* [RFC PATCH v3 01/18] xen/arm: Add bitmap_find_next_zero_area helper function
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-22 14:14 ` Julien Grall
2015-06-22 12:01 ` [RFC PATCH v3 02/18] xen: Add log2 functionality vijay.kilari
` (19 subsequent siblings)
20 siblings, 1 reply; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
bitmap_find_next_zero_area helper function will be used
by physical ITS driver imported from linux
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v3: Moved changes to xen/common/bitmap.c and
xen/include/xen/bitmap.h
---
xen/common/bitmap.c | 39 +++++++++++++++++++++++++++++++++++++++
xen/include/xen/bitmap.h | 5 +++++
2 files changed, 44 insertions(+)
diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c
index 61d1ea4..ba060b2 100644
--- a/xen/common/bitmap.c
+++ b/xen/common/bitmap.c
@@ -489,6 +489,45 @@ int bitmap_allocate_region(unsigned long *bitmap, int pos, int order)
}
EXPORT_SYMBOL(bitmap_allocate_region);
+/*
+ * bitmap_find_next_zero_area - find a contiguous aligned zero area
+ * @map: The address to base the search on
+ * @size: The bitmap size in bits
+ * @start: The bitnumber to start searching at
+ * @nr: The number of zeroed bits we're looking for
+ * @align_mask: Alignment mask for zero area
+ *
+ * The @align_mask should be one less than a power of 2; the effect is that
+ * the bit offset of all zero areas this function finds is multiples of that
+ * power of 2. A @align_mask of 0 means no alignment is required.
+ */
+#define ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
+
+unsigned long bitmap_find_next_zero_area(unsigned long *map,
+ unsigned long size,
+ unsigned long start,
+ unsigned int nr,
+ unsigned long align_mask)
+{
+ unsigned long index, end, i;
+again:
+ index = find_next_zero_bit(map, size, start);
+
+ /* Align allocation */
+ index = ALIGN_MASK(index, align_mask);
+
+ end = index + nr;
+ if (end > size)
+ return end;
+ i = find_next_bit(map, end, index);
+ if (i < end) {
+ start = i + 1;
+ goto again;
+ }
+ return index;
+}
+EXPORT_SYMBOL(bitmap_find_next_zero_area);
+
#ifdef __BIG_ENDIAN
void bitmap_long_to_byte(uint8_t *bp, const unsigned long *lp, int nbits)
diff --git a/xen/include/xen/bitmap.h b/xen/include/xen/bitmap.h
index e2a3686..5c48ac3 100644
--- a/xen/include/xen/bitmap.h
+++ b/xen/include/xen/bitmap.h
@@ -101,6 +101,11 @@ extern int bitmap_scnlistprintf(char *buf, unsigned int len,
extern int bitmap_find_free_region(unsigned long *bitmap, int bits, int order);
extern void bitmap_release_region(unsigned long *bitmap, int pos, int order);
extern int bitmap_allocate_region(unsigned long *bitmap, int pos, int order);
+extern unsigned long bitmap_find_next_zero_area(unsigned long *map,
+ unsigned long size,
+ unsigned long start,
+ unsigned int nr,
+ unsigned long align_mask);
#define BITMAP_LAST_WORD_MASK(nbits) \
( \
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 01/18] xen/arm: Add bitmap_find_next_zero_area helper function
2015-06-22 12:01 ` [RFC PATCH v3 01/18] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
@ 2015-06-22 14:14 ` Julien Grall
0 siblings, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-06-22 14:14 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi,
Please include all the relevant maintainers. See scripts/get_maintainers.pl.
On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> bitmap_find_next_zero_area helper function will be used
> by physical ITS driver imported from linux
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> v3: Moved changes to xen/common/bitmap.c and
> xen/include/xen/bitmap.h
> ---
> xen/common/bitmap.c | 39 +++++++++++++++++++++++++++++++++++++++
> xen/include/xen/bitmap.h | 5 +++++
> 2 files changed, 44 insertions(+)
>
> diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c
> index 61d1ea4..ba060b2 100644
> --- a/xen/common/bitmap.c
> +++ b/xen/common/bitmap.c
> @@ -489,6 +489,45 @@ int bitmap_allocate_region(unsigned long *bitmap, int pos, int order)
> }
> EXPORT_SYMBOL(bitmap_allocate_region);
>
> +/*
> + * bitmap_find_next_zero_area - find a contiguous aligned zero area
> + * @map: The address to base the search on
> + * @size: The bitmap size in bits
> + * @start: The bitnumber to start searching at
> + * @nr: The number of zeroed bits we're looking for
> + * @align_mask: Alignment mask for zero area
> + *
> + * The @align_mask should be one less than a power of 2; the effect is that
> + * the bit offset of all zero areas this function finds is multiples of that
> + * power of 2. A @align_mask of 0 means no alignment is required.
> + */
> +#define ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
This is not bitmap specific, I would move this macro in lib.h
> +unsigned long bitmap_find_next_zero_area(unsigned long *map,
> + unsigned long size,
> + unsigned long start,
> + unsigned int nr,
> + unsigned long align_mask)
> +{
> + unsigned long index, end, i;
> +again:
> + index = find_next_zero_bit(map, size, start);
> +
> + /* Align allocation */
> + index = ALIGN_MASK(index, align_mask);
> +
> + end = index + nr;
> + if (end > size)
> + return end;
> + i = find_next_bit(map, end, index);
> + if (i < end) {
> + start = i + 1;
> + goto again;
> + }
> + return index;
> +}
> +EXPORT_SYMBOL(bitmap_find_next_zero_area);
> +
bitmap.c is using Linux coding style which include hard tabs. Please
retain them.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 02/18] xen: Add log2 functionality
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
2015-06-22 12:01 ` [RFC PATCH v3 01/18] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-22 13:17 ` Jan Beulich
2015-06-22 12:01 ` [RFC PATCH v3 03/18] xen: console: Add ratelimit support for error message vijay.kilari
` (18 subsequent siblings)
20 siblings, 1 reply; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
log2 helper apis are ported from linux
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/include/xen/bitops.h | 8 ++
xen/include/xen/log2.h | 205 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 213 insertions(+)
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index cb56f24..b01333c 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -117,6 +117,14 @@ static inline int generic_fls64(__u64 x)
# endif
#endif
+static inline unsigned fls_long(unsigned long l)
+{
+ if (sizeof(l) == 4)
+ return fls(l);
+
+ return fls64(l);
+}
+
static __inline__ int get_bitmask_order(unsigned int count)
{
int order;
diff --git a/xen/include/xen/log2.h b/xen/include/xen/log2.h
new file mode 100644
index 0000000..1e73ae2
--- /dev/null
+++ b/xen/include/xen/log2.h
@@ -0,0 +1,205 @@
+/*
+ * Copyright (C) 2006 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _LINUX_LOG2_H
+#define _LINUX_LOG2_H
+
+#include <xen/types.h>
+#include <xen/bitops.h>
+
+/*
+ * deal with unrepresentable constant logarithms
+ */
+extern __attribute__((const))
+int ____ilog2_NaN(void);
+
+/*
+ * non-constant log of base 2 calculators
+ * - the arch may override these in asm/bitops.h if they can be implemented
+ * more efficiently than using fls() and fls64()
+ * - the arch is not required to handle n==0 if implementing the fallback
+ */
+static inline __attribute__((const))
+int __ilog2_u32(u32 n)
+{
+ return fls(n) - 1;
+}
+
+static inline __attribute__((const))
+int __ilog2_u64(u64 n)
+{
+ return fls64(n) - 1;
+}
+
+/*
+ * Determine whether some value is a power of two, where zero is
+ * *not* considered a power of two.
+ */
+
+static inline __attribute__((const))
+bool is_power_of_2(unsigned long n)
+{
+ return (n != 0 && ((n & (n - 1)) == 0));
+}
+
+#if 1
+/*
+ * round up to nearest power of two
+ */
+static inline __attribute__((const))
+unsigned long __roundup_pow_of_two(unsigned long n)
+{
+ return 1UL << fls_long(n - 1);
+}
+
+/*
+ * round down to nearest power of two
+ */
+static inline __attribute__((const))
+unsigned long __rounddown_pow_of_two(unsigned long n)
+{
+ return 1UL << (fls_long(n) - 1);
+}
+#endif
+/**
+ * ilog2 - log of base 2 of 32-bit or a 64-bit unsigned value
+ * @n - parameter
+ *
+ * constant-capable log of base 2 calculation
+ * - this can be used to initialise global variables from constant data, hence
+ * the massive ternary operator construction
+ *
+ * selects the appropriately-sized optimised version depending on sizeof(n)
+ */
+#define ilog2(n) \
+( \
+ __builtin_constant_p(n) ? ( \
+ (n) < 1 ? ____ilog2_NaN() : \
+ (n) & (1ULL << 63) ? 63 : \
+ (n) & (1ULL << 62) ? 62 : \
+ (n) & (1ULL << 61) ? 61 : \
+ (n) & (1ULL << 60) ? 60 : \
+ (n) & (1ULL << 59) ? 59 : \
+ (n) & (1ULL << 58) ? 58 : \
+ (n) & (1ULL << 57) ? 57 : \
+ (n) & (1ULL << 56) ? 56 : \
+ (n) & (1ULL << 55) ? 55 : \
+ (n) & (1ULL << 54) ? 54 : \
+ (n) & (1ULL << 53) ? 53 : \
+ (n) & (1ULL << 52) ? 52 : \
+ (n) & (1ULL << 51) ? 51 : \
+ (n) & (1ULL << 50) ? 50 : \
+ (n) & (1ULL << 49) ? 49 : \
+ (n) & (1ULL << 48) ? 48 : \
+ (n) & (1ULL << 47) ? 47 : \
+ (n) & (1ULL << 46) ? 46 : \
+ (n) & (1ULL << 45) ? 45 : \
+ (n) & (1ULL << 44) ? 44 : \
+ (n) & (1ULL << 43) ? 43 : \
+ (n) & (1ULL << 42) ? 42 : \
+ (n) & (1ULL << 41) ? 41 : \
+ (n) & (1ULL << 40) ? 40 : \
+ (n) & (1ULL << 39) ? 39 : \
+ (n) & (1ULL << 38) ? 38 : \
+ (n) & (1ULL << 37) ? 37 : \
+ (n) & (1ULL << 36) ? 36 : \
+ (n) & (1ULL << 35) ? 35 : \
+ (n) & (1ULL << 34) ? 34 : \
+ (n) & (1ULL << 33) ? 33 : \
+ (n) & (1ULL << 32) ? 32 : \
+ (n) & (1ULL << 31) ? 31 : \
+ (n) & (1ULL << 30) ? 30 : \
+ (n) & (1ULL << 29) ? 29 : \
+ (n) & (1ULL << 28) ? 28 : \
+ (n) & (1ULL << 27) ? 27 : \
+ (n) & (1ULL << 26) ? 26 : \
+ (n) & (1ULL << 25) ? 25 : \
+ (n) & (1ULL << 24) ? 24 : \
+ (n) & (1ULL << 23) ? 23 : \
+ (n) & (1ULL << 22) ? 22 : \
+ (n) & (1ULL << 21) ? 21 : \
+ (n) & (1ULL << 20) ? 20 : \
+ (n) & (1ULL << 19) ? 19 : \
+ (n) & (1ULL << 18) ? 18 : \
+ (n) & (1ULL << 17) ? 17 : \
+ (n) & (1ULL << 16) ? 16 : \
+ (n) & (1ULL << 15) ? 15 : \
+ (n) & (1ULL << 14) ? 14 : \
+ (n) & (1ULL << 13) ? 13 : \
+ (n) & (1ULL << 12) ? 12 : \
+ (n) & (1ULL << 11) ? 11 : \
+ (n) & (1ULL << 10) ? 10 : \
+ (n) & (1ULL << 9) ? 9 : \
+ (n) & (1ULL << 8) ? 8 : \
+ (n) & (1ULL << 7) ? 7 : \
+ (n) & (1ULL << 6) ? 6 : \
+ (n) & (1ULL << 5) ? 5 : \
+ (n) & (1ULL << 4) ? 4 : \
+ (n) & (1ULL << 3) ? 3 : \
+ (n) & (1ULL << 2) ? 2 : \
+ (n) & (1ULL << 1) ? 1 : \
+ (n) & (1ULL << 0) ? 0 : \
+ ____ilog2_NaN() \
+ ) : \
+ (sizeof(n) <= 4) ? \
+ __ilog2_u32(n) : \
+ __ilog2_u64(n) \
+ )
+
+#if 1
+/**
+ * roundup_pow_of_two - round the given value up to nearest power of two
+ * @n - parameter
+ *
+ * round the given value up to the nearest power of two
+ * - the result is undefined when n == 0
+ * - this can be used to initialise global variables from constant data
+ */
+#define roundup_pow_of_two(n) \
+( \
+ __builtin_constant_p(n) ? ( \
+ (n == 1) ? 1 : \
+ (1UL << (ilog2((n) - 1) + 1)) \
+ ) : \
+ __roundup_pow_of_two(n) \
+ )
+
+/**
+ * rounddown_pow_of_two - round the given value down to nearest power of two
+ * @n - parameter
+ *
+ * round the given value down to the nearest power of two
+ * - the result is undefined when n == 0
+ * - this can be used to initialise global variables from constant data
+ */
+#define rounddown_pow_of_two(n) \
+( \
+ __builtin_constant_p(n) ? ( \
+ (1UL << ilog2(n))) : \
+ __rounddown_pow_of_two(n) \
+ )
+
+/**
+ * order_base_2 - calculate the (rounded up) base 2 order of the argument
+ * @n: parameter
+ *
+ * The first few values calculated by this routine:
+ * ob2(0) = 0
+ * ob2(1) = 0
+ * ob2(2) = 1
+ * ob2(3) = 2
+ * ob2(4) = 2
+ * ob2(5) = 3
+ * ... and so on.
+ */
+
+#define order_base_2(n) ilog2(roundup_pow_of_two(n))
+#endif
+#endif /* _LINUX_LOG2_H */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 02/18] xen: Add log2 functionality
2015-06-22 12:01 ` [RFC PATCH v3 02/18] xen: Add log2 functionality vijay.kilari
@ 2015-06-22 13:17 ` Jan Beulich
2015-06-24 13:05 ` Vijay Kilari
0 siblings, 1 reply; 88+ messages in thread
From: Jan Beulich @ 2015-06-22 13:17 UTC (permalink / raw)
To: vijay.kilari
Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, manish.jaggi,
tim, xen-devel, julien.grall, stefano.stabellini, Vijaya Kumar K
>>> On 22.06.15 at 14:01, <vijay.kilari@gmail.com> wrote:
First of all, please Cc _all_ relevant maintainers.
> --- a/xen/include/xen/bitops.h
> +++ b/xen/include/xen/bitops.h
> @@ -117,6 +117,14 @@ static inline int generic_fls64(__u64 x)
> # endif
> #endif
>
> +static inline unsigned fls_long(unsigned long l)
> +{
> + if (sizeof(l) == 4)
> + return fls(l);
> +
> + return fls64(l);
> +}
I'm not really opposed to this, but did you really verify that there's
no suitable functionality in tree already (even if named differently)?
I can't, e.g., see why flsl() wouldn't fit your needs.
> --- /dev/null
> +++ b/xen/include/xen/log2.h
> @@ -0,0 +1,205 @@
> +/*
> + * Copyright (C) 2006 Red Hat, Inc. All Rights Reserved.
> + * Written by David Howells (dhowells@redhat.com)
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#ifndef _LINUX_LOG2_H
> +#define _LINUX_LOG2_H
LINUX?
> +/*
> + * deal with unrepresentable constant logarithms
> + */
> +extern __attribute__((const))
> +int ____ilog2_NaN(void);
-ETOOMANYUNDERSCORES
> +#if 1
??? (at least one more below)
Also, are you really needing all of what you add here?
Jan
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 02/18] xen: Add log2 functionality
2015-06-22 13:17 ` Jan Beulich
@ 2015-06-24 13:05 ` Vijay Kilari
2015-06-24 13:21 ` Jan Beulich
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-06-24 13:05 UTC (permalink / raw)
To: Jan Beulich
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel, Julien Grall, Stefano Stabellini,
Vijaya Kumar K
On Mon, Jun 22, 2015 at 6:47 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 22.06.15 at 14:01, <vijay.kilari@gmail.com> wrote:
>
> First of all, please Cc _all_ relevant maintainers.
>
>> --- a/xen/include/xen/bitops.h
>> +++ b/xen/include/xen/bitops.h
>> @@ -117,6 +117,14 @@ static inline int generic_fls64(__u64 x)
>> # endif
>> #endif
>>
>> +static inline unsigned fls_long(unsigned long l)
>> +{
>> + if (sizeof(l) == 4)
>> + return fls(l);
>> +
>> + return fls64(l);
>> +}
>
> I'm not really opposed to this, but did you really verify that there's
> no suitable functionality in tree already (even if named differently)?
> I can't, e.g., see why flsl() wouldn't fit your needs.
#define fls64 flsl
So flsl also should be fine
>
>> --- /dev/null
>> +++ b/xen/include/xen/log2.h
>> @@ -0,0 +1,205 @@
>> +/*
>> + * Copyright (C) 2006 Red Hat, Inc. All Rights Reserved.
>> + * Written by David Howells (dhowells@redhat.com)
>> + *
>> + * This program is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU General Public License
>> + * as published by the Free Software Foundation; either version
>> + * 2 of the License, or (at your option) any later version.
>> + */
>> +
>> +#ifndef _LINUX_LOG2_H
>> +#define _LINUX_LOG2_H
>
> LINUX?
Most of the include/xen/*.h files has LINUX kept.
Anyway, I will remove it
>
>> +/*
>> + * deal with unrepresentable constant logarithms
>> + */
>> +extern __attribute__((const))
>> +int ____ilog2_NaN(void);
>
> -ETOOMANYUNDERSCORES
I have taken this from linux. I have not introduced it.
>
>> +#if 1
>
> ??? (at least one more below)
>
> Also, are you really needing all of what you add here?
I have taken complete log2 file from linux.
Regards
Vijay
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 02/18] xen: Add log2 functionality
2015-06-24 13:05 ` Vijay Kilari
@ 2015-06-24 13:21 ` Jan Beulich
0 siblings, 0 replies; 88+ messages in thread
From: Jan Beulich @ 2015-06-24 13:21 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel, Julien Grall, Stefano Stabellini,
Vijaya Kumar K
>>> On 24.06.15 at 15:05, <vijay.kilari@gmail.com> wrote:
> On Mon, Jun 22, 2015 at 6:47 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 22.06.15 at 14:01, <vijay.kilari@gmail.com> wrote:
>>> --- a/xen/include/xen/bitops.h
>>> +++ b/xen/include/xen/bitops.h
>>> @@ -117,6 +117,14 @@ static inline int generic_fls64(__u64 x)
>>> # endif
>>> #endif
>>>
>>> +static inline unsigned fls_long(unsigned long l)
>>> +{
>>> + if (sizeof(l) == 4)
>>> + return fls(l);
>>> +
>>> + return fls64(l);
>>> +}
>>
>> I'm not really opposed to this, but did you really verify that there's
>> no suitable functionality in tree already (even if named differently)?
>> I can't, e.g., see why flsl() wouldn't fit your needs.
>
> #define fls64 flsl
> So flsl also should be fine
No idea what you mean to say with the first of the two lines.
>>> +/*
>>> + * deal with unrepresentable constant logarithms
>>> + */
>>> +extern __attribute__((const))
>>> +int ____ilog2_NaN(void);
>>
>> -ETOOMANYUNDERSCORES
>
> I have taken this from linux. I have not introduced it.
>
>>
>>> +#if 1
>>
>> ??? (at least one more below)
>>
>> Also, are you really needing all of what you add here?
>
> I have taken complete log2 file from linux.
And how much of it do you really need? In any event I'd like to ask
not to copy rubbish like this. Also, in the description you say "ported"
while taking the complete file should be expressed differently imo
(and should indicate the version it was taken from so that people
wanting to update it later will know what to diff against).
Jan
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 03/18] xen: console: Add ratelimit support for error message
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
2015-06-22 12:01 ` [RFC PATCH v3 01/18] xen/arm: Add bitmap_find_next_zero_area helper function vijay.kilari
2015-06-22 12:01 ` [RFC PATCH v3 02/18] xen: Add log2 functionality vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-22 13:21 ` Jan Beulich
2015-06-22 12:01 ` [RFC PATCH v3 04/18] xen/arm: gicv3: Refactor redistributor information vijay.kilari
` (17 subsequent siblings)
20 siblings, 1 reply; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
XENLOG_ERR_RATE_LIMIT and XENLOG_G_ERR_RATE_LIMIT
log levels are added to support rate limit for error messages
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/drivers/char/console.c | 17 +++++++++--------
xen/include/xen/config.h | 8 ++++++--
2 files changed, 15 insertions(+), 10 deletions(-)
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index fce4cc8..b01fde5 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -92,15 +92,15 @@ static DEFINE_SPINLOCK(console_lock);
* the lower threshold equal to the upper.
*/
#ifdef NDEBUG
-#define XENLOG_UPPER_THRESHOLD 2 /* Do not print INFO and DEBUG */
+#define XENLOG_UPPER_THRESHOLD 3 /* Do not print INFO and DEBUG */
#define XENLOG_LOWER_THRESHOLD 2 /* Always print ERR and WARNING */
-#define XENLOG_GUEST_UPPER_THRESHOLD 2 /* Do not print INFO and DEBUG */
+#define XENLOG_GUEST_UPPER_THRESHOLD 3 /* Do not print INFO and DEBUG */
#define XENLOG_GUEST_LOWER_THRESHOLD 0 /* Rate-limit ERR and WARNING */
#else
-#define XENLOG_UPPER_THRESHOLD 4 /* Do not discard anything */
-#define XENLOG_LOWER_THRESHOLD 4 /* Print everything */
-#define XENLOG_GUEST_UPPER_THRESHOLD 4 /* Do not discard anything */
-#define XENLOG_GUEST_LOWER_THRESHOLD 4 /* Print everything */
+#define XENLOG_UPPER_THRESHOLD 5 /* Do not discard anything */
+#define XENLOG_LOWER_THRESHOLD 5 /* Print everything */
+#define XENLOG_GUEST_UPPER_THRESHOLD 5 /* Do not discard anything */
+#define XENLOG_GUEST_LOWER_THRESHOLD 5 /* Print everything */
#endif
/*
* The XENLOG_DEFAULT is the default given to printks that
@@ -540,7 +540,7 @@ static int printk_prefix_check(char *p, char **pp)
if ( loglvl == -1 )
loglvl = XENLOG_GUEST_DEFAULT;
break;
- case '0' ... '3':
+ case '0' ... '4':
loglvl = p[1] - '0';
break;
}
@@ -554,7 +554,8 @@ static int printk_prefix_check(char *p, char **pp)
return ((atomic_read(&print_everything) != 0) ||
(loglvl < lower_thresh) ||
- ((loglvl < upper_thresh) && printk_ratelimit()));
+ ((loglvl >= lower_thresh && loglvl < upper_thresh) &&
+ printk_ratelimit()));
}
static void __init parse_console_timestamps(char *s)
diff --git a/xen/include/xen/config.h b/xen/include/xen/config.h
index f7258c7..b2ce0fe 100644
--- a/xen/include/xen/config.h
+++ b/xen/include/xen/config.h
@@ -18,6 +18,8 @@
/*
* The following log levels are as follows:
*
+ * XENLOG_ERR_RATE_LIMIT: Rate limit error message from Xen and Guest
+ *
* XENLOG_ERR: Fatal errors, either Xen, Guest or Dom0
* is about to crash.
*
@@ -41,11 +43,13 @@
*/
#define XENLOG_ERR "<0>"
#define XENLOG_WARNING "<1>"
-#define XENLOG_INFO "<2>"
-#define XENLOG_DEBUG "<3>"
+#define XENLOG_ERR_RATE_LIMIT "<2>"
+#define XENLOG_INFO "<3>"
+#define XENLOG_DEBUG "<4>"
#define XENLOG_GUEST "<G>"
+#define XENLOG_G_ERR_RATE_LIMIT XENLOG_GUEST XENLOG_ERR_RATE_LIMIT
#define XENLOG_G_ERR XENLOG_GUEST XENLOG_ERR
#define XENLOG_G_WARNING XENLOG_GUEST XENLOG_WARNING
#define XENLOG_G_INFO XENLOG_GUEST XENLOG_INFO
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 03/18] xen: console: Add ratelimit support for error message
2015-06-22 12:01 ` [RFC PATCH v3 03/18] xen: console: Add ratelimit support for error message vijay.kilari
@ 2015-06-22 13:21 ` Jan Beulich
2015-06-25 13:14 ` Vijay Kilari
0 siblings, 1 reply; 88+ messages in thread
From: Jan Beulich @ 2015-06-22 13:21 UTC (permalink / raw)
To: vijay.kilari
Cc: Ian.Campbell, stefano.stabellini, Prasun.Kapoor, manish.jaggi,
tim, xen-devel, julien.grall, stefano.stabellini, Vijaya Kumar K
>>> On 22.06.15 at 14:01, <vijay.kilari@gmail.com> wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> XENLOG_ERR_RATE_LIMIT and XENLOG_G_ERR_RATE_LIMIT
> log levels are added to support rate limit for error messages
If you mean to say that rate limiting currently doesn't work for
XENLOG_ERR messages, then that's a problem to be fixed by
adjusting existing code, not by adding yet another log level.
Jan
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 03/18] xen: console: Add ratelimit support for error message
2015-06-22 13:21 ` Jan Beulich
@ 2015-06-25 13:14 ` Vijay Kilari
2015-06-25 13:21 ` Andrew Cooper
2015-06-25 13:31 ` Jan Beulich
0 siblings, 2 replies; 88+ messages in thread
From: Vijay Kilari @ 2015-06-25 13:14 UTC (permalink / raw)
To: Jan Beulich
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel, Julien Grall, Stefano Stabellini,
Vijaya Kumar K
On Mon, Jun 22, 2015 at 6:51 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 22.06.15 at 14:01, <vijay.kilari@gmail.com> wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> XENLOG_ERR_RATE_LIMIT and XENLOG_G_ERR_RATE_LIMIT
>> log levels are added to support rate limit for error messages
>
> If you mean to say that rate limiting currently doesn't work for
> XENLOG_ERR messages, then that's a problem to be fixed by
> adjusting existing code, not by adding yet another log level.
For GUEST messages ERR and WARN are rate limited by
setting lower threshold to 0 and upper threshold to 2 as below
#define XENLOG_GUEST_UPPER_THRESHOLD 2 /* Do not print INFO and DEBUG */
#define XENLOG_GUEST_LOWER_THRESHOLD 0 /* Rate-limit ERR and WARNING */
So do you recommend to set same threshold levels to Xen messages
there by ERR & WARN are rate limited?
Regards
Vijay
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 03/18] xen: console: Add ratelimit support for error message
2015-06-25 13:14 ` Vijay Kilari
@ 2015-06-25 13:21 ` Andrew Cooper
2015-06-25 13:31 ` Jan Beulich
1 sibling, 0 replies; 88+ messages in thread
From: Andrew Cooper @ 2015-06-25 13:21 UTC (permalink / raw)
To: Vijay Kilari, Jan Beulich
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel, Julien Grall, Stefano Stabellini,
Vijaya Kumar K
On 25/06/15 14:14, Vijay Kilari wrote:
> On Mon, Jun 22, 2015 at 6:51 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 22.06.15 at 14:01, <vijay.kilari@gmail.com> wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> XENLOG_ERR_RATE_LIMIT and XENLOG_G_ERR_RATE_LIMIT
>>> log levels are added to support rate limit for error messages
>> If you mean to say that rate limiting currently doesn't work for
>> XENLOG_ERR messages, then that's a problem to be fixed by
>> adjusting existing code, not by adding yet another log level.
> For GUEST messages ERR and WARN are rate limited by
> setting lower threshold to 0 and upper threshold to 2 as below
>
> #define XENLOG_GUEST_UPPER_THRESHOLD 2 /* Do not print INFO and DEBUG */
> #define XENLOG_GUEST_LOWER_THRESHOLD 0 /* Rate-limit ERR and WARNING */
>
> So do you recommend to set same threshold levels to Xen messages
> there by ERR & WARN are rate limited?
Xen already has guest rate limiting logic, which has separate visible
and ratelimited boundaries.
http://xenbits.xen.org/docs/unstable/misc/xen-command-line.html
Look at guest_loglvl.
If the existing logic isn't working then it should be fixed. We should
not introduce a second way of doing this.
~Andrew
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 03/18] xen: console: Add ratelimit support for error message
2015-06-25 13:14 ` Vijay Kilari
2015-06-25 13:21 ` Andrew Cooper
@ 2015-06-25 13:31 ` Jan Beulich
1 sibling, 0 replies; 88+ messages in thread
From: Jan Beulich @ 2015-06-25 13:31 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, manish.jaggi,
Tim Deegan, xen-devel, Julien Grall, Stefano Stabellini,
Vijaya Kumar K
>>> On 25.06.15 at 15:14, <vijay.kilari@gmail.com> wrote:
> On Mon, Jun 22, 2015 at 6:51 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 22.06.15 at 14:01, <vijay.kilari@gmail.com> wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> XENLOG_ERR_RATE_LIMIT and XENLOG_G_ERR_RATE_LIMIT
>>> log levels are added to support rate limit for error messages
>>
>> If you mean to say that rate limiting currently doesn't work for
>> XENLOG_ERR messages, then that's a problem to be fixed by
>> adjusting existing code, not by adding yet another log level.
>
> For GUEST messages ERR and WARN are rate limited by
> setting lower threshold to 0 and upper threshold to 2 as below
>
> #define XENLOG_GUEST_UPPER_THRESHOLD 2 /* Do not print INFO and DEBUG */
> #define XENLOG_GUEST_LOWER_THRESHOLD 0 /* Rate-limit ERR and WARNING */
>
> So do you recommend to set same threshold levels to Xen messages
> there by ERR & WARN are rate limited?
I'm not sure I understand what you're asking: I recommend no
change at all, unless you see something broken (in which case
that's what I want clearly described).
Jan
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 04/18] xen/arm: gicv3: Refactor redistributor information
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (2 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 03/18] xen: console: Add ratelimit support for error message vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-22 15:00 ` Julien Grall
2015-06-29 11:09 ` Ian Campbell
2015-06-22 12:01 ` [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen vijay.kilari
` (16 subsequent siblings)
20 siblings, 2 replies; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Separate redistributor information into rdist and rdist_prop
structures.
The rdist_prop holds the redistributor common information
and rdist holds the per cpu specific information.
This percpu rdist defined as global and shared with ITS
driver
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3.c | 15 ++++++++++-----
xen/include/asm-arm/gic_v3_defs.h | 15 +++++++++++++++
2 files changed, 25 insertions(+), 5 deletions(-)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 30682cf..b5c59f6 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -53,6 +53,7 @@ static struct {
paddr_t dbase; /* Address of distributor registers */
paddr_t dbase_size;
void __iomem *map_dbase; /* Mapped address of distributor registers */
+ struct rdist_prop rdist_data;
struct rdist_region *rdist_regions;
uint32_t rdist_stride;
unsigned int rdist_count; /* Number of rdist regions count */
@@ -63,10 +64,10 @@ static struct {
static struct gic_info gicv3_info;
/* per-cpu re-distributor base */
-static DEFINE_PER_CPU(void __iomem*, rbase);
+DEFINE_PER_CPU(struct rdist, rdist);
#define GICD (gicv3.map_dbase)
-#define GICD_RDIST_BASE (this_cpu(rbase))
+#define GICD_RDIST_BASE (per_cpu(rdist, smp_processor_id()).rbase)
#define GICD_RDIST_SGI_BASE (GICD_RDIST_BASE + SZ_64K)
/*
@@ -613,6 +614,7 @@ static int __init gicv3_populate_rdist(void)
uint32_t aff;
uint32_t reg;
uint64_t typer;
+ uint64_t offset;
uint64_t mpidr = cpu_logical_map(smp_processor_id());
/*
@@ -648,9 +650,12 @@ static int __init gicv3_populate_rdist(void)
if ( (typer >> 32) == aff )
{
- this_cpu(rbase) = ptr;
- printk("GICv3: CPU%d: Found redistributor in region %d @%p\n",
- smp_processor_id(), i, ptr);
+ offset = ptr - gicv3.rdist_regions[i].map_base;
+ per_cpu(rdist, smp_processor_id()).rbase = ptr;
+ per_cpu(rdist, smp_processor_id()).phys_base = gicv3.rdist_regions[i].base + offset;
+ printk("GICv3: CPU%d: Found redistributor in region %d @%"PRIpaddr"\n",
+ smp_processor_id(), i,
+ per_cpu(rdist, smp_processor_id()).phys_base);
return 0;
}
if ( gicv3.rdist_stride )
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index 556f114..acbb906 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -152,6 +152,21 @@
#define ICH_SGI_IRQ_SHIFT 24
#define ICH_SGI_IRQ_MASK 0xf
#define ICH_SGI_TARGETLIST_MASK 0xffff
+
+struct rdist {
+ void __iomem *rbase;
+ void * pend_page;
+ paddr_t phys_base;
+};
+
+struct rdist_prop {
+ void * prop_page;
+ int id_bits;
+ uint64_t flags;
+};
+
+DECLARE_PER_CPU(struct rdist, rdist);
+
#endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 04/18] xen/arm: gicv3: Refactor redistributor information
2015-06-22 12:01 ` [RFC PATCH v3 04/18] xen/arm: gicv3: Refactor redistributor information vijay.kilari
@ 2015-06-22 15:00 ` Julien Grall
2015-06-29 11:09 ` Ian Campbell
1 sibling, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-06-22 15:00 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi,
On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Separate redistributor information into rdist and rdist_prop
> structures.
>
> The rdist_prop holds the redistributor common information
> and rdist holds the per cpu specific information.
>
> This percpu rdist defined as global and shared with ITS
> driver
This patch does more than refactoring, you are adding new fields which
are not use here.
Please explain why you need them or move them in the patch where they
are used. Comment on the usage of the field would be nice too.
FWIW, I would be in favor of the later.
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/gic-v3.c | 15 ++++++++++-----
> xen/include/asm-arm/gic_v3_defs.h | 15 +++++++++++++++
> 2 files changed, 25 insertions(+), 5 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 30682cf..b5c59f6 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -53,6 +53,7 @@ static struct {
> paddr_t dbase; /* Address of distributor registers */
> paddr_t dbase_size;
> void __iomem *map_dbase; /* Mapped address of distributor registers */
> + struct rdist_prop rdist_data;
> struct rdist_region *rdist_regions;
> uint32_t rdist_stride;
> unsigned int rdist_count; /* Number of rdist regions count */
> @@ -63,10 +64,10 @@ static struct {
> static struct gic_info gicv3_info;
>
> /* per-cpu re-distributor base */
> -static DEFINE_PER_CPU(void __iomem*, rbase);
> +DEFINE_PER_CPU(struct rdist, rdist);
>
> #define GICD (gicv3.map_dbase)
> -#define GICD_RDIST_BASE (this_cpu(rbase))
> +#define GICD_RDIST_BASE (per_cpu(rdist, smp_processor_id()).rbase)
> #define GICD_RDIST_SGI_BASE (GICD_RDIST_BASE + SZ_64K)
>
> /*
> @@ -613,6 +614,7 @@ static int __init gicv3_populate_rdist(void)
> uint32_t aff;
> uint32_t reg;
> uint64_t typer;
> + uint64_t offset;
> uint64_t mpidr = cpu_logical_map(smp_processor_id());
>
> /*
> @@ -648,9 +650,12 @@ static int __init gicv3_populate_rdist(void)
>
> if ( (typer >> 32) == aff )
> {
> - this_cpu(rbase) = ptr;
> - printk("GICv3: CPU%d: Found redistributor in region %d @%p\n",
> - smp_processor_id(), i, ptr);
> + offset = ptr - gicv3.rdist_regions[i].map_base;
> + per_cpu(rdist, smp_processor_id()).rbase = ptr;
> + per_cpu(rdist, smp_processor_id()).phys_base = gicv3.rdist_regions[i].base + offset;
> + printk("GICv3: CPU%d: Found redistributor in region %d @%"PRIpaddr"\n",
> + smp_processor_id(), i,
> + per_cpu(rdist, smp_processor_id()).phys_base);
> return 0;
> }
> if ( gicv3.rdist_stride )
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index 556f114..acbb906 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -152,6 +152,21 @@
> #define ICH_SGI_IRQ_SHIFT 24
> #define ICH_SGI_IRQ_MASK 0xf
> #define ICH_SGI_TARGETLIST_MASK 0xffff
> +
> +struct rdist {
> + void __iomem *rbase;
> + void * pend_page;
void *pend_page;
> + paddr_t phys_base;
> +};
> +
> +struct rdist_prop {
What does "prop" stand for?
> + void * prop_page;
Ditto.
> + int id_bits;
> + uint64_t flags;
> +};
> +
> +DECLARE_PER_CPU(struct rdist, rdist);
> +
> #endif /* __ASM_ARM_GIC_V3_DEFS_H__ */
>
> /*
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 04/18] xen/arm: gicv3: Refactor redistributor information
2015-06-22 12:01 ` [RFC PATCH v3 04/18] xen/arm: gicv3: Refactor redistributor information vijay.kilari
2015-06-22 15:00 ` Julien Grall
@ 2015-06-29 11:09 ` Ian Campbell
1 sibling, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 11:09 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Separate redistributor information into rdist and rdist_prop
> structures.
>
> The rdist_prop holds the redistributor common information
> and rdist holds the per cpu specific information.
>
> This percpu rdist defined as global and shared with ITS
> driver
The vits or pits driver? The latter might be ok, the former would
suggest some sort of layering violation I think.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (3 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 04/18] xen/arm: gicv3: Refactor redistributor information vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-22 17:16 ` Julien Grall
2015-06-29 11:39 ` Ian Campbell
2015-06-22 12:01 ` [RFC PATCH v3 06/18] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
` (15 subsequent siblings)
20 siblings, 2 replies; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Only required changes from Linux ITS driver is ported
and compiled
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
v3:
- Only required changes from Linux ITS driver is ported
- Xen coding style is followed.
---
xen/arch/arm/Makefile | 1 +
xen/arch/arm/gic-v3-its.c | 1067 +++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 214 ++++++++
xen/include/asm-arm/gic.h | 1 +
xen/include/asm-arm/gic_v3_defs.h | 127 ++++-
5 files changed, 1408 insertions(+), 2 deletions(-)
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 935999e..1821ed2 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -32,6 +32,7 @@ obj-y += shutdown.o
obj-y += traps.o
obj-y += vgic.o vgic-v2.o
obj-$(CONFIG_ARM_64) += vgic-v3.o
+obj-$(CONFIG_ARM_64) += gic-v3-its.o
obj-y += vtimer.o
obj-y += vuart.o
obj-y += hvm.o
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
new file mode 100644
index 0000000..b1a97c1
--- /dev/null
+++ b/xen/arch/arm/gic-v3-its.c
@@ -0,0 +1,1067 @@
+/*
+ * Copyright (C) 2013, 2014 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * Xen changes:
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ * Copyright (C) 2014, 2015 Cavium Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/config.h>
+#include <xen/bitops.h>
+#include <xen/init.h>
+#include <xen/mm.h>
+#include <xen/irq.h>
+#include <xen/sched.h>
+#include <xen/errno.h>
+#include <xen/delay.h>
+#include <xen/list.h>
+#include <xen/sizes.h>
+#include <xen/vmap.h>
+#include <asm/p2m.h>
+#include <asm/domain.h>
+#include <asm/io.h>
+#include <asm/device.h>
+#include <asm/gic.h>
+#include <asm/gic_v3_defs.h>
+#include <asm/gic-its.h>
+#include <xen/log2.h>
+
+#define its_print(lvl, fmt, ...) \
+ printk("GIC-ITS:" fmt, ## __VA_ARGS__)
+
+#define its_err(fmt, ...) its_print(XENLOG_ERR, fmt, ## __VA_ARGS__)
+
+#define its_err_ratelimit(fmt, ...) \
+ its_print(XENLOG_ERR_RATE_LIMIT, fmt, ## __VA_ARGS__)
+
+#define its_dbg(fmt, ...) \
+ its_print(XENLOG_DEBUG, fmt, ## __VA_ARGS__)
+
+#define its_info(fmt, ...) \
+ its_print(XENLOG_INFO, fmt, ## __VA_ARGS__)
+
+#define its_warn(fmt, ...) \
+ its_print(XENLOG_WARNING, fmt, ## __VA_ARGS__)
+
+//#define DEBUG_GIC_ITS
+
+#ifdef DEBUG_GIC_ITS
+# define DPRINTK(fmt, args...) printk(XENLOG_DEBUG fmt, ##args)
+#else
+# define DPRINTK(fmt, args...) do {} while ( 0 )
+#endif
+
+#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1 << 0)
+#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)
+
+/*
+ * The ITS structure - contains most of the infrastructure, with the
+ * msi_controller, the command queue, the collections, and the list of
+ * devices writing to it.
+ */
+struct its_node {
+ spinlock_t lock;
+ struct list_head entry;
+ void __iomem *base;
+ unsigned long phys_base;
+ unsigned long phys_size;
+ its_cmd_block *cmd_base;
+ its_cmd_block *cmd_write;
+ void *tables[GITS_BASER_NR_REGS];
+ struct its_collection *collections;
+ u64 flags;
+ u32 ite_size;
+ struct dt_device_node *dt_node;
+};
+
+#define ITS_ITT_ALIGN SZ_256
+
+static LIST_HEAD(its_nodes);
+static DEFINE_SPINLOCK(its_lock);
+static struct rdist_prop *gic_rdists;
+
+#define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
+#define gic_data_rdist_rd_base() (per_cpu(rdist, smp_processor_id()).rbase)
+
+/*
+ * ITS command descriptors - parameters to be encoded in a command
+ * block.
+ */
+struct its_cmd_desc {
+ union {
+ struct {
+ struct its_collection *col;
+ u32 event_id;
+ u32 dev_id;
+ } its_inv_cmd;
+
+ struct {
+ struct its_device *dev;
+ int valid;
+ } its_mapd_cmd;
+
+ struct {
+ struct its_collection *col;
+ int valid;
+ } its_mapc_cmd;
+
+ struct {
+ struct its_device *dev;
+ struct its_collection *col;
+ u32 phys_id;
+ u32 event_id;
+ } its_mapvi_cmd;
+
+ struct {
+ struct its_device *dev;
+ struct its_collection *col;
+ u32 id;
+ } its_movi_cmd;
+
+ struct {
+ struct its_collection *col;
+ } its_invall_cmd;
+ };
+};
+
+#ifdef DEBUG_GIC_ITS
+void dump_cmd(its_cmd_block *cmd)
+{
+ printk("ITS: Phys_cmd CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
+ cmd->raw_cmd[0], cmd->raw_cmd[1], cmd->raw_cmd[2], cmd->raw_cmd[3]);
+}
+#endif
+
+#define ITS_CMD_QUEUE_SZ SZ_64K
+#define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(its_cmd_block))
+
+#define ALIGN(x, a) (((x) + (a - 1)) & ~(a - 1))
+
+typedef struct its_collection *(*its_cmd_builder_t)(its_cmd_block *,
+ struct its_cmd_desc *);
+
+static struct its_collection *its_build_mapd_cmd(its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ unsigned long itt_addr;
+ u8 size;
+
+ size = max(order_base_2(desc->its_mapd_cmd.dev->nr_ites), 1);
+ itt_addr = __pa(desc->its_mapd_cmd.dev->itt_addr);
+ itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
+
+ memset(cmd, 0x0, sizeof(its_cmd_block));
+ cmd->mapd.cmd = GITS_CMD_MAPD;
+ cmd->mapd.devid = desc->its_mapd_cmd.dev->device_id;
+ cmd->mapd.size = size - 1;
+ cmd->mapd.itt = itt_addr >> 8;
+ cmd->mapd.valid = desc->its_mapd_cmd.valid;
+
+#ifdef DEBUG_GIC_ITS
+ dump_cmd(cmd);
+#endif
+
+ /* Take first collection for sync */
+ return &desc->its_mapd_cmd.dev->its->collections[0];
+}
+
+static struct its_collection *its_build_mapc_cmd(its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ memset(cmd, 0x0, sizeof(its_cmd_block));
+ cmd->mapc.cmd = GITS_CMD_MAPC;
+ cmd->mapc.col = desc->its_mapc_cmd.col->col_id;
+ /*
+ * Thought target address field is only 32 bit.
+ * So take bit[48:16]
+ */
+ cmd->mapc.ta = desc->its_mapc_cmd.col->target_address >> 16;
+ cmd->mapc.valid = desc->its_mapc_cmd.valid;
+
+#ifdef DEBUG_GIC_ITS
+ dump_cmd(cmd);
+#endif
+ return desc->its_mapc_cmd.col;
+}
+
+static struct its_collection *its_build_mapvi_cmd(its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ memset(cmd, 0x0, sizeof(its_cmd_block));
+ cmd->mapvi.cmd = GITS_CMD_MAPVI;
+ cmd->mapvi.devid = desc->its_mapvi_cmd.dev->device_id;
+ cmd->mapvi.event = desc->its_mapvi_cmd.event_id;
+ cmd->mapvi.phy_id = desc->its_mapvi_cmd.phys_id;
+ cmd->mapvi.col = desc->its_mapvi_cmd.col->col_id;
+
+#ifdef DEBUG_GIC_ITS
+ dump_cmd(cmd);
+#endif
+
+ return desc->its_mapvi_cmd.col;
+}
+
+static struct its_collection *its_build_movi_cmd(its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ memset(cmd, 0x0, sizeof(its_cmd_block));
+ cmd->movi.cmd = GITS_CMD_MOVI;
+ cmd->movi.devid = desc->its_movi_cmd.dev->device_id;
+ cmd->movi.event = desc->its_movi_cmd.id;
+ cmd->movi.col = desc->its_movi_cmd.col->col_id;
+
+#ifdef DEBUG_GIC_ITS
+ dump_cmd(cmd);
+#endif
+
+ return desc->its_movi_cmd.col;
+}
+
+static struct its_collection *its_build_inv_cmd(its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ memset(cmd, 0x0, sizeof(its_cmd_block));
+ cmd->inv.cmd = GITS_CMD_INV;
+ cmd->inv.devid = desc->its_inv_cmd.dev_id;
+ cmd->inv.event = desc->its_inv_cmd.event_id;
+
+#ifdef DEBUG_GIC_ITS
+ dump_cmd(cmd);
+#endif
+
+ return desc->its_inv_cmd.col;
+}
+
+static struct its_collection *its_build_invall_cmd(its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ memset(cmd, 0x0, sizeof(its_cmd_block));
+ cmd->invall.cmd = GITS_CMD_INVALL;
+ cmd->invall.col = desc->its_mapc_cmd.col->col_id;
+
+#ifdef DEBUG_GIC_ITS
+ dump_cmd(cmd);
+#endif
+
+ return NULL;
+}
+
+static u64 its_cmd_ptr_to_offset(struct its_node *its, its_cmd_block *ptr)
+{
+ return (ptr - its->cmd_base) * sizeof(*ptr);
+}
+
+static int its_queue_full(struct its_node *its)
+{
+ int widx;
+ int ridx;
+
+ widx = its->cmd_write - its->cmd_base;
+ ridx = readl_relaxed(its->base + GITS_CREADR) / sizeof(its_cmd_block);
+
+ /* This is incredibly unlikely to happen, unless the ITS locks up. */
+ if ( ((widx + 1) % ITS_CMD_QUEUE_NR_ENTRIES) == ridx )
+ return 1;
+
+ return 0;
+}
+
+static its_cmd_block *its_allocate_entry(struct its_node *its)
+{
+ its_cmd_block *cmd;
+ u32 count = 1000000; /* 1s! */
+
+ while ( its_queue_full(its) )
+ {
+ count--;
+ if ( !count )
+ {
+ its_err_ratelimit("ITS queue not draining\n");
+ return NULL;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+
+ cmd = its->cmd_write++;
+
+ /* Handle queue wrapping */
+ if (its->cmd_write == (its->cmd_base + ITS_CMD_QUEUE_NR_ENTRIES))
+ its->cmd_write = its->cmd_base;
+
+ return cmd;
+}
+
+static its_cmd_block *its_post_commands(struct its_node *its)
+{
+ u64 wr = its_cmd_ptr_to_offset(its, its->cmd_write);
+
+ writel_relaxed(wr, its->base + GITS_CWRITER);
+
+ return its->cmd_write;
+}
+
+static void its_flush_cmd(struct its_node *its, its_cmd_block *cmd)
+{
+ /*
+ * Make sure the commands written to memory are observable by
+ * the ITS.
+ */
+ if ( its->flags & ITS_FLAGS_CMDQ_NEEDS_FLUSHING )
+ clean_and_invalidate_dcache_va_range(cmd, sizeof(*cmd));
+ else
+ dsb(ishst);
+}
+
+static void its_wait_for_range_completion(struct its_node *its,
+ its_cmd_block *from,
+ its_cmd_block *to)
+{
+ u64 rd_idx, from_idx, to_idx;
+ u32 count = 1000000; /* 1s! */
+
+ from_idx = its_cmd_ptr_to_offset(its, from);
+ to_idx = its_cmd_ptr_to_offset(its, to);
+
+ while ( 1 )
+ {
+ rd_idx = readl_relaxed(its->base + GITS_CREADR);
+ if ( rd_idx >= to_idx || rd_idx < from_idx )
+ break;
+
+ count--;
+ if ( !count )
+ {
+ its_err_ratelimit("ITS queue timeout\n");
+ return;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static void its_send_single_command(struct its_node *its,
+ its_cmd_builder_t builder,
+ struct its_cmd_desc *desc)
+{
+ its_cmd_block *cmd, *sync_cmd, *next_cmd;
+ struct its_collection *sync_col;
+ unsigned long flags;
+
+ spin_lock_irqsave(&its->lock, flags);
+
+ cmd = its_allocate_entry(its);
+ if ( !cmd )
+ { /* We're soooooo screewed... */
+ its_err_ratelimit("ITS can't allocate, dropping command\n");
+ spin_unlock_irqrestore(&its->lock, flags);
+ return;
+ }
+ sync_col = builder(cmd, desc);
+ its_flush_cmd(its, cmd);
+
+ if ( sync_col )
+ {
+ sync_cmd = its_allocate_entry(its);
+ if ( !sync_cmd )
+ {
+ its_err_ratelimit("ITS can't SYNC, skipping\n");
+ goto post;
+ }
+ sync_cmd->sync.cmd = GITS_CMD_SYNC;
+ sync_cmd->sync.ta = sync_col->target_address >> 16;
+ its_flush_cmd(its, sync_cmd);
+ }
+
+post:
+ next_cmd = its_post_commands(its);
+ spin_unlock_irqrestore(&its->lock, flags);
+
+ its_wait_for_range_completion(its, cmd, next_cmd);
+}
+
+void its_send_inv(struct its_device *dev, struct its_collection *col,
+ u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_inv_cmd.dev_id = dev->device_id;
+ desc.its_inv_cmd.event_id = event_id;
+ desc.its_inv_cmd.col = col;
+
+ its_send_single_command(dev->its, its_build_inv_cmd, &desc);
+}
+
+void its_send_mapd(struct its_device *dev, int valid)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapd_cmd.dev = dev;
+ desc.its_mapd_cmd.valid = !!valid;
+
+ its_send_single_command(dev->its, its_build_mapd_cmd, &desc);
+}
+
+static void its_send_mapc(struct its_node *its, struct its_collection *col,
+ int valid)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapc_cmd.col = col;
+ desc.its_mapc_cmd.valid = !!valid;
+
+ its_send_single_command(its, its_build_mapc_cmd, &desc);
+}
+
+void its_send_mapvi(struct its_device *dev, struct its_collection *col,
+ u32 phys_id, u32 event)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapvi_cmd.dev = dev;
+ desc.its_mapvi_cmd.phys_id = phys_id;
+ desc.its_mapvi_cmd.event_id = event;
+ desc.its_mapvi_cmd.col = col;
+
+ its_send_single_command(dev->its, its_build_mapvi_cmd, &desc);
+}
+
+void its_send_movi(struct its_device *dev, struct its_collection *col,
+ u32 event)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_movi_cmd.dev = dev;
+ desc.its_movi_cmd.col = col;
+ desc.its_movi_cmd.id = event;
+
+ its_send_single_command(dev->its, its_build_movi_cmd, &desc);
+}
+
+static void its_send_invall(struct its_node *its, struct its_collection *col)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_invall_cmd.col = col;
+
+ its_send_single_command(its, its_build_invall_cmd, &desc);
+}
+
+/*
+ * How we allocate LPIs:
+ *
+ * The GIC has id_bits bits for interrupt identifiers. From there, we
+ * must subtract 8192 which are reserved for SGIs/PPIs/SPIs. Then, as
+ * we allocate LPIs by chunks of 32, we can shift the whole thing by 5
+ * bits to the right.
+ *
+ * This gives us (((1UL << id_bits) - 8192) >> 5) possible allocations.
+ */
+#define IRQS_PER_CHUNK_SHIFT 5
+#define IRQS_PER_CHUNK (1 << IRQS_PER_CHUNK_SHIFT)
+
+static unsigned long *lpi_bitmap;
+static u32 lpi_chunks;
+static DEFINE_SPINLOCK(lpi_lock);
+
+static int its_lpi_to_chunk(int lpi)
+{
+ return (lpi - 8192) >> IRQS_PER_CHUNK_SHIFT;
+}
+
+static int its_chunk_to_lpi(int chunk)
+{
+ return (chunk << IRQS_PER_CHUNK_SHIFT) + 8192;
+}
+
+int its_lpi_init(u32 id_bits)
+{
+ lpi_chunks = its_lpi_to_chunk(1UL << id_bits);
+
+ lpi_bitmap = xzalloc_bytes(BITS_TO_LONGS(lpi_chunks) * sizeof(long));
+ if ( !lpi_bitmap )
+ {
+ lpi_chunks = 0;
+ return -ENOMEM;
+ }
+
+ its_info("ITS: Allocated %d chunks for LPIs\n", (int)lpi_chunks);
+
+ return 0;
+}
+
+unsigned long *its_lpi_alloc_chunks(int nirqs, int *base, int *nr_ids)
+{
+ unsigned long *bitmap = NULL;
+ int chunk_id;
+ int nr_chunks;
+ int i;
+
+ nr_chunks = DIV_ROUND_UP(nirqs, IRQS_PER_CHUNK);
+
+ spin_lock(&lpi_lock);
+
+ do {
+ chunk_id = bitmap_find_next_zero_area(lpi_bitmap, lpi_chunks,
+ 0, nr_chunks, 0);
+ if ( chunk_id < lpi_chunks )
+ break;
+
+ nr_chunks--;
+ } while ( nr_chunks > 0 );
+
+ if ( !nr_chunks )
+ goto out;
+
+ bitmap = xzalloc_bytes(BITS_TO_LONGS(nr_chunks * IRQS_PER_CHUNK) *
+ sizeof (long));
+ if ( !bitmap )
+ goto out;
+
+ for ( i = 0; i < nr_chunks; i++ )
+ set_bit(chunk_id + i, lpi_bitmap);
+
+ *base = its_chunk_to_lpi(chunk_id);
+ *nr_ids = nr_chunks * IRQS_PER_CHUNK;
+
+out:
+ spin_unlock(&lpi_lock);
+
+ return bitmap;
+}
+
+/*
+ * We allocate 64kB for PROPBASE. That gives us at most 64K LPIs to
+ * deal with (one configuration byte per interrupt). PENDBASE has to
+ * be 64kB aligned (one bit per LPI, plus 8192 bits for SPI/PPI/SGI).
+ */
+#define LPI_PROPBASE_SZ SZ_64K
+#define LPI_PENDBASE_SZ (LPI_PROPBASE_SZ / 8 + SZ_1K)
+
+/*
+ * This is how many bits of ID we need, including the useless ones.
+ */
+#define LPI_NRBITS fls(LPI_PROPBASE_SZ + SZ_8K) - 1
+
+static int __init its_alloc_lpi_tables(void)
+{
+ paddr_t paddr;
+
+ gic_rdists->prop_page =
+ alloc_xenheap_pages(get_order_from_bytes(LPI_PROPBASE_SZ), 0);
+
+ if ( !gic_rdists->prop_page )
+ {
+ its_err("Failed to allocate PROPBASE\n");
+ return -ENOMEM;
+ }
+
+ paddr = __pa(gic_rdists->prop_page);
+ its_info("GIC: using LPI property table @%pa\n", &paddr);
+
+ /* Priority 0xa0, Group-1, disabled */
+ memset(gic_rdists->prop_page,
+ GIC_PRI_IRQ | LPI_PROP_GROUP1,
+ LPI_PROPBASE_SZ);
+
+ /* Make sure the GIC will observe the written configuration */
+ clean_and_invalidate_dcache_va_range(gic_rdists->prop_page,
+ LPI_PROPBASE_SZ);
+
+ return 0;
+}
+
+static const char *its_base_type_string[] = {
+ [GITS_BASER_TYPE_DEVICE] = "Devices",
+ [GITS_BASER_TYPE_VCPU] = "Virtual CPUs",
+ [GITS_BASER_TYPE_CPU] = "Physical CPUs",
+ [GITS_BASER_TYPE_COLLECTION] = "Interrupt Collections",
+ [GITS_BASER_TYPE_RESERVED5] = "Reserved (5)",
+ [GITS_BASER_TYPE_RESERVED6] = "Reserved (6)",
+ [GITS_BASER_TYPE_RESERVED7] = "Reserved (7)",
+};
+
+static void its_free_tables(struct its_node *its)
+{
+ int i;
+
+ for ( i = 0; i < GITS_BASER_NR_REGS; i++ )
+ {
+ if ( its->tables[i] )
+ {
+ xfree(its->tables[i]);
+ its->tables[i] = NULL;
+ }
+ }
+}
+
+static int its_alloc_tables(struct its_node *its)
+{
+ int err;
+ int i;
+ int psz = SZ_64K;
+ u64 shr = GITS_BASER_InnerShareable;
+
+ for ( i = 0; i < GITS_BASER_NR_REGS; i++ )
+ {
+ u64 val = readq_relaxed(its->base + GITS_BASER + i * 8);
+ u64 type = GITS_BASER_TYPE(val);
+ u64 entry_size = GITS_BASER_ENTRY_SIZE(val);
+ int order = get_order_from_bytes(psz);
+ int alloc_size;
+ u64 tmp;
+ void *base;
+
+ if (type == GITS_BASER_TYPE_NONE)
+ continue;
+
+ /*
+ * Allocate as many entries as required to fit the
+ * range of device IDs that the ITS can grok... The ID
+ * space being incredibly sparse, this results in a
+ * massive waste of memory.
+ *
+ * For other tables, only allocate a single page.
+ */
+ if ( type == GITS_BASER_TYPE_DEVICE )
+ {
+ u64 typer = readq_relaxed(its->base + GITS_TYPER);
+ u32 ids = GITS_TYPER_DEVBITS(typer);
+
+ order = get_order_from_bytes((1UL << ids) * 8);
+ if (order >= MAX_ORDER)
+ {
+ order = MAX_ORDER - 1;
+ its_warn("Device Table too large, reduce its page order to %u\n",
+ order);
+ }
+ }
+
+ alloc_size = (1 << order) * PAGE_SIZE;
+ base = alloc_xenheap_pages(order, 0);
+ if ( !base )
+ {
+ err = -ENOMEM;
+ goto out_free;
+ }
+ memset(base, 0, alloc_size);
+ its->tables[i] = base;
+retry_baser:
+ val = (__pa(base) |
+ (type << GITS_BASER_TYPE_SHIFT) |
+ ((entry_size - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
+ GITS_BASER_WaWb |
+ shr |
+ GITS_BASER_VALID);
+
+ switch (psz) {
+ case SZ_4K:
+ val |= GITS_BASER_PAGE_SIZE_4K;
+ break;
+ case SZ_16K:
+ val |= GITS_BASER_PAGE_SIZE_16K;
+ break;
+ case SZ_64K:
+ val |= GITS_BASER_PAGE_SIZE_64K;
+ break;
+ }
+
+ val |= (((alloc_size / psz) - 1) & 0xffUL);
+
+ writeq_relaxed(val, its->base + GITS_BASER + i * 8);
+ tmp = readq_relaxed(its->base + GITS_BASER + i * 8);
+
+ if ( (val ^ tmp) & GITS_BASER_SHAREABILITY_MASK )
+ {
+ /*
+ * Shareability didn't stick. Just use
+ * whatever the read reported, which is likely
+ * to be the only thing this redistributor
+ * supports.
+ */
+ shr = tmp & GITS_BASER_SHAREABILITY_MASK;
+ goto retry_baser;
+ }
+
+ if ( (val ^ tmp) & GITS_BASER_PAGE_SIZE_MASK )
+ {
+ /*
+ * Page size didn't stick. Let's try a smaller
+ * size and retry. If we reach 4K, then
+ * something is horribly wrong...
+ */
+ switch (psz) {
+ case SZ_16K:
+ psz = SZ_4K;
+ goto retry_baser;
+ case SZ_64K:
+ psz = SZ_16K;
+ goto retry_baser;
+ }
+ }
+
+ /* skip comparin cacheability fields as they are implementation
+ * defined.
+ */
+ val = val << 5;
+ tmp = tmp << 5;
+
+ if ( val != tmp )
+ {
+ its_err("ITS: GITS_BASER%d doesn't stick: %lx %lx\n",
+ i, (unsigned long) val, (unsigned long) tmp);
+ err = -ENXIO;
+ goto out_free;
+ }
+
+ its_info("ITS: allocated %d %s @%lx (psz %dK, shr %d)\n",
+ (int)(alloc_size / entry_size),
+ its_base_type_string[type],
+ (unsigned long)__pa(base),
+ psz / SZ_1K, (int)shr >> GITS_BASER_SHAREABILITY_SHIFT);
+ }
+
+ return 0;
+
+out_free:
+ its_free_tables(its);
+
+ return err;
+}
+
+static int its_alloc_collections(struct its_node *its)
+{
+ its->collections = xzalloc_array(struct its_collection, nr_cpu_ids);
+ if ( !its->collections )
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void its_cpu_init_lpis(void)
+{
+ void __iomem *rbase = gic_data_rdist_rd_base();
+ void *pend_page;
+ u64 val, tmp;
+
+ /* If we didn't allocate the pending table yet, do it now */
+ pend_page = gic_data_rdist().pend_page;
+ if ( !pend_page )
+ {
+ paddr_t paddr;
+ u32 order;
+ /*
+ * The pending pages have to be at least 64kB aligned,
+ * hence the 'max(LPI_PENDBASE_SZ, SZ_64K)' below.
+ */
+ order = get_order_from_bytes(max(LPI_PENDBASE_SZ, SZ_64K));
+ pend_page = alloc_xenheap_pages(order, 0);
+ if ( !pend_page )
+ {
+ its_err("Failed to allocate PENDBASE for CPU%d with order %d\n",
+ smp_processor_id(), order);
+ return;
+ }
+
+ memset(pend_page, 0, max(LPI_PENDBASE_SZ, SZ_64K));
+ /* Make sure the GIC will observe the zero-ed page */
+ clean_and_invalidate_dcache_va_range(pend_page, LPI_PENDBASE_SZ);
+
+ paddr = __pa(pend_page);
+
+ its_info("CPU%d: using LPI pending table @%pa\n",
+ smp_processor_id(), &paddr);
+
+ gic_data_rdist().pend_page = pend_page;
+ }
+
+ /* Disable LPIs */
+ val = readl_relaxed(rbase + GICR_CTLR);
+ val &= ~GICR_CTLR_ENABLE_LPIS;
+ writel_relaxed(val, rbase + GICR_CTLR);
+
+ /*
+ * Make sure any change to the table is observable by the GIC.
+ */
+ dsb(sy);
+
+ /* set PROPBASE */
+ val = (__pa(gic_rdists->prop_page) |
+ GICR_PROPBASER_InnerShareable |
+ GICR_PROPBASER_WaWb |
+ ((LPI_NRBITS - 1) & GICR_PROPBASER_IDBITS_MASK));
+
+ writeq_relaxed(val, rbase + GICR_PROPBASER);
+ tmp = readq_relaxed(rbase + GICR_PROPBASER);
+
+ if ( (tmp ^ val) & GICR_PROPBASER_SHAREABILITY_MASK )
+ {
+ its_info("GIC: using cache flushing for LPI property table\n");
+ gic_rdists->flags |= RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING;
+ }
+
+ /* set PENDBASE */
+ val = (__pa(pend_page) |
+ GICR_PROPBASER_InnerShareable |
+ GICR_PROPBASER_WaWb);
+
+ writeq_relaxed(val, rbase + GICR_PENDBASER);
+
+ /* Enable LPIs */
+ val = readl_relaxed(rbase + GICR_CTLR);
+ val |= GICR_CTLR_ENABLE_LPIS;
+ writel_relaxed(val, rbase + GICR_CTLR);
+
+ /* Make sure the GIC has seen the above */
+ dsb(sy);
+}
+
+static void its_cpu_init_collection(void)
+{
+ struct its_node *its;
+ int cpu;
+
+ spin_lock(&its_lock);
+ cpu = smp_processor_id();
+
+ list_for_each_entry(its, &its_nodes, entry)
+ {
+ u64 target;
+ /*
+ * We now have to bind each collection to its target
+ * redistributor.
+ */
+ if ( readq_relaxed(its->base + GITS_TYPER) & GITS_TYPER_PTA )
+ {
+ /*
+ * This ITS wants the physical address of the
+ * redistributor.
+ */
+ target = gic_data_rdist().phys_base;
+ }
+ else
+ {
+ /*
+ * This ITS wants a linear CPU number.
+ */
+ target = readq_relaxed(gic_data_rdist_rd_base() + GICR_TYPER);
+ target = GICR_TYPER_CPU_NUMBER(target);
+ }
+
+ /* Perform collection mapping */
+ its->collections[cpu].target_address = target;
+ its->collections[cpu].col_id = cpu;
+
+ its_send_mapc(its, &its->collections[cpu], 1);
+ its_send_invall(its, &its->collections[cpu]);
+ }
+
+ spin_unlock(&its_lock);
+}
+
+static int its_force_quiescent(void __iomem *base)
+{
+ u32 count = 1000000; /* 1s */
+ u32 val;
+
+ val = readl_relaxed(base + GITS_CTLR);
+ if ( val & GITS_CTLR_QUIESCENT )
+ return 0;
+
+ /* Disable the generation of all interrupts to this ITS */
+ val &= ~GITS_CTLR_ENABLE;
+ writel_relaxed(val, base + GITS_CTLR);
+
+ /* Poll GITS_CTLR and wait until ITS becomes quiescent */
+ while ( 1 )
+ {
+ val = readl_relaxed(base + GITS_CTLR);
+ if ( val & GITS_CTLR_QUIESCENT )
+ return 0;
+
+ count--;
+ if ( !count )
+ return -EBUSY;
+
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static int its_probe(struct dt_device_node *node)
+{
+ paddr_t its_addr, its_size;
+ struct its_node *its;
+ void __iomem *its_base;
+ u32 val, typer;
+ u64 baser, tmp;
+ int err;
+
+ err = dt_device_get_address(node, 0, &its_addr, &its_size);
+ if ( err )
+ {
+ its_warn("%s: no regs?\n", node->full_name);
+ return -ENXIO;
+ }
+
+ its_base = ioremap_nocache(its_addr, its_size);
+ if ( !its_base )
+ {
+ its_warn("%s: unable to map registers\n", node->full_name);
+ return -ENOMEM;
+ }
+
+ val = readl_relaxed(its_base + GITS_PIDR2) & GIC_PIDR2_ARCH_REV_MASK;
+ if ( val != 0x30 && val != 0x40 )
+ {
+ its_warn("%s: no ITS detected, giving up\n", node->full_name);
+ err = -ENODEV;
+ goto out_unmap;
+ }
+
+ err = its_force_quiescent(its_base);
+ if ( err )
+ {
+ its_warn("%s: failed to quiesce, giving up\n",
+ node->full_name);
+ goto out_unmap;
+ }
+
+ its_info("ITS: %s\n", node->full_name);
+
+ its = xzalloc(struct its_node);
+ if ( !its )
+ {
+ err = -ENOMEM;
+ goto out_unmap;
+ }
+
+ spin_lock_init(&its->lock);
+ INIT_LIST_HEAD(&its->entry);
+ its->dt_node = node;
+ its->base = its_base;
+ its->phys_base = its_addr;
+ its->phys_size = its_size;
+ typer = readl_relaxed(its_base + GITS_TYPER);
+ its->ite_size = ((typer >> 4) & 0xf) + 1;
+
+ its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
+ if ( !its->cmd_base )
+ {
+ err = -ENOMEM;
+ goto out_free_its;
+ }
+ its->cmd_write = its->cmd_base;
+
+ err = its_alloc_tables(its);
+ if ( err )
+ goto out_free_cmd;
+
+ err = its_alloc_collections(its);
+ if ( err )
+ goto out_free_tables;
+
+ baser = (__pa(its->cmd_base) |
+ GITS_CBASER_WaWb |
+ GITS_CBASER_InnerShareable |
+ (ITS_CMD_QUEUE_SZ / SZ_4K - 1) |
+ GITS_CBASER_VALID);
+
+ writeq_relaxed(baser, its->base + GITS_CBASER);
+ tmp = readq_relaxed(its->base + GITS_CBASER);
+ writeq_relaxed(0, its->base + GITS_CWRITER);
+ writel_relaxed(GITS_CTLR_ENABLE, its->base + GITS_CTLR);
+
+ if ( (tmp ^ baser) & GITS_BASER_SHAREABILITY_MASK )
+ {
+ its_info("ITS: using cache flushing for cmd queue\n");
+ its->flags |= ITS_FLAGS_CMDQ_NEEDS_FLUSHING;
+ }
+ spin_lock(&its_lock);
+ list_add(&its->entry, &its_nodes);
+ spin_unlock(&its_lock);
+
+ return 0;
+
+out_free_tables:
+ its_free_tables(its);
+out_free_cmd:
+ xfree(its->cmd_base);
+out_free_its:
+ xfree(its);
+out_unmap:
+ iounmap(its_base);
+ its_err("ITS: failed probing %s (%d)\n", node->full_name, err);
+ return err;
+}
+
+static bool gic_rdists_supports_plpis(void)
+{
+ return !!(readl_relaxed(gic_data_rdist_rd_base() + GICR_TYPER) & GICR_TYPER_PLPIS);
+}
+
+int its_cpu_init(void)
+{
+ if ( !list_empty(&its_nodes) )
+ {
+ if ( !gic_rdists_supports_plpis() )
+ {
+ its_info("CPU%d: LPIs not supported\n", smp_processor_id());
+ return -ENXIO;
+ }
+ its_cpu_init_lpis();
+ its_cpu_init_collection();
+ }
+
+ return 0;
+}
+
+int its_init(struct rdist_prop *rdists)
+{
+ struct dt_device_node *np = NULL;
+
+ static const struct dt_device_match its_device_ids[] __initconst =
+ {
+ DT_MATCH_GIC_ITS,
+ { /* sentinel */ },
+ };
+
+ for (np = dt_find_matching_node(NULL, its_device_ids); np;
+ np = dt_find_matching_node(np, its_device_ids))
+ its_probe(np);
+
+ if ( list_empty(&its_nodes) )
+ {
+ its_warn("ITS: No ITS available, not enabling LPIs\n");
+ return -ENXIO;
+ }
+
+ gic_rdists = rdists;
+ its_alloc_lpi_tables();
+
+ BUILD_BUG_ON(sizeof(its_cmd_block) != 32);
+
+ return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
new file mode 100644
index 0000000..4e42f7f
--- /dev/null
+++ b/xen/include/asm-arm/gic-its.h
@@ -0,0 +1,214 @@
+/*
+ * Copyright (C) 2015 Cavium Inc.
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_ARM_GIC_ITS_H__
+#define __ASM_ARM_GIC_ITS_H__
+
+#include <asm/gic_v3_defs.h>
+
+/*
+ * Collection structure - just an ID, and a redistributor address to
+ * ping. We use one per CPU as a bag of interrupts assigned to this
+ * CPU.
+ */
+struct its_collection {
+ u64 target_address;
+ u16 col_id;
+};
+
+/* ITS command structures */
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:56;
+ u64 res2:64;
+ u64 col:16;
+ u64 ta:32;
+ u64 res3:15;
+ u64 valid:1;
+ u64 res4:64;
+}mapc_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:24;
+ u64 devid:32;
+ u64 size:5;
+ u64 res2:59;
+ u64 res3:8;
+ u64 itt:40;
+ u64 res4:15;
+ u64 valid:1;
+ u64 res5:64;
+}mapd_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:24;
+ u64 devid:32;
+ u64 event:32;
+ u64 res2:32;
+ u64 col:16;
+ u64 res3:48;
+ u64 res4:64;
+}mapi_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:24;
+ u64 devid:32;
+ u64 event:32;
+ u64 phy_id:32;
+ u64 col:16;
+ u64 res3:48;
+ u64 res4:64;
+}mapvi_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:24;
+ u64 devid:32;
+ u64 event:32;
+ u64 res2:32;
+ u64 col:16;
+ u64 res3:48;
+ u64 res4:64;
+}movi_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:24;
+ u64 devid:32;
+ u64 event:32;
+ u64 res2:32;
+ u64 res3:64;
+ u64 res4:64;
+}discard_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:24;
+ u64 devid:32;
+ u64 event:32;
+ u64 res2:32;
+ u64 res3:64;
+ u64 res4:64;
+}inv_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:56;
+ u64 res2:64;
+ u64 res3:16;
+ u64 ta1:32;
+ u64 res4:16;
+ u64 res5:16;
+ u64 ta2:32;
+ u64 res6:16;
+}movall_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:56;
+ u64 res2:64;
+ u64 col:16;
+ u64 res3:48;
+ u64 res4:64;
+}invall_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:24;
+ u64 devid:32;
+ u64 event:32;
+ u64 res2:32;
+ u64 res3:64;
+ u64 res4:64;
+}int_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:24;
+ u64 devid:32;
+ u64 event:32;
+ u64 res2:32;
+ u64 res3:64;
+ u64 res4:64;
+}clear_cmd_t;
+
+typedef struct __packed {
+ u64 cmd:8;
+ u64 res1:56;
+ u64 res2:64;
+ u64 res3:16;
+ u64 ta:32;
+ u64 res4:16;
+ u64 res5:64;
+}sync_cmd_t;
+
+typedef union {
+ u64 raw_cmd[4];
+ mapc_cmd_t mapc;
+ mapd_cmd_t mapd;
+ mapi_cmd_t mapi;
+ mapvi_cmd_t mapvi;
+ movi_cmd_t movi;
+ movall_cmd_t movall;
+ inv_cmd_t inv;
+ invall_cmd_t invall;
+ discard_cmd_t discard;
+ int_cmd_t int_cmd;
+ clear_cmd_t clear;
+ sync_cmd_t sync;
+}its_cmd_block;
+/*
+ * The ITS view of a device - belongs to an ITS, a collection, owns an
+ * interrupt translation table, and a list of interrupts.
+ */
+struct its_device {
+ /* Physical ITS */
+ struct its_node *its;
+ /* Device ITT address */
+ paddr_t itt_addr;
+ /* Device ITT size */
+ unsigned long itt_size;
+ /* Physical LPI map */
+ unsigned long *lpi_map;
+ /* First Physical LPI number assigned */
+ u32 lpi_base;
+ /* Number of Physical LPIs assigned */
+ int nr_lpis;
+ /* Number of ITES entries */
+ u32 nr_ites;
+ /* Physical Device id */
+ u32 device_id;
+};
+
+static inline uint8_t its_decode_cmd(its_cmd_block *cmd)
+{
+ return cmd->raw_cmd[0] & 0xff;
+}
+
+#endif /* __ASM_ARM_GIC_ITS_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 9e2acb7..e9d5f36 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -161,6 +161,7 @@
DT_MATCH_COMPATIBLE("arm,gic-400")
#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
+#define DT_MATCH_GIC_ITS DT_MATCH_COMPATIBLE("arm,gic-v3-its")
/*
* GICv3 registers that needs to be saved/restored
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index acbb906..dc4fe14 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -45,9 +45,11 @@
#define GICC_SRE_EL2_DIB (1UL << 2)
#define GICC_SRE_EL2_ENEL1 (1UL << 3)
+#define GICR_CTL_ENABLE (1U << 0)
/* Additional bits in GICD_TYPER defined by GICv3 */
#define GICD_TYPE_ID_BITS_SHIFT 19
+#define GICD_TYPER_LPIS_SUPPORTED (1U << 17)
#define GICD_CTLR_RWP (1UL << 31)
#define GICD_CTLR_ARE_NS (1U << 4)
#define GICD_CTLR_ENABLE_G1A (1U << 1)
@@ -59,11 +61,12 @@
#define GICR_WAKER_ProcessorSleep (1U << 1)
#define GICR_WAKER_ChildrenAsleep (1U << 2)
-#define GICD_PIDR2_ARCH_REV_MASK (0xf0)
+#define GIC_PIDR2_ARCH_REV_MASK (0xf0)
+#define GICD_PIDR2_ARCH_REV_MASK GIC_PIDR2_ARCH_REV_MASK
#define GICD_PIDR2_ARCH_REV_SHIFT (0x4)
#define GICD_PIDR2_ARCH_GICV3 (0x3)
-#define GICR_PIDR2_ARCH_REV_MASK GICD_PIDR2_ARCH_REV_MASK
+#define GICR_PIDR2_ARCH_REV_MASK GIC_PIDR2_ARCH_REV_MASK
#define GICR_PIDR2_ARCH_REV_SHIFT GICD_PIDR2_ARCH_REV_SHIFT
#define GICR_PIDR2_ARCH_GICV3 GICD_PIDR2_ARCH_GICV3
@@ -113,8 +116,25 @@
#define GICR_ICFGR1 (0x0C04)
#define GICR_NSACR (0x0E00)
+#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
+#define GICR_TYPER_CPU_NUMBER(r) (((r) >> 8) & 0xffff)
+
+#define GICR_PROPBASER_NonShareable (0U << 10)
+#define GICR_PROPBASER_InnerShareable (1U << 10)
+#define GICR_PROPBASER_OuterShareable (2U << 10)
+#define GICR_PROPBASER_SHAREABILITY_MASK (3UL << 10)
+#define GICR_PROPBASER_nCnB (0U << 7)
+#define GICR_PROPBASER_nC (1U << 7)
+#define GICR_PROPBASER_RaWt (2U << 7)
+#define GICR_PROPBASER_RaWb (3U << 7)
+#define GICR_PROPBASER_WaWt (4U << 7)
+#define GICR_PROPBASER_WaWb (5U << 7)
+#define GICR_PROPBASER_RaWaWt (6U << 7)
+#define GICR_PROPBASER_RaWaWb (7U << 7)
+#define GICR_PROPBASER_IDBITS_MASK (0x1f)
#define GICR_TYPER_PLPIS (1U << 0)
#define GICR_TYPER_VLPIS (1U << 1)
+#define GICR_TYPER_DISTRIBUTED_IMP (1U << 3)
#define GICR_TYPER_LAST (1U << 4)
#define DEFAULT_PMR_VALUE 0xff
@@ -152,6 +172,109 @@
#define ICH_SGI_IRQ_SHIFT 24
#define ICH_SGI_IRQ_MASK 0xf
#define ICH_SGI_TARGETLIST_MASK 0xffff
+#define LPI_PROP_GROUP1 (1 << 1)
+#define LPI_PROP_ENABLED (1 << 0)
+
+/*
+ * ITS registers, offsets from ITS_base
+ */
+#define GITS_CTLR 0x0000
+#define GITS_IIDR 0x0004
+#define GITS_TYPER 0x0008
+#define GITS_CBASER 0x0080
+#define GITS_CWRITER 0x0088
+#define GITS_CREADR 0x0090
+#define GITS_BASER0 0x0100
+#define GITS_BASER1 0x0108
+#define GITS_BASER 0x0100
+#define GITS_BASERN 0x013c
+#define GITS_PIDR0 GICR_PIDR0
+#define GITS_PIDR1 GICR_PIDR1
+#define GITS_PIDR2 GICR_PIDR2
+#define GITS_PIDR3 GICR_PIDR3
+#define GITS_PIDR4 GICR_PIDR4
+#define GITS_PIDR5 GICR_PIDR5
+#define GITS_PIDR7 GICR_PIDR7
+
+#define GITS_TRANSLATER 0x10040
+#define GITS_CTLR_QUIESCENT (1U << 31)
+#define GITS_CTLR_ENABLE (1U << 0)
+
+#define GITS_TYPER_DEVBITS_SHIFT 13
+#define GITS_TYPER_DEVBITS(r) ((((r) >> GITS_TYPER_DEVBITS_SHIFT) & 0x1f) + 1)
+#define GITS_TYPER_IDBITS_SHIFT 8
+#define GITS_TYPER_IDBITS(r) ((((r) >> GITS_TYPER_IDBITS_SHIFT) & 0x1f) + 1)
+#define GITS_TYPER_PTA (1UL << 19)
+#define GITS_TYPER_HCC_SHIFT (24)
+
+#define GITS_CBASER_VALID (1UL << 63)
+#define GITS_CBASER_nCnB (0UL << 59)
+#define GITS_CBASER_nC (1UL << 59)
+#define GITS_CBASER_RaWt (2UL << 59)
+#define GITS_CBASER_RaWb (3UL << 59)
+#define GITS_CBASER_WaWt (4UL << 59)
+#define GITS_CBASER_WaWb (5UL << 59)
+#define GITS_CBASER_RaWaWt (6UL << 59)
+#define GITS_CBASER_RaWaWb (7UL << 59)
+#define GITS_CBASER_NonShareable (0UL << 10)
+#define GITS_CBASER_InnerShareable (1UL << 10)
+#define GITS_CBASER_OuterShareable (2UL << 10)
+#define GITS_CBASER_SHAREABILITY_MASK (3UL << 10)
+
+#define GITS_BASER_NR_REGS 8
+
+#define GITS_BASER_VALID (1UL << 63)
+#define GITS_BASER_nCnB (0UL << 59)
+#define GITS_BASER_nC (1UL << 59)
+#define GITS_BASER_RaWt (2UL << 59)
+#define GITS_BASER_RaWb (3UL << 59)
+#define GITS_BASER_WaWt (4UL << 59)
+#define GITS_BASER_WaWb (5UL << 59)
+#define GITS_BASER_RaWaWt (6UL << 59)
+#define GITS_BASER_RaWaWb (7UL << 59)
+#define GITS_BASER_TYPE_SHIFT (56)
+#define GITS_BASER_TYPE_MASK (0x7)
+#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
+#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
+#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0xff) + 1)
+#define GITS_BASER_NonShareable (0UL << 10)
+#define GITS_BASER_InnerShareable (1UL << 10)
+#define GITS_BASER_OuterShareable (2UL << 10)
+#define GITS_BASER_SHAREABILITY_SHIFT (10)
+#define GITS_BASER_SHAREABILITY_MASK (3UL << GITS_BASER_SHAREABILITY_SHIFT)
+#define GITS_BASER_PAGE_SIZE_SHIFT (8)
+#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_4K_VAL (0)
+#define GITS_BASER_PAGE_SIZE_16K_VAL (1)
+#define GITS_BASER_PAGE_SIZE_MASK_VAL (0x3)
+#define GITS_BASER_PAGES_MASK_VAL (0xff)
+#define GITS_BASER_TYPE_NONE 0
+#define GITS_BASER_TYPE_DEVICE 1
+#define GITS_BASER_TYPE_VCPU 2
+#define GITS_BASER_TYPE_CPU 3
+#define GITS_BASER_TYPE_COLLECTION 4
+#define GITS_BASER_TYPE_RESERVED5 5
+#define GITS_BASER_TYPE_RESERVED6 6
+#define GITS_BASER_TYPE_RESERVED7 7
+
+/*
+ * ITS commands
+ */
+#define GITS_CMD_MAPD 0x08
+#define GITS_CMD_MAPC 0x09
+#define GITS_CMD_MAPVI 0x0a
+#define GITS_CMD_MAPI 0x0b
+#define GITS_CMD_MOVI 0x01
+#define GITS_CMD_DISCARD 0x0f
+#define GITS_CMD_INV 0x0c
+#define GITS_CMD_MOVALL 0x0e
+#define GITS_CMD_INVALL 0x0d
+#define GITS_CMD_INT 0x03
+#define GITS_CMD_CLEAR 0x04
+#define GITS_CMD_SYNC 0x05
struct rdist {
void __iomem *rbase;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen
2015-06-22 12:01 ` [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen vijay.kilari
@ 2015-06-22 17:16 ` Julien Grall
2015-06-26 9:19 ` Vijay Kilari
2015-06-29 11:39 ` Ian Campbell
1 sibling, 1 reply; 88+ messages in thread
From: Julien Grall @ 2015-06-22 17:16 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi,
On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Only required changes from Linux ITS driver is ported
> and compiled
Can you list the changes you took from Linux? The coding style is not
the same so it's hard to know what is the difference.
It would also have been nice to get the latest fix from Linux and say on
which version you are based.
[..]
> +/*
> + * The ITS structure - contains most of the infrastructure, with the
> + * msi_controller, the command queue, the collections, and the list of
> + * devices writing to it.
> + */
> +struct its_node {
> + spinlock_t lock;
> + struct list_head entry;
> + void __iomem *base;
The indentation seems wrong.
> + unsigned long phys_base;
> + unsigned long phys_size;
> + its_cmd_block *cmd_base;
> + its_cmd_block *cmd_write;
> + void *tables[GITS_BASER_NR_REGS];
> + struct its_collection *collections;
> + u64 flags;
> + u32 ite_size;
> + struct dt_device_node *dt_node;
> +};
> +
> +#define ITS_ITT_ALIGN SZ_256
> +
> +static LIST_HEAD(its_nodes);
> +static DEFINE_SPINLOCK(its_lock);
> +static struct rdist_prop *gic_rdists;
> +
> +#define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
> +#define gic_data_rdist_rd_base() (per_cpu(rdist, smp_processor_id()).rbase)
You could use gic_data_rdist().rbase
[..]
> +#ifdef DEBUG_GIC_ITS
> +void dump_cmd(its_cmd_block *cmd)
> +{
> + printk("ITS: Phys_cmd CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
> + cmd->raw_cmd[0], cmd->raw_cmd[1], cmd->raw_cmd[2], cmd->raw_cmd[3]);
> +}
> +#endif
> +
> +#define ITS_CMD_QUEUE_SZ SZ_64K
> +#define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(its_cmd_block))
> +
> +#define ALIGN(x, a) (((x) + (a - 1)) & ~(a - 1))
Please use ROUNDUP which does the same job.
> +
> +typedef struct its_collection *(*its_cmd_builder_t)(its_cmd_block *,
> + struct its_cmd_desc *);
> +
> +static struct its_collection *its_build_mapd_cmd(its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + unsigned long itt_addr;
> + u8 size;
> +
> + size = max(order_base_2(desc->its_mapd_cmd.dev->nr_ites), 1);
Why did you replace the ilog2 by order_base_2?
> + itt_addr = __pa(desc->its_mapd_cmd.dev->itt_addr);
> + itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
> +
> + memset(cmd, 0x0, sizeof(its_cmd_block));
You duplicate this line in ever build_*_cmd function. I think you should
call only in one place before the builder.
> + cmd->mapd.cmd = GITS_CMD_MAPD;
> + cmd->mapd.devid = desc->its_mapd_cmd.dev->device_id;
> + cmd->mapd.size = size - 1;
> + cmd->mapd.itt = itt_addr >> 8;
I think the code is more difficult to read without the helpers. You
opencode every trick in all the builder rather than in a single place.
> + cmd->mapd.valid = desc->its_mapd_cmd.valid;
> +
> +#ifdef DEBUG_GIC_ITS
> + dump_cmd(cmd);
> +#endif
The dump can be done in a single place rather than in every builder.
> + /* Take first collection for sync */
> + return &desc->its_mapd_cmd.dev->its->collections[0];
> +}
> +
> +static struct its_collection *its_build_mapc_cmd(its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + memset(cmd, 0x0, sizeof(its_cmd_block));
> + cmd->mapc.cmd = GITS_CMD_MAPC;
> + cmd->mapc.col = desc->its_mapc_cmd.col->col_id;
> + /*
> + * Thought target address field is only 32 bit.
I don't understand what you mean with "thought".
> + * So take bit[48:16]
Can you explain why you are using only the [48:16]? I.e that target
addresses must be 64KB aligned...
> + */
> + cmd->mapc.ta = desc->its_mapc_cmd.col->target_address >> 16;
> + cmd->mapc.valid = desc->its_mapc_cmd.valid;
> +
> +#ifdef DEBUG_GIC_ITS
> + dump_cmd(cmd);
> +#endif
Missing newline.
[..]
> +static void its_send_single_command(struct its_node *its,
> + its_cmd_builder_t builder,
> + struct its_cmd_desc *desc)
> +{
> + its_cmd_block *cmd, *sync_cmd, *next_cmd;
> + struct its_collection *sync_col;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&its->lock, flags);
> +
> + cmd = its_allocate_entry(its);
> + if ( !cmd )
> + { /* We're soooooo screewed... */
I think the /* ... */ should go on a separate line.
> + its_err_ratelimit("ITS can't allocate, dropping command\n");
> + spin_unlock_irqrestore(&its->lock, flags);
> + return;
> + }
> + sync_col = builder(cmd, desc);
> + its_flush_cmd(its, cmd);
> +
> + if ( sync_col )
> + {
> + sync_cmd = its_allocate_entry(its);
> + if ( !sync_cmd )
> + {
> + its_err_ratelimit("ITS can't SYNC, skipping\n");
> + goto post;
> + }
> + sync_cmd->sync.cmd = GITS_CMD_SYNC;
> + sync_cmd->sync.ta = sync_col->target_address >> 16;
This is the second place where you use target_address with ">> 16". May
I ask why you dropped the helpers? At least it keeping such shift in a
single place.
Also, given that all the usage of target_address is done with shift.
Shouldn't you just do the shift once at setup?
[..]
> +/*
> + * We allocate 64kB for PROPBASE. That gives us at most 64K LPIs to
> + * deal with (one configuration byte per interrupt). PENDBASE has to
> + * be 64kB aligned (one bit per LPI, plus 8192 bits for SPI/PPI/SGI).
> + */
> +#define LPI_PROPBASE_SZ SZ_64K
> +#define LPI_PENDBASE_SZ (LPI_PROPBASE_SZ / 8 + SZ_1K)
> +
> +/*
> + * This is how many bits of ID we need, including the useless ones.
> + */
> +#define LPI_NRBITS fls(LPI_PROPBASE_SZ + SZ_8K) - 1
Missing ( ... ) to ensure that LPI_NRBITS will be expanded correctly.
Although there is no need to switch to fls(...) you recently introduced
ilog2.
> +
> +static int __init its_alloc_lpi_tables(void)
> +{
> + paddr_t paddr;
> +
> + gic_rdists->prop_page =
> + alloc_xenheap_pages(get_order_from_bytes(LPI_PROPBASE_SZ), 0);
> +
> + if ( !gic_rdists->prop_page )
> + {
> + its_err("Failed to allocate PROPBASE\n");
> + return -ENOMEM;
> + }
> +
> + paddr = __pa(gic_rdists->prop_page);
> + its_info("GIC: using LPI property table @%pa\n", &paddr);
> +
> + /* Priority 0xa0, Group-1, disabled */
Well, you are not sure that GIC_PRI_IRQ is equal to 0xa0.
> + memset(gic_rdists->prop_page,
> + GIC_PRI_IRQ | LPI_PROP_GROUP1,
> + LPI_PROPBASE_SZ);
> +
> + /* Make sure the GIC will observe the written configuration */
> + clean_and_invalidate_dcache_va_range(gic_rdists->prop_page,
> + LPI_PROPBASE_SZ);
> +
> + return 0;
> +}
> +
> +static const char *its_base_type_string[] = {
> + [GITS_BASER_TYPE_DEVICE] = "Devices",
> + [GITS_BASER_TYPE_VCPU] = "Virtual CPUs",
> + [GITS_BASER_TYPE_CPU] = "Physical CPUs",
> + [GITS_BASER_TYPE_COLLECTION] = "Interrupt Collections",
> + [GITS_BASER_TYPE_RESERVED5] = "Reserved (5)",
> + [GITS_BASER_TYPE_RESERVED6] = "Reserved (6)",
> + [GITS_BASER_TYPE_RESERVED7] = "Reserved (7)",
> +};
> +
> +static void its_free_tables(struct its_node *its)
> +{
> + int i;
> +
> + for ( i = 0; i < GITS_BASER_NR_REGS; i++ )
> + {
> + if ( its->tables[i] )
> + {
> + xfree(its->tables[i]);
As said on the previous version, the memory for the table is allocated
via alloc_xenheap_pages. So freeing the memory should be done via
free_xenheap_pages.
> + its->tables[i] = NULL;
> + }
> + }
> +}
> +
> +static int its_alloc_tables(struct its_node *its)
> +{
> + int err;
> + int i;
> + int psz = SZ_64K;
> + u64 shr = GITS_BASER_InnerShareable;
> +
> + for ( i = 0; i < GITS_BASER_NR_REGS; i++ )
> + {
> + u64 val = readq_relaxed(its->base + GITS_BASER + i * 8);
> + u64 type = GITS_BASER_TYPE(val);
> + u64 entry_size = GITS_BASER_ENTRY_SIZE(val);
> + int order = get_order_from_bytes(psz);
> + int alloc_size;
> + u64 tmp;
> + void *base;
> +
> + if (type == GITS_BASER_TYPE_NONE)
if ( ... )
> + continue;
> +
> + /*
> + * Allocate as many entries as required to fit the
> + * range of device IDs that the ITS can grok... The ID
> + * space being incredibly sparse, this results in a
> + * massive waste of memory.
> + *
> + * For other tables, only allocate a single page.
> + */
> + if ( type == GITS_BASER_TYPE_DEVICE )
> + {
> + u64 typer = readq_relaxed(its->base + GITS_TYPER);
> + u32 ids = GITS_TYPER_DEVBITS(typer);
> +
> + order = get_order_from_bytes((1UL << ids) * 8);
Why 8 and not entry_size?
Also, the latest Linux version is using max(...) to round up to the
default page granularity if the size is smaller.
> + if (order >= MAX_ORDER)
> + {
> + order = MAX_ORDER - 1;
> + its_warn("Device Table too large, reduce its page order to %u\n",
> + order);
> + }
> + }
> +
> + alloc_size = (1 << order) * PAGE_SIZE;
> + base = alloc_xenheap_pages(order, 0);
> + if ( !base )
> + {
> + err = -ENOMEM;
> + goto out_free;
> + }
> + memset(base, 0, alloc_size);
> + its->tables[i] = base;
> +retry_baser:
> + val = (__pa(base) |
> + (type << GITS_BASER_TYPE_SHIFT) |
> + ((entry_size - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
> + GITS_BASER_WaWb |
Same here, there was Linux changes to use non-cacheable where there is
no shareability.
> + shr |
> + GITS_BASER_VALID);
> +
> + switch (psz) {
> + case SZ_4K:
> + val |= GITS_BASER_PAGE_SIZE_4K;
> + break;
> + case SZ_16K:
> + val |= GITS_BASER_PAGE_SIZE_16K;
> + break;
> + case SZ_64K:
> + val |= GITS_BASER_PAGE_SIZE_64K;
> + break;
> + }
> +
> + val |= (((alloc_size / psz) - 1) & 0xffUL);
I don't think the & 0xffUL is useful. If alloc_size is too big it would
result to an invalid val anyway.
FWIW, Linux doesn't have it.
> +
> + writeq_relaxed(val, its->base + GITS_BASER + i * 8);
> + tmp = readq_relaxed(its->base + GITS_BASER + i * 8);
> +
> + if ( (val ^ tmp) & GITS_BASER_SHAREABILITY_MASK )
> + {
> + /*
> + * Shareability didn't stick. Just use
> + * whatever the read reported, which is likely
> + * to be the only thing this redistributor
> + * supports.
> + */
> + shr = tmp & GITS_BASER_SHAREABILITY_MASK;
> + goto retry_baser;
> + }
> +
> + if ( (val ^ tmp) & GITS_BASER_PAGE_SIZE_MASK )
> + {
> + /*
> + * Page size didn't stick. Let's try a smaller
> + * size and retry. If we reach 4K, then
> + * something is horribly wrong...
> + */
> + switch (psz) {
> + case SZ_16K:
> + psz = SZ_4K;
> + goto retry_baser;
> + case SZ_64K:
> + psz = SZ_16K;
> + goto retry_baser;
> + }
> + }
> +
> + /* skip comparin cacheability fields as they are implementation
/*
* Fooo
*/
And s/comparin/comparing/
> + * defined.
> + */
> + val = val << 5;
> + tmp = tmp << 5;
Why do you need a such trick? The whole purpose of the below check is to
verify that the value is correctly taken by the ITS. As the field should
be RW, we would be in big trouble if the cache is not taken into account
by the ITS.
FWIW, Linux doesn't do it.
> + if ( val != tmp )
> + {
> + its_err("ITS: GITS_BASER%d doesn't stick: %lx %lx\n",
> + i, (unsigned long) val, (unsigned long) tmp);
You won't print a correct value here with your "val <<".
[..]
> + /*
> + * Make sure any change to the table is observable by the GIC.
> + */
> + dsb(sy);
> +
> + /* set PROPBASE */
> + val = (__pa(gic_rdists->prop_page) |
> + GICR_PROPBASER_InnerShareable |
> + GICR_PROPBASER_WaWb |
> + ((LPI_NRBITS - 1) & GICR_PROPBASER_IDBITS_MASK));
> +
> + writeq_relaxed(val, rbase + GICR_PROPBASER);
> + tmp = readq_relaxed(rbase + GICR_PROPBASER);
> +
> + if ( (tmp ^ val) & GICR_PROPBASER_SHAREABILITY_MASK )
> + {
If there is no shareability, Linux is dropping the cacheability.
> + its_info("GIC: using cache flushing for LPI property table\n");
> + gic_rdists->flags |= RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING;
> + }
> +
> + /* set PENDBASE */
> + val = (__pa(pend_page) |
> + GICR_PROPBASER_InnerShareable |
> + GICR_PROPBASER_WaWb);
> +
> + writeq_relaxed(val, rbase + GICR_PENDBASER);
> +
Ditto
[..]
> +static void its_cpu_init_collection(void)
> +{
> + struct its_node *its;
> + int cpu;
> +
> + spin_lock(&its_lock);
> + cpu = smp_processor_id();
> +
> + list_for_each_entry(its, &its_nodes, entry)
> + {
> + u64 target;
> + /*
> + * We now have to bind each collection to its target
> + * redistributor.
> + */
> + if ( readq_relaxed(its->base + GITS_TYPER) & GITS_TYPER_PTA )
> + {
> + /*
> + * This ITS wants the physical address of the
> + * redistributor.
> + */
> + target = gic_data_rdist().phys_base;
> + }
> + else
> + {
> + /*
> + * This ITS wants a linear CPU number.
> + */
> + target = readq_relaxed(gic_data_rdist_rd_base() + GICR_TYPER);
> + target = GICR_TYPER_CPU_NUMBER(target);
Why did you drop the << 16?
Although, as said earlier, given the usage of target_address you could
do shift >> directly in this function rather than on multiple__ place.
[..]
> +
> +static int its_probe(struct dt_device_node *node)
> +{
[..]
> + err = its_alloc_collections(its);
> + if ( err )
> + goto out_free_tables;
> +
> + baser = (__pa(its->cmd_base) |
> + GITS_CBASER_WaWb |
> + GITS_CBASER_InnerShareable |
> + (ITS_CMD_QUEUE_SZ / SZ_4K - 1) |
> + GITS_CBASER_VALID);
> +
> + writeq_relaxed(baser, its->base + GITS_CBASER);
> + tmp = readq_relaxed(its->base + GITS_CBASER);
Missing checking the shareability.
> + writeq_relaxed(0, its->base + GITS_CWRITER);
> + writel_relaxed(GITS_CTLR_ENABLE, its->base + GITS_CTLR);
> +
> + if ( (tmp ^ baser) & GITS_BASER_SHAREABILITY_MASK )
> + {
> + its_info("ITS: using cache flushing for cmd queue\n");
> + its->flags |= ITS_FLAGS_CMDQ_NEEDS_FLUSHING;
> + }
Technically you should check that the ITS node contains a property
"msi-controller".
[..]
> +int its_init(struct rdist_prop *rdists)
> +{
> + struct dt_device_node *np = NULL;
> +
> + static const struct dt_device_match its_device_ids[] __initconst =
If the structure is __initconst, the function should be __init too.
> + {
> + DT_MATCH_GIC_ITS,
> + { /* sentinel */ },
> + };
> +
> + for (np = dt_find_matching_node(NULL, its_device_ids); np;
> + np = dt_find_matching_node(np, its_device_ids))
> + its_probe(np);
> +
> + if ( list_empty(&its_nodes) )
> + {
> + its_warn("ITS: No ITS available, not enabling LPIs\n");
> + return -ENXIO;
> + }
> +
> + gic_rdists = rdists;
> + its_alloc_lpi_tables();
> +
> + BUILD_BUG_ON(sizeof(its_cmd_block) != 32);
Why this build bug on here? Shouldn't it be part of the builder code?
[..]
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> new file mode 100644
> index 0000000..4e42f7f
> --- /dev/null
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -0,0 +1,214 @@
> +/*
> + * Copyright (C) 2015 Cavium Inc.
> + * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#ifndef __ASM_ARM_GIC_ITS_H__
> +#define __ASM_ARM_GIC_ITS_H__
> +
> +#include <asm/gic_v3_defs.h>
> +
> +/*
> + * Collection structure - just an ID, and a redistributor address to
> + * ping. We use one per CPU as a bag of interrupts assigned to this
s/bag/collection/
> + * CPU.
> + */
> +struct its_collection {
> + u64 target_address;
> + u16 col_id;
> +};
> +
> +/* ITS command structures */
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:56;
> + u64 res2:64;
> + u64 col:16;
> + u64 ta:32;
> + u64 res3:15;
> + u64 valid:1;
> + u64 res4:64;
> +}mapc_cmd_t;
Missing space
> +
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:24;
> + u64 devid:32;
> + u64 size:5;
> + u64 res2:59;
> + u64 res3:8;
> + u64 itt:40;
> + u64 res4:15;
> + u64 valid:1;
> + u64 res5:64;
> +}mapd_cmd_t;
ditto
> +
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:24;
> + u64 devid:32;
> + u64 event:32;
> + u64 res2:32;
> + u64 col:16;
> + u64 res3:48;
> + u64 res4:64;
> +}mapi_cmd_t;
> +
ditto
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:24;
> + u64 devid:32;
> + u64 event:32;
> + u64 phy_id:32;
> + u64 col:16;
> + u64 res3:48;
> + u64 res4:64;
> +}mapvi_cmd_t;
> +
ditto
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:24;
> + u64 devid:32;
> + u64 event:32;
> + u64 res2:32;
> + u64 col:16;
> + u64 res3:48;
> + u64 res4:64;
> +}movi_cmd_t;
ditto
> +
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:24;
> + u64 devid:32;
> + u64 event:32;
> + u64 res2:32;
> + u64 res3:64;
> + u64 res4:64;
> +}discard_cmd_t;
ditto
> +
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:24;
> + u64 devid:32;
> + u64 event:32;
> + u64 res2:32;
> + u64 res3:64;
> + u64 res4:64;
> +}inv_cmd_t;
ditto
> +
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:56;
> + u64 res2:64;
> + u64 res3:16;
> + u64 ta1:32;
> + u64 res4:16;
> + u64 res5:16;
> + u64 ta2:32;
> + u64 res6:16;
> +}movall_cmd_t;
ditto
> +
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:56;
> + u64 res2:64;
> + u64 col:16;
> + u64 res3:48;
> + u64 res4:64;
> +}invall_cmd_t;
ditto
> +
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:24;
> + u64 devid:32;
> + u64 event:32;
> + u64 res2:32;
> + u64 res3:64;
> + u64 res4:64;
> +}int_cmd_t;
ditto
> +
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:24;
> + u64 devid:32;
> + u64 event:32;
> + u64 res2:32;
> + u64 res3:64;
> + u64 res4:64;
> +}clear_cmd_t;
ditto
> +
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:56;
> + u64 res2:64;
> + u64 res3:16;
> + u64 ta:32;
> + u64 res4:16;
> + u64 res5:64;
> +}sync_cmd_t;
ditto
> +
> +typedef union {
> + u64 raw_cmd[4];
> + mapc_cmd_t mapc;
> + mapd_cmd_t mapd;
> + mapi_cmd_t mapi;
> + mapvi_cmd_t mapvi;
> + movi_cmd_t movi;
> + movall_cmd_t movall;
> + inv_cmd_t inv;
> + invall_cmd_t invall;
> + discard_cmd_t discard;
> + int_cmd_t int_cmd;
> + clear_cmd_t clear;
> + sync_cmd_t sync;
> +}its_cmd_block;
ditto
> +/*
> + * The ITS view of a device - belongs to an ITS, a collection, owns an
> + * interrupt translation table, and a list of interrupts.
> + */
> +struct its_device {
> + /* Physical ITS */
> + struct its_node *its;
> + /* Device ITT address */
> + paddr_t itt_addr;
> + /* Device ITT size */
> + unsigned long itt_size;
> + /* Physical LPI map */
> + unsigned long *lpi_map;
> + /* First Physical LPI number assigned */
> + u32 lpi_base;
> + /* Number of Physical LPIs assigned */
> + int nr_lpis;
> + /* Number of ITES entries */
> + u32 nr_ites;
> + /* Physical Device id */
> + u32 device_id;
> +};
> +
> +static inline uint8_t its_decode_cmd(its_cmd_block *cmd)
> +{
> + return cmd->raw_cmd[0] & 0xff;
> +}
Please defined any exported function of gic-v3-its.c here and not in
patches where they are used.
[..]
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 9e2acb7..e9d5f36 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -161,6 +161,7 @@
> DT_MATCH_COMPATIBLE("arm,gic-400")
>
> #define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
> +#define DT_MATCH_GIC_ITS DT_MATCH_COMPATIBLE("arm,gic-v3-its")
>
> /*
> * GICv3 registers that needs to be saved/restored
> diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
> index acbb906..dc4fe14 100644
> --- a/xen/include/asm-arm/gic_v3_defs.h
> +++ b/xen/include/asm-arm/gic_v3_defs.h
> @@ -45,9 +45,11 @@
> #define GICC_SRE_EL2_DIB (1UL << 2)
> #define GICC_SRE_EL2_ENEL1 (1UL << 3)
>
> +#define GICR_CTL_ENABLE (1U << 0)
Why? You've defined GICR_CTLR_ENABLE_LPIS below and don't use it.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen
2015-06-22 17:16 ` Julien Grall
@ 2015-06-26 9:19 ` Vijay Kilari
2015-06-26 9:52 ` Julien Grall
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-06-26 9:19 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Julien,
On Mon, Jun 22, 2015 at 10:46 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi,
>> + cmd->mapd.cmd = GITS_CMD_MAPD;
>> + cmd->mapd.devid = desc->its_mapd_cmd.dev->device_id;
>> + cmd->mapd.size = size - 1;
>> + cmd->mapd.itt = itt_addr >> 8;
>
> I think the code is more difficult to read without the helpers. You
> opencode every trick in all the builder rather than in a single place.
ITS commands does not consider all the bits of the values esp. when
encoding addresses. So we need to shift and write it.
Helpers might be useful, but usage is hardly once or twice in the code.
>> +static void its_cpu_init_collection(void)
>> +{
>> + struct its_node *its;
>> + int cpu;
>> +
>> + spin_lock(&its_lock);
>> + cpu = smp_processor_id();
>> +
>> + list_for_each_entry(its, &its_nodes, entry)
>> + {
>> + u64 target;
>> + /*
>> + * We now have to bind each collection to its target
>> + * redistributor.
>> + */
>> + if ( readq_relaxed(its->base + GITS_TYPER) & GITS_TYPER_PTA )
>> + {
>> + /*
>> + * This ITS wants the physical address of the
>> + * redistributor.
>> + */
>> + target = gic_data_rdist().phys_base;
>> + }
>> + else
>> + {
>> + /*
>> + * This ITS wants a linear CPU number.
>> + */
>> + target = readq_relaxed(gic_data_rdist_rd_base() + GICR_TYPER);
>> + target = GICR_TYPER_CPU_NUMBER(target);
>
> Why did you drop the << 16?
>
> Although, as said earlier, given the usage of target_address you could
> do shift >> directly in this function rather than on multiple__ place.
Yes, we can shift at the setup. But we lose actual value of target_address.
Also we shift because ITS command skips lower 16 bits
>> +int its_init(struct rdist_prop *rdists)
>> +{
>> + struct dt_device_node *np = NULL;
>> +
>> + static const struct dt_device_match its_device_ids[] __initconst =
>
> If the structure is __initconst, the function should be __init too.
>
>> + {
>> + DT_MATCH_GIC_ITS,
>> + { /* sentinel */ },
>> + };
>> +
>> + for (np = dt_find_matching_node(NULL, its_device_ids); np;
>> + np = dt_find_matching_node(np, its_device_ids))
>> + its_probe(np);
>> +
>> + if ( list_empty(&its_nodes) )
>> + {
>> + its_warn("ITS: No ITS available, not enabling LPIs\n");
>> + return -ENXIO;
>> + }
>> +
>> + gic_rdists = rdists;
>> + its_alloc_lpi_tables();
>> +
>> + BUILD_BUG_ON(sizeof(its_cmd_block) != 32);
>
> Why this build bug on here? Shouldn't it be part of the builder code?
Where is builder code in xen?
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen
2015-06-26 9:19 ` Vijay Kilari
@ 2015-06-26 9:52 ` Julien Grall
0 siblings, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-06-26 9:52 UTC (permalink / raw)
To: Vijay Kilari, Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 26/06/2015 11:19, Vijay Kilari wrote:
> Hi Julien,
Hi Vijay,
> On Mon, Jun 22, 2015 at 10:46 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi,
>
>>> + cmd->mapd.cmd = GITS_CMD_MAPD;
>>> + cmd->mapd.devid = desc->its_mapd_cmd.dev->device_id;
>>> + cmd->mapd.size = size - 1;
>>> + cmd->mapd.itt = itt_addr >> 8;
>>
>> I think the code is more difficult to read without the helpers. You
>> opencode every trick in all the builder rather than in a single place.
>
> ITS commands does not consider all the bits of the values esp. when
> encoding addresses. So we need to shift and write it.
A comment on the code would have been welcomed (either here or in the
definition of the structure). By naming the field itt it gives the
impression that we have to pass all the ITT address.
> Helpers might be useful, but usage is hardly once or twice in the code.
IHMO, once is okay but twice is too much. It means that you duplicated
code that may be non straigh-forward to understand (such as shift).
Anyway, I will let Ian & Stefano decide.
>
>>> +static void its_cpu_init_collection(void)
>>> +{
>>> + struct its_node *its;
>>> + int cpu;
>>> +
>>> + spin_lock(&its_lock);
>>> + cpu = smp_processor_id();
>>> +
>>> + list_for_each_entry(its, &its_nodes, entry)
>>> + {
>>> + u64 target;
>>> + /*
>>> + * We now have to bind each collection to its target
>>> + * redistributor.
>>> + */
>>> + if ( readq_relaxed(its->base + GITS_TYPER) & GITS_TYPER_PTA )
>>> + {
>>> + /*
>>> + * This ITS wants the physical address of the
>>> + * redistributor.
>>> + */
>>> + target = gic_data_rdist().phys_base;
>>> + }
>>> + else
>>> + {
>>> + /*
>>> + * This ITS wants a linear CPU number.
>>> + */
>>> + target = readq_relaxed(gic_data_rdist_rd_base() + GICR_TYPER);
>>> + target = GICR_TYPER_CPU_NUMBER(target);
>>
>> Why did you drop the << 16?
>>
>> Although, as said earlier, given the usage of target_address you could
>> do shift >> directly in this function rather than on multiple__ place.
>
> Yes, we can shift at the setup. But we lose actual value of target_address.
Do we really need to get the target_address intact? I believe not.
>>> + gic_rdists = rdists;
>>> + its_alloc_lpi_tables();
>>> +
>>> + BUILD_BUG_ON(sizeof(its_cmd_block) != 32);
>>
>> Why this build bug on here? Shouldn't it be part of the builder code?
>
> Where is builder code in xen?
By builder I meant the function who built the command i.e
its_send_single_command.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen
2015-06-22 12:01 ` [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen vijay.kilari
2015-06-22 17:16 ` Julien Grall
@ 2015-06-29 11:39 ` Ian Campbell
2015-06-29 15:43 ` Vijay Kilari
1 sibling, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 11:39 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
[...]
> +/*
> + * ITS command descriptors - parameters to be encoded in a command
> + * block.
> + */
> +struct its_cmd_desc {
> + union {
> + struct {
> + struct its_collection *col;
> + u32 event_id;
> + u32 dev_id;
> + } its_inv_cmd;
[...]
> +static struct its_collection *its_build_inv_cmd(its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + memset(cmd, 0x0, sizeof(its_cmd_block));
> + cmd->inv.cmd = GITS_CMD_INV;
> + cmd->inv.devid = desc->its_inv_cmd.dev_id;
> + cmd->inv.event = desc->its_inv_cmd.event_id;
> +
> +#ifdef DEBUG_GIC_ITS
> + dump_cmd(cmd);
> +#endif
> +
> + return desc->its_inv_cmd.col;
> +}
[...]
> +void its_send_inv(struct its_device *dev, struct its_collection *col,
> + u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_inv_cmd.dev_id = dev->device_id;
> + desc.its_inv_cmd.event_id = event_id;
> + desc.its_inv_cmd.col = col;
> +
> + its_send_single_command(dev->its, its_build_inv_cmd, &desc);
> +}
[...]
> +typedef struct __packed {
> + u64 cmd:8;
> + u64 res1:24;
> + u64 devid:32;
> + u64 event:32;
> + u64 res2:32;
> + u64 res3:64;
> + u64 res4:64;
> +}inv_cmd_t;
(I've trimmed this to just the INV command, but it's the same for all of
them)
I suppose this is a mix of the way the Linux code was structured and my
request to use a struct/union to encode these things, but I'm afraid
this is not how I intended to suggest things be done.
What I expected was something analogous to the hsr or lpae_t types, e.g.
a single:
union its_cmd {
uint64_t bits[N];
struct {
uint8_t cmd;
uint8_t pad[...];
} hdr;
struct {
uint8_t cmd;
uint8_t res1[3];
uint32_t devid;
uint32_t event;
uint64_t res2[2];
} inv;
};
So its_send_single_command can take a "union its_cmd *" and its_send_inv
should look like:
void its_send_inv(struct its_device *dev, struct its_collection *col,
u32 event_id)
{
union its_cmd cmd;
/* memset perhaps, or sets .bits = {0,} */
cmd.inv.cmd = GITS_CMD_INV;
cmd.inv.dev_id = dev->device_id;
cmd.inv.event_id = event_id;
cmd.inv.col = col;
return its_send_single_command(dev->its, &cmd);
}
I've omitted the its_ prefix and _cmd/_desc suffix where they aren't
needed in the context they are used. (so not cmd.cmd_inv etc).
I've also used proper types where possible instead of bitfields of u64
(although unsigned long bitfields should still be used for sub 8-bit
fields).
The "hdr" member of the union should contain any field which is global
to all commands and which generic code (i.e. which isn't aware which
command is in the cmd in its hand) can use. Off the top of my head that
is just the cmd code itself.
You should also consider doing the collection sync in the caller as
appropriate instead of pushing it down into its_send_single_command.
IMHO its_send_single_command should do just that, not optionally do some
other command too.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen
2015-06-29 11:39 ` Ian Campbell
@ 2015-06-29 15:43 ` Vijay Kilari
2015-06-29 15:47 ` Vijay Kilari
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-06-29 15:43 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Mon, Jun 29, 2015 at 5:09 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> [...]
>> +/*
>> + * ITS command descriptors - parameters to be encoded in a command
>> + * block.
>> + */
>> +struct its_cmd_desc {
>> + union {
>> + struct {
>> + struct its_collection *col;
>> + u32 event_id;
>> + u32 dev_id;
>> + } its_inv_cmd;
> [...]
>> +static struct its_collection *its_build_inv_cmd(its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + memset(cmd, 0x0, sizeof(its_cmd_block));
>> + cmd->inv.cmd = GITS_CMD_INV;
>> + cmd->inv.devid = desc->its_inv_cmd.dev_id;
>> + cmd->inv.event = desc->its_inv_cmd.event_id;
>> +
>> +#ifdef DEBUG_GIC_ITS
>> + dump_cmd(cmd);
>> +#endif
>> +
>> + return desc->its_inv_cmd.col;
>> +}
> [...]
>> +void its_send_inv(struct its_device *dev, struct its_collection *col,
>> + u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_inv_cmd.dev_id = dev->device_id;
>> + desc.its_inv_cmd.event_id = event_id;
>> + desc.its_inv_cmd.col = col;
>> +
>> + its_send_single_command(dev->its, its_build_inv_cmd, &desc);
>> +}
> [...]
>> +typedef struct __packed {
>> + u64 cmd:8;
>> + u64 res1:24;
>> + u64 devid:32;
>> + u64 event:32;
>> + u64 res2:32;
>> + u64 res3:64;
>> + u64 res4:64;
>> +}inv_cmd_t;
>
> (I've trimmed this to just the INV command, but it's the same for all of
> them)
>
> I suppose this is a mix of the way the Linux code was structured and my
> request to use a struct/union to encode these things, but I'm afraid
> this is not how I intended to suggest things be done.
>
> What I expected was something analogous to the hsr or lpae_t types, e.g.
> a single:
> union its_cmd {
> uint64_t bits[N];
>
> struct {
> uint8_t cmd;
> uint8_t pad[...];
> } hdr;
>
> struct {
> uint8_t cmd;
> uint8_t res1[3];
> uint32_t devid;
> uint32_t event;
> uint64_t res2[2];
> } inv;
> };
Commands like MAPD
>
> So its_send_single_command can take a "union its_cmd *" and its_send_inv
> should look like:
>
> void its_send_inv(struct its_device *dev, struct its_collection *col,
> u32 event_id)
> {
> union its_cmd cmd;
> /* memset perhaps, or sets .bits = {0,} */
>
> cmd.inv.cmd = GITS_CMD_INV;
> cmd.inv.dev_id = dev->device_id;
> cmd.inv.event_id = event_id;
> cmd.inv.col = col;
>
> return its_send_single_command(dev->its, &cmd);
> }
>
> I've omitted the its_ prefix and _cmd/_desc suffix where they aren't
> needed in the context they are used. (so not cmd.cmd_inv etc).
>
> I've also used proper types where possible instead of bitfields of u64
> (although unsigned long bitfields should still be used for sub 8-bit
> fields).
>
> The "hdr" member of the union should contain any field which is global
> to all commands and which generic code (i.e. which isn't aware which
> command is in the cmd in its hand) can use. Off the top of my head that
> is just the cmd code itself.
>
> You should also consider doing the collection sync in the caller as
> appropriate instead of pushing it down into its_send_single_command.
> IMHO its_send_single_command should do just that, not optionally do some
> other command too.
>
> Ian.
>
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen
2015-06-29 15:43 ` Vijay Kilari
@ 2015-06-29 15:47 ` Vijay Kilari
2015-06-29 16:49 ` Ian Campbell
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-06-29 15:47 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Mon, Jun 29, 2015 at 9:13 PM, Vijay Kilari <vijay.kilari@gmail.com> wrote:
> On Mon, Jun 29, 2015 at 5:09 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
>> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>> [...]
>>> +/*
>>> + * ITS command descriptors - parameters to be encoded in a command
>>> + * block.
>>> + */
>>> +struct its_cmd_desc {
>>> + union {
>>> + struct {
>>> + struct its_collection *col;
>>> + u32 event_id;
>>> + u32 dev_id;
>>> + } its_inv_cmd;
>> [...]
>>> +static struct its_collection *its_build_inv_cmd(its_cmd_block *cmd,
>>> + struct its_cmd_desc *desc)
>>> +{
>>> + memset(cmd, 0x0, sizeof(its_cmd_block));
>>> + cmd->inv.cmd = GITS_CMD_INV;
>>> + cmd->inv.devid = desc->its_inv_cmd.dev_id;
>>> + cmd->inv.event = desc->its_inv_cmd.event_id;
>>> +
>>> +#ifdef DEBUG_GIC_ITS
>>> + dump_cmd(cmd);
>>> +#endif
>>> +
>>> + return desc->its_inv_cmd.col;
>>> +}
>> [...]
>>> +void its_send_inv(struct its_device *dev, struct its_collection *col,
>>> + u32 event_id)
>>> +{
>>> + struct its_cmd_desc desc;
>>> +
>>> + desc.its_inv_cmd.dev_id = dev->device_id;
>>> + desc.its_inv_cmd.event_id = event_id;
>>> + desc.its_inv_cmd.col = col;
>>> +
>>> + its_send_single_command(dev->its, its_build_inv_cmd, &desc);
>>> +}
>> [...]
>>> +typedef struct __packed {
>>> + u64 cmd:8;
>>> + u64 res1:24;
>>> + u64 devid:32;
>>> + u64 event:32;
>>> + u64 res2:32;
>>> + u64 res3:64;
>>> + u64 res4:64;
>>> +}inv_cmd_t;
>>
>> (I've trimmed this to just the INV command, but it's the same for all of
>> them)
>>
>> I suppose this is a mix of the way the Linux code was structured and my
>> request to use a struct/union to encode these things, but I'm afraid
>> this is not how I intended to suggest things be done.
>>
>> What I expected was something analogous to the hsr or lpae_t types, e.g.
>> a single:
>> union its_cmd {
>> uint64_t bits[N];
>>
>> struct {
>> uint8_t cmd;
>> uint8_t pad[...];
>> } hdr;
>>
>> struct {
>> uint8_t cmd;
>> uint8_t res1[3];
>> uint32_t devid;
>> uint32_t event;
>> uint64_t res2[2];
>> } inv;
>> };
>
Commands like MAPD has 40 bit ITT, 5 bit Size and 1 bit valid bit. So we have
to still manage with bit fields to manage with ease. So we have to use
mix of bit fields
and normal types (uint8_t & uint32_t etc.,)
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen
2015-06-29 15:47 ` Vijay Kilari
@ 2015-06-29 16:49 ` Ian Campbell
0 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 16:49 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Mon, 2015-06-29 at 21:17 +0530, Vijay Kilari wrote:
> On Mon, Jun 29, 2015 at 9:13 PM, Vijay Kilari <vijay.kilari@gmail.com> wrote:
> > On Mon, Jun 29, 2015 at 5:09 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> >> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> >> [...]
> >>> +/*
> >>> + * ITS command descriptors - parameters to be encoded in a command
> >>> + * block.
> >>> + */
> >>> +struct its_cmd_desc {
> >>> + union {
> >>> + struct {
> >>> + struct its_collection *col;
> >>> + u32 event_id;
> >>> + u32 dev_id;
> >>> + } its_inv_cmd;
> >> [...]
> >>> +static struct its_collection *its_build_inv_cmd(its_cmd_block *cmd,
> >>> + struct its_cmd_desc *desc)
> >>> +{
> >>> + memset(cmd, 0x0, sizeof(its_cmd_block));
> >>> + cmd->inv.cmd = GITS_CMD_INV;
> >>> + cmd->inv.devid = desc->its_inv_cmd.dev_id;
> >>> + cmd->inv.event = desc->its_inv_cmd.event_id;
> >>> +
> >>> +#ifdef DEBUG_GIC_ITS
> >>> + dump_cmd(cmd);
> >>> +#endif
> >>> +
> >>> + return desc->its_inv_cmd.col;
> >>> +}
> >> [...]
> >>> +void its_send_inv(struct its_device *dev, struct its_collection *col,
> >>> + u32 event_id)
> >>> +{
> >>> + struct its_cmd_desc desc;
> >>> +
> >>> + desc.its_inv_cmd.dev_id = dev->device_id;
> >>> + desc.its_inv_cmd.event_id = event_id;
> >>> + desc.its_inv_cmd.col = col;
> >>> +
> >>> + its_send_single_command(dev->its, its_build_inv_cmd, &desc);
> >>> +}
> >> [...]
> >>> +typedef struct __packed {
> >>> + u64 cmd:8;
> >>> + u64 res1:24;
> >>> + u64 devid:32;
> >>> + u64 event:32;
> >>> + u64 res2:32;
> >>> + u64 res3:64;
> >>> + u64 res4:64;
> >>> +}inv_cmd_t;
> >>
> >> (I've trimmed this to just the INV command, but it's the same for all of
> >> them)
> >>
> >> I suppose this is a mix of the way the Linux code was structured and my
> >> request to use a struct/union to encode these things, but I'm afraid
> >> this is not how I intended to suggest things be done.
> >>
> >> What I expected was something analogous to the hsr or lpae_t types, e.g.
> >> a single:
> >> union its_cmd {
> >> uint64_t bits[N];
> >>
> >> struct {
> >> uint8_t cmd;
> >> uint8_t pad[...];
> >> } hdr;
> >>
> >> struct {
> >> uint8_t cmd;
> >> uint8_t res1[3];
> >> uint32_t devid;
> >> uint32_t event;
> >> uint64_t res2[2];
> >> } inv;
> >> };
> >
>
> Commands like MAPD has 40 bit ITT, 5 bit Size and 1 bit valid bit. So we have
> to still manage with bit fields to manage with ease. So we have to use
> mix of bit fields
> and normal types (uint8_t & uint32_t etc.,)
That's fine. Just use normal types for fields are sized that way and
bitfields for the rest. e.g. for mapd:
struct {
uint8_t cmd;
uint8_t res1[3];
uint32_t devid;
uint64_t size:5;
uint64_t res2:59;
...
};
Looking at ITT_addr you might consider including the res0 bits in bits
0..7 of that double word in the itt_addr, allowing you to avoid a shift
on use of that field (this assumes that the reason for those res0 bits
is the alignment constraints on itt_addr).
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 06/18] xen/arm: ITS: Add helper functions to manage its_devices
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (4 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 05/18] xen/arm: ITS: Port ITS driver to xen vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-23 10:21 ` Julien Grall
2015-06-22 12:01 ` [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs vijay.kilari
` (14 subsequent siblings)
20 siblings, 1 reply; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Helper functions to mange its devices using RB-tree
are introduced in physical ITS driver.
This is global list of all the devices.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3-its.c | 49 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 3 +++
2 files changed, 52 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index b1a97c1..349d0bb 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -92,6 +92,7 @@ struct its_node {
static LIST_HEAD(its_nodes);
static DEFINE_SPINLOCK(its_lock);
static struct rdist_prop *gic_rdists;
+static struct rb_root rb_its_dev;
#define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
#define gic_data_rdist_rd_base() (per_cpu(rdist, smp_processor_id()).rbase)
@@ -145,6 +146,53 @@ void dump_cmd(its_cmd_block *cmd)
}
#endif
+/* RB-tree helpers for its_device */
+struct its_device * find_its_device(struct rb_root *root, u32 devid)
+{
+ struct rb_node *node = root->rb_node;
+
+ while ( node )
+ {
+ struct its_device *dev;
+
+ dev = container_of(node, struct its_device, node);
+ if ( devid < dev->device_id )
+ node = node->rb_left;
+ else if ( devid > dev->device_id )
+ node = node->rb_right;
+ else
+ return dev;
+ }
+
+ return NULL;
+}
+
+int insert_its_device(struct rb_root *root, struct its_device *dev)
+{
+ struct rb_node **new, *parent;
+
+ new = &root->rb_node;
+ parent = NULL;
+ while ( *new )
+ {
+ struct its_device *this;
+
+ this = container_of(*new, struct its_device, node);
+ parent = *new;
+ if ( dev->device_id < this->device_id )
+ new = &((*new)->rb_left);
+ else if ( dev->device_id > this->device_id )
+ new = &((*new)->rb_right);
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node(&dev->node, parent, new);
+ rb_insert_color(&dev->node, root);
+
+ return 0;
+}
+
#define ITS_CMD_QUEUE_SZ SZ_64K
#define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(its_cmd_block))
@@ -994,6 +1042,7 @@ static int its_probe(struct dt_device_node *node)
list_add(&its->entry, &its_nodes);
spin_unlock(&its_lock);
+ rb_its_dev = RB_ROOT;
return 0;
out_free_tables:
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 4e42f7f..59a6490 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -19,6 +19,7 @@
#define __ASM_ARM_GIC_ITS_H__
#include <asm/gic_v3_defs.h>
+#include <xen/rbtree.h>
/*
* Collection structure - just an ID, and a redistributor address to
@@ -195,6 +196,8 @@ struct its_device {
u32 nr_ites;
/* Physical Device id */
u32 device_id;
+ /* RB-tree entry */
+ struct rb_node node;
};
static inline uint8_t its_decode_cmd(its_cmd_block *cmd)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 06/18] xen/arm: ITS: Add helper functions to manage its_devices
2015-06-22 12:01 ` [RFC PATCH v3 06/18] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
@ 2015-06-23 10:21 ` Julien Grall
0 siblings, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-06-23 10:21 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 22/06/2015 13:01, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Helper functions to mange its devices using RB-tree
s/mange/manage/
> are introduced in physical ITS driver.
>
> This is global list of all the devices.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/gic-v3-its.c | 49 +++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic-its.h | 3 +++
> 2 files changed, 52 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index b1a97c1..349d0bb 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -92,6 +92,7 @@ struct its_node {
> static LIST_HEAD(its_nodes);
> static DEFINE_SPINLOCK(its_lock);
> static struct rdist_prop *gic_rdists;
> +static struct rb_root rb_its_dev;
>
> #define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
> #define gic_data_rdist_rd_base() (per_cpu(rdist, smp_processor_id()).rbase)
> @@ -145,6 +146,53 @@ void dump_cmd(its_cmd_block *cmd)
> }
> #endif
>
> +/* RB-tree helpers for its_device */
> +struct its_device * find_its_device(struct rb_root *root, u32 devid)
coding style: struct its_device *find
I would rename to its_find_device to keep consistent with the function
name within this file.
Also, for any exported function you have to declare the prototype in the
header within the same patch.
> +{
> + struct rb_node *node = root->rb_node;
> +
> + while ( node )
> + {
> + struct its_device *dev;
> +
> + dev = container_of(node, struct its_device, node);
> + if ( devid < dev->device_id )
> + node = node->rb_left;
> + else if ( devid > dev->device_id )
> + node = node->rb_right;
> + else
> + return dev;
> + }
> +
> + return NULL;
> +}
> +
> +int insert_its_device(struct rb_root *root, struct its_device *dev)
Why do you need the root in parameter? You already it within the file.
Also, I would rename to its_add_device.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (5 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 06/18] xen/arm: ITS: Add helper functions to manage its_devices vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-23 14:32 ` Julien Grall
2015-06-22 12:01 ` [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver vijay.kilari
` (13 subsequent siblings)
20 siblings, 1 reply; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Implements hw_irq_controller api's required
to handle LPI's
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3-its.c | 39 +++++++++++++++++++++++++++++++++++++++
xen/arch/arm/gic-v3.c | 26 +++++++++++++++++++-------
xen/arch/arm/irq.c | 16 ++++++++++++++++
xen/include/asm-arm/gic-its.h | 10 ++++++++++
xen/include/asm-arm/gic.h | 4 ++++
xen/include/asm-arm/irq.h | 4 +++-
6 files changed, 91 insertions(+), 8 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 349d0bb..535fc53 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -508,6 +508,45 @@ static void its_send_invall(struct its_node *its, struct its_collection *col)
its_send_single_command(its, its_build_invall_cmd, &desc);
}
+void lpi_set_config(struct irq_desc *desc, int enable)
+{
+ struct its_collection *col;
+ struct its_device *its_dev = get_irq_device(desc);
+ u8 *cfg;
+ u32 virq = irq_to_virq(desc);
+
+ ASSERT(virq < its_dev->nr_lpis);
+
+ cfg = gic_rdists->prop_page + desc->irq - NR_GIC_LPI;
+ if ( enable )
+ *cfg |= LPI_PROP_ENABLED;
+ else
+ *cfg &= ~LPI_PROP_ENABLED;
+
+ /*
+ * Make the above write visible to the redistributors.
+ * And yes, we're flushing exactly: One. Single. Byte.
+ * Humpf...
+ */
+ if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
+ clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
+ else
+ dsb(ishst);
+
+ /* Get collection id for this event id */
+ col = &its_dev->its->collections[virq % num_online_cpus()];
+ its_send_inv(its_dev, col, virq);
+}
+
+void its_set_affinity(struct irq_desc *desc, int cpu)
+{
+ struct its_device *its_dev = get_irq_device(desc);
+ struct its_collection *target_col;
+
+ /* Physical collection id */
+ target_col = &its_dev->its->collections[cpu];
+ its_send_movi(its_dev, target_col, irq_to_virq(desc));
+}
/*
* How we allocate LPIs:
*
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index b5c59f6..556b291 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -40,6 +40,7 @@
#include <asm/device.h>
#include <asm/gic.h>
#include <asm/gic_v3_defs.h>
+#include <asm/gic-its.h>
#include <asm/cpufeature.h>
struct rdist_region {
@@ -427,12 +428,18 @@ static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset)
static void gicv3_unmask_irq(struct irq_desc *irqd)
{
- gicv3_poke_irq(irqd, GICD_ISENABLER);
+ if ( is_lpi(irqd->irq) )
+ lpi_set_config(irqd, 1);
+ else
+ gicv3_poke_irq(irqd, GICD_ISENABLER);
}
static void gicv3_mask_irq(struct irq_desc *irqd)
{
- gicv3_poke_irq(irqd, GICD_ICENABLER);
+ if ( is_lpi(irqd->irq) )
+ lpi_set_config(irqd, 0);
+ else
+ gicv3_poke_irq(irqd, GICD_ICENABLER);
}
static void gicv3_eoi_irq(struct irq_desc *irqd)
@@ -1095,13 +1102,18 @@ static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
spin_lock(&gicv3.lock);
cpu = gicv3_get_cpu_from_mask(mask);
- affinity = gicv3_mpidr_to_affinity(cpu);
- /* Make sure we don't broadcast the interrupt */
- affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
- if ( desc->irq >= NR_GIC_LOCAL_IRQS )
- writeq_relaxed(affinity, (GICD + GICD_IROUTER + desc->irq * 8));
+ if ( is_lpi(desc->irq) )
+ its_set_affinity(desc, cpu);
+ else
+ {
+ affinity = gicv3_mpidr_to_affinity(cpu);
+ /* Make sure we don't broadcast the interrupt */
+ affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
+ if ( desc->irq >= NR_GIC_LOCAL_IRQS )
+ writeq_relaxed(affinity, (GICD + GICD_IROUTER + desc->irq * 8));
+ }
spin_unlock(&gicv3.lock);
}
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 2dd43ee..9dbdf7d 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -36,6 +36,7 @@ struct irq_guest
{
struct domain *d;
unsigned int virq;
+ struct its_device *dev;
};
static void ack_none(struct irq_desc *irq)
@@ -143,6 +144,21 @@ static inline struct domain *irq_get_domain(struct irq_desc *desc)
return irq_get_guest_info(desc)->d;
}
+unsigned int irq_to_virq(struct irq_desc *desc)
+{
+ return irq_get_guest_info(desc)->virq;
+}
+
+struct its_device *get_irq_device(struct irq_desc *desc)
+{
+ return irq_get_guest_info(desc)->dev;
+}
+
+void set_irq_device(struct irq_desc *desc, struct its_device *dev)
+{
+ irq_get_guest_info(desc)->dev = dev;
+}
+
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
{
if ( desc != NULL )
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 59a6490..a47cf26 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -205,6 +205,16 @@ static inline uint8_t its_decode_cmd(its_cmd_block *cmd)
return cmd->raw_cmd[0] & 0xff;
}
+static inline uint32_t its_decode_devid(struct domain *d, its_cmd_block *cmd)
+{
+ /* TODO: Use pci helper function to get physical id */
+ return (cmd->raw_cmd[0] >> 32);
+}
+
+void its_set_affinity(struct irq_desc *desc, int cpu);
+void lpi_set_config(struct irq_desc *desc, int enable);
+uint32_t its_get_nr_events(void);
+
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index e9d5f36..0209cc5 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -20,6 +20,9 @@
#define NR_GIC_LOCAL_IRQS NR_LOCAL_IRQS
#define NR_GIC_SGI 16
+#define NR_GIC_LPI 8192
+#define MAX_LPI (8192 + 4096)
+#define MAX_NR_LPIS 4096
#define MAX_RDIST_COUNT 4
#define GICD_CTLR (0x000)
@@ -163,6 +166,7 @@
#define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
#define DT_MATCH_GIC_ITS DT_MATCH_COMPATIBLE("arm,gic-v3-its")
+#define is_lpi(lpi) (lpi >= NR_GIC_LPI && lpi < MAX_LPI)
/*
* GICv3 registers that needs to be saved/restored
*/
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index 34b492b..3b29182 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -50,7 +50,9 @@ void arch_move_irqs(struct vcpu *v);
/* Set IRQ type for an SPI */
int irq_set_spi_type(unsigned int spi, unsigned int type);
-
+unsigned int irq_to_virq(struct irq_desc *desc);
+struct its_device *get_irq_device(struct irq_desc *desc);
+void set_irq_device(struct irq_desc *desc, struct its_device *dev);
int platform_get_irq(const struct dt_device_node *device, int index);
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-22 12:01 ` [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs vijay.kilari
@ 2015-06-23 14:32 ` Julien Grall
2015-06-26 12:54 ` Vijay Kilari
` (2 more replies)
0 siblings, 3 replies; 88+ messages in thread
From: Julien Grall @ 2015-06-23 14:32 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Implements hw_irq_controller api's required
> to handle LPI's
This patch doesn't hw_irq_controller for LPI but just hack around the
current GICv3 host hw_irq_controller.
As said on the previous version, the goal of hw_irq_controller is too
keep things simple (i.e few conditional code). Please introduce a
separate hw_irq_controller for LPIs.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/gic-v3-its.c | 39 +++++++++++++++++++++++++++++++++++++++
> xen/arch/arm/gic-v3.c | 26 +++++++++++++++++++-------
> xen/arch/arm/irq.c | 16 ++++++++++++++++
> xen/include/asm-arm/gic-its.h | 10 ++++++++++
> xen/include/asm-arm/gic.h | 4 ++++
> xen/include/asm-arm/irq.h | 4 +++-
> 6 files changed, 91 insertions(+), 8 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 349d0bb..535fc53 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -508,6 +508,45 @@ static void its_send_invall(struct its_node *its, struct its_collection *col)
> its_send_single_command(its, its_build_invall_cmd, &desc);
> }
>
> +void lpi_set_config(struct irq_desc *desc, int enable)
Any function exported should have their prototype defined within the
same patch...
> +{
> + struct its_collection *col;
> + struct its_device *its_dev = get_irq_device(desc);
> + u8 *cfg;
> + u32 virq = irq_to_virq(desc);
> +
> + ASSERT(virq < its_dev->nr_lpis);
> +
> + cfg = gic_rdists->prop_page + desc->irq - NR_GIC_LPI;
> + if ( enable )
> + *cfg |= LPI_PROP_ENABLED;
> + else
> + *cfg &= ~LPI_PROP_ENABLED;
> +
> + /*
> + * Make the above write visible to the redistributors.
> + * And yes, we're flushing exactly: One. Single. Byte.
> + * Humpf...
> + */
> + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
> + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
> + else
> + dsb(ishst);
> +
> + /* Get collection id for this event id */
> + col = &its_dev->its->collections[virq % num_online_cpus()];
This is fragile, you are assuming that num_online_cpus() will never
change. Why don't you store the collection in every irq_desc?
> + its_send_inv(its_dev, col, virq);
> +}
> +
> +void its_set_affinity(struct irq_desc *desc, int cpu)
> +{
> + struct its_device *its_dev = get_irq_device(desc);
> + struct its_collection *target_col;
> +
> + /* Physical collection id */
> + target_col = &its_dev->its->collections[cpu];
> + its_send_movi(its_dev, target_col, irq_to_virq(desc));
The field "virq" in the structure irq_guest refers to the guest virtual
IRQ and not the event ID. As Ian suggested in the proposal [1], please
use an union to make this code clears.
Furthermore, when you set the LPI configuration (see lpi_set_config) you
are using a round robin to get the collection. This won't work anymore
if Xen decides to change the affinity... So you may want to drop
affinity support for now.
> +}
Missing newline.
[..]
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 2dd43ee..9dbdf7d 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -36,6 +36,7 @@ struct irq_guest
> {
> struct domain *d;
> unsigned int virq;
> + struct its_device *dev;
I know that this was suggested in the proposal [1]. But the goal of
irq_guest is to store anything specific to the guest. The event ID and
the its_device assigned are known when the device is added to Xen and
hence can be set in irq_desc (with a small memory impact, but we have
plenty of memory on ARM64).
This would avoid you to set dev and virq every time the device is
passthrough to a guest.
> };
>
> static void ack_none(struct irq_desc *irq)
> @@ -143,6 +144,21 @@ static inline struct domain *irq_get_domain(struct irq_desc *desc)
> return irq_get_guest_info(desc)->d;
> }
>
> +unsigned int irq_to_virq(struct irq_desc *desc)
> +{
> + return irq_get_guest_info(desc)->virq;
> +}
> +
> +struct its_device *get_irq_device(struct irq_desc *desc)
> +{
> + return irq_get_guest_info(desc)->dev;
> +}
> +
> +void set_irq_device(struct irq_desc *desc, struct its_device *dev)
> +{
> + irq_get_guest_info(desc)->dev = dev;
> +}
The goal of route_irq_guest is to setup correctly irq_guest. If the
current version doesn't fit your usage please update it or add a new
helper and no workaround the code.
> +
> void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
> {
> if ( desc != NULL )
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 59a6490..a47cf26 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -205,6 +205,16 @@ static inline uint8_t its_decode_cmd(its_cmd_block *cmd)
> return cmd->raw_cmd[0] & 0xff;
> }
>
> +static inline uint32_t its_decode_devid(struct domain *d, its_cmd_block *cmd)
> +{
> + /* TODO: Use pci helper function to get physical id */
> + return (cmd->raw_cmd[0] >> 32);
> +}
This doesn't belong to this patch. And without more comment it makes
little sense why you are using cmd.
> +
> +void its_set_affinity(struct irq_desc *desc, int cpu);
> +void lpi_set_config(struct irq_desc *desc, int enable);
> +uint32_t its_get_nr_events(void);
> +
> #endif /* __ASM_ARM_GIC_ITS_H__ */
>
> /*
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index e9d5f36..0209cc5 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -20,6 +20,9 @@
>
> #define NR_GIC_LOCAL_IRQS NR_LOCAL_IRQS
> #define NR_GIC_SGI 16
> +#define NR_GIC_LPI 8192
> +#define MAX_LPI (8192 + 4096)
> +#define MAX_NR_LPIS 4096
For me NR_GIC_LPI, MAX_LPI and MAX_NR_LPIS means the exactly the same
things but you are using 3 different value.
Please name the correctly:
- NR_GIC_LPI => FIRST_GIC_LPI
- MAX_NR_LPIS => NR_GIC_LPI
- Drop MAX_LPI
Although, why do you hardcode the number of LPIs? This should be
retrieved from the GIC hardware configuration.
> #define MAX_RDIST_COUNT 4
>
> #define GICD_CTLR (0x000)
> @@ -163,6 +166,7 @@
> #define DT_MATCH_GIC_V3 DT_MATCH_COMPATIBLE("arm,gic-v3")
> #define DT_MATCH_GIC_ITS DT_MATCH_COMPATIBLE("arm,gic-v3-its")
>
> +#define is_lpi(lpi) (lpi >= NR_GIC_LPI && lpi < MAX_LPI)
Missing newline.
Regards,
[1]
http://xenbits.xensource.com/people/ianc/vits/draftF.html#virtual-lpi-interrupt-injection
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-23 14:32 ` Julien Grall
@ 2015-06-26 12:54 ` Vijay Kilari
2015-06-26 15:05 ` Julien Grall
2015-06-26 14:25 ` Vijay Kilari
2015-06-29 11:59 ` Ian Campbell
2 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-06-26 12:54 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Tue, Jun 23, 2015 at 8:02 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Implements hw_irq_controller api's required
>> to handle LPI's
>
> This patch doesn't hw_irq_controller for LPI but just hack around the
> current GICv3 host hw_irq_controller.
>
> As said on the previous version, the goal of hw_irq_controller is too
> keep things simple (i.e few conditional code). Please introduce a
> separate hw_irq_controller for LPIs.
If new hw_irq_controller is introduced for LPIs, then this has to
be exported using some lpi structure which holds pointer to hw_irq_controller
for guest & host type similar to gic_hw_ops
>> +{
>> + struct its_collection *col;
>> + struct its_device *its_dev = get_irq_device(desc);
>> + u8 *cfg;
>> + u32 virq = irq_to_virq(desc);
>> +
>> + ASSERT(virq < its_dev->nr_lpis);
>> +
>> + cfg = gic_rdists->prop_page + desc->irq - NR_GIC_LPI;
>> + if ( enable )
>> + *cfg |= LPI_PROP_ENABLED;
>> + else
>> + *cfg &= ~LPI_PROP_ENABLED;
>> +
>> + /*
>> + * Make the above write visible to the redistributors.
>> + * And yes, we're flushing exactly: One. Single. Byte.
>> + * Humpf...
>> + */
>> + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
>> + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
>> + else
>> + dsb(ishst);
>> +
>> + /* Get collection id for this event id */
>> + col = &its_dev->its->collections[virq % num_online_cpus()];
>
> This is fragile, you are assuming that num_online_cpus() will never
> change. Why don't you store the collection in every irq_desc?
This will add additional 8 bytes overhead for each irq_desc.
Also is there a macro to get number of actual number processors in system.?
AUI, nr_cpu_ids always returns 128
Regards
Vijay
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-26 12:54 ` Vijay Kilari
@ 2015-06-26 15:05 ` Julien Grall
2015-06-29 11:53 ` Ian Campbell
2015-07-02 12:15 ` Vijay Kilari
0 siblings, 2 replies; 88+ messages in thread
From: Julien Grall @ 2015-06-26 15:05 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Vijay,
On 26/06/2015 14:54, Vijay Kilari wrote:
> On Tue, Jun 23, 2015 at 8:02 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi Vijay,
>>
>> On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> Implements hw_irq_controller api's required
>>> to handle LPI's
>>
>> This patch doesn't hw_irq_controller for LPI but just hack around the
>> current GICv3 host hw_irq_controller.
>>
>> As said on the previous version, the goal of hw_irq_controller is too
>> keep things simple (i.e few conditional code). Please introduce a
>> separate hw_irq_controller for LPIs.
>
> If new hw_irq_controller is introduced for LPIs, then this has to
> be exported using some lpi structure which holds pointer to hw_irq_controller
> for guest & host type similar to gic_hw_ops
The interface is not set in stone, you are free to change what you want
as long as we keep something clean and comprehensible. It's the same for
the functions (I have in mind route_irq_to_guest).
In this case, I would prefer to see 2 callbacks (one for the host the
other for the guest) which return the correct IRQ controller for a
specific IRQ. I have in mind something like:
get_guest_hw_irq_controller(unsigned int irq)
{
if ( !is_lpi )
return &gicv3_guest_irq_controller
else
return &gicv3_guest_lpi_controller
}
Same for the host irq controller. So the selection of the IRQ controller
would be hidden from gic.c and keep the code a generic as possible.
>>> + /*
>>> + * Make the above write visible to the redistributors.
>>> + * And yes, we're flushing exactly: One. Single. Byte.
>>> + * Humpf...
>>> + */
>>> + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
>>> + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
>>> + else
>>> + dsb(ishst);
>>> +
>>> + /* Get collection id for this event id */
>>> + col = &its_dev->its->collections[virq % num_online_cpus()];
>>
>> This is fragile, you are assuming that num_online_cpus() will never
>> change. Why don't you store the collection in every irq_desc?
>
> This will add additional 8 bytes overhead for each irq_desc.
>
> Also is there a macro to get number of actual number processors in system.?
> AUI, nr_cpu_ids always returns 128
AFAIU, nr_cpu_ids should reflect the number of CPU of the platform. x86
correctly set it when parsing the ACPI. So I think this is a bug in the
ARM code.
In fact, I wasn't able to find a place in the ARM code where this value
is changed.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-26 15:05 ` Julien Grall
@ 2015-06-29 11:53 ` Ian Campbell
2015-06-29 12:46 ` Julien Grall
2015-07-02 12:15 ` Vijay Kilari
1 sibling, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 11:53 UTC (permalink / raw)
To: Julien Grall
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Fri, 2015-06-26 at 17:05 +0200, Julien Grall wrote:
> Hi Vijay,
>
> On 26/06/2015 14:54, Vijay Kilari wrote:
> > On Tue, Jun 23, 2015 at 8:02 PM, Julien Grall <julien.grall@citrix.com> wrote:
> >> Hi Vijay,
> >>
> >> On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> >>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> >>>
> >>> Implements hw_irq_controller api's required
> >>> to handle LPI's
> >>
> >> This patch doesn't hw_irq_controller for LPI but just hack around the
> >> current GICv3 host hw_irq_controller.
> >>
> >> As said on the previous version, the goal of hw_irq_controller is too
> >> keep things simple (i.e few conditional code). Please introduce a
> >> separate hw_irq_controller for LPIs.
> >
> > If new hw_irq_controller is introduced for LPIs, then this has to
> > be exported using some lpi structure which holds pointer to hw_irq_controller
> > for guest & host type similar to gic_hw_ops
>
> The interface is not set in stone, you are free to change what you want
> as long as we keep something clean and comprehensible. It's the same for
> the functions (I have in mind route_irq_to_guest).
Ack.
> In this case, I would prefer to see 2 callbacks (one for the host the
> other for the guest) which return the correct IRQ controller for a
> specific IRQ. I have in mind something like:
>
> get_guest_hw_irq_controller(unsigned int irq)
> {
> if ( !is_lpi )
> return &gicv3_guest_irq_controller
> else
> return &gicv3_guest_lpi_controller
> }
>
> Same for the host irq controller. So the selection of the IRQ controller
> would be hidden from gic.c and keep the code a generic as possible.
Yes, this is how I would expect it too.
Alternatively I notice that the pattern today is:
desc->handler = gic_hw_ops->gic_(host|guest)_irq_type;
[set_bit(_IRQ_GUEST, &desc->status) or not]
gic_set_irq_properties(desc,[...]);
So an alternative might be for the set_irq_properties hook in the ops to
also setup the handler (based on desc->status&_IRQ_GUEST and desc->irq),
perhaps renaming it to something less "property" based. Both callers are
git_route_irq_to_... so perhaps gic_route_irq?
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-29 11:53 ` Ian Campbell
@ 2015-06-29 12:46 ` Julien Grall
0 siblings, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-06-29 12:46 UTC (permalink / raw)
To: Ian Campbell
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 29/06/15 12:53, Ian Campbell wrote:
>> In this case, I would prefer to see 2 callbacks (one for the host the
>> other for the guest) which return the correct IRQ controller for a
>> specific IRQ. I have in mind something like:
>>
>> get_guest_hw_irq_controller(unsigned int irq)
>> {
>> if ( !is_lpi )
>> return &gicv3_guest_irq_controller
>> else
>> return &gicv3_guest_lpi_controller
>> }
>>
>> Same for the host irq controller. So the selection of the IRQ controller
>> would be hidden from gic.c and keep the code a generic as possible.
>
> Yes, this is how I would expect it too.
>
> Alternatively I notice that the pattern today is:
> desc->handler = gic_hw_ops->gic_(host|guest)_irq_type;
> [set_bit(_IRQ_GUEST, &desc->status) or not]
> gic_set_irq_properties(desc,[...]);
>
> So an alternative might be for the set_irq_properties hook in the ops to
> also setup the handler (based on desc->status&_IRQ_GUEST and desc->irq),
> perhaps renaming it to something less "property" based. Both callers are
> git_route_irq_to_... so perhaps gic_route_irq?
Sounds good for me.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-26 15:05 ` Julien Grall
2015-06-29 11:53 ` Ian Campbell
@ 2015-07-02 12:15 ` Vijay Kilari
1 sibling, 0 replies; 88+ messages in thread
From: Vijay Kilari @ 2015-07-02 12:15 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Fri, Jun 26, 2015 at 8:35 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
> On 26/06/2015 14:54, Vijay Kilari wrote:
>>
>> On Tue, Jun 23, 2015 at 8:02 PM, Julien Grall <julien.grall@citrix.com>
>> wrote:
>>>
>>> Hi Vijay,
>>>
>>> On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
>>>>
>>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>>
>>>> Implements hw_irq_controller api's required
>>>> to handle LPI's
>>>
>>>
>>> This patch doesn't hw_irq_controller for LPI but just hack around the
>>> current GICv3 host hw_irq_controller.
>>>
>>> As said on the previous version, the goal of hw_irq_controller is too
>>> keep things simple (i.e few conditional code). Please introduce a
>>> separate hw_irq_controller for LPIs.
>>
>>
>> If new hw_irq_controller is introduced for LPIs, then this has to
>> be exported using some lpi structure which holds pointer to
>> hw_irq_controller
>> for guest & host type similar to gic_hw_ops
>
>
> The interface is not set in stone, you are free to change what you want as
> long as we keep something clean and comprehensible. It's the same for the
> functions (I have in mind route_irq_to_guest).
>
> In this case, I would prefer to see 2 callbacks (one for the host the other
> for the guest) which return the correct IRQ controller for a specific IRQ. I
> have in mind something like:
>
> get_guest_hw_irq_controller(unsigned int irq)
> {
> if ( !is_lpi )
> return &gicv3_guest_irq_controller
> else
> return &gicv3_guest_lpi_controller
> }
>
> Same for the host irq controller. So the selection of the IRQ controller
> would be hidden from gic.c and keep the code a generic as possible.
>
>>>> + /*
>>>> + * Make the above write visible to the redistributors.
>>>> + * And yes, we're flushing exactly: One. Single. Byte.
>>>> + * Humpf...
>>>> + */
>>>> + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
>>>> + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
>>>> + else
>>>> + dsb(ishst);
>>>> +
>>>> + /* Get collection id for this event id */
>>>> + col = &its_dev->its->collections[virq % num_online_cpus()];
>>>
>>>
>>> This is fragile, you are assuming that num_online_cpus() will never
>>> change. Why don't you store the collection in every irq_desc?
>>
>>
>> This will add additional 8 bytes overhead for each irq_desc.
>>
>> Also is there a macro to get number of actual number processors in
>> system.?
>> AUI, nr_cpu_ids always returns 128
>
>
> AFAIU, nr_cpu_ids should reflect the number of CPU of the platform. x86
> correctly set it when parsing the ACPI. So I think this is a bug in the ARM
> code.
>
> In fact, I wasn't able to find a place in the ARM code where this value is
> changed.
nr_cpu_ids is not changed in case of ARM. I think this value has
to be updated in start_xen with cpus value
void __init start_xen(unsigned long boot_phys_offset,
unsigned long fdt_paddr,
unsigned long cpuid)
....
cpus = smp_get_max_cpus();
..
}
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-23 14:32 ` Julien Grall
2015-06-26 12:54 ` Vijay Kilari
@ 2015-06-26 14:25 ` Vijay Kilari
2015-06-26 15:15 ` Julien Grall
2015-06-29 11:59 ` Ian Campbell
2 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-06-26 14:25 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Julien,
>> + its_send_inv(its_dev, col, virq);
>> +}
>> +
>> +void its_set_affinity(struct irq_desc *desc, int cpu)
>> +{
>> + struct its_device *its_dev = get_irq_device(desc);
>> + struct its_collection *target_col;
>> +
>> + /* Physical collection id */
>> + target_col = &its_dev->its->collections[cpu];
>> + its_send_movi(its_dev, target_col, irq_to_virq(desc));
>
> The field "virq" in the structure irq_guest refers to the guest virtual
> IRQ and not the event ID. As Ian suggested in the proposal [1], please
> use an union to make this code clears.
Apart from adding union, do you recommend to add macros irq_to_vid()
and irq_to_virq() and use appropriately?
>
> Furthermore, when you set the LPI configuration (see lpi_set_config) you
> are using a round robin to get the collection. This won't work anymore
> if Xen decides to change the affinity... So you may want to drop
> affinity support for now.
>
>> +}
>
> Missing newline.
>
> [..]
>
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 2dd43ee..9dbdf7d 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -36,6 +36,7 @@ struct irq_guest
>> {
>> struct domain *d;
>> unsigned int virq;
>> + struct its_device *dev;
>
> I know that this was suggested in the proposal [1]. But the goal of
> irq_guest is to store anything specific to the guest. The event ID and
> the its_device assigned are known when the device is added to Xen and
> hence can be set in irq_desc (with a small memory impact, but we have
> plenty of memory on ARM64).
Do you mean adding its_device to irq_desc instead of irq_guest?
If so, irq_desc is common for both x86 & ARM.
Regards
Vijay
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-26 14:25 ` Vijay Kilari
@ 2015-06-26 15:15 ` Julien Grall
0 siblings, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-06-26 15:15 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 26/06/2015 16:25, Vijay Kilari wrote:
> Hi Julien,
Hi Vijay,
>>> + its_send_inv(its_dev, col, virq);
>>> +}
>>> +
>>> +void its_set_affinity(struct irq_desc *desc, int cpu)
>>> +{
>>> + struct its_device *its_dev = get_irq_device(desc);
>>> + struct its_collection *target_col;
>>> +
>>> + /* Physical collection id */
>>> + target_col = &its_dev->its->collections[cpu];
>>> + its_send_movi(its_dev, target_col, irq_to_virq(desc));
>>
>> The field "virq" in the structure irq_guest refers to the guest virtual
>> IRQ and not the event ID. As Ian suggested in the proposal [1], please
>> use an union to make this code clears.
>
> Apart from adding union, do you recommend to add macros irq_to_vid()
> and irq_to_virq() and use appropriately?
If it makes to code clearer yes. Although it may not be necessary if we
move the value into irq_desc (see below).
[..]
>>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>>> index 2dd43ee..9dbdf7d 100644
>>> --- a/xen/arch/arm/irq.c
>>> +++ b/xen/arch/arm/irq.c
>>> @@ -36,6 +36,7 @@ struct irq_guest
>>> {
>>> struct domain *d;
>>> unsigned int virq;
>>> + struct its_device *dev;
>>
>> I know that this was suggested in the proposal [1]. But the goal of
>> irq_guest is to store anything specific to the guest. The event ID and
>> the its_device assigned are known when the device is added to Xen and
>> hence can be set in irq_desc (with a small memory impact, but we have
>> plenty of memory on ARM64).
>
> Do you mean adding its_device to irq_desc instead of irq_guest?
> If so, irq_desc is common for both x86 & ARM.
There is an arch specific structure for irq_desc (see arch_irq_desc).
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-23 14:32 ` Julien Grall
2015-06-26 12:54 ` Vijay Kilari
2015-06-26 14:25 ` Vijay Kilari
@ 2015-06-29 11:59 ` Ian Campbell
2015-07-02 12:21 ` Vijay Kilari
2 siblings, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 11:59 UTC (permalink / raw)
To: Julien Grall
Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
tim, xen-devel, stefano.stabellini, manish.jaggi
On Tue, 2015-06-23 at 15:32 +0100, Julien Grall wrote:
[...]
> > +{
> > + struct its_collection *col;
> > + struct its_device *its_dev = get_irq_device(desc);
> > + u8 *cfg;
> > + u32 virq = irq_to_virq(desc);
> > +
> > + ASSERT(virq < its_dev->nr_lpis);
> > +
> > + cfg = gic_rdists->prop_page + desc->irq - NR_GIC_LPI;
> > + if ( enable )
> > + *cfg |= LPI_PROP_ENABLED;
> > + else
> > + *cfg &= ~LPI_PROP_ENABLED;
> > +
> > + /*
> > + * Make the above write visible to the redistributors.
> > + * And yes, we're flushing exactly: One. Single. Byte.
> > + * Humpf...
> > + */
> > + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
> > + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
> > + else
> > + dsb(ishst);
> > +
> > + /* Get collection id for this event id */
> > + col = &its_dev->its->collections[virq % num_online_cpus()];
>
> This is fragile, you are assuming that num_online_cpus() will never
> change. Why don't you store the collection in every irq_desc?
The original Linux code upon which this is based doesn't seem to need to
lookup the collection here, why is flushing needed for us but not Linux?
I'm also confused by the use of the variable name "virq" in a function
called set_lpi_config which appears to be dealing with host level
physical LPIs. It seems like this function would be broken for LPIs
which were delivered to Xen and not to a guest, and that the irq_to_virq
in here ought to be desc->irq, no?
BTW, the Linux original calls what this calls "virq" "id" instead, which
is much less confusing.
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-06-29 11:59 ` Ian Campbell
@ 2015-07-02 12:21 ` Vijay Kilari
2015-07-02 12:35 ` Ian Campbell
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-07-02 12:21 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Mon, Jun 29, 2015 at 5:29 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Tue, 2015-06-23 at 15:32 +0100, Julien Grall wrote:
> [...]
>> > +{
>> > + struct its_collection *col;
>> > + struct its_device *its_dev = get_irq_device(desc);
>> > + u8 *cfg;
>> > + u32 virq = irq_to_virq(desc);
>> > +
>> > + ASSERT(virq < its_dev->nr_lpis);
>> > +
>> > + cfg = gic_rdists->prop_page + desc->irq - NR_GIC_LPI;
>> > + if ( enable )
>> > + *cfg |= LPI_PROP_ENABLED;
>> > + else
>> > + *cfg &= ~LPI_PROP_ENABLED;
>> > +
>> > + /*
>> > + * Make the above write visible to the redistributors.
>> > + * And yes, we're flushing exactly: One. Single. Byte.
>> > + * Humpf...
>> > + */
>> > + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
>> > + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
>> > + else
>> > + dsb(ishst);
>> > +
>> > + /* Get collection id for this event id */
>> > + col = &its_dev->its->collections[virq % num_online_cpus()];
>>
>> This is fragile, you are assuming that num_online_cpus() will never
>> change. Why don't you store the collection in every irq_desc?
>
> The original Linux code upon which this is based doesn't seem to need to
> lookup the collection here, why is flushing needed for us but not Linux?
We are writing to lpi property table. Even linux code flushes it.
>
> I'm also confused by the use of the variable name "virq" in a function
> called set_lpi_config which appears to be dealing with host level
> physical LPIs. It seems like this function would be broken for LPIs
> which were delivered to Xen and not to a guest, and that the irq_to_virq
> in here ought to be desc->irq, no?
I am using desc->irq for updating lpi property table but using vid to
send inv command
>
> BTW, the Linux original calls what this calls "virq" "id" instead, which
> is much less confusing.
OK will rename virq to vid
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-02 12:21 ` Vijay Kilari
@ 2015-07-02 12:35 ` Ian Campbell
2015-07-02 12:44 ` Vijay Kilari
0 siblings, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-07-02 12:35 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Thu, 2015-07-02 at 17:51 +0530, Vijay Kilari wrote:
> On Mon, Jun 29, 2015 at 5:29 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> > On Tue, 2015-06-23 at 15:32 +0100, Julien Grall wrote:
> > [...]
> >> > +{
> >> > + struct its_collection *col;
> >> > + struct its_device *its_dev = get_irq_device(desc);
> >> > + u8 *cfg;
> >> > + u32 virq = irq_to_virq(desc);
> >> > +
> >> > + ASSERT(virq < its_dev->nr_lpis);
> >> > +
> >> > + cfg = gic_rdists->prop_page + desc->irq - NR_GIC_LPI;
> >> > + if ( enable )
> >> > + *cfg |= LPI_PROP_ENABLED;
> >> > + else
> >> > + *cfg &= ~LPI_PROP_ENABLED;
> >> > +
> >> > + /*
> >> > + * Make the above write visible to the redistributors.
> >> > + * And yes, we're flushing exactly: One. Single. Byte.
> >> > + * Humpf...
> >> > + */
> >> > + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
> >> > + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
> >> > + else
> >> > + dsb(ishst);
> >> > +
> >> > + /* Get collection id for this event id */
> >> > + col = &its_dev->its->collections[virq % num_online_cpus()];
> >>
> >> This is fragile, you are assuming that num_online_cpus() will never
> >> change. Why don't you store the collection in every irq_desc?
> >
> > The original Linux code upon which this is based doesn't seem to need to
> > lookup the collection here, why is flushing needed for us but not Linux?
>
> We are writing to lpi property table. Even linux code flushes it.
Sorry I was referring to the collection look up and inv, not the cache
flush, i.e. this bit:
+ /* Get collection id for this event id */
+ col = &its_dev->its->collections[virq % num_online_cpus()];
+ its_send_inv(its_dev, col, virq);
Linux doesn't seem to do that INV there.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-02 12:35 ` Ian Campbell
@ 2015-07-02 12:44 ` Vijay Kilari
2015-07-02 12:59 ` Ian Campbell
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-07-02 12:44 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Thu, Jul 2, 2015 at 6:05 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Thu, 2015-07-02 at 17:51 +0530, Vijay Kilari wrote:
>> On Mon, Jun 29, 2015 at 5:29 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
>> > On Tue, 2015-06-23 at 15:32 +0100, Julien Grall wrote:
>> > [...]
>> >> > +{
>> >> > + struct its_collection *col;
>> >> > + struct its_device *its_dev = get_irq_device(desc);
>> >> > + u8 *cfg;
>> >> > + u32 virq = irq_to_virq(desc);
>> >> > +
>> >> > + ASSERT(virq < its_dev->nr_lpis);
>> >> > +
>> >> > + cfg = gic_rdists->prop_page + desc->irq - NR_GIC_LPI;
>> >> > + if ( enable )
>> >> > + *cfg |= LPI_PROP_ENABLED;
>> >> > + else
>> >> > + *cfg &= ~LPI_PROP_ENABLED;
>> >> > +
>> >> > + /*
>> >> > + * Make the above write visible to the redistributors.
>> >> > + * And yes, we're flushing exactly: One. Single. Byte.
>> >> > + * Humpf...
>> >> > + */
>> >> > + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
>> >> > + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
>> >> > + else
>> >> > + dsb(ishst);
>> >> > +
>> >> > + /* Get collection id for this event id */
>> >> > + col = &its_dev->its->collections[virq % num_online_cpus()];
>> >>
>> >> This is fragile, you are assuming that num_online_cpus() will never
>> >> change. Why don't you store the collection in every irq_desc?
>> >
>> > The original Linux code upon which this is based doesn't seem to need to
>> > lookup the collection here, why is flushing needed for us but not Linux?
>>
>> We are writing to lpi property table. Even linux code flushes it.
>
> Sorry I was referring to the collection look up and inv, not the cache
> flush, i.e. this bit:
>
> + /* Get collection id for this event id */
> + col = &its_dev->its->collections[virq % num_online_cpus()];
> + its_send_inv(its_dev, col, virq);
>
> Linux doesn't seem to do that INV there.
Linux does INV.
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/irqchip/irq-gic-v3-its.c?id=refs/tags/v4.1
line 555
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-02 12:44 ` Vijay Kilari
@ 2015-07-02 12:59 ` Ian Campbell
2015-07-02 16:11 ` Julien Grall
0 siblings, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-07-02 12:59 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Thu, 2015-07-02 at 18:14 +0530, Vijay Kilari wrote:
> On Thu, Jul 2, 2015 at 6:05 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> > On Thu, 2015-07-02 at 17:51 +0530, Vijay Kilari wrote:
> >> On Mon, Jun 29, 2015 at 5:29 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> >> > On Tue, 2015-06-23 at 15:32 +0100, Julien Grall wrote:
> >> > [...]
> >> >> > +{
> >> >> > + struct its_collection *col;
> >> >> > + struct its_device *its_dev = get_irq_device(desc);
> >> >> > + u8 *cfg;
> >> >> > + u32 virq = irq_to_virq(desc);
> >> >> > +
> >> >> > + ASSERT(virq < its_dev->nr_lpis);
> >> >> > +
> >> >> > + cfg = gic_rdists->prop_page + desc->irq - NR_GIC_LPI;
> >> >> > + if ( enable )
> >> >> > + *cfg |= LPI_PROP_ENABLED;
> >> >> > + else
> >> >> > + *cfg &= ~LPI_PROP_ENABLED;
> >> >> > +
> >> >> > + /*
> >> >> > + * Make the above write visible to the redistributors.
> >> >> > + * And yes, we're flushing exactly: One. Single. Byte.
> >> >> > + * Humpf...
> >> >> > + */
> >> >> > + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
> >> >> > + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
> >> >> > + else
> >> >> > + dsb(ishst);
> >> >> > +
> >> >> > + /* Get collection id for this event id */
> >> >> > + col = &its_dev->its->collections[virq % num_online_cpus()];
> >> >>
> >> >> This is fragile, you are assuming that num_online_cpus() will never
> >> >> change. Why don't you store the collection in every irq_desc?
> >> >
> >> > The original Linux code upon which this is based doesn't seem to need to
> >> > lookup the collection here, why is flushing needed for us but not Linux?
> >>
> >> We are writing to lpi property table. Even linux code flushes it.
> >
> > Sorry I was referring to the collection look up and inv, not the cache
> > flush, i.e. this bit:
> >
> > + /* Get collection id for this event id */
> > + col = &its_dev->its->collections[virq % num_online_cpus()];
> > + its_send_inv(its_dev, col, virq);
> >
> > Linux doesn't seem to do that INV there.
>
> Linux does INV.
>
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/irqchip/irq-gic-v3-its.c?id=refs/tags/v4.1
>
> line 555
So it does, not sure how I missed that when I first looked.
Linux's approach of saving collection in the its_dev seems preferable to
looking it up like this here though.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs
2015-07-02 12:59 ` Ian Campbell
@ 2015-07-02 16:11 ` Julien Grall
0 siblings, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-07-02 16:11 UTC (permalink / raw)
To: Ian Campbell, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On 02/07/15 13:59, Ian Campbell wrote:
> On Thu, 2015-07-02 at 18:14 +0530, Vijay Kilari wrote:
>> On Thu, Jul 2, 2015 at 6:05 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
>>> On Thu, 2015-07-02 at 17:51 +0530, Vijay Kilari wrote:
>>>> On Mon, Jun 29, 2015 at 5:29 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
>>>>> On Tue, 2015-06-23 at 15:32 +0100, Julien Grall wrote:
>>>>> [...]
>>>>>>> +{
>>>>>>> + struct its_collection *col;
>>>>>>> + struct its_device *its_dev = get_irq_device(desc);
>>>>>>> + u8 *cfg;
>>>>>>> + u32 virq = irq_to_virq(desc);
>>>>>>> +
>>>>>>> + ASSERT(virq < its_dev->nr_lpis);
>>>>>>> +
>>>>>>> + cfg = gic_rdists->prop_page + desc->irq - NR_GIC_LPI;
>>>>>>> + if ( enable )
>>>>>>> + *cfg |= LPI_PROP_ENABLED;
>>>>>>> + else
>>>>>>> + *cfg &= ~LPI_PROP_ENABLED;
>>>>>>> +
>>>>>>> + /*
>>>>>>> + * Make the above write visible to the redistributors.
>>>>>>> + * And yes, we're flushing exactly: One. Single. Byte.
>>>>>>> + * Humpf...
>>>>>>> + */
>>>>>>> + if ( gic_rdists->flags & RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING )
>>>>>>> + clean_and_invalidate_dcache_va_range(cfg, sizeof(*cfg));
>>>>>>> + else
>>>>>>> + dsb(ishst);
>>>>>>> +
>>>>>>> + /* Get collection id for this event id */
>>>>>>> + col = &its_dev->its->collections[virq % num_online_cpus()];
>>>>>>
>>>>>> This is fragile, you are assuming that num_online_cpus() will never
>>>>>> change. Why don't you store the collection in every irq_desc?
>>>>>
>>>>> The original Linux code upon which this is based doesn't seem to need to
>>>>> lookup the collection here, why is flushing needed for us but not Linux?
>>>>
>>>> We are writing to lpi property table. Even linux code flushes it.
>>>
>>> Sorry I was referring to the collection look up and inv, not the cache
>>> flush, i.e. this bit:
>>>
>>> + /* Get collection id for this event id */
>>> + col = &its_dev->its->collections[virq % num_online_cpus()];
>>> + its_send_inv(its_dev, col, virq);
>>>
>>> Linux doesn't seem to do that INV there.
>>
>> Linux does INV.
>>
>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/irqchip/irq-gic-v3-its.c?id=refs/tags/v4.1
>>
>> line 555
>
> So it does, not sure how I missed that when I first looked.
>
> Linux's approach of saving collection in the its_dev seems preferable to
> looking it up like this here though.
That would be a better idea. We have too many place in this series where
the way to retrieve the collection for an interrupt.
Hence, the irq_desc should have a pointer to its_dev (as suggested in
another patch).
struct device would even be better, but I can live with its_dev for now.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (6 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 07/18] xen/arm: ITS: implement hw_irq_controller for LPIs vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-23 16:39 ` Julien Grall
` (2 more replies)
2015-06-22 12:01 ` [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
` (12 subsequent siblings)
20 siblings, 3 replies; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
This patch introduces virtual ITS driver with following
functionality
- Introduces helper functions to manage device table and
ITT table in guest memory
- Helper function to handle virtual ITS devices assigned
to domain
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/Makefile | 1 +
xen/arch/arm/vgic-v3-its.c | 266 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/domain.h | 2 +
xen/include/asm-arm/gic-its.h | 49 ++++++++
4 files changed, 318 insertions(+)
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 1821ed2..8590846 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -33,6 +33,7 @@ obj-y += traps.o
obj-y += vgic.o vgic-v2.o
obj-$(CONFIG_ARM_64) += vgic-v3.o
obj-$(CONFIG_ARM_64) += gic-v3-its.o
+obj-$(CONFIG_ARM_64) += vgic-v3-its.o
obj-y += vtimer.o
obj-y += vuart.o
obj-y += hvm.o
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
new file mode 100644
index 0000000..ea52a87
--- /dev/null
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -0,0 +1,266 @@
+/*
+ * Copyright (C) 2015 Cavium Inc.
+ * Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/bitops.h>
+#include <xen/config.h>
+#include <xen/init.h>
+#include <xen/irq.h>
+#include <xen/list.h>
+#include <xen/sched.h>
+#include <xen/sizes.h>
+#include <asm/device.h>
+#include <asm/mmio.h>
+#include <asm/io.h>
+#include <asm/gic_v3_defs.h>
+#include <asm/gic.h>
+#include <asm/vgic.h>
+#include <asm/gic-its.h>
+#include <xen/log2.h>
+
+/* GITS register definitions */
+#define VITS_GITS_TYPER_HCC (0xffU << 24)
+#define VITS_GITS_TYPER_PTA_SHIFT (19)
+#define VITS_GITS_DEV_BITS (0x14U << 13)
+#define VITS_GITS_ID_BITS (0x13U << 8)
+#define VITS_GITS_ITT_SIZE (0x7U << 4)
+#define VITS_GITS_DISTRIBUTED (0x1U << 3)
+#define VITS_GITS_PLPIS (0x1U << 0)
+
+/* GITS_PIDRn register values for ARM implementations */
+#define GITS_PIDR0_VAL (0x94)
+#define GITS_PIDR1_VAL (0xb4)
+#define GITS_PIDR2_VAL (0x3b)
+#define GITS_PIDR3_VAL (0x00)
+#define GITS_PIDR4_VAL (0x04)
+
+// #define DEBUG_ITS
+
+#ifdef DEBUG_ITS
+# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
+#else
+# define DPRINTK(fmt, args...) do {} while ( 0 )
+#endif
+
+#ifdef DEBUG_ITS
+static void dump_cmd(its_cmd_block *cmd)
+{
+ printk("CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
+ cmd->raw_cmd[0], cmd->raw_cmd[1], cmd->raw_cmd[2], cmd->raw_cmd[3]);
+}
+#endif
+
+/* ITS device table helper functions */
+int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
+ struct vdevice_table *entry, int set)
+{
+ uint64_t offset;
+ paddr_t dt_entry;
+ struct page_info *page;
+ p2m_type_t p2mt;
+ void *p;
+
+ offset = dev_id * sizeof(struct vdevice_table);
+ if ( offset > d->arch.vits->dt_size )
+ {
+ dprintk(XENLOG_G_ERR,
+ "vITS:d%dv%d: Out of range offset 0x%lx id 0x%x size 0x%lx\n",
+ d->domain_id, current->vcpu_id, offset, dev_id,
+ d->arch.vits->dt_size);
+ return -EINVAL;
+ }
+
+ dt_entry = d->arch.vits->dt_ipa + offset;
+
+ page = get_page_from_gfn(d, paddr_to_pfn(dt_entry), &p2mt, P2M_ALLOC);
+ if ( !page )
+ {
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to get VITT device table\n",
+ d->domain_id, current->vcpu_id);
+ return -EINVAL;
+ }
+
+ if ( !p2m_is_ram(p2mt) )
+ {
+ put_page(page);
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d with wrong attributes\n",
+ d->domain_id, current->vcpu_id);
+ return -EINVAL;
+ return -EINVAL;
+ }
+
+ p = __map_domain_page(page);
+ /* Offset within the mapped page */
+ offset = dt_entry & (PAGE_SIZE - 1);
+
+ if ( set )
+ memcpy(p + offset, entry, sizeof(struct vdevice_table));
+ else
+ memcpy(entry, p + offset, sizeof(struct vdevice_table));
+
+ unmap_domain_page(p);
+ put_page(page);
+
+ return 0;
+}
+
+int vits_set_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry)
+{
+ return vits_vdevice_entry(d, devid, entry, 1);
+}
+
+int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry)
+{
+ return vits_vdevice_entry(d, devid, entry, 0);
+}
+
+int vits_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry, int set)
+{
+ struct vdevice_table dt_entry;
+ struct page_info *page;
+ paddr_t vitt_entry;
+ p2m_type_t p2mt;
+ uint64_t offset;
+ void *p;
+
+ if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
+ {
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d Fail to get vdevice for dev 0x%x\n",
+ d->domain_id, current->vcpu_id, devid);
+ return -EINVAL;
+ }
+
+ /* dt_entry is validated when read */
+ offset = event * sizeof(struct vitt);
+ if ( offset > dt_entry.vitt_size )
+ {
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d ITT out of range\n",
+ d->domain_id, current->vcpu_id);
+ return -EINVAL;
+ }
+
+ vitt_entry = dt_entry.vitt_ipa + offset;
+ page = get_page_from_gfn(d, paddr_to_pfn(vitt_entry), &p2mt, P2M_ALLOC);
+ if ( !page )
+ {
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to get VITT device table\n",
+ d->domain_id, current->vcpu_id);
+ return -EINVAL;
+ }
+
+ if ( !p2m_is_ram(p2mt) )
+ {
+ put_page(page);
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d with wrong attributes\n",
+ d->domain_id, current->vcpu_id);
+ return -EINVAL;
+ }
+
+ p = __map_domain_page(page);
+ /* Offset within the mapped page */
+ offset = vitt_entry & (PAGE_SIZE - 1);
+
+ if ( set )
+ memcpy(p + offset, entry, sizeof(struct vitt));
+ else
+ memcpy(entry, p + offset, sizeof(struct vitt));
+
+ unmap_domain_page(p);
+ put_page(page);
+
+ return 0;
+}
+
+int vits_set_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry)
+{
+ return vits_vitt_entry(d, devid, event, entry, 1);
+}
+
+int vits_get_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry)
+{
+ return vits_vitt_entry(d, devid, event, entry, 0);
+}
+
+/* RB-tree helpers for vits_device attached to a domain */
+struct vits_device * find_vits_device(struct rb_root *root, uint32_t devid)
+{
+ struct rb_node *node = root->rb_node;
+
+ while ( node )
+ {
+ struct vits_device *dev;
+
+ dev = container_of(node, struct vits_device, node);
+
+ if ( devid < dev->vdevid )
+ node = node->rb_left;
+ else if ( devid > dev->vdevid )
+ node = node->rb_right;
+ else
+ return dev;
+ }
+
+ return NULL;
+}
+
+int insert_vits_device(struct rb_root *root, struct vits_device *dev)
+{
+ struct rb_node **new, *parent;
+
+ new = &root->rb_node;
+ parent = NULL;
+ while ( *new )
+ {
+ struct vits_device *this;
+
+ this = container_of(*new, struct vits_device, node);
+
+ parent = *new;
+ if ( dev->vdevid < this->vdevid )
+ new = &((*new)->rb_left);
+ else if ( dev->vdevid > this->vdevid )
+ new = &((*new)->rb_right);
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node(&dev->node, parent, new);
+ rb_insert_color(&dev->node, root);
+
+ return 0;
+}
+
+int remove_vits_device(struct rb_root *root, struct vits_device *dev)
+{
+ if ( dev )
+ rb_erase(&dev->node, root);
+
+ return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index f1a087e..da73cf5 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -115,6 +115,8 @@ struct arch_domain
#endif
} vgic;
+ struct vgic_its *vits;
+
struct vuart {
#define VUART_BUF_SIZE 128
char *buf;
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index a47cf26..a1099a1 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -31,6 +31,35 @@ struct its_collection {
u16 col_id;
};
+/*
+ * Per domain virtual ITS structure.
+ * One per Physical ITS node available for the domain
+ */
+struct vgic_its
+{
+ spinlock_t lock;
+ /* Command queue base */
+ paddr_t cmd_base;
+ /* Command queue write pointer */
+ paddr_t cmd_write;
+ /* Command queue write saved pointer */
+ paddr_t cmd_write_save;
+ /* Command queue read pointer */
+ paddr_t cmd_read;
+ /* Command queue size */
+ unsigned long cmd_qsize;
+ /* ITS physical node */
+ struct its_node *its;
+ /* vITT device table ipa */
+ paddr_t dt_ipa;
+ /* vITT device table size */
+ uint64_t dt_size;
+ /* Radix-tree root of devices attached to this domain */
+ struct rb_root dev_root;
+ /* collections mapped */
+ struct its_collection *collections;
+};
+
/* ITS command structures */
typedef struct __packed {
u64 cmd:8;
@@ -200,6 +229,26 @@ struct its_device {
struct rb_node node;
};
+struct vits_device {
+ uint32_t vdevid;
+ uint32_t pdevid;
+ struct its_device *its_dev;
+ struct rb_node node;
+};
+
+struct vdevice_table {
+ uint64_t vitt_ipa;
+ uint32_t vitt_size;
+ uint32_t padding;
+};
+
+struct vitt {
+ uint16_t valid:1;
+ uint16_t pad:15;
+ uint16_t vcollection;
+ uint32_t vlpi;
+};
+
static inline uint8_t its_decode_cmd(its_cmd_block *cmd)
{
return cmd->raw_cmd[0] & 0xff;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver
2015-06-22 12:01 ` [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver vijay.kilari
@ 2015-06-23 16:39 ` Julien Grall
2015-07-02 13:33 ` Vijay Kilari
2015-06-24 9:20 ` Julien Grall
2015-06-29 12:13 ` Ian Campbell
2 siblings, 1 reply; 88+ messages in thread
From: Julien Grall @ 2015-06-23 16:39 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> This patch introduces virtual ITS driver with following
> functionality
> - Introduces helper functions to manage device table and
> ITT table in guest memory
> - Helper function to handle virtual ITS devices assigned
> to domain
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/Makefile | 1 +
> xen/arch/arm/vgic-v3-its.c | 266 +++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/domain.h | 2 +
> xen/include/asm-arm/gic-its.h | 49 ++++++++
> 4 files changed, 318 insertions(+)
>
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 1821ed2..8590846 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -33,6 +33,7 @@ obj-y += traps.o
> obj-y += vgic.o vgic-v2.o
> obj-$(CONFIG_ARM_64) += vgic-v3.o
> obj-$(CONFIG_ARM_64) += gic-v3-its.o
> +obj-$(CONFIG_ARM_64) += vgic-v3-its.o
I would prefer to the vgic-v3-its code not compiled until you finish to
implement it rather than having every function exported while it should
be static.
> obj-y += vtimer.o
> obj-y += vuart.o
> obj-y += hvm.o
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> new file mode 100644
> index 0000000..ea52a87
> --- /dev/null
> +++ b/xen/arch/arm/vgic-v3-its.c
[..]
> +/* GITS register definitions */
> +#define VITS_GITS_TYPER_HCC (0xffU << 24)
Where does this value comes from? The number of supported collection
should be the maximum number of VCPUs supported by the domain.
Note that the hardware can only hold 256 collections, all the other
requires provisioning. But it's fine for now, although a BUILD_BUG_ON in
the vGIC code would be nice.
> +#define VITS_GITS_TYPER_PTA_SHIFT (19)
> +#define VITS_GITS_DEV_BITS (0x14U << 13)
> +#define VITS_GITS_ID_BITS (0x13U << 8)
Where do those values come from? This needs at least to come from the
hardware GIC configuration if we plan to only support DOM0 for now.
> +#define VITS_GITS_ITT_SIZE (0x7U << 4)
The name is misleading, this is not the size of the ITT but the value to
store in the GITS_TYPER register.
> +#define VITS_GITS_DISTRIBUTED (0x1U << 3)
Why? This bit is implementation defined in the spec.
> +#define VITS_GITS_PLPIS (0x1U << 0)
For all these defines, why do you hardcode the shift rather using
GITS_TYPER_* you provide in patch #5? It would make the code more easier
to understand.
> +
> +/* GITS_PIDRn register values for ARM implementations */
> +#define GITS_PIDR0_VAL (0x94)
> +#define GITS_PIDR1_VAL (0xb4)
> +#define GITS_PIDR2_VAL (0x3b)
> +#define GITS_PIDR3_VAL (0x00)
> +#define GITS_PIDR4_VAL (0x04)
> +
> +// #define DEBUG_ITS
> +
> +#ifdef DEBUG_ITS
> +# define DPRINTK(fmt, args...) dprintk(XENLOG_DEBUG, fmt, ##args)
> +#else
> +# define DPRINTK(fmt, args...) do {} while ( 0 )
> +#endif
> +
> +#ifdef DEBUG_ITS
> +static void dump_cmd(its_cmd_block *cmd)
> +{
> + printk("CMD[0] = 0x%lx CMD[1] = 0x%lx CMD[2] = 0x%lx CMD[3] = 0x%lx\n",
> + cmd->raw_cmd[0], cmd->raw_cmd[1], cmd->raw_cmd[2], cmd->raw_cmd[3]);
> +}
> +#endif
> +
> +/* ITS device table helper functions */
> +int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
> + struct vdevice_table *entry, int set)
Any functions not exported should be static. I won't repeat for all the
others within this file.
Also, please use bool_t for set.
> +{
> + uint64_t offset;
> + paddr_t dt_entry;
> + struct page_info *page;
> + p2m_type_t p2mt;
> + void *p;
> +
> + offset = dev_id * sizeof(struct vdevice_table);
> + if ( offset > d->arch.vits->dt_size )
If you gonna let the guest to write deci
> + {
> + dprintk(XENLOG_G_ERR,
> + "vITS:d%dv%d: Out of range offset 0x%lx id 0x%x size 0x%lx\n",
You can use %pv to print the domain/vcpuid (see an example in vgic-v3.c).
Also, all the value can be printed in decimal because we have to use
them for addition...
> + d->domain_id, current->vcpu_id, offset, dev_id,
Please don't mix d and current.
> + d->arch.vits->dt_size);
> + return -EINVAL;
> + }
> +
> + dt_entry = d->arch.vits->dt_ipa + offset;
> +
> + page = get_page_from_gfn(d, paddr_to_pfn(dt_entry), &p2mt, P2M_ALLOC);
> + if ( !page )
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to get VITT device table\n",
> + d->domain_id, current->vcpu_id);
> + return -EINVAL;
> + }
> +
> + if ( !p2m_is_ram(p2mt) )
> + {
> + put_page(page);
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d with wrong attributes\n",
> + d->domain_id, current->vcpu_id);
> + return -EINVAL;
> + return -EINVAL;
Duplicate "return -EINVAL".
> + }
> +
> + p = __map_domain_page(page);
> + /* Offset within the mapped page */
> + offset = dt_entry & (PAGE_SIZE - 1);
> +
> + if ( set )
> + memcpy(p + offset, entry, sizeof(struct vdevice_table));
> + else
> + memcpy(entry, p + offset, sizeof(struct vdevice_table));
> +
> + unmap_domain_page(p);
> + put_page(page);
> +
> + return 0;
> +}
> +
> +int vits_set_vdevice_entry(struct domain *d, uint32_t devid,
> + struct vdevice_table *entry)
> +{
> + return vits_vdevice_entry(d, devid, entry, 1);
> +}
> +
> +int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
> + struct vdevice_table *entry)
> +{
> + return vits_vdevice_entry(d, devid, entry, 0);
> +}
> +
> +int vits_vitt_entry(struct domain *d, uint32_t devid,
> + uint32_t event, struct vitt *entry, int set)
> +{
> + struct vdevice_table dt_entry;
> + struct page_info *page;
> + paddr_t vitt_entry;
> + p2m_type_t p2mt;
> + uint64_t offset;
> + void *p;
> +
> + if ( vits_get_vdevice_entry(d, devid, &dt_entry) )
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d Fail to get vdevice for dev 0x%x\n",
> + d->domain_id, current->vcpu_id, devid);
> + return -EINVAL;
> + }
> +
> + /* dt_entry is validated when read */
> + offset = event * sizeof(struct vitt);
> + if ( offset > dt_entry.vitt_size )
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d ITT out of range\n",
> + d->domain_id, current->vcpu_id);
> + return -EINVAL;
> + }
> +
> + vitt_entry = dt_entry.vitt_ipa + offset;
> + page = get_page_from_gfn(d, paddr_to_pfn(vitt_entry), &p2mt, P2M_ALLOC);
> + if ( !page )
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to get VITT device table\n",
> + d->domain_id, current->vcpu_id);
> + return -EINVAL;
> + }
> +
> + if ( !p2m_is_ram(p2mt) )
> + {
> + put_page(page);
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d with wrong attributes\n",
> + d->domain_id, current->vcpu_id);
> + return -EINVAL;
> + }
> +
> + p = __map_domain_page(page);
> + /* Offset within the mapped page */
> + offset = vitt_entry & (PAGE_SIZE - 1);
> +
> + if ( set )
> + memcpy(p + offset, entry, sizeof(struct vitt));
> + else
> + memcpy(entry, p + offset, sizeof(struct vitt));
> +
> + unmap_domain_page(p);
> + put_page(page);
The code to transfer the data is exactly the same as vits_vdevice_entry
except the size of the copy. Can you make an helper? It would avoid
duplicate code.
> +
> + return 0;
> +}
> +
> +int vits_set_vitt_entry(struct domain *d, uint32_t devid,
> + uint32_t event, struct vitt *entry)
> +{
> + return vits_vitt_entry(d, devid, event, entry, 1);
> +}
> +
> +int vits_get_vitt_entry(struct domain *d, uint32_t devid,
> + uint32_t event, struct vitt *entry)
> +{
> + return vits_vitt_entry(d, devid, event, entry, 0);
> +}
> +
> +/* RB-tree helpers for vits_device attached to a domain */
> +struct vits_device * find_vits_device(struct rb_root *root, uint32_t devid)
Coding style: vits_device *find_
> +{
> + struct rb_node *node = root->rb_node;
> +
> + while ( node )
> + {
> + struct vits_device *dev;
> +
> + dev = container_of(node, struct vits_device, node);
> +
> + if ( devid < dev->vdevid )
> + node = node->rb_left;
> + else if ( devid > dev->vdevid )
> + node = node->rb_right;
> + else
> + return dev;
> + }
> +
> + return NULL;
> +}
> +
> +int insert_vits_device(struct rb_root *root, struct vits_device *dev)
> +{
> + struct rb_node **new, *parent;
> +
> + new = &root->rb_node;
> + parent = NULL;
> + while ( *new )
> + {
> + struct vits_device *this;
> +
> + this = container_of(*new, struct vits_device, node);
> +
> + parent = *new;
> + if ( dev->vdevid < this->vdevid )
> + new = &((*new)->rb_left);
> + else if ( dev->vdevid > this->vdevid )
> + new = &((*new)->rb_right);
> + else
> + return -EEXIST;
> + }
> +
> + rb_link_node(&dev->node, parent, new);
> + rb_insert_color(&dev->node, root);
> +
> + return 0;
> +}
> +
> +int remove_vits_device(struct rb_root *root, struct vits_device *dev)
This function can be void.
> +{
> + if ( dev )
> + rb_erase(&dev->node, root);
> +
> + return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index f1a087e..da73cf5 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -115,6 +115,8 @@ struct arch_domain
> #endif
> } vgic;
>
> + struct vgic_its *vits;
> +
> struct vuart {
> #define VUART_BUF_SIZE 128
> char *buf;
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index a47cf26..a1099a1 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -31,6 +31,35 @@ struct its_collection {
> u16 col_id;
> };
>
> +/*
> + * Per domain virtual ITS structure.
> + * One per Physical ITS node available for the domain
Left over of the previous version? AFAICT, the agreed design [1] only
expose one vITS per domain.
> + */
> +struct vgic_its
> +{
> + spinlock_t lock;
> + /* Command queue base */
> + paddr_t cmd_base;
> + /* Command queue write pointer */
> + paddr_t cmd_write;
> + /* Command queue write saved pointer */
> + paddr_t cmd_write_save;
> + /* Command queue read pointer */
> + paddr_t cmd_read;
> + /* Command queue size */
> + unsigned long cmd_qsize;
> + /* ITS physical node */
> + struct its_node *its;
This is not necessary.
> + /* vITT device table ipa */
> + paddr_t dt_ipa;
> + /* vITT device table size */
> + uint64_t dt_size;
> + /* Radix-tree root of devices attached to this domain */
> + struct rb_root dev_root;
> + /* collections mapped */
> + struct its_collection *collections;
> +};
> +
> /* ITS command structures */
> typedef struct __packed {
> u64 cmd:8;
> @@ -200,6 +229,26 @@ struct its_device {
> struct rb_node node;
> };
>
> +struct vits_device {
> + uint32_t vdevid;
> + uint32_t pdevid;
> + struct its_device *its_dev;
> + struct rb_node node;
> +};
We spoke about a specific structure in the design [2] but you introduced
a new one. Why?
Having everything in the its_device would help to catch a device
attached to 2 different domains...
Also, the field pdevid is not vits specific but its.
> +
> +struct vdevice_table {
> + uint64_t vitt_ipa;
> + uint32_t vitt_size;
> + uint32_t padding;
> +};
> +
> +struct vitt {
> + uint16_t valid:1;
> + uint16_t pad:15;
> + uint16_t vcollection;
> + uint32_t vlpi;
> +};
You forgot to add the BUILD_BUG_ON for each structure here. It's very
important as you decided to hardcoded the value in the registers.
> +
> static inline uint8_t its_decode_cmd(its_cmd_block *cmd)
> {
> return cmd->raw_cmd[0] & 0xff;
>
Regards,
[1]
http://xenbits.xensource.com/people/ianc/vits/draftF.html#vits-to-pits-mapping
[2]
http://xenbits.xensource.com/people/ianc/vits/draftF.html#vits-to-pits-mapping
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver
2015-06-23 16:39 ` Julien Grall
@ 2015-07-02 13:33 ` Vijay Kilari
2015-07-02 14:30 ` Ian Campbell
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-07-02 13:33 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
Hi Julien,
On Tue, Jun 23, 2015 at 10:09 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi Vijay,
>
>>
>> +struct vits_device {
>> + uint32_t vdevid;
>> + uint32_t pdevid;
>> + struct its_device *its_dev;
>> + struct rb_node node;
>> +};
>
> We spoke about a specific structure in the design [2] but you introduced
> a new one. Why?
Section 6 of DraftG specifies to manage separate tree for device assignment.
This helps to manage RB-tree per domain to hold list of devices
assigned to this domain index with vdevid.
This helps to check if device is assigned to this domain before processing
any ITS command with that vdevid.
>
> Having everything in the its_device would help to catch a device
> attached to 2 different domains...
One option is to introduce a new variable inside its_device to know
to which domain the device is currently assigned.
>
> Also, the field pdevid is not vits specific but its.
pdevid can be removed as its_device structure already has it
Regards
Vijay
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver
2015-07-02 13:33 ` Vijay Kilari
@ 2015-07-02 14:30 ` Ian Campbell
0 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-07-02 14:30 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Thu, 2015-07-02 at 19:03 +0530, Vijay Kilari wrote:
> Hi Julien,
>
> On Tue, Jun 23, 2015 at 10:09 PM, Julien Grall <julien.grall@citrix.com> wrote:
> > Hi Vijay,
> >
> >>
> >> +struct vits_device {
> >> + uint32_t vdevid;
> >> + uint32_t pdevid;
> >> + struct its_device *its_dev;
> >> + struct rb_node node;
> >> +};
> >
> > We spoke about a specific structure in the design [2] but you introduced
> > a new one. Why?
>
> Section 6 of DraftG specifies to manage separate tree for device assignment.
> This helps to manage RB-tree per domain to hold list of devices
> assigned to this domain index with vdevid.
>
> This helps to check if device is assigned to this domain before processing
> any ITS command with that vdevid.
>
> >
> > Having everything in the its_device would help to catch a device
> > attached to 2 different domains...
>
> One option is to introduce a new variable inside its_device to know
> to which domain the device is currently assigned.
IIRC that's what I intended, e.g. two trees referencing the same
underlying data structure. Sorry that wasn't clear.
>
> >
> > Also, the field pdevid is not vits specific but its.
> pdevid can be removed as its_device structure already has it
>
> Regards
> Vijay
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver
2015-06-22 12:01 ` [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver vijay.kilari
2015-06-23 16:39 ` Julien Grall
@ 2015-06-24 9:20 ` Julien Grall
2015-06-29 12:13 ` Ian Campbell
2 siblings, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-06-24 9:20 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi,
On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index f1a087e..da73cf5 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -115,6 +115,8 @@ struct arch_domain
> #endif
> } vgic;
>
> + struct vgic_its *vits;
> +
I forgot to mention that this should only be compiled when GICv3 is used
(i.e only for ARM64).
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver
2015-06-22 12:01 ` [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver vijay.kilari
2015-06-23 16:39 ` Julien Grall
2015-06-24 9:20 ` Julien Grall
@ 2015-06-29 12:13 ` Ian Campbell
2 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 12:13 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> + if ( !p2m_is_ram(p2mt) )
> + {
> + put_page(page);
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d with wrong attributes\n",
> + d->domain_id, current->vcpu_id);
"d%dv%d" should be done (everywhere) with the "%pv" format code passing
current as the argument. See docs/misc/printk-formats.txt.
> + return -EINVAL;
> + return -EINVAL;
> + }
> +
> + p = __map_domain_page(page);
> + /* Offset within the mapped page */
> + offset = dt_entry & (PAGE_SIZE - 1);
This should be "& ~PAGE_MASK" please.
> +
> + if ( set )
> + memcpy(p + offset, entry, sizeof(struct vdevice_table));
> + else
> + memcpy(entry, p + offset, sizeof(struct vdevice_table));
Personally I'd have done this with *entry = *(struct ..*)(p + offset)
and vice versa and let the compiler figure out how to achieve that.
> + /* dt_entry is validated when read */
The vits_get_vdevice_entry call is right above and in this
implementation I don't see any checks in that function, so I don't think
this comment is strictly true.
Since it is very important that information living in this guest
accessible memory is correctly validated before use I think these
comments need to be 100% trustworthy.
I think vits_get_vdevice_entry is indeed the right place for those
checks as the comment suggests.
If there are no actual checks needed (e.g. because everything is in
terms of IPA) then there should be a comment in that function explaining
exactly why nothing actually needs to be validated.
> + offset = event * sizeof(struct vitt);
> + if ( offset > dt_entry.vitt_size )
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d ITT out of range\n",
> + d->domain_id, current->vcpu_id);
> + return -EINVAL;
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (7 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 08/18] xen/arm: vITS: Add virtual ITS driver vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-24 10:29 ` Julien Grall
` (2 more replies)
2015-06-22 12:01 ` [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
` (11 subsequent siblings)
20 siblings, 3 replies; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add Virtual ITS command processing support to Virtual ITS driver
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3-its.c | 7 +
xen/arch/arm/vgic-v3-its.c | 393 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 400 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 535fc53..2a4fa97 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -89,6 +89,7 @@ struct its_node {
#define ITS_ITT_ALIGN SZ_256
+static u32 id_bits;
static LIST_HEAD(its_nodes);
static DEFINE_SPINLOCK(its_lock);
static struct rdist_prop *gic_rdists;
@@ -146,6 +147,11 @@ void dump_cmd(its_cmd_block *cmd)
}
#endif
+u32 its_get_nr_events(void)
+{
+ return (1 << id_bits);
+}
+
/* RB-tree helpers for its_device */
struct its_device * find_its_device(struct rb_root *root, u32 devid)
{
@@ -1044,6 +1050,7 @@ static int its_probe(struct dt_device_node *node)
its->phys_size = its_size;
typer = readl_relaxed(its_base + GITS_TYPER);
its->ite_size = ((typer >> 4) & 0xf) + 1;
+ id_bits = GITS_TYPER_IDBITS(typer);
its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
if ( !its->cmd_base )
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index ea52a87..0671434 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -256,6 +256,399 @@ int remove_vits_device(struct rb_root *root, struct vits_device *dev)
return 0;
}
+static int vgic_its_process_sync(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* XXX: Ignored */
+ DPRINTK("vITS:d%dv%d SYNC: ta 0x%x \n",
+ v->domain->domain_id, v->vcpu_id, virt_cmd->sync.ta);
+
+ return 0;
+}
+
+static int vgic_its_process_mapvi(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct vits_device *vdev;
+ uint8_t vcol_id, cmd;
+ uint32_t vid, dev_id, event;
+
+ vcol_id = virt_cmd->mapvi.col;
+ vid = virt_cmd->mapvi.phy_id;
+ dev_id = its_decode_devid(v->domain, virt_cmd);
+ cmd = virt_cmd->mapvi.cmd;
+
+ DPRINTK("vITS:d%dv%d MAPVI: dev_id 0x%x vcol_id %d vid %d \n",
+ v->domain->domain_id, v->vcpu_id, dev_id, vcol_id, vid);
+
+ if ( vcol_id > (v->domain->max_vcpus + 1) || vid > its_get_nr_events() )
+ return -EINVAL;
+
+ /* XXX: Enable validation later */
+ vdev = find_vits_device(&v->domain->arch.vits->dev_root, dev_id);
+ if ( !vdev && !vdev->its_dev )
+ return -EINVAL;
+
+ entry.valid = true;
+ entry.vcollection = vcol_id;
+ entry.vlpi = vid;
+
+ if ( cmd == GITS_CMD_MAPI )
+ vits_set_vitt_entry(v->domain, dev_id, vid, &entry);
+ else
+ {
+ event = virt_cmd->mapvi.event;
+ if ( event > its_get_nr_events() )
+ return -EINVAL;
+
+ vits_set_vitt_entry(v->domain, dev_id, event, &entry);
+ }
+
+ return 0;
+}
+
+static int vgic_its_process_movi(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt entry;
+ struct vits_device *vdev;
+ uint32_t dev_id, event;
+ uint8_t vcol_id;
+
+ dev_id = its_decode_devid(v->domain, virt_cmd);
+ vcol_id = virt_cmd->movi.col;
+ event = virt_cmd->movi.event;
+
+ DPRINTK("vITS:d%dv%d MOVI: dev_id 0x%x vcol_id %d event %d\n",
+ v->domain->domain_id, v->vcpu_id, dev_id, vcol_id, event);
+ if ( vcol_id > (v->domain->max_vcpus + 1) || event > its_get_nr_events() )
+ return -EINVAL;
+
+ /* Enable validation later when device assignment is done */
+ vdev = find_vits_device(&v->domain->arch.vits->dev_root, dev_id);
+ if ( !vdev && !vdev->its_dev )
+ return -EINVAL;
+
+ if ( vits_get_vitt_entry(v->domain, dev_id, event, &entry) )
+ return -EINVAL;
+ entry.vcollection = vcol_id;
+ if ( vits_set_vitt_entry(v->domain, dev_id, event, &entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vgic_its_process_discard(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vits_device *vdev;
+ struct vitt entry;
+ uint32_t event, dev_id;
+
+ event = virt_cmd->discard.event;
+ dev_id = its_decode_devid(v->domain, virt_cmd);
+
+ DPRINTK("vITS:d%dv%d DISCARD: dev_id 0x%x id %d\n",
+ v->domain->domain_id, v->vcpu_id, dev_id, event);
+ if ( event > its_get_nr_events() )
+ return -EINVAL;
+
+ /* Validated later */
+ vdev = find_vits_device(&v->domain->arch.vits->dev_root, dev_id);
+ if ( !vdev && !vdev->its_dev )
+ return -EINVAL;
+
+ if ( vits_get_vitt_entry(v->domain, dev_id, event, &entry) )
+ return -EINVAL;
+ entry.valid = false;
+ if ( vits_set_vitt_entry(v->domain, dev_id, event, &entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vgic_its_process_inv(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* XXX: Ignored */
+ DPRINTK("vITS:d%dv%d INV: dev_id 0x%x id %d\n",
+ v->domain->domain_id, v->vcpu_id, virt_cmd->inv.devid,
+ virt_cmd->inv.event);
+
+ return 0;
+}
+
+static int vgic_its_process_clear(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* XXX: Ignored */
+ DPRINTK("vITS:d%dv%d CLEAR: dev_id 0x%x id %d\n",
+ v->domain->domain_id, v->vcpu_id, virt_cmd->clear.devid,
+ virt_cmd->clear.event);
+
+ return 0;
+}
+
+static int vgic_its_process_invall(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ /* XXX: Ignored */
+ DPRINTK("vITS:d%dv%d INVALL: vCID %d\n",
+ v->domain->domain_id, v->vcpu_id, virt_cmd->invall.col);
+
+ return 0;
+}
+
+static int vgic_its_process_int(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct vitt vitt_entry;
+ uint32_t event, dev_id, col_id;
+ event = virt_cmd->int_cmd.cmd;
+ dev_id = its_decode_devid(v->domain, virt_cmd);
+
+ DPRINTK("vITS:d%dv%d INT: Device 0x%x id %d\n",
+ v->domain->domain_id, v->vcpu_id, dev_id, event);
+ if ( event > its_get_nr_events() )
+ return -EINVAL;
+
+ if ( vits_get_vitt_entry(v->domain, dev_id, event, &vitt_entry) )
+ return -EINVAL;
+
+ if ( !vitt_entry.valid )
+ {
+ dprintk(XENLOG_G_ERR,
+ "vITS:d%dv%d INT CMD invalid, event %d for dev 0x%x\n",
+ v->domain->domain_id, v->vcpu_id, event, dev_id);
+ return -EINVAL;
+ }
+
+ col_id = vitt_entry.vcollection;
+ ASSERT(col_id < v->domain->max_vcpus);
+
+ vgic_vcpu_inject_irq(v->domain->vcpu[col_id], vitt_entry.vlpi);
+
+ return 0;
+}
+
+static int vgic_its_add_device(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct domain *d = v->domain;
+ struct vdevice_table dt_entry;
+ uint32_t devid = its_decode_devid(v->domain, virt_cmd);
+
+ DPRINTK("vITS:d%dv%dd Add device dev_id 0x%x vitt_ipa = 0x%lx size %d\n",
+ v->domain->domain_id, v->vcpu_id, devid, (u64)virt_cmd->mapd.itt << 8,
+ virt_cmd->mapd.size);
+ /* XXX: Validate devid with physical ITS driver ? */
+ if ( virt_cmd->mapd.valid )
+ {
+ /* itt field is 40 bit. extract 48 bit address by shifting */
+ dt_entry.vitt_ipa = virt_cmd->mapd.itt << 8;
+
+ dt_entry.vitt_size = (1 << (virt_cmd->mapd.size + 1)) * 8;
+ }
+ else
+ {
+ dt_entry.vitt_ipa = INVALID_PADDR;
+ dt_entry.vitt_size = 0;
+ }
+
+ if ( vits_set_vdevice_entry(d, devid, &dt_entry) )
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vgic_its_process_mapc(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ uint8_t vcol_id;
+ uint64_t vta = 0;
+
+ vcol_id = virt_cmd->mapc.col;
+ vta = virt_cmd->mapc.ta;
+
+ DPRINTK("vITS:d%dv%d MAPC: vCID %d vTA 0x%lx valid %d \n",
+ v->domain->domain_id, v->vcpu_id, vcol_id, vta,
+ virt_cmd->mapc.valid);
+ if ( vcol_id > (v->domain->max_vcpus + 1) || vta > v->domain->max_vcpus )
+ return -EINVAL;
+
+ if ( virt_cmd->mapc.valid )
+ v->domain->arch.vits->collections[vcol_id].target_address = vta;
+ else
+ v->domain->arch.vits->collections[vcol_id].target_address = ~0UL;
+
+ return 0;
+}
+
+static void vgic_its_update_read_ptr(struct vcpu *v, struct vgic_its *vits)
+{
+ vits->cmd_read = vits->cmd_write;
+}
+
+#ifdef DEBUG_ITS
+char *cmd_str[] = {
+ [GITS_CMD_MOVI] = "MOVI",
+ [GITS_CMD_INT] = "INT",
+ [GITS_CMD_CLEAR] = "CLEAR",
+ [GITS_CMD_SYNC] = "SYNC",
+ [GITS_CMD_MAPD] = "MAPD",
+ [GITS_CMD_MAPC] = "MAPC",
+ [GITS_CMD_MAPVI] = "MAPVI",
+ [GITS_CMD_MAPI] = "MAPI",
+ [GITS_CMD_INV] = "INV",
+ [GITS_CMD_INVALL] = "INVALL",
+ [GITS_CMD_MOVALL] = "MOVALL",
+ [GITS_CMD_DISCARD] = "DISCARD",
+ };
+#endif
+
+static int vgic_its_parse_its_command(struct vcpu *v, struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ uint8_t cmd = its_decode_cmd(virt_cmd);
+ int ret;
+
+#ifdef DEBUG_ITS
+ DPRINTK("vITS:d%dv%d Received cmd %s (0x%x)\n",
+ v->domain->domain_id, v->vcpu_id, cmd_str[cmd], cmd);
+ DPRINTK("Dump Virt cmd: ");
+ dump_cmd(virt_cmd);
+#endif
+
+ switch ( cmd )
+ {
+ case GITS_CMD_MAPD:
+ ret = vgic_its_add_device(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MAPC:
+ ret = vgic_its_process_mapc(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MAPI:
+ /* MAPI is same as MAPVI */
+ case GITS_CMD_MAPVI:
+ ret = vgic_its_process_mapvi(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_MOVI:
+ ret = vgic_its_process_movi(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_DISCARD:
+ ret = vgic_its_process_discard(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INV:
+ ret = vgic_its_process_inv(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INVALL:
+ ret = vgic_its_process_invall(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_INT:
+ ret = vgic_its_process_int(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_CLEAR:
+ ret = vgic_its_process_clear(v, vits, virt_cmd);
+ break;
+ case GITS_CMD_SYNC:
+ ret = vgic_its_process_sync(v, vits, virt_cmd);
+ break;
+ /*TODO: GITS_CMD_MOVALL not implemented */
+ default:
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d Unhandled command cmd %d\n",
+ v->domain->domain_id, v->vcpu_id, cmd);
+ return 1;
+ }
+
+ if ( ret )
+ {
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to handle cmd %d\n",
+ v->domain->domain_id, v->vcpu_id, cmd);
+ return 1;
+ }
+
+ return 0;
+}
+
+/* Called with its lock held */
+static int vgic_its_read_virt_cmd(struct vcpu *v,
+ struct vgic_its *vits,
+ its_cmd_block *virt_cmd)
+{
+ struct page_info * page;
+ uint64_t offset;
+ unsigned long maddr;
+ void *p;
+
+ /* CMD Q can be more than 1 page. Map only page that is required */
+ maddr = ((vits->cmd_base & 0xfffffffff000UL) +
+ vits->cmd_write_save) & PAGE_MASK;
+
+ DPRINTK("vITS:d%dv%d Mapping CMD Q maddr 0x%lx write_save 0x%lx \n",
+ v->domain->domain_id, v->vcpu_id, maddr, vits->cmd_write_save);
+
+ page = get_page_from_gfn(v->domain, paddr_to_pfn(maddr), NULL, P2M_ALLOC);
+ if ( page == NULL )
+ {
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to get command page\n",
+ v->domain->domain_id, v->vcpu_id);
+ return -EINVAL;
+ }
+
+ p = __map_domain_page(page);
+
+ /* Offset within the mapped 4K page to read */
+ offset = vits->cmd_write_save & 0xfff;
+
+ memcpy(virt_cmd, p + offset, sizeof(its_cmd_block));
+
+ /* No command queue is created by vits to check on Q full */
+ vits->cmd_write_save += 0x20;
+ if ( vits->cmd_write_save == vits->cmd_qsize )
+ {
+ DPRINTK("vITS:d%dv%d Reset write_save 0x%lx qsize 0x%lx \n",
+ v->domain->domain_id, v->vcpu_id, vits->cmd_write_save,
+ vits->cmd_qsize);
+ vits->cmd_write_save = 0x0;
+ }
+
+ unmap_domain_page(p);
+ put_page(page);
+
+ return 0;
+}
+
+int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
+{
+ its_cmd_block virt_cmd;
+
+ /* XXX: Currently we are processing one cmd at a time */
+ ASSERT(spin_is_locked(&vits->lock));
+
+ do {
+ if ( vgic_its_read_virt_cmd(v, vits, &virt_cmd) )
+ goto err;
+ if ( vgic_its_parse_its_command(v, vits, &virt_cmd) )
+ goto err;
+ } while ( vits->cmd_write != vits->cmd_write_save );
+
+ vits->cmd_write_save = vits->cmd_write;
+ DPRINTK("vITS:d%dv%d write_save 0x%lx write 0x%lx\n",
+ v->domain->domain_id, v->vcpu_id, vits->cmd_write_save,
+ vits->cmd_write);
+ /* XXX: Currently we are processing one cmd at a time */
+ vgic_its_update_read_ptr(v, vits);
+
+ dsb(ishst);
+
+ return 1;
+err:
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to process guest cmd\n",
+ v->domain->domain_id, v->vcpu_id);
+ return 0;
+}
+
/*
* Local variables:
* mode: C
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support
2015-06-22 12:01 ` [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
@ 2015-06-24 10:29 ` Julien Grall
2015-06-29 12:16 ` Ian Campbell
2015-06-29 12:18 ` Ian Campbell
2015-06-29 12:23 ` Ian Campbell
2 siblings, 1 reply; 88+ messages in thread
From: Julien Grall @ 2015-06-24 10:29 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add Virtual ITS command processing support to Virtual ITS driver
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/gic-v3-its.c | 7 +
> xen/arch/arm/vgic-v3-its.c | 393 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 400 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 535fc53..2a4fa97 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -89,6 +89,7 @@ struct its_node {
>
> #define ITS_ITT_ALIGN SZ_256
>
> +static u32 id_bits;
> static LIST_HEAD(its_nodes);
> static DEFINE_SPINLOCK(its_lock);
> static struct rdist_prop *gic_rdists;
> @@ -146,6 +147,11 @@ void dump_cmd(its_cmd_block *cmd)
> }
> #endif
>
> +u32 its_get_nr_events(void)
> +{
> + return (1 << id_bits);
> +}
> +
> /* RB-tree helpers for its_device */
> struct its_device * find_its_device(struct rb_root *root, u32 devid)
> {
> @@ -1044,6 +1050,7 @@ static int its_probe(struct dt_device_node *node)
> its->phys_size = its_size;
> typer = readl_relaxed(its_base + GITS_TYPER);
> its->ite_size = ((typer >> 4) & 0xf) + 1;
> + id_bits = GITS_TYPER_IDBITS(typer);
>
> its->cmd_base = xzalloc_bytes(ITS_CMD_QUEUE_SZ);
> if ( !its->cmd_base )
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index ea52a87..0671434 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -256,6 +256,399 @@ int remove_vits_device(struct rb_root *root, struct vits_device *dev)
> return 0;
> }
>
> +static int vgic_its_process_sync(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
virt_cmd is not modified, please use const.
> +{
> + /* XXX: Ignored */
IHMO, XXX means TODO which is not the case here.
> + DPRINTK("vITS:d%dv%d SYNC: ta 0x%x \n",
> + v->domain->domain_id, v->vcpu_id, virt_cmd->sync.ta);
You can use %pv rather than d%dv%d an directly pass the vcpu.
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_mapvi(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
Please use const for the virt_cmd.
> +{
> + struct vitt entry;
> + struct vits_device *vdev;
> + uint8_t vcol_id, cmd;
> + uint32_t vid, dev_id, event;
struct domain *d = v->domain for a better abstraction in the code.
> +
> + vcol_id = virt_cmd->mapvi.col;
> + vid = virt_cmd->mapvi.phy_id;
> + dev_id = its_decode_devid(v->domain, virt_cmd);
AFAIU, the its_decode_devid will return a physical devID... although you
function find_vits_device is working on virtual devID. This will also
not work on fake device. Did I miss something?
> + cmd = virt_cmd->mapvi.cmd;
> +
> + DPRINTK("vITS:d%dv%d MAPVI: dev_id 0x%x vcol_id %d vid %d \n",
> + v->domain->domain_id, v->vcpu_id, dev_id, vcol_id, vid);
> +
> + if ( vcol_id > (v->domain->max_vcpus + 1) || vid > its_get_nr_events() )
Checking the validity is pointless as a malicious guest can rewrite the
ITT. We only need to check it when the LPI is effectively injected.
> + return -EINVAL;
> +
> + /* XXX: Enable validation later */
What do you mean?
> + vdev = find_vits_device(&v->domain->arch.vits->dev_root, dev_id);
> + if ( !vdev && !vdev->its_dev )
> + return -EINVAL;
You deny the possibility to have fake device in the domain.
Anyway, this check is not necessary too.
> +
> + entry.valid = true;
> + entry.vcollection = vcol_id;
> + entry.vlpi = vid;
> +
> + if ( cmd == GITS_CMD_MAPI )
> + vits_set_vitt_entry(v->domain, dev_id, vid, &entry);
> + else
> + {
> + event = virt_cmd->mapvi.event;
> + if ( event > its_get_nr_events() )
You have hardcoded the number of event in the vGIC but you are using the
physical ITS to check the value.
IHMO, we should introduce a new field in the vITS to specify the number
of events supported by the domain. For DOM0 it will be equal to the
physical ITS.
But this check is also unnecessary.
> + return -EINVAL;
> +
> + vits_set_vitt_entry(v->domain, dev_id, event, &entry);
> + }
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_movi(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
virt_cmd is not modified, please use const.
> +{
> + struct vitt entry;
> + struct vits_device *vdev;
> + uint32_t dev_id, event;
> + uint8_t vcol_id;
struct domain *d = v->domain for a better abstraction in the code.
> +
> + dev_id = its_decode_devid(v->domain, virt_cmd);
> + vcol_id = virt_cmd->movi.col;
> + event = virt_cmd->movi.event;
> +
> + DPRINTK("vITS:d%dv%d MOVI: dev_id 0x%x vcol_id %d event %d\n",
> + v->domain->domain_id, v->vcpu_id, dev_id, vcol_id, event);
> + if ( vcol_id > (v->domain->max_vcpus + 1) || event > its_get_nr_events() )
> + return -EINVAL;
> +
> + /* Enable validation later when device assignment is done */
> + vdev = find_vits_device(&v->domain->arch.vits->dev_root, dev_id);
> + if ( !vdev && !vdev->its_dev )
> + return -EINVAL;
You deny the possibility to create the fake device. The only things you
care is dev_id is within the limit of devID supported by the domain. I
think the below helper take care of it.
> +
> + if ( vits_get_vitt_entry(v->domain, dev_id, event, &entry) )
> + return -EINVAL;
> + entry.vcollection = vcol_id;
> + if ( vits_set_vitt_entry(v->domain, dev_id, event, &entry) )
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_discard(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
Most of my comments on the previous functions are valid within
vgic_its_process_discard.
> + struct vits_device *vdev;
> + struct vitt entry;
> + uint32_t event, dev_id;
> +
> + event = virt_cmd->discard.event;
> + dev_id = its_decode_devid(v->domain, virt_cmd);
> +
> + DPRINTK("vITS:d%dv%d DISCARD: dev_id 0x%x id %d\n",
> + v->domain->domain_id, v->vcpu_id, dev_id, event);
> + if ( event > its_get_nr_events() )
> + return -EINVAL;
> +
> + /* Validated later */
> + vdev = find_vits_device(&v->domain->arch.vits->dev_root, dev_id);
> + if ( !vdev && !vdev->its_dev )
> + return -EINVAL;
> +
> + if ( vits_get_vitt_entry(v->domain, dev_id, event, &entry) )
> + return -EINVAL;
> + entry.valid = false;
> + if ( vits_set_vitt_entry(v->domain, dev_id, event, &entry) )
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_inv(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
ditto
> + /* XXX: Ignored */
> + DPRINTK("vITS:d%dv%d INV: dev_id 0x%x id %d\n",
> + v->domain->domain_id, v->vcpu_id, virt_cmd->inv.devid,
> + virt_cmd->inv.event);
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_clear(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
ditto
> + /* XXX: Ignored */
> + DPRINTK("vITS:d%dv%d CLEAR: dev_id 0x%x id %d\n",
> + v->domain->domain_id, v->vcpu_id, virt_cmd->clear.devid,
> + virt_cmd->clear.event);
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_invall(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
ditto
> + /* XXX: Ignored */
> + DPRINTK("vITS:d%dv%d INVALL: vCID %d\n",
> + v->domain->domain_id, v->vcpu_id, virt_cmd->invall.col);
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_int(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
ditto
> + struct vitt vitt_entry;
> + uint32_t event, dev_id, col_id;
Newline after the declaration of the variable.
> + event = virt_cmd->int_cmd.cmd;
> + dev_id = its_decode_devid(v->domain, virt_cmd);
> +
> + DPRINTK("vITS:d%dv%d INT: Device 0x%x id %d\n",
> + v->domain->domain_id, v->vcpu_id, dev_id, event);
> + if ( event > its_get_nr_events() )
> + return -EINVAL;
> +
> + if ( vits_get_vitt_entry(v->domain, dev_id, event, &vitt_entry) )
> + return -EINVAL;
> +
> + if ( !vitt_entry.valid )
> + {
> + dprintk(XENLOG_G_ERR,
> + "vITS:d%dv%d INT CMD invalid, event %d for dev 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, event, dev_id);
> + return -EINVAL;
> + }
> +
> + col_id = vitt_entry.vcollection;
> + ASSERT(col_id < v->domain->max_vcpus);
This need a proper check, a malicious guest can rewrite the VITT and
crash Xen.
> +
> + vgic_vcpu_inject_irq(v->domain->vcpu[col_id], vitt_entry.vlpi);
On the draft [1], the implementation suggest for INT was to call
vgic_vcpu_inject_lpi. Is there any issue to do it?
Also, you have to translate the col_id into to a VCPU ID.
> +
> + return 0;
> +}
> +
> +static int vgic_its_add_device(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
Most of my comment on the previous functions can apply here.
> +{
> + struct domain *d = v->domain;
> + struct vdevice_table dt_entry;
> + uint32_t devid = its_decode_devid(v->domain, virt_cmd);
> +
> + DPRINTK("vITS:d%dv%dd Add device dev_id 0x%x vitt_ipa = 0x%lx size %d\n",
> + v->domain->domain_id, v->vcpu_id, devid, (u64)virt_cmd->mapd.itt << 8,
> + virt_cmd->mapd.size);
> + /* XXX: Validate devid with physical ITS driver ? */
No need.
> + if ( virt_cmd->mapd.valid )
> + {
> + /* itt field is 40 bit. extract 48 bit address by shifting */
> + dt_entry.vitt_ipa = virt_cmd->mapd.itt << 8;
> +
> + dt_entry.vitt_size = (1 << (virt_cmd->mapd.size + 1)) * 8;
Where does come from the 8? Is it the size of the ITT? If so, you should
use the proper define.
> + }
> + else
> + {
> + dt_entry.vitt_ipa = INVALID_PADDR;
> + dt_entry.vitt_size = 0;
> + }
> +
> + if ( vits_set_vdevice_entry(d, devid, &dt_entry) )
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int vgic_its_process_mapc(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + uint8_t vcol_id;
> + uint64_t vta = 0;
> +
> + vcol_id = virt_cmd->mapc.col;
> + vta = virt_cmd->mapc.ta;
> +
> + DPRINTK("vITS:d%dv%d MAPC: vCID %d vTA 0x%lx valid %d \n",
> + v->domain->domain_id, v->vcpu_id, vcol_id, vta,
> + virt_cmd->mapc.valid);
> + if ( vcol_id > (v->domain->max_vcpus + 1) || vta > v->domain->max_vcpus )
> + return -EINVAL;
> +
> + if ( virt_cmd->mapc.valid )
> + v->domain->arch.vits->collections[vcol_id].target_address = vta;
> + else
> + v->domain->arch.vits->collections[vcol_id].target_address = ~0UL;
> +
> + return 0;
> +}
> +
> +static void vgic_its_update_read_ptr(struct vcpu *v, struct vgic_its *vits)
> +{
> + vits->cmd_read = vits->cmd_write;
> +}
> +
> +#ifdef DEBUG_ITS
> +char *cmd_str[] = {
> + [GITS_CMD_MOVI] = "MOVI",
> + [GITS_CMD_INT] = "INT",
> + [GITS_CMD_CLEAR] = "CLEAR",
> + [GITS_CMD_SYNC] = "SYNC",
> + [GITS_CMD_MAPD] = "MAPD",
> + [GITS_CMD_MAPC] = "MAPC",
> + [GITS_CMD_MAPVI] = "MAPVI",
> + [GITS_CMD_MAPI] = "MAPI",
> + [GITS_CMD_INV] = "INV",
> + [GITS_CMD_INVALL] = "INVALL",
> + [GITS_CMD_MOVALL] = "MOVALL",
> + [GITS_CMD_DISCARD] = "DISCARD",
> + };
> +#endif
> +
> +static int vgic_its_parse_its_command(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
const its_cmd_block *virt_cmd;
> +{
> + uint8_t cmd = its_decode_cmd(virt_cmd);
> + int ret;
> +
> +#ifdef DEBUG_ITS
> + DPRINTK("vITS:d%dv%d Received cmd %s (0x%x)\n",
> + v->domain->domain_id, v->vcpu_id, cmd_str[cmd], cmd);
%pv
> + DPRINTK("Dump Virt cmd: ");
> + dump_cmd(virt_cmd);
> +#endif
> +
> + switch ( cmd )
> + {
> + case GITS_CMD_MAPD:
> + ret = vgic_its_add_device(v, vits, virt_cmd);
> + break;
> + case GITS_CMD_MAPC:
> + ret = vgic_its_process_mapc(v, vits, virt_cmd);
> + break;
> + case GITS_CMD_MAPI:
> + /* MAPI is same as MAPVI */
> + case GITS_CMD_MAPVI:
> + ret = vgic_its_process_mapvi(v, vits, virt_cmd);
> + break;
> + case GITS_CMD_MOVI:
> + ret = vgic_its_process_movi(v, vits, virt_cmd);
> + break;
> + case GITS_CMD_DISCARD:
> + ret = vgic_its_process_discard(v, vits, virt_cmd);
> + break;
> + case GITS_CMD_INV:
> + ret = vgic_its_process_inv(v, vits, virt_cmd);
> + break;
> + case GITS_CMD_INVALL:
> + ret = vgic_its_process_invall(v, vits, virt_cmd);
> + break;
> + case GITS_CMD_INT:
> + ret = vgic_its_process_int(v, vits, virt_cmd);
> + break;
> + case GITS_CMD_CLEAR:
> + ret = vgic_its_process_clear(v, vits, virt_cmd);
> + break;
> + case GITS_CMD_SYNC:
> + ret = vgic_its_process_sync(v, vits, virt_cmd);
> + break;
> + /*TODO: GITS_CMD_MOVALL not implemented */
I think we have to ignore it as we do for SYNC, INV, INVALL.
> + default:
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d Unhandled command cmd %d\n",
> + v->domain->domain_id, v->vcpu_id, cmd);
> + return 1;
> + }
> +
> + if ( ret )
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to handle cmd %d\n",
> + v->domain->domain_id, v->vcpu_id, cmd);
%pv
> + return 1;
> + }
> +
> + return 0;
> +}
> +
> +/* Called with its lock held */
Please add an ASSERT to validate this assumption
> +static int vgic_its_read_virt_cmd(struct vcpu *v,
> + struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct page_info * page;
struct page_info *page;
> + uint64_t offset;
> + unsigned long maddr;
> + void *p;
> +
> + /* CMD Q can be more than 1 page. Map only page that is required */
> + maddr = ((vits->cmd_base & 0xfffffffff000UL) +
> + vits->cmd_write_save) & PAGE_MASK;
Define a mask would make the code clearer.
Also I don't see any check on GITS_CBASER.Valid in this code and in the
register emulation.
If GITS_CBASER.Valid = 0, command should be ignore.
> +
> + DPRINTK("vITS:d%dv%d Mapping CMD Q maddr 0x%lx write_save 0x%lx \n",
> + v->domain->domain_id, v->vcpu_id, maddr, vits->cmd_write_save);
> +
> + page = get_page_from_gfn(v->domain, paddr_to_pfn(maddr), NULL, P2M_ALLOC);
> + if ( page == NULL )
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to get command page\n",
> + v->domain->domain_id, v->vcpu_id);
> + return -EINVAL;
> + }
> +
> + p = __map_domain_page(page);
> +
> + /* Offset within the mapped 4K page to read */
> + offset = vits->cmd_write_save & 0xfff;
> +
> + memcpy(virt_cmd, p + offset, sizeof(its_cmd_block));
> +
> + /* No command queue is created by vits to check on Q full */
> + vits->cmd_write_save += 0x20;
0x20 is confusing. I would use sizeof (struct its_cmd_block) or add a
command to explain the 0x20.
> + if ( vits->cmd_write_save == vits->cmd_qsize )
> + {
> + DPRINTK("vITS:d%dv%d Reset write_save 0x%lx qsize 0x%lx \n",
> + v->domain->domain_id, v->vcpu_id, vits->cmd_write_save,
> + vits->cmd_qsize);
> + vits->cmd_write_save = 0x0;
> + }
> +
> + unmap_domain_page(p);
> + put_page(page);
As suggested above, I would create an helper to read/write based on an IPA.
> +
> + return 0;
> +}
> +
> +int vgic_its_process_cmd(struct vcpu *v, struct vgic_its *vits)
Missing a static
> +{
> + its_cmd_block virt_cmd;
> +
> + /* XXX: Currently we are processing one cmd at a time */
This comment seems wrong. You are handling multiple commands at the same
time.
> + ASSERT(spin_is_locked(&vits->lock));
> +
> + do {
> + if ( vgic_its_read_virt_cmd(v, vits, &virt_cmd) )
> + goto err;
> + if ( vgic_its_parse_its_command(v, vits, &virt_cmd) )
> + goto err;
> + } while ( vits->cmd_write != vits->cmd_write_save );
> +
> + vits->cmd_write_save = vits->cmd_write;
Why this line? At the end of the loop cmd_write_save is equals to
write_save. And technically, vits->cmd_write should be updated after
each command.
> + DPRINTK("vITS:d%dv%d write_save 0x%lx write 0x%lx\n",
> + v->domain->domain_id, v->vcpu_id, vits->cmd_write_save,
> + vits->cmd_write);
> + /* XXX: Currently we are processing one cmd at a time */
> + vgic_its_update_read_ptr(v, vits);
Ditto for the command.
> +
> + dsb(ishst);
Why the dsb?
> +
> + return 1;
> +err:
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to process guest cmd\n",
> + v->domain->domain_id, v->vcpu_id);
> + return 0;
returning 0 here means that we will inject a data abort to the guest
pointing to GITS_CWRITER. Although I'm not sure this what we want.
> +}
> +
> /*
> * Local variables:
> * mode: C
>
Regards,
[1]
http://xenbits.xensource.com/people/ianc/vits/draftF.html#int-generate-an-interrupt
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support
2015-06-24 10:29 ` Julien Grall
@ 2015-06-29 12:16 ` Ian Campbell
0 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 12:16 UTC (permalink / raw)
To: Julien Grall
Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
tim, xen-devel, stefano.stabellini, manish.jaggi
On Wed, 2015-06-24 at 11:29 +0100, Julien Grall wrote:
> > + if ( virt_cmd->mapd.valid )
> > + {
> > + /* itt field is 40 bit. extract 48 bit address by shifting */
> > + dt_entry.vitt_ipa = virt_cmd->mapd.itt << 8;
> > +
> > + dt_entry.vitt_size = (1 << (virt_cmd->mapd.size + 1)) * 8;
>
> Where does come from the 8? Is it the size of the ITT? If so, you should
> use the proper define.
sizeof(struct vitt) is what was meant, I think?
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support
2015-06-22 12:01 ` [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
2015-06-24 10:29 ` Julien Grall
@ 2015-06-29 12:18 ` Ian Campbell
2015-06-29 12:23 ` Ian Campbell
2 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 12:18 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
[...]
> + /* XXX: Enable validation later */
> + vdev = find_vits_device(&v->domain->arch.vits->dev_root, dev_id);
> + if ( !vdev && !vdev->its_dev )
> + return -EINVAL;
vdev is unused I think. Which is good because according to the draft
spec it should never be needed here. I believe that should be the case
for all vgic_its_process_*.
Avoiding this lookup was part of the point of the major rework done in
draftC of the spec.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support
2015-06-22 12:01 ` [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
2015-06-24 10:29 ` Julien Grall
2015-06-29 12:18 ` Ian Campbell
@ 2015-06-29 12:23 ` Ian Campbell
2015-07-03 6:50 ` Vijay Kilari
2 siblings, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 12:23 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> +static int vgic_its_process_mapvi(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + struct vitt entry;
> + struct vits_device *vdev;
> + uint8_t vcol_id, cmd;
> + uint32_t vid, dev_id, event;
> +
> + vcol_id = virt_cmd->mapvi.col;
> + vid = virt_cmd->mapvi.phy_id;
> + dev_id = its_decode_devid(v->domain, virt_cmd);
If you used the "union its_cmd" I proposed earlier for the virt_cmd
argument then this would just be "virt_command->mapvi.devid".
[...]
> +static int vgic_its_parse_its_command(struct vcpu *v, struct vgic_its *vits,
> + its_cmd_block *virt_cmd)
> +{
> + uint8_t cmd = its_decode_cmd(virt_cmd);
and this would be cirt_cmd->hdr.cmd.
[...]
> + memcpy(virt_cmd, p + offset, sizeof(its_cmd_block));
and this would be
memcpy(&virt_cmd.bits[0], p+offset, sizeof(union its_cmd));
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support
2015-06-29 12:23 ` Ian Campbell
@ 2015-07-03 6:50 ` Vijay Kilari
2015-07-03 8:41 ` Ian Campbell
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-07-03 6:50 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Mon, Jun 29, 2015 at 5:53 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>> +static int vgic_its_process_mapvi(struct vcpu *v, struct vgic_its *vits,
>> + its_cmd_block *virt_cmd)
>> +{
>> + struct vitt entry;
>> + struct vits_device *vdev;
>> + uint8_t vcol_id, cmd;
>> + uint32_t vid, dev_id, event;
>> +
>> + vcol_id = virt_cmd->mapvi.col;
>> + vid = virt_cmd->mapvi.phy_id;
>> + dev_id = its_decode_devid(v->domain, virt_cmd);
>
> If you used the "union its_cmd" I proposed earlier for the virt_cmd
> argument then this would just be "virt_command->mapvi.devid".
its_decode_devid() is used to get physical devid for a given
virtual device id. I have to use virt_command->mapvi.devid.
If vitt entries are made base on physical dev id then
- This helps where in if interrupt is received we can extract
physical dev id from
the its_device structure held by irq_desc. With this physical device id we can
search vitt entry to get collection/vcpu and vlpi information.
However fake devices cannot have physical device id. We cannot search vitt
entries. Esp INT command has only virtual dev id. So I think this
approach is not
viable.
I propose:
Manage vitt entries based on virtual dev id and manage domain specific
RB-tree (which holds list of devices assigned to domain) with physical devid.
Now on receiving interrupt, will search for RB-tree based on physical dev id and
get virtual dev id. With virtual dev id search vitt entries to get
vcpu/collection & vlpi
info.
Also for fake devices, we search vitt entries with vdevice id for INT command.
Any suggestions?
Regards
Vijay
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support
2015-07-03 6:50 ` Vijay Kilari
@ 2015-07-03 8:41 ` Ian Campbell
0 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-07-03 8:41 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Fri, 2015-07-03 at 12:20 +0530, Vijay Kilari wrote:
> On Mon, Jun 29, 2015 at 5:53 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> > On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> >> +static int vgic_its_process_mapvi(struct vcpu *v, struct vgic_its *vits,
> >> + its_cmd_block *virt_cmd)
> >> +{
> >> + struct vitt entry;
> >> + struct vits_device *vdev;
> >> + uint8_t vcol_id, cmd;
> >> + uint32_t vid, dev_id, event;
> >> +
> >> + vcol_id = virt_cmd->mapvi.col;
> >> + vid = virt_cmd->mapvi.phy_id;
> >> + dev_id = its_decode_devid(v->domain, virt_cmd);
> >
> > If you used the "union its_cmd" I proposed earlier for the virt_cmd
> > argument then this would just be "virt_command->mapvi.devid".
>
> its_decode_devid() is used to get physical devid for a given
> virtual device id. I have to use virt_command->mapvi.devid.
>
> If vitt entries are made base on physical dev id then
>
> - This helps where in if interrupt is received we can extract
> physical dev id from
> the its_device structure held by irq_desc. With this physical device id we can
> search vitt entry to get collection/vcpu and vlpi information.
http://xenbits.xen.org/people/ianc/vits/draftG.html#virtual-lpi-injection shows that there is no need for the physical device id when handling an pLPI and turning it into a vPLI injection.
In the pLPI case the vlpi comes out of the irq_desc and
vgic_vcpu_inject_lpi only requires a vpli, not physical, which it then
uses as indexes into the per-domain irq_pending
(http://xenbits.xen.org/people/ianc/vits/draftG.html#per-domain-struct-pending_irq-for-vlpis)
See also
http://xenbits.xen.org/people/ianc/vits/draftG.html#mapvi-map-an-input-identifier-to-a-physical-interrupt-and-an-interrupt-collection.
which describes how a MAPVI from the guest needs to be handled.
> However fake devices cannot have physical device id. We cannot search vitt
> entries. Esp INT command has only virtual dev id. So I think this
> approach is not
> viable.
As shown above vgic_vcpu_inject_lpi only takes a vpli param, so the fact
there that there is no physical device id for an INT command on a fake
command doesn't matter.
> I propose:
> Manage vitt entries based on virtual dev id and manage domain specific
> RB-tree (which holds list of devices assigned to domain) with physical devid.
>
> Now on receiving interrupt, will search for RB-tree based on physical dev id and
> get virtual dev id. With virtual dev id search vitt entries to get
> vcpu/collection & vlpi
> info.
The design document should shows there is no need for this.
You are very welcome to propose an alternative design, but I'm afraid
that in that case I'm going to have to ask you to produce a
comprehensive document covering all aspects of your proposed design,
either from scratch or by modifying the existing draft to account for
the consequences of your proposed changes across the board, i.e. it
needs remain a coherent design across the board after your changes.
The fact that you are proposing an RB-tree looked during interrupt
however is a problematic aspect of your proposal, in the existing design
has been arranged such that this is not necessary.
> Also for fake devices, we search vitt entries with vdevice id for INT command.
>
> Any suggestions?
>
> Regards
> Vijay
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (8 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 09/18] xen/arm: ITS: Add virtual ITS commands support vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-24 11:38 ` Julien Grall
2015-06-29 12:29 ` Ian Campbell
2015-06-22 12:01 ` [RFC PATCH v3 11/18] xen/arm: ITS: Add GITS registers emulation vijay.kilari
` (10 subsequent siblings)
20 siblings, 2 replies; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add APIs to add devices to RB-tree, assign and remove
devices to domain.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3-its.c | 246 ++++++++++++++++++++++++++++++++++++++++-
xen/include/asm-arm/gic-its.h | 4 +-
2 files changed, 246 insertions(+), 4 deletions(-)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 2a4fa97..4471669 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -94,6 +94,7 @@ static LIST_HEAD(its_nodes);
static DEFINE_SPINLOCK(its_lock);
static struct rdist_prop *gic_rdists;
static struct rb_root rb_its_dev;
+static DEFINE_SPINLOCK(rb_its_dev_lock);
#define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
#define gic_data_rdist_rd_base() (per_cpu(rdist, smp_processor_id()).rbase)
@@ -152,6 +153,21 @@ u32 its_get_nr_events(void)
return (1 << id_bits);
}
+static struct its_node * its_get_phys_node(u32 dev_id)
+{
+ struct its_node *its;
+
+ /* TODO: For now return ITS0 node.
+ * Need Query PCI helper function to get on which
+ * ITS node the device is attached
+ */
+ list_for_each_entry(its, &its_nodes, entry)
+ {
+ return its;
+ }
+
+ return NULL;
+}
/* RB-tree helpers for its_device */
struct its_device * find_its_device(struct rb_root *root, u32 devid)
{
@@ -459,7 +475,7 @@ void its_send_inv(struct its_device *dev, struct its_collection *col,
its_send_single_command(dev->its, its_build_inv_cmd, &desc);
}
-void its_send_mapd(struct its_device *dev, int valid)
+static void its_send_mapd(struct its_device *dev, int valid)
{
struct its_cmd_desc desc;
@@ -493,7 +509,7 @@ void its_send_mapvi(struct its_device *dev, struct its_collection *col,
its_send_single_command(dev->its, its_build_mapvi_cmd, &desc);
}
-void its_send_movi(struct its_device *dev, struct its_collection *col,
+static void its_send_movi(struct its_device *dev, struct its_collection *col,
u32 event)
{
struct its_cmd_desc desc;
@@ -596,7 +612,7 @@ int its_lpi_init(u32 id_bits)
return 0;
}
-unsigned long *its_lpi_alloc_chunks(int nirqs, int *base, int *nr_ids)
+static unsigned long *its_lpi_alloc_chunks(int nirqs, int *base, int *nr_ids)
{
unsigned long *bitmap = NULL;
int chunk_id;
@@ -636,6 +652,230 @@ out:
return bitmap;
}
+static void its_lpi_free(unsigned long *bitmap, int base, int nr_ids)
+{
+ int lpi;
+
+ spin_lock(&lpi_lock);
+
+ for ( lpi = base; lpi < (base + nr_ids); lpi += IRQS_PER_CHUNK )
+ {
+ int chunk = its_lpi_to_chunk(lpi);
+
+ BUG_ON(chunk > lpi_chunks);
+ if ( test_bit(chunk, lpi_bitmap) )
+ clear_bit(chunk, lpi_bitmap);
+ else
+ its_err("Bad LPI chunk %d\n", chunk);
+ }
+
+ spin_unlock(&lpi_lock);
+
+ xfree(bitmap);
+}
+
+static int its_alloc_device_irq(struct its_device *dev, u32 *hwirq)
+{
+ int idx;
+
+ idx = find_first_zero_bit(dev->lpi_map, dev->nr_lpis);
+ if ( idx == dev->nr_lpis )
+ return -ENOSPC;
+
+ *hwirq = dev->lpi_base + idx;
+ set_bit(idx, dev->lpi_map);
+
+ return 0;
+}
+
+static u32 its_get_plpi(struct its_device *dev, u32 event)
+{
+ ASSERT(event < dev->nr_lpis);
+ return dev->lpi_base + event;
+}
+
+/* Device assignment. Should be called from pci_device_add */
+int its_add_device(struct domain *d, u32 devid)
+{
+ struct its_device *dev;
+ unsigned long *lpi_map;
+ void *itt;
+ int lpi_base, nr_lpis, sz;
+ u32 i, nr_ites, plpi, nr_cpus;
+ struct its_collection *col;
+
+ spin_lock(&rb_its_dev_lock);
+ dev = find_its_device(&rb_its_dev, devid);
+ if ( dev )
+ {
+ spin_unlock(&rb_its_dev_lock);
+ dprintk(XENLOG_G_ERR, "ITS:d%dv%d Device already exists dev 0x%x\n",
+ d->domain_id, current->vcpu_id, dev->device_id);
+ return -EEXIST;
+ }
+ spin_unlock(&rb_its_dev_lock);
+
+ DPRINTK("ITS:d%dv%d Add device devid 0x%x\n",
+ d->domain_id, current->vcpu_id, devid);
+
+ dev = xzalloc(struct its_device);
+ if ( dev == NULL )
+ return -ENOMEM;
+
+ dev->its = its_get_phys_node(devid);
+ /* TODO: Use pci helper to get nvecs */
+ nr_ites = 64;
+ sz = nr_ites * dev->its->ite_size;
+ sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
+ itt = xzalloc_bytes(sz);
+ if ( !itt )
+ return -ENOMEM;
+
+ lpi_map = its_lpi_alloc_chunks(nr_ites, &lpi_base, &nr_lpis);
+ if ( !lpi_map || (nr_lpis != nr_ites) )
+ {
+ xfree(dev);
+ xfree(itt);
+ xfree(lpi_map);
+ return -EINVAL;
+ }
+
+ dev->itt_addr = (u64)itt;
+ dev->nr_ites = nr_ites;
+ dev->lpi_map = lpi_map;
+ dev->lpi_base = lpi_base;
+ /* nr_lpis should be always equal to nvecs */
+ dev->nr_lpis = nr_lpis;
+ dev->device_id = devid;
+
+ DPRINTK("ITS:d%dv%d Adding Device with id 0x%x nvecs %d lpi_base 0x%x\n",
+ d->domain_id, current->vcpu_id, dev->device_id,
+ dev->nr_lpis, dev->lpi_base);
+
+ /* Map device to its ITT */
+ its_send_mapd(dev, 1);
+
+ nr_cpus = num_online_cpus();
+ for ( i = 0; i < nr_ites; i++ )
+ {
+ /* Reserve pLPI */
+ if ( its_alloc_device_irq(dev, &plpi) )
+ return -ENOSPC;
+
+ /* For each pLPI send MAPVI command */
+ col = &dev->its->collections[(i % nr_cpus)];
+ its_send_mapvi(dev, col, plpi, i);
+ }
+
+ spin_lock(&rb_its_dev_lock);
+ if ( insert_its_device(&rb_its_dev, dev) )
+ {
+ spin_unlock(&rb_its_dev_lock);
+ return -EINVAL;
+ }
+ spin_unlock(&rb_its_dev_lock);
+
+ return 0;
+}
+
+int its_assign_device(struct domain *d, u32 vdevid)
+{
+ struct its_device *pdev;
+ struct vits_device *vdev;
+ struct irq_desc *desc;
+ u32 plpi, i, pdevid;
+
+ DPRINTK("ITS:d%dv%d Assign request for virtual device 0x%x\n",
+ d->domain_id, current->vcpu_id, vdevid);
+ vdev = find_vits_device(&d->arch.vits->dev_root, vdevid);
+ if ( vdev )
+ return -ENODEV;
+
+ /* TODO: Use PCI helper to convert vDevid to pDevid */
+ pdevid = vdevid;
+ spin_lock(&rb_its_dev_lock);
+ pdev = find_its_device(&rb_its_dev, pdevid);
+ if ( !pdev )
+ {
+ spin_unlock(&rb_its_dev_lock);
+ return -ENODEV;
+ }
+ spin_unlock(&rb_its_dev_lock);
+
+ vdev = xzalloc_bytes(sizeof(struct vits_device));
+ if ( !pdev )
+ return -ENOMEM;
+
+ vdev->its_dev = pdev;
+ vdev->vdevid = vdevid;
+ vdev->pdevid = pdevid;
+
+ /* Insert to domains' list */
+ if ( insert_vits_device(&d->arch.vits->dev_root, vdev) )
+ return -EINVAL;
+
+ DPRINTK("ITS:d%dv%d Assign pdevid 0x%x with nvecs %d\n",
+ d->domain_id, current->vcpu_id, pdev->device_id, pdev->nr_lpis);
+
+ /* XXX: Event id always starts from 0? */
+ for ( i = 0; i < pdev->nr_lpis; i++ )
+ {
+ plpi = its_get_plpi(pdev, i);
+ route_irq_to_guest(d, i, plpi, "LPI");
+ /* Enable all the event in LPI configuration table */
+ desc = irq_to_desc(plpi);
+ set_irq_device(desc, pdev);
+ lpi_set_config(desc, 1);
+ }
+
+ return 0;
+}
+
+int its_detach_device(struct domain *d, u32 vdevid)
+{
+ struct its_device *pdev;
+ struct vits_device *vdev;
+ struct irq_desc *desc;
+ u32 plpi, i, pdevid;
+
+ DPRINTK("ITS:d%dv%d Detach request for virtual device 0x%x\n",
+ d->domain_id, current->vcpu_id, vdevid);
+ vdev = find_vits_device(&d->arch.vits->dev_root, vdevid);
+ if ( !vdev )
+ return -EINVAL;
+
+ /* XXX: convert vDevid to pDevid ? */
+ pdevid = vdevid;
+ spin_lock(&rb_its_dev_lock);
+ pdev = find_its_device(&rb_its_dev, pdevid);
+ if ( !pdev )
+ {
+ spin_unlock(&rb_its_dev_lock);
+ return -EINVAL;
+ }
+ spin_unlock(&rb_its_dev_lock);
+
+ DPRINTK("ITS:d%dv%d Detach pdevid 0x%x with nvecs %d\n",
+ d->domain_id, current->vcpu_id, pdev->device_id, pdev->nr_lpis);
+ /* Remove vits_device from domain list */
+ remove_vits_device(&d->arch.vits->dev_root, vdev);
+ xfree(vdev);
+
+ for ( i = 0; i < pdev->nr_lpis; i++ )
+ {
+ plpi = its_get_plpi(pdev, i);
+ /* XXX: Event id always starts from 0 */
+ release_irq(plpi, d);
+ /* Disable all the event in LPI configuration table */
+ desc = irq_to_desc(plpi);
+ set_irq_device(desc, NULL);
+ lpi_set_config(desc, 0);
+ }
+
+ its_lpi_free(pdev->lpi_map, pdev->lpi_base, pdev->nr_lpis);
+
+ return 0;
+}
/*
* We allocate 64kB for PROPBASE. That gives us at most 64K LPIs to
* deal with (one configuration byte per interrupt). PENDBASE has to
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index a1099a1..8f898a6 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -263,7 +263,9 @@ static inline uint32_t its_decode_devid(struct domain *d, its_cmd_block *cmd)
void its_set_affinity(struct irq_desc *desc, int cpu);
void lpi_set_config(struct irq_desc *desc, int enable);
uint32_t its_get_nr_events(void);
-
+struct vits_device * find_vits_device(struct rb_root *root, uint32_t devid);
+int insert_vits_device(struct rb_root *root, struct vits_device *dev);
+int remove_vits_device(struct rb_root *root, struct vits_device *dev);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device
2015-06-22 12:01 ` [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
@ 2015-06-24 11:38 ` Julien Grall
2015-06-29 12:25 ` Ian Campbell
2015-06-29 12:29 ` Ian Campbell
1 sibling, 1 reply; 88+ messages in thread
From: Julien Grall @ 2015-06-24 11:38 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add APIs to add devices to RB-tree, assign and remove
> devices to domain.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/gic-v3-its.c | 246 ++++++++++++++++++++++++++++++++++++++++-
> xen/include/asm-arm/gic-its.h | 4 +-
> 2 files changed, 246 insertions(+), 4 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 2a4fa97..4471669 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -94,6 +94,7 @@ static LIST_HEAD(its_nodes);
> static DEFINE_SPINLOCK(its_lock);
> static struct rdist_prop *gic_rdists;
> static struct rb_root rb_its_dev;
> +static DEFINE_SPINLOCK(rb_its_dev_lock);
Can't you use the its_lock to handle locking for the rb_its_dev?
>
> #define gic_data_rdist() (per_cpu(rdist, smp_processor_id()))
> #define gic_data_rdist_rd_base() (per_cpu(rdist, smp_processor_id()).rbase)
> @@ -152,6 +153,21 @@ u32 its_get_nr_events(void)
> return (1 << id_bits);
> }
>
> +static struct its_node * its_get_phys_node(u32 dev_id)
> +{
> + struct its_node *its;
> +
> + /* TODO: For now return ITS0 node.
> + * Need Query PCI helper function to get on which
> + * ITS node the device is attached
> + */
> + list_for_each_entry(its, &its_nodes, entry)
> + {
> + return its;
> + }
> +
> + return NULL;
> +}
> /* RB-tree helpers for its_device */
> struct its_device * find_its_device(struct rb_root *root, u32 devid)
> {
> @@ -459,7 +475,7 @@ void its_send_inv(struct its_device *dev, struct its_collection *col,
> its_send_single_command(dev->its, its_build_inv_cmd, &desc);
> }
>
> -void its_send_mapd(struct its_device *dev, int valid)
> +static void its_send_mapd(struct its_device *dev, int valid)
I would prefer to see this static where the function has been introduced
and delay the compilation until the end.
> {
> struct its_cmd_desc desc;
>
> @@ -493,7 +509,7 @@ void its_send_mapvi(struct its_device *dev, struct its_collection *col,
> its_send_single_command(dev->its, its_build_mapvi_cmd, &desc);
> }
>
> -void its_send_movi(struct its_device *dev, struct its_collection *col,
> +static void its_send_movi(struct its_device *dev, struct its_collection *col,
> u32 event)
Ditto
> {
> struct its_cmd_desc desc;
> @@ -596,7 +612,7 @@ int its_lpi_init(u32 id_bits)
> return 0;
> }
>
> -unsigned long *its_lpi_alloc_chunks(int nirqs, int *base, int *nr_ids)
> +static unsigned long *its_lpi_alloc_chunks(int nirqs, int *base, int *nr_ids)
Ditto
> {
> unsigned long *bitmap = NULL;
> int chunk_id;
> @@ -636,6 +652,230 @@ out:
> return bitmap;
> }
>
> +static void its_lpi_free(unsigned long *bitmap, int base, int nr_ids)
> +{
> + int lpi;
> +
> + spin_lock(&lpi_lock);
> +
> + for ( lpi = base; lpi < (base + nr_ids); lpi += IRQS_PER_CHUNK )
> + {
> + int chunk = its_lpi_to_chunk(lpi);
> +
> + BUG_ON(chunk > lpi_chunks);
> + if ( test_bit(chunk, lpi_bitmap) )
> + clear_bit(chunk, lpi_bitmap);
> + else
> + its_err("Bad LPI chunk %d\n", chunk);
> + }
> +
> + spin_unlock(&lpi_lock);
> +
> + xfree(bitmap);
> +}
> +
> +static int its_alloc_device_irq(struct its_device *dev, u32 *hwirq)
> +{
> + int idx;
> +
> + idx = find_first_zero_bit(dev->lpi_map, dev->nr_lpis);
> + if ( idx == dev->nr_lpis )
> + return -ENOSPC;
> +
> + *hwirq = dev->lpi_base + idx;
You could use its_get_plpi here.
> + set_bit(idx, dev->lpi_map);
> +
> + return 0;
> +}
> +
> +static u32 its_get_plpi(struct its_device *dev, u32 event)
> +{
> + ASSERT(event < dev->nr_lpis);
> + return dev->lpi_base + event;
> +}
> +
> +/* Device assignment. Should be called from pci_device_add */
I guess you mean PHYSDEVOP_pci_device_add?
> +int its_add_device(struct domain *d, u32 devid)
Why do you pass a domain in parameter? This function only adds a device
into the ITS, there is nothing related to a specific domain.
For more abstraction, I would create an helper which take a struct
device in parameter and retrieving all the necessary informations
(devid, number of MSI...) and then call this function.
> +{
> + struct its_device *dev;
> + unsigned long *lpi_map;
> + void *itt;
> + int lpi_base, nr_lpis, sz;
> + u32 i, nr_ites, plpi, nr_cpus;
> + struct its_collection *col;
> +
> + spin_lock(&rb_its_dev_lock);
> + dev = find_its_device(&rb_its_dev, devid);
> + if ( dev )
This check is not useful. As you release the lock before adding the
device someone else can have time to add the device in the RB-tree.
> + {
> + spin_unlock(&rb_its_dev_lock);
> + dprintk(XENLOG_G_ERR, "ITS:d%dv%d Device already exists dev 0x%x\n",
> + d->domain_id, current->vcpu_id, dev->device_id);
> + return -EEXIST;
> + }
> + spin_unlock(&rb_its_dev_lock);
> +
> + DPRINTK("ITS:d%dv%d Add device devid 0x%x\n",
> + d->domain_id, current->vcpu_id, devid);
> +
> + dev = xzalloc(struct its_device);
> + if ( dev == NULL )
> + return -ENOMEM;
> +
> + dev->its = its_get_phys_node(devid);
> + /* TODO: Use pci helper to get nvecs */
> + nr_ites = 64;
How did you choose this value?
> + sz = nr_ites * dev->its->ite_size;
> + sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
> + itt = xzalloc_bytes(sz);
> + if ( !itt )
> + return -ENOMEM;
You forgot to free dev if we fail to allocate itt.
As Linux does, you can do the allocation check in one place. But you
will have to create a local variable for the ITS.
> +
> + lpi_map = its_lpi_alloc_chunks(nr_ites, &lpi_base, &nr_lpis);
> + if ( !lpi_map || (nr_lpis != nr_ites) )
The check "nr_lpis != nr_ites" is here to ensure that
its_lpi_alloc_chuncks allocate the right number of LPI, right? If so,
given this is the only place you call its_lpi_alloc_chunks, why don't
you put the check within the function? It would avoid one parameter.
> + {
> + xfree(dev);
> + xfree(itt);
> + xfree(lpi_map);
> + return -EINVAL;
> + }
> +
> + dev->itt_addr = (u64)itt;
The cast is not necessary.
> + dev->nr_ites = nr_ites;
> + dev->lpi_map = lpi_map;
> + dev->lpi_base = lpi_base;
> + /* nr_lpis should be always equal to nvecs */
nvecs doesn't have any meaning within this function. Do you mean nr_ites?
> + dev->nr_lpis = nr_lpis;
> + dev->device_id = devid;
> +
> + DPRINTK("ITS:d%dv%d Adding Device with id 0x%x nvecs %d lpi_base 0x%x\n",
> + d->domain_id, current->vcpu_id, dev->device_id,
> + dev->nr_lpis, dev->lpi_base);
> +
> + /* Map device to its ITT */
> + its_send_mapd(dev, 1);
> +
> + nr_cpus = num_online_cpus();
> + for ( i = 0; i < nr_ites; i++ )
> + {
> + /* Reserve pLPI */
> + if ( its_alloc_device_irq(dev, &plpi) )
> + return -ENOSPC;
> +
> + /* For each pLPI send MAPVI command */
> + col = &dev->its->collections[(i % nr_cpus)];
Hmmmm.... In patch #7 you are using a different way to get the
collection for the LPIs. Can you introduce an helper to get the collection?
> + its_send_mapvi(dev, col, plpi, i);
> + }
> +
> + spin_lock(&rb_its_dev_lock);
> + if ( insert_its_device(&rb_its_dev, dev) )
You have to undo the mapd, mapvi, and free all the allocated memory if
the insertion fail.
You may want to rework the code in to avoid taking care of that.
> + {
> + spin_unlock(&rb_its_dev_lock);
> + return -EINVAL;
> + }
> + spin_unlock(&rb_its_dev_lock);
> +
> + return 0;
> +}
> +
> +int its_assign_device(struct domain *d, u32 vdevid)
> +{
> + struct its_device *pdev;
> + struct vits_device *vdev;
> + struct irq_desc *desc;
> + u32 plpi, i, pdevid;
> +
> + DPRINTK("ITS:d%dv%d Assign request for virtual device 0x%x\n",
> + d->domain_id, current->vcpu_id, vdevid);
Please don't mix d and current. The first one point to the domain where
the device will be assigned, the latter is the vcpu which issue the
hypercall to assign.
You don't need to print current here.
> + vdev = find_vits_device(&d->arch.vits->dev_root, vdevid);
> + if ( vdev )
> + return -ENODEV;
ENODEV mean there is not device. In this case the vDevID already exists.
You may want to use -EEXISTS.
> +
> + /* TODO: Use PCI helper to convert vDevid to pDevid */
> + pdevid = vdevid;
How the PCI helper would convert a vDevID to pDevID? This is entirely up
to the toolstack.
It would be preferrable that this function take a device and the vDevID
in parameter.
> + spin_lock(&rb_its_dev_lock);
> + pdev = find_its_device(&rb_its_dev, pdevid);
> + if ( !pdev )
> + {
> + spin_unlock(&rb_its_dev_lock);
> + return -ENODEV;
> + }
> + spin_unlock(&rb_its_dev_lock);
Same remark as in its_add_device for finding a device.
> +
> + vdev = xzalloc_bytes(sizeof(struct vits_device));
> + if ( !pdev )
> + return -ENOMEM;
> +
> + vdev->its_dev = pdev;
> + vdev->vdevid = vdevid;
> + vdev->pdevid = pdevid;
> +
> + /* Insert to domains' list */
> + if ( insert_vits_device(&d->arch.vits->dev_root, vdev) )
> + return -EINVAL;
> +
> + DPRINTK("ITS:d%dv%d Assign pdevid 0x%x with nvecs %d\n",
> + d->domain_id, current->vcpu_id, pdev->device_id, pdev->nr_lpis);
ditto for d and current
> +
> + /* XXX: Event id always starts from 0? */
> + for ( i = 0; i < pdev->nr_lpis; i++ )
> + {
> + plpi = its_get_plpi(pdev, i);
> + route_irq_to_guest(d, i, plpi, "LPI");
> + /* Enable all the event in LPI configuration table */
> + desc = irq_to_desc(plpi);
> + set_irq_device(desc, pdev);
> + lpi_set_config(desc, 1);
route_irq_to_guest expects to see a vIRQ in parameter and not an
eventID. Actually, all the check are based on this assumption (even
after you changes in patch #16).
The correct solution is to introduce a new function to route the LPI and
set correctly the required information.
> + }
> +
> + return 0;
> +}
> +
> +int its_detach_device(struct domain *d, u32 vdevid)
> +{
> + struct its_device *pdev;
> + struct vits_device *vdev;
> + struct irq_desc *desc;
> + u32 plpi, i, pdevid;
> +
> + DPRINTK("ITS:d%dv%d Detach request for virtual device 0x%x\n",
> + d->domain_id, current->vcpu_id, vdevid);
ditto for d and current.
> + vdev = find_vits_device(&d->arch.vits->dev_root, vdevid);
> + if ( !vdev )
> + return -EINVAL;
> +
> + /* XXX: convert vDevid to pDevid ? */
> + pdevid = vdevid;
> + spin_lock(&rb_its_dev_lock);
> + pdev = find_its_device(&rb_its_dev, pdevid);
> + if ( !pdev )
> + {
> + spin_unlock(&rb_its_dev_lock);
> + return -EINVAL;
> + }
> + spin_unlock(&rb_its_dev_lock);
Why do you need this? You have the vDev which contains a pointer to the
its_device structure.
> +
> + DPRINTK("ITS:d%dv%d Detach pdevid 0x%x with nvecs %d\n",
> + d->domain_id, current->vcpu_id, pdev->device_id, pdev->nr_lpis);
> + /* Remove vits_device from domain list */
> + remove_vits_device(&d->arch.vits->dev_root, vdev);
> + xfree(vdev);
> +
> + for ( i = 0; i < pdev->nr_lpis; i++ )
> + {
> + plpi = its_get_plpi(pdev, i);
> + /* XXX: Event id always starts from 0 */
> + release_irq(plpi, d);
> + /* Disable all the event in LPI configuration table */
> + desc = irq_to_desc(plpi);
> + set_irq_device(desc, NULL);
release_irq free irq_guest so this will segfault. As for
route_irq_to_guest please introduce a function to release the LPI.
> + lpi_set_config(desc, 0);
The interrupt should be disabled before deassigning from the guest.
IHMO, all this stuff should be done the a new release function.
> + }
> +
> + its_lpi_free(pdev->lpi_map, pdev->lpi_base, pdev->nr_lpis);
Why do you free the physical LPI chunk? We want to be able to reassign
the guest later.
> + return 0;
> +}
> /*
> * We allocate 64kB for PROPBASE. That gives us at most 64K LPIs to
> * deal with (one configuration byte per interrupt). PENDBASE has to
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index a1099a1..8f898a6 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -263,7 +263,9 @@ static inline uint32_t its_decode_devid(struct domain *d, its_cmd_block *cmd)
> void its_set_affinity(struct irq_desc *desc, int cpu);
> void lpi_set_config(struct irq_desc *desc, int enable);
> uint32_t its_get_nr_events(void);
> -
> +struct vits_device * find_vits_device(struct rb_root *root, uint32_t devid);
Coding style. And in general, the function prototype should be declared
in the patch where the function has been added not when it's used.
> +int insert_vits_device(struct rb_root *root, struct vits_device *dev);
> +int remove_vits_device(struct rb_root *root, struct vits_device *dev);
> #endif /* __ASM_ARM_GIC_ITS_H__ */
>
> /*
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device
2015-06-24 11:38 ` Julien Grall
@ 2015-06-29 12:25 ` Ian Campbell
0 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 12:25 UTC (permalink / raw)
To: Julien Grall
Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
tim, xen-devel, stefano.stabellini, manish.jaggi
On Wed, 2015-06-24 at 12:38 +0100, Julien Grall wrote:
[...]
> > @@ -459,7 +475,7 @@ void its_send_inv(struct its_device *dev, struct its_collection *col,
> > its_send_single_command(dev->its, its_build_inv_cmd, &desc);
> > }
> >
> > -void its_send_mapd(struct its_device *dev, int valid)
> > +static void its_send_mapd(struct its_device *dev, int valid)
>
> I would prefer to see this static where the function has been introduced
> and delay the compilation until the end.
Yes, please.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device
2015-06-22 12:01 ` [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
2015-06-24 11:38 ` Julien Grall
@ 2015-06-29 12:29 ` Ian Campbell
2015-07-02 8:40 ` Vijay Kilari
1 sibling, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 12:29 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
CIOn Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> +/* Device assignment. Should be called from pci_device_add */
> +int its_add_device(struct domain *d, u32 devid)
> +{
Prior to the PCI series landing, and to enable dom0 to use ITS it might
be possible to call this from xen/arch/arm/platforms/thunderx.c via the
specific_mappings platform hook, which would also expose the PCI
controller to dom0 via a series of specific mmio mappings (look at
xen/arch/arm/platforms/xgene-storm.c for the sort of thing I mean).
That would, I think, give basic PCI functionality for dom0 (i.e.
allowing us to boot on thunderx) and decouple things from the PCI series
somewhat, which ought to make things easier overall IMHO.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device
2015-06-29 12:29 ` Ian Campbell
@ 2015-07-02 8:40 ` Vijay Kilari
2015-07-02 9:01 ` Ian Campbell
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-07-02 8:40 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Mon, Jun 29, 2015 at 5:59 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> CIOn Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>> +/* Device assignment. Should be called from pci_device_add */
>> +int its_add_device(struct domain *d, u32 devid)
>> +{
>
> Prior to the PCI series landing, and to enable dom0 to use ITS it might
> be possible to call this from xen/arch/arm/platforms/thunderx.c via the
> specific_mappings platform hook, which would also expose the PCI
> controller to dom0 via a series of specific mmio mappings (look at
> xen/arch/arm/platforms/xgene-storm.c for the sort of thing I mean).
>
> That would, I think, give basic PCI functionality for dom0 (i.e.
> allowing us to boot on thunderx) and decouple things from the PCI series
> somewhat, which ought to make things easier overall IMHO.
In case ThunderX, mmio mappings PCI RC is parsed to find devices.
How do we know device ids upfront to call its_add_device?
>
> Ian.
>
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device
2015-07-02 8:40 ` Vijay Kilari
@ 2015-07-02 9:01 ` Ian Campbell
2015-07-02 10:30 ` Julien Grall
0 siblings, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-07-02 9:01 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Thu, 2015-07-02 at 14:10 +0530, Vijay Kilari wrote:
> On Mon, Jun 29, 2015 at 5:59 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> > CIOn Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> >> +/* Device assignment. Should be called from pci_device_add */
> >> +int its_add_device(struct domain *d, u32 devid)
> >> +{
> >
> > Prior to the PCI series landing, and to enable dom0 to use ITS it might
> > be possible to call this from xen/arch/arm/platforms/thunderx.c via the
> > specific_mappings platform hook, which would also expose the PCI
> > controller to dom0 via a series of specific mmio mappings (look at
> > xen/arch/arm/platforms/xgene-storm.c for the sort of thing I mean).
> >
> > That would, I think, give basic PCI functionality for dom0 (i.e.
> > allowing us to boot on thunderx) and decouple things from the PCI series
> > somewhat, which ought to make things easier overall IMHO.
>
> In case ThunderX, mmio mappings PCI RC is parsed to find devices.
> How do we know device ids upfront to call its_add_device?
Ah yes, this approach probably doesn't work for ITS. Which is a shame.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device
2015-07-02 9:01 ` Ian Campbell
@ 2015-07-02 10:30 ` Julien Grall
0 siblings, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-07-02 10:30 UTC (permalink / raw)
To: Ian Campbell, Vijay Kilari
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
On 02/07/15 10:01, Ian Campbell wrote:
> On Thu, 2015-07-02 at 14:10 +0530, Vijay Kilari wrote:
>> On Mon, Jun 29, 2015 at 5:59 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
>>> CIOn Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>>>> +/* Device assignment. Should be called from pci_device_add */
>>>> +int its_add_device(struct domain *d, u32 devid)
>>>> +{
>>>
>>> Prior to the PCI series landing, and to enable dom0 to use ITS it might
>>> be possible to call this from xen/arch/arm/platforms/thunderx.c via the
>>> specific_mappings platform hook, which would also expose the PCI
>>> controller to dom0 via a series of specific mmio mappings (look at
>>> xen/arch/arm/platforms/xgene-storm.c for the sort of thing I mean).
>>>
>>> That would, I think, give basic PCI functionality for dom0 (i.e.
>>> allowing us to boot on thunderx) and decouple things from the PCI series
>>> somewhat, which ought to make things easier overall IMHO.
>>
>> In case ThunderX, mmio mappings PCI RC is parsed to find devices.
>> How do we know device ids upfront to call its_add_device?
>
> Ah yes, this approach probably doesn't work for ITS. Which is a shame.
I guess the number of device IDs supported by thunderX is bounded and
not so big. So you could do a loop which register all the IDs from 0 to N.
Though, it may be possible to be smarter.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 11/18] xen/arm: ITS: Add GITS registers emulation
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (9 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 10/18] xen/arm: ITS: Add APIs to add and assign device vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-26 12:51 ` Julien Grall
2015-07-08 12:11 ` Ian Campbell
2015-06-22 12:01 ` [RFC PATCH v3 12/18] xen/arm: ITS: Add GICR register emulation vijay.kilari
` (9 subsequent siblings)
20 siblings, 2 replies; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Emulate GITS* registers and handle LPI configuration
table update trap.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/vgic-v3-its.c | 516 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 14 ++
2 files changed, 530 insertions(+)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 0671434..fa9dccc 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -63,6 +63,46 @@ static void dump_cmd(its_cmd_block *cmd)
}
#endif
+void vgic_its_disable_lpis(struct vcpu *v, uint32_t vlpi)
+{
+ struct pending_irq *p;
+ unsigned long flags;
+
+ p = irq_to_pending(v, vlpi);
+ clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+ gic_remove_from_queues(v, vlpi);
+ if ( p->desc != NULL )
+ {
+ spin_lock_irqsave(&p->desc->lock, flags);
+ p->desc->handler->disable(p->desc);
+ spin_unlock_irqrestore(&p->desc->lock, flags);
+ }
+}
+
+void vgic_its_enable_lpis(struct vcpu *v, uint32_t vlpi, uint8_t priority)
+{
+ struct pending_irq *p;
+ unsigned long flags;
+
+ /* Get plpi for the given vlpi */
+ p = irq_to_pending(v, vlpi);
+ p->priority = priority;
+ set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
+
+ spin_lock_irqsave(&v->arch.vgic.lock, flags);
+
+ if ( !list_empty(&p->inflight) &&
+ !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
+ gic_raise_guest_irq(v, irq_to_virq(p->desc), p->priority);
+
+ spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
+ if ( p->desc != NULL )
+ {
+ spin_lock_irqsave(&p->desc->lock, flags);
+ p->desc->handler->enable(p->desc);
+ spin_unlock_irqrestore(&p->desc->lock, flags);
+ }
+}
/* ITS device table helper functions */
int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
struct vdevice_table *entry, int set)
@@ -649,6 +689,482 @@ err:
return 0;
}
+static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+ uint32_t offset;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+ uint8_t cfg;
+
+ offset = info->gpa -
+ (v->domain->arch.vits->propbase & 0xfffffffff000UL);
+
+ if ( offset < SZ_64K )
+ {
+ DPRINTK("vITS:d%dv%d LPI Table read offset 0x%x\n",
+ v->domain->domain_id, v->vcpu_id, offset);
+ cfg = readb_relaxed(v->domain->arch.vits->prop_page + offset);
+ *r = cfg;
+ return 1;
+ }
+ else
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d LPI Table read with wrong offset 0x%x\n",
+ v->domain->domain_id, v->vcpu_id, offset);
+
+
+ return 0;
+}
+
+static int vgic_v3_gits_lpi_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+ uint32_t offset;
+ uint32_t vid;
+ uint8_t cfg;
+ bool_t enable;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+
+ offset = info->gpa -
+ (v->domain->arch.vits->propbase & 0xfffffffff000UL);
+
+ vid = offset + NR_GIC_LPI;
+ if ( offset < SZ_64K )
+ {
+ DPRINTK("vITS:d%dv%d LPI Table write offset 0x%x\n",
+ v->domain->domain_id, v->vcpu_id, offset);
+ cfg = readb_relaxed(v->domain->arch.vits->prop_page + offset);
+ enable = (cfg & *r) & 0x1;
+
+ if ( !enable )
+ vgic_its_enable_lpis(v, vid, (*r & 0xfc));
+ else
+ vgic_its_disable_lpis(v, vid);
+
+ /* Update virtual prop page */
+ writeb_relaxed((*r & 0xff),
+ v->domain->arch.vits->prop_page + offset);
+
+ return 1;
+ }
+ else
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d LPI Table invalid write @ 0x%x\n",
+ v->domain->domain_id, v->vcpu_id, offset);
+
+ return 0;
+}
+
+static const struct mmio_handler_ops vgic_gits_lpi_mmio_handler = {
+ .read_handler = vgic_v3_gits_lpi_mmio_read,
+ .write_handler = vgic_v3_gits_lpi_mmio_write,
+};
+
+int vgic_its_unmap_lpi_prop(struct vcpu *v)
+{
+ paddr_t maddr;
+ uint32_t lpi_size;
+ int i;
+
+ maddr = v->domain->arch.vits->propbase & 0xfffffffff000UL;
+ lpi_size = 1UL << ((v->domain->arch.vits->propbase & 0x1f) + 1);
+
+ DPRINTK("vITS:d%dv%d Unmap guest LPI conf table maddr 0x%lx lpi_size 0x%x\n",
+ v->domain->domain_id, v->vcpu_id, maddr, lpi_size);
+
+ if ( lpi_size < SZ_64K )
+ {
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d LPI Prop page < 64K\n",
+ v->domain->domain_id, v->vcpu_id);
+ return 0;
+ }
+
+ /* XXX: As per 4.8.9 each re-distributor shares a common LPI configuration table
+ * So one set of mmio handlers to manage configuration table is enough
+ */
+ for ( i = 0; i < lpi_size / PAGE_SIZE; i++ )
+ guest_physmap_remove_page(v->domain, paddr_to_pfn(maddr),
+ gmfn_to_mfn(v->domain, paddr_to_pfn(maddr)), 0);
+
+ /* Register mmio handlers for this region */
+ register_mmio_handler(v->domain, &vgic_gits_lpi_mmio_handler,
+ maddr, lpi_size);
+
+ /* Allocate Virtual LPI Property table */
+ v->domain->arch.vits->prop_page =
+ alloc_xenheap_pages(get_order_from_bytes(lpi_size), 0);
+ if ( !v->domain->arch.vits->prop_page )
+ {
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to allocate LPI Prop page\n",
+ v->domain->domain_id, v->vcpu_id);
+ return 0;
+ }
+
+ memset(v->domain->arch.vits->prop_page, 0xa2, lpi_size);
+
+ return 1;
+}
+
+static inline void vits_spin_lock(struct vgic_its *vits)
+{
+ spin_lock(&vits->lock);
+}
+
+static inline void vits_spin_unlock(struct vgic_its *vits)
+{
+ spin_unlock(&vits->lock);
+}
+
+static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
+{
+ struct vgic_its *vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+ uint64_t val = 0;
+ uint32_t index, gits_reg;
+
+ vits = v->domain->arch.vits;
+
+ gits_reg = info->gpa - vits->phys_base;
+
+ if ( gits_reg >= SZ_64K )
+ {
+ gdprintk(XENLOG_G_WARNING,
+ "vITS:d%dv%d unknown gpa read address %"PRIpaddr"\n",
+ v->domain->domain_id, v->vcpu_id, info->gpa);
+ return 0;
+ }
+
+ switch ( gits_reg )
+ {
+ case GITS_CTLR:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = 0;
+ return 1;
+ case GITS_IIDR:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = 0;
+ return 1;
+ case GITS_TYPER:
+ vits_spin_lock(vits);
+ val = (((v->domain->max_vcpus + 1) << GITS_TYPER_HCC_SHIFT ) |
+ VITS_GITS_DEV_BITS | VITS_GITS_ID_BITS |
+ VITS_GITS_ITT_SIZE | VITS_GITS_DISTRIBUTED |
+ VITS_GITS_PLPIS);
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ *r = val;
+ else if ( dabt.size == DABT_WORD )
+ *r = (u32)val;
+ else
+ {
+ vits_spin_unlock(vits);
+ goto bad_width;
+ }
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_TYPER + 4:
+ if (dabt.size != DABT_WORD ) goto bad_width;
+ vits_spin_lock(vits);
+ val = (((v->domain->max_vcpus + 1) << GITS_TYPER_HCC_SHIFT ) |
+ VITS_GITS_DEV_BITS | VITS_GITS_ID_BITS |
+ VITS_GITS_ITT_SIZE | VITS_GITS_DISTRIBUTED |
+ VITS_GITS_PLPIS);
+ *r = (u32)(val >> 32);
+ vits_spin_unlock(vits);
+ return 1;
+ case 0x0010 ... 0x007c:
+ case 0xc000 ... 0xffcc:
+ /* Implementation defined -- read ignored */
+ dprintk(XENLOG_ERR,
+ "vITS:d%dv%d read unknown 0x000c - 0x007c r%d offset %#08x\n",
+ v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
+ goto read_as_zero;
+ case GITS_CBASER:
+ /* XXX: Only read support 32/64-bit access */
+ vits_spin_lock(vits);
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ *r = vits->cmd_base && 0xc7ffffffffffffffUL;
+ else if ( dabt.size == DABT_WORD )
+ *r = (u32)vits->cmd_base;
+ else
+ {
+ vits_spin_unlock(vits);
+ goto bad_width;
+ }
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CBASER + 4:
+ if (dabt.size != DABT_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ *r = (u32)(vits->cmd_base >> 32);
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CWRITER:
+ /* XXX: Only read support 32/64-bit access */
+ vits_spin_lock(vits);
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ *r = vits->cmd_write;
+ else if ( dabt.size == DABT_WORD )
+ *r = (u32)vits->cmd_write;
+ else
+ {
+ vits_spin_unlock(vits);
+ goto bad_width;
+ }
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CWRITER + 4:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ *r = (u32)(vits->cmd_write >> 32);
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CREADR:
+ /* XXX: Only read support 32/64-bit access */
+ vits_spin_lock(vits);
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ *r = vits->cmd_read;
+ else if ( dabt.size == DABT_WORD )
+ *r = (u32)vits->cmd_read;
+ else
+ {
+ vits_spin_unlock(vits);
+ goto bad_width;
+ }
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CREADR + 4:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ *r = (u32)(vits->cmd_read >> 32);
+ vits_spin_unlock(vits);
+ return 1;
+ case 0x0098 ... 0x009c:
+ case 0x00a0 ... 0x00fc:
+ case 0x0140 ... 0xbffc:
+ /* Reserved -- read ignored */
+ dprintk(XENLOG_ERR,
+ "vITS:d%dv%d read unknown 0x0098-9c or 0x00a0-fc r%d offset %#08x\n",
+ v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
+ goto read_as_zero;
+ case GITS_BASER ... GITS_BASERN:
+ /* Supports only 64-bit access */
+ if ( dabt.size != DABT_DOUBLE_WORD )
+ goto bad_width;
+ if ( (gits_reg % 8) != 0 )
+ goto bad_width;
+ vits_spin_lock(vits);
+ index = (gits_reg - GITS_BASER) / 8;
+ *r = vits->baser[index];
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_PIDR0:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR0_VAL;
+ return 1;
+ case GITS_PIDR1:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR1_VAL;
+ return 1;
+ case GITS_PIDR2:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR2_VAL;
+ return 1;
+ case GITS_PIDR3:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR3_VAL;
+ return 1;
+ case GITS_PIDR4:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = GITS_PIDR4_VAL;
+ return 1;
+ case GITS_PIDR5 ... GITS_PIDR7:
+ goto read_as_zero;
+ default:
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d unhandled read r%d offset %#08x\n",
+ v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
+ return 0;
+ }
+
+bad_width:
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d bad read width %d r%d offset %#08x\n",
+ v->domain->domain_id, v->vcpu_id, dabt.size, dabt.reg, gits_reg);
+ domain_crash_synchronous();
+ return 0;
+
+read_as_zero:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ *r = 0;
+ return 1;
+}
+
+#define GITS_BASER_MASK (~((0x7UL << GITS_BASER_TYPE_SHIFT) | \
+ (0xffUL << GITS_BASER_ENTRY_SIZE_SHIFT) | \
+ (0x3UL << GITS_BASER_SHAREABILITY_SHIFT)))
+
+static int vgic_v3_gits_mmio_write(struct vcpu *v, mmio_info_t *info)
+{
+ struct vgic_its *vits;
+ struct hsr_dabt dabt = info->dabt;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ register_t *r = select_user_reg(regs, dabt.reg);
+ int ret;
+ uint32_t index, gits_reg, sz, psz;
+ uint64_t val;
+
+ vits = v->domain->arch.vits;
+
+ gits_reg = info->gpa - vits->phys_base;
+
+ if ( gits_reg >= SZ_64K )
+ {
+ gdprintk(XENLOG_G_WARNING,
+ "vITS:d%dv%d unknown gpa write address %"PRIpaddr"\n",
+ v->domain->domain_id, v->vcpu_id, info->gpa);
+ return 0;
+ }
+ switch ( gits_reg )
+ {
+ case GITS_CTLR:
+ if ( dabt.size != DABT_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ vits->ctrl = *r;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_IIDR:
+ /* R0 -- write ignored */
+ goto write_ignore;
+ case GITS_TYPER:
+ case GITS_TYPER + 4:
+ /* R0 -- write ignored */
+ goto write_ignore;
+ case 0x0010 ... 0x007c:
+ case 0xc000 ... 0xffcc:
+ /* Implementation defined -- write ignored */
+ dprintk(XENLOG_G_ERR,
+ "vITS:d%dv%d write to unknown 0x000c - 0x007c r%d offset %#08x\n",
+ v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
+ goto write_ignore;
+ case GITS_CBASER:
+ if ( dabt.size != DABT_DOUBLE_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ vits->cmd_base = *r;
+ vits->cmd_qsize = SZ_4K * ((*r & 0xff) + 1);
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_CBASER + 4:
+ /* XXX: Does not support word write */
+ goto bad_width;
+ case GITS_CWRITER:
+ vits_spin_lock(vits);
+ if ( dabt.size == DABT_DOUBLE_WORD )
+ vits->cmd_write = *r;
+ else if ( dabt.size == DABT_WORD)
+ {
+ val = vits->cmd_write & 0xffffffff00000000UL;
+ val = (*r) | val;
+ vits->cmd_write = val;
+ }
+ else
+ {
+ vits_spin_unlock(vits);
+ goto bad_width;
+ }
+ ret = vgic_its_process_cmd(v, vits);
+ vits_spin_unlock(vits);
+ return ret;
+ case GITS_CWRITER + 4:
+ if (dabt.size != DABT_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ val = vits->cmd_write & 0xffffffffUL;
+ val = ((*r & 0xffffffffUL) << 32) | val;
+ vits->cmd_write = val;
+ ret = vgic_its_process_cmd(v, vits);
+ vits_spin_unlock(vits);
+ return ret;
+ case GITS_CREADR:
+ /* R0 -- write ignored */
+ goto write_ignore;
+ case 0x0098 ... 0x009c:
+ case 0x00a0 ... 0x00fc:
+ case 0x0140 ... 0xbffc:
+ /* Reserved -- write ignored */
+ dprintk(XENLOG_G_ERR,
+ "vITS:d%dv%d write to unknown 0x98-9c or 0xa0-fc r%d offset %#08x\n",
+ v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
+ goto write_ignore;
+ case GITS_BASER0:
+ if ( dabt.size != DABT_DOUBLE_WORD )
+ goto bad_width;
+ vits_spin_lock(vits);
+ vits->baser[0] = vits->baser[0] | (GITS_BASER_MASK & *r);
+ vits->dt_ipa = vits->baser[0] & 0xfffffffff000UL;
+ psz = (vits->baser[0] >> GITS_BASER_PAGE_SIZE_SHIFT) &
+ GITS_BASER_PAGE_SIZE_MASK_VAL;
+ if ( psz == GITS_BASER_PAGE_SIZE_4K_VAL )
+ sz = 4;
+ else if ( psz == GITS_BASER_PAGE_SIZE_16K_VAL )
+ sz = 16;
+ else
+ sz = 64;
+
+ vits->dt_size = (vits->baser[0] & GITS_BASER_PAGES_MASK_VAL)
+ * sz * SZ_1K;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_BASER1 ... GITS_BASERN:
+ /* Nothing to do with this values. Just store and emulate */
+ if ( dabt.size != DABT_DOUBLE_WORD )
+ goto bad_width;
+ if ( (gits_reg % 8) != 0 )
+ goto bad_width;
+ vits_spin_lock(vits);
+ index = (gits_reg - GITS_BASER) / 8;
+ vits->baser[index] = *r;
+ vits_spin_unlock(vits);
+ return 1;
+ case GITS_PIDR7 ... GITS_PIDR0:
+ /* R0 -- write ignored */
+ goto write_ignore;
+ default:
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d unhandled write r%d offset %#08x\n",
+ v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
+ return 0;
+ }
+
+bad_width:
+ dprintk(XENLOG_G_ERR, "vITS:d%dv%d bad write width %d r%d offset %#08x\n",
+ v->domain->domain_id, v->vcpu_id, dabt.size, dabt.reg, gits_reg);
+ domain_crash_synchronous();
+ return 0;
+
+write_ignore:
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ *r = 0;
+ return 1;
+}
+
+
+static const struct mmio_handler_ops vgic_gits_mmio_handler = {
+ .read_handler = vgic_v3_gits_mmio_read,
+ .write_handler = vgic_v3_gits_mmio_write,
+};
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 8f898a6..3271477 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -38,6 +38,8 @@ struct its_collection {
struct vgic_its
{
spinlock_t lock;
+ /* Emulation of BASER */
+ paddr_t baser[8];
/* Command queue base */
paddr_t cmd_base;
/* Command queue write pointer */
@@ -48,8 +50,20 @@ struct vgic_its
paddr_t cmd_read;
/* Command queue size */
unsigned long cmd_qsize;
+ /* ITS mmio physical base */
+ paddr_t phys_base;
+ /* ITS mmio physical size */
+ unsigned long phys_size;
/* ITS physical node */
struct its_node *its;
+ /* GICR ctrl register */
+ uint32_t ctrl;
+ /* LPI propbase */
+ paddr_t propbase;
+ /* percpu pendbase */
+ paddr_t pendbase[MAX_VIRT_CPUS];
+ /* Virtual LPI property table */
+ void * prop_page;
/* vITT device table ipa */
paddr_t dt_ipa;
/* vITT device table size */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 11/18] xen/arm: ITS: Add GITS registers emulation
2015-06-22 12:01 ` [RFC PATCH v3 11/18] xen/arm: ITS: Add GITS registers emulation vijay.kilari
@ 2015-06-26 12:51 ` Julien Grall
2015-07-08 12:11 ` Ian Campbell
1 sibling, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-06-26 12:51 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 22/06/2015 14:01, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Emulate GITS* registers and handle LPI configuration
> table update trap.
The LPI configuration table as nothing to do with the GITS registers.
It's related to the GICR. Why didn't you implement it on the next patch?
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 516 +++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic-its.h | 14 ++
> 2 files changed, 530 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 0671434..fa9dccc 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -63,6 +63,46 @@ static void dump_cmd(its_cmd_block *cmd)
> }
> #endif
>
> +void vgic_its_disable_lpis(struct vcpu *v, uint32_t vlpi)
Any function non-exported should be static. Also you are disabling only
one lpi, so I would rename the function vgic_its_disable_lpi.
> +{
> + struct pending_irq *p;
> + unsigned long flags;
> +
> + p = irq_to_pending(v, vlpi);
> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> + gic_remove_from_queues(v, vlpi);
> + if ( p->desc != NULL )
> + {
> + spin_lock_irqsave(&p->desc->lock, flags);
> + p->desc->handler->disable(p->desc);
> + spin_unlock_irqrestore(&p->desc->lock, flags);
> + }
It's not possible to associate a unique pLPI to a vLPI. This function
should only update the pending_state.
> +}
> +
> +void vgic_its_enable_lpis(struct vcpu *v, uint32_t vlpi, uint8_t priority)
Ditto for static & name.
> +{
> + struct pending_irq *p;
> + unsigned long flags;
> +
> + /* Get plpi for the given vlpi */
> + p = irq_to_pending(v, vlpi);
> + p->priority = priority;
Why? There is already a callback to get the IRQ priority (see
get_irq_priority). It will retrieve the correct priority when the IRQ is
injected (and not when it's has been enabled).
In anycase, this is not safe to write in p->pending like that.
> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> +
> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
The vIRQ is protected by the CPU where it's routed not the one which
enabled. You have to find the right vCPU.
> +
> + if ( !list_empty(&p->inflight) &&
> + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
> + gic_raise_guest_irq(v, irq_to_virq(p->desc), p->priority);
> +
> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> + if ( p->desc != NULL )
> + {
> + spin_lock_irqsave(&p->desc->lock, flags);
> + p->desc->handler->enable(p->desc);
> + spin_unlock_irqrestore(&p->desc->lock, flags);
> + }
Ditto for the pLPI to vLPI mapping.
> +}
Missing newline
> /* ITS device table helper functions */
> int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
> struct vdevice_table *entry, int set)
> @@ -649,6 +689,482 @@ err:
> return 0;
> }
>
> +static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
The LPI configuration table is not ITS specific but GICv3. Although, I'm
fine if you let this code here for the time being.
> +{
> + uint32_t offset;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> + uint8_t cfg;
> +
> + offset = info->gpa -
> + (v->domain->arch.vits->propbase & 0xfffffffff000UL);
You are using 0xffff... in multiple place. Please add a define and use it.
> +
> + if ( offset < SZ_64K )
AFAICT, the LPI configuration table can be smaller and higher than 64K.
Although, this check is not necessary because you have registered the
handler on a valid MMIO range.
> + {
> + DPRINTK("vITS:d%dv%d LPI Table read offset 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, offset);
%pv rather than d%dv%d
> + cfg = readb_relaxed(v->domain->arch.vits->prop_page + offset);
Why do you use readb? Those helpers have been created to read MMIO not
Xen memory.
Although what about other access? a 64/32/16 bits access are valid and
will return the wrong value.
> + *r = cfg;
> + return 1;
> + }
> + else
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d LPI Table read with wrong offset 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, offset);
> +
> +
> + return 0;
> +}
> +
> +static int vgic_v3_gits_lpi_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
> + uint32_t offset;
> + uint32_t vid;
> + uint8_t cfg;
> + bool_t enable;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> +
> + offset = info->gpa -
> + (v->domain->arch.vits->propbase & 0xfffffffff000UL);
> +
> + vid = offset + NR_GIC_LPI;
> + if ( offset < SZ_64K )
> + {
> + DPRINTK("vITS:d%dv%d LPI Table write offset 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, offset);
%pv rather than d%dv%d
> + cfg = readb_relaxed(v->domain->arch.vits->prop_page + offset);
> + enable = (cfg & *r) & 0x1;
Please use a define rather than 0x1. It would have been more clear that
we are checking the enable bit.
> +
> + if ( !enable )
> + vgic_its_enable_lpis(v, vid, (*r & 0xfc));
Please a define for the priority mask.
> + else
> + vgic_its_disable_lpis(v, vid);
> +
> + /* Update virtual prop page */
> + writeb_relaxed((*r & 0xff),
> + v->domain->arch.vits->prop_page + offset);
Same remark as readb_relaxed. You also need to handle all the other
possible write access.
Finally, this function is racy. Nothing prevent 2 vCPUs to write in the
LPI configuration table. This may result to incoherency between the
state of the pending_irq and the Configuration Table.
You have to provide at least a locking for the Configuration Table.
> +
> + return 1;
> + }
> + else
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d LPI Table invalid write @ 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, offset);
> +
> + return 0;
> +}
> +
> +static const struct mmio_handler_ops vgic_gits_lpi_mmio_handler = {
> + .read_handler = vgic_v3_gits_lpi_mmio_read,
> + .write_handler = vgic_v3_gits_lpi_mmio_write,
> +};
> +
> +int vgic_its_unmap_lpi_prop(struct vcpu *v)
Please define the prototype of this function in the header within the
same file if you plan to export. If not, it should be static.
> +{
> + paddr_t maddr;
> + uint32_t lpi_size;
> + int i;
> +
> + maddr = v->domain->arch.vits->propbase & 0xfffffffff000UL;
> + lpi_size = 1UL << ((v->domain->arch.vits->propbase & 0x1f) + 1);
Please use a define for the 0x1f.
> +
> + DPRINTK("vITS:d%dv%d Unmap guest LPI conf table maddr 0x%lx lpi_size 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, maddr, lpi_size);
> +
> + if ( lpi_size < SZ_64K )
Why this restriction? The IDbits can encode up to 32 bits interrupt
identifier.
You have to check this value against GICD_TYPER.IDbits.
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d LPI Prop page < 64K\n",
> + v->domain->domain_id, v->vcpu_id);
> + return 0;
> + }
> +
> + /* XXX: As per 4.8.9 each re-distributor shares a common LPI configuration table
Coding style:
/*
* ...
*/
Also XXX means TODO.
4.8.9 of what? Please provide a section for the public doc (i.e
IHI0069A...) and not private.
> + * So one set of mmio handlers to manage configuration table is enough
> + */
> + for ( i = 0; i < lpi_size / PAGE_SIZE; i++ )
> + guest_physmap_remove_page(v->domain, paddr_to_pfn(maddr),
> + gmfn_to_mfn(v->domain, paddr_to_pfn(maddr)), 0);
No validation at all on the address pass for the guest? gmfn_to_mfn can
return an invalid MFN and I'm not sure what would happen if the guest is
trying to pass other things than RAM.
> +
> + /* Register mmio handlers for this region */
> + register_mmio_handler(v->domain, &vgic_gits_lpi_mmio_handler,
> + maddr, lpi_size);
> +
> + /* Allocate Virtual LPI Property table */
> + v->domain->arch.vits->prop_page =
> + alloc_xenheap_pages(get_order_from_bytes(lpi_size), 0);
I wasn't able to find a place where you free the pages allocated...
Please ensure that any allocated memory is freed when the guest is
destroyed.
Rather than allocating the page from Xen memory, I'm wondering if we can
re-use the page unmapped.
If not, I would prefer to see this allocation at domain creation time
rather than when the domain is setting PROPBASER. The main reason is
with the current implementation, the guest could abuse of PROPBASER and
exhaust the Xen memory. It would require to do some management here.
Also, what if the guest decide to change it's property table? This is
valid and I think we will hit with the suspend/resume case. I'm fine if
you don't want to handle it now, but please make it obvious that you
don't handle it.
> + if ( !v->domain->arch.vits->prop_page )
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to allocate LPI Prop page\n",
> + v->domain->domain_id, v->vcpu_id);
> + return 0;
> + }
> +
> + memset(v->domain->arch.vits->prop_page, 0xa2, lpi_size);
Where does this value come from? It's up to the guest to decide what
will be the priority of vLPI. Also, have you a doc reference to say that
we should ignore any value in the initial set of pages?
> +
> + return 1;
> +}
> +
> +static inline void vits_spin_lock(struct vgic_its *vits)
> +{
> + spin_lock(&vits->lock);
> +}
> +
> +static inline void vits_spin_unlock(struct vgic_its *vits)
> +{
> + spin_unlock(&vits->lock);
> +}
> +
> +static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> + struct vgic_its *vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> + uint64_t val = 0;
> + uint32_t index, gits_reg;
> +
> + vits = v->domain->arch.vits;
You can directly do struct vgic_its *vits = v->domain->arch.vits
> +
> + gits_reg = info->gpa - vits->phys_base;
> +
> + if ( gits_reg >= SZ_64K )
This can never happen, you always register a valid 64K range.
> + {
> + gdprintk(XENLOG_G_WARNING,
> + "vITS:d%dv%d unknown gpa read address %"PRIpaddr"\n",
> + v->domain->domain_id, v->vcpu_id, info->gpa);
> + return 0;
> + }
> +
> + switch ( gits_reg )
> + {
> + case GITS_CTLR:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = 0;
Missing implementation of GITS_CTLR.
> + return 1;
> + case GITS_IIDR:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = 0;
Missing implementaiton of GITS_IIDR
> + return 1;
> + case GITS_TYPER:
> + vits_spin_lock(vits);
Why the lock?
Please add a comment explaining the configuration of this field. It
would avoid to scratch his head try to understand why PTA is not
mentioned...
> + val = (((v->domain->max_vcpus + 1) << GITS_TYPER_HCC_SHIFT ) |
I would prefer to see a macro to compute the number of collection.
Although, HCC is only able to encore 16-bit of collection ID (i.e 256
collection => 255 VCPUs). Please make a note somewhere and add a
BUILD_ON_BUG on anything else to ensure that it will never happen.
HCC is only able to encode 16-bit collection ID (i.e 255 collection).
> + VITS_GITS_DEV_BITS | VITS_GITS_ID_BITS |
Please introduce new fields in vits to store the number of deviceID bits
and eventID bits. Don't harcode them.
> + VITS_GITS_ITT_SIZE | VITS_GITS_DISTRIBUTED |
The bit 3 is marked as implementation defined. Why do you set and name
it DISTRIBUTED?
> + VITS_GITS_PLPIS);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = val;
> + else if ( dabt.size == DABT_WORD )
> + *r = (u32)val;
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_TYPER + 4:
I don't like the idea to duplicate the code for GITS_TYPER just for
reading the top word. Isn't possible to merge the 2 switch case?
> + if (dabt.size != DABT_WORD ) goto bad_width;
> + vits_spin_lock(vits);
> + val = (((v->domain->max_vcpus + 1) << GITS_TYPER_HCC_SHIFT ) |
> + VITS_GITS_DEV_BITS | VITS_GITS_ID_BITS |
> + VITS_GITS_ITT_SIZE | VITS_GITS_DISTRIBUTED |
> + VITS_GITS_PLPIS);
> + *r = (u32)(val >> 32);
> + vits_spin_unlock(vits);
> + return 1;
> + case 0x0010 ... 0x007c:
> + case 0xc000 ... 0xffcc:
> + /* Implementation defined -- read ignored */
> + dprintk(XENLOG_ERR,
> + "vITS:d%dv%d read unknown 0x000c - 0x007c r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
Please don't use XENLOG_ERR in guest code. Also, this printk is not
useful and has been dropped in other emulation.
> + goto read_as_zero;
> + case GITS_CBASER:
> + /* XXX: Only read support 32/64-bit access */
XXX means TODO which is not the case here.
> + vits_spin_lock(vits);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = vits->cmd_base && 0xc7ffffffffffffffUL;
Why the && and what does mean this constant?
> + else if ( dabt.size == DABT_WORD )
> + *r = (u32)vits->cmd_base;
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + vits_spin_unlock(vits);
The lock section could have been smaller and avoid to duplicat
vits_spin_unlock by first reading the value then setting the register.
> + return 1;
> + case GITS_CBASER + 4:
Same remark as GITS_TYPER for the support for reading word.
> + if (dabt.size != DABT_WORD )
if ( ... )
> + goto bad_width;
> + vits_spin_lock(vits);
> + *r = (u32)(vits->cmd_base >> 32);
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CWRITER:
See all my comment on CBASER.
> + /* XXX: Only read support 32/64-bit access */
> + vits_spin_lock(vits);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = vits->cmd_write;
> + else if ( dabt.size == DABT_WORD )
> + *r = (u32)vits->cmd_write;
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CWRITER + 4:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + *r = (u32)(vits->cmd_write >> 32);
> + vits_spin_unlock(vits);
> + return 1;
ditto for word-read
> + case GITS_CREADR:
> + /* XXX: Only read support 32/64-bit access */
> + vits_spin_lock(vits);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = vits->cmd_read;
> + else if ( dabt.size == DABT_WORD )
> + *r = (u32)vits->cmd_read;
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CREADR + 4:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + *r = (u32)(vits->cmd_read >> 32);
> + vits_spin_unlock(vits);
> + return 1;
ditto
> + case 0x0098 ... 0x009c:
> + case 0x00a0 ... 0x00fc:
> + case 0x0140 ... 0xbffc:
> + /* Reserved -- read ignored */
> + dprintk(XENLOG_ERR,
> + "vITS:d%dv%d read unknown 0x0098-9c or 0x00a0-fc r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
No need to the message.
> + goto read_as_zero;
> + case GITS_BASER ... GITS_BASERN:
> + /* Supports only 64-bit access */
The XXX would have been useful here ;)
> + if ( dabt.size != DABT_DOUBLE_WORD )
> + goto bad_width;
> + if ( (gits_reg % 8) != 0 )
What is used for?
> + goto bad_width;
> + vits_spin_lock(vits);
> + index = (gits_reg - GITS_BASER) / 8;
> + *r = vits->baser[index];
> + vits_spin_unlock(vits);
Based on the spec (8.19.1 in IHI0069A), any unimplemented register
should be RES0.
As we only support BASER0, I would only implemented it with a check all
the other would go to the read_as_zero.
> + return 1;
> + case GITS_PIDR0:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR0_VAL;
> + return 1;
> + case GITS_PIDR1:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR1_VAL;
> + return 1;
> + case GITS_PIDR2:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR2_VAL;
> + return 1;
> + case GITS_PIDR3:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR3_VAL;
> + return 1;
> + case GITS_PIDR4:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR4_VAL;
> + return 1;
> + case GITS_PIDR5 ... GITS_PIDR7:
> + goto read_as_zero;
Please check that we access is done via a word-access by introducing a
new label read_as_zero_32 (for instance see the vgic v2 emulation).
> + default:
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d unhandled read r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
> + return 0;
> + }
> +
> +bad_width:
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d bad read width %d r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.size, dabt.reg, gits_reg);
printk(XENLOG_G_ERR "%pv: ....", v,...)
> + domain_crash_synchronous();
> + return 0;
> +
> +read_as_zero:
> + if ( dabt.size != DABT_WORD )
How do you know that all RAZ access 32-bit access? See implementation
defined registers for instance.
I would prefer to introduce multiple label:
read_as_zero_32: /* RAZ 32-bit */
if ( dabt.size != DABT_WORD ) goto bad_width;
read_as_zero: /* Not check necessary */
*r = 0;
And use the correctly label for goto in the emulation. So the code would
be self-documented too.
> + goto bad_width;
> + *r = 0;
> + return 1;
> +}
> +
> +#define GITS_BASER_MASK (~((0x7UL << GITS_BASER_TYPE_SHIFT) | \
> + (0xffUL << GITS_BASER_ENTRY_SIZE_SHIFT) | \
> + (0x3UL << GITS_BASER_SHAREABILITY_SHIFT)))
Please add a comment to explain what the mask is used for.
> +
> +static int vgic_v3_gits_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
> + struct vgic_its *vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> + int ret;
> + uint32_t index, gits_reg, sz, psz;
> + uint64_t val;
> +
> + vits = v->domain->arch.vits;
> +
> + gits_reg = info->gpa - vits->phys_base;
> +
> + if ( gits_reg >= SZ_64K )
> + {
> + gdprintk(XENLOG_G_WARNING,
> + "vITS:d%dv%d unknown gpa write address %"PRIpaddr"\n",
> + v->domain->domain_id, v->vcpu_id, info->gpa);
> + return 0;
> + }
This check is not necessary.
> + switch ( gits_reg )
> + {
> + case GITS_CTLR:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + vits->ctrl = *r;
> + vits_spin_unlock(vits);
Only bit[0] (Enabled) is writable.
> + return 1;
> + case GITS_IIDR:
> + /* R0 -- write ignored */
> + goto write_ignore;
> + case GITS_TYPER:
> + case GITS_TYPER + 4:
> + /* R0 -- write ignored */
> + goto write_ignore;
Please explicitly check the access size. That would avoid to crash the
guest when TYPER is write with a 64-bit access.
> + case 0x0010 ... 0x007c:
> + case 0xc000 ... 0xffcc:
> + /* Implementation defined -- write ignored */
> + dprintk(XENLOG_G_ERR,
> + "vITS:d%dv%d write to unknown 0x000c - 0x007c r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
Please drop the dprintk.
> + goto write_ignore;
> + case GITS_CBASER:
GITS_CBASER is read-only when GITS_CTLR.Enable is Zero or
GITS_CTLR.Quiescent is zero.
> + if ( dabt.size != DABT_DOUBLE_WORD )
Missing a TODO for 32bit access support.
> + goto bad_width;
> + vits_spin_lock(vits);
> + vits->cmd_base = *r;
> + vits->cmd_qsize = SZ_4K * ((*r & 0xff) + 1);
Please use a define for the mask.
Also I would use cmd_qsize to know if the valid is set or not. I.e
cmd_qsize = 0 => command queue not valid.
You forgot to update GITS_CREADR (i.e setting to 0) when GITS_CBASER is
successfully written (8.19.2 in IHI0069A).
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CBASER + 4:
> + /* XXX: Does not support word write */
> + goto bad_width;
> + case GITS_CWRITER:
> + vits_spin_lock(vits);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + vits->cmd_write = *r;
Only Bits[19:5] are writable.
> + else if ( dabt.size == DABT_WORD)
> + {
> + val = vits->cmd_write & 0xffffffff00000000UL;
> + val = (*r) | val;
Only Bits[19:5] are writable.
> + vits->cmd_write = val;
> + }
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
This could be simplified to avoid 2 vits_spin_unlock.
Also, no validation of the value written by the guest? Given your
implementation of the command processing, any invalid value will end up
to an infinite loop in the hypervisor. Whoops :).
> + ret = vgic_its_process_cmd(v, vits);
> + vits_spin_unlock(vits);
> + return ret;
> + case GITS_CWRITER + 4:
Only the Bits[19:5] of GITS_CWRITER are writable. So the emulation is
pointless here.
> + if (dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + val = vits->cmd_write & 0xffffffffUL;
> + val = ((*r & 0xffffffffUL) << 32) | val;
> + vits->cmd_write = val;
> + ret = vgic_its_process_cmd(v, vits);
> + vits_spin_unlock(vits);
> + return ret;
> + case GITS_CREADR:
> + /* R0 -- write ignored */
> + goto write_ignore;
> + case 0x0098 ... 0x009c:
> + case 0x00a0 ... 0x00fc:
> + case 0x0140 ... 0xbffc:
> + /* Reserved -- write ignored */
> + dprintk(XENLOG_G_ERR,
> + "vITS:d%dv%d write to unknown 0x98-9c or 0xa0-fc r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
Please drop the printk
> + goto write_ignore;
> + case GITS_BASER0:
> + if ( dabt.size != DABT_DOUBLE_WORD )
/* XXX: Support 32 bit access */
> + goto bad_width;
> + vits_spin_lock(vits);
> + vits->baser[0] = vits->baser[0] | (GITS_BASER_MASK & *r);
I would prefer to see *r & GITS_BASER_MASK which is more logical to read.
Although this doesn't do what you want. This will not erase bit set to 1
in baser.
> + vits->dt_ipa = vits->baser[0] & 0xfffffffff000UL;
> + psz = (vits->baser[0] >> GITS_BASER_PAGE_SIZE_SHIFT) &
> + GITS_BASER_PAGE_SIZE_MASK_VAL;
> + if ( psz == GITS_BASER_PAGE_SIZE_4K_VAL )
> + sz = 4;
> + else if ( psz == GITS_BASER_PAGE_SIZE_16K_VAL )
> + sz = 16;
> + else
> + sz = 64;
> +
> + vits->dt_size = (vits->baser[0] & GITS_BASER_PAGES_MASK_VAL)
> + * sz * SZ_1K;
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_BASER1 ... GITS_BASERN:
> + /* Nothing to do with this values. Just store and emulate */
From the spec these register should be RES0. There is nothing to do here.
> + if ( dabt.size != DABT_DOUBLE_WORD )
> + goto bad_width;
> + if ( (gits_reg % 8) != 0 )
> + goto bad_width;
> + vits_spin_lock(vits);
> + index = (gits_reg - GITS_BASER) / 8;
> + vits->baser[index] = *r;
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_PIDR7 ... GITS_PIDR0:
> + /* R0 -- write ignored */
> + goto write_ignore;
> + default:
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d unhandled write r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
printk(XENLOG_G_ERR "VITS %pv: .....", v, ....);
> + return 0;
> + }
> +
> +bad_width:
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d bad write width %d r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.size, dabt.reg, gits_reg);
Ditto
> + domain_crash_synchronous();
> + return 0;
> +
> +write_ignore:
Same remark as read_ignore.
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + *r = 0;
> + return 1;
> +}
> +
> +
> +static const struct mmio_handler_ops vgic_gits_mmio_handler = {
> + .read_handler = vgic_v3_gits_mmio_read,
> + .write_handler = vgic_v3_gits_mmio_write,
> +};
> +
Any patch should be compilable on standalone to avoid breaking
bisection. I believe this is not the case here.
As suggested earlier, I would prefer the Makefile change to include this
file when everything is done. This would avoid you s//static/ changes...
> /*
> * Local variables:
> * mode: C
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 8f898a6..3271477 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -38,6 +38,8 @@ struct its_collection {
> struct vgic_its
> {
> spinlock_t lock;
> + /* Emulation of BASER */
> + paddr_t baser[8];
I don't think this is useful.
> /* Command queue base */
> paddr_t cmd_base;
> /* Command queue write pointer */
> @@ -48,8 +50,20 @@ struct vgic_its
> paddr_t cmd_read;
> /* Command queue size */
> unsigned long cmd_qsize;
> + /* ITS mmio physical base */
> + paddr_t phys_base;
> + /* ITS mmio physical size */
> + unsigned long phys_size;
The names are misleading, I though it was referring to the hardware address.
I would prefer if you rename into gits_base and gits_size.
> /* ITS physical node */
> struct its_node *its;
> + /* GICR ctrl register */
> + uint32_t ctrl;
> + /* LPI propbase */
> + paddr_t propbase;
> + /* percpu pendbase */
> + paddr_t pendbase[MAX_VIRT_CPUS];
> + /* Virtual LPI property table */
> + void * prop_page;
Coding style: void *prop_page
> /* vITT device table ipa */
> paddr_t dt_ipa;
> /* vITT device table size */
>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 11/18] xen/arm: ITS: Add GITS registers emulation
2015-06-22 12:01 ` [RFC PATCH v3 11/18] xen/arm: ITS: Add GITS registers emulation vijay.kilari
2015-06-26 12:51 ` Julien Grall
@ 2015-07-08 12:11 ` Ian Campbell
1 sibling, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-07-08 12:11 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Emulate GITS* registers and handle LPI configuration
> table update trap.
These need to only be exposed to a guest which has been configured with
an ITS. For dom0 that means at a minimum it needs to be based on the
capabilities of the underlying hardware.
The same is true of the next patch adding the GICR registers.
For domU it seems there is currently no ITS exposed to them, since there
is no toolstack changes here, so the emulation should be configured
accordingly.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/vgic-v3-its.c | 516 +++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic-its.h | 14 ++
> 2 files changed, 530 insertions(+)
>
> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
> index 0671434..fa9dccc 100644
> --- a/xen/arch/arm/vgic-v3-its.c
> +++ b/xen/arch/arm/vgic-v3-its.c
> @@ -63,6 +63,46 @@ static void dump_cmd(its_cmd_block *cmd)
> }
> #endif
>
> +void vgic_its_disable_lpis(struct vcpu *v, uint32_t vlpi)
> +{
> + struct pending_irq *p;
> + unsigned long flags;
> +
> + p = irq_to_pending(v, vlpi);
> + clear_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> + gic_remove_from_queues(v, vlpi);
> + if ( p->desc != NULL )
> + {
> + spin_lock_irqsave(&p->desc->lock, flags);
> + p->desc->handler->disable(p->desc);
> + spin_unlock_irqrestore(&p->desc->lock, flags);
> + }
> +}
> +
> +void vgic_its_enable_lpis(struct vcpu *v, uint32_t vlpi, uint8_t priority)
> +{
> + struct pending_irq *p;
> + unsigned long flags;
> +
> + /* Get plpi for the given vlpi */
> + p = irq_to_pending(v, vlpi);
> + p->priority = priority;
> + set_bit(GIC_IRQ_GUEST_ENABLED, &p->status);
> +
> + spin_lock_irqsave(&v->arch.vgic.lock, flags);
> +
> + if ( !list_empty(&p->inflight) &&
> + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) )
> + gic_raise_guest_irq(v, irq_to_virq(p->desc), p->priority);
> +
> + spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
> + if ( p->desc != NULL )
> + {
> + spin_lock_irqsave(&p->desc->lock, flags);
> + p->desc->handler->enable(p->desc);
> + spin_unlock_irqrestore(&p->desc->lock, flags);
> + }
> +}
> /* ITS device table helper functions */
> int vits_vdevice_entry(struct domain *d, uint32_t dev_id,
> struct vdevice_table *entry, int set)
> @@ -649,6 +689,482 @@ err:
> return 0;
> }
>
> +static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> + uint32_t offset;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> + uint8_t cfg;
> +
> + offset = info->gpa -
> + (v->domain->arch.vits->propbase & 0xfffffffff000UL);
> +
> + if ( offset < SZ_64K )
> + {
> + DPRINTK("vITS:d%dv%d LPI Table read offset 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, offset);
> + cfg = readb_relaxed(v->domain->arch.vits->prop_page + offset);
> + *r = cfg;
> + return 1;
> + }
> + else
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d LPI Table read with wrong offset 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, offset);
> +
> +
> + return 0;
> +}
> +
> +static int vgic_v3_gits_lpi_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
> + uint32_t offset;
> + uint32_t vid;
> + uint8_t cfg;
> + bool_t enable;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> +
> + offset = info->gpa -
> + (v->domain->arch.vits->propbase & 0xfffffffff000UL);
> +
> + vid = offset + NR_GIC_LPI;
> + if ( offset < SZ_64K )
> + {
> + DPRINTK("vITS:d%dv%d LPI Table write offset 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, offset);
> + cfg = readb_relaxed(v->domain->arch.vits->prop_page + offset);
> + enable = (cfg & *r) & 0x1;
> +
> + if ( !enable )
> + vgic_its_enable_lpis(v, vid, (*r & 0xfc));
> + else
> + vgic_its_disable_lpis(v, vid);
> +
> + /* Update virtual prop page */
> + writeb_relaxed((*r & 0xff),
> + v->domain->arch.vits->prop_page + offset);
> +
> + return 1;
> + }
> + else
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d LPI Table invalid write @ 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, offset);
> +
> + return 0;
> +}
> +
> +static const struct mmio_handler_ops vgic_gits_lpi_mmio_handler = {
> + .read_handler = vgic_v3_gits_lpi_mmio_read,
> + .write_handler = vgic_v3_gits_lpi_mmio_write,
> +};
> +
> +int vgic_its_unmap_lpi_prop(struct vcpu *v)
> +{
> + paddr_t maddr;
> + uint32_t lpi_size;
> + int i;
> +
> + maddr = v->domain->arch.vits->propbase & 0xfffffffff000UL;
> + lpi_size = 1UL << ((v->domain->arch.vits->propbase & 0x1f) + 1);
> +
> + DPRINTK("vITS:d%dv%d Unmap guest LPI conf table maddr 0x%lx lpi_size 0x%x\n",
> + v->domain->domain_id, v->vcpu_id, maddr, lpi_size);
> +
> + if ( lpi_size < SZ_64K )
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d LPI Prop page < 64K\n",
> + v->domain->domain_id, v->vcpu_id);
> + return 0;
> + }
> +
> + /* XXX: As per 4.8.9 each re-distributor shares a common LPI configuration table
> + * So one set of mmio handlers to manage configuration table is enough
> + */
> + for ( i = 0; i < lpi_size / PAGE_SIZE; i++ )
> + guest_physmap_remove_page(v->domain, paddr_to_pfn(maddr),
> + gmfn_to_mfn(v->domain, paddr_to_pfn(maddr)), 0);
> +
> + /* Register mmio handlers for this region */
> + register_mmio_handler(v->domain, &vgic_gits_lpi_mmio_handler,
> + maddr, lpi_size);
> +
> + /* Allocate Virtual LPI Property table */
> + v->domain->arch.vits->prop_page =
> + alloc_xenheap_pages(get_order_from_bytes(lpi_size), 0);
> + if ( !v->domain->arch.vits->prop_page )
> + {
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d Failed to allocate LPI Prop page\n",
> + v->domain->domain_id, v->vcpu_id);
> + return 0;
> + }
> +
> + memset(v->domain->arch.vits->prop_page, 0xa2, lpi_size);
> +
> + return 1;
> +}
> +
> +static inline void vits_spin_lock(struct vgic_its *vits)
> +{
> + spin_lock(&vits->lock);
> +}
> +
> +static inline void vits_spin_unlock(struct vgic_its *vits)
> +{
> + spin_unlock(&vits->lock);
> +}
> +
> +static int vgic_v3_gits_mmio_read(struct vcpu *v, mmio_info_t *info)
> +{
> + struct vgic_its *vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> + uint64_t val = 0;
> + uint32_t index, gits_reg;
> +
> + vits = v->domain->arch.vits;
> +
> + gits_reg = info->gpa - vits->phys_base;
> +
> + if ( gits_reg >= SZ_64K )
> + {
> + gdprintk(XENLOG_G_WARNING,
> + "vITS:d%dv%d unknown gpa read address %"PRIpaddr"\n",
> + v->domain->domain_id, v->vcpu_id, info->gpa);
> + return 0;
> + }
> +
> + switch ( gits_reg )
> + {
> + case GITS_CTLR:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = 0;
> + return 1;
> + case GITS_IIDR:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = 0;
> + return 1;
> + case GITS_TYPER:
> + vits_spin_lock(vits);
> + val = (((v->domain->max_vcpus + 1) << GITS_TYPER_HCC_SHIFT ) |
> + VITS_GITS_DEV_BITS | VITS_GITS_ID_BITS |
> + VITS_GITS_ITT_SIZE | VITS_GITS_DISTRIBUTED |
> + VITS_GITS_PLPIS);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = val;
> + else if ( dabt.size == DABT_WORD )
> + *r = (u32)val;
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_TYPER + 4:
> + if (dabt.size != DABT_WORD ) goto bad_width;
> + vits_spin_lock(vits);
> + val = (((v->domain->max_vcpus + 1) << GITS_TYPER_HCC_SHIFT ) |
> + VITS_GITS_DEV_BITS | VITS_GITS_ID_BITS |
> + VITS_GITS_ITT_SIZE | VITS_GITS_DISTRIBUTED |
> + VITS_GITS_PLPIS);
> + *r = (u32)(val >> 32);
> + vits_spin_unlock(vits);
> + return 1;
> + case 0x0010 ... 0x007c:
> + case 0xc000 ... 0xffcc:
> + /* Implementation defined -- read ignored */
> + dprintk(XENLOG_ERR,
> + "vITS:d%dv%d read unknown 0x000c - 0x007c r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
> + goto read_as_zero;
> + case GITS_CBASER:
> + /* XXX: Only read support 32/64-bit access */
> + vits_spin_lock(vits);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = vits->cmd_base && 0xc7ffffffffffffffUL;
> + else if ( dabt.size == DABT_WORD )
> + *r = (u32)vits->cmd_base;
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CBASER + 4:
> + if (dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + *r = (u32)(vits->cmd_base >> 32);
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CWRITER:
> + /* XXX: Only read support 32/64-bit access */
> + vits_spin_lock(vits);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = vits->cmd_write;
> + else if ( dabt.size == DABT_WORD )
> + *r = (u32)vits->cmd_write;
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CWRITER + 4:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + *r = (u32)(vits->cmd_write >> 32);
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CREADR:
> + /* XXX: Only read support 32/64-bit access */
> + vits_spin_lock(vits);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + *r = vits->cmd_read;
> + else if ( dabt.size == DABT_WORD )
> + *r = (u32)vits->cmd_read;
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CREADR + 4:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + *r = (u32)(vits->cmd_read >> 32);
> + vits_spin_unlock(vits);
> + return 1;
> + case 0x0098 ... 0x009c:
> + case 0x00a0 ... 0x00fc:
> + case 0x0140 ... 0xbffc:
> + /* Reserved -- read ignored */
> + dprintk(XENLOG_ERR,
> + "vITS:d%dv%d read unknown 0x0098-9c or 0x00a0-fc r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
> + goto read_as_zero;
> + case GITS_BASER ... GITS_BASERN:
> + /* Supports only 64-bit access */
> + if ( dabt.size != DABT_DOUBLE_WORD )
> + goto bad_width;
> + if ( (gits_reg % 8) != 0 )
> + goto bad_width;
> + vits_spin_lock(vits);
> + index = (gits_reg - GITS_BASER) / 8;
> + *r = vits->baser[index];
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_PIDR0:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR0_VAL;
> + return 1;
> + case GITS_PIDR1:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR1_VAL;
> + return 1;
> + case GITS_PIDR2:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR2_VAL;
> + return 1;
> + case GITS_PIDR3:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR3_VAL;
> + return 1;
> + case GITS_PIDR4:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = GITS_PIDR4_VAL;
> + return 1;
> + case GITS_PIDR5 ... GITS_PIDR7:
> + goto read_as_zero;
> + default:
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d unhandled read r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
> + return 0;
> + }
> +
> +bad_width:
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d bad read width %d r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.size, dabt.reg, gits_reg);
> + domain_crash_synchronous();
> + return 0;
> +
> +read_as_zero:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + *r = 0;
> + return 1;
> +}
> +
> +#define GITS_BASER_MASK (~((0x7UL << GITS_BASER_TYPE_SHIFT) | \
> + (0xffUL << GITS_BASER_ENTRY_SIZE_SHIFT) | \
> + (0x3UL << GITS_BASER_SHAREABILITY_SHIFT)))
> +
> +static int vgic_v3_gits_mmio_write(struct vcpu *v, mmio_info_t *info)
> +{
> + struct vgic_its *vits;
> + struct hsr_dabt dabt = info->dabt;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + register_t *r = select_user_reg(regs, dabt.reg);
> + int ret;
> + uint32_t index, gits_reg, sz, psz;
> + uint64_t val;
> +
> + vits = v->domain->arch.vits;
> +
> + gits_reg = info->gpa - vits->phys_base;
> +
> + if ( gits_reg >= SZ_64K )
> + {
> + gdprintk(XENLOG_G_WARNING,
> + "vITS:d%dv%d unknown gpa write address %"PRIpaddr"\n",
> + v->domain->domain_id, v->vcpu_id, info->gpa);
> + return 0;
> + }
> + switch ( gits_reg )
> + {
> + case GITS_CTLR:
> + if ( dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + vits->ctrl = *r;
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_IIDR:
> + /* R0 -- write ignored */
> + goto write_ignore;
> + case GITS_TYPER:
> + case GITS_TYPER + 4:
> + /* R0 -- write ignored */
> + goto write_ignore;
> + case 0x0010 ... 0x007c:
> + case 0xc000 ... 0xffcc:
> + /* Implementation defined -- write ignored */
> + dprintk(XENLOG_G_ERR,
> + "vITS:d%dv%d write to unknown 0x000c - 0x007c r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
> + goto write_ignore;
> + case GITS_CBASER:
> + if ( dabt.size != DABT_DOUBLE_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + vits->cmd_base = *r;
> + vits->cmd_qsize = SZ_4K * ((*r & 0xff) + 1);
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_CBASER + 4:
> + /* XXX: Does not support word write */
> + goto bad_width;
> + case GITS_CWRITER:
> + vits_spin_lock(vits);
> + if ( dabt.size == DABT_DOUBLE_WORD )
> + vits->cmd_write = *r;
> + else if ( dabt.size == DABT_WORD)
> + {
> + val = vits->cmd_write & 0xffffffff00000000UL;
> + val = (*r) | val;
> + vits->cmd_write = val;
> + }
> + else
> + {
> + vits_spin_unlock(vits);
> + goto bad_width;
> + }
> + ret = vgic_its_process_cmd(v, vits);
> + vits_spin_unlock(vits);
> + return ret;
> + case GITS_CWRITER + 4:
> + if (dabt.size != DABT_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + val = vits->cmd_write & 0xffffffffUL;
> + val = ((*r & 0xffffffffUL) << 32) | val;
> + vits->cmd_write = val;
> + ret = vgic_its_process_cmd(v, vits);
> + vits_spin_unlock(vits);
> + return ret;
> + case GITS_CREADR:
> + /* R0 -- write ignored */
> + goto write_ignore;
> + case 0x0098 ... 0x009c:
> + case 0x00a0 ... 0x00fc:
> + case 0x0140 ... 0xbffc:
> + /* Reserved -- write ignored */
> + dprintk(XENLOG_G_ERR,
> + "vITS:d%dv%d write to unknown 0x98-9c or 0xa0-fc r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
> + goto write_ignore;
> + case GITS_BASER0:
> + if ( dabt.size != DABT_DOUBLE_WORD )
> + goto bad_width;
> + vits_spin_lock(vits);
> + vits->baser[0] = vits->baser[0] | (GITS_BASER_MASK & *r);
> + vits->dt_ipa = vits->baser[0] & 0xfffffffff000UL;
> + psz = (vits->baser[0] >> GITS_BASER_PAGE_SIZE_SHIFT) &
> + GITS_BASER_PAGE_SIZE_MASK_VAL;
> + if ( psz == GITS_BASER_PAGE_SIZE_4K_VAL )
> + sz = 4;
> + else if ( psz == GITS_BASER_PAGE_SIZE_16K_VAL )
> + sz = 16;
> + else
> + sz = 64;
> +
> + vits->dt_size = (vits->baser[0] & GITS_BASER_PAGES_MASK_VAL)
> + * sz * SZ_1K;
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_BASER1 ... GITS_BASERN:
> + /* Nothing to do with this values. Just store and emulate */
> + if ( dabt.size != DABT_DOUBLE_WORD )
> + goto bad_width;
> + if ( (gits_reg % 8) != 0 )
> + goto bad_width;
> + vits_spin_lock(vits);
> + index = (gits_reg - GITS_BASER) / 8;
> + vits->baser[index] = *r;
> + vits_spin_unlock(vits);
> + return 1;
> + case GITS_PIDR7 ... GITS_PIDR0:
> + /* R0 -- write ignored */
> + goto write_ignore;
> + default:
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d unhandled write r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.reg, gits_reg);
> + return 0;
> + }
> +
> +bad_width:
> + dprintk(XENLOG_G_ERR, "vITS:d%dv%d bad write width %d r%d offset %#08x\n",
> + v->domain->domain_id, v->vcpu_id, dabt.size, dabt.reg, gits_reg);
> + domain_crash_synchronous();
> + return 0;
> +
> +write_ignore:
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + *r = 0;
> + return 1;
> +}
> +
> +
> +static const struct mmio_handler_ops vgic_gits_mmio_handler = {
> + .read_handler = vgic_v3_gits_mmio_read,
> + .write_handler = vgic_v3_gits_mmio_write,
> +};
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 8f898a6..3271477 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -38,6 +38,8 @@ struct its_collection {
> struct vgic_its
> {
> spinlock_t lock;
> + /* Emulation of BASER */
> + paddr_t baser[8];
> /* Command queue base */
> paddr_t cmd_base;
> /* Command queue write pointer */
> @@ -48,8 +50,20 @@ struct vgic_its
> paddr_t cmd_read;
> /* Command queue size */
> unsigned long cmd_qsize;
> + /* ITS mmio physical base */
> + paddr_t phys_base;
> + /* ITS mmio physical size */
> + unsigned long phys_size;
> /* ITS physical node */
> struct its_node *its;
> + /* GICR ctrl register */
> + uint32_t ctrl;
> + /* LPI propbase */
> + paddr_t propbase;
> + /* percpu pendbase */
> + paddr_t pendbase[MAX_VIRT_CPUS];
> + /* Virtual LPI property table */
> + void * prop_page;
> /* vITT device table ipa */
> paddr_t dt_ipa;
> /* vITT device table size */
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 12/18] xen/arm: ITS: Add GICR register emulation
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (10 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 11/18] xen/arm: ITS: Add GITS registers emulation vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-22 12:01 ` [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs vijay.kilari
` (8 subsequent siblings)
20 siblings, 0 replies; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Emulate LPI related changes to GICR registers
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3.c | 15 ++++++++
xen/arch/arm/gic.c | 10 ++++++
xen/arch/arm/vgic-v3.c | 69 ++++++++++++++++++++++++++++++-------
xen/include/asm-arm/domain.h | 1 +
xen/include/asm-arm/gic-its.h | 1 +
xen/include/asm-arm/gic.h | 9 +++++
xen/include/asm-arm/gic_v3_defs.h | 1 +
7 files changed, 94 insertions(+), 12 deletions(-)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 556b291..737646c 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -683,6 +683,11 @@ static int __init gicv3_populate_rdist(void)
return -ENODEV;
}
+static int gicv3_dist_supports_lpis(void)
+{
+ return readl_relaxed(GICD + GICD_TYPER) & GICD_TYPER_LPIS_SUPPORTED;
+}
+
static int __cpuinit gicv3_cpu_init(void)
{
int i;
@@ -1304,10 +1309,20 @@ static int __init gicv3_init(void)
gicv3.rdist_regions[0].size, gicv3.rdist_regions[0].map_base,
gicv3_info.maintenance_irq);
+ reg = readl_relaxed(GICD + GICD_TYPER);
+
+ gicv3.rdist_data.id_bits = ((reg >> 19) & 0x1f) + 1;
+ gicv3_info.nr_id_bits = gicv3.rdist_data.id_bits;
+
spin_lock_init(&gicv3.lock);
spin_lock(&gicv3.lock);
+ if ( gicv3_dist_supports_lpis() )
+ gicv3_info.lpi_supported = 1;
+ else
+ gicv3_info.lpi_supported = 0;
+
gicv3_dist_init();
res = gicv3_cpu_init();
gicv3_hyp_init();
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index c41e82e..cfc9c42 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -67,6 +67,16 @@ unsigned int gic_number_lines(void)
return gic_hw_ops->info->nr_lines;
}
+unsigned int gic_nr_id_bits(void)
+{
+ return gic_hw_ops->info->nr_id_bits;
+}
+
+bool_t gic_lpi_supported(void)
+{
+ return gic_hw_ops->info->lpi_supported;
+}
+
void gic_save_state(struct vcpu *v)
{
ASSERT(!local_irq_is_enabled());
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 4af5a84..44922fb 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -30,6 +30,7 @@
#include <asm/mmio.h>
#include <asm/gic_v3_defs.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
/* GICD_PIDRn register values for ARM implementations */
@@ -93,8 +94,15 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
switch ( gicr_reg )
{
case GICR_CTLR:
- /* We have not implemented LPI's, read zero */
- goto read_as_zero_32;
+ /*
+ * Enable LPI's for ITS. Direct injection of LPI
+ * by writing to GICR_{SET,CLR}LPIR are not supported
+ */
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vgic_lock(v);
+ *r = v->domain->arch.vgic.gicr_ctlr;
+ vgic_unlock(v);
+ return 1;
case GICR_IIDR:
if ( dabt.size != DABT_WORD ) goto bad_width;
*r = GICV3_GICR_IIDR_VAL;
@@ -106,6 +114,10 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 2) << 48 |
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 1) << 40 |
MPIDR_AFFINITY_LEVEL(v->arch.vmpidr, 0) << 32);
+ /* Set LPI support */
+ aff |= (GICR_TYPER_DISTRIBUTED_IMP | GICR_TYPER_PLPIS);
+ /* GITS_TYPER.PTA is 0. Provice vcpu number as ta */
+ aff |= (v->vcpu_id << GICR_TYPER_PROCESSOR_SHIFT);
*r = aff;
if ( v->arch.vgic.flags & VGIC_V3_RDIST_LAST )
@@ -125,11 +137,13 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, mmio_info_t *info,
/* WO. Read as zero */
goto read_as_zero_64;
case GICR_PROPBASER:
- /* LPI's not implemented */
- goto read_as_zero_64;
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ /* Remove shareability attribute we don't want dom to flush */
+ *r = v->domain->arch.vits->propbase;
+ return 1;
case GICR_PENDBASER:
- /* LPI's not implemented */
- goto read_as_zero_64;
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ *r = v->domain->arch.vits->pendbase[v->vcpu_id];
case GICR_INVLPIR:
/* WO. Read as zero */
goto read_as_zero_64;
@@ -203,8 +217,15 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
switch ( gicr_reg )
{
case GICR_CTLR:
- /* LPI's not implemented */
- goto write_ignore_32;
+ /*
+ * Enable LPI's for ITS. Direct injection of LPI
+ * by writing to GICR_{SET,CLR}LPIR are not supported
+ */
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ vgic_lock(v);
+ v->domain->arch.vgic.gicr_ctlr = (*r) & GICR_CTL_ENABLE;
+ vgic_unlock(v);
+ return 1;
case GICR_IIDR:
/* RO */
goto write_ignore_32;
@@ -224,11 +245,27 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
/* LPI is not implemented */
goto write_ignore_64;
case GICR_PROPBASER:
- /* LPI is not implemented */
- goto write_ignore_64;
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ vgic_lock(v);
+ /* LPI configuration tables are shared across cpus. Should be same */
+ if ( (v->domain->arch.vits->propbase != 0) &&
+ ((v->domain->arch.vits->propbase & 0xfffffffff000UL) !=
+ (*r & 0xfffffffff000UL)) )
+ {
+ dprintk(XENLOG_G_ERR,
+ "vGICv3: vITS: Wrong configuration of LPI_PROPBASER\n");
+ return 0;
+ }
+ v->domain->arch.vits->propbase = *r;
+ vgic_unlock(v);
+ return vgic_its_unmap_lpi_prop(v);
case GICR_PENDBASER:
- /* LPI is not implemented */
- goto write_ignore_64;
+ /* Just hold pendbaser value for guest read */
+ if ( dabt.size != DABT_DOUBLE_WORD ) goto bad_width;
+ vgic_lock(v);
+ v->domain->arch.vits->pendbase[v->vcpu_id] = *r;
+ vgic_unlock(v);
+ return 1;
case GICR_INVLPIR:
/* LPI is not implemented */
goto write_ignore_64;
@@ -694,6 +731,14 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
*r = ((ncpus - 1) << GICD_TYPE_CPUS_SHIFT |
DIV_ROUND_UP(v->domain->arch.vgic.nr_spis, 32));
+ if ( gic_lpi_supported() )
+ {
+ irq_bits = gic_nr_id_bits();
+ *r |= GICD_TYPE_LPIS;
+ }
+ else
+ irq_bits = get_count_order(vgic_num_irqs(v->domain));
+
*r |= (irq_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
return 1;
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index da73cf5..db1d1db 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -102,6 +102,7 @@ struct arch_domain
paddr_t dbase; /* Distributor base address */
paddr_t cbase; /* CPU base address */
#ifdef CONFIG_ARM_64
+ int gicr_ctlr;
/* GIC V3 addressing */
paddr_t dbase_size; /* Distributor base size */
/* List of contiguous occupied by the redistributors */
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 3271477..1de57a7 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -277,6 +277,7 @@ static inline uint32_t its_decode_devid(struct domain *d, its_cmd_block *cmd)
void its_set_affinity(struct irq_desc *desc, int cpu);
void lpi_set_config(struct irq_desc *desc, int enable);
uint32_t its_get_nr_events(void);
+int vgic_its_unmap_lpi_prop(struct vcpu *v);
struct vits_device * find_vits_device(struct rb_root *root, uint32_t devid);
int insert_vits_device(struct rb_root *root, struct vits_device *dev);
int remove_vits_device(struct rb_root *root, struct vits_device *dev);
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 0209cc5..ee612de 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -99,6 +99,7 @@
#define GICD_TYPE_CPUS_SHIFT 5
#define GICD_TYPE_CPUS 0x0e0
#define GICD_TYPE_SEC 0x400
+#define GICD_TYPE_LPIS (0x1UL << 17)
#define GICC_CTL_ENABLE 0x1
#define GICC_CTL_EOI (0x1 << 9)
@@ -282,6 +283,10 @@ extern void gic_dump_info(struct vcpu *v);
/* Number of interrupt lines */
extern unsigned int gic_number_lines(void);
+/* Number of interrupt id bits supported */
+extern unsigned int gic_nr_id_bits(void);
+/* LPI support info */
+bool_t gic_lpi_supported(void);
/* IRQ translation function for the device tree */
int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
@@ -299,6 +304,10 @@ struct gic_info {
unsigned int maintenance_irq;
/* Pointer to the device tree node representing the interrupt controller */
const struct dt_device_node *node;
+ /* Number of IRQ ID bits supported */
+ uint32_t nr_id_bits;
+ /* LPIs are support information */
+ bool_t lpi_supported;
};
struct gic_hw_operations {
diff --git a/xen/include/asm-arm/gic_v3_defs.h b/xen/include/asm-arm/gic_v3_defs.h
index dc4fe14..0872097 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -136,6 +136,7 @@
#define GICR_TYPER_VLPIS (1U << 1)
#define GICR_TYPER_DISTRIBUTED_IMP (1U << 3)
#define GICR_TYPER_LAST (1U << 4)
+#define GICR_TYPER_PROCESSOR_SHIFT (8)
#define DEFAULT_PMR_VALUE 0xff
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (11 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 12/18] xen/arm: ITS: Add GICR register emulation vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-29 12:58 ` Ian Campbell
2015-06-22 12:01 ` [RFC PATCH v3 14/18] xen/arm: ITS: Initialize physical ITS vijay.kilari
` (7 subsequent siblings)
20 siblings, 1 reply; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add irq descriptors for LPIs and route
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3.c | 8 +++-
xen/arch/arm/gic.c | 17 +++++++-
xen/arch/arm/irq.c | 38 +++++++++++++----
xen/arch/arm/vgic-v3-its.c | 9 +++++
xen/arch/arm/vgic.c | 90 ++++++++++++++++++++++++++++++++++++++---
xen/include/asm-arm/domain.h | 2 +
xen/include/asm-arm/gic-its.h | 6 +++
xen/include/asm-arm/gic.h | 3 ++
xen/include/asm-arm/vgic.h | 1 +
9 files changed, 157 insertions(+), 17 deletions(-)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 737646c..793f2f0 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -899,9 +899,13 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
val = (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT) | grp;
val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
- val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
- if ( p->desc != NULL )
+ if ( is_lpi(p->irq) )
+ val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
+ else
+ val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
+
+ if ( p->desc != NULL && !(is_lpi(p->irq)) )
val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
<< GICH_LR_PHYSICAL_SHIFT);
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index cfc9c42..091f7e5 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -124,18 +124,31 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
unsigned int priority)
{
ASSERT(priority <= 0xff); /* Only 8 bits of priority */
- ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
+ /* Can't route interrupts that don't exist */
+ ASSERT(desc->irq < gic_number_lines() && is_lpi(desc->irq));
ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
ASSERT(spin_is_locked(&desc->lock));
desc->handler = gic_hw_ops->gic_host_irq_type;
- gic_set_irq_properties(desc, cpu_mask, priority);
+ if ( !is_lpi(desc->irq) )
+ gic_set_irq_properties(desc, cpu_mask, priority);
}
/* Program the GIC to route an interrupt to a guest
* - desc.lock must be held
*/
+int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
+ struct irq_desc *desc, unsigned int priority)
+{
+ ASSERT(spin_is_locked(&desc->lock));
+
+ desc->handler = gic_hw_ops->gic_guest_irq_type;
+ set_bit(_IRQ_GUEST, &desc->status);
+
+ return 0;
+}
+
int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
struct irq_desc *desc, unsigned int priority)
{
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 9dbdf7d..105ef85 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -57,12 +57,22 @@ hw_irq_controller no_irq_type = {
};
static irq_desc_t irq_desc[NR_IRQS];
+static irq_desc_t irq_desc_lpi[MAX_NR_LPIS];
static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
irq_desc_t *__irq_to_desc(int irq)
{
+ struct irq_desc *desc = NULL;
if (irq < NR_LOCAL_IRQS) return &this_cpu(local_irq_desc)[irq];
- return &irq_desc[irq-NR_LOCAL_IRQS];
+ else if ( irq >= NR_LOCAL_IRQS && irq < NR_IRQS)
+ return &irq_desc[irq-NR_LOCAL_IRQS];
+ else
+ {
+ if ( is_lpi(irq) )
+ return &irq_desc_lpi[irq - NR_GIC_LPI];
+ }
+
+ return desc;
}
int __init arch_init_one_irq_desc(struct irq_desc *desc)
@@ -83,6 +93,13 @@ static int __init init_irq_data(void)
desc->action = NULL;
}
+ for ( irq = NR_GIC_LPI; irq < MAX_LPI; irq++ )
+ {
+ struct irq_desc *desc = irq_to_desc(irq);
+ init_one_irq_desc(desc);
+ desc->irq = irq;
+ desc->action = NULL;
+ }
return 0;
}
@@ -178,7 +195,7 @@ int request_irq(unsigned int irq, unsigned int irqflags,
* which interrupt is which (messes up the interrupt freeing
* logic etc).
*/
- if ( irq >= nr_irqs )
+ if ( irq >= nr_irqs && !is_lpi(irq) )
return -EINVAL;
if ( !handler )
return -EINVAL;
@@ -237,9 +254,12 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
set_bit(_IRQ_INPROGRESS, &desc->status);
desc->arch.eoi_cpu = smp_processor_id();
+ if ( is_lpi(irq) )
+ vgic_vcpu_inject_lpi(info->d, irq);
+ else
/* the irq cannot be a PPI, we only support delivery of SPIs to
* guests */
- vgic_vcpu_inject_spi(info->d, info->virq);
+ vgic_vcpu_inject_spi(info->d, info->virq);
goto out_no_end;
}
@@ -405,6 +425,8 @@ err:
bool_t is_assignable_irq(unsigned int irq)
{
+ if ( is_lpi(irq) )
+ return 1;
/* For now, we can only route SPIs to the guest */
return ((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines()));
}
@@ -422,7 +444,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
unsigned long flags;
int retval = 0;
- if ( virq >= vgic_num_irqs(d) )
+ if ( virq >= vgic_num_irqs(d) && !is_lpi(irq) )
{
printk(XENLOG_G_ERR
"the vIRQ number %u is too high for domain %u (max = %u)\n",
@@ -431,7 +453,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
}
/* Only routing to virtual SPIs is supported */
- if ( virq < NR_LOCAL_IRQS )
+ if ( virq < NR_LOCAL_IRQS && !is_lpi(irq) )
{
printk(XENLOG_G_ERR "IRQ can only be routed to an SPI\n");
return -EINVAL;
@@ -507,8 +529,10 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
retval = __setup_irq(desc, 0, action);
if ( retval )
goto out;
-
- retval = gic_route_irq_to_guest(d, virq, desc, GIC_PRI_IRQ);
+ if ( is_lpi(irq) )
+ retval = gic_route_lpi_to_guest(d, virq, desc, GIC_PRI_IRQ);
+ else
+ retval = gic_route_irq_to_guest(d, virq, desc, GIC_PRI_IRQ);
spin_unlock_irqrestore(&desc->lock, flags);
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index fa9dccc..543db91 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -689,6 +689,15 @@ err:
return 0;
}
+uint8_t vgic_its_get_priority(struct vcpu *v, uint32_t pid)
+{
+ uint8_t priority;
+
+ priority = readb_relaxed(v->domain->arch.vits->prop_page + pid);
+ priority &= 0xfc;
+
+ return priority;
+}
static int vgic_v3_gits_lpi_mmio_read(struct vcpu *v, mmio_info_t *info)
{
uint32_t offset;
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 73a6f7e..6074431 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -30,6 +30,7 @@
#include <asm/mmio.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <asm/vgic.h>
static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v, int rank)
@@ -111,6 +112,13 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
for (i=0; i<d->arch.vgic.nr_spis; i++)
vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i], i + 32);
+ d->arch.vgic.pending_lpis = xzalloc_array(struct pending_irq, MAX_NR_LPIS);
+ if ( d->arch.vgic.pending_lpis == NULL )
+ return -ENOMEM;
+
+ for ( i = 0; i < MAX_NR_LPIS; i++ )
+ vgic_init_pending_irq(&d->arch.vgic.pending_lpis[i], i);
+
for (i=0; i<DOMAIN_NR_RANKS(d); i++)
spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
@@ -377,13 +385,18 @@ int vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode, int
struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
{
- struct pending_irq *n;
+ struct pending_irq *n = NULL;
/* Pending irqs allocation strategy: the first vgic.nr_spis irqs
* are used for SPIs; the rests are used for per cpu irqs */
if ( irq < 32 )
n = &v->arch.vgic.pending_irqs[irq];
- else
+ else if ( irq < NR_IRQS )
n = &v->domain->arch.vgic.pending_irqs[irq - 32];
+ else
+ {
+ if ( is_lpi(irq) )
+ n = &v->domain->arch.vgic.pending_lpis[irq - 8192];
+ }
return n;
}
@@ -409,14 +422,20 @@ void vgic_clear_pending_irqs(struct vcpu *v)
void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq)
{
uint8_t priority;
- struct vgic_irq_rank *rank = vgic_rank_irq(v, virq);
+ struct vgic_irq_rank *rank;
struct pending_irq *iter, *n = irq_to_pending(v, virq);
unsigned long flags;
bool_t running;
- vgic_lock_rank(v, rank, flags);
- priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
- vgic_unlock_rank(v, rank, flags);
+ if ( virq < NR_GIC_LPI )
+ {
+ rank = vgic_rank_irq(v, virq);
+ vgic_lock_rank(v, rank, flags);
+ priority = v->domain->arch.vgic.handler->get_irq_priority(v, virq);
+ vgic_unlock_rank(v, rank, flags);
+ }
+ else
+ priority = vgic_its_get_priority(v, virq);
spin_lock_irqsave(&v->arch.vgic.lock, flags);
@@ -473,6 +492,65 @@ void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq)
vgic_vcpu_inject_irq(v, virq);
}
+void vgic_vcpu_inject_lpi(struct domain *d, unsigned int irq)
+{
+ struct irq_desc *desc;
+ struct pending_irq *p;
+ struct its_device *dev;
+ struct vitt vitt_entry;
+ struct vdevice_table dt_entry;
+ uint32_t devid, col_id;
+ int event;
+
+ desc = irq_to_desc(irq);
+ event = irq_to_virq(desc);
+
+ dev = get_irq_device(desc);
+ devid = dev->device_id;
+ event = irq - dev->lpi_base;
+ if ( event < 0 && event > dev->nr_lpis)
+ {
+ dprintk(XENLOG_WARNING,
+ "LPI %d received for dev 0x%x is not valid..dropping \n",
+ irq, devid);
+ return;
+ }
+
+ /* validity of device is checheck on vitt entry request */
+ vits_get_vdevice_entry(d, devid, &dt_entry);
+ if ( dt_entry.vitt_ipa != INVALID_PADDR )
+ {
+ dprintk(XENLOG_WARNING,
+ "LPI %d received for dev 0x%x which is disabled..dropping \n",
+ irq, devid);
+ return;
+ }
+
+ if ( vits_get_vitt_entry(d, devid, event, &vitt_entry) )
+ {
+ dprintk(XENLOG_WARNING,
+ "LPI %d received for dev 0x%x which is disabled..dropping \n",
+ irq, devid);
+ return;
+ }
+ if ( !vitt_entry.valid )
+ {
+ dprintk(XENLOG_WARNING,
+ "LPI %d received for dev 0x%x which is not valid..dropping \n",
+ irq, devid);
+ return;
+ }
+ p = irq_to_pending(d->vcpu[0], vitt_entry.vlpi);
+ col_id = vitt_entry.vcollection;
+
+ ASSERT(col_id < d->max_vcpus);
+
+ p->desc = desc;
+
+ vgic_vcpu_inject_irq(d->vcpu[col_id], vitt_entry.vlpi);
+}
+
+
void arch_evtchn_inject(struct vcpu *v)
{
vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index db1d1db..06105cc 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -98,6 +98,8 @@ struct arch_domain
* struct arch_vcpu.
*/
struct pending_irq *pending_irqs;
+ struct pending_irq *pending_lpis;
+
/* Base address for guest GIC */
paddr_t dbase; /* Distributor base address */
paddr_t cbase; /* CPU base address */
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 1de57a7..f34a207 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -276,11 +276,17 @@ static inline uint32_t its_decode_devid(struct domain *d, its_cmd_block *cmd)
void its_set_affinity(struct irq_desc *desc, int cpu);
void lpi_set_config(struct irq_desc *desc, int enable);
+uint8_t vgic_its_get_priority(struct vcpu *v, uint32_t pid);
+
uint32_t its_get_nr_events(void);
int vgic_its_unmap_lpi_prop(struct vcpu *v);
struct vits_device * find_vits_device(struct rb_root *root, uint32_t devid);
int insert_vits_device(struct rb_root *root, struct vits_device *dev);
int remove_vits_device(struct rb_root *root, struct vits_device *dev);
+int vits_get_vitt_entry(struct domain *d, uint32_t devid,
+ uint32_t event, struct vitt *entry);
+int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
+ struct vdevice_table *entry);
#endif /* __ASM_ARM_GIC_ITS_H__ */
/*
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index ee612de..9e94d65 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -225,6 +225,9 @@ extern void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mas
extern int gic_route_irq_to_guest(struct domain *, unsigned int virq,
struct irq_desc *desc,
unsigned int priority);
+extern int gic_route_lpi_to_guest(struct domain *d, unsigned int virq,
+ struct irq_desc *desc,
+ unsigned int priority);
/* Remove an IRQ passthrough to a guest */
int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 8d22532..f8928ab 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -183,6 +183,7 @@ extern int vcpu_vgic_init(struct vcpu *v);
extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq);
extern void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int virq);
extern void vgic_vcpu_inject_spi(struct domain *d, unsigned int virq);
+extern void vgic_vcpu_inject_lpi(struct domain *d, unsigned int virq);
extern void vgic_clear_pending_irqs(struct vcpu *v);
extern struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq);
extern struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs
2015-06-22 12:01 ` [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs vijay.kilari
@ 2015-06-29 12:58 ` Ian Campbell
2015-06-29 13:11 ` Julien Grall
0 siblings, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 12:58 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add irq descriptors for LPIs and route
This seems to also do interrupt injection for LPIs and more. Please
check that your commit messages are accurately describing the contents
of the patch. If it turns into a long list of unrelated sounding things
then that might suggest the patch should be split up.
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/gic-v3.c | 8 +++-
> xen/arch/arm/gic.c | 17 +++++++-
> xen/arch/arm/irq.c | 38 +++++++++++++----
> xen/arch/arm/vgic-v3-its.c | 9 +++++
> xen/arch/arm/vgic.c | 90 ++++++++++++++++++++++++++++++++++++++---
> xen/include/asm-arm/domain.h | 2 +
> xen/include/asm-arm/gic-its.h | 6 +++
> xen/include/asm-arm/gic.h | 3 ++
> xen/include/asm-arm/vgic.h | 1 +
> 9 files changed, 157 insertions(+), 17 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 737646c..793f2f0 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -899,9 +899,13 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
>
> val = (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT) | grp;
> val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
> - val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>
> - if ( p->desc != NULL )
> + if ( is_lpi(p->irq) )
> + val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
> + else
> + val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
Is there supposed to be something different between these two cases? (Or
am I missing it?)
> +
> + if ( p->desc != NULL && !(is_lpi(p->irq)) )
> val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
> << GICH_LR_PHYSICAL_SHIFT);
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index cfc9c42..091f7e5 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -124,18 +124,31 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> unsigned int priority)
> {
> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
> + /* Can't route interrupts that don't exist */
> + ASSERT(desc->irq < gic_number_lines() && is_lpi(desc->irq));
||, surely? Otherwise doesn't this hit for every possible irq?
> ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
> ASSERT(spin_is_locked(&desc->lock));
>
> desc->handler = gic_hw_ops->gic_host_irq_type;
>
> - gic_set_irq_properties(desc, cpu_mask, priority);
> + if ( !is_lpi(desc->irq) )
> + gic_set_irq_properties(desc, cpu_mask, priority);
I think any lack of support for affinity or priority should be pushed
down into the lower level drivers, rather than encoding that lack of
current support in the pits driver into the generic code.
> int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
> struct irq_desc *desc, unsigned int priority)
> {
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 9dbdf7d..105ef85 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -57,12 +57,22 @@ hw_irq_controller no_irq_type = {
> };
>
> static irq_desc_t irq_desc[NR_IRQS];
> +static irq_desc_t irq_desc_lpi[MAX_NR_LPIS];
> static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
>
> irq_desc_t *__irq_to_desc(int irq)
> {
> + struct irq_desc *desc = NULL;
> if (irq < NR_LOCAL_IRQS) return &this_cpu(local_irq_desc)[irq];
> - return &irq_desc[irq-NR_LOCAL_IRQS];
> + else if ( irq >= NR_LOCAL_IRQS && irq < NR_IRQS)
> + return &irq_desc[irq-NR_LOCAL_IRQS];
> + else
> + {
> + if ( is_lpi(irq) )
> + return &irq_desc_lpi[irq - NR_GIC_LPI];
> + }
> +
> + return desc;
I'd write this as just a chain of "if (...) return", no need for else or
{}s.
> +void vgic_vcpu_inject_lpi(struct domain *d, unsigned int irq)
> +{
> + struct irq_desc *desc;
> + struct pending_irq *p;
> + struct its_device *dev;
> + struct vitt vitt_entry;
> + struct vdevice_table dt_entry;
> + uint32_t devid, col_id;
> + int event;
> +
> + desc = irq_to_desc(irq);
> + event = irq_to_virq(desc);
> +
> + dev = get_irq_device(desc);
> + devid = dev->device_id;
> + event = irq - dev->lpi_base;
> + if ( event < 0 && event > dev->nr_lpis)
> + {
> + dprintk(XENLOG_WARNING,
> + "LPI %d received for dev 0x%x is not valid..dropping \n",
> + irq, devid);
> + return;
> + }
> +
> + /* validity of device is checheck on vitt entry request */
Typo
Also trailing whitespace, please clean that up everywhere.
> + vits_get_vdevice_entry(d, devid, &dt_entry);
> + if ( dt_entry.vitt_ipa != INVALID_PADDR )
> + {
> + dprintk(XENLOG_WARNING,
> + "LPI %d received for dev 0x%x which is disabled..dropping \n",
> + irq, devid);
This drops if the entry is _not_ invalid, is that really right?
> + return;
> + }
> +
> + if ( vits_get_vitt_entry(d, devid, event, &vitt_entry) )
> + {
> + dprintk(XENLOG_WARNING,
> + "LPI %d received for dev 0x%x which is disabled..dropping \n",
> + irq, devid);
> + return;
> + }
> + if ( !vitt_entry.valid )
> + {
> + dprintk(XENLOG_WARNING,
> + "LPI %d received for dev 0x%x which is not valid..dropping \n",
> + irq, devid);
> + return;
> + }
> + p = irq_to_pending(d->vcpu[0], vitt_entry.vlpi);
Need to bounds check vitt_entry.vlpi somewhere, since the guest might
have messed with it.
> + col_id = vitt_entry.vcollection;
> +
> + ASSERT(col_id < d->max_vcpus);
Likewise a guest which writes vitt_entry.vcollection can now crash the
host.
Very great care needs to be taken with both dt_entry and vitt_entry and
all of their fields.
> +int vits_get_vitt_entry(struct domain *d, uint32_t devid,
> + uint32_t event, struct vitt *entry);
> +int vits_get_vdevice_entry(struct domain *d, uint32_t devid,
> + struct vdevice_table *entry);
Please add all prototypes in the same patch as the implementation of the
function, not the addition of the first caller.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs
2015-06-29 12:58 ` Ian Campbell
@ 2015-06-29 13:11 ` Julien Grall
2015-07-07 11:00 ` Vijay Kilari
0 siblings, 1 reply; 88+ messages in thread
From: Julien Grall @ 2015-06-29 13:11 UTC (permalink / raw)
To: Ian Campbell, vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
stefano.stabellini, manish.jaggi
On 29/06/15 13:58, Ian Campbell wrote:
> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>
>> Add irq descriptors for LPIs and route
>
> This seems to also do interrupt injection for LPIs and more. Please
> check that your commit messages are accurately describing the contents
> of the patch. If it turns into a long list of unrelated sounding things
> then that might suggest the patch should be split up.
>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>> ---
>> xen/arch/arm/gic-v3.c | 8 +++-
>> xen/arch/arm/gic.c | 17 +++++++-
>> xen/arch/arm/irq.c | 38 +++++++++++++----
>> xen/arch/arm/vgic-v3-its.c | 9 +++++
>> xen/arch/arm/vgic.c | 90 ++++++++++++++++++++++++++++++++++++++---
>> xen/include/asm-arm/domain.h | 2 +
>> xen/include/asm-arm/gic-its.h | 6 +++
>> xen/include/asm-arm/gic.h | 3 ++
>> xen/include/asm-arm/vgic.h | 1 +
>> 9 files changed, 157 insertions(+), 17 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>> index 737646c..793f2f0 100644
>> --- a/xen/arch/arm/gic-v3.c
>> +++ b/xen/arch/arm/gic-v3.c
>> @@ -899,9 +899,13 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
>>
>> val = (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT) | grp;
>> val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
>> - val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>>
>> - if ( p->desc != NULL )
>> + if ( is_lpi(p->irq) )
>> + val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>> + else
>> + val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>
> Is there supposed to be something different between these two cases? (Or
> am I missing it?)
>
>> +
>> + if ( p->desc != NULL && !(is_lpi(p->irq)) )
>> val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
>> << GICH_LR_PHYSICAL_SHIFT);
>>
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index cfc9c42..091f7e5 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -124,18 +124,31 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
>> unsigned int priority)
>> {
>> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
>> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
>> + /* Can't route interrupts that don't exist */
>> + ASSERT(desc->irq < gic_number_lines() && is_lpi(desc->irq));
>
> ||, surely? Otherwise doesn't this hit for every possible irq?
Aside that, the change in the ASSERT is wrong. The goal of the helper
gic_number_lines is to return the number of IRQs (i.e PPIs, LPIs, SPIs)
present in the hardware. We use in few places to ensure the validity of
the IRQ.
Although, this will require some extra care in places where an interrupt
is assigned to a domain in order to only allow SPIs.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs
2015-06-29 13:11 ` Julien Grall
@ 2015-07-07 11:00 ` Vijay Kilari
2015-07-07 11:16 ` Julien Grall
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-07-07 11:00 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Mon, Jun 29, 2015 at 6:41 PM, Julien Grall <julien.grall@citrix.com> wrote:
> On 29/06/15 13:58, Ian Campbell wrote:
>> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>
>>> Add irq descriptors for LPIs and route
>>
>> This seems to also do interrupt injection for LPIs and more. Please
>> check that your commit messages are accurately describing the contents
>> of the patch. If it turns into a long list of unrelated sounding things
>> then that might suggest the patch should be split up.
>>
>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>> ---
>>> xen/arch/arm/gic-v3.c | 8 +++-
>>> xen/arch/arm/gic.c | 17 +++++++-
>>> xen/arch/arm/irq.c | 38 +++++++++++++----
>>> xen/arch/arm/vgic-v3-its.c | 9 +++++
>>> xen/arch/arm/vgic.c | 90 ++++++++++++++++++++++++++++++++++++++---
>>> xen/include/asm-arm/domain.h | 2 +
>>> xen/include/asm-arm/gic-its.h | 6 +++
>>> xen/include/asm-arm/gic.h | 3 ++
>>> xen/include/asm-arm/vgic.h | 1 +
>>> 9 files changed, 157 insertions(+), 17 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>>> index 737646c..793f2f0 100644
>>> --- a/xen/arch/arm/gic-v3.c
>>> +++ b/xen/arch/arm/gic-v3.c
>>> @@ -899,9 +899,13 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
>>>
>>> val = (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT) | grp;
>>> val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
>>> - val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>>>
>>> - if ( p->desc != NULL )
>>> + if ( is_lpi(p->irq) )
>>> + val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>>> + else
>>> + val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>>
>> Is there supposed to be something different between these two cases? (Or
>> am I missing it?)
>>
>>> +
>>> + if ( p->desc != NULL && !(is_lpi(p->irq)) )
>>> val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
>>> << GICH_LR_PHYSICAL_SHIFT);
>>>
>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>>> index cfc9c42..091f7e5 100644
>>> --- a/xen/arch/arm/gic.c
>>> +++ b/xen/arch/arm/gic.c
>>> @@ -124,18 +124,31 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
>>> unsigned int priority)
>>> {
>>> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
>>> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
>>> + /* Can't route interrupts that don't exist */
>>> + ASSERT(desc->irq < gic_number_lines() && is_lpi(desc->irq));
>>
>> ||, surely? Otherwise doesn't this hit for every possible irq?
>
> Aside that, the change in the ASSERT is wrong. The goal of the helper
> gic_number_lines is to return the number of IRQs (i.e PPIs, LPIs, SPIs)
> present in the hardware. We use in few places to ensure the validity of
> the IRQ.
>
> Although, this will require some extra care in places where an interrupt
> is assigned to a domain in order to only allow SPIs.
Today, gic_number_lines returns number of SPI+PPI supported.
If we want to include number LPIs supported, then we cannot
compare against with irq number because of hole in irq numbers from
1024 - 8196.
Regards
Vijay
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs
2015-07-07 11:00 ` Vijay Kilari
@ 2015-07-07 11:16 ` Julien Grall
2015-07-07 15:50 ` Ian Campbell
0 siblings, 1 reply; 88+ messages in thread
From: Julien Grall @ 2015-07-07 11:16 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 07/07/15 12:00, Vijay Kilari wrote:
> On Mon, Jun 29, 2015 at 6:41 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> On 29/06/15 13:58, Ian Campbell wrote:
>>> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>>
>>>> Add irq descriptors for LPIs and route
>>>
>>> This seems to also do interrupt injection for LPIs and more. Please
>>> check that your commit messages are accurately describing the contents
>>> of the patch. If it turns into a long list of unrelated sounding things
>>> then that might suggest the patch should be split up.
>>>
>>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>>>> ---
>>>> xen/arch/arm/gic-v3.c | 8 +++-
>>>> xen/arch/arm/gic.c | 17 +++++++-
>>>> xen/arch/arm/irq.c | 38 +++++++++++++----
>>>> xen/arch/arm/vgic-v3-its.c | 9 +++++
>>>> xen/arch/arm/vgic.c | 90 ++++++++++++++++++++++++++++++++++++++---
>>>> xen/include/asm-arm/domain.h | 2 +
>>>> xen/include/asm-arm/gic-its.h | 6 +++
>>>> xen/include/asm-arm/gic.h | 3 ++
>>>> xen/include/asm-arm/vgic.h | 1 +
>>>> 9 files changed, 157 insertions(+), 17 deletions(-)
>>>>
>>>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>>>> index 737646c..793f2f0 100644
>>>> --- a/xen/arch/arm/gic-v3.c
>>>> +++ b/xen/arch/arm/gic-v3.c
>>>> @@ -899,9 +899,13 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
>>>>
>>>> val = (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT) | grp;
>>>> val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
>>>> - val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>>>>
>>>> - if ( p->desc != NULL )
>>>> + if ( is_lpi(p->irq) )
>>>> + val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>>>> + else
>>>> + val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
>>>
>>> Is there supposed to be something different between these two cases? (Or
>>> am I missing it?)
>>>
>>>> +
>>>> + if ( p->desc != NULL && !(is_lpi(p->irq)) )
>>>> val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
>>>> << GICH_LR_PHYSICAL_SHIFT);
>>>>
>>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>>>> index cfc9c42..091f7e5 100644
>>>> --- a/xen/arch/arm/gic.c
>>>> +++ b/xen/arch/arm/gic.c
>>>> @@ -124,18 +124,31 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
>>>> unsigned int priority)
>>>> {
>>>> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
>>>> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
>>>> + /* Can't route interrupts that don't exist */
>>>> + ASSERT(desc->irq < gic_number_lines() && is_lpi(desc->irq));
>>>
>>> ||, surely? Otherwise doesn't this hit for every possible irq?
>>
>> Aside that, the change in the ASSERT is wrong. The goal of the helper
>> gic_number_lines is to return the number of IRQs (i.e PPIs, LPIs, SPIs)
>> present in the hardware. We use in few places to ensure the validity of
>> the IRQ.
>>
>> Although, this will require some extra care in places where an interrupt
>> is assigned to a domain in order to only allow SPIs.
>
> Today, gic_number_lines returns number of SPI+PPI supported.
> If we want to include number LPIs supported, then we cannot
> compare against with irq number because of hole in irq numbers from
> 1024 - 8196.
I don't see any problem... All the SPI/PPI reported by the GIC may not
be implemented. This ASSERT is here for a sanity check of the IRQ value.
If the caller decides to pass an unused IRQ then it's own problem.
There is not much reason to not extend gic_number_lines. The common GIC
code should not care whether the IRQ is a LPI, SPI...
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs
2015-07-07 11:16 ` Julien Grall
@ 2015-07-07 15:50 ` Ian Campbell
2015-07-07 15:52 ` Julien Grall
0 siblings, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-07-07 15:50 UTC (permalink / raw)
To: Julien Grall
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Tue, 2015-07-07 at 12:16 +0100, Julien Grall wrote:
> On 07/07/15 12:00, Vijay Kilari wrote:
> > On Mon, Jun 29, 2015 at 6:41 PM, Julien Grall <julien.grall@citrix.com> wrote:
> >> On 29/06/15 13:58, Ian Campbell wrote:
> >>> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> >>>> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> >>>>
> >>>> Add irq descriptors for LPIs and route
> >>>
> >>> This seems to also do interrupt injection for LPIs and more. Please
> >>> check that your commit messages are accurately describing the contents
> >>> of the patch. If it turns into a long list of unrelated sounding things
> >>> then that might suggest the patch should be split up.
> >>>
> >>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> >>>> ---
> >>>> xen/arch/arm/gic-v3.c | 8 +++-
> >>>> xen/arch/arm/gic.c | 17 +++++++-
> >>>> xen/arch/arm/irq.c | 38 +++++++++++++----
> >>>> xen/arch/arm/vgic-v3-its.c | 9 +++++
> >>>> xen/arch/arm/vgic.c | 90 ++++++++++++++++++++++++++++++++++++++---
> >>>> xen/include/asm-arm/domain.h | 2 +
> >>>> xen/include/asm-arm/gic-its.h | 6 +++
> >>>> xen/include/asm-arm/gic.h | 3 ++
> >>>> xen/include/asm-arm/vgic.h | 1 +
> >>>> 9 files changed, 157 insertions(+), 17 deletions(-)
> >>>>
> >>>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> >>>> index 737646c..793f2f0 100644
> >>>> --- a/xen/arch/arm/gic-v3.c
> >>>> +++ b/xen/arch/arm/gic-v3.c
> >>>> @@ -899,9 +899,13 @@ static void gicv3_update_lr(int lr, const struct pending_irq *p,
> >>>>
> >>>> val = (((uint64_t)state & 0x3) << GICH_LR_STATE_SHIFT) | grp;
> >>>> val |= ((uint64_t)p->priority & 0xff) << GICH_LR_PRIORITY_SHIFT;
> >>>> - val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
> >>>>
> >>>> - if ( p->desc != NULL )
> >>>> + if ( is_lpi(p->irq) )
> >>>> + val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
> >>>> + else
> >>>> + val |= ((uint64_t)p->irq & GICH_LR_VIRTUAL_MASK) << GICH_LR_VIRTUAL_SHIFT;
> >>>
> >>> Is there supposed to be something different between these two cases? (Or
> >>> am I missing it?)
> >>>
> >>>> +
> >>>> + if ( p->desc != NULL && !(is_lpi(p->irq)) )
> >>>> val |= GICH_LR_HW | (((uint64_t)p->desc->irq & GICH_LR_PHYSICAL_MASK)
> >>>> << GICH_LR_PHYSICAL_SHIFT);
> >>>>
> >>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> >>>> index cfc9c42..091f7e5 100644
> >>>> --- a/xen/arch/arm/gic.c
> >>>> +++ b/xen/arch/arm/gic.c
> >>>> @@ -124,18 +124,31 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> >>>> unsigned int priority)
> >>>> {
> >>>> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
> >>>> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
> >>>> + /* Can't route interrupts that don't exist */
> >>>> + ASSERT(desc->irq < gic_number_lines() && is_lpi(desc->irq));
> >>>
> >>> ||, surely? Otherwise doesn't this hit for every possible irq?
> >>
> >> Aside that, the change in the ASSERT is wrong. The goal of the helper
> >> gic_number_lines is to return the number of IRQs (i.e PPIs, LPIs, SPIs)
> >> present in the hardware. We use in few places to ensure the validity of
> >> the IRQ.
> >>
> >> Although, this will require some extra care in places where an interrupt
> >> is assigned to a domain in order to only allow SPIs.
> >
> > Today, gic_number_lines returns number of SPI+PPI supported.
> > If we want to include number LPIs supported, then we cannot
> > compare against with irq number because of hole in irq numbers from
> > 1024 - 8196.
>
> I don't see any problem... All the SPI/PPI reported by the GIC may not
> be implemented. This ASSERT is here for a sanity check of the IRQ value.
>
> If the caller decides to pass an unused IRQ then it's own problem.
>
> There is not much reason to not extend gic_number_lines. The common GIC
> code should not care whether the IRQ is a LPI, SPI...
I'd rather we refactored these checks into some sort of is_valid_irq()
helper which could check vs. nr_lines (which remains the
GICD_TYPER.ITLinesNumber based thing) and the number of LPIs separately.
Combining the two risks considering interrupt in the 1025..8191 range as
valid and open coding anything more complex than the existing single <
check in all the relevant places isn't going to scale.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs
2015-07-07 15:50 ` Ian Campbell
@ 2015-07-07 15:52 ` Julien Grall
2015-07-07 16:04 ` Ian Campbell
0 siblings, 1 reply; 88+ messages in thread
From: Julien Grall @ 2015-07-07 15:52 UTC (permalink / raw)
To: Ian Campbell
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 07/07/15 16:50, Ian Campbell wrote:
> I'd rather we refactored these checks into some sort of is_valid_irq()
> helper which could check vs. nr_lines (which remains the
> GICD_TYPER.ITLinesNumber based thing) and the number of LPIs separately.
>
> Combining the two risks considering interrupt in the 1025..8191 range as
> valid and open coding anything more complex than the existing single <
> check in all the relevant places isn't going to scale.
I would be fine with that too. Although, I think we need to add a
comment on top of gic_number_lines to clearly specify this is not
included LPIs.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs
2015-07-07 15:52 ` Julien Grall
@ 2015-07-07 16:04 ` Ian Campbell
0 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-07-07 16:04 UTC (permalink / raw)
To: Julien Grall
Cc: Vijay Kilari, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Tue, 2015-07-07 at 16:52 +0100, Julien Grall wrote:
> On 07/07/15 16:50, Ian Campbell wrote:
> > I'd rather we refactored these checks into some sort of is_valid_irq()
> > helper which could check vs. nr_lines (which remains the
> > GICD_TYPER.ITLinesNumber based thing) and the number of LPIs separately.
> >
> > Combining the two risks considering interrupt in the 1025..8191 range as
> > valid and open coding anything more complex than the existing single <
> > check in all the relevant places isn't going to scale.
>
> I would be fine with that too. Although, I think we need to add a
> comment on top of gic_number_lines to clearly specify this is not
> included LPIs.
Strictly speaking an LPI being message signalled is not associated with
a line, which is why GICD_TYPER.ITLinesNumber also doesn't consider
them. But that's not necessarily all that obvious, so a comment would be
appropriate, yes.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 14/18] xen/arm: ITS: Initialize physical ITS
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (12 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 13/18] xen/arm: ITS: Add irq descriptors for LPIs vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-22 12:01 ` [RFC PATCH v3 15/18] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
` (6 subsequent siblings)
20 siblings, 0 replies; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Initialize physical ITS driver from GIC v3 driver
if LPIs are supported by hardware
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3.c | 7 +++++++
xen/include/asm-arm/gic-its.h | 2 ++
2 files changed, 9 insertions(+)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 793f2f0..18971ed 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -700,6 +700,10 @@ static int __cpuinit gicv3_cpu_init(void)
if ( gicv3_enable_redist() )
return -ENODEV;
+ /* Give LPIs a spin */
+ if ( gicv3_info.lpi_supported )
+ its_cpu_init();
+
/* Set priority on PPI and SGI interrupts */
priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI << 8 |
GIC_PRI_IPI);
@@ -1327,6 +1331,9 @@ static int __init gicv3_init(void)
else
gicv3_info.lpi_supported = 0;
+ if ( gicv3_info.lpi_supported )
+ its_init(&gicv3.rdist_data);
+
gicv3_dist_init();
res = gicv3_cpu_init();
gicv3_hyp_init();
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index f34a207..c674b3f 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -274,6 +274,8 @@ static inline uint32_t its_decode_devid(struct domain *d, its_cmd_block *cmd)
return (cmd->raw_cmd[0] >> 32);
}
+int its_cpu_init(void);
+int its_init(struct rdist_prop *rdist);
void its_set_affinity(struct irq_desc *desc, int cpu);
void lpi_set_config(struct irq_desc *desc, int enable);
uint8_t vgic_its_get_priority(struct vcpu *v, uint32_t pid);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* [RFC PATCH v3 15/18] xen/arm: ITS: Add domain specific ITS initialization
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (13 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 14/18] xen/arm: ITS: Initialize physical ITS vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-29 13:01 ` Ian Campbell
2015-06-22 12:01 ` [RFC PATCH v3 16/18] xen/arm: ITS: Handle LPI interrupts vijay.kilari
` (5 subsequent siblings)
20 siblings, 1 reply; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Add Domain and vcpu specific ITS initialization
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic-v3-its.c | 17 ++++++++++++++++
xen/arch/arm/setup.c | 1 +
xen/arch/arm/vgic-v3-its.c | 45 +++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/vgic-v3.c | 1 +
xen/include/asm-arm/gic-its.h | 3 +++
xen/include/asm-arm/vgic.h | 1 +
6 files changed, 68 insertions(+)
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 4471669..8aa1ec5 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -1234,6 +1234,23 @@ static int its_force_quiescent(void __iomem *base)
}
}
+void its_domain_init(struct domain *d)
+{
+ struct its_node *its;
+
+ if ( is_hardware_domain(d) )
+ {
+ list_for_each_entry(its, &its_nodes, entry)
+ {
+ /* XXX: Assign only first physical ITS address */
+ d->arch.vits->phys_base = its->phys_base;
+ d->arch.vits->phys_size = its->phys_size;
+ break;
+ }
+ }
+ /* TODO: DomU */
+}
+
static int its_probe(struct dt_device_node *node)
{
paddr_t its_addr, its_size;
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 06f8e54..0c1081a 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -776,6 +776,7 @@ void __init start_xen(unsigned long boot_phys_offset,
init_xen_time();
gic_init();
+ vgic_its_init();
p2m_vmid_allocator_init();
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 543db91..00e210f 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -1174,6 +1174,51 @@ static const struct mmio_handler_ops vgic_gits_mmio_handler = {
.write_handler = vgic_v3_gits_mmio_write,
};
+int vgic_its_domain_init(struct domain *d)
+{
+ int i;
+
+ d->arch.vits = xzalloc(struct vgic_its);
+ if ( !d->arch.vits )
+ return -ENOMEM;
+
+ spin_lock_init(&d->arch.vits->lock);
+
+ d->arch.vits->collections = xzalloc_array(struct its_collection,
+ nr_cpu_ids);
+ if ( !d->arch.vits->collections )
+ return -ENOMEM;
+
+ for ( i = 0; i < nr_cpu_ids; i++ )
+ d->arch.vits->collections[i].target_address = ~0UL;
+
+ for ( i = 0; i < GITS_BASER_NR_REGS; i++)
+ {
+ /* XXX: Fix this */
+ if ( i == 0 )
+ d->arch.vits->baser[i] = 0x107000000000c00;
+ else
+ d->arch.vits->baser[i] = 0x0;
+ }
+
+ d->arch.vits->dev_root = RB_ROOT;
+
+ spin_lock_init(&d->arch.vits[i].lock);
+
+ its_domain_init(d);
+ register_mmio_handler(d, &vgic_gits_mmio_handler,
+ d->arch.vits->phys_base,
+ SZ_64K);
+
+ return 0;
+}
+
+void vgic_its_init(void)
+{
+ if ( gic_lpi_supported() )
+ its_lpi_init(gic_nr_id_bits());
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 44922fb..abb1457 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -1186,6 +1186,7 @@ static int vgic_v3_domain_init(struct domain *d)
d->arch.vgic.ctlr = VGICD_CTLR_DEFAULT;
+ vgic_its_domain_init(d);
return 0;
}
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index c674b3f..7e25f5b 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -276,6 +276,9 @@ static inline uint32_t its_decode_devid(struct domain *d, its_cmd_block *cmd)
int its_cpu_init(void);
int its_init(struct rdist_prop *rdist);
+void its_domain_init(struct domain *d);
+int its_lpi_init(u32 id_bits);
+int vgic_its_domain_init(struct domain *d);
void its_set_affinity(struct irq_desc *desc, int cpu);
void lpi_set_config(struct irq_desc *desc, int enable);
uint8_t vgic_its_get_priority(struct vcpu *v, uint32_t pid);
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index f8928ab..4509e9a 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -178,6 +178,7 @@ enum gic_sgi_mode;
#define vgic_num_irqs(d) ((d)->arch.vgic.nr_spis + 32)
extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
+extern void vgic_its_init(void);
extern void domain_vgic_free(struct domain *d);
extern int vcpu_vgic_init(struct vcpu *v);
extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 15/18] xen/arm: ITS: Add domain specific ITS initialization
2015-06-22 12:01 ` [RFC PATCH v3 15/18] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
@ 2015-06-29 13:01 ` Ian Campbell
0 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 13:01 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K, tim,
xen-devel, julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Add Domain and vcpu specific ITS initialization
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/gic-v3-its.c | 17 ++++++++++++++++
> xen/arch/arm/setup.c | 1 +
> xen/arch/arm/vgic-v3-its.c | 45 +++++++++++++++++++++++++++++++++++++++++
> xen/arch/arm/vgic-v3.c | 1 +
> xen/include/asm-arm/gic-its.h | 3 +++
> xen/include/asm-arm/vgic.h | 1 +
> 6 files changed, 68 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 4471669..8aa1ec5 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -1234,6 +1234,23 @@ static int its_force_quiescent(void __iomem *base)
> }
> }
>
> +void its_domain_init(struct domain *d)
> +{
> + struct its_node *its;
> +
> + if ( is_hardware_domain(d) )
> + {
> + list_for_each_entry(its, &its_nodes, entry)
> + {
> + /* XXX: Assign only first physical ITS address */
I think we settled upon rewriting the msi-parent properties in the dts
to all refer to a single vits, in which case putting that vits in the
first pits' hole seem ok.
Rather than an XXX (which is a TODO) a comment explaining what is going
on would be preferable.
Ian
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 16/18] xen/arm: ITS: Handle LPI interrupts
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (14 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 15/18] xen/arm: ITS: Add domain specific ITS initialization vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-29 13:03 ` Ian Campbell
2015-06-22 12:01 ` [RFC PATCH v3 17/18] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
` (4 subsequent siblings)
20 siblings, 1 reply; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Handle and route LPI interrupts
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/gic.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 091f7e5..802f82f 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -665,6 +665,11 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
do {
/* Reading IRQ will ACK it */
irq = gic_hw_ops->read_irq();
+ if ( is_lpi(irq) ) {
+ /* TODO: Enable irqs */
+ do_IRQ(regs, irq, is_fiq);
+ continue;
+ }
if ( likely(irq >= 16 && irq < 1020) )
{
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 16/18] xen/arm: ITS: Handle LPI interrupts
2015-06-22 12:01 ` [RFC PATCH v3 16/18] xen/arm: ITS: Handle LPI interrupts vijay.kilari
@ 2015-06-29 13:03 ` Ian Campbell
0 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 13:03 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K, tim,
xen-devel, julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Handle and route LPI interrupts
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/gic.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 091f7e5..802f82f 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -665,6 +665,11 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
> do {
> /* Reading IRQ will ACK it */
> irq = gic_hw_ops->read_irq();
> + if ( is_lpi(irq) ) {
> + /* TODO: Enable irqs */
What was the problem with just doing that?
> + do_IRQ(regs, irq, is_fiq);
> + continue;
Either this should be folded with the SPI/PPI handling case below or it
should be part of the same if/if else/else chain as the other cases.
Ideally the former (i.e. fix the interrupt enabling problem).
> + }
>
> if ( likely(irq >= 16 && irq < 1020) )
> {
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 17/18] xen/arm: ITS: Generate ITS node for Dom0
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (15 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 16/18] xen/arm: ITS: Handle LPI interrupts vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-29 13:06 ` Ian Campbell
2015-06-22 12:01 ` [RFC PATCH v3 18/18] xen/arm: ITS: Map ITS translation space vijay.kilari
` (3 subsequent siblings)
20 siblings, 1 reply; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Parse host dt and generate ITS node for Dom0.
ITS node resides inside GIC node so when GIC node
is encountered look for ITS node.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/domain_build.c | 50 +++++++++++++++++++++++++++++++++++-
xen/arch/arm/gic-v3-its.c | 57 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-arm/gic-its.h | 2 ++
3 files changed, 108 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index e9cb8a9..0de5a8b 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -20,6 +20,7 @@
#include <asm/cpufeature.h>
#include <asm/gic.h>
+#include <asm/gic-its.h>
#include <xen/irq.h>
#include "kernel.h"
@@ -803,6 +804,34 @@ static int make_cpus_node(const struct domain *d, void *fdt,
return res;
}
+static int make_its_node(const struct domain *d, void *fdt,
+ const struct dt_device_node *node)
+{
+ int res = 0;
+
+ DPRINT("Create GIC ITS node\n");
+
+ res = its_make_dt_node(d, node, fdt);
+ if ( res )
+ return res;
+
+ /*
+ * The value of the property "phandle" in the property "interrupts"
+ * to know on which interrupt controller the interrupt is wired.
+ */
+ if ( node->phandle )
+ {
+ DPRINT(" Set phandle = 0x%x\n", node->phandle);
+ res = fdt_property_cell(fdt, "phandle", node->phandle);
+ if ( res )
+ return res;
+ }
+
+ res = fdt_end_node(fdt);
+
+ return res;
+}
+
static int make_gic_node(const struct domain *d, void *fdt,
const struct dt_device_node *node)
{
@@ -1119,7 +1148,13 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
DT_MATCH_TIMER,
{ /* sentinel */ },
};
+ static const struct dt_device_match gits_matches[] __initconst =
+ {
+ DT_MATCH_GIC_ITS,
+ { /* sentinel */ },
+ };
struct dt_device_node *child;
+ struct dt_device_node *gic_child;
int res;
const char *name;
const char *path;
@@ -1143,7 +1178,20 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
/* Replace these nodes with our own. Note that the original may be
* used_by DOMID_XEN so this check comes first. */
if ( device_get_class(node) == DEVICE_GIC )
- return make_gic_node(d, kinfo->fdt, node);
+ {
+ if ( !make_gic_node(d, kinfo->fdt, node) )
+ {
+ dt_for_each_child_node(node, gic_child)
+ {
+ if ( gic_child != NULL )
+ {
+ if ( dt_match_node(gits_matches, gic_child) )
+ return make_its_node(d, kinfo->fdt, gic_child);
+ }
+ }
+ }
+ return 0;
+ }
if ( dt_match_node(timer_matches, node) )
return make_timer_node(d, kinfo->fdt, node);
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 8aa1ec5..fc853d4 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -27,6 +27,8 @@
#include <xen/sched.h>
#include <xen/errno.h>
#include <xen/delay.h>
+#include <xen/device_tree.h>
+#include <xen/libfdt/libfdt.h>
#include <xen/list.h>
#include <xen/sizes.h>
#include <xen/vmap.h>
@@ -1205,6 +1207,61 @@ static void its_cpu_init_collection(void)
spin_unlock(&its_lock);
}
+int its_make_dt_node(const struct domain *d,
+ const struct dt_device_node *node, void *fdt)
+{
+ struct its_node *its;
+ const struct dt_device_node *gic;
+ const void *compatible = NULL;
+ u32 len;
+ __be32 *new_cells, *tmp;
+ int res = 0;
+
+ /* Will pass only first ITS node info */
+ /* TODO: Handle multi node */
+ its = list_first_entry(&its_nodes, struct its_node, entry);
+ if ( !its )
+ {
+ dprintk(XENLOG_ERR, "ITS node not found\n");
+ return -FDT_ERR_XEN(ENOENT);
+ }
+
+ gic = its->dt_node;
+
+ compatible = dt_get_property(gic, "compatible", &len);
+ if ( !compatible )
+ {
+ dprintk(XENLOG_ERR, "Can't find compatible property for the its node\n");
+ return -FDT_ERR_XEN(ENOENT);
+ }
+
+ res = fdt_begin_node(fdt, "gic-its");
+ if ( res )
+ return res;
+
+ res = fdt_property(fdt, "compatible", compatible, len);
+ if ( res )
+ return res;
+
+ res = fdt_property(fdt, "msi-controller", NULL, 0);
+ if ( res )
+ return res;
+
+ len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
+
+ new_cells = xzalloc_bytes(len);
+ if ( new_cells == NULL )
+ return -FDT_ERR_XEN(ENOMEM);
+ tmp = new_cells;
+
+ dt_set_range(&tmp, node, its->phys_base, its->phys_size);
+
+ res = fdt_property(fdt, "reg", new_cells, len);
+ xfree(new_cells);
+
+ return res;
+}
+
static int its_force_quiescent(void __iomem *base)
{
u32 count = 1000000; /* 1s */
diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
index 7e25f5b..c4c2a07 100644
--- a/xen/include/asm-arm/gic-its.h
+++ b/xen/include/asm-arm/gic-its.h
@@ -274,6 +274,8 @@ static inline uint32_t its_decode_devid(struct domain *d, its_cmd_block *cmd)
return (cmd->raw_cmd[0] >> 32);
}
+int its_make_dt_node(const struct domain *d,
+ const struct dt_device_node *node, void *fdt);
int its_cpu_init(void);
int its_init(struct rdist_prop *rdist);
void its_domain_init(struct domain *d);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 17/18] xen/arm: ITS: Generate ITS node for Dom0
2015-06-22 12:01 ` [RFC PATCH v3 17/18] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
@ 2015-06-29 13:06 ` Ian Campbell
2015-07-07 5:31 ` Vijay Kilari
0 siblings, 1 reply; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 13:06 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K, tim,
xen-devel, julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> Parse host dt and generate ITS node for Dom0.
> ITS node resides inside GIC node so when GIC node
> is encountered look for ITS node.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> ---
> xen/arch/arm/domain_build.c | 50 +++++++++++++++++++++++++++++++++++-
> xen/arch/arm/gic-v3-its.c | 57 +++++++++++++++++++++++++++++++++++++++++
> xen/include/asm-arm/gic-its.h | 2 ++
> 3 files changed, 108 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index e9cb8a9..0de5a8b 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -20,6 +20,7 @@
> #include <asm/cpufeature.h>
>
> #include <asm/gic.h>
> +#include <asm/gic-its.h>
> #include <xen/irq.h>
> #include "kernel.h"
>
> @@ -803,6 +804,34 @@ static int make_cpus_node(const struct domain *d, void *fdt,
> return res;
> }
>
> +static int make_its_node(const struct domain *d, void *fdt,
> + const struct dt_device_node *node)
> +{
> + int res = 0;
> +
> + DPRINT("Create GIC ITS node\n");
> +
> + res = its_make_dt_node(d, node, fdt);
> + if ( res )
> + return res;
> +
> + /*
> + * The value of the property "phandle" in the property "interrupts"
> + * to know on which interrupt controller the interrupt is wired.
> + */
> + if ( node->phandle )
> + {
> + DPRINT(" Set phandle = 0x%x\n", node->phandle);
> + res = fdt_property_cell(fdt, "phandle", node->phandle);
> + if ( res )
> + return res;
> + }
> +
> + res = fdt_end_node(fdt);
> +
> + return res;
> +}
> +
> static int make_gic_node(const struct domain *d, void *fdt,
> const struct dt_device_node *node)
> {
> @@ -1119,7 +1148,13 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
> DT_MATCH_TIMER,
> { /* sentinel */ },
> };
> + static const struct dt_device_match gits_matches[] __initconst =
> + {
> + DT_MATCH_GIC_ITS,
> + { /* sentinel */ },
> + };
> struct dt_device_node *child;
> + struct dt_device_node *gic_child;
> int res;
> const char *name;
> const char *path;
> @@ -1143,7 +1178,20 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
> /* Replace these nodes with our own. Note that the original may be
> * used_by DOMID_XEN so this check comes first. */
> if ( device_get_class(node) == DEVICE_GIC )
> - return make_gic_node(d, kinfo->fdt, node);
> + {
> + if ( !make_gic_node(d, kinfo->fdt, node) )
> + {
> + dt_for_each_child_node(node, gic_child)
> + {
> + if ( gic_child != NULL )
> + {
> + if ( dt_match_node(gits_matches, gic_child) )
> + return make_its_node(d, kinfo->fdt, gic_child);
This will create multiple ITS nodes, we only want one and we need the
msi-parent properties everywhere else to be changed to point to it.
> + }
> + }
> + }
> + return 0;
> + }
> if ( dt_match_node(timer_matches, node) )
> return make_timer_node(d, kinfo->fdt, node);
>
> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
> index 8aa1ec5..fc853d4 100644
> --- a/xen/arch/arm/gic-v3-its.c
> +++ b/xen/arch/arm/gic-v3-its.c
> @@ -27,6 +27,8 @@
> #include <xen/sched.h>
> #include <xen/errno.h>
> #include <xen/delay.h>
> +#include <xen/device_tree.h>
> +#include <xen/libfdt/libfdt.h>
> #include <xen/list.h>
> #include <xen/sizes.h>
> #include <xen/vmap.h>
> @@ -1205,6 +1207,61 @@ static void its_cpu_init_collection(void)
> spin_unlock(&its_lock);
> }
>
> +int its_make_dt_node(const struct domain *d,
> + const struct dt_device_node *node, void *fdt)
> +{
> + struct its_node *its;
> + const struct dt_device_node *gic;
> + const void *compatible = NULL;
> + u32 len;
> + __be32 *new_cells, *tmp;
> + int res = 0;
> +
> + /* Will pass only first ITS node info */
> + /* TODO: Handle multi node */
> + its = list_first_entry(&its_nodes, struct its_node, entry);
I think this should be done at the top level by not walking all children
and by blacklisting all ITS nodes to be replaced by our own.
It's also not clear why list_first_entry should be NULL for all but the
first, but nevermind.
> + if ( !its )
> + {
> + dprintk(XENLOG_ERR, "ITS node not found\n");
> + return -FDT_ERR_XEN(ENOENT);
> + }
> +
> + gic = its->dt_node;
> +
> + compatible = dt_get_property(gic, "compatible", &len);
> + if ( !compatible )
> + {
> + dprintk(XENLOG_ERR, "Can't find compatible property for the its node\n");
> + return -FDT_ERR_XEN(ENOENT);
> + }
> +
> + res = fdt_begin_node(fdt, "gic-its");
> + if ( res )
> + return res;
> +
> + res = fdt_property(fdt, "compatible", compatible, len);
> + if ( res )
> + return res;
> +
> + res = fdt_property(fdt, "msi-controller", NULL, 0);
> + if ( res )
> + return res;
> +
> + len = dt_cells_to_size(dt_n_addr_cells(node) + dt_n_size_cells(node));
> +
> + new_cells = xzalloc_bytes(len);
> + if ( new_cells == NULL )
> + return -FDT_ERR_XEN(ENOMEM);
> + tmp = new_cells;
> +
> + dt_set_range(&tmp, node, its->phys_base, its->phys_size);
> +
> + res = fdt_property(fdt, "reg", new_cells, len);
> + xfree(new_cells);
> +
> + return res;
> +}
> +
> static int its_force_quiescent(void __iomem *base)
> {
> u32 count = 1000000; /* 1s */
> diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h
> index 7e25f5b..c4c2a07 100644
> --- a/xen/include/asm-arm/gic-its.h
> +++ b/xen/include/asm-arm/gic-its.h
> @@ -274,6 +274,8 @@ static inline uint32_t its_decode_devid(struct domain *d, its_cmd_block *cmd)
> return (cmd->raw_cmd[0] >> 32);
> }
>
> +int its_make_dt_node(const struct domain *d,
> + const struct dt_device_node *node, void *fdt);
> int its_cpu_init(void);
> int its_init(struct rdist_prop *rdist);
> void its_domain_init(struct domain *d);
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 17/18] xen/arm: ITS: Generate ITS node for Dom0
2015-06-29 13:06 ` Ian Campbell
@ 2015-07-07 5:31 ` Vijay Kilari
2015-07-07 8:21 ` Julien Grall
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-07-07 5:31 UTC (permalink / raw)
To: Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Julien Grall, Stefano Stabellini, manish.jaggi
On Mon, Jun 29, 2015 at 6:36 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>>
>> +int its_make_dt_node(const struct domain *d,
>> + const struct dt_device_node *node, void *fdt)
>> +{
>> + struct its_node *its;
>> + const struct dt_device_node *gic;
>> + const void *compatible = NULL;
>> + u32 len;
>> + __be32 *new_cells, *tmp;
>> + int res = 0;
>> +
>> + /* Will pass only first ITS node info */
>> + /* TODO: Handle multi node */
>> + its = list_first_entry(&its_nodes, struct its_node, entry);
>
> I think this should be done at the top level by not walking all children
> and by blacklisting all ITS nodes to be replaced by our own.
I have added TODO for multinode.
For now, I plan to support only single ITS node. I don't have
platform support right now to test multinode
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 17/18] xen/arm: ITS: Generate ITS node for Dom0
2015-07-07 5:31 ` Vijay Kilari
@ 2015-07-07 8:21 ` Julien Grall
2015-07-07 10:25 ` Vijay Kilari
0 siblings, 1 reply; 88+ messages in thread
From: Julien Grall @ 2015-07-07 8:21 UTC (permalink / raw)
To: Vijay Kilari, Ian Campbell
Cc: Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K, Tim Deegan,
xen-devel, Stefano Stabellini, manish.jaggi
Hi,
On 07/07/2015 06:31, Vijay Kilari wrote:
> On Mon, Jun 29, 2015 at 6:36 PM, Ian Campbell <ian.campbell@citrix.com> wrote:
>> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>
>>>
>>> +int its_make_dt_node(const struct domain *d,
>>> + const struct dt_device_node *node, void *fdt)
>>> +{
>>> + struct its_node *its;
>>> + const struct dt_device_node *gic;
>>> + const void *compatible = NULL;
>>> + u32 len;
>>> + __be32 *new_cells, *tmp;
>>> + int res = 0;
>>> +
>>> + /* Will pass only first ITS node info */
>>> + /* TODO: Handle multi node */
>>> + its = list_first_entry(&its_nodes, struct its_node, entry);
>>
>> I think this should be done at the top level by not walking all children
>> and by blacklisting all ITS nodes to be replaced by our own.
>
> I have added TODO for multinode.
> For now, I plan to support only single ITS node. I don't have
> platform support right now to test multinode
I'm not sure to understand what you want to support multi-node here...
This function is used to create the ITS node for DOM0. From the design
document, guest domain only sees a single node.
Blacklisted ITS node in the top level function is just one line (you
have to give the compatible string)>
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 17/18] xen/arm: ITS: Generate ITS node for Dom0
2015-07-07 8:21 ` Julien Grall
@ 2015-07-07 10:25 ` Vijay Kilari
2015-07-07 11:07 ` Julien Grall
0 siblings, 1 reply; 88+ messages in thread
From: Vijay Kilari @ 2015-07-07 10:25 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On Tue, Jul 7, 2015 at 1:51 PM, Julien Grall <julien.grall@citrix.com> wrote:
> Hi,
>
>
> On 07/07/2015 06:31, Vijay Kilari wrote:
>>
>> On Mon, Jun 29, 2015 at 6:36 PM, Ian Campbell <ian.campbell@citrix.com>
>> wrote:
>>>
>>> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>>
>>
>>>>
>>>> +int its_make_dt_node(const struct domain *d,
>>>> + const struct dt_device_node *node, void *fdt)
>>>> +{
>>>> + struct its_node *its;
>>>> + const struct dt_device_node *gic;
>>>> + const void *compatible = NULL;
>>>> + u32 len;
>>>> + __be32 *new_cells, *tmp;
>>>> + int res = 0;
>>>> +
>>>> + /* Will pass only first ITS node info */
>>>> + /* TODO: Handle multi node */
>>>> + its = list_first_entry(&its_nodes, struct its_node, entry);
>>>
>>>
>>> I think this should be done at the top level by not walking all children
>>> and by blacklisting all ITS nodes to be replaced by our own.
>>
>>
>> I have added TODO for multinode.
>> For now, I plan to support only single ITS node. I don't have
>> platform support right now to test multinode
>
>
> I'm not sure to understand what you want to support multi-node here...
>
> This function is used to create the ITS node for DOM0. From the design
> document, guest domain only sees a single node.
>
> Blacklisted ITS node in the top level function is just one line (you have to
> give the compatible string)>
AIUI, blacklisting ITS node mean, skipping the node getting generated for dom0
by adding compatible string to skip_matches[] list in handle_node()
function of domain_build.c right?
If so, ITS node resides within GIC node, I don't think adding to
skip_matches[] list skips it.
So In the new patch, within GIC node I have checked for child nodes
which matches to ITS compatible
string and generated only one node for Dom0 as below.
if ( device_get_class(node) == DEVICE_GIC )
{
if ( !make_gic_node(d, kinfo->fdt, node) )
{
res = 0;
dt_for_each_child_node(node, gic_child)
{
if ( gic_child != NULL )
{
if ( dt_match_node(gits_matches, gic_child) )
{
res = make_its_node(d, kinfo->fdt, gic_child);
break;
}
}
}
return res;
}
return 0;
}
However if DT has more than one ITS node, then all references in the
DT (PCI nodes)
should be changed to pointed to single (generated) ITS node. This is
what I meant
for TODO. May be TODO is placed wrongly in its driver
Regards
Vijay
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 17/18] xen/arm: ITS: Generate ITS node for Dom0
2015-07-07 10:25 ` Vijay Kilari
@ 2015-07-07 11:07 ` Julien Grall
0 siblings, 0 replies; 88+ messages in thread
From: Julien Grall @ 2015-07-07 11:07 UTC (permalink / raw)
To: Vijay Kilari
Cc: Ian Campbell, Stefano Stabellini, Prasun Kapoor, Vijaya Kumar K,
Tim Deegan, xen-devel, Stefano Stabellini, manish.jaggi
On 07/07/15 11:25, Vijay Kilari wrote:
> On Tue, Jul 7, 2015 at 1:51 PM, Julien Grall <julien.grall@citrix.com> wrote:
>> Hi,
>>
>>
>> On 07/07/2015 06:31, Vijay Kilari wrote:
>>>
>>> On Mon, Jun 29, 2015 at 6:36 PM, Ian Campbell <ian.campbell@citrix.com>
>>> wrote:
>>>>
>>>> On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
>>>
>>>
>>>>>
>>>>> +int its_make_dt_node(const struct domain *d,
>>>>> + const struct dt_device_node *node, void *fdt)
>>>>> +{
>>>>> + struct its_node *its;
>>>>> + const struct dt_device_node *gic;
>>>>> + const void *compatible = NULL;
>>>>> + u32 len;
>>>>> + __be32 *new_cells, *tmp;
>>>>> + int res = 0;
>>>>> +
>>>>> + /* Will pass only first ITS node info */
>>>>> + /* TODO: Handle multi node */
>>>>> + its = list_first_entry(&its_nodes, struct its_node, entry);
>>>>
>>>>
>>>> I think this should be done at the top level by not walking all children
>>>> and by blacklisting all ITS nodes to be replaced by our own.
>>>
>>>
>>> I have added TODO for multinode.
>>> For now, I plan to support only single ITS node. I don't have
>>> platform support right now to test multinode
>>
>>
>> I'm not sure to understand what you want to support multi-node here...
>>
>> This function is used to create the ITS node for DOM0. From the design
>> document, guest domain only sees a single node.
>>
>> Blacklisted ITS node in the top level function is just one line (you have to
>> give the compatible string)>
>
> AIUI, blacklisting ITS node mean, skipping the node getting generated for dom0
> by adding compatible string to skip_matches[] list in handle_node()
> function of domain_build.c right?
Yes.
> If so, ITS node resides within GIC node, I don't think adding to
> skip_matches[] list skips it.
In the case of the GIC node, we skip all the childs so ITS node are
already blacklisted by default.
You can create the ITS node in the hwdom_dt_node callback. I don't see
why you need to browse all the ITS nodes in the function handle_device,
which is common to every one.
> However if DT has more than one ITS node, then all references in the
> DT (PCI nodes)
> should be changed to pointed to single (generated) ITS node. This is
> what I meant
> for TODO. May be TODO is placed wrongly in its driver
The changes in only a couple of lines in write_properties. If you don't
plan to handle multiple ITS in your series, you need to add a big
warning/panic rather letting Xen booting on it fine.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread
* [RFC PATCH v3 18/18] xen/arm: ITS: Map ITS translation space
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (16 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 17/18] xen/arm: ITS: Generate ITS node for Dom0 vijay.kilari
@ 2015-06-22 12:01 ` vijay.kilari
2015-06-22 13:52 ` [RFC PATCH v3 00/18] Add ITS support Julien Grall
` (2 subsequent siblings)
20 siblings, 0 replies; 88+ messages in thread
From: vijay.kilari @ 2015-06-22 12:01 UTC (permalink / raw)
To: Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi, vijay.kilari
From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
ITS translation space contains GITS_TRANSLATOR
register which is written by device to raise
LPI. This space needs to mapped to every domain
address space for all physical ITS available,
so that device can access GITS_TRANSLATOR
register using SMMU.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
---
xen/arch/arm/vgic-v3-its.c | 32 +++++++++++++++++++++++++++++++-
1 file changed, 31 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c
index 00e210f..7cf029a 100644
--- a/xen/arch/arm/vgic-v3-its.c
+++ b/xen/arch/arm/vgic-v3-its.c
@@ -1174,6 +1174,36 @@ static const struct mmio_handler_ops vgic_gits_mmio_handler = {
.write_handler = vgic_v3_gits_mmio_write,
};
+/*
+ * Map the 64K ITS translation space in guest.
+ * This is required purely for device smmu writes.
+*/
+
+//static int vgic_map_translation_space(uint32_t nr_its, struct domain *d)
+static int vgic_map_translation_space(struct domain *d)
+{
+ uint64_t addr, size;
+ int ret;
+
+ addr = d->arch.vits->phys_base + SZ_64K;
+ size = SZ_64K;
+
+ ret = map_mmio_regions(d,
+ paddr_to_pfn(addr & PAGE_MASK),
+ DIV_ROUND_UP(size, PAGE_SIZE),
+ paddr_to_pfn(addr & PAGE_MASK));
+
+ if ( ret )
+ {
+ dprintk(XENLOG_G_ERR, "vITS: Unable to map to dom%d access to"
+ " 0x%"PRIx64" - 0x%"PRIx64"\n",
+ d->domain_id,
+ addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
+ }
+
+ return ret;
+}
+
int vgic_its_domain_init(struct domain *d)
{
int i;
@@ -1210,7 +1240,7 @@ int vgic_its_domain_init(struct domain *d)
d->arch.vits->phys_base,
SZ_64K);
- return 0;
+ return vgic_map_translation_space(d);
}
void vgic_its_init(void)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 00/18] Add ITS support
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (17 preceding siblings ...)
2015-06-22 12:01 ` [RFC PATCH v3 18/18] xen/arm: ITS: Map ITS translation space vijay.kilari
@ 2015-06-22 13:52 ` Julien Grall
2015-06-22 13:56 ` Ian Campbell
2015-06-24 10:02 ` Ian Campbell
2015-06-29 13:11 ` Ian Campbell
20 siblings, 1 reply; 88+ messages in thread
From: Julien Grall @ 2015-06-22 13:52 UTC (permalink / raw)
To: vijay.kilari, Ian.Campbell, julien.grall, stefano.stabellini,
stefano.stabellini, tim, xen-devel
Cc: Prasun.Kapoor, Vijaya Kumar K, manish.jaggi
Hi Vijay,
On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> This is based on DraftF version
> http://xenbits.xen.org/people/ianc/vits/draftF.pdf
>
> Following major features are supported
> - GICv3 ITS support for arm64 platform
> - Only Dom0 is supported
>
> Basic boot is tested with single ITS node with hacks
> in pci driver.
What about a proper support without hack?
> All Comments from V2 version are not
> fixed as this series is based on new design.
I agree that any comments on code that is not present anymore should be
skipped. But all the others should have been fixed and it's not the case.
The time I will spend to find the same errors and write the same
comments could have been spend more usefully to review the new design...
Please help us to speed up the review and make possible to merge ITS
support in Xen 4.6.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 00/18] Add ITS support
2015-06-22 13:52 ` [RFC PATCH v3 00/18] Add ITS support Julien Grall
@ 2015-06-22 13:56 ` Ian Campbell
0 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-22 13:56 UTC (permalink / raw)
To: Julien Grall
Cc: vijay.kilari, stefano.stabellini, Prasun.Kapoor, Vijaya Kumar K,
tim, xen-devel, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 14:52 +0100, Julien Grall wrote:
> Hi Vijay,
>
> On 22/06/15 13:01, vijay.kilari@gmail.com wrote:
> > From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
> >
> > This is based on DraftF version
> > http://xenbits.xen.org/people/ianc/vits/draftF.pdf
> >
> > Following major features are supported
> > - GICv3 ITS support for arm64 platform
> > - Only Dom0 is supported
> >
> > Basic boot is tested with single ITS node with hacks
> > in pci driver.
>
> What about a proper support without hack?
I asked Vijay to send an RFC without waiting for that.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [RFC PATCH v3 00/18] Add ITS support
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (18 preceding siblings ...)
2015-06-22 13:52 ` [RFC PATCH v3 00/18] Add ITS support Julien Grall
@ 2015-06-24 10:02 ` Ian Campbell
2015-06-29 13:11 ` Ian Campbell
20 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-24 10:02 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
>
> This is based on DraftF version
> http://xenbits.xen.org/people/ianc/vits/draftF.pdf
I had some local edits due to comments made on IRC and in response to
questions you asked while implementing etc. It's nothing major and I
don't think it will have an impact on the implementation but rather than
have it sitting on my HDD I've stuck it at
http://xenbits.xen.org/people/ianc/vits/draftG.pdf and below.
Ian.
% Xen on ARM vITS Handling
% Ian Campbell <ian.campbell@citrix.com>
% Draft G
# Changelog
## Since Draft F
* Fix wording in `INV` and `INVALL` cases.
* Drop attempt at preemptible `GITS_CWRITER` handling.
* Clarify physical vs virtual collections and physical vs virtual LPIs
in some places.
## Since Draft E
* Discussion of `struct pending_irq`
* Fix handling of enable/disable, requiring switching back to trapping
the virtual cfg table again. get_vlpi_cfg is no longer needed.
* Fix p2m_lookup to also use get_page_from_gfn.
## Since Draft D
* Fixed assumptions about vLPI->pLPI mapping, which is not
possible. This lead to changes to the model for enabling and
disabling pLPI and vLPI and the handling of the virtual LPI
configuration table, resolving _Unresolved Issue 1_.
* Made the pLPI and vLPI interrupt priorities explicit.
* Attempted to clarify the trust issues regarding in-guest data
structures.
* Mandate a particular cacheability for tables in guest memory.
## Since Draft C
* _Major_ rework, in an attempt to simplify everything into something
more likely to be achievable for 4.6.
* Made some simplifying assumptions.
* Reduced the scope of some support.
* Command emulation is now mostly trivial.
* Expanded detail on host setup, allowing other assumptions to be
made during emulation.
* Many other things lost in the noise of the above.
## Since Draft B
* Details of command translation (thanks to Julien and Vijay)
* Added background on LPI Translation and Pending tables
* Added background on Collections
* Settled on `N:N` scheme for vITS:pat's mapping.
* Rejigged section nesting a bit.
* Since we now thing translation should be cheap, settle on
translation at scheduling time.
* Lazy `INVALL` and `SYNC`
## Since Draft A
* Added discussion of when/where command translation occurs.
* Contention on scheduler lock, suggestion to use SOFTIRQ.
* Handling of domain shutdown.
* More detailed discussion of multiple vs single vits pros/cons.
# Introduction
ARM systems containing a GIC version 3 or later may contain one or
more ITS logical blocks. An ITS is used to route Message Signalled
interrupts from devices into an LPI injection on the processor.
The following summarises the ITS hardware design and serves as a set
of assumptions for the vITS software design. For full details of the
ITS see the "GIC Architecture Specification".
## Locality-specific Peripheral Interrupts (`LPI`)
This is a new class of message signalled interrupts introduced in
GICv3. They occupy the interrupt ID space from `8192..(2^32)-1`.
The number of LPIs support by an ITS is exposed via
`GITS_TYPER.IDbits` (as number of bits - 1), it may be up to
2^32. _Note_: This field also contains the number of Event IDs
supported by the ITS.
### LPI Configuration Table
Each LPI has an associated configuration byte in the LPI Configuration
Table (managed via the GIC Redistributor and placed at
`GICR_PROPBASER` or `GICR_VPROPBASER`). This byte configures:
* The LPI's priority;
* Whether the LPI is enabled or disabled.
Software updates the Configuration Table directly but must then issue
an invalidate command (per-device `INV` ITS command, global `INVALL`
ITS command or write `GICR_INVLPIR`) for the affect to be guaranteed
to become visible (possibly requiring an ITS `SYNC` command to ensure
completion of the `INV` or `INVALL`). Note that it is valid for an
implementation to reread the configuration table at any time (IOW it
is _not_ guaranteed that a change to the LPI Configuration Table won't
be visible until an invalidate is issued).
### LPI Pending Table
Each LPI also has an associated bit in the LPI Pending Table (managed
by the GIC redistributor). This bit signals whether the LPI is pending
or not.
This region may contain out of date information and the mechanism to
synchronise is `IMPLEMENTATION DEFINED`.
## Interrupt Translation Service (`ITS`)
### Device Identifiers
Each device using the ITS is associated with a unique "Device
Identifier".
The device IDs are properties of the implementation and are typically
described via system firmware, e.g. the ACPI IORT table or via device
tree.
The number of device ids in a system depends on the implementation and
can be discovered via `GITS_TYPER.Devbits`. This field allows an ITS
to have up to 2^32 devices.
### Events
Each device can generate "Events" (called `ID` in the spec) these
correspond to possible interrupt sources in the device (e.g. MSI
offset).
The maximum number of interrupt sources is device specific. It is
usually discovered either from firmware tables (e.g. DT or ACPI) or
from bus specific mechanisms (e.g. PCI config space).
The maximum number of events ids support by an ITS is exposed via
`GITS_TYPER.IDbits` (as number of bits - 1), it may be up to
2^32. _Note_: This field also contains the number of `LPIs` supported
by the ITS.
### Interrupt Collections
Each interrupt is a member of an "Interrupt Collection". This allows
software to manage large numbers of physical interrupts with a small
number of commands rather than issuing one command per interrupt.
On a system with N processors, the ITS must provide at least N+1
collections.
An ITS may support some number of internal collections (indicated by
`GITS_TYPER.HCC`) and external ones which require memory provisioned
by the Operating System via a `GITS_BASERn` register.
### Target Addresses
The Target Address correspond to a specific GIC re-distributor. The
format of this field depends on the value of the `GITS_TYPER.PTA` bit:
* 1: the base address of the re-distributor target is used
* 0: a unique processor number is used. The mapping between the
processor affinity value (`MPIDR`) and the processor number is
discoverable via `GICR_TYPER.ProcessorNumber`.
This value is up to the ITS implementer (`GITS_TYPER` is a read-only
register).
### Device Table
A Device Table is configured in each ITS which maps incoming device
identifiers into an ITS Interrupt Translation Table.
### Interrupt Translation Table (`ITT`) and Collection Table
An `Event` generated by a `Device` is translated into an `LPI` via a
per-Device Interrupt Translation Table. The structure of this table is
described in GIC Spec 4.9.12.
The ITS translation table maps the device id of the originating device
into a physical interrupt (`LPI`) and an Interrupt Collection.
The Collection is in turn looked up in the Collection Table to produce
a Target Address, indicating a redistributor (AKA CPU) to which the
LPI is delivered.
### OS Provisioned Memory Regions
The ITS hardware design provides mechanisms for an ITS to be provided
with various blocks of memory by the OS for ITS internal use, this
include the per-device ITT (established with `MAPD`) and memory
regions for Device Tables, Virtual Processors and Interrupt
Collections. Up to 8 such regions can be requested by the ITS and
provisioned by the OS via the `GITS_BASERn` registers.
### ITS Configuration
The ITS is configured and managed, including establishing and
configuring the Translation Tables and Collection Table, via an in
memory ring shared between the CPU and the ITS controller. The ring is
managed via the `GITS_CBASER` register and indexed by `GITS_CWRITER`
and `GITS_CREADR` registers.
A processor adds commands to the shared ring and then updates
`GITS_CWRITER` to make them visible to the ITS controller.
The ITS controller processes commands from the ring and then updates
`GITS_CREADR` to indicate the the processor that the command has been
processed.
Commands are processed sequentially.
Commands sent on the ring include operational commands:
* Routing interrupts to processors;
* Generating interrupts;
* Clearing the pending state of interrupts;
* Synchronising the command queue
and maintenance commands:
* Map device/collection/processor;
* Map virtual interrupt;
* Clean interrupts;
* Discard interrupts;
The field `GITS_CBASER.Size` encodes the number of 4KB pages minus 0
consisting of the command queue. This field is 8 bits which means the
maximum size is 2^8 * 4KB = 1MB. Given that each command is 32 bytes,
there is a maximum of 32768 commands in the queue.
The ITS provides no specific completion notification
mechanism. Completion is monitored by a combination of a `SYNC`
command and either polling `GITS_CREADR` or notification via an
interrupt generated via the `INT` command.
Note that the interrupt generation via `INT` requires an originating
device ID to be supplied (which is then translated via the ITS into an
LPI). No specific device ID is defined for this purpose and so the OS
software is expected to fabricate one.
Possible ways of inventing such a device ID are:
* Enumerate all device ids in the system and pick another one;
* Use a PCI BDF associated with a non-existent device function (such
as an unused one relating to the PCI root-bridge) and translate that
(via firmware tables) into a suitable device id;
* ???
# LPI Handling in Xen
## IRQ descriptors
Currently all SGI/PPI/SPI interrupts are covered by a single static
array of `struct irq_desc` with ~1024 entries (the maximum interrupt
number in that set of interrupt types).
The addition of LPIs in GICv3 means that the largest potential
interrupt specifier is much larger.
Therefore a second dynamically allocated array will be added to cover
the range `8192..nr_lpis`. The `irq_to_desc` function will determine
which array to use (static `0..1024` or dynamic `8192..end` lpi desc
array) based on the input irq number. Two arrays are used to avoid a
wasteful allocation covering the unused/unusable) `1024..8191` range.
## Virtual LPI interrupt injection
A physical interrupt which is routed to a guest vCPU has the
`_IRQ_GUEST` flag set in the `irq_desc` status mask. Such interrupts
have an associated instance of `struct irq_guest` which contains the
target `struct domain` pointer and virtual interrupt number.
In Xen a virtual interrupt (either arising from a physical interrupt
or completely virtual) is ultimately injected to a VCPU using the
`vgic_vcpu_inject_irq` function, or `vgic_vcpu_inject_lpi`.
This mechanism will likely need updating to handle the injection of
virtual LPIs. In particular rather than `GICD_ITARGERRn` or
`GICD_IROUTERn` routing of LPIs is performed via the ITS collections
mechanism. This is discussed below (In _vITS_:_Virtual LPI injection_).
# Scope
The ITS is rather complicated, especially when combined with
virtualisation. To simplify things we initially omit the following
functionality:
- Interrupt -> vCPU -> pCPU affinity. The management of physical vs
virtual Collections is a feature of GICv4, thus is omitted in this
design for GICv3. Physical interrupts which occur on a pCPU where
the target vCPU is not already resident will be forwarded (via IPI)
to the correct pCPU for injection via the existing
`vgic_vcpu_inject_irq` mechanism (extended to handle LPI injection
correctly).
- Clearing of the pending state of an LPI under various circumstances
(`MOVI`, `DISCARD`, `CLEAR` commands) is not done. This will result
in guests seeing some perhaps spurious interrupts.
- vITS functionality will only be available on 64-bit ARM hosts,
avoiding the need to worry about fast access to guest owned data
structures (64-bit uses a direct map). (NB: 32-bit guests on 64-bit
hosts can be considered to have access)
# pITS
## Assumptions
It is assumed that `GITS_TYPER.IDbits` is large enough that there are
sufficient LPIs available to cover the sum of the number of possible
events generated by each device in the system (that is the sum of the
actual events for each bit of hardware, rather than the notional
per-device maximum from `GITS_TYPER.Idbits`).
This assumption avoids the need to do memory allocations and interrupt
routing at run time, e.g. during command processing by allowing us to
setup everything up front.
## Driver
The physical driver will provide functions for enabling, disabling
routing etc a specified interrupt, via the usual Xen APIs for doing
such things.
This will likely involve interacting with the physical ITS command
queue etc. In this document such interactions are considered internal
to the driver (i.e. we care that the API to enable an interrupt
exists, not how it is implemented).
The physical ITS will be provisioned with whatever tables it requests
via its `GITS_BASERn` registers.
## Collections
The `pITS` will be configured at start of day with 1 Collection mapped
to each physical processor, using the `MAPC` command on the physical
ITS.
## Per Device Information
Each physical device in the system which can be used together with an
ITS (whether using passthrough or not) will have associated with it a
data structure:
struct its_device {
struct pits *pits;
uintNN_t phys_device_id;
uintNN_t virt_device_id;
unsigned int *events;
unsigned int nr_events;
struct page_info *pitt;
unsigned int nr_pitt_pages;
/* Other fields relating to pITS maintenance but unrelated to vITS */
};
Where:
- `pits`: Pointer to the associated physical ITS.
- `phys_device_id`: The physical device ID of the physical device
- `virt_device_id`: The virtual device ID if the device is accessible
to a domain
- `events`: An array mapping a per-device event number into a physical
LPI.
- `nr_events`: The number of events which this device is able to
generate.
- `pitt`, `nr_pitt_pages`: Records allocation of pages for physical
ITT (not directly accessible).
During its lifetime this structure may be referenced by several
different mappings (e.g. physical and virtual device id maps, virtual
collection device id).
## Device Discovery/Registration and Configuration
Per device information will be discovered based on firmware tables (DT
or ACPI) and information provided by dom0 (e.g. reading associated PCI
cfg space, registration via PHYSDEVOP_pci_device_add or new custom
hypercalls).
This information shall include at least:
- The Device ID of the device.
- The maximum number of Events which the device is capable of
generating.
When a device is discovered/registered (i.e. when all necessary
information is available) then:
- `struct its_device` and the embedded `events` array will be
allocated (the latter with `nr_events` elements).
- The `struct its_device` will be inserted into a mapping (possibly an
R-B tree) from its physical Device ID to the `struct its`.
- `nr_events` physical LPIs will be allocated and recorded in the
`events` array.
- An ITT table will be allocated for the device and the appropriate
`MAPD` command will be issued to the physical ITS. The location will
be recorded in `struct its_device.pitt`.
- Each Event which the device may generate will be mapped to the
corresponding LPI in the `events` array and a collection, by issuing
a series of `MAPVI` commands. Events will be assigned to physical
collections in a round-robin fashion.
This setup must occur for a given device before any ITS interrupts may
be configured for the device and certainly before a device is passed
through to a guest. This implies that dom0 cannot use MSIs on a PCI
device before having called `PHYSDEVOP_pci_device_add`.
# Device Assignment
Each domain will have an associated mapping from virtual device ids
into a data structure describing the physical device, including a
reference to the relevant `struct its_device`.
The number of possible device IDs may be large so a simple array or
list is likely unsuitable. A tree (e.g. Red-Black may be a suitable
data structure. Currently we do not need to perform lookups in this
tree on any hot paths.
_Note_: In the context of virtualised device ids (especially for domU)
it may be possible to arrange for the upper bound on the number of
device IDs to be lower allowing a more efficient data structure to be
used. This is left for a future improvement.
When a device is assigned to a domain (including to domain 0) the
mapping for the new virtual device ID will be entered into the tree.
During assignment all LPIs associated with the device will be routed
to the guest (i.e. `route_irq_to_guest` will be called for each LPI in
the `struct its_device.events` array) and the pLPI will be enabled in
the physical LPI configuration table with a priority of `GIC_PRI_IRQ`
(not any priority from the guest).
# vITS
A guest domain which is allowed to use ITS functionality (i.e. has
been assigned pass-through devices which can generate MSIs) will be
presented with a virtualised ITS.
Accesses to the vITS registers will trap to Xen and be emulated and a
virtualised Command Queue will be provided.
Commands entered onto the virtual Command Queue will be translated
into physical commands, as described later in this document.
There are other aspects to virtualising the ITS (LPI collection
management, assignment of LPI ranges to guests, device
management). However these are only considered here to the extent
needed for describing the vITS emulation.
## Xen interaction with guest OS provisioned vITS memory
Memory which the guest provisions to the vITS (ITT via `MAPD` or other
tables via `GITS_BASERn`) needs careful handling in Xen.
### Trust
Since Xen cannot trust data in data structures contained in such
memory if a guest can trample over it at will. Therefore Xen either
must take great care when accessing data structures stored in such
memory to validate the contents e.g. not trust that values are within
the required limits or it must take steps to restrict guest access to
the memory when it is provisioned. Since the data structures are
simple and most accessors need to do bounds check anyway it is
considered sufficient to simply do the necessary checks on access.
**Any information read memory which has been provisioned by the guest
OS should not be trusted and must be carefully checked (e.g. ranges
etc) before use.**
### Mapping
Most data structures stored in this shared memory are accessed on the
hot interrupt injection path and must therefore be quickly accessible
from within Xen. Since we have restricted vits support to 64-bit hosts
only `map_domain_page` is fast enough to be used on the fly and
therefore we do not need to be concerned about unbounded amounts of
permanently mapped memory consumed by each `MAPD` command.
Although `map_domain_page` is fast, `p2m_lookup` (translation from IPA
to PA) is not necessarily so and is used by `get_page_from_gfn`. For
now we accept this, as a future extension a sparse mapping of the
guest device table in vmap space could be considered, with limits on
the total amount of vmap space which we allow each domain to consume.
The `GITS_BASERn` registers allow for the guest to specify cache
attributes for the memory. For now we require that these have the same
attributes as hypercall arguments in general (see `public/arch-arm.h`)
In addition while `GITS_BASERn` allows the Cacheability to be
specified as `Device-nGnRnE` we require that the tables provided be in
normal guest RAM (not MMIO, not granted memory etc), that is it must
have type `p2m_ram_rw`.
## vITS properties
The vITS implementation shall have:
- `GITS_TYPER.HCC == nr_vcpus + 1`.
- `GITS_TYPER.PTA == 0`. Target addresses are linear processor numbers.
- `GITS_TYPER.Devbits == See below`.
- `GITS_TYPER.IDbits == See below`.
- `GITS_TYPER.ITT Entry Size == 7`, meaning 8 bytes, which is the size
of `struct vitt` (defined below).
`GITS_TYPER.Devbits` and `GITS_TYPER.Idbits` will need to be chosen to
reflect the host and guest configurations (number of LPIs, maximum
device ID etc).
Other fields (not mentioned here) will be set to some sensible (or
mandated) value.
The `GITS_BASER0` will be setup to request sufficient memory for a
device table consisting of entries of:
struct vdevice_table {
uint64_t vitt_ipa;
uint32_t vitt_size;
uint32_t padding;
};
BUILD_BUG_ON(sizeof(struct vdevice_table) != 16);
On write to `GITS_BASER0` the relevant details of the Device Table
(IPA, size, cache attributes to use when mapping) will be recorded in
`struct domain`.
All other `GITS_BASERn.Valid == 0`.
## vITS to pITS mapping
A physical system may have multiple physical ITSs.
With the simplified vits command model presented here only a single
`vits` is required.
In the future a more complex arrangement may be desired. Since the
choice of model is internal to the hypervisor/tools and is
communicated to the guest via firmware tables we are not tied to this
model as an ABI if we decide to change.
When constructing dom0 it will therefore be necessary to rewrite any
DTS properties which refer to an ITS to point to the single provided
ITS, as well as dropping all ITS nodes and replacing them with a
single node representing the vITS.
## Mapping from `vLPI` back to `pLPI`
While we have arranged for a (`pDevice`,`pEvent`) to map to a single
`pLPI` we cannot guarantee that a given `vLPI` is mapped by a single
(`vDevice`,`vEvent`) since the guest may setup multiple ITT tables
such that this is not the case. Enforcing that this is the case is
prohibitively expensive.
Therefore it is not in general possible to associate a `vLPI` with a
`pLPI`.
## Per-domain `struct pending_irq` for `vLPI`s
Internally Xen uses a `struct pending_irq` to track the status of any
pending virtual IRQ, including a virtual LPI.
Upon domain creation an array of such `struct pending_irq`'s will be
allocated to cover the range `8192..nr_lpis` (for the number of LPIs
which the guest is configured with) and a pointer this array will be
stored in the `struct domain`. The function `irq_to_pending` will be
modified to lookup interupts in the LPI range in this array.
## Handling of unrouted/spurious LPIs
Since there is no 1:1 link between a `vLPI` and `pLPI` enabling and
disabling of physical LPIs cannot be driven from the state of an
associated vLPI.
Each `pLPI` is routed and enabled during device assignment, therefore
it is possible to receive a physical LPI which has yet to be routed
(via a `vITS`) to a `vLPI`.
Similarly if a guest routes multiple Events to a single `vLPI` the
interrupt may already be pending when we attempt to deliver it.
Such `pLPI`s shall be ignored and left in the priority dropped state
(per the read from `GICC_IAR`). They will not be `EOI`-d in order to
avoid a possible interrupt storm.
On device deassignment (including as part of domain destroy) after
resetting the device it will be necessary to EOI any interrupts in
such a state by walking over all events in the corresponding `struct
its_device`.
## Enabling and disabling LPIs
Two new functions `vgic_enable_lpi` and `vgic_disable_lpi` will be
provided which are analogous to `vgic_enable_irqs` and
`vgic_disable_irqs` but work for the LPI interface. (Alternatively,
refactoring the existing functions to work for all caes would be
acceptable too).
A `vLPI` which has not yet be enabled will automatically be queued, by
the existing vgic injection machinery, until a call to
`vgic_enable_lpi` is made (in response to a trapped access to the
virtual cfg table).
## LPI Configuration Table Virtualisation
A guest's write accesses to its LPI Configuration Table (which is just
an area of guest RAM which the guest has nominated) will be trapped to
the hypervisor, using stage 2 MMU permissions, in order for changes to
be propagated into the host interrupt configuration.
On write `bit[0]` of the written byte is the enable/disable state for
the irq and is handled thus, for each byte in the written value:
vlpi = lpi corresponding to byte offset (addr - table_base);
pending_irq = irq_to_pending(current, vlpi);
pending_irq->priority = byte & 0xfc; /* XXX: or byte >> 2 */
if ( byte & 0x1 )
vgic_enable_lpi(current, vlpi);
else
vgic_disable_lpi(current, vlpi);
Note that physical interrupts are always configured with a priority of
`GIC_PRI_IRQ`, regardless of the priority of any virtual interrupt.
## LPI Pending Table Virtualisation
According to GIC spec 4.8.5 this table is not necessarily in sync and
the mechanism to force a sync is `IMPLEMENTATION DEFINED`, hence we
don't need to do anything.
## Device Table Virtualisation
The IPA, size and cacheability attributes of the guest device table
will be recorded in `struct domain` upon write to `GITS_BASER0`.
In order to lookup an entry for `device`:
define {get,set}_vdevice_entry(domain, device, struct device_table *entry):
offset = device*sizeof(struct vdevice_table)
if offset > <DT size>: error
dt_entry = <DT base IPA> + device*sizeof(struct vdevice_table)
page = get_page_from_gfn(current->domain, dt_entry>>PAGE_SHIFT, &p2mt, P2M_ALLOC);
if !page: error
if !page_is_ram(p2mt): put_page(page); error;
dt_mapping = map_domain_page(page)
if (set)
dt_mapping[<appropriate page offset from device>] = *entry;
else
*entry = dt_mapping[<appropriate page offset>];
unmap_domain_page(dt_mapping)
put_page(page)
Since everything is based upon IPA (guest addresses) a malicious guest
can only reference its own RAM here.
## ITT Virtualisation
The location of a VITS will have been recorded in the domain Device
Table by a `MAPI` or `MAPVI` command and is looked up as above.
The `vitt` is a `struct vitt`:
struct vitt {
uint16_t valid:1;
uint16_t pad:15;
uint16_t vcollection;
uint32_t vlpi;
};
BUILD_BUG_ON(sizeof(struct vitt) != 8);
A lookup occurs similar to for a device table, the offset is range
checked against the `vitt_size` from the device table. To lookup
`event` on `device`:
define {get,set}_vitt_entry(domain, device, event, struct vitt *entry):
get_vdevice_entry(domain, device, &dt)
offset = event*sizeof(struct vitt);
if offset > dt->vitt_size: error
vitt_entry = dt->vita_ipa + event*sizeof(struct vitt)
page = get_page_from_gfn(current->domain, vitt_entry>>PAGE_SHIFT, &p2mt, P2M_ALLOC);
if !page: error
if !page_is_ram(p2mt): put_page(page); error;
vitt_mapping = map_domain_page(page)
if (set)
vitt_mapping[<appropriate page offset from event>] = *entry;
else
*entry = vitt_mapping[<appropriate page offset>];
unmap_domain_page(entry)
put_page(page)
Again since this is IPA based a malicious guest can only point things
to its own ram.
## Collection Table Virtualisation
A pointer to a dynamically allocated array `its_collections` mapping
collection ID to vcpu ID will be added to `struct domain`. The array
shall have `nr_vcpus + 1` entries and resets to ~0 (or another
explicitly invalid vpcu nr).
## Virtual LPI injection
As discussed above the `vgic_vcpu_inject_irq` functionality will need
to be extended to cover this new case, most likely via a new
`vgic_vcpu_inject_lpi` frontend function. `vgic_vcpu_inject_irq` will
also require some refactoring to allow the priority to be passed in
from the caller (since `LPI` proprity comes from the `LPI` CFG table,
while `SPI` and `PPI` priority is configured via other means).
`vgic_vcpu_inject_lpi` receives a `struct domain *` and a virtual
interrupt number (corresponding to a vLPI) and needs to figure out
which vcpu this should map to.
To do this it must look up the Collection ID associated (via the vITS)
with that LPI.
Proposal: Add a new `its_device` field to `struct irq_guest`, a
pointer to the associated `struct its_device`. The existing `struct
irq_guest.virq` field contains the event ID (perhaps use a `union`
to give a more appropriate name) and _not_ the virtual LPI. Injection
then consists of:
d = irq_guest->domain
virq = irq_guest->virq
its_device = irq_guest->its_device
get_vitt_entry(d, its_device->virt_device_id, virq, &vitt)
vcpu = d->its_collections[vitt.vcollection]
if !is_valid_lpi(vitt.vlpi): error
vgic_vcpu_inject_lpi(&d->vcpus[vcpu], vitt.vlpi)
If the LPI is currently disabled then it will be queued by
`vgic_vcpu_inject_lpi` and injected in response to a subsequent
`vgic_enable_lpi` call.
## Command Queue Virtualisation
The command translation/emulation in this design has been arranged to
be as cheap as possible (e.g. in many cases the actions are NOPs),
avoiding previous concerns about the length of time which an emulated
write to a `CWRITER` register may block the vcpu.
Each domain will have an associated `struct vits_cq`:
struct vits_cq {
spinlock_t lock;
atomic_t creadr; /* Virtual creadr */
[ Reference to command queue memory ]
};
On read from `GITS_CREADR` the current value of `vits_cq.creadr` is
read and returned using `atomic_read`. No lock is required.
Writes to `GITS_CWRITER` are handled synchronously as follows, given a
write of `val`:
spin_lock(vits_cq.lock)
while vits_cq.creadr != val:
process one command from offset vits_cq.creadr
atomic_add(vits_cq.creadr, 0x10); /* XXX handle wrap around */
spin_unlock(vits_cq.lock)
increment PC
return
Adding preemption to this is tricky with multiple vcpus present. e.g,
a subtle case:
- VCPUA writes GITS_CWRITER W1, processes a proportion and is
preempted.
- VCPUB write GITS_CWRITER W2 (>W1) and processes a proportion such that
W1<GITS_CREADR<W2.
- VCPUA resumes, sees GITS_CREADR != W1, processes junk until it
wraps around.
## ITS Command Translation
This section is based on the section 5.13 of GICv3 specification
(PRD03-GENC-010745 24.0) and provides concrete ideas of how this can
be interpreted for Xen.
The ITS provides 12 commands in order to manage interrupt collections,
devices and interrupts. Possible command parameters are:
- Device ID (`Device`) (called `device` in the spec).
- Event ID (`Event`) (called `ID` in the spec). This is an index into
a devices `ITT`.
- Collection ID (`Collection`) (called `collection` in the spec)
- LPI ID (`LPI`) (called `pID` in the spec)
- Target Address (`TA`) (called `TA` in the spec`)
These parameters need to be validated and translated from Virtual (`v`
prefix) to Physical (`p` prefix).
Note, we differ from the naming in the GIC spec for clarity, in
particular we use `Event` not `ID` and `LPI` not `pID` to reduce
confusion, especially when `v` and `p` suffixes are used due to
virtualisation.
### Parameter Validation / Translation
Each command contains parameters that needs to be validated before any
usage in Xen or passing to the hardware.
#### Device ID (`Device`)
Corresponding ITT obtained by looking up as described above.
The physical `struct its_device` can be found by looking up in the
domain's device map.
If lookup fails or the resulting device table entry is invalid then
the Device is invalid.
#### Event ID (`Event`)
Validated against emulated `GITS_TYPER.IDbits`.
It is not necessary to translate a `vEvent`.
#### LPI (`LPI`)
Validated against emulated `GITS_TYPER.IDbits`.
It is not necessary to translate a `vLPI` into a `pLPI` since the
tables all contain `vLPI`. (Translation from `pLPI` to `vLPI` happens
via `struct irq_guest` when we receive the IRQ).
#### Interrupt Collection (`Collection`)
The `Collection` is validated against the size of the per-domain
`its_collections` array (i.e. nr_vcpus + 1) and then translated by a
simple lookup in that array.
vcpu_nr = d->its_collections[Collection]
A result > `nr_cpus` is invalid
#### Target Address (`TA`)
This parameter is used in commands which manage collections. It is a
unique identifier per processor.
We have chosen to implement `GITS_TYPER.PTA` as 0, hence `vTA` simply
corresponds to the `vcpu_id`, so only needs bounds checking against
`nr_vcpus`.
### Commands
To be read with reference to spec for each command (which includes
error checks etc which are omitted here).
It is assumed that inputs will be bounds and validity checked as
described above, thus error handling is omitted for brevity (i.e. if
get and/or set fail then so be it). In general invalid commands are
simply ignored.
#### `MAPD`: Map a physical device to an ITT.
_Format_: `MAPD Device, Valid, ITT Address, ITT Size`.
_Spec_: 5.13.11
`MAPD` is sent with `Valid` bit set if the mapping is to be added and
reset when mapping is removed.
When the `Valid` bit is set then the range `ITT Address` to `ITT
Address` + `ITT Size` need not be validated, this is done in
`{get,set}_vdevice_entry` when calling the `get_page_from_gfn`
function. Validating the memory at `MAPD` time would serve no purpose
since the guest could subsequently balloon it out or grant map over it etc.
The domain's device table is updated with the provided information.
The `vitt_mapd` field is set according to the `Valid` flag in the
command:
dt_entry.vitt_ipa = ITT Address
dt_entry.vitt_size = ITT Size
set_vdevice_entry(current->domain, Device, &dt_entry)
#### `MAPC`: Map an interrupt collection to a target processor
_Format_: `MAPC Collection, TA`
_Spec_: 5.13.12
The updated `vTA` (a vcpu number) is recorded in the `its_collections`
array of the domain struct:
d->its_collections[Collection] = TA
#### `MAPI`: Map an interrupt to an interrupt collection.
_Format_: `MAPI Device, LPI, Collection`
_Spec_: 5.13.13
After validation:
vitt.valid = True
vitt.collection = Collection
vitt.vlpi = LPI
set_vitt_entry(current->domian, Device, LPI, &vitt)
#### `MAPVI`: Map an input identifier to a physical interrupt and an interrupt collection.
Format: `MAPVI Device, Event, LPI, Collection`
vitt.valid = True
vitt.collection = Collection
vitt.vlpi = LPI
set_vitt_entry(current->odmian, Device, Event, &vitt)
#### `MOVI`: Redirect interrupt to an interrupt collection
_Format_: `MOVI Device, Event, Collection`
_Spec_: 5.13.15
get_vitt_entry(current->domain, Device, Event, &vitt)
vitt.collection = Collection
set_vitt_entry(current->domain, Device, Event, &vitt)
XXX consider helper which sets field without mapping/unmapping
twice.
This command is supposed to move any pending interrupts associated
with `Event` to the vcpu implied by the new `Collection`, which is
tricky. For now we ignore this requirement (as we do for
`GICD_IROUTERn` and `GICD_TARGETRn` for other interrupt types).
#### `DISCARD`: Discard interrupt requests
_Format_: `DISCARD Device, Event`
_Spec_: 5.13.16
get_vitt_entry(current->domain, Device, Event, &vitt)
vitt.valid = False
set_vitt_entry(current->domain, Device, Event, &vitt)
XXX consider helper which sets field without mapping/unmapping
twice.
This command is supposed to clear the pending state of any associated
interrupt. This requirement is ignored (guest may see a spurious
interrupt).
#### `INV`: Clean any caches associated with interrupt
_Format_: `INV Device, Event`
_Spec_: 5.13.17
Since LPI Configuration table updates are handled synchronously, there
is nothing to do here.
#### `INVALL`: Clean any caches associated with an interrupt collection
_Format_: `INVALL Collection`
_Spec_: 5.13.19
Since LPI Configuration table updates are handled synchronously, there
is nothing to do here.
#### `INT`: Generate an interrupt
_Format_: `INT Device, Event`
_Spec_: 5.13.20
The `vitt` entry corresonding to `Device,Event` is looked up and then:
get_vitt_entry(current->domain, Device, Event, &vitt)
vgic_vcpu_inject_lpi(current->domain, vitt.vlpi)
_Note_: Where (Device,Event) is real may need consideration of
interactions with real LPIs being delivered: Julien had concerns about
Xen's internal IRQ State tracking. if this is a problem then may need
changes to IRQ state tracking, or to inject as a real IRQ and let
physical IRQ injection handle it, or write to `GICR_SETLPIR`?
#### `CLEAR`: Clear the pending state of an interrupt
_Format_: `CLEAR Device, Event`
_Spec_: 5.13.21
Should clear pending state of LPI. Ignore (guest may see a spurious
interrupt).
#### `SYNC`: Wait for completion of any outstanding ITS actions for collection
_Format_: `SYNC TA`
_Spec_: 5.13.22
This command can be ignored.
# GICv4 Direct Interrupt Injection
GICv4 will directly mark the LPIs pending in the virtual pending table
which is per-redistributor (i.e per-vCPU).
LPIs will be received by the guest the same way as an SPIs. I.e trap in
IRQ mode then read ICC_IAR1_EL1 (for GICv3).
Therefore GICv4 will not require one vITS per pITS.
# Event Channels
It has been proposed that it might be nice to inject event channels as
LPIs in the future. Whether or not that would involve any sort of vITS
is unclear, but if it did then it would likely be a separate emulation
to the vITS emulation used with a pITS and as such is not considered
further here.
# Glossary
* _MSI_: Message Signalled Interrupt
* _ITS_: Interrupt Translation Service
* _GIC_: Generic Interrupt Controller
* _LPI_: Locality-specific Peripheral Interrupt
# References
"GIC Architecture Specification" PRD03-GENC-010745 24.0.
"IO Remapping Table System Software on ARM® Platforms" ARM DEN 0049A.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 88+ messages in thread* Re: [RFC PATCH v3 00/18] Add ITS support
2015-06-22 12:01 [RFC PATCH v3 00/18] Add ITS support vijay.kilari
` (19 preceding siblings ...)
2015-06-24 10:02 ` Ian Campbell
@ 2015-06-29 13:11 ` Ian Campbell
20 siblings, 0 replies; 88+ messages in thread
From: Ian Campbell @ 2015-06-29 13:11 UTC (permalink / raw)
To: vijay.kilari
Cc: stefano.stabellini, Prasun.Kapoor, vijaya.kumar, tim, xen-devel,
julien.grall, stefano.stabellini, manish.jaggi
On Mon, 2015-06-22 at 17:31 +0530, vijay.kilari@gmail.com wrote:
> Vijaya Kumar K (18):
> xen/arm: Add bitmap_find_next_zero_area helper function
> xen: Add log2 functionality
> xen: console: Add ratelimit support for error message
> xen/arm: gicv3: Refactor redistributor information
> xen/arm: ITS: Port ITS driver to xen
> xen/arm: ITS: Add helper functions to manage its_devices
> xen/arm: ITS: implement hw_irq_controller for LPIs
> xen/arm: vITS: Add virtual ITS driver
> xen/arm: ITS: Add virtual ITS commands support
> xen/arm: ITS: Add APIs to add and assign device
> xen/arm: ITS: Add GITS registers emulation
> xen/arm: ITS: Add GICR register emulation
> xen/arm: ITS: Add irq descriptors for LPIs
> xen/arm: ITS: Initialize physical ITS
> xen/arm: ITS: Add domain specific ITS initialization
> xen/arm: ITS: Handle LPI interrupts
> xen/arm: ITS: Generate ITS node for Dom0
> xen/arm: ITS: Map ITS translation space
FWIW I think it would be more logical to order this as:
* Preparation, precursor cleanups etc
* PITS support
* VITS support, which uses pits stuff
* Domain vits init, including map the ITS translation page, which
uses vits stuff
* Device Tree mangling which finally exposes the result to the
guest.
But I think far more critical would be to make function which whould
eventually be static be so when it is introduced (not later) and to
include prototypes of functions in the patch which adds the function.
Also lots more const correctness would be good.
Ian.
^ permalink raw reply [flat|nested] 88+ messages in thread