* [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
v6:
- rebased to latest master
- several other changes thanks to Andre and Alex, changes in
individual patch change logs
- some code cleanups
v5:
- fix arm32/gicv3 compile [drew]
- use modern register names [Andre]
- one Andre r-b
v4:
- Eric's r-b's
- Andre's suggestion to only take defines we need
- several other changes listed in individual patches
v3:
- Rebased on latest master
- Added Alex's r-b's
v2:
Rebased on latest master + my "populate argv[0]" series (will
send a REPOST for that shortly. Additionally a few patches got
fixes/features;
07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
all interrupts as non-secure Group-1" in order to continue
working over TCG, as the gicv3 code for TCG removed a hack
it had there to make Linux happy.
08/10 added more output for when things fail (if they fail)
09/10 switched gicv3 broadcast implementation to using IRM. This
found a bug in a recent (but not tip) kernel, which I was
about to fix, but then I saw MarcZ beat me to it.
10/10 actually check that the input irq is the received irq
Import defines, and steal enough helper functions, from Linux to
enable programming of the gic (v2 and v3). Then use the framework
to add an initial test (an ipi test; self, target-list, broadcast).
It's my hope that this framework will be a suitable base on which
more tests may be easily added, particularly because we have
vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
vgic-new and tcg gicv3 are merged now)
To run it, along with other tests, just do
./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
make
export QEMU=$PATH_TO_QEMU
./run_tests.sh
To run it separately do, e.g.
$QEMU -machine virt,accel=tcg -cpu cortex-a57 \
-device virtio-serial-device \
-device virtconsole,chardev=ctd -chardev testdev,id=ctd \
-display none -serial stdio \
-kernel arm/gic.flat \
-smp 123 -machine gic-version=3 -append ipi
^^ note, we can go nuts with nr-cpus on TCG :-)
Or, a KVM example using a different "sender" cpu and irq (other than zero)
$QEMU -machine virt,accel=kvm -cpu host \
-device virtio-serial-device \
-device virtconsole,chardev=ctd -chardev testdev,id=ctd \
-display none -serial stdio \
-kernel arm/gic.flat \
-smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
Patches:
01-05: fixes and functionality needed by the later gic patches
06-07: enable gicv2 and gicv2 IPI test
08-10: enable gicv3 and gicv3 IPI test
11: extend the IPI tests to take variable sender and irq
Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
Andrew Jones (10):
lib: xstr: allow multiple args
arm64: fix get_"sysreg32" and make MPIDR 64bit
arm/arm64: smp: support more than 8 cpus
arm/arm64: add some delay routines
arm/arm64: irq enable/disable
arm/arm64: add initial gicv2 support
arm/arm64: gicv2: add an IPI test
arm/arm64: add initial gicv3 support
arm/arm64: gicv3: add an IPI test
arm/arm64: gic: don't just use zero
Peter Xu (1):
libcflat: add IS_ALIGNED() macro, and page sizes
arm/Makefile.common | 9 +-
arm/gic.c | 340 +++++++++++++++++++++++++++++++++++++++++++++
arm/run | 19 ++-
arm/selftest.c | 5 +-
arm/unittests.cfg | 14 ++
lib/arm/asm/arch_gicv3.h | 70 ++++++++++
lib/arm/asm/gic-v2.h | 36 +++++
lib/arm/asm/gic-v3.h | 112 +++++++++++++++
lib/arm/asm/gic.h | 106 ++++++++++++++
lib/arm/asm/processor.h | 42 +++++-
lib/arm/asm/setup.h | 4 +-
lib/arm/gic.c | 267 +++++++++++++++++++++++++++++++++++
lib/arm/processor.c | 15 ++
lib/arm/setup.c | 10 ++
lib/arm64/asm/arch_gicv3.h | 66 +++++++++
lib/arm64/asm/gic-v2.h | 1 +
lib/arm64/asm/gic-v3.h | 1 +
lib/arm64/asm/gic.h | 1 +
lib/arm64/asm/processor.h | 53 +++++--
lib/arm64/asm/sysreg.h | 44 ++++++
lib/arm64/processor.c | 15 ++
lib/libcflat.h | 10 +-
22 files changed, 1212 insertions(+), 28 deletions(-)
create mode 100644 arm/gic.c
create mode 100644 lib/arm/asm/arch_gicv3.h
create mode 100644 lib/arm/asm/gic-v2.h
create mode 100644 lib/arm/asm/gic-v3.h
create mode 100644 lib/arm/asm/gic.h
create mode 100644 lib/arm/gic.c
create mode 100644 lib/arm64/asm/arch_gicv3.h
create mode 100644 lib/arm64/asm/gic-v2.h
create mode 100644 lib/arm64/asm/gic-v3.h
create mode 100644 lib/arm64/asm/gic.h
create mode 100644 lib/arm64/asm/sysreg.h
--
2.7.4
^ permalink raw reply [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
v6:
- rebased to latest master
- several other changes thanks to Andre and Alex, changes in
individual patch change logs
- some code cleanups
v5:
- fix arm32/gicv3 compile [drew]
- use modern register names [Andre]
- one Andre r-b
v4:
- Eric's r-b's
- Andre's suggestion to only take defines we need
- several other changes listed in individual patches
v3:
- Rebased on latest master
- Added Alex's r-b's
v2:
Rebased on latest master + my "populate argv[0]" series (will
send a REPOST for that shortly. Additionally a few patches got
fixes/features;
07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
all interrupts as non-secure Group-1" in order to continue
working over TCG, as the gicv3 code for TCG removed a hack
it had there to make Linux happy.
08/10 added more output for when things fail (if they fail)
09/10 switched gicv3 broadcast implementation to using IRM. This
found a bug in a recent (but not tip) kernel, which I was
about to fix, but then I saw MarcZ beat me to it.
10/10 actually check that the input irq is the received irq
Import defines, and steal enough helper functions, from Linux to
enable programming of the gic (v2 and v3). Then use the framework
to add an initial test (an ipi test; self, target-list, broadcast).
It's my hope that this framework will be a suitable base on which
more tests may be easily added, particularly because we have
vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
vgic-new and tcg gicv3 are merged now)
To run it, along with other tests, just do
./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
make
export QEMU=$PATH_TO_QEMU
./run_tests.sh
To run it separately do, e.g.
$QEMU -machine virt,accel=tcg -cpu cortex-a57 \
-device virtio-serial-device \
-device virtconsole,chardev=ctd -chardev testdev,id=ctd \
-display none -serial stdio \
-kernel arm/gic.flat \
-smp 123 -machine gic-version=3 -append ipi
^^ note, we can go nuts with nr-cpus on TCG :-)
Or, a KVM example using a different "sender" cpu and irq (other than zero)
$QEMU -machine virt,accel=kvm -cpu host \
-device virtio-serial-device \
-device virtconsole,chardev=ctd -chardev testdev,id=ctd \
-display none -serial stdio \
-kernel arm/gic.flat \
-smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
Patches:
01-05: fixes and functionality needed by the later gic patches
06-07: enable gicv2 and gicv2 IPI test
08-10: enable gicv3 and gicv3 IPI test
11: extend the IPI tests to take variable sender and irq
Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
Andrew Jones (10):
lib: xstr: allow multiple args
arm64: fix get_"sysreg32" and make MPIDR 64bit
arm/arm64: smp: support more than 8 cpus
arm/arm64: add some delay routines
arm/arm64: irq enable/disable
arm/arm64: add initial gicv2 support
arm/arm64: gicv2: add an IPI test
arm/arm64: add initial gicv3 support
arm/arm64: gicv3: add an IPI test
arm/arm64: gic: don't just use zero
Peter Xu (1):
libcflat: add IS_ALIGNED() macro, and page sizes
arm/Makefile.common | 9 +-
arm/gic.c | 340 +++++++++++++++++++++++++++++++++++++++++++++
arm/run | 19 ++-
arm/selftest.c | 5 +-
arm/unittests.cfg | 14 ++
lib/arm/asm/arch_gicv3.h | 70 ++++++++++
lib/arm/asm/gic-v2.h | 36 +++++
lib/arm/asm/gic-v3.h | 112 +++++++++++++++
lib/arm/asm/gic.h | 106 ++++++++++++++
lib/arm/asm/processor.h | 42 +++++-
lib/arm/asm/setup.h | 4 +-
lib/arm/gic.c | 267 +++++++++++++++++++++++++++++++++++
lib/arm/processor.c | 15 ++
lib/arm/setup.c | 10 ++
lib/arm64/asm/arch_gicv3.h | 66 +++++++++
lib/arm64/asm/gic-v2.h | 1 +
lib/arm64/asm/gic-v3.h | 1 +
lib/arm64/asm/gic.h | 1 +
lib/arm64/asm/processor.h | 53 +++++--
lib/arm64/asm/sysreg.h | 44 ++++++
lib/arm64/processor.c | 15 ++
lib/libcflat.h | 10 +-
22 files changed, 1212 insertions(+), 28 deletions(-)
create mode 100644 arm/gic.c
create mode 100644 lib/arm/asm/arch_gicv3.h
create mode 100644 lib/arm/asm/gic-v2.h
create mode 100644 lib/arm/asm/gic-v3.h
create mode 100644 lib/arm/asm/gic.h
create mode 100644 lib/arm/gic.c
create mode 100644 lib/arm64/asm/arch_gicv3.h
create mode 100644 lib/arm64/asm/gic-v2.h
create mode 100644 lib/arm64/asm/gic-v3.h
create mode 100644 lib/arm64/asm/gic.h
create mode 100644 lib/arm64/asm/sysreg.h
--
2.7.4
^ permalink raw reply [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 01/11] lib: xstr: allow multiple args
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini
Make implementation equivalent to Linux's include/linux/stringify.h
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
lib/libcflat.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/libcflat.h b/lib/libcflat.h
index 72b1bf9668ef..82005f5d014f 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -27,8 +27,8 @@
#define __unused __attribute__((__unused__))
-#define xstr(s) xxstr(s)
-#define xxstr(s) #s
+#define xstr(s...) xxstr(s)
+#define xxstr(s...) #s
#define __ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
#define __ALIGN(x, a) __ALIGN_MASK(x, (typeof(x))(a) - 1)
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 01/11] lib: xstr: allow multiple args
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Make implementation equivalent to Linux's include/linux/stringify.h
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
lib/libcflat.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/libcflat.h b/lib/libcflat.h
index 72b1bf9668ef..82005f5d014f 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -27,8 +27,8 @@
#define __unused __attribute__((__unused__))
-#define xstr(s) xxstr(s)
-#define xxstr(s) #s
+#define xstr(s...) xxstr(s)
+#define xxstr(s...) #s
#define __ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
#define __ALIGN(x, a) __ALIGN_MASK(x, (typeof(x))(a) - 1)
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 02/11] arm64: fix get_"sysreg32" and make MPIDR 64bit
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
mrs is always 64bit, so we should always use a 64bit register.
Sometimes we'll only want to return the lower 32, but not for
MPIDR, as that does define fields in the upper 32.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v5: switch arm32's get_mpidr to 'unsigned long' too, to be
consistent with arm64 [Andre]
---
lib/arm/asm/processor.h | 4 ++--
lib/arm64/asm/processor.h | 15 +++++++++------
2 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index f25e7eee3666..02f912f99974 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -33,9 +33,9 @@ static inline unsigned long current_cpsr(void)
#define current_mode() (current_cpsr() & MODE_MASK)
-static inline unsigned int get_mpidr(void)
+static inline unsigned long get_mpidr(void)
{
- unsigned int mpidr;
+ unsigned long mpidr;
asm volatile("mrc p15, 0, %0, c0, c0, 5" : "=r" (mpidr));
return mpidr;
}
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 84d5c7ce752b..9a208ff729b7 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -66,14 +66,17 @@ static inline unsigned long current_level(void)
return el & 0xc;
}
-#define DEFINE_GET_SYSREG32(reg) \
-static inline unsigned int get_##reg(void) \
+#define DEFINE_GET_SYSREG(reg, type) \
+static inline type get_##reg(void) \
{ \
- unsigned int reg; \
- asm volatile("mrs %0, " #reg "_el1" : "=r" (reg)); \
- return reg; \
+ unsigned long r; \
+ asm volatile("mrs %0, " #reg "_el1" : "=r" (r)); \
+ return (type)r; \
}
-DEFINE_GET_SYSREG32(mpidr)
+#define DEFINE_GET_SYSREG32(reg) DEFINE_GET_SYSREG(reg, unsigned int)
+#define DEFINE_GET_SYSREG64(reg) DEFINE_GET_SYSREG(reg, unsigned long)
+
+DEFINE_GET_SYSREG64(mpidr)
/* Only support Aff0 for now, gicv2 only */
#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 02/11] arm64: fix get_"sysreg32" and make MPIDR 64bit
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
mrs is always 64bit, so we should always use a 64bit register.
Sometimes we'll only want to return the lower 32, but not for
MPIDR, as that does define fields in the upper 32.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v5: switch arm32's get_mpidr to 'unsigned long' too, to be
consistent with arm64 [Andre]
---
lib/arm/asm/processor.h | 4 ++--
lib/arm64/asm/processor.h | 15 +++++++++------
2 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index f25e7eee3666..02f912f99974 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -33,9 +33,9 @@ static inline unsigned long current_cpsr(void)
#define current_mode() (current_cpsr() & MODE_MASK)
-static inline unsigned int get_mpidr(void)
+static inline unsigned long get_mpidr(void)
{
- unsigned int mpidr;
+ unsigned long mpidr;
asm volatile("mrc p15, 0, %0, c0, c0, 5" : "=r" (mpidr));
return mpidr;
}
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 84d5c7ce752b..9a208ff729b7 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -66,14 +66,17 @@ static inline unsigned long current_level(void)
return el & 0xc;
}
-#define DEFINE_GET_SYSREG32(reg) \
-static inline unsigned int get_##reg(void) \
+#define DEFINE_GET_SYSREG(reg, type) \
+static inline type get_##reg(void) \
{ \
- unsigned int reg; \
- asm volatile("mrs %0, " #reg "_el1" : "=r" (reg)); \
- return reg; \
+ unsigned long r; \
+ asm volatile("mrs %0, " #reg "_el1" : "=r" (r)); \
+ return (type)r; \
}
-DEFINE_GET_SYSREG32(mpidr)
+#define DEFINE_GET_SYSREG32(reg) DEFINE_GET_SYSREG(reg, unsigned int)
+#define DEFINE_GET_SYSREG64(reg) DEFINE_GET_SYSREG(reg, unsigned long)
+
+DEFINE_GET_SYSREG64(mpidr)
/* Only support Aff0 for now, gicv2 only */
#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 03/11] arm/arm64: smp: support more than 8 cpus
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini
By adding support for launching with gicv3 we can break the 8 vcpu
limit. This patch adds support to smp code and also selects the
vgic model corresponding to the host. The vgic model may also be
manually selected by adding e.g. -machine gic-version=3 to
extra_params.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v5: left cpus a u32 for now. Changing to u64 requires a change to
devicetree. Will do it later. [Andre]
v4: improved commit message
---
arm/run | 19 ++++++++++++-------
arm/selftest.c | 5 ++++-
lib/arm/asm/processor.h | 9 +++++++--
lib/arm/asm/setup.h | 4 ++--
lib/arm/setup.c | 10 ++++++++++
lib/arm64/asm/processor.h | 9 +++++++--
6 files changed, 42 insertions(+), 14 deletions(-)
diff --git a/arm/run b/arm/run
index 1ee6231599d6..a35952b28b46 100755
--- a/arm/run
+++ b/arm/run
@@ -31,13 +31,6 @@ if [ -z "$ACCEL" ]; then
fi
fi
-if [ "$HOST" = "aarch64" ] && [ "$ACCEL" = "kvm" ]; then
- processor="host"
- if [ "$ARCH" = "arm" ]; then
- processor+=",aarch64=off"
- fi
-fi
-
qemu="${QEMU:-qemu-system-$ARCH_NAME}"
qpath=$(which $qemu 2>/dev/null)
@@ -53,6 +46,18 @@ fi
M='-machine virt'
+if [ "$ACCEL" = "kvm" ]; then
+ if $qemu $M,\? 2>&1 | grep gic-version > /dev/null; then
+ M+=',gic-version=host'
+ fi
+ if [ "$HOST" = "aarch64" ]; then
+ processor="host"
+ if [ "$ARCH" = "arm" ]; then
+ processor+=",aarch64=off"
+ fi
+ fi
+fi
+
if ! $qemu $M -device '?' 2>&1 | grep virtconsole > /dev/null; then
echo "$qpath doesn't support virtio-console for chr-testdev. Exiting."
exit 2
diff --git a/arm/selftest.c b/arm/selftest.c
index 196164f5313d..2f117f795d2d 100644
--- a/arm/selftest.c
+++ b/arm/selftest.c
@@ -312,9 +312,10 @@ static bool psci_check(void)
static cpumask_t smp_reported;
static void cpu_report(void)
{
+ unsigned long mpidr = get_mpidr();
int cpu = smp_processor_id();
- report("CPU%d online", true, cpu);
+ report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == cpu, cpu, mpidr);
cpumask_set_cpu(cpu, &smp_reported);
halt();
}
@@ -343,6 +344,7 @@ int main(int argc, char **argv)
} else if (strcmp(argv[1], "smp") == 0) {
+ unsigned long mpidr = get_mpidr();
int cpu;
report("PSCI version", psci_check());
@@ -353,6 +355,7 @@ int main(int argc, char **argv)
smp_boot_secondary(cpu, cpu_report);
}
+ report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == 0, 0, mpidr);
cpumask_set_cpu(0, &smp_reported);
while (!cpumask_full(&smp_reported))
cpu_relax();
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index 02f912f99974..ecf5bbe1824a 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -40,8 +40,13 @@ static inline unsigned long get_mpidr(void)
return mpidr;
}
-/* Only support Aff0 for now, up to 4 cpus */
-#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
+#define MPIDR_HWID_BITMASK 0xffffff
+extern int mpidr_to_cpu(unsigned long mpidr);
+
+#define MPIDR_LEVEL_SHIFT(level) \
+ (((1 << level) >> 1) << 3)
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+ ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
extern bool is_user(void);
diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
index cb8fdbd38dd5..1de99dd184d1 100644
--- a/lib/arm/asm/setup.h
+++ b/lib/arm/asm/setup.h
@@ -10,8 +10,8 @@
#include <asm/page.h>
#include <asm/pgtable-hwdef.h>
-#define NR_CPUS 8
-extern u32 cpus[NR_CPUS];
+#define NR_CPUS 255
+extern u32 cpus[NR_CPUS]; /* per-cpu IDs (MPIDRs) */
extern int nr_cpus;
#define NR_MEM_REGIONS 8
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 7e7b39f11dde..241bf9410447 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -30,6 +30,16 @@ int nr_cpus;
struct mem_region mem_regions[NR_MEM_REGIONS];
phys_addr_t __phys_offset, __phys_end;
+int mpidr_to_cpu(unsigned long mpidr)
+{
+ int i;
+
+ for (i = 0; i < nr_cpus; ++i)
+ if (cpus[i] == (mpidr & MPIDR_HWID_BITMASK))
+ return i;
+ return -1;
+}
+
static void cpu_set(int fdtnode __unused, u32 regval, void *info __unused)
{
int cpu = nr_cpus++;
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 9a208ff729b7..7e448dc81a6a 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -78,8 +78,13 @@ static inline type get_##reg(void) \
DEFINE_GET_SYSREG64(mpidr)
-/* Only support Aff0 for now, gicv2 only */
-#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
+#define MPIDR_HWID_BITMASK 0xff00ffffff
+extern int mpidr_to_cpu(unsigned long mpidr);
+
+#define MPIDR_LEVEL_SHIFT(level) \
+ (((1 << level) >> 1) << 3)
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+ ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
extern bool is_user(void);
--
2.7.4
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 03/11] arm/arm64: smp: support more than 8 cpus
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
By adding support for launching with gicv3 we can break the 8 vcpu
limit. This patch adds support to smp code and also selects the
vgic model corresponding to the host. The vgic model may also be
manually selected by adding e.g. -machine gic-version=3 to
extra_params.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v5: left cpus a u32 for now. Changing to u64 requires a change to
devicetree. Will do it later. [Andre]
v4: improved commit message
---
arm/run | 19 ++++++++++++-------
arm/selftest.c | 5 ++++-
lib/arm/asm/processor.h | 9 +++++++--
lib/arm/asm/setup.h | 4 ++--
lib/arm/setup.c | 10 ++++++++++
lib/arm64/asm/processor.h | 9 +++++++--
6 files changed, 42 insertions(+), 14 deletions(-)
diff --git a/arm/run b/arm/run
index 1ee6231599d6..a35952b28b46 100755
--- a/arm/run
+++ b/arm/run
@@ -31,13 +31,6 @@ if [ -z "$ACCEL" ]; then
fi
fi
-if [ "$HOST" = "aarch64" ] && [ "$ACCEL" = "kvm" ]; then
- processor="host"
- if [ "$ARCH" = "arm" ]; then
- processor+=",aarch64=off"
- fi
-fi
-
qemu="${QEMU:-qemu-system-$ARCH_NAME}"
qpath=$(which $qemu 2>/dev/null)
@@ -53,6 +46,18 @@ fi
M='-machine virt'
+if [ "$ACCEL" = "kvm" ]; then
+ if $qemu $M,\? 2>&1 | grep gic-version > /dev/null; then
+ M+=',gic-version=host'
+ fi
+ if [ "$HOST" = "aarch64" ]; then
+ processor="host"
+ if [ "$ARCH" = "arm" ]; then
+ processor+=",aarch64=off"
+ fi
+ fi
+fi
+
if ! $qemu $M -device '?' 2>&1 | grep virtconsole > /dev/null; then
echo "$qpath doesn't support virtio-console for chr-testdev. Exiting."
exit 2
diff --git a/arm/selftest.c b/arm/selftest.c
index 196164f5313d..2f117f795d2d 100644
--- a/arm/selftest.c
+++ b/arm/selftest.c
@@ -312,9 +312,10 @@ static bool psci_check(void)
static cpumask_t smp_reported;
static void cpu_report(void)
{
+ unsigned long mpidr = get_mpidr();
int cpu = smp_processor_id();
- report("CPU%d online", true, cpu);
+ report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == cpu, cpu, mpidr);
cpumask_set_cpu(cpu, &smp_reported);
halt();
}
@@ -343,6 +344,7 @@ int main(int argc, char **argv)
} else if (strcmp(argv[1], "smp") == 0) {
+ unsigned long mpidr = get_mpidr();
int cpu;
report("PSCI version", psci_check());
@@ -353,6 +355,7 @@ int main(int argc, char **argv)
smp_boot_secondary(cpu, cpu_report);
}
+ report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == 0, 0, mpidr);
cpumask_set_cpu(0, &smp_reported);
while (!cpumask_full(&smp_reported))
cpu_relax();
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index 02f912f99974..ecf5bbe1824a 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -40,8 +40,13 @@ static inline unsigned long get_mpidr(void)
return mpidr;
}
-/* Only support Aff0 for now, up to 4 cpus */
-#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
+#define MPIDR_HWID_BITMASK 0xffffff
+extern int mpidr_to_cpu(unsigned long mpidr);
+
+#define MPIDR_LEVEL_SHIFT(level) \
+ (((1 << level) >> 1) << 3)
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+ ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
extern bool is_user(void);
diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
index cb8fdbd38dd5..1de99dd184d1 100644
--- a/lib/arm/asm/setup.h
+++ b/lib/arm/asm/setup.h
@@ -10,8 +10,8 @@
#include <asm/page.h>
#include <asm/pgtable-hwdef.h>
-#define NR_CPUS 8
-extern u32 cpus[NR_CPUS];
+#define NR_CPUS 255
+extern u32 cpus[NR_CPUS]; /* per-cpu IDs (MPIDRs) */
extern int nr_cpus;
#define NR_MEM_REGIONS 8
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 7e7b39f11dde..241bf9410447 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -30,6 +30,16 @@ int nr_cpus;
struct mem_region mem_regions[NR_MEM_REGIONS];
phys_addr_t __phys_offset, __phys_end;
+int mpidr_to_cpu(unsigned long mpidr)
+{
+ int i;
+
+ for (i = 0; i < nr_cpus; ++i)
+ if (cpus[i] == (mpidr & MPIDR_HWID_BITMASK))
+ return i;
+ return -1;
+}
+
static void cpu_set(int fdtnode __unused, u32 regval, void *info __unused)
{
int cpu = nr_cpus++;
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 9a208ff729b7..7e448dc81a6a 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -78,8 +78,13 @@ static inline type get_##reg(void) \
DEFINE_GET_SYSREG64(mpidr)
-/* Only support Aff0 for now, gicv2 only */
-#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
+#define MPIDR_HWID_BITMASK 0xff00ffffff
+extern int mpidr_to_cpu(unsigned long mpidr);
+
+#define MPIDR_LEVEL_SHIFT(level) \
+ (((1 << level) >> 1) << 3)
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+ ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
extern bool is_user(void);
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 04/11] arm/arm64: add some delay routines
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini
Allow a thread to wait some specified amount of time. Can
specify in cycles, usecs, and msecs.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
lib/arm/asm/processor.h | 19 +++++++++++++++++++
lib/arm/processor.c | 15 +++++++++++++++
lib/arm64/asm/processor.h | 19 +++++++++++++++++++
lib/arm64/processor.c | 15 +++++++++++++++
4 files changed, 68 insertions(+)
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index ecf5bbe1824a..bc46d1f980ee 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -5,7 +5,9 @@
*
* This work is licensed under the terms of the GNU LGPL, version 2.
*/
+#include <libcflat.h>
#include <asm/ptrace.h>
+#include <asm/barrier.h>
enum vector {
EXCPTN_RST,
@@ -51,4 +53,21 @@ extern int mpidr_to_cpu(unsigned long mpidr);
extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
extern bool is_user(void);
+static inline u64 get_cntvct(void)
+{
+ u64 vct;
+ isb();
+ asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (vct));
+ return vct;
+}
+
+extern void delay(u64 cycles);
+extern void udelay(unsigned long usecs);
+
+static inline void mdelay(unsigned long msecs)
+{
+ while (msecs--)
+ udelay(1000);
+}
+
#endif /* _ASMARM_PROCESSOR_H_ */
diff --git a/lib/arm/processor.c b/lib/arm/processor.c
index 54fdb87ef019..c2ee360df688 100644
--- a/lib/arm/processor.c
+++ b/lib/arm/processor.c
@@ -9,6 +9,7 @@
#include <asm/ptrace.h>
#include <asm/processor.h>
#include <asm/thread_info.h>
+#include <asm/barrier.h>
static const char *processor_modes[] = {
"USER_26", "FIQ_26" , "IRQ_26" , "SVC_26" ,
@@ -141,3 +142,17 @@ bool is_user(void)
{
return current_thread_info()->flags & TIF_USER_MODE;
}
+
+void delay(u64 cycles)
+{
+ u64 start = get_cntvct();
+ while ((get_cntvct() - start) < cycles)
+ cpu_relax();
+}
+
+void udelay(unsigned long usec)
+{
+ unsigned int frq;
+ asm volatile("mrc p15, 0, %0, c14, c0, 0" : "=r" (frq));
+ delay((u64)usec * frq / 1000000);
+}
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 7e448dc81a6a..94f7ce35b65c 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -17,8 +17,10 @@
#define SCTLR_EL1_M (1 << 0)
#ifndef __ASSEMBLY__
+#include <libcflat.h>
#include <asm/ptrace.h>
#include <asm/esr.h>
+#include <asm/barrier.h>
enum vector {
EL1T_SYNC,
@@ -89,5 +91,22 @@ extern int mpidr_to_cpu(unsigned long mpidr);
extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
extern bool is_user(void);
+static inline u64 get_cntvct(void)
+{
+ u64 vct;
+ isb();
+ asm volatile("mrs %0, cntvct_el0" : "=r" (vct));
+ return vct;
+}
+
+extern void delay(u64 cycles);
+extern void udelay(unsigned long usecs);
+
+static inline void mdelay(unsigned long msecs)
+{
+ while (msecs--)
+ udelay(1000);
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASMARM64_PROCESSOR_H_ */
diff --git a/lib/arm64/processor.c b/lib/arm64/processor.c
index deeab4ec9c8a..50fa835c6f1e 100644
--- a/lib/arm64/processor.c
+++ b/lib/arm64/processor.c
@@ -9,6 +9,7 @@
#include <asm/ptrace.h>
#include <asm/processor.h>
#include <asm/thread_info.h>
+#include <asm/barrier.h>
static const char *vector_names[] = {
"el1t_sync",
@@ -253,3 +254,17 @@ bool is_user(void)
{
return current_thread_info()->flags & TIF_USER_MODE;
}
+
+void delay(u64 cycles)
+{
+ u64 start = get_cntvct();
+ while ((get_cntvct() - start) < cycles)
+ cpu_relax();
+}
+
+void udelay(unsigned long usec)
+{
+ unsigned int frq;
+ asm volatile("mrs %0, cntfrq_el0" : "=r" (frq));
+ delay((u64)usec * frq / 1000000);
+}
--
2.7.4
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 04/11] arm/arm64: add some delay routines
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Allow a thread to wait some specified amount of time. Can
specify in cycles, usecs, and msecs.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
lib/arm/asm/processor.h | 19 +++++++++++++++++++
lib/arm/processor.c | 15 +++++++++++++++
lib/arm64/asm/processor.h | 19 +++++++++++++++++++
lib/arm64/processor.c | 15 +++++++++++++++
4 files changed, 68 insertions(+)
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index ecf5bbe1824a..bc46d1f980ee 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -5,7 +5,9 @@
*
* This work is licensed under the terms of the GNU LGPL, version 2.
*/
+#include <libcflat.h>
#include <asm/ptrace.h>
+#include <asm/barrier.h>
enum vector {
EXCPTN_RST,
@@ -51,4 +53,21 @@ extern int mpidr_to_cpu(unsigned long mpidr);
extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
extern bool is_user(void);
+static inline u64 get_cntvct(void)
+{
+ u64 vct;
+ isb();
+ asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (vct));
+ return vct;
+}
+
+extern void delay(u64 cycles);
+extern void udelay(unsigned long usecs);
+
+static inline void mdelay(unsigned long msecs)
+{
+ while (msecs--)
+ udelay(1000);
+}
+
#endif /* _ASMARM_PROCESSOR_H_ */
diff --git a/lib/arm/processor.c b/lib/arm/processor.c
index 54fdb87ef019..c2ee360df688 100644
--- a/lib/arm/processor.c
+++ b/lib/arm/processor.c
@@ -9,6 +9,7 @@
#include <asm/ptrace.h>
#include <asm/processor.h>
#include <asm/thread_info.h>
+#include <asm/barrier.h>
static const char *processor_modes[] = {
"USER_26", "FIQ_26" , "IRQ_26" , "SVC_26" ,
@@ -141,3 +142,17 @@ bool is_user(void)
{
return current_thread_info()->flags & TIF_USER_MODE;
}
+
+void delay(u64 cycles)
+{
+ u64 start = get_cntvct();
+ while ((get_cntvct() - start) < cycles)
+ cpu_relax();
+}
+
+void udelay(unsigned long usec)
+{
+ unsigned int frq;
+ asm volatile("mrc p15, 0, %0, c14, c0, 0" : "=r" (frq));
+ delay((u64)usec * frq / 1000000);
+}
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 7e448dc81a6a..94f7ce35b65c 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -17,8 +17,10 @@
#define SCTLR_EL1_M (1 << 0)
#ifndef __ASSEMBLY__
+#include <libcflat.h>
#include <asm/ptrace.h>
#include <asm/esr.h>
+#include <asm/barrier.h>
enum vector {
EL1T_SYNC,
@@ -89,5 +91,22 @@ extern int mpidr_to_cpu(unsigned long mpidr);
extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
extern bool is_user(void);
+static inline u64 get_cntvct(void)
+{
+ u64 vct;
+ isb();
+ asm volatile("mrs %0, cntvct_el0" : "=r" (vct));
+ return vct;
+}
+
+extern void delay(u64 cycles);
+extern void udelay(unsigned long usecs);
+
+static inline void mdelay(unsigned long msecs)
+{
+ while (msecs--)
+ udelay(1000);
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASMARM64_PROCESSOR_H_ */
diff --git a/lib/arm64/processor.c b/lib/arm64/processor.c
index deeab4ec9c8a..50fa835c6f1e 100644
--- a/lib/arm64/processor.c
+++ b/lib/arm64/processor.c
@@ -9,6 +9,7 @@
#include <asm/ptrace.h>
#include <asm/processor.h>
#include <asm/thread_info.h>
+#include <asm/barrier.h>
static const char *vector_names[] = {
"el1t_sync",
@@ -253,3 +254,17 @@ bool is_user(void)
{
return current_thread_info()->flags & TIF_USER_MODE;
}
+
+void delay(u64 cycles)
+{
+ u64 start = get_cntvct();
+ while ((get_cntvct() - start) < cycles)
+ cpu_relax();
+}
+
+void udelay(unsigned long usec)
+{
+ unsigned int frq;
+ asm volatile("mrs %0, cntfrq_el0" : "=r" (frq));
+ delay((u64)usec * frq / 1000000);
+}
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 05/11] arm/arm64: irq enable/disable
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
lib/arm/asm/processor.h | 10 ++++++++++
lib/arm64/asm/processor.h | 10 ++++++++++
2 files changed, 20 insertions(+)
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index bc46d1f980ee..959ecda5dced 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -35,6 +35,16 @@ static inline unsigned long current_cpsr(void)
#define current_mode() (current_cpsr() & MODE_MASK)
+static inline void local_irq_enable(void)
+{
+ asm volatile("cpsie i" : : : "memory", "cc");
+}
+
+static inline void local_irq_disable(void)
+{
+ asm volatile("cpsid i" : : : "memory", "cc");
+}
+
static inline unsigned long get_mpidr(void)
{
unsigned long mpidr;
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 94f7ce35b65c..d54a4ed1c187 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -68,6 +68,16 @@ static inline unsigned long current_level(void)
return el & 0xc;
}
+static inline void local_irq_enable(void)
+{
+ asm volatile("msr daifclr, #2" : : : "memory");
+}
+
+static inline void local_irq_disable(void)
+{
+ asm volatile("msr daifset, #2" : : : "memory");
+}
+
#define DEFINE_GET_SYSREG(reg, type) \
static inline type get_##reg(void) \
{ \
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 05/11] arm/arm64: irq enable/disable
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
lib/arm/asm/processor.h | 10 ++++++++++
lib/arm64/asm/processor.h | 10 ++++++++++
2 files changed, 20 insertions(+)
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index bc46d1f980ee..959ecda5dced 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -35,6 +35,16 @@ static inline unsigned long current_cpsr(void)
#define current_mode() (current_cpsr() & MODE_MASK)
+static inline void local_irq_enable(void)
+{
+ asm volatile("cpsie i" : : : "memory", "cc");
+}
+
+static inline void local_irq_disable(void)
+{
+ asm volatile("cpsid i" : : : "memory", "cc");
+}
+
static inline unsigned long get_mpidr(void)
{
unsigned long mpidr;
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 94f7ce35b65c..d54a4ed1c187 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -68,6 +68,16 @@ static inline unsigned long current_level(void)
return el & 0xc;
}
+static inline void local_irq_enable(void)
+{
+ asm volatile("msr daifclr, #2" : : : "memory");
+}
+
+static inline void local_irq_disable(void)
+{
+ asm volatile("msr daifset, #2" : : : "memory");
+}
+
#define DEFINE_GET_SYSREG(reg, type) \
static inline type get_##reg(void) \
{ \
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 06/11] arm/arm64: add initial gicv2 support
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Add some gicv2 support. This just adds init and enable
functions, allowing unit tests to start messing with it.
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6: added comments (register offset headers) [Alex]
v5: share/use only the modern register names [Andre]
v4:
- only take defines from kernel we need now [Andre]
- moved defines to asm/gic.h so they'll be shared with v3 [drew]
- simplify enable by not caring if we reinit the distributor [drew]
- init all GICD_INT_DEF_PRI_X4 registers [Eric]
---
arm/Makefile.common | 1 +
lib/arm/asm/gic-v2.h | 34 ++++++++++++++++++++++
lib/arm/asm/gic.h | 39 ++++++++++++++++++++++++++
lib/arm/gic.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
lib/arm64/asm/gic-v2.h | 1 +
lib/arm64/asm/gic.h | 1 +
6 files changed, 152 insertions(+)
create mode 100644 lib/arm/asm/gic-v2.h
create mode 100644 lib/arm/asm/gic.h
create mode 100644 lib/arm/gic.c
create mode 100644 lib/arm64/asm/gic-v2.h
create mode 100644 lib/arm64/asm/gic.h
diff --git a/arm/Makefile.common b/arm/Makefile.common
index f37b5c2a3de4..6f56015c43c4 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -46,6 +46,7 @@ cflatobjs += lib/arm/mmu.o
cflatobjs += lib/arm/bitops.o
cflatobjs += lib/arm/psci.o
cflatobjs += lib/arm/smp.o
+cflatobjs += lib/arm/gic.o
libeabi = lib/arm/libeabi.a
eabiobjs = lib/arm/eabi_compat.o
diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
new file mode 100644
index 000000000000..c2d5fecd4886
--- /dev/null
+++ b/lib/arm/asm/gic-v2.h
@@ -0,0 +1,34 @@
+/*
+ * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V2_H_
+#define _ASMARM_GIC_V2_H_
+
+#ifndef _ASMARM_GIC_H_
+#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
+#endif
+
+#define GICD_ENABLE 0x1
+#define GICC_ENABLE 0x1
+
+#ifndef __ASSEMBLY__
+
+struct gicv2_data {
+ void *dist_base;
+ void *cpu_base;
+ unsigned int irq_nr;
+};
+extern struct gicv2_data gicv2_data;
+
+#define gicv2_dist_base() (gicv2_data.dist_base)
+#define gicv2_cpu_base() (gicv2_data.cpu_base)
+
+extern int gicv2_init(void);
+extern void gicv2_enable_defaults(void);
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V2_H_ */
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
new file mode 100644
index 000000000000..e3580bd1d42d
--- /dev/null
+++ b/lib/arm/asm/gic.h
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_H_
+#define _ASMARM_GIC_H_
+
+#include <asm/gic-v2.h>
+
+/* Distributor registers */
+#define GICD_CTLR 0x0000
+#define GICD_TYPER 0x0004
+#define GICD_ISENABLER 0x0100
+#define GICD_IPRIORITYR 0x0400
+
+#define GICD_TYPER_IRQS(typer) ((((typer) & 0x1f) + 1) * 32)
+#define GICD_INT_EN_SET_SGI 0x0000ffff
+#define GICD_INT_DEF_PRI_X4 0xa0a0a0a0
+
+/* CPU interface registers */
+#define GICC_CTLR 0x0000
+#define GICC_PMR 0x0004
+
+#define GICC_INT_PRI_THRESHOLD 0xf0
+
+#ifndef __ASSEMBLY__
+
+/*
+ * gic_init will try to find all known gics, and then
+ * initialize the gic data for the one found.
+ * returns
+ * 0 : no gic was found
+ * > 0 : the gic version of the gic found
+ */
+extern int gic_init(void);
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_H_ */
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
new file mode 100644
index 000000000000..d655105e058b
--- /dev/null
+++ b/lib/arm/gic.c
@@ -0,0 +1,76 @@
+/*
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <devicetree.h>
+#include <asm/gic.h>
+#include <asm/io.h>
+
+struct gicv2_data gicv2_data;
+
+/*
+ * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
+ */
+static bool
+gic_get_dt_bases(const char *compatible, void **base1, void **base2)
+{
+ struct dt_pbus_reg reg;
+ struct dt_device gic;
+ struct dt_bus bus;
+ int node, ret;
+
+ dt_bus_init_defaults(&bus);
+ dt_device_init(&gic, &bus, NULL);
+
+ node = dt_device_find_compatible(&gic, compatible);
+ assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
+
+ if (node == -FDT_ERR_NOTFOUND)
+ return false;
+
+ dt_device_bind_node(&gic, node);
+
+ ret = dt_pbus_translate(&gic, 0, ®);
+ assert(ret == 0);
+ *base1 = ioremap(reg.addr, reg.size);
+
+ ret = dt_pbus_translate(&gic, 1, ®);
+ assert(ret == 0);
+ *base2 = ioremap(reg.addr, reg.size);
+
+ return true;
+}
+
+int gicv2_init(void)
+{
+ return gic_get_dt_bases("arm,cortex-a15-gic",
+ &gicv2_data.dist_base, &gicv2_data.cpu_base);
+}
+
+int gic_init(void)
+{
+ if (gicv2_init())
+ return 2;
+ return 0;
+}
+
+void gicv2_enable_defaults(void)
+{
+ void *dist = gicv2_dist_base();
+ void *cpu_base = gicv2_cpu_base();
+ unsigned int i;
+
+ gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
+ if (gicv2_data.irq_nr > 1020)
+ gicv2_data.irq_nr = 1020;
+
+ for (i = 0; i < gicv2_data.irq_nr; i += 4)
+ writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
+
+ writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
+ writel(GICD_ENABLE, dist + GICD_CTLR);
+
+ writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
+ writel(GICC_ENABLE, cpu_base + GICC_CTLR);
+}
diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
new file mode 100644
index 000000000000..52226624a209
--- /dev/null
+++ b/lib/arm64/asm/gic-v2.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v2.h"
diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
new file mode 100644
index 000000000000..e5eb302a31b4
--- /dev/null
+++ b/lib/arm64/asm/gic.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic.h"
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 06/11] arm/arm64: add initial gicv2 support
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Add some gicv2 support. This just adds init and enable
functions, allowing unit tests to start messing with it.
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6: added comments (register offset headers) [Alex]
v5: share/use only the modern register names [Andre]
v4:
- only take defines from kernel we need now [Andre]
- moved defines to asm/gic.h so they'll be shared with v3 [drew]
- simplify enable by not caring if we reinit the distributor [drew]
- init all GICD_INT_DEF_PRI_X4 registers [Eric]
---
arm/Makefile.common | 1 +
lib/arm/asm/gic-v2.h | 34 ++++++++++++++++++++++
lib/arm/asm/gic.h | 39 ++++++++++++++++++++++++++
lib/arm/gic.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
lib/arm64/asm/gic-v2.h | 1 +
lib/arm64/asm/gic.h | 1 +
6 files changed, 152 insertions(+)
create mode 100644 lib/arm/asm/gic-v2.h
create mode 100644 lib/arm/asm/gic.h
create mode 100644 lib/arm/gic.c
create mode 100644 lib/arm64/asm/gic-v2.h
create mode 100644 lib/arm64/asm/gic.h
diff --git a/arm/Makefile.common b/arm/Makefile.common
index f37b5c2a3de4..6f56015c43c4 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -46,6 +46,7 @@ cflatobjs += lib/arm/mmu.o
cflatobjs += lib/arm/bitops.o
cflatobjs += lib/arm/psci.o
cflatobjs += lib/arm/smp.o
+cflatobjs += lib/arm/gic.o
libeabi = lib/arm/libeabi.a
eabiobjs = lib/arm/eabi_compat.o
diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
new file mode 100644
index 000000000000..c2d5fecd4886
--- /dev/null
+++ b/lib/arm/asm/gic-v2.h
@@ -0,0 +1,34 @@
+/*
+ * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V2_H_
+#define _ASMARM_GIC_V2_H_
+
+#ifndef _ASMARM_GIC_H_
+#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
+#endif
+
+#define GICD_ENABLE 0x1
+#define GICC_ENABLE 0x1
+
+#ifndef __ASSEMBLY__
+
+struct gicv2_data {
+ void *dist_base;
+ void *cpu_base;
+ unsigned int irq_nr;
+};
+extern struct gicv2_data gicv2_data;
+
+#define gicv2_dist_base() (gicv2_data.dist_base)
+#define gicv2_cpu_base() (gicv2_data.cpu_base)
+
+extern int gicv2_init(void);
+extern void gicv2_enable_defaults(void);
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V2_H_ */
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
new file mode 100644
index 000000000000..e3580bd1d42d
--- /dev/null
+++ b/lib/arm/asm/gic.h
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_H_
+#define _ASMARM_GIC_H_
+
+#include <asm/gic-v2.h>
+
+/* Distributor registers */
+#define GICD_CTLR 0x0000
+#define GICD_TYPER 0x0004
+#define GICD_ISENABLER 0x0100
+#define GICD_IPRIORITYR 0x0400
+
+#define GICD_TYPER_IRQS(typer) ((((typer) & 0x1f) + 1) * 32)
+#define GICD_INT_EN_SET_SGI 0x0000ffff
+#define GICD_INT_DEF_PRI_X4 0xa0a0a0a0
+
+/* CPU interface registers */
+#define GICC_CTLR 0x0000
+#define GICC_PMR 0x0004
+
+#define GICC_INT_PRI_THRESHOLD 0xf0
+
+#ifndef __ASSEMBLY__
+
+/*
+ * gic_init will try to find all known gics, and then
+ * initialize the gic data for the one found.
+ * returns
+ * 0 : no gic was found
+ * > 0 : the gic version of the gic found
+ */
+extern int gic_init(void);
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_H_ */
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
new file mode 100644
index 000000000000..d655105e058b
--- /dev/null
+++ b/lib/arm/gic.c
@@ -0,0 +1,76 @@
+/*
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <devicetree.h>
+#include <asm/gic.h>
+#include <asm/io.h>
+
+struct gicv2_data gicv2_data;
+
+/*
+ * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
+ */
+static bool
+gic_get_dt_bases(const char *compatible, void **base1, void **base2)
+{
+ struct dt_pbus_reg reg;
+ struct dt_device gic;
+ struct dt_bus bus;
+ int node, ret;
+
+ dt_bus_init_defaults(&bus);
+ dt_device_init(&gic, &bus, NULL);
+
+ node = dt_device_find_compatible(&gic, compatible);
+ assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
+
+ if (node == -FDT_ERR_NOTFOUND)
+ return false;
+
+ dt_device_bind_node(&gic, node);
+
+ ret = dt_pbus_translate(&gic, 0, ®);
+ assert(ret == 0);
+ *base1 = ioremap(reg.addr, reg.size);
+
+ ret = dt_pbus_translate(&gic, 1, ®);
+ assert(ret == 0);
+ *base2 = ioremap(reg.addr, reg.size);
+
+ return true;
+}
+
+int gicv2_init(void)
+{
+ return gic_get_dt_bases("arm,cortex-a15-gic",
+ &gicv2_data.dist_base, &gicv2_data.cpu_base);
+}
+
+int gic_init(void)
+{
+ if (gicv2_init())
+ return 2;
+ return 0;
+}
+
+void gicv2_enable_defaults(void)
+{
+ void *dist = gicv2_dist_base();
+ void *cpu_base = gicv2_cpu_base();
+ unsigned int i;
+
+ gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
+ if (gicv2_data.irq_nr > 1020)
+ gicv2_data.irq_nr = 1020;
+
+ for (i = 0; i < gicv2_data.irq_nr; i += 4)
+ writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
+
+ writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
+ writel(GICD_ENABLE, dist + GICD_CTLR);
+
+ writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
+ writel(GICC_ENABLE, cpu_base + GICC_CTLR);
+}
diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
new file mode 100644
index 000000000000..52226624a209
--- /dev/null
+++ b/lib/arm64/asm/gic-v2.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v2.h"
diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
new file mode 100644
index 000000000000..e5eb302a31b4
--- /dev/null
+++ b/lib/arm64/asm/gic.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic.h"
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 07/11] arm/arm64: gicv2: add an IPI test
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6: move the spurious check to its own check_ function [drew]
v5: use modern registers [Andre]
v4: properly mask irqnr in ipi_handler
v2: add more details in the output if a test fails,
report spurious interrupts if we get them
---
arm/Makefile.common | 8 +--
arm/gic.c | 199 +++++++++++++++++++++++++++++++++++++++++++++++++++
arm/unittests.cfg | 8 +++
lib/arm/asm/gic-v2.h | 2 +
lib/arm/asm/gic.h | 4 ++
5 files changed, 217 insertions(+), 4 deletions(-)
create mode 100644 arm/gic.c
diff --git a/arm/Makefile.common b/arm/Makefile.common
index 6f56015c43c4..2fe7aeeca6d4 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -9,10 +9,10 @@ ifeq ($(LOADADDR),)
LOADADDR = 0x40000000
endif
-tests-common = \
- $(TEST_DIR)/selftest.flat \
- $(TEST_DIR)/spinlock-test.flat \
- $(TEST_DIR)/pci-test.flat
+tests-common = $(TEST_DIR)/selftest.flat
+tests-common += $(TEST_DIR)/spinlock-test.flat
+tests-common += $(TEST_DIR)/pci-test.flat
+tests-common += $(TEST_DIR)/gic.flat
all: test_cases
diff --git a/arm/gic.c b/arm/gic.c
new file mode 100644
index 000000000000..b42c2b1ca1e1
--- /dev/null
+++ b/arm/gic.c
@@ -0,0 +1,199 @@
+/*
+ * GIC tests
+ *
+ * GICv2
+ * + test sending/receiving IPIs
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <libcflat.h>
+#include <asm/setup.h>
+#include <asm/processor.h>
+#include <asm/gic.h>
+#include <asm/smp.h>
+#include <asm/barrier.h>
+#include <asm/io.h>
+
+static int gic_version;
+static int acked[NR_CPUS], spurious[NR_CPUS];
+static cpumask_t ready;
+
+static void nr_cpu_check(int nr)
+{
+ if (nr_cpus < nr)
+ report_abort("At least %d cpus required", nr);
+}
+
+static void wait_on_ready(void)
+{
+ cpumask_set_cpu(smp_processor_id(), &ready);
+ while (!cpumask_full(&ready))
+ cpu_relax();
+}
+
+static void check_acked(cpumask_t *mask)
+{
+ int missing = 0, extra = 0, unexpected = 0;
+ int nr_pass, cpu, i;
+
+ /* Wait up to 5s for all interrupts to be delivered */
+ for (i = 0; i < 50; ++i) {
+ mdelay(100);
+ nr_pass = 0;
+ for_each_present_cpu(cpu) {
+ smp_rmb();
+ nr_pass += cpumask_test_cpu(cpu, mask) ?
+ acked[cpu] == 1 : acked[cpu] == 0;
+ }
+ if (nr_pass == nr_cpus) {
+ report("Completed in %d ms", true, ++i * 100);
+ return;
+ }
+ }
+
+ for_each_present_cpu(cpu) {
+ if (cpumask_test_cpu(cpu, mask)) {
+ if (!acked[cpu])
+ ++missing;
+ else if (acked[cpu] > 1)
+ ++extra;
+ } else {
+ if (acked[cpu])
+ ++unexpected;
+ }
+ }
+
+ report("Timed-out (5s). ACKS: missing=%d extra=%d unexpected=%d",
+ false, missing, extra, unexpected);
+}
+
+static void check_spurious(void)
+{
+ int cpu;
+
+ smp_rmb();
+ for_each_present_cpu(cpu) {
+ if (spurious[cpu])
+ printf("ipi: WARN: cpu%d got %d spurious interrupts\n",
+ spurious[cpu], smp_processor_id());
+ }
+}
+
+static void ipi_handler(struct pt_regs *regs __unused)
+{
+ u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
+ u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
+
+ if (irqnr != GICC_INT_SPURIOUS) {
+ writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+ smp_rmb(); /* pairs with wmb in ipi_test functions */
+ ++acked[smp_processor_id()];
+ smp_wmb(); /* pairs with rmb in check_acked */
+ } else {
+ ++spurious[smp_processor_id()];
+ smp_wmb();
+ }
+}
+
+static void ipi_test_self(void)
+{
+ cpumask_t mask;
+
+ report_prefix_push("self");
+ memset(acked, 0, sizeof(acked));
+ smp_wmb();
+ cpumask_clear(&mask);
+ cpumask_set_cpu(0, &mask);
+ writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+ check_acked(&mask);
+ report_prefix_pop();
+}
+
+static void ipi_test_smp(void)
+{
+ cpumask_t mask;
+ unsigned long tlist;
+
+ report_prefix_push("target-list");
+ memset(acked, 0, sizeof(acked));
+ smp_wmb();
+ tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
+ cpumask_bits(&mask)[0] = tlist;
+ writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
+ check_acked(&mask);
+ report_prefix_pop();
+
+ report_prefix_push("broadcast");
+ memset(acked, 0, sizeof(acked));
+ smp_wmb();
+ cpumask_copy(&mask, &cpu_present_mask);
+ cpumask_clear_cpu(0, &mask);
+ writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+ check_acked(&mask);
+ report_prefix_pop();
+}
+
+static void ipi_enable(void)
+{
+ gicv2_enable_defaults();
+#ifdef __arm__
+ install_exception_handler(EXCPTN_IRQ, ipi_handler);
+#else
+ install_irq_handler(EL1H_IRQ, ipi_handler);
+#endif
+ local_irq_enable();
+}
+
+static void ipi_recv(void)
+{
+ ipi_enable();
+ cpumask_set_cpu(smp_processor_id(), &ready);
+ while (1)
+ wfi();
+}
+
+int main(int argc, char **argv)
+{
+ char pfx[8];
+ int cpu;
+
+ gic_version = gic_init();
+ if (!gic_version)
+ report_abort("No gic present!");
+
+ snprintf(pfx, sizeof(pfx), "gicv%d", gic_version);
+ report_prefix_push(pfx);
+
+ if (argc < 2) {
+
+ report_prefix_push("ipi");
+ ipi_enable();
+ ipi_test_self();
+ check_spurious();
+ report_prefix_pop();
+
+ } else if (strcmp(argv[1], "ipi") == 0) {
+
+ report_prefix_push(argv[1]);
+ nr_cpu_check(2);
+
+ for_each_present_cpu(cpu) {
+ if (cpu == 0)
+ continue;
+ smp_boot_secondary(cpu, ipi_recv);
+ }
+ ipi_enable();
+ wait_on_ready();
+ ipi_test_self();
+ ipi_test_smp();
+ check_spurious();
+ report_prefix_pop();
+
+ } else {
+ report_abort("Unknown subtest '%s'", argv[1]);
+ }
+
+ return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index ae32a42a91c3..e631c35e2bbb 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -55,6 +55,14 @@ smp = $MAX_SMP
extra_params = -append 'smp'
groups = selftest
+# pci-testdev
[pci-test]
file = pci-test.flat
groups = pci
+
+# Test GIC emulation
+[gicv2-ipi]
+file = gic.flat
+smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
+extra_params = -machine gic-version=2 -append 'ipi'
+groups = gic
diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
index c2d5fecd4886..8b3f7ed6790c 100644
--- a/lib/arm/asm/gic-v2.h
+++ b/lib/arm/asm/gic-v2.h
@@ -13,7 +13,9 @@
#endif
#define GICD_ENABLE 0x1
+
#define GICC_ENABLE 0x1
+#define GICC_IAR_INT_ID_MASK 0x3ff
#ifndef __ASSEMBLY__
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index e3580bd1d42d..d816b96e46b4 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -13,6 +13,7 @@
#define GICD_TYPER 0x0004
#define GICD_ISENABLER 0x0100
#define GICD_IPRIORITYR 0x0400
+#define GICD_SGIR 0x0f00
#define GICD_TYPER_IRQS(typer) ((((typer) & 0x1f) + 1) * 32)
#define GICD_INT_EN_SET_SGI 0x0000ffff
@@ -21,8 +22,11 @@
/* CPU interface registers */
#define GICC_CTLR 0x0000
#define GICC_PMR 0x0004
+#define GICC_IAR 0x000c
+#define GICC_EOIR 0x0010
#define GICC_INT_PRI_THRESHOLD 0xf0
+#define GICC_INT_SPURIOUS 0x3ff
#ifndef __ASSEMBLY__
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 07/11] arm/arm64: gicv2: add an IPI test
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6: move the spurious check to its own check_ function [drew]
v5: use modern registers [Andre]
v4: properly mask irqnr in ipi_handler
v2: add more details in the output if a test fails,
report spurious interrupts if we get them
---
arm/Makefile.common | 8 +--
arm/gic.c | 199 +++++++++++++++++++++++++++++++++++++++++++++++++++
arm/unittests.cfg | 8 +++
lib/arm/asm/gic-v2.h | 2 +
lib/arm/asm/gic.h | 4 ++
5 files changed, 217 insertions(+), 4 deletions(-)
create mode 100644 arm/gic.c
diff --git a/arm/Makefile.common b/arm/Makefile.common
index 6f56015c43c4..2fe7aeeca6d4 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -9,10 +9,10 @@ ifeq ($(LOADADDR),)
LOADADDR = 0x40000000
endif
-tests-common = \
- $(TEST_DIR)/selftest.flat \
- $(TEST_DIR)/spinlock-test.flat \
- $(TEST_DIR)/pci-test.flat
+tests-common = $(TEST_DIR)/selftest.flat
+tests-common += $(TEST_DIR)/spinlock-test.flat
+tests-common += $(TEST_DIR)/pci-test.flat
+tests-common += $(TEST_DIR)/gic.flat
all: test_cases
diff --git a/arm/gic.c b/arm/gic.c
new file mode 100644
index 000000000000..b42c2b1ca1e1
--- /dev/null
+++ b/arm/gic.c
@@ -0,0 +1,199 @@
+/*
+ * GIC tests
+ *
+ * GICv2
+ * + test sending/receiving IPIs
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <libcflat.h>
+#include <asm/setup.h>
+#include <asm/processor.h>
+#include <asm/gic.h>
+#include <asm/smp.h>
+#include <asm/barrier.h>
+#include <asm/io.h>
+
+static int gic_version;
+static int acked[NR_CPUS], spurious[NR_CPUS];
+static cpumask_t ready;
+
+static void nr_cpu_check(int nr)
+{
+ if (nr_cpus < nr)
+ report_abort("At least %d cpus required", nr);
+}
+
+static void wait_on_ready(void)
+{
+ cpumask_set_cpu(smp_processor_id(), &ready);
+ while (!cpumask_full(&ready))
+ cpu_relax();
+}
+
+static void check_acked(cpumask_t *mask)
+{
+ int missing = 0, extra = 0, unexpected = 0;
+ int nr_pass, cpu, i;
+
+ /* Wait up to 5s for all interrupts to be delivered */
+ for (i = 0; i < 50; ++i) {
+ mdelay(100);
+ nr_pass = 0;
+ for_each_present_cpu(cpu) {
+ smp_rmb();
+ nr_pass += cpumask_test_cpu(cpu, mask) ?
+ acked[cpu] == 1 : acked[cpu] == 0;
+ }
+ if (nr_pass == nr_cpus) {
+ report("Completed in %d ms", true, ++i * 100);
+ return;
+ }
+ }
+
+ for_each_present_cpu(cpu) {
+ if (cpumask_test_cpu(cpu, mask)) {
+ if (!acked[cpu])
+ ++missing;
+ else if (acked[cpu] > 1)
+ ++extra;
+ } else {
+ if (acked[cpu])
+ ++unexpected;
+ }
+ }
+
+ report("Timed-out (5s). ACKS: missing=%d extra=%d unexpected=%d",
+ false, missing, extra, unexpected);
+}
+
+static void check_spurious(void)
+{
+ int cpu;
+
+ smp_rmb();
+ for_each_present_cpu(cpu) {
+ if (spurious[cpu])
+ printf("ipi: WARN: cpu%d got %d spurious interrupts\n",
+ spurious[cpu], smp_processor_id());
+ }
+}
+
+static void ipi_handler(struct pt_regs *regs __unused)
+{
+ u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
+ u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
+
+ if (irqnr != GICC_INT_SPURIOUS) {
+ writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+ smp_rmb(); /* pairs with wmb in ipi_test functions */
+ ++acked[smp_processor_id()];
+ smp_wmb(); /* pairs with rmb in check_acked */
+ } else {
+ ++spurious[smp_processor_id()];
+ smp_wmb();
+ }
+}
+
+static void ipi_test_self(void)
+{
+ cpumask_t mask;
+
+ report_prefix_push("self");
+ memset(acked, 0, sizeof(acked));
+ smp_wmb();
+ cpumask_clear(&mask);
+ cpumask_set_cpu(0, &mask);
+ writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+ check_acked(&mask);
+ report_prefix_pop();
+}
+
+static void ipi_test_smp(void)
+{
+ cpumask_t mask;
+ unsigned long tlist;
+
+ report_prefix_push("target-list");
+ memset(acked, 0, sizeof(acked));
+ smp_wmb();
+ tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
+ cpumask_bits(&mask)[0] = tlist;
+ writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
+ check_acked(&mask);
+ report_prefix_pop();
+
+ report_prefix_push("broadcast");
+ memset(acked, 0, sizeof(acked));
+ smp_wmb();
+ cpumask_copy(&mask, &cpu_present_mask);
+ cpumask_clear_cpu(0, &mask);
+ writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+ check_acked(&mask);
+ report_prefix_pop();
+}
+
+static void ipi_enable(void)
+{
+ gicv2_enable_defaults();
+#ifdef __arm__
+ install_exception_handler(EXCPTN_IRQ, ipi_handler);
+#else
+ install_irq_handler(EL1H_IRQ, ipi_handler);
+#endif
+ local_irq_enable();
+}
+
+static void ipi_recv(void)
+{
+ ipi_enable();
+ cpumask_set_cpu(smp_processor_id(), &ready);
+ while (1)
+ wfi();
+}
+
+int main(int argc, char **argv)
+{
+ char pfx[8];
+ int cpu;
+
+ gic_version = gic_init();
+ if (!gic_version)
+ report_abort("No gic present!");
+
+ snprintf(pfx, sizeof(pfx), "gicv%d", gic_version);
+ report_prefix_push(pfx);
+
+ if (argc < 2) {
+
+ report_prefix_push("ipi");
+ ipi_enable();
+ ipi_test_self();
+ check_spurious();
+ report_prefix_pop();
+
+ } else if (strcmp(argv[1], "ipi") == 0) {
+
+ report_prefix_push(argv[1]);
+ nr_cpu_check(2);
+
+ for_each_present_cpu(cpu) {
+ if (cpu == 0)
+ continue;
+ smp_boot_secondary(cpu, ipi_recv);
+ }
+ ipi_enable();
+ wait_on_ready();
+ ipi_test_self();
+ ipi_test_smp();
+ check_spurious();
+ report_prefix_pop();
+
+ } else {
+ report_abort("Unknown subtest '%s'", argv[1]);
+ }
+
+ return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index ae32a42a91c3..e631c35e2bbb 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -55,6 +55,14 @@ smp = $MAX_SMP
extra_params = -append 'smp'
groups = selftest
+# pci-testdev
[pci-test]
file = pci-test.flat
groups = pci
+
+# Test GIC emulation
+[gicv2-ipi]
+file = gic.flat
+smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
+extra_params = -machine gic-version=2 -append 'ipi'
+groups = gic
diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
index c2d5fecd4886..8b3f7ed6790c 100644
--- a/lib/arm/asm/gic-v2.h
+++ b/lib/arm/asm/gic-v2.h
@@ -13,7 +13,9 @@
#endif
#define GICD_ENABLE 0x1
+
#define GICC_ENABLE 0x1
+#define GICC_IAR_INT_ID_MASK 0x3ff
#ifndef __ASSEMBLY__
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index e3580bd1d42d..d816b96e46b4 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -13,6 +13,7 @@
#define GICD_TYPER 0x0004
#define GICD_ISENABLER 0x0100
#define GICD_IPRIORITYR 0x0400
+#define GICD_SGIR 0x0f00
#define GICD_TYPER_IRQS(typer) ((((typer) & 0x1f) + 1) * 32)
#define GICD_INT_EN_SET_SGI 0x0000ffff
@@ -21,8 +22,11 @@
/* CPU interface registers */
#define GICC_CTLR 0x0000
#define GICC_PMR 0x0004
+#define GICC_IAR 0x000c
+#define GICC_EOIR 0x0010
#define GICC_INT_PRI_THRESHOLD 0xf0
+#define GICC_INT_SPURIOUS 0x3ff
#ifndef __ASSEMBLY__
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 08/11] libcflat: add IS_ALIGNED() macro, and page sizes
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini
From: Peter Xu <peterx@redhat.com>
These macros will be useful to do page alignment checks.
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
[drew: also added SZ_64K and changed to shifts]
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6: change to shifts [Alex]
---
lib/libcflat.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/lib/libcflat.h b/lib/libcflat.h
index 82005f5d014f..244e40a724be 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -33,6 +33,12 @@
#define __ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
#define __ALIGN(x, a) __ALIGN_MASK(x, (typeof(x))(a) - 1)
#define ALIGN(x, a) __ALIGN((x), (a))
+#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
+
+#define SZ_4K (1 << 12)
+#define SZ_64K (1 << 16)
+#define SZ_2M (1 << 21)
+#define SZ_1G (1 << 30)
typedef uint8_t u8;
typedef int8_t s8;
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 08/11] libcflat: add IS_ALIGNED() macro, and page sizes
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall, Peter Xu
From: Peter Xu <peterx@redhat.com>
These macros will be useful to do page alignment checks.
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
[drew: also added SZ_64K and changed to shifts]
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6: change to shifts [Alex]
---
lib/libcflat.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/lib/libcflat.h b/lib/libcflat.h
index 82005f5d014f..244e40a724be 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -33,6 +33,12 @@
#define __ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
#define __ALIGN(x, a) __ALIGN_MASK(x, (typeof(x))(a) - 1)
#define ALIGN(x, a) __ALIGN((x), (a))
+#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
+
+#define SZ_4K (1 << 12)
+#define SZ_64K (1 << 16)
+#define SZ_2M (1 << 21)
+#define SZ_1G (1 << 30)
typedef uint8_t u8;
typedef int8_t s8;
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 09/11] arm/arm64: add initial gicv3 support
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6:
- added comments [Alex]
- added stride parameter to gicv3_set_redist_base [Andre]
- redist-wait s/rwp/uwp/ and comment [Andre]
- removed unnecessary wait-for-rwps [Andre]
v5: use modern register names [Andre]
v4:
- only take defines from kernel we need now [Andre]
- simplify enable by not caring if we reinit the distributor [drew]
v2:
- configure irqs as NS GRP1
---
lib/arm/asm/arch_gicv3.h | 47 ++++++++++++++++++++
lib/arm/asm/gic-v3.h | 104 +++++++++++++++++++++++++++++++++++++++++++++
lib/arm/asm/gic.h | 5 ++-
lib/arm/gic.c | 64 ++++++++++++++++++++++++++++
lib/arm64/asm/arch_gicv3.h | 44 +++++++++++++++++++
lib/arm64/asm/gic-v3.h | 1 +
lib/arm64/asm/sysreg.h | 44 +++++++++++++++++++
7 files changed, 308 insertions(+), 1 deletion(-)
create mode 100644 lib/arm/asm/arch_gicv3.h
create mode 100644 lib/arm/asm/gic-v3.h
create mode 100644 lib/arm64/asm/arch_gicv3.h
create mode 100644 lib/arm64/asm/gic-v3.h
create mode 100644 lib/arm64/asm/sysreg.h
diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
new file mode 100644
index 000000000000..276577452a14
--- /dev/null
+++ b/lib/arm/asm/arch_gicv3.h
@@ -0,0 +1,47 @@
+/*
+ * All ripped off from arch/arm/include/asm/arch_gicv3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_ARCH_GICV3_H_
+#define _ASMARM_ARCH_GICV3_H_
+
+#ifndef __ASSEMBLY__
+#include <libcflat.h>
+#include <asm/barrier.h>
+#include <asm/io.h>
+
+#define __stringify xstr
+
+#define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
+
+#define ICC_PMR __ACCESS_CP15(c4, 0, c6, 0)
+#define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
+
+static inline void gicv3_write_pmr(u32 val)
+{
+ asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
+}
+
+static inline void gicv3_write_grpen1(u32 val)
+{
+ asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
+ isb();
+}
+
+/*
+ * We may access GICR_TYPER and GITS_TYPER by reading both the TYPER
+ * offset and the following offset (+ 4) and then combining them to
+ * form a 64-bit address.
+ */
+static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
+{
+ u64 val = readl(addr);
+ val |= (u64)readl(addr + 4) << 32;
+ return val;
+}
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_ARCH_GICV3_H_ */
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
new file mode 100644
index 000000000000..73ade4681d21
--- /dev/null
+++ b/lib/arm/asm/gic-v3.h
@@ -0,0 +1,104 @@
+/*
+ * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V3_H_
+#define _ASMARM_GIC_V3_H_
+
+#ifndef _ASMARM_GIC_H_
+#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
+#endif
+
+/*
+ * Distributor registers
+ *
+ * We expect to be run in Non-secure mode, thus we define the
+ * group1 enable bits with respect to that view.
+ */
+#define GICD_CTLR_RWP (1U << 31)
+#define GICD_CTLR_ARE_NS (1U << 4)
+#define GICD_CTLR_ENABLE_G1A (1U << 1)
+#define GICD_CTLR_ENABLE_G1 (1U << 0)
+
+/* Re-Distributor registers, offsets from RD_base */
+#define GICR_TYPER 0x0008
+
+#define GICR_TYPER_LAST (1U << 4)
+
+/* Re-Distributor registers, offsets from SGI_base */
+#define GICR_IGROUPR0 GICD_IGROUPR
+#define GICR_ISENABLER0 GICD_ISENABLER
+#define GICR_IPRIORITYR0 GICD_IPRIORITYR
+
+#include <asm/arch_gicv3.h>
+
+#ifndef __ASSEMBLY__
+#include <asm/setup.h>
+#include <asm/smp.h>
+#include <asm/processor.h>
+#include <asm/io.h>
+
+struct gicv3_data {
+ void *dist_base;
+ void *redist_base[NR_CPUS];
+ unsigned int irq_nr;
+};
+extern struct gicv3_data gicv3_data;
+
+#define gicv3_dist_base() (gicv3_data.dist_base)
+#define gicv3_redist_base() (gicv3_data.redist_base[smp_processor_id()])
+#define gicv3_sgi_base() (gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
+
+extern int gicv3_init(void);
+extern void gicv3_enable_defaults(void);
+extern void gicv3_set_redist_base(size_t stride);
+
+static inline void gicv3_do_wait_for_rwp(void *base)
+{
+ int count = 100000; /* 1s */
+
+ while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
+ if (!--count) {
+ printf("GICv3: RWP timeout!\n");
+ abort();
+ }
+ cpu_relax();
+ udelay(10);
+ };
+}
+
+static inline void gicv3_dist_wait_for_rwp(void)
+{
+ gicv3_do_wait_for_rwp(gicv3_dist_base());
+}
+
+static inline void gicv3_redist_wait_for_uwp(void)
+{
+ /*
+ * We can build on gic_do_wait_for_rwp, which uses GICD_ registers
+ * because GICD_CTLR == GICR_CTLR and GICD_CTLR_RWP == GICR_CTLR_UWP
+ */
+ gicv3_do_wait_for_rwp(gicv3_redist_base());
+}
+
+static inline u32 mpidr_compress(u64 mpidr)
+{
+ u64 compressed = mpidr & MPIDR_HWID_BITMASK;
+
+ compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
+ return compressed;
+}
+
+static inline u64 mpidr_uncompress(u32 compressed)
+{
+ u64 mpidr = ((u64)compressed >> 24) << 32;
+
+ mpidr |= compressed & MPIDR_HWID_BITMASK;
+ return mpidr;
+}
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V3_H_ */
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index d816b96e46b4..21511997f2a9 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -6,11 +6,11 @@
#ifndef _ASMARM_GIC_H_
#define _ASMARM_GIC_H_
-#include <asm/gic-v2.h>
/* Distributor registers */
#define GICD_CTLR 0x0000
#define GICD_TYPER 0x0004
+#define GICD_IGROUPR 0x0080
#define GICD_ISENABLER 0x0100
#define GICD_IPRIORITYR 0x0400
#define GICD_SGIR 0x0f00
@@ -28,6 +28,9 @@
#define GICC_INT_PRI_THRESHOLD 0xf0
#define GICC_INT_SPURIOUS 0x3ff
+#include <asm/gic-v2.h>
+#include <asm/gic-v3.h>
+
#ifndef __ASSEMBLY__
/*
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index d655105e058b..d703ad96a37e 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -8,9 +8,11 @@
#include <asm/io.h>
struct gicv2_data gicv2_data;
+struct gicv3_data gicv3_data;
/*
* Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
+ * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
*/
static bool
gic_get_dt_bases(const char *compatible, void **base1, void **base2)
@@ -48,10 +50,18 @@ int gicv2_init(void)
&gicv2_data.dist_base, &gicv2_data.cpu_base);
}
+int gicv3_init(void)
+{
+ return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
+ &gicv3_data.redist_base[0]);
+}
+
int gic_init(void)
{
if (gicv2_init())
return 2;
+ else if (gicv3_init())
+ return 3;
return 0;
}
@@ -74,3 +84,57 @@ void gicv2_enable_defaults(void)
writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
writel(GICC_ENABLE, cpu_base + GICC_CTLR);
}
+
+void gicv3_set_redist_base(size_t stride)
+{
+ u32 aff = mpidr_compress(get_mpidr());
+ void *ptr = gicv3_data.redist_base[0];
+ u64 typer;
+
+ do {
+ typer = gicv3_read_typer(ptr + GICR_TYPER);
+ if ((typer >> 32) == aff) {
+ gicv3_redist_base() = ptr;
+ return;
+ }
+ ptr += stride; /* skip RD_base, SGI_base, etc. */
+ } while (!(typer & GICR_TYPER_LAST));
+
+ /* should never reach here */
+ assert(0);
+}
+
+void gicv3_enable_defaults(void)
+{
+ void *dist = gicv3_dist_base();
+ void *sgi_base;
+ unsigned int i;
+
+ gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
+ if (gicv3_data.irq_nr > 1020)
+ gicv3_data.irq_nr = 1020;
+
+ writel(0, dist + GICD_CTLR);
+ gicv3_dist_wait_for_rwp();
+
+ writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
+ dist + GICD_CTLR);
+ gicv3_dist_wait_for_rwp();
+
+ for (i = 0; i < gicv3_data.irq_nr; i += 4)
+ writel(~0, dist + GICD_IGROUPR + i);
+
+ if (!gicv3_redist_base())
+ gicv3_set_redist_base(SZ_64K * 2);
+ sgi_base = gicv3_sgi_base();
+
+ writel(~0, sgi_base + GICR_IGROUPR0);
+
+ for (i = 0; i < 16; i += 4)
+ writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
+
+ writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
+
+ gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
+ gicv3_write_grpen1(1);
+}
diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
new file mode 100644
index 000000000000..6d353567f56a
--- /dev/null
+++ b/lib/arm64/asm/arch_gicv3.h
@@ -0,0 +1,44 @@
+/*
+ * All ripped off from arch/arm64/include/asm/arch_gicv3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM64_ARCH_GICV3_H_
+#define _ASMARM64_ARCH_GICV3_H_
+
+#include <asm/sysreg.h>
+
+#define ICC_PMR_EL1 sys_reg(3, 0, 4, 6, 0)
+#define ICC_GRPEN1_EL1 sys_reg(3, 0, 12, 12, 7)
+
+#ifndef __ASSEMBLY__
+
+#include <libcflat.h>
+#include <asm/barrier.h>
+
+#define __stringify xstr
+
+/*
+ * Low-level accessors
+ *
+ * These system registers are 32 bits, but we make sure that the compiler
+ * sets the GP register's most significant bits to 0 with an explicit cast.
+ */
+
+static inline void gicv3_write_pmr(u32 val)
+{
+ asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
+}
+
+static inline void gicv3_write_grpen1(u32 val)
+{
+ asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
+ isb();
+}
+
+#define gicv3_read_typer(c) readq(c)
+
+#endif /* __ASSEMBLY__ */
+#endif /* _ASMARM64_ARCH_GICV3_H_ */
diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
new file mode 100644
index 000000000000..8ee5d4d9c181
--- /dev/null
+++ b/lib/arm64/asm/gic-v3.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v3.h"
diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
new file mode 100644
index 000000000000..544a46cb8cc5
--- /dev/null
+++ b/lib/arm64/asm/sysreg.h
@@ -0,0 +1,44 @@
+/*
+ * Ripped off from arch/arm64/include/asm/sysreg.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM64_SYSREG_H_
+#define _ASMARM64_SYSREG_H_
+
+#define sys_reg(op0, op1, crn, crm, op2) \
+ ((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
+
+#ifdef __ASSEMBLY__
+ .irp num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
+ .equ .L__reg_num_x\num, \num
+ .endr
+ .equ .L__reg_num_xzr, 31
+
+ .macro mrs_s, rt, sreg
+ .inst 0xd5200000|(\sreg)|(.L__reg_num_\rt)
+ .endm
+
+ .macro msr_s, sreg, rt
+ .inst 0xd5000000|(\sreg)|(.L__reg_num_\rt)
+ .endm
+#else
+asm(
+" .irp num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
+" .equ .L__reg_num_x\\num, \\num\n"
+" .endr\n"
+" .equ .L__reg_num_xzr, 31\n"
+"\n"
+" .macro mrs_s, rt, sreg\n"
+" .inst 0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
+" .endm\n"
+"\n"
+" .macro msr_s, sreg, rt\n"
+" .inst 0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
+" .endm\n"
+);
+#endif
+
+#endif /* _ASMARM64_SYSREG_H_ */
--
2.7.4
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 09/11] arm/arm64: add initial gicv3 support
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6:
- added comments [Alex]
- added stride parameter to gicv3_set_redist_base [Andre]
- redist-wait s/rwp/uwp/ and comment [Andre]
- removed unnecessary wait-for-rwps [Andre]
v5: use modern register names [Andre]
v4:
- only take defines from kernel we need now [Andre]
- simplify enable by not caring if we reinit the distributor [drew]
v2:
- configure irqs as NS GRP1
---
lib/arm/asm/arch_gicv3.h | 47 ++++++++++++++++++++
lib/arm/asm/gic-v3.h | 104 +++++++++++++++++++++++++++++++++++++++++++++
lib/arm/asm/gic.h | 5 ++-
lib/arm/gic.c | 64 ++++++++++++++++++++++++++++
lib/arm64/asm/arch_gicv3.h | 44 +++++++++++++++++++
lib/arm64/asm/gic-v3.h | 1 +
lib/arm64/asm/sysreg.h | 44 +++++++++++++++++++
7 files changed, 308 insertions(+), 1 deletion(-)
create mode 100644 lib/arm/asm/arch_gicv3.h
create mode 100644 lib/arm/asm/gic-v3.h
create mode 100644 lib/arm64/asm/arch_gicv3.h
create mode 100644 lib/arm64/asm/gic-v3.h
create mode 100644 lib/arm64/asm/sysreg.h
diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
new file mode 100644
index 000000000000..276577452a14
--- /dev/null
+++ b/lib/arm/asm/arch_gicv3.h
@@ -0,0 +1,47 @@
+/*
+ * All ripped off from arch/arm/include/asm/arch_gicv3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_ARCH_GICV3_H_
+#define _ASMARM_ARCH_GICV3_H_
+
+#ifndef __ASSEMBLY__
+#include <libcflat.h>
+#include <asm/barrier.h>
+#include <asm/io.h>
+
+#define __stringify xstr
+
+#define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
+
+#define ICC_PMR __ACCESS_CP15(c4, 0, c6, 0)
+#define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
+
+static inline void gicv3_write_pmr(u32 val)
+{
+ asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
+}
+
+static inline void gicv3_write_grpen1(u32 val)
+{
+ asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
+ isb();
+}
+
+/*
+ * We may access GICR_TYPER and GITS_TYPER by reading both the TYPER
+ * offset and the following offset (+ 4) and then combining them to
+ * form a 64-bit address.
+ */
+static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
+{
+ u64 val = readl(addr);
+ val |= (u64)readl(addr + 4) << 32;
+ return val;
+}
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_ARCH_GICV3_H_ */
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
new file mode 100644
index 000000000000..73ade4681d21
--- /dev/null
+++ b/lib/arm/asm/gic-v3.h
@@ -0,0 +1,104 @@
+/*
+ * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V3_H_
+#define _ASMARM_GIC_V3_H_
+
+#ifndef _ASMARM_GIC_H_
+#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
+#endif
+
+/*
+ * Distributor registers
+ *
+ * We expect to be run in Non-secure mode, thus we define the
+ * group1 enable bits with respect to that view.
+ */
+#define GICD_CTLR_RWP (1U << 31)
+#define GICD_CTLR_ARE_NS (1U << 4)
+#define GICD_CTLR_ENABLE_G1A (1U << 1)
+#define GICD_CTLR_ENABLE_G1 (1U << 0)
+
+/* Re-Distributor registers, offsets from RD_base */
+#define GICR_TYPER 0x0008
+
+#define GICR_TYPER_LAST (1U << 4)
+
+/* Re-Distributor registers, offsets from SGI_base */
+#define GICR_IGROUPR0 GICD_IGROUPR
+#define GICR_ISENABLER0 GICD_ISENABLER
+#define GICR_IPRIORITYR0 GICD_IPRIORITYR
+
+#include <asm/arch_gicv3.h>
+
+#ifndef __ASSEMBLY__
+#include <asm/setup.h>
+#include <asm/smp.h>
+#include <asm/processor.h>
+#include <asm/io.h>
+
+struct gicv3_data {
+ void *dist_base;
+ void *redist_base[NR_CPUS];
+ unsigned int irq_nr;
+};
+extern struct gicv3_data gicv3_data;
+
+#define gicv3_dist_base() (gicv3_data.dist_base)
+#define gicv3_redist_base() (gicv3_data.redist_base[smp_processor_id()])
+#define gicv3_sgi_base() (gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
+
+extern int gicv3_init(void);
+extern void gicv3_enable_defaults(void);
+extern void gicv3_set_redist_base(size_t stride);
+
+static inline void gicv3_do_wait_for_rwp(void *base)
+{
+ int count = 100000; /* 1s */
+
+ while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
+ if (!--count) {
+ printf("GICv3: RWP timeout!\n");
+ abort();
+ }
+ cpu_relax();
+ udelay(10);
+ };
+}
+
+static inline void gicv3_dist_wait_for_rwp(void)
+{
+ gicv3_do_wait_for_rwp(gicv3_dist_base());
+}
+
+static inline void gicv3_redist_wait_for_uwp(void)
+{
+ /*
+ * We can build on gic_do_wait_for_rwp, which uses GICD_ registers
+ * because GICD_CTLR == GICR_CTLR and GICD_CTLR_RWP == GICR_CTLR_UWP
+ */
+ gicv3_do_wait_for_rwp(gicv3_redist_base());
+}
+
+static inline u32 mpidr_compress(u64 mpidr)
+{
+ u64 compressed = mpidr & MPIDR_HWID_BITMASK;
+
+ compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
+ return compressed;
+}
+
+static inline u64 mpidr_uncompress(u32 compressed)
+{
+ u64 mpidr = ((u64)compressed >> 24) << 32;
+
+ mpidr |= compressed & MPIDR_HWID_BITMASK;
+ return mpidr;
+}
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V3_H_ */
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index d816b96e46b4..21511997f2a9 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -6,11 +6,11 @@
#ifndef _ASMARM_GIC_H_
#define _ASMARM_GIC_H_
-#include <asm/gic-v2.h>
/* Distributor registers */
#define GICD_CTLR 0x0000
#define GICD_TYPER 0x0004
+#define GICD_IGROUPR 0x0080
#define GICD_ISENABLER 0x0100
#define GICD_IPRIORITYR 0x0400
#define GICD_SGIR 0x0f00
@@ -28,6 +28,9 @@
#define GICC_INT_PRI_THRESHOLD 0xf0
#define GICC_INT_SPURIOUS 0x3ff
+#include <asm/gic-v2.h>
+#include <asm/gic-v3.h>
+
#ifndef __ASSEMBLY__
/*
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index d655105e058b..d703ad96a37e 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -8,9 +8,11 @@
#include <asm/io.h>
struct gicv2_data gicv2_data;
+struct gicv3_data gicv3_data;
/*
* Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
+ * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
*/
static bool
gic_get_dt_bases(const char *compatible, void **base1, void **base2)
@@ -48,10 +50,18 @@ int gicv2_init(void)
&gicv2_data.dist_base, &gicv2_data.cpu_base);
}
+int gicv3_init(void)
+{
+ return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
+ &gicv3_data.redist_base[0]);
+}
+
int gic_init(void)
{
if (gicv2_init())
return 2;
+ else if (gicv3_init())
+ return 3;
return 0;
}
@@ -74,3 +84,57 @@ void gicv2_enable_defaults(void)
writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
writel(GICC_ENABLE, cpu_base + GICC_CTLR);
}
+
+void gicv3_set_redist_base(size_t stride)
+{
+ u32 aff = mpidr_compress(get_mpidr());
+ void *ptr = gicv3_data.redist_base[0];
+ u64 typer;
+
+ do {
+ typer = gicv3_read_typer(ptr + GICR_TYPER);
+ if ((typer >> 32) == aff) {
+ gicv3_redist_base() = ptr;
+ return;
+ }
+ ptr += stride; /* skip RD_base, SGI_base, etc. */
+ } while (!(typer & GICR_TYPER_LAST));
+
+ /* should never reach here */
+ assert(0);
+}
+
+void gicv3_enable_defaults(void)
+{
+ void *dist = gicv3_dist_base();
+ void *sgi_base;
+ unsigned int i;
+
+ gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
+ if (gicv3_data.irq_nr > 1020)
+ gicv3_data.irq_nr = 1020;
+
+ writel(0, dist + GICD_CTLR);
+ gicv3_dist_wait_for_rwp();
+
+ writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
+ dist + GICD_CTLR);
+ gicv3_dist_wait_for_rwp();
+
+ for (i = 0; i < gicv3_data.irq_nr; i += 4)
+ writel(~0, dist + GICD_IGROUPR + i);
+
+ if (!gicv3_redist_base())
+ gicv3_set_redist_base(SZ_64K * 2);
+ sgi_base = gicv3_sgi_base();
+
+ writel(~0, sgi_base + GICR_IGROUPR0);
+
+ for (i = 0; i < 16; i += 4)
+ writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
+
+ writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
+
+ gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
+ gicv3_write_grpen1(1);
+}
diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
new file mode 100644
index 000000000000..6d353567f56a
--- /dev/null
+++ b/lib/arm64/asm/arch_gicv3.h
@@ -0,0 +1,44 @@
+/*
+ * All ripped off from arch/arm64/include/asm/arch_gicv3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM64_ARCH_GICV3_H_
+#define _ASMARM64_ARCH_GICV3_H_
+
+#include <asm/sysreg.h>
+
+#define ICC_PMR_EL1 sys_reg(3, 0, 4, 6, 0)
+#define ICC_GRPEN1_EL1 sys_reg(3, 0, 12, 12, 7)
+
+#ifndef __ASSEMBLY__
+
+#include <libcflat.h>
+#include <asm/barrier.h>
+
+#define __stringify xstr
+
+/*
+ * Low-level accessors
+ *
+ * These system registers are 32 bits, but we make sure that the compiler
+ * sets the GP register's most significant bits to 0 with an explicit cast.
+ */
+
+static inline void gicv3_write_pmr(u32 val)
+{
+ asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
+}
+
+static inline void gicv3_write_grpen1(u32 val)
+{
+ asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
+ isb();
+}
+
+#define gicv3_read_typer(c) readq(c)
+
+#endif /* __ASSEMBLY__ */
+#endif /* _ASMARM64_ARCH_GICV3_H_ */
diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
new file mode 100644
index 000000000000..8ee5d4d9c181
--- /dev/null
+++ b/lib/arm64/asm/gic-v3.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v3.h"
diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
new file mode 100644
index 000000000000..544a46cb8cc5
--- /dev/null
+++ b/lib/arm64/asm/sysreg.h
@@ -0,0 +1,44 @@
+/*
+ * Ripped off from arch/arm64/include/asm/sysreg.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM64_SYSREG_H_
+#define _ASMARM64_SYSREG_H_
+
+#define sys_reg(op0, op1, crn, crm, op2) \
+ ((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
+
+#ifdef __ASSEMBLY__
+ .irp num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
+ .equ .L__reg_num_x\num, \num
+ .endr
+ .equ .L__reg_num_xzr, 31
+
+ .macro mrs_s, rt, sreg
+ .inst 0xd5200000|(\sreg)|(.L__reg_num_\rt)
+ .endm
+
+ .macro msr_s, sreg, rt
+ .inst 0xd5000000|(\sreg)|(.L__reg_num_\rt)
+ .endm
+#else
+asm(
+" .irp num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
+" .equ .L__reg_num_x\\num, \\num\n"
+" .endr\n"
+" .equ .L__reg_num_xzr, 31\n"
+"\n"
+" .macro mrs_s, rt, sreg\n"
+" .inst 0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
+" .endm\n"
+"\n"
+" .macro msr_s, sreg, rt\n"
+" .inst 0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
+" .endm\n"
+);
+#endif
+
+#endif /* _ASMARM64_SYSREG_H_ */
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 10/11] arm/arm64: gicv3: add an IPI test
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6: move most gicv2/gicv3 wrappers to common code [Alex]
v5:
- fix copy+paste error in gicv3_write_eoir [drew]
- use modern register names [Andre]
v4:
- heavily comment gicv3_ipi_send_tlist() [Eric]
- changes needed for gicv2 iar/irqstat fix to other patch
v2:
- use IRM for gicv3 broadcast
---
arm/gic.c | 97 +++++++++++++++++++++++++-----
arm/unittests.cfg | 6 ++
lib/arm/asm/arch_gicv3.h | 23 +++++++
lib/arm/asm/gic-v3.h | 10 +++-
lib/arm/asm/gic.h | 60 +++++++++++++++++++
lib/arm/gic.c | 145 ++++++++++++++++++++++++++++++++++++++++++---
lib/arm64/asm/arch_gicv3.h | 22 +++++++
7 files changed, 338 insertions(+), 25 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index b42c2b1ca1e1..d954a3775c26 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -3,6 +3,8 @@
*
* GICv2
* + test sending/receiving IPIs
+ * GICv3
+ * + test sending/receiving IPIs
*
* Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
*
@@ -16,7 +18,15 @@
#include <asm/barrier.h>
#include <asm/io.h>
-static int gic_version;
+struct gic {
+ struct {
+ void (*send_self)(void);
+ void (*send_tlist)(cpumask_t *mask, int irq);
+ void (*send_broadcast)(void);
+ } ipi;
+};
+
+static struct gic *gic;
static int acked[NR_CPUS], spurious[NR_CPUS];
static cpumask_t ready;
@@ -83,11 +93,11 @@ static void check_spurious(void)
static void ipi_handler(struct pt_regs *regs __unused)
{
- u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
- u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
+ u32 irqstat = gic_read_iar();
+ u32 irqnr = gic_iar_irqnr(irqstat);
if (irqnr != GICC_INT_SPURIOUS) {
- writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+ gic_write_eoir(irqstat);
smp_rmb(); /* pairs with wmb in ipi_test functions */
++acked[smp_processor_id()];
smp_wmb(); /* pairs with rmb in check_acked */
@@ -97,6 +107,38 @@ static void ipi_handler(struct pt_regs *regs __unused)
}
}
+static void gicv2_ipi_send_self(void)
+{
+ writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+}
+
+static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
+{
+ u8 tlist = (u8)cpumask_bits(mask)[0];
+
+ writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
+}
+
+static void gicv2_ipi_send_broadcast(void)
+{
+ writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+}
+
+static void gicv3_ipi_send_self(void)
+{
+ cpumask_t mask;
+
+ cpumask_clear(&mask);
+ cpumask_set_cpu(smp_processor_id(), &mask);
+ gicv3_ipi_send_tlist(&mask, 0);
+}
+
+static void gicv3_ipi_send_broadcast(void)
+{
+ gicv3_write_sgi1r(1ULL << 40);
+ isb();
+}
+
static void ipi_test_self(void)
{
cpumask_t mask;
@@ -106,7 +148,7 @@ static void ipi_test_self(void)
smp_wmb();
cpumask_clear(&mask);
cpumask_set_cpu(0, &mask);
- writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+ gic->ipi.send_self();
check_acked(&mask);
report_prefix_pop();
}
@@ -114,14 +156,15 @@ static void ipi_test_self(void)
static void ipi_test_smp(void)
{
cpumask_t mask;
- unsigned long tlist;
+ int i;
report_prefix_push("target-list");
memset(acked, 0, sizeof(acked));
smp_wmb();
- tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
- cpumask_bits(&mask)[0] = tlist;
- writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
+ cpumask_copy(&mask, &cpu_present_mask);
+ for (i = 0; i < nr_cpus; i += 2)
+ cpumask_clear_cpu(i, &mask);
+ gic->ipi.send_tlist(&mask, 0);
check_acked(&mask);
report_prefix_pop();
@@ -130,14 +173,14 @@ static void ipi_test_smp(void)
smp_wmb();
cpumask_copy(&mask, &cpu_present_mask);
cpumask_clear_cpu(0, &mask);
- writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+ gic->ipi.send_broadcast();
check_acked(&mask);
report_prefix_pop();
}
static void ipi_enable(void)
{
- gicv2_enable_defaults();
+ gic_enable_defaults();
#ifdef __arm__
install_exception_handler(EXCPTN_IRQ, ipi_handler);
#else
@@ -154,18 +197,42 @@ static void ipi_recv(void)
wfi();
}
+static struct gic gicv2 = {
+ .ipi = {
+ .send_self = gicv2_ipi_send_self,
+ .send_tlist = gicv2_ipi_send_tlist,
+ .send_broadcast = gicv2_ipi_send_broadcast,
+ },
+};
+
+static struct gic gicv3 = {
+ .ipi = {
+ .send_self = gicv3_ipi_send_self,
+ .send_tlist = gicv3_ipi_send_tlist,
+ .send_broadcast = gicv3_ipi_send_broadcast,
+ },
+};
+
int main(int argc, char **argv)
{
char pfx[8];
int cpu;
- gic_version = gic_init();
- if (!gic_version)
- report_abort("No gic present!");
+ if (!gic_init())
+ report_abort("No supported gic present!");
- snprintf(pfx, sizeof(pfx), "gicv%d", gic_version);
+ snprintf(pfx, sizeof(pfx), "gicv%d", gic_version());
report_prefix_push(pfx);
+ switch (gic_version()) {
+ case 2:
+ gic = &gicv2;
+ break;
+ case 3:
+ gic = &gicv3;
+ break;
+ }
+
if (argc < 2) {
report_prefix_push("ipi");
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index e631c35e2bbb..c7392c747f98 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -66,3 +66,9 @@ file = gic.flat
smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
extra_params = -machine gic-version=2 -append 'ipi'
groups = gic
+
+[gicv3-ipi]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'ipi'
+groups = gic
diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
index 276577452a14..b47cd2e0090b 100644
--- a/lib/arm/asm/arch_gicv3.h
+++ b/lib/arm/asm/arch_gicv3.h
@@ -16,10 +16,28 @@
#define __stringify xstr
#define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
+#define __ACCESS_CP15_64(Op1, CRm) p15, Op1, %Q0, %R0, CRm
+#define ICC_EOIR1 __ACCESS_CP15(c12, 0, c12, 1)
+#define ICC_IAR1 __ACCESS_CP15(c12, 0, c12, 0)
+#define ICC_SGI1R __ACCESS_CP15_64(0, c12)
#define ICC_PMR __ACCESS_CP15(c4, 0, c6, 0)
#define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
+static inline void gicv3_write_eoir(u32 irq)
+{
+ asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
+ isb();
+}
+
+static inline u32 gicv3_read_iar(void)
+{
+ u32 irqstat;
+ asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
+ dsb(sy);
+ return irqstat;
+}
+
static inline void gicv3_write_pmr(u32 val)
{
asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
@@ -31,6 +49,11 @@ static inline void gicv3_write_grpen1(u32 val)
isb();
}
+static inline void gicv3_write_sgi1r(u64 val)
+{
+ asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
+}
+
/*
* We may access GICR_TYPER and GITS_TYPER by reading both the TYPER
* offset and the following offset (+ 4) and then combining them to
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index 73ade4681d21..43f9ffce56de 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -33,12 +33,19 @@
#define GICR_ISENABLER0 GICD_ISENABLER
#define GICR_IPRIORITYR0 GICD_IPRIORITYR
+#define ICC_SGI1R_AFFINITY_1_SHIFT 16
+#define ICC_SGI1R_AFFINITY_2_SHIFT 32
+#define ICC_SGI1R_AFFINITY_3_SHIFT 48
+#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
+ (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
+
#include <asm/arch_gicv3.h>
#ifndef __ASSEMBLY__
#include <asm/setup.h>
-#include <asm/smp.h>
#include <asm/processor.h>
+#include <asm/cpumask.h>
+#include <asm/smp.h>
#include <asm/io.h>
struct gicv3_data {
@@ -55,6 +62,7 @@ extern struct gicv3_data gicv3_data;
extern int gicv3_init(void);
extern void gicv3_enable_defaults(void);
extern void gicv3_set_redist_base(size_t stride);
+extern void gicv3_ipi_send_tlist(cpumask_t *mask, int irq);
static inline void gicv3_do_wait_for_rwp(void *base)
{
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index 21511997f2a9..c2267b6b3937 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -42,5 +42,65 @@
*/
extern int gic_init(void);
+/*
+ * gic_common_ops collects some functions that we provide unit
+ * tests that don't care which gic version they're using.
+ */
+struct gic_common_ops {
+ int gic_version;
+ u32 (*read_iar)(void);
+ u32 (*iar_irqnr)(u32 iar);
+ void (*write_eoir)(u32 irqstat);
+ void (*ipi_send)(int cpu, int irq);
+};
+
+extern struct gic_common_ops *gic_common_ops;
+
+static inline int gic_version(void)
+{
+ assert(gic_common_ops);
+ return gic_common_ops->gic_version;
+}
+
+static inline u32 gic_read_iar(void)
+{
+ assert(gic_common_ops && gic_common_ops->read_iar);
+ return gic_common_ops->read_iar();
+}
+
+static inline u32 gic_iar_irqnr(u32 iar)
+{
+ assert(gic_common_ops && gic_common_ops->iar_irqnr);
+ return gic_common_ops->iar_irqnr(iar);
+}
+
+static inline void gic_write_eoir(u32 irqstat)
+{
+ assert(gic_common_ops && gic_common_ops->write_eoir);
+ gic_common_ops->write_eoir(irqstat);
+}
+
+static inline void gic_ipi_send(int cpu, int irq)
+{
+ assert(gic_common_ops && gic_common_ops->ipi_send);
+ gic_common_ops->ipi_send(cpu, irq);
+}
+
+static inline void gic_enable_defaults(void)
+{
+ switch (gic_version()) {
+ case 2:
+ gicv2_enable_defaults();
+ break;
+ case 3:
+ gicv3_enable_defaults();
+ break;
+ default:
+ printf("%s: Unknown gic version %d\n", __func__,
+ gic_version());
+ abort();
+ }
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASMARM_GIC_H_ */
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index d703ad96a37e..4f67363e073b 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -56,15 +56,6 @@ int gicv3_init(void)
&gicv3_data.redist_base[0]);
}
-int gic_init(void)
-{
- if (gicv2_init())
- return 2;
- else if (gicv3_init())
- return 3;
- return 0;
-}
-
void gicv2_enable_defaults(void)
{
void *dist = gicv2_dist_base();
@@ -85,6 +76,28 @@ void gicv2_enable_defaults(void)
writel(GICC_ENABLE, cpu_base + GICC_CTLR);
}
+static u32 gicv2_read_iar(void)
+{
+ return readl(gicv2_cpu_base() + GICC_IAR);
+}
+
+static u32 gicv2_iar_irqnr(u32 iar)
+{
+ return iar & GICC_IAR_INT_ID_MASK;
+}
+
+static void gicv2_write_eoir(u32 irqstat)
+{
+ writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+}
+
+static void gicv2_ipi_send(int cpu, int irq)
+{
+ assert(cpu < 8);
+ assert(irq < 16);
+ writel(1 << (cpu + 16) | irq, gicv2_dist_base() + GICD_SGIR);
+}
+
void gicv3_set_redist_base(size_t stride)
{
u32 aff = mpidr_compress(get_mpidr());
@@ -138,3 +151,117 @@ void gicv3_enable_defaults(void)
gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
gicv3_write_grpen1(1);
}
+
+static u32 gicv3_iar_irqnr(u32 iar)
+{
+ return iar;
+}
+
+void gicv3_ipi_send_tlist(cpumask_t *mask, int irq)
+{
+ u16 tlist;
+ int cpu;
+
+ assert(irq < 16);
+
+ /*
+ * For each cpu in the mask collect its peers, which are also in
+ * the mask, in order to form target lists.
+ */
+ for_each_cpu(cpu, mask) {
+ u64 mpidr = cpus[cpu], sgi1r;
+ u64 cluster_id;
+
+ /*
+ * GICv3 can send IPIs to up 16 peer cpus with a single
+ * write to ICC_SGI1R_EL1 (using the target list). Peers
+ * are cpus that have nearly identical MPIDRs, the only
+ * difference being Aff0. The matching upper affinity
+ * levels form the cluster ID.
+ */
+ cluster_id = mpidr & ~0xffUL;
+ tlist = 0;
+
+ /*
+ * Sort of open code for_each_cpu in order to have a
+ * nested for_each_cpu loop.
+ */
+ while (cpu < nr_cpus) {
+ if ((mpidr & 0xff) >= 16) {
+ printf("cpu%d MPIDR:aff0 is %d (>= 16)!\n",
+ cpu, (int)(mpidr & 0xff));
+ break;
+ }
+
+ tlist |= 1 << (mpidr & 0xf);
+
+ cpu = cpumask_next(cpu, mask);
+ if (cpu >= nr_cpus)
+ break;
+
+ mpidr = cpus[cpu];
+
+ if (cluster_id != (mpidr & ~0xffUL)) {
+ /*
+ * The next cpu isn't in our cluster. Roll
+ * back the cpu index allowing the outer
+ * for_each_cpu to find it again with
+ * cpumask_next
+ */
+ --cpu;
+ break;
+ }
+ }
+
+ /* Send the IPIs for the target list of this cluster */
+ sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3) |
+ MPIDR_TO_SGI_AFFINITY(cluster_id, 2) |
+ irq << 24 |
+ MPIDR_TO_SGI_AFFINITY(cluster_id, 1) |
+ tlist);
+
+ gicv3_write_sgi1r(sgi1r);
+ }
+
+ /* Force the above writes to ICC_SGI1R_EL1 to be executed */
+ isb();
+}
+
+static void gicv3_ipi_send(int cpu, int irq)
+{
+ cpumask_t mask;
+
+ cpumask_clear(&mask);
+ cpumask_set_cpu(cpu, &mask);
+ gicv3_ipi_send_tlist(&mask, irq);
+}
+
+static struct gic_common_ops gicv2_common_ops = {
+ .gic_version = 2,
+ .read_iar = gicv2_read_iar,
+ .iar_irqnr = gicv2_iar_irqnr,
+ .write_eoir = gicv2_write_eoir,
+ .ipi_send = gicv2_ipi_send,
+};
+
+static struct gic_common_ops gicv3_common_ops = {
+ .gic_version = 3,
+ .read_iar = gicv3_read_iar,
+ .iar_irqnr = gicv3_iar_irqnr,
+ .write_eoir = gicv3_write_eoir,
+ .ipi_send = gicv3_ipi_send,
+};
+
+struct gic_common_ops *gic_common_ops;
+
+int gic_init(void)
+{
+ if (gicv2_init()) {
+ gic_common_ops = &gicv2_common_ops;
+ return 2;
+ } else if (gicv3_init()) {
+ gic_common_ops = &gicv3_common_ops;
+ return 3;
+ }
+ return 0;
+}
diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
index 6d353567f56a..874775828016 100644
--- a/lib/arm64/asm/arch_gicv3.h
+++ b/lib/arm64/asm/arch_gicv3.h
@@ -10,6 +10,9 @@
#include <asm/sysreg.h>
+#define ICC_EOIR1_EL1 sys_reg(3, 0, 12, 12, 1)
+#define ICC_IAR1_EL1 sys_reg(3, 0, 12, 12, 0)
+#define ICC_SGI1R_EL1 sys_reg(3, 0, 12, 11, 5)
#define ICC_PMR_EL1 sys_reg(3, 0, 4, 6, 0)
#define ICC_GRPEN1_EL1 sys_reg(3, 0, 12, 12, 7)
@@ -27,6 +30,20 @@
* sets the GP register's most significant bits to 0 with an explicit cast.
*/
+static inline void gicv3_write_eoir(u32 irq)
+{
+ asm volatile("msr_s " __stringify(ICC_EOIR1_EL1) ", %0" : : "r" ((u64)irq));
+ isb();
+}
+
+static inline u32 gicv3_read_iar(void)
+{
+ u64 irqstat;
+ asm volatile("mrs_s %0, " __stringify(ICC_IAR1_EL1) : "=r" (irqstat));
+ dsb(sy);
+ return (u64)irqstat;
+}
+
static inline void gicv3_write_pmr(u32 val)
{
asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
@@ -38,6 +55,11 @@ static inline void gicv3_write_grpen1(u32 val)
isb();
}
+static inline void gicv3_write_sgi1r(u64 val)
+{
+ asm volatile("msr_s " __stringify(ICC_SGI1R_EL1) ", %0" : : "r" (val));
+}
+
#define gicv3_read_typer(c) readq(c)
#endif /* __ASSEMBLY__ */
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6: move most gicv2/gicv3 wrappers to common code [Alex]
v5:
- fix copy+paste error in gicv3_write_eoir [drew]
- use modern register names [Andre]
v4:
- heavily comment gicv3_ipi_send_tlist() [Eric]
- changes needed for gicv2 iar/irqstat fix to other patch
v2:
- use IRM for gicv3 broadcast
---
arm/gic.c | 97 +++++++++++++++++++++++++-----
arm/unittests.cfg | 6 ++
lib/arm/asm/arch_gicv3.h | 23 +++++++
lib/arm/asm/gic-v3.h | 10 +++-
lib/arm/asm/gic.h | 60 +++++++++++++++++++
lib/arm/gic.c | 145 ++++++++++++++++++++++++++++++++++++++++++---
lib/arm64/asm/arch_gicv3.h | 22 +++++++
7 files changed, 338 insertions(+), 25 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index b42c2b1ca1e1..d954a3775c26 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -3,6 +3,8 @@
*
* GICv2
* + test sending/receiving IPIs
+ * GICv3
+ * + test sending/receiving IPIs
*
* Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
*
@@ -16,7 +18,15 @@
#include <asm/barrier.h>
#include <asm/io.h>
-static int gic_version;
+struct gic {
+ struct {
+ void (*send_self)(void);
+ void (*send_tlist)(cpumask_t *mask, int irq);
+ void (*send_broadcast)(void);
+ } ipi;
+};
+
+static struct gic *gic;
static int acked[NR_CPUS], spurious[NR_CPUS];
static cpumask_t ready;
@@ -83,11 +93,11 @@ static void check_spurious(void)
static void ipi_handler(struct pt_regs *regs __unused)
{
- u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
- u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
+ u32 irqstat = gic_read_iar();
+ u32 irqnr = gic_iar_irqnr(irqstat);
if (irqnr != GICC_INT_SPURIOUS) {
- writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+ gic_write_eoir(irqstat);
smp_rmb(); /* pairs with wmb in ipi_test functions */
++acked[smp_processor_id()];
smp_wmb(); /* pairs with rmb in check_acked */
@@ -97,6 +107,38 @@ static void ipi_handler(struct pt_regs *regs __unused)
}
}
+static void gicv2_ipi_send_self(void)
+{
+ writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+}
+
+static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
+{
+ u8 tlist = (u8)cpumask_bits(mask)[0];
+
+ writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
+}
+
+static void gicv2_ipi_send_broadcast(void)
+{
+ writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+}
+
+static void gicv3_ipi_send_self(void)
+{
+ cpumask_t mask;
+
+ cpumask_clear(&mask);
+ cpumask_set_cpu(smp_processor_id(), &mask);
+ gicv3_ipi_send_tlist(&mask, 0);
+}
+
+static void gicv3_ipi_send_broadcast(void)
+{
+ gicv3_write_sgi1r(1ULL << 40);
+ isb();
+}
+
static void ipi_test_self(void)
{
cpumask_t mask;
@@ -106,7 +148,7 @@ static void ipi_test_self(void)
smp_wmb();
cpumask_clear(&mask);
cpumask_set_cpu(0, &mask);
- writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+ gic->ipi.send_self();
check_acked(&mask);
report_prefix_pop();
}
@@ -114,14 +156,15 @@ static void ipi_test_self(void)
static void ipi_test_smp(void)
{
cpumask_t mask;
- unsigned long tlist;
+ int i;
report_prefix_push("target-list");
memset(acked, 0, sizeof(acked));
smp_wmb();
- tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
- cpumask_bits(&mask)[0] = tlist;
- writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
+ cpumask_copy(&mask, &cpu_present_mask);
+ for (i = 0; i < nr_cpus; i += 2)
+ cpumask_clear_cpu(i, &mask);
+ gic->ipi.send_tlist(&mask, 0);
check_acked(&mask);
report_prefix_pop();
@@ -130,14 +173,14 @@ static void ipi_test_smp(void)
smp_wmb();
cpumask_copy(&mask, &cpu_present_mask);
cpumask_clear_cpu(0, &mask);
- writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+ gic->ipi.send_broadcast();
check_acked(&mask);
report_prefix_pop();
}
static void ipi_enable(void)
{
- gicv2_enable_defaults();
+ gic_enable_defaults();
#ifdef __arm__
install_exception_handler(EXCPTN_IRQ, ipi_handler);
#else
@@ -154,18 +197,42 @@ static void ipi_recv(void)
wfi();
}
+static struct gic gicv2 = {
+ .ipi = {
+ .send_self = gicv2_ipi_send_self,
+ .send_tlist = gicv2_ipi_send_tlist,
+ .send_broadcast = gicv2_ipi_send_broadcast,
+ },
+};
+
+static struct gic gicv3 = {
+ .ipi = {
+ .send_self = gicv3_ipi_send_self,
+ .send_tlist = gicv3_ipi_send_tlist,
+ .send_broadcast = gicv3_ipi_send_broadcast,
+ },
+};
+
int main(int argc, char **argv)
{
char pfx[8];
int cpu;
- gic_version = gic_init();
- if (!gic_version)
- report_abort("No gic present!");
+ if (!gic_init())
+ report_abort("No supported gic present!");
- snprintf(pfx, sizeof(pfx), "gicv%d", gic_version);
+ snprintf(pfx, sizeof(pfx), "gicv%d", gic_version());
report_prefix_push(pfx);
+ switch (gic_version()) {
+ case 2:
+ gic = &gicv2;
+ break;
+ case 3:
+ gic = &gicv3;
+ break;
+ }
+
if (argc < 2) {
report_prefix_push("ipi");
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index e631c35e2bbb..c7392c747f98 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -66,3 +66,9 @@ file = gic.flat
smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
extra_params = -machine gic-version=2 -append 'ipi'
groups = gic
+
+[gicv3-ipi]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'ipi'
+groups = gic
diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
index 276577452a14..b47cd2e0090b 100644
--- a/lib/arm/asm/arch_gicv3.h
+++ b/lib/arm/asm/arch_gicv3.h
@@ -16,10 +16,28 @@
#define __stringify xstr
#define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
+#define __ACCESS_CP15_64(Op1, CRm) p15, Op1, %Q0, %R0, CRm
+#define ICC_EOIR1 __ACCESS_CP15(c12, 0, c12, 1)
+#define ICC_IAR1 __ACCESS_CP15(c12, 0, c12, 0)
+#define ICC_SGI1R __ACCESS_CP15_64(0, c12)
#define ICC_PMR __ACCESS_CP15(c4, 0, c6, 0)
#define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
+static inline void gicv3_write_eoir(u32 irq)
+{
+ asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
+ isb();
+}
+
+static inline u32 gicv3_read_iar(void)
+{
+ u32 irqstat;
+ asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
+ dsb(sy);
+ return irqstat;
+}
+
static inline void gicv3_write_pmr(u32 val)
{
asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
@@ -31,6 +49,11 @@ static inline void gicv3_write_grpen1(u32 val)
isb();
}
+static inline void gicv3_write_sgi1r(u64 val)
+{
+ asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
+}
+
/*
* We may access GICR_TYPER and GITS_TYPER by reading both the TYPER
* offset and the following offset (+ 4) and then combining them to
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index 73ade4681d21..43f9ffce56de 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -33,12 +33,19 @@
#define GICR_ISENABLER0 GICD_ISENABLER
#define GICR_IPRIORITYR0 GICD_IPRIORITYR
+#define ICC_SGI1R_AFFINITY_1_SHIFT 16
+#define ICC_SGI1R_AFFINITY_2_SHIFT 32
+#define ICC_SGI1R_AFFINITY_3_SHIFT 48
+#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
+ (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
+
#include <asm/arch_gicv3.h>
#ifndef __ASSEMBLY__
#include <asm/setup.h>
-#include <asm/smp.h>
#include <asm/processor.h>
+#include <asm/cpumask.h>
+#include <asm/smp.h>
#include <asm/io.h>
struct gicv3_data {
@@ -55,6 +62,7 @@ extern struct gicv3_data gicv3_data;
extern int gicv3_init(void);
extern void gicv3_enable_defaults(void);
extern void gicv3_set_redist_base(size_t stride);
+extern void gicv3_ipi_send_tlist(cpumask_t *mask, int irq);
static inline void gicv3_do_wait_for_rwp(void *base)
{
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index 21511997f2a9..c2267b6b3937 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -42,5 +42,65 @@
*/
extern int gic_init(void);
+/*
+ * gic_common_ops collects some functions that we provide unit
+ * tests that don't care which gic version they're using.
+ */
+struct gic_common_ops {
+ int gic_version;
+ u32 (*read_iar)(void);
+ u32 (*iar_irqnr)(u32 iar);
+ void (*write_eoir)(u32 irqstat);
+ void (*ipi_send)(int cpu, int irq);
+};
+
+extern struct gic_common_ops *gic_common_ops;
+
+static inline int gic_version(void)
+{
+ assert(gic_common_ops);
+ return gic_common_ops->gic_version;
+}
+
+static inline u32 gic_read_iar(void)
+{
+ assert(gic_common_ops && gic_common_ops->read_iar);
+ return gic_common_ops->read_iar();
+}
+
+static inline u32 gic_iar_irqnr(u32 iar)
+{
+ assert(gic_common_ops && gic_common_ops->iar_irqnr);
+ return gic_common_ops->iar_irqnr(iar);
+}
+
+static inline void gic_write_eoir(u32 irqstat)
+{
+ assert(gic_common_ops && gic_common_ops->write_eoir);
+ gic_common_ops->write_eoir(irqstat);
+}
+
+static inline void gic_ipi_send(int cpu, int irq)
+{
+ assert(gic_common_ops && gic_common_ops->ipi_send);
+ gic_common_ops->ipi_send(cpu, irq);
+}
+
+static inline void gic_enable_defaults(void)
+{
+ switch (gic_version()) {
+ case 2:
+ gicv2_enable_defaults();
+ break;
+ case 3:
+ gicv3_enable_defaults();
+ break;
+ default:
+ printf("%s: Unknown gic version %d\n", __func__,
+ gic_version());
+ abort();
+ }
+}
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASMARM_GIC_H_ */
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index d703ad96a37e..4f67363e073b 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -56,15 +56,6 @@ int gicv3_init(void)
&gicv3_data.redist_base[0]);
}
-int gic_init(void)
-{
- if (gicv2_init())
- return 2;
- else if (gicv3_init())
- return 3;
- return 0;
-}
-
void gicv2_enable_defaults(void)
{
void *dist = gicv2_dist_base();
@@ -85,6 +76,28 @@ void gicv2_enable_defaults(void)
writel(GICC_ENABLE, cpu_base + GICC_CTLR);
}
+static u32 gicv2_read_iar(void)
+{
+ return readl(gicv2_cpu_base() + GICC_IAR);
+}
+
+static u32 gicv2_iar_irqnr(u32 iar)
+{
+ return iar & GICC_IAR_INT_ID_MASK;
+}
+
+static void gicv2_write_eoir(u32 irqstat)
+{
+ writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+}
+
+static void gicv2_ipi_send(int cpu, int irq)
+{
+ assert(cpu < 8);
+ assert(irq < 16);
+ writel(1 << (cpu + 16) | irq, gicv2_dist_base() + GICD_SGIR);
+}
+
void gicv3_set_redist_base(size_t stride)
{
u32 aff = mpidr_compress(get_mpidr());
@@ -138,3 +151,117 @@ void gicv3_enable_defaults(void)
gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
gicv3_write_grpen1(1);
}
+
+static u32 gicv3_iar_irqnr(u32 iar)
+{
+ return iar;
+}
+
+void gicv3_ipi_send_tlist(cpumask_t *mask, int irq)
+{
+ u16 tlist;
+ int cpu;
+
+ assert(irq < 16);
+
+ /*
+ * For each cpu in the mask collect its peers, which are also in
+ * the mask, in order to form target lists.
+ */
+ for_each_cpu(cpu, mask) {
+ u64 mpidr = cpus[cpu], sgi1r;
+ u64 cluster_id;
+
+ /*
+ * GICv3 can send IPIs to up 16 peer cpus with a single
+ * write to ICC_SGI1R_EL1 (using the target list). Peers
+ * are cpus that have nearly identical MPIDRs, the only
+ * difference being Aff0. The matching upper affinity
+ * levels form the cluster ID.
+ */
+ cluster_id = mpidr & ~0xffUL;
+ tlist = 0;
+
+ /*
+ * Sort of open code for_each_cpu in order to have a
+ * nested for_each_cpu loop.
+ */
+ while (cpu < nr_cpus) {
+ if ((mpidr & 0xff) >= 16) {
+ printf("cpu%d MPIDR:aff0 is %d (>= 16)!\n",
+ cpu, (int)(mpidr & 0xff));
+ break;
+ }
+
+ tlist |= 1 << (mpidr & 0xf);
+
+ cpu = cpumask_next(cpu, mask);
+ if (cpu >= nr_cpus)
+ break;
+
+ mpidr = cpus[cpu];
+
+ if (cluster_id != (mpidr & ~0xffUL)) {
+ /*
+ * The next cpu isn't in our cluster. Roll
+ * back the cpu index allowing the outer
+ * for_each_cpu to find it again with
+ * cpumask_next
+ */
+ --cpu;
+ break;
+ }
+ }
+
+ /* Send the IPIs for the target list of this cluster */
+ sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3) |
+ MPIDR_TO_SGI_AFFINITY(cluster_id, 2) |
+ irq << 24 |
+ MPIDR_TO_SGI_AFFINITY(cluster_id, 1) |
+ tlist);
+
+ gicv3_write_sgi1r(sgi1r);
+ }
+
+ /* Force the above writes to ICC_SGI1R_EL1 to be executed */
+ isb();
+}
+
+static void gicv3_ipi_send(int cpu, int irq)
+{
+ cpumask_t mask;
+
+ cpumask_clear(&mask);
+ cpumask_set_cpu(cpu, &mask);
+ gicv3_ipi_send_tlist(&mask, irq);
+}
+
+static struct gic_common_ops gicv2_common_ops = {
+ .gic_version = 2,
+ .read_iar = gicv2_read_iar,
+ .iar_irqnr = gicv2_iar_irqnr,
+ .write_eoir = gicv2_write_eoir,
+ .ipi_send = gicv2_ipi_send,
+};
+
+static struct gic_common_ops gicv3_common_ops = {
+ .gic_version = 3,
+ .read_iar = gicv3_read_iar,
+ .iar_irqnr = gicv3_iar_irqnr,
+ .write_eoir = gicv3_write_eoir,
+ .ipi_send = gicv3_ipi_send,
+};
+
+struct gic_common_ops *gic_common_ops;
+
+int gic_init(void)
+{
+ if (gicv2_init()) {
+ gic_common_ops = &gicv2_common_ops;
+ return 2;
+ } else if (gicv3_init()) {
+ gic_common_ops = &gicv3_common_ops;
+ return 3;
+ }
+ return 0;
+}
diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
index 6d353567f56a..874775828016 100644
--- a/lib/arm64/asm/arch_gicv3.h
+++ b/lib/arm64/asm/arch_gicv3.h
@@ -10,6 +10,9 @@
#include <asm/sysreg.h>
+#define ICC_EOIR1_EL1 sys_reg(3, 0, 12, 12, 1)
+#define ICC_IAR1_EL1 sys_reg(3, 0, 12, 12, 0)
+#define ICC_SGI1R_EL1 sys_reg(3, 0, 12, 11, 5)
#define ICC_PMR_EL1 sys_reg(3, 0, 4, 6, 0)
#define ICC_GRPEN1_EL1 sys_reg(3, 0, 12, 12, 7)
@@ -27,6 +30,20 @@
* sets the GP register's most significant bits to 0 with an explicit cast.
*/
+static inline void gicv3_write_eoir(u32 irq)
+{
+ asm volatile("msr_s " __stringify(ICC_EOIR1_EL1) ", %0" : : "r" ((u64)irq));
+ isb();
+}
+
+static inline u32 gicv3_read_iar(void)
+{
+ u64 irqstat;
+ asm volatile("mrs_s %0, " __stringify(ICC_IAR1_EL1) : "=r" (irqstat));
+ dsb(sy);
+ return (u64)irqstat;
+}
+
static inline void gicv3_write_pmr(u32 val)
{
asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
@@ -38,6 +55,11 @@ static inline void gicv3_write_grpen1(u32 val)
isb();
}
+static inline void gicv3_write_sgi1r(u64 val)
+{
+ asm volatile("msr_s " __stringify(ICC_SGI1R_EL1) ", %0" : : "r" (val));
+}
+
#define gicv3_read_typer(c) readq(c)
#endif /* __ASSEMBLY__ */
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-14 21:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini
Allow user to select who sends ipis and with which irq,
rather than just always sending irq=0 from cpu0.
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6:
- make sender/irq names more future-proof [drew]
- sanity check inputs [drew]
- introduce check_sender/irq and bad_sender/irq to more
cleanly do checks [drew]
- default sender and irq to 1, instead of still zero [drew]
v4: improve structure and make sure spurious checking is
done even when the sender isn't cpu0
v2: actually check that the irq received was the irq sent,
and (for gicv2) that the sender is the expected one.
---
arm/gic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 99 insertions(+), 25 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index d954a3775c26..638b8b140c96 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -11,6 +11,7 @@
* This work is licensed under the terms of the GNU LGPL, version 2.
*/
#include <libcflat.h>
+#include <util.h>
#include <asm/setup.h>
#include <asm/processor.h>
#include <asm/gic.h>
@@ -28,6 +29,8 @@ struct gic {
static struct gic *gic;
static int acked[NR_CPUS], spurious[NR_CPUS];
+static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
+static int cmdl_sender = 1, cmdl_irq = 1;
static cpumask_t ready;
static void nr_cpu_check(int nr)
@@ -43,10 +46,23 @@ static void wait_on_ready(void)
cpu_relax();
}
+static void stats_reset(void)
+{
+ int i;
+
+ for (i = 0; i < nr_cpus; ++i) {
+ acked[i] = 0;
+ bad_sender[i] = -1;
+ bad_irq[i] = -1;
+ }
+ smp_wmb();
+}
+
static void check_acked(cpumask_t *mask)
{
int missing = 0, extra = 0, unexpected = 0;
int nr_pass, cpu, i;
+ bool bad = false;
/* Wait up to 5s for all interrupts to be delivered */
for (i = 0; i < 50; ++i) {
@@ -56,9 +72,21 @@ static void check_acked(cpumask_t *mask)
smp_rmb();
nr_pass += cpumask_test_cpu(cpu, mask) ?
acked[cpu] == 1 : acked[cpu] == 0;
+
+ if (bad_sender[cpu] != -1) {
+ printf("cpu%d received IPI from wrong sender %d\n",
+ cpu, bad_sender[cpu]);
+ bad = true;
+ }
+
+ if (bad_irq[cpu] != -1) {
+ printf("cpu%d received wrong irq %d\n",
+ cpu, bad_irq[cpu]);
+ bad = true;
+ }
}
if (nr_pass == nr_cpus) {
- report("Completed in %d ms", true, ++i * 100);
+ report("Completed in %d ms", !bad, ++i * 100);
return;
}
}
@@ -91,6 +119,22 @@ static void check_spurious(void)
}
}
+static void check_ipi_sender(u32 irqstat)
+{
+ if (gic_version() == 2) {
+ int src = (irqstat >> 10) & 7;
+
+ if (src != cmdl_sender)
+ bad_sender[smp_processor_id()] = src;
+ }
+}
+
+static void check_irqnr(u32 irqnr)
+{
+ if (irqnr != (u32)cmdl_irq)
+ bad_irq[smp_processor_id()] = irqnr;
+}
+
static void ipi_handler(struct pt_regs *regs __unused)
{
u32 irqstat = gic_read_iar();
@@ -98,8 +142,10 @@ static void ipi_handler(struct pt_regs *regs __unused)
if (irqnr != GICC_INT_SPURIOUS) {
gic_write_eoir(irqstat);
- smp_rmb(); /* pairs with wmb in ipi_test functions */
+ smp_rmb(); /* pairs with wmb in stats_reset */
++acked[smp_processor_id()];
+ check_ipi_sender(irqstat);
+ check_irqnr(irqnr);
smp_wmb(); /* pairs with rmb in check_acked */
} else {
++spurious[smp_processor_id()];
@@ -109,19 +155,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
static void gicv2_ipi_send_self(void)
{
- writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+ writel(2 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
}
-static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
+static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq)
{
u8 tlist = (u8)cpumask_bits(mask)[0];
- writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
+ writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
}
static void gicv2_ipi_send_broadcast(void)
{
- writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+ writel(1 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
}
static void gicv3_ipi_send_self(void)
@@ -130,12 +176,12 @@ static void gicv3_ipi_send_self(void)
cpumask_clear(&mask);
cpumask_set_cpu(smp_processor_id(), &mask);
- gicv3_ipi_send_tlist(&mask, 0);
+ gicv3_ipi_send_tlist(&mask, cmdl_irq);
}
static void gicv3_ipi_send_broadcast(void)
{
- gicv3_write_sgi1r(1ULL << 40);
+ gicv3_write_sgi1r(1ULL << 40 | cmdl_irq << 24);
isb();
}
@@ -144,10 +190,9 @@ static void ipi_test_self(void)
cpumask_t mask;
report_prefix_push("self");
- memset(acked, 0, sizeof(acked));
- smp_wmb();
+ stats_reset();
cpumask_clear(&mask);
- cpumask_set_cpu(0, &mask);
+ cpumask_set_cpu(smp_processor_id(), &mask);
gic->ipi.send_self();
check_acked(&mask);
report_prefix_pop();
@@ -159,20 +204,18 @@ static void ipi_test_smp(void)
int i;
report_prefix_push("target-list");
- memset(acked, 0, sizeof(acked));
- smp_wmb();
+ stats_reset();
cpumask_copy(&mask, &cpu_present_mask);
- for (i = 0; i < nr_cpus; i += 2)
+ for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
cpumask_clear_cpu(i, &mask);
- gic->ipi.send_tlist(&mask, 0);
+ gic->ipi.send_tlist(&mask, cmdl_irq);
check_acked(&mask);
report_prefix_pop();
report_prefix_push("broadcast");
- memset(acked, 0, sizeof(acked));
- smp_wmb();
+ stats_reset();
cpumask_copy(&mask, &cpu_present_mask);
- cpumask_clear_cpu(0, &mask);
+ cpumask_clear_cpu(smp_processor_id(), &mask);
gic->ipi.send_broadcast();
check_acked(&mask);
report_prefix_pop();
@@ -189,6 +232,16 @@ static void ipi_enable(void)
local_irq_enable();
}
+static void ipi_send(void)
+{
+ ipi_enable();
+ wait_on_ready();
+ ipi_test_self();
+ ipi_test_smp();
+ check_spurious();
+ exit(report_summary());
+}
+
static void ipi_recv(void)
{
ipi_enable();
@@ -197,6 +250,14 @@ static void ipi_recv(void)
wfi();
}
+static void ipi_test(void)
+{
+ if (smp_processor_id() == cmdl_sender)
+ ipi_send();
+ else
+ ipi_recv();
+}
+
static struct gic gicv2 = {
.ipi = {
.send_self = gicv2_ipi_send_self,
@@ -242,21 +303,34 @@ int main(int argc, char **argv)
report_prefix_pop();
} else if (strcmp(argv[1], "ipi") == 0) {
+ int off, i = 1;
+ long val;
report_prefix_push(argv[1]);
nr_cpu_check(2);
+ while (--argc != 1) {
+ off = parse_keyval(argv[++i], &val);
+ if (off == -1)
+ continue;
+ argv[i][off] = '\0';
+ if (strcmp(argv[i], "sender") == 0) {
+ if (val >= nr_cpus)
+ report_abort("invalid sender %d, nr_cpus=%d", val, nr_cpus);
+ cmdl_sender = val;
+ } else if (strcmp(argv[i], "irq") == 0) {
+ if (val > 15)
+ report_abort("irq (SGI) must be < 16");
+ cmdl_irq = val;
+ }
+ }
+
for_each_present_cpu(cpu) {
if (cpu == 0)
continue;
- smp_boot_secondary(cpu, ipi_recv);
+ smp_boot_secondary(cpu, ipi_test);
}
- ipi_enable();
- wait_on_ready();
- ipi_test_self();
- ipi_test_smp();
- check_spurious();
- report_prefix_pop();
+ ipi_test();
} else {
report_abort("Unknown subtest '%s'", argv[1]);
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero
@ 2016-11-14 21:08 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-14 21:08 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, eric.auger, christoffer.dall
Allow user to select who sends ipis and with which irq,
rather than just always sending irq=0 from cpu0.
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v6:
- make sender/irq names more future-proof [drew]
- sanity check inputs [drew]
- introduce check_sender/irq and bad_sender/irq to more
cleanly do checks [drew]
- default sender and irq to 1, instead of still zero [drew]
v4: improve structure and make sure spurious checking is
done even when the sender isn't cpu0
v2: actually check that the irq received was the irq sent,
and (for gicv2) that the sender is the expected one.
---
arm/gic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 99 insertions(+), 25 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index d954a3775c26..638b8b140c96 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -11,6 +11,7 @@
* This work is licensed under the terms of the GNU LGPL, version 2.
*/
#include <libcflat.h>
+#include <util.h>
#include <asm/setup.h>
#include <asm/processor.h>
#include <asm/gic.h>
@@ -28,6 +29,8 @@ struct gic {
static struct gic *gic;
static int acked[NR_CPUS], spurious[NR_CPUS];
+static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
+static int cmdl_sender = 1, cmdl_irq = 1;
static cpumask_t ready;
static void nr_cpu_check(int nr)
@@ -43,10 +46,23 @@ static void wait_on_ready(void)
cpu_relax();
}
+static void stats_reset(void)
+{
+ int i;
+
+ for (i = 0; i < nr_cpus; ++i) {
+ acked[i] = 0;
+ bad_sender[i] = -1;
+ bad_irq[i] = -1;
+ }
+ smp_wmb();
+}
+
static void check_acked(cpumask_t *mask)
{
int missing = 0, extra = 0, unexpected = 0;
int nr_pass, cpu, i;
+ bool bad = false;
/* Wait up to 5s for all interrupts to be delivered */
for (i = 0; i < 50; ++i) {
@@ -56,9 +72,21 @@ static void check_acked(cpumask_t *mask)
smp_rmb();
nr_pass += cpumask_test_cpu(cpu, mask) ?
acked[cpu] == 1 : acked[cpu] == 0;
+
+ if (bad_sender[cpu] != -1) {
+ printf("cpu%d received IPI from wrong sender %d\n",
+ cpu, bad_sender[cpu]);
+ bad = true;
+ }
+
+ if (bad_irq[cpu] != -1) {
+ printf("cpu%d received wrong irq %d\n",
+ cpu, bad_irq[cpu]);
+ bad = true;
+ }
}
if (nr_pass == nr_cpus) {
- report("Completed in %d ms", true, ++i * 100);
+ report("Completed in %d ms", !bad, ++i * 100);
return;
}
}
@@ -91,6 +119,22 @@ static void check_spurious(void)
}
}
+static void check_ipi_sender(u32 irqstat)
+{
+ if (gic_version() == 2) {
+ int src = (irqstat >> 10) & 7;
+
+ if (src != cmdl_sender)
+ bad_sender[smp_processor_id()] = src;
+ }
+}
+
+static void check_irqnr(u32 irqnr)
+{
+ if (irqnr != (u32)cmdl_irq)
+ bad_irq[smp_processor_id()] = irqnr;
+}
+
static void ipi_handler(struct pt_regs *regs __unused)
{
u32 irqstat = gic_read_iar();
@@ -98,8 +142,10 @@ static void ipi_handler(struct pt_regs *regs __unused)
if (irqnr != GICC_INT_SPURIOUS) {
gic_write_eoir(irqstat);
- smp_rmb(); /* pairs with wmb in ipi_test functions */
+ smp_rmb(); /* pairs with wmb in stats_reset */
++acked[smp_processor_id()];
+ check_ipi_sender(irqstat);
+ check_irqnr(irqnr);
smp_wmb(); /* pairs with rmb in check_acked */
} else {
++spurious[smp_processor_id()];
@@ -109,19 +155,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
static void gicv2_ipi_send_self(void)
{
- writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+ writel(2 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
}
-static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
+static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq)
{
u8 tlist = (u8)cpumask_bits(mask)[0];
- writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
+ writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
}
static void gicv2_ipi_send_broadcast(void)
{
- writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+ writel(1 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
}
static void gicv3_ipi_send_self(void)
@@ -130,12 +176,12 @@ static void gicv3_ipi_send_self(void)
cpumask_clear(&mask);
cpumask_set_cpu(smp_processor_id(), &mask);
- gicv3_ipi_send_tlist(&mask, 0);
+ gicv3_ipi_send_tlist(&mask, cmdl_irq);
}
static void gicv3_ipi_send_broadcast(void)
{
- gicv3_write_sgi1r(1ULL << 40);
+ gicv3_write_sgi1r(1ULL << 40 | cmdl_irq << 24);
isb();
}
@@ -144,10 +190,9 @@ static void ipi_test_self(void)
cpumask_t mask;
report_prefix_push("self");
- memset(acked, 0, sizeof(acked));
- smp_wmb();
+ stats_reset();
cpumask_clear(&mask);
- cpumask_set_cpu(0, &mask);
+ cpumask_set_cpu(smp_processor_id(), &mask);
gic->ipi.send_self();
check_acked(&mask);
report_prefix_pop();
@@ -159,20 +204,18 @@ static void ipi_test_smp(void)
int i;
report_prefix_push("target-list");
- memset(acked, 0, sizeof(acked));
- smp_wmb();
+ stats_reset();
cpumask_copy(&mask, &cpu_present_mask);
- for (i = 0; i < nr_cpus; i += 2)
+ for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
cpumask_clear_cpu(i, &mask);
- gic->ipi.send_tlist(&mask, 0);
+ gic->ipi.send_tlist(&mask, cmdl_irq);
check_acked(&mask);
report_prefix_pop();
report_prefix_push("broadcast");
- memset(acked, 0, sizeof(acked));
- smp_wmb();
+ stats_reset();
cpumask_copy(&mask, &cpu_present_mask);
- cpumask_clear_cpu(0, &mask);
+ cpumask_clear_cpu(smp_processor_id(), &mask);
gic->ipi.send_broadcast();
check_acked(&mask);
report_prefix_pop();
@@ -189,6 +232,16 @@ static void ipi_enable(void)
local_irq_enable();
}
+static void ipi_send(void)
+{
+ ipi_enable();
+ wait_on_ready();
+ ipi_test_self();
+ ipi_test_smp();
+ check_spurious();
+ exit(report_summary());
+}
+
static void ipi_recv(void)
{
ipi_enable();
@@ -197,6 +250,14 @@ static void ipi_recv(void)
wfi();
}
+static void ipi_test(void)
+{
+ if (smp_processor_id() == cmdl_sender)
+ ipi_send();
+ else
+ ipi_recv();
+}
+
static struct gic gicv2 = {
.ipi = {
.send_self = gicv2_ipi_send_self,
@@ -242,21 +303,34 @@ int main(int argc, char **argv)
report_prefix_pop();
} else if (strcmp(argv[1], "ipi") == 0) {
+ int off, i = 1;
+ long val;
report_prefix_push(argv[1]);
nr_cpu_check(2);
+ while (--argc != 1) {
+ off = parse_keyval(argv[++i], &val);
+ if (off == -1)
+ continue;
+ argv[i][off] = '\0';
+ if (strcmp(argv[i], "sender") == 0) {
+ if (val >= nr_cpus)
+ report_abort("invalid sender %d, nr_cpus=%d", val, nr_cpus);
+ cmdl_sender = val;
+ } else if (strcmp(argv[i], "irq") == 0) {
+ if (val > 15)
+ report_abort("irq (SGI) must be < 16");
+ cmdl_irq = val;
+ }
+ }
+
for_each_present_cpu(cpu) {
if (cpu == 0)
continue;
- smp_boot_secondary(cpu, ipi_recv);
+ smp_boot_secondary(cpu, ipi_test);
}
- ipi_enable();
- wait_on_ready();
- ipi_test_self();
- ipi_test_smp();
- check_spurious();
- report_prefix_pop();
+ ipi_test();
} else {
report_abort("Unknown subtest '%s'", argv[1]);
--
2.7.4
^ permalink raw reply related [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
` (11 preceding siblings ...)
(?)
@ 2016-11-22 18:45 ` Andrew Jones
2016-11-23 9:55 ` Andre Przywara
2016-11-23 10:09 ` Alex Bennée
-1 siblings, 2 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-22 18:45 UTC (permalink / raw)
To: kvm, kvmarm, qemu-devel, qemu-arm
Cc: peter.maydell, marc.zyngier, andre.przywara, eric.auger,
pbonzini, alex.bennee, christoffer.dall
Andre, Alex, Eric, anybody,
Any more comments on this? If not, I'll send a pull request
to Radim and Paolo to finally get this merged.
Thanks,
drew
On Mon, Nov 14, 2016 at 10:08:28PM +0100, Andrew Jones wrote:
> v6:
> - rebased to latest master
> - several other changes thanks to Andre and Alex, changes in
> individual patch change logs
> - some code cleanups
>
> v5:
> - fix arm32/gicv3 compile [drew]
> - use modern register names [Andre]
> - one Andre r-b
>
> v4:
> - Eric's r-b's
> - Andre's suggestion to only take defines we need
> - several other changes listed in individual patches
>
> v3:
> - Rebased on latest master
> - Added Alex's r-b's
>
> v2:
> Rebased on latest master + my "populate argv[0]" series (will
> send a REPOST for that shortly. Additionally a few patches got
> fixes/features;
> 07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
> all interrupts as non-secure Group-1" in order to continue
> working over TCG, as the gicv3 code for TCG removed a hack
> it had there to make Linux happy.
> 08/10 added more output for when things fail (if they fail)
> 09/10 switched gicv3 broadcast implementation to using IRM. This
> found a bug in a recent (but not tip) kernel, which I was
> about to fix, but then I saw MarcZ beat me to it.
> 10/10 actually check that the input irq is the received irq
>
>
> Import defines, and steal enough helper functions, from Linux to
> enable programming of the gic (v2 and v3). Then use the framework
> to add an initial test (an ipi test; self, target-list, broadcast).
>
> It's my hope that this framework will be a suitable base on which
> more tests may be easily added, particularly because we have
> vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
> vgic-new and tcg gicv3 are merged now)
>
> To run it, along with other tests, just do
>
> ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
> make
> export QEMU=$PATH_TO_QEMU
> ./run_tests.sh
>
> To run it separately do, e.g.
>
> $QEMU -machine virt,accel=tcg -cpu cortex-a57 \
> -device virtio-serial-device \
> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
> -display none -serial stdio \
> -kernel arm/gic.flat \
> -smp 123 -machine gic-version=3 -append ipi
> ^^ note, we can go nuts with nr-cpus on TCG :-)
>
> Or, a KVM example using a different "sender" cpu and irq (other than zero)
>
> $QEMU -machine virt,accel=kvm -cpu host \
> -device virtio-serial-device \
> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
> -display none -serial stdio \
> -kernel arm/gic.flat \
> -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
>
>
> Patches:
> 01-05: fixes and functionality needed by the later gic patches
> 06-07: enable gicv2 and gicv2 IPI test
> 08-10: enable gicv3 and gicv3 IPI test
> 11: extend the IPI tests to take variable sender and irq
>
> Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
>
>
> Andrew Jones (10):
> lib: xstr: allow multiple args
> arm64: fix get_"sysreg32" and make MPIDR 64bit
> arm/arm64: smp: support more than 8 cpus
> arm/arm64: add some delay routines
> arm/arm64: irq enable/disable
> arm/arm64: add initial gicv2 support
> arm/arm64: gicv2: add an IPI test
> arm/arm64: add initial gicv3 support
> arm/arm64: gicv3: add an IPI test
> arm/arm64: gic: don't just use zero
>
> Peter Xu (1):
> libcflat: add IS_ALIGNED() macro, and page sizes
>
> arm/Makefile.common | 9 +-
> arm/gic.c | 340 +++++++++++++++++++++++++++++++++++++++++++++
> arm/run | 19 ++-
> arm/selftest.c | 5 +-
> arm/unittests.cfg | 14 ++
> lib/arm/asm/arch_gicv3.h | 70 ++++++++++
> lib/arm/asm/gic-v2.h | 36 +++++
> lib/arm/asm/gic-v3.h | 112 +++++++++++++++
> lib/arm/asm/gic.h | 106 ++++++++++++++
> lib/arm/asm/processor.h | 42 +++++-
> lib/arm/asm/setup.h | 4 +-
> lib/arm/gic.c | 267 +++++++++++++++++++++++++++++++++++
> lib/arm/processor.c | 15 ++
> lib/arm/setup.c | 10 ++
> lib/arm64/asm/arch_gicv3.h | 66 +++++++++
> lib/arm64/asm/gic-v2.h | 1 +
> lib/arm64/asm/gic-v3.h | 1 +
> lib/arm64/asm/gic.h | 1 +
> lib/arm64/asm/processor.h | 53 +++++--
> lib/arm64/asm/sysreg.h | 44 ++++++
> lib/arm64/processor.c | 15 ++
> lib/libcflat.h | 10 +-
> 22 files changed, 1212 insertions(+), 28 deletions(-)
> create mode 100644 arm/gic.c
> create mode 100644 lib/arm/asm/arch_gicv3.h
> create mode 100644 lib/arm/asm/gic-v2.h
> create mode 100644 lib/arm/asm/gic-v3.h
> create mode 100644 lib/arm/asm/gic.h
> create mode 100644 lib/arm/gic.c
> create mode 100644 lib/arm64/asm/arch_gicv3.h
> create mode 100644 lib/arm64/asm/gic-v2.h
> create mode 100644 lib/arm64/asm/gic-v3.h
> create mode 100644 lib/arm64/asm/gic.h
> create mode 100644 lib/arm64/asm/sysreg.h
>
> --
> 2.7.4
>
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework
2016-11-22 18:45 ` [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework Andrew Jones
@ 2016-11-23 9:55 ` Andre Przywara
2016-11-23 10:09 ` Alex Bennée
1 sibling, 0 replies; 52+ messages in thread
From: Andre Przywara @ 2016-11-23 9:55 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, pbonzini
Hi,
On 22/11/16 18:45, Andrew Jones wrote:
>
> Andre, Alex, Eric, anybody,
>
> Any more comments on this? If not, I'll send a pull request
> to Radim and Paolo to finally get this merged.
I didn't manage to look in detail at the IPI test, but I am OK with the
rest of the GIC framework.
So I'd say: go ahead.
Cheers,
Andre.
> On Mon, Nov 14, 2016 at 10:08:28PM +0100, Andrew Jones wrote:
>> v6:
>> - rebased to latest master
>> - several other changes thanks to Andre and Alex, changes in
>> individual patch change logs
>> - some code cleanups
>>
>> v5:
>> - fix arm32/gicv3 compile [drew]
>> - use modern register names [Andre]
>> - one Andre r-b
>>
>> v4:
>> - Eric's r-b's
>> - Andre's suggestion to only take defines we need
>> - several other changes listed in individual patches
>>
>> v3:
>> - Rebased on latest master
>> - Added Alex's r-b's
>>
>> v2:
>> Rebased on latest master + my "populate argv[0]" series (will
>> send a REPOST for that shortly. Additionally a few patches got
>> fixes/features;
>> 07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
>> all interrupts as non-secure Group-1" in order to continue
>> working over TCG, as the gicv3 code for TCG removed a hack
>> it had there to make Linux happy.
>> 08/10 added more output for when things fail (if they fail)
>> 09/10 switched gicv3 broadcast implementation to using IRM. This
>> found a bug in a recent (but not tip) kernel, which I was
>> about to fix, but then I saw MarcZ beat me to it.
>> 10/10 actually check that the input irq is the received irq
>>
>>
>> Import defines, and steal enough helper functions, from Linux to
>> enable programming of the gic (v2 and v3). Then use the framework
>> to add an initial test (an ipi test; self, target-list, broadcast).
>>
>> It's my hope that this framework will be a suitable base on which
>> more tests may be easily added, particularly because we have
>> vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
>> vgic-new and tcg gicv3 are merged now)
>>
>> To run it, along with other tests, just do
>>
>> ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
>> make
>> export QEMU=$PATH_TO_QEMU
>> ./run_tests.sh
>>
>> To run it separately do, e.g.
>>
>> $QEMU -machine virt,accel=tcg -cpu cortex-a57 \
>> -device virtio-serial-device \
>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>> -display none -serial stdio \
>> -kernel arm/gic.flat \
>> -smp 123 -machine gic-version=3 -append ipi
>> ^^ note, we can go nuts with nr-cpus on TCG :-)
>>
>> Or, a KVM example using a different "sender" cpu and irq (other than zero)
>>
>> $QEMU -machine virt,accel=kvm -cpu host \
>> -device virtio-serial-device \
>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>> -display none -serial stdio \
>> -kernel arm/gic.flat \
>> -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
>>
>>
>> Patches:
>> 01-05: fixes and functionality needed by the later gic patches
>> 06-07: enable gicv2 and gicv2 IPI test
>> 08-10: enable gicv3 and gicv3 IPI test
>> 11: extend the IPI tests to take variable sender and irq
>>
>> Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
>>
>>
>> Andrew Jones (10):
>> lib: xstr: allow multiple args
>> arm64: fix get_"sysreg32" and make MPIDR 64bit
>> arm/arm64: smp: support more than 8 cpus
>> arm/arm64: add some delay routines
>> arm/arm64: irq enable/disable
>> arm/arm64: add initial gicv2 support
>> arm/arm64: gicv2: add an IPI test
>> arm/arm64: add initial gicv3 support
>> arm/arm64: gicv3: add an IPI test
>> arm/arm64: gic: don't just use zero
>>
>> Peter Xu (1):
>> libcflat: add IS_ALIGNED() macro, and page sizes
>>
>> arm/Makefile.common | 9 +-
>> arm/gic.c | 340 +++++++++++++++++++++++++++++++++++++++++++++
>> arm/run | 19 ++-
>> arm/selftest.c | 5 +-
>> arm/unittests.cfg | 14 ++
>> lib/arm/asm/arch_gicv3.h | 70 ++++++++++
>> lib/arm/asm/gic-v2.h | 36 +++++
>> lib/arm/asm/gic-v3.h | 112 +++++++++++++++
>> lib/arm/asm/gic.h | 106 ++++++++++++++
>> lib/arm/asm/processor.h | 42 +++++-
>> lib/arm/asm/setup.h | 4 +-
>> lib/arm/gic.c | 267 +++++++++++++++++++++++++++++++++++
>> lib/arm/processor.c | 15 ++
>> lib/arm/setup.c | 10 ++
>> lib/arm64/asm/arch_gicv3.h | 66 +++++++++
>> lib/arm64/asm/gic-v2.h | 1 +
>> lib/arm64/asm/gic-v3.h | 1 +
>> lib/arm64/asm/gic.h | 1 +
>> lib/arm64/asm/processor.h | 53 +++++--
>> lib/arm64/asm/sysreg.h | 44 ++++++
>> lib/arm64/processor.c | 15 ++
>> lib/libcflat.h | 10 +-
>> 22 files changed, 1212 insertions(+), 28 deletions(-)
>> create mode 100644 arm/gic.c
>> create mode 100644 lib/arm/asm/arch_gicv3.h
>> create mode 100644 lib/arm/asm/gic-v2.h
>> create mode 100644 lib/arm/asm/gic-v3.h
>> create mode 100644 lib/arm/asm/gic.h
>> create mode 100644 lib/arm/gic.c
>> create mode 100644 lib/arm64/asm/arch_gicv3.h
>> create mode 100644 lib/arm64/asm/gic-v2.h
>> create mode 100644 lib/arm64/asm/gic-v3.h
>> create mode 100644 lib/arm64/asm/gic.h
>> create mode 100644 lib/arm64/asm/sysreg.h
>>
>> --
>> 2.7.4
>>
>>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework
@ 2016-11-23 9:55 ` Andre Przywara
0 siblings, 0 replies; 52+ messages in thread
From: Andre Przywara @ 2016-11-23 9:55 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: peter.maydell, marc.zyngier, eric.auger, pbonzini, alex.bennee,
christoffer.dall
Hi,
On 22/11/16 18:45, Andrew Jones wrote:
>
> Andre, Alex, Eric, anybody,
>
> Any more comments on this? If not, I'll send a pull request
> to Radim and Paolo to finally get this merged.
I didn't manage to look in detail at the IPI test, but I am OK with the
rest of the GIC framework.
So I'd say: go ahead.
Cheers,
Andre.
> On Mon, Nov 14, 2016 at 10:08:28PM +0100, Andrew Jones wrote:
>> v6:
>> - rebased to latest master
>> - several other changes thanks to Andre and Alex, changes in
>> individual patch change logs
>> - some code cleanups
>>
>> v5:
>> - fix arm32/gicv3 compile [drew]
>> - use modern register names [Andre]
>> - one Andre r-b
>>
>> v4:
>> - Eric's r-b's
>> - Andre's suggestion to only take defines we need
>> - several other changes listed in individual patches
>>
>> v3:
>> - Rebased on latest master
>> - Added Alex's r-b's
>>
>> v2:
>> Rebased on latest master + my "populate argv[0]" series (will
>> send a REPOST for that shortly. Additionally a few patches got
>> fixes/features;
>> 07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
>> all interrupts as non-secure Group-1" in order to continue
>> working over TCG, as the gicv3 code for TCG removed a hack
>> it had there to make Linux happy.
>> 08/10 added more output for when things fail (if they fail)
>> 09/10 switched gicv3 broadcast implementation to using IRM. This
>> found a bug in a recent (but not tip) kernel, which I was
>> about to fix, but then I saw MarcZ beat me to it.
>> 10/10 actually check that the input irq is the received irq
>>
>>
>> Import defines, and steal enough helper functions, from Linux to
>> enable programming of the gic (v2 and v3). Then use the framework
>> to add an initial test (an ipi test; self, target-list, broadcast).
>>
>> It's my hope that this framework will be a suitable base on which
>> more tests may be easily added, particularly because we have
>> vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
>> vgic-new and tcg gicv3 are merged now)
>>
>> To run it, along with other tests, just do
>>
>> ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
>> make
>> export QEMU=$PATH_TO_QEMU
>> ./run_tests.sh
>>
>> To run it separately do, e.g.
>>
>> $QEMU -machine virt,accel=tcg -cpu cortex-a57 \
>> -device virtio-serial-device \
>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>> -display none -serial stdio \
>> -kernel arm/gic.flat \
>> -smp 123 -machine gic-version=3 -append ipi
>> ^^ note, we can go nuts with nr-cpus on TCG :-)
>>
>> Or, a KVM example using a different "sender" cpu and irq (other than zero)
>>
>> $QEMU -machine virt,accel=kvm -cpu host \
>> -device virtio-serial-device \
>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>> -display none -serial stdio \
>> -kernel arm/gic.flat \
>> -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
>>
>>
>> Patches:
>> 01-05: fixes and functionality needed by the later gic patches
>> 06-07: enable gicv2 and gicv2 IPI test
>> 08-10: enable gicv3 and gicv3 IPI test
>> 11: extend the IPI tests to take variable sender and irq
>>
>> Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
>>
>>
>> Andrew Jones (10):
>> lib: xstr: allow multiple args
>> arm64: fix get_"sysreg32" and make MPIDR 64bit
>> arm/arm64: smp: support more than 8 cpus
>> arm/arm64: add some delay routines
>> arm/arm64: irq enable/disable
>> arm/arm64: add initial gicv2 support
>> arm/arm64: gicv2: add an IPI test
>> arm/arm64: add initial gicv3 support
>> arm/arm64: gicv3: add an IPI test
>> arm/arm64: gic: don't just use zero
>>
>> Peter Xu (1):
>> libcflat: add IS_ALIGNED() macro, and page sizes
>>
>> arm/Makefile.common | 9 +-
>> arm/gic.c | 340 +++++++++++++++++++++++++++++++++++++++++++++
>> arm/run | 19 ++-
>> arm/selftest.c | 5 +-
>> arm/unittests.cfg | 14 ++
>> lib/arm/asm/arch_gicv3.h | 70 ++++++++++
>> lib/arm/asm/gic-v2.h | 36 +++++
>> lib/arm/asm/gic-v3.h | 112 +++++++++++++++
>> lib/arm/asm/gic.h | 106 ++++++++++++++
>> lib/arm/asm/processor.h | 42 +++++-
>> lib/arm/asm/setup.h | 4 +-
>> lib/arm/gic.c | 267 +++++++++++++++++++++++++++++++++++
>> lib/arm/processor.c | 15 ++
>> lib/arm/setup.c | 10 ++
>> lib/arm64/asm/arch_gicv3.h | 66 +++++++++
>> lib/arm64/asm/gic-v2.h | 1 +
>> lib/arm64/asm/gic-v3.h | 1 +
>> lib/arm64/asm/gic.h | 1 +
>> lib/arm64/asm/processor.h | 53 +++++--
>> lib/arm64/asm/sysreg.h | 44 ++++++
>> lib/arm64/processor.c | 15 ++
>> lib/libcflat.h | 10 +-
>> 22 files changed, 1212 insertions(+), 28 deletions(-)
>> create mode 100644 arm/gic.c
>> create mode 100644 lib/arm/asm/arch_gicv3.h
>> create mode 100644 lib/arm/asm/gic-v2.h
>> create mode 100644 lib/arm/asm/gic-v3.h
>> create mode 100644 lib/arm/asm/gic.h
>> create mode 100644 lib/arm/gic.c
>> create mode 100644 lib/arm64/asm/arch_gicv3.h
>> create mode 100644 lib/arm64/asm/gic-v2.h
>> create mode 100644 lib/arm64/asm/gic-v3.h
>> create mode 100644 lib/arm64/asm/gic.h
>> create mode 100644 lib/arm64/asm/sysreg.h
>>
>> --
>> 2.7.4
>>
>>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework
2016-11-22 18:45 ` [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework Andrew Jones
@ 2016-11-23 10:09 ` Alex Bennée
2016-11-23 10:09 ` Alex Bennée
1 sibling, 0 replies; 52+ messages in thread
From: Alex Bennée @ 2016-11-23 10:09 UTC (permalink / raw)
To: Andrew Jones
Cc: kvm, marc.zyngier, andre.przywara, qemu-devel, qemu-arm,
pbonzini, kvmarm
Andrew Jones <drjones@redhat.com> writes:
> Andre, Alex, Eric, anybody,
>
> Any more comments on this? If not, I'll send a pull request
> to Radim and Paolo to finally get this merged.
Looks good to me. I successfully re-based my TCG tests on top and they
work fine ;-)
Tested-by: Alex Bennée <alex.bennee@linaro.org>
>
> Thanks,
> drew
>
>
> On Mon, Nov 14, 2016 at 10:08:28PM +0100, Andrew Jones wrote:
>> v6:
>> - rebased to latest master
>> - several other changes thanks to Andre and Alex, changes in
>> individual patch change logs
>> - some code cleanups
>>
>> v5:
>> - fix arm32/gicv3 compile [drew]
>> - use modern register names [Andre]
>> - one Andre r-b
>>
>> v4:
>> - Eric's r-b's
>> - Andre's suggestion to only take defines we need
>> - several other changes listed in individual patches
>>
>> v3:
>> - Rebased on latest master
>> - Added Alex's r-b's
>>
>> v2:
>> Rebased on latest master + my "populate argv[0]" series (will
>> send a REPOST for that shortly. Additionally a few patches got
>> fixes/features;
>> 07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
>> all interrupts as non-secure Group-1" in order to continue
>> working over TCG, as the gicv3 code for TCG removed a hack
>> it had there to make Linux happy.
>> 08/10 added more output for when things fail (if they fail)
>> 09/10 switched gicv3 broadcast implementation to using IRM. This
>> found a bug in a recent (but not tip) kernel, which I was
>> about to fix, but then I saw MarcZ beat me to it.
>> 10/10 actually check that the input irq is the received irq
>>
>>
>> Import defines, and steal enough helper functions, from Linux to
>> enable programming of the gic (v2 and v3). Then use the framework
>> to add an initial test (an ipi test; self, target-list, broadcast).
>>
>> It's my hope that this framework will be a suitable base on which
>> more tests may be easily added, particularly because we have
>> vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
>> vgic-new and tcg gicv3 are merged now)
>>
>> To run it, along with other tests, just do
>>
>> ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
>> make
>> export QEMU=$PATH_TO_QEMU
>> ./run_tests.sh
>>
>> To run it separately do, e.g.
>>
>> $QEMU -machine virt,accel=tcg -cpu cortex-a57 \
>> -device virtio-serial-device \
>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>> -display none -serial stdio \
>> -kernel arm/gic.flat \
>> -smp 123 -machine gic-version=3 -append ipi
>> ^^ note, we can go nuts with nr-cpus on TCG :-)
>>
>> Or, a KVM example using a different "sender" cpu and irq (other than zero)
>>
>> $QEMU -machine virt,accel=kvm -cpu host \
>> -device virtio-serial-device \
>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>> -display none -serial stdio \
>> -kernel arm/gic.flat \
>> -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
>>
>>
>> Patches:
>> 01-05: fixes and functionality needed by the later gic patches
>> 06-07: enable gicv2 and gicv2 IPI test
>> 08-10: enable gicv3 and gicv3 IPI test
>> 11: extend the IPI tests to take variable sender and irq
>>
>> Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
>>
>>
>> Andrew Jones (10):
>> lib: xstr: allow multiple args
>> arm64: fix get_"sysreg32" and make MPIDR 64bit
>> arm/arm64: smp: support more than 8 cpus
>> arm/arm64: add some delay routines
>> arm/arm64: irq enable/disable
>> arm/arm64: add initial gicv2 support
>> arm/arm64: gicv2: add an IPI test
>> arm/arm64: add initial gicv3 support
>> arm/arm64: gicv3: add an IPI test
>> arm/arm64: gic: don't just use zero
>>
>> Peter Xu (1):
>> libcflat: add IS_ALIGNED() macro, and page sizes
>>
>> arm/Makefile.common | 9 +-
>> arm/gic.c | 340 +++++++++++++++++++++++++++++++++++++++++++++
>> arm/run | 19 ++-
>> arm/selftest.c | 5 +-
>> arm/unittests.cfg | 14 ++
>> lib/arm/asm/arch_gicv3.h | 70 ++++++++++
>> lib/arm/asm/gic-v2.h | 36 +++++
>> lib/arm/asm/gic-v3.h | 112 +++++++++++++++
>> lib/arm/asm/gic.h | 106 ++++++++++++++
>> lib/arm/asm/processor.h | 42 +++++-
>> lib/arm/asm/setup.h | 4 +-
>> lib/arm/gic.c | 267 +++++++++++++++++++++++++++++++++++
>> lib/arm/processor.c | 15 ++
>> lib/arm/setup.c | 10 ++
>> lib/arm64/asm/arch_gicv3.h | 66 +++++++++
>> lib/arm64/asm/gic-v2.h | 1 +
>> lib/arm64/asm/gic-v3.h | 1 +
>> lib/arm64/asm/gic.h | 1 +
>> lib/arm64/asm/processor.h | 53 +++++--
>> lib/arm64/asm/sysreg.h | 44 ++++++
>> lib/arm64/processor.c | 15 ++
>> lib/libcflat.h | 10 +-
>> 22 files changed, 1212 insertions(+), 28 deletions(-)
>> create mode 100644 arm/gic.c
>> create mode 100644 lib/arm/asm/arch_gicv3.h
>> create mode 100644 lib/arm/asm/gic-v2.h
>> create mode 100644 lib/arm/asm/gic-v3.h
>> create mode 100644 lib/arm/asm/gic.h
>> create mode 100644 lib/arm/gic.c
>> create mode 100644 lib/arm64/asm/arch_gicv3.h
>> create mode 100644 lib/arm64/asm/gic-v2.h
>> create mode 100644 lib/arm64/asm/gic-v3.h
>> create mode 100644 lib/arm64/asm/gic.h
>> create mode 100644 lib/arm64/asm/sysreg.h
>>
>> --
>> 2.7.4
>>
>>
--
Alex Bennée
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework
@ 2016-11-23 10:09 ` Alex Bennée
0 siblings, 0 replies; 52+ messages in thread
From: Alex Bennée @ 2016-11-23 10:09 UTC (permalink / raw)
To: Andrew Jones
Cc: kvm, kvmarm, qemu-devel, qemu-arm, peter.maydell, marc.zyngier,
andre.przywara, eric.auger, pbonzini, christoffer.dall
Andrew Jones <drjones@redhat.com> writes:
> Andre, Alex, Eric, anybody,
>
> Any more comments on this? If not, I'll send a pull request
> to Radim and Paolo to finally get this merged.
Looks good to me. I successfully re-based my TCG tests on top and they
work fine ;-)
Tested-by: Alex Bennée <alex.bennee@linaro.org>
>
> Thanks,
> drew
>
>
> On Mon, Nov 14, 2016 at 10:08:28PM +0100, Andrew Jones wrote:
>> v6:
>> - rebased to latest master
>> - several other changes thanks to Andre and Alex, changes in
>> individual patch change logs
>> - some code cleanups
>>
>> v5:
>> - fix arm32/gicv3 compile [drew]
>> - use modern register names [Andre]
>> - one Andre r-b
>>
>> v4:
>> - Eric's r-b's
>> - Andre's suggestion to only take defines we need
>> - several other changes listed in individual patches
>>
>> v3:
>> - Rebased on latest master
>> - Added Alex's r-b's
>>
>> v2:
>> Rebased on latest master + my "populate argv[0]" series (will
>> send a REPOST for that shortly. Additionally a few patches got
>> fixes/features;
>> 07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
>> all interrupts as non-secure Group-1" in order to continue
>> working over TCG, as the gicv3 code for TCG removed a hack
>> it had there to make Linux happy.
>> 08/10 added more output for when things fail (if they fail)
>> 09/10 switched gicv3 broadcast implementation to using IRM. This
>> found a bug in a recent (but not tip) kernel, which I was
>> about to fix, but then I saw MarcZ beat me to it.
>> 10/10 actually check that the input irq is the received irq
>>
>>
>> Import defines, and steal enough helper functions, from Linux to
>> enable programming of the gic (v2 and v3). Then use the framework
>> to add an initial test (an ipi test; self, target-list, broadcast).
>>
>> It's my hope that this framework will be a suitable base on which
>> more tests may be easily added, particularly because we have
>> vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
>> vgic-new and tcg gicv3 are merged now)
>>
>> To run it, along with other tests, just do
>>
>> ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
>> make
>> export QEMU=$PATH_TO_QEMU
>> ./run_tests.sh
>>
>> To run it separately do, e.g.
>>
>> $QEMU -machine virt,accel=tcg -cpu cortex-a57 \
>> -device virtio-serial-device \
>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>> -display none -serial stdio \
>> -kernel arm/gic.flat \
>> -smp 123 -machine gic-version=3 -append ipi
>> ^^ note, we can go nuts with nr-cpus on TCG :-)
>>
>> Or, a KVM example using a different "sender" cpu and irq (other than zero)
>>
>> $QEMU -machine virt,accel=kvm -cpu host \
>> -device virtio-serial-device \
>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>> -display none -serial stdio \
>> -kernel arm/gic.flat \
>> -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
>>
>>
>> Patches:
>> 01-05: fixes and functionality needed by the later gic patches
>> 06-07: enable gicv2 and gicv2 IPI test
>> 08-10: enable gicv3 and gicv3 IPI test
>> 11: extend the IPI tests to take variable sender and irq
>>
>> Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
>>
>>
>> Andrew Jones (10):
>> lib: xstr: allow multiple args
>> arm64: fix get_"sysreg32" and make MPIDR 64bit
>> arm/arm64: smp: support more than 8 cpus
>> arm/arm64: add some delay routines
>> arm/arm64: irq enable/disable
>> arm/arm64: add initial gicv2 support
>> arm/arm64: gicv2: add an IPI test
>> arm/arm64: add initial gicv3 support
>> arm/arm64: gicv3: add an IPI test
>> arm/arm64: gic: don't just use zero
>>
>> Peter Xu (1):
>> libcflat: add IS_ALIGNED() macro, and page sizes
>>
>> arm/Makefile.common | 9 +-
>> arm/gic.c | 340 +++++++++++++++++++++++++++++++++++++++++++++
>> arm/run | 19 ++-
>> arm/selftest.c | 5 +-
>> arm/unittests.cfg | 14 ++
>> lib/arm/asm/arch_gicv3.h | 70 ++++++++++
>> lib/arm/asm/gic-v2.h | 36 +++++
>> lib/arm/asm/gic-v3.h | 112 +++++++++++++++
>> lib/arm/asm/gic.h | 106 ++++++++++++++
>> lib/arm/asm/processor.h | 42 +++++-
>> lib/arm/asm/setup.h | 4 +-
>> lib/arm/gic.c | 267 +++++++++++++++++++++++++++++++++++
>> lib/arm/processor.c | 15 ++
>> lib/arm/setup.c | 10 ++
>> lib/arm64/asm/arch_gicv3.h | 66 +++++++++
>> lib/arm64/asm/gic-v2.h | 1 +
>> lib/arm64/asm/gic-v3.h | 1 +
>> lib/arm64/asm/gic.h | 1 +
>> lib/arm64/asm/processor.h | 53 +++++--
>> lib/arm64/asm/sysreg.h | 44 ++++++
>> lib/arm64/processor.c | 15 ++
>> lib/libcflat.h | 10 +-
>> 22 files changed, 1212 insertions(+), 28 deletions(-)
>> create mode 100644 arm/gic.c
>> create mode 100644 lib/arm/asm/arch_gicv3.h
>> create mode 100644 lib/arm/asm/gic-v2.h
>> create mode 100644 lib/arm/asm/gic-v3.h
>> create mode 100644 lib/arm/asm/gic.h
>> create mode 100644 lib/arm/gic.c
>> create mode 100644 lib/arm64/asm/arch_gicv3.h
>> create mode 100644 lib/arm64/asm/gic-v2.h
>> create mode 100644 lib/arm64/asm/gic-v3.h
>> create mode 100644 lib/arm64/asm/gic.h
>> create mode 100644 lib/arm64/asm/sysreg.h
>>
>> --
>> 2.7.4
>>
>>
--
Alex Bennée
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [kvm-unit-tests PATCH v6 08/11] libcflat: add IS_ALIGNED() macro, and page sizes
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-23 10:38 ` Auger Eric
-1 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 10:38 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: marc.zyngier, andre.przywara, pbonzini
Hi,
On 14/11/2016 22:08, Andrew Jones wrote:
> From: Peter Xu <peterx@redhat.com>
>
> These macros will be useful to do page alignment checks.
>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> [drew: also added SZ_64K and changed to shifts]
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v6: change to shifts [Alex]
> ---
> lib/libcflat.h | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/lib/libcflat.h b/lib/libcflat.h
> index 82005f5d014f..244e40a724be 100644
> --- a/lib/libcflat.h
> +++ b/lib/libcflat.h
> @@ -33,6 +33,12 @@
> #define __ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
> #define __ALIGN(x, a) __ALIGN_MASK(x, (typeof(x))(a) - 1)
> #define ALIGN(x, a) __ALIGN((x), (a))
> +#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
> +
> +#define SZ_4K (1 << 12)
> +#define SZ_64K (1 << 16)
> +#define SZ_2M (1 << 21)
> +#define SZ_1G (1 << 30)
>
> typedef uint8_t u8;
> typedef int8_t s8;
>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Eric
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 08/11] libcflat: add IS_ALIGNED() macro, and page sizes
@ 2016-11-23 10:38 ` Auger Eric
0 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 10:38 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, christoffer.dall, Peter Xu
Hi,
On 14/11/2016 22:08, Andrew Jones wrote:
> From: Peter Xu <peterx@redhat.com>
>
> These macros will be useful to do page alignment checks.
>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> [drew: also added SZ_64K and changed to shifts]
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v6: change to shifts [Alex]
> ---
> lib/libcflat.h | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/lib/libcflat.h b/lib/libcflat.h
> index 82005f5d014f..244e40a724be 100644
> --- a/lib/libcflat.h
> +++ b/lib/libcflat.h
> @@ -33,6 +33,12 @@
> #define __ALIGN_MASK(x, mask) (((x) + (mask)) & ~(mask))
> #define __ALIGN(x, a) __ALIGN_MASK(x, (typeof(x))(a) - 1)
> #define ALIGN(x, a) __ALIGN((x), (a))
> +#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
> +
> +#define SZ_4K (1 << 12)
> +#define SZ_64K (1 << 16)
> +#define SZ_2M (1 << 21)
> +#define SZ_1G (1 << 30)
>
> typedef uint8_t u8;
> typedef int8_t s8;
>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Eric
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [kvm-unit-tests PATCH v6 06/11] arm/arm64: add initial gicv2 support
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-23 10:38 ` Auger Eric
-1 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 10:38 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, christoffer.dall
Hi,
On 14/11/2016 22:08, Andrew Jones wrote:
> Add some gicv2 support. This just adds init and enable
> functions, allowing unit tests to start messing with it.
>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v6: added comments (register offset headers) [Alex]
> v5: share/use only the modern register names [Andre]
> v4:
> - only take defines from kernel we need now [Andre]
> - moved defines to asm/gic.h so they'll be shared with v3 [drew]
> - simplify enable by not caring if we reinit the distributor [drew]
> - init all GICD_INT_DEF_PRI_X4 registers [Eric]
> ---
> arm/Makefile.common | 1 +
> lib/arm/asm/gic-v2.h | 34 ++++++++++++++++++++++
> lib/arm/asm/gic.h | 39 ++++++++++++++++++++++++++
> lib/arm/gic.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
> lib/arm64/asm/gic-v2.h | 1 +
> lib/arm64/asm/gic.h | 1 +
> 6 files changed, 152 insertions(+)
> create mode 100644 lib/arm/asm/gic-v2.h
> create mode 100644 lib/arm/asm/gic.h
> create mode 100644 lib/arm/gic.c
> create mode 100644 lib/arm64/asm/gic-v2.h
> create mode 100644 lib/arm64/asm/gic.h
>
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index f37b5c2a3de4..6f56015c43c4 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -46,6 +46,7 @@ cflatobjs += lib/arm/mmu.o
> cflatobjs += lib/arm/bitops.o
> cflatobjs += lib/arm/psci.o
> cflatobjs += lib/arm/smp.o
> +cflatobjs += lib/arm/gic.o
>
> libeabi = lib/arm/libeabi.a
> eabiobjs = lib/arm/eabi_compat.o
> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
> new file mode 100644
> index 000000000000..c2d5fecd4886
> --- /dev/null
> +++ b/lib/arm/asm/gic-v2.h
> @@ -0,0 +1,34 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V2_H_
> +#define _ASMARM_GIC_V2_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
> +#endif
> +
> +#define GICD_ENABLE 0x1
> +#define GICC_ENABLE 0x1
> +
> +#ifndef __ASSEMBLY__
> +
> +struct gicv2_data {
> + void *dist_base;
> + void *cpu_base;
> + unsigned int irq_nr;
> +};
> +extern struct gicv2_data gicv2_data;
> +
> +#define gicv2_dist_base() (gicv2_data.dist_base)
> +#define gicv2_cpu_base() (gicv2_data.cpu_base)
> +
> +extern int gicv2_init(void);
> +extern void gicv2_enable_defaults(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V2_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> new file mode 100644
> index 000000000000..e3580bd1d42d
> --- /dev/null
> +++ b/lib/arm/asm/gic.h
> @@ -0,0 +1,39 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_H_
> +#define _ASMARM_GIC_H_
> +
> +#include <asm/gic-v2.h>
> +
> +/* Distributor registers */
> +#define GICD_CTLR 0x0000
> +#define GICD_TYPER 0x0004
> +#define GICD_ISENABLER 0x0100
> +#define GICD_IPRIORITYR 0x0400
> +
> +#define GICD_TYPER_IRQS(typer) ((((typer) & 0x1f) + 1) * 32)
> +#define GICD_INT_EN_SET_SGI 0x0000ffff
> +#define GICD_INT_DEF_PRI_X4 0xa0a0a0a0
> +
> +/* CPU interface registers */
> +#define GICC_CTLR 0x0000
> +#define GICC_PMR 0x0004
> +
> +#define GICC_INT_PRI_THRESHOLD 0xf0
> +
> +#ifndef __ASSEMBLY__
> +
> +/*
> + * gic_init will try to find all known gics, and then
> + * initialize the gic data for the one found.
> + * returns
> + * 0 : no gic was found
> + * > 0 : the gic version of the gic found
> + */
> +extern int gic_init(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_H_ */
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> new file mode 100644
> index 000000000000..d655105e058b
> --- /dev/null
> +++ b/lib/arm/gic.c
> @@ -0,0 +1,76 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <devicetree.h>
> +#include <asm/gic.h>
> +#include <asm/io.h>
> +
> +struct gicv2_data gicv2_data;
> +
> +/*
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + */
> +static bool
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +{
> + struct dt_pbus_reg reg;
> + struct dt_device gic;
> + struct dt_bus bus;
> + int node, ret;
> +
> + dt_bus_init_defaults(&bus);
> + dt_device_init(&gic, &bus, NULL);
> +
> + node = dt_device_find_compatible(&gic, compatible);
> + assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
> +
> + if (node == -FDT_ERR_NOTFOUND)
> + return false;
> +
> + dt_device_bind_node(&gic, node);
> +
> + ret = dt_pbus_translate(&gic, 0, ®);
> + assert(ret == 0);
> + *base1 = ioremap(reg.addr, reg.size);
> +
> + ret = dt_pbus_translate(&gic, 1, ®);
> + assert(ret == 0);
> + *base2 = ioremap(reg.addr, reg.size);
> +
> + return true;
> +}
> +
> +int gicv2_init(void)
> +{
> + return gic_get_dt_bases("arm,cortex-a15-gic",
> + &gicv2_data.dist_base, &gicv2_data.cpu_base);
> +}
> +
> +int gic_init(void)
> +{
> + if (gicv2_init())
> + return 2;
> + return 0;
> +}
> +
> +void gicv2_enable_defaults(void)
> +{
> + void *dist = gicv2_dist_base();
> + void *cpu_base = gicv2_cpu_base();
> + unsigned int i;
> +
> + gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> + if (gicv2_data.irq_nr > 1020)
> + gicv2_data.irq_nr = 1020;
> +
> + for (i = 0; i < gicv2_data.irq_nr; i += 4)
> + writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
> +
> + writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
> + writel(GICD_ENABLE, dist + GICD_CTLR);
> +
> + writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> + writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> +}
> diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
> new file mode 100644
> index 000000000000..52226624a209
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v2.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v2.h"
> diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
> new file mode 100644
> index 000000000000..e5eb302a31b4
> --- /dev/null
> +++ b/lib/arm64/asm/gic.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic.h"
>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Eric
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 06/11] arm/arm64: add initial gicv2 support
@ 2016-11-23 10:38 ` Auger Eric
0 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 10:38 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, christoffer.dall
Hi,
On 14/11/2016 22:08, Andrew Jones wrote:
> Add some gicv2 support. This just adds init and enable
> functions, allowing unit tests to start messing with it.
>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v6: added comments (register offset headers) [Alex]
> v5: share/use only the modern register names [Andre]
> v4:
> - only take defines from kernel we need now [Andre]
> - moved defines to asm/gic.h so they'll be shared with v3 [drew]
> - simplify enable by not caring if we reinit the distributor [drew]
> - init all GICD_INT_DEF_PRI_X4 registers [Eric]
> ---
> arm/Makefile.common | 1 +
> lib/arm/asm/gic-v2.h | 34 ++++++++++++++++++++++
> lib/arm/asm/gic.h | 39 ++++++++++++++++++++++++++
> lib/arm/gic.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
> lib/arm64/asm/gic-v2.h | 1 +
> lib/arm64/asm/gic.h | 1 +
> 6 files changed, 152 insertions(+)
> create mode 100644 lib/arm/asm/gic-v2.h
> create mode 100644 lib/arm/asm/gic.h
> create mode 100644 lib/arm/gic.c
> create mode 100644 lib/arm64/asm/gic-v2.h
> create mode 100644 lib/arm64/asm/gic.h
>
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index f37b5c2a3de4..6f56015c43c4 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -46,6 +46,7 @@ cflatobjs += lib/arm/mmu.o
> cflatobjs += lib/arm/bitops.o
> cflatobjs += lib/arm/psci.o
> cflatobjs += lib/arm/smp.o
> +cflatobjs += lib/arm/gic.o
>
> libeabi = lib/arm/libeabi.a
> eabiobjs = lib/arm/eabi_compat.o
> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
> new file mode 100644
> index 000000000000..c2d5fecd4886
> --- /dev/null
> +++ b/lib/arm/asm/gic-v2.h
> @@ -0,0 +1,34 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V2_H_
> +#define _ASMARM_GIC_V2_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
> +#endif
> +
> +#define GICD_ENABLE 0x1
> +#define GICC_ENABLE 0x1
> +
> +#ifndef __ASSEMBLY__
> +
> +struct gicv2_data {
> + void *dist_base;
> + void *cpu_base;
> + unsigned int irq_nr;
> +};
> +extern struct gicv2_data gicv2_data;
> +
> +#define gicv2_dist_base() (gicv2_data.dist_base)
> +#define gicv2_cpu_base() (gicv2_data.cpu_base)
> +
> +extern int gicv2_init(void);
> +extern void gicv2_enable_defaults(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V2_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> new file mode 100644
> index 000000000000..e3580bd1d42d
> --- /dev/null
> +++ b/lib/arm/asm/gic.h
> @@ -0,0 +1,39 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_H_
> +#define _ASMARM_GIC_H_
> +
> +#include <asm/gic-v2.h>
> +
> +/* Distributor registers */
> +#define GICD_CTLR 0x0000
> +#define GICD_TYPER 0x0004
> +#define GICD_ISENABLER 0x0100
> +#define GICD_IPRIORITYR 0x0400
> +
> +#define GICD_TYPER_IRQS(typer) ((((typer) & 0x1f) + 1) * 32)
> +#define GICD_INT_EN_SET_SGI 0x0000ffff
> +#define GICD_INT_DEF_PRI_X4 0xa0a0a0a0
> +
> +/* CPU interface registers */
> +#define GICC_CTLR 0x0000
> +#define GICC_PMR 0x0004
> +
> +#define GICC_INT_PRI_THRESHOLD 0xf0
> +
> +#ifndef __ASSEMBLY__
> +
> +/*
> + * gic_init will try to find all known gics, and then
> + * initialize the gic data for the one found.
> + * returns
> + * 0 : no gic was found
> + * > 0 : the gic version of the gic found
> + */
> +extern int gic_init(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_H_ */
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> new file mode 100644
> index 000000000000..d655105e058b
> --- /dev/null
> +++ b/lib/arm/gic.c
> @@ -0,0 +1,76 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <devicetree.h>
> +#include <asm/gic.h>
> +#include <asm/io.h>
> +
> +struct gicv2_data gicv2_data;
> +
> +/*
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + */
> +static bool
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +{
> + struct dt_pbus_reg reg;
> + struct dt_device gic;
> + struct dt_bus bus;
> + int node, ret;
> +
> + dt_bus_init_defaults(&bus);
> + dt_device_init(&gic, &bus, NULL);
> +
> + node = dt_device_find_compatible(&gic, compatible);
> + assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
> +
> + if (node == -FDT_ERR_NOTFOUND)
> + return false;
> +
> + dt_device_bind_node(&gic, node);
> +
> + ret = dt_pbus_translate(&gic, 0, ®);
> + assert(ret == 0);
> + *base1 = ioremap(reg.addr, reg.size);
> +
> + ret = dt_pbus_translate(&gic, 1, ®);
> + assert(ret == 0);
> + *base2 = ioremap(reg.addr, reg.size);
> +
> + return true;
> +}
> +
> +int gicv2_init(void)
> +{
> + return gic_get_dt_bases("arm,cortex-a15-gic",
> + &gicv2_data.dist_base, &gicv2_data.cpu_base);
> +}
> +
> +int gic_init(void)
> +{
> + if (gicv2_init())
> + return 2;
> + return 0;
> +}
> +
> +void gicv2_enable_defaults(void)
> +{
> + void *dist = gicv2_dist_base();
> + void *cpu_base = gicv2_cpu_base();
> + unsigned int i;
> +
> + gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> + if (gicv2_data.irq_nr > 1020)
> + gicv2_data.irq_nr = 1020;
> +
> + for (i = 0; i < gicv2_data.irq_nr; i += 4)
> + writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
> +
> + writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
> + writel(GICD_ENABLE, dist + GICD_CTLR);
> +
> + writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> + writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> +}
> diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
> new file mode 100644
> index 000000000000..52226624a209
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v2.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v2.h"
> diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
> new file mode 100644
> index 000000000000..e5eb302a31b4
> --- /dev/null
> +++ b/lib/arm64/asm/gic.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic.h"
>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Eric
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [kvm-unit-tests PATCH v6 09/11] arm/arm64: add initial gicv3 support
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-23 10:38 ` Auger Eric
-1 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 10:38 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: marc.zyngier, andre.przywara, pbonzini
Hi Drew,
On 14/11/2016 22:08, Andrew Jones wrote:
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v6:
> - added comments [Alex]
> - added stride parameter to gicv3_set_redist_base [Andre]
> - redist-wait s/rwp/uwp/ and comment [Andre]
> - removed unnecessary wait-for-rwps [Andre]
> v5: use modern register names [Andre]
> v4:
> - only take defines from kernel we need now [Andre]
> - simplify enable by not caring if we reinit the distributor [drew]
> v2:
> - configure irqs as NS GRP1
> ---
> lib/arm/asm/arch_gicv3.h | 47 ++++++++++++++++++++
> lib/arm/asm/gic-v3.h | 104 +++++++++++++++++++++++++++++++++++++++++++++
> lib/arm/asm/gic.h | 5 ++-
> lib/arm/gic.c | 64 ++++++++++++++++++++++++++++
> lib/arm64/asm/arch_gicv3.h | 44 +++++++++++++++++++
> lib/arm64/asm/gic-v3.h | 1 +
> lib/arm64/asm/sysreg.h | 44 +++++++++++++++++++
> 7 files changed, 308 insertions(+), 1 deletion(-)
> create mode 100644 lib/arm/asm/arch_gicv3.h
> create mode 100644 lib/arm/asm/gic-v3.h
> create mode 100644 lib/arm64/asm/arch_gicv3.h
> create mode 100644 lib/arm64/asm/gic-v3.h
> create mode 100644 lib/arm64/asm/sysreg.h
>
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..276577452a14
> --- /dev/null
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -0,0 +1,47 @@
> +/*
> + * All ripped off from arch/arm/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_ARCH_GICV3_H_
> +#define _ASMARM_ARCH_GICV3_H_
> +
> +#ifndef __ASSEMBLY__
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +#include <asm/io.h>
> +
> +#define __stringify xstr
> +
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
> +
> +#define ICC_PMR __ACCESS_CP15(c4, 0, c6, 0)
> +#define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> + asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> + asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> + isb();
> +}
> +
> +/*
> + * We may access GICR_TYPER and GITS_TYPER by reading both the TYPER
> + * offset and the following offset (+ 4) and then combining them to
> + * form a 64-bit address.
> + */
> +static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
> +{
> + u64 val = readl(addr);
> + val |= (u64)readl(addr + 4) << 32;
> + return val;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_ARCH_GICV3_H_ */
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> new file mode 100644
> index 000000000000..73ade4681d21
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3.h
> @@ -0,0 +1,104 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_H_
> +#define _ASMARM_GIC_V3_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
> +#endif
> +
> +/*
> + * Distributor registers
> + *
> + * We expect to be run in Non-secure mode, thus we define the
> + * group1 enable bits with respect to that view.
> + */
> +#define GICD_CTLR_RWP (1U << 31)
> +#define GICD_CTLR_ARE_NS (1U << 4)
> +#define GICD_CTLR_ENABLE_G1A (1U << 1)
> +#define GICD_CTLR_ENABLE_G1 (1U << 0)
> +
> +/* Re-Distributor registers, offsets from RD_base */
> +#define GICR_TYPER 0x0008
> +
> +#define GICR_TYPER_LAST (1U << 4)
> +
> +/* Re-Distributor registers, offsets from SGI_base */
> +#define GICR_IGROUPR0 GICD_IGROUPR
> +#define GICR_ISENABLER0 GICD_ISENABLER
> +#define GICR_IPRIORITYR0 GICD_IPRIORITYR
> +
> +#include <asm/arch_gicv3.h>
> +
> +#ifndef __ASSEMBLY__
> +#include <asm/setup.h>
> +#include <asm/smp.h>
> +#include <asm/processor.h>
> +#include <asm/io.h>
> +
> +struct gicv3_data {
> + void *dist_base;
> + void *redist_base[NR_CPUS];
> + unsigned int irq_nr;
> +};
> +extern struct gicv3_data gicv3_data;
> +
> +#define gicv3_dist_base() (gicv3_data.dist_base)
> +#define gicv3_redist_base() (gicv3_data.redist_base[smp_processor_id()])
> +#define gicv3_sgi_base() (gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
> +
> +extern int gicv3_init(void);
> +extern void gicv3_enable_defaults(void);
> +extern void gicv3_set_redist_base(size_t stride);
> +
> +static inline void gicv3_do_wait_for_rwp(void *base)
> +{
> + int count = 100000; /* 1s */
> +
> + while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
> + if (!--count) {
> + printf("GICv3: RWP timeout!\n");
> + abort();
> + }
> + cpu_relax();
> + udelay(10);
> + };
> +}
> +
> +static inline void gicv3_dist_wait_for_rwp(void)
> +{
> + gicv3_do_wait_for_rwp(gicv3_dist_base());
> +}
> +
> +static inline void gicv3_redist_wait_for_uwp(void)
> +{
> + /*
> + * We can build on gic_do_wait_for_rwp, which uses GICD_ registers
> + * because GICD_CTLR == GICR_CTLR and GICD_CTLR_RWP == GICR_CTLR_UWP
> + */
> + gicv3_do_wait_for_rwp(gicv3_redist_base());
> +}
> +
> +static inline u32 mpidr_compress(u64 mpidr)
> +{
> + u64 compressed = mpidr & MPIDR_HWID_BITMASK;
> +
> + compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
> + return compressed;
> +}
> +
> +static inline u64 mpidr_uncompress(u32 compressed)
> +{
> + u64 mpidr = ((u64)compressed >> 24) << 32;
> +
> + mpidr |= compressed & MPIDR_HWID_BITMASK;
> + return mpidr;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> index d816b96e46b4..21511997f2a9 100644
> --- a/lib/arm/asm/gic.h
> +++ b/lib/arm/asm/gic.h
> @@ -6,11 +6,11 @@
> #ifndef _ASMARM_GIC_H_
> #define _ASMARM_GIC_H_
>
> -#include <asm/gic-v2.h>
>
> /* Distributor registers */
> #define GICD_CTLR 0x0000
> #define GICD_TYPER 0x0004
> +#define GICD_IGROUPR 0x0080
> #define GICD_ISENABLER 0x0100
> #define GICD_IPRIORITYR 0x0400
> #define GICD_SGIR 0x0f00
> @@ -28,6 +28,9 @@
> #define GICC_INT_PRI_THRESHOLD 0xf0
> #define GICC_INT_SPURIOUS 0x3ff
>
> +#include <asm/gic-v2.h>
> +#include <asm/gic-v3.h>
> +
> #ifndef __ASSEMBLY__
>
> /*
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index d655105e058b..d703ad96a37e 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -8,9 +8,11 @@
> #include <asm/io.h>
>
> struct gicv2_data gicv2_data;
> +struct gicv3_data gicv3_data;
>
> /*
> * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> */
> static bool
> gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> @@ -48,10 +50,18 @@ int gicv2_init(void)
> &gicv2_data.dist_base, &gicv2_data.cpu_base);
> }
>
> +int gicv3_init(void)
> +{
> + return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> + &gicv3_data.redist_base[0]);
> +}
> +
> int gic_init(void)
> {
> if (gicv2_init())
> return 2;
> + else if (gicv3_init())
> + return 3;
> return 0;
> }
>
> @@ -74,3 +84,57 @@ void gicv2_enable_defaults(void)
> writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> }
> +
> +void gicv3_set_redist_base(size_t stride)
> +{
> + u32 aff = mpidr_compress(get_mpidr());
> + void *ptr = gicv3_data.redist_base[0];
> + u64 typer;
> +
> + do {
> + typer = gicv3_read_typer(ptr + GICR_TYPER);
> + if ((typer >> 32) == aff) {
> + gicv3_redist_base() = ptr;
> + return;
> + }
> + ptr += stride; /* skip RD_base, SGI_base, etc. */
> + } while (!(typer & GICR_TYPER_LAST));
> +
> + /* should never reach here */
> + assert(0);
> +}
> +
> +void gicv3_enable_defaults(void)
> +{
> + void *dist = gicv3_dist_base();
> + void *sgi_base;
> + unsigned int i;
> +
> + gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> + if (gicv3_data.irq_nr > 1020)
> + gicv3_data.irq_nr = 1020;
> +
> + writel(0, dist + GICD_CTLR);
> + gicv3_dist_wait_for_rwp();
> +
> + writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
> + dist + GICD_CTLR);
> + gicv3_dist_wait_for_rwp();
> +
> + for (i = 0; i < gicv3_data.irq_nr; i += 4)
> + writel(~0, dist + GICD_IGROUPR + i);
> +
> + if (!gicv3_redist_base())
> + gicv3_set_redist_base(SZ_64K * 2);
> + sgi_base = gicv3_sgi_base();
> +
> + writel(~0, sgi_base + GICR_IGROUPR0);
> +
> + for (i = 0; i < 16; i += 4)
> + writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
> +
> + writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
> +
> + gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
> + gicv3_write_grpen1(1);
> +}
> diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..6d353567f56a
> --- /dev/null
> +++ b/lib/arm64/asm/arch_gicv3.h
> @@ -0,0 +1,44 @@
> +/*
> + * All ripped off from arch/arm64/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_ARCH_GICV3_H_
> +#define _ASMARM64_ARCH_GICV3_H_
> +
> +#include <asm/sysreg.h>
> +
> +#define ICC_PMR_EL1 sys_reg(3, 0, 4, 6, 0)
> +#define ICC_GRPEN1_EL1 sys_reg(3, 0, 12, 12, 7)
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +
> +#define __stringify xstr
> +
> +/*
> + * Low-level accessors
> + *
> + * These system registers are 32 bits, but we make sure that the compiler
> + * sets the GP register's most significant bits to 0 with an explicit cast.
> + */
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> + asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> + asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
> + isb();
> +}
> +
> +#define gicv3_read_typer(c) readq(c)
> +
> +#endif /* __ASSEMBLY__ */
> +#endif /* _ASMARM64_ARCH_GICV3_H_ */
> diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
> new file mode 100644
> index 000000000000..8ee5d4d9c181
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3.h"
> diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
> new file mode 100644
> index 000000000000..544a46cb8cc5
> --- /dev/null
> +++ b/lib/arm64/asm/sysreg.h
> @@ -0,0 +1,44 @@
> +/*
> + * Ripped off from arch/arm64/include/asm/sysreg.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_SYSREG_H_
> +#define _ASMARM64_SYSREG_H_
> +
> +#define sys_reg(op0, op1, crn, crm, op2) \
> + ((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
> +
> +#ifdef __ASSEMBLY__
> + .irp num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
> + .equ .L__reg_num_x\num, \num
> + .endr
> + .equ .L__reg_num_xzr, 31
> +
> + .macro mrs_s, rt, sreg
> + .inst 0xd5200000|(\sreg)|(.L__reg_num_\rt)
> + .endm
> +
> + .macro msr_s, sreg, rt
> + .inst 0xd5000000|(\sreg)|(.L__reg_num_\rt)
> + .endm
> +#else
> +asm(
> +" .irp num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
> +" .equ .L__reg_num_x\\num, \\num\n"
> +" .endr\n"
> +" .equ .L__reg_num_xzr, 31\n"
> +"\n"
> +" .macro mrs_s, rt, sreg\n"
> +" .inst 0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +" .endm\n"
> +"\n"
> +" .macro msr_s, sreg, rt\n"
> +" .inst 0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +" .endm\n"
> +);
> +#endif
> +
> +#endif /* _ASMARM64_SYSREG_H_ */
>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Eric
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 09/11] arm/arm64: add initial gicv3 support
@ 2016-11-23 10:38 ` Auger Eric
0 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 10:38 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, christoffer.dall
Hi Drew,
On 14/11/2016 22:08, Andrew Jones wrote:
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v6:
> - added comments [Alex]
> - added stride parameter to gicv3_set_redist_base [Andre]
> - redist-wait s/rwp/uwp/ and comment [Andre]
> - removed unnecessary wait-for-rwps [Andre]
> v5: use modern register names [Andre]
> v4:
> - only take defines from kernel we need now [Andre]
> - simplify enable by not caring if we reinit the distributor [drew]
> v2:
> - configure irqs as NS GRP1
> ---
> lib/arm/asm/arch_gicv3.h | 47 ++++++++++++++++++++
> lib/arm/asm/gic-v3.h | 104 +++++++++++++++++++++++++++++++++++++++++++++
> lib/arm/asm/gic.h | 5 ++-
> lib/arm/gic.c | 64 ++++++++++++++++++++++++++++
> lib/arm64/asm/arch_gicv3.h | 44 +++++++++++++++++++
> lib/arm64/asm/gic-v3.h | 1 +
> lib/arm64/asm/sysreg.h | 44 +++++++++++++++++++
> 7 files changed, 308 insertions(+), 1 deletion(-)
> create mode 100644 lib/arm/asm/arch_gicv3.h
> create mode 100644 lib/arm/asm/gic-v3.h
> create mode 100644 lib/arm64/asm/arch_gicv3.h
> create mode 100644 lib/arm64/asm/gic-v3.h
> create mode 100644 lib/arm64/asm/sysreg.h
>
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..276577452a14
> --- /dev/null
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -0,0 +1,47 @@
> +/*
> + * All ripped off from arch/arm/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_ARCH_GICV3_H_
> +#define _ASMARM_ARCH_GICV3_H_
> +
> +#ifndef __ASSEMBLY__
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +#include <asm/io.h>
> +
> +#define __stringify xstr
> +
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
> +
> +#define ICC_PMR __ACCESS_CP15(c4, 0, c6, 0)
> +#define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> + asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> + asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> + isb();
> +}
> +
> +/*
> + * We may access GICR_TYPER and GITS_TYPER by reading both the TYPER
> + * offset and the following offset (+ 4) and then combining them to
> + * form a 64-bit address.
> + */
> +static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
> +{
> + u64 val = readl(addr);
> + val |= (u64)readl(addr + 4) << 32;
> + return val;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_ARCH_GICV3_H_ */
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> new file mode 100644
> index 000000000000..73ade4681d21
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3.h
> @@ -0,0 +1,104 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_H_
> +#define _ASMARM_GIC_V3_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
> +#endif
> +
> +/*
> + * Distributor registers
> + *
> + * We expect to be run in Non-secure mode, thus we define the
> + * group1 enable bits with respect to that view.
> + */
> +#define GICD_CTLR_RWP (1U << 31)
> +#define GICD_CTLR_ARE_NS (1U << 4)
> +#define GICD_CTLR_ENABLE_G1A (1U << 1)
> +#define GICD_CTLR_ENABLE_G1 (1U << 0)
> +
> +/* Re-Distributor registers, offsets from RD_base */
> +#define GICR_TYPER 0x0008
> +
> +#define GICR_TYPER_LAST (1U << 4)
> +
> +/* Re-Distributor registers, offsets from SGI_base */
> +#define GICR_IGROUPR0 GICD_IGROUPR
> +#define GICR_ISENABLER0 GICD_ISENABLER
> +#define GICR_IPRIORITYR0 GICD_IPRIORITYR
> +
> +#include <asm/arch_gicv3.h>
> +
> +#ifndef __ASSEMBLY__
> +#include <asm/setup.h>
> +#include <asm/smp.h>
> +#include <asm/processor.h>
> +#include <asm/io.h>
> +
> +struct gicv3_data {
> + void *dist_base;
> + void *redist_base[NR_CPUS];
> + unsigned int irq_nr;
> +};
> +extern struct gicv3_data gicv3_data;
> +
> +#define gicv3_dist_base() (gicv3_data.dist_base)
> +#define gicv3_redist_base() (gicv3_data.redist_base[smp_processor_id()])
> +#define gicv3_sgi_base() (gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
> +
> +extern int gicv3_init(void);
> +extern void gicv3_enable_defaults(void);
> +extern void gicv3_set_redist_base(size_t stride);
> +
> +static inline void gicv3_do_wait_for_rwp(void *base)
> +{
> + int count = 100000; /* 1s */
> +
> + while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
> + if (!--count) {
> + printf("GICv3: RWP timeout!\n");
> + abort();
> + }
> + cpu_relax();
> + udelay(10);
> + };
> +}
> +
> +static inline void gicv3_dist_wait_for_rwp(void)
> +{
> + gicv3_do_wait_for_rwp(gicv3_dist_base());
> +}
> +
> +static inline void gicv3_redist_wait_for_uwp(void)
> +{
> + /*
> + * We can build on gic_do_wait_for_rwp, which uses GICD_ registers
> + * because GICD_CTLR == GICR_CTLR and GICD_CTLR_RWP == GICR_CTLR_UWP
> + */
> + gicv3_do_wait_for_rwp(gicv3_redist_base());
> +}
> +
> +static inline u32 mpidr_compress(u64 mpidr)
> +{
> + u64 compressed = mpidr & MPIDR_HWID_BITMASK;
> +
> + compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
> + return compressed;
> +}
> +
> +static inline u64 mpidr_uncompress(u32 compressed)
> +{
> + u64 mpidr = ((u64)compressed >> 24) << 32;
> +
> + mpidr |= compressed & MPIDR_HWID_BITMASK;
> + return mpidr;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> index d816b96e46b4..21511997f2a9 100644
> --- a/lib/arm/asm/gic.h
> +++ b/lib/arm/asm/gic.h
> @@ -6,11 +6,11 @@
> #ifndef _ASMARM_GIC_H_
> #define _ASMARM_GIC_H_
>
> -#include <asm/gic-v2.h>
>
> /* Distributor registers */
> #define GICD_CTLR 0x0000
> #define GICD_TYPER 0x0004
> +#define GICD_IGROUPR 0x0080
> #define GICD_ISENABLER 0x0100
> #define GICD_IPRIORITYR 0x0400
> #define GICD_SGIR 0x0f00
> @@ -28,6 +28,9 @@
> #define GICC_INT_PRI_THRESHOLD 0xf0
> #define GICC_INT_SPURIOUS 0x3ff
>
> +#include <asm/gic-v2.h>
> +#include <asm/gic-v3.h>
> +
> #ifndef __ASSEMBLY__
>
> /*
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index d655105e058b..d703ad96a37e 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -8,9 +8,11 @@
> #include <asm/io.h>
>
> struct gicv2_data gicv2_data;
> +struct gicv3_data gicv3_data;
>
> /*
> * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> */
> static bool
> gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> @@ -48,10 +50,18 @@ int gicv2_init(void)
> &gicv2_data.dist_base, &gicv2_data.cpu_base);
> }
>
> +int gicv3_init(void)
> +{
> + return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> + &gicv3_data.redist_base[0]);
> +}
> +
> int gic_init(void)
> {
> if (gicv2_init())
> return 2;
> + else if (gicv3_init())
> + return 3;
> return 0;
> }
>
> @@ -74,3 +84,57 @@ void gicv2_enable_defaults(void)
> writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> }
> +
> +void gicv3_set_redist_base(size_t stride)
> +{
> + u32 aff = mpidr_compress(get_mpidr());
> + void *ptr = gicv3_data.redist_base[0];
> + u64 typer;
> +
> + do {
> + typer = gicv3_read_typer(ptr + GICR_TYPER);
> + if ((typer >> 32) == aff) {
> + gicv3_redist_base() = ptr;
> + return;
> + }
> + ptr += stride; /* skip RD_base, SGI_base, etc. */
> + } while (!(typer & GICR_TYPER_LAST));
> +
> + /* should never reach here */
> + assert(0);
> +}
> +
> +void gicv3_enable_defaults(void)
> +{
> + void *dist = gicv3_dist_base();
> + void *sgi_base;
> + unsigned int i;
> +
> + gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> + if (gicv3_data.irq_nr > 1020)
> + gicv3_data.irq_nr = 1020;
> +
> + writel(0, dist + GICD_CTLR);
> + gicv3_dist_wait_for_rwp();
> +
> + writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
> + dist + GICD_CTLR);
> + gicv3_dist_wait_for_rwp();
> +
> + for (i = 0; i < gicv3_data.irq_nr; i += 4)
> + writel(~0, dist + GICD_IGROUPR + i);
> +
> + if (!gicv3_redist_base())
> + gicv3_set_redist_base(SZ_64K * 2);
> + sgi_base = gicv3_sgi_base();
> +
> + writel(~0, sgi_base + GICR_IGROUPR0);
> +
> + for (i = 0; i < 16; i += 4)
> + writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
> +
> + writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
> +
> + gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
> + gicv3_write_grpen1(1);
> +}
> diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..6d353567f56a
> --- /dev/null
> +++ b/lib/arm64/asm/arch_gicv3.h
> @@ -0,0 +1,44 @@
> +/*
> + * All ripped off from arch/arm64/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_ARCH_GICV3_H_
> +#define _ASMARM64_ARCH_GICV3_H_
> +
> +#include <asm/sysreg.h>
> +
> +#define ICC_PMR_EL1 sys_reg(3, 0, 4, 6, 0)
> +#define ICC_GRPEN1_EL1 sys_reg(3, 0, 12, 12, 7)
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +
> +#define __stringify xstr
> +
> +/*
> + * Low-level accessors
> + *
> + * These system registers are 32 bits, but we make sure that the compiler
> + * sets the GP register's most significant bits to 0 with an explicit cast.
> + */
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> + asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> + asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
> + isb();
> +}
> +
> +#define gicv3_read_typer(c) readq(c)
> +
> +#endif /* __ASSEMBLY__ */
> +#endif /* _ASMARM64_ARCH_GICV3_H_ */
> diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
> new file mode 100644
> index 000000000000..8ee5d4d9c181
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3.h"
> diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
> new file mode 100644
> index 000000000000..544a46cb8cc5
> --- /dev/null
> +++ b/lib/arm64/asm/sysreg.h
> @@ -0,0 +1,44 @@
> +/*
> + * Ripped off from arch/arm64/include/asm/sysreg.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_SYSREG_H_
> +#define _ASMARM64_SYSREG_H_
> +
> +#define sys_reg(op0, op1, crn, crm, op2) \
> + ((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
> +
> +#ifdef __ASSEMBLY__
> + .irp num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
> + .equ .L__reg_num_x\num, \num
> + .endr
> + .equ .L__reg_num_xzr, 31
> +
> + .macro mrs_s, rt, sreg
> + .inst 0xd5200000|(\sreg)|(.L__reg_num_\rt)
> + .endm
> +
> + .macro msr_s, sreg, rt
> + .inst 0xd5000000|(\sreg)|(.L__reg_num_\rt)
> + .endm
> +#else
> +asm(
> +" .irp num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
> +" .equ .L__reg_num_x\\num, \\num\n"
> +" .endr\n"
> +" .equ .L__reg_num_xzr, 31\n"
> +"\n"
> +" .macro mrs_s, rt, sreg\n"
> +" .inst 0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +" .endm\n"
> +"\n"
> +" .macro msr_s, sreg, rt\n"
> +" .inst 0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +" .endm\n"
> +);
> +#endif
> +
> +#endif /* _ASMARM64_SYSREG_H_ */
>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Eric
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [kvm-unit-tests PATCH v6 03/11] arm/arm64: smp: support more than 8 cpus
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-23 10:38 ` Auger Eric
-1 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 10:38 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: marc.zyngier, andre.przywara, pbonzini
Hi Drew,
On 14/11/2016 22:08, Andrew Jones wrote:
> By adding support for launching with gicv3 we can break the 8 vcpu
> limit. This patch adds support to smp code and also selects the
> vgic model corresponding to the host. The vgic model may also be
> manually selected by adding e.g. -machine gic-version=3 to
> extra_params.
>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v5: left cpus a u32 for now. Changing to u64 requires a change to
> devicetree. Will do it later. [Andre]
> v4: improved commit message
> ---
> arm/run | 19 ++++++++++++-------
> arm/selftest.c | 5 ++++-
> lib/arm/asm/processor.h | 9 +++++++--
> lib/arm/asm/setup.h | 4 ++--
> lib/arm/setup.c | 10 ++++++++++
> lib/arm64/asm/processor.h | 9 +++++++--
> 6 files changed, 42 insertions(+), 14 deletions(-)
>
> diff --git a/arm/run b/arm/run
> index 1ee6231599d6..a35952b28b46 100755
> --- a/arm/run
> +++ b/arm/run
> @@ -31,13 +31,6 @@ if [ -z "$ACCEL" ]; then
> fi
> fi
>
> -if [ "$HOST" = "aarch64" ] && [ "$ACCEL" = "kvm" ]; then
> - processor="host"
> - if [ "$ARCH" = "arm" ]; then
> - processor+=",aarch64=off"
> - fi
> -fi
> -
> qemu="${QEMU:-qemu-system-$ARCH_NAME}"
> qpath=$(which $qemu 2>/dev/null)
>
> @@ -53,6 +46,18 @@ fi
>
> M='-machine virt'
>
> +if [ "$ACCEL" = "kvm" ]; then
> + if $qemu $M,\? 2>&1 | grep gic-version > /dev/null; then
> + M+=',gic-version=host'
> + fi
> + if [ "$HOST" = "aarch64" ]; then
> + processor="host"
> + if [ "$ARCH" = "arm" ]; then
> + processor+=",aarch64=off"
> + fi
> + fi
> +fi
> +
> if ! $qemu $M -device '?' 2>&1 | grep virtconsole > /dev/null; then
> echo "$qpath doesn't support virtio-console for chr-testdev. Exiting."
> exit 2
> diff --git a/arm/selftest.c b/arm/selftest.c
> index 196164f5313d..2f117f795d2d 100644
> --- a/arm/selftest.c
> +++ b/arm/selftest.c
> @@ -312,9 +312,10 @@ static bool psci_check(void)
> static cpumask_t smp_reported;
> static void cpu_report(void)
> {
> + unsigned long mpidr = get_mpidr();
> int cpu = smp_processor_id();
>
> - report("CPU%d online", true, cpu);
> + report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == cpu, cpu, mpidr);
> cpumask_set_cpu(cpu, &smp_reported);
> halt();
> }
> @@ -343,6 +344,7 @@ int main(int argc, char **argv)
>
> } else if (strcmp(argv[1], "smp") == 0) {
>
> + unsigned long mpidr = get_mpidr();
> int cpu;
>
> report("PSCI version", psci_check());
> @@ -353,6 +355,7 @@ int main(int argc, char **argv)
> smp_boot_secondary(cpu, cpu_report);
> }
>
> + report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == 0, 0, mpidr);
> cpumask_set_cpu(0, &smp_reported);
> while (!cpumask_full(&smp_reported))
> cpu_relax();
> diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
> index 02f912f99974..ecf5bbe1824a 100644
> --- a/lib/arm/asm/processor.h
> +++ b/lib/arm/asm/processor.h
> @@ -40,8 +40,13 @@ static inline unsigned long get_mpidr(void)
> return mpidr;
> }
>
> -/* Only support Aff0 for now, up to 4 cpus */
> -#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
> +#define MPIDR_HWID_BITMASK 0xffffff
> +extern int mpidr_to_cpu(unsigned long mpidr);
> +
> +#define MPIDR_LEVEL_SHIFT(level) \
> + (((1 << level) >> 1) << 3)
> +#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> + ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
>
> extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
> extern bool is_user(void);
> diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
> index cb8fdbd38dd5..1de99dd184d1 100644
> --- a/lib/arm/asm/setup.h
> +++ b/lib/arm/asm/setup.h
> @@ -10,8 +10,8 @@
> #include <asm/page.h>
> #include <asm/pgtable-hwdef.h>
>
> -#define NR_CPUS 8
> -extern u32 cpus[NR_CPUS];
> +#define NR_CPUS 255
> +extern u32 cpus[NR_CPUS]; /* per-cpu IDs (MPIDRs) */
> extern int nr_cpus;
>
> #define NR_MEM_REGIONS 8
> diff --git a/lib/arm/setup.c b/lib/arm/setup.c
> index 7e7b39f11dde..241bf9410447 100644
> --- a/lib/arm/setup.c
> +++ b/lib/arm/setup.c
> @@ -30,6 +30,16 @@ int nr_cpus;
> struct mem_region mem_regions[NR_MEM_REGIONS];
> phys_addr_t __phys_offset, __phys_end;
>
> +int mpidr_to_cpu(unsigned long mpidr)
> +{
> + int i;
> +
> + for (i = 0; i < nr_cpus; ++i)
> + if (cpus[i] == (mpidr & MPIDR_HWID_BITMASK))
> + return i;
> + return -1;
> +}
> +
> static void cpu_set(int fdtnode __unused, u32 regval, void *info __unused)
> {
> int cpu = nr_cpus++;
> diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
> index 9a208ff729b7..7e448dc81a6a 100644
> --- a/lib/arm64/asm/processor.h
> +++ b/lib/arm64/asm/processor.h
> @@ -78,8 +78,13 @@ static inline type get_##reg(void) \
>
> DEFINE_GET_SYSREG64(mpidr)
>
> -/* Only support Aff0 for now, gicv2 only */
> -#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
> +#define MPIDR_HWID_BITMASK 0xff00ffffff
> +extern int mpidr_to_cpu(unsigned long mpidr);
> +
> +#define MPIDR_LEVEL_SHIFT(level) \
> + (((1 << level) >> 1) << 3)
> +#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> + ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
>
> extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
> extern bool is_user(void);
>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Thanks
Eric
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 03/11] arm/arm64: smp: support more than 8 cpus
@ 2016-11-23 10:38 ` Auger Eric
0 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 10:38 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, christoffer.dall
Hi Drew,
On 14/11/2016 22:08, Andrew Jones wrote:
> By adding support for launching with gicv3 we can break the 8 vcpu
> limit. This patch adds support to smp code and also selects the
> vgic model corresponding to the host. The vgic model may also be
> manually selected by adding e.g. -machine gic-version=3 to
> extra_params.
>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v5: left cpus a u32 for now. Changing to u64 requires a change to
> devicetree. Will do it later. [Andre]
> v4: improved commit message
> ---
> arm/run | 19 ++++++++++++-------
> arm/selftest.c | 5 ++++-
> lib/arm/asm/processor.h | 9 +++++++--
> lib/arm/asm/setup.h | 4 ++--
> lib/arm/setup.c | 10 ++++++++++
> lib/arm64/asm/processor.h | 9 +++++++--
> 6 files changed, 42 insertions(+), 14 deletions(-)
>
> diff --git a/arm/run b/arm/run
> index 1ee6231599d6..a35952b28b46 100755
> --- a/arm/run
> +++ b/arm/run
> @@ -31,13 +31,6 @@ if [ -z "$ACCEL" ]; then
> fi
> fi
>
> -if [ "$HOST" = "aarch64" ] && [ "$ACCEL" = "kvm" ]; then
> - processor="host"
> - if [ "$ARCH" = "arm" ]; then
> - processor+=",aarch64=off"
> - fi
> -fi
> -
> qemu="${QEMU:-qemu-system-$ARCH_NAME}"
> qpath=$(which $qemu 2>/dev/null)
>
> @@ -53,6 +46,18 @@ fi
>
> M='-machine virt'
>
> +if [ "$ACCEL" = "kvm" ]; then
> + if $qemu $M,\? 2>&1 | grep gic-version > /dev/null; then
> + M+=',gic-version=host'
> + fi
> + if [ "$HOST" = "aarch64" ]; then
> + processor="host"
> + if [ "$ARCH" = "arm" ]; then
> + processor+=",aarch64=off"
> + fi
> + fi
> +fi
> +
> if ! $qemu $M -device '?' 2>&1 | grep virtconsole > /dev/null; then
> echo "$qpath doesn't support virtio-console for chr-testdev. Exiting."
> exit 2
> diff --git a/arm/selftest.c b/arm/selftest.c
> index 196164f5313d..2f117f795d2d 100644
> --- a/arm/selftest.c
> +++ b/arm/selftest.c
> @@ -312,9 +312,10 @@ static bool psci_check(void)
> static cpumask_t smp_reported;
> static void cpu_report(void)
> {
> + unsigned long mpidr = get_mpidr();
> int cpu = smp_processor_id();
>
> - report("CPU%d online", true, cpu);
> + report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == cpu, cpu, mpidr);
> cpumask_set_cpu(cpu, &smp_reported);
> halt();
> }
> @@ -343,6 +344,7 @@ int main(int argc, char **argv)
>
> } else if (strcmp(argv[1], "smp") == 0) {
>
> + unsigned long mpidr = get_mpidr();
> int cpu;
>
> report("PSCI version", psci_check());
> @@ -353,6 +355,7 @@ int main(int argc, char **argv)
> smp_boot_secondary(cpu, cpu_report);
> }
>
> + report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == 0, 0, mpidr);
> cpumask_set_cpu(0, &smp_reported);
> while (!cpumask_full(&smp_reported))
> cpu_relax();
> diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
> index 02f912f99974..ecf5bbe1824a 100644
> --- a/lib/arm/asm/processor.h
> +++ b/lib/arm/asm/processor.h
> @@ -40,8 +40,13 @@ static inline unsigned long get_mpidr(void)
> return mpidr;
> }
>
> -/* Only support Aff0 for now, up to 4 cpus */
> -#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
> +#define MPIDR_HWID_BITMASK 0xffffff
> +extern int mpidr_to_cpu(unsigned long mpidr);
> +
> +#define MPIDR_LEVEL_SHIFT(level) \
> + (((1 << level) >> 1) << 3)
> +#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> + ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
>
> extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
> extern bool is_user(void);
> diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
> index cb8fdbd38dd5..1de99dd184d1 100644
> --- a/lib/arm/asm/setup.h
> +++ b/lib/arm/asm/setup.h
> @@ -10,8 +10,8 @@
> #include <asm/page.h>
> #include <asm/pgtable-hwdef.h>
>
> -#define NR_CPUS 8
> -extern u32 cpus[NR_CPUS];
> +#define NR_CPUS 255
> +extern u32 cpus[NR_CPUS]; /* per-cpu IDs (MPIDRs) */
> extern int nr_cpus;
>
> #define NR_MEM_REGIONS 8
> diff --git a/lib/arm/setup.c b/lib/arm/setup.c
> index 7e7b39f11dde..241bf9410447 100644
> --- a/lib/arm/setup.c
> +++ b/lib/arm/setup.c
> @@ -30,6 +30,16 @@ int nr_cpus;
> struct mem_region mem_regions[NR_MEM_REGIONS];
> phys_addr_t __phys_offset, __phys_end;
>
> +int mpidr_to_cpu(unsigned long mpidr)
> +{
> + int i;
> +
> + for (i = 0; i < nr_cpus; ++i)
> + if (cpus[i] == (mpidr & MPIDR_HWID_BITMASK))
> + return i;
> + return -1;
> +}
> +
> static void cpu_set(int fdtnode __unused, u32 regval, void *info __unused)
> {
> int cpu = nr_cpus++;
> diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
> index 9a208ff729b7..7e448dc81a6a 100644
> --- a/lib/arm64/asm/processor.h
> +++ b/lib/arm64/asm/processor.h
> @@ -78,8 +78,13 @@ static inline type get_##reg(void) \
>
> DEFINE_GET_SYSREG64(mpidr)
>
> -/* Only support Aff0 for now, gicv2 only */
> -#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
> +#define MPIDR_HWID_BITMASK 0xff00ffffff
> +extern int mpidr_to_cpu(unsigned long mpidr);
> +
> +#define MPIDR_LEVEL_SHIFT(level) \
> + (((1 << level) >> 1) << 3)
> +#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> + ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
>
> extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
> extern bool is_user(void);
>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Thanks
Eric
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [kvm-unit-tests PATCH v6 10/11] arm/arm64: gicv3: add an IPI test
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-23 11:05 ` Auger Eric
-1 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 11:05 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: marc.zyngier, andre.przywara, pbonzini
Hi Drew,
On 14/11/2016 22:08, Andrew Jones wrote:
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v6: move most gicv2/gicv3 wrappers to common code [Alex]
> v5:
> - fix copy+paste error in gicv3_write_eoir [drew]
> - use modern register names [Andre]
> v4:
> - heavily comment gicv3_ipi_send_tlist() [Eric]
> - changes needed for gicv2 iar/irqstat fix to other patch
> v2:
> - use IRM for gicv3 broadcast
> ---
> arm/gic.c | 97 +++++++++++++++++++++++++-----
> arm/unittests.cfg | 6 ++
> lib/arm/asm/arch_gicv3.h | 23 +++++++
> lib/arm/asm/gic-v3.h | 10 +++-
> lib/arm/asm/gic.h | 60 +++++++++++++++++++
> lib/arm/gic.c | 145 ++++++++++++++++++++++++++++++++++++++++++---
> lib/arm64/asm/arch_gicv3.h | 22 +++++++
> 7 files changed, 338 insertions(+), 25 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index b42c2b1ca1e1..d954a3775c26 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -3,6 +3,8 @@
> *
> * GICv2
> * + test sending/receiving IPIs
> + * GICv3
> + * + test sending/receiving IPIs
> *
> * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> *
> @@ -16,7 +18,15 @@
> #include <asm/barrier.h>
> #include <asm/io.h>
>
> -static int gic_version;
> +struct gic {
> + struct {
> + void (*send_self)(void);
> + void (*send_tlist)(cpumask_t *mask, int irq);
> + void (*send_broadcast)(void);
> + } ipi;
> +};
what is the rationale behind not putting this into common_ops?
> +
> +static struct gic *gic;
> static int acked[NR_CPUS], spurious[NR_CPUS];
> static cpumask_t ready;
>
> @@ -83,11 +93,11 @@ static void check_spurious(void)
>
> static void ipi_handler(struct pt_regs *regs __unused)
> {
> - u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
> - u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
> + u32 irqstat = gic_read_iar();
> + u32 irqnr = gic_iar_irqnr(irqstat);
>
> if (irqnr != GICC_INT_SPURIOUS) {
> - writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
> + gic_write_eoir(irqstat);
> smp_rmb(); /* pairs with wmb in ipi_test functions */
> ++acked[smp_processor_id()];
> smp_wmb(); /* pairs with rmb in check_acked */
> @@ -97,6 +107,38 @@ static void ipi_handler(struct pt_regs *regs __unused)
> }
> }
>
> +static void gicv2_ipi_send_self(void)
> +{
> + writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
> +{
> + u8 tlist = (u8)cpumask_bits(mask)[0];
> +
> + writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +static void gicv2_ipi_send_broadcast(void)
> +{
> + writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +static void gicv3_ipi_send_self(void)
> +{
> + cpumask_t mask;
> +
> + cpumask_clear(&mask);
> + cpumask_set_cpu(smp_processor_id(), &mask);
> + gicv3_ipi_send_tlist(&mask, 0);
> +}
> +
> +static void gicv3_ipi_send_broadcast(void)
> +{
> + gicv3_write_sgi1r(1ULL << 40);
> + isb();
> +}
> +
> static void ipi_test_self(void)
> {
> cpumask_t mask;
> @@ -106,7 +148,7 @@ static void ipi_test_self(void)
> smp_wmb();
> cpumask_clear(&mask);
> cpumask_set_cpu(0, &mask);
> - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> + gic->ipi.send_self();
> check_acked(&mask);
> report_prefix_pop();
> }
> @@ -114,14 +156,15 @@ static void ipi_test_self(void)
> static void ipi_test_smp(void)
> {
> cpumask_t mask;
> - unsigned long tlist;
> + int i;
>
> report_prefix_push("target-list");
> memset(acked, 0, sizeof(acked));
> smp_wmb();
> - tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
> - cpumask_bits(&mask)[0] = tlist;
> - writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
> + cpumask_copy(&mask, &cpu_present_mask);
> + for (i = 0; i < nr_cpus; i += 2)
> + cpumask_clear_cpu(i, &mask);
> + gic->ipi.send_tlist(&mask, 0);
> check_acked(&mask);
> report_prefix_pop();
>
> @@ -130,14 +173,14 @@ static void ipi_test_smp(void)
> smp_wmb();
> cpumask_copy(&mask, &cpu_present_mask);
> cpumask_clear_cpu(0, &mask);
> - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> + gic->ipi.send_broadcast();
> check_acked(&mask);
> report_prefix_pop();
> }
>
> static void ipi_enable(void)
> {
> - gicv2_enable_defaults();
> + gic_enable_defaults();
> #ifdef __arm__
> install_exception_handler(EXCPTN_IRQ, ipi_handler);
> #else
> @@ -154,18 +197,42 @@ static void ipi_recv(void)
> wfi();
> }
>
> +static struct gic gicv2 = {
> + .ipi = {
> + .send_self = gicv2_ipi_send_self,
> + .send_tlist = gicv2_ipi_send_tlist,
> + .send_broadcast = gicv2_ipi_send_broadcast,
> + },
> +};
> +
> +static struct gic gicv3 = {
> + .ipi = {
> + .send_self = gicv3_ipi_send_self,
> + .send_tlist = gicv3_ipi_send_tlist,
> + .send_broadcast = gicv3_ipi_send_broadcast,
> + },
> +};
> +
> int main(int argc, char **argv)
> {
> char pfx[8];
> int cpu;
>
> - gic_version = gic_init();
> - if (!gic_version)
> - report_abort("No gic present!");
> + if (!gic_init())
> + report_abort("No supported gic present!");
>
> - snprintf(pfx, sizeof(pfx), "gicv%d", gic_version);
> + snprintf(pfx, sizeof(pfx), "gicv%d", gic_version());
> report_prefix_push(pfx);
>
> + switch (gic_version()) {
> + case 2:
> + gic = &gicv2;
> + break;
> + case 3:
> + gic = &gicv3;
> + break;
> + }
> +
> if (argc < 2) {
>
> report_prefix_push("ipi");
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index e631c35e2bbb..c7392c747f98 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -66,3 +66,9 @@ file = gic.flat
> smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
> extra_params = -machine gic-version=2 -append 'ipi'
> groups = gic
> +
> +[gicv3-ipi]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'ipi'
> +groups = gic
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> index 276577452a14..b47cd2e0090b 100644
> --- a/lib/arm/asm/arch_gicv3.h
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -16,10 +16,28 @@
> #define __stringify xstr
>
> #define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
> +#define __ACCESS_CP15_64(Op1, CRm) p15, Op1, %Q0, %R0, CRm
>
> +#define ICC_EOIR1 __ACCESS_CP15(c12, 0, c12, 1)
> +#define ICC_IAR1 __ACCESS_CP15(c12, 0, c12, 0)
> +#define ICC_SGI1R __ACCESS_CP15_64(0, c12)
> #define ICC_PMR __ACCESS_CP15(c4, 0, c6, 0)
> #define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
>
> +static inline void gicv3_write_eoir(u32 irq)
> +{
> + asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> + isb();
> +}
> +
> +static inline u32 gicv3_read_iar(void)
> +{
> + u32 irqstat;
> + asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
> + dsb(sy);
> + return irqstat;
> +}
> +
> static inline void gicv3_write_pmr(u32 val)
> {
> asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> @@ -31,6 +49,11 @@ static inline void gicv3_write_grpen1(u32 val)
> isb();
> }
>
> +static inline void gicv3_write_sgi1r(u64 val)
> +{
> + asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> +}
> +
> /*
> * We may access GICR_TYPER and GITS_TYPER by reading both the TYPER
> * offset and the following offset (+ 4) and then combining them to
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index 73ade4681d21..43f9ffce56de 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -33,12 +33,19 @@
> #define GICR_ISENABLER0 GICD_ISENABLER
> #define GICR_IPRIORITYR0 GICD_IPRIORITYR
>
> +#define ICC_SGI1R_AFFINITY_1_SHIFT 16
> +#define ICC_SGI1R_AFFINITY_2_SHIFT 32
> +#define ICC_SGI1R_AFFINITY_3_SHIFT 48
> +#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> + (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
> +
> #include <asm/arch_gicv3.h>
>
> #ifndef __ASSEMBLY__
> #include <asm/setup.h>
> -#include <asm/smp.h>
> #include <asm/processor.h>
> +#include <asm/cpumask.h>
> +#include <asm/smp.h>
> #include <asm/io.h>
>
> struct gicv3_data {
> @@ -55,6 +62,7 @@ extern struct gicv3_data gicv3_data;
> extern int gicv3_init(void);
> extern void gicv3_enable_defaults(void);
> extern void gicv3_set_redist_base(size_t stride);
> +extern void gicv3_ipi_send_tlist(cpumask_t *mask, int irq);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> index 21511997f2a9..c2267b6b3937 100644
> --- a/lib/arm/asm/gic.h
> +++ b/lib/arm/asm/gic.h
> @@ -42,5 +42,65 @@
> */
> extern int gic_init(void);
>
> +/*
> + * gic_common_ops collects some functions that we provide unit
> + * tests that don't care which gic version they're using.
nit: wording?
> + */
> +struct gic_common_ops {
> + int gic_version;
> + u32 (*read_iar)(void);
> + u32 (*iar_irqnr)(u32 iar);
> + void (*write_eoir)(u32 irqstat);
> + void (*ipi_send)(int cpu, int irq);
what is the rationale behind not putting enable_defaults here?
lib/arm/gic.c becomes bigger and bigger. Woudn't it make sense to have
separate lib/arm/gic-v2/v3.c and arm/gic.c only call generic ops?
Typically ipi functions could go in lib
But nethertheless this can be changed later.
Thanks
Eric
> +};
> +
> +extern struct gic_common_ops *gic_common_ops;
> +
> +static inline int gic_version(void)
> +{
> + assert(gic_common_ops);
> + return gic_common_ops->gic_version;
> +}
> +
> +static inline u32 gic_read_iar(void)
> +{
> + assert(gic_common_ops && gic_common_ops->read_iar);
> + return gic_common_ops->read_iar();
> +}
> +
> +static inline u32 gic_iar_irqnr(u32 iar)
> +{
> + assert(gic_common_ops && gic_common_ops->iar_irqnr);
> + return gic_common_ops->iar_irqnr(iar);
> +}
> +
> +static inline void gic_write_eoir(u32 irqstat)
> +{
> + assert(gic_common_ops && gic_common_ops->write_eoir);
> + gic_common_ops->write_eoir(irqstat);
> +}
> +
> +static inline void gic_ipi_send(int cpu, int irq)
> +{
> + assert(gic_common_ops && gic_common_ops->ipi_send);
> + gic_common_ops->ipi_send(cpu, irq);
> +}
> +
> +static inline void gic_enable_defaults(void)
> +{
> + switch (gic_version()) {
> + case 2:
> + gicv2_enable_defaults();
> + break;
> + case 3:
> + gicv3_enable_defaults();
> + break;
> + default:
> + printf("%s: Unknown gic version %d\n", __func__,
> + gic_version());
> + abort();
> + }
> +}
> +
> #endif /* !__ASSEMBLY__ */
> #endif /* _ASMARM_GIC_H_ */
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index d703ad96a37e..4f67363e073b 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -56,15 +56,6 @@ int gicv3_init(void)
> &gicv3_data.redist_base[0]);
> }
>
> -int gic_init(void)
> -{
> - if (gicv2_init())
> - return 2;
> - else if (gicv3_init())
> - return 3;
> - return 0;
> -}
> -
> void gicv2_enable_defaults(void)
> {
> void *dist = gicv2_dist_base();
> @@ -85,6 +76,28 @@ void gicv2_enable_defaults(void)
> writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> }
>
> +static u32 gicv2_read_iar(void)
> +{
> + return readl(gicv2_cpu_base() + GICC_IAR);
> +}
> +
> +static u32 gicv2_iar_irqnr(u32 iar)
> +{
> + return iar & GICC_IAR_INT_ID_MASK;
> +}
> +
> +static void gicv2_write_eoir(u32 irqstat)
> +{
> + writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
> +}
> +
> +static void gicv2_ipi_send(int cpu, int irq)
> +{
> + assert(cpu < 8);
> + assert(irq < 16);
> + writel(1 << (cpu + 16) | irq, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> void gicv3_set_redist_base(size_t stride)
> {
> u32 aff = mpidr_compress(get_mpidr());
> @@ -138,3 +151,117 @@ void gicv3_enable_defaults(void)
> gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
> gicv3_write_grpen1(1);
> }
> +
> +static u32 gicv3_iar_irqnr(u32 iar)
> +{
> + return iar;
> +}
> +
> +void gicv3_ipi_send_tlist(cpumask_t *mask, int irq)
> +{
> + u16 tlist;
> + int cpu;
> +
> + assert(irq < 16);
> +
> + /*
> + * For each cpu in the mask collect its peers, which are also in
> + * the mask, in order to form target lists.
> + */
> + for_each_cpu(cpu, mask) {
> + u64 mpidr = cpus[cpu], sgi1r;
> + u64 cluster_id;
> +
> + /*
> + * GICv3 can send IPIs to up 16 peer cpus with a single
> + * write to ICC_SGI1R_EL1 (using the target list). Peers
> + * are cpus that have nearly identical MPIDRs, the only
> + * difference being Aff0. The matching upper affinity
> + * levels form the cluster ID.
> + */
> + cluster_id = mpidr & ~0xffUL;
> + tlist = 0;
> +
> + /*
> + * Sort of open code for_each_cpu in order to have a
> + * nested for_each_cpu loop.
> + */
> + while (cpu < nr_cpus) {
> + if ((mpidr & 0xff) >= 16) {
> + printf("cpu%d MPIDR:aff0 is %d (>= 16)!\n",
> + cpu, (int)(mpidr & 0xff));
> + break;
> + }
> +
> + tlist |= 1 << (mpidr & 0xf);
> +
> + cpu = cpumask_next(cpu, mask);
> + if (cpu >= nr_cpus)
> + break;
> +
> + mpidr = cpus[cpu];
> +
> + if (cluster_id != (mpidr & ~0xffUL)) {
> + /*
> + * The next cpu isn't in our cluster. Roll
> + * back the cpu index allowing the outer
> + * for_each_cpu to find it again with
> + * cpumask_next
> + */
> + --cpu;
> + break;
> + }
> + }
> +
> + /* Send the IPIs for the target list of this cluster */
> + sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3) |
> + MPIDR_TO_SGI_AFFINITY(cluster_id, 2) |
> + irq << 24 |
> + MPIDR_TO_SGI_AFFINITY(cluster_id, 1) |
> + tlist);
> +
> + gicv3_write_sgi1r(sgi1r);
> + }
> +
> + /* Force the above writes to ICC_SGI1R_EL1 to be executed */
> + isb();
> +}
> +
> +static void gicv3_ipi_send(int cpu, int irq)
> +{
> + cpumask_t mask;
> +
> + cpumask_clear(&mask);
> + cpumask_set_cpu(cpu, &mask);
> + gicv3_ipi_send_tlist(&mask, irq);
> +}
> +
> +static struct gic_common_ops gicv2_common_ops = {
> + .gic_version = 2,
> + .read_iar = gicv2_read_iar,
> + .iar_irqnr = gicv2_iar_irqnr,
> + .write_eoir = gicv2_write_eoir,
> + .ipi_send = gicv2_ipi_send,
> +};
> +
> +static struct gic_common_ops gicv3_common_ops = {
> + .gic_version = 3,
> + .read_iar = gicv3_read_iar,
> + .iar_irqnr = gicv3_iar_irqnr,
> + .write_eoir = gicv3_write_eoir,
> + .ipi_send = gicv3_ipi_send,
> +};
> +
> +struct gic_common_ops *gic_common_ops;
> +
> +int gic_init(void)
> +{
> + if (gicv2_init()) {
> + gic_common_ops = &gicv2_common_ops;
> + return 2;
> + } else if (gicv3_init()) {
> + gic_common_ops = &gicv3_common_ops;
> + return 3;
> + }
> + return 0;
> +}
> diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> index 6d353567f56a..874775828016 100644
> --- a/lib/arm64/asm/arch_gicv3.h
> +++ b/lib/arm64/asm/arch_gicv3.h
> @@ -10,6 +10,9 @@
>
> #include <asm/sysreg.h>
>
> +#define ICC_EOIR1_EL1 sys_reg(3, 0, 12, 12, 1)
> +#define ICC_IAR1_EL1 sys_reg(3, 0, 12, 12, 0)
> +#define ICC_SGI1R_EL1 sys_reg(3, 0, 12, 11, 5)
> #define ICC_PMR_EL1 sys_reg(3, 0, 4, 6, 0)
> #define ICC_GRPEN1_EL1 sys_reg(3, 0, 12, 12, 7)
>
> @@ -27,6 +30,20 @@
> * sets the GP register's most significant bits to 0 with an explicit cast.
> */
>
> +static inline void gicv3_write_eoir(u32 irq)
> +{
> + asm volatile("msr_s " __stringify(ICC_EOIR1_EL1) ", %0" : : "r" ((u64)irq));
> + isb();
> +}
> +
> +static inline u32 gicv3_read_iar(void)
> +{
> + u64 irqstat;
> + asm volatile("mrs_s %0, " __stringify(ICC_IAR1_EL1) : "=r" (irqstat));
> + dsb(sy);
> + return (u64)irqstat;
> +}
> +
> static inline void gicv3_write_pmr(u32 val)
> {
> asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> @@ -38,6 +55,11 @@ static inline void gicv3_write_grpen1(u32 val)
> isb();
> }
>
> +static inline void gicv3_write_sgi1r(u64 val)
> +{
> + asm volatile("msr_s " __stringify(ICC_SGI1R_EL1) ", %0" : : "r" (val));
> +}
> +
> #define gicv3_read_typer(c) readq(c)
>
> #endif /* __ASSEMBLY__ */
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-23 11:05 ` Auger Eric
0 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 11:05 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, christoffer.dall
Hi Drew,
On 14/11/2016 22:08, Andrew Jones wrote:
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v6: move most gicv2/gicv3 wrappers to common code [Alex]
> v5:
> - fix copy+paste error in gicv3_write_eoir [drew]
> - use modern register names [Andre]
> v4:
> - heavily comment gicv3_ipi_send_tlist() [Eric]
> - changes needed for gicv2 iar/irqstat fix to other patch
> v2:
> - use IRM for gicv3 broadcast
> ---
> arm/gic.c | 97 +++++++++++++++++++++++++-----
> arm/unittests.cfg | 6 ++
> lib/arm/asm/arch_gicv3.h | 23 +++++++
> lib/arm/asm/gic-v3.h | 10 +++-
> lib/arm/asm/gic.h | 60 +++++++++++++++++++
> lib/arm/gic.c | 145 ++++++++++++++++++++++++++++++++++++++++++---
> lib/arm64/asm/arch_gicv3.h | 22 +++++++
> 7 files changed, 338 insertions(+), 25 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index b42c2b1ca1e1..d954a3775c26 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -3,6 +3,8 @@
> *
> * GICv2
> * + test sending/receiving IPIs
> + * GICv3
> + * + test sending/receiving IPIs
> *
> * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> *
> @@ -16,7 +18,15 @@
> #include <asm/barrier.h>
> #include <asm/io.h>
>
> -static int gic_version;
> +struct gic {
> + struct {
> + void (*send_self)(void);
> + void (*send_tlist)(cpumask_t *mask, int irq);
> + void (*send_broadcast)(void);
> + } ipi;
> +};
what is the rationale behind not putting this into common_ops?
> +
> +static struct gic *gic;
> static int acked[NR_CPUS], spurious[NR_CPUS];
> static cpumask_t ready;
>
> @@ -83,11 +93,11 @@ static void check_spurious(void)
>
> static void ipi_handler(struct pt_regs *regs __unused)
> {
> - u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
> - u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
> + u32 irqstat = gic_read_iar();
> + u32 irqnr = gic_iar_irqnr(irqstat);
>
> if (irqnr != GICC_INT_SPURIOUS) {
> - writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
> + gic_write_eoir(irqstat);
> smp_rmb(); /* pairs with wmb in ipi_test functions */
> ++acked[smp_processor_id()];
> smp_wmb(); /* pairs with rmb in check_acked */
> @@ -97,6 +107,38 @@ static void ipi_handler(struct pt_regs *regs __unused)
> }
> }
>
> +static void gicv2_ipi_send_self(void)
> +{
> + writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
> +{
> + u8 tlist = (u8)cpumask_bits(mask)[0];
> +
> + writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +static void gicv2_ipi_send_broadcast(void)
> +{
> + writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +static void gicv3_ipi_send_self(void)
> +{
> + cpumask_t mask;
> +
> + cpumask_clear(&mask);
> + cpumask_set_cpu(smp_processor_id(), &mask);
> + gicv3_ipi_send_tlist(&mask, 0);
> +}
> +
> +static void gicv3_ipi_send_broadcast(void)
> +{
> + gicv3_write_sgi1r(1ULL << 40);
> + isb();
> +}
> +
> static void ipi_test_self(void)
> {
> cpumask_t mask;
> @@ -106,7 +148,7 @@ static void ipi_test_self(void)
> smp_wmb();
> cpumask_clear(&mask);
> cpumask_set_cpu(0, &mask);
> - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> + gic->ipi.send_self();
> check_acked(&mask);
> report_prefix_pop();
> }
> @@ -114,14 +156,15 @@ static void ipi_test_self(void)
> static void ipi_test_smp(void)
> {
> cpumask_t mask;
> - unsigned long tlist;
> + int i;
>
> report_prefix_push("target-list");
> memset(acked, 0, sizeof(acked));
> smp_wmb();
> - tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
> - cpumask_bits(&mask)[0] = tlist;
> - writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
> + cpumask_copy(&mask, &cpu_present_mask);
> + for (i = 0; i < nr_cpus; i += 2)
> + cpumask_clear_cpu(i, &mask);
> + gic->ipi.send_tlist(&mask, 0);
> check_acked(&mask);
> report_prefix_pop();
>
> @@ -130,14 +173,14 @@ static void ipi_test_smp(void)
> smp_wmb();
> cpumask_copy(&mask, &cpu_present_mask);
> cpumask_clear_cpu(0, &mask);
> - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> + gic->ipi.send_broadcast();
> check_acked(&mask);
> report_prefix_pop();
> }
>
> static void ipi_enable(void)
> {
> - gicv2_enable_defaults();
> + gic_enable_defaults();
> #ifdef __arm__
> install_exception_handler(EXCPTN_IRQ, ipi_handler);
> #else
> @@ -154,18 +197,42 @@ static void ipi_recv(void)
> wfi();
> }
>
> +static struct gic gicv2 = {
> + .ipi = {
> + .send_self = gicv2_ipi_send_self,
> + .send_tlist = gicv2_ipi_send_tlist,
> + .send_broadcast = gicv2_ipi_send_broadcast,
> + },
> +};
> +
> +static struct gic gicv3 = {
> + .ipi = {
> + .send_self = gicv3_ipi_send_self,
> + .send_tlist = gicv3_ipi_send_tlist,
> + .send_broadcast = gicv3_ipi_send_broadcast,
> + },
> +};
> +
> int main(int argc, char **argv)
> {
> char pfx[8];
> int cpu;
>
> - gic_version = gic_init();
> - if (!gic_version)
> - report_abort("No gic present!");
> + if (!gic_init())
> + report_abort("No supported gic present!");
>
> - snprintf(pfx, sizeof(pfx), "gicv%d", gic_version);
> + snprintf(pfx, sizeof(pfx), "gicv%d", gic_version());
> report_prefix_push(pfx);
>
> + switch (gic_version()) {
> + case 2:
> + gic = &gicv2;
> + break;
> + case 3:
> + gic = &gicv3;
> + break;
> + }
> +
> if (argc < 2) {
>
> report_prefix_push("ipi");
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index e631c35e2bbb..c7392c747f98 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -66,3 +66,9 @@ file = gic.flat
> smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
> extra_params = -machine gic-version=2 -append 'ipi'
> groups = gic
> +
> +[gicv3-ipi]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'ipi'
> +groups = gic
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> index 276577452a14..b47cd2e0090b 100644
> --- a/lib/arm/asm/arch_gicv3.h
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -16,10 +16,28 @@
> #define __stringify xstr
>
> #define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
> +#define __ACCESS_CP15_64(Op1, CRm) p15, Op1, %Q0, %R0, CRm
>
> +#define ICC_EOIR1 __ACCESS_CP15(c12, 0, c12, 1)
> +#define ICC_IAR1 __ACCESS_CP15(c12, 0, c12, 0)
> +#define ICC_SGI1R __ACCESS_CP15_64(0, c12)
> #define ICC_PMR __ACCESS_CP15(c4, 0, c6, 0)
> #define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
>
> +static inline void gicv3_write_eoir(u32 irq)
> +{
> + asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> + isb();
> +}
> +
> +static inline u32 gicv3_read_iar(void)
> +{
> + u32 irqstat;
> + asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
> + dsb(sy);
> + return irqstat;
> +}
> +
> static inline void gicv3_write_pmr(u32 val)
> {
> asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> @@ -31,6 +49,11 @@ static inline void gicv3_write_grpen1(u32 val)
> isb();
> }
>
> +static inline void gicv3_write_sgi1r(u64 val)
> +{
> + asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> +}
> +
> /*
> * We may access GICR_TYPER and GITS_TYPER by reading both the TYPER
> * offset and the following offset (+ 4) and then combining them to
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index 73ade4681d21..43f9ffce56de 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -33,12 +33,19 @@
> #define GICR_ISENABLER0 GICD_ISENABLER
> #define GICR_IPRIORITYR0 GICD_IPRIORITYR
>
> +#define ICC_SGI1R_AFFINITY_1_SHIFT 16
> +#define ICC_SGI1R_AFFINITY_2_SHIFT 32
> +#define ICC_SGI1R_AFFINITY_3_SHIFT 48
> +#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> + (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
> +
> #include <asm/arch_gicv3.h>
>
> #ifndef __ASSEMBLY__
> #include <asm/setup.h>
> -#include <asm/smp.h>
> #include <asm/processor.h>
> +#include <asm/cpumask.h>
> +#include <asm/smp.h>
> #include <asm/io.h>
>
> struct gicv3_data {
> @@ -55,6 +62,7 @@ extern struct gicv3_data gicv3_data;
> extern int gicv3_init(void);
> extern void gicv3_enable_defaults(void);
> extern void gicv3_set_redist_base(size_t stride);
> +extern void gicv3_ipi_send_tlist(cpumask_t *mask, int irq);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> index 21511997f2a9..c2267b6b3937 100644
> --- a/lib/arm/asm/gic.h
> +++ b/lib/arm/asm/gic.h
> @@ -42,5 +42,65 @@
> */
> extern int gic_init(void);
>
> +/*
> + * gic_common_ops collects some functions that we provide unit
> + * tests that don't care which gic version they're using.
nit: wording?
> + */
> +struct gic_common_ops {
> + int gic_version;
> + u32 (*read_iar)(void);
> + u32 (*iar_irqnr)(u32 iar);
> + void (*write_eoir)(u32 irqstat);
> + void (*ipi_send)(int cpu, int irq);
what is the rationale behind not putting enable_defaults here?
lib/arm/gic.c becomes bigger and bigger. Woudn't it make sense to have
separate lib/arm/gic-v2/v3.c and arm/gic.c only call generic ops?
Typically ipi functions could go in lib
But nethertheless this can be changed later.
Thanks
Eric
> +};
> +
> +extern struct gic_common_ops *gic_common_ops;
> +
> +static inline int gic_version(void)
> +{
> + assert(gic_common_ops);
> + return gic_common_ops->gic_version;
> +}
> +
> +static inline u32 gic_read_iar(void)
> +{
> + assert(gic_common_ops && gic_common_ops->read_iar);
> + return gic_common_ops->read_iar();
> +}
> +
> +static inline u32 gic_iar_irqnr(u32 iar)
> +{
> + assert(gic_common_ops && gic_common_ops->iar_irqnr);
> + return gic_common_ops->iar_irqnr(iar);
> +}
> +
> +static inline void gic_write_eoir(u32 irqstat)
> +{
> + assert(gic_common_ops && gic_common_ops->write_eoir);
> + gic_common_ops->write_eoir(irqstat);
> +}
> +
> +static inline void gic_ipi_send(int cpu, int irq)
> +{
> + assert(gic_common_ops && gic_common_ops->ipi_send);
> + gic_common_ops->ipi_send(cpu, irq);
> +}
> +
> +static inline void gic_enable_defaults(void)
> +{
> + switch (gic_version()) {
> + case 2:
> + gicv2_enable_defaults();
> + break;
> + case 3:
> + gicv3_enable_defaults();
> + break;
> + default:
> + printf("%s: Unknown gic version %d\n", __func__,
> + gic_version());
> + abort();
> + }
> +}
> +
> #endif /* !__ASSEMBLY__ */
> #endif /* _ASMARM_GIC_H_ */
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index d703ad96a37e..4f67363e073b 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -56,15 +56,6 @@ int gicv3_init(void)
> &gicv3_data.redist_base[0]);
> }
>
> -int gic_init(void)
> -{
> - if (gicv2_init())
> - return 2;
> - else if (gicv3_init())
> - return 3;
> - return 0;
> -}
> -
> void gicv2_enable_defaults(void)
> {
> void *dist = gicv2_dist_base();
> @@ -85,6 +76,28 @@ void gicv2_enable_defaults(void)
> writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> }
>
> +static u32 gicv2_read_iar(void)
> +{
> + return readl(gicv2_cpu_base() + GICC_IAR);
> +}
> +
> +static u32 gicv2_iar_irqnr(u32 iar)
> +{
> + return iar & GICC_IAR_INT_ID_MASK;
> +}
> +
> +static void gicv2_write_eoir(u32 irqstat)
> +{
> + writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
> +}
> +
> +static void gicv2_ipi_send(int cpu, int irq)
> +{
> + assert(cpu < 8);
> + assert(irq < 16);
> + writel(1 << (cpu + 16) | irq, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> void gicv3_set_redist_base(size_t stride)
> {
> u32 aff = mpidr_compress(get_mpidr());
> @@ -138,3 +151,117 @@ void gicv3_enable_defaults(void)
> gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
> gicv3_write_grpen1(1);
> }
> +
> +static u32 gicv3_iar_irqnr(u32 iar)
> +{
> + return iar;
> +}
> +
> +void gicv3_ipi_send_tlist(cpumask_t *mask, int irq)
> +{
> + u16 tlist;
> + int cpu;
> +
> + assert(irq < 16);
> +
> + /*
> + * For each cpu in the mask collect its peers, which are also in
> + * the mask, in order to form target lists.
> + */
> + for_each_cpu(cpu, mask) {
> + u64 mpidr = cpus[cpu], sgi1r;
> + u64 cluster_id;
> +
> + /*
> + * GICv3 can send IPIs to up 16 peer cpus with a single
> + * write to ICC_SGI1R_EL1 (using the target list). Peers
> + * are cpus that have nearly identical MPIDRs, the only
> + * difference being Aff0. The matching upper affinity
> + * levels form the cluster ID.
> + */
> + cluster_id = mpidr & ~0xffUL;
> + tlist = 0;
> +
> + /*
> + * Sort of open code for_each_cpu in order to have a
> + * nested for_each_cpu loop.
> + */
> + while (cpu < nr_cpus) {
> + if ((mpidr & 0xff) >= 16) {
> + printf("cpu%d MPIDR:aff0 is %d (>= 16)!\n",
> + cpu, (int)(mpidr & 0xff));
> + break;
> + }
> +
> + tlist |= 1 << (mpidr & 0xf);
> +
> + cpu = cpumask_next(cpu, mask);
> + if (cpu >= nr_cpus)
> + break;
> +
> + mpidr = cpus[cpu];
> +
> + if (cluster_id != (mpidr & ~0xffUL)) {
> + /*
> + * The next cpu isn't in our cluster. Roll
> + * back the cpu index allowing the outer
> + * for_each_cpu to find it again with
> + * cpumask_next
> + */
> + --cpu;
> + break;
> + }
> + }
> +
> + /* Send the IPIs for the target list of this cluster */
> + sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3) |
> + MPIDR_TO_SGI_AFFINITY(cluster_id, 2) |
> + irq << 24 |
> + MPIDR_TO_SGI_AFFINITY(cluster_id, 1) |
> + tlist);
> +
> + gicv3_write_sgi1r(sgi1r);
> + }
> +
> + /* Force the above writes to ICC_SGI1R_EL1 to be executed */
> + isb();
> +}
> +
> +static void gicv3_ipi_send(int cpu, int irq)
> +{
> + cpumask_t mask;
> +
> + cpumask_clear(&mask);
> + cpumask_set_cpu(cpu, &mask);
> + gicv3_ipi_send_tlist(&mask, irq);
> +}
> +
> +static struct gic_common_ops gicv2_common_ops = {
> + .gic_version = 2,
> + .read_iar = gicv2_read_iar,
> + .iar_irqnr = gicv2_iar_irqnr,
> + .write_eoir = gicv2_write_eoir,
> + .ipi_send = gicv2_ipi_send,
> +};
> +
> +static struct gic_common_ops gicv3_common_ops = {
> + .gic_version = 3,
> + .read_iar = gicv3_read_iar,
> + .iar_irqnr = gicv3_iar_irqnr,
> + .write_eoir = gicv3_write_eoir,
> + .ipi_send = gicv3_ipi_send,
> +};
> +
> +struct gic_common_ops *gic_common_ops;
> +
> +int gic_init(void)
> +{
> + if (gicv2_init()) {
> + gic_common_ops = &gicv2_common_ops;
> + return 2;
> + } else if (gicv3_init()) {
> + gic_common_ops = &gicv3_common_ops;
> + return 3;
> + }
> + return 0;
> +}
> diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> index 6d353567f56a..874775828016 100644
> --- a/lib/arm64/asm/arch_gicv3.h
> +++ b/lib/arm64/asm/arch_gicv3.h
> @@ -10,6 +10,9 @@
>
> #include <asm/sysreg.h>
>
> +#define ICC_EOIR1_EL1 sys_reg(3, 0, 12, 12, 1)
> +#define ICC_IAR1_EL1 sys_reg(3, 0, 12, 12, 0)
> +#define ICC_SGI1R_EL1 sys_reg(3, 0, 12, 11, 5)
> #define ICC_PMR_EL1 sys_reg(3, 0, 4, 6, 0)
> #define ICC_GRPEN1_EL1 sys_reg(3, 0, 12, 12, 7)
>
> @@ -27,6 +30,20 @@
> * sets the GP register's most significant bits to 0 with an explicit cast.
> */
>
> +static inline void gicv3_write_eoir(u32 irq)
> +{
> + asm volatile("msr_s " __stringify(ICC_EOIR1_EL1) ", %0" : : "r" ((u64)irq));
> + isb();
> +}
> +
> +static inline u32 gicv3_read_iar(void)
> +{
> + u64 irqstat;
> + asm volatile("mrs_s %0, " __stringify(ICC_IAR1_EL1) : "=r" (irqstat));
> + dsb(sy);
> + return (u64)irqstat;
> +}
> +
> static inline void gicv3_write_pmr(u32 val)
> {
> asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> @@ -38,6 +55,11 @@ static inline void gicv3_write_grpen1(u32 val)
> isb();
> }
>
> +static inline void gicv3_write_sgi1r(u64 val)
> +{
> + asm volatile("msr_s " __stringify(ICC_SGI1R_EL1) ", %0" : : "r" (val));
> +}
> +
> #define gicv3_read_typer(c) readq(c)
>
> #endif /* __ASSEMBLY__ */
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
@ 2016-11-23 11:28 ` Auger Eric
-1 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 11:28 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, christoffer.dall
Hi,
On 14/11/2016 22:08, Andrew Jones wrote:
> Allow user to select who sends ipis and with which irq,
> rather than just always sending irq=0 from cpu0.
>From a user point of view is there a way to know the list of available
tests and their arg?
>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v6:
> - make sender/irq names more future-proof [drew]
> - sanity check inputs [drew]
> - introduce check_sender/irq and bad_sender/irq to more
> cleanly do checks [drew]
> - default sender and irq to 1, instead of still zero [drew]
> v4: improve structure and make sure spurious checking is
> done even when the sender isn't cpu0
> v2: actually check that the irq received was the irq sent,
> and (for gicv2) that the sender is the expected one.
> ---
> arm/gic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
> 1 file changed, 99 insertions(+), 25 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index d954a3775c26..638b8b140c96 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -11,6 +11,7 @@
> * This work is licensed under the terms of the GNU LGPL, version 2.
> */
> #include <libcflat.h>
> +#include <util.h>
> #include <asm/setup.h>
> #include <asm/processor.h>
> #include <asm/gic.h>
> @@ -28,6 +29,8 @@ struct gic {
>
> static struct gic *gic;
> static int acked[NR_CPUS], spurious[NR_CPUS];
> +static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
> +static int cmdl_sender = 1, cmdl_irq = 1;
> static cpumask_t ready;
>
> static void nr_cpu_check(int nr)
> @@ -43,10 +46,23 @@ static void wait_on_ready(void)
> cpu_relax();
> }
>
> +static void stats_reset(void)
> +{
> + int i;
> +
> + for (i = 0; i < nr_cpus; ++i) {
> + acked[i] = 0;
> + bad_sender[i] = -1;
> + bad_irq[i] = -1;
> + }
> + smp_wmb();
> +}
> +
> static void check_acked(cpumask_t *mask)
> {
> int missing = 0, extra = 0, unexpected = 0;
> int nr_pass, cpu, i;
> + bool bad = false;
>
> /* Wait up to 5s for all interrupts to be delivered */
> for (i = 0; i < 50; ++i) {
> @@ -56,9 +72,21 @@ static void check_acked(cpumask_t *mask)
> smp_rmb();
> nr_pass += cpumask_test_cpu(cpu, mask) ?
> acked[cpu] == 1 : acked[cpu] == 0;
> +
> + if (bad_sender[cpu] != -1) {
> + printf("cpu%d received IPI from wrong sender %d\n",
> + cpu, bad_sender[cpu]);
> + bad = true;
> + }
> +
> + if (bad_irq[cpu] != -1) {
> + printf("cpu%d received wrong irq %d\n",
> + cpu, bad_irq[cpu]);
> + bad = true;
> + }
> }
> if (nr_pass == nr_cpus) {
> - report("Completed in %d ms", true, ++i * 100);
> + report("Completed in %d ms", !bad, ++i * 100);
> return;
> }
> }
> @@ -91,6 +119,22 @@ static void check_spurious(void)
> }
> }
>
> +static void check_ipi_sender(u32 irqstat)
> +{
> + if (gic_version() == 2) {
> + int src = (irqstat >> 10) & 7;
> +
> + if (src != cmdl_sender)
> + bad_sender[smp_processor_id()] = src;
> + }
> +}
> +
> +static void check_irqnr(u32 irqnr)
> +{
> + if (irqnr != (u32)cmdl_irq)
> + bad_irq[smp_processor_id()] = irqnr;
> +}
> +
> static void ipi_handler(struct pt_regs *regs __unused)
> {
> u32 irqstat = gic_read_iar();
> @@ -98,8 +142,10 @@ static void ipi_handler(struct pt_regs *regs __unused)
>
> if (irqnr != GICC_INT_SPURIOUS) {
> gic_write_eoir(irqstat);
> - smp_rmb(); /* pairs with wmb in ipi_test functions */
> + smp_rmb(); /* pairs with wmb in stats_reset */
> ++acked[smp_processor_id()];
> + check_ipi_sender(irqstat);
> + check_irqnr(irqnr);
> smp_wmb(); /* pairs with rmb in check_acked */
> } else {
> ++spurious[smp_processor_id()];
> @@ -109,19 +155,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
>
> static void gicv2_ipi_send_self(void)
> {
> - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> + writel(2 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> }
>
> -static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
> +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq)
> {
> u8 tlist = (u8)cpumask_bits(mask)[0];
>
> - writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> + writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
> }
>
> static void gicv2_ipi_send_broadcast(void)
> {
> - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> + writel(1 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> }
>
> static void gicv3_ipi_send_self(void)
> @@ -130,12 +176,12 @@ static void gicv3_ipi_send_self(void)
>
> cpumask_clear(&mask);
> cpumask_set_cpu(smp_processor_id(), &mask);
> - gicv3_ipi_send_tlist(&mask, 0);
> + gicv3_ipi_send_tlist(&mask, cmdl_irq);
> }
>
> static void gicv3_ipi_send_broadcast(void)
> {
> - gicv3_write_sgi1r(1ULL << 40);
> + gicv3_write_sgi1r(1ULL << 40 | cmdl_irq << 24);
> isb();
> }
>
> @@ -144,10 +190,9 @@ static void ipi_test_self(void)
> cpumask_t mask;
>
> report_prefix_push("self");
> - memset(acked, 0, sizeof(acked));
> - smp_wmb();
> + stats_reset();
> cpumask_clear(&mask);
> - cpumask_set_cpu(0, &mask);
> + cpumask_set_cpu(smp_processor_id(), &mask);
> gic->ipi.send_self();
> check_acked(&mask);
> report_prefix_pop();
> @@ -159,20 +204,18 @@ static void ipi_test_smp(void)
> int i;
>
> report_prefix_push("target-list");
> - memset(acked, 0, sizeof(acked));
> - smp_wmb();
> + stats_reset();
> cpumask_copy(&mask, &cpu_present_mask);
> - for (i = 0; i < nr_cpus; i += 2)
> + for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
> cpumask_clear_cpu(i, &mask);
> - gic->ipi.send_tlist(&mask, 0);
> + gic->ipi.send_tlist(&mask, cmdl_irq);
> check_acked(&mask);
> report_prefix_pop();
>
> report_prefix_push("broadcast");
> - memset(acked, 0, sizeof(acked));
> - smp_wmb();
> + stats_reset();
> cpumask_copy(&mask, &cpu_present_mask);
> - cpumask_clear_cpu(0, &mask);
> + cpumask_clear_cpu(smp_processor_id(), &mask);
> gic->ipi.send_broadcast();
> check_acked(&mask);
> report_prefix_pop();
> @@ -189,6 +232,16 @@ static void ipi_enable(void)
> local_irq_enable();
> }
>
> +static void ipi_send(void)
> +{
> + ipi_enable();
> + wait_on_ready();
> + ipi_test_self();
> + ipi_test_smp();
> + check_spurious();
> + exit(report_summary());
> +}
> +
> static void ipi_recv(void)
> {
> ipi_enable();
> @@ -197,6 +250,14 @@ static void ipi_recv(void)
> wfi();
> }
>
> +static void ipi_test(void)
> +{
> + if (smp_processor_id() == cmdl_sender)
> + ipi_send();
> + else
> + ipi_recv();
> +}
> +
> static struct gic gicv2 = {
> .ipi = {
> .send_self = gicv2_ipi_send_self,
> @@ -242,21 +303,34 @@ int main(int argc, char **argv)
> report_prefix_pop();
>
> } else if (strcmp(argv[1], "ipi") == 0) {
> + int off, i = 1;
> + long val;
>
> report_prefix_push(argv[1]);
> nr_cpu_check(2);
>
> + while (--argc != 1) {
> + off = parse_keyval(argv[++i], &val);
> + if (off == -1)
> + continue;
> + argv[i][off] = '\0';
> + if (strcmp(argv[i], "sender") == 0) {
> + if (val >= nr_cpus)
> + report_abort("invalid sender %d, nr_cpus=%d", val, nr_cpus);
> + cmdl_sender = val;
> + } else if (strcmp(argv[i], "irq") == 0) {
> + if (val > 15)
> + report_abort("irq (SGI) must be < 16");
> + cmdl_irq = val;
> + }
wrong arg could potentially be tested.
Besides
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Eric
> + }
> +
> for_each_present_cpu(cpu) {
> if (cpu == 0)
> continue;
> - smp_boot_secondary(cpu, ipi_recv);
> + smp_boot_secondary(cpu, ipi_test);
> }
> - ipi_enable();
> - wait_on_ready();
> - ipi_test_self();
> - ipi_test_smp();
> - check_spurious();
> - report_prefix_pop();
> + ipi_test();
>
> } else {
> report_abort("Unknown subtest '%s'", argv[1]);
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero
@ 2016-11-23 11:28 ` Auger Eric
0 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 11:28 UTC (permalink / raw)
To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
marc.zyngier, christoffer.dall
Hi,
On 14/11/2016 22:08, Andrew Jones wrote:
> Allow user to select who sends ipis and with which irq,
> rather than just always sending irq=0 from cpu0.
>From a user point of view is there a way to know the list of available
tests and their arg?
>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v6:
> - make sender/irq names more future-proof [drew]
> - sanity check inputs [drew]
> - introduce check_sender/irq and bad_sender/irq to more
> cleanly do checks [drew]
> - default sender and irq to 1, instead of still zero [drew]
> v4: improve structure and make sure spurious checking is
> done even when the sender isn't cpu0
> v2: actually check that the irq received was the irq sent,
> and (for gicv2) that the sender is the expected one.
> ---
> arm/gic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
> 1 file changed, 99 insertions(+), 25 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index d954a3775c26..638b8b140c96 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -11,6 +11,7 @@
> * This work is licensed under the terms of the GNU LGPL, version 2.
> */
> #include <libcflat.h>
> +#include <util.h>
> #include <asm/setup.h>
> #include <asm/processor.h>
> #include <asm/gic.h>
> @@ -28,6 +29,8 @@ struct gic {
>
> static struct gic *gic;
> static int acked[NR_CPUS], spurious[NR_CPUS];
> +static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
> +static int cmdl_sender = 1, cmdl_irq = 1;
> static cpumask_t ready;
>
> static void nr_cpu_check(int nr)
> @@ -43,10 +46,23 @@ static void wait_on_ready(void)
> cpu_relax();
> }
>
> +static void stats_reset(void)
> +{
> + int i;
> +
> + for (i = 0; i < nr_cpus; ++i) {
> + acked[i] = 0;
> + bad_sender[i] = -1;
> + bad_irq[i] = -1;
> + }
> + smp_wmb();
> +}
> +
> static void check_acked(cpumask_t *mask)
> {
> int missing = 0, extra = 0, unexpected = 0;
> int nr_pass, cpu, i;
> + bool bad = false;
>
> /* Wait up to 5s for all interrupts to be delivered */
> for (i = 0; i < 50; ++i) {
> @@ -56,9 +72,21 @@ static void check_acked(cpumask_t *mask)
> smp_rmb();
> nr_pass += cpumask_test_cpu(cpu, mask) ?
> acked[cpu] == 1 : acked[cpu] == 0;
> +
> + if (bad_sender[cpu] != -1) {
> + printf("cpu%d received IPI from wrong sender %d\n",
> + cpu, bad_sender[cpu]);
> + bad = true;
> + }
> +
> + if (bad_irq[cpu] != -1) {
> + printf("cpu%d received wrong irq %d\n",
> + cpu, bad_irq[cpu]);
> + bad = true;
> + }
> }
> if (nr_pass == nr_cpus) {
> - report("Completed in %d ms", true, ++i * 100);
> + report("Completed in %d ms", !bad, ++i * 100);
> return;
> }
> }
> @@ -91,6 +119,22 @@ static void check_spurious(void)
> }
> }
>
> +static void check_ipi_sender(u32 irqstat)
> +{
> + if (gic_version() == 2) {
> + int src = (irqstat >> 10) & 7;
> +
> + if (src != cmdl_sender)
> + bad_sender[smp_processor_id()] = src;
> + }
> +}
> +
> +static void check_irqnr(u32 irqnr)
> +{
> + if (irqnr != (u32)cmdl_irq)
> + bad_irq[smp_processor_id()] = irqnr;
> +}
> +
> static void ipi_handler(struct pt_regs *regs __unused)
> {
> u32 irqstat = gic_read_iar();
> @@ -98,8 +142,10 @@ static void ipi_handler(struct pt_regs *regs __unused)
>
> if (irqnr != GICC_INT_SPURIOUS) {
> gic_write_eoir(irqstat);
> - smp_rmb(); /* pairs with wmb in ipi_test functions */
> + smp_rmb(); /* pairs with wmb in stats_reset */
> ++acked[smp_processor_id()];
> + check_ipi_sender(irqstat);
> + check_irqnr(irqnr);
> smp_wmb(); /* pairs with rmb in check_acked */
> } else {
> ++spurious[smp_processor_id()];
> @@ -109,19 +155,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
>
> static void gicv2_ipi_send_self(void)
> {
> - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> + writel(2 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> }
>
> -static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
> +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq)
> {
> u8 tlist = (u8)cpumask_bits(mask)[0];
>
> - writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> + writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
> }
>
> static void gicv2_ipi_send_broadcast(void)
> {
> - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> + writel(1 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> }
>
> static void gicv3_ipi_send_self(void)
> @@ -130,12 +176,12 @@ static void gicv3_ipi_send_self(void)
>
> cpumask_clear(&mask);
> cpumask_set_cpu(smp_processor_id(), &mask);
> - gicv3_ipi_send_tlist(&mask, 0);
> + gicv3_ipi_send_tlist(&mask, cmdl_irq);
> }
>
> static void gicv3_ipi_send_broadcast(void)
> {
> - gicv3_write_sgi1r(1ULL << 40);
> + gicv3_write_sgi1r(1ULL << 40 | cmdl_irq << 24);
> isb();
> }
>
> @@ -144,10 +190,9 @@ static void ipi_test_self(void)
> cpumask_t mask;
>
> report_prefix_push("self");
> - memset(acked, 0, sizeof(acked));
> - smp_wmb();
> + stats_reset();
> cpumask_clear(&mask);
> - cpumask_set_cpu(0, &mask);
> + cpumask_set_cpu(smp_processor_id(), &mask);
> gic->ipi.send_self();
> check_acked(&mask);
> report_prefix_pop();
> @@ -159,20 +204,18 @@ static void ipi_test_smp(void)
> int i;
>
> report_prefix_push("target-list");
> - memset(acked, 0, sizeof(acked));
> - smp_wmb();
> + stats_reset();
> cpumask_copy(&mask, &cpu_present_mask);
> - for (i = 0; i < nr_cpus; i += 2)
> + for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
> cpumask_clear_cpu(i, &mask);
> - gic->ipi.send_tlist(&mask, 0);
> + gic->ipi.send_tlist(&mask, cmdl_irq);
> check_acked(&mask);
> report_prefix_pop();
>
> report_prefix_push("broadcast");
> - memset(acked, 0, sizeof(acked));
> - smp_wmb();
> + stats_reset();
> cpumask_copy(&mask, &cpu_present_mask);
> - cpumask_clear_cpu(0, &mask);
> + cpumask_clear_cpu(smp_processor_id(), &mask);
> gic->ipi.send_broadcast();
> check_acked(&mask);
> report_prefix_pop();
> @@ -189,6 +232,16 @@ static void ipi_enable(void)
> local_irq_enable();
> }
>
> +static void ipi_send(void)
> +{
> + ipi_enable();
> + wait_on_ready();
> + ipi_test_self();
> + ipi_test_smp();
> + check_spurious();
> + exit(report_summary());
> +}
> +
> static void ipi_recv(void)
> {
> ipi_enable();
> @@ -197,6 +250,14 @@ static void ipi_recv(void)
> wfi();
> }
>
> +static void ipi_test(void)
> +{
> + if (smp_processor_id() == cmdl_sender)
> + ipi_send();
> + else
> + ipi_recv();
> +}
> +
> static struct gic gicv2 = {
> .ipi = {
> .send_self = gicv2_ipi_send_self,
> @@ -242,21 +303,34 @@ int main(int argc, char **argv)
> report_prefix_pop();
>
> } else if (strcmp(argv[1], "ipi") == 0) {
> + int off, i = 1;
> + long val;
>
> report_prefix_push(argv[1]);
> nr_cpu_check(2);
>
> + while (--argc != 1) {
> + off = parse_keyval(argv[++i], &val);
> + if (off == -1)
> + continue;
> + argv[i][off] = '\0';
> + if (strcmp(argv[i], "sender") == 0) {
> + if (val >= nr_cpus)
> + report_abort("invalid sender %d, nr_cpus=%d", val, nr_cpus);
> + cmdl_sender = val;
> + } else if (strcmp(argv[i], "irq") == 0) {
> + if (val > 15)
> + report_abort("irq (SGI) must be < 16");
> + cmdl_irq = val;
> + }
wrong arg could potentially be tested.
Besides
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Eric
> + }
> +
> for_each_present_cpu(cpu) {
> if (cpu == 0)
> continue;
> - smp_boot_secondary(cpu, ipi_recv);
> + smp_boot_secondary(cpu, ipi_test);
> }
> - ipi_enable();
> - wait_on_ready();
> - ipi_test_self();
> - ipi_test_smp();
> - check_spurious();
> - report_prefix_pop();
> + ipi_test();
>
> } else {
> report_abort("Unknown subtest '%s'", argv[1]);
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework
2016-11-23 10:09 ` Alex Bennée
@ 2016-11-23 11:33 ` Auger Eric
-1 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 11:33 UTC (permalink / raw)
To: Alex Bennée, Andrew Jones
Cc: kvm, marc.zyngier, andre.przywara, qemu-devel, qemu-arm,
pbonzini, kvmarm
Hi,
On 23/11/2016 11:09, Alex Bennée wrote:
>
> Andrew Jones <drjones@redhat.com> writes:
>
>> Andre, Alex, Eric, anybody,
>>
>> Any more comments on this? If not, I'll send a pull request
>> to Radim and Paolo to finally get this merged.
>
> Looks good to me. I successfully re-based my TCG tests on top and they
> work fine ;-)
no blocking point for me either. That's already a huge work and when
adding new tests we may end up rationalizing things in separate v2/v3
lib files and ops.
Tested on Cavium ThunderX
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Eric Auger <eric.auger@redhat.com>
Thanks
Eric
>
> Tested-by: Alex Bennée <alex.bennee@linaro.org>
>
>>
>> Thanks,
>> drew
>>
>>
>> On Mon, Nov 14, 2016 at 10:08:28PM +0100, Andrew Jones wrote:
>>> v6:
>>> - rebased to latest master
>>> - several other changes thanks to Andre and Alex, changes in
>>> individual patch change logs
>>> - some code cleanups
>>>
>>> v5:
>>> - fix arm32/gicv3 compile [drew]
>>> - use modern register names [Andre]
>>> - one Andre r-b
>>>
>>> v4:
>>> - Eric's r-b's
>>> - Andre's suggestion to only take defines we need
>>> - several other changes listed in individual patches
>>>
>>> v3:
>>> - Rebased on latest master
>>> - Added Alex's r-b's
>>>
>>> v2:
>>> Rebased on latest master + my "populate argv[0]" series (will
>>> send a REPOST for that shortly. Additionally a few patches got
>>> fixes/features;
>>> 07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
>>> all interrupts as non-secure Group-1" in order to continue
>>> working over TCG, as the gicv3 code for TCG removed a hack
>>> it had there to make Linux happy.
>>> 08/10 added more output for when things fail (if they fail)
>>> 09/10 switched gicv3 broadcast implementation to using IRM. This
>>> found a bug in a recent (but not tip) kernel, which I was
>>> about to fix, but then I saw MarcZ beat me to it.
>>> 10/10 actually check that the input irq is the received irq
>>>
>>>
>>> Import defines, and steal enough helper functions, from Linux to
>>> enable programming of the gic (v2 and v3). Then use the framework
>>> to add an initial test (an ipi test; self, target-list, broadcast).
>>>
>>> It's my hope that this framework will be a suitable base on which
>>> more tests may be easily added, particularly because we have
>>> vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
>>> vgic-new and tcg gicv3 are merged now)
>>>
>>> To run it, along with other tests, just do
>>>
>>> ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
>>> make
>>> export QEMU=$PATH_TO_QEMU
>>> ./run_tests.sh
>>>
>>> To run it separately do, e.g.
>>>
>>> $QEMU -machine virt,accel=tcg -cpu cortex-a57 \
>>> -device virtio-serial-device \
>>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>>> -display none -serial stdio \
>>> -kernel arm/gic.flat \
>>> -smp 123 -machine gic-version=3 -append ipi
>>> ^^ note, we can go nuts with nr-cpus on TCG :-)
>>>
>>> Or, a KVM example using a different "sender" cpu and irq (other than zero)
>>>
>>> $QEMU -machine virt,accel=kvm -cpu host \
>>> -device virtio-serial-device \
>>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>>> -display none -serial stdio \
>>> -kernel arm/gic.flat \
>>> -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
>>>
>>>
>>> Patches:
>>> 01-05: fixes and functionality needed by the later gic patches
>>> 06-07: enable gicv2 and gicv2 IPI test
>>> 08-10: enable gicv3 and gicv3 IPI test
>>> 11: extend the IPI tests to take variable sender and irq
>>>
>>> Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
>>>
>>>
>>> Andrew Jones (10):
>>> lib: xstr: allow multiple args
>>> arm64: fix get_"sysreg32" and make MPIDR 64bit
>>> arm/arm64: smp: support more than 8 cpus
>>> arm/arm64: add some delay routines
>>> arm/arm64: irq enable/disable
>>> arm/arm64: add initial gicv2 support
>>> arm/arm64: gicv2: add an IPI test
>>> arm/arm64: add initial gicv3 support
>>> arm/arm64: gicv3: add an IPI test
>>> arm/arm64: gic: don't just use zero
>>>
>>> Peter Xu (1):
>>> libcflat: add IS_ALIGNED() macro, and page sizes
>>>
>>> arm/Makefile.common | 9 +-
>>> arm/gic.c | 340 +++++++++++++++++++++++++++++++++++++++++++++
>>> arm/run | 19 ++-
>>> arm/selftest.c | 5 +-
>>> arm/unittests.cfg | 14 ++
>>> lib/arm/asm/arch_gicv3.h | 70 ++++++++++
>>> lib/arm/asm/gic-v2.h | 36 +++++
>>> lib/arm/asm/gic-v3.h | 112 +++++++++++++++
>>> lib/arm/asm/gic.h | 106 ++++++++++++++
>>> lib/arm/asm/processor.h | 42 +++++-
>>> lib/arm/asm/setup.h | 4 +-
>>> lib/arm/gic.c | 267 +++++++++++++++++++++++++++++++++++
>>> lib/arm/processor.c | 15 ++
>>> lib/arm/setup.c | 10 ++
>>> lib/arm64/asm/arch_gicv3.h | 66 +++++++++
>>> lib/arm64/asm/gic-v2.h | 1 +
>>> lib/arm64/asm/gic-v3.h | 1 +
>>> lib/arm64/asm/gic.h | 1 +
>>> lib/arm64/asm/processor.h | 53 +++++--
>>> lib/arm64/asm/sysreg.h | 44 ++++++
>>> lib/arm64/processor.c | 15 ++
>>> lib/libcflat.h | 10 +-
>>> 22 files changed, 1212 insertions(+), 28 deletions(-)
>>> create mode 100644 arm/gic.c
>>> create mode 100644 lib/arm/asm/arch_gicv3.h
>>> create mode 100644 lib/arm/asm/gic-v2.h
>>> create mode 100644 lib/arm/asm/gic-v3.h
>>> create mode 100644 lib/arm/asm/gic.h
>>> create mode 100644 lib/arm/gic.c
>>> create mode 100644 lib/arm64/asm/arch_gicv3.h
>>> create mode 100644 lib/arm64/asm/gic-v2.h
>>> create mode 100644 lib/arm64/asm/gic-v3.h
>>> create mode 100644 lib/arm64/asm/gic.h
>>> create mode 100644 lib/arm64/asm/sysreg.h
>>>
>>> --
>>> 2.7.4
>>>
>>>
>
>
> --
> Alex Bennée
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework
@ 2016-11-23 11:33 ` Auger Eric
0 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 11:33 UTC (permalink / raw)
To: Alex Bennée, Andrew Jones
Cc: kvm, kvmarm, qemu-devel, qemu-arm, peter.maydell, marc.zyngier,
andre.przywara, pbonzini, christoffer.dall
Hi,
On 23/11/2016 11:09, Alex Bennée wrote:
>
> Andrew Jones <drjones@redhat.com> writes:
>
>> Andre, Alex, Eric, anybody,
>>
>> Any more comments on this? If not, I'll send a pull request
>> to Radim and Paolo to finally get this merged.
>
> Looks good to me. I successfully re-based my TCG tests on top and they
> work fine ;-)
no blocking point for me either. That's already a huge work and when
adding new tests we may end up rationalizing things in separate v2/v3
lib files and ops.
Tested on Cavium ThunderX
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Eric Auger <eric.auger@redhat.com>
Thanks
Eric
>
> Tested-by: Alex Bennée <alex.bennee@linaro.org>
>
>>
>> Thanks,
>> drew
>>
>>
>> On Mon, Nov 14, 2016 at 10:08:28PM +0100, Andrew Jones wrote:
>>> v6:
>>> - rebased to latest master
>>> - several other changes thanks to Andre and Alex, changes in
>>> individual patch change logs
>>> - some code cleanups
>>>
>>> v5:
>>> - fix arm32/gicv3 compile [drew]
>>> - use modern register names [Andre]
>>> - one Andre r-b
>>>
>>> v4:
>>> - Eric's r-b's
>>> - Andre's suggestion to only take defines we need
>>> - several other changes listed in individual patches
>>>
>>> v3:
>>> - Rebased on latest master
>>> - Added Alex's r-b's
>>>
>>> v2:
>>> Rebased on latest master + my "populate argv[0]" series (will
>>> send a REPOST for that shortly. Additionally a few patches got
>>> fixes/features;
>>> 07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
>>> all interrupts as non-secure Group-1" in order to continue
>>> working over TCG, as the gicv3 code for TCG removed a hack
>>> it had there to make Linux happy.
>>> 08/10 added more output for when things fail (if they fail)
>>> 09/10 switched gicv3 broadcast implementation to using IRM. This
>>> found a bug in a recent (but not tip) kernel, which I was
>>> about to fix, but then I saw MarcZ beat me to it.
>>> 10/10 actually check that the input irq is the received irq
>>>
>>>
>>> Import defines, and steal enough helper functions, from Linux to
>>> enable programming of the gic (v2 and v3). Then use the framework
>>> to add an initial test (an ipi test; self, target-list, broadcast).
>>>
>>> It's my hope that this framework will be a suitable base on which
>>> more tests may be easily added, particularly because we have
>>> vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
>>> vgic-new and tcg gicv3 are merged now)
>>>
>>> To run it, along with other tests, just do
>>>
>>> ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
>>> make
>>> export QEMU=$PATH_TO_QEMU
>>> ./run_tests.sh
>>>
>>> To run it separately do, e.g.
>>>
>>> $QEMU -machine virt,accel=tcg -cpu cortex-a57 \
>>> -device virtio-serial-device \
>>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>>> -display none -serial stdio \
>>> -kernel arm/gic.flat \
>>> -smp 123 -machine gic-version=3 -append ipi
>>> ^^ note, we can go nuts with nr-cpus on TCG :-)
>>>
>>> Or, a KVM example using a different "sender" cpu and irq (other than zero)
>>>
>>> $QEMU -machine virt,accel=kvm -cpu host \
>>> -device virtio-serial-device \
>>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>>> -display none -serial stdio \
>>> -kernel arm/gic.flat \
>>> -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
>>>
>>>
>>> Patches:
>>> 01-05: fixes and functionality needed by the later gic patches
>>> 06-07: enable gicv2 and gicv2 IPI test
>>> 08-10: enable gicv3 and gicv3 IPI test
>>> 11: extend the IPI tests to take variable sender and irq
>>>
>>> Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
>>>
>>>
>>> Andrew Jones (10):
>>> lib: xstr: allow multiple args
>>> arm64: fix get_"sysreg32" and make MPIDR 64bit
>>> arm/arm64: smp: support more than 8 cpus
>>> arm/arm64: add some delay routines
>>> arm/arm64: irq enable/disable
>>> arm/arm64: add initial gicv2 support
>>> arm/arm64: gicv2: add an IPI test
>>> arm/arm64: add initial gicv3 support
>>> arm/arm64: gicv3: add an IPI test
>>> arm/arm64: gic: don't just use zero
>>>
>>> Peter Xu (1):
>>> libcflat: add IS_ALIGNED() macro, and page sizes
>>>
>>> arm/Makefile.common | 9 +-
>>> arm/gic.c | 340 +++++++++++++++++++++++++++++++++++++++++++++
>>> arm/run | 19 ++-
>>> arm/selftest.c | 5 +-
>>> arm/unittests.cfg | 14 ++
>>> lib/arm/asm/arch_gicv3.h | 70 ++++++++++
>>> lib/arm/asm/gic-v2.h | 36 +++++
>>> lib/arm/asm/gic-v3.h | 112 +++++++++++++++
>>> lib/arm/asm/gic.h | 106 ++++++++++++++
>>> lib/arm/asm/processor.h | 42 +++++-
>>> lib/arm/asm/setup.h | 4 +-
>>> lib/arm/gic.c | 267 +++++++++++++++++++++++++++++++++++
>>> lib/arm/processor.c | 15 ++
>>> lib/arm/setup.c | 10 ++
>>> lib/arm64/asm/arch_gicv3.h | 66 +++++++++
>>> lib/arm64/asm/gic-v2.h | 1 +
>>> lib/arm64/asm/gic-v3.h | 1 +
>>> lib/arm64/asm/gic.h | 1 +
>>> lib/arm64/asm/processor.h | 53 +++++--
>>> lib/arm64/asm/sysreg.h | 44 ++++++
>>> lib/arm64/processor.c | 15 ++
>>> lib/libcflat.h | 10 +-
>>> 22 files changed, 1212 insertions(+), 28 deletions(-)
>>> create mode 100644 arm/gic.c
>>> create mode 100644 lib/arm/asm/arch_gicv3.h
>>> create mode 100644 lib/arm/asm/gic-v2.h
>>> create mode 100644 lib/arm/asm/gic-v3.h
>>> create mode 100644 lib/arm/asm/gic.h
>>> create mode 100644 lib/arm/gic.c
>>> create mode 100644 lib/arm64/asm/arch_gicv3.h
>>> create mode 100644 lib/arm64/asm/gic-v2.h
>>> create mode 100644 lib/arm64/asm/gic-v3.h
>>> create mode 100644 lib/arm64/asm/gic.h
>>> create mode 100644 lib/arm64/asm/sysreg.h
>>>
>>> --
>>> 2.7.4
>>>
>>>
>
>
> --
> Alex Bennée
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 10/11] arm/arm64: gicv3: add an IPI test
2016-11-23 11:05 ` [Qemu-devel] " Auger Eric
@ 2016-11-23 12:57 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-23 12:57 UTC (permalink / raw)
To: Auger Eric
Cc: kvm, marc.zyngier, andre.przywara, qemu-devel, qemu-arm,
pbonzini, kvmarm
On Wed, Nov 23, 2016 at 12:05:42PM +0100, Auger Eric wrote:
> Hi Drew,
>
> On 14/11/2016 22:08, Andrew Jones wrote:
> > Signed-off-by: Andrew Jones <drjones@redhat.com>
> >
> > ---
> > v6: move most gicv2/gicv3 wrappers to common code [Alex]
> > v5:
> > - fix copy+paste error in gicv3_write_eoir [drew]
> > - use modern register names [Andre]
> > v4:
> > - heavily comment gicv3_ipi_send_tlist() [Eric]
> > - changes needed for gicv2 iar/irqstat fix to other patch
> > v2:
> > - use IRM for gicv3 broadcast
> > ---
> > arm/gic.c | 97 +++++++++++++++++++++++++-----
> > arm/unittests.cfg | 6 ++
> > lib/arm/asm/arch_gicv3.h | 23 +++++++
> > lib/arm/asm/gic-v3.h | 10 +++-
> > lib/arm/asm/gic.h | 60 +++++++++++++++++++
> > lib/arm/gic.c | 145 ++++++++++++++++++++++++++++++++++++++++++---
> > lib/arm64/asm/arch_gicv3.h | 22 +++++++
> > 7 files changed, 338 insertions(+), 25 deletions(-)
> >
> > diff --git a/arm/gic.c b/arm/gic.c
> > index b42c2b1ca1e1..d954a3775c26 100644
> > --- a/arm/gic.c
> > +++ b/arm/gic.c
> > @@ -3,6 +3,8 @@
> > *
> > * GICv2
> > * + test sending/receiving IPIs
> > + * GICv3
> > + * + test sending/receiving IPIs
> > *
> > * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > *
> > @@ -16,7 +18,15 @@
> > #include <asm/barrier.h>
> > #include <asm/io.h>
> >
> > -static int gic_version;
> > +struct gic {
> > + struct {
> > + void (*send_self)(void);
> > + void (*send_tlist)(cpumask_t *mask, int irq);
> > + void (*send_broadcast)(void);
> > + } ipi;
> > +};
> what is the rationale behind not putting this into common_ops?
I added gic_ipi_send(cpu, irq) to common_ops, which I think will
satisfy most users (it certainly covers send_self easily enough).
gicv3_ipi_send_tlist is also available for gicv3 users. The rest
are trivial, so I didn't want to "clutter" common ops with them.
Keep in mind that this struct is for the unit test, and so it
may choose to define ops with less general uses and/or interfaces.
I.e. in all cases above the irq used is stored in a global variable.
common_ops would want that to be a parameter.
> > +
> > +static struct gic *gic;
> > static int acked[NR_CPUS], spurious[NR_CPUS];
> > static cpumask_t ready;
> >
> > @@ -83,11 +93,11 @@ static void check_spurious(void)
> >
> > static void ipi_handler(struct pt_regs *regs __unused)
> > {
> > - u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
> > - u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
> > + u32 irqstat = gic_read_iar();
> > + u32 irqnr = gic_iar_irqnr(irqstat);
> >
> > if (irqnr != GICC_INT_SPURIOUS) {
> > - writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
> > + gic_write_eoir(irqstat);
> > smp_rmb(); /* pairs with wmb in ipi_test functions */
> > ++acked[smp_processor_id()];
> > smp_wmb(); /* pairs with rmb in check_acked */
> > @@ -97,6 +107,38 @@ static void ipi_handler(struct pt_regs *regs __unused)
> > }
> > }
> >
> > +static void gicv2_ipi_send_self(void)
> > +{
> > + writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> > +}
> > +
> > +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
> > +{
> > + u8 tlist = (u8)cpumask_bits(mask)[0];
> > +
> > + writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> > +}
> > +
> > +static void gicv2_ipi_send_broadcast(void)
> > +{
> > + writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> > +}
> > +
> > +static void gicv3_ipi_send_self(void)
> > +{
> > + cpumask_t mask;
> > +
> > + cpumask_clear(&mask);
> > + cpumask_set_cpu(smp_processor_id(), &mask);
> > + gicv3_ipi_send_tlist(&mask, 0);
> > +}
> > +
> > +static void gicv3_ipi_send_broadcast(void)
> > +{
> > + gicv3_write_sgi1r(1ULL << 40);
> > + isb();
> > +}
> > +
> > static void ipi_test_self(void)
> > {
> > cpumask_t mask;
> > @@ -106,7 +148,7 @@ static void ipi_test_self(void)
> > smp_wmb();
> > cpumask_clear(&mask);
> > cpumask_set_cpu(0, &mask);
> > - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> > + gic->ipi.send_self();
> > check_acked(&mask);
> > report_prefix_pop();
> > }
> > @@ -114,14 +156,15 @@ static void ipi_test_self(void)
> > static void ipi_test_smp(void)
> > {
> > cpumask_t mask;
> > - unsigned long tlist;
> > + int i;
> >
> > report_prefix_push("target-list");
> > memset(acked, 0, sizeof(acked));
> > smp_wmb();
> > - tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
> > - cpumask_bits(&mask)[0] = tlist;
> > - writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
> > + cpumask_copy(&mask, &cpu_present_mask);
> > + for (i = 0; i < nr_cpus; i += 2)
> > + cpumask_clear_cpu(i, &mask);
> > + gic->ipi.send_tlist(&mask, 0);
> > check_acked(&mask);
> > report_prefix_pop();
> >
> > @@ -130,14 +173,14 @@ static void ipi_test_smp(void)
> > smp_wmb();
> > cpumask_copy(&mask, &cpu_present_mask);
> > cpumask_clear_cpu(0, &mask);
> > - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> > + gic->ipi.send_broadcast();
> > check_acked(&mask);
> > report_prefix_pop();
> > }
> >
> > static void ipi_enable(void)
> > {
> > - gicv2_enable_defaults();
> > + gic_enable_defaults();
> > #ifdef __arm__
> > install_exception_handler(EXCPTN_IRQ, ipi_handler);
> > #else
> > @@ -154,18 +197,42 @@ static void ipi_recv(void)
> > wfi();
> > }
> >
> > +static struct gic gicv2 = {
> > + .ipi = {
> > + .send_self = gicv2_ipi_send_self,
> > + .send_tlist = gicv2_ipi_send_tlist,
> > + .send_broadcast = gicv2_ipi_send_broadcast,
> > + },
> > +};
> > +
> > +static struct gic gicv3 = {
> > + .ipi = {
> > + .send_self = gicv3_ipi_send_self,
> > + .send_tlist = gicv3_ipi_send_tlist,
> > + .send_broadcast = gicv3_ipi_send_broadcast,
> > + },
> > +};
> > +
> > int main(int argc, char **argv)
> > {
> > char pfx[8];
> > int cpu;
> >
> > - gic_version = gic_init();
> > - if (!gic_version)
> > - report_abort("No gic present!");
> > + if (!gic_init())
> > + report_abort("No supported gic present!");
> >
> > - snprintf(pfx, sizeof(pfx), "gicv%d", gic_version);
> > + snprintf(pfx, sizeof(pfx), "gicv%d", gic_version());
> > report_prefix_push(pfx);
> >
> > + switch (gic_version()) {
> > + case 2:
> > + gic = &gicv2;
> > + break;
> > + case 3:
> > + gic = &gicv3;
> > + break;
> > + }
> > +
> > if (argc < 2) {
> >
> > report_prefix_push("ipi");
> > diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> > index e631c35e2bbb..c7392c747f98 100644
> > --- a/arm/unittests.cfg
> > +++ b/arm/unittests.cfg
> > @@ -66,3 +66,9 @@ file = gic.flat
> > smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
> > extra_params = -machine gic-version=2 -append 'ipi'
> > groups = gic
> > +
> > +[gicv3-ipi]
> > +file = gic.flat
> > +smp = $MAX_SMP
> > +extra_params = -machine gic-version=3 -append 'ipi'
> > +groups = gic
> > diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> > index 276577452a14..b47cd2e0090b 100644
> > --- a/lib/arm/asm/arch_gicv3.h
> > +++ b/lib/arm/asm/arch_gicv3.h
> > @@ -16,10 +16,28 @@
> > #define __stringify xstr
> >
> > #define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
> > +#define __ACCESS_CP15_64(Op1, CRm) p15, Op1, %Q0, %R0, CRm
> >
> > +#define ICC_EOIR1 __ACCESS_CP15(c12, 0, c12, 1)
> > +#define ICC_IAR1 __ACCESS_CP15(c12, 0, c12, 0)
> > +#define ICC_SGI1R __ACCESS_CP15_64(0, c12)
> > #define ICC_PMR __ACCESS_CP15(c4, 0, c6, 0)
> > #define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
> >
> > +static inline void gicv3_write_eoir(u32 irq)
> > +{
> > + asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> > + isb();
> > +}
> > +
> > +static inline u32 gicv3_read_iar(void)
> > +{
> > + u32 irqstat;
> > + asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
> > + dsb(sy);
> > + return irqstat;
> > +}
> > +
> > static inline void gicv3_write_pmr(u32 val)
> > {
> > asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> > @@ -31,6 +49,11 @@ static inline void gicv3_write_grpen1(u32 val)
> > isb();
> > }
> >
> > +static inline void gicv3_write_sgi1r(u64 val)
> > +{
> > + asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> > +}
> > +
> > /*
> > * We may access GICR_TYPER and GITS_TYPER by reading both the TYPER
> > * offset and the following offset (+ 4) and then combining them to
> > diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> > index 73ade4681d21..43f9ffce56de 100644
> > --- a/lib/arm/asm/gic-v3.h
> > +++ b/lib/arm/asm/gic-v3.h
> > @@ -33,12 +33,19 @@
> > #define GICR_ISENABLER0 GICD_ISENABLER
> > #define GICR_IPRIORITYR0 GICD_IPRIORITYR
> >
> > +#define ICC_SGI1R_AFFINITY_1_SHIFT 16
> > +#define ICC_SGI1R_AFFINITY_2_SHIFT 32
> > +#define ICC_SGI1R_AFFINITY_3_SHIFT 48
> > +#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> > + (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
> > +
> > #include <asm/arch_gicv3.h>
> >
> > #ifndef __ASSEMBLY__
> > #include <asm/setup.h>
> > -#include <asm/smp.h>
> > #include <asm/processor.h>
> > +#include <asm/cpumask.h>
> > +#include <asm/smp.h>
> > #include <asm/io.h>
> >
> > struct gicv3_data {
> > @@ -55,6 +62,7 @@ extern struct gicv3_data gicv3_data;
> > extern int gicv3_init(void);
> > extern void gicv3_enable_defaults(void);
> > extern void gicv3_set_redist_base(size_t stride);
> > +extern void gicv3_ipi_send_tlist(cpumask_t *mask, int irq);
> >
> > static inline void gicv3_do_wait_for_rwp(void *base)
> > {
> > diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> > index 21511997f2a9..c2267b6b3937 100644
> > --- a/lib/arm/asm/gic.h
> > +++ b/lib/arm/asm/gic.h
> > @@ -42,5 +42,65 @@
> > */
> > extern int gic_init(void);
> >
> > +/*
> > + * gic_common_ops collects some functions that we provide unit
> > + * tests that don't care which gic version they're using.
> nit: wording?
How about
gic_common_ops collects useful functions for unit tests which
aren't concerned with the gic version they're using.
> > + */
> > +struct gic_common_ops {
> > + int gic_version;
> > + u32 (*read_iar)(void);
> > + u32 (*iar_irqnr)(u32 iar);
> > + void (*write_eoir)(u32 irqstat);
> > + void (*ipi_send)(int cpu, int irq);
> what is the rationale behind not putting enable_defaults here?
Hmm, I don't have a great rationale for that. I was thinking that
'enable_defaults' was a strange op, that 'enable' would be more
appropriate, but I didn't want to set .enable to enable_defaults,
as that may compel unit test writers to use it when they should
instead write their own.
Adding an 'enable_defaults' op makes more sense though... I'll
change this.
>
> lib/arm/gic.c becomes bigger and bigger. Woudn't it make sense to have
> separate lib/arm/gic-v2/v3.c and arm/gic.c only call generic ops?
I agree with the split.
>
> Typically ipi functions could go in lib
We have the one, gic_ipi_send(cpu, irq). I could also add a
gic_ipi_send_all(irq), as that may also be useful.
Or... I see my struct gic_common_ops is similar to Linux's
struct irq_chip. irq_chip has ipi_send_single and ipi_send_mask,
so I could copy those choices, and even names.
Thanks,
drew
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-23 12:57 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-23 12:57 UTC (permalink / raw)
To: Auger Eric
Cc: kvm, kvmarm, qemu-devel, qemu-arm, peter.maydell, marc.zyngier,
andre.przywara, pbonzini, alex.bennee, christoffer.dall
On Wed, Nov 23, 2016 at 12:05:42PM +0100, Auger Eric wrote:
> Hi Drew,
>
> On 14/11/2016 22:08, Andrew Jones wrote:
> > Signed-off-by: Andrew Jones <drjones@redhat.com>
> >
> > ---
> > v6: move most gicv2/gicv3 wrappers to common code [Alex]
> > v5:
> > - fix copy+paste error in gicv3_write_eoir [drew]
> > - use modern register names [Andre]
> > v4:
> > - heavily comment gicv3_ipi_send_tlist() [Eric]
> > - changes needed for gicv2 iar/irqstat fix to other patch
> > v2:
> > - use IRM for gicv3 broadcast
> > ---
> > arm/gic.c | 97 +++++++++++++++++++++++++-----
> > arm/unittests.cfg | 6 ++
> > lib/arm/asm/arch_gicv3.h | 23 +++++++
> > lib/arm/asm/gic-v3.h | 10 +++-
> > lib/arm/asm/gic.h | 60 +++++++++++++++++++
> > lib/arm/gic.c | 145 ++++++++++++++++++++++++++++++++++++++++++---
> > lib/arm64/asm/arch_gicv3.h | 22 +++++++
> > 7 files changed, 338 insertions(+), 25 deletions(-)
> >
> > diff --git a/arm/gic.c b/arm/gic.c
> > index b42c2b1ca1e1..d954a3775c26 100644
> > --- a/arm/gic.c
> > +++ b/arm/gic.c
> > @@ -3,6 +3,8 @@
> > *
> > * GICv2
> > * + test sending/receiving IPIs
> > + * GICv3
> > + * + test sending/receiving IPIs
> > *
> > * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > *
> > @@ -16,7 +18,15 @@
> > #include <asm/barrier.h>
> > #include <asm/io.h>
> >
> > -static int gic_version;
> > +struct gic {
> > + struct {
> > + void (*send_self)(void);
> > + void (*send_tlist)(cpumask_t *mask, int irq);
> > + void (*send_broadcast)(void);
> > + } ipi;
> > +};
> what is the rationale behind not putting this into common_ops?
I added gic_ipi_send(cpu, irq) to common_ops, which I think will
satisfy most users (it certainly covers send_self easily enough).
gicv3_ipi_send_tlist is also available for gicv3 users. The rest
are trivial, so I didn't want to "clutter" common ops with them.
Keep in mind that this struct is for the unit test, and so it
may choose to define ops with less general uses and/or interfaces.
I.e. in all cases above the irq used is stored in a global variable.
common_ops would want that to be a parameter.
> > +
> > +static struct gic *gic;
> > static int acked[NR_CPUS], spurious[NR_CPUS];
> > static cpumask_t ready;
> >
> > @@ -83,11 +93,11 @@ static void check_spurious(void)
> >
> > static void ipi_handler(struct pt_regs *regs __unused)
> > {
> > - u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
> > - u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
> > + u32 irqstat = gic_read_iar();
> > + u32 irqnr = gic_iar_irqnr(irqstat);
> >
> > if (irqnr != GICC_INT_SPURIOUS) {
> > - writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
> > + gic_write_eoir(irqstat);
> > smp_rmb(); /* pairs with wmb in ipi_test functions */
> > ++acked[smp_processor_id()];
> > smp_wmb(); /* pairs with rmb in check_acked */
> > @@ -97,6 +107,38 @@ static void ipi_handler(struct pt_regs *regs __unused)
> > }
> > }
> >
> > +static void gicv2_ipi_send_self(void)
> > +{
> > + writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> > +}
> > +
> > +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
> > +{
> > + u8 tlist = (u8)cpumask_bits(mask)[0];
> > +
> > + writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> > +}
> > +
> > +static void gicv2_ipi_send_broadcast(void)
> > +{
> > + writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> > +}
> > +
> > +static void gicv3_ipi_send_self(void)
> > +{
> > + cpumask_t mask;
> > +
> > + cpumask_clear(&mask);
> > + cpumask_set_cpu(smp_processor_id(), &mask);
> > + gicv3_ipi_send_tlist(&mask, 0);
> > +}
> > +
> > +static void gicv3_ipi_send_broadcast(void)
> > +{
> > + gicv3_write_sgi1r(1ULL << 40);
> > + isb();
> > +}
> > +
> > static void ipi_test_self(void)
> > {
> > cpumask_t mask;
> > @@ -106,7 +148,7 @@ static void ipi_test_self(void)
> > smp_wmb();
> > cpumask_clear(&mask);
> > cpumask_set_cpu(0, &mask);
> > - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> > + gic->ipi.send_self();
> > check_acked(&mask);
> > report_prefix_pop();
> > }
> > @@ -114,14 +156,15 @@ static void ipi_test_self(void)
> > static void ipi_test_smp(void)
> > {
> > cpumask_t mask;
> > - unsigned long tlist;
> > + int i;
> >
> > report_prefix_push("target-list");
> > memset(acked, 0, sizeof(acked));
> > smp_wmb();
> > - tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
> > - cpumask_bits(&mask)[0] = tlist;
> > - writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
> > + cpumask_copy(&mask, &cpu_present_mask);
> > + for (i = 0; i < nr_cpus; i += 2)
> > + cpumask_clear_cpu(i, &mask);
> > + gic->ipi.send_tlist(&mask, 0);
> > check_acked(&mask);
> > report_prefix_pop();
> >
> > @@ -130,14 +173,14 @@ static void ipi_test_smp(void)
> > smp_wmb();
> > cpumask_copy(&mask, &cpu_present_mask);
> > cpumask_clear_cpu(0, &mask);
> > - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> > + gic->ipi.send_broadcast();
> > check_acked(&mask);
> > report_prefix_pop();
> > }
> >
> > static void ipi_enable(void)
> > {
> > - gicv2_enable_defaults();
> > + gic_enable_defaults();
> > #ifdef __arm__
> > install_exception_handler(EXCPTN_IRQ, ipi_handler);
> > #else
> > @@ -154,18 +197,42 @@ static void ipi_recv(void)
> > wfi();
> > }
> >
> > +static struct gic gicv2 = {
> > + .ipi = {
> > + .send_self = gicv2_ipi_send_self,
> > + .send_tlist = gicv2_ipi_send_tlist,
> > + .send_broadcast = gicv2_ipi_send_broadcast,
> > + },
> > +};
> > +
> > +static struct gic gicv3 = {
> > + .ipi = {
> > + .send_self = gicv3_ipi_send_self,
> > + .send_tlist = gicv3_ipi_send_tlist,
> > + .send_broadcast = gicv3_ipi_send_broadcast,
> > + },
> > +};
> > +
> > int main(int argc, char **argv)
> > {
> > char pfx[8];
> > int cpu;
> >
> > - gic_version = gic_init();
> > - if (!gic_version)
> > - report_abort("No gic present!");
> > + if (!gic_init())
> > + report_abort("No supported gic present!");
> >
> > - snprintf(pfx, sizeof(pfx), "gicv%d", gic_version);
> > + snprintf(pfx, sizeof(pfx), "gicv%d", gic_version());
> > report_prefix_push(pfx);
> >
> > + switch (gic_version()) {
> > + case 2:
> > + gic = &gicv2;
> > + break;
> > + case 3:
> > + gic = &gicv3;
> > + break;
> > + }
> > +
> > if (argc < 2) {
> >
> > report_prefix_push("ipi");
> > diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> > index e631c35e2bbb..c7392c747f98 100644
> > --- a/arm/unittests.cfg
> > +++ b/arm/unittests.cfg
> > @@ -66,3 +66,9 @@ file = gic.flat
> > smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
> > extra_params = -machine gic-version=2 -append 'ipi'
> > groups = gic
> > +
> > +[gicv3-ipi]
> > +file = gic.flat
> > +smp = $MAX_SMP
> > +extra_params = -machine gic-version=3 -append 'ipi'
> > +groups = gic
> > diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> > index 276577452a14..b47cd2e0090b 100644
> > --- a/lib/arm/asm/arch_gicv3.h
> > +++ b/lib/arm/asm/arch_gicv3.h
> > @@ -16,10 +16,28 @@
> > #define __stringify xstr
> >
> > #define __ACCESS_CP15(CRn, Op1, CRm, Op2) p15, Op1, %0, CRn, CRm, Op2
> > +#define __ACCESS_CP15_64(Op1, CRm) p15, Op1, %Q0, %R0, CRm
> >
> > +#define ICC_EOIR1 __ACCESS_CP15(c12, 0, c12, 1)
> > +#define ICC_IAR1 __ACCESS_CP15(c12, 0, c12, 0)
> > +#define ICC_SGI1R __ACCESS_CP15_64(0, c12)
> > #define ICC_PMR __ACCESS_CP15(c4, 0, c6, 0)
> > #define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7)
> >
> > +static inline void gicv3_write_eoir(u32 irq)
> > +{
> > + asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> > + isb();
> > +}
> > +
> > +static inline u32 gicv3_read_iar(void)
> > +{
> > + u32 irqstat;
> > + asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
> > + dsb(sy);
> > + return irqstat;
> > +}
> > +
> > static inline void gicv3_write_pmr(u32 val)
> > {
> > asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> > @@ -31,6 +49,11 @@ static inline void gicv3_write_grpen1(u32 val)
> > isb();
> > }
> >
> > +static inline void gicv3_write_sgi1r(u64 val)
> > +{
> > + asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> > +}
> > +
> > /*
> > * We may access GICR_TYPER and GITS_TYPER by reading both the TYPER
> > * offset and the following offset (+ 4) and then combining them to
> > diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> > index 73ade4681d21..43f9ffce56de 100644
> > --- a/lib/arm/asm/gic-v3.h
> > +++ b/lib/arm/asm/gic-v3.h
> > @@ -33,12 +33,19 @@
> > #define GICR_ISENABLER0 GICD_ISENABLER
> > #define GICR_IPRIORITYR0 GICD_IPRIORITYR
> >
> > +#define ICC_SGI1R_AFFINITY_1_SHIFT 16
> > +#define ICC_SGI1R_AFFINITY_2_SHIFT 32
> > +#define ICC_SGI1R_AFFINITY_3_SHIFT 48
> > +#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> > + (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
> > +
> > #include <asm/arch_gicv3.h>
> >
> > #ifndef __ASSEMBLY__
> > #include <asm/setup.h>
> > -#include <asm/smp.h>
> > #include <asm/processor.h>
> > +#include <asm/cpumask.h>
> > +#include <asm/smp.h>
> > #include <asm/io.h>
> >
> > struct gicv3_data {
> > @@ -55,6 +62,7 @@ extern struct gicv3_data gicv3_data;
> > extern int gicv3_init(void);
> > extern void gicv3_enable_defaults(void);
> > extern void gicv3_set_redist_base(size_t stride);
> > +extern void gicv3_ipi_send_tlist(cpumask_t *mask, int irq);
> >
> > static inline void gicv3_do_wait_for_rwp(void *base)
> > {
> > diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> > index 21511997f2a9..c2267b6b3937 100644
> > --- a/lib/arm/asm/gic.h
> > +++ b/lib/arm/asm/gic.h
> > @@ -42,5 +42,65 @@
> > */
> > extern int gic_init(void);
> >
> > +/*
> > + * gic_common_ops collects some functions that we provide unit
> > + * tests that don't care which gic version they're using.
> nit: wording?
How about
gic_common_ops collects useful functions for unit tests which
aren't concerned with the gic version they're using.
> > + */
> > +struct gic_common_ops {
> > + int gic_version;
> > + u32 (*read_iar)(void);
> > + u32 (*iar_irqnr)(u32 iar);
> > + void (*write_eoir)(u32 irqstat);
> > + void (*ipi_send)(int cpu, int irq);
> what is the rationale behind not putting enable_defaults here?
Hmm, I don't have a great rationale for that. I was thinking that
'enable_defaults' was a strange op, that 'enable' would be more
appropriate, but I didn't want to set .enable to enable_defaults,
as that may compel unit test writers to use it when they should
instead write their own.
Adding an 'enable_defaults' op makes more sense though... I'll
change this.
>
> lib/arm/gic.c becomes bigger and bigger. Woudn't it make sense to have
> separate lib/arm/gic-v2/v3.c and arm/gic.c only call generic ops?
I agree with the split.
>
> Typically ipi functions could go in lib
We have the one, gic_ipi_send(cpu, irq). I could also add a
gic_ipi_send_all(irq), as that may also be useful.
Or... I see my struct gic_common_ops is similar to Linux's
struct irq_chip. irq_chip has ipi_send_single and ipi_send_mask,
so I could copy those choices, and even names.
Thanks,
drew
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero
2016-11-23 11:28 ` [Qemu-devel] " Auger Eric
@ 2016-11-23 13:01 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-23 13:01 UTC (permalink / raw)
To: Auger Eric
Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
peter.maydell, alex.bennee, marc.zyngier, christoffer.dall
On Wed, Nov 23, 2016 at 12:28:34PM +0100, Auger Eric wrote:
> Hi,
>
> On 14/11/2016 22:08, Andrew Jones wrote:
> > Allow user to select who sends ipis and with which irq,
> > rather than just always sending irq=0 from cpu0.
> From a user point of view is there a way to know the list of available
> tests and their arg?
Not at the moment. I could change the arg-less run and/or add
a '-h' though that would provide a usage. So far unit tests
have never had -h's, but it's not a bad idea.
> >
> > Signed-off-by: Andrew Jones <drjones@redhat.com>
> >
> > ---
> > v6:
> > - make sender/irq names more future-proof [drew]
> > - sanity check inputs [drew]
> > - introduce check_sender/irq and bad_sender/irq to more
> > cleanly do checks [drew]
> > - default sender and irq to 1, instead of still zero [drew]
> > v4: improve structure and make sure spurious checking is
> > done even when the sender isn't cpu0
> > v2: actually check that the irq received was the irq sent,
> > and (for gicv2) that the sender is the expected one.
> > ---
> > arm/gic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
> > 1 file changed, 99 insertions(+), 25 deletions(-)
> >
> > diff --git a/arm/gic.c b/arm/gic.c
> > index d954a3775c26..638b8b140c96 100644
> > --- a/arm/gic.c
> > +++ b/arm/gic.c
> > @@ -11,6 +11,7 @@
> > * This work is licensed under the terms of the GNU LGPL, version 2.
> > */
> > #include <libcflat.h>
> > +#include <util.h>
> > #include <asm/setup.h>
> > #include <asm/processor.h>
> > #include <asm/gic.h>
> > @@ -28,6 +29,8 @@ struct gic {
> >
> > static struct gic *gic;
> > static int acked[NR_CPUS], spurious[NR_CPUS];
> > +static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
> > +static int cmdl_sender = 1, cmdl_irq = 1;
> > static cpumask_t ready;
> >
> > static void nr_cpu_check(int nr)
> > @@ -43,10 +46,23 @@ static void wait_on_ready(void)
> > cpu_relax();
> > }
> >
> > +static void stats_reset(void)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < nr_cpus; ++i) {
> > + acked[i] = 0;
> > + bad_sender[i] = -1;
> > + bad_irq[i] = -1;
> > + }
> > + smp_wmb();
> > +}
> > +
> > static void check_acked(cpumask_t *mask)
> > {
> > int missing = 0, extra = 0, unexpected = 0;
> > int nr_pass, cpu, i;
> > + bool bad = false;
> >
> > /* Wait up to 5s for all interrupts to be delivered */
> > for (i = 0; i < 50; ++i) {
> > @@ -56,9 +72,21 @@ static void check_acked(cpumask_t *mask)
> > smp_rmb();
> > nr_pass += cpumask_test_cpu(cpu, mask) ?
> > acked[cpu] == 1 : acked[cpu] == 0;
> > +
> > + if (bad_sender[cpu] != -1) {
> > + printf("cpu%d received IPI from wrong sender %d\n",
> > + cpu, bad_sender[cpu]);
> > + bad = true;
> > + }
> > +
> > + if (bad_irq[cpu] != -1) {
> > + printf("cpu%d received wrong irq %d\n",
> > + cpu, bad_irq[cpu]);
> > + bad = true;
> > + }
> > }
> > if (nr_pass == nr_cpus) {
> > - report("Completed in %d ms", true, ++i * 100);
> > + report("Completed in %d ms", !bad, ++i * 100);
> > return;
> > }
> > }
> > @@ -91,6 +119,22 @@ static void check_spurious(void)
> > }
> > }
> >
> > +static void check_ipi_sender(u32 irqstat)
> > +{
> > + if (gic_version() == 2) {
> > + int src = (irqstat >> 10) & 7;
> > +
> > + if (src != cmdl_sender)
> > + bad_sender[smp_processor_id()] = src;
> > + }
> > +}
> > +
> > +static void check_irqnr(u32 irqnr)
> > +{
> > + if (irqnr != (u32)cmdl_irq)
> > + bad_irq[smp_processor_id()] = irqnr;
> > +}
> > +
> > static void ipi_handler(struct pt_regs *regs __unused)
> > {
> > u32 irqstat = gic_read_iar();
> > @@ -98,8 +142,10 @@ static void ipi_handler(struct pt_regs *regs __unused)
> >
> > if (irqnr != GICC_INT_SPURIOUS) {
> > gic_write_eoir(irqstat);
> > - smp_rmb(); /* pairs with wmb in ipi_test functions */
> > + smp_rmb(); /* pairs with wmb in stats_reset */
> > ++acked[smp_processor_id()];
> > + check_ipi_sender(irqstat);
> > + check_irqnr(irqnr);
> > smp_wmb(); /* pairs with rmb in check_acked */
> > } else {
> > ++spurious[smp_processor_id()];
> > @@ -109,19 +155,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
> >
> > static void gicv2_ipi_send_self(void)
> > {
> > - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> > + writel(2 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> > }
> >
> > -static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
> > +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq)
> > {
> > u8 tlist = (u8)cpumask_bits(mask)[0];
> >
> > - writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> > + writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
> > }
> >
> > static void gicv2_ipi_send_broadcast(void)
> > {
> > - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> > + writel(1 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> > }
> >
> > static void gicv3_ipi_send_self(void)
> > @@ -130,12 +176,12 @@ static void gicv3_ipi_send_self(void)
> >
> > cpumask_clear(&mask);
> > cpumask_set_cpu(smp_processor_id(), &mask);
> > - gicv3_ipi_send_tlist(&mask, 0);
> > + gicv3_ipi_send_tlist(&mask, cmdl_irq);
> > }
> >
> > static void gicv3_ipi_send_broadcast(void)
> > {
> > - gicv3_write_sgi1r(1ULL << 40);
> > + gicv3_write_sgi1r(1ULL << 40 | cmdl_irq << 24);
> > isb();
> > }
> >
> > @@ -144,10 +190,9 @@ static void ipi_test_self(void)
> > cpumask_t mask;
> >
> > report_prefix_push("self");
> > - memset(acked, 0, sizeof(acked));
> > - smp_wmb();
> > + stats_reset();
> > cpumask_clear(&mask);
> > - cpumask_set_cpu(0, &mask);
> > + cpumask_set_cpu(smp_processor_id(), &mask);
> > gic->ipi.send_self();
> > check_acked(&mask);
> > report_prefix_pop();
> > @@ -159,20 +204,18 @@ static void ipi_test_smp(void)
> > int i;
> >
> > report_prefix_push("target-list");
> > - memset(acked, 0, sizeof(acked));
> > - smp_wmb();
> > + stats_reset();
> > cpumask_copy(&mask, &cpu_present_mask);
> > - for (i = 0; i < nr_cpus; i += 2)
> > + for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
> > cpumask_clear_cpu(i, &mask);
> > - gic->ipi.send_tlist(&mask, 0);
> > + gic->ipi.send_tlist(&mask, cmdl_irq);
> > check_acked(&mask);
> > report_prefix_pop();
> >
> > report_prefix_push("broadcast");
> > - memset(acked, 0, sizeof(acked));
> > - smp_wmb();
> > + stats_reset();
> > cpumask_copy(&mask, &cpu_present_mask);
> > - cpumask_clear_cpu(0, &mask);
> > + cpumask_clear_cpu(smp_processor_id(), &mask);
> > gic->ipi.send_broadcast();
> > check_acked(&mask);
> > report_prefix_pop();
> > @@ -189,6 +232,16 @@ static void ipi_enable(void)
> > local_irq_enable();
> > }
> >
> > +static void ipi_send(void)
> > +{
> > + ipi_enable();
> > + wait_on_ready();
> > + ipi_test_self();
> > + ipi_test_smp();
> > + check_spurious();
> > + exit(report_summary());
> > +}
> > +
> > static void ipi_recv(void)
> > {
> > ipi_enable();
> > @@ -197,6 +250,14 @@ static void ipi_recv(void)
> > wfi();
> > }
> >
> > +static void ipi_test(void)
> > +{
> > + if (smp_processor_id() == cmdl_sender)
> > + ipi_send();
> > + else
> > + ipi_recv();
> > +}
> > +
> > static struct gic gicv2 = {
> > .ipi = {
> > .send_self = gicv2_ipi_send_self,
> > @@ -242,21 +303,34 @@ int main(int argc, char **argv)
> > report_prefix_pop();
> >
> > } else if (strcmp(argv[1], "ipi") == 0) {
> > + int off, i = 1;
> > + long val;
> >
> > report_prefix_push(argv[1]);
> > nr_cpu_check(2);
> >
> > + while (--argc != 1) {
> > + off = parse_keyval(argv[++i], &val);
> > + if (off == -1)
> > + continue;
> > + argv[i][off] = '\0';
> > + if (strcmp(argv[i], "sender") == 0) {
> > + if (val >= nr_cpus)
> > + report_abort("invalid sender %d, nr_cpus=%d", val, nr_cpus);
> > + cmdl_sender = val;
> > + } else if (strcmp(argv[i], "irq") == 0) {
> > + if (val > 15)
> > + report_abort("irq (SGI) must be < 16");
> > + cmdl_irq = val;
> > + }
> wrong arg could potentially be tested.
What do you mean by 'wrong arg'? Can you give me an example?
>
> Besides
> Reviewed-by: Eric Auger <eric.auger@redhat.com>
Thanks!
drew
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero
@ 2016-11-23 13:01 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-23 13:01 UTC (permalink / raw)
To: Auger Eric
Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
peter.maydell, alex.bennee, marc.zyngier, christoffer.dall
On Wed, Nov 23, 2016 at 12:28:34PM +0100, Auger Eric wrote:
> Hi,
>
> On 14/11/2016 22:08, Andrew Jones wrote:
> > Allow user to select who sends ipis and with which irq,
> > rather than just always sending irq=0 from cpu0.
> From a user point of view is there a way to know the list of available
> tests and their arg?
Not at the moment. I could change the arg-less run and/or add
a '-h' though that would provide a usage. So far unit tests
have never had -h's, but it's not a bad idea.
> >
> > Signed-off-by: Andrew Jones <drjones@redhat.com>
> >
> > ---
> > v6:
> > - make sender/irq names more future-proof [drew]
> > - sanity check inputs [drew]
> > - introduce check_sender/irq and bad_sender/irq to more
> > cleanly do checks [drew]
> > - default sender and irq to 1, instead of still zero [drew]
> > v4: improve structure and make sure spurious checking is
> > done even when the sender isn't cpu0
> > v2: actually check that the irq received was the irq sent,
> > and (for gicv2) that the sender is the expected one.
> > ---
> > arm/gic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
> > 1 file changed, 99 insertions(+), 25 deletions(-)
> >
> > diff --git a/arm/gic.c b/arm/gic.c
> > index d954a3775c26..638b8b140c96 100644
> > --- a/arm/gic.c
> > +++ b/arm/gic.c
> > @@ -11,6 +11,7 @@
> > * This work is licensed under the terms of the GNU LGPL, version 2.
> > */
> > #include <libcflat.h>
> > +#include <util.h>
> > #include <asm/setup.h>
> > #include <asm/processor.h>
> > #include <asm/gic.h>
> > @@ -28,6 +29,8 @@ struct gic {
> >
> > static struct gic *gic;
> > static int acked[NR_CPUS], spurious[NR_CPUS];
> > +static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
> > +static int cmdl_sender = 1, cmdl_irq = 1;
> > static cpumask_t ready;
> >
> > static void nr_cpu_check(int nr)
> > @@ -43,10 +46,23 @@ static void wait_on_ready(void)
> > cpu_relax();
> > }
> >
> > +static void stats_reset(void)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < nr_cpus; ++i) {
> > + acked[i] = 0;
> > + bad_sender[i] = -1;
> > + bad_irq[i] = -1;
> > + }
> > + smp_wmb();
> > +}
> > +
> > static void check_acked(cpumask_t *mask)
> > {
> > int missing = 0, extra = 0, unexpected = 0;
> > int nr_pass, cpu, i;
> > + bool bad = false;
> >
> > /* Wait up to 5s for all interrupts to be delivered */
> > for (i = 0; i < 50; ++i) {
> > @@ -56,9 +72,21 @@ static void check_acked(cpumask_t *mask)
> > smp_rmb();
> > nr_pass += cpumask_test_cpu(cpu, mask) ?
> > acked[cpu] == 1 : acked[cpu] == 0;
> > +
> > + if (bad_sender[cpu] != -1) {
> > + printf("cpu%d received IPI from wrong sender %d\n",
> > + cpu, bad_sender[cpu]);
> > + bad = true;
> > + }
> > +
> > + if (bad_irq[cpu] != -1) {
> > + printf("cpu%d received wrong irq %d\n",
> > + cpu, bad_irq[cpu]);
> > + bad = true;
> > + }
> > }
> > if (nr_pass == nr_cpus) {
> > - report("Completed in %d ms", true, ++i * 100);
> > + report("Completed in %d ms", !bad, ++i * 100);
> > return;
> > }
> > }
> > @@ -91,6 +119,22 @@ static void check_spurious(void)
> > }
> > }
> >
> > +static void check_ipi_sender(u32 irqstat)
> > +{
> > + if (gic_version() == 2) {
> > + int src = (irqstat >> 10) & 7;
> > +
> > + if (src != cmdl_sender)
> > + bad_sender[smp_processor_id()] = src;
> > + }
> > +}
> > +
> > +static void check_irqnr(u32 irqnr)
> > +{
> > + if (irqnr != (u32)cmdl_irq)
> > + bad_irq[smp_processor_id()] = irqnr;
> > +}
> > +
> > static void ipi_handler(struct pt_regs *regs __unused)
> > {
> > u32 irqstat = gic_read_iar();
> > @@ -98,8 +142,10 @@ static void ipi_handler(struct pt_regs *regs __unused)
> >
> > if (irqnr != GICC_INT_SPURIOUS) {
> > gic_write_eoir(irqstat);
> > - smp_rmb(); /* pairs with wmb in ipi_test functions */
> > + smp_rmb(); /* pairs with wmb in stats_reset */
> > ++acked[smp_processor_id()];
> > + check_ipi_sender(irqstat);
> > + check_irqnr(irqnr);
> > smp_wmb(); /* pairs with rmb in check_acked */
> > } else {
> > ++spurious[smp_processor_id()];
> > @@ -109,19 +155,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
> >
> > static void gicv2_ipi_send_self(void)
> > {
> > - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> > + writel(2 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> > }
> >
> > -static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
> > +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq)
> > {
> > u8 tlist = (u8)cpumask_bits(mask)[0];
> >
> > - writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> > + writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
> > }
> >
> > static void gicv2_ipi_send_broadcast(void)
> > {
> > - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> > + writel(1 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> > }
> >
> > static void gicv3_ipi_send_self(void)
> > @@ -130,12 +176,12 @@ static void gicv3_ipi_send_self(void)
> >
> > cpumask_clear(&mask);
> > cpumask_set_cpu(smp_processor_id(), &mask);
> > - gicv3_ipi_send_tlist(&mask, 0);
> > + gicv3_ipi_send_tlist(&mask, cmdl_irq);
> > }
> >
> > static void gicv3_ipi_send_broadcast(void)
> > {
> > - gicv3_write_sgi1r(1ULL << 40);
> > + gicv3_write_sgi1r(1ULL << 40 | cmdl_irq << 24);
> > isb();
> > }
> >
> > @@ -144,10 +190,9 @@ static void ipi_test_self(void)
> > cpumask_t mask;
> >
> > report_prefix_push("self");
> > - memset(acked, 0, sizeof(acked));
> > - smp_wmb();
> > + stats_reset();
> > cpumask_clear(&mask);
> > - cpumask_set_cpu(0, &mask);
> > + cpumask_set_cpu(smp_processor_id(), &mask);
> > gic->ipi.send_self();
> > check_acked(&mask);
> > report_prefix_pop();
> > @@ -159,20 +204,18 @@ static void ipi_test_smp(void)
> > int i;
> >
> > report_prefix_push("target-list");
> > - memset(acked, 0, sizeof(acked));
> > - smp_wmb();
> > + stats_reset();
> > cpumask_copy(&mask, &cpu_present_mask);
> > - for (i = 0; i < nr_cpus; i += 2)
> > + for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
> > cpumask_clear_cpu(i, &mask);
> > - gic->ipi.send_tlist(&mask, 0);
> > + gic->ipi.send_tlist(&mask, cmdl_irq);
> > check_acked(&mask);
> > report_prefix_pop();
> >
> > report_prefix_push("broadcast");
> > - memset(acked, 0, sizeof(acked));
> > - smp_wmb();
> > + stats_reset();
> > cpumask_copy(&mask, &cpu_present_mask);
> > - cpumask_clear_cpu(0, &mask);
> > + cpumask_clear_cpu(smp_processor_id(), &mask);
> > gic->ipi.send_broadcast();
> > check_acked(&mask);
> > report_prefix_pop();
> > @@ -189,6 +232,16 @@ static void ipi_enable(void)
> > local_irq_enable();
> > }
> >
> > +static void ipi_send(void)
> > +{
> > + ipi_enable();
> > + wait_on_ready();
> > + ipi_test_self();
> > + ipi_test_smp();
> > + check_spurious();
> > + exit(report_summary());
> > +}
> > +
> > static void ipi_recv(void)
> > {
> > ipi_enable();
> > @@ -197,6 +250,14 @@ static void ipi_recv(void)
> > wfi();
> > }
> >
> > +static void ipi_test(void)
> > +{
> > + if (smp_processor_id() == cmdl_sender)
> > + ipi_send();
> > + else
> > + ipi_recv();
> > +}
> > +
> > static struct gic gicv2 = {
> > .ipi = {
> > .send_self = gicv2_ipi_send_self,
> > @@ -242,21 +303,34 @@ int main(int argc, char **argv)
> > report_prefix_pop();
> >
> > } else if (strcmp(argv[1], "ipi") == 0) {
> > + int off, i = 1;
> > + long val;
> >
> > report_prefix_push(argv[1]);
> > nr_cpu_check(2);
> >
> > + while (--argc != 1) {
> > + off = parse_keyval(argv[++i], &val);
> > + if (off == -1)
> > + continue;
> > + argv[i][off] = '\0';
> > + if (strcmp(argv[i], "sender") == 0) {
> > + if (val >= nr_cpus)
> > + report_abort("invalid sender %d, nr_cpus=%d", val, nr_cpus);
> > + cmdl_sender = val;
> > + } else if (strcmp(argv[i], "irq") == 0) {
> > + if (val > 15)
> > + report_abort("irq (SGI) must be < 16");
> > + cmdl_irq = val;
> > + }
> wrong arg could potentially be tested.
What do you mean by 'wrong arg'? Can you give me an example?
>
> Besides
> Reviewed-by: Eric Auger <eric.auger@redhat.com>
Thanks!
drew
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework
2016-11-23 11:33 ` Auger Eric
(?)
@ 2016-11-23 13:08 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-23 13:08 UTC (permalink / raw)
To: Auger Eric
Cc: Alex Bennée, peter.maydell, kvm, marc.zyngier,
andre.przywara, qemu-devel, qemu-arm, pbonzini, kvmarm,
christoffer.dall
On Wed, Nov 23, 2016 at 12:33:53PM +0100, Auger Eric wrote:
> Hi,
> On 23/11/2016 11:09, Alex Bennée wrote:
> >
> > Andrew Jones <drjones@redhat.com> writes:
> >
> >> Andre, Alex, Eric, anybody,
> >>
> >> Any more comments on this? If not, I'll send a pull request
> >> to Radim and Paolo to finally get this merged.
> >
> > Looks good to me. I successfully re-based my TCG tests on top and they
> > work fine ;-)
> no blocking point for me either. That's already a huge work and when
> adding new tests we may end up rationalizing things in separate v2/v3
> lib files and ops.
>
> Tested on Cavium ThunderX
>
> Reviewed-by: Eric Auger <eric.auger@redhat.com>
> Tested-by: Eric Auger <eric.auger@redhat.com>
Thanks Eric!
I like your suggestions for "gicv3: add an IPI test", so I'll
spin a v7.
drew
>
> Thanks
>
> Eric
>
>
> >
> > Tested-by: Alex Bennée <alex.bennee@linaro.org>
> >
> >>
> >> Thanks,
> >> drew
> >>
> >>
> >> On Mon, Nov 14, 2016 at 10:08:28PM +0100, Andrew Jones wrote:
> >>> v6:
> >>> - rebased to latest master
> >>> - several other changes thanks to Andre and Alex, changes in
> >>> individual patch change logs
> >>> - some code cleanups
> >>>
> >>> v5:
> >>> - fix arm32/gicv3 compile [drew]
> >>> - use modern register names [Andre]
> >>> - one Andre r-b
> >>>
> >>> v4:
> >>> - Eric's r-b's
> >>> - Andre's suggestion to only take defines we need
> >>> - several other changes listed in individual patches
> >>>
> >>> v3:
> >>> - Rebased on latest master
> >>> - Added Alex's r-b's
> >>>
> >>> v2:
> >>> Rebased on latest master + my "populate argv[0]" series (will
> >>> send a REPOST for that shortly. Additionally a few patches got
> >>> fixes/features;
> >>> 07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
> >>> all interrupts as non-secure Group-1" in order to continue
> >>> working over TCG, as the gicv3 code for TCG removed a hack
> >>> it had there to make Linux happy.
> >>> 08/10 added more output for when things fail (if they fail)
> >>> 09/10 switched gicv3 broadcast implementation to using IRM. This
> >>> found a bug in a recent (but not tip) kernel, which I was
> >>> about to fix, but then I saw MarcZ beat me to it.
> >>> 10/10 actually check that the input irq is the received irq
> >>>
> >>>
> >>> Import defines, and steal enough helper functions, from Linux to
> >>> enable programming of the gic (v2 and v3). Then use the framework
> >>> to add an initial test (an ipi test; self, target-list, broadcast).
> >>>
> >>> It's my hope that this framework will be a suitable base on which
> >>> more tests may be easily added, particularly because we have
> >>> vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
> >>> vgic-new and tcg gicv3 are merged now)
> >>>
> >>> To run it, along with other tests, just do
> >>>
> >>> ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
> >>> make
> >>> export QEMU=$PATH_TO_QEMU
> >>> ./run_tests.sh
> >>>
> >>> To run it separately do, e.g.
> >>>
> >>> $QEMU -machine virt,accel=tcg -cpu cortex-a57 \
> >>> -device virtio-serial-device \
> >>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
> >>> -display none -serial stdio \
> >>> -kernel arm/gic.flat \
> >>> -smp 123 -machine gic-version=3 -append ipi
> >>> ^^ note, we can go nuts with nr-cpus on TCG :-)
> >>>
> >>> Or, a KVM example using a different "sender" cpu and irq (other than zero)
> >>>
> >>> $QEMU -machine virt,accel=kvm -cpu host \
> >>> -device virtio-serial-device \
> >>> -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
> >>> -display none -serial stdio \
> >>> -kernel arm/gic.flat \
> >>> -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
> >>>
> >>>
> >>> Patches:
> >>> 01-05: fixes and functionality needed by the later gic patches
> >>> 06-07: enable gicv2 and gicv2 IPI test
> >>> 08-10: enable gicv3 and gicv3 IPI test
> >>> 11: extend the IPI tests to take variable sender and irq
> >>>
> >>> Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
> >>>
> >>>
> >>> Andrew Jones (10):
> >>> lib: xstr: allow multiple args
> >>> arm64: fix get_"sysreg32" and make MPIDR 64bit
> >>> arm/arm64: smp: support more than 8 cpus
> >>> arm/arm64: add some delay routines
> >>> arm/arm64: irq enable/disable
> >>> arm/arm64: add initial gicv2 support
> >>> arm/arm64: gicv2: add an IPI test
> >>> arm/arm64: add initial gicv3 support
> >>> arm/arm64: gicv3: add an IPI test
> >>> arm/arm64: gic: don't just use zero
> >>>
> >>> Peter Xu (1):
> >>> libcflat: add IS_ALIGNED() macro, and page sizes
> >>>
> >>> arm/Makefile.common | 9 +-
> >>> arm/gic.c | 340 +++++++++++++++++++++++++++++++++++++++++++++
> >>> arm/run | 19 ++-
> >>> arm/selftest.c | 5 +-
> >>> arm/unittests.cfg | 14 ++
> >>> lib/arm/asm/arch_gicv3.h | 70 ++++++++++
> >>> lib/arm/asm/gic-v2.h | 36 +++++
> >>> lib/arm/asm/gic-v3.h | 112 +++++++++++++++
> >>> lib/arm/asm/gic.h | 106 ++++++++++++++
> >>> lib/arm/asm/processor.h | 42 +++++-
> >>> lib/arm/asm/setup.h | 4 +-
> >>> lib/arm/gic.c | 267 +++++++++++++++++++++++++++++++++++
> >>> lib/arm/processor.c | 15 ++
> >>> lib/arm/setup.c | 10 ++
> >>> lib/arm64/asm/arch_gicv3.h | 66 +++++++++
> >>> lib/arm64/asm/gic-v2.h | 1 +
> >>> lib/arm64/asm/gic-v3.h | 1 +
> >>> lib/arm64/asm/gic.h | 1 +
> >>> lib/arm64/asm/processor.h | 53 +++++--
> >>> lib/arm64/asm/sysreg.h | 44 ++++++
> >>> lib/arm64/processor.c | 15 ++
> >>> lib/libcflat.h | 10 +-
> >>> 22 files changed, 1212 insertions(+), 28 deletions(-)
> >>> create mode 100644 arm/gic.c
> >>> create mode 100644 lib/arm/asm/arch_gicv3.h
> >>> create mode 100644 lib/arm/asm/gic-v2.h
> >>> create mode 100644 lib/arm/asm/gic-v3.h
> >>> create mode 100644 lib/arm/asm/gic.h
> >>> create mode 100644 lib/arm/gic.c
> >>> create mode 100644 lib/arm64/asm/arch_gicv3.h
> >>> create mode 100644 lib/arm64/asm/gic-v2.h
> >>> create mode 100644 lib/arm64/asm/gic-v3.h
> >>> create mode 100644 lib/arm64/asm/gic.h
> >>> create mode 100644 lib/arm64/asm/sysreg.h
> >>>
> >>> --
> >>> 2.7.4
> >>>
> >>>
> >
> >
> > --
> > Alex Bennée
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero
2016-11-23 13:01 ` [Qemu-devel] " Andrew Jones
@ 2016-11-23 13:33 ` Auger Eric
-1 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 13:33 UTC (permalink / raw)
To: Andrew Jones
Cc: kvm, marc.zyngier, andre.przywara, qemu-devel, qemu-arm,
pbonzini, kvmarm
On 23/11/2016 14:01, Andrew Jones wrote:
> On Wed, Nov 23, 2016 at 12:28:34PM +0100, Auger Eric wrote:
>> Hi,
>>
>> On 14/11/2016 22:08, Andrew Jones wrote:
>>> Allow user to select who sends ipis and with which irq,
>>> rather than just always sending irq=0 from cpu0.
>> From a user point of view is there a way to know the list of available
>> tests and their arg?
>
> Not at the moment. I could change the arg-less run and/or add
> a '-h' though that would provide a usage. So far unit tests
> have never had -h's, but it's not a bad idea.
OK thanks for the info
>
>>>
>>> Signed-off-by: Andrew Jones <drjones@redhat.com>
>>>
>>> ---
>>> v6:
>>> - make sender/irq names more future-proof [drew]
>>> - sanity check inputs [drew]
>>> - introduce check_sender/irq and bad_sender/irq to more
>>> cleanly do checks [drew]
>>> - default sender and irq to 1, instead of still zero [drew]
>>> v4: improve structure and make sure spurious checking is
>>> done even when the sender isn't cpu0
>>> v2: actually check that the irq received was the irq sent,
>>> and (for gicv2) that the sender is the expected one.
>>> ---
>>> arm/gic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
>>> 1 file changed, 99 insertions(+), 25 deletions(-)
>>>
>>> diff --git a/arm/gic.c b/arm/gic.c
>>> index d954a3775c26..638b8b140c96 100644
>>> --- a/arm/gic.c
>>> +++ b/arm/gic.c
>>> @@ -11,6 +11,7 @@
>>> * This work is licensed under the terms of the GNU LGPL, version 2.
>>> */
>>> #include <libcflat.h>
>>> +#include <util.h>
>>> #include <asm/setup.h>
>>> #include <asm/processor.h>
>>> #include <asm/gic.h>
>>> @@ -28,6 +29,8 @@ struct gic {
>>>
>>> static struct gic *gic;
>>> static int acked[NR_CPUS], spurious[NR_CPUS];
>>> +static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
>>> +static int cmdl_sender = 1, cmdl_irq = 1;
>>> static cpumask_t ready;
>>>
>>> static void nr_cpu_check(int nr)
>>> @@ -43,10 +46,23 @@ static void wait_on_ready(void)
>>> cpu_relax();
>>> }
>>>
>>> +static void stats_reset(void)
>>> +{
>>> + int i;
>>> +
>>> + for (i = 0; i < nr_cpus; ++i) {
>>> + acked[i] = 0;
>>> + bad_sender[i] = -1;
>>> + bad_irq[i] = -1;
>>> + }
>>> + smp_wmb();
>>> +}
>>> +
>>> static void check_acked(cpumask_t *mask)
>>> {
>>> int missing = 0, extra = 0, unexpected = 0;
>>> int nr_pass, cpu, i;
>>> + bool bad = false;
>>>
>>> /* Wait up to 5s for all interrupts to be delivered */
>>> for (i = 0; i < 50; ++i) {
>>> @@ -56,9 +72,21 @@ static void check_acked(cpumask_t *mask)
>>> smp_rmb();
>>> nr_pass += cpumask_test_cpu(cpu, mask) ?
>>> acked[cpu] == 1 : acked[cpu] == 0;
>>> +
>>> + if (bad_sender[cpu] != -1) {
>>> + printf("cpu%d received IPI from wrong sender %d\n",
>>> + cpu, bad_sender[cpu]);
>>> + bad = true;
>>> + }
>>> +
>>> + if (bad_irq[cpu] != -1) {
>>> + printf("cpu%d received wrong irq %d\n",
>>> + cpu, bad_irq[cpu]);
>>> + bad = true;
>>> + }
>>> }
>>> if (nr_pass == nr_cpus) {
>>> - report("Completed in %d ms", true, ++i * 100);
>>> + report("Completed in %d ms", !bad, ++i * 100);
>>> return;
>>> }
>>> }
>>> @@ -91,6 +119,22 @@ static void check_spurious(void)
>>> }
>>> }
>>>
>>> +static void check_ipi_sender(u32 irqstat)
>>> +{
>>> + if (gic_version() == 2) {
>>> + int src = (irqstat >> 10) & 7;
>>> +
>>> + if (src != cmdl_sender)
>>> + bad_sender[smp_processor_id()] = src;
>>> + }
>>> +}
>>> +
>>> +static void check_irqnr(u32 irqnr)
>>> +{
>>> + if (irqnr != (u32)cmdl_irq)
>>> + bad_irq[smp_processor_id()] = irqnr;
>>> +}
>>> +
>>> static void ipi_handler(struct pt_regs *regs __unused)
>>> {
>>> u32 irqstat = gic_read_iar();
>>> @@ -98,8 +142,10 @@ static void ipi_handler(struct pt_regs *regs __unused)
>>>
>>> if (irqnr != GICC_INT_SPURIOUS) {
>>> gic_write_eoir(irqstat);
>>> - smp_rmb(); /* pairs with wmb in ipi_test functions */
>>> + smp_rmb(); /* pairs with wmb in stats_reset */
>>> ++acked[smp_processor_id()];
>>> + check_ipi_sender(irqstat);
>>> + check_irqnr(irqnr);
>>> smp_wmb(); /* pairs with rmb in check_acked */
>>> } else {
>>> ++spurious[smp_processor_id()];
>>> @@ -109,19 +155,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
>>>
>>> static void gicv2_ipi_send_self(void)
>>> {
>>> - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
>>> + writel(2 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
>>> }
>>>
>>> -static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
>>> +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq)
>>> {
>>> u8 tlist = (u8)cpumask_bits(mask)[0];
>>>
>>> - writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
>>> + writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
>>> }
>>>
>>> static void gicv2_ipi_send_broadcast(void)
>>> {
>>> - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
>>> + writel(1 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
>>> }
>>>
>>> static void gicv3_ipi_send_self(void)
>>> @@ -130,12 +176,12 @@ static void gicv3_ipi_send_self(void)
>>>
>>> cpumask_clear(&mask);
>>> cpumask_set_cpu(smp_processor_id(), &mask);
>>> - gicv3_ipi_send_tlist(&mask, 0);
>>> + gicv3_ipi_send_tlist(&mask, cmdl_irq);
>>> }
>>>
>>> static void gicv3_ipi_send_broadcast(void)
>>> {
>>> - gicv3_write_sgi1r(1ULL << 40);
>>> + gicv3_write_sgi1r(1ULL << 40 | cmdl_irq << 24);
>>> isb();
>>> }
>>>
>>> @@ -144,10 +190,9 @@ static void ipi_test_self(void)
>>> cpumask_t mask;
>>>
>>> report_prefix_push("self");
>>> - memset(acked, 0, sizeof(acked));
>>> - smp_wmb();
>>> + stats_reset();
>>> cpumask_clear(&mask);
>>> - cpumask_set_cpu(0, &mask);
>>> + cpumask_set_cpu(smp_processor_id(), &mask);
>>> gic->ipi.send_self();
>>> check_acked(&mask);
>>> report_prefix_pop();
>>> @@ -159,20 +204,18 @@ static void ipi_test_smp(void)
>>> int i;
>>>
>>> report_prefix_push("target-list");
>>> - memset(acked, 0, sizeof(acked));
>>> - smp_wmb();
>>> + stats_reset();
>>> cpumask_copy(&mask, &cpu_present_mask);
>>> - for (i = 0; i < nr_cpus; i += 2)
>>> + for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
>>> cpumask_clear_cpu(i, &mask);
>>> - gic->ipi.send_tlist(&mask, 0);
>>> + gic->ipi.send_tlist(&mask, cmdl_irq);
>>> check_acked(&mask);
>>> report_prefix_pop();
>>>
>>> report_prefix_push("broadcast");
>>> - memset(acked, 0, sizeof(acked));
>>> - smp_wmb();
>>> + stats_reset();
>>> cpumask_copy(&mask, &cpu_present_mask);
>>> - cpumask_clear_cpu(0, &mask);
>>> + cpumask_clear_cpu(smp_processor_id(), &mask);
>>> gic->ipi.send_broadcast();
>>> check_acked(&mask);
>>> report_prefix_pop();
>>> @@ -189,6 +232,16 @@ static void ipi_enable(void)
>>> local_irq_enable();
>>> }
>>>
>>> +static void ipi_send(void)
>>> +{
>>> + ipi_enable();
>>> + wait_on_ready();
>>> + ipi_test_self();
>>> + ipi_test_smp();
>>> + check_spurious();
>>> + exit(report_summary());
>>> +}
>>> +
>>> static void ipi_recv(void)
>>> {
>>> ipi_enable();
>>> @@ -197,6 +250,14 @@ static void ipi_recv(void)
>>> wfi();
>>> }
>>>
>>> +static void ipi_test(void)
>>> +{
>>> + if (smp_processor_id() == cmdl_sender)
>>> + ipi_send();
>>> + else
>>> + ipi_recv();
>>> +}
>>> +
>>> static struct gic gicv2 = {
>>> .ipi = {
>>> .send_self = gicv2_ipi_send_self,
>>> @@ -242,21 +303,34 @@ int main(int argc, char **argv)
>>> report_prefix_pop();
>>>
>>> } else if (strcmp(argv[1], "ipi") == 0) {
>>> + int off, i = 1;
>>> + long val;
>>>
>>> report_prefix_push(argv[1]);
>>> nr_cpu_check(2);
>>>
>>> + while (--argc != 1) {
>>> + off = parse_keyval(argv[++i], &val);
>>> + if (off == -1)
>>> + continue;
>>> + argv[i][off] = '\0';
>>> + if (strcmp(argv[i], "sender") == 0) {
>>> + if (val >= nr_cpus)
>>> + report_abort("invalid sender %d, nr_cpus=%d", val, nr_cpus);
>>> + cmdl_sender = val;
>>> + } else if (strcmp(argv[i], "irq") == 0) {
>>> + if (val > 15)
>>> + report_abort("irq (SGI) must be < 16");
>>> + cmdl_irq = val;
>>> + }
>> wrong arg could potentially be tested.
>
> What do you mean by 'wrong arg'? Can you give me an example?
I meant bad option, ie if the tester makes a typo such as sende=3, this
gets ignored, right?.
Thanks
Eric
>
>>
>> Besides
>> Reviewed-by: Eric Auger <eric.auger@redhat.com>
>
> Thanks!
>
> drew
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero
@ 2016-11-23 13:33 ` Auger Eric
0 siblings, 0 replies; 52+ messages in thread
From: Auger Eric @ 2016-11-23 13:33 UTC (permalink / raw)
To: Andrew Jones
Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
peter.maydell, alex.bennee, marc.zyngier, christoffer.dall
On 23/11/2016 14:01, Andrew Jones wrote:
> On Wed, Nov 23, 2016 at 12:28:34PM +0100, Auger Eric wrote:
>> Hi,
>>
>> On 14/11/2016 22:08, Andrew Jones wrote:
>>> Allow user to select who sends ipis and with which irq,
>>> rather than just always sending irq=0 from cpu0.
>> From a user point of view is there a way to know the list of available
>> tests and their arg?
>
> Not at the moment. I could change the arg-less run and/or add
> a '-h' though that would provide a usage. So far unit tests
> have never had -h's, but it's not a bad idea.
OK thanks for the info
>
>>>
>>> Signed-off-by: Andrew Jones <drjones@redhat.com>
>>>
>>> ---
>>> v6:
>>> - make sender/irq names more future-proof [drew]
>>> - sanity check inputs [drew]
>>> - introduce check_sender/irq and bad_sender/irq to more
>>> cleanly do checks [drew]
>>> - default sender and irq to 1, instead of still zero [drew]
>>> v4: improve structure and make sure spurious checking is
>>> done even when the sender isn't cpu0
>>> v2: actually check that the irq received was the irq sent,
>>> and (for gicv2) that the sender is the expected one.
>>> ---
>>> arm/gic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
>>> 1 file changed, 99 insertions(+), 25 deletions(-)
>>>
>>> diff --git a/arm/gic.c b/arm/gic.c
>>> index d954a3775c26..638b8b140c96 100644
>>> --- a/arm/gic.c
>>> +++ b/arm/gic.c
>>> @@ -11,6 +11,7 @@
>>> * This work is licensed under the terms of the GNU LGPL, version 2.
>>> */
>>> #include <libcflat.h>
>>> +#include <util.h>
>>> #include <asm/setup.h>
>>> #include <asm/processor.h>
>>> #include <asm/gic.h>
>>> @@ -28,6 +29,8 @@ struct gic {
>>>
>>> static struct gic *gic;
>>> static int acked[NR_CPUS], spurious[NR_CPUS];
>>> +static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
>>> +static int cmdl_sender = 1, cmdl_irq = 1;
>>> static cpumask_t ready;
>>>
>>> static void nr_cpu_check(int nr)
>>> @@ -43,10 +46,23 @@ static void wait_on_ready(void)
>>> cpu_relax();
>>> }
>>>
>>> +static void stats_reset(void)
>>> +{
>>> + int i;
>>> +
>>> + for (i = 0; i < nr_cpus; ++i) {
>>> + acked[i] = 0;
>>> + bad_sender[i] = -1;
>>> + bad_irq[i] = -1;
>>> + }
>>> + smp_wmb();
>>> +}
>>> +
>>> static void check_acked(cpumask_t *mask)
>>> {
>>> int missing = 0, extra = 0, unexpected = 0;
>>> int nr_pass, cpu, i;
>>> + bool bad = false;
>>>
>>> /* Wait up to 5s for all interrupts to be delivered */
>>> for (i = 0; i < 50; ++i) {
>>> @@ -56,9 +72,21 @@ static void check_acked(cpumask_t *mask)
>>> smp_rmb();
>>> nr_pass += cpumask_test_cpu(cpu, mask) ?
>>> acked[cpu] == 1 : acked[cpu] == 0;
>>> +
>>> + if (bad_sender[cpu] != -1) {
>>> + printf("cpu%d received IPI from wrong sender %d\n",
>>> + cpu, bad_sender[cpu]);
>>> + bad = true;
>>> + }
>>> +
>>> + if (bad_irq[cpu] != -1) {
>>> + printf("cpu%d received wrong irq %d\n",
>>> + cpu, bad_irq[cpu]);
>>> + bad = true;
>>> + }
>>> }
>>> if (nr_pass == nr_cpus) {
>>> - report("Completed in %d ms", true, ++i * 100);
>>> + report("Completed in %d ms", !bad, ++i * 100);
>>> return;
>>> }
>>> }
>>> @@ -91,6 +119,22 @@ static void check_spurious(void)
>>> }
>>> }
>>>
>>> +static void check_ipi_sender(u32 irqstat)
>>> +{
>>> + if (gic_version() == 2) {
>>> + int src = (irqstat >> 10) & 7;
>>> +
>>> + if (src != cmdl_sender)
>>> + bad_sender[smp_processor_id()] = src;
>>> + }
>>> +}
>>> +
>>> +static void check_irqnr(u32 irqnr)
>>> +{
>>> + if (irqnr != (u32)cmdl_irq)
>>> + bad_irq[smp_processor_id()] = irqnr;
>>> +}
>>> +
>>> static void ipi_handler(struct pt_regs *regs __unused)
>>> {
>>> u32 irqstat = gic_read_iar();
>>> @@ -98,8 +142,10 @@ static void ipi_handler(struct pt_regs *regs __unused)
>>>
>>> if (irqnr != GICC_INT_SPURIOUS) {
>>> gic_write_eoir(irqstat);
>>> - smp_rmb(); /* pairs with wmb in ipi_test functions */
>>> + smp_rmb(); /* pairs with wmb in stats_reset */
>>> ++acked[smp_processor_id()];
>>> + check_ipi_sender(irqstat);
>>> + check_irqnr(irqnr);
>>> smp_wmb(); /* pairs with rmb in check_acked */
>>> } else {
>>> ++spurious[smp_processor_id()];
>>> @@ -109,19 +155,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
>>>
>>> static void gicv2_ipi_send_self(void)
>>> {
>>> - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
>>> + writel(2 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
>>> }
>>>
>>> -static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
>>> +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq)
>>> {
>>> u8 tlist = (u8)cpumask_bits(mask)[0];
>>>
>>> - writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
>>> + writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
>>> }
>>>
>>> static void gicv2_ipi_send_broadcast(void)
>>> {
>>> - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
>>> + writel(1 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
>>> }
>>>
>>> static void gicv3_ipi_send_self(void)
>>> @@ -130,12 +176,12 @@ static void gicv3_ipi_send_self(void)
>>>
>>> cpumask_clear(&mask);
>>> cpumask_set_cpu(smp_processor_id(), &mask);
>>> - gicv3_ipi_send_tlist(&mask, 0);
>>> + gicv3_ipi_send_tlist(&mask, cmdl_irq);
>>> }
>>>
>>> static void gicv3_ipi_send_broadcast(void)
>>> {
>>> - gicv3_write_sgi1r(1ULL << 40);
>>> + gicv3_write_sgi1r(1ULL << 40 | cmdl_irq << 24);
>>> isb();
>>> }
>>>
>>> @@ -144,10 +190,9 @@ static void ipi_test_self(void)
>>> cpumask_t mask;
>>>
>>> report_prefix_push("self");
>>> - memset(acked, 0, sizeof(acked));
>>> - smp_wmb();
>>> + stats_reset();
>>> cpumask_clear(&mask);
>>> - cpumask_set_cpu(0, &mask);
>>> + cpumask_set_cpu(smp_processor_id(), &mask);
>>> gic->ipi.send_self();
>>> check_acked(&mask);
>>> report_prefix_pop();
>>> @@ -159,20 +204,18 @@ static void ipi_test_smp(void)
>>> int i;
>>>
>>> report_prefix_push("target-list");
>>> - memset(acked, 0, sizeof(acked));
>>> - smp_wmb();
>>> + stats_reset();
>>> cpumask_copy(&mask, &cpu_present_mask);
>>> - for (i = 0; i < nr_cpus; i += 2)
>>> + for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
>>> cpumask_clear_cpu(i, &mask);
>>> - gic->ipi.send_tlist(&mask, 0);
>>> + gic->ipi.send_tlist(&mask, cmdl_irq);
>>> check_acked(&mask);
>>> report_prefix_pop();
>>>
>>> report_prefix_push("broadcast");
>>> - memset(acked, 0, sizeof(acked));
>>> - smp_wmb();
>>> + stats_reset();
>>> cpumask_copy(&mask, &cpu_present_mask);
>>> - cpumask_clear_cpu(0, &mask);
>>> + cpumask_clear_cpu(smp_processor_id(), &mask);
>>> gic->ipi.send_broadcast();
>>> check_acked(&mask);
>>> report_prefix_pop();
>>> @@ -189,6 +232,16 @@ static void ipi_enable(void)
>>> local_irq_enable();
>>> }
>>>
>>> +static void ipi_send(void)
>>> +{
>>> + ipi_enable();
>>> + wait_on_ready();
>>> + ipi_test_self();
>>> + ipi_test_smp();
>>> + check_spurious();
>>> + exit(report_summary());
>>> +}
>>> +
>>> static void ipi_recv(void)
>>> {
>>> ipi_enable();
>>> @@ -197,6 +250,14 @@ static void ipi_recv(void)
>>> wfi();
>>> }
>>>
>>> +static void ipi_test(void)
>>> +{
>>> + if (smp_processor_id() == cmdl_sender)
>>> + ipi_send();
>>> + else
>>> + ipi_recv();
>>> +}
>>> +
>>> static struct gic gicv2 = {
>>> .ipi = {
>>> .send_self = gicv2_ipi_send_self,
>>> @@ -242,21 +303,34 @@ int main(int argc, char **argv)
>>> report_prefix_pop();
>>>
>>> } else if (strcmp(argv[1], "ipi") == 0) {
>>> + int off, i = 1;
>>> + long val;
>>>
>>> report_prefix_push(argv[1]);
>>> nr_cpu_check(2);
>>>
>>> + while (--argc != 1) {
>>> + off = parse_keyval(argv[++i], &val);
>>> + if (off == -1)
>>> + continue;
>>> + argv[i][off] = '\0';
>>> + if (strcmp(argv[i], "sender") == 0) {
>>> + if (val >= nr_cpus)
>>> + report_abort("invalid sender %d, nr_cpus=%d", val, nr_cpus);
>>> + cmdl_sender = val;
>>> + } else if (strcmp(argv[i], "irq") == 0) {
>>> + if (val > 15)
>>> + report_abort("irq (SGI) must be < 16");
>>> + cmdl_irq = val;
>>> + }
>> wrong arg could potentially be tested.
>
> What do you mean by 'wrong arg'? Can you give me an example?
I meant bad option, ie if the tester makes a typo such as sende=3, this
gets ignored, right?.
Thanks
Eric
>
>>
>> Besides
>> Reviewed-by: Eric Auger <eric.auger@redhat.com>
>
> Thanks!
>
> drew
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero
2016-11-23 13:33 ` [Qemu-devel] " Auger Eric
@ 2016-11-23 13:46 ` Andrew Jones
-1 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-23 13:46 UTC (permalink / raw)
To: Auger Eric
Cc: kvm, marc.zyngier, andre.przywara, qemu-devel, qemu-arm,
pbonzini, kvmarm
On Wed, Nov 23, 2016 at 02:33:31PM +0100, Auger Eric wrote:
>
>
> On 23/11/2016 14:01, Andrew Jones wrote:
> > On Wed, Nov 23, 2016 at 12:28:34PM +0100, Auger Eric wrote:
> >> Hi,
> >>
> >> On 14/11/2016 22:08, Andrew Jones wrote:
> >>> Allow user to select who sends ipis and with which irq,
> >>> rather than just always sending irq=0 from cpu0.
> >> From a user point of view is there a way to know the list of available
> >> tests and their arg?
> >
> > Not at the moment. I could change the arg-less run and/or add
> > a '-h' though that would provide a usage. So far unit tests
> > have never had -h's, but it's not a bad idea.
>
> OK thanks for the info
> >
> >>>
> >>> Signed-off-by: Andrew Jones <drjones@redhat.com>
> >>>
> >>> ---
> >>> v6:
> >>> - make sender/irq names more future-proof [drew]
> >>> - sanity check inputs [drew]
> >>> - introduce check_sender/irq and bad_sender/irq to more
> >>> cleanly do checks [drew]
> >>> - default sender and irq to 1, instead of still zero [drew]
> >>> v4: improve structure and make sure spurious checking is
> >>> done even when the sender isn't cpu0
> >>> v2: actually check that the irq received was the irq sent,
> >>> and (for gicv2) that the sender is the expected one.
> >>> ---
> >>> arm/gic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
> >>> 1 file changed, 99 insertions(+), 25 deletions(-)
> >>>
> >>> diff --git a/arm/gic.c b/arm/gic.c
> >>> index d954a3775c26..638b8b140c96 100644
> >>> --- a/arm/gic.c
> >>> +++ b/arm/gic.c
> >>> @@ -11,6 +11,7 @@
> >>> * This work is licensed under the terms of the GNU LGPL, version 2.
> >>> */
> >>> #include <libcflat.h>
> >>> +#include <util.h>
> >>> #include <asm/setup.h>
> >>> #include <asm/processor.h>
> >>> #include <asm/gic.h>
> >>> @@ -28,6 +29,8 @@ struct gic {
> >>>
> >>> static struct gic *gic;
> >>> static int acked[NR_CPUS], spurious[NR_CPUS];
> >>> +static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
> >>> +static int cmdl_sender = 1, cmdl_irq = 1;
> >>> static cpumask_t ready;
> >>>
> >>> static void nr_cpu_check(int nr)
> >>> @@ -43,10 +46,23 @@ static void wait_on_ready(void)
> >>> cpu_relax();
> >>> }
> >>>
> >>> +static void stats_reset(void)
> >>> +{
> >>> + int i;
> >>> +
> >>> + for (i = 0; i < nr_cpus; ++i) {
> >>> + acked[i] = 0;
> >>> + bad_sender[i] = -1;
> >>> + bad_irq[i] = -1;
> >>> + }
> >>> + smp_wmb();
> >>> +}
> >>> +
> >>> static void check_acked(cpumask_t *mask)
> >>> {
> >>> int missing = 0, extra = 0, unexpected = 0;
> >>> int nr_pass, cpu, i;
> >>> + bool bad = false;
> >>>
> >>> /* Wait up to 5s for all interrupts to be delivered */
> >>> for (i = 0; i < 50; ++i) {
> >>> @@ -56,9 +72,21 @@ static void check_acked(cpumask_t *mask)
> >>> smp_rmb();
> >>> nr_pass += cpumask_test_cpu(cpu, mask) ?
> >>> acked[cpu] == 1 : acked[cpu] == 0;
> >>> +
> >>> + if (bad_sender[cpu] != -1) {
> >>> + printf("cpu%d received IPI from wrong sender %d\n",
> >>> + cpu, bad_sender[cpu]);
> >>> + bad = true;
> >>> + }
> >>> +
> >>> + if (bad_irq[cpu] != -1) {
> >>> + printf("cpu%d received wrong irq %d\n",
> >>> + cpu, bad_irq[cpu]);
> >>> + bad = true;
> >>> + }
> >>> }
> >>> if (nr_pass == nr_cpus) {
> >>> - report("Completed in %d ms", true, ++i * 100);
> >>> + report("Completed in %d ms", !bad, ++i * 100);
> >>> return;
> >>> }
> >>> }
> >>> @@ -91,6 +119,22 @@ static void check_spurious(void)
> >>> }
> >>> }
> >>>
> >>> +static void check_ipi_sender(u32 irqstat)
> >>> +{
> >>> + if (gic_version() == 2) {
> >>> + int src = (irqstat >> 10) & 7;
> >>> +
> >>> + if (src != cmdl_sender)
> >>> + bad_sender[smp_processor_id()] = src;
> >>> + }
> >>> +}
> >>> +
> >>> +static void check_irqnr(u32 irqnr)
> >>> +{
> >>> + if (irqnr != (u32)cmdl_irq)
> >>> + bad_irq[smp_processor_id()] = irqnr;
> >>> +}
> >>> +
> >>> static void ipi_handler(struct pt_regs *regs __unused)
> >>> {
> >>> u32 irqstat = gic_read_iar();
> >>> @@ -98,8 +142,10 @@ static void ipi_handler(struct pt_regs *regs __unused)
> >>>
> >>> if (irqnr != GICC_INT_SPURIOUS) {
> >>> gic_write_eoir(irqstat);
> >>> - smp_rmb(); /* pairs with wmb in ipi_test functions */
> >>> + smp_rmb(); /* pairs with wmb in stats_reset */
> >>> ++acked[smp_processor_id()];
> >>> + check_ipi_sender(irqstat);
> >>> + check_irqnr(irqnr);
> >>> smp_wmb(); /* pairs with rmb in check_acked */
> >>> } else {
> >>> ++spurious[smp_processor_id()];
> >>> @@ -109,19 +155,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
> >>>
> >>> static void gicv2_ipi_send_self(void)
> >>> {
> >>> - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> >>> + writel(2 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> >>> }
> >>>
> >>> -static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
> >>> +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq)
> >>> {
> >>> u8 tlist = (u8)cpumask_bits(mask)[0];
> >>>
> >>> - writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> >>> + writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
> >>> }
> >>>
> >>> static void gicv2_ipi_send_broadcast(void)
> >>> {
> >>> - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> >>> + writel(1 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> >>> }
> >>>
> >>> static void gicv3_ipi_send_self(void)
> >>> @@ -130,12 +176,12 @@ static void gicv3_ipi_send_self(void)
> >>>
> >>> cpumask_clear(&mask);
> >>> cpumask_set_cpu(smp_processor_id(), &mask);
> >>> - gicv3_ipi_send_tlist(&mask, 0);
> >>> + gicv3_ipi_send_tlist(&mask, cmdl_irq);
> >>> }
> >>>
> >>> static void gicv3_ipi_send_broadcast(void)
> >>> {
> >>> - gicv3_write_sgi1r(1ULL << 40);
> >>> + gicv3_write_sgi1r(1ULL << 40 | cmdl_irq << 24);
> >>> isb();
> >>> }
> >>>
> >>> @@ -144,10 +190,9 @@ static void ipi_test_self(void)
> >>> cpumask_t mask;
> >>>
> >>> report_prefix_push("self");
> >>> - memset(acked, 0, sizeof(acked));
> >>> - smp_wmb();
> >>> + stats_reset();
> >>> cpumask_clear(&mask);
> >>> - cpumask_set_cpu(0, &mask);
> >>> + cpumask_set_cpu(smp_processor_id(), &mask);
> >>> gic->ipi.send_self();
> >>> check_acked(&mask);
> >>> report_prefix_pop();
> >>> @@ -159,20 +204,18 @@ static void ipi_test_smp(void)
> >>> int i;
> >>>
> >>> report_prefix_push("target-list");
> >>> - memset(acked, 0, sizeof(acked));
> >>> - smp_wmb();
> >>> + stats_reset();
> >>> cpumask_copy(&mask, &cpu_present_mask);
> >>> - for (i = 0; i < nr_cpus; i += 2)
> >>> + for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
> >>> cpumask_clear_cpu(i, &mask);
> >>> - gic->ipi.send_tlist(&mask, 0);
> >>> + gic->ipi.send_tlist(&mask, cmdl_irq);
> >>> check_acked(&mask);
> >>> report_prefix_pop();
> >>>
> >>> report_prefix_push("broadcast");
> >>> - memset(acked, 0, sizeof(acked));
> >>> - smp_wmb();
> >>> + stats_reset();
> >>> cpumask_copy(&mask, &cpu_present_mask);
> >>> - cpumask_clear_cpu(0, &mask);
> >>> + cpumask_clear_cpu(smp_processor_id(), &mask);
> >>> gic->ipi.send_broadcast();
> >>> check_acked(&mask);
> >>> report_prefix_pop();
> >>> @@ -189,6 +232,16 @@ static void ipi_enable(void)
> >>> local_irq_enable();
> >>> }
> >>>
> >>> +static void ipi_send(void)
> >>> +{
> >>> + ipi_enable();
> >>> + wait_on_ready();
> >>> + ipi_test_self();
> >>> + ipi_test_smp();
> >>> + check_spurious();
> >>> + exit(report_summary());
> >>> +}
> >>> +
> >>> static void ipi_recv(void)
> >>> {
> >>> ipi_enable();
> >>> @@ -197,6 +250,14 @@ static void ipi_recv(void)
> >>> wfi();
> >>> }
> >>>
> >>> +static void ipi_test(void)
> >>> +{
> >>> + if (smp_processor_id() == cmdl_sender)
> >>> + ipi_send();
> >>> + else
> >>> + ipi_recv();
> >>> +}
> >>> +
> >>> static struct gic gicv2 = {
> >>> .ipi = {
> >>> .send_self = gicv2_ipi_send_self,
> >>> @@ -242,21 +303,34 @@ int main(int argc, char **argv)
> >>> report_prefix_pop();
> >>>
> >>> } else if (strcmp(argv[1], "ipi") == 0) {
> >>> + int off, i = 1;
> >>> + long val;
> >>>
> >>> report_prefix_push(argv[1]);
> >>> nr_cpu_check(2);
> >>>
> >>> + while (--argc != 1) {
> >>> + off = parse_keyval(argv[++i], &val);
> >>> + if (off == -1)
> >>> + continue;
> >>> + argv[i][off] = '\0';
> >>> + if (strcmp(argv[i], "sender") == 0) {
> >>> + if (val >= nr_cpus)
> >>> + report_abort("invalid sender %d, nr_cpus=%d", val, nr_cpus);
> >>> + cmdl_sender = val;
> >>> + } else if (strcmp(argv[i], "irq") == 0) {
> >>> + if (val > 15)
> >>> + report_abort("irq (SGI) must be < 16");
> >>> + cmdl_irq = val;
> >>> + }
> >> wrong arg could potentially be tested.
> >
> > What do you mean by 'wrong arg'? Can you give me an example?
> I meant bad option, ie if the tester makes a typo such as sende=3, this
> gets ignored, right?.
Right, it just gets ignored. I can check for that and add a usage output
here.
Thanks,
drew
>
> Thanks
>
> Eric
> >
> >>
> >> Besides
> >> Reviewed-by: Eric Auger <eric.auger@redhat.com>
> >
> > Thanks!
> >
> > drew
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero
@ 2016-11-23 13:46 ` Andrew Jones
0 siblings, 0 replies; 52+ messages in thread
From: Andrew Jones @ 2016-11-23 13:46 UTC (permalink / raw)
To: Auger Eric
Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
peter.maydell, alex.bennee, marc.zyngier, christoffer.dall
On Wed, Nov 23, 2016 at 02:33:31PM +0100, Auger Eric wrote:
>
>
> On 23/11/2016 14:01, Andrew Jones wrote:
> > On Wed, Nov 23, 2016 at 12:28:34PM +0100, Auger Eric wrote:
> >> Hi,
> >>
> >> On 14/11/2016 22:08, Andrew Jones wrote:
> >>> Allow user to select who sends ipis and with which irq,
> >>> rather than just always sending irq=0 from cpu0.
> >> From a user point of view is there a way to know the list of available
> >> tests and their arg?
> >
> > Not at the moment. I could change the arg-less run and/or add
> > a '-h' though that would provide a usage. So far unit tests
> > have never had -h's, but it's not a bad idea.
>
> OK thanks for the info
> >
> >>>
> >>> Signed-off-by: Andrew Jones <drjones@redhat.com>
> >>>
> >>> ---
> >>> v6:
> >>> - make sender/irq names more future-proof [drew]
> >>> - sanity check inputs [drew]
> >>> - introduce check_sender/irq and bad_sender/irq to more
> >>> cleanly do checks [drew]
> >>> - default sender and irq to 1, instead of still zero [drew]
> >>> v4: improve structure and make sure spurious checking is
> >>> done even when the sender isn't cpu0
> >>> v2: actually check that the irq received was the irq sent,
> >>> and (for gicv2) that the sender is the expected one.
> >>> ---
> >>> arm/gic.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++++-------------
> >>> 1 file changed, 99 insertions(+), 25 deletions(-)
> >>>
> >>> diff --git a/arm/gic.c b/arm/gic.c
> >>> index d954a3775c26..638b8b140c96 100644
> >>> --- a/arm/gic.c
> >>> +++ b/arm/gic.c
> >>> @@ -11,6 +11,7 @@
> >>> * This work is licensed under the terms of the GNU LGPL, version 2.
> >>> */
> >>> #include <libcflat.h>
> >>> +#include <util.h>
> >>> #include <asm/setup.h>
> >>> #include <asm/processor.h>
> >>> #include <asm/gic.h>
> >>> @@ -28,6 +29,8 @@ struct gic {
> >>>
> >>> static struct gic *gic;
> >>> static int acked[NR_CPUS], spurious[NR_CPUS];
> >>> +static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
> >>> +static int cmdl_sender = 1, cmdl_irq = 1;
> >>> static cpumask_t ready;
> >>>
> >>> static void nr_cpu_check(int nr)
> >>> @@ -43,10 +46,23 @@ static void wait_on_ready(void)
> >>> cpu_relax();
> >>> }
> >>>
> >>> +static void stats_reset(void)
> >>> +{
> >>> + int i;
> >>> +
> >>> + for (i = 0; i < nr_cpus; ++i) {
> >>> + acked[i] = 0;
> >>> + bad_sender[i] = -1;
> >>> + bad_irq[i] = -1;
> >>> + }
> >>> + smp_wmb();
> >>> +}
> >>> +
> >>> static void check_acked(cpumask_t *mask)
> >>> {
> >>> int missing = 0, extra = 0, unexpected = 0;
> >>> int nr_pass, cpu, i;
> >>> + bool bad = false;
> >>>
> >>> /* Wait up to 5s for all interrupts to be delivered */
> >>> for (i = 0; i < 50; ++i) {
> >>> @@ -56,9 +72,21 @@ static void check_acked(cpumask_t *mask)
> >>> smp_rmb();
> >>> nr_pass += cpumask_test_cpu(cpu, mask) ?
> >>> acked[cpu] == 1 : acked[cpu] == 0;
> >>> +
> >>> + if (bad_sender[cpu] != -1) {
> >>> + printf("cpu%d received IPI from wrong sender %d\n",
> >>> + cpu, bad_sender[cpu]);
> >>> + bad = true;
> >>> + }
> >>> +
> >>> + if (bad_irq[cpu] != -1) {
> >>> + printf("cpu%d received wrong irq %d\n",
> >>> + cpu, bad_irq[cpu]);
> >>> + bad = true;
> >>> + }
> >>> }
> >>> if (nr_pass == nr_cpus) {
> >>> - report("Completed in %d ms", true, ++i * 100);
> >>> + report("Completed in %d ms", !bad, ++i * 100);
> >>> return;
> >>> }
> >>> }
> >>> @@ -91,6 +119,22 @@ static void check_spurious(void)
> >>> }
> >>> }
> >>>
> >>> +static void check_ipi_sender(u32 irqstat)
> >>> +{
> >>> + if (gic_version() == 2) {
> >>> + int src = (irqstat >> 10) & 7;
> >>> +
> >>> + if (src != cmdl_sender)
> >>> + bad_sender[smp_processor_id()] = src;
> >>> + }
> >>> +}
> >>> +
> >>> +static void check_irqnr(u32 irqnr)
> >>> +{
> >>> + if (irqnr != (u32)cmdl_irq)
> >>> + bad_irq[smp_processor_id()] = irqnr;
> >>> +}
> >>> +
> >>> static void ipi_handler(struct pt_regs *regs __unused)
> >>> {
> >>> u32 irqstat = gic_read_iar();
> >>> @@ -98,8 +142,10 @@ static void ipi_handler(struct pt_regs *regs __unused)
> >>>
> >>> if (irqnr != GICC_INT_SPURIOUS) {
> >>> gic_write_eoir(irqstat);
> >>> - smp_rmb(); /* pairs with wmb in ipi_test functions */
> >>> + smp_rmb(); /* pairs with wmb in stats_reset */
> >>> ++acked[smp_processor_id()];
> >>> + check_ipi_sender(irqstat);
> >>> + check_irqnr(irqnr);
> >>> smp_wmb(); /* pairs with rmb in check_acked */
> >>> } else {
> >>> ++spurious[smp_processor_id()];
> >>> @@ -109,19 +155,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
> >>>
> >>> static void gicv2_ipi_send_self(void)
> >>> {
> >>> - writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> >>> + writel(2 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> >>> }
> >>>
> >>> -static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq __unused)
> >>> +static void gicv2_ipi_send_tlist(cpumask_t *mask, int irq)
> >>> {
> >>> u8 tlist = (u8)cpumask_bits(mask)[0];
> >>>
> >>> - writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> >>> + writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
> >>> }
> >>>
> >>> static void gicv2_ipi_send_broadcast(void)
> >>> {
> >>> - writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> >>> + writel(1 << 24 | cmdl_irq, gicv2_dist_base() + GICD_SGIR);
> >>> }
> >>>
> >>> static void gicv3_ipi_send_self(void)
> >>> @@ -130,12 +176,12 @@ static void gicv3_ipi_send_self(void)
> >>>
> >>> cpumask_clear(&mask);
> >>> cpumask_set_cpu(smp_processor_id(), &mask);
> >>> - gicv3_ipi_send_tlist(&mask, 0);
> >>> + gicv3_ipi_send_tlist(&mask, cmdl_irq);
> >>> }
> >>>
> >>> static void gicv3_ipi_send_broadcast(void)
> >>> {
> >>> - gicv3_write_sgi1r(1ULL << 40);
> >>> + gicv3_write_sgi1r(1ULL << 40 | cmdl_irq << 24);
> >>> isb();
> >>> }
> >>>
> >>> @@ -144,10 +190,9 @@ static void ipi_test_self(void)
> >>> cpumask_t mask;
> >>>
> >>> report_prefix_push("self");
> >>> - memset(acked, 0, sizeof(acked));
> >>> - smp_wmb();
> >>> + stats_reset();
> >>> cpumask_clear(&mask);
> >>> - cpumask_set_cpu(0, &mask);
> >>> + cpumask_set_cpu(smp_processor_id(), &mask);
> >>> gic->ipi.send_self();
> >>> check_acked(&mask);
> >>> report_prefix_pop();
> >>> @@ -159,20 +204,18 @@ static void ipi_test_smp(void)
> >>> int i;
> >>>
> >>> report_prefix_push("target-list");
> >>> - memset(acked, 0, sizeof(acked));
> >>> - smp_wmb();
> >>> + stats_reset();
> >>> cpumask_copy(&mask, &cpu_present_mask);
> >>> - for (i = 0; i < nr_cpus; i += 2)
> >>> + for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
> >>> cpumask_clear_cpu(i, &mask);
> >>> - gic->ipi.send_tlist(&mask, 0);
> >>> + gic->ipi.send_tlist(&mask, cmdl_irq);
> >>> check_acked(&mask);
> >>> report_prefix_pop();
> >>>
> >>> report_prefix_push("broadcast");
> >>> - memset(acked, 0, sizeof(acked));
> >>> - smp_wmb();
> >>> + stats_reset();
> >>> cpumask_copy(&mask, &cpu_present_mask);
> >>> - cpumask_clear_cpu(0, &mask);
> >>> + cpumask_clear_cpu(smp_processor_id(), &mask);
> >>> gic->ipi.send_broadcast();
> >>> check_acked(&mask);
> >>> report_prefix_pop();
> >>> @@ -189,6 +232,16 @@ static void ipi_enable(void)
> >>> local_irq_enable();
> >>> }
> >>>
> >>> +static void ipi_send(void)
> >>> +{
> >>> + ipi_enable();
> >>> + wait_on_ready();
> >>> + ipi_test_self();
> >>> + ipi_test_smp();
> >>> + check_spurious();
> >>> + exit(report_summary());
> >>> +}
> >>> +
> >>> static void ipi_recv(void)
> >>> {
> >>> ipi_enable();
> >>> @@ -197,6 +250,14 @@ static void ipi_recv(void)
> >>> wfi();
> >>> }
> >>>
> >>> +static void ipi_test(void)
> >>> +{
> >>> + if (smp_processor_id() == cmdl_sender)
> >>> + ipi_send();
> >>> + else
> >>> + ipi_recv();
> >>> +}
> >>> +
> >>> static struct gic gicv2 = {
> >>> .ipi = {
> >>> .send_self = gicv2_ipi_send_self,
> >>> @@ -242,21 +303,34 @@ int main(int argc, char **argv)
> >>> report_prefix_pop();
> >>>
> >>> } else if (strcmp(argv[1], "ipi") == 0) {
> >>> + int off, i = 1;
> >>> + long val;
> >>>
> >>> report_prefix_push(argv[1]);
> >>> nr_cpu_check(2);
> >>>
> >>> + while (--argc != 1) {
> >>> + off = parse_keyval(argv[++i], &val);
> >>> + if (off == -1)
> >>> + continue;
> >>> + argv[i][off] = '\0';
> >>> + if (strcmp(argv[i], "sender") == 0) {
> >>> + if (val >= nr_cpus)
> >>> + report_abort("invalid sender %d, nr_cpus=%d", val, nr_cpus);
> >>> + cmdl_sender = val;
> >>> + } else if (strcmp(argv[i], "irq") == 0) {
> >>> + if (val > 15)
> >>> + report_abort("irq (SGI) must be < 16");
> >>> + cmdl_irq = val;
> >>> + }
> >> wrong arg could potentially be tested.
> >
> > What do you mean by 'wrong arg'? Can you give me an example?
> I meant bad option, ie if the tester makes a typo such as sende=3, this
> gets ignored, right?.
Right, it just gets ignored. I can check for that and add a usage output
here.
Thanks,
drew
>
> Thanks
>
> Eric
> >
> >>
> >> Besides
> >> Reviewed-by: Eric Auger <eric.auger@redhat.com>
> >
> > Thanks!
> >
> > drew
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 52+ messages in thread
end of thread, other threads:[~2016-11-23 13:46 UTC | newest]
Thread overview: 52+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-14 21:08 [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 01/11] lib: xstr: allow multiple args Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 02/11] arm64: fix get_"sysreg32" and make MPIDR 64bit Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 03/11] arm/arm64: smp: support more than 8 cpus Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-23 10:38 ` Auger Eric
2016-11-23 10:38 ` [Qemu-devel] " Auger Eric
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 04/11] arm/arm64: add some delay routines Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 05/11] arm/arm64: irq enable/disable Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 06/11] arm/arm64: add initial gicv2 support Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-23 10:38 ` Auger Eric
2016-11-23 10:38 ` [Qemu-devel] " Auger Eric
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 07/11] arm/arm64: gicv2: add an IPI test Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 08/11] libcflat: add IS_ALIGNED() macro, and page sizes Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-23 10:38 ` Auger Eric
2016-11-23 10:38 ` [Qemu-devel] " Auger Eric
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 09/11] arm/arm64: add initial gicv3 support Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-23 10:38 ` Auger Eric
2016-11-23 10:38 ` [Qemu-devel] " Auger Eric
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 10/11] arm/arm64: gicv3: add an IPI test Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-23 11:05 ` Auger Eric
2016-11-23 11:05 ` [Qemu-devel] " Auger Eric
2016-11-23 12:57 ` Andrew Jones
2016-11-23 12:57 ` Andrew Jones
2016-11-14 21:08 ` [kvm-unit-tests PATCH v6 11/11] arm/arm64: gic: don't just use zero Andrew Jones
2016-11-14 21:08 ` [Qemu-devel] " Andrew Jones
2016-11-23 11:28 ` Auger Eric
2016-11-23 11:28 ` [Qemu-devel] " Auger Eric
2016-11-23 13:01 ` Andrew Jones
2016-11-23 13:01 ` [Qemu-devel] " Andrew Jones
2016-11-23 13:33 ` Auger Eric
2016-11-23 13:33 ` [Qemu-devel] " Auger Eric
2016-11-23 13:46 ` Andrew Jones
2016-11-23 13:46 ` [Qemu-devel] " Andrew Jones
2016-11-22 18:45 ` [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework Andrew Jones
2016-11-23 9:55 ` Andre Przywara
2016-11-23 9:55 ` Andre Przywara
2016-11-23 10:09 ` Alex Bennée
2016-11-23 10:09 ` Alex Bennée
2016-11-23 11:33 ` Auger Eric
2016-11-23 11:33 ` Auger Eric
2016-11-23 13:08 ` Andrew Jones
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.