All of lore.kernel.org
 help / color / mirror / Atom feed
* [kvm-unit-tests PATCH v5 00/11] arm/arm64: add gic framework
@ 2016-11-10 17:21 ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini

v5:
 - fix arm32/gicv3 compile [drew]
 - use modern register names [Andre]
 - one Andre r-b

v4:
 - Eric's r-b's
 - Andre's suggestion to only take defines we need
 - several other changes listed in individual patches

v3:
 - Rebased on latest master
 - Added Alex's r-b's

v2:
 Rebased on latest master + my "populate argv[0]" series (will
 send a REPOST for that shortly. Additionally a few patches got
 fixes/features;
 07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
       all interrupts as non-secure Group-1" in order to continue
       working over TCG, as the gicv3 code for TCG removed a hack
       it had there to make Linux happy.
 08/10 added more output for when things fail (if they fail)
 09/10 switched gicv3 broadcast implementation to using IRM. This
       found a bug in a recent (but not tip) kernel, which I was
       about to fix, but then I saw MarcZ beat me to it.
 10/10 actually check that the input irq is the received irq


Import defines, and steal enough helper functions, from Linux to
enable programming of the gic (v2 and v3). Then use the framework
to add an initial test (an ipi test; self, target-list, broadcast).

It's my hope that this framework will be a suitable base on which
more tests may be easily added, particularly because we have
vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
vgic-new and tcg gicv3 are merged now)

To run it, along with other tests, just do

 ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
 make
 export QEMU=$PATH_TO_QEMU
 ./run_tests.sh

To run it separately do, e.g.

$QEMU -machine virt,accel=tcg -cpu cortex-a57 \
 -device virtio-serial-device \
 -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
 -display none -serial stdio \
 -kernel arm/gic.flat \
 -smp 123 -machine gic-version=3 -append ipi
      ^^ note, we can go nuts with nr-cpus on TCG :-)

Or, a KVM example using a different "sender" cpu and irq (other than zero)

$QEMU -machine virt,accel=kvm -cpu host \
 -device virtio-serial-device \
 -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
 -display none -serial stdio \
 -kernel arm/gic.flat \
 -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'


Patches:
01-05: fixes and functionality needed by the later gic patches
06-07: enable gicv2 and gicv2 IPI test
08-10: enable gicv3 and gicv3 IPI test
   11: extend the IPI tests to take variable sender and irq

Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v5


Andrew Jones (10):
  lib: xstr: allow multiple args
  arm64: fix get_"sysreg32" and make MPIDR 64bit
  arm/arm64: smp: support more than 8 cpus
  arm/arm64: add some delay routines
  arm/arm64: irq enable/disable
  arm/arm64: add initial gicv2 support
  arm/arm64: gicv2: add an IPI test
  arm/arm64: add initial gicv3 support
  arm/arm64: gicv3: add an IPI test
  arm/arm64: gic: don't just use zero

Peter Xu (1):
  libcflat: add IS_ALIGNED() macro, and page sizes

 arm/Makefile.common        |   7 +-
 arm/gic.c                  | 417 +++++++++++++++++++++++++++++++++++++++++++++
 arm/run                    |  19 ++-
 arm/selftest.c             |   5 +-
 arm/unittests.cfg          |  13 ++
 lib/arm/asm/arch_gicv3.h   |  65 +++++++
 lib/arm/asm/gic-v2.h       |  36 ++++
 lib/arm/asm/gic-v3.h       |  94 ++++++++++
 lib/arm/asm/gic.h          |  43 +++++
 lib/arm/asm/processor.h    |  42 ++++-
 lib/arm/asm/setup.h        |   4 +-
 lib/arm/gic.c              | 141 +++++++++++++++
 lib/arm/processor.c        |  15 ++
 lib/arm/setup.c            |  10 ++
 lib/arm64/asm/arch_gicv3.h |  66 +++++++
 lib/arm64/asm/gic-v2.h     |   1 +
 lib/arm64/asm/gic-v3.h     |   1 +
 lib/arm64/asm/gic.h        |   1 +
 lib/arm64/asm/processor.h  |  53 +++++-
 lib/arm64/asm/sysreg.h     |  44 +++++
 lib/arm64/processor.c      |  15 ++
 lib/libcflat.h             |  10 +-
 22 files changed, 1075 insertions(+), 27 deletions(-)
 create mode 100644 arm/gic.c
 create mode 100644 lib/arm/asm/arch_gicv3.h
 create mode 100644 lib/arm/asm/gic-v2.h
 create mode 100644 lib/arm/asm/gic-v3.h
 create mode 100644 lib/arm/asm/gic.h
 create mode 100644 lib/arm/gic.c
 create mode 100644 lib/arm64/asm/arch_gicv3.h
 create mode 100644 lib/arm64/asm/gic-v2.h
 create mode 100644 lib/arm64/asm/gic-v3.h
 create mode 100644 lib/arm64/asm/gic.h
 create mode 100644 lib/arm64/asm/sysreg.h

-- 
2.7.4

^ permalink raw reply	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 00/11] arm/arm64: add gic framework
@ 2016-11-10 17:21 ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

v5:
 - fix arm32/gicv3 compile [drew]
 - use modern register names [Andre]
 - one Andre r-b

v4:
 - Eric's r-b's
 - Andre's suggestion to only take defines we need
 - several other changes listed in individual patches

v3:
 - Rebased on latest master
 - Added Alex's r-b's

v2:
 Rebased on latest master + my "populate argv[0]" series (will
 send a REPOST for that shortly. Additionally a few patches got
 fixes/features;
 07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
       all interrupts as non-secure Group-1" in order to continue
       working over TCG, as the gicv3 code for TCG removed a hack
       it had there to make Linux happy.
 08/10 added more output for when things fail (if they fail)
 09/10 switched gicv3 broadcast implementation to using IRM. This
       found a bug in a recent (but not tip) kernel, which I was
       about to fix, but then I saw MarcZ beat me to it.
 10/10 actually check that the input irq is the received irq


Import defines, and steal enough helper functions, from Linux to
enable programming of the gic (v2 and v3). Then use the framework
to add an initial test (an ipi test; self, target-list, broadcast).

It's my hope that this framework will be a suitable base on which
more tests may be easily added, particularly because we have
vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
vgic-new and tcg gicv3 are merged now)

To run it, along with other tests, just do

 ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
 make
 export QEMU=$PATH_TO_QEMU
 ./run_tests.sh

To run it separately do, e.g.

$QEMU -machine virt,accel=tcg -cpu cortex-a57 \
 -device virtio-serial-device \
 -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
 -display none -serial stdio \
 -kernel arm/gic.flat \
 -smp 123 -machine gic-version=3 -append ipi
      ^^ note, we can go nuts with nr-cpus on TCG :-)

Or, a KVM example using a different "sender" cpu and irq (other than zero)

$QEMU -machine virt,accel=kvm -cpu host \
 -device virtio-serial-device \
 -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
 -display none -serial stdio \
 -kernel arm/gic.flat \
 -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'


Patches:
01-05: fixes and functionality needed by the later gic patches
06-07: enable gicv2 and gicv2 IPI test
08-10: enable gicv3 and gicv3 IPI test
   11: extend the IPI tests to take variable sender and irq

Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v5


Andrew Jones (10):
  lib: xstr: allow multiple args
  arm64: fix get_"sysreg32" and make MPIDR 64bit
  arm/arm64: smp: support more than 8 cpus
  arm/arm64: add some delay routines
  arm/arm64: irq enable/disable
  arm/arm64: add initial gicv2 support
  arm/arm64: gicv2: add an IPI test
  arm/arm64: add initial gicv3 support
  arm/arm64: gicv3: add an IPI test
  arm/arm64: gic: don't just use zero

Peter Xu (1):
  libcflat: add IS_ALIGNED() macro, and page sizes

 arm/Makefile.common        |   7 +-
 arm/gic.c                  | 417 +++++++++++++++++++++++++++++++++++++++++++++
 arm/run                    |  19 ++-
 arm/selftest.c             |   5 +-
 arm/unittests.cfg          |  13 ++
 lib/arm/asm/arch_gicv3.h   |  65 +++++++
 lib/arm/asm/gic-v2.h       |  36 ++++
 lib/arm/asm/gic-v3.h       |  94 ++++++++++
 lib/arm/asm/gic.h          |  43 +++++
 lib/arm/asm/processor.h    |  42 ++++-
 lib/arm/asm/setup.h        |   4 +-
 lib/arm/gic.c              | 141 +++++++++++++++
 lib/arm/processor.c        |  15 ++
 lib/arm/setup.c            |  10 ++
 lib/arm64/asm/arch_gicv3.h |  66 +++++++
 lib/arm64/asm/gic-v2.h     |   1 +
 lib/arm64/asm/gic-v3.h     |   1 +
 lib/arm64/asm/gic.h        |   1 +
 lib/arm64/asm/processor.h  |  53 +++++-
 lib/arm64/asm/sysreg.h     |  44 +++++
 lib/arm64/processor.c      |  15 ++
 lib/libcflat.h             |  10 +-
 22 files changed, 1075 insertions(+), 27 deletions(-)
 create mode 100644 arm/gic.c
 create mode 100644 lib/arm/asm/arch_gicv3.h
 create mode 100644 lib/arm/asm/gic-v2.h
 create mode 100644 lib/arm/asm/gic-v3.h
 create mode 100644 lib/arm/asm/gic.h
 create mode 100644 lib/arm/gic.c
 create mode 100644 lib/arm64/asm/arch_gicv3.h
 create mode 100644 lib/arm64/asm/gic-v2.h
 create mode 100644 lib/arm64/asm/gic-v3.h
 create mode 100644 lib/arm64/asm/gic.h
 create mode 100644 lib/arm64/asm/sysreg.h

-- 
2.7.4

^ permalink raw reply	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 01/11] lib: xstr: allow multiple args
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini

Make implementation equivalent to Linux's include/linux/stringify.h

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 lib/libcflat.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/libcflat.h b/lib/libcflat.h
index 72b1bf9668ef..82005f5d014f 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -27,8 +27,8 @@
 
 #define __unused __attribute__((__unused__))
 
-#define xstr(s) xxstr(s)
-#define xxstr(s) #s
+#define xstr(s...) xxstr(s)
+#define xxstr(s...) #s
 
 #define __ALIGN_MASK(x, mask)	(((x) + (mask)) & ~(mask))
 #define __ALIGN(x, a)		__ALIGN_MASK(x, (typeof(x))(a) - 1)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 01/11] lib: xstr: allow multiple args
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Make implementation equivalent to Linux's include/linux/stringify.h

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 lib/libcflat.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/libcflat.h b/lib/libcflat.h
index 72b1bf9668ef..82005f5d014f 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -27,8 +27,8 @@
 
 #define __unused __attribute__((__unused__))
 
-#define xstr(s) xxstr(s)
-#define xxstr(s) #s
+#define xstr(s...) xxstr(s)
+#define xxstr(s...) #s
 
 #define __ALIGN_MASK(x, mask)	(((x) + (mask)) & ~(mask))
 #define __ALIGN(x, a)		__ALIGN_MASK(x, (typeof(x))(a) - 1)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 02/11] arm64: fix get_"sysreg32" and make MPIDR 64bit
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

mrs is always 64bit, so we should always use a 64bit register.
Sometimes we'll only want to return the lower 32, but not for
MPIDR, as that does define fields in the upper 32.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v5: switch arm32's get_mpidr to 'unsigned long' too, to be
    consistent with arm64 [Andre]
---
 lib/arm/asm/processor.h   |  4 ++--
 lib/arm64/asm/processor.h | 15 +++++++++------
 2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index f25e7eee3666..02f912f99974 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -33,9 +33,9 @@ static inline unsigned long current_cpsr(void)
 
 #define current_mode() (current_cpsr() & MODE_MASK)
 
-static inline unsigned int get_mpidr(void)
+static inline unsigned long get_mpidr(void)
 {
-	unsigned int mpidr;
+	unsigned long mpidr;
 	asm volatile("mrc p15, 0, %0, c0, c0, 5" : "=r" (mpidr));
 	return mpidr;
 }
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 84d5c7ce752b..9a208ff729b7 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -66,14 +66,17 @@ static inline unsigned long current_level(void)
 	return el & 0xc;
 }
 
-#define DEFINE_GET_SYSREG32(reg)				\
-static inline unsigned int get_##reg(void)			\
+#define DEFINE_GET_SYSREG(reg, type)				\
+static inline type get_##reg(void)				\
 {								\
-	unsigned int reg;					\
-	asm volatile("mrs %0, " #reg "_el1" : "=r" (reg));	\
-	return reg;						\
+	unsigned long r;					\
+	asm volatile("mrs %0, " #reg "_el1" : "=r" (r));	\
+	return (type)r;						\
 }
-DEFINE_GET_SYSREG32(mpidr)
+#define DEFINE_GET_SYSREG32(reg) DEFINE_GET_SYSREG(reg, unsigned int)
+#define DEFINE_GET_SYSREG64(reg) DEFINE_GET_SYSREG(reg, unsigned long)
+
+DEFINE_GET_SYSREG64(mpidr)
 
 /* Only support Aff0 for now, gicv2 only */
 #define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 02/11] arm64: fix get_"sysreg32" and make MPIDR 64bit
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

mrs is always 64bit, so we should always use a 64bit register.
Sometimes we'll only want to return the lower 32, but not for
MPIDR, as that does define fields in the upper 32.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v5: switch arm32's get_mpidr to 'unsigned long' too, to be
    consistent with arm64 [Andre]
---
 lib/arm/asm/processor.h   |  4 ++--
 lib/arm64/asm/processor.h | 15 +++++++++------
 2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index f25e7eee3666..02f912f99974 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -33,9 +33,9 @@ static inline unsigned long current_cpsr(void)
 
 #define current_mode() (current_cpsr() & MODE_MASK)
 
-static inline unsigned int get_mpidr(void)
+static inline unsigned long get_mpidr(void)
 {
-	unsigned int mpidr;
+	unsigned long mpidr;
 	asm volatile("mrc p15, 0, %0, c0, c0, 5" : "=r" (mpidr));
 	return mpidr;
 }
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 84d5c7ce752b..9a208ff729b7 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -66,14 +66,17 @@ static inline unsigned long current_level(void)
 	return el & 0xc;
 }
 
-#define DEFINE_GET_SYSREG32(reg)				\
-static inline unsigned int get_##reg(void)			\
+#define DEFINE_GET_SYSREG(reg, type)				\
+static inline type get_##reg(void)				\
 {								\
-	unsigned int reg;					\
-	asm volatile("mrs %0, " #reg "_el1" : "=r" (reg));	\
-	return reg;						\
+	unsigned long r;					\
+	asm volatile("mrs %0, " #reg "_el1" : "=r" (r));	\
+	return (type)r;						\
 }
-DEFINE_GET_SYSREG32(mpidr)
+#define DEFINE_GET_SYSREG32(reg) DEFINE_GET_SYSREG(reg, unsigned int)
+#define DEFINE_GET_SYSREG64(reg) DEFINE_GET_SYSREG(reg, unsigned long)
+
+DEFINE_GET_SYSREG64(mpidr)
 
 /* Only support Aff0 for now, gicv2 only */
 #define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 03/11] arm/arm64: smp: support more than 8 cpus
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini

By adding support for launching with gicv3 we can break the 8 vcpu
limit. This patch adds support to smp code and also selects the
vgic model corresponding to the host. The vgic model may also be
manually selected by adding e.g. -machine gic-version=3 to
extra_params.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v5: left cpus a u32 for now. Changing to u64 requires a change to
    devicetree. Will do it later. [Andre]
v4: improved commit message
---
 arm/run                   | 19 ++++++++++++-------
 arm/selftest.c            |  5 ++++-
 lib/arm/asm/processor.h   |  9 +++++++--
 lib/arm/asm/setup.h       |  4 ++--
 lib/arm/setup.c           | 10 ++++++++++
 lib/arm64/asm/processor.h |  9 +++++++--
 6 files changed, 42 insertions(+), 14 deletions(-)

diff --git a/arm/run b/arm/run
index a2f35ef6a7e6..2d0698619606 100755
--- a/arm/run
+++ b/arm/run
@@ -31,13 +31,6 @@ if [ -z "$ACCEL" ]; then
 	fi
 fi
 
-if [ "$HOST" = "aarch64" ] && [ "$ACCEL" = "kvm" ]; then
-	processor="host"
-	if [ "$ARCH" = "arm" ]; then
-		processor+=",aarch64=off"
-	fi
-fi
-
 qemu="${QEMU:-qemu-system-$ARCH_NAME}"
 qpath=$(which $qemu 2>/dev/null)
 
@@ -53,6 +46,18 @@ fi
 
 M='-machine virt'
 
+if [ "$ACCEL" = "kvm" ]; then
+	if $qemu $M,\? 2>&1 | grep gic-version > /dev/null; then
+		M+=',gic-version=host'
+	fi
+	if [ "$HOST" = "aarch64" ]; then
+		processor="host"
+		if [ "$ARCH" = "arm" ]; then
+			processor+=",aarch64=off"
+		fi
+	fi
+fi
+
 if ! $qemu $M -device '?' 2>&1 | grep virtconsole > /dev/null; then
 	echo "$qpath doesn't support virtio-console for chr-testdev. Exiting."
 	exit 2
diff --git a/arm/selftest.c b/arm/selftest.c
index 196164f5313d..2f117f795d2d 100644
--- a/arm/selftest.c
+++ b/arm/selftest.c
@@ -312,9 +312,10 @@ static bool psci_check(void)
 static cpumask_t smp_reported;
 static void cpu_report(void)
 {
+	unsigned long mpidr = get_mpidr();
 	int cpu = smp_processor_id();
 
-	report("CPU%d online", true, cpu);
+	report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == cpu, cpu, mpidr);
 	cpumask_set_cpu(cpu, &smp_reported);
 	halt();
 }
@@ -343,6 +344,7 @@ int main(int argc, char **argv)
 
 	} else if (strcmp(argv[1], "smp") == 0) {
 
+		unsigned long mpidr = get_mpidr();
 		int cpu;
 
 		report("PSCI version", psci_check());
@@ -353,6 +355,7 @@ int main(int argc, char **argv)
 			smp_boot_secondary(cpu, cpu_report);
 		}
 
+		report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == 0, 0, mpidr);
 		cpumask_set_cpu(0, &smp_reported);
 		while (!cpumask_full(&smp_reported))
 			cpu_relax();
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index 02f912f99974..ecf5bbe1824a 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -40,8 +40,13 @@ static inline unsigned long get_mpidr(void)
 	return mpidr;
 }
 
-/* Only support Aff0 for now, up to 4 cpus */
-#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
+#define MPIDR_HWID_BITMASK 0xffffff
+extern int mpidr_to_cpu(unsigned long mpidr);
+
+#define MPIDR_LEVEL_SHIFT(level) \
+	(((1 << level) >> 1) << 3)
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+	((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
 
 extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
 extern bool is_user(void);
diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
index cb8fdbd38dd5..1de99dd184d1 100644
--- a/lib/arm/asm/setup.h
+++ b/lib/arm/asm/setup.h
@@ -10,8 +10,8 @@
 #include <asm/page.h>
 #include <asm/pgtable-hwdef.h>
 
-#define NR_CPUS			8
-extern u32 cpus[NR_CPUS];
+#define NR_CPUS			255
+extern u32 cpus[NR_CPUS];	/* per-cpu IDs (MPIDRs) */
 extern int nr_cpus;
 
 #define NR_MEM_REGIONS		8
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 7e7b39f11dde..241bf9410447 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -30,6 +30,16 @@ int nr_cpus;
 struct mem_region mem_regions[NR_MEM_REGIONS];
 phys_addr_t __phys_offset, __phys_end;
 
+int mpidr_to_cpu(unsigned long mpidr)
+{
+	int i;
+
+	for (i = 0; i < nr_cpus; ++i)
+		if (cpus[i] == (mpidr & MPIDR_HWID_BITMASK))
+			return i;
+	return -1;
+}
+
 static void cpu_set(int fdtnode __unused, u32 regval, void *info __unused)
 {
 	int cpu = nr_cpus++;
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 9a208ff729b7..7e448dc81a6a 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -78,8 +78,13 @@ static inline type get_##reg(void)				\
 
 DEFINE_GET_SYSREG64(mpidr)
 
-/* Only support Aff0 for now, gicv2 only */
-#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
+#define MPIDR_HWID_BITMASK 0xff00ffffff
+extern int mpidr_to_cpu(unsigned long mpidr);
+
+#define MPIDR_LEVEL_SHIFT(level) \
+	(((1 << level) >> 1) << 3)
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+	((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
 
 extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
 extern bool is_user(void);
-- 
2.7.4

_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 03/11] arm/arm64: smp: support more than 8 cpus
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

By adding support for launching with gicv3 we can break the 8 vcpu
limit. This patch adds support to smp code and also selects the
vgic model corresponding to the host. The vgic model may also be
manually selected by adding e.g. -machine gic-version=3 to
extra_params.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v5: left cpus a u32 for now. Changing to u64 requires a change to
    devicetree. Will do it later. [Andre]
v4: improved commit message
---
 arm/run                   | 19 ++++++++++++-------
 arm/selftest.c            |  5 ++++-
 lib/arm/asm/processor.h   |  9 +++++++--
 lib/arm/asm/setup.h       |  4 ++--
 lib/arm/setup.c           | 10 ++++++++++
 lib/arm64/asm/processor.h |  9 +++++++--
 6 files changed, 42 insertions(+), 14 deletions(-)

diff --git a/arm/run b/arm/run
index a2f35ef6a7e6..2d0698619606 100755
--- a/arm/run
+++ b/arm/run
@@ -31,13 +31,6 @@ if [ -z "$ACCEL" ]; then
 	fi
 fi
 
-if [ "$HOST" = "aarch64" ] && [ "$ACCEL" = "kvm" ]; then
-	processor="host"
-	if [ "$ARCH" = "arm" ]; then
-		processor+=",aarch64=off"
-	fi
-fi
-
 qemu="${QEMU:-qemu-system-$ARCH_NAME}"
 qpath=$(which $qemu 2>/dev/null)
 
@@ -53,6 +46,18 @@ fi
 
 M='-machine virt'
 
+if [ "$ACCEL" = "kvm" ]; then
+	if $qemu $M,\? 2>&1 | grep gic-version > /dev/null; then
+		M+=',gic-version=host'
+	fi
+	if [ "$HOST" = "aarch64" ]; then
+		processor="host"
+		if [ "$ARCH" = "arm" ]; then
+			processor+=",aarch64=off"
+		fi
+	fi
+fi
+
 if ! $qemu $M -device '?' 2>&1 | grep virtconsole > /dev/null; then
 	echo "$qpath doesn't support virtio-console for chr-testdev. Exiting."
 	exit 2
diff --git a/arm/selftest.c b/arm/selftest.c
index 196164f5313d..2f117f795d2d 100644
--- a/arm/selftest.c
+++ b/arm/selftest.c
@@ -312,9 +312,10 @@ static bool psci_check(void)
 static cpumask_t smp_reported;
 static void cpu_report(void)
 {
+	unsigned long mpidr = get_mpidr();
 	int cpu = smp_processor_id();
 
-	report("CPU%d online", true, cpu);
+	report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == cpu, cpu, mpidr);
 	cpumask_set_cpu(cpu, &smp_reported);
 	halt();
 }
@@ -343,6 +344,7 @@ int main(int argc, char **argv)
 
 	} else if (strcmp(argv[1], "smp") == 0) {
 
+		unsigned long mpidr = get_mpidr();
 		int cpu;
 
 		report("PSCI version", psci_check());
@@ -353,6 +355,7 @@ int main(int argc, char **argv)
 			smp_boot_secondary(cpu, cpu_report);
 		}
 
+		report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == 0, 0, mpidr);
 		cpumask_set_cpu(0, &smp_reported);
 		while (!cpumask_full(&smp_reported))
 			cpu_relax();
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index 02f912f99974..ecf5bbe1824a 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -40,8 +40,13 @@ static inline unsigned long get_mpidr(void)
 	return mpidr;
 }
 
-/* Only support Aff0 for now, up to 4 cpus */
-#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
+#define MPIDR_HWID_BITMASK 0xffffff
+extern int mpidr_to_cpu(unsigned long mpidr);
+
+#define MPIDR_LEVEL_SHIFT(level) \
+	(((1 << level) >> 1) << 3)
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+	((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
 
 extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
 extern bool is_user(void);
diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
index cb8fdbd38dd5..1de99dd184d1 100644
--- a/lib/arm/asm/setup.h
+++ b/lib/arm/asm/setup.h
@@ -10,8 +10,8 @@
 #include <asm/page.h>
 #include <asm/pgtable-hwdef.h>
 
-#define NR_CPUS			8
-extern u32 cpus[NR_CPUS];
+#define NR_CPUS			255
+extern u32 cpus[NR_CPUS];	/* per-cpu IDs (MPIDRs) */
 extern int nr_cpus;
 
 #define NR_MEM_REGIONS		8
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 7e7b39f11dde..241bf9410447 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -30,6 +30,16 @@ int nr_cpus;
 struct mem_region mem_regions[NR_MEM_REGIONS];
 phys_addr_t __phys_offset, __phys_end;
 
+int mpidr_to_cpu(unsigned long mpidr)
+{
+	int i;
+
+	for (i = 0; i < nr_cpus; ++i)
+		if (cpus[i] == (mpidr & MPIDR_HWID_BITMASK))
+			return i;
+	return -1;
+}
+
 static void cpu_set(int fdtnode __unused, u32 regval, void *info __unused)
 {
 	int cpu = nr_cpus++;
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 9a208ff729b7..7e448dc81a6a 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -78,8 +78,13 @@ static inline type get_##reg(void)				\
 
 DEFINE_GET_SYSREG64(mpidr)
 
-/* Only support Aff0 for now, gicv2 only */
-#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
+#define MPIDR_HWID_BITMASK 0xff00ffffff
+extern int mpidr_to_cpu(unsigned long mpidr);
+
+#define MPIDR_LEVEL_SHIFT(level) \
+	(((1 << level) >> 1) << 3)
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+	((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
 
 extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
 extern bool is_user(void);
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 04/11] arm/arm64: add some delay routines
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Allow a thread to wait some specified amount of time. Can
specify in cycles, usecs, and msecs.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 lib/arm/asm/processor.h   | 19 +++++++++++++++++++
 lib/arm/processor.c       | 15 +++++++++++++++
 lib/arm64/asm/processor.h | 19 +++++++++++++++++++
 lib/arm64/processor.c     | 15 +++++++++++++++
 4 files changed, 68 insertions(+)

diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index ecf5bbe1824a..bc46d1f980ee 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -5,7 +5,9 @@
  *
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
+#include <libcflat.h>
 #include <asm/ptrace.h>
+#include <asm/barrier.h>
 
 enum vector {
 	EXCPTN_RST,
@@ -51,4 +53,21 @@ extern int mpidr_to_cpu(unsigned long mpidr);
 extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
 extern bool is_user(void);
 
+static inline u64 get_cntvct(void)
+{
+	u64 vct;
+	isb();
+	asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (vct));
+	return vct;
+}
+
+extern void delay(u64 cycles);
+extern void udelay(unsigned long usecs);
+
+static inline void mdelay(unsigned long msecs)
+{
+	while (msecs--)
+		udelay(1000);
+}
+
 #endif /* _ASMARM_PROCESSOR_H_ */
diff --git a/lib/arm/processor.c b/lib/arm/processor.c
index 54fdb87ef019..c2ee360df688 100644
--- a/lib/arm/processor.c
+++ b/lib/arm/processor.c
@@ -9,6 +9,7 @@
 #include <asm/ptrace.h>
 #include <asm/processor.h>
 #include <asm/thread_info.h>
+#include <asm/barrier.h>
 
 static const char *processor_modes[] = {
 	"USER_26", "FIQ_26" , "IRQ_26" , "SVC_26" ,
@@ -141,3 +142,17 @@ bool is_user(void)
 {
 	return current_thread_info()->flags & TIF_USER_MODE;
 }
+
+void delay(u64 cycles)
+{
+	u64 start = get_cntvct();
+	while ((get_cntvct() - start) < cycles)
+		cpu_relax();
+}
+
+void udelay(unsigned long usec)
+{
+	unsigned int frq;
+	asm volatile("mrc p15, 0, %0, c14, c0, 0" : "=r" (frq));
+	delay((u64)usec * frq / 1000000);
+}
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 7e448dc81a6a..94f7ce35b65c 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -17,8 +17,10 @@
 #define SCTLR_EL1_M	(1 << 0)
 
 #ifndef __ASSEMBLY__
+#include <libcflat.h>
 #include <asm/ptrace.h>
 #include <asm/esr.h>
+#include <asm/barrier.h>
 
 enum vector {
 	EL1T_SYNC,
@@ -89,5 +91,22 @@ extern int mpidr_to_cpu(unsigned long mpidr);
 extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
 extern bool is_user(void);
 
+static inline u64 get_cntvct(void)
+{
+	u64 vct;
+	isb();
+	asm volatile("mrs %0, cntvct_el0" : "=r" (vct));
+	return vct;
+}
+
+extern void delay(u64 cycles);
+extern void udelay(unsigned long usecs);
+
+static inline void mdelay(unsigned long msecs)
+{
+	while (msecs--)
+		udelay(1000);
+}
+
 #endif /* !__ASSEMBLY__ */
 #endif /* _ASMARM64_PROCESSOR_H_ */
diff --git a/lib/arm64/processor.c b/lib/arm64/processor.c
index deeab4ec9c8a..50fa835c6f1e 100644
--- a/lib/arm64/processor.c
+++ b/lib/arm64/processor.c
@@ -9,6 +9,7 @@
 #include <asm/ptrace.h>
 #include <asm/processor.h>
 #include <asm/thread_info.h>
+#include <asm/barrier.h>
 
 static const char *vector_names[] = {
 	"el1t_sync",
@@ -253,3 +254,17 @@ bool is_user(void)
 {
 	return current_thread_info()->flags & TIF_USER_MODE;
 }
+
+void delay(u64 cycles)
+{
+	u64 start = get_cntvct();
+	while ((get_cntvct() - start) < cycles)
+		cpu_relax();
+}
+
+void udelay(unsigned long usec)
+{
+	unsigned int frq;
+	asm volatile("mrs %0, cntfrq_el0" : "=r" (frq));
+	delay((u64)usec * frq / 1000000);
+}
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 04/11] arm/arm64: add some delay routines
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Allow a thread to wait some specified amount of time. Can
specify in cycles, usecs, and msecs.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 lib/arm/asm/processor.h   | 19 +++++++++++++++++++
 lib/arm/processor.c       | 15 +++++++++++++++
 lib/arm64/asm/processor.h | 19 +++++++++++++++++++
 lib/arm64/processor.c     | 15 +++++++++++++++
 4 files changed, 68 insertions(+)

diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index ecf5bbe1824a..bc46d1f980ee 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -5,7 +5,9 @@
  *
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
+#include <libcflat.h>
 #include <asm/ptrace.h>
+#include <asm/barrier.h>
 
 enum vector {
 	EXCPTN_RST,
@@ -51,4 +53,21 @@ extern int mpidr_to_cpu(unsigned long mpidr);
 extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
 extern bool is_user(void);
 
+static inline u64 get_cntvct(void)
+{
+	u64 vct;
+	isb();
+	asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (vct));
+	return vct;
+}
+
+extern void delay(u64 cycles);
+extern void udelay(unsigned long usecs);
+
+static inline void mdelay(unsigned long msecs)
+{
+	while (msecs--)
+		udelay(1000);
+}
+
 #endif /* _ASMARM_PROCESSOR_H_ */
diff --git a/lib/arm/processor.c b/lib/arm/processor.c
index 54fdb87ef019..c2ee360df688 100644
--- a/lib/arm/processor.c
+++ b/lib/arm/processor.c
@@ -9,6 +9,7 @@
 #include <asm/ptrace.h>
 #include <asm/processor.h>
 #include <asm/thread_info.h>
+#include <asm/barrier.h>
 
 static const char *processor_modes[] = {
 	"USER_26", "FIQ_26" , "IRQ_26" , "SVC_26" ,
@@ -141,3 +142,17 @@ bool is_user(void)
 {
 	return current_thread_info()->flags & TIF_USER_MODE;
 }
+
+void delay(u64 cycles)
+{
+	u64 start = get_cntvct();
+	while ((get_cntvct() - start) < cycles)
+		cpu_relax();
+}
+
+void udelay(unsigned long usec)
+{
+	unsigned int frq;
+	asm volatile("mrc p15, 0, %0, c14, c0, 0" : "=r" (frq));
+	delay((u64)usec * frq / 1000000);
+}
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 7e448dc81a6a..94f7ce35b65c 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -17,8 +17,10 @@
 #define SCTLR_EL1_M	(1 << 0)
 
 #ifndef __ASSEMBLY__
+#include <libcflat.h>
 #include <asm/ptrace.h>
 #include <asm/esr.h>
+#include <asm/barrier.h>
 
 enum vector {
 	EL1T_SYNC,
@@ -89,5 +91,22 @@ extern int mpidr_to_cpu(unsigned long mpidr);
 extern void start_usr(void (*func)(void *arg), void *arg, unsigned long sp_usr);
 extern bool is_user(void);
 
+static inline u64 get_cntvct(void)
+{
+	u64 vct;
+	isb();
+	asm volatile("mrs %0, cntvct_el0" : "=r" (vct));
+	return vct;
+}
+
+extern void delay(u64 cycles);
+extern void udelay(unsigned long usecs);
+
+static inline void mdelay(unsigned long msecs)
+{
+	while (msecs--)
+		udelay(1000);
+}
+
 #endif /* !__ASSEMBLY__ */
 #endif /* _ASMARM64_PROCESSOR_H_ */
diff --git a/lib/arm64/processor.c b/lib/arm64/processor.c
index deeab4ec9c8a..50fa835c6f1e 100644
--- a/lib/arm64/processor.c
+++ b/lib/arm64/processor.c
@@ -9,6 +9,7 @@
 #include <asm/ptrace.h>
 #include <asm/processor.h>
 #include <asm/thread_info.h>
+#include <asm/barrier.h>
 
 static const char *vector_names[] = {
 	"el1t_sync",
@@ -253,3 +254,17 @@ bool is_user(void)
 {
 	return current_thread_info()->flags & TIF_USER_MODE;
 }
+
+void delay(u64 cycles)
+{
+	u64 start = get_cntvct();
+	while ((get_cntvct() - start) < cycles)
+		cpu_relax();
+}
+
+void udelay(unsigned long usec)
+{
+	unsigned int frq;
+	asm volatile("mrs %0, cntfrq_el0" : "=r" (frq));
+	delay((u64)usec * frq / 1000000);
+}
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 05/11] arm/arm64: irq enable/disable
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 lib/arm/asm/processor.h   | 10 ++++++++++
 lib/arm64/asm/processor.h | 10 ++++++++++
 2 files changed, 20 insertions(+)

diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index bc46d1f980ee..959ecda5dced 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -35,6 +35,16 @@ static inline unsigned long current_cpsr(void)
 
 #define current_mode() (current_cpsr() & MODE_MASK)
 
+static inline void local_irq_enable(void)
+{
+	asm volatile("cpsie i" : : : "memory", "cc");
+}
+
+static inline void local_irq_disable(void)
+{
+	asm volatile("cpsid i" : : : "memory", "cc");
+}
+
 static inline unsigned long get_mpidr(void)
 {
 	unsigned long mpidr;
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 94f7ce35b65c..d54a4ed1c187 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -68,6 +68,16 @@ static inline unsigned long current_level(void)
 	return el & 0xc;
 }
 
+static inline void local_irq_enable(void)
+{
+	asm volatile("msr daifclr, #2" : : : "memory");
+}
+
+static inline void local_irq_disable(void)
+{
+	asm volatile("msr daifset, #2" : : : "memory");
+}
+
 #define DEFINE_GET_SYSREG(reg, type)				\
 static inline type get_##reg(void)				\
 {								\
-- 
2.7.4

_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 05/11] arm/arm64: irq enable/disable
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 lib/arm/asm/processor.h   | 10 ++++++++++
 lib/arm64/asm/processor.h | 10 ++++++++++
 2 files changed, 20 insertions(+)

diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index bc46d1f980ee..959ecda5dced 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -35,6 +35,16 @@ static inline unsigned long current_cpsr(void)
 
 #define current_mode() (current_cpsr() & MODE_MASK)
 
+static inline void local_irq_enable(void)
+{
+	asm volatile("cpsie i" : : : "memory", "cc");
+}
+
+static inline void local_irq_disable(void)
+{
+	asm volatile("cpsid i" : : : "memory", "cc");
+}
+
 static inline unsigned long get_mpidr(void)
 {
 	unsigned long mpidr;
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 94f7ce35b65c..d54a4ed1c187 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -68,6 +68,16 @@ static inline unsigned long current_level(void)
 	return el & 0xc;
 }
 
+static inline void local_irq_enable(void)
+{
+	asm volatile("msr daifclr, #2" : : : "memory");
+}
+
+static inline void local_irq_disable(void)
+{
+	asm volatile("msr daifset, #2" : : : "memory");
+}
+
 #define DEFINE_GET_SYSREG(reg, type)				\
 static inline type get_##reg(void)				\
 {								\
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Add some gicv2 support. This just adds init and enable
functions, allowing unit tests to start messing with it.

Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v5: share/use only the modern register names [Andre]
v4:
 - only take defines from kernel we need now [Andre]
 - moved defines to asm/gic.h so they'll be shared with v3 [drew]
 - simplify enable by not caring if we reinit the distributor [drew]
 - init all GICD_INT_DEF_PRI_X4 registers [Eric]
---
 arm/Makefile.common    |  1 +
 lib/arm/asm/gic-v2.h   | 34 ++++++++++++++++++++++
 lib/arm/asm/gic.h      | 37 ++++++++++++++++++++++++
 lib/arm/gic.c          | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
 lib/arm64/asm/gic-v2.h |  1 +
 lib/arm64/asm/gic.h    |  1 +
 6 files changed, 150 insertions(+)
 create mode 100644 lib/arm/asm/gic-v2.h
 create mode 100644 lib/arm/asm/gic.h
 create mode 100644 lib/arm/gic.c
 create mode 100644 lib/arm64/asm/gic-v2.h
 create mode 100644 lib/arm64/asm/gic.h

diff --git a/arm/Makefile.common b/arm/Makefile.common
index ccb554d9251a..41239c37e092 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
 cflatobjs += lib/arm/bitops.o
 cflatobjs += lib/arm/psci.o
 cflatobjs += lib/arm/smp.o
+cflatobjs += lib/arm/gic.o
 
 libeabi = lib/arm/libeabi.a
 eabiobjs = lib/arm/eabi_compat.o
diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
new file mode 100644
index 000000000000..c2d5fecd4886
--- /dev/null
+++ b/lib/arm/asm/gic-v2.h
@@ -0,0 +1,34 @@
+/*
+ * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V2_H_
+#define _ASMARM_GIC_V2_H_
+
+#ifndef _ASMARM_GIC_H_
+#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
+#endif
+
+#define GICD_ENABLE			0x1
+#define GICC_ENABLE			0x1
+
+#ifndef __ASSEMBLY__
+
+struct gicv2_data {
+	void *dist_base;
+	void *cpu_base;
+	unsigned int irq_nr;
+};
+extern struct gicv2_data gicv2_data;
+
+#define gicv2_dist_base()		(gicv2_data.dist_base)
+#define gicv2_cpu_base()		(gicv2_data.cpu_base)
+
+extern int gicv2_init(void);
+extern void gicv2_enable_defaults(void);
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V2_H_ */
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
new file mode 100644
index 000000000000..d44e47bcf404
--- /dev/null
+++ b/lib/arm/asm/gic.h
@@ -0,0 +1,37 @@
+/*
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_H_
+#define _ASMARM_GIC_H_
+
+#include <asm/gic-v2.h>
+
+#define GICD_CTLR			0x0000
+#define GICD_TYPER			0x0004
+#define GICD_ISENABLER			0x0100
+#define GICD_IPRIORITYR			0x0400
+
+#define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1) * 32)
+#define GICD_INT_EN_SET_SGI		0x0000ffff
+#define GICD_INT_DEF_PRI_X4		0xa0a0a0a0
+
+#define GICC_CTLR			0x0000
+#define GICC_PMR			0x0004
+
+#define GICC_INT_PRI_THRESHOLD		0xf0
+
+#ifndef __ASSEMBLY__
+
+/*
+ * gic_init will try to find all known gics, and then
+ * initialize the gic data for the one found.
+ * returns
+ *  0   : no gic was found
+ *  > 0 : the gic version of the gic found
+ */
+extern int gic_init(void);
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_H_ */
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
new file mode 100644
index 000000000000..d655105e058b
--- /dev/null
+++ b/lib/arm/gic.c
@@ -0,0 +1,76 @@
+/*
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <devicetree.h>
+#include <asm/gic.h>
+#include <asm/io.h>
+
+struct gicv2_data gicv2_data;
+
+/*
+ * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
+ */
+static bool
+gic_get_dt_bases(const char *compatible, void **base1, void **base2)
+{
+	struct dt_pbus_reg reg;
+	struct dt_device gic;
+	struct dt_bus bus;
+	int node, ret;
+
+	dt_bus_init_defaults(&bus);
+	dt_device_init(&gic, &bus, NULL);
+
+	node = dt_device_find_compatible(&gic, compatible);
+	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
+
+	if (node == -FDT_ERR_NOTFOUND)
+		return false;
+
+	dt_device_bind_node(&gic, node);
+
+	ret = dt_pbus_translate(&gic, 0, &reg);
+	assert(ret == 0);
+	*base1 = ioremap(reg.addr, reg.size);
+
+	ret = dt_pbus_translate(&gic, 1, &reg);
+	assert(ret == 0);
+	*base2 = ioremap(reg.addr, reg.size);
+
+	return true;
+}
+
+int gicv2_init(void)
+{
+	return gic_get_dt_bases("arm,cortex-a15-gic",
+			&gicv2_data.dist_base, &gicv2_data.cpu_base);
+}
+
+int gic_init(void)
+{
+	if (gicv2_init())
+		return 2;
+	return 0;
+}
+
+void gicv2_enable_defaults(void)
+{
+	void *dist = gicv2_dist_base();
+	void *cpu_base = gicv2_cpu_base();
+	unsigned int i;
+
+	gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
+	if (gicv2_data.irq_nr > 1020)
+		gicv2_data.irq_nr = 1020;
+
+	for (i = 0; i < gicv2_data.irq_nr; i += 4)
+		writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
+
+	writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
+	writel(GICD_ENABLE, dist + GICD_CTLR);
+
+	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
+	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
+}
diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
new file mode 100644
index 000000000000..52226624a209
--- /dev/null
+++ b/lib/arm64/asm/gic-v2.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v2.h"
diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
new file mode 100644
index 000000000000..e5eb302a31b4
--- /dev/null
+++ b/lib/arm64/asm/gic.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic.h"
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Add some gicv2 support. This just adds init and enable
functions, allowing unit tests to start messing with it.

Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v5: share/use only the modern register names [Andre]
v4:
 - only take defines from kernel we need now [Andre]
 - moved defines to asm/gic.h so they'll be shared with v3 [drew]
 - simplify enable by not caring if we reinit the distributor [drew]
 - init all GICD_INT_DEF_PRI_X4 registers [Eric]
---
 arm/Makefile.common    |  1 +
 lib/arm/asm/gic-v2.h   | 34 ++++++++++++++++++++++
 lib/arm/asm/gic.h      | 37 ++++++++++++++++++++++++
 lib/arm/gic.c          | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
 lib/arm64/asm/gic-v2.h |  1 +
 lib/arm64/asm/gic.h    |  1 +
 6 files changed, 150 insertions(+)
 create mode 100644 lib/arm/asm/gic-v2.h
 create mode 100644 lib/arm/asm/gic.h
 create mode 100644 lib/arm/gic.c
 create mode 100644 lib/arm64/asm/gic-v2.h
 create mode 100644 lib/arm64/asm/gic.h

diff --git a/arm/Makefile.common b/arm/Makefile.common
index ccb554d9251a..41239c37e092 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
 cflatobjs += lib/arm/bitops.o
 cflatobjs += lib/arm/psci.o
 cflatobjs += lib/arm/smp.o
+cflatobjs += lib/arm/gic.o
 
 libeabi = lib/arm/libeabi.a
 eabiobjs = lib/arm/eabi_compat.o
diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
new file mode 100644
index 000000000000..c2d5fecd4886
--- /dev/null
+++ b/lib/arm/asm/gic-v2.h
@@ -0,0 +1,34 @@
+/*
+ * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V2_H_
+#define _ASMARM_GIC_V2_H_
+
+#ifndef _ASMARM_GIC_H_
+#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
+#endif
+
+#define GICD_ENABLE			0x1
+#define GICC_ENABLE			0x1
+
+#ifndef __ASSEMBLY__
+
+struct gicv2_data {
+	void *dist_base;
+	void *cpu_base;
+	unsigned int irq_nr;
+};
+extern struct gicv2_data gicv2_data;
+
+#define gicv2_dist_base()		(gicv2_data.dist_base)
+#define gicv2_cpu_base()		(gicv2_data.cpu_base)
+
+extern int gicv2_init(void);
+extern void gicv2_enable_defaults(void);
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V2_H_ */
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
new file mode 100644
index 000000000000..d44e47bcf404
--- /dev/null
+++ b/lib/arm/asm/gic.h
@@ -0,0 +1,37 @@
+/*
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_H_
+#define _ASMARM_GIC_H_
+
+#include <asm/gic-v2.h>
+
+#define GICD_CTLR			0x0000
+#define GICD_TYPER			0x0004
+#define GICD_ISENABLER			0x0100
+#define GICD_IPRIORITYR			0x0400
+
+#define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1) * 32)
+#define GICD_INT_EN_SET_SGI		0x0000ffff
+#define GICD_INT_DEF_PRI_X4		0xa0a0a0a0
+
+#define GICC_CTLR			0x0000
+#define GICC_PMR			0x0004
+
+#define GICC_INT_PRI_THRESHOLD		0xf0
+
+#ifndef __ASSEMBLY__
+
+/*
+ * gic_init will try to find all known gics, and then
+ * initialize the gic data for the one found.
+ * returns
+ *  0   : no gic was found
+ *  > 0 : the gic version of the gic found
+ */
+extern int gic_init(void);
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_H_ */
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
new file mode 100644
index 000000000000..d655105e058b
--- /dev/null
+++ b/lib/arm/gic.c
@@ -0,0 +1,76 @@
+/*
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <devicetree.h>
+#include <asm/gic.h>
+#include <asm/io.h>
+
+struct gicv2_data gicv2_data;
+
+/*
+ * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
+ */
+static bool
+gic_get_dt_bases(const char *compatible, void **base1, void **base2)
+{
+	struct dt_pbus_reg reg;
+	struct dt_device gic;
+	struct dt_bus bus;
+	int node, ret;
+
+	dt_bus_init_defaults(&bus);
+	dt_device_init(&gic, &bus, NULL);
+
+	node = dt_device_find_compatible(&gic, compatible);
+	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
+
+	if (node == -FDT_ERR_NOTFOUND)
+		return false;
+
+	dt_device_bind_node(&gic, node);
+
+	ret = dt_pbus_translate(&gic, 0, &reg);
+	assert(ret == 0);
+	*base1 = ioremap(reg.addr, reg.size);
+
+	ret = dt_pbus_translate(&gic, 1, &reg);
+	assert(ret == 0);
+	*base2 = ioremap(reg.addr, reg.size);
+
+	return true;
+}
+
+int gicv2_init(void)
+{
+	return gic_get_dt_bases("arm,cortex-a15-gic",
+			&gicv2_data.dist_base, &gicv2_data.cpu_base);
+}
+
+int gic_init(void)
+{
+	if (gicv2_init())
+		return 2;
+	return 0;
+}
+
+void gicv2_enable_defaults(void)
+{
+	void *dist = gicv2_dist_base();
+	void *cpu_base = gicv2_cpu_base();
+	unsigned int i;
+
+	gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
+	if (gicv2_data.irq_nr > 1020)
+		gicv2_data.irq_nr = 1020;
+
+	for (i = 0; i < gicv2_data.irq_nr; i += 4)
+		writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
+
+	writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
+	writel(GICD_ENABLE, dist + GICD_CTLR);
+
+	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
+	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
+}
diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
new file mode 100644
index 000000000000..52226624a209
--- /dev/null
+++ b/lib/arm64/asm/gic-v2.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v2.h"
diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
new file mode 100644
index 000000000000..e5eb302a31b4
--- /dev/null
+++ b/lib/arm64/asm/gic.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic.h"
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 07/11] arm/arm64: gicv2: add an IPI test
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v5: use modern registers [Andre]
v4: properly mask irqnr in ipi_handler
v2: add more details in the output if a test fails,
    report spurious interrupts if we get them
---
 arm/Makefile.common  |   6 +-
 arm/gic.c            | 195 +++++++++++++++++++++++++++++++++++++++++++++++++++
 arm/unittests.cfg    |   7 ++
 lib/arm/asm/gic-v2.h |   2 +
 lib/arm/asm/gic.h    |   4 ++
 5 files changed, 211 insertions(+), 3 deletions(-)
 create mode 100644 arm/gic.c

diff --git a/arm/Makefile.common b/arm/Makefile.common
index 41239c37e092..bc38183ab86e 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -9,9 +9,9 @@ ifeq ($(LOADADDR),)
 	LOADADDR = 0x40000000
 endif
 
-tests-common = \
-	$(TEST_DIR)/selftest.flat \
-	$(TEST_DIR)/spinlock-test.flat
+tests-common  = $(TEST_DIR)/selftest.flat
+tests-common += $(TEST_DIR)/spinlock-test.flat
+tests-common += $(TEST_DIR)/gic.flat
 
 all: test_cases
 
diff --git a/arm/gic.c b/arm/gic.c
new file mode 100644
index 000000000000..06092aec7c08
--- /dev/null
+++ b/arm/gic.c
@@ -0,0 +1,195 @@
+/*
+ * GIC tests
+ *
+ * GICv2
+ *   + test sending/receiving IPIs
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <libcflat.h>
+#include <asm/setup.h>
+#include <asm/processor.h>
+#include <asm/gic.h>
+#include <asm/smp.h>
+#include <asm/barrier.h>
+#include <asm/io.h>
+
+static int gic_version;
+static int acked[NR_CPUS], spurious[NR_CPUS];
+static cpumask_t ready;
+
+static void nr_cpu_check(int nr)
+{
+	if (nr_cpus < nr)
+		report_abort("At least %d cpus required", nr);
+}
+
+static void wait_on_ready(void)
+{
+	cpumask_set_cpu(smp_processor_id(), &ready);
+	while (!cpumask_full(&ready))
+		cpu_relax();
+}
+
+static void check_acked(cpumask_t *mask)
+{
+	int missing = 0, extra = 0, unexpected = 0;
+	int nr_pass, cpu, i;
+
+	/* Wait up to 5s for all interrupts to be delivered */
+	for (i = 0; i < 50; ++i) {
+		mdelay(100);
+		nr_pass = 0;
+		for_each_present_cpu(cpu) {
+			smp_rmb();
+			nr_pass += cpumask_test_cpu(cpu, mask) ?
+				acked[cpu] == 1 : acked[cpu] == 0;
+		}
+		if (nr_pass == nr_cpus) {
+			report("Completed in %d ms", true, ++i * 100);
+			return;
+		}
+	}
+
+	for_each_present_cpu(cpu) {
+		if (cpumask_test_cpu(cpu, mask)) {
+			if (!acked[cpu])
+				++missing;
+			else if (acked[cpu] > 1)
+				++extra;
+		} else {
+			if (acked[cpu])
+				++unexpected;
+		}
+	}
+
+	report("Timed-out (5s). ACKS: missing=%d extra=%d unexpected=%d",
+	       false, missing, extra, unexpected);
+}
+
+static void ipi_handler(struct pt_regs *regs __unused)
+{
+	u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
+	u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
+
+	if (irqnr != GICC_INT_SPURIOUS) {
+		writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+		smp_rmb(); /* pairs with wmb in ipi_test functions */
+		++acked[smp_processor_id()];
+		smp_wmb(); /* pairs with rmb in check_acked */
+	} else {
+		++spurious[smp_processor_id()];
+		smp_wmb();
+	}
+}
+
+static void ipi_test_self(void)
+{
+	cpumask_t mask;
+
+	report_prefix_push("self");
+	memset(acked, 0, sizeof(acked));
+	smp_wmb();
+	cpumask_clear(&mask);
+	cpumask_set_cpu(0, &mask);
+	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+	check_acked(&mask);
+	report_prefix_pop();
+}
+
+static void ipi_test_smp(void)
+{
+	cpumask_t mask;
+	unsigned long tlist;
+
+	report_prefix_push("target-list");
+	memset(acked, 0, sizeof(acked));
+	smp_wmb();
+	tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
+	cpumask_bits(&mask)[0] = tlist;
+	writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
+	check_acked(&mask);
+	report_prefix_pop();
+
+	report_prefix_push("broadcast");
+	memset(acked, 0, sizeof(acked));
+	smp_wmb();
+	cpumask_copy(&mask, &cpu_present_mask);
+	cpumask_clear_cpu(0, &mask);
+	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+	check_acked(&mask);
+	report_prefix_pop();
+}
+
+static void ipi_enable(void)
+{
+	gicv2_enable_defaults();
+#ifdef __arm__
+	install_exception_handler(EXCPTN_IRQ, ipi_handler);
+#else
+	install_irq_handler(EL1H_IRQ, ipi_handler);
+#endif
+	local_irq_enable();
+}
+
+static void ipi_recv(void)
+{
+	ipi_enable();
+	cpumask_set_cpu(smp_processor_id(), &ready);
+	while (1)
+		wfi();
+}
+
+int main(int argc, char **argv)
+{
+	char pfx[8];
+	int cpu;
+
+	gic_version = gic_init();
+	if (!gic_version)
+		report_abort("No gic present!");
+
+	snprintf(pfx, 8, "gicv%d", gic_version);
+	report_prefix_push(pfx);
+
+	if (argc < 2) {
+
+		report_prefix_push("ipi");
+		ipi_enable();
+		ipi_test_self();
+		report_prefix_pop();
+
+	} else if (!strcmp(argv[1], "ipi")) {
+
+		report_prefix_push(argv[1]);
+		nr_cpu_check(2);
+
+		for_each_present_cpu(cpu) {
+			if (cpu == 0)
+				continue;
+			smp_boot_secondary(cpu, ipi_recv);
+		}
+		ipi_enable();
+		wait_on_ready();
+		ipi_test_self();
+		ipi_test_smp();
+
+		smp_rmb();
+		for_each_present_cpu(cpu) {
+			if (spurious[cpu]) {
+				printf("ipi: WARN: cpu%d got %d spurious "
+				       "interrupts\n",
+				       spurious[cpu], smp_processor_id());
+			}
+		}
+
+		report_prefix_pop();
+
+	} else {
+		report_abort("Unknown subtest '%s'", argv[1]);
+	}
+
+	return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 3f6fa45c587e..68bf5cd6008f 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -54,3 +54,10 @@ file = selftest.flat
 smp = $MAX_SMP
 extra_params = -append 'smp'
 groups = selftest
+
+# Test GIC emulation
+[gicv2-ipi]
+file = gic.flat
+smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
+extra_params = -machine gic-version=2 -append 'ipi'
+groups = gic
diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
index c2d5fecd4886..8b3f7ed6790c 100644
--- a/lib/arm/asm/gic-v2.h
+++ b/lib/arm/asm/gic-v2.h
@@ -13,7 +13,9 @@
 #endif
 
 #define GICD_ENABLE			0x1
+
 #define GICC_ENABLE			0x1
+#define GICC_IAR_INT_ID_MASK		0x3ff
 
 #ifndef __ASSEMBLY__
 
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index d44e47bcf404..a16645708c35 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -12,6 +12,7 @@
 #define GICD_TYPER			0x0004
 #define GICD_ISENABLER			0x0100
 #define GICD_IPRIORITYR			0x0400
+#define GICD_SGIR			0x0f00
 
 #define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1) * 32)
 #define GICD_INT_EN_SET_SGI		0x0000ffff
@@ -19,8 +20,11 @@
 
 #define GICC_CTLR			0x0000
 #define GICC_PMR			0x0004
+#define GICC_IAR			0x000c
+#define GICC_EOIR			0x0010
 
 #define GICC_INT_PRI_THRESHOLD		0xf0
+#define GICC_INT_SPURIOUS		0x3ff
 
 #ifndef __ASSEMBLY__
 
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 07/11] arm/arm64: gicv2: add an IPI test
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
v5: use modern registers [Andre]
v4: properly mask irqnr in ipi_handler
v2: add more details in the output if a test fails,
    report spurious interrupts if we get them
---
 arm/Makefile.common  |   6 +-
 arm/gic.c            | 195 +++++++++++++++++++++++++++++++++++++++++++++++++++
 arm/unittests.cfg    |   7 ++
 lib/arm/asm/gic-v2.h |   2 +
 lib/arm/asm/gic.h    |   4 ++
 5 files changed, 211 insertions(+), 3 deletions(-)
 create mode 100644 arm/gic.c

diff --git a/arm/Makefile.common b/arm/Makefile.common
index 41239c37e092..bc38183ab86e 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -9,9 +9,9 @@ ifeq ($(LOADADDR),)
 	LOADADDR = 0x40000000
 endif
 
-tests-common = \
-	$(TEST_DIR)/selftest.flat \
-	$(TEST_DIR)/spinlock-test.flat
+tests-common  = $(TEST_DIR)/selftest.flat
+tests-common += $(TEST_DIR)/spinlock-test.flat
+tests-common += $(TEST_DIR)/gic.flat
 
 all: test_cases
 
diff --git a/arm/gic.c b/arm/gic.c
new file mode 100644
index 000000000000..06092aec7c08
--- /dev/null
+++ b/arm/gic.c
@@ -0,0 +1,195 @@
+/*
+ * GIC tests
+ *
+ * GICv2
+ *   + test sending/receiving IPIs
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <libcflat.h>
+#include <asm/setup.h>
+#include <asm/processor.h>
+#include <asm/gic.h>
+#include <asm/smp.h>
+#include <asm/barrier.h>
+#include <asm/io.h>
+
+static int gic_version;
+static int acked[NR_CPUS], spurious[NR_CPUS];
+static cpumask_t ready;
+
+static void nr_cpu_check(int nr)
+{
+	if (nr_cpus < nr)
+		report_abort("At least %d cpus required", nr);
+}
+
+static void wait_on_ready(void)
+{
+	cpumask_set_cpu(smp_processor_id(), &ready);
+	while (!cpumask_full(&ready))
+		cpu_relax();
+}
+
+static void check_acked(cpumask_t *mask)
+{
+	int missing = 0, extra = 0, unexpected = 0;
+	int nr_pass, cpu, i;
+
+	/* Wait up to 5s for all interrupts to be delivered */
+	for (i = 0; i < 50; ++i) {
+		mdelay(100);
+		nr_pass = 0;
+		for_each_present_cpu(cpu) {
+			smp_rmb();
+			nr_pass += cpumask_test_cpu(cpu, mask) ?
+				acked[cpu] == 1 : acked[cpu] == 0;
+		}
+		if (nr_pass == nr_cpus) {
+			report("Completed in %d ms", true, ++i * 100);
+			return;
+		}
+	}
+
+	for_each_present_cpu(cpu) {
+		if (cpumask_test_cpu(cpu, mask)) {
+			if (!acked[cpu])
+				++missing;
+			else if (acked[cpu] > 1)
+				++extra;
+		} else {
+			if (acked[cpu])
+				++unexpected;
+		}
+	}
+
+	report("Timed-out (5s). ACKS: missing=%d extra=%d unexpected=%d",
+	       false, missing, extra, unexpected);
+}
+
+static void ipi_handler(struct pt_regs *regs __unused)
+{
+	u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
+	u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
+
+	if (irqnr != GICC_INT_SPURIOUS) {
+		writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+		smp_rmb(); /* pairs with wmb in ipi_test functions */
+		++acked[smp_processor_id()];
+		smp_wmb(); /* pairs with rmb in check_acked */
+	} else {
+		++spurious[smp_processor_id()];
+		smp_wmb();
+	}
+}
+
+static void ipi_test_self(void)
+{
+	cpumask_t mask;
+
+	report_prefix_push("self");
+	memset(acked, 0, sizeof(acked));
+	smp_wmb();
+	cpumask_clear(&mask);
+	cpumask_set_cpu(0, &mask);
+	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+	check_acked(&mask);
+	report_prefix_pop();
+}
+
+static void ipi_test_smp(void)
+{
+	cpumask_t mask;
+	unsigned long tlist;
+
+	report_prefix_push("target-list");
+	memset(acked, 0, sizeof(acked));
+	smp_wmb();
+	tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
+	cpumask_bits(&mask)[0] = tlist;
+	writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
+	check_acked(&mask);
+	report_prefix_pop();
+
+	report_prefix_push("broadcast");
+	memset(acked, 0, sizeof(acked));
+	smp_wmb();
+	cpumask_copy(&mask, &cpu_present_mask);
+	cpumask_clear_cpu(0, &mask);
+	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+	check_acked(&mask);
+	report_prefix_pop();
+}
+
+static void ipi_enable(void)
+{
+	gicv2_enable_defaults();
+#ifdef __arm__
+	install_exception_handler(EXCPTN_IRQ, ipi_handler);
+#else
+	install_irq_handler(EL1H_IRQ, ipi_handler);
+#endif
+	local_irq_enable();
+}
+
+static void ipi_recv(void)
+{
+	ipi_enable();
+	cpumask_set_cpu(smp_processor_id(), &ready);
+	while (1)
+		wfi();
+}
+
+int main(int argc, char **argv)
+{
+	char pfx[8];
+	int cpu;
+
+	gic_version = gic_init();
+	if (!gic_version)
+		report_abort("No gic present!");
+
+	snprintf(pfx, 8, "gicv%d", gic_version);
+	report_prefix_push(pfx);
+
+	if (argc < 2) {
+
+		report_prefix_push("ipi");
+		ipi_enable();
+		ipi_test_self();
+		report_prefix_pop();
+
+	} else if (!strcmp(argv[1], "ipi")) {
+
+		report_prefix_push(argv[1]);
+		nr_cpu_check(2);
+
+		for_each_present_cpu(cpu) {
+			if (cpu == 0)
+				continue;
+			smp_boot_secondary(cpu, ipi_recv);
+		}
+		ipi_enable();
+		wait_on_ready();
+		ipi_test_self();
+		ipi_test_smp();
+
+		smp_rmb();
+		for_each_present_cpu(cpu) {
+			if (spurious[cpu]) {
+				printf("ipi: WARN: cpu%d got %d spurious "
+				       "interrupts\n",
+				       spurious[cpu], smp_processor_id());
+			}
+		}
+
+		report_prefix_pop();
+
+	} else {
+		report_abort("Unknown subtest '%s'", argv[1]);
+	}
+
+	return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 3f6fa45c587e..68bf5cd6008f 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -54,3 +54,10 @@ file = selftest.flat
 smp = $MAX_SMP
 extra_params = -append 'smp'
 groups = selftest
+
+# Test GIC emulation
+[gicv2-ipi]
+file = gic.flat
+smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
+extra_params = -machine gic-version=2 -append 'ipi'
+groups = gic
diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
index c2d5fecd4886..8b3f7ed6790c 100644
--- a/lib/arm/asm/gic-v2.h
+++ b/lib/arm/asm/gic-v2.h
@@ -13,7 +13,9 @@
 #endif
 
 #define GICD_ENABLE			0x1
+
 #define GICC_ENABLE			0x1
+#define GICC_IAR_INT_ID_MASK		0x3ff
 
 #ifndef __ASSEMBLY__
 
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index d44e47bcf404..a16645708c35 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -12,6 +12,7 @@
 #define GICD_TYPER			0x0004
 #define GICD_ISENABLER			0x0100
 #define GICD_IPRIORITYR			0x0400
+#define GICD_SGIR			0x0f00
 
 #define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1) * 32)
 #define GICD_INT_EN_SET_SGI		0x0000ffff
@@ -19,8 +20,11 @@
 
 #define GICC_CTLR			0x0000
 #define GICC_PMR			0x0004
+#define GICC_IAR			0x000c
+#define GICC_EOIR			0x0010
 
 #define GICC_INT_PRI_THRESHOLD		0xf0
+#define GICC_INT_SPURIOUS		0x3ff
 
 #ifndef __ASSEMBLY__
 
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 08/11] libcflat: add IS_ALIGNED() macro, and page sizes
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall, Peter Xu

From: Peter Xu <peterx@redhat.com>

These macros will be useful to do page alignment checks.

Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
[drew: also added SZ_64K]
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 lib/libcflat.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/libcflat.h b/lib/libcflat.h
index 82005f5d014f..143fc53061fe 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -33,6 +33,12 @@
 #define __ALIGN_MASK(x, mask)	(((x) + (mask)) & ~(mask))
 #define __ALIGN(x, a)		__ALIGN_MASK(x, (typeof(x))(a) - 1)
 #define ALIGN(x, a)		__ALIGN((x), (a))
+#define IS_ALIGNED(x, a)	(((x) & ((typeof(x))(a) - 1)) == 0)
+
+#define SZ_4K			(0x1000)
+#define SZ_64K			(0x10000)
+#define SZ_2M			(0x200000)
+#define SZ_1G			(0x40000000)
 
 typedef uint8_t		u8;
 typedef int8_t		s8;
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 08/11] libcflat: add IS_ALIGNED() macro, and page sizes
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall, Peter Xu

From: Peter Xu <peterx@redhat.com>

These macros will be useful to do page alignment checks.

Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
[drew: also added SZ_64K]
Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 lib/libcflat.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/libcflat.h b/lib/libcflat.h
index 82005f5d014f..143fc53061fe 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -33,6 +33,12 @@
 #define __ALIGN_MASK(x, mask)	(((x) + (mask)) & ~(mask))
 #define __ALIGN(x, a)		__ALIGN_MASK(x, (typeof(x))(a) - 1)
 #define ALIGN(x, a)		__ALIGN((x), (a))
+#define IS_ALIGNED(x, a)	(((x) & ((typeof(x))(a) - 1)) == 0)
+
+#define SZ_4K			(0x1000)
+#define SZ_64K			(0x10000)
+#define SZ_2M			(0x200000)
+#define SZ_1G			(0x40000000)
 
 typedef uint8_t		u8;
 typedef int8_t		s8;
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 09/11] arm/arm64: add initial gicv3 support
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini

Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v5: use modern register names [Andre]
v4:
 - only take defines from kernel we need now [Andre]
 - simplify enable by not caring if we reinit the distributor [drew]
v2:
 - configure irqs as NS GRP1
---
 lib/arm/asm/arch_gicv3.h   | 42 +++++++++++++++++++++
 lib/arm/asm/gic-v3.h       | 94 ++++++++++++++++++++++++++++++++++++++++++++++
 lib/arm/asm/gic.h          |  6 ++-
 lib/arm/gic.c              | 65 ++++++++++++++++++++++++++++++++
 lib/arm64/asm/arch_gicv3.h | 44 ++++++++++++++++++++++
 lib/arm64/asm/gic-v3.h     |  1 +
 lib/arm64/asm/sysreg.h     | 44 ++++++++++++++++++++++
 7 files changed, 294 insertions(+), 2 deletions(-)
 create mode 100644 lib/arm/asm/arch_gicv3.h
 create mode 100644 lib/arm/asm/gic-v3.h
 create mode 100644 lib/arm64/asm/arch_gicv3.h
 create mode 100644 lib/arm64/asm/gic-v3.h
 create mode 100644 lib/arm64/asm/sysreg.h

diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
new file mode 100644
index 000000000000..81a1e5f6c29c
--- /dev/null
+++ b/lib/arm/asm/arch_gicv3.h
@@ -0,0 +1,42 @@
+/*
+ * All ripped off from arch/arm/include/asm/arch_gicv3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_ARCH_GICV3_H_
+#define _ASMARM_ARCH_GICV3_H_
+
+#ifndef __ASSEMBLY__
+#include <libcflat.h>
+#include <asm/barrier.h>
+#include <asm/io.h>
+
+#define __stringify xstr
+
+#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
+
+#define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
+#define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
+
+static inline void gicv3_write_pmr(u32 val)
+{
+	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
+}
+
+static inline void gicv3_write_grpen1(u32 val)
+{
+	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
+	isb();
+}
+
+static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
+{
+	u64 val = readl(addr);
+	val |= (u64)readl(addr + 4) << 32;
+	return val;
+}
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_ARCH_GICV3_H_ */
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
new file mode 100644
index 000000000000..e0f303d82508
--- /dev/null
+++ b/lib/arm/asm/gic-v3.h
@@ -0,0 +1,94 @@
+/*
+ * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V3_H_
+#define _ASMARM_GIC_V3_H_
+
+#ifndef _ASMARM_GIC_H_
+#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
+#endif
+
+#define GICD_CTLR_RWP			(1U << 31)
+#define GICD_CTLR_ARE_NS		(1U << 4)
+#define GICD_CTLR_ENABLE_G1A		(1U << 1)
+#define GICD_CTLR_ENABLE_G1		(1U << 0)
+
+/* Re-Distributor registers, offsets from RD_base */
+#define GICR_TYPER			0x0008
+
+#define GICR_TYPER_LAST			(1U << 4)
+
+/* Re-Distributor registers, offsets from SGI_base */
+#define GICR_IGROUPR0			GICD_IGROUPR
+#define GICR_ISENABLER0			GICD_ISENABLER
+#define GICR_IPRIORITYR0		GICD_IPRIORITYR
+
+#include <asm/arch_gicv3.h>
+
+#ifndef __ASSEMBLY__
+#include <asm/setup.h>
+#include <asm/smp.h>
+#include <asm/processor.h>
+#include <asm/io.h>
+
+struct gicv3_data {
+	void *dist_base;
+	void *redist_base[NR_CPUS];
+	unsigned int irq_nr;
+};
+extern struct gicv3_data gicv3_data;
+
+#define gicv3_dist_base()		(gicv3_data.dist_base)
+#define gicv3_redist_base()		(gicv3_data.redist_base[smp_processor_id()])
+#define gicv3_sgi_base()		(gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
+
+extern int gicv3_init(void);
+extern void gicv3_enable_defaults(void);
+extern void gicv3_set_redist_base(void);
+
+static inline void gicv3_do_wait_for_rwp(void *base)
+{
+	int count = 100000;	/* 1s */
+
+	while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
+		if (!--count) {
+			printf("GICv3: RWP timeout!\n");
+			abort();
+		}
+		cpu_relax();
+		udelay(10);
+	};
+}
+
+static inline void gicv3_dist_wait_for_rwp(void)
+{
+	gicv3_do_wait_for_rwp(gicv3_dist_base());
+}
+
+static inline void gicv3_redist_wait_for_rwp(void)
+{
+	gicv3_do_wait_for_rwp(gicv3_redist_base());
+}
+
+static inline u32 mpidr_compress(u64 mpidr)
+{
+	u64 compressed = mpidr & MPIDR_HWID_BITMASK;
+
+	compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
+	return compressed;
+}
+
+static inline u64 mpidr_uncompress(u32 compressed)
+{
+	u64 mpidr = ((u64)compressed >> 24) << 32;
+
+	mpidr |= compressed & MPIDR_HWID_BITMASK;
+	return mpidr;
+}
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V3_H_ */
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index a16645708c35..981518620d18 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -6,10 +6,9 @@
 #ifndef _ASMARM_GIC_H_
 #define _ASMARM_GIC_H_
 
-#include <asm/gic-v2.h>
-
 #define GICD_CTLR			0x0000
 #define GICD_TYPER			0x0004
+#define GICD_IGROUPR			0x0080
 #define GICD_ISENABLER			0x0100
 #define GICD_IPRIORITYR			0x0400
 #define GICD_SGIR			0x0f00
@@ -26,6 +25,9 @@
 #define GICC_INT_PRI_THRESHOLD		0xf0
 #define GICC_INT_SPURIOUS		0x3ff
 
+#include <asm/gic-v2.h>
+#include <asm/gic-v3.h>
+
 #ifndef __ASSEMBLY__
 
 /*
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index d655105e058b..d929d3f0fa05 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -8,9 +8,11 @@
 #include <asm/io.h>
 
 struct gicv2_data gicv2_data;
+struct gicv3_data gicv3_data;
 
 /*
  * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
+ * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
  */
 static bool
 gic_get_dt_bases(const char *compatible, void **base1, void **base2)
@@ -48,10 +50,18 @@ int gicv2_init(void)
 			&gicv2_data.dist_base, &gicv2_data.cpu_base);
 }
 
+int gicv3_init(void)
+{
+	return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
+			&gicv3_data.redist_base[0]);
+}
+
 int gic_init(void)
 {
 	if (gicv2_init())
 		return 2;
+	else if (gicv3_init())
+		return 3;
 	return 0;
 }
 
@@ -74,3 +84,58 @@ void gicv2_enable_defaults(void)
 	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
 	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
 }
+
+void gicv3_set_redist_base(void)
+{
+	u32 aff = mpidr_compress(get_mpidr());
+	void *ptr = gicv3_data.redist_base[0];
+	u64 typer;
+
+	do {
+		typer = gicv3_read_typer(ptr + GICR_TYPER);
+		if ((typer >> 32) == aff) {
+			gicv3_redist_base() = ptr;
+			return;
+		}
+		ptr += SZ_64K * 2; /* skip RD_base and SGI_base */
+	} while (!(typer & GICR_TYPER_LAST));
+	assert(0);
+}
+
+void gicv3_enable_defaults(void)
+{
+	void *dist = gicv3_dist_base();
+	void *sgi_base;
+	unsigned int i;
+
+	gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
+	if (gicv3_data.irq_nr > 1020)
+		gicv3_data.irq_nr = 1020;
+
+	writel(0, dist + GICD_CTLR);
+	gicv3_dist_wait_for_rwp();
+
+	writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
+	       dist + GICD_CTLR);
+	gicv3_dist_wait_for_rwp();
+
+	for (i = 0; i < gicv3_data.irq_nr; i += 4)
+		writel(~0, dist + GICD_IGROUPR + i);
+	gicv3_dist_wait_for_rwp();
+
+	if (!gicv3_redist_base())
+		gicv3_set_redist_base();
+	sgi_base = gicv3_sgi_base();
+
+	writel(~0, sgi_base + GICR_IGROUPR0);
+
+	for (i = 0; i < 16; i += 4)
+		writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
+
+	writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
+
+	gicv3_redist_wait_for_rwp();
+
+	gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
+	gicv3_write_grpen1(1);
+}
diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
new file mode 100644
index 000000000000..6d353567f56a
--- /dev/null
+++ b/lib/arm64/asm/arch_gicv3.h
@@ -0,0 +1,44 @@
+/*
+ * All ripped off from arch/arm64/include/asm/arch_gicv3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM64_ARCH_GICV3_H_
+#define _ASMARM64_ARCH_GICV3_H_
+
+#include <asm/sysreg.h>
+
+#define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
+#define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
+
+#ifndef __ASSEMBLY__
+
+#include <libcflat.h>
+#include <asm/barrier.h>
+
+#define __stringify xstr
+
+/*
+ * Low-level accessors
+ *
+ * These system registers are 32 bits, but we make sure that the compiler
+ * sets the GP register's most significant bits to 0 with an explicit cast.
+ */
+
+static inline void gicv3_write_pmr(u32 val)
+{
+	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
+}
+
+static inline void gicv3_write_grpen1(u32 val)
+{
+	asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
+	isb();
+}
+
+#define gicv3_read_typer(c)		readq(c)
+
+#endif /* __ASSEMBLY__ */
+#endif /* _ASMARM64_ARCH_GICV3_H_ */
diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
new file mode 100644
index 000000000000..8ee5d4d9c181
--- /dev/null
+++ b/lib/arm64/asm/gic-v3.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v3.h"
diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
new file mode 100644
index 000000000000..544a46cb8cc5
--- /dev/null
+++ b/lib/arm64/asm/sysreg.h
@@ -0,0 +1,44 @@
+/*
+ * Ripped off from arch/arm64/include/asm/sysreg.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM64_SYSREG_H_
+#define _ASMARM64_SYSREG_H_
+
+#define sys_reg(op0, op1, crn, crm, op2) \
+	((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
+
+#ifdef __ASSEMBLY__
+	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
+	.equ	.L__reg_num_x\num, \num
+	.endr
+	.equ	.L__reg_num_xzr, 31
+
+	.macro	mrs_s, rt, sreg
+	.inst	0xd5200000|(\sreg)|(.L__reg_num_\rt)
+	.endm
+
+	.macro	msr_s, sreg, rt
+	.inst	0xd5000000|(\sreg)|(.L__reg_num_\rt)
+	.endm
+#else
+asm(
+"	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
+"	.equ	.L__reg_num_x\\num, \\num\n"
+"	.endr\n"
+"	.equ	.L__reg_num_xzr, 31\n"
+"\n"
+"	.macro	mrs_s, rt, sreg\n"
+"	.inst	0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
+"	.endm\n"
+"\n"
+"	.macro	msr_s, sreg, rt\n"
+"	.inst	0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
+"	.endm\n"
+);
+#endif
+
+#endif /* _ASMARM64_SYSREG_H_ */
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 09/11] arm/arm64: add initial gicv3 support
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v5: use modern register names [Andre]
v4:
 - only take defines from kernel we need now [Andre]
 - simplify enable by not caring if we reinit the distributor [drew]
v2:
 - configure irqs as NS GRP1
---
 lib/arm/asm/arch_gicv3.h   | 42 +++++++++++++++++++++
 lib/arm/asm/gic-v3.h       | 94 ++++++++++++++++++++++++++++++++++++++++++++++
 lib/arm/asm/gic.h          |  6 ++-
 lib/arm/gic.c              | 65 ++++++++++++++++++++++++++++++++
 lib/arm64/asm/arch_gicv3.h | 44 ++++++++++++++++++++++
 lib/arm64/asm/gic-v3.h     |  1 +
 lib/arm64/asm/sysreg.h     | 44 ++++++++++++++++++++++
 7 files changed, 294 insertions(+), 2 deletions(-)
 create mode 100644 lib/arm/asm/arch_gicv3.h
 create mode 100644 lib/arm/asm/gic-v3.h
 create mode 100644 lib/arm64/asm/arch_gicv3.h
 create mode 100644 lib/arm64/asm/gic-v3.h
 create mode 100644 lib/arm64/asm/sysreg.h

diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
new file mode 100644
index 000000000000..81a1e5f6c29c
--- /dev/null
+++ b/lib/arm/asm/arch_gicv3.h
@@ -0,0 +1,42 @@
+/*
+ * All ripped off from arch/arm/include/asm/arch_gicv3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_ARCH_GICV3_H_
+#define _ASMARM_ARCH_GICV3_H_
+
+#ifndef __ASSEMBLY__
+#include <libcflat.h>
+#include <asm/barrier.h>
+#include <asm/io.h>
+
+#define __stringify xstr
+
+#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
+
+#define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
+#define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
+
+static inline void gicv3_write_pmr(u32 val)
+{
+	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
+}
+
+static inline void gicv3_write_grpen1(u32 val)
+{
+	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
+	isb();
+}
+
+static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
+{
+	u64 val = readl(addr);
+	val |= (u64)readl(addr + 4) << 32;
+	return val;
+}
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_ARCH_GICV3_H_ */
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
new file mode 100644
index 000000000000..e0f303d82508
--- /dev/null
+++ b/lib/arm/asm/gic-v3.h
@@ -0,0 +1,94 @@
+/*
+ * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V3_H_
+#define _ASMARM_GIC_V3_H_
+
+#ifndef _ASMARM_GIC_H_
+#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
+#endif
+
+#define GICD_CTLR_RWP			(1U << 31)
+#define GICD_CTLR_ARE_NS		(1U << 4)
+#define GICD_CTLR_ENABLE_G1A		(1U << 1)
+#define GICD_CTLR_ENABLE_G1		(1U << 0)
+
+/* Re-Distributor registers, offsets from RD_base */
+#define GICR_TYPER			0x0008
+
+#define GICR_TYPER_LAST			(1U << 4)
+
+/* Re-Distributor registers, offsets from SGI_base */
+#define GICR_IGROUPR0			GICD_IGROUPR
+#define GICR_ISENABLER0			GICD_ISENABLER
+#define GICR_IPRIORITYR0		GICD_IPRIORITYR
+
+#include <asm/arch_gicv3.h>
+
+#ifndef __ASSEMBLY__
+#include <asm/setup.h>
+#include <asm/smp.h>
+#include <asm/processor.h>
+#include <asm/io.h>
+
+struct gicv3_data {
+	void *dist_base;
+	void *redist_base[NR_CPUS];
+	unsigned int irq_nr;
+};
+extern struct gicv3_data gicv3_data;
+
+#define gicv3_dist_base()		(gicv3_data.dist_base)
+#define gicv3_redist_base()		(gicv3_data.redist_base[smp_processor_id()])
+#define gicv3_sgi_base()		(gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
+
+extern int gicv3_init(void);
+extern void gicv3_enable_defaults(void);
+extern void gicv3_set_redist_base(void);
+
+static inline void gicv3_do_wait_for_rwp(void *base)
+{
+	int count = 100000;	/* 1s */
+
+	while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
+		if (!--count) {
+			printf("GICv3: RWP timeout!\n");
+			abort();
+		}
+		cpu_relax();
+		udelay(10);
+	};
+}
+
+static inline void gicv3_dist_wait_for_rwp(void)
+{
+	gicv3_do_wait_for_rwp(gicv3_dist_base());
+}
+
+static inline void gicv3_redist_wait_for_rwp(void)
+{
+	gicv3_do_wait_for_rwp(gicv3_redist_base());
+}
+
+static inline u32 mpidr_compress(u64 mpidr)
+{
+	u64 compressed = mpidr & MPIDR_HWID_BITMASK;
+
+	compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
+	return compressed;
+}
+
+static inline u64 mpidr_uncompress(u32 compressed)
+{
+	u64 mpidr = ((u64)compressed >> 24) << 32;
+
+	mpidr |= compressed & MPIDR_HWID_BITMASK;
+	return mpidr;
+}
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V3_H_ */
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index a16645708c35..981518620d18 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -6,10 +6,9 @@
 #ifndef _ASMARM_GIC_H_
 #define _ASMARM_GIC_H_
 
-#include <asm/gic-v2.h>
-
 #define GICD_CTLR			0x0000
 #define GICD_TYPER			0x0004
+#define GICD_IGROUPR			0x0080
 #define GICD_ISENABLER			0x0100
 #define GICD_IPRIORITYR			0x0400
 #define GICD_SGIR			0x0f00
@@ -26,6 +25,9 @@
 #define GICC_INT_PRI_THRESHOLD		0xf0
 #define GICC_INT_SPURIOUS		0x3ff
 
+#include <asm/gic-v2.h>
+#include <asm/gic-v3.h>
+
 #ifndef __ASSEMBLY__
 
 /*
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index d655105e058b..d929d3f0fa05 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -8,9 +8,11 @@
 #include <asm/io.h>
 
 struct gicv2_data gicv2_data;
+struct gicv3_data gicv3_data;
 
 /*
  * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
+ * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
  */
 static bool
 gic_get_dt_bases(const char *compatible, void **base1, void **base2)
@@ -48,10 +50,18 @@ int gicv2_init(void)
 			&gicv2_data.dist_base, &gicv2_data.cpu_base);
 }
 
+int gicv3_init(void)
+{
+	return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
+			&gicv3_data.redist_base[0]);
+}
+
 int gic_init(void)
 {
 	if (gicv2_init())
 		return 2;
+	else if (gicv3_init())
+		return 3;
 	return 0;
 }
 
@@ -74,3 +84,58 @@ void gicv2_enable_defaults(void)
 	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
 	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
 }
+
+void gicv3_set_redist_base(void)
+{
+	u32 aff = mpidr_compress(get_mpidr());
+	void *ptr = gicv3_data.redist_base[0];
+	u64 typer;
+
+	do {
+		typer = gicv3_read_typer(ptr + GICR_TYPER);
+		if ((typer >> 32) == aff) {
+			gicv3_redist_base() = ptr;
+			return;
+		}
+		ptr += SZ_64K * 2; /* skip RD_base and SGI_base */
+	} while (!(typer & GICR_TYPER_LAST));
+	assert(0);
+}
+
+void gicv3_enable_defaults(void)
+{
+	void *dist = gicv3_dist_base();
+	void *sgi_base;
+	unsigned int i;
+
+	gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
+	if (gicv3_data.irq_nr > 1020)
+		gicv3_data.irq_nr = 1020;
+
+	writel(0, dist + GICD_CTLR);
+	gicv3_dist_wait_for_rwp();
+
+	writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
+	       dist + GICD_CTLR);
+	gicv3_dist_wait_for_rwp();
+
+	for (i = 0; i < gicv3_data.irq_nr; i += 4)
+		writel(~0, dist + GICD_IGROUPR + i);
+	gicv3_dist_wait_for_rwp();
+
+	if (!gicv3_redist_base())
+		gicv3_set_redist_base();
+	sgi_base = gicv3_sgi_base();
+
+	writel(~0, sgi_base + GICR_IGROUPR0);
+
+	for (i = 0; i < 16; i += 4)
+		writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
+
+	writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
+
+	gicv3_redist_wait_for_rwp();
+
+	gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
+	gicv3_write_grpen1(1);
+}
diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
new file mode 100644
index 000000000000..6d353567f56a
--- /dev/null
+++ b/lib/arm64/asm/arch_gicv3.h
@@ -0,0 +1,44 @@
+/*
+ * All ripped off from arch/arm64/include/asm/arch_gicv3.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM64_ARCH_GICV3_H_
+#define _ASMARM64_ARCH_GICV3_H_
+
+#include <asm/sysreg.h>
+
+#define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
+#define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
+
+#ifndef __ASSEMBLY__
+
+#include <libcflat.h>
+#include <asm/barrier.h>
+
+#define __stringify xstr
+
+/*
+ * Low-level accessors
+ *
+ * These system registers are 32 bits, but we make sure that the compiler
+ * sets the GP register's most significant bits to 0 with an explicit cast.
+ */
+
+static inline void gicv3_write_pmr(u32 val)
+{
+	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
+}
+
+static inline void gicv3_write_grpen1(u32 val)
+{
+	asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
+	isb();
+}
+
+#define gicv3_read_typer(c)		readq(c)
+
+#endif /* __ASSEMBLY__ */
+#endif /* _ASMARM64_ARCH_GICV3_H_ */
diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
new file mode 100644
index 000000000000..8ee5d4d9c181
--- /dev/null
+++ b/lib/arm64/asm/gic-v3.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v3.h"
diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
new file mode 100644
index 000000000000..544a46cb8cc5
--- /dev/null
+++ b/lib/arm64/asm/sysreg.h
@@ -0,0 +1,44 @@
+/*
+ * Ripped off from arch/arm64/include/asm/sysreg.h
+ *
+ * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM64_SYSREG_H_
+#define _ASMARM64_SYSREG_H_
+
+#define sys_reg(op0, op1, crn, crm, op2) \
+	((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
+
+#ifdef __ASSEMBLY__
+	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
+	.equ	.L__reg_num_x\num, \num
+	.endr
+	.equ	.L__reg_num_xzr, 31
+
+	.macro	mrs_s, rt, sreg
+	.inst	0xd5200000|(\sreg)|(.L__reg_num_\rt)
+	.endm
+
+	.macro	msr_s, sreg, rt
+	.inst	0xd5000000|(\sreg)|(.L__reg_num_\rt)
+	.endm
+#else
+asm(
+"	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
+"	.equ	.L__reg_num_x\\num, \\num\n"
+"	.endr\n"
+"	.equ	.L__reg_num_xzr, 31\n"
+"\n"
+"	.macro	mrs_s, rt, sreg\n"
+"	.inst	0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
+"	.endm\n"
+"\n"
+"	.macro	msr_s, sreg, rt\n"
+"	.inst	0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
+"	.endm\n"
+);
+#endif
+
+#endif /* _ASMARM64_SYSREG_H_ */
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, andre.przywara, pbonzini

Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v5:
 - fix copy+paste error in gicv3_write_eoir [drew]
 - use modern register names [Andre]
v4:
 - heavily comment gicv3_ipi_send_tlist() [Eric]
 - changes needed for gicv2 iar/irqstat fix to other patch
v2:
 - use IRM for gicv3 broadcast
---
 arm/gic.c                  | 195 ++++++++++++++++++++++++++++++++++++++++++---
 arm/unittests.cfg          |   6 ++
 lib/arm/asm/arch_gicv3.h   |  23 ++++++
 lib/arm64/asm/arch_gicv3.h |  22 +++++
 4 files changed, 236 insertions(+), 10 deletions(-)

diff --git a/arm/gic.c b/arm/gic.c
index 06092aec7c08..ca68f8ad1cb9 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -3,6 +3,8 @@
  *
  * GICv2
  *   + test sending/receiving IPIs
+ * GICv3
+ *   + test sending/receiving IPIs
  *
  * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
  *
@@ -16,6 +18,19 @@
 #include <asm/barrier.h>
 #include <asm/io.h>
 
+struct gic {
+	struct {
+		void (*enable)(void);
+		void (*send_self)(void);
+		void (*send_tlist)(cpumask_t *);
+		void (*send_broadcast)(void);
+	} ipi;
+	u32 (*read_iar)(void);
+	u32 (*irqnr)(u32 iar);
+	void (*write_eoi)(u32);
+};
+
+static struct gic *gic;
 static int gic_version;
 static int acked[NR_CPUS], spurious[NR_CPUS];
 static cpumask_t ready;
@@ -69,13 +84,33 @@ static void check_acked(cpumask_t *mask)
 	       false, missing, extra, unexpected);
 }
 
+static u32 gicv2_read_iar(void)
+{
+	return readl(gicv2_cpu_base() + GICC_IAR);
+}
+
+static u32 gicv2_irqnr(u32 iar)
+{
+	return iar & GICC_IAR_INT_ID_MASK;
+}
+
+static void gicv2_write_eoi(u32 irqstat)
+{
+	writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+}
+
+static u32 gicv3_irqnr(u32 iar)
+{
+	return iar;
+}
+
 static void ipi_handler(struct pt_regs *regs __unused)
 {
-	u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
-	u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
+	u32 irqstat = gic->read_iar();
+	u32 irqnr = gic->irqnr(irqstat);
 
 	if (irqnr != GICC_INT_SPURIOUS) {
-		writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+		gic->write_eoi(irqstat);
 		smp_rmb(); /* pairs with wmb in ipi_test functions */
 		++acked[smp_processor_id()];
 		smp_wmb(); /* pairs with rmb in check_acked */
@@ -85,6 +120,112 @@ static void ipi_handler(struct pt_regs *regs __unused)
 	}
 }
 
+static void gicv2_ipi_send_self(void)
+{
+	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+}
+
+static void gicv2_ipi_send_tlist(cpumask_t *mask)
+{
+	u8 tlist = (u8)cpumask_bits(mask)[0];
+
+	writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
+}
+
+static void gicv2_ipi_send_broadcast(void)
+{
+	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+}
+
+#define ICC_SGI1R_AFFINITY_1_SHIFT	16
+#define ICC_SGI1R_AFFINITY_2_SHIFT	32
+#define ICC_SGI1R_AFFINITY_3_SHIFT	48
+#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
+	(MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
+
+static void gicv3_ipi_send_tlist(cpumask_t *mask)
+{
+	u16 tlist;
+	int cpu;
+
+	/*
+	 * For each cpu in the mask collect its peers, which are also in
+	 * the mask, in order to form target lists.
+	 */
+	for_each_cpu(cpu, mask) {
+		u64 mpidr = cpus[cpu], sgi1r;
+		u64 cluster_id;
+
+		/*
+		 * GICv3 can send IPIs to up 16 peer cpus with a single
+		 * write to ICC_SGI1R_EL1 (using the target list). Peers
+		 * are cpus that have nearly identical MPIDRs, the only
+		 * difference being Aff0. The matching upper affinity
+		 * levels form the cluster ID.
+		 */
+		cluster_id = mpidr & ~0xffUL;
+		tlist = 0;
+
+		/*
+		 * Sort of open code for_each_cpu in order to have a
+		 * nested for_each_cpu loop.
+		 */
+		while (cpu < nr_cpus) {
+			if ((mpidr & 0xff) >= 16) {
+				printf("cpu%d MPIDR:aff0 is %d (>= 16)!\n",
+					cpu, (int)(mpidr & 0xff));
+				break;
+			}
+
+			tlist |= 1 << (mpidr & 0xf);
+
+			cpu = cpumask_next(cpu, mask);
+			if (cpu >= nr_cpus)
+				break;
+
+			mpidr = cpus[cpu];
+
+			if (cluster_id != (mpidr & ~0xffUL)) {
+				/*
+				 * The next cpu isn't in our cluster. Roll
+				 * back the cpu index allowing the outer
+				 * for_each_cpu to find it again with
+				 * cpumask_next
+				 */
+				--cpu;
+				break;
+			}
+		}
+
+		/* Send the IPIs for the target list of this cluster */
+		sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3)	|
+			 MPIDR_TO_SGI_AFFINITY(cluster_id, 2)	|
+			 /* irq << 24				| */
+			 MPIDR_TO_SGI_AFFINITY(cluster_id, 1)	|
+			 tlist);
+
+		gicv3_write_sgi1r(sgi1r);
+	}
+
+	/* Force the above writes to ICC_SGI1R_EL1 to be executed */
+	isb();
+}
+
+static void gicv3_ipi_send_self(void)
+{
+	cpumask_t mask;
+
+	cpumask_clear(&mask);
+	cpumask_set_cpu(smp_processor_id(), &mask);
+	gicv3_ipi_send_tlist(&mask);
+}
+
+static void gicv3_ipi_send_broadcast(void)
+{
+	gicv3_write_sgi1r(1ULL << 40);
+	isb();
+}
+
 static void ipi_test_self(void)
 {
 	cpumask_t mask;
@@ -94,7 +235,7 @@ static void ipi_test_self(void)
 	smp_wmb();
 	cpumask_clear(&mask);
 	cpumask_set_cpu(0, &mask);
-	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+	gic->ipi.send_self();
 	check_acked(&mask);
 	report_prefix_pop();
 }
@@ -102,14 +243,15 @@ static void ipi_test_self(void)
 static void ipi_test_smp(void)
 {
 	cpumask_t mask;
-	unsigned long tlist;
+	int i;
 
 	report_prefix_push("target-list");
 	memset(acked, 0, sizeof(acked));
 	smp_wmb();
-	tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
-	cpumask_bits(&mask)[0] = tlist;
-	writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
+	cpumask_copy(&mask, &cpu_present_mask);
+	for (i = 0; i < nr_cpus; i += 2)
+		cpumask_clear_cpu(i, &mask);
+	gic->ipi.send_tlist(&mask);
 	check_acked(&mask);
 	report_prefix_pop();
 
@@ -118,14 +260,14 @@ static void ipi_test_smp(void)
 	smp_wmb();
 	cpumask_copy(&mask, &cpu_present_mask);
 	cpumask_clear_cpu(0, &mask);
-	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+	gic->ipi.send_broadcast();
 	check_acked(&mask);
 	report_prefix_pop();
 }
 
 static void ipi_enable(void)
 {
-	gicv2_enable_defaults();
+	gic->ipi.enable();
 #ifdef __arm__
 	install_exception_handler(EXCPTN_IRQ, ipi_handler);
 #else
@@ -142,6 +284,30 @@ static void ipi_recv(void)
 		wfi();
 }
 
+struct gic gicv2 = {
+	.ipi = {
+		.enable = gicv2_enable_defaults,
+		.send_self = gicv2_ipi_send_self,
+		.send_tlist = gicv2_ipi_send_tlist,
+		.send_broadcast = gicv2_ipi_send_broadcast,
+	},
+	.read_iar = gicv2_read_iar,
+	.irqnr = gicv2_irqnr,
+	.write_eoi = gicv2_write_eoi,
+};
+
+struct gic gicv3 = {
+	.ipi = {
+		.enable = gicv3_enable_defaults,
+		.send_self = gicv3_ipi_send_self,
+		.send_tlist = gicv3_ipi_send_tlist,
+		.send_broadcast = gicv3_ipi_send_broadcast,
+	},
+	.read_iar = gicv3_read_iar,
+	.irqnr = gicv3_irqnr,
+	.write_eoi = gicv3_write_eoir,
+};
+
 int main(int argc, char **argv)
 {
 	char pfx[8];
@@ -154,6 +320,15 @@ int main(int argc, char **argv)
 	snprintf(pfx, 8, "gicv%d", gic_version);
 	report_prefix_push(pfx);
 
+	switch (gic_version) {
+	case 2:
+		gic = &gicv2;
+		break;
+	case 3:
+		gic = &gicv3;
+		break;
+	}
+
 	if (argc < 2) {
 
 		report_prefix_push("ipi");
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 68bf5cd6008f..ce3cee73c4bf 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -61,3 +61,9 @@ file = gic.flat
 smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
 extra_params = -machine gic-version=2 -append 'ipi'
 groups = gic
+
+[gicv3-ipi]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'ipi'
+groups = gic
diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
index 81a1e5f6c29c..615a3393e439 100644
--- a/lib/arm/asm/arch_gicv3.h
+++ b/lib/arm/asm/arch_gicv3.h
@@ -16,10 +16,28 @@
 #define __stringify xstr
 
 #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
+#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
 
+#define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
+#define ICC_IAR1			__ACCESS_CP15(c12, 0, c12, 0)
+#define ICC_SGI1R			__ACCESS_CP15_64(0, c12)
 #define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
 #define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
 
+static inline void gicv3_write_eoir(u32 irq)
+{
+	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
+	isb();
+}
+
+static inline u32 gicv3_read_iar(void)
+{
+	u32 irqstat;
+	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
+	dsb(sy);
+	return irqstat;
+}
+
 static inline void gicv3_write_pmr(u32 val)
 {
 	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
@@ -31,6 +49,11 @@ static inline void gicv3_write_grpen1(u32 val)
 	isb();
 }
 
+static inline void gicv3_write_sgi1r(u64 val)
+{
+	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
+}
+
 static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
 {
 	u64 val = readl(addr);
diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
index 6d353567f56a..874775828016 100644
--- a/lib/arm64/asm/arch_gicv3.h
+++ b/lib/arm64/asm/arch_gicv3.h
@@ -10,6 +10,9 @@
 
 #include <asm/sysreg.h>
 
+#define ICC_EOIR1_EL1			sys_reg(3, 0, 12, 12, 1)
+#define ICC_IAR1_EL1			sys_reg(3, 0, 12, 12, 0)
+#define ICC_SGI1R_EL1			sys_reg(3, 0, 12, 11, 5)
 #define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
 #define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
 
@@ -27,6 +30,20 @@
  * sets the GP register's most significant bits to 0 with an explicit cast.
  */
 
+static inline void gicv3_write_eoir(u32 irq)
+{
+	asm volatile("msr_s " __stringify(ICC_EOIR1_EL1) ", %0" : : "r" ((u64)irq));
+	isb();
+}
+
+static inline u32 gicv3_read_iar(void)
+{
+	u64 irqstat;
+	asm volatile("mrs_s %0, " __stringify(ICC_IAR1_EL1) : "=r" (irqstat));
+	dsb(sy);
+	return (u64)irqstat;
+}
+
 static inline void gicv3_write_pmr(u32 val)
 {
 	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
@@ -38,6 +55,11 @@ static inline void gicv3_write_grpen1(u32 val)
 	isb();
 }
 
+static inline void gicv3_write_sgi1r(u64 val)
+{
+	asm volatile("msr_s " __stringify(ICC_SGI1R_EL1) ", %0" : : "r" (val));
+}
+
 #define gicv3_read_typer(c)		readq(c)
 
 #endif /* __ASSEMBLY__ */
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v5:
 - fix copy+paste error in gicv3_write_eoir [drew]
 - use modern register names [Andre]
v4:
 - heavily comment gicv3_ipi_send_tlist() [Eric]
 - changes needed for gicv2 iar/irqstat fix to other patch
v2:
 - use IRM for gicv3 broadcast
---
 arm/gic.c                  | 195 ++++++++++++++++++++++++++++++++++++++++++---
 arm/unittests.cfg          |   6 ++
 lib/arm/asm/arch_gicv3.h   |  23 ++++++
 lib/arm64/asm/arch_gicv3.h |  22 +++++
 4 files changed, 236 insertions(+), 10 deletions(-)

diff --git a/arm/gic.c b/arm/gic.c
index 06092aec7c08..ca68f8ad1cb9 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -3,6 +3,8 @@
  *
  * GICv2
  *   + test sending/receiving IPIs
+ * GICv3
+ *   + test sending/receiving IPIs
  *
  * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
  *
@@ -16,6 +18,19 @@
 #include <asm/barrier.h>
 #include <asm/io.h>
 
+struct gic {
+	struct {
+		void (*enable)(void);
+		void (*send_self)(void);
+		void (*send_tlist)(cpumask_t *);
+		void (*send_broadcast)(void);
+	} ipi;
+	u32 (*read_iar)(void);
+	u32 (*irqnr)(u32 iar);
+	void (*write_eoi)(u32);
+};
+
+static struct gic *gic;
 static int gic_version;
 static int acked[NR_CPUS], spurious[NR_CPUS];
 static cpumask_t ready;
@@ -69,13 +84,33 @@ static void check_acked(cpumask_t *mask)
 	       false, missing, extra, unexpected);
 }
 
+static u32 gicv2_read_iar(void)
+{
+	return readl(gicv2_cpu_base() + GICC_IAR);
+}
+
+static u32 gicv2_irqnr(u32 iar)
+{
+	return iar & GICC_IAR_INT_ID_MASK;
+}
+
+static void gicv2_write_eoi(u32 irqstat)
+{
+	writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+}
+
+static u32 gicv3_irqnr(u32 iar)
+{
+	return iar;
+}
+
 static void ipi_handler(struct pt_regs *regs __unused)
 {
-	u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
-	u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
+	u32 irqstat = gic->read_iar();
+	u32 irqnr = gic->irqnr(irqstat);
 
 	if (irqnr != GICC_INT_SPURIOUS) {
-		writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
+		gic->write_eoi(irqstat);
 		smp_rmb(); /* pairs with wmb in ipi_test functions */
 		++acked[smp_processor_id()];
 		smp_wmb(); /* pairs with rmb in check_acked */
@@ -85,6 +120,112 @@ static void ipi_handler(struct pt_regs *regs __unused)
 	}
 }
 
+static void gicv2_ipi_send_self(void)
+{
+	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+}
+
+static void gicv2_ipi_send_tlist(cpumask_t *mask)
+{
+	u8 tlist = (u8)cpumask_bits(mask)[0];
+
+	writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
+}
+
+static void gicv2_ipi_send_broadcast(void)
+{
+	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+}
+
+#define ICC_SGI1R_AFFINITY_1_SHIFT	16
+#define ICC_SGI1R_AFFINITY_2_SHIFT	32
+#define ICC_SGI1R_AFFINITY_3_SHIFT	48
+#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
+	(MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
+
+static void gicv3_ipi_send_tlist(cpumask_t *mask)
+{
+	u16 tlist;
+	int cpu;
+
+	/*
+	 * For each cpu in the mask collect its peers, which are also in
+	 * the mask, in order to form target lists.
+	 */
+	for_each_cpu(cpu, mask) {
+		u64 mpidr = cpus[cpu], sgi1r;
+		u64 cluster_id;
+
+		/*
+		 * GICv3 can send IPIs to up 16 peer cpus with a single
+		 * write to ICC_SGI1R_EL1 (using the target list). Peers
+		 * are cpus that have nearly identical MPIDRs, the only
+		 * difference being Aff0. The matching upper affinity
+		 * levels form the cluster ID.
+		 */
+		cluster_id = mpidr & ~0xffUL;
+		tlist = 0;
+
+		/*
+		 * Sort of open code for_each_cpu in order to have a
+		 * nested for_each_cpu loop.
+		 */
+		while (cpu < nr_cpus) {
+			if ((mpidr & 0xff) >= 16) {
+				printf("cpu%d MPIDR:aff0 is %d (>= 16)!\n",
+					cpu, (int)(mpidr & 0xff));
+				break;
+			}
+
+			tlist |= 1 << (mpidr & 0xf);
+
+			cpu = cpumask_next(cpu, mask);
+			if (cpu >= nr_cpus)
+				break;
+
+			mpidr = cpus[cpu];
+
+			if (cluster_id != (mpidr & ~0xffUL)) {
+				/*
+				 * The next cpu isn't in our cluster. Roll
+				 * back the cpu index allowing the outer
+				 * for_each_cpu to find it again with
+				 * cpumask_next
+				 */
+				--cpu;
+				break;
+			}
+		}
+
+		/* Send the IPIs for the target list of this cluster */
+		sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3)	|
+			 MPIDR_TO_SGI_AFFINITY(cluster_id, 2)	|
+			 /* irq << 24				| */
+			 MPIDR_TO_SGI_AFFINITY(cluster_id, 1)	|
+			 tlist);
+
+		gicv3_write_sgi1r(sgi1r);
+	}
+
+	/* Force the above writes to ICC_SGI1R_EL1 to be executed */
+	isb();
+}
+
+static void gicv3_ipi_send_self(void)
+{
+	cpumask_t mask;
+
+	cpumask_clear(&mask);
+	cpumask_set_cpu(smp_processor_id(), &mask);
+	gicv3_ipi_send_tlist(&mask);
+}
+
+static void gicv3_ipi_send_broadcast(void)
+{
+	gicv3_write_sgi1r(1ULL << 40);
+	isb();
+}
+
 static void ipi_test_self(void)
 {
 	cpumask_t mask;
@@ -94,7 +235,7 @@ static void ipi_test_self(void)
 	smp_wmb();
 	cpumask_clear(&mask);
 	cpumask_set_cpu(0, &mask);
-	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+	gic->ipi.send_self();
 	check_acked(&mask);
 	report_prefix_pop();
 }
@@ -102,14 +243,15 @@ static void ipi_test_self(void)
 static void ipi_test_smp(void)
 {
 	cpumask_t mask;
-	unsigned long tlist;
+	int i;
 
 	report_prefix_push("target-list");
 	memset(acked, 0, sizeof(acked));
 	smp_wmb();
-	tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
-	cpumask_bits(&mask)[0] = tlist;
-	writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
+	cpumask_copy(&mask, &cpu_present_mask);
+	for (i = 0; i < nr_cpus; i += 2)
+		cpumask_clear_cpu(i, &mask);
+	gic->ipi.send_tlist(&mask);
 	check_acked(&mask);
 	report_prefix_pop();
 
@@ -118,14 +260,14 @@ static void ipi_test_smp(void)
 	smp_wmb();
 	cpumask_copy(&mask, &cpu_present_mask);
 	cpumask_clear_cpu(0, &mask);
-	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+	gic->ipi.send_broadcast();
 	check_acked(&mask);
 	report_prefix_pop();
 }
 
 static void ipi_enable(void)
 {
-	gicv2_enable_defaults();
+	gic->ipi.enable();
 #ifdef __arm__
 	install_exception_handler(EXCPTN_IRQ, ipi_handler);
 #else
@@ -142,6 +284,30 @@ static void ipi_recv(void)
 		wfi();
 }
 
+struct gic gicv2 = {
+	.ipi = {
+		.enable = gicv2_enable_defaults,
+		.send_self = gicv2_ipi_send_self,
+		.send_tlist = gicv2_ipi_send_tlist,
+		.send_broadcast = gicv2_ipi_send_broadcast,
+	},
+	.read_iar = gicv2_read_iar,
+	.irqnr = gicv2_irqnr,
+	.write_eoi = gicv2_write_eoi,
+};
+
+struct gic gicv3 = {
+	.ipi = {
+		.enable = gicv3_enable_defaults,
+		.send_self = gicv3_ipi_send_self,
+		.send_tlist = gicv3_ipi_send_tlist,
+		.send_broadcast = gicv3_ipi_send_broadcast,
+	},
+	.read_iar = gicv3_read_iar,
+	.irqnr = gicv3_irqnr,
+	.write_eoi = gicv3_write_eoir,
+};
+
 int main(int argc, char **argv)
 {
 	char pfx[8];
@@ -154,6 +320,15 @@ int main(int argc, char **argv)
 	snprintf(pfx, 8, "gicv%d", gic_version);
 	report_prefix_push(pfx);
 
+	switch (gic_version) {
+	case 2:
+		gic = &gicv2;
+		break;
+	case 3:
+		gic = &gicv3;
+		break;
+	}
+
 	if (argc < 2) {
 
 		report_prefix_push("ipi");
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 68bf5cd6008f..ce3cee73c4bf 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -61,3 +61,9 @@ file = gic.flat
 smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
 extra_params = -machine gic-version=2 -append 'ipi'
 groups = gic
+
+[gicv3-ipi]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'ipi'
+groups = gic
diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
index 81a1e5f6c29c..615a3393e439 100644
--- a/lib/arm/asm/arch_gicv3.h
+++ b/lib/arm/asm/arch_gicv3.h
@@ -16,10 +16,28 @@
 #define __stringify xstr
 
 #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
+#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
 
+#define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
+#define ICC_IAR1			__ACCESS_CP15(c12, 0, c12, 0)
+#define ICC_SGI1R			__ACCESS_CP15_64(0, c12)
 #define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
 #define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
 
+static inline void gicv3_write_eoir(u32 irq)
+{
+	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
+	isb();
+}
+
+static inline u32 gicv3_read_iar(void)
+{
+	u32 irqstat;
+	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
+	dsb(sy);
+	return irqstat;
+}
+
 static inline void gicv3_write_pmr(u32 val)
 {
 	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
@@ -31,6 +49,11 @@ static inline void gicv3_write_grpen1(u32 val)
 	isb();
 }
 
+static inline void gicv3_write_sgi1r(u64 val)
+{
+	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
+}
+
 static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
 {
 	u64 val = readl(addr);
diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
index 6d353567f56a..874775828016 100644
--- a/lib/arm64/asm/arch_gicv3.h
+++ b/lib/arm64/asm/arch_gicv3.h
@@ -10,6 +10,9 @@
 
 #include <asm/sysreg.h>
 
+#define ICC_EOIR1_EL1			sys_reg(3, 0, 12, 12, 1)
+#define ICC_IAR1_EL1			sys_reg(3, 0, 12, 12, 0)
+#define ICC_SGI1R_EL1			sys_reg(3, 0, 12, 11, 5)
 #define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
 #define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
 
@@ -27,6 +30,20 @@
  * sets the GP register's most significant bits to 0 with an explicit cast.
  */
 
+static inline void gicv3_write_eoir(u32 irq)
+{
+	asm volatile("msr_s " __stringify(ICC_EOIR1_EL1) ", %0" : : "r" ((u64)irq));
+	isb();
+}
+
+static inline u32 gicv3_read_iar(void)
+{
+	u64 irqstat;
+	asm volatile("mrs_s %0, " __stringify(ICC_IAR1_EL1) : "=r" (irqstat));
+	dsb(sy);
+	return (u64)irqstat;
+}
+
 static inline void gicv3_write_pmr(u32 val)
 {
 	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
@@ -38,6 +55,11 @@ static inline void gicv3_write_grpen1(u32 val)
 	isb();
 }
 
+static inline void gicv3_write_sgi1r(u64 val)
+{
+	asm volatile("msr_s " __stringify(ICC_SGI1R_EL1) ", %0" : : "r" (val));
+}
+
 #define gicv3_read_typer(c)		readq(c)
 
 #endif /* __ASSEMBLY__ */
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [kvm-unit-tests PATCH v5 11/11] arm/arm64: gic: don't just use zero
  2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 17:21   ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Allow user to select who sends ipis and with which irq,
rather than just always sending irq=0 from cpu0.

Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v4: improve structure and make sure spurious checking is
    done even when the sender isn't cpu0
v2: actually check that the irq received was the irq sent,
    and (for gicv2) that the sender is the expected one.
---
 arm/gic.c | 99 ++++++++++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 73 insertions(+), 26 deletions(-)

diff --git a/arm/gic.c b/arm/gic.c
index ca68f8ad1cb9..af9e745e2e5c 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -11,6 +11,7 @@
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
 #include <libcflat.h>
+#include <util.h>
 #include <asm/setup.h>
 #include <asm/processor.h>
 #include <asm/gic.h>
@@ -34,6 +35,8 @@ static struct gic *gic;
 static int gic_version;
 static int acked[NR_CPUS], spurious[NR_CPUS];
 static cpumask_t ready;
+static int sender;
+static u32 irq;
 
 static void nr_cpu_check(int nr)
 {
@@ -86,7 +89,16 @@ static void check_acked(cpumask_t *mask)
 
 static u32 gicv2_read_iar(void)
 {
-	return readl(gicv2_cpu_base() + GICC_IAR);
+	u32 iar = readl(gicv2_cpu_base() + GICC_IAR);
+	int src = (iar >> 10) & 7;
+
+	if (src != sender) {
+		report("cpu%d received IPI from unexpected source cpu%d "
+		       "(expected cpu%d)",
+		       false, smp_processor_id(), src, sender);
+	}
+
+	return iar;
 }
 
 static u32 gicv2_irqnr(u32 iar)
@@ -111,9 +123,15 @@ static void ipi_handler(struct pt_regs *regs __unused)
 
 	if (irqnr != GICC_INT_SPURIOUS) {
 		gic->write_eoi(irqstat);
-		smp_rmb(); /* pairs with wmb in ipi_test functions */
-		++acked[smp_processor_id()];
-		smp_wmb(); /* pairs with rmb in check_acked */
+		if (irqnr == irq) {
+			smp_rmb(); /* pairs with wmb in ipi_test functions */
+			++acked[smp_processor_id()];
+			smp_wmb(); /* pairs with rmb in check_acked */
+		} else {
+			report("cpu%d received unexpected irq %u "
+			       "(expected %u)",
+			       false, smp_processor_id(), irqnr, irq);
+		}
 	} else {
 		++spurious[smp_processor_id()];
 		smp_wmb();
@@ -122,19 +140,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
 
 static void gicv2_ipi_send_self(void)
 {
-	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+	writel(2 << 24 | irq, gicv2_dist_base() + GICD_SGIR);
 }
 
 static void gicv2_ipi_send_tlist(cpumask_t *mask)
 {
 	u8 tlist = (u8)cpumask_bits(mask)[0];
 
-	writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
+	writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
 }
 
 static void gicv2_ipi_send_broadcast(void)
 {
-	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+	writel(1 << 24 | irq, gicv2_dist_base() + GICD_SGIR);
 }
 
 #define ICC_SGI1R_AFFINITY_1_SHIFT	16
@@ -200,7 +218,7 @@ static void gicv3_ipi_send_tlist(cpumask_t *mask)
 		/* Send the IPIs for the target list of this cluster */
 		sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3)	|
 			 MPIDR_TO_SGI_AFFINITY(cluster_id, 2)	|
-			 /* irq << 24				| */
+			 irq << 24				|
 			 MPIDR_TO_SGI_AFFINITY(cluster_id, 1)	|
 			 tlist);
 
@@ -222,7 +240,7 @@ static void gicv3_ipi_send_self(void)
 
 static void gicv3_ipi_send_broadcast(void)
 {
-	gicv3_write_sgi1r(1ULL << 40);
+	gicv3_write_sgi1r(1ULL << 40 | irq << 24);
 	isb();
 }
 
@@ -234,7 +252,7 @@ static void ipi_test_self(void)
 	memset(acked, 0, sizeof(acked));
 	smp_wmb();
 	cpumask_clear(&mask);
-	cpumask_set_cpu(0, &mask);
+	cpumask_set_cpu(smp_processor_id(), &mask);
 	gic->ipi.send_self();
 	check_acked(&mask);
 	report_prefix_pop();
@@ -249,7 +267,7 @@ static void ipi_test_smp(void)
 	memset(acked, 0, sizeof(acked));
 	smp_wmb();
 	cpumask_copy(&mask, &cpu_present_mask);
-	for (i = 0; i < nr_cpus; i += 2)
+	for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
 		cpumask_clear_cpu(i, &mask);
 	gic->ipi.send_tlist(&mask);
 	check_acked(&mask);
@@ -259,7 +277,7 @@ static void ipi_test_smp(void)
 	memset(acked, 0, sizeof(acked));
 	smp_wmb();
 	cpumask_copy(&mask, &cpu_present_mask);
-	cpumask_clear_cpu(0, &mask);
+	cpumask_clear_cpu(smp_processor_id(), &mask);
 	gic->ipi.send_broadcast();
 	check_acked(&mask);
 	report_prefix_pop();
@@ -276,6 +294,27 @@ static void ipi_enable(void)
 	local_irq_enable();
 }
 
+static void ipi_send(void)
+{
+	int cpu;
+
+	ipi_enable();
+	wait_on_ready();
+	ipi_test_self();
+	ipi_test_smp();
+
+	smp_rmb();
+	for_each_present_cpu(cpu) {
+		if (spurious[cpu]) {
+			printf("ipi: WARN: cpu%d got %d spurious "
+			       "interrupts\n",
+			       spurious[cpu], smp_processor_id());
+		}
+	}
+
+	exit(report_summary());
+}
+
 static void ipi_recv(void)
 {
 	ipi_enable();
@@ -284,6 +323,14 @@ static void ipi_recv(void)
 		wfi();
 }
 
+static void ipi_test(void)
+{
+	if (smp_processor_id() == sender)
+		ipi_send();
+	else
+		ipi_recv();
+}
+
 struct gic gicv2 = {
 	.ipi = {
 		.enable = gicv2_enable_defaults,
@@ -337,30 +384,30 @@ int main(int argc, char **argv)
 		report_prefix_pop();
 
 	} else if (!strcmp(argv[1], "ipi")) {
+		int off, i = 1;
+		long val;
 
 		report_prefix_push(argv[1]);
 		nr_cpu_check(2);
 
-		for_each_present_cpu(cpu) {
-			if (cpu == 0)
+		while (--argc != 1) {
+			off = parse_keyval(argv[++i], &val);
+			if (off == -1)
 				continue;
-			smp_boot_secondary(cpu, ipi_recv);
+			argv[i][off] = '\0';
+			if (strcmp(argv[i], "sender") == 0)
+				sender = val;
+			else if (strcmp(argv[i], "irq") == 0)
+				irq = val;
 		}
-		ipi_enable();
-		wait_on_ready();
-		ipi_test_self();
-		ipi_test_smp();
 
-		smp_rmb();
 		for_each_present_cpu(cpu) {
-			if (spurious[cpu]) {
-				printf("ipi: WARN: cpu%d got %d spurious "
-				       "interrupts\n",
-				       spurious[cpu], smp_processor_id());
-			}
+			if (cpu == 0)
+				continue;
+			smp_boot_secondary(cpu, ipi_test);
 		}
 
-		report_prefix_pop();
+		ipi_test();
 
 	} else {
 		report_abort("Unknown subtest '%s'", argv[1]);
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 65+ messages in thread
* [Qemu-devel] [kvm-unit-tests PATCH v5 11/11] arm/arm64: gic: don't just use zero
@ 2016-11-10 17:21   ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 17:21 UTC (permalink / raw)
  To: kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, andre.przywara, peter.maydell, alex.bennee,
	marc.zyngier, eric.auger, christoffer.dall

Allow user to select who sends ipis and with which irq,
rather than just always sending irq=0 from cpu0.

Signed-off-by: Andrew Jones <drjones@redhat.com>

---
v4: improve structure and make sure spurious checking is
    done even when the sender isn't cpu0
v2: actually check that the irq received was the irq sent,
    and (for gicv2) that the sender is the expected one.
---
 arm/gic.c | 99 ++++++++++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 73 insertions(+), 26 deletions(-)

diff --git a/arm/gic.c b/arm/gic.c
index ca68f8ad1cb9..af9e745e2e5c 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -11,6 +11,7 @@
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
 #include <libcflat.h>
+#include <util.h>
 #include <asm/setup.h>
 #include <asm/processor.h>
 #include <asm/gic.h>
@@ -34,6 +35,8 @@ static struct gic *gic;
 static int gic_version;
 static int acked[NR_CPUS], spurious[NR_CPUS];
 static cpumask_t ready;
+static int sender;
+static u32 irq;
 
 static void nr_cpu_check(int nr)
 {
@@ -86,7 +89,16 @@ static void check_acked(cpumask_t *mask)
 
 static u32 gicv2_read_iar(void)
 {
-	return readl(gicv2_cpu_base() + GICC_IAR);
+	u32 iar = readl(gicv2_cpu_base() + GICC_IAR);
+	int src = (iar >> 10) & 7;
+
+	if (src != sender) {
+		report("cpu%d received IPI from unexpected source cpu%d "
+		       "(expected cpu%d)",
+		       false, smp_processor_id(), src, sender);
+	}
+
+	return iar;
 }
 
 static u32 gicv2_irqnr(u32 iar)
@@ -111,9 +123,15 @@ static void ipi_handler(struct pt_regs *regs __unused)
 
 	if (irqnr != GICC_INT_SPURIOUS) {
 		gic->write_eoi(irqstat);
-		smp_rmb(); /* pairs with wmb in ipi_test functions */
-		++acked[smp_processor_id()];
-		smp_wmb(); /* pairs with rmb in check_acked */
+		if (irqnr == irq) {
+			smp_rmb(); /* pairs with wmb in ipi_test functions */
+			++acked[smp_processor_id()];
+			smp_wmb(); /* pairs with rmb in check_acked */
+		} else {
+			report("cpu%d received unexpected irq %u "
+			       "(expected %u)",
+			       false, smp_processor_id(), irqnr, irq);
+		}
 	} else {
 		++spurious[smp_processor_id()];
 		smp_wmb();
@@ -122,19 +140,19 @@ static void ipi_handler(struct pt_regs *regs __unused)
 
 static void gicv2_ipi_send_self(void)
 {
-	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
+	writel(2 << 24 | irq, gicv2_dist_base() + GICD_SGIR);
 }
 
 static void gicv2_ipi_send_tlist(cpumask_t *mask)
 {
 	u8 tlist = (u8)cpumask_bits(mask)[0];
 
-	writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
+	writel(tlist << 16 | irq, gicv2_dist_base() + GICD_SGIR);
 }
 
 static void gicv2_ipi_send_broadcast(void)
 {
-	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
+	writel(1 << 24 | irq, gicv2_dist_base() + GICD_SGIR);
 }
 
 #define ICC_SGI1R_AFFINITY_1_SHIFT	16
@@ -200,7 +218,7 @@ static void gicv3_ipi_send_tlist(cpumask_t *mask)
 		/* Send the IPIs for the target list of this cluster */
 		sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3)	|
 			 MPIDR_TO_SGI_AFFINITY(cluster_id, 2)	|
-			 /* irq << 24				| */
+			 irq << 24				|
 			 MPIDR_TO_SGI_AFFINITY(cluster_id, 1)	|
 			 tlist);
 
@@ -222,7 +240,7 @@ static void gicv3_ipi_send_self(void)
 
 static void gicv3_ipi_send_broadcast(void)
 {
-	gicv3_write_sgi1r(1ULL << 40);
+	gicv3_write_sgi1r(1ULL << 40 | irq << 24);
 	isb();
 }
 
@@ -234,7 +252,7 @@ static void ipi_test_self(void)
 	memset(acked, 0, sizeof(acked));
 	smp_wmb();
 	cpumask_clear(&mask);
-	cpumask_set_cpu(0, &mask);
+	cpumask_set_cpu(smp_processor_id(), &mask);
 	gic->ipi.send_self();
 	check_acked(&mask);
 	report_prefix_pop();
@@ -249,7 +267,7 @@ static void ipi_test_smp(void)
 	memset(acked, 0, sizeof(acked));
 	smp_wmb();
 	cpumask_copy(&mask, &cpu_present_mask);
-	for (i = 0; i < nr_cpus; i += 2)
+	for (i = smp_processor_id() & 1; i < nr_cpus; i += 2)
 		cpumask_clear_cpu(i, &mask);
 	gic->ipi.send_tlist(&mask);
 	check_acked(&mask);
@@ -259,7 +277,7 @@ static void ipi_test_smp(void)
 	memset(acked, 0, sizeof(acked));
 	smp_wmb();
 	cpumask_copy(&mask, &cpu_present_mask);
-	cpumask_clear_cpu(0, &mask);
+	cpumask_clear_cpu(smp_processor_id(), &mask);
 	gic->ipi.send_broadcast();
 	check_acked(&mask);
 	report_prefix_pop();
@@ -276,6 +294,27 @@ static void ipi_enable(void)
 	local_irq_enable();
 }
 
+static void ipi_send(void)
+{
+	int cpu;
+
+	ipi_enable();
+	wait_on_ready();
+	ipi_test_self();
+	ipi_test_smp();
+
+	smp_rmb();
+	for_each_present_cpu(cpu) {
+		if (spurious[cpu]) {
+			printf("ipi: WARN: cpu%d got %d spurious "
+			       "interrupts\n",
+			       spurious[cpu], smp_processor_id());
+		}
+	}
+
+	exit(report_summary());
+}
+
 static void ipi_recv(void)
 {
 	ipi_enable();
@@ -284,6 +323,14 @@ static void ipi_recv(void)
 		wfi();
 }
 
+static void ipi_test(void)
+{
+	if (smp_processor_id() == sender)
+		ipi_send();
+	else
+		ipi_recv();
+}
+
 struct gic gicv2 = {
 	.ipi = {
 		.enable = gicv2_enable_defaults,
@@ -337,30 +384,30 @@ int main(int argc, char **argv)
 		report_prefix_pop();
 
 	} else if (!strcmp(argv[1], "ipi")) {
+		int off, i = 1;
+		long val;
 
 		report_prefix_push(argv[1]);
 		nr_cpu_check(2);
 
-		for_each_present_cpu(cpu) {
-			if (cpu == 0)
+		while (--argc != 1) {
+			off = parse_keyval(argv[++i], &val);
+			if (off == -1)
 				continue;
-			smp_boot_secondary(cpu, ipi_recv);
+			argv[i][off] = '\0';
+			if (strcmp(argv[i], "sender") == 0)
+				sender = val;
+			else if (strcmp(argv[i], "irq") == 0)
+				irq = val;
 		}
-		ipi_enable();
-		wait_on_ready();
-		ipi_test_self();
-		ipi_test_smp();
 
-		smp_rmb();
 		for_each_present_cpu(cpu) {
-			if (spurious[cpu]) {
-				printf("ipi: WARN: cpu%d got %d spurious "
-				       "interrupts\n",
-				       spurious[cpu], smp_processor_id());
-			}
+			if (cpu == 0)
+				continue;
+			smp_boot_secondary(cpu, ipi_test);
 		}
 
-		report_prefix_pop();
+		ipi_test();
 
 	} else {
 		report_abort("Unknown subtest '%s'", argv[1]);
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
  2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
@ 2016-11-10 19:53     ` Alex Bennée
  -1 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-10 19:53 UTC (permalink / raw)
  To: Andrew Jones
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
	peter.maydell, marc.zyngier, eric.auger, christoffer.dall


Andrew Jones <drjones@redhat.com> writes:

> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v5:
>  - fix copy+paste error in gicv3_write_eoir [drew]
>  - use modern register names [Andre]
> v4:
>  - heavily comment gicv3_ipi_send_tlist() [Eric]
>  - changes needed for gicv2 iar/irqstat fix to other patch
> v2:
>  - use IRM for gicv3 broadcast
> ---
>  arm/gic.c                  | 195 ++++++++++++++++++++++++++++++++++++++++++---
>  arm/unittests.cfg          |   6 ++
>  lib/arm/asm/arch_gicv3.h   |  23 ++++++
>  lib/arm64/asm/arch_gicv3.h |  22 +++++
>  4 files changed, 236 insertions(+), 10 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index 06092aec7c08..ca68f8ad1cb9 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -3,6 +3,8 @@
>   *
>   * GICv2
>   *   + test sending/receiving IPIs
> + * GICv3
> + *   + test sending/receiving IPIs
>   *
>   * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
>   *
> @@ -16,6 +18,19 @@
>  #include <asm/barrier.h>
>  #include <asm/io.h>
>
> +struct gic {
> +	struct {
> +		void (*enable)(void);
> +		void (*send_self)(void);
> +		void (*send_tlist)(cpumask_t *);
> +		void (*send_broadcast)(void);
> +	} ipi;
> +	u32 (*read_iar)(void);
> +	u32 (*irqnr)(u32 iar);
> +	void (*write_eoi)(u32);
> +};
> +
> +static struct gic *gic;
>  static int gic_version;
>  static int acked[NR_CPUS], spurious[NR_CPUS];
>  static cpumask_t ready;
> @@ -69,13 +84,33 @@ static void check_acked(cpumask_t *mask)
>  	       false, missing, extra, unexpected);
>  }
>
> +static u32 gicv2_read_iar(void)
> +{
> +	return readl(gicv2_cpu_base() + GICC_IAR);
> +}
> +
> +static u32 gicv2_irqnr(u32 iar)
> +{
> +	return iar & GICC_IAR_INT_ID_MASK;
> +}
> +
> +static void gicv2_write_eoi(u32 irqstat)
> +{
> +	writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
> +}
> +
> +static u32 gicv3_irqnr(u32 iar)
> +{
> +	return iar;
> +}
> +
>  static void ipi_handler(struct pt_regs *regs __unused)
>  {
> -	u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
> -	u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
> +	u32 irqstat = gic->read_iar();
> +	u32 irqnr = gic->irqnr(irqstat);
>
>  	if (irqnr != GICC_INT_SPURIOUS) {
> -		writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
> +		gic->write_eoi(irqstat);
>  		smp_rmb(); /* pairs with wmb in ipi_test functions */
>  		++acked[smp_processor_id()];
>  		smp_wmb(); /* pairs with rmb in check_acked */
> @@ -85,6 +120,112 @@ static void ipi_handler(struct pt_regs *regs __unused)
>  	}
>  }
>
> +static void gicv2_ipi_send_self(void)
> +{
> +	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +static void gicv2_ipi_send_tlist(cpumask_t *mask)
> +{
> +	u8 tlist = (u8)cpumask_bits(mask)[0];
> +
> +	writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +static void gicv2_ipi_send_broadcast(void)
> +{
> +	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +#define ICC_SGI1R_AFFINITY_1_SHIFT	16
> +#define ICC_SGI1R_AFFINITY_2_SHIFT	32
> +#define ICC_SGI1R_AFFINITY_3_SHIFT	48
> +#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> +	(MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
> +
> +static void gicv3_ipi_send_tlist(cpumask_t *mask)
> +{
> +	u16 tlist;
> +	int cpu;
> +
> +	/*
> +	 * For each cpu in the mask collect its peers, which are also in
> +	 * the mask, in order to form target lists.
> +	 */
> +	for_each_cpu(cpu, mask) {
> +		u64 mpidr = cpus[cpu], sgi1r;
> +		u64 cluster_id;
> +
> +		/*
> +		 * GICv3 can send IPIs to up 16 peer cpus with a single
> +		 * write to ICC_SGI1R_EL1 (using the target list). Peers
> +		 * are cpus that have nearly identical MPIDRs, the only
> +		 * difference being Aff0. The matching upper affinity
> +		 * levels form the cluster ID.
> +		 */
> +		cluster_id = mpidr & ~0xffUL;
> +		tlist = 0;
> +
> +		/*
> +		 * Sort of open code for_each_cpu in order to have a
> +		 * nested for_each_cpu loop.
> +		 */
> +		while (cpu < nr_cpus) {
> +			if ((mpidr & 0xff) >= 16) {
> +				printf("cpu%d MPIDR:aff0 is %d (>= 16)!\n",
> +					cpu, (int)(mpidr & 0xff));
> +				break;
> +			}
> +
> +			tlist |= 1 << (mpidr & 0xf);
> +
> +			cpu = cpumask_next(cpu, mask);
> +			if (cpu >= nr_cpus)
> +				break;
> +
> +			mpidr = cpus[cpu];
> +
> +			if (cluster_id != (mpidr & ~0xffUL)) {
> +				/*
> +				 * The next cpu isn't in our cluster. Roll
> +				 * back the cpu index allowing the outer
> +				 * for_each_cpu to find it again with
> +				 * cpumask_next
> +				 */
> +				--cpu;
> +				break;
> +			}
> +		}
> +
> +		/* Send the IPIs for the target list of this cluster */
> +		sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3)	|
> +			 MPIDR_TO_SGI_AFFINITY(cluster_id, 2)	|
> +			 /* irq << 24				| */
> +			 MPIDR_TO_SGI_AFFINITY(cluster_id, 1)	|
> +			 tlist);
> +
> +		gicv3_write_sgi1r(sgi1r);
> +	}
> +
> +	/* Force the above writes to ICC_SGI1R_EL1 to be executed */
> +	isb();
> +}
> +
> +static void gicv3_ipi_send_self(void)
> +{
> +	cpumask_t mask;
> +
> +	cpumask_clear(&mask);
> +	cpumask_set_cpu(smp_processor_id(), &mask);
> +	gicv3_ipi_send_tlist(&mask);
> +}
> +
> +static void gicv3_ipi_send_broadcast(void)
> +{
> +	gicv3_write_sgi1r(1ULL << 40);
> +	isb();
> +}
> +
>  static void ipi_test_self(void)
>  {
>  	cpumask_t mask;
> @@ -94,7 +235,7 @@ static void ipi_test_self(void)
>  	smp_wmb();
>  	cpumask_clear(&mask);
>  	cpumask_set_cpu(0, &mask);
> -	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> +	gic->ipi.send_self();
>  	check_acked(&mask);
>  	report_prefix_pop();
>  }
> @@ -102,14 +243,15 @@ static void ipi_test_self(void)
>  static void ipi_test_smp(void)
>  {
>  	cpumask_t mask;
> -	unsigned long tlist;
> +	int i;
>
>  	report_prefix_push("target-list");
>  	memset(acked, 0, sizeof(acked));
>  	smp_wmb();
> -	tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
> -	cpumask_bits(&mask)[0] = tlist;
> -	writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
> +	cpumask_copy(&mask, &cpu_present_mask);
> +	for (i = 0; i < nr_cpus; i += 2)
> +		cpumask_clear_cpu(i, &mask);
> +	gic->ipi.send_tlist(&mask);
>  	check_acked(&mask);
>  	report_prefix_pop();
>
> @@ -118,14 +260,14 @@ static void ipi_test_smp(void)
>  	smp_wmb();
>  	cpumask_copy(&mask, &cpu_present_mask);
>  	cpumask_clear_cpu(0, &mask);
> -	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> +	gic->ipi.send_broadcast();
>  	check_acked(&mask);
>  	report_prefix_pop();
>  }
>
>  static void ipi_enable(void)
>  {
> -	gicv2_enable_defaults();
> +	gic->ipi.enable();
>  #ifdef __arm__
>  	install_exception_handler(EXCPTN_IRQ, ipi_handler);
>  #else
> @@ -142,6 +284,30 @@ static void ipi_recv(void)
>  		wfi();
>  }
>
> +struct gic gicv2 = {
> +	.ipi = {
> +		.enable = gicv2_enable_defaults,
> +		.send_self = gicv2_ipi_send_self,
> +		.send_tlist = gicv2_ipi_send_tlist,
> +		.send_broadcast = gicv2_ipi_send_broadcast,
> +	},
> +	.read_iar = gicv2_read_iar,
> +	.irqnr = gicv2_irqnr,
> +	.write_eoi = gicv2_write_eoi,
> +};
> +
> +struct gic gicv3 = {
> +	.ipi = {
> +		.enable = gicv3_enable_defaults,
> +		.send_self = gicv3_ipi_send_self,
> +		.send_tlist = gicv3_ipi_send_tlist,
> +		.send_broadcast = gicv3_ipi_send_broadcast,
> +	},
> +	.read_iar = gicv3_read_iar,
> +	.irqnr = gicv3_irqnr,
> +	.write_eoi = gicv3_write_eoir,
> +};
> +

So I was re-basing my kvm-unit-tests against your GIC rework and found
myself copy and pasting a bunch of this into my tests that fire IRQs.
That makes me think the abstraction should be in the library code so
other tests can fiddle with sending IRQs.

What do you think?

>  int main(int argc, char **argv)
>  {
>  	char pfx[8];
> @@ -154,6 +320,15 @@ int main(int argc, char **argv)
>  	snprintf(pfx, 8, "gicv%d", gic_version);
>  	report_prefix_push(pfx);
>
> +	switch (gic_version) {
> +	case 2:
> +		gic = &gicv2;
> +		break;
> +	case 3:
> +		gic = &gicv3;
> +		break;
> +	}
> +
>  	if (argc < 2) {
>
>  		report_prefix_push("ipi");
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index 68bf5cd6008f..ce3cee73c4bf 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -61,3 +61,9 @@ file = gic.flat
>  smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
>  extra_params = -machine gic-version=2 -append 'ipi'
>  groups = gic
> +
> +[gicv3-ipi]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'ipi'
> +groups = gic
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> index 81a1e5f6c29c..615a3393e439 100644
> --- a/lib/arm/asm/arch_gicv3.h
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -16,10 +16,28 @@
>  #define __stringify xstr
>
>  #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> +#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
>
> +#define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
> +#define ICC_IAR1			__ACCESS_CP15(c12, 0, c12, 0)
> +#define ICC_SGI1R			__ACCESS_CP15_64(0, c12)
>  #define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
>  #define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
>
> +static inline void gicv3_write_eoir(u32 irq)
> +{
> +	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> +	isb();
> +}
> +
> +static inline u32 gicv3_read_iar(void)
> +{
> +	u32 irqstat;
> +	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
> +	dsb(sy);
> +	return irqstat;
> +}
> +
>  static inline void gicv3_write_pmr(u32 val)
>  {
>  	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> @@ -31,6 +49,11 @@ static inline void gicv3_write_grpen1(u32 val)
>  	isb();
>  }
>
> +static inline void gicv3_write_sgi1r(u64 val)
> +{
> +	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> +}
> +
>  static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
>  {
>  	u64 val = readl(addr);
> diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> index 6d353567f56a..874775828016 100644
> --- a/lib/arm64/asm/arch_gicv3.h
> +++ b/lib/arm64/asm/arch_gicv3.h
> @@ -10,6 +10,9 @@
>
>  #include <asm/sysreg.h>
>
> +#define ICC_EOIR1_EL1			sys_reg(3, 0, 12, 12, 1)
> +#define ICC_IAR1_EL1			sys_reg(3, 0, 12, 12, 0)
> +#define ICC_SGI1R_EL1			sys_reg(3, 0, 12, 11, 5)
>  #define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
>  #define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
>
> @@ -27,6 +30,20 @@
>   * sets the GP register's most significant bits to 0 with an explicit cast.
>   */
>
> +static inline void gicv3_write_eoir(u32 irq)
> +{
> +	asm volatile("msr_s " __stringify(ICC_EOIR1_EL1) ", %0" : : "r" ((u64)irq));
> +	isb();
> +}
> +
> +static inline u32 gicv3_read_iar(void)
> +{
> +	u64 irqstat;
> +	asm volatile("mrs_s %0, " __stringify(ICC_IAR1_EL1) : "=r" (irqstat));
> +	dsb(sy);
> +	return (u64)irqstat;
> +}
> +
>  static inline void gicv3_write_pmr(u32 val)
>  {
>  	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> @@ -38,6 +55,11 @@ static inline void gicv3_write_grpen1(u32 val)
>  	isb();
>  }
>
> +static inline void gicv3_write_sgi1r(u64 val)
> +{
> +	asm volatile("msr_s " __stringify(ICC_SGI1R_EL1) ", %0" : : "r" (val));
> +}
> +
>  #define gicv3_read_typer(c)		readq(c)
>
>  #endif /* __ASSEMBLY__ */


--
Alex Bennée

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-10 19:53     ` Alex Bennée
  0 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-10 19:53 UTC (permalink / raw)
  To: Andrew Jones
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
	peter.maydell, marc.zyngier, eric.auger, christoffer.dall


Andrew Jones <drjones@redhat.com> writes:

> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v5:
>  - fix copy+paste error in gicv3_write_eoir [drew]
>  - use modern register names [Andre]
> v4:
>  - heavily comment gicv3_ipi_send_tlist() [Eric]
>  - changes needed for gicv2 iar/irqstat fix to other patch
> v2:
>  - use IRM for gicv3 broadcast
> ---
>  arm/gic.c                  | 195 ++++++++++++++++++++++++++++++++++++++++++---
>  arm/unittests.cfg          |   6 ++
>  lib/arm/asm/arch_gicv3.h   |  23 ++++++
>  lib/arm64/asm/arch_gicv3.h |  22 +++++
>  4 files changed, 236 insertions(+), 10 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index 06092aec7c08..ca68f8ad1cb9 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -3,6 +3,8 @@
>   *
>   * GICv2
>   *   + test sending/receiving IPIs
> + * GICv3
> + *   + test sending/receiving IPIs
>   *
>   * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
>   *
> @@ -16,6 +18,19 @@
>  #include <asm/barrier.h>
>  #include <asm/io.h>
>
> +struct gic {
> +	struct {
> +		void (*enable)(void);
> +		void (*send_self)(void);
> +		void (*send_tlist)(cpumask_t *);
> +		void (*send_broadcast)(void);
> +	} ipi;
> +	u32 (*read_iar)(void);
> +	u32 (*irqnr)(u32 iar);
> +	void (*write_eoi)(u32);
> +};
> +
> +static struct gic *gic;
>  static int gic_version;
>  static int acked[NR_CPUS], spurious[NR_CPUS];
>  static cpumask_t ready;
> @@ -69,13 +84,33 @@ static void check_acked(cpumask_t *mask)
>  	       false, missing, extra, unexpected);
>  }
>
> +static u32 gicv2_read_iar(void)
> +{
> +	return readl(gicv2_cpu_base() + GICC_IAR);
> +}
> +
> +static u32 gicv2_irqnr(u32 iar)
> +{
> +	return iar & GICC_IAR_INT_ID_MASK;
> +}
> +
> +static void gicv2_write_eoi(u32 irqstat)
> +{
> +	writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
> +}
> +
> +static u32 gicv3_irqnr(u32 iar)
> +{
> +	return iar;
> +}
> +
>  static void ipi_handler(struct pt_regs *regs __unused)
>  {
> -	u32 irqstat = readl(gicv2_cpu_base() + GICC_IAR);
> -	u32 irqnr = irqstat & GICC_IAR_INT_ID_MASK;
> +	u32 irqstat = gic->read_iar();
> +	u32 irqnr = gic->irqnr(irqstat);
>
>  	if (irqnr != GICC_INT_SPURIOUS) {
> -		writel(irqstat, gicv2_cpu_base() + GICC_EOIR);
> +		gic->write_eoi(irqstat);
>  		smp_rmb(); /* pairs with wmb in ipi_test functions */
>  		++acked[smp_processor_id()];
>  		smp_wmb(); /* pairs with rmb in check_acked */
> @@ -85,6 +120,112 @@ static void ipi_handler(struct pt_regs *regs __unused)
>  	}
>  }
>
> +static void gicv2_ipi_send_self(void)
> +{
> +	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +static void gicv2_ipi_send_tlist(cpumask_t *mask)
> +{
> +	u8 tlist = (u8)cpumask_bits(mask)[0];
> +
> +	writel(tlist << 16, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +static void gicv2_ipi_send_broadcast(void)
> +{
> +	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> +}
> +
> +#define ICC_SGI1R_AFFINITY_1_SHIFT	16
> +#define ICC_SGI1R_AFFINITY_2_SHIFT	32
> +#define ICC_SGI1R_AFFINITY_3_SHIFT	48
> +#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> +	(MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
> +
> +static void gicv3_ipi_send_tlist(cpumask_t *mask)
> +{
> +	u16 tlist;
> +	int cpu;
> +
> +	/*
> +	 * For each cpu in the mask collect its peers, which are also in
> +	 * the mask, in order to form target lists.
> +	 */
> +	for_each_cpu(cpu, mask) {
> +		u64 mpidr = cpus[cpu], sgi1r;
> +		u64 cluster_id;
> +
> +		/*
> +		 * GICv3 can send IPIs to up 16 peer cpus with a single
> +		 * write to ICC_SGI1R_EL1 (using the target list). Peers
> +		 * are cpus that have nearly identical MPIDRs, the only
> +		 * difference being Aff0. The matching upper affinity
> +		 * levels form the cluster ID.
> +		 */
> +		cluster_id = mpidr & ~0xffUL;
> +		tlist = 0;
> +
> +		/*
> +		 * Sort of open code for_each_cpu in order to have a
> +		 * nested for_each_cpu loop.
> +		 */
> +		while (cpu < nr_cpus) {
> +			if ((mpidr & 0xff) >= 16) {
> +				printf("cpu%d MPIDR:aff0 is %d (>= 16)!\n",
> +					cpu, (int)(mpidr & 0xff));
> +				break;
> +			}
> +
> +			tlist |= 1 << (mpidr & 0xf);
> +
> +			cpu = cpumask_next(cpu, mask);
> +			if (cpu >= nr_cpus)
> +				break;
> +
> +			mpidr = cpus[cpu];
> +
> +			if (cluster_id != (mpidr & ~0xffUL)) {
> +				/*
> +				 * The next cpu isn't in our cluster. Roll
> +				 * back the cpu index allowing the outer
> +				 * for_each_cpu to find it again with
> +				 * cpumask_next
> +				 */
> +				--cpu;
> +				break;
> +			}
> +		}
> +
> +		/* Send the IPIs for the target list of this cluster */
> +		sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3)	|
> +			 MPIDR_TO_SGI_AFFINITY(cluster_id, 2)	|
> +			 /* irq << 24				| */
> +			 MPIDR_TO_SGI_AFFINITY(cluster_id, 1)	|
> +			 tlist);
> +
> +		gicv3_write_sgi1r(sgi1r);
> +	}
> +
> +	/* Force the above writes to ICC_SGI1R_EL1 to be executed */
> +	isb();
> +}
> +
> +static void gicv3_ipi_send_self(void)
> +{
> +	cpumask_t mask;
> +
> +	cpumask_clear(&mask);
> +	cpumask_set_cpu(smp_processor_id(), &mask);
> +	gicv3_ipi_send_tlist(&mask);
> +}
> +
> +static void gicv3_ipi_send_broadcast(void)
> +{
> +	gicv3_write_sgi1r(1ULL << 40);
> +	isb();
> +}
> +
>  static void ipi_test_self(void)
>  {
>  	cpumask_t mask;
> @@ -94,7 +235,7 @@ static void ipi_test_self(void)
>  	smp_wmb();
>  	cpumask_clear(&mask);
>  	cpumask_set_cpu(0, &mask);
> -	writel(2 << 24, gicv2_dist_base() + GICD_SGIR);
> +	gic->ipi.send_self();
>  	check_acked(&mask);
>  	report_prefix_pop();
>  }
> @@ -102,14 +243,15 @@ static void ipi_test_self(void)
>  static void ipi_test_smp(void)
>  {
>  	cpumask_t mask;
> -	unsigned long tlist;
> +	int i;
>
>  	report_prefix_push("target-list");
>  	memset(acked, 0, sizeof(acked));
>  	smp_wmb();
> -	tlist = cpumask_bits(&cpu_present_mask)[0] & 0xaa;
> -	cpumask_bits(&mask)[0] = tlist;
> -	writel((u8)tlist << 16, gicv2_dist_base() + GICD_SGIR);
> +	cpumask_copy(&mask, &cpu_present_mask);
> +	for (i = 0; i < nr_cpus; i += 2)
> +		cpumask_clear_cpu(i, &mask);
> +	gic->ipi.send_tlist(&mask);
>  	check_acked(&mask);
>  	report_prefix_pop();
>
> @@ -118,14 +260,14 @@ static void ipi_test_smp(void)
>  	smp_wmb();
>  	cpumask_copy(&mask, &cpu_present_mask);
>  	cpumask_clear_cpu(0, &mask);
> -	writel(1 << 24, gicv2_dist_base() + GICD_SGIR);
> +	gic->ipi.send_broadcast();
>  	check_acked(&mask);
>  	report_prefix_pop();
>  }
>
>  static void ipi_enable(void)
>  {
> -	gicv2_enable_defaults();
> +	gic->ipi.enable();
>  #ifdef __arm__
>  	install_exception_handler(EXCPTN_IRQ, ipi_handler);
>  #else
> @@ -142,6 +284,30 @@ static void ipi_recv(void)
>  		wfi();
>  }
>
> +struct gic gicv2 = {
> +	.ipi = {
> +		.enable = gicv2_enable_defaults,
> +		.send_self = gicv2_ipi_send_self,
> +		.send_tlist = gicv2_ipi_send_tlist,
> +		.send_broadcast = gicv2_ipi_send_broadcast,
> +	},
> +	.read_iar = gicv2_read_iar,
> +	.irqnr = gicv2_irqnr,
> +	.write_eoi = gicv2_write_eoi,
> +};
> +
> +struct gic gicv3 = {
> +	.ipi = {
> +		.enable = gicv3_enable_defaults,
> +		.send_self = gicv3_ipi_send_self,
> +		.send_tlist = gicv3_ipi_send_tlist,
> +		.send_broadcast = gicv3_ipi_send_broadcast,
> +	},
> +	.read_iar = gicv3_read_iar,
> +	.irqnr = gicv3_irqnr,
> +	.write_eoi = gicv3_write_eoir,
> +};
> +

So I was re-basing my kvm-unit-tests against your GIC rework and found
myself copy and pasting a bunch of this into my tests that fire IRQs.
That makes me think the abstraction should be in the library code so
other tests can fiddle with sending IRQs.

What do you think?

>  int main(int argc, char **argv)
>  {
>  	char pfx[8];
> @@ -154,6 +320,15 @@ int main(int argc, char **argv)
>  	snprintf(pfx, 8, "gicv%d", gic_version);
>  	report_prefix_push(pfx);
>
> +	switch (gic_version) {
> +	case 2:
> +		gic = &gicv2;
> +		break;
> +	case 3:
> +		gic = &gicv3;
> +		break;
> +	}
> +
>  	if (argc < 2) {
>
>  		report_prefix_push("ipi");
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index 68bf5cd6008f..ce3cee73c4bf 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -61,3 +61,9 @@ file = gic.flat
>  smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
>  extra_params = -machine gic-version=2 -append 'ipi'
>  groups = gic
> +
> +[gicv3-ipi]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'ipi'
> +groups = gic
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> index 81a1e5f6c29c..615a3393e439 100644
> --- a/lib/arm/asm/arch_gicv3.h
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -16,10 +16,28 @@
>  #define __stringify xstr
>
>  #define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> +#define __ACCESS_CP15_64(Op1, CRm)		p15, Op1, %Q0, %R0, CRm
>
> +#define ICC_EOIR1			__ACCESS_CP15(c12, 0, c12, 1)
> +#define ICC_IAR1			__ACCESS_CP15(c12, 0, c12, 0)
> +#define ICC_SGI1R			__ACCESS_CP15_64(0, c12)
>  #define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
>  #define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
>
> +static inline void gicv3_write_eoir(u32 irq)
> +{
> +	asm volatile("mcr " __stringify(ICC_EOIR1) : : "r" (irq));
> +	isb();
> +}
> +
> +static inline u32 gicv3_read_iar(void)
> +{
> +	u32 irqstat;
> +	asm volatile("mrc " __stringify(ICC_IAR1) : "=r" (irqstat));
> +	dsb(sy);
> +	return irqstat;
> +}
> +
>  static inline void gicv3_write_pmr(u32 val)
>  {
>  	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> @@ -31,6 +49,11 @@ static inline void gicv3_write_grpen1(u32 val)
>  	isb();
>  }
>
> +static inline void gicv3_write_sgi1r(u64 val)
> +{
> +	asm volatile("mcrr " __stringify(ICC_SGI1R) : : "r" (val));
> +}
> +
>  static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
>  {
>  	u64 val = readl(addr);
> diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> index 6d353567f56a..874775828016 100644
> --- a/lib/arm64/asm/arch_gicv3.h
> +++ b/lib/arm64/asm/arch_gicv3.h
> @@ -10,6 +10,9 @@
>
>  #include <asm/sysreg.h>
>
> +#define ICC_EOIR1_EL1			sys_reg(3, 0, 12, 12, 1)
> +#define ICC_IAR1_EL1			sys_reg(3, 0, 12, 12, 0)
> +#define ICC_SGI1R_EL1			sys_reg(3, 0, 12, 11, 5)
>  #define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
>  #define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
>
> @@ -27,6 +30,20 @@
>   * sets the GP register's most significant bits to 0 with an explicit cast.
>   */
>
> +static inline void gicv3_write_eoir(u32 irq)
> +{
> +	asm volatile("msr_s " __stringify(ICC_EOIR1_EL1) ", %0" : : "r" ((u64)irq));
> +	isb();
> +}
> +
> +static inline u32 gicv3_read_iar(void)
> +{
> +	u64 irqstat;
> +	asm volatile("mrs_s %0, " __stringify(ICC_IAR1_EL1) : "=r" (irqstat));
> +	dsb(sy);
> +	return (u64)irqstat;
> +}
> +
>  static inline void gicv3_write_pmr(u32 val)
>  {
>  	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> @@ -38,6 +55,11 @@ static inline void gicv3_write_grpen1(u32 val)
>  	isb();
>  }
>
> +static inline void gicv3_write_sgi1r(u64 val)
> +{
> +	asm volatile("msr_s " __stringify(ICC_SGI1R_EL1) ", %0" : : "r" (val));
> +}
> +
>  #define gicv3_read_typer(c)		readq(c)
>
>  #endif /* __ASSEMBLY__ */


--
Alex Bennée

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
  2016-11-10 19:53     ` [Qemu-devel] " Alex Bennée
@ 2016-11-10 20:37       ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 20:37 UTC (permalink / raw)
  To: Alex Bennée
  Cc: kvm, marc.zyngier, andre.przywara, qemu-devel, qemu-arm,
	pbonzini, kvmarm

On Thu, Nov 10, 2016 at 07:53:58PM +0000, Alex Bennée wrote:
[...]
> > +struct gic gicv2 = {
> > +	.ipi = {
> > +		.enable = gicv2_enable_defaults,
> > +		.send_self = gicv2_ipi_send_self,
> > +		.send_tlist = gicv2_ipi_send_tlist,
> > +		.send_broadcast = gicv2_ipi_send_broadcast,
> > +	},
> > +	.read_iar = gicv2_read_iar,
> > +	.irqnr = gicv2_irqnr,
> > +	.write_eoi = gicv2_write_eoi,
> > +};
> > +
> > +struct gic gicv3 = {
> > +	.ipi = {
> > +		.enable = gicv3_enable_defaults,
> > +		.send_self = gicv3_ipi_send_self,
> > +		.send_tlist = gicv3_ipi_send_tlist,
> > +		.send_broadcast = gicv3_ipi_send_broadcast,
> > +	},
> > +	.read_iar = gicv3_read_iar,
> > +	.irqnr = gicv3_irqnr,
> > +	.write_eoi = gicv3_write_eoir,
> > +};
> > +
> 
> So I was re-basing my kvm-unit-tests against your GIC rework and found
> myself copy and pasting a bunch of this into my tests that fire IRQs.
> That makes me think the abstraction should be in the library code so
> other tests can fiddle with sending IRQs.
> 
> What do you think?
>

I guess you mean moving the above two structs and their corresponding
functions (all which aren't already common) to lib/arm/ ? Or do you
just mean the one non-trivial function gicv3_ipi_send_tlist? I think
agree with gicv3_ipi_send_tlist getting shared, but the others are
mostly one-liners, so I'm not sure. I guess I'd have to see how you're
using them first.

Thanks,
drew

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-10 20:37       ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-10 20:37 UTC (permalink / raw)
  To: Alex Bennée
  Cc: peter.maydell, kvm, marc.zyngier, andre.przywara, qemu-devel,
	eric.auger, qemu-arm, pbonzini, kvmarm, christoffer.dall

On Thu, Nov 10, 2016 at 07:53:58PM +0000, Alex Bennée wrote:
[...]
> > +struct gic gicv2 = {
> > +	.ipi = {
> > +		.enable = gicv2_enable_defaults,
> > +		.send_self = gicv2_ipi_send_self,
> > +		.send_tlist = gicv2_ipi_send_tlist,
> > +		.send_broadcast = gicv2_ipi_send_broadcast,
> > +	},
> > +	.read_iar = gicv2_read_iar,
> > +	.irqnr = gicv2_irqnr,
> > +	.write_eoi = gicv2_write_eoi,
> > +};
> > +
> > +struct gic gicv3 = {
> > +	.ipi = {
> > +		.enable = gicv3_enable_defaults,
> > +		.send_self = gicv3_ipi_send_self,
> > +		.send_tlist = gicv3_ipi_send_tlist,
> > +		.send_broadcast = gicv3_ipi_send_broadcast,
> > +	},
> > +	.read_iar = gicv3_read_iar,
> > +	.irqnr = gicv3_irqnr,
> > +	.write_eoi = gicv3_write_eoir,
> > +};
> > +
> 
> So I was re-basing my kvm-unit-tests against your GIC rework and found
> myself copy and pasting a bunch of this into my tests that fire IRQs.
> That makes me think the abstraction should be in the library code so
> other tests can fiddle with sending IRQs.
> 
> What do you think?
>

I guess you mean moving the above two structs and their corresponding
functions (all which aren't already common) to lib/arm/ ? Or do you
just mean the one non-trivial function gicv3_ipi_send_tlist? I think
agree with gicv3_ipi_send_tlist getting shared, but the others are
mostly one-liners, so I'm not sure. I guess I'd have to see how you're
using them first.

Thanks,
drew

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
  2016-11-10 19:53     ` [Qemu-devel] " Alex Bennée
@ 2016-11-11  9:21       ` Andre Przywara
  -1 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11  9:21 UTC (permalink / raw)
  To: Alex Bennée, Andrew Jones
  Cc: kvm, marc.zyngier, qemu-devel, qemu-arm, pbonzini, kvmarm

Hi,

On 10/11/16 19:53, Alex Bennée wrote:

....
> So I was re-basing my kvm-unit-tests against your GIC rework and found
> myself copy and pasting a bunch of this into my tests that fire IRQs.

So I take it you are working on (or already have) code to test SPIs,
probably via GICD_ISPENDR?
Just asking because I was thinking about going there and thus could save
my time if you are on it already...

> That makes me think the abstraction should be in the library code so
> other tests can fiddle with sending IRQs.

...because I was wondering the same.

Cheers,
Andre.
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-11  9:21       ` Andre Przywara
  0 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11  9:21 UTC (permalink / raw)
  To: Alex Bennée, Andrew Jones
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, peter.maydell,
	marc.zyngier, eric.auger, christoffer.dall

Hi,

On 10/11/16 19:53, Alex Bennée wrote:

....
> So I was re-basing my kvm-unit-tests against your GIC rework and found
> myself copy and pasting a bunch of this into my tests that fire IRQs.

So I take it you are working on (or already have) code to test SPIs,
probably via GICD_ISPENDR?
Just asking because I was thinking about going there and thus could save
my time if you are on it already...

> That makes me think the abstraction should be in the library code so
> other tests can fiddle with sending IRQs.

...because I was wondering the same.

Cheers,
Andre.

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
  2016-11-11  9:21       ` [Qemu-devel] " Andre Przywara
@ 2016-11-11 10:00         ` Alex Bennée
  -1 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 10:00 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, qemu-devel, qemu-arm, pbonzini, kvmarm


Andre Przywara <andre.przywara@arm.com> writes:

> Hi,
>
> On 10/11/16 19:53, Alex Bennée wrote:
>
> ....
>> So I was re-basing my kvm-unit-tests against your GIC rework and found
>> myself copy and pasting a bunch of this into my tests that fire IRQs.
>
> So I take it you are working on (or already have) code to test SPIs,
> probably via GICD_ISPENDR?
> Just asking because I was thinking about going there and thus could save
> my time if you are on it already...

In my case I wanted to trigger SPIs to exercise my TCG tests for MTTCG:

  https://github.com/stsquad/kvm-unit-tests/blob/mttcg/current-tests-v6/arm/tcg-test.c#L113

>> That makes me think the abstraction should be in the library code so
>> other tests can fiddle with sending IRQs.
>
> ...because I was wondering the same.

To answer Andrew's question from the other post I would be happy with a
common:

  gic_enable
  gic_send_spi(cpu, irq)
  gic_irq_ack() which returns the iar.


>
> Cheers,
> Andre.


--
Alex Bennée
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-11 10:00         ` Alex Bennée
  0 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 10:00 UTC (permalink / raw)
  To: Andre Przywara
  Cc: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm, pbonzini,
	peter.maydell, marc.zyngier, eric.auger, christoffer.dall


Andre Przywara <andre.przywara@arm.com> writes:

> Hi,
>
> On 10/11/16 19:53, Alex Bennée wrote:
>
> ....
>> So I was re-basing my kvm-unit-tests against your GIC rework and found
>> myself copy and pasting a bunch of this into my tests that fire IRQs.
>
> So I take it you are working on (or already have) code to test SPIs,
> probably via GICD_ISPENDR?
> Just asking because I was thinking about going there and thus could save
> my time if you are on it already...

In my case I wanted to trigger SPIs to exercise my TCG tests for MTTCG:

  https://github.com/stsquad/kvm-unit-tests/blob/mttcg/current-tests-v6/arm/tcg-test.c#L113

>> That makes me think the abstraction should be in the library code so
>> other tests can fiddle with sending IRQs.
>
> ...because I was wondering the same.

To answer Andrew's question from the other post I would be happy with a
common:

  gic_enable
  gic_send_spi(cpu, irq)
  gic_irq_ack() which returns the iar.


>
> Cheers,
> Andre.


--
Alex Bennée

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
  2016-11-10 20:37       ` Andrew Jones
@ 2016-11-11 10:02         ` Alex Bennée
  -1 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 10:02 UTC (permalink / raw)
  To: Andrew Jones
  Cc: kvm, marc.zyngier, andre.przywara, qemu-devel, qemu-arm,
	pbonzini, kvmarm


Andrew Jones <drjones@redhat.com> writes:

> On Thu, Nov 10, 2016 at 07:53:58PM +0000, Alex Bennée wrote:
> [...]
>> > +struct gic gicv2 = {
>> > +	.ipi = {
>> > +		.enable = gicv2_enable_defaults,
>> > +		.send_self = gicv2_ipi_send_self,
>> > +		.send_tlist = gicv2_ipi_send_tlist,
>> > +		.send_broadcast = gicv2_ipi_send_broadcast,
>> > +	},
>> > +	.read_iar = gicv2_read_iar,
>> > +	.irqnr = gicv2_irqnr,
>> > +	.write_eoi = gicv2_write_eoi,
>> > +};
>> > +
>> > +struct gic gicv3 = {
>> > +	.ipi = {
>> > +		.enable = gicv3_enable_defaults,
>> > +		.send_self = gicv3_ipi_send_self,
>> > +		.send_tlist = gicv3_ipi_send_tlist,
>> > +		.send_broadcast = gicv3_ipi_send_broadcast,
>> > +	},
>> > +	.read_iar = gicv3_read_iar,
>> > +	.irqnr = gicv3_irqnr,
>> > +	.write_eoi = gicv3_write_eoir,
>> > +};
>> > +
>>
>> So I was re-basing my kvm-unit-tests against your GIC rework and found
>> myself copy and pasting a bunch of this into my tests that fire IRQs.
>> That makes me think the abstraction should be in the library code so
>> other tests can fiddle with sending IRQs.
>>
>> What do you think?
>>
>
> I guess you mean moving the above two structs and their corresponding
> functions (all which aren't already common) to lib/arm/ ? Or do you
> just mean the one non-trivial function gicv3_ipi_send_tlist? I think
> agree with gicv3_ipi_send_tlist getting shared, but the others are
> mostly one-liners, so I'm not sure. I guess I'd have to see how you're
> using them first.

So it looked like there were some functions in the common code for one
GIC which had local test defined functions for the other. They should at
least be consistent.

For my use case I could do with a common:

  gic_enable
  gic_send_spi(cpu, irq)
  gic_irq_ack() which returns the iar.

See:

  https://github.com/stsquad/kvm-unit-tests/blob/mttcg/current-tests-v6/arm/tcg-test.c#L113

>
> Thanks,
> drew


--
Alex Bennée
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-11 10:02         ` Alex Bennée
  0 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 10:02 UTC (permalink / raw)
  To: Andrew Jones
  Cc: peter.maydell, kvm, marc.zyngier, andre.przywara, qemu-devel,
	eric.auger, qemu-arm, pbonzini, kvmarm, christoffer.dall


Andrew Jones <drjones@redhat.com> writes:

> On Thu, Nov 10, 2016 at 07:53:58PM +0000, Alex Bennée wrote:
> [...]
>> > +struct gic gicv2 = {
>> > +	.ipi = {
>> > +		.enable = gicv2_enable_defaults,
>> > +		.send_self = gicv2_ipi_send_self,
>> > +		.send_tlist = gicv2_ipi_send_tlist,
>> > +		.send_broadcast = gicv2_ipi_send_broadcast,
>> > +	},
>> > +	.read_iar = gicv2_read_iar,
>> > +	.irqnr = gicv2_irqnr,
>> > +	.write_eoi = gicv2_write_eoi,
>> > +};
>> > +
>> > +struct gic gicv3 = {
>> > +	.ipi = {
>> > +		.enable = gicv3_enable_defaults,
>> > +		.send_self = gicv3_ipi_send_self,
>> > +		.send_tlist = gicv3_ipi_send_tlist,
>> > +		.send_broadcast = gicv3_ipi_send_broadcast,
>> > +	},
>> > +	.read_iar = gicv3_read_iar,
>> > +	.irqnr = gicv3_irqnr,
>> > +	.write_eoi = gicv3_write_eoir,
>> > +};
>> > +
>>
>> So I was re-basing my kvm-unit-tests against your GIC rework and found
>> myself copy and pasting a bunch of this into my tests that fire IRQs.
>> That makes me think the abstraction should be in the library code so
>> other tests can fiddle with sending IRQs.
>>
>> What do you think?
>>
>
> I guess you mean moving the above two structs and their corresponding
> functions (all which aren't already common) to lib/arm/ ? Or do you
> just mean the one non-trivial function gicv3_ipi_send_tlist? I think
> agree with gicv3_ipi_send_tlist getting shared, but the others are
> mostly one-liners, so I'm not sure. I guess I'd have to see how you're
> using them first.

So it looked like there were some functions in the common code for one
GIC which had local test defined functions for the other. They should at
least be consistent.

For my use case I could do with a common:

  gic_enable
  gic_send_spi(cpu, irq)
  gic_irq_ack() which returns the iar.

See:

  https://github.com/stsquad/kvm-unit-tests/blob/mttcg/current-tests-v6/arm/tcg-test.c#L113

>
> Thanks,
> drew


--
Alex Bennée

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 07/11] arm/arm64: gicv2: add an IPI test
  2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
@ 2016-11-11 11:13     ` Andre Przywara
  -1 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 11:13 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
  Cc: peter.maydell, marc.zyngier, eric.auger, pbonzini, alex.bennee,
	christoffer.dall

Hi,

more a comment loosely related to this patch ...

> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index 3f6fa45c587e..68bf5cd6008f 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -54,3 +54,10 @@ file = selftest.flat
>  smp = $MAX_SMP
>  extra_params = -append 'smp'
>  groups = selftest
> +
> +# Test GIC emulation
> +[gicv2-ipi]
> +file = gic.flat
> +smp = $((($MAX_SMP < 8)?$MAX_SMP:8))

So here we always go with the maximum number of VCPUs in the guest.
However (as you also noted in your cover-letter) running with a
different number of CPUs might be interesting, for instance with less
than 8 CPUs on a GICv2 (the ITARGETSR register must be masked) or in
general with an odd number (both literally and in the broader sense). I
have a test case with passes with 8 VCPUs but fails with less.

Is there any good way to run some tests multiple times with different
numbers of VCPUS?
Shall we add some "set" functionality to the smp parameter, so that we
can specify a list of desired test points?

Cheers,
Andre.

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 07/11] arm/arm64: gicv2: add an IPI test
@ 2016-11-11 11:13     ` Andre Przywara
  0 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 11:13 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, peter.maydell, alex.bennee, marc.zyngier, eric.auger,
	christoffer.dall

Hi,

more a comment loosely related to this patch ...

> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index 3f6fa45c587e..68bf5cd6008f 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -54,3 +54,10 @@ file = selftest.flat
>  smp = $MAX_SMP
>  extra_params = -append 'smp'
>  groups = selftest
> +
> +# Test GIC emulation
> +[gicv2-ipi]
> +file = gic.flat
> +smp = $((($MAX_SMP < 8)?$MAX_SMP:8))

So here we always go with the maximum number of VCPUs in the guest.
However (as you also noted in your cover-letter) running with a
different number of CPUs might be interesting, for instance with less
than 8 CPUs on a GICv2 (the ITARGETSR register must be masked) or in
general with an odd number (both literally and in the broader sense). I
have a test case with passes with 8 VCPUs but fails with less.

Is there any good way to run some tests multiple times with different
numbers of VCPUS?
Shall we add some "set" functionality to the smp parameter, so that we
can specify a list of desired test points?

Cheers,
Andre.

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 07/11] arm/arm64: gicv2: add an IPI test
  2016-11-11 11:13     ` [Qemu-devel] " Andre Przywara
@ 2016-11-11 13:13       ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-11 13:13 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, qemu-devel, qemu-arm, pbonzini, kvmarm

On Fri, Nov 11, 2016 at 11:13:46AM +0000, Andre Przywara wrote:
> Hi,
> 
> more a comment loosely related to this patch ...
> 
> > diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> > index 3f6fa45c587e..68bf5cd6008f 100644
> > --- a/arm/unittests.cfg
> > +++ b/arm/unittests.cfg
> > @@ -54,3 +54,10 @@ file = selftest.flat
> >  smp = $MAX_SMP
> >  extra_params = -append 'smp'
> >  groups = selftest
> > +
> > +# Test GIC emulation
> > +[gicv2-ipi]
> > +file = gic.flat
> > +smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
> 
> So here we always go with the maximum number of VCPUs in the guest.
> However (as you also noted in your cover-letter) running with a
> different number of CPUs might be interesting, for instance with less
> than 8 CPUs on a GICv2 (the ITARGETSR register must be masked) or in
> general with an odd number (both literally and in the broader sense). I
> have a test case with passes with 8 VCPUs but fails with less.
> 
> Is there any good way to run some tests multiple times with different
> numbers of VCPUS?
> Shall we add some "set" functionality to the smp parameter, so that we
> can specify a list of desired test points?
>

We can just add multiple entries, e.g.

[gicv2-ipi]
file = gic.flat
smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
[gicv2-ipi-3]
file = gic.flat
smp = $((($MAX_SMP > 3)?3:$MAX_SMP))

or whatever. But we need to always consider MAX_SMP, since some
machines may less than 8.

Thanks,
drew

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 07/11] arm/arm64: gicv2: add an IPI test
@ 2016-11-11 13:13       ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-11 13:13 UTC (permalink / raw)
  To: Andre Przywara
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, peter.maydell, marc.zyngier,
	eric.auger, pbonzini, alex.bennee, christoffer.dall

On Fri, Nov 11, 2016 at 11:13:46AM +0000, Andre Przywara wrote:
> Hi,
> 
> more a comment loosely related to this patch ...
> 
> > diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> > index 3f6fa45c587e..68bf5cd6008f 100644
> > --- a/arm/unittests.cfg
> > +++ b/arm/unittests.cfg
> > @@ -54,3 +54,10 @@ file = selftest.flat
> >  smp = $MAX_SMP
> >  extra_params = -append 'smp'
> >  groups = selftest
> > +
> > +# Test GIC emulation
> > +[gicv2-ipi]
> > +file = gic.flat
> > +smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
> 
> So here we always go with the maximum number of VCPUs in the guest.
> However (as you also noted in your cover-letter) running with a
> different number of CPUs might be interesting, for instance with less
> than 8 CPUs on a GICv2 (the ITARGETSR register must be masked) or in
> general with an odd number (both literally and in the broader sense). I
> have a test case with passes with 8 VCPUs but fails with less.
> 
> Is there any good way to run some tests multiple times with different
> numbers of VCPUS?
> Shall we add some "set" functionality to the smp parameter, so that we
> can specify a list of desired test points?
>

We can just add multiple entries, e.g.

[gicv2-ipi]
file = gic.flat
smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
[gicv2-ipi-3]
file = gic.flat
smp = $((($MAX_SMP > 3)?3:$MAX_SMP))

or whatever. But we need to always consider MAX_SMP, since some
machines may less than 8.

Thanks,
drew

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
  2016-11-11 10:02         ` Alex Bennée
@ 2016-11-11 13:54           ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-11 13:54 UTC (permalink / raw)
  To: Alex Bennée
  Cc: kvm, marc.zyngier, andre.przywara, qemu-devel, qemu-arm,
	pbonzini, kvmarm

On Fri, Nov 11, 2016 at 10:02:59AM +0000, Alex Bennée wrote:
> 
> Andrew Jones <drjones@redhat.com> writes:
> 
> > On Thu, Nov 10, 2016 at 07:53:58PM +0000, Alex Bennée wrote:
> > [...]
> >> > +struct gic gicv2 = {
> >> > +	.ipi = {
> >> > +		.enable = gicv2_enable_defaults,
> >> > +		.send_self = gicv2_ipi_send_self,
> >> > +		.send_tlist = gicv2_ipi_send_tlist,
> >> > +		.send_broadcast = gicv2_ipi_send_broadcast,
> >> > +	},
> >> > +	.read_iar = gicv2_read_iar,
> >> > +	.irqnr = gicv2_irqnr,
> >> > +	.write_eoi = gicv2_write_eoi,
> >> > +};
> >> > +
> >> > +struct gic gicv3 = {
> >> > +	.ipi = {
> >> > +		.enable = gicv3_enable_defaults,
> >> > +		.send_self = gicv3_ipi_send_self,
> >> > +		.send_tlist = gicv3_ipi_send_tlist,
> >> > +		.send_broadcast = gicv3_ipi_send_broadcast,
> >> > +	},
> >> > +	.read_iar = gicv3_read_iar,
> >> > +	.irqnr = gicv3_irqnr,
> >> > +	.write_eoi = gicv3_write_eoir,
> >> > +};
> >> > +
> >>
> >> So I was re-basing my kvm-unit-tests against your GIC rework and found
> >> myself copy and pasting a bunch of this into my tests that fire IRQs.
> >> That makes me think the abstraction should be in the library code so
> >> other tests can fiddle with sending IRQs.
> >>
> >> What do you think?
> >>
> >
> > I guess you mean moving the above two structs and their corresponding
> > functions (all which aren't already common) to lib/arm/ ? Or do you
> > just mean the one non-trivial function gicv3_ipi_send_tlist? I think
> > agree with gicv3_ipi_send_tlist getting shared, but the others are
> > mostly one-liners, so I'm not sure. I guess I'd have to see how you're
> > using them first.
> 
> So it looked like there were some functions in the common code for one
> GIC which had local test defined functions for the other. They should at
> least be consistent.

gicv3_read_iar and gicv3_write_eoir being common already is a product of
being sysreg wrappers, allowing for both arm32 and arm64 to use functions
of the same names, not because I wanted gicv3 to be inconsistent with
gicv2 (which uses MMIO and thus doesn't need wrappers)

> 
> For my use case I could do with a common:
> 
>   gic_enable

OK, I can extend gic_init() to initialize a 'struct gic_common_ops' that
includes an enable -> *_enable_defaults(void), ipi_send(int cpu),
read_iar(void), iar_irqnr(u32 iar), and write_eoi(u32 irqstat). And also
provide the wrappers gic_enable, gic_ipi_send(cpu), ...

>   gic_send_spi(cpu, irq)

I'll let you add this one to the new common ops struct :-)

>   gic_irq_ack() which returns the iar.

This one will be called read_iar.

Would that work for you, Alex?

Andre,

Would this also satisfy your needs for more common code?

Thanks,
drew

> 
> See:
> 
>   https://github.com/stsquad/kvm-unit-tests/blob/mttcg/current-tests-v6/arm/tcg-test.c#L113
> 
> >
> > Thanks,
> > drew
> 
> 
> --
> Alex Bennée
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-11 13:54           ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-11 13:54 UTC (permalink / raw)
  To: Alex Bennée
  Cc: peter.maydell, kvm, marc.zyngier, andre.przywara, qemu-devel,
	eric.auger, qemu-arm, pbonzini, kvmarm, christoffer.dall

On Fri, Nov 11, 2016 at 10:02:59AM +0000, Alex Bennée wrote:
> 
> Andrew Jones <drjones@redhat.com> writes:
> 
> > On Thu, Nov 10, 2016 at 07:53:58PM +0000, Alex Bennée wrote:
> > [...]
> >> > +struct gic gicv2 = {
> >> > +	.ipi = {
> >> > +		.enable = gicv2_enable_defaults,
> >> > +		.send_self = gicv2_ipi_send_self,
> >> > +		.send_tlist = gicv2_ipi_send_tlist,
> >> > +		.send_broadcast = gicv2_ipi_send_broadcast,
> >> > +	},
> >> > +	.read_iar = gicv2_read_iar,
> >> > +	.irqnr = gicv2_irqnr,
> >> > +	.write_eoi = gicv2_write_eoi,
> >> > +};
> >> > +
> >> > +struct gic gicv3 = {
> >> > +	.ipi = {
> >> > +		.enable = gicv3_enable_defaults,
> >> > +		.send_self = gicv3_ipi_send_self,
> >> > +		.send_tlist = gicv3_ipi_send_tlist,
> >> > +		.send_broadcast = gicv3_ipi_send_broadcast,
> >> > +	},
> >> > +	.read_iar = gicv3_read_iar,
> >> > +	.irqnr = gicv3_irqnr,
> >> > +	.write_eoi = gicv3_write_eoir,
> >> > +};
> >> > +
> >>
> >> So I was re-basing my kvm-unit-tests against your GIC rework and found
> >> myself copy and pasting a bunch of this into my tests that fire IRQs.
> >> That makes me think the abstraction should be in the library code so
> >> other tests can fiddle with sending IRQs.
> >>
> >> What do you think?
> >>
> >
> > I guess you mean moving the above two structs and their corresponding
> > functions (all which aren't already common) to lib/arm/ ? Or do you
> > just mean the one non-trivial function gicv3_ipi_send_tlist? I think
> > agree with gicv3_ipi_send_tlist getting shared, but the others are
> > mostly one-liners, so I'm not sure. I guess I'd have to see how you're
> > using them first.
> 
> So it looked like there were some functions in the common code for one
> GIC which had local test defined functions for the other. They should at
> least be consistent.

gicv3_read_iar and gicv3_write_eoir being common already is a product of
being sysreg wrappers, allowing for both arm32 and arm64 to use functions
of the same names, not because I wanted gicv3 to be inconsistent with
gicv2 (which uses MMIO and thus doesn't need wrappers)

> 
> For my use case I could do with a common:
> 
>   gic_enable

OK, I can extend gic_init() to initialize a 'struct gic_common_ops' that
includes an enable -> *_enable_defaults(void), ipi_send(int cpu),
read_iar(void), iar_irqnr(u32 iar), and write_eoi(u32 irqstat). And also
provide the wrappers gic_enable, gic_ipi_send(cpu), ...

>   gic_send_spi(cpu, irq)

I'll let you add this one to the new common ops struct :-)

>   gic_irq_ack() which returns the iar.

This one will be called read_iar.

Would that work for you, Alex?

Andre,

Would this also satisfy your needs for more common code?

Thanks,
drew

> 
> See:
> 
>   https://github.com/stsquad/kvm-unit-tests/blob/mttcg/current-tests-v6/arm/tcg-test.c#L113
> 
> >
> > Thanks,
> > drew
> 
> 
> --
> Alex Bennée
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support
  2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
@ 2016-11-11 14:52     ` Alex Bennée
  -1 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 14:52 UTC (permalink / raw)
  To: Andrew Jones
  Cc: kvm, marc.zyngier, andre.przywara, qemu-devel, qemu-arm,
	pbonzini, kvmarm


Andrew Jones <drjones@redhat.com> writes:

> Add some gicv2 support. This just adds init and enable
> functions, allowing unit tests to start messing with it.
>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v5: share/use only the modern register names [Andre]
> v4:
>  - only take defines from kernel we need now [Andre]
>  - moved defines to asm/gic.h so they'll be shared with v3 [drew]
>  - simplify enable by not caring if we reinit the distributor [drew]
>  - init all GICD_INT_DEF_PRI_X4 registers [Eric]
> ---
>  arm/Makefile.common    |  1 +
>  lib/arm/asm/gic-v2.h   | 34 ++++++++++++++++++++++
>  lib/arm/asm/gic.h      | 37 ++++++++++++++++++++++++
>  lib/arm/gic.c          | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  lib/arm64/asm/gic-v2.h |  1 +
>  lib/arm64/asm/gic.h    |  1 +
>  6 files changed, 150 insertions(+)
>  create mode 100644 lib/arm/asm/gic-v2.h
>  create mode 100644 lib/arm/asm/gic.h
>  create mode 100644 lib/arm/gic.c
>  create mode 100644 lib/arm64/asm/gic-v2.h
>  create mode 100644 lib/arm64/asm/gic.h
>
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index ccb554d9251a..41239c37e092 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
>  cflatobjs += lib/arm/bitops.o
>  cflatobjs += lib/arm/psci.o
>  cflatobjs += lib/arm/smp.o
> +cflatobjs += lib/arm/gic.o
>
>  libeabi = lib/arm/libeabi.a
>  eabiobjs = lib/arm/eabi_compat.o
> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
> new file mode 100644
> index 000000000000..c2d5fecd4886
> --- /dev/null
> +++ b/lib/arm/asm/gic-v2.h
> @@ -0,0 +1,34 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V2_H_
> +#define _ASMARM_GIC_V2_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
> +#endif
> +
> +#define GICD_ENABLE			0x1
> +#define GICC_ENABLE			0x1
> +
> +#ifndef __ASSEMBLY__
> +
> +struct gicv2_data {
> +	void *dist_base;
> +	void *cpu_base;
> +	unsigned int irq_nr;
> +};
> +extern struct gicv2_data gicv2_data;
> +
> +#define gicv2_dist_base()		(gicv2_data.dist_base)
> +#define gicv2_cpu_base()		(gicv2_data.cpu_base)
> +
> +extern int gicv2_init(void);
> +extern void gicv2_enable_defaults(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V2_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> new file mode 100644
> index 000000000000..d44e47bcf404
> --- /dev/null
> +++ b/lib/arm/asm/gic.h
> @@ -0,0 +1,37 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_H_
> +#define _ASMARM_GIC_H_
> +
> +#include <asm/gic-v2.h>
> +
> +#define GICD_CTLR			0x0000
> +#define GICD_TYPER			0x0004
> +#define GICD_ISENABLER			0x0100
> +#define GICD_IPRIORITYR			0x0400

Maybe GICD_ISENABLER_BASE and GICD_IPRIORITYR_BASE as they are the start
of a series of registers? Also what happened to the formatting?

> +
> +#define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1)
> * 32)
> +#define GICD_INT_EN_SET_SGI		0x0000ffff
> +#define GICD_INT_DEF_PRI_X4		0xa0a0a0a0

This doesn't seem to be used and I'm not sure what GICD_TYPER_IRQS it is
trying to achieve.

A comment above and here to make it clear we are talking about offsets
in the distributor and cpu register maps would aid confusion.

> +
> +#define GICC_CTLR			0x0000
> +#define GICC_PMR			0x0004
> +
> +#define GICC_INT_PRI_THRESHOLD		0xf0
> +
> +#ifndef __ASSEMBLY__
> +
> +/*
> + * gic_init will try to find all known gics, and then
> + * initialize the gic data for the one found.
> + * returns
> + *  0   : no gic was found
> + *  > 0 : the gic version of the gic found
> + */
> +extern int gic_init(void);

If we are going to make the library API agnostic I guess returning NULL
or an ops structure would be best here?

> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_H_ */
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> new file mode 100644
> index 000000000000..d655105e058b
> --- /dev/null
> +++ b/lib/arm/gic.c
> @@ -0,0 +1,76 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <devicetree.h>
> +#include <asm/gic.h>
> +#include <asm/io.h>
> +
> +struct gicv2_data gicv2_data;
> +
> +/*
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + */
> +static bool
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +{
> +	struct dt_pbus_reg reg;
> +	struct dt_device gic;
> +	struct dt_bus bus;
> +	int node, ret;
> +
> +	dt_bus_init_defaults(&bus);
> +	dt_device_init(&gic, &bus, NULL);
> +
> +	node = dt_device_find_compatible(&gic, compatible);
> +	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
> +
> +	if (node == -FDT_ERR_NOTFOUND)
> +		return false;
> +
> +	dt_device_bind_node(&gic, node);
> +
> +	ret = dt_pbus_translate(&gic, 0, &reg);
> +	assert(ret == 0);
> +	*base1 = ioremap(reg.addr, reg.size);
> +
> +	ret = dt_pbus_translate(&gic, 1, &reg);
> +	assert(ret == 0);
> +	*base2 = ioremap(reg.addr, reg.size);
> +
> +	return true;
> +}
> +
> +int gicv2_init(void)
> +{
> +	return gic_get_dt_bases("arm,cortex-a15-gic",
> +			&gicv2_data.dist_base, &gicv2_data.cpu_base);
> +}
> +
> +int gic_init(void)
> +{
> +	if (gicv2_init())
> +		return 2;
> +	return 0;
> +}
> +
> +void gicv2_enable_defaults(void)
> +{
> +	void *dist = gicv2_dist_base();
> +	void *cpu_base = gicv2_cpu_base();
> +	unsigned int i;
> +
> +	gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> +	if (gicv2_data.irq_nr > 1020)
> +		gicv2_data.irq_nr = 1020;
> +
> +	for (i = 0; i < gicv2_data.irq_nr; i += 4)
> +		writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
> +
> +	writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
> +	writel(GICD_ENABLE, dist + GICD_CTLR);
> +
> +	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> +	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> +}
> diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
> new file mode 100644
> index 000000000000..52226624a209
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v2.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v2.h"
> diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
> new file mode 100644
> index 000000000000..e5eb302a31b4
> --- /dev/null
> +++ b/lib/arm64/asm/gic.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic.h"


--
Alex Bennée
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support
@ 2016-11-11 14:52     ` Alex Bennée
  0 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 14:52 UTC (permalink / raw)
  To: Andrew Jones
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
	peter.maydell, marc.zyngier, eric.auger, christoffer.dall


Andrew Jones <drjones@redhat.com> writes:

> Add some gicv2 support. This just adds init and enable
> functions, allowing unit tests to start messing with it.
>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v5: share/use only the modern register names [Andre]
> v4:
>  - only take defines from kernel we need now [Andre]
>  - moved defines to asm/gic.h so they'll be shared with v3 [drew]
>  - simplify enable by not caring if we reinit the distributor [drew]
>  - init all GICD_INT_DEF_PRI_X4 registers [Eric]
> ---
>  arm/Makefile.common    |  1 +
>  lib/arm/asm/gic-v2.h   | 34 ++++++++++++++++++++++
>  lib/arm/asm/gic.h      | 37 ++++++++++++++++++++++++
>  lib/arm/gic.c          | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  lib/arm64/asm/gic-v2.h |  1 +
>  lib/arm64/asm/gic.h    |  1 +
>  6 files changed, 150 insertions(+)
>  create mode 100644 lib/arm/asm/gic-v2.h
>  create mode 100644 lib/arm/asm/gic.h
>  create mode 100644 lib/arm/gic.c
>  create mode 100644 lib/arm64/asm/gic-v2.h
>  create mode 100644 lib/arm64/asm/gic.h
>
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index ccb554d9251a..41239c37e092 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
>  cflatobjs += lib/arm/bitops.o
>  cflatobjs += lib/arm/psci.o
>  cflatobjs += lib/arm/smp.o
> +cflatobjs += lib/arm/gic.o
>
>  libeabi = lib/arm/libeabi.a
>  eabiobjs = lib/arm/eabi_compat.o
> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
> new file mode 100644
> index 000000000000..c2d5fecd4886
> --- /dev/null
> +++ b/lib/arm/asm/gic-v2.h
> @@ -0,0 +1,34 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V2_H_
> +#define _ASMARM_GIC_V2_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
> +#endif
> +
> +#define GICD_ENABLE			0x1
> +#define GICC_ENABLE			0x1
> +
> +#ifndef __ASSEMBLY__
> +
> +struct gicv2_data {
> +	void *dist_base;
> +	void *cpu_base;
> +	unsigned int irq_nr;
> +};
> +extern struct gicv2_data gicv2_data;
> +
> +#define gicv2_dist_base()		(gicv2_data.dist_base)
> +#define gicv2_cpu_base()		(gicv2_data.cpu_base)
> +
> +extern int gicv2_init(void);
> +extern void gicv2_enable_defaults(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V2_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> new file mode 100644
> index 000000000000..d44e47bcf404
> --- /dev/null
> +++ b/lib/arm/asm/gic.h
> @@ -0,0 +1,37 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_H_
> +#define _ASMARM_GIC_H_
> +
> +#include <asm/gic-v2.h>
> +
> +#define GICD_CTLR			0x0000
> +#define GICD_TYPER			0x0004
> +#define GICD_ISENABLER			0x0100
> +#define GICD_IPRIORITYR			0x0400

Maybe GICD_ISENABLER_BASE and GICD_IPRIORITYR_BASE as they are the start
of a series of registers? Also what happened to the formatting?

> +
> +#define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1)
> * 32)
> +#define GICD_INT_EN_SET_SGI		0x0000ffff
> +#define GICD_INT_DEF_PRI_X4		0xa0a0a0a0

This doesn't seem to be used and I'm not sure what GICD_TYPER_IRQS it is
trying to achieve.

A comment above and here to make it clear we are talking about offsets
in the distributor and cpu register maps would aid confusion.

> +
> +#define GICC_CTLR			0x0000
> +#define GICC_PMR			0x0004
> +
> +#define GICC_INT_PRI_THRESHOLD		0xf0
> +
> +#ifndef __ASSEMBLY__
> +
> +/*
> + * gic_init will try to find all known gics, and then
> + * initialize the gic data for the one found.
> + * returns
> + *  0   : no gic was found
> + *  > 0 : the gic version of the gic found
> + */
> +extern int gic_init(void);

If we are going to make the library API agnostic I guess returning NULL
or an ops structure would be best here?

> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_H_ */
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> new file mode 100644
> index 000000000000..d655105e058b
> --- /dev/null
> +++ b/lib/arm/gic.c
> @@ -0,0 +1,76 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <devicetree.h>
> +#include <asm/gic.h>
> +#include <asm/io.h>
> +
> +struct gicv2_data gicv2_data;
> +
> +/*
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + */
> +static bool
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +{
> +	struct dt_pbus_reg reg;
> +	struct dt_device gic;
> +	struct dt_bus bus;
> +	int node, ret;
> +
> +	dt_bus_init_defaults(&bus);
> +	dt_device_init(&gic, &bus, NULL);
> +
> +	node = dt_device_find_compatible(&gic, compatible);
> +	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
> +
> +	if (node == -FDT_ERR_NOTFOUND)
> +		return false;
> +
> +	dt_device_bind_node(&gic, node);
> +
> +	ret = dt_pbus_translate(&gic, 0, &reg);
> +	assert(ret == 0);
> +	*base1 = ioremap(reg.addr, reg.size);
> +
> +	ret = dt_pbus_translate(&gic, 1, &reg);
> +	assert(ret == 0);
> +	*base2 = ioremap(reg.addr, reg.size);
> +
> +	return true;
> +}
> +
> +int gicv2_init(void)
> +{
> +	return gic_get_dt_bases("arm,cortex-a15-gic",
> +			&gicv2_data.dist_base, &gicv2_data.cpu_base);
> +}
> +
> +int gic_init(void)
> +{
> +	if (gicv2_init())
> +		return 2;
> +	return 0;
> +}
> +
> +void gicv2_enable_defaults(void)
> +{
> +	void *dist = gicv2_dist_base();
> +	void *cpu_base = gicv2_cpu_base();
> +	unsigned int i;
> +
> +	gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> +	if (gicv2_data.irq_nr > 1020)
> +		gicv2_data.irq_nr = 1020;
> +
> +	for (i = 0; i < gicv2_data.irq_nr; i += 4)
> +		writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
> +
> +	writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
> +	writel(GICD_ENABLE, dist + GICD_CTLR);
> +
> +	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> +	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> +}
> diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
> new file mode 100644
> index 000000000000..52226624a209
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v2.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v2.h"
> diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
> new file mode 100644
> index 000000000000..e5eb302a31b4
> --- /dev/null
> +++ b/lib/arm64/asm/gic.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic.h"


--
Alex Bennée

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
  2016-11-11 13:54           ` Andrew Jones
  (?)
@ 2016-11-11 14:53           ` Alex Bennée
  2016-11-11 15:20               ` Andre Przywara
  -1 siblings, 1 reply; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 14:53 UTC (permalink / raw)
  To: Andrew Jones
  Cc: peter.maydell, kvm, marc.zyngier, andre.przywara, qemu-devel,
	eric.auger, qemu-arm, pbonzini, kvmarm, christoffer.dall


Andrew Jones <drjones@redhat.com> writes:

> On Fri, Nov 11, 2016 at 10:02:59AM +0000, Alex Bennée wrote:
>>
>> Andrew Jones <drjones@redhat.com> writes:
>>
>> > On Thu, Nov 10, 2016 at 07:53:58PM +0000, Alex Bennée wrote:
>> > [...]
>> >> > +struct gic gicv2 = {
>> >> > +	.ipi = {
>> >> > +		.enable = gicv2_enable_defaults,
>> >> > +		.send_self = gicv2_ipi_send_self,
>> >> > +		.send_tlist = gicv2_ipi_send_tlist,
>> >> > +		.send_broadcast = gicv2_ipi_send_broadcast,
>> >> > +	},
>> >> > +	.read_iar = gicv2_read_iar,
>> >> > +	.irqnr = gicv2_irqnr,
>> >> > +	.write_eoi = gicv2_write_eoi,
>> >> > +};
>> >> > +
>> >> > +struct gic gicv3 = {
>> >> > +	.ipi = {
>> >> > +		.enable = gicv3_enable_defaults,
>> >> > +		.send_self = gicv3_ipi_send_self,
>> >> > +		.send_tlist = gicv3_ipi_send_tlist,
>> >> > +		.send_broadcast = gicv3_ipi_send_broadcast,
>> >> > +	},
>> >> > +	.read_iar = gicv3_read_iar,
>> >> > +	.irqnr = gicv3_irqnr,
>> >> > +	.write_eoi = gicv3_write_eoir,
>> >> > +};
>> >> > +
>> >>
>> >> So I was re-basing my kvm-unit-tests against your GIC rework and found
>> >> myself copy and pasting a bunch of this into my tests that fire IRQs.
>> >> That makes me think the abstraction should be in the library code so
>> >> other tests can fiddle with sending IRQs.
>> >>
>> >> What do you think?
>> >>
>> >
>> > I guess you mean moving the above two structs and their corresponding
>> > functions (all which aren't already common) to lib/arm/ ? Or do you
>> > just mean the one non-trivial function gicv3_ipi_send_tlist? I think
>> > agree with gicv3_ipi_send_tlist getting shared, but the others are
>> > mostly one-liners, so I'm not sure. I guess I'd have to see how you're
>> > using them first.
>>
>> So it looked like there were some functions in the common code for one
>> GIC which had local test defined functions for the other. They should at
>> least be consistent.
>
> gicv3_read_iar and gicv3_write_eoir being common already is a product of
> being sysreg wrappers, allowing for both arm32 and arm64 to use functions
> of the same names, not because I wanted gicv3 to be inconsistent with
> gicv2 (which uses MMIO and thus doesn't need wrappers)
>
>>
>> For my use case I could do with a common:
>>
>>   gic_enable
>
> OK, I can extend gic_init() to initialize a 'struct gic_common_ops' that
> includes an enable -> *_enable_defaults(void), ipi_send(int cpu),
> read_iar(void), iar_irqnr(u32 iar), and write_eoi(u32 irqstat). And also
> provide the wrappers gic_enable, gic_ipi_send(cpu), ...
>
>>   gic_send_spi(cpu, irq)
>
> I'll let you add this one to the new common ops struct :-)
>
>>   gic_irq_ack() which returns the iar.
>
> This one will be called read_iar.
>
> Would that work for you, Alex?

Sounds good to me :-)

>
> Andre,
>
> Would this also satisfy your needs for more common code?
>
> Thanks,
> drew
>
>>
>> See:
>>
>>   https://github.com/stsquad/kvm-unit-tests/blob/mttcg/current-tests-v6/arm/tcg-test.c#L113
>>
>> >
>> > Thanks,
>> > drew
>>
>>
>> --
>> Alex Bennée
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
Alex Bennée

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 08/11] libcflat: add IS_ALIGNED() macro, and page sizes
  2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
@ 2016-11-11 15:02     ` Alex Bennée
  -1 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 15:02 UTC (permalink / raw)
  To: Andrew Jones
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
	peter.maydell, marc.zyngier, eric.auger, christoffer.dall,
	Peter Xu


Andrew Jones <drjones@redhat.com> writes:

> From: Peter Xu <peterx@redhat.com>
>
> These macros will be useful to do page alignment checks.
>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> [drew: also added SZ_64K]
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> ---
>  lib/libcflat.h | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/lib/libcflat.h b/lib/libcflat.h
> index 82005f5d014f..143fc53061fe 100644
> --- a/lib/libcflat.h
> +++ b/lib/libcflat.h
> @@ -33,6 +33,12 @@
>  #define __ALIGN_MASK(x, mask)	(((x) + (mask)) & ~(mask))
>  #define __ALIGN(x, a)		__ALIGN_MASK(x, (typeof(x))(a) - 1)
>  #define ALIGN(x, a)		__ALIGN((x), (a))
> +#define IS_ALIGNED(x, a)	(((x) & ((typeof(x))(a) - 1)) == 0)
> +
> +#define SZ_4K			(0x1000)
> +#define SZ_64K			(0x10000)
> +#define SZ_2M			(0x200000)
> +#define SZ_1G			(0x40000000)

We don't seem to use IS_ALIGNED, or in fact anything apart from SZ_64K
(which is multiplied by 2 anyway) so I'm not sure if this patch is worth
it for this series.

Stylistically I thought (1 << foo) was preferred for setting of
individual bits? Otherwise I'd be tempted to line the values up with
zero padding to make it easier to read the bit position.

>
>  typedef uint8_t		u8;
>  typedef int8_t		s8;


--
Alex Bennée

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 08/11] libcflat: add IS_ALIGNED() macro, and page sizes
@ 2016-11-11 15:02     ` Alex Bennée
  0 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 15:02 UTC (permalink / raw)
  To: Andrew Jones
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
	peter.maydell, marc.zyngier, eric.auger, christoffer.dall,
	Peter Xu


Andrew Jones <drjones@redhat.com> writes:

> From: Peter Xu <peterx@redhat.com>
>
> These macros will be useful to do page alignment checks.
>
> Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> [drew: also added SZ_64K]
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> ---
>  lib/libcflat.h | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/lib/libcflat.h b/lib/libcflat.h
> index 82005f5d014f..143fc53061fe 100644
> --- a/lib/libcflat.h
> +++ b/lib/libcflat.h
> @@ -33,6 +33,12 @@
>  #define __ALIGN_MASK(x, mask)	(((x) + (mask)) & ~(mask))
>  #define __ALIGN(x, a)		__ALIGN_MASK(x, (typeof(x))(a) - 1)
>  #define ALIGN(x, a)		__ALIGN((x), (a))
> +#define IS_ALIGNED(x, a)	(((x) & ((typeof(x))(a) - 1)) == 0)
> +
> +#define SZ_4K			(0x1000)
> +#define SZ_64K			(0x10000)
> +#define SZ_2M			(0x200000)
> +#define SZ_1G			(0x40000000)

We don't seem to use IS_ALIGNED, or in fact anything apart from SZ_64K
(which is multiplied by 2 anyway) so I'm not sure if this patch is worth
it for this series.

Stylistically I thought (1 << foo) was preferred for setting of
individual bits? Otherwise I'd be tempted to line the values up with
zero padding to make it easier to read the bit position.

>
>  typedef uint8_t		u8;
>  typedef int8_t		s8;


--
Alex Bennée

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support
  2016-11-11 14:52     ` [Qemu-devel] " Alex Bennée
@ 2016-11-11 15:17       ` Andre Przywara
  -1 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 15:17 UTC (permalink / raw)
  To: Alex Bennée, Andrew Jones
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, peter.maydell,
	marc.zyngier, eric.auger, christoffer.dall

Hi,

On 11/11/16 14:52, Alex Bennée wrote:
> 
> Andrew Jones <drjones@redhat.com> writes:
> 
>> Add some gicv2 support. This just adds init and enable
>> functions, allowing unit tests to start messing with it.
>>
>> Signed-off-by: Andrew Jones <drjones@redhat.com>
>>
>> ---
>> v5: share/use only the modern register names [Andre]
>> v4:
>>  - only take defines from kernel we need now [Andre]
>>  - moved defines to asm/gic.h so they'll be shared with v3 [drew]
>>  - simplify enable by not caring if we reinit the distributor [drew]
>>  - init all GICD_INT_DEF_PRI_X4 registers [Eric]
>> ---
>>  arm/Makefile.common    |  1 +
>>  lib/arm/asm/gic-v2.h   | 34 ++++++++++++++++++++++
>>  lib/arm/asm/gic.h      | 37 ++++++++++++++++++++++++
>>  lib/arm/gic.c          | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>  lib/arm64/asm/gic-v2.h |  1 +
>>  lib/arm64/asm/gic.h    |  1 +
>>  6 files changed, 150 insertions(+)
>>  create mode 100644 lib/arm/asm/gic-v2.h
>>  create mode 100644 lib/arm/asm/gic.h
>>  create mode 100644 lib/arm/gic.c
>>  create mode 100644 lib/arm64/asm/gic-v2.h
>>  create mode 100644 lib/arm64/asm/gic.h
>>
>> diff --git a/arm/Makefile.common b/arm/Makefile.common
>> index ccb554d9251a..41239c37e092 100644
>> --- a/arm/Makefile.common
>> +++ b/arm/Makefile.common
>> @@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
>>  cflatobjs += lib/arm/bitops.o
>>  cflatobjs += lib/arm/psci.o
>>  cflatobjs += lib/arm/smp.o
>> +cflatobjs += lib/arm/gic.o
>>
>>  libeabi = lib/arm/libeabi.a
>>  eabiobjs = lib/arm/eabi_compat.o
>> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
>> new file mode 100644
>> index 000000000000..c2d5fecd4886
>> --- /dev/null
>> +++ b/lib/arm/asm/gic-v2.h
>> @@ -0,0 +1,34 @@
>> +/*
>> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
>> + *
>> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#ifndef _ASMARM_GIC_V2_H_
>> +#define _ASMARM_GIC_V2_H_
>> +
>> +#ifndef _ASMARM_GIC_H_
>> +#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
>> +#endif
>> +
>> +#define GICD_ENABLE			0x1
>> +#define GICC_ENABLE			0x1
>> +
>> +#ifndef __ASSEMBLY__
>> +
>> +struct gicv2_data {
>> +	void *dist_base;
>> +	void *cpu_base;
>> +	unsigned int irq_nr;
>> +};
>> +extern struct gicv2_data gicv2_data;
>> +
>> +#define gicv2_dist_base()		(gicv2_data.dist_base)
>> +#define gicv2_cpu_base()		(gicv2_data.cpu_base)
>> +
>> +extern int gicv2_init(void);
>> +extern void gicv2_enable_defaults(void);
>> +
>> +#endif /* !__ASSEMBLY__ */
>> +#endif /* _ASMARM_GIC_V2_H_ */
>> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
>> new file mode 100644
>> index 000000000000..d44e47bcf404
>> --- /dev/null
>> +++ b/lib/arm/asm/gic.h
>> @@ -0,0 +1,37 @@
>> +/*
>> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#ifndef _ASMARM_GIC_H_
>> +#define _ASMARM_GIC_H_
>> +
>> +#include <asm/gic-v2.h>
>> +
>> +#define GICD_CTLR			0x0000
>> +#define GICD_TYPER			0x0004
>> +#define GICD_ISENABLER			0x0100
>> +#define GICD_IPRIORITYR			0x0400
> 
> Maybe GICD_ISENABLER_BASE and GICD_IPRIORITYR_BASE as they are the start
> of a series of registers?

We should keep the naming consistent to both the spec and the Linux headers.

> Also what happened to the formatting?

Isn't that the usual diff artifact caused by prepending a single
character? Which moves the tab stops, also the mailer adding quotation
characters?

>> +
>> +#define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1)
>> * 32)
>> +#define GICD_INT_EN_SET_SGI		0x0000ffff
>> +#define GICD_INT_DEF_PRI_X4		0xa0a0a0a0
> 
> This doesn't seem to be used and I'm not sure what GICD_TYPER_IRQS it is
> trying to achieve.

The idea is to calculate the number of implemented SPIs. But I am not a
big fan of copying a macro from the emulation code base to the test code.

Cheers,
Andre.

> A comment above and here to make it clear we are talking about offsets
> in the distributor and cpu register maps would aid confusion.
> 
>> +
>> +#define GICC_CTLR			0x0000
>> +#define GICC_PMR			0x0004
>> +
>> +#define GICC_INT_PRI_THRESHOLD		0xf0
>> +
>> +#ifndef __ASSEMBLY__
>> +
>> +/*
>> + * gic_init will try to find all known gics, and then
>> + * initialize the gic data for the one found.
>> + * returns
>> + *  0   : no gic was found
>> + *  > 0 : the gic version of the gic found
>> + */
>> +extern int gic_init(void);
> 
> If we are going to make the library API agnostic I guess returning NULL
> or an ops structure would be best here?
> 
>> +
>> +#endif /* !__ASSEMBLY__ */
>> +#endif /* _ASMARM_GIC_H_ */
>> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
>> new file mode 100644
>> index 000000000000..d655105e058b
>> --- /dev/null
>> +++ b/lib/arm/gic.c
>> @@ -0,0 +1,76 @@
>> +/*
>> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#include <devicetree.h>
>> +#include <asm/gic.h>
>> +#include <asm/io.h>
>> +
>> +struct gicv2_data gicv2_data;
>> +
>> +/*
>> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
>> + */
>> +static bool
>> +gic_get_dt_bases(const char *compatible, void **base1, void **base2)
>> +{
>> +	struct dt_pbus_reg reg;
>> +	struct dt_device gic;
>> +	struct dt_bus bus;
>> +	int node, ret;
>> +
>> +	dt_bus_init_defaults(&bus);
>> +	dt_device_init(&gic, &bus, NULL);
>> +
>> +	node = dt_device_find_compatible(&gic, compatible);
>> +	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
>> +
>> +	if (node == -FDT_ERR_NOTFOUND)
>> +		return false;
>> +
>> +	dt_device_bind_node(&gic, node);
>> +
>> +	ret = dt_pbus_translate(&gic, 0, &reg);
>> +	assert(ret == 0);
>> +	*base1 = ioremap(reg.addr, reg.size);
>> +
>> +	ret = dt_pbus_translate(&gic, 1, &reg);
>> +	assert(ret == 0);
>> +	*base2 = ioremap(reg.addr, reg.size);
>> +
>> +	return true;
>> +}
>> +
>> +int gicv2_init(void)
>> +{
>> +	return gic_get_dt_bases("arm,cortex-a15-gic",
>> +			&gicv2_data.dist_base, &gicv2_data.cpu_base);
>> +}
>> +
>> +int gic_init(void)
>> +{
>> +	if (gicv2_init())
>> +		return 2;
>> +	return 0;
>> +}
>> +
>> +void gicv2_enable_defaults(void)
>> +{
>> +	void *dist = gicv2_dist_base();
>> +	void *cpu_base = gicv2_cpu_base();
>> +	unsigned int i;
>> +
>> +	gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
>> +	if (gicv2_data.irq_nr > 1020)
>> +		gicv2_data.irq_nr = 1020;
>> +
>> +	for (i = 0; i < gicv2_data.irq_nr; i += 4)
>> +		writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
>> +
>> +	writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
>> +	writel(GICD_ENABLE, dist + GICD_CTLR);
>> +
>> +	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
>> +	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
>> +}
>> diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
>> new file mode 100644
>> index 000000000000..52226624a209
>> --- /dev/null
>> +++ b/lib/arm64/asm/gic-v2.h
>> @@ -0,0 +1 @@
>> +#include "../../arm/asm/gic-v2.h"
>> diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
>> new file mode 100644
>> index 000000000000..e5eb302a31b4
>> --- /dev/null
>> +++ b/lib/arm64/asm/gic.h
>> @@ -0,0 +1 @@
>> +#include "../../arm/asm/gic.h"
> 
> 
> --
> Alex Bennée
> 

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support
@ 2016-11-11 15:17       ` Andre Przywara
  0 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 15:17 UTC (permalink / raw)
  To: Alex Bennée, Andrew Jones
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, peter.maydell,
	marc.zyngier, eric.auger, christoffer.dall

Hi,

On 11/11/16 14:52, Alex Bennée wrote:
> 
> Andrew Jones <drjones@redhat.com> writes:
> 
>> Add some gicv2 support. This just adds init and enable
>> functions, allowing unit tests to start messing with it.
>>
>> Signed-off-by: Andrew Jones <drjones@redhat.com>
>>
>> ---
>> v5: share/use only the modern register names [Andre]
>> v4:
>>  - only take defines from kernel we need now [Andre]
>>  - moved defines to asm/gic.h so they'll be shared with v3 [drew]
>>  - simplify enable by not caring if we reinit the distributor [drew]
>>  - init all GICD_INT_DEF_PRI_X4 registers [Eric]
>> ---
>>  arm/Makefile.common    |  1 +
>>  lib/arm/asm/gic-v2.h   | 34 ++++++++++++++++++++++
>>  lib/arm/asm/gic.h      | 37 ++++++++++++++++++++++++
>>  lib/arm/gic.c          | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>  lib/arm64/asm/gic-v2.h |  1 +
>>  lib/arm64/asm/gic.h    |  1 +
>>  6 files changed, 150 insertions(+)
>>  create mode 100644 lib/arm/asm/gic-v2.h
>>  create mode 100644 lib/arm/asm/gic.h
>>  create mode 100644 lib/arm/gic.c
>>  create mode 100644 lib/arm64/asm/gic-v2.h
>>  create mode 100644 lib/arm64/asm/gic.h
>>
>> diff --git a/arm/Makefile.common b/arm/Makefile.common
>> index ccb554d9251a..41239c37e092 100644
>> --- a/arm/Makefile.common
>> +++ b/arm/Makefile.common
>> @@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
>>  cflatobjs += lib/arm/bitops.o
>>  cflatobjs += lib/arm/psci.o
>>  cflatobjs += lib/arm/smp.o
>> +cflatobjs += lib/arm/gic.o
>>
>>  libeabi = lib/arm/libeabi.a
>>  eabiobjs = lib/arm/eabi_compat.o
>> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
>> new file mode 100644
>> index 000000000000..c2d5fecd4886
>> --- /dev/null
>> +++ b/lib/arm/asm/gic-v2.h
>> @@ -0,0 +1,34 @@
>> +/*
>> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
>> + *
>> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#ifndef _ASMARM_GIC_V2_H_
>> +#define _ASMARM_GIC_V2_H_
>> +
>> +#ifndef _ASMARM_GIC_H_
>> +#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
>> +#endif
>> +
>> +#define GICD_ENABLE			0x1
>> +#define GICC_ENABLE			0x1
>> +
>> +#ifndef __ASSEMBLY__
>> +
>> +struct gicv2_data {
>> +	void *dist_base;
>> +	void *cpu_base;
>> +	unsigned int irq_nr;
>> +};
>> +extern struct gicv2_data gicv2_data;
>> +
>> +#define gicv2_dist_base()		(gicv2_data.dist_base)
>> +#define gicv2_cpu_base()		(gicv2_data.cpu_base)
>> +
>> +extern int gicv2_init(void);
>> +extern void gicv2_enable_defaults(void);
>> +
>> +#endif /* !__ASSEMBLY__ */
>> +#endif /* _ASMARM_GIC_V2_H_ */
>> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
>> new file mode 100644
>> index 000000000000..d44e47bcf404
>> --- /dev/null
>> +++ b/lib/arm/asm/gic.h
>> @@ -0,0 +1,37 @@
>> +/*
>> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#ifndef _ASMARM_GIC_H_
>> +#define _ASMARM_GIC_H_
>> +
>> +#include <asm/gic-v2.h>
>> +
>> +#define GICD_CTLR			0x0000
>> +#define GICD_TYPER			0x0004
>> +#define GICD_ISENABLER			0x0100
>> +#define GICD_IPRIORITYR			0x0400
> 
> Maybe GICD_ISENABLER_BASE and GICD_IPRIORITYR_BASE as they are the start
> of a series of registers?

We should keep the naming consistent to both the spec and the Linux headers.

> Also what happened to the formatting?

Isn't that the usual diff artifact caused by prepending a single
character? Which moves the tab stops, also the mailer adding quotation
characters?

>> +
>> +#define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1)
>> * 32)
>> +#define GICD_INT_EN_SET_SGI		0x0000ffff
>> +#define GICD_INT_DEF_PRI_X4		0xa0a0a0a0
> 
> This doesn't seem to be used and I'm not sure what GICD_TYPER_IRQS it is
> trying to achieve.

The idea is to calculate the number of implemented SPIs. But I am not a
big fan of copying a macro from the emulation code base to the test code.

Cheers,
Andre.

> A comment above and here to make it clear we are talking about offsets
> in the distributor and cpu register maps would aid confusion.
> 
>> +
>> +#define GICC_CTLR			0x0000
>> +#define GICC_PMR			0x0004
>> +
>> +#define GICC_INT_PRI_THRESHOLD		0xf0
>> +
>> +#ifndef __ASSEMBLY__
>> +
>> +/*
>> + * gic_init will try to find all known gics, and then
>> + * initialize the gic data for the one found.
>> + * returns
>> + *  0   : no gic was found
>> + *  > 0 : the gic version of the gic found
>> + */
>> +extern int gic_init(void);
> 
> If we are going to make the library API agnostic I guess returning NULL
> or an ops structure would be best here?
> 
>> +
>> +#endif /* !__ASSEMBLY__ */
>> +#endif /* _ASMARM_GIC_H_ */
>> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
>> new file mode 100644
>> index 000000000000..d655105e058b
>> --- /dev/null
>> +++ b/lib/arm/gic.c
>> @@ -0,0 +1,76 @@
>> +/*
>> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#include <devicetree.h>
>> +#include <asm/gic.h>
>> +#include <asm/io.h>
>> +
>> +struct gicv2_data gicv2_data;
>> +
>> +/*
>> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
>> + */
>> +static bool
>> +gic_get_dt_bases(const char *compatible, void **base1, void **base2)
>> +{
>> +	struct dt_pbus_reg reg;
>> +	struct dt_device gic;
>> +	struct dt_bus bus;
>> +	int node, ret;
>> +
>> +	dt_bus_init_defaults(&bus);
>> +	dt_device_init(&gic, &bus, NULL);
>> +
>> +	node = dt_device_find_compatible(&gic, compatible);
>> +	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
>> +
>> +	if (node == -FDT_ERR_NOTFOUND)
>> +		return false;
>> +
>> +	dt_device_bind_node(&gic, node);
>> +
>> +	ret = dt_pbus_translate(&gic, 0, &reg);
>> +	assert(ret == 0);
>> +	*base1 = ioremap(reg.addr, reg.size);
>> +
>> +	ret = dt_pbus_translate(&gic, 1, &reg);
>> +	assert(ret == 0);
>> +	*base2 = ioremap(reg.addr, reg.size);
>> +
>> +	return true;
>> +}
>> +
>> +int gicv2_init(void)
>> +{
>> +	return gic_get_dt_bases("arm,cortex-a15-gic",
>> +			&gicv2_data.dist_base, &gicv2_data.cpu_base);
>> +}
>> +
>> +int gic_init(void)
>> +{
>> +	if (gicv2_init())
>> +		return 2;
>> +	return 0;
>> +}
>> +
>> +void gicv2_enable_defaults(void)
>> +{
>> +	void *dist = gicv2_dist_base();
>> +	void *cpu_base = gicv2_cpu_base();
>> +	unsigned int i;
>> +
>> +	gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
>> +	if (gicv2_data.irq_nr > 1020)
>> +		gicv2_data.irq_nr = 1020;
>> +
>> +	for (i = 0; i < gicv2_data.irq_nr; i += 4)
>> +		writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
>> +
>> +	writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
>> +	writel(GICD_ENABLE, dist + GICD_CTLR);
>> +
>> +	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
>> +	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
>> +}
>> diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
>> new file mode 100644
>> index 000000000000..52226624a209
>> --- /dev/null
>> +++ b/lib/arm64/asm/gic-v2.h
>> @@ -0,0 +1 @@
>> +#include "../../arm/asm/gic-v2.h"
>> diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
>> new file mode 100644
>> index 000000000000..e5eb302a31b4
>> --- /dev/null
>> +++ b/lib/arm64/asm/gic.h
>> @@ -0,0 +1 @@
>> +#include "../../arm/asm/gic.h"
> 
> 
> --
> Alex Bennée
> 

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
  2016-11-11 14:53           ` Alex Bennée
@ 2016-11-11 15:20               ` Andre Przywara
  0 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 15:20 UTC (permalink / raw)
  To: Alex Bennée, Andrew Jones
  Cc: kvm, marc.zyngier, qemu-devel, qemu-arm, pbonzini, kvmarm

Hi,

On 11/11/16 14:53, Alex Bennée wrote:
> 
> Andrew Jones <drjones@redhat.com> writes:
> 
>> On Fri, Nov 11, 2016 at 10:02:59AM +0000, Alex Bennée wrote:
>>>
>>> Andrew Jones <drjones@redhat.com> writes:
>>>
>>>> On Thu, Nov 10, 2016 at 07:53:58PM +0000, Alex Bennée wrote:
>>>> [...]
>>>>>> +struct gic gicv2 = {
>>>>>> +	.ipi = {
>>>>>> +		.enable = gicv2_enable_defaults,
>>>>>> +		.send_self = gicv2_ipi_send_self,
>>>>>> +		.send_tlist = gicv2_ipi_send_tlist,
>>>>>> +		.send_broadcast = gicv2_ipi_send_broadcast,
>>>>>> +	},
>>>>>> +	.read_iar = gicv2_read_iar,
>>>>>> +	.irqnr = gicv2_irqnr,
>>>>>> +	.write_eoi = gicv2_write_eoi,
>>>>>> +};
>>>>>> +
>>>>>> +struct gic gicv3 = {
>>>>>> +	.ipi = {
>>>>>> +		.enable = gicv3_enable_defaults,
>>>>>> +		.send_self = gicv3_ipi_send_self,
>>>>>> +		.send_tlist = gicv3_ipi_send_tlist,
>>>>>> +		.send_broadcast = gicv3_ipi_send_broadcast,
>>>>>> +	},
>>>>>> +	.read_iar = gicv3_read_iar,
>>>>>> +	.irqnr = gicv3_irqnr,
>>>>>> +	.write_eoi = gicv3_write_eoir,
>>>>>> +};
>>>>>> +
>>>>>
>>>>> So I was re-basing my kvm-unit-tests against your GIC rework and found
>>>>> myself copy and pasting a bunch of this into my tests that fire IRQs.
>>>>> That makes me think the abstraction should be in the library code so
>>>>> other tests can fiddle with sending IRQs.
>>>>>
>>>>> What do you think?
>>>>>
>>>>
>>>> I guess you mean moving the above two structs and their corresponding
>>>> functions (all which aren't already common) to lib/arm/ ? Or do you
>>>> just mean the one non-trivial function gicv3_ipi_send_tlist? I think
>>>> agree with gicv3_ipi_send_tlist getting shared, but the others are
>>>> mostly one-liners, so I'm not sure. I guess I'd have to see how you're
>>>> using them first.
>>>
>>> So it looked like there were some functions in the common code for one
>>> GIC which had local test defined functions for the other. They should at
>>> least be consistent.
>>
>> gicv3_read_iar and gicv3_write_eoir being common already is a product of
>> being sysreg wrappers, allowing for both arm32 and arm64 to use functions
>> of the same names, not because I wanted gicv3 to be inconsistent with
>> gicv2 (which uses MMIO and thus doesn't need wrappers)
>>
>>>
>>> For my use case I could do with a common:
>>>
>>>   gic_enable
>>
>> OK, I can extend gic_init() to initialize a 'struct gic_common_ops' that
>> includes an enable -> *_enable_defaults(void), ipi_send(int cpu),
>> read_iar(void), iar_irqnr(u32 iar), and write_eoi(u32 irqstat). And also
>> provide the wrappers gic_enable, gic_ipi_send(cpu), ...
>>
>>>   gic_send_spi(cpu, irq)
>>
>> I'll let you add this one to the new common ops struct :-)
>>
>>>   gic_irq_ack() which returns the iar.
>>
>> This one will be called read_iar.
>>
>> Would that work for you, Alex?
> 
> Sounds good to me :-)
> 
>>
>> Andre,
>>
>> Would this also satisfy your needs for more common code?

TBH I haven't look deeply enough to give an educated answer. I just
wanted to +1 Alex for the general direction. So I guess it's OK. ;-)

I guess we may need some refactoring later anyway, so any missing pieces
can be added/refactored later once we exactly know what we need.

Cheers,
Andre.
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test
@ 2016-11-11 15:20               ` Andre Przywara
  0 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 15:20 UTC (permalink / raw)
  To: Alex Bennée, Andrew Jones
  Cc: peter.maydell, kvm, marc.zyngier, qemu-devel, eric.auger,
	qemu-arm, pbonzini, kvmarm, christoffer.dall

Hi,

On 11/11/16 14:53, Alex Bennée wrote:
> 
> Andrew Jones <drjones@redhat.com> writes:
> 
>> On Fri, Nov 11, 2016 at 10:02:59AM +0000, Alex Bennée wrote:
>>>
>>> Andrew Jones <drjones@redhat.com> writes:
>>>
>>>> On Thu, Nov 10, 2016 at 07:53:58PM +0000, Alex Bennée wrote:
>>>> [...]
>>>>>> +struct gic gicv2 = {
>>>>>> +	.ipi = {
>>>>>> +		.enable = gicv2_enable_defaults,
>>>>>> +		.send_self = gicv2_ipi_send_self,
>>>>>> +		.send_tlist = gicv2_ipi_send_tlist,
>>>>>> +		.send_broadcast = gicv2_ipi_send_broadcast,
>>>>>> +	},
>>>>>> +	.read_iar = gicv2_read_iar,
>>>>>> +	.irqnr = gicv2_irqnr,
>>>>>> +	.write_eoi = gicv2_write_eoi,
>>>>>> +};
>>>>>> +
>>>>>> +struct gic gicv3 = {
>>>>>> +	.ipi = {
>>>>>> +		.enable = gicv3_enable_defaults,
>>>>>> +		.send_self = gicv3_ipi_send_self,
>>>>>> +		.send_tlist = gicv3_ipi_send_tlist,
>>>>>> +		.send_broadcast = gicv3_ipi_send_broadcast,
>>>>>> +	},
>>>>>> +	.read_iar = gicv3_read_iar,
>>>>>> +	.irqnr = gicv3_irqnr,
>>>>>> +	.write_eoi = gicv3_write_eoir,
>>>>>> +};
>>>>>> +
>>>>>
>>>>> So I was re-basing my kvm-unit-tests against your GIC rework and found
>>>>> myself copy and pasting a bunch of this into my tests that fire IRQs.
>>>>> That makes me think the abstraction should be in the library code so
>>>>> other tests can fiddle with sending IRQs.
>>>>>
>>>>> What do you think?
>>>>>
>>>>
>>>> I guess you mean moving the above two structs and their corresponding
>>>> functions (all which aren't already common) to lib/arm/ ? Or do you
>>>> just mean the one non-trivial function gicv3_ipi_send_tlist? I think
>>>> agree with gicv3_ipi_send_tlist getting shared, but the others are
>>>> mostly one-liners, so I'm not sure. I guess I'd have to see how you're
>>>> using them first.
>>>
>>> So it looked like there were some functions in the common code for one
>>> GIC which had local test defined functions for the other. They should at
>>> least be consistent.
>>
>> gicv3_read_iar and gicv3_write_eoir being common already is a product of
>> being sysreg wrappers, allowing for both arm32 and arm64 to use functions
>> of the same names, not because I wanted gicv3 to be inconsistent with
>> gicv2 (which uses MMIO and thus doesn't need wrappers)
>>
>>>
>>> For my use case I could do with a common:
>>>
>>>   gic_enable
>>
>> OK, I can extend gic_init() to initialize a 'struct gic_common_ops' that
>> includes an enable -> *_enable_defaults(void), ipi_send(int cpu),
>> read_iar(void), iar_irqnr(u32 iar), and write_eoi(u32 irqstat). And also
>> provide the wrappers gic_enable, gic_ipi_send(cpu), ...
>>
>>>   gic_send_spi(cpu, irq)
>>
>> I'll let you add this one to the new common ops struct :-)
>>
>>>   gic_irq_ack() which returns the iar.
>>
>> This one will be called read_iar.
>>
>> Would that work for you, Alex?
> 
> Sounds good to me :-)
> 
>>
>> Andre,
>>
>> Would this also satisfy your needs for more common code?

TBH I haven't look deeply enough to give an educated answer. I just
wanted to +1 Alex for the general direction. So I guess it's OK. ;-)

I guess we may need some refactoring later anyway, so any missing pieces
can be added/refactored later once we exactly know what we need.

Cheers,
Andre.

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support
  2016-11-11 14:52     ` [Qemu-devel] " Alex Bennée
  (?)
  (?)
@ 2016-11-11 15:24     ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-11 15:24 UTC (permalink / raw)
  To: Alex Bennée
  Cc: peter.maydell, kvm, marc.zyngier, andre.przywara, qemu-devel,
	eric.auger, qemu-arm, pbonzini, kvmarm, christoffer.dall

On Fri, Nov 11, 2016 at 02:52:40PM +0000, Alex Bennée wrote:
> 
> Andrew Jones <drjones@redhat.com> writes:
> 
> > Add some gicv2 support. This just adds init and enable
> > functions, allowing unit tests to start messing with it.
> >
> > Signed-off-by: Andrew Jones <drjones@redhat.com>
> >
> > ---
> > v5: share/use only the modern register names [Andre]
> > v4:
> >  - only take defines from kernel we need now [Andre]
> >  - moved defines to asm/gic.h so they'll be shared with v3 [drew]
> >  - simplify enable by not caring if we reinit the distributor [drew]
> >  - init all GICD_INT_DEF_PRI_X4 registers [Eric]
> > ---
> >  arm/Makefile.common    |  1 +
> >  lib/arm/asm/gic-v2.h   | 34 ++++++++++++++++++++++
> >  lib/arm/asm/gic.h      | 37 ++++++++++++++++++++++++
> >  lib/arm/gic.c          | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
> >  lib/arm64/asm/gic-v2.h |  1 +
> >  lib/arm64/asm/gic.h    |  1 +
> >  6 files changed, 150 insertions(+)
> >  create mode 100644 lib/arm/asm/gic-v2.h
> >  create mode 100644 lib/arm/asm/gic.h
> >  create mode 100644 lib/arm/gic.c
> >  create mode 100644 lib/arm64/asm/gic-v2.h
> >  create mode 100644 lib/arm64/asm/gic.h
> >
> > diff --git a/arm/Makefile.common b/arm/Makefile.common
> > index ccb554d9251a..41239c37e092 100644
> > --- a/arm/Makefile.common
> > +++ b/arm/Makefile.common
> > @@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
> >  cflatobjs += lib/arm/bitops.o
> >  cflatobjs += lib/arm/psci.o
> >  cflatobjs += lib/arm/smp.o
> > +cflatobjs += lib/arm/gic.o
> >
> >  libeabi = lib/arm/libeabi.a
> >  eabiobjs = lib/arm/eabi_compat.o
> > diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
> > new file mode 100644
> > index 000000000000..c2d5fecd4886
> > --- /dev/null
> > +++ b/lib/arm/asm/gic-v2.h
> > @@ -0,0 +1,34 @@
> > +/*
> > + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
> > + *
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#ifndef _ASMARM_GIC_V2_H_
> > +#define _ASMARM_GIC_V2_H_
> > +
> > +#ifndef _ASMARM_GIC_H_
> > +#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
> > +#endif
> > +
> > +#define GICD_ENABLE			0x1
> > +#define GICC_ENABLE			0x1
> > +
> > +#ifndef __ASSEMBLY__
> > +
> > +struct gicv2_data {
> > +	void *dist_base;
> > +	void *cpu_base;
> > +	unsigned int irq_nr;
> > +};
> > +extern struct gicv2_data gicv2_data;
> > +
> > +#define gicv2_dist_base()		(gicv2_data.dist_base)
> > +#define gicv2_cpu_base()		(gicv2_data.cpu_base)
> > +
> > +extern int gicv2_init(void);
> > +extern void gicv2_enable_defaults(void);
> > +
> > +#endif /* !__ASSEMBLY__ */
> > +#endif /* _ASMARM_GIC_V2_H_ */
> > diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> > new file mode 100644
> > index 000000000000..d44e47bcf404
> > --- /dev/null
> > +++ b/lib/arm/asm/gic.h
> > @@ -0,0 +1,37 @@
> > +/*
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#ifndef _ASMARM_GIC_H_
> > +#define _ASMARM_GIC_H_
> > +
> > +#include <asm/gic-v2.h>
> > +
> > +#define GICD_CTLR			0x0000
> > +#define GICD_TYPER			0x0004
> > +#define GICD_ISENABLER			0x0100
> > +#define GICD_IPRIORITYR			0x0400
> 
> Maybe GICD_ISENABLER_BASE and GICD_IPRIORITYR_BASE as they are the start
> of a series of registers? Also what happened to the formatting?

These names match the kernel's naming (and the ARM spec) The formatting
is fine after applying, diff just bumped them over prepending the '+'

> 
> > +
> > +#define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1)
> > * 32)
> > +#define GICD_INT_EN_SET_SGI		0x0000ffff
> > +#define GICD_INT_DEF_PRI_X4		0xa0a0a0a0
> 
> This doesn't seem to be used and I'm not sure what GICD_TYPER_IRQS it is
> trying to achieve.

All used in lib/arm/gic.c. See the spec for how to interpret the
ITLinesNumber field of GICD_TYPER

> 
> A comment above and here to make it clear we are talking about offsets
> in the distributor and cpu register maps would aid confusion.

Better not add them then, as I wouldn't want to *aid* confusion :-)

I can add some comments (I do add a couple for gicv3), but I think
here it's pretty clear by the name prefix (GICD vs. GICC).

> 
> > +
> > +#define GICC_CTLR			0x0000
> > +#define GICC_PMR			0x0004
> > +
> > +#define GICC_INT_PRI_THRESHOLD		0xf0
> > +
> > +#ifndef __ASSEMBLY__
> > +
> > +/*
> > + * gic_init will try to find all known gics, and then
> > + * initialize the gic data for the one found.
> > + * returns
> > + *  0   : no gic was found
> > + *  > 0 : the gic version of the gic found
> > + */
> > +extern int gic_init(void);
> 
> If we are going to make the library API agnostic I guess returning NULL
> or an ops structure would be best here?

While this function is gic-version agnostic, it's also the only function
that identifies the gic version to the unit test. unit test writers may
need to know what they have. Indeed, they may only care about one type,
and then call report_skip or something when they don't get the version
they want. Anyway, if a unit test really doesn't care, it can simply use
this like a bool, telling them they have a supported gic or not.

The ops structure doesn't need to be returned, that'll be
pseudo-private, only referenced by lib wrapper functions.

> 
> > +
> > +#endif /* !__ASSEMBLY__ */
> > +#endif /* _ASMARM_GIC_H_ */
> > diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> > new file mode 100644
> > index 000000000000..d655105e058b
> > --- /dev/null
> > +++ b/lib/arm/gic.c
> > @@ -0,0 +1,76 @@
> > +/*
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#include <devicetree.h>
> > +#include <asm/gic.h>
> > +#include <asm/io.h>
> > +
> > +struct gicv2_data gicv2_data;
> > +
> > +/*
> > + * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> > + */
> > +static bool
> > +gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> > +{
> > +	struct dt_pbus_reg reg;
> > +	struct dt_device gic;
> > +	struct dt_bus bus;
> > +	int node, ret;
> > +
> > +	dt_bus_init_defaults(&bus);
> > +	dt_device_init(&gic, &bus, NULL);
> > +
> > +	node = dt_device_find_compatible(&gic, compatible);
> > +	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
> > +
> > +	if (node == -FDT_ERR_NOTFOUND)
> > +		return false;
> > +
> > +	dt_device_bind_node(&gic, node);
> > +
> > +	ret = dt_pbus_translate(&gic, 0, &reg);
> > +	assert(ret == 0);
> > +	*base1 = ioremap(reg.addr, reg.size);
> > +
> > +	ret = dt_pbus_translate(&gic, 1, &reg);
> > +	assert(ret == 0);
> > +	*base2 = ioremap(reg.addr, reg.size);
> > +
> > +	return true;
> > +}
> > +
> > +int gicv2_init(void)
> > +{
> > +	return gic_get_dt_bases("arm,cortex-a15-gic",
> > +			&gicv2_data.dist_base, &gicv2_data.cpu_base);
> > +}
> > +
> > +int gic_init(void)
> > +{
> > +	if (gicv2_init())
> > +		return 2;
> > +	return 0;
> > +}
> > +
> > +void gicv2_enable_defaults(void)
> > +{
> > +	void *dist = gicv2_dist_base();
> > +	void *cpu_base = gicv2_cpu_base();
> > +	unsigned int i;
> > +
> > +	gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> > +	if (gicv2_data.irq_nr > 1020)
> > +		gicv2_data.irq_nr = 1020;
> > +
> > +	for (i = 0; i < gicv2_data.irq_nr; i += 4)
> > +		writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
> > +
> > +	writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
> > +	writel(GICD_ENABLE, dist + GICD_CTLR);
> > +
> > +	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> > +	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> > +}
> > diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
> > new file mode 100644
> > index 000000000000..52226624a209
> > --- /dev/null
> > +++ b/lib/arm64/asm/gic-v2.h
> > @@ -0,0 +1 @@
> > +#include "../../arm/asm/gic-v2.h"
> > diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
> > new file mode 100644
> > index 000000000000..e5eb302a31b4
> > --- /dev/null
> > +++ b/lib/arm64/asm/gic.h
> > @@ -0,0 +1 @@
> > +#include "../../arm/asm/gic.h"
>

Thanks,
drew

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 03/11] arm/arm64: smp: support more than 8 cpus
  2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
@ 2016-11-11 15:27     ` Andre Przywara
  -1 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 15:27 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, peter.maydell, alex.bennee, marc.zyngier, eric.auger,
	christoffer.dall

Hi,

On 10/11/16 17:21, Andrew Jones wrote:
> By adding support for launching with gicv3 we can break the 8 vcpu
> limit. This patch adds support to smp code and also selects the
> vgic model corresponding to the host. The vgic model may also be
> manually selected by adding e.g. -machine gic-version=3 to
> extra_params.
> 
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> 
> ---
> v5: left cpus a u32 for now. Changing to u64 requires a change to
>     devicetree. Will do it later. [Andre]

Given that we address this in the future:

Reviewed-by: Andre Przywara <andre.przywara@arm.com>



^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 03/11] arm/arm64: smp: support more than 8 cpus
@ 2016-11-11 15:27     ` Andre Przywara
  0 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 15:27 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, peter.maydell, alex.bennee, marc.zyngier, eric.auger,
	christoffer.dall

Hi,

On 10/11/16 17:21, Andrew Jones wrote:
> By adding support for launching with gicv3 we can break the 8 vcpu
> limit. This patch adds support to smp code and also selects the
> vgic model corresponding to the host. The vgic model may also be
> manually selected by adding e.g. -machine gic-version=3 to
> extra_params.
> 
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> 
> ---
> v5: left cpus a u32 for now. Changing to u64 requires a change to
>     devicetree. Will do it later. [Andre]

Given that we address this in the future:

Reviewed-by: Andre Przywara <andre.przywara@arm.com>

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support
  2016-11-11 15:17       ` [Qemu-devel] " Andre Przywara
  (?)
@ 2016-11-11 15:30       ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-11 15:30 UTC (permalink / raw)
  To: Andre Przywara
  Cc: Alex Bennée, peter.maydell, kvm, marc.zyngier, qemu-devel,
	eric.auger, qemu-arm, pbonzini, kvmarm, christoffer.dall

On Fri, Nov 11, 2016 at 03:17:25PM +0000, Andre Przywara wrote:
> > 
> > This doesn't seem to be used and I'm not sure what GICD_TYPER_IRQS it is
> > trying to achieve.
> 
> The idea is to calculate the number of implemented SPIs. But I am not a
> big fan of copying a macro from the emulation code base to the test code.
>

Consider it rewritten then :-) I did review the spec while copying
over stuff from Linux, so Linux has been getting a review at the
same time. I agree that we should be cautious with copy+paste and
we should probably not bring over super complex macros that are hard
to understand, because we want kvm-unit-tests to be easy to read.
Let's not say "no macros from Linux", but rather case-by-case it.
I think this macro is a nice addition to our little API.

Thanks,
drew

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support
  2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
@ 2016-11-11 15:34     ` Andre Przywara
  -1 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 15:34 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, pbonzini

Hi,

On 10/11/16 17:21, Andrew Jones wrote:
> Add some gicv2 support. This just adds init and enable
> functions, allowing unit tests to start messing with it.
> 
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> 
> ---
> v5: share/use only the modern register names [Andre]

Thanks! That looks much better now.

Reviewed-by: Andre Przywara <andre.przywara@arm.com>

> v4:
>  - only take defines from kernel we need now [Andre]
>  - moved defines to asm/gic.h so they'll be shared with v3 [drew]
>  - simplify enable by not caring if we reinit the distributor [drew]
>  - init all GICD_INT_DEF_PRI_X4 registers [Eric]
> ---
>  arm/Makefile.common    |  1 +
>  lib/arm/asm/gic-v2.h   | 34 ++++++++++++++++++++++
>  lib/arm/asm/gic.h      | 37 ++++++++++++++++++++++++
>  lib/arm/gic.c          | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  lib/arm64/asm/gic-v2.h |  1 +
>  lib/arm64/asm/gic.h    |  1 +
>  6 files changed, 150 insertions(+)
>  create mode 100644 lib/arm/asm/gic-v2.h
>  create mode 100644 lib/arm/asm/gic.h
>  create mode 100644 lib/arm/gic.c
>  create mode 100644 lib/arm64/asm/gic-v2.h
>  create mode 100644 lib/arm64/asm/gic.h
> 
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index ccb554d9251a..41239c37e092 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
>  cflatobjs += lib/arm/bitops.o
>  cflatobjs += lib/arm/psci.o
>  cflatobjs += lib/arm/smp.o
> +cflatobjs += lib/arm/gic.o
>  
>  libeabi = lib/arm/libeabi.a
>  eabiobjs = lib/arm/eabi_compat.o
> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
> new file mode 100644
> index 000000000000..c2d5fecd4886
> --- /dev/null
> +++ b/lib/arm/asm/gic-v2.h
> @@ -0,0 +1,34 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V2_H_
> +#define _ASMARM_GIC_V2_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
> +#endif
> +
> +#define GICD_ENABLE			0x1
> +#define GICC_ENABLE			0x1
> +
> +#ifndef __ASSEMBLY__
> +
> +struct gicv2_data {
> +	void *dist_base;
> +	void *cpu_base;
> +	unsigned int irq_nr;
> +};
> +extern struct gicv2_data gicv2_data;
> +
> +#define gicv2_dist_base()		(gicv2_data.dist_base)
> +#define gicv2_cpu_base()		(gicv2_data.cpu_base)
> +
> +extern int gicv2_init(void);
> +extern void gicv2_enable_defaults(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V2_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> new file mode 100644
> index 000000000000..d44e47bcf404
> --- /dev/null
> +++ b/lib/arm/asm/gic.h
> @@ -0,0 +1,37 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_H_
> +#define _ASMARM_GIC_H_
> +
> +#include <asm/gic-v2.h>
> +
> +#define GICD_CTLR			0x0000
> +#define GICD_TYPER			0x0004
> +#define GICD_ISENABLER			0x0100
> +#define GICD_IPRIORITYR			0x0400
> +
> +#define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1) * 32)
> +#define GICD_INT_EN_SET_SGI		0x0000ffff
> +#define GICD_INT_DEF_PRI_X4		0xa0a0a0a0
> +
> +#define GICC_CTLR			0x0000
> +#define GICC_PMR			0x0004
> +
> +#define GICC_INT_PRI_THRESHOLD		0xf0
> +
> +#ifndef __ASSEMBLY__
> +
> +/*
> + * gic_init will try to find all known gics, and then
> + * initialize the gic data for the one found.
> + * returns
> + *  0   : no gic was found
> + *  > 0 : the gic version of the gic found
> + */
> +extern int gic_init(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_H_ */
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> new file mode 100644
> index 000000000000..d655105e058b
> --- /dev/null
> +++ b/lib/arm/gic.c
> @@ -0,0 +1,76 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <devicetree.h>
> +#include <asm/gic.h>
> +#include <asm/io.h>
> +
> +struct gicv2_data gicv2_data;
> +
> +/*
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + */
> +static bool
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +{
> +	struct dt_pbus_reg reg;
> +	struct dt_device gic;
> +	struct dt_bus bus;
> +	int node, ret;
> +
> +	dt_bus_init_defaults(&bus);
> +	dt_device_init(&gic, &bus, NULL);
> +
> +	node = dt_device_find_compatible(&gic, compatible);
> +	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
> +
> +	if (node == -FDT_ERR_NOTFOUND)
> +		return false;
> +
> +	dt_device_bind_node(&gic, node);
> +
> +	ret = dt_pbus_translate(&gic, 0, &reg);
> +	assert(ret == 0);
> +	*base1 = ioremap(reg.addr, reg.size);
> +
> +	ret = dt_pbus_translate(&gic, 1, &reg);
> +	assert(ret == 0);
> +	*base2 = ioremap(reg.addr, reg.size);
> +
> +	return true;
> +}
> +
> +int gicv2_init(void)
> +{
> +	return gic_get_dt_bases("arm,cortex-a15-gic",
> +			&gicv2_data.dist_base, &gicv2_data.cpu_base);
> +}
> +
> +int gic_init(void)
> +{
> +	if (gicv2_init())
> +		return 2;
> +	return 0;
> +}
> +
> +void gicv2_enable_defaults(void)
> +{
> +	void *dist = gicv2_dist_base();
> +	void *cpu_base = gicv2_cpu_base();
> +	unsigned int i;
> +
> +	gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> +	if (gicv2_data.irq_nr > 1020)
> +		gicv2_data.irq_nr = 1020;
> +
> +	for (i = 0; i < gicv2_data.irq_nr; i += 4)
> +		writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
> +
> +	writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
> +	writel(GICD_ENABLE, dist + GICD_CTLR);
> +
> +	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> +	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> +}
> diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
> new file mode 100644
> index 000000000000..52226624a209
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v2.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v2.h"
> diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
> new file mode 100644
> index 000000000000..e5eb302a31b4
> --- /dev/null
> +++ b/lib/arm64/asm/gic.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic.h"
> 

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support
@ 2016-11-11 15:34     ` Andre Przywara
  0 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 15:34 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, peter.maydell, alex.bennee, marc.zyngier, eric.auger,
	christoffer.dall

Hi,

On 10/11/16 17:21, Andrew Jones wrote:
> Add some gicv2 support. This just adds init and enable
> functions, allowing unit tests to start messing with it.
> 
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> 
> ---
> v5: share/use only the modern register names [Andre]

Thanks! That looks much better now.

Reviewed-by: Andre Przywara <andre.przywara@arm.com>

> v4:
>  - only take defines from kernel we need now [Andre]
>  - moved defines to asm/gic.h so they'll be shared with v3 [drew]
>  - simplify enable by not caring if we reinit the distributor [drew]
>  - init all GICD_INT_DEF_PRI_X4 registers [Eric]
> ---
>  arm/Makefile.common    |  1 +
>  lib/arm/asm/gic-v2.h   | 34 ++++++++++++++++++++++
>  lib/arm/asm/gic.h      | 37 ++++++++++++++++++++++++
>  lib/arm/gic.c          | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  lib/arm64/asm/gic-v2.h |  1 +
>  lib/arm64/asm/gic.h    |  1 +
>  6 files changed, 150 insertions(+)
>  create mode 100644 lib/arm/asm/gic-v2.h
>  create mode 100644 lib/arm/asm/gic.h
>  create mode 100644 lib/arm/gic.c
>  create mode 100644 lib/arm64/asm/gic-v2.h
>  create mode 100644 lib/arm64/asm/gic.h
> 
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index ccb554d9251a..41239c37e092 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
>  cflatobjs += lib/arm/bitops.o
>  cflatobjs += lib/arm/psci.o
>  cflatobjs += lib/arm/smp.o
> +cflatobjs += lib/arm/gic.o
>  
>  libeabi = lib/arm/libeabi.a
>  eabiobjs = lib/arm/eabi_compat.o
> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
> new file mode 100644
> index 000000000000..c2d5fecd4886
> --- /dev/null
> +++ b/lib/arm/asm/gic-v2.h
> @@ -0,0 +1,34 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V2_H_
> +#define _ASMARM_GIC_V2_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v2.h>. Include <asm/gic.h>
> +#endif
> +
> +#define GICD_ENABLE			0x1
> +#define GICC_ENABLE			0x1
> +
> +#ifndef __ASSEMBLY__
> +
> +struct gicv2_data {
> +	void *dist_base;
> +	void *cpu_base;
> +	unsigned int irq_nr;
> +};
> +extern struct gicv2_data gicv2_data;
> +
> +#define gicv2_dist_base()		(gicv2_data.dist_base)
> +#define gicv2_cpu_base()		(gicv2_data.cpu_base)
> +
> +extern int gicv2_init(void);
> +extern void gicv2_enable_defaults(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V2_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> new file mode 100644
> index 000000000000..d44e47bcf404
> --- /dev/null
> +++ b/lib/arm/asm/gic.h
> @@ -0,0 +1,37 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_H_
> +#define _ASMARM_GIC_H_
> +
> +#include <asm/gic-v2.h>
> +
> +#define GICD_CTLR			0x0000
> +#define GICD_TYPER			0x0004
> +#define GICD_ISENABLER			0x0100
> +#define GICD_IPRIORITYR			0x0400
> +
> +#define GICD_TYPER_IRQS(typer)		((((typer) & 0x1f) + 1) * 32)
> +#define GICD_INT_EN_SET_SGI		0x0000ffff
> +#define GICD_INT_DEF_PRI_X4		0xa0a0a0a0
> +
> +#define GICC_CTLR			0x0000
> +#define GICC_PMR			0x0004
> +
> +#define GICC_INT_PRI_THRESHOLD		0xf0
> +
> +#ifndef __ASSEMBLY__
> +
> +/*
> + * gic_init will try to find all known gics, and then
> + * initialize the gic data for the one found.
> + * returns
> + *  0   : no gic was found
> + *  > 0 : the gic version of the gic found
> + */
> +extern int gic_init(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_H_ */
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> new file mode 100644
> index 000000000000..d655105e058b
> --- /dev/null
> +++ b/lib/arm/gic.c
> @@ -0,0 +1,76 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <devicetree.h>
> +#include <asm/gic.h>
> +#include <asm/io.h>
> +
> +struct gicv2_data gicv2_data;
> +
> +/*
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + */
> +static bool
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +{
> +	struct dt_pbus_reg reg;
> +	struct dt_device gic;
> +	struct dt_bus bus;
> +	int node, ret;
> +
> +	dt_bus_init_defaults(&bus);
> +	dt_device_init(&gic, &bus, NULL);
> +
> +	node = dt_device_find_compatible(&gic, compatible);
> +	assert(node >= 0 || node == -FDT_ERR_NOTFOUND);
> +
> +	if (node == -FDT_ERR_NOTFOUND)
> +		return false;
> +
> +	dt_device_bind_node(&gic, node);
> +
> +	ret = dt_pbus_translate(&gic, 0, &reg);
> +	assert(ret == 0);
> +	*base1 = ioremap(reg.addr, reg.size);
> +
> +	ret = dt_pbus_translate(&gic, 1, &reg);
> +	assert(ret == 0);
> +	*base2 = ioremap(reg.addr, reg.size);
> +
> +	return true;
> +}
> +
> +int gicv2_init(void)
> +{
> +	return gic_get_dt_bases("arm,cortex-a15-gic",
> +			&gicv2_data.dist_base, &gicv2_data.cpu_base);
> +}
> +
> +int gic_init(void)
> +{
> +	if (gicv2_init())
> +		return 2;
> +	return 0;
> +}
> +
> +void gicv2_enable_defaults(void)
> +{
> +	void *dist = gicv2_dist_base();
> +	void *cpu_base = gicv2_cpu_base();
> +	unsigned int i;
> +
> +	gicv2_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> +	if (gicv2_data.irq_nr > 1020)
> +		gicv2_data.irq_nr = 1020;
> +
> +	for (i = 0; i < gicv2_data.irq_nr; i += 4)
> +		writel(GICD_INT_DEF_PRI_X4, dist + GICD_IPRIORITYR + i);
> +
> +	writel(GICD_INT_EN_SET_SGI, dist + GICD_ISENABLER + 0);
> +	writel(GICD_ENABLE, dist + GICD_CTLR);
> +
> +	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> +	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> +}
> diff --git a/lib/arm64/asm/gic-v2.h b/lib/arm64/asm/gic-v2.h
> new file mode 100644
> index 000000000000..52226624a209
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v2.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v2.h"
> diff --git a/lib/arm64/asm/gic.h b/lib/arm64/asm/gic.h
> new file mode 100644
> index 000000000000..e5eb302a31b4
> --- /dev/null
> +++ b/lib/arm64/asm/gic.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic.h"
> 

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 09/11] arm/arm64: add initial gicv3 support
  2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
@ 2016-11-11 15:35     ` Alex Bennée
  -1 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 15:35 UTC (permalink / raw)
  To: Andrew Jones
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
	peter.maydell, marc.zyngier, eric.auger, christoffer.dall


Andrew Jones <drjones@redhat.com> writes:

> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v5: use modern register names [Andre]
> v4:
>  - only take defines from kernel we need now [Andre]
>  - simplify enable by not caring if we reinit the distributor [drew]
> v2:
>  - configure irqs as NS GRP1
> ---
>  lib/arm/asm/arch_gicv3.h   | 42 +++++++++++++++++++++
>  lib/arm/asm/gic-v3.h       | 94 ++++++++++++++++++++++++++++++++++++++++++++++
>  lib/arm/asm/gic.h          |  6 ++-
>  lib/arm/gic.c              | 65 ++++++++++++++++++++++++++++++++
>  lib/arm64/asm/arch_gicv3.h | 44 ++++++++++++++++++++++
>  lib/arm64/asm/gic-v3.h     |  1 +
>  lib/arm64/asm/sysreg.h     | 44 ++++++++++++++++++++++
>  7 files changed, 294 insertions(+), 2 deletions(-)
>  create mode 100644 lib/arm/asm/arch_gicv3.h
>  create mode 100644 lib/arm/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/arch_gicv3.h
>  create mode 100644 lib/arm64/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/sysreg.h
>
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..81a1e5f6c29c
> --- /dev/null
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -0,0 +1,42 @@
> +/*
> + * All ripped off from arch/arm/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_ARCH_GICV3_H_
> +#define _ASMARM_ARCH_GICV3_H_
> +
> +#ifndef __ASSEMBLY__
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +#include <asm/io.h>
> +
> +#define __stringify xstr
> +
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> +
> +#define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
> +#define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> +	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> +	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> +	isb();
> +}
> +
> +static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
> +{
> +	u64 val = readl(addr);
> +	val |= (u64)readl(addr + 4) << 32;

I'd be tempted to wrap the cast in additional parentheses for clarity:

  val |= ((u64)readl(addr + 4)) << 32;

> +	return val;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_ARCH_GICV3_H_ */
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> new file mode 100644
> index 000000000000..e0f303d82508
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3.h
> @@ -0,0 +1,94 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_H_
> +#define _ASMARM_GIC_V3_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
> +#endif
> +
> +#define GICD_CTLR_RWP			(1U << 31)
> +#define GICD_CTLR_ARE_NS		(1U << 4)
> +#define GICD_CTLR_ENABLE_G1A		(1U << 1)
> +#define GICD_CTLR_ENABLE_G1		(1U << 0)

I got confused when looking at the data sheet until I noticed Secure and
Non-secure worlds have subtly different bit positions. It might be worth
making that clear in a comment.

> +
> +/* Re-Distributor registers, offsets from RD_base */
> +#define GICR_TYPER			0x0008
> +
> +#define GICR_TYPER_LAST			(1U << 4)
> +
> +/* Re-Distributor registers, offsets from SGI_base */
> +#define GICR_IGROUPR0			GICD_IGROUPR
> +#define GICR_ISENABLER0			GICD_ISENABLER
> +#define GICR_IPRIORITYR0		GICD_IPRIORITYR
> +
> +#include <asm/arch_gicv3.h>
> +
> +#ifndef __ASSEMBLY__
> +#include <asm/setup.h>
> +#include <asm/smp.h>
> +#include <asm/processor.h>
> +#include <asm/io.h>
> +
> +struct gicv3_data {
> +	void *dist_base;
> +	void *redist_base[NR_CPUS];
> +	unsigned int irq_nr;
> +};
> +extern struct gicv3_data gicv3_data;
> +
> +#define gicv3_dist_base()		(gicv3_data.dist_base)
> +#define gicv3_redist_base()		(gicv3_data.redist_base[smp_processor_id()])
> +#define gicv3_sgi_base()		(gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
> +
> +extern int gicv3_init(void);
> +extern void gicv3_enable_defaults(void);
> +extern void gicv3_set_redist_base(void);
> +
> +static inline void gicv3_do_wait_for_rwp(void *base)
> +{
> +	int count = 100000;	/* 1s */
> +
> +	while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
> +		if (!--count) {
> +			printf("GICv3: RWP timeout!\n");
> +			abort();
> +		}
> +		cpu_relax();
> +		udelay(10);
> +	};
> +}
> +
> +static inline void gicv3_dist_wait_for_rwp(void)
> +{
> +	gicv3_do_wait_for_rwp(gicv3_dist_base());
> +}
> +
> +static inline void gicv3_redist_wait_for_rwp(void)
> +{
> +	gicv3_do_wait_for_rwp(gicv3_redist_base());
> +}
> +
> +static inline u32 mpidr_compress(u64 mpidr)
> +{
> +	u64 compressed = mpidr & MPIDR_HWID_BITMASK;
> +
> +	compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
> +	return compressed;
> +}
> +
> +static inline u64 mpidr_uncompress(u32 compressed)
> +{
> +	u64 mpidr = ((u64)compressed >> 24) << 32;
> +
> +	mpidr |= compressed & MPIDR_HWID_BITMASK;
> +	return mpidr;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> index a16645708c35..981518620d18 100644
> --- a/lib/arm/asm/gic.h
> +++ b/lib/arm/asm/gic.h
> @@ -6,10 +6,9 @@
>  #ifndef _ASMARM_GIC_H_
>  #define _ASMARM_GIC_H_
>
> -#include <asm/gic-v2.h>
> -
>  #define GICD_CTLR			0x0000
>  #define GICD_TYPER			0x0004
> +#define GICD_IGROUPR			0x0080
>  #define GICD_ISENABLER			0x0100
>  #define GICD_IPRIORITYR			0x0400
>  #define GICD_SGIR			0x0f00
> @@ -26,6 +25,9 @@
>  #define GICC_INT_PRI_THRESHOLD		0xf0
>  #define GICC_INT_SPURIOUS		0x3ff
>
> +#include <asm/gic-v2.h>
> +#include <asm/gic-v3.h>
> +
>  #ifndef __ASSEMBLY__
>
>  /*
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index d655105e058b..d929d3f0fa05 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -8,9 +8,11 @@
>  #include <asm/io.h>
>
>  struct gicv2_data gicv2_data;
> +struct gicv3_data gicv3_data;
>
>  /*
>   * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>   */
>  static bool
>  gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> @@ -48,10 +50,18 @@ int gicv2_init(void)
>  			&gicv2_data.dist_base, &gicv2_data.cpu_base);
>  }
>
> +int gicv3_init(void)
> +{
> +	return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> +			&gicv3_data.redist_base[0]);
> +}
> +
>  int gic_init(void)
>  {
>  	if (gicv2_init())
>  		return 2;
> +	else if (gicv3_init())
> +		return 3;
>  	return 0;
>  }
>
> @@ -74,3 +84,58 @@ void gicv2_enable_defaults(void)
>  	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
>  	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
>  }
> +
> +void gicv3_set_redist_base(void)
> +{
> +	u32 aff = mpidr_compress(get_mpidr());
> +	void *ptr = gicv3_data.redist_base[0];
> +	u64 typer;
> +
> +	do {
> +		typer = gicv3_read_typer(ptr + GICR_TYPER);
> +		if ((typer >> 32) == aff) {
> +			gicv3_redist_base() = ptr;
> +			return;
> +		}
> +		ptr += SZ_64K * 2; /* skip RD_base and SGI_base */
> +	} while (!(typer & GICR_TYPER_LAST));
> +	assert(0);

Maybe:

  /* should never reach here */
  assert(false);

> +}
> +
> +void gicv3_enable_defaults(void)
> +{
> +	void *dist = gicv3_dist_base();
> +	void *sgi_base;
> +	unsigned int i;
> +
> +	gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> +	if (gicv3_data.irq_nr > 1020)
> +		gicv3_data.irq_nr = 1020;
> +
> +	writel(0, dist + GICD_CTLR);
> +	gicv3_dist_wait_for_rwp();
> +
> +	writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
> +	       dist + GICD_CTLR);
> +	gicv3_dist_wait_for_rwp();
> +
> +	for (i = 0; i < gicv3_data.irq_nr; i += 4)
> +		writel(~0, dist + GICD_IGROUPR + i);
> +	gicv3_dist_wait_for_rwp();
> +
> +	if (!gicv3_redist_base())
> +		gicv3_set_redist_base();
> +	sgi_base = gicv3_sgi_base();
> +
> +	writel(~0, sgi_base + GICR_IGROUPR0);
> +
> +	for (i = 0; i < 16; i += 4)
> +		writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
> +
> +	writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
> +
> +	gicv3_redist_wait_for_rwp();
> +
> +	gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
> +	gicv3_write_grpen1(1);
> +}
> diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..6d353567f56a
> --- /dev/null
> +++ b/lib/arm64/asm/arch_gicv3.h
> @@ -0,0 +1,44 @@
> +/*
> + * All ripped off from arch/arm64/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_ARCH_GICV3_H_
> +#define _ASMARM64_ARCH_GICV3_H_
> +
> +#include <asm/sysreg.h>
> +
> +#define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
> +#define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +
> +#define __stringify xstr
> +
> +/*
> + * Low-level accessors
> + *
> + * These system registers are 32 bits, but we make sure that the compiler
> + * sets the GP register's most significant bits to 0 with an explicit cast.
> + */
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> +	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> +	asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
> +	isb();
> +}
> +
> +#define gicv3_read_typer(c)		readq(c)
> +
> +#endif /* __ASSEMBLY__ */
> +#endif /* _ASMARM64_ARCH_GICV3_H_ */
> diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
> new file mode 100644
> index 000000000000..8ee5d4d9c181
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3.h"
> diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
> new file mode 100644
> index 000000000000..544a46cb8cc5
> --- /dev/null
> +++ b/lib/arm64/asm/sysreg.h
> @@ -0,0 +1,44 @@
> +/*
> + * Ripped off from arch/arm64/include/asm/sysreg.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_SYSREG_H_
> +#define _ASMARM64_SYSREG_H_
> +
> +#define sys_reg(op0, op1, crn, crm, op2) \
> +	((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
> +
> +#ifdef __ASSEMBLY__
> +	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
> +	.equ	.L__reg_num_x\num, \num
> +	.endr
> +	.equ	.L__reg_num_xzr, 31
> +
> +	.macro	mrs_s, rt, sreg
> +	.inst	0xd5200000|(\sreg)|(.L__reg_num_\rt)
> +	.endm
> +
> +	.macro	msr_s, sreg, rt
> +	.inst	0xd5000000|(\sreg)|(.L__reg_num_\rt)
> +	.endm
> +#else
> +asm(
> +"	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
> +"	.equ	.L__reg_num_x\\num, \\num\n"
> +"	.endr\n"
> +"	.equ	.L__reg_num_xzr, 31\n"
> +"\n"
> +"	.macro	mrs_s, rt, sreg\n"
> +"	.inst	0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +"	.endm\n"
> +"\n"
> +"	.macro	msr_s, sreg, rt\n"
> +"	.inst	0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +"	.endm\n"
> +);
> +#endif
> +
> +#endif /* _ASMARM64_SYSREG_H_ */

Otherwise:

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

--
Alex Bennée

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 09/11] arm/arm64: add initial gicv3 support
@ 2016-11-11 15:35     ` Alex Bennée
  0 siblings, 0 replies; 65+ messages in thread
From: Alex Bennée @ 2016-11-11 15:35 UTC (permalink / raw)
  To: Andrew Jones
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
	peter.maydell, marc.zyngier, eric.auger, christoffer.dall


Andrew Jones <drjones@redhat.com> writes:

> Signed-off-by: Andrew Jones <drjones@redhat.com>
>
> ---
> v5: use modern register names [Andre]
> v4:
>  - only take defines from kernel we need now [Andre]
>  - simplify enable by not caring if we reinit the distributor [drew]
> v2:
>  - configure irqs as NS GRP1
> ---
>  lib/arm/asm/arch_gicv3.h   | 42 +++++++++++++++++++++
>  lib/arm/asm/gic-v3.h       | 94 ++++++++++++++++++++++++++++++++++++++++++++++
>  lib/arm/asm/gic.h          |  6 ++-
>  lib/arm/gic.c              | 65 ++++++++++++++++++++++++++++++++
>  lib/arm64/asm/arch_gicv3.h | 44 ++++++++++++++++++++++
>  lib/arm64/asm/gic-v3.h     |  1 +
>  lib/arm64/asm/sysreg.h     | 44 ++++++++++++++++++++++
>  7 files changed, 294 insertions(+), 2 deletions(-)
>  create mode 100644 lib/arm/asm/arch_gicv3.h
>  create mode 100644 lib/arm/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/arch_gicv3.h
>  create mode 100644 lib/arm64/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/sysreg.h
>
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..81a1e5f6c29c
> --- /dev/null
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -0,0 +1,42 @@
> +/*
> + * All ripped off from arch/arm/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_ARCH_GICV3_H_
> +#define _ASMARM_ARCH_GICV3_H_
> +
> +#ifndef __ASSEMBLY__
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +#include <asm/io.h>
> +
> +#define __stringify xstr
> +
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> +
> +#define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
> +#define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> +	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> +	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> +	isb();
> +}
> +
> +static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
> +{
> +	u64 val = readl(addr);
> +	val |= (u64)readl(addr + 4) << 32;

I'd be tempted to wrap the cast in additional parentheses for clarity:

  val |= ((u64)readl(addr + 4)) << 32;

> +	return val;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_ARCH_GICV3_H_ */
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> new file mode 100644
> index 000000000000..e0f303d82508
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3.h
> @@ -0,0 +1,94 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_H_
> +#define _ASMARM_GIC_V3_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
> +#endif
> +
> +#define GICD_CTLR_RWP			(1U << 31)
> +#define GICD_CTLR_ARE_NS		(1U << 4)
> +#define GICD_CTLR_ENABLE_G1A		(1U << 1)
> +#define GICD_CTLR_ENABLE_G1		(1U << 0)

I got confused when looking at the data sheet until I noticed Secure and
Non-secure worlds have subtly different bit positions. It might be worth
making that clear in a comment.

> +
> +/* Re-Distributor registers, offsets from RD_base */
> +#define GICR_TYPER			0x0008
> +
> +#define GICR_TYPER_LAST			(1U << 4)
> +
> +/* Re-Distributor registers, offsets from SGI_base */
> +#define GICR_IGROUPR0			GICD_IGROUPR
> +#define GICR_ISENABLER0			GICD_ISENABLER
> +#define GICR_IPRIORITYR0		GICD_IPRIORITYR
> +
> +#include <asm/arch_gicv3.h>
> +
> +#ifndef __ASSEMBLY__
> +#include <asm/setup.h>
> +#include <asm/smp.h>
> +#include <asm/processor.h>
> +#include <asm/io.h>
> +
> +struct gicv3_data {
> +	void *dist_base;
> +	void *redist_base[NR_CPUS];
> +	unsigned int irq_nr;
> +};
> +extern struct gicv3_data gicv3_data;
> +
> +#define gicv3_dist_base()		(gicv3_data.dist_base)
> +#define gicv3_redist_base()		(gicv3_data.redist_base[smp_processor_id()])
> +#define gicv3_sgi_base()		(gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
> +
> +extern int gicv3_init(void);
> +extern void gicv3_enable_defaults(void);
> +extern void gicv3_set_redist_base(void);
> +
> +static inline void gicv3_do_wait_for_rwp(void *base)
> +{
> +	int count = 100000;	/* 1s */
> +
> +	while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
> +		if (!--count) {
> +			printf("GICv3: RWP timeout!\n");
> +			abort();
> +		}
> +		cpu_relax();
> +		udelay(10);
> +	};
> +}
> +
> +static inline void gicv3_dist_wait_for_rwp(void)
> +{
> +	gicv3_do_wait_for_rwp(gicv3_dist_base());
> +}
> +
> +static inline void gicv3_redist_wait_for_rwp(void)
> +{
> +	gicv3_do_wait_for_rwp(gicv3_redist_base());
> +}
> +
> +static inline u32 mpidr_compress(u64 mpidr)
> +{
> +	u64 compressed = mpidr & MPIDR_HWID_BITMASK;
> +
> +	compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
> +	return compressed;
> +}
> +
> +static inline u64 mpidr_uncompress(u32 compressed)
> +{
> +	u64 mpidr = ((u64)compressed >> 24) << 32;
> +
> +	mpidr |= compressed & MPIDR_HWID_BITMASK;
> +	return mpidr;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> index a16645708c35..981518620d18 100644
> --- a/lib/arm/asm/gic.h
> +++ b/lib/arm/asm/gic.h
> @@ -6,10 +6,9 @@
>  #ifndef _ASMARM_GIC_H_
>  #define _ASMARM_GIC_H_
>
> -#include <asm/gic-v2.h>
> -
>  #define GICD_CTLR			0x0000
>  #define GICD_TYPER			0x0004
> +#define GICD_IGROUPR			0x0080
>  #define GICD_ISENABLER			0x0100
>  #define GICD_IPRIORITYR			0x0400
>  #define GICD_SGIR			0x0f00
> @@ -26,6 +25,9 @@
>  #define GICC_INT_PRI_THRESHOLD		0xf0
>  #define GICC_INT_SPURIOUS		0x3ff
>
> +#include <asm/gic-v2.h>
> +#include <asm/gic-v3.h>
> +
>  #ifndef __ASSEMBLY__
>
>  /*
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index d655105e058b..d929d3f0fa05 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -8,9 +8,11 @@
>  #include <asm/io.h>
>
>  struct gicv2_data gicv2_data;
> +struct gicv3_data gicv3_data;
>
>  /*
>   * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>   */
>  static bool
>  gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> @@ -48,10 +50,18 @@ int gicv2_init(void)
>  			&gicv2_data.dist_base, &gicv2_data.cpu_base);
>  }
>
> +int gicv3_init(void)
> +{
> +	return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> +			&gicv3_data.redist_base[0]);
> +}
> +
>  int gic_init(void)
>  {
>  	if (gicv2_init())
>  		return 2;
> +	else if (gicv3_init())
> +		return 3;
>  	return 0;
>  }
>
> @@ -74,3 +84,58 @@ void gicv2_enable_defaults(void)
>  	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
>  	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
>  }
> +
> +void gicv3_set_redist_base(void)
> +{
> +	u32 aff = mpidr_compress(get_mpidr());
> +	void *ptr = gicv3_data.redist_base[0];
> +	u64 typer;
> +
> +	do {
> +		typer = gicv3_read_typer(ptr + GICR_TYPER);
> +		if ((typer >> 32) == aff) {
> +			gicv3_redist_base() = ptr;
> +			return;
> +		}
> +		ptr += SZ_64K * 2; /* skip RD_base and SGI_base */
> +	} while (!(typer & GICR_TYPER_LAST));
> +	assert(0);

Maybe:

  /* should never reach here */
  assert(false);

> +}
> +
> +void gicv3_enable_defaults(void)
> +{
> +	void *dist = gicv3_dist_base();
> +	void *sgi_base;
> +	unsigned int i;
> +
> +	gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> +	if (gicv3_data.irq_nr > 1020)
> +		gicv3_data.irq_nr = 1020;
> +
> +	writel(0, dist + GICD_CTLR);
> +	gicv3_dist_wait_for_rwp();
> +
> +	writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
> +	       dist + GICD_CTLR);
> +	gicv3_dist_wait_for_rwp();
> +
> +	for (i = 0; i < gicv3_data.irq_nr; i += 4)
> +		writel(~0, dist + GICD_IGROUPR + i);
> +	gicv3_dist_wait_for_rwp();
> +
> +	if (!gicv3_redist_base())
> +		gicv3_set_redist_base();
> +	sgi_base = gicv3_sgi_base();
> +
> +	writel(~0, sgi_base + GICR_IGROUPR0);
> +
> +	for (i = 0; i < 16; i += 4)
> +		writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
> +
> +	writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
> +
> +	gicv3_redist_wait_for_rwp();
> +
> +	gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
> +	gicv3_write_grpen1(1);
> +}
> diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..6d353567f56a
> --- /dev/null
> +++ b/lib/arm64/asm/arch_gicv3.h
> @@ -0,0 +1,44 @@
> +/*
> + * All ripped off from arch/arm64/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_ARCH_GICV3_H_
> +#define _ASMARM64_ARCH_GICV3_H_
> +
> +#include <asm/sysreg.h>
> +
> +#define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
> +#define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +
> +#define __stringify xstr
> +
> +/*
> + * Low-level accessors
> + *
> + * These system registers are 32 bits, but we make sure that the compiler
> + * sets the GP register's most significant bits to 0 with an explicit cast.
> + */
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> +	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> +	asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
> +	isb();
> +}
> +
> +#define gicv3_read_typer(c)		readq(c)
> +
> +#endif /* __ASSEMBLY__ */
> +#endif /* _ASMARM64_ARCH_GICV3_H_ */
> diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
> new file mode 100644
> index 000000000000..8ee5d4d9c181
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3.h"
> diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
> new file mode 100644
> index 000000000000..544a46cb8cc5
> --- /dev/null
> +++ b/lib/arm64/asm/sysreg.h
> @@ -0,0 +1,44 @@
> +/*
> + * Ripped off from arch/arm64/include/asm/sysreg.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_SYSREG_H_
> +#define _ASMARM64_SYSREG_H_
> +
> +#define sys_reg(op0, op1, crn, crm, op2) \
> +	((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
> +
> +#ifdef __ASSEMBLY__
> +	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
> +	.equ	.L__reg_num_x\num, \num
> +	.endr
> +	.equ	.L__reg_num_xzr, 31
> +
> +	.macro	mrs_s, rt, sreg
> +	.inst	0xd5200000|(\sreg)|(.L__reg_num_\rt)
> +	.endm
> +
> +	.macro	msr_s, sreg, rt
> +	.inst	0xd5000000|(\sreg)|(.L__reg_num_\rt)
> +	.endm
> +#else
> +asm(
> +"	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
> +"	.equ	.L__reg_num_x\\num, \\num\n"
> +"	.endr\n"
> +"	.equ	.L__reg_num_xzr, 31\n"
> +"\n"
> +"	.macro	mrs_s, rt, sreg\n"
> +"	.inst	0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +"	.endm\n"
> +"\n"
> +"	.macro	msr_s, sreg, rt\n"
> +"	.inst	0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +"	.endm\n"
> +);
> +#endif
> +
> +#endif /* _ASMARM64_SYSREG_H_ */

Otherwise:

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

--
Alex Bennée

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 08/11] libcflat: add IS_ALIGNED() macro, and page sizes
  2016-11-11 15:02     ` [Qemu-devel] " Alex Bennée
@ 2016-11-11 15:35       ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-11 15:35 UTC (permalink / raw)
  To: Alex Bennée
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
	peter.maydell, marc.zyngier, eric.auger, christoffer.dall,
	Peter Xu

On Fri, Nov 11, 2016 at 03:02:42PM +0000, Alex Bennée wrote:
> 
> Andrew Jones <drjones@redhat.com> writes:
> 
> > From: Peter Xu <peterx@redhat.com>
> >
> > These macros will be useful to do page alignment checks.
> >
> > Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> > Signed-off-by: Peter Xu <peterx@redhat.com>
> > [drew: also added SZ_64K]
> > Signed-off-by: Andrew Jones <drjones@redhat.com>
> > ---
> >  lib/libcflat.h | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > diff --git a/lib/libcflat.h b/lib/libcflat.h
> > index 82005f5d014f..143fc53061fe 100644
> > --- a/lib/libcflat.h
> > +++ b/lib/libcflat.h
> > @@ -33,6 +33,12 @@
> >  #define __ALIGN_MASK(x, mask)	(((x) + (mask)) & ~(mask))
> >  #define __ALIGN(x, a)		__ALIGN_MASK(x, (typeof(x))(a) - 1)
> >  #define ALIGN(x, a)		__ALIGN((x), (a))
> > +#define IS_ALIGNED(x, a)	(((x) & ((typeof(x))(a) - 1)) == 0)
> > +
> > +#define SZ_4K			(0x1000)
> > +#define SZ_64K			(0x10000)
> > +#define SZ_2M			(0x200000)
> > +#define SZ_1G			(0x40000000)
> 
> We don't seem to use IS_ALIGNED, or in fact anything apart from SZ_64K
> (which is multiplied by 2 anyway) so I'm not sure if this patch is worth
> it for this series.

I cherry-picked this patch (and modified it to add SZ_64K) from a
different series, one currently in-flight from Peter Xu.

> 
> Stylistically I thought (1 << foo) was preferred for setting of
> individual bits? Otherwise I'd be tempted to line the values up with
> zero padding to make it easier to read the bit position.

I agree with the style comments and will do that for v6.

> 
> >
> >  typedef uint8_t		u8;
> >  typedef int8_t		s8;
> 
>

Thanks,
drew 

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 08/11] libcflat: add IS_ALIGNED() macro, and page sizes
@ 2016-11-11 15:35       ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-11 15:35 UTC (permalink / raw)
  To: Alex Bennée
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, pbonzini, andre.przywara,
	peter.maydell, marc.zyngier, eric.auger, christoffer.dall,
	Peter Xu

On Fri, Nov 11, 2016 at 03:02:42PM +0000, Alex Bennée wrote:
> 
> Andrew Jones <drjones@redhat.com> writes:
> 
> > From: Peter Xu <peterx@redhat.com>
> >
> > These macros will be useful to do page alignment checks.
> >
> > Reviewed-by: Andre Przywara <andre.przywara@arm.com>
> > Signed-off-by: Peter Xu <peterx@redhat.com>
> > [drew: also added SZ_64K]
> > Signed-off-by: Andrew Jones <drjones@redhat.com>
> > ---
> >  lib/libcflat.h | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > diff --git a/lib/libcflat.h b/lib/libcflat.h
> > index 82005f5d014f..143fc53061fe 100644
> > --- a/lib/libcflat.h
> > +++ b/lib/libcflat.h
> > @@ -33,6 +33,12 @@
> >  #define __ALIGN_MASK(x, mask)	(((x) + (mask)) & ~(mask))
> >  #define __ALIGN(x, a)		__ALIGN_MASK(x, (typeof(x))(a) - 1)
> >  #define ALIGN(x, a)		__ALIGN((x), (a))
> > +#define IS_ALIGNED(x, a)	(((x) & ((typeof(x))(a) - 1)) == 0)
> > +
> > +#define SZ_4K			(0x1000)
> > +#define SZ_64K			(0x10000)
> > +#define SZ_2M			(0x200000)
> > +#define SZ_1G			(0x40000000)
> 
> We don't seem to use IS_ALIGNED, or in fact anything apart from SZ_64K
> (which is multiplied by 2 anyway) so I'm not sure if this patch is worth
> it for this series.

I cherry-picked this patch (and modified it to add SZ_64K) from a
different series, one currently in-flight from Peter Xu.

> 
> Stylistically I thought (1 << foo) was preferred for setting of
> individual bits? Otherwise I'd be tempted to line the values up with
> zero padding to make it easier to read the bit position.

I agree with the style comments and will do that for v6.

> 
> >
> >  typedef uint8_t		u8;
> >  typedef int8_t		s8;
> 
>

Thanks,
drew 

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [kvm-unit-tests PATCH v5 09/11] arm/arm64: add initial gicv3 support
  2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
@ 2016-11-11 16:31     ` Andre Przywara
  -1 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 16:31 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm; +Cc: marc.zyngier, pbonzini

Hi,

On 10/11/16 17:21, Andrew Jones wrote:
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> 
> ---
> v5: use modern register names [Andre]
> v4:
>  - only take defines from kernel we need now [Andre]
>  - simplify enable by not caring if we reinit the distributor [drew]
> v2:
>  - configure irqs as NS GRP1
> ---
>  lib/arm/asm/arch_gicv3.h   | 42 +++++++++++++++++++++
>  lib/arm/asm/gic-v3.h       | 94 ++++++++++++++++++++++++++++++++++++++++++++++
>  lib/arm/asm/gic.h          |  6 ++-
>  lib/arm/gic.c              | 65 ++++++++++++++++++++++++++++++++
>  lib/arm64/asm/arch_gicv3.h | 44 ++++++++++++++++++++++
>  lib/arm64/asm/gic-v3.h     |  1 +
>  lib/arm64/asm/sysreg.h     | 44 ++++++++++++++++++++++
>  7 files changed, 294 insertions(+), 2 deletions(-)
>  create mode 100644 lib/arm/asm/arch_gicv3.h
>  create mode 100644 lib/arm/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/arch_gicv3.h
>  create mode 100644 lib/arm64/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/sysreg.h
> 
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..81a1e5f6c29c
> --- /dev/null
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -0,0 +1,42 @@
> +/*
> + * All ripped off from arch/arm/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_ARCH_GICV3_H_
> +#define _ASMARM_ARCH_GICV3_H_
> +
> +#ifndef __ASSEMBLY__
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +#include <asm/io.h>
> +
> +#define __stringify xstr
> +
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> +
> +#define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
> +#define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> +	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> +	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> +	isb();
> +}
> +
> +static inline u64 gicv3_read_typer(const volatile void __iomem *addr)

It may be worth to add that this is for GICR_TYPER (or GITS_TYPER),
because GICD_TYPER is 32-bit only.
Or to make the naming generic (because the code actually is), along the
lines of read_64bit_reg or the like?

> +{
> +	u64 val = readl(addr);
> +	val |= (u64)readl(addr + 4) << 32;
> +	return val;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_ARCH_GICV3_H_ */
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> new file mode 100644
> index 000000000000..e0f303d82508
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3.h
> @@ -0,0 +1,94 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_H_
> +#define _ASMARM_GIC_V3_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
> +#endif
> +
> +#define GICD_CTLR_RWP			(1U << 31)
> +#define GICD_CTLR_ARE_NS		(1U << 4)
> +#define GICD_CTLR_ENABLE_G1A		(1U << 1)
> +#define GICD_CTLR_ENABLE_G1		(1U << 0)

+1 to Alex for adding a comment noting the non-secure view here.

> +
> +/* Re-Distributor registers, offsets from RD_base */
> +#define GICR_TYPER			0x0008
> +
> +#define GICR_TYPER_LAST			(1U << 4)
> +
> +/* Re-Distributor registers, offsets from SGI_base */
> +#define GICR_IGROUPR0			GICD_IGROUPR
> +#define GICR_ISENABLER0			GICD_ISENABLER
> +#define GICR_IPRIORITYR0		GICD_IPRIORITYR
> +
> +#include <asm/arch_gicv3.h>
> +
> +#ifndef __ASSEMBLY__
> +#include <asm/setup.h>
> +#include <asm/smp.h>
> +#include <asm/processor.h>
> +#include <asm/io.h>
> +
> +struct gicv3_data {
> +	void *dist_base;
> +	void *redist_base[NR_CPUS];
> +	unsigned int irq_nr;
> +};
> +extern struct gicv3_data gicv3_data;
> +
> +#define gicv3_dist_base()		(gicv3_data.dist_base)
> +#define gicv3_redist_base()		(gicv3_data.redist_base[smp_processor_id()])
> +#define gicv3_sgi_base()		(gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
> +
> +extern int gicv3_init(void);
> +extern void gicv3_enable_defaults(void);
> +extern void gicv3_set_redist_base(void);
> +
> +static inline void gicv3_do_wait_for_rwp(void *base)
> +{
> +	int count = 100000;	/* 1s */
> +
> +	while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
> +		if (!--count) {
> +			printf("GICv3: RWP timeout!\n");
> +			abort();
> +		}
> +		cpu_relax();
> +		udelay(10);
> +	};
> +}
> +
> +static inline void gicv3_dist_wait_for_rwp(void)
> +{
> +	gicv3_do_wait_for_rwp(gicv3_dist_base());
> +}
> +
> +static inline void gicv3_redist_wait_for_rwp(void)

Careful here. RWP is bit 3 in GICR_CTLR, while UWP (with a slightly
different semantic) is bit 31. I guess it's bit 3 you are after, so this
has to be taken into account.

> +{
> +	gicv3_do_wait_for_rwp(gicv3_redist_base());
> +}
> +
> +static inline u32 mpidr_compress(u64 mpidr)
> +{
> +	u64 compressed = mpidr & MPIDR_HWID_BITMASK;
> +
> +	compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
> +	return compressed;
> +}
> +
> +static inline u64 mpidr_uncompress(u32 compressed)
> +{
> +	u64 mpidr = ((u64)compressed >> 24) << 32;
> +
> +	mpidr |= compressed & MPIDR_HWID_BITMASK;
> +	return mpidr;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> index a16645708c35..981518620d18 100644
> --- a/lib/arm/asm/gic.h
> +++ b/lib/arm/asm/gic.h
> @@ -6,10 +6,9 @@
>  #ifndef _ASMARM_GIC_H_
>  #define _ASMARM_GIC_H_
>  
> -#include <asm/gic-v2.h>
> -
>  #define GICD_CTLR			0x0000
>  #define GICD_TYPER			0x0004
> +#define GICD_IGROUPR			0x0080
>  #define GICD_ISENABLER			0x0100
>  #define GICD_IPRIORITYR			0x0400
>  #define GICD_SGIR			0x0f00
> @@ -26,6 +25,9 @@
>  #define GICC_INT_PRI_THRESHOLD		0xf0
>  #define GICC_INT_SPURIOUS		0x3ff
>  
> +#include <asm/gic-v2.h>
> +#include <asm/gic-v3.h>
> +
>  #ifndef __ASSEMBLY__
>  
>  /*
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index d655105e058b..d929d3f0fa05 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -8,9 +8,11 @@
>  #include <asm/io.h>
>  
>  struct gicv2_data gicv2_data;
> +struct gicv3_data gicv3_data;
>  
>  /*
>   * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>   */
>  static bool
>  gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> @@ -48,10 +50,18 @@ int gicv2_init(void)
>  			&gicv2_data.dist_base, &gicv2_data.cpu_base);
>  }
>  
> +int gicv3_init(void)
> +{
> +	return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> +			&gicv3_data.redist_base[0]);
> +}
> +
>  int gic_init(void)
>  {
>  	if (gicv2_init())
>  		return 2;
> +	else if (gicv3_init())
> +		return 3;
>  	return 0;
>  }
>  
> @@ -74,3 +84,58 @@ void gicv2_enable_defaults(void)
>  	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
>  	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
>  }
> +
> +void gicv3_set_redist_base(void)
> +{
> +	u32 aff = mpidr_compress(get_mpidr());
> +	void *ptr = gicv3_data.redist_base[0];
> +	u64 typer;
> +
> +	do {
> +		typer = gicv3_read_typer(ptr + GICR_TYPER);
> +		if ((typer >> 32) == aff) {
> +			gicv3_redist_base() = ptr;
> +			return;
> +		}
> +		ptr += SZ_64K * 2; /* skip RD_base and SGI_base */

For a GICv4 the stride is four 64K pages instead of 2.
I guess we don't need to bother atm, but maybe worth a comment or even a
TODO?

> +	} while (!(typer & GICR_TYPER_LAST));
> +	assert(0);
> +}
> +
> +void gicv3_enable_defaults(void)
> +{
> +	void *dist = gicv3_dist_base();
> +	void *sgi_base;
> +	unsigned int i;
> +
> +	gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> +	if (gicv3_data.irq_nr > 1020)
> +		gicv3_data.irq_nr = 1020;
> +
> +	writel(0, dist + GICD_CTLR);
> +	gicv3_dist_wait_for_rwp();
> +
> +	writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
> +	       dist + GICD_CTLR);
> +	gicv3_dist_wait_for_rwp();
> +
> +	for (i = 0; i < gicv3_data.irq_nr; i += 4)
> +		writel(~0, dist + GICD_IGROUPR + i);
> +	gicv3_dist_wait_for_rwp();

I don't think we need this. The spec says that IGROUPR accesses are not
tracked by this bit.

> +
> +	if (!gicv3_redist_base())
> +		gicv3_set_redist_base();
> +	sgi_base = gicv3_sgi_base();
> +
> +	writel(~0, sgi_base + GICR_IGROUPR0);
> +
> +	for (i = 0; i < 16; i += 4)
> +		writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
> +
> +	writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
> +
> +	gicv3_redist_wait_for_rwp();

I think we don't need this either, only for clear enable. That applies
to both RWP (= bit 3) and UWP (= bit 31).

Cheers,
Andre.

> +
> +	gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
> +	gicv3_write_grpen1(1);
> +}
> diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..6d353567f56a
> --- /dev/null
> +++ b/lib/arm64/asm/arch_gicv3.h
> @@ -0,0 +1,44 @@
> +/*
> + * All ripped off from arch/arm64/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_ARCH_GICV3_H_
> +#define _ASMARM64_ARCH_GICV3_H_
> +
> +#include <asm/sysreg.h>
> +
> +#define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
> +#define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +
> +#define __stringify xstr
> +
> +/*
> + * Low-level accessors
> + *
> + * These system registers are 32 bits, but we make sure that the compiler
> + * sets the GP register's most significant bits to 0 with an explicit cast.
> + */
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> +	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> +	asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
> +	isb();
> +}
> +
> +#define gicv3_read_typer(c)		readq(c)
> +
> +#endif /* __ASSEMBLY__ */
> +#endif /* _ASMARM64_ARCH_GICV3_H_ */
> diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
> new file mode 100644
> index 000000000000..8ee5d4d9c181
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3.h"
> diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
> new file mode 100644
> index 000000000000..544a46cb8cc5
> --- /dev/null
> +++ b/lib/arm64/asm/sysreg.h
> @@ -0,0 +1,44 @@
> +/*
> + * Ripped off from arch/arm64/include/asm/sysreg.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_SYSREG_H_
> +#define _ASMARM64_SYSREG_H_
> +
> +#define sys_reg(op0, op1, crn, crm, op2) \
> +	((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
> +
> +#ifdef __ASSEMBLY__
> +	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
> +	.equ	.L__reg_num_x\num, \num
> +	.endr
> +	.equ	.L__reg_num_xzr, 31
> +
> +	.macro	mrs_s, rt, sreg
> +	.inst	0xd5200000|(\sreg)|(.L__reg_num_\rt)
> +	.endm
> +
> +	.macro	msr_s, sreg, rt
> +	.inst	0xd5000000|(\sreg)|(.L__reg_num_\rt)
> +	.endm
> +#else
> +asm(
> +"	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
> +"	.equ	.L__reg_num_x\\num, \\num\n"
> +"	.endr\n"
> +"	.equ	.L__reg_num_xzr, 31\n"
> +"\n"
> +"	.macro	mrs_s, rt, sreg\n"
> +"	.inst	0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +"	.endm\n"
> +"\n"
> +"	.macro	msr_s, sreg, rt\n"
> +"	.inst	0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +"	.endm\n"
> +);
> +#endif
> +
> +#endif /* _ASMARM64_SYSREG_H_ */
> 

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 09/11] arm/arm64: add initial gicv3 support
@ 2016-11-11 16:31     ` Andre Przywara
  0 siblings, 0 replies; 65+ messages in thread
From: Andre Przywara @ 2016-11-11 16:31 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvmarm, qemu-devel, qemu-arm
  Cc: pbonzini, peter.maydell, alex.bennee, marc.zyngier, eric.auger,
	christoffer.dall

Hi,

On 10/11/16 17:21, Andrew Jones wrote:
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> 
> ---
> v5: use modern register names [Andre]
> v4:
>  - only take defines from kernel we need now [Andre]
>  - simplify enable by not caring if we reinit the distributor [drew]
> v2:
>  - configure irqs as NS GRP1
> ---
>  lib/arm/asm/arch_gicv3.h   | 42 +++++++++++++++++++++
>  lib/arm/asm/gic-v3.h       | 94 ++++++++++++++++++++++++++++++++++++++++++++++
>  lib/arm/asm/gic.h          |  6 ++-
>  lib/arm/gic.c              | 65 ++++++++++++++++++++++++++++++++
>  lib/arm64/asm/arch_gicv3.h | 44 ++++++++++++++++++++++
>  lib/arm64/asm/gic-v3.h     |  1 +
>  lib/arm64/asm/sysreg.h     | 44 ++++++++++++++++++++++
>  7 files changed, 294 insertions(+), 2 deletions(-)
>  create mode 100644 lib/arm/asm/arch_gicv3.h
>  create mode 100644 lib/arm/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/arch_gicv3.h
>  create mode 100644 lib/arm64/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/sysreg.h
> 
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..81a1e5f6c29c
> --- /dev/null
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -0,0 +1,42 @@
> +/*
> + * All ripped off from arch/arm/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_ARCH_GICV3_H_
> +#define _ASMARM_ARCH_GICV3_H_
> +
> +#ifndef __ASSEMBLY__
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +#include <asm/io.h>
> +
> +#define __stringify xstr
> +
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> +
> +#define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
> +#define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> +	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> +	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> +	isb();
> +}
> +
> +static inline u64 gicv3_read_typer(const volatile void __iomem *addr)

It may be worth to add that this is for GICR_TYPER (or GITS_TYPER),
because GICD_TYPER is 32-bit only.
Or to make the naming generic (because the code actually is), along the
lines of read_64bit_reg or the like?

> +{
> +	u64 val = readl(addr);
> +	val |= (u64)readl(addr + 4) << 32;
> +	return val;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_ARCH_GICV3_H_ */
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> new file mode 100644
> index 000000000000..e0f303d82508
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3.h
> @@ -0,0 +1,94 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_H_
> +#define _ASMARM_GIC_V3_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
> +#endif
> +
> +#define GICD_CTLR_RWP			(1U << 31)
> +#define GICD_CTLR_ARE_NS		(1U << 4)
> +#define GICD_CTLR_ENABLE_G1A		(1U << 1)
> +#define GICD_CTLR_ENABLE_G1		(1U << 0)

+1 to Alex for adding a comment noting the non-secure view here.

> +
> +/* Re-Distributor registers, offsets from RD_base */
> +#define GICR_TYPER			0x0008
> +
> +#define GICR_TYPER_LAST			(1U << 4)
> +
> +/* Re-Distributor registers, offsets from SGI_base */
> +#define GICR_IGROUPR0			GICD_IGROUPR
> +#define GICR_ISENABLER0			GICD_ISENABLER
> +#define GICR_IPRIORITYR0		GICD_IPRIORITYR
> +
> +#include <asm/arch_gicv3.h>
> +
> +#ifndef __ASSEMBLY__
> +#include <asm/setup.h>
> +#include <asm/smp.h>
> +#include <asm/processor.h>
> +#include <asm/io.h>
> +
> +struct gicv3_data {
> +	void *dist_base;
> +	void *redist_base[NR_CPUS];
> +	unsigned int irq_nr;
> +};
> +extern struct gicv3_data gicv3_data;
> +
> +#define gicv3_dist_base()		(gicv3_data.dist_base)
> +#define gicv3_redist_base()		(gicv3_data.redist_base[smp_processor_id()])
> +#define gicv3_sgi_base()		(gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
> +
> +extern int gicv3_init(void);
> +extern void gicv3_enable_defaults(void);
> +extern void gicv3_set_redist_base(void);
> +
> +static inline void gicv3_do_wait_for_rwp(void *base)
> +{
> +	int count = 100000;	/* 1s */
> +
> +	while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
> +		if (!--count) {
> +			printf("GICv3: RWP timeout!\n");
> +			abort();
> +		}
> +		cpu_relax();
> +		udelay(10);
> +	};
> +}
> +
> +static inline void gicv3_dist_wait_for_rwp(void)
> +{
> +	gicv3_do_wait_for_rwp(gicv3_dist_base());
> +}
> +
> +static inline void gicv3_redist_wait_for_rwp(void)

Careful here. RWP is bit 3 in GICR_CTLR, while UWP (with a slightly
different semantic) is bit 31. I guess it's bit 3 you are after, so this
has to be taken into account.

> +{
> +	gicv3_do_wait_for_rwp(gicv3_redist_base());
> +}
> +
> +static inline u32 mpidr_compress(u64 mpidr)
> +{
> +	u64 compressed = mpidr & MPIDR_HWID_BITMASK;
> +
> +	compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
> +	return compressed;
> +}
> +
> +static inline u64 mpidr_uncompress(u32 compressed)
> +{
> +	u64 mpidr = ((u64)compressed >> 24) << 32;
> +
> +	mpidr |= compressed & MPIDR_HWID_BITMASK;
> +	return mpidr;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> index a16645708c35..981518620d18 100644
> --- a/lib/arm/asm/gic.h
> +++ b/lib/arm/asm/gic.h
> @@ -6,10 +6,9 @@
>  #ifndef _ASMARM_GIC_H_
>  #define _ASMARM_GIC_H_
>  
> -#include <asm/gic-v2.h>
> -
>  #define GICD_CTLR			0x0000
>  #define GICD_TYPER			0x0004
> +#define GICD_IGROUPR			0x0080
>  #define GICD_ISENABLER			0x0100
>  #define GICD_IPRIORITYR			0x0400
>  #define GICD_SGIR			0x0f00
> @@ -26,6 +25,9 @@
>  #define GICC_INT_PRI_THRESHOLD		0xf0
>  #define GICC_INT_SPURIOUS		0x3ff
>  
> +#include <asm/gic-v2.h>
> +#include <asm/gic-v3.h>
> +
>  #ifndef __ASSEMBLY__
>  
>  /*
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index d655105e058b..d929d3f0fa05 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -8,9 +8,11 @@
>  #include <asm/io.h>
>  
>  struct gicv2_data gicv2_data;
> +struct gicv3_data gicv3_data;
>  
>  /*
>   * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>   */
>  static bool
>  gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> @@ -48,10 +50,18 @@ int gicv2_init(void)
>  			&gicv2_data.dist_base, &gicv2_data.cpu_base);
>  }
>  
> +int gicv3_init(void)
> +{
> +	return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> +			&gicv3_data.redist_base[0]);
> +}
> +
>  int gic_init(void)
>  {
>  	if (gicv2_init())
>  		return 2;
> +	else if (gicv3_init())
> +		return 3;
>  	return 0;
>  }
>  
> @@ -74,3 +84,58 @@ void gicv2_enable_defaults(void)
>  	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
>  	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
>  }
> +
> +void gicv3_set_redist_base(void)
> +{
> +	u32 aff = mpidr_compress(get_mpidr());
> +	void *ptr = gicv3_data.redist_base[0];
> +	u64 typer;
> +
> +	do {
> +		typer = gicv3_read_typer(ptr + GICR_TYPER);
> +		if ((typer >> 32) == aff) {
> +			gicv3_redist_base() = ptr;
> +			return;
> +		}
> +		ptr += SZ_64K * 2; /* skip RD_base and SGI_base */

For a GICv4 the stride is four 64K pages instead of 2.
I guess we don't need to bother atm, but maybe worth a comment or even a
TODO?

> +	} while (!(typer & GICR_TYPER_LAST));
> +	assert(0);
> +}
> +
> +void gicv3_enable_defaults(void)
> +{
> +	void *dist = gicv3_dist_base();
> +	void *sgi_base;
> +	unsigned int i;
> +
> +	gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> +	if (gicv3_data.irq_nr > 1020)
> +		gicv3_data.irq_nr = 1020;
> +
> +	writel(0, dist + GICD_CTLR);
> +	gicv3_dist_wait_for_rwp();
> +
> +	writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
> +	       dist + GICD_CTLR);
> +	gicv3_dist_wait_for_rwp();
> +
> +	for (i = 0; i < gicv3_data.irq_nr; i += 4)
> +		writel(~0, dist + GICD_IGROUPR + i);
> +	gicv3_dist_wait_for_rwp();

I don't think we need this. The spec says that IGROUPR accesses are not
tracked by this bit.

> +
> +	if (!gicv3_redist_base())
> +		gicv3_set_redist_base();
> +	sgi_base = gicv3_sgi_base();
> +
> +	writel(~0, sgi_base + GICR_IGROUPR0);
> +
> +	for (i = 0; i < 16; i += 4)
> +		writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
> +
> +	writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
> +
> +	gicv3_redist_wait_for_rwp();

I think we don't need this either, only for clear enable. That applies
to both RWP (= bit 3) and UWP (= bit 31).

Cheers,
Andre.

> +
> +	gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
> +	gicv3_write_grpen1(1);
> +}
> diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> new file mode 100644
> index 000000000000..6d353567f56a
> --- /dev/null
> +++ b/lib/arm64/asm/arch_gicv3.h
> @@ -0,0 +1,44 @@
> +/*
> + * All ripped off from arch/arm64/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_ARCH_GICV3_H_
> +#define _ASMARM64_ARCH_GICV3_H_
> +
> +#include <asm/sysreg.h>
> +
> +#define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
> +#define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <libcflat.h>
> +#include <asm/barrier.h>
> +
> +#define __stringify xstr
> +
> +/*
> + * Low-level accessors
> + *
> + * These system registers are 32 bits, but we make sure that the compiler
> + * sets the GP register's most significant bits to 0 with an explicit cast.
> + */
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> +	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> +	asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
> +	isb();
> +}
> +
> +#define gicv3_read_typer(c)		readq(c)
> +
> +#endif /* __ASSEMBLY__ */
> +#endif /* _ASMARM64_ARCH_GICV3_H_ */
> diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
> new file mode 100644
> index 000000000000..8ee5d4d9c181
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3.h"
> diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
> new file mode 100644
> index 000000000000..544a46cb8cc5
> --- /dev/null
> +++ b/lib/arm64/asm/sysreg.h
> @@ -0,0 +1,44 @@
> +/*
> + * Ripped off from arch/arm64/include/asm/sysreg.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM64_SYSREG_H_
> +#define _ASMARM64_SYSREG_H_
> +
> +#define sys_reg(op0, op1, crn, crm, op2) \
> +	((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
> +
> +#ifdef __ASSEMBLY__
> +	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
> +	.equ	.L__reg_num_x\num, \num
> +	.endr
> +	.equ	.L__reg_num_xzr, 31
> +
> +	.macro	mrs_s, rt, sreg
> +	.inst	0xd5200000|(\sreg)|(.L__reg_num_\rt)
> +	.endm
> +
> +	.macro	msr_s, sreg, rt
> +	.inst	0xd5000000|(\sreg)|(.L__reg_num_\rt)
> +	.endm
> +#else
> +asm(
> +"	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
> +"	.equ	.L__reg_num_x\\num, \\num\n"
> +"	.endr\n"
> +"	.equ	.L__reg_num_xzr, 31\n"
> +"\n"
> +"	.macro	mrs_s, rt, sreg\n"
> +"	.inst	0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +"	.endm\n"
> +"\n"
> +"	.macro	msr_s, sreg, rt\n"
> +"	.inst	0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
> +"	.endm\n"
> +);
> +#endif
> +
> +#endif /* _ASMARM64_SYSREG_H_ */
> 

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 07/11] arm/arm64: gicv2: add an IPI test
  2016-11-11 13:13       ` Andrew Jones
@ 2016-11-14 14:12         ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-14 14:12 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, qemu-devel, qemu-arm, pbonzini, kvmarm

On Fri, Nov 11, 2016 at 02:13:31PM +0100, Andrew Jones wrote:
> On Fri, Nov 11, 2016 at 11:13:46AM +0000, Andre Przywara wrote:
> > Hi,
> > 
> > more a comment loosely related to this patch ...
> > 
> > > diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> > > index 3f6fa45c587e..68bf5cd6008f 100644
> > > --- a/arm/unittests.cfg
> > > +++ b/arm/unittests.cfg
> > > @@ -54,3 +54,10 @@ file = selftest.flat
> > >  smp = $MAX_SMP
> > >  extra_params = -append 'smp'
> > >  groups = selftest
> > > +
> > > +# Test GIC emulation
> > > +[gicv2-ipi]
> > > +file = gic.flat
> > > +smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
> > 
> > So here we always go with the maximum number of VCPUs in the guest.
> > However (as you also noted in your cover-letter) running with a
> > different number of CPUs might be interesting, for instance with less
> > than 8 CPUs on a GICv2 (the ITARGETSR register must be masked) or in
> > general with an odd number (both literally and in the broader sense). I
> > have a test case with passes with 8 VCPUs but fails with less.
> > 
> > Is there any good way to run some tests multiple times with different
> > numbers of VCPUS?
> > Shall we add some "set" functionality to the smp parameter, so that we
> > can specify a list of desired test points?
> >
> 
> We can just add multiple entries, e.g.
> 
> [gicv2-ipi]
> file = gic.flat
> smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
> [gicv2-ipi-3]
> file = gic.flat
> smp = $((($MAX_SMP > 3)?3:$MAX_SMP))
> 
> or whatever. But we need to always consider MAX_SMP, since some
> machines may less than 8.
>

Hmm, thinking about this some more, the unit test needs to know how
many processors the test wants, in order to ensure it's testing
correctly. We should provide the number to both -smp and -append,
like we do for selftest-setup.

So, we can have one test that doesn't care, just uses MAX_SMP or 8,
like this patch introduces, but then for each test that does care we
need, e.g.

 smp = 3
 extra_params = '... smp=3 ...'

Then the unit test will start exactly 2 secondaries (or abort if
they're not available)

Anyway, I don't think this is something we should extend the framework
for, but rather address it with unittests.cfg and unit test input
validation.

Thanks,
drew

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 07/11] arm/arm64: gicv2: add an IPI test
@ 2016-11-14 14:12         ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-14 14:12 UTC (permalink / raw)
  To: Andre Przywara
  Cc: peter.maydell, kvm, marc.zyngier, qemu-devel, eric.auger,
	qemu-arm, pbonzini, alex.bennee, kvmarm, christoffer.dall

On Fri, Nov 11, 2016 at 02:13:31PM +0100, Andrew Jones wrote:
> On Fri, Nov 11, 2016 at 11:13:46AM +0000, Andre Przywara wrote:
> > Hi,
> > 
> > more a comment loosely related to this patch ...
> > 
> > > diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> > > index 3f6fa45c587e..68bf5cd6008f 100644
> > > --- a/arm/unittests.cfg
> > > +++ b/arm/unittests.cfg
> > > @@ -54,3 +54,10 @@ file = selftest.flat
> > >  smp = $MAX_SMP
> > >  extra_params = -append 'smp'
> > >  groups = selftest
> > > +
> > > +# Test GIC emulation
> > > +[gicv2-ipi]
> > > +file = gic.flat
> > > +smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
> > 
> > So here we always go with the maximum number of VCPUs in the guest.
> > However (as you also noted in your cover-letter) running with a
> > different number of CPUs might be interesting, for instance with less
> > than 8 CPUs on a GICv2 (the ITARGETSR register must be masked) or in
> > general with an odd number (both literally and in the broader sense). I
> > have a test case with passes with 8 VCPUs but fails with less.
> > 
> > Is there any good way to run some tests multiple times with different
> > numbers of VCPUS?
> > Shall we add some "set" functionality to the smp parameter, so that we
> > can specify a list of desired test points?
> >
> 
> We can just add multiple entries, e.g.
> 
> [gicv2-ipi]
> file = gic.flat
> smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
> [gicv2-ipi-3]
> file = gic.flat
> smp = $((($MAX_SMP > 3)?3:$MAX_SMP))
> 
> or whatever. But we need to always consider MAX_SMP, since some
> machines may less than 8.
>

Hmm, thinking about this some more, the unit test needs to know how
many processors the test wants, in order to ensure it's testing
correctly. We should provide the number to both -smp and -append,
like we do for selftest-setup.

So, we can have one test that doesn't care, just uses MAX_SMP or 8,
like this patch introduces, but then for each test that does care we
need, e.g.

 smp = 3
 extra_params = '... smp=3 ...'

Then the unit test will start exactly 2 secondaries (or abort if
they're not available)

Anyway, I don't think this is something we should extend the framework
for, but rather address it with unittests.cfg and unit test input
validation.

Thanks,
drew

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 09/11] arm/arm64: add initial gicv3 support
  2016-11-11 16:31     ` [Qemu-devel] " Andre Przywara
@ 2016-11-14 15:17       ` Andrew Jones
  -1 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-14 15:17 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, qemu-devel, qemu-arm, pbonzini, kvmarm

On Fri, Nov 11, 2016 at 04:31:36PM +0000, Andre Przywara wrote:
> Hi,
> 
> On 10/11/16 17:21, Andrew Jones wrote:
> > Signed-off-by: Andrew Jones <drjones@redhat.com>
> > 
> > ---
> > v5: use modern register names [Andre]
> > v4:
> >  - only take defines from kernel we need now [Andre]
> >  - simplify enable by not caring if we reinit the distributor [drew]
> > v2:
> >  - configure irqs as NS GRP1
> > ---
> >  lib/arm/asm/arch_gicv3.h   | 42 +++++++++++++++++++++
> >  lib/arm/asm/gic-v3.h       | 94 ++++++++++++++++++++++++++++++++++++++++++++++
> >  lib/arm/asm/gic.h          |  6 ++-
> >  lib/arm/gic.c              | 65 ++++++++++++++++++++++++++++++++
> >  lib/arm64/asm/arch_gicv3.h | 44 ++++++++++++++++++++++
> >  lib/arm64/asm/gic-v3.h     |  1 +
> >  lib/arm64/asm/sysreg.h     | 44 ++++++++++++++++++++++
> >  7 files changed, 294 insertions(+), 2 deletions(-)
> >  create mode 100644 lib/arm/asm/arch_gicv3.h
> >  create mode 100644 lib/arm/asm/gic-v3.h
> >  create mode 100644 lib/arm64/asm/arch_gicv3.h
> >  create mode 100644 lib/arm64/asm/gic-v3.h
> >  create mode 100644 lib/arm64/asm/sysreg.h
> > 
> > diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> > new file mode 100644
> > index 000000000000..81a1e5f6c29c
> > --- /dev/null
> > +++ b/lib/arm/asm/arch_gicv3.h
> > @@ -0,0 +1,42 @@
> > +/*
> > + * All ripped off from arch/arm/include/asm/arch_gicv3.h
> > + *
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#ifndef _ASMARM_ARCH_GICV3_H_
> > +#define _ASMARM_ARCH_GICV3_H_
> > +
> > +#ifndef __ASSEMBLY__
> > +#include <libcflat.h>
> > +#include <asm/barrier.h>
> > +#include <asm/io.h>
> > +
> > +#define __stringify xstr
> > +
> > +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> > +
> > +#define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
> > +#define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
> > +
> > +static inline void gicv3_write_pmr(u32 val)
> > +{
> > +	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> > +}
> > +
> > +static inline void gicv3_write_grpen1(u32 val)
> > +{
> > +	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> > +	isb();
> > +}
> > +
> > +static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
> 
> It may be worth to add that this is for GICR_TYPER (or GITS_TYPER),
> because GICD_TYPER is 32-bit only.
> Or to make the naming generic (because the code actually is), along the
> lines of read_64bit_reg or the like?

Hmm, the fact that these two consecutive mmio addresses allow me to
read and combine them into one address isn't a general property, but
rather one of this particular register. So I think we want typer in
the name. I'm not sure how to improve on the name, since it's useful
for both GICR_ and GITS_. I'll just add a comment above it.

> 
> > +{
> > +	u64 val = readl(addr);
> > +	val |= (u64)readl(addr + 4) << 32;
> > +	return val;
> > +}
> > +
> > +#endif /* !__ASSEMBLY__ */
> > +#endif /* _ASMARM_ARCH_GICV3_H_ */
> > diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> > new file mode 100644
> > index 000000000000..e0f303d82508
> > --- /dev/null
> > +++ b/lib/arm/asm/gic-v3.h
> > @@ -0,0 +1,94 @@
> > +/*
> > + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> > + *
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#ifndef _ASMARM_GIC_V3_H_
> > +#define _ASMARM_GIC_V3_H_
> > +
> > +#ifndef _ASMARM_GIC_H_
> > +#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
> > +#endif
> > +
> > +#define GICD_CTLR_RWP			(1U << 31)
> > +#define GICD_CTLR_ARE_NS		(1U << 4)
> > +#define GICD_CTLR_ENABLE_G1A		(1U << 1)
> > +#define GICD_CTLR_ENABLE_G1		(1U << 0)
> 
> +1 to Alex for adding a comment noting the non-secure view here.

Will do.

> 
> > +
> > +/* Re-Distributor registers, offsets from RD_base */
> > +#define GICR_TYPER			0x0008
> > +
> > +#define GICR_TYPER_LAST			(1U << 4)
> > +
> > +/* Re-Distributor registers, offsets from SGI_base */
> > +#define GICR_IGROUPR0			GICD_IGROUPR
> > +#define GICR_ISENABLER0			GICD_ISENABLER
> > +#define GICR_IPRIORITYR0		GICD_IPRIORITYR
> > +
> > +#include <asm/arch_gicv3.h>
> > +
> > +#ifndef __ASSEMBLY__
> > +#include <asm/setup.h>
> > +#include <asm/smp.h>
> > +#include <asm/processor.h>
> > +#include <asm/io.h>
> > +
> > +struct gicv3_data {
> > +	void *dist_base;
> > +	void *redist_base[NR_CPUS];
> > +	unsigned int irq_nr;
> > +};
> > +extern struct gicv3_data gicv3_data;
> > +
> > +#define gicv3_dist_base()		(gicv3_data.dist_base)
> > +#define gicv3_redist_base()		(gicv3_data.redist_base[smp_processor_id()])
> > +#define gicv3_sgi_base()		(gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
> > +
> > +extern int gicv3_init(void);
> > +extern void gicv3_enable_defaults(void);
> > +extern void gicv3_set_redist_base(void);
> > +
> > +static inline void gicv3_do_wait_for_rwp(void *base)
> > +{
> > +	int count = 100000;	/* 1s */
> > +
> > +	while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
> > +		if (!--count) {
> > +			printf("GICv3: RWP timeout!\n");
> > +			abort();
> > +		}
> > +		cpu_relax();
> > +		udelay(10);
> > +	};
> > +}
> > +
> > +static inline void gicv3_dist_wait_for_rwp(void)
> > +{
> > +	gicv3_do_wait_for_rwp(gicv3_dist_base());
> > +}
> > +
> > +static inline void gicv3_redist_wait_for_rwp(void)
> 
> Careful here. RWP is bit 3 in GICR_CTLR, while UWP (with a slightly
> different semantic) is bit 31. I guess it's bit 3 you are after, so this
> has to be taken into account.

When I stole this from the kernel I noticed that GICD_CTLR_RWP wasn't
mapped the same as GICR_CTLR_RWP, but GICR_CTLR_UWP looked "stronger"
to me, so I figured that was a subtle, but by design decision. I
could make our version less subtle by renaming to _uwp, and also adding
a comment that we abuse the gicr-uwp==gicd-rwp mapping. Maybe that's
something the kernel would like to do too, assuming I'm correct to do it
here...

> 
> > +{
> > +	gicv3_do_wait_for_rwp(gicv3_redist_base());
> > +}
> > +
> > +static inline u32 mpidr_compress(u64 mpidr)
> > +{
> > +	u64 compressed = mpidr & MPIDR_HWID_BITMASK;
> > +
> > +	compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
> > +	return compressed;
> > +}
> > +
> > +static inline u64 mpidr_uncompress(u32 compressed)
> > +{
> > +	u64 mpidr = ((u64)compressed >> 24) << 32;
> > +
> > +	mpidr |= compressed & MPIDR_HWID_BITMASK;
> > +	return mpidr;
> > +}
> > +
> > +#endif /* !__ASSEMBLY__ */
> > +#endif /* _ASMARM_GIC_V3_H_ */
> > diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> > index a16645708c35..981518620d18 100644
> > --- a/lib/arm/asm/gic.h
> > +++ b/lib/arm/asm/gic.h
> > @@ -6,10 +6,9 @@
> >  #ifndef _ASMARM_GIC_H_
> >  #define _ASMARM_GIC_H_
> >  
> > -#include <asm/gic-v2.h>
> > -
> >  #define GICD_CTLR			0x0000
> >  #define GICD_TYPER			0x0004
> > +#define GICD_IGROUPR			0x0080
> >  #define GICD_ISENABLER			0x0100
> >  #define GICD_IPRIORITYR			0x0400
> >  #define GICD_SGIR			0x0f00
> > @@ -26,6 +25,9 @@
> >  #define GICC_INT_PRI_THRESHOLD		0xf0
> >  #define GICC_INT_SPURIOUS		0x3ff
> >  
> > +#include <asm/gic-v2.h>
> > +#include <asm/gic-v3.h>
> > +
> >  #ifndef __ASSEMBLY__
> >  
> >  /*
> > diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> > index d655105e058b..d929d3f0fa05 100644
> > --- a/lib/arm/gic.c
> > +++ b/lib/arm/gic.c
> > @@ -8,9 +8,11 @@
> >  #include <asm/io.h>
> >  
> >  struct gicv2_data gicv2_data;
> > +struct gicv3_data gicv3_data;
> >  
> >  /*
> >   * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> > + * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> >   */
> >  static bool
> >  gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> > @@ -48,10 +50,18 @@ int gicv2_init(void)
> >  			&gicv2_data.dist_base, &gicv2_data.cpu_base);
> >  }
> >  
> > +int gicv3_init(void)
> > +{
> > +	return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> > +			&gicv3_data.redist_base[0]);
> > +}
> > +
> >  int gic_init(void)
> >  {
> >  	if (gicv2_init())
> >  		return 2;
> > +	else if (gicv3_init())
> > +		return 3;
> >  	return 0;
> >  }
> >  
> > @@ -74,3 +84,58 @@ void gicv2_enable_defaults(void)
> >  	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> >  	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> >  }
> > +
> > +void gicv3_set_redist_base(void)
> > +{
> > +	u32 aff = mpidr_compress(get_mpidr());
> > +	void *ptr = gicv3_data.redist_base[0];
> > +	u64 typer;
> > +
> > +	do {
> > +		typer = gicv3_read_typer(ptr + GICR_TYPER);
> > +		if ((typer >> 32) == aff) {
> > +			gicv3_redist_base() = ptr;
> > +			return;
> > +		}
> > +		ptr += SZ_64K * 2; /* skip RD_base and SGI_base */
> 
> For a GICv4 the stride is four 64K pages instead of 2.
> I guess we don't need to bother atm, but maybe worth a comment or even a
> TODO?

Will fix for v6 by adding a stride parameter to this function

> 
> > +	} while (!(typer & GICR_TYPER_LAST));
> > +	assert(0);
> > +}
> > +
> > +void gicv3_enable_defaults(void)
> > +{
> > +	void *dist = gicv3_dist_base();
> > +	void *sgi_base;
> > +	unsigned int i;
> > +
> > +	gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> > +	if (gicv3_data.irq_nr > 1020)
> > +		gicv3_data.irq_nr = 1020;
> > +
> > +	writel(0, dist + GICD_CTLR);
> > +	gicv3_dist_wait_for_rwp();
> > +
> > +	writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
> > +	       dist + GICD_CTLR);
> > +	gicv3_dist_wait_for_rwp();
> > +
> > +	for (i = 0; i < gicv3_data.irq_nr; i += 4)
> > +		writel(~0, dist + GICD_IGROUPR + i);
> > +	gicv3_dist_wait_for_rwp();
> 
> I don't think we need this. The spec says that IGROUPR accesses are not
> tracked by this bit.

Indeed, will drop.

> 
> > +
> > +	if (!gicv3_redist_base())
> > +		gicv3_set_redist_base();
> > +	sgi_base = gicv3_sgi_base();
> > +
> > +	writel(~0, sgi_base + GICR_IGROUPR0);
> > +
> > +	for (i = 0; i < 16; i += 4)
> > +		writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
> > +
> > +	writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
> > +
> > +	gicv3_redist_wait_for_rwp();
> 
> I think we don't need this either, only for clear enable. That applies
> to both RWP (= bit 3) and UWP (= bit 31).

Seems to work without it, will drop.

Thanks,
drew

> 
> Cheers,
> Andre.
> 
> > +
> > +	gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
> > +	gicv3_write_grpen1(1);
> > +}
> > diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> > new file mode 100644
> > index 000000000000..6d353567f56a
> > --- /dev/null
> > +++ b/lib/arm64/asm/arch_gicv3.h
> > @@ -0,0 +1,44 @@
> > +/*
> > + * All ripped off from arch/arm64/include/asm/arch_gicv3.h
> > + *
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#ifndef _ASMARM64_ARCH_GICV3_H_
> > +#define _ASMARM64_ARCH_GICV3_H_
> > +
> > +#include <asm/sysreg.h>
> > +
> > +#define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
> > +#define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
> > +
> > +#ifndef __ASSEMBLY__
> > +
> > +#include <libcflat.h>
> > +#include <asm/barrier.h>
> > +
> > +#define __stringify xstr
> > +
> > +/*
> > + * Low-level accessors
> > + *
> > + * These system registers are 32 bits, but we make sure that the compiler
> > + * sets the GP register's most significant bits to 0 with an explicit cast.
> > + */
> > +
> > +static inline void gicv3_write_pmr(u32 val)
> > +{
> > +	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> > +}
> > +
> > +static inline void gicv3_write_grpen1(u32 val)
> > +{
> > +	asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
> > +	isb();
> > +}
> > +
> > +#define gicv3_read_typer(c)		readq(c)
> > +
> > +#endif /* __ASSEMBLY__ */
> > +#endif /* _ASMARM64_ARCH_GICV3_H_ */
> > diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
> > new file mode 100644
> > index 000000000000..8ee5d4d9c181
> > --- /dev/null
> > +++ b/lib/arm64/asm/gic-v3.h
> > @@ -0,0 +1 @@
> > +#include "../../arm/asm/gic-v3.h"
> > diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
> > new file mode 100644
> > index 000000000000..544a46cb8cc5
> > --- /dev/null
> > +++ b/lib/arm64/asm/sysreg.h
> > @@ -0,0 +1,44 @@
> > +/*
> > + * Ripped off from arch/arm64/include/asm/sysreg.h
> > + *
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#ifndef _ASMARM64_SYSREG_H_
> > +#define _ASMARM64_SYSREG_H_
> > +
> > +#define sys_reg(op0, op1, crn, crm, op2) \
> > +	((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
> > +
> > +#ifdef __ASSEMBLY__
> > +	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
> > +	.equ	.L__reg_num_x\num, \num
> > +	.endr
> > +	.equ	.L__reg_num_xzr, 31
> > +
> > +	.macro	mrs_s, rt, sreg
> > +	.inst	0xd5200000|(\sreg)|(.L__reg_num_\rt)
> > +	.endm
> > +
> > +	.macro	msr_s, sreg, rt
> > +	.inst	0xd5000000|(\sreg)|(.L__reg_num_\rt)
> > +	.endm
> > +#else
> > +asm(
> > +"	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
> > +"	.equ	.L__reg_num_x\\num, \\num\n"
> > +"	.endr\n"
> > +"	.equ	.L__reg_num_xzr, 31\n"
> > +"\n"
> > +"	.macro	mrs_s, rt, sreg\n"
> > +"	.inst	0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
> > +"	.endm\n"
> > +"\n"
> > +"	.macro	msr_s, sreg, rt\n"
> > +"	.inst	0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
> > +"	.endm\n"
> > +);
> > +#endif
> > +
> > +#endif /* _ASMARM64_SYSREG_H_ */
> > 
> 

^ permalink raw reply	[flat|nested] 65+ messages in thread
* Re: [Qemu-devel] [kvm-unit-tests PATCH v5 09/11] arm/arm64: add initial gicv3 support
@ 2016-11-14 15:17       ` Andrew Jones
  0 siblings, 0 replies; 65+ messages in thread
From: Andrew Jones @ 2016-11-14 15:17 UTC (permalink / raw)
  To: Andre Przywara
  Cc: kvm, kvmarm, qemu-devel, qemu-arm, peter.maydell, marc.zyngier,
	eric.auger, pbonzini, alex.bennee, christoffer.dall

On Fri, Nov 11, 2016 at 04:31:36PM +0000, Andre Przywara wrote:
> Hi,
> 
> On 10/11/16 17:21, Andrew Jones wrote:
> > Signed-off-by: Andrew Jones <drjones@redhat.com>
> > 
> > ---
> > v5: use modern register names [Andre]
> > v4:
> >  - only take defines from kernel we need now [Andre]
> >  - simplify enable by not caring if we reinit the distributor [drew]
> > v2:
> >  - configure irqs as NS GRP1
> > ---
> >  lib/arm/asm/arch_gicv3.h   | 42 +++++++++++++++++++++
> >  lib/arm/asm/gic-v3.h       | 94 ++++++++++++++++++++++++++++++++++++++++++++++
> >  lib/arm/asm/gic.h          |  6 ++-
> >  lib/arm/gic.c              | 65 ++++++++++++++++++++++++++++++++
> >  lib/arm64/asm/arch_gicv3.h | 44 ++++++++++++++++++++++
> >  lib/arm64/asm/gic-v3.h     |  1 +
> >  lib/arm64/asm/sysreg.h     | 44 ++++++++++++++++++++++
> >  7 files changed, 294 insertions(+), 2 deletions(-)
> >  create mode 100644 lib/arm/asm/arch_gicv3.h
> >  create mode 100644 lib/arm/asm/gic-v3.h
> >  create mode 100644 lib/arm64/asm/arch_gicv3.h
> >  create mode 100644 lib/arm64/asm/gic-v3.h
> >  create mode 100644 lib/arm64/asm/sysreg.h
> > 
> > diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> > new file mode 100644
> > index 000000000000..81a1e5f6c29c
> > --- /dev/null
> > +++ b/lib/arm/asm/arch_gicv3.h
> > @@ -0,0 +1,42 @@
> > +/*
> > + * All ripped off from arch/arm/include/asm/arch_gicv3.h
> > + *
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#ifndef _ASMARM_ARCH_GICV3_H_
> > +#define _ASMARM_ARCH_GICV3_H_
> > +
> > +#ifndef __ASSEMBLY__
> > +#include <libcflat.h>
> > +#include <asm/barrier.h>
> > +#include <asm/io.h>
> > +
> > +#define __stringify xstr
> > +
> > +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)	p15, Op1, %0, CRn, CRm, Op2
> > +
> > +#define ICC_PMR				__ACCESS_CP15(c4, 0, c6, 0)
> > +#define ICC_IGRPEN1			__ACCESS_CP15(c12, 0, c12, 7)
> > +
> > +static inline void gicv3_write_pmr(u32 val)
> > +{
> > +	asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> > +}
> > +
> > +static inline void gicv3_write_grpen1(u32 val)
> > +{
> > +	asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> > +	isb();
> > +}
> > +
> > +static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
> 
> It may be worth to add that this is for GICR_TYPER (or GITS_TYPER),
> because GICD_TYPER is 32-bit only.
> Or to make the naming generic (because the code actually is), along the
> lines of read_64bit_reg or the like?

Hmm, the fact that these two consecutive mmio addresses allow me to
read and combine them into one address isn't a general property, but
rather one of this particular register. So I think we want typer in
the name. I'm not sure how to improve on the name, since it's useful
for both GICR_ and GITS_. I'll just add a comment above it.

> 
> > +{
> > +	u64 val = readl(addr);
> > +	val |= (u64)readl(addr + 4) << 32;
> > +	return val;
> > +}
> > +
> > +#endif /* !__ASSEMBLY__ */
> > +#endif /* _ASMARM_ARCH_GICV3_H_ */
> > diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> > new file mode 100644
> > index 000000000000..e0f303d82508
> > --- /dev/null
> > +++ b/lib/arm/asm/gic-v3.h
> > @@ -0,0 +1,94 @@
> > +/*
> > + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> > + *
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#ifndef _ASMARM_GIC_V3_H_
> > +#define _ASMARM_GIC_V3_H_
> > +
> > +#ifndef _ASMARM_GIC_H_
> > +#error Do not directly include <asm/gic-v3.h>. Include <asm/gic.h>
> > +#endif
> > +
> > +#define GICD_CTLR_RWP			(1U << 31)
> > +#define GICD_CTLR_ARE_NS		(1U << 4)
> > +#define GICD_CTLR_ENABLE_G1A		(1U << 1)
> > +#define GICD_CTLR_ENABLE_G1		(1U << 0)
> 
> +1 to Alex for adding a comment noting the non-secure view here.

Will do.

> 
> > +
> > +/* Re-Distributor registers, offsets from RD_base */
> > +#define GICR_TYPER			0x0008
> > +
> > +#define GICR_TYPER_LAST			(1U << 4)
> > +
> > +/* Re-Distributor registers, offsets from SGI_base */
> > +#define GICR_IGROUPR0			GICD_IGROUPR
> > +#define GICR_ISENABLER0			GICD_ISENABLER
> > +#define GICR_IPRIORITYR0		GICD_IPRIORITYR
> > +
> > +#include <asm/arch_gicv3.h>
> > +
> > +#ifndef __ASSEMBLY__
> > +#include <asm/setup.h>
> > +#include <asm/smp.h>
> > +#include <asm/processor.h>
> > +#include <asm/io.h>
> > +
> > +struct gicv3_data {
> > +	void *dist_base;
> > +	void *redist_base[NR_CPUS];
> > +	unsigned int irq_nr;
> > +};
> > +extern struct gicv3_data gicv3_data;
> > +
> > +#define gicv3_dist_base()		(gicv3_data.dist_base)
> > +#define gicv3_redist_base()		(gicv3_data.redist_base[smp_processor_id()])
> > +#define gicv3_sgi_base()		(gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
> > +
> > +extern int gicv3_init(void);
> > +extern void gicv3_enable_defaults(void);
> > +extern void gicv3_set_redist_base(void);
> > +
> > +static inline void gicv3_do_wait_for_rwp(void *base)
> > +{
> > +	int count = 100000;	/* 1s */
> > +
> > +	while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
> > +		if (!--count) {
> > +			printf("GICv3: RWP timeout!\n");
> > +			abort();
> > +		}
> > +		cpu_relax();
> > +		udelay(10);
> > +	};
> > +}
> > +
> > +static inline void gicv3_dist_wait_for_rwp(void)
> > +{
> > +	gicv3_do_wait_for_rwp(gicv3_dist_base());
> > +}
> > +
> > +static inline void gicv3_redist_wait_for_rwp(void)
> 
> Careful here. RWP is bit 3 in GICR_CTLR, while UWP (with a slightly
> different semantic) is bit 31. I guess it's bit 3 you are after, so this
> has to be taken into account.

When I stole this from the kernel I noticed that GICD_CTLR_RWP wasn't
mapped the same as GICR_CTLR_RWP, but GICR_CTLR_UWP looked "stronger"
to me, so I figured that was a subtle, but by design decision. I
could make our version less subtle by renaming to _uwp, and also adding
a comment that we abuse the gicr-uwp==gicd-rwp mapping. Maybe that's
something the kernel would like to do too, assuming I'm correct to do it
here...

> 
> > +{
> > +	gicv3_do_wait_for_rwp(gicv3_redist_base());
> > +}
> > +
> > +static inline u32 mpidr_compress(u64 mpidr)
> > +{
> > +	u64 compressed = mpidr & MPIDR_HWID_BITMASK;
> > +
> > +	compressed = (((compressed >> 32) & 0xff) << 24) | compressed;
> > +	return compressed;
> > +}
> > +
> > +static inline u64 mpidr_uncompress(u32 compressed)
> > +{
> > +	u64 mpidr = ((u64)compressed >> 24) << 32;
> > +
> > +	mpidr |= compressed & MPIDR_HWID_BITMASK;
> > +	return mpidr;
> > +}
> > +
> > +#endif /* !__ASSEMBLY__ */
> > +#endif /* _ASMARM_GIC_V3_H_ */
> > diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> > index a16645708c35..981518620d18 100644
> > --- a/lib/arm/asm/gic.h
> > +++ b/lib/arm/asm/gic.h
> > @@ -6,10 +6,9 @@
> >  #ifndef _ASMARM_GIC_H_
> >  #define _ASMARM_GIC_H_
> >  
> > -#include <asm/gic-v2.h>
> > -
> >  #define GICD_CTLR			0x0000
> >  #define GICD_TYPER			0x0004
> > +#define GICD_IGROUPR			0x0080
> >  #define GICD_ISENABLER			0x0100
> >  #define GICD_IPRIORITYR			0x0400
> >  #define GICD_SGIR			0x0f00
> > @@ -26,6 +25,9 @@
> >  #define GICC_INT_PRI_THRESHOLD		0xf0
> >  #define GICC_INT_SPURIOUS		0x3ff
> >  
> > +#include <asm/gic-v2.h>
> > +#include <asm/gic-v3.h>
> > +
> >  #ifndef __ASSEMBLY__
> >  
> >  /*
> > diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> > index d655105e058b..d929d3f0fa05 100644
> > --- a/lib/arm/gic.c
> > +++ b/lib/arm/gic.c
> > @@ -8,9 +8,11 @@
> >  #include <asm/io.h>
> >  
> >  struct gicv2_data gicv2_data;
> > +struct gicv3_data gicv3_data;
> >  
> >  /*
> >   * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> > + * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> >   */
> >  static bool
> >  gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> > @@ -48,10 +50,18 @@ int gicv2_init(void)
> >  			&gicv2_data.dist_base, &gicv2_data.cpu_base);
> >  }
> >  
> > +int gicv3_init(void)
> > +{
> > +	return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> > +			&gicv3_data.redist_base[0]);
> > +}
> > +
> >  int gic_init(void)
> >  {
> >  	if (gicv2_init())
> >  		return 2;
> > +	else if (gicv3_init())
> > +		return 3;
> >  	return 0;
> >  }
> >  
> > @@ -74,3 +84,58 @@ void gicv2_enable_defaults(void)
> >  	writel(GICC_INT_PRI_THRESHOLD, cpu_base + GICC_PMR);
> >  	writel(GICC_ENABLE, cpu_base + GICC_CTLR);
> >  }
> > +
> > +void gicv3_set_redist_base(void)
> > +{
> > +	u32 aff = mpidr_compress(get_mpidr());
> > +	void *ptr = gicv3_data.redist_base[0];
> > +	u64 typer;
> > +
> > +	do {
> > +		typer = gicv3_read_typer(ptr + GICR_TYPER);
> > +		if ((typer >> 32) == aff) {
> > +			gicv3_redist_base() = ptr;
> > +			return;
> > +		}
> > +		ptr += SZ_64K * 2; /* skip RD_base and SGI_base */
> 
> For a GICv4 the stride is four 64K pages instead of 2.
> I guess we don't need to bother atm, but maybe worth a comment or even a
> TODO?

Will fix for v6 by adding a stride parameter to this function

> 
> > +	} while (!(typer & GICR_TYPER_LAST));
> > +	assert(0);
> > +}
> > +
> > +void gicv3_enable_defaults(void)
> > +{
> > +	void *dist = gicv3_dist_base();
> > +	void *sgi_base;
> > +	unsigned int i;
> > +
> > +	gicv3_data.irq_nr = GICD_TYPER_IRQS(readl(dist + GICD_TYPER));
> > +	if (gicv3_data.irq_nr > 1020)
> > +		gicv3_data.irq_nr = 1020;
> > +
> > +	writel(0, dist + GICD_CTLR);
> > +	gicv3_dist_wait_for_rwp();
> > +
> > +	writel(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
> > +	       dist + GICD_CTLR);
> > +	gicv3_dist_wait_for_rwp();
> > +
> > +	for (i = 0; i < gicv3_data.irq_nr; i += 4)
> > +		writel(~0, dist + GICD_IGROUPR + i);
> > +	gicv3_dist_wait_for_rwp();
> 
> I don't think we need this. The spec says that IGROUPR accesses are not
> tracked by this bit.

Indeed, will drop.

> 
> > +
> > +	if (!gicv3_redist_base())
> > +		gicv3_set_redist_base();
> > +	sgi_base = gicv3_sgi_base();
> > +
> > +	writel(~0, sgi_base + GICR_IGROUPR0);
> > +
> > +	for (i = 0; i < 16; i += 4)
> > +		writel(GICD_INT_DEF_PRI_X4, sgi_base + GICR_IPRIORITYR0 + i);
> > +
> > +	writel(GICD_INT_EN_SET_SGI, sgi_base + GICR_ISENABLER0);
> > +
> > +	gicv3_redist_wait_for_rwp();
> 
> I think we don't need this either, only for clear enable. That applies
> to both RWP (= bit 3) and UWP (= bit 31).

Seems to work without it, will drop.

Thanks,
drew

> 
> Cheers,
> Andre.
> 
> > +
> > +	gicv3_write_pmr(GICC_INT_PRI_THRESHOLD);
> > +	gicv3_write_grpen1(1);
> > +}
> > diff --git a/lib/arm64/asm/arch_gicv3.h b/lib/arm64/asm/arch_gicv3.h
> > new file mode 100644
> > index 000000000000..6d353567f56a
> > --- /dev/null
> > +++ b/lib/arm64/asm/arch_gicv3.h
> > @@ -0,0 +1,44 @@
> > +/*
> > + * All ripped off from arch/arm64/include/asm/arch_gicv3.h
> > + *
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#ifndef _ASMARM64_ARCH_GICV3_H_
> > +#define _ASMARM64_ARCH_GICV3_H_
> > +
> > +#include <asm/sysreg.h>
> > +
> > +#define ICC_PMR_EL1			sys_reg(3, 0, 4, 6, 0)
> > +#define ICC_GRPEN1_EL1			sys_reg(3, 0, 12, 12, 7)
> > +
> > +#ifndef __ASSEMBLY__
> > +
> > +#include <libcflat.h>
> > +#include <asm/barrier.h>
> > +
> > +#define __stringify xstr
> > +
> > +/*
> > + * Low-level accessors
> > + *
> > + * These system registers are 32 bits, but we make sure that the compiler
> > + * sets the GP register's most significant bits to 0 with an explicit cast.
> > + */
> > +
> > +static inline void gicv3_write_pmr(u32 val)
> > +{
> > +	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" ((u64)val));
> > +}
> > +
> > +static inline void gicv3_write_grpen1(u32 val)
> > +{
> > +	asm volatile("msr_s " __stringify(ICC_GRPEN1_EL1) ", %0" : : "r" ((u64)val));
> > +	isb();
> > +}
> > +
> > +#define gicv3_read_typer(c)		readq(c)
> > +
> > +#endif /* __ASSEMBLY__ */
> > +#endif /* _ASMARM64_ARCH_GICV3_H_ */
> > diff --git a/lib/arm64/asm/gic-v3.h b/lib/arm64/asm/gic-v3.h
> > new file mode 100644
> > index 000000000000..8ee5d4d9c181
> > --- /dev/null
> > +++ b/lib/arm64/asm/gic-v3.h
> > @@ -0,0 +1 @@
> > +#include "../../arm/asm/gic-v3.h"
> > diff --git a/lib/arm64/asm/sysreg.h b/lib/arm64/asm/sysreg.h
> > new file mode 100644
> > index 000000000000..544a46cb8cc5
> > --- /dev/null
> > +++ b/lib/arm64/asm/sysreg.h
> > @@ -0,0 +1,44 @@
> > +/*
> > + * Ripped off from arch/arm64/include/asm/sysreg.h
> > + *
> > + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU LGPL, version 2.
> > + */
> > +#ifndef _ASMARM64_SYSREG_H_
> > +#define _ASMARM64_SYSREG_H_
> > +
> > +#define sys_reg(op0, op1, crn, crm, op2) \
> > +	((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5))
> > +
> > +#ifdef __ASSEMBLY__
> > +	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30
> > +	.equ	.L__reg_num_x\num, \num
> > +	.endr
> > +	.equ	.L__reg_num_xzr, 31
> > +
> > +	.macro	mrs_s, rt, sreg
> > +	.inst	0xd5200000|(\sreg)|(.L__reg_num_\rt)
> > +	.endm
> > +
> > +	.macro	msr_s, sreg, rt
> > +	.inst	0xd5000000|(\sreg)|(.L__reg_num_\rt)
> > +	.endm
> > +#else
> > +asm(
> > +"	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
> > +"	.equ	.L__reg_num_x\\num, \\num\n"
> > +"	.endr\n"
> > +"	.equ	.L__reg_num_xzr, 31\n"
> > +"\n"
> > +"	.macro	mrs_s, rt, sreg\n"
> > +"	.inst	0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
> > +"	.endm\n"
> > +"\n"
> > +"	.macro	msr_s, sreg, rt\n"
> > +"	.inst	0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
> > +"	.endm\n"
> > +);
> > +#endif
> > +
> > +#endif /* _ASMARM64_SYSREG_H_ */
> > 
> 

^ permalink raw reply	[flat|nested] 65+ messages in thread
end of thread, other threads:[~2016-11-14 15:17 UTC | newest]

Thread overview: 65+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-10 17:21 [kvm-unit-tests PATCH v5 00/11] arm/arm64: add gic framework Andrew Jones
2016-11-10 17:21 ` [Qemu-devel] " Andrew Jones
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 01/11] lib: xstr: allow multiple args Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 02/11] arm64: fix get_"sysreg32" and make MPIDR 64bit Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 03/11] arm/arm64: smp: support more than 8 cpus Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
2016-11-11 15:27   ` Andre Przywara
2016-11-11 15:27     ` [Qemu-devel] " Andre Przywara
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 04/11] arm/arm64: add some delay routines Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 05/11] arm/arm64: irq enable/disable Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
2016-11-11 14:52   ` Alex Bennée
2016-11-11 14:52     ` [Qemu-devel] " Alex Bennée
2016-11-11 15:17     ` Andre Przywara
2016-11-11 15:17       ` [Qemu-devel] " Andre Przywara
2016-11-11 15:30       ` Andrew Jones
2016-11-11 15:24     ` Andrew Jones
2016-11-11 15:34   ` Andre Przywara
2016-11-11 15:34     ` [Qemu-devel] " Andre Przywara
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 07/11] arm/arm64: gicv2: add an IPI test Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
2016-11-11 11:13   ` Andre Przywara
2016-11-11 11:13     ` [Qemu-devel] " Andre Przywara
2016-11-11 13:13     ` Andrew Jones
2016-11-11 13:13       ` Andrew Jones
2016-11-14 14:12       ` Andrew Jones
2016-11-14 14:12         ` Andrew Jones
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 08/11] libcflat: add IS_ALIGNED() macro, and page sizes Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
2016-11-11 15:02   ` Alex Bennée
2016-11-11 15:02     ` [Qemu-devel] " Alex Bennée
2016-11-11 15:35     ` Andrew Jones
2016-11-11 15:35       ` [Qemu-devel] " Andrew Jones
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 09/11] arm/arm64: add initial gicv3 support Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
2016-11-11 15:35   ` Alex Bennée
2016-11-11 15:35     ` [Qemu-devel] " Alex Bennée
2016-11-11 16:31   ` Andre Przywara
2016-11-11 16:31     ` [Qemu-devel] " Andre Przywara
2016-11-14 15:17     ` Andrew Jones
2016-11-14 15:17       ` Andrew Jones
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones
2016-11-10 19:53   ` Alex Bennée
2016-11-10 19:53     ` [Qemu-devel] " Alex Bennée
2016-11-10 20:37     ` Andrew Jones
2016-11-10 20:37       ` Andrew Jones
2016-11-11 10:02       ` Alex Bennée
2016-11-11 10:02         ` Alex Bennée
2016-11-11 13:54         ` Andrew Jones
2016-11-11 13:54           ` Andrew Jones
2016-11-11 14:53           ` Alex Bennée
2016-11-11 15:20             ` Andre Przywara
2016-11-11 15:20               ` Andre Przywara
2016-11-11  9:21     ` Andre Przywara
2016-11-11  9:21       ` [Qemu-devel] " Andre Przywara
2016-11-11 10:00       ` Alex Bennée
2016-11-11 10:00         ` [Qemu-devel] " Alex Bennée
2016-11-10 17:21 ` [kvm-unit-tests PATCH v5 11/11] arm/arm64: gic: don't just use zero Andrew Jones
2016-11-10 17:21   ` [Qemu-devel] " Andrew Jones

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.