* [kvm-unit-tests PATCH v3 00/14] arm/arm64: Add ITS tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
This series is a revival of an RFC series sent in Dec 2016 [1].
Given the amount of code and the lack of traction at that time,
I haven't respinned until now. However a recent bug found related
to the ITS migration convinced me that this work may deserve to be
respinned and enhanced.
Tests exercise main ITS commands and also test migration.
With the migration framework, we are able to trigger the
migration from guest and that is very practical actually.
What is particular with the ITS programming is that most of
the commands are passed through queues and there is real error
handling. Invalid commands are just ignored and that is not
really tester friendly.
This series includes Andre's patch: "arm: gic: Provide
per-IRQ helper functions" [2]
test_migrate_unmapped_collection is currently failing with
upstream kernel. See [3].
The series can be fount at:
https://github.com/eauger/kut/tree/its-v3
Best Regards
Eric
History:
v2 -> v3:
- fix 32b compilation
- take into account Drew's comments (see individual diff logs)
v1 -> v2:
- took into account Zenghui's comments
- collect R-b's from Thomas
References:
[1] [kvm-unit-tests RFC 00/15] arm/arm64: add ITS framework
https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg00575.html
[2] [kvm-unit-tests PATCH 00/17] arm: gic: Test SPIs and interrupt groups
https://patchwork.kernel.org/cover/11234975/
[3] [PATCH] KVM: arm/arm64: vgic-its: Fix restoration of unmapped
collections (https://lkml.org/lkml/2019/12/13/205)
Execution:
x For ITS migration testing use:
./run_tests.sh -g migration (block on TCG)
x For other ITS tests:
./run_tests.sh -g its
x non migration tests can be launched invidually. For instance:
./arm-run arm/gic.flat -smp 8 -append 'its-trigger'
Andre Przywara (1):
arm: gic: Provide per-IRQ helper functions
Eric Auger (13):
libcflat: Add other size defines
arm/arm64: gic: Introduce setup_irq() helper
arm/arm64: gicv3: Add some re-distributor defines
arm/arm64: ITS: Introspection tests
arm/arm64: gicv3: Set the LPI config and pending tables
arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
arm/arm64: ITS: its_enable_defaults
arm/arm64: ITS: Device and collection Initialization
arm/arm64: ITS: commands
arm/arm64: ITS: INT functional tests
arm/run: Allow Migration tests
arm/arm64: ITS: migration tests
arm/arm64: ITS: pending table migration test
arm/Makefile.arm64 | 1 +
arm/Makefile.common | 2 +-
arm/gic.c | 488 +++++++++++++++++++++++++++++++++++--
arm/run | 2 +-
arm/unittests.cfg | 38 +++
lib/arm/asm/gic-v3-its.h | 167 +++++++++++++
lib/arm/asm/gic-v3.h | 25 ++
lib/arm/asm/gic.h | 8 +
lib/arm/gic-v3-its-cmd.c | 454 ++++++++++++++++++++++++++++++++++
lib/arm/gic-v3-its.c | 243 ++++++++++++++++++
lib/arm/gic-v3.c | 81 ++++++
lib/arm/gic.c | 120 ++++++++-
lib/arm/io.c | 28 +++
lib/arm64/asm/gic-v3-its.h | 1 +
lib/libcflat.h | 3 +
15 files changed, 1633 insertions(+), 28 deletions(-)
create mode 100644 lib/arm/asm/gic-v3-its.h
create mode 100644 lib/arm/gic-v3-its-cmd.c
create mode 100644 lib/arm/gic-v3-its.c
create mode 100644 lib/arm64/asm/gic-v3-its.h
--
2.20.1
^ permalink raw reply [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 00/14] arm/arm64: Add ITS tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
This series is a revival of an RFC series sent in Dec 2016 [1].
Given the amount of code and the lack of traction at that time,
I haven't respinned until now. However a recent bug found related
to the ITS migration convinced me that this work may deserve to be
respinned and enhanced.
Tests exercise main ITS commands and also test migration.
With the migration framework, we are able to trigger the
migration from guest and that is very practical actually.
What is particular with the ITS programming is that most of
the commands are passed through queues and there is real error
handling. Invalid commands are just ignored and that is not
really tester friendly.
This series includes Andre's patch: "arm: gic: Provide
per-IRQ helper functions" [2]
test_migrate_unmapped_collection is currently failing with
upstream kernel. See [3].
The series can be fount at:
https://github.com/eauger/kut/tree/its-v3
Best Regards
Eric
History:
v2 -> v3:
- fix 32b compilation
- take into account Drew's comments (see individual diff logs)
v1 -> v2:
- took into account Zenghui's comments
- collect R-b's from Thomas
References:
[1] [kvm-unit-tests RFC 00/15] arm/arm64: add ITS framework
https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg00575.html
[2] [kvm-unit-tests PATCH 00/17] arm: gic: Test SPIs and interrupt groups
https://patchwork.kernel.org/cover/11234975/
[3] [PATCH] KVM: arm/arm64: vgic-its: Fix restoration of unmapped
collections (https://lkml.org/lkml/2019/12/13/205)
Execution:
x For ITS migration testing use:
./run_tests.sh -g migration (block on TCG)
x For other ITS tests:
./run_tests.sh -g its
x non migration tests can be launched invidually. For instance:
./arm-run arm/gic.flat -smp 8 -append 'its-trigger'
Andre Przywara (1):
arm: gic: Provide per-IRQ helper functions
Eric Auger (13):
libcflat: Add other size defines
arm/arm64: gic: Introduce setup_irq() helper
arm/arm64: gicv3: Add some re-distributor defines
arm/arm64: ITS: Introspection tests
arm/arm64: gicv3: Set the LPI config and pending tables
arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
arm/arm64: ITS: its_enable_defaults
arm/arm64: ITS: Device and collection Initialization
arm/arm64: ITS: commands
arm/arm64: ITS: INT functional tests
arm/run: Allow Migration tests
arm/arm64: ITS: migration tests
arm/arm64: ITS: pending table migration test
arm/Makefile.arm64 | 1 +
arm/Makefile.common | 2 +-
arm/gic.c | 488 +++++++++++++++++++++++++++++++++++--
arm/run | 2 +-
arm/unittests.cfg | 38 +++
lib/arm/asm/gic-v3-its.h | 167 +++++++++++++
lib/arm/asm/gic-v3.h | 25 ++
lib/arm/asm/gic.h | 8 +
lib/arm/gic-v3-its-cmd.c | 454 ++++++++++++++++++++++++++++++++++
lib/arm/gic-v3-its.c | 243 ++++++++++++++++++
lib/arm/gic-v3.c | 81 ++++++
lib/arm/gic.c | 120 ++++++++-
lib/arm/io.c | 28 +++
lib/arm64/asm/gic-v3-its.h | 1 +
lib/libcflat.h | 3 +
15 files changed, 1633 insertions(+), 28 deletions(-)
create mode 100644 lib/arm/asm/gic-v3-its.h
create mode 100644 lib/arm/gic-v3-its-cmd.c
create mode 100644 lib/arm/gic-v3-its.c
create mode 100644 lib/arm64/asm/gic-v3-its.h
--
2.20.1
^ permalink raw reply [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 00/14] arm/arm64: Add ITS tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
This series is a revival of an RFC series sent in Dec 2016 [1].
Given the amount of code and the lack of traction at that time,
I haven't respinned until now. However a recent bug found related
to the ITS migration convinced me that this work may deserve to be
respinned and enhanced.
Tests exercise main ITS commands and also test migration.
With the migration framework, we are able to trigger the
migration from guest and that is very practical actually.
What is particular with the ITS programming is that most of
the commands are passed through queues and there is real error
handling. Invalid commands are just ignored and that is not
really tester friendly.
This series includes Andre's patch: "arm: gic: Provide
per-IRQ helper functions" [2]
test_migrate_unmapped_collection is currently failing with
upstream kernel. See [3].
The series can be fount at:
https://github.com/eauger/kut/tree/its-v3
Best Regards
Eric
History:
v2 -> v3:
- fix 32b compilation
- take into account Drew's comments (see individual diff logs)
v1 -> v2:
- took into account Zenghui's comments
- collect R-b's from Thomas
References:
[1] [kvm-unit-tests RFC 00/15] arm/arm64: add ITS framework
https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg00575.html
[2] [kvm-unit-tests PATCH 00/17] arm: gic: Test SPIs and interrupt groups
https://patchwork.kernel.org/cover/11234975/
[3] [PATCH] KVM: arm/arm64: vgic-its: Fix restoration of unmapped
collections (https://lkml.org/lkml/2019/12/13/205)
Execution:
x For ITS migration testing use:
./run_tests.sh -g migration (block on TCG)
x For other ITS tests:
./run_tests.sh -g its
x non migration tests can be launched invidually. For instance:
./arm-run arm/gic.flat -smp 8 -append 'its-trigger'
Andre Przywara (1):
arm: gic: Provide per-IRQ helper functions
Eric Auger (13):
libcflat: Add other size defines
arm/arm64: gic: Introduce setup_irq() helper
arm/arm64: gicv3: Add some re-distributor defines
arm/arm64: ITS: Introspection tests
arm/arm64: gicv3: Set the LPI config and pending tables
arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
arm/arm64: ITS: its_enable_defaults
arm/arm64: ITS: Device and collection Initialization
arm/arm64: ITS: commands
arm/arm64: ITS: INT functional tests
arm/run: Allow Migration tests
arm/arm64: ITS: migration tests
arm/arm64: ITS: pending table migration test
arm/Makefile.arm64 | 1 +
arm/Makefile.common | 2 +-
arm/gic.c | 488 +++++++++++++++++++++++++++++++++++--
arm/run | 2 +-
arm/unittests.cfg | 38 +++
lib/arm/asm/gic-v3-its.h | 167 +++++++++++++
lib/arm/asm/gic-v3.h | 25 ++
lib/arm/asm/gic.h | 8 +
lib/arm/gic-v3-its-cmd.c | 454 ++++++++++++++++++++++++++++++++++
lib/arm/gic-v3-its.c | 243 ++++++++++++++++++
lib/arm/gic-v3.c | 81 ++++++
lib/arm/gic.c | 120 ++++++++-
lib/arm/io.c | 28 +++
lib/arm64/asm/gic-v3-its.h | 1 +
lib/libcflat.h | 3 +
15 files changed, 1633 insertions(+), 28 deletions(-)
create mode 100644 lib/arm/asm/gic-v3-its.h
create mode 100644 lib/arm/gic-v3-its-cmd.c
create mode 100644 lib/arm/gic-v3-its.c
create mode 100644 lib/arm64/asm/gic-v3-its.h
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 01/14] libcflat: Add other size defines
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
Introduce additional SZ_256, SZ_8K, SZ_16K macros that will
be used by ITS tests.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
lib/libcflat.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/lib/libcflat.h b/lib/libcflat.h
index ea19f61..7092af2 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -36,7 +36,10 @@
#define ALIGN(x, a) __ALIGN((x), (a))
#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
+#define SZ_256 (1 << 8)
#define SZ_4K (1 << 12)
+#define SZ_8K (1 << 13)
+#define SZ_16K (1 << 14)
#define SZ_64K (1 << 16)
#define SZ_2M (1 << 21)
#define SZ_1G (1 << 30)
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 01/14] libcflat: Add other size defines
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Introduce additional SZ_256, SZ_8K, SZ_16K macros that will
be used by ITS tests.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
lib/libcflat.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/lib/libcflat.h b/lib/libcflat.h
index ea19f61..7092af2 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -36,7 +36,10 @@
#define ALIGN(x, a) __ALIGN((x), (a))
#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
+#define SZ_256 (1 << 8)
#define SZ_4K (1 << 12)
+#define SZ_8K (1 << 13)
+#define SZ_16K (1 << 14)
#define SZ_64K (1 << 16)
#define SZ_2M (1 << 21)
#define SZ_1G (1 << 30)
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 01/14] libcflat: Add other size defines
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Introduce additional SZ_256, SZ_8K, SZ_16K macros that will
be used by ITS tests.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
lib/libcflat.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/lib/libcflat.h b/lib/libcflat.h
index ea19f61..7092af2 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -36,7 +36,10 @@
#define ALIGN(x, a) __ALIGN((x), (a))
#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
+#define SZ_256 (1 << 8)
#define SZ_4K (1 << 12)
+#define SZ_8K (1 << 13)
+#define SZ_16K (1 << 14)
#define SZ_64K (1 << 16)
#define SZ_2M (1 << 21)
#define SZ_1G (1 << 30)
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 02/14] arm: gic: Provide per-IRQ helper functions
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
From: Andre Przywara <andre.przywara@arm.com>
A common theme when accessing per-IRQ parameters in the GIC distributor
is to set fields of a certain bit width in a range of MMIO registers.
Examples are the enabled status (one bit per IRQ), the level/edge
configuration (2 bits per IRQ) or the priority (8 bits per IRQ).
Add a generic helper function which is able to mask and set the
respective number of bits, given the IRQ number and the MMIO offset.
Provide wrappers using this function to easily allow configuring an IRQ.
For now assume that private IRQ numbers always refer to the current CPU.
In a GICv2 accessing the "other" private IRQs is not easily doable (the
registers are banked per CPU on the same MMIO address), so we impose the
same limitation on GICv3, even though those registers are not banked
there anymore.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
initialize reg
---
lib/arm/asm/gic-v3.h | 2 +
lib/arm/asm/gic.h | 8 ++++
lib/arm/gic.c | 90 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 100 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index 0dc838b..6beeab6 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -23,6 +23,8 @@
#define GICD_CTLR_ENABLE_G1A (1U << 1)
#define GICD_CTLR_ENABLE_G1 (1U << 0)
+#define GICD_IROUTER 0x6000
+
/* Re-Distributor registers, offsets from RD_base */
#define GICR_TYPER 0x0008
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index 09826fd..21cdb58 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -74,5 +74,13 @@ extern void gic_write_eoir(u32 irqstat);
extern void gic_ipi_send_single(int irq, int cpu);
extern void gic_ipi_send_mask(int irq, const cpumask_t *dest);
+void gic_set_irq_bit(int irq, int offset);
+void gic_enable_irq(int irq);
+void gic_disable_irq(int irq);
+void gic_set_irq_priority(int irq, u8 prio);
+void gic_set_irq_target(int irq, int cpu);
+void gic_set_irq_group(int irq, int group);
+int gic_get_irq_group(int irq);
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASMARM_GIC_H_ */
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index 9430116..aa9cb86 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -146,3 +146,93 @@ void gic_ipi_send_mask(int irq, const cpumask_t *dest)
assert(gic_common_ops && gic_common_ops->ipi_send_mask);
gic_common_ops->ipi_send_mask(irq, dest);
}
+
+enum gic_bit_access {
+ ACCESS_READ,
+ ACCESS_SET,
+ ACCESS_RMW
+};
+
+static u8 gic_masked_irq_bits(int irq, int offset, int bits, u8 value,
+ enum gic_bit_access access)
+{
+ void *base;
+ int split = 32 / bits;
+ int shift = (irq % split) * bits;
+ u32 reg = 0, mask = ((1U << bits) - 1) << shift;
+
+ switch (gic_version()) {
+ case 2:
+ base = gicv2_dist_base();
+ break;
+ case 3:
+ if (irq < 32)
+ base = gicv3_sgi_base();
+ else
+ base = gicv3_dist_base();
+ break;
+ default:
+ return 0;
+ }
+ base += offset + (irq / split) * 4;
+
+ switch (access) {
+ case ACCESS_READ:
+ return (readl(base) & mask) >> shift;
+ case ACCESS_SET:
+ reg = 0;
+ break;
+ case ACCESS_RMW:
+ reg = readl(base) & ~mask;
+ break;
+ }
+
+ writel(reg | ((u32)value << shift), base);
+
+ return 0;
+}
+
+void gic_set_irq_bit(int irq, int offset)
+{
+ gic_masked_irq_bits(irq, offset, 1, 1, ACCESS_SET);
+}
+
+void gic_enable_irq(int irq)
+{
+ gic_set_irq_bit(irq, GICD_ISENABLER);
+}
+
+void gic_disable_irq(int irq)
+{
+ gic_set_irq_bit(irq, GICD_ICENABLER);
+}
+
+void gic_set_irq_priority(int irq, u8 prio)
+{
+ gic_masked_irq_bits(irq, GICD_IPRIORITYR, 8, prio, ACCESS_RMW);
+}
+
+void gic_set_irq_target(int irq, int cpu)
+{
+ if (irq < 32)
+ return;
+
+ if (gic_version() == 2) {
+ gic_masked_irq_bits(irq, GICD_ITARGETSR, 8, 1U << cpu,
+ ACCESS_RMW);
+
+ return;
+ }
+
+ writeq(cpus[cpu], gicv3_dist_base() + GICD_IROUTER + irq * 8);
+}
+
+void gic_set_irq_group(int irq, int group)
+{
+ gic_masked_irq_bits(irq, GICD_IGROUPR, 1, group, ACCESS_RMW);
+}
+
+int gic_get_irq_group(int irq)
+{
+ return gic_masked_irq_bits(irq, GICD_IGROUPR, 1, 0, ACCESS_READ);
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 02/14] arm: gic: Provide per-IRQ helper functions
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
From: Andre Przywara <andre.przywara@arm.com>
A common theme when accessing per-IRQ parameters in the GIC distributor
is to set fields of a certain bit width in a range of MMIO registers.
Examples are the enabled status (one bit per IRQ), the level/edge
configuration (2 bits per IRQ) or the priority (8 bits per IRQ).
Add a generic helper function which is able to mask and set the
respective number of bits, given the IRQ number and the MMIO offset.
Provide wrappers using this function to easily allow configuring an IRQ.
For now assume that private IRQ numbers always refer to the current CPU.
In a GICv2 accessing the "other" private IRQs is not easily doable (the
registers are banked per CPU on the same MMIO address), so we impose the
same limitation on GICv3, even though those registers are not banked
there anymore.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
initialize reg
---
lib/arm/asm/gic-v3.h | 2 +
lib/arm/asm/gic.h | 8 ++++
lib/arm/gic.c | 90 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 100 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index 0dc838b..6beeab6 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -23,6 +23,8 @@
#define GICD_CTLR_ENABLE_G1A (1U << 1)
#define GICD_CTLR_ENABLE_G1 (1U << 0)
+#define GICD_IROUTER 0x6000
+
/* Re-Distributor registers, offsets from RD_base */
#define GICR_TYPER 0x0008
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index 09826fd..21cdb58 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -74,5 +74,13 @@ extern void gic_write_eoir(u32 irqstat);
extern void gic_ipi_send_single(int irq, int cpu);
extern void gic_ipi_send_mask(int irq, const cpumask_t *dest);
+void gic_set_irq_bit(int irq, int offset);
+void gic_enable_irq(int irq);
+void gic_disable_irq(int irq);
+void gic_set_irq_priority(int irq, u8 prio);
+void gic_set_irq_target(int irq, int cpu);
+void gic_set_irq_group(int irq, int group);
+int gic_get_irq_group(int irq);
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASMARM_GIC_H_ */
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index 9430116..aa9cb86 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -146,3 +146,93 @@ void gic_ipi_send_mask(int irq, const cpumask_t *dest)
assert(gic_common_ops && gic_common_ops->ipi_send_mask);
gic_common_ops->ipi_send_mask(irq, dest);
}
+
+enum gic_bit_access {
+ ACCESS_READ,
+ ACCESS_SET,
+ ACCESS_RMW
+};
+
+static u8 gic_masked_irq_bits(int irq, int offset, int bits, u8 value,
+ enum gic_bit_access access)
+{
+ void *base;
+ int split = 32 / bits;
+ int shift = (irq % split) * bits;
+ u32 reg = 0, mask = ((1U << bits) - 1) << shift;
+
+ switch (gic_version()) {
+ case 2:
+ base = gicv2_dist_base();
+ break;
+ case 3:
+ if (irq < 32)
+ base = gicv3_sgi_base();
+ else
+ base = gicv3_dist_base();
+ break;
+ default:
+ return 0;
+ }
+ base += offset + (irq / split) * 4;
+
+ switch (access) {
+ case ACCESS_READ:
+ return (readl(base) & mask) >> shift;
+ case ACCESS_SET:
+ reg = 0;
+ break;
+ case ACCESS_RMW:
+ reg = readl(base) & ~mask;
+ break;
+ }
+
+ writel(reg | ((u32)value << shift), base);
+
+ return 0;
+}
+
+void gic_set_irq_bit(int irq, int offset)
+{
+ gic_masked_irq_bits(irq, offset, 1, 1, ACCESS_SET);
+}
+
+void gic_enable_irq(int irq)
+{
+ gic_set_irq_bit(irq, GICD_ISENABLER);
+}
+
+void gic_disable_irq(int irq)
+{
+ gic_set_irq_bit(irq, GICD_ICENABLER);
+}
+
+void gic_set_irq_priority(int irq, u8 prio)
+{
+ gic_masked_irq_bits(irq, GICD_IPRIORITYR, 8, prio, ACCESS_RMW);
+}
+
+void gic_set_irq_target(int irq, int cpu)
+{
+ if (irq < 32)
+ return;
+
+ if (gic_version() == 2) {
+ gic_masked_irq_bits(irq, GICD_ITARGETSR, 8, 1U << cpu,
+ ACCESS_RMW);
+
+ return;
+ }
+
+ writeq(cpus[cpu], gicv3_dist_base() + GICD_IROUTER + irq * 8);
+}
+
+void gic_set_irq_group(int irq, int group)
+{
+ gic_masked_irq_bits(irq, GICD_IGROUPR, 1, group, ACCESS_RMW);
+}
+
+int gic_get_irq_group(int irq)
+{
+ return gic_masked_irq_bits(irq, GICD_IGROUPR, 1, 0, ACCESS_READ);
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 02/14] arm: gic: Provide per-IRQ helper functions
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
From: Andre Przywara <andre.przywara@arm.com>
A common theme when accessing per-IRQ parameters in the GIC distributor
is to set fields of a certain bit width in a range of MMIO registers.
Examples are the enabled status (one bit per IRQ), the level/edge
configuration (2 bits per IRQ) or the priority (8 bits per IRQ).
Add a generic helper function which is able to mask and set the
respective number of bits, given the IRQ number and the MMIO offset.
Provide wrappers using this function to easily allow configuring an IRQ.
For now assume that private IRQ numbers always refer to the current CPU.
In a GICv2 accessing the "other" private IRQs is not easily doable (the
registers are banked per CPU on the same MMIO address), so we impose the
same limitation on GICv3, even though those registers are not banked
there anymore.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
initialize reg
---
lib/arm/asm/gic-v3.h | 2 +
lib/arm/asm/gic.h | 8 ++++
lib/arm/gic.c | 90 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 100 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index 0dc838b..6beeab6 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -23,6 +23,8 @@
#define GICD_CTLR_ENABLE_G1A (1U << 1)
#define GICD_CTLR_ENABLE_G1 (1U << 0)
+#define GICD_IROUTER 0x6000
+
/* Re-Distributor registers, offsets from RD_base */
#define GICR_TYPER 0x0008
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index 09826fd..21cdb58 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -74,5 +74,13 @@ extern void gic_write_eoir(u32 irqstat);
extern void gic_ipi_send_single(int irq, int cpu);
extern void gic_ipi_send_mask(int irq, const cpumask_t *dest);
+void gic_set_irq_bit(int irq, int offset);
+void gic_enable_irq(int irq);
+void gic_disable_irq(int irq);
+void gic_set_irq_priority(int irq, u8 prio);
+void gic_set_irq_target(int irq, int cpu);
+void gic_set_irq_group(int irq, int group);
+int gic_get_irq_group(int irq);
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASMARM_GIC_H_ */
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index 9430116..aa9cb86 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -146,3 +146,93 @@ void gic_ipi_send_mask(int irq, const cpumask_t *dest)
assert(gic_common_ops && gic_common_ops->ipi_send_mask);
gic_common_ops->ipi_send_mask(irq, dest);
}
+
+enum gic_bit_access {
+ ACCESS_READ,
+ ACCESS_SET,
+ ACCESS_RMW
+};
+
+static u8 gic_masked_irq_bits(int irq, int offset, int bits, u8 value,
+ enum gic_bit_access access)
+{
+ void *base;
+ int split = 32 / bits;
+ int shift = (irq % split) * bits;
+ u32 reg = 0, mask = ((1U << bits) - 1) << shift;
+
+ switch (gic_version()) {
+ case 2:
+ base = gicv2_dist_base();
+ break;
+ case 3:
+ if (irq < 32)
+ base = gicv3_sgi_base();
+ else
+ base = gicv3_dist_base();
+ break;
+ default:
+ return 0;
+ }
+ base += offset + (irq / split) * 4;
+
+ switch (access) {
+ case ACCESS_READ:
+ return (readl(base) & mask) >> shift;
+ case ACCESS_SET:
+ reg = 0;
+ break;
+ case ACCESS_RMW:
+ reg = readl(base) & ~mask;
+ break;
+ }
+
+ writel(reg | ((u32)value << shift), base);
+
+ return 0;
+}
+
+void gic_set_irq_bit(int irq, int offset)
+{
+ gic_masked_irq_bits(irq, offset, 1, 1, ACCESS_SET);
+}
+
+void gic_enable_irq(int irq)
+{
+ gic_set_irq_bit(irq, GICD_ISENABLER);
+}
+
+void gic_disable_irq(int irq)
+{
+ gic_set_irq_bit(irq, GICD_ICENABLER);
+}
+
+void gic_set_irq_priority(int irq, u8 prio)
+{
+ gic_masked_irq_bits(irq, GICD_IPRIORITYR, 8, prio, ACCESS_RMW);
+}
+
+void gic_set_irq_target(int irq, int cpu)
+{
+ if (irq < 32)
+ return;
+
+ if (gic_version() == 2) {
+ gic_masked_irq_bits(irq, GICD_ITARGETSR, 8, 1U << cpu,
+ ACCESS_RMW);
+
+ return;
+ }
+
+ writeq(cpus[cpu], gicv3_dist_base() + GICD_IROUTER + irq * 8);
+}
+
+void gic_set_irq_group(int irq, int group)
+{
+ gic_masked_irq_bits(irq, GICD_IGROUPR, 1, group, ACCESS_RMW);
+}
+
+int gic_get_irq_group(int irq)
+{
+ return gic_masked_irq_bits(irq, GICD_IGROUPR, 1, 0, ACCESS_READ);
+}
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 03/14] arm/arm64: gic: Introduce setup_irq() helper
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
ipi_enable() code would be reusable for other interrupts
than IPI. Let's rename it setup_irq() and pass an interrupt
handler pointer.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- do not export setup_irq anymore
---
arm/gic.c | 20 +++++++-------------
1 file changed, 7 insertions(+), 13 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index fcf4c1f..abf08c7 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -34,6 +34,7 @@ static struct gic *gic;
static int acked[NR_CPUS], spurious[NR_CPUS];
static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
static cpumask_t ready;
+typedef void (*handler_t)(struct pt_regs *regs __unused);
static void nr_cpu_check(int nr)
{
@@ -215,20 +216,20 @@ static void ipi_test_smp(void)
report_prefix_pop();
}
-static void ipi_enable(void)
+static void setup_irq(handler_t handler)
{
gic_enable_defaults();
#ifdef __arm__
- install_exception_handler(EXCPTN_IRQ, ipi_handler);
+ install_exception_handler(EXCPTN_IRQ, handler);
#else
- install_irq_handler(EL1H_IRQ, ipi_handler);
+ install_irq_handler(EL1H_IRQ, handler);
#endif
local_irq_enable();
}
static void ipi_send(void)
{
- ipi_enable();
+ setup_irq(ipi_handler);
wait_on_ready();
ipi_test_self();
ipi_test_smp();
@@ -238,7 +239,7 @@ static void ipi_send(void)
static void ipi_recv(void)
{
- ipi_enable();
+ setup_irq(ipi_handler);
cpumask_set_cpu(smp_processor_id(), &ready);
while (1)
wfi();
@@ -295,14 +296,7 @@ static void ipi_clear_active_handler(struct pt_regs *regs __unused)
static void run_active_clear_test(void)
{
report_prefix_push("active");
- gic_enable_defaults();
-#ifdef __arm__
- install_exception_handler(EXCPTN_IRQ, ipi_clear_active_handler);
-#else
- install_irq_handler(EL1H_IRQ, ipi_clear_active_handler);
-#endif
- local_irq_enable();
-
+ setup_irq(ipi_clear_active_handler);
ipi_test_self();
report_prefix_pop();
}
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 03/14] arm/arm64: gic: Introduce setup_irq() helper
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
ipi_enable() code would be reusable for other interrupts
than IPI. Let's rename it setup_irq() and pass an interrupt
handler pointer.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- do not export setup_irq anymore
---
arm/gic.c | 20 +++++++-------------
1 file changed, 7 insertions(+), 13 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index fcf4c1f..abf08c7 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -34,6 +34,7 @@ static struct gic *gic;
static int acked[NR_CPUS], spurious[NR_CPUS];
static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
static cpumask_t ready;
+typedef void (*handler_t)(struct pt_regs *regs __unused);
static void nr_cpu_check(int nr)
{
@@ -215,20 +216,20 @@ static void ipi_test_smp(void)
report_prefix_pop();
}
-static void ipi_enable(void)
+static void setup_irq(handler_t handler)
{
gic_enable_defaults();
#ifdef __arm__
- install_exception_handler(EXCPTN_IRQ, ipi_handler);
+ install_exception_handler(EXCPTN_IRQ, handler);
#else
- install_irq_handler(EL1H_IRQ, ipi_handler);
+ install_irq_handler(EL1H_IRQ, handler);
#endif
local_irq_enable();
}
static void ipi_send(void)
{
- ipi_enable();
+ setup_irq(ipi_handler);
wait_on_ready();
ipi_test_self();
ipi_test_smp();
@@ -238,7 +239,7 @@ static void ipi_send(void)
static void ipi_recv(void)
{
- ipi_enable();
+ setup_irq(ipi_handler);
cpumask_set_cpu(smp_processor_id(), &ready);
while (1)
wfi();
@@ -295,14 +296,7 @@ static void ipi_clear_active_handler(struct pt_regs *regs __unused)
static void run_active_clear_test(void)
{
report_prefix_push("active");
- gic_enable_defaults();
-#ifdef __arm__
- install_exception_handler(EXCPTN_IRQ, ipi_clear_active_handler);
-#else
- install_irq_handler(EL1H_IRQ, ipi_clear_active_handler);
-#endif
- local_irq_enable();
-
+ setup_irq(ipi_clear_active_handler);
ipi_test_self();
report_prefix_pop();
}
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 03/14] arm/arm64: gic: Introduce setup_irq() helper
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
ipi_enable() code would be reusable for other interrupts
than IPI. Let's rename it setup_irq() and pass an interrupt
handler pointer.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- do not export setup_irq anymore
---
arm/gic.c | 20 +++++++-------------
1 file changed, 7 insertions(+), 13 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index fcf4c1f..abf08c7 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -34,6 +34,7 @@ static struct gic *gic;
static int acked[NR_CPUS], spurious[NR_CPUS];
static int bad_sender[NR_CPUS], bad_irq[NR_CPUS];
static cpumask_t ready;
+typedef void (*handler_t)(struct pt_regs *regs __unused);
static void nr_cpu_check(int nr)
{
@@ -215,20 +216,20 @@ static void ipi_test_smp(void)
report_prefix_pop();
}
-static void ipi_enable(void)
+static void setup_irq(handler_t handler)
{
gic_enable_defaults();
#ifdef __arm__
- install_exception_handler(EXCPTN_IRQ, ipi_handler);
+ install_exception_handler(EXCPTN_IRQ, handler);
#else
- install_irq_handler(EL1H_IRQ, ipi_handler);
+ install_irq_handler(EL1H_IRQ, handler);
#endif
local_irq_enable();
}
static void ipi_send(void)
{
- ipi_enable();
+ setup_irq(ipi_handler);
wait_on_ready();
ipi_test_self();
ipi_test_smp();
@@ -238,7 +239,7 @@ static void ipi_send(void)
static void ipi_recv(void)
{
- ipi_enable();
+ setup_irq(ipi_handler);
cpumask_set_cpu(smp_processor_id(), &ready);
while (1)
wfi();
@@ -295,14 +296,7 @@ static void ipi_clear_active_handler(struct pt_regs *regs __unused)
static void run_active_clear_test(void)
{
report_prefix_push("active");
- gic_enable_defaults();
-#ifdef __arm__
- install_exception_handler(EXCPTN_IRQ, ipi_clear_active_handler);
-#else
- install_irq_handler(EL1H_IRQ, ipi_clear_active_handler);
-#endif
- local_irq_enable();
-
+ setup_irq(ipi_clear_active_handler);
ipi_test_self();
report_prefix_pop();
}
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 04/14] arm/arm64: gicv3: Add some re-distributor defines
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
PROPBASER, PENDBASE and GICR_CTRL will be used for LPI management.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
lib/arm/asm/gic-v3.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index 6beeab6..ffb2e26 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -18,6 +18,7 @@
* We expect to be run in Non-secure mode, thus we define the
* group1 enable bits with respect to that view.
*/
+#define GICD_CTLR 0x0000
#define GICD_CTLR_RWP (1U << 31)
#define GICD_CTLR_ARE_NS (1U << 4)
#define GICD_CTLR_ENABLE_G1A (1U << 1)
@@ -36,6 +37,11 @@
#define GICR_ICENABLER0 GICD_ICENABLER
#define GICR_IPRIORITYR0 GICD_IPRIORITYR
+#define GICR_PROPBASER 0x0070
+#define GICR_PENDBASER 0x0078
+#define GICR_CTLR GICD_CTLR
+#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
+
#define ICC_SGI1R_AFFINITY_1_SHIFT 16
#define ICC_SGI1R_AFFINITY_2_SHIFT 32
#define ICC_SGI1R_AFFINITY_3_SHIFT 48
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 04/14] arm/arm64: gicv3: Add some re-distributor defines
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
PROPBASER, PENDBASE and GICR_CTRL will be used for LPI management.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
lib/arm/asm/gic-v3.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index 6beeab6..ffb2e26 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -18,6 +18,7 @@
* We expect to be run in Non-secure mode, thus we define the
* group1 enable bits with respect to that view.
*/
+#define GICD_CTLR 0x0000
#define GICD_CTLR_RWP (1U << 31)
#define GICD_CTLR_ARE_NS (1U << 4)
#define GICD_CTLR_ENABLE_G1A (1U << 1)
@@ -36,6 +37,11 @@
#define GICR_ICENABLER0 GICD_ICENABLER
#define GICR_IPRIORITYR0 GICD_IPRIORITYR
+#define GICR_PROPBASER 0x0070
+#define GICR_PENDBASER 0x0078
+#define GICR_CTLR GICD_CTLR
+#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
+
#define ICC_SGI1R_AFFINITY_1_SHIFT 16
#define ICC_SGI1R_AFFINITY_2_SHIFT 32
#define ICC_SGI1R_AFFINITY_3_SHIFT 48
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 04/14] arm/arm64: gicv3: Add some re-distributor defines
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
PROPBASER, PENDBASE and GICR_CTRL will be used for LPI management.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
lib/arm/asm/gic-v3.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index 6beeab6..ffb2e26 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -18,6 +18,7 @@
* We expect to be run in Non-secure mode, thus we define the
* group1 enable bits with respect to that view.
*/
+#define GICD_CTLR 0x0000
#define GICD_CTLR_RWP (1U << 31)
#define GICD_CTLR_ARE_NS (1U << 4)
#define GICD_CTLR_ENABLE_G1A (1U << 1)
@@ -36,6 +37,11 @@
#define GICR_ICENABLER0 GICD_ICENABLER
#define GICR_IPRIORITYR0 GICD_IPRIORITYR
+#define GICR_PROPBASER 0x0070
+#define GICR_PENDBASER 0x0078
+#define GICR_CTLR GICD_CTLR
+#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
+
#define ICC_SGI1R_AFFINITY_1_SHIFT 16
#define ICC_SGI1R_AFFINITY_2_SHIFT 32
#define ICC_SGI1R_AFFINITY_3_SHIFT 48
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
Detect the presence of an ITS as part of the GICv3 init
routine, initialize its base address and read few registers
the IIDR, the TYPER to store its dimensioning parameters.
Also parse the BASER registers.
This is our first ITS test, belonging to a new "its" group.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- updated dates and changed author
- squash "arm/arm64: ITS: Test BASER" into this patch but
removes setup_baser which will be introduced later.
- only compile on aarch64
- restrict the new test to aarch64
v1 -> v2:
- clean GITS_TYPER macros and unused fields in typer struct
- remove memory attribute related macros
- remove everything related to memory attributes
- s/dev_baser/coll_baser/ in report_info
- add extra line
- removed index filed in its_baser
---
arm/Makefile.arm64 | 1 +
arm/gic.c | 49 ++++++++++++++++++
arm/unittests.cfg | 7 +++
lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
lib/arm/gic.c | 30 +++++++++--
lib/arm64/asm/gic-v3-its.h | 1 +
7 files changed, 274 insertions(+), 5 deletions(-)
create mode 100644 lib/arm/asm/gic-v3-its.h
create mode 100644 lib/arm/gic-v3-its.c
create mode 100644 lib/arm64/asm/gic-v3-its.h
diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 6d3dc2c..2571ffb 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -19,6 +19,7 @@ endef
cstart.o = $(TEST_DIR)/cstart64.o
cflatobjs += lib/arm64/processor.o
cflatobjs += lib/arm64/spinlock.o
+cflatobjs += lib/arm/gic-v3-its.o
OBJDIRS += lib/arm64
diff --git a/arm/gic.c b/arm/gic.c
index abf08c7..4d7dd03 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -16,6 +16,7 @@
#include <asm/processor.h>
#include <asm/delay.h>
#include <asm/gic.h>
+#include <asm/gic-v3-its.h>
#include <asm/smp.h>
#include <asm/barrier.h>
#include <asm/io.h>
@@ -518,6 +519,50 @@ static void gic_test_mmio(void)
test_targets(nr_irqs);
}
+#if defined(__arm__)
+
+static void test_its_introspection(void) {}
+
+#else /* __arch64__ */
+
+static void test_its_introspection(void)
+{
+ struct its_baser *dev_baser, *coll_baser;
+ struct its_typer *typer = &its_data.typer;
+
+ if (!gicv3_its_base()) {
+ report_skip("No ITS, skip ...");
+ return;
+ }
+
+ /* IIDR */
+ report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
+ "GITS_IIDR is read-only"),
+
+ /* TYPER */
+ report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
+ "GITS_TYPER is read-only");
+
+ report(typer->phys_lpi, "ITS supports physical LPIs");
+ report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
+ report_info("ITT entry size = 0x%x", typer->ite_size);
+ report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
+ typer->eventid_bits, typer->deviceid_bits,
+ typer->collid_bits);
+ report(typer->eventid_bits && typer->deviceid_bits &&
+ typer->collid_bits, "ID spaces");
+ report_info("Target address format %s",
+ typer->pta ? "Redist basse address" : "PE #");
+
+ dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
+ coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
+ report(dev_baser && coll_baser, "detect device and collection BASER");
+ report_info("device baser entry_size = 0x%x", dev_baser->esz);
+ report_info("collection baser entry_size = 0x%x", coll_baser->esz);
+}
+
+#endif
+
int main(int argc, char **argv)
{
if (!gic_init()) {
@@ -549,6 +594,10 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
gic_test_mmio();
report_prefix_pop();
+ } else if (strcmp(argv[1], "its-introspection") == 0) {
+ report_prefix_push(argv[1]);
+ test_its_introspection();
+ report_prefix_pop();
} else {
report_abort("Unknown subtest '%s'", argv[1]);
}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index daeb5a0..ba2b31b 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -122,6 +122,13 @@ smp = $MAX_SMP
extra_params = -machine gic-version=3 -append 'active'
groups = gic
+[its-introspection]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'its-introspection'
+groups = its
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
new file mode 100644
index 0000000..815c515
--- /dev/null
+++ b/lib/arm/asm/gic-v3-its.h
@@ -0,0 +1,103 @@
+/*
+ * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
+ *
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V3_ITS_H_
+#define _ASMARM_GIC_V3_ITS_H_
+
+#ifndef __ASSEMBLY__
+
+struct its_typer {
+ unsigned int ite_size;
+ unsigned int eventid_bits;
+ unsigned int deviceid_bits;
+ unsigned int collid_bits;
+ bool pta;
+ bool phys_lpi;
+ bool virt_lpi;
+};
+
+struct its_baser {
+ int type;
+ size_t psz;
+ int nr_pages;
+ bool indirect;
+ phys_addr_t table_addr;
+ bool valid;
+ int esz;
+};
+
+#define GITS_BASER_NR_REGS 8
+
+struct its_data {
+ void *base;
+ struct its_typer typer;
+ struct its_baser baser[GITS_BASER_NR_REGS];
+};
+
+extern struct its_data its_data;
+
+#define gicv3_its_base() (its_data.base)
+
+#if defined(__aarch64__)
+
+#define GITS_CTLR 0x0000
+#define GITS_IIDR 0x0004
+#define GITS_TYPER 0x0008
+#define GITS_CBASER 0x0080
+#define GITS_CWRITER 0x0088
+#define GITS_CREADR 0x0090
+#define GITS_BASER 0x0100
+
+#define GITS_TYPER_PLPIS BIT(0)
+#define GITS_TYPER_VLPIS BIT(1)
+#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
+#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
+#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
+#define GITS_TYPER_IDBITS_SHIFT 8
+#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
+#define GITS_TYPER_DEVBITS_SHIFT 13
+#define GITS_TYPER_PTA BIT(19)
+#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
+#define GITS_TYPER_CIDBITS_SHIFT 32
+#define GITS_TYPER_CIL BIT(36)
+
+#define GITS_CTLR_ENABLE (1U << 0)
+
+#define GITS_CBASER_VALID (1UL << 63)
+
+#define GITS_BASER_VALID BIT(63)
+#define GITS_BASER_INDIRECT BIT(62)
+#define GITS_BASER_TYPE_SHIFT (56)
+#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
+#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
+#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
+#define GITS_BASER_PAGE_SIZE_SHIFT (8)
+#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGES_MAX 256
+#define GITS_BASER_PAGES_SHIFT (0)
+#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
+#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
+#define GITS_BASER_TYPE_NONE 0
+#define GITS_BASER_TYPE_DEVICE 1
+#define GITS_BASER_TYPE_COLLECTION 4
+
+extern void its_parse_typer(void);
+extern void its_init(void);
+extern int its_parse_baser(int i, struct its_baser *baser);
+extern struct its_baser *its_lookup_baser(int type);
+
+#else /* __arm__ */
+
+static inline void its_init(void) {}
+
+#endif
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V3_ITS_H_ */
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
new file mode 100644
index 0000000..2c0ce13
--- /dev/null
+++ b/lib/arm/gic-v3-its.c
@@ -0,0 +1,88 @@
+/*
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <asm/gic.h>
+#include <alloc_page.h>
+#include <asm/gic-v3-its.h>
+
+void its_parse_typer(void)
+{
+ u64 typer = readq(gicv3_its_base() + GITS_TYPER);
+
+ its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
+ GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
+ its_data.typer.pta = typer & GITS_TYPER_PTA;
+ its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
+ GITS_TYPER_IDBITS_SHIFT) + 1;
+ its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
+ GITS_TYPER_DEVBITS_SHIFT) + 1;
+
+ if (typer & GITS_TYPER_CIL)
+ its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
+ GITS_TYPER_CIDBITS_SHIFT) + 1;
+ else
+ its_data.typer.collid_bits = 16;
+
+ its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
+ its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
+}
+
+int its_parse_baser(int i, struct its_baser *baser)
+{
+ void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
+ u64 val = readq(reg_addr);
+
+ if (!val) {
+ memset(baser, 0, sizeof(*baser));
+ return -1;
+ }
+
+ baser->valid = val & GITS_BASER_VALID;
+ baser->indirect = val & GITS_BASER_INDIRECT;
+ baser->type = GITS_BASER_TYPE(val);
+ baser->esz = GITS_BASER_ENTRY_SIZE(val);
+ baser->nr_pages = GITS_BASER_NR_PAGES(val);
+ baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
+ switch (val & GITS_BASER_PAGE_SIZE_MASK) {
+ case GITS_BASER_PAGE_SIZE_4K:
+ baser->psz = SZ_4K;
+ break;
+ case GITS_BASER_PAGE_SIZE_16K:
+ baser->psz = SZ_16K;
+ break;
+ case GITS_BASER_PAGE_SIZE_64K:
+ baser->psz = SZ_64K;
+ break;
+ default:
+ baser->psz = SZ_64K;
+ }
+ return 0;
+}
+
+struct its_baser *its_lookup_baser(int type)
+{
+ int i;
+
+ for (i = 0; i < GITS_BASER_NR_REGS; i++) {
+ struct its_baser *baser = &its_data.baser[i];
+
+ if (baser->type == type)
+ return baser;
+ }
+ return NULL;
+}
+
+void its_init(void)
+{
+ int i;
+
+ if (!its_data.base)
+ return;
+
+ its_parse_typer();
+ for (i = 0; i < GITS_BASER_NR_REGS; i++)
+ its_parse_baser(i, &its_data.baser[i]);
+}
+
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index aa9cb86..6b70b05 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -6,9 +6,11 @@
#include <devicetree.h>
#include <asm/gic.h>
#include <asm/io.h>
+#include <asm/gic-v3-its.h>
struct gicv2_data gicv2_data;
struct gicv3_data gicv3_data;
+struct its_data its_data;
struct gic_common_ops {
void (*enable_defaults)(void);
@@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
* Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
*/
static bool
-gic_get_dt_bases(const char *compatible, void **base1, void **base2)
+gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
{
struct dt_pbus_reg reg;
- struct dt_device gic;
+ struct dt_device gic, its;
struct dt_bus bus;
- int node, ret, i;
+ int node, subnode, ret, i, len;
+ const void *fdt = dt_fdt();
dt_bus_init_defaults(&bus);
dt_device_init(&gic, &bus, NULL);
@@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
base2[i] = ioremap(reg.addr, reg.size);
}
+ if (base3 && !strcmp(compatible, "arm,gic-v3")) {
+ dt_for_each_subnode(node, subnode) {
+ const struct fdt_property *prop;
+
+ prop = fdt_get_property(fdt, subnode, "compatible", &len);
+ if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
+ dt_device_bind_node(&its, subnode);
+ ret = dt_pbus_translate(&its, 0, ®);
+ assert(ret == 0);
+ *base3 = ioremap(reg.addr, reg.size);
+ break;
+ }
+ }
+
+ }
+
return true;
}
int gicv2_init(void)
{
return gic_get_dt_bases("arm,cortex-a15-gic",
- &gicv2_data.dist_base, &gicv2_data.cpu_base);
+ &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
}
int gicv3_init(void)
{
return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
- &gicv3_data.redist_bases[0]);
+ &gicv3_data.redist_bases[0], &its_data.base);
}
int gic_version(void)
@@ -104,6 +123,7 @@ int gic_init(void)
gic_common_ops = &gicv2_common_ops;
else if (gicv3_init())
gic_common_ops = &gicv3_common_ops;
+ its_init();
return gic_version();
}
diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
new file mode 100644
index 0000000..083cba4
--- /dev/null
+++ b/lib/arm64/asm/gic-v3-its.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v3-its.h"
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Detect the presence of an ITS as part of the GICv3 init
routine, initialize its base address and read few registers
the IIDR, the TYPER to store its dimensioning parameters.
Also parse the BASER registers.
This is our first ITS test, belonging to a new "its" group.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- updated dates and changed author
- squash "arm/arm64: ITS: Test BASER" into this patch but
removes setup_baser which will be introduced later.
- only compile on aarch64
- restrict the new test to aarch64
v1 -> v2:
- clean GITS_TYPER macros and unused fields in typer struct
- remove memory attribute related macros
- remove everything related to memory attributes
- s/dev_baser/coll_baser/ in report_info
- add extra line
- removed index filed in its_baser
---
arm/Makefile.arm64 | 1 +
arm/gic.c | 49 ++++++++++++++++++
arm/unittests.cfg | 7 +++
lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
lib/arm/gic.c | 30 +++++++++--
lib/arm64/asm/gic-v3-its.h | 1 +
7 files changed, 274 insertions(+), 5 deletions(-)
create mode 100644 lib/arm/asm/gic-v3-its.h
create mode 100644 lib/arm/gic-v3-its.c
create mode 100644 lib/arm64/asm/gic-v3-its.h
diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 6d3dc2c..2571ffb 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -19,6 +19,7 @@ endef
cstart.o = $(TEST_DIR)/cstart64.o
cflatobjs += lib/arm64/processor.o
cflatobjs += lib/arm64/spinlock.o
+cflatobjs += lib/arm/gic-v3-its.o
OBJDIRS += lib/arm64
diff --git a/arm/gic.c b/arm/gic.c
index abf08c7..4d7dd03 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -16,6 +16,7 @@
#include <asm/processor.h>
#include <asm/delay.h>
#include <asm/gic.h>
+#include <asm/gic-v3-its.h>
#include <asm/smp.h>
#include <asm/barrier.h>
#include <asm/io.h>
@@ -518,6 +519,50 @@ static void gic_test_mmio(void)
test_targets(nr_irqs);
}
+#if defined(__arm__)
+
+static void test_its_introspection(void) {}
+
+#else /* __arch64__ */
+
+static void test_its_introspection(void)
+{
+ struct its_baser *dev_baser, *coll_baser;
+ struct its_typer *typer = &its_data.typer;
+
+ if (!gicv3_its_base()) {
+ report_skip("No ITS, skip ...");
+ return;
+ }
+
+ /* IIDR */
+ report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
+ "GITS_IIDR is read-only"),
+
+ /* TYPER */
+ report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
+ "GITS_TYPER is read-only");
+
+ report(typer->phys_lpi, "ITS supports physical LPIs");
+ report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
+ report_info("ITT entry size = 0x%x", typer->ite_size);
+ report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
+ typer->eventid_bits, typer->deviceid_bits,
+ typer->collid_bits);
+ report(typer->eventid_bits && typer->deviceid_bits &&
+ typer->collid_bits, "ID spaces");
+ report_info("Target address format %s",
+ typer->pta ? "Redist basse address" : "PE #");
+
+ dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
+ coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
+ report(dev_baser && coll_baser, "detect device and collection BASER");
+ report_info("device baser entry_size = 0x%x", dev_baser->esz);
+ report_info("collection baser entry_size = 0x%x", coll_baser->esz);
+}
+
+#endif
+
int main(int argc, char **argv)
{
if (!gic_init()) {
@@ -549,6 +594,10 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
gic_test_mmio();
report_prefix_pop();
+ } else if (strcmp(argv[1], "its-introspection") == 0) {
+ report_prefix_push(argv[1]);
+ test_its_introspection();
+ report_prefix_pop();
} else {
report_abort("Unknown subtest '%s'", argv[1]);
}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index daeb5a0..ba2b31b 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -122,6 +122,13 @@ smp = $MAX_SMP
extra_params = -machine gic-version=3 -append 'active'
groups = gic
+[its-introspection]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'its-introspection'
+groups = its
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
new file mode 100644
index 0000000..815c515
--- /dev/null
+++ b/lib/arm/asm/gic-v3-its.h
@@ -0,0 +1,103 @@
+/*
+ * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
+ *
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V3_ITS_H_
+#define _ASMARM_GIC_V3_ITS_H_
+
+#ifndef __ASSEMBLY__
+
+struct its_typer {
+ unsigned int ite_size;
+ unsigned int eventid_bits;
+ unsigned int deviceid_bits;
+ unsigned int collid_bits;
+ bool pta;
+ bool phys_lpi;
+ bool virt_lpi;
+};
+
+struct its_baser {
+ int type;
+ size_t psz;
+ int nr_pages;
+ bool indirect;
+ phys_addr_t table_addr;
+ bool valid;
+ int esz;
+};
+
+#define GITS_BASER_NR_REGS 8
+
+struct its_data {
+ void *base;
+ struct its_typer typer;
+ struct its_baser baser[GITS_BASER_NR_REGS];
+};
+
+extern struct its_data its_data;
+
+#define gicv3_its_base() (its_data.base)
+
+#if defined(__aarch64__)
+
+#define GITS_CTLR 0x0000
+#define GITS_IIDR 0x0004
+#define GITS_TYPER 0x0008
+#define GITS_CBASER 0x0080
+#define GITS_CWRITER 0x0088
+#define GITS_CREADR 0x0090
+#define GITS_BASER 0x0100
+
+#define GITS_TYPER_PLPIS BIT(0)
+#define GITS_TYPER_VLPIS BIT(1)
+#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
+#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
+#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
+#define GITS_TYPER_IDBITS_SHIFT 8
+#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
+#define GITS_TYPER_DEVBITS_SHIFT 13
+#define GITS_TYPER_PTA BIT(19)
+#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
+#define GITS_TYPER_CIDBITS_SHIFT 32
+#define GITS_TYPER_CIL BIT(36)
+
+#define GITS_CTLR_ENABLE (1U << 0)
+
+#define GITS_CBASER_VALID (1UL << 63)
+
+#define GITS_BASER_VALID BIT(63)
+#define GITS_BASER_INDIRECT BIT(62)
+#define GITS_BASER_TYPE_SHIFT (56)
+#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
+#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
+#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
+#define GITS_BASER_PAGE_SIZE_SHIFT (8)
+#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGES_MAX 256
+#define GITS_BASER_PAGES_SHIFT (0)
+#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
+#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
+#define GITS_BASER_TYPE_NONE 0
+#define GITS_BASER_TYPE_DEVICE 1
+#define GITS_BASER_TYPE_COLLECTION 4
+
+extern void its_parse_typer(void);
+extern void its_init(void);
+extern int its_parse_baser(int i, struct its_baser *baser);
+extern struct its_baser *its_lookup_baser(int type);
+
+#else /* __arm__ */
+
+static inline void its_init(void) {}
+
+#endif
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V3_ITS_H_ */
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
new file mode 100644
index 0000000..2c0ce13
--- /dev/null
+++ b/lib/arm/gic-v3-its.c
@@ -0,0 +1,88 @@
+/*
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <asm/gic.h>
+#include <alloc_page.h>
+#include <asm/gic-v3-its.h>
+
+void its_parse_typer(void)
+{
+ u64 typer = readq(gicv3_its_base() + GITS_TYPER);
+
+ its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
+ GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
+ its_data.typer.pta = typer & GITS_TYPER_PTA;
+ its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
+ GITS_TYPER_IDBITS_SHIFT) + 1;
+ its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
+ GITS_TYPER_DEVBITS_SHIFT) + 1;
+
+ if (typer & GITS_TYPER_CIL)
+ its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
+ GITS_TYPER_CIDBITS_SHIFT) + 1;
+ else
+ its_data.typer.collid_bits = 16;
+
+ its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
+ its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
+}
+
+int its_parse_baser(int i, struct its_baser *baser)
+{
+ void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
+ u64 val = readq(reg_addr);
+
+ if (!val) {
+ memset(baser, 0, sizeof(*baser));
+ return -1;
+ }
+
+ baser->valid = val & GITS_BASER_VALID;
+ baser->indirect = val & GITS_BASER_INDIRECT;
+ baser->type = GITS_BASER_TYPE(val);
+ baser->esz = GITS_BASER_ENTRY_SIZE(val);
+ baser->nr_pages = GITS_BASER_NR_PAGES(val);
+ baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
+ switch (val & GITS_BASER_PAGE_SIZE_MASK) {
+ case GITS_BASER_PAGE_SIZE_4K:
+ baser->psz = SZ_4K;
+ break;
+ case GITS_BASER_PAGE_SIZE_16K:
+ baser->psz = SZ_16K;
+ break;
+ case GITS_BASER_PAGE_SIZE_64K:
+ baser->psz = SZ_64K;
+ break;
+ default:
+ baser->psz = SZ_64K;
+ }
+ return 0;
+}
+
+struct its_baser *its_lookup_baser(int type)
+{
+ int i;
+
+ for (i = 0; i < GITS_BASER_NR_REGS; i++) {
+ struct its_baser *baser = &its_data.baser[i];
+
+ if (baser->type == type)
+ return baser;
+ }
+ return NULL;
+}
+
+void its_init(void)
+{
+ int i;
+
+ if (!its_data.base)
+ return;
+
+ its_parse_typer();
+ for (i = 0; i < GITS_BASER_NR_REGS; i++)
+ its_parse_baser(i, &its_data.baser[i]);
+}
+
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index aa9cb86..6b70b05 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -6,9 +6,11 @@
#include <devicetree.h>
#include <asm/gic.h>
#include <asm/io.h>
+#include <asm/gic-v3-its.h>
struct gicv2_data gicv2_data;
struct gicv3_data gicv3_data;
+struct its_data its_data;
struct gic_common_ops {
void (*enable_defaults)(void);
@@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
* Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
*/
static bool
-gic_get_dt_bases(const char *compatible, void **base1, void **base2)
+gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
{
struct dt_pbus_reg reg;
- struct dt_device gic;
+ struct dt_device gic, its;
struct dt_bus bus;
- int node, ret, i;
+ int node, subnode, ret, i, len;
+ const void *fdt = dt_fdt();
dt_bus_init_defaults(&bus);
dt_device_init(&gic, &bus, NULL);
@@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
base2[i] = ioremap(reg.addr, reg.size);
}
+ if (base3 && !strcmp(compatible, "arm,gic-v3")) {
+ dt_for_each_subnode(node, subnode) {
+ const struct fdt_property *prop;
+
+ prop = fdt_get_property(fdt, subnode, "compatible", &len);
+ if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
+ dt_device_bind_node(&its, subnode);
+ ret = dt_pbus_translate(&its, 0, ®);
+ assert(ret == 0);
+ *base3 = ioremap(reg.addr, reg.size);
+ break;
+ }
+ }
+
+ }
+
return true;
}
int gicv2_init(void)
{
return gic_get_dt_bases("arm,cortex-a15-gic",
- &gicv2_data.dist_base, &gicv2_data.cpu_base);
+ &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
}
int gicv3_init(void)
{
return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
- &gicv3_data.redist_bases[0]);
+ &gicv3_data.redist_bases[0], &its_data.base);
}
int gic_version(void)
@@ -104,6 +123,7 @@ int gic_init(void)
gic_common_ops = &gicv2_common_ops;
else if (gicv3_init())
gic_common_ops = &gicv3_common_ops;
+ its_init();
return gic_version();
}
diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
new file mode 100644
index 0000000..083cba4
--- /dev/null
+++ b/lib/arm64/asm/gic-v3-its.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v3-its.h"
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Detect the presence of an ITS as part of the GICv3 init
routine, initialize its base address and read few registers
the IIDR, the TYPER to store its dimensioning parameters.
Also parse the BASER registers.
This is our first ITS test, belonging to a new "its" group.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- updated dates and changed author
- squash "arm/arm64: ITS: Test BASER" into this patch but
removes setup_baser which will be introduced later.
- only compile on aarch64
- restrict the new test to aarch64
v1 -> v2:
- clean GITS_TYPER macros and unused fields in typer struct
- remove memory attribute related macros
- remove everything related to memory attributes
- s/dev_baser/coll_baser/ in report_info
- add extra line
- removed index filed in its_baser
---
arm/Makefile.arm64 | 1 +
arm/gic.c | 49 ++++++++++++++++++
arm/unittests.cfg | 7 +++
lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
lib/arm/gic.c | 30 +++++++++--
lib/arm64/asm/gic-v3-its.h | 1 +
7 files changed, 274 insertions(+), 5 deletions(-)
create mode 100644 lib/arm/asm/gic-v3-its.h
create mode 100644 lib/arm/gic-v3-its.c
create mode 100644 lib/arm64/asm/gic-v3-its.h
diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 6d3dc2c..2571ffb 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -19,6 +19,7 @@ endef
cstart.o = $(TEST_DIR)/cstart64.o
cflatobjs += lib/arm64/processor.o
cflatobjs += lib/arm64/spinlock.o
+cflatobjs += lib/arm/gic-v3-its.o
OBJDIRS += lib/arm64
diff --git a/arm/gic.c b/arm/gic.c
index abf08c7..4d7dd03 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -16,6 +16,7 @@
#include <asm/processor.h>
#include <asm/delay.h>
#include <asm/gic.h>
+#include <asm/gic-v3-its.h>
#include <asm/smp.h>
#include <asm/barrier.h>
#include <asm/io.h>
@@ -518,6 +519,50 @@ static void gic_test_mmio(void)
test_targets(nr_irqs);
}
+#if defined(__arm__)
+
+static void test_its_introspection(void) {}
+
+#else /* __arch64__ */
+
+static void test_its_introspection(void)
+{
+ struct its_baser *dev_baser, *coll_baser;
+ struct its_typer *typer = &its_data.typer;
+
+ if (!gicv3_its_base()) {
+ report_skip("No ITS, skip ...");
+ return;
+ }
+
+ /* IIDR */
+ report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
+ "GITS_IIDR is read-only"),
+
+ /* TYPER */
+ report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
+ "GITS_TYPER is read-only");
+
+ report(typer->phys_lpi, "ITS supports physical LPIs");
+ report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
+ report_info("ITT entry size = 0x%x", typer->ite_size);
+ report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
+ typer->eventid_bits, typer->deviceid_bits,
+ typer->collid_bits);
+ report(typer->eventid_bits && typer->deviceid_bits &&
+ typer->collid_bits, "ID spaces");
+ report_info("Target address format %s",
+ typer->pta ? "Redist basse address" : "PE #");
+
+ dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
+ coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
+ report(dev_baser && coll_baser, "detect device and collection BASER");
+ report_info("device baser entry_size = 0x%x", dev_baser->esz);
+ report_info("collection baser entry_size = 0x%x", coll_baser->esz);
+}
+
+#endif
+
int main(int argc, char **argv)
{
if (!gic_init()) {
@@ -549,6 +594,10 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
gic_test_mmio();
report_prefix_pop();
+ } else if (strcmp(argv[1], "its-introspection") == 0) {
+ report_prefix_push(argv[1]);
+ test_its_introspection();
+ report_prefix_pop();
} else {
report_abort("Unknown subtest '%s'", argv[1]);
}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index daeb5a0..ba2b31b 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -122,6 +122,13 @@ smp = $MAX_SMP
extra_params = -machine gic-version=3 -append 'active'
groups = gic
+[its-introspection]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'its-introspection'
+groups = its
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
new file mode 100644
index 0000000..815c515
--- /dev/null
+++ b/lib/arm/asm/gic-v3-its.h
@@ -0,0 +1,103 @@
+/*
+ * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
+ *
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V3_ITS_H_
+#define _ASMARM_GIC_V3_ITS_H_
+
+#ifndef __ASSEMBLY__
+
+struct its_typer {
+ unsigned int ite_size;
+ unsigned int eventid_bits;
+ unsigned int deviceid_bits;
+ unsigned int collid_bits;
+ bool pta;
+ bool phys_lpi;
+ bool virt_lpi;
+};
+
+struct its_baser {
+ int type;
+ size_t psz;
+ int nr_pages;
+ bool indirect;
+ phys_addr_t table_addr;
+ bool valid;
+ int esz;
+};
+
+#define GITS_BASER_NR_REGS 8
+
+struct its_data {
+ void *base;
+ struct its_typer typer;
+ struct its_baser baser[GITS_BASER_NR_REGS];
+};
+
+extern struct its_data its_data;
+
+#define gicv3_its_base() (its_data.base)
+
+#if defined(__aarch64__)
+
+#define GITS_CTLR 0x0000
+#define GITS_IIDR 0x0004
+#define GITS_TYPER 0x0008
+#define GITS_CBASER 0x0080
+#define GITS_CWRITER 0x0088
+#define GITS_CREADR 0x0090
+#define GITS_BASER 0x0100
+
+#define GITS_TYPER_PLPIS BIT(0)
+#define GITS_TYPER_VLPIS BIT(1)
+#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
+#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
+#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
+#define GITS_TYPER_IDBITS_SHIFT 8
+#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
+#define GITS_TYPER_DEVBITS_SHIFT 13
+#define GITS_TYPER_PTA BIT(19)
+#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
+#define GITS_TYPER_CIDBITS_SHIFT 32
+#define GITS_TYPER_CIL BIT(36)
+
+#define GITS_CTLR_ENABLE (1U << 0)
+
+#define GITS_CBASER_VALID (1UL << 63)
+
+#define GITS_BASER_VALID BIT(63)
+#define GITS_BASER_INDIRECT BIT(62)
+#define GITS_BASER_TYPE_SHIFT (56)
+#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
+#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
+#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
+#define GITS_BASER_PAGE_SIZE_SHIFT (8)
+#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGES_MAX 256
+#define GITS_BASER_PAGES_SHIFT (0)
+#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
+#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
+#define GITS_BASER_TYPE_NONE 0
+#define GITS_BASER_TYPE_DEVICE 1
+#define GITS_BASER_TYPE_COLLECTION 4
+
+extern void its_parse_typer(void);
+extern void its_init(void);
+extern int its_parse_baser(int i, struct its_baser *baser);
+extern struct its_baser *its_lookup_baser(int type);
+
+#else /* __arm__ */
+
+static inline void its_init(void) {}
+
+#endif
+
+#endif /* !__ASSEMBLY__ */
+#endif /* _ASMARM_GIC_V3_ITS_H_ */
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
new file mode 100644
index 0000000..2c0ce13
--- /dev/null
+++ b/lib/arm/gic-v3-its.c
@@ -0,0 +1,88 @@
+/*
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <asm/gic.h>
+#include <alloc_page.h>
+#include <asm/gic-v3-its.h>
+
+void its_parse_typer(void)
+{
+ u64 typer = readq(gicv3_its_base() + GITS_TYPER);
+
+ its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
+ GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
+ its_data.typer.pta = typer & GITS_TYPER_PTA;
+ its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
+ GITS_TYPER_IDBITS_SHIFT) + 1;
+ its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
+ GITS_TYPER_DEVBITS_SHIFT) + 1;
+
+ if (typer & GITS_TYPER_CIL)
+ its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
+ GITS_TYPER_CIDBITS_SHIFT) + 1;
+ else
+ its_data.typer.collid_bits = 16;
+
+ its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
+ its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
+}
+
+int its_parse_baser(int i, struct its_baser *baser)
+{
+ void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
+ u64 val = readq(reg_addr);
+
+ if (!val) {
+ memset(baser, 0, sizeof(*baser));
+ return -1;
+ }
+
+ baser->valid = val & GITS_BASER_VALID;
+ baser->indirect = val & GITS_BASER_INDIRECT;
+ baser->type = GITS_BASER_TYPE(val);
+ baser->esz = GITS_BASER_ENTRY_SIZE(val);
+ baser->nr_pages = GITS_BASER_NR_PAGES(val);
+ baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
+ switch (val & GITS_BASER_PAGE_SIZE_MASK) {
+ case GITS_BASER_PAGE_SIZE_4K:
+ baser->psz = SZ_4K;
+ break;
+ case GITS_BASER_PAGE_SIZE_16K:
+ baser->psz = SZ_16K;
+ break;
+ case GITS_BASER_PAGE_SIZE_64K:
+ baser->psz = SZ_64K;
+ break;
+ default:
+ baser->psz = SZ_64K;
+ }
+ return 0;
+}
+
+struct its_baser *its_lookup_baser(int type)
+{
+ int i;
+
+ for (i = 0; i < GITS_BASER_NR_REGS; i++) {
+ struct its_baser *baser = &its_data.baser[i];
+
+ if (baser->type == type)
+ return baser;
+ }
+ return NULL;
+}
+
+void its_init(void)
+{
+ int i;
+
+ if (!its_data.base)
+ return;
+
+ its_parse_typer();
+ for (i = 0; i < GITS_BASER_NR_REGS; i++)
+ its_parse_baser(i, &its_data.baser[i]);
+}
+
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index aa9cb86..6b70b05 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -6,9 +6,11 @@
#include <devicetree.h>
#include <asm/gic.h>
#include <asm/io.h>
+#include <asm/gic-v3-its.h>
struct gicv2_data gicv2_data;
struct gicv3_data gicv3_data;
+struct its_data its_data;
struct gic_common_ops {
void (*enable_defaults)(void);
@@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
* Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
*/
static bool
-gic_get_dt_bases(const char *compatible, void **base1, void **base2)
+gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
{
struct dt_pbus_reg reg;
- struct dt_device gic;
+ struct dt_device gic, its;
struct dt_bus bus;
- int node, ret, i;
+ int node, subnode, ret, i, len;
+ const void *fdt = dt_fdt();
dt_bus_init_defaults(&bus);
dt_device_init(&gic, &bus, NULL);
@@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
base2[i] = ioremap(reg.addr, reg.size);
}
+ if (base3 && !strcmp(compatible, "arm,gic-v3")) {
+ dt_for_each_subnode(node, subnode) {
+ const struct fdt_property *prop;
+
+ prop = fdt_get_property(fdt, subnode, "compatible", &len);
+ if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
+ dt_device_bind_node(&its, subnode);
+ ret = dt_pbus_translate(&its, 0, ®);
+ assert(ret == 0);
+ *base3 = ioremap(reg.addr, reg.size);
+ break;
+ }
+ }
+
+ }
+
return true;
}
int gicv2_init(void)
{
return gic_get_dt_bases("arm,cortex-a15-gic",
- &gicv2_data.dist_base, &gicv2_data.cpu_base);
+ &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
}
int gicv3_init(void)
{
return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
- &gicv3_data.redist_bases[0]);
+ &gicv3_data.redist_bases[0], &its_data.base);
}
int gic_version(void)
@@ -104,6 +123,7 @@ int gic_init(void)
gic_common_ops = &gicv2_common_ops;
else if (gicv3_init())
gic_common_ops = &gicv3_common_ops;
+ its_init();
return gic_version();
}
diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
new file mode 100644
index 0000000..083cba4
--- /dev/null
+++ b/lib/arm64/asm/gic-v3-its.h
@@ -0,0 +1 @@
+#include "../../arm/asm/gic-v3-its.h"
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
Allocate the LPI configuration and per re-distributor pending table.
Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
by default in the config table.
Also introduce a helper routine that allows to set the pending table
bit for a given LPI.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
and add _lpi prefix too
v1 -> v2:
- remove memory attributes
---
lib/arm/asm/gic-v3.h | 16 +++++++++++
lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 80 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index ffb2e26..ec2a6f0 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -48,6 +48,16 @@
#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
(MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
+#define GICR_PROPBASER_IDBITS_MASK (0x1f)
+
+#define GICR_PENDBASER_PTZ BIT_ULL(62)
+
+#define LPI_PROP_GROUP1 (1 << 1)
+#define LPI_PROP_ENABLED (1 << 0)
+#define LPI_PROP_DEFAULT_PRIO 0xa0
+#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
+ LPI_PROP_ENABLED)
+
#include <asm/arch_gicv3.h>
#ifndef __ASSEMBLY__
@@ -64,6 +74,8 @@ struct gicv3_data {
void *dist_base;
void *redist_bases[GICV3_NR_REDISTS];
void *redist_base[NR_CPUS];
+ void *lpi_prop;
+ void *lpi_pend[NR_CPUS];
unsigned int irq_nr;
};
extern struct gicv3_data gicv3_data;
@@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
extern void gicv3_ipi_send_single(int irq, int cpu);
extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
extern void gicv3_set_redist_base(size_t stride);
+extern void gicv3_lpi_set_config(int n, u8 val);
+extern u8 gicv3_lpi_get_config(int n);
+extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
+extern void gicv3_lpi_alloc_tables(void);
static inline void gicv3_do_wait_for_rwp(void *base)
{
diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
index feecb5e..c33f883 100644
--- a/lib/arm/gic-v3.c
+++ b/lib/arm/gic-v3.c
@@ -5,6 +5,7 @@
*/
#include <asm/gic.h>
#include <asm/io.h>
+#include <alloc_page.h>
void gicv3_set_redist_base(size_t stride)
{
@@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
cpumask_set_cpu(cpu, &dest);
gicv3_ipi_send_mask(irq, &dest);
}
+
+#if defined(__aarch64__)
+/* alloc_lpi_tables: Allocate LPI config and pending tables */
+void gicv3_lpi_alloc_tables(void)
+{
+ unsigned long n = SZ_64K >> PAGE_SHIFT;
+ unsigned long order = fls(n);
+ u64 prop_val;
+ int cpu;
+
+ gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
+
+ /* ID bits = 13, ie. up to 14b LPI INTID */
+ prop_val = (u64)gicv3_data.lpi_prop | 13;
+
+ /*
+ * Allocate pending tables for each redistributor
+ * and set PROPBASER and PENDBASER
+ */
+ for_each_present_cpu(cpu) {
+ u64 pend_val;
+ void *ptr;
+
+ ptr = gicv3_data.redist_base[cpu];
+
+ writeq(prop_val, ptr + GICR_PROPBASER);
+
+ gicv3_data.lpi_pend[cpu] = (void *)virt_to_phys(alloc_pages(order));
+
+ pend_val = (u64)gicv3_data.lpi_pend[cpu];
+
+ writeq(pend_val, ptr + GICR_PENDBASER);
+ }
+}
+
+void gicv3_lpi_set_config(int n, u8 value)
+{
+ u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
+
+ *entry = value;
+}
+
+u8 gicv3_lpi_get_config(int n)
+{
+ u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
+
+ return *entry;
+}
+
+void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
+{
+ u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
+ u8 mask = 1 << (n % 8), byte;
+
+ ptr += (n / 8);
+ byte = *ptr;
+ if (set)
+ byte |= mask;
+ else
+ byte &= ~mask;
+ *ptr = byte;
+}
+#endif /* __aarch64__ */
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Allocate the LPI configuration and per re-distributor pending table.
Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
by default in the config table.
Also introduce a helper routine that allows to set the pending table
bit for a given LPI.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
and add _lpi prefix too
v1 -> v2:
- remove memory attributes
---
lib/arm/asm/gic-v3.h | 16 +++++++++++
lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 80 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index ffb2e26..ec2a6f0 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -48,6 +48,16 @@
#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
(MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
+#define GICR_PROPBASER_IDBITS_MASK (0x1f)
+
+#define GICR_PENDBASER_PTZ BIT_ULL(62)
+
+#define LPI_PROP_GROUP1 (1 << 1)
+#define LPI_PROP_ENABLED (1 << 0)
+#define LPI_PROP_DEFAULT_PRIO 0xa0
+#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
+ LPI_PROP_ENABLED)
+
#include <asm/arch_gicv3.h>
#ifndef __ASSEMBLY__
@@ -64,6 +74,8 @@ struct gicv3_data {
void *dist_base;
void *redist_bases[GICV3_NR_REDISTS];
void *redist_base[NR_CPUS];
+ void *lpi_prop;
+ void *lpi_pend[NR_CPUS];
unsigned int irq_nr;
};
extern struct gicv3_data gicv3_data;
@@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
extern void gicv3_ipi_send_single(int irq, int cpu);
extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
extern void gicv3_set_redist_base(size_t stride);
+extern void gicv3_lpi_set_config(int n, u8 val);
+extern u8 gicv3_lpi_get_config(int n);
+extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
+extern void gicv3_lpi_alloc_tables(void);
static inline void gicv3_do_wait_for_rwp(void *base)
{
diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
index feecb5e..c33f883 100644
--- a/lib/arm/gic-v3.c
+++ b/lib/arm/gic-v3.c
@@ -5,6 +5,7 @@
*/
#include <asm/gic.h>
#include <asm/io.h>
+#include <alloc_page.h>
void gicv3_set_redist_base(size_t stride)
{
@@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
cpumask_set_cpu(cpu, &dest);
gicv3_ipi_send_mask(irq, &dest);
}
+
+#if defined(__aarch64__)
+/* alloc_lpi_tables: Allocate LPI config and pending tables */
+void gicv3_lpi_alloc_tables(void)
+{
+ unsigned long n = SZ_64K >> PAGE_SHIFT;
+ unsigned long order = fls(n);
+ u64 prop_val;
+ int cpu;
+
+ gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
+
+ /* ID bits = 13, ie. up to 14b LPI INTID */
+ prop_val = (u64)gicv3_data.lpi_prop | 13;
+
+ /*
+ * Allocate pending tables for each redistributor
+ * and set PROPBASER and PENDBASER
+ */
+ for_each_present_cpu(cpu) {
+ u64 pend_val;
+ void *ptr;
+
+ ptr = gicv3_data.redist_base[cpu];
+
+ writeq(prop_val, ptr + GICR_PROPBASER);
+
+ gicv3_data.lpi_pend[cpu] = (void *)virt_to_phys(alloc_pages(order));
+
+ pend_val = (u64)gicv3_data.lpi_pend[cpu];
+
+ writeq(pend_val, ptr + GICR_PENDBASER);
+ }
+}
+
+void gicv3_lpi_set_config(int n, u8 value)
+{
+ u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
+
+ *entry = value;
+}
+
+u8 gicv3_lpi_get_config(int n)
+{
+ u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
+
+ return *entry;
+}
+
+void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
+{
+ u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
+ u8 mask = 1 << (n % 8), byte;
+
+ ptr += (n / 8);
+ byte = *ptr;
+ if (set)
+ byte |= mask;
+ else
+ byte &= ~mask;
+ *ptr = byte;
+}
+#endif /* __aarch64__ */
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Allocate the LPI configuration and per re-distributor pending table.
Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
by default in the config table.
Also introduce a helper routine that allows to set the pending table
bit for a given LPI.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
and add _lpi prefix too
v1 -> v2:
- remove memory attributes
---
lib/arm/asm/gic-v3.h | 16 +++++++++++
lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 80 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index ffb2e26..ec2a6f0 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -48,6 +48,16 @@
#define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
(MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
+#define GICR_PROPBASER_IDBITS_MASK (0x1f)
+
+#define GICR_PENDBASER_PTZ BIT_ULL(62)
+
+#define LPI_PROP_GROUP1 (1 << 1)
+#define LPI_PROP_ENABLED (1 << 0)
+#define LPI_PROP_DEFAULT_PRIO 0xa0
+#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
+ LPI_PROP_ENABLED)
+
#include <asm/arch_gicv3.h>
#ifndef __ASSEMBLY__
@@ -64,6 +74,8 @@ struct gicv3_data {
void *dist_base;
void *redist_bases[GICV3_NR_REDISTS];
void *redist_base[NR_CPUS];
+ void *lpi_prop;
+ void *lpi_pend[NR_CPUS];
unsigned int irq_nr;
};
extern struct gicv3_data gicv3_data;
@@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
extern void gicv3_ipi_send_single(int irq, int cpu);
extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
extern void gicv3_set_redist_base(size_t stride);
+extern void gicv3_lpi_set_config(int n, u8 val);
+extern u8 gicv3_lpi_get_config(int n);
+extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
+extern void gicv3_lpi_alloc_tables(void);
static inline void gicv3_do_wait_for_rwp(void *base)
{
diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
index feecb5e..c33f883 100644
--- a/lib/arm/gic-v3.c
+++ b/lib/arm/gic-v3.c
@@ -5,6 +5,7 @@
*/
#include <asm/gic.h>
#include <asm/io.h>
+#include <alloc_page.h>
void gicv3_set_redist_base(size_t stride)
{
@@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
cpumask_set_cpu(cpu, &dest);
gicv3_ipi_send_mask(irq, &dest);
}
+
+#if defined(__aarch64__)
+/* alloc_lpi_tables: Allocate LPI config and pending tables */
+void gicv3_lpi_alloc_tables(void)
+{
+ unsigned long n = SZ_64K >> PAGE_SHIFT;
+ unsigned long order = fls(n);
+ u64 prop_val;
+ int cpu;
+
+ gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
+
+ /* ID bits = 13, ie. up to 14b LPI INTID */
+ prop_val = (u64)gicv3_data.lpi_prop | 13;
+
+ /*
+ * Allocate pending tables for each redistributor
+ * and set PROPBASER and PENDBASER
+ */
+ for_each_present_cpu(cpu) {
+ u64 pend_val;
+ void *ptr;
+
+ ptr = gicv3_data.redist_base[cpu];
+
+ writeq(prop_val, ptr + GICR_PROPBASER);
+
+ gicv3_data.lpi_pend[cpu] = (void *)virt_to_phys(alloc_pages(order));
+
+ pend_val = (u64)gicv3_data.lpi_pend[cpu];
+
+ writeq(pend_val, ptr + GICR_PENDBASER);
+ }
+}
+
+void gicv3_lpi_set_config(int n, u8 value)
+{
+ u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
+
+ *entry = value;
+}
+
+u8 gicv3_lpi_get_config(int n)
+{
+ u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
+
+ return *entry;
+}
+
+void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
+{
+ u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
+ u8 mask = 1 << (n % 8), byte;
+
+ ptr += (n / 8);
+ byte = *ptr;
+ if (set)
+ byte |= mask;
+ else
+ byte &= ~mask;
+ *ptr = byte;
+}
+#endif /* __aarch64__ */
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 07/14] arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
This helper function controls the signaling of LPIs at
redistributor level.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- move the helper in lib/arm/gic-v3.c
- rename the function with gicv3_lpi_ prefix
- s/report_abort/assert
---
lib/arm/asm/gic-v3.h | 1 +
lib/arm/gic-v3.c | 17 +++++++++++++++++
2 files changed, 18 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index ec2a6f0..734c0c0 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -96,6 +96,7 @@ extern void gicv3_lpi_set_config(int n, u8 val);
extern u8 gicv3_lpi_get_config(int n);
extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
extern void gicv3_lpi_alloc_tables(void);
+extern void gicv3_lpi_rdist_ctrl(u32 redist, bool set);
static inline void gicv3_do_wait_for_rwp(void *base)
{
diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
index c33f883..7865d01 100644
--- a/lib/arm/gic-v3.c
+++ b/lib/arm/gic-v3.c
@@ -210,4 +210,21 @@ void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
byte &= ~mask;
*ptr = byte;
}
+
+void gicv3_lpi_rdist_ctrl(u32 redist, bool set)
+{
+ void *ptr;
+ u64 val;
+
+ assert(redist < nr_cpus);
+
+ ptr = gicv3_data.redist_base[redist];
+ val = readl(ptr + GICR_CTLR);
+ if (set)
+ val |= GICR_CTLR_ENABLE_LPIS;
+ else
+ val &= ~GICR_CTLR_ENABLE_LPIS;
+ writel(val, ptr + GICR_CTLR);
+}
#endif /* __aarch64__ */
+
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 07/14] arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
This helper function controls the signaling of LPIs at
redistributor level.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- move the helper in lib/arm/gic-v3.c
- rename the function with gicv3_lpi_ prefix
- s/report_abort/assert
---
lib/arm/asm/gic-v3.h | 1 +
lib/arm/gic-v3.c | 17 +++++++++++++++++
2 files changed, 18 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index ec2a6f0..734c0c0 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -96,6 +96,7 @@ extern void gicv3_lpi_set_config(int n, u8 val);
extern u8 gicv3_lpi_get_config(int n);
extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
extern void gicv3_lpi_alloc_tables(void);
+extern void gicv3_lpi_rdist_ctrl(u32 redist, bool set);
static inline void gicv3_do_wait_for_rwp(void *base)
{
diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
index c33f883..7865d01 100644
--- a/lib/arm/gic-v3.c
+++ b/lib/arm/gic-v3.c
@@ -210,4 +210,21 @@ void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
byte &= ~mask;
*ptr = byte;
}
+
+void gicv3_lpi_rdist_ctrl(u32 redist, bool set)
+{
+ void *ptr;
+ u64 val;
+
+ assert(redist < nr_cpus);
+
+ ptr = gicv3_data.redist_base[redist];
+ val = readl(ptr + GICR_CTLR);
+ if (set)
+ val |= GICR_CTLR_ENABLE_LPIS;
+ else
+ val &= ~GICR_CTLR_ENABLE_LPIS;
+ writel(val, ptr + GICR_CTLR);
+}
#endif /* __aarch64__ */
+
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 07/14] arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
This helper function controls the signaling of LPIs at
redistributor level.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- move the helper in lib/arm/gic-v3.c
- rename the function with gicv3_lpi_ prefix
- s/report_abort/assert
---
lib/arm/asm/gic-v3.h | 1 +
lib/arm/gic-v3.c | 17 +++++++++++++++++
2 files changed, 18 insertions(+)
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index ec2a6f0..734c0c0 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -96,6 +96,7 @@ extern void gicv3_lpi_set_config(int n, u8 val);
extern u8 gicv3_lpi_get_config(int n);
extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
extern void gicv3_lpi_alloc_tables(void);
+extern void gicv3_lpi_rdist_ctrl(u32 redist, bool set);
static inline void gicv3_do_wait_for_rwp(void *base)
{
diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
index c33f883..7865d01 100644
--- a/lib/arm/gic-v3.c
+++ b/lib/arm/gic-v3.c
@@ -210,4 +210,21 @@ void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
byte &= ~mask;
*ptr = byte;
}
+
+void gicv3_lpi_rdist_ctrl(u32 redist, bool set)
+{
+ void *ptr;
+ u64 val;
+
+ assert(redist < nr_cpus);
+
+ ptr = gicv3_data.redist_base[redist];
+ val = readl(ptr + GICR_CTLR);
+ if (set)
+ val |= GICR_CTLR_ENABLE_LPIS;
+ else
+ val &= ~GICR_CTLR_ENABLE_LPIS;
+ writel(val, ptr + GICR_CTLR);
+}
#endif /* __aarch64__ */
+
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
its_enable_defaults() is the top init function that allocates the
command queue and all the requested tables (device, collection,
lpi config and pending tables), enable LPIs at distributor level
and ITS level.
gicv3_enable_defaults must be called before.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- introduce its_setup_baser in this patch
- squash "arm/arm64: ITS: Init the command queue" in this patch.
---
lib/arm/asm/gic-v3-its.h | 8 ++++
lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 97 insertions(+)
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index 815c515..fe73c04 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -36,6 +36,8 @@ struct its_data {
void *base;
struct its_typer typer;
struct its_baser baser[GITS_BASER_NR_REGS];
+ struct its_cmd_block *cmd_base;
+ struct its_cmd_block *cmd_write;
};
extern struct its_data its_data;
@@ -88,10 +90,16 @@ extern struct its_data its_data;
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_COLLECTION 4
+
+struct its_cmd_block {
+ u64 raw_cmd[4];
+};
+
extern void its_parse_typer(void);
extern void its_init(void);
extern int its_parse_baser(int i, struct its_baser *baser);
extern struct its_baser *its_lookup_baser(int type);
+extern void its_enable_defaults(void);
#else /* __arm__ */
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
index 2c0ce13..d1e7e52 100644
--- a/lib/arm/gic-v3-its.c
+++ b/lib/arm/gic-v3-its.c
@@ -86,3 +86,92 @@ void its_init(void)
its_parse_baser(i, &its_data.baser[i]);
}
+static void its_setup_baser(int i, struct its_baser *baser)
+{
+ unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
+ unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
+ u64 val;
+
+ baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
+
+ val = ((u64)baser->table_addr |
+ ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
+ ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
+ ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
+ (u64)baser->indirect << 62 |
+ (u64)baser->valid << 63);
+
+ switch (baser->psz) {
+ case SZ_4K:
+ val |= GITS_BASER_PAGE_SIZE_4K;
+ break;
+ case SZ_16K:
+ val |= GITS_BASER_PAGE_SIZE_16K;
+ break;
+ case SZ_64K:
+ val |= GITS_BASER_PAGE_SIZE_64K;
+ break;
+ }
+
+ writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
+}
+
+/**
+ * init_cmd_queue: Allocate the command queue and initialize
+ * CBASER, CREADR, CWRITER
+ */
+static void its_cmd_queue_init(void)
+{
+ unsigned long n = SZ_64K >> PAGE_SHIFT;
+ unsigned long order = fls(n);
+ u64 cbaser;
+
+ its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
+
+ cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1) | GITS_CBASER_VALID);
+
+ writeq(cbaser, its_data.base + GITS_CBASER);
+
+ its_data.cmd_write = its_data.cmd_base;
+ writeq(0, its_data.base + GITS_CWRITER);
+}
+
+void its_enable_defaults(void)
+{
+ unsigned int i;
+
+ its_parse_typer();
+
+ /* Allocate BASER tables (device and collection tables) */
+ for (i = 0; i < GITS_BASER_NR_REGS; i++) {
+ struct its_baser *baser = &its_data.baser[i];
+ int ret;
+
+ ret = its_parse_baser(i, baser);
+ if (ret)
+ continue;
+
+ switch (baser->type) {
+ case GITS_BASER_TYPE_DEVICE:
+ baser->valid = true;
+ its_setup_baser(i, baser);
+ break;
+ case GITS_BASER_TYPE_COLLECTION:
+ baser->valid = true;
+ its_setup_baser(i, baser);
+ break;
+ default:
+ break;
+ }
+ }
+
+ /* Allocate LPI config and pending tables */
+ gicv3_lpi_alloc_tables();
+
+ its_cmd_queue_init();
+
+ for (i = 0; i < nr_cpus; i++)
+ gicv3_lpi_rdist_ctrl(i, true);
+
+ writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
its_enable_defaults() is the top init function that allocates the
command queue and all the requested tables (device, collection,
lpi config and pending tables), enable LPIs at distributor level
and ITS level.
gicv3_enable_defaults must be called before.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- introduce its_setup_baser in this patch
- squash "arm/arm64: ITS: Init the command queue" in this patch.
---
lib/arm/asm/gic-v3-its.h | 8 ++++
lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 97 insertions(+)
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index 815c515..fe73c04 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -36,6 +36,8 @@ struct its_data {
void *base;
struct its_typer typer;
struct its_baser baser[GITS_BASER_NR_REGS];
+ struct its_cmd_block *cmd_base;
+ struct its_cmd_block *cmd_write;
};
extern struct its_data its_data;
@@ -88,10 +90,16 @@ extern struct its_data its_data;
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_COLLECTION 4
+
+struct its_cmd_block {
+ u64 raw_cmd[4];
+};
+
extern void its_parse_typer(void);
extern void its_init(void);
extern int its_parse_baser(int i, struct its_baser *baser);
extern struct its_baser *its_lookup_baser(int type);
+extern void its_enable_defaults(void);
#else /* __arm__ */
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
index 2c0ce13..d1e7e52 100644
--- a/lib/arm/gic-v3-its.c
+++ b/lib/arm/gic-v3-its.c
@@ -86,3 +86,92 @@ void its_init(void)
its_parse_baser(i, &its_data.baser[i]);
}
+static void its_setup_baser(int i, struct its_baser *baser)
+{
+ unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
+ unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
+ u64 val;
+
+ baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
+
+ val = ((u64)baser->table_addr |
+ ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
+ ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
+ ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
+ (u64)baser->indirect << 62 |
+ (u64)baser->valid << 63);
+
+ switch (baser->psz) {
+ case SZ_4K:
+ val |= GITS_BASER_PAGE_SIZE_4K;
+ break;
+ case SZ_16K:
+ val |= GITS_BASER_PAGE_SIZE_16K;
+ break;
+ case SZ_64K:
+ val |= GITS_BASER_PAGE_SIZE_64K;
+ break;
+ }
+
+ writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
+}
+
+/**
+ * init_cmd_queue: Allocate the command queue and initialize
+ * CBASER, CREADR, CWRITER
+ */
+static void its_cmd_queue_init(void)
+{
+ unsigned long n = SZ_64K >> PAGE_SHIFT;
+ unsigned long order = fls(n);
+ u64 cbaser;
+
+ its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
+
+ cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1) | GITS_CBASER_VALID);
+
+ writeq(cbaser, its_data.base + GITS_CBASER);
+
+ its_data.cmd_write = its_data.cmd_base;
+ writeq(0, its_data.base + GITS_CWRITER);
+}
+
+void its_enable_defaults(void)
+{
+ unsigned int i;
+
+ its_parse_typer();
+
+ /* Allocate BASER tables (device and collection tables) */
+ for (i = 0; i < GITS_BASER_NR_REGS; i++) {
+ struct its_baser *baser = &its_data.baser[i];
+ int ret;
+
+ ret = its_parse_baser(i, baser);
+ if (ret)
+ continue;
+
+ switch (baser->type) {
+ case GITS_BASER_TYPE_DEVICE:
+ baser->valid = true;
+ its_setup_baser(i, baser);
+ break;
+ case GITS_BASER_TYPE_COLLECTION:
+ baser->valid = true;
+ its_setup_baser(i, baser);
+ break;
+ default:
+ break;
+ }
+ }
+
+ /* Allocate LPI config and pending tables */
+ gicv3_lpi_alloc_tables();
+
+ its_cmd_queue_init();
+
+ for (i = 0; i < nr_cpus; i++)
+ gicv3_lpi_rdist_ctrl(i, true);
+
+ writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
its_enable_defaults() is the top init function that allocates the
command queue and all the requested tables (device, collection,
lpi config and pending tables), enable LPIs at distributor level
and ITS level.
gicv3_enable_defaults must be called before.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- introduce its_setup_baser in this patch
- squash "arm/arm64: ITS: Init the command queue" in this patch.
---
lib/arm/asm/gic-v3-its.h | 8 ++++
lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 97 insertions(+)
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index 815c515..fe73c04 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -36,6 +36,8 @@ struct its_data {
void *base;
struct its_typer typer;
struct its_baser baser[GITS_BASER_NR_REGS];
+ struct its_cmd_block *cmd_base;
+ struct its_cmd_block *cmd_write;
};
extern struct its_data its_data;
@@ -88,10 +90,16 @@ extern struct its_data its_data;
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_COLLECTION 4
+
+struct its_cmd_block {
+ u64 raw_cmd[4];
+};
+
extern void its_parse_typer(void);
extern void its_init(void);
extern int its_parse_baser(int i, struct its_baser *baser);
extern struct its_baser *its_lookup_baser(int type);
+extern void its_enable_defaults(void);
#else /* __arm__ */
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
index 2c0ce13..d1e7e52 100644
--- a/lib/arm/gic-v3-its.c
+++ b/lib/arm/gic-v3-its.c
@@ -86,3 +86,92 @@ void its_init(void)
its_parse_baser(i, &its_data.baser[i]);
}
+static void its_setup_baser(int i, struct its_baser *baser)
+{
+ unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
+ unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
+ u64 val;
+
+ baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
+
+ val = ((u64)baser->table_addr |
+ ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
+ ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
+ ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
+ (u64)baser->indirect << 62 |
+ (u64)baser->valid << 63);
+
+ switch (baser->psz) {
+ case SZ_4K:
+ val |= GITS_BASER_PAGE_SIZE_4K;
+ break;
+ case SZ_16K:
+ val |= GITS_BASER_PAGE_SIZE_16K;
+ break;
+ case SZ_64K:
+ val |= GITS_BASER_PAGE_SIZE_64K;
+ break;
+ }
+
+ writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
+}
+
+/**
+ * init_cmd_queue: Allocate the command queue and initialize
+ * CBASER, CREADR, CWRITER
+ */
+static void its_cmd_queue_init(void)
+{
+ unsigned long n = SZ_64K >> PAGE_SHIFT;
+ unsigned long order = fls(n);
+ u64 cbaser;
+
+ its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
+
+ cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1) | GITS_CBASER_VALID);
+
+ writeq(cbaser, its_data.base + GITS_CBASER);
+
+ its_data.cmd_write = its_data.cmd_base;
+ writeq(0, its_data.base + GITS_CWRITER);
+}
+
+void its_enable_defaults(void)
+{
+ unsigned int i;
+
+ its_parse_typer();
+
+ /* Allocate BASER tables (device and collection tables) */
+ for (i = 0; i < GITS_BASER_NR_REGS; i++) {
+ struct its_baser *baser = &its_data.baser[i];
+ int ret;
+
+ ret = its_parse_baser(i, baser);
+ if (ret)
+ continue;
+
+ switch (baser->type) {
+ case GITS_BASER_TYPE_DEVICE:
+ baser->valid = true;
+ its_setup_baser(i, baser);
+ break;
+ case GITS_BASER_TYPE_COLLECTION:
+ baser->valid = true;
+ its_setup_baser(i, baser);
+ break;
+ default:
+ break;
+ }
+ }
+
+ /* Allocate LPI config and pending tables */
+ gicv3_lpi_alloc_tables();
+
+ its_cmd_queue_init();
+
+ for (i = 0; i < nr_cpus; i++)
+ gicv3_lpi_rdist_ctrl(i, true);
+
+ writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
+}
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
Introduce an helper functions to register
- a new device, characterized by its device id and the
max number of event IDs that dimension its ITT (Interrupt
Translation Table). The function allocates the ITT.
- a new collection, characterized by its ID and the
target processing engine (PE).
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- s/report_abort/assert
v1 -> v2:
- s/nb_/nr_
---
lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 63 insertions(+), 1 deletion(-)
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index fe73c04..acd97a9 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -31,6 +31,19 @@ struct its_baser {
};
#define GITS_BASER_NR_REGS 8
+#define GITS_MAX_DEVICES 8
+#define GITS_MAX_COLLECTIONS 8
+
+struct its_device {
+ u32 device_id; /* device ID */
+ u32 nr_ites; /* Max Interrupt Translation Entries */
+ void *itt; /* Interrupt Translation Table GPA */
+};
+
+struct its_collection {
+ u64 target_address;
+ u16 col_id;
+};
struct its_data {
void *base;
@@ -38,6 +51,10 @@ struct its_data {
struct its_baser baser[GITS_BASER_NR_REGS];
struct its_cmd_block *cmd_base;
struct its_cmd_block *cmd_write;
+ struct its_device devices[GITS_MAX_DEVICES];
+ u32 nr_devices; /* Allocated Devices */
+ struct its_collection collections[GITS_MAX_COLLECTIONS];
+ u32 nr_collections; /* Allocated Collections */
};
extern struct its_data its_data;
@@ -90,7 +107,6 @@ extern struct its_data its_data;
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_COLLECTION 4
-
struct its_cmd_block {
u64 raw_cmd[4];
};
@@ -100,6 +116,8 @@ extern void its_init(void);
extern int its_parse_baser(int i, struct its_baser *baser);
extern struct its_baser *its_lookup_baser(int type);
extern void its_enable_defaults(void);
+extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
+extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
#else /* __arm__ */
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
index d1e7e52..c2dcd01 100644
--- a/lib/arm/gic-v3-its.c
+++ b/lib/arm/gic-v3-its.c
@@ -175,3 +175,47 @@ void its_enable_defaults(void)
writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
}
+
+struct its_device *its_create_device(u32 device_id, int nr_ites)
+{
+ struct its_baser *baser;
+ struct its_device *new;
+ unsigned long n, order;
+
+ assert(its_data.nr_devices < GITS_MAX_DEVICES);
+
+ baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
+ if (!baser)
+ return NULL;
+
+ new = &its_data.devices[its_data.nr_devices];
+
+ new->device_id = device_id;
+ new->nr_ites = nr_ites;
+
+ n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
+ order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
+ new->itt = (void *)virt_to_phys(alloc_pages(order));
+
+ its_data.nr_devices++;
+ return new;
+}
+
+struct its_collection *its_create_collection(u32 col_id, u32 pe)
+{
+ struct its_collection *new;
+
+ assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
+
+ new = &its_data.collections[its_data.nr_collections];
+
+ new->col_id = col_id;
+
+ if (its_data.typer.pta)
+ new->target_address = (u64)gicv3_data.redist_base[pe];
+ else
+ new->target_address = pe << 16;
+
+ its_data.nr_collections++;
+ return new;
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Introduce an helper functions to register
- a new device, characterized by its device id and the
max number of event IDs that dimension its ITT (Interrupt
Translation Table). The function allocates the ITT.
- a new collection, characterized by its ID and the
target processing engine (PE).
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- s/report_abort/assert
v1 -> v2:
- s/nb_/nr_
---
lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 63 insertions(+), 1 deletion(-)
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index fe73c04..acd97a9 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -31,6 +31,19 @@ struct its_baser {
};
#define GITS_BASER_NR_REGS 8
+#define GITS_MAX_DEVICES 8
+#define GITS_MAX_COLLECTIONS 8
+
+struct its_device {
+ u32 device_id; /* device ID */
+ u32 nr_ites; /* Max Interrupt Translation Entries */
+ void *itt; /* Interrupt Translation Table GPA */
+};
+
+struct its_collection {
+ u64 target_address;
+ u16 col_id;
+};
struct its_data {
void *base;
@@ -38,6 +51,10 @@ struct its_data {
struct its_baser baser[GITS_BASER_NR_REGS];
struct its_cmd_block *cmd_base;
struct its_cmd_block *cmd_write;
+ struct its_device devices[GITS_MAX_DEVICES];
+ u32 nr_devices; /* Allocated Devices */
+ struct its_collection collections[GITS_MAX_COLLECTIONS];
+ u32 nr_collections; /* Allocated Collections */
};
extern struct its_data its_data;
@@ -90,7 +107,6 @@ extern struct its_data its_data;
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_COLLECTION 4
-
struct its_cmd_block {
u64 raw_cmd[4];
};
@@ -100,6 +116,8 @@ extern void its_init(void);
extern int its_parse_baser(int i, struct its_baser *baser);
extern struct its_baser *its_lookup_baser(int type);
extern void its_enable_defaults(void);
+extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
+extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
#else /* __arm__ */
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
index d1e7e52..c2dcd01 100644
--- a/lib/arm/gic-v3-its.c
+++ b/lib/arm/gic-v3-its.c
@@ -175,3 +175,47 @@ void its_enable_defaults(void)
writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
}
+
+struct its_device *its_create_device(u32 device_id, int nr_ites)
+{
+ struct its_baser *baser;
+ struct its_device *new;
+ unsigned long n, order;
+
+ assert(its_data.nr_devices < GITS_MAX_DEVICES);
+
+ baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
+ if (!baser)
+ return NULL;
+
+ new = &its_data.devices[its_data.nr_devices];
+
+ new->device_id = device_id;
+ new->nr_ites = nr_ites;
+
+ n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
+ order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
+ new->itt = (void *)virt_to_phys(alloc_pages(order));
+
+ its_data.nr_devices++;
+ return new;
+}
+
+struct its_collection *its_create_collection(u32 col_id, u32 pe)
+{
+ struct its_collection *new;
+
+ assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
+
+ new = &its_data.collections[its_data.nr_collections];
+
+ new->col_id = col_id;
+
+ if (its_data.typer.pta)
+ new->target_address = (u64)gicv3_data.redist_base[pe];
+ else
+ new->target_address = pe << 16;
+
+ its_data.nr_collections++;
+ return new;
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Introduce an helper functions to register
- a new device, characterized by its device id and the
max number of event IDs that dimension its ITT (Interrupt
Translation Table). The function allocates the ITT.
- a new collection, characterized by its ID and the
target processing engine (PE).
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- s/report_abort/assert
v1 -> v2:
- s/nb_/nr_
---
lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 63 insertions(+), 1 deletion(-)
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index fe73c04..acd97a9 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -31,6 +31,19 @@ struct its_baser {
};
#define GITS_BASER_NR_REGS 8
+#define GITS_MAX_DEVICES 8
+#define GITS_MAX_COLLECTIONS 8
+
+struct its_device {
+ u32 device_id; /* device ID */
+ u32 nr_ites; /* Max Interrupt Translation Entries */
+ void *itt; /* Interrupt Translation Table GPA */
+};
+
+struct its_collection {
+ u64 target_address;
+ u16 col_id;
+};
struct its_data {
void *base;
@@ -38,6 +51,10 @@ struct its_data {
struct its_baser baser[GITS_BASER_NR_REGS];
struct its_cmd_block *cmd_base;
struct its_cmd_block *cmd_write;
+ struct its_device devices[GITS_MAX_DEVICES];
+ u32 nr_devices; /* Allocated Devices */
+ struct its_collection collections[GITS_MAX_COLLECTIONS];
+ u32 nr_collections; /* Allocated Collections */
};
extern struct its_data its_data;
@@ -90,7 +107,6 @@ extern struct its_data its_data;
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_COLLECTION 4
-
struct its_cmd_block {
u64 raw_cmd[4];
};
@@ -100,6 +116,8 @@ extern void its_init(void);
extern int its_parse_baser(int i, struct its_baser *baser);
extern struct its_baser *its_lookup_baser(int type);
extern void its_enable_defaults(void);
+extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
+extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
#else /* __arm__ */
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
index d1e7e52..c2dcd01 100644
--- a/lib/arm/gic-v3-its.c
+++ b/lib/arm/gic-v3-its.c
@@ -175,3 +175,47 @@ void its_enable_defaults(void)
writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
}
+
+struct its_device *its_create_device(u32 device_id, int nr_ites)
+{
+ struct its_baser *baser;
+ struct its_device *new;
+ unsigned long n, order;
+
+ assert(its_data.nr_devices < GITS_MAX_DEVICES);
+
+ baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
+ if (!baser)
+ return NULL;
+
+ new = &its_data.devices[its_data.nr_devices];
+
+ new->device_id = device_id;
+ new->nr_ites = nr_ites;
+
+ n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
+ order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
+ new->itt = (void *)virt_to_phys(alloc_pages(order));
+
+ its_data.nr_devices++;
+ return new;
+}
+
+struct its_collection *its_create_collection(u32 col_id, u32 pe)
+{
+ struct its_collection *new;
+
+ assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
+
+ new = &its_data.collections[its_data.nr_collections];
+
+ new->col_id = col_id;
+
+ if (its_data.typer.pta)
+ new->target_address = (u64)gicv3_data.redist_base[pe];
+ else
+ new->target_address = pe << 16;
+
+ its_data.nr_collections++;
+ return new;
+}
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 10/14] arm/arm64: ITS: commands
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
Implement main ITS commands. The code is largely inherited from
the ITS driver.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- do not use report() anymore
- assert if cmd_write exceeds the queue capacity
v1 -> v2:
- removed its_print_cmd_state
---
arm/Makefile.arm64 | 2 +-
lib/arm/asm/gic-v3-its.h | 38 +++-
lib/arm/gic-v3-its-cmd.c | 454 +++++++++++++++++++++++++++++++++++++++
3 files changed, 492 insertions(+), 2 deletions(-)
create mode 100644 lib/arm/gic-v3-its-cmd.c
diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 2571ffb..d12aea5 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -19,7 +19,7 @@ endef
cstart.o = $(TEST_DIR)/cstart64.o
cflatobjs += lib/arm64/processor.o
cflatobjs += lib/arm64/spinlock.o
-cflatobjs += lib/arm/gic-v3-its.o
+cflatobjs += lib/arm/gic-v3-its.o lib/arm/gic-v3-its-cmd.o
OBJDIRS += lib/arm64
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index acd97a9..0e5c5b6 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -45,6 +45,8 @@ struct its_collection {
u16 col_id;
};
+struct its_cmd_block;
+
struct its_data {
void *base;
struct its_typer typer;
@@ -107,6 +109,24 @@ extern struct its_data its_data;
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_COLLECTION 4
+/*
+ * ITS commands
+ */
+#define GITS_CMD_MAPD 0x08
+#define GITS_CMD_MAPC 0x09
+#define GITS_CMD_MAPTI 0x0a
+/* older GIC documentation used MAPVI for this command */
+#define GITS_CMD_MAPVI GITS_CMD_MAPTI
+#define GITS_CMD_MAPI 0x0b
+#define GITS_CMD_MOVI 0x01
+#define GITS_CMD_DISCARD 0x0f
+#define GITS_CMD_INV 0x0c
+#define GITS_CMD_MOVALL 0x0e
+#define GITS_CMD_INVALL 0x0d
+#define GITS_CMD_INT 0x03
+#define GITS_CMD_CLEAR 0x04
+#define GITS_CMD_SYNC 0x05
+
struct its_cmd_block {
u64 raw_cmd[4];
};
@@ -119,11 +139,27 @@ extern void its_enable_defaults(void);
extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
+extern void its_send_mapd(struct its_device *dev, int valid);
+extern void its_send_mapc(struct its_collection *col, int valid);
+extern void its_send_mapti(struct its_device *dev, u32 irq_id,
+ u32 event_id, struct its_collection *col);
+extern void its_send_int(struct its_device *dev, u32 event_id);
+extern void its_send_inv(struct its_device *dev, u32 event_id);
+extern void its_send_discard(struct its_device *dev, u32 event_id);
+extern void its_send_clear(struct its_device *dev, u32 event_id);
+extern void its_send_invall(struct its_collection *col);
+extern void its_send_movi(struct its_device *dev,
+ struct its_collection *col, u32 id);
+extern void its_send_sync(struct its_collection *col);
+
+#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
+#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
+#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)
+
#else /* __arm__ */
static inline void its_init(void) {}
#endif
-
#endif /* !__ASSEMBLY__ */
#endif /* _ASMARM_GIC_V3_ITS_H_ */
diff --git a/lib/arm/gic-v3-its-cmd.c b/lib/arm/gic-v3-its-cmd.c
new file mode 100644
index 0000000..fb4364c
--- /dev/null
+++ b/lib/arm/gic-v3-its-cmd.c
@@ -0,0 +1,454 @@
+/*
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * Most of the code is copy-pasted from:
+ * drivers/irqchip/irq-gic-v3-its.c
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <asm/io.h>
+#include <asm/gic.h>
+#include <asm/gic-v3-its.h>
+
+#define ITS_ITT_ALIGN SZ_256
+
+static const char * const its_cmd_string[] = {
+ [GITS_CMD_MAPD] = "MAPD",
+ [GITS_CMD_MAPC] = "MAPC",
+ [GITS_CMD_MAPTI] = "MAPTI",
+ [GITS_CMD_MAPI] = "MAPI",
+ [GITS_CMD_MOVI] = "MOVI",
+ [GITS_CMD_DISCARD] = "DISCARD",
+ [GITS_CMD_INV] = "INV",
+ [GITS_CMD_MOVALL] = "MOVALL",
+ [GITS_CMD_INVALL] = "INVALL",
+ [GITS_CMD_INT] = "INT",
+ [GITS_CMD_CLEAR] = "CLEAR",
+ [GITS_CMD_SYNC] = "SYNC",
+};
+
+struct its_cmd_desc {
+ union {
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_inv_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_int_cmd;
+
+ struct {
+ struct its_device *dev;
+ bool valid;
+ } its_mapd_cmd;
+
+ struct {
+ struct its_collection *col;
+ bool valid;
+ } its_mapc_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 phys_id;
+ u32 event_id;
+ u32 col_id;
+ } its_mapti_cmd;
+
+ struct {
+ struct its_device *dev;
+ struct its_collection *col;
+ u32 event_id;
+ } its_movi_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_discard_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_clear_cmd;
+
+ struct {
+ struct its_collection *col;
+ } its_invall_cmd;
+
+ struct {
+ struct its_collection *col;
+ } its_sync_cmd;
+ };
+};
+
+typedef void (*its_cmd_builder_t)(struct its_cmd_block *,
+ struct its_cmd_desc *);
+
+/* ITS COMMANDS */
+
+static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
+{
+ cmd->raw_cmd[0] &= ~0xffUL;
+ cmd->raw_cmd[0] |= cmd_nr;
+}
+
+static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
+{
+ cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
+ cmd->raw_cmd[0] |= ((u64)devid) << 32;
+}
+
+static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
+{
+ cmd->raw_cmd[1] &= ~0xffffffffUL;
+ cmd->raw_cmd[1] |= id;
+}
+
+static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
+{
+ cmd->raw_cmd[1] &= 0xffffffffUL;
+ cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
+}
+
+static void its_encode_size(struct its_cmd_block *cmd, u8 size)
+{
+ cmd->raw_cmd[1] &= ~0x1fUL;
+ cmd->raw_cmd[1] |= size & 0x1f;
+}
+
+static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
+{
+ cmd->raw_cmd[2] &= ~0xffffffffffffUL;
+ cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
+}
+
+static void its_encode_valid(struct its_cmd_block *cmd, int valid)
+{
+ cmd->raw_cmd[2] &= ~(1UL << 63);
+ cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
+}
+
+static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
+{
+ cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
+ cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
+}
+
+static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
+{
+ cmd->raw_cmd[2] &= ~0xffffUL;
+ cmd->raw_cmd[2] |= col;
+}
+
+static inline void its_fixup_cmd(struct its_cmd_block *cmd)
+{
+ /* Let's fixup BE commands */
+ cmd->raw_cmd[0] = cpu_to_le64(cmd->raw_cmd[0]);
+ cmd->raw_cmd[1] = cpu_to_le64(cmd->raw_cmd[1]);
+ cmd->raw_cmd[2] = cpu_to_le64(cmd->raw_cmd[2]);
+ cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
+}
+
+static u64 its_cmd_ptr_to_offset(struct its_cmd_block *ptr)
+{
+ return (ptr - its_data.cmd_base) * sizeof(*ptr);
+}
+
+static struct its_cmd_block *its_post_commands(void)
+{
+ u64 wr = its_cmd_ptr_to_offset(its_data.cmd_write);
+
+ writeq(wr, its_data.base + GITS_CWRITER);
+ return its_data.cmd_write;
+}
+
+
+static struct its_cmd_block *its_allocate_entry(void)
+{
+ struct its_cmd_block *cmd;
+
+ assert((u64)its_data.cmd_write < (u64)its_data.cmd_base + SZ_64K);
+ cmd = its_data.cmd_write++;
+ return cmd;
+}
+
+static void its_wait_for_range_completion(struct its_cmd_block *from,
+ struct its_cmd_block *to)
+{
+ u64 rd_idx, from_idx, to_idx;
+ u32 count = 1000000; /* 1s! */
+
+ from_idx = its_cmd_ptr_to_offset(from);
+ to_idx = its_cmd_ptr_to_offset(to);
+ while (1) {
+ rd_idx = readq(its_data.base + GITS_CREADR);
+ if (rd_idx >= to_idx || rd_idx < from_idx)
+ break;
+
+ count--;
+ if (!count) {
+ unsigned int cmd_id = from->raw_cmd[0] & 0xFF;
+
+ assert_msg(false, "%s timeout!",
+ cmd_id <= 0xF ? its_cmd_string[cmd_id] :
+ "Unexpected");
+ return;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static void its_send_single_command(its_cmd_builder_t builder,
+ struct its_cmd_desc *desc)
+{
+ struct its_cmd_block *cmd, *next_cmd;
+
+ cmd = its_allocate_entry();
+ builder(cmd, desc);
+ next_cmd = its_post_commands();
+
+ its_wait_for_range_completion(cmd, next_cmd);
+}
+
+
+static void its_build_mapd_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ unsigned long itt_addr;
+ u8 size = 12; /* 4096 eventids */
+
+ itt_addr = (unsigned long)desc->its_mapd_cmd.dev->itt;
+ itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
+
+ its_encode_cmd(cmd, GITS_CMD_MAPD);
+ its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
+ its_encode_size(cmd, size - 1);
+ its_encode_itt(cmd, itt_addr);
+ its_encode_valid(cmd, desc->its_mapd_cmd.valid);
+
+ its_fixup_cmd(cmd);
+ printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
+ desc->its_mapd_cmd.dev->device_id,
+ size, itt_addr, desc->its_mapd_cmd.valid);
+
+}
+
+static void its_build_mapc_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_MAPC);
+ its_encode_collection(cmd, desc->its_mapc_cmd.col->col_id);
+ its_encode_target(cmd, desc->its_mapc_cmd.col->target_address);
+ its_encode_valid(cmd, desc->its_mapc_cmd.valid);
+
+ its_fixup_cmd(cmd);
+ report_info("MAPC col_id=%d target_addr = 0x%lx valid=%d",
+ desc->its_mapc_cmd.col->col_id,
+ desc->its_mapc_cmd.col->target_address,
+ desc->its_mapc_cmd.valid);
+}
+
+static void its_build_mapti_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_MAPTI);
+ its_encode_devid(cmd, desc->its_mapti_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_mapti_cmd.event_id);
+ its_encode_phys_id(cmd, desc->its_mapti_cmd.phys_id);
+ its_encode_collection(cmd, desc->its_mapti_cmd.col_id);
+
+ its_fixup_cmd(cmd);
+ report_info("MAPTI dev_id=%d event_id=%d -> phys_id=%d, col_id=%d",
+ desc->its_mapti_cmd.dev->device_id,
+ desc->its_mapti_cmd.event_id,
+ desc->its_mapti_cmd.phys_id,
+ desc->its_mapti_cmd.col_id);
+}
+
+static void its_build_invall_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_INVALL);
+ its_encode_collection(cmd, desc->its_invall_cmd.col->col_id);
+
+ its_fixup_cmd(cmd);
+ report_info("INVALL col_id=%d", desc->its_invall_cmd.col->col_id);
+}
+
+static void its_build_clear_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_CLEAR);
+ its_encode_devid(cmd, desc->its_clear_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_clear_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("CLEAR col_id=%d", desc->its_invall_cmd.col->col_id);
+}
+
+static void its_build_discard_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_DISCARD);
+ its_encode_devid(cmd, desc->its_discard_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_discard_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("DISCARD col_id=%d", desc->its_invall_cmd.col->col_id);
+}
+
+static void its_build_inv_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_INV);
+ its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("INV dev_id=%d event_id=%d",
+ desc->its_inv_cmd.dev->device_id,
+ desc->its_inv_cmd.event_id);
+}
+
+static void its_build_int_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_INT);
+ its_encode_devid(cmd, desc->its_int_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_int_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("INT dev_id=%d event_id=%d",
+ desc->its_int_cmd.dev->device_id,
+ desc->its_int_cmd.event_id);
+}
+
+static void its_build_sync_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_SYNC);
+ its_encode_target(cmd, desc->its_sync_cmd.col->target_address);
+ its_fixup_cmd(cmd);
+ report_info("SYNC target_addr = 0x%lx",
+ desc->its_sync_cmd.col->target_address);
+}
+
+static void its_build_movi_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_MOVI);
+ its_encode_devid(cmd, desc->its_movi_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_movi_cmd.event_id);
+ its_encode_collection(cmd, desc->its_movi_cmd.col->col_id);
+
+ its_fixup_cmd(cmd);
+ report_info("MOVI dev_id=%d event_id = %d col_id=%d",
+ desc->its_movi_cmd.dev->device_id,
+ desc->its_movi_cmd.event_id,
+ desc->its_movi_cmd.col->col_id);
+}
+
+void its_send_mapd(struct its_device *dev, int valid)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapd_cmd.dev = dev;
+ desc.its_mapd_cmd.valid = !!valid;
+
+ its_send_single_command(its_build_mapd_cmd, &desc);
+}
+
+void its_send_mapc(struct its_collection *col, int valid)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapc_cmd.col = col;
+ desc.its_mapc_cmd.valid = !!valid;
+
+ its_send_single_command(its_build_mapc_cmd, &desc);
+}
+
+void its_send_mapti(struct its_device *dev, u32 irq_id,
+ u32 event_id, struct its_collection *col)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapti_cmd.dev = dev;
+ desc.its_mapti_cmd.phys_id = irq_id;
+ desc.its_mapti_cmd.event_id = event_id;
+ desc.its_mapti_cmd.col_id = col->col_id;
+
+ its_send_single_command(its_build_mapti_cmd, &desc);
+}
+
+void its_send_int(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_int_cmd.dev = dev;
+ desc.its_int_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_int_cmd, &desc);
+}
+
+void its_send_movi(struct its_device *dev,
+ struct its_collection *col, u32 id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_movi_cmd.dev = dev;
+ desc.its_movi_cmd.col = col;
+ desc.its_movi_cmd.event_id = id;
+
+ its_send_single_command(its_build_movi_cmd, &desc);
+}
+
+void its_send_invall(struct its_collection *col)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_invall_cmd.col = col;
+
+ its_send_single_command(its_build_invall_cmd, &desc);
+}
+
+void its_send_inv(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_inv_cmd.dev = dev;
+ desc.its_inv_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_inv_cmd, &desc);
+}
+
+void its_send_discard(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_discard_cmd.dev = dev;
+ desc.its_discard_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_discard_cmd, &desc);
+}
+
+void its_send_clear(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_clear_cmd.dev = dev;
+ desc.its_clear_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_clear_cmd, &desc);
+}
+
+void its_send_sync(struct its_collection *col)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_sync_cmd.col = col;
+
+ its_send_single_command(its_build_sync_cmd, &desc);
+}
+
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 10/14] arm/arm64: ITS: commands
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Implement main ITS commands. The code is largely inherited from
the ITS driver.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- do not use report() anymore
- assert if cmd_write exceeds the queue capacity
v1 -> v2:
- removed its_print_cmd_state
---
arm/Makefile.arm64 | 2 +-
lib/arm/asm/gic-v3-its.h | 38 +++-
lib/arm/gic-v3-its-cmd.c | 454 +++++++++++++++++++++++++++++++++++++++
3 files changed, 492 insertions(+), 2 deletions(-)
create mode 100644 lib/arm/gic-v3-its-cmd.c
diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 2571ffb..d12aea5 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -19,7 +19,7 @@ endef
cstart.o = $(TEST_DIR)/cstart64.o
cflatobjs += lib/arm64/processor.o
cflatobjs += lib/arm64/spinlock.o
-cflatobjs += lib/arm/gic-v3-its.o
+cflatobjs += lib/arm/gic-v3-its.o lib/arm/gic-v3-its-cmd.o
OBJDIRS += lib/arm64
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index acd97a9..0e5c5b6 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -45,6 +45,8 @@ struct its_collection {
u16 col_id;
};
+struct its_cmd_block;
+
struct its_data {
void *base;
struct its_typer typer;
@@ -107,6 +109,24 @@ extern struct its_data its_data;
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_COLLECTION 4
+/*
+ * ITS commands
+ */
+#define GITS_CMD_MAPD 0x08
+#define GITS_CMD_MAPC 0x09
+#define GITS_CMD_MAPTI 0x0a
+/* older GIC documentation used MAPVI for this command */
+#define GITS_CMD_MAPVI GITS_CMD_MAPTI
+#define GITS_CMD_MAPI 0x0b
+#define GITS_CMD_MOVI 0x01
+#define GITS_CMD_DISCARD 0x0f
+#define GITS_CMD_INV 0x0c
+#define GITS_CMD_MOVALL 0x0e
+#define GITS_CMD_INVALL 0x0d
+#define GITS_CMD_INT 0x03
+#define GITS_CMD_CLEAR 0x04
+#define GITS_CMD_SYNC 0x05
+
struct its_cmd_block {
u64 raw_cmd[4];
};
@@ -119,11 +139,27 @@ extern void its_enable_defaults(void);
extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
+extern void its_send_mapd(struct its_device *dev, int valid);
+extern void its_send_mapc(struct its_collection *col, int valid);
+extern void its_send_mapti(struct its_device *dev, u32 irq_id,
+ u32 event_id, struct its_collection *col);
+extern void its_send_int(struct its_device *dev, u32 event_id);
+extern void its_send_inv(struct its_device *dev, u32 event_id);
+extern void its_send_discard(struct its_device *dev, u32 event_id);
+extern void its_send_clear(struct its_device *dev, u32 event_id);
+extern void its_send_invall(struct its_collection *col);
+extern void its_send_movi(struct its_device *dev,
+ struct its_collection *col, u32 id);
+extern void its_send_sync(struct its_collection *col);
+
+#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
+#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
+#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)
+
#else /* __arm__ */
static inline void its_init(void) {}
#endif
-
#endif /* !__ASSEMBLY__ */
#endif /* _ASMARM_GIC_V3_ITS_H_ */
diff --git a/lib/arm/gic-v3-its-cmd.c b/lib/arm/gic-v3-its-cmd.c
new file mode 100644
index 0000000..fb4364c
--- /dev/null
+++ b/lib/arm/gic-v3-its-cmd.c
@@ -0,0 +1,454 @@
+/*
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * Most of the code is copy-pasted from:
+ * drivers/irqchip/irq-gic-v3-its.c
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <asm/io.h>
+#include <asm/gic.h>
+#include <asm/gic-v3-its.h>
+
+#define ITS_ITT_ALIGN SZ_256
+
+static const char * const its_cmd_string[] = {
+ [GITS_CMD_MAPD] = "MAPD",
+ [GITS_CMD_MAPC] = "MAPC",
+ [GITS_CMD_MAPTI] = "MAPTI",
+ [GITS_CMD_MAPI] = "MAPI",
+ [GITS_CMD_MOVI] = "MOVI",
+ [GITS_CMD_DISCARD] = "DISCARD",
+ [GITS_CMD_INV] = "INV",
+ [GITS_CMD_MOVALL] = "MOVALL",
+ [GITS_CMD_INVALL] = "INVALL",
+ [GITS_CMD_INT] = "INT",
+ [GITS_CMD_CLEAR] = "CLEAR",
+ [GITS_CMD_SYNC] = "SYNC",
+};
+
+struct its_cmd_desc {
+ union {
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_inv_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_int_cmd;
+
+ struct {
+ struct its_device *dev;
+ bool valid;
+ } its_mapd_cmd;
+
+ struct {
+ struct its_collection *col;
+ bool valid;
+ } its_mapc_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 phys_id;
+ u32 event_id;
+ u32 col_id;
+ } its_mapti_cmd;
+
+ struct {
+ struct its_device *dev;
+ struct its_collection *col;
+ u32 event_id;
+ } its_movi_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_discard_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_clear_cmd;
+
+ struct {
+ struct its_collection *col;
+ } its_invall_cmd;
+
+ struct {
+ struct its_collection *col;
+ } its_sync_cmd;
+ };
+};
+
+typedef void (*its_cmd_builder_t)(struct its_cmd_block *,
+ struct its_cmd_desc *);
+
+/* ITS COMMANDS */
+
+static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
+{
+ cmd->raw_cmd[0] &= ~0xffUL;
+ cmd->raw_cmd[0] |= cmd_nr;
+}
+
+static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
+{
+ cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
+ cmd->raw_cmd[0] |= ((u64)devid) << 32;
+}
+
+static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
+{
+ cmd->raw_cmd[1] &= ~0xffffffffUL;
+ cmd->raw_cmd[1] |= id;
+}
+
+static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
+{
+ cmd->raw_cmd[1] &= 0xffffffffUL;
+ cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
+}
+
+static void its_encode_size(struct its_cmd_block *cmd, u8 size)
+{
+ cmd->raw_cmd[1] &= ~0x1fUL;
+ cmd->raw_cmd[1] |= size & 0x1f;
+}
+
+static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
+{
+ cmd->raw_cmd[2] &= ~0xffffffffffffUL;
+ cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
+}
+
+static void its_encode_valid(struct its_cmd_block *cmd, int valid)
+{
+ cmd->raw_cmd[2] &= ~(1UL << 63);
+ cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
+}
+
+static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
+{
+ cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
+ cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
+}
+
+static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
+{
+ cmd->raw_cmd[2] &= ~0xffffUL;
+ cmd->raw_cmd[2] |= col;
+}
+
+static inline void its_fixup_cmd(struct its_cmd_block *cmd)
+{
+ /* Let's fixup BE commands */
+ cmd->raw_cmd[0] = cpu_to_le64(cmd->raw_cmd[0]);
+ cmd->raw_cmd[1] = cpu_to_le64(cmd->raw_cmd[1]);
+ cmd->raw_cmd[2] = cpu_to_le64(cmd->raw_cmd[2]);
+ cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
+}
+
+static u64 its_cmd_ptr_to_offset(struct its_cmd_block *ptr)
+{
+ return (ptr - its_data.cmd_base) * sizeof(*ptr);
+}
+
+static struct its_cmd_block *its_post_commands(void)
+{
+ u64 wr = its_cmd_ptr_to_offset(its_data.cmd_write);
+
+ writeq(wr, its_data.base + GITS_CWRITER);
+ return its_data.cmd_write;
+}
+
+
+static struct its_cmd_block *its_allocate_entry(void)
+{
+ struct its_cmd_block *cmd;
+
+ assert((u64)its_data.cmd_write < (u64)its_data.cmd_base + SZ_64K);
+ cmd = its_data.cmd_write++;
+ return cmd;
+}
+
+static void its_wait_for_range_completion(struct its_cmd_block *from,
+ struct its_cmd_block *to)
+{
+ u64 rd_idx, from_idx, to_idx;
+ u32 count = 1000000; /* 1s! */
+
+ from_idx = its_cmd_ptr_to_offset(from);
+ to_idx = its_cmd_ptr_to_offset(to);
+ while (1) {
+ rd_idx = readq(its_data.base + GITS_CREADR);
+ if (rd_idx >= to_idx || rd_idx < from_idx)
+ break;
+
+ count--;
+ if (!count) {
+ unsigned int cmd_id = from->raw_cmd[0] & 0xFF;
+
+ assert_msg(false, "%s timeout!",
+ cmd_id <= 0xF ? its_cmd_string[cmd_id] :
+ "Unexpected");
+ return;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static void its_send_single_command(its_cmd_builder_t builder,
+ struct its_cmd_desc *desc)
+{
+ struct its_cmd_block *cmd, *next_cmd;
+
+ cmd = its_allocate_entry();
+ builder(cmd, desc);
+ next_cmd = its_post_commands();
+
+ its_wait_for_range_completion(cmd, next_cmd);
+}
+
+
+static void its_build_mapd_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ unsigned long itt_addr;
+ u8 size = 12; /* 4096 eventids */
+
+ itt_addr = (unsigned long)desc->its_mapd_cmd.dev->itt;
+ itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
+
+ its_encode_cmd(cmd, GITS_CMD_MAPD);
+ its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
+ its_encode_size(cmd, size - 1);
+ its_encode_itt(cmd, itt_addr);
+ its_encode_valid(cmd, desc->its_mapd_cmd.valid);
+
+ its_fixup_cmd(cmd);
+ printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
+ desc->its_mapd_cmd.dev->device_id,
+ size, itt_addr, desc->its_mapd_cmd.valid);
+
+}
+
+static void its_build_mapc_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_MAPC);
+ its_encode_collection(cmd, desc->its_mapc_cmd.col->col_id);
+ its_encode_target(cmd, desc->its_mapc_cmd.col->target_address);
+ its_encode_valid(cmd, desc->its_mapc_cmd.valid);
+
+ its_fixup_cmd(cmd);
+ report_info("MAPC col_id=%d target_addr = 0x%lx valid=%d",
+ desc->its_mapc_cmd.col->col_id,
+ desc->its_mapc_cmd.col->target_address,
+ desc->its_mapc_cmd.valid);
+}
+
+static void its_build_mapti_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_MAPTI);
+ its_encode_devid(cmd, desc->its_mapti_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_mapti_cmd.event_id);
+ its_encode_phys_id(cmd, desc->its_mapti_cmd.phys_id);
+ its_encode_collection(cmd, desc->its_mapti_cmd.col_id);
+
+ its_fixup_cmd(cmd);
+ report_info("MAPTI dev_id=%d event_id=%d -> phys_id=%d, col_id=%d",
+ desc->its_mapti_cmd.dev->device_id,
+ desc->its_mapti_cmd.event_id,
+ desc->its_mapti_cmd.phys_id,
+ desc->its_mapti_cmd.col_id);
+}
+
+static void its_build_invall_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_INVALL);
+ its_encode_collection(cmd, desc->its_invall_cmd.col->col_id);
+
+ its_fixup_cmd(cmd);
+ report_info("INVALL col_id=%d", desc->its_invall_cmd.col->col_id);
+}
+
+static void its_build_clear_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_CLEAR);
+ its_encode_devid(cmd, desc->its_clear_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_clear_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("CLEAR col_id=%d", desc->its_invall_cmd.col->col_id);
+}
+
+static void its_build_discard_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_DISCARD);
+ its_encode_devid(cmd, desc->its_discard_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_discard_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("DISCARD col_id=%d", desc->its_invall_cmd.col->col_id);
+}
+
+static void its_build_inv_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_INV);
+ its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("INV dev_id=%d event_id=%d",
+ desc->its_inv_cmd.dev->device_id,
+ desc->its_inv_cmd.event_id);
+}
+
+static void its_build_int_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_INT);
+ its_encode_devid(cmd, desc->its_int_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_int_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("INT dev_id=%d event_id=%d",
+ desc->its_int_cmd.dev->device_id,
+ desc->its_int_cmd.event_id);
+}
+
+static void its_build_sync_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_SYNC);
+ its_encode_target(cmd, desc->its_sync_cmd.col->target_address);
+ its_fixup_cmd(cmd);
+ report_info("SYNC target_addr = 0x%lx",
+ desc->its_sync_cmd.col->target_address);
+}
+
+static void its_build_movi_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_MOVI);
+ its_encode_devid(cmd, desc->its_movi_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_movi_cmd.event_id);
+ its_encode_collection(cmd, desc->its_movi_cmd.col->col_id);
+
+ its_fixup_cmd(cmd);
+ report_info("MOVI dev_id=%d event_id = %d col_id=%d",
+ desc->its_movi_cmd.dev->device_id,
+ desc->its_movi_cmd.event_id,
+ desc->its_movi_cmd.col->col_id);
+}
+
+void its_send_mapd(struct its_device *dev, int valid)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapd_cmd.dev = dev;
+ desc.its_mapd_cmd.valid = !!valid;
+
+ its_send_single_command(its_build_mapd_cmd, &desc);
+}
+
+void its_send_mapc(struct its_collection *col, int valid)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapc_cmd.col = col;
+ desc.its_mapc_cmd.valid = !!valid;
+
+ its_send_single_command(its_build_mapc_cmd, &desc);
+}
+
+void its_send_mapti(struct its_device *dev, u32 irq_id,
+ u32 event_id, struct its_collection *col)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapti_cmd.dev = dev;
+ desc.its_mapti_cmd.phys_id = irq_id;
+ desc.its_mapti_cmd.event_id = event_id;
+ desc.its_mapti_cmd.col_id = col->col_id;
+
+ its_send_single_command(its_build_mapti_cmd, &desc);
+}
+
+void its_send_int(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_int_cmd.dev = dev;
+ desc.its_int_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_int_cmd, &desc);
+}
+
+void its_send_movi(struct its_device *dev,
+ struct its_collection *col, u32 id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_movi_cmd.dev = dev;
+ desc.its_movi_cmd.col = col;
+ desc.its_movi_cmd.event_id = id;
+
+ its_send_single_command(its_build_movi_cmd, &desc);
+}
+
+void its_send_invall(struct its_collection *col)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_invall_cmd.col = col;
+
+ its_send_single_command(its_build_invall_cmd, &desc);
+}
+
+void its_send_inv(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_inv_cmd.dev = dev;
+ desc.its_inv_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_inv_cmd, &desc);
+}
+
+void its_send_discard(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_discard_cmd.dev = dev;
+ desc.its_discard_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_discard_cmd, &desc);
+}
+
+void its_send_clear(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_clear_cmd.dev = dev;
+ desc.its_clear_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_clear_cmd, &desc);
+}
+
+void its_send_sync(struct its_collection *col)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_sync_cmd.col = col;
+
+ its_send_single_command(its_build_sync_cmd, &desc);
+}
+
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 10/14] arm/arm64: ITS: commands
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Implement main ITS commands. The code is largely inherited from
the ITS driver.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- do not use report() anymore
- assert if cmd_write exceeds the queue capacity
v1 -> v2:
- removed its_print_cmd_state
---
arm/Makefile.arm64 | 2 +-
lib/arm/asm/gic-v3-its.h | 38 +++-
lib/arm/gic-v3-its-cmd.c | 454 +++++++++++++++++++++++++++++++++++++++
3 files changed, 492 insertions(+), 2 deletions(-)
create mode 100644 lib/arm/gic-v3-its-cmd.c
diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 2571ffb..d12aea5 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -19,7 +19,7 @@ endef
cstart.o = $(TEST_DIR)/cstart64.o
cflatobjs += lib/arm64/processor.o
cflatobjs += lib/arm64/spinlock.o
-cflatobjs += lib/arm/gic-v3-its.o
+cflatobjs += lib/arm/gic-v3-its.o lib/arm/gic-v3-its-cmd.o
OBJDIRS += lib/arm64
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index acd97a9..0e5c5b6 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -45,6 +45,8 @@ struct its_collection {
u16 col_id;
};
+struct its_cmd_block;
+
struct its_data {
void *base;
struct its_typer typer;
@@ -107,6 +109,24 @@ extern struct its_data its_data;
#define GITS_BASER_TYPE_DEVICE 1
#define GITS_BASER_TYPE_COLLECTION 4
+/*
+ * ITS commands
+ */
+#define GITS_CMD_MAPD 0x08
+#define GITS_CMD_MAPC 0x09
+#define GITS_CMD_MAPTI 0x0a
+/* older GIC documentation used MAPVI for this command */
+#define GITS_CMD_MAPVI GITS_CMD_MAPTI
+#define GITS_CMD_MAPI 0x0b
+#define GITS_CMD_MOVI 0x01
+#define GITS_CMD_DISCARD 0x0f
+#define GITS_CMD_INV 0x0c
+#define GITS_CMD_MOVALL 0x0e
+#define GITS_CMD_INVALL 0x0d
+#define GITS_CMD_INT 0x03
+#define GITS_CMD_CLEAR 0x04
+#define GITS_CMD_SYNC 0x05
+
struct its_cmd_block {
u64 raw_cmd[4];
};
@@ -119,11 +139,27 @@ extern void its_enable_defaults(void);
extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
+extern void its_send_mapd(struct its_device *dev, int valid);
+extern void its_send_mapc(struct its_collection *col, int valid);
+extern void its_send_mapti(struct its_device *dev, u32 irq_id,
+ u32 event_id, struct its_collection *col);
+extern void its_send_int(struct its_device *dev, u32 event_id);
+extern void its_send_inv(struct its_device *dev, u32 event_id);
+extern void its_send_discard(struct its_device *dev, u32 event_id);
+extern void its_send_clear(struct its_device *dev, u32 event_id);
+extern void its_send_invall(struct its_collection *col);
+extern void its_send_movi(struct its_device *dev,
+ struct its_collection *col, u32 id);
+extern void its_send_sync(struct its_collection *col);
+
+#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
+#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
+#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)
+
#else /* __arm__ */
static inline void its_init(void) {}
#endif
-
#endif /* !__ASSEMBLY__ */
#endif /* _ASMARM_GIC_V3_ITS_H_ */
diff --git a/lib/arm/gic-v3-its-cmd.c b/lib/arm/gic-v3-its-cmd.c
new file mode 100644
index 0000000..fb4364c
--- /dev/null
+++ b/lib/arm/gic-v3-its-cmd.c
@@ -0,0 +1,454 @@
+/*
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * Most of the code is copy-pasted from:
+ * drivers/irqchip/irq-gic-v3-its.c
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <asm/io.h>
+#include <asm/gic.h>
+#include <asm/gic-v3-its.h>
+
+#define ITS_ITT_ALIGN SZ_256
+
+static const char * const its_cmd_string[] = {
+ [GITS_CMD_MAPD] = "MAPD",
+ [GITS_CMD_MAPC] = "MAPC",
+ [GITS_CMD_MAPTI] = "MAPTI",
+ [GITS_CMD_MAPI] = "MAPI",
+ [GITS_CMD_MOVI] = "MOVI",
+ [GITS_CMD_DISCARD] = "DISCARD",
+ [GITS_CMD_INV] = "INV",
+ [GITS_CMD_MOVALL] = "MOVALL",
+ [GITS_CMD_INVALL] = "INVALL",
+ [GITS_CMD_INT] = "INT",
+ [GITS_CMD_CLEAR] = "CLEAR",
+ [GITS_CMD_SYNC] = "SYNC",
+};
+
+struct its_cmd_desc {
+ union {
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_inv_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_int_cmd;
+
+ struct {
+ struct its_device *dev;
+ bool valid;
+ } its_mapd_cmd;
+
+ struct {
+ struct its_collection *col;
+ bool valid;
+ } its_mapc_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 phys_id;
+ u32 event_id;
+ u32 col_id;
+ } its_mapti_cmd;
+
+ struct {
+ struct its_device *dev;
+ struct its_collection *col;
+ u32 event_id;
+ } its_movi_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_discard_cmd;
+
+ struct {
+ struct its_device *dev;
+ u32 event_id;
+ } its_clear_cmd;
+
+ struct {
+ struct its_collection *col;
+ } its_invall_cmd;
+
+ struct {
+ struct its_collection *col;
+ } its_sync_cmd;
+ };
+};
+
+typedef void (*its_cmd_builder_t)(struct its_cmd_block *,
+ struct its_cmd_desc *);
+
+/* ITS COMMANDS */
+
+static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
+{
+ cmd->raw_cmd[0] &= ~0xffUL;
+ cmd->raw_cmd[0] |= cmd_nr;
+}
+
+static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
+{
+ cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
+ cmd->raw_cmd[0] |= ((u64)devid) << 32;
+}
+
+static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
+{
+ cmd->raw_cmd[1] &= ~0xffffffffUL;
+ cmd->raw_cmd[1] |= id;
+}
+
+static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
+{
+ cmd->raw_cmd[1] &= 0xffffffffUL;
+ cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
+}
+
+static void its_encode_size(struct its_cmd_block *cmd, u8 size)
+{
+ cmd->raw_cmd[1] &= ~0x1fUL;
+ cmd->raw_cmd[1] |= size & 0x1f;
+}
+
+static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
+{
+ cmd->raw_cmd[2] &= ~0xffffffffffffUL;
+ cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
+}
+
+static void its_encode_valid(struct its_cmd_block *cmd, int valid)
+{
+ cmd->raw_cmd[2] &= ~(1UL << 63);
+ cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
+}
+
+static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
+{
+ cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
+ cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
+}
+
+static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
+{
+ cmd->raw_cmd[2] &= ~0xffffUL;
+ cmd->raw_cmd[2] |= col;
+}
+
+static inline void its_fixup_cmd(struct its_cmd_block *cmd)
+{
+ /* Let's fixup BE commands */
+ cmd->raw_cmd[0] = cpu_to_le64(cmd->raw_cmd[0]);
+ cmd->raw_cmd[1] = cpu_to_le64(cmd->raw_cmd[1]);
+ cmd->raw_cmd[2] = cpu_to_le64(cmd->raw_cmd[2]);
+ cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
+}
+
+static u64 its_cmd_ptr_to_offset(struct its_cmd_block *ptr)
+{
+ return (ptr - its_data.cmd_base) * sizeof(*ptr);
+}
+
+static struct its_cmd_block *its_post_commands(void)
+{
+ u64 wr = its_cmd_ptr_to_offset(its_data.cmd_write);
+
+ writeq(wr, its_data.base + GITS_CWRITER);
+ return its_data.cmd_write;
+}
+
+
+static struct its_cmd_block *its_allocate_entry(void)
+{
+ struct its_cmd_block *cmd;
+
+ assert((u64)its_data.cmd_write < (u64)its_data.cmd_base + SZ_64K);
+ cmd = its_data.cmd_write++;
+ return cmd;
+}
+
+static void its_wait_for_range_completion(struct its_cmd_block *from,
+ struct its_cmd_block *to)
+{
+ u64 rd_idx, from_idx, to_idx;
+ u32 count = 1000000; /* 1s! */
+
+ from_idx = its_cmd_ptr_to_offset(from);
+ to_idx = its_cmd_ptr_to_offset(to);
+ while (1) {
+ rd_idx = readq(its_data.base + GITS_CREADR);
+ if (rd_idx >= to_idx || rd_idx < from_idx)
+ break;
+
+ count--;
+ if (!count) {
+ unsigned int cmd_id = from->raw_cmd[0] & 0xFF;
+
+ assert_msg(false, "%s timeout!",
+ cmd_id <= 0xF ? its_cmd_string[cmd_id] :
+ "Unexpected");
+ return;
+ }
+ cpu_relax();
+ udelay(1);
+ }
+}
+
+static void its_send_single_command(its_cmd_builder_t builder,
+ struct its_cmd_desc *desc)
+{
+ struct its_cmd_block *cmd, *next_cmd;
+
+ cmd = its_allocate_entry();
+ builder(cmd, desc);
+ next_cmd = its_post_commands();
+
+ its_wait_for_range_completion(cmd, next_cmd);
+}
+
+
+static void its_build_mapd_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ unsigned long itt_addr;
+ u8 size = 12; /* 4096 eventids */
+
+ itt_addr = (unsigned long)desc->its_mapd_cmd.dev->itt;
+ itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
+
+ its_encode_cmd(cmd, GITS_CMD_MAPD);
+ its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
+ its_encode_size(cmd, size - 1);
+ its_encode_itt(cmd, itt_addr);
+ its_encode_valid(cmd, desc->its_mapd_cmd.valid);
+
+ its_fixup_cmd(cmd);
+ printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
+ desc->its_mapd_cmd.dev->device_id,
+ size, itt_addr, desc->its_mapd_cmd.valid);
+
+}
+
+static void its_build_mapc_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_MAPC);
+ its_encode_collection(cmd, desc->its_mapc_cmd.col->col_id);
+ its_encode_target(cmd, desc->its_mapc_cmd.col->target_address);
+ its_encode_valid(cmd, desc->its_mapc_cmd.valid);
+
+ its_fixup_cmd(cmd);
+ report_info("MAPC col_id=%d target_addr = 0x%lx valid=%d",
+ desc->its_mapc_cmd.col->col_id,
+ desc->its_mapc_cmd.col->target_address,
+ desc->its_mapc_cmd.valid);
+}
+
+static void its_build_mapti_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_MAPTI);
+ its_encode_devid(cmd, desc->its_mapti_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_mapti_cmd.event_id);
+ its_encode_phys_id(cmd, desc->its_mapti_cmd.phys_id);
+ its_encode_collection(cmd, desc->its_mapti_cmd.col_id);
+
+ its_fixup_cmd(cmd);
+ report_info("MAPTI dev_id=%d event_id=%d -> phys_id=%d, col_id=%d",
+ desc->its_mapti_cmd.dev->device_id,
+ desc->its_mapti_cmd.event_id,
+ desc->its_mapti_cmd.phys_id,
+ desc->its_mapti_cmd.col_id);
+}
+
+static void its_build_invall_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_INVALL);
+ its_encode_collection(cmd, desc->its_invall_cmd.col->col_id);
+
+ its_fixup_cmd(cmd);
+ report_info("INVALL col_id=%d", desc->its_invall_cmd.col->col_id);
+}
+
+static void its_build_clear_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_CLEAR);
+ its_encode_devid(cmd, desc->its_clear_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_clear_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("CLEAR col_id=%d", desc->its_invall_cmd.col->col_id);
+}
+
+static void its_build_discard_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_DISCARD);
+ its_encode_devid(cmd, desc->its_discard_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_discard_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("DISCARD col_id=%d", desc->its_invall_cmd.col->col_id);
+}
+
+static void its_build_inv_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_INV);
+ its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("INV dev_id=%d event_id=%d",
+ desc->its_inv_cmd.dev->device_id,
+ desc->its_inv_cmd.event_id);
+}
+
+static void its_build_int_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_INT);
+ its_encode_devid(cmd, desc->its_int_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_int_cmd.event_id);
+
+ its_fixup_cmd(cmd);
+ report_info("INT dev_id=%d event_id=%d",
+ desc->its_int_cmd.dev->device_id,
+ desc->its_int_cmd.event_id);
+}
+
+static void its_build_sync_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_SYNC);
+ its_encode_target(cmd, desc->its_sync_cmd.col->target_address);
+ its_fixup_cmd(cmd);
+ report_info("SYNC target_addr = 0x%lx",
+ desc->its_sync_cmd.col->target_address);
+}
+
+static void its_build_movi_cmd(struct its_cmd_block *cmd,
+ struct its_cmd_desc *desc)
+{
+ its_encode_cmd(cmd, GITS_CMD_MOVI);
+ its_encode_devid(cmd, desc->its_movi_cmd.dev->device_id);
+ its_encode_event_id(cmd, desc->its_movi_cmd.event_id);
+ its_encode_collection(cmd, desc->its_movi_cmd.col->col_id);
+
+ its_fixup_cmd(cmd);
+ report_info("MOVI dev_id=%d event_id = %d col_id=%d",
+ desc->its_movi_cmd.dev->device_id,
+ desc->its_movi_cmd.event_id,
+ desc->its_movi_cmd.col->col_id);
+}
+
+void its_send_mapd(struct its_device *dev, int valid)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapd_cmd.dev = dev;
+ desc.its_mapd_cmd.valid = !!valid;
+
+ its_send_single_command(its_build_mapd_cmd, &desc);
+}
+
+void its_send_mapc(struct its_collection *col, int valid)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapc_cmd.col = col;
+ desc.its_mapc_cmd.valid = !!valid;
+
+ its_send_single_command(its_build_mapc_cmd, &desc);
+}
+
+void its_send_mapti(struct its_device *dev, u32 irq_id,
+ u32 event_id, struct its_collection *col)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_mapti_cmd.dev = dev;
+ desc.its_mapti_cmd.phys_id = irq_id;
+ desc.its_mapti_cmd.event_id = event_id;
+ desc.its_mapti_cmd.col_id = col->col_id;
+
+ its_send_single_command(its_build_mapti_cmd, &desc);
+}
+
+void its_send_int(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_int_cmd.dev = dev;
+ desc.its_int_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_int_cmd, &desc);
+}
+
+void its_send_movi(struct its_device *dev,
+ struct its_collection *col, u32 id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_movi_cmd.dev = dev;
+ desc.its_movi_cmd.col = col;
+ desc.its_movi_cmd.event_id = id;
+
+ its_send_single_command(its_build_movi_cmd, &desc);
+}
+
+void its_send_invall(struct its_collection *col)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_invall_cmd.col = col;
+
+ its_send_single_command(its_build_invall_cmd, &desc);
+}
+
+void its_send_inv(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_inv_cmd.dev = dev;
+ desc.its_inv_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_inv_cmd, &desc);
+}
+
+void its_send_discard(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_discard_cmd.dev = dev;
+ desc.its_discard_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_discard_cmd, &desc);
+}
+
+void its_send_clear(struct its_device *dev, u32 event_id)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_clear_cmd.dev = dev;
+ desc.its_clear_cmd.event_id = event_id;
+
+ its_send_single_command(its_build_clear_cmd, &desc);
+}
+
+void its_send_sync(struct its_collection *col)
+{
+ struct its_cmd_desc desc;
+
+ desc.its_sync_cmd.col = col;
+
+ its_send_single_command(its_build_sync_cmd, &desc);
+}
+
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
Triggers LPIs through the INT command.
the test checks the LPI hits the right CPU and triggers
the right LPI intid, ie. the translation is correct.
Updates to the config table also are tested, along with inv
and invall commands.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- add comments
- keep the report_skip in case there aren't 4 vcpus to be able to
run other tests in the its category.
- fix the prefix pop
- move its_event and its_stats to arm/gic.c
---
arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
arm/unittests.cfg | 7 ++
2 files changed, 224 insertions(+), 11 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index 4d7dd03..50104b1 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
}
}
+static void setup_irq(handler_t handler)
+{
+ gic_enable_defaults();
+#ifdef __arm__
+ install_exception_handler(EXCPTN_IRQ, handler);
+#else
+ install_irq_handler(EL1H_IRQ, handler);
+#endif
+ local_irq_enable();
+}
+
+#if defined(__aarch64__)
+struct its_event {
+ int cpu_id;
+ int lpi_id;
+};
+
+struct its_stats {
+ struct its_event expected;
+ struct its_event observed;
+};
+
+static struct its_stats lpi_stats;
+
+static void lpi_handler(struct pt_regs *regs __unused)
+{
+ u32 irqstat = gic_read_iar();
+ int irqnr = gic_iar_irqnr(irqstat);
+
+ gic_write_eoir(irqstat);
+ if (irqnr < 8192)
+ report(false, "Unexpected non LPI interrupt received");
+ smp_rmb(); /* pairs with wmb in lpi_stats_expect */
+ lpi_stats.observed.cpu_id = smp_processor_id();
+ lpi_stats.observed.lpi_id = irqnr;
+ smp_wmb(); /* pairs with rmb in check_lpi_stats */
+}
+
+static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
+{
+ lpi_stats.expected.cpu_id = exp_cpu_id;
+ lpi_stats.expected.lpi_id = exp_lpi_id;
+ lpi_stats.observed.cpu_id = -1;
+ lpi_stats.observed.lpi_id = -1;
+ smp_wmb(); /* pairs with rmb in handler */
+}
+
+static void check_lpi_stats(void)
+{
+ mdelay(100);
+ smp_rmb(); /* pairs with wmb in lpi_handler */
+ if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
+ (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
+ if (lpi_stats.observed.cpu_id == -1 &&
+ lpi_stats.observed.lpi_id == -1) {
+ report(false,
+ "No LPI received whereas (cpuid=%d, intid=%d) "
+ "was expected", lpi_stats.expected.cpu_id,
+ lpi_stats.expected.lpi_id);
+ } else {
+ report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
+ lpi_stats.observed.cpu_id,
+ lpi_stats.observed.lpi_id);
+ }
+ } else if (lpi_stats.expected.lpi_id != -1) {
+ report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
+ lpi_stats.observed.cpu_id);
+ } else {
+ report(true, "no LPI received, as expected");
+ }
+}
+
+static void secondary_lpi_test(void)
+{
+ setup_irq(lpi_handler);
+ cpumask_set_cpu(smp_processor_id(), &ready);
+ while (1)
+ wfi();
+}
+#endif
+
static void gicv2_ipi_send_self(void)
{
writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
@@ -217,17 +298,6 @@ static void ipi_test_smp(void)
report_prefix_pop();
}
-static void setup_irq(handler_t handler)
-{
- gic_enable_defaults();
-#ifdef __arm__
- install_exception_handler(EXCPTN_IRQ, handler);
-#else
- install_irq_handler(EL1H_IRQ, handler);
-#endif
- local_irq_enable();
-}
-
static void ipi_send(void)
{
setup_irq(ipi_handler);
@@ -522,6 +592,7 @@ static void gic_test_mmio(void)
#if defined(__arm__)
static void test_its_introspection(void) {}
+static void test_its_trigger(void) {}
#else /* __arch64__ */
@@ -561,6 +632,137 @@ static void test_its_introspection(void)
report_info("collection baser entry_size = 0x%x", coll_baser->esz);
}
+static bool its_prerequisites(int nb_cpus)
+{
+ int cpu;
+
+ if (!gicv3_its_base()) {
+ report_skip("No ITS, skip ...");
+ return true;
+ }
+
+ if (nr_cpus < 4) {
+ report_skip("Test requires at least %d vcpus", nb_cpus);
+ return true;
+ }
+
+ stats_reset();
+
+ setup_irq(lpi_handler);
+
+ for_each_present_cpu(cpu) {
+ if (cpu == 0)
+ continue;
+ smp_boot_secondary(cpu, secondary_lpi_test);
+ }
+ wait_on_ready();
+
+ its_enable_defaults();
+
+ lpi_stats_expect(-1, -1);
+ check_lpi_stats();
+
+ return false;
+}
+
+static void test_its_trigger(void)
+{
+ struct its_collection *col3, *col2;
+ struct its_device *dev2, *dev7;
+
+ if (its_prerequisites(4))
+ return;
+
+ dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
+ dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
+
+ col3 = its_create_collection(3 /* col id */, 3/* target PE */);
+ col2 = its_create_collection(2 /* col id */, 2/* target PE */);
+
+ gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
+ gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
+
+ its_send_invall(col2);
+ its_send_invall(col3);
+
+ report_prefix_push("int");
+ /*
+ * dev=2, eventid=20 -> lpi= 8195, col=3
+ * dev=7, eventid=255 -> lpi= 8196, col=2
+ * Trigger dev2, eventid=20 and dev7, eventid=255
+ * Check both LPIs hit
+ */
+
+ its_send_mapd(dev2, true);
+ its_send_mapd(dev7, true);
+
+ its_send_mapc(col3, true);
+ its_send_mapc(col2, true);
+
+ its_send_mapti(dev2, 8195 /* lpi id */,
+ 20 /* event id */, col3);
+ its_send_mapti(dev7, 8196 /* lpi id */,
+ 255 /* event id */, col2);
+
+ lpi_stats_expect(3, 8195);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ lpi_stats_expect(2, 8196);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+
+ report_prefix_pop();
+
+ report_prefix_push("inv/invall");
+
+ /*
+ * disable 8195, check dev2/eventid=20 does not trigger the
+ * corresponding LPI
+ */
+ gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
+ its_send_inv(dev2, 20);
+
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ /*
+ * re-enable the LPI but willingly do not call invall
+ * so the change in config is not taken into account.
+ * The LPI should not hit
+ */
+ gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ /* Now call the invall and check the LPI hits */
+ its_send_invall(col3);
+ lpi_stats_expect(3, 8195);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ report_prefix_pop();
+ /*
+ * Unmap device 2 and check the eventid 20 formerly
+ * attached to it does not hit anymore
+ */
+ report_prefix_push("mapd valid=false");
+ its_send_mapd(dev2, false);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+ report_prefix_pop();
+
+ /* Unmap the collection this time and check no LPI does hit */
+ report_prefix_push("mapc valid=false");
+ its_send_mapc(col2, false);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+ report_prefix_pop();
+}
#endif
int main(int argc, char **argv)
@@ -594,6 +796,10 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
gic_test_mmio();
report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-trigger")) {
+ report_prefix_push(argv[1]);
+ test_its_trigger();
+ report_prefix_pop();
} else if (strcmp(argv[1], "its-introspection") == 0) {
report_prefix_push(argv[1]);
test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index ba2b31b..bfafec5 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
groups = its
arch = arm64
+[its-trigger]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'its-trigger'
+groups = its
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Triggers LPIs through the INT command.
the test checks the LPI hits the right CPU and triggers
the right LPI intid, ie. the translation is correct.
Updates to the config table also are tested, along with inv
and invall commands.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- add comments
- keep the report_skip in case there aren't 4 vcpus to be able to
run other tests in the its category.
- fix the prefix pop
- move its_event and its_stats to arm/gic.c
---
arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
arm/unittests.cfg | 7 ++
2 files changed, 224 insertions(+), 11 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index 4d7dd03..50104b1 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
}
}
+static void setup_irq(handler_t handler)
+{
+ gic_enable_defaults();
+#ifdef __arm__
+ install_exception_handler(EXCPTN_IRQ, handler);
+#else
+ install_irq_handler(EL1H_IRQ, handler);
+#endif
+ local_irq_enable();
+}
+
+#if defined(__aarch64__)
+struct its_event {
+ int cpu_id;
+ int lpi_id;
+};
+
+struct its_stats {
+ struct its_event expected;
+ struct its_event observed;
+};
+
+static struct its_stats lpi_stats;
+
+static void lpi_handler(struct pt_regs *regs __unused)
+{
+ u32 irqstat = gic_read_iar();
+ int irqnr = gic_iar_irqnr(irqstat);
+
+ gic_write_eoir(irqstat);
+ if (irqnr < 8192)
+ report(false, "Unexpected non LPI interrupt received");
+ smp_rmb(); /* pairs with wmb in lpi_stats_expect */
+ lpi_stats.observed.cpu_id = smp_processor_id();
+ lpi_stats.observed.lpi_id = irqnr;
+ smp_wmb(); /* pairs with rmb in check_lpi_stats */
+}
+
+static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
+{
+ lpi_stats.expected.cpu_id = exp_cpu_id;
+ lpi_stats.expected.lpi_id = exp_lpi_id;
+ lpi_stats.observed.cpu_id = -1;
+ lpi_stats.observed.lpi_id = -1;
+ smp_wmb(); /* pairs with rmb in handler */
+}
+
+static void check_lpi_stats(void)
+{
+ mdelay(100);
+ smp_rmb(); /* pairs with wmb in lpi_handler */
+ if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
+ (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
+ if (lpi_stats.observed.cpu_id == -1 &&
+ lpi_stats.observed.lpi_id == -1) {
+ report(false,
+ "No LPI received whereas (cpuid=%d, intid=%d) "
+ "was expected", lpi_stats.expected.cpu_id,
+ lpi_stats.expected.lpi_id);
+ } else {
+ report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
+ lpi_stats.observed.cpu_id,
+ lpi_stats.observed.lpi_id);
+ }
+ } else if (lpi_stats.expected.lpi_id != -1) {
+ report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
+ lpi_stats.observed.cpu_id);
+ } else {
+ report(true, "no LPI received, as expected");
+ }
+}
+
+static void secondary_lpi_test(void)
+{
+ setup_irq(lpi_handler);
+ cpumask_set_cpu(smp_processor_id(), &ready);
+ while (1)
+ wfi();
+}
+#endif
+
static void gicv2_ipi_send_self(void)
{
writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
@@ -217,17 +298,6 @@ static void ipi_test_smp(void)
report_prefix_pop();
}
-static void setup_irq(handler_t handler)
-{
- gic_enable_defaults();
-#ifdef __arm__
- install_exception_handler(EXCPTN_IRQ, handler);
-#else
- install_irq_handler(EL1H_IRQ, handler);
-#endif
- local_irq_enable();
-}
-
static void ipi_send(void)
{
setup_irq(ipi_handler);
@@ -522,6 +592,7 @@ static void gic_test_mmio(void)
#if defined(__arm__)
static void test_its_introspection(void) {}
+static void test_its_trigger(void) {}
#else /* __arch64__ */
@@ -561,6 +632,137 @@ static void test_its_introspection(void)
report_info("collection baser entry_size = 0x%x", coll_baser->esz);
}
+static bool its_prerequisites(int nb_cpus)
+{
+ int cpu;
+
+ if (!gicv3_its_base()) {
+ report_skip("No ITS, skip ...");
+ return true;
+ }
+
+ if (nr_cpus < 4) {
+ report_skip("Test requires at least %d vcpus", nb_cpus);
+ return true;
+ }
+
+ stats_reset();
+
+ setup_irq(lpi_handler);
+
+ for_each_present_cpu(cpu) {
+ if (cpu == 0)
+ continue;
+ smp_boot_secondary(cpu, secondary_lpi_test);
+ }
+ wait_on_ready();
+
+ its_enable_defaults();
+
+ lpi_stats_expect(-1, -1);
+ check_lpi_stats();
+
+ return false;
+}
+
+static void test_its_trigger(void)
+{
+ struct its_collection *col3, *col2;
+ struct its_device *dev2, *dev7;
+
+ if (its_prerequisites(4))
+ return;
+
+ dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
+ dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
+
+ col3 = its_create_collection(3 /* col id */, 3/* target PE */);
+ col2 = its_create_collection(2 /* col id */, 2/* target PE */);
+
+ gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
+ gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
+
+ its_send_invall(col2);
+ its_send_invall(col3);
+
+ report_prefix_push("int");
+ /*
+ * dev=2, eventid=20 -> lpi= 8195, col=3
+ * dev=7, eventid=255 -> lpi= 8196, col=2
+ * Trigger dev2, eventid=20 and dev7, eventid=255
+ * Check both LPIs hit
+ */
+
+ its_send_mapd(dev2, true);
+ its_send_mapd(dev7, true);
+
+ its_send_mapc(col3, true);
+ its_send_mapc(col2, true);
+
+ its_send_mapti(dev2, 8195 /* lpi id */,
+ 20 /* event id */, col3);
+ its_send_mapti(dev7, 8196 /* lpi id */,
+ 255 /* event id */, col2);
+
+ lpi_stats_expect(3, 8195);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ lpi_stats_expect(2, 8196);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+
+ report_prefix_pop();
+
+ report_prefix_push("inv/invall");
+
+ /*
+ * disable 8195, check dev2/eventid=20 does not trigger the
+ * corresponding LPI
+ */
+ gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
+ its_send_inv(dev2, 20);
+
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ /*
+ * re-enable the LPI but willingly do not call invall
+ * so the change in config is not taken into account.
+ * The LPI should not hit
+ */
+ gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ /* Now call the invall and check the LPI hits */
+ its_send_invall(col3);
+ lpi_stats_expect(3, 8195);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ report_prefix_pop();
+ /*
+ * Unmap device 2 and check the eventid 20 formerly
+ * attached to it does not hit anymore
+ */
+ report_prefix_push("mapd valid=false");
+ its_send_mapd(dev2, false);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+ report_prefix_pop();
+
+ /* Unmap the collection this time and check no LPI does hit */
+ report_prefix_push("mapc valid=false");
+ its_send_mapc(col2, false);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+ report_prefix_pop();
+}
#endif
int main(int argc, char **argv)
@@ -594,6 +796,10 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
gic_test_mmio();
report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-trigger")) {
+ report_prefix_push(argv[1]);
+ test_its_trigger();
+ report_prefix_pop();
} else if (strcmp(argv[1], "its-introspection") == 0) {
report_prefix_push(argv[1]);
test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index ba2b31b..bfafec5 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
groups = its
arch = arm64
+[its-trigger]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'its-trigger'
+groups = its
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Triggers LPIs through the INT command.
the test checks the LPI hits the right CPU and triggers
the right LPI intid, ie. the translation is correct.
Updates to the config table also are tested, along with inv
and invall commands.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- add comments
- keep the report_skip in case there aren't 4 vcpus to be able to
run other tests in the its category.
- fix the prefix pop
- move its_event and its_stats to arm/gic.c
---
arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
arm/unittests.cfg | 7 ++
2 files changed, 224 insertions(+), 11 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index 4d7dd03..50104b1 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
}
}
+static void setup_irq(handler_t handler)
+{
+ gic_enable_defaults();
+#ifdef __arm__
+ install_exception_handler(EXCPTN_IRQ, handler);
+#else
+ install_irq_handler(EL1H_IRQ, handler);
+#endif
+ local_irq_enable();
+}
+
+#if defined(__aarch64__)
+struct its_event {
+ int cpu_id;
+ int lpi_id;
+};
+
+struct its_stats {
+ struct its_event expected;
+ struct its_event observed;
+};
+
+static struct its_stats lpi_stats;
+
+static void lpi_handler(struct pt_regs *regs __unused)
+{
+ u32 irqstat = gic_read_iar();
+ int irqnr = gic_iar_irqnr(irqstat);
+
+ gic_write_eoir(irqstat);
+ if (irqnr < 8192)
+ report(false, "Unexpected non LPI interrupt received");
+ smp_rmb(); /* pairs with wmb in lpi_stats_expect */
+ lpi_stats.observed.cpu_id = smp_processor_id();
+ lpi_stats.observed.lpi_id = irqnr;
+ smp_wmb(); /* pairs with rmb in check_lpi_stats */
+}
+
+static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
+{
+ lpi_stats.expected.cpu_id = exp_cpu_id;
+ lpi_stats.expected.lpi_id = exp_lpi_id;
+ lpi_stats.observed.cpu_id = -1;
+ lpi_stats.observed.lpi_id = -1;
+ smp_wmb(); /* pairs with rmb in handler */
+}
+
+static void check_lpi_stats(void)
+{
+ mdelay(100);
+ smp_rmb(); /* pairs with wmb in lpi_handler */
+ if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
+ (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
+ if (lpi_stats.observed.cpu_id == -1 &&
+ lpi_stats.observed.lpi_id == -1) {
+ report(false,
+ "No LPI received whereas (cpuid=%d, intid=%d) "
+ "was expected", lpi_stats.expected.cpu_id,
+ lpi_stats.expected.lpi_id);
+ } else {
+ report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
+ lpi_stats.observed.cpu_id,
+ lpi_stats.observed.lpi_id);
+ }
+ } else if (lpi_stats.expected.lpi_id != -1) {
+ report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
+ lpi_stats.observed.cpu_id);
+ } else {
+ report(true, "no LPI received, as expected");
+ }
+}
+
+static void secondary_lpi_test(void)
+{
+ setup_irq(lpi_handler);
+ cpumask_set_cpu(smp_processor_id(), &ready);
+ while (1)
+ wfi();
+}
+#endif
+
static void gicv2_ipi_send_self(void)
{
writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
@@ -217,17 +298,6 @@ static void ipi_test_smp(void)
report_prefix_pop();
}
-static void setup_irq(handler_t handler)
-{
- gic_enable_defaults();
-#ifdef __arm__
- install_exception_handler(EXCPTN_IRQ, handler);
-#else
- install_irq_handler(EL1H_IRQ, handler);
-#endif
- local_irq_enable();
-}
-
static void ipi_send(void)
{
setup_irq(ipi_handler);
@@ -522,6 +592,7 @@ static void gic_test_mmio(void)
#if defined(__arm__)
static void test_its_introspection(void) {}
+static void test_its_trigger(void) {}
#else /* __arch64__ */
@@ -561,6 +632,137 @@ static void test_its_introspection(void)
report_info("collection baser entry_size = 0x%x", coll_baser->esz);
}
+static bool its_prerequisites(int nb_cpus)
+{
+ int cpu;
+
+ if (!gicv3_its_base()) {
+ report_skip("No ITS, skip ...");
+ return true;
+ }
+
+ if (nr_cpus < 4) {
+ report_skip("Test requires at least %d vcpus", nb_cpus);
+ return true;
+ }
+
+ stats_reset();
+
+ setup_irq(lpi_handler);
+
+ for_each_present_cpu(cpu) {
+ if (cpu == 0)
+ continue;
+ smp_boot_secondary(cpu, secondary_lpi_test);
+ }
+ wait_on_ready();
+
+ its_enable_defaults();
+
+ lpi_stats_expect(-1, -1);
+ check_lpi_stats();
+
+ return false;
+}
+
+static void test_its_trigger(void)
+{
+ struct its_collection *col3, *col2;
+ struct its_device *dev2, *dev7;
+
+ if (its_prerequisites(4))
+ return;
+
+ dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
+ dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
+
+ col3 = its_create_collection(3 /* col id */, 3/* target PE */);
+ col2 = its_create_collection(2 /* col id */, 2/* target PE */);
+
+ gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
+ gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
+
+ its_send_invall(col2);
+ its_send_invall(col3);
+
+ report_prefix_push("int");
+ /*
+ * dev=2, eventid=20 -> lpi= 8195, col=3
+ * dev=7, eventid=255 -> lpi= 8196, col=2
+ * Trigger dev2, eventid=20 and dev7, eventid=255
+ * Check both LPIs hit
+ */
+
+ its_send_mapd(dev2, true);
+ its_send_mapd(dev7, true);
+
+ its_send_mapc(col3, true);
+ its_send_mapc(col2, true);
+
+ its_send_mapti(dev2, 8195 /* lpi id */,
+ 20 /* event id */, col3);
+ its_send_mapti(dev7, 8196 /* lpi id */,
+ 255 /* event id */, col2);
+
+ lpi_stats_expect(3, 8195);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ lpi_stats_expect(2, 8196);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+
+ report_prefix_pop();
+
+ report_prefix_push("inv/invall");
+
+ /*
+ * disable 8195, check dev2/eventid=20 does not trigger the
+ * corresponding LPI
+ */
+ gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
+ its_send_inv(dev2, 20);
+
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ /*
+ * re-enable the LPI but willingly do not call invall
+ * so the change in config is not taken into account.
+ * The LPI should not hit
+ */
+ gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ /* Now call the invall and check the LPI hits */
+ its_send_invall(col3);
+ lpi_stats_expect(3, 8195);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ report_prefix_pop();
+ /*
+ * Unmap device 2 and check the eventid 20 formerly
+ * attached to it does not hit anymore
+ */
+ report_prefix_push("mapd valid=false");
+ its_send_mapd(dev2, false);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+ report_prefix_pop();
+
+ /* Unmap the collection this time and check no LPI does hit */
+ report_prefix_push("mapc valid=false");
+ its_send_mapc(col2, false);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+ report_prefix_pop();
+}
#endif
int main(int argc, char **argv)
@@ -594,6 +796,10 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
gic_test_mmio();
report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-trigger")) {
+ report_prefix_push(argv[1]);
+ test_its_trigger();
+ report_prefix_pop();
} else if (strcmp(argv[1], "its-introspection") == 0) {
report_prefix_push(argv[1]);
test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index ba2b31b..bfafec5 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
groups = its
arch = arm64
+[its-trigger]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'its-trigger'
+groups = its
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 12/14] arm/run: Allow Migration tests
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
Let's link getchar.o to use puts and getchar from the
tests.
Then allow tests belonging to the migration group to
trigger the migration from the test code by putting
"migrate" into the uart. Then the code can wait for the
migration completion by using getchar().
The __getchar implement is minimalist as it just reads the
data register. It is just meant to read the single character
emitted at the end of the migration by the runner script.
It is not meant to read more data (FIFOs are not enabled).
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- take the lock
- assert if more than 16 chars
- removed Thomas' R-b
---
arm/Makefile.common | 2 +-
arm/run | 2 +-
lib/arm/io.c | 28 ++++++++++++++++++++++++++++
3 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arm/Makefile.common b/arm/Makefile.common
index b8988f2..a123e85 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -32,7 +32,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
asm-offsets = lib/$(ARCH)/asm-offsets.h
include $(SRCDIR)/scripts/asm-offsets.mak
-cflatobjs += lib/util.o
+cflatobjs += lib/util.o lib/getchar.o
cflatobjs += lib/alloc_phys.o
cflatobjs += lib/alloc_page.o
cflatobjs += lib/vmalloc.o
diff --git a/arm/run b/arm/run
index 277db9b..a390ca5 100755
--- a/arm/run
+++ b/arm/run
@@ -61,6 +61,6 @@ fi
M+=",accel=$ACCEL"
command="$qemu -nodefaults $M -cpu $processor $chr_testdev $pci_testdev"
command+=" -display none -serial stdio -kernel"
-command="$(timeout_cmd) $command"
+command="$(migration_cmd) $(timeout_cmd) $command"
run_qemu $command "$@"
diff --git a/lib/arm/io.c b/lib/arm/io.c
index 99fd315..d8e7745 100644
--- a/lib/arm/io.c
+++ b/lib/arm/io.c
@@ -87,6 +87,34 @@ void puts(const char *s)
spin_unlock(&uart_lock);
}
+static int ____getchar(void)
+{
+ int c;
+
+ spin_lock(&uart_lock);
+ c = readb(uart0_base);
+ spin_unlock(&uart_lock);
+
+ return c ? : -1;
+}
+
+/*
+ * Minimalist implementation for migration completion detection.
+ * Without FIFOs enabled on the QEMU UART device we just read
+ * the data register: we cannot read more than 16 characters.
+ */
+int __getchar(void)
+{
+ int c = ____getchar();
+ static int count;
+
+ if (c != -1)
+ ++count;
+
+ assert(count < 16);
+
+ return c;
+}
/*
* Defining halt to take 'code' as an argument guarantees that it will
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 12/14] arm/run: Allow Migration tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Let's link getchar.o to use puts and getchar from the
tests.
Then allow tests belonging to the migration group to
trigger the migration from the test code by putting
"migrate" into the uart. Then the code can wait for the
migration completion by using getchar().
The __getchar implement is minimalist as it just reads the
data register. It is just meant to read the single character
emitted at the end of the migration by the runner script.
It is not meant to read more data (FIFOs are not enabled).
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- take the lock
- assert if more than 16 chars
- removed Thomas' R-b
---
arm/Makefile.common | 2 +-
arm/run | 2 +-
lib/arm/io.c | 28 ++++++++++++++++++++++++++++
3 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arm/Makefile.common b/arm/Makefile.common
index b8988f2..a123e85 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -32,7 +32,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
asm-offsets = lib/$(ARCH)/asm-offsets.h
include $(SRCDIR)/scripts/asm-offsets.mak
-cflatobjs += lib/util.o
+cflatobjs += lib/util.o lib/getchar.o
cflatobjs += lib/alloc_phys.o
cflatobjs += lib/alloc_page.o
cflatobjs += lib/vmalloc.o
diff --git a/arm/run b/arm/run
index 277db9b..a390ca5 100755
--- a/arm/run
+++ b/arm/run
@@ -61,6 +61,6 @@ fi
M+=",accel=$ACCEL"
command="$qemu -nodefaults $M -cpu $processor $chr_testdev $pci_testdev"
command+=" -display none -serial stdio -kernel"
-command="$(timeout_cmd) $command"
+command="$(migration_cmd) $(timeout_cmd) $command"
run_qemu $command "$@"
diff --git a/lib/arm/io.c b/lib/arm/io.c
index 99fd315..d8e7745 100644
--- a/lib/arm/io.c
+++ b/lib/arm/io.c
@@ -87,6 +87,34 @@ void puts(const char *s)
spin_unlock(&uart_lock);
}
+static int ____getchar(void)
+{
+ int c;
+
+ spin_lock(&uart_lock);
+ c = readb(uart0_base);
+ spin_unlock(&uart_lock);
+
+ return c ? : -1;
+}
+
+/*
+ * Minimalist implementation for migration completion detection.
+ * Without FIFOs enabled on the QEMU UART device we just read
+ * the data register: we cannot read more than 16 characters.
+ */
+int __getchar(void)
+{
+ int c = ____getchar();
+ static int count;
+
+ if (c != -1)
+ ++count;
+
+ assert(count < 16);
+
+ return c;
+}
/*
* Defining halt to take 'code' as an argument guarantees that it will
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 12/14] arm/run: Allow Migration tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Let's link getchar.o to use puts and getchar from the
tests.
Then allow tests belonging to the migration group to
trigger the migration from the test code by putting
"migrate" into the uart. Then the code can wait for the
migration completion by using getchar().
The __getchar implement is minimalist as it just reads the
data register. It is just meant to read the single character
emitted at the end of the migration by the runner script.
It is not meant to read more data (FIFOs are not enabled).
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- take the lock
- assert if more than 16 chars
- removed Thomas' R-b
---
arm/Makefile.common | 2 +-
arm/run | 2 +-
lib/arm/io.c | 28 ++++++++++++++++++++++++++++
3 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arm/Makefile.common b/arm/Makefile.common
index b8988f2..a123e85 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -32,7 +32,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
asm-offsets = lib/$(ARCH)/asm-offsets.h
include $(SRCDIR)/scripts/asm-offsets.mak
-cflatobjs += lib/util.o
+cflatobjs += lib/util.o lib/getchar.o
cflatobjs += lib/alloc_phys.o
cflatobjs += lib/alloc_page.o
cflatobjs += lib/vmalloc.o
diff --git a/arm/run b/arm/run
index 277db9b..a390ca5 100755
--- a/arm/run
+++ b/arm/run
@@ -61,6 +61,6 @@ fi
M+=",accel=$ACCEL"
command="$qemu -nodefaults $M -cpu $processor $chr_testdev $pci_testdev"
command+=" -display none -serial stdio -kernel"
-command="$(timeout_cmd) $command"
+command="$(migration_cmd) $(timeout_cmd) $command"
run_qemu $command "$@"
diff --git a/lib/arm/io.c b/lib/arm/io.c
index 99fd315..d8e7745 100644
--- a/lib/arm/io.c
+++ b/lib/arm/io.c
@@ -87,6 +87,34 @@ void puts(const char *s)
spin_unlock(&uart_lock);
}
+static int ____getchar(void)
+{
+ int c;
+
+ spin_lock(&uart_lock);
+ c = readb(uart0_base);
+ spin_unlock(&uart_lock);
+
+ return c ? : -1;
+}
+
+/*
+ * Minimalist implementation for migration completion detection.
+ * Without FIFOs enabled on the QEMU UART device we just read
+ * the data register: we cannot read more than 16 characters.
+ */
+int __getchar(void)
+{
+ int c = ____getchar();
+ static int count;
+
+ if (c != -1)
+ ++count;
+
+ assert(count < 16);
+
+ return c;
+}
/*
* Defining halt to take 'code' as an argument guarantees that it will
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 13/14] arm/arm64: ITS: migration tests
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
This test maps LPIs (populates the device table, the collection table,
interrupt translation tables, configuration table), migrates and make
sure the translation is correct on the destination.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
arm/gic.c | 59 ++++++++++++++++++++++++++++++++++++----
arm/unittests.cfg | 8 ++++++
lib/arm/asm/gic-v3-its.h | 2 ++
lib/arm/gic-v3-its.c | 22 +++++++++++++++
4 files changed, 85 insertions(+), 6 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index 50104b1..fa8626a 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -593,6 +593,7 @@ static void gic_test_mmio(void)
static void test_its_introspection(void) {}
static void test_its_trigger(void) {}
+static void test_its_migration(void) {}
#else /* __arch64__ */
@@ -665,13 +666,19 @@ static bool its_prerequisites(int nb_cpus)
return false;
}
-static void test_its_trigger(void)
+/*
+ * Setup the configuration for those mappings:
+ * dev_id=2 event=20 -> vcpu 3, intid=8195
+ * dev_id=7 event=255 -> vcpu 2, intid=8196
+ * LPIs ready to hit
+ */
+static int its_setup1(void)
{
struct its_collection *col3, *col2;
struct its_device *dev2, *dev7;
if (its_prerequisites(4))
- return;
+ return -1;
dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
@@ -685,14 +692,10 @@ static void test_its_trigger(void)
its_send_invall(col2);
its_send_invall(col3);
- report_prefix_push("int");
/*
* dev=2, eventid=20 -> lpi= 8195, col=3
* dev=7, eventid=255 -> lpi= 8196, col=2
- * Trigger dev2, eventid=20 and dev7, eventid=255
- * Check both LPIs hit
*/
-
its_send_mapd(dev2, true);
its_send_mapd(dev7, true);
@@ -703,6 +706,23 @@ static void test_its_trigger(void)
20 /* event id */, col3);
its_send_mapti(dev7, 8196 /* lpi id */,
255 /* event id */, col2);
+ return 0;
+}
+
+static void test_its_trigger(void)
+{
+ struct its_collection *col3, *col2;
+ struct its_device *dev2, *dev7;
+
+ if (its_setup1())
+ return;
+
+ col3 = its_get_collection(3);
+ col2 = its_get_collection(2);
+ dev2 = its_get_device(2);
+ dev7 = its_get_device(7);
+
+ report_prefix_push("int");
lpi_stats_expect(3, 8195);
its_send_int(dev2, 20);
@@ -763,6 +783,29 @@ static void test_its_trigger(void)
check_lpi_stats();
report_prefix_pop();
}
+
+static void test_its_migration(void)
+{
+ struct its_device *dev2, *dev7;
+
+ if (its_setup1())
+ return;
+
+ dev2 = its_get_device(2);
+ dev7 = its_get_device(7);
+
+ puts("Now migrate the VM, then press a key to continue...\n");
+ (void)getchar();
+ report(true, "Migration complete");
+
+ lpi_stats_expect(3, 8195);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ lpi_stats_expect(2, 8196);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+}
#endif
int main(int argc, char **argv)
@@ -800,6 +843,10 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
test_its_trigger();
report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-migration")) {
+ report_prefix_push(argv[1]);
+ test_its_migration();
+ report_prefix_pop();
} else if (strcmp(argv[1], "its-introspection") == 0) {
report_prefix_push(argv[1]);
test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index bfafec5..8b8ec79 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
groups = its
arch = arm64
+[its-migration]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-migration'
+groups = its migration
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index 0e5c5b6..febc2b2 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -151,6 +151,8 @@ extern void its_send_invall(struct its_collection *col);
extern void its_send_movi(struct its_device *dev,
struct its_collection *col, u32 id);
extern void its_send_sync(struct its_collection *col);
+extern struct its_device *its_get_device(u32 id);
+extern struct its_collection *its_get_collection(u32 id);
#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
index c2dcd01..099940e 100644
--- a/lib/arm/gic-v3-its.c
+++ b/lib/arm/gic-v3-its.c
@@ -219,3 +219,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
its_data.nr_collections++;
return new;
}
+
+struct its_device *its_get_device(u32 id)
+{
+ int i;
+
+ for (i = 0; i < GITS_MAX_DEVICES; i++) {
+ if (its_data.devices[i].device_id == id)
+ return &its_data.devices[i];
+ }
+ return NULL;
+}
+
+struct its_collection *its_get_collection(u32 id)
+{
+ int i;
+
+ for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
+ if (its_data.collections[i].col_id == id)
+ return &its_data.collections[i];
+ }
+ return NULL;
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 13/14] arm/arm64: ITS: migration tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
This test maps LPIs (populates the device table, the collection table,
interrupt translation tables, configuration table), migrates and make
sure the translation is correct on the destination.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
arm/gic.c | 59 ++++++++++++++++++++++++++++++++++++----
arm/unittests.cfg | 8 ++++++
lib/arm/asm/gic-v3-its.h | 2 ++
lib/arm/gic-v3-its.c | 22 +++++++++++++++
4 files changed, 85 insertions(+), 6 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index 50104b1..fa8626a 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -593,6 +593,7 @@ static void gic_test_mmio(void)
static void test_its_introspection(void) {}
static void test_its_trigger(void) {}
+static void test_its_migration(void) {}
#else /* __arch64__ */
@@ -665,13 +666,19 @@ static bool its_prerequisites(int nb_cpus)
return false;
}
-static void test_its_trigger(void)
+/*
+ * Setup the configuration for those mappings:
+ * dev_id=2 event=20 -> vcpu 3, intid=8195
+ * dev_id=7 event=255 -> vcpu 2, intid=8196
+ * LPIs ready to hit
+ */
+static int its_setup1(void)
{
struct its_collection *col3, *col2;
struct its_device *dev2, *dev7;
if (its_prerequisites(4))
- return;
+ return -1;
dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
@@ -685,14 +692,10 @@ static void test_its_trigger(void)
its_send_invall(col2);
its_send_invall(col3);
- report_prefix_push("int");
/*
* dev=2, eventid=20 -> lpi= 8195, col=3
* dev=7, eventid=255 -> lpi= 8196, col=2
- * Trigger dev2, eventid=20 and dev7, eventid=255
- * Check both LPIs hit
*/
-
its_send_mapd(dev2, true);
its_send_mapd(dev7, true);
@@ -703,6 +706,23 @@ static void test_its_trigger(void)
20 /* event id */, col3);
its_send_mapti(dev7, 8196 /* lpi id */,
255 /* event id */, col2);
+ return 0;
+}
+
+static void test_its_trigger(void)
+{
+ struct its_collection *col3, *col2;
+ struct its_device *dev2, *dev7;
+
+ if (its_setup1())
+ return;
+
+ col3 = its_get_collection(3);
+ col2 = its_get_collection(2);
+ dev2 = its_get_device(2);
+ dev7 = its_get_device(7);
+
+ report_prefix_push("int");
lpi_stats_expect(3, 8195);
its_send_int(dev2, 20);
@@ -763,6 +783,29 @@ static void test_its_trigger(void)
check_lpi_stats();
report_prefix_pop();
}
+
+static void test_its_migration(void)
+{
+ struct its_device *dev2, *dev7;
+
+ if (its_setup1())
+ return;
+
+ dev2 = its_get_device(2);
+ dev7 = its_get_device(7);
+
+ puts("Now migrate the VM, then press a key to continue...\n");
+ (void)getchar();
+ report(true, "Migration complete");
+
+ lpi_stats_expect(3, 8195);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ lpi_stats_expect(2, 8196);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+}
#endif
int main(int argc, char **argv)
@@ -800,6 +843,10 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
test_its_trigger();
report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-migration")) {
+ report_prefix_push(argv[1]);
+ test_its_migration();
+ report_prefix_pop();
} else if (strcmp(argv[1], "its-introspection") == 0) {
report_prefix_push(argv[1]);
test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index bfafec5..8b8ec79 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
groups = its
arch = arm64
+[its-migration]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-migration'
+groups = its migration
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index 0e5c5b6..febc2b2 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -151,6 +151,8 @@ extern void its_send_invall(struct its_collection *col);
extern void its_send_movi(struct its_device *dev,
struct its_collection *col, u32 id);
extern void its_send_sync(struct its_collection *col);
+extern struct its_device *its_get_device(u32 id);
+extern struct its_collection *its_get_collection(u32 id);
#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
index c2dcd01..099940e 100644
--- a/lib/arm/gic-v3-its.c
+++ b/lib/arm/gic-v3-its.c
@@ -219,3 +219,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
its_data.nr_collections++;
return new;
}
+
+struct its_device *its_get_device(u32 id)
+{
+ int i;
+
+ for (i = 0; i < GITS_MAX_DEVICES; i++) {
+ if (its_data.devices[i].device_id == id)
+ return &its_data.devices[i];
+ }
+ return NULL;
+}
+
+struct its_collection *its_get_collection(u32 id)
+{
+ int i;
+
+ for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
+ if (its_data.collections[i].col_id == id)
+ return &its_data.collections[i];
+ }
+ return NULL;
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 13/14] arm/arm64: ITS: migration tests
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
This test maps LPIs (populates the device table, the collection table,
interrupt translation tables, configuration table), migrates and make
sure the translation is correct on the destination.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
arm/gic.c | 59 ++++++++++++++++++++++++++++++++++++----
arm/unittests.cfg | 8 ++++++
lib/arm/asm/gic-v3-its.h | 2 ++
lib/arm/gic-v3-its.c | 22 +++++++++++++++
4 files changed, 85 insertions(+), 6 deletions(-)
diff --git a/arm/gic.c b/arm/gic.c
index 50104b1..fa8626a 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -593,6 +593,7 @@ static void gic_test_mmio(void)
static void test_its_introspection(void) {}
static void test_its_trigger(void) {}
+static void test_its_migration(void) {}
#else /* __arch64__ */
@@ -665,13 +666,19 @@ static bool its_prerequisites(int nb_cpus)
return false;
}
-static void test_its_trigger(void)
+/*
+ * Setup the configuration for those mappings:
+ * dev_id=2 event=20 -> vcpu 3, intid=8195
+ * dev_id=7 event=255 -> vcpu 2, intid=8196
+ * LPIs ready to hit
+ */
+static int its_setup1(void)
{
struct its_collection *col3, *col2;
struct its_device *dev2, *dev7;
if (its_prerequisites(4))
- return;
+ return -1;
dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
@@ -685,14 +692,10 @@ static void test_its_trigger(void)
its_send_invall(col2);
its_send_invall(col3);
- report_prefix_push("int");
/*
* dev=2, eventid=20 -> lpi= 8195, col=3
* dev=7, eventid=255 -> lpi= 8196, col=2
- * Trigger dev2, eventid=20 and dev7, eventid=255
- * Check both LPIs hit
*/
-
its_send_mapd(dev2, true);
its_send_mapd(dev7, true);
@@ -703,6 +706,23 @@ static void test_its_trigger(void)
20 /* event id */, col3);
its_send_mapti(dev7, 8196 /* lpi id */,
255 /* event id */, col2);
+ return 0;
+}
+
+static void test_its_trigger(void)
+{
+ struct its_collection *col3, *col2;
+ struct its_device *dev2, *dev7;
+
+ if (its_setup1())
+ return;
+
+ col3 = its_get_collection(3);
+ col2 = its_get_collection(2);
+ dev2 = its_get_device(2);
+ dev7 = its_get_device(7);
+
+ report_prefix_push("int");
lpi_stats_expect(3, 8195);
its_send_int(dev2, 20);
@@ -763,6 +783,29 @@ static void test_its_trigger(void)
check_lpi_stats();
report_prefix_pop();
}
+
+static void test_its_migration(void)
+{
+ struct its_device *dev2, *dev7;
+
+ if (its_setup1())
+ return;
+
+ dev2 = its_get_device(2);
+ dev7 = its_get_device(7);
+
+ puts("Now migrate the VM, then press a key to continue...\n");
+ (void)getchar();
+ report(true, "Migration complete");
+
+ lpi_stats_expect(3, 8195);
+ its_send_int(dev2, 20);
+ check_lpi_stats();
+
+ lpi_stats_expect(2, 8196);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+}
#endif
int main(int argc, char **argv)
@@ -800,6 +843,10 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
test_its_trigger();
report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-migration")) {
+ report_prefix_push(argv[1]);
+ test_its_migration();
+ report_prefix_pop();
} else if (strcmp(argv[1], "its-introspection") == 0) {
report_prefix_push(argv[1]);
test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index bfafec5..8b8ec79 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
groups = its
arch = arm64
+[its-migration]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-migration'
+groups = its migration
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index 0e5c5b6..febc2b2 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -151,6 +151,8 @@ extern void its_send_invall(struct its_collection *col);
extern void its_send_movi(struct its_device *dev,
struct its_collection *col, u32 id);
extern void its_send_sync(struct its_collection *col);
+extern struct its_device *its_get_device(u32 id);
+extern struct its_collection *its_get_collection(u32 id);
#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
index c2dcd01..099940e 100644
--- a/lib/arm/gic-v3-its.c
+++ b/lib/arm/gic-v3-its.c
@@ -219,3 +219,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
its_data.nr_collections++;
return new;
}
+
+struct its_device *its_get_device(u32 id)
+{
+ int i;
+
+ for (i = 0; i < GITS_MAX_DEVICES; i++) {
+ if (its_data.devices[i].device_id == id)
+ return &its_data.devices[i];
+ }
+ return NULL;
+}
+
+struct its_collection *its_get_collection(u32 id)
+{
+ int i;
+
+ for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
+ if (its_data.collections[i].col_id == id)
+ return &its_data.collections[i];
+ }
+ return NULL;
+}
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-28 10:34 ` Eric Auger
-1 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui,
alexandru.elisei, thuth
Add two new migration tests. One testing the migration of
a topology where collection were unmapped. The second test
checks the migration of the pending table.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- tests belong to both its and migration groups
---
arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
arm/unittests.cfg | 16 +++++
2 files changed, 166 insertions(+)
diff --git a/arm/gic.c b/arm/gic.c
index fa8626a..ec3dd3a 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
smp_rmb(); /* pairs with wmb in lpi_stats_expect */
lpi_stats.observed.cpu_id = smp_processor_id();
lpi_stats.observed.lpi_id = irqnr;
+ acked[lpi_stats.observed.cpu_id]++;
smp_wmb(); /* pairs with rmb in check_lpi_stats */
}
@@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
while (1)
wfi();
}
+
+static void check_lpi_hits(int *expected)
+{
+ int i;
+
+ for (i = 0; i < nr_cpus; i++) {
+ if (acked[i] != expected[i])
+ report(false, "expected %d LPIs on PE #%d, %d observed",
+ expected[i], i, acked[i]);
+ }
+ report(true, "check LPI on all vcpus");
+}
#endif
static void gicv2_ipi_send_self(void)
@@ -594,6 +607,8 @@ static void gic_test_mmio(void)
static void test_its_introspection(void) {}
static void test_its_trigger(void) {}
static void test_its_migration(void) {}
+static void test_migrate_unmapped_collection(void) {}
+static void test_its_pending_migration(void) {}
#else /* __arch64__ */
@@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
return false;
}
+static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
+ struct its_collection *col)
+{
+ if (!dev || !col)
+ report_abort("wrong device or collection");
+
+ its_send_mapti(dev, physid, eventid, col);
+
+ gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
+ its_send_invall(col);
+}
+
/*
* Setup the configuration for those mappings:
* dev_id=2 event=20 -> vcpu 3, intid=8195
@@ -806,6 +833,121 @@ static void test_its_migration(void)
its_send_int(dev7, 255);
check_lpi_stats();
}
+
+static void test_migrate_unmapped_collection(void)
+{
+ struct its_collection *col;
+ struct its_device *dev2, *dev7;
+ u8 config;
+
+ if (its_setup1())
+ return;
+
+ col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
+ dev2 = its_get_device(2);
+ dev7 = its_get_device(7);
+
+ /* MAPTI with the collection unmapped */
+ set_lpi(dev2, 0, 8192, col);
+
+ puts("Now migrate the VM, then press a key to continue...\n");
+ (void)getchar();
+ report(true, "Migration complete");
+
+ /* on the destination, map the collection */
+ its_send_mapc(col, true);
+
+ lpi_stats_expect(2, 8196);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+
+ config = gicv3_lpi_get_config(8192);
+ report(config == LPI_PROP_DEFAULT,
+ "Config of LPI 8192 was properly migrated");
+
+ lpi_stats_expect(nr_cpus - 1, 8192);
+ its_send_int(dev2, 0);
+ check_lpi_stats();
+
+ /* unmap the collection */
+ its_send_mapc(col, false);
+
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 0);
+ check_lpi_stats();
+
+ /* remap event 0 onto lpiid 8193 */
+ set_lpi(dev2, 0, 8193, col);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 0);
+ check_lpi_stats();
+
+ /* remap the collection */
+ its_send_mapc(col, true);
+ lpi_stats_expect(nr_cpus - 1, 8193);
+}
+
+static void test_its_pending_migration(void)
+{
+ struct its_device *dev;
+ struct its_collection *collection[2];
+ int expected[NR_CPUS];
+ u64 pendbaser;
+ void *ptr;
+ int i;
+
+ if (its_prerequisites(4))
+ return;
+
+ dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
+ its_send_mapd(dev, true);
+
+ collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
+ collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
+ its_send_mapc(collection[0], true);
+ its_send_mapc(collection[1], true);
+
+ /* disable lpi at redist level */
+ gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
+ gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
+
+ /* even lpis are assigned to even cpu */
+ for (i = 0; i < 256; i++) {
+ struct its_collection *col = i % 2 ? collection[0] :
+ collection[1];
+ int vcpu = col->target_address >> 16;
+
+ its_send_mapti(dev, 8192 + i, i, col);
+ gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
+ gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
+ }
+ its_send_invall(collection[0]);
+ its_send_invall(collection[1]);
+
+ /* Set the PTZ bit on each pendbaser */
+
+ expected[nr_cpus - 1] = 128;
+ expected[nr_cpus - 2] = 128;
+
+ ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
+ pendbaser = readq(ptr);
+ writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
+
+ ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
+ pendbaser = readq(ptr);
+ writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
+
+ gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
+ gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
+
+ puts("Now migrate the VM, then press a key to continue...\n");
+ (void)getchar();
+ report(true, "Migration complete");
+
+ mdelay(1000);
+
+ check_lpi_hits(expected);
+}
#endif
int main(int argc, char **argv)
@@ -847,6 +989,14 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
test_its_migration();
report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-pending-migration")) {
+ report_prefix_push(argv[1]);
+ test_its_pending_migration();
+ report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
+ report_prefix_push(argv[1]);
+ test_migrate_unmapped_collection();
+ report_prefix_pop();
} else if (strcmp(argv[1], "its-introspection") == 0) {
report_prefix_push(argv[1]);
test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 8b8ec79..d917157 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
groups = its migration
arch = arm64
+[its-pending-migration]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-pending-migration'
+groups = its migration
+arch = arm64
+
+[its-migrate-unmapped-collection]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
+groups = its migration
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Add two new migration tests. One testing the migration of
a topology where collection were unmapped. The second test
checks the migration of the pending table.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- tests belong to both its and migration groups
---
arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
arm/unittests.cfg | 16 +++++
2 files changed, 166 insertions(+)
diff --git a/arm/gic.c b/arm/gic.c
index fa8626a..ec3dd3a 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
smp_rmb(); /* pairs with wmb in lpi_stats_expect */
lpi_stats.observed.cpu_id = smp_processor_id();
lpi_stats.observed.lpi_id = irqnr;
+ acked[lpi_stats.observed.cpu_id]++;
smp_wmb(); /* pairs with rmb in check_lpi_stats */
}
@@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
while (1)
wfi();
}
+
+static void check_lpi_hits(int *expected)
+{
+ int i;
+
+ for (i = 0; i < nr_cpus; i++) {
+ if (acked[i] != expected[i])
+ report(false, "expected %d LPIs on PE #%d, %d observed",
+ expected[i], i, acked[i]);
+ }
+ report(true, "check LPI on all vcpus");
+}
#endif
static void gicv2_ipi_send_self(void)
@@ -594,6 +607,8 @@ static void gic_test_mmio(void)
static void test_its_introspection(void) {}
static void test_its_trigger(void) {}
static void test_its_migration(void) {}
+static void test_migrate_unmapped_collection(void) {}
+static void test_its_pending_migration(void) {}
#else /* __arch64__ */
@@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
return false;
}
+static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
+ struct its_collection *col)
+{
+ if (!dev || !col)
+ report_abort("wrong device or collection");
+
+ its_send_mapti(dev, physid, eventid, col);
+
+ gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
+ its_send_invall(col);
+}
+
/*
* Setup the configuration for those mappings:
* dev_id=2 event=20 -> vcpu 3, intid=8195
@@ -806,6 +833,121 @@ static void test_its_migration(void)
its_send_int(dev7, 255);
check_lpi_stats();
}
+
+static void test_migrate_unmapped_collection(void)
+{
+ struct its_collection *col;
+ struct its_device *dev2, *dev7;
+ u8 config;
+
+ if (its_setup1())
+ return;
+
+ col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
+ dev2 = its_get_device(2);
+ dev7 = its_get_device(7);
+
+ /* MAPTI with the collection unmapped */
+ set_lpi(dev2, 0, 8192, col);
+
+ puts("Now migrate the VM, then press a key to continue...\n");
+ (void)getchar();
+ report(true, "Migration complete");
+
+ /* on the destination, map the collection */
+ its_send_mapc(col, true);
+
+ lpi_stats_expect(2, 8196);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+
+ config = gicv3_lpi_get_config(8192);
+ report(config == LPI_PROP_DEFAULT,
+ "Config of LPI 8192 was properly migrated");
+
+ lpi_stats_expect(nr_cpus - 1, 8192);
+ its_send_int(dev2, 0);
+ check_lpi_stats();
+
+ /* unmap the collection */
+ its_send_mapc(col, false);
+
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 0);
+ check_lpi_stats();
+
+ /* remap event 0 onto lpiid 8193 */
+ set_lpi(dev2, 0, 8193, col);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 0);
+ check_lpi_stats();
+
+ /* remap the collection */
+ its_send_mapc(col, true);
+ lpi_stats_expect(nr_cpus - 1, 8193);
+}
+
+static void test_its_pending_migration(void)
+{
+ struct its_device *dev;
+ struct its_collection *collection[2];
+ int expected[NR_CPUS];
+ u64 pendbaser;
+ void *ptr;
+ int i;
+
+ if (its_prerequisites(4))
+ return;
+
+ dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
+ its_send_mapd(dev, true);
+
+ collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
+ collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
+ its_send_mapc(collection[0], true);
+ its_send_mapc(collection[1], true);
+
+ /* disable lpi at redist level */
+ gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
+ gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
+
+ /* even lpis are assigned to even cpu */
+ for (i = 0; i < 256; i++) {
+ struct its_collection *col = i % 2 ? collection[0] :
+ collection[1];
+ int vcpu = col->target_address >> 16;
+
+ its_send_mapti(dev, 8192 + i, i, col);
+ gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
+ gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
+ }
+ its_send_invall(collection[0]);
+ its_send_invall(collection[1]);
+
+ /* Set the PTZ bit on each pendbaser */
+
+ expected[nr_cpus - 1] = 128;
+ expected[nr_cpus - 2] = 128;
+
+ ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
+ pendbaser = readq(ptr);
+ writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
+
+ ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
+ pendbaser = readq(ptr);
+ writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
+
+ gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
+ gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
+
+ puts("Now migrate the VM, then press a key to continue...\n");
+ (void)getchar();
+ report(true, "Migration complete");
+
+ mdelay(1000);
+
+ check_lpi_hits(expected);
+}
#endif
int main(int argc, char **argv)
@@ -847,6 +989,14 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
test_its_migration();
report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-pending-migration")) {
+ report_prefix_push(argv[1]);
+ test_its_pending_migration();
+ report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
+ report_prefix_push(argv[1]);
+ test_migrate_unmapped_collection();
+ report_prefix_pop();
} else if (strcmp(argv[1], "its-introspection") == 0) {
report_prefix_push(argv[1]);
test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 8b8ec79..d917157 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
groups = its migration
arch = arm64
+[its-pending-migration]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-pending-migration'
+groups = its migration
+arch = arm64
+
+[its-migrate-unmapped-collection]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
+groups = its migration
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
--
2.20.1
^ permalink raw reply related [flat|nested] 137+ messages in thread
* [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
@ 2020-01-28 10:34 ` Eric Auger
0 siblings, 0 replies; 137+ messages in thread
From: Eric Auger @ 2020-01-28 10:34 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Add two new migration tests. One testing the migration of
a topology where collection were unmapped. The second test
checks the migration of the pending table.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v2 -> v3:
- tests belong to both its and migration groups
---
arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
arm/unittests.cfg | 16 +++++
2 files changed, 166 insertions(+)
diff --git a/arm/gic.c b/arm/gic.c
index fa8626a..ec3dd3a 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
smp_rmb(); /* pairs with wmb in lpi_stats_expect */
lpi_stats.observed.cpu_id = smp_processor_id();
lpi_stats.observed.lpi_id = irqnr;
+ acked[lpi_stats.observed.cpu_id]++;
smp_wmb(); /* pairs with rmb in check_lpi_stats */
}
@@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
while (1)
wfi();
}
+
+static void check_lpi_hits(int *expected)
+{
+ int i;
+
+ for (i = 0; i < nr_cpus; i++) {
+ if (acked[i] != expected[i])
+ report(false, "expected %d LPIs on PE #%d, %d observed",
+ expected[i], i, acked[i]);
+ }
+ report(true, "check LPI on all vcpus");
+}
#endif
static void gicv2_ipi_send_self(void)
@@ -594,6 +607,8 @@ static void gic_test_mmio(void)
static void test_its_introspection(void) {}
static void test_its_trigger(void) {}
static void test_its_migration(void) {}
+static void test_migrate_unmapped_collection(void) {}
+static void test_its_pending_migration(void) {}
#else /* __arch64__ */
@@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
return false;
}
+static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
+ struct its_collection *col)
+{
+ if (!dev || !col)
+ report_abort("wrong device or collection");
+
+ its_send_mapti(dev, physid, eventid, col);
+
+ gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
+ its_send_invall(col);
+}
+
/*
* Setup the configuration for those mappings:
* dev_id=2 event=20 -> vcpu 3, intid=8195
@@ -806,6 +833,121 @@ static void test_its_migration(void)
its_send_int(dev7, 255);
check_lpi_stats();
}
+
+static void test_migrate_unmapped_collection(void)
+{
+ struct its_collection *col;
+ struct its_device *dev2, *dev7;
+ u8 config;
+
+ if (its_setup1())
+ return;
+
+ col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
+ dev2 = its_get_device(2);
+ dev7 = its_get_device(7);
+
+ /* MAPTI with the collection unmapped */
+ set_lpi(dev2, 0, 8192, col);
+
+ puts("Now migrate the VM, then press a key to continue...\n");
+ (void)getchar();
+ report(true, "Migration complete");
+
+ /* on the destination, map the collection */
+ its_send_mapc(col, true);
+
+ lpi_stats_expect(2, 8196);
+ its_send_int(dev7, 255);
+ check_lpi_stats();
+
+ config = gicv3_lpi_get_config(8192);
+ report(config == LPI_PROP_DEFAULT,
+ "Config of LPI 8192 was properly migrated");
+
+ lpi_stats_expect(nr_cpus - 1, 8192);
+ its_send_int(dev2, 0);
+ check_lpi_stats();
+
+ /* unmap the collection */
+ its_send_mapc(col, false);
+
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 0);
+ check_lpi_stats();
+
+ /* remap event 0 onto lpiid 8193 */
+ set_lpi(dev2, 0, 8193, col);
+ lpi_stats_expect(-1, -1);
+ its_send_int(dev2, 0);
+ check_lpi_stats();
+
+ /* remap the collection */
+ its_send_mapc(col, true);
+ lpi_stats_expect(nr_cpus - 1, 8193);
+}
+
+static void test_its_pending_migration(void)
+{
+ struct its_device *dev;
+ struct its_collection *collection[2];
+ int expected[NR_CPUS];
+ u64 pendbaser;
+ void *ptr;
+ int i;
+
+ if (its_prerequisites(4))
+ return;
+
+ dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
+ its_send_mapd(dev, true);
+
+ collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
+ collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
+ its_send_mapc(collection[0], true);
+ its_send_mapc(collection[1], true);
+
+ /* disable lpi at redist level */
+ gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
+ gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
+
+ /* even lpis are assigned to even cpu */
+ for (i = 0; i < 256; i++) {
+ struct its_collection *col = i % 2 ? collection[0] :
+ collection[1];
+ int vcpu = col->target_address >> 16;
+
+ its_send_mapti(dev, 8192 + i, i, col);
+ gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
+ gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
+ }
+ its_send_invall(collection[0]);
+ its_send_invall(collection[1]);
+
+ /* Set the PTZ bit on each pendbaser */
+
+ expected[nr_cpus - 1] = 128;
+ expected[nr_cpus - 2] = 128;
+
+ ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
+ pendbaser = readq(ptr);
+ writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
+
+ ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
+ pendbaser = readq(ptr);
+ writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
+
+ gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
+ gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
+
+ puts("Now migrate the VM, then press a key to continue...\n");
+ (void)getchar();
+ report(true, "Migration complete");
+
+ mdelay(1000);
+
+ check_lpi_hits(expected);
+}
#endif
int main(int argc, char **argv)
@@ -847,6 +989,14 @@ int main(int argc, char **argv)
report_prefix_push(argv[1]);
test_its_migration();
report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-pending-migration")) {
+ report_prefix_push(argv[1]);
+ test_its_pending_migration();
+ report_prefix_pop();
+ } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
+ report_prefix_push(argv[1]);
+ test_migrate_unmapped_collection();
+ report_prefix_pop();
} else if (strcmp(argv[1], "its-introspection") == 0) {
report_prefix_push(argv[1]);
test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 8b8ec79..d917157 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
groups = its migration
arch = arm64
+[its-pending-migration]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-pending-migration'
+groups = its migration
+arch = arm64
+
+[its-migrate-unmapped-collection]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
+groups = its migration
+arch = arm64
+
# Test PSCI emulation
[psci]
file = psci.flat
--
2.20.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 12/14] arm/run: Allow Migration tests
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-01-29 8:07 ` Thomas Huth
-1 siblings, 0 replies; 137+ messages in thread
From: Thomas Huth @ 2020-01-29 8:07 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui, alexandru.elisei
On 28/01/2020 11.34, Eric Auger wrote:
> Let's link getchar.o to use puts and getchar from the
> tests.
>
> Then allow tests belonging to the migration group to
> trigger the migration from the test code by putting
> "migrate" into the uart. Then the code can wait for the
> migration completion by using getchar().
>
> The __getchar implement is minimalist as it just reads the
> data register. It is just meant to read the single character
> emitted at the end of the migration by the runner script.
>
> It is not meant to read more data (FIFOs are not enabled).
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - take the lock
> - assert if more than 16 chars
> - removed Thomas' R-b
> ---
> arm/Makefile.common | 2 +-
> arm/run | 2 +-
> lib/arm/io.c | 28 ++++++++++++++++++++++++++++
> 3 files changed, 30 insertions(+), 2 deletions(-)
>
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index b8988f2..a123e85 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -32,7 +32,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
> asm-offsets = lib/$(ARCH)/asm-offsets.h
> include $(SRCDIR)/scripts/asm-offsets.mak
>
> -cflatobjs += lib/util.o
> +cflatobjs += lib/util.o lib/getchar.o
> cflatobjs += lib/alloc_phys.o
> cflatobjs += lib/alloc_page.o
> cflatobjs += lib/vmalloc.o
> diff --git a/arm/run b/arm/run
> index 277db9b..a390ca5 100755
> --- a/arm/run
> +++ b/arm/run
> @@ -61,6 +61,6 @@ fi
> M+=",accel=$ACCEL"
> command="$qemu -nodefaults $M -cpu $processor $chr_testdev $pci_testdev"
> command+=" -display none -serial stdio -kernel"
> -command="$(timeout_cmd) $command"
> +command="$(migration_cmd) $(timeout_cmd) $command"
>
> run_qemu $command "$@"
> diff --git a/lib/arm/io.c b/lib/arm/io.c
> index 99fd315..d8e7745 100644
> --- a/lib/arm/io.c
> +++ b/lib/arm/io.c
> @@ -87,6 +87,34 @@ void puts(const char *s)
> spin_unlock(&uart_lock);
> }
>
> +static int ____getchar(void)
Three underscores? ... that's quite a lot already. I'd maybe rather name
the function "do_getchar" or something similar instead. Or simply merge
the code into the __getchar function below - it's just three lines.
> +{
> + int c;
> +
> + spin_lock(&uart_lock);
> + c = readb(uart0_base);
> + spin_unlock(&uart_lock);
> +
> + return c ? : -1;
Just a matter of taste, but I prefer the elvis operator without space in
between.
> +}
> +
> +/*
> + * Minimalist implementation for migration completion detection.
> + * Without FIFOs enabled on the QEMU UART device we just read
> + * the data register: we cannot read more than 16 characters.
Where are the 16 bytes buffered if FIFOs are disabled?
> + */
> +int __getchar(void)
> +{
> + int c = ____getchar();
> + static int count;
> +
> + if (c != -1)
> + ++count;
> +
> + assert(count < 16);
> +
> + return c;
> +}
The above comments were only nits ... feel free to ignore them if you
don't want to respin the series just because of this.
Thomas
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 12/14] arm/run: Allow Migration tests
@ 2020-01-29 8:07 ` Thomas Huth
0 siblings, 0 replies; 137+ messages in thread
From: Thomas Huth @ 2020-01-29 8:07 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: yuzenghui, andre.przywara, drjones, alexandru.elisei, peter.maydell
On 28/01/2020 11.34, Eric Auger wrote:
> Let's link getchar.o to use puts and getchar from the
> tests.
>
> Then allow tests belonging to the migration group to
> trigger the migration from the test code by putting
> "migrate" into the uart. Then the code can wait for the
> migration completion by using getchar().
>
> The __getchar implement is minimalist as it just reads the
> data register. It is just meant to read the single character
> emitted at the end of the migration by the runner script.
>
> It is not meant to read more data (FIFOs are not enabled).
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - take the lock
> - assert if more than 16 chars
> - removed Thomas' R-b
> ---
> arm/Makefile.common | 2 +-
> arm/run | 2 +-
> lib/arm/io.c | 28 ++++++++++++++++++++++++++++
> 3 files changed, 30 insertions(+), 2 deletions(-)
>
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index b8988f2..a123e85 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -32,7 +32,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
> asm-offsets = lib/$(ARCH)/asm-offsets.h
> include $(SRCDIR)/scripts/asm-offsets.mak
>
> -cflatobjs += lib/util.o
> +cflatobjs += lib/util.o lib/getchar.o
> cflatobjs += lib/alloc_phys.o
> cflatobjs += lib/alloc_page.o
> cflatobjs += lib/vmalloc.o
> diff --git a/arm/run b/arm/run
> index 277db9b..a390ca5 100755
> --- a/arm/run
> +++ b/arm/run
> @@ -61,6 +61,6 @@ fi
> M+=",accel=$ACCEL"
> command="$qemu -nodefaults $M -cpu $processor $chr_testdev $pci_testdev"
> command+=" -display none -serial stdio -kernel"
> -command="$(timeout_cmd) $command"
> +command="$(migration_cmd) $(timeout_cmd) $command"
>
> run_qemu $command "$@"
> diff --git a/lib/arm/io.c b/lib/arm/io.c
> index 99fd315..d8e7745 100644
> --- a/lib/arm/io.c
> +++ b/lib/arm/io.c
> @@ -87,6 +87,34 @@ void puts(const char *s)
> spin_unlock(&uart_lock);
> }
>
> +static int ____getchar(void)
Three underscores? ... that's quite a lot already. I'd maybe rather name
the function "do_getchar" or something similar instead. Or simply merge
the code into the __getchar function below - it's just three lines.
> +{
> + int c;
> +
> + spin_lock(&uart_lock);
> + c = readb(uart0_base);
> + spin_unlock(&uart_lock);
> +
> + return c ? : -1;
Just a matter of taste, but I prefer the elvis operator without space in
between.
> +}
> +
> +/*
> + * Minimalist implementation for migration completion detection.
> + * Without FIFOs enabled on the QEMU UART device we just read
> + * the data register: we cannot read more than 16 characters.
Where are the 16 bytes buffered if FIFOs are disabled?
> + */
> +int __getchar(void)
> +{
> + int c = ____getchar();
> + static int count;
> +
> + if (c != -1)
> + ++count;
> +
> + assert(count < 16);
> +
> + return c;
> +}
The above comments were only nits ... feel free to ignore them if you
don't want to respin the series just because of this.
Thomas
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 12/14] arm/run: Allow Migration tests
@ 2020-01-29 8:07 ` Thomas Huth
0 siblings, 0 replies; 137+ messages in thread
From: Thomas Huth @ 2020-01-29 8:07 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara
On 28/01/2020 11.34, Eric Auger wrote:
> Let's link getchar.o to use puts and getchar from the
> tests.
>
> Then allow tests belonging to the migration group to
> trigger the migration from the test code by putting
> "migrate" into the uart. Then the code can wait for the
> migration completion by using getchar().
>
> The __getchar implement is minimalist as it just reads the
> data register. It is just meant to read the single character
> emitted at the end of the migration by the runner script.
>
> It is not meant to read more data (FIFOs are not enabled).
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - take the lock
> - assert if more than 16 chars
> - removed Thomas' R-b
> ---
> arm/Makefile.common | 2 +-
> arm/run | 2 +-
> lib/arm/io.c | 28 ++++++++++++++++++++++++++++
> 3 files changed, 30 insertions(+), 2 deletions(-)
>
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index b8988f2..a123e85 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -32,7 +32,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
> asm-offsets = lib/$(ARCH)/asm-offsets.h
> include $(SRCDIR)/scripts/asm-offsets.mak
>
> -cflatobjs += lib/util.o
> +cflatobjs += lib/util.o lib/getchar.o
> cflatobjs += lib/alloc_phys.o
> cflatobjs += lib/alloc_page.o
> cflatobjs += lib/vmalloc.o
> diff --git a/arm/run b/arm/run
> index 277db9b..a390ca5 100755
> --- a/arm/run
> +++ b/arm/run
> @@ -61,6 +61,6 @@ fi
> M+=",accel=$ACCEL"
> command="$qemu -nodefaults $M -cpu $processor $chr_testdev $pci_testdev"
> command+=" -display none -serial stdio -kernel"
> -command="$(timeout_cmd) $command"
> +command="$(migration_cmd) $(timeout_cmd) $command"
>
> run_qemu $command "$@"
> diff --git a/lib/arm/io.c b/lib/arm/io.c
> index 99fd315..d8e7745 100644
> --- a/lib/arm/io.c
> +++ b/lib/arm/io.c
> @@ -87,6 +87,34 @@ void puts(const char *s)
> spin_unlock(&uart_lock);
> }
>
> +static int ____getchar(void)
Three underscores? ... that's quite a lot already. I'd maybe rather name
the function "do_getchar" or something similar instead. Or simply merge
the code into the __getchar function below - it's just three lines.
> +{
> + int c;
> +
> + spin_lock(&uart_lock);
> + c = readb(uart0_base);
> + spin_unlock(&uart_lock);
> +
> + return c ? : -1;
Just a matter of taste, but I prefer the elvis operator without space in
between.
> +}
> +
> +/*
> + * Minimalist implementation for migration completion detection.
> + * Without FIFOs enabled on the QEMU UART device we just read
> + * the data register: we cannot read more than 16 characters.
Where are the 16 bytes buffered if FIFOs are disabled?
> + */
> +int __getchar(void)
> +{
> + int c = ____getchar();
> + static int count;
> +
> + if (c != -1)
> + ++count;
> +
> + assert(count < 16);
> +
> + return c;
> +}
The above comments were only nits ... feel free to ignore them if you
don't want to respin the series just because of this.
Thomas
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 12/14] arm/run: Allow Migration tests
2020-01-29 8:07 ` Thomas Huth
(?)
@ 2020-01-29 9:29 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-01-29 9:29 UTC (permalink / raw)
To: Thomas Huth, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, yuzenghui, alexandru.elisei
Hi Thomas,
On 1/29/20 9:07 AM, Thomas Huth wrote:
> On 28/01/2020 11.34, Eric Auger wrote:
>> Let's link getchar.o to use puts and getchar from the
>> tests.
>>
>> Then allow tests belonging to the migration group to
>> trigger the migration from the test code by putting
>> "migrate" into the uart. Then the code can wait for the
>> migration completion by using getchar().
>>
>> The __getchar implement is minimalist as it just reads the
>> data register. It is just meant to read the single character
>> emitted at the end of the migration by the runner script.
>>
>> It is not meant to read more data (FIFOs are not enabled).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - take the lock
>> - assert if more than 16 chars
>> - removed Thomas' R-b
>> ---
>> arm/Makefile.common | 2 +-
>> arm/run | 2 +-
>> lib/arm/io.c | 28 ++++++++++++++++++++++++++++
>> 3 files changed, 30 insertions(+), 2 deletions(-)
>>
>> diff --git a/arm/Makefile.common b/arm/Makefile.common
>> index b8988f2..a123e85 100644
>> --- a/arm/Makefile.common
>> +++ b/arm/Makefile.common
>> @@ -32,7 +32,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
>> asm-offsets = lib/$(ARCH)/asm-offsets.h
>> include $(SRCDIR)/scripts/asm-offsets.mak
>>
>> -cflatobjs += lib/util.o
>> +cflatobjs += lib/util.o lib/getchar.o
>> cflatobjs += lib/alloc_phys.o
>> cflatobjs += lib/alloc_page.o
>> cflatobjs += lib/vmalloc.o
>> diff --git a/arm/run b/arm/run
>> index 277db9b..a390ca5 100755
>> --- a/arm/run
>> +++ b/arm/run
>> @@ -61,6 +61,6 @@ fi
>> M+=",accel=$ACCEL"
>> command="$qemu -nodefaults $M -cpu $processor $chr_testdev $pci_testdev"
>> command+=" -display none -serial stdio -kernel"
>> -command="$(timeout_cmd) $command"
>> +command="$(migration_cmd) $(timeout_cmd) $command"
>>
>> run_qemu $command "$@"
>> diff --git a/lib/arm/io.c b/lib/arm/io.c
>> index 99fd315..d8e7745 100644
>> --- a/lib/arm/io.c
>> +++ b/lib/arm/io.c
>> @@ -87,6 +87,34 @@ void puts(const char *s)
>> spin_unlock(&uart_lock);
>> }
>>
>> +static int ____getchar(void)
>
> Three underscores? ... that's quite a lot already. I'd maybe rather name
> the function "do_getchar" or something similar instead. Or simply merge
> the code into the __getchar function below - it's just three lines.
OK
>
>> +{
>> + int c;
>> +
>> + spin_lock(&uart_lock);
>> + c = readb(uart0_base);
>> + spin_unlock(&uart_lock);
>> +
>> + return c ? : -1;
>
> Just a matter of taste, but I prefer the elvis operator without space in
> between.
OK
>
>> +}
>> +
>> +/*
>> + * Minimalist implementation for migration completion detection.
>> + * Without FIFOs enabled on the QEMU UART device we just read
>> + * the data register: we cannot read more than 16 characters.
>
> Where are the 16 bytes buffered if FIFOs are disabled?
I think this is in the PL011 data register (UARTDR), 4 words.
https://developer.arm.com/docs/ddi0183/latest/programmers-model/register-descriptions/data-register-uartdr
>
>> + */
>> +int __getchar(void)
>> +{
>> + int c = ____getchar();
>> + static int count;
>> +
>> + if (c != -1)
>> + ++count;
>> +
>> + assert(count < 16);
>> +
>> + return c;
>> +}
>
> The above comments were only nits ... feel free to ignore them if you
> don't want to respin the series just because of this.
No Problem. Thank you for your time.
Thanks
Eric
>
> Thomas
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 12/14] arm/run: Allow Migration tests
@ 2020-01-29 9:29 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-01-29 9:29 UTC (permalink / raw)
To: Thomas Huth, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: yuzenghui, andre.przywara, drjones, alexandru.elisei, peter.maydell
Hi Thomas,
On 1/29/20 9:07 AM, Thomas Huth wrote:
> On 28/01/2020 11.34, Eric Auger wrote:
>> Let's link getchar.o to use puts and getchar from the
>> tests.
>>
>> Then allow tests belonging to the migration group to
>> trigger the migration from the test code by putting
>> "migrate" into the uart. Then the code can wait for the
>> migration completion by using getchar().
>>
>> The __getchar implement is minimalist as it just reads the
>> data register. It is just meant to read the single character
>> emitted at the end of the migration by the runner script.
>>
>> It is not meant to read more data (FIFOs are not enabled).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - take the lock
>> - assert if more than 16 chars
>> - removed Thomas' R-b
>> ---
>> arm/Makefile.common | 2 +-
>> arm/run | 2 +-
>> lib/arm/io.c | 28 ++++++++++++++++++++++++++++
>> 3 files changed, 30 insertions(+), 2 deletions(-)
>>
>> diff --git a/arm/Makefile.common b/arm/Makefile.common
>> index b8988f2..a123e85 100644
>> --- a/arm/Makefile.common
>> +++ b/arm/Makefile.common
>> @@ -32,7 +32,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
>> asm-offsets = lib/$(ARCH)/asm-offsets.h
>> include $(SRCDIR)/scripts/asm-offsets.mak
>>
>> -cflatobjs += lib/util.o
>> +cflatobjs += lib/util.o lib/getchar.o
>> cflatobjs += lib/alloc_phys.o
>> cflatobjs += lib/alloc_page.o
>> cflatobjs += lib/vmalloc.o
>> diff --git a/arm/run b/arm/run
>> index 277db9b..a390ca5 100755
>> --- a/arm/run
>> +++ b/arm/run
>> @@ -61,6 +61,6 @@ fi
>> M+=",accel=$ACCEL"
>> command="$qemu -nodefaults $M -cpu $processor $chr_testdev $pci_testdev"
>> command+=" -display none -serial stdio -kernel"
>> -command="$(timeout_cmd) $command"
>> +command="$(migration_cmd) $(timeout_cmd) $command"
>>
>> run_qemu $command "$@"
>> diff --git a/lib/arm/io.c b/lib/arm/io.c
>> index 99fd315..d8e7745 100644
>> --- a/lib/arm/io.c
>> +++ b/lib/arm/io.c
>> @@ -87,6 +87,34 @@ void puts(const char *s)
>> spin_unlock(&uart_lock);
>> }
>>
>> +static int ____getchar(void)
>
> Three underscores? ... that's quite a lot already. I'd maybe rather name
> the function "do_getchar" or something similar instead. Or simply merge
> the code into the __getchar function below - it's just three lines.
OK
>
>> +{
>> + int c;
>> +
>> + spin_lock(&uart_lock);
>> + c = readb(uart0_base);
>> + spin_unlock(&uart_lock);
>> +
>> + return c ? : -1;
>
> Just a matter of taste, but I prefer the elvis operator without space in
> between.
OK
>
>> +}
>> +
>> +/*
>> + * Minimalist implementation for migration completion detection.
>> + * Without FIFOs enabled on the QEMU UART device we just read
>> + * the data register: we cannot read more than 16 characters.
>
> Where are the 16 bytes buffered if FIFOs are disabled?
I think this is in the PL011 data register (UARTDR), 4 words.
https://developer.arm.com/docs/ddi0183/latest/programmers-model/register-descriptions/data-register-uartdr
>
>> + */
>> +int __getchar(void)
>> +{
>> + int c = ____getchar();
>> + static int count;
>> +
>> + if (c != -1)
>> + ++count;
>> +
>> + assert(count < 16);
>> +
>> + return c;
>> +}
>
> The above comments were only nits ... feel free to ignore them if you
> don't want to respin the series just because of this.
No Problem. Thank you for your time.
Thanks
Eric
>
> Thomas
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 12/14] arm/run: Allow Migration tests
@ 2020-01-29 9:29 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-01-29 9:29 UTC (permalink / raw)
To: Thomas Huth, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara
Hi Thomas,
On 1/29/20 9:07 AM, Thomas Huth wrote:
> On 28/01/2020 11.34, Eric Auger wrote:
>> Let's link getchar.o to use puts and getchar from the
>> tests.
>>
>> Then allow tests belonging to the migration group to
>> trigger the migration from the test code by putting
>> "migrate" into the uart. Then the code can wait for the
>> migration completion by using getchar().
>>
>> The __getchar implement is minimalist as it just reads the
>> data register. It is just meant to read the single character
>> emitted at the end of the migration by the runner script.
>>
>> It is not meant to read more data (FIFOs are not enabled).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - take the lock
>> - assert if more than 16 chars
>> - removed Thomas' R-b
>> ---
>> arm/Makefile.common | 2 +-
>> arm/run | 2 +-
>> lib/arm/io.c | 28 ++++++++++++++++++++++++++++
>> 3 files changed, 30 insertions(+), 2 deletions(-)
>>
>> diff --git a/arm/Makefile.common b/arm/Makefile.common
>> index b8988f2..a123e85 100644
>> --- a/arm/Makefile.common
>> +++ b/arm/Makefile.common
>> @@ -32,7 +32,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
>> asm-offsets = lib/$(ARCH)/asm-offsets.h
>> include $(SRCDIR)/scripts/asm-offsets.mak
>>
>> -cflatobjs += lib/util.o
>> +cflatobjs += lib/util.o lib/getchar.o
>> cflatobjs += lib/alloc_phys.o
>> cflatobjs += lib/alloc_page.o
>> cflatobjs += lib/vmalloc.o
>> diff --git a/arm/run b/arm/run
>> index 277db9b..a390ca5 100755
>> --- a/arm/run
>> +++ b/arm/run
>> @@ -61,6 +61,6 @@ fi
>> M+=",accel=$ACCEL"
>> command="$qemu -nodefaults $M -cpu $processor $chr_testdev $pci_testdev"
>> command+=" -display none -serial stdio -kernel"
>> -command="$(timeout_cmd) $command"
>> +command="$(migration_cmd) $(timeout_cmd) $command"
>>
>> run_qemu $command "$@"
>> diff --git a/lib/arm/io.c b/lib/arm/io.c
>> index 99fd315..d8e7745 100644
>> --- a/lib/arm/io.c
>> +++ b/lib/arm/io.c
>> @@ -87,6 +87,34 @@ void puts(const char *s)
>> spin_unlock(&uart_lock);
>> }
>>
>> +static int ____getchar(void)
>
> Three underscores? ... that's quite a lot already. I'd maybe rather name
> the function "do_getchar" or something similar instead. Or simply merge
> the code into the __getchar function below - it's just three lines.
OK
>
>> +{
>> + int c;
>> +
>> + spin_lock(&uart_lock);
>> + c = readb(uart0_base);
>> + spin_unlock(&uart_lock);
>> +
>> + return c ? : -1;
>
> Just a matter of taste, but I prefer the elvis operator without space in
> between.
OK
>
>> +}
>> +
>> +/*
>> + * Minimalist implementation for migration completion detection.
>> + * Without FIFOs enabled on the QEMU UART device we just read
>> + * the data register: we cannot read more than 16 characters.
>
> Where are the 16 bytes buffered if FIFOs are disabled?
I think this is in the PL011 data register (UARTDR), 4 words.
https://developer.arm.com/docs/ddi0183/latest/programmers-model/register-descriptions/data-register-uartdr
>
>> + */
>> +int __getchar(void)
>> +{
>> + int c = ____getchar();
>> + static int count;
>> +
>> + if (c != -1)
>> + ++count;
>> +
>> + assert(count < 16);
>> +
>> + return c;
>> +}
>
> The above comments were only nits ... feel free to ignore them if you
> don't want to respin the series just because of this.
No Problem. Thank you for your time.
Thanks
Eric
>
> Thomas
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 04/14] arm/arm64: gicv3: Add some re-distributor defines
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-06 14:35 ` Zenghui Yu
-1 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-06 14:35 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, alexandru.elisei, thuth
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> PROPBASER, PENDBASE and GICR_CTRL will be used for LPI management.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> ---
> lib/arm/asm/gic-v3.h | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index 6beeab6..ffb2e26 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -18,6 +18,7 @@
> * We expect to be run in Non-secure mode, thus we define the
> * group1 enable bits with respect to that view.
> */
> +#define GICD_CTLR 0x0000
> #define GICD_CTLR_RWP (1U << 31)
> #define GICD_CTLR_ARE_NS (1U << 4)
> #define GICD_CTLR_ENABLE_G1A (1U << 1)
> @@ -36,6 +37,11 @@
> #define GICR_ICENABLER0 GICD_ICENABLER
> #define GICR_IPRIORITYR0 GICD_IPRIORITYR
>
> +#define GICR_PROPBASER 0x0070
> +#define GICR_PENDBASER 0x0078
nit: we'd better use tab instead of space.
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
> +#define GICR_CTLR GICD_CTLR
> +#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
> +
> #define ICC_SGI1R_AFFINITY_1_SHIFT 16
> #define ICC_SGI1R_AFFINITY_2_SHIFT 32
> #define ICC_SGI1R_AFFINITY_3_SHIFT 48
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 04/14] arm/arm64: gicv3: Add some re-distributor defines
@ 2020-02-06 14:35 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-06 14:35 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, drjones, alexandru.elisei, thuth, peter.maydell
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> PROPBASER, PENDBASE and GICR_CTRL will be used for LPI management.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> ---
> lib/arm/asm/gic-v3.h | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index 6beeab6..ffb2e26 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -18,6 +18,7 @@
> * We expect to be run in Non-secure mode, thus we define the
> * group1 enable bits with respect to that view.
> */
> +#define GICD_CTLR 0x0000
> #define GICD_CTLR_RWP (1U << 31)
> #define GICD_CTLR_ARE_NS (1U << 4)
> #define GICD_CTLR_ENABLE_G1A (1U << 1)
> @@ -36,6 +37,11 @@
> #define GICR_ICENABLER0 GICD_ICENABLER
> #define GICR_IPRIORITYR0 GICD_IPRIORITYR
>
> +#define GICR_PROPBASER 0x0070
> +#define GICR_PENDBASER 0x0078
nit: we'd better use tab instead of space.
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
> +#define GICR_CTLR GICD_CTLR
> +#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
> +
> #define ICC_SGI1R_AFFINITY_1_SHIFT 16
> #define ICC_SGI1R_AFFINITY_2_SHIFT 32
> #define ICC_SGI1R_AFFINITY_3_SHIFT 48
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 04/14] arm/arm64: gicv3: Add some re-distributor defines
@ 2020-02-06 14:35 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-06 14:35 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> PROPBASER, PENDBASE and GICR_CTRL will be used for LPI management.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> ---
> lib/arm/asm/gic-v3.h | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index 6beeab6..ffb2e26 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -18,6 +18,7 @@
> * We expect to be run in Non-secure mode, thus we define the
> * group1 enable bits with respect to that view.
> */
> +#define GICD_CTLR 0x0000
> #define GICD_CTLR_RWP (1U << 31)
> #define GICD_CTLR_ARE_NS (1U << 4)
> #define GICD_CTLR_ENABLE_G1A (1U << 1)
> @@ -36,6 +37,11 @@
> #define GICR_ICENABLER0 GICD_ICENABLER
> #define GICR_IPRIORITYR0 GICD_IPRIORITYR
>
> +#define GICR_PROPBASER 0x0070
> +#define GICR_PENDBASER 0x0078
nit: we'd better use tab instead of space.
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
> +#define GICR_CTLR GICD_CTLR
> +#define GICR_CTLR_ENABLE_LPIS (1UL << 0)
> +
> #define ICC_SGI1R_AFFINITY_1_SHIFT 16
> #define ICC_SGI1R_AFFINITY_2_SHIFT 32
> #define ICC_SGI1R_AFFINITY_3_SHIFT 48
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-06 15:12 ` Zenghui Yu
-1 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-06 15:12 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, alexandru.elisei, thuth
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> Detect the presence of an ITS as part of the GICv3 init
> routine, initialize its base address and read few registers
> the IIDR, the TYPER to store its dimensioning parameters.
> Also parse the BASER registers.
>
> This is our first ITS test, belonging to a new "its" group.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - updated dates and changed author
> - squash "arm/arm64: ITS: Test BASER" into this patch but
> removes setup_baser which will be introduced later.
> - only compile on aarch64
> - restrict the new test to aarch64
>
> v1 -> v2:
> - clean GITS_TYPER macros and unused fields in typer struct
> - remove memory attribute related macros
> - remove everything related to memory attributes
> - s/dev_baser/coll_baser/ in report_info
> - add extra line
> - removed index filed in its_baser
> ---
> arm/Makefile.arm64 | 1 +
> arm/gic.c | 49 ++++++++++++++++++
> arm/unittests.cfg | 7 +++
> lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
> lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
> lib/arm/gic.c | 30 +++++++++--
> lib/arm64/asm/gic-v3-its.h | 1 +
> 7 files changed, 274 insertions(+), 5 deletions(-)
> create mode 100644 lib/arm/asm/gic-v3-its.h
> create mode 100644 lib/arm/gic-v3-its.c
> create mode 100644 lib/arm64/asm/gic-v3-its.h
>
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 6d3dc2c..2571ffb 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -19,6 +19,7 @@ endef
> cstart.o = $(TEST_DIR)/cstart64.o
> cflatobjs += lib/arm64/processor.o
> cflatobjs += lib/arm64/spinlock.o
> +cflatobjs += lib/arm/gic-v3-its.o
>
> OBJDIRS += lib/arm64
>
> diff --git a/arm/gic.c b/arm/gic.c
> index abf08c7..4d7dd03 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -16,6 +16,7 @@
> #include <asm/processor.h>
> #include <asm/delay.h>
> #include <asm/gic.h>
> +#include <asm/gic-v3-its.h>
> #include <asm/smp.h>
> #include <asm/barrier.h>
> #include <asm/io.h>
> @@ -518,6 +519,50 @@ static void gic_test_mmio(void)
> test_targets(nr_irqs);
> }
>
> +#if defined(__arm__)
> +
> +static void test_its_introspection(void) {}
> +
> +#else /* __arch64__ */
> +
> +static void test_its_introspection(void)
> +{
> + struct its_baser *dev_baser, *coll_baser;
> + struct its_typer *typer = &its_data.typer;
> +
> + if (!gicv3_its_base()) {
> + report_skip("No ITS, skip ...");
> + return;
> + }
> +
> + /* IIDR */
> + report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
> + "GITS_IIDR is read-only"),
> +
> + /* TYPER */
> + report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
> + "GITS_TYPER is read-only");
> +
> + report(typer->phys_lpi, "ITS supports physical LPIs");
> + report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
> + report_info("ITT entry size = 0x%x", typer->ite_size);
> + report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
> + typer->eventid_bits, typer->deviceid_bits,
> + typer->collid_bits);
> + report(typer->eventid_bits && typer->deviceid_bits &&
> + typer->collid_bits, "ID spaces");
> + report_info("Target address format %s",
> + typer->pta ? "Redist basse address" : "PE #");
typo: s/basse/base/
> +
> + dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
> + report(dev_baser && coll_baser, "detect device and collection BASER");
> + report_info("device baser entry_size = 0x%x", dev_baser->esz);
> + report_info("collection baser entry_size = 0x%x", coll_baser->esz);
How about "device table entry_size = ..." and "collection table
entry_size = ..."?
> +}
> +
> +#endif
> +
> int main(int argc, char **argv)
> {
> if (!gic_init()) {
> @@ -549,6 +594,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> gic_test_mmio();
> report_prefix_pop();
> + } else if (strcmp(argv[1], "its-introspection") == 0) {
> + report_prefix_push(argv[1]);
> + test_its_introspection();
> + report_prefix_pop();
> } else {
> report_abort("Unknown subtest '%s'", argv[1]);
> }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index daeb5a0..ba2b31b 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -122,6 +122,13 @@ smp = $MAX_SMP
> extra_params = -machine gic-version=3 -append 'active'
> groups = gic
>
> +[its-introspection]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'its-introspection'
> +groups = its
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..815c515
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -0,0 +1,103 @@
> +/*
> + * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_ITS_H_
> +#define _ASMARM_GIC_V3_ITS_H_
> +
> +#ifndef __ASSEMBLY__
> +
> +struct its_typer {
> + unsigned int ite_size;
> + unsigned int eventid_bits;
> + unsigned int deviceid_bits;
> + unsigned int collid_bits;
> + bool pta;
> + bool phys_lpi;
> + bool virt_lpi;
> +};
> +
> +struct its_baser {
> + int type;
> + size_t psz;
> + int nr_pages;
> + bool indirect;
> + phys_addr_t table_addr;
> + bool valid;
> + int esz;
> +};
> +
> +#define GITS_BASER_NR_REGS 8
> +
> +struct its_data {
> + void *base;
> + struct its_typer typer;
> + struct its_baser baser[GITS_BASER_NR_REGS];
> +};
> +
> +extern struct its_data its_data;
> +
> +#define gicv3_its_base() (its_data.base)
> +
> +#if defined(__aarch64__)
> +
> +#define GITS_CTLR 0x0000
> +#define GITS_IIDR 0x0004
> +#define GITS_TYPER 0x0008
> +#define GITS_CBASER 0x0080
> +#define GITS_CWRITER 0x0088
> +#define GITS_CREADR 0x0090
> +#define GITS_BASER 0x0100
> +
> +#define GITS_TYPER_PLPIS BIT(0)
> +#define GITS_TYPER_VLPIS BIT(1)
> +#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
> +#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
> +#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
> +#define GITS_TYPER_IDBITS_SHIFT 8
> +#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
> +#define GITS_TYPER_DEVBITS_SHIFT 13
> +#define GITS_TYPER_PTA BIT(19)
> +#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
> +#define GITS_TYPER_CIDBITS_SHIFT 32
> +#define GITS_TYPER_CIL BIT(36)
> +
> +#define GITS_CTLR_ENABLE (1U << 0)
> +
> +#define GITS_CBASER_VALID (1UL << 63)
> +
> +#define GITS_BASER_VALID BIT(63)
> +#define GITS_BASER_INDIRECT BIT(62)
> +#define GITS_BASER_TYPE_SHIFT (56)
> +#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
> +#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
> +#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
> +#define GITS_BASER_PAGE_SIZE_SHIFT (8)
> +#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGES_MAX 256
> +#define GITS_BASER_PAGES_SHIFT (0)
> +#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
> +#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
> +#define GITS_BASER_TYPE_NONE 0
> +#define GITS_BASER_TYPE_DEVICE 1
> +#define GITS_BASER_TYPE_COLLECTION 4
> +
> +extern void its_parse_typer(void);
> +extern void its_init(void);
> +extern int its_parse_baser(int i, struct its_baser *baser);
> +extern struct its_baser *its_lookup_baser(int type);
> +
> +#else /* __arm__ */
> +
> +static inline void its_init(void) {}
> +
> +#endif
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_ITS_H_ */
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> new file mode 100644
> index 0000000..2c0ce13
> --- /dev/null
> +++ b/lib/arm/gic-v3-its.c
> @@ -0,0 +1,88 @@
> +/*
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <asm/gic.h>
> +#include <alloc_page.h>
> +#include <asm/gic-v3-its.h>
> +
> +void its_parse_typer(void)
> +{
> + u64 typer = readq(gicv3_its_base() + GITS_TYPER);
> +
> + its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
> + GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
> + its_data.typer.pta = typer & GITS_TYPER_PTA;
> + its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
> + GITS_TYPER_IDBITS_SHIFT) + 1;
> + its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
> + GITS_TYPER_DEVBITS_SHIFT) + 1;
> +
> + if (typer & GITS_TYPER_CIL)
> + its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
> + GITS_TYPER_CIDBITS_SHIFT) + 1;
> + else
> + its_data.typer.collid_bits = 16;
> +
> + its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
> + its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
> +}
> +
> +int its_parse_baser(int i, struct its_baser *baser)
> +{
> + void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
> + u64 val = readq(reg_addr);
> +
> + if (!val) {
> + memset(baser, 0, sizeof(*baser));
> + return -1;
> + }
Unimplemented BASERs? How about using something like:
if (GITS_BASER_TYPE(val) == GITS_BASER_TYPE_NONE) {
[...]
}
to make it a bit more explicit?
But feel free to just ignore it because you're right that
the unimplemented BASERs are RES0.
Thanks,
Zenghui
> +
> + baser->valid = val & GITS_BASER_VALID;
> + baser->indirect = val & GITS_BASER_INDIRECT;
> + baser->type = GITS_BASER_TYPE(val);
> + baser->esz = GITS_BASER_ENTRY_SIZE(val);
> + baser->nr_pages = GITS_BASER_NR_PAGES(val);
> + baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
> + switch (val & GITS_BASER_PAGE_SIZE_MASK) {
> + case GITS_BASER_PAGE_SIZE_4K:
> + baser->psz = SZ_4K;
> + break;
> + case GITS_BASER_PAGE_SIZE_16K:
> + baser->psz = SZ_16K;
> + break;
> + case GITS_BASER_PAGE_SIZE_64K:
> + baser->psz = SZ_64K;
> + break;
> + default:
> + baser->psz = SZ_64K;
> + }
> + return 0;
> +}
> +
> +struct its_baser *its_lookup_baser(int type)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
> + struct its_baser *baser = &its_data.baser[i];
> +
> + if (baser->type == type)
> + return baser;
> + }
> + return NULL;
> +}
> +
> +void its_init(void)
> +{
> + int i;
> +
> + if (!its_data.base)
> + return;
> +
> + its_parse_typer();
> + for (i = 0; i < GITS_BASER_NR_REGS; i++)
> + its_parse_baser(i, &its_data.baser[i]);
> +}
> +
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index aa9cb86..6b70b05 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -6,9 +6,11 @@
> #include <devicetree.h>
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <asm/gic-v3-its.h>
>
> struct gicv2_data gicv2_data;
> struct gicv3_data gicv3_data;
> +struct its_data its_data;
>
> struct gic_common_ops {
> void (*enable_defaults)(void);
> @@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
> * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> */
> static bool
> -gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
> {
> struct dt_pbus_reg reg;
> - struct dt_device gic;
> + struct dt_device gic, its;
> struct dt_bus bus;
> - int node, ret, i;
> + int node, subnode, ret, i, len;
> + const void *fdt = dt_fdt();
>
> dt_bus_init_defaults(&bus);
> dt_device_init(&gic, &bus, NULL);
> @@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> base2[i] = ioremap(reg.addr, reg.size);
> }
>
> + if (base3 && !strcmp(compatible, "arm,gic-v3")) {
> + dt_for_each_subnode(node, subnode) {
> + const struct fdt_property *prop;
> +
> + prop = fdt_get_property(fdt, subnode, "compatible", &len);
> + if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
> + dt_device_bind_node(&its, subnode);
> + ret = dt_pbus_translate(&its, 0, ®);
> + assert(ret == 0);
> + *base3 = ioremap(reg.addr, reg.size);
> + break;
> + }
> + }
> +
> + }
> +
> return true;
> }
>
> int gicv2_init(void)
> {
> return gic_get_dt_bases("arm,cortex-a15-gic",
> - &gicv2_data.dist_base, &gicv2_data.cpu_base);
> + &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
> }
>
> int gicv3_init(void)
> {
> return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> - &gicv3_data.redist_bases[0]);
> + &gicv3_data.redist_bases[0], &its_data.base);
> }
>
> int gic_version(void)
> @@ -104,6 +123,7 @@ int gic_init(void)
> gic_common_ops = &gicv2_common_ops;
> else if (gicv3_init())
> gic_common_ops = &gicv3_common_ops;
> + its_init();
> return gic_version();
> }
>
> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..083cba4
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3-its.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3-its.h"
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
@ 2020-02-06 15:12 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-06 15:12 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, drjones, alexandru.elisei, thuth, peter.maydell
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> Detect the presence of an ITS as part of the GICv3 init
> routine, initialize its base address and read few registers
> the IIDR, the TYPER to store its dimensioning parameters.
> Also parse the BASER registers.
>
> This is our first ITS test, belonging to a new "its" group.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - updated dates and changed author
> - squash "arm/arm64: ITS: Test BASER" into this patch but
> removes setup_baser which will be introduced later.
> - only compile on aarch64
> - restrict the new test to aarch64
>
> v1 -> v2:
> - clean GITS_TYPER macros and unused fields in typer struct
> - remove memory attribute related macros
> - remove everything related to memory attributes
> - s/dev_baser/coll_baser/ in report_info
> - add extra line
> - removed index filed in its_baser
> ---
> arm/Makefile.arm64 | 1 +
> arm/gic.c | 49 ++++++++++++++++++
> arm/unittests.cfg | 7 +++
> lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
> lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
> lib/arm/gic.c | 30 +++++++++--
> lib/arm64/asm/gic-v3-its.h | 1 +
> 7 files changed, 274 insertions(+), 5 deletions(-)
> create mode 100644 lib/arm/asm/gic-v3-its.h
> create mode 100644 lib/arm/gic-v3-its.c
> create mode 100644 lib/arm64/asm/gic-v3-its.h
>
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 6d3dc2c..2571ffb 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -19,6 +19,7 @@ endef
> cstart.o = $(TEST_DIR)/cstart64.o
> cflatobjs += lib/arm64/processor.o
> cflatobjs += lib/arm64/spinlock.o
> +cflatobjs += lib/arm/gic-v3-its.o
>
> OBJDIRS += lib/arm64
>
> diff --git a/arm/gic.c b/arm/gic.c
> index abf08c7..4d7dd03 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -16,6 +16,7 @@
> #include <asm/processor.h>
> #include <asm/delay.h>
> #include <asm/gic.h>
> +#include <asm/gic-v3-its.h>
> #include <asm/smp.h>
> #include <asm/barrier.h>
> #include <asm/io.h>
> @@ -518,6 +519,50 @@ static void gic_test_mmio(void)
> test_targets(nr_irqs);
> }
>
> +#if defined(__arm__)
> +
> +static void test_its_introspection(void) {}
> +
> +#else /* __arch64__ */
> +
> +static void test_its_introspection(void)
> +{
> + struct its_baser *dev_baser, *coll_baser;
> + struct its_typer *typer = &its_data.typer;
> +
> + if (!gicv3_its_base()) {
> + report_skip("No ITS, skip ...");
> + return;
> + }
> +
> + /* IIDR */
> + report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
> + "GITS_IIDR is read-only"),
> +
> + /* TYPER */
> + report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
> + "GITS_TYPER is read-only");
> +
> + report(typer->phys_lpi, "ITS supports physical LPIs");
> + report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
> + report_info("ITT entry size = 0x%x", typer->ite_size);
> + report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
> + typer->eventid_bits, typer->deviceid_bits,
> + typer->collid_bits);
> + report(typer->eventid_bits && typer->deviceid_bits &&
> + typer->collid_bits, "ID spaces");
> + report_info("Target address format %s",
> + typer->pta ? "Redist basse address" : "PE #");
typo: s/basse/base/
> +
> + dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
> + report(dev_baser && coll_baser, "detect device and collection BASER");
> + report_info("device baser entry_size = 0x%x", dev_baser->esz);
> + report_info("collection baser entry_size = 0x%x", coll_baser->esz);
How about "device table entry_size = ..." and "collection table
entry_size = ..."?
> +}
> +
> +#endif
> +
> int main(int argc, char **argv)
> {
> if (!gic_init()) {
> @@ -549,6 +594,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> gic_test_mmio();
> report_prefix_pop();
> + } else if (strcmp(argv[1], "its-introspection") == 0) {
> + report_prefix_push(argv[1]);
> + test_its_introspection();
> + report_prefix_pop();
> } else {
> report_abort("Unknown subtest '%s'", argv[1]);
> }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index daeb5a0..ba2b31b 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -122,6 +122,13 @@ smp = $MAX_SMP
> extra_params = -machine gic-version=3 -append 'active'
> groups = gic
>
> +[its-introspection]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'its-introspection'
> +groups = its
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..815c515
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -0,0 +1,103 @@
> +/*
> + * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_ITS_H_
> +#define _ASMARM_GIC_V3_ITS_H_
> +
> +#ifndef __ASSEMBLY__
> +
> +struct its_typer {
> + unsigned int ite_size;
> + unsigned int eventid_bits;
> + unsigned int deviceid_bits;
> + unsigned int collid_bits;
> + bool pta;
> + bool phys_lpi;
> + bool virt_lpi;
> +};
> +
> +struct its_baser {
> + int type;
> + size_t psz;
> + int nr_pages;
> + bool indirect;
> + phys_addr_t table_addr;
> + bool valid;
> + int esz;
> +};
> +
> +#define GITS_BASER_NR_REGS 8
> +
> +struct its_data {
> + void *base;
> + struct its_typer typer;
> + struct its_baser baser[GITS_BASER_NR_REGS];
> +};
> +
> +extern struct its_data its_data;
> +
> +#define gicv3_its_base() (its_data.base)
> +
> +#if defined(__aarch64__)
> +
> +#define GITS_CTLR 0x0000
> +#define GITS_IIDR 0x0004
> +#define GITS_TYPER 0x0008
> +#define GITS_CBASER 0x0080
> +#define GITS_CWRITER 0x0088
> +#define GITS_CREADR 0x0090
> +#define GITS_BASER 0x0100
> +
> +#define GITS_TYPER_PLPIS BIT(0)
> +#define GITS_TYPER_VLPIS BIT(1)
> +#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
> +#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
> +#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
> +#define GITS_TYPER_IDBITS_SHIFT 8
> +#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
> +#define GITS_TYPER_DEVBITS_SHIFT 13
> +#define GITS_TYPER_PTA BIT(19)
> +#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
> +#define GITS_TYPER_CIDBITS_SHIFT 32
> +#define GITS_TYPER_CIL BIT(36)
> +
> +#define GITS_CTLR_ENABLE (1U << 0)
> +
> +#define GITS_CBASER_VALID (1UL << 63)
> +
> +#define GITS_BASER_VALID BIT(63)
> +#define GITS_BASER_INDIRECT BIT(62)
> +#define GITS_BASER_TYPE_SHIFT (56)
> +#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
> +#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
> +#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
> +#define GITS_BASER_PAGE_SIZE_SHIFT (8)
> +#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGES_MAX 256
> +#define GITS_BASER_PAGES_SHIFT (0)
> +#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
> +#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
> +#define GITS_BASER_TYPE_NONE 0
> +#define GITS_BASER_TYPE_DEVICE 1
> +#define GITS_BASER_TYPE_COLLECTION 4
> +
> +extern void its_parse_typer(void);
> +extern void its_init(void);
> +extern int its_parse_baser(int i, struct its_baser *baser);
> +extern struct its_baser *its_lookup_baser(int type);
> +
> +#else /* __arm__ */
> +
> +static inline void its_init(void) {}
> +
> +#endif
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_ITS_H_ */
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> new file mode 100644
> index 0000000..2c0ce13
> --- /dev/null
> +++ b/lib/arm/gic-v3-its.c
> @@ -0,0 +1,88 @@
> +/*
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <asm/gic.h>
> +#include <alloc_page.h>
> +#include <asm/gic-v3-its.h>
> +
> +void its_parse_typer(void)
> +{
> + u64 typer = readq(gicv3_its_base() + GITS_TYPER);
> +
> + its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
> + GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
> + its_data.typer.pta = typer & GITS_TYPER_PTA;
> + its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
> + GITS_TYPER_IDBITS_SHIFT) + 1;
> + its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
> + GITS_TYPER_DEVBITS_SHIFT) + 1;
> +
> + if (typer & GITS_TYPER_CIL)
> + its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
> + GITS_TYPER_CIDBITS_SHIFT) + 1;
> + else
> + its_data.typer.collid_bits = 16;
> +
> + its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
> + its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
> +}
> +
> +int its_parse_baser(int i, struct its_baser *baser)
> +{
> + void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
> + u64 val = readq(reg_addr);
> +
> + if (!val) {
> + memset(baser, 0, sizeof(*baser));
> + return -1;
> + }
Unimplemented BASERs? How about using something like:
if (GITS_BASER_TYPE(val) == GITS_BASER_TYPE_NONE) {
[...]
}
to make it a bit more explicit?
But feel free to just ignore it because you're right that
the unimplemented BASERs are RES0.
Thanks,
Zenghui
> +
> + baser->valid = val & GITS_BASER_VALID;
> + baser->indirect = val & GITS_BASER_INDIRECT;
> + baser->type = GITS_BASER_TYPE(val);
> + baser->esz = GITS_BASER_ENTRY_SIZE(val);
> + baser->nr_pages = GITS_BASER_NR_PAGES(val);
> + baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
> + switch (val & GITS_BASER_PAGE_SIZE_MASK) {
> + case GITS_BASER_PAGE_SIZE_4K:
> + baser->psz = SZ_4K;
> + break;
> + case GITS_BASER_PAGE_SIZE_16K:
> + baser->psz = SZ_16K;
> + break;
> + case GITS_BASER_PAGE_SIZE_64K:
> + baser->psz = SZ_64K;
> + break;
> + default:
> + baser->psz = SZ_64K;
> + }
> + return 0;
> +}
> +
> +struct its_baser *its_lookup_baser(int type)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
> + struct its_baser *baser = &its_data.baser[i];
> +
> + if (baser->type == type)
> + return baser;
> + }
> + return NULL;
> +}
> +
> +void its_init(void)
> +{
> + int i;
> +
> + if (!its_data.base)
> + return;
> +
> + its_parse_typer();
> + for (i = 0; i < GITS_BASER_NR_REGS; i++)
> + its_parse_baser(i, &its_data.baser[i]);
> +}
> +
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index aa9cb86..6b70b05 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -6,9 +6,11 @@
> #include <devicetree.h>
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <asm/gic-v3-its.h>
>
> struct gicv2_data gicv2_data;
> struct gicv3_data gicv3_data;
> +struct its_data its_data;
>
> struct gic_common_ops {
> void (*enable_defaults)(void);
> @@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
> * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> */
> static bool
> -gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
> {
> struct dt_pbus_reg reg;
> - struct dt_device gic;
> + struct dt_device gic, its;
> struct dt_bus bus;
> - int node, ret, i;
> + int node, subnode, ret, i, len;
> + const void *fdt = dt_fdt();
>
> dt_bus_init_defaults(&bus);
> dt_device_init(&gic, &bus, NULL);
> @@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> base2[i] = ioremap(reg.addr, reg.size);
> }
>
> + if (base3 && !strcmp(compatible, "arm,gic-v3")) {
> + dt_for_each_subnode(node, subnode) {
> + const struct fdt_property *prop;
> +
> + prop = fdt_get_property(fdt, subnode, "compatible", &len);
> + if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
> + dt_device_bind_node(&its, subnode);
> + ret = dt_pbus_translate(&its, 0, ®);
> + assert(ret == 0);
> + *base3 = ioremap(reg.addr, reg.size);
> + break;
> + }
> + }
> +
> + }
> +
> return true;
> }
>
> int gicv2_init(void)
> {
> return gic_get_dt_bases("arm,cortex-a15-gic",
> - &gicv2_data.dist_base, &gicv2_data.cpu_base);
> + &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
> }
>
> int gicv3_init(void)
> {
> return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> - &gicv3_data.redist_bases[0]);
> + &gicv3_data.redist_bases[0], &its_data.base);
> }
>
> int gic_version(void)
> @@ -104,6 +123,7 @@ int gic_init(void)
> gic_common_ops = &gicv2_common_ops;
> else if (gicv3_init())
> gic_common_ops = &gicv3_common_ops;
> + its_init();
> return gic_version();
> }
>
> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..083cba4
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3-its.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3-its.h"
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
@ 2020-02-06 15:12 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-06 15:12 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> Detect the presence of an ITS as part of the GICv3 init
> routine, initialize its base address and read few registers
> the IIDR, the TYPER to store its dimensioning parameters.
> Also parse the BASER registers.
>
> This is our first ITS test, belonging to a new "its" group.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - updated dates and changed author
> - squash "arm/arm64: ITS: Test BASER" into this patch but
> removes setup_baser which will be introduced later.
> - only compile on aarch64
> - restrict the new test to aarch64
>
> v1 -> v2:
> - clean GITS_TYPER macros and unused fields in typer struct
> - remove memory attribute related macros
> - remove everything related to memory attributes
> - s/dev_baser/coll_baser/ in report_info
> - add extra line
> - removed index filed in its_baser
> ---
> arm/Makefile.arm64 | 1 +
> arm/gic.c | 49 ++++++++++++++++++
> arm/unittests.cfg | 7 +++
> lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
> lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
> lib/arm/gic.c | 30 +++++++++--
> lib/arm64/asm/gic-v3-its.h | 1 +
> 7 files changed, 274 insertions(+), 5 deletions(-)
> create mode 100644 lib/arm/asm/gic-v3-its.h
> create mode 100644 lib/arm/gic-v3-its.c
> create mode 100644 lib/arm64/asm/gic-v3-its.h
>
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 6d3dc2c..2571ffb 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -19,6 +19,7 @@ endef
> cstart.o = $(TEST_DIR)/cstart64.o
> cflatobjs += lib/arm64/processor.o
> cflatobjs += lib/arm64/spinlock.o
> +cflatobjs += lib/arm/gic-v3-its.o
>
> OBJDIRS += lib/arm64
>
> diff --git a/arm/gic.c b/arm/gic.c
> index abf08c7..4d7dd03 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -16,6 +16,7 @@
> #include <asm/processor.h>
> #include <asm/delay.h>
> #include <asm/gic.h>
> +#include <asm/gic-v3-its.h>
> #include <asm/smp.h>
> #include <asm/barrier.h>
> #include <asm/io.h>
> @@ -518,6 +519,50 @@ static void gic_test_mmio(void)
> test_targets(nr_irqs);
> }
>
> +#if defined(__arm__)
> +
> +static void test_its_introspection(void) {}
> +
> +#else /* __arch64__ */
> +
> +static void test_its_introspection(void)
> +{
> + struct its_baser *dev_baser, *coll_baser;
> + struct its_typer *typer = &its_data.typer;
> +
> + if (!gicv3_its_base()) {
> + report_skip("No ITS, skip ...");
> + return;
> + }
> +
> + /* IIDR */
> + report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
> + "GITS_IIDR is read-only"),
> +
> + /* TYPER */
> + report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
> + "GITS_TYPER is read-only");
> +
> + report(typer->phys_lpi, "ITS supports physical LPIs");
> + report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
> + report_info("ITT entry size = 0x%x", typer->ite_size);
> + report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
> + typer->eventid_bits, typer->deviceid_bits,
> + typer->collid_bits);
> + report(typer->eventid_bits && typer->deviceid_bits &&
> + typer->collid_bits, "ID spaces");
> + report_info("Target address format %s",
> + typer->pta ? "Redist basse address" : "PE #");
typo: s/basse/base/
> +
> + dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
> + report(dev_baser && coll_baser, "detect device and collection BASER");
> + report_info("device baser entry_size = 0x%x", dev_baser->esz);
> + report_info("collection baser entry_size = 0x%x", coll_baser->esz);
How about "device table entry_size = ..." and "collection table
entry_size = ..."?
> +}
> +
> +#endif
> +
> int main(int argc, char **argv)
> {
> if (!gic_init()) {
> @@ -549,6 +594,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> gic_test_mmio();
> report_prefix_pop();
> + } else if (strcmp(argv[1], "its-introspection") == 0) {
> + report_prefix_push(argv[1]);
> + test_its_introspection();
> + report_prefix_pop();
> } else {
> report_abort("Unknown subtest '%s'", argv[1]);
> }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index daeb5a0..ba2b31b 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -122,6 +122,13 @@ smp = $MAX_SMP
> extra_params = -machine gic-version=3 -append 'active'
> groups = gic
>
> +[its-introspection]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'its-introspection'
> +groups = its
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..815c515
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -0,0 +1,103 @@
> +/*
> + * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_ITS_H_
> +#define _ASMARM_GIC_V3_ITS_H_
> +
> +#ifndef __ASSEMBLY__
> +
> +struct its_typer {
> + unsigned int ite_size;
> + unsigned int eventid_bits;
> + unsigned int deviceid_bits;
> + unsigned int collid_bits;
> + bool pta;
> + bool phys_lpi;
> + bool virt_lpi;
> +};
> +
> +struct its_baser {
> + int type;
> + size_t psz;
> + int nr_pages;
> + bool indirect;
> + phys_addr_t table_addr;
> + bool valid;
> + int esz;
> +};
> +
> +#define GITS_BASER_NR_REGS 8
> +
> +struct its_data {
> + void *base;
> + struct its_typer typer;
> + struct its_baser baser[GITS_BASER_NR_REGS];
> +};
> +
> +extern struct its_data its_data;
> +
> +#define gicv3_its_base() (its_data.base)
> +
> +#if defined(__aarch64__)
> +
> +#define GITS_CTLR 0x0000
> +#define GITS_IIDR 0x0004
> +#define GITS_TYPER 0x0008
> +#define GITS_CBASER 0x0080
> +#define GITS_CWRITER 0x0088
> +#define GITS_CREADR 0x0090
> +#define GITS_BASER 0x0100
> +
> +#define GITS_TYPER_PLPIS BIT(0)
> +#define GITS_TYPER_VLPIS BIT(1)
> +#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
> +#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
> +#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
> +#define GITS_TYPER_IDBITS_SHIFT 8
> +#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
> +#define GITS_TYPER_DEVBITS_SHIFT 13
> +#define GITS_TYPER_PTA BIT(19)
> +#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
> +#define GITS_TYPER_CIDBITS_SHIFT 32
> +#define GITS_TYPER_CIL BIT(36)
> +
> +#define GITS_CTLR_ENABLE (1U << 0)
> +
> +#define GITS_CBASER_VALID (1UL << 63)
> +
> +#define GITS_BASER_VALID BIT(63)
> +#define GITS_BASER_INDIRECT BIT(62)
> +#define GITS_BASER_TYPE_SHIFT (56)
> +#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
> +#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
> +#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
> +#define GITS_BASER_PAGE_SIZE_SHIFT (8)
> +#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGES_MAX 256
> +#define GITS_BASER_PAGES_SHIFT (0)
> +#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
> +#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
> +#define GITS_BASER_TYPE_NONE 0
> +#define GITS_BASER_TYPE_DEVICE 1
> +#define GITS_BASER_TYPE_COLLECTION 4
> +
> +extern void its_parse_typer(void);
> +extern void its_init(void);
> +extern int its_parse_baser(int i, struct its_baser *baser);
> +extern struct its_baser *its_lookup_baser(int type);
> +
> +#else /* __arm__ */
> +
> +static inline void its_init(void) {}
> +
> +#endif
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_ITS_H_ */
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> new file mode 100644
> index 0000000..2c0ce13
> --- /dev/null
> +++ b/lib/arm/gic-v3-its.c
> @@ -0,0 +1,88 @@
> +/*
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <asm/gic.h>
> +#include <alloc_page.h>
> +#include <asm/gic-v3-its.h>
> +
> +void its_parse_typer(void)
> +{
> + u64 typer = readq(gicv3_its_base() + GITS_TYPER);
> +
> + its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
> + GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
> + its_data.typer.pta = typer & GITS_TYPER_PTA;
> + its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
> + GITS_TYPER_IDBITS_SHIFT) + 1;
> + its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
> + GITS_TYPER_DEVBITS_SHIFT) + 1;
> +
> + if (typer & GITS_TYPER_CIL)
> + its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
> + GITS_TYPER_CIDBITS_SHIFT) + 1;
> + else
> + its_data.typer.collid_bits = 16;
> +
> + its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
> + its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
> +}
> +
> +int its_parse_baser(int i, struct its_baser *baser)
> +{
> + void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
> + u64 val = readq(reg_addr);
> +
> + if (!val) {
> + memset(baser, 0, sizeof(*baser));
> + return -1;
> + }
Unimplemented BASERs? How about using something like:
if (GITS_BASER_TYPE(val) == GITS_BASER_TYPE_NONE) {
[...]
}
to make it a bit more explicit?
But feel free to just ignore it because you're right that
the unimplemented BASERs are RES0.
Thanks,
Zenghui
> +
> + baser->valid = val & GITS_BASER_VALID;
> + baser->indirect = val & GITS_BASER_INDIRECT;
> + baser->type = GITS_BASER_TYPE(val);
> + baser->esz = GITS_BASER_ENTRY_SIZE(val);
> + baser->nr_pages = GITS_BASER_NR_PAGES(val);
> + baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
> + switch (val & GITS_BASER_PAGE_SIZE_MASK) {
> + case GITS_BASER_PAGE_SIZE_4K:
> + baser->psz = SZ_4K;
> + break;
> + case GITS_BASER_PAGE_SIZE_16K:
> + baser->psz = SZ_16K;
> + break;
> + case GITS_BASER_PAGE_SIZE_64K:
> + baser->psz = SZ_64K;
> + break;
> + default:
> + baser->psz = SZ_64K;
> + }
> + return 0;
> +}
> +
> +struct its_baser *its_lookup_baser(int type)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
> + struct its_baser *baser = &its_data.baser[i];
> +
> + if (baser->type == type)
> + return baser;
> + }
> + return NULL;
> +}
> +
> +void its_init(void)
> +{
> + int i;
> +
> + if (!its_data.base)
> + return;
> +
> + its_parse_typer();
> + for (i = 0; i < GITS_BASER_NR_REGS; i++)
> + its_parse_baser(i, &its_data.baser[i]);
> +}
> +
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index aa9cb86..6b70b05 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -6,9 +6,11 @@
> #include <devicetree.h>
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <asm/gic-v3-its.h>
>
> struct gicv2_data gicv2_data;
> struct gicv3_data gicv3_data;
> +struct its_data its_data;
>
> struct gic_common_ops {
> void (*enable_defaults)(void);
> @@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
> * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> */
> static bool
> -gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
> {
> struct dt_pbus_reg reg;
> - struct dt_device gic;
> + struct dt_device gic, its;
> struct dt_bus bus;
> - int node, ret, i;
> + int node, subnode, ret, i, len;
> + const void *fdt = dt_fdt();
>
> dt_bus_init_defaults(&bus);
> dt_device_init(&gic, &bus, NULL);
> @@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> base2[i] = ioremap(reg.addr, reg.size);
> }
>
> + if (base3 && !strcmp(compatible, "arm,gic-v3")) {
> + dt_for_each_subnode(node, subnode) {
> + const struct fdt_property *prop;
> +
> + prop = fdt_get_property(fdt, subnode, "compatible", &len);
> + if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
> + dt_device_bind_node(&its, subnode);
> + ret = dt_pbus_translate(&its, 0, ®);
> + assert(ret == 0);
> + *base3 = ioremap(reg.addr, reg.size);
> + break;
> + }
> + }
> +
> + }
> +
> return true;
> }
>
> int gicv2_init(void)
> {
> return gic_get_dt_bases("arm,cortex-a15-gic",
> - &gicv2_data.dist_base, &gicv2_data.cpu_base);
> + &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
> }
>
> int gicv3_init(void)
> {
> return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> - &gicv3_data.redist_bases[0]);
> + &gicv3_data.redist_bases[0], &its_data.base);
> }
>
> int gic_version(void)
> @@ -104,6 +123,7 @@ int gic_init(void)
> gic_common_ops = &gicv2_common_ops;
> else if (gicv3_init())
> gic_common_ops = &gicv3_common_ops;
> + its_init();
> return gic_version();
> }
>
> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..083cba4
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3-its.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3-its.h"
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 2:12 ` Zenghui Yu
-1 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-07 2:12 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, alexandru.elisei, thuth
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> Allocate the LPI configuration and per re-distributor pending table.
> Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
> by default in the config table.
>
> Also introduce a helper routine that allows to set the pending table
> bit for a given LPI.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
> and add _lpi prefix too
>
> v1 -> v2:
> - remove memory attributes
> ---
> lib/arm/asm/gic-v3.h | 16 +++++++++++
> lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 80 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index ffb2e26..ec2a6f0 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -48,6 +48,16 @@
> #define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
>
> +#define GICR_PROPBASER_IDBITS_MASK (0x1f)
This is not being used. You can use it when calculating prop_val
or just drop it.
> +
> +#define GICR_PENDBASER_PTZ BIT_ULL(62)
> +
> +#define LPI_PROP_GROUP1 (1 << 1)
> +#define LPI_PROP_ENABLED (1 << 0)
> +#define LPI_PROP_DEFAULT_PRIO 0xa0
> +#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
> + LPI_PROP_ENABLED)
> +
> #include <asm/arch_gicv3.h>
>
> #ifndef __ASSEMBLY__
> @@ -64,6 +74,8 @@ struct gicv3_data {
> void *dist_base;
> void *redist_bases[GICV3_NR_REDISTS];
> void *redist_base[NR_CPUS];
> + void *lpi_prop;
> + void *lpi_pend[NR_CPUS];
> unsigned int irq_nr;
> };
> extern struct gicv3_data gicv3_data;
> @@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
> extern void gicv3_ipi_send_single(int irq, int cpu);
> extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
> extern void gicv3_set_redist_base(size_t stride);
> +extern void gicv3_lpi_set_config(int n, u8 val);
> +extern u8 gicv3_lpi_get_config(int n);
> +extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> +extern void gicv3_lpi_alloc_tables(void);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> index feecb5e..c33f883 100644
> --- a/lib/arm/gic-v3.c
> +++ b/lib/arm/gic-v3.c
> @@ -5,6 +5,7 @@
> */
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <alloc_page.h>
>
> void gicv3_set_redist_base(size_t stride)
> {
> @@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
> cpumask_set_cpu(cpu, &dest);
> gicv3_ipi_send_mask(irq, &dest);
> }
> +
> +#if defined(__aarch64__)
> +/* alloc_lpi_tables: Allocate LPI config and pending tables */
> +void gicv3_lpi_alloc_tables(void)
> +{
> + unsigned long n = SZ_64K >> PAGE_SHIFT;
> + unsigned long order = fls(n);
> + u64 prop_val;
> + int cpu;
> +
> + gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
> +
> + /* ID bits = 13, ie. up to 14b LPI INTID */
> + prop_val = (u64)gicv3_data.lpi_prop | 13;
> +
> + /*
> + * Allocate pending tables for each redistributor
> + * and set PROPBASER and PENDBASER
> + */
> + for_each_present_cpu(cpu) {
> + u64 pend_val;
> + void *ptr;
> +
> + ptr = gicv3_data.redist_base[cpu];
> +
> + writeq(prop_val, ptr + GICR_PROPBASER);
> +
> + gicv3_data.lpi_pend[cpu] = (void *)virt_to_phys(alloc_pages(order));
> +
> + pend_val = (u64)gicv3_data.lpi_pend[cpu];
> +
> + writeq(pend_val, ptr + GICR_PENDBASER);
> + }
> +}
> +
> +void gicv3_lpi_set_config(int n, u8 value)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
But this is actually the *physical* address, shouldn't it be
converted by phys_to_virt() before reading/writing something?
Like what you've done for the 'lpi_pend[rdist]' before writing
pending bit. Or I'm missing some points here?
> +
> + *entry = value;
> +}
> +
> +u8 gicv3_lpi_get_config(int n)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
The same as above.
Thanks,
Zenghui
> +
> + return *entry;
> +}
> +
> +void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
> +{
> + u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
> + u8 mask = 1 << (n % 8), byte;
> +
> + ptr += (n / 8);
> + byte = *ptr;
> + if (set)
> + byte |= mask;
> + else
> + byte &= ~mask;
> + *ptr = byte;
> +}
> +#endif /* __aarch64__ */
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
@ 2020-02-07 2:12 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-07 2:12 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, drjones, alexandru.elisei, thuth, peter.maydell
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> Allocate the LPI configuration and per re-distributor pending table.
> Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
> by default in the config table.
>
> Also introduce a helper routine that allows to set the pending table
> bit for a given LPI.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
> and add _lpi prefix too
>
> v1 -> v2:
> - remove memory attributes
> ---
> lib/arm/asm/gic-v3.h | 16 +++++++++++
> lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 80 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index ffb2e26..ec2a6f0 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -48,6 +48,16 @@
> #define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
>
> +#define GICR_PROPBASER_IDBITS_MASK (0x1f)
This is not being used. You can use it when calculating prop_val
or just drop it.
> +
> +#define GICR_PENDBASER_PTZ BIT_ULL(62)
> +
> +#define LPI_PROP_GROUP1 (1 << 1)
> +#define LPI_PROP_ENABLED (1 << 0)
> +#define LPI_PROP_DEFAULT_PRIO 0xa0
> +#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
> + LPI_PROP_ENABLED)
> +
> #include <asm/arch_gicv3.h>
>
> #ifndef __ASSEMBLY__
> @@ -64,6 +74,8 @@ struct gicv3_data {
> void *dist_base;
> void *redist_bases[GICV3_NR_REDISTS];
> void *redist_base[NR_CPUS];
> + void *lpi_prop;
> + void *lpi_pend[NR_CPUS];
> unsigned int irq_nr;
> };
> extern struct gicv3_data gicv3_data;
> @@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
> extern void gicv3_ipi_send_single(int irq, int cpu);
> extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
> extern void gicv3_set_redist_base(size_t stride);
> +extern void gicv3_lpi_set_config(int n, u8 val);
> +extern u8 gicv3_lpi_get_config(int n);
> +extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> +extern void gicv3_lpi_alloc_tables(void);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> index feecb5e..c33f883 100644
> --- a/lib/arm/gic-v3.c
> +++ b/lib/arm/gic-v3.c
> @@ -5,6 +5,7 @@
> */
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <alloc_page.h>
>
> void gicv3_set_redist_base(size_t stride)
> {
> @@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
> cpumask_set_cpu(cpu, &dest);
> gicv3_ipi_send_mask(irq, &dest);
> }
> +
> +#if defined(__aarch64__)
> +/* alloc_lpi_tables: Allocate LPI config and pending tables */
> +void gicv3_lpi_alloc_tables(void)
> +{
> + unsigned long n = SZ_64K >> PAGE_SHIFT;
> + unsigned long order = fls(n);
> + u64 prop_val;
> + int cpu;
> +
> + gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
> +
> + /* ID bits = 13, ie. up to 14b LPI INTID */
> + prop_val = (u64)gicv3_data.lpi_prop | 13;
> +
> + /*
> + * Allocate pending tables for each redistributor
> + * and set PROPBASER and PENDBASER
> + */
> + for_each_present_cpu(cpu) {
> + u64 pend_val;
> + void *ptr;
> +
> + ptr = gicv3_data.redist_base[cpu];
> +
> + writeq(prop_val, ptr + GICR_PROPBASER);
> +
> + gicv3_data.lpi_pend[cpu] = (void *)virt_to_phys(alloc_pages(order));
> +
> + pend_val = (u64)gicv3_data.lpi_pend[cpu];
> +
> + writeq(pend_val, ptr + GICR_PENDBASER);
> + }
> +}
> +
> +void gicv3_lpi_set_config(int n, u8 value)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
But this is actually the *physical* address, shouldn't it be
converted by phys_to_virt() before reading/writing something?
Like what you've done for the 'lpi_pend[rdist]' before writing
pending bit. Or I'm missing some points here?
> +
> + *entry = value;
> +}
> +
> +u8 gicv3_lpi_get_config(int n)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
The same as above.
Thanks,
Zenghui
> +
> + return *entry;
> +}
> +
> +void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
> +{
> + u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
> + u8 mask = 1 << (n % 8), byte;
> +
> + ptr += (n / 8);
> + byte = *ptr;
> + if (set)
> + byte |= mask;
> + else
> + byte &= ~mask;
> + *ptr = byte;
> +}
> +#endif /* __aarch64__ */
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
@ 2020-02-07 2:12 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-07 2:12 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> Allocate the LPI configuration and per re-distributor pending table.
> Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
> by default in the config table.
>
> Also introduce a helper routine that allows to set the pending table
> bit for a given LPI.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
> and add _lpi prefix too
>
> v1 -> v2:
> - remove memory attributes
> ---
> lib/arm/asm/gic-v3.h | 16 +++++++++++
> lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 80 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index ffb2e26..ec2a6f0 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -48,6 +48,16 @@
> #define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
>
> +#define GICR_PROPBASER_IDBITS_MASK (0x1f)
This is not being used. You can use it when calculating prop_val
or just drop it.
> +
> +#define GICR_PENDBASER_PTZ BIT_ULL(62)
> +
> +#define LPI_PROP_GROUP1 (1 << 1)
> +#define LPI_PROP_ENABLED (1 << 0)
> +#define LPI_PROP_DEFAULT_PRIO 0xa0
> +#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
> + LPI_PROP_ENABLED)
> +
> #include <asm/arch_gicv3.h>
>
> #ifndef __ASSEMBLY__
> @@ -64,6 +74,8 @@ struct gicv3_data {
> void *dist_base;
> void *redist_bases[GICV3_NR_REDISTS];
> void *redist_base[NR_CPUS];
> + void *lpi_prop;
> + void *lpi_pend[NR_CPUS];
> unsigned int irq_nr;
> };
> extern struct gicv3_data gicv3_data;
> @@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
> extern void gicv3_ipi_send_single(int irq, int cpu);
> extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
> extern void gicv3_set_redist_base(size_t stride);
> +extern void gicv3_lpi_set_config(int n, u8 val);
> +extern u8 gicv3_lpi_get_config(int n);
> +extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> +extern void gicv3_lpi_alloc_tables(void);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> index feecb5e..c33f883 100644
> --- a/lib/arm/gic-v3.c
> +++ b/lib/arm/gic-v3.c
> @@ -5,6 +5,7 @@
> */
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <alloc_page.h>
>
> void gicv3_set_redist_base(size_t stride)
> {
> @@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
> cpumask_set_cpu(cpu, &dest);
> gicv3_ipi_send_mask(irq, &dest);
> }
> +
> +#if defined(__aarch64__)
> +/* alloc_lpi_tables: Allocate LPI config and pending tables */
> +void gicv3_lpi_alloc_tables(void)
> +{
> + unsigned long n = SZ_64K >> PAGE_SHIFT;
> + unsigned long order = fls(n);
> + u64 prop_val;
> + int cpu;
> +
> + gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
> +
> + /* ID bits = 13, ie. up to 14b LPI INTID */
> + prop_val = (u64)gicv3_data.lpi_prop | 13;
> +
> + /*
> + * Allocate pending tables for each redistributor
> + * and set PROPBASER and PENDBASER
> + */
> + for_each_present_cpu(cpu) {
> + u64 pend_val;
> + void *ptr;
> +
> + ptr = gicv3_data.redist_base[cpu];
> +
> + writeq(prop_val, ptr + GICR_PROPBASER);
> +
> + gicv3_data.lpi_pend[cpu] = (void *)virt_to_phys(alloc_pages(order));
> +
> + pend_val = (u64)gicv3_data.lpi_pend[cpu];
> +
> + writeq(pend_val, ptr + GICR_PENDBASER);
> + }
> +}
> +
> +void gicv3_lpi_set_config(int n, u8 value)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
But this is actually the *physical* address, shouldn't it be
converted by phys_to_virt() before reading/writing something?
Like what you've done for the 'lpi_pend[rdist]' before writing
pending bit. Or I'm missing some points here?
> +
> + *entry = value;
> +}
> +
> +u8 gicv3_lpi_get_config(int n)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
The same as above.
Thanks,
Zenghui
> +
> + return *entry;
> +}
> +
> +void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
> +{
> + u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
> + u8 mask = 1 << (n % 8), byte;
> +
> + ptr += (n / 8);
> + byte = *ptr;
> + if (set)
> + byte |= mask;
> + else
> + byte &= ~mask;
> + *ptr = byte;
> +}
> +#endif /* __aarch64__ */
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 3:20 ` Zenghui Yu
-1 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-07 3:20 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, alexandru.elisei, thuth
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> its_enable_defaults() is the top init function that allocates the
> command queue and all the requested tables (device, collection,
> lpi config and pending tables), enable LPIs at distributor level
> and ITS level.
>
> gicv3_enable_defaults must be called before.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - introduce its_setup_baser in this patch
> - squash "arm/arm64: ITS: Init the command queue" in this patch.
> ---
> lib/arm/asm/gic-v3-its.h | 8 ++++
> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 97 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index 815c515..fe73c04 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -36,6 +36,8 @@ struct its_data {
> void *base;
> struct its_typer typer;
> struct its_baser baser[GITS_BASER_NR_REGS];
> + struct its_cmd_block *cmd_base;
> + struct its_cmd_block *cmd_write;
> };
>
> extern struct its_data its_data;
> @@ -88,10 +90,16 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> +
> +struct its_cmd_block {
> + u64 raw_cmd[4];
> +};
> +
> extern void its_parse_typer(void);
> extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> +extern void its_enable_defaults(void);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index 2c0ce13..d1e7e52 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -86,3 +86,92 @@ void its_init(void)
> its_parse_baser(i, &its_data.baser[i]);
> }
>
> +static void its_setup_baser(int i, struct its_baser *baser)
> +{
> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
> + u64 val;
> +
> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
> +
> + val = ((u64)baser->table_addr |
> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
> + (u64)baser->indirect << 62 |
I haven't seen the 'nr_pages' and 'indirect' are programmed anywhere
except in its_parse_baser(). It looks like they're treated as RO (but
they shouldn't) and I now don't think it makes sense to parse them in
its_parse_baser(), in patch#5.
> + (u64)baser->valid << 63);
> +
> + switch (baser->psz) {
> + case SZ_4K:
> + val |= GITS_BASER_PAGE_SIZE_4K;
> + break;
> + case SZ_16K:
> + val |= GITS_BASER_PAGE_SIZE_16K;
> + break;
> + case SZ_64K:
> + val |= GITS_BASER_PAGE_SIZE_64K;
> + break;
> + }
> +
> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
> +}
> +
> +/**
> + * init_cmd_queue: Allocate the command queue and initialize
> + * CBASER, CREADR, CWRITER
no 'CREADR'.
Thanks,
Zenghui
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-02-07 3:20 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-07 3:20 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, drjones, alexandru.elisei, thuth, peter.maydell
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> its_enable_defaults() is the top init function that allocates the
> command queue and all the requested tables (device, collection,
> lpi config and pending tables), enable LPIs at distributor level
> and ITS level.
>
> gicv3_enable_defaults must be called before.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - introduce its_setup_baser in this patch
> - squash "arm/arm64: ITS: Init the command queue" in this patch.
> ---
> lib/arm/asm/gic-v3-its.h | 8 ++++
> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 97 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index 815c515..fe73c04 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -36,6 +36,8 @@ struct its_data {
> void *base;
> struct its_typer typer;
> struct its_baser baser[GITS_BASER_NR_REGS];
> + struct its_cmd_block *cmd_base;
> + struct its_cmd_block *cmd_write;
> };
>
> extern struct its_data its_data;
> @@ -88,10 +90,16 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> +
> +struct its_cmd_block {
> + u64 raw_cmd[4];
> +};
> +
> extern void its_parse_typer(void);
> extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> +extern void its_enable_defaults(void);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index 2c0ce13..d1e7e52 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -86,3 +86,92 @@ void its_init(void)
> its_parse_baser(i, &its_data.baser[i]);
> }
>
> +static void its_setup_baser(int i, struct its_baser *baser)
> +{
> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
> + u64 val;
> +
> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
> +
> + val = ((u64)baser->table_addr |
> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
> + (u64)baser->indirect << 62 |
I haven't seen the 'nr_pages' and 'indirect' are programmed anywhere
except in its_parse_baser(). It looks like they're treated as RO (but
they shouldn't) and I now don't think it makes sense to parse them in
its_parse_baser(), in patch#5.
> + (u64)baser->valid << 63);
> +
> + switch (baser->psz) {
> + case SZ_4K:
> + val |= GITS_BASER_PAGE_SIZE_4K;
> + break;
> + case SZ_16K:
> + val |= GITS_BASER_PAGE_SIZE_16K;
> + break;
> + case SZ_64K:
> + val |= GITS_BASER_PAGE_SIZE_64K;
> + break;
> + }
> +
> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
> +}
> +
> +/**
> + * init_cmd_queue: Allocate the command queue and initialize
> + * CBASER, CREADR, CWRITER
no 'CREADR'.
Thanks,
Zenghui
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-02-07 3:20 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-07 3:20 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> its_enable_defaults() is the top init function that allocates the
> command queue and all the requested tables (device, collection,
> lpi config and pending tables), enable LPIs at distributor level
> and ITS level.
>
> gicv3_enable_defaults must be called before.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - introduce its_setup_baser in this patch
> - squash "arm/arm64: ITS: Init the command queue" in this patch.
> ---
> lib/arm/asm/gic-v3-its.h | 8 ++++
> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 97 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index 815c515..fe73c04 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -36,6 +36,8 @@ struct its_data {
> void *base;
> struct its_typer typer;
> struct its_baser baser[GITS_BASER_NR_REGS];
> + struct its_cmd_block *cmd_base;
> + struct its_cmd_block *cmd_write;
> };
>
> extern struct its_data its_data;
> @@ -88,10 +90,16 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> +
> +struct its_cmd_block {
> + u64 raw_cmd[4];
> +};
> +
> extern void its_parse_typer(void);
> extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> +extern void its_enable_defaults(void);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index 2c0ce13..d1e7e52 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -86,3 +86,92 @@ void its_init(void)
> its_parse_baser(i, &its_data.baser[i]);
> }
>
> +static void its_setup_baser(int i, struct its_baser *baser)
> +{
> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
> + u64 val;
> +
> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
> +
> + val = ((u64)baser->table_addr |
> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
> + (u64)baser->indirect << 62 |
I haven't seen the 'nr_pages' and 'indirect' are programmed anywhere
except in its_parse_baser(). It looks like they're treated as RO (but
they shouldn't) and I now don't think it makes sense to parse them in
its_parse_baser(), in patch#5.
> + (u64)baser->valid << 63);
> +
> + switch (baser->psz) {
> + case SZ_4K:
> + val |= GITS_BASER_PAGE_SIZE_4K;
> + break;
> + case SZ_16K:
> + val |= GITS_BASER_PAGE_SIZE_16K;
> + break;
> + case SZ_64K:
> + val |= GITS_BASER_PAGE_SIZE_64K;
> + break;
> + }
> +
> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
> +}
> +
> +/**
> + * init_cmd_queue: Allocate the command queue and initialize
> + * CBASER, CREADR, CWRITER
no 'CREADR'.
Thanks,
Zenghui
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 5:41 ` Zenghui Yu
-1 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-07 5:41 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, alexandru.elisei, thuth
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> Introduce an helper functions to register
> - a new device, characterized by its device id and the
> max number of event IDs that dimension its ITT (Interrupt
> Translation Table). The function allocates the ITT.
>
> - a new collection, characterized by its ID and the
> target processing engine (PE).
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - s/report_abort/assert
>
> v1 -> v2:
> - s/nb_/nr_
> ---
> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 63 insertions(+), 1 deletion(-)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index fe73c04..acd97a9 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -31,6 +31,19 @@ struct its_baser {
> };
>
> #define GITS_BASER_NR_REGS 8
> +#define GITS_MAX_DEVICES 8
> +#define GITS_MAX_COLLECTIONS 8
> +
> +struct its_device {
> + u32 device_id; /* device ID */
> + u32 nr_ites; /* Max Interrupt Translation Entries */
> + void *itt; /* Interrupt Translation Table GPA */
> +};
> +
> +struct its_collection {
> + u64 target_address;
> + u16 col_id;
> +};
>
> struct its_data {
> void *base;
> @@ -38,6 +51,10 @@ struct its_data {
> struct its_baser baser[GITS_BASER_NR_REGS];
> struct its_cmd_block *cmd_base;
> struct its_cmd_block *cmd_write;
> + struct its_device devices[GITS_MAX_DEVICES];
> + u32 nr_devices; /* Allocated Devices */
> + struct its_collection collections[GITS_MAX_COLLECTIONS];
> + u32 nr_collections; /* Allocated Collections */
> };
>
> extern struct its_data its_data;
> @@ -90,7 +107,6 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> -
> struct its_cmd_block {
> u64 raw_cmd[4];
> };
> @@ -100,6 +116,8 @@ extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> extern void its_enable_defaults(void);
> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> +extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index d1e7e52..c2dcd01 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>
> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
> }
> +
> +struct its_device *its_create_device(u32 device_id, int nr_ites)
> +{
> + struct its_baser *baser;
> + struct its_device *new;
> + unsigned long n, order;
> +
> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
> +
> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + if (!baser)
> + return NULL;
I think there's no need to lookup the device baser here. As the
device baser should have already been setup at initialization
time (i.e. in its_enable_defaults). And anyway, 'baser' is not
being used in this function.
Thanks,
Zenghui
> +
> + new = &its_data.devices[its_data.nr_devices];
> +
> + new->device_id = device_id;
> + new->nr_ites = nr_ites;
> +
> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
> + new->itt = (void *)virt_to_phys(alloc_pages(order));
> +
> + its_data.nr_devices++;
> + return new;
> +}
> +
> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
> +{
> + struct its_collection *new;
> +
> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
> +
> + new = &its_data.collections[its_data.nr_collections];
> +
> + new->col_id = col_id;
> +
> + if (its_data.typer.pta)
> + new->target_address = (u64)gicv3_data.redist_base[pe];
> + else
> + new->target_address = pe << 16;
> +
> + its_data.nr_collections++;
> + return new;
> +}
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
@ 2020-02-07 5:41 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-07 5:41 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, drjones, alexandru.elisei, thuth, peter.maydell
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> Introduce an helper functions to register
> - a new device, characterized by its device id and the
> max number of event IDs that dimension its ITT (Interrupt
> Translation Table). The function allocates the ITT.
>
> - a new collection, characterized by its ID and the
> target processing engine (PE).
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - s/report_abort/assert
>
> v1 -> v2:
> - s/nb_/nr_
> ---
> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 63 insertions(+), 1 deletion(-)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index fe73c04..acd97a9 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -31,6 +31,19 @@ struct its_baser {
> };
>
> #define GITS_BASER_NR_REGS 8
> +#define GITS_MAX_DEVICES 8
> +#define GITS_MAX_COLLECTIONS 8
> +
> +struct its_device {
> + u32 device_id; /* device ID */
> + u32 nr_ites; /* Max Interrupt Translation Entries */
> + void *itt; /* Interrupt Translation Table GPA */
> +};
> +
> +struct its_collection {
> + u64 target_address;
> + u16 col_id;
> +};
>
> struct its_data {
> void *base;
> @@ -38,6 +51,10 @@ struct its_data {
> struct its_baser baser[GITS_BASER_NR_REGS];
> struct its_cmd_block *cmd_base;
> struct its_cmd_block *cmd_write;
> + struct its_device devices[GITS_MAX_DEVICES];
> + u32 nr_devices; /* Allocated Devices */
> + struct its_collection collections[GITS_MAX_COLLECTIONS];
> + u32 nr_collections; /* Allocated Collections */
> };
>
> extern struct its_data its_data;
> @@ -90,7 +107,6 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> -
> struct its_cmd_block {
> u64 raw_cmd[4];
> };
> @@ -100,6 +116,8 @@ extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> extern void its_enable_defaults(void);
> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> +extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index d1e7e52..c2dcd01 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>
> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
> }
> +
> +struct its_device *its_create_device(u32 device_id, int nr_ites)
> +{
> + struct its_baser *baser;
> + struct its_device *new;
> + unsigned long n, order;
> +
> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
> +
> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + if (!baser)
> + return NULL;
I think there's no need to lookup the device baser here. As the
device baser should have already been setup at initialization
time (i.e. in its_enable_defaults). And anyway, 'baser' is not
being used in this function.
Thanks,
Zenghui
> +
> + new = &its_data.devices[its_data.nr_devices];
> +
> + new->device_id = device_id;
> + new->nr_ites = nr_ites;
> +
> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
> + new->itt = (void *)virt_to_phys(alloc_pages(order));
> +
> + its_data.nr_devices++;
> + return new;
> +}
> +
> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
> +{
> + struct its_collection *new;
> +
> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
> +
> + new = &its_data.collections[its_data.nr_collections];
> +
> + new->col_id = col_id;
> +
> + if (its_data.typer.pta)
> + new->target_address = (u64)gicv3_data.redist_base[pe];
> + else
> + new->target_address = pe << 16;
> +
> + its_data.nr_collections++;
> + return new;
> +}
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
@ 2020-02-07 5:41 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-02-07 5:41 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Hi Eric,
On 2020/1/28 18:34, Eric Auger wrote:
> Introduce an helper functions to register
> - a new device, characterized by its device id and the
> max number of event IDs that dimension its ITT (Interrupt
> Translation Table). The function allocates the ITT.
>
> - a new collection, characterized by its ID and the
> target processing engine (PE).
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - s/report_abort/assert
>
> v1 -> v2:
> - s/nb_/nr_
> ---
> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 63 insertions(+), 1 deletion(-)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index fe73c04..acd97a9 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -31,6 +31,19 @@ struct its_baser {
> };
>
> #define GITS_BASER_NR_REGS 8
> +#define GITS_MAX_DEVICES 8
> +#define GITS_MAX_COLLECTIONS 8
> +
> +struct its_device {
> + u32 device_id; /* device ID */
> + u32 nr_ites; /* Max Interrupt Translation Entries */
> + void *itt; /* Interrupt Translation Table GPA */
> +};
> +
> +struct its_collection {
> + u64 target_address;
> + u16 col_id;
> +};
>
> struct its_data {
> void *base;
> @@ -38,6 +51,10 @@ struct its_data {
> struct its_baser baser[GITS_BASER_NR_REGS];
> struct its_cmd_block *cmd_base;
> struct its_cmd_block *cmd_write;
> + struct its_device devices[GITS_MAX_DEVICES];
> + u32 nr_devices; /* Allocated Devices */
> + struct its_collection collections[GITS_MAX_COLLECTIONS];
> + u32 nr_collections; /* Allocated Collections */
> };
>
> extern struct its_data its_data;
> @@ -90,7 +107,6 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> -
> struct its_cmd_block {
> u64 raw_cmd[4];
> };
> @@ -100,6 +116,8 @@ extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> extern void its_enable_defaults(void);
> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> +extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index d1e7e52..c2dcd01 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>
> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
> }
> +
> +struct its_device *its_create_device(u32 device_id, int nr_ites)
> +{
> + struct its_baser *baser;
> + struct its_device *new;
> + unsigned long n, order;
> +
> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
> +
> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + if (!baser)
> + return NULL;
I think there's no need to lookup the device baser here. As the
device baser should have already been setup at initialization
time (i.e. in its_enable_defaults). And anyway, 'baser' is not
being used in this function.
Thanks,
Zenghui
> +
> + new = &its_data.devices[its_data.nr_devices];
> +
> + new->device_id = device_id;
> + new->nr_ites = nr_ites;
> +
> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
> + new->itt = (void *)virt_to_phys(alloc_pages(order));
> +
> + its_data.nr_devices++;
> + return new;
> +}
> +
> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
> +{
> + struct its_collection *new;
> +
> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
> +
> + new = &its_data.collections[its_data.nr_collections];
> +
> + new->col_id = col_id;
> +
> + if (its_data.typer.pta)
> + new->target_address = (u64)gicv3_data.redist_base[pe];
> + else
> + new->target_address = pe << 16;
> +
> + its_data.nr_collections++;
> + return new;
> +}
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 10:19 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 10:19 UTC (permalink / raw)
To: Eric Auger
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
andre.przywara, peter.maydell, yuzenghui, alexandru.elisei,
thuth
On Tue, Jan 28, 2020 at 11:34:50AM +0100, Eric Auger wrote:
> Detect the presence of an ITS as part of the GICv3 init
> routine, initialize its base address and read few registers
> the IIDR, the TYPER to store its dimensioning parameters.
> Also parse the BASER registers.
>
> This is our first ITS test, belonging to a new "its" group.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - updated dates and changed author
> - squash "arm/arm64: ITS: Test BASER" into this patch but
> removes setup_baser which will be introduced later.
> - only compile on aarch64
> - restrict the new test to aarch64
>
> v1 -> v2:
> - clean GITS_TYPER macros and unused fields in typer struct
> - remove memory attribute related macros
> - remove everything related to memory attributes
> - s/dev_baser/coll_baser/ in report_info
> - add extra line
> - removed index filed in its_baser
> ---
> arm/Makefile.arm64 | 1 +
> arm/gic.c | 49 ++++++++++++++++++
> arm/unittests.cfg | 7 +++
> lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
> lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
> lib/arm/gic.c | 30 +++++++++--
> lib/arm64/asm/gic-v3-its.h | 1 +
> 7 files changed, 274 insertions(+), 5 deletions(-)
> create mode 100644 lib/arm/asm/gic-v3-its.h
> create mode 100644 lib/arm/gic-v3-its.c
> create mode 100644 lib/arm64/asm/gic-v3-its.h
>
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 6d3dc2c..2571ffb 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -19,6 +19,7 @@ endef
> cstart.o = $(TEST_DIR)/cstart64.o
> cflatobjs += lib/arm64/processor.o
> cflatobjs += lib/arm64/spinlock.o
> +cflatobjs += lib/arm/gic-v3-its.o
If gic-v3-its.c will never be compiled for arm, then it should
probably go lib/arm64, not lib/arm. Same comment for all other
new source and header files. The only problem with that is...
>
> OBJDIRS += lib/arm64
>
> diff --git a/arm/gic.c b/arm/gic.c
> index abf08c7..4d7dd03 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -16,6 +16,7 @@
> #include <asm/processor.h>
> #include <asm/delay.h>
> #include <asm/gic.h>
> +#include <asm/gic-v3-its.h>
...here where we'd have to do
#if defined(__aarch64__)
#include <asm/gic-v3-its.h>
#endif
which is ugly. However that can be avoided by adding an
empty lib/arm/asm/gic-v3-its.h file.
> #include <asm/smp.h>
> #include <asm/barrier.h>
> #include <asm/io.h>
> @@ -518,6 +519,50 @@ static void gic_test_mmio(void)
> test_targets(nr_irqs);
> }
>
> +#if defined(__arm__)
> +
> +static void test_its_introspection(void) {}
static void test_its_introspection(void)
{
report_abort(...);
}
> +
> +#else /* __arch64__ */
> +
> +static void test_its_introspection(void)
> +{
> + struct its_baser *dev_baser, *coll_baser;
> + struct its_typer *typer = &its_data.typer;
> +
> + if (!gicv3_its_base()) {
> + report_skip("No ITS, skip ...");
> + return;
> + }
> +
> + /* IIDR */
> + report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
> + "GITS_IIDR is read-only"),
> +
> + /* TYPER */
> + report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
> + "GITS_TYPER is read-only");
> +
> + report(typer->phys_lpi, "ITS supports physical LPIs");
> + report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
> + report_info("ITT entry size = 0x%x", typer->ite_size);
> + report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
> + typer->eventid_bits, typer->deviceid_bits,
> + typer->collid_bits);
> + report(typer->eventid_bits && typer->deviceid_bits &&
> + typer->collid_bits, "ID spaces");
> + report_info("Target address format %s",
> + typer->pta ? "Redist basse address" : "PE #");
> +
> + dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
> + report(dev_baser && coll_baser, "detect device and collection BASER");
> + report_info("device baser entry_size = 0x%x", dev_baser->esz);
> + report_info("collection baser entry_size = 0x%x", coll_baser->esz);
> +}
> +
> +#endif
> +
> int main(int argc, char **argv)
> {
> if (!gic_init()) {
> @@ -549,6 +594,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> gic_test_mmio();
> report_prefix_pop();
> + } else if (strcmp(argv[1], "its-introspection") == 0) {
> + report_prefix_push(argv[1]);
> + test_its_introspection();
> + report_prefix_pop();
> } else {
> report_abort("Unknown subtest '%s'", argv[1]);
> }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index daeb5a0..ba2b31b 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -122,6 +122,13 @@ smp = $MAX_SMP
> extra_params = -machine gic-version=3 -append 'active'
> groups = gic
>
> +[its-introspection]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'its-introspection'
> +groups = its
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..815c515
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -0,0 +1,103 @@
> +/*
> + * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_ITS_H_
> +#define _ASMARM_GIC_V3_ITS_H_
> +
> +#ifndef __ASSEMBLY__
Doesn't look like you use the #else /* __ASSEMBLY__ */ side of this.
I'd leave out the #ifndef until we add defines we need to access
from assembler.
> +
> +struct its_typer {
> + unsigned int ite_size;
> + unsigned int eventid_bits;
> + unsigned int deviceid_bits;
> + unsigned int collid_bits;
> + bool pta;
> + bool phys_lpi;
> + bool virt_lpi;
> +};
> +
> +struct its_baser {
> + int type;
> + size_t psz;
> + int nr_pages;
> + bool indirect;
> + phys_addr_t table_addr;
> + bool valid;
> + int esz;
> +};
> +
> +#define GITS_BASER_NR_REGS 8
> +
> +struct its_data {
> + void *base;
> + struct its_typer typer;
> + struct its_baser baser[GITS_BASER_NR_REGS];
> +};
> +
> +extern struct its_data its_data;
> +
> +#define gicv3_its_base() (its_data.base)
> +
> +#if defined(__aarch64__)
> +
> +#define GITS_CTLR 0x0000
> +#define GITS_IIDR 0x0004
> +#define GITS_TYPER 0x0008
> +#define GITS_CBASER 0x0080
> +#define GITS_CWRITER 0x0088
> +#define GITS_CREADR 0x0090
> +#define GITS_BASER 0x0100
> +
> +#define GITS_TYPER_PLPIS BIT(0)
> +#define GITS_TYPER_VLPIS BIT(1)
> +#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
> +#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
> +#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
> +#define GITS_TYPER_IDBITS_SHIFT 8
> +#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
> +#define GITS_TYPER_DEVBITS_SHIFT 13
> +#define GITS_TYPER_PTA BIT(19)
> +#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
> +#define GITS_TYPER_CIDBITS_SHIFT 32
> +#define GITS_TYPER_CIL BIT(36)
> +
> +#define GITS_CTLR_ENABLE (1U << 0)
> +
> +#define GITS_CBASER_VALID (1UL << 63)
> +
> +#define GITS_BASER_VALID BIT(63)
> +#define GITS_BASER_INDIRECT BIT(62)
> +#define GITS_BASER_TYPE_SHIFT (56)
> +#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
> +#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
> +#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
> +#define GITS_BASER_PAGE_SIZE_SHIFT (8)
> +#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGES_MAX 256
> +#define GITS_BASER_PAGES_SHIFT (0)
> +#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
> +#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
> +#define GITS_BASER_TYPE_NONE 0
> +#define GITS_BASER_TYPE_DEVICE 1
> +#define GITS_BASER_TYPE_COLLECTION 4
> +
> +extern void its_parse_typer(void);
> +extern void its_init(void);
> +extern int its_parse_baser(int i, struct its_baser *baser);
> +extern struct its_baser *its_lookup_baser(int type);
> +
> +#else /* __arm__ */
> +
> +static inline void its_init(void) {}
Looks like the empty gic-v3-its.h I suggested creating above will actually
be useful. We can add stubs like this in it.
> +
> +#endif
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_ITS_H_ */
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> new file mode 100644
> index 0000000..2c0ce13
> --- /dev/null
> +++ b/lib/arm/gic-v3-its.c
> @@ -0,0 +1,88 @@
> +/*
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <asm/gic.h>
> +#include <alloc_page.h>
> +#include <asm/gic-v3-its.h>
> +
> +void its_parse_typer(void)
> +{
> + u64 typer = readq(gicv3_its_base() + GITS_TYPER);
> +
> + its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
> + GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
> + its_data.typer.pta = typer & GITS_TYPER_PTA;
> + its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
> + GITS_TYPER_IDBITS_SHIFT) + 1;
> + its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
> + GITS_TYPER_DEVBITS_SHIFT) + 1;
> +
> + if (typer & GITS_TYPER_CIL)
> + its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
> + GITS_TYPER_CIDBITS_SHIFT) + 1;
nit: please consider aligning like this
((typer & MASK) >>
SHIFT) + 1;
Or, maybe better to macro it
#define TYPER_FIELD(typer, mask, shift) (((type) & (mask) >> (shift)) + 1)
And, rather than have a bunch of 'its_data.typer's we could use an alias,
helping us stay within 120 chars.
struct its_typer *t = &its_data.typer;
t->ite_size = TYPER_FIELD(typer, GITS_TYPER_ITT_ENTRY_SIZE,
GITS_TYPER_ITT_ENTRY_SIZE_SHIFT);
> + else
> + its_data.typer.collid_bits = 16;
> +
> + its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
> + its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
> +}
> +
> +int its_parse_baser(int i, struct its_baser *baser)
> +{
> + void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
> + u64 val = readq(reg_addr);
> +
> + if (!val) {
> + memset(baser, 0, sizeof(*baser));
> + return -1;
> + }
> +
> + baser->valid = val & GITS_BASER_VALID;
> + baser->indirect = val & GITS_BASER_INDIRECT;
> + baser->type = GITS_BASER_TYPE(val);
> + baser->esz = GITS_BASER_ENTRY_SIZE(val);
> + baser->nr_pages = GITS_BASER_NR_PAGES(val);
> + baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
> + switch (val & GITS_BASER_PAGE_SIZE_MASK) {
> + case GITS_BASER_PAGE_SIZE_4K:
> + baser->psz = SZ_4K;
> + break;
> + case GITS_BASER_PAGE_SIZE_16K:
> + baser->psz = SZ_16K;
> + break;
> + case GITS_BASER_PAGE_SIZE_64K:
> + baser->psz = SZ_64K;
> + break;
> + default:
> + baser->psz = SZ_64K;
> + }
> + return 0;
> +}
> +
> +struct its_baser *its_lookup_baser(int type)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
> + struct its_baser *baser = &its_data.baser[i];
> +
> + if (baser->type == type)
> + return baser;
> + }
> + return NULL;
> +}
> +
> +void its_init(void)
> +{
> + int i;
> +
> + if (!its_data.base)
> + return;
> +
> + its_parse_typer();
> + for (i = 0; i < GITS_BASER_NR_REGS; i++)
> + its_parse_baser(i, &its_data.baser[i]);
> +}
> +
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index aa9cb86..6b70b05 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -6,9 +6,11 @@
> #include <devicetree.h>
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <asm/gic-v3-its.h>
>
> struct gicv2_data gicv2_data;
> struct gicv3_data gicv3_data;
> +struct its_data its_data;
>
> struct gic_common_ops {
> void (*enable_defaults)(void);
> @@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
> * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> */
> static bool
> -gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
> {
> struct dt_pbus_reg reg;
> - struct dt_device gic;
> + struct dt_device gic, its;
> struct dt_bus bus;
> - int node, ret, i;
> + int node, subnode, ret, i, len;
> + const void *fdt = dt_fdt();
>
> dt_bus_init_defaults(&bus);
> dt_device_init(&gic, &bus, NULL);
> @@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> base2[i] = ioremap(reg.addr, reg.size);
> }
>
> + if (base3 && !strcmp(compatible, "arm,gic-v3")) {
If base != NULL, then we could assert(strcmp(compatible, "arm,cortex-a15-gic") != 0)
> + dt_for_each_subnode(node, subnode) {
> + const struct fdt_property *prop;
> +
> + prop = fdt_get_property(fdt, subnode, "compatible", &len);
> + if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
> + dt_device_bind_node(&its, subnode);
> + ret = dt_pbus_translate(&its, 0, ®);
> + assert(ret == 0);
> + *base3 = ioremap(reg.addr, reg.size);
> + break;
> + }
> + }
> +
> + }
> +
> return true;
> }
>
> int gicv2_init(void)
> {
> return gic_get_dt_bases("arm,cortex-a15-gic",
> - &gicv2_data.dist_base, &gicv2_data.cpu_base);
> + &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
> }
>
> int gicv3_init(void)
> {
> return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> - &gicv3_data.redist_bases[0]);
> + &gicv3_data.redist_bases[0], &its_data.base);
> }
>
> int gic_version(void)
> @@ -104,6 +123,7 @@ int gic_init(void)
> gic_common_ops = &gicv2_common_ops;
> else if (gicv3_init())
> gic_common_ops = &gicv3_common_ops;
> + its_init();
> return gic_version();
> }
>
> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..083cba4
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3-its.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3-its.h"
> --
> 2.20.1
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
@ 2020-02-07 10:19 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 10:19 UTC (permalink / raw)
To: Eric Auger
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:50AM +0100, Eric Auger wrote:
> Detect the presence of an ITS as part of the GICv3 init
> routine, initialize its base address and read few registers
> the IIDR, the TYPER to store its dimensioning parameters.
> Also parse the BASER registers.
>
> This is our first ITS test, belonging to a new "its" group.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - updated dates and changed author
> - squash "arm/arm64: ITS: Test BASER" into this patch but
> removes setup_baser which will be introduced later.
> - only compile on aarch64
> - restrict the new test to aarch64
>
> v1 -> v2:
> - clean GITS_TYPER macros and unused fields in typer struct
> - remove memory attribute related macros
> - remove everything related to memory attributes
> - s/dev_baser/coll_baser/ in report_info
> - add extra line
> - removed index filed in its_baser
> ---
> arm/Makefile.arm64 | 1 +
> arm/gic.c | 49 ++++++++++++++++++
> arm/unittests.cfg | 7 +++
> lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
> lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
> lib/arm/gic.c | 30 +++++++++--
> lib/arm64/asm/gic-v3-its.h | 1 +
> 7 files changed, 274 insertions(+), 5 deletions(-)
> create mode 100644 lib/arm/asm/gic-v3-its.h
> create mode 100644 lib/arm/gic-v3-its.c
> create mode 100644 lib/arm64/asm/gic-v3-its.h
>
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 6d3dc2c..2571ffb 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -19,6 +19,7 @@ endef
> cstart.o = $(TEST_DIR)/cstart64.o
> cflatobjs += lib/arm64/processor.o
> cflatobjs += lib/arm64/spinlock.o
> +cflatobjs += lib/arm/gic-v3-its.o
If gic-v3-its.c will never be compiled for arm, then it should
probably go lib/arm64, not lib/arm. Same comment for all other
new source and header files. The only problem with that is...
>
> OBJDIRS += lib/arm64
>
> diff --git a/arm/gic.c b/arm/gic.c
> index abf08c7..4d7dd03 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -16,6 +16,7 @@
> #include <asm/processor.h>
> #include <asm/delay.h>
> #include <asm/gic.h>
> +#include <asm/gic-v3-its.h>
...here where we'd have to do
#if defined(__aarch64__)
#include <asm/gic-v3-its.h>
#endif
which is ugly. However that can be avoided by adding an
empty lib/arm/asm/gic-v3-its.h file.
> #include <asm/smp.h>
> #include <asm/barrier.h>
> #include <asm/io.h>
> @@ -518,6 +519,50 @@ static void gic_test_mmio(void)
> test_targets(nr_irqs);
> }
>
> +#if defined(__arm__)
> +
> +static void test_its_introspection(void) {}
static void test_its_introspection(void)
{
report_abort(...);
}
> +
> +#else /* __arch64__ */
> +
> +static void test_its_introspection(void)
> +{
> + struct its_baser *dev_baser, *coll_baser;
> + struct its_typer *typer = &its_data.typer;
> +
> + if (!gicv3_its_base()) {
> + report_skip("No ITS, skip ...");
> + return;
> + }
> +
> + /* IIDR */
> + report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
> + "GITS_IIDR is read-only"),
> +
> + /* TYPER */
> + report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
> + "GITS_TYPER is read-only");
> +
> + report(typer->phys_lpi, "ITS supports physical LPIs");
> + report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
> + report_info("ITT entry size = 0x%x", typer->ite_size);
> + report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
> + typer->eventid_bits, typer->deviceid_bits,
> + typer->collid_bits);
> + report(typer->eventid_bits && typer->deviceid_bits &&
> + typer->collid_bits, "ID spaces");
> + report_info("Target address format %s",
> + typer->pta ? "Redist basse address" : "PE #");
> +
> + dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
> + report(dev_baser && coll_baser, "detect device and collection BASER");
> + report_info("device baser entry_size = 0x%x", dev_baser->esz);
> + report_info("collection baser entry_size = 0x%x", coll_baser->esz);
> +}
> +
> +#endif
> +
> int main(int argc, char **argv)
> {
> if (!gic_init()) {
> @@ -549,6 +594,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> gic_test_mmio();
> report_prefix_pop();
> + } else if (strcmp(argv[1], "its-introspection") == 0) {
> + report_prefix_push(argv[1]);
> + test_its_introspection();
> + report_prefix_pop();
> } else {
> report_abort("Unknown subtest '%s'", argv[1]);
> }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index daeb5a0..ba2b31b 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -122,6 +122,13 @@ smp = $MAX_SMP
> extra_params = -machine gic-version=3 -append 'active'
> groups = gic
>
> +[its-introspection]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'its-introspection'
> +groups = its
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..815c515
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -0,0 +1,103 @@
> +/*
> + * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_ITS_H_
> +#define _ASMARM_GIC_V3_ITS_H_
> +
> +#ifndef __ASSEMBLY__
Doesn't look like you use the #else /* __ASSEMBLY__ */ side of this.
I'd leave out the #ifndef until we add defines we need to access
from assembler.
> +
> +struct its_typer {
> + unsigned int ite_size;
> + unsigned int eventid_bits;
> + unsigned int deviceid_bits;
> + unsigned int collid_bits;
> + bool pta;
> + bool phys_lpi;
> + bool virt_lpi;
> +};
> +
> +struct its_baser {
> + int type;
> + size_t psz;
> + int nr_pages;
> + bool indirect;
> + phys_addr_t table_addr;
> + bool valid;
> + int esz;
> +};
> +
> +#define GITS_BASER_NR_REGS 8
> +
> +struct its_data {
> + void *base;
> + struct its_typer typer;
> + struct its_baser baser[GITS_BASER_NR_REGS];
> +};
> +
> +extern struct its_data its_data;
> +
> +#define gicv3_its_base() (its_data.base)
> +
> +#if defined(__aarch64__)
> +
> +#define GITS_CTLR 0x0000
> +#define GITS_IIDR 0x0004
> +#define GITS_TYPER 0x0008
> +#define GITS_CBASER 0x0080
> +#define GITS_CWRITER 0x0088
> +#define GITS_CREADR 0x0090
> +#define GITS_BASER 0x0100
> +
> +#define GITS_TYPER_PLPIS BIT(0)
> +#define GITS_TYPER_VLPIS BIT(1)
> +#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
> +#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
> +#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
> +#define GITS_TYPER_IDBITS_SHIFT 8
> +#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
> +#define GITS_TYPER_DEVBITS_SHIFT 13
> +#define GITS_TYPER_PTA BIT(19)
> +#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
> +#define GITS_TYPER_CIDBITS_SHIFT 32
> +#define GITS_TYPER_CIL BIT(36)
> +
> +#define GITS_CTLR_ENABLE (1U << 0)
> +
> +#define GITS_CBASER_VALID (1UL << 63)
> +
> +#define GITS_BASER_VALID BIT(63)
> +#define GITS_BASER_INDIRECT BIT(62)
> +#define GITS_BASER_TYPE_SHIFT (56)
> +#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
> +#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
> +#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
> +#define GITS_BASER_PAGE_SIZE_SHIFT (8)
> +#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGES_MAX 256
> +#define GITS_BASER_PAGES_SHIFT (0)
> +#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
> +#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
> +#define GITS_BASER_TYPE_NONE 0
> +#define GITS_BASER_TYPE_DEVICE 1
> +#define GITS_BASER_TYPE_COLLECTION 4
> +
> +extern void its_parse_typer(void);
> +extern void its_init(void);
> +extern int its_parse_baser(int i, struct its_baser *baser);
> +extern struct its_baser *its_lookup_baser(int type);
> +
> +#else /* __arm__ */
> +
> +static inline void its_init(void) {}
Looks like the empty gic-v3-its.h I suggested creating above will actually
be useful. We can add stubs like this in it.
> +
> +#endif
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_ITS_H_ */
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> new file mode 100644
> index 0000000..2c0ce13
> --- /dev/null
> +++ b/lib/arm/gic-v3-its.c
> @@ -0,0 +1,88 @@
> +/*
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <asm/gic.h>
> +#include <alloc_page.h>
> +#include <asm/gic-v3-its.h>
> +
> +void its_parse_typer(void)
> +{
> + u64 typer = readq(gicv3_its_base() + GITS_TYPER);
> +
> + its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
> + GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
> + its_data.typer.pta = typer & GITS_TYPER_PTA;
> + its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
> + GITS_TYPER_IDBITS_SHIFT) + 1;
> + its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
> + GITS_TYPER_DEVBITS_SHIFT) + 1;
> +
> + if (typer & GITS_TYPER_CIL)
> + its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
> + GITS_TYPER_CIDBITS_SHIFT) + 1;
nit: please consider aligning like this
((typer & MASK) >>
SHIFT) + 1;
Or, maybe better to macro it
#define TYPER_FIELD(typer, mask, shift) (((type) & (mask) >> (shift)) + 1)
And, rather than have a bunch of 'its_data.typer's we could use an alias,
helping us stay within 120 chars.
struct its_typer *t = &its_data.typer;
t->ite_size = TYPER_FIELD(typer, GITS_TYPER_ITT_ENTRY_SIZE,
GITS_TYPER_ITT_ENTRY_SIZE_SHIFT);
> + else
> + its_data.typer.collid_bits = 16;
> +
> + its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
> + its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
> +}
> +
> +int its_parse_baser(int i, struct its_baser *baser)
> +{
> + void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
> + u64 val = readq(reg_addr);
> +
> + if (!val) {
> + memset(baser, 0, sizeof(*baser));
> + return -1;
> + }
> +
> + baser->valid = val & GITS_BASER_VALID;
> + baser->indirect = val & GITS_BASER_INDIRECT;
> + baser->type = GITS_BASER_TYPE(val);
> + baser->esz = GITS_BASER_ENTRY_SIZE(val);
> + baser->nr_pages = GITS_BASER_NR_PAGES(val);
> + baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
> + switch (val & GITS_BASER_PAGE_SIZE_MASK) {
> + case GITS_BASER_PAGE_SIZE_4K:
> + baser->psz = SZ_4K;
> + break;
> + case GITS_BASER_PAGE_SIZE_16K:
> + baser->psz = SZ_16K;
> + break;
> + case GITS_BASER_PAGE_SIZE_64K:
> + baser->psz = SZ_64K;
> + break;
> + default:
> + baser->psz = SZ_64K;
> + }
> + return 0;
> +}
> +
> +struct its_baser *its_lookup_baser(int type)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
> + struct its_baser *baser = &its_data.baser[i];
> +
> + if (baser->type == type)
> + return baser;
> + }
> + return NULL;
> +}
> +
> +void its_init(void)
> +{
> + int i;
> +
> + if (!its_data.base)
> + return;
> +
> + its_parse_typer();
> + for (i = 0; i < GITS_BASER_NR_REGS; i++)
> + its_parse_baser(i, &its_data.baser[i]);
> +}
> +
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index aa9cb86..6b70b05 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -6,9 +6,11 @@
> #include <devicetree.h>
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <asm/gic-v3-its.h>
>
> struct gicv2_data gicv2_data;
> struct gicv3_data gicv3_data;
> +struct its_data its_data;
>
> struct gic_common_ops {
> void (*enable_defaults)(void);
> @@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
> * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> */
> static bool
> -gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
> {
> struct dt_pbus_reg reg;
> - struct dt_device gic;
> + struct dt_device gic, its;
> struct dt_bus bus;
> - int node, ret, i;
> + int node, subnode, ret, i, len;
> + const void *fdt = dt_fdt();
>
> dt_bus_init_defaults(&bus);
> dt_device_init(&gic, &bus, NULL);
> @@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> base2[i] = ioremap(reg.addr, reg.size);
> }
>
> + if (base3 && !strcmp(compatible, "arm,gic-v3")) {
If base != NULL, then we could assert(strcmp(compatible, "arm,cortex-a15-gic") != 0)
> + dt_for_each_subnode(node, subnode) {
> + const struct fdt_property *prop;
> +
> + prop = fdt_get_property(fdt, subnode, "compatible", &len);
> + if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
> + dt_device_bind_node(&its, subnode);
> + ret = dt_pbus_translate(&its, 0, ®);
> + assert(ret == 0);
> + *base3 = ioremap(reg.addr, reg.size);
> + break;
> + }
> + }
> +
> + }
> +
> return true;
> }
>
> int gicv2_init(void)
> {
> return gic_get_dt_bases("arm,cortex-a15-gic",
> - &gicv2_data.dist_base, &gicv2_data.cpu_base);
> + &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
> }
>
> int gicv3_init(void)
> {
> return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> - &gicv3_data.redist_bases[0]);
> + &gicv3_data.redist_bases[0], &its_data.base);
> }
>
> int gic_version(void)
> @@ -104,6 +123,7 @@ int gic_init(void)
> gic_common_ops = &gicv2_common_ops;
> else if (gicv3_init())
> gic_common_ops = &gicv3_common_ops;
> + its_init();
> return gic_version();
> }
>
> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..083cba4
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3-its.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3-its.h"
> --
> 2.20.1
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
@ 2020-02-07 10:19 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 10:19 UTC (permalink / raw)
To: Eric Auger
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:50AM +0100, Eric Auger wrote:
> Detect the presence of an ITS as part of the GICv3 init
> routine, initialize its base address and read few registers
> the IIDR, the TYPER to store its dimensioning parameters.
> Also parse the BASER registers.
>
> This is our first ITS test, belonging to a new "its" group.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - updated dates and changed author
> - squash "arm/arm64: ITS: Test BASER" into this patch but
> removes setup_baser which will be introduced later.
> - only compile on aarch64
> - restrict the new test to aarch64
>
> v1 -> v2:
> - clean GITS_TYPER macros and unused fields in typer struct
> - remove memory attribute related macros
> - remove everything related to memory attributes
> - s/dev_baser/coll_baser/ in report_info
> - add extra line
> - removed index filed in its_baser
> ---
> arm/Makefile.arm64 | 1 +
> arm/gic.c | 49 ++++++++++++++++++
> arm/unittests.cfg | 7 +++
> lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
> lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
> lib/arm/gic.c | 30 +++++++++--
> lib/arm64/asm/gic-v3-its.h | 1 +
> 7 files changed, 274 insertions(+), 5 deletions(-)
> create mode 100644 lib/arm/asm/gic-v3-its.h
> create mode 100644 lib/arm/gic-v3-its.c
> create mode 100644 lib/arm64/asm/gic-v3-its.h
>
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 6d3dc2c..2571ffb 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -19,6 +19,7 @@ endef
> cstart.o = $(TEST_DIR)/cstart64.o
> cflatobjs += lib/arm64/processor.o
> cflatobjs += lib/arm64/spinlock.o
> +cflatobjs += lib/arm/gic-v3-its.o
If gic-v3-its.c will never be compiled for arm, then it should
probably go lib/arm64, not lib/arm. Same comment for all other
new source and header files. The only problem with that is...
>
> OBJDIRS += lib/arm64
>
> diff --git a/arm/gic.c b/arm/gic.c
> index abf08c7..4d7dd03 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -16,6 +16,7 @@
> #include <asm/processor.h>
> #include <asm/delay.h>
> #include <asm/gic.h>
> +#include <asm/gic-v3-its.h>
...here where we'd have to do
#if defined(__aarch64__)
#include <asm/gic-v3-its.h>
#endif
which is ugly. However that can be avoided by adding an
empty lib/arm/asm/gic-v3-its.h file.
> #include <asm/smp.h>
> #include <asm/barrier.h>
> #include <asm/io.h>
> @@ -518,6 +519,50 @@ static void gic_test_mmio(void)
> test_targets(nr_irqs);
> }
>
> +#if defined(__arm__)
> +
> +static void test_its_introspection(void) {}
static void test_its_introspection(void)
{
report_abort(...);
}
> +
> +#else /* __arch64__ */
> +
> +static void test_its_introspection(void)
> +{
> + struct its_baser *dev_baser, *coll_baser;
> + struct its_typer *typer = &its_data.typer;
> +
> + if (!gicv3_its_base()) {
> + report_skip("No ITS, skip ...");
> + return;
> + }
> +
> + /* IIDR */
> + report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
> + "GITS_IIDR is read-only"),
> +
> + /* TYPER */
> + report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
> + "GITS_TYPER is read-only");
> +
> + report(typer->phys_lpi, "ITS supports physical LPIs");
> + report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
> + report_info("ITT entry size = 0x%x", typer->ite_size);
> + report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
> + typer->eventid_bits, typer->deviceid_bits,
> + typer->collid_bits);
> + report(typer->eventid_bits && typer->deviceid_bits &&
> + typer->collid_bits, "ID spaces");
> + report_info("Target address format %s",
> + typer->pta ? "Redist basse address" : "PE #");
> +
> + dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
> + report(dev_baser && coll_baser, "detect device and collection BASER");
> + report_info("device baser entry_size = 0x%x", dev_baser->esz);
> + report_info("collection baser entry_size = 0x%x", coll_baser->esz);
> +}
> +
> +#endif
> +
> int main(int argc, char **argv)
> {
> if (!gic_init()) {
> @@ -549,6 +594,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> gic_test_mmio();
> report_prefix_pop();
> + } else if (strcmp(argv[1], "its-introspection") == 0) {
> + report_prefix_push(argv[1]);
> + test_its_introspection();
> + report_prefix_pop();
> } else {
> report_abort("Unknown subtest '%s'", argv[1]);
> }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index daeb5a0..ba2b31b 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -122,6 +122,13 @@ smp = $MAX_SMP
> extra_params = -machine gic-version=3 -append 'active'
> groups = gic
>
> +[its-introspection]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'its-introspection'
> +groups = its
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..815c515
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -0,0 +1,103 @@
> +/*
> + * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_ITS_H_
> +#define _ASMARM_GIC_V3_ITS_H_
> +
> +#ifndef __ASSEMBLY__
Doesn't look like you use the #else /* __ASSEMBLY__ */ side of this.
I'd leave out the #ifndef until we add defines we need to access
from assembler.
> +
> +struct its_typer {
> + unsigned int ite_size;
> + unsigned int eventid_bits;
> + unsigned int deviceid_bits;
> + unsigned int collid_bits;
> + bool pta;
> + bool phys_lpi;
> + bool virt_lpi;
> +};
> +
> +struct its_baser {
> + int type;
> + size_t psz;
> + int nr_pages;
> + bool indirect;
> + phys_addr_t table_addr;
> + bool valid;
> + int esz;
> +};
> +
> +#define GITS_BASER_NR_REGS 8
> +
> +struct its_data {
> + void *base;
> + struct its_typer typer;
> + struct its_baser baser[GITS_BASER_NR_REGS];
> +};
> +
> +extern struct its_data its_data;
> +
> +#define gicv3_its_base() (its_data.base)
> +
> +#if defined(__aarch64__)
> +
> +#define GITS_CTLR 0x0000
> +#define GITS_IIDR 0x0004
> +#define GITS_TYPER 0x0008
> +#define GITS_CBASER 0x0080
> +#define GITS_CWRITER 0x0088
> +#define GITS_CREADR 0x0090
> +#define GITS_BASER 0x0100
> +
> +#define GITS_TYPER_PLPIS BIT(0)
> +#define GITS_TYPER_VLPIS BIT(1)
> +#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
> +#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
> +#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
> +#define GITS_TYPER_IDBITS_SHIFT 8
> +#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
> +#define GITS_TYPER_DEVBITS_SHIFT 13
> +#define GITS_TYPER_PTA BIT(19)
> +#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
> +#define GITS_TYPER_CIDBITS_SHIFT 32
> +#define GITS_TYPER_CIL BIT(36)
> +
> +#define GITS_CTLR_ENABLE (1U << 0)
> +
> +#define GITS_CBASER_VALID (1UL << 63)
> +
> +#define GITS_BASER_VALID BIT(63)
> +#define GITS_BASER_INDIRECT BIT(62)
> +#define GITS_BASER_TYPE_SHIFT (56)
> +#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
> +#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
> +#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
> +#define GITS_BASER_PAGE_SIZE_SHIFT (8)
> +#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
> +#define GITS_BASER_PAGES_MAX 256
> +#define GITS_BASER_PAGES_SHIFT (0)
> +#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
> +#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
> +#define GITS_BASER_TYPE_NONE 0
> +#define GITS_BASER_TYPE_DEVICE 1
> +#define GITS_BASER_TYPE_COLLECTION 4
> +
> +extern void its_parse_typer(void);
> +extern void its_init(void);
> +extern int its_parse_baser(int i, struct its_baser *baser);
> +extern struct its_baser *its_lookup_baser(int type);
> +
> +#else /* __arm__ */
> +
> +static inline void its_init(void) {}
Looks like the empty gic-v3-its.h I suggested creating above will actually
be useful. We can add stubs like this in it.
> +
> +#endif
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V3_ITS_H_ */
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> new file mode 100644
> index 0000000..2c0ce13
> --- /dev/null
> +++ b/lib/arm/gic-v3-its.c
> @@ -0,0 +1,88 @@
> +/*
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <asm/gic.h>
> +#include <alloc_page.h>
> +#include <asm/gic-v3-its.h>
> +
> +void its_parse_typer(void)
> +{
> + u64 typer = readq(gicv3_its_base() + GITS_TYPER);
> +
> + its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
> + GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
> + its_data.typer.pta = typer & GITS_TYPER_PTA;
> + its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
> + GITS_TYPER_IDBITS_SHIFT) + 1;
> + its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
> + GITS_TYPER_DEVBITS_SHIFT) + 1;
> +
> + if (typer & GITS_TYPER_CIL)
> + its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
> + GITS_TYPER_CIDBITS_SHIFT) + 1;
nit: please consider aligning like this
((typer & MASK) >>
SHIFT) + 1;
Or, maybe better to macro it
#define TYPER_FIELD(typer, mask, shift) (((type) & (mask) >> (shift)) + 1)
And, rather than have a bunch of 'its_data.typer's we could use an alias,
helping us stay within 120 chars.
struct its_typer *t = &its_data.typer;
t->ite_size = TYPER_FIELD(typer, GITS_TYPER_ITT_ENTRY_SIZE,
GITS_TYPER_ITT_ENTRY_SIZE_SHIFT);
> + else
> + its_data.typer.collid_bits = 16;
> +
> + its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
> + its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
> +}
> +
> +int its_parse_baser(int i, struct its_baser *baser)
> +{
> + void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
> + u64 val = readq(reg_addr);
> +
> + if (!val) {
> + memset(baser, 0, sizeof(*baser));
> + return -1;
> + }
> +
> + baser->valid = val & GITS_BASER_VALID;
> + baser->indirect = val & GITS_BASER_INDIRECT;
> + baser->type = GITS_BASER_TYPE(val);
> + baser->esz = GITS_BASER_ENTRY_SIZE(val);
> + baser->nr_pages = GITS_BASER_NR_PAGES(val);
> + baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
> + switch (val & GITS_BASER_PAGE_SIZE_MASK) {
> + case GITS_BASER_PAGE_SIZE_4K:
> + baser->psz = SZ_4K;
> + break;
> + case GITS_BASER_PAGE_SIZE_16K:
> + baser->psz = SZ_16K;
> + break;
> + case GITS_BASER_PAGE_SIZE_64K:
> + baser->psz = SZ_64K;
> + break;
> + default:
> + baser->psz = SZ_64K;
> + }
> + return 0;
> +}
> +
> +struct its_baser *its_lookup_baser(int type)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
> + struct its_baser *baser = &its_data.baser[i];
> +
> + if (baser->type == type)
> + return baser;
> + }
> + return NULL;
> +}
> +
> +void its_init(void)
> +{
> + int i;
> +
> + if (!its_data.base)
> + return;
> +
> + its_parse_typer();
> + for (i = 0; i < GITS_BASER_NR_REGS; i++)
> + its_parse_baser(i, &its_data.baser[i]);
> +}
> +
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index aa9cb86..6b70b05 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -6,9 +6,11 @@
> #include <devicetree.h>
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <asm/gic-v3-its.h>
>
> struct gicv2_data gicv2_data;
> struct gicv3_data gicv3_data;
> +struct its_data its_data;
>
> struct gic_common_ops {
> void (*enable_defaults)(void);
> @@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
> * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> */
> static bool
> -gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> +gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
> {
> struct dt_pbus_reg reg;
> - struct dt_device gic;
> + struct dt_device gic, its;
> struct dt_bus bus;
> - int node, ret, i;
> + int node, subnode, ret, i, len;
> + const void *fdt = dt_fdt();
>
> dt_bus_init_defaults(&bus);
> dt_device_init(&gic, &bus, NULL);
> @@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
> base2[i] = ioremap(reg.addr, reg.size);
> }
>
> + if (base3 && !strcmp(compatible, "arm,gic-v3")) {
If base != NULL, then we could assert(strcmp(compatible, "arm,cortex-a15-gic") != 0)
> + dt_for_each_subnode(node, subnode) {
> + const struct fdt_property *prop;
> +
> + prop = fdt_get_property(fdt, subnode, "compatible", &len);
> + if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
> + dt_device_bind_node(&its, subnode);
> + ret = dt_pbus_translate(&its, 0, ®);
> + assert(ret == 0);
> + *base3 = ioremap(reg.addr, reg.size);
> + break;
> + }
> + }
> +
> + }
> +
> return true;
> }
>
> int gicv2_init(void)
> {
> return gic_get_dt_bases("arm,cortex-a15-gic",
> - &gicv2_data.dist_base, &gicv2_data.cpu_base);
> + &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
> }
>
> int gicv3_init(void)
> {
> return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
> - &gicv3_data.redist_bases[0]);
> + &gicv3_data.redist_bases[0], &its_data.base);
> }
>
> int gic_version(void)
> @@ -104,6 +123,7 @@ int gic_init(void)
> gic_common_ops = &gicv2_common_ops;
> else if (gicv3_init())
> gic_common_ops = &gicv3_common_ops;
> + its_init();
> return gic_version();
> }
>
> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
> new file mode 100644
> index 0000000..083cba4
> --- /dev/null
> +++ b/lib/arm64/asm/gic-v3-its.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/gic-v3-its.h"
> --
> 2.20.1
>
Thanks,
drew
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 12:11 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:11 UTC (permalink / raw)
To: Eric Auger
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
peter.maydell, andre.przywara, thuth, yuzenghui,
alexandru.elisei
On Tue, Jan 28, 2020 at 11:34:51AM +0100, Eric Auger wrote:
> Allocate the LPI configuration and per re-distributor pending table.
> Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
> by default in the config table.
>
> Also introduce a helper routine that allows to set the pending table
> bit for a given LPI.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
> and add _lpi prefix too
>
> v1 -> v2:
> - remove memory attributes
> ---
> lib/arm/asm/gic-v3.h | 16 +++++++++++
> lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 80 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index ffb2e26..ec2a6f0 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -48,6 +48,16 @@
> #define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
>
> +#define GICR_PROPBASER_IDBITS_MASK (0x1f)
> +
> +#define GICR_PENDBASER_PTZ BIT_ULL(62)
Strange indent here, too far and not tabs
> +
> +#define LPI_PROP_GROUP1 (1 << 1)
> +#define LPI_PROP_ENABLED (1 << 0)
> +#define LPI_PROP_DEFAULT_PRIO 0xa0
> +#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
> + LPI_PROP_ENABLED)
> +
> #include <asm/arch_gicv3.h>
>
> #ifndef __ASSEMBLY__
> @@ -64,6 +74,8 @@ struct gicv3_data {
> void *dist_base;
> void *redist_bases[GICV3_NR_REDISTS];
> void *redist_base[NR_CPUS];
> + void *lpi_prop;
> + void *lpi_pend[NR_CPUS];
> unsigned int irq_nr;
> };
> extern struct gicv3_data gicv3_data;
> @@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
> extern void gicv3_ipi_send_single(int irq, int cpu);
> extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
> extern void gicv3_set_redist_base(size_t stride);
> +extern void gicv3_lpi_set_config(int n, u8 val);
> +extern u8 gicv3_lpi_get_config(int n);
> +extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> +extern void gicv3_lpi_alloc_tables(void);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> index feecb5e..c33f883 100644
> --- a/lib/arm/gic-v3.c
> +++ b/lib/arm/gic-v3.c
> @@ -5,6 +5,7 @@
> */
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <alloc_page.h>
>
> void gicv3_set_redist_base(size_t stride)
> {
> @@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
> cpumask_set_cpu(cpu, &dest);
> gicv3_ipi_send_mask(irq, &dest);
> }
> +
> +#if defined(__aarch64__)
> +/* alloc_lpi_tables: Allocate LPI config and pending tables */
> +void gicv3_lpi_alloc_tables(void)
> +{
> + unsigned long n = SZ_64K >> PAGE_SHIFT;
> + unsigned long order = fls(n);
> + u64 prop_val;
> + int cpu;
> +
> + gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
> +
> + /* ID bits = 13, ie. up to 14b LPI INTID */
> + prop_val = (u64)gicv3_data.lpi_prop | 13;
> +
> + /*
> + * Allocate pending tables for each redistributor
> + * and set PROPBASER and PENDBASER
> + */
> + for_each_present_cpu(cpu) {
> + u64 pend_val;
> + void *ptr;
> +
> + ptr = gicv3_data.redist_base[cpu];
> +
> + writeq(prop_val, ptr + GICR_PROPBASER);
> +
> + gicv3_data.lpi_pend[cpu] = (void *)virt_to_phys(alloc_pages(order));
> +
> + pend_val = (u64)gicv3_data.lpi_pend[cpu];
> +
> + writeq(pend_val, ptr + GICR_PENDBASER);
nit: probably don't need all the blank lines
> + }
> +}
> +
> +void gicv3_lpi_set_config(int n, u8 value)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
> +
> + *entry = value;
> +}
> +
> +u8 gicv3_lpi_get_config(int n)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
> +
> + return *entry;
> +}
We probably need a define for the 8192. LPI_ID_BASE? And, like PPI and SPI
maybe we need an LPI(id)? '#define LPI(id) ((id) + LPI_ID_BASE)'?
Also, is lpi_prop an array of u8's? If so, then I'd think we could
declare it as a u8 * in gicv3_data. Then, instead of the above two
functions just do something like
#define gicv3_lpi_set_config(n, val) ({
gicv3_data.lpi_prop[n] = (val);
})
#define gicv3_lpi_get_config(n) (gicv3_data.lpi_prop[n])
> +
> +void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
These types of functions that can be used both as a setter and clearer
usually have a name like foo_set_clr_bar(). That would make this long
name a bit longer, but do we need the '_table_bit' part of the name?
> +{
> + u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
If lpi_pend is a physical address then why not declare it like that
in gicv3_data, i.e. phys_addr_t lpi_pend[NR_CPUS] ?
> + u8 mask = 1 << (n % 8), byte;
> +
> + ptr += (n / 8);
> + byte = *ptr;
And Zenghui already pointed out that this doesn't look right for
a physical address.
> + if (set)
> + byte |= mask;
> + else
> + byte &= ~mask;
> + *ptr = byte;
> +}
> +#endif /* __aarch64__ */
> --
> 2.20.1
>
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
@ 2020-02-07 12:11 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:11 UTC (permalink / raw)
To: Eric Auger
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:51AM +0100, Eric Auger wrote:
> Allocate the LPI configuration and per re-distributor pending table.
> Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
> by default in the config table.
>
> Also introduce a helper routine that allows to set the pending table
> bit for a given LPI.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
> and add _lpi prefix too
>
> v1 -> v2:
> - remove memory attributes
> ---
> lib/arm/asm/gic-v3.h | 16 +++++++++++
> lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 80 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index ffb2e26..ec2a6f0 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -48,6 +48,16 @@
> #define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
>
> +#define GICR_PROPBASER_IDBITS_MASK (0x1f)
> +
> +#define GICR_PENDBASER_PTZ BIT_ULL(62)
Strange indent here, too far and not tabs
> +
> +#define LPI_PROP_GROUP1 (1 << 1)
> +#define LPI_PROP_ENABLED (1 << 0)
> +#define LPI_PROP_DEFAULT_PRIO 0xa0
> +#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
> + LPI_PROP_ENABLED)
> +
> #include <asm/arch_gicv3.h>
>
> #ifndef __ASSEMBLY__
> @@ -64,6 +74,8 @@ struct gicv3_data {
> void *dist_base;
> void *redist_bases[GICV3_NR_REDISTS];
> void *redist_base[NR_CPUS];
> + void *lpi_prop;
> + void *lpi_pend[NR_CPUS];
> unsigned int irq_nr;
> };
> extern struct gicv3_data gicv3_data;
> @@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
> extern void gicv3_ipi_send_single(int irq, int cpu);
> extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
> extern void gicv3_set_redist_base(size_t stride);
> +extern void gicv3_lpi_set_config(int n, u8 val);
> +extern u8 gicv3_lpi_get_config(int n);
> +extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> +extern void gicv3_lpi_alloc_tables(void);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> index feecb5e..c33f883 100644
> --- a/lib/arm/gic-v3.c
> +++ b/lib/arm/gic-v3.c
> @@ -5,6 +5,7 @@
> */
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <alloc_page.h>
>
> void gicv3_set_redist_base(size_t stride)
> {
> @@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
> cpumask_set_cpu(cpu, &dest);
> gicv3_ipi_send_mask(irq, &dest);
> }
> +
> +#if defined(__aarch64__)
> +/* alloc_lpi_tables: Allocate LPI config and pending tables */
> +void gicv3_lpi_alloc_tables(void)
> +{
> + unsigned long n = SZ_64K >> PAGE_SHIFT;
> + unsigned long order = fls(n);
> + u64 prop_val;
> + int cpu;
> +
> + gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
> +
> + /* ID bits = 13, ie. up to 14b LPI INTID */
> + prop_val = (u64)gicv3_data.lpi_prop | 13;
> +
> + /*
> + * Allocate pending tables for each redistributor
> + * and set PROPBASER and PENDBASER
> + */
> + for_each_present_cpu(cpu) {
> + u64 pend_val;
> + void *ptr;
> +
> + ptr = gicv3_data.redist_base[cpu];
> +
> + writeq(prop_val, ptr + GICR_PROPBASER);
> +
> + gicv3_data.lpi_pend[cpu] = (void *)virt_to_phys(alloc_pages(order));
> +
> + pend_val = (u64)gicv3_data.lpi_pend[cpu];
> +
> + writeq(pend_val, ptr + GICR_PENDBASER);
nit: probably don't need all the blank lines
> + }
> +}
> +
> +void gicv3_lpi_set_config(int n, u8 value)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
> +
> + *entry = value;
> +}
> +
> +u8 gicv3_lpi_get_config(int n)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
> +
> + return *entry;
> +}
We probably need a define for the 8192. LPI_ID_BASE? And, like PPI and SPI
maybe we need an LPI(id)? '#define LPI(id) ((id) + LPI_ID_BASE)'?
Also, is lpi_prop an array of u8's? If so, then I'd think we could
declare it as a u8 * in gicv3_data. Then, instead of the above two
functions just do something like
#define gicv3_lpi_set_config(n, val) ({
gicv3_data.lpi_prop[n] = (val);
})
#define gicv3_lpi_get_config(n) (gicv3_data.lpi_prop[n])
> +
> +void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
These types of functions that can be used both as a setter and clearer
usually have a name like foo_set_clr_bar(). That would make this long
name a bit longer, but do we need the '_table_bit' part of the name?
> +{
> + u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
If lpi_pend is a physical address then why not declare it like that
in gicv3_data, i.e. phys_addr_t lpi_pend[NR_CPUS] ?
> + u8 mask = 1 << (n % 8), byte;
> +
> + ptr += (n / 8);
> + byte = *ptr;
And Zenghui already pointed out that this doesn't look right for
a physical address.
> + if (set)
> + byte |= mask;
> + else
> + byte &= ~mask;
> + *ptr = byte;
> +}
> +#endif /* __aarch64__ */
> --
> 2.20.1
>
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
@ 2020-02-07 12:11 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:11 UTC (permalink / raw)
To: Eric Auger
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:51AM +0100, Eric Auger wrote:
> Allocate the LPI configuration and per re-distributor pending table.
> Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
> by default in the config table.
>
> Also introduce a helper routine that allows to set the pending table
> bit for a given LPI.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
> and add _lpi prefix too
>
> v1 -> v2:
> - remove memory attributes
> ---
> lib/arm/asm/gic-v3.h | 16 +++++++++++
> lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 80 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index ffb2e26..ec2a6f0 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -48,6 +48,16 @@
> #define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
> (MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
>
> +#define GICR_PROPBASER_IDBITS_MASK (0x1f)
> +
> +#define GICR_PENDBASER_PTZ BIT_ULL(62)
Strange indent here, too far and not tabs
> +
> +#define LPI_PROP_GROUP1 (1 << 1)
> +#define LPI_PROP_ENABLED (1 << 0)
> +#define LPI_PROP_DEFAULT_PRIO 0xa0
> +#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
> + LPI_PROP_ENABLED)
> +
> #include <asm/arch_gicv3.h>
>
> #ifndef __ASSEMBLY__
> @@ -64,6 +74,8 @@ struct gicv3_data {
> void *dist_base;
> void *redist_bases[GICV3_NR_REDISTS];
> void *redist_base[NR_CPUS];
> + void *lpi_prop;
> + void *lpi_pend[NR_CPUS];
> unsigned int irq_nr;
> };
> extern struct gicv3_data gicv3_data;
> @@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
> extern void gicv3_ipi_send_single(int irq, int cpu);
> extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
> extern void gicv3_set_redist_base(size_t stride);
> +extern void gicv3_lpi_set_config(int n, u8 val);
> +extern u8 gicv3_lpi_get_config(int n);
> +extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> +extern void gicv3_lpi_alloc_tables(void);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> index feecb5e..c33f883 100644
> --- a/lib/arm/gic-v3.c
> +++ b/lib/arm/gic-v3.c
> @@ -5,6 +5,7 @@
> */
> #include <asm/gic.h>
> #include <asm/io.h>
> +#include <alloc_page.h>
>
> void gicv3_set_redist_base(size_t stride)
> {
> @@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
> cpumask_set_cpu(cpu, &dest);
> gicv3_ipi_send_mask(irq, &dest);
> }
> +
> +#if defined(__aarch64__)
> +/* alloc_lpi_tables: Allocate LPI config and pending tables */
> +void gicv3_lpi_alloc_tables(void)
> +{
> + unsigned long n = SZ_64K >> PAGE_SHIFT;
> + unsigned long order = fls(n);
> + u64 prop_val;
> + int cpu;
> +
> + gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
> +
> + /* ID bits = 13, ie. up to 14b LPI INTID */
> + prop_val = (u64)gicv3_data.lpi_prop | 13;
> +
> + /*
> + * Allocate pending tables for each redistributor
> + * and set PROPBASER and PENDBASER
> + */
> + for_each_present_cpu(cpu) {
> + u64 pend_val;
> + void *ptr;
> +
> + ptr = gicv3_data.redist_base[cpu];
> +
> + writeq(prop_val, ptr + GICR_PROPBASER);
> +
> + gicv3_data.lpi_pend[cpu] = (void *)virt_to_phys(alloc_pages(order));
> +
> + pend_val = (u64)gicv3_data.lpi_pend[cpu];
> +
> + writeq(pend_val, ptr + GICR_PENDBASER);
nit: probably don't need all the blank lines
> + }
> +}
> +
> +void gicv3_lpi_set_config(int n, u8 value)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
> +
> + *entry = value;
> +}
> +
> +u8 gicv3_lpi_get_config(int n)
> +{
> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
> +
> + return *entry;
> +}
We probably need a define for the 8192. LPI_ID_BASE? And, like PPI and SPI
maybe we need an LPI(id)? '#define LPI(id) ((id) + LPI_ID_BASE)'?
Also, is lpi_prop an array of u8's? If so, then I'd think we could
declare it as a u8 * in gicv3_data. Then, instead of the above two
functions just do something like
#define gicv3_lpi_set_config(n, val) ({
gicv3_data.lpi_prop[n] = (val);
})
#define gicv3_lpi_get_config(n) (gicv3_data.lpi_prop[n])
> +
> +void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
These types of functions that can be used both as a setter and clearer
usually have a name like foo_set_clr_bar(). That would make this long
name a bit longer, but do we need the '_table_bit' part of the name?
> +{
> + u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
If lpi_pend is a physical address then why not declare it like that
in gicv3_data, i.e. phys_addr_t lpi_pend[NR_CPUS] ?
> + u8 mask = 1 << (n % 8), byte;
> +
> + ptr += (n / 8);
> + byte = *ptr;
And Zenghui already pointed out that this doesn't look right for
a physical address.
> + if (set)
> + byte |= mask;
> + else
> + byte &= ~mask;
> + *ptr = byte;
> +}
> +#endif /* __aarch64__ */
> --
> 2.20.1
>
>
Thanks,
drew
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 07/14] arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 12:14 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:14 UTC (permalink / raw)
To: Eric Auger
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
andre.przywara, peter.maydell, yuzenghui, alexandru.elisei,
thuth
On Tue, Jan 28, 2020 at 11:34:52AM +0100, Eric Auger wrote:
> This helper function controls the signaling of LPIs at
> redistributor level.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - move the helper in lib/arm/gic-v3.c
> - rename the function with gicv3_lpi_ prefix
> - s/report_abort/assert
> ---
> lib/arm/asm/gic-v3.h | 1 +
> lib/arm/gic-v3.c | 17 +++++++++++++++++
> 2 files changed, 18 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index ec2a6f0..734c0c0 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -96,6 +96,7 @@ extern void gicv3_lpi_set_config(int n, u8 val);
> extern u8 gicv3_lpi_get_config(int n);
> extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> extern void gicv3_lpi_alloc_tables(void);
> +extern void gicv3_lpi_rdist_ctrl(u32 redist, bool set);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> index c33f883..7865d01 100644
> --- a/lib/arm/gic-v3.c
> +++ b/lib/arm/gic-v3.c
> @@ -210,4 +210,21 @@ void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
> byte &= ~mask;
> *ptr = byte;
> }
> +
> +void gicv3_lpi_rdist_ctrl(u32 redist, bool set)
_set_clr_ ?
> +{
> + void *ptr;
> + u64 val;
> +
> + assert(redist < nr_cpus);
> +
> + ptr = gicv3_data.redist_base[redist];
> + val = readl(ptr + GICR_CTLR);
> + if (set)
> + val |= GICR_CTLR_ENABLE_LPIS;
> + else
> + val &= ~GICR_CTLR_ENABLE_LPIS;
> + writel(val, ptr + GICR_CTLR);
> +}
> #endif /* __aarch64__ */
> +
stray blank line here
> --
> 2.20.1
>
I'm not sure why this needs its own patch. I could just be part of the
next patch.
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 07/14] arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
@ 2020-02-07 12:14 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:14 UTC (permalink / raw)
To: Eric Auger
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:52AM +0100, Eric Auger wrote:
> This helper function controls the signaling of LPIs at
> redistributor level.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - move the helper in lib/arm/gic-v3.c
> - rename the function with gicv3_lpi_ prefix
> - s/report_abort/assert
> ---
> lib/arm/asm/gic-v3.h | 1 +
> lib/arm/gic-v3.c | 17 +++++++++++++++++
> 2 files changed, 18 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index ec2a6f0..734c0c0 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -96,6 +96,7 @@ extern void gicv3_lpi_set_config(int n, u8 val);
> extern u8 gicv3_lpi_get_config(int n);
> extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> extern void gicv3_lpi_alloc_tables(void);
> +extern void gicv3_lpi_rdist_ctrl(u32 redist, bool set);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> index c33f883..7865d01 100644
> --- a/lib/arm/gic-v3.c
> +++ b/lib/arm/gic-v3.c
> @@ -210,4 +210,21 @@ void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
> byte &= ~mask;
> *ptr = byte;
> }
> +
> +void gicv3_lpi_rdist_ctrl(u32 redist, bool set)
_set_clr_ ?
> +{
> + void *ptr;
> + u64 val;
> +
> + assert(redist < nr_cpus);
> +
> + ptr = gicv3_data.redist_base[redist];
> + val = readl(ptr + GICR_CTLR);
> + if (set)
> + val |= GICR_CTLR_ENABLE_LPIS;
> + else
> + val &= ~GICR_CTLR_ENABLE_LPIS;
> + writel(val, ptr + GICR_CTLR);
> +}
> #endif /* __aarch64__ */
> +
stray blank line here
> --
> 2.20.1
>
I'm not sure why this needs its own patch. I could just be part of the
next patch.
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 07/14] arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
@ 2020-02-07 12:14 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:14 UTC (permalink / raw)
To: Eric Auger
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:52AM +0100, Eric Auger wrote:
> This helper function controls the signaling of LPIs at
> redistributor level.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - move the helper in lib/arm/gic-v3.c
> - rename the function with gicv3_lpi_ prefix
> - s/report_abort/assert
> ---
> lib/arm/asm/gic-v3.h | 1 +
> lib/arm/gic-v3.c | 17 +++++++++++++++++
> 2 files changed, 18 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index ec2a6f0..734c0c0 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -96,6 +96,7 @@ extern void gicv3_lpi_set_config(int n, u8 val);
> extern u8 gicv3_lpi_get_config(int n);
> extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> extern void gicv3_lpi_alloc_tables(void);
> +extern void gicv3_lpi_rdist_ctrl(u32 redist, bool set);
>
> static inline void gicv3_do_wait_for_rwp(void *base)
> {
> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> index c33f883..7865d01 100644
> --- a/lib/arm/gic-v3.c
> +++ b/lib/arm/gic-v3.c
> @@ -210,4 +210,21 @@ void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
> byte &= ~mask;
> *ptr = byte;
> }
> +
> +void gicv3_lpi_rdist_ctrl(u32 redist, bool set)
_set_clr_ ?
> +{
> + void *ptr;
> + u64 val;
> +
> + assert(redist < nr_cpus);
> +
> + ptr = gicv3_data.redist_base[redist];
> + val = readl(ptr + GICR_CTLR);
> + if (set)
> + val |= GICR_CTLR_ENABLE_LPIS;
> + else
> + val &= ~GICR_CTLR_ENABLE_LPIS;
> + writel(val, ptr + GICR_CTLR);
> +}
> #endif /* __aarch64__ */
> +
stray blank line here
> --
> 2.20.1
>
I'm not sure why this needs its own patch. I could just be part of the
next patch.
Thanks,
drew
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 07/14] arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
2020-02-07 12:14 ` Andrew Jones
(?)
@ 2020-02-07 12:19 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:19 UTC (permalink / raw)
To: Eric Auger
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
andre.przywara, peter.maydell, yuzenghui, alexandru.elisei,
thuth
On Fri, Feb 07, 2020 at 01:14:37PM +0100, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:52AM +0100, Eric Auger wrote:
> > This helper function controls the signaling of LPIs at
> > redistributor level.
> >
> > Signed-off-by: Eric Auger <eric.auger@redhat.com>
> >
> > ---
> >
> > v2 -> v3:
> > - move the helper in lib/arm/gic-v3.c
> > - rename the function with gicv3_lpi_ prefix
> > - s/report_abort/assert
> > ---
> > lib/arm/asm/gic-v3.h | 1 +
> > lib/arm/gic-v3.c | 17 +++++++++++++++++
> > 2 files changed, 18 insertions(+)
> >
> > diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> > index ec2a6f0..734c0c0 100644
> > --- a/lib/arm/asm/gic-v3.h
> > +++ b/lib/arm/asm/gic-v3.h
> > @@ -96,6 +96,7 @@ extern void gicv3_lpi_set_config(int n, u8 val);
> > extern u8 gicv3_lpi_get_config(int n);
> > extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> > extern void gicv3_lpi_alloc_tables(void);
> > +extern void gicv3_lpi_rdist_ctrl(u32 redist, bool set);
> >
> > static inline void gicv3_do_wait_for_rwp(void *base)
> > {
> > diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> > index c33f883..7865d01 100644
> > --- a/lib/arm/gic-v3.c
> > +++ b/lib/arm/gic-v3.c
> > @@ -210,4 +210,21 @@ void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
> > byte &= ~mask;
> > *ptr = byte;
> > }
> > +
> > +void gicv3_lpi_rdist_ctrl(u32 redist, bool set)
>
> _set_clr_ ?
No, probably not _set_clr_ here. The function could be
static though, with other functions to enable/disable
void gicv3_lpi_rdist_enable(redist) { gicv3_lpi_rdist_ctrl(redist, true); }
void gicv3_lpi_rdist_disable(redist) { gicv3_lpi_rdist_ctrl(redist, false); }
But whatever.
>
> > +{
> > + void *ptr;
> > + u64 val;
> > +
> > + assert(redist < nr_cpus);
> > +
> > + ptr = gicv3_data.redist_base[redist];
> > + val = readl(ptr + GICR_CTLR);
> > + if (set)
> > + val |= GICR_CTLR_ENABLE_LPIS;
> > + else
> > + val &= ~GICR_CTLR_ENABLE_LPIS;
> > + writel(val, ptr + GICR_CTLR);
> > +}
> > #endif /* __aarch64__ */
> > +
>
> stray blank line here
>
> > --
> > 2.20.1
> >
>
> I'm not sure why this needs its own patch. I could just be part of the
> next patch.
>
> Thanks,
> drew
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 07/14] arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
@ 2020-02-07 12:19 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:19 UTC (permalink / raw)
To: Eric Auger
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Fri, Feb 07, 2020 at 01:14:37PM +0100, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:52AM +0100, Eric Auger wrote:
> > This helper function controls the signaling of LPIs at
> > redistributor level.
> >
> > Signed-off-by: Eric Auger <eric.auger@redhat.com>
> >
> > ---
> >
> > v2 -> v3:
> > - move the helper in lib/arm/gic-v3.c
> > - rename the function with gicv3_lpi_ prefix
> > - s/report_abort/assert
> > ---
> > lib/arm/asm/gic-v3.h | 1 +
> > lib/arm/gic-v3.c | 17 +++++++++++++++++
> > 2 files changed, 18 insertions(+)
> >
> > diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> > index ec2a6f0..734c0c0 100644
> > --- a/lib/arm/asm/gic-v3.h
> > +++ b/lib/arm/asm/gic-v3.h
> > @@ -96,6 +96,7 @@ extern void gicv3_lpi_set_config(int n, u8 val);
> > extern u8 gicv3_lpi_get_config(int n);
> > extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> > extern void gicv3_lpi_alloc_tables(void);
> > +extern void gicv3_lpi_rdist_ctrl(u32 redist, bool set);
> >
> > static inline void gicv3_do_wait_for_rwp(void *base)
> > {
> > diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> > index c33f883..7865d01 100644
> > --- a/lib/arm/gic-v3.c
> > +++ b/lib/arm/gic-v3.c
> > @@ -210,4 +210,21 @@ void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
> > byte &= ~mask;
> > *ptr = byte;
> > }
> > +
> > +void gicv3_lpi_rdist_ctrl(u32 redist, bool set)
>
> _set_clr_ ?
No, probably not _set_clr_ here. The function could be
static though, with other functions to enable/disable
void gicv3_lpi_rdist_enable(redist) { gicv3_lpi_rdist_ctrl(redist, true); }
void gicv3_lpi_rdist_disable(redist) { gicv3_lpi_rdist_ctrl(redist, false); }
But whatever.
>
> > +{
> > + void *ptr;
> > + u64 val;
> > +
> > + assert(redist < nr_cpus);
> > +
> > + ptr = gicv3_data.redist_base[redist];
> > + val = readl(ptr + GICR_CTLR);
> > + if (set)
> > + val |= GICR_CTLR_ENABLE_LPIS;
> > + else
> > + val &= ~GICR_CTLR_ENABLE_LPIS;
> > + writel(val, ptr + GICR_CTLR);
> > +}
> > #endif /* __aarch64__ */
> > +
>
> stray blank line here
>
> > --
> > 2.20.1
> >
>
> I'm not sure why this needs its own patch. I could just be part of the
> next patch.
>
> Thanks,
> drew
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 07/14] arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level
@ 2020-02-07 12:19 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:19 UTC (permalink / raw)
To: Eric Auger
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Fri, Feb 07, 2020 at 01:14:37PM +0100, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:52AM +0100, Eric Auger wrote:
> > This helper function controls the signaling of LPIs at
> > redistributor level.
> >
> > Signed-off-by: Eric Auger <eric.auger@redhat.com>
> >
> > ---
> >
> > v2 -> v3:
> > - move the helper in lib/arm/gic-v3.c
> > - rename the function with gicv3_lpi_ prefix
> > - s/report_abort/assert
> > ---
> > lib/arm/asm/gic-v3.h | 1 +
> > lib/arm/gic-v3.c | 17 +++++++++++++++++
> > 2 files changed, 18 insertions(+)
> >
> > diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> > index ec2a6f0..734c0c0 100644
> > --- a/lib/arm/asm/gic-v3.h
> > +++ b/lib/arm/asm/gic-v3.h
> > @@ -96,6 +96,7 @@ extern void gicv3_lpi_set_config(int n, u8 val);
> > extern u8 gicv3_lpi_get_config(int n);
> > extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
> > extern void gicv3_lpi_alloc_tables(void);
> > +extern void gicv3_lpi_rdist_ctrl(u32 redist, bool set);
> >
> > static inline void gicv3_do_wait_for_rwp(void *base)
> > {
> > diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
> > index c33f883..7865d01 100644
> > --- a/lib/arm/gic-v3.c
> > +++ b/lib/arm/gic-v3.c
> > @@ -210,4 +210,21 @@ void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
> > byte &= ~mask;
> > *ptr = byte;
> > }
> > +
> > +void gicv3_lpi_rdist_ctrl(u32 redist, bool set)
>
> _set_clr_ ?
No, probably not _set_clr_ here. The function could be
static though, with other functions to enable/disable
void gicv3_lpi_rdist_enable(redist) { gicv3_lpi_rdist_ctrl(redist, true); }
void gicv3_lpi_rdist_disable(redist) { gicv3_lpi_rdist_ctrl(redist, false); }
But whatever.
>
> > +{
> > + void *ptr;
> > + u64 val;
> > +
> > + assert(redist < nr_cpus);
> > +
> > + ptr = gicv3_data.redist_base[redist];
> > + val = readl(ptr + GICR_CTLR);
> > + if (set)
> > + val |= GICR_CTLR_ENABLE_LPIS;
> > + else
> > + val &= ~GICR_CTLR_ENABLE_LPIS;
> > + writel(val, ptr + GICR_CTLR);
> > +}
> > #endif /* __aarch64__ */
> > +
>
> stray blank line here
>
> > --
> > 2.20.1
> >
>
> I'm not sure why this needs its own patch. I could just be part of the
> next patch.
>
> Thanks,
> drew
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 12:41 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:41 UTC (permalink / raw)
To: Eric Auger
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
peter.maydell, andre.przywara, thuth, yuzenghui,
alexandru.elisei
On Tue, Jan 28, 2020 at 11:34:53AM +0100, Eric Auger wrote:
> its_enable_defaults() is the top init function that allocates the
> command queue and all the requested tables (device, collection,
> lpi config and pending tables), enable LPIs at distributor level
> and ITS level.
>
> gicv3_enable_defaults must be called before.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - introduce its_setup_baser in this patch
> - squash "arm/arm64: ITS: Init the command queue" in this patch.
> ---
> lib/arm/asm/gic-v3-its.h | 8 ++++
> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 97 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index 815c515..fe73c04 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -36,6 +36,8 @@ struct its_data {
> void *base;
> struct its_typer typer;
> struct its_baser baser[GITS_BASER_NR_REGS];
> + struct its_cmd_block *cmd_base;
> + struct its_cmd_block *cmd_write;
> };
>
> extern struct its_data its_data;
> @@ -88,10 +90,16 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> +
> +struct its_cmd_block {
> + u64 raw_cmd[4];
> +};
> +
> extern void its_parse_typer(void);
> extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> +extern void its_enable_defaults(void);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index 2c0ce13..d1e7e52 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -86,3 +86,92 @@ void its_init(void)
> its_parse_baser(i, &its_data.baser[i]);
> }
>
> +static void its_setup_baser(int i, struct its_baser *baser)
> +{
> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
> + u64 val;
> +
> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
> +
> + val = ((u64)baser->table_addr |
> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
> + (u64)baser->indirect << 62 |
> + (u64)baser->valid << 63);
<< GITS_BASER_INDIRECT
<< GITS_BASER_VALID
> +
> + switch (baser->psz) {
> + case SZ_4K:
> + val |= GITS_BASER_PAGE_SIZE_4K;
> + break;
> + case SZ_16K:
> + val |= GITS_BASER_PAGE_SIZE_16K;
> + break;
> + case SZ_64K:
> + val |= GITS_BASER_PAGE_SIZE_64K;
> + break;
> + }
> +
> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
> +}
> +
> +/**
> + * init_cmd_queue: Allocate the command queue and initialize
> + * CBASER, CREADR, CWRITER
> + */
> +static void its_cmd_queue_init(void)
> +{
> + unsigned long n = SZ_64K >> PAGE_SHIFT;
> + unsigned long order = fls(n);
> + u64 cbaser;
> +
> + its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
> +
> + cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1) | GITS_CBASER_VALID);
> +
> + writeq(cbaser, its_data.base + GITS_CBASER);
> +
> + its_data.cmd_write = its_data.cmd_base;
> + writeq(0, its_data.base + GITS_CWRITER);
> +}
> +
> +void its_enable_defaults(void)
> +{
> + unsigned int i;
> +
> + its_parse_typer();
> +
> + /* Allocate BASER tables (device and collection tables) */
> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
> + struct its_baser *baser = &its_data.baser[i];
> + int ret;
> +
> + ret = its_parse_baser(i, baser);
> + if (ret)
> + continue;
Didn't we already parse typer and baser at its_init time? How/why would
its_parse_baser fail? Should we assert when it does?
> +
> + switch (baser->type) {
> + case GITS_BASER_TYPE_DEVICE:
> + baser->valid = true;
> + its_setup_baser(i, baser);
> + break;
> + case GITS_BASER_TYPE_COLLECTION:
> + baser->valid = true;
> + its_setup_baser(i, baser);
> + break;
> + default:
> + break;
assert() ?
> + }
> + }
> +
> + /* Allocate LPI config and pending tables */
> + gicv3_lpi_alloc_tables();
> +
> + its_cmd_queue_init();
> +
> + for (i = 0; i < nr_cpus; i++)
> + gicv3_lpi_rdist_ctrl(i, true);
> +
> + writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
> +}
> --
> 2.20.1
>
>
It's looking like we don't have a clean separation between its_init and
its_enable_defaults. I'd expect its_init to do all the allocating of
memory and its_enable_defaults to do all the write's to the device.
We should only do its_init once and its_enable should be something
we can do again (after a disable, on reset, etc.). Is that not possible
with the ITS device?
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-02-07 12:41 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:41 UTC (permalink / raw)
To: Eric Auger
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:53AM +0100, Eric Auger wrote:
> its_enable_defaults() is the top init function that allocates the
> command queue and all the requested tables (device, collection,
> lpi config and pending tables), enable LPIs at distributor level
> and ITS level.
>
> gicv3_enable_defaults must be called before.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - introduce its_setup_baser in this patch
> - squash "arm/arm64: ITS: Init the command queue" in this patch.
> ---
> lib/arm/asm/gic-v3-its.h | 8 ++++
> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 97 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index 815c515..fe73c04 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -36,6 +36,8 @@ struct its_data {
> void *base;
> struct its_typer typer;
> struct its_baser baser[GITS_BASER_NR_REGS];
> + struct its_cmd_block *cmd_base;
> + struct its_cmd_block *cmd_write;
> };
>
> extern struct its_data its_data;
> @@ -88,10 +90,16 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> +
> +struct its_cmd_block {
> + u64 raw_cmd[4];
> +};
> +
> extern void its_parse_typer(void);
> extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> +extern void its_enable_defaults(void);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index 2c0ce13..d1e7e52 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -86,3 +86,92 @@ void its_init(void)
> its_parse_baser(i, &its_data.baser[i]);
> }
>
> +static void its_setup_baser(int i, struct its_baser *baser)
> +{
> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
> + u64 val;
> +
> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
> +
> + val = ((u64)baser->table_addr |
> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
> + (u64)baser->indirect << 62 |
> + (u64)baser->valid << 63);
<< GITS_BASER_INDIRECT
<< GITS_BASER_VALID
> +
> + switch (baser->psz) {
> + case SZ_4K:
> + val |= GITS_BASER_PAGE_SIZE_4K;
> + break;
> + case SZ_16K:
> + val |= GITS_BASER_PAGE_SIZE_16K;
> + break;
> + case SZ_64K:
> + val |= GITS_BASER_PAGE_SIZE_64K;
> + break;
> + }
> +
> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
> +}
> +
> +/**
> + * init_cmd_queue: Allocate the command queue and initialize
> + * CBASER, CREADR, CWRITER
> + */
> +static void its_cmd_queue_init(void)
> +{
> + unsigned long n = SZ_64K >> PAGE_SHIFT;
> + unsigned long order = fls(n);
> + u64 cbaser;
> +
> + its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
> +
> + cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1) | GITS_CBASER_VALID);
> +
> + writeq(cbaser, its_data.base + GITS_CBASER);
> +
> + its_data.cmd_write = its_data.cmd_base;
> + writeq(0, its_data.base + GITS_CWRITER);
> +}
> +
> +void its_enable_defaults(void)
> +{
> + unsigned int i;
> +
> + its_parse_typer();
> +
> + /* Allocate BASER tables (device and collection tables) */
> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
> + struct its_baser *baser = &its_data.baser[i];
> + int ret;
> +
> + ret = its_parse_baser(i, baser);
> + if (ret)
> + continue;
Didn't we already parse typer and baser at its_init time? How/why would
its_parse_baser fail? Should we assert when it does?
> +
> + switch (baser->type) {
> + case GITS_BASER_TYPE_DEVICE:
> + baser->valid = true;
> + its_setup_baser(i, baser);
> + break;
> + case GITS_BASER_TYPE_COLLECTION:
> + baser->valid = true;
> + its_setup_baser(i, baser);
> + break;
> + default:
> + break;
assert() ?
> + }
> + }
> +
> + /* Allocate LPI config and pending tables */
> + gicv3_lpi_alloc_tables();
> +
> + its_cmd_queue_init();
> +
> + for (i = 0; i < nr_cpus; i++)
> + gicv3_lpi_rdist_ctrl(i, true);
> +
> + writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
> +}
> --
> 2.20.1
>
>
It's looking like we don't have a clean separation between its_init and
its_enable_defaults. I'd expect its_init to do all the allocating of
memory and its_enable_defaults to do all the write's to the device.
We should only do its_init once and its_enable should be something
we can do again (after a disable, on reset, etc.). Is that not possible
with the ITS device?
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-02-07 12:41 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:41 UTC (permalink / raw)
To: Eric Auger
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:53AM +0100, Eric Auger wrote:
> its_enable_defaults() is the top init function that allocates the
> command queue and all the requested tables (device, collection,
> lpi config and pending tables), enable LPIs at distributor level
> and ITS level.
>
> gicv3_enable_defaults must be called before.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - introduce its_setup_baser in this patch
> - squash "arm/arm64: ITS: Init the command queue" in this patch.
> ---
> lib/arm/asm/gic-v3-its.h | 8 ++++
> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 97 insertions(+)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index 815c515..fe73c04 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -36,6 +36,8 @@ struct its_data {
> void *base;
> struct its_typer typer;
> struct its_baser baser[GITS_BASER_NR_REGS];
> + struct its_cmd_block *cmd_base;
> + struct its_cmd_block *cmd_write;
> };
>
> extern struct its_data its_data;
> @@ -88,10 +90,16 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> +
> +struct its_cmd_block {
> + u64 raw_cmd[4];
> +};
> +
> extern void its_parse_typer(void);
> extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> +extern void its_enable_defaults(void);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index 2c0ce13..d1e7e52 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -86,3 +86,92 @@ void its_init(void)
> its_parse_baser(i, &its_data.baser[i]);
> }
>
> +static void its_setup_baser(int i, struct its_baser *baser)
> +{
> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
> + u64 val;
> +
> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
> +
> + val = ((u64)baser->table_addr |
> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
> + (u64)baser->indirect << 62 |
> + (u64)baser->valid << 63);
<< GITS_BASER_INDIRECT
<< GITS_BASER_VALID
> +
> + switch (baser->psz) {
> + case SZ_4K:
> + val |= GITS_BASER_PAGE_SIZE_4K;
> + break;
> + case SZ_16K:
> + val |= GITS_BASER_PAGE_SIZE_16K;
> + break;
> + case SZ_64K:
> + val |= GITS_BASER_PAGE_SIZE_64K;
> + break;
> + }
> +
> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
> +}
> +
> +/**
> + * init_cmd_queue: Allocate the command queue and initialize
> + * CBASER, CREADR, CWRITER
> + */
> +static void its_cmd_queue_init(void)
> +{
> + unsigned long n = SZ_64K >> PAGE_SHIFT;
> + unsigned long order = fls(n);
> + u64 cbaser;
> +
> + its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
> +
> + cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1) | GITS_CBASER_VALID);
> +
> + writeq(cbaser, its_data.base + GITS_CBASER);
> +
> + its_data.cmd_write = its_data.cmd_base;
> + writeq(0, its_data.base + GITS_CWRITER);
> +}
> +
> +void its_enable_defaults(void)
> +{
> + unsigned int i;
> +
> + its_parse_typer();
> +
> + /* Allocate BASER tables (device and collection tables) */
> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
> + struct its_baser *baser = &its_data.baser[i];
> + int ret;
> +
> + ret = its_parse_baser(i, baser);
> + if (ret)
> + continue;
Didn't we already parse typer and baser at its_init time? How/why would
its_parse_baser fail? Should we assert when it does?
> +
> + switch (baser->type) {
> + case GITS_BASER_TYPE_DEVICE:
> + baser->valid = true;
> + its_setup_baser(i, baser);
> + break;
> + case GITS_BASER_TYPE_COLLECTION:
> + baser->valid = true;
> + its_setup_baser(i, baser);
> + break;
> + default:
> + break;
assert() ?
> + }
> + }
> +
> + /* Allocate LPI config and pending tables */
> + gicv3_lpi_alloc_tables();
> +
> + its_cmd_queue_init();
> +
> + for (i = 0; i < nr_cpus; i++)
> + gicv3_lpi_rdist_ctrl(i, true);
> +
> + writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
> +}
> --
> 2.20.1
>
>
It's looking like we don't have a clean separation between its_init and
its_enable_defaults. I'd expect its_init to do all the allocating of
memory and its_enable_defaults to do all the write's to the device.
We should only do its_init once and its_enable should be something
we can do again (after a disable, on reset, etc.). Is that not possible
with the ITS device?
Thanks,
drew
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 12:51 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:51 UTC (permalink / raw)
To: Eric Auger
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
andre.przywara, peter.maydell, yuzenghui, alexandru.elisei,
thuth
On Tue, Jan 28, 2020 at 11:34:54AM +0100, Eric Auger wrote:
> Introduce an helper functions to register
> - a new device, characterized by its device id and the
> max number of event IDs that dimension its ITT (Interrupt
> Translation Table). The function allocates the ITT.
>
> - a new collection, characterized by its ID and the
> target processing engine (PE).
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - s/report_abort/assert
>
> v1 -> v2:
> - s/nb_/nr_
> ---
> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 63 insertions(+), 1 deletion(-)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index fe73c04..acd97a9 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -31,6 +31,19 @@ struct its_baser {
> };
>
> #define GITS_BASER_NR_REGS 8
> +#define GITS_MAX_DEVICES 8
> +#define GITS_MAX_COLLECTIONS 8
> +
> +struct its_device {
> + u32 device_id; /* device ID */
> + u32 nr_ites; /* Max Interrupt Translation Entries */
> + void *itt; /* Interrupt Translation Table GPA */
> +};
> +
> +struct its_collection {
> + u64 target_address;
> + u16 col_id;
> +};
>
> struct its_data {
> void *base;
> @@ -38,6 +51,10 @@ struct its_data {
> struct its_baser baser[GITS_BASER_NR_REGS];
> struct its_cmd_block *cmd_base;
> struct its_cmd_block *cmd_write;
> + struct its_device devices[GITS_MAX_DEVICES];
> + u32 nr_devices; /* Allocated Devices */
> + struct its_collection collections[GITS_MAX_COLLECTIONS];
> + u32 nr_collections; /* Allocated Collections */
> };
>
> extern struct its_data its_data;
> @@ -90,7 +107,6 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> -
> struct its_cmd_block {
> u64 raw_cmd[4];
> };
> @@ -100,6 +116,8 @@ extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> extern void its_enable_defaults(void);
> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> +extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index d1e7e52..c2dcd01 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>
> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
> }
> +
> +struct its_device *its_create_device(u32 device_id, int nr_ites)
> +{
> + struct its_baser *baser;
> + struct its_device *new;
> + unsigned long n, order;
> +
> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
> +
> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + if (!baser)
> + return NULL;
Should we assert here if we can't find a GITS_BASER_TYPE_DEVICE ?
It seems none of the callers of its_create_device are checking
for null.
> +
> + new = &its_data.devices[its_data.nr_devices];
> +
> + new->device_id = device_id;
> + new->nr_ites = nr_ites;
> +
> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
I've seen this calculation several times now, so I think an
arch-neutral order calculator is in order:
int get_order(size_t size);
> + new->itt = (void *)virt_to_phys(alloc_pages(order));
If this is a physical address then shouldn't itt be phys_addr_t ?
> +
> + its_data.nr_devices++;
> + return new;
> +}
> +
> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
> +{
> + struct its_collection *new;
> +
> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
> +
> + new = &its_data.collections[its_data.nr_collections];
> +
> + new->col_id = col_id;
> +
> + if (its_data.typer.pta)
> + new->target_address = (u64)gicv3_data.redist_base[pe];
> + else
> + new->target_address = pe << 16;
> +
> + its_data.nr_collections++;
> + return new;
> +}
> --
> 2.20.1
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
@ 2020-02-07 12:51 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:51 UTC (permalink / raw)
To: Eric Auger
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:54AM +0100, Eric Auger wrote:
> Introduce an helper functions to register
> - a new device, characterized by its device id and the
> max number of event IDs that dimension its ITT (Interrupt
> Translation Table). The function allocates the ITT.
>
> - a new collection, characterized by its ID and the
> target processing engine (PE).
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - s/report_abort/assert
>
> v1 -> v2:
> - s/nb_/nr_
> ---
> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 63 insertions(+), 1 deletion(-)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index fe73c04..acd97a9 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -31,6 +31,19 @@ struct its_baser {
> };
>
> #define GITS_BASER_NR_REGS 8
> +#define GITS_MAX_DEVICES 8
> +#define GITS_MAX_COLLECTIONS 8
> +
> +struct its_device {
> + u32 device_id; /* device ID */
> + u32 nr_ites; /* Max Interrupt Translation Entries */
> + void *itt; /* Interrupt Translation Table GPA */
> +};
> +
> +struct its_collection {
> + u64 target_address;
> + u16 col_id;
> +};
>
> struct its_data {
> void *base;
> @@ -38,6 +51,10 @@ struct its_data {
> struct its_baser baser[GITS_BASER_NR_REGS];
> struct its_cmd_block *cmd_base;
> struct its_cmd_block *cmd_write;
> + struct its_device devices[GITS_MAX_DEVICES];
> + u32 nr_devices; /* Allocated Devices */
> + struct its_collection collections[GITS_MAX_COLLECTIONS];
> + u32 nr_collections; /* Allocated Collections */
> };
>
> extern struct its_data its_data;
> @@ -90,7 +107,6 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> -
> struct its_cmd_block {
> u64 raw_cmd[4];
> };
> @@ -100,6 +116,8 @@ extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> extern void its_enable_defaults(void);
> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> +extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index d1e7e52..c2dcd01 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>
> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
> }
> +
> +struct its_device *its_create_device(u32 device_id, int nr_ites)
> +{
> + struct its_baser *baser;
> + struct its_device *new;
> + unsigned long n, order;
> +
> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
> +
> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + if (!baser)
> + return NULL;
Should we assert here if we can't find a GITS_BASER_TYPE_DEVICE ?
It seems none of the callers of its_create_device are checking
for null.
> +
> + new = &its_data.devices[its_data.nr_devices];
> +
> + new->device_id = device_id;
> + new->nr_ites = nr_ites;
> +
> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
I've seen this calculation several times now, so I think an
arch-neutral order calculator is in order:
int get_order(size_t size);
> + new->itt = (void *)virt_to_phys(alloc_pages(order));
If this is a physical address then shouldn't itt be phys_addr_t ?
> +
> + its_data.nr_devices++;
> + return new;
> +}
> +
> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
> +{
> + struct its_collection *new;
> +
> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
> +
> + new = &its_data.collections[its_data.nr_collections];
> +
> + new->col_id = col_id;
> +
> + if (its_data.typer.pta)
> + new->target_address = (u64)gicv3_data.redist_base[pe];
> + else
> + new->target_address = pe << 16;
> +
> + its_data.nr_collections++;
> + return new;
> +}
> --
> 2.20.1
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
@ 2020-02-07 12:51 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 12:51 UTC (permalink / raw)
To: Eric Auger
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:54AM +0100, Eric Auger wrote:
> Introduce an helper functions to register
> - a new device, characterized by its device id and the
> max number of event IDs that dimension its ITT (Interrupt
> Translation Table). The function allocates the ITT.
>
> - a new collection, characterized by its ID and the
> target processing engine (PE).
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - s/report_abort/assert
>
> v1 -> v2:
> - s/nb_/nr_
> ---
> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 63 insertions(+), 1 deletion(-)
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index fe73c04..acd97a9 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -31,6 +31,19 @@ struct its_baser {
> };
>
> #define GITS_BASER_NR_REGS 8
> +#define GITS_MAX_DEVICES 8
> +#define GITS_MAX_COLLECTIONS 8
> +
> +struct its_device {
> + u32 device_id; /* device ID */
> + u32 nr_ites; /* Max Interrupt Translation Entries */
> + void *itt; /* Interrupt Translation Table GPA */
> +};
> +
> +struct its_collection {
> + u64 target_address;
> + u16 col_id;
> +};
>
> struct its_data {
> void *base;
> @@ -38,6 +51,10 @@ struct its_data {
> struct its_baser baser[GITS_BASER_NR_REGS];
> struct its_cmd_block *cmd_base;
> struct its_cmd_block *cmd_write;
> + struct its_device devices[GITS_MAX_DEVICES];
> + u32 nr_devices; /* Allocated Devices */
> + struct its_collection collections[GITS_MAX_COLLECTIONS];
> + u32 nr_collections; /* Allocated Collections */
> };
>
> extern struct its_data its_data;
> @@ -90,7 +107,6 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> -
> struct its_cmd_block {
> u64 raw_cmd[4];
> };
> @@ -100,6 +116,8 @@ extern void its_init(void);
> extern int its_parse_baser(int i, struct its_baser *baser);
> extern struct its_baser *its_lookup_baser(int type);
> extern void its_enable_defaults(void);
> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> +extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>
> #else /* __arm__ */
>
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index d1e7e52..c2dcd01 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>
> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
> }
> +
> +struct its_device *its_create_device(u32 device_id, int nr_ites)
> +{
> + struct its_baser *baser;
> + struct its_device *new;
> + unsigned long n, order;
> +
> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
> +
> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
> + if (!baser)
> + return NULL;
Should we assert here if we can't find a GITS_BASER_TYPE_DEVICE ?
It seems none of the callers of its_create_device are checking
for null.
> +
> + new = &its_data.devices[its_data.nr_devices];
> +
> + new->device_id = device_id;
> + new->nr_ites = nr_ites;
> +
> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
I've seen this calculation several times now, so I think an
arch-neutral order calculator is in order:
int get_order(size_t size);
> + new->itt = (void *)virt_to_phys(alloc_pages(order));
If this is a physical address then shouldn't itt be phys_addr_t ?
> +
> + its_data.nr_devices++;
> + return new;
> +}
> +
> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
> +{
> + struct its_collection *new;
> +
> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
> +
> + new = &its_data.collections[its_data.nr_collections];
> +
> + new->col_id = col_id;
> +
> + if (its_data.typer.pta)
> + new->target_address = (u64)gicv3_data.redist_base[pe];
> + else
> + new->target_address = pe << 16;
> +
> + its_data.nr_collections++;
> + return new;
> +}
> --
> 2.20.1
>
Thanks,
drew
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 13:15 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 13:15 UTC (permalink / raw)
To: Eric Auger
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
andre.przywara, peter.maydell, yuzenghui, alexandru.elisei,
thuth
On Tue, Jan 28, 2020 at 11:34:56AM +0100, Eric Auger wrote:
> Triggers LPIs through the INT command.
>
> the test checks the LPI hits the right CPU and triggers
> the right LPI intid, ie. the translation is correct.
>
> Updates to the config table also are tested, along with inv
> and invall commands.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - add comments
> - keep the report_skip in case there aren't 4 vcpus to be able to
> run other tests in the its category.
> - fix the prefix pop
> - move its_event and its_stats to arm/gic.c
> ---
> arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
> arm/unittests.cfg | 7 ++
> 2 files changed, 224 insertions(+), 11 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index 4d7dd03..50104b1 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
> }
> }
>
> +static void setup_irq(handler_t handler)
> +{
> + gic_enable_defaults();
> +#ifdef __arm__
> + install_exception_handler(EXCPTN_IRQ, handler);
> +#else
> + install_irq_handler(EL1H_IRQ, handler);
> +#endif
> + local_irq_enable();
> +}
> +
> +#if defined(__aarch64__)
> +struct its_event {
> + int cpu_id;
> + int lpi_id;
> +};
> +
> +struct its_stats {
> + struct its_event expected;
> + struct its_event observed;
> +};
> +
> +static struct its_stats lpi_stats;
> +
> +static void lpi_handler(struct pt_regs *regs __unused)
> +{
> + u32 irqstat = gic_read_iar();
> + int irqnr = gic_iar_irqnr(irqstat);
> +
> + gic_write_eoir(irqstat);
> + if (irqnr < 8192)
> + report(false, "Unexpected non LPI interrupt received");
report_info
> + smp_rmb(); /* pairs with wmb in lpi_stats_expect */
> + lpi_stats.observed.cpu_id = smp_processor_id();
> + lpi_stats.observed.lpi_id = irqnr;
> + smp_wmb(); /* pairs with rmb in check_lpi_stats */
> +}
> +
> +static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
> +{
> + lpi_stats.expected.cpu_id = exp_cpu_id;
> + lpi_stats.expected.lpi_id = exp_lpi_id;
> + lpi_stats.observed.cpu_id = -1;
> + lpi_stats.observed.lpi_id = -1;
> + smp_wmb(); /* pairs with rmb in handler */
> +}
> +
> +static void check_lpi_stats(void)
static void check_lpi_stats(const char *testname)
{
bool pass = false;
> +{
> + mdelay(100);
> + smp_rmb(); /* pairs with wmb in lpi_handler */
> + if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
> + (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
nit: extra ()
> + if (lpi_stats.observed.cpu_id == -1 &&
> + lpi_stats.observed.lpi_id == -1) {
> + report(false,
> + "No LPI received whereas (cpuid=%d, intid=%d) "
> + "was expected", lpi_stats.expected.cpu_id,
> + lpi_stats.expected.lpi_id);
report_info
> + } else {
> + report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
> + lpi_stats.observed.cpu_id,
> + lpi_stats.observed.lpi_id);
report_info
> + }
pass = false;
> + } else if (lpi_stats.expected.lpi_id != -1) {
> + report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
> + lpi_stats.observed.cpu_id);
report_info
> + } else {
> + report(true, "no LPI received, as expected");
report_info
> + }
report(pass, "%s", testname);
> +}
> +
> +static void secondary_lpi_test(void)
> +{
> + setup_irq(lpi_handler);
> + cpumask_set_cpu(smp_processor_id(), &ready);
> + while (1)
> + wfi();
> +}
> +#endif
> +
> static void gicv2_ipi_send_self(void)
> {
> writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
> @@ -217,17 +298,6 @@ static void ipi_test_smp(void)
> report_prefix_pop();
> }
>
> -static void setup_irq(handler_t handler)
> -{
> - gic_enable_defaults();
> -#ifdef __arm__
> - install_exception_handler(EXCPTN_IRQ, handler);
> -#else
> - install_irq_handler(EL1H_IRQ, handler);
> -#endif
> - local_irq_enable();
> -}
> -
> static void ipi_send(void)
> {
> setup_irq(ipi_handler);
> @@ -522,6 +592,7 @@ static void gic_test_mmio(void)
> #if defined(__arm__)
>
> static void test_its_introspection(void) {}
> +static void test_its_trigger(void) {}
>
> #else /* __arch64__ */
>
> @@ -561,6 +632,137 @@ static void test_its_introspection(void)
> report_info("collection baser entry_size = 0x%x", coll_baser->esz);
> }
>
> +static bool its_prerequisites(int nb_cpus)
> +{
> + int cpu;
> +
> + if (!gicv3_its_base()) {
> + report_skip("No ITS, skip ...");
> + return true;
> + }
> +
> + if (nr_cpus < 4) {
nr_cpus < nb_cpus, or just drop the nb_cpus parameter and hard code 4
here.
> + report_skip("Test requires at least %d vcpus", nb_cpus);
> + return true;
> + }
> +
> + stats_reset();
> +
> + setup_irq(lpi_handler);
> +
> + for_each_present_cpu(cpu) {
> + if (cpu == 0)
> + continue;
> + smp_boot_secondary(cpu, secondary_lpi_test);
> + }
> + wait_on_ready();
> +
> + its_enable_defaults();
> +
> + lpi_stats_expect(-1, -1);
> + check_lpi_stats();
> +
> + return false;
Reverse logic. I'd expect 'return true' for success.
> +}
> +
> +static void test_its_trigger(void)
> +{
> + struct its_collection *col3, *col2;
> + struct its_device *dev2, *dev7;
> +
> + if (its_prerequisites(4))
if (!its_prerequisites(...))
> + return;
> +
> + dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> + dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
> +
> + col3 = its_create_collection(3 /* col id */, 3/* target PE */);
> + col2 = its_create_collection(2 /* col id */, 2/* target PE */);
> +
> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
> + gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
> +
> + its_send_invall(col2);
> + its_send_invall(col3);
> +
> + report_prefix_push("int");
> + /*
> + * dev=2, eventid=20 -> lpi= 8195, col=3
> + * dev=7, eventid=255 -> lpi= 8196, col=2
> + * Trigger dev2, eventid=20 and dev7, eventid=255
> + * Check both LPIs hit
> + */
> +
> + its_send_mapd(dev2, true);
> + its_send_mapd(dev7, true);
> +
> + its_send_mapc(col3, true);
> + its_send_mapc(col2, true);
> +
> + its_send_mapti(dev2, 8195 /* lpi id */,
> + 20 /* event id */, col3);
> + its_send_mapti(dev7, 8196 /* lpi id */,
> + 255 /* event id */, col2);
No need for line breaks, with the embedded comments it's hard to read
> +
> + lpi_stats_expect(3, 8195);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + lpi_stats_expect(2, 8196);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> +
> + report_prefix_pop();
I think a table of parameters and loop would be nicer than all the
repeated function calls.
> +
> + report_prefix_push("inv/invall");
> +
> + /*
> + * disable 8195, check dev2/eventid=20 does not trigger the
> + * corresponding LPI
> + */
> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED
> + its_send_inv(dev2, 20);
> +
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + /*
> + * re-enable the LPI but willingly do not call invall
> + * so the change in config is not taken into account.
> + * The LPI should not hit
> + */
> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + /* Now call the invall and check the LPI hits */
> + its_send_invall(col3);
> + lpi_stats_expect(3, 8195);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + report_prefix_pop();
Need blank line here.
> + /*
> + * Unmap device 2 and check the eventid 20 formerly
> + * attached to it does not hit anymore
> + */
> + report_prefix_push("mapd valid=false");
Above you have the prefix-push before the comment explaining the test.
After is probably better, but whatever, as long as it's consistent.
> + its_send_mapd(dev2, false);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> + report_prefix_pop();
> +
> + /* Unmap the collection this time and check no LPI does hit */
> + report_prefix_push("mapc valid=false");
> + its_send_mapc(col2, false);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> + report_prefix_pop();
> +}
> #endif
>
> int main(int argc, char **argv)
> @@ -594,6 +796,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> gic_test_mmio();
> report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-trigger")) {
> + report_prefix_push(argv[1]);
> + test_its_trigger();
> + report_prefix_pop();
> } else if (strcmp(argv[1], "its-introspection") == 0) {
> report_prefix_push(argv[1]);
> test_its_introspection();
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index ba2b31b..bfafec5 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
> groups = its
> arch = arm64
>
> +[its-trigger]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'its-trigger'
> +groups = its
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> --
> 2.20.1
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
@ 2020-02-07 13:15 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 13:15 UTC (permalink / raw)
To: Eric Auger
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:56AM +0100, Eric Auger wrote:
> Triggers LPIs through the INT command.
>
> the test checks the LPI hits the right CPU and triggers
> the right LPI intid, ie. the translation is correct.
>
> Updates to the config table also are tested, along with inv
> and invall commands.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - add comments
> - keep the report_skip in case there aren't 4 vcpus to be able to
> run other tests in the its category.
> - fix the prefix pop
> - move its_event and its_stats to arm/gic.c
> ---
> arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
> arm/unittests.cfg | 7 ++
> 2 files changed, 224 insertions(+), 11 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index 4d7dd03..50104b1 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
> }
> }
>
> +static void setup_irq(handler_t handler)
> +{
> + gic_enable_defaults();
> +#ifdef __arm__
> + install_exception_handler(EXCPTN_IRQ, handler);
> +#else
> + install_irq_handler(EL1H_IRQ, handler);
> +#endif
> + local_irq_enable();
> +}
> +
> +#if defined(__aarch64__)
> +struct its_event {
> + int cpu_id;
> + int lpi_id;
> +};
> +
> +struct its_stats {
> + struct its_event expected;
> + struct its_event observed;
> +};
> +
> +static struct its_stats lpi_stats;
> +
> +static void lpi_handler(struct pt_regs *regs __unused)
> +{
> + u32 irqstat = gic_read_iar();
> + int irqnr = gic_iar_irqnr(irqstat);
> +
> + gic_write_eoir(irqstat);
> + if (irqnr < 8192)
> + report(false, "Unexpected non LPI interrupt received");
report_info
> + smp_rmb(); /* pairs with wmb in lpi_stats_expect */
> + lpi_stats.observed.cpu_id = smp_processor_id();
> + lpi_stats.observed.lpi_id = irqnr;
> + smp_wmb(); /* pairs with rmb in check_lpi_stats */
> +}
> +
> +static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
> +{
> + lpi_stats.expected.cpu_id = exp_cpu_id;
> + lpi_stats.expected.lpi_id = exp_lpi_id;
> + lpi_stats.observed.cpu_id = -1;
> + lpi_stats.observed.lpi_id = -1;
> + smp_wmb(); /* pairs with rmb in handler */
> +}
> +
> +static void check_lpi_stats(void)
static void check_lpi_stats(const char *testname)
{
bool pass = false;
> +{
> + mdelay(100);
> + smp_rmb(); /* pairs with wmb in lpi_handler */
> + if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
> + (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
nit: extra ()
> + if (lpi_stats.observed.cpu_id == -1 &&
> + lpi_stats.observed.lpi_id == -1) {
> + report(false,
> + "No LPI received whereas (cpuid=%d, intid=%d) "
> + "was expected", lpi_stats.expected.cpu_id,
> + lpi_stats.expected.lpi_id);
report_info
> + } else {
> + report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
> + lpi_stats.observed.cpu_id,
> + lpi_stats.observed.lpi_id);
report_info
> + }
pass = false;
> + } else if (lpi_stats.expected.lpi_id != -1) {
> + report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
> + lpi_stats.observed.cpu_id);
report_info
> + } else {
> + report(true, "no LPI received, as expected");
report_info
> + }
report(pass, "%s", testname);
> +}
> +
> +static void secondary_lpi_test(void)
> +{
> + setup_irq(lpi_handler);
> + cpumask_set_cpu(smp_processor_id(), &ready);
> + while (1)
> + wfi();
> +}
> +#endif
> +
> static void gicv2_ipi_send_self(void)
> {
> writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
> @@ -217,17 +298,6 @@ static void ipi_test_smp(void)
> report_prefix_pop();
> }
>
> -static void setup_irq(handler_t handler)
> -{
> - gic_enable_defaults();
> -#ifdef __arm__
> - install_exception_handler(EXCPTN_IRQ, handler);
> -#else
> - install_irq_handler(EL1H_IRQ, handler);
> -#endif
> - local_irq_enable();
> -}
> -
> static void ipi_send(void)
> {
> setup_irq(ipi_handler);
> @@ -522,6 +592,7 @@ static void gic_test_mmio(void)
> #if defined(__arm__)
>
> static void test_its_introspection(void) {}
> +static void test_its_trigger(void) {}
>
> #else /* __arch64__ */
>
> @@ -561,6 +632,137 @@ static void test_its_introspection(void)
> report_info("collection baser entry_size = 0x%x", coll_baser->esz);
> }
>
> +static bool its_prerequisites(int nb_cpus)
> +{
> + int cpu;
> +
> + if (!gicv3_its_base()) {
> + report_skip("No ITS, skip ...");
> + return true;
> + }
> +
> + if (nr_cpus < 4) {
nr_cpus < nb_cpus, or just drop the nb_cpus parameter and hard code 4
here.
> + report_skip("Test requires at least %d vcpus", nb_cpus);
> + return true;
> + }
> +
> + stats_reset();
> +
> + setup_irq(lpi_handler);
> +
> + for_each_present_cpu(cpu) {
> + if (cpu == 0)
> + continue;
> + smp_boot_secondary(cpu, secondary_lpi_test);
> + }
> + wait_on_ready();
> +
> + its_enable_defaults();
> +
> + lpi_stats_expect(-1, -1);
> + check_lpi_stats();
> +
> + return false;
Reverse logic. I'd expect 'return true' for success.
> +}
> +
> +static void test_its_trigger(void)
> +{
> + struct its_collection *col3, *col2;
> + struct its_device *dev2, *dev7;
> +
> + if (its_prerequisites(4))
if (!its_prerequisites(...))
> + return;
> +
> + dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> + dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
> +
> + col3 = its_create_collection(3 /* col id */, 3/* target PE */);
> + col2 = its_create_collection(2 /* col id */, 2/* target PE */);
> +
> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
> + gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
> +
> + its_send_invall(col2);
> + its_send_invall(col3);
> +
> + report_prefix_push("int");
> + /*
> + * dev=2, eventid=20 -> lpi= 8195, col=3
> + * dev=7, eventid=255 -> lpi= 8196, col=2
> + * Trigger dev2, eventid=20 and dev7, eventid=255
> + * Check both LPIs hit
> + */
> +
> + its_send_mapd(dev2, true);
> + its_send_mapd(dev7, true);
> +
> + its_send_mapc(col3, true);
> + its_send_mapc(col2, true);
> +
> + its_send_mapti(dev2, 8195 /* lpi id */,
> + 20 /* event id */, col3);
> + its_send_mapti(dev7, 8196 /* lpi id */,
> + 255 /* event id */, col2);
No need for line breaks, with the embedded comments it's hard to read
> +
> + lpi_stats_expect(3, 8195);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + lpi_stats_expect(2, 8196);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> +
> + report_prefix_pop();
I think a table of parameters and loop would be nicer than all the
repeated function calls.
> +
> + report_prefix_push("inv/invall");
> +
> + /*
> + * disable 8195, check dev2/eventid=20 does not trigger the
> + * corresponding LPI
> + */
> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED
> + its_send_inv(dev2, 20);
> +
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + /*
> + * re-enable the LPI but willingly do not call invall
> + * so the change in config is not taken into account.
> + * The LPI should not hit
> + */
> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + /* Now call the invall and check the LPI hits */
> + its_send_invall(col3);
> + lpi_stats_expect(3, 8195);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + report_prefix_pop();
Need blank line here.
> + /*
> + * Unmap device 2 and check the eventid 20 formerly
> + * attached to it does not hit anymore
> + */
> + report_prefix_push("mapd valid=false");
Above you have the prefix-push before the comment explaining the test.
After is probably better, but whatever, as long as it's consistent.
> + its_send_mapd(dev2, false);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> + report_prefix_pop();
> +
> + /* Unmap the collection this time and check no LPI does hit */
> + report_prefix_push("mapc valid=false");
> + its_send_mapc(col2, false);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> + report_prefix_pop();
> +}
> #endif
>
> int main(int argc, char **argv)
> @@ -594,6 +796,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> gic_test_mmio();
> report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-trigger")) {
> + report_prefix_push(argv[1]);
> + test_its_trigger();
> + report_prefix_pop();
> } else if (strcmp(argv[1], "its-introspection") == 0) {
> report_prefix_push(argv[1]);
> test_its_introspection();
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index ba2b31b..bfafec5 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
> groups = its
> arch = arm64
>
> +[its-trigger]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'its-trigger'
> +groups = its
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> --
> 2.20.1
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
@ 2020-02-07 13:15 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 13:15 UTC (permalink / raw)
To: Eric Auger
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:56AM +0100, Eric Auger wrote:
> Triggers LPIs through the INT command.
>
> the test checks the LPI hits the right CPU and triggers
> the right LPI intid, ie. the translation is correct.
>
> Updates to the config table also are tested, along with inv
> and invall commands.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - add comments
> - keep the report_skip in case there aren't 4 vcpus to be able to
> run other tests in the its category.
> - fix the prefix pop
> - move its_event and its_stats to arm/gic.c
> ---
> arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
> arm/unittests.cfg | 7 ++
> 2 files changed, 224 insertions(+), 11 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index 4d7dd03..50104b1 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
> }
> }
>
> +static void setup_irq(handler_t handler)
> +{
> + gic_enable_defaults();
> +#ifdef __arm__
> + install_exception_handler(EXCPTN_IRQ, handler);
> +#else
> + install_irq_handler(EL1H_IRQ, handler);
> +#endif
> + local_irq_enable();
> +}
> +
> +#if defined(__aarch64__)
> +struct its_event {
> + int cpu_id;
> + int lpi_id;
> +};
> +
> +struct its_stats {
> + struct its_event expected;
> + struct its_event observed;
> +};
> +
> +static struct its_stats lpi_stats;
> +
> +static void lpi_handler(struct pt_regs *regs __unused)
> +{
> + u32 irqstat = gic_read_iar();
> + int irqnr = gic_iar_irqnr(irqstat);
> +
> + gic_write_eoir(irqstat);
> + if (irqnr < 8192)
> + report(false, "Unexpected non LPI interrupt received");
report_info
> + smp_rmb(); /* pairs with wmb in lpi_stats_expect */
> + lpi_stats.observed.cpu_id = smp_processor_id();
> + lpi_stats.observed.lpi_id = irqnr;
> + smp_wmb(); /* pairs with rmb in check_lpi_stats */
> +}
> +
> +static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
> +{
> + lpi_stats.expected.cpu_id = exp_cpu_id;
> + lpi_stats.expected.lpi_id = exp_lpi_id;
> + lpi_stats.observed.cpu_id = -1;
> + lpi_stats.observed.lpi_id = -1;
> + smp_wmb(); /* pairs with rmb in handler */
> +}
> +
> +static void check_lpi_stats(void)
static void check_lpi_stats(const char *testname)
{
bool pass = false;
> +{
> + mdelay(100);
> + smp_rmb(); /* pairs with wmb in lpi_handler */
> + if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
> + (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
nit: extra ()
> + if (lpi_stats.observed.cpu_id == -1 &&
> + lpi_stats.observed.lpi_id == -1) {
> + report(false,
> + "No LPI received whereas (cpuid=%d, intid=%d) "
> + "was expected", lpi_stats.expected.cpu_id,
> + lpi_stats.expected.lpi_id);
report_info
> + } else {
> + report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
> + lpi_stats.observed.cpu_id,
> + lpi_stats.observed.lpi_id);
report_info
> + }
pass = false;
> + } else if (lpi_stats.expected.lpi_id != -1) {
> + report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
> + lpi_stats.observed.cpu_id);
report_info
> + } else {
> + report(true, "no LPI received, as expected");
report_info
> + }
report(pass, "%s", testname);
> +}
> +
> +static void secondary_lpi_test(void)
> +{
> + setup_irq(lpi_handler);
> + cpumask_set_cpu(smp_processor_id(), &ready);
> + while (1)
> + wfi();
> +}
> +#endif
> +
> static void gicv2_ipi_send_self(void)
> {
> writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
> @@ -217,17 +298,6 @@ static void ipi_test_smp(void)
> report_prefix_pop();
> }
>
> -static void setup_irq(handler_t handler)
> -{
> - gic_enable_defaults();
> -#ifdef __arm__
> - install_exception_handler(EXCPTN_IRQ, handler);
> -#else
> - install_irq_handler(EL1H_IRQ, handler);
> -#endif
> - local_irq_enable();
> -}
> -
> static void ipi_send(void)
> {
> setup_irq(ipi_handler);
> @@ -522,6 +592,7 @@ static void gic_test_mmio(void)
> #if defined(__arm__)
>
> static void test_its_introspection(void) {}
> +static void test_its_trigger(void) {}
>
> #else /* __arch64__ */
>
> @@ -561,6 +632,137 @@ static void test_its_introspection(void)
> report_info("collection baser entry_size = 0x%x", coll_baser->esz);
> }
>
> +static bool its_prerequisites(int nb_cpus)
> +{
> + int cpu;
> +
> + if (!gicv3_its_base()) {
> + report_skip("No ITS, skip ...");
> + return true;
> + }
> +
> + if (nr_cpus < 4) {
nr_cpus < nb_cpus, or just drop the nb_cpus parameter and hard code 4
here.
> + report_skip("Test requires at least %d vcpus", nb_cpus);
> + return true;
> + }
> +
> + stats_reset();
> +
> + setup_irq(lpi_handler);
> +
> + for_each_present_cpu(cpu) {
> + if (cpu == 0)
> + continue;
> + smp_boot_secondary(cpu, secondary_lpi_test);
> + }
> + wait_on_ready();
> +
> + its_enable_defaults();
> +
> + lpi_stats_expect(-1, -1);
> + check_lpi_stats();
> +
> + return false;
Reverse logic. I'd expect 'return true' for success.
> +}
> +
> +static void test_its_trigger(void)
> +{
> + struct its_collection *col3, *col2;
> + struct its_device *dev2, *dev7;
> +
> + if (its_prerequisites(4))
if (!its_prerequisites(...))
> + return;
> +
> + dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> + dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
> +
> + col3 = its_create_collection(3 /* col id */, 3/* target PE */);
> + col2 = its_create_collection(2 /* col id */, 2/* target PE */);
> +
> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
> + gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
> +
> + its_send_invall(col2);
> + its_send_invall(col3);
> +
> + report_prefix_push("int");
> + /*
> + * dev=2, eventid=20 -> lpi= 8195, col=3
> + * dev=7, eventid=255 -> lpi= 8196, col=2
> + * Trigger dev2, eventid=20 and dev7, eventid=255
> + * Check both LPIs hit
> + */
> +
> + its_send_mapd(dev2, true);
> + its_send_mapd(dev7, true);
> +
> + its_send_mapc(col3, true);
> + its_send_mapc(col2, true);
> +
> + its_send_mapti(dev2, 8195 /* lpi id */,
> + 20 /* event id */, col3);
> + its_send_mapti(dev7, 8196 /* lpi id */,
> + 255 /* event id */, col2);
No need for line breaks, with the embedded comments it's hard to read
> +
> + lpi_stats_expect(3, 8195);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + lpi_stats_expect(2, 8196);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> +
> + report_prefix_pop();
I think a table of parameters and loop would be nicer than all the
repeated function calls.
> +
> + report_prefix_push("inv/invall");
> +
> + /*
> + * disable 8195, check dev2/eventid=20 does not trigger the
> + * corresponding LPI
> + */
> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED
> + its_send_inv(dev2, 20);
> +
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + /*
> + * re-enable the LPI but willingly do not call invall
> + * so the change in config is not taken into account.
> + * The LPI should not hit
> + */
> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + /* Now call the invall and check the LPI hits */
> + its_send_invall(col3);
> + lpi_stats_expect(3, 8195);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + report_prefix_pop();
Need blank line here.
> + /*
> + * Unmap device 2 and check the eventid 20 formerly
> + * attached to it does not hit anymore
> + */
> + report_prefix_push("mapd valid=false");
Above you have the prefix-push before the comment explaining the test.
After is probably better, but whatever, as long as it's consistent.
> + its_send_mapd(dev2, false);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> + report_prefix_pop();
> +
> + /* Unmap the collection this time and check no LPI does hit */
> + report_prefix_push("mapc valid=false");
> + its_send_mapc(col2, false);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> + report_prefix_pop();
> +}
> #endif
>
> int main(int argc, char **argv)
> @@ -594,6 +796,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> gic_test_mmio();
> report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-trigger")) {
> + report_prefix_push(argv[1]);
> + test_its_trigger();
> + report_prefix_pop();
> } else if (strcmp(argv[1], "its-introspection") == 0) {
> report_prefix_push(argv[1]);
> test_its_introspection();
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index ba2b31b..bfafec5 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
> groups = its
> arch = arm64
>
> +[its-trigger]
> +file = gic.flat
> +smp = $MAX_SMP
> +extra_params = -machine gic-version=3 -append 'its-trigger'
> +groups = its
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> --
> 2.20.1
>
Thanks,
drew
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 10/14] arm/arm64: ITS: commands
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 13:37 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 13:37 UTC (permalink / raw)
To: Eric Auger
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
peter.maydell, andre.przywara, thuth, yuzenghui,
alexandru.elisei
On Tue, Jan 28, 2020 at 11:34:55AM +0100, Eric Auger wrote:
> Implement main ITS commands. The code is largely inherited from
> the ITS driver.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - do not use report() anymore
> - assert if cmd_write exceeds the queue capacity
>
> v1 -> v2:
> - removed its_print_cmd_state
> ---
> arm/Makefile.arm64 | 2 +-
> lib/arm/asm/gic-v3-its.h | 38 +++-
> lib/arm/gic-v3-its-cmd.c | 454 +++++++++++++++++++++++++++++++++++++++
> 3 files changed, 492 insertions(+), 2 deletions(-)
> create mode 100644 lib/arm/gic-v3-its-cmd.c
>
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 2571ffb..d12aea5 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -19,7 +19,7 @@ endef
> cstart.o = $(TEST_DIR)/cstart64.o
> cflatobjs += lib/arm64/processor.o
> cflatobjs += lib/arm64/spinlock.o
> -cflatobjs += lib/arm/gic-v3-its.o
> +cflatobjs += lib/arm/gic-v3-its.o lib/arm/gic-v3-its-cmd.o
>
> OBJDIRS += lib/arm64
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index acd97a9..0e5c5b6 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -45,6 +45,8 @@ struct its_collection {
> u16 col_id;
> };
>
> +struct its_cmd_block;
> +
This isn't necessary. If it was, then it should have been added in a
previous patch.
> struct its_data {
> void *base;
> struct its_typer typer;
> @@ -107,6 +109,24 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> +/*
> + * ITS commands
> + */
> +#define GITS_CMD_MAPD 0x08
> +#define GITS_CMD_MAPC 0x09
> +#define GITS_CMD_MAPTI 0x0a
> +/* older GIC documentation used MAPVI for this command */
> +#define GITS_CMD_MAPVI GITS_CMD_MAPTI
> +#define GITS_CMD_MAPI 0x0b
> +#define GITS_CMD_MOVI 0x01
> +#define GITS_CMD_DISCARD 0x0f
> +#define GITS_CMD_INV 0x0c
> +#define GITS_CMD_MOVALL 0x0e
> +#define GITS_CMD_INVALL 0x0d
> +#define GITS_CMD_INT 0x03
> +#define GITS_CMD_CLEAR 0x04
> +#define GITS_CMD_SYNC 0x05
Please use tabs.
> +
> struct its_cmd_block {
> u64 raw_cmd[4];
> };
> @@ -119,11 +139,27 @@ extern void its_enable_defaults(void);
> extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>
> +extern void its_send_mapd(struct its_device *dev, int valid);
> +extern void its_send_mapc(struct its_collection *col, int valid);
> +extern void its_send_mapti(struct its_device *dev, u32 irq_id,
> + u32 event_id, struct its_collection *col);
> +extern void its_send_int(struct its_device *dev, u32 event_id);
> +extern void its_send_inv(struct its_device *dev, u32 event_id);
> +extern void its_send_discard(struct its_device *dev, u32 event_id);
> +extern void its_send_clear(struct its_device *dev, u32 event_id);
> +extern void its_send_invall(struct its_collection *col);
> +extern void its_send_movi(struct its_device *dev,
> + struct its_collection *col, u32 id);
> +extern void its_send_sync(struct its_collection *col);
> +
> +#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
> +#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
> +#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)
What are these flags for?
> +
> #else /* __arm__ */
>
> static inline void its_init(void) {}
>
> #endif
> -
> #endif /* !__ASSEMBLY__ */
> #endif /* _ASMARM_GIC_V3_ITS_H_ */
> diff --git a/lib/arm/gic-v3-its-cmd.c b/lib/arm/gic-v3-its-cmd.c
> new file mode 100644
> index 0000000..fb4364c
> --- /dev/null
> +++ b/lib/arm/gic-v3-its-cmd.c
> @@ -0,0 +1,454 @@
> +/*
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * Most of the code is copy-pasted from:
> + * drivers/irqchip/irq-gic-v3-its.c
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <asm/io.h>
> +#include <asm/gic.h>
> +#include <asm/gic-v3-its.h>
> +
> +#define ITS_ITT_ALIGN SZ_256
tabs
> +
> +static const char * const its_cmd_string[] = {
> + [GITS_CMD_MAPD] = "MAPD",
> + [GITS_CMD_MAPC] = "MAPC",
> + [GITS_CMD_MAPTI] = "MAPTI",
> + [GITS_CMD_MAPI] = "MAPI",
> + [GITS_CMD_MOVI] = "MOVI",
> + [GITS_CMD_DISCARD] = "DISCARD",
> + [GITS_CMD_INV] = "INV",
> + [GITS_CMD_MOVALL] = "MOVALL",
> + [GITS_CMD_INVALL] = "INVALL",
> + [GITS_CMD_INT] = "INT",
> + [GITS_CMD_CLEAR] = "CLEAR",
> + [GITS_CMD_SYNC] = "SYNC",
> +};
> +
> +struct its_cmd_desc {
> + union {
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_inv_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_int_cmd;
> +
> + struct {
> + struct its_device *dev;
> + bool valid;
> + } its_mapd_cmd;
> +
> + struct {
> + struct its_collection *col;
> + bool valid;
> + } its_mapc_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 phys_id;
> + u32 event_id;
> + u32 col_id;
> + } its_mapti_cmd;
> +
> + struct {
> + struct its_device *dev;
> + struct its_collection *col;
> + u32 event_id;
> + } its_movi_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_discard_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_clear_cmd;
> +
> + struct {
> + struct its_collection *col;
> + } its_invall_cmd;
> +
> + struct {
> + struct its_collection *col;
> + } its_sync_cmd;
> + };
> +};
> +
> +typedef void (*its_cmd_builder_t)(struct its_cmd_block *,
> + struct its_cmd_desc *);
> +
> +/* ITS COMMANDS */
> +
> +static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
> +{
> + cmd->raw_cmd[0] &= ~0xffUL;
> + cmd->raw_cmd[0] |= cmd_nr;
> +}
> +
> +static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
> +{
> + cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
> + cmd->raw_cmd[0] |= ((u64)devid) << 32;
> +}
> +
> +static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
> +{
> + cmd->raw_cmd[1] &= ~0xffffffffUL;
> + cmd->raw_cmd[1] |= id;
> +}
> +
> +static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
> +{
> + cmd->raw_cmd[1] &= 0xffffffffUL;
> + cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
> +}
> +
> +static void its_encode_size(struct its_cmd_block *cmd, u8 size)
> +{
> + cmd->raw_cmd[1] &= ~0x1fUL;
> + cmd->raw_cmd[1] |= size & 0x1f;
> +}
> +
> +static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
> +{
> + cmd->raw_cmd[2] &= ~0xffffffffffffUL;
> + cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
> +}
> +
> +static void its_encode_valid(struct its_cmd_block *cmd, int valid)
> +{
> + cmd->raw_cmd[2] &= ~(1UL << 63);
> + cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
> +}
> +
> +static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
> +{
> + cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
> + cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
> +}
> +
> +static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
> +{
> + cmd->raw_cmd[2] &= ~0xffffUL;
> + cmd->raw_cmd[2] |= col;
> +}
> +
> +static inline void its_fixup_cmd(struct its_cmd_block *cmd)
> +{
> + /* Let's fixup BE commands */
> + cmd->raw_cmd[0] = cpu_to_le64(cmd->raw_cmd[0]);
> + cmd->raw_cmd[1] = cpu_to_le64(cmd->raw_cmd[1]);
> + cmd->raw_cmd[2] = cpu_to_le64(cmd->raw_cmd[2]);
> + cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
> +}
> +
> +static u64 its_cmd_ptr_to_offset(struct its_cmd_block *ptr)
> +{
> + return (ptr - its_data.cmd_base) * sizeof(*ptr);
> +}
> +
> +static struct its_cmd_block *its_post_commands(void)
> +{
> + u64 wr = its_cmd_ptr_to_offset(its_data.cmd_write);
> +
> + writeq(wr, its_data.base + GITS_CWRITER);
> + return its_data.cmd_write;
> +}
> +
> +
extra blank line
> +static struct its_cmd_block *its_allocate_entry(void)
> +{
> + struct its_cmd_block *cmd;
> +
> + assert((u64)its_data.cmd_write < (u64)its_data.cmd_base + SZ_64K);
> + cmd = its_data.cmd_write++;
> + return cmd;
> +}
> +
> +static void its_wait_for_range_completion(struct its_cmd_block *from,
> + struct its_cmd_block *to)
> +{
> + u64 rd_idx, from_idx, to_idx;
> + u32 count = 1000000; /* 1s! */
> +
> + from_idx = its_cmd_ptr_to_offset(from);
> + to_idx = its_cmd_ptr_to_offset(to);
> + while (1) {
> + rd_idx = readq(its_data.base + GITS_CREADR);
> + if (rd_idx >= to_idx || rd_idx < from_idx)
> + break;
> +
> + count--;
> + if (!count) {
> + unsigned int cmd_id = from->raw_cmd[0] & 0xFF;
> +
> + assert_msg(false, "%s timeout!",
> + cmd_id <= 0xF ? its_cmd_string[cmd_id] :
> + "Unexpected");
> + return;
No need for 'return' after assert.
> + }
> + cpu_relax();
no need for cpu_relax right before udelay which calls cpu_relax
> + udelay(1);
> + }
> +}
> +
> +static void its_send_single_command(its_cmd_builder_t builder,
> + struct its_cmd_desc *desc)
> +{
> + struct its_cmd_block *cmd, *next_cmd;
> +
> + cmd = its_allocate_entry();
> + builder(cmd, desc);
> + next_cmd = its_post_commands();
> +
> + its_wait_for_range_completion(cmd, next_cmd);
> +}
> +
> +
> +static void its_build_mapd_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + unsigned long itt_addr;
> + u8 size = 12; /* 4096 eventids */
> +
> + itt_addr = (unsigned long)desc->its_mapd_cmd.dev->itt;
> + itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
> +
> + its_encode_cmd(cmd, GITS_CMD_MAPD);
> + its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
> + its_encode_size(cmd, size - 1);
> + its_encode_itt(cmd, itt_addr);
> + its_encode_valid(cmd, desc->its_mapd_cmd.valid);
> +
> + its_fixup_cmd(cmd);
> + printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
> + desc->its_mapd_cmd.dev->device_id,
> + size, itt_addr, desc->its_mapd_cmd.valid);
> +
> +}
> +
> +static void its_build_mapc_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_MAPC);
> + its_encode_collection(cmd, desc->its_mapc_cmd.col->col_id);
> + its_encode_target(cmd, desc->its_mapc_cmd.col->target_address);
> + its_encode_valid(cmd, desc->its_mapc_cmd.valid);
> +
> + its_fixup_cmd(cmd);
> + report_info("MAPC col_id=%d target_addr = 0x%lx valid=%d",
> + desc->its_mapc_cmd.col->col_id,
> + desc->its_mapc_cmd.col->target_address,
> + desc->its_mapc_cmd.valid);
printf, but better yet, leave the printing to the callers. We're in
library code here, so if a unit test doesn't want this verbosity
then they shouldn't have to have it. Same comment for the above printf
and all the below report_infos.
> +}
> +
> +static void its_build_mapti_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_MAPTI);
> + its_encode_devid(cmd, desc->its_mapti_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_mapti_cmd.event_id);
> + its_encode_phys_id(cmd, desc->its_mapti_cmd.phys_id);
> + its_encode_collection(cmd, desc->its_mapti_cmd.col_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("MAPTI dev_id=%d event_id=%d -> phys_id=%d, col_id=%d",
> + desc->its_mapti_cmd.dev->device_id,
> + desc->its_mapti_cmd.event_id,
> + desc->its_mapti_cmd.phys_id,
> + desc->its_mapti_cmd.col_id);
> +}
> +
> +static void its_build_invall_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_INVALL);
> + its_encode_collection(cmd, desc->its_invall_cmd.col->col_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("INVALL col_id=%d", desc->its_invall_cmd.col->col_id);
> +}
> +
> +static void its_build_clear_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_CLEAR);
> + its_encode_devid(cmd, desc->its_clear_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_clear_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("CLEAR col_id=%d", desc->its_invall_cmd.col->col_id);
> +}
> +
> +static void its_build_discard_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_DISCARD);
> + its_encode_devid(cmd, desc->its_discard_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_discard_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("DISCARD col_id=%d", desc->its_invall_cmd.col->col_id);
> +}
> +
> +static void its_build_inv_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_INV);
> + its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("INV dev_id=%d event_id=%d",
> + desc->its_inv_cmd.dev->device_id,
> + desc->its_inv_cmd.event_id);
> +}
> +
> +static void its_build_int_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_INT);
> + its_encode_devid(cmd, desc->its_int_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_int_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("INT dev_id=%d event_id=%d",
> + desc->its_int_cmd.dev->device_id,
> + desc->its_int_cmd.event_id);
> +}
> +
> +static void its_build_sync_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_SYNC);
> + its_encode_target(cmd, desc->its_sync_cmd.col->target_address);
> + its_fixup_cmd(cmd);
All the rest of the blocks have a blank line before its_fixup_cmd,
but I actually like this one better. The blanks are unnecessary.
> + report_info("SYNC target_addr = 0x%lx",
> + desc->its_sync_cmd.col->target_address);
> +}
> +
> +static void its_build_movi_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_MOVI);
> + its_encode_devid(cmd, desc->its_movi_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_movi_cmd.event_id);
> + its_encode_collection(cmd, desc->its_movi_cmd.col->col_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("MOVI dev_id=%d event_id = %d col_id=%d",
> + desc->its_movi_cmd.dev->device_id,
> + desc->its_movi_cmd.event_id,
> + desc->its_movi_cmd.col->col_id);
> +}
> +
> +void its_send_mapd(struct its_device *dev, int valid)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_mapd_cmd.dev = dev;
> + desc.its_mapd_cmd.valid = !!valid;
> +
> + its_send_single_command(its_build_mapd_cmd, &desc);
> +}
> +
> +void its_send_mapc(struct its_collection *col, int valid)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_mapc_cmd.col = col;
> + desc.its_mapc_cmd.valid = !!valid;
> +
> + its_send_single_command(its_build_mapc_cmd, &desc);
> +}
> +
> +void its_send_mapti(struct its_device *dev, u32 irq_id,
> + u32 event_id, struct its_collection *col)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_mapti_cmd.dev = dev;
> + desc.its_mapti_cmd.phys_id = irq_id;
> + desc.its_mapti_cmd.event_id = event_id;
> + desc.its_mapti_cmd.col_id = col->col_id;
> +
> + its_send_single_command(its_build_mapti_cmd, &desc);
> +}
> +
> +void its_send_int(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_int_cmd.dev = dev;
> + desc.its_int_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_int_cmd, &desc);
> +}
> +
> +void its_send_movi(struct its_device *dev,
> + struct its_collection *col, u32 id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_movi_cmd.dev = dev;
> + desc.its_movi_cmd.col = col;
> + desc.its_movi_cmd.event_id = id;
> +
> + its_send_single_command(its_build_movi_cmd, &desc);
> +}
> +
> +void its_send_invall(struct its_collection *col)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_invall_cmd.col = col;
> +
> + its_send_single_command(its_build_invall_cmd, &desc);
> +}
> +
> +void its_send_inv(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_inv_cmd.dev = dev;
> + desc.its_inv_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_inv_cmd, &desc);
> +}
> +
> +void its_send_discard(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_discard_cmd.dev = dev;
> + desc.its_discard_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_discard_cmd, &desc);
> +}
> +
> +void its_send_clear(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_clear_cmd.dev = dev;
> + desc.its_clear_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_clear_cmd, &desc);
> +}
> +
> +void its_send_sync(struct its_collection *col)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_sync_cmd.col = col;
> +
> + its_send_single_command(its_build_sync_cmd, &desc);
> +}
> +
> --
> 2.20.1
>
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 10/14] arm/arm64: ITS: commands
@ 2020-02-07 13:37 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 13:37 UTC (permalink / raw)
To: Eric Auger
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:55AM +0100, Eric Auger wrote:
> Implement main ITS commands. The code is largely inherited from
> the ITS driver.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - do not use report() anymore
> - assert if cmd_write exceeds the queue capacity
>
> v1 -> v2:
> - removed its_print_cmd_state
> ---
> arm/Makefile.arm64 | 2 +-
> lib/arm/asm/gic-v3-its.h | 38 +++-
> lib/arm/gic-v3-its-cmd.c | 454 +++++++++++++++++++++++++++++++++++++++
> 3 files changed, 492 insertions(+), 2 deletions(-)
> create mode 100644 lib/arm/gic-v3-its-cmd.c
>
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 2571ffb..d12aea5 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -19,7 +19,7 @@ endef
> cstart.o = $(TEST_DIR)/cstart64.o
> cflatobjs += lib/arm64/processor.o
> cflatobjs += lib/arm64/spinlock.o
> -cflatobjs += lib/arm/gic-v3-its.o
> +cflatobjs += lib/arm/gic-v3-its.o lib/arm/gic-v3-its-cmd.o
>
> OBJDIRS += lib/arm64
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index acd97a9..0e5c5b6 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -45,6 +45,8 @@ struct its_collection {
> u16 col_id;
> };
>
> +struct its_cmd_block;
> +
This isn't necessary. If it was, then it should have been added in a
previous patch.
> struct its_data {
> void *base;
> struct its_typer typer;
> @@ -107,6 +109,24 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> +/*
> + * ITS commands
> + */
> +#define GITS_CMD_MAPD 0x08
> +#define GITS_CMD_MAPC 0x09
> +#define GITS_CMD_MAPTI 0x0a
> +/* older GIC documentation used MAPVI for this command */
> +#define GITS_CMD_MAPVI GITS_CMD_MAPTI
> +#define GITS_CMD_MAPI 0x0b
> +#define GITS_CMD_MOVI 0x01
> +#define GITS_CMD_DISCARD 0x0f
> +#define GITS_CMD_INV 0x0c
> +#define GITS_CMD_MOVALL 0x0e
> +#define GITS_CMD_INVALL 0x0d
> +#define GITS_CMD_INT 0x03
> +#define GITS_CMD_CLEAR 0x04
> +#define GITS_CMD_SYNC 0x05
Please use tabs.
> +
> struct its_cmd_block {
> u64 raw_cmd[4];
> };
> @@ -119,11 +139,27 @@ extern void its_enable_defaults(void);
> extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>
> +extern void its_send_mapd(struct its_device *dev, int valid);
> +extern void its_send_mapc(struct its_collection *col, int valid);
> +extern void its_send_mapti(struct its_device *dev, u32 irq_id,
> + u32 event_id, struct its_collection *col);
> +extern void its_send_int(struct its_device *dev, u32 event_id);
> +extern void its_send_inv(struct its_device *dev, u32 event_id);
> +extern void its_send_discard(struct its_device *dev, u32 event_id);
> +extern void its_send_clear(struct its_device *dev, u32 event_id);
> +extern void its_send_invall(struct its_collection *col);
> +extern void its_send_movi(struct its_device *dev,
> + struct its_collection *col, u32 id);
> +extern void its_send_sync(struct its_collection *col);
> +
> +#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
> +#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
> +#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)
What are these flags for?
> +
> #else /* __arm__ */
>
> static inline void its_init(void) {}
>
> #endif
> -
> #endif /* !__ASSEMBLY__ */
> #endif /* _ASMARM_GIC_V3_ITS_H_ */
> diff --git a/lib/arm/gic-v3-its-cmd.c b/lib/arm/gic-v3-its-cmd.c
> new file mode 100644
> index 0000000..fb4364c
> --- /dev/null
> +++ b/lib/arm/gic-v3-its-cmd.c
> @@ -0,0 +1,454 @@
> +/*
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * Most of the code is copy-pasted from:
> + * drivers/irqchip/irq-gic-v3-its.c
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <asm/io.h>
> +#include <asm/gic.h>
> +#include <asm/gic-v3-its.h>
> +
> +#define ITS_ITT_ALIGN SZ_256
tabs
> +
> +static const char * const its_cmd_string[] = {
> + [GITS_CMD_MAPD] = "MAPD",
> + [GITS_CMD_MAPC] = "MAPC",
> + [GITS_CMD_MAPTI] = "MAPTI",
> + [GITS_CMD_MAPI] = "MAPI",
> + [GITS_CMD_MOVI] = "MOVI",
> + [GITS_CMD_DISCARD] = "DISCARD",
> + [GITS_CMD_INV] = "INV",
> + [GITS_CMD_MOVALL] = "MOVALL",
> + [GITS_CMD_INVALL] = "INVALL",
> + [GITS_CMD_INT] = "INT",
> + [GITS_CMD_CLEAR] = "CLEAR",
> + [GITS_CMD_SYNC] = "SYNC",
> +};
> +
> +struct its_cmd_desc {
> + union {
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_inv_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_int_cmd;
> +
> + struct {
> + struct its_device *dev;
> + bool valid;
> + } its_mapd_cmd;
> +
> + struct {
> + struct its_collection *col;
> + bool valid;
> + } its_mapc_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 phys_id;
> + u32 event_id;
> + u32 col_id;
> + } its_mapti_cmd;
> +
> + struct {
> + struct its_device *dev;
> + struct its_collection *col;
> + u32 event_id;
> + } its_movi_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_discard_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_clear_cmd;
> +
> + struct {
> + struct its_collection *col;
> + } its_invall_cmd;
> +
> + struct {
> + struct its_collection *col;
> + } its_sync_cmd;
> + };
> +};
> +
> +typedef void (*its_cmd_builder_t)(struct its_cmd_block *,
> + struct its_cmd_desc *);
> +
> +/* ITS COMMANDS */
> +
> +static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
> +{
> + cmd->raw_cmd[0] &= ~0xffUL;
> + cmd->raw_cmd[0] |= cmd_nr;
> +}
> +
> +static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
> +{
> + cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
> + cmd->raw_cmd[0] |= ((u64)devid) << 32;
> +}
> +
> +static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
> +{
> + cmd->raw_cmd[1] &= ~0xffffffffUL;
> + cmd->raw_cmd[1] |= id;
> +}
> +
> +static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
> +{
> + cmd->raw_cmd[1] &= 0xffffffffUL;
> + cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
> +}
> +
> +static void its_encode_size(struct its_cmd_block *cmd, u8 size)
> +{
> + cmd->raw_cmd[1] &= ~0x1fUL;
> + cmd->raw_cmd[1] |= size & 0x1f;
> +}
> +
> +static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
> +{
> + cmd->raw_cmd[2] &= ~0xffffffffffffUL;
> + cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
> +}
> +
> +static void its_encode_valid(struct its_cmd_block *cmd, int valid)
> +{
> + cmd->raw_cmd[2] &= ~(1UL << 63);
> + cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
> +}
> +
> +static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
> +{
> + cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
> + cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
> +}
> +
> +static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
> +{
> + cmd->raw_cmd[2] &= ~0xffffUL;
> + cmd->raw_cmd[2] |= col;
> +}
> +
> +static inline void its_fixup_cmd(struct its_cmd_block *cmd)
> +{
> + /* Let's fixup BE commands */
> + cmd->raw_cmd[0] = cpu_to_le64(cmd->raw_cmd[0]);
> + cmd->raw_cmd[1] = cpu_to_le64(cmd->raw_cmd[1]);
> + cmd->raw_cmd[2] = cpu_to_le64(cmd->raw_cmd[2]);
> + cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
> +}
> +
> +static u64 its_cmd_ptr_to_offset(struct its_cmd_block *ptr)
> +{
> + return (ptr - its_data.cmd_base) * sizeof(*ptr);
> +}
> +
> +static struct its_cmd_block *its_post_commands(void)
> +{
> + u64 wr = its_cmd_ptr_to_offset(its_data.cmd_write);
> +
> + writeq(wr, its_data.base + GITS_CWRITER);
> + return its_data.cmd_write;
> +}
> +
> +
extra blank line
> +static struct its_cmd_block *its_allocate_entry(void)
> +{
> + struct its_cmd_block *cmd;
> +
> + assert((u64)its_data.cmd_write < (u64)its_data.cmd_base + SZ_64K);
> + cmd = its_data.cmd_write++;
> + return cmd;
> +}
> +
> +static void its_wait_for_range_completion(struct its_cmd_block *from,
> + struct its_cmd_block *to)
> +{
> + u64 rd_idx, from_idx, to_idx;
> + u32 count = 1000000; /* 1s! */
> +
> + from_idx = its_cmd_ptr_to_offset(from);
> + to_idx = its_cmd_ptr_to_offset(to);
> + while (1) {
> + rd_idx = readq(its_data.base + GITS_CREADR);
> + if (rd_idx >= to_idx || rd_idx < from_idx)
> + break;
> +
> + count--;
> + if (!count) {
> + unsigned int cmd_id = from->raw_cmd[0] & 0xFF;
> +
> + assert_msg(false, "%s timeout!",
> + cmd_id <= 0xF ? its_cmd_string[cmd_id] :
> + "Unexpected");
> + return;
No need for 'return' after assert.
> + }
> + cpu_relax();
no need for cpu_relax right before udelay which calls cpu_relax
> + udelay(1);
> + }
> +}
> +
> +static void its_send_single_command(its_cmd_builder_t builder,
> + struct its_cmd_desc *desc)
> +{
> + struct its_cmd_block *cmd, *next_cmd;
> +
> + cmd = its_allocate_entry();
> + builder(cmd, desc);
> + next_cmd = its_post_commands();
> +
> + its_wait_for_range_completion(cmd, next_cmd);
> +}
> +
> +
> +static void its_build_mapd_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + unsigned long itt_addr;
> + u8 size = 12; /* 4096 eventids */
> +
> + itt_addr = (unsigned long)desc->its_mapd_cmd.dev->itt;
> + itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
> +
> + its_encode_cmd(cmd, GITS_CMD_MAPD);
> + its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
> + its_encode_size(cmd, size - 1);
> + its_encode_itt(cmd, itt_addr);
> + its_encode_valid(cmd, desc->its_mapd_cmd.valid);
> +
> + its_fixup_cmd(cmd);
> + printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
> + desc->its_mapd_cmd.dev->device_id,
> + size, itt_addr, desc->its_mapd_cmd.valid);
> +
> +}
> +
> +static void its_build_mapc_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_MAPC);
> + its_encode_collection(cmd, desc->its_mapc_cmd.col->col_id);
> + its_encode_target(cmd, desc->its_mapc_cmd.col->target_address);
> + its_encode_valid(cmd, desc->its_mapc_cmd.valid);
> +
> + its_fixup_cmd(cmd);
> + report_info("MAPC col_id=%d target_addr = 0x%lx valid=%d",
> + desc->its_mapc_cmd.col->col_id,
> + desc->its_mapc_cmd.col->target_address,
> + desc->its_mapc_cmd.valid);
printf, but better yet, leave the printing to the callers. We're in
library code here, so if a unit test doesn't want this verbosity
then they shouldn't have to have it. Same comment for the above printf
and all the below report_infos.
> +}
> +
> +static void its_build_mapti_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_MAPTI);
> + its_encode_devid(cmd, desc->its_mapti_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_mapti_cmd.event_id);
> + its_encode_phys_id(cmd, desc->its_mapti_cmd.phys_id);
> + its_encode_collection(cmd, desc->its_mapti_cmd.col_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("MAPTI dev_id=%d event_id=%d -> phys_id=%d, col_id=%d",
> + desc->its_mapti_cmd.dev->device_id,
> + desc->its_mapti_cmd.event_id,
> + desc->its_mapti_cmd.phys_id,
> + desc->its_mapti_cmd.col_id);
> +}
> +
> +static void its_build_invall_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_INVALL);
> + its_encode_collection(cmd, desc->its_invall_cmd.col->col_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("INVALL col_id=%d", desc->its_invall_cmd.col->col_id);
> +}
> +
> +static void its_build_clear_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_CLEAR);
> + its_encode_devid(cmd, desc->its_clear_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_clear_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("CLEAR col_id=%d", desc->its_invall_cmd.col->col_id);
> +}
> +
> +static void its_build_discard_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_DISCARD);
> + its_encode_devid(cmd, desc->its_discard_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_discard_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("DISCARD col_id=%d", desc->its_invall_cmd.col->col_id);
> +}
> +
> +static void its_build_inv_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_INV);
> + its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("INV dev_id=%d event_id=%d",
> + desc->its_inv_cmd.dev->device_id,
> + desc->its_inv_cmd.event_id);
> +}
> +
> +static void its_build_int_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_INT);
> + its_encode_devid(cmd, desc->its_int_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_int_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("INT dev_id=%d event_id=%d",
> + desc->its_int_cmd.dev->device_id,
> + desc->its_int_cmd.event_id);
> +}
> +
> +static void its_build_sync_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_SYNC);
> + its_encode_target(cmd, desc->its_sync_cmd.col->target_address);
> + its_fixup_cmd(cmd);
All the rest of the blocks have a blank line before its_fixup_cmd,
but I actually like this one better. The blanks are unnecessary.
> + report_info("SYNC target_addr = 0x%lx",
> + desc->its_sync_cmd.col->target_address);
> +}
> +
> +static void its_build_movi_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_MOVI);
> + its_encode_devid(cmd, desc->its_movi_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_movi_cmd.event_id);
> + its_encode_collection(cmd, desc->its_movi_cmd.col->col_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("MOVI dev_id=%d event_id = %d col_id=%d",
> + desc->its_movi_cmd.dev->device_id,
> + desc->its_movi_cmd.event_id,
> + desc->its_movi_cmd.col->col_id);
> +}
> +
> +void its_send_mapd(struct its_device *dev, int valid)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_mapd_cmd.dev = dev;
> + desc.its_mapd_cmd.valid = !!valid;
> +
> + its_send_single_command(its_build_mapd_cmd, &desc);
> +}
> +
> +void its_send_mapc(struct its_collection *col, int valid)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_mapc_cmd.col = col;
> + desc.its_mapc_cmd.valid = !!valid;
> +
> + its_send_single_command(its_build_mapc_cmd, &desc);
> +}
> +
> +void its_send_mapti(struct its_device *dev, u32 irq_id,
> + u32 event_id, struct its_collection *col)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_mapti_cmd.dev = dev;
> + desc.its_mapti_cmd.phys_id = irq_id;
> + desc.its_mapti_cmd.event_id = event_id;
> + desc.its_mapti_cmd.col_id = col->col_id;
> +
> + its_send_single_command(its_build_mapti_cmd, &desc);
> +}
> +
> +void its_send_int(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_int_cmd.dev = dev;
> + desc.its_int_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_int_cmd, &desc);
> +}
> +
> +void its_send_movi(struct its_device *dev,
> + struct its_collection *col, u32 id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_movi_cmd.dev = dev;
> + desc.its_movi_cmd.col = col;
> + desc.its_movi_cmd.event_id = id;
> +
> + its_send_single_command(its_build_movi_cmd, &desc);
> +}
> +
> +void its_send_invall(struct its_collection *col)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_invall_cmd.col = col;
> +
> + its_send_single_command(its_build_invall_cmd, &desc);
> +}
> +
> +void its_send_inv(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_inv_cmd.dev = dev;
> + desc.its_inv_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_inv_cmd, &desc);
> +}
> +
> +void its_send_discard(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_discard_cmd.dev = dev;
> + desc.its_discard_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_discard_cmd, &desc);
> +}
> +
> +void its_send_clear(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_clear_cmd.dev = dev;
> + desc.its_clear_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_clear_cmd, &desc);
> +}
> +
> +void its_send_sync(struct its_collection *col)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_sync_cmd.col = col;
> +
> + its_send_single_command(its_build_sync_cmd, &desc);
> +}
> +
> --
> 2.20.1
>
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 10/14] arm/arm64: ITS: commands
@ 2020-02-07 13:37 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 13:37 UTC (permalink / raw)
To: Eric Auger
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:55AM +0100, Eric Auger wrote:
> Implement main ITS commands. The code is largely inherited from
> the ITS driver.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - do not use report() anymore
> - assert if cmd_write exceeds the queue capacity
>
> v1 -> v2:
> - removed its_print_cmd_state
> ---
> arm/Makefile.arm64 | 2 +-
> lib/arm/asm/gic-v3-its.h | 38 +++-
> lib/arm/gic-v3-its-cmd.c | 454 +++++++++++++++++++++++++++++++++++++++
> 3 files changed, 492 insertions(+), 2 deletions(-)
> create mode 100644 lib/arm/gic-v3-its-cmd.c
>
> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
> index 2571ffb..d12aea5 100644
> --- a/arm/Makefile.arm64
> +++ b/arm/Makefile.arm64
> @@ -19,7 +19,7 @@ endef
> cstart.o = $(TEST_DIR)/cstart64.o
> cflatobjs += lib/arm64/processor.o
> cflatobjs += lib/arm64/spinlock.o
> -cflatobjs += lib/arm/gic-v3-its.o
> +cflatobjs += lib/arm/gic-v3-its.o lib/arm/gic-v3-its-cmd.o
>
> OBJDIRS += lib/arm64
>
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index acd97a9..0e5c5b6 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -45,6 +45,8 @@ struct its_collection {
> u16 col_id;
> };
>
> +struct its_cmd_block;
> +
This isn't necessary. If it was, then it should have been added in a
previous patch.
> struct its_data {
> void *base;
> struct its_typer typer;
> @@ -107,6 +109,24 @@ extern struct its_data its_data;
> #define GITS_BASER_TYPE_DEVICE 1
> #define GITS_BASER_TYPE_COLLECTION 4
>
> +/*
> + * ITS commands
> + */
> +#define GITS_CMD_MAPD 0x08
> +#define GITS_CMD_MAPC 0x09
> +#define GITS_CMD_MAPTI 0x0a
> +/* older GIC documentation used MAPVI for this command */
> +#define GITS_CMD_MAPVI GITS_CMD_MAPTI
> +#define GITS_CMD_MAPI 0x0b
> +#define GITS_CMD_MOVI 0x01
> +#define GITS_CMD_DISCARD 0x0f
> +#define GITS_CMD_INV 0x0c
> +#define GITS_CMD_MOVALL 0x0e
> +#define GITS_CMD_INVALL 0x0d
> +#define GITS_CMD_INT 0x03
> +#define GITS_CMD_CLEAR 0x04
> +#define GITS_CMD_SYNC 0x05
Please use tabs.
> +
> struct its_cmd_block {
> u64 raw_cmd[4];
> };
> @@ -119,11 +139,27 @@ extern void its_enable_defaults(void);
> extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>
> +extern void its_send_mapd(struct its_device *dev, int valid);
> +extern void its_send_mapc(struct its_collection *col, int valid);
> +extern void its_send_mapti(struct its_device *dev, u32 irq_id,
> + u32 event_id, struct its_collection *col);
> +extern void its_send_int(struct its_device *dev, u32 event_id);
> +extern void its_send_inv(struct its_device *dev, u32 event_id);
> +extern void its_send_discard(struct its_device *dev, u32 event_id);
> +extern void its_send_clear(struct its_device *dev, u32 event_id);
> +extern void its_send_invall(struct its_collection *col);
> +extern void its_send_movi(struct its_device *dev,
> + struct its_collection *col, u32 id);
> +extern void its_send_sync(struct its_collection *col);
> +
> +#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
> +#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
> +#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)
What are these flags for?
> +
> #else /* __arm__ */
>
> static inline void its_init(void) {}
>
> #endif
> -
> #endif /* !__ASSEMBLY__ */
> #endif /* _ASMARM_GIC_V3_ITS_H_ */
> diff --git a/lib/arm/gic-v3-its-cmd.c b/lib/arm/gic-v3-its-cmd.c
> new file mode 100644
> index 0000000..fb4364c
> --- /dev/null
> +++ b/lib/arm/gic-v3-its-cmd.c
> @@ -0,0 +1,454 @@
> +/*
> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
> + *
> + * Most of the code is copy-pasted from:
> + * drivers/irqchip/irq-gic-v3-its.c
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include <asm/io.h>
> +#include <asm/gic.h>
> +#include <asm/gic-v3-its.h>
> +
> +#define ITS_ITT_ALIGN SZ_256
tabs
> +
> +static const char * const its_cmd_string[] = {
> + [GITS_CMD_MAPD] = "MAPD",
> + [GITS_CMD_MAPC] = "MAPC",
> + [GITS_CMD_MAPTI] = "MAPTI",
> + [GITS_CMD_MAPI] = "MAPI",
> + [GITS_CMD_MOVI] = "MOVI",
> + [GITS_CMD_DISCARD] = "DISCARD",
> + [GITS_CMD_INV] = "INV",
> + [GITS_CMD_MOVALL] = "MOVALL",
> + [GITS_CMD_INVALL] = "INVALL",
> + [GITS_CMD_INT] = "INT",
> + [GITS_CMD_CLEAR] = "CLEAR",
> + [GITS_CMD_SYNC] = "SYNC",
> +};
> +
> +struct its_cmd_desc {
> + union {
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_inv_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_int_cmd;
> +
> + struct {
> + struct its_device *dev;
> + bool valid;
> + } its_mapd_cmd;
> +
> + struct {
> + struct its_collection *col;
> + bool valid;
> + } its_mapc_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 phys_id;
> + u32 event_id;
> + u32 col_id;
> + } its_mapti_cmd;
> +
> + struct {
> + struct its_device *dev;
> + struct its_collection *col;
> + u32 event_id;
> + } its_movi_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_discard_cmd;
> +
> + struct {
> + struct its_device *dev;
> + u32 event_id;
> + } its_clear_cmd;
> +
> + struct {
> + struct its_collection *col;
> + } its_invall_cmd;
> +
> + struct {
> + struct its_collection *col;
> + } its_sync_cmd;
> + };
> +};
> +
> +typedef void (*its_cmd_builder_t)(struct its_cmd_block *,
> + struct its_cmd_desc *);
> +
> +/* ITS COMMANDS */
> +
> +static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
> +{
> + cmd->raw_cmd[0] &= ~0xffUL;
> + cmd->raw_cmd[0] |= cmd_nr;
> +}
> +
> +static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
> +{
> + cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
> + cmd->raw_cmd[0] |= ((u64)devid) << 32;
> +}
> +
> +static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
> +{
> + cmd->raw_cmd[1] &= ~0xffffffffUL;
> + cmd->raw_cmd[1] |= id;
> +}
> +
> +static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
> +{
> + cmd->raw_cmd[1] &= 0xffffffffUL;
> + cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
> +}
> +
> +static void its_encode_size(struct its_cmd_block *cmd, u8 size)
> +{
> + cmd->raw_cmd[1] &= ~0x1fUL;
> + cmd->raw_cmd[1] |= size & 0x1f;
> +}
> +
> +static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
> +{
> + cmd->raw_cmd[2] &= ~0xffffffffffffUL;
> + cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
> +}
> +
> +static void its_encode_valid(struct its_cmd_block *cmd, int valid)
> +{
> + cmd->raw_cmd[2] &= ~(1UL << 63);
> + cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
> +}
> +
> +static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
> +{
> + cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
> + cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
> +}
> +
> +static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
> +{
> + cmd->raw_cmd[2] &= ~0xffffUL;
> + cmd->raw_cmd[2] |= col;
> +}
> +
> +static inline void its_fixup_cmd(struct its_cmd_block *cmd)
> +{
> + /* Let's fixup BE commands */
> + cmd->raw_cmd[0] = cpu_to_le64(cmd->raw_cmd[0]);
> + cmd->raw_cmd[1] = cpu_to_le64(cmd->raw_cmd[1]);
> + cmd->raw_cmd[2] = cpu_to_le64(cmd->raw_cmd[2]);
> + cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
> +}
> +
> +static u64 its_cmd_ptr_to_offset(struct its_cmd_block *ptr)
> +{
> + return (ptr - its_data.cmd_base) * sizeof(*ptr);
> +}
> +
> +static struct its_cmd_block *its_post_commands(void)
> +{
> + u64 wr = its_cmd_ptr_to_offset(its_data.cmd_write);
> +
> + writeq(wr, its_data.base + GITS_CWRITER);
> + return its_data.cmd_write;
> +}
> +
> +
extra blank line
> +static struct its_cmd_block *its_allocate_entry(void)
> +{
> + struct its_cmd_block *cmd;
> +
> + assert((u64)its_data.cmd_write < (u64)its_data.cmd_base + SZ_64K);
> + cmd = its_data.cmd_write++;
> + return cmd;
> +}
> +
> +static void its_wait_for_range_completion(struct its_cmd_block *from,
> + struct its_cmd_block *to)
> +{
> + u64 rd_idx, from_idx, to_idx;
> + u32 count = 1000000; /* 1s! */
> +
> + from_idx = its_cmd_ptr_to_offset(from);
> + to_idx = its_cmd_ptr_to_offset(to);
> + while (1) {
> + rd_idx = readq(its_data.base + GITS_CREADR);
> + if (rd_idx >= to_idx || rd_idx < from_idx)
> + break;
> +
> + count--;
> + if (!count) {
> + unsigned int cmd_id = from->raw_cmd[0] & 0xFF;
> +
> + assert_msg(false, "%s timeout!",
> + cmd_id <= 0xF ? its_cmd_string[cmd_id] :
> + "Unexpected");
> + return;
No need for 'return' after assert.
> + }
> + cpu_relax();
no need for cpu_relax right before udelay which calls cpu_relax
> + udelay(1);
> + }
> +}
> +
> +static void its_send_single_command(its_cmd_builder_t builder,
> + struct its_cmd_desc *desc)
> +{
> + struct its_cmd_block *cmd, *next_cmd;
> +
> + cmd = its_allocate_entry();
> + builder(cmd, desc);
> + next_cmd = its_post_commands();
> +
> + its_wait_for_range_completion(cmd, next_cmd);
> +}
> +
> +
> +static void its_build_mapd_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + unsigned long itt_addr;
> + u8 size = 12; /* 4096 eventids */
> +
> + itt_addr = (unsigned long)desc->its_mapd_cmd.dev->itt;
> + itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
> +
> + its_encode_cmd(cmd, GITS_CMD_MAPD);
> + its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
> + its_encode_size(cmd, size - 1);
> + its_encode_itt(cmd, itt_addr);
> + its_encode_valid(cmd, desc->its_mapd_cmd.valid);
> +
> + its_fixup_cmd(cmd);
> + printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
> + desc->its_mapd_cmd.dev->device_id,
> + size, itt_addr, desc->its_mapd_cmd.valid);
> +
> +}
> +
> +static void its_build_mapc_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_MAPC);
> + its_encode_collection(cmd, desc->its_mapc_cmd.col->col_id);
> + its_encode_target(cmd, desc->its_mapc_cmd.col->target_address);
> + its_encode_valid(cmd, desc->its_mapc_cmd.valid);
> +
> + its_fixup_cmd(cmd);
> + report_info("MAPC col_id=%d target_addr = 0x%lx valid=%d",
> + desc->its_mapc_cmd.col->col_id,
> + desc->its_mapc_cmd.col->target_address,
> + desc->its_mapc_cmd.valid);
printf, but better yet, leave the printing to the callers. We're in
library code here, so if a unit test doesn't want this verbosity
then they shouldn't have to have it. Same comment for the above printf
and all the below report_infos.
> +}
> +
> +static void its_build_mapti_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_MAPTI);
> + its_encode_devid(cmd, desc->its_mapti_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_mapti_cmd.event_id);
> + its_encode_phys_id(cmd, desc->its_mapti_cmd.phys_id);
> + its_encode_collection(cmd, desc->its_mapti_cmd.col_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("MAPTI dev_id=%d event_id=%d -> phys_id=%d, col_id=%d",
> + desc->its_mapti_cmd.dev->device_id,
> + desc->its_mapti_cmd.event_id,
> + desc->its_mapti_cmd.phys_id,
> + desc->its_mapti_cmd.col_id);
> +}
> +
> +static void its_build_invall_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_INVALL);
> + its_encode_collection(cmd, desc->its_invall_cmd.col->col_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("INVALL col_id=%d", desc->its_invall_cmd.col->col_id);
> +}
> +
> +static void its_build_clear_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_CLEAR);
> + its_encode_devid(cmd, desc->its_clear_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_clear_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("CLEAR col_id=%d", desc->its_invall_cmd.col->col_id);
> +}
> +
> +static void its_build_discard_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_DISCARD);
> + its_encode_devid(cmd, desc->its_discard_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_discard_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("DISCARD col_id=%d", desc->its_invall_cmd.col->col_id);
> +}
> +
> +static void its_build_inv_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_INV);
> + its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("INV dev_id=%d event_id=%d",
> + desc->its_inv_cmd.dev->device_id,
> + desc->its_inv_cmd.event_id);
> +}
> +
> +static void its_build_int_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_INT);
> + its_encode_devid(cmd, desc->its_int_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_int_cmd.event_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("INT dev_id=%d event_id=%d",
> + desc->its_int_cmd.dev->device_id,
> + desc->its_int_cmd.event_id);
> +}
> +
> +static void its_build_sync_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_SYNC);
> + its_encode_target(cmd, desc->its_sync_cmd.col->target_address);
> + its_fixup_cmd(cmd);
All the rest of the blocks have a blank line before its_fixup_cmd,
but I actually like this one better. The blanks are unnecessary.
> + report_info("SYNC target_addr = 0x%lx",
> + desc->its_sync_cmd.col->target_address);
> +}
> +
> +static void its_build_movi_cmd(struct its_cmd_block *cmd,
> + struct its_cmd_desc *desc)
> +{
> + its_encode_cmd(cmd, GITS_CMD_MOVI);
> + its_encode_devid(cmd, desc->its_movi_cmd.dev->device_id);
> + its_encode_event_id(cmd, desc->its_movi_cmd.event_id);
> + its_encode_collection(cmd, desc->its_movi_cmd.col->col_id);
> +
> + its_fixup_cmd(cmd);
> + report_info("MOVI dev_id=%d event_id = %d col_id=%d",
> + desc->its_movi_cmd.dev->device_id,
> + desc->its_movi_cmd.event_id,
> + desc->its_movi_cmd.col->col_id);
> +}
> +
> +void its_send_mapd(struct its_device *dev, int valid)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_mapd_cmd.dev = dev;
> + desc.its_mapd_cmd.valid = !!valid;
> +
> + its_send_single_command(its_build_mapd_cmd, &desc);
> +}
> +
> +void its_send_mapc(struct its_collection *col, int valid)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_mapc_cmd.col = col;
> + desc.its_mapc_cmd.valid = !!valid;
> +
> + its_send_single_command(its_build_mapc_cmd, &desc);
> +}
> +
> +void its_send_mapti(struct its_device *dev, u32 irq_id,
> + u32 event_id, struct its_collection *col)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_mapti_cmd.dev = dev;
> + desc.its_mapti_cmd.phys_id = irq_id;
> + desc.its_mapti_cmd.event_id = event_id;
> + desc.its_mapti_cmd.col_id = col->col_id;
> +
> + its_send_single_command(its_build_mapti_cmd, &desc);
> +}
> +
> +void its_send_int(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_int_cmd.dev = dev;
> + desc.its_int_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_int_cmd, &desc);
> +}
> +
> +void its_send_movi(struct its_device *dev,
> + struct its_collection *col, u32 id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_movi_cmd.dev = dev;
> + desc.its_movi_cmd.col = col;
> + desc.its_movi_cmd.event_id = id;
> +
> + its_send_single_command(its_build_movi_cmd, &desc);
> +}
> +
> +void its_send_invall(struct its_collection *col)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_invall_cmd.col = col;
> +
> + its_send_single_command(its_build_invall_cmd, &desc);
> +}
> +
> +void its_send_inv(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_inv_cmd.dev = dev;
> + desc.its_inv_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_inv_cmd, &desc);
> +}
> +
> +void its_send_discard(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_discard_cmd.dev = dev;
> + desc.its_discard_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_discard_cmd, &desc);
> +}
> +
> +void its_send_clear(struct its_device *dev, u32 event_id)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_clear_cmd.dev = dev;
> + desc.its_clear_cmd.event_id = event_id;
> +
> + its_send_single_command(its_build_clear_cmd, &desc);
> +}
> +
> +void its_send_sync(struct its_collection *col)
> +{
> + struct its_cmd_desc desc;
> +
> + desc.its_sync_cmd.col = col;
> +
> + its_send_single_command(its_build_sync_cmd, &desc);
> +}
> +
> --
> 2.20.1
>
>
Thanks,
drew
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 13/14] arm/arm64: ITS: migration tests
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 13:49 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 13:49 UTC (permalink / raw)
To: Eric Auger
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
andre.przywara, peter.maydell, yuzenghui, alexandru.elisei,
thuth
On Tue, Jan 28, 2020 at 11:34:58AM +0100, Eric Auger wrote:
> This test maps LPIs (populates the device table, the collection table,
> interrupt translation tables, configuration table), migrates and make
> sure the translation is correct on the destination.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> ---
> arm/gic.c | 59 ++++++++++++++++++++++++++++++++++++----
> arm/unittests.cfg | 8 ++++++
> lib/arm/asm/gic-v3-its.h | 2 ++
> lib/arm/gic-v3-its.c | 22 +++++++++++++++
> 4 files changed, 85 insertions(+), 6 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index 50104b1..fa8626a 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -593,6 +593,7 @@ static void gic_test_mmio(void)
>
> static void test_its_introspection(void) {}
> static void test_its_trigger(void) {}
> +static void test_its_migration(void) {}
>
> #else /* __arch64__ */
>
> @@ -665,13 +666,19 @@ static bool its_prerequisites(int nb_cpus)
> return false;
> }
>
> -static void test_its_trigger(void)
> +/*
> + * Setup the configuration for those mappings:
> + * dev_id=2 event=20 -> vcpu 3, intid=8195
> + * dev_id=7 event=255 -> vcpu 2, intid=8196
> + * LPIs ready to hit
> + */
> +static int its_setup1(void)
> {
> struct its_collection *col3, *col2;
> struct its_device *dev2, *dev7;
>
> if (its_prerequisites(4))
> - return;
> + return -1;
Why not make its_setup1 a bool? Where true means success and false mean
failure?
>
> dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
> @@ -685,14 +692,10 @@ static void test_its_trigger(void)
> its_send_invall(col2);
> its_send_invall(col3);
>
> - report_prefix_push("int");
> /*
> * dev=2, eventid=20 -> lpi= 8195, col=3
> * dev=7, eventid=255 -> lpi= 8196, col=2
> - * Trigger dev2, eventid=20 and dev7, eventid=255
> - * Check both LPIs hit
> */
> -
> its_send_mapd(dev2, true);
> its_send_mapd(dev7, true);
>
> @@ -703,6 +706,23 @@ static void test_its_trigger(void)
> 20 /* event id */, col3);
> its_send_mapti(dev7, 8196 /* lpi id */,
> 255 /* event id */, col2);
> + return 0;
> +}
> +
> +static void test_its_trigger(void)
> +{
> + struct its_collection *col3, *col2;
> + struct its_device *dev2, *dev7;
> +
> + if (its_setup1())
> + return;
> +
> + col3 = its_get_collection(3);
> + col2 = its_get_collection(2);
> + dev2 = its_get_device(2);
> + dev7 = its_get_device(7);
> +
> + report_prefix_push("int");
>
> lpi_stats_expect(3, 8195);
> its_send_int(dev2, 20);
> @@ -763,6 +783,29 @@ static void test_its_trigger(void)
> check_lpi_stats();
> report_prefix_pop();
> }
> +
> +static void test_its_migration(void)
> +{
> + struct its_device *dev2, *dev7;
> +
> + if (its_setup1())
> + return;
> +
> + dev2 = its_get_device(2);
> + dev7 = its_get_device(7);
> +
> + puts("Now migrate the VM, then press a key to continue...\n");
> + (void)getchar();
> + report(true, "Migration complete");
This seems more like a report_info place. If migration fails and
we don't complete we'll never get the report FAIL anyway.
> +
> + lpi_stats_expect(3, 8195);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + lpi_stats_expect(2, 8196);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> +}
> #endif
>
> int main(int argc, char **argv)
> @@ -800,6 +843,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> test_its_trigger();
> report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-migration")) {
> + report_prefix_push(argv[1]);
> + test_its_migration();
> + report_prefix_pop();
> } else if (strcmp(argv[1], "its-introspection") == 0) {
> report_prefix_push(argv[1]);
> test_its_introspection();
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index bfafec5..8b8ec79 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
> groups = its
> arch = arm64
>
> +[its-migration]
> +file = gic.flat
> +smp = $MAX_SMP
> +accel = kvm
> +extra_params = -machine gic-version=3 -append 'its-migration'
> +groups = its migration
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index 0e5c5b6..febc2b2 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -151,6 +151,8 @@ extern void its_send_invall(struct its_collection *col);
> extern void its_send_movi(struct its_device *dev,
> struct its_collection *col, u32 id);
> extern void its_send_sync(struct its_collection *col);
> +extern struct its_device *its_get_device(u32 id);
> +extern struct its_collection *its_get_collection(u32 id);
>
> #define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
> #define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index c2dcd01..099940e 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -219,3 +219,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
> its_data.nr_collections++;
> return new;
> }
> +
> +struct its_device *its_get_device(u32 id)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_MAX_DEVICES; i++) {
> + if (its_data.devices[i].device_id == id)
> + return &its_data.devices[i];
> + }
> + return NULL;
> +}
> +
> +struct its_collection *its_get_collection(u32 id)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
> + if (its_data.collections[i].col_id == id)
> + return &its_data.collections[i];
> + }
> + return NULL;
> +}
The callers aren't checking for NULL. Should we assert here
or in the caller?
Thanks,
drew
> --
> 2.20.1
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 13/14] arm/arm64: ITS: migration tests
@ 2020-02-07 13:49 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 13:49 UTC (permalink / raw)
To: Eric Auger
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:58AM +0100, Eric Auger wrote:
> This test maps LPIs (populates the device table, the collection table,
> interrupt translation tables, configuration table), migrates and make
> sure the translation is correct on the destination.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> ---
> arm/gic.c | 59 ++++++++++++++++++++++++++++++++++++----
> arm/unittests.cfg | 8 ++++++
> lib/arm/asm/gic-v3-its.h | 2 ++
> lib/arm/gic-v3-its.c | 22 +++++++++++++++
> 4 files changed, 85 insertions(+), 6 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index 50104b1..fa8626a 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -593,6 +593,7 @@ static void gic_test_mmio(void)
>
> static void test_its_introspection(void) {}
> static void test_its_trigger(void) {}
> +static void test_its_migration(void) {}
>
> #else /* __arch64__ */
>
> @@ -665,13 +666,19 @@ static bool its_prerequisites(int nb_cpus)
> return false;
> }
>
> -static void test_its_trigger(void)
> +/*
> + * Setup the configuration for those mappings:
> + * dev_id=2 event=20 -> vcpu 3, intid=8195
> + * dev_id=7 event=255 -> vcpu 2, intid=8196
> + * LPIs ready to hit
> + */
> +static int its_setup1(void)
> {
> struct its_collection *col3, *col2;
> struct its_device *dev2, *dev7;
>
> if (its_prerequisites(4))
> - return;
> + return -1;
Why not make its_setup1 a bool? Where true means success and false mean
failure?
>
> dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
> @@ -685,14 +692,10 @@ static void test_its_trigger(void)
> its_send_invall(col2);
> its_send_invall(col3);
>
> - report_prefix_push("int");
> /*
> * dev=2, eventid=20 -> lpi= 8195, col=3
> * dev=7, eventid=255 -> lpi= 8196, col=2
> - * Trigger dev2, eventid=20 and dev7, eventid=255
> - * Check both LPIs hit
> */
> -
> its_send_mapd(dev2, true);
> its_send_mapd(dev7, true);
>
> @@ -703,6 +706,23 @@ static void test_its_trigger(void)
> 20 /* event id */, col3);
> its_send_mapti(dev7, 8196 /* lpi id */,
> 255 /* event id */, col2);
> + return 0;
> +}
> +
> +static void test_its_trigger(void)
> +{
> + struct its_collection *col3, *col2;
> + struct its_device *dev2, *dev7;
> +
> + if (its_setup1())
> + return;
> +
> + col3 = its_get_collection(3);
> + col2 = its_get_collection(2);
> + dev2 = its_get_device(2);
> + dev7 = its_get_device(7);
> +
> + report_prefix_push("int");
>
> lpi_stats_expect(3, 8195);
> its_send_int(dev2, 20);
> @@ -763,6 +783,29 @@ static void test_its_trigger(void)
> check_lpi_stats();
> report_prefix_pop();
> }
> +
> +static void test_its_migration(void)
> +{
> + struct its_device *dev2, *dev7;
> +
> + if (its_setup1())
> + return;
> +
> + dev2 = its_get_device(2);
> + dev7 = its_get_device(7);
> +
> + puts("Now migrate the VM, then press a key to continue...\n");
> + (void)getchar();
> + report(true, "Migration complete");
This seems more like a report_info place. If migration fails and
we don't complete we'll never get the report FAIL anyway.
> +
> + lpi_stats_expect(3, 8195);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + lpi_stats_expect(2, 8196);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> +}
> #endif
>
> int main(int argc, char **argv)
> @@ -800,6 +843,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> test_its_trigger();
> report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-migration")) {
> + report_prefix_push(argv[1]);
> + test_its_migration();
> + report_prefix_pop();
> } else if (strcmp(argv[1], "its-introspection") == 0) {
> report_prefix_push(argv[1]);
> test_its_introspection();
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index bfafec5..8b8ec79 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
> groups = its
> arch = arm64
>
> +[its-migration]
> +file = gic.flat
> +smp = $MAX_SMP
> +accel = kvm
> +extra_params = -machine gic-version=3 -append 'its-migration'
> +groups = its migration
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index 0e5c5b6..febc2b2 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -151,6 +151,8 @@ extern void its_send_invall(struct its_collection *col);
> extern void its_send_movi(struct its_device *dev,
> struct its_collection *col, u32 id);
> extern void its_send_sync(struct its_collection *col);
> +extern struct its_device *its_get_device(u32 id);
> +extern struct its_collection *its_get_collection(u32 id);
>
> #define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
> #define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index c2dcd01..099940e 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -219,3 +219,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
> its_data.nr_collections++;
> return new;
> }
> +
> +struct its_device *its_get_device(u32 id)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_MAX_DEVICES; i++) {
> + if (its_data.devices[i].device_id == id)
> + return &its_data.devices[i];
> + }
> + return NULL;
> +}
> +
> +struct its_collection *its_get_collection(u32 id)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
> + if (its_data.collections[i].col_id == id)
> + return &its_data.collections[i];
> + }
> + return NULL;
> +}
The callers aren't checking for NULL. Should we assert here
or in the caller?
Thanks,
drew
> --
> 2.20.1
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 13/14] arm/arm64: ITS: migration tests
@ 2020-02-07 13:49 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 13:49 UTC (permalink / raw)
To: Eric Auger
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:58AM +0100, Eric Auger wrote:
> This test maps LPIs (populates the device table, the collection table,
> interrupt translation tables, configuration table), migrates and make
> sure the translation is correct on the destination.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> ---
> arm/gic.c | 59 ++++++++++++++++++++++++++++++++++++----
> arm/unittests.cfg | 8 ++++++
> lib/arm/asm/gic-v3-its.h | 2 ++
> lib/arm/gic-v3-its.c | 22 +++++++++++++++
> 4 files changed, 85 insertions(+), 6 deletions(-)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index 50104b1..fa8626a 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -593,6 +593,7 @@ static void gic_test_mmio(void)
>
> static void test_its_introspection(void) {}
> static void test_its_trigger(void) {}
> +static void test_its_migration(void) {}
>
> #else /* __arch64__ */
>
> @@ -665,13 +666,19 @@ static bool its_prerequisites(int nb_cpus)
> return false;
> }
>
> -static void test_its_trigger(void)
> +/*
> + * Setup the configuration for those mappings:
> + * dev_id=2 event=20 -> vcpu 3, intid=8195
> + * dev_id=7 event=255 -> vcpu 2, intid=8196
> + * LPIs ready to hit
> + */
> +static int its_setup1(void)
> {
> struct its_collection *col3, *col2;
> struct its_device *dev2, *dev7;
>
> if (its_prerequisites(4))
> - return;
> + return -1;
Why not make its_setup1 a bool? Where true means success and false mean
failure?
>
> dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
> @@ -685,14 +692,10 @@ static void test_its_trigger(void)
> its_send_invall(col2);
> its_send_invall(col3);
>
> - report_prefix_push("int");
> /*
> * dev=2, eventid=20 -> lpi= 8195, col=3
> * dev=7, eventid=255 -> lpi= 8196, col=2
> - * Trigger dev2, eventid=20 and dev7, eventid=255
> - * Check both LPIs hit
> */
> -
> its_send_mapd(dev2, true);
> its_send_mapd(dev7, true);
>
> @@ -703,6 +706,23 @@ static void test_its_trigger(void)
> 20 /* event id */, col3);
> its_send_mapti(dev7, 8196 /* lpi id */,
> 255 /* event id */, col2);
> + return 0;
> +}
> +
> +static void test_its_trigger(void)
> +{
> + struct its_collection *col3, *col2;
> + struct its_device *dev2, *dev7;
> +
> + if (its_setup1())
> + return;
> +
> + col3 = its_get_collection(3);
> + col2 = its_get_collection(2);
> + dev2 = its_get_device(2);
> + dev7 = its_get_device(7);
> +
> + report_prefix_push("int");
>
> lpi_stats_expect(3, 8195);
> its_send_int(dev2, 20);
> @@ -763,6 +783,29 @@ static void test_its_trigger(void)
> check_lpi_stats();
> report_prefix_pop();
> }
> +
> +static void test_its_migration(void)
> +{
> + struct its_device *dev2, *dev7;
> +
> + if (its_setup1())
> + return;
> +
> + dev2 = its_get_device(2);
> + dev7 = its_get_device(7);
> +
> + puts("Now migrate the VM, then press a key to continue...\n");
> + (void)getchar();
> + report(true, "Migration complete");
This seems more like a report_info place. If migration fails and
we don't complete we'll never get the report FAIL anyway.
> +
> + lpi_stats_expect(3, 8195);
> + its_send_int(dev2, 20);
> + check_lpi_stats();
> +
> + lpi_stats_expect(2, 8196);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> +}
> #endif
>
> int main(int argc, char **argv)
> @@ -800,6 +843,10 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> test_its_trigger();
> report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-migration")) {
> + report_prefix_push(argv[1]);
> + test_its_migration();
> + report_prefix_pop();
> } else if (strcmp(argv[1], "its-introspection") == 0) {
> report_prefix_push(argv[1]);
> test_its_introspection();
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index bfafec5..8b8ec79 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
> groups = its
> arch = arm64
>
> +[its-migration]
> +file = gic.flat
> +smp = $MAX_SMP
> +accel = kvm
> +extra_params = -machine gic-version=3 -append 'its-migration'
> +groups = its migration
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
> index 0e5c5b6..febc2b2 100644
> --- a/lib/arm/asm/gic-v3-its.h
> +++ b/lib/arm/asm/gic-v3-its.h
> @@ -151,6 +151,8 @@ extern void its_send_invall(struct its_collection *col);
> extern void its_send_movi(struct its_device *dev,
> struct its_collection *col, u32 id);
> extern void its_send_sync(struct its_collection *col);
> +extern struct its_device *its_get_device(u32 id);
> +extern struct its_collection *its_get_collection(u32 id);
>
> #define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
> #define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
> index c2dcd01..099940e 100644
> --- a/lib/arm/gic-v3-its.c
> +++ b/lib/arm/gic-v3-its.c
> @@ -219,3 +219,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
> its_data.nr_collections++;
> return new;
> }
> +
> +struct its_device *its_get_device(u32 id)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_MAX_DEVICES; i++) {
> + if (its_data.devices[i].device_id == id)
> + return &its_data.devices[i];
> + }
> + return NULL;
> +}
> +
> +struct its_collection *its_get_collection(u32 id)
> +{
> + int i;
> +
> + for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
> + if (its_data.collections[i].col_id == id)
> + return &its_data.collections[i];
> + }
> + return NULL;
> +}
The callers aren't checking for NULL. Should we assert here
or in the caller?
Thanks,
drew
> --
> 2.20.1
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
2020-01-28 10:34 ` Eric Auger
(?)
@ 2020-02-07 14:06 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 14:06 UTC (permalink / raw)
To: Eric Auger
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
peter.maydell, andre.przywara, thuth, yuzenghui,
alexandru.elisei
On Tue, Jan 28, 2020 at 11:34:59AM +0100, Eric Auger wrote:
> Add two new migration tests. One testing the migration of
> a topology where collection were unmapped. The second test
> checks the migration of the pending table.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - tests belong to both its and migration groups
> ---
> arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
> arm/unittests.cfg | 16 +++++
> 2 files changed, 166 insertions(+)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index fa8626a..ec3dd3a 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
> smp_rmb(); /* pairs with wmb in lpi_stats_expect */
> lpi_stats.observed.cpu_id = smp_processor_id();
> lpi_stats.observed.lpi_id = irqnr;
> + acked[lpi_stats.observed.cpu_id]++;
> smp_wmb(); /* pairs with rmb in check_lpi_stats */
> }
>
> @@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
> while (1)
> wfi();
> }
> +
> +static void check_lpi_hits(int *expected)
> +{
> + int i;
> +
> + for (i = 0; i < nr_cpus; i++) {
> + if (acked[i] != expected[i])
> + report(false, "expected %d LPIs on PE #%d, %d observed",
> + expected[i], i, acked[i]);
report_info
pass = false
> + }
> + report(true, "check LPI on all vcpus");
report(pass, ...);
> +}
> #endif
>
> static void gicv2_ipi_send_self(void)
> @@ -594,6 +607,8 @@ static void gic_test_mmio(void)
> static void test_its_introspection(void) {}
> static void test_its_trigger(void) {}
> static void test_its_migration(void) {}
> +static void test_migrate_unmapped_collection(void) {}
> +static void test_its_pending_migration(void) {}
I'm not sure what's worse. This pile of stubs or one #ifdef in main()
wrapping all the calls.
>
> #else /* __arch64__ */
>
> @@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
> return false;
> }
>
> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
> + struct its_collection *col)
> +{
> + if (!dev || !col)
I don't think col can be null, and this doesn't look like the right place
to check if dev is null. If we're bothiner to call set_lpi, then I
think we should already expect dev to be good to go.
> + report_abort("wrong device or collection");
> +
> + its_send_mapti(dev, physid, eventid, col);
> +
> + gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
> + its_send_invall(col);
> +}
> +
> /*
> * Setup the configuration for those mappings:
> * dev_id=2 event=20 -> vcpu 3, intid=8195
> @@ -806,6 +833,121 @@ static void test_its_migration(void)
> its_send_int(dev7, 255);
> check_lpi_stats();
> }
> +
> +static void test_migrate_unmapped_collection(void)
> +{
> + struct its_collection *col;
> + struct its_device *dev2, *dev7;
> + u8 config;
> +
> + if (its_setup1())
> + return;
> +
> + col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
> + dev2 = its_get_device(2);
> + dev7 = its_get_device(7);
> +
> + /* MAPTI with the collection unmapped */
> + set_lpi(dev2, 0, 8192, col);
> +
> + puts("Now migrate the VM, then press a key to continue...\n");
> + (void)getchar();
> + report(true, "Migration complete");
report_info
> +
> + /* on the destination, map the collection */
> + its_send_mapc(col, true);
> +
> + lpi_stats_expect(2, 8196);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> +
> + config = gicv3_lpi_get_config(8192);
> + report(config == LPI_PROP_DEFAULT,
> + "Config of LPI 8192 was properly migrated");
> +
> + lpi_stats_expect(nr_cpus - 1, 8192);
> + its_send_int(dev2, 0);
> + check_lpi_stats();
> +
> + /* unmap the collection */
> + its_send_mapc(col, false);
> +
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 0);
> + check_lpi_stats();
> +
> + /* remap event 0 onto lpiid 8193 */
> + set_lpi(dev2, 0, 8193, col);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 0);
> + check_lpi_stats();
> +
> + /* remap the collection */
> + its_send_mapc(col, true);
> + lpi_stats_expect(nr_cpus - 1, 8193);
> +}
> +
> +static void test_its_pending_migration(void)
> +{
> + struct its_device *dev;
> + struct its_collection *collection[2];
> + int expected[NR_CPUS];
expected = malloc(nr_cpus * sizeof(int));
I know there are other places using NR_CPUS right now that don't have to,
but we shouldn't add more. Eventually I'll change the other places too.
> + u64 pendbaser;
> + void *ptr;
> + int i;
> +
> + if (its_prerequisites(4))
> + return;
> +
> + dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> + its_send_mapd(dev, true);
> +
> + collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
> + collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
> + its_send_mapc(collection[0], true);
> + its_send_mapc(collection[1], true);
> +
> + /* disable lpi at redist level */
> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
> +
> + /* even lpis are assigned to even cpu */
> + for (i = 0; i < 256; i++) {
> + struct its_collection *col = i % 2 ? collection[0] :
> + collection[1];
> + int vcpu = col->target_address >> 16;
I'm lost with the even/odd (nr_cpus - 1)/(nr_cpus - 2) stuff, and won't
it swap if nr_cpus is odd vs. even?
Shouldn't we just have something like
pe1 = nr_cpus - 1;
pe2 = nr_cpus - 2;
col1 = its_create_collection(pe1, pe1);
col2 = its_create_collection(pe2, pe2);
without mentioning even and odd?
> +
> + its_send_mapti(dev, 8192 + i, i, col);
> + gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
> + gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
> + }
> + its_send_invall(collection[0]);
> + its_send_invall(collection[1]);
> +
> + /* Set the PTZ bit on each pendbaser */
> +
> + expected[nr_cpus - 1] = 128;
> + expected[nr_cpus - 2] = 128;
> +
> + ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
> + pendbaser = readq(ptr);
> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> +
> + ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
> + pendbaser = readq(ptr);
> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> +
> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
> +
> + puts("Now migrate the VM, then press a key to continue...\n");
> + (void)getchar();
> + report(true, "Migration complete");
report_info
> +
> + mdelay(1000);
This delay needs a comment explaining why it's here.
> +
> + check_lpi_hits(expected);
> +}
> #endif
>
> int main(int argc, char **argv)
> @@ -847,6 +989,14 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> test_its_migration();
> report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-pending-migration")) {
> + report_prefix_push(argv[1]);
> + test_its_pending_migration();
> + report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
> + report_prefix_push(argv[1]);
> + test_migrate_unmapped_collection();
> + report_prefix_pop();
> } else if (strcmp(argv[1], "its-introspection") == 0) {
> report_prefix_push(argv[1]);
> test_its_introspection();
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index 8b8ec79..d917157 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
> groups = its migration
> arch = arm64
>
> +[its-pending-migration]
> +file = gic.flat
> +smp = $MAX_SMP
> +accel = kvm
> +extra_params = -machine gic-version=3 -append 'its-pending-migration'
> +groups = its migration
> +arch = arm64
> +
> +[its-migrate-unmapped-collection]
> +file = gic.flat
> +smp = $MAX_SMP
> +accel = kvm
> +extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
> +groups = its migration
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> --
> 2.20.1
>
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
@ 2020-02-07 14:06 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 14:06 UTC (permalink / raw)
To: Eric Auger
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:59AM +0100, Eric Auger wrote:
> Add two new migration tests. One testing the migration of
> a topology where collection were unmapped. The second test
> checks the migration of the pending table.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - tests belong to both its and migration groups
> ---
> arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
> arm/unittests.cfg | 16 +++++
> 2 files changed, 166 insertions(+)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index fa8626a..ec3dd3a 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
> smp_rmb(); /* pairs with wmb in lpi_stats_expect */
> lpi_stats.observed.cpu_id = smp_processor_id();
> lpi_stats.observed.lpi_id = irqnr;
> + acked[lpi_stats.observed.cpu_id]++;
> smp_wmb(); /* pairs with rmb in check_lpi_stats */
> }
>
> @@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
> while (1)
> wfi();
> }
> +
> +static void check_lpi_hits(int *expected)
> +{
> + int i;
> +
> + for (i = 0; i < nr_cpus; i++) {
> + if (acked[i] != expected[i])
> + report(false, "expected %d LPIs on PE #%d, %d observed",
> + expected[i], i, acked[i]);
report_info
pass = false
> + }
> + report(true, "check LPI on all vcpus");
report(pass, ...);
> +}
> #endif
>
> static void gicv2_ipi_send_self(void)
> @@ -594,6 +607,8 @@ static void gic_test_mmio(void)
> static void test_its_introspection(void) {}
> static void test_its_trigger(void) {}
> static void test_its_migration(void) {}
> +static void test_migrate_unmapped_collection(void) {}
> +static void test_its_pending_migration(void) {}
I'm not sure what's worse. This pile of stubs or one #ifdef in main()
wrapping all the calls.
>
> #else /* __arch64__ */
>
> @@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
> return false;
> }
>
> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
> + struct its_collection *col)
> +{
> + if (!dev || !col)
I don't think col can be null, and this doesn't look like the right place
to check if dev is null. If we're bothiner to call set_lpi, then I
think we should already expect dev to be good to go.
> + report_abort("wrong device or collection");
> +
> + its_send_mapti(dev, physid, eventid, col);
> +
> + gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
> + its_send_invall(col);
> +}
> +
> /*
> * Setup the configuration for those mappings:
> * dev_id=2 event=20 -> vcpu 3, intid=8195
> @@ -806,6 +833,121 @@ static void test_its_migration(void)
> its_send_int(dev7, 255);
> check_lpi_stats();
> }
> +
> +static void test_migrate_unmapped_collection(void)
> +{
> + struct its_collection *col;
> + struct its_device *dev2, *dev7;
> + u8 config;
> +
> + if (its_setup1())
> + return;
> +
> + col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
> + dev2 = its_get_device(2);
> + dev7 = its_get_device(7);
> +
> + /* MAPTI with the collection unmapped */
> + set_lpi(dev2, 0, 8192, col);
> +
> + puts("Now migrate the VM, then press a key to continue...\n");
> + (void)getchar();
> + report(true, "Migration complete");
report_info
> +
> + /* on the destination, map the collection */
> + its_send_mapc(col, true);
> +
> + lpi_stats_expect(2, 8196);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> +
> + config = gicv3_lpi_get_config(8192);
> + report(config == LPI_PROP_DEFAULT,
> + "Config of LPI 8192 was properly migrated");
> +
> + lpi_stats_expect(nr_cpus - 1, 8192);
> + its_send_int(dev2, 0);
> + check_lpi_stats();
> +
> + /* unmap the collection */
> + its_send_mapc(col, false);
> +
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 0);
> + check_lpi_stats();
> +
> + /* remap event 0 onto lpiid 8193 */
> + set_lpi(dev2, 0, 8193, col);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 0);
> + check_lpi_stats();
> +
> + /* remap the collection */
> + its_send_mapc(col, true);
> + lpi_stats_expect(nr_cpus - 1, 8193);
> +}
> +
> +static void test_its_pending_migration(void)
> +{
> + struct its_device *dev;
> + struct its_collection *collection[2];
> + int expected[NR_CPUS];
expected = malloc(nr_cpus * sizeof(int));
I know there are other places using NR_CPUS right now that don't have to,
but we shouldn't add more. Eventually I'll change the other places too.
> + u64 pendbaser;
> + void *ptr;
> + int i;
> +
> + if (its_prerequisites(4))
> + return;
> +
> + dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> + its_send_mapd(dev, true);
> +
> + collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
> + collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
> + its_send_mapc(collection[0], true);
> + its_send_mapc(collection[1], true);
> +
> + /* disable lpi at redist level */
> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
> +
> + /* even lpis are assigned to even cpu */
> + for (i = 0; i < 256; i++) {
> + struct its_collection *col = i % 2 ? collection[0] :
> + collection[1];
> + int vcpu = col->target_address >> 16;
I'm lost with the even/odd (nr_cpus - 1)/(nr_cpus - 2) stuff, and won't
it swap if nr_cpus is odd vs. even?
Shouldn't we just have something like
pe1 = nr_cpus - 1;
pe2 = nr_cpus - 2;
col1 = its_create_collection(pe1, pe1);
col2 = its_create_collection(pe2, pe2);
without mentioning even and odd?
> +
> + its_send_mapti(dev, 8192 + i, i, col);
> + gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
> + gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
> + }
> + its_send_invall(collection[0]);
> + its_send_invall(collection[1]);
> +
> + /* Set the PTZ bit on each pendbaser */
> +
> + expected[nr_cpus - 1] = 128;
> + expected[nr_cpus - 2] = 128;
> +
> + ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
> + pendbaser = readq(ptr);
> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> +
> + ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
> + pendbaser = readq(ptr);
> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> +
> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
> +
> + puts("Now migrate the VM, then press a key to continue...\n");
> + (void)getchar();
> + report(true, "Migration complete");
report_info
> +
> + mdelay(1000);
This delay needs a comment explaining why it's here.
> +
> + check_lpi_hits(expected);
> +}
> #endif
>
> int main(int argc, char **argv)
> @@ -847,6 +989,14 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> test_its_migration();
> report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-pending-migration")) {
> + report_prefix_push(argv[1]);
> + test_its_pending_migration();
> + report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
> + report_prefix_push(argv[1]);
> + test_migrate_unmapped_collection();
> + report_prefix_pop();
> } else if (strcmp(argv[1], "its-introspection") == 0) {
> report_prefix_push(argv[1]);
> test_its_introspection();
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index 8b8ec79..d917157 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
> groups = its migration
> arch = arm64
>
> +[its-pending-migration]
> +file = gic.flat
> +smp = $MAX_SMP
> +accel = kvm
> +extra_params = -machine gic-version=3 -append 'its-pending-migration'
> +groups = its migration
> +arch = arm64
> +
> +[its-migrate-unmapped-collection]
> +file = gic.flat
> +smp = $MAX_SMP
> +accel = kvm
> +extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
> +groups = its migration
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> --
> 2.20.1
>
>
Thanks,
drew
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
@ 2020-02-07 14:06 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-02-07 14:06 UTC (permalink / raw)
To: Eric Auger
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Tue, Jan 28, 2020 at 11:34:59AM +0100, Eric Auger wrote:
> Add two new migration tests. One testing the migration of
> a topology where collection were unmapped. The second test
> checks the migration of the pending table.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - tests belong to both its and migration groups
> ---
> arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
> arm/unittests.cfg | 16 +++++
> 2 files changed, 166 insertions(+)
>
> diff --git a/arm/gic.c b/arm/gic.c
> index fa8626a..ec3dd3a 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
> smp_rmb(); /* pairs with wmb in lpi_stats_expect */
> lpi_stats.observed.cpu_id = smp_processor_id();
> lpi_stats.observed.lpi_id = irqnr;
> + acked[lpi_stats.observed.cpu_id]++;
> smp_wmb(); /* pairs with rmb in check_lpi_stats */
> }
>
> @@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
> while (1)
> wfi();
> }
> +
> +static void check_lpi_hits(int *expected)
> +{
> + int i;
> +
> + for (i = 0; i < nr_cpus; i++) {
> + if (acked[i] != expected[i])
> + report(false, "expected %d LPIs on PE #%d, %d observed",
> + expected[i], i, acked[i]);
report_info
pass = false
> + }
> + report(true, "check LPI on all vcpus");
report(pass, ...);
> +}
> #endif
>
> static void gicv2_ipi_send_self(void)
> @@ -594,6 +607,8 @@ static void gic_test_mmio(void)
> static void test_its_introspection(void) {}
> static void test_its_trigger(void) {}
> static void test_its_migration(void) {}
> +static void test_migrate_unmapped_collection(void) {}
> +static void test_its_pending_migration(void) {}
I'm not sure what's worse. This pile of stubs or one #ifdef in main()
wrapping all the calls.
>
> #else /* __arch64__ */
>
> @@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
> return false;
> }
>
> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
> + struct its_collection *col)
> +{
> + if (!dev || !col)
I don't think col can be null, and this doesn't look like the right place
to check if dev is null. If we're bothiner to call set_lpi, then I
think we should already expect dev to be good to go.
> + report_abort("wrong device or collection");
> +
> + its_send_mapti(dev, physid, eventid, col);
> +
> + gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
> + its_send_invall(col);
> +}
> +
> /*
> * Setup the configuration for those mappings:
> * dev_id=2 event=20 -> vcpu 3, intid=8195
> @@ -806,6 +833,121 @@ static void test_its_migration(void)
> its_send_int(dev7, 255);
> check_lpi_stats();
> }
> +
> +static void test_migrate_unmapped_collection(void)
> +{
> + struct its_collection *col;
> + struct its_device *dev2, *dev7;
> + u8 config;
> +
> + if (its_setup1())
> + return;
> +
> + col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
> + dev2 = its_get_device(2);
> + dev7 = its_get_device(7);
> +
> + /* MAPTI with the collection unmapped */
> + set_lpi(dev2, 0, 8192, col);
> +
> + puts("Now migrate the VM, then press a key to continue...\n");
> + (void)getchar();
> + report(true, "Migration complete");
report_info
> +
> + /* on the destination, map the collection */
> + its_send_mapc(col, true);
> +
> + lpi_stats_expect(2, 8196);
> + its_send_int(dev7, 255);
> + check_lpi_stats();
> +
> + config = gicv3_lpi_get_config(8192);
> + report(config == LPI_PROP_DEFAULT,
> + "Config of LPI 8192 was properly migrated");
> +
> + lpi_stats_expect(nr_cpus - 1, 8192);
> + its_send_int(dev2, 0);
> + check_lpi_stats();
> +
> + /* unmap the collection */
> + its_send_mapc(col, false);
> +
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 0);
> + check_lpi_stats();
> +
> + /* remap event 0 onto lpiid 8193 */
> + set_lpi(dev2, 0, 8193, col);
> + lpi_stats_expect(-1, -1);
> + its_send_int(dev2, 0);
> + check_lpi_stats();
> +
> + /* remap the collection */
> + its_send_mapc(col, true);
> + lpi_stats_expect(nr_cpus - 1, 8193);
> +}
> +
> +static void test_its_pending_migration(void)
> +{
> + struct its_device *dev;
> + struct its_collection *collection[2];
> + int expected[NR_CPUS];
expected = malloc(nr_cpus * sizeof(int));
I know there are other places using NR_CPUS right now that don't have to,
but we shouldn't add more. Eventually I'll change the other places too.
> + u64 pendbaser;
> + void *ptr;
> + int i;
> +
> + if (its_prerequisites(4))
> + return;
> +
> + dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> + its_send_mapd(dev, true);
> +
> + collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
> + collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
> + its_send_mapc(collection[0], true);
> + its_send_mapc(collection[1], true);
> +
> + /* disable lpi at redist level */
> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
> +
> + /* even lpis are assigned to even cpu */
> + for (i = 0; i < 256; i++) {
> + struct its_collection *col = i % 2 ? collection[0] :
> + collection[1];
> + int vcpu = col->target_address >> 16;
I'm lost with the even/odd (nr_cpus - 1)/(nr_cpus - 2) stuff, and won't
it swap if nr_cpus is odd vs. even?
Shouldn't we just have something like
pe1 = nr_cpus - 1;
pe2 = nr_cpus - 2;
col1 = its_create_collection(pe1, pe1);
col2 = its_create_collection(pe2, pe2);
without mentioning even and odd?
> +
> + its_send_mapti(dev, 8192 + i, i, col);
> + gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
> + gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
> + }
> + its_send_invall(collection[0]);
> + its_send_invall(collection[1]);
> +
> + /* Set the PTZ bit on each pendbaser */
> +
> + expected[nr_cpus - 1] = 128;
> + expected[nr_cpus - 2] = 128;
> +
> + ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
> + pendbaser = readq(ptr);
> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> +
> + ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
> + pendbaser = readq(ptr);
> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> +
> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
> +
> + puts("Now migrate the VM, then press a key to continue...\n");
> + (void)getchar();
> + report(true, "Migration complete");
report_info
> +
> + mdelay(1000);
This delay needs a comment explaining why it's here.
> +
> + check_lpi_hits(expected);
> +}
> #endif
>
> int main(int argc, char **argv)
> @@ -847,6 +989,14 @@ int main(int argc, char **argv)
> report_prefix_push(argv[1]);
> test_its_migration();
> report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-pending-migration")) {
> + report_prefix_push(argv[1]);
> + test_its_pending_migration();
> + report_prefix_pop();
> + } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
> + report_prefix_push(argv[1]);
> + test_migrate_unmapped_collection();
> + report_prefix_pop();
> } else if (strcmp(argv[1], "its-introspection") == 0) {
> report_prefix_push(argv[1]);
> test_its_introspection();
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index 8b8ec79..d917157 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
> groups = its migration
> arch = arm64
>
> +[its-pending-migration]
> +file = gic.flat
> +smp = $MAX_SMP
> +accel = kvm
> +extra_params = -machine gic-version=3 -append 'its-pending-migration'
> +groups = its migration
> +arch = arm64
> +
> +[its-migrate-unmapped-collection]
> +file = gic.flat
> +smp = $MAX_SMP
> +accel = kvm
> +extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
> +groups = its migration
> +arch = arm64
> +
> # Test PSCI emulation
> [psci]
> file = psci.flat
> --
> 2.20.1
>
>
Thanks,
drew
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
2020-02-07 10:19 ` Andrew Jones
(?)
@ 2020-03-04 14:20 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-04 14:20 UTC (permalink / raw)
To: Andrew Jones
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
andre.przywara, peter.maydell, yuzenghui, alexandru.elisei,
thuth
Hi Drew,
On 2/7/20 11:19 AM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:50AM +0100, Eric Auger wrote:
>> Detect the presence of an ITS as part of the GICv3 init
>> routine, initialize its base address and read few registers
>> the IIDR, the TYPER to store its dimensioning parameters.
>> Also parse the BASER registers.
>>
>> This is our first ITS test, belonging to a new "its" group.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - updated dates and changed author
>> - squash "arm/arm64: ITS: Test BASER" into this patch but
>> removes setup_baser which will be introduced later.
>> - only compile on aarch64
>> - restrict the new test to aarch64
>>
>> v1 -> v2:
>> - clean GITS_TYPER macros and unused fields in typer struct
>> - remove memory attribute related macros
>> - remove everything related to memory attributes
>> - s/dev_baser/coll_baser/ in report_info
>> - add extra line
>> - removed index filed in its_baser
>> ---
>> arm/Makefile.arm64 | 1 +
>> arm/gic.c | 49 ++++++++++++++++++
>> arm/unittests.cfg | 7 +++
>> lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
>> lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
>> lib/arm/gic.c | 30 +++++++++--
>> lib/arm64/asm/gic-v3-its.h | 1 +
>> 7 files changed, 274 insertions(+), 5 deletions(-)
>> create mode 100644 lib/arm/asm/gic-v3-its.h
>> create mode 100644 lib/arm/gic-v3-its.c
>> create mode 100644 lib/arm64/asm/gic-v3-its.h
>>
>> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
>> index 6d3dc2c..2571ffb 100644
>> --- a/arm/Makefile.arm64
>> +++ b/arm/Makefile.arm64
>> @@ -19,6 +19,7 @@ endef
>> cstart.o = $(TEST_DIR)/cstart64.o
>> cflatobjs += lib/arm64/processor.o
>> cflatobjs += lib/arm64/spinlock.o
>> +cflatobjs += lib/arm/gic-v3-its.o
>
> If gic-v3-its.c will never be compiled for arm, then it should
> probably go lib/arm64, not lib/arm. Same comment for all other
> new source and header files. The only problem with that is...>
>>
>> OBJDIRS += lib/arm64
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index abf08c7..4d7dd03 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -16,6 +16,7 @@
>> #include <asm/processor.h>
>> #include <asm/delay.h>
>> #include <asm/gic.h>
>> +#include <asm/gic-v3-its.h>
>
> ...here where we'd have to do
>
> #if defined(__aarch64__)
> #include <asm/gic-v3-its.h>
> #endif
>
> which is ugly. However that can be avoided by adding an
> empty lib/arm/asm/gic-v3-its.h file.
OK made those changes with this latter containing stubs.
>
>
>> #include <asm/smp.h>
>> #include <asm/barrier.h>
>> #include <asm/io.h>
>> @@ -518,6 +519,50 @@ static void gic_test_mmio(void)
>> test_targets(nr_irqs);
>> }
>>
>> +#if defined(__arm__)
>> +
>> +static void test_its_introspection(void) {}
>
> static void test_its_introspection(void)
> {
> report_abort(...);
> }
OK
>
>> +
>> +#else /* __arch64__ */
>> +
>> +static void test_its_introspection(void)
>> +{
>> + struct its_baser *dev_baser, *coll_baser;
>> + struct its_typer *typer = &its_data.typer;
>> +
>> + if (!gicv3_its_base()) {
>> + report_skip("No ITS, skip ...");
>> + return;
>> + }
>> +
>> + /* IIDR */
>> + report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
>> + "GITS_IIDR is read-only"),
>> +
>> + /* TYPER */
>> + report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
>> + "GITS_TYPER is read-only");
>> +
>> + report(typer->phys_lpi, "ITS supports physical LPIs");
>> + report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
>> + report_info("ITT entry size = 0x%x", typer->ite_size);
>> + report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
>> + typer->eventid_bits, typer->deviceid_bits,
>> + typer->collid_bits);
>> + report(typer->eventid_bits && typer->deviceid_bits &&
>> + typer->collid_bits, "ID spaces");
>> + report_info("Target address format %s",
>> + typer->pta ? "Redist basse address" : "PE #");
>> +
>> + dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
>> + coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
>> + report(dev_baser && coll_baser, "detect device and collection BASER");
>> + report_info("device baser entry_size = 0x%x", dev_baser->esz);
>> + report_info("collection baser entry_size = 0x%x", coll_baser->esz);
>> +}
>> +
>> +#endif
>> +
>> int main(int argc, char **argv)
>> {
>> if (!gic_init()) {
>> @@ -549,6 +594,10 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> gic_test_mmio();
>> report_prefix_pop();
>> + } else if (strcmp(argv[1], "its-introspection") == 0) {
>> + report_prefix_push(argv[1]);
>> + test_its_introspection();
>> + report_prefix_pop();
>> } else {
>> report_abort("Unknown subtest '%s'", argv[1]);
>> }
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index daeb5a0..ba2b31b 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -122,6 +122,13 @@ smp = $MAX_SMP
>> extra_params = -machine gic-version=3 -append 'active'
>> groups = gic
>>
>> +[its-introspection]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +extra_params = -machine gic-version=3 -append 'its-introspection'
>> +groups = its
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> new file mode 100644
>> index 0000000..815c515
>> --- /dev/null
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -0,0 +1,103 @@
>> +/*
>> + * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
>> + *
>> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#ifndef _ASMARM_GIC_V3_ITS_H_
>> +#define _ASMARM_GIC_V3_ITS_H_
>> +
>> +#ifndef __ASSEMBLY__
>
> Doesn't look like you use the #else /* __ASSEMBLY__ */ side of this.
> I'd leave out the #ifndef until we add defines we need to access
> from assembler.
OK
>
>> +
>> +struct its_typer {
>> + unsigned int ite_size;
>> + unsigned int eventid_bits;
>> + unsigned int deviceid_bits;
>> + unsigned int collid_bits;
>> + bool pta;
>> + bool phys_lpi;
>> + bool virt_lpi;
>> +};
>> +
>> +struct its_baser {
>> + int type;
>> + size_t psz;
>> + int nr_pages;
>> + bool indirect;
>> + phys_addr_t table_addr;
>> + bool valid;
>> + int esz;
>> +};
>> +
>> +#define GITS_BASER_NR_REGS 8
>> +
>> +struct its_data {
>> + void *base;
>> + struct its_typer typer;
>> + struct its_baser baser[GITS_BASER_NR_REGS];
>> +};
>> +
>> +extern struct its_data its_data;
>> +
>> +#define gicv3_its_base() (its_data.base)
>> +
>> +#if defined(__aarch64__)
>> +
>> +#define GITS_CTLR 0x0000
>> +#define GITS_IIDR 0x0004
>> +#define GITS_TYPER 0x0008
>> +#define GITS_CBASER 0x0080
>> +#define GITS_CWRITER 0x0088
>> +#define GITS_CREADR 0x0090
>> +#define GITS_BASER 0x0100
>> +
>> +#define GITS_TYPER_PLPIS BIT(0)
>> +#define GITS_TYPER_VLPIS BIT(1)
>> +#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
>> +#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
>> +#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
>> +#define GITS_TYPER_IDBITS_SHIFT 8
>> +#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
>> +#define GITS_TYPER_DEVBITS_SHIFT 13
>> +#define GITS_TYPER_PTA BIT(19)
>> +#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
>> +#define GITS_TYPER_CIDBITS_SHIFT 32
>> +#define GITS_TYPER_CIL BIT(36)
>> +
>> +#define GITS_CTLR_ENABLE (1U << 0)
>> +
>> +#define GITS_CBASER_VALID (1UL << 63)
>> +
>> +#define GITS_BASER_VALID BIT(63)
>> +#define GITS_BASER_INDIRECT BIT(62)
>> +#define GITS_BASER_TYPE_SHIFT (56)
>> +#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
>> +#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
>> +#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
>> +#define GITS_BASER_PAGE_SIZE_SHIFT (8)
>> +#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGES_MAX 256
>> +#define GITS_BASER_PAGES_SHIFT (0)
>> +#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
>> +#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
>> +#define GITS_BASER_TYPE_NONE 0
>> +#define GITS_BASER_TYPE_DEVICE 1
>> +#define GITS_BASER_TYPE_COLLECTION 4
>> +
>> +extern void its_parse_typer(void);
>> +extern void its_init(void);
>> +extern int its_parse_baser(int i, struct its_baser *baser);
>> +extern struct its_baser *its_lookup_baser(int type);
>> +
>> +#else /* __arm__ */
>> +
>> +static inline void its_init(void) {}
>
> Looks like the empty gic-v3-its.h I suggested creating above will actually
> be useful. We can add stubs like this in it.
OK
>
>> +
>> +#endif
>> +
>> +#endif /* !__ASSEMBLY__ */
>> +#endif /* _ASMARM_GIC_V3_ITS_H_ */
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> new file mode 100644
>> index 0000000..2c0ce13
>> --- /dev/null
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -0,0 +1,88 @@
>> +/*
>> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#include <asm/gic.h>
>> +#include <alloc_page.h>
>> +#include <asm/gic-v3-its.h>
>> +
>> +void its_parse_typer(void)
>> +{
>> + u64 typer = readq(gicv3_its_base() + GITS_TYPER);
>> +
>> + its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
>> + GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
>> + its_data.typer.pta = typer & GITS_TYPER_PTA;
>> + its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
>> + GITS_TYPER_IDBITS_SHIFT) + 1;
>> + its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
>> + GITS_TYPER_DEVBITS_SHIFT) + 1;
>> +
>> + if (typer & GITS_TYPER_CIL)
>> + its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
>> + GITS_TYPER_CIDBITS_SHIFT) + 1;
>
> nit: please consider aligning like this
>
> ((typer & MASK) >>
> SHIFT) + 1;
>
> Or, maybe better to macro it
>
> #define TYPER_FIELD(typer, mask, shift) (((type) & (mask) >> (shift)) + 1)
>
> And, rather than have a bunch of 'its_data.typer's we could use an alias,
> helping us stay within 120 chars.
>
> struct its_typer *t = &its_data.typer;
yes used that trick and relied on 120 char max.
>
> t->ite_size = TYPER_FIELD(typer, GITS_TYPER_ITT_ENTRY_SIZE,
> GITS_TYPER_ITT_ENTRY_SIZE_SHIFT);
>
>
>> + else
>> + its_data.typer.collid_bits = 16;
>> +
>> + its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
>> + its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
>> +}
>> +
>> +int its_parse_baser(int i, struct its_baser *baser)
>> +{
>> + void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
>> + u64 val = readq(reg_addr);
>> +
>> + if (!val) {
>> + memset(baser, 0, sizeof(*baser));
>> + return -1;
>> + }
>> +
>> + baser->valid = val & GITS_BASER_VALID;
>> + baser->indirect = val & GITS_BASER_INDIRECT;
>> + baser->type = GITS_BASER_TYPE(val);
>> + baser->esz = GITS_BASER_ENTRY_SIZE(val);
>> + baser->nr_pages = GITS_BASER_NR_PAGES(val);
>> + baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
>> + switch (val & GITS_BASER_PAGE_SIZE_MASK) {
>> + case GITS_BASER_PAGE_SIZE_4K:
>> + baser->psz = SZ_4K;
>> + break;
>> + case GITS_BASER_PAGE_SIZE_16K:
>> + baser->psz = SZ_16K;
>> + break;
>> + case GITS_BASER_PAGE_SIZE_64K:
>> + baser->psz = SZ_64K;
>> + break;
>> + default:
>> + baser->psz = SZ_64K;
>> + }
>> + return 0;
>> +}
>> +
>> +struct its_baser *its_lookup_baser(int type)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
>> + struct its_baser *baser = &its_data.baser[i];
>> +
>> + if (baser->type == type)
>> + return baser;
>> + }
>> + return NULL;
>> +}
>> +
>> +void its_init(void)
>> +{
>> + int i;
>> +
>> + if (!its_data.base)
>> + return;
>> +
>> + its_parse_typer();
>> + for (i = 0; i < GITS_BASER_NR_REGS; i++)
>> + its_parse_baser(i, &its_data.baser[i]);
>> +}
>> +
>> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
>> index aa9cb86..6b70b05 100644
>> --- a/lib/arm/gic.c
>> +++ b/lib/arm/gic.c
>> @@ -6,9 +6,11 @@
>> #include <devicetree.h>
>> #include <asm/gic.h>
>> #include <asm/io.h>
>> +#include <asm/gic-v3-its.h>
>>
>> struct gicv2_data gicv2_data;
>> struct gicv3_data gicv3_data;
>> +struct its_data its_data;
>>
>> struct gic_common_ops {
>> void (*enable_defaults)(void);
>> @@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
>> * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>> */
>> static bool
>> -gic_get_dt_bases(const char *compatible, void **base1, void **base2)
>> +gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
>> {
>> struct dt_pbus_reg reg;
>> - struct dt_device gic;
>> + struct dt_device gic, its;
>> struct dt_bus bus;
>> - int node, ret, i;
>> + int node, subnode, ret, i, len;
>> + const void *fdt = dt_fdt();
>>
>> dt_bus_init_defaults(&bus);
>> dt_device_init(&gic, &bus, NULL);
>> @@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
>> base2[i] = ioremap(reg.addr, reg.size);
>> }
>>
>> + if (base3 && !strcmp(compatible, "arm,gic-v3")) {
>
> If base != NULL, then we could assert(strcmp(compatible, "arm,cortex-a15-gic") != 0)
OK
Thanks
Eric
>
>> + dt_for_each_subnode(node, subnode) {
>> + const struct fdt_property *prop;
>> +
>> + prop = fdt_get_property(fdt, subnode, "compatible", &len);
>> + if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
>> + dt_device_bind_node(&its, subnode);
>> + ret = dt_pbus_translate(&its, 0, ®);
>> + assert(ret == 0);
>> + *base3 = ioremap(reg.addr, reg.size);
>> + break;
>> + }
>> + }
>> +
>> + }
>> +
>> return true;
>> }
>>
>> int gicv2_init(void)
>> {
>> return gic_get_dt_bases("arm,cortex-a15-gic",
>> - &gicv2_data.dist_base, &gicv2_data.cpu_base);
>> + &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
>> }
>>
>> int gicv3_init(void)
>> {
>> return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
>> - &gicv3_data.redist_bases[0]);
>> + &gicv3_data.redist_bases[0], &its_data.base);
>> }
>>
>> int gic_version(void)
>> @@ -104,6 +123,7 @@ int gic_init(void)
>> gic_common_ops = &gicv2_common_ops;
>> else if (gicv3_init())
>> gic_common_ops = &gicv3_common_ops;
>> + its_init();
>> return gic_version();
>> }
>>
>> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
>> new file mode 100644
>> index 0000000..083cba4
>> --- /dev/null
>> +++ b/lib/arm64/asm/gic-v3-its.h
>> @@ -0,0 +1 @@
>> +#include "../../arm/asm/gic-v3-its.h"
>> --
>> 2.20.1
>>
>
> Thanks,
> drew
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
@ 2020-03-04 14:20 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-04 14:20 UTC (permalink / raw)
To: Andrew Jones
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 11:19 AM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:50AM +0100, Eric Auger wrote:
>> Detect the presence of an ITS as part of the GICv3 init
>> routine, initialize its base address and read few registers
>> the IIDR, the TYPER to store its dimensioning parameters.
>> Also parse the BASER registers.
>>
>> This is our first ITS test, belonging to a new "its" group.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - updated dates and changed author
>> - squash "arm/arm64: ITS: Test BASER" into this patch but
>> removes setup_baser which will be introduced later.
>> - only compile on aarch64
>> - restrict the new test to aarch64
>>
>> v1 -> v2:
>> - clean GITS_TYPER macros and unused fields in typer struct
>> - remove memory attribute related macros
>> - remove everything related to memory attributes
>> - s/dev_baser/coll_baser/ in report_info
>> - add extra line
>> - removed index filed in its_baser
>> ---
>> arm/Makefile.arm64 | 1 +
>> arm/gic.c | 49 ++++++++++++++++++
>> arm/unittests.cfg | 7 +++
>> lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
>> lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
>> lib/arm/gic.c | 30 +++++++++--
>> lib/arm64/asm/gic-v3-its.h | 1 +
>> 7 files changed, 274 insertions(+), 5 deletions(-)
>> create mode 100644 lib/arm/asm/gic-v3-its.h
>> create mode 100644 lib/arm/gic-v3-its.c
>> create mode 100644 lib/arm64/asm/gic-v3-its.h
>>
>> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
>> index 6d3dc2c..2571ffb 100644
>> --- a/arm/Makefile.arm64
>> +++ b/arm/Makefile.arm64
>> @@ -19,6 +19,7 @@ endef
>> cstart.o = $(TEST_DIR)/cstart64.o
>> cflatobjs += lib/arm64/processor.o
>> cflatobjs += lib/arm64/spinlock.o
>> +cflatobjs += lib/arm/gic-v3-its.o
>
> If gic-v3-its.c will never be compiled for arm, then it should
> probably go lib/arm64, not lib/arm. Same comment for all other
> new source and header files. The only problem with that is...>
>>
>> OBJDIRS += lib/arm64
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index abf08c7..4d7dd03 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -16,6 +16,7 @@
>> #include <asm/processor.h>
>> #include <asm/delay.h>
>> #include <asm/gic.h>
>> +#include <asm/gic-v3-its.h>
>
> ...here where we'd have to do
>
> #if defined(__aarch64__)
> #include <asm/gic-v3-its.h>
> #endif
>
> which is ugly. However that can be avoided by adding an
> empty lib/arm/asm/gic-v3-its.h file.
OK made those changes with this latter containing stubs.
>
>
>> #include <asm/smp.h>
>> #include <asm/barrier.h>
>> #include <asm/io.h>
>> @@ -518,6 +519,50 @@ static void gic_test_mmio(void)
>> test_targets(nr_irqs);
>> }
>>
>> +#if defined(__arm__)
>> +
>> +static void test_its_introspection(void) {}
>
> static void test_its_introspection(void)
> {
> report_abort(...);
> }
OK
>
>> +
>> +#else /* __arch64__ */
>> +
>> +static void test_its_introspection(void)
>> +{
>> + struct its_baser *dev_baser, *coll_baser;
>> + struct its_typer *typer = &its_data.typer;
>> +
>> + if (!gicv3_its_base()) {
>> + report_skip("No ITS, skip ...");
>> + return;
>> + }
>> +
>> + /* IIDR */
>> + report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
>> + "GITS_IIDR is read-only"),
>> +
>> + /* TYPER */
>> + report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
>> + "GITS_TYPER is read-only");
>> +
>> + report(typer->phys_lpi, "ITS supports physical LPIs");
>> + report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
>> + report_info("ITT entry size = 0x%x", typer->ite_size);
>> + report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
>> + typer->eventid_bits, typer->deviceid_bits,
>> + typer->collid_bits);
>> + report(typer->eventid_bits && typer->deviceid_bits &&
>> + typer->collid_bits, "ID spaces");
>> + report_info("Target address format %s",
>> + typer->pta ? "Redist basse address" : "PE #");
>> +
>> + dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
>> + coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
>> + report(dev_baser && coll_baser, "detect device and collection BASER");
>> + report_info("device baser entry_size = 0x%x", dev_baser->esz);
>> + report_info("collection baser entry_size = 0x%x", coll_baser->esz);
>> +}
>> +
>> +#endif
>> +
>> int main(int argc, char **argv)
>> {
>> if (!gic_init()) {
>> @@ -549,6 +594,10 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> gic_test_mmio();
>> report_prefix_pop();
>> + } else if (strcmp(argv[1], "its-introspection") == 0) {
>> + report_prefix_push(argv[1]);
>> + test_its_introspection();
>> + report_prefix_pop();
>> } else {
>> report_abort("Unknown subtest '%s'", argv[1]);
>> }
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index daeb5a0..ba2b31b 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -122,6 +122,13 @@ smp = $MAX_SMP
>> extra_params = -machine gic-version=3 -append 'active'
>> groups = gic
>>
>> +[its-introspection]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +extra_params = -machine gic-version=3 -append 'its-introspection'
>> +groups = its
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> new file mode 100644
>> index 0000000..815c515
>> --- /dev/null
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -0,0 +1,103 @@
>> +/*
>> + * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
>> + *
>> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#ifndef _ASMARM_GIC_V3_ITS_H_
>> +#define _ASMARM_GIC_V3_ITS_H_
>> +
>> +#ifndef __ASSEMBLY__
>
> Doesn't look like you use the #else /* __ASSEMBLY__ */ side of this.
> I'd leave out the #ifndef until we add defines we need to access
> from assembler.
OK
>
>> +
>> +struct its_typer {
>> + unsigned int ite_size;
>> + unsigned int eventid_bits;
>> + unsigned int deviceid_bits;
>> + unsigned int collid_bits;
>> + bool pta;
>> + bool phys_lpi;
>> + bool virt_lpi;
>> +};
>> +
>> +struct its_baser {
>> + int type;
>> + size_t psz;
>> + int nr_pages;
>> + bool indirect;
>> + phys_addr_t table_addr;
>> + bool valid;
>> + int esz;
>> +};
>> +
>> +#define GITS_BASER_NR_REGS 8
>> +
>> +struct its_data {
>> + void *base;
>> + struct its_typer typer;
>> + struct its_baser baser[GITS_BASER_NR_REGS];
>> +};
>> +
>> +extern struct its_data its_data;
>> +
>> +#define gicv3_its_base() (its_data.base)
>> +
>> +#if defined(__aarch64__)
>> +
>> +#define GITS_CTLR 0x0000
>> +#define GITS_IIDR 0x0004
>> +#define GITS_TYPER 0x0008
>> +#define GITS_CBASER 0x0080
>> +#define GITS_CWRITER 0x0088
>> +#define GITS_CREADR 0x0090
>> +#define GITS_BASER 0x0100
>> +
>> +#define GITS_TYPER_PLPIS BIT(0)
>> +#define GITS_TYPER_VLPIS BIT(1)
>> +#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
>> +#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
>> +#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
>> +#define GITS_TYPER_IDBITS_SHIFT 8
>> +#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
>> +#define GITS_TYPER_DEVBITS_SHIFT 13
>> +#define GITS_TYPER_PTA BIT(19)
>> +#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
>> +#define GITS_TYPER_CIDBITS_SHIFT 32
>> +#define GITS_TYPER_CIL BIT(36)
>> +
>> +#define GITS_CTLR_ENABLE (1U << 0)
>> +
>> +#define GITS_CBASER_VALID (1UL << 63)
>> +
>> +#define GITS_BASER_VALID BIT(63)
>> +#define GITS_BASER_INDIRECT BIT(62)
>> +#define GITS_BASER_TYPE_SHIFT (56)
>> +#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
>> +#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
>> +#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
>> +#define GITS_BASER_PAGE_SIZE_SHIFT (8)
>> +#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGES_MAX 256
>> +#define GITS_BASER_PAGES_SHIFT (0)
>> +#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
>> +#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
>> +#define GITS_BASER_TYPE_NONE 0
>> +#define GITS_BASER_TYPE_DEVICE 1
>> +#define GITS_BASER_TYPE_COLLECTION 4
>> +
>> +extern void its_parse_typer(void);
>> +extern void its_init(void);
>> +extern int its_parse_baser(int i, struct its_baser *baser);
>> +extern struct its_baser *its_lookup_baser(int type);
>> +
>> +#else /* __arm__ */
>> +
>> +static inline void its_init(void) {}
>
> Looks like the empty gic-v3-its.h I suggested creating above will actually
> be useful. We can add stubs like this in it.
OK
>
>> +
>> +#endif
>> +
>> +#endif /* !__ASSEMBLY__ */
>> +#endif /* _ASMARM_GIC_V3_ITS_H_ */
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> new file mode 100644
>> index 0000000..2c0ce13
>> --- /dev/null
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -0,0 +1,88 @@
>> +/*
>> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#include <asm/gic.h>
>> +#include <alloc_page.h>
>> +#include <asm/gic-v3-its.h>
>> +
>> +void its_parse_typer(void)
>> +{
>> + u64 typer = readq(gicv3_its_base() + GITS_TYPER);
>> +
>> + its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
>> + GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
>> + its_data.typer.pta = typer & GITS_TYPER_PTA;
>> + its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
>> + GITS_TYPER_IDBITS_SHIFT) + 1;
>> + its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
>> + GITS_TYPER_DEVBITS_SHIFT) + 1;
>> +
>> + if (typer & GITS_TYPER_CIL)
>> + its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
>> + GITS_TYPER_CIDBITS_SHIFT) + 1;
>
> nit: please consider aligning like this
>
> ((typer & MASK) >>
> SHIFT) + 1;
>
> Or, maybe better to macro it
>
> #define TYPER_FIELD(typer, mask, shift) (((type) & (mask) >> (shift)) + 1)
>
> And, rather than have a bunch of 'its_data.typer's we could use an alias,
> helping us stay within 120 chars.
>
> struct its_typer *t = &its_data.typer;
yes used that trick and relied on 120 char max.
>
> t->ite_size = TYPER_FIELD(typer, GITS_TYPER_ITT_ENTRY_SIZE,
> GITS_TYPER_ITT_ENTRY_SIZE_SHIFT);
>
>
>> + else
>> + its_data.typer.collid_bits = 16;
>> +
>> + its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
>> + its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
>> +}
>> +
>> +int its_parse_baser(int i, struct its_baser *baser)
>> +{
>> + void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
>> + u64 val = readq(reg_addr);
>> +
>> + if (!val) {
>> + memset(baser, 0, sizeof(*baser));
>> + return -1;
>> + }
>> +
>> + baser->valid = val & GITS_BASER_VALID;
>> + baser->indirect = val & GITS_BASER_INDIRECT;
>> + baser->type = GITS_BASER_TYPE(val);
>> + baser->esz = GITS_BASER_ENTRY_SIZE(val);
>> + baser->nr_pages = GITS_BASER_NR_PAGES(val);
>> + baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
>> + switch (val & GITS_BASER_PAGE_SIZE_MASK) {
>> + case GITS_BASER_PAGE_SIZE_4K:
>> + baser->psz = SZ_4K;
>> + break;
>> + case GITS_BASER_PAGE_SIZE_16K:
>> + baser->psz = SZ_16K;
>> + break;
>> + case GITS_BASER_PAGE_SIZE_64K:
>> + baser->psz = SZ_64K;
>> + break;
>> + default:
>> + baser->psz = SZ_64K;
>> + }
>> + return 0;
>> +}
>> +
>> +struct its_baser *its_lookup_baser(int type)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
>> + struct its_baser *baser = &its_data.baser[i];
>> +
>> + if (baser->type == type)
>> + return baser;
>> + }
>> + return NULL;
>> +}
>> +
>> +void its_init(void)
>> +{
>> + int i;
>> +
>> + if (!its_data.base)
>> + return;
>> +
>> + its_parse_typer();
>> + for (i = 0; i < GITS_BASER_NR_REGS; i++)
>> + its_parse_baser(i, &its_data.baser[i]);
>> +}
>> +
>> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
>> index aa9cb86..6b70b05 100644
>> --- a/lib/arm/gic.c
>> +++ b/lib/arm/gic.c
>> @@ -6,9 +6,11 @@
>> #include <devicetree.h>
>> #include <asm/gic.h>
>> #include <asm/io.h>
>> +#include <asm/gic-v3-its.h>
>>
>> struct gicv2_data gicv2_data;
>> struct gicv3_data gicv3_data;
>> +struct its_data its_data;
>>
>> struct gic_common_ops {
>> void (*enable_defaults)(void);
>> @@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
>> * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>> */
>> static bool
>> -gic_get_dt_bases(const char *compatible, void **base1, void **base2)
>> +gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
>> {
>> struct dt_pbus_reg reg;
>> - struct dt_device gic;
>> + struct dt_device gic, its;
>> struct dt_bus bus;
>> - int node, ret, i;
>> + int node, subnode, ret, i, len;
>> + const void *fdt = dt_fdt();
>>
>> dt_bus_init_defaults(&bus);
>> dt_device_init(&gic, &bus, NULL);
>> @@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
>> base2[i] = ioremap(reg.addr, reg.size);
>> }
>>
>> + if (base3 && !strcmp(compatible, "arm,gic-v3")) {
>
> If base != NULL, then we could assert(strcmp(compatible, "arm,cortex-a15-gic") != 0)
OK
Thanks
Eric
>
>> + dt_for_each_subnode(node, subnode) {
>> + const struct fdt_property *prop;
>> +
>> + prop = fdt_get_property(fdt, subnode, "compatible", &len);
>> + if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
>> + dt_device_bind_node(&its, subnode);
>> + ret = dt_pbus_translate(&its, 0, ®);
>> + assert(ret == 0);
>> + *base3 = ioremap(reg.addr, reg.size);
>> + break;
>> + }
>> + }
>> +
>> + }
>> +
>> return true;
>> }
>>
>> int gicv2_init(void)
>> {
>> return gic_get_dt_bases("arm,cortex-a15-gic",
>> - &gicv2_data.dist_base, &gicv2_data.cpu_base);
>> + &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
>> }
>>
>> int gicv3_init(void)
>> {
>> return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
>> - &gicv3_data.redist_bases[0]);
>> + &gicv3_data.redist_bases[0], &its_data.base);
>> }
>>
>> int gic_version(void)
>> @@ -104,6 +123,7 @@ int gic_init(void)
>> gic_common_ops = &gicv2_common_ops;
>> else if (gicv3_init())
>> gic_common_ops = &gicv3_common_ops;
>> + its_init();
>> return gic_version();
>> }
>>
>> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
>> new file mode 100644
>> index 0000000..083cba4
>> --- /dev/null
>> +++ b/lib/arm64/asm/gic-v3-its.h
>> @@ -0,0 +1 @@
>> +#include "../../arm/asm/gic-v3-its.h"
>> --
>> 2.20.1
>>
>
> Thanks,
> drew
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests
@ 2020-03-04 14:20 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-04 14:20 UTC (permalink / raw)
To: Andrew Jones
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 11:19 AM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:50AM +0100, Eric Auger wrote:
>> Detect the presence of an ITS as part of the GICv3 init
>> routine, initialize its base address and read few registers
>> the IIDR, the TYPER to store its dimensioning parameters.
>> Also parse the BASER registers.
>>
>> This is our first ITS test, belonging to a new "its" group.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - updated dates and changed author
>> - squash "arm/arm64: ITS: Test BASER" into this patch but
>> removes setup_baser which will be introduced later.
>> - only compile on aarch64
>> - restrict the new test to aarch64
>>
>> v1 -> v2:
>> - clean GITS_TYPER macros and unused fields in typer struct
>> - remove memory attribute related macros
>> - remove everything related to memory attributes
>> - s/dev_baser/coll_baser/ in report_info
>> - add extra line
>> - removed index filed in its_baser
>> ---
>> arm/Makefile.arm64 | 1 +
>> arm/gic.c | 49 ++++++++++++++++++
>> arm/unittests.cfg | 7 +++
>> lib/arm/asm/gic-v3-its.h | 103 +++++++++++++++++++++++++++++++++++++
>> lib/arm/gic-v3-its.c | 88 +++++++++++++++++++++++++++++++
>> lib/arm/gic.c | 30 +++++++++--
>> lib/arm64/asm/gic-v3-its.h | 1 +
>> 7 files changed, 274 insertions(+), 5 deletions(-)
>> create mode 100644 lib/arm/asm/gic-v3-its.h
>> create mode 100644 lib/arm/gic-v3-its.c
>> create mode 100644 lib/arm64/asm/gic-v3-its.h
>>
>> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
>> index 6d3dc2c..2571ffb 100644
>> --- a/arm/Makefile.arm64
>> +++ b/arm/Makefile.arm64
>> @@ -19,6 +19,7 @@ endef
>> cstart.o = $(TEST_DIR)/cstart64.o
>> cflatobjs += lib/arm64/processor.o
>> cflatobjs += lib/arm64/spinlock.o
>> +cflatobjs += lib/arm/gic-v3-its.o
>
> If gic-v3-its.c will never be compiled for arm, then it should
> probably go lib/arm64, not lib/arm. Same comment for all other
> new source and header files. The only problem with that is...>
>>
>> OBJDIRS += lib/arm64
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index abf08c7..4d7dd03 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -16,6 +16,7 @@
>> #include <asm/processor.h>
>> #include <asm/delay.h>
>> #include <asm/gic.h>
>> +#include <asm/gic-v3-its.h>
>
> ...here where we'd have to do
>
> #if defined(__aarch64__)
> #include <asm/gic-v3-its.h>
> #endif
>
> which is ugly. However that can be avoided by adding an
> empty lib/arm/asm/gic-v3-its.h file.
OK made those changes with this latter containing stubs.
>
>
>> #include <asm/smp.h>
>> #include <asm/barrier.h>
>> #include <asm/io.h>
>> @@ -518,6 +519,50 @@ static void gic_test_mmio(void)
>> test_targets(nr_irqs);
>> }
>>
>> +#if defined(__arm__)
>> +
>> +static void test_its_introspection(void) {}
>
> static void test_its_introspection(void)
> {
> report_abort(...);
> }
OK
>
>> +
>> +#else /* __arch64__ */
>> +
>> +static void test_its_introspection(void)
>> +{
>> + struct its_baser *dev_baser, *coll_baser;
>> + struct its_typer *typer = &its_data.typer;
>> +
>> + if (!gicv3_its_base()) {
>> + report_skip("No ITS, skip ...");
>> + return;
>> + }
>> +
>> + /* IIDR */
>> + report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
>> + "GITS_IIDR is read-only"),
>> +
>> + /* TYPER */
>> + report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
>> + "GITS_TYPER is read-only");
>> +
>> + report(typer->phys_lpi, "ITS supports physical LPIs");
>> + report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
>> + report_info("ITT entry size = 0x%x", typer->ite_size);
>> + report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
>> + typer->eventid_bits, typer->deviceid_bits,
>> + typer->collid_bits);
>> + report(typer->eventid_bits && typer->deviceid_bits &&
>> + typer->collid_bits, "ID spaces");
>> + report_info("Target address format %s",
>> + typer->pta ? "Redist basse address" : "PE #");
>> +
>> + dev_baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
>> + coll_baser = its_lookup_baser(GITS_BASER_TYPE_COLLECTION);
>> + report(dev_baser && coll_baser, "detect device and collection BASER");
>> + report_info("device baser entry_size = 0x%x", dev_baser->esz);
>> + report_info("collection baser entry_size = 0x%x", coll_baser->esz);
>> +}
>> +
>> +#endif
>> +
>> int main(int argc, char **argv)
>> {
>> if (!gic_init()) {
>> @@ -549,6 +594,10 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> gic_test_mmio();
>> report_prefix_pop();
>> + } else if (strcmp(argv[1], "its-introspection") == 0) {
>> + report_prefix_push(argv[1]);
>> + test_its_introspection();
>> + report_prefix_pop();
>> } else {
>> report_abort("Unknown subtest '%s'", argv[1]);
>> }
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index daeb5a0..ba2b31b 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -122,6 +122,13 @@ smp = $MAX_SMP
>> extra_params = -machine gic-version=3 -append 'active'
>> groups = gic
>>
>> +[its-introspection]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +extra_params = -machine gic-version=3 -append 'its-introspection'
>> +groups = its
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> new file mode 100644
>> index 0000000..815c515
>> --- /dev/null
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -0,0 +1,103 @@
>> +/*
>> + * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
>> + *
>> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#ifndef _ASMARM_GIC_V3_ITS_H_
>> +#define _ASMARM_GIC_V3_ITS_H_
>> +
>> +#ifndef __ASSEMBLY__
>
> Doesn't look like you use the #else /* __ASSEMBLY__ */ side of this.
> I'd leave out the #ifndef until we add defines we need to access
> from assembler.
OK
>
>> +
>> +struct its_typer {
>> + unsigned int ite_size;
>> + unsigned int eventid_bits;
>> + unsigned int deviceid_bits;
>> + unsigned int collid_bits;
>> + bool pta;
>> + bool phys_lpi;
>> + bool virt_lpi;
>> +};
>> +
>> +struct its_baser {
>> + int type;
>> + size_t psz;
>> + int nr_pages;
>> + bool indirect;
>> + phys_addr_t table_addr;
>> + bool valid;
>> + int esz;
>> +};
>> +
>> +#define GITS_BASER_NR_REGS 8
>> +
>> +struct its_data {
>> + void *base;
>> + struct its_typer typer;
>> + struct its_baser baser[GITS_BASER_NR_REGS];
>> +};
>> +
>> +extern struct its_data its_data;
>> +
>> +#define gicv3_its_base() (its_data.base)
>> +
>> +#if defined(__aarch64__)
>> +
>> +#define GITS_CTLR 0x0000
>> +#define GITS_IIDR 0x0004
>> +#define GITS_TYPER 0x0008
>> +#define GITS_CBASER 0x0080
>> +#define GITS_CWRITER 0x0088
>> +#define GITS_CREADR 0x0090
>> +#define GITS_BASER 0x0100
>> +
>> +#define GITS_TYPER_PLPIS BIT(0)
>> +#define GITS_TYPER_VLPIS BIT(1)
>> +#define GITS_TYPER_ITT_ENTRY_SIZE GENMASK_ULL(7, 4)
>> +#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT 4
>> +#define GITS_TYPER_IDBITS GENMASK_ULL(8, 12)
>> +#define GITS_TYPER_IDBITS_SHIFT 8
>> +#define GITS_TYPER_DEVBITS GENMASK_ULL(13, 17)
>> +#define GITS_TYPER_DEVBITS_SHIFT 13
>> +#define GITS_TYPER_PTA BIT(19)
>> +#define GITS_TYPER_CIDBITS GENMASK_ULL(32, 35)
>> +#define GITS_TYPER_CIDBITS_SHIFT 32
>> +#define GITS_TYPER_CIL BIT(36)
>> +
>> +#define GITS_CTLR_ENABLE (1U << 0)
>> +
>> +#define GITS_CBASER_VALID (1UL << 63)
>> +
>> +#define GITS_BASER_VALID BIT(63)
>> +#define GITS_BASER_INDIRECT BIT(62)
>> +#define GITS_BASER_TYPE_SHIFT (56)
>> +#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
>> +#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
>> +#define GITS_BASER_ENTRY_SIZE(r) ((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
>> +#define GITS_BASER_PAGE_SIZE_SHIFT (8)
>> +#define GITS_BASER_PAGE_SIZE_4K (0UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGE_SIZE_16K (1UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGE_SIZE_64K (2UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGE_SIZE_MASK (3UL << GITS_BASER_PAGE_SIZE_SHIFT)
>> +#define GITS_BASER_PAGES_MAX 256
>> +#define GITS_BASER_PAGES_SHIFT (0)
>> +#define GITS_BASER_NR_PAGES(r) (((r) & 0xff) + 1)
>> +#define GITS_BASER_PHYS_ADDR_MASK 0xFFFFFFFFF000
>> +#define GITS_BASER_TYPE_NONE 0
>> +#define GITS_BASER_TYPE_DEVICE 1
>> +#define GITS_BASER_TYPE_COLLECTION 4
>> +
>> +extern void its_parse_typer(void);
>> +extern void its_init(void);
>> +extern int its_parse_baser(int i, struct its_baser *baser);
>> +extern struct its_baser *its_lookup_baser(int type);
>> +
>> +#else /* __arm__ */
>> +
>> +static inline void its_init(void) {}
>
> Looks like the empty gic-v3-its.h I suggested creating above will actually
> be useful. We can add stubs like this in it.
OK
>
>> +
>> +#endif
>> +
>> +#endif /* !__ASSEMBLY__ */
>> +#endif /* _ASMARM_GIC_V3_ITS_H_ */
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> new file mode 100644
>> index 0000000..2c0ce13
>> --- /dev/null
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -0,0 +1,88 @@
>> +/*
>> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#include <asm/gic.h>
>> +#include <alloc_page.h>
>> +#include <asm/gic-v3-its.h>
>> +
>> +void its_parse_typer(void)
>> +{
>> + u64 typer = readq(gicv3_its_base() + GITS_TYPER);
>> +
>> + its_data.typer.ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >>
>> + GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
>> + its_data.typer.pta = typer & GITS_TYPER_PTA;
>> + its_data.typer.eventid_bits = ((typer & GITS_TYPER_IDBITS) >>
>> + GITS_TYPER_IDBITS_SHIFT) + 1;
>> + its_data.typer.deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >>
>> + GITS_TYPER_DEVBITS_SHIFT) + 1;
>> +
>> + if (typer & GITS_TYPER_CIL)
>> + its_data.typer.collid_bits = ((typer & GITS_TYPER_CIDBITS) >>
>> + GITS_TYPER_CIDBITS_SHIFT) + 1;
>
> nit: please consider aligning like this
>
> ((typer & MASK) >>
> SHIFT) + 1;
>
> Or, maybe better to macro it
>
> #define TYPER_FIELD(typer, mask, shift) (((type) & (mask) >> (shift)) + 1)
>
> And, rather than have a bunch of 'its_data.typer's we could use an alias,
> helping us stay within 120 chars.
>
> struct its_typer *t = &its_data.typer;
yes used that trick and relied on 120 char max.
>
> t->ite_size = TYPER_FIELD(typer, GITS_TYPER_ITT_ENTRY_SIZE,
> GITS_TYPER_ITT_ENTRY_SIZE_SHIFT);
>
>
>> + else
>> + its_data.typer.collid_bits = 16;
>> +
>> + its_data.typer.virt_lpi = typer & GITS_TYPER_VLPIS;
>> + its_data.typer.phys_lpi = typer & GITS_TYPER_PLPIS;
>> +}
>> +
>> +int its_parse_baser(int i, struct its_baser *baser)
>> +{
>> + void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
>> + u64 val = readq(reg_addr);
>> +
>> + if (!val) {
>> + memset(baser, 0, sizeof(*baser));
>> + return -1;
>> + }
>> +
>> + baser->valid = val & GITS_BASER_VALID;
>> + baser->indirect = val & GITS_BASER_INDIRECT;
>> + baser->type = GITS_BASER_TYPE(val);
>> + baser->esz = GITS_BASER_ENTRY_SIZE(val);
>> + baser->nr_pages = GITS_BASER_NR_PAGES(val);
>> + baser->table_addr = val & GITS_BASER_PHYS_ADDR_MASK;
>> + switch (val & GITS_BASER_PAGE_SIZE_MASK) {
>> + case GITS_BASER_PAGE_SIZE_4K:
>> + baser->psz = SZ_4K;
>> + break;
>> + case GITS_BASER_PAGE_SIZE_16K:
>> + baser->psz = SZ_16K;
>> + break;
>> + case GITS_BASER_PAGE_SIZE_64K:
>> + baser->psz = SZ_64K;
>> + break;
>> + default:
>> + baser->psz = SZ_64K;
>> + }
>> + return 0;
>> +}
>> +
>> +struct its_baser *its_lookup_baser(int type)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
>> + struct its_baser *baser = &its_data.baser[i];
>> +
>> + if (baser->type == type)
>> + return baser;
>> + }
>> + return NULL;
>> +}
>> +
>> +void its_init(void)
>> +{
>> + int i;
>> +
>> + if (!its_data.base)
>> + return;
>> +
>> + its_parse_typer();
>> + for (i = 0; i < GITS_BASER_NR_REGS; i++)
>> + its_parse_baser(i, &its_data.baser[i]);
>> +}
>> +
>> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
>> index aa9cb86..6b70b05 100644
>> --- a/lib/arm/gic.c
>> +++ b/lib/arm/gic.c
>> @@ -6,9 +6,11 @@
>> #include <devicetree.h>
>> #include <asm/gic.h>
>> #include <asm/io.h>
>> +#include <asm/gic-v3-its.h>
>>
>> struct gicv2_data gicv2_data;
>> struct gicv3_data gicv3_data;
>> +struct its_data its_data;
>>
>> struct gic_common_ops {
>> void (*enable_defaults)(void);
>> @@ -44,12 +46,13 @@ static const struct gic_common_ops gicv3_common_ops = {
>> * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
>> */
>> static bool
>> -gic_get_dt_bases(const char *compatible, void **base1, void **base2)
>> +gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
>> {
>> struct dt_pbus_reg reg;
>> - struct dt_device gic;
>> + struct dt_device gic, its;
>> struct dt_bus bus;
>> - int node, ret, i;
>> + int node, subnode, ret, i, len;
>> + const void *fdt = dt_fdt();
>>
>> dt_bus_init_defaults(&bus);
>> dt_device_init(&gic, &bus, NULL);
>> @@ -74,19 +77,35 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
>> base2[i] = ioremap(reg.addr, reg.size);
>> }
>>
>> + if (base3 && !strcmp(compatible, "arm,gic-v3")) {
>
> If base != NULL, then we could assert(strcmp(compatible, "arm,cortex-a15-gic") != 0)
OK
Thanks
Eric
>
>> + dt_for_each_subnode(node, subnode) {
>> + const struct fdt_property *prop;
>> +
>> + prop = fdt_get_property(fdt, subnode, "compatible", &len);
>> + if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
>> + dt_device_bind_node(&its, subnode);
>> + ret = dt_pbus_translate(&its, 0, ®);
>> + assert(ret == 0);
>> + *base3 = ioremap(reg.addr, reg.size);
>> + break;
>> + }
>> + }
>> +
>> + }
>> +
>> return true;
>> }
>>
>> int gicv2_init(void)
>> {
>> return gic_get_dt_bases("arm,cortex-a15-gic",
>> - &gicv2_data.dist_base, &gicv2_data.cpu_base);
>> + &gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
>> }
>>
>> int gicv3_init(void)
>> {
>> return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
>> - &gicv3_data.redist_bases[0]);
>> + &gicv3_data.redist_bases[0], &its_data.base);
>> }
>>
>> int gic_version(void)
>> @@ -104,6 +123,7 @@ int gic_init(void)
>> gic_common_ops = &gicv2_common_ops;
>> else if (gicv3_init())
>> gic_common_ops = &gicv3_common_ops;
>> + its_init();
>> return gic_version();
>> }
>>
>> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
>> new file mode 100644
>> index 0000000..083cba4
>> --- /dev/null
>> +++ b/lib/arm64/asm/gic-v3-its.h
>> @@ -0,0 +1 @@
>> +#include "../../arm/asm/gic-v3-its.h"
>> --
>> 2.20.1
>>
>
> Thanks,
> drew
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
2020-02-07 3:20 ` Zenghui Yu
(?)
@ 2020-03-04 14:26 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-04 14:26 UTC (permalink / raw)
To: Zenghui Yu, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, alexandru.elisei, thuth
Hi Zenghui,
On 2/7/20 4:20 AM, Zenghui Yu wrote:
> Hi Eric,
>
> On 2020/1/28 18:34, Eric Auger wrote:
>> its_enable_defaults() is the top init function that allocates the
>> command queue and all the requested tables (device, collection,
>> lpi config and pending tables), enable LPIs at distributor level
>> and ITS level.
>>
>> gicv3_enable_defaults must be called before.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - introduce its_setup_baser in this patch
>> - squash "arm/arm64: ITS: Init the command queue" in this patch.
>> ---
>> lib/arm/asm/gic-v3-its.h | 8 ++++
>> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 97 insertions(+)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index 815c515..fe73c04 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -36,6 +36,8 @@ struct its_data {
>> void *base;
>> struct its_typer typer;
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> + struct its_cmd_block *cmd_base;
>> + struct its_cmd_block *cmd_write;
>> };
>> extern struct its_data its_data;
>> @@ -88,10 +90,16 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>> +
>> +struct its_cmd_block {
>> + u64 raw_cmd[4];
>> +};
>> +
>> extern void its_parse_typer(void);
>> extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> +extern void its_enable_defaults(void);
>> #else /* __arm__ */
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index 2c0ce13..d1e7e52 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -86,3 +86,92 @@ void its_init(void)
>> its_parse_baser(i, &its_data.baser[i]);
>> }
>> +static void its_setup_baser(int i, struct its_baser *baser)
>> +{
>> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
>> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>> + u64 val;
>> +
>> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
>> +
>> + val = ((u64)baser->table_addr |
>> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
>> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
>> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
>> + (u64)baser->indirect << 62 |
>
> I haven't seen the 'nr_pages' and 'indirect' are programmed anywhere
> except in its_parse_baser(). It looks like they're treated as RO (but
> they shouldn't) and I now don't think it makes sense to parse them in
> its_parse_baser(), in patch#5.
First of all please forgive me for the delay.
I agree with you on nr_pages. However indirect also indicates the BASER
capability to support or not 2 level tables. So I think it makes sense
to read it on init.
>
>> + (u64)baser->valid << 63);
>> +
>> + switch (baser->psz) {
>> + case SZ_4K:
>> + val |= GITS_BASER_PAGE_SIZE_4K;
>> + break;
>> + case SZ_16K:
>> + val |= GITS_BASER_PAGE_SIZE_16K;
>> + break;
>> + case SZ_64K:
>> + val |= GITS_BASER_PAGE_SIZE_64K;
>> + break;
>> + }
>> +
>> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
>> +}
>> +
>> +/**
>> + * init_cmd_queue: Allocate the command queue and initialize
>> + * CBASER, CREADR, CWRITER
>
> no 'CREADR'.
OK
Thanks
Eric
>
>
> Thanks,
> Zenghui
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-03-04 14:26 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-04 14:26 UTC (permalink / raw)
To: Zenghui Yu, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, drjones, alexandru.elisei, thuth, peter.maydell
Hi Zenghui,
On 2/7/20 4:20 AM, Zenghui Yu wrote:
> Hi Eric,
>
> On 2020/1/28 18:34, Eric Auger wrote:
>> its_enable_defaults() is the top init function that allocates the
>> command queue and all the requested tables (device, collection,
>> lpi config and pending tables), enable LPIs at distributor level
>> and ITS level.
>>
>> gicv3_enable_defaults must be called before.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - introduce its_setup_baser in this patch
>> - squash "arm/arm64: ITS: Init the command queue" in this patch.
>> ---
>> lib/arm/asm/gic-v3-its.h | 8 ++++
>> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 97 insertions(+)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index 815c515..fe73c04 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -36,6 +36,8 @@ struct its_data {
>> void *base;
>> struct its_typer typer;
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> + struct its_cmd_block *cmd_base;
>> + struct its_cmd_block *cmd_write;
>> };
>> extern struct its_data its_data;
>> @@ -88,10 +90,16 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>> +
>> +struct its_cmd_block {
>> + u64 raw_cmd[4];
>> +};
>> +
>> extern void its_parse_typer(void);
>> extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> +extern void its_enable_defaults(void);
>> #else /* __arm__ */
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index 2c0ce13..d1e7e52 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -86,3 +86,92 @@ void its_init(void)
>> its_parse_baser(i, &its_data.baser[i]);
>> }
>> +static void its_setup_baser(int i, struct its_baser *baser)
>> +{
>> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
>> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>> + u64 val;
>> +
>> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
>> +
>> + val = ((u64)baser->table_addr |
>> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
>> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
>> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
>> + (u64)baser->indirect << 62 |
>
> I haven't seen the 'nr_pages' and 'indirect' are programmed anywhere
> except in its_parse_baser(). It looks like they're treated as RO (but
> they shouldn't) and I now don't think it makes sense to parse them in
> its_parse_baser(), in patch#5.
First of all please forgive me for the delay.
I agree with you on nr_pages. However indirect also indicates the BASER
capability to support or not 2 level tables. So I think it makes sense
to read it on init.
>
>> + (u64)baser->valid << 63);
>> +
>> + switch (baser->psz) {
>> + case SZ_4K:
>> + val |= GITS_BASER_PAGE_SIZE_4K;
>> + break;
>> + case SZ_16K:
>> + val |= GITS_BASER_PAGE_SIZE_16K;
>> + break;
>> + case SZ_64K:
>> + val |= GITS_BASER_PAGE_SIZE_64K;
>> + break;
>> + }
>> +
>> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
>> +}
>> +
>> +/**
>> + * init_cmd_queue: Allocate the command queue and initialize
>> + * CBASER, CREADR, CWRITER
>
> no 'CREADR'.
OK
Thanks
Eric
>
>
> Thanks,
> Zenghui
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-03-04 14:26 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-04 14:26 UTC (permalink / raw)
To: Zenghui Yu, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Hi Zenghui,
On 2/7/20 4:20 AM, Zenghui Yu wrote:
> Hi Eric,
>
> On 2020/1/28 18:34, Eric Auger wrote:
>> its_enable_defaults() is the top init function that allocates the
>> command queue and all the requested tables (device, collection,
>> lpi config and pending tables), enable LPIs at distributor level
>> and ITS level.
>>
>> gicv3_enable_defaults must be called before.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - introduce its_setup_baser in this patch
>> - squash "arm/arm64: ITS: Init the command queue" in this patch.
>> ---
>> lib/arm/asm/gic-v3-its.h | 8 ++++
>> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 97 insertions(+)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index 815c515..fe73c04 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -36,6 +36,8 @@ struct its_data {
>> void *base;
>> struct its_typer typer;
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> + struct its_cmd_block *cmd_base;
>> + struct its_cmd_block *cmd_write;
>> };
>> extern struct its_data its_data;
>> @@ -88,10 +90,16 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>> +
>> +struct its_cmd_block {
>> + u64 raw_cmd[4];
>> +};
>> +
>> extern void its_parse_typer(void);
>> extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> +extern void its_enable_defaults(void);
>> #else /* __arm__ */
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index 2c0ce13..d1e7e52 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -86,3 +86,92 @@ void its_init(void)
>> its_parse_baser(i, &its_data.baser[i]);
>> }
>> +static void its_setup_baser(int i, struct its_baser *baser)
>> +{
>> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
>> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>> + u64 val;
>> +
>> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
>> +
>> + val = ((u64)baser->table_addr |
>> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
>> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
>> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
>> + (u64)baser->indirect << 62 |
>
> I haven't seen the 'nr_pages' and 'indirect' are programmed anywhere
> except in its_parse_baser(). It looks like they're treated as RO (but
> they shouldn't) and I now don't think it makes sense to parse them in
> its_parse_baser(), in patch#5.
First of all please forgive me for the delay.
I agree with you on nr_pages. However indirect also indicates the BASER
capability to support or not 2 level tables. So I think it makes sense
to read it on init.
>
>> + (u64)baser->valid << 63);
>> +
>> + switch (baser->psz) {
>> + case SZ_4K:
>> + val |= GITS_BASER_PAGE_SIZE_4K;
>> + break;
>> + case SZ_16K:
>> + val |= GITS_BASER_PAGE_SIZE_16K;
>> + break;
>> + case SZ_64K:
>> + val |= GITS_BASER_PAGE_SIZE_64K;
>> + break;
>> + }
>> +
>> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
>> +}
>> +
>> +/**
>> + * init_cmd_queue: Allocate the command queue and initialize
>> + * CBASER, CREADR, CWRITER
>
> no 'CREADR'.
OK
Thanks
Eric
>
>
> Thanks,
> Zenghui
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
2020-03-04 14:26 ` Auger Eric
(?)
@ 2020-03-05 6:30 ` Zenghui Yu
-1 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-03-05 6:30 UTC (permalink / raw)
To: Auger Eric, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, alexandru.elisei, thuth
Hi Eric,
On 2020/3/4 22:26, Auger Eric wrote:
> Hi Zenghui,
> On 2/7/20 4:20 AM, Zenghui Yu wrote:
>> Hi Eric,
>>
>> On 2020/1/28 18:34, Eric Auger wrote:
>>> its_enable_defaults() is the top init function that allocates the
>>> command queue and all the requested tables (device, collection,
>>> lpi config and pending tables), enable LPIs at distributor level
>>> and ITS level.
>>>
>>> gicv3_enable_defaults must be called before.
>>>
>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>
>>> ---
>>>
>>> v2 -> v3:
>>> - introduce its_setup_baser in this patch
>>> - squash "arm/arm64: ITS: Init the command queue" in this patch.
>>> ---
>>> lib/arm/asm/gic-v3-its.h | 8 ++++
>>> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
>>> 2 files changed, 97 insertions(+)
>>>
>>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>>> index 815c515..fe73c04 100644
>>> --- a/lib/arm/asm/gic-v3-its.h
>>> +++ b/lib/arm/asm/gic-v3-its.h
>>> @@ -36,6 +36,8 @@ struct its_data {
>>> void *base;
>>> struct its_typer typer;
>>> struct its_baser baser[GITS_BASER_NR_REGS];
>>> + struct its_cmd_block *cmd_base;
>>> + struct its_cmd_block *cmd_write;
>>> };
>>> extern struct its_data its_data;
>>> @@ -88,10 +90,16 @@ extern struct its_data its_data;
>>> #define GITS_BASER_TYPE_DEVICE 1
>>> #define GITS_BASER_TYPE_COLLECTION 4
>>> +
>>> +struct its_cmd_block {
>>> + u64 raw_cmd[4];
>>> +};
>>> +
>>> extern void its_parse_typer(void);
>>> extern void its_init(void);
>>> extern int its_parse_baser(int i, struct its_baser *baser);
>>> extern struct its_baser *its_lookup_baser(int type);
>>> +extern void its_enable_defaults(void);
>>> #else /* __arm__ */
>>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>>> index 2c0ce13..d1e7e52 100644
>>> --- a/lib/arm/gic-v3-its.c
>>> +++ b/lib/arm/gic-v3-its.c
>>> @@ -86,3 +86,92 @@ void its_init(void)
>>> its_parse_baser(i, &its_data.baser[i]);
>>> }
>>> +static void its_setup_baser(int i, struct its_baser *baser)
>>> +{
>>> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
>>> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>>> + u64 val;
>>> +
>>> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
>>> +
>>> + val = ((u64)baser->table_addr |
>>> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
>>> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
>>> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
>>> + (u64)baser->indirect << 62 |
>>
>> I haven't seen the 'nr_pages' and 'indirect' are programmed anywhere
>> except in its_parse_baser(). It looks like they're treated as RO (but
>> they shouldn't) and I now don't think it makes sense to parse them in
>> its_parse_baser(), in patch#5.
>
> First of all please forgive me for the delay.
Never mind.
>
> I agree with you on nr_pages. However indirect also indicates the BASER
> capability to support or not 2 level tables. So I think it makes sense
> to read it on init.
Yes, you're right. As the spec says, the Indirect field "is RAZ/WI for
GIC implementations that only support flat tables".
Thanks,
Zenghui
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-03-05 6:30 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-03-05 6:30 UTC (permalink / raw)
To: Auger Eric, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, drjones, alexandru.elisei, thuth, peter.maydell
Hi Eric,
On 2020/3/4 22:26, Auger Eric wrote:
> Hi Zenghui,
> On 2/7/20 4:20 AM, Zenghui Yu wrote:
>> Hi Eric,
>>
>> On 2020/1/28 18:34, Eric Auger wrote:
>>> its_enable_defaults() is the top init function that allocates the
>>> command queue and all the requested tables (device, collection,
>>> lpi config and pending tables), enable LPIs at distributor level
>>> and ITS level.
>>>
>>> gicv3_enable_defaults must be called before.
>>>
>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>
>>> ---
>>>
>>> v2 -> v3:
>>> - introduce its_setup_baser in this patch
>>> - squash "arm/arm64: ITS: Init the command queue" in this patch.
>>> ---
>>> lib/arm/asm/gic-v3-its.h | 8 ++++
>>> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
>>> 2 files changed, 97 insertions(+)
>>>
>>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>>> index 815c515..fe73c04 100644
>>> --- a/lib/arm/asm/gic-v3-its.h
>>> +++ b/lib/arm/asm/gic-v3-its.h
>>> @@ -36,6 +36,8 @@ struct its_data {
>>> void *base;
>>> struct its_typer typer;
>>> struct its_baser baser[GITS_BASER_NR_REGS];
>>> + struct its_cmd_block *cmd_base;
>>> + struct its_cmd_block *cmd_write;
>>> };
>>> extern struct its_data its_data;
>>> @@ -88,10 +90,16 @@ extern struct its_data its_data;
>>> #define GITS_BASER_TYPE_DEVICE 1
>>> #define GITS_BASER_TYPE_COLLECTION 4
>>> +
>>> +struct its_cmd_block {
>>> + u64 raw_cmd[4];
>>> +};
>>> +
>>> extern void its_parse_typer(void);
>>> extern void its_init(void);
>>> extern int its_parse_baser(int i, struct its_baser *baser);
>>> extern struct its_baser *its_lookup_baser(int type);
>>> +extern void its_enable_defaults(void);
>>> #else /* __arm__ */
>>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>>> index 2c0ce13..d1e7e52 100644
>>> --- a/lib/arm/gic-v3-its.c
>>> +++ b/lib/arm/gic-v3-its.c
>>> @@ -86,3 +86,92 @@ void its_init(void)
>>> its_parse_baser(i, &its_data.baser[i]);
>>> }
>>> +static void its_setup_baser(int i, struct its_baser *baser)
>>> +{
>>> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
>>> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>>> + u64 val;
>>> +
>>> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
>>> +
>>> + val = ((u64)baser->table_addr |
>>> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
>>> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
>>> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
>>> + (u64)baser->indirect << 62 |
>>
>> I haven't seen the 'nr_pages' and 'indirect' are programmed anywhere
>> except in its_parse_baser(). It looks like they're treated as RO (but
>> they shouldn't) and I now don't think it makes sense to parse them in
>> its_parse_baser(), in patch#5.
>
> First of all please forgive me for the delay.
Never mind.
>
> I agree with you on nr_pages. However indirect also indicates the BASER
> capability to support or not 2 level tables. So I think it makes sense
> to read it on init.
Yes, you're right. As the spec says, the Indirect field "is RAZ/WI for
GIC implementations that only support flat tables".
Thanks,
Zenghui
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-03-05 6:30 ` Zenghui Yu
0 siblings, 0 replies; 137+ messages in thread
From: Zenghui Yu @ 2020-03-05 6:30 UTC (permalink / raw)
To: Auger Eric, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Hi Eric,
On 2020/3/4 22:26, Auger Eric wrote:
> Hi Zenghui,
> On 2/7/20 4:20 AM, Zenghui Yu wrote:
>> Hi Eric,
>>
>> On 2020/1/28 18:34, Eric Auger wrote:
>>> its_enable_defaults() is the top init function that allocates the
>>> command queue and all the requested tables (device, collection,
>>> lpi config and pending tables), enable LPIs at distributor level
>>> and ITS level.
>>>
>>> gicv3_enable_defaults must be called before.
>>>
>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>
>>> ---
>>>
>>> v2 -> v3:
>>> - introduce its_setup_baser in this patch
>>> - squash "arm/arm64: ITS: Init the command queue" in this patch.
>>> ---
>>> lib/arm/asm/gic-v3-its.h | 8 ++++
>>> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
>>> 2 files changed, 97 insertions(+)
>>>
>>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>>> index 815c515..fe73c04 100644
>>> --- a/lib/arm/asm/gic-v3-its.h
>>> +++ b/lib/arm/asm/gic-v3-its.h
>>> @@ -36,6 +36,8 @@ struct its_data {
>>> void *base;
>>> struct its_typer typer;
>>> struct its_baser baser[GITS_BASER_NR_REGS];
>>> + struct its_cmd_block *cmd_base;
>>> + struct its_cmd_block *cmd_write;
>>> };
>>> extern struct its_data its_data;
>>> @@ -88,10 +90,16 @@ extern struct its_data its_data;
>>> #define GITS_BASER_TYPE_DEVICE 1
>>> #define GITS_BASER_TYPE_COLLECTION 4
>>> +
>>> +struct its_cmd_block {
>>> + u64 raw_cmd[4];
>>> +};
>>> +
>>> extern void its_parse_typer(void);
>>> extern void its_init(void);
>>> extern int its_parse_baser(int i, struct its_baser *baser);
>>> extern struct its_baser *its_lookup_baser(int type);
>>> +extern void its_enable_defaults(void);
>>> #else /* __arm__ */
>>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>>> index 2c0ce13..d1e7e52 100644
>>> --- a/lib/arm/gic-v3-its.c
>>> +++ b/lib/arm/gic-v3-its.c
>>> @@ -86,3 +86,92 @@ void its_init(void)
>>> its_parse_baser(i, &its_data.baser[i]);
>>> }
>>> +static void its_setup_baser(int i, struct its_baser *baser)
>>> +{
>>> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
>>> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>>> + u64 val;
>>> +
>>> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
>>> +
>>> + val = ((u64)baser->table_addr |
>>> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
>>> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
>>> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
>>> + (u64)baser->indirect << 62 |
>>
>> I haven't seen the 'nr_pages' and 'indirect' are programmed anywhere
>> except in its_parse_baser(). It looks like they're treated as RO (but
>> they shouldn't) and I now don't think it makes sense to parse them in
>> its_parse_baser(), in patch#5.
>
> First of all please forgive me for the delay.
Never mind.
>
> I agree with you on nr_pages. However indirect also indicates the BASER
> capability to support or not 2 level tables. So I think it makes sense
> to read it on init.
Yes, you're right. As the spec says, the Indirect field "is RAZ/WI for
GIC implementations that only support flat tables".
Thanks,
Zenghui
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
2020-02-07 12:41 ` Andrew Jones
(?)
@ 2020-03-05 17:59 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-05 17:59 UTC (permalink / raw)
To: Andrew Jones
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
peter.maydell, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Hi Drew,
On 2/7/20 1:41 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:53AM +0100, Eric Auger wrote:
>> its_enable_defaults() is the top init function that allocates the
>> command queue and all the requested tables (device, collection,
>> lpi config and pending tables), enable LPIs at distributor level
>> and ITS level.
>>
>> gicv3_enable_defaults must be called before.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - introduce its_setup_baser in this patch
>> - squash "arm/arm64: ITS: Init the command queue" in this patch.
>> ---
>> lib/arm/asm/gic-v3-its.h | 8 ++++
>> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 97 insertions(+)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index 815c515..fe73c04 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -36,6 +36,8 @@ struct its_data {
>> void *base;
>> struct its_typer typer;
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> + struct its_cmd_block *cmd_base;
>> + struct its_cmd_block *cmd_write;
>> };
>>
>> extern struct its_data its_data;
>> @@ -88,10 +90,16 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>>
>> +
>> +struct its_cmd_block {
>> + u64 raw_cmd[4];
>> +};
>> +
>> extern void its_parse_typer(void);
>> extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> +extern void its_enable_defaults(void);
>>
>> #else /* __arm__ */
>>
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index 2c0ce13..d1e7e52 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -86,3 +86,92 @@ void its_init(void)
>> its_parse_baser(i, &its_data.baser[i]);
>> }
>>
>> +static void its_setup_baser(int i, struct its_baser *baser)
>> +{
>> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
>> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>> + u64 val;
>> +
>> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
>> +
>> + val = ((u64)baser->table_addr |
>> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
>> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
>> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
>> + (u64)baser->indirect << 62 |
>> + (u64)baser->valid << 63);
>
> << GITS_BASER_INDIRECT
> << GITS_BASER_VALID
Those are BIT()
if (baser->indirect)
val |= GITS_BASER_INDIRECT;
if (baser->valid)
val |= GITS_BASER_VALID;
>
>> +
>> + switch (baser->psz) {
>> + case SZ_4K:
>> + val |= GITS_BASER_PAGE_SIZE_4K;
>> + break;
>> + case SZ_16K:
>> + val |= GITS_BASER_PAGE_SIZE_16K;
>> + break;
>> + case SZ_64K:
>> + val |= GITS_BASER_PAGE_SIZE_64K;
>> + break;
>> + }
>> +
>> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
>> +}
>> +
>> +/**
>> + * init_cmd_queue: Allocate the command queue and initialize
>> + * CBASER, CREADR, CWRITER
>> + */
>> +static void its_cmd_queue_init(void)
>> +{
>> + unsigned long n = SZ_64K >> PAGE_SHIFT;
>> + unsigned long order = fls(n);
>> + u64 cbaser;
>> +
>> + its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
>> +
>> + cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1) | GITS_CBASER_VALID);
>> +
>> + writeq(cbaser, its_data.base + GITS_CBASER);
>> +
>> + its_data.cmd_write = its_data.cmd_base;
>> + writeq(0, its_data.base + GITS_CWRITER);
>> +}
>> +
>> +void its_enable_defaults(void)
>> +{
>> + unsigned int i;
>> +
>> + its_parse_typer();
>> +
>> + /* Allocate BASER tables (device and collection tables) */
>> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
>> + struct its_baser *baser = &its_data.baser[i];
>> + int ret;
>> +
>> + ret = its_parse_baser(i, baser);
>> + if (ret)
>> + continue;
>
> Didn't we already parse typer and baser at its_init time? How/why would
> its_parse_baser fail? Should we assert when it does?
Yes we parsed them already so I can directly use
baser = &its_data.baser[i];
More generally I have simplified and only cares about the device and
collection baser now.
>
>> +
>> + switch (baser->type) {
>> + case GITS_BASER_TYPE_DEVICE:
>> + baser->valid = true;
>> + its_setup_baser(i, baser);
>> + break;
>> + case GITS_BASER_TYPE_COLLECTION:
>> + baser->valid = true;
>> + its_setup_baser(i, baser);
>> + break;
>> + default:
>> + break;
>
> assert() ?
>
>> + }
>> + }
>> +
>> + /* Allocate LPI config and pending tables */
>> + gicv3_lpi_alloc_tables();
>> +
>> + its_cmd_queue_init();
>> +
>> + for (i = 0; i < nr_cpus; i++)
>> + gicv3_lpi_rdist_ctrl(i, true);
>> +
>> + writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
>> +}
>> --
>> 2.20.1
>>
>>
>
> It's looking like we don't have a clean separation between its_init and
> its_enable_defaults. I'd expect its_init to do all the allocating of
> memory and its_enable_defaults to do all the write's to the device.
> We should only do its_init once and its_enable should be something
> we can do again (after a disable, on reset, etc.). Is that not possible
> with the ITS device?
I can moved the BASER device & collection table allocation, + the queue
init in its_init()
However gicv3_lpi_alloc_tables() must be called after
gicv3_enable_defaults as it uses the redist_base[] initialized there.
Thanks
Eric
>
> Thanks,
> drew
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-03-05 17:59 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-05 17:59 UTC (permalink / raw)
To: Andrew Jones
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 1:41 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:53AM +0100, Eric Auger wrote:
>> its_enable_defaults() is the top init function that allocates the
>> command queue and all the requested tables (device, collection,
>> lpi config and pending tables), enable LPIs at distributor level
>> and ITS level.
>>
>> gicv3_enable_defaults must be called before.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - introduce its_setup_baser in this patch
>> - squash "arm/arm64: ITS: Init the command queue" in this patch.
>> ---
>> lib/arm/asm/gic-v3-its.h | 8 ++++
>> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 97 insertions(+)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index 815c515..fe73c04 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -36,6 +36,8 @@ struct its_data {
>> void *base;
>> struct its_typer typer;
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> + struct its_cmd_block *cmd_base;
>> + struct its_cmd_block *cmd_write;
>> };
>>
>> extern struct its_data its_data;
>> @@ -88,10 +90,16 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>>
>> +
>> +struct its_cmd_block {
>> + u64 raw_cmd[4];
>> +};
>> +
>> extern void its_parse_typer(void);
>> extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> +extern void its_enable_defaults(void);
>>
>> #else /* __arm__ */
>>
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index 2c0ce13..d1e7e52 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -86,3 +86,92 @@ void its_init(void)
>> its_parse_baser(i, &its_data.baser[i]);
>> }
>>
>> +static void its_setup_baser(int i, struct its_baser *baser)
>> +{
>> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
>> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>> + u64 val;
>> +
>> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
>> +
>> + val = ((u64)baser->table_addr |
>> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
>> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
>> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
>> + (u64)baser->indirect << 62 |
>> + (u64)baser->valid << 63);
>
> << GITS_BASER_INDIRECT
> << GITS_BASER_VALID
Those are BIT()
if (baser->indirect)
val |= GITS_BASER_INDIRECT;
if (baser->valid)
val |= GITS_BASER_VALID;
>
>> +
>> + switch (baser->psz) {
>> + case SZ_4K:
>> + val |= GITS_BASER_PAGE_SIZE_4K;
>> + break;
>> + case SZ_16K:
>> + val |= GITS_BASER_PAGE_SIZE_16K;
>> + break;
>> + case SZ_64K:
>> + val |= GITS_BASER_PAGE_SIZE_64K;
>> + break;
>> + }
>> +
>> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
>> +}
>> +
>> +/**
>> + * init_cmd_queue: Allocate the command queue and initialize
>> + * CBASER, CREADR, CWRITER
>> + */
>> +static void its_cmd_queue_init(void)
>> +{
>> + unsigned long n = SZ_64K >> PAGE_SHIFT;
>> + unsigned long order = fls(n);
>> + u64 cbaser;
>> +
>> + its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
>> +
>> + cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1) | GITS_CBASER_VALID);
>> +
>> + writeq(cbaser, its_data.base + GITS_CBASER);
>> +
>> + its_data.cmd_write = its_data.cmd_base;
>> + writeq(0, its_data.base + GITS_CWRITER);
>> +}
>> +
>> +void its_enable_defaults(void)
>> +{
>> + unsigned int i;
>> +
>> + its_parse_typer();
>> +
>> + /* Allocate BASER tables (device and collection tables) */
>> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
>> + struct its_baser *baser = &its_data.baser[i];
>> + int ret;
>> +
>> + ret = its_parse_baser(i, baser);
>> + if (ret)
>> + continue;
>
> Didn't we already parse typer and baser at its_init time? How/why would
> its_parse_baser fail? Should we assert when it does?
Yes we parsed them already so I can directly use
baser = &its_data.baser[i];
More generally I have simplified and only cares about the device and
collection baser now.
>
>> +
>> + switch (baser->type) {
>> + case GITS_BASER_TYPE_DEVICE:
>> + baser->valid = true;
>> + its_setup_baser(i, baser);
>> + break;
>> + case GITS_BASER_TYPE_COLLECTION:
>> + baser->valid = true;
>> + its_setup_baser(i, baser);
>> + break;
>> + default:
>> + break;
>
> assert() ?
>
>> + }
>> + }
>> +
>> + /* Allocate LPI config and pending tables */
>> + gicv3_lpi_alloc_tables();
>> +
>> + its_cmd_queue_init();
>> +
>> + for (i = 0; i < nr_cpus; i++)
>> + gicv3_lpi_rdist_ctrl(i, true);
>> +
>> + writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
>> +}
>> --
>> 2.20.1
>>
>>
>
> It's looking like we don't have a clean separation between its_init and
> its_enable_defaults. I'd expect its_init to do all the allocating of
> memory and its_enable_defaults to do all the write's to the device.
> We should only do its_init once and its_enable should be something
> we can do again (after a disable, on reset, etc.). Is that not possible
> with the ITS device?
I can moved the BASER device & collection table allocation, + the queue
init in its_init()
However gicv3_lpi_alloc_tables() must be called after
gicv3_enable_defaults as it uses the redist_base[] initialized there.
Thanks
Eric
>
> Thanks,
> drew
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults
@ 2020-03-05 17:59 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-05 17:59 UTC (permalink / raw)
To: Andrew Jones
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 1:41 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:53AM +0100, Eric Auger wrote:
>> its_enable_defaults() is the top init function that allocates the
>> command queue and all the requested tables (device, collection,
>> lpi config and pending tables), enable LPIs at distributor level
>> and ITS level.
>>
>> gicv3_enable_defaults must be called before.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - introduce its_setup_baser in this patch
>> - squash "arm/arm64: ITS: Init the command queue" in this patch.
>> ---
>> lib/arm/asm/gic-v3-its.h | 8 ++++
>> lib/arm/gic-v3-its.c | 89 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 97 insertions(+)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index 815c515..fe73c04 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -36,6 +36,8 @@ struct its_data {
>> void *base;
>> struct its_typer typer;
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> + struct its_cmd_block *cmd_base;
>> + struct its_cmd_block *cmd_write;
>> };
>>
>> extern struct its_data its_data;
>> @@ -88,10 +90,16 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>>
>> +
>> +struct its_cmd_block {
>> + u64 raw_cmd[4];
>> +};
>> +
>> extern void its_parse_typer(void);
>> extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> +extern void its_enable_defaults(void);
>>
>> #else /* __arm__ */
>>
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index 2c0ce13..d1e7e52 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -86,3 +86,92 @@ void its_init(void)
>> its_parse_baser(i, &its_data.baser[i]);
>> }
>>
>> +static void its_setup_baser(int i, struct its_baser *baser)
>> +{
>> + unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
>> + unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>> + u64 val;
>> +
>> + baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
>> +
>> + val = ((u64)baser->table_addr |
>> + ((u64)baser->type << GITS_BASER_TYPE_SHIFT) |
>> + ((u64)(baser->esz - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
>> + ((baser->nr_pages - 1) << GITS_BASER_PAGES_SHIFT) |
>> + (u64)baser->indirect << 62 |
>> + (u64)baser->valid << 63);
>
> << GITS_BASER_INDIRECT
> << GITS_BASER_VALID
Those are BIT()
if (baser->indirect)
val |= GITS_BASER_INDIRECT;
if (baser->valid)
val |= GITS_BASER_VALID;
>
>> +
>> + switch (baser->psz) {
>> + case SZ_4K:
>> + val |= GITS_BASER_PAGE_SIZE_4K;
>> + break;
>> + case SZ_16K:
>> + val |= GITS_BASER_PAGE_SIZE_16K;
>> + break;
>> + case SZ_64K:
>> + val |= GITS_BASER_PAGE_SIZE_64K;
>> + break;
>> + }
>> +
>> + writeq(val, gicv3_its_base() + GITS_BASER + i * 8);
>> +}
>> +
>> +/**
>> + * init_cmd_queue: Allocate the command queue and initialize
>> + * CBASER, CREADR, CWRITER
>> + */
>> +static void its_cmd_queue_init(void)
>> +{
>> + unsigned long n = SZ_64K >> PAGE_SHIFT;
>> + unsigned long order = fls(n);
>> + u64 cbaser;
>> +
>> + its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
>> +
>> + cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1) | GITS_CBASER_VALID);
>> +
>> + writeq(cbaser, its_data.base + GITS_CBASER);
>> +
>> + its_data.cmd_write = its_data.cmd_base;
>> + writeq(0, its_data.base + GITS_CWRITER);
>> +}
>> +
>> +void its_enable_defaults(void)
>> +{
>> + unsigned int i;
>> +
>> + its_parse_typer();
>> +
>> + /* Allocate BASER tables (device and collection tables) */
>> + for (i = 0; i < GITS_BASER_NR_REGS; i++) {
>> + struct its_baser *baser = &its_data.baser[i];
>> + int ret;
>> +
>> + ret = its_parse_baser(i, baser);
>> + if (ret)
>> + continue;
>
> Didn't we already parse typer and baser at its_init time? How/why would
> its_parse_baser fail? Should we assert when it does?
Yes we parsed them already so I can directly use
baser = &its_data.baser[i];
More generally I have simplified and only cares about the device and
collection baser now.
>
>> +
>> + switch (baser->type) {
>> + case GITS_BASER_TYPE_DEVICE:
>> + baser->valid = true;
>> + its_setup_baser(i, baser);
>> + break;
>> + case GITS_BASER_TYPE_COLLECTION:
>> + baser->valid = true;
>> + its_setup_baser(i, baser);
>> + break;
>> + default:
>> + break;
>
> assert() ?
>
>> + }
>> + }
>> +
>> + /* Allocate LPI config and pending tables */
>> + gicv3_lpi_alloc_tables();
>> +
>> + its_cmd_queue_init();
>> +
>> + for (i = 0; i < nr_cpus; i++)
>> + gicv3_lpi_rdist_ctrl(i, true);
>> +
>> + writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
>> +}
>> --
>> 2.20.1
>>
>>
>
> It's looking like we don't have a clean separation between its_init and
> its_enable_defaults. I'd expect its_init to do all the allocating of
> memory and its_enable_defaults to do all the write's to the device.
> We should only do its_init once and its_enable should be something
> we can do again (after a disable, on reset, etc.). Is that not possible
> with the ITS device?
I can moved the BASER device & collection table allocation, + the queue
init in its_init()
However gicv3_lpi_alloc_tables() must be called after
gicv3_enable_defaults as it uses the redist_base[] initialized there.
Thanks
Eric
>
> Thanks,
> drew
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
2020-02-07 2:12 ` Zenghui Yu
(?)
@ 2020-03-05 19:40 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-05 19:40 UTC (permalink / raw)
To: Zenghui Yu, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, alexandru.elisei, thuth
Hi Zenghui,
On 2/7/20 3:12 AM, Zenghui Yu wrote:
> Hi Eric,
>
> On 2020/1/28 18:34, Eric Auger wrote:
>> Allocate the LPI configuration and per re-distributor pending table.
>> Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
>> by default in the config table.
>>
>> Also introduce a helper routine that allows to set the pending table
>> bit for a given LPI.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
>> and add _lpi prefix too
>>
>> v1 -> v2:
>> - remove memory attributes
>> ---
>> lib/arm/asm/gic-v3.h | 16 +++++++++++
>> lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 80 insertions(+)
>>
>> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
>> index ffb2e26..ec2a6f0 100644
>> --- a/lib/arm/asm/gic-v3.h
>> +++ b/lib/arm/asm/gic-v3.h
>> @@ -48,6 +48,16 @@
>> #define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
>> (MPIDR_AFFINITY_LEVEL(cluster_id, level) <<
>> ICC_SGI1R_AFFINITY_## level ## _SHIFT)
>> +#define GICR_PROPBASER_IDBITS_MASK (0x1f)
>
> This is not being used. You can use it when calculating prop_val
> or just drop it.
yep dropped it.
>
>> +
>> +#define GICR_PENDBASER_PTZ BIT_ULL(62)
>> +
>> +#define LPI_PROP_GROUP1 (1 << 1)
>> +#define LPI_PROP_ENABLED (1 << 0)
>> +#define LPI_PROP_DEFAULT_PRIO 0xa0
>> +#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
>> + LPI_PROP_ENABLED)
>> +
>> #include <asm/arch_gicv3.h>
>> #ifndef __ASSEMBLY__
>> @@ -64,6 +74,8 @@ struct gicv3_data {
>> void *dist_base;
>> void *redist_bases[GICV3_NR_REDISTS];
>> void *redist_base[NR_CPUS];
>> + void *lpi_prop;
>> + void *lpi_pend[NR_CPUS];
>> unsigned int irq_nr;
>> };
>> extern struct gicv3_data gicv3_data;
>> @@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
>> extern void gicv3_ipi_send_single(int irq, int cpu);
>> extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
>> extern void gicv3_set_redist_base(size_t stride);
>> +extern void gicv3_lpi_set_config(int n, u8 val);
>> +extern u8 gicv3_lpi_get_config(int n);
>> +extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
>> +extern void gicv3_lpi_alloc_tables(void);
>> static inline void gicv3_do_wait_for_rwp(void *base)
>> {
>> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
>> index feecb5e..c33f883 100644
>> --- a/lib/arm/gic-v3.c
>> +++ b/lib/arm/gic-v3.c
>> @@ -5,6 +5,7 @@
>> */
>> #include <asm/gic.h>
>> #include <asm/io.h>
>> +#include <alloc_page.h>
>> void gicv3_set_redist_base(size_t stride)
>> {
>> @@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
>> cpumask_set_cpu(cpu, &dest);
>> gicv3_ipi_send_mask(irq, &dest);
>> }
>> +
>> +#if defined(__aarch64__)
>> +/* alloc_lpi_tables: Allocate LPI config and pending tables */
>> +void gicv3_lpi_alloc_tables(void)
>> +{
>> + unsigned long n = SZ_64K >> PAGE_SHIFT;
>> + unsigned long order = fls(n);
>> + u64 prop_val;
>> + int cpu;
>> +
>> + gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
>> +
>> + /* ID bits = 13, ie. up to 14b LPI INTID */
>> + prop_val = (u64)gicv3_data.lpi_prop | 13;
>> +
>> + /*
>> + * Allocate pending tables for each redistributor
>> + * and set PROPBASER and PENDBASER
>> + */
>> + for_each_present_cpu(cpu) {
>> + u64 pend_val;
>> + void *ptr;
>> +
>> + ptr = gicv3_data.redist_base[cpu];
>> +
>> + writeq(prop_val, ptr + GICR_PROPBASER);
>> +
>> + gicv3_data.lpi_pend[cpu] = (void
>> *)virt_to_phys(alloc_pages(order));
>> +
>> + pend_val = (u64)gicv3_data.lpi_pend[cpu];
>> +
>> + writeq(pend_val, ptr + GICR_PENDBASER);
>> + }
>> +}
>> +
>> +void gicv3_lpi_set_config(int n, u8 value)
>> +{
>> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
>
> But this is actually the *physical* address, shouldn't it be
> converted by phys_to_virt() before reading/writing something?
> Like what you've done for the 'lpi_pend[rdist]' before writing
> pending bit. Or I'm missing some points here?
Agreed! Thanks
Eric
>
>> +
>> + *entry = value;
>> +}
>> +
>> +u8 gicv3_lpi_get_config(int n)
>> +{
>> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
>
> The same as above.
>
>
> Thanks,
> Zenghui
>
>> +
>> + return *entry;
>> +}
>> +
>> +void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
>> +{
>> + u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
>> + u8 mask = 1 << (n % 8), byte;
>> +
>> + ptr += (n / 8);
>> + byte = *ptr;
>> + if (set)
>> + byte |= mask;
>> + else
>> + byte &= ~mask;
>> + *ptr = byte;
>> +}
>> +#endif /* __aarch64__ */
>>
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
@ 2020-03-05 19:40 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-05 19:40 UTC (permalink / raw)
To: Zenghui Yu, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, drjones, alexandru.elisei, thuth, peter.maydell
Hi Zenghui,
On 2/7/20 3:12 AM, Zenghui Yu wrote:
> Hi Eric,
>
> On 2020/1/28 18:34, Eric Auger wrote:
>> Allocate the LPI configuration and per re-distributor pending table.
>> Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
>> by default in the config table.
>>
>> Also introduce a helper routine that allows to set the pending table
>> bit for a given LPI.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
>> and add _lpi prefix too
>>
>> v1 -> v2:
>> - remove memory attributes
>> ---
>> lib/arm/asm/gic-v3.h | 16 +++++++++++
>> lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 80 insertions(+)
>>
>> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
>> index ffb2e26..ec2a6f0 100644
>> --- a/lib/arm/asm/gic-v3.h
>> +++ b/lib/arm/asm/gic-v3.h
>> @@ -48,6 +48,16 @@
>> #define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
>> (MPIDR_AFFINITY_LEVEL(cluster_id, level) <<
>> ICC_SGI1R_AFFINITY_## level ## _SHIFT)
>> +#define GICR_PROPBASER_IDBITS_MASK (0x1f)
>
> This is not being used. You can use it when calculating prop_val
> or just drop it.
yep dropped it.
>
>> +
>> +#define GICR_PENDBASER_PTZ BIT_ULL(62)
>> +
>> +#define LPI_PROP_GROUP1 (1 << 1)
>> +#define LPI_PROP_ENABLED (1 << 0)
>> +#define LPI_PROP_DEFAULT_PRIO 0xa0
>> +#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
>> + LPI_PROP_ENABLED)
>> +
>> #include <asm/arch_gicv3.h>
>> #ifndef __ASSEMBLY__
>> @@ -64,6 +74,8 @@ struct gicv3_data {
>> void *dist_base;
>> void *redist_bases[GICV3_NR_REDISTS];
>> void *redist_base[NR_CPUS];
>> + void *lpi_prop;
>> + void *lpi_pend[NR_CPUS];
>> unsigned int irq_nr;
>> };
>> extern struct gicv3_data gicv3_data;
>> @@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
>> extern void gicv3_ipi_send_single(int irq, int cpu);
>> extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
>> extern void gicv3_set_redist_base(size_t stride);
>> +extern void gicv3_lpi_set_config(int n, u8 val);
>> +extern u8 gicv3_lpi_get_config(int n);
>> +extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
>> +extern void gicv3_lpi_alloc_tables(void);
>> static inline void gicv3_do_wait_for_rwp(void *base)
>> {
>> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
>> index feecb5e..c33f883 100644
>> --- a/lib/arm/gic-v3.c
>> +++ b/lib/arm/gic-v3.c
>> @@ -5,6 +5,7 @@
>> */
>> #include <asm/gic.h>
>> #include <asm/io.h>
>> +#include <alloc_page.h>
>> void gicv3_set_redist_base(size_t stride)
>> {
>> @@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
>> cpumask_set_cpu(cpu, &dest);
>> gicv3_ipi_send_mask(irq, &dest);
>> }
>> +
>> +#if defined(__aarch64__)
>> +/* alloc_lpi_tables: Allocate LPI config and pending tables */
>> +void gicv3_lpi_alloc_tables(void)
>> +{
>> + unsigned long n = SZ_64K >> PAGE_SHIFT;
>> + unsigned long order = fls(n);
>> + u64 prop_val;
>> + int cpu;
>> +
>> + gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
>> +
>> + /* ID bits = 13, ie. up to 14b LPI INTID */
>> + prop_val = (u64)gicv3_data.lpi_prop | 13;
>> +
>> + /*
>> + * Allocate pending tables for each redistributor
>> + * and set PROPBASER and PENDBASER
>> + */
>> + for_each_present_cpu(cpu) {
>> + u64 pend_val;
>> + void *ptr;
>> +
>> + ptr = gicv3_data.redist_base[cpu];
>> +
>> + writeq(prop_val, ptr + GICR_PROPBASER);
>> +
>> + gicv3_data.lpi_pend[cpu] = (void
>> *)virt_to_phys(alloc_pages(order));
>> +
>> + pend_val = (u64)gicv3_data.lpi_pend[cpu];
>> +
>> + writeq(pend_val, ptr + GICR_PENDBASER);
>> + }
>> +}
>> +
>> +void gicv3_lpi_set_config(int n, u8 value)
>> +{
>> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
>
> But this is actually the *physical* address, shouldn't it be
> converted by phys_to_virt() before reading/writing something?
> Like what you've done for the 'lpi_pend[rdist]' before writing
> pending bit. Or I'm missing some points here?
Agreed! Thanks
Eric
>
>> +
>> + *entry = value;
>> +}
>> +
>> +u8 gicv3_lpi_get_config(int n)
>> +{
>> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
>
> The same as above.
>
>
> Thanks,
> Zenghui
>
>> +
>> + return *entry;
>> +}
>> +
>> +void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
>> +{
>> + u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
>> + u8 mask = 1 << (n % 8), byte;
>> +
>> + ptr += (n / 8);
>> + byte = *ptr;
>> + if (set)
>> + byte |= mask;
>> + else
>> + byte &= ~mask;
>> + *ptr = byte;
>> +}
>> +#endif /* __aarch64__ */
>>
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables
@ 2020-03-05 19:40 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-05 19:40 UTC (permalink / raw)
To: Zenghui Yu, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Hi Zenghui,
On 2/7/20 3:12 AM, Zenghui Yu wrote:
> Hi Eric,
>
> On 2020/1/28 18:34, Eric Auger wrote:
>> Allocate the LPI configuration and per re-distributor pending table.
>> Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
>> by default in the config table.
>>
>> Also introduce a helper routine that allows to set the pending table
>> bit for a given LPI.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
>> and add _lpi prefix too
>>
>> v1 -> v2:
>> - remove memory attributes
>> ---
>> lib/arm/asm/gic-v3.h | 16 +++++++++++
>> lib/arm/gic-v3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 80 insertions(+)
>>
>> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
>> index ffb2e26..ec2a6f0 100644
>> --- a/lib/arm/asm/gic-v3.h
>> +++ b/lib/arm/asm/gic-v3.h
>> @@ -48,6 +48,16 @@
>> #define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
>> (MPIDR_AFFINITY_LEVEL(cluster_id, level) <<
>> ICC_SGI1R_AFFINITY_## level ## _SHIFT)
>> +#define GICR_PROPBASER_IDBITS_MASK (0x1f)
>
> This is not being used. You can use it when calculating prop_val
> or just drop it.
yep dropped it.
>
>> +
>> +#define GICR_PENDBASER_PTZ BIT_ULL(62)
>> +
>> +#define LPI_PROP_GROUP1 (1 << 1)
>> +#define LPI_PROP_ENABLED (1 << 0)
>> +#define LPI_PROP_DEFAULT_PRIO 0xa0
>> +#define LPI_PROP_DEFAULT (LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | \
>> + LPI_PROP_ENABLED)
>> +
>> #include <asm/arch_gicv3.h>
>> #ifndef __ASSEMBLY__
>> @@ -64,6 +74,8 @@ struct gicv3_data {
>> void *dist_base;
>> void *redist_bases[GICV3_NR_REDISTS];
>> void *redist_base[NR_CPUS];
>> + void *lpi_prop;
>> + void *lpi_pend[NR_CPUS];
>> unsigned int irq_nr;
>> };
>> extern struct gicv3_data gicv3_data;
>> @@ -80,6 +92,10 @@ extern void gicv3_write_eoir(u32 irqstat);
>> extern void gicv3_ipi_send_single(int irq, int cpu);
>> extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
>> extern void gicv3_set_redist_base(size_t stride);
>> +extern void gicv3_lpi_set_config(int n, u8 val);
>> +extern u8 gicv3_lpi_get_config(int n);
>> +extern void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set);
>> +extern void gicv3_lpi_alloc_tables(void);
>> static inline void gicv3_do_wait_for_rwp(void *base)
>> {
>> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
>> index feecb5e..c33f883 100644
>> --- a/lib/arm/gic-v3.c
>> +++ b/lib/arm/gic-v3.c
>> @@ -5,6 +5,7 @@
>> */
>> #include <asm/gic.h>
>> #include <asm/io.h>
>> +#include <alloc_page.h>
>> void gicv3_set_redist_base(size_t stride)
>> {
>> @@ -147,3 +148,66 @@ void gicv3_ipi_send_single(int irq, int cpu)
>> cpumask_set_cpu(cpu, &dest);
>> gicv3_ipi_send_mask(irq, &dest);
>> }
>> +
>> +#if defined(__aarch64__)
>> +/* alloc_lpi_tables: Allocate LPI config and pending tables */
>> +void gicv3_lpi_alloc_tables(void)
>> +{
>> + unsigned long n = SZ_64K >> PAGE_SHIFT;
>> + unsigned long order = fls(n);
>> + u64 prop_val;
>> + int cpu;
>> +
>> + gicv3_data.lpi_prop = (void *)virt_to_phys(alloc_pages(order));
>> +
>> + /* ID bits = 13, ie. up to 14b LPI INTID */
>> + prop_val = (u64)gicv3_data.lpi_prop | 13;
>> +
>> + /*
>> + * Allocate pending tables for each redistributor
>> + * and set PROPBASER and PENDBASER
>> + */
>> + for_each_present_cpu(cpu) {
>> + u64 pend_val;
>> + void *ptr;
>> +
>> + ptr = gicv3_data.redist_base[cpu];
>> +
>> + writeq(prop_val, ptr + GICR_PROPBASER);
>> +
>> + gicv3_data.lpi_pend[cpu] = (void
>> *)virt_to_phys(alloc_pages(order));
>> +
>> + pend_val = (u64)gicv3_data.lpi_pend[cpu];
>> +
>> + writeq(pend_val, ptr + GICR_PENDBASER);
>> + }
>> +}
>> +
>> +void gicv3_lpi_set_config(int n, u8 value)
>> +{
>> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
>
> But this is actually the *physical* address, shouldn't it be
> converted by phys_to_virt() before reading/writing something?
> Like what you've done for the 'lpi_pend[rdist]' before writing
> pending bit. Or I'm missing some points here?
Agreed! Thanks
Eric
>
>> +
>> + *entry = value;
>> +}
>> +
>> +u8 gicv3_lpi_get_config(int n)
>> +{
>> + u8 *entry = (u8 *)(gicv3_data.lpi_prop + (n - 8192));
>
> The same as above.
>
>
> Thanks,
> Zenghui
>
>> +
>> + return *entry;
>> +}
>> +
>> +void gicv3_lpi_set_pending_table_bit(int rdist, int n, bool set)
>> +{
>> + u8 *ptr = phys_to_virt((phys_addr_t)gicv3_data.lpi_pend[rdist]);
>> + u8 mask = 1 << (n % 8), byte;
>> +
>> + ptr += (n / 8);
>> + byte = *ptr;
>> + if (set)
>> + byte |= mask;
>> + else
>> + byte &= ~mask;
>> + *ptr = byte;
>> +}
>> +#endif /* __aarch64__ */
>>
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
2020-02-07 5:41 ` Zenghui Yu
(?)
@ 2020-03-05 19:42 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-05 19:42 UTC (permalink / raw)
To: Zenghui Yu, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: drjones, andre.przywara, peter.maydell, alexandru.elisei, thuth
Hi Zenghui,
On 2/7/20 6:41 AM, Zenghui Yu wrote:
> Hi Eric,
>
> On 2020/1/28 18:34, Eric Auger wrote:
>> Introduce an helper functions to register
>> - a new device, characterized by its device id and the
>> max number of event IDs that dimension its ITT (Interrupt
>> Translation Table). The function allocates the ITT.
>>
>> - a new collection, characterized by its ID and the
>> target processing engine (PE).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - s/report_abort/assert
>>
>> v1 -> v2:
>> - s/nb_/nr_
>> ---
>> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
>> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 63 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index fe73c04..acd97a9 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -31,6 +31,19 @@ struct its_baser {
>> };
>> #define GITS_BASER_NR_REGS 8
>> +#define GITS_MAX_DEVICES 8
>> +#define GITS_MAX_COLLECTIONS 8
>> +
>> +struct its_device {
>> + u32 device_id; /* device ID */
>> + u32 nr_ites; /* Max Interrupt Translation Entries */
>> + void *itt; /* Interrupt Translation Table GPA */
>> +};
>> +
>> +struct its_collection {
>> + u64 target_address;
>> + u16 col_id;
>> +};
>> struct its_data {
>> void *base;
>> @@ -38,6 +51,10 @@ struct its_data {
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> struct its_cmd_block *cmd_base;
>> struct its_cmd_block *cmd_write;
>> + struct its_device devices[GITS_MAX_DEVICES];
>> + u32 nr_devices; /* Allocated Devices */
>> + struct its_collection collections[GITS_MAX_COLLECTIONS];
>> + u32 nr_collections; /* Allocated Collections */
>> };
>> extern struct its_data its_data;
>> @@ -90,7 +107,6 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>> -
>> struct its_cmd_block {
>> u64 raw_cmd[4];
>> };
>> @@ -100,6 +116,8 @@ extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> extern void its_enable_defaults(void);
>> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
>> +extern struct its_collection *its_create_collection(u32 col_id, u32
>> target_pe);
>> #else /* __arm__ */
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index d1e7e52..c2dcd01 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
>> }
>> +
>> +struct its_device *its_create_device(u32 device_id, int nr_ites)
>> +{
>> + struct its_baser *baser;
>> + struct its_device *new;
>> + unsigned long n, order;
>> +
>> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
>> +
>
>
>> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
>> + if (!baser)
>> + return NULL;
>
> I think there's no need to lookup the device baser here. As the
> device baser should have already been setup at initialization
> time (i.e. in its_enable_defaults). And anyway, 'baser' is not
> being used in this function.
I fully agree. I Removed that.
Thanks!
Eric
>
>
> Thanks,
> Zenghui
>
>> +
>> + new = &its_data.devices[its_data.nr_devices];
>> +
>> + new->device_id = device_id;
>> + new->nr_ites = nr_ites;
>> +
>> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
>> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>> + new->itt = (void *)virt_to_phys(alloc_pages(order));
>> +
>> + its_data.nr_devices++;
>> + return new;
>> +}
>> +
>> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
>> +{
>> + struct its_collection *new;
>> +
>> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
>> +
>> + new = &its_data.collections[its_data.nr_collections];
>> +
>> + new->col_id = col_id;
>> +
>> + if (its_data.typer.pta)
>> + new->target_address = (u64)gicv3_data.redist_base[pe];
>> + else
>> + new->target_address = pe << 16;
>> +
>> + its_data.nr_collections++;
>> + return new;
>> +}
>>
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
@ 2020-03-05 19:42 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-05 19:42 UTC (permalink / raw)
To: Zenghui Yu, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, drjones, alexandru.elisei, thuth, peter.maydell
Hi Zenghui,
On 2/7/20 6:41 AM, Zenghui Yu wrote:
> Hi Eric,
>
> On 2020/1/28 18:34, Eric Auger wrote:
>> Introduce an helper functions to register
>> - a new device, characterized by its device id and the
>> max number of event IDs that dimension its ITT (Interrupt
>> Translation Table). The function allocates the ITT.
>>
>> - a new collection, characterized by its ID and the
>> target processing engine (PE).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - s/report_abort/assert
>>
>> v1 -> v2:
>> - s/nb_/nr_
>> ---
>> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
>> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 63 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index fe73c04..acd97a9 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -31,6 +31,19 @@ struct its_baser {
>> };
>> #define GITS_BASER_NR_REGS 8
>> +#define GITS_MAX_DEVICES 8
>> +#define GITS_MAX_COLLECTIONS 8
>> +
>> +struct its_device {
>> + u32 device_id; /* device ID */
>> + u32 nr_ites; /* Max Interrupt Translation Entries */
>> + void *itt; /* Interrupt Translation Table GPA */
>> +};
>> +
>> +struct its_collection {
>> + u64 target_address;
>> + u16 col_id;
>> +};
>> struct its_data {
>> void *base;
>> @@ -38,6 +51,10 @@ struct its_data {
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> struct its_cmd_block *cmd_base;
>> struct its_cmd_block *cmd_write;
>> + struct its_device devices[GITS_MAX_DEVICES];
>> + u32 nr_devices; /* Allocated Devices */
>> + struct its_collection collections[GITS_MAX_COLLECTIONS];
>> + u32 nr_collections; /* Allocated Collections */
>> };
>> extern struct its_data its_data;
>> @@ -90,7 +107,6 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>> -
>> struct its_cmd_block {
>> u64 raw_cmd[4];
>> };
>> @@ -100,6 +116,8 @@ extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> extern void its_enable_defaults(void);
>> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
>> +extern struct its_collection *its_create_collection(u32 col_id, u32
>> target_pe);
>> #else /* __arm__ */
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index d1e7e52..c2dcd01 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
>> }
>> +
>> +struct its_device *its_create_device(u32 device_id, int nr_ites)
>> +{
>> + struct its_baser *baser;
>> + struct its_device *new;
>> + unsigned long n, order;
>> +
>> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
>> +
>
>
>> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
>> + if (!baser)
>> + return NULL;
>
> I think there's no need to lookup the device baser here. As the
> device baser should have already been setup at initialization
> time (i.e. in its_enable_defaults). And anyway, 'baser' is not
> being used in this function.
I fully agree. I Removed that.
Thanks!
Eric
>
>
> Thanks,
> Zenghui
>
>> +
>> + new = &its_data.devices[its_data.nr_devices];
>> +
>> + new->device_id = device_id;
>> + new->nr_ites = nr_ites;
>> +
>> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
>> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>> + new->itt = (void *)virt_to_phys(alloc_pages(order));
>> +
>> + its_data.nr_devices++;
>> + return new;
>> +}
>> +
>> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
>> +{
>> + struct its_collection *new;
>> +
>> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
>> +
>> + new = &its_data.collections[its_data.nr_collections];
>> +
>> + new->col_id = col_id;
>> +
>> + if (its_data.typer.pta)
>> + new->target_address = (u64)gicv3_data.redist_base[pe];
>> + else
>> + new->target_address = pe << 16;
>> +
>> + its_data.nr_collections++;
>> + return new;
>> +}
>>
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
@ 2020-03-05 19:42 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-05 19:42 UTC (permalink / raw)
To: Zenghui Yu, eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm
Cc: andre.przywara, thuth
Hi Zenghui,
On 2/7/20 6:41 AM, Zenghui Yu wrote:
> Hi Eric,
>
> On 2020/1/28 18:34, Eric Auger wrote:
>> Introduce an helper functions to register
>> - a new device, characterized by its device id and the
>> max number of event IDs that dimension its ITT (Interrupt
>> Translation Table). The function allocates the ITT.
>>
>> - a new collection, characterized by its ID and the
>> target processing engine (PE).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - s/report_abort/assert
>>
>> v1 -> v2:
>> - s/nb_/nr_
>> ---
>> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
>> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 63 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index fe73c04..acd97a9 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -31,6 +31,19 @@ struct its_baser {
>> };
>> #define GITS_BASER_NR_REGS 8
>> +#define GITS_MAX_DEVICES 8
>> +#define GITS_MAX_COLLECTIONS 8
>> +
>> +struct its_device {
>> + u32 device_id; /* device ID */
>> + u32 nr_ites; /* Max Interrupt Translation Entries */
>> + void *itt; /* Interrupt Translation Table GPA */
>> +};
>> +
>> +struct its_collection {
>> + u64 target_address;
>> + u16 col_id;
>> +};
>> struct its_data {
>> void *base;
>> @@ -38,6 +51,10 @@ struct its_data {
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> struct its_cmd_block *cmd_base;
>> struct its_cmd_block *cmd_write;
>> + struct its_device devices[GITS_MAX_DEVICES];
>> + u32 nr_devices; /* Allocated Devices */
>> + struct its_collection collections[GITS_MAX_COLLECTIONS];
>> + u32 nr_collections; /* Allocated Collections */
>> };
>> extern struct its_data its_data;
>> @@ -90,7 +107,6 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>> -
>> struct its_cmd_block {
>> u64 raw_cmd[4];
>> };
>> @@ -100,6 +116,8 @@ extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> extern void its_enable_defaults(void);
>> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
>> +extern struct its_collection *its_create_collection(u32 col_id, u32
>> target_pe);
>> #else /* __arm__ */
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index d1e7e52..c2dcd01 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
>> }
>> +
>> +struct its_device *its_create_device(u32 device_id, int nr_ites)
>> +{
>> + struct its_baser *baser;
>> + struct its_device *new;
>> + unsigned long n, order;
>> +
>> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
>> +
>
>
>> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
>> + if (!baser)
>> + return NULL;
>
> I think there's no need to lookup the device baser here. As the
> device baser should have already been setup at initialization
> time (i.e. in its_enable_defaults). And anyway, 'baser' is not
> being used in this function.
I fully agree. I Removed that.
Thanks!
Eric
>
>
> Thanks,
> Zenghui
>
>> +
>> + new = &its_data.devices[its_data.nr_devices];
>> +
>> + new->device_id = device_id;
>> + new->nr_ites = nr_ites;
>> +
>> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
>> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>> + new->itt = (void *)virt_to_phys(alloc_pages(order));
>> +
>> + its_data.nr_devices++;
>> + return new;
>> +}
>> +
>> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
>> +{
>> + struct its_collection *new;
>> +
>> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
>> +
>> + new = &its_data.collections[its_data.nr_collections];
>> +
>> + new->col_id = col_id;
>> +
>> + if (its_data.typer.pta)
>> + new->target_address = (u64)gicv3_data.redist_base[pe];
>> + else
>> + new->target_address = pe << 16;
>> +
>> + its_data.nr_collections++;
>> + return new;
>> +}
>>
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
2020-02-07 12:51 ` Andrew Jones
(?)
@ 2020-03-06 8:47 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 8:47 UTC (permalink / raw)
To: Andrew Jones
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
andre.przywara, peter.maydell, yuzenghui, alexandru.elisei,
thuth
Hi Drew,
On 2/7/20 1:51 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:54AM +0100, Eric Auger wrote:
>> Introduce an helper functions to register
>> - a new device, characterized by its device id and the
>> max number of event IDs that dimension its ITT (Interrupt
>> Translation Table). The function allocates the ITT.
>>
>> - a new collection, characterized by its ID and the
>> target processing engine (PE).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - s/report_abort/assert
>>
>> v1 -> v2:
>> - s/nb_/nr_
>> ---
>> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
>> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 63 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index fe73c04..acd97a9 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -31,6 +31,19 @@ struct its_baser {
>> };
>>
>> #define GITS_BASER_NR_REGS 8
>> +#define GITS_MAX_DEVICES 8
>> +#define GITS_MAX_COLLECTIONS 8
>> +
>> +struct its_device {
>> + u32 device_id; /* device ID */
>> + u32 nr_ites; /* Max Interrupt Translation Entries */
>> + void *itt; /* Interrupt Translation Table GPA */
>> +};
>> +
>> +struct its_collection {
>> + u64 target_address;
>> + u16 col_id;
>> +};
>>
>> struct its_data {
>> void *base;
>> @@ -38,6 +51,10 @@ struct its_data {
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> struct its_cmd_block *cmd_base;
>> struct its_cmd_block *cmd_write;
>> + struct its_device devices[GITS_MAX_DEVICES];
>> + u32 nr_devices; /* Allocated Devices */
>> + struct its_collection collections[GITS_MAX_COLLECTIONS];
>> + u32 nr_collections; /* Allocated Collections */
>> };
>>
>> extern struct its_data its_data;
>> @@ -90,7 +107,6 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>>
>> -
>> struct its_cmd_block {
>> u64 raw_cmd[4];
>> };
>> @@ -100,6 +116,8 @@ extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> extern void its_enable_defaults(void);
>> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
>> +extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>>
>> #else /* __arm__ */
>>
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index d1e7e52..c2dcd01 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>>
>> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
>> }
>> +
>> +struct its_device *its_create_device(u32 device_id, int nr_ites)
>> +{
>> + struct its_baser *baser;
>> + struct its_device *new;
>> + unsigned long n, order;
>> +
>> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
>> +
>> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
>> + if (!baser)
>> + return NULL;
>
> Should we assert here if we can't find a GITS_BASER_TYPE_DEVICE ?
> It seems none of the callers of its_create_device are checking
> for null.
I removed that code which is useless.
>
>> +
>> + new = &its_data.devices[its_data.nr_devices];
>> +
>> + new->device_id = device_id;
>> + new->nr_ites = nr_ites;
>> +
>> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
>> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>
> I've seen this calculation several times now, so I think an
> arch-neutral order calculator is in order:
>
> int get_order(size_t size);
>
>> + new->itt = (void *)virt_to_phys(alloc_pages(order));
I added this get_order helper in lib/alloc_page.h/c
>
> If this is a physical address then shouldn't itt be phys_addr_t ?
>
>> +
>> + its_data.nr_devices++;
>> + return new;
>> +}
ITT now is a GVA (void *).
>> +
>> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
>> +{
>> + struct its_collection *new;
>> +
>> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
>> +
>> + new = &its_data.collections[its_data.nr_collections];
>> +
>> + new->col_id = col_id;
>> +
>> + if (its_data.typer.pta)
>> + new->target_address = (u64)gicv3_data.redist_base[pe];
>> + else
>> + new->target_address = pe << 16;
>> +
>> + its_data.nr_collections++;
>> + return new;
>> +}
>> --
>> 2.20.1
>>
>
> Thanks,
> drew
>
Thanks
Eric
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
@ 2020-03-06 8:47 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 8:47 UTC (permalink / raw)
To: Andrew Jones
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 1:51 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:54AM +0100, Eric Auger wrote:
>> Introduce an helper functions to register
>> - a new device, characterized by its device id and the
>> max number of event IDs that dimension its ITT (Interrupt
>> Translation Table). The function allocates the ITT.
>>
>> - a new collection, characterized by its ID and the
>> target processing engine (PE).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - s/report_abort/assert
>>
>> v1 -> v2:
>> - s/nb_/nr_
>> ---
>> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
>> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 63 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index fe73c04..acd97a9 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -31,6 +31,19 @@ struct its_baser {
>> };
>>
>> #define GITS_BASER_NR_REGS 8
>> +#define GITS_MAX_DEVICES 8
>> +#define GITS_MAX_COLLECTIONS 8
>> +
>> +struct its_device {
>> + u32 device_id; /* device ID */
>> + u32 nr_ites; /* Max Interrupt Translation Entries */
>> + void *itt; /* Interrupt Translation Table GPA */
>> +};
>> +
>> +struct its_collection {
>> + u64 target_address;
>> + u16 col_id;
>> +};
>>
>> struct its_data {
>> void *base;
>> @@ -38,6 +51,10 @@ struct its_data {
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> struct its_cmd_block *cmd_base;
>> struct its_cmd_block *cmd_write;
>> + struct its_device devices[GITS_MAX_DEVICES];
>> + u32 nr_devices; /* Allocated Devices */
>> + struct its_collection collections[GITS_MAX_COLLECTIONS];
>> + u32 nr_collections; /* Allocated Collections */
>> };
>>
>> extern struct its_data its_data;
>> @@ -90,7 +107,6 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>>
>> -
>> struct its_cmd_block {
>> u64 raw_cmd[4];
>> };
>> @@ -100,6 +116,8 @@ extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> extern void its_enable_defaults(void);
>> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
>> +extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>>
>> #else /* __arm__ */
>>
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index d1e7e52..c2dcd01 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>>
>> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
>> }
>> +
>> +struct its_device *its_create_device(u32 device_id, int nr_ites)
>> +{
>> + struct its_baser *baser;
>> + struct its_device *new;
>> + unsigned long n, order;
>> +
>> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
>> +
>> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
>> + if (!baser)
>> + return NULL;
>
> Should we assert here if we can't find a GITS_BASER_TYPE_DEVICE ?
> It seems none of the callers of its_create_device are checking
> for null.
I removed that code which is useless.
>
>> +
>> + new = &its_data.devices[its_data.nr_devices];
>> +
>> + new->device_id = device_id;
>> + new->nr_ites = nr_ites;
>> +
>> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
>> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>
> I've seen this calculation several times now, so I think an
> arch-neutral order calculator is in order:
>
> int get_order(size_t size);
>
>> + new->itt = (void *)virt_to_phys(alloc_pages(order));
I added this get_order helper in lib/alloc_page.h/c
>
> If this is a physical address then shouldn't itt be phys_addr_t ?
>
>> +
>> + its_data.nr_devices++;
>> + return new;
>> +}
ITT now is a GVA (void *).
>> +
>> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
>> +{
>> + struct its_collection *new;
>> +
>> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
>> +
>> + new = &its_data.collections[its_data.nr_collections];
>> +
>> + new->col_id = col_id;
>> +
>> + if (its_data.typer.pta)
>> + new->target_address = (u64)gicv3_data.redist_base[pe];
>> + else
>> + new->target_address = pe << 16;
>> +
>> + its_data.nr_collections++;
>> + return new;
>> +}
>> --
>> 2.20.1
>>
>
> Thanks,
> drew
>
Thanks
Eric
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization
@ 2020-03-06 8:47 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 8:47 UTC (permalink / raw)
To: Andrew Jones
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 1:51 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:54AM +0100, Eric Auger wrote:
>> Introduce an helper functions to register
>> - a new device, characterized by its device id and the
>> max number of event IDs that dimension its ITT (Interrupt
>> Translation Table). The function allocates the ITT.
>>
>> - a new collection, characterized by its ID and the
>> target processing engine (PE).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - s/report_abort/assert
>>
>> v1 -> v2:
>> - s/nb_/nr_
>> ---
>> lib/arm/asm/gic-v3-its.h | 20 +++++++++++++++++-
>> lib/arm/gic-v3-its.c | 44 ++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 63 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index fe73c04..acd97a9 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -31,6 +31,19 @@ struct its_baser {
>> };
>>
>> #define GITS_BASER_NR_REGS 8
>> +#define GITS_MAX_DEVICES 8
>> +#define GITS_MAX_COLLECTIONS 8
>> +
>> +struct its_device {
>> + u32 device_id; /* device ID */
>> + u32 nr_ites; /* Max Interrupt Translation Entries */
>> + void *itt; /* Interrupt Translation Table GPA */
>> +};
>> +
>> +struct its_collection {
>> + u64 target_address;
>> + u16 col_id;
>> +};
>>
>> struct its_data {
>> void *base;
>> @@ -38,6 +51,10 @@ struct its_data {
>> struct its_baser baser[GITS_BASER_NR_REGS];
>> struct its_cmd_block *cmd_base;
>> struct its_cmd_block *cmd_write;
>> + struct its_device devices[GITS_MAX_DEVICES];
>> + u32 nr_devices; /* Allocated Devices */
>> + struct its_collection collections[GITS_MAX_COLLECTIONS];
>> + u32 nr_collections; /* Allocated Collections */
>> };
>>
>> extern struct its_data its_data;
>> @@ -90,7 +107,6 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>>
>> -
>> struct its_cmd_block {
>> u64 raw_cmd[4];
>> };
>> @@ -100,6 +116,8 @@ extern void its_init(void);
>> extern int its_parse_baser(int i, struct its_baser *baser);
>> extern struct its_baser *its_lookup_baser(int type);
>> extern void its_enable_defaults(void);
>> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
>> +extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>>
>> #else /* __arm__ */
>>
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index d1e7e52..c2dcd01 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -175,3 +175,47 @@ void its_enable_defaults(void)
>>
>> writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
>> }
>> +
>> +struct its_device *its_create_device(u32 device_id, int nr_ites)
>> +{
>> + struct its_baser *baser;
>> + struct its_device *new;
>> + unsigned long n, order;
>> +
>> + assert(its_data.nr_devices < GITS_MAX_DEVICES);
>> +
>> + baser = its_lookup_baser(GITS_BASER_TYPE_DEVICE);
>> + if (!baser)
>> + return NULL;
>
> Should we assert here if we can't find a GITS_BASER_TYPE_DEVICE ?
> It seems none of the callers of its_create_device are checking
> for null.
I removed that code which is useless.
>
>> +
>> + new = &its_data.devices[its_data.nr_devices];
>> +
>> + new->device_id = device_id;
>> + new->nr_ites = nr_ites;
>> +
>> + n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
>> + order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
>
> I've seen this calculation several times now, so I think an
> arch-neutral order calculator is in order:
>
> int get_order(size_t size);
>
>> + new->itt = (void *)virt_to_phys(alloc_pages(order));
I added this get_order helper in lib/alloc_page.h/c
>
> If this is a physical address then shouldn't itt be phys_addr_t ?
>
>> +
>> + its_data.nr_devices++;
>> + return new;
>> +}
ITT now is a GVA (void *).
>> +
>> +struct its_collection *its_create_collection(u32 col_id, u32 pe)
>> +{
>> + struct its_collection *new;
>> +
>> + assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
>> +
>> + new = &its_data.collections[its_data.nr_collections];
>> +
>> + new->col_id = col_id;
>> +
>> + if (its_data.typer.pta)
>> + new->target_address = (u64)gicv3_data.redist_base[pe];
>> + else
>> + new->target_address = pe << 16;
>> +
>> + its_data.nr_collections++;
>> + return new;
>> +}
>> --
>> 2.20.1
>>
>
> Thanks,
> drew
>
Thanks
Eric
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 10/14] arm/arm64: ITS: commands
2020-02-07 13:37 ` Andrew Jones
(?)
@ 2020-03-06 9:13 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 9:13 UTC (permalink / raw)
To: Andrew Jones
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
peter.maydell, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Hi Drew,
On 2/7/20 2:37 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:55AM +0100, Eric Auger wrote:
>> Implement main ITS commands. The code is largely inherited from
>> the ITS driver.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - do not use report() anymore
>> - assert if cmd_write exceeds the queue capacity
>>
>> v1 -> v2:
>> - removed its_print_cmd_state
>> ---
>> arm/Makefile.arm64 | 2 +-
>> lib/arm/asm/gic-v3-its.h | 38 +++-
>> lib/arm/gic-v3-its-cmd.c | 454 +++++++++++++++++++++++++++++++++++++++
>> 3 files changed, 492 insertions(+), 2 deletions(-)
>> create mode 100644 lib/arm/gic-v3-its-cmd.c
>>
>> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
>> index 2571ffb..d12aea5 100644
>> --- a/arm/Makefile.arm64
>> +++ b/arm/Makefile.arm64
>> @@ -19,7 +19,7 @@ endef
>> cstart.o = $(TEST_DIR)/cstart64.o
>> cflatobjs += lib/arm64/processor.o
>> cflatobjs += lib/arm64/spinlock.o
>> -cflatobjs += lib/arm/gic-v3-its.o
>> +cflatobjs += lib/arm/gic-v3-its.o lib/arm/gic-v3-its-cmd.o
>>
>> OBJDIRS += lib/arm64
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index acd97a9..0e5c5b6 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -45,6 +45,8 @@ struct its_collection {
>> u16 col_id;
>> };
>>
>> +struct its_cmd_block;
>> +
>
> This isn't necessary. If it was, then it should have been added in a
> previous patch.
removed
>
>> struct its_data {
>> void *base;
>> struct its_typer typer;
>> @@ -107,6 +109,24 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>>
>> +/*
>> + * ITS commands
>> + */
>> +#define GITS_CMD_MAPD 0x08
>> +#define GITS_CMD_MAPC 0x09
>> +#define GITS_CMD_MAPTI 0x0a
>> +/* older GIC documentation used MAPVI for this command */
>> +#define GITS_CMD_MAPVI GITS_CMD_MAPTI
>> +#define GITS_CMD_MAPI 0x0b
>> +#define GITS_CMD_MOVI 0x01
>> +#define GITS_CMD_DISCARD 0x0f
>> +#define GITS_CMD_INV 0x0c
>> +#define GITS_CMD_MOVALL 0x0e
>> +#define GITS_CMD_INVALL 0x0d
>> +#define GITS_CMD_INT 0x03
>> +#define GITS_CMD_CLEAR 0x04
>> +#define GITS_CMD_SYNC 0x05
>
> Please use tabs.
done
>
>> +
>> struct its_cmd_block {
>> u64 raw_cmd[4];
>> };
>> @@ -119,11 +139,27 @@ extern void its_enable_defaults(void);
>> extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
>> extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>>
>> +extern void its_send_mapd(struct its_device *dev, int valid);
>> +extern void its_send_mapc(struct its_collection *col, int valid);
>> +extern void its_send_mapti(struct its_device *dev, u32 irq_id,
>> + u32 event_id, struct its_collection *col);
>> +extern void its_send_int(struct its_device *dev, u32 event_id);
>> +extern void its_send_inv(struct its_device *dev, u32 event_id);
>> +extern void its_send_discard(struct its_device *dev, u32 event_id);
>> +extern void its_send_clear(struct its_device *dev, u32 event_id);
>> +extern void its_send_invall(struct its_collection *col);
>> +extern void its_send_movi(struct its_device *dev,
>> + struct its_collection *col, u32 id);
>> +extern void its_send_sync(struct its_collection *col);
>> +
>> +#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
>> +#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
>> +#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)
>
> What are these flags for?
removed
>
>> +
>> #else /* __arm__ */
>>
>> static inline void its_init(void) {}
>>
>> #endif
>> -
>> #endif /* !__ASSEMBLY__ */
>> #endif /* _ASMARM_GIC_V3_ITS_H_ */
>> diff --git a/lib/arm/gic-v3-its-cmd.c b/lib/arm/gic-v3-its-cmd.c
>> new file mode 100644
>> index 0000000..fb4364c
>> --- /dev/null
>> +++ b/lib/arm/gic-v3-its-cmd.c
>> @@ -0,0 +1,454 @@
>> +/*
>> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
>> + *
>> + * Most of the code is copy-pasted from:
>> + * drivers/irqchip/irq-gic-v3-its.c
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#include <asm/io.h>
>> +#include <asm/gic.h>
>> +#include <asm/gic-v3-its.h>
>> +
>> +#define ITS_ITT_ALIGN SZ_256
>
> tabs
done
>
>> +
>> +static const char * const its_cmd_string[] = {
>> + [GITS_CMD_MAPD] = "MAPD",
>> + [GITS_CMD_MAPC] = "MAPC",
>> + [GITS_CMD_MAPTI] = "MAPTI",
>> + [GITS_CMD_MAPI] = "MAPI",
>> + [GITS_CMD_MOVI] = "MOVI",
>> + [GITS_CMD_DISCARD] = "DISCARD",
>> + [GITS_CMD_INV] = "INV",
>> + [GITS_CMD_MOVALL] = "MOVALL",
>> + [GITS_CMD_INVALL] = "INVALL",
>> + [GITS_CMD_INT] = "INT",
>> + [GITS_CMD_CLEAR] = "CLEAR",
>> + [GITS_CMD_SYNC] = "SYNC",
>> +};
>> +
>> +struct its_cmd_desc {
>> + union {
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_inv_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_int_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + bool valid;
>> + } its_mapd_cmd;
>> +
>> + struct {
>> + struct its_collection *col;
>> + bool valid;
>> + } its_mapc_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 phys_id;
>> + u32 event_id;
>> + u32 col_id;
>> + } its_mapti_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + struct its_collection *col;
>> + u32 event_id;
>> + } its_movi_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_discard_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_clear_cmd;
>> +
>> + struct {
>> + struct its_collection *col;
>> + } its_invall_cmd;
>> +
>> + struct {
>> + struct its_collection *col;
>> + } its_sync_cmd;
>> + };
>> +};
>> +
>> +typedef void (*its_cmd_builder_t)(struct its_cmd_block *,
>> + struct its_cmd_desc *);
>> +
>> +/* ITS COMMANDS */
>> +
>> +static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
>> +{
>> + cmd->raw_cmd[0] &= ~0xffUL;
>> + cmd->raw_cmd[0] |= cmd_nr;
>> +}
>> +
>> +static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
>> +{
>> + cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
>> + cmd->raw_cmd[0] |= ((u64)devid) << 32;
>> +}
>> +
>> +static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
>> +{
>> + cmd->raw_cmd[1] &= ~0xffffffffUL;
>> + cmd->raw_cmd[1] |= id;
>> +}
>> +
>> +static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
>> +{
>> + cmd->raw_cmd[1] &= 0xffffffffUL;
>> + cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
>> +}
>> +
>> +static void its_encode_size(struct its_cmd_block *cmd, u8 size)
>> +{
>> + cmd->raw_cmd[1] &= ~0x1fUL;
>> + cmd->raw_cmd[1] |= size & 0x1f;
>> +}
>> +
>> +static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
>> +{
>> + cmd->raw_cmd[2] &= ~0xffffffffffffUL;
>> + cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
>> +}
>> +
>> +static void its_encode_valid(struct its_cmd_block *cmd, int valid)
>> +{
>> + cmd->raw_cmd[2] &= ~(1UL << 63);
>> + cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
>> +}
>> +
>> +static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
>> +{
>> + cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
>> + cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
>> +}
>> +
>> +static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
>> +{
>> + cmd->raw_cmd[2] &= ~0xffffUL;
>> + cmd->raw_cmd[2] |= col;
>> +}
>> +
>> +static inline void its_fixup_cmd(struct its_cmd_block *cmd)
>> +{
>> + /* Let's fixup BE commands */
>> + cmd->raw_cmd[0] = cpu_to_le64(cmd->raw_cmd[0]);
>> + cmd->raw_cmd[1] = cpu_to_le64(cmd->raw_cmd[1]);
>> + cmd->raw_cmd[2] = cpu_to_le64(cmd->raw_cmd[2]);
>> + cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
>> +}
>> +
>> +static u64 its_cmd_ptr_to_offset(struct its_cmd_block *ptr)
>> +{
>> + return (ptr - its_data.cmd_base) * sizeof(*ptr);
>> +}
>> +
>> +static struct its_cmd_block *its_post_commands(void)
>> +{
>> + u64 wr = its_cmd_ptr_to_offset(its_data.cmd_write);
>> +
>> + writeq(wr, its_data.base + GITS_CWRITER);
>> + return its_data.cmd_write;
>> +}
>> +
>> +
>
> extra blank line
removed
>
>> +static struct its_cmd_block *its_allocate_entry(void)
>> +{
>> + struct its_cmd_block *cmd;
>> +
>> + assert((u64)its_data.cmd_write < (u64)its_data.cmd_base + SZ_64K);
>> + cmd = its_data.cmd_write++;
>> + return cmd;
>> +}
>> +
>> +static void its_wait_for_range_completion(struct its_cmd_block *from,
>> + struct its_cmd_block *to)
>> +{
>> + u64 rd_idx, from_idx, to_idx;
>> + u32 count = 1000000; /* 1s! */
>> +
>> + from_idx = its_cmd_ptr_to_offset(from);
>> + to_idx = its_cmd_ptr_to_offset(to);
>> + while (1) {
>> + rd_idx = readq(its_data.base + GITS_CREADR);
>> + if (rd_idx >= to_idx || rd_idx < from_idx)
>> + break;
>> +
>> + count--;
>> + if (!count) {
>> + unsigned int cmd_id = from->raw_cmd[0] & 0xFF;
>> +
>> + assert_msg(false, "%s timeout!",
>> + cmd_id <= 0xF ? its_cmd_string[cmd_id] :
>> + "Unexpected");
>> + return;
>
> No need for 'return' after assert.
removed
>
>> + }
>> + cpu_relax();
>
> no need for cpu_relax right before udelay which calls cpu_relax
removed
>
>> + udelay(1);
>> + }
>> +}
>> +
>> +static void its_send_single_command(its_cmd_builder_t builder,
>> + struct its_cmd_desc *desc)
>> +{
>> + struct its_cmd_block *cmd, *next_cmd;
>> +
>> + cmd = its_allocate_entry();
>> + builder(cmd, desc);
>> + next_cmd = its_post_commands();
>> +
>> + its_wait_for_range_completion(cmd, next_cmd);
>> +}
>> +
>> +
>> +static void its_build_mapd_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + unsigned long itt_addr;
>> + u8 size = 12; /* 4096 eventids */
>> +
>> + itt_addr = (unsigned long)desc->its_mapd_cmd.dev->itt;
>> + itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
>> +
>> + its_encode_cmd(cmd, GITS_CMD_MAPD);
>> + its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
>> + its_encode_size(cmd, size - 1);
>> + its_encode_itt(cmd, itt_addr);
>> + its_encode_valid(cmd, desc->its_mapd_cmd.valid);
>> +
>> + its_fixup_cmd(cmd);
>> + printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
>> + desc->its_mapd_cmd.dev->device_id,
>> + size, itt_addr, desc->its_mapd_cmd.valid);
>> +
>> +}
>> +
>> +static void its_build_mapc_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_MAPC);
>> + its_encode_collection(cmd, desc->its_mapc_cmd.col->col_id);
>> + its_encode_target(cmd, desc->its_mapc_cmd.col->target_address);
>> + its_encode_valid(cmd, desc->its_mapc_cmd.valid);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("MAPC col_id=%d target_addr = 0x%lx valid=%d",
>> + desc->its_mapc_cmd.col->col_id,
>> + desc->its_mapc_cmd.col->target_address,
>> + desc->its_mapc_cmd.valid);
>
> printf, but better yet, leave the printing to the callers. We're in
> library code here, so if a unit test doesn't want this verbosity
> then they shouldn't have to have it. Same comment for the above printf
> and all the below report_infos.
I switched to printf. I added a verbose field to desc so that send*
function can pass a verbose argument. It is easier to trace at low
levels as all the fields are resolved there.
>
>> +}
>> +
>> +static void its_build_mapti_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_MAPTI);
>> + its_encode_devid(cmd, desc->its_mapti_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_mapti_cmd.event_id);
>> + its_encode_phys_id(cmd, desc->its_mapti_cmd.phys_id);
>> + its_encode_collection(cmd, desc->its_mapti_cmd.col_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("MAPTI dev_id=%d event_id=%d -> phys_id=%d, col_id=%d",
>> + desc->its_mapti_cmd.dev->device_id,
>> + desc->its_mapti_cmd.event_id,
>> + desc->its_mapti_cmd.phys_id,
>> + desc->its_mapti_cmd.col_id);
>> +}
>> +
>> +static void its_build_invall_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_INVALL);
>> + its_encode_collection(cmd, desc->its_invall_cmd.col->col_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("INVALL col_id=%d", desc->its_invall_cmd.col->col_id);
>> +}
>> +
>> +static void its_build_clear_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_CLEAR);
>> + its_encode_devid(cmd, desc->its_clear_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_clear_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("CLEAR col_id=%d", desc->its_invall_cmd.col->col_id);
>> +}
>> +
>> +static void its_build_discard_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_DISCARD);
>> + its_encode_devid(cmd, desc->its_discard_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_discard_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("DISCARD col_id=%d", desc->its_invall_cmd.col->col_id);
>> +}
>> +
>> +static void its_build_inv_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_INV);
>> + its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("INV dev_id=%d event_id=%d",
>> + desc->its_inv_cmd.dev->device_id,
>> + desc->its_inv_cmd.event_id);
>> +}
>> +
>> +static void its_build_int_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_INT);
>> + its_encode_devid(cmd, desc->its_int_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_int_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("INT dev_id=%d event_id=%d",
>> + desc->its_int_cmd.dev->device_id,
>> + desc->its_int_cmd.event_id);
>> +}
>> +
>> +static void its_build_sync_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_SYNC);
>> + its_encode_target(cmd, desc->its_sync_cmd.col->target_address);
>> + its_fixup_cmd(cmd);
>
> All the rest of the blocks have a blank line before its_fixup_cmd,
> but I actually like this one better. The blanks are unnecessary.
removed
>
>> + report_info("SYNC target_addr = 0x%lx",
>> + desc->its_sync_cmd.col->target_address);
>> +}
>> +
>> +static void its_build_movi_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_MOVI);
>> + its_encode_devid(cmd, desc->its_movi_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_movi_cmd.event_id);
>> + its_encode_collection(cmd, desc->its_movi_cmd.col->col_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("MOVI dev_id=%d event_id = %d col_id=%d",
>> + desc->its_movi_cmd.dev->device_id,
>> + desc->its_movi_cmd.event_id,
>> + desc->its_movi_cmd.col->col_id);
>> +}
>> +
>> +void its_send_mapd(struct its_device *dev, int valid)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_mapd_cmd.dev = dev;
>> + desc.its_mapd_cmd.valid = !!valid;
>> +
>> + its_send_single_command(its_build_mapd_cmd, &desc);
>> +}
>> +
>> +void its_send_mapc(struct its_collection *col, int valid)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_mapc_cmd.col = col;
>> + desc.its_mapc_cmd.valid = !!valid;
>> +
>> + its_send_single_command(its_build_mapc_cmd, &desc);
>> +}
>> +
>> +void its_send_mapti(struct its_device *dev, u32 irq_id,
>> + u32 event_id, struct its_collection *col)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_mapti_cmd.dev = dev;
>> + desc.its_mapti_cmd.phys_id = irq_id;
>> + desc.its_mapti_cmd.event_id = event_id;
>> + desc.its_mapti_cmd.col_id = col->col_id;
>> +
>> + its_send_single_command(its_build_mapti_cmd, &desc);
>> +}
>> +
>> +void its_send_int(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_int_cmd.dev = dev;
>> + desc.its_int_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_int_cmd, &desc);
>> +}
>> +
>> +void its_send_movi(struct its_device *dev,
>> + struct its_collection *col, u32 id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_movi_cmd.dev = dev;
>> + desc.its_movi_cmd.col = col;
>> + desc.its_movi_cmd.event_id = id;
>> +
>> + its_send_single_command(its_build_movi_cmd, &desc);
>> +}
>> +
>> +void its_send_invall(struct its_collection *col)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_invall_cmd.col = col;
>> +
>> + its_send_single_command(its_build_invall_cmd, &desc);
>> +}
>> +
>> +void its_send_inv(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_inv_cmd.dev = dev;
>> + desc.its_inv_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_inv_cmd, &desc);
>> +}
>> +
>> +void its_send_discard(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_discard_cmd.dev = dev;
>> + desc.its_discard_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_discard_cmd, &desc);
>> +}
>> +
>> +void its_send_clear(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_clear_cmd.dev = dev;
>> + desc.its_clear_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_clear_cmd, &desc);
>> +}
>> +
>> +void its_send_sync(struct its_collection *col)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_sync_cmd.col = col;
>> +
>> + its_send_single_command(its_build_sync_cmd, &desc);
>> +}
>> +
>> --
>> 2.20.1
>>
>>
>
> Thanks,
> drew
>
Thanks
Eric
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 10/14] arm/arm64: ITS: commands
@ 2020-03-06 9:13 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 9:13 UTC (permalink / raw)
To: Andrew Jones
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 2:37 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:55AM +0100, Eric Auger wrote:
>> Implement main ITS commands. The code is largely inherited from
>> the ITS driver.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - do not use report() anymore
>> - assert if cmd_write exceeds the queue capacity
>>
>> v1 -> v2:
>> - removed its_print_cmd_state
>> ---
>> arm/Makefile.arm64 | 2 +-
>> lib/arm/asm/gic-v3-its.h | 38 +++-
>> lib/arm/gic-v3-its-cmd.c | 454 +++++++++++++++++++++++++++++++++++++++
>> 3 files changed, 492 insertions(+), 2 deletions(-)
>> create mode 100644 lib/arm/gic-v3-its-cmd.c
>>
>> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
>> index 2571ffb..d12aea5 100644
>> --- a/arm/Makefile.arm64
>> +++ b/arm/Makefile.arm64
>> @@ -19,7 +19,7 @@ endef
>> cstart.o = $(TEST_DIR)/cstart64.o
>> cflatobjs += lib/arm64/processor.o
>> cflatobjs += lib/arm64/spinlock.o
>> -cflatobjs += lib/arm/gic-v3-its.o
>> +cflatobjs += lib/arm/gic-v3-its.o lib/arm/gic-v3-its-cmd.o
>>
>> OBJDIRS += lib/arm64
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index acd97a9..0e5c5b6 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -45,6 +45,8 @@ struct its_collection {
>> u16 col_id;
>> };
>>
>> +struct its_cmd_block;
>> +
>
> This isn't necessary. If it was, then it should have been added in a
> previous patch.
removed
>
>> struct its_data {
>> void *base;
>> struct its_typer typer;
>> @@ -107,6 +109,24 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>>
>> +/*
>> + * ITS commands
>> + */
>> +#define GITS_CMD_MAPD 0x08
>> +#define GITS_CMD_MAPC 0x09
>> +#define GITS_CMD_MAPTI 0x0a
>> +/* older GIC documentation used MAPVI for this command */
>> +#define GITS_CMD_MAPVI GITS_CMD_MAPTI
>> +#define GITS_CMD_MAPI 0x0b
>> +#define GITS_CMD_MOVI 0x01
>> +#define GITS_CMD_DISCARD 0x0f
>> +#define GITS_CMD_INV 0x0c
>> +#define GITS_CMD_MOVALL 0x0e
>> +#define GITS_CMD_INVALL 0x0d
>> +#define GITS_CMD_INT 0x03
>> +#define GITS_CMD_CLEAR 0x04
>> +#define GITS_CMD_SYNC 0x05
>
> Please use tabs.
done
>
>> +
>> struct its_cmd_block {
>> u64 raw_cmd[4];
>> };
>> @@ -119,11 +139,27 @@ extern void its_enable_defaults(void);
>> extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
>> extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>>
>> +extern void its_send_mapd(struct its_device *dev, int valid);
>> +extern void its_send_mapc(struct its_collection *col, int valid);
>> +extern void its_send_mapti(struct its_device *dev, u32 irq_id,
>> + u32 event_id, struct its_collection *col);
>> +extern void its_send_int(struct its_device *dev, u32 event_id);
>> +extern void its_send_inv(struct its_device *dev, u32 event_id);
>> +extern void its_send_discard(struct its_device *dev, u32 event_id);
>> +extern void its_send_clear(struct its_device *dev, u32 event_id);
>> +extern void its_send_invall(struct its_collection *col);
>> +extern void its_send_movi(struct its_device *dev,
>> + struct its_collection *col, u32 id);
>> +extern void its_send_sync(struct its_collection *col);
>> +
>> +#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
>> +#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
>> +#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)
>
> What are these flags for?
removed
>
>> +
>> #else /* __arm__ */
>>
>> static inline void its_init(void) {}
>>
>> #endif
>> -
>> #endif /* !__ASSEMBLY__ */
>> #endif /* _ASMARM_GIC_V3_ITS_H_ */
>> diff --git a/lib/arm/gic-v3-its-cmd.c b/lib/arm/gic-v3-its-cmd.c
>> new file mode 100644
>> index 0000000..fb4364c
>> --- /dev/null
>> +++ b/lib/arm/gic-v3-its-cmd.c
>> @@ -0,0 +1,454 @@
>> +/*
>> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
>> + *
>> + * Most of the code is copy-pasted from:
>> + * drivers/irqchip/irq-gic-v3-its.c
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#include <asm/io.h>
>> +#include <asm/gic.h>
>> +#include <asm/gic-v3-its.h>
>> +
>> +#define ITS_ITT_ALIGN SZ_256
>
> tabs
done
>
>> +
>> +static const char * const its_cmd_string[] = {
>> + [GITS_CMD_MAPD] = "MAPD",
>> + [GITS_CMD_MAPC] = "MAPC",
>> + [GITS_CMD_MAPTI] = "MAPTI",
>> + [GITS_CMD_MAPI] = "MAPI",
>> + [GITS_CMD_MOVI] = "MOVI",
>> + [GITS_CMD_DISCARD] = "DISCARD",
>> + [GITS_CMD_INV] = "INV",
>> + [GITS_CMD_MOVALL] = "MOVALL",
>> + [GITS_CMD_INVALL] = "INVALL",
>> + [GITS_CMD_INT] = "INT",
>> + [GITS_CMD_CLEAR] = "CLEAR",
>> + [GITS_CMD_SYNC] = "SYNC",
>> +};
>> +
>> +struct its_cmd_desc {
>> + union {
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_inv_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_int_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + bool valid;
>> + } its_mapd_cmd;
>> +
>> + struct {
>> + struct its_collection *col;
>> + bool valid;
>> + } its_mapc_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 phys_id;
>> + u32 event_id;
>> + u32 col_id;
>> + } its_mapti_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + struct its_collection *col;
>> + u32 event_id;
>> + } its_movi_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_discard_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_clear_cmd;
>> +
>> + struct {
>> + struct its_collection *col;
>> + } its_invall_cmd;
>> +
>> + struct {
>> + struct its_collection *col;
>> + } its_sync_cmd;
>> + };
>> +};
>> +
>> +typedef void (*its_cmd_builder_t)(struct its_cmd_block *,
>> + struct its_cmd_desc *);
>> +
>> +/* ITS COMMANDS */
>> +
>> +static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
>> +{
>> + cmd->raw_cmd[0] &= ~0xffUL;
>> + cmd->raw_cmd[0] |= cmd_nr;
>> +}
>> +
>> +static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
>> +{
>> + cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
>> + cmd->raw_cmd[0] |= ((u64)devid) << 32;
>> +}
>> +
>> +static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
>> +{
>> + cmd->raw_cmd[1] &= ~0xffffffffUL;
>> + cmd->raw_cmd[1] |= id;
>> +}
>> +
>> +static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
>> +{
>> + cmd->raw_cmd[1] &= 0xffffffffUL;
>> + cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
>> +}
>> +
>> +static void its_encode_size(struct its_cmd_block *cmd, u8 size)
>> +{
>> + cmd->raw_cmd[1] &= ~0x1fUL;
>> + cmd->raw_cmd[1] |= size & 0x1f;
>> +}
>> +
>> +static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
>> +{
>> + cmd->raw_cmd[2] &= ~0xffffffffffffUL;
>> + cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
>> +}
>> +
>> +static void its_encode_valid(struct its_cmd_block *cmd, int valid)
>> +{
>> + cmd->raw_cmd[2] &= ~(1UL << 63);
>> + cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
>> +}
>> +
>> +static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
>> +{
>> + cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
>> + cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
>> +}
>> +
>> +static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
>> +{
>> + cmd->raw_cmd[2] &= ~0xffffUL;
>> + cmd->raw_cmd[2] |= col;
>> +}
>> +
>> +static inline void its_fixup_cmd(struct its_cmd_block *cmd)
>> +{
>> + /* Let's fixup BE commands */
>> + cmd->raw_cmd[0] = cpu_to_le64(cmd->raw_cmd[0]);
>> + cmd->raw_cmd[1] = cpu_to_le64(cmd->raw_cmd[1]);
>> + cmd->raw_cmd[2] = cpu_to_le64(cmd->raw_cmd[2]);
>> + cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
>> +}
>> +
>> +static u64 its_cmd_ptr_to_offset(struct its_cmd_block *ptr)
>> +{
>> + return (ptr - its_data.cmd_base) * sizeof(*ptr);
>> +}
>> +
>> +static struct its_cmd_block *its_post_commands(void)
>> +{
>> + u64 wr = its_cmd_ptr_to_offset(its_data.cmd_write);
>> +
>> + writeq(wr, its_data.base + GITS_CWRITER);
>> + return its_data.cmd_write;
>> +}
>> +
>> +
>
> extra blank line
removed
>
>> +static struct its_cmd_block *its_allocate_entry(void)
>> +{
>> + struct its_cmd_block *cmd;
>> +
>> + assert((u64)its_data.cmd_write < (u64)its_data.cmd_base + SZ_64K);
>> + cmd = its_data.cmd_write++;
>> + return cmd;
>> +}
>> +
>> +static void its_wait_for_range_completion(struct its_cmd_block *from,
>> + struct its_cmd_block *to)
>> +{
>> + u64 rd_idx, from_idx, to_idx;
>> + u32 count = 1000000; /* 1s! */
>> +
>> + from_idx = its_cmd_ptr_to_offset(from);
>> + to_idx = its_cmd_ptr_to_offset(to);
>> + while (1) {
>> + rd_idx = readq(its_data.base + GITS_CREADR);
>> + if (rd_idx >= to_idx || rd_idx < from_idx)
>> + break;
>> +
>> + count--;
>> + if (!count) {
>> + unsigned int cmd_id = from->raw_cmd[0] & 0xFF;
>> +
>> + assert_msg(false, "%s timeout!",
>> + cmd_id <= 0xF ? its_cmd_string[cmd_id] :
>> + "Unexpected");
>> + return;
>
> No need for 'return' after assert.
removed
>
>> + }
>> + cpu_relax();
>
> no need for cpu_relax right before udelay which calls cpu_relax
removed
>
>> + udelay(1);
>> + }
>> +}
>> +
>> +static void its_send_single_command(its_cmd_builder_t builder,
>> + struct its_cmd_desc *desc)
>> +{
>> + struct its_cmd_block *cmd, *next_cmd;
>> +
>> + cmd = its_allocate_entry();
>> + builder(cmd, desc);
>> + next_cmd = its_post_commands();
>> +
>> + its_wait_for_range_completion(cmd, next_cmd);
>> +}
>> +
>> +
>> +static void its_build_mapd_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + unsigned long itt_addr;
>> + u8 size = 12; /* 4096 eventids */
>> +
>> + itt_addr = (unsigned long)desc->its_mapd_cmd.dev->itt;
>> + itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
>> +
>> + its_encode_cmd(cmd, GITS_CMD_MAPD);
>> + its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
>> + its_encode_size(cmd, size - 1);
>> + its_encode_itt(cmd, itt_addr);
>> + its_encode_valid(cmd, desc->its_mapd_cmd.valid);
>> +
>> + its_fixup_cmd(cmd);
>> + printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
>> + desc->its_mapd_cmd.dev->device_id,
>> + size, itt_addr, desc->its_mapd_cmd.valid);
>> +
>> +}
>> +
>> +static void its_build_mapc_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_MAPC);
>> + its_encode_collection(cmd, desc->its_mapc_cmd.col->col_id);
>> + its_encode_target(cmd, desc->its_mapc_cmd.col->target_address);
>> + its_encode_valid(cmd, desc->its_mapc_cmd.valid);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("MAPC col_id=%d target_addr = 0x%lx valid=%d",
>> + desc->its_mapc_cmd.col->col_id,
>> + desc->its_mapc_cmd.col->target_address,
>> + desc->its_mapc_cmd.valid);
>
> printf, but better yet, leave the printing to the callers. We're in
> library code here, so if a unit test doesn't want this verbosity
> then they shouldn't have to have it. Same comment for the above printf
> and all the below report_infos.
I switched to printf. I added a verbose field to desc so that send*
function can pass a verbose argument. It is easier to trace at low
levels as all the fields are resolved there.
>
>> +}
>> +
>> +static void its_build_mapti_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_MAPTI);
>> + its_encode_devid(cmd, desc->its_mapti_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_mapti_cmd.event_id);
>> + its_encode_phys_id(cmd, desc->its_mapti_cmd.phys_id);
>> + its_encode_collection(cmd, desc->its_mapti_cmd.col_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("MAPTI dev_id=%d event_id=%d -> phys_id=%d, col_id=%d",
>> + desc->its_mapti_cmd.dev->device_id,
>> + desc->its_mapti_cmd.event_id,
>> + desc->its_mapti_cmd.phys_id,
>> + desc->its_mapti_cmd.col_id);
>> +}
>> +
>> +static void its_build_invall_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_INVALL);
>> + its_encode_collection(cmd, desc->its_invall_cmd.col->col_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("INVALL col_id=%d", desc->its_invall_cmd.col->col_id);
>> +}
>> +
>> +static void its_build_clear_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_CLEAR);
>> + its_encode_devid(cmd, desc->its_clear_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_clear_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("CLEAR col_id=%d", desc->its_invall_cmd.col->col_id);
>> +}
>> +
>> +static void its_build_discard_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_DISCARD);
>> + its_encode_devid(cmd, desc->its_discard_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_discard_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("DISCARD col_id=%d", desc->its_invall_cmd.col->col_id);
>> +}
>> +
>> +static void its_build_inv_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_INV);
>> + its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("INV dev_id=%d event_id=%d",
>> + desc->its_inv_cmd.dev->device_id,
>> + desc->its_inv_cmd.event_id);
>> +}
>> +
>> +static void its_build_int_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_INT);
>> + its_encode_devid(cmd, desc->its_int_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_int_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("INT dev_id=%d event_id=%d",
>> + desc->its_int_cmd.dev->device_id,
>> + desc->its_int_cmd.event_id);
>> +}
>> +
>> +static void its_build_sync_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_SYNC);
>> + its_encode_target(cmd, desc->its_sync_cmd.col->target_address);
>> + its_fixup_cmd(cmd);
>
> All the rest of the blocks have a blank line before its_fixup_cmd,
> but I actually like this one better. The blanks are unnecessary.
removed
>
>> + report_info("SYNC target_addr = 0x%lx",
>> + desc->its_sync_cmd.col->target_address);
>> +}
>> +
>> +static void its_build_movi_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_MOVI);
>> + its_encode_devid(cmd, desc->its_movi_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_movi_cmd.event_id);
>> + its_encode_collection(cmd, desc->its_movi_cmd.col->col_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("MOVI dev_id=%d event_id = %d col_id=%d",
>> + desc->its_movi_cmd.dev->device_id,
>> + desc->its_movi_cmd.event_id,
>> + desc->its_movi_cmd.col->col_id);
>> +}
>> +
>> +void its_send_mapd(struct its_device *dev, int valid)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_mapd_cmd.dev = dev;
>> + desc.its_mapd_cmd.valid = !!valid;
>> +
>> + its_send_single_command(its_build_mapd_cmd, &desc);
>> +}
>> +
>> +void its_send_mapc(struct its_collection *col, int valid)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_mapc_cmd.col = col;
>> + desc.its_mapc_cmd.valid = !!valid;
>> +
>> + its_send_single_command(its_build_mapc_cmd, &desc);
>> +}
>> +
>> +void its_send_mapti(struct its_device *dev, u32 irq_id,
>> + u32 event_id, struct its_collection *col)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_mapti_cmd.dev = dev;
>> + desc.its_mapti_cmd.phys_id = irq_id;
>> + desc.its_mapti_cmd.event_id = event_id;
>> + desc.its_mapti_cmd.col_id = col->col_id;
>> +
>> + its_send_single_command(its_build_mapti_cmd, &desc);
>> +}
>> +
>> +void its_send_int(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_int_cmd.dev = dev;
>> + desc.its_int_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_int_cmd, &desc);
>> +}
>> +
>> +void its_send_movi(struct its_device *dev,
>> + struct its_collection *col, u32 id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_movi_cmd.dev = dev;
>> + desc.its_movi_cmd.col = col;
>> + desc.its_movi_cmd.event_id = id;
>> +
>> + its_send_single_command(its_build_movi_cmd, &desc);
>> +}
>> +
>> +void its_send_invall(struct its_collection *col)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_invall_cmd.col = col;
>> +
>> + its_send_single_command(its_build_invall_cmd, &desc);
>> +}
>> +
>> +void its_send_inv(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_inv_cmd.dev = dev;
>> + desc.its_inv_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_inv_cmd, &desc);
>> +}
>> +
>> +void its_send_discard(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_discard_cmd.dev = dev;
>> + desc.its_discard_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_discard_cmd, &desc);
>> +}
>> +
>> +void its_send_clear(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_clear_cmd.dev = dev;
>> + desc.its_clear_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_clear_cmd, &desc);
>> +}
>> +
>> +void its_send_sync(struct its_collection *col)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_sync_cmd.col = col;
>> +
>> + its_send_single_command(its_build_sync_cmd, &desc);
>> +}
>> +
>> --
>> 2.20.1
>>
>>
>
> Thanks,
> drew
>
Thanks
Eric
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 10/14] arm/arm64: ITS: commands
@ 2020-03-06 9:13 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 9:13 UTC (permalink / raw)
To: Andrew Jones
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 2:37 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:55AM +0100, Eric Auger wrote:
>> Implement main ITS commands. The code is largely inherited from
>> the ITS driver.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - do not use report() anymore
>> - assert if cmd_write exceeds the queue capacity
>>
>> v1 -> v2:
>> - removed its_print_cmd_state
>> ---
>> arm/Makefile.arm64 | 2 +-
>> lib/arm/asm/gic-v3-its.h | 38 +++-
>> lib/arm/gic-v3-its-cmd.c | 454 +++++++++++++++++++++++++++++++++++++++
>> 3 files changed, 492 insertions(+), 2 deletions(-)
>> create mode 100644 lib/arm/gic-v3-its-cmd.c
>>
>> diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
>> index 2571ffb..d12aea5 100644
>> --- a/arm/Makefile.arm64
>> +++ b/arm/Makefile.arm64
>> @@ -19,7 +19,7 @@ endef
>> cstart.o = $(TEST_DIR)/cstart64.o
>> cflatobjs += lib/arm64/processor.o
>> cflatobjs += lib/arm64/spinlock.o
>> -cflatobjs += lib/arm/gic-v3-its.o
>> +cflatobjs += lib/arm/gic-v3-its.o lib/arm/gic-v3-its-cmd.o
>>
>> OBJDIRS += lib/arm64
>>
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index acd97a9..0e5c5b6 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -45,6 +45,8 @@ struct its_collection {
>> u16 col_id;
>> };
>>
>> +struct its_cmd_block;
>> +
>
> This isn't necessary. If it was, then it should have been added in a
> previous patch.
removed
>
>> struct its_data {
>> void *base;
>> struct its_typer typer;
>> @@ -107,6 +109,24 @@ extern struct its_data its_data;
>> #define GITS_BASER_TYPE_DEVICE 1
>> #define GITS_BASER_TYPE_COLLECTION 4
>>
>> +/*
>> + * ITS commands
>> + */
>> +#define GITS_CMD_MAPD 0x08
>> +#define GITS_CMD_MAPC 0x09
>> +#define GITS_CMD_MAPTI 0x0a
>> +/* older GIC documentation used MAPVI for this command */
>> +#define GITS_CMD_MAPVI GITS_CMD_MAPTI
>> +#define GITS_CMD_MAPI 0x0b
>> +#define GITS_CMD_MOVI 0x01
>> +#define GITS_CMD_DISCARD 0x0f
>> +#define GITS_CMD_INV 0x0c
>> +#define GITS_CMD_MOVALL 0x0e
>> +#define GITS_CMD_INVALL 0x0d
>> +#define GITS_CMD_INT 0x03
>> +#define GITS_CMD_CLEAR 0x04
>> +#define GITS_CMD_SYNC 0x05
>
> Please use tabs.
done
>
>> +
>> struct its_cmd_block {
>> u64 raw_cmd[4];
>> };
>> @@ -119,11 +139,27 @@ extern void its_enable_defaults(void);
>> extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
>> extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
>>
>> +extern void its_send_mapd(struct its_device *dev, int valid);
>> +extern void its_send_mapc(struct its_collection *col, int valid);
>> +extern void its_send_mapti(struct its_device *dev, u32 irq_id,
>> + u32 event_id, struct its_collection *col);
>> +extern void its_send_int(struct its_device *dev, u32 event_id);
>> +extern void its_send_inv(struct its_device *dev, u32 event_id);
>> +extern void its_send_discard(struct its_device *dev, u32 event_id);
>> +extern void its_send_clear(struct its_device *dev, u32 event_id);
>> +extern void its_send_invall(struct its_collection *col);
>> +extern void its_send_movi(struct its_device *dev,
>> + struct its_collection *col, u32 id);
>> +extern void its_send_sync(struct its_collection *col);
>> +
>> +#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
>> +#define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
>> +#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)
>
> What are these flags for?
removed
>
>> +
>> #else /* __arm__ */
>>
>> static inline void its_init(void) {}
>>
>> #endif
>> -
>> #endif /* !__ASSEMBLY__ */
>> #endif /* _ASMARM_GIC_V3_ITS_H_ */
>> diff --git a/lib/arm/gic-v3-its-cmd.c b/lib/arm/gic-v3-its-cmd.c
>> new file mode 100644
>> index 0000000..fb4364c
>> --- /dev/null
>> +++ b/lib/arm/gic-v3-its-cmd.c
>> @@ -0,0 +1,454 @@
>> +/*
>> + * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
>> + *
>> + * Most of the code is copy-pasted from:
>> + * drivers/irqchip/irq-gic-v3-its.c
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#include <asm/io.h>
>> +#include <asm/gic.h>
>> +#include <asm/gic-v3-its.h>
>> +
>> +#define ITS_ITT_ALIGN SZ_256
>
> tabs
done
>
>> +
>> +static const char * const its_cmd_string[] = {
>> + [GITS_CMD_MAPD] = "MAPD",
>> + [GITS_CMD_MAPC] = "MAPC",
>> + [GITS_CMD_MAPTI] = "MAPTI",
>> + [GITS_CMD_MAPI] = "MAPI",
>> + [GITS_CMD_MOVI] = "MOVI",
>> + [GITS_CMD_DISCARD] = "DISCARD",
>> + [GITS_CMD_INV] = "INV",
>> + [GITS_CMD_MOVALL] = "MOVALL",
>> + [GITS_CMD_INVALL] = "INVALL",
>> + [GITS_CMD_INT] = "INT",
>> + [GITS_CMD_CLEAR] = "CLEAR",
>> + [GITS_CMD_SYNC] = "SYNC",
>> +};
>> +
>> +struct its_cmd_desc {
>> + union {
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_inv_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_int_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + bool valid;
>> + } its_mapd_cmd;
>> +
>> + struct {
>> + struct its_collection *col;
>> + bool valid;
>> + } its_mapc_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 phys_id;
>> + u32 event_id;
>> + u32 col_id;
>> + } its_mapti_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + struct its_collection *col;
>> + u32 event_id;
>> + } its_movi_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_discard_cmd;
>> +
>> + struct {
>> + struct its_device *dev;
>> + u32 event_id;
>> + } its_clear_cmd;
>> +
>> + struct {
>> + struct its_collection *col;
>> + } its_invall_cmd;
>> +
>> + struct {
>> + struct its_collection *col;
>> + } its_sync_cmd;
>> + };
>> +};
>> +
>> +typedef void (*its_cmd_builder_t)(struct its_cmd_block *,
>> + struct its_cmd_desc *);
>> +
>> +/* ITS COMMANDS */
>> +
>> +static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
>> +{
>> + cmd->raw_cmd[0] &= ~0xffUL;
>> + cmd->raw_cmd[0] |= cmd_nr;
>> +}
>> +
>> +static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
>> +{
>> + cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
>> + cmd->raw_cmd[0] |= ((u64)devid) << 32;
>> +}
>> +
>> +static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
>> +{
>> + cmd->raw_cmd[1] &= ~0xffffffffUL;
>> + cmd->raw_cmd[1] |= id;
>> +}
>> +
>> +static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
>> +{
>> + cmd->raw_cmd[1] &= 0xffffffffUL;
>> + cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
>> +}
>> +
>> +static void its_encode_size(struct its_cmd_block *cmd, u8 size)
>> +{
>> + cmd->raw_cmd[1] &= ~0x1fUL;
>> + cmd->raw_cmd[1] |= size & 0x1f;
>> +}
>> +
>> +static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
>> +{
>> + cmd->raw_cmd[2] &= ~0xffffffffffffUL;
>> + cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
>> +}
>> +
>> +static void its_encode_valid(struct its_cmd_block *cmd, int valid)
>> +{
>> + cmd->raw_cmd[2] &= ~(1UL << 63);
>> + cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
>> +}
>> +
>> +static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
>> +{
>> + cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
>> + cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
>> +}
>> +
>> +static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
>> +{
>> + cmd->raw_cmd[2] &= ~0xffffUL;
>> + cmd->raw_cmd[2] |= col;
>> +}
>> +
>> +static inline void its_fixup_cmd(struct its_cmd_block *cmd)
>> +{
>> + /* Let's fixup BE commands */
>> + cmd->raw_cmd[0] = cpu_to_le64(cmd->raw_cmd[0]);
>> + cmd->raw_cmd[1] = cpu_to_le64(cmd->raw_cmd[1]);
>> + cmd->raw_cmd[2] = cpu_to_le64(cmd->raw_cmd[2]);
>> + cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
>> +}
>> +
>> +static u64 its_cmd_ptr_to_offset(struct its_cmd_block *ptr)
>> +{
>> + return (ptr - its_data.cmd_base) * sizeof(*ptr);
>> +}
>> +
>> +static struct its_cmd_block *its_post_commands(void)
>> +{
>> + u64 wr = its_cmd_ptr_to_offset(its_data.cmd_write);
>> +
>> + writeq(wr, its_data.base + GITS_CWRITER);
>> + return its_data.cmd_write;
>> +}
>> +
>> +
>
> extra blank line
removed
>
>> +static struct its_cmd_block *its_allocate_entry(void)
>> +{
>> + struct its_cmd_block *cmd;
>> +
>> + assert((u64)its_data.cmd_write < (u64)its_data.cmd_base + SZ_64K);
>> + cmd = its_data.cmd_write++;
>> + return cmd;
>> +}
>> +
>> +static void its_wait_for_range_completion(struct its_cmd_block *from,
>> + struct its_cmd_block *to)
>> +{
>> + u64 rd_idx, from_idx, to_idx;
>> + u32 count = 1000000; /* 1s! */
>> +
>> + from_idx = its_cmd_ptr_to_offset(from);
>> + to_idx = its_cmd_ptr_to_offset(to);
>> + while (1) {
>> + rd_idx = readq(its_data.base + GITS_CREADR);
>> + if (rd_idx >= to_idx || rd_idx < from_idx)
>> + break;
>> +
>> + count--;
>> + if (!count) {
>> + unsigned int cmd_id = from->raw_cmd[0] & 0xFF;
>> +
>> + assert_msg(false, "%s timeout!",
>> + cmd_id <= 0xF ? its_cmd_string[cmd_id] :
>> + "Unexpected");
>> + return;
>
> No need for 'return' after assert.
removed
>
>> + }
>> + cpu_relax();
>
> no need for cpu_relax right before udelay which calls cpu_relax
removed
>
>> + udelay(1);
>> + }
>> +}
>> +
>> +static void its_send_single_command(its_cmd_builder_t builder,
>> + struct its_cmd_desc *desc)
>> +{
>> + struct its_cmd_block *cmd, *next_cmd;
>> +
>> + cmd = its_allocate_entry();
>> + builder(cmd, desc);
>> + next_cmd = its_post_commands();
>> +
>> + its_wait_for_range_completion(cmd, next_cmd);
>> +}
>> +
>> +
>> +static void its_build_mapd_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + unsigned long itt_addr;
>> + u8 size = 12; /* 4096 eventids */
>> +
>> + itt_addr = (unsigned long)desc->its_mapd_cmd.dev->itt;
>> + itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
>> +
>> + its_encode_cmd(cmd, GITS_CMD_MAPD);
>> + its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
>> + its_encode_size(cmd, size - 1);
>> + its_encode_itt(cmd, itt_addr);
>> + its_encode_valid(cmd, desc->its_mapd_cmd.valid);
>> +
>> + its_fixup_cmd(cmd);
>> + printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
>> + desc->its_mapd_cmd.dev->device_id,
>> + size, itt_addr, desc->its_mapd_cmd.valid);
>> +
>> +}
>> +
>> +static void its_build_mapc_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_MAPC);
>> + its_encode_collection(cmd, desc->its_mapc_cmd.col->col_id);
>> + its_encode_target(cmd, desc->its_mapc_cmd.col->target_address);
>> + its_encode_valid(cmd, desc->its_mapc_cmd.valid);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("MAPC col_id=%d target_addr = 0x%lx valid=%d",
>> + desc->its_mapc_cmd.col->col_id,
>> + desc->its_mapc_cmd.col->target_address,
>> + desc->its_mapc_cmd.valid);
>
> printf, but better yet, leave the printing to the callers. We're in
> library code here, so if a unit test doesn't want this verbosity
> then they shouldn't have to have it. Same comment for the above printf
> and all the below report_infos.
I switched to printf. I added a verbose field to desc so that send*
function can pass a verbose argument. It is easier to trace at low
levels as all the fields are resolved there.
>
>> +}
>> +
>> +static void its_build_mapti_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_MAPTI);
>> + its_encode_devid(cmd, desc->its_mapti_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_mapti_cmd.event_id);
>> + its_encode_phys_id(cmd, desc->its_mapti_cmd.phys_id);
>> + its_encode_collection(cmd, desc->its_mapti_cmd.col_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("MAPTI dev_id=%d event_id=%d -> phys_id=%d, col_id=%d",
>> + desc->its_mapti_cmd.dev->device_id,
>> + desc->its_mapti_cmd.event_id,
>> + desc->its_mapti_cmd.phys_id,
>> + desc->its_mapti_cmd.col_id);
>> +}
>> +
>> +static void its_build_invall_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_INVALL);
>> + its_encode_collection(cmd, desc->its_invall_cmd.col->col_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("INVALL col_id=%d", desc->its_invall_cmd.col->col_id);
>> +}
>> +
>> +static void its_build_clear_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_CLEAR);
>> + its_encode_devid(cmd, desc->its_clear_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_clear_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("CLEAR col_id=%d", desc->its_invall_cmd.col->col_id);
>> +}
>> +
>> +static void its_build_discard_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_DISCARD);
>> + its_encode_devid(cmd, desc->its_discard_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_discard_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("DISCARD col_id=%d", desc->its_invall_cmd.col->col_id);
>> +}
>> +
>> +static void its_build_inv_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_INV);
>> + its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("INV dev_id=%d event_id=%d",
>> + desc->its_inv_cmd.dev->device_id,
>> + desc->its_inv_cmd.event_id);
>> +}
>> +
>> +static void its_build_int_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_INT);
>> + its_encode_devid(cmd, desc->its_int_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_int_cmd.event_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("INT dev_id=%d event_id=%d",
>> + desc->its_int_cmd.dev->device_id,
>> + desc->its_int_cmd.event_id);
>> +}
>> +
>> +static void its_build_sync_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_SYNC);
>> + its_encode_target(cmd, desc->its_sync_cmd.col->target_address);
>> + its_fixup_cmd(cmd);
>
> All the rest of the blocks have a blank line before its_fixup_cmd,
> but I actually like this one better. The blanks are unnecessary.
removed
>
>> + report_info("SYNC target_addr = 0x%lx",
>> + desc->its_sync_cmd.col->target_address);
>> +}
>> +
>> +static void its_build_movi_cmd(struct its_cmd_block *cmd,
>> + struct its_cmd_desc *desc)
>> +{
>> + its_encode_cmd(cmd, GITS_CMD_MOVI);
>> + its_encode_devid(cmd, desc->its_movi_cmd.dev->device_id);
>> + its_encode_event_id(cmd, desc->its_movi_cmd.event_id);
>> + its_encode_collection(cmd, desc->its_movi_cmd.col->col_id);
>> +
>> + its_fixup_cmd(cmd);
>> + report_info("MOVI dev_id=%d event_id = %d col_id=%d",
>> + desc->its_movi_cmd.dev->device_id,
>> + desc->its_movi_cmd.event_id,
>> + desc->its_movi_cmd.col->col_id);
>> +}
>> +
>> +void its_send_mapd(struct its_device *dev, int valid)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_mapd_cmd.dev = dev;
>> + desc.its_mapd_cmd.valid = !!valid;
>> +
>> + its_send_single_command(its_build_mapd_cmd, &desc);
>> +}
>> +
>> +void its_send_mapc(struct its_collection *col, int valid)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_mapc_cmd.col = col;
>> + desc.its_mapc_cmd.valid = !!valid;
>> +
>> + its_send_single_command(its_build_mapc_cmd, &desc);
>> +}
>> +
>> +void its_send_mapti(struct its_device *dev, u32 irq_id,
>> + u32 event_id, struct its_collection *col)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_mapti_cmd.dev = dev;
>> + desc.its_mapti_cmd.phys_id = irq_id;
>> + desc.its_mapti_cmd.event_id = event_id;
>> + desc.its_mapti_cmd.col_id = col->col_id;
>> +
>> + its_send_single_command(its_build_mapti_cmd, &desc);
>> +}
>> +
>> +void its_send_int(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_int_cmd.dev = dev;
>> + desc.its_int_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_int_cmd, &desc);
>> +}
>> +
>> +void its_send_movi(struct its_device *dev,
>> + struct its_collection *col, u32 id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_movi_cmd.dev = dev;
>> + desc.its_movi_cmd.col = col;
>> + desc.its_movi_cmd.event_id = id;
>> +
>> + its_send_single_command(its_build_movi_cmd, &desc);
>> +}
>> +
>> +void its_send_invall(struct its_collection *col)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_invall_cmd.col = col;
>> +
>> + its_send_single_command(its_build_invall_cmd, &desc);
>> +}
>> +
>> +void its_send_inv(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_inv_cmd.dev = dev;
>> + desc.its_inv_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_inv_cmd, &desc);
>> +}
>> +
>> +void its_send_discard(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_discard_cmd.dev = dev;
>> + desc.its_discard_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_discard_cmd, &desc);
>> +}
>> +
>> +void its_send_clear(struct its_device *dev, u32 event_id)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_clear_cmd.dev = dev;
>> + desc.its_clear_cmd.event_id = event_id;
>> +
>> + its_send_single_command(its_build_clear_cmd, &desc);
>> +}
>> +
>> +void its_send_sync(struct its_collection *col)
>> +{
>> + struct its_cmd_desc desc;
>> +
>> + desc.its_sync_cmd.col = col;
>> +
>> + its_send_single_command(its_build_sync_cmd, &desc);
>> +}
>> +
>> --
>> 2.20.1
>>
>>
>
> Thanks,
> drew
>
Thanks
Eric
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
2020-02-07 13:15 ` Andrew Jones
(?)
@ 2020-03-06 12:55 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 12:55 UTC (permalink / raw)
To: Andrew Jones
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
andre.przywara, peter.maydell, yuzenghui, alexandru.elisei,
thuth
Hi Drew,
On 2/7/20 2:15 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:56AM +0100, Eric Auger wrote:
>> Triggers LPIs through the INT command.
>>
>> the test checks the LPI hits the right CPU and triggers
>> the right LPI intid, ie. the translation is correct.
>>
>> Updates to the config table also are tested, along with inv
>> and invall commands.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - add comments
>> - keep the report_skip in case there aren't 4 vcpus to be able to
>> run other tests in the its category.
>> - fix the prefix pop
>> - move its_event and its_stats to arm/gic.c
>> ---
>> arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
>> arm/unittests.cfg | 7 ++
>> 2 files changed, 224 insertions(+), 11 deletions(-)
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index 4d7dd03..50104b1 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
>> }
>> }
>>
>> +static void setup_irq(handler_t handler)
>> +{
>> + gic_enable_defaults();
>> +#ifdef __arm__
>> + install_exception_handler(EXCPTN_IRQ, handler);
>> +#else
>> + install_irq_handler(EL1H_IRQ, handler);
>> +#endif
>> + local_irq_enable();
>> +}
>> +
>> +#if defined(__aarch64__)
>> +struct its_event {
>> + int cpu_id;
>> + int lpi_id;
>> +};
>> +
>> +struct its_stats {
>> + struct its_event expected;
>> + struct its_event observed;
>> +};
>> +
>> +static struct its_stats lpi_stats;
>> +
>> +static void lpi_handler(struct pt_regs *regs __unused)
>> +{
>> + u32 irqstat = gic_read_iar();
>> + int irqnr = gic_iar_irqnr(irqstat);
>> +
>> + gic_write_eoir(irqstat);
>> + if (irqnr < 8192)
>> + report(false, "Unexpected non LPI interrupt received");
>
> report_info
why? This is an error case. We do not expect other interrupts than LPIs
>
>> + smp_rmb(); /* pairs with wmb in lpi_stats_expect */
>> + lpi_stats.observed.cpu_id = smp_processor_id();
>> + lpi_stats.observed.lpi_id = irqnr;
>> + smp_wmb(); /* pairs with rmb in check_lpi_stats */
>> +}
>> +
>> +static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
>> +{
>> + lpi_stats.expected.cpu_id = exp_cpu_id;
>> + lpi_stats.expected.lpi_id = exp_lpi_id;
>> + lpi_stats.observed.cpu_id = -1;
>> + lpi_stats.observed.lpi_id = -1;
>> + smp_wmb(); /* pairs with rmb in handler */
>> +}
>> +
>> +static void check_lpi_stats(void)
>
> static void check_lpi_stats(const char *testname)
> {
> bool pass = false;
>
>> +{
>> + mdelay(100);
>> + smp_rmb(); /* pairs with wmb in lpi_handler */
>> + if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
>> + (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
>
> nit: extra ()
>
>> + if (lpi_stats.observed.cpu_id == -1 &&
>> + lpi_stats.observed.lpi_id == -1) {
>> + report(false,
>> + "No LPI received whereas (cpuid=%d, intid=%d) "
>> + "was expected", lpi_stats.expected.cpu_id,
>> + lpi_stats.expected.lpi_id);
>
> report_info
What's the problem keeping those. Those are error reports. The message
is something like that:
FAIL: gicv3: its-trigger: mapc valid=false: No LPI received whereas
(cpuid=1, intid=8192) was expected.
So the testname is already part of the message.
>
>> + } else {
>> + report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
>> + lpi_stats.observed.cpu_id,
>> + lpi_stats.observed.lpi_id);
>
> report_info
>
>> + }
>
> pass = false;
>
>> + } else if (lpi_stats.expected.lpi_id != -1) {
>> + report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
>> + lpi_stats.observed.cpu_id);
>
> report_info
>
>> + } else {
>> + report(true, "no LPI received, as expected");
>
> report_info
>
>
>> + }
>
> report(pass, "%s", testname);
>
>> +}
>> +
>> +static void secondary_lpi_test(void)
>> +{
>> + setup_irq(lpi_handler);
>> + cpumask_set_cpu(smp_processor_id(), &ready);
>> + while (1)
>> + wfi();
>> +}
>> +#endif
>> +
>> static void gicv2_ipi_send_self(void)
>> {
>> writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
>> @@ -217,17 +298,6 @@ static void ipi_test_smp(void)
>> report_prefix_pop();
>> }
>>
>> -static void setup_irq(handler_t handler)
>> -{
>> - gic_enable_defaults();
>> -#ifdef __arm__
>> - install_exception_handler(EXCPTN_IRQ, handler);
>> -#else
>> - install_irq_handler(EL1H_IRQ, handler);
>> -#endif
>> - local_irq_enable();
>> -}
>> -
>> static void ipi_send(void)
>> {
>> setup_irq(ipi_handler);
>> @@ -522,6 +592,7 @@ static void gic_test_mmio(void)
>> #if defined(__arm__)
>>
>> static void test_its_introspection(void) {}
>> +static void test_its_trigger(void) {}
>>
>> #else /* __arch64__ */
>>
>> @@ -561,6 +632,137 @@ static void test_its_introspection(void)
>> report_info("collection baser entry_size = 0x%x", coll_baser->esz);
>> }
>>
>> +static bool its_prerequisites(int nb_cpus)
>> +{
>> + int cpu;
>> +
>> + if (!gicv3_its_base()) {
>> + report_skip("No ITS, skip ...");
>> + return true;
>> + }
>> +
>> + if (nr_cpus < 4) {
>
> nr_cpus < nb_cpus, or just drop the nb_cpus parameter and hard code 4
> here.
sure
>
>> + report_skip("Test requires at least %d vcpus", nb_cpus);
>> + return true;
>> + }
>> +
>> + stats_reset();
>> +
>> + setup_irq(lpi_handler);
>> +
>> + for_each_present_cpu(cpu) {
>> + if (cpu == 0)
>> + continue;
>> + smp_boot_secondary(cpu, secondary_lpi_test);
>> + }
>> + wait_on_ready();
>> +
>> + its_enable_defaults();
>> +
>> + lpi_stats_expect(-1, -1);
>> + check_lpi_stats();
>> +
>> + return false;
>
> Reverse logic. I'd expect 'return true' for success.
I am going to return an int. In case of error a std negative error will
be returned.
>
>> +}
>> +
>> +static void test_its_trigger(void)
>> +{
>> + struct its_collection *col3, *col2;
>> + struct its_device *dev2, *dev7;
>> +
>> + if (its_prerequisites(4))
>
> if (!its_prerequisites(...))
>
>> + return;
>> +
>> + dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>> + dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
>> +
>> + col3 = its_create_collection(3 /* col id */, 3/* target PE */);
>> + col2 = its_create_collection(2 /* col id */, 2/* target PE */);
>> +
>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>> + gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
>> +
>> + its_send_invall(col2);
>> + its_send_invall(col3);
>> +
>> + report_prefix_push("int");
>> + /*
>> + * dev=2, eventid=20 -> lpi= 8195, col=3
>> + * dev=7, eventid=255 -> lpi= 8196, col=2
>> + * Trigger dev2, eventid=20 and dev7, eventid=255
>> + * Check both LPIs hit
>> + */
>> +
>> + its_send_mapd(dev2, true);
>> + its_send_mapd(dev7, true);
>> +
>> + its_send_mapc(col3, true);
>> + its_send_mapc(col2, true);
>> +
>> + its_send_mapti(dev2, 8195 /* lpi id */,
>> + 20 /* event id */, col3);
>> + its_send_mapti(dev7, 8196 /* lpi id */,
>> + 255 /* event id */, col2);
>
> No need for line breaks, with the embedded comments it's hard to read
OK
>
>> +
>> + lpi_stats_expect(3, 8195);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + lpi_stats_expect(2, 8196);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> +
>> + report_prefix_pop();
>
> I think a table of parameters and loop would be nicer than all the
> repeated function calls.
Frankly speaking I am not sure this would really help. We are just
enabling 2 translation paths. I think I prefer to manipulate the low
level objects and helpers rather than playing with a loop and potential
new structs of params.
>
>> +
>> + report_prefix_push("inv/invall");
>> +
>> + /*
>> + * disable 8195, check dev2/eventid=20 does not trigger the
>> + * corresponding LPI
>> + */
>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
>
> LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED
ok
>
>> + its_send_inv(dev2, 20);
>> +
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + /*
>> + * re-enable the LPI but willingly do not call invall
>> + * so the change in config is not taken into account.
>> + * The LPI should not hit
>> + */
>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + /* Now call the invall and check the LPI hits */
>> + its_send_invall(col3);
>> + lpi_stats_expect(3, 8195);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + report_prefix_pop();
>
> Need blank line here.
OK
>
>> + /*
>> + * Unmap device 2 and check the eventid 20 formerly
>> + * attached to it does not hit anymore
>> + */
>> + report_prefix_push("mapd valid=false");
>
> Above you have the prefix-push before the comment explaining the test.
> After is probably better, but whatever, as long as it's consistent.
moved after
>
>> + its_send_mapd(dev2, false);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> + report_prefix_pop();
>> +
>> + /* Unmap the collection this time and check no LPI does hit */
>> + report_prefix_push("mapc valid=false");
>> + its_send_mapc(col2, false);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> + report_prefix_pop();
>> +}
>> #endif
>>
>> int main(int argc, char **argv)
>> @@ -594,6 +796,10 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> gic_test_mmio();
>> report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-trigger")) {
>> + report_prefix_push(argv[1]);
>> + test_its_trigger();
>> + report_prefix_pop();
>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>> report_prefix_push(argv[1]);
>> test_its_introspection();
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index ba2b31b..bfafec5 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
>> groups = its
>> arch = arm64
>>
>> +[its-trigger]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +extra_params = -machine gic-version=3 -append 'its-trigger'
>> +groups = its
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> --
>> 2.20.1
>>
>
> Thanks,
> drew
>
Thanks
Eric
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
@ 2020-03-06 12:55 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 12:55 UTC (permalink / raw)
To: Andrew Jones
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 2:15 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:56AM +0100, Eric Auger wrote:
>> Triggers LPIs through the INT command.
>>
>> the test checks the LPI hits the right CPU and triggers
>> the right LPI intid, ie. the translation is correct.
>>
>> Updates to the config table also are tested, along with inv
>> and invall commands.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - add comments
>> - keep the report_skip in case there aren't 4 vcpus to be able to
>> run other tests in the its category.
>> - fix the prefix pop
>> - move its_event and its_stats to arm/gic.c
>> ---
>> arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
>> arm/unittests.cfg | 7 ++
>> 2 files changed, 224 insertions(+), 11 deletions(-)
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index 4d7dd03..50104b1 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
>> }
>> }
>>
>> +static void setup_irq(handler_t handler)
>> +{
>> + gic_enable_defaults();
>> +#ifdef __arm__
>> + install_exception_handler(EXCPTN_IRQ, handler);
>> +#else
>> + install_irq_handler(EL1H_IRQ, handler);
>> +#endif
>> + local_irq_enable();
>> +}
>> +
>> +#if defined(__aarch64__)
>> +struct its_event {
>> + int cpu_id;
>> + int lpi_id;
>> +};
>> +
>> +struct its_stats {
>> + struct its_event expected;
>> + struct its_event observed;
>> +};
>> +
>> +static struct its_stats lpi_stats;
>> +
>> +static void lpi_handler(struct pt_regs *regs __unused)
>> +{
>> + u32 irqstat = gic_read_iar();
>> + int irqnr = gic_iar_irqnr(irqstat);
>> +
>> + gic_write_eoir(irqstat);
>> + if (irqnr < 8192)
>> + report(false, "Unexpected non LPI interrupt received");
>
> report_info
why? This is an error case. We do not expect other interrupts than LPIs
>
>> + smp_rmb(); /* pairs with wmb in lpi_stats_expect */
>> + lpi_stats.observed.cpu_id = smp_processor_id();
>> + lpi_stats.observed.lpi_id = irqnr;
>> + smp_wmb(); /* pairs with rmb in check_lpi_stats */
>> +}
>> +
>> +static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
>> +{
>> + lpi_stats.expected.cpu_id = exp_cpu_id;
>> + lpi_stats.expected.lpi_id = exp_lpi_id;
>> + lpi_stats.observed.cpu_id = -1;
>> + lpi_stats.observed.lpi_id = -1;
>> + smp_wmb(); /* pairs with rmb in handler */
>> +}
>> +
>> +static void check_lpi_stats(void)
>
> static void check_lpi_stats(const char *testname)
> {
> bool pass = false;
>
>> +{
>> + mdelay(100);
>> + smp_rmb(); /* pairs with wmb in lpi_handler */
>> + if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
>> + (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
>
> nit: extra ()
>
>> + if (lpi_stats.observed.cpu_id == -1 &&
>> + lpi_stats.observed.lpi_id == -1) {
>> + report(false,
>> + "No LPI received whereas (cpuid=%d, intid=%d) "
>> + "was expected", lpi_stats.expected.cpu_id,
>> + lpi_stats.expected.lpi_id);
>
> report_info
What's the problem keeping those. Those are error reports. The message
is something like that:
FAIL: gicv3: its-trigger: mapc valid=false: No LPI received whereas
(cpuid=1, intid=8192) was expected.
So the testname is already part of the message.
>
>> + } else {
>> + report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
>> + lpi_stats.observed.cpu_id,
>> + lpi_stats.observed.lpi_id);
>
> report_info
>
>> + }
>
> pass = false;
>
>> + } else if (lpi_stats.expected.lpi_id != -1) {
>> + report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
>> + lpi_stats.observed.cpu_id);
>
> report_info
>
>> + } else {
>> + report(true, "no LPI received, as expected");
>
> report_info
>
>
>> + }
>
> report(pass, "%s", testname);
>
>> +}
>> +
>> +static void secondary_lpi_test(void)
>> +{
>> + setup_irq(lpi_handler);
>> + cpumask_set_cpu(smp_processor_id(), &ready);
>> + while (1)
>> + wfi();
>> +}
>> +#endif
>> +
>> static void gicv2_ipi_send_self(void)
>> {
>> writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
>> @@ -217,17 +298,6 @@ static void ipi_test_smp(void)
>> report_prefix_pop();
>> }
>>
>> -static void setup_irq(handler_t handler)
>> -{
>> - gic_enable_defaults();
>> -#ifdef __arm__
>> - install_exception_handler(EXCPTN_IRQ, handler);
>> -#else
>> - install_irq_handler(EL1H_IRQ, handler);
>> -#endif
>> - local_irq_enable();
>> -}
>> -
>> static void ipi_send(void)
>> {
>> setup_irq(ipi_handler);
>> @@ -522,6 +592,7 @@ static void gic_test_mmio(void)
>> #if defined(__arm__)
>>
>> static void test_its_introspection(void) {}
>> +static void test_its_trigger(void) {}
>>
>> #else /* __arch64__ */
>>
>> @@ -561,6 +632,137 @@ static void test_its_introspection(void)
>> report_info("collection baser entry_size = 0x%x", coll_baser->esz);
>> }
>>
>> +static bool its_prerequisites(int nb_cpus)
>> +{
>> + int cpu;
>> +
>> + if (!gicv3_its_base()) {
>> + report_skip("No ITS, skip ...");
>> + return true;
>> + }
>> +
>> + if (nr_cpus < 4) {
>
> nr_cpus < nb_cpus, or just drop the nb_cpus parameter and hard code 4
> here.
sure
>
>> + report_skip("Test requires at least %d vcpus", nb_cpus);
>> + return true;
>> + }
>> +
>> + stats_reset();
>> +
>> + setup_irq(lpi_handler);
>> +
>> + for_each_present_cpu(cpu) {
>> + if (cpu == 0)
>> + continue;
>> + smp_boot_secondary(cpu, secondary_lpi_test);
>> + }
>> + wait_on_ready();
>> +
>> + its_enable_defaults();
>> +
>> + lpi_stats_expect(-1, -1);
>> + check_lpi_stats();
>> +
>> + return false;
>
> Reverse logic. I'd expect 'return true' for success.
I am going to return an int. In case of error a std negative error will
be returned.
>
>> +}
>> +
>> +static void test_its_trigger(void)
>> +{
>> + struct its_collection *col3, *col2;
>> + struct its_device *dev2, *dev7;
>> +
>> + if (its_prerequisites(4))
>
> if (!its_prerequisites(...))
>
>> + return;
>> +
>> + dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>> + dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
>> +
>> + col3 = its_create_collection(3 /* col id */, 3/* target PE */);
>> + col2 = its_create_collection(2 /* col id */, 2/* target PE */);
>> +
>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>> + gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
>> +
>> + its_send_invall(col2);
>> + its_send_invall(col3);
>> +
>> + report_prefix_push("int");
>> + /*
>> + * dev=2, eventid=20 -> lpi= 8195, col=3
>> + * dev=7, eventid=255 -> lpi= 8196, col=2
>> + * Trigger dev2, eventid=20 and dev7, eventid=255
>> + * Check both LPIs hit
>> + */
>> +
>> + its_send_mapd(dev2, true);
>> + its_send_mapd(dev7, true);
>> +
>> + its_send_mapc(col3, true);
>> + its_send_mapc(col2, true);
>> +
>> + its_send_mapti(dev2, 8195 /* lpi id */,
>> + 20 /* event id */, col3);
>> + its_send_mapti(dev7, 8196 /* lpi id */,
>> + 255 /* event id */, col2);
>
> No need for line breaks, with the embedded comments it's hard to read
OK
>
>> +
>> + lpi_stats_expect(3, 8195);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + lpi_stats_expect(2, 8196);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> +
>> + report_prefix_pop();
>
> I think a table of parameters and loop would be nicer than all the
> repeated function calls.
Frankly speaking I am not sure this would really help. We are just
enabling 2 translation paths. I think I prefer to manipulate the low
level objects and helpers rather than playing with a loop and potential
new structs of params.
>
>> +
>> + report_prefix_push("inv/invall");
>> +
>> + /*
>> + * disable 8195, check dev2/eventid=20 does not trigger the
>> + * corresponding LPI
>> + */
>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
>
> LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED
ok
>
>> + its_send_inv(dev2, 20);
>> +
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + /*
>> + * re-enable the LPI but willingly do not call invall
>> + * so the change in config is not taken into account.
>> + * The LPI should not hit
>> + */
>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + /* Now call the invall and check the LPI hits */
>> + its_send_invall(col3);
>> + lpi_stats_expect(3, 8195);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + report_prefix_pop();
>
> Need blank line here.
OK
>
>> + /*
>> + * Unmap device 2 and check the eventid 20 formerly
>> + * attached to it does not hit anymore
>> + */
>> + report_prefix_push("mapd valid=false");
>
> Above you have the prefix-push before the comment explaining the test.
> After is probably better, but whatever, as long as it's consistent.
moved after
>
>> + its_send_mapd(dev2, false);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> + report_prefix_pop();
>> +
>> + /* Unmap the collection this time and check no LPI does hit */
>> + report_prefix_push("mapc valid=false");
>> + its_send_mapc(col2, false);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> + report_prefix_pop();
>> +}
>> #endif
>>
>> int main(int argc, char **argv)
>> @@ -594,6 +796,10 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> gic_test_mmio();
>> report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-trigger")) {
>> + report_prefix_push(argv[1]);
>> + test_its_trigger();
>> + report_prefix_pop();
>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>> report_prefix_push(argv[1]);
>> test_its_introspection();
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index ba2b31b..bfafec5 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
>> groups = its
>> arch = arm64
>>
>> +[its-trigger]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +extra_params = -machine gic-version=3 -append 'its-trigger'
>> +groups = its
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> --
>> 2.20.1
>>
>
> Thanks,
> drew
>
Thanks
Eric
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
@ 2020-03-06 12:55 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 12:55 UTC (permalink / raw)
To: Andrew Jones
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 2:15 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:56AM +0100, Eric Auger wrote:
>> Triggers LPIs through the INT command.
>>
>> the test checks the LPI hits the right CPU and triggers
>> the right LPI intid, ie. the translation is correct.
>>
>> Updates to the config table also are tested, along with inv
>> and invall commands.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - add comments
>> - keep the report_skip in case there aren't 4 vcpus to be able to
>> run other tests in the its category.
>> - fix the prefix pop
>> - move its_event and its_stats to arm/gic.c
>> ---
>> arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
>> arm/unittests.cfg | 7 ++
>> 2 files changed, 224 insertions(+), 11 deletions(-)
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index 4d7dd03..50104b1 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
>> }
>> }
>>
>> +static void setup_irq(handler_t handler)
>> +{
>> + gic_enable_defaults();
>> +#ifdef __arm__
>> + install_exception_handler(EXCPTN_IRQ, handler);
>> +#else
>> + install_irq_handler(EL1H_IRQ, handler);
>> +#endif
>> + local_irq_enable();
>> +}
>> +
>> +#if defined(__aarch64__)
>> +struct its_event {
>> + int cpu_id;
>> + int lpi_id;
>> +};
>> +
>> +struct its_stats {
>> + struct its_event expected;
>> + struct its_event observed;
>> +};
>> +
>> +static struct its_stats lpi_stats;
>> +
>> +static void lpi_handler(struct pt_regs *regs __unused)
>> +{
>> + u32 irqstat = gic_read_iar();
>> + int irqnr = gic_iar_irqnr(irqstat);
>> +
>> + gic_write_eoir(irqstat);
>> + if (irqnr < 8192)
>> + report(false, "Unexpected non LPI interrupt received");
>
> report_info
why? This is an error case. We do not expect other interrupts than LPIs
>
>> + smp_rmb(); /* pairs with wmb in lpi_stats_expect */
>> + lpi_stats.observed.cpu_id = smp_processor_id();
>> + lpi_stats.observed.lpi_id = irqnr;
>> + smp_wmb(); /* pairs with rmb in check_lpi_stats */
>> +}
>> +
>> +static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
>> +{
>> + lpi_stats.expected.cpu_id = exp_cpu_id;
>> + lpi_stats.expected.lpi_id = exp_lpi_id;
>> + lpi_stats.observed.cpu_id = -1;
>> + lpi_stats.observed.lpi_id = -1;
>> + smp_wmb(); /* pairs with rmb in handler */
>> +}
>> +
>> +static void check_lpi_stats(void)
>
> static void check_lpi_stats(const char *testname)
> {
> bool pass = false;
>
>> +{
>> + mdelay(100);
>> + smp_rmb(); /* pairs with wmb in lpi_handler */
>> + if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
>> + (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
>
> nit: extra ()
>
>> + if (lpi_stats.observed.cpu_id == -1 &&
>> + lpi_stats.observed.lpi_id == -1) {
>> + report(false,
>> + "No LPI received whereas (cpuid=%d, intid=%d) "
>> + "was expected", lpi_stats.expected.cpu_id,
>> + lpi_stats.expected.lpi_id);
>
> report_info
What's the problem keeping those. Those are error reports. The message
is something like that:
FAIL: gicv3: its-trigger: mapc valid=false: No LPI received whereas
(cpuid=1, intid=8192) was expected.
So the testname is already part of the message.
>
>> + } else {
>> + report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
>> + lpi_stats.observed.cpu_id,
>> + lpi_stats.observed.lpi_id);
>
> report_info
>
>> + }
>
> pass = false;
>
>> + } else if (lpi_stats.expected.lpi_id != -1) {
>> + report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
>> + lpi_stats.observed.cpu_id);
>
> report_info
>
>> + } else {
>> + report(true, "no LPI received, as expected");
>
> report_info
>
>
>> + }
>
> report(pass, "%s", testname);
>
>> +}
>> +
>> +static void secondary_lpi_test(void)
>> +{
>> + setup_irq(lpi_handler);
>> + cpumask_set_cpu(smp_processor_id(), &ready);
>> + while (1)
>> + wfi();
>> +}
>> +#endif
>> +
>> static void gicv2_ipi_send_self(void)
>> {
>> writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
>> @@ -217,17 +298,6 @@ static void ipi_test_smp(void)
>> report_prefix_pop();
>> }
>>
>> -static void setup_irq(handler_t handler)
>> -{
>> - gic_enable_defaults();
>> -#ifdef __arm__
>> - install_exception_handler(EXCPTN_IRQ, handler);
>> -#else
>> - install_irq_handler(EL1H_IRQ, handler);
>> -#endif
>> - local_irq_enable();
>> -}
>> -
>> static void ipi_send(void)
>> {
>> setup_irq(ipi_handler);
>> @@ -522,6 +592,7 @@ static void gic_test_mmio(void)
>> #if defined(__arm__)
>>
>> static void test_its_introspection(void) {}
>> +static void test_its_trigger(void) {}
>>
>> #else /* __arch64__ */
>>
>> @@ -561,6 +632,137 @@ static void test_its_introspection(void)
>> report_info("collection baser entry_size = 0x%x", coll_baser->esz);
>> }
>>
>> +static bool its_prerequisites(int nb_cpus)
>> +{
>> + int cpu;
>> +
>> + if (!gicv3_its_base()) {
>> + report_skip("No ITS, skip ...");
>> + return true;
>> + }
>> +
>> + if (nr_cpus < 4) {
>
> nr_cpus < nb_cpus, or just drop the nb_cpus parameter and hard code 4
> here.
sure
>
>> + report_skip("Test requires at least %d vcpus", nb_cpus);
>> + return true;
>> + }
>> +
>> + stats_reset();
>> +
>> + setup_irq(lpi_handler);
>> +
>> + for_each_present_cpu(cpu) {
>> + if (cpu == 0)
>> + continue;
>> + smp_boot_secondary(cpu, secondary_lpi_test);
>> + }
>> + wait_on_ready();
>> +
>> + its_enable_defaults();
>> +
>> + lpi_stats_expect(-1, -1);
>> + check_lpi_stats();
>> +
>> + return false;
>
> Reverse logic. I'd expect 'return true' for success.
I am going to return an int. In case of error a std negative error will
be returned.
>
>> +}
>> +
>> +static void test_its_trigger(void)
>> +{
>> + struct its_collection *col3, *col2;
>> + struct its_device *dev2, *dev7;
>> +
>> + if (its_prerequisites(4))
>
> if (!its_prerequisites(...))
>
>> + return;
>> +
>> + dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>> + dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
>> +
>> + col3 = its_create_collection(3 /* col id */, 3/* target PE */);
>> + col2 = its_create_collection(2 /* col id */, 2/* target PE */);
>> +
>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>> + gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
>> +
>> + its_send_invall(col2);
>> + its_send_invall(col3);
>> +
>> + report_prefix_push("int");
>> + /*
>> + * dev=2, eventid=20 -> lpi= 8195, col=3
>> + * dev=7, eventid=255 -> lpi= 8196, col=2
>> + * Trigger dev2, eventid=20 and dev7, eventid=255
>> + * Check both LPIs hit
>> + */
>> +
>> + its_send_mapd(dev2, true);
>> + its_send_mapd(dev7, true);
>> +
>> + its_send_mapc(col3, true);
>> + its_send_mapc(col2, true);
>> +
>> + its_send_mapti(dev2, 8195 /* lpi id */,
>> + 20 /* event id */, col3);
>> + its_send_mapti(dev7, 8196 /* lpi id */,
>> + 255 /* event id */, col2);
>
> No need for line breaks, with the embedded comments it's hard to read
OK
>
>> +
>> + lpi_stats_expect(3, 8195);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + lpi_stats_expect(2, 8196);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> +
>> + report_prefix_pop();
>
> I think a table of parameters and loop would be nicer than all the
> repeated function calls.
Frankly speaking I am not sure this would really help. We are just
enabling 2 translation paths. I think I prefer to manipulate the low
level objects and helpers rather than playing with a loop and potential
new structs of params.
>
>> +
>> + report_prefix_push("inv/invall");
>> +
>> + /*
>> + * disable 8195, check dev2/eventid=20 does not trigger the
>> + * corresponding LPI
>> + */
>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
>
> LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED
ok
>
>> + its_send_inv(dev2, 20);
>> +
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + /*
>> + * re-enable the LPI but willingly do not call invall
>> + * so the change in config is not taken into account.
>> + * The LPI should not hit
>> + */
>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + /* Now call the invall and check the LPI hits */
>> + its_send_invall(col3);
>> + lpi_stats_expect(3, 8195);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + report_prefix_pop();
>
> Need blank line here.
OK
>
>> + /*
>> + * Unmap device 2 and check the eventid 20 formerly
>> + * attached to it does not hit anymore
>> + */
>> + report_prefix_push("mapd valid=false");
>
> Above you have the prefix-push before the comment explaining the test.
> After is probably better, but whatever, as long as it's consistent.
moved after
>
>> + its_send_mapd(dev2, false);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> + report_prefix_pop();
>> +
>> + /* Unmap the collection this time and check no LPI does hit */
>> + report_prefix_push("mapc valid=false");
>> + its_send_mapc(col2, false);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> + report_prefix_pop();
>> +}
>> #endif
>>
>> int main(int argc, char **argv)
>> @@ -594,6 +796,10 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> gic_test_mmio();
>> report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-trigger")) {
>> + report_prefix_push(argv[1]);
>> + test_its_trigger();
>> + report_prefix_pop();
>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>> report_prefix_push(argv[1]);
>> test_its_introspection();
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index ba2b31b..bfafec5 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
>> groups = its
>> arch = arm64
>>
>> +[its-trigger]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +extra_params = -machine gic-version=3 -append 'its-trigger'
>> +groups = its
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> --
>> 2.20.1
>>
>
> Thanks,
> drew
>
Thanks
Eric
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 13/14] arm/arm64: ITS: migration tests
2020-02-07 13:49 ` Andrew Jones
(?)
@ 2020-03-06 13:06 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 13:06 UTC (permalink / raw)
To: Andrew Jones
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
andre.przywara, peter.maydell, yuzenghui, alexandru.elisei,
thuth
Hi Drew,
On 2/7/20 2:49 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:58AM +0100, Eric Auger wrote:
>> This test maps LPIs (populates the device table, the collection table,
>> interrupt translation tables, configuration table), migrates and make
>> sure the translation is correct on the destination.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>> ---
>> arm/gic.c | 59 ++++++++++++++++++++++++++++++++++++----
>> arm/unittests.cfg | 8 ++++++
>> lib/arm/asm/gic-v3-its.h | 2 ++
>> lib/arm/gic-v3-its.c | 22 +++++++++++++++
>> 4 files changed, 85 insertions(+), 6 deletions(-)
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index 50104b1..fa8626a 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -593,6 +593,7 @@ static void gic_test_mmio(void)
>>
>> static void test_its_introspection(void) {}
>> static void test_its_trigger(void) {}
>> +static void test_its_migration(void) {}
>>
>> #else /* __arch64__ */
>>
>> @@ -665,13 +666,19 @@ static bool its_prerequisites(int nb_cpus)
>> return false;
>> }
>>
>> -static void test_its_trigger(void)
>> +/*
>> + * Setup the configuration for those mappings:
>> + * dev_id=2 event=20 -> vcpu 3, intid=8195
>> + * dev_id=7 event=255 -> vcpu 2, intid=8196
>> + * LPIs ready to hit
>> + */
>> +static int its_setup1(void)
>> {
>> struct its_collection *col3, *col2;
>> struct its_device *dev2, *dev7;
>>
>> if (its_prerequisites(4))
>> - return;
>> + return -1;
>
> Why not make its_setup1 a bool? Where true means success and false mean
> failure?
I tend to prefer the std error return value convention that the bool
return value. I aligned its_prerequisites accordingly.
>
>>
>> dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>> dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
>> @@ -685,14 +692,10 @@ static void test_its_trigger(void)
>> its_send_invall(col2);
>> its_send_invall(col3);
>>
>> - report_prefix_push("int");
>> /*
>> * dev=2, eventid=20 -> lpi= 8195, col=3
>> * dev=7, eventid=255 -> lpi= 8196, col=2
>> - * Trigger dev2, eventid=20 and dev7, eventid=255
>> - * Check both LPIs hit
>> */
>> -
>> its_send_mapd(dev2, true);
>> its_send_mapd(dev7, true);
>>
>> @@ -703,6 +706,23 @@ static void test_its_trigger(void)
>> 20 /* event id */, col3);
>> its_send_mapti(dev7, 8196 /* lpi id */,
>> 255 /* event id */, col2);
>> + return 0;
>> +}
>> +
>> +static void test_its_trigger(void)
>> +{
>> + struct its_collection *col3, *col2;
>> + struct its_device *dev2, *dev7;
>> +
>> + if (its_setup1())
>> + return;
>> +
>> + col3 = its_get_collection(3);
>> + col2 = its_get_collection(2);
>> + dev2 = its_get_device(2);
>> + dev7 = its_get_device(7);
>> +
>> + report_prefix_push("int");
>>
>> lpi_stats_expect(3, 8195);
>> its_send_int(dev2, 20);
>> @@ -763,6 +783,29 @@ static void test_its_trigger(void)
>> check_lpi_stats();
>> report_prefix_pop();
>> }
>> +
>> +static void test_its_migration(void)
>> +{
>> + struct its_device *dev2, *dev7;
>> +
>> + if (its_setup1())
>> + return;
>> +
>> + dev2 = its_get_device(2);
>> + dev7 = its_get_device(7);
>> +
>> + puts("Now migrate the VM, then press a key to continue...\n");
>> + (void)getchar();
>> + report(true, "Migration complete");
>
> This seems more like a report_info place. If migration fails and
> we don't complete we'll never get the report FAIL anyway.
OK
>
>> +
>> + lpi_stats_expect(3, 8195);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + lpi_stats_expect(2, 8196);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> +}
>> #endif
>>
>> int main(int argc, char **argv)
>> @@ -800,6 +843,10 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> test_its_trigger();
>> report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-migration")) {
>> + report_prefix_push(argv[1]);
>> + test_its_migration();
>> + report_prefix_pop();
>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>> report_prefix_push(argv[1]);
>> test_its_introspection();
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index bfafec5..8b8ec79 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
>> groups = its
>> arch = arm64
>>
>> +[its-migration]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +accel = kvm
>> +extra_params = -machine gic-version=3 -append 'its-migration'
>> +groups = its migration
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index 0e5c5b6..febc2b2 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -151,6 +151,8 @@ extern void its_send_invall(struct its_collection *col);
>> extern void its_send_movi(struct its_device *dev,
>> struct its_collection *col, u32 id);
>> extern void its_send_sync(struct its_collection *col);
>> +extern struct its_device *its_get_device(u32 id);
>> +extern struct its_collection *its_get_collection(u32 id);
>>
>> #define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
>> #define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index c2dcd01..099940e 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -219,3 +219,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
>> its_data.nr_collections++;
>> return new;
>> }
>> +
>> +struct its_device *its_get_device(u32 id)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < GITS_MAX_DEVICES; i++) {
>> + if (its_data.devices[i].device_id == id)
>> + return &its_data.devices[i];
>> + }
>> + return NULL;
>> +}
>> +
>> +struct its_collection *its_get_collection(u32 id)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
>> + if (its_data.collections[i].col_id == id)
>> + return &its_data.collections[i];
>> + }
>> + return NULL;
>> +}
>
> The callers aren't checking for NULL. Should we assert here
> or in the caller?
Added the assert here.
Thanks
Eric
>
> Thanks,
> drew
>
>
>> --
>> 2.20.1
>>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 13/14] arm/arm64: ITS: migration tests
@ 2020-03-06 13:06 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 13:06 UTC (permalink / raw)
To: Andrew Jones
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 2:49 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:58AM +0100, Eric Auger wrote:
>> This test maps LPIs (populates the device table, the collection table,
>> interrupt translation tables, configuration table), migrates and make
>> sure the translation is correct on the destination.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>> ---
>> arm/gic.c | 59 ++++++++++++++++++++++++++++++++++++----
>> arm/unittests.cfg | 8 ++++++
>> lib/arm/asm/gic-v3-its.h | 2 ++
>> lib/arm/gic-v3-its.c | 22 +++++++++++++++
>> 4 files changed, 85 insertions(+), 6 deletions(-)
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index 50104b1..fa8626a 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -593,6 +593,7 @@ static void gic_test_mmio(void)
>>
>> static void test_its_introspection(void) {}
>> static void test_its_trigger(void) {}
>> +static void test_its_migration(void) {}
>>
>> #else /* __arch64__ */
>>
>> @@ -665,13 +666,19 @@ static bool its_prerequisites(int nb_cpus)
>> return false;
>> }
>>
>> -static void test_its_trigger(void)
>> +/*
>> + * Setup the configuration for those mappings:
>> + * dev_id=2 event=20 -> vcpu 3, intid=8195
>> + * dev_id=7 event=255 -> vcpu 2, intid=8196
>> + * LPIs ready to hit
>> + */
>> +static int its_setup1(void)
>> {
>> struct its_collection *col3, *col2;
>> struct its_device *dev2, *dev7;
>>
>> if (its_prerequisites(4))
>> - return;
>> + return -1;
>
> Why not make its_setup1 a bool? Where true means success and false mean
> failure?
I tend to prefer the std error return value convention that the bool
return value. I aligned its_prerequisites accordingly.
>
>>
>> dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>> dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
>> @@ -685,14 +692,10 @@ static void test_its_trigger(void)
>> its_send_invall(col2);
>> its_send_invall(col3);
>>
>> - report_prefix_push("int");
>> /*
>> * dev=2, eventid=20 -> lpi= 8195, col=3
>> * dev=7, eventid=255 -> lpi= 8196, col=2
>> - * Trigger dev2, eventid=20 and dev7, eventid=255
>> - * Check both LPIs hit
>> */
>> -
>> its_send_mapd(dev2, true);
>> its_send_mapd(dev7, true);
>>
>> @@ -703,6 +706,23 @@ static void test_its_trigger(void)
>> 20 /* event id */, col3);
>> its_send_mapti(dev7, 8196 /* lpi id */,
>> 255 /* event id */, col2);
>> + return 0;
>> +}
>> +
>> +static void test_its_trigger(void)
>> +{
>> + struct its_collection *col3, *col2;
>> + struct its_device *dev2, *dev7;
>> +
>> + if (its_setup1())
>> + return;
>> +
>> + col3 = its_get_collection(3);
>> + col2 = its_get_collection(2);
>> + dev2 = its_get_device(2);
>> + dev7 = its_get_device(7);
>> +
>> + report_prefix_push("int");
>>
>> lpi_stats_expect(3, 8195);
>> its_send_int(dev2, 20);
>> @@ -763,6 +783,29 @@ static void test_its_trigger(void)
>> check_lpi_stats();
>> report_prefix_pop();
>> }
>> +
>> +static void test_its_migration(void)
>> +{
>> + struct its_device *dev2, *dev7;
>> +
>> + if (its_setup1())
>> + return;
>> +
>> + dev2 = its_get_device(2);
>> + dev7 = its_get_device(7);
>> +
>> + puts("Now migrate the VM, then press a key to continue...\n");
>> + (void)getchar();
>> + report(true, "Migration complete");
>
> This seems more like a report_info place. If migration fails and
> we don't complete we'll never get the report FAIL anyway.
OK
>
>> +
>> + lpi_stats_expect(3, 8195);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + lpi_stats_expect(2, 8196);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> +}
>> #endif
>>
>> int main(int argc, char **argv)
>> @@ -800,6 +843,10 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> test_its_trigger();
>> report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-migration")) {
>> + report_prefix_push(argv[1]);
>> + test_its_migration();
>> + report_prefix_pop();
>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>> report_prefix_push(argv[1]);
>> test_its_introspection();
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index bfafec5..8b8ec79 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
>> groups = its
>> arch = arm64
>>
>> +[its-migration]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +accel = kvm
>> +extra_params = -machine gic-version=3 -append 'its-migration'
>> +groups = its migration
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index 0e5c5b6..febc2b2 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -151,6 +151,8 @@ extern void its_send_invall(struct its_collection *col);
>> extern void its_send_movi(struct its_device *dev,
>> struct its_collection *col, u32 id);
>> extern void its_send_sync(struct its_collection *col);
>> +extern struct its_device *its_get_device(u32 id);
>> +extern struct its_collection *its_get_collection(u32 id);
>>
>> #define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
>> #define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index c2dcd01..099940e 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -219,3 +219,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
>> its_data.nr_collections++;
>> return new;
>> }
>> +
>> +struct its_device *its_get_device(u32 id)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < GITS_MAX_DEVICES; i++) {
>> + if (its_data.devices[i].device_id == id)
>> + return &its_data.devices[i];
>> + }
>> + return NULL;
>> +}
>> +
>> +struct its_collection *its_get_collection(u32 id)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
>> + if (its_data.collections[i].col_id == id)
>> + return &its_data.collections[i];
>> + }
>> + return NULL;
>> +}
>
> The callers aren't checking for NULL. Should we assert here
> or in the caller?
Added the assert here.
Thanks
Eric
>
> Thanks,
> drew
>
>
>> --
>> 2.20.1
>>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 13/14] arm/arm64: ITS: migration tests
@ 2020-03-06 13:06 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 13:06 UTC (permalink / raw)
To: Andrew Jones
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 2:49 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:58AM +0100, Eric Auger wrote:
>> This test maps LPIs (populates the device table, the collection table,
>> interrupt translation tables, configuration table), migrates and make
>> sure the translation is correct on the destination.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>> ---
>> arm/gic.c | 59 ++++++++++++++++++++++++++++++++++++----
>> arm/unittests.cfg | 8 ++++++
>> lib/arm/asm/gic-v3-its.h | 2 ++
>> lib/arm/gic-v3-its.c | 22 +++++++++++++++
>> 4 files changed, 85 insertions(+), 6 deletions(-)
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index 50104b1..fa8626a 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -593,6 +593,7 @@ static void gic_test_mmio(void)
>>
>> static void test_its_introspection(void) {}
>> static void test_its_trigger(void) {}
>> +static void test_its_migration(void) {}
>>
>> #else /* __arch64__ */
>>
>> @@ -665,13 +666,19 @@ static bool its_prerequisites(int nb_cpus)
>> return false;
>> }
>>
>> -static void test_its_trigger(void)
>> +/*
>> + * Setup the configuration for those mappings:
>> + * dev_id=2 event=20 -> vcpu 3, intid=8195
>> + * dev_id=7 event=255 -> vcpu 2, intid=8196
>> + * LPIs ready to hit
>> + */
>> +static int its_setup1(void)
>> {
>> struct its_collection *col3, *col2;
>> struct its_device *dev2, *dev7;
>>
>> if (its_prerequisites(4))
>> - return;
>> + return -1;
>
> Why not make its_setup1 a bool? Where true means success and false mean
> failure?
I tend to prefer the std error return value convention that the bool
return value. I aligned its_prerequisites accordingly.
>
>>
>> dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>> dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
>> @@ -685,14 +692,10 @@ static void test_its_trigger(void)
>> its_send_invall(col2);
>> its_send_invall(col3);
>>
>> - report_prefix_push("int");
>> /*
>> * dev=2, eventid=20 -> lpi= 8195, col=3
>> * dev=7, eventid=255 -> lpi= 8196, col=2
>> - * Trigger dev2, eventid=20 and dev7, eventid=255
>> - * Check both LPIs hit
>> */
>> -
>> its_send_mapd(dev2, true);
>> its_send_mapd(dev7, true);
>>
>> @@ -703,6 +706,23 @@ static void test_its_trigger(void)
>> 20 /* event id */, col3);
>> its_send_mapti(dev7, 8196 /* lpi id */,
>> 255 /* event id */, col2);
>> + return 0;
>> +}
>> +
>> +static void test_its_trigger(void)
>> +{
>> + struct its_collection *col3, *col2;
>> + struct its_device *dev2, *dev7;
>> +
>> + if (its_setup1())
>> + return;
>> +
>> + col3 = its_get_collection(3);
>> + col2 = its_get_collection(2);
>> + dev2 = its_get_device(2);
>> + dev7 = its_get_device(7);
>> +
>> + report_prefix_push("int");
>>
>> lpi_stats_expect(3, 8195);
>> its_send_int(dev2, 20);
>> @@ -763,6 +783,29 @@ static void test_its_trigger(void)
>> check_lpi_stats();
>> report_prefix_pop();
>> }
>> +
>> +static void test_its_migration(void)
>> +{
>> + struct its_device *dev2, *dev7;
>> +
>> + if (its_setup1())
>> + return;
>> +
>> + dev2 = its_get_device(2);
>> + dev7 = its_get_device(7);
>> +
>> + puts("Now migrate the VM, then press a key to continue...\n");
>> + (void)getchar();
>> + report(true, "Migration complete");
>
> This seems more like a report_info place. If migration fails and
> we don't complete we'll never get the report FAIL anyway.
OK
>
>> +
>> + lpi_stats_expect(3, 8195);
>> + its_send_int(dev2, 20);
>> + check_lpi_stats();
>> +
>> + lpi_stats_expect(2, 8196);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> +}
>> #endif
>>
>> int main(int argc, char **argv)
>> @@ -800,6 +843,10 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> test_its_trigger();
>> report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-migration")) {
>> + report_prefix_push(argv[1]);
>> + test_its_migration();
>> + report_prefix_pop();
>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>> report_prefix_push(argv[1]);
>> test_its_introspection();
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index bfafec5..8b8ec79 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
>> groups = its
>> arch = arm64
>>
>> +[its-migration]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +accel = kvm
>> +extra_params = -machine gic-version=3 -append 'its-migration'
>> +groups = its migration
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
>> index 0e5c5b6..febc2b2 100644
>> --- a/lib/arm/asm/gic-v3-its.h
>> +++ b/lib/arm/asm/gic-v3-its.h
>> @@ -151,6 +151,8 @@ extern void its_send_invall(struct its_collection *col);
>> extern void its_send_movi(struct its_device *dev,
>> struct its_collection *col, u32 id);
>> extern void its_send_sync(struct its_collection *col);
>> +extern struct its_device *its_get_device(u32 id);
>> +extern struct its_collection *its_get_collection(u32 id);
>>
>> #define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)
>> #define ITS_FLAGS_WORKAROUND_CAVIUM_22375 (1ULL << 1)
>> diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
>> index c2dcd01..099940e 100644
>> --- a/lib/arm/gic-v3-its.c
>> +++ b/lib/arm/gic-v3-its.c
>> @@ -219,3 +219,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
>> its_data.nr_collections++;
>> return new;
>> }
>> +
>> +struct its_device *its_get_device(u32 id)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < GITS_MAX_DEVICES; i++) {
>> + if (its_data.devices[i].device_id == id)
>> + return &its_data.devices[i];
>> + }
>> + return NULL;
>> +}
>> +
>> +struct its_collection *its_get_collection(u32 id)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
>> + if (its_data.collections[i].col_id == id)
>> + return &its_data.collections[i];
>> + }
>> + return NULL;
>> +}
>
> The callers aren't checking for NULL. Should we assert here
> or in the caller?
Added the assert here.
Thanks
Eric
>
> Thanks,
> drew
>
>
>> --
>> 2.20.1
>>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
2020-02-07 14:06 ` Andrew Jones
(?)
@ 2020-03-06 13:21 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 13:21 UTC (permalink / raw)
To: Andrew Jones
Cc: eric.auger.pro, maz, kvmarm, kvm, qemu-devel, qemu-arm,
peter.maydell, andre.przywara, thuth, yuzenghui,
alexandru.elisei
Hi Drew,
On 2/7/20 3:06 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:59AM +0100, Eric Auger wrote:
>> Add two new migration tests. One testing the migration of
>> a topology where collection were unmapped. The second test
>> checks the migration of the pending table.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - tests belong to both its and migration groups
>> ---
>> arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
>> arm/unittests.cfg | 16 +++++
>> 2 files changed, 166 insertions(+)
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index fa8626a..ec3dd3a 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
>> smp_rmb(); /* pairs with wmb in lpi_stats_expect */
>> lpi_stats.observed.cpu_id = smp_processor_id();
>> lpi_stats.observed.lpi_id = irqnr;
>> + acked[lpi_stats.observed.cpu_id]++;
>> smp_wmb(); /* pairs with rmb in check_lpi_stats */
>> }
>>
>> @@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
>> while (1)
>> wfi();
>> }
>> +
>> +static void check_lpi_hits(int *expected)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < nr_cpus; i++) {
>> + if (acked[i] != expected[i])
>> + report(false, "expected %d LPIs on PE #%d, %d observed",
>> + expected[i], i, acked[i]);
>
> report_info
> pass = false
>
>> + }
>> + report(true, "check LPI on all vcpus");
>
> report(pass, ...);
I still don't get the issue.
>
>> +}
>> #endif
>>
>> static void gicv2_ipi_send_self(void)
>> @@ -594,6 +607,8 @@ static void gic_test_mmio(void)
>> static void test_its_introspection(void) {}
>> static void test_its_trigger(void) {}
>> static void test_its_migration(void) {}
>> +static void test_migrate_unmapped_collection(void) {}
>> +static void test_its_pending_migration(void) {}
>
> I'm not sure what's worse. This pile of stubs or one #ifdef in main()
> wrapping all the calls.
Those stubs now are in the arm header.
>
>>
>> #else /* __arch64__ */
>>
>> @@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
>> return false;
>> }
>>
>> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
>> + struct its_collection *col)
>> +{
>> + if (!dev || !col)
>
> I don't think col can be null, and this doesn't look like the right place
> to check if dev is null. If we're bothiner to call set_lpi, then I
> think we should already expect dev to be good to go.
put an assert() instead
>
>> + report_abort("wrong device or collection");
>> +
>> + its_send_mapti(dev, physid, eventid, col);
>> +
>> + gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
>> + its_send_invall(col);
>> +}
>> +
>> /*
>> * Setup the configuration for those mappings:
>> * dev_id=2 event=20 -> vcpu 3, intid=8195
>> @@ -806,6 +833,121 @@ static void test_its_migration(void)
>> its_send_int(dev7, 255);
>> check_lpi_stats();
>> }
>> +
>> +static void test_migrate_unmapped_collection(void)
>> +{
>> + struct its_collection *col;
>> + struct its_device *dev2, *dev7;
>> + u8 config;
>> +
>> + if (its_setup1())
>> + return;
>> +
>> + col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
>> + dev2 = its_get_device(2);
>> + dev7 = its_get_device(7);
>> +
>> + /* MAPTI with the collection unmapped */
>> + set_lpi(dev2, 0, 8192, col);
>> +
>> + puts("Now migrate the VM, then press a key to continue...\n");
>> + (void)getchar();
>> + report(true, "Migration complete");
>
> report_info
yep
>
>> +
>> + /* on the destination, map the collection */
>> + its_send_mapc(col, true);
>> +
>> + lpi_stats_expect(2, 8196);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> +
>> + config = gicv3_lpi_get_config(8192);
>> + report(config == LPI_PROP_DEFAULT,
>> + "Config of LPI 8192 was properly migrated");
>> +
>> + lpi_stats_expect(nr_cpus - 1, 8192);
>> + its_send_int(dev2, 0);
>> + check_lpi_stats();
>> +
>> + /* unmap the collection */
>> + its_send_mapc(col, false);
>> +
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 0);
>> + check_lpi_stats();
>> +
>> + /* remap event 0 onto lpiid 8193 */
>> + set_lpi(dev2, 0, 8193, col);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 0);
>> + check_lpi_stats();
>> +
>> + /* remap the collection */
>> + its_send_mapc(col, true);
>> + lpi_stats_expect(nr_cpus - 1, 8193);
>> +}
>> +
>> +static void test_its_pending_migration(void)
>> +{
>> + struct its_device *dev;
>> + struct its_collection *collection[2];
>> + int expected[NR_CPUS];
>
> expected = malloc(nr_cpus * sizeof(int));
>
> I know there are other places using NR_CPUS right now that don't have to,
> but we shouldn't add more. Eventually I'll change the other places too.
OK
>
>> + u64 pendbaser;
>> + void *ptr;
>> + int i;
>> +
>> + if (its_prerequisites(4))
>> + return;
>> +
>> + dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>> + its_send_mapd(dev, true);
>> +
>> + collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
>> + collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
>> + its_send_mapc(collection[0], true);
>> + its_send_mapc(collection[1], true);
>> +
>> + /* disable lpi at redist level */
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
>> +
>> + /* even lpis are assigned to even cpu */
>> + for (i = 0; i < 256; i++) {
>> + struct its_collection *col = i % 2 ? collection[0] :
>> + collection[1];
>> + int vcpu = col->target_address >> 16;
>
> I'm lost with the even/odd (nr_cpus - 1)/(nr_cpus - 2) stuff, and won't
> it swap if nr_cpus is odd vs. even?
>
> Shouldn't we just have something like
>
> pe1 = nr_cpus - 1;
> pe2 = nr_cpus - 2;
> col1 = its_create_collection(pe1, pe1);
> col2 = its_create_collection(pe2, pe2);
>
> without mentioning even and odd?
OK
>
>> +
>> + its_send_mapti(dev, 8192 + i, i, col);
>> + gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
>> + gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
>> + }
>> + its_send_invall(collection[0]);
>> + its_send_invall(collection[1]);
>> +
>> + /* Set the PTZ bit on each pendbaser */
>> +
>> + expected[nr_cpus - 1] = 128;
>> + expected[nr_cpus - 2] = 128;
>> +
>> + ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
>> + pendbaser = readq(ptr);
>> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>> +
>> + ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
>> + pendbaser = readq(ptr);
>> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>> +
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
>> +
>> + puts("Now migrate the VM, then press a key to continue...\n");
>> + (void)getchar();
>> + report(true, "Migration complete");
>
> report_info
OK
>
>> +
>> + mdelay(1000);
>
> This delay needs a comment explaining why it's here.
OK
Thanks
Eric
>
>> +
>> + check_lpi_hits(expected);
>> +}
>> #endif
>>
>> int main(int argc, char **argv)
>> @@ -847,6 +989,14 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> test_its_migration();
>> report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-pending-migration")) {
>> + report_prefix_push(argv[1]);
>> + test_its_pending_migration();
>> + report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
>> + report_prefix_push(argv[1]);
>> + test_migrate_unmapped_collection();
>> + report_prefix_pop();
>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>> report_prefix_push(argv[1]);
>> test_its_introspection();
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index 8b8ec79..d917157 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
>> groups = its migration
>> arch = arm64
>>
>> +[its-pending-migration]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +accel = kvm
>> +extra_params = -machine gic-version=3 -append 'its-pending-migration'
>> +groups = its migration
>> +arch = arm64
>> +
>> +[its-migrate-unmapped-collection]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +accel = kvm
>> +extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
>> +groups = its migration
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> --
>> 2.20.1
>>
>>
>
> Thanks,
> drew
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
@ 2020-03-06 13:21 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 13:21 UTC (permalink / raw)
To: Andrew Jones
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 3:06 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:59AM +0100, Eric Auger wrote:
>> Add two new migration tests. One testing the migration of
>> a topology where collection were unmapped. The second test
>> checks the migration of the pending table.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - tests belong to both its and migration groups
>> ---
>> arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
>> arm/unittests.cfg | 16 +++++
>> 2 files changed, 166 insertions(+)
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index fa8626a..ec3dd3a 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
>> smp_rmb(); /* pairs with wmb in lpi_stats_expect */
>> lpi_stats.observed.cpu_id = smp_processor_id();
>> lpi_stats.observed.lpi_id = irqnr;
>> + acked[lpi_stats.observed.cpu_id]++;
>> smp_wmb(); /* pairs with rmb in check_lpi_stats */
>> }
>>
>> @@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
>> while (1)
>> wfi();
>> }
>> +
>> +static void check_lpi_hits(int *expected)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < nr_cpus; i++) {
>> + if (acked[i] != expected[i])
>> + report(false, "expected %d LPIs on PE #%d, %d observed",
>> + expected[i], i, acked[i]);
>
> report_info
> pass = false
>
>> + }
>> + report(true, "check LPI on all vcpus");
>
> report(pass, ...);
I still don't get the issue.
>
>> +}
>> #endif
>>
>> static void gicv2_ipi_send_self(void)
>> @@ -594,6 +607,8 @@ static void gic_test_mmio(void)
>> static void test_its_introspection(void) {}
>> static void test_its_trigger(void) {}
>> static void test_its_migration(void) {}
>> +static void test_migrate_unmapped_collection(void) {}
>> +static void test_its_pending_migration(void) {}
>
> I'm not sure what's worse. This pile of stubs or one #ifdef in main()
> wrapping all the calls.
Those stubs now are in the arm header.
>
>>
>> #else /* __arch64__ */
>>
>> @@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
>> return false;
>> }
>>
>> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
>> + struct its_collection *col)
>> +{
>> + if (!dev || !col)
>
> I don't think col can be null, and this doesn't look like the right place
> to check if dev is null. If we're bothiner to call set_lpi, then I
> think we should already expect dev to be good to go.
put an assert() instead
>
>> + report_abort("wrong device or collection");
>> +
>> + its_send_mapti(dev, physid, eventid, col);
>> +
>> + gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
>> + its_send_invall(col);
>> +}
>> +
>> /*
>> * Setup the configuration for those mappings:
>> * dev_id=2 event=20 -> vcpu 3, intid=8195
>> @@ -806,6 +833,121 @@ static void test_its_migration(void)
>> its_send_int(dev7, 255);
>> check_lpi_stats();
>> }
>> +
>> +static void test_migrate_unmapped_collection(void)
>> +{
>> + struct its_collection *col;
>> + struct its_device *dev2, *dev7;
>> + u8 config;
>> +
>> + if (its_setup1())
>> + return;
>> +
>> + col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
>> + dev2 = its_get_device(2);
>> + dev7 = its_get_device(7);
>> +
>> + /* MAPTI with the collection unmapped */
>> + set_lpi(dev2, 0, 8192, col);
>> +
>> + puts("Now migrate the VM, then press a key to continue...\n");
>> + (void)getchar();
>> + report(true, "Migration complete");
>
> report_info
yep
>
>> +
>> + /* on the destination, map the collection */
>> + its_send_mapc(col, true);
>> +
>> + lpi_stats_expect(2, 8196);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> +
>> + config = gicv3_lpi_get_config(8192);
>> + report(config == LPI_PROP_DEFAULT,
>> + "Config of LPI 8192 was properly migrated");
>> +
>> + lpi_stats_expect(nr_cpus - 1, 8192);
>> + its_send_int(dev2, 0);
>> + check_lpi_stats();
>> +
>> + /* unmap the collection */
>> + its_send_mapc(col, false);
>> +
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 0);
>> + check_lpi_stats();
>> +
>> + /* remap event 0 onto lpiid 8193 */
>> + set_lpi(dev2, 0, 8193, col);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 0);
>> + check_lpi_stats();
>> +
>> + /* remap the collection */
>> + its_send_mapc(col, true);
>> + lpi_stats_expect(nr_cpus - 1, 8193);
>> +}
>> +
>> +static void test_its_pending_migration(void)
>> +{
>> + struct its_device *dev;
>> + struct its_collection *collection[2];
>> + int expected[NR_CPUS];
>
> expected = malloc(nr_cpus * sizeof(int));
>
> I know there are other places using NR_CPUS right now that don't have to,
> but we shouldn't add more. Eventually I'll change the other places too.
OK
>
>> + u64 pendbaser;
>> + void *ptr;
>> + int i;
>> +
>> + if (its_prerequisites(4))
>> + return;
>> +
>> + dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>> + its_send_mapd(dev, true);
>> +
>> + collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
>> + collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
>> + its_send_mapc(collection[0], true);
>> + its_send_mapc(collection[1], true);
>> +
>> + /* disable lpi at redist level */
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
>> +
>> + /* even lpis are assigned to even cpu */
>> + for (i = 0; i < 256; i++) {
>> + struct its_collection *col = i % 2 ? collection[0] :
>> + collection[1];
>> + int vcpu = col->target_address >> 16;
>
> I'm lost with the even/odd (nr_cpus - 1)/(nr_cpus - 2) stuff, and won't
> it swap if nr_cpus is odd vs. even?
>
> Shouldn't we just have something like
>
> pe1 = nr_cpus - 1;
> pe2 = nr_cpus - 2;
> col1 = its_create_collection(pe1, pe1);
> col2 = its_create_collection(pe2, pe2);
>
> without mentioning even and odd?
OK
>
>> +
>> + its_send_mapti(dev, 8192 + i, i, col);
>> + gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
>> + gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
>> + }
>> + its_send_invall(collection[0]);
>> + its_send_invall(collection[1]);
>> +
>> + /* Set the PTZ bit on each pendbaser */
>> +
>> + expected[nr_cpus - 1] = 128;
>> + expected[nr_cpus - 2] = 128;
>> +
>> + ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
>> + pendbaser = readq(ptr);
>> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>> +
>> + ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
>> + pendbaser = readq(ptr);
>> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>> +
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
>> +
>> + puts("Now migrate the VM, then press a key to continue...\n");
>> + (void)getchar();
>> + report(true, "Migration complete");
>
> report_info
OK
>
>> +
>> + mdelay(1000);
>
> This delay needs a comment explaining why it's here.
OK
Thanks
Eric
>
>> +
>> + check_lpi_hits(expected);
>> +}
>> #endif
>>
>> int main(int argc, char **argv)
>> @@ -847,6 +989,14 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> test_its_migration();
>> report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-pending-migration")) {
>> + report_prefix_push(argv[1]);
>> + test_its_pending_migration();
>> + report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
>> + report_prefix_push(argv[1]);
>> + test_migrate_unmapped_collection();
>> + report_prefix_pop();
>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>> report_prefix_push(argv[1]);
>> test_its_introspection();
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index 8b8ec79..d917157 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
>> groups = its migration
>> arch = arm64
>>
>> +[its-pending-migration]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +accel = kvm
>> +extra_params = -machine gic-version=3 -append 'its-pending-migration'
>> +groups = its migration
>> +arch = arm64
>> +
>> +[its-migrate-unmapped-collection]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +accel = kvm
>> +extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
>> +groups = its migration
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> --
>> 2.20.1
>>
>>
>
> Thanks,
> drew
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
@ 2020-03-06 13:21 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 13:21 UTC (permalink / raw)
To: Andrew Jones
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
Hi Drew,
On 2/7/20 3:06 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 11:34:59AM +0100, Eric Auger wrote:
>> Add two new migration tests. One testing the migration of
>> a topology where collection were unmapped. The second test
>> checks the migration of the pending table.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - tests belong to both its and migration groups
>> ---
>> arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
>> arm/unittests.cfg | 16 +++++
>> 2 files changed, 166 insertions(+)
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index fa8626a..ec3dd3a 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
>> smp_rmb(); /* pairs with wmb in lpi_stats_expect */
>> lpi_stats.observed.cpu_id = smp_processor_id();
>> lpi_stats.observed.lpi_id = irqnr;
>> + acked[lpi_stats.observed.cpu_id]++;
>> smp_wmb(); /* pairs with rmb in check_lpi_stats */
>> }
>>
>> @@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
>> while (1)
>> wfi();
>> }
>> +
>> +static void check_lpi_hits(int *expected)
>> +{
>> + int i;
>> +
>> + for (i = 0; i < nr_cpus; i++) {
>> + if (acked[i] != expected[i])
>> + report(false, "expected %d LPIs on PE #%d, %d observed",
>> + expected[i], i, acked[i]);
>
> report_info
> pass = false
>
>> + }
>> + report(true, "check LPI on all vcpus");
>
> report(pass, ...);
I still don't get the issue.
>
>> +}
>> #endif
>>
>> static void gicv2_ipi_send_self(void)
>> @@ -594,6 +607,8 @@ static void gic_test_mmio(void)
>> static void test_its_introspection(void) {}
>> static void test_its_trigger(void) {}
>> static void test_its_migration(void) {}
>> +static void test_migrate_unmapped_collection(void) {}
>> +static void test_its_pending_migration(void) {}
>
> I'm not sure what's worse. This pile of stubs or one #ifdef in main()
> wrapping all the calls.
Those stubs now are in the arm header.
>
>>
>> #else /* __arch64__ */
>>
>> @@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
>> return false;
>> }
>>
>> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
>> + struct its_collection *col)
>> +{
>> + if (!dev || !col)
>
> I don't think col can be null, and this doesn't look like the right place
> to check if dev is null. If we're bothiner to call set_lpi, then I
> think we should already expect dev to be good to go.
put an assert() instead
>
>> + report_abort("wrong device or collection");
>> +
>> + its_send_mapti(dev, physid, eventid, col);
>> +
>> + gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
>> + its_send_invall(col);
>> +}
>> +
>> /*
>> * Setup the configuration for those mappings:
>> * dev_id=2 event=20 -> vcpu 3, intid=8195
>> @@ -806,6 +833,121 @@ static void test_its_migration(void)
>> its_send_int(dev7, 255);
>> check_lpi_stats();
>> }
>> +
>> +static void test_migrate_unmapped_collection(void)
>> +{
>> + struct its_collection *col;
>> + struct its_device *dev2, *dev7;
>> + u8 config;
>> +
>> + if (its_setup1())
>> + return;
>> +
>> + col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
>> + dev2 = its_get_device(2);
>> + dev7 = its_get_device(7);
>> +
>> + /* MAPTI with the collection unmapped */
>> + set_lpi(dev2, 0, 8192, col);
>> +
>> + puts("Now migrate the VM, then press a key to continue...\n");
>> + (void)getchar();
>> + report(true, "Migration complete");
>
> report_info
yep
>
>> +
>> + /* on the destination, map the collection */
>> + its_send_mapc(col, true);
>> +
>> + lpi_stats_expect(2, 8196);
>> + its_send_int(dev7, 255);
>> + check_lpi_stats();
>> +
>> + config = gicv3_lpi_get_config(8192);
>> + report(config == LPI_PROP_DEFAULT,
>> + "Config of LPI 8192 was properly migrated");
>> +
>> + lpi_stats_expect(nr_cpus - 1, 8192);
>> + its_send_int(dev2, 0);
>> + check_lpi_stats();
>> +
>> + /* unmap the collection */
>> + its_send_mapc(col, false);
>> +
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 0);
>> + check_lpi_stats();
>> +
>> + /* remap event 0 onto lpiid 8193 */
>> + set_lpi(dev2, 0, 8193, col);
>> + lpi_stats_expect(-1, -1);
>> + its_send_int(dev2, 0);
>> + check_lpi_stats();
>> +
>> + /* remap the collection */
>> + its_send_mapc(col, true);
>> + lpi_stats_expect(nr_cpus - 1, 8193);
>> +}
>> +
>> +static void test_its_pending_migration(void)
>> +{
>> + struct its_device *dev;
>> + struct its_collection *collection[2];
>> + int expected[NR_CPUS];
>
> expected = malloc(nr_cpus * sizeof(int));
>
> I know there are other places using NR_CPUS right now that don't have to,
> but we shouldn't add more. Eventually I'll change the other places too.
OK
>
>> + u64 pendbaser;
>> + void *ptr;
>> + int i;
>> +
>> + if (its_prerequisites(4))
>> + return;
>> +
>> + dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>> + its_send_mapd(dev, true);
>> +
>> + collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
>> + collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
>> + its_send_mapc(collection[0], true);
>> + its_send_mapc(collection[1], true);
>> +
>> + /* disable lpi at redist level */
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
>> +
>> + /* even lpis are assigned to even cpu */
>> + for (i = 0; i < 256; i++) {
>> + struct its_collection *col = i % 2 ? collection[0] :
>> + collection[1];
>> + int vcpu = col->target_address >> 16;
>
> I'm lost with the even/odd (nr_cpus - 1)/(nr_cpus - 2) stuff, and won't
> it swap if nr_cpus is odd vs. even?
>
> Shouldn't we just have something like
>
> pe1 = nr_cpus - 1;
> pe2 = nr_cpus - 2;
> col1 = its_create_collection(pe1, pe1);
> col2 = its_create_collection(pe2, pe2);
>
> without mentioning even and odd?
OK
>
>> +
>> + its_send_mapti(dev, 8192 + i, i, col);
>> + gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
>> + gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
>> + }
>> + its_send_invall(collection[0]);
>> + its_send_invall(collection[1]);
>> +
>> + /* Set the PTZ bit on each pendbaser */
>> +
>> + expected[nr_cpus - 1] = 128;
>> + expected[nr_cpus - 2] = 128;
>> +
>> + ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
>> + pendbaser = readq(ptr);
>> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>> +
>> + ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
>> + pendbaser = readq(ptr);
>> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>> +
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
>> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
>> +
>> + puts("Now migrate the VM, then press a key to continue...\n");
>> + (void)getchar();
>> + report(true, "Migration complete");
>
> report_info
OK
>
>> +
>> + mdelay(1000);
>
> This delay needs a comment explaining why it's here.
OK
Thanks
Eric
>
>> +
>> + check_lpi_hits(expected);
>> +}
>> #endif
>>
>> int main(int argc, char **argv)
>> @@ -847,6 +989,14 @@ int main(int argc, char **argv)
>> report_prefix_push(argv[1]);
>> test_its_migration();
>> report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-pending-migration")) {
>> + report_prefix_push(argv[1]);
>> + test_its_pending_migration();
>> + report_prefix_pop();
>> + } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
>> + report_prefix_push(argv[1]);
>> + test_migrate_unmapped_collection();
>> + report_prefix_pop();
>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>> report_prefix_push(argv[1]);
>> test_its_introspection();
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index 8b8ec79..d917157 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
>> groups = its migration
>> arch = arm64
>>
>> +[its-pending-migration]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +accel = kvm
>> +extra_params = -machine gic-version=3 -append 'its-pending-migration'
>> +groups = its migration
>> +arch = arm64
>> +
>> +[its-migrate-unmapped-collection]
>> +file = gic.flat
>> +smp = $MAX_SMP
>> +accel = kvm
>> +extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
>> +groups = its migration
>> +arch = arm64
>> +
>> # Test PSCI emulation
>> [psci]
>> file = psci.flat
>> --
>> 2.20.1
>>
>>
>
> Thanks,
> drew
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
2020-03-06 12:55 ` Auger Eric
@ 2020-03-06 13:29 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-03-06 13:29 UTC (permalink / raw)
To: Auger Eric
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Fri, Mar 06, 2020 at 01:55:09PM +0100, Auger Eric wrote:
> Hi Drew,
>
> On 2/7/20 2:15 PM, Andrew Jones wrote:
> > On Tue, Jan 28, 2020 at 11:34:56AM +0100, Eric Auger wrote:
> >> Triggers LPIs through the INT command.
> >>
> >> the test checks the LPI hits the right CPU and triggers
> >> the right LPI intid, ie. the translation is correct.
> >>
> >> Updates to the config table also are tested, along with inv
> >> and invall commands.
> >>
> >> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> >>
> >> ---
> >>
> >> v2 -> v3:
> >> - add comments
> >> - keep the report_skip in case there aren't 4 vcpus to be able to
> >> run other tests in the its category.
> >> - fix the prefix pop
> >> - move its_event and its_stats to arm/gic.c
> >> ---
> >> arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
> >> arm/unittests.cfg | 7 ++
> >> 2 files changed, 224 insertions(+), 11 deletions(-)
> >>
> >> diff --git a/arm/gic.c b/arm/gic.c
> >> index 4d7dd03..50104b1 100644
> >> --- a/arm/gic.c
> >> +++ b/arm/gic.c
> >> @@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
> >> }
> >> }
> >>
> >> +static void setup_irq(handler_t handler)
> >> +{
> >> + gic_enable_defaults();
> >> +#ifdef __arm__
> >> + install_exception_handler(EXCPTN_IRQ, handler);
> >> +#else
> >> + install_irq_handler(EL1H_IRQ, handler);
> >> +#endif
> >> + local_irq_enable();
> >> +}
> >> +
> >> +#if defined(__aarch64__)
> >> +struct its_event {
> >> + int cpu_id;
> >> + int lpi_id;
> >> +};
> >> +
> >> +struct its_stats {
> >> + struct its_event expected;
> >> + struct its_event observed;
> >> +};
> >> +
> >> +static struct its_stats lpi_stats;
> >> +
> >> +static void lpi_handler(struct pt_regs *regs __unused)
> >> +{
> >> + u32 irqstat = gic_read_iar();
> >> + int irqnr = gic_iar_irqnr(irqstat);
> >> +
> >> + gic_write_eoir(irqstat);
> >> + if (irqnr < 8192)
> >> + report(false, "Unexpected non LPI interrupt received");
> >
> > report_info
> why? This is an error case. We do not expect other interrupts than LPIs
If there's almost no chance this will happen and it means something quite
unexpected has occurred, then it should probably be an assert. If this is
a real test case, then it should be
report(irqnr >= 8192, "Got LPI");
or something like that. If it's something that shouldn't ever happen, so
it doesn't really deserve its own PASS/FAIL test output each execution
of the unit test, but you don't want to assert for some reason, then it
should be a report_info, but it should probably also contain a "WARNING"
prefix in that case.
> >
> >> + smp_rmb(); /* pairs with wmb in lpi_stats_expect */
> >> + lpi_stats.observed.cpu_id = smp_processor_id();
> >> + lpi_stats.observed.lpi_id = irqnr;
> >> + smp_wmb(); /* pairs with rmb in check_lpi_stats */
> >> +}
> >> +
> >> +static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
> >> +{
> >> + lpi_stats.expected.cpu_id = exp_cpu_id;
> >> + lpi_stats.expected.lpi_id = exp_lpi_id;
> >> + lpi_stats.observed.cpu_id = -1;
> >> + lpi_stats.observed.lpi_id = -1;
> >> + smp_wmb(); /* pairs with rmb in handler */
> >> +}
> >> +
> >> +static void check_lpi_stats(void)
> >
> > static void check_lpi_stats(const char *testname)
> > {
> > bool pass = false;
> >
> >> +{
> >> + mdelay(100);
> >> + smp_rmb(); /* pairs with wmb in lpi_handler */
> >> + if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
> >> + (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
> >
> > nit: extra ()
> >
> >> + if (lpi_stats.observed.cpu_id == -1 &&
> >> + lpi_stats.observed.lpi_id == -1) {
> >> + report(false,
> >> + "No LPI received whereas (cpuid=%d, intid=%d) "
> >> + "was expected", lpi_stats.expected.cpu_id,
> >> + lpi_stats.expected.lpi_id);
> >
> > report_info
> What's the problem keeping those. Those are error reports. The message
> is something like that:
> FAIL: gicv3: its-trigger: mapc valid=false: No LPI received whereas
> (cpuid=1, intid=8192) was expected.
>
> So the testname is already part of the message.
This one has two problems with being report() vs. report_info. The same
comment as above, where the condition for a report() should be the test,
rather than if (cond) report(false, ...), which implies it's not expected
to report at all. A pattern like that needs to be extended at least to
something like this
if (cond)
report(true, ...)
else
report(false, ...)
so we get the PASS/FAIL each execution. The other problem with this
particular report() is the dynamic info in it (cpuid and maybe intid).
A report() should only have consistent info so test output parsers
can count on finding the PASS/FAIL for a given report line. If you
need a test like this, then it can be structured like
report_info(...); // dynamic info
if (cond) {
report(true, MSG1); // no dynamic info
report(true, MSG2); // no dynamic info
} else {
report(false, MSG1); // no dynamic info
report(false, MSG2); // no dynamic info
}
Notice how the MSG's match on both paths of the condition.
Or just
report_info(...);
report(cond, ...);
> >
> >> + } else {
> >> + report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
> >> + lpi_stats.observed.cpu_id,
> >> + lpi_stats.observed.lpi_id);
> >
> > report_info
> >
> >> + }
> >
> > pass = false;
> >
> >> + } else if (lpi_stats.expected.lpi_id != -1) {
> >> + report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
> >> + lpi_stats.observed.cpu_id);
> >
> > report_info
> >
> >> + } else {
> >> + report(true, "no LPI received, as expected");
> >
> > report_info
This if, else if, ..., else with report() would be fine if the messages
would all match, resulting in a single 'PASS: MSG' line. report_info can
be used to get the dynamic info output too.
> >
> >
> >> + }
> >
> > report(pass, "%s", testname);
> >
> >> +}
> >> +
> >> +static void secondary_lpi_test(void)
> >> +{
> >> + setup_irq(lpi_handler);
> >> + cpumask_set_cpu(smp_processor_id(), &ready);
> >> + while (1)
> >> + wfi();
> >> +}
> >> +#endif
> >> +
> >> static void gicv2_ipi_send_self(void)
> >> {
> >> writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
> >> @@ -217,17 +298,6 @@ static void ipi_test_smp(void)
> >> report_prefix_pop();
> >> }
> >>
> >> -static void setup_irq(handler_t handler)
> >> -{
> >> - gic_enable_defaults();
> >> -#ifdef __arm__
> >> - install_exception_handler(EXCPTN_IRQ, handler);
> >> -#else
> >> - install_irq_handler(EL1H_IRQ, handler);
> >> -#endif
> >> - local_irq_enable();
> >> -}
> >> -
> >> static void ipi_send(void)
> >> {
> >> setup_irq(ipi_handler);
> >> @@ -522,6 +592,7 @@ static void gic_test_mmio(void)
> >> #if defined(__arm__)
> >>
> >> static void test_its_introspection(void) {}
> >> +static void test_its_trigger(void) {}
> >>
> >> #else /* __arch64__ */
> >>
> >> @@ -561,6 +632,137 @@ static void test_its_introspection(void)
> >> report_info("collection baser entry_size = 0x%x", coll_baser->esz);
> >> }
> >>
> >> +static bool its_prerequisites(int nb_cpus)
> >> +{
> >> + int cpu;
> >> +
> >> + if (!gicv3_its_base()) {
> >> + report_skip("No ITS, skip ...");
> >> + return true;
> >> + }
> >> +
> >> + if (nr_cpus < 4) {
> >
> > nr_cpus < nb_cpus, or just drop the nb_cpus parameter and hard code 4
> > here.
> sure
> >
> >> + report_skip("Test requires at least %d vcpus", nb_cpus);
> >> + return true;
> >> + }
> >> +
> >> + stats_reset();
> >> +
> >> + setup_irq(lpi_handler);
> >> +
> >> + for_each_present_cpu(cpu) {
> >> + if (cpu == 0)
> >> + continue;
> >> + smp_boot_secondary(cpu, secondary_lpi_test);
> >> + }
> >> + wait_on_ready();
> >> +
> >> + its_enable_defaults();
> >> +
> >> + lpi_stats_expect(-1, -1);
> >> + check_lpi_stats();
> >> +
> >> + return false;
> >
> > Reverse logic. I'd expect 'return true' for success.
> I am going to return an int. In case of error a std negative error will
> be returned.
> >
> >> +}
> >> +
> >> +static void test_its_trigger(void)
> >> +{
> >> + struct its_collection *col3, *col2;
> >> + struct its_device *dev2, *dev7;
> >> +
> >> + if (its_prerequisites(4))
> >
> > if (!its_prerequisites(...))
> >
> >> + return;
> >> +
> >> + dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> >> + dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
> >> +
> >> + col3 = its_create_collection(3 /* col id */, 3/* target PE */);
> >> + col2 = its_create_collection(2 /* col id */, 2/* target PE */);
> >> +
> >> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
> >> + gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
> >> +
> >> + its_send_invall(col2);
> >> + its_send_invall(col3);
> >> +
> >> + report_prefix_push("int");
> >> + /*
> >> + * dev=2, eventid=20 -> lpi= 8195, col=3
> >> + * dev=7, eventid=255 -> lpi= 8196, col=2
> >> + * Trigger dev2, eventid=20 and dev7, eventid=255
> >> + * Check both LPIs hit
> >> + */
> >> +
> >> + its_send_mapd(dev2, true);
> >> + its_send_mapd(dev7, true);
> >> +
> >> + its_send_mapc(col3, true);
> >> + its_send_mapc(col2, true);
> >> +
> >> + its_send_mapti(dev2, 8195 /* lpi id */,
> >> + 20 /* event id */, col3);
> >> + its_send_mapti(dev7, 8196 /* lpi id */,
> >> + 255 /* event id */, col2);
> >
> > No need for line breaks, with the embedded comments it's hard to read
> OK
> >
> >> +
> >> + lpi_stats_expect(3, 8195);
> >> + its_send_int(dev2, 20);
> >> + check_lpi_stats();
> >> +
> >> + lpi_stats_expect(2, 8196);
> >> + its_send_int(dev7, 255);
> >> + check_lpi_stats();
> >> +
> >> + report_prefix_pop();
> >
> > I think a table of parameters and loop would be nicer than all the
> > repeated function calls.
> Frankly speaking I am not sure this would really help. We are just
> enabling 2 translation paths. I think I prefer to manipulate the low
> level objects and helpers rather than playing with a loop and potential
> new structs of params.
OK, but you could probably at least wrap the common sequence into one
function
void master_function(a1, a2, a3, a4)
{
lpi_stats_expect(a1, a2);
its_send_int(a3, a4);
check_lpi_stats();
}
but whatever, it's not so important.
> >
> >> +
> >> + report_prefix_push("inv/invall");
> >> +
> >> + /*
> >> + * disable 8195, check dev2/eventid=20 does not trigger the
> >> + * corresponding LPI
> >> + */
> >> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
> >
> > LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED
> ok
> >
> >> + its_send_inv(dev2, 20);
> >> +
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev2, 20);
> >> + check_lpi_stats();
> >> +
> >> + /*
> >> + * re-enable the LPI but willingly do not call invall
> >> + * so the change in config is not taken into account.
> >> + * The LPI should not hit
> >> + */
> >> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev2, 20);
> >> + check_lpi_stats();
> >> +
> >> + /* Now call the invall and check the LPI hits */
> >> + its_send_invall(col3);
> >> + lpi_stats_expect(3, 8195);
> >> + its_send_int(dev2, 20);
> >> + check_lpi_stats();
> >> +
> >> + report_prefix_pop();
> >
> > Need blank line here.
> OK
> >
> >> + /*
> >> + * Unmap device 2 and check the eventid 20 formerly
> >> + * attached to it does not hit anymore
> >> + */
> >> + report_prefix_push("mapd valid=false");
> >
> > Above you have the prefix-push before the comment explaining the test.
> > After is probably better, but whatever, as long as it's consistent.
> moved after
> >
> >> + its_send_mapd(dev2, false);
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev2, 20);
> >> + check_lpi_stats();
> >> + report_prefix_pop();
> >> +
> >> + /* Unmap the collection this time and check no LPI does hit */
> >> + report_prefix_push("mapc valid=false");
> >> + its_send_mapc(col2, false);
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev7, 255);
> >> + check_lpi_stats();
> >> + report_prefix_pop();
> >> +}
> >> #endif
> >>
> >> int main(int argc, char **argv)
> >> @@ -594,6 +796,10 @@ int main(int argc, char **argv)
> >> report_prefix_push(argv[1]);
> >> gic_test_mmio();
> >> report_prefix_pop();
> >> + } else if (!strcmp(argv[1], "its-trigger")) {
> >> + report_prefix_push(argv[1]);
> >> + test_its_trigger();
> >> + report_prefix_pop();
> >> } else if (strcmp(argv[1], "its-introspection") == 0) {
> >> report_prefix_push(argv[1]);
> >> test_its_introspection();
> >> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> >> index ba2b31b..bfafec5 100644
> >> --- a/arm/unittests.cfg
> >> +++ b/arm/unittests.cfg
> >> @@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
> >> groups = its
> >> arch = arm64
> >>
> >> +[its-trigger]
> >> +file = gic.flat
> >> +smp = $MAX_SMP
> >> +extra_params = -machine gic-version=3 -append 'its-trigger'
> >> +groups = its
> >> +arch = arm64
> >> +
> >> # Test PSCI emulation
> >> [psci]
> >> file = psci.flat
> >> --
> >> 2.20.1
> >>
> >
> > Thanks,
> > drew
> >
> Thanks
>
> Eric
>
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
@ 2020-03-06 13:29 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-03-06 13:29 UTC (permalink / raw)
To: Auger Eric
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Fri, Mar 06, 2020 at 01:55:09PM +0100, Auger Eric wrote:
> Hi Drew,
>
> On 2/7/20 2:15 PM, Andrew Jones wrote:
> > On Tue, Jan 28, 2020 at 11:34:56AM +0100, Eric Auger wrote:
> >> Triggers LPIs through the INT command.
> >>
> >> the test checks the LPI hits the right CPU and triggers
> >> the right LPI intid, ie. the translation is correct.
> >>
> >> Updates to the config table also are tested, along with inv
> >> and invall commands.
> >>
> >> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> >>
> >> ---
> >>
> >> v2 -> v3:
> >> - add comments
> >> - keep the report_skip in case there aren't 4 vcpus to be able to
> >> run other tests in the its category.
> >> - fix the prefix pop
> >> - move its_event and its_stats to arm/gic.c
> >> ---
> >> arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
> >> arm/unittests.cfg | 7 ++
> >> 2 files changed, 224 insertions(+), 11 deletions(-)
> >>
> >> diff --git a/arm/gic.c b/arm/gic.c
> >> index 4d7dd03..50104b1 100644
> >> --- a/arm/gic.c
> >> +++ b/arm/gic.c
> >> @@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
> >> }
> >> }
> >>
> >> +static void setup_irq(handler_t handler)
> >> +{
> >> + gic_enable_defaults();
> >> +#ifdef __arm__
> >> + install_exception_handler(EXCPTN_IRQ, handler);
> >> +#else
> >> + install_irq_handler(EL1H_IRQ, handler);
> >> +#endif
> >> + local_irq_enable();
> >> +}
> >> +
> >> +#if defined(__aarch64__)
> >> +struct its_event {
> >> + int cpu_id;
> >> + int lpi_id;
> >> +};
> >> +
> >> +struct its_stats {
> >> + struct its_event expected;
> >> + struct its_event observed;
> >> +};
> >> +
> >> +static struct its_stats lpi_stats;
> >> +
> >> +static void lpi_handler(struct pt_regs *regs __unused)
> >> +{
> >> + u32 irqstat = gic_read_iar();
> >> + int irqnr = gic_iar_irqnr(irqstat);
> >> +
> >> + gic_write_eoir(irqstat);
> >> + if (irqnr < 8192)
> >> + report(false, "Unexpected non LPI interrupt received");
> >
> > report_info
> why? This is an error case. We do not expect other interrupts than LPIs
If there's almost no chance this will happen and it means something quite
unexpected has occurred, then it should probably be an assert. If this is
a real test case, then it should be
report(irqnr >= 8192, "Got LPI");
or something like that. If it's something that shouldn't ever happen, so
it doesn't really deserve its own PASS/FAIL test output each execution
of the unit test, but you don't want to assert for some reason, then it
should be a report_info, but it should probably also contain a "WARNING"
prefix in that case.
> >
> >> + smp_rmb(); /* pairs with wmb in lpi_stats_expect */
> >> + lpi_stats.observed.cpu_id = smp_processor_id();
> >> + lpi_stats.observed.lpi_id = irqnr;
> >> + smp_wmb(); /* pairs with rmb in check_lpi_stats */
> >> +}
> >> +
> >> +static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
> >> +{
> >> + lpi_stats.expected.cpu_id = exp_cpu_id;
> >> + lpi_stats.expected.lpi_id = exp_lpi_id;
> >> + lpi_stats.observed.cpu_id = -1;
> >> + lpi_stats.observed.lpi_id = -1;
> >> + smp_wmb(); /* pairs with rmb in handler */
> >> +}
> >> +
> >> +static void check_lpi_stats(void)
> >
> > static void check_lpi_stats(const char *testname)
> > {
> > bool pass = false;
> >
> >> +{
> >> + mdelay(100);
> >> + smp_rmb(); /* pairs with wmb in lpi_handler */
> >> + if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
> >> + (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
> >
> > nit: extra ()
> >
> >> + if (lpi_stats.observed.cpu_id == -1 &&
> >> + lpi_stats.observed.lpi_id == -1) {
> >> + report(false,
> >> + "No LPI received whereas (cpuid=%d, intid=%d) "
> >> + "was expected", lpi_stats.expected.cpu_id,
> >> + lpi_stats.expected.lpi_id);
> >
> > report_info
> What's the problem keeping those. Those are error reports. The message
> is something like that:
> FAIL: gicv3: its-trigger: mapc valid=false: No LPI received whereas
> (cpuid=1, intid=8192) was expected.
>
> So the testname is already part of the message.
This one has two problems with being report() vs. report_info. The same
comment as above, where the condition for a report() should be the test,
rather than if (cond) report(false, ...), which implies it's not expected
to report at all. A pattern like that needs to be extended at least to
something like this
if (cond)
report(true, ...)
else
report(false, ...)
so we get the PASS/FAIL each execution. The other problem with this
particular report() is the dynamic info in it (cpuid and maybe intid).
A report() should only have consistent info so test output parsers
can count on finding the PASS/FAIL for a given report line. If you
need a test like this, then it can be structured like
report_info(...); // dynamic info
if (cond) {
report(true, MSG1); // no dynamic info
report(true, MSG2); // no dynamic info
} else {
report(false, MSG1); // no dynamic info
report(false, MSG2); // no dynamic info
}
Notice how the MSG's match on both paths of the condition.
Or just
report_info(...);
report(cond, ...);
> >
> >> + } else {
> >> + report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
> >> + lpi_stats.observed.cpu_id,
> >> + lpi_stats.observed.lpi_id);
> >
> > report_info
> >
> >> + }
> >
> > pass = false;
> >
> >> + } else if (lpi_stats.expected.lpi_id != -1) {
> >> + report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
> >> + lpi_stats.observed.cpu_id);
> >
> > report_info
> >
> >> + } else {
> >> + report(true, "no LPI received, as expected");
> >
> > report_info
This if, else if, ..., else with report() would be fine if the messages
would all match, resulting in a single 'PASS: MSG' line. report_info can
be used to get the dynamic info output too.
> >
> >
> >> + }
> >
> > report(pass, "%s", testname);
> >
> >> +}
> >> +
> >> +static void secondary_lpi_test(void)
> >> +{
> >> + setup_irq(lpi_handler);
> >> + cpumask_set_cpu(smp_processor_id(), &ready);
> >> + while (1)
> >> + wfi();
> >> +}
> >> +#endif
> >> +
> >> static void gicv2_ipi_send_self(void)
> >> {
> >> writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
> >> @@ -217,17 +298,6 @@ static void ipi_test_smp(void)
> >> report_prefix_pop();
> >> }
> >>
> >> -static void setup_irq(handler_t handler)
> >> -{
> >> - gic_enable_defaults();
> >> -#ifdef __arm__
> >> - install_exception_handler(EXCPTN_IRQ, handler);
> >> -#else
> >> - install_irq_handler(EL1H_IRQ, handler);
> >> -#endif
> >> - local_irq_enable();
> >> -}
> >> -
> >> static void ipi_send(void)
> >> {
> >> setup_irq(ipi_handler);
> >> @@ -522,6 +592,7 @@ static void gic_test_mmio(void)
> >> #if defined(__arm__)
> >>
> >> static void test_its_introspection(void) {}
> >> +static void test_its_trigger(void) {}
> >>
> >> #else /* __arch64__ */
> >>
> >> @@ -561,6 +632,137 @@ static void test_its_introspection(void)
> >> report_info("collection baser entry_size = 0x%x", coll_baser->esz);
> >> }
> >>
> >> +static bool its_prerequisites(int nb_cpus)
> >> +{
> >> + int cpu;
> >> +
> >> + if (!gicv3_its_base()) {
> >> + report_skip("No ITS, skip ...");
> >> + return true;
> >> + }
> >> +
> >> + if (nr_cpus < 4) {
> >
> > nr_cpus < nb_cpus, or just drop the nb_cpus parameter and hard code 4
> > here.
> sure
> >
> >> + report_skip("Test requires at least %d vcpus", nb_cpus);
> >> + return true;
> >> + }
> >> +
> >> + stats_reset();
> >> +
> >> + setup_irq(lpi_handler);
> >> +
> >> + for_each_present_cpu(cpu) {
> >> + if (cpu == 0)
> >> + continue;
> >> + smp_boot_secondary(cpu, secondary_lpi_test);
> >> + }
> >> + wait_on_ready();
> >> +
> >> + its_enable_defaults();
> >> +
> >> + lpi_stats_expect(-1, -1);
> >> + check_lpi_stats();
> >> +
> >> + return false;
> >
> > Reverse logic. I'd expect 'return true' for success.
> I am going to return an int. In case of error a std negative error will
> be returned.
> >
> >> +}
> >> +
> >> +static void test_its_trigger(void)
> >> +{
> >> + struct its_collection *col3, *col2;
> >> + struct its_device *dev2, *dev7;
> >> +
> >> + if (its_prerequisites(4))
> >
> > if (!its_prerequisites(...))
> >
> >> + return;
> >> +
> >> + dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> >> + dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
> >> +
> >> + col3 = its_create_collection(3 /* col id */, 3/* target PE */);
> >> + col2 = its_create_collection(2 /* col id */, 2/* target PE */);
> >> +
> >> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
> >> + gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
> >> +
> >> + its_send_invall(col2);
> >> + its_send_invall(col3);
> >> +
> >> + report_prefix_push("int");
> >> + /*
> >> + * dev=2, eventid=20 -> lpi= 8195, col=3
> >> + * dev=7, eventid=255 -> lpi= 8196, col=2
> >> + * Trigger dev2, eventid=20 and dev7, eventid=255
> >> + * Check both LPIs hit
> >> + */
> >> +
> >> + its_send_mapd(dev2, true);
> >> + its_send_mapd(dev7, true);
> >> +
> >> + its_send_mapc(col3, true);
> >> + its_send_mapc(col2, true);
> >> +
> >> + its_send_mapti(dev2, 8195 /* lpi id */,
> >> + 20 /* event id */, col3);
> >> + its_send_mapti(dev7, 8196 /* lpi id */,
> >> + 255 /* event id */, col2);
> >
> > No need for line breaks, with the embedded comments it's hard to read
> OK
> >
> >> +
> >> + lpi_stats_expect(3, 8195);
> >> + its_send_int(dev2, 20);
> >> + check_lpi_stats();
> >> +
> >> + lpi_stats_expect(2, 8196);
> >> + its_send_int(dev7, 255);
> >> + check_lpi_stats();
> >> +
> >> + report_prefix_pop();
> >
> > I think a table of parameters and loop would be nicer than all the
> > repeated function calls.
> Frankly speaking I am not sure this would really help. We are just
> enabling 2 translation paths. I think I prefer to manipulate the low
> level objects and helpers rather than playing with a loop and potential
> new structs of params.
OK, but you could probably at least wrap the common sequence into one
function
void master_function(a1, a2, a3, a4)
{
lpi_stats_expect(a1, a2);
its_send_int(a3, a4);
check_lpi_stats();
}
but whatever, it's not so important.
> >
> >> +
> >> + report_prefix_push("inv/invall");
> >> +
> >> + /*
> >> + * disable 8195, check dev2/eventid=20 does not trigger the
> >> + * corresponding LPI
> >> + */
> >> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
> >
> > LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED
> ok
> >
> >> + its_send_inv(dev2, 20);
> >> +
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev2, 20);
> >> + check_lpi_stats();
> >> +
> >> + /*
> >> + * re-enable the LPI but willingly do not call invall
> >> + * so the change in config is not taken into account.
> >> + * The LPI should not hit
> >> + */
> >> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev2, 20);
> >> + check_lpi_stats();
> >> +
> >> + /* Now call the invall and check the LPI hits */
> >> + its_send_invall(col3);
> >> + lpi_stats_expect(3, 8195);
> >> + its_send_int(dev2, 20);
> >> + check_lpi_stats();
> >> +
> >> + report_prefix_pop();
> >
> > Need blank line here.
> OK
> >
> >> + /*
> >> + * Unmap device 2 and check the eventid 20 formerly
> >> + * attached to it does not hit anymore
> >> + */
> >> + report_prefix_push("mapd valid=false");
> >
> > Above you have the prefix-push before the comment explaining the test.
> > After is probably better, but whatever, as long as it's consistent.
> moved after
> >
> >> + its_send_mapd(dev2, false);
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev2, 20);
> >> + check_lpi_stats();
> >> + report_prefix_pop();
> >> +
> >> + /* Unmap the collection this time and check no LPI does hit */
> >> + report_prefix_push("mapc valid=false");
> >> + its_send_mapc(col2, false);
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev7, 255);
> >> + check_lpi_stats();
> >> + report_prefix_pop();
> >> +}
> >> #endif
> >>
> >> int main(int argc, char **argv)
> >> @@ -594,6 +796,10 @@ int main(int argc, char **argv)
> >> report_prefix_push(argv[1]);
> >> gic_test_mmio();
> >> report_prefix_pop();
> >> + } else if (!strcmp(argv[1], "its-trigger")) {
> >> + report_prefix_push(argv[1]);
> >> + test_its_trigger();
> >> + report_prefix_pop();
> >> } else if (strcmp(argv[1], "its-introspection") == 0) {
> >> report_prefix_push(argv[1]);
> >> test_its_introspection();
> >> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> >> index ba2b31b..bfafec5 100644
> >> --- a/arm/unittests.cfg
> >> +++ b/arm/unittests.cfg
> >> @@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
> >> groups = its
> >> arch = arm64
> >>
> >> +[its-trigger]
> >> +file = gic.flat
> >> +smp = $MAX_SMP
> >> +extra_params = -machine gic-version=3 -append 'its-trigger'
> >> +groups = its
> >> +arch = arm64
> >> +
> >> # Test PSCI emulation
> >> [psci]
> >> file = psci.flat
> >> --
> >> 2.20.1
> >>
> >
> > Thanks,
> > drew
> >
> Thanks
>
> Eric
>
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
2020-03-06 13:21 ` Auger Eric
@ 2020-03-06 13:36 ` Andrew Jones
-1 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-03-06 13:36 UTC (permalink / raw)
To: Auger Eric
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
On Fri, Mar 06, 2020 at 02:21:37PM +0100, Auger Eric wrote:
> Hi Drew,
>
> On 2/7/20 3:06 PM, Andrew Jones wrote:
> > On Tue, Jan 28, 2020 at 11:34:59AM +0100, Eric Auger wrote:
> >> Add two new migration tests. One testing the migration of
> >> a topology where collection were unmapped. The second test
> >> checks the migration of the pending table.
> >>
> >> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> >>
> >> ---
> >>
> >> v2 -> v3:
> >> - tests belong to both its and migration groups
> >> ---
> >> arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
> >> arm/unittests.cfg | 16 +++++
> >> 2 files changed, 166 insertions(+)
> >>
> >> diff --git a/arm/gic.c b/arm/gic.c
> >> index fa8626a..ec3dd3a 100644
> >> --- a/arm/gic.c
> >> +++ b/arm/gic.c
> >> @@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
> >> smp_rmb(); /* pairs with wmb in lpi_stats_expect */
> >> lpi_stats.observed.cpu_id = smp_processor_id();
> >> lpi_stats.observed.lpi_id = irqnr;
> >> + acked[lpi_stats.observed.cpu_id]++;
> >> smp_wmb(); /* pairs with rmb in check_lpi_stats */
> >> }
> >>
> >> @@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
> >> while (1)
> >> wfi();
> >> }
> >> +
> >> +static void check_lpi_hits(int *expected)
> >> +{
> >> + int i;
> >> +
> >> + for (i = 0; i < nr_cpus; i++) {
> >> + if (acked[i] != expected[i])
> >> + report(false, "expected %d LPIs on PE #%d, %d observed",
> >> + expected[i], i, acked[i]);
> >
> > report_info
> > pass = false
> >
> >> + }
> >> + report(true, "check LPI on all vcpus");
> >
> > report(pass, ...);
> I still don't get the issue.
Your messages aren't consistent and have dynamic info. You need
report_info(...); // dynamic info
if (failure_condition) {
pass_boolean = false;
}
report(pass_boolean, single_message_for_both_pass_and_failure);
> >
> >> +}
> >> #endif
> >>
> >> static void gicv2_ipi_send_self(void)
> >> @@ -594,6 +607,8 @@ static void gic_test_mmio(void)
> >> static void test_its_introspection(void) {}
> >> static void test_its_trigger(void) {}
> >> static void test_its_migration(void) {}
> >> +static void test_migrate_unmapped_collection(void) {}
> >> +static void test_its_pending_migration(void) {}
> >
> > I'm not sure what's worse. This pile of stubs or one #ifdef in main()
> > wrapping all the calls.
> Those stubs now are in the arm header.
> >
> >>
> >> #else /* __arch64__ */
> >>
> >> @@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
> >> return false;
> >> }
> >>
> >> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
> >> + struct its_collection *col)
> >> +{
> >> + if (!dev || !col)
> >
> > I don't think col can be null, and this doesn't look like the right place
> > to check if dev is null. If we're bothiner to call set_lpi, then I
> > think we should already expect dev to be good to go.
> put an assert() instead
> >
> >> + report_abort("wrong device or collection");
> >> +
> >> + its_send_mapti(dev, physid, eventid, col);
> >> +
> >> + gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
> >> + its_send_invall(col);
> >> +}
> >> +
> >> /*
> >> * Setup the configuration for those mappings:
> >> * dev_id=2 event=20 -> vcpu 3, intid=8195
> >> @@ -806,6 +833,121 @@ static void test_its_migration(void)
> >> its_send_int(dev7, 255);
> >> check_lpi_stats();
> >> }
> >> +
> >> +static void test_migrate_unmapped_collection(void)
> >> +{
> >> + struct its_collection *col;
> >> + struct its_device *dev2, *dev7;
> >> + u8 config;
> >> +
> >> + if (its_setup1())
> >> + return;
> >> +
> >> + col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
> >> + dev2 = its_get_device(2);
> >> + dev7 = its_get_device(7);
> >> +
> >> + /* MAPTI with the collection unmapped */
> >> + set_lpi(dev2, 0, 8192, col);
> >> +
> >> + puts("Now migrate the VM, then press a key to continue...\n");
> >> + (void)getchar();
> >> + report(true, "Migration complete");
> >
> > report_info
> yep
> >
> >> +
> >> + /* on the destination, map the collection */
> >> + its_send_mapc(col, true);
> >> +
> >> + lpi_stats_expect(2, 8196);
> >> + its_send_int(dev7, 255);
> >> + check_lpi_stats();
> >> +
> >> + config = gicv3_lpi_get_config(8192);
> >> + report(config == LPI_PROP_DEFAULT,
> >> + "Config of LPI 8192 was properly migrated");
> >> +
> >> + lpi_stats_expect(nr_cpus - 1, 8192);
> >> + its_send_int(dev2, 0);
> >> + check_lpi_stats();
> >> +
> >> + /* unmap the collection */
> >> + its_send_mapc(col, false);
> >> +
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev2, 0);
> >> + check_lpi_stats();
> >> +
> >> + /* remap event 0 onto lpiid 8193 */
> >> + set_lpi(dev2, 0, 8193, col);
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev2, 0);
> >> + check_lpi_stats();
> >> +
> >> + /* remap the collection */
> >> + its_send_mapc(col, true);
> >> + lpi_stats_expect(nr_cpus - 1, 8193);
> >> +}
> >> +
> >> +static void test_its_pending_migration(void)
> >> +{
> >> + struct its_device *dev;
> >> + struct its_collection *collection[2];
> >> + int expected[NR_CPUS];
> >
> > expected = malloc(nr_cpus * sizeof(int));
> >
> > I know there are other places using NR_CPUS right now that don't have to,
> > but we shouldn't add more. Eventually I'll change the other places too.
> OK
> >
> >> + u64 pendbaser;
> >> + void *ptr;
> >> + int i;
> >> +
> >> + if (its_prerequisites(4))
> >> + return;
> >> +
> >> + dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> >> + its_send_mapd(dev, true);
> >> +
> >> + collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
> >> + collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
> >> + its_send_mapc(collection[0], true);
> >> + its_send_mapc(collection[1], true);
> >> +
> >> + /* disable lpi at redist level */
> >> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
> >> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
> >> +
> >> + /* even lpis are assigned to even cpu */
> >> + for (i = 0; i < 256; i++) {
> >> + struct its_collection *col = i % 2 ? collection[0] :
> >> + collection[1];
> >> + int vcpu = col->target_address >> 16;
> >
> > I'm lost with the even/odd (nr_cpus - 1)/(nr_cpus - 2) stuff, and won't
> > it swap if nr_cpus is odd vs. even?
> >
> > Shouldn't we just have something like
> >
> > pe1 = nr_cpus - 1;
> > pe2 = nr_cpus - 2;
> > col1 = its_create_collection(pe1, pe1);
> > col2 = its_create_collection(pe2, pe2);
> >
> > without mentioning even and odd?
> OK
> >
> >> +
> >> + its_send_mapti(dev, 8192 + i, i, col);
> >> + gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
> >> + gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
> >> + }
> >> + its_send_invall(collection[0]);
> >> + its_send_invall(collection[1]);
> >> +
> >> + /* Set the PTZ bit on each pendbaser */
> >> +
> >> + expected[nr_cpus - 1] = 128;
> >> + expected[nr_cpus - 2] = 128;
> >> +
> >> + ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
> >> + pendbaser = readq(ptr);
> >> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> >> +
> >> + ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
> >> + pendbaser = readq(ptr);
> >> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> >> +
> >> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
> >> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
> >> +
> >> + puts("Now migrate the VM, then press a key to continue...\n");
> >> + (void)getchar();
> >> + report(true, "Migration complete");
> >
> > report_info
> OK
> >
> >> +
> >> + mdelay(1000);
> >
> > This delay needs a comment explaining why it's here.
> OK
>
> Thanks
>
> Eric
> >
> >> +
> >> + check_lpi_hits(expected);
> >> +}
> >> #endif
> >>
> >> int main(int argc, char **argv)
> >> @@ -847,6 +989,14 @@ int main(int argc, char **argv)
> >> report_prefix_push(argv[1]);
> >> test_its_migration();
> >> report_prefix_pop();
> >> + } else if (!strcmp(argv[1], "its-pending-migration")) {
> >> + report_prefix_push(argv[1]);
> >> + test_its_pending_migration();
> >> + report_prefix_pop();
> >> + } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
> >> + report_prefix_push(argv[1]);
> >> + test_migrate_unmapped_collection();
> >> + report_prefix_pop();
> >> } else if (strcmp(argv[1], "its-introspection") == 0) {
> >> report_prefix_push(argv[1]);
> >> test_its_introspection();
> >> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> >> index 8b8ec79..d917157 100644
> >> --- a/arm/unittests.cfg
> >> +++ b/arm/unittests.cfg
> >> @@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
> >> groups = its migration
> >> arch = arm64
> >>
> >> +[its-pending-migration]
> >> +file = gic.flat
> >> +smp = $MAX_SMP
> >> +accel = kvm
> >> +extra_params = -machine gic-version=3 -append 'its-pending-migration'
> >> +groups = its migration
> >> +arch = arm64
> >> +
> >> +[its-migrate-unmapped-collection]
> >> +file = gic.flat
> >> +smp = $MAX_SMP
> >> +accel = kvm
> >> +extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
> >> +groups = its migration
> >> +arch = arm64
> >> +
> >> # Test PSCI emulation
> >> [psci]
> >> file = psci.flat
> >> --
> >> 2.20.1
> >>
> >>
> >
> > Thanks,
> > drew
> >
>
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
@ 2020-03-06 13:36 ` Andrew Jones
0 siblings, 0 replies; 137+ messages in thread
From: Andrew Jones @ 2020-03-06 13:36 UTC (permalink / raw)
To: Auger Eric
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
On Fri, Mar 06, 2020 at 02:21:37PM +0100, Auger Eric wrote:
> Hi Drew,
>
> On 2/7/20 3:06 PM, Andrew Jones wrote:
> > On Tue, Jan 28, 2020 at 11:34:59AM +0100, Eric Auger wrote:
> >> Add two new migration tests. One testing the migration of
> >> a topology where collection were unmapped. The second test
> >> checks the migration of the pending table.
> >>
> >> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> >>
> >> ---
> >>
> >> v2 -> v3:
> >> - tests belong to both its and migration groups
> >> ---
> >> arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
> >> arm/unittests.cfg | 16 +++++
> >> 2 files changed, 166 insertions(+)
> >>
> >> diff --git a/arm/gic.c b/arm/gic.c
> >> index fa8626a..ec3dd3a 100644
> >> --- a/arm/gic.c
> >> +++ b/arm/gic.c
> >> @@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
> >> smp_rmb(); /* pairs with wmb in lpi_stats_expect */
> >> lpi_stats.observed.cpu_id = smp_processor_id();
> >> lpi_stats.observed.lpi_id = irqnr;
> >> + acked[lpi_stats.observed.cpu_id]++;
> >> smp_wmb(); /* pairs with rmb in check_lpi_stats */
> >> }
> >>
> >> @@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
> >> while (1)
> >> wfi();
> >> }
> >> +
> >> +static void check_lpi_hits(int *expected)
> >> +{
> >> + int i;
> >> +
> >> + for (i = 0; i < nr_cpus; i++) {
> >> + if (acked[i] != expected[i])
> >> + report(false, "expected %d LPIs on PE #%d, %d observed",
> >> + expected[i], i, acked[i]);
> >
> > report_info
> > pass = false
> >
> >> + }
> >> + report(true, "check LPI on all vcpus");
> >
> > report(pass, ...);
> I still don't get the issue.
Your messages aren't consistent and have dynamic info. You need
report_info(...); // dynamic info
if (failure_condition) {
pass_boolean = false;
}
report(pass_boolean, single_message_for_both_pass_and_failure);
> >
> >> +}
> >> #endif
> >>
> >> static void gicv2_ipi_send_self(void)
> >> @@ -594,6 +607,8 @@ static void gic_test_mmio(void)
> >> static void test_its_introspection(void) {}
> >> static void test_its_trigger(void) {}
> >> static void test_its_migration(void) {}
> >> +static void test_migrate_unmapped_collection(void) {}
> >> +static void test_its_pending_migration(void) {}
> >
> > I'm not sure what's worse. This pile of stubs or one #ifdef in main()
> > wrapping all the calls.
> Those stubs now are in the arm header.
> >
> >>
> >> #else /* __arch64__ */
> >>
> >> @@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
> >> return false;
> >> }
> >>
> >> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
> >> + struct its_collection *col)
> >> +{
> >> + if (!dev || !col)
> >
> > I don't think col can be null, and this doesn't look like the right place
> > to check if dev is null. If we're bothiner to call set_lpi, then I
> > think we should already expect dev to be good to go.
> put an assert() instead
> >
> >> + report_abort("wrong device or collection");
> >> +
> >> + its_send_mapti(dev, physid, eventid, col);
> >> +
> >> + gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
> >> + its_send_invall(col);
> >> +}
> >> +
> >> /*
> >> * Setup the configuration for those mappings:
> >> * dev_id=2 event=20 -> vcpu 3, intid=8195
> >> @@ -806,6 +833,121 @@ static void test_its_migration(void)
> >> its_send_int(dev7, 255);
> >> check_lpi_stats();
> >> }
> >> +
> >> +static void test_migrate_unmapped_collection(void)
> >> +{
> >> + struct its_collection *col;
> >> + struct its_device *dev2, *dev7;
> >> + u8 config;
> >> +
> >> + if (its_setup1())
> >> + return;
> >> +
> >> + col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
> >> + dev2 = its_get_device(2);
> >> + dev7 = its_get_device(7);
> >> +
> >> + /* MAPTI with the collection unmapped */
> >> + set_lpi(dev2, 0, 8192, col);
> >> +
> >> + puts("Now migrate the VM, then press a key to continue...\n");
> >> + (void)getchar();
> >> + report(true, "Migration complete");
> >
> > report_info
> yep
> >
> >> +
> >> + /* on the destination, map the collection */
> >> + its_send_mapc(col, true);
> >> +
> >> + lpi_stats_expect(2, 8196);
> >> + its_send_int(dev7, 255);
> >> + check_lpi_stats();
> >> +
> >> + config = gicv3_lpi_get_config(8192);
> >> + report(config == LPI_PROP_DEFAULT,
> >> + "Config of LPI 8192 was properly migrated");
> >> +
> >> + lpi_stats_expect(nr_cpus - 1, 8192);
> >> + its_send_int(dev2, 0);
> >> + check_lpi_stats();
> >> +
> >> + /* unmap the collection */
> >> + its_send_mapc(col, false);
> >> +
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev2, 0);
> >> + check_lpi_stats();
> >> +
> >> + /* remap event 0 onto lpiid 8193 */
> >> + set_lpi(dev2, 0, 8193, col);
> >> + lpi_stats_expect(-1, -1);
> >> + its_send_int(dev2, 0);
> >> + check_lpi_stats();
> >> +
> >> + /* remap the collection */
> >> + its_send_mapc(col, true);
> >> + lpi_stats_expect(nr_cpus - 1, 8193);
> >> +}
> >> +
> >> +static void test_its_pending_migration(void)
> >> +{
> >> + struct its_device *dev;
> >> + struct its_collection *collection[2];
> >> + int expected[NR_CPUS];
> >
> > expected = malloc(nr_cpus * sizeof(int));
> >
> > I know there are other places using NR_CPUS right now that don't have to,
> > but we shouldn't add more. Eventually I'll change the other places too.
> OK
> >
> >> + u64 pendbaser;
> >> + void *ptr;
> >> + int i;
> >> +
> >> + if (its_prerequisites(4))
> >> + return;
> >> +
> >> + dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> >> + its_send_mapd(dev, true);
> >> +
> >> + collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
> >> + collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
> >> + its_send_mapc(collection[0], true);
> >> + its_send_mapc(collection[1], true);
> >> +
> >> + /* disable lpi at redist level */
> >> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
> >> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
> >> +
> >> + /* even lpis are assigned to even cpu */
> >> + for (i = 0; i < 256; i++) {
> >> + struct its_collection *col = i % 2 ? collection[0] :
> >> + collection[1];
> >> + int vcpu = col->target_address >> 16;
> >
> > I'm lost with the even/odd (nr_cpus - 1)/(nr_cpus - 2) stuff, and won't
> > it swap if nr_cpus is odd vs. even?
> >
> > Shouldn't we just have something like
> >
> > pe1 = nr_cpus - 1;
> > pe2 = nr_cpus - 2;
> > col1 = its_create_collection(pe1, pe1);
> > col2 = its_create_collection(pe2, pe2);
> >
> > without mentioning even and odd?
> OK
> >
> >> +
> >> + its_send_mapti(dev, 8192 + i, i, col);
> >> + gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
> >> + gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
> >> + }
> >> + its_send_invall(collection[0]);
> >> + its_send_invall(collection[1]);
> >> +
> >> + /* Set the PTZ bit on each pendbaser */
> >> +
> >> + expected[nr_cpus - 1] = 128;
> >> + expected[nr_cpus - 2] = 128;
> >> +
> >> + ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
> >> + pendbaser = readq(ptr);
> >> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> >> +
> >> + ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
> >> + pendbaser = readq(ptr);
> >> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> >> +
> >> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
> >> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
> >> +
> >> + puts("Now migrate the VM, then press a key to continue...\n");
> >> + (void)getchar();
> >> + report(true, "Migration complete");
> >
> > report_info
> OK
> >
> >> +
> >> + mdelay(1000);
> >
> > This delay needs a comment explaining why it's here.
> OK
>
> Thanks
>
> Eric
> >
> >> +
> >> + check_lpi_hits(expected);
> >> +}
> >> #endif
> >>
> >> int main(int argc, char **argv)
> >> @@ -847,6 +989,14 @@ int main(int argc, char **argv)
> >> report_prefix_push(argv[1]);
> >> test_its_migration();
> >> report_prefix_pop();
> >> + } else if (!strcmp(argv[1], "its-pending-migration")) {
> >> + report_prefix_push(argv[1]);
> >> + test_its_pending_migration();
> >> + report_prefix_pop();
> >> + } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
> >> + report_prefix_push(argv[1]);
> >> + test_migrate_unmapped_collection();
> >> + report_prefix_pop();
> >> } else if (strcmp(argv[1], "its-introspection") == 0) {
> >> report_prefix_push(argv[1]);
> >> test_its_introspection();
> >> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> >> index 8b8ec79..d917157 100644
> >> --- a/arm/unittests.cfg
> >> +++ b/arm/unittests.cfg
> >> @@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
> >> groups = its migration
> >> arch = arm64
> >>
> >> +[its-pending-migration]
> >> +file = gic.flat
> >> +smp = $MAX_SMP
> >> +accel = kvm
> >> +extra_params = -machine gic-version=3 -append 'its-pending-migration'
> >> +groups = its migration
> >> +arch = arm64
> >> +
> >> +[its-migrate-unmapped-collection]
> >> +file = gic.flat
> >> +smp = $MAX_SMP
> >> +accel = kvm
> >> +extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
> >> +groups = its migration
> >> +arch = arm64
> >> +
> >> # Test PSCI emulation
> >> [psci]
> >> file = psci.flat
> >> --
> >> 2.20.1
> >>
> >>
> >
> > Thanks,
> > drew
> >
>
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
2020-03-06 13:29 ` Andrew Jones
@ 2020-03-06 13:40 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 13:40 UTC (permalink / raw)
To: Andrew Jones
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
Hi Drew,
On 3/6/20 2:29 PM, Andrew Jones wrote:
> On Fri, Mar 06, 2020 at 01:55:09PM +0100, Auger Eric wrote:
>> Hi Drew,
>>
>> On 2/7/20 2:15 PM, Andrew Jones wrote:
>>> On Tue, Jan 28, 2020 at 11:34:56AM +0100, Eric Auger wrote:
>>>> Triggers LPIs through the INT command.
>>>>
>>>> the test checks the LPI hits the right CPU and triggers
>>>> the right LPI intid, ie. the translation is correct.
>>>>
>>>> Updates to the config table also are tested, along with inv
>>>> and invall commands.
>>>>
>>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>>
>>>> ---
>>>>
>>>> v2 -> v3:
>>>> - add comments
>>>> - keep the report_skip in case there aren't 4 vcpus to be able to
>>>> run other tests in the its category.
>>>> - fix the prefix pop
>>>> - move its_event and its_stats to arm/gic.c
>>>> ---
>>>> arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
>>>> arm/unittests.cfg | 7 ++
>>>> 2 files changed, 224 insertions(+), 11 deletions(-)
>>>>
>>>> diff --git a/arm/gic.c b/arm/gic.c
>>>> index 4d7dd03..50104b1 100644
>>>> --- a/arm/gic.c
>>>> +++ b/arm/gic.c
>>>> @@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
>>>> }
>>>> }
>>>>
>>>> +static void setup_irq(handler_t handler)
>>>> +{
>>>> + gic_enable_defaults();
>>>> +#ifdef __arm__
>>>> + install_exception_handler(EXCPTN_IRQ, handler);
>>>> +#else
>>>> + install_irq_handler(EL1H_IRQ, handler);
>>>> +#endif
>>>> + local_irq_enable();
>>>> +}
>>>> +
>>>> +#if defined(__aarch64__)
>>>> +struct its_event {
>>>> + int cpu_id;
>>>> + int lpi_id;
>>>> +};
>>>> +
>>>> +struct its_stats {
>>>> + struct its_event expected;
>>>> + struct its_event observed;
>>>> +};
>>>> +
>>>> +static struct its_stats lpi_stats;
>>>> +
>>>> +static void lpi_handler(struct pt_regs *regs __unused)
>>>> +{
>>>> + u32 irqstat = gic_read_iar();
>>>> + int irqnr = gic_iar_irqnr(irqstat);
>>>> +
>>>> + gic_write_eoir(irqstat);
>>>> + if (irqnr < 8192)
>>>> + report(false, "Unexpected non LPI interrupt received");
>>>
>>> report_info
>> why? This is an error case. We do not expect other interrupts than LPIs
>
> If there's almost no chance this will happen and it means something quite
> unexpected has occurred, then it should probably be an assert. If this is
> a real test case, then it should be
>
> report(irqnr >= 8192, "Got LPI");
>
> or something like that. If it's something that shouldn't ever happen, so
> it doesn't really deserve its own PASS/FAIL test output each execution
> of the unit test, but you don't want to assert for some reason, then it
> should be a report_info, but it should probably also contain a "WARNING"
> prefix in that case.
OK so the assert should be OK.
>
>>>
>>>> + smp_rmb(); /* pairs with wmb in lpi_stats_expect */
>>>> + lpi_stats.observed.cpu_id = smp_processor_id();
>>>> + lpi_stats.observed.lpi_id = irqnr;
>>>> + smp_wmb(); /* pairs with rmb in check_lpi_stats */
>>>> +}
>>>> +
>>>> +static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
>>>> +{
>>>> + lpi_stats.expected.cpu_id = exp_cpu_id;
>>>> + lpi_stats.expected.lpi_id = exp_lpi_id;
>>>> + lpi_stats.observed.cpu_id = -1;
>>>> + lpi_stats.observed.lpi_id = -1;
>>>> + smp_wmb(); /* pairs with rmb in handler */
>>>> +}
>>>> +
>>>> +static void check_lpi_stats(void)
>>>
>>> static void check_lpi_stats(const char *testname)
>>> {
>>> bool pass = false;
>>>
>>>> +{
>>>> + mdelay(100);
>>>> + smp_rmb(); /* pairs with wmb in lpi_handler */
>>>> + if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
>>>> + (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
>>>
>>> nit: extra ()
>>>
>>>> + if (lpi_stats.observed.cpu_id == -1 &&
>>>> + lpi_stats.observed.lpi_id == -1) {
>>>> + report(false,
>>>> + "No LPI received whereas (cpuid=%d, intid=%d) "
>>>> + "was expected", lpi_stats.expected.cpu_id,
>>>> + lpi_stats.expected.lpi_id);
>>>
>>> report_info
>> What's the problem keeping those. Those are error reports. The message
>> is something like that:
>> FAIL: gicv3: its-trigger: mapc valid=false: No LPI received whereas
>> (cpuid=1, intid=8192) was expected.
>>
>> So the testname is already part of the message.
>
> This one has two problems with being report() vs. report_info. The same
> comment as above, where the condition for a report() should be the test,
> rather than if (cond) report(false, ...), which implies it's not expected
> to report at all. A pattern like that needs to be extended at least to
> something like this
>
> if (cond)
> report(true, ...)
> else
> report(false, ...)
OK understood. I should have use the test as the 1st param.
>
> so we get the PASS/FAIL each execution. The other problem with this
> particular report() is the dynamic info in it (cpuid and maybe intid).
> A report() should only have consistent info so test output parsers
> can count on finding the PASS/FAIL for a given report line. If you
> need a test like this, then it can be structured like
>
> report_info(...); // dynamic info
> if (cond) {
> report(true, MSG1); // no dynamic info
> report(true, MSG2); // no dynamic info
> } else {
> report(false, MSG1); // no dynamic info
> report(false, MSG2); // no dynamic info
> }
>
> Notice how the MSG's match on both paths of the condition.
>
> Or just
>
> report_info(...);
> report(cond, ...);
OK I see what you mean now. I will rewrite it accordingly.
Thank you for the extra explanation
Eric
>
>>>
>>>> + } else {
>>>> + report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
>>>> + lpi_stats.observed.cpu_id,
>>>> + lpi_stats.observed.lpi_id);
>>>
>>> report_info
>>>
>>>> + }
>>>
>>> pass = false;
>>>
>>>> + } else if (lpi_stats.expected.lpi_id != -1) {
>>>> + report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
>>>> + lpi_stats.observed.cpu_id);
>>>
>>> report_info
>>>
>>>> + } else {
>>>> + report(true, "no LPI received, as expected");
>>>
>>> report_info
>
> This if, else if, ..., else with report() would be fine if the messages
> would all match, resulting in a single 'PASS: MSG' line. report_info can
> be used to get the dynamic info output too.
>
>>>
>>>
>>>> + }
>>>
>>> report(pass, "%s", testname);
>>>
>>>> +}
>>>> +
>>>> +static void secondary_lpi_test(void)
>>>> +{
>>>> + setup_irq(lpi_handler);
>>>> + cpumask_set_cpu(smp_processor_id(), &ready);
>>>> + while (1)
>>>> + wfi();
>>>> +}
>>>> +#endif
>>>> +
>>>> static void gicv2_ipi_send_self(void)
>>>> {
>>>> writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
>>>> @@ -217,17 +298,6 @@ static void ipi_test_smp(void)
>>>> report_prefix_pop();
>>>> }
>>>>
>>>> -static void setup_irq(handler_t handler)
>>>> -{
>>>> - gic_enable_defaults();
>>>> -#ifdef __arm__
>>>> - install_exception_handler(EXCPTN_IRQ, handler);
>>>> -#else
>>>> - install_irq_handler(EL1H_IRQ, handler);
>>>> -#endif
>>>> - local_irq_enable();
>>>> -}
>>>> -
>>>> static void ipi_send(void)
>>>> {
>>>> setup_irq(ipi_handler);
>>>> @@ -522,6 +592,7 @@ static void gic_test_mmio(void)
>>>> #if defined(__arm__)
>>>>
>>>> static void test_its_introspection(void) {}
>>>> +static void test_its_trigger(void) {}
>>>>
>>>> #else /* __arch64__ */
>>>>
>>>> @@ -561,6 +632,137 @@ static void test_its_introspection(void)
>>>> report_info("collection baser entry_size = 0x%x", coll_baser->esz);
>>>> }
>>>>
>>>> +static bool its_prerequisites(int nb_cpus)
>>>> +{
>>>> + int cpu;
>>>> +
>>>> + if (!gicv3_its_base()) {
>>>> + report_skip("No ITS, skip ...");
>>>> + return true;
>>>> + }
>>>> +
>>>> + if (nr_cpus < 4) {
>>>
>>> nr_cpus < nb_cpus, or just drop the nb_cpus parameter and hard code 4
>>> here.
>> sure
>>>
>>>> + report_skip("Test requires at least %d vcpus", nb_cpus);
>>>> + return true;
>>>> + }
>>>> +
>>>> + stats_reset();
>>>> +
>>>> + setup_irq(lpi_handler);
>>>> +
>>>> + for_each_present_cpu(cpu) {
>>>> + if (cpu == 0)
>>>> + continue;
>>>> + smp_boot_secondary(cpu, secondary_lpi_test);
>>>> + }
>>>> + wait_on_ready();
>>>> +
>>>> + its_enable_defaults();
>>>> +
>>>> + lpi_stats_expect(-1, -1);
>>>> + check_lpi_stats();
>>>> +
>>>> + return false;
>>>
>>> Reverse logic. I'd expect 'return true' for success.
>> I am going to return an int. In case of error a std negative error will
>> be returned.
>>>
>>>> +}
>>>> +
>>>> +static void test_its_trigger(void)
>>>> +{
>>>> + struct its_collection *col3, *col2;
>>>> + struct its_device *dev2, *dev7;
>>>> +
>>>> + if (its_prerequisites(4))
>>>
>>> if (!its_prerequisites(...))
>>>
>>>> + return;
>>>> +
>>>> + dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>>>> + dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
>>>> +
>>>> + col3 = its_create_collection(3 /* col id */, 3/* target PE */);
>>>> + col2 = its_create_collection(2 /* col id */, 2/* target PE */);
>>>> +
>>>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>>>> + gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
>>>> +
>>>> + its_send_invall(col2);
>>>> + its_send_invall(col3);
>>>> +
>>>> + report_prefix_push("int");
>>>> + /*
>>>> + * dev=2, eventid=20 -> lpi= 8195, col=3
>>>> + * dev=7, eventid=255 -> lpi= 8196, col=2
>>>> + * Trigger dev2, eventid=20 and dev7, eventid=255
>>>> + * Check both LPIs hit
>>>> + */
>>>> +
>>>> + its_send_mapd(dev2, true);
>>>> + its_send_mapd(dev7, true);
>>>> +
>>>> + its_send_mapc(col3, true);
>>>> + its_send_mapc(col2, true);
>>>> +
>>>> + its_send_mapti(dev2, 8195 /* lpi id */,
>>>> + 20 /* event id */, col3);
>>>> + its_send_mapti(dev7, 8196 /* lpi id */,
>>>> + 255 /* event id */, col2);
>>>
>>> No need for line breaks, with the embedded comments it's hard to read
>> OK
>>>
>>>> +
>>>> + lpi_stats_expect(3, 8195);
>>>> + its_send_int(dev2, 20);
>>>> + check_lpi_stats();
>>>> +
>>>> + lpi_stats_expect(2, 8196);
>>>> + its_send_int(dev7, 255);
>>>> + check_lpi_stats();
>>>> +
>>>> + report_prefix_pop();
>>>
>>> I think a table of parameters and loop would be nicer than all the
>>> repeated function calls.
>> Frankly speaking I am not sure this would really help. We are just
>> enabling 2 translation paths. I think I prefer to manipulate the low
>> level objects and helpers rather than playing with a loop and potential
>> new structs of params.
>
> OK, but you could probably at least wrap the common sequence into one
> function
>
> void master_function(a1, a2, a3, a4)
> {
> lpi_stats_expect(a1, a2);
> its_send_int(a3, a4);
> check_lpi_stats();
> }
>
> but whatever, it's not so important.
>
>>>
>>>> +
>>>> + report_prefix_push("inv/invall");
>>>> +
>>>> + /*
>>>> + * disable 8195, check dev2/eventid=20 does not trigger the
>>>> + * corresponding LPI
>>>> + */
>>>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
>>>
>>> LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED
>> ok
>>>
>>>> + its_send_inv(dev2, 20);
>>>> +
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev2, 20);
>>>> + check_lpi_stats();
>>>> +
>>>> + /*
>>>> + * re-enable the LPI but willingly do not call invall
>>>> + * so the change in config is not taken into account.
>>>> + * The LPI should not hit
>>>> + */
>>>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev2, 20);
>>>> + check_lpi_stats();
>>>> +
>>>> + /* Now call the invall and check the LPI hits */
>>>> + its_send_invall(col3);
>>>> + lpi_stats_expect(3, 8195);
>>>> + its_send_int(dev2, 20);
>>>> + check_lpi_stats();
>>>> +
>>>> + report_prefix_pop();
>>>
>>> Need blank line here.
>> OK
>>>
>>>> + /*
>>>> + * Unmap device 2 and check the eventid 20 formerly
>>>> + * attached to it does not hit anymore
>>>> + */
>>>> + report_prefix_push("mapd valid=false");
>>>
>>> Above you have the prefix-push before the comment explaining the test.
>>> After is probably better, but whatever, as long as it's consistent.
>> moved after
>>>
>>>> + its_send_mapd(dev2, false);
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev2, 20);
>>>> + check_lpi_stats();
>>>> + report_prefix_pop();
>>>> +
>>>> + /* Unmap the collection this time and check no LPI does hit */
>>>> + report_prefix_push("mapc valid=false");
>>>> + its_send_mapc(col2, false);
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev7, 255);
>>>> + check_lpi_stats();
>>>> + report_prefix_pop();
>>>> +}
>>>> #endif
>>>>
>>>> int main(int argc, char **argv)
>>>> @@ -594,6 +796,10 @@ int main(int argc, char **argv)
>>>> report_prefix_push(argv[1]);
>>>> gic_test_mmio();
>>>> report_prefix_pop();
>>>> + } else if (!strcmp(argv[1], "its-trigger")) {
>>>> + report_prefix_push(argv[1]);
>>>> + test_its_trigger();
>>>> + report_prefix_pop();
>>>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>>>> report_prefix_push(argv[1]);
>>>> test_its_introspection();
>>>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>>>> index ba2b31b..bfafec5 100644
>>>> --- a/arm/unittests.cfg
>>>> +++ b/arm/unittests.cfg
>>>> @@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
>>>> groups = its
>>>> arch = arm64
>>>>
>>>> +[its-trigger]
>>>> +file = gic.flat
>>>> +smp = $MAX_SMP
>>>> +extra_params = -machine gic-version=3 -append 'its-trigger'
>>>> +groups = its
>>>> +arch = arm64
>>>> +
>>>> # Test PSCI emulation
>>>> [psci]
>>>> file = psci.flat
>>>> --
>>>> 2.20.1
>>>>
>>>
>>> Thanks,
>>> drew
>>>
>> Thanks
>>
>> Eric
>>
>>
>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests
@ 2020-03-06 13:40 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 13:40 UTC (permalink / raw)
To: Andrew Jones
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
Hi Drew,
On 3/6/20 2:29 PM, Andrew Jones wrote:
> On Fri, Mar 06, 2020 at 01:55:09PM +0100, Auger Eric wrote:
>> Hi Drew,
>>
>> On 2/7/20 2:15 PM, Andrew Jones wrote:
>>> On Tue, Jan 28, 2020 at 11:34:56AM +0100, Eric Auger wrote:
>>>> Triggers LPIs through the INT command.
>>>>
>>>> the test checks the LPI hits the right CPU and triggers
>>>> the right LPI intid, ie. the translation is correct.
>>>>
>>>> Updates to the config table also are tested, along with inv
>>>> and invall commands.
>>>>
>>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>>
>>>> ---
>>>>
>>>> v2 -> v3:
>>>> - add comments
>>>> - keep the report_skip in case there aren't 4 vcpus to be able to
>>>> run other tests in the its category.
>>>> - fix the prefix pop
>>>> - move its_event and its_stats to arm/gic.c
>>>> ---
>>>> arm/gic.c | 228 +++++++++++++++++++++++++++++++++++++++++++---
>>>> arm/unittests.cfg | 7 ++
>>>> 2 files changed, 224 insertions(+), 11 deletions(-)
>>>>
>>>> diff --git a/arm/gic.c b/arm/gic.c
>>>> index 4d7dd03..50104b1 100644
>>>> --- a/arm/gic.c
>>>> +++ b/arm/gic.c
>>>> @@ -160,6 +160,87 @@ static void ipi_handler(struct pt_regs *regs __unused)
>>>> }
>>>> }
>>>>
>>>> +static void setup_irq(handler_t handler)
>>>> +{
>>>> + gic_enable_defaults();
>>>> +#ifdef __arm__
>>>> + install_exception_handler(EXCPTN_IRQ, handler);
>>>> +#else
>>>> + install_irq_handler(EL1H_IRQ, handler);
>>>> +#endif
>>>> + local_irq_enable();
>>>> +}
>>>> +
>>>> +#if defined(__aarch64__)
>>>> +struct its_event {
>>>> + int cpu_id;
>>>> + int lpi_id;
>>>> +};
>>>> +
>>>> +struct its_stats {
>>>> + struct its_event expected;
>>>> + struct its_event observed;
>>>> +};
>>>> +
>>>> +static struct its_stats lpi_stats;
>>>> +
>>>> +static void lpi_handler(struct pt_regs *regs __unused)
>>>> +{
>>>> + u32 irqstat = gic_read_iar();
>>>> + int irqnr = gic_iar_irqnr(irqstat);
>>>> +
>>>> + gic_write_eoir(irqstat);
>>>> + if (irqnr < 8192)
>>>> + report(false, "Unexpected non LPI interrupt received");
>>>
>>> report_info
>> why? This is an error case. We do not expect other interrupts than LPIs
>
> If there's almost no chance this will happen and it means something quite
> unexpected has occurred, then it should probably be an assert. If this is
> a real test case, then it should be
>
> report(irqnr >= 8192, "Got LPI");
>
> or something like that. If it's something that shouldn't ever happen, so
> it doesn't really deserve its own PASS/FAIL test output each execution
> of the unit test, but you don't want to assert for some reason, then it
> should be a report_info, but it should probably also contain a "WARNING"
> prefix in that case.
OK so the assert should be OK.
>
>>>
>>>> + smp_rmb(); /* pairs with wmb in lpi_stats_expect */
>>>> + lpi_stats.observed.cpu_id = smp_processor_id();
>>>> + lpi_stats.observed.lpi_id = irqnr;
>>>> + smp_wmb(); /* pairs with rmb in check_lpi_stats */
>>>> +}
>>>> +
>>>> +static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
>>>> +{
>>>> + lpi_stats.expected.cpu_id = exp_cpu_id;
>>>> + lpi_stats.expected.lpi_id = exp_lpi_id;
>>>> + lpi_stats.observed.cpu_id = -1;
>>>> + lpi_stats.observed.lpi_id = -1;
>>>> + smp_wmb(); /* pairs with rmb in handler */
>>>> +}
>>>> +
>>>> +static void check_lpi_stats(void)
>>>
>>> static void check_lpi_stats(const char *testname)
>>> {
>>> bool pass = false;
>>>
>>>> +{
>>>> + mdelay(100);
>>>> + smp_rmb(); /* pairs with wmb in lpi_handler */
>>>> + if ((lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id) ||
>>>> + (lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id)) {
>>>
>>> nit: extra ()
>>>
>>>> + if (lpi_stats.observed.cpu_id == -1 &&
>>>> + lpi_stats.observed.lpi_id == -1) {
>>>> + report(false,
>>>> + "No LPI received whereas (cpuid=%d, intid=%d) "
>>>> + "was expected", lpi_stats.expected.cpu_id,
>>>> + lpi_stats.expected.lpi_id);
>>>
>>> report_info
>> What's the problem keeping those. Those are error reports. The message
>> is something like that:
>> FAIL: gicv3: its-trigger: mapc valid=false: No LPI received whereas
>> (cpuid=1, intid=8192) was expected.
>>
>> So the testname is already part of the message.
>
> This one has two problems with being report() vs. report_info. The same
> comment as above, where the condition for a report() should be the test,
> rather than if (cond) report(false, ...), which implies it's not expected
> to report at all. A pattern like that needs to be extended at least to
> something like this
>
> if (cond)
> report(true, ...)
> else
> report(false, ...)
OK understood. I should have use the test as the 1st param.
>
> so we get the PASS/FAIL each execution. The other problem with this
> particular report() is the dynamic info in it (cpuid and maybe intid).
> A report() should only have consistent info so test output parsers
> can count on finding the PASS/FAIL for a given report line. If you
> need a test like this, then it can be structured like
>
> report_info(...); // dynamic info
> if (cond) {
> report(true, MSG1); // no dynamic info
> report(true, MSG2); // no dynamic info
> } else {
> report(false, MSG1); // no dynamic info
> report(false, MSG2); // no dynamic info
> }
>
> Notice how the MSG's match on both paths of the condition.
>
> Or just
>
> report_info(...);
> report(cond, ...);
OK I see what you mean now. I will rewrite it accordingly.
Thank you for the extra explanation
Eric
>
>>>
>>>> + } else {
>>>> + report(false, "Unexpected LPI (cpuid=%d, intid=%d)",
>>>> + lpi_stats.observed.cpu_id,
>>>> + lpi_stats.observed.lpi_id);
>>>
>>> report_info
>>>
>>>> + }
>>>
>>> pass = false;
>>>
>>>> + } else if (lpi_stats.expected.lpi_id != -1) {
>>>> + report(true, "LPI %d on CPU %d", lpi_stats.observed.lpi_id,
>>>> + lpi_stats.observed.cpu_id);
>>>
>>> report_info
>>>
>>>> + } else {
>>>> + report(true, "no LPI received, as expected");
>>>
>>> report_info
>
> This if, else if, ..., else with report() would be fine if the messages
> would all match, resulting in a single 'PASS: MSG' line. report_info can
> be used to get the dynamic info output too.
>
>>>
>>>
>>>> + }
>>>
>>> report(pass, "%s", testname);
>>>
>>>> +}
>>>> +
>>>> +static void secondary_lpi_test(void)
>>>> +{
>>>> + setup_irq(lpi_handler);
>>>> + cpumask_set_cpu(smp_processor_id(), &ready);
>>>> + while (1)
>>>> + wfi();
>>>> +}
>>>> +#endif
>>>> +
>>>> static void gicv2_ipi_send_self(void)
>>>> {
>>>> writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
>>>> @@ -217,17 +298,6 @@ static void ipi_test_smp(void)
>>>> report_prefix_pop();
>>>> }
>>>>
>>>> -static void setup_irq(handler_t handler)
>>>> -{
>>>> - gic_enable_defaults();
>>>> -#ifdef __arm__
>>>> - install_exception_handler(EXCPTN_IRQ, handler);
>>>> -#else
>>>> - install_irq_handler(EL1H_IRQ, handler);
>>>> -#endif
>>>> - local_irq_enable();
>>>> -}
>>>> -
>>>> static void ipi_send(void)
>>>> {
>>>> setup_irq(ipi_handler);
>>>> @@ -522,6 +592,7 @@ static void gic_test_mmio(void)
>>>> #if defined(__arm__)
>>>>
>>>> static void test_its_introspection(void) {}
>>>> +static void test_its_trigger(void) {}
>>>>
>>>> #else /* __arch64__ */
>>>>
>>>> @@ -561,6 +632,137 @@ static void test_its_introspection(void)
>>>> report_info("collection baser entry_size = 0x%x", coll_baser->esz);
>>>> }
>>>>
>>>> +static bool its_prerequisites(int nb_cpus)
>>>> +{
>>>> + int cpu;
>>>> +
>>>> + if (!gicv3_its_base()) {
>>>> + report_skip("No ITS, skip ...");
>>>> + return true;
>>>> + }
>>>> +
>>>> + if (nr_cpus < 4) {
>>>
>>> nr_cpus < nb_cpus, or just drop the nb_cpus parameter and hard code 4
>>> here.
>> sure
>>>
>>>> + report_skip("Test requires at least %d vcpus", nb_cpus);
>>>> + return true;
>>>> + }
>>>> +
>>>> + stats_reset();
>>>> +
>>>> + setup_irq(lpi_handler);
>>>> +
>>>> + for_each_present_cpu(cpu) {
>>>> + if (cpu == 0)
>>>> + continue;
>>>> + smp_boot_secondary(cpu, secondary_lpi_test);
>>>> + }
>>>> + wait_on_ready();
>>>> +
>>>> + its_enable_defaults();
>>>> +
>>>> + lpi_stats_expect(-1, -1);
>>>> + check_lpi_stats();
>>>> +
>>>> + return false;
>>>
>>> Reverse logic. I'd expect 'return true' for success.
>> I am going to return an int. In case of error a std negative error will
>> be returned.
>>>
>>>> +}
>>>> +
>>>> +static void test_its_trigger(void)
>>>> +{
>>>> + struct its_collection *col3, *col2;
>>>> + struct its_device *dev2, *dev7;
>>>> +
>>>> + if (its_prerequisites(4))
>>>
>>> if (!its_prerequisites(...))
>>>
>>>> + return;
>>>> +
>>>> + dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>>>> + dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
>>>> +
>>>> + col3 = its_create_collection(3 /* col id */, 3/* target PE */);
>>>> + col2 = its_create_collection(2 /* col id */, 2/* target PE */);
>>>> +
>>>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>>>> + gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
>>>> +
>>>> + its_send_invall(col2);
>>>> + its_send_invall(col3);
>>>> +
>>>> + report_prefix_push("int");
>>>> + /*
>>>> + * dev=2, eventid=20 -> lpi= 8195, col=3
>>>> + * dev=7, eventid=255 -> lpi= 8196, col=2
>>>> + * Trigger dev2, eventid=20 and dev7, eventid=255
>>>> + * Check both LPIs hit
>>>> + */
>>>> +
>>>> + its_send_mapd(dev2, true);
>>>> + its_send_mapd(dev7, true);
>>>> +
>>>> + its_send_mapc(col3, true);
>>>> + its_send_mapc(col2, true);
>>>> +
>>>> + its_send_mapti(dev2, 8195 /* lpi id */,
>>>> + 20 /* event id */, col3);
>>>> + its_send_mapti(dev7, 8196 /* lpi id */,
>>>> + 255 /* event id */, col2);
>>>
>>> No need for line breaks, with the embedded comments it's hard to read
>> OK
>>>
>>>> +
>>>> + lpi_stats_expect(3, 8195);
>>>> + its_send_int(dev2, 20);
>>>> + check_lpi_stats();
>>>> +
>>>> + lpi_stats_expect(2, 8196);
>>>> + its_send_int(dev7, 255);
>>>> + check_lpi_stats();
>>>> +
>>>> + report_prefix_pop();
>>>
>>> I think a table of parameters and loop would be nicer than all the
>>> repeated function calls.
>> Frankly speaking I am not sure this would really help. We are just
>> enabling 2 translation paths. I think I prefer to manipulate the low
>> level objects and helpers rather than playing with a loop and potential
>> new structs of params.
>
> OK, but you could probably at least wrap the common sequence into one
> function
>
> void master_function(a1, a2, a3, a4)
> {
> lpi_stats_expect(a1, a2);
> its_send_int(a3, a4);
> check_lpi_stats();
> }
>
> but whatever, it's not so important.
>
>>>
>>>> +
>>>> + report_prefix_push("inv/invall");
>>>> +
>>>> + /*
>>>> + * disable 8195, check dev2/eventid=20 does not trigger the
>>>> + * corresponding LPI
>>>> + */
>>>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~0x1);
>>>
>>> LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED
>> ok
>>>
>>>> + its_send_inv(dev2, 20);
>>>> +
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev2, 20);
>>>> + check_lpi_stats();
>>>> +
>>>> + /*
>>>> + * re-enable the LPI but willingly do not call invall
>>>> + * so the change in config is not taken into account.
>>>> + * The LPI should not hit
>>>> + */
>>>> + gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev2, 20);
>>>> + check_lpi_stats();
>>>> +
>>>> + /* Now call the invall and check the LPI hits */
>>>> + its_send_invall(col3);
>>>> + lpi_stats_expect(3, 8195);
>>>> + its_send_int(dev2, 20);
>>>> + check_lpi_stats();
>>>> +
>>>> + report_prefix_pop();
>>>
>>> Need blank line here.
>> OK
>>>
>>>> + /*
>>>> + * Unmap device 2 and check the eventid 20 formerly
>>>> + * attached to it does not hit anymore
>>>> + */
>>>> + report_prefix_push("mapd valid=false");
>>>
>>> Above you have the prefix-push before the comment explaining the test.
>>> After is probably better, but whatever, as long as it's consistent.
>> moved after
>>>
>>>> + its_send_mapd(dev2, false);
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev2, 20);
>>>> + check_lpi_stats();
>>>> + report_prefix_pop();
>>>> +
>>>> + /* Unmap the collection this time and check no LPI does hit */
>>>> + report_prefix_push("mapc valid=false");
>>>> + its_send_mapc(col2, false);
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev7, 255);
>>>> + check_lpi_stats();
>>>> + report_prefix_pop();
>>>> +}
>>>> #endif
>>>>
>>>> int main(int argc, char **argv)
>>>> @@ -594,6 +796,10 @@ int main(int argc, char **argv)
>>>> report_prefix_push(argv[1]);
>>>> gic_test_mmio();
>>>> report_prefix_pop();
>>>> + } else if (!strcmp(argv[1], "its-trigger")) {
>>>> + report_prefix_push(argv[1]);
>>>> + test_its_trigger();
>>>> + report_prefix_pop();
>>>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>>>> report_prefix_push(argv[1]);
>>>> test_its_introspection();
>>>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>>>> index ba2b31b..bfafec5 100644
>>>> --- a/arm/unittests.cfg
>>>> +++ b/arm/unittests.cfg
>>>> @@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
>>>> groups = its
>>>> arch = arm64
>>>>
>>>> +[its-trigger]
>>>> +file = gic.flat
>>>> +smp = $MAX_SMP
>>>> +extra_params = -machine gic-version=3 -append 'its-trigger'
>>>> +groups = its
>>>> +arch = arm64
>>>> +
>>>> # Test PSCI emulation
>>>> [psci]
>>>> file = psci.flat
>>>> --
>>>> 2.20.1
>>>>
>>>
>>> Thanks,
>>> drew
>>>
>> Thanks
>>
>> Eric
>>
>>
>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
2020-03-06 13:36 ` Andrew Jones
@ 2020-03-06 13:41 ` Auger Eric
-1 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 13:41 UTC (permalink / raw)
To: Andrew Jones
Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
andre.przywara, yuzenghui, alexandru.elisei, kvmarm,
eric.auger.pro
Hi
On 3/6/20 2:36 PM, Andrew Jones wrote:
> On Fri, Mar 06, 2020 at 02:21:37PM +0100, Auger Eric wrote:
>> Hi Drew,
>>
>> On 2/7/20 3:06 PM, Andrew Jones wrote:
>>> On Tue, Jan 28, 2020 at 11:34:59AM +0100, Eric Auger wrote:
>>>> Add two new migration tests. One testing the migration of
>>>> a topology where collection were unmapped. The second test
>>>> checks the migration of the pending table.
>>>>
>>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>>
>>>> ---
>>>>
>>>> v2 -> v3:
>>>> - tests belong to both its and migration groups
>>>> ---
>>>> arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
>>>> arm/unittests.cfg | 16 +++++
>>>> 2 files changed, 166 insertions(+)
>>>>
>>>> diff --git a/arm/gic.c b/arm/gic.c
>>>> index fa8626a..ec3dd3a 100644
>>>> --- a/arm/gic.c
>>>> +++ b/arm/gic.c
>>>> @@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
>>>> smp_rmb(); /* pairs with wmb in lpi_stats_expect */
>>>> lpi_stats.observed.cpu_id = smp_processor_id();
>>>> lpi_stats.observed.lpi_id = irqnr;
>>>> + acked[lpi_stats.observed.cpu_id]++;
>>>> smp_wmb(); /* pairs with rmb in check_lpi_stats */
>>>> }
>>>>
>>>> @@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
>>>> while (1)
>>>> wfi();
>>>> }
>>>> +
>>>> +static void check_lpi_hits(int *expected)
>>>> +{
>>>> + int i;
>>>> +
>>>> + for (i = 0; i < nr_cpus; i++) {
>>>> + if (acked[i] != expected[i])
>>>> + report(false, "expected %d LPIs on PE #%d, %d observed",
>>>> + expected[i], i, acked[i]);
>>>
>>> report_info
>>> pass = false
>>>
>>>> + }
>>>> + report(true, "check LPI on all vcpus");
>>>
>>> report(pass, ...);
>> I still don't get the issue.
>
> Your messages aren't consistent and have dynamic info. You need
>
> report_info(...); // dynamic info
> if (failure_condition) {
> pass_boolean = false;
> }
> report(pass_boolean, single_message_for_both_pass_and_failure);
OK understood.
Thanks!
Eric
>
>>>
>>>> +}
>>>> #endif
>>>>
>>>> static void gicv2_ipi_send_self(void)
>>>> @@ -594,6 +607,8 @@ static void gic_test_mmio(void)
>>>> static void test_its_introspection(void) {}
>>>> static void test_its_trigger(void) {}
>>>> static void test_its_migration(void) {}
>>>> +static void test_migrate_unmapped_collection(void) {}
>>>> +static void test_its_pending_migration(void) {}
>>>
>>> I'm not sure what's worse. This pile of stubs or one #ifdef in main()
>>> wrapping all the calls.
>> Those stubs now are in the arm header.
>>>
>>>>
>>>> #else /* __arch64__ */
>>>>
>>>> @@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
>>>> return false;
>>>> }
>>>>
>>>> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
>>>> + struct its_collection *col)
>>>> +{
>>>> + if (!dev || !col)
>>>
>>> I don't think col can be null, and this doesn't look like the right place
>>> to check if dev is null. If we're bothiner to call set_lpi, then I
>>> think we should already expect dev to be good to go.
>> put an assert() instead
>>>
>>>> + report_abort("wrong device or collection");
>>>> +
>>>> + its_send_mapti(dev, physid, eventid, col);
>>>> +
>>>> + gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
>>>> + its_send_invall(col);
>>>> +}
>>>> +
>>>> /*
>>>> * Setup the configuration for those mappings:
>>>> * dev_id=2 event=20 -> vcpu 3, intid=8195
>>>> @@ -806,6 +833,121 @@ static void test_its_migration(void)
>>>> its_send_int(dev7, 255);
>>>> check_lpi_stats();
>>>> }
>>>> +
>>>> +static void test_migrate_unmapped_collection(void)
>>>> +{
>>>> + struct its_collection *col;
>>>> + struct its_device *dev2, *dev7;
>>>> + u8 config;
>>>> +
>>>> + if (its_setup1())
>>>> + return;
>>>> +
>>>> + col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
>>>> + dev2 = its_get_device(2);
>>>> + dev7 = its_get_device(7);
>>>> +
>>>> + /* MAPTI with the collection unmapped */
>>>> + set_lpi(dev2, 0, 8192, col);
>>>> +
>>>> + puts("Now migrate the VM, then press a key to continue...\n");
>>>> + (void)getchar();
>>>> + report(true, "Migration complete");
>>>
>>> report_info
>> yep
>>>
>>>> +
>>>> + /* on the destination, map the collection */
>>>> + its_send_mapc(col, true);
>>>> +
>>>> + lpi_stats_expect(2, 8196);
>>>> + its_send_int(dev7, 255);
>>>> + check_lpi_stats();
>>>> +
>>>> + config = gicv3_lpi_get_config(8192);
>>>> + report(config == LPI_PROP_DEFAULT,
>>>> + "Config of LPI 8192 was properly migrated");
>>>> +
>>>> + lpi_stats_expect(nr_cpus - 1, 8192);
>>>> + its_send_int(dev2, 0);
>>>> + check_lpi_stats();
>>>> +
>>>> + /* unmap the collection */
>>>> + its_send_mapc(col, false);
>>>> +
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev2, 0);
>>>> + check_lpi_stats();
>>>> +
>>>> + /* remap event 0 onto lpiid 8193 */
>>>> + set_lpi(dev2, 0, 8193, col);
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev2, 0);
>>>> + check_lpi_stats();
>>>> +
>>>> + /* remap the collection */
>>>> + its_send_mapc(col, true);
>>>> + lpi_stats_expect(nr_cpus - 1, 8193);
>>>> +}
>>>> +
>>>> +static void test_its_pending_migration(void)
>>>> +{
>>>> + struct its_device *dev;
>>>> + struct its_collection *collection[2];
>>>> + int expected[NR_CPUS];
>>>
>>> expected = malloc(nr_cpus * sizeof(int));
>>>
>>> I know there are other places using NR_CPUS right now that don't have to,
>>> but we shouldn't add more. Eventually I'll change the other places too.
>> OK
>>>
>>>> + u64 pendbaser;
>>>> + void *ptr;
>>>> + int i;
>>>> +
>>>> + if (its_prerequisites(4))
>>>> + return;
>>>> +
>>>> + dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>>>> + its_send_mapd(dev, true);
>>>> +
>>>> + collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
>>>> + collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
>>>> + its_send_mapc(collection[0], true);
>>>> + its_send_mapc(collection[1], true);
>>>> +
>>>> + /* disable lpi at redist level */
>>>> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
>>>> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
>>>> +
>>>> + /* even lpis are assigned to even cpu */
>>>> + for (i = 0; i < 256; i++) {
>>>> + struct its_collection *col = i % 2 ? collection[0] :
>>>> + collection[1];
>>>> + int vcpu = col->target_address >> 16;
>>>
>>> I'm lost with the even/odd (nr_cpus - 1)/(nr_cpus - 2) stuff, and won't
>>> it swap if nr_cpus is odd vs. even?
>>>
>>> Shouldn't we just have something like
>>>
>>> pe1 = nr_cpus - 1;
>>> pe2 = nr_cpus - 2;
>>> col1 = its_create_collection(pe1, pe1);
>>> col2 = its_create_collection(pe2, pe2);
>>>
>>> without mentioning even and odd?
>> OK
>>>
>>>> +
>>>> + its_send_mapti(dev, 8192 + i, i, col);
>>>> + gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
>>>> + gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
>>>> + }
>>>> + its_send_invall(collection[0]);
>>>> + its_send_invall(collection[1]);
>>>> +
>>>> + /* Set the PTZ bit on each pendbaser */
>>>> +
>>>> + expected[nr_cpus - 1] = 128;
>>>> + expected[nr_cpus - 2] = 128;
>>>> +
>>>> + ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
>>>> + pendbaser = readq(ptr);
>>>> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>>>> +
>>>> + ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
>>>> + pendbaser = readq(ptr);
>>>> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>>>> +
>>>> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
>>>> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
>>>> +
>>>> + puts("Now migrate the VM, then press a key to continue...\n");
>>>> + (void)getchar();
>>>> + report(true, "Migration complete");
>>>
>>> report_info
>> OK
>>>
>>>> +
>>>> + mdelay(1000);
>>>
>>> This delay needs a comment explaining why it's here.
>> OK
>>
>> Thanks
>>
>> Eric
>>>
>>>> +
>>>> + check_lpi_hits(expected);
>>>> +}
>>>> #endif
>>>>
>>>> int main(int argc, char **argv)
>>>> @@ -847,6 +989,14 @@ int main(int argc, char **argv)
>>>> report_prefix_push(argv[1]);
>>>> test_its_migration();
>>>> report_prefix_pop();
>>>> + } else if (!strcmp(argv[1], "its-pending-migration")) {
>>>> + report_prefix_push(argv[1]);
>>>> + test_its_pending_migration();
>>>> + report_prefix_pop();
>>>> + } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
>>>> + report_prefix_push(argv[1]);
>>>> + test_migrate_unmapped_collection();
>>>> + report_prefix_pop();
>>>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>>>> report_prefix_push(argv[1]);
>>>> test_its_introspection();
>>>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>>>> index 8b8ec79..d917157 100644
>>>> --- a/arm/unittests.cfg
>>>> +++ b/arm/unittests.cfg
>>>> @@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
>>>> groups = its migration
>>>> arch = arm64
>>>>
>>>> +[its-pending-migration]
>>>> +file = gic.flat
>>>> +smp = $MAX_SMP
>>>> +accel = kvm
>>>> +extra_params = -machine gic-version=3 -append 'its-pending-migration'
>>>> +groups = its migration
>>>> +arch = arm64
>>>> +
>>>> +[its-migrate-unmapped-collection]
>>>> +file = gic.flat
>>>> +smp = $MAX_SMP
>>>> +accel = kvm
>>>> +extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
>>>> +groups = its migration
>>>> +arch = arm64
>>>> +
>>>> # Test PSCI emulation
>>>> [psci]
>>>> file = psci.flat
>>>> --
>>>> 2.20.1
>>>>
>>>>
>>>
>>> Thanks,
>>> drew
>>>
>>
>>
^ permalink raw reply [flat|nested] 137+ messages in thread
* Re: [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test
@ 2020-03-06 13:41 ` Auger Eric
0 siblings, 0 replies; 137+ messages in thread
From: Auger Eric @ 2020-03-06 13:41 UTC (permalink / raw)
To: Andrew Jones
Cc: thuth, kvm, maz, qemu-devel, qemu-arm, andre.przywara, kvmarm,
eric.auger.pro
Hi
On 3/6/20 2:36 PM, Andrew Jones wrote:
> On Fri, Mar 06, 2020 at 02:21:37PM +0100, Auger Eric wrote:
>> Hi Drew,
>>
>> On 2/7/20 3:06 PM, Andrew Jones wrote:
>>> On Tue, Jan 28, 2020 at 11:34:59AM +0100, Eric Auger wrote:
>>>> Add two new migration tests. One testing the migration of
>>>> a topology where collection were unmapped. The second test
>>>> checks the migration of the pending table.
>>>>
>>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>>
>>>> ---
>>>>
>>>> v2 -> v3:
>>>> - tests belong to both its and migration groups
>>>> ---
>>>> arm/gic.c | 150 ++++++++++++++++++++++++++++++++++++++++++++++
>>>> arm/unittests.cfg | 16 +++++
>>>> 2 files changed, 166 insertions(+)
>>>>
>>>> diff --git a/arm/gic.c b/arm/gic.c
>>>> index fa8626a..ec3dd3a 100644
>>>> --- a/arm/gic.c
>>>> +++ b/arm/gic.c
>>>> @@ -195,6 +195,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
>>>> smp_rmb(); /* pairs with wmb in lpi_stats_expect */
>>>> lpi_stats.observed.cpu_id = smp_processor_id();
>>>> lpi_stats.observed.lpi_id = irqnr;
>>>> + acked[lpi_stats.observed.cpu_id]++;
>>>> smp_wmb(); /* pairs with rmb in check_lpi_stats */
>>>> }
>>>>
>>>> @@ -239,6 +240,18 @@ static void secondary_lpi_test(void)
>>>> while (1)
>>>> wfi();
>>>> }
>>>> +
>>>> +static void check_lpi_hits(int *expected)
>>>> +{
>>>> + int i;
>>>> +
>>>> + for (i = 0; i < nr_cpus; i++) {
>>>> + if (acked[i] != expected[i])
>>>> + report(false, "expected %d LPIs on PE #%d, %d observed",
>>>> + expected[i], i, acked[i]);
>>>
>>> report_info
>>> pass = false
>>>
>>>> + }
>>>> + report(true, "check LPI on all vcpus");
>>>
>>> report(pass, ...);
>> I still don't get the issue.
>
> Your messages aren't consistent and have dynamic info. You need
>
> report_info(...); // dynamic info
> if (failure_condition) {
> pass_boolean = false;
> }
> report(pass_boolean, single_message_for_both_pass_and_failure);
OK understood.
Thanks!
Eric
>
>>>
>>>> +}
>>>> #endif
>>>>
>>>> static void gicv2_ipi_send_self(void)
>>>> @@ -594,6 +607,8 @@ static void gic_test_mmio(void)
>>>> static void test_its_introspection(void) {}
>>>> static void test_its_trigger(void) {}
>>>> static void test_its_migration(void) {}
>>>> +static void test_migrate_unmapped_collection(void) {}
>>>> +static void test_its_pending_migration(void) {}
>>>
>>> I'm not sure what's worse. This pile of stubs or one #ifdef in main()
>>> wrapping all the calls.
>> Those stubs now are in the arm header.
>>>
>>>>
>>>> #else /* __arch64__ */
>>>>
>>>> @@ -666,6 +681,18 @@ static bool its_prerequisites(int nb_cpus)
>>>> return false;
>>>> }
>>>>
>>>> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
>>>> + struct its_collection *col)
>>>> +{
>>>> + if (!dev || !col)
>>>
>>> I don't think col can be null, and this doesn't look like the right place
>>> to check if dev is null. If we're bothiner to call set_lpi, then I
>>> think we should already expect dev to be good to go.
>> put an assert() instead
>>>
>>>> + report_abort("wrong device or collection");
>>>> +
>>>> + its_send_mapti(dev, physid, eventid, col);
>>>> +
>>>> + gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
>>>> + its_send_invall(col);
>>>> +}
>>>> +
>>>> /*
>>>> * Setup the configuration for those mappings:
>>>> * dev_id=2 event=20 -> vcpu 3, intid=8195
>>>> @@ -806,6 +833,121 @@ static void test_its_migration(void)
>>>> its_send_int(dev7, 255);
>>>> check_lpi_stats();
>>>> }
>>>> +
>>>> +static void test_migrate_unmapped_collection(void)
>>>> +{
>>>> + struct its_collection *col;
>>>> + struct its_device *dev2, *dev7;
>>>> + u8 config;
>>>> +
>>>> + if (its_setup1())
>>>> + return;
>>>> +
>>>> + col = its_create_collection(nr_cpus - 1, nr_cpus - 1);
>>>> + dev2 = its_get_device(2);
>>>> + dev7 = its_get_device(7);
>>>> +
>>>> + /* MAPTI with the collection unmapped */
>>>> + set_lpi(dev2, 0, 8192, col);
>>>> +
>>>> + puts("Now migrate the VM, then press a key to continue...\n");
>>>> + (void)getchar();
>>>> + report(true, "Migration complete");
>>>
>>> report_info
>> yep
>>>
>>>> +
>>>> + /* on the destination, map the collection */
>>>> + its_send_mapc(col, true);
>>>> +
>>>> + lpi_stats_expect(2, 8196);
>>>> + its_send_int(dev7, 255);
>>>> + check_lpi_stats();
>>>> +
>>>> + config = gicv3_lpi_get_config(8192);
>>>> + report(config == LPI_PROP_DEFAULT,
>>>> + "Config of LPI 8192 was properly migrated");
>>>> +
>>>> + lpi_stats_expect(nr_cpus - 1, 8192);
>>>> + its_send_int(dev2, 0);
>>>> + check_lpi_stats();
>>>> +
>>>> + /* unmap the collection */
>>>> + its_send_mapc(col, false);
>>>> +
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev2, 0);
>>>> + check_lpi_stats();
>>>> +
>>>> + /* remap event 0 onto lpiid 8193 */
>>>> + set_lpi(dev2, 0, 8193, col);
>>>> + lpi_stats_expect(-1, -1);
>>>> + its_send_int(dev2, 0);
>>>> + check_lpi_stats();
>>>> +
>>>> + /* remap the collection */
>>>> + its_send_mapc(col, true);
>>>> + lpi_stats_expect(nr_cpus - 1, 8193);
>>>> +}
>>>> +
>>>> +static void test_its_pending_migration(void)
>>>> +{
>>>> + struct its_device *dev;
>>>> + struct its_collection *collection[2];
>>>> + int expected[NR_CPUS];
>>>
>>> expected = malloc(nr_cpus * sizeof(int));
>>>
>>> I know there are other places using NR_CPUS right now that don't have to,
>>> but we shouldn't add more. Eventually I'll change the other places too.
>> OK
>>>
>>>> + u64 pendbaser;
>>>> + void *ptr;
>>>> + int i;
>>>> +
>>>> + if (its_prerequisites(4))
>>>> + return;
>>>> +
>>>> + dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>>>> + its_send_mapd(dev, true);
>>>> +
>>>> + collection[0] = its_create_collection(nr_cpus - 1, nr_cpus - 1);
>>>> + collection[1] = its_create_collection(nr_cpus - 2, nr_cpus - 2);
>>>> + its_send_mapc(collection[0], true);
>>>> + its_send_mapc(collection[1], true);
>>>> +
>>>> + /* disable lpi at redist level */
>>>> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, false);
>>>> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, false);
>>>> +
>>>> + /* even lpis are assigned to even cpu */
>>>> + for (i = 0; i < 256; i++) {
>>>> + struct its_collection *col = i % 2 ? collection[0] :
>>>> + collection[1];
>>>> + int vcpu = col->target_address >> 16;
>>>
>>> I'm lost with the even/odd (nr_cpus - 1)/(nr_cpus - 2) stuff, and won't
>>> it swap if nr_cpus is odd vs. even?
>>>
>>> Shouldn't we just have something like
>>>
>>> pe1 = nr_cpus - 1;
>>> pe2 = nr_cpus - 2;
>>> col1 = its_create_collection(pe1, pe1);
>>> col2 = its_create_collection(pe2, pe2);
>>>
>>> without mentioning even and odd?
>> OK
>>>
>>>> +
>>>> + its_send_mapti(dev, 8192 + i, i, col);
>>>> + gicv3_lpi_set_config(8192 + i, LPI_PROP_DEFAULT);
>>>> + gicv3_lpi_set_pending_table_bit(vcpu, 8192 + i, true);
>>>> + }
>>>> + its_send_invall(collection[0]);
>>>> + its_send_invall(collection[1]);
>>>> +
>>>> + /* Set the PTZ bit on each pendbaser */
>>>> +
>>>> + expected[nr_cpus - 1] = 128;
>>>> + expected[nr_cpus - 2] = 128;
>>>> +
>>>> + ptr = gicv3_data.redist_base[nr_cpus - 1] + GICR_PENDBASER;
>>>> + pendbaser = readq(ptr);
>>>> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>>>> +
>>>> + ptr = gicv3_data.redist_base[nr_cpus - 2] + GICR_PENDBASER;
>>>> + pendbaser = readq(ptr);
>>>> + writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>>>> +
>>>> + gicv3_lpi_rdist_ctrl(nr_cpus - 1, true);
>>>> + gicv3_lpi_rdist_ctrl(nr_cpus - 2, true);
>>>> +
>>>> + puts("Now migrate the VM, then press a key to continue...\n");
>>>> + (void)getchar();
>>>> + report(true, "Migration complete");
>>>
>>> report_info
>> OK
>>>
>>>> +
>>>> + mdelay(1000);
>>>
>>> This delay needs a comment explaining why it's here.
>> OK
>>
>> Thanks
>>
>> Eric
>>>
>>>> +
>>>> + check_lpi_hits(expected);
>>>> +}
>>>> #endif
>>>>
>>>> int main(int argc, char **argv)
>>>> @@ -847,6 +989,14 @@ int main(int argc, char **argv)
>>>> report_prefix_push(argv[1]);
>>>> test_its_migration();
>>>> report_prefix_pop();
>>>> + } else if (!strcmp(argv[1], "its-pending-migration")) {
>>>> + report_prefix_push(argv[1]);
>>>> + test_its_pending_migration();
>>>> + report_prefix_pop();
>>>> + } else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
>>>> + report_prefix_push(argv[1]);
>>>> + test_migrate_unmapped_collection();
>>>> + report_prefix_pop();
>>>> } else if (strcmp(argv[1], "its-introspection") == 0) {
>>>> report_prefix_push(argv[1]);
>>>> test_its_introspection();
>>>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>>>> index 8b8ec79..d917157 100644
>>>> --- a/arm/unittests.cfg
>>>> +++ b/arm/unittests.cfg
>>>> @@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
>>>> groups = its migration
>>>> arch = arm64
>>>>
>>>> +[its-pending-migration]
>>>> +file = gic.flat
>>>> +smp = $MAX_SMP
>>>> +accel = kvm
>>>> +extra_params = -machine gic-version=3 -append 'its-pending-migration'
>>>> +groups = its migration
>>>> +arch = arm64
>>>> +
>>>> +[its-migrate-unmapped-collection]
>>>> +file = gic.flat
>>>> +smp = $MAX_SMP
>>>> +accel = kvm
>>>> +extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
>>>> +groups = its migration
>>>> +arch = arm64
>>>> +
>>>> # Test PSCI emulation
>>>> [psci]
>>>> file = psci.flat
>>>> --
>>>> 2.20.1
>>>>
>>>>
>>>
>>> Thanks,
>>> drew
>>>
>>
>>
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 137+ messages in thread
end of thread, other threads:[~2020-03-06 13:41 UTC | newest]
Thread overview: 137+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-28 10:34 [kvm-unit-tests PATCH v3 00/14] arm/arm64: Add ITS tests Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 01/14] libcflat: Add other size defines Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 02/14] arm: gic: Provide per-IRQ helper functions Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 03/14] arm/arm64: gic: Introduce setup_irq() helper Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 04/14] arm/arm64: gicv3: Add some re-distributor defines Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-02-06 14:35 ` Zenghui Yu
2020-02-06 14:35 ` Zenghui Yu
2020-02-06 14:35 ` Zenghui Yu
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 05/14] arm/arm64: ITS: Introspection tests Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-02-06 15:12 ` Zenghui Yu
2020-02-06 15:12 ` Zenghui Yu
2020-02-06 15:12 ` Zenghui Yu
2020-02-07 10:19 ` Andrew Jones
2020-02-07 10:19 ` Andrew Jones
2020-02-07 10:19 ` Andrew Jones
2020-03-04 14:20 ` Auger Eric
2020-03-04 14:20 ` Auger Eric
2020-03-04 14:20 ` Auger Eric
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 06/14] arm/arm64: gicv3: Set the LPI config and pending tables Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-02-07 2:12 ` Zenghui Yu
2020-02-07 2:12 ` Zenghui Yu
2020-02-07 2:12 ` Zenghui Yu
2020-03-05 19:40 ` Auger Eric
2020-03-05 19:40 ` Auger Eric
2020-03-05 19:40 ` Auger Eric
2020-02-07 12:11 ` Andrew Jones
2020-02-07 12:11 ` Andrew Jones
2020-02-07 12:11 ` Andrew Jones
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 07/14] arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-02-07 12:14 ` Andrew Jones
2020-02-07 12:14 ` Andrew Jones
2020-02-07 12:14 ` Andrew Jones
2020-02-07 12:19 ` Andrew Jones
2020-02-07 12:19 ` Andrew Jones
2020-02-07 12:19 ` Andrew Jones
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-02-07 3:20 ` Zenghui Yu
2020-02-07 3:20 ` Zenghui Yu
2020-02-07 3:20 ` Zenghui Yu
2020-03-04 14:26 ` Auger Eric
2020-03-04 14:26 ` Auger Eric
2020-03-04 14:26 ` Auger Eric
2020-03-05 6:30 ` Zenghui Yu
2020-03-05 6:30 ` Zenghui Yu
2020-03-05 6:30 ` Zenghui Yu
2020-02-07 12:41 ` Andrew Jones
2020-02-07 12:41 ` Andrew Jones
2020-02-07 12:41 ` Andrew Jones
2020-03-05 17:59 ` Auger Eric
2020-03-05 17:59 ` Auger Eric
2020-03-05 17:59 ` Auger Eric
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 09/14] arm/arm64: ITS: Device and collection Initialization Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-02-07 5:41 ` Zenghui Yu
2020-02-07 5:41 ` Zenghui Yu
2020-02-07 5:41 ` Zenghui Yu
2020-03-05 19:42 ` Auger Eric
2020-03-05 19:42 ` Auger Eric
2020-03-05 19:42 ` Auger Eric
2020-02-07 12:51 ` Andrew Jones
2020-02-07 12:51 ` Andrew Jones
2020-02-07 12:51 ` Andrew Jones
2020-03-06 8:47 ` Auger Eric
2020-03-06 8:47 ` Auger Eric
2020-03-06 8:47 ` Auger Eric
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 10/14] arm/arm64: ITS: commands Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-02-07 13:37 ` Andrew Jones
2020-02-07 13:37 ` Andrew Jones
2020-02-07 13:37 ` Andrew Jones
2020-03-06 9:13 ` Auger Eric
2020-03-06 9:13 ` Auger Eric
2020-03-06 9:13 ` Auger Eric
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 11/14] arm/arm64: ITS: INT functional tests Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-02-07 13:15 ` Andrew Jones
2020-02-07 13:15 ` Andrew Jones
2020-02-07 13:15 ` Andrew Jones
2020-03-06 12:55 ` Auger Eric
2020-03-06 12:55 ` Auger Eric
2020-03-06 12:55 ` Auger Eric
2020-03-06 13:29 ` Andrew Jones
2020-03-06 13:29 ` Andrew Jones
2020-03-06 13:40 ` Auger Eric
2020-03-06 13:40 ` Auger Eric
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 12/14] arm/run: Allow Migration tests Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-29 8:07 ` Thomas Huth
2020-01-29 8:07 ` Thomas Huth
2020-01-29 8:07 ` Thomas Huth
2020-01-29 9:29 ` Auger Eric
2020-01-29 9:29 ` Auger Eric
2020-01-29 9:29 ` Auger Eric
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 13/14] arm/arm64: ITS: migration tests Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-02-07 13:49 ` Andrew Jones
2020-02-07 13:49 ` Andrew Jones
2020-02-07 13:49 ` Andrew Jones
2020-03-06 13:06 ` Auger Eric
2020-03-06 13:06 ` Auger Eric
2020-03-06 13:06 ` Auger Eric
2020-01-28 10:34 ` [kvm-unit-tests PATCH v3 14/14] arm/arm64: ITS: pending table migration test Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-01-28 10:34 ` Eric Auger
2020-02-07 14:06 ` Andrew Jones
2020-02-07 14:06 ` Andrew Jones
2020-02-07 14:06 ` Andrew Jones
2020-03-06 13:21 ` Auger Eric
2020-03-06 13:21 ` Auger Eric
2020-03-06 13:21 ` Auger Eric
2020-03-06 13:36 ` Andrew Jones
2020-03-06 13:36 ` Andrew Jones
2020-03-06 13:41 ` Auger Eric
2020-03-06 13:41 ` Auger Eric
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.