QEMU-Devel Archive on lore.kernel.org
 help / color / Atom feed
* [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests
@ 2020-03-20  9:24 Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 01/13] libcflat: Add other size defines Eric Auger
                   ` (12 more replies)
  0 siblings, 13 replies; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

This series is a revival of an RFC series sent in Dec 2016 [1].
Given the amount of code and the lack of traction at that time,
I haven't respinned until now. However a recent bug found related
to the ITS migration convinced me that this work may deserve to be
respinned and enhanced.

Tests exercise main ITS commands and also test migration.
With the migration framework, we are able to trigger the
migration from guest and that is very practical actually.

What is particular with the ITS programming is that most of
the commands are passed through queues and there is real error
handling. Invalid commands are just ignored and that is not
really tester friendly.

The series can be fount at:
https://github.com/eauger/kut/tree/its-v7

Applies on top of arm/queue.

Best Regards

Eric

History:
v6 -> v7:
- Fixed issue reported by Drew in its-migrate-unmapped-collection (a collection
  could be allocated twice depending on the SMP value)
- Integrated changes prepared by Drew
- Use for_each_present_cpu()

v5 -> v6:
- Took into account Zenghui's comments, mostly functional: see invidual
  history logs
- fix wrong assert!

v4 -> v5:
- 32b stubs moved back to arm/gic.c
- some changes reordering
- minor style issues

v3 -> v4:
- addressed comments from Drew and Zenghui
- added "page_alloc: Introduce get_order()"
- removed "arm: gic: Provide per-IRQ helper functions"
- ITS files moved to lib64
- and many more, see individual logs

v2 -> v3:
- fix 32b compilation
- take into account Drew's comments (see individual diff logs)

v1 -> v2:
- took into account Zenghui's comments
- collect R-b's from Thomas

References:
[1] [kvm-unit-tests RFC 00/15] arm/arm64: add ITS framework
    https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg00575.html

Execution:
x For other ITS tests:
  ./run_tests.sh -g its

x non migration tests can be launched invidually. For instance:
  ./arm-run arm/gic.flat -smp 8 -append 'its-trigger'

x also tests can be compiled in standalone mode with
  "make standalone" and tests (even migration ones) can also be
  launched that way, for instance:
  MAX_SMP=4 tests/its-migrate-unmapped-collection

Eric Auger (13):
  libcflat: Add other size defines
  page_alloc: Introduce get_order()
  arm/arm64: gic: Introduce setup_irq() helper
  arm/arm64: gicv3: Add some re-distributor defines
  arm/arm64: gicv3: Set the LPI config and pending tables
  arm/arm64: ITS: Introspection tests
  arm/arm64: ITS: its_enable_defaults
  arm/arm64: ITS: Device and collection Initialization
  arm/arm64: ITS: Commands
  arm/arm64: ITS: INT functional tests
  arm/run: Allow Migration tests
  arm/arm64: ITS: migration tests
  arm/arm64: ITS: pending table migration test

 arm/Makefile.arm64         |   1 +
 arm/Makefile.common        |   2 +-
 arm/gic.c                  | 460 ++++++++++++++++++++++++++++++++++--
 arm/run                    |   2 +-
 arm/unittests.cfg          |  38 +++
 lib/alloc_page.c           |   7 +-
 lib/alloc_page.h           |   1 +
 lib/arm/asm/gic-v3-its.h   |  27 +++
 lib/arm/asm/gic-v3.h       |  29 +++
 lib/arm/asm/gic.h          |   1 +
 lib/arm/asm/processor.h    |   2 +
 lib/arm/gic-v3.c           |  78 +++++++
 lib/arm/gic.c              |  40 +++-
 lib/arm/io.c               |  28 +++
 lib/arm64/asm/gic-v3-its.h | 174 ++++++++++++++
 lib/arm64/gic-v3-its-cmd.c | 463 +++++++++++++++++++++++++++++++++++++
 lib/arm64/gic-v3-its.c     | 171 ++++++++++++++
 lib/libcflat.h             |   3 +
 18 files changed, 1496 insertions(+), 31 deletions(-)
 create mode 100644 lib/arm/asm/gic-v3-its.h
 create mode 100644 lib/arm64/asm/gic-v3-its.h
 create mode 100644 lib/arm64/gic-v3-its-cmd.c
 create mode 100644 lib/arm64/gic-v3-its.c

-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 01/13] libcflat: Add other size defines
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 02/13] page_alloc: Introduce get_order() Eric Auger
                   ` (11 subsequent siblings)
  12 siblings, 0 replies; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

Introduce additional SZ_256, SZ_8K, SZ_16K macros that will
be used by ITS tests.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
 lib/libcflat.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/libcflat.h b/lib/libcflat.h
index ea19f61..7092af2 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -36,7 +36,10 @@
 #define ALIGN(x, a)		__ALIGN((x), (a))
 #define IS_ALIGNED(x, a)	(((x) & ((typeof(x))(a) - 1)) == 0)
 
+#define SZ_256			(1 << 8)
 #define SZ_4K			(1 << 12)
+#define SZ_8K			(1 << 13)
+#define SZ_16K			(1 << 14)
 #define SZ_64K			(1 << 16)
 #define SZ_2M			(1 << 21)
 #define SZ_1G			(1 << 30)
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 02/13] page_alloc: Introduce get_order()
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 01/13] libcflat: Add other size defines Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 03/13] arm/arm64: gic: Introduce setup_irq() helper Eric Auger
                   ` (10 subsequent siblings)
  12 siblings, 0 replies; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

Compute the power of 2 order of a size. Use it in
page_memalign. Other users are looming.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
 lib/alloc_page.c | 7 ++++++-
 lib/alloc_page.h | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/lib/alloc_page.c b/lib/alloc_page.c
index ed23638..7c8461a 100644
--- a/lib/alloc_page.c
+++ b/lib/alloc_page.c
@@ -155,7 +155,7 @@ static void *page_memalign(size_t alignment, size_t size)
 	if (!size)
 		return NULL;
 
-	order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
+	order = get_order(n);
 
 	return alloc_pages(order);
 }
@@ -175,3 +175,8 @@ void page_alloc_ops_enable(void)
 {
 	alloc_ops = &page_alloc_ops;
 }
+
+int get_order(size_t size)
+{
+	return is_power_of_2(size) ? fls(size) : fls(size) + 1;
+}
diff --git a/lib/alloc_page.h b/lib/alloc_page.h
index 739a91d..e6a51d2 100644
--- a/lib/alloc_page.h
+++ b/lib/alloc_page.h
@@ -15,5 +15,6 @@ void *alloc_pages(unsigned long order);
 void free_page(void *page);
 void free_pages(void *mem, unsigned long size);
 void free_pages_by_order(void *mem, unsigned long order);
+int get_order(size_t size);
 
 #endif
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 03/13] arm/arm64: gic: Introduce setup_irq() helper
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 01/13] libcflat: Add other size defines Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 02/13] page_alloc: Introduce get_order() Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 04/13] arm/arm64: gicv3: Add some re-distributor defines Eric Auger
                   ` (9 subsequent siblings)
  12 siblings, 0 replies; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

ipi_enable() code would be reusable for other interrupts
than IPI. Let's rename it setup_irq() and pass an interrupt
handler pointer.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---

v4 -> v5:
- s/handler_t/irq_handler_t
- also add irq_handler_fn in lib/arm/asm/processor.h

v2 -> v3:
- do not export setup_irq anymore
---
 arm/gic.c               | 19 ++++++-------------
 lib/arm/asm/processor.h |  2 ++
 2 files changed, 8 insertions(+), 13 deletions(-)

diff --git a/arm/gic.c b/arm/gic.c
index fcf4c1f..2f904b0 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -215,20 +215,20 @@ static void ipi_test_smp(void)
 	report_prefix_pop();
 }
 
-static void ipi_enable(void)
+static void setup_irq(irq_handler_fn handler)
 {
 	gic_enable_defaults();
 #ifdef __arm__
-	install_exception_handler(EXCPTN_IRQ, ipi_handler);
+	install_exception_handler(EXCPTN_IRQ, handler);
 #else
-	install_irq_handler(EL1H_IRQ, ipi_handler);
+	install_irq_handler(EL1H_IRQ, handler);
 #endif
 	local_irq_enable();
 }
 
 static void ipi_send(void)
 {
-	ipi_enable();
+	setup_irq(ipi_handler);
 	wait_on_ready();
 	ipi_test_self();
 	ipi_test_smp();
@@ -238,7 +238,7 @@ static void ipi_send(void)
 
 static void ipi_recv(void)
 {
-	ipi_enable();
+	setup_irq(ipi_handler);
 	cpumask_set_cpu(smp_processor_id(), &ready);
 	while (1)
 		wfi();
@@ -295,14 +295,7 @@ static void ipi_clear_active_handler(struct pt_regs *regs __unused)
 static void run_active_clear_test(void)
 {
 	report_prefix_push("active");
-	gic_enable_defaults();
-#ifdef __arm__
-	install_exception_handler(EXCPTN_IRQ, ipi_clear_active_handler);
-#else
-	install_irq_handler(EL1H_IRQ, ipi_clear_active_handler);
-#endif
-	local_irq_enable();
-
+	setup_irq(ipi_clear_active_handler);
 	ipi_test_self();
 	report_prefix_pop();
 }
diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
index 1e1132d..e26ef89 100644
--- a/lib/arm/asm/processor.h
+++ b/lib/arm/asm/processor.h
@@ -26,7 +26,9 @@ enum vector {
 	EXCPTN_MAX,
 };
 
+typedef void (*irq_handler_fn)(struct pt_regs *regs);
 typedef void (*exception_fn)(struct pt_regs *);
+
 extern void install_exception_handler(enum vector v, exception_fn fn);
 
 extern void show_regs(struct pt_regs *regs);
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 04/13] arm/arm64: gicv3: Add some re-distributor defines
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
                   ` (2 preceding siblings ...)
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 03/13] arm/arm64: gic: Introduce setup_irq() helper Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 05/13] arm/arm64: gicv3: Set the LPI config and pending tables Eric Auger
                   ` (8 subsequent siblings)
  12 siblings, 0 replies; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

PROPBASER, PENDBASE and GICR_CTRL will be used for LPI management.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>

---

v3 -> v4:
- replace some spaces by tabs and added Zenghui's R-b
---
 lib/arm/asm/gic-v3.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index e2736a1..47df051 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -18,6 +18,7 @@
  * We expect to be run in Non-secure mode, thus we define the
  * group1 enable bits with respect to that view.
  */
+#define GICD_CTLR			0x0000
 #define GICD_CTLR_RWP			(1U << 31)
 #define GICD_CTLR_ARE_NS		(1U << 4)
 #define GICD_CTLR_ENABLE_G1A		(1U << 1)
@@ -38,6 +39,11 @@
 #define GICR_ICACTIVER0			GICD_ICACTIVER
 #define GICR_IPRIORITYR0		GICD_IPRIORITYR
 
+#define GICR_PROPBASER			0x0070
+#define GICR_PENDBASER			0x0078
+#define GICR_CTLR			GICD_CTLR
+#define GICR_CTLR_ENABLE_LPIS		(1UL << 0)
+
 #define ICC_SGI1R_AFFINITY_1_SHIFT	16
 #define ICC_SGI1R_AFFINITY_2_SHIFT	32
 #define ICC_SGI1R_AFFINITY_3_SHIFT	48
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 05/13] arm/arm64: gicv3: Set the LPI config and pending tables
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
                   ` (3 preceding siblings ...)
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 04/13] arm/arm64: gicv3: Add some re-distributor defines Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests Eric Auger
                   ` (7 subsequent siblings)
  12 siblings, 0 replies; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

Allocate the LPI configuration and per re-distributor pending table.
Set redistributor's PROPBASER and PENDBASER. The LPIs are enabled
by default in the config table.

Also introduce a helper routine that allows to set the pending
table bit for a given LPI and macros to set/get its configuration.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>

---

v5 -> v6:
- fix the assert()
- remove GICR_PROPBASER_IDBITS_MASK
- remove gicv3_lpi_set_config and gicv3_lpi_get_config declarations
  and move macros in this patch

v4 -> v5:
- Moved some reformattings previously done in
  "arm/arm64: ITS: its_enable_defaults", in this patch
- added assert(!gicv3_redist_base()) in gicv3_lpi_alloc_tables()
- revert for_each_present_cpu() change

v2 -> v3:
- Move the helpers in lib/arm/gic-v3.c and prefix them with "gicv3_"
  and add _lpi prefix too

v1 -> v2:
- remove memory attributes
---
 lib/arm/asm/gic-v3.h | 17 ++++++++++++++
 lib/arm/gic-v3.c     | 53 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+)

diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index 47df051..fedffa8 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -50,6 +50,13 @@
 #define MPIDR_TO_SGI_AFFINITY(cluster_id, level) \
 	(MPIDR_AFFINITY_LEVEL(cluster_id, level) << ICC_SGI1R_AFFINITY_## level ## _SHIFT)
 
+#define GICR_PENDBASER_PTZ		BIT_ULL(62)
+
+#define LPI_PROP_GROUP1			(1 << 1)
+#define LPI_PROP_ENABLED		(1 << 0)
+#define LPI_PROP_DEFAULT_PRIO		0xa0
+#define LPI_PROP_DEFAULT		(LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | LPI_PROP_ENABLED)
+
 #include <asm/arch_gicv3.h>
 
 #ifndef __ASSEMBLY__
@@ -66,6 +73,8 @@ struct gicv3_data {
 	void *dist_base;
 	void *redist_bases[GICV3_NR_REDISTS];
 	void *redist_base[NR_CPUS];
+	u8 *lpi_prop;
+	void *lpi_pend[NR_CPUS];
 	unsigned int irq_nr;
 };
 extern struct gicv3_data gicv3_data;
@@ -82,6 +91,8 @@ extern void gicv3_write_eoir(u32 irqstat);
 extern void gicv3_ipi_send_single(int irq, int cpu);
 extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
 extern void gicv3_set_redist_base(size_t stride);
+extern void gicv3_lpi_set_clr_pending(int rdist, int n, bool set);
+extern void gicv3_lpi_alloc_tables(void);
 
 static inline void gicv3_do_wait_for_rwp(void *base)
 {
@@ -127,5 +138,11 @@ static inline u64 mpidr_uncompress(u32 compressed)
 	return mpidr;
 }
 
+#define gicv3_lpi_set_config(intid, value) ({		\
+	gicv3_data.lpi_prop[LPI_OFFSET(intid)] = value;	\
+})
+
+#define gicv3_lpi_get_config(intid) (gicv3_data.lpi_prop[LPI_OFFSET(intid)])
+
 #endif /* !__ASSEMBLY__ */
 #endif /* _ASMARM_GIC_V3_H_ */
diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
index feecb5e..6cf1d1d 100644
--- a/lib/arm/gic-v3.c
+++ b/lib/arm/gic-v3.c
@@ -5,6 +5,7 @@
  */
 #include <asm/gic.h>
 #include <asm/io.h>
+#include <alloc_page.h>
 
 void gicv3_set_redist_base(size_t stride)
 {
@@ -147,3 +148,55 @@ void gicv3_ipi_send_single(int irq, int cpu)
 	cpumask_set_cpu(cpu, &dest);
 	gicv3_ipi_send_mask(irq, &dest);
 }
+
+#if defined(__aarch64__)
+
+/*
+ * alloc_lpi_tables - Allocate LPI config and pending tables
+ * and set PROPBASER (shared by all rdistributors) and per
+ * redistributor PENDBASER.
+ *
+ * gicv3_set_redist_base() must be called before
+ */
+void gicv3_lpi_alloc_tables(void)
+{
+	unsigned long n = SZ_64K >> PAGE_SHIFT;
+	unsigned long order = fls(n);
+	u64 prop_val;
+	int cpu;
+
+	assert(gicv3_redist_base());
+
+	gicv3_data.lpi_prop = alloc_pages(order);
+
+	/* ID bits = 13, ie. up to 14b LPI INTID */
+	prop_val = (u64)(virt_to_phys(gicv3_data.lpi_prop)) | 13;
+
+	for_each_present_cpu(cpu) {
+		u64 pend_val;
+		void *ptr;
+
+		ptr = gicv3_data.redist_base[cpu];
+
+		writeq(prop_val, ptr + GICR_PROPBASER);
+
+		gicv3_data.lpi_pend[cpu] = alloc_pages(order);
+		pend_val = (u64)(virt_to_phys(gicv3_data.lpi_pend[cpu]));
+		writeq(pend_val, ptr + GICR_PENDBASER);
+	}
+}
+
+void gicv3_lpi_set_clr_pending(int rdist, int n, bool set)
+{
+	u8 *ptr = gicv3_data.lpi_pend[rdist];
+	u8 mask = 1 << (n % 8), byte;
+
+	ptr += (n / 8);
+	byte = *ptr;
+	if (set)
+		byte |=  mask;
+	else
+		byte &= ~mask;
+	*ptr = byte;
+}
+#endif /* __aarch64__ */
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
                   ` (4 preceding siblings ...)
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 05/13] arm/arm64: gicv3: Set the LPI config and pending tables Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-30  8:30   ` Zenghui Yu
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 07/13] arm/arm64: ITS: its_enable_defaults Eric Auger
                   ` (6 subsequent siblings)
  12 siblings, 1 reply; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

Detect the presence of an ITS as part of the GICv3 init
routine, initialize its base address and read few registers
the IIDR, the TYPER to store its dimensioning parameters.
Parse the BASER registers. As part of the init sequence we
also init all the requested tables.

This is our first ITS test, belonging to a new "its" group.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---
v6 -> v7:
- use assert_msg() in its_init
- Only call its_init for arm64-gicv3
- Not including gic-v3-its.h anywhere directly
- One spaces to tab

v5 -> v6:
- fix some GENMASK_ULL and tabs

v4 -> v5:
- Moved test_its_introspection() stub from
  lib/arm/asm/gic-v3-its.h back to arm/gic.c
- 32b its_init does report_abort()
- remove kerneldoc style comment
- remove alloc_lpi_tables from its_init()

v3 -> v4:
- fixed some typos, refine trace msgs
- move its files to lib/arm64 instead of lib/arm
- create lib/arm/asm/gic-v3-its.h containing stubs
- rework gic_get_dt_bases
- rework baser parsing
- move table allocation to init routine
- use get_order()

v2 -> v3:
- updated dates and changed author
- squash "arm/arm64: ITS: Test BASER" into this patch but
  removes setup_baser which will be introduced later.
- only compile on aarch64
- restrict the new test to aarch64

v1 -> v2:
- clean GITS_TYPER macros and unused fields in typer struct
- remove memory attribute related macros
- remove everything related to memory attributes
- s/dev_baser/coll_baser/ in report_info
- add extra line
- removed index filed in its_baser
---
 arm/Makefile.arm64         |  1 +
 arm/gic.c                  | 48 +++++++++++++++++++
 arm/unittests.cfg          |  7 +++
 lib/arm/asm/gic-v3-its.h   | 27 +++++++++++
 lib/arm/asm/gic.h          |  1 +
 lib/arm/gic.c              | 40 +++++++++++++---
 lib/arm64/asm/gic-v3-its.h | 96 +++++++++++++++++++++++++++++++++++++
 lib/arm64/gic-v3-its.c     | 98 ++++++++++++++++++++++++++++++++++++++
 8 files changed, 311 insertions(+), 7 deletions(-)
 create mode 100644 lib/arm/asm/gic-v3-its.h
 create mode 100644 lib/arm64/asm/gic-v3-its.h
 create mode 100644 lib/arm64/gic-v3-its.c

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 6d3dc2c..60182ae 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -19,6 +19,7 @@ endef
 cstart.o = $(TEST_DIR)/cstart64.o
 cflatobjs += lib/arm64/processor.o
 cflatobjs += lib/arm64/spinlock.o
+cflatobjs += lib/arm64/gic-v3-its.o
 
 OBJDIRS += lib/arm64
 
diff --git a/arm/gic.c b/arm/gic.c
index 2f904b0..649ed81 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -16,6 +16,7 @@
 #include <asm/processor.h>
 #include <asm/delay.h>
 #include <asm/gic.h>
+#include <asm/gic-v3-its.h>
 #include <asm/smp.h>
 #include <asm/barrier.h>
 #include <asm/io.h>
@@ -517,6 +518,49 @@ static void gic_test_mmio(void)
 		test_targets(nr_irqs);
 }
 
+#if defined(__arm__)
+
+static void test_its_introspection(void) {}
+
+#else /* __aarch64__ */
+
+static void test_its_introspection(void)
+{
+	struct its_baser *dev_baser = &its_data.device_baser;
+	struct its_baser *coll_baser = &its_data.coll_baser;
+	struct its_typer *typer = &its_data.typer;
+
+	if (!gicv3_its_base()) {
+		report_skip("No ITS, skip ...");
+		return;
+	}
+
+	/* IIDR */
+	report(test_readonly_32(gicv3_its_base() + GITS_IIDR, false),
+	       "GITS_IIDR is read-only"),
+
+	/* TYPER */
+	report(test_readonly_32(gicv3_its_base() + GITS_TYPER, false),
+	       "GITS_TYPER is read-only");
+
+	report(typer->phys_lpi, "ITS supports physical LPIs");
+	report_info("vLPI support: %s", typer->virt_lpi ? "yes" : "no");
+	report_info("ITT entry size = 0x%x", typer->ite_size);
+	report_info("Bit Count: EventID=%d DeviceId=%d CollId=%d",
+		    typer->eventid_bits, typer->deviceid_bits,
+		    typer->collid_bits);
+	report(typer->eventid_bits && typer->deviceid_bits &&
+	       typer->collid_bits, "ID spaces");
+	report_info("Target address format %s",
+			typer->pta ? "Redist base address" : "PE #");
+
+	report(dev_baser && coll_baser, "detect device and collection BASER");
+	report_info("device table entry_size = 0x%x", dev_baser->esz);
+	report_info("collection table entry_size = 0x%x", coll_baser->esz);
+}
+
+#endif
+
 int main(int argc, char **argv)
 {
 	if (!gic_init()) {
@@ -548,6 +592,10 @@ int main(int argc, char **argv)
 		report_prefix_push(argv[1]);
 		gic_test_mmio();
 		report_prefix_pop();
+	} else if (strcmp(argv[1], "its-introspection") == 0) {
+		report_prefix_push(argv[1]);
+		test_its_introspection();
+		report_prefix_pop();
 	} else {
 		report_abort("Unknown subtest '%s'", argv[1]);
 	}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 017958d..23d378e 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -122,6 +122,13 @@ smp = $MAX_SMP
 extra_params = -machine gic-version=3 -append 'active'
 groups = gic
 
+[its-introspection]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'its-introspection'
+groups = its
+arch = arm64
+
 # Test PSCI emulation
 [psci]
 file = psci.flat
diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
new file mode 100644
index 0000000..efd8f67
--- /dev/null
+++ b/lib/arm/asm/gic-v3-its.h
@@ -0,0 +1,27 @@
+/*
+ * ITS 32-bit stubs
+ *
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM_GIC_V3_ITS_H_
+#define _ASMARM_GIC_V3_ITS_H_
+
+#ifndef _ASMARM_GIC_H_
+#error Do not directly include <asm/gic-v3-its.h>. Include <asm/gic.h>
+#endif
+
+#include <libcflat.h>
+
+/* dummy its_data struct to allow gic_get_dt_bases() call */
+struct its_data {
+	void *base;
+};
+
+static inline void its_init(void)
+{
+	assert_msg(false, "not supported on 32-bit");
+}
+
+#endif /* _ASMARM_GIC_V3_ITS_H_ */
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index 922cbe9..9564d4f 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -40,6 +40,7 @@
 
 #include <asm/gic-v2.h>
 #include <asm/gic-v3.h>
+#include <asm/gic-v3-its.h>
 
 #define PPI(irq)			((irq) + 16)
 #define SPI(irq)			((irq) + GIC_FIRST_SPI)
diff --git a/lib/arm/gic.c b/lib/arm/gic.c
index c3c5f6b..a807d5f 100644
--- a/lib/arm/gic.c
+++ b/lib/arm/gic.c
@@ -9,6 +9,7 @@
 
 struct gicv2_data gicv2_data;
 struct gicv3_data gicv3_data;
+struct its_data its_data;
 
 struct gic_common_ops {
 	void (*enable_defaults)(void);
@@ -44,12 +45,13 @@ static const struct gic_common_ops gicv3_common_ops = {
  * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
  */
 static bool
-gic_get_dt_bases(const char *compatible, void **base1, void **base2)
+gic_get_dt_bases(const char *compatible, void **base1, void **base2, void **base3)
 {
 	struct dt_pbus_reg reg;
-	struct dt_device gic;
+	struct dt_device gic, its;
 	struct dt_bus bus;
-	int node, ret, i;
+	int node, subnode, ret, i, len;
+	const void *fdt = dt_fdt();
 
 	dt_bus_init_defaults(&bus);
 	dt_device_init(&gic, &bus, NULL);
@@ -74,19 +76,39 @@ gic_get_dt_bases(const char *compatible, void **base1, void **base2)
 		base2[i] = ioremap(reg.addr, reg.size);
 	}
 
+	if (!base3) {
+		assert(!strcmp(compatible, "arm,cortex-a15-gic"));
+		return true;
+	}
+
+	assert(!strcmp(compatible, "arm,gic-v3"));
+
+	dt_for_each_subnode(node, subnode) {
+		const struct fdt_property *prop;
+
+		prop = fdt_get_property(fdt, subnode, "compatible", &len);
+		if (!strcmp((char *)prop->data, "arm,gic-v3-its")) {
+			dt_device_bind_node(&its, subnode);
+			ret = dt_pbus_translate(&its, 0, &reg);
+			assert(ret == 0);
+			*base3 = ioremap(reg.addr, reg.size);
+			break;
+		}
+	}
+
 	return true;
 }
 
 int gicv2_init(void)
 {
 	return gic_get_dt_bases("arm,cortex-a15-gic",
-			&gicv2_data.dist_base, &gicv2_data.cpu_base);
+			&gicv2_data.dist_base, &gicv2_data.cpu_base, NULL);
 }
 
 int gicv3_init(void)
 {
 	return gic_get_dt_bases("arm,gic-v3", &gicv3_data.dist_base,
-			&gicv3_data.redist_bases[0]);
+			&gicv3_data.redist_bases[0], &its_data.base);
 }
 
 int gic_version(void)
@@ -100,10 +122,14 @@ int gic_version(void)
 
 int gic_init(void)
 {
-	if (gicv2_init())
+	if (gicv2_init()) {
 		gic_common_ops = &gicv2_common_ops;
-	else if (gicv3_init())
+	} else if (gicv3_init()) {
 		gic_common_ops = &gicv3_common_ops;
+#ifdef __aarch64__
+		its_init();
+#endif
+	}
 	return gic_version();
 }
 
diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
new file mode 100644
index 0000000..30f2d90
--- /dev/null
+++ b/lib/arm64/asm/gic-v3-its.h
@@ -0,0 +1,96 @@
+/*
+ * All ITS* defines are lifted from include/linux/irqchip/arm-gic-v3.h
+ *
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#ifndef _ASMARM64_GIC_V3_ITS_H_
+#define _ASMARM64_GIC_V3_ITS_H_
+
+#ifndef _ASMARM_GIC_H_
+#error Do not directly include <asm/gic-v3-its.h>. Include <asm/gic.h>
+#endif
+
+struct its_typer {
+	unsigned int ite_size;
+	unsigned int eventid_bits;
+	unsigned int deviceid_bits;
+	unsigned int collid_bits;
+	bool pta;
+	bool phys_lpi;
+	bool virt_lpi;
+};
+
+struct its_baser {
+	int index;
+	size_t psz;
+	int esz;
+	bool indirect;
+	phys_addr_t table_addr;
+};
+
+#define GITS_BASER_NR_REGS		8
+
+struct its_data {
+	void *base;
+	struct its_typer typer;
+	struct its_baser device_baser;
+	struct its_baser coll_baser;
+	struct its_cmd_block *cmd_base;
+	struct its_cmd_block *cmd_write;
+};
+
+extern struct its_data its_data;
+
+#define gicv3_its_base()		(its_data.base)
+
+#define GITS_CTLR			0x0000
+#define GITS_IIDR			0x0004
+#define GITS_TYPER			0x0008
+#define GITS_CBASER			0x0080
+#define GITS_CWRITER			0x0088
+#define GITS_CREADR			0x0090
+#define GITS_BASER			0x0100
+
+#define GITS_TYPER_PLPIS		BIT(0)
+#define GITS_TYPER_VLPIS		BIT(1)
+#define GITS_TYPER_ITT_ENTRY_SIZE	GENMASK_ULL(7, 4)
+#define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT	4
+#define GITS_TYPER_IDBITS		GENMASK_ULL(12, 8)
+#define GITS_TYPER_IDBITS_SHIFT		8
+#define GITS_TYPER_DEVBITS		GENMASK_ULL(17, 13)
+#define GITS_TYPER_DEVBITS_SHIFT	13
+#define GITS_TYPER_PTA			BIT(19)
+#define GITS_TYPER_CIDBITS		GENMASK_ULL(35, 32)
+#define GITS_TYPER_CIDBITS_SHIFT	32
+#define GITS_TYPER_CIL			BIT(36)
+
+#define GITS_CTLR_ENABLE		(1U << 0)
+
+#define GITS_CBASER_VALID		(1UL << 63)
+
+#define GITS_BASER_VALID		BIT(63)
+#define GITS_BASER_INDIRECT		BIT(62)
+#define GITS_BASER_TYPE_SHIFT		(56)
+#define GITS_BASER_TYPE(r)		(((r) >> GITS_BASER_TYPE_SHIFT) & 7)
+#define GITS_BASER_ENTRY_SIZE_SHIFT	(48)
+#define GITS_BASER_ENTRY_SIZE(r)	((((r) >> GITS_BASER_ENTRY_SIZE_SHIFT) & 0x1f) + 1)
+#define GITS_BASER_PAGE_SIZE_SHIFT	(8)
+#define GITS_BASER_PAGE_SIZE_4K		(0UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_16K	(1UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_64K	(2UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_MASK	(3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGES_MAX		256
+#define GITS_BASER_PAGES_SHIFT		(0)
+#define GITS_BASER_NR_PAGES(r)		(((r) & 0xff) + 1)
+#define GITS_BASER_PHYS_ADDR_MASK	0xFFFFFFFFF000
+#define GITS_BASER_TYPE_NONE		0
+#define GITS_BASER_TYPE_DEVICE		1
+#define GITS_BASER_TYPE_COLLECTION	4
+
+extern void its_parse_typer(void);
+extern void its_init(void);
+extern int its_baser_lookup(int i, struct its_baser *baser);
+
+#endif /* _ASMARM64_GIC_V3_ITS_H_ */
diff --git a/lib/arm64/gic-v3-its.c b/lib/arm64/gic-v3-its.c
new file mode 100644
index 0000000..fb8e3f2
--- /dev/null
+++ b/lib/arm64/gic-v3-its.c
@@ -0,0 +1,98 @@
+/*
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <asm/gic.h>
+#include <alloc_page.h>
+
+void its_parse_typer(void)
+{
+	u64 typer = readq(gicv3_its_base() + GITS_TYPER);
+	struct its_typer *t = &its_data.typer;
+
+	t->ite_size = ((typer & GITS_TYPER_ITT_ENTRY_SIZE) >> GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) + 1;
+	t->pta = typer & GITS_TYPER_PTA;
+	t->eventid_bits = ((typer & GITS_TYPER_IDBITS) >> GITS_TYPER_IDBITS_SHIFT) + 1;
+	t->deviceid_bits = ((typer & GITS_TYPER_DEVBITS) >> GITS_TYPER_DEVBITS_SHIFT) + 1;
+
+	if (typer & GITS_TYPER_CIL)
+		t->collid_bits = ((typer & GITS_TYPER_CIDBITS) >> GITS_TYPER_CIDBITS_SHIFT) + 1;
+	else
+		t->collid_bits = 16;
+
+	t->virt_lpi = typer & GITS_TYPER_VLPIS;
+	t->phys_lpi = typer & GITS_TYPER_PLPIS;
+}
+
+int its_baser_lookup(int type, struct its_baser *baser)
+{
+	int i;
+
+	for (i = 0; i < GITS_BASER_NR_REGS; i++) {
+		void *reg_addr = gicv3_its_base() + GITS_BASER + i * 8;
+		u64 val = readq(reg_addr);
+
+		if (GITS_BASER_TYPE(val) == type) {
+			assert((val & GITS_BASER_PAGE_SIZE_MASK) == GITS_BASER_PAGE_SIZE_64K);
+			baser->esz = GITS_BASER_ENTRY_SIZE(val);
+			baser->indirect = val & GITS_BASER_INDIRECT;
+			baser->index = i;
+			return 0;
+		}
+	}
+	return -1;
+}
+
+/*
+ * Allocate the BASER table (a single page of size @baser->psz)
+ * and set the BASER valid
+ */
+static void its_baser_alloc_table(struct its_baser *baser, size_t size)
+{
+	unsigned long order = get_order(size >> PAGE_SHIFT);
+	void *reg_addr = gicv3_its_base() + GITS_BASER + baser->index * 8;
+	u64 val = readq(reg_addr);
+
+	baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
+
+	val |= (u64)baser->table_addr | GITS_BASER_VALID;
+
+	writeq(val, reg_addr);
+}
+
+/*
+ * init_cmd_queue - Allocate the command queue and initialize
+ * CBASER, CWRITER
+ */
+static void its_cmd_queue_init(void)
+{
+	unsigned long order = get_order(SZ_64K >> PAGE_SHIFT);
+	u64 cbaser;
+
+	its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
+
+	cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1)	| GITS_CBASER_VALID);
+
+	writeq(cbaser, its_data.base + GITS_CBASER);
+
+	its_data.cmd_write = its_data.cmd_base;
+	writeq(0, its_data.base + GITS_CWRITER);
+}
+
+void its_init(void)
+{
+	if (!its_data.base)
+		return;
+
+	its_parse_typer();
+
+	assert(!its_baser_lookup(GITS_BASER_TYPE_DEVICE, &its_data.device_baser));
+	assert(!its_baser_lookup(GITS_BASER_TYPE_COLLECTION, &its_data.coll_baser));
+
+	its_baser_alloc_table(&its_data.device_baser, SZ_64K);
+	its_baser_alloc_table(&its_data.coll_baser, SZ_64K);
+
+	its_cmd_queue_init();
+}
+
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 07/13] arm/arm64: ITS: its_enable_defaults
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
                   ` (5 preceding siblings ...)
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 08/13] arm/arm64: ITS: Device and collection Initialization Eric Auger
                   ` (5 subsequent siblings)
  12 siblings, 0 replies; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

its_enable_defaults() enable LPIs at redistributor level
and ITS level.

gicv3_enable_defaults must be called before.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>

---
v6 -> v7:
- use for_each_present_cpu in its_enable_defaults

v5 -> v6:
- gicv3_lpi_set/get_config introduced before this patch
- dist/redist in commit msg
- Added Zenghui's R-b

v4 -> v5:
- some reformattings moved to earlier patch
- add assert(!gicv3_redist_base()) in alloc_lpi_tables()
- revert the usage of for_each_present_cpu()

v3 -> v4:
- use GITS_BASER_INDIRECT & GITS_BASER_VALID in its_setup_baser()
- don't parse BASERs again in its_enable_defaults
- rename its_setup_baser into its_baser_alloc_table
- All allocations moved to the init function
- squashed "arm/arm64: gicv3: Enable/Disable LPIs at re-distributor level"
  into this patch
- introduce gicv3_lpi_rdist_enable and gicv3_lpi_rdist_disable
- pend and prop table bases stored as virt addresses
- move some init functions from enable() to its_init
- removed GICR_PROPBASER_IDBITS_MASK
- introduced LPI_OFFSET
- lpi_prop becomes u8 *
- gicv3_lpi_set_config/get_config became macro
- renamed gicv3_lpi_set_pending_table_bit into gicv3_lpi_set_clr_pending

v2 -> v3:
- introduce its_setup_baser in this patch
- squash "arm/arm64: ITS: Init the command queue" in this patch.
---
 lib/arm/asm/gic-v3.h       |  6 ++++++
 lib/arm/gic-v3.c           | 25 +++++++++++++++++++++++++
 lib/arm64/asm/gic-v3-its.h |  1 +
 lib/arm64/gic-v3-its.c     | 13 +++++++++++++
 4 files changed, 45 insertions(+)

diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index fedffa8..cb72922 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -57,6 +57,10 @@
 #define LPI_PROP_DEFAULT_PRIO		0xa0
 #define LPI_PROP_DEFAULT		(LPI_PROP_DEFAULT_PRIO | LPI_PROP_GROUP1 | LPI_PROP_ENABLED)
 
+#define LPI_ID_BASE			8192
+#define LPI(lpi)			((lpi) + LPI_ID_BASE)
+#define LPI_OFFSET(intid)		((intid) - LPI_ID_BASE)
+
 #include <asm/arch_gicv3.h>
 
 #ifndef __ASSEMBLY__
@@ -93,6 +97,8 @@ extern void gicv3_ipi_send_mask(int irq, const cpumask_t *dest);
 extern void gicv3_set_redist_base(size_t stride);
 extern void gicv3_lpi_set_clr_pending(int rdist, int n, bool set);
 extern void gicv3_lpi_alloc_tables(void);
+extern void gicv3_lpi_rdist_enable(int redist);
+extern void gicv3_lpi_rdist_disable(int redist);
 
 static inline void gicv3_do_wait_for_rwp(void *base)
 {
diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
index 6cf1d1d..a7e2cb8 100644
--- a/lib/arm/gic-v3.c
+++ b/lib/arm/gic-v3.c
@@ -199,4 +199,29 @@ void gicv3_lpi_set_clr_pending(int rdist, int n, bool set)
 		byte &= ~mask;
 	*ptr = byte;
 }
+
+static void gicv3_lpi_rdist_ctrl(u32 redist, bool set)
+{
+	void *ptr;
+	u64 val;
+
+	assert(redist < nr_cpus);
+
+	ptr = gicv3_data.redist_base[redist];
+	val = readl(ptr + GICR_CTLR);
+	if (set)
+		val |= GICR_CTLR_ENABLE_LPIS;
+	else
+		val &= ~GICR_CTLR_ENABLE_LPIS;
+	writel(val,  ptr + GICR_CTLR);
+}
+
+void gicv3_lpi_rdist_enable(int redist)
+{
+	gicv3_lpi_rdist_ctrl(redist, true);
+}
+void gicv3_lpi_rdist_disable(int redist)
+{
+	gicv3_lpi_rdist_ctrl(redist, false);
+}
 #endif /* __aarch64__ */
diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
index 30f2d90..4683011 100644
--- a/lib/arm64/asm/gic-v3-its.h
+++ b/lib/arm64/asm/gic-v3-its.h
@@ -92,5 +92,6 @@ extern struct its_data its_data;
 extern void its_parse_typer(void);
 extern void its_init(void);
 extern int its_baser_lookup(int i, struct its_baser *baser);
+extern void its_enable_defaults(void);
 
 #endif /* _ASMARM64_GIC_V3_ITS_H_ */
diff --git a/lib/arm64/gic-v3-its.c b/lib/arm64/gic-v3-its.c
index fb8e3f2..afc66a3 100644
--- a/lib/arm64/gic-v3-its.c
+++ b/lib/arm64/gic-v3-its.c
@@ -96,3 +96,16 @@ void its_init(void)
 	its_cmd_queue_init();
 }
 
+/* must be called after gicv3_enable_defaults */
+void its_enable_defaults(void)
+{
+	int cpu;
+
+	/* Allocate LPI config and pending tables */
+	gicv3_lpi_alloc_tables();
+
+	for_each_present_cpu(cpu)
+		gicv3_lpi_rdist_enable(cpu);
+
+	writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
+}
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 08/13] arm/arm64: ITS: Device and collection Initialization
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
                   ` (6 preceding siblings ...)
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 07/13] arm/arm64: ITS: its_enable_defaults Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-25  8:10   ` Zenghui Yu
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 09/13] arm/arm64: ITS: Commands Eric Auger
                   ` (4 subsequent siblings)
  12 siblings, 1 reply; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

Introduce an helper functions to register
- a new device, characterized by its device id and the
  max number of event IDs that dimension its ITT (Interrupt
  Translation Table).  The function allocates the ITT.

- a new collection, characterized by its ID and the
  target processing engine (PE).

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---

v3 -> v4:
- remove unused its_baser variable from its_create_device()
- use get_order()
- device->itt becomes a GVA instead of GPA

v2 -> v3:
- s/report_abort/assert

v1 -> v2:
- s/nb_/nr_
---
 lib/arm64/asm/gic-v3-its.h | 19 +++++++++++++++++++
 lib/arm64/gic-v3-its.c     | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 57 insertions(+)

diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
index 4683011..adcb642 100644
--- a/lib/arm64/asm/gic-v3-its.h
+++ b/lib/arm64/asm/gic-v3-its.h
@@ -31,6 +31,19 @@ struct its_baser {
 };
 
 #define GITS_BASER_NR_REGS		8
+#define GITS_MAX_DEVICES		8
+#define GITS_MAX_COLLECTIONS		8
+
+struct its_device {
+	u32 device_id;	/* device ID */
+	u32 nr_ites;	/* Max Interrupt Translation Entries */
+	void *itt;	/* Interrupt Translation Table GVA */
+};
+
+struct its_collection {
+	u64 target_address;
+	u16 col_id;
+};
 
 struct its_data {
 	void *base;
@@ -39,6 +52,10 @@ struct its_data {
 	struct its_baser coll_baser;
 	struct its_cmd_block *cmd_base;
 	struct its_cmd_block *cmd_write;
+	struct its_device devices[GITS_MAX_DEVICES];
+	u32 nr_devices;		/* Allocated Devices */
+	struct its_collection collections[GITS_MAX_COLLECTIONS];
+	u32 nr_collections;	/* Allocated Collections */
 };
 
 extern struct its_data its_data;
@@ -93,5 +110,7 @@ extern void its_parse_typer(void);
 extern void its_init(void);
 extern int its_baser_lookup(int i, struct its_baser *baser);
 extern void its_enable_defaults(void);
+extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
+extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
 
 #endif /* _ASMARM64_GIC_V3_ITS_H_ */
diff --git a/lib/arm64/gic-v3-its.c b/lib/arm64/gic-v3-its.c
index afc66a3..f0a0381 100644
--- a/lib/arm64/gic-v3-its.c
+++ b/lib/arm64/gic-v3-its.c
@@ -109,3 +109,41 @@ void its_enable_defaults(void)
 
 	writel(GITS_CTLR_ENABLE, its_data.base + GITS_CTLR);
 }
+
+struct its_device *its_create_device(u32 device_id, int nr_ites)
+{
+	struct its_device *new;
+	unsigned long n;
+
+	assert(its_data.nr_devices < GITS_MAX_DEVICES);
+
+	new = &its_data.devices[its_data.nr_devices];
+
+	new->device_id = device_id;
+	new->nr_ites = nr_ites;
+
+	n = (its_data.typer.ite_size * nr_ites) >> PAGE_SHIFT;
+	new->itt = alloc_pages(get_order(n));
+
+	its_data.nr_devices++;
+	return new;
+}
+
+struct its_collection *its_create_collection(u32 col_id, u32 pe)
+{
+	struct its_collection *new;
+
+	assert(its_data.nr_collections < GITS_MAX_COLLECTIONS);
+
+	new = &its_data.collections[its_data.nr_collections];
+
+	new->col_id = col_id;
+
+	if (its_data.typer.pta)
+		new->target_address = (u64)gicv3_data.redist_base[pe];
+	else
+		new->target_address = pe << 16;
+
+	its_data.nr_collections++;
+	return new;
+}
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 09/13] arm/arm64: ITS: Commands
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
                   ` (7 preceding siblings ...)
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 08/13] arm/arm64: ITS: Device and collection Initialization Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-30  9:22   ` Zenghui Yu
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 10/13] arm/arm64: ITS: INT functional tests Eric Auger
                   ` (3 subsequent siblings)
  12 siblings, 1 reply; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

Implement main ITS commands. The code is largely inherited from
the ITS driver.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---

v6 -> v7:
- remove #include <asm/gic-v3-its.h>

v5 -> v6:
- fix 2 printfs
- removed GITS_CMD_MAPVI

v3 -> v4:
- device's itt now is a VGA
- pass verbose to choose whether we shall print the cmd
- use printf instead of report_info

v2 -> v3:
- do not use report() anymore
- assert if cmd_write exceeds the queue capacity

v1 -> v2:
- removed its_print_cmd_state
---
 arm/Makefile.arm64         |   2 +-
 lib/arm64/asm/gic-v3-its.h |  55 +++++
 lib/arm64/gic-v3-its-cmd.c | 463 +++++++++++++++++++++++++++++++++++++
 3 files changed, 519 insertions(+), 1 deletion(-)
 create mode 100644 lib/arm64/gic-v3-its-cmd.c

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 60182ae..dfd0c56 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -19,7 +19,7 @@ endef
 cstart.o = $(TEST_DIR)/cstart64.o
 cflatobjs += lib/arm64/processor.o
 cflatobjs += lib/arm64/spinlock.o
-cflatobjs += lib/arm64/gic-v3-its.o
+cflatobjs += lib/arm64/gic-v3-its.o lib/arm64/gic-v3-its-cmd.o
 
 OBJDIRS += lib/arm64
 
diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
index adcb642..e1e42c3 100644
--- a/lib/arm64/asm/gic-v3-its.h
+++ b/lib/arm64/asm/gic-v3-its.h
@@ -106,6 +106,26 @@ extern struct its_data its_data;
 #define GITS_BASER_TYPE_DEVICE		1
 #define GITS_BASER_TYPE_COLLECTION	4
 
+/*
+ * ITS commands
+ */
+#define GITS_CMD_MAPD			0x08
+#define GITS_CMD_MAPC			0x09
+#define GITS_CMD_MAPTI			0x0a
+#define GITS_CMD_MAPI			0x0b
+#define GITS_CMD_MOVI			0x01
+#define GITS_CMD_DISCARD		0x0f
+#define GITS_CMD_INV			0x0c
+#define GITS_CMD_MOVALL			0x0e
+#define GITS_CMD_INVALL			0x0d
+#define GITS_CMD_INT			0x03
+#define GITS_CMD_CLEAR			0x04
+#define GITS_CMD_SYNC			0x05
+
+struct its_cmd_block {
+	u64 raw_cmd[4];
+};
+
 extern void its_parse_typer(void);
 extern void its_init(void);
 extern int its_baser_lookup(int i, struct its_baser *baser);
@@ -113,4 +133,39 @@ extern void its_enable_defaults(void);
 extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
 extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);
 
+extern void __its_send_mapd(struct its_device *dev, int valid, bool verbose);
+extern void __its_send_mapc(struct its_collection *col, int valid, bool verbose);
+extern void __its_send_mapti(struct its_device *dev, u32 irq_id, u32 event_id,
+			     struct its_collection *col, bool verbose);
+extern void __its_send_int(struct its_device *dev, u32 event_id, bool verbose);
+extern void __its_send_inv(struct its_device *dev, u32 event_id, bool verbose);
+extern void __its_send_discard(struct its_device *dev, u32 event_id, bool verbose);
+extern void __its_send_clear(struct its_device *dev, u32 event_id, bool verbose);
+extern void __its_send_invall(struct its_collection *col, bool verbose);
+extern void __its_send_movi(struct its_device *dev, struct its_collection *col,
+			    u32 id, bool verbose);
+extern void __its_send_sync(struct its_collection *col, bool verbose);
+
+#define its_send_mapd(dev, valid)			__its_send_mapd(dev, valid, true)
+#define its_send_mapc(col, valid)			__its_send_mapc(col, valid, true)
+#define its_send_mapti(dev, irqid, eventid, col)	__its_send_mapti(dev, irqid, eventid, col, true)
+#define its_send_int(dev, eventid)			__its_send_int(dev, eventid, true)
+#define its_send_inv(dev, eventid)			__its_send_inv(dev, eventid, true)
+#define its_send_discard(dev, eventid)			__its_send_discard(dev, eventid, true)
+#define its_send_clear(dev, eventid)			__its_send_clear(dev, eventid, true)
+#define its_send_invall(col)				__its_send_invall(col, true)
+#define its_send_movi(dev, col, id)			__its_send_movi(dev, col, id, true)
+#define its_send_sync(col)				__its_send_sync(col, true)
+
+#define its_send_mapd_nv(dev, valid)			__its_send_mapd(dev, valid, false)
+#define its_send_mapc_nv(col, valid)			__its_send_mapc(col, valid, false)
+#define its_send_mapti_nv(dev, irqid, eventid, col)	__its_send_mapti(dev, irqid, eventid, col, false)
+#define its_send_int_nv(dev, eventid)			__its_send_int(dev, eventid, false)
+#define its_send_inv_nv(dev, eventid)			__its_send_inv(dev, eventid, false)
+#define its_send_discard_nv(dev, eventid)		__its_send_discard(dev, eventid, false)
+#define its_send_clear_nv(dev, eventid)			__its_send_clear(dev, eventidn false)
+#define its_send_invall_nv(col)				__its_send_invall(col, false)
+#define its_send_movi_nv(dev, col, id)			__its_send_movi(dev, col, id, false)
+#define its_send_sync_nv(col)				__its_send_sync(col, false)
+
 #endif /* _ASMARM64_GIC_V3_ITS_H_ */
diff --git a/lib/arm64/gic-v3-its-cmd.c b/lib/arm64/gic-v3-its-cmd.c
new file mode 100644
index 0000000..65f1c8c
--- /dev/null
+++ b/lib/arm64/gic-v3-its-cmd.c
@@ -0,0 +1,463 @@
+/*
+ * Copyright (C) 2020, Red Hat Inc, Eric Auger <eric.auger@redhat.com>
+ *
+ * Most of the code is copy-pasted from:
+ * drivers/irqchip/irq-gic-v3-its.c
+ * This work is licensed under the terms of the GNU LGPL, version 2.
+ */
+#include <asm/io.h>
+#include <asm/gic.h>
+
+#define ITS_ITT_ALIGN		SZ_256
+
+static const char * const its_cmd_string[] = {
+	[GITS_CMD_MAPD]		= "MAPD",
+	[GITS_CMD_MAPC]		= "MAPC",
+	[GITS_CMD_MAPTI]	= "MAPTI",
+	[GITS_CMD_MAPI]		= "MAPI",
+	[GITS_CMD_MOVI]		= "MOVI",
+	[GITS_CMD_DISCARD]	= "DISCARD",
+	[GITS_CMD_INV]		= "INV",
+	[GITS_CMD_MOVALL]	= "MOVALL",
+	[GITS_CMD_INVALL]	= "INVALL",
+	[GITS_CMD_INT]		= "INT",
+	[GITS_CMD_CLEAR]	= "CLEAR",
+	[GITS_CMD_SYNC]		= "SYNC",
+};
+
+struct its_cmd_desc {
+	union {
+		struct {
+			struct its_device *dev;
+			u32 event_id;
+		} its_inv_cmd;
+
+		struct {
+			struct its_device *dev;
+			u32 event_id;
+		} its_int_cmd;
+
+		struct {
+			struct its_device *dev;
+			bool valid;
+		} its_mapd_cmd;
+
+		struct {
+			struct its_collection *col;
+			bool valid;
+		} its_mapc_cmd;
+
+		struct {
+			struct its_device *dev;
+			u32 phys_id;
+			u32 event_id;
+			u32 col_id;
+		} its_mapti_cmd;
+
+		struct {
+			struct its_device *dev;
+			struct its_collection *col;
+			u32 event_id;
+		} its_movi_cmd;
+
+		struct {
+			struct its_device *dev;
+			u32 event_id;
+		} its_discard_cmd;
+
+		struct {
+			struct its_device *dev;
+			u32 event_id;
+		} its_clear_cmd;
+
+		struct {
+			struct its_collection *col;
+		} its_invall_cmd;
+
+		struct {
+			struct its_collection *col;
+		} its_sync_cmd;
+	};
+	bool verbose;
+};
+
+typedef void (*its_cmd_builder_t)(struct its_cmd_block *,
+				  struct its_cmd_desc *);
+
+/* ITS COMMANDS */
+
+static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
+{
+	cmd->raw_cmd[0] &= ~0xffUL;
+	cmd->raw_cmd[0] |= cmd_nr;
+}
+
+static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
+{
+	cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
+	cmd->raw_cmd[0] |= ((u64)devid) << 32;
+}
+
+static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
+{
+	cmd->raw_cmd[1] &= ~0xffffffffUL;
+	cmd->raw_cmd[1] |= id;
+}
+
+static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
+{
+	cmd->raw_cmd[1] &= 0xffffffffUL;
+	cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
+}
+
+static void its_encode_size(struct its_cmd_block *cmd, u8 size)
+{
+	cmd->raw_cmd[1] &= ~0x1fUL;
+	cmd->raw_cmd[1] |= size & 0x1f;
+}
+
+static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
+{
+	cmd->raw_cmd[2] &= ~0xffffffffffffUL;
+	cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
+}
+
+static void its_encode_valid(struct its_cmd_block *cmd, int valid)
+{
+	cmd->raw_cmd[2] &= ~(1UL << 63);
+	cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
+}
+
+static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
+{
+	cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
+	cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
+}
+
+static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
+{
+	cmd->raw_cmd[2] &= ~0xffffUL;
+	cmd->raw_cmd[2] |= col;
+}
+
+static inline void its_fixup_cmd(struct its_cmd_block *cmd)
+{
+	/* Let's fixup BE commands */
+	cmd->raw_cmd[0] = cpu_to_le64(cmd->raw_cmd[0]);
+	cmd->raw_cmd[1] = cpu_to_le64(cmd->raw_cmd[1]);
+	cmd->raw_cmd[2] = cpu_to_le64(cmd->raw_cmd[2]);
+	cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
+}
+
+static u64 its_cmd_ptr_to_offset(struct its_cmd_block *ptr)
+{
+	return (ptr - its_data.cmd_base) * sizeof(*ptr);
+}
+
+static struct its_cmd_block *its_post_commands(void)
+{
+	u64 wr = its_cmd_ptr_to_offset(its_data.cmd_write);
+
+	writeq(wr, its_data.base + GITS_CWRITER);
+	return its_data.cmd_write;
+}
+
+static struct its_cmd_block *its_allocate_entry(void)
+{
+	struct its_cmd_block *cmd;
+
+	assert((u64)its_data.cmd_write < (u64)its_data.cmd_base + SZ_64K);
+	cmd = its_data.cmd_write++;
+	return cmd;
+}
+
+static void its_wait_for_range_completion(struct its_cmd_block *from,
+					  struct its_cmd_block *to)
+{
+	u64 rd_idx, from_idx, to_idx;
+	u32 count = 1000000;    /* 1s! */
+
+	from_idx = its_cmd_ptr_to_offset(from);
+	to_idx = its_cmd_ptr_to_offset(to);
+	while (1) {
+		rd_idx = readq(its_data.base + GITS_CREADR);
+		if (rd_idx >= to_idx || rd_idx < from_idx)
+			break;
+
+		count--;
+		if (!count) {
+			unsigned int cmd_id = from->raw_cmd[0] & 0xFF;
+
+			assert_msg(false, "%s timeout!",
+			       cmd_id <= 0xF ? its_cmd_string[cmd_id] :
+			       "Unexpected");
+		}
+		udelay(1);
+	}
+}
+
+static void its_send_single_command(its_cmd_builder_t builder,
+				    struct its_cmd_desc *desc)
+{
+	struct its_cmd_block *cmd, *next_cmd;
+
+	cmd = its_allocate_entry();
+	builder(cmd, desc);
+	next_cmd = its_post_commands();
+
+	its_wait_for_range_completion(cmd, next_cmd);
+}
+
+
+static void its_build_mapd_cmd(struct its_cmd_block *cmd,
+			       struct its_cmd_desc *desc)
+{
+	unsigned long itt_addr;
+	u8 size = 12; /* 4096 eventids */
+
+	itt_addr = (unsigned long)(virt_to_phys(desc->its_mapd_cmd.dev->itt));
+	itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
+
+	its_encode_cmd(cmd, GITS_CMD_MAPD);
+	its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
+	its_encode_size(cmd, size - 1);
+	its_encode_itt(cmd, itt_addr);
+	its_encode_valid(cmd, desc->its_mapd_cmd.valid);
+	its_fixup_cmd(cmd);
+	if (desc->verbose)
+		printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
+			desc->its_mapd_cmd.dev->device_id,
+			size, itt_addr, desc->its_mapd_cmd.valid);
+
+}
+
+static void its_build_mapc_cmd(struct its_cmd_block *cmd,
+			       struct its_cmd_desc *desc)
+{
+	its_encode_cmd(cmd, GITS_CMD_MAPC);
+	its_encode_collection(cmd, desc->its_mapc_cmd.col->col_id);
+	its_encode_target(cmd, desc->its_mapc_cmd.col->target_address);
+	its_encode_valid(cmd, desc->its_mapc_cmd.valid);
+	its_fixup_cmd(cmd);
+	if (desc->verbose)
+		printf("MAPC col_id=%d target_addr = 0x%lx valid=%d\n",
+		       desc->its_mapc_cmd.col->col_id,
+		       desc->its_mapc_cmd.col->target_address,
+		       desc->its_mapc_cmd.valid);
+}
+
+static void its_build_mapti_cmd(struct its_cmd_block *cmd,
+				struct its_cmd_desc *desc)
+{
+	its_encode_cmd(cmd, GITS_CMD_MAPTI);
+	its_encode_devid(cmd, desc->its_mapti_cmd.dev->device_id);
+	its_encode_event_id(cmd, desc->its_mapti_cmd.event_id);
+	its_encode_phys_id(cmd, desc->its_mapti_cmd.phys_id);
+	its_encode_collection(cmd, desc->its_mapti_cmd.col_id);
+	its_fixup_cmd(cmd);
+	if (desc->verbose)
+		printf("MAPTI dev_id=%d event_id=%d -> phys_id=%d, col_id=%d\n",
+		       desc->its_mapti_cmd.dev->device_id,
+		       desc->its_mapti_cmd.event_id,
+		       desc->its_mapti_cmd.phys_id,
+		       desc->its_mapti_cmd.col_id);
+}
+
+static void its_build_invall_cmd(struct its_cmd_block *cmd,
+			      struct its_cmd_desc *desc)
+{
+	its_encode_cmd(cmd, GITS_CMD_INVALL);
+	its_encode_collection(cmd, desc->its_invall_cmd.col->col_id);
+	its_fixup_cmd(cmd);
+	if (desc->verbose)
+		printf("INVALL col_id=%d\n", desc->its_invall_cmd.col->col_id);
+}
+
+static void its_build_clear_cmd(struct its_cmd_block *cmd,
+				struct its_cmd_desc *desc)
+{
+	its_encode_cmd(cmd, GITS_CMD_CLEAR);
+	its_encode_devid(cmd, desc->its_clear_cmd.dev->device_id);
+	its_encode_event_id(cmd, desc->its_clear_cmd.event_id);
+	its_fixup_cmd(cmd);
+	if (desc->verbose)
+		printf("CLEAR dev_id=%d event_id=%d\n", desc->its_clear_cmd.dev->device_id, desc->its_clear_cmd.event_id);
+}
+
+static void its_build_discard_cmd(struct its_cmd_block *cmd,
+				  struct its_cmd_desc *desc)
+{
+	its_encode_cmd(cmd, GITS_CMD_DISCARD);
+	its_encode_devid(cmd, desc->its_discard_cmd.dev->device_id);
+	its_encode_event_id(cmd, desc->its_discard_cmd.event_id);
+	its_fixup_cmd(cmd);
+	if (desc->verbose)
+		printf("DISCARD dev_id=%d event_id=%d\n",
+			desc->its_clear_cmd.dev->device_id, desc->its_clear_cmd.event_id);
+}
+
+static void its_build_inv_cmd(struct its_cmd_block *cmd,
+			      struct its_cmd_desc *desc)
+{
+	its_encode_cmd(cmd, GITS_CMD_INV);
+	its_encode_devid(cmd, desc->its_inv_cmd.dev->device_id);
+	its_encode_event_id(cmd, desc->its_inv_cmd.event_id);
+	its_fixup_cmd(cmd);
+	if (desc->verbose)
+		printf("INV dev_id=%d event_id=%d\n",
+		       desc->its_inv_cmd.dev->device_id,
+		       desc->its_inv_cmd.event_id);
+}
+
+static void its_build_int_cmd(struct its_cmd_block *cmd,
+			      struct its_cmd_desc *desc)
+{
+	its_encode_cmd(cmd, GITS_CMD_INT);
+	its_encode_devid(cmd, desc->its_int_cmd.dev->device_id);
+	its_encode_event_id(cmd, desc->its_int_cmd.event_id);
+	its_fixup_cmd(cmd);
+	if (desc->verbose)
+		printf("INT dev_id=%d event_id=%d\n",
+		       desc->its_int_cmd.dev->device_id,
+		       desc->its_int_cmd.event_id);
+}
+
+static void its_build_sync_cmd(struct its_cmd_block *cmd,
+			       struct its_cmd_desc *desc)
+{
+	its_encode_cmd(cmd, GITS_CMD_SYNC);
+	its_encode_target(cmd, desc->its_sync_cmd.col->target_address);
+	its_fixup_cmd(cmd);
+	if (desc->verbose)
+		printf("SYNC target_addr = 0x%lx\n",
+		       desc->its_sync_cmd.col->target_address);
+}
+
+static void its_build_movi_cmd(struct its_cmd_block *cmd,
+			       struct its_cmd_desc *desc)
+{
+	its_encode_cmd(cmd, GITS_CMD_MOVI);
+	its_encode_devid(cmd, desc->its_movi_cmd.dev->device_id);
+	its_encode_event_id(cmd, desc->its_movi_cmd.event_id);
+	its_encode_collection(cmd, desc->its_movi_cmd.col->col_id);
+	its_fixup_cmd(cmd);
+	if (desc->verbose)
+		printf("MOVI dev_id=%d event_id = %d col_id=%d\n",
+		       desc->its_movi_cmd.dev->device_id,
+		       desc->its_movi_cmd.event_id,
+		       desc->its_movi_cmd.col->col_id);
+}
+
+void __its_send_mapd(struct its_device *dev, int valid, bool verbose)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_mapd_cmd.dev = dev;
+	desc.its_mapd_cmd.valid = !!valid;
+	desc.verbose = verbose;
+
+	its_send_single_command(its_build_mapd_cmd, &desc);
+}
+
+void __its_send_mapc(struct its_collection *col, int valid, bool verbose)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_mapc_cmd.col = col;
+	desc.its_mapc_cmd.valid = !!valid;
+	desc.verbose = verbose;
+
+	its_send_single_command(its_build_mapc_cmd, &desc);
+}
+
+void __its_send_mapti(struct its_device *dev, u32 irq_id,
+		      u32 event_id, struct its_collection *col, bool verbose)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_mapti_cmd.dev = dev;
+	desc.its_mapti_cmd.phys_id = irq_id;
+	desc.its_mapti_cmd.event_id = event_id;
+	desc.its_mapti_cmd.col_id = col->col_id;
+	desc.verbose = verbose;
+
+	its_send_single_command(its_build_mapti_cmd, &desc);
+}
+
+void __its_send_int(struct its_device *dev, u32 event_id, bool verbose)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_int_cmd.dev = dev;
+	desc.its_int_cmd.event_id = event_id;
+	desc.verbose = verbose;
+
+	its_send_single_command(its_build_int_cmd, &desc);
+}
+
+void __its_send_movi(struct its_device *dev, struct its_collection *col,
+		     u32 id, bool verbose)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_movi_cmd.dev = dev;
+	desc.its_movi_cmd.col = col;
+	desc.its_movi_cmd.event_id = id;
+	desc.verbose = verbose;
+
+	its_send_single_command(its_build_movi_cmd, &desc);
+}
+
+void __its_send_invall(struct its_collection *col, bool verbose)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_invall_cmd.col = col;
+	desc.verbose = verbose;
+
+	its_send_single_command(its_build_invall_cmd, &desc);
+}
+
+void __its_send_inv(struct its_device *dev, u32 event_id, bool verbose)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_inv_cmd.dev = dev;
+	desc.its_inv_cmd.event_id = event_id;
+	desc.verbose = verbose;
+
+	its_send_single_command(its_build_inv_cmd, &desc);
+}
+
+void __its_send_discard(struct its_device *dev, u32 event_id, bool verbose)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_discard_cmd.dev = dev;
+	desc.its_discard_cmd.event_id = event_id;
+	desc.verbose = verbose;
+
+	its_send_single_command(its_build_discard_cmd, &desc);
+}
+
+void __its_send_clear(struct its_device *dev, u32 event_id, bool verbose)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_clear_cmd.dev = dev;
+	desc.its_clear_cmd.event_id = event_id;
+	desc.verbose = verbose;
+
+	its_send_single_command(its_build_clear_cmd, &desc);
+}
+
+void __its_send_sync(struct its_collection *col, bool verbose)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_sync_cmd.col = col;
+	desc.verbose = verbose;
+
+	its_send_single_command(its_build_sync_cmd, &desc);
+}
+
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 10/13] arm/arm64: ITS: INT functional tests
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
                   ` (8 preceding siblings ...)
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 09/13] arm/arm64: ITS: Commands Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-30 10:43   ` Zenghui Yu
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 11/13] arm/run: Allow Migration tests Eric Auger
                   ` (2 subsequent siblings)
  12 siblings, 1 reply; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

Triggers LPIs through the INT command.

the test checks the LPI hits the right CPU and triggers
the right LPI intid, ie. the translation is correct.

Updates to the config table also are tested, along with inv
and invall commands.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---
v5 -> v6:
- removed collection-unmap test
- moved the collection invalidation after the MAPCs

v4 -> v5:
- move the test stub from the header to arm/gic.c

v3 -> v4:
- assert in lpi_handler if the interrupt is not an LPI
- remove check_lpi_stats from its_prerequisites()

v2 -> v3:
- add comments
- keep the report_skip in case there aren't 4 vcpus to be able to
  run other tests in the its category.
- fix the prefix pop
- move its_event and its_stats to arm/gic.c
---
 arm/gic.c         | 215 +++++++++++++++++++++++++++++++++++++++++++---
 arm/unittests.cfg |   7 ++
 2 files changed, 211 insertions(+), 11 deletions(-)

diff --git a/arm/gic.c b/arm/gic.c
index 649ed81..5f1e595 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -159,6 +159,85 @@ static void ipi_handler(struct pt_regs *regs __unused)
 	}
 }
 
+static void setup_irq(irq_handler_fn handler)
+{
+	gic_enable_defaults();
+#ifdef __arm__
+	install_exception_handler(EXCPTN_IRQ, handler);
+#else
+	install_irq_handler(EL1H_IRQ, handler);
+#endif
+	local_irq_enable();
+}
+
+#if defined(__aarch64__)
+struct its_event {
+	int cpu_id;
+	int lpi_id;
+};
+
+struct its_stats {
+	struct its_event expected;
+	struct its_event observed;
+};
+
+static struct its_stats lpi_stats;
+
+static void lpi_handler(struct pt_regs *regs __unused)
+{
+	u32 irqstat = gic_read_iar();
+	int irqnr = gic_iar_irqnr(irqstat);
+
+	gic_write_eoir(irqstat);
+	assert(irqnr >= 8192);
+	smp_rmb(); /* pairs with wmb in lpi_stats_expect */
+	lpi_stats.observed.cpu_id = smp_processor_id();
+	lpi_stats.observed.lpi_id = irqnr;
+	smp_wmb(); /* pairs with rmb in check_lpi_stats */
+}
+
+static void lpi_stats_expect(int exp_cpu_id, int exp_lpi_id)
+{
+	lpi_stats.expected.cpu_id = exp_cpu_id;
+	lpi_stats.expected.lpi_id = exp_lpi_id;
+	lpi_stats.observed.cpu_id = -1;
+	lpi_stats.observed.lpi_id = -1;
+	smp_wmb(); /* pairs with rmb in handler */
+}
+
+static void check_lpi_stats(const char *msg)
+{
+	bool pass = false;
+
+	mdelay(100);
+	smp_rmb(); /* pairs with wmb in lpi_handler */
+	if (lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id ||
+	    lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id) {
+		if (lpi_stats.observed.cpu_id == -1 &&
+		    lpi_stats.observed.lpi_id == -1) {
+			report_info("No LPI received whereas (cpuid=%d, intid=%d) "
+				    "was expected", lpi_stats.expected.cpu_id,
+				    lpi_stats.expected.lpi_id);
+		} else {
+			report_info("Unexpected LPI (cpuid=%d, intid=%d)",
+				    lpi_stats.observed.cpu_id,
+				    lpi_stats.observed.lpi_id);
+		}
+	} else {
+		pass = true;
+	}
+	report(pass, "%s", msg);
+}
+
+static void secondary_lpi_test(void)
+{
+	setup_irq(lpi_handler);
+	cpumask_set_cpu(smp_processor_id(), &ready);
+	while (1)
+		wfi();
+}
+#endif
+
 static void gicv2_ipi_send_self(void)
 {
 	writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR);
@@ -216,17 +295,6 @@ static void ipi_test_smp(void)
 	report_prefix_pop();
 }
 
-static void setup_irq(irq_handler_fn handler)
-{
-	gic_enable_defaults();
-#ifdef __arm__
-	install_exception_handler(EXCPTN_IRQ, handler);
-#else
-	install_irq_handler(EL1H_IRQ, handler);
-#endif
-	local_irq_enable();
-}
-
 static void ipi_send(void)
 {
 	setup_irq(ipi_handler);
@@ -521,6 +589,7 @@ static void gic_test_mmio(void)
 #if defined(__arm__)
 
 static void test_its_introspection(void) {}
+static void test_its_trigger(void) {}
 
 #else /* __aarch64__ */
 
@@ -559,6 +628,126 @@ static void test_its_introspection(void)
 	report_info("collection table entry_size = 0x%x", coll_baser->esz);
 }
 
+static int its_prerequisites(int nb_cpus)
+{
+	int cpu;
+
+	if (!gicv3_its_base()) {
+		report_skip("No ITS, skip ...");
+		return -1;
+	}
+
+	if (nr_cpus < nb_cpus) {
+		report_skip("Test requires at least %d vcpus", nb_cpus);
+		return -1;
+	}
+
+	stats_reset();
+
+	setup_irq(lpi_handler);
+
+	for_each_present_cpu(cpu) {
+		if (cpu == 0)
+			continue;
+		smp_boot_secondary(cpu, secondary_lpi_test);
+	}
+	wait_on_ready();
+
+	its_enable_defaults();
+
+	return 0;
+}
+
+static void test_its_trigger(void)
+{
+	struct its_collection *col3, *col2;
+	struct its_device *dev2, *dev7;
+
+	if (its_prerequisites(4))
+		return;
+
+	dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
+	dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
+
+	col3 = its_create_collection(3 /* col id */, 3/* target PE */);
+	col2 = its_create_collection(2 /* col id */, 2/* target PE */);
+
+	gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
+	gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
+
+	report_prefix_push("int");
+	/*
+	 * dev=2, eventid=20  -> lpi= 8195, col=3
+	 * dev=7, eventid=255 -> lpi= 8196, col=2
+	 * Trigger dev2, eventid=20 and dev7, eventid=255
+	 * Check both LPIs hit
+	 */
+
+	its_send_mapd(dev2, true);
+	its_send_mapd(dev7, true);
+
+	its_send_mapc(col3, true);
+	its_send_mapc(col2, true);
+
+	its_send_invall(col2);
+	its_send_invall(col3);
+
+	its_send_mapti(dev2, 8195 /* lpi id */, 20 /* event id */, col3);
+	its_send_mapti(dev7, 8196 /* lpi id */, 255 /* event id */, col2);
+
+	lpi_stats_expect(3, 8195);
+	its_send_int(dev2, 20);
+	check_lpi_stats("dev=2, eventid=20  -> lpi= 8195, col=3");
+
+	lpi_stats_expect(2, 8196);
+	its_send_int(dev7, 255);
+	check_lpi_stats("dev=7, eventid=255 -> lpi= 8196, col=2");
+
+	report_prefix_pop();
+
+	report_prefix_push("inv/invall");
+
+	/*
+	 * disable 8195, check dev2/eventid=20 does not trigger the
+	 * corresponding LPI
+	 */
+	gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT & ~LPI_PROP_ENABLED);
+	its_send_inv(dev2, 20);
+
+	lpi_stats_expect(-1, -1);
+	its_send_int(dev2, 20);
+	check_lpi_stats("dev2/eventid=20 does not trigger any LPI");
+
+	/*
+	 * re-enable the LPI but willingly do not call invall
+	 * so the change in config is not taken into account.
+	 * The LPI should not hit
+	 */
+	gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
+	lpi_stats_expect(-1, -1);
+	its_send_int(dev2, 20);
+	check_lpi_stats("dev2/eventid=20 still does not trigger any LPI");
+
+	/* Now call the invall and check the LPI hits */
+	its_send_invall(col3);
+	lpi_stats_expect(3, 8195);
+	its_send_int(dev2, 20);
+	check_lpi_stats("dev2/eventid=20 now triggers an LPI");
+
+	report_prefix_pop();
+
+	report_prefix_push("mapd valid=false");
+	/*
+	 * Unmap device 2 and check the eventid 20 formerly
+	 * attached to it does not hit anymore
+	 */
+
+	its_send_mapd(dev2, false);
+	lpi_stats_expect(-1, -1);
+	its_send_int(dev2, 20);
+	check_lpi_stats("no LPI after device unmap");
+	report_prefix_pop();
+}
 #endif
 
 int main(int argc, char **argv)
@@ -592,6 +781,10 @@ int main(int argc, char **argv)
 		report_prefix_push(argv[1]);
 		gic_test_mmio();
 		report_prefix_pop();
+	} else if (!strcmp(argv[1], "its-trigger")) {
+		report_prefix_push(argv[1]);
+		test_its_trigger();
+		report_prefix_pop();
 	} else if (strcmp(argv[1], "its-introspection") == 0) {
 		report_prefix_push(argv[1]);
 		test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 23d378e..b9a7a2c 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -129,6 +129,13 @@ extra_params = -machine gic-version=3 -append 'its-introspection'
 groups = its
 arch = arm64
 
+[its-trigger]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'its-trigger'
+groups = its
+arch = arm64
+
 # Test PSCI emulation
 [psci]
 file = psci.flat
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 11/13] arm/run: Allow Migration tests
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
                   ` (9 preceding siblings ...)
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 10/13] arm/arm64: ITS: INT functional tests Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 12/13] arm/arm64: ITS: migration tests Eric Auger
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 13/13] arm/arm64: ITS: pending table migration test Eric Auger
  12 siblings, 0 replies; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

Let's link getchar.o to use puts and getchar from the
tests.

Then allow tests belonging to the migration group to
trigger the migration from the test code by putting
"migrate" into the uart. Then the code can wait for the
migration completion by using getchar().

The __getchar implement is minimalist as it just reads the
data register. It is just meant to read the single character
emitted at the end of the migration by the runner script.

It is not meant to read more data (FIFOs are not enabled).

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---

v3 -> v4:
- remove space around Elvis operator
- rename ___getchar into do_getchar

v2 -> v3:
- take the lock
- assert if more than 16 chars
- removed Thomas' R-b
---
 arm/Makefile.common |  2 +-
 arm/run             |  2 +-
 lib/arm/io.c        | 28 ++++++++++++++++++++++++++++
 3 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/arm/Makefile.common b/arm/Makefile.common
index b8988f2..a123e85 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -32,7 +32,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
 asm-offsets = lib/$(ARCH)/asm-offsets.h
 include $(SRCDIR)/scripts/asm-offsets.mak
 
-cflatobjs += lib/util.o
+cflatobjs += lib/util.o lib/getchar.o
 cflatobjs += lib/alloc_phys.o
 cflatobjs += lib/alloc_page.o
 cflatobjs += lib/vmalloc.o
diff --git a/arm/run b/arm/run
index 277db9b..a390ca5 100755
--- a/arm/run
+++ b/arm/run
@@ -61,6 +61,6 @@ fi
 M+=",accel=$ACCEL"
 command="$qemu -nodefaults $M -cpu $processor $chr_testdev $pci_testdev"
 command+=" -display none -serial stdio -kernel"
-command="$(timeout_cmd) $command"
+command="$(migration_cmd) $(timeout_cmd) $command"
 
 run_qemu $command "$@"
diff --git a/lib/arm/io.c b/lib/arm/io.c
index 99fd315..343e108 100644
--- a/lib/arm/io.c
+++ b/lib/arm/io.c
@@ -87,6 +87,34 @@ void puts(const char *s)
 	spin_unlock(&uart_lock);
 }
 
+static int do_getchar(void)
+{
+	int c;
+
+	spin_lock(&uart_lock);
+	c = readb(uart0_base);
+	spin_unlock(&uart_lock);
+
+	return c ?: -1;
+}
+
+/*
+ * Minimalist implementation for migration completion detection.
+ * Without FIFOs enabled on the QEMU UART device we just read
+ * the data register: we cannot read more than 16 characters.
+ */
+int __getchar(void)
+{
+	int c = do_getchar();
+	static int count;
+
+	if (c != -1)
+		++count;
+
+	assert(count < 16);
+
+	return c;
+}
 
 /*
  * Defining halt to take 'code' as an argument guarantees that it will
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 12/13] arm/arm64: ITS: migration tests
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
                   ` (10 preceding siblings ...)
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 11/13] arm/run: Allow Migration tests Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-30 10:55   ` Zenghui Yu
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 13/13] arm/arm64: ITS: pending table migration test Eric Auger
  12 siblings, 1 reply; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

This test maps LPIs (populates the device table, the collection table,
interrupt translation tables, configuration table), migrates and make
sure the translation is correct on the destination.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---
v4 -> v5:
- move stub from header to arm/gic.c

v3 -> v4:
- assert in its_get_device/collection if the id is not found
---
 arm/gic.c                  | 58 ++++++++++++++++++++++++++++++++++----
 arm/unittests.cfg          |  8 ++++++
 lib/arm64/asm/gic-v3-its.h |  3 ++
 lib/arm64/gic-v3-its.c     | 22 +++++++++++++++
 4 files changed, 85 insertions(+), 6 deletions(-)

diff --git a/arm/gic.c b/arm/gic.c
index 5f1e595..6ecfdbc 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -590,6 +590,7 @@ static void gic_test_mmio(void)
 
 static void test_its_introspection(void) {}
 static void test_its_trigger(void) {}
+static void test_its_migration(void) {}
 
 #else /* __aarch64__ */
 
@@ -658,13 +659,19 @@ static int its_prerequisites(int nb_cpus)
 	return 0;
 }
 
-static void test_its_trigger(void)
+/*
+ * Setup the configuration for those mappings:
+ * dev_id=2 event=20 -> vcpu 3, intid=8195
+ * dev_id=7 event=255 -> vcpu 2, intid=8196
+ * LPIs ready to hit
+ */
+static int its_setup1(void)
 {
 	struct its_collection *col3, *col2;
 	struct its_device *dev2, *dev7;
 
 	if (its_prerequisites(4))
-		return;
+		return -1;
 
 	dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
 	dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
@@ -675,14 +682,10 @@ static void test_its_trigger(void)
 	gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
 	gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
 
-	report_prefix_push("int");
 	/*
 	 * dev=2, eventid=20  -> lpi= 8195, col=3
 	 * dev=7, eventid=255 -> lpi= 8196, col=2
-	 * Trigger dev2, eventid=20 and dev7, eventid=255
-	 * Check both LPIs hit
 	 */
-
 	its_send_mapd(dev2, true);
 	its_send_mapd(dev7, true);
 
@@ -694,6 +697,22 @@ static void test_its_trigger(void)
 
 	its_send_mapti(dev2, 8195 /* lpi id */, 20 /* event id */, col3);
 	its_send_mapti(dev7, 8196 /* lpi id */, 255 /* event id */, col2);
+	return 0;
+}
+
+static void test_its_trigger(void)
+{
+	struct its_collection *col3;
+	struct its_device *dev2, *dev7;
+
+	if (its_setup1())
+		return;
+
+	col3 = its_get_collection(3);
+	dev2 = its_get_device(2);
+	dev7 = its_get_device(7);
+
+	report_prefix_push("int");
 
 	lpi_stats_expect(3, 8195);
 	its_send_int(dev2, 20);
@@ -748,6 +767,29 @@ static void test_its_trigger(void)
 	check_lpi_stats("no LPI after device unmap");
 	report_prefix_pop();
 }
+
+static void test_its_migration(void)
+{
+	struct its_device *dev2, *dev7;
+
+	if (its_setup1())
+		return;
+
+	dev2 = its_get_device(2);
+	dev7 = its_get_device(7);
+
+	puts("Now migrate the VM, then press a key to continue...\n");
+	(void)getchar();
+	report_info("Migration complete");
+
+	lpi_stats_expect(3, 8195);
+	its_send_int(dev2, 20);
+	check_lpi_stats("dev2/eventid=20 triggers LPI 8195 en PE #3 after migration");
+
+	lpi_stats_expect(2, 8196);
+	its_send_int(dev7, 255);
+	check_lpi_stats("dev7/eventid=255 triggers LPI 8196 on PE #2 after migration");
+}
 #endif
 
 int main(int argc, char **argv)
@@ -785,6 +827,10 @@ int main(int argc, char **argv)
 		report_prefix_push(argv[1]);
 		test_its_trigger();
 		report_prefix_pop();
+	} else if (!strcmp(argv[1], "its-migration")) {
+		report_prefix_push(argv[1]);
+		test_its_migration();
+		report_prefix_pop();
 	} else if (strcmp(argv[1], "its-introspection") == 0) {
 		report_prefix_push(argv[1]);
 		test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index b9a7a2c..480adec 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
 groups = its
 arch = arm64
 
+[its-migration]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-migration'
+groups = its migration
+arch = arm64
+
 # Test PSCI emulation
 [psci]
 file = psci.flat
diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
index e1e42c3..412f438 100644
--- a/lib/arm64/asm/gic-v3-its.h
+++ b/lib/arm64/asm/gic-v3-its.h
@@ -168,4 +168,7 @@ extern void __its_send_sync(struct its_collection *col, bool verbose);
 #define its_send_movi_nv(dev, col, id)			__its_send_movi(dev, col, id, false)
 #define its_send_sync_nv(col)				__its_send_sync(col, false)
 
+extern struct its_device *its_get_device(u32 id);
+extern struct its_collection *its_get_collection(u32 id);
+
 #endif /* _ASMARM64_GIC_V3_ITS_H_ */
diff --git a/lib/arm64/gic-v3-its.c b/lib/arm64/gic-v3-its.c
index f0a0381..c7755d9 100644
--- a/lib/arm64/gic-v3-its.c
+++ b/lib/arm64/gic-v3-its.c
@@ -147,3 +147,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
 	its_data.nr_collections++;
 	return new;
 }
+
+struct its_device *its_get_device(u32 id)
+{
+	int i;
+
+	for (i = 0; i < GITS_MAX_DEVICES; i++) {
+		if (its_data.devices[i].device_id == id)
+			return &its_data.devices[i];
+	}
+	assert(0);
+}
+
+struct its_collection *its_get_collection(u32 id)
+{
+	int i;
+
+	for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
+		if (its_data.collections[i].col_id == id)
+			return &its_data.collections[i];
+	}
+	assert(0);
+}
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* [kvm-unit-tests PATCH v7 13/13] arm/arm64: ITS: pending table migration test
  2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
                   ` (11 preceding siblings ...)
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 12/13] arm/arm64: ITS: migration tests Eric Auger
@ 2020-03-20  9:24 ` Eric Auger
  2020-03-30 12:06   ` Zenghui Yu
  12 siblings, 1 reply; 34+ messages in thread
From: Eric Auger @ 2020-03-20  9:24 UTC (permalink / raw)
  To: eric.auger.pro, eric.auger, maz, kvmarm, kvm, qemu-devel, qemu-arm
  Cc: peter.maydell, drjones, andre.przywara, thuth, yuzenghui,
	alexandru.elisei

Add two new migration tests. One testing the migration of
a topology where collection were unmapped. The second test
checks the migration of the pending table.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---

v6 -> v7:
- test_migrate_unmapped_collection now uses pe0=0. Otherwise,
  depending on SMP value it collides with collections created
  by setup1()
- use of for_each_present_cpu

v5 -> v6:
- s/Set the PTZ/Clear the PTZ
- Move the collection inval after MAPC
- remove the unmap collection test

v4 -> v5:
- move stub from header to arm/gic.c

v3 -> v4:
- do not talk about odd/even CPUs, use pe0 and pe1
- comment the delay

v2 -> v3:
- tests belong to both its and migration groups
- use LPI(i)
- gicv3_lpi_set_pending_table_bit renamed into gicv3_lpi_set_clr_pending
---
 arm/gic.c         | 138 ++++++++++++++++++++++++++++++++++++++++++++++
 arm/unittests.cfg |  16 ++++++
 2 files changed, 154 insertions(+)

diff --git a/arm/gic.c b/arm/gic.c
index 6ecfdbc..2c661dc 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -193,6 +193,7 @@ static void lpi_handler(struct pt_regs *regs __unused)
 	smp_rmb(); /* pairs with wmb in lpi_stats_expect */
 	lpi_stats.observed.cpu_id = smp_processor_id();
 	lpi_stats.observed.lpi_id = irqnr;
+	acked[lpi_stats.observed.cpu_id]++;
 	smp_wmb(); /* pairs with rmb in check_lpi_stats */
 }
 
@@ -236,6 +237,22 @@ static void secondary_lpi_test(void)
 	while (1)
 		wfi();
 }
+
+static void check_lpi_hits(int *expected, const char *msg)
+{
+	bool pass = true;
+	int i;
+
+	for_each_present_cpu(i) {
+		if (acked[i] != expected[i]) {
+			report_info("expected %d LPIs on PE #%d, %d observed",
+				    expected[i], i, acked[i]);
+			pass = false;
+			break;
+		}
+	}
+	report(pass, "%s", msg);
+}
 #endif
 
 static void gicv2_ipi_send_self(void)
@@ -591,6 +608,8 @@ static void gic_test_mmio(void)
 static void test_its_introspection(void) {}
 static void test_its_trigger(void) {}
 static void test_its_migration(void) {}
+static void test_its_pending_migration(void) {}
+static void test_migrate_unmapped_collection(void) {}
 
 #else /* __aarch64__ */
 
@@ -659,6 +678,15 @@ static int its_prerequisites(int nb_cpus)
 	return 0;
 }
 
+static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
+		    struct its_collection *col)
+{
+	assert(dev && col);
+
+	its_send_mapti(dev, physid, eventid, col);
+	gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
+}
+
 /*
  * Setup the configuration for those mappings:
  * dev_id=2 event=20 -> vcpu 3, intid=8195
@@ -790,6 +818,108 @@ static void test_its_migration(void)
 	its_send_int(dev7, 255);
 	check_lpi_stats("dev7/eventid=255 triggers LPI 8196 on PE #2 after migration");
 }
+
+static void test_migrate_unmapped_collection(void)
+{
+	struct its_collection *col;
+	struct its_device *dev2, *dev7;
+	int pe0 = 0;
+	u8 config;
+
+	if (its_setup1())
+		return;
+
+	col = its_create_collection(pe0, pe0);
+	dev2 = its_get_device(2);
+	dev7 = its_get_device(7);
+
+	/* MAPTI with the collection unmapped */
+	set_lpi(dev2, 0, 8192, col);
+
+	puts("Now migrate the VM, then press a key to continue...\n");
+	(void)getchar();
+	report_info("Migration complete");
+
+	/* on the destination, map the collection */
+	its_send_mapc(col, true);
+	its_send_invall(col);
+
+	lpi_stats_expect(2, 8196);
+	its_send_int(dev7, 255);
+	check_lpi_stats("dev7/eventid= 255 triggered LPI 8196 on PE #2");
+
+	config = gicv3_lpi_get_config(8192);
+	report(config == LPI_PROP_DEFAULT,
+	       "Config of LPI 8192 was properly migrated");
+
+	lpi_stats_expect(pe0, 8192);
+	its_send_int(dev2, 0);
+	check_lpi_stats("dev2/eventid = 0 triggered LPI 8192 on PE0");
+}
+
+static void test_its_pending_migration(void)
+{
+	struct its_device *dev;
+	struct its_collection *collection[2];
+	int *expected = malloc(nr_cpus * sizeof(int));
+	int pe0 = nr_cpus - 1, pe1 = nr_cpus - 2;
+	u64 pendbaser;
+	void *ptr;
+	int i;
+
+	if (its_prerequisites(4))
+		return;
+
+	dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
+	its_send_mapd(dev, true);
+
+	collection[0] = its_create_collection(pe0, pe0);
+	collection[1] = its_create_collection(pe1, pe1);
+	its_send_mapc(collection[0], true);
+	its_send_mapc(collection[1], true);
+
+	/* disable lpi at redist level */
+	gicv3_lpi_rdist_disable(pe0);
+	gicv3_lpi_rdist_disable(pe1);
+
+	/* lpis are interleaved inbetween the 2 PEs */
+	for (i = 0; i < 256; i++) {
+		struct its_collection *col = i % 2 ? collection[0] :
+						     collection[1];
+		int vcpu = col->target_address >> 16;
+
+		its_send_mapti(dev, LPI(i), i, col);
+		gicv3_lpi_set_config(LPI(i), LPI_PROP_DEFAULT);
+		gicv3_lpi_set_clr_pending(vcpu, LPI(i), true);
+	}
+	its_send_invall(collection[0]);
+	its_send_invall(collection[1]);
+
+	/* Clear the PTZ bit on each pendbaser */
+
+	expected[pe0] = 128;
+	expected[pe1] = 128;
+
+	ptr = gicv3_data.redist_base[pe0] + GICR_PENDBASER;
+	pendbaser = readq(ptr);
+	writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
+
+	ptr = gicv3_data.redist_base[pe1] + GICR_PENDBASER;
+	pendbaser = readq(ptr);
+	writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
+
+	gicv3_lpi_rdist_enable(pe0);
+	gicv3_lpi_rdist_enable(pe1);
+
+	puts("Now migrate the VM, then press a key to continue...\n");
+	(void)getchar();
+	report_info("Migration complete");
+
+	/* let's wait for the 256 LPIs to be handled */
+	mdelay(1000);
+
+	check_lpi_hits(expected, "128 LPIs on both PE0 and PE1 after migration");
+}
 #endif
 
 int main(int argc, char **argv)
@@ -831,6 +961,14 @@ int main(int argc, char **argv)
 		report_prefix_push(argv[1]);
 		test_its_migration();
 		report_prefix_pop();
+	} else if (!strcmp(argv[1], "its-pending-migration")) {
+		report_prefix_push(argv[1]);
+		test_its_pending_migration();
+		report_prefix_pop();
+	} else if (!strcmp(argv[1], "its-migrate-unmapped-collection")) {
+		report_prefix_push(argv[1]);
+		test_migrate_unmapped_collection();
+		report_prefix_pop();
 	} else if (strcmp(argv[1], "its-introspection") == 0) {
 		report_prefix_push(argv[1]);
 		test_its_introspection();
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 480adec..b96f0a1 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -144,6 +144,22 @@ extra_params = -machine gic-version=3 -append 'its-migration'
 groups = its migration
 arch = arm64
 
+[its-pending-migration]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-pending-migration'
+groups = its migration
+arch = arm64
+
+[its-migrate-unmapped-collection]
+file = gic.flat
+smp = $MAX_SMP
+accel = kvm
+extra_params = -machine gic-version=3 -append 'its-migrate-unmapped-collection'
+groups = its migration
+arch = arm64
+
 # Test PSCI emulation
 [psci]
 file = psci.flat
-- 
2.20.1



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 08/13] arm/arm64: ITS: Device and collection Initialization
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 08/13] arm/arm64: ITS: Device and collection Initialization Eric Auger
@ 2020-03-25  8:10   ` Zenghui Yu
  2020-03-25 21:20     ` Auger Eric
  0 siblings, 1 reply; 34+ messages in thread
From: Zenghui Yu @ 2020-03-25  8:10 UTC (permalink / raw)
  To: Eric Auger
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Eric,

On 2020/3/20 17:24, Eric Auger wrote:
> Introduce an helper functions to register
> - a new device, characterized by its device id and the
>    max number of event IDs that dimension its ITT (Interrupt
>    Translation Table).  The function allocates the ITT.
> 
> - a new collection, characterized by its ID and the
>    target processing engine (PE).
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> 
> ---
> 
> v3 -> v4:
> - remove unused its_baser variable from its_create_device()
> - use get_order()
> - device->itt becomes a GVA instead of GPA
> 
> v2 -> v3:
> - s/report_abort/assert
> 
> v1 -> v2:
> - s/nb_/nr_
> ---
>   lib/arm64/asm/gic-v3-its.h | 19 +++++++++++++++++++
>   lib/arm64/gic-v3-its.c     | 38 ++++++++++++++++++++++++++++++++++++++
>   2 files changed, 57 insertions(+)
> 
> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
> index 4683011..adcb642 100644
> --- a/lib/arm64/asm/gic-v3-its.h
> +++ b/lib/arm64/asm/gic-v3-its.h
> @@ -31,6 +31,19 @@ struct its_baser {
>   };
>   
>   #define GITS_BASER_NR_REGS		8
> +#define GITS_MAX_DEVICES		8
> +#define GITS_MAX_COLLECTIONS		8
> +
> +struct its_device {
> +	u32 device_id;	/* device ID */
> +	u32 nr_ites;	/* Max Interrupt Translation Entries */
> +	void *itt;	/* Interrupt Translation Table GVA */
> +};
> +
> +struct its_collection {
> +	u64 target_address;
> +	u16 col_id;
> +};
>   
>   struct its_data {
>   	void *base;
> @@ -39,6 +52,10 @@ struct its_data {
>   	struct its_baser coll_baser;
>   	struct its_cmd_block *cmd_base;
>   	struct its_cmd_block *cmd_write;
> +	struct its_device devices[GITS_MAX_DEVICES];
> +	u32 nr_devices;		/* Allocated Devices */
> +	struct its_collection collections[GITS_MAX_COLLECTIONS];
> +	u32 nr_collections;	/* Allocated Collections */
>   };
>   
>   extern struct its_data its_data;
> @@ -93,5 +110,7 @@ extern void its_parse_typer(void);
>   extern void its_init(void);
>   extern int its_baser_lookup(int i, struct its_baser *baser);
>   extern void its_enable_defaults(void);
> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> +extern struct its_collection *its_create_collection(u32 col_id, u32 target_pe);

Maybe use 'u16 col_id'?

Besides,
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>


Thanks



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 08/13] arm/arm64: ITS: Device and collection Initialization
  2020-03-25  8:10   ` Zenghui Yu
@ 2020-03-25 21:20     ` Auger Eric
  2020-03-30  9:13       ` Andrew Jones
  0 siblings, 1 reply; 34+ messages in thread
From: Auger Eric @ 2020-03-25 21:20 UTC (permalink / raw)
  To: Zenghui Yu
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Zenghui,

On 3/25/20 9:10 AM, Zenghui Yu wrote:
> Hi Eric,
> 
> On 2020/3/20 17:24, Eric Auger wrote:
>> Introduce an helper functions to register
>> - a new device, characterized by its device id and the
>>    max number of event IDs that dimension its ITT (Interrupt
>>    Translation Table).  The function allocates the ITT.
>>
>> - a new collection, characterized by its ID and the
>>    target processing engine (PE).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v3 -> v4:
>> - remove unused its_baser variable from its_create_device()
>> - use get_order()
>> - device->itt becomes a GVA instead of GPA
>>
>> v2 -> v3:
>> - s/report_abort/assert
>>
>> v1 -> v2:
>> - s/nb_/nr_
>> ---
>>   lib/arm64/asm/gic-v3-its.h | 19 +++++++++++++++++++
>>   lib/arm64/gic-v3-its.c     | 38 ++++++++++++++++++++++++++++++++++++++
>>   2 files changed, 57 insertions(+)
>>
>> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
>> index 4683011..adcb642 100644
>> --- a/lib/arm64/asm/gic-v3-its.h
>> +++ b/lib/arm64/asm/gic-v3-its.h
>> @@ -31,6 +31,19 @@ struct its_baser {
>>   };
>>     #define GITS_BASER_NR_REGS        8
>> +#define GITS_MAX_DEVICES        8
>> +#define GITS_MAX_COLLECTIONS        8
>> +
>> +struct its_device {
>> +    u32 device_id;    /* device ID */
>> +    u32 nr_ites;    /* Max Interrupt Translation Entries */
>> +    void *itt;    /* Interrupt Translation Table GVA */
>> +};
>> +
>> +struct its_collection {
>> +    u64 target_address;
>> +    u16 col_id;
>> +};
>>     struct its_data {
>>       void *base;
>> @@ -39,6 +52,10 @@ struct its_data {
>>       struct its_baser coll_baser;
>>       struct its_cmd_block *cmd_base;
>>       struct its_cmd_block *cmd_write;
>> +    struct its_device devices[GITS_MAX_DEVICES];
>> +    u32 nr_devices;        /* Allocated Devices */
>> +    struct its_collection collections[GITS_MAX_COLLECTIONS];
>> +    u32 nr_collections;    /* Allocated Collections */
>>   };
>>     extern struct its_data its_data;
>> @@ -93,5 +110,7 @@ extern void its_parse_typer(void);
>>   extern void its_init(void);
>>   extern int its_baser_lookup(int i, struct its_baser *baser);
>>   extern void its_enable_defaults(void);
>> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
>> +extern struct its_collection *its_create_collection(u32 col_id, u32
>> target_pe);
> 
> Maybe use 'u16 col_id'?
fair enough. At this point, not sure this is worth a respin though ;-)
> 
> Besides,
> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
Thanks!

Eric




> 
> 
> Thanks
> 
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests Eric Auger
@ 2020-03-30  8:30   ` Zenghui Yu
  2020-03-30  8:46     ` Auger Eric
  0 siblings, 1 reply; 34+ messages in thread
From: Zenghui Yu @ 2020-03-30  8:30 UTC (permalink / raw)
  To: Eric Auger
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Eric,

On 2020/3/20 17:24, Eric Auger wrote:
> +static void its_cmd_queue_init(void)
> +{
> +	unsigned long order = get_order(SZ_64K >> PAGE_SHIFT);
> +	u64 cbaser;
> +
> +	its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));

Shouldn't the cmd_base (and the cmd_write) be set as a GVA?

Otherwise I think we will end-up with memory corruption when writing
the command queue.  But it seems that everything just works fine ...
So I'm really confused here :-/

> +
> +	cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1)	| GITS_CBASER_VALID);
> +
> +	writeq(cbaser, its_data.base + GITS_CBASER);
> +
> +	its_data.cmd_write = its_data.cmd_base;
> +	writeq(0, its_data.base + GITS_CWRITER);
> +}

Otherwise this looks good,
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>


Thanks



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests
  2020-03-30  8:30   ` Zenghui Yu
@ 2020-03-30  8:46     ` Auger Eric
  2020-03-30  9:11       ` Andrew Jones
  0 siblings, 1 reply; 34+ messages in thread
From: Auger Eric @ 2020-03-30  8:46 UTC (permalink / raw)
  To: Zenghui Yu
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Zenghui,

On 3/30/20 10:30 AM, Zenghui Yu wrote:
> Hi Eric,
> 
> On 2020/3/20 17:24, Eric Auger wrote:
>> +static void its_cmd_queue_init(void)
>> +{
>> +    unsigned long order = get_order(SZ_64K >> PAGE_SHIFT);
>> +    u64 cbaser;
>> +
>> +    its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
> 
> Shouldn't the cmd_base (and the cmd_write) be set as a GVA?
yes it should
> 
> Otherwise I think we will end-up with memory corruption when writing
> the command queue.  But it seems that everything just works fine ...
> So I'm really confused here :-/
I was told by Paolo that the VA/PA memory map is flat in kvmunit test.

> 
>> +
>> +    cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1)    |
>> GITS_CBASER_VALID);
>> +
>> +    writeq(cbaser, its_data.base + GITS_CBASER);
>> +
>> +    its_data.cmd_write = its_data.cmd_base;
>> +    writeq(0, its_data.base + GITS_CWRITER);
>> +}
> 
> Otherwise this looks good,
> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
Thanks!

Eric
> 
> 
> Thanks
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests
  2020-03-30  8:46     ` Auger Eric
@ 2020-03-30  9:11       ` Andrew Jones
  2020-03-30  9:56         ` Auger Eric
  2020-03-30 12:20         ` Zenghui Yu
  0 siblings, 2 replies; 34+ messages in thread
From: Andrew Jones @ 2020-03-30  9:11 UTC (permalink / raw)
  To: Auger Eric
  Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, Zenghui Yu, alexandru.elisei, kvmarm,
	eric.auger.pro

On Mon, Mar 30, 2020 at 10:46:57AM +0200, Auger Eric wrote:
> Hi Zenghui,
> 
> On 3/30/20 10:30 AM, Zenghui Yu wrote:
> > Hi Eric,
> > 
> > On 2020/3/20 17:24, Eric Auger wrote:
> >> +static void its_cmd_queue_init(void)
> >> +{
> >> +    unsigned long order = get_order(SZ_64K >> PAGE_SHIFT);
> >> +    u64 cbaser;
> >> +
> >> +    its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
> > 
> > Shouldn't the cmd_base (and the cmd_write) be set as a GVA?
> yes it should

If it's supposed to be a virtual address, when why do the virt_to_phys?

> > 
> > Otherwise I think we will end-up with memory corruption when writing
> > the command queue.  But it seems that everything just works fine ...
> > So I'm really confused here :-/
> I was told by Paolo that the VA/PA memory map is flat in kvmunit test.

What does flat mean? kvm-unit-tests, at least arm/arm64, does prepare
an identity map of all physical memory, which explains why the above
is working. It's doing virt_to_phys(some-virt-addr), which gets a
phys addr, but when the ITS uses it as a virt addr it works because
we *also* have a virt addr == phys addr mapping in the default page
table, which is named "idmap" for good reason.

I think it would be better to test with the non-identity mapped addresses
though.

Thanks,
drew

> 
> > 
> >> +
> >> +    cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1)    |
> >> GITS_CBASER_VALID);
> >> +
> >> +    writeq(cbaser, its_data.base + GITS_CBASER);
> >> +
> >> +    its_data.cmd_write = its_data.cmd_base;
> >> +    writeq(0, its_data.base + GITS_CWRITER);
> >> +}
> > 
> > Otherwise this looks good,
> > Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
> Thanks!
> 
> Eric
> > 
> > 
> > Thanks
> > 
> 
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 08/13] arm/arm64: ITS: Device and collection Initialization
  2020-03-25 21:20     ` Auger Eric
@ 2020-03-30  9:13       ` Andrew Jones
  0 siblings, 0 replies; 34+ messages in thread
From: Andrew Jones @ 2020-03-30  9:13 UTC (permalink / raw)
  To: Auger Eric
  Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, Zenghui Yu, alexandru.elisei, kvmarm,
	eric.auger.pro

On Wed, Mar 25, 2020 at 10:20:43PM +0100, Auger Eric wrote:
> Hi Zenghui,
> 
> On 3/25/20 9:10 AM, Zenghui Yu wrote:
> > Hi Eric,
> > 
> > On 2020/3/20 17:24, Eric Auger wrote:
> >> Introduce an helper functions to register
> >> - a new device, characterized by its device id and the
> >>    max number of event IDs that dimension its ITT (Interrupt
> >>    Translation Table).  The function allocates the ITT.
> >>
> >> - a new collection, characterized by its ID and the
> >>    target processing engine (PE).
> >>
> >> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> >>
> >> ---
> >>
> >> v3 -> v4:
> >> - remove unused its_baser variable from its_create_device()
> >> - use get_order()
> >> - device->itt becomes a GVA instead of GPA
> >>
> >> v2 -> v3:
> >> - s/report_abort/assert
> >>
> >> v1 -> v2:
> >> - s/nb_/nr_
> >> ---
> >>   lib/arm64/asm/gic-v3-its.h | 19 +++++++++++++++++++
> >>   lib/arm64/gic-v3-its.c     | 38 ++++++++++++++++++++++++++++++++++++++
> >>   2 files changed, 57 insertions(+)
> >>
> >> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
> >> index 4683011..adcb642 100644
> >> --- a/lib/arm64/asm/gic-v3-its.h
> >> +++ b/lib/arm64/asm/gic-v3-its.h
> >> @@ -31,6 +31,19 @@ struct its_baser {
> >>   };
> >>     #define GITS_BASER_NR_REGS        8
> >> +#define GITS_MAX_DEVICES        8
> >> +#define GITS_MAX_COLLECTIONS        8
> >> +
> >> +struct its_device {
> >> +    u32 device_id;    /* device ID */
> >> +    u32 nr_ites;    /* Max Interrupt Translation Entries */
> >> +    void *itt;    /* Interrupt Translation Table GVA */
> >> +};
> >> +
> >> +struct its_collection {
> >> +    u64 target_address;
> >> +    u16 col_id;
> >> +};
> >>     struct its_data {
> >>       void *base;
> >> @@ -39,6 +52,10 @@ struct its_data {
> >>       struct its_baser coll_baser;
> >>       struct its_cmd_block *cmd_base;
> >>       struct its_cmd_block *cmd_write;
> >> +    struct its_device devices[GITS_MAX_DEVICES];
> >> +    u32 nr_devices;        /* Allocated Devices */
> >> +    struct its_collection collections[GITS_MAX_COLLECTIONS];
> >> +    u32 nr_collections;    /* Allocated Collections */
> >>   };
> >>     extern struct its_data its_data;
> >> @@ -93,5 +110,7 @@ extern void its_parse_typer(void);
> >>   extern void its_init(void);
> >>   extern int its_baser_lookup(int i, struct its_baser *baser);
> >>   extern void its_enable_defaults(void);
> >> +extern struct its_device *its_create_device(u32 dev_id, int nr_ites);
> >> +extern struct its_collection *its_create_collection(u32 col_id, u32
> >> target_pe);
> > 
> > Maybe use 'u16 col_id'?
> fair enough. At this point, not sure this is worth a respin though ;-)

I'd like all the virt_to_phys calls removed where there are not necessary,
which was pointed out in a different patch. This can be fixed up at the
same time.

Thanks,
drew

> > 
> > Besides,
> > Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
> Thanks!
> 
> Eric
> 
> 
> 
> 
> > 
> > 
> > Thanks
> > 
> > 
> 
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 09/13] arm/arm64: ITS: Commands
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 09/13] arm/arm64: ITS: Commands Eric Auger
@ 2020-03-30  9:22   ` Zenghui Yu
  2020-03-30  9:57     ` Auger Eric
  0 siblings, 1 reply; 34+ messages in thread
From: Zenghui Yu @ 2020-03-30  9:22 UTC (permalink / raw)
  To: Eric Auger
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Eric,

On 2020/3/20 17:24, Eric Auger wrote:
> Implement main ITS commands. The code is largely inherited from
> the ITS driver.
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>

[...]

> +/* ITS COMMANDS */
> +
> +static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
> +{
> +	cmd->raw_cmd[0] &= ~0xffUL;
> +	cmd->raw_cmd[0] |= cmd_nr;
> +}
> +
> +static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
> +{
> +	cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
> +	cmd->raw_cmd[0] |= ((u64)devid) << 32;
> +}
> +
> +static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
> +{
> +	cmd->raw_cmd[1] &= ~0xffffffffUL;
> +	cmd->raw_cmd[1] |= id;
> +}
> +
> +static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
> +{
> +	cmd->raw_cmd[1] &= 0xffffffffUL;
> +	cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
> +}
> +
> +static void its_encode_size(struct its_cmd_block *cmd, u8 size)
> +{
> +	cmd->raw_cmd[1] &= ~0x1fUL;
> +	cmd->raw_cmd[1] |= size & 0x1f;
> +}
> +
> +static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
> +{
> +	cmd->raw_cmd[2] &= ~0xffffffffffffUL;
> +	cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
> +}
> +
> +static void its_encode_valid(struct its_cmd_block *cmd, int valid)
> +{
> +	cmd->raw_cmd[2] &= ~(1UL << 63);
> +	cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
> +}
> +
> +static void its_encode_target(struct its_cmd_block *cmd, u64 target_addr)
> +{
> +	cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
> +	cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
> +}
> +
> +static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
> +{
> +	cmd->raw_cmd[2] &= ~0xffffUL;
> +	cmd->raw_cmd[2] |= col;
> +}

The command encoding can be refactored like:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4d36f136d57aea6f6440886106e246bb7e5918d8

which will look much clearer.

[...]

> +static void its_send_single_command(its_cmd_builder_t builder,
> +				    struct its_cmd_desc *desc)
> +{
> +	struct its_cmd_block *cmd, *next_cmd;
> +
> +	cmd = its_allocate_entry();
> +	builder(cmd, desc);
> +	next_cmd = its_post_commands();
> +
> +	its_wait_for_range_completion(cmd, next_cmd);
> +}
> +
> +

extra line.

> +static void its_build_mapd_cmd(struct its_cmd_block *cmd,
> +			       struct its_cmd_desc *desc)
> +{
> +	unsigned long itt_addr;
> +	u8 size = 12; /* 4096 eventids */

Maybe use desc->its_mapd_cmd.dev->nr_ites instead as we already have it?

> +
> +	itt_addr = (unsigned long)(virt_to_phys(desc->its_mapd_cmd.dev->itt));
> +	itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
> +
> +	its_encode_cmd(cmd, GITS_CMD_MAPD);
> +	its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
> +	its_encode_size(cmd, size - 1);
> +	its_encode_itt(cmd, itt_addr);
> +	its_encode_valid(cmd, desc->its_mapd_cmd.valid);
> +	its_fixup_cmd(cmd);
> +	if (desc->verbose)
> +		printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
> +			desc->its_mapd_cmd.dev->device_id,
> +			size, itt_addr, desc->its_mapd_cmd.valid);
> +

extra line.

All of these are trivial things and feel free to ignore them,
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>


Thanks



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests
  2020-03-30  9:11       ` Andrew Jones
@ 2020-03-30  9:56         ` Auger Eric
  2020-03-30 10:19           ` Andrew Jones
  2020-03-30 12:20         ` Zenghui Yu
  1 sibling, 1 reply; 34+ messages in thread
From: Auger Eric @ 2020-03-30  9:56 UTC (permalink / raw)
  To: Andrew Jones
  Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, Zenghui Yu, alexandru.elisei, kvmarm,
	eric.auger.pro

Hi,

On 3/30/20 11:11 AM, Andrew Jones wrote:
> On Mon, Mar 30, 2020 at 10:46:57AM +0200, Auger Eric wrote:
>> Hi Zenghui,
>>
>> On 3/30/20 10:30 AM, Zenghui Yu wrote:
>>> Hi Eric,
>>>
>>> On 2020/3/20 17:24, Eric Auger wrote:
>>>> +static void its_cmd_queue_init(void)
>>>> +{
>>>> +    unsigned long order = get_order(SZ_64K >> PAGE_SHIFT);
>>>> +    u64 cbaser;
>>>> +
>>>> +    its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
>>>
>>> Shouldn't the cmd_base (and the cmd_write) be set as a GVA?
>> yes it should
> 
> If it's supposed to be a virtual address, when why do the virt_to_phys?
What is programmed in CBASER register is a physical address. So the
virt_to_phys() is relevant. The inconsistency is in its_allocate_entry()
introduced later on where I return the physical address instead of the
virtual address. I will fix that.


> 
>>>
>>> Otherwise I think we will end-up with memory corruption when writing
>>> the command queue.  But it seems that everything just works fine ...
>>> So I'm really confused here :-/
>> I was told by Paolo that the VA/PA memory map is flat in kvmunit test.
> 
> What does flat mean?

Yes I meant an identity map.

 kvm-unit-tests, at least arm/arm64, does prepare
> an identity map of all physical memory, which explains why the above
> is working.

should be the same on x86

 It's doing virt_to_phys(some-virt-addr), which gets a
> phys addr, but when the ITS uses it as a virt addr it works because
> we *also* have a virt addr == phys addr mapping in the default page
> table, which is named "idmap" for good reason.
> 
> I think it would be better to test with the non-identity mapped addresses
> though.

is there any way to exercise a non idmap?

Thanks

Eric
> 
> Thanks,
> drew
> 
>>
>>>
>>>> +
>>>> +    cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1)    |
>>>> GITS_CBASER_VALID);
>>>> +
>>>> +    writeq(cbaser, its_data.base + GITS_CBASER);
>>>> +
>>>> +    its_data.cmd_write = its_data.cmd_base;
>>>> +    writeq(0, its_data.base + GITS_CWRITER);
>>>> +}
>>>
>>> Otherwise this looks good,
>>> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
>> Thanks!
>>
>> Eric
>>>
>>>
>>> Thanks
>>>
>>
>>



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 09/13] arm/arm64: ITS: Commands
  2020-03-30  9:22   ` Zenghui Yu
@ 2020-03-30  9:57     ` Auger Eric
  0 siblings, 0 replies; 34+ messages in thread
From: Auger Eric @ 2020-03-30  9:57 UTC (permalink / raw)
  To: Zenghui Yu
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Zenghui,

On 3/30/20 11:22 AM, Zenghui Yu wrote:
> Hi Eric,
> 
> On 2020/3/20 17:24, Eric Auger wrote:
>> Implement main ITS commands. The code is largely inherited from
>> the ITS driver.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> 
> [...]
> 
>> +/* ITS COMMANDS */
>> +
>> +static void its_encode_cmd(struct its_cmd_block *cmd, u8 cmd_nr)
>> +{
>> +    cmd->raw_cmd[0] &= ~0xffUL;
>> +    cmd->raw_cmd[0] |= cmd_nr;
>> +}
>> +
>> +static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
>> +{
>> +    cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
>> +    cmd->raw_cmd[0] |= ((u64)devid) << 32;
>> +}
>> +
>> +static void its_encode_event_id(struct its_cmd_block *cmd, u32 id)
>> +{
>> +    cmd->raw_cmd[1] &= ~0xffffffffUL;
>> +    cmd->raw_cmd[1] |= id;
>> +}
>> +
>> +static void its_encode_phys_id(struct its_cmd_block *cmd, u32 phys_id)
>> +{
>> +    cmd->raw_cmd[1] &= 0xffffffffUL;
>> +    cmd->raw_cmd[1] |= ((u64)phys_id) << 32;
>> +}
>> +
>> +static void its_encode_size(struct its_cmd_block *cmd, u8 size)
>> +{
>> +    cmd->raw_cmd[1] &= ~0x1fUL;
>> +    cmd->raw_cmd[1] |= size & 0x1f;
>> +}
>> +
>> +static void its_encode_itt(struct its_cmd_block *cmd, u64 itt_addr)
>> +{
>> +    cmd->raw_cmd[2] &= ~0xffffffffffffUL;
>> +    cmd->raw_cmd[2] |= itt_addr & 0xffffffffff00UL;
>> +}
>> +
>> +static void its_encode_valid(struct its_cmd_block *cmd, int valid)
>> +{
>> +    cmd->raw_cmd[2] &= ~(1UL << 63);
>> +    cmd->raw_cmd[2] |= ((u64)!!valid) << 63;
>> +}
>> +
>> +static void its_encode_target(struct its_cmd_block *cmd, u64
>> target_addr)
>> +{
>> +    cmd->raw_cmd[2] &= ~(0xfffffffffUL << 16);
>> +    cmd->raw_cmd[2] |= (target_addr & (0xffffffffUL << 16));
>> +}
>> +
>> +static void its_encode_collection(struct its_cmd_block *cmd, u16 col)
>> +{
>> +    cmd->raw_cmd[2] &= ~0xffffUL;
>> +    cmd->raw_cmd[2] |= col;
>> +}
> 
> The command encoding can be refactored like:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4d36f136d57aea6f6440886106e246bb7e5918d8
> 
> 
> which will look much clearer.
OK
> 
> [...]
> 
>> +static void its_send_single_command(its_cmd_builder_t builder,
>> +                    struct its_cmd_desc *desc)
>> +{
>> +    struct its_cmd_block *cmd, *next_cmd;
>> +
>> +    cmd = its_allocate_entry();
>> +    builder(cmd, desc);
>> +    next_cmd = its_post_commands();
>> +
>> +    its_wait_for_range_completion(cmd, next_cmd);
>> +}
>> +
>> +
> 
> extra line.
OK
> 
>> +static void its_build_mapd_cmd(struct its_cmd_block *cmd,
>> +                   struct its_cmd_desc *desc)
>> +{
>> +    unsigned long itt_addr;
>> +    u8 size = 12; /* 4096 eventids */
> 
> Maybe use desc->its_mapd_cmd.dev->nr_ites instead as we already have it?
OK
> 
>> +
>> +    itt_addr = (unsigned
>> long)(virt_to_phys(desc->its_mapd_cmd.dev->itt));
>> +    itt_addr = ALIGN(itt_addr, ITS_ITT_ALIGN);
>> +
>> +    its_encode_cmd(cmd, GITS_CMD_MAPD);
>> +    its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id);
>> +    its_encode_size(cmd, size - 1);
>> +    its_encode_itt(cmd, itt_addr);
>> +    its_encode_valid(cmd, desc->its_mapd_cmd.valid);
>> +    its_fixup_cmd(cmd);
>> +    if (desc->verbose)
>> +        printf("ITS: MAPD devid=%d size = 0x%x itt=0x%lx valid=%d\n",
>> +            desc->its_mapd_cmd.dev->device_id,
>> +            size, itt_addr, desc->its_mapd_cmd.valid);
>> +
> 
> extra line.
> 
> All of these are trivial things and feel free to ignore them,
> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
thanks!

Eric
> 
> 
> Thanks
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests
  2020-03-30  9:56         ` Auger Eric
@ 2020-03-30 10:19           ` Andrew Jones
  2020-03-30 10:24             ` Auger Eric
  0 siblings, 1 reply; 34+ messages in thread
From: Andrew Jones @ 2020-03-30 10:19 UTC (permalink / raw)
  To: Auger Eric
  Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, Zenghui Yu, alexandru.elisei, kvmarm,
	eric.auger.pro

On Mon, Mar 30, 2020 at 11:56:00AM +0200, Auger Eric wrote:
> Hi,
> 
> On 3/30/20 11:11 AM, Andrew Jones wrote:
> > On Mon, Mar 30, 2020 at 10:46:57AM +0200, Auger Eric wrote:
> >> Hi Zenghui,
> >>
> >> On 3/30/20 10:30 AM, Zenghui Yu wrote:
> >>> Hi Eric,
> >>>
> >>> On 2020/3/20 17:24, Eric Auger wrote:
> >>>> +static void its_cmd_queue_init(void)
> >>>> +{
> >>>> +    unsigned long order = get_order(SZ_64K >> PAGE_SHIFT);
> >>>> +    u64 cbaser;
> >>>> +
> >>>> +    its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
> >>>
> >>> Shouldn't the cmd_base (and the cmd_write) be set as a GVA?
> >> yes it should
> > 
> > If it's supposed to be a virtual address, when why do the virt_to_phys?
> What is programmed in CBASER register is a physical address. So the
> virt_to_phys() is relevant. The inconsistency is in its_allocate_entry()
> introduced later on where I return the physical address instead of the
> virtual address. I will fix that.
> 
> 
> > 
> >>>
> >>> Otherwise I think we will end-up with memory corruption when writing
> >>> the command queue.  But it seems that everything just works fine ...
> >>> So I'm really confused here :-/
> >> I was told by Paolo that the VA/PA memory map is flat in kvmunit test.
> > 
> > What does flat mean?
> 
> Yes I meant an identity map.
> 
>  kvm-unit-tests, at least arm/arm64, does prepare
> > an identity map of all physical memory, which explains why the above
> > is working.
> 
> should be the same on x86

Maybe, but I didn't write or review how x86 does their default map, so I
don't know.

> 
>  It's doing virt_to_phys(some-virt-addr), which gets a
> > phys addr, but when the ITS uses it as a virt addr it works because
> > we *also* have a virt addr == phys addr mapping in the default page
> > table, which is named "idmap" for good reason.
> > 
> > I think it would be better to test with the non-identity mapped addresses
> > though.
> 
> is there any way to exercise a non idmap?

You could create your own map and then switch to it. See lib/arm/asm/mmu-api.h

But, you don't have to switch the whole map to use non-identity mapped
addresses. Just create new virt mappings to the phys addresses you're
interested in, and then use those new virt addresses instead. If you're
worried that somewhere an identity mapped virt address is getting used
because of a phys/virt address mix up, then you could probably sprinkle
some assert(virt_to_phys(addr) != addr)'s around to ensure it.

Thanks,
drew

> 
> Thanks
> 
> Eric
> > 
> > Thanks,
> > drew
> > 
> >>
> >>>
> >>>> +
> >>>> +    cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1)    |
> >>>> GITS_CBASER_VALID);
> >>>> +
> >>>> +    writeq(cbaser, its_data.base + GITS_CBASER);
> >>>> +
> >>>> +    its_data.cmd_write = its_data.cmd_base;
> >>>> +    writeq(0, its_data.base + GITS_CWRITER);
> >>>> +}
> >>>
> >>> Otherwise this looks good,
> >>> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
> >> Thanks!
> >>
> >> Eric
> >>>
> >>>
> >>> Thanks
> >>>
> >>
> >>
> 
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests
  2020-03-30 10:19           ` Andrew Jones
@ 2020-03-30 10:24             ` Auger Eric
  0 siblings, 0 replies; 34+ messages in thread
From: Auger Eric @ 2020-03-30 10:24 UTC (permalink / raw)
  To: Andrew Jones
  Cc: peter.maydell, thuth, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, Zenghui Yu, alexandru.elisei, kvmarm,
	eric.auger.pro

Hi Drew,

On 3/30/20 12:19 PM, Andrew Jones wrote:
> On Mon, Mar 30, 2020 at 11:56:00AM +0200, Auger Eric wrote:
>> Hi,
>>
>> On 3/30/20 11:11 AM, Andrew Jones wrote:
>>> On Mon, Mar 30, 2020 at 10:46:57AM +0200, Auger Eric wrote:
>>>> Hi Zenghui,
>>>>
>>>> On 3/30/20 10:30 AM, Zenghui Yu wrote:
>>>>> Hi Eric,
>>>>>
>>>>> On 2020/3/20 17:24, Eric Auger wrote:
>>>>>> +static void its_cmd_queue_init(void)
>>>>>> +{
>>>>>> +    unsigned long order = get_order(SZ_64K >> PAGE_SHIFT);
>>>>>> +    u64 cbaser;
>>>>>> +
>>>>>> +    its_data.cmd_base = (void *)virt_to_phys(alloc_pages(order));
>>>>>
>>>>> Shouldn't the cmd_base (and the cmd_write) be set as a GVA?
>>>> yes it should
>>>
>>> If it's supposed to be a virtual address, when why do the virt_to_phys?
>> What is programmed in CBASER register is a physical address. So the
>> virt_to_phys() is relevant. The inconsistency is in its_allocate_entry()
>> introduced later on where I return the physical address instead of the
>> virtual address. I will fix that.
>>
>>
>>>
>>>>>
>>>>> Otherwise I think we will end-up with memory corruption when writing
>>>>> the command queue.  But it seems that everything just works fine ...
>>>>> So I'm really confused here :-/
>>>> I was told by Paolo that the VA/PA memory map is flat in kvmunit test.
>>>
>>> What does flat mean?
>>
>> Yes I meant an identity map.
>>
>>  kvm-unit-tests, at least arm/arm64, does prepare
>>> an identity map of all physical memory, which explains why the above
>>> is working.
>>
>> should be the same on x86
> 
> Maybe, but I didn't write or review how x86 does their default map, so I
> don't know.
> 
>>
>>  It's doing virt_to_phys(some-virt-addr), which gets a
>>> phys addr, but when the ITS uses it as a virt addr it works because
>>> we *also* have a virt addr == phys addr mapping in the default page
>>> table, which is named "idmap" for good reason.
>>>
>>> I think it would be better to test with the non-identity mapped addresses
>>> though.
>>
>> is there any way to exercise a non idmap?
> 
> You could create your own map and then switch to it. See lib/arm/asm/mmu-api.h
> 
> But, you don't have to switch the whole map to use non-identity mapped
> addresses. Just create new virt mappings to the phys addresses you're
> interested in, and then use those new virt addresses instead. If you're
> worried that somewhere an identity mapped virt address is getting used
> because of a phys/virt address mix up, then you could probably sprinkle
> some assert(virt_to_phys(addr) != addr)'s around to ensure it.

OK. Well I don't know if it is worth the candle. I will review the code
again and fix the remaining inconsistencies I can see.

Thanks

Eric
> 
> Thanks,
> drew
> 
>>
>> Thanks
>>
>> Eric
>>>
>>> Thanks,
>>> drew
>>>
>>>>
>>>>>
>>>>>> +
>>>>>> +    cbaser = ((u64)its_data.cmd_base | (SZ_64K / SZ_4K - 1)    |
>>>>>> GITS_CBASER_VALID);
>>>>>> +
>>>>>> +    writeq(cbaser, its_data.base + GITS_CBASER);
>>>>>> +
>>>>>> +    its_data.cmd_write = its_data.cmd_base;
>>>>>> +    writeq(0, its_data.base + GITS_CWRITER);
>>>>>> +}
>>>>>
>>>>> Otherwise this looks good,
>>>>> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
>>>> Thanks!
>>>>
>>>> Eric
>>>>>
>>>>>
>>>>> Thanks
>>>>>
>>>>
>>>>
>>
>>



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 10/13] arm/arm64: ITS: INT functional tests
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 10/13] arm/arm64: ITS: INT functional tests Eric Auger
@ 2020-03-30 10:43   ` Zenghui Yu
  2020-04-02  8:50     ` Auger Eric
  0 siblings, 1 reply; 34+ messages in thread
From: Zenghui Yu @ 2020-03-30 10:43 UTC (permalink / raw)
  To: Eric Auger
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Eric,

On 2020/3/20 17:24, Eric Auger wrote:
> Triggers LPIs through the INT command.
> 
> the test checks the LPI hits the right CPU and triggers
> the right LPI intid, ie. the translation is correct.
> 
> Updates to the config table also are tested, along with inv
> and invall commands.
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>

[...]

So I've tested this series and found that the "INT" test will sometimes
fail.

"not ok 12 - gicv3: its-migration: dev2/eventid=20 triggers LPI 8195 en 
PE #3 after migration
not ok 13 - gicv3: its-migration: dev7/eventid=255 triggers LPI 8196 on 
PE #2 after migration"

 From logs:
"INFO: gicv3: its-migration: Migration complete
INT dev_id=2 event_id=20
INFO: gicv3: its-migration: No LPI received whereas (cpuid=3, 
intid=8195) was expected
FAIL: gicv3: its-migration: dev2/eventid=20 triggers LPI 8195 en PE #3 
after migration
INT dev_id=7 event_id=255
INFO: gicv3: its-migration: No LPI received whereas (cpuid=2, 
intid=8196) was expected
FAIL: gicv3: its-migration: dev7/eventid=255 triggers LPI 8196 on PE #2 
after migration"

> +static void check_lpi_stats(const char *msg)
> +{
> +	bool pass = false;
> +
> +	mdelay(100);

After changing this to 'mdelay(1000)', the above error doesn't show up
anymore. But it sounds strange that 100ms is not enough to deliver a
single LPI. I haven't dig it further but will get back here later.

> +	smp_rmb(); /* pairs with wmb in lpi_handler */
> +	if (lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id ||
> +	    lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id) {
> +		if (lpi_stats.observed.cpu_id == -1 &&
> +		    lpi_stats.observed.lpi_id == -1) {
> +			report_info("No LPI received whereas (cpuid=%d, intid=%d) "
> +				    "was expected", lpi_stats.expected.cpu_id,
> +				    lpi_stats.expected.lpi_id);
> +		} else {
> +			report_info("Unexpected LPI (cpuid=%d, intid=%d)",
> +				    lpi_stats.observed.cpu_id,
> +				    lpi_stats.observed.lpi_id);
> +		}
> +	} else {
> +		pass = true;
> +	}
> +	report(pass, "%s", msg);
> +}

This patch itself looks good to me,
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>


Thanks



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 12/13] arm/arm64: ITS: migration tests
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 12/13] arm/arm64: ITS: migration tests Eric Auger
@ 2020-03-30 10:55   ` Zenghui Yu
  0 siblings, 0 replies; 34+ messages in thread
From: Zenghui Yu @ 2020-03-30 10:55 UTC (permalink / raw)
  To: Eric Auger
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Eric,

On 2020/3/20 17:24, Eric Auger wrote:
> This test maps LPIs (populates the device table, the collection table,
> interrupt translation tables, configuration table), migrates and make
> sure the translation is correct on the destination.
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> 
> ---
> v4 -> v5:
> - move stub from header to arm/gic.c
> 
> v3 -> v4:
> - assert in its_get_device/collection if the id is not found
> ---
>   arm/gic.c                  | 58 ++++++++++++++++++++++++++++++++++----
>   arm/unittests.cfg          |  8 ++++++
>   lib/arm64/asm/gic-v3-its.h |  3 ++
>   lib/arm64/gic-v3-its.c     | 22 +++++++++++++++
>   4 files changed, 85 insertions(+), 6 deletions(-)
> 
> diff --git a/arm/gic.c b/arm/gic.c
> index 5f1e595..6ecfdbc 100644
> --- a/arm/gic.c
> +++ b/arm/gic.c
> @@ -590,6 +590,7 @@ static void gic_test_mmio(void)
>   
>   static void test_its_introspection(void) {}
>   static void test_its_trigger(void) {}
> +static void test_its_migration(void) {}
>   
>   #else /* __aarch64__ */
>   
> @@ -658,13 +659,19 @@ static int its_prerequisites(int nb_cpus)
>   	return 0;
>   }
>   
> -static void test_its_trigger(void)
> +/*
> + * Setup the configuration for those mappings:
> + * dev_id=2 event=20 -> vcpu 3, intid=8195
> + * dev_id=7 event=255 -> vcpu 2, intid=8196
> + * LPIs ready to hit
> + */
> +static int its_setup1(void)
>   {
>   	struct its_collection *col3, *col2;
>   	struct its_device *dev2, *dev7;
>   
>   	if (its_prerequisites(4))
> -		return;
> +		return -1;
>   
>   	dev2 = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>   	dev7 = its_create_device(7 /* dev id */, 8 /* nb_ites */);
> @@ -675,14 +682,10 @@ static void test_its_trigger(void)
>   	gicv3_lpi_set_config(8195, LPI_PROP_DEFAULT);
>   	gicv3_lpi_set_config(8196, LPI_PROP_DEFAULT);
>   
> -	report_prefix_push("int");
>   	/*
>   	 * dev=2, eventid=20  -> lpi= 8195, col=3
>   	 * dev=7, eventid=255 -> lpi= 8196, col=2
> -	 * Trigger dev2, eventid=20 and dev7, eventid=255
> -	 * Check both LPIs hit
>   	 */
> -
>   	its_send_mapd(dev2, true);
>   	its_send_mapd(dev7, true);
>   
> @@ -694,6 +697,22 @@ static void test_its_trigger(void)
>   
>   	its_send_mapti(dev2, 8195 /* lpi id */, 20 /* event id */, col3);
>   	its_send_mapti(dev7, 8196 /* lpi id */, 255 /* event id */, col2);
> +	return 0;
> +}
> +
> +static void test_its_trigger(void)
> +{
> +	struct its_collection *col3;
> +	struct its_device *dev2, *dev7;
> +
> +	if (its_setup1())
> +		return;
> +
> +	col3 = its_get_collection(3);
> +	dev2 = its_get_device(2);
> +	dev7 = its_get_device(7);
> +
> +	report_prefix_push("int");
>   
>   	lpi_stats_expect(3, 8195);
>   	its_send_int(dev2, 20);
> @@ -748,6 +767,29 @@ static void test_its_trigger(void)
>   	check_lpi_stats("no LPI after device unmap");
>   	report_prefix_pop();
>   }
> +
> +static void test_its_migration(void)
> +{
> +	struct its_device *dev2, *dev7;
> +
> +	if (its_setup1())
> +		return;
> +
> +	dev2 = its_get_device(2);
> +	dev7 = its_get_device(7);
> +
> +	puts("Now migrate the VM, then press a key to continue...\n");
> +	(void)getchar();
> +	report_info("Migration complete");
> +
> +	lpi_stats_expect(3, 8195);
> +	its_send_int(dev2, 20);
> +	check_lpi_stats("dev2/eventid=20 triggers LPI 8195 en PE #3 after migration");

typo: s/en PE #3/on PE #3/

Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>


Thanks

> +
> +	lpi_stats_expect(2, 8196);
> +	its_send_int(dev7, 255);
> +	check_lpi_stats("dev7/eventid=255 triggers LPI 8196 on PE #2 after migration");
> +}
>   #endif
>   
>   int main(int argc, char **argv)
> @@ -785,6 +827,10 @@ int main(int argc, char **argv)
>   		report_prefix_push(argv[1]);
>   		test_its_trigger();
>   		report_prefix_pop();
> +	} else if (!strcmp(argv[1], "its-migration")) {
> +		report_prefix_push(argv[1]);
> +		test_its_migration();
> +		report_prefix_pop();
>   	} else if (strcmp(argv[1], "its-introspection") == 0) {
>   		report_prefix_push(argv[1]);
>   		test_its_introspection();
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index b9a7a2c..480adec 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -136,6 +136,14 @@ extra_params = -machine gic-version=3 -append 'its-trigger'
>   groups = its
>   arch = arm64
>   
> +[its-migration]
> +file = gic.flat
> +smp = $MAX_SMP
> +accel = kvm
> +extra_params = -machine gic-version=3 -append 'its-migration'
> +groups = its migration
> +arch = arm64
> +
>   # Test PSCI emulation
>   [psci]
>   file = psci.flat
> diff --git a/lib/arm64/asm/gic-v3-its.h b/lib/arm64/asm/gic-v3-its.h
> index e1e42c3..412f438 100644
> --- a/lib/arm64/asm/gic-v3-its.h
> +++ b/lib/arm64/asm/gic-v3-its.h
> @@ -168,4 +168,7 @@ extern void __its_send_sync(struct its_collection *col, bool verbose);
>   #define its_send_movi_nv(dev, col, id)			__its_send_movi(dev, col, id, false)
>   #define its_send_sync_nv(col)				__its_send_sync(col, false)
>   
> +extern struct its_device *its_get_device(u32 id);
> +extern struct its_collection *its_get_collection(u32 id);
> +
>   #endif /* _ASMARM64_GIC_V3_ITS_H_ */
> diff --git a/lib/arm64/gic-v3-its.c b/lib/arm64/gic-v3-its.c
> index f0a0381..c7755d9 100644
> --- a/lib/arm64/gic-v3-its.c
> +++ b/lib/arm64/gic-v3-its.c
> @@ -147,3 +147,25 @@ struct its_collection *its_create_collection(u32 col_id, u32 pe)
>   	its_data.nr_collections++;
>   	return new;
>   }
> +
> +struct its_device *its_get_device(u32 id)
> +{
> +	int i;
> +
> +	for (i = 0; i < GITS_MAX_DEVICES; i++) {
> +		if (its_data.devices[i].device_id == id)
> +			return &its_data.devices[i];
> +	}
> +	assert(0);
> +}
> +
> +struct its_collection *its_get_collection(u32 id)
> +{
> +	int i;
> +
> +	for (i = 0; i < GITS_MAX_COLLECTIONS; i++) {
> +		if (its_data.collections[i].col_id == id)
> +			return &its_data.collections[i];
> +	}
> +	assert(0);
> +}
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 13/13] arm/arm64: ITS: pending table migration test
  2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 13/13] arm/arm64: ITS: pending table migration test Eric Auger
@ 2020-03-30 12:06   ` Zenghui Yu
  2020-03-30 12:38     ` Auger Eric
  0 siblings, 1 reply; 34+ messages in thread
From: Zenghui Yu @ 2020-03-30 12:06 UTC (permalink / raw)
  To: Eric Auger
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Eric,

On 2020/3/20 17:24, Eric Auger wrote:
> Add two new migration tests. One testing the migration of
> a topology where collection were unmapped. The second test
> checks the migration of the pending table.
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>

[...]

> @@ -659,6 +678,15 @@ static int its_prerequisites(int nb_cpus)
>   	return 0;
>   }
>   
> +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
> +		    struct its_collection *col)
> +{
> +	assert(dev && col);
> +
> +	its_send_mapti(dev, physid, eventid, col);
> +	gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
> +}

I'd say we can just drop this helper and open-code it anywhere
necessarily. The name 'set_lpi' doesn't tell me too much about
what has been implemented inside the helper.

> +
>   /*
>    * Setup the configuration for those mappings:
>    * dev_id=2 event=20 -> vcpu 3, intid=8195
> @@ -790,6 +818,108 @@ static void test_its_migration(void)
>   	its_send_int(dev7, 255);
>   	check_lpi_stats("dev7/eventid=255 triggers LPI 8196 on PE #2 after migration");
>   }
> +
> +static void test_migrate_unmapped_collection(void)
> +{
> +	struct its_collection *col;
> +	struct its_device *dev2, *dev7;
> +	int pe0 = 0;
> +	u8 config;
> +
> +	if (its_setup1())
> +		return;
> +
> +	col = its_create_collection(pe0, pe0);
> +	dev2 = its_get_device(2);
> +	dev7 = its_get_device(7);
> +
> +	/* MAPTI with the collection unmapped */
> +	set_lpi(dev2, 0, 8192, col);

... and it's only invoked here.

> +
> +	puts("Now migrate the VM, then press a key to continue...\n");
> +	(void)getchar();
> +	report_info("Migration complete");
> +
> +	/* on the destination, map the collection */
> +	its_send_mapc(col, true);
> +	its_send_invall(col);
> +
> +	lpi_stats_expect(2, 8196);
> +	its_send_int(dev7, 255);
> +	check_lpi_stats("dev7/eventid= 255 triggered LPI 8196 on PE #2");
> +
> +	config = gicv3_lpi_get_config(8192);
> +	report(config == LPI_PROP_DEFAULT,
> +	       "Config of LPI 8192 was properly migrated");
> +
> +	lpi_stats_expect(pe0, 8192);
> +	its_send_int(dev2, 0);
> +	check_lpi_stats("dev2/eventid = 0 triggered LPI 8192 on PE0");
> +}
> +
> +static void test_its_pending_migration(void)
> +{
> +	struct its_device *dev;
> +	struct its_collection *collection[2];
> +	int *expected = malloc(nr_cpus * sizeof(int));
> +	int pe0 = nr_cpus - 1, pe1 = nr_cpus - 2;
> +	u64 pendbaser;
> +	void *ptr;
> +	int i;
> +
> +	if (its_prerequisites(4))
> +		return;
> +
> +	dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
> +	its_send_mapd(dev, true);
> +
> +	collection[0] = its_create_collection(pe0, pe0);
> +	collection[1] = its_create_collection(pe1, pe1);
> +	its_send_mapc(collection[0], true);
> +	its_send_mapc(collection[1], true);
> +
> +	/* disable lpi at redist level */
> +	gicv3_lpi_rdist_disable(pe0);
> +	gicv3_lpi_rdist_disable(pe1);
> +
> +	/* lpis are interleaved inbetween the 2 PEs */
> +	for (i = 0; i < 256; i++) {
> +		struct its_collection *col = i % 2 ? collection[0] :
> +						     collection[1];
> +		int vcpu = col->target_address >> 16;
> +
> +		its_send_mapti(dev, LPI(i), i, col);
> +		gicv3_lpi_set_config(LPI(i), LPI_PROP_DEFAULT);
> +		gicv3_lpi_set_clr_pending(vcpu, LPI(i), true);
> +	}
> +	its_send_invall(collection[0]);
> +	its_send_invall(collection[1]);
> +
> +	/* Clear the PTZ bit on each pendbaser */
> +
> +	expected[pe0] = 128;
> +	expected[pe1] = 128;

Do we need to initialize expected[] for other PEs? Or it has always
been zeroed by the kvm-unit-tests implementation of malloc()?

> +
> +	ptr = gicv3_data.redist_base[pe0] + GICR_PENDBASER;
> +	pendbaser = readq(ptr);
> +	writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> +
> +	ptr = gicv3_data.redist_base[pe1] + GICR_PENDBASER;
> +	pendbaser = readq(ptr);
> +	writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
> +
> +	gicv3_lpi_rdist_enable(pe0);
> +	gicv3_lpi_rdist_enable(pe1);

I don't know how the migration gets implemented in kvm-unit-tests.
But is there any guarantee that the LPIs will only be triggered on the
destination side? As once the EnableLPIs bit becomes 1, VGIC will start
reading the pending bit in guest memory and potentially injecting LPIs
into the target vcpu (in the source side).

> +
> +	puts("Now migrate the VM, then press a key to continue...\n");
> +	(void)getchar();
> +	report_info("Migration complete");
> +
> +	/* let's wait for the 256 LPIs to be handled */
> +	mdelay(1000);
> +
> +	check_lpi_hits(expected, "128 LPIs on both PE0 and PE1 after migration");
> +}


Thanks,
Zenghui



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests
  2020-03-30  9:11       ` Andrew Jones
  2020-03-30  9:56         ` Auger Eric
@ 2020-03-30 12:20         ` Zenghui Yu
  1 sibling, 0 replies; 34+ messages in thread
From: Zenghui Yu @ 2020-03-30 12:20 UTC (permalink / raw)
  To: Andrew Jones, Auger Eric
  Cc: peter.maydell, thuth, kvm, andre.przywara, qemu-devel, qemu-arm,
	maz, alexandru.elisei, kvmarm, eric.auger.pro

Hi Drew,

On 2020/3/30 17:11, Andrew Jones wrote:
> On Mon, Mar 30, 2020 at 10:46:57AM +0200, Auger Eric wrote:
>> Hi Zenghui,
>>
>> On 3/30/20 10:30 AM, Zenghui Yu wrote:

[...]

>>> Otherwise I think we will end-up with memory corruption when writing
>>> the command queue.  But it seems that everything just works fine ...
>>> So I'm really confused here :-/
>> I was told by Paolo that the VA/PA memory map is flat in kvmunit test.
> 
> What does flat mean? kvm-unit-tests, at least arm/arm64, does prepare
> an identity map of all physical memory, which explains why the above
> is working. It's doing virt_to_phys(some-virt-addr), which gets a
> phys addr, but when the ITS uses it as a virt addr it works because
> we *also* have a virt addr == phys addr mapping in the default page
> table, which is named "idmap" for good reason.

Thanks for the explanation :-). I will have a look at the arm/arm64 mm
code to learn it.


Thanks,
Zenghui



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 13/13] arm/arm64: ITS: pending table migration test
  2020-03-30 12:06   ` Zenghui Yu
@ 2020-03-30 12:38     ` Auger Eric
  2020-03-30 13:17       ` Zenghui Yu
  0 siblings, 1 reply; 34+ messages in thread
From: Auger Eric @ 2020-03-30 12:38 UTC (permalink / raw)
  To: Zenghui Yu
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Zenghui,

On 3/30/20 2:06 PM, Zenghui Yu wrote:
> Hi Eric,
> 
> On 2020/3/20 17:24, Eric Auger wrote:
>> Add two new migration tests. One testing the migration of
>> a topology where collection were unmapped. The second test
>> checks the migration of the pending table.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> 
> [...]
> 
>> @@ -659,6 +678,15 @@ static int its_prerequisites(int nb_cpus)
>>       return 0;
>>   }
>>   +static void set_lpi(struct its_device *dev, u32 eventid, u32 physid,
>> +            struct its_collection *col)
>> +{
>> +    assert(dev && col);
>> +
>> +    its_send_mapti(dev, physid, eventid, col);
>> +    gicv3_lpi_set_config(physid, LPI_PROP_DEFAULT);
>> +}
> 
> I'd say we can just drop this helper and open-code it anywhere
> necessarily. The name 'set_lpi' doesn't tell me too much about
> what has been implemented inside the helper.
> 
>> +
>>   /*
>>    * Setup the configuration for those mappings:
>>    * dev_id=2 event=20 -> vcpu 3, intid=8195
>> @@ -790,6 +818,108 @@ static void test_its_migration(void)
>>       its_send_int(dev7, 255);
>>       check_lpi_stats("dev7/eventid=255 triggers LPI 8196 on PE #2
>> after migration");
>>   }
>> +
>> +static void test_migrate_unmapped_collection(void)
>> +{
>> +    struct its_collection *col;
>> +    struct its_device *dev2, *dev7;
>> +    int pe0 = 0;
>> +    u8 config;
>> +
>> +    if (its_setup1())
>> +        return;
>> +
>> +    col = its_create_collection(pe0, pe0);
>> +    dev2 = its_get_device(2);
>> +    dev7 = its_get_device(7);
>> +
>> +    /* MAPTI with the collection unmapped */
>> +    set_lpi(dev2, 0, 8192, col);
> 
> ... and it's only invoked here.
> 
>> +
>> +    puts("Now migrate the VM, then press a key to continue...\n");
>> +    (void)getchar();
>> +    report_info("Migration complete");
>> +
>> +    /* on the destination, map the collection */
>> +    its_send_mapc(col, true);
>> +    its_send_invall(col);
>> +
>> +    lpi_stats_expect(2, 8196);
>> +    its_send_int(dev7, 255);
>> +    check_lpi_stats("dev7/eventid= 255 triggered LPI 8196 on PE #2");
>> +
>> +    config = gicv3_lpi_get_config(8192);
>> +    report(config == LPI_PROP_DEFAULT,
>> +           "Config of LPI 8192 was properly migrated");
>> +
>> +    lpi_stats_expect(pe0, 8192);
>> +    its_send_int(dev2, 0);
>> +    check_lpi_stats("dev2/eventid = 0 triggered LPI 8192 on PE0");
>> +}
>> +
>> +static void test_its_pending_migration(void)
>> +{
>> +    struct its_device *dev;
>> +    struct its_collection *collection[2];
>> +    int *expected = malloc(nr_cpus * sizeof(int));
>> +    int pe0 = nr_cpus - 1, pe1 = nr_cpus - 2;
>> +    u64 pendbaser;
>> +    void *ptr;
>> +    int i;
>> +
>> +    if (its_prerequisites(4))
>> +        return;
>> +
>> +    dev = its_create_device(2 /* dev id */, 8 /* nb_ites */);
>> +    its_send_mapd(dev, true);
>> +
>> +    collection[0] = its_create_collection(pe0, pe0);
>> +    collection[1] = its_create_collection(pe1, pe1);
>> +    its_send_mapc(collection[0], true);
>> +    its_send_mapc(collection[1], true);
>> +
>> +    /* disable lpi at redist level */
>> +    gicv3_lpi_rdist_disable(pe0);
>> +    gicv3_lpi_rdist_disable(pe1);
>> +
>> +    /* lpis are interleaved inbetween the 2 PEs */
>> +    for (i = 0; i < 256; i++) {
>> +        struct its_collection *col = i % 2 ? collection[0] :
>> +                             collection[1];
>> +        int vcpu = col->target_address >> 16;
>> +
>> +        its_send_mapti(dev, LPI(i), i, col);
>> +        gicv3_lpi_set_config(LPI(i), LPI_PROP_DEFAULT);
>> +        gicv3_lpi_set_clr_pending(vcpu, LPI(i), true);
>> +    }
>> +    its_send_invall(collection[0]);
>> +    its_send_invall(collection[1]);
>> +
>> +    /* Clear the PTZ bit on each pendbaser */
>> +
>> +    expected[pe0] = 128;
>> +    expected[pe1] = 128;
> 
> Do we need to initialize expected[] for other PEs? Or it has always
> been zeroed by the kvm-unit-tests implementation of malloc()?
> 
>> +
>> +    ptr = gicv3_data.redist_base[pe0] + GICR_PENDBASER;
>> +    pendbaser = readq(ptr);
>> +    writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>> +
>> +    ptr = gicv3_data.redist_base[pe1] + GICR_PENDBASER;
>> +    pendbaser = readq(ptr);
>> +    writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>> +
>> +    gicv3_lpi_rdist_enable(pe0);
>> +    gicv3_lpi_rdist_enable(pe1);
> 
> I don't know how the migration gets implemented in kvm-unit-tests.
> But is there any guarantee that the LPIs will only be triggered on the
> destination side? As once the EnableLPIs bit becomes 1, VGIC will start
> reading the pending bit in guest memory and potentially injecting LPIs
> into the target vcpu (in the source side).

I expect some LPIs to hit on source and some others to hit on the
destination. To me, this does not really matter as long as the handlers
gets called and accumulate the stats. Given the number of LPIs, we will
at least test the migration of some of the pending bits and especially
adjacent ones. It does work as it allows to test your fix:

ca185b260951  KVM: arm/arm64: vgic: Don't rely on the wrong pending table

Thanks

Eric
> 
>> +
>> +    puts("Now migrate the VM, then press a key to continue...\n");
>> +    (void)getchar();
>> +    report_info("Migration complete");
>> +
>> +    /* let's wait for the 256 LPIs to be handled */
>> +    mdelay(1000);
>> +
>> +    check_lpi_hits(expected, "128 LPIs on both PE0 and PE1 after
>> migration");
>> +}
> 
> 
> Thanks,
> Zenghui
> 
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 13/13] arm/arm64: ITS: pending table migration test
  2020-03-30 12:38     ` Auger Eric
@ 2020-03-30 13:17       ` Zenghui Yu
  0 siblings, 0 replies; 34+ messages in thread
From: Zenghui Yu @ 2020-03-30 13:17 UTC (permalink / raw)
  To: Auger Eric
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

On 2020/3/30 20:38, Auger Eric wrote:
> Hi Zenghui,

[...]

>>> +
>>> +    ptr = gicv3_data.redist_base[pe0] + GICR_PENDBASER;
>>> +    pendbaser = readq(ptr);
>>> +    writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>>> +
>>> +    ptr = gicv3_data.redist_base[pe1] + GICR_PENDBASER;
>>> +    pendbaser = readq(ptr);
>>> +    writeq(pendbaser & ~GICR_PENDBASER_PTZ, ptr);
>>> +
>>> +    gicv3_lpi_rdist_enable(pe0);
>>> +    gicv3_lpi_rdist_enable(pe1);
>>
>> I don't know how the migration gets implemented in kvm-unit-tests.
>> But is there any guarantee that the LPIs will only be triggered on the
>> destination side? As once the EnableLPIs bit becomes 1, VGIC will start
>> reading the pending bit in guest memory and potentially injecting LPIs
>> into the target vcpu (in the source side).
> 
> I expect some LPIs to hit on source and some others to hit on the
> destination. To me, this does not really matter as long as the handlers
> gets called and accumulate the stats. Given the number of LPIs, we will
> at least test the migration of some of the pending bits and especially
> adjacent ones. It does work as it allows to test your fix:
> 
> ca185b260951  KVM: arm/arm64: vgic: Don't rely on the wrong pending table

Fair enough. Thanks for your explanation!


Zenghui



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 10/13] arm/arm64: ITS: INT functional tests
  2020-03-30 10:43   ` Zenghui Yu
@ 2020-04-02  8:50     ` Auger Eric
  2020-04-02 12:40       ` Zenghui Yu
  0 siblings, 1 reply; 34+ messages in thread
From: Auger Eric @ 2020-04-02  8:50 UTC (permalink / raw)
  To: Zenghui Yu
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Zenghui,

On 3/30/20 12:43 PM, Zenghui Yu wrote:
> Hi Eric,
> 
> On 2020/3/20 17:24, Eric Auger wrote:
>> Triggers LPIs through the INT command.
>>
>> the test checks the LPI hits the right CPU and triggers
>> the right LPI intid, ie. the translation is correct.
>>
>> Updates to the config table also are tested, along with inv
>> and invall commands.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> 
> [...]
> 
> So I've tested this series and found that the "INT" test will sometimes
> fail.
> 
> "not ok 12 - gicv3: its-migration: dev2/eventid=20 triggers LPI 8195 en
> PE #3 after migration
> not ok 13 - gicv3: its-migration: dev7/eventid=255 triggers LPI 8196 on
> PE #2 after migration"
> 
> From logs:
> "INFO: gicv3: its-migration: Migration complete
> INT dev_id=2 event_id=20
> INFO: gicv3: its-migration: No LPI received whereas (cpuid=3,
> intid=8195) was expected
> FAIL: gicv3: its-migration: dev2/eventid=20 triggers LPI 8195 en PE #3
> after migration
> INT dev_id=7 event_id=255
> INFO: gicv3: its-migration: No LPI received whereas (cpuid=2,
> intid=8196) was expected
> FAIL: gicv3: its-migration: dev7/eventid=255 triggers LPI 8196 on PE #2
> after migration"
> 
>> +static void check_lpi_stats(const char *msg)
>> +{
>> +    bool pass = false;
>> +
>> +    mdelay(100);
> 
> After changing this to 'mdelay(1000)', the above error doesn't show up
> anymore. But it sounds strange that 100ms is not enough to deliver a
> single LPI. I haven't dig it further but will get back here later.

Did you find some time to investigate this issue. Changing 100 to 1000
has a huge impact on the overall test duration and I don't think it is
sensible. Could you see what is your minimal value that pass the tests?

Thanks

Eric
> 
>> +    smp_rmb(); /* pairs with wmb in lpi_handler */
>> +    if (lpi_stats.observed.cpu_id != lpi_stats.expected.cpu_id ||
>> +        lpi_stats.observed.lpi_id != lpi_stats.expected.lpi_id) {
>> +        if (lpi_stats.observed.cpu_id == -1 &&
>> +            lpi_stats.observed.lpi_id == -1) {
>> +            report_info("No LPI received whereas (cpuid=%d, intid=%d) "
>> +                    "was expected", lpi_stats.expected.cpu_id,
>> +                    lpi_stats.expected.lpi_id);
>> +        } else {
>> +            report_info("Unexpected LPI (cpuid=%d, intid=%d)",
>> +                    lpi_stats.observed.cpu_id,
>> +                    lpi_stats.observed.lpi_id);
>> +        }
>> +    } else {
>> +        pass = true;
>> +    }
>> +    report(pass, "%s", msg);
>> +}
> 
> This patch itself looks good to me,
> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
> 
> 
> Thanks
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 10/13] arm/arm64: ITS: INT functional tests
  2020-04-02  8:50     ` Auger Eric
@ 2020-04-02 12:40       ` Zenghui Yu
  2020-04-02 14:41         ` Andrew Jones
  0 siblings, 1 reply; 34+ messages in thread
From: Zenghui Yu @ 2020-04-02 12:40 UTC (permalink / raw)
  To: Auger Eric
  Cc: peter.maydell, drjones, kvm, maz, qemu-devel, qemu-arm,
	andre.przywara, thuth, alexandru.elisei, kvmarm, eric.auger.pro

Hi Eric,

On 2020/4/2 16:50, Auger Eric wrote:
> Hi Zenghui,
> 
> On 3/30/20 12:43 PM, Zenghui Yu wrote:
>> Hi Eric,
>>
>> On 2020/3/20 17:24, Eric Auger wrote:
>>> Triggers LPIs through the INT command.
>>>
>>> the test checks the LPI hits the right CPU and triggers
>>> the right LPI intid, ie. the translation is correct.
>>>
>>> Updates to the config table also are tested, along with inv
>>> and invall commands.
>>>
>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> [...]
>>
>> So I've tested this series and found that the "INT" test will sometimes
>> fail.
>>
>> "not ok 12 - gicv3: its-migration: dev2/eventid=20 triggers LPI 8195 en
>> PE #3 after migration
>> not ok 13 - gicv3: its-migration: dev7/eventid=255 triggers LPI 8196 on
>> PE #2 after migration"
>>
>>  From logs:
>> "INFO: gicv3: its-migration: Migration complete
>> INT dev_id=2 event_id=20
>> INFO: gicv3: its-migration: No LPI received whereas (cpuid=3,
>> intid=8195) was expected
>> FAIL: gicv3: its-migration: dev2/eventid=20 triggers LPI 8195 en PE #3
>> after migration
>> INT dev_id=7 event_id=255
>> INFO: gicv3: its-migration: No LPI received whereas (cpuid=2,
>> intid=8196) was expected
>> FAIL: gicv3: its-migration: dev7/eventid=255 triggers LPI 8196 on PE #2
>> after migration"
>>
>>> +static void check_lpi_stats(const char *msg)
>>> +{
>>> +    bool pass = false;
>>> +
>>> +    mdelay(100);
>>
>> After changing this to 'mdelay(1000)', the above error doesn't show up
>> anymore. But it sounds strange that 100ms is not enough to deliver a
>> single LPI. I haven't dig it further but will get back here later.
> 
> Did you find some time to investigate this issue. Changing 100 to 1000
> has a huge impact on the overall test duration and I don't think it is
> sensible. Could you see what is your minimal value that pass the tests?

I can reproduce this issue with a very *low* probability so I failed
to investigate it :-(.  (It might because the LPI was delivered to a
busy vcpu...)

You can leave it as it is until someone else complain about it again.
Or take the similar approach as check_acked() - wait up to 5s for the
interrupt to be delivered, and bail out as soon as we see it.


Thanks,
Zenghui



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [kvm-unit-tests PATCH v7 10/13] arm/arm64: ITS: INT functional tests
  2020-04-02 12:40       ` Zenghui Yu
@ 2020-04-02 14:41         ` Andrew Jones
  0 siblings, 0 replies; 34+ messages in thread
From: Andrew Jones @ 2020-04-02 14:41 UTC (permalink / raw)
  To: Zenghui Yu
  Cc: peter.maydell, thuth, kvm, maz, qemu-devel, Auger Eric, qemu-arm,
	andre.przywara, alexandru.elisei, kvmarm, eric.auger.pro

On Thu, Apr 02, 2020 at 08:40:42PM +0800, Zenghui Yu wrote:
> Hi Eric,
> 
> On 2020/4/2 16:50, Auger Eric wrote:
> > Hi Zenghui,
> > 
> > On 3/30/20 12:43 PM, Zenghui Yu wrote:
> > > Hi Eric,
> > > 
> > > On 2020/3/20 17:24, Eric Auger wrote:
> > > > Triggers LPIs through the INT command.
> > > > 
> > > > the test checks the LPI hits the right CPU and triggers
> > > > the right LPI intid, ie. the translation is correct.
> > > > 
> > > > Updates to the config table also are tested, along with inv
> > > > and invall commands.
> > > > 
> > > > Signed-off-by: Eric Auger <eric.auger@redhat.com>
> > > 
> > > [...]
> > > 
> > > So I've tested this series and found that the "INT" test will sometimes
> > > fail.
> > > 
> > > "not ok 12 - gicv3: its-migration: dev2/eventid=20 triggers LPI 8195 en
> > > PE #3 after migration
> > > not ok 13 - gicv3: its-migration: dev7/eventid=255 triggers LPI 8196 on
> > > PE #2 after migration"
> > > 
> > >  From logs:
> > > "INFO: gicv3: its-migration: Migration complete
> > > INT dev_id=2 event_id=20
> > > INFO: gicv3: its-migration: No LPI received whereas (cpuid=3,
> > > intid=8195) was expected
> > > FAIL: gicv3: its-migration: dev2/eventid=20 triggers LPI 8195 en PE #3
> > > after migration
> > > INT dev_id=7 event_id=255
> > > INFO: gicv3: its-migration: No LPI received whereas (cpuid=2,
> > > intid=8196) was expected
> > > FAIL: gicv3: its-migration: dev7/eventid=255 triggers LPI 8196 on PE #2
> > > after migration"
> > > 
> > > > +static void check_lpi_stats(const char *msg)
> > > > +{
> > > > +    bool pass = false;
> > > > +
> > > > +    mdelay(100);
> > > 
> > > After changing this to 'mdelay(1000)', the above error doesn't show up
> > > anymore. But it sounds strange that 100ms is not enough to deliver a
> > > single LPI. I haven't dig it further but will get back here later.
> > 
> > Did you find some time to investigate this issue. Changing 100 to 1000
> > has a huge impact on the overall test duration and I don't think it is
> > sensible. Could you see what is your minimal value that pass the tests?
> 
> I can reproduce this issue with a very *low* probability so I failed
> to investigate it :-(.  (It might because the LPI was delivered to a
> busy vcpu...)
> 
> You can leave it as it is until someone else complain about it again.
> Or take the similar approach as check_acked() - wait up to 5s for the
> interrupt to be delivered, and bail out as soon as we see it.

I think the check_acked approach would be the best approach.

Thanks,
drew

> 
> 
> Thanks,
> Zenghui
> 
> 



^ permalink raw reply	[flat|nested] 34+ messages in thread

end of thread, back to index

Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-20  9:24 [kvm-unit-tests PATCH v7 00/13] arm/arm64: Add ITS tests Eric Auger
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 01/13] libcflat: Add other size defines Eric Auger
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 02/13] page_alloc: Introduce get_order() Eric Auger
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 03/13] arm/arm64: gic: Introduce setup_irq() helper Eric Auger
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 04/13] arm/arm64: gicv3: Add some re-distributor defines Eric Auger
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 05/13] arm/arm64: gicv3: Set the LPI config and pending tables Eric Auger
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 06/13] arm/arm64: ITS: Introspection tests Eric Auger
2020-03-30  8:30   ` Zenghui Yu
2020-03-30  8:46     ` Auger Eric
2020-03-30  9:11       ` Andrew Jones
2020-03-30  9:56         ` Auger Eric
2020-03-30 10:19           ` Andrew Jones
2020-03-30 10:24             ` Auger Eric
2020-03-30 12:20         ` Zenghui Yu
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 07/13] arm/arm64: ITS: its_enable_defaults Eric Auger
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 08/13] arm/arm64: ITS: Device and collection Initialization Eric Auger
2020-03-25  8:10   ` Zenghui Yu
2020-03-25 21:20     ` Auger Eric
2020-03-30  9:13       ` Andrew Jones
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 09/13] arm/arm64: ITS: Commands Eric Auger
2020-03-30  9:22   ` Zenghui Yu
2020-03-30  9:57     ` Auger Eric
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 10/13] arm/arm64: ITS: INT functional tests Eric Auger
2020-03-30 10:43   ` Zenghui Yu
2020-04-02  8:50     ` Auger Eric
2020-04-02 12:40       ` Zenghui Yu
2020-04-02 14:41         ` Andrew Jones
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 11/13] arm/run: Allow Migration tests Eric Auger
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 12/13] arm/arm64: ITS: migration tests Eric Auger
2020-03-30 10:55   ` Zenghui Yu
2020-03-20  9:24 ` [kvm-unit-tests PATCH v7 13/13] arm/arm64: ITS: pending table migration test Eric Auger
2020-03-30 12:06   ` Zenghui Yu
2020-03-30 12:38     ` Auger Eric
2020-03-30 13:17       ` Zenghui Yu

QEMU-Devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/qemu-devel/0 qemu-devel/git/0.git
	git clone --mirror https://lore.kernel.org/qemu-devel/1 qemu-devel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 qemu-devel qemu-devel/ https://lore.kernel.org/qemu-devel \
		qemu-devel@nongnu.org
	public-inbox-index qemu-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.nongnu.qemu-devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git