kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V
@ 2024-01-26 14:23 Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 01/24] configure: Add ARCH_LIBDIR Andrew Jones
                   ` (24 more replies)
  0 siblings, 25 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

v2:
 - While basing [1] on this series I found two bugs (one in exception
   return and another in isa string parsing). I also decided to expose
   a get_pte() function to unit tests and also an isa-extension-by-name
   function to check for arbitrary extensions. Finally, I picked up
   Thomas' gitlab-ci suggestion and his tags.

(Having [1] and the selftests running makes me pretty happy with the
series, so, unless somebody shouts, I'll merge this sometime next week.)

[1] https://gitlab.com/jones-drew/kvm-unit-tests/-/commit/e9c6c58b1c799de77fd39970b358a7592ecd048f

Thanks,
drew

Original cover letter follows:

This series adds another architecture to kvm-unit-tests (RISC-V, both
32-bit and 64-bit). Much of the code is borrowed from arm/arm64 by
mimicking its patterns or by first making the arm code more generic
and moving it to the common lib.

This series brings UART, SMP, MMU, and exception handling support.
One should be able to start writing CPU validation tests in a mix
of C and asm as well as write SBI tests, as is the plan for the SBI
verification framework. kvm-unit-tests provides backtraces on asserts
and input can be given to the tests through command line arguments,
environment variables, and the DT (there's already an ISA string
parser for extension detection).

This series only targets QEMU TCG and KVM, but OpenSBI may be replaced
with other SBI implementations, such as RustSBI. It's a goal to target
bare-metal as soon as possible, so EFI support is already in progress
and will be posted soon. More follow on series will come as well,
bringing interrupt controller support for timer and PMU testing,
support to run tests in usermode, and whatever else people need for
their tests.


Andrew Jones (24):
  configure: Add ARCH_LIBDIR
  riscv: Initial port, hello world
  arm/arm64: Move cpumask.h to common lib
  arm/arm64: Share cpu online, present and idle masks
  riscv: Add DT parsing
  riscv: Add initial SBI support
  riscv: Add run script and unittests.cfg
  riscv: Add riscv32 support
  riscv: Add exception handling
  riscv: Add backtrace support
  arm/arm64: Generalize wfe/sev names in smp.c
  arm/arm64: Remove spinlocks from on_cpu_async
  arm/arm64: Share on_cpus
  riscv: Compile with march
  riscv: Add SMP support
  arm/arm64: Share memregions
  riscv: Populate memregions and switch to page allocator
  riscv: Add MMU support
  riscv: Enable the MMU in secondaries
  riscv: Enable vmalloc
  lib: Add strcasecmp and strncasecmp
  riscv: Add isa string parsing
  gitlab-ci: Add riscv64 tests
  MAINTAINERS: Add riscv

 .gitlab-ci.yml               |  17 +++
 MAINTAINERS                  |   8 ++
 Makefile                     |   2 +-
 arm/Makefile.common          |   2 +
 arm/selftest.c               |   3 +-
 configure                    |  16 +++
 lib/arm/asm/gic-v2.h         |   2 +-
 lib/arm/asm/gic-v3.h         |   2 +-
 lib/arm/asm/gic.h            |   2 +-
 lib/arm/asm/setup.h          |  14 --
 lib/arm/asm/smp.h            |  45 +-----
 lib/arm/mmu.c                |   3 +-
 lib/arm/setup.c              |  93 +++----------
 lib/arm/smp.c                | 135 +-----------------
 lib/arm64/asm/cpumask.h      |   1 -
 lib/{arm/asm => }/cpumask.h  |  42 +++++-
 lib/ctype.h                  |  10 ++
 lib/elf.h                    |  11 ++
 lib/ldiv32.c                 |  16 +++
 lib/linux/const.h            |   2 +
 lib/memregions.c             |  82 +++++++++++
 lib/memregions.h             |  29 ++++
 lib/on-cpus.c                | 154 +++++++++++++++++++++
 lib/on-cpus.h                |  14 ++
 lib/riscv/.gitignore         |   1 +
 lib/riscv/asm-offsets.c      |  62 +++++++++
 lib/riscv/asm/asm-offsets.h  |   1 +
 lib/riscv/asm/barrier.h      |  20 +++
 lib/riscv/asm/bitops.h       |  21 +++
 lib/riscv/asm/bug.h          |  20 +++
 lib/riscv/asm/csr.h          | 100 ++++++++++++++
 lib/riscv/asm/io.h           |  87 ++++++++++++
 lib/riscv/asm/isa.h          |  33 +++++
 lib/riscv/asm/memory_areas.h |   1 +
 lib/riscv/asm/mmu.h          |  32 +++++
 lib/riscv/asm/page.h         |  21 +++
 lib/riscv/asm/pgtable.h      |  42 ++++++
 lib/riscv/asm/processor.h    |  29 ++++
 lib/riscv/asm/ptrace.h       |  46 +++++++
 lib/riscv/asm/sbi.h          |  54 ++++++++
 lib/riscv/asm/setup.h        |  15 ++
 lib/riscv/asm/smp.h          |  29 ++++
 lib/riscv/asm/spinlock.h     |   7 +
 lib/riscv/asm/stack.h        |  12 ++
 lib/riscv/bitops.c           |  47 +++++++
 lib/riscv/io.c               |  97 +++++++++++++
 lib/riscv/isa.c              | 126 +++++++++++++++++
 lib/riscv/mmu.c              | 205 +++++++++++++++++++++++++++
 lib/riscv/processor.c        |  64 +++++++++
 lib/riscv/sbi.c              |  40 ++++++
 lib/riscv/setup.c            | 188 +++++++++++++++++++++++++
 lib/riscv/smp.c              |  70 ++++++++++
 lib/riscv/stack.c            |  32 +++++
 lib/string.c                 |  14 ++
 lib/string.h                 |   2 +
 riscv/Makefile               | 106 ++++++++++++++
 riscv/cstart.S               | 259 +++++++++++++++++++++++++++++++++++
 riscv/flat.lds               |  75 ++++++++++
 riscv/run                    |  41 ++++++
 riscv/sbi.c                  |  41 ++++++
 riscv/selftest.c             | 100 ++++++++++++++
 riscv/sieve.c                |   1 +
 riscv/unittests.cfg          |  37 +++++
 63 files changed, 2616 insertions(+), 267 deletions(-)
 delete mode 100644 lib/arm64/asm/cpumask.h
 rename lib/{arm/asm => }/cpumask.h (72%)
 create mode 100644 lib/memregions.c
 create mode 100644 lib/memregions.h
 create mode 100644 lib/on-cpus.c
 create mode 100644 lib/on-cpus.h
 create mode 100644 lib/riscv/.gitignore
 create mode 100644 lib/riscv/asm-offsets.c
 create mode 100644 lib/riscv/asm/asm-offsets.h
 create mode 100644 lib/riscv/asm/barrier.h
 create mode 100644 lib/riscv/asm/bitops.h
 create mode 100644 lib/riscv/asm/bug.h
 create mode 100644 lib/riscv/asm/csr.h
 create mode 100644 lib/riscv/asm/io.h
 create mode 100644 lib/riscv/asm/isa.h
 create mode 100644 lib/riscv/asm/memory_areas.h
 create mode 100644 lib/riscv/asm/mmu.h
 create mode 100644 lib/riscv/asm/page.h
 create mode 100644 lib/riscv/asm/pgtable.h
 create mode 100644 lib/riscv/asm/processor.h
 create mode 100644 lib/riscv/asm/ptrace.h
 create mode 100644 lib/riscv/asm/sbi.h
 create mode 100644 lib/riscv/asm/setup.h
 create mode 100644 lib/riscv/asm/smp.h
 create mode 100644 lib/riscv/asm/spinlock.h
 create mode 100644 lib/riscv/asm/stack.h
 create mode 100644 lib/riscv/bitops.c
 create mode 100644 lib/riscv/io.c
 create mode 100644 lib/riscv/isa.c
 create mode 100644 lib/riscv/mmu.c
 create mode 100644 lib/riscv/processor.c
 create mode 100644 lib/riscv/sbi.c
 create mode 100644 lib/riscv/setup.c
 create mode 100644 lib/riscv/smp.c
 create mode 100644 lib/riscv/stack.c
 create mode 100644 riscv/Makefile
 create mode 100644 riscv/cstart.S
 create mode 100644 riscv/flat.lds
 create mode 100755 riscv/run
 create mode 100644 riscv/sbi.c
 create mode 100644 riscv/selftest.c
 create mode 120000 riscv/sieve.c
 create mode 100644 riscv/unittests.cfg

-- 
2.43.0


^ permalink raw reply	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 01/24] configure: Add ARCH_LIBDIR
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-01  8:29   ` Eric Auger
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 02/24] riscv: Initial port, hello world Andrew Jones
                   ` (23 subsequent siblings)
  24 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Prepare for an architecture which will share the same lib/$ARCH
directory, but be configured with different arch names for different
bit widths, i.e. riscv32 -> lib/riscv and riscv64 -> lib/riscv.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 Makefile  | 2 +-
 configure | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 602910dda11b..4f35fffc685b 100644
--- a/Makefile
+++ b/Makefile
@@ -10,7 +10,7 @@ include config.mak
 VPATH = $(SRCDIR)
 
 libdirs-get = $(shell [ -d "lib/$(1)" ] && echo "lib/$(1) lib/$(1)/asm")
-ARCH_LIBDIRS := $(call libdirs-get,$(ARCH)) $(call libdirs-get,$(TEST_DIR))
+ARCH_LIBDIRS := $(call libdirs-get,$(ARCH_LIBDIR)) $(call libdirs-get,$(TEST_DIR))
 OBJDIRS := $(ARCH_LIBDIRS)
 
 DESTDIR := $(PREFIX)/share/kvm-unit-tests/
diff --git a/configure b/configure
index 6ee9b27a6af2..ada6512702a1 100755
--- a/configure
+++ b/configure
@@ -198,6 +198,7 @@ fi
 arch_name=$arch
 [ "$arch" = "aarch64" ] && arch="arm64"
 [ "$arch_name" = "arm64" ] && arch_name="aarch64"
+arch_libdir=$arch
 
 if [ -z "$target" ]; then
     target="qemu"
@@ -391,6 +392,7 @@ PREFIX=$prefix
 HOST=$host
 ARCH=$arch
 ARCH_NAME=$arch_name
+ARCH_LIBDIR=$arch_libdir
 PROCESSOR=$processor
 CC=$cross_prefix$cc
 CFLAGS=$cflags
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 02/24] riscv: Initial port, hello world
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 01/24] configure: Add ARCH_LIBDIR Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-01  8:29   ` Eric Auger
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 03/24] arm/arm64: Move cpumask.h to common lib Andrew Jones
                   ` (22 subsequent siblings)
  24 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Add the minimal amount of code possible in order to launch a first
test, which just prints "Hello, world" using the expected UART
address of the QEMU virt machine. Add files, stubs, and some support,
such as barriers and MMIO read/write along the way in order to
satisfy the compiler. Basically everything is either copied from
the arm64 port of kvm-unit-tests, or at least inspired by it, and,
in that case, the RISC-V Linux kernel code was copied.

Run with
  qemu-system-riscv64 -nographic -M virt -kernel riscv/selftest.flat

and then go to the monitor (ctrl-a c) and use 'q' to quit, since
the unit test will just hang after printing hello world and the
exit code.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 configure                   | 14 ++++++
 lib/riscv/.gitignore        |  1 +
 lib/riscv/asm-offsets.c     |  6 +++
 lib/riscv/asm/asm-offsets.h |  1 +
 lib/riscv/asm/barrier.h     | 13 ++++++
 lib/riscv/asm/csr.h         |  7 +++
 lib/riscv/asm/io.h          | 78 +++++++++++++++++++++++++++++++
 lib/riscv/asm/page.h        |  7 +++
 lib/riscv/asm/setup.h       |  7 +++
 lib/riscv/asm/spinlock.h    |  7 +++
 lib/riscv/asm/stack.h       |  9 ++++
 lib/riscv/io.c              | 44 ++++++++++++++++++
 lib/riscv/setup.c           | 12 +++++
 riscv/Makefile              | 83 +++++++++++++++++++++++++++++++++
 riscv/cstart.S              | 92 +++++++++++++++++++++++++++++++++++++
 riscv/flat.lds              | 75 ++++++++++++++++++++++++++++++
 riscv/selftest.c            | 13 ++++++
 17 files changed, 469 insertions(+)
 create mode 100644 lib/riscv/.gitignore
 create mode 100644 lib/riscv/asm-offsets.c
 create mode 100644 lib/riscv/asm/asm-offsets.h
 create mode 100644 lib/riscv/asm/barrier.h
 create mode 100644 lib/riscv/asm/csr.h
 create mode 100644 lib/riscv/asm/io.h
 create mode 100644 lib/riscv/asm/page.h
 create mode 100644 lib/riscv/asm/setup.h
 create mode 100644 lib/riscv/asm/spinlock.h
 create mode 100644 lib/riscv/asm/stack.h
 create mode 100644 lib/riscv/io.c
 create mode 100644 lib/riscv/setup.c
 create mode 100644 riscv/Makefile
 create mode 100644 riscv/cstart.S
 create mode 100644 riscv/flat.lds
 create mode 100644 riscv/selftest.c

diff --git a/configure b/configure
index ada6512702a1..05e6702eab06 100755
--- a/configure
+++ b/configure
@@ -200,6 +200,11 @@ arch_name=$arch
 [ "$arch_name" = "arm64" ] && arch_name="aarch64"
 arch_libdir=$arch
 
+if [ "$arch" = "riscv" ]; then
+    echo "riscv32 or riscv64 must be specified"
+    exit 1
+fi
+
 if [ -z "$target" ]; then
     target="qemu"
 else
@@ -307,6 +312,9 @@ elif [ "$arch" = "ppc64" ]; then
         echo "You must provide endianness (big or little)!"
         usage
     fi
+elif [ "$arch" = "riscv32" ] || [ "$arch" = "riscv64" ]; then
+    testdir=riscv
+    arch_libdir=riscv
 else
     testdir=$arch
 fi
@@ -438,6 +446,12 @@ cat <<EOF >> lib/config.h
 #define CONFIG_ERRATA_FORCE ${errata_force}
 #define CONFIG_PAGE_SIZE _AC(${page_size}, UL)
 
+EOF
+elif [ "$arch" = "riscv32" ] || [ "$arch" = "riscv64" ]; then
+cat <<EOF >> lib/config.h
+
+#define CONFIG_UART_EARLY_BASE 0x10000000
+
 EOF
 fi
 echo "#endif" >> lib/config.h
diff --git a/lib/riscv/.gitignore b/lib/riscv/.gitignore
new file mode 100644
index 000000000000..82da12e6bd4e
--- /dev/null
+++ b/lib/riscv/.gitignore
@@ -0,0 +1 @@
+/asm-offsets.[hs]
diff --git a/lib/riscv/asm-offsets.c b/lib/riscv/asm-offsets.c
new file mode 100644
index 000000000000..4a74df9e4a09
--- /dev/null
+++ b/lib/riscv/asm-offsets.c
@@ -0,0 +1,6 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+int main(void)
+{
+	return 0;
+}
diff --git a/lib/riscv/asm/asm-offsets.h b/lib/riscv/asm/asm-offsets.h
new file mode 100644
index 000000000000..d370ee36a182
--- /dev/null
+++ b/lib/riscv/asm/asm-offsets.h
@@ -0,0 +1 @@
+#include <generated/asm-offsets.h>
diff --git a/lib/riscv/asm/barrier.h b/lib/riscv/asm/barrier.h
new file mode 100644
index 000000000000..c6a09066b2c7
--- /dev/null
+++ b/lib/riscv/asm/barrier.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_BARRIER_H_
+#define _ASMRISCV_BARRIER_H_
+
+#define RISCV_FENCE(p, s) \
+	__asm__ __volatile__ ("fence " #p "," #s : : : "memory")
+
+/* These barriers need to enforce ordering on both devices or memory. */
+#define mb()		RISCV_FENCE(iorw,iorw)
+#define rmb()		RISCV_FENCE(ir,ir)
+#define wmb()		RISCV_FENCE(ow,ow)
+
+#endif /* _ASMRISCV_BARRIER_H_ */
diff --git a/lib/riscv/asm/csr.h b/lib/riscv/asm/csr.h
new file mode 100644
index 000000000000..5c4f2de34f64
--- /dev/null
+++ b/lib/riscv/asm/csr.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_CSR_H_
+#define _ASMRISCV_CSR_H_
+
+#define CSR_SSCRATCH		0x140
+
+#endif /* _ASMRISCV_CSR_H_ */
diff --git a/lib/riscv/asm/io.h b/lib/riscv/asm/io.h
new file mode 100644
index 000000000000..d2eb3acc9fda
--- /dev/null
+++ b/lib/riscv/asm/io.h
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * From Linux arch/riscv/include/asm/mmio.h
+ */
+#ifndef _ASMRISCV_IO_H_
+#define _ASMRISCV_IO_H_
+#include <libcflat.h>
+
+#define __iomem
+
+/* Generic IO read/write.  These perform native-endian accesses. */
+#define __raw_writeb __raw_writeb
+static inline void __raw_writeb(u8 val, volatile void __iomem *addr)
+{
+	asm volatile("sb %0, 0(%1)" : : "r" (val), "r" (addr));
+}
+
+#define __raw_writew __raw_writew
+static inline void __raw_writew(u16 val, volatile void __iomem *addr)
+{
+	asm volatile("sh %0, 0(%1)" : : "r" (val), "r" (addr));
+}
+
+#define __raw_writel __raw_writel
+static inline void __raw_writel(u32 val, volatile void __iomem *addr)
+{
+	asm volatile("sw %0, 0(%1)" : : "r" (val), "r" (addr));
+}
+
+#ifdef CONFIG_64BIT
+#define __raw_writeq __raw_writeq
+static inline void __raw_writeq(u64 val, volatile void __iomem *addr)
+{
+	asm volatile("sd %0, 0(%1)" : : "r" (val), "r" (addr));
+}
+#endif
+
+#define __raw_readb __raw_readb
+static inline u8 __raw_readb(const volatile void __iomem *addr)
+{
+	u8 val;
+
+	asm volatile("lb %0, 0(%1)" : "=r" (val) : "r" (addr));
+	return val;
+}
+
+#define __raw_readw __raw_readw
+static inline u16 __raw_readw(const volatile void __iomem *addr)
+{
+	u16 val;
+
+	asm volatile("lh %0, 0(%1)" : "=r" (val) : "r" (addr));
+	return val;
+}
+
+#define __raw_readl __raw_readl
+static inline u32 __raw_readl(const volatile void __iomem *addr)
+{
+	u32 val;
+
+	asm volatile("lw %0, 0(%1)" : "=r" (val) : "r" (addr));
+	return val;
+}
+
+#ifdef CONFIG_64BIT
+#define __raw_readq __raw_readq
+static inline u64 __raw_readq(const volatile void __iomem *addr)
+{
+	u64 val;
+
+	asm volatile("ld %0, 0(%1)" : "=r" (val) : "r" (addr));
+	return val;
+}
+#endif
+
+#include <asm-generic/io.h>
+
+#endif /* _ASMRISCV_IO_H_ */
diff --git a/lib/riscv/asm/page.h b/lib/riscv/asm/page.h
new file mode 100644
index 000000000000..7d7c9191605a
--- /dev/null
+++ b/lib/riscv/asm/page.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_PAGE_H_
+#define _ASMRISCV_PAGE_H_
+
+#include <asm-generic/page.h>
+
+#endif /* _ASMRISCV_PAGE_H_ */
diff --git a/lib/riscv/asm/setup.h b/lib/riscv/asm/setup.h
new file mode 100644
index 000000000000..385455f341cc
--- /dev/null
+++ b/lib/riscv/asm/setup.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_SETUP_H_
+#define _ASMRISCV_SETUP_H_
+
+void setup(const void *fdt, phys_addr_t freemem_start);
+
+#endif /* _ASMRISCV_SETUP_H_ */
diff --git a/lib/riscv/asm/spinlock.h b/lib/riscv/asm/spinlock.h
new file mode 100644
index 000000000000..6e2b3009abf3
--- /dev/null
+++ b/lib/riscv/asm/spinlock.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_SPINLOCK_H_
+#define _ASMRISCV_SPINLOCK_H_
+
+#include <asm-generic/spinlock.h>
+
+#endif /* _ASMRISCV_SPINLOCK_H_ */
diff --git a/lib/riscv/asm/stack.h b/lib/riscv/asm/stack.h
new file mode 100644
index 000000000000..d081d0716d7b
--- /dev/null
+++ b/lib/riscv/asm/stack.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_STACK_H_
+#define _ASMRISCV_STACK_H_
+
+#ifndef _STACK_H_
+#error Do not directly include <asm/stack.h>. Just use <stack.h>.
+#endif
+
+#endif
diff --git a/lib/riscv/io.c b/lib/riscv/io.c
new file mode 100644
index 000000000000..3cfc235d19a6
--- /dev/null
+++ b/lib/riscv/io.c
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Each architecture must implement puts() and exit() with the I/O
+ * devices exposed from QEMU, e.g. ns16550a.
+ *
+ * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
+ */
+#include <libcflat.h>
+#include <config.h>
+#include <asm/io.h>
+#include <asm/spinlock.h>
+
+/*
+ * Use this guess for the uart base in order to make an attempt at
+ * having earlier printf support. We'll overwrite it with the real
+ * base address that we read from the device tree later. This is
+ * the address we expect the virtual machine manager to put in
+ * its generated device tree.
+ */
+#define UART_EARLY_BASE ((u8 *)(unsigned long)CONFIG_UART_EARLY_BASE)
+static volatile u8 *uart0_base = UART_EARLY_BASE;
+static struct spinlock uart_lock;
+
+void puts(const char *s)
+{
+	spin_lock(&uart_lock);
+	while (*s)
+		writeb(*s++, uart0_base);
+	spin_unlock(&uart_lock);
+}
+
+/*
+ * Defining halt to take 'code' as an argument guarantees that it will
+ * be in a0 when we halt. That gives us a final chance to see the exit
+ * status while inspecting the halted unit test state.
+ */
+void halt(int code);
+
+void exit(int code)
+{
+	printf("\nEXIT: STATUS=%d\n", ((code) << 1) | 1);
+	halt(code);
+	__builtin_unreachable();
+}
diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c
new file mode 100644
index 000000000000..8937525ccb7f
--- /dev/null
+++ b/lib/riscv/setup.c
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Initialize machine setup information and I/O.
+ *
+ * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
+ */
+#include <libcflat.h>
+#include <asm/setup.h>
+
+void setup(const void *fdt, phys_addr_t freemem_start)
+{
+}
diff --git a/riscv/Makefile b/riscv/Makefile
new file mode 100644
index 000000000000..f2e89f0e4c38
--- /dev/null
+++ b/riscv/Makefile
@@ -0,0 +1,83 @@
+#
+# riscv makefile
+#
+# Authors: Andrew Jones <ajones@ventanamicro.com>
+#
+
+ifeq ($(CONFIG_EFI),y)
+exe = efi
+else
+exe = flat
+endif
+
+tests =
+tests += $(TEST_DIR)/selftest.$(exe)
+#tests += $(TEST_DIR)/sieve.$(exe)
+
+all: $(tests)
+
+$(TEST_DIR)/sieve.elf: AUXFLAGS = 0x1
+
+cstart.o = $(TEST_DIR)/cstart.o
+
+cflatobjs += lib/riscv/io.o
+cflatobjs += lib/riscv/setup.o
+
+########################################
+
+OBJDIRS += lib/riscv
+FLATLIBS = $(libcflat) $(LIBFDT_archive)
+
+AUXFLAGS ?= 0x0
+
+# stack.o relies on frame pointers.
+KEEP_FRAME_POINTER := y
+
+# We want to keep intermediate files
+.PRECIOUS: %.elf %.o
+
+define arch_elf_check =
+	$(if $(shell ! $(READELF) -rW $(1) >&/dev/null && echo "nok"),
+		$(error $(shell $(READELF) -rW $(1) 2>&1)))
+	$(if $(shell $(READELF) -rW $(1) | grep R_ | grep -v R_RISCV_RELATIVE),
+		$(error $(1) has unsupported reloc types))
+endef
+
+ifeq ($(ARCH),riscv64)
+CFLAGS += -DCONFIG_64BIT
+endif
+CFLAGS += -DCONFIG_RELOC
+CFLAGS += -mcmodel=medany
+CFLAGS += -mstrict-align
+CFLAGS += -std=gnu99
+CFLAGS += -ffreestanding
+CFLAGS += -O2
+CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt
+
+asm-offsets = lib/riscv/asm-offsets.h
+include $(SRCDIR)/scripts/asm-offsets.mak
+
+ifeq ($(CONFIG_EFI),y)
+	# TODO
+else
+%.elf: LDFLAGS += -pie -n -z notext
+%.elf: %.o $(FLATLIBS) $(SRCDIR)/riscv/flat.lds $(cstart.o)
+	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) $(SRCDIR)/lib/auxinfo.c \
+		-DPROGNAME=\"$(notdir $(@:.elf=.flat))\" -DAUXFLAGS=$(AUXFLAGS)
+	$(LD) $(LDFLAGS) -o $@ -T $(SRCDIR)/riscv/flat.lds \
+		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+	$(RM) $(@:.elf=.aux.o)
+	@chmod a-x $@
+
+%.flat: %.elf
+	$(call arch_elf_check, $^)
+	$(OBJCOPY) -O binary $^ $@
+	@chmod a-x $@
+endif
+
+generated-files = $(asm-offsets)
+$(tests:.$(exe)=.o) $(cstart.o) $(cflatobjs): $(generated-files)
+
+arch_clean: asm_offsets_clean
+	$(RM) $(TEST_DIR)/*.{o,flat,elf,so,efi,debug} \
+	      $(TEST_DIR)/.*.d $(TEST_DIR)/efi/.*.d lib/riscv/.*.d
diff --git a/riscv/cstart.S b/riscv/cstart.S
new file mode 100644
index 000000000000..a28d75e8021e
--- /dev/null
+++ b/riscv/cstart.S
@@ -0,0 +1,92 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Boot entry point and assembler functions for riscv.
+ *
+ * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
+ */
+#include <asm/csr.h>
+
+.macro zero_range, tmp1, tmp2
+9998:	beq	\tmp1, \tmp2, 9997f
+	sd	zero, 0(\tmp1)
+	addi	\tmp1, \tmp1, 8
+	j	9998b
+9997:
+.endm
+
+	.section .init
+
+/*
+ * The hartid of the current core is in a0
+ * The address of the devicetree is in a1
+ *
+ * See Linux kernel doc Documentation/riscv/boot.rst
+ */
+.global start
+start:
+	/*
+	 * Stash the hartid in scratch and shift the dtb
+	 * address into a0
+	 */
+	csrw	CSR_SSCRATCH, a0
+	mv	a0, a1
+
+	/*
+	 * Update all R_RISCV_RELATIVE relocations using the table
+	 * of Elf64_Rela entries between reloc_start/end. The build
+	 * will not emit other relocation types.
+	 *
+	 * struct Elf64_Rela {
+	 * 	uint64_t r_offset;
+	 * 	uint64_t r_info;
+	 * 	int64_t  r_addend;
+	 * }
+	 */
+	la	a1, reloc_start
+	la	a2, reloc_end
+	la	a3, start			// base
+1:
+	bge	a1, a2, 1f
+	ld	a4, 0(a1)			// r_offset
+	ld	a5, 16(a1)			// r_addend
+	add	a4, a3, a4			// addr = base + r_offset
+	add	a5, a3, a5			// val = base + r_addend
+	sd	a5, 0(a4)			// *addr = val
+	addi	a1, a1, 24
+	j	1b
+
+1:
+	/* zero BSS */
+	la	a1, bss
+	la	a2, ebss
+	zero_range a1, a2
+
+	/* zero and set up stack */
+	la	sp, stacktop
+	li	a1, -8192
+	add	a1, sp, a1
+	zero_range a1, sp
+
+	/* set up exception handling */
+	//TODO
+
+	/* complete setup */
+	la	a1, stacktop			// a1 is the base of free memory
+	call	setup				// a0 is the addr of the dtb
+
+	/* run the test */
+	la	a0, __argc
+	ld	a0, 0(a0)
+	la	a1, __argv
+	la	a2, __environ
+	call	main
+	call	exit
+	j	halt
+
+	.text
+
+.balign 4
+.global halt
+halt:
+1:	wfi
+	j	1b
diff --git a/riscv/flat.lds b/riscv/flat.lds
new file mode 100644
index 000000000000..d4853f82ba1c
--- /dev/null
+++ b/riscv/flat.lds
@@ -0,0 +1,75 @@
+/*
+ * init::start will pass stacktop to setup() as the base of free memory.
+ * setup() will then move the FDT and initrd to that base before calling
+ * mem_init(). With those movements and this linker script, we'll end up
+ * having the following memory layout:
+ *
+ *    +----------------------+   <-- top of physical memory
+ *    |                      |
+ *    ~                      ~
+ *    |                      |
+ *    +----------------------+   <-- top of initrd
+ *    |                      |
+ *    +----------------------+   <-- top of FDT
+ *    |                      |
+ *    +----------------------+   <-- top of cpu0's stack
+ *    |                      |
+ *    +----------------------+   <-- top of text/data/bss sections
+ *    |                      |
+ *    |                      |
+ *    +----------------------+   <-- load address
+ *    |                      |
+ *    +----------------------+   <-- physical address 0x0
+ */
+
+PHDRS
+{
+    text PT_LOAD FLAGS(5);
+    data PT_LOAD FLAGS(6);
+}
+
+SECTIONS
+{
+    PROVIDE(_text = .);
+    .text : { *(.init) *(.text) *(.text.*) } :text
+    . = ALIGN(4K);
+    PROVIDE(_etext = .);
+
+    PROVIDE(reloc_start = .);
+    .rela.dyn : { *(.rela.dyn) }
+    PROVIDE(reloc_end = .);
+    .dynsym   : { *(.dynsym) }
+    .dynstr   : { *(.dynstr) }
+    .hash     : { *(.hash) }
+    .gnu.hash : { *(.gnu.hash) }
+    .got      : { *(.got) *(.got.plt) }
+    .eh_frame : { *(.eh_frame) }
+
+    .rodata   : { *(.rodata*) } :data
+    .data     : { *(.data) } :data
+    . = ALIGN(16);
+    PROVIDE(bss = .);
+    .bss      : { *(.bss) }
+    . = ALIGN(16);
+    PROVIDE(ebss = .);
+    . = ALIGN(4K);
+    PROVIDE(edata = .);
+
+    /*
+     * stack depth is 8K and sp must be 16 byte aligned
+     * sp must always be strictly less than the true stacktop
+     */
+    . += 12K;
+    . = ALIGN(4K);
+    PROVIDE(stackptr = . - 16);
+    PROVIDE(stacktop = .);
+
+    /DISCARD/ : {
+        *(.note*)
+        *(.interp)
+        *(.comment)
+        *(.dynamic)
+    }
+}
+
+ENTRY(start)
diff --git a/riscv/selftest.c b/riscv/selftest.c
new file mode 100644
index 000000000000..88afa732649e
--- /dev/null
+++ b/riscv/selftest.c
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Test the framework itself. These tests confirm that setup works.
+ *
+ * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
+ */
+#include <libcflat.h>
+
+int main(void)
+{
+	puts("Hello, world\n");
+	return 0;
+}
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 03/24] arm/arm64: Move cpumask.h to common lib
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 01/24] configure: Add ARCH_LIBDIR Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 02/24] riscv: Initial port, hello world Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-01  8:29   ` Eric Auger
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 04/24] arm/arm64: Share cpu online, present and idle masks Andrew Jones
                   ` (21 subsequent siblings)
  24 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

RISC-V will also make use of cpumask.h, so move it to the arch-common
directory.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/arm/asm/gic-v2.h        | 2 +-
 lib/arm/asm/gic-v3.h        | 2 +-
 lib/arm/asm/gic.h           | 2 +-
 lib/arm/asm/smp.h           | 2 +-
 lib/arm/mmu.c               | 2 +-
 lib/arm/smp.c               | 2 +-
 lib/arm64/asm/cpumask.h     | 1 -
 lib/{arm/asm => }/cpumask.h | 9 ++++-----
 8 files changed, 10 insertions(+), 12 deletions(-)
 delete mode 100644 lib/arm64/asm/cpumask.h
 rename lib/{arm/asm => }/cpumask.h (94%)

diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
index 1fcfd43c8075..ff11afb15d30 100644
--- a/lib/arm/asm/gic-v2.h
+++ b/lib/arm/asm/gic-v2.h
@@ -18,7 +18,7 @@
 #define GICC_IAR_INT_ID_MASK		0x3ff
 
 #ifndef __ASSEMBLY__
-#include <asm/cpumask.h>
+#include <cpumask.h>
 
 struct gicv2_data {
 	void *dist_base;
diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
index b4ce130e56c6..a1cc62a298b8 100644
--- a/lib/arm/asm/gic-v3.h
+++ b/lib/arm/asm/gic-v3.h
@@ -67,10 +67,10 @@
 #include <asm/arch_gicv3.h>
 
 #ifndef __ASSEMBLY__
+#include <cpumask.h>
 #include <asm/setup.h>
 #include <asm/processor.h>
 #include <asm/delay.h>
-#include <asm/cpumask.h>
 #include <asm/smp.h>
 #include <asm/io.h>
 
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index 189840014b02..dc8cc18c0fbd 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -47,7 +47,7 @@
 #define SPI(irq)			((irq) + GIC_FIRST_SPI)
 
 #ifndef __ASSEMBLY__
-#include <asm/cpumask.h>
+#include <cpumask.h>
 
 enum gic_irq_state {
 	GIC_IRQ_STATE_INACTIVE,
diff --git a/lib/arm/asm/smp.h b/lib/arm/asm/smp.h
index dee4c1a883e7..bb3e71a55e8c 100644
--- a/lib/arm/asm/smp.h
+++ b/lib/arm/asm/smp.h
@@ -5,8 +5,8 @@
  *
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
+#include <cpumask.h>
 #include <asm/thread_info.h>
-#include <asm/cpumask.h>
 
 #define smp_processor_id()		(current_thread_info()->cpu)
 
diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index 2f4ec815a35d..b16517a3200d 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -5,9 +5,9 @@
  *
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
+#include <cpumask.h>
 #include <asm/setup.h>
 #include <asm/thread_info.h>
-#include <asm/cpumask.h>
 #include <asm/mmu.h>
 #include <asm/setup.h>
 #include <asm/page.h>
diff --git a/lib/arm/smp.c b/lib/arm/smp.c
index 1d470d1aab45..78fc1656cefa 100644
--- a/lib/arm/smp.c
+++ b/lib/arm/smp.c
@@ -7,9 +7,9 @@
  */
 #include <libcflat.h>
 #include <auxinfo.h>
+#include <cpumask.h>
 #include <asm/thread_info.h>
 #include <asm/spinlock.h>
-#include <asm/cpumask.h>
 #include <asm/barrier.h>
 #include <asm/mmu.h>
 #include <asm/psci.h>
diff --git a/lib/arm64/asm/cpumask.h b/lib/arm64/asm/cpumask.h
deleted file mode 100644
index d1421e7abe31..000000000000
--- a/lib/arm64/asm/cpumask.h
+++ /dev/null
@@ -1 +0,0 @@
-#include "../../arm/asm/cpumask.h"
diff --git a/lib/arm/asm/cpumask.h b/lib/cpumask.h
similarity index 94%
rename from lib/arm/asm/cpumask.h
rename to lib/cpumask.h
index 3fa57bfb17c6..d30e14cda09e 100644
--- a/lib/arm/asm/cpumask.h
+++ b/lib/cpumask.h
@@ -1,12 +1,11 @@
-#ifndef _ASMARM_CPUMASK_H_
-#define _ASMARM_CPUMASK_H_
+/* SPDX-License-Identifier: GPL-2.0-only */
 /*
  * Simple cpumask implementation
  *
  * Copyright (C) 2015, Red Hat Inc, Andrew Jones <drjones@redhat.com>
- *
- * This work is licensed under the terms of the GNU LGPL, version 2.
  */
+#ifndef _CPUMASK_H_
+#define _CPUMASK_H_
 #include <asm/setup.h>
 #include <bitops.h>
 
@@ -120,4 +119,4 @@ static inline int cpumask_next(int cpu, const cpumask_t *mask)
 			(cpu) < nr_cpus; 			\
 			(cpu) = cpumask_next(cpu, mask))
 
-#endif /* _ASMARM_CPUMASK_H_ */
+#endif /* _CPUMASK_H_ */
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 04/24] arm/arm64: Share cpu online, present and idle masks
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (2 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 03/24] arm/arm64: Move cpumask.h to common lib Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-01  8:29   ` Eric Auger
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 05/24] riscv: Add DT parsing Andrew Jones
                   ` (20 subsequent siblings)
  24 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

RISC-V will also use Arm's three cpumasks. These were in smp.h,
but they can be in cpumask.h instead, so move them there, which
is now shared.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/arm/asm/smp.h | 33 ---------------------------------
 lib/cpumask.h     | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+), 33 deletions(-)

diff --git a/lib/arm/asm/smp.h b/lib/arm/asm/smp.h
index bb3e71a55e8c..b89a68dd344f 100644
--- a/lib/arm/asm/smp.h
+++ b/lib/arm/asm/smp.h
@@ -23,39 +23,6 @@ extern bool cpu0_calls_idle;
 extern void halt(void);
 extern void do_idle(void);
 
-extern cpumask_t cpu_present_mask;
-extern cpumask_t cpu_online_mask;
-extern cpumask_t cpu_idle_mask;
-#define cpu_present(cpu)		cpumask_test_cpu(cpu, &cpu_present_mask)
-#define cpu_online(cpu)			cpumask_test_cpu(cpu, &cpu_online_mask)
-#define cpu_idle(cpu)			cpumask_test_cpu(cpu, &cpu_idle_mask)
-#define for_each_present_cpu(cpu)	for_each_cpu(cpu, &cpu_present_mask)
-#define for_each_online_cpu(cpu)	for_each_cpu(cpu, &cpu_online_mask)
-
-static inline void set_cpu_present(int cpu, bool present)
-{
-	if (present)
-		cpumask_set_cpu(cpu, &cpu_present_mask);
-	else
-		cpumask_clear_cpu(cpu, &cpu_present_mask);
-}
-
-static inline void set_cpu_online(int cpu, bool online)
-{
-	if (online)
-		cpumask_set_cpu(cpu, &cpu_online_mask);
-	else
-		cpumask_clear_cpu(cpu, &cpu_online_mask);
-}
-
-static inline void set_cpu_idle(int cpu, bool idle)
-{
-	if (idle)
-		cpumask_set_cpu(cpu, &cpu_idle_mask);
-	else
-		cpumask_clear_cpu(cpu, &cpu_idle_mask);
-}
-
 extern void smp_boot_secondary(int cpu, secondary_entry_fn entry);
 extern void on_cpu_async(int cpu, void (*func)(void *data), void *data);
 extern void on_cpu(int cpu, void (*func)(void *data), void *data);
diff --git a/lib/cpumask.h b/lib/cpumask.h
index d30e14cda09e..be1919234d8e 100644
--- a/lib/cpumask.h
+++ b/lib/cpumask.h
@@ -119,4 +119,37 @@ static inline int cpumask_next(int cpu, const cpumask_t *mask)
 			(cpu) < nr_cpus; 			\
 			(cpu) = cpumask_next(cpu, mask))
 
+extern cpumask_t cpu_present_mask;
+extern cpumask_t cpu_online_mask;
+extern cpumask_t cpu_idle_mask;
+#define cpu_present(cpu)		cpumask_test_cpu(cpu, &cpu_present_mask)
+#define cpu_online(cpu)			cpumask_test_cpu(cpu, &cpu_online_mask)
+#define cpu_idle(cpu)			cpumask_test_cpu(cpu, &cpu_idle_mask)
+#define for_each_present_cpu(cpu)	for_each_cpu(cpu, &cpu_present_mask)
+#define for_each_online_cpu(cpu)	for_each_cpu(cpu, &cpu_online_mask)
+
+static inline void set_cpu_present(int cpu, bool present)
+{
+	if (present)
+		cpumask_set_cpu(cpu, &cpu_present_mask);
+	else
+		cpumask_clear_cpu(cpu, &cpu_present_mask);
+}
+
+static inline void set_cpu_online(int cpu, bool online)
+{
+	if (online)
+		cpumask_set_cpu(cpu, &cpu_online_mask);
+	else
+		cpumask_clear_cpu(cpu, &cpu_online_mask);
+}
+
+static inline void set_cpu_idle(int cpu, bool idle)
+{
+	if (idle)
+		cpumask_set_cpu(cpu, &cpu_idle_mask);
+	else
+		cpumask_clear_cpu(cpu, &cpu_idle_mask);
+}
+
 #endif /* _CPUMASK_H_ */
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 05/24] riscv: Add DT parsing
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (3 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 04/24] arm/arm64: Share cpu online, present and idle masks Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 06/24] riscv: Add initial SBI support Andrew Jones
                   ` (19 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Start building setup() by copying code from Arm and adding
dependencies along the way like bitops and a few more barriers.
We now parse the DT for the UART base address to be sure we
find what we expect with the early base. We also parse the
CPU nodes to get the hartids, even though we don't yet support
booting secondaries. Finally, add parsing of bootargs to get
the command line and parsing of the environ to set the environment
variables, and then extend the selftest to ensure it all works.

We don't do proper memory setup yet, only just enough to allocate
memory for the environment variables and any another small mallocs
that may be necessary.

Note, we've added a banner, which Arm doesn't have, because we
need to separate the test output from OpenSBI output.

Run with
  qemu-system-riscv64 -nographic -M virt \
      -kernel riscv/selftest.flat \
      -append 'foo bar baz' \
      -initrd test-env \
      -smp 16

where test-env is a text file with the environment, i.e.
$ cat test-env
FOO=foo
BAR=bar
BAZ=baz

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/linux/const.h       |   2 +
 lib/riscv/asm/barrier.h |   5 ++
 lib/riscv/asm/bitops.h  |  21 ++++++++
 lib/riscv/asm/csr.h     |  64 +++++++++++++++++++++++
 lib/riscv/asm/setup.h   |   7 +++
 lib/riscv/bitops.c      |  47 +++++++++++++++++
 lib/riscv/io.c          |  51 +++++++++++++++++++
 lib/riscv/setup.c       | 109 ++++++++++++++++++++++++++++++++++++++++
 lib/riscv/smp.c         |   6 +++
 riscv/Makefile          |   5 ++
 riscv/selftest.c        |  44 ++++++++++++++--
 11 files changed, 358 insertions(+), 3 deletions(-)
 create mode 100644 lib/riscv/asm/bitops.h
 create mode 100644 lib/riscv/bitops.c
 create mode 100644 lib/riscv/smp.c

diff --git a/lib/linux/const.h b/lib/linux/const.h
index be114dc4a553..f622fa852ced 100644
--- a/lib/linux/const.h
+++ b/lib/linux/const.h
@@ -15,10 +15,12 @@
 #if defined(__ASSEMBLY__) || defined(__ASSEMBLER__)
 #define _AC(X,Y)	X
 #define _AT(T,X)	X
+#define __ASM_STR(X)	X
 #else
 #define __AC(X,Y)	(X##Y)
 #define _AC(X,Y)	__AC(X,Y)
 #define _AT(T,X)	((T)(X))
+#define __ASM_STR(X)	#X
 #endif
 
 #define _BITUL(x)	(_AC(1,UL) << (x))
diff --git a/lib/riscv/asm/barrier.h b/lib/riscv/asm/barrier.h
index c6a09066b2c7..6036d66af76f 100644
--- a/lib/riscv/asm/barrier.h
+++ b/lib/riscv/asm/barrier.h
@@ -10,4 +10,9 @@
 #define rmb()		RISCV_FENCE(ir,ir)
 #define wmb()		RISCV_FENCE(ow,ow)
 
+/* These barriers do not need to enforce ordering on devices, just memory. */
+#define smp_mb()	RISCV_FENCE(rw,rw)
+#define smp_rmb()	RISCV_FENCE(r,r)
+#define smp_wmb()	RISCV_FENCE(w,w)
+
 #endif /* _ASMRISCV_BARRIER_H_ */
diff --git a/lib/riscv/asm/bitops.h b/lib/riscv/asm/bitops.h
new file mode 100644
index 000000000000..0d982507c7e7
--- /dev/null
+++ b/lib/riscv/asm/bitops.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_BITOPS_H_
+#define _ASMRISCV_BITOPS_H_
+
+#ifndef _BITOPS_H_
+#error only <bitops.h> can be included directly
+#endif
+
+#ifdef CONFIG_64BIT
+#define BITS_PER_LONG	64
+#else
+#define BITS_PER_LONG	32
+#endif
+
+void set_bit(int nr, volatile unsigned long *addr);
+void clear_bit(int nr, volatile unsigned long *addr);
+int test_bit(int nr, const volatile unsigned long *addr);
+int test_and_set_bit(int nr, volatile unsigned long *addr);
+int test_and_clear_bit(int nr, volatile unsigned long *addr);
+
+#endif /* _ASMRISCV_BITOPS_H_ */
diff --git a/lib/riscv/asm/csr.h b/lib/riscv/asm/csr.h
index 5c4f2de34f64..356ae054bfff 100644
--- a/lib/riscv/asm/csr.h
+++ b/lib/riscv/asm/csr.h
@@ -1,7 +1,71 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 #ifndef _ASMRISCV_CSR_H_
 #define _ASMRISCV_CSR_H_
+#include <linux/const.h>
 
 #define CSR_SSCRATCH		0x140
 
+#ifndef __ASSEMBLY__
+
+#define csr_swap(csr, val)					\
+({								\
+	unsigned long __v = (unsigned long)(val);		\
+	__asm__ __volatile__ ("csrrw %0, " __ASM_STR(csr) ", %1"\
+				: "=r" (__v) : "rK" (__v)	\
+				: "memory");			\
+	__v;							\
+})
+
+#define csr_read(csr)						\
+({								\
+	register unsigned long __v;				\
+	__asm__ __volatile__ ("csrr %0, " __ASM_STR(csr)	\
+				: "=r" (__v) :			\
+				: "memory");			\
+	__v;							\
+})
+
+#define csr_write(csr, val)					\
+({								\
+	unsigned long __v = (unsigned long)(val);		\
+	__asm__ __volatile__ ("csrw " __ASM_STR(csr) ", %0"	\
+				: : "rK" (__v)			\
+				: "memory");			\
+})
+
+#define csr_read_set(csr, val)					\
+({								\
+	unsigned long __v = (unsigned long)(val);		\
+	__asm__ __volatile__ ("csrrs %0, " __ASM_STR(csr) ", %1"\
+				: "=r" (__v) : "rK" (__v)	\
+				: "memory");			\
+	__v;							\
+})
+
+#define csr_set(csr, val)					\
+({								\
+	unsigned long __v = (unsigned long)(val);		\
+	__asm__ __volatile__ ("csrs " __ASM_STR(csr) ", %0"	\
+				: : "rK" (__v)			\
+				: "memory");			\
+})
+
+#define csr_read_clear(csr, val)				\
+({								\
+	unsigned long __v = (unsigned long)(val);		\
+	__asm__ __volatile__ ("csrrc %0, " __ASM_STR(csr) ", %1"\
+				: "=r" (__v) : "rK" (__v)	\
+				: "memory");			\
+	__v;							\
+})
+
+#define csr_clear(csr, val)					\
+({								\
+	unsigned long __v = (unsigned long)(val);		\
+	__asm__ __volatile__ ("csrc " __ASM_STR(csr) ", %0"	\
+				: : "rK" (__v)			\
+				: "memory");			\
+})
+
+#endif /* !__ASSEMBLY__ */
 #endif /* _ASMRISCV_CSR_H_ */
diff --git a/lib/riscv/asm/setup.h b/lib/riscv/asm/setup.h
index 385455f341cc..c8cfebb4f2c1 100644
--- a/lib/riscv/asm/setup.h
+++ b/lib/riscv/asm/setup.h
@@ -1,7 +1,14 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 #ifndef _ASMRISCV_SETUP_H_
 #define _ASMRISCV_SETUP_H_
+#include <libcflat.h>
 
+#define NR_CPUS 16
+extern unsigned long cpus[NR_CPUS];       /* per-cpu IDs (hartids) */
+extern int nr_cpus;
+int hartid_to_cpu(unsigned long hartid);
+
+void io_init(void);
 void setup(const void *fdt, phys_addr_t freemem_start);
 
 #endif /* _ASMRISCV_SETUP_H_ */
diff --git a/lib/riscv/bitops.c b/lib/riscv/bitops.c
new file mode 100644
index 000000000000..f9d4d9ad45c3
--- /dev/null
+++ b/lib/riscv/bitops.c
@@ -0,0 +1,47 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
+ */
+#include <bitops.h>
+
+void set_bit(int nr, volatile unsigned long *addr)
+{
+	volatile unsigned long *word = addr + BIT_WORD(nr);
+	unsigned long mask = BIT_MASK(nr);
+
+	__sync_or_and_fetch(word, mask);
+}
+
+void clear_bit(int nr, volatile unsigned long *addr)
+{
+	volatile unsigned long *word = addr + BIT_WORD(nr);
+	unsigned long mask = BIT_MASK(nr);
+
+	__sync_and_and_fetch(word, ~mask);
+}
+
+int test_bit(int nr, const volatile unsigned long *addr)
+{
+	const volatile unsigned long *word = addr + BIT_WORD(nr);
+	unsigned long mask = BIT_MASK(nr);
+
+	return (*word & mask) != 0;
+}
+
+int test_and_set_bit(int nr, volatile unsigned long *addr)
+{
+	volatile unsigned long *word = addr + BIT_WORD(nr);
+	unsigned long mask = BIT_MASK(nr);
+	unsigned long old = __sync_fetch_and_or(word, mask);
+
+	return (old & mask) != 0;
+}
+
+int test_and_clear_bit(int nr, volatile unsigned long *addr)
+{
+	volatile unsigned long *word = addr + BIT_WORD(nr);
+	unsigned long mask = BIT_MASK(nr);
+	unsigned long old = __sync_fetch_and_and(word, ~mask);
+
+	return (old & mask) != 0;
+}
diff --git a/lib/riscv/io.c b/lib/riscv/io.c
index 3cfc235d19a6..aeda74be61ee 100644
--- a/lib/riscv/io.c
+++ b/lib/riscv/io.c
@@ -7,7 +7,9 @@
  */
 #include <libcflat.h>
 #include <config.h>
+#include <devicetree.h>
 #include <asm/io.h>
+#include <asm/setup.h>
 #include <asm/spinlock.h>
 
 /*
@@ -21,6 +23,55 @@
 static volatile u8 *uart0_base = UART_EARLY_BASE;
 static struct spinlock uart_lock;
 
+static void uart0_init_fdt(void)
+{
+	const char *compatible[] = {"ns16550a"};
+	struct dt_pbus_reg base;
+	int i, ret;
+
+	ret = dt_get_default_console_node();
+	assert(ret >= 0 || ret == -FDT_ERR_NOTFOUND);
+
+	if (ret == -FDT_ERR_NOTFOUND) {
+		for (i = 0; i < ARRAY_SIZE(compatible); i++) {
+			ret = dt_pbus_get_base_compatible(compatible[i], &base);
+			assert(ret == 0 || ret == -FDT_ERR_NOTFOUND);
+			if (ret == 0)
+				break;
+		}
+
+		if (ret) {
+			printf("%s: Compatible uart not found in the device tree, aborting...\n",
+			       __func__);
+			abort();
+		}
+	} else {
+		ret = dt_pbus_translate_node(ret, 0, &base);
+		assert(ret == 0);
+	}
+
+	uart0_base = ioremap(base.addr, base.size);
+}
+
+static void uart0_init_acpi(void)
+{
+	assert_msg(false, "ACPI not available");
+}
+
+void io_init(void)
+{
+	if (dt_available())
+		uart0_init_fdt();
+	else
+		uart0_init_acpi();
+
+	if (uart0_base != UART_EARLY_BASE) {
+		printf("WARNING: early print support may not work. "
+		       "Found uart at %p, but early base is %p.\n",
+		       uart0_base, UART_EARLY_BASE);
+	}
+}
+
 void puts(const char *s)
 {
 	spin_lock(&uart_lock);
diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c
index 8937525ccb7f..44c26b125a27 100644
--- a/lib/riscv/setup.c
+++ b/lib/riscv/setup.c
@@ -5,8 +5,117 @@
  * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
  */
 #include <libcflat.h>
+#include <alloc.h>
+#include <alloc_phys.h>
+#include <argv.h>
+#include <cpumask.h>
+#include <devicetree.h>
+#include <asm/csr.h>
+#include <asm/page.h>
 #include <asm/setup.h>
 
+char *initrd;
+u32 initrd_size;
+
+unsigned long cpus[NR_CPUS] = { [0 ... NR_CPUS - 1] = ~0UL };
+int nr_cpus;
+
+int hartid_to_cpu(unsigned long hartid)
+{
+	int cpu;
+
+	for_each_present_cpu(cpu)
+		if (cpus[cpu] == hartid)
+			return cpu;
+	return -1;
+}
+
+static void cpu_set_fdt(int fdtnode __unused, u64 regval, void *info __unused)
+{
+	int cpu = nr_cpus++;
+
+	assert_msg(cpu < NR_CPUS, "Number cpus exceeds maximum supported (%d).", NR_CPUS);
+
+	cpus[cpu] = regval;
+	set_cpu_present(cpu, true);
+}
+
+static void cpu_init_acpi(void)
+{
+	assert_msg(false, "ACPI not available");
+}
+
+static void cpu_init(void)
+{
+	int ret;
+
+	nr_cpus = 0;
+	if (dt_available()) {
+		ret = dt_for_each_cpu_node(cpu_set_fdt, NULL);
+		assert(ret == 0);
+	} else {
+		cpu_init_acpi();
+	}
+
+	set_cpu_online(hartid_to_cpu(csr_read(CSR_SSCRATCH)), true);
+}
+
+static void mem_init(phys_addr_t freemem_start)
+{
+	//TODO - for now just assume we've got some memory available
+	phys_alloc_init(freemem_start, 16 * SZ_1M);
+}
+
+static void banner(void)
+{
+	puts("\n");
+	puts("##########################################################################\n");
+	puts("#    kvm-unit-tests\n");
+	puts("##########################################################################\n");
+	puts("\n");
+}
+
 void setup(const void *fdt, phys_addr_t freemem_start)
 {
+	void *freemem;
+	const char *bootargs, *tmp;
+	u32 fdt_size;
+	int ret;
+
+	assert(sizeof(long) == 8 || freemem_start < (3ul << 30));
+	freemem = (void *)(unsigned long)freemem_start;
+
+	/* Move the FDT to the base of free memory */
+	fdt_size = fdt_totalsize(fdt);
+	ret = fdt_move(fdt, freemem, fdt_size);
+	assert(ret == 0);
+	ret = dt_init(freemem);
+	assert(ret == 0);
+	freemem += fdt_size;
+
+	/* Move the initrd to the top of the FDT */
+	ret = dt_get_initrd(&tmp, &initrd_size);
+	assert(ret == 0 || ret == -FDT_ERR_NOTFOUND);
+	if (ret == 0) {
+		initrd = freemem;
+		memmove(initrd, tmp, initrd_size);
+		freemem += initrd_size;
+	}
+
+	mem_init(PAGE_ALIGN((unsigned long)freemem));
+	cpu_init();
+	io_init();
+
+	ret = dt_get_bootargs(&bootargs);
+	assert(ret == 0 || ret == -FDT_ERR_NOTFOUND);
+	setup_args_progname(bootargs);
+
+	if (initrd) {
+		/* environ is currently the only file in the initrd */
+		char *env = malloc(initrd_size);
+		memcpy(env, initrd, initrd_size);
+		setup_env(env, initrd_size);
+	}
+
+	banner();
 }
diff --git a/lib/riscv/smp.c b/lib/riscv/smp.c
new file mode 100644
index 000000000000..a89b59d8dd03
--- /dev/null
+++ b/lib/riscv/smp.c
@@ -0,0 +1,6 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <cpumask.h>
+
+cpumask_t cpu_present_mask;
+cpumask_t cpu_online_mask;
+cpumask_t cpu_idle_mask;
diff --git a/riscv/Makefile b/riscv/Makefile
index f2e89f0e4c38..ddf2a0e016a8 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -20,8 +20,13 @@ $(TEST_DIR)/sieve.elf: AUXFLAGS = 0x1
 
 cstart.o = $(TEST_DIR)/cstart.o
 
+cflatobjs += lib/alloc.o
+cflatobjs += lib/alloc_phys.o
+cflatobjs += lib/devicetree.o
+cflatobjs += lib/riscv/bitops.o
 cflatobjs += lib/riscv/io.o
 cflatobjs += lib/riscv/setup.o
+cflatobjs += lib/riscv/smp.o
 
 ########################################
 
diff --git a/riscv/selftest.c b/riscv/selftest.c
index 88afa732649e..d3b269cf6255 100644
--- a/riscv/selftest.c
+++ b/riscv/selftest.c
@@ -5,9 +5,47 @@
  * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
  */
 #include <libcflat.h>
+#include <cpumask.h>
+#include <asm/setup.h>
 
-int main(void)
+static void check_cpus(void)
 {
-	puts("Hello, world\n");
-	return 0;
+	int cpu;
+
+	for_each_present_cpu(cpu)
+		report_info("CPU%3d: hartid=%08lx", cpu, cpus[cpu]);
+}
+
+int main(int argc, char **argv)
+{
+	bool r;
+
+	report_prefix_push("selftest");
+
+	report(!strncmp(argv[0], "selftest", 8), "program name");
+
+	if (argc > 1) {
+		r = !strcmp(argv[1], "foo");
+		if (argc > 2)
+			r &= !strcmp(argv[2], "bar");
+		if (argc > 3)
+			r &= !strcmp(argv[3], "baz");
+		report_info("matched %d command line parameters", argc - 1);
+		report(r, "command line parsing");
+	} else {
+		report_skip("command line parsing");
+	}
+
+	if (getenv("FOO")) {
+		r = !strcmp(getenv("FOO"), "foo");
+		r &= !strcmp(getenv("BAR"), "bar");
+		r &= !strcmp(getenv("BAZ"), "baz");
+		report(r, "environ parsing");
+	} else {
+		report_skip("environ parsing");
+	}
+
+	check_cpus();
+
+	return report_summary();
 }
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 06/24] riscv: Add initial SBI support
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (4 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 05/24] riscv: Add DT parsing Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 07/24] riscv: Add run script and unittests.cfg Andrew Jones
                   ` (18 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Add an SBI call function and immediately apply it to properly
exiting the test (instead of hanging) by invoking SBI shutdown
from exit(). Also seed an SBI test file with a simple SBI test
that checks mvendorid is correctly extracted.

Run with e.g.
  qemu-system-riscv64 -nographic -M virt \
      -kernel riscv/sbi.flat \
      -cpu rv64,mvendorid=45 \
      -initrd sbi-env

and be happy that ctrl-a c q is no longer necessary to return to
the shell prompt. sbi-env has MVENDORID=45 in it.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/riscv/asm/sbi.h | 32 ++++++++++++++++++++++++++++++++
 lib/riscv/io.c      |  2 ++
 lib/riscv/sbi.c     | 35 +++++++++++++++++++++++++++++++++++
 riscv/Makefile      |  2 ++
 riscv/sbi.c         | 41 +++++++++++++++++++++++++++++++++++++++++
 5 files changed, 112 insertions(+)
 create mode 100644 lib/riscv/asm/sbi.h
 create mode 100644 lib/riscv/sbi.c
 create mode 100644 riscv/sbi.c

diff --git a/lib/riscv/asm/sbi.h b/lib/riscv/asm/sbi.h
new file mode 100644
index 000000000000..aeff07f6f1a8
--- /dev/null
+++ b/lib/riscv/asm/sbi.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_SBI_H_
+#define _ASMRISCV_SBI_H_
+
+enum sbi_ext_id {
+	SBI_EXT_BASE = 0x10,
+	SBI_EXT_SRST = 0x53525354,
+};
+
+enum sbi_ext_base_fid {
+	SBI_EXT_BASE_GET_SPEC_VERSION = 0,
+	SBI_EXT_BASE_GET_IMP_ID,
+	SBI_EXT_BASE_GET_IMP_VERSION,
+	SBI_EXT_BASE_PROBE_EXT,
+	SBI_EXT_BASE_GET_MVENDORID,
+	SBI_EXT_BASE_GET_MARCHID,
+	SBI_EXT_BASE_GET_MIMPID,
+};
+
+struct sbiret {
+	long error;
+	long value;
+};
+
+struct sbiret sbi_ecall(int ext, int fid, unsigned long arg0,
+			unsigned long arg1, unsigned long arg2,
+			unsigned long arg3, unsigned long arg4,
+			unsigned long arg5);
+
+void sbi_shutdown(void);
+
+#endif /* _ASMRISCV_SBI_H_ */
diff --git a/lib/riscv/io.c b/lib/riscv/io.c
index aeda74be61ee..b3f587bb68ca 100644
--- a/lib/riscv/io.c
+++ b/lib/riscv/io.c
@@ -9,6 +9,7 @@
 #include <config.h>
 #include <devicetree.h>
 #include <asm/io.h>
+#include <asm/sbi.h>
 #include <asm/setup.h>
 #include <asm/spinlock.h>
 
@@ -90,6 +91,7 @@ void halt(int code);
 void exit(int code)
 {
 	printf("\nEXIT: STATUS=%d\n", ((code) << 1) | 1);
+	sbi_shutdown();
 	halt(code);
 	__builtin_unreachable();
 }
diff --git a/lib/riscv/sbi.c b/lib/riscv/sbi.c
new file mode 100644
index 000000000000..fd758555b888
--- /dev/null
+++ b/lib/riscv/sbi.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <libcflat.h>
+#include <asm/sbi.h>
+
+struct sbiret sbi_ecall(int ext, int fid, unsigned long arg0,
+			unsigned long arg1, unsigned long arg2,
+			unsigned long arg3, unsigned long arg4,
+			unsigned long arg5)
+{
+	register uintptr_t a0 asm ("a0") = (uintptr_t)(arg0);
+	register uintptr_t a1 asm ("a1") = (uintptr_t)(arg1);
+	register uintptr_t a2 asm ("a2") = (uintptr_t)(arg2);
+	register uintptr_t a3 asm ("a3") = (uintptr_t)(arg3);
+	register uintptr_t a4 asm ("a4") = (uintptr_t)(arg4);
+	register uintptr_t a5 asm ("a5") = (uintptr_t)(arg5);
+	register uintptr_t a6 asm ("a6") = (uintptr_t)(fid);
+	register uintptr_t a7 asm ("a7") = (uintptr_t)(ext);
+	struct sbiret ret;
+
+	asm volatile (
+		"ecall"
+		: "+r" (a0), "+r" (a1)
+		: "r" (a2), "r" (a3), "r" (a4), "r" (a5), "r" (a6), "r" (a7)
+		: "memory");
+	ret.error = a0;
+	ret.value = a1;
+
+	return ret;
+}
+
+void sbi_shutdown(void)
+{
+	sbi_ecall(SBI_EXT_SRST, 0, 0, 0, 0, 0, 0, 0);
+	puts("SBI shutdown failed!\n");
+}
diff --git a/riscv/Makefile b/riscv/Makefile
index ddf2a0e016a8..4e7fcc538ba1 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -11,6 +11,7 @@ exe = flat
 endif
 
 tests =
+tests += $(TEST_DIR)/sbi.$(exe)
 tests += $(TEST_DIR)/selftest.$(exe)
 #tests += $(TEST_DIR)/sieve.$(exe)
 
@@ -25,6 +26,7 @@ cflatobjs += lib/alloc_phys.o
 cflatobjs += lib/devicetree.o
 cflatobjs += lib/riscv/bitops.o
 cflatobjs += lib/riscv/io.o
+cflatobjs += lib/riscv/sbi.o
 cflatobjs += lib/riscv/setup.o
 cflatobjs += lib/riscv/smp.o
 
diff --git a/riscv/sbi.c b/riscv/sbi.c
new file mode 100644
index 000000000000..ffb07a256ff6
--- /dev/null
+++ b/riscv/sbi.c
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * SBI verification
+ *
+ * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
+ */
+#include <libcflat.h>
+#include <stdlib.h>
+#include <asm/sbi.h>
+
+static void help(void)
+{
+	puts("Test SBI\n");
+	puts("An environ must be provided where expected values are given.\n");
+}
+
+int main(int argc, char **argv)
+{
+	struct sbiret ret;
+	long expected;
+
+	if (argc > 1 && !strcmp(argv[1], "-h")) {
+		help();
+		exit(0);
+	}
+
+	report_prefix_push("sbi");
+
+	if (!getenv("MVENDORID")) {
+		report_skip("mvendorid: missing MVENDORID environment variable");
+		goto done;
+	}
+	expected = strtol(getenv("MVENDORID"), NULL, 0);
+
+	ret = sbi_ecall(SBI_EXT_BASE, SBI_EXT_BASE_GET_MVENDORID, 0, 0, 0, 0, 0, 0);
+	report(!ret.error, "mvendorid: no error");
+	report(ret.value == expected, "mvendorid");
+
+done:
+	return report_summary();
+}
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 07/24] riscv: Add run script and unittests.cfg
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (5 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 06/24] riscv: Add initial SBI support Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 08/24] riscv: Add riscv32 support Andrew Jones
                   ` (17 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

With a run script and a unittests.cfg file we can now run

 $ KVM_UNIT_TESTS_ENV=test-env ./run_tests.sh

where test-env has the environment variables needed for both tests.

Note, to change the SBI implementation under test, for example to
RustSBI, QEMU needs the -bios parameter. The full command line
would be

  qemu-system-riscv64 -nographic -M virt -cpu rv64 \
      -kernel riscv/sbi.flat \
      -bios $PATH_TO_RUSTSBI

and with the run script, it's

  SBI_IMPL="-bios $PATH_TO_RUSTSBI" ./run_tests.sh -g sbi

(note the '-g sbi' to only run the SBI test group)

Finally, with the addition of the run script, 'make standalone' now
also works.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 riscv/run           | 41 +++++++++++++++++++++++++++++++++++++++++
 riscv/unittests.cfg | 37 +++++++++++++++++++++++++++++++++++++
 2 files changed, 78 insertions(+)
 create mode 100755 riscv/run
 create mode 100644 riscv/unittests.cfg

diff --git a/riscv/run b/riscv/run
new file mode 100755
index 000000000000..cbe5dd792dcd
--- /dev/null
+++ b/riscv/run
@@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+
+if [ -z "$KUT_STANDALONE" ]; then
+	if [ ! -f config.mak ]; then
+		echo "run ./configure && make first. See ./configure -h"
+		exit 2
+	fi
+	source config.mak
+	source scripts/arch-run.bash
+fi
+
+# Allow user overrides of some config.mak variables
+processor=$PROCESSOR_OVERRIDE
+firmware=$FIRMWARE_OVERRIDE
+
+[ "$PROCESSOR" = "$ARCH" ] && PROCESSOR="max"
+: "${processor:=$PROCESSOR}"
+: "${firmware:=$FIRMWARE}"
+[ "$firmware" ] && firmware="-bios $firmware"
+
+set_qemu_accelerator || exit $?
+[ "$ACCEL" = "kvm" ] && QEMU_ARCH=$HOST
+acc="-accel $ACCEL$ACCEL_PROPS"
+
+qemu=$(search_qemu_binary) || exit $?
+if ! $qemu -machine '?' | grep -q 'RISC-V VirtIO board'; then
+	echo "$qemu doesn't support mach-virt ('-machine virt'). Exiting."
+	exit 2
+fi
+mach='-machine virt'
+
+command="$qemu -nodefaults -nographic -serial mon:stdio"
+command+=" $mach $acc $firmware -cpu $processor "
+command="$(migration_cmd) $(timeout_cmd) $command"
+
+if [ "$EFI_RUN" = "y" ]; then
+	ENVIRON_DEFAULT=n run_qemu_status $command "$@"
+else
+	# We return the exit code via stdout, not via the QEMU return code
+	run_qemu_status $command -kernel "$@"
+fi
diff --git a/riscv/unittests.cfg b/riscv/unittests.cfg
new file mode 100644
index 000000000000..5a23bed9cdd6
--- /dev/null
+++ b/riscv/unittests.cfg
@@ -0,0 +1,37 @@
+##############################################################################
+# unittest configuration
+#
+# [unittest_name]
+# file = <name>.flat		# Name of the flat file to be used.
+# smp  = <num>			# Number of processors the VM will use
+#				# during this test. Use $MAX_SMP to use
+#				# the maximum the host supports. Defaults
+#				# to one.
+# extra_params = -append <params...>	# Additional parameters used.
+# arch = riscv32|riscv64		# Select one if the test case is
+#					# specific to only one.
+# groups = <group_name1> <group_name2> ...	# Used to identify test cases
+#						# with run_tests -g ...
+#						# Specify group_name=nodefault
+#						# to have test not run by
+#						# default
+# accel = kvm|tcg		# Optionally specify if test must run with
+#				# kvm or tcg. If not specified, then kvm will
+#				# be used when available.
+# timeout = <duration>		# Optionally specify a timeout.
+# check = <path>=<value> # check a file for a particular value before running
+#                        # a test. The check line can contain multiple files
+#                        # to check separated by a space but each check
+#                        # parameter needs to be of the form <path>=<value>
+##############################################################################
+
+[selftest]
+file = selftest.flat
+smp = 16
+extra_params = -append 'foo bar baz'
+groups = selftest
+
+# Set $FIRMWARE_OVERRIDE to /path/to/firmware to select the SBI implementation.
+[sbi]
+file = sbi.flat
+groups = sbi
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 08/24] riscv: Add riscv32 support
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (6 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 07/24] riscv: Add run script and unittests.cfg Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-01 15:24   ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 09/24] riscv: Add exception handling Andrew Jones
                   ` (16 subsequent siblings)
  24 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Make a few tweaks to allow also building and running riscv32.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/elf.h               | 11 +++++++++++
 lib/ldiv32.c            | 16 ++++++++++++++++
 lib/riscv/asm-offsets.c | 11 +++++++++++
 riscv/Makefile          |  3 +++
 riscv/cstart.S          | 35 +++++++++++++++++++++--------------
 5 files changed, 62 insertions(+), 14 deletions(-)

diff --git a/lib/elf.h b/lib/elf.h
index abd5cf4beaad..7a7db57774cd 100644
--- a/lib/elf.h
+++ b/lib/elf.h
@@ -8,6 +8,11 @@
 
 #include <libcflat.h>
 
+/* 32-bit ELF base types. */
+typedef u32	Elf32_Addr;
+typedef u32	Elf32_Xword;
+typedef s32	Elf32_Sxword;
+
 /* 64-bit ELF base types. */
 typedef u64	Elf64_Addr;
 typedef u64	Elf64_Xword;
@@ -26,6 +31,12 @@ typedef struct elf64_rel {
 	Elf64_Xword r_info;     /* index and type of relocation */
 } Elf64_Rel;
 
+typedef struct elf32_rela {
+	Elf32_Addr r_offset;    /* Location at which to apply the action */
+	Elf32_Xword r_info;     /* index and type of relocation */
+	Elf32_Sxword r_addend;  /* Constant addend used to compute value */
+} Elf32_Rela;
+
 typedef struct elf64_rela {
 	Elf64_Addr r_offset;    /* Location at which to apply the action */
 	Elf64_Xword r_info;     /* index and type of relocation */
diff --git a/lib/ldiv32.c b/lib/ldiv32.c
index 897a4b9cd39e..9ce2a6a1faf0 100644
--- a/lib/ldiv32.c
+++ b/lib/ldiv32.c
@@ -1,5 +1,21 @@
 #include <stdint.h>
 
+#if __riscv_xlen == 32
+int __clzdi2(unsigned long);
+
+int __clzdi2(unsigned long a)
+{
+	int n = 0;
+
+	while (a) {
+		++n;
+		a >>= 1;
+	}
+
+	return 32 - n;
+}
+#endif
+
 extern uint64_t __udivmoddi4(uint64_t num, uint64_t den, uint64_t *p_rem);
 extern int64_t __divmoddi4(int64_t num, int64_t den, int64_t *p_rem);
 extern int64_t __moddi3(int64_t num, int64_t den);
diff --git a/lib/riscv/asm-offsets.c b/lib/riscv/asm-offsets.c
index 4a74df9e4a09..eb337b7547b8 100644
--- a/lib/riscv/asm-offsets.c
+++ b/lib/riscv/asm-offsets.c
@@ -1,6 +1,17 @@
 // SPDX-License-Identifier: GPL-2.0-only
+#include <kbuild.h>
+#include <elf.h>
 
 int main(void)
 {
+#if __riscv_xlen == 32
+	OFFSET(ELF_RELA_OFFSET, elf32_rela, r_offset);
+	OFFSET(ELF_RELA_ADDEND, elf32_rela, r_addend);
+	DEFINE(ELF_RELA_SIZE, sizeof(struct elf32_rela));
+#elif __riscv_xlen == 64
+	OFFSET(ELF_RELA_OFFSET, elf64_rela, r_offset);
+	OFFSET(ELF_RELA_ADDEND, elf64_rela, r_addend);
+	DEFINE(ELF_RELA_SIZE, sizeof(struct elf64_rela));
+#endif
 	return 0;
 }
diff --git a/riscv/Makefile b/riscv/Makefile
index 4e7fcc538ba1..fb97e678a456 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -29,6 +29,9 @@ cflatobjs += lib/riscv/io.o
 cflatobjs += lib/riscv/sbi.o
 cflatobjs += lib/riscv/setup.o
 cflatobjs += lib/riscv/smp.o
+ifeq ($(ARCH),riscv32)
+cflatobjs += lib/ldiv32.o
+endif
 
 ########################################
 
diff --git a/riscv/cstart.S b/riscv/cstart.S
index a28d75e8021e..6ec2231e5812 100644
--- a/riscv/cstart.S
+++ b/riscv/cstart.S
@@ -4,11 +4,23 @@
  *
  * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
  */
+#include <asm/asm-offsets.h>
 #include <asm/csr.h>
 
+#if __riscv_xlen == 64
+#define __REG_SEL(a, b) a
+#elif __riscv_xlen == 32
+#define __REG_SEL(a, b) b
+#else
+#error "Unexpected __riscv_xlen"
+#endif
+
+#define REG_L	__REG_SEL(ld, lw)
+#define REG_S	__REG_SEL(sd, sw)
+
 .macro zero_range, tmp1, tmp2
 9998:	beq	\tmp1, \tmp2, 9997f
-	sd	zero, 0(\tmp1)
+	REG_S	zero, 0(\tmp1)
 	addi	\tmp1, \tmp1, 8
 	j	9998b
 9997:
@@ -33,26 +45,20 @@ start:
 
 	/*
 	 * Update all R_RISCV_RELATIVE relocations using the table
-	 * of Elf64_Rela entries between reloc_start/end. The build
-	 * will not emit other relocation types.
-	 *
-	 * struct Elf64_Rela {
-	 * 	uint64_t r_offset;
-	 * 	uint64_t r_info;
-	 * 	int64_t  r_addend;
-	 * }
+	 * of Elf32_Rela/Elf64_Rela entries between reloc_start/end.
+	 * The build will not emit other relocation types.
 	 */
 	la	a1, reloc_start
 	la	a2, reloc_end
 	la	a3, start			// base
 1:
 	bge	a1, a2, 1f
-	ld	a4, 0(a1)			// r_offset
-	ld	a5, 16(a1)			// r_addend
+	REG_L	a4, ELF_RELA_OFFSET(a1)		// r_offset
+	REG_L	a5, ELF_RELA_ADDEND(a1)		// r_addend
 	add	a4, a3, a4			// addr = base + r_offset
 	add	a5, a3, a5			// val = base + r_addend
-	sd	a5, 0(a4)			// *addr = val
-	addi	a1, a1, 24
+	REG_S	a5, 0(a4)			// *addr = val
+	addi	a1, a1, ELF_RELA_SIZE
 	j	1b
 
 1:
@@ -72,11 +78,12 @@ start:
 
 	/* complete setup */
 	la	a1, stacktop			// a1 is the base of free memory
+	mv	a2, zero			// clear a2 for xlen=32
 	call	setup				// a0 is the addr of the dtb
 
 	/* run the test */
 	la	a0, __argc
-	ld	a0, 0(a0)
+	REG_L	a0, 0(a0)
 	la	a1, __argv
 	la	a2, __environ
 	call	main
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 09/24] riscv: Add exception handling
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (7 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 08/24] riscv: Add riscv32 support Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 10/24] riscv: Add backtrace support Andrew Jones
                   ` (15 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Steal more code from Linux to implement exception handling, but with
the same kvm-unit-tests API that Arm has. Also introduce struct
thread_info like Arm has in order to hold the handler pointers.
Finally, as usual, extend the selftest to make sure it all works.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/riscv/asm-offsets.c   |  38 +++++++++++++
 lib/riscv/asm/bug.h       |  20 +++++++
 lib/riscv/asm/csr.h       |  28 +++++++++
 lib/riscv/asm/processor.h |  28 +++++++++
 lib/riscv/asm/ptrace.h    |  46 +++++++++++++++
 lib/riscv/asm/setup.h     |   3 +-
 lib/riscv/processor.c     |  60 +++++++++++++++++++
 lib/riscv/setup.c         |   9 ++-
 riscv/Makefile            |   1 +
 riscv/cstart.S            | 117 +++++++++++++++++++++++++++++++++++++-
 riscv/selftest.c          |  20 ++++++-
 11 files changed, 362 insertions(+), 8 deletions(-)
 create mode 100644 lib/riscv/asm/bug.h
 create mode 100644 lib/riscv/asm/processor.h
 create mode 100644 lib/riscv/asm/ptrace.h
 create mode 100644 lib/riscv/processor.c

diff --git a/lib/riscv/asm-offsets.c b/lib/riscv/asm-offsets.c
index eb337b7547b8..7b88d16fd0e4 100644
--- a/lib/riscv/asm-offsets.c
+++ b/lib/riscv/asm-offsets.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0-only
 #include <kbuild.h>
 #include <elf.h>
+#include <asm/ptrace.h>
 
 int main(void)
 {
@@ -13,5 +14,42 @@ int main(void)
 	OFFSET(ELF_RELA_ADDEND, elf64_rela, r_addend);
 	DEFINE(ELF_RELA_SIZE, sizeof(struct elf64_rela));
 #endif
+	OFFSET(PT_EPC, pt_regs, epc);
+	OFFSET(PT_RA, pt_regs, ra);
+	OFFSET(PT_SP, pt_regs, sp);
+	OFFSET(PT_GP, pt_regs, gp);
+	OFFSET(PT_TP, pt_regs, tp);
+	OFFSET(PT_T0, pt_regs, t0);
+	OFFSET(PT_T1, pt_regs, t1);
+	OFFSET(PT_T2, pt_regs, t2);
+	OFFSET(PT_S0, pt_regs, s0);
+	OFFSET(PT_S1, pt_regs, s1);
+	OFFSET(PT_A0, pt_regs, a0);
+	OFFSET(PT_A1, pt_regs, a1);
+	OFFSET(PT_A2, pt_regs, a2);
+	OFFSET(PT_A3, pt_regs, a3);
+	OFFSET(PT_A4, pt_regs, a4);
+	OFFSET(PT_A5, pt_regs, a5);
+	OFFSET(PT_A6, pt_regs, a6);
+	OFFSET(PT_A7, pt_regs, a7);
+	OFFSET(PT_S2, pt_regs, s2);
+	OFFSET(PT_S3, pt_regs, s3);
+	OFFSET(PT_S4, pt_regs, s4);
+	OFFSET(PT_S5, pt_regs, s5);
+	OFFSET(PT_S6, pt_regs, s6);
+	OFFSET(PT_S7, pt_regs, s7);
+	OFFSET(PT_S8, pt_regs, s8);
+	OFFSET(PT_S9, pt_regs, s9);
+	OFFSET(PT_S10, pt_regs, s10);
+	OFFSET(PT_S11, pt_regs, s11);
+	OFFSET(PT_T3, pt_regs, t3);
+	OFFSET(PT_T4, pt_regs, t4);
+	OFFSET(PT_T5, pt_regs, t5);
+	OFFSET(PT_T6, pt_regs, t6);
+	OFFSET(PT_STATUS, pt_regs, status);
+	OFFSET(PT_BADADDR, pt_regs, badaddr);
+	OFFSET(PT_CAUSE, pt_regs, cause);
+	OFFSET(PT_ORIG_A0, pt_regs, orig_a0);
+	DEFINE(PT_SIZE, sizeof(struct pt_regs));
 	return 0;
 }
diff --git a/lib/riscv/asm/bug.h b/lib/riscv/asm/bug.h
new file mode 100644
index 000000000000..a6f4136ba1b6
--- /dev/null
+++ b/lib/riscv/asm/bug.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_BUG_H_
+#define _ASMRISCV_BUG_H_
+
+#ifndef __ASSEMBLY__
+
+static inline void bug(void)
+{
+	asm volatile("ebreak");
+}
+
+#else
+
+.macro bug
+	ebreak
+.endm
+
+#endif
+
+#endif /* _ASMRISCV_BUG_H_ */
diff --git a/lib/riscv/asm/csr.h b/lib/riscv/asm/csr.h
index 356ae054bfff..39ffd2a146be 100644
--- a/lib/riscv/asm/csr.h
+++ b/lib/riscv/asm/csr.h
@@ -3,7 +3,35 @@
 #define _ASMRISCV_CSR_H_
 #include <linux/const.h>
 
+#define CSR_SSTATUS		0x100
+#define CSR_STVEC		0x105
 #define CSR_SSCRATCH		0x140
+#define CSR_SEPC		0x141
+#define CSR_SCAUSE		0x142
+#define CSR_STVAL		0x143
+
+/* Exception cause high bit - is an interrupt if set */
+#define CAUSE_IRQ_FLAG		(_AC(1, UL) << (__riscv_xlen - 1))
+
+/* Exception causes */
+#define EXC_INST_MISALIGNED	0
+#define EXC_INST_ACCESS		1
+#define EXC_INST_ILLEGAL	2
+#define EXC_BREAKPOINT		3
+#define EXC_LOAD_MISALIGNED	4
+#define EXC_LOAD_ACCESS		5
+#define EXC_STORE_MISALIGNED	6
+#define EXC_STORE_ACCESS	7
+#define EXC_SYSCALL		8
+#define EXC_HYPERVISOR_SYSCALL	9
+#define EXC_SUPERVISOR_SYSCALL	10
+#define EXC_INST_PAGE_FAULT	12
+#define EXC_LOAD_PAGE_FAULT	13
+#define EXC_STORE_PAGE_FAULT	15
+#define EXC_INST_GUEST_PAGE_FAULT	20
+#define EXC_LOAD_GUEST_PAGE_FAULT	21
+#define EXC_VIRTUAL_INST_FAULT		22
+#define EXC_STORE_GUEST_PAGE_FAULT	23
 
 #ifndef __ASSEMBLY__
 
diff --git a/lib/riscv/asm/processor.h b/lib/riscv/asm/processor.h
new file mode 100644
index 000000000000..f20774d02d8e
--- /dev/null
+++ b/lib/riscv/asm/processor.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_PROCESSOR_H_
+#define _ASMRISCV_PROCESSOR_H_
+#include <asm/csr.h>
+#include <asm/ptrace.h>
+
+#define EXCEPTION_CAUSE_MAX	16
+
+typedef void (*exception_fn)(struct pt_regs *);
+
+struct thread_info {
+	int cpu;
+	unsigned long hartid;
+	exception_fn exception_handlers[EXCEPTION_CAUSE_MAX];
+};
+
+static inline struct thread_info *current_thread_info(void)
+{
+	return (struct thread_info *)csr_read(CSR_SSCRATCH);
+}
+
+void install_exception_handler(unsigned long cause, void (*handler)(struct pt_regs *));
+void do_handle_exception(struct pt_regs *regs);
+void thread_info_init(void);
+
+void show_regs(struct pt_regs *regs);
+
+#endif /* _ASMRISCV_PROCESSOR_H_ */
diff --git a/lib/riscv/asm/ptrace.h b/lib/riscv/asm/ptrace.h
new file mode 100644
index 000000000000..0873a8ae749f
--- /dev/null
+++ b/lib/riscv/asm/ptrace.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_PTRACE_H_
+#define _ASMRISCV_PTRACE_H_
+
+struct pt_regs {
+	unsigned long epc;
+	unsigned long ra;
+	unsigned long sp;
+	unsigned long gp;
+	unsigned long tp;
+	unsigned long t0;
+	unsigned long t1;
+	unsigned long t2;
+	unsigned long s0;
+	unsigned long s1;
+	unsigned long a0;
+	unsigned long a1;
+	unsigned long a2;
+	unsigned long a3;
+	unsigned long a4;
+	unsigned long a5;
+	unsigned long a6;
+	unsigned long a7;
+	unsigned long s2;
+	unsigned long s3;
+	unsigned long s4;
+	unsigned long s5;
+	unsigned long s6;
+	unsigned long s7;
+	unsigned long s8;
+	unsigned long s9;
+	unsigned long s10;
+	unsigned long s11;
+	unsigned long t3;
+	unsigned long t4;
+	unsigned long t5;
+	unsigned long t6;
+	/* Supervisor/Machine CSRs */
+	unsigned long status;
+	unsigned long badaddr;
+	unsigned long cause;
+	/* a0 value before the syscall */
+	unsigned long orig_a0;
+};
+
+#endif /* _ASMRISCV_PTRACE_H_ */
diff --git a/lib/riscv/asm/setup.h b/lib/riscv/asm/setup.h
index c8cfebb4f2c1..e58dd53071ae 100644
--- a/lib/riscv/asm/setup.h
+++ b/lib/riscv/asm/setup.h
@@ -2,9 +2,10 @@
 #ifndef _ASMRISCV_SETUP_H_
 #define _ASMRISCV_SETUP_H_
 #include <libcflat.h>
+#include <asm/processor.h>
 
 #define NR_CPUS 16
-extern unsigned long cpus[NR_CPUS];       /* per-cpu IDs (hartids) */
+extern struct thread_info cpus[NR_CPUS];
 extern int nr_cpus;
 int hartid_to_cpu(unsigned long hartid);
 
diff --git a/lib/riscv/processor.c b/lib/riscv/processor.c
new file mode 100644
index 000000000000..fafa0f864179
--- /dev/null
+++ b/lib/riscv/processor.c
@@ -0,0 +1,60 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
+ */
+#include <libcflat.h>
+#include <asm/csr.h>
+#include <asm/processor.h>
+#include <asm/setup.h>
+
+extern unsigned long _text;
+
+void show_regs(struct pt_regs *regs)
+{
+	uintptr_t text = (uintptr_t)&_text;
+	unsigned int w = __riscv_xlen / 4;
+
+	printf("Load address: %" PRIxPTR "\n", text);
+	printf("status : %.*lx\n", w, regs->status);
+	printf("cause  : %.*lx\n", w, regs->cause);
+	printf("badaddr: %.*lx\n", w, regs->badaddr);
+	printf("pc: %.*lx ra: %.*lx\n", w, regs->epc, w, regs->ra);
+	printf("sp: %.*lx gp: %.*lx tp : %.*lx\n", w, regs->sp, w, regs->gp, w, regs->tp);
+	printf("a0: %.*lx a1: %.*lx a2 : %.*lx a3 : %.*lx\n", w, regs->a0, w, regs->a1, w, regs->a2, w, regs->a3);
+	printf("a4: %.*lx a5: %.*lx a6 : %.*lx a7 : %.*lx\n", w, regs->a4, w, regs->a5, w, regs->a6, w, regs->a7);
+	printf("t0: %.*lx t1: %.*lx t2 : %.*lx t3 : %.*lx\n", w, regs->t0, w, regs->t1, w, regs->t2, w, regs->t3);
+	printf("t4: %.*lx t5: %.*lx t6 : %.*lx\n", w, regs->t4, w, regs->t5, w, regs->t6);
+	printf("s0: %.*lx s1: %.*lx s2 : %.*lx s3 : %.*lx\n", w, regs->s0, w, regs->s1, w, regs->s2, w, regs->s3);
+	printf("s4: %.*lx s5: %.*lx s6 : %.*lx s7 : %.*lx\n", w, regs->s4, w, regs->s5, w, regs->s6, w, regs->s7);
+	printf("s8: %.*lx s9: %.*lx s10: %.*lx s11: %.*lx\n", w, regs->s8, w, regs->s9, w, regs->s10, w, regs->s11);
+}
+
+void do_handle_exception(struct pt_regs *regs)
+{
+	struct thread_info *info = current_thread_info();
+
+	assert(regs->cause < EXCEPTION_CAUSE_MAX);
+	if (info->exception_handlers[regs->cause]) {
+		info->exception_handlers[regs->cause](regs);
+		return;
+	}
+
+	show_regs(regs);
+	assert(0);
+}
+
+void install_exception_handler(unsigned long cause, void (*handler)(struct pt_regs *))
+{
+	struct thread_info *info = current_thread_info();
+
+	assert(cause < EXCEPTION_CAUSE_MAX);
+	info->exception_handlers[cause] = handler;
+}
+
+void thread_info_init(void)
+{
+	unsigned long hartid = csr_read(CSR_SSCRATCH);
+	int cpu = hartid_to_cpu(hartid);
+
+	csr_write(CSR_SSCRATCH, &cpus[cpu]);
+}
diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c
index 44c26b125a27..57eb4797f798 100644
--- a/lib/riscv/setup.c
+++ b/lib/riscv/setup.c
@@ -12,12 +12,13 @@
 #include <devicetree.h>
 #include <asm/csr.h>
 #include <asm/page.h>
+#include <asm/processor.h>
 #include <asm/setup.h>
 
 char *initrd;
 u32 initrd_size;
 
-unsigned long cpus[NR_CPUS] = { [0 ... NR_CPUS - 1] = ~0UL };
+struct thread_info cpus[NR_CPUS];
 int nr_cpus;
 
 int hartid_to_cpu(unsigned long hartid)
@@ -25,7 +26,7 @@ int hartid_to_cpu(unsigned long hartid)
 	int cpu;
 
 	for_each_present_cpu(cpu)
-		if (cpus[cpu] == hartid)
+		if (cpus[cpu].hartid == hartid)
 			return cpu;
 	return -1;
 }
@@ -36,7 +37,8 @@ static void cpu_set_fdt(int fdtnode __unused, u64 regval, void *info __unused)
 
 	assert_msg(cpu < NR_CPUS, "Number cpus exceeds maximum supported (%d).", NR_CPUS);
 
-	cpus[cpu] = regval;
+	cpus[cpu].cpu = cpu;
+	cpus[cpu].hartid = regval;
 	set_cpu_present(cpu, true);
 }
 
@@ -104,6 +106,7 @@ void setup(const void *fdt, phys_addr_t freemem_start)
 
 	mem_init(PAGE_ALIGN((unsigned long)freemem));
 	cpu_init();
+	thread_info_init();
 	io_init();
 
 	ret = dt_get_bootargs(&bootargs);
diff --git a/riscv/Makefile b/riscv/Makefile
index fb97e678a456..1243be125c00 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -26,6 +26,7 @@ cflatobjs += lib/alloc_phys.o
 cflatobjs += lib/devicetree.o
 cflatobjs += lib/riscv/bitops.o
 cflatobjs += lib/riscv/io.o
+cflatobjs += lib/riscv/processor.o
 cflatobjs += lib/riscv/sbi.o
 cflatobjs += lib/riscv/setup.o
 cflatobjs += lib/riscv/smp.o
diff --git a/riscv/cstart.S b/riscv/cstart.S
index 6ec2231e5812..b3842d667309 100644
--- a/riscv/cstart.S
+++ b/riscv/cstart.S
@@ -37,8 +37,9 @@
 .global start
 start:
 	/*
-	 * Stash the hartid in scratch and shift the dtb
-	 * address into a0
+	 * Stash the hartid in scratch and shift the dtb address into a0.
+	 * thread_info_init() will later promote scratch to point at thread
+	 * local storage.
 	 */
 	csrw	CSR_SSCRATCH, a0
 	mv	a0, a1
@@ -74,7 +75,8 @@ start:
 	zero_range a1, sp
 
 	/* set up exception handling */
-	//TODO
+	la	a1, exception_vectors
+	csrw	CSR_STVEC, a1
 
 	/* complete setup */
 	la	a1, stacktop			// a1 is the base of free memory
@@ -97,3 +99,112 @@ start:
 halt:
 1:	wfi
 	j	1b
+
+/*
+ * Save context to address in a0.
+ * For a0, sets PT_A0(a0) to the contents of PT_ORIG_A0(a0).
+ * Clobbers a1.
+ */
+.macro save_context
+	REG_S	ra, PT_RA(a0)			// x1
+	REG_S	sp, PT_SP(a0)			// x2
+	REG_S	gp, PT_GP(a0)			// x3
+	REG_S	tp, PT_TP(a0)			// x4
+	REG_S	t0, PT_T0(a0)			// x5
+	REG_S	t1, PT_T1(a0)			// x6
+	REG_S	t2, PT_T2(a0)			// x7
+	REG_S	s0, PT_S0(a0)			// x8 / fp
+	REG_S	s1, PT_S1(a0)			// x9
+	/* a0 */				// x10
+	REG_S   a1, PT_A1(a0)			// x11
+	REG_S	a2, PT_A2(a0)			// x12
+	REG_S	a3, PT_A3(a0)			// x13
+	REG_S	a4, PT_A4(a0)			// x14
+	REG_S	a5, PT_A5(a0)			// x15
+	REG_S	a6, PT_A6(a0)			// x16
+	REG_S	a7, PT_A7(a0)			// x17
+	REG_S	s2, PT_S2(a0)			// x18
+	REG_S	s3, PT_S3(a0)			// x19
+	REG_S	s4, PT_S4(a0)			// x20
+	REG_S	s5, PT_S5(a0)			// x21
+	REG_S	s6, PT_S6(a0)			// x22
+	REG_S	s7, PT_S7(a0)			// x23
+	REG_S	s8, PT_S8(a0)			// x24
+	REG_S	s9, PT_S9(a0)			// x25
+	REG_S	s10, PT_S10(a0)			// x26
+	REG_S	s11, PT_S11(a0)			// x27
+	REG_S	t3, PT_T3(a0)			// x28
+	REG_S	t4, PT_T4(a0)			// x29
+	REG_S	t5, PT_T5(a0)			// x30
+	REG_S	t6, PT_T6(a0)			// x31
+	csrr	a1, CSR_SEPC
+	REG_S	a1, PT_EPC(a0)
+	csrr	a1, CSR_SSTATUS
+	REG_S	a1, PT_STATUS(a0)
+	csrr	a1, CSR_STVAL
+	REG_S	a1, PT_BADADDR(a0)
+	csrr	a1, CSR_SCAUSE
+	REG_S	a1, PT_CAUSE(a0)
+	REG_L	a1, PT_ORIG_A0(a0)
+	REG_S	a1, PT_A0(a0)
+.endm
+
+/*
+ * Restore context from address in a0.
+ * Also restores a0.
+ */
+.macro restore_context
+	REG_L	ra, PT_RA(a0)			// x1
+	REG_L	sp, PT_SP(a0)			// x2
+	REG_L	gp, PT_GP(a0)			// x3
+	REG_L	tp, PT_TP(a0)			// x4
+	REG_L	t0, PT_T0(a0)			// x5
+	REG_L	t1, PT_T1(a0)			// x6
+	REG_L	t2, PT_T2(a0)			// x7
+	REG_L	s0, PT_S0(a0)			// x8 / fp
+	REG_L	s1, PT_S1(a0)			// x9
+	/* a0 */				// x10
+	/* a1 */				// x11
+	REG_L	a2, PT_A2(a0)			// x12
+	REG_L	a3, PT_A3(a0)			// x13
+	REG_L	a4, PT_A4(a0)			// x14
+	REG_L	a5, PT_A5(a0)			// x15
+	REG_L	a6, PT_A6(a0)			// x16
+	REG_L	a7, PT_A7(a0)			// x17
+	REG_L	s2, PT_S2(a0)			// x18
+	REG_L	s3, PT_S3(a0)			// x19
+	REG_L	s4, PT_S4(a0)			// x20
+	REG_L	s5, PT_S5(a0)			// x21
+	REG_L	s6, PT_S6(a0)			// x22
+	REG_L	s7, PT_S7(a0)			// x23
+	REG_L	s8, PT_S8(a0)			// x24
+	REG_L	s9, PT_S9(a0)			// x25
+	REG_L	s10, PT_S10(a0)			// x26
+	REG_L	s11, PT_S11(a0)			// x27
+	REG_L	t3, PT_T3(a0)			// x28
+	REG_L	t4, PT_T4(a0)			// x29
+	REG_L	t5, PT_T5(a0)			// x30
+	REG_L	t6, PT_T6(a0)			// x31
+	REG_L	a1, PT_EPC(a0)
+	csrw	CSR_SEPC, a1
+	REG_L	a1, PT_STATUS(a0)
+	csrw	CSR_SSTATUS, a1
+	REG_L	a1, PT_BADADDR(a0)
+	csrw	CSR_STVAL, a1
+	REG_L	a1, PT_CAUSE(a0)
+	csrw	CSR_SCAUSE, a1
+	REG_L	a1, PT_A1(a0)
+	REG_L	a0, PT_A0(a0)
+.endm
+
+.balign 4
+.global exception_vectors
+exception_vectors:
+	REG_S	a0, (-PT_SIZE + PT_ORIG_A0)(sp)
+	addi	a0, sp, -PT_SIZE
+	save_context
+	mv	sp, a0
+	call	do_handle_exception
+	mv	a0, sp
+	restore_context
+	sret
diff --git a/riscv/selftest.c b/riscv/selftest.c
index d3b269cf6255..219093489b62 100644
--- a/riscv/selftest.c
+++ b/riscv/selftest.c
@@ -6,6 +6,7 @@
  */
 #include <libcflat.h>
 #include <cpumask.h>
+#include <asm/processor.h>
 #include <asm/setup.h>
 
 static void check_cpus(void)
@@ -13,7 +14,23 @@ static void check_cpus(void)
 	int cpu;
 
 	for_each_present_cpu(cpu)
-		report_info("CPU%3d: hartid=%08lx", cpu, cpus[cpu]);
+		report_info("CPU%3d: hartid=%08lx", cpu, cpus[cpu].hartid);
+}
+
+static bool exceptions_work;
+
+static void handler(struct pt_regs *regs)
+{
+	exceptions_work = true;
+	regs->epc += 2;
+}
+
+static void check_exceptions(void)
+{
+	install_exception_handler(EXC_INST_ILLEGAL, handler);
+	asm volatile(".4byte 0");
+	install_exception_handler(EXC_INST_ILLEGAL, NULL);
+	report(exceptions_work, "exceptions");
 }
 
 int main(int argc, char **argv)
@@ -45,6 +62,7 @@ int main(int argc, char **argv)
 		report_skip("environ parsing");
 	}
 
+	check_exceptions();
 	check_cpus();
 
 	return report_summary();
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 10/24] riscv: Add backtrace support
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (8 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 09/24] riscv: Add exception handling Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 11/24] arm/arm64: Generalize wfe/sev names in smp.c Andrew Jones
                   ` (14 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Enable stack unwinding, even when going through an exception, by
implementing backtrace() and pushing a frame pointer on the stack
in exception_vectors.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/riscv/asm/stack.h |  3 +++
 lib/riscv/stack.c     | 32 ++++++++++++++++++++++++++++++++
 riscv/Makefile        |  1 +
 riscv/cstart.S        | 28 ++++++++++++++++++++++++++--
 4 files changed, 62 insertions(+), 2 deletions(-)
 create mode 100644 lib/riscv/stack.c

diff --git a/lib/riscv/asm/stack.h b/lib/riscv/asm/stack.h
index d081d0716d7b..f003ca37c913 100644
--- a/lib/riscv/asm/stack.h
+++ b/lib/riscv/asm/stack.h
@@ -6,4 +6,7 @@
 #error Do not directly include <asm/stack.h>. Just use <stack.h>.
 #endif
 
+#define HAVE_ARCH_BACKTRACE_FRAME
+#define HAVE_ARCH_BACKTRACE
+
 #endif
diff --git a/lib/riscv/stack.c b/lib/riscv/stack.c
new file mode 100644
index 000000000000..712a5478d547
--- /dev/null
+++ b/lib/riscv/stack.c
@@ -0,0 +1,32 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <libcflat.h>
+#include <stack.h>
+
+int backtrace_frame(const void *frame, const void **return_addrs, int max_depth)
+{
+	static bool walking;
+	const unsigned long *fp = (unsigned long *)frame;
+	int depth;
+
+	if (walking) {
+		printf("RECURSIVE STACK WALK!!!\n");
+		return 0;
+	}
+	walking = true;
+
+	for (depth = 0; fp && depth < max_depth; ++depth) {
+		return_addrs[depth] = (void *)fp[-1];
+		if (return_addrs[depth] == 0)
+			break;
+		fp = (unsigned long *)fp[-2];
+	}
+
+	walking = false;
+	return depth;
+}
+
+int backtrace(const void **return_addrs, int max_depth)
+{
+	return backtrace_frame(__builtin_frame_address(0),
+			       return_addrs, max_depth);
+}
diff --git a/riscv/Makefile b/riscv/Makefile
index 1243be125c00..4a83f27f7df2 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -30,6 +30,7 @@ cflatobjs += lib/riscv/processor.o
 cflatobjs += lib/riscv/sbi.o
 cflatobjs += lib/riscv/setup.o
 cflatobjs += lib/riscv/smp.o
+cflatobjs += lib/riscv/stack.o
 ifeq ($(ARCH),riscv32)
 cflatobjs += lib/ldiv32.o
 endif
diff --git a/riscv/cstart.S b/riscv/cstart.S
index b3842d667309..2066e37d1ef6 100644
--- a/riscv/cstart.S
+++ b/riscv/cstart.S
@@ -17,6 +17,22 @@
 
 #define REG_L	__REG_SEL(ld, lw)
 #define REG_S	__REG_SEL(sd, sw)
+#define SZREG	__REG_SEL(8, 4)
+
+#define FP_SIZE 16
+
+.macro push_fp, ra=ra
+	addi	sp, sp, -FP_SIZE
+	REG_S	\ra, (FP_SIZE - SZREG)(sp)
+	REG_S	fp, (FP_SIZE - 2*SZREG)(sp)
+	addi	fp, sp, FP_SIZE
+.endm
+
+.macro pop_fp
+	REG_L	ra, (FP_SIZE - SZREG)(sp)
+	REG_L	fp, (FP_SIZE - 2*SZREG)(sp)
+	addi	sp, sp, FP_SIZE
+.endm
 
 .macro zero_range, tmp1, tmp2
 9998:	beq	\tmp1, \tmp2, 9997f
@@ -73,6 +89,7 @@ start:
 	li	a1, -8192
 	add	a1, sp, a1
 	zero_range a1, sp
+	mv	fp, zero			// Ensure fp starts out as zero
 
 	/* set up exception handling */
 	la	a1, exception_vectors
@@ -200,9 +217,16 @@ halt:
 .balign 4
 .global exception_vectors
 exception_vectors:
-	REG_S	a0, (-PT_SIZE + PT_ORIG_A0)(sp)
-	addi	a0, sp, -PT_SIZE
+	REG_S	a0, (-PT_SIZE - FP_SIZE + PT_ORIG_A0)(sp)
+	addi	a0, sp, -PT_SIZE - FP_SIZE
 	save_context
+	/*
+	 * Set a frame pointer "ra" which points to the last instruction.
+	 * Add 1 to it, because pretty_print_stacks.py subtracts 1.
+	 */
+	REG_L	a1, PT_EPC(a0)
+	addi	a1, a1, 1
+	push_fp	a1
 	mv	sp, a0
 	call	do_handle_exception
 	mv	a0, sp
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 11/24] arm/arm64: Generalize wfe/sev names in smp.c
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (9 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 10/24] riscv: Add backtrace support Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-01  9:22   ` Eric Auger
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 12/24] arm/arm64: Remove spinlocks from on_cpu_async Andrew Jones
                   ` (13 subsequent siblings)
  24 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Most of Arm's on_cpus() implementation can be shared by any
architecture which has the possible, present, and idle cpumasks,
like riscv does. Rename the exceptions (wfe/sve) to something
more generic in order to prepare to share the functions.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/arm/asm/smp.h |  4 ++++
 lib/arm/smp.c     | 16 ++++++++--------
 2 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/lib/arm/asm/smp.h b/lib/arm/asm/smp.h
index b89a68dd344f..9f6d839ab568 100644
--- a/lib/arm/asm/smp.h
+++ b/lib/arm/asm/smp.h
@@ -6,6 +6,7 @@
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
 #include <cpumask.h>
+#include <asm/barrier.h>
 #include <asm/thread_info.h>
 
 #define smp_processor_id()		(current_thread_info()->cpu)
@@ -18,6 +19,9 @@ struct secondary_data {
 };
 extern struct secondary_data secondary_data;
 
+#define smp_wait_for_event()	wfe()
+#define smp_send_event()	sev()
+
 extern bool cpu0_calls_idle;
 
 extern void halt(void);
diff --git a/lib/arm/smp.c b/lib/arm/smp.c
index 78fc1656cefa..c00fda2efb03 100644
--- a/lib/arm/smp.c
+++ b/lib/arm/smp.c
@@ -45,7 +45,7 @@ secondary_entry_fn secondary_cinit(void)
 	 */
 	entry = secondary_data.entry;
 	set_cpu_online(ti->cpu, true);
-	sev();
+	smp_send_event();
 
 	/*
 	 * Return to the assembly stub, allowing entry to be called
@@ -65,7 +65,7 @@ static void __smp_boot_secondary(int cpu, secondary_entry_fn entry)
 	assert(ret == 0);
 
 	while (!cpu_online(cpu))
-		wfe();
+		smp_wait_for_event();
 }
 
 void smp_boot_secondary(int cpu, secondary_entry_fn entry)
@@ -122,7 +122,7 @@ static void cpu_wait(int cpu)
 	cpumask_set_cpu(me, &on_cpu_info[cpu].waiters);
 	deadlock_check(me, cpu);
 	while (!cpu_idle(cpu))
-		wfe();
+		smp_wait_for_event();
 	cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
 }
 
@@ -134,17 +134,17 @@ void do_idle(void)
 		cpu0_calls_idle = true;
 
 	set_cpu_idle(cpu, true);
-	sev();
+	smp_send_event();
 
 	for (;;) {
 		while (cpu_idle(cpu))
-			wfe();
+			smp_wait_for_event();
 		smp_rmb();
 		on_cpu_info[cpu].func(on_cpu_info[cpu].data);
 		on_cpu_info[cpu].func = NULL;
 		smp_wmb();
 		set_cpu_idle(cpu, true);
-		sev();
+		smp_send_event();
 	}
 }
 
@@ -174,7 +174,7 @@ void on_cpu_async(int cpu, void (*func)(void *data), void *data)
 	on_cpu_info[cpu].data = data;
 	spin_unlock(&lock);
 	set_cpu_idle(cpu, false);
-	sev();
+	smp_send_event();
 }
 
 void on_cpu(int cpu, void (*func)(void *data), void *data)
@@ -201,7 +201,7 @@ void on_cpus(void (*func)(void *data), void *data)
 		deadlock_check(me, cpu);
 	}
 	while (cpumask_weight(&cpu_idle_mask) < nr_cpus - 1)
-		wfe();
+		smp_wait_for_event();
 	for_each_present_cpu(cpu)
 		cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
 }
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 12/24] arm/arm64: Remove spinlocks from on_cpu_async
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (10 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 11/24] arm/arm64: Generalize wfe/sev names in smp.c Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-01  9:34   ` Eric Auger
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 13/24] arm/arm64: Share on_cpus Andrew Jones
                   ` (12 subsequent siblings)
  24 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Remove spinlocks from on_cpu_async() by pulling some of their
use into a new function and also by narrowing the locking to a
single on_cpu_info structure by introducing yet another cpumask.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/arm/asm/smp.h |  4 +++-
 lib/arm/smp.c     | 37 ++++++++++++++++++++++++++++---------
 2 files changed, 31 insertions(+), 10 deletions(-)

diff --git a/lib/arm/asm/smp.h b/lib/arm/asm/smp.h
index 9f6d839ab568..f0c0f97a19f8 100644
--- a/lib/arm/asm/smp.h
+++ b/lib/arm/asm/smp.h
@@ -27,9 +27,11 @@ extern bool cpu0_calls_idle;
 extern void halt(void);
 extern void do_idle(void);
 
-extern void smp_boot_secondary(int cpu, secondary_entry_fn entry);
 extern void on_cpu_async(int cpu, void (*func)(void *data), void *data);
 extern void on_cpu(int cpu, void (*func)(void *data), void *data);
 extern void on_cpus(void (*func)(void *data), void *data);
 
+extern void smp_boot_secondary(int cpu, secondary_entry_fn entry);
+extern void smp_boot_secondary_nofail(int cpu, secondary_entry_fn entry);
+
 #endif /* _ASMARM_SMP_H_ */
diff --git a/lib/arm/smp.c b/lib/arm/smp.c
index c00fda2efb03..e0872a1a72c2 100644
--- a/lib/arm/smp.c
+++ b/lib/arm/smp.c
@@ -76,12 +76,32 @@ void smp_boot_secondary(int cpu, secondary_entry_fn entry)
 	spin_unlock(&lock);
 }
 
+void smp_boot_secondary_nofail(int cpu, secondary_entry_fn entry)
+{
+	spin_lock(&lock);
+	if (!cpu_online(cpu))
+		__smp_boot_secondary(cpu, entry);
+	spin_unlock(&lock);
+}
+
 struct on_cpu_info {
 	void (*func)(void *data);
 	void *data;
 	cpumask_t waiters;
 };
 static struct on_cpu_info on_cpu_info[NR_CPUS];
+static cpumask_t on_cpu_info_lock;
+
+static bool get_on_cpu_info(int cpu)
+{
+	return !cpumask_test_and_set_cpu(cpu, &on_cpu_info_lock);
+}
+
+static void put_on_cpu_info(int cpu)
+{
+	int ret = cpumask_test_and_clear_cpu(cpu, &on_cpu_info_lock);
+	assert(ret);
+}
 
 static void __deadlock_check(int cpu, const cpumask_t *waiters, bool *found)
 {
@@ -158,22 +178,21 @@ void on_cpu_async(int cpu, void (*func)(void *data), void *data)
 	assert_msg(cpu != 0 || cpu0_calls_idle, "Waiting on CPU0, which is unlikely to idle. "
 						"If this is intended set cpu0_calls_idle=1");
 
-	spin_lock(&lock);
-	if (!cpu_online(cpu))
-		__smp_boot_secondary(cpu, do_idle);
-	spin_unlock(&lock);
+	smp_boot_secondary_nofail(cpu, do_idle);
 
 	for (;;) {
 		cpu_wait(cpu);
-		spin_lock(&lock);
-		if ((volatile void *)on_cpu_info[cpu].func == NULL)
-			break;
-		spin_unlock(&lock);
+		if (get_on_cpu_info(cpu)) {
+			if ((volatile void *)on_cpu_info[cpu].func == NULL)
+				break;
+			put_on_cpu_info(cpu);
+		}
 	}
+
 	on_cpu_info[cpu].func = func;
 	on_cpu_info[cpu].data = data;
-	spin_unlock(&lock);
 	set_cpu_idle(cpu, false);
+	put_on_cpu_info(cpu);
 	smp_send_event();
 }
 
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 13/24] arm/arm64: Share on_cpus
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (11 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 12/24] arm/arm64: Remove spinlocks from on_cpu_async Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-01  9:36   ` Eric Auger
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 14/24] riscv: Compile with march Andrew Jones
                   ` (11 subsequent siblings)
  24 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Now that the previous patches have cleaned up Arm's on_cpus
implementation we can move it to the common lib where riscv
will share it.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 arm/Makefile.common |   1 +
 lib/arm/asm/smp.h   |   8 +--
 lib/arm/smp.c       | 144 -----------------------------------------
 lib/on-cpus.c       | 154 ++++++++++++++++++++++++++++++++++++++++++++
 lib/on-cpus.h       |  14 ++++
 5 files changed, 170 insertions(+), 151 deletions(-)
 create mode 100644 lib/on-cpus.c
 create mode 100644 lib/on-cpus.h

diff --git a/arm/Makefile.common b/arm/Makefile.common
index 5214c8acdab3..dc92a7433350 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -43,6 +43,7 @@ cflatobjs += lib/vmalloc.o
 cflatobjs += lib/alloc.o
 cflatobjs += lib/devicetree.o
 cflatobjs += lib/migrate.o
+cflatobjs += lib/on-cpus.o
 cflatobjs += lib/pci.o
 cflatobjs += lib/pci-host-generic.o
 cflatobjs += lib/pci-testdev.o
diff --git a/lib/arm/asm/smp.h b/lib/arm/asm/smp.h
index f0c0f97a19f8..2e1dc27f7bd8 100644
--- a/lib/arm/asm/smp.h
+++ b/lib/arm/asm/smp.h
@@ -6,6 +6,7 @@
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
 #include <cpumask.h>
+#include <on-cpus.h>
 #include <asm/barrier.h>
 #include <asm/thread_info.h>
 
@@ -22,14 +23,7 @@ extern struct secondary_data secondary_data;
 #define smp_wait_for_event()	wfe()
 #define smp_send_event()	sev()
 
-extern bool cpu0_calls_idle;
-
 extern void halt(void);
-extern void do_idle(void);
-
-extern void on_cpu_async(int cpu, void (*func)(void *data), void *data);
-extern void on_cpu(int cpu, void (*func)(void *data), void *data);
-extern void on_cpus(void (*func)(void *data), void *data);
 
 extern void smp_boot_secondary(int cpu, secondary_entry_fn entry);
 extern void smp_boot_secondary_nofail(int cpu, secondary_entry_fn entry);
diff --git a/lib/arm/smp.c b/lib/arm/smp.c
index e0872a1a72c2..0207ca2a7d57 100644
--- a/lib/arm/smp.c
+++ b/lib/arm/smp.c
@@ -10,13 +10,10 @@
 #include <cpumask.h>
 #include <asm/thread_info.h>
 #include <asm/spinlock.h>
-#include <asm/barrier.h>
 #include <asm/mmu.h>
 #include <asm/psci.h>
 #include <asm/smp.h>
 
-bool cpu0_calls_idle;
-
 cpumask_t cpu_present_mask;
 cpumask_t cpu_online_mask;
 cpumask_t cpu_idle_mask;
@@ -83,144 +80,3 @@ void smp_boot_secondary_nofail(int cpu, secondary_entry_fn entry)
 		__smp_boot_secondary(cpu, entry);
 	spin_unlock(&lock);
 }
-
-struct on_cpu_info {
-	void (*func)(void *data);
-	void *data;
-	cpumask_t waiters;
-};
-static struct on_cpu_info on_cpu_info[NR_CPUS];
-static cpumask_t on_cpu_info_lock;
-
-static bool get_on_cpu_info(int cpu)
-{
-	return !cpumask_test_and_set_cpu(cpu, &on_cpu_info_lock);
-}
-
-static void put_on_cpu_info(int cpu)
-{
-	int ret = cpumask_test_and_clear_cpu(cpu, &on_cpu_info_lock);
-	assert(ret);
-}
-
-static void __deadlock_check(int cpu, const cpumask_t *waiters, bool *found)
-{
-	int i;
-
-	for_each_cpu(i, waiters) {
-		if (i == cpu) {
-			printf("CPU%d", cpu);
-			*found = true;
-			return;
-		}
-		__deadlock_check(cpu, &on_cpu_info[i].waiters, found);
-		if (*found) {
-			printf(" <=> CPU%d", i);
-			return;
-		}
-	}
-}
-
-static void deadlock_check(int me, int cpu)
-{
-	bool found = false;
-
-	__deadlock_check(cpu, &on_cpu_info[me].waiters, &found);
-	if (found) {
-		printf(" <=> CPU%d deadlock detectd\n", me);
-		assert(0);
-	}
-}
-
-static void cpu_wait(int cpu)
-{
-	int me = smp_processor_id();
-
-	if (cpu == me)
-		return;
-
-	cpumask_set_cpu(me, &on_cpu_info[cpu].waiters);
-	deadlock_check(me, cpu);
-	while (!cpu_idle(cpu))
-		smp_wait_for_event();
-	cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
-}
-
-void do_idle(void)
-{
-	int cpu = smp_processor_id();
-
-	if (cpu == 0)
-		cpu0_calls_idle = true;
-
-	set_cpu_idle(cpu, true);
-	smp_send_event();
-
-	for (;;) {
-		while (cpu_idle(cpu))
-			smp_wait_for_event();
-		smp_rmb();
-		on_cpu_info[cpu].func(on_cpu_info[cpu].data);
-		on_cpu_info[cpu].func = NULL;
-		smp_wmb();
-		set_cpu_idle(cpu, true);
-		smp_send_event();
-	}
-}
-
-void on_cpu_async(int cpu, void (*func)(void *data), void *data)
-{
-	if (cpu == smp_processor_id()) {
-		func(data);
-		return;
-	}
-
-	assert_msg(cpu != 0 || cpu0_calls_idle, "Waiting on CPU0, which is unlikely to idle. "
-						"If this is intended set cpu0_calls_idle=1");
-
-	smp_boot_secondary_nofail(cpu, do_idle);
-
-	for (;;) {
-		cpu_wait(cpu);
-		if (get_on_cpu_info(cpu)) {
-			if ((volatile void *)on_cpu_info[cpu].func == NULL)
-				break;
-			put_on_cpu_info(cpu);
-		}
-	}
-
-	on_cpu_info[cpu].func = func;
-	on_cpu_info[cpu].data = data;
-	set_cpu_idle(cpu, false);
-	put_on_cpu_info(cpu);
-	smp_send_event();
-}
-
-void on_cpu(int cpu, void (*func)(void *data), void *data)
-{
-	on_cpu_async(cpu, func, data);
-	cpu_wait(cpu);
-}
-
-void on_cpus(void (*func)(void *data), void *data)
-{
-	int cpu, me = smp_processor_id();
-
-	for_each_present_cpu(cpu) {
-		if (cpu == me)
-			continue;
-		on_cpu_async(cpu, func, data);
-	}
-	func(data);
-
-	for_each_present_cpu(cpu) {
-		if (cpu == me)
-			continue;
-		cpumask_set_cpu(me, &on_cpu_info[cpu].waiters);
-		deadlock_check(me, cpu);
-	}
-	while (cpumask_weight(&cpu_idle_mask) < nr_cpus - 1)
-		smp_wait_for_event();
-	for_each_present_cpu(cpu)
-		cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
-}
diff --git a/lib/on-cpus.c b/lib/on-cpus.c
new file mode 100644
index 000000000000..aed70f7b27b2
--- /dev/null
+++ b/lib/on-cpus.c
@@ -0,0 +1,154 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * on_cpus() support based on cpumasks.
+ *
+ * Copyright (C) 2015, Red Hat Inc, Andrew Jones <drjones@redhat.com>
+ */
+#include <libcflat.h>
+#include <cpumask.h>
+#include <on-cpus.h>
+#include <asm/barrier.h>
+#include <asm/smp.h>
+
+bool cpu0_calls_idle;
+
+struct on_cpu_info {
+	void (*func)(void *data);
+	void *data;
+	cpumask_t waiters;
+};
+static struct on_cpu_info on_cpu_info[NR_CPUS];
+static cpumask_t on_cpu_info_lock;
+
+static bool get_on_cpu_info(int cpu)
+{
+	return !cpumask_test_and_set_cpu(cpu, &on_cpu_info_lock);
+}
+
+static void put_on_cpu_info(int cpu)
+{
+	int ret = cpumask_test_and_clear_cpu(cpu, &on_cpu_info_lock);
+	assert(ret);
+}
+
+static void __deadlock_check(int cpu, const cpumask_t *waiters, bool *found)
+{
+	int i;
+
+	for_each_cpu(i, waiters) {
+		if (i == cpu) {
+			printf("CPU%d", cpu);
+			*found = true;
+			return;
+		}
+		__deadlock_check(cpu, &on_cpu_info[i].waiters, found);
+		if (*found) {
+			printf(" <=> CPU%d", i);
+			return;
+		}
+	}
+}
+
+static void deadlock_check(int me, int cpu)
+{
+	bool found = false;
+
+	__deadlock_check(cpu, &on_cpu_info[me].waiters, &found);
+	if (found) {
+		printf(" <=> CPU%d deadlock detectd\n", me);
+		assert(0);
+	}
+}
+
+static void cpu_wait(int cpu)
+{
+	int me = smp_processor_id();
+
+	if (cpu == me)
+		return;
+
+	cpumask_set_cpu(me, &on_cpu_info[cpu].waiters);
+	deadlock_check(me, cpu);
+	while (!cpu_idle(cpu))
+		smp_wait_for_event();
+	cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
+}
+
+void do_idle(void)
+{
+	int cpu = smp_processor_id();
+
+	if (cpu == 0)
+		cpu0_calls_idle = true;
+
+	set_cpu_idle(cpu, true);
+	smp_send_event();
+
+	for (;;) {
+		while (cpu_idle(cpu))
+			smp_wait_for_event();
+		smp_rmb();
+		on_cpu_info[cpu].func(on_cpu_info[cpu].data);
+		on_cpu_info[cpu].func = NULL;
+		smp_wmb();
+		set_cpu_idle(cpu, true);
+		smp_send_event();
+	}
+}
+
+void on_cpu_async(int cpu, void (*func)(void *data), void *data)
+{
+	if (cpu == smp_processor_id()) {
+		func(data);
+		return;
+	}
+
+	assert_msg(cpu != 0 || cpu0_calls_idle, "Waiting on CPU0, which is unlikely to idle. "
+						"If this is intended set cpu0_calls_idle=1");
+
+	smp_boot_secondary_nofail(cpu, do_idle);
+
+	for (;;) {
+		cpu_wait(cpu);
+		if (get_on_cpu_info(cpu)) {
+			if ((volatile void *)on_cpu_info[cpu].func == NULL)
+				break;
+			put_on_cpu_info(cpu);
+		}
+	}
+
+	on_cpu_info[cpu].func = func;
+	on_cpu_info[cpu].data = data;
+	set_cpu_idle(cpu, false);
+	put_on_cpu_info(cpu);
+	smp_send_event();
+}
+
+void on_cpu(int cpu, void (*func)(void *data), void *data)
+{
+	on_cpu_async(cpu, func, data);
+	cpu_wait(cpu);
+}
+
+void on_cpus(void (*func)(void *data), void *data)
+{
+	int cpu, me = smp_processor_id();
+
+	for_each_present_cpu(cpu) {
+		if (cpu == me)
+			continue;
+		on_cpu_async(cpu, func, data);
+	}
+	func(data);
+
+	for_each_present_cpu(cpu) {
+		if (cpu == me)
+			continue;
+		cpumask_set_cpu(me, &on_cpu_info[cpu].waiters);
+		deadlock_check(me, cpu);
+	}
+	while (cpumask_weight(&cpu_idle_mask) < nr_cpus - 1)
+		smp_wait_for_event();
+	for_each_present_cpu(cpu)
+		cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
+}
diff --git a/lib/on-cpus.h b/lib/on-cpus.h
new file mode 100644
index 000000000000..41103b0245c7
--- /dev/null
+++ b/lib/on-cpus.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ON_CPUS_H_
+#define _ON_CPUS_H_
+#include <stdbool.h>
+
+extern bool cpu0_calls_idle;
+
+void do_idle(void);
+
+void on_cpu_async(int cpu, void (*func)(void *data), void *data);
+void on_cpu(int cpu, void (*func)(void *data), void *data);
+void on_cpus(void (*func)(void *data), void *data);
+
+#endif /* _ON_CPUS_H_ */
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 14/24] riscv: Compile with march
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (12 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 13/24] arm/arm64: Share on_cpus Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 15/24] riscv: Add SMP support Andrew Jones
                   ` (10 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Tell the compiler to provide mnemonics for instructions we depend on,
such as 'pause'. Specifying march also allows extensions which affect
compilation to be individually [un]selected. For example, building
without compressed (2 byte) instructions may be desirable, so 'c' may
be removed from the march isa string.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 riscv/Makefile | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/riscv/Makefile b/riscv/Makefile
index 4a83f27f7df2..697a3beb2703 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -55,8 +55,13 @@ define arch_elf_check =
 		$(error $(1) has unsupported reloc types))
 endef
 
+ISA_COMMON = mafdc_zicsr_zifencei_zihintpause
+
 ifeq ($(ARCH),riscv64)
+CFLAGS += -march=rv64i$(ISA_COMMON)
 CFLAGS += -DCONFIG_64BIT
+else ifeq ($(ARCH),riscv32)
+CFLAGS += -march=rv32i$(ISA_COMMON)
 endif
 CFLAGS += -DCONFIG_RELOC
 CFLAGS += -mcmodel=medany
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 15/24] riscv: Add SMP support
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (13 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 14/24] riscv: Compile with march Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 16/24] arm/arm64: Share memregions Andrew Jones
                   ` (9 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Implement the same SMP API that Arm has but using an SBI HSM
call instead of PSCI. Unlike Arm, riscv needs to always set
cpu0_calls_idle, because the boot hart doesn't have to be the
first hart described in the DT, which means cpu0 may well be
a secondary. As usual, add a couple tests to selftest.c to
make sure everything works.

(The secondary boot process is also improved over Arm's a bit
by keeping boot data percpu, dropping the need for a lock. We
could create percpu data for Arm too, but that's left as future
work.)

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/riscv/asm-offsets.c |  6 ++++
 lib/riscv/asm/barrier.h |  2 ++
 lib/riscv/asm/sbi.h     | 22 +++++++++++++++
 lib/riscv/asm/smp.h     | 28 +++++++++++++++++++
 lib/riscv/processor.c   |  2 ++
 lib/riscv/sbi.c         |  5 ++++
 lib/riscv/setup.c       |  2 ++
 lib/riscv/smp.c         | 61 +++++++++++++++++++++++++++++++++++++++++
 riscv/Makefile          |  1 +
 riscv/cstart.S          | 25 +++++++++++++++++
 riscv/selftest.c        | 31 +++++++++++++++++++++
 11 files changed, 185 insertions(+)
 create mode 100644 lib/riscv/asm/smp.h

diff --git a/lib/riscv/asm-offsets.c b/lib/riscv/asm-offsets.c
index 7b88d16fd0e4..f5beeeb45e09 100644
--- a/lib/riscv/asm-offsets.c
+++ b/lib/riscv/asm-offsets.c
@@ -2,6 +2,7 @@
 #include <kbuild.h>
 #include <elf.h>
 #include <asm/ptrace.h>
+#include <asm/smp.h>
 
 int main(void)
 {
@@ -51,5 +52,10 @@ int main(void)
 	OFFSET(PT_CAUSE, pt_regs, cause);
 	OFFSET(PT_ORIG_A0, pt_regs, orig_a0);
 	DEFINE(PT_SIZE, sizeof(struct pt_regs));
+
+	OFFSET(SECONDARY_STVEC, secondary_data, stvec);
+	OFFSET(SECONDARY_FUNC, secondary_data, func);
+	DEFINE(SECONDARY_DATA_SIZE, sizeof(struct secondary_data));
+
 	return 0;
 }
diff --git a/lib/riscv/asm/barrier.h b/lib/riscv/asm/barrier.h
index 6036d66af76f..4fef120a0fe8 100644
--- a/lib/riscv/asm/barrier.h
+++ b/lib/riscv/asm/barrier.h
@@ -15,4 +15,6 @@
 #define smp_rmb()	RISCV_FENCE(r,r)
 #define smp_wmb()	RISCV_FENCE(w,w)
 
+#define cpu_relax()	__asm__ __volatile__ ("pause")
+
 #endif /* _ASMRISCV_BARRIER_H_ */
diff --git a/lib/riscv/asm/sbi.h b/lib/riscv/asm/sbi.h
index aeff07f6f1a8..d82a384da5ce 100644
--- a/lib/riscv/asm/sbi.h
+++ b/lib/riscv/asm/sbi.h
@@ -2,8 +2,21 @@
 #ifndef _ASMRISCV_SBI_H_
 #define _ASMRISCV_SBI_H_
 
+#define SBI_SUCCESS			0
+#define SBI_ERR_FAILURE			-1
+#define SBI_ERR_NOT_SUPPORTED		-2
+#define SBI_ERR_INVALID_PARAM		-3
+#define SBI_ERR_DENIED			-4
+#define SBI_ERR_INVALID_ADDRESS		-5
+#define SBI_ERR_ALREADY_AVAILABLE	-6
+#define SBI_ERR_ALREADY_STARTED		-7
+#define SBI_ERR_ALREADY_STOPPED		-8
+
+#ifndef __ASSEMBLY__
+
 enum sbi_ext_id {
 	SBI_EXT_BASE = 0x10,
+	SBI_EXT_HSM = 0x48534d,
 	SBI_EXT_SRST = 0x53525354,
 };
 
@@ -17,6 +30,13 @@ enum sbi_ext_base_fid {
 	SBI_EXT_BASE_GET_MIMPID,
 };
 
+enum sbi_ext_hsm_fid {
+	SBI_EXT_HSM_HART_START = 0,
+	SBI_EXT_HSM_HART_STOP,
+	SBI_EXT_HSM_HART_STATUS,
+	SBI_EXT_HSM_HART_SUSPEND,
+};
+
 struct sbiret {
 	long error;
 	long value;
@@ -28,5 +48,7 @@ struct sbiret sbi_ecall(int ext, int fid, unsigned long arg0,
 			unsigned long arg5);
 
 void sbi_shutdown(void);
+struct sbiret sbi_hart_start(unsigned long hartid, unsigned long entry, unsigned long sp);
 
+#endif /* !__ASSEMBLY__ */
 #endif /* _ASMRISCV_SBI_H_ */
diff --git a/lib/riscv/asm/smp.h b/lib/riscv/asm/smp.h
new file mode 100644
index 000000000000..931766dc3969
--- /dev/null
+++ b/lib/riscv/asm/smp.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_SMP_H_
+#define _ASMRISCV_SMP_H_
+#include <asm/barrier.h>
+#include <asm/processor.h>
+
+#define smp_wait_for_event()	cpu_relax()
+#define smp_send_event()	cpu_relax()
+
+static inline int smp_processor_id(void)
+{
+	return current_thread_info()->cpu;
+}
+
+typedef void (*secondary_func_t)(void);
+
+struct secondary_data {
+	unsigned long stvec;
+	secondary_func_t func;
+} __attribute__((aligned(16)));
+
+void secondary_entry(unsigned long hartid, unsigned long sp_phys);
+secondary_func_t secondary_cinit(struct secondary_data *data);
+
+void smp_boot_secondary(int cpu, void (*func)(void));
+void smp_boot_secondary_nofail(int cpu, void (*func)(void));
+
+#endif /* _ASMRISCV_SMP_H_ */
diff --git a/lib/riscv/processor.c b/lib/riscv/processor.c
index fafa0f864179..2bfbd4e9b274 100644
--- a/lib/riscv/processor.c
+++ b/lib/riscv/processor.c
@@ -11,10 +11,12 @@ extern unsigned long _text;
 
 void show_regs(struct pt_regs *regs)
 {
+	struct thread_info *info = current_thread_info();
 	uintptr_t text = (uintptr_t)&_text;
 	unsigned int w = __riscv_xlen / 4;
 
 	printf("Load address: %" PRIxPTR "\n", text);
+	printf("CPU%3d : hartid=%lx\n", info->cpu, info->hartid);
 	printf("status : %.*lx\n", w, regs->status);
 	printf("cause  : %.*lx\n", w, regs->cause);
 	printf("badaddr: %.*lx\n", w, regs->badaddr);
diff --git a/lib/riscv/sbi.c b/lib/riscv/sbi.c
index fd758555b888..f39134c4d77e 100644
--- a/lib/riscv/sbi.c
+++ b/lib/riscv/sbi.c
@@ -33,3 +33,8 @@ void sbi_shutdown(void)
 	sbi_ecall(SBI_EXT_SRST, 0, 0, 0, 0, 0, 0, 0);
 	puts("SBI shutdown failed!\n");
 }
+
+struct sbiret sbi_hart_start(unsigned long hartid, unsigned long entry, unsigned long sp)
+{
+	return sbi_ecall(SBI_EXT_HSM, SBI_EXT_HSM_HART_START, hartid, entry, sp, 0, 0, 0);
+}
diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c
index 57eb4797f798..9ff446b5e171 100644
--- a/lib/riscv/setup.c
+++ b/lib/riscv/setup.c
@@ -10,6 +10,7 @@
 #include <argv.h>
 #include <cpumask.h>
 #include <devicetree.h>
+#include <on-cpus.h>
 #include <asm/csr.h>
 #include <asm/page.h>
 #include <asm/processor.h>
@@ -60,6 +61,7 @@ static void cpu_init(void)
 	}
 
 	set_cpu_online(hartid_to_cpu(csr_read(CSR_SSCRATCH)), true);
+	cpu0_calls_idle = true;
 }
 
 static void mem_init(phys_addr_t freemem_start)
diff --git a/lib/riscv/smp.c b/lib/riscv/smp.c
index a89b59d8dd03..ed7984e75608 100644
--- a/lib/riscv/smp.c
+++ b/lib/riscv/smp.c
@@ -1,6 +1,67 @@
 // SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Boot secondary CPUs
+ *
+ * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
+ */
+#include <libcflat.h>
+#include <alloc.h>
 #include <cpumask.h>
+#include <asm/csr.h>
+#include <asm/page.h>
+#include <asm/processor.h>
+#include <asm/sbi.h>
+#include <asm/smp.h>
 
 cpumask_t cpu_present_mask;
 cpumask_t cpu_online_mask;
 cpumask_t cpu_idle_mask;
+
+static cpumask_t cpu_started;
+
+secondary_func_t secondary_cinit(struct secondary_data *data)
+{
+	struct thread_info *info;
+
+	thread_info_init();
+	info = current_thread_info();
+	set_cpu_online(info->cpu, true);
+	smp_send_event();
+
+	return data->func;
+}
+
+static void __smp_boot_secondary(int cpu, secondary_func_t func)
+{
+	struct secondary_data *sp = memalign(16, SZ_8K) + SZ_8K - 16;
+	struct sbiret ret;
+
+	sp -= sizeof(struct secondary_data);
+	sp->stvec = csr_read(CSR_STVEC);
+	sp->func = func;
+
+	ret = sbi_hart_start(cpus[cpu].hartid, (unsigned long)&secondary_entry, __pa(sp));
+	assert(ret.error == SBI_SUCCESS);
+}
+
+void smp_boot_secondary(int cpu, void (*func)(void))
+{
+	int ret = cpumask_test_and_set_cpu(cpu, &cpu_started);
+
+	assert_msg(!ret, "CPU%d already boot once", cpu);
+	__smp_boot_secondary(cpu, func);
+
+	while (!cpu_online(cpu))
+		smp_wait_for_event();
+}
+
+void smp_boot_secondary_nofail(int cpu, void (*func)(void))
+{
+	int ret = cpumask_test_and_set_cpu(cpu, &cpu_started);
+
+	if (!ret)
+		__smp_boot_secondary(cpu, func);
+
+	while (!cpu_online(cpu))
+		smp_wait_for_event();
+}
diff --git a/riscv/Makefile b/riscv/Makefile
index 697a3beb2703..932f3378264c 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -24,6 +24,7 @@ cstart.o = $(TEST_DIR)/cstart.o
 cflatobjs += lib/alloc.o
 cflatobjs += lib/alloc_phys.o
 cflatobjs += lib/devicetree.o
+cflatobjs += lib/on-cpus.o
 cflatobjs += lib/riscv/bitops.o
 cflatobjs += lib/riscv/io.o
 cflatobjs += lib/riscv/processor.o
diff --git a/riscv/cstart.S b/riscv/cstart.S
index 2066e37d1ef6..c935467ff6a1 100644
--- a/riscv/cstart.S
+++ b/riscv/cstart.S
@@ -117,6 +117,31 @@ halt:
 1:	wfi
 	j	1b
 
+.balign 4
+.global secondary_entry
+secondary_entry:
+	/*
+	 * From the "HSM Hart Start Register State" table of the SBI spec:
+	 *	satp		0
+	 *	sstatus.SIE	0
+	 *	a0		hartid
+	 *	a1		opaque parameter
+	 *
+	 * __smp_boot_secondary() sets the opaque parameter (a1) to the physical
+	 * address of the stack and the stack contains the secondary data.
+	 */
+	csrw	CSR_SSCRATCH, a0
+	mv	sp, a1
+	mv	fp, zero
+	REG_L	a0, SECONDARY_STVEC(sp)
+	csrw	CSR_STVEC, a0
+	mv	a0, sp
+	call	secondary_cinit
+	addi	sp, sp, SECONDARY_DATA_SIZE
+	jr	a0
+	la	a0, do_idle
+	jr	a0
+
 /*
  * Save context to address in a0.
  * For a0, sets PT_A0(a0) to the contents of PT_ORIG_A0(a0).
diff --git a/riscv/selftest.c b/riscv/selftest.c
index 219093489b62..da13c622dba7 100644
--- a/riscv/selftest.c
+++ b/riscv/selftest.c
@@ -6,8 +6,10 @@
  */
 #include <libcflat.h>
 #include <cpumask.h>
+#include <on-cpus.h>
 #include <asm/processor.h>
 #include <asm/setup.h>
+#include <asm/smp.h>
 
 static void check_cpus(void)
 {
@@ -33,6 +35,34 @@ static void check_exceptions(void)
 	report(exceptions_work, "exceptions");
 }
 
+static cpumask_t cpus_alive;
+
+static void check_secondary(void *data)
+{
+	cpumask_set_cpu(smp_processor_id(), &cpus_alive);
+}
+
+static void check_smp(void)
+{
+	int cpu, me = smp_processor_id();
+	bool fail = false;
+
+	on_cpus(check_secondary, NULL);
+
+	report(cpumask_full(&cpu_online_mask), "Brought up all cpus");
+	report(cpumask_full(&cpus_alive), "check_secondary");
+
+	for_each_present_cpu(cpu) {
+		if (cpu == me)
+			continue;
+		if (!cpu_idle(cpu)) {
+			fail = true;
+			break;
+		}
+	}
+	report(!fail, "All secondaries are idle");
+}
+
 int main(int argc, char **argv)
 {
 	bool r;
@@ -64,6 +94,7 @@ int main(int argc, char **argv)
 
 	check_exceptions();
 	check_cpus();
+	check_smp();
 
 	return report_summary();
 }
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 16/24] arm/arm64: Share memregions
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (14 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 15/24] riscv: Add SMP support Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-01 12:03   ` Eric Auger
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 17/24] riscv: Populate memregions and switch to page allocator Andrew Jones
                   ` (8 subsequent siblings)
  24 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

arm/arm64 (and to a small extent powerpc) have memory regions which
get built from hardware descriptions (DT/ACPI/EFI) and then used to
build page tables. Move memregions to common code, tweaking the API
a bit at the same time, e.g. change 'mem_region' to 'memregions'.
The biggest change is there is now a default number of memory regions
which, if too small, should be overridden at setup time with a new
init function, memregions_init().

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 arm/Makefile.common |  1 +
 arm/selftest.c      |  3 +-
 lib/arm/asm/setup.h | 14 -------
 lib/arm/mmu.c       |  1 +
 lib/arm/setup.c     | 93 ++++++++++-----------------------------------
 lib/memregions.c    | 82 +++++++++++++++++++++++++++++++++++++++
 lib/memregions.h    | 29 ++++++++++++++
 7 files changed, 136 insertions(+), 87 deletions(-)
 create mode 100644 lib/memregions.c
 create mode 100644 lib/memregions.h

diff --git a/arm/Makefile.common b/arm/Makefile.common
index dc92a7433350..4dfd570fa59e 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -42,6 +42,7 @@ cflatobjs += lib/alloc_page.o
 cflatobjs += lib/vmalloc.o
 cflatobjs += lib/alloc.o
 cflatobjs += lib/devicetree.o
+cflatobjs += lib/memregions.o
 cflatobjs += lib/migrate.o
 cflatobjs += lib/on-cpus.o
 cflatobjs += lib/pci.o
diff --git a/arm/selftest.c b/arm/selftest.c
index 9f459ed3d571..007d2309d01c 100644
--- a/arm/selftest.c
+++ b/arm/selftest.c
@@ -8,6 +8,7 @@
 #include <libcflat.h>
 #include <util.h>
 #include <devicetree.h>
+#include <memregions.h>
 #include <vmalloc.h>
 #include <asm/setup.h>
 #include <asm/ptrace.h>
@@ -90,7 +91,7 @@ static bool check_pabt_init(void)
 			highest_end = PAGE_ALIGN(r->end);
 	}
 
-	if (mem_region_get_flags(highest_end) != MR_F_UNKNOWN)
+	if (memregions_get_flags(highest_end) != MR_F_UNKNOWN)
 		return false;
 
 	vaddr = (unsigned long)vmap(highest_end, PAGE_SIZE);
diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
index 060691165a20..9f8ef82efb90 100644
--- a/lib/arm/asm/setup.h
+++ b/lib/arm/asm/setup.h
@@ -13,22 +13,8 @@
 extern u64 cpus[NR_CPUS];	/* per-cpu IDs (MPIDRs) */
 extern int nr_cpus;
 
-#define MR_F_IO			(1U << 0)
-#define MR_F_CODE		(1U << 1)
-#define MR_F_RESERVED		(1U << 2)
-#define MR_F_UNKNOWN		(1U << 31)
-
-struct mem_region {
-	phys_addr_t start;
-	phys_addr_t end;
-	unsigned int flags;
-};
-extern struct mem_region *mem_regions;
 extern phys_addr_t __phys_offset, __phys_end;
 
-extern struct mem_region *mem_region_find(phys_addr_t paddr);
-extern unsigned int mem_region_get_flags(phys_addr_t paddr);
-
 #define PHYS_OFFSET		(__phys_offset)
 #define PHYS_END		(__phys_end)
 
diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
index b16517a3200d..eb5e82a95f06 100644
--- a/lib/arm/mmu.c
+++ b/lib/arm/mmu.c
@@ -6,6 +6,7 @@
  * This work is licensed under the terms of the GNU LGPL, version 2.
  */
 #include <cpumask.h>
+#include <memregions.h>
 #include <asm/setup.h>
 #include <asm/thread_info.h>
 #include <asm/mmu.h>
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index b6fc453e5b31..0382cbdaf5a1 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -13,6 +13,7 @@
 #include <libcflat.h>
 #include <libfdt/libfdt.h>
 #include <devicetree.h>
+#include <memregions.h>
 #include <alloc.h>
 #include <alloc_phys.h>
 #include <alloc_page.h>
@@ -31,7 +32,7 @@
 
 #define MAX_DT_MEM_REGIONS	16
 #define NR_EXTRA_MEM_REGIONS	64
-#define NR_INITIAL_MEM_REGIONS	(MAX_DT_MEM_REGIONS + NR_EXTRA_MEM_REGIONS)
+#define NR_MEM_REGIONS		(MAX_DT_MEM_REGIONS + NR_EXTRA_MEM_REGIONS)
 
 extern unsigned long _text, _etext, _data, _edata;
 
@@ -41,8 +42,7 @@ u32 initrd_size;
 u64 cpus[NR_CPUS] = { [0 ... NR_CPUS-1] = (u64)~0 };
 int nr_cpus;
 
-static struct mem_region __initial_mem_regions[NR_INITIAL_MEM_REGIONS + 1];
-struct mem_region *mem_regions = __initial_mem_regions;
+static struct mem_region arm_mem_regions[NR_MEM_REGIONS + 1];
 phys_addr_t __phys_offset = (phys_addr_t)-1, __phys_end = 0;
 
 extern void exceptions_init(void);
@@ -114,68 +114,14 @@ static void cpu_init(void)
 	set_cpu_online(0, true);
 }
 
-static void mem_region_add(struct mem_region *r)
+static void arm_memregions_add_assumed(void)
 {
-	struct mem_region *r_next = mem_regions;
-	int i = 0;
-
-	for (; r_next->end; ++r_next, ++i)
-		;
-	assert(i < NR_INITIAL_MEM_REGIONS);
-
-	*r_next = *r;
-}
-
-static void mem_regions_add_dt_regions(void)
-{
-	struct dt_pbus_reg regs[MAX_DT_MEM_REGIONS];
-	int nr_regs, i;
-
-	nr_regs = dt_get_memory_params(regs, MAX_DT_MEM_REGIONS);
-	assert(nr_regs > 0);
-
-	for (i = 0; i < nr_regs; ++i) {
-		mem_region_add(&(struct mem_region){
-			.start = regs[i].addr,
-			.end = regs[i].addr + regs[i].size,
-		});
-	}
-}
-
-struct mem_region *mem_region_find(phys_addr_t paddr)
-{
-	struct mem_region *r;
-
-	for (r = mem_regions; r->end; ++r)
-		if (paddr >= r->start && paddr < r->end)
-			return r;
-	return NULL;
-}
-
-unsigned int mem_region_get_flags(phys_addr_t paddr)
-{
-	struct mem_region *r = mem_region_find(paddr);
-	return r ? r->flags : MR_F_UNKNOWN;
-}
-
-static void mem_regions_add_assumed(void)
-{
-	phys_addr_t code_end = (phys_addr_t)(unsigned long)&_etext;
-	struct mem_region *r;
-
-	r = mem_region_find(code_end - 1);
-	assert(r);
+	struct mem_region *code, *data;
 
 	/* Split the region with the code into two regions; code and data */
-	mem_region_add(&(struct mem_region){
-		.start = code_end,
-		.end = r->end,
-	});
-	*r = (struct mem_region){
-		.start = r->start,
-		.end = code_end,
-		.flags = MR_F_CODE,
-	};
+	memregions_split((unsigned long)&_etext, &code, &data);
+	assert(code);
+	code->flags |= MR_F_CODE;
 
 	/*
 	 * mach-virt I/O regions:
@@ -183,10 +129,10 @@ static void mem_regions_add_assumed(void)
 	 *   - 512M at 256G (arm64, arm uses highmem=off)
 	 *   - 512G at 512G (arm64, arm uses highmem=off)
 	 */
-	mem_region_add(&(struct mem_region){ 0, (1ul << 30), MR_F_IO });
+	memregions_add(&(struct mem_region){ 0, (1ul << 30), MR_F_IO });
 #ifdef __aarch64__
-	mem_region_add(&(struct mem_region){ (1ul << 38), (1ul << 38) | (1ul << 29), MR_F_IO });
-	mem_region_add(&(struct mem_region){ (1ul << 39), (1ul << 40), MR_F_IO });
+	memregions_add(&(struct mem_region){ (1ul << 38), (1ul << 38) | (1ul << 29), MR_F_IO });
+	memregions_add(&(struct mem_region){ (1ul << 39), (1ul << 40), MR_F_IO });
 #endif
 }
 
@@ -197,7 +143,7 @@ static void mem_init(phys_addr_t freemem_start)
 		.start = (phys_addr_t)-1,
 	};
 
-	freemem = mem_region_find(freemem_start);
+	freemem = memregions_find(freemem_start);
 	assert(freemem && !(freemem->flags & (MR_F_IO | MR_F_CODE)));
 
 	for (r = mem_regions; r->end; ++r) {
@@ -212,9 +158,9 @@ static void mem_init(phys_addr_t freemem_start)
 	mem.end &= PHYS_MASK;
 
 	/* Check for holes */
-	r = mem_region_find(mem.start);
+	r = memregions_find(mem.start);
 	while (r && r->end != mem.end)
-		r = mem_region_find(r->end);
+		r = memregions_find(r->end);
 	assert(r);
 
 	/* Ensure our selected freemem range is somewhere in our full range */
@@ -263,8 +209,9 @@ void setup(const void *fdt, phys_addr_t freemem_start)
 		freemem += initrd_size;
 	}
 
-	mem_regions_add_dt_regions();
-	mem_regions_add_assumed();
+	memregions_init(arm_mem_regions, NR_MEM_REGIONS);
+	memregions_add_dt_regions(MAX_DT_MEM_REGIONS);
+	arm_memregions_add_assumed();
 	mem_init(PAGE_ALIGN((unsigned long)freemem));
 
 	psci_set_conduit();
@@ -371,7 +318,7 @@ static efi_status_t efi_mem_init(efi_bootinfo_t *efi_bootinfo)
 				assert(edata <= r.end);
 				r.flags = MR_F_CODE;
 				r.end = data;
-				mem_region_add(&r);
+				memregions_add(&r);
 				r.start = data;
 				r.end = tmp;
 				r.flags = 0;
@@ -393,7 +340,7 @@ static efi_status_t efi_mem_init(efi_bootinfo_t *efi_bootinfo)
 			if (r.end > __phys_end)
 				__phys_end = r.end;
 		}
-		mem_region_add(&r);
+		memregions_add(&r);
 	}
 	if (fdt) {
 		/* Move the FDT to the base of free memory */
@@ -439,6 +386,8 @@ efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo)
 
 	exceptions_init();
 
+	memregions_init(arm_mem_regions, NR_MEM_REGIONS);
+
 	status = efi_mem_init(efi_bootinfo);
 	if (status != EFI_SUCCESS) {
 		printf("Failed to initialize memory: ");
diff --git a/lib/memregions.c b/lib/memregions.c
new file mode 100644
index 000000000000..96de86b27333
--- /dev/null
+++ b/lib/memregions.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <libcflat.h>
+#include <devicetree.h>
+#include <memregions.h>
+
+static struct mem_region __initial_mem_regions[NR_INITIAL_MEM_REGIONS + 1];
+static size_t nr_regions = NR_INITIAL_MEM_REGIONS;
+
+struct mem_region *mem_regions = __initial_mem_regions;
+
+void memregions_init(struct mem_region regions[], size_t nr)
+{
+	mem_regions = regions;
+	nr_regions = nr;
+}
+
+struct mem_region *memregions_add(struct mem_region *r)
+{
+	struct mem_region *r_next = mem_regions;
+	int i = 0;
+
+	for (; r_next->end; ++r_next, ++i)
+		;
+	assert(i < nr_regions);
+
+	*r_next = *r;
+
+	return r_next;
+}
+
+struct mem_region *memregions_find(phys_addr_t paddr)
+{
+	struct mem_region *r;
+
+	for (r = mem_regions; r->end; ++r)
+		if (paddr >= r->start && paddr < r->end)
+			return r;
+	return NULL;
+}
+
+uint32_t memregions_get_flags(phys_addr_t paddr)
+{
+	struct mem_region *r = memregions_find(paddr);
+
+	return r ? r->flags : MR_F_UNKNOWN;
+}
+
+void memregions_split(phys_addr_t addr, struct mem_region **r1, struct mem_region **r2)
+{
+	*r1 = memregions_find(addr);
+	assert(*r1);
+
+	if ((*r1)->start == addr) {
+		*r2 = *r1;
+		*r1 = NULL;
+		return;
+	}
+
+	*r2 = memregions_add(&(struct mem_region){
+		.start = addr,
+		.end = (*r1)->end,
+		.flags = (*r1)->flags,
+	});
+
+	(*r1)->end = addr;
+}
+
+void memregions_add_dt_regions(size_t max_nr)
+{
+	struct dt_pbus_reg regs[max_nr];
+	int nr_regs, i;
+
+	nr_regs = dt_get_memory_params(regs, max_nr);
+	assert(nr_regs > 0);
+
+	for (i = 0; i < nr_regs; ++i) {
+		memregions_add(&(struct mem_region){
+			.start = regs[i].addr,
+			.end = regs[i].addr + regs[i].size,
+		});
+	}
+}
diff --git a/lib/memregions.h b/lib/memregions.h
new file mode 100644
index 000000000000..9a8e33182fe5
--- /dev/null
+++ b/lib/memregions.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _MEMREGIONS_H_
+#define _MEMREGIONS_H_
+#include <libcflat.h>
+#include <bitops.h>
+
+#define NR_INITIAL_MEM_REGIONS		8
+
+#define MR_F_IO				BIT(0)
+#define MR_F_CODE			BIT(1)
+#define MR_F_RESERVED			BIT(2)
+#define MR_F_UNKNOWN			BIT(31)
+
+struct mem_region {
+	phys_addr_t start;
+	phys_addr_t end;
+	uint32_t flags;
+};
+
+extern struct mem_region *mem_regions;
+
+void memregions_init(struct mem_region regions[], size_t nr);
+struct mem_region *memregions_add(struct mem_region *r);
+struct mem_region *memregions_find(phys_addr_t paddr);
+uint32_t memregions_get_flags(phys_addr_t paddr);
+void memregions_split(phys_addr_t addr, struct mem_region **r1, struct mem_region **r2);
+void memregions_add_dt_regions(size_t max_nr);
+
+#endif /* _MEMREGIONS_H_ */
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 17/24] riscv: Populate memregions and switch to page allocator
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (15 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 16/24] arm/arm64: Share memregions Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 18/24] riscv: Add MMU support Andrew Jones
                   ` (7 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Get the memory regions from the DT rather than just assuming we have
enough. Getting memory regions and setting their flags is also a
first step to enabling the MMU. Also switch to the page allocator,
which is a second step to enabling the MMU.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/riscv/asm/memory_areas.h |  1 +
 lib/riscv/setup.c            | 55 ++++++++++++++++++++++++++++++++++--
 riscv/Makefile               |  2 ++
 3 files changed, 55 insertions(+), 3 deletions(-)
 create mode 100644 lib/riscv/asm/memory_areas.h

diff --git a/lib/riscv/asm/memory_areas.h b/lib/riscv/asm/memory_areas.h
new file mode 100644
index 000000000000..2b34e63502dd
--- /dev/null
+++ b/lib/riscv/asm/memory_areas.h
@@ -0,0 +1 @@
+#include <asm-generic/memory_areas.h>
diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c
index 9ff446b5e171..848ec8e83496 100644
--- a/lib/riscv/setup.c
+++ b/lib/riscv/setup.c
@@ -6,22 +6,31 @@
  */
 #include <libcflat.h>
 #include <alloc.h>
+#include <alloc_page.h>
 #include <alloc_phys.h>
 #include <argv.h>
 #include <cpumask.h>
 #include <devicetree.h>
+#include <memregions.h>
 #include <on-cpus.h>
 #include <asm/csr.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 #include <asm/setup.h>
 
+#define VA_BASE			((phys_addr_t)3 * SZ_1G)
+
+#define MAX_DT_MEM_REGIONS	16
+#define NR_MEM_REGIONS		(MAX_DT_MEM_REGIONS + 16)
+
 char *initrd;
 u32 initrd_size;
 
 struct thread_info cpus[NR_CPUS];
 int nr_cpus;
 
+static struct mem_region riscv_mem_regions[NR_MEM_REGIONS + 1];
+
 int hartid_to_cpu(unsigned long hartid)
 {
 	int cpu;
@@ -64,10 +73,50 @@ static void cpu_init(void)
 	cpu0_calls_idle = true;
 }
 
+extern unsigned long _etext;
+
 static void mem_init(phys_addr_t freemem_start)
 {
-	//TODO - for now just assume we've got some memory available
-	phys_alloc_init(freemem_start, 16 * SZ_1M);
+	struct mem_region *freemem, *code, *data;
+	phys_addr_t freemem_end, base, top;
+
+	memregions_init(riscv_mem_regions, NR_MEM_REGIONS);
+	memregions_add_dt_regions(MAX_DT_MEM_REGIONS);
+
+	/* Split the region with the code into two regions; code and data */
+	memregions_split((unsigned long)&_etext, &code, &data);
+	assert(code);
+	code->flags |= MR_F_CODE;
+
+	freemem = memregions_find(freemem_start);
+	assert(freemem && !(freemem->flags & (MR_F_IO | MR_F_CODE)));
+
+	freemem_end = freemem->end & PAGE_MASK;
+
+	/*
+	 * The assert below is mostly checking that the free memory doesn't
+	 * start in the 3G-4G range, which is reserved for virtual addresses,
+	 * but it also confirms that there is some free memory (the amount
+	 * is arbitrarily selected, but should be sufficient for a unit test)
+	 *
+	 * TODO: Allow the VA range to shrink and move.
+	 */
+	if (freemem_end > VA_BASE)
+		freemem_end = VA_BASE;
+	assert(freemem_end - freemem_start >= SZ_1M * 16);
+
+	/*
+	 * TODO: Remove the need for this phys allocator dance, since, as we
+	 * can see with the assert, we could have gone straight to the page
+	 * allocator.
+	 */
+	phys_alloc_init(freemem_start, freemem_end - freemem_start);
+	phys_alloc_set_minimum_alignment(PAGE_SIZE);
+	phys_alloc_get_unused(&base, &top);
+	assert(base == freemem_start && top == freemem_end);
+
+	page_alloc_init_area(0, freemem_start >> PAGE_SHIFT, freemem_end >> PAGE_SHIFT);
+	page_alloc_ops_enable();
 }
 
 static void banner(void)
@@ -86,7 +135,7 @@ void setup(const void *fdt, phys_addr_t freemem_start)
 	u32 fdt_size;
 	int ret;
 
-	assert(sizeof(long) == 8 || freemem_start < (3ul << 30));
+	assert(sizeof(long) == 8 || freemem_start < VA_BASE);
 	freemem = (void *)(unsigned long)freemem_start;
 
 	/* Move the FDT to the base of free memory */
diff --git a/riscv/Makefile b/riscv/Makefile
index 932f3378264c..ed1a14025ed2 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -22,8 +22,10 @@ $(TEST_DIR)/sieve.elf: AUXFLAGS = 0x1
 cstart.o = $(TEST_DIR)/cstart.o
 
 cflatobjs += lib/alloc.o
+cflatobjs += lib/alloc_page.o
 cflatobjs += lib/alloc_phys.o
 cflatobjs += lib/devicetree.o
+cflatobjs += lib/memregions.o
 cflatobjs += lib/on-cpus.o
 cflatobjs += lib/riscv/bitops.o
 cflatobjs += lib/riscv/io.o
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 18/24] riscv: Add MMU support
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (16 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 17/24] riscv: Populate memregions and switch to page allocator Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 19/24] riscv: Enable the MMU in secondaries Andrew Jones
                   ` (6 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Add minimal page table defines and functions in order to build page
tables and enable the MMU.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/riscv/asm/csr.h     |   1 +
 lib/riscv/asm/io.h      |   3 +
 lib/riscv/asm/mmu.h     |  34 +++++++++
 lib/riscv/asm/page.h    |  14 ++++
 lib/riscv/asm/pgtable.h |  42 +++++++++++
 lib/riscv/mmu.c         | 150 ++++++++++++++++++++++++++++++++++++++++
 lib/riscv/setup.c       |   3 +
 riscv/Makefile          |   1 +
 8 files changed, 248 insertions(+)
 create mode 100644 lib/riscv/asm/mmu.h
 create mode 100644 lib/riscv/asm/pgtable.h
 create mode 100644 lib/riscv/mmu.c

diff --git a/lib/riscv/asm/csr.h b/lib/riscv/asm/csr.h
index 39ffd2a146be..52608512b68d 100644
--- a/lib/riscv/asm/csr.h
+++ b/lib/riscv/asm/csr.h
@@ -9,6 +9,7 @@
 #define CSR_SEPC		0x141
 #define CSR_SCAUSE		0x142
 #define CSR_STVAL		0x143
+#define CSR_SATP		0x180
 
 /* Exception cause high bit - is an interrupt if set */
 #define CAUSE_IRQ_FLAG		(_AC(1, UL) << (__riscv_xlen - 1))
diff --git a/lib/riscv/asm/io.h b/lib/riscv/asm/io.h
index d2eb3acc9fda..6fe111289102 100644
--- a/lib/riscv/asm/io.h
+++ b/lib/riscv/asm/io.h
@@ -73,6 +73,9 @@ static inline u64 __raw_readq(const volatile void __iomem *addr)
 }
 #endif
 
+#define ioremap ioremap
+void __iomem *ioremap(phys_addr_t phys_addr, size_t size);
+
 #include <asm-generic/io.h>
 
 #endif /* _ASMRISCV_IO_H_ */
diff --git a/lib/riscv/asm/mmu.h b/lib/riscv/asm/mmu.h
new file mode 100644
index 000000000000..18d39e75ba30
--- /dev/null
+++ b/lib/riscv/asm/mmu.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_MMU_H_
+#define _ASMRISCV_MMU_H_
+#include <libcflat.h>
+#include <asm/csr.h>
+#include <asm/page.h>
+#include <asm/pgtable.h>
+
+static inline pgd_t *current_pgtable(void)
+{
+	return (pgd_t *)((csr_read(CSR_SATP) & SATP_PPN) << PAGE_SHIFT);
+}
+
+void mmu_set_range_ptes(pgd_t *pgtable, uintptr_t virt_offset,
+			phys_addr_t phys_start, phys_addr_t phys_end,
+			pgprot_t prot, bool flush);
+void __mmu_enable(unsigned long satp);
+void mmu_enable(unsigned long mode, pgd_t *pgtable);
+void mmu_disable(void);
+
+void setup_mmu(void);
+
+static inline void local_flush_tlb_page(unsigned long addr)
+{
+	asm volatile("sfence.vma %0" : : "r" (addr) : "memory");
+}
+
+/*
+ * Get the pte pointer for a virtual address, even if it's not mapped.
+ * Constructs upper levels of the table as necessary.
+ */
+pte_t *get_pte(pgd_t *pgtable, uintptr_t vaddr);
+
+#endif /* _ASMRISCV_MMU_H_ */
diff --git a/lib/riscv/asm/page.h b/lib/riscv/asm/page.h
index 7d7c9191605a..07b482f76176 100644
--- a/lib/riscv/asm/page.h
+++ b/lib/riscv/asm/page.h
@@ -2,6 +2,20 @@
 #ifndef _ASMRISCV_PAGE_H_
 #define _ASMRISCV_PAGE_H_
 
+#ifndef __ASSEMBLY__
+
+typedef unsigned long pgd_t;
+typedef unsigned long pte_t;
+typedef unsigned long pgprot_t;
+typedef unsigned long pteval_t;
+
+#define pte_val(x)		((pteval_t)(x))
+#define pgprot_val(x)		((pteval_t)(x))
+#define __pte(x)		((pte_t)(x))
+#define __pgprot(x)		((pgprot_t)(x))
+
+#endif /* !__ASSEMBLY__ */
+
 #include <asm-generic/page.h>
 
 #endif /* _ASMRISCV_PAGE_H_ */
diff --git a/lib/riscv/asm/pgtable.h b/lib/riscv/asm/pgtable.h
new file mode 100644
index 000000000000..98d41ff9f661
--- /dev/null
+++ b/lib/riscv/asm/pgtable.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_PGTABLE_H_
+#define _ASMRISCV_PGTABLE_H_
+#include <linux/const.h>
+
+#if __riscv_xlen == 32
+#define SATP_PPN		_AC(0x003FFFFF, UL)
+#define SATP_MODE_32		_AC(0x80000000, UL)
+#define SATP_MODE_SHIFT		31
+#define NR_LEVELS		2
+#define PGDIR_BITS		10
+#define PGDIR_MASK		_AC(0x3FF, UL)
+#define PTE_PPN			_AC(0xFFFFFC00, UL)
+
+#define SATP_MODE_DEFAULT	SATP_MODE_32
+
+#else
+#define SATP_PPN		_AC(0x00000FFFFFFFFFFF, UL)
+#define SATP_MODE_39		_AC(0x8000000000000000, UL)
+#define SATP_MODE_SHIFT		60
+#define NR_LEVELS		3
+#define PGDIR_BITS		9
+#define PGDIR_MASK		_AC(0x1FF, UL)
+#define PTE_PPN			_AC(0x3FFFFFFFFFFC00, UL)
+
+#define SATP_MODE_DEFAULT	SATP_MODE_39
+
+#endif
+
+#define PPN_SHIFT		10
+
+#define _PAGE_PRESENT		(1 << 0)
+#define _PAGE_READ		(1 << 1)
+#define _PAGE_WRITE		(1 << 2)
+#define _PAGE_EXEC		(1 << 3)
+#define _PAGE_USER		(1 << 4)
+#define _PAGE_GLOBAL		(1 << 5)
+#define _PAGE_ACCESSED		(1 << 6)
+#define _PAGE_DIRTY		(1 << 7)
+#define _PAGE_SOFT		(3 << 8)
+
+#endif /* _ASMRISCV_PGTABLE_H_ */
diff --git a/lib/riscv/mmu.c b/lib/riscv/mmu.c
new file mode 100644
index 000000000000..4d13a3034ccc
--- /dev/null
+++ b/lib/riscv/mmu.c
@@ -0,0 +1,150 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
+ */
+#include <libcflat.h>
+#include <alloc_page.h>
+#include <memregions.h>
+#include <asm/csr.h>
+#include <asm/io.h>
+#include <asm/mmu.h>
+#include <asm/page.h>
+
+static pgd_t *__initial_pgtable;
+
+static int pte_index(uintptr_t vaddr, int level)
+{
+	return (vaddr >> (PGDIR_BITS * level + PAGE_SHIFT)) & PGDIR_MASK;
+}
+
+static pte_t *pteval_to_ptep(pteval_t pteval)
+{
+	return (pte_t *)(((pteval & PTE_PPN) >> PPN_SHIFT) << PAGE_SHIFT);
+}
+
+static pteval_t ptep_to_pteval(pte_t *ptep)
+{
+	return ((pteval_t)ptep >> PAGE_SHIFT) << PPN_SHIFT;
+}
+
+pte_t *get_pte(pgd_t *pgtable, uintptr_t vaddr)
+{
+	pte_t *ptep = (pte_t *)pgtable;
+
+	assert(pgtable && !((uintptr_t)pgtable & ~PAGE_MASK));
+
+	for (int level = NR_LEVELS - 1; level > 0; --level) {
+		pte_t *next = &ptep[pte_index(vaddr, level)];
+		if (!pte_val(*next)) {
+			void *page = alloc_page();
+			*next = __pte(ptep_to_pteval(page) | _PAGE_PRESENT);
+		}
+		ptep = pteval_to_ptep(pte_val(*next));
+	}
+	ptep = &ptep[pte_index(vaddr, 0)];
+
+	return ptep;
+}
+
+static pteval_t *__install_page(pgd_t *pgtable, phys_addr_t paddr,
+				uintptr_t vaddr, pgprot_t prot, bool flush)
+{
+	phys_addr_t ppn = (paddr >> PAGE_SHIFT) << PPN_SHIFT;
+	pteval_t pte = (pteval_t)ppn;
+	pte_t *ptep;
+
+	assert(!(ppn & ~PTE_PPN));
+
+	ptep = get_pte(pgtable, vaddr);
+	*ptep = __pte(pte | pgprot_val(prot) | _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_DIRTY);
+
+	if (flush)
+		local_flush_tlb_page(vaddr);
+
+	return (pteval_t *)ptep;
+}
+
+void mmu_set_range_ptes(pgd_t *pgtable, uintptr_t virt_offset,
+			phys_addr_t phys_start, phys_addr_t phys_end,
+			pgprot_t prot, bool flush)
+{
+	phys_addr_t paddr = phys_start & PAGE_MASK;
+	uintptr_t vaddr = virt_offset & PAGE_MASK;
+	uintptr_t virt_end = phys_end - paddr + vaddr;
+
+	assert(phys_start < phys_end);
+
+	for (; vaddr < virt_end; vaddr += PAGE_SIZE, paddr += PAGE_SIZE)
+		__install_page(pgtable, paddr, vaddr, prot, flush);
+}
+
+void mmu_disable(void)
+{
+	__asm__ __volatile__ (
+	"	csrw	" xstr(CSR_SATP) ", zero\n"
+	"	sfence.vma\n"
+	: : : "memory");
+}
+
+void __mmu_enable(unsigned long satp)
+{
+	__asm__ __volatile__ (
+	"	sfence.vma\n"
+	"	csrw	" xstr(CSR_SATP) ", %0\n"
+	: : "r" (satp) : "memory");
+}
+
+void mmu_enable(unsigned long mode, pgd_t *pgtable)
+{
+	unsigned long ppn = (unsigned long)pgtable >> PAGE_SHIFT;
+	unsigned long satp = mode | ppn;
+
+	assert(!(ppn & ~SATP_PPN));
+	__mmu_enable(satp);
+}
+
+void setup_mmu(void)
+{
+	struct mem_region *r;
+	pgd_t *pgtable;
+
+	if (!__initial_pgtable)
+		__initial_pgtable = alloc_page();
+	pgtable = __initial_pgtable;
+
+	for (r = mem_regions; r->end; ++r) {
+		if (r->flags & (MR_F_IO | MR_F_RESERVED))
+			continue;
+		if (r->flags & MR_F_CODE) {
+			mmu_set_range_ptes(pgtable, r->start, r->start, r->end,
+					   __pgprot(_PAGE_READ | _PAGE_EXEC), false);
+		} else {
+			mmu_set_range_ptes(pgtable, r->start, r->start, r->end,
+					   __pgprot(_PAGE_READ | _PAGE_WRITE), false);
+		}
+	}
+
+	mmu_enable(SATP_MODE_DEFAULT, pgtable);
+}
+
+void __iomem *ioremap(phys_addr_t phys_addr, size_t size)
+{
+	phys_addr_t start = phys_addr & PAGE_MASK;
+	phys_addr_t end = PAGE_ALIGN(phys_addr + size);
+	pgd_t *pgtable = current_pgtable();
+	bool flush = true;
+
+	assert(sizeof(long) == 8 || !(phys_addr >> 32));
+
+	if (!pgtable) {
+		if (!__initial_pgtable)
+			__initial_pgtable = alloc_page();
+		pgtable = __initial_pgtable;
+		flush = false;
+	}
+
+	mmu_set_range_ptes(pgtable, start, start, end,
+			   __pgprot(_PAGE_READ | _PAGE_WRITE), flush);
+
+	return (void __iomem *)(unsigned long)phys_addr;
+}
diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c
index 848ec8e83496..c4c1bd58b337 100644
--- a/lib/riscv/setup.c
+++ b/lib/riscv/setup.c
@@ -14,6 +14,7 @@
 #include <memregions.h>
 #include <on-cpus.h>
 #include <asm/csr.h>
+#include <asm/mmu.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 #include <asm/setup.h>
@@ -171,5 +172,7 @@ void setup(const void *fdt, phys_addr_t freemem_start)
 		setup_env(env, initrd_size);
 	}
 
+	setup_mmu();
+
 	banner();
 }
diff --git a/riscv/Makefile b/riscv/Makefile
index ed1a14025ed2..821891b719e7 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -29,6 +29,7 @@ cflatobjs += lib/memregions.o
 cflatobjs += lib/on-cpus.o
 cflatobjs += lib/riscv/bitops.o
 cflatobjs += lib/riscv/io.o
+cflatobjs += lib/riscv/mmu.o
 cflatobjs += lib/riscv/processor.o
 cflatobjs += lib/riscv/sbi.o
 cflatobjs += lib/riscv/setup.o
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 19/24] riscv: Enable the MMU in secondaries
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (17 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 18/24] riscv: Add MMU support Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 20/24] riscv: Enable vmalloc Andrew Jones
                   ` (5 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Set each secondary satp to the same as the primary's and enable the
MMU when starting. We also change the memalign() to alloc_pages()
to prepare for enabling vmalloc_ops. We always want an address
for the stack where its virtual address is the same as its physical
address, but vmalloc_ops.memalign wouldn't provide that.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/riscv/asm-offsets.c | 1 +
 lib/riscv/asm/smp.h     | 1 +
 lib/riscv/smp.c         | 7 +++++--
 3 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/lib/riscv/asm-offsets.c b/lib/riscv/asm-offsets.c
index f5beeeb45e09..a2a32438e075 100644
--- a/lib/riscv/asm-offsets.c
+++ b/lib/riscv/asm-offsets.c
@@ -53,6 +53,7 @@ int main(void)
 	OFFSET(PT_ORIG_A0, pt_regs, orig_a0);
 	DEFINE(PT_SIZE, sizeof(struct pt_regs));
 
+	OFFSET(SECONDARY_SATP, secondary_data, satp);
 	OFFSET(SECONDARY_STVEC, secondary_data, stvec);
 	OFFSET(SECONDARY_FUNC, secondary_data, func);
 	DEFINE(SECONDARY_DATA_SIZE, sizeof(struct secondary_data));
diff --git a/lib/riscv/asm/smp.h b/lib/riscv/asm/smp.h
index 931766dc3969..b3ead4e86433 100644
--- a/lib/riscv/asm/smp.h
+++ b/lib/riscv/asm/smp.h
@@ -15,6 +15,7 @@ static inline int smp_processor_id(void)
 typedef void (*secondary_func_t)(void);
 
 struct secondary_data {
+	unsigned long satp;
 	unsigned long stvec;
 	secondary_func_t func;
 } __attribute__((aligned(16)));
diff --git a/lib/riscv/smp.c b/lib/riscv/smp.c
index ed7984e75608..7e4bb5b76903 100644
--- a/lib/riscv/smp.c
+++ b/lib/riscv/smp.c
@@ -5,9 +5,10 @@
  * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
  */
 #include <libcflat.h>
-#include <alloc.h>
+#include <alloc_page.h>
 #include <cpumask.h>
 #include <asm/csr.h>
+#include <asm/mmu.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 #include <asm/sbi.h>
@@ -23,6 +24,7 @@ secondary_func_t secondary_cinit(struct secondary_data *data)
 {
 	struct thread_info *info;
 
+	__mmu_enable(data->satp);
 	thread_info_init();
 	info = current_thread_info();
 	set_cpu_online(info->cpu, true);
@@ -33,10 +35,11 @@ secondary_func_t secondary_cinit(struct secondary_data *data)
 
 static void __smp_boot_secondary(int cpu, secondary_func_t func)
 {
-	struct secondary_data *sp = memalign(16, SZ_8K) + SZ_8K - 16;
+	struct secondary_data *sp = alloc_pages(1) + SZ_8K - 16;
 	struct sbiret ret;
 
 	sp -= sizeof(struct secondary_data);
+	sp->satp = csr_read(CSR_SATP);
 	sp->stvec = csr_read(CSR_STVEC);
 	sp->func = func;
 
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 20/24] riscv: Enable vmalloc
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (18 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 19/24] riscv: Enable the MMU in secondaries Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 21/24] lib: Add strcasecmp and strncasecmp Andrew Jones
                   ` (4 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Implement the functions that vmalloc depends on and let it enable the
MMU through setup_vm(). We can now also run the sieve test, so we
add it as well.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/riscv/asm/io.h  |  6 +++++
 lib/riscv/asm/mmu.h |  2 --
 lib/riscv/mmu.c     | 57 ++++++++++++++++++++++++++++++++++++++++++++-
 lib/riscv/setup.c   | 16 ++++++++++---
 riscv/Makefile      |  3 ++-
 riscv/sieve.c       |  1 +
 6 files changed, 78 insertions(+), 7 deletions(-)
 create mode 120000 riscv/sieve.c

diff --git a/lib/riscv/asm/io.h b/lib/riscv/asm/io.h
index 6fe111289102..37a130e533c9 100644
--- a/lib/riscv/asm/io.h
+++ b/lib/riscv/asm/io.h
@@ -76,6 +76,12 @@ static inline u64 __raw_readq(const volatile void __iomem *addr)
 #define ioremap ioremap
 void __iomem *ioremap(phys_addr_t phys_addr, size_t size);
 
+#define virt_to_phys virt_to_phys
+unsigned long virt_to_phys(volatile void *address);
+
+#define phys_to_virt phys_to_virt
+void *phys_to_virt(unsigned long address);
+
 #include <asm-generic/io.h>
 
 #endif /* _ASMRISCV_IO_H_ */
diff --git a/lib/riscv/asm/mmu.h b/lib/riscv/asm/mmu.h
index 18d39e75ba30..bb60f0895e2b 100644
--- a/lib/riscv/asm/mmu.h
+++ b/lib/riscv/asm/mmu.h
@@ -18,8 +18,6 @@ void __mmu_enable(unsigned long satp);
 void mmu_enable(unsigned long mode, pgd_t *pgtable);
 void mmu_disable(void);
 
-void setup_mmu(void);
-
 static inline void local_flush_tlb_page(unsigned long addr)
 {
 	asm volatile("sfence.vma %0" : : "r" (addr) : "memory");
diff --git a/lib/riscv/mmu.c b/lib/riscv/mmu.c
index 4d13a3034ccc..bd006881a95f 100644
--- a/lib/riscv/mmu.c
+++ b/lib/riscv/mmu.c
@@ -5,6 +5,7 @@
 #include <libcflat.h>
 #include <alloc_page.h>
 #include <memregions.h>
+#include <vmalloc.h>
 #include <asm/csr.h>
 #include <asm/io.h>
 #include <asm/mmu.h>
@@ -64,6 +65,15 @@ static pteval_t *__install_page(pgd_t *pgtable, phys_addr_t paddr,
 	return (pteval_t *)ptep;
 }
 
+pteval_t *install_page(pgd_t *pgtable, phys_addr_t phys, void *virt)
+{
+	phys_addr_t paddr = phys & PAGE_MASK;
+	uintptr_t vaddr = (uintptr_t)virt & PAGE_MASK;
+
+	return __install_page(pgtable, paddr, vaddr,
+			      __pgprot(_PAGE_READ | _PAGE_WRITE), true);
+}
+
 void mmu_set_range_ptes(pgd_t *pgtable, uintptr_t virt_offset,
 			phys_addr_t phys_start, phys_addr_t phys_end,
 			pgprot_t prot, bool flush)
@@ -103,7 +113,7 @@ void mmu_enable(unsigned long mode, pgd_t *pgtable)
 	__mmu_enable(satp);
 }
 
-void setup_mmu(void)
+void *setup_mmu(phys_addr_t top, void *opaque)
 {
 	struct mem_region *r;
 	pgd_t *pgtable;
@@ -125,6 +135,8 @@ void setup_mmu(void)
 	}
 
 	mmu_enable(SATP_MODE_DEFAULT, pgtable);
+
+	return pgtable;
 }
 
 void __iomem *ioremap(phys_addr_t phys_addr, size_t size)
@@ -148,3 +160,46 @@ void __iomem *ioremap(phys_addr_t phys_addr, size_t size)
 
 	return (void __iomem *)(unsigned long)phys_addr;
 }
+
+phys_addr_t virt_to_pte_phys(pgd_t *pgtable, void *virt)
+{
+	uintptr_t vaddr = (uintptr_t)virt;
+	pte_t *ptep = (pte_t *)pgtable;
+
+	assert(pgtable && !((uintptr_t)pgtable & ~PAGE_MASK));
+
+	for (int level = NR_LEVELS - 1; level > 0; --level) {
+		pte_t *next = &ptep[pte_index(vaddr, level)];
+		if (!pte_val(*next))
+			return 0;
+		ptep = pteval_to_ptep(pte_val(*next));
+	}
+	ptep = &ptep[pte_index(vaddr, 0)];
+
+	if (!pte_val(*ptep))
+		return 0;
+
+	return __pa(pteval_to_ptep(pte_val(*ptep)));
+}
+
+unsigned long virt_to_phys(volatile void *address)
+{
+	unsigned long satp = csr_read(CSR_SATP);
+	pgd_t *pgtable = (pgd_t *)((satp & SATP_PPN) << PAGE_SHIFT);
+	phys_addr_t paddr;
+
+	if ((satp >> SATP_MODE_SHIFT) == 0)
+		return __pa(address);
+
+	paddr = virt_to_pte_phys(pgtable, (void *)address);
+	assert(sizeof(long) == 8 || !(paddr >> 32));
+
+	return (unsigned long)paddr;
+}
+
+void *phys_to_virt(unsigned long address)
+{
+	/* @address must have an identity mapping for this to work. */
+	assert(virt_to_phys(__va(address)) == address);
+	return __va(address);
+}
diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c
index c4c1bd58b337..40ff26a24cfc 100644
--- a/lib/riscv/setup.c
+++ b/lib/riscv/setup.c
@@ -9,10 +9,12 @@
 #include <alloc_page.h>
 #include <alloc_phys.h>
 #include <argv.h>
+#include <auxinfo.h>
 #include <cpumask.h>
 #include <devicetree.h>
 #include <memregions.h>
 #include <on-cpus.h>
+#include <vmalloc.h>
 #include <asm/csr.h>
 #include <asm/mmu.h>
 #include <asm/page.h>
@@ -20,6 +22,11 @@
 #include <asm/setup.h>
 
 #define VA_BASE			((phys_addr_t)3 * SZ_1G)
+#if __riscv_xlen == 64
+#define VA_TOP			((phys_addr_t)4 * SZ_1G)
+#else
+#define VA_TOP			((phys_addr_t)0)
+#endif
 
 #define MAX_DT_MEM_REGIONS	16
 #define NR_MEM_REGIONS		(MAX_DT_MEM_REGIONS + 16)
@@ -106,6 +113,8 @@ static void mem_init(phys_addr_t freemem_start)
 		freemem_end = VA_BASE;
 	assert(freemem_end - freemem_start >= SZ_1M * 16);
 
+	init_alloc_vpage(__va(VA_TOP));
+
 	/*
 	 * TODO: Remove the need for this phys allocator dance, since, as we
 	 * can see with the assert, we could have gone straight to the page
@@ -137,7 +146,7 @@ void setup(const void *fdt, phys_addr_t freemem_start)
 	int ret;
 
 	assert(sizeof(long) == 8 || freemem_start < VA_BASE);
-	freemem = (void *)(unsigned long)freemem_start;
+	freemem = __va(freemem_start);
 
 	/* Move the FDT to the base of free memory */
 	fdt_size = fdt_totalsize(fdt);
@@ -156,7 +165,7 @@ void setup(const void *fdt, phys_addr_t freemem_start)
 		freemem += initrd_size;
 	}
 
-	mem_init(PAGE_ALIGN((unsigned long)freemem));
+	mem_init(PAGE_ALIGN(__pa(freemem)));
 	cpu_init();
 	thread_info_init();
 	io_init();
@@ -172,7 +181,8 @@ void setup(const void *fdt, phys_addr_t freemem_start)
 		setup_env(env, initrd_size);
 	}
 
-	setup_mmu();
+	if (!(auxinfo.flags & AUXINFO_MMU_OFF))
+		setup_vm();
 
 	banner();
 }
diff --git a/riscv/Makefile b/riscv/Makefile
index 821891b719e7..61a1ff88d8ec 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -13,7 +13,7 @@ endif
 tests =
 tests += $(TEST_DIR)/sbi.$(exe)
 tests += $(TEST_DIR)/selftest.$(exe)
-#tests += $(TEST_DIR)/sieve.$(exe)
+tests += $(TEST_DIR)/sieve.$(exe)
 
 all: $(tests)
 
@@ -27,6 +27,7 @@ cflatobjs += lib/alloc_phys.o
 cflatobjs += lib/devicetree.o
 cflatobjs += lib/memregions.o
 cflatobjs += lib/on-cpus.o
+cflatobjs += lib/vmalloc.o
 cflatobjs += lib/riscv/bitops.o
 cflatobjs += lib/riscv/io.o
 cflatobjs += lib/riscv/mmu.o
diff --git a/riscv/sieve.c b/riscv/sieve.c
new file mode 120000
index 000000000000..8f14a5c3d4aa
--- /dev/null
+++ b/riscv/sieve.c
@@ -0,0 +1 @@
+../x86/sieve.c
\ No newline at end of file
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 21/24] lib: Add strcasecmp and strncasecmp
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (19 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 20/24] riscv: Enable vmalloc Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-01  9:45   ` Eric Auger
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 22/24] riscv: Add isa string parsing Andrew Jones
                   ` (3 subsequent siblings)
  24 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

We'll soon want a case insensitive string comparison. Add toupper()
and tolower() too (the latter gets used by the new string functions).

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/ctype.h  | 10 ++++++++++
 lib/string.c | 14 ++++++++++++++
 lib/string.h |  2 ++
 3 files changed, 26 insertions(+)

diff --git a/lib/ctype.h b/lib/ctype.h
index 48a9c16300f8..45c96f111e19 100644
--- a/lib/ctype.h
+++ b/lib/ctype.h
@@ -37,4 +37,14 @@ static inline int isspace(int c)
         return c == ' ' || c == '\t' || c == '\r' || c == '\n' || c == '\v' || c == '\f';
 }
 
+static inline int toupper(int c)
+{
+	return islower(c) ? c - 'a' + 'A' : c;
+}
+
+static inline int tolower(int c)
+{
+	return isupper(c) ? c - 'A' + 'a' : c;
+}
+
 #endif /* _CTYPE_H_ */
diff --git a/lib/string.c b/lib/string.c
index 6d8a6380db92..ab6a724a3144 100644
--- a/lib/string.c
+++ b/lib/string.c
@@ -54,11 +54,25 @@ int strncmp(const char *a, const char *b, size_t n)
 	return 0;
 }
 
+int strncasecmp(const char *a, const char *b, size_t n)
+{
+	for (; n--; ++a, ++b)
+		if (tolower(*a) != tolower(*b) || *a == '\0')
+			return tolower(*a) - tolower(*b);
+
+	return 0;
+}
+
 int strcmp(const char *a, const char *b)
 {
 	return strncmp(a, b, SIZE_MAX);
 }
 
+int strcasecmp(const char *a, const char *b)
+{
+	return strncasecmp(a, b, SIZE_MAX);
+}
+
 char *strchr(const char *s, int c)
 {
 	while (*s != (char)c)
diff --git a/lib/string.h b/lib/string.h
index 758dca8af36a..a28d75641530 100644
--- a/lib/string.h
+++ b/lib/string.h
@@ -15,6 +15,8 @@ extern char *strcat(char *dest, const char *src);
 extern char *strcpy(char *dest, const char *src);
 extern int strcmp(const char *a, const char *b);
 extern int strncmp(const char *a, const char *b, size_t n);
+int strcasecmp(const char *a, const char *b);
+int strncasecmp(const char *a, const char *b, size_t n);
 extern char *strchr(const char *s, int c);
 extern char *strrchr(const char *s, int c);
 extern char *strchrnul(const char *s, int c);
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 22/24] riscv: Add isa string parsing
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (20 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 21/24] lib: Add strcasecmp and strncasecmp Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 23/24] gitlab-ci: Add riscv64 tests Andrew Jones
                   ` (2 subsequent siblings)
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

We can probably get away with just assuming several important
and popular extensions (at least everything covered by G), but
we'll also want to use some extensions which we should ensure
are present by parsing the isa string. Add a parser and already
apply it to Sstc.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 lib/riscv/asm/isa.h       |  33 ++++++++++
 lib/riscv/asm/processor.h |   1 +
 lib/riscv/isa.c           | 126 ++++++++++++++++++++++++++++++++++++++
 lib/riscv/processor.c     |   2 +
 riscv/Makefile            |   1 +
 5 files changed, 163 insertions(+)
 create mode 100644 lib/riscv/asm/isa.h
 create mode 100644 lib/riscv/isa.c

diff --git a/lib/riscv/asm/isa.h b/lib/riscv/asm/isa.h
new file mode 100644
index 000000000000..df874173f4ed
--- /dev/null
+++ b/lib/riscv/asm/isa.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASMRISCV_ISA_H_
+#define _ASMRISCV_ISA_H_
+#include <bitops.h>
+#include <asm/setup.h>
+
+/*
+ * We assume and use several extensions, such as Zicsr and Zifencei.
+ * Here we only track extensions which we don't assume and the
+ * framework may want to use. Unit tests may check for extensions
+ * by name not tracked here with cpu_has_extension_name()
+ */
+enum {
+	ISA_SSTC,
+	ISA_MAX,
+};
+_Static_assert(ISA_MAX <= __riscv_xlen, "Need to increase thread_info.isa");
+
+static inline bool cpu_has_extension(int cpu, int ext)
+{
+	return test_bit(ext, cpus[cpu].isa);
+}
+
+bool cpu_has_extension_name(int cpu, const char *ext);
+
+static inline bool has_ext(const char *ext)
+{
+	return cpu_has_extension_name(current_thread_info()->cpu, ext);
+}
+
+void isa_init(struct thread_info *info);
+
+#endif /* _ASMRISCV_ISA_H_ */
diff --git a/lib/riscv/asm/processor.h b/lib/riscv/asm/processor.h
index f20774d02d8e..32c499d0c0ab 100644
--- a/lib/riscv/asm/processor.h
+++ b/lib/riscv/asm/processor.h
@@ -11,6 +11,7 @@ typedef void (*exception_fn)(struct pt_regs *);
 struct thread_info {
 	int cpu;
 	unsigned long hartid;
+	unsigned long isa[1];
 	exception_fn exception_handlers[EXCEPTION_CAUSE_MAX];
 };
 
diff --git a/lib/riscv/isa.c b/lib/riscv/isa.c
new file mode 100644
index 000000000000..bc1c9c72045c
--- /dev/null
+++ b/lib/riscv/isa.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
+ */
+#include <libcflat.h>
+#include <bitops.h>
+#include <devicetree.h>
+#include <string.h>
+#include <asm/isa.h>
+#include <asm/setup.h>
+
+typedef void (*isa_func_t)(const char *, int, void *);
+
+struct isa_info {
+	unsigned long hartid;
+	isa_func_t func;
+	void *data;
+};
+
+static bool isa_match(const char *ext, const char *name, int len)
+{
+	return len == strlen(ext) && !strncasecmp(name, ext, len);
+}
+
+struct isa_check {
+	const char *ext;
+	bool found;
+};
+
+static void isa_name(const char *name, int len, void *data)
+{
+	struct isa_check *check = (struct isa_check *)data;
+
+	if (isa_match(check->ext, name, len))
+		check->found = true;
+}
+
+static void isa_bit(const char *name, int len, void *data)
+{
+	struct thread_info *info = (struct thread_info *)data;
+
+	if (isa_match("sstc", name, len))
+		set_bit(ISA_SSTC, info->isa);
+}
+
+static void isa_parse(const char *isa_string, int len, struct isa_info *info)
+{
+	assert(isa_string[0] == 'r' && isa_string[1] == 'v');
+#if __riscv_xlen == 32
+	assert(isa_string[2] == '3' && isa_string[3] == '2');
+#else
+	assert(isa_string[2] == '6' && isa_string[3] == '4');
+#endif
+
+	for (int i = 4; i < len; ++i) {
+		if (isa_string[i] == '_') {
+			const char *multi = &isa_string[++i];
+			int start = i;
+
+			while (i < len - 1 && isa_string[i] != '_')
+				++i;
+			info->func(multi, i - start, info->data);
+			if (i < len - 1)
+				--i;
+		} else {
+			info->func(&isa_string[i], 1, info->data);
+		}
+	}
+}
+
+static void isa_parse_fdt(int cpu_node, u64 hartid, void *data)
+{
+	struct isa_info *info = (struct isa_info *)data;
+	const struct fdt_property *prop;
+	int len;
+
+	if (hartid != info->hartid)
+		return;
+
+	prop = fdt_get_property(dt_fdt(), cpu_node, "riscv,isa", &len);
+	assert(prop);
+
+	isa_parse(prop->data, len, info);
+}
+
+static void isa_init_acpi(void)
+{
+	assert_msg(false, "ACPI not available");
+}
+
+void isa_init(struct thread_info *ti)
+{
+	struct isa_info info = {
+		.hartid = ti->hartid,
+		.func = isa_bit,
+		.data = ti,
+	};
+	int ret;
+
+	if (dt_available()) {
+		ret = dt_for_each_cpu_node(isa_parse_fdt, &info);
+		assert(ret == 0);
+	} else {
+		isa_init_acpi();
+	}
+}
+
+bool cpu_has_extension_name(int cpu, const char *ext)
+{
+	struct isa_info info = {
+		.hartid = cpus[cpu].hartid,
+		.func = isa_name,
+		.data = &(struct isa_check){ .ext = ext, },
+	};
+	struct isa_check *check = info.data;
+	int ret;
+
+	if (dt_available()) {
+		ret = dt_for_each_cpu_node(isa_parse_fdt, &info);
+		assert(ret == 0);
+	} else {
+		assert_msg(false, "ACPI not available");
+	}
+
+	return check->found;
+}
diff --git a/lib/riscv/processor.c b/lib/riscv/processor.c
index 2bfbd4e9b274..e0904209c0da 100644
--- a/lib/riscv/processor.c
+++ b/lib/riscv/processor.c
@@ -4,6 +4,7 @@
  */
 #include <libcflat.h>
 #include <asm/csr.h>
+#include <asm/isa.h>
 #include <asm/processor.h>
 #include <asm/setup.h>
 
@@ -58,5 +59,6 @@ void thread_info_init(void)
 	unsigned long hartid = csr_read(CSR_SSCRATCH);
 	int cpu = hartid_to_cpu(hartid);
 
+	isa_init(&cpus[cpu]);
 	csr_write(CSR_SSCRATCH, &cpus[cpu]);
 }
diff --git a/riscv/Makefile b/riscv/Makefile
index 61a1ff88d8ec..b51d9edfb792 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -30,6 +30,7 @@ cflatobjs += lib/on-cpus.o
 cflatobjs += lib/vmalloc.o
 cflatobjs += lib/riscv/bitops.o
 cflatobjs += lib/riscv/io.o
+cflatobjs += lib/riscv/isa.o
 cflatobjs += lib/riscv/mmu.o
 cflatobjs += lib/riscv/processor.o
 cflatobjs += lib/riscv/sbi.o
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 23/24] gitlab-ci: Add riscv64 tests
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (21 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 22/24] riscv: Add isa string parsing Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 24/24] MAINTAINERS: Add riscv Andrew Jones
  2024-02-02 14:22 ` [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Add build/run tests for riscv64. We would also add riscv32, but Fedora
doesn't package what we need for that.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.yml | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 273ec9a7224b..71d986e9884e 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -87,6 +87,23 @@ build-ppc64le:
      | tee results.txt
  - if grep -q FAIL results.txt ; then exit 1 ; fi
 
+# build-riscv32:
+# Fedora doesn't package a riscv32 compiler for QEMU. Oh, well.
+
+# Select 'rv64' with PROCESSOR_OVERRIDE in case QEMU is too old to have 'max'
+build-riscv64:
+ extends: .intree_template
+ script:
+ - dnf install -y qemu-system-riscv gcc-riscv64-linux-gnu
+ - ./configure --arch=riscv64 --cross-prefix=riscv64-linux-gnu-
+ - make -j2
+ - printf "FOO=foo\nBAR=bar\nBAZ=baz\nMVENDORID=0\n" >test-env
+ - PROCESSOR_OVERRIDE=rv64 ACCEL=tcg KVM_UNIT_TESTS_ENV=test-env ./run_tests.sh
+      selftest
+      sbi
+      | tee results.txt
+ - grep -q PASS results.txt && ! grep -q FAIL results.txt
+
 build-s390x:
  extends: .outoftree_template
  script:
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* [kvm-unit-tests PATCH v2 24/24] MAINTAINERS: Add riscv
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (22 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 23/24] gitlab-ci: Add riscv64 tests Andrew Jones
@ 2024-01-26 14:23 ` Andrew Jones
  2024-02-02 14:22 ` [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-01-26 14:23 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

Most of the support for riscv is now in place. Let's make it official
and start adding tests!

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
---
 MAINTAINERS | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 958735fbfd79..a2fa437da5c9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -94,6 +94,14 @@ F: powerpc/
 F: lib/powerpc/
 F: lib/ppc64/
 
+RISCV
+M: Andrew Jones <andrew.jones@linux.dev>
+S: Supported
+L: kvm-riscv@lists.infradead.org
+F: riscv/
+F: lib/riscv/
+T: https://gitlab.com/jones-drew/kvm-unit-tests.git
+
 S390X
 M: Janosch Frank <frankja@linux.ibm.com>
 M: Claudio Imbrenda <imbrenda@linux.ibm.com>
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 04/24] arm/arm64: Share cpu online, present and idle masks
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 04/24] arm/arm64: Share cpu online, present and idle masks Andrew Jones
@ 2024-02-01  8:29   ` Eric Auger
  0 siblings, 0 replies; 39+ messages in thread
From: Eric Auger @ 2024-02-01  8:29 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei

Hi Drew,

On 1/26/24 15:23, Andrew Jones wrote:
> RISC-V will also use Arm's three cpumasks. These were in smp.h,
> but they can be in cpumask.h instead, so move them there, which
> is now shared.
>
> Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Acked-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Eric Auger <eric.auger@redhat.com>


Eric
> ---
>  lib/arm/asm/smp.h | 33 ---------------------------------
>  lib/cpumask.h     | 33 +++++++++++++++++++++++++++++++++
>  2 files changed, 33 insertions(+), 33 deletions(-)
>
> diff --git a/lib/arm/asm/smp.h b/lib/arm/asm/smp.h
> index bb3e71a55e8c..b89a68dd344f 100644
> --- a/lib/arm/asm/smp.h
> +++ b/lib/arm/asm/smp.h
> @@ -23,39 +23,6 @@ extern bool cpu0_calls_idle;
>  extern void halt(void);
>  extern void do_idle(void);
>  
> -extern cpumask_t cpu_present_mask;
> -extern cpumask_t cpu_online_mask;
> -extern cpumask_t cpu_idle_mask;
> -#define cpu_present(cpu)		cpumask_test_cpu(cpu, &cpu_present_mask)
> -#define cpu_online(cpu)			cpumask_test_cpu(cpu, &cpu_online_mask)
> -#define cpu_idle(cpu)			cpumask_test_cpu(cpu, &cpu_idle_mask)
> -#define for_each_present_cpu(cpu)	for_each_cpu(cpu, &cpu_present_mask)
> -#define for_each_online_cpu(cpu)	for_each_cpu(cpu, &cpu_online_mask)
> -
> -static inline void set_cpu_present(int cpu, bool present)
> -{
> -	if (present)
> -		cpumask_set_cpu(cpu, &cpu_present_mask);
> -	else
> -		cpumask_clear_cpu(cpu, &cpu_present_mask);
> -}
> -
> -static inline void set_cpu_online(int cpu, bool online)
> -{
> -	if (online)
> -		cpumask_set_cpu(cpu, &cpu_online_mask);
> -	else
> -		cpumask_clear_cpu(cpu, &cpu_online_mask);
> -}
> -
> -static inline void set_cpu_idle(int cpu, bool idle)
> -{
> -	if (idle)
> -		cpumask_set_cpu(cpu, &cpu_idle_mask);
> -	else
> -		cpumask_clear_cpu(cpu, &cpu_idle_mask);
> -}
> -
>  extern void smp_boot_secondary(int cpu, secondary_entry_fn entry);
>  extern void on_cpu_async(int cpu, void (*func)(void *data), void *data);
>  extern void on_cpu(int cpu, void (*func)(void *data), void *data);
> diff --git a/lib/cpumask.h b/lib/cpumask.h
> index d30e14cda09e..be1919234d8e 100644
> --- a/lib/cpumask.h
> +++ b/lib/cpumask.h
> @@ -119,4 +119,37 @@ static inline int cpumask_next(int cpu, const cpumask_t *mask)
>  			(cpu) < nr_cpus; 			\
>  			(cpu) = cpumask_next(cpu, mask))
>  
> +extern cpumask_t cpu_present_mask;
> +extern cpumask_t cpu_online_mask;
> +extern cpumask_t cpu_idle_mask;
> +#define cpu_present(cpu)		cpumask_test_cpu(cpu, &cpu_present_mask)
> +#define cpu_online(cpu)			cpumask_test_cpu(cpu, &cpu_online_mask)
> +#define cpu_idle(cpu)			cpumask_test_cpu(cpu, &cpu_idle_mask)
> +#define for_each_present_cpu(cpu)	for_each_cpu(cpu, &cpu_present_mask)
> +#define for_each_online_cpu(cpu)	for_each_cpu(cpu, &cpu_online_mask)
> +
> +static inline void set_cpu_present(int cpu, bool present)
> +{
> +	if (present)
> +		cpumask_set_cpu(cpu, &cpu_present_mask);
> +	else
> +		cpumask_clear_cpu(cpu, &cpu_present_mask);
> +}
> +
> +static inline void set_cpu_online(int cpu, bool online)
> +{
> +	if (online)
> +		cpumask_set_cpu(cpu, &cpu_online_mask);
> +	else
> +		cpumask_clear_cpu(cpu, &cpu_online_mask);
> +}
> +
> +static inline void set_cpu_idle(int cpu, bool idle)
> +{
> +	if (idle)
> +		cpumask_set_cpu(cpu, &cpu_idle_mask);
> +	else
> +		cpumask_clear_cpu(cpu, &cpu_idle_mask);
> +}
> +
>  #endif /* _CPUMASK_H_ */


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 03/24] arm/arm64: Move cpumask.h to common lib
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 03/24] arm/arm64: Move cpumask.h to common lib Andrew Jones
@ 2024-02-01  8:29   ` Eric Auger
  0 siblings, 0 replies; 39+ messages in thread
From: Eric Auger @ 2024-02-01  8:29 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei



On 1/26/24 15:23, Andrew Jones wrote:
> RISC-V will also make use of cpumask.h, so move it to the arch-common
> directory.
>
> Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  lib/arm/asm/gic-v2.h        | 2 +-
>  lib/arm/asm/gic-v3.h        | 2 +-
>  lib/arm/asm/gic.h           | 2 +-
>  lib/arm/asm/smp.h           | 2 +-
>  lib/arm/mmu.c               | 2 +-
>  lib/arm/smp.c               | 2 +-
>  lib/arm64/asm/cpumask.h     | 1 -
>  lib/{arm/asm => }/cpumask.h | 9 ++++-----
>  8 files changed, 10 insertions(+), 12 deletions(-)
>  delete mode 100644 lib/arm64/asm/cpumask.h
>  rename lib/{arm/asm => }/cpumask.h (94%)
>
> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
> index 1fcfd43c8075..ff11afb15d30 100644
> --- a/lib/arm/asm/gic-v2.h
> +++ b/lib/arm/asm/gic-v2.h
> @@ -18,7 +18,7 @@
>  #define GICC_IAR_INT_ID_MASK		0x3ff
>  
>  #ifndef __ASSEMBLY__
> -#include <asm/cpumask.h>
> +#include <cpumask.h>
>  
>  struct gicv2_data {
>  	void *dist_base;
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> index b4ce130e56c6..a1cc62a298b8 100644
> --- a/lib/arm/asm/gic-v3.h
> +++ b/lib/arm/asm/gic-v3.h
> @@ -67,10 +67,10 @@
>  #include <asm/arch_gicv3.h>
>  
>  #ifndef __ASSEMBLY__
> +#include <cpumask.h>
>  #include <asm/setup.h>
>  #include <asm/processor.h>
>  #include <asm/delay.h>
> -#include <asm/cpumask.h>
>  #include <asm/smp.h>
>  #include <asm/io.h>
>  
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> index 189840014b02..dc8cc18c0fbd 100644
> --- a/lib/arm/asm/gic.h
> +++ b/lib/arm/asm/gic.h
> @@ -47,7 +47,7 @@
>  #define SPI(irq)			((irq) + GIC_FIRST_SPI)
>  
>  #ifndef __ASSEMBLY__
> -#include <asm/cpumask.h>
> +#include <cpumask.h>
>  
>  enum gic_irq_state {
>  	GIC_IRQ_STATE_INACTIVE,
> diff --git a/lib/arm/asm/smp.h b/lib/arm/asm/smp.h
> index dee4c1a883e7..bb3e71a55e8c 100644
> --- a/lib/arm/asm/smp.h
> +++ b/lib/arm/asm/smp.h
> @@ -5,8 +5,8 @@
>   *
>   * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
> +#include <cpumask.h>
>  #include <asm/thread_info.h>
> -#include <asm/cpumask.h>
>  
>  #define smp_processor_id()		(current_thread_info()->cpu)
>  
> diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
> index 2f4ec815a35d..b16517a3200d 100644
> --- a/lib/arm/mmu.c
> +++ b/lib/arm/mmu.c
> @@ -5,9 +5,9 @@
>   *
>   * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
> +#include <cpumask.h>
>  #include <asm/setup.h>
>  #include <asm/thread_info.h>
> -#include <asm/cpumask.h>
>  #include <asm/mmu.h>
>  #include <asm/setup.h>
>  #include <asm/page.h>
> diff --git a/lib/arm/smp.c b/lib/arm/smp.c
> index 1d470d1aab45..78fc1656cefa 100644
> --- a/lib/arm/smp.c
> +++ b/lib/arm/smp.c
> @@ -7,9 +7,9 @@
>   */
>  #include <libcflat.h>
>  #include <auxinfo.h>
> +#include <cpumask.h>
>  #include <asm/thread_info.h>
>  #include <asm/spinlock.h>
> -#include <asm/cpumask.h>
>  #include <asm/barrier.h>
>  #include <asm/mmu.h>
>  #include <asm/psci.h>
> diff --git a/lib/arm64/asm/cpumask.h b/lib/arm64/asm/cpumask.h
> deleted file mode 100644
> index d1421e7abe31..000000000000
> --- a/lib/arm64/asm/cpumask.h
> +++ /dev/null
> @@ -1 +0,0 @@
> -#include "../../arm/asm/cpumask.h"
> diff --git a/lib/arm/asm/cpumask.h b/lib/cpumask.h
> similarity index 94%
> rename from lib/arm/asm/cpumask.h
> rename to lib/cpumask.h
> index 3fa57bfb17c6..d30e14cda09e 100644
> --- a/lib/arm/asm/cpumask.h
> +++ b/lib/cpumask.h
> @@ -1,12 +1,11 @@
> -#ifndef _ASMARM_CPUMASK_H_
> -#define _ASMARM_CPUMASK_H_
> +/* SPDX-License-Identifier: GPL-2.0-only */
>  /*
>   * Simple cpumask implementation
>   *
>   * Copyright (C) 2015, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> - *
> - * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
> +#ifndef _CPUMASK_H_
> +#define _CPUMASK_H_
>  #include <asm/setup.h>
>  #include <bitops.h>
>  
> @@ -120,4 +119,4 @@ static inline int cpumask_next(int cpu, const cpumask_t *mask)
>  			(cpu) < nr_cpus; 			\
>  			(cpu) = cpumask_next(cpu, mask))
>  
> -#endif /* _ASMARM_CPUMASK_H_ */
> +#endif /* _CPUMASK_H_ */


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 02/24] riscv: Initial port, hello world
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 02/24] riscv: Initial port, hello world Andrew Jones
@ 2024-02-01  8:29   ` Eric Auger
  2024-02-01 14:07     ` Andrew Jones
  0 siblings, 1 reply; 39+ messages in thread
From: Eric Auger @ 2024-02-01  8:29 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei



On 1/26/24 15:23, Andrew Jones wrote:
> Add the minimal amount of code possible in order to launch a first
> test, which just prints "Hello, world" using the expected UART
> address of the QEMU virt machine. Add files, stubs, and some support,
> such as barriers and MMIO read/write along the way in order to
> satisfy the compiler. Basically everything is either copied from
> the arm64 port of kvm-unit-tests, or at least inspired by it, and,
> in that case, the RISC-V Linux kernel code was copied.
>
> Run with
>   qemu-system-riscv64 -nographic -M virt -kernel riscv/selftest.flat
>
> and then go to the monitor (ctrl-a c) and use 'q' to quit, since
> the unit test will just hang after printing hello world and the
> exit code.
>
> Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
> Acked-by: Thomas Huth <thuth@redhat.com>
> ---
>  configure                   | 14 ++++++
>  lib/riscv/.gitignore        |  1 +
>  lib/riscv/asm-offsets.c     |  6 +++
>  lib/riscv/asm/asm-offsets.h |  1 +
>  lib/riscv/asm/barrier.h     | 13 ++++++
>  lib/riscv/asm/csr.h         |  7 +++
>  lib/riscv/asm/io.h          | 78 +++++++++++++++++++++++++++++++
>  lib/riscv/asm/page.h        |  7 +++
>  lib/riscv/asm/setup.h       |  7 +++
>  lib/riscv/asm/spinlock.h    |  7 +++
>  lib/riscv/asm/stack.h       |  9 ++++
>  lib/riscv/io.c              | 44 ++++++++++++++++++
>  lib/riscv/setup.c           | 12 +++++
>  riscv/Makefile              | 83 +++++++++++++++++++++++++++++++++
>  riscv/cstart.S              | 92 +++++++++++++++++++++++++++++++++++++
>  riscv/flat.lds              | 75 ++++++++++++++++++++++++++++++
>  riscv/selftest.c            | 13 ++++++
>  17 files changed, 469 insertions(+)
>  create mode 100644 lib/riscv/.gitignore
>  create mode 100644 lib/riscv/asm-offsets.c
>  create mode 100644 lib/riscv/asm/asm-offsets.h
>  create mode 100644 lib/riscv/asm/barrier.h
>  create mode 100644 lib/riscv/asm/csr.h
>  create mode 100644 lib/riscv/asm/io.h
>  create mode 100644 lib/riscv/asm/page.h
>  create mode 100644 lib/riscv/asm/setup.h
>  create mode 100644 lib/riscv/asm/spinlock.h
>  create mode 100644 lib/riscv/asm/stack.h
>  create mode 100644 lib/riscv/io.c
>  create mode 100644 lib/riscv/setup.c
>  create mode 100644 riscv/Makefile
>  create mode 100644 riscv/cstart.S
>  create mode 100644 riscv/flat.lds
>  create mode 100644 riscv/selftest.c
>
> diff --git a/configure b/configure
> index ada6512702a1..05e6702eab06 100755
> --- a/configure
> +++ b/configure
> @@ -200,6 +200,11 @@ arch_name=$arch
>  [ "$arch_name" = "arm64" ] && arch_name="aarch64"
>  arch_libdir=$arch
>  
> +if [ "$arch" = "riscv" ]; then
> +    echo "riscv32 or riscv64 must be specified"
> +    exit 1
> +fi
> +
>  if [ -z "$target" ]; then
>      target="qemu"
>  else
> @@ -307,6 +312,9 @@ elif [ "$arch" = "ppc64" ]; then
>          echo "You must provide endianness (big or little)!"
>          usage
>      fi
> +elif [ "$arch" = "riscv32" ] || [ "$arch" = "riscv64" ]; then
> +    testdir=riscv
> +    arch_libdir=riscv
>  else
>      testdir=$arch
>  fi
> @@ -438,6 +446,12 @@ cat <<EOF >> lib/config.h
>  #define CONFIG_ERRATA_FORCE ${errata_force}
>  #define CONFIG_PAGE_SIZE _AC(${page_size}, UL)
>  
> +EOF
> +elif [ "$arch" = "riscv32" ] || [ "$arch" = "riscv64" ]; then
> +cat <<EOF >> lib/config.h
> +
> +#define CONFIG_UART_EARLY_BASE 0x10000000
> +
>  EOF
>  fi
>  echo "#endif" >> lib/config.h
> diff --git a/lib/riscv/.gitignore b/lib/riscv/.gitignore
> new file mode 100644
> index 000000000000..82da12e6bd4e
> --- /dev/null
> +++ b/lib/riscv/.gitignore
> @@ -0,0 +1 @@
> +/asm-offsets.[hs]
> diff --git a/lib/riscv/asm-offsets.c b/lib/riscv/asm-offsets.c
> new file mode 100644
> index 000000000000..4a74df9e4a09
> --- /dev/null
> +++ b/lib/riscv/asm-offsets.c
> @@ -0,0 +1,6 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +
> +int main(void)
> +{
> +	return 0;
> +}
> diff --git a/lib/riscv/asm/asm-offsets.h b/lib/riscv/asm/asm-offsets.h
> new file mode 100644
> index 000000000000..d370ee36a182
> --- /dev/null
> +++ b/lib/riscv/asm/asm-offsets.h
> @@ -0,0 +1 @@
> +#include <generated/asm-offsets.h>
> diff --git a/lib/riscv/asm/barrier.h b/lib/riscv/asm/barrier.h
> new file mode 100644
> index 000000000000..c6a09066b2c7
> --- /dev/null
> +++ b/lib/riscv/asm/barrier.h
> @@ -0,0 +1,13 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _ASMRISCV_BARRIER_H_
> +#define _ASMRISCV_BARRIER_H_
> +
> +#define RISCV_FENCE(p, s) \
> +	__asm__ __volatile__ ("fence " #p "," #s : : : "memory")
> +
> +/* These barriers need to enforce ordering on both devices or memory. */
> +#define mb()		RISCV_FENCE(iorw,iorw)
> +#define rmb()		RISCV_FENCE(ir,ir)
> +#define wmb()		RISCV_FENCE(ow,ow)
> +
> +#endif /* _ASMRISCV_BARRIER_H_ */
> diff --git a/lib/riscv/asm/csr.h b/lib/riscv/asm/csr.h
> new file mode 100644
> index 000000000000..5c4f2de34f64
> --- /dev/null
> +++ b/lib/riscv/asm/csr.h
> @@ -0,0 +1,7 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _ASMRISCV_CSR_H_
> +#define _ASMRISCV_CSR_H_
> +
> +#define CSR_SSCRATCH		0x140
> +
> +#endif /* _ASMRISCV_CSR_H_ */
> diff --git a/lib/riscv/asm/io.h b/lib/riscv/asm/io.h
> new file mode 100644
> index 000000000000..d2eb3acc9fda
> --- /dev/null
> +++ b/lib/riscv/asm/io.h
> @@ -0,0 +1,78 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * From Linux arch/riscv/include/asm/mmio.h
> + */
> +#ifndef _ASMRISCV_IO_H_
> +#define _ASMRISCV_IO_H_
> +#include <libcflat.h>
> +
> +#define __iomem
> +
> +/* Generic IO read/write.  These perform native-endian accesses. */
> +#define __raw_writeb __raw_writeb
> +static inline void __raw_writeb(u8 val, volatile void __iomem *addr)
> +{
> +	asm volatile("sb %0, 0(%1)" : : "r" (val), "r" (addr));
> +}
> +
> +#define __raw_writew __raw_writew
> +static inline void __raw_writew(u16 val, volatile void __iomem *addr)
> +{
> +	asm volatile("sh %0, 0(%1)" : : "r" (val), "r" (addr));
> +}
> +
> +#define __raw_writel __raw_writel
> +static inline void __raw_writel(u32 val, volatile void __iomem *addr)
> +{
> +	asm volatile("sw %0, 0(%1)" : : "r" (val), "r" (addr));
> +}
> +
> +#ifdef CONFIG_64BIT
> +#define __raw_writeq __raw_writeq
> +static inline void __raw_writeq(u64 val, volatile void __iomem *addr)
> +{
> +	asm volatile("sd %0, 0(%1)" : : "r" (val), "r" (addr));
> +}
> +#endif
> +
> +#define __raw_readb __raw_readb
> +static inline u8 __raw_readb(const volatile void __iomem *addr)
> +{
> +	u8 val;
> +
> +	asm volatile("lb %0, 0(%1)" : "=r" (val) : "r" (addr));
> +	return val;
> +}
> +
> +#define __raw_readw __raw_readw
> +static inline u16 __raw_readw(const volatile void __iomem *addr)
> +{
> +	u16 val;
> +
> +	asm volatile("lh %0, 0(%1)" : "=r" (val) : "r" (addr));
> +	return val;
> +}
> +
> +#define __raw_readl __raw_readl
> +static inline u32 __raw_readl(const volatile void __iomem *addr)
> +{
> +	u32 val;
> +
> +	asm volatile("lw %0, 0(%1)" : "=r" (val) : "r" (addr));
> +	return val;
> +}
> +
> +#ifdef CONFIG_64BIT
> +#define __raw_readq __raw_readq
> +static inline u64 __raw_readq(const volatile void __iomem *addr)
> +{
> +	u64 val;
> +
> +	asm volatile("ld %0, 0(%1)" : "=r" (val) : "r" (addr));
> +	return val;
> +}
> +#endif
> +
> +#include <asm-generic/io.h>
> +
> +#endif /* _ASMRISCV_IO_H_ */
> diff --git a/lib/riscv/asm/page.h b/lib/riscv/asm/page.h
> new file mode 100644
> index 000000000000..7d7c9191605a
> --- /dev/null
> +++ b/lib/riscv/asm/page.h
> @@ -0,0 +1,7 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _ASMRISCV_PAGE_H_
> +#define _ASMRISCV_PAGE_H_
> +
> +#include <asm-generic/page.h>
> +
> +#endif /* _ASMRISCV_PAGE_H_ */
> diff --git a/lib/riscv/asm/setup.h b/lib/riscv/asm/setup.h
> new file mode 100644
> index 000000000000..385455f341cc
> --- /dev/null
> +++ b/lib/riscv/asm/setup.h
> @@ -0,0 +1,7 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _ASMRISCV_SETUP_H_
> +#define _ASMRISCV_SETUP_H_
> +
> +void setup(const void *fdt, phys_addr_t freemem_start);
> +
> +#endif /* _ASMRISCV_SETUP_H_ */
> diff --git a/lib/riscv/asm/spinlock.h b/lib/riscv/asm/spinlock.h
> new file mode 100644
> index 000000000000..6e2b3009abf3
> --- /dev/null
> +++ b/lib/riscv/asm/spinlock.h
> @@ -0,0 +1,7 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _ASMRISCV_SPINLOCK_H_
> +#define _ASMRISCV_SPINLOCK_H_
> +
> +#include <asm-generic/spinlock.h>
> +
> +#endif /* _ASMRISCV_SPINLOCK_H_ */
> diff --git a/lib/riscv/asm/stack.h b/lib/riscv/asm/stack.h
> new file mode 100644
> index 000000000000..d081d0716d7b
> --- /dev/null
> +++ b/lib/riscv/asm/stack.h
> @@ -0,0 +1,9 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _ASMRISCV_STACK_H_
> +#define _ASMRISCV_STACK_H_
> +
> +#ifndef _STACK_H_
> +#error Do not directly include <asm/stack.h>. Just use <stack.h>.
> +#endif
> +
> +#endif
> diff --git a/lib/riscv/io.c b/lib/riscv/io.c
> new file mode 100644
> index 000000000000..3cfc235d19a6
> --- /dev/null
> +++ b/lib/riscv/io.c
> @@ -0,0 +1,44 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Each architecture must implement puts() and exit() with the I/O
> + * devices exposed from QEMU, e.g. ns16550a.
> + *
> + * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
> + */
> +#include <libcflat.h>
> +#include <config.h>
> +#include <asm/io.h>
> +#include <asm/spinlock.h>
> +
> +/*
> + * Use this guess for the uart base in order to make an attempt at
> + * having earlier printf support. We'll overwrite it with the real
> + * base address that we read from the device tree later. This is
> + * the address we expect the virtual machine manager to put in
> + * its generated device tree.
> + */
> +#define UART_EARLY_BASE ((u8 *)(unsigned long)CONFIG_UART_EARLY_BASE)
> +static volatile u8 *uart0_base = UART_EARLY_BASE;
> +static struct spinlock uart_lock;
> +
> +void puts(const char *s)
> +{
> +	spin_lock(&uart_lock);
> +	while (*s)
> +		writeb(*s++, uart0_base);
> +	spin_unlock(&uart_lock);
> +}
> +
> +/*
> + * Defining halt to take 'code' as an argument guarantees that it will
> + * be in a0 when we halt. That gives us a final chance to see the exit
> + * status while inspecting the halted unit test state.
> + */
> +void halt(int code);
> +
> +void exit(int code)
> +{
> +	printf("\nEXIT: STATUS=%d\n", ((code) << 1) | 1);
> +	halt(code);
> +	__builtin_unreachable();
> +}
> diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c
> new file mode 100644
> index 000000000000..8937525ccb7f
> --- /dev/null
> +++ b/lib/riscv/setup.c
> @@ -0,0 +1,12 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Initialize machine setup information and I/O.
> + *
> + * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
> + */
> +#include <libcflat.h>
> +#include <asm/setup.h>
> +
> +void setup(const void *fdt, phys_addr_t freemem_start)
> +{
> +}
> diff --git a/riscv/Makefile b/riscv/Makefile
> new file mode 100644
> index 000000000000..f2e89f0e4c38
> --- /dev/null
> +++ b/riscv/Makefile
> @@ -0,0 +1,83 @@
> +#
> +# riscv makefile
> +#
> +# Authors: Andrew Jones <ajones@ventanamicro.com>
> +#
> +
> +ifeq ($(CONFIG_EFI),y)
> +exe = efi
> +else
> +exe = flat
> +endif
> +
> +tests =
> +tests += $(TEST_DIR)/selftest.$(exe)
> +#tests += $(TEST_DIR)/sieve.$(exe)
> +
> +all: $(tests)
> +
> +$(TEST_DIR)/sieve.elf: AUXFLAGS = 0x1
> +
> +cstart.o = $(TEST_DIR)/cstart.o
> +
> +cflatobjs += lib/riscv/io.o
> +cflatobjs += lib/riscv/setup.o
> +
> +########################################
> +
> +OBJDIRS += lib/riscv
> +FLATLIBS = $(libcflat) $(LIBFDT_archive)
> +
> +AUXFLAGS ?= 0x0
> +
> +# stack.o relies on frame pointers.
> +KEEP_FRAME_POINTER := y
> +
> +# We want to keep intermediate files
> +.PRECIOUS: %.elf %.o
> +
> +define arch_elf_check =
> +	$(if $(shell ! $(READELF) -rW $(1) >&/dev/null && echo "nok"),
> +		$(error $(shell $(READELF) -rW $(1) 2>&1)))
> +	$(if $(shell $(READELF) -rW $(1) | grep R_ | grep -v R_RISCV_RELATIVE),
> +		$(error $(1) has unsupported reloc types))
> +endef
> +
> +ifeq ($(ARCH),riscv64)
> +CFLAGS += -DCONFIG_64BIT
> +endif
> +CFLAGS += -DCONFIG_RELOC
> +CFLAGS += -mcmodel=medany
> +CFLAGS += -mstrict-align
> +CFLAGS += -std=gnu99
> +CFLAGS += -ffreestanding
> +CFLAGS += -O2
> +CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt
> +
> +asm-offsets = lib/riscv/asm-offsets.h
> +include $(SRCDIR)/scripts/asm-offsets.mak
> +
> +ifeq ($(CONFIG_EFI),y)
> +	# TODO
> +else
> +%.elf: LDFLAGS += -pie -n -z notext
> +%.elf: %.o $(FLATLIBS) $(SRCDIR)/riscv/flat.lds $(cstart.o)
> +	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) $(SRCDIR)/lib/auxinfo.c \
> +		-DPROGNAME=\"$(notdir $(@:.elf=.flat))\" -DAUXFLAGS=$(AUXFLAGS)
> +	$(LD) $(LDFLAGS) -o $@ -T $(SRCDIR)/riscv/flat.lds \
> +		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
> +	$(RM) $(@:.elf=.aux.o)
> +	@chmod a-x $@
> +
> +%.flat: %.elf
> +	$(call arch_elf_check, $^)
> +	$(OBJCOPY) -O binary $^ $@
> +	@chmod a-x $@
> +endif
> +
> +generated-files = $(asm-offsets)
> +$(tests:.$(exe)=.o) $(cstart.o) $(cflatobjs): $(generated-files)
> +
> +arch_clean: asm_offsets_clean
> +	$(RM) $(TEST_DIR)/*.{o,flat,elf,so,efi,debug} \
> +	      $(TEST_DIR)/.*.d $(TEST_DIR)/efi/.*.d lib/riscv/.*.d
> diff --git a/riscv/cstart.S b/riscv/cstart.S
> new file mode 100644
> index 000000000000..a28d75e8021e
> --- /dev/null
> +++ b/riscv/cstart.S
> @@ -0,0 +1,92 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Boot entry point and assembler functions for riscv.
> + *
> + * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
> + */
> +#include <asm/csr.h>
> +
> +.macro zero_range, tmp1, tmp2
For my education what were the tmp3/4 args used for on arm?
> +9998:	beq	\tmp1, \tmp2, 9997f
> +	sd	zero, 0(\tmp1)
> +	addi	\tmp1, \tmp1, 8
> +	j	9998b
> +9997:
> +.endm
> +
> +	.section .init
> +
> +/*
> + * The hartid of the current core is in a0
> + * The address of the devicetree is in a1
> + *
> + * See Linux kernel doc Documentation/riscv/boot.rst
> + */
> +.global start
> +start:
> +	/*
> +	 * Stash the hartid in scratch and shift the dtb
> +	 * address into a0
> +	 */
> +	csrw	CSR_SSCRATCH, a0
> +	mv	a0, a1
> +
> +	/*
> +	 * Update all R_RISCV_RELATIVE relocations using the table
> +	 * of Elf64_Rela entries between reloc_start/end. The build
> +	 * will not emit other relocation types.
> +	 *
> +	 * struct Elf64_Rela {
> +	 * 	uint64_t r_offset;
> +	 * 	uint64_t r_info;
> +	 * 	int64_t  r_addend;
> +	 * }
> +	 */
> +	la	a1, reloc_start
> +	la	a2, reloc_end
> +	la	a3, start			// base
> +1:
> +	bge	a1, a2, 1f
> +	ld	a4, 0(a1)			// r_offset
> +	ld	a5, 16(a1)			// r_addend
> +	add	a4, a3, a4			// addr = base + r_offset
> +	add	a5, a3, a5			// val = base + r_addend
> +	sd	a5, 0(a4)			// *addr = val
> +	addi	a1, a1, 24
> +	j	1b
> +
> +1:
> +	/* zero BSS */
> +	la	a1, bss
> +	la	a2, ebss
> +	zero_range a1, a2
> +
> +	/* zero and set up stack */
> +	la	sp, stacktop
> +	li	a1, -8192
> +	add	a1, sp, a1
> +	zero_range a1, sp
> +
> +	/* set up exception handling */
> +	//TODO
> +
> +	/* complete setup */
> +	la	a1, stacktop			// a1 is the base of free memory
> +	call	setup				// a0 is the addr of the dtb
> +
> +	/* run the test */
> +	la	a0, __argc
> +	ld	a0, 0(a0)
> +	la	a1, __argv
> +	la	a2, __environ
> +	call	main
> +	call	exit
> +	j	halt
> +
> +	.text
> +
> +.balign 4
> +.global halt
> +halt:
> +1:	wfi
> +	j	1b
> diff --git a/riscv/flat.lds b/riscv/flat.lds
> new file mode 100644
> index 000000000000..d4853f82ba1c
> --- /dev/null
> +++ b/riscv/flat.lds
> @@ -0,0 +1,75 @@
> +/*
> + * init::start will pass stacktop to setup() as the base of free memory.
> + * setup() will then move the FDT and initrd to that base before calling
> + * mem_init(). With those movements and this linker script, we'll end up
> + * having the following memory layout:
> + *
> + *    +----------------------+   <-- top of physical memory
> + *    |                      |
> + *    ~                      ~
> + *    |                      |
> + *    +----------------------+   <-- top of initrd
> + *    |                      |
> + *    +----------------------+   <-- top of FDT
> + *    |                      |
> + *    +----------------------+   <-- top of cpu0's stack
> + *    |                      |
> + *    +----------------------+   <-- top of text/data/bss sections
> + *    |                      |
> + *    |                      |
> + *    +----------------------+   <-- load address
> + *    |                      |
> + *    +----------------------+   <-- physical address 0x0
> + */
> +
> +PHDRS
> +{
> +    text PT_LOAD FLAGS(5);
> +    data PT_LOAD FLAGS(6);
> +}
> +
> +SECTIONS
> +{
> +    PROVIDE(_text = .);
> +    .text : { *(.init) *(.text) *(.text.*) } :text
> +    . = ALIGN(4K);
> +    PROVIDE(_etext = .);
> +
> +    PROVIDE(reloc_start = .);
> +    .rela.dyn : { *(.rela.dyn) }
> +    PROVIDE(reloc_end = .);
> +    .dynsym   : { *(.dynsym) }
> +    .dynstr   : { *(.dynstr) }
> +    .hash     : { *(.hash) }
> +    .gnu.hash : { *(.gnu.hash) }
> +    .got      : { *(.got) *(.got.plt) }
> +    .eh_frame : { *(.eh_frame) }
> +
> +    .rodata   : { *(.rodata*) } :data
> +    .data     : { *(.data) } :data
> +    . = ALIGN(16);
> +    PROVIDE(bss = .);
> +    .bss      : { *(.bss) }
> +    . = ALIGN(16);
> +    PROVIDE(ebss = .);
> +    . = ALIGN(4K);
> +    PROVIDE(edata = .);
> +
> +    /*
> +     * stack depth is 8K and sp must be 16 byte aligned
> +     * sp must always be strictly less than the true stacktop
> +     */
> +    . += 12K;
> +    . = ALIGN(4K);
> +    PROVIDE(stackptr = . - 16);
> +    PROVIDE(stacktop = .);
> +
> +    /DISCARD/ : {
> +        *(.note*)
> +        *(.interp)
> +        *(.comment)
> +        *(.dynamic)
> +    }
> +}
> +
> +ENTRY(start)
> diff --git a/riscv/selftest.c b/riscv/selftest.c
> new file mode 100644
> index 000000000000..88afa732649e
> --- /dev/null
> +++ b/riscv/selftest.c
> @@ -0,0 +1,13 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Test the framework itself. These tests confirm that setup works.
> + *
> + * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
> + */
> +#include <libcflat.h>
> +
> +int main(void)
> +{
> +	puts("Hello, world\n");
> +	return 0;
> +}
Thanks

Eric


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 01/24] configure: Add ARCH_LIBDIR
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 01/24] configure: Add ARCH_LIBDIR Andrew Jones
@ 2024-02-01  8:29   ` Eric Auger
  0 siblings, 0 replies; 39+ messages in thread
From: Eric Auger @ 2024-02-01  8:29 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei

Hi Drew,

On 1/26/24 15:23, Andrew Jones wrote:
> Prepare for an architecture which will share the same lib/$ARCH
> directory, but be configured with different arch names for different
> bit widths, i.e. riscv32 -> lib/riscv and riscv64 -> lib/riscv.
>
> Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  Makefile  | 2 +-
>  configure | 2 ++
>  2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index 602910dda11b..4f35fffc685b 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -10,7 +10,7 @@ include config.mak
>  VPATH = $(SRCDIR)
>  
>  libdirs-get = $(shell [ -d "lib/$(1)" ] && echo "lib/$(1) lib/$(1)/asm")
> -ARCH_LIBDIRS := $(call libdirs-get,$(ARCH)) $(call libdirs-get,$(TEST_DIR))
> +ARCH_LIBDIRS := $(call libdirs-get,$(ARCH_LIBDIR)) $(call libdirs-get,$(TEST_DIR))
>  OBJDIRS := $(ARCH_LIBDIRS)
>  
>  DESTDIR := $(PREFIX)/share/kvm-unit-tests/
> diff --git a/configure b/configure
> index 6ee9b27a6af2..ada6512702a1 100755
> --- a/configure
> +++ b/configure
> @@ -198,6 +198,7 @@ fi
>  arch_name=$arch
>  [ "$arch" = "aarch64" ] && arch="arm64"
>  [ "$arch_name" = "arm64" ] && arch_name="aarch64"
> +arch_libdir=$arch
>  
>  if [ -z "$target" ]; then
>      target="qemu"
> @@ -391,6 +392,7 @@ PREFIX=$prefix
>  HOST=$host
>  ARCH=$arch
>  ARCH_NAME=$arch_name
> +ARCH_LIBDIR=$arch_libdir
>  PROCESSOR=$processor
>  CC=$cross_prefix$cc
>  CFLAGS=$cflags


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 11/24] arm/arm64: Generalize wfe/sev names in smp.c
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 11/24] arm/arm64: Generalize wfe/sev names in smp.c Andrew Jones
@ 2024-02-01  9:22   ` Eric Auger
  0 siblings, 0 replies; 39+ messages in thread
From: Eric Auger @ 2024-02-01  9:22 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei



On 1/26/24 15:23, Andrew Jones wrote:
> Most of Arm's on_cpus() implementation can be shared by any
> architecture which has the possible, present, and idle cpumasks,
> like riscv does. Rename the exceptions (wfe/sve) to something
> more generic in order to prepare to share the functions.
>
> Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
> Acked-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric

> ---
>  lib/arm/asm/smp.h |  4 ++++
>  lib/arm/smp.c     | 16 ++++++++--------
>  2 files changed, 12 insertions(+), 8 deletions(-)
>
> diff --git a/lib/arm/asm/smp.h b/lib/arm/asm/smp.h
> index b89a68dd344f..9f6d839ab568 100644
> --- a/lib/arm/asm/smp.h
> +++ b/lib/arm/asm/smp.h
> @@ -6,6 +6,7 @@
>   * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
>  #include <cpumask.h>
> +#include <asm/barrier.h>
>  #include <asm/thread_info.h>
>  
>  #define smp_processor_id()		(current_thread_info()->cpu)
> @@ -18,6 +19,9 @@ struct secondary_data {
>  };
>  extern struct secondary_data secondary_data;
>  
> +#define smp_wait_for_event()	wfe()
> +#define smp_send_event()	sev()
> +
>  extern bool cpu0_calls_idle;
>  
>  extern void halt(void);
> diff --git a/lib/arm/smp.c b/lib/arm/smp.c
> index 78fc1656cefa..c00fda2efb03 100644
> --- a/lib/arm/smp.c
> +++ b/lib/arm/smp.c
> @@ -45,7 +45,7 @@ secondary_entry_fn secondary_cinit(void)
>  	 */
>  	entry = secondary_data.entry;
>  	set_cpu_online(ti->cpu, true);
> -	sev();
> +	smp_send_event();
>  
>  	/*
>  	 * Return to the assembly stub, allowing entry to be called
> @@ -65,7 +65,7 @@ static void __smp_boot_secondary(int cpu, secondary_entry_fn entry)
>  	assert(ret == 0);
>  
>  	while (!cpu_online(cpu))
> -		wfe();
> +		smp_wait_for_event();
>  }
>  
>  void smp_boot_secondary(int cpu, secondary_entry_fn entry)
> @@ -122,7 +122,7 @@ static void cpu_wait(int cpu)
>  	cpumask_set_cpu(me, &on_cpu_info[cpu].waiters);
>  	deadlock_check(me, cpu);
>  	while (!cpu_idle(cpu))
> -		wfe();
> +		smp_wait_for_event();
>  	cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
>  }
>  
> @@ -134,17 +134,17 @@ void do_idle(void)
>  		cpu0_calls_idle = true;
>  
>  	set_cpu_idle(cpu, true);
> -	sev();
> +	smp_send_event();
>  
>  	for (;;) {
>  		while (cpu_idle(cpu))
> -			wfe();
> +			smp_wait_for_event();
>  		smp_rmb();
>  		on_cpu_info[cpu].func(on_cpu_info[cpu].data);
>  		on_cpu_info[cpu].func = NULL;
>  		smp_wmb();
>  		set_cpu_idle(cpu, true);
> -		sev();
> +		smp_send_event();
>  	}
>  }
>  
> @@ -174,7 +174,7 @@ void on_cpu_async(int cpu, void (*func)(void *data), void *data)
>  	on_cpu_info[cpu].data = data;
>  	spin_unlock(&lock);
>  	set_cpu_idle(cpu, false);
> -	sev();
> +	smp_send_event();
>  }
>  
>  void on_cpu(int cpu, void (*func)(void *data), void *data)
> @@ -201,7 +201,7 @@ void on_cpus(void (*func)(void *data), void *data)
>  		deadlock_check(me, cpu);
>  	}
>  	while (cpumask_weight(&cpu_idle_mask) < nr_cpus - 1)
> -		wfe();
> +		smp_wait_for_event();
>  	for_each_present_cpu(cpu)
>  		cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
>  }


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 12/24] arm/arm64: Remove spinlocks from on_cpu_async
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 12/24] arm/arm64: Remove spinlocks from on_cpu_async Andrew Jones
@ 2024-02-01  9:34   ` Eric Auger
  0 siblings, 0 replies; 39+ messages in thread
From: Eric Auger @ 2024-02-01  9:34 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei



On 1/26/24 15:23, Andrew Jones wrote:
> Remove spinlocks from on_cpu_async() by pulling some of their
> use into a new function and also by narrowing the locking to a
> single on_cpu_info structure by introducing yet another cpumask.
>
> Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
> Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  lib/arm/asm/smp.h |  4 +++-
>  lib/arm/smp.c     | 37 ++++++++++++++++++++++++++++---------
>  2 files changed, 31 insertions(+), 10 deletions(-)
>
> diff --git a/lib/arm/asm/smp.h b/lib/arm/asm/smp.h
> index 9f6d839ab568..f0c0f97a19f8 100644
> --- a/lib/arm/asm/smp.h
> +++ b/lib/arm/asm/smp.h
> @@ -27,9 +27,11 @@ extern bool cpu0_calls_idle;
>  extern void halt(void);
>  extern void do_idle(void);
>  
> -extern void smp_boot_secondary(int cpu, secondary_entry_fn entry);
>  extern void on_cpu_async(int cpu, void (*func)(void *data), void *data);
>  extern void on_cpu(int cpu, void (*func)(void *data), void *data);
>  extern void on_cpus(void (*func)(void *data), void *data);
>  
> +extern void smp_boot_secondary(int cpu, secondary_entry_fn entry);
> +extern void smp_boot_secondary_nofail(int cpu, secondary_entry_fn entry);
> +
>  #endif /* _ASMARM_SMP_H_ */
> diff --git a/lib/arm/smp.c b/lib/arm/smp.c
> index c00fda2efb03..e0872a1a72c2 100644
> --- a/lib/arm/smp.c
> +++ b/lib/arm/smp.c
> @@ -76,12 +76,32 @@ void smp_boot_secondary(int cpu, secondary_entry_fn entry)
>  	spin_unlock(&lock);
>  }
>  
> +void smp_boot_secondary_nofail(int cpu, secondary_entry_fn entry)
> +{
> +	spin_lock(&lock);
> +	if (!cpu_online(cpu))
> +		__smp_boot_secondary(cpu, entry);
> +	spin_unlock(&lock);
> +}
> +
>  struct on_cpu_info {
>  	void (*func)(void *data);
>  	void *data;
>  	cpumask_t waiters;
>  };
>  static struct on_cpu_info on_cpu_info[NR_CPUS];
> +static cpumask_t on_cpu_info_lock;
> +
> +static bool get_on_cpu_info(int cpu)
> +{
> +	return !cpumask_test_and_set_cpu(cpu, &on_cpu_info_lock);
> +}
> +
> +static void put_on_cpu_info(int cpu)
> +{
> +	int ret = cpumask_test_and_clear_cpu(cpu, &on_cpu_info_lock);
> +	assert(ret);
> +}
>  
>  static void __deadlock_check(int cpu, const cpumask_t *waiters, bool *found)
>  {
> @@ -158,22 +178,21 @@ void on_cpu_async(int cpu, void (*func)(void *data), void *data)
>  	assert_msg(cpu != 0 || cpu0_calls_idle, "Waiting on CPU0, which is unlikely to idle. "
>  						"If this is intended set cpu0_calls_idle=1");
>  
> -	spin_lock(&lock);
> -	if (!cpu_online(cpu))
> -		__smp_boot_secondary(cpu, do_idle);
> -	spin_unlock(&lock);
> +	smp_boot_secondary_nofail(cpu, do_idle);
>  
>  	for (;;) {
>  		cpu_wait(cpu);
> -		spin_lock(&lock);
> -		if ((volatile void *)on_cpu_info[cpu].func == NULL)
> -			break;
> -		spin_unlock(&lock);
> +		if (get_on_cpu_info(cpu)) {
> +			if ((volatile void *)on_cpu_info[cpu].func == NULL)
> +				break;
> +			put_on_cpu_info(cpu);
> +		}
>  	}
> +
>  	on_cpu_info[cpu].func = func;
>  	on_cpu_info[cpu].data = data;
> -	spin_unlock(&lock);
>  	set_cpu_idle(cpu, false);
> +	put_on_cpu_info(cpu);
>  	smp_send_event();
>  }
>  


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 13/24] arm/arm64: Share on_cpus
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 13/24] arm/arm64: Share on_cpus Andrew Jones
@ 2024-02-01  9:36   ` Eric Auger
  0 siblings, 0 replies; 39+ messages in thread
From: Eric Auger @ 2024-02-01  9:36 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei



On 1/26/24 15:23, Andrew Jones wrote:
> Now that the previous patches have cleaned up Arm's on_cpus
> implementation we can move it to the common lib where riscv
> will share it.
>
> Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
> Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  arm/Makefile.common |   1 +
>  lib/arm/asm/smp.h   |   8 +--
>  lib/arm/smp.c       | 144 -----------------------------------------
>  lib/on-cpus.c       | 154 ++++++++++++++++++++++++++++++++++++++++++++
>  lib/on-cpus.h       |  14 ++++
>  5 files changed, 170 insertions(+), 151 deletions(-)
>  create mode 100644 lib/on-cpus.c
>  create mode 100644 lib/on-cpus.h
>
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index 5214c8acdab3..dc92a7433350 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -43,6 +43,7 @@ cflatobjs += lib/vmalloc.o
>  cflatobjs += lib/alloc.o
>  cflatobjs += lib/devicetree.o
>  cflatobjs += lib/migrate.o
> +cflatobjs += lib/on-cpus.o
>  cflatobjs += lib/pci.o
>  cflatobjs += lib/pci-host-generic.o
>  cflatobjs += lib/pci-testdev.o
> diff --git a/lib/arm/asm/smp.h b/lib/arm/asm/smp.h
> index f0c0f97a19f8..2e1dc27f7bd8 100644
> --- a/lib/arm/asm/smp.h
> +++ b/lib/arm/asm/smp.h
> @@ -6,6 +6,7 @@
>   * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
>  #include <cpumask.h>
> +#include <on-cpus.h>
>  #include <asm/barrier.h>
>  #include <asm/thread_info.h>
>  
> @@ -22,14 +23,7 @@ extern struct secondary_data secondary_data;
>  #define smp_wait_for_event()	wfe()
>  #define smp_send_event()	sev()
>  
> -extern bool cpu0_calls_idle;
> -
>  extern void halt(void);
> -extern void do_idle(void);
> -
> -extern void on_cpu_async(int cpu, void (*func)(void *data), void *data);
> -extern void on_cpu(int cpu, void (*func)(void *data), void *data);
> -extern void on_cpus(void (*func)(void *data), void *data);
>  
>  extern void smp_boot_secondary(int cpu, secondary_entry_fn entry);
>  extern void smp_boot_secondary_nofail(int cpu, secondary_entry_fn entry);
> diff --git a/lib/arm/smp.c b/lib/arm/smp.c
> index e0872a1a72c2..0207ca2a7d57 100644
> --- a/lib/arm/smp.c
> +++ b/lib/arm/smp.c
> @@ -10,13 +10,10 @@
>  #include <cpumask.h>
>  #include <asm/thread_info.h>
>  #include <asm/spinlock.h>
> -#include <asm/barrier.h>
>  #include <asm/mmu.h>
>  #include <asm/psci.h>
>  #include <asm/smp.h>
>  
> -bool cpu0_calls_idle;
> -
>  cpumask_t cpu_present_mask;
>  cpumask_t cpu_online_mask;
>  cpumask_t cpu_idle_mask;
> @@ -83,144 +80,3 @@ void smp_boot_secondary_nofail(int cpu, secondary_entry_fn entry)
>  		__smp_boot_secondary(cpu, entry);
>  	spin_unlock(&lock);
>  }
> -
> -struct on_cpu_info {
> -	void (*func)(void *data);
> -	void *data;
> -	cpumask_t waiters;
> -};
> -static struct on_cpu_info on_cpu_info[NR_CPUS];
> -static cpumask_t on_cpu_info_lock;
> -
> -static bool get_on_cpu_info(int cpu)
> -{
> -	return !cpumask_test_and_set_cpu(cpu, &on_cpu_info_lock);
> -}
> -
> -static void put_on_cpu_info(int cpu)
> -{
> -	int ret = cpumask_test_and_clear_cpu(cpu, &on_cpu_info_lock);
> -	assert(ret);
> -}
> -
> -static void __deadlock_check(int cpu, const cpumask_t *waiters, bool *found)
> -{
> -	int i;
> -
> -	for_each_cpu(i, waiters) {
> -		if (i == cpu) {
> -			printf("CPU%d", cpu);
> -			*found = true;
> -			return;
> -		}
> -		__deadlock_check(cpu, &on_cpu_info[i].waiters, found);
> -		if (*found) {
> -			printf(" <=> CPU%d", i);
> -			return;
> -		}
> -	}
> -}
> -
> -static void deadlock_check(int me, int cpu)
> -{
> -	bool found = false;
> -
> -	__deadlock_check(cpu, &on_cpu_info[me].waiters, &found);
> -	if (found) {
> -		printf(" <=> CPU%d deadlock detectd\n", me);
> -		assert(0);
> -	}
> -}
> -
> -static void cpu_wait(int cpu)
> -{
> -	int me = smp_processor_id();
> -
> -	if (cpu == me)
> -		return;
> -
> -	cpumask_set_cpu(me, &on_cpu_info[cpu].waiters);
> -	deadlock_check(me, cpu);
> -	while (!cpu_idle(cpu))
> -		smp_wait_for_event();
> -	cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
> -}
> -
> -void do_idle(void)
> -{
> -	int cpu = smp_processor_id();
> -
> -	if (cpu == 0)
> -		cpu0_calls_idle = true;
> -
> -	set_cpu_idle(cpu, true);
> -	smp_send_event();
> -
> -	for (;;) {
> -		while (cpu_idle(cpu))
> -			smp_wait_for_event();
> -		smp_rmb();
> -		on_cpu_info[cpu].func(on_cpu_info[cpu].data);
> -		on_cpu_info[cpu].func = NULL;
> -		smp_wmb();
> -		set_cpu_idle(cpu, true);
> -		smp_send_event();
> -	}
> -}
> -
> -void on_cpu_async(int cpu, void (*func)(void *data), void *data)
> -{
> -	if (cpu == smp_processor_id()) {
> -		func(data);
> -		return;
> -	}
> -
> -	assert_msg(cpu != 0 || cpu0_calls_idle, "Waiting on CPU0, which is unlikely to idle. "
> -						"If this is intended set cpu0_calls_idle=1");
> -
> -	smp_boot_secondary_nofail(cpu, do_idle);
> -
> -	for (;;) {
> -		cpu_wait(cpu);
> -		if (get_on_cpu_info(cpu)) {
> -			if ((volatile void *)on_cpu_info[cpu].func == NULL)
> -				break;
> -			put_on_cpu_info(cpu);
> -		}
> -	}
> -
> -	on_cpu_info[cpu].func = func;
> -	on_cpu_info[cpu].data = data;
> -	set_cpu_idle(cpu, false);
> -	put_on_cpu_info(cpu);
> -	smp_send_event();
> -}
> -
> -void on_cpu(int cpu, void (*func)(void *data), void *data)
> -{
> -	on_cpu_async(cpu, func, data);
> -	cpu_wait(cpu);
> -}
> -
> -void on_cpus(void (*func)(void *data), void *data)
> -{
> -	int cpu, me = smp_processor_id();
> -
> -	for_each_present_cpu(cpu) {
> -		if (cpu == me)
> -			continue;
> -		on_cpu_async(cpu, func, data);
> -	}
> -	func(data);
> -
> -	for_each_present_cpu(cpu) {
> -		if (cpu == me)
> -			continue;
> -		cpumask_set_cpu(me, &on_cpu_info[cpu].waiters);
> -		deadlock_check(me, cpu);
> -	}
> -	while (cpumask_weight(&cpu_idle_mask) < nr_cpus - 1)
> -		smp_wait_for_event();
> -	for_each_present_cpu(cpu)
> -		cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
> -}
> diff --git a/lib/on-cpus.c b/lib/on-cpus.c
> new file mode 100644
> index 000000000000..aed70f7b27b2
> --- /dev/null
> +++ b/lib/on-cpus.c
> @@ -0,0 +1,154 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * on_cpus() support based on cpumasks.
> + *
> + * Copyright (C) 2015, Red Hat Inc, Andrew Jones <drjones@redhat.com>
> + */
> +#include <libcflat.h>
> +#include <cpumask.h>
> +#include <on-cpus.h>
> +#include <asm/barrier.h>
> +#include <asm/smp.h>
> +
> +bool cpu0_calls_idle;
> +
> +struct on_cpu_info {
> +	void (*func)(void *data);
> +	void *data;
> +	cpumask_t waiters;
> +};
> +static struct on_cpu_info on_cpu_info[NR_CPUS];
> +static cpumask_t on_cpu_info_lock;
> +
> +static bool get_on_cpu_info(int cpu)
> +{
> +	return !cpumask_test_and_set_cpu(cpu, &on_cpu_info_lock);
> +}
> +
> +static void put_on_cpu_info(int cpu)
> +{
> +	int ret = cpumask_test_and_clear_cpu(cpu, &on_cpu_info_lock);
> +	assert(ret);
> +}
> +
> +static void __deadlock_check(int cpu, const cpumask_t *waiters, bool *found)
> +{
> +	int i;
> +
> +	for_each_cpu(i, waiters) {
> +		if (i == cpu) {
> +			printf("CPU%d", cpu);
> +			*found = true;
> +			return;
> +		}
> +		__deadlock_check(cpu, &on_cpu_info[i].waiters, found);
> +		if (*found) {
> +			printf(" <=> CPU%d", i);
> +			return;
> +		}
> +	}
> +}
> +
> +static void deadlock_check(int me, int cpu)
> +{
> +	bool found = false;
> +
> +	__deadlock_check(cpu, &on_cpu_info[me].waiters, &found);
> +	if (found) {
> +		printf(" <=> CPU%d deadlock detectd\n", me);
> +		assert(0);
> +	}
> +}
> +
> +static void cpu_wait(int cpu)
> +{
> +	int me = smp_processor_id();
> +
> +	if (cpu == me)
> +		return;
> +
> +	cpumask_set_cpu(me, &on_cpu_info[cpu].waiters);
> +	deadlock_check(me, cpu);
> +	while (!cpu_idle(cpu))
> +		smp_wait_for_event();
> +	cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
> +}
> +
> +void do_idle(void)
> +{
> +	int cpu = smp_processor_id();
> +
> +	if (cpu == 0)
> +		cpu0_calls_idle = true;
> +
> +	set_cpu_idle(cpu, true);
> +	smp_send_event();
> +
> +	for (;;) {
> +		while (cpu_idle(cpu))
> +			smp_wait_for_event();
> +		smp_rmb();
> +		on_cpu_info[cpu].func(on_cpu_info[cpu].data);
> +		on_cpu_info[cpu].func = NULL;
> +		smp_wmb();
> +		set_cpu_idle(cpu, true);
> +		smp_send_event();
> +	}
> +}
> +
> +void on_cpu_async(int cpu, void (*func)(void *data), void *data)
> +{
> +	if (cpu == smp_processor_id()) {
> +		func(data);
> +		return;
> +	}
> +
> +	assert_msg(cpu != 0 || cpu0_calls_idle, "Waiting on CPU0, which is unlikely to idle. "
> +						"If this is intended set cpu0_calls_idle=1");
> +
> +	smp_boot_secondary_nofail(cpu, do_idle);
> +
> +	for (;;) {
> +		cpu_wait(cpu);
> +		if (get_on_cpu_info(cpu)) {
> +			if ((volatile void *)on_cpu_info[cpu].func == NULL)
> +				break;
> +			put_on_cpu_info(cpu);
> +		}
> +	}
> +
> +	on_cpu_info[cpu].func = func;
> +	on_cpu_info[cpu].data = data;
> +	set_cpu_idle(cpu, false);
> +	put_on_cpu_info(cpu);
> +	smp_send_event();
> +}
> +
> +void on_cpu(int cpu, void (*func)(void *data), void *data)
> +{
> +	on_cpu_async(cpu, func, data);
> +	cpu_wait(cpu);
> +}
> +
> +void on_cpus(void (*func)(void *data), void *data)
> +{
> +	int cpu, me = smp_processor_id();
> +
> +	for_each_present_cpu(cpu) {
> +		if (cpu == me)
> +			continue;
> +		on_cpu_async(cpu, func, data);
> +	}
> +	func(data);
> +
> +	for_each_present_cpu(cpu) {
> +		if (cpu == me)
> +			continue;
> +		cpumask_set_cpu(me, &on_cpu_info[cpu].waiters);
> +		deadlock_check(me, cpu);
> +	}
> +	while (cpumask_weight(&cpu_idle_mask) < nr_cpus - 1)
> +		smp_wait_for_event();
> +	for_each_present_cpu(cpu)
> +		cpumask_clear_cpu(me, &on_cpu_info[cpu].waiters);
> +}
> diff --git a/lib/on-cpus.h b/lib/on-cpus.h
> new file mode 100644
> index 000000000000..41103b0245c7
> --- /dev/null
> +++ b/lib/on-cpus.h
> @@ -0,0 +1,14 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _ON_CPUS_H_
> +#define _ON_CPUS_H_
> +#include <stdbool.h>
> +
> +extern bool cpu0_calls_idle;
> +
> +void do_idle(void);
> +
> +void on_cpu_async(int cpu, void (*func)(void *data), void *data);
> +void on_cpu(int cpu, void (*func)(void *data), void *data);
> +void on_cpus(void (*func)(void *data), void *data);
> +
> +#endif /* _ON_CPUS_H_ */


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 21/24] lib: Add strcasecmp and strncasecmp
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 21/24] lib: Add strcasecmp and strncasecmp Andrew Jones
@ 2024-02-01  9:45   ` Eric Auger
  0 siblings, 0 replies; 39+ messages in thread
From: Eric Auger @ 2024-02-01  9:45 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei



On 1/26/24 15:23, Andrew Jones wrote:
> We'll soon want a case insensitive string comparison. Add toupper()
> and tolower() too (the latter gets used by the new string functions).
>
> Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric
> ---
>  lib/ctype.h  | 10 ++++++++++
>  lib/string.c | 14 ++++++++++++++
>  lib/string.h |  2 ++
>  3 files changed, 26 insertions(+)
>
> diff --git a/lib/ctype.h b/lib/ctype.h
> index 48a9c16300f8..45c96f111e19 100644
> --- a/lib/ctype.h
> +++ b/lib/ctype.h
> @@ -37,4 +37,14 @@ static inline int isspace(int c)
>          return c == ' ' || c == '\t' || c == '\r' || c == '\n' || c == '\v' || c == '\f';
>  }
>  
> +static inline int toupper(int c)
> +{
> +	return islower(c) ? c - 'a' + 'A' : c;
> +}
> +
> +static inline int tolower(int c)
> +{
> +	return isupper(c) ? c - 'A' + 'a' : c;
> +}
> +
>  #endif /* _CTYPE_H_ */
> diff --git a/lib/string.c b/lib/string.c
> index 6d8a6380db92..ab6a724a3144 100644
> --- a/lib/string.c
> +++ b/lib/string.c
> @@ -54,11 +54,25 @@ int strncmp(const char *a, const char *b, size_t n)
>  	return 0;
>  }
>  
> +int strncasecmp(const char *a, const char *b, size_t n)
> +{
> +	for (; n--; ++a, ++b)
> +		if (tolower(*a) != tolower(*b) || *a == '\0')
> +			return tolower(*a) - tolower(*b);
> +
> +	return 0;
> +}
> +
>  int strcmp(const char *a, const char *b)
>  {
>  	return strncmp(a, b, SIZE_MAX);
>  }
>  
> +int strcasecmp(const char *a, const char *b)
> +{
> +	return strncasecmp(a, b, SIZE_MAX);
> +}
> +
>  char *strchr(const char *s, int c)
>  {
>  	while (*s != (char)c)
> diff --git a/lib/string.h b/lib/string.h
> index 758dca8af36a..a28d75641530 100644
> --- a/lib/string.h
> +++ b/lib/string.h
> @@ -15,6 +15,8 @@ extern char *strcat(char *dest, const char *src);
>  extern char *strcpy(char *dest, const char *src);
>  extern int strcmp(const char *a, const char *b);
>  extern int strncmp(const char *a, const char *b, size_t n);
> +int strcasecmp(const char *a, const char *b);
> +int strncasecmp(const char *a, const char *b, size_t n);
>  extern char *strchr(const char *s, int c);
>  extern char *strrchr(const char *s, int c);
>  extern char *strchrnul(const char *s, int c);


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 16/24] arm/arm64: Share memregions
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 16/24] arm/arm64: Share memregions Andrew Jones
@ 2024-02-01 12:03   ` Eric Auger
  2024-02-01 14:21     ` Andrew Jones
  0 siblings, 1 reply; 39+ messages in thread
From: Eric Auger @ 2024-02-01 12:03 UTC (permalink / raw)
  To: Andrew Jones, kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei

Hi Drew,

On 1/26/24 15:23, Andrew Jones wrote:
> arm/arm64 (and to a small extent powerpc) have memory regions which
> get built from hardware descriptions (DT/ACPI/EFI) and then used to
> build page tables. Move memregions to common code, tweaking the API
> a bit at the same time, e.g. change 'mem_region' to 'memregions'.
> The biggest change is there is now a default number of memory regions
> which, if too small, should be overridden at setup time with a new
> init function, memregions_init().
>
> Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
> Acked-by: Thomas Huth <thuth@redhat.com>
> ---
>  arm/Makefile.common |  1 +
>  arm/selftest.c      |  3 +-
>  lib/arm/asm/setup.h | 14 -------
>  lib/arm/mmu.c       |  1 +
>  lib/arm/setup.c     | 93 ++++++++++-----------------------------------
>  lib/memregions.c    | 82 +++++++++++++++++++++++++++++++++++++++
>  lib/memregions.h    | 29 ++++++++++++++
>  7 files changed, 136 insertions(+), 87 deletions(-)
>  create mode 100644 lib/memregions.c
>  create mode 100644 lib/memregions.h
>
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index dc92a7433350..4dfd570fa59e 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -42,6 +42,7 @@ cflatobjs += lib/alloc_page.o
>  cflatobjs += lib/vmalloc.o
>  cflatobjs += lib/alloc.o
>  cflatobjs += lib/devicetree.o
> +cflatobjs += lib/memregions.o
>  cflatobjs += lib/migrate.o
>  cflatobjs += lib/on-cpus.o
>  cflatobjs += lib/pci.o
> diff --git a/arm/selftest.c b/arm/selftest.c
> index 9f459ed3d571..007d2309d01c 100644
> --- a/arm/selftest.c
> +++ b/arm/selftest.c
> @@ -8,6 +8,7 @@
>  #include <libcflat.h>
>  #include <util.h>
>  #include <devicetree.h>
> +#include <memregions.h>
>  #include <vmalloc.h>
>  #include <asm/setup.h>
>  #include <asm/ptrace.h>
> @@ -90,7 +91,7 @@ static bool check_pabt_init(void)
>  			highest_end = PAGE_ALIGN(r->end);
>  	}
>  
> -	if (mem_region_get_flags(highest_end) != MR_F_UNKNOWN)
> +	if (memregions_get_flags(highest_end) != MR_F_UNKNOWN)
>  		return false;
>  
>  	vaddr = (unsigned long)vmap(highest_end, PAGE_SIZE);
> diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
> index 060691165a20..9f8ef82efb90 100644
> --- a/lib/arm/asm/setup.h
> +++ b/lib/arm/asm/setup.h
> @@ -13,22 +13,8 @@
>  extern u64 cpus[NR_CPUS];	/* per-cpu IDs (MPIDRs) */
>  extern int nr_cpus;
>  
> -#define MR_F_IO			(1U << 0)
> -#define MR_F_CODE		(1U << 1)
> -#define MR_F_RESERVED		(1U << 2)
> -#define MR_F_UNKNOWN		(1U << 31)
> -
> -struct mem_region {
> -	phys_addr_t start;
> -	phys_addr_t end;
> -	unsigned int flags;
> -};
> -extern struct mem_region *mem_regions;
>  extern phys_addr_t __phys_offset, __phys_end;
>  
> -extern struct mem_region *mem_region_find(phys_addr_t paddr);
> -extern unsigned int mem_region_get_flags(phys_addr_t paddr);
> -
>  #define PHYS_OFFSET		(__phys_offset)
>  #define PHYS_END		(__phys_end)
>  
> diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c
> index b16517a3200d..eb5e82a95f06 100644
> --- a/lib/arm/mmu.c
> +++ b/lib/arm/mmu.c
> @@ -6,6 +6,7 @@
>   * This work is licensed under the terms of the GNU LGPL, version 2.
>   */
>  #include <cpumask.h>
> +#include <memregions.h>
>  #include <asm/setup.h>
>  #include <asm/thread_info.h>
>  #include <asm/mmu.h>
> diff --git a/lib/arm/setup.c b/lib/arm/setup.c
> index b6fc453e5b31..0382cbdaf5a1 100644
> --- a/lib/arm/setup.c
> +++ b/lib/arm/setup.c
> @@ -13,6 +13,7 @@
>  #include <libcflat.h>
>  #include <libfdt/libfdt.h>
>  #include <devicetree.h>
> +#include <memregions.h>
>  #include <alloc.h>
>  #include <alloc_phys.h>
>  #include <alloc_page.h>
> @@ -31,7 +32,7 @@
>  
>  #define MAX_DT_MEM_REGIONS	16
>  #define NR_EXTRA_MEM_REGIONS	64
> -#define NR_INITIAL_MEM_REGIONS	(MAX_DT_MEM_REGIONS + NR_EXTRA_MEM_REGIONS)
> +#define NR_MEM_REGIONS		(MAX_DT_MEM_REGIONS + NR_EXTRA_MEM_REGIONS)
>  
>  extern unsigned long _text, _etext, _data, _edata;
>  
> @@ -41,8 +42,7 @@ u32 initrd_size;
>  u64 cpus[NR_CPUS] = { [0 ... NR_CPUS-1] = (u64)~0 };
>  int nr_cpus;
>  
> -static struct mem_region __initial_mem_regions[NR_INITIAL_MEM_REGIONS + 1];
> -struct mem_region *mem_regions = __initial_mem_regions;
> +static struct mem_region arm_mem_regions[NR_MEM_REGIONS + 1];
>  phys_addr_t __phys_offset = (phys_addr_t)-1, __phys_end = 0;
>  
>  extern void exceptions_init(void);
> @@ -114,68 +114,14 @@ static void cpu_init(void)
>  	set_cpu_online(0, true);
>  }
>  
> -static void mem_region_add(struct mem_region *r)
> +static void arm_memregions_add_assumed(void)
>  {
> -	struct mem_region *r_next = mem_regions;
> -	int i = 0;
> -
> -	for (; r_next->end; ++r_next, ++i)
> -		;
> -	assert(i < NR_INITIAL_MEM_REGIONS);
> -
> -	*r_next = *r;
> -}
> -
> -static void mem_regions_add_dt_regions(void)
> -{
> -	struct dt_pbus_reg regs[MAX_DT_MEM_REGIONS];
> -	int nr_regs, i;
> -
> -	nr_regs = dt_get_memory_params(regs, MAX_DT_MEM_REGIONS);
> -	assert(nr_regs > 0);
> -
> -	for (i = 0; i < nr_regs; ++i) {
> -		mem_region_add(&(struct mem_region){
> -			.start = regs[i].addr,
> -			.end = regs[i].addr + regs[i].size,
> -		});
> -	}
> -}
> -
> -struct mem_region *mem_region_find(phys_addr_t paddr)
> -{
> -	struct mem_region *r;
> -
> -	for (r = mem_regions; r->end; ++r)
> -		if (paddr >= r->start && paddr < r->end)
> -			return r;
> -	return NULL;
> -}
> -
> -unsigned int mem_region_get_flags(phys_addr_t paddr)
> -{
> -	struct mem_region *r = mem_region_find(paddr);
> -	return r ? r->flags : MR_F_UNKNOWN;
> -}
> -
> -static void mem_regions_add_assumed(void)
> -{
> -	phys_addr_t code_end = (phys_addr_t)(unsigned long)&_etext;
> -	struct mem_region *r;
> -
> -	r = mem_region_find(code_end - 1);
> -	assert(r);
> +	struct mem_region *code, *data;
>  
>  	/* Split the region with the code into two regions; code and data */
> -	mem_region_add(&(struct mem_region){
> -		.start = code_end,
> -		.end = r->end,
> -	});
> -	*r = (struct mem_region){
> -		.start = r->start,
> -		.end = code_end,
> -		.flags = MR_F_CODE,
> -	};
> +	memregions_split((unsigned long)&_etext, &code, &data);
> +	assert(code);
> +	code->flags |= MR_F_CODE;
I think this would deserve to be split into several patches, esp. this
change in the implementation of

mem_regions_add_assumed and the init changes. At the moment this is pretty difficult to review

Eric

>  
>  	/*
>  	 * mach-virt I/O regions:
> @@ -183,10 +129,10 @@ static void mem_regions_add_assumed(void)
>  	 *   - 512M at 256G (arm64, arm uses highmem=off)
>  	 *   - 512G at 512G (arm64, arm uses highmem=off)
>  	 */
> -	mem_region_add(&(struct mem_region){ 0, (1ul << 30), MR_F_IO });
> +	memregions_add(&(struct mem_region){ 0, (1ul << 30), MR_F_IO });
>  #ifdef __aarch64__
> -	mem_region_add(&(struct mem_region){ (1ul << 38), (1ul << 38) | (1ul << 29), MR_F_IO });
> -	mem_region_add(&(struct mem_region){ (1ul << 39), (1ul << 40), MR_F_IO });
> +	memregions_add(&(struct mem_region){ (1ul << 38), (1ul << 38) | (1ul << 29), MR_F_IO });
> +	memregions_add(&(struct mem_region){ (1ul << 39), (1ul << 40), MR_F_IO });
>  #endif
>  }
>  
> @@ -197,7 +143,7 @@ static void mem_init(phys_addr_t freemem_start)
>  		.start = (phys_addr_t)-1,
>  	};
>  
> -	freemem = mem_region_find(freemem_start);
> +	freemem = memregions_find(freemem_start);
>  	assert(freemem && !(freemem->flags & (MR_F_IO | MR_F_CODE)));
>  
>  	for (r = mem_regions; r->end; ++r) {
> @@ -212,9 +158,9 @@ static void mem_init(phys_addr_t freemem_start)
>  	mem.end &= PHYS_MASK;
>  
>  	/* Check for holes */
> -	r = mem_region_find(mem.start);
> +	r = memregions_find(mem.start);
>  	while (r && r->end != mem.end)
> -		r = mem_region_find(r->end);
> +		r = memregions_find(r->end);
>  	assert(r);
>  
>  	/* Ensure our selected freemem range is somewhere in our full range */
> @@ -263,8 +209,9 @@ void setup(const void *fdt, phys_addr_t freemem_start)
>  		freemem += initrd_size;
>  	}
>  
> -	mem_regions_add_dt_regions();
> -	mem_regions_add_assumed();
> +	memregions_init(arm_mem_regions, NR_MEM_REGIONS);
> +	memregions_add_dt_regions(MAX_DT_MEM_REGIONS);
> +	arm_memregions_add_assumed();
>  	mem_init(PAGE_ALIGN((unsigned long)freemem));
>  
>  	psci_set_conduit();
> @@ -371,7 +318,7 @@ static efi_status_t efi_mem_init(efi_bootinfo_t *efi_bootinfo)
>  				assert(edata <= r.end);
>  				r.flags = MR_F_CODE;
>  				r.end = data;
> -				mem_region_add(&r);
> +				memregions_add(&r);
>  				r.start = data;
>  				r.end = tmp;
>  				r.flags = 0;
> @@ -393,7 +340,7 @@ static efi_status_t efi_mem_init(efi_bootinfo_t *efi_bootinfo)
>  			if (r.end > __phys_end)
>  				__phys_end = r.end;
>  		}
> -		mem_region_add(&r);
> +		memregions_add(&r);
>  	}
>  	if (fdt) {
>  		/* Move the FDT to the base of free memory */
> @@ -439,6 +386,8 @@ efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo)
>  
>  	exceptions_init();
>  
> +	memregions_init(arm_mem_regions, NR_MEM_REGIONS);
> +
>  	status = efi_mem_init(efi_bootinfo);
>  	if (status != EFI_SUCCESS) {
>  		printf("Failed to initialize memory: ");
> diff --git a/lib/memregions.c b/lib/memregions.c
> new file mode 100644
> index 000000000000..96de86b27333
> --- /dev/null
> +++ b/lib/memregions.c
> @@ -0,0 +1,82 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +#include <libcflat.h>
> +#include <devicetree.h>
> +#include <memregions.h>
> +
> +static struct mem_region __initial_mem_regions[NR_INITIAL_MEM_REGIONS + 1];
> +static size_t nr_regions = NR_INITIAL_MEM_REGIONS;
> +
> +struct mem_region *mem_regions = __initial_mem_regions;
> +
> +void memregions_init(struct mem_region regions[], size_t nr)
> +{
> +	mem_regions = regions;
> +	nr_regions = nr;
> +}
> +
> +struct mem_region *memregions_add(struct mem_region *r)
> +{
> +	struct mem_region *r_next = mem_regions;
> +	int i = 0;
> +
> +	for (; r_next->end; ++r_next, ++i)
> +		;
> +	assert(i < nr_regions);
> +
> +	*r_next = *r;
> +
> +	return r_next;
> +}
> +
> +struct mem_region *memregions_find(phys_addr_t paddr)
> +{
> +	struct mem_region *r;
> +
> +	for (r = mem_regions; r->end; ++r)
> +		if (paddr >= r->start && paddr < r->end)
> +			return r;
> +	return NULL;
> +}
> +
> +uint32_t memregions_get_flags(phys_addr_t paddr)
> +{
> +	struct mem_region *r = memregions_find(paddr);
> +
> +	return r ? r->flags : MR_F_UNKNOWN;
> +}
> +
> +void memregions_split(phys_addr_t addr, struct mem_region **r1, struct mem_region **r2)
> +{
> +	*r1 = memregions_find(addr);
> +	assert(*r1);
> +
> +	if ((*r1)->start == addr) {
> +		*r2 = *r1;
> +		*r1 = NULL;
> +		return;
> +	}
> +
> +	*r2 = memregions_add(&(struct mem_region){
> +		.start = addr,
> +		.end = (*r1)->end,
> +		.flags = (*r1)->flags,
> +	});
> +
> +	(*r1)->end = addr;
> +}
> +
> +void memregions_add_dt_regions(size_t max_nr)
> +{
> +	struct dt_pbus_reg regs[max_nr];
> +	int nr_regs, i;
> +
> +	nr_regs = dt_get_memory_params(regs, max_nr);
> +	assert(nr_regs > 0);
> +
> +	for (i = 0; i < nr_regs; ++i) {
> +		memregions_add(&(struct mem_region){
> +			.start = regs[i].addr,
> +			.end = regs[i].addr + regs[i].size,
> +		});
> +	}
> +}
> diff --git a/lib/memregions.h b/lib/memregions.h
> new file mode 100644
> index 000000000000..9a8e33182fe5
> --- /dev/null
> +++ b/lib/memregions.h
> @@ -0,0 +1,29 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _MEMREGIONS_H_
> +#define _MEMREGIONS_H_
> +#include <libcflat.h>
> +#include <bitops.h>
> +
> +#define NR_INITIAL_MEM_REGIONS		8
> +
> +#define MR_F_IO				BIT(0)
> +#define MR_F_CODE			BIT(1)
> +#define MR_F_RESERVED			BIT(2)
> +#define MR_F_UNKNOWN			BIT(31)
> +
> +struct mem_region {
> +	phys_addr_t start;
> +	phys_addr_t end;
> +	uint32_t flags;
> +};
> +
> +extern struct mem_region *mem_regions;
> +
> +void memregions_init(struct mem_region regions[], size_t nr);
> +struct mem_region *memregions_add(struct mem_region *r);
> +struct mem_region *memregions_find(phys_addr_t paddr);
> +uint32_t memregions_get_flags(phys_addr_t paddr);
> +void memregions_split(phys_addr_t addr, struct mem_region **r1, struct mem_region **r2);
> +void memregions_add_dt_regions(size_t max_nr);
> +
> +#endif /* _MEMREGIONS_H_ */


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Re: [kvm-unit-tests PATCH v2 02/24] riscv: Initial port, hello world
  2024-02-01  8:29   ` Eric Auger
@ 2024-02-01 14:07     ` Andrew Jones
  0 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-02-01 14:07 UTC (permalink / raw)
  To: Eric Auger
  Cc: kvm, kvm-riscv, kvmarm, ajones, anup, atishp, pbonzini, thuth,
	alexandru.elisei

On Thu, Feb 01, 2024 at 09:29:36AM +0100, Eric Auger wrote:
...
> > --- /dev/null
> > +++ b/riscv/cstart.S
> > @@ -0,0 +1,92 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Boot entry point and assembler functions for riscv.
> > + *
> > + * Copyright (C) 2023, Ventana Micro Systems Inc., Andrew Jones <ajones@ventanamicro.com>
> > + */
> > +#include <asm/csr.h>
> > +
> > +.macro zero_range, tmp1, tmp2
> For my education what were the tmp3/4 args used for on arm?
> > +9998:	beq	\tmp1, \tmp2, 9997f
> > +	sd	zero, 0(\tmp1)
> > +	addi	\tmp1, \tmp1, 8
> > +	j	9998b
> > +9997:
> > +.endm
> > +

arm doesn't have a zero register like arm64 and riscv32/64 have, so at
least one extra tmp register is needed to hold the zero stored to the
memory. We use two tmp registers because arm has a 'strd' instruction
allowing us to write two at once, as long as the first register is
an even-numbered register and the second is the immediately following
odd-numbered register. (We should probably write a comment about the
purpose and even/odd constraints of tmp3/4 above the zero_range macro
in arm/cstart.S)

Thanks,
drew

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Re: [kvm-unit-tests PATCH v2 16/24] arm/arm64: Share memregions
  2024-02-01 12:03   ` Eric Auger
@ 2024-02-01 14:21     ` Andrew Jones
  2024-02-01 17:46       ` Eric Auger
  0 siblings, 1 reply; 39+ messages in thread
From: Andrew Jones @ 2024-02-01 14:21 UTC (permalink / raw)
  To: Eric Auger
  Cc: kvm, kvm-riscv, kvmarm, ajones, anup, atishp, pbonzini, thuth,
	alexandru.elisei

On Thu, Feb 01, 2024 at 01:03:54PM +0100, Eric Auger wrote:
> Hi Drew,
> 
> On 1/26/24 15:23, Andrew Jones wrote:
...
> > -static void mem_regions_add_assumed(void)
> > -{
> > -	phys_addr_t code_end = (phys_addr_t)(unsigned long)&_etext;
> > -	struct mem_region *r;
> > -
> > -	r = mem_region_find(code_end - 1);
> > -	assert(r);
> > +	struct mem_region *code, *data;
> >  
> >  	/* Split the region with the code into two regions; code and data */
> > -	mem_region_add(&(struct mem_region){
> > -		.start = code_end,
> > -		.end = r->end,
> > -	});
> > -	*r = (struct mem_region){
> > -		.start = r->start,
> > -		.end = code_end,
> > -		.flags = MR_F_CODE,
> > -	};
> > +	memregions_split((unsigned long)&_etext, &code, &data);
> > +	assert(code);
> > +	code->flags |= MR_F_CODE;
> I think this would deserve to be split into several patches, esp. this
> change in the implementation of
> 
> mem_regions_add_assumed and the init changes. At the moment this is pretty difficult to review
>

Darn, you called me out on this one :-) I had a feeling I should split out
the introduction of memregions_split(), since it was sneaking a bit more
into the patch than just code motion as advertised, but then I hoped I
get away with putting a bit more burden on the reviewer instead. If you
haven't already convinced yourself that the new function is equivalent to
the old code, then I'll respin with the splitting and also create a new
patch for the 'mem_region' to 'memregions' rename while at it (so there
will be three patches instead of one). But, if you're already good with
it, then I'll leave it as is, since patch splitting is a pain...

Thanks,
drew

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 08/24] riscv: Add riscv32 support
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 08/24] riscv: Add riscv32 support Andrew Jones
@ 2024-02-01 15:24   ` Andrew Jones
  0 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-02-01 15:24 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

On Fri, Jan 26, 2024 at 03:23:33PM +0100, Andrew Jones wrote:
...
> diff --git a/lib/ldiv32.c b/lib/ldiv32.c
> index 897a4b9cd39e..9ce2a6a1faf0 100644
> --- a/lib/ldiv32.c
> +++ b/lib/ldiv32.c
> @@ -1,5 +1,21 @@
>  #include <stdint.h>
>  
> +#if __riscv_xlen == 32
> +int __clzdi2(unsigned long);
> +
> +int __clzdi2(unsigned long a)
> +{
> +	int n = 0;
> +
> +	while (a) {
> +		++n;
> +		a >>= 1;
> +	}
> +
> +	return 32 - n;
> +}
> +#endif
> +

On riscv32, when attempting to do printf("%llx\n", x), where x is a 64-bit
type, I found a bug with the above. It turns out that despite [1] stating
that __clzdi2() takes an unsigned long, libgcc code which generates calls
to it expect it to take an unsigned long long. I've fixed this for v3 by
renaming the above function to __clzsi2() and adding

 int __clzdi2(uint64_t num)
 {
     return num >> 32 ? __clzsi2(num >> 32) : __clzsi2(num) + 32;
 }

[1] https://gcc.gnu.org/onlinedocs/gccint/Integer-library-routines.html

Thanks,
drew

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 16/24] arm/arm64: Share memregions
  2024-02-01 14:21     ` Andrew Jones
@ 2024-02-01 17:46       ` Eric Auger
  0 siblings, 0 replies; 39+ messages in thread
From: Eric Auger @ 2024-02-01 17:46 UTC (permalink / raw)
  To: Andrew Jones
  Cc: kvm, kvm-riscv, kvmarm, ajones, anup, atishp, pbonzini, thuth,
	alexandru.elisei



On 2/1/24 15:21, Andrew Jones wrote:
> On Thu, Feb 01, 2024 at 01:03:54PM +0100, Eric Auger wrote:
>> Hi Drew,
>>
>> On 1/26/24 15:23, Andrew Jones wrote:
> ...
>>> -static void mem_regions_add_assumed(void)
>>> -{
>>> -	phys_addr_t code_end = (phys_addr_t)(unsigned long)&_etext;
>>> -	struct mem_region *r;
>>> -
>>> -	r = mem_region_find(code_end - 1);
>>> -	assert(r);
>>> +	struct mem_region *code, *data;
>>>  
>>>  	/* Split the region with the code into two regions; code and data */
>>> -	mem_region_add(&(struct mem_region){
>>> -		.start = code_end,
>>> -		.end = r->end,
>>> -	});
>>> -	*r = (struct mem_region){
>>> -		.start = r->start,
>>> -		.end = code_end,
>>> -		.flags = MR_F_CODE,
>>> -	};
>>> +	memregions_split((unsigned long)&_etext, &code, &data);
>>> +	assert(code);
>>> +	code->flags |= MR_F_CODE;
>> I think this would deserve to be split into several patches, esp. this
>> change in the implementation of
>>
>> mem_regions_add_assumed and the init changes. At the moment this is pretty difficult to review
>>
> Darn, you called me out on this one :-) I had a feeling I should split out
> the introduction of memregions_split(), since it was sneaking a bit more
> into the patch than just code motion as advertised, but then I hoped I
> get away with putting a bit more burden on the reviewer instead. If you
> haven't already convinced yourself that the new function is equivalent to
> the old code, then I'll respin with the splitting and also create a new
> patch for the 'mem_region' to 'memregions' rename while at it (so there
> will be three patches instead of one). But, if you're already good with
> it, then I'll leave it as is, since patch splitting is a pain...
frankly I would prefer you split. But maybe somebody smarter than me
will be able to review as is, maybe just wait a little bit until you
respin ;-)

Eric
>
> Thanks,
> drew
>


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V
  2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
                   ` (23 preceding siblings ...)
  2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 24/24] MAINTAINERS: Add riscv Andrew Jones
@ 2024-02-02 14:22 ` Andrew Jones
  24 siblings, 0 replies; 39+ messages in thread
From: Andrew Jones @ 2024-02-02 14:22 UTC (permalink / raw)
  To: kvm, kvm-riscv, kvmarm
  Cc: ajones, anup, atishp, pbonzini, thuth, alexandru.elisei, eric.auger

On Fri, Jan 26, 2024 at 03:23:25PM +0100, Andrew Jones wrote:
> v2:
>  - While basing [1] on this series I found two bugs (one in exception
>    return and another in isa string parsing). I also decided to expose
>    a get_pte() function to unit tests and also an isa-extension-by-name
>    function to check for arbitrary extensions. Finally, I picked up
>    Thomas' gitlab-ci suggestion and his tags.
> 
> (Having [1] and the selftests running makes me pretty happy with the
> series, so, unless somebody shouts, I'll merge this sometime next week.)
> 
> [1] https://gitlab.com/jones-drew/kvm-unit-tests/-/commit/e9c6c58b1c799de77fd39970b358a7592ecd048f

With the fix for __clzdi2(), the makefile refactoring to add the %.aux.o
target, and Eric's tags, I'm pleased to announce that I've created a v3
branch and merged it.  Bring on the riscv tests!

Thanks,
drew


> 
> Thanks,
> drew
> 
> Original cover letter follows:
> 
> This series adds another architecture to kvm-unit-tests (RISC-V, both
> 32-bit and 64-bit). Much of the code is borrowed from arm/arm64 by
> mimicking its patterns or by first making the arm code more generic
> and moving it to the common lib.
> 
> This series brings UART, SMP, MMU, and exception handling support.
> One should be able to start writing CPU validation tests in a mix
> of C and asm as well as write SBI tests, as is the plan for the SBI
> verification framework. kvm-unit-tests provides backtraces on asserts
> and input can be given to the tests through command line arguments,
> environment variables, and the DT (there's already an ISA string
> parser for extension detection).
> 
> This series only targets QEMU TCG and KVM, but OpenSBI may be replaced
> with other SBI implementations, such as RustSBI. It's a goal to target
> bare-metal as soon as possible, so EFI support is already in progress
> and will be posted soon. More follow on series will come as well,
> bringing interrupt controller support for timer and PMU testing,
> support to run tests in usermode, and whatever else people need for
> their tests.
> 
> 
> Andrew Jones (24):
>   configure: Add ARCH_LIBDIR
>   riscv: Initial port, hello world
>   arm/arm64: Move cpumask.h to common lib
>   arm/arm64: Share cpu online, present and idle masks
>   riscv: Add DT parsing
>   riscv: Add initial SBI support
>   riscv: Add run script and unittests.cfg
>   riscv: Add riscv32 support
>   riscv: Add exception handling
>   riscv: Add backtrace support
>   arm/arm64: Generalize wfe/sev names in smp.c
>   arm/arm64: Remove spinlocks from on_cpu_async
>   arm/arm64: Share on_cpus
>   riscv: Compile with march
>   riscv: Add SMP support
>   arm/arm64: Share memregions
>   riscv: Populate memregions and switch to page allocator
>   riscv: Add MMU support
>   riscv: Enable the MMU in secondaries
>   riscv: Enable vmalloc
>   lib: Add strcasecmp and strncasecmp
>   riscv: Add isa string parsing
>   gitlab-ci: Add riscv64 tests
>   MAINTAINERS: Add riscv
> 
>  .gitlab-ci.yml               |  17 +++
>  MAINTAINERS                  |   8 ++
>  Makefile                     |   2 +-
>  arm/Makefile.common          |   2 +
>  arm/selftest.c               |   3 +-
>  configure                    |  16 +++
>  lib/arm/asm/gic-v2.h         |   2 +-
>  lib/arm/asm/gic-v3.h         |   2 +-
>  lib/arm/asm/gic.h            |   2 +-
>  lib/arm/asm/setup.h          |  14 --
>  lib/arm/asm/smp.h            |  45 +-----
>  lib/arm/mmu.c                |   3 +-
>  lib/arm/setup.c              |  93 +++----------
>  lib/arm/smp.c                | 135 +-----------------
>  lib/arm64/asm/cpumask.h      |   1 -
>  lib/{arm/asm => }/cpumask.h  |  42 +++++-
>  lib/ctype.h                  |  10 ++
>  lib/elf.h                    |  11 ++
>  lib/ldiv32.c                 |  16 +++
>  lib/linux/const.h            |   2 +
>  lib/memregions.c             |  82 +++++++++++
>  lib/memregions.h             |  29 ++++
>  lib/on-cpus.c                | 154 +++++++++++++++++++++
>  lib/on-cpus.h                |  14 ++
>  lib/riscv/.gitignore         |   1 +
>  lib/riscv/asm-offsets.c      |  62 +++++++++
>  lib/riscv/asm/asm-offsets.h  |   1 +
>  lib/riscv/asm/barrier.h      |  20 +++
>  lib/riscv/asm/bitops.h       |  21 +++
>  lib/riscv/asm/bug.h          |  20 +++
>  lib/riscv/asm/csr.h          | 100 ++++++++++++++
>  lib/riscv/asm/io.h           |  87 ++++++++++++
>  lib/riscv/asm/isa.h          |  33 +++++
>  lib/riscv/asm/memory_areas.h |   1 +
>  lib/riscv/asm/mmu.h          |  32 +++++
>  lib/riscv/asm/page.h         |  21 +++
>  lib/riscv/asm/pgtable.h      |  42 ++++++
>  lib/riscv/asm/processor.h    |  29 ++++
>  lib/riscv/asm/ptrace.h       |  46 +++++++
>  lib/riscv/asm/sbi.h          |  54 ++++++++
>  lib/riscv/asm/setup.h        |  15 ++
>  lib/riscv/asm/smp.h          |  29 ++++
>  lib/riscv/asm/spinlock.h     |   7 +
>  lib/riscv/asm/stack.h        |  12 ++
>  lib/riscv/bitops.c           |  47 +++++++
>  lib/riscv/io.c               |  97 +++++++++++++
>  lib/riscv/isa.c              | 126 +++++++++++++++++
>  lib/riscv/mmu.c              | 205 +++++++++++++++++++++++++++
>  lib/riscv/processor.c        |  64 +++++++++
>  lib/riscv/sbi.c              |  40 ++++++
>  lib/riscv/setup.c            | 188 +++++++++++++++++++++++++
>  lib/riscv/smp.c              |  70 ++++++++++
>  lib/riscv/stack.c            |  32 +++++
>  lib/string.c                 |  14 ++
>  lib/string.h                 |   2 +
>  riscv/Makefile               | 106 ++++++++++++++
>  riscv/cstart.S               | 259 +++++++++++++++++++++++++++++++++++
>  riscv/flat.lds               |  75 ++++++++++
>  riscv/run                    |  41 ++++++
>  riscv/sbi.c                  |  41 ++++++
>  riscv/selftest.c             | 100 ++++++++++++++
>  riscv/sieve.c                |   1 +
>  riscv/unittests.cfg          |  37 +++++
>  63 files changed, 2616 insertions(+), 267 deletions(-)
>  delete mode 100644 lib/arm64/asm/cpumask.h
>  rename lib/{arm/asm => }/cpumask.h (72%)
>  create mode 100644 lib/memregions.c
>  create mode 100644 lib/memregions.h
>  create mode 100644 lib/on-cpus.c
>  create mode 100644 lib/on-cpus.h
>  create mode 100644 lib/riscv/.gitignore
>  create mode 100644 lib/riscv/asm-offsets.c
>  create mode 100644 lib/riscv/asm/asm-offsets.h
>  create mode 100644 lib/riscv/asm/barrier.h
>  create mode 100644 lib/riscv/asm/bitops.h
>  create mode 100644 lib/riscv/asm/bug.h
>  create mode 100644 lib/riscv/asm/csr.h
>  create mode 100644 lib/riscv/asm/io.h
>  create mode 100644 lib/riscv/asm/isa.h
>  create mode 100644 lib/riscv/asm/memory_areas.h
>  create mode 100644 lib/riscv/asm/mmu.h
>  create mode 100644 lib/riscv/asm/page.h
>  create mode 100644 lib/riscv/asm/pgtable.h
>  create mode 100644 lib/riscv/asm/processor.h
>  create mode 100644 lib/riscv/asm/ptrace.h
>  create mode 100644 lib/riscv/asm/sbi.h
>  create mode 100644 lib/riscv/asm/setup.h
>  create mode 100644 lib/riscv/asm/smp.h
>  create mode 100644 lib/riscv/asm/spinlock.h
>  create mode 100644 lib/riscv/asm/stack.h
>  create mode 100644 lib/riscv/bitops.c
>  create mode 100644 lib/riscv/io.c
>  create mode 100644 lib/riscv/isa.c
>  create mode 100644 lib/riscv/mmu.c
>  create mode 100644 lib/riscv/processor.c
>  create mode 100644 lib/riscv/sbi.c
>  create mode 100644 lib/riscv/setup.c
>  create mode 100644 lib/riscv/smp.c
>  create mode 100644 lib/riscv/stack.c
>  create mode 100644 riscv/Makefile
>  create mode 100644 riscv/cstart.S
>  create mode 100644 riscv/flat.lds
>  create mode 100755 riscv/run
>  create mode 100644 riscv/sbi.c
>  create mode 100644 riscv/selftest.c
>  create mode 120000 riscv/sieve.c
>  create mode 100644 riscv/unittests.cfg
> 
> -- 
> 2.43.0
> 

^ permalink raw reply	[flat|nested] 39+ messages in thread

end of thread, other threads:[~2024-02-02 14:23 UTC | newest]

Thread overview: 39+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-01-26 14:23 [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 01/24] configure: Add ARCH_LIBDIR Andrew Jones
2024-02-01  8:29   ` Eric Auger
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 02/24] riscv: Initial port, hello world Andrew Jones
2024-02-01  8:29   ` Eric Auger
2024-02-01 14:07     ` Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 03/24] arm/arm64: Move cpumask.h to common lib Andrew Jones
2024-02-01  8:29   ` Eric Auger
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 04/24] arm/arm64: Share cpu online, present and idle masks Andrew Jones
2024-02-01  8:29   ` Eric Auger
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 05/24] riscv: Add DT parsing Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 06/24] riscv: Add initial SBI support Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 07/24] riscv: Add run script and unittests.cfg Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 08/24] riscv: Add riscv32 support Andrew Jones
2024-02-01 15:24   ` Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 09/24] riscv: Add exception handling Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 10/24] riscv: Add backtrace support Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 11/24] arm/arm64: Generalize wfe/sev names in smp.c Andrew Jones
2024-02-01  9:22   ` Eric Auger
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 12/24] arm/arm64: Remove spinlocks from on_cpu_async Andrew Jones
2024-02-01  9:34   ` Eric Auger
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 13/24] arm/arm64: Share on_cpus Andrew Jones
2024-02-01  9:36   ` Eric Auger
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 14/24] riscv: Compile with march Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 15/24] riscv: Add SMP support Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 16/24] arm/arm64: Share memregions Andrew Jones
2024-02-01 12:03   ` Eric Auger
2024-02-01 14:21     ` Andrew Jones
2024-02-01 17:46       ` Eric Auger
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 17/24] riscv: Populate memregions and switch to page allocator Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 18/24] riscv: Add MMU support Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 19/24] riscv: Enable the MMU in secondaries Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 20/24] riscv: Enable vmalloc Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 21/24] lib: Add strcasecmp and strncasecmp Andrew Jones
2024-02-01  9:45   ` Eric Auger
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 22/24] riscv: Add isa string parsing Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 23/24] gitlab-ci: Add riscv64 tests Andrew Jones
2024-01-26 14:23 ` [kvm-unit-tests PATCH v2 24/24] MAINTAINERS: Add riscv Andrew Jones
2024-02-02 14:22 ` [kvm-unit-tests PATCH v2 00/24] Introduce RISC-V Andrew Jones

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).