* [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-03 16:43 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Mark Rutland, Marc Poulhiès, linux-doc, Alexander Shishkin,
Jan Kiszka, Sebastian Reichel, Marius Gligor, Oleg Nesterov,
linux-mm, Louis Morhet, Krzysztof Kozlowski, Luc Michel,
linux-riscv, Ashley Lesdalons, Will Deacon, Ard Biesheuvel,
linux-arch, Marc Zyngier, Thomas Costis, Jonathan Corbet,
Jonathan Borne, Aneesh Kumar K.V, Samuel Jones, Peter Zijlstra,
Ingo Molnar, Steven Rostedt, Jean-Christophe Pince, Waiman Long,
Clement Leger, Masami Hiramatsu, devicetree, Albert Ou,
Kees Cook, linux-pm, Yann Sionneau, Boqun Feng,
Guillaume Thouvenin, Julian Vetter, Arnaldo Carvalho de Melo,
Julien Hascoet, Jason Baron, Rob Herring, Nick Piggin,
Paul Walmsley, Namhyung Kim, Thomas Gleixner, Josh Poimboeuf,
linux-arm-kernel, Julien Villette, Jules Maselbas,
Christian Brauner, Andrew Morton, Benjamin Mugnier,
Vincent Chardon, Guillaume Missonnier, linux-kernel, Eric Paris,
Alex Michon, linux-perf-users, linux-audit, Palmer Dabbelt,
Eric Biederman, Jiri Olsa, Kieran Bingham, bpf
This patch series adds support for the kv3-1 CPU architecture of the kvx family
found in the Coolidge (aka MPPA3-80) SoC of Kalray.
This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
therefore this patch series cannot be merged into Linux for now.
The goal is to have preliminary reviews and to fix problems early.
The Kalray VLIW processor family (kvx) has the following features:
* 32/64 bits execution mode
* 6-issue VLIW architecture
* 64 x 64bits general purpose registers
* SIMD instructions
* little-endian
* deep learning co-processor
Kalray kv3-1 core which is the third of the kvx family is embedded in Kalray
Coolidge SoC currently used on K200 and K200-LP boards.
The Coolidge SoC contains 5 clusters each of which is made of:
* 4MiB of on-chip memory (SMEM)
* 1 dedicated safety/security core (kv3-1 core).
* 16 PEs (Processing Elements) (kv3-1 cores).
* 16 Co-processors (one per PE)
* 2 Crypto accelerators
The Coolidge SoC contains the following features:
* 5 Clusters
* 2 100G Ethernet controllers
* 8 PCIe GEN4 controllers (Root Complex and Endpoint capable)
* 2 USB 2.0 controllers
* 1 Octal SPI-NOR flash controller
* 1 eMMC controller
* 3 Quad SPI controllers
* 6 UART
* 5 I2C controllers (3 of which are SMBus capable)
* 4 CAN controllers
* 1 OTP memory
A kvx toolchain can be built using:
# install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
$ git clone https://github.com/kalray/build-scripts
$ cd build-scripts
$ source last.refs
$ ./build-kvx-xgcc.sh output
The kvx toolchain will be installed in the "output" directory.
A buildroot image (kernel+rootfs) and toolchain can be built using:
$ git clone -b coolidge-for-upstream https://github.com/kalray/buildroot
$ cd buildroot
$ make O=build_kvx kvx_defconfig
$ make O=build_kvx
The vmlinux image can be found in buildroot/build_kvx/images/vmlinux.
If you are just interested in building the Linux kernel with no rootfs you can
just do this with the kvx-elf- toolchain:
$ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- default_defconfig
$ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- -j$(($(nproc) + 1))
The vmlinux ELF can be run with qemu by doing:
# install dependencies: ninja pkg-config libglib-2.0-dev cmake libfdt-dev libpixman-1-dev zlib1g-dev
$ git clone https://github.com/kalray/qemu-builder
$ cd qemu-builder
$ git submodule update --init
$ make -j$(($(nproc) + 1))
$ ./qemu-system-kvx -m 1024 -nographic -kernel <path/to/vmlinux>
Yann Sionneau (25):
Documentation: kvx: Add basic documentation
kvx: Add ELF-related definitions
kvx: Add build infrastructure
kvx: Add CPU definition headers
kvx: Add atomic/locking headers
kvx: Add other common headers
kvx: Add boot and setup routines
kvx: Add exception/interrupt handling
kvx: irqchip: Add support for irq controllers
kvx: Add process management
kvx: Add memory management
kvx: Add system call support
kvx: Add signal handling support
kvx: Add ELF relocations and module support
kvx: Add misc common routines
kvx: Add some library functions
kvx: Add multi-processor (SMP) support
kvx: Add kvx default config file
kvx: power: scall poweroff driver
kvx: gdb: add kvx related gdb helpers
kvx: Add support for ftrace
kvx: Add support for jump labels
kvx: Add debugging related support
kvx: Add support for CPU Perf Monitors
kvx: Add support for cpuinfo
.../kalray,kvx-core-intc.txt | 22 +
.../devicetree/bindings/perf/kalray-pm.txt | 21 +
Documentation/kvx/kvx-exceptions.txt | 246 +
Documentation/kvx/kvx-iommu.txt | 183 +
Documentation/kvx/kvx-mmu.txt | 272 +
Documentation/kvx/kvx-smp.txt | 36 +
Documentation/kvx/kvx.txt | 268 +
arch/kvx/Kconfig | 249 +
arch/kvx/Kconfig.debug | 70 +
arch/kvx/Makefile | 52 +
arch/kvx/configs/default_defconfig | 130 +
arch/kvx/include/asm/Kbuild | 20 +
arch/kvx/include/asm/asm-prototypes.h | 14 +
arch/kvx/include/asm/atomic.h | 104 +
arch/kvx/include/asm/barrier.h | 15 +
arch/kvx/include/asm/bitops.h | 207 +
arch/kvx/include/asm/bitrev.h | 32 +
arch/kvx/include/asm/break_hook.h | 69 +
arch/kvx/include/asm/bug.h | 67 +
arch/kvx/include/asm/cache.h | 46 +
arch/kvx/include/asm/cacheflush.h | 181 +
arch/kvx/include/asm/clocksource.h | 17 +
arch/kvx/include/asm/cmpxchg.h | 185 +
arch/kvx/include/asm/current.h | 22 +
arch/kvx/include/asm/dame.h | 31 +
arch/kvx/include/asm/debug.h | 35 +
arch/kvx/include/asm/elf.h | 155 +
arch/kvx/include/asm/fixmap.h | 47 +
arch/kvx/include/asm/ftrace.h | 41 +
arch/kvx/include/asm/futex.h | 141 +
arch/kvx/include/asm/hardirq.h | 14 +
arch/kvx/include/asm/hugetlb.h | 36 +
arch/kvx/include/asm/hw_breakpoint.h | 72 +
arch/kvx/include/asm/hw_irq.h | 14 +
arch/kvx/include/asm/insns.h | 16 +
arch/kvx/include/asm/insns_defs.h | 197 +
arch/kvx/include/asm/io.h | 34 +
arch/kvx/include/asm/ipi.h | 16 +
arch/kvx/include/asm/irqflags.h | 58 +
arch/kvx/include/asm/jump_label.h | 59 +
arch/kvx/include/asm/l2_cache.h | 75 +
arch/kvx/include/asm/l2_cache_defs.h | 64 +
arch/kvx/include/asm/linkage.h | 13 +
arch/kvx/include/asm/mem_map.h | 44 +
arch/kvx/include/asm/mmu.h | 296 +
arch/kvx/include/asm/mmu_context.h | 156 +
arch/kvx/include/asm/mmu_stats.h | 38 +
arch/kvx/include/asm/page.h | 187 +
arch/kvx/include/asm/page_size.h | 29 +
arch/kvx/include/asm/pci.h | 36 +
arch/kvx/include/asm/perf_event.h | 90 +
arch/kvx/include/asm/pgalloc.h | 101 +
arch/kvx/include/asm/pgtable-bits.h | 102 +
arch/kvx/include/asm/pgtable.h | 451 ++
arch/kvx/include/asm/privilege.h | 211 +
arch/kvx/include/asm/processor.h | 176 +
arch/kvx/include/asm/ptrace.h | 217 +
arch/kvx/include/asm/pwr_ctrl.h | 45 +
arch/kvx/include/asm/rm_fw.h | 16 +
arch/kvx/include/asm/sections.h | 18 +
arch/kvx/include/asm/setup.h | 29 +
arch/kvx/include/asm/sfr.h | 107 +
arch/kvx/include/asm/sfr_defs.h | 5028 +++++++++++++++++
arch/kvx/include/asm/smp.h | 42 +
arch/kvx/include/asm/sparsemem.h | 15 +
arch/kvx/include/asm/spinlock.h | 16 +
arch/kvx/include/asm/spinlock_types.h | 17 +
arch/kvx/include/asm/stackprotector.h | 47 +
arch/kvx/include/asm/stacktrace.h | 44 +
arch/kvx/include/asm/string.h | 20 +
arch/kvx/include/asm/swab.h | 48 +
arch/kvx/include/asm/switch_to.h | 21 +
arch/kvx/include/asm/symbols.h | 16 +
arch/kvx/include/asm/sys_arch.h | 51 +
arch/kvx/include/asm/syscall.h | 73 +
arch/kvx/include/asm/syscalls.h | 21 +
arch/kvx/include/asm/thread_info.h | 78 +
arch/kvx/include/asm/timex.h | 20 +
arch/kvx/include/asm/tlb.h | 24 +
arch/kvx/include/asm/tlb_defs.h | 131 +
arch/kvx/include/asm/tlbflush.h | 58 +
arch/kvx/include/asm/traps.h | 76 +
arch/kvx/include/asm/types.h | 12 +
arch/kvx/include/asm/uaccess.h | 324 ++
arch/kvx/include/asm/unistd.h | 11 +
arch/kvx/include/asm/vermagic.h | 12 +
arch/kvx/include/asm/vmalloc.h | 10 +
arch/kvx/include/uapi/asm/Kbuild | 1 +
arch/kvx/include/uapi/asm/bitsperlong.h | 14 +
arch/kvx/include/uapi/asm/byteorder.h | 12 +
arch/kvx/include/uapi/asm/cachectl.h | 25 +
arch/kvx/include/uapi/asm/ptrace.h | 114 +
arch/kvx/include/uapi/asm/sigcontext.h | 16 +
arch/kvx/include/uapi/asm/unistd.h | 16 +
arch/kvx/kernel/Makefile | 27 +
arch/kvx/kernel/asm-offsets.c | 157 +
arch/kvx/kernel/break_hook.c | 77 +
arch/kvx/kernel/common.c | 11 +
arch/kvx/kernel/cpuinfo.c | 96 +
arch/kvx/kernel/dame_handler.c | 113 +
arch/kvx/kernel/debug.c | 64 +
arch/kvx/kernel/entry.S | 1759 ++++++
arch/kvx/kernel/ftrace.c | 339 ++
arch/kvx/kernel/head.S | 612 ++
arch/kvx/kernel/hw_breakpoint.c | 556 ++
arch/kvx/kernel/insns.c | 146 +
arch/kvx/kernel/io.c | 96 +
arch/kvx/kernel/irq.c | 78 +
arch/kvx/kernel/jump_label.c | 34 +
arch/kvx/kernel/kvx_ksyms.c | 29 +
arch/kvx/kernel/l2_cache.c | 448 ++
arch/kvx/kernel/mcount.S | 340 ++
arch/kvx/kernel/module.c | 148 +
arch/kvx/kernel/perf_event.c | 609 ++
arch/kvx/kernel/process.c | 212 +
arch/kvx/kernel/prom.c | 24 +
arch/kvx/kernel/ptrace.c | 461 ++
arch/kvx/kernel/reset.c | 37 +
arch/kvx/kernel/return_address.c | 55 +
arch/kvx/kernel/setup.c | 178 +
arch/kvx/kernel/signal.c | 266 +
arch/kvx/kernel/smp.c | 110 +
arch/kvx/kernel/smpboot.c | 127 +
arch/kvx/kernel/stacktrace.c | 173 +
arch/kvx/kernel/sys_kvx.c | 58 +
arch/kvx/kernel/syscall_table.c | 19 +
arch/kvx/kernel/time.c | 242 +
arch/kvx/kernel/traps.c | 243 +
arch/kvx/kernel/vdso.c | 87 +
arch/kvx/kernel/vmlinux.lds.S | 173 +
arch/kvx/lib/Makefile | 6 +
arch/kvx/lib/clear_page.S | 40 +
arch/kvx/lib/copy_page.S | 90 +
arch/kvx/lib/delay.c | 39 +
arch/kvx/lib/memcpy.c | 70 +
arch/kvx/lib/memset.S | 351 ++
arch/kvx/lib/strlen.S | 122 +
arch/kvx/lib/usercopy.S | 90 +
arch/kvx/mm/Makefile | 10 +
arch/kvx/mm/cacheflush.c | 154 +
arch/kvx/mm/dma-mapping.c | 95 +
arch/kvx/mm/extable.c | 24 +
arch/kvx/mm/fault.c | 264 +
arch/kvx/mm/hugetlbpage.c | 317 ++
arch/kvx/mm/init.c | 527 ++
arch/kvx/mm/kernel_rwx.c | 228 +
arch/kvx/mm/mmap.c | 31 +
arch/kvx/mm/mmu.c | 204 +
arch/kvx/mm/mmu_stats.c | 94 +
arch/kvx/mm/tlb.c | 433 ++
arch/kvx/platform/Makefile | 7 +
arch/kvx/platform/ipi.c | 110 +
arch/kvx/platform/pwr_ctrl.c | 93 +
drivers/irqchip/Kconfig | 27 +
drivers/irqchip/Makefile | 4 +
drivers/irqchip/irq-kvx-apic-gic.c | 349 ++
drivers/irqchip/irq-kvx-apic-mailbox.c | 465 ++
drivers/irqchip/irq-kvx-core-intc.c | 82 +
drivers/irqchip/irq-kvx-itgen.c | 224 +
drivers/power/reset/kvx-scall-poweroff.c | 53 +
include/linux/cpuhotplug.h | 2 +
include/linux/irqchip/irq-kvx-apic-gic.h | 21 +
include/linux/irqchip/irq-kvx-apic-mailbox.h | 29 +
include/linux/irqchip/irq-kvx-itgen.h | 24 +
include/uapi/linux/audit.h | 1 +
include/uapi/linux/elf-em.h | 1 +
include/uapi/linux/elf.h | 1 +
scripts/gdb/arch/Makefile | 11 +
scripts/gdb/arch/__init__.py | 1 +
scripts/gdb/arch/kvx/Makefile | 25 +
scripts/gdb/arch/kvx/__init__.py | 1 +
scripts/gdb/arch/kvx/constants.py.in | 74 +
scripts/gdb/arch/kvx/mmu.py | 199 +
scripts/gdb/arch/kvx/page_table_walk.py | 207 +
tools/include/uapi/asm/bitsperlong.h | 2 +
175 files changed, 25814 insertions(+)
create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt
create mode 100644 Documentation/devicetree/bindings/perf/kalray-pm.txt
create mode 100644 Documentation/kvx/kvx-exceptions.txt
create mode 100644 Documentation/kvx/kvx-iommu.txt
create mode 100644 Documentation/kvx/kvx-mmu.txt
create mode 100644 Documentation/kvx/kvx-smp.txt
create mode 100644 Documentation/kvx/kvx.txt
create mode 100644 arch/kvx/Kconfig
create mode 100644 arch/kvx/Kconfig.debug
create mode 100644 arch/kvx/Makefile
create mode 100644 arch/kvx/configs/default_defconfig
create mode 100644 arch/kvx/include/asm/Kbuild
create mode 100644 arch/kvx/include/asm/asm-prototypes.h
create mode 100644 arch/kvx/include/asm/atomic.h
create mode 100644 arch/kvx/include/asm/barrier.h
create mode 100644 arch/kvx/include/asm/bitops.h
create mode 100644 arch/kvx/include/asm/bitrev.h
create mode 100644 arch/kvx/include/asm/break_hook.h
create mode 100644 arch/kvx/include/asm/bug.h
create mode 100644 arch/kvx/include/asm/cache.h
create mode 100644 arch/kvx/include/asm/cacheflush.h
create mode 100644 arch/kvx/include/asm/clocksource.h
create mode 100644 arch/kvx/include/asm/cmpxchg.h
create mode 100644 arch/kvx/include/asm/current.h
create mode 100644 arch/kvx/include/asm/dame.h
create mode 100644 arch/kvx/include/asm/debug.h
create mode 100644 arch/kvx/include/asm/elf.h
create mode 100644 arch/kvx/include/asm/fixmap.h
create mode 100644 arch/kvx/include/asm/ftrace.h
create mode 100644 arch/kvx/include/asm/futex.h
create mode 100644 arch/kvx/include/asm/hardirq.h
create mode 100644 arch/kvx/include/asm/hugetlb.h
create mode 100644 arch/kvx/include/asm/hw_breakpoint.h
create mode 100644 arch/kvx/include/asm/hw_irq.h
create mode 100644 arch/kvx/include/asm/insns.h
create mode 100644 arch/kvx/include/asm/insns_defs.h
create mode 100644 arch/kvx/include/asm/io.h
create mode 100644 arch/kvx/include/asm/ipi.h
create mode 100644 arch/kvx/include/asm/irqflags.h
create mode 100644 arch/kvx/include/asm/jump_label.h
create mode 100644 arch/kvx/include/asm/l2_cache.h
create mode 100644 arch/kvx/include/asm/l2_cache_defs.h
create mode 100644 arch/kvx/include/asm/linkage.h
create mode 100644 arch/kvx/include/asm/mem_map.h
create mode 100644 arch/kvx/include/asm/mmu.h
create mode 100644 arch/kvx/include/asm/mmu_context.h
create mode 100644 arch/kvx/include/asm/mmu_stats.h
create mode 100644 arch/kvx/include/asm/page.h
create mode 100644 arch/kvx/include/asm/page_size.h
create mode 100644 arch/kvx/include/asm/pci.h
create mode 100644 arch/kvx/include/asm/perf_event.h
create mode 100644 arch/kvx/include/asm/pgalloc.h
create mode 100644 arch/kvx/include/asm/pgtable-bits.h
create mode 100644 arch/kvx/include/asm/pgtable.h
create mode 100644 arch/kvx/include/asm/privilege.h
create mode 100644 arch/kvx/include/asm/processor.h
create mode 100644 arch/kvx/include/asm/ptrace.h
create mode 100644 arch/kvx/include/asm/pwr_ctrl.h
create mode 100644 arch/kvx/include/asm/rm_fw.h
create mode 100644 arch/kvx/include/asm/sections.h
create mode 100644 arch/kvx/include/asm/setup.h
create mode 100644 arch/kvx/include/asm/sfr.h
create mode 100644 arch/kvx/include/asm/sfr_defs.h
create mode 100644 arch/kvx/include/asm/smp.h
create mode 100644 arch/kvx/include/asm/sparsemem.h
create mode 100644 arch/kvx/include/asm/spinlock.h
create mode 100644 arch/kvx/include/asm/spinlock_types.h
create mode 100644 arch/kvx/include/asm/stackprotector.h
create mode 100644 arch/kvx/include/asm/stacktrace.h
create mode 100644 arch/kvx/include/asm/string.h
create mode 100644 arch/kvx/include/asm/swab.h
create mode 100644 arch/kvx/include/asm/switch_to.h
create mode 100644 arch/kvx/include/asm/symbols.h
create mode 100644 arch/kvx/include/asm/sys_arch.h
create mode 100644 arch/kvx/include/asm/syscall.h
create mode 100644 arch/kvx/include/asm/syscalls.h
create mode 100644 arch/kvx/include/asm/thread_info.h
create mode 100644 arch/kvx/include/asm/timex.h
create mode 100644 arch/kvx/include/asm/tlb.h
create mode 100644 arch/kvx/include/asm/tlb_defs.h
create mode 100644 arch/kvx/include/asm/tlbflush.h
create mode 100644 arch/kvx/include/asm/traps.h
create mode 100644 arch/kvx/include/asm/types.h
create mode 100644 arch/kvx/include/asm/uaccess.h
create mode 100644 arch/kvx/include/asm/unistd.h
create mode 100644 arch/kvx/include/asm/vermagic.h
create mode 100644 arch/kvx/include/asm/vmalloc.h
create mode 100644 arch/kvx/include/uapi/asm/Kbuild
create mode 100644 arch/kvx/include/uapi/asm/bitsperlong.h
create mode 100644 arch/kvx/include/uapi/asm/byteorder.h
create mode 100644 arch/kvx/include/uapi/asm/cachectl.h
create mode 100644 arch/kvx/include/uapi/asm/ptrace.h
create mode 100644 arch/kvx/include/uapi/asm/sigcontext.h
create mode 100644 arch/kvx/include/uapi/asm/unistd.h
create mode 100644 arch/kvx/kernel/Makefile
create mode 100644 arch/kvx/kernel/asm-offsets.c
create mode 100644 arch/kvx/kernel/break_hook.c
create mode 100644 arch/kvx/kernel/common.c
create mode 100644 arch/kvx/kernel/cpuinfo.c
create mode 100644 arch/kvx/kernel/dame_handler.c
create mode 100644 arch/kvx/kernel/debug.c
create mode 100644 arch/kvx/kernel/entry.S
create mode 100644 arch/kvx/kernel/ftrace.c
create mode 100644 arch/kvx/kernel/head.S
create mode 100644 arch/kvx/kernel/hw_breakpoint.c
create mode 100644 arch/kvx/kernel/insns.c
create mode 100644 arch/kvx/kernel/io.c
create mode 100644 arch/kvx/kernel/irq.c
create mode 100644 arch/kvx/kernel/jump_label.c
create mode 100644 arch/kvx/kernel/kvx_ksyms.c
create mode 100644 arch/kvx/kernel/l2_cache.c
create mode 100644 arch/kvx/kernel/mcount.S
create mode 100644 arch/kvx/kernel/module.c
create mode 100644 arch/kvx/kernel/perf_event.c
create mode 100644 arch/kvx/kernel/process.c
create mode 100644 arch/kvx/kernel/prom.c
create mode 100644 arch/kvx/kernel/ptrace.c
create mode 100644 arch/kvx/kernel/reset.c
create mode 100644 arch/kvx/kernel/return_address.c
create mode 100644 arch/kvx/kernel/setup.c
create mode 100644 arch/kvx/kernel/signal.c
create mode 100644 arch/kvx/kernel/smp.c
create mode 100644 arch/kvx/kernel/smpboot.c
create mode 100644 arch/kvx/kernel/stacktrace.c
create mode 100644 arch/kvx/kernel/sys_kvx.c
create mode 100644 arch/kvx/kernel/syscall_table.c
create mode 100644 arch/kvx/kernel/time.c
create mode 100644 arch/kvx/kernel/traps.c
create mode 100644 arch/kvx/kernel/vdso.c
create mode 100644 arch/kvx/kernel/vmlinux.lds.S
create mode 100644 arch/kvx/lib/Makefile
create mode 100644 arch/kvx/lib/clear_page.S
create mode 100644 arch/kvx/lib/copy_page.S
create mode 100644 arch/kvx/lib/delay.c
create mode 100644 arch/kvx/lib/memcpy.c
create mode 100644 arch/kvx/lib/memset.S
create mode 100644 arch/kvx/lib/strlen.S
create mode 100644 arch/kvx/lib/usercopy.S
create mode 100644 arch/kvx/mm/Makefile
create mode 100644 arch/kvx/mm/cacheflush.c
create mode 100644 arch/kvx/mm/dma-mapping.c
create mode 100644 arch/kvx/mm/extable.c
create mode 100644 arch/kvx/mm/fault.c
create mode 100644 arch/kvx/mm/hugetlbpage.c
create mode 100644 arch/kvx/mm/init.c
create mode 100644 arch/kvx/mm/kernel_rwx.c
create mode 100644 arch/kvx/mm/mmap.c
create mode 100644 arch/kvx/mm/mmu.c
create mode 100644 arch/kvx/mm/mmu_stats.c
create mode 100644 arch/kvx/mm/tlb.c
create mode 100644 arch/kvx/platform/Makefile
create mode 100644 arch/kvx/platform/ipi.c
create mode 100644 arch/kvx/platform/pwr_ctrl.c
create mode 100644 drivers/irqchip/irq-kvx-apic-gic.c
create mode 100644 drivers/irqchip/irq-kvx-apic-mailbox.c
create mode 100644 drivers/irqchip/irq-kvx-core-intc.c
create mode 100644 drivers/irqchip/irq-kvx-itgen.c
create mode 100644 drivers/power/reset/kvx-scall-poweroff.c
create mode 100644 include/linux/irqchip/irq-kvx-apic-gic.h
create mode 100644 include/linux/irqchip/irq-kvx-apic-mailbox.h
create mode 100644 include/linux/irqchip/irq-kvx-itgen.h
create mode 100644 scripts/gdb/arch/Makefile
create mode 100644 scripts/gdb/arch/__init__.py
create mode 100644 scripts/gdb/arch/kvx/Makefile
create mode 100644 scripts/gdb/arch/kvx/__init__.py
create mode 100644 scripts/gdb/arch/kvx/constants.py.in
create mode 100644 scripts/gdb/arch/kvx/mmu.py
create mode 100644 scripts/gdb/arch/kvx/page_table_walk.py
--
2.37.2
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-03 16:43 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Yann Sionneau, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
This patch series adds support for the kv3-1 CPU architecture of the kvx family
found in the Coolidge (aka MPPA3-80) SoC of Kalray.
This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
therefore this patch series cannot be merged into Linux for now.
The goal is to have preliminary reviews and to fix problems early.
The Kalray VLIW processor family (kvx) has the following features:
* 32/64 bits execution mode
* 6-issue VLIW architecture
* 64 x 64bits general purpose registers
* SIMD instructions
* little-endian
* deep learning co-processor
Kalray kv3-1 core which is the third of the kvx family is embedded in Kalray
Coolidge SoC currently used on K200 and K200-LP boards.
The Coolidge SoC contains 5 clusters each of which is made of:
* 4MiB of on-chip memory (SMEM)
* 1 dedicated safety/security core (kv3-1 core).
* 16 PEs (Processing Elements) (kv3-1 cores).
* 16 Co-processors (one per PE)
* 2 Crypto accelerators
The Coolidge SoC contains the following features:
* 5 Clusters
* 2 100G Ethernet controllers
* 8 PCIe GEN4 controllers (Root Complex and Endpoint capable)
* 2 USB 2.0 controllers
* 1 Octal SPI-NOR flash controller
* 1 eMMC controller
* 3 Quad SPI controllers
* 6 UART
* 5 I2C controllers (3 of which are SMBus capable)
* 4 CAN controllers
* 1 OTP memory
A kvx toolchain can be built using:
# install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
$ git clone https://github.com/kalray/build-scripts
$ cd build-scripts
$ source last.refs
$ ./build-kvx-xgcc.sh output
The kvx toolchain will be installed in the "output" directory.
A buildroot image (kernel+rootfs) and toolchain can be built using:
$ git clone -b coolidge-for-upstream https://github.com/kalray/buildroot
$ cd buildroot
$ make O=build_kvx kvx_defconfig
$ make O=build_kvx
The vmlinux image can be found in buildroot/build_kvx/images/vmlinux.
If you are just interested in building the Linux kernel with no rootfs you can
just do this with the kvx-elf- toolchain:
$ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- default_defconfig
$ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- -j$(($(nproc) + 1))
The vmlinux ELF can be run with qemu by doing:
# install dependencies: ninja pkg-config libglib-2.0-dev cmake libfdt-dev libpixman-1-dev zlib1g-dev
$ git clone https://github.com/kalray/qemu-builder
$ cd qemu-builder
$ git submodule update --init
$ make -j$(($(nproc) + 1))
$ ./qemu-system-kvx -m 1024 -nographic -kernel <path/to/vmlinux>
Yann Sionneau (25):
Documentation: kvx: Add basic documentation
kvx: Add ELF-related definitions
kvx: Add build infrastructure
kvx: Add CPU definition headers
kvx: Add atomic/locking headers
kvx: Add other common headers
kvx: Add boot and setup routines
kvx: Add exception/interrupt handling
kvx: irqchip: Add support for irq controllers
kvx: Add process management
kvx: Add memory management
kvx: Add system call support
kvx: Add signal handling support
kvx: Add ELF relocations and module support
kvx: Add misc common routines
kvx: Add some library functions
kvx: Add multi-processor (SMP) support
kvx: Add kvx default config file
kvx: power: scall poweroff driver
kvx: gdb: add kvx related gdb helpers
kvx: Add support for ftrace
kvx: Add support for jump labels
kvx: Add debugging related support
kvx: Add support for CPU Perf Monitors
kvx: Add support for cpuinfo
.../kalray,kvx-core-intc.txt | 22 +
.../devicetree/bindings/perf/kalray-pm.txt | 21 +
Documentation/kvx/kvx-exceptions.txt | 246 +
Documentation/kvx/kvx-iommu.txt | 183 +
Documentation/kvx/kvx-mmu.txt | 272 +
Documentation/kvx/kvx-smp.txt | 36 +
Documentation/kvx/kvx.txt | 268 +
arch/kvx/Kconfig | 249 +
arch/kvx/Kconfig.debug | 70 +
arch/kvx/Makefile | 52 +
arch/kvx/configs/default_defconfig | 130 +
arch/kvx/include/asm/Kbuild | 20 +
arch/kvx/include/asm/asm-prototypes.h | 14 +
arch/kvx/include/asm/atomic.h | 104 +
arch/kvx/include/asm/barrier.h | 15 +
arch/kvx/include/asm/bitops.h | 207 +
arch/kvx/include/asm/bitrev.h | 32 +
arch/kvx/include/asm/break_hook.h | 69 +
arch/kvx/include/asm/bug.h | 67 +
arch/kvx/include/asm/cache.h | 46 +
arch/kvx/include/asm/cacheflush.h | 181 +
arch/kvx/include/asm/clocksource.h | 17 +
arch/kvx/include/asm/cmpxchg.h | 185 +
arch/kvx/include/asm/current.h | 22 +
arch/kvx/include/asm/dame.h | 31 +
arch/kvx/include/asm/debug.h | 35 +
arch/kvx/include/asm/elf.h | 155 +
arch/kvx/include/asm/fixmap.h | 47 +
arch/kvx/include/asm/ftrace.h | 41 +
arch/kvx/include/asm/futex.h | 141 +
arch/kvx/include/asm/hardirq.h | 14 +
arch/kvx/include/asm/hugetlb.h | 36 +
arch/kvx/include/asm/hw_breakpoint.h | 72 +
arch/kvx/include/asm/hw_irq.h | 14 +
arch/kvx/include/asm/insns.h | 16 +
arch/kvx/include/asm/insns_defs.h | 197 +
arch/kvx/include/asm/io.h | 34 +
arch/kvx/include/asm/ipi.h | 16 +
arch/kvx/include/asm/irqflags.h | 58 +
arch/kvx/include/asm/jump_label.h | 59 +
arch/kvx/include/asm/l2_cache.h | 75 +
arch/kvx/include/asm/l2_cache_defs.h | 64 +
arch/kvx/include/asm/linkage.h | 13 +
arch/kvx/include/asm/mem_map.h | 44 +
arch/kvx/include/asm/mmu.h | 296 +
arch/kvx/include/asm/mmu_context.h | 156 +
arch/kvx/include/asm/mmu_stats.h | 38 +
arch/kvx/include/asm/page.h | 187 +
arch/kvx/include/asm/page_size.h | 29 +
arch/kvx/include/asm/pci.h | 36 +
arch/kvx/include/asm/perf_event.h | 90 +
arch/kvx/include/asm/pgalloc.h | 101 +
arch/kvx/include/asm/pgtable-bits.h | 102 +
arch/kvx/include/asm/pgtable.h | 451 ++
arch/kvx/include/asm/privilege.h | 211 +
arch/kvx/include/asm/processor.h | 176 +
arch/kvx/include/asm/ptrace.h | 217 +
arch/kvx/include/asm/pwr_ctrl.h | 45 +
arch/kvx/include/asm/rm_fw.h | 16 +
arch/kvx/include/asm/sections.h | 18 +
arch/kvx/include/asm/setup.h | 29 +
arch/kvx/include/asm/sfr.h | 107 +
arch/kvx/include/asm/sfr_defs.h | 5028 +++++++++++++++++
arch/kvx/include/asm/smp.h | 42 +
arch/kvx/include/asm/sparsemem.h | 15 +
arch/kvx/include/asm/spinlock.h | 16 +
arch/kvx/include/asm/spinlock_types.h | 17 +
arch/kvx/include/asm/stackprotector.h | 47 +
arch/kvx/include/asm/stacktrace.h | 44 +
arch/kvx/include/asm/string.h | 20 +
arch/kvx/include/asm/swab.h | 48 +
arch/kvx/include/asm/switch_to.h | 21 +
arch/kvx/include/asm/symbols.h | 16 +
arch/kvx/include/asm/sys_arch.h | 51 +
arch/kvx/include/asm/syscall.h | 73 +
arch/kvx/include/asm/syscalls.h | 21 +
arch/kvx/include/asm/thread_info.h | 78 +
arch/kvx/include/asm/timex.h | 20 +
arch/kvx/include/asm/tlb.h | 24 +
arch/kvx/include/asm/tlb_defs.h | 131 +
arch/kvx/include/asm/tlbflush.h | 58 +
arch/kvx/include/asm/traps.h | 76 +
arch/kvx/include/asm/types.h | 12 +
arch/kvx/include/asm/uaccess.h | 324 ++
arch/kvx/include/asm/unistd.h | 11 +
arch/kvx/include/asm/vermagic.h | 12 +
arch/kvx/include/asm/vmalloc.h | 10 +
arch/kvx/include/uapi/asm/Kbuild | 1 +
arch/kvx/include/uapi/asm/bitsperlong.h | 14 +
arch/kvx/include/uapi/asm/byteorder.h | 12 +
arch/kvx/include/uapi/asm/cachectl.h | 25 +
arch/kvx/include/uapi/asm/ptrace.h | 114 +
arch/kvx/include/uapi/asm/sigcontext.h | 16 +
arch/kvx/include/uapi/asm/unistd.h | 16 +
arch/kvx/kernel/Makefile | 27 +
arch/kvx/kernel/asm-offsets.c | 157 +
arch/kvx/kernel/break_hook.c | 77 +
arch/kvx/kernel/common.c | 11 +
arch/kvx/kernel/cpuinfo.c | 96 +
arch/kvx/kernel/dame_handler.c | 113 +
arch/kvx/kernel/debug.c | 64 +
arch/kvx/kernel/entry.S | 1759 ++++++
arch/kvx/kernel/ftrace.c | 339 ++
arch/kvx/kernel/head.S | 612 ++
arch/kvx/kernel/hw_breakpoint.c | 556 ++
arch/kvx/kernel/insns.c | 146 +
arch/kvx/kernel/io.c | 96 +
arch/kvx/kernel/irq.c | 78 +
arch/kvx/kernel/jump_label.c | 34 +
arch/kvx/kernel/kvx_ksyms.c | 29 +
arch/kvx/kernel/l2_cache.c | 448 ++
arch/kvx/kernel/mcount.S | 340 ++
arch/kvx/kernel/module.c | 148 +
arch/kvx/kernel/perf_event.c | 609 ++
arch/kvx/kernel/process.c | 212 +
arch/kvx/kernel/prom.c | 24 +
arch/kvx/kernel/ptrace.c | 461 ++
arch/kvx/kernel/reset.c | 37 +
arch/kvx/kernel/return_address.c | 55 +
arch/kvx/kernel/setup.c | 178 +
arch/kvx/kernel/signal.c | 266 +
arch/kvx/kernel/smp.c | 110 +
arch/kvx/kernel/smpboot.c | 127 +
arch/kvx/kernel/stacktrace.c | 173 +
arch/kvx/kernel/sys_kvx.c | 58 +
arch/kvx/kernel/syscall_table.c | 19 +
arch/kvx/kernel/time.c | 242 +
arch/kvx/kernel/traps.c | 243 +
arch/kvx/kernel/vdso.c | 87 +
arch/kvx/kernel/vmlinux.lds.S | 173 +
arch/kvx/lib/Makefile | 6 +
arch/kvx/lib/clear_page.S | 40 +
arch/kvx/lib/copy_page.S | 90 +
arch/kvx/lib/delay.c | 39 +
arch/kvx/lib/memcpy.c | 70 +
arch/kvx/lib/memset.S | 351 ++
arch/kvx/lib/strlen.S | 122 +
arch/kvx/lib/usercopy.S | 90 +
arch/kvx/mm/Makefile | 10 +
arch/kvx/mm/cacheflush.c | 154 +
arch/kvx/mm/dma-mapping.c | 95 +
arch/kvx/mm/extable.c | 24 +
arch/kvx/mm/fault.c | 264 +
arch/kvx/mm/hugetlbpage.c | 317 ++
arch/kvx/mm/init.c | 527 ++
arch/kvx/mm/kernel_rwx.c | 228 +
arch/kvx/mm/mmap.c | 31 +
arch/kvx/mm/mmu.c | 204 +
arch/kvx/mm/mmu_stats.c | 94 +
arch/kvx/mm/tlb.c | 433 ++
arch/kvx/platform/Makefile | 7 +
arch/kvx/platform/ipi.c | 110 +
arch/kvx/platform/pwr_ctrl.c | 93 +
drivers/irqchip/Kconfig | 27 +
drivers/irqchip/Makefile | 4 +
drivers/irqchip/irq-kvx-apic-gic.c | 349 ++
drivers/irqchip/irq-kvx-apic-mailbox.c | 465 ++
drivers/irqchip/irq-kvx-core-intc.c | 82 +
drivers/irqchip/irq-kvx-itgen.c | 224 +
drivers/power/reset/kvx-scall-poweroff.c | 53 +
include/linux/cpuhotplug.h | 2 +
include/linux/irqchip/irq-kvx-apic-gic.h | 21 +
include/linux/irqchip/irq-kvx-apic-mailbox.h | 29 +
include/linux/irqchip/irq-kvx-itgen.h | 24 +
include/uapi/linux/audit.h | 1 +
include/uapi/linux/elf-em.h | 1 +
include/uapi/linux/elf.h | 1 +
scripts/gdb/arch/Makefile | 11 +
scripts/gdb/arch/__init__.py | 1 +
scripts/gdb/arch/kvx/Makefile | 25 +
scripts/gdb/arch/kvx/__init__.py | 1 +
scripts/gdb/arch/kvx/constants.py.in | 74 +
scripts/gdb/arch/kvx/mmu.py | 199 +
scripts/gdb/arch/kvx/page_table_walk.py | 207 +
tools/include/uapi/asm/bitsperlong.h | 2 +
175 files changed, 25814 insertions(+)
create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt
create mode 100644 Documentation/devicetree/bindings/perf/kalray-pm.txt
create mode 100644 Documentation/kvx/kvx-exceptions.txt
create mode 100644 Documentation/kvx/kvx-iommu.txt
create mode 100644 Documentation/kvx/kvx-mmu.txt
create mode 100644 Documentation/kvx/kvx-smp.txt
create mode 100644 Documentation/kvx/kvx.txt
create mode 100644 arch/kvx/Kconfig
create mode 100644 arch/kvx/Kconfig.debug
create mode 100644 arch/kvx/Makefile
create mode 100644 arch/kvx/configs/default_defconfig
create mode 100644 arch/kvx/include/asm/Kbuild
create mode 100644 arch/kvx/include/asm/asm-prototypes.h
create mode 100644 arch/kvx/include/asm/atomic.h
create mode 100644 arch/kvx/include/asm/barrier.h
create mode 100644 arch/kvx/include/asm/bitops.h
create mode 100644 arch/kvx/include/asm/bitrev.h
create mode 100644 arch/kvx/include/asm/break_hook.h
create mode 100644 arch/kvx/include/asm/bug.h
create mode 100644 arch/kvx/include/asm/cache.h
create mode 100644 arch/kvx/include/asm/cacheflush.h
create mode 100644 arch/kvx/include/asm/clocksource.h
create mode 100644 arch/kvx/include/asm/cmpxchg.h
create mode 100644 arch/kvx/include/asm/current.h
create mode 100644 arch/kvx/include/asm/dame.h
create mode 100644 arch/kvx/include/asm/debug.h
create mode 100644 arch/kvx/include/asm/elf.h
create mode 100644 arch/kvx/include/asm/fixmap.h
create mode 100644 arch/kvx/include/asm/ftrace.h
create mode 100644 arch/kvx/include/asm/futex.h
create mode 100644 arch/kvx/include/asm/hardirq.h
create mode 100644 arch/kvx/include/asm/hugetlb.h
create mode 100644 arch/kvx/include/asm/hw_breakpoint.h
create mode 100644 arch/kvx/include/asm/hw_irq.h
create mode 100644 arch/kvx/include/asm/insns.h
create mode 100644 arch/kvx/include/asm/insns_defs.h
create mode 100644 arch/kvx/include/asm/io.h
create mode 100644 arch/kvx/include/asm/ipi.h
create mode 100644 arch/kvx/include/asm/irqflags.h
create mode 100644 arch/kvx/include/asm/jump_label.h
create mode 100644 arch/kvx/include/asm/l2_cache.h
create mode 100644 arch/kvx/include/asm/l2_cache_defs.h
create mode 100644 arch/kvx/include/asm/linkage.h
create mode 100644 arch/kvx/include/asm/mem_map.h
create mode 100644 arch/kvx/include/asm/mmu.h
create mode 100644 arch/kvx/include/asm/mmu_context.h
create mode 100644 arch/kvx/include/asm/mmu_stats.h
create mode 100644 arch/kvx/include/asm/page.h
create mode 100644 arch/kvx/include/asm/page_size.h
create mode 100644 arch/kvx/include/asm/pci.h
create mode 100644 arch/kvx/include/asm/perf_event.h
create mode 100644 arch/kvx/include/asm/pgalloc.h
create mode 100644 arch/kvx/include/asm/pgtable-bits.h
create mode 100644 arch/kvx/include/asm/pgtable.h
create mode 100644 arch/kvx/include/asm/privilege.h
create mode 100644 arch/kvx/include/asm/processor.h
create mode 100644 arch/kvx/include/asm/ptrace.h
create mode 100644 arch/kvx/include/asm/pwr_ctrl.h
create mode 100644 arch/kvx/include/asm/rm_fw.h
create mode 100644 arch/kvx/include/asm/sections.h
create mode 100644 arch/kvx/include/asm/setup.h
create mode 100644 arch/kvx/include/asm/sfr.h
create mode 100644 arch/kvx/include/asm/sfr_defs.h
create mode 100644 arch/kvx/include/asm/smp.h
create mode 100644 arch/kvx/include/asm/sparsemem.h
create mode 100644 arch/kvx/include/asm/spinlock.h
create mode 100644 arch/kvx/include/asm/spinlock_types.h
create mode 100644 arch/kvx/include/asm/stackprotector.h
create mode 100644 arch/kvx/include/asm/stacktrace.h
create mode 100644 arch/kvx/include/asm/string.h
create mode 100644 arch/kvx/include/asm/swab.h
create mode 100644 arch/kvx/include/asm/switch_to.h
create mode 100644 arch/kvx/include/asm/symbols.h
create mode 100644 arch/kvx/include/asm/sys_arch.h
create mode 100644 arch/kvx/include/asm/syscall.h
create mode 100644 arch/kvx/include/asm/syscalls.h
create mode 100644 arch/kvx/include/asm/thread_info.h
create mode 100644 arch/kvx/include/asm/timex.h
create mode 100644 arch/kvx/include/asm/tlb.h
create mode 100644 arch/kvx/include/asm/tlb_defs.h
create mode 100644 arch/kvx/include/asm/tlbflush.h
create mode 100644 arch/kvx/include/asm/traps.h
create mode 100644 arch/kvx/include/asm/types.h
create mode 100644 arch/kvx/include/asm/uaccess.h
create mode 100644 arch/kvx/include/asm/unistd.h
create mode 100644 arch/kvx/include/asm/vermagic.h
create mode 100644 arch/kvx/include/asm/vmalloc.h
create mode 100644 arch/kvx/include/uapi/asm/Kbuild
create mode 100644 arch/kvx/include/uapi/asm/bitsperlong.h
create mode 100644 arch/kvx/include/uapi/asm/byteorder.h
create mode 100644 arch/kvx/include/uapi/asm/cachectl.h
create mode 100644 arch/kvx/include/uapi/asm/ptrace.h
create mode 100644 arch/kvx/include/uapi/asm/sigcontext.h
create mode 100644 arch/kvx/include/uapi/asm/unistd.h
create mode 100644 arch/kvx/kernel/Makefile
create mode 100644 arch/kvx/kernel/asm-offsets.c
create mode 100644 arch/kvx/kernel/break_hook.c
create mode 100644 arch/kvx/kernel/common.c
create mode 100644 arch/kvx/kernel/cpuinfo.c
create mode 100644 arch/kvx/kernel/dame_handler.c
create mode 100644 arch/kvx/kernel/debug.c
create mode 100644 arch/kvx/kernel/entry.S
create mode 100644 arch/kvx/kernel/ftrace.c
create mode 100644 arch/kvx/kernel/head.S
create mode 100644 arch/kvx/kernel/hw_breakpoint.c
create mode 100644 arch/kvx/kernel/insns.c
create mode 100644 arch/kvx/kernel/io.c
create mode 100644 arch/kvx/kernel/irq.c
create mode 100644 arch/kvx/kernel/jump_label.c
create mode 100644 arch/kvx/kernel/kvx_ksyms.c
create mode 100644 arch/kvx/kernel/l2_cache.c
create mode 100644 arch/kvx/kernel/mcount.S
create mode 100644 arch/kvx/kernel/module.c
create mode 100644 arch/kvx/kernel/perf_event.c
create mode 100644 arch/kvx/kernel/process.c
create mode 100644 arch/kvx/kernel/prom.c
create mode 100644 arch/kvx/kernel/ptrace.c
create mode 100644 arch/kvx/kernel/reset.c
create mode 100644 arch/kvx/kernel/return_address.c
create mode 100644 arch/kvx/kernel/setup.c
create mode 100644 arch/kvx/kernel/signal.c
create mode 100644 arch/kvx/kernel/smp.c
create mode 100644 arch/kvx/kernel/smpboot.c
create mode 100644 arch/kvx/kernel/stacktrace.c
create mode 100644 arch/kvx/kernel/sys_kvx.c
create mode 100644 arch/kvx/kernel/syscall_table.c
create mode 100644 arch/kvx/kernel/time.c
create mode 100644 arch/kvx/kernel/traps.c
create mode 100644 arch/kvx/kernel/vdso.c
create mode 100644 arch/kvx/kernel/vmlinux.lds.S
create mode 100644 arch/kvx/lib/Makefile
create mode 100644 arch/kvx/lib/clear_page.S
create mode 100644 arch/kvx/lib/copy_page.S
create mode 100644 arch/kvx/lib/delay.c
create mode 100644 arch/kvx/lib/memcpy.c
create mode 100644 arch/kvx/lib/memset.S
create mode 100644 arch/kvx/lib/strlen.S
create mode 100644 arch/kvx/lib/usercopy.S
create mode 100644 arch/kvx/mm/Makefile
create mode 100644 arch/kvx/mm/cacheflush.c
create mode 100644 arch/kvx/mm/dma-mapping.c
create mode 100644 arch/kvx/mm/extable.c
create mode 100644 arch/kvx/mm/fault.c
create mode 100644 arch/kvx/mm/hugetlbpage.c
create mode 100644 arch/kvx/mm/init.c
create mode 100644 arch/kvx/mm/kernel_rwx.c
create mode 100644 arch/kvx/mm/mmap.c
create mode 100644 arch/kvx/mm/mmu.c
create mode 100644 arch/kvx/mm/mmu_stats.c
create mode 100644 arch/kvx/mm/tlb.c
create mode 100644 arch/kvx/platform/Makefile
create mode 100644 arch/kvx/platform/ipi.c
create mode 100644 arch/kvx/platform/pwr_ctrl.c
create mode 100644 drivers/irqchip/irq-kvx-apic-gic.c
create mode 100644 drivers/irqchip/irq-kvx-apic-mailbox.c
create mode 100644 drivers/irqchip/irq-kvx-core-intc.c
create mode 100644 drivers/irqchip/irq-kvx-itgen.c
create mode 100644 drivers/power/reset/kvx-scall-poweroff.c
create mode 100644 include/linux/irqchip/irq-kvx-apic-gic.h
create mode 100644 include/linux/irqchip/irq-kvx-apic-mailbox.h
create mode 100644 include/linux/irqchip/irq-kvx-itgen.h
create mode 100644 scripts/gdb/arch/Makefile
create mode 100644 scripts/gdb/arch/__init__.py
create mode 100644 scripts/gdb/arch/kvx/Makefile
create mode 100644 scripts/gdb/arch/kvx/__init__.py
create mode 100644 scripts/gdb/arch/kvx/constants.py.in
create mode 100644 scripts/gdb/arch/kvx/mmu.py
create mode 100644 scripts/gdb/arch/kvx/page_table_walk.py
--
2.37.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 01/25] Documentation: kvx: Add basic documentation
2023-01-03 16:43 ` Yann Sionneau
(?)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-03 17:50 ` Jonathan Corbet
` (2 more replies)
-1 siblings, 3 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Jonathan Corbet, linux-doc, linux-kernel,
Clement Leger, Guillaume Thouvenin
Add some documentation for kvx arch and its Linux port.
CC: Jonathan Corbet <corbet@lwn.net>
CC: linux-doc@vger.kernel.org
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
Documentation/kvx/kvx-exceptions.txt | 246 ++++++++++++++++++++++++
Documentation/kvx/kvx-iommu.txt | 183 ++++++++++++++++++
Documentation/kvx/kvx-mmu.txt | 272 +++++++++++++++++++++++++++
Documentation/kvx/kvx-smp.txt | 36 ++++
Documentation/kvx/kvx.txt | 268 ++++++++++++++++++++++++++
5 files changed, 1005 insertions(+)
create mode 100644 Documentation/kvx/kvx-exceptions.txt
create mode 100644 Documentation/kvx/kvx-iommu.txt
create mode 100644 Documentation/kvx/kvx-mmu.txt
create mode 100644 Documentation/kvx/kvx-smp.txt
create mode 100644 Documentation/kvx/kvx.txt
diff --git a/Documentation/kvx/kvx-exceptions.txt b/Documentation/kvx/kvx-exceptions.txt
new file mode 100644
index 000000000000..11368287bd48
--- /dev/null
+++ b/Documentation/kvx/kvx-exceptions.txt
@@ -0,0 +1,246 @@
+Exceptions
+==========
+On kvx, handlers are set using $ev (exception vector) register which
+specifies a base address.
+An offset is added to $ev upon exception and the result is used as
+"Next $pc".
+The offset depends on which exception vector the cpu wants to jump to:
+* $ev + 0x00 for debug
+* $ev + 0x40 for trap
+* $ev + 0x80 for interrupt
+* $ev + 0xc0 for syscall
+
+Then, handlers are laid in the following order:
+
+ _____________
+ | |
+ | Syscall |
+ |_____________|
+ | |
+ | Interrupts |
+ |_____________|
+ | |
+ | Traps |
+ |_____________|
+ | | ^
+ | Debug | | Stride
+BASE -> |_____________| v
+
+
+Interrupts, and traps are serviced similarly, ie:
+- Jump to handler
+- Save all registers
+- Prepare the call (do_IRQ or trap_handler)
+- restore all registers
+- return from exception
+
+entry.S file is (as for other architectures) the entry point into the kernel.
+It contains all assembly routines related to interrupts/traps/syscall.
+
+Syscall handling
+================
+
+When executing a syscall, it must be done using "scall $r6"
+where $r6 contains the syscall number. Using this convention allow to
+modify and restart a syscall from the kernel.
+
+Syscalls are handled differently than interrupts/exceptions. From an ABI
+point of view, scalls are like function calls: any caller saved register
+can be clobbered by the syscall. However, syscall parameters are passed
+using registers r0 through r7. These registers must be preserved to avoid
+cloberring them before the actual syscall function.
+
+On syscall from userspace (scall instruction), the processor will put
+the syscall number in $es.sn and switch from user to kernel privilege
+mode. kvx_syscall_handler will be called in kernel mode.
+
+The following steps are then taken:
+
+- Switch to kernel stack
+- Extract syscall number
+- Check that the syscall number is not bogus
+ - If so, set syscall func to a not implemented one
+- Check if tracing is enabled
+ - If so, jump to trace_syscall_enter
+ - Save syscall arguments (r0 -> r7) on stack in pt_regs
+ - Call do_trace_syscall_enter function
+- Restore syscall arguments since they have been modified by C call
+- Call the syscall function
+- Save $r0 in pt_regs since it can be cloberred afterward
+- If tracing was enabled, call trace_syscall_exit
+- Call work_pending
+- Return to user !
+
+The trace call is handled out of the fast path. All slow path handling
+is done in another part of code to avoid messing with the cache.
+
+Signals
+=======
+
+Signals are handled when exiting kernel before returning to user.
+When handling a signal, the path is the following:
+
+1 - User application is executing normally
+ Then any exception happens (syscall, interrupt, trap)
+2 - The exception handling path is taken
+ and before returning to user, pending signals are checked
+3 - Signal are handled by do_signal
+ Registers are saved and a special part of the stack is modified
+ to create a trampoline to call rt_sigreturn
+ $spc is modified to jump to user signal handler
+ $ra is modified to jump to sigreturn trampoline directly after
+ returning from user signal handler.
+4 - User signal handler is called after rfe from exception
+ when returning, $ra is retored to $pc, resulting in a call
+ to the syscall trampoline.
+5 - syscall trampoline is executed, leading to rt_sigreturn syscall
+6 - rt_sigreturn syscall is executed
+ Previous registers are restored to allow returning to user correctly
+7 - User application is restored at the exact point it was interrupted
+ before.
+
+
+ +----------+
+ | 1 |
+ | User app | @func
+ | (user) |
+ +---+------+
+ |
+ | it/trap/scall
+ |
+ +---v-------+
+ | 2 |
+ | exception |
+ | handling |
+ | (kernel) |
+ +---+-------+
+ |
+ | Check if signal are pending, if so, handle signals
+ |
+ +---v--------+
+ | 3 |
+ | do_signal |
+ | handling |
+ | (kernel) |
+ +----+-------+
+ |
+ | Return to user signal handler
+ |
+ +----v------+
+ | 4 |
+ | signal |
+ | handler |
+ | (user) |
+ +----+------+
+ |
+ | Return to sigreturn trampoline
+ |
+ +----v-------+
+ | 5 |
+ | syscall |
+ |rt_sigreturn|
+ | (user) |
+ +----+-------+
+ |
+ | Syscall to rt_sigreturn
+ |
+ +----v-------+
+ | 6 |
+ | sigreturn |
+ | handler |
+ | (kernel) |
+ +----+-------+
+ |
+ | Modify context to return to original func
+ |
+ +----v-----+
+ | 7 |
+ | User app | @func
+ | (user) |
+ +----------+
+
+Registers handling
+==================
+
+MMU is disabled in all exceptions paths, during register save and restoration.
+This will prevent from triggering MMU fault (such as TLB miss) which could
+clobber the current register state. Such event can occurs when RWX mode is
+enabled and the memory accessed to save register can trigger a TLB miss.
+Aside from that which is common for all exceptions path, registers are saved
+differently regarding the type of exception.
+
+Interrupts and traps
+--------------------
+
+When interrupt and traps are triggered, we only save the caller-saved registers.
+Indeed, we rely on the fact that C code will save and restore callee-saved and
+hence, there is no need to save them. This path is the following:
+
+ +------------+ +-----------+ +---------------+
+IT | Save caller| C Call | Execute C | Ret | Restore caller| Ret from IT
++--->+ saved +--------->+ handler +------->+ saved +----->
+ | registers | +-----------+ | registers |
+ +------------+ +---------------+
+
+However, when returning to user, we check if there is work_pending. If a signal
+is pending and there is a signal handler to be called, then we need all
+registers to be saved on the stack in the pt_regs before executing the signal
+handler and restored after that. Since we only saved caller-saved registers, we
+need to also save callee-saved registers to restore them correctly when
+returning to user. This path is the following (a bit more complicated !):
+
+ +------------+
+ | Save caller| +-----------+ Ret +------------+
+ IT | saved | C Call | Execute C | to asm | Check work |
+ +--->+ registers +--------->+ handler +------->+ pending |
+ | to pt_regs | +-----------+ +--+---+-----+
+ +------------+ | |
+ Work pending | | No work pending
+ +--------------------------------------------+ |
+ | |
+ | +------------+
+ v |
+ +------+------+ v
+ | Save callee | +-------+-------+
+ | saved | | Restore caller| RFE from IT
+ | registers | | saved +------->
+ | to pt_regs | | registers |
+ +--+-------+--+ | from pt_regs |
+ | | +-------+-------+
+ | | +---------+ ^
+ | | | Execute | |
+ | +-------->+ needed +-----------+
+ | | work |
+ | +---------+
+ |Signal handler ?
+ v
++----+----------+ RFE to user +-------------+ +--------------+
+| Copy all | handler | Execute | ret | rt_sigreturn |
+| registers +------------>+ user signal +------>+ trampoline |
+| from pt_regs | | handler | | to kernel |
+| to user stack | +-------------+ +------+-------+
++---------------+ |
+ syscall rt_sigreturn |
+ +-------------------------------------------------+
+ |
+ v
++--------+-------+ +-------------+
+| Recopy all | | Restore all | RFE
+| registers from +--------------------->+ saved +------->
+| user stack | Return | registers |
+| to pt_regs | from sigreturn |from pt_regs |
++----------------+ (via ret_from_fork) +-------------+
+
+
+Syscalls
+--------
+As explained before, for syscalls, we can use whatever callee-saved registers
+we want since syscall are seen as a "classic" call from ABI pov.
+Only different path is the one for clone. For this path, since the child expects
+to find same callee-registers content than his parent, we must save them before
+executing the clone syscall and restore them after that for the child. This is
+done via a redefinition of __sys_clone in assembly which will be called in place
+of the standard sys_clone. This new call will save callee saved registers
+in pt_regs. Parent will return using the syscall standard path. Freshly spawned
+child however will be woken up via ret_from_fork which will restore all
+registers (even if caller saved are not needed).
diff --git a/Documentation/kvx/kvx-iommu.txt b/Documentation/kvx/kvx-iommu.txt
new file mode 100644
index 000000000000..96b74ce71acb
--- /dev/null
+++ b/Documentation/kvx/kvx-iommu.txt
@@ -0,0 +1,183 @@
+IOMMU
+=====
+
+General Overview
+----------------
+
+To exchange data between device and users through memory, the driver has to
+set up a buffer by doing some kernel allocation. The address of the buffer is
+virtual and the physical one is obtained through the MMU. When the device wants
+to access the same physical memory space it uses a bus address. This address is
+obtained by using the DMA mapping API. The Coolidge SoC includes several IOMMUs for clusters,
+PCIe peripherals, SoC peripherals, and more; that will translate this "bus address"
+into a physical one during DMA operations.
+
+The bus addresses are IOVA (I/O Virtual Address) or DMA addresses. This
+addresses can be obtained by calling the allocation functions of the DMA APIs.
+It can also be obtained through classical kernel allocation of physical
+contiguous memory and then calling mapping functions of the DMA API.
+
+In order to be able to use the kvx IOMMU we have implemented the IOMMU DMA
+interface in arch/kvx/mm/dma-mapping.c. DMA functions are registered by
+implementing arch_setup_dma_ops() and generic IOMMU functions. Generic IOMMU
+are calling our specific IOMMU functions that adding or removing mappings
+between DMA addresses and physical addresses in the IOMMU TLB.
+
+Specifics IOMMU functions are defined in the kvx IOMMU driver. A kvx IOMMU
+driver is managing two physical hardware IOMMU used for TX and RX. In the next
+section we described the HW IOMMUs.
+
+
+Cluster IOMMUs
+--------------
+
+IOMMUs on cluster are used for DMA and cryptographic accelerators.
+There are six IOMMUs connected to the:
+ - cluster DMA tx
+ - cluster DMA rx
+ - first non secure cryptographic accelerator
+ - second non secure cryptographic accelerator
+ - first secure cryptographic accelerator
+ - second secure cryptographic accelerator
+
+SoC peripherals IOMMUs
+----------------------
+
+Since SoC peripherals are connected to an AXI bus, two IOMMUs are used: one for
+each AXI channel (read and write). These two IOMMUs are shared between all master
+devices and DMA. These two IOMMUs will have the same entries but need to be configured
+independently.
+
+PCIe IOMMUs
+-----------
+
+There is a slave IOMMU (read and write from the MPPA to the PCIe endpoint)
+and a master IOMMU (read and write from a PCIe endpoint to system DDR).
+The PCIe root complex and the MSI/MSI-X controller have been designed to use
+the IOMMU feature when enabled. (For example for supporting endpoint that
+support only 32 bits addresses and allow them to access any memory in a
+64 bits address space). For security reason it is highly recommended to
+activate the IOMMU for PCIe.
+
+IOMMU implementation
+--------------------
+
+The kvx is providing several IOMMUs. Here is a simplified view of all IOMMUs
+and translations that occurs between memory and devices:
+
+ +---------------------------------------------------------------------+
+ | +------------+ +---------+ | CLUSTER X |
+ | | Cores 0-15 +---->+ Crypto | +-----------|
+ | +-----+------+ +----+----+ |
+ | | | |
+ | v v |
+ | +-------+ +------------------------------+ |
+ | | MMU | +----+ IOMMU x4 (secure + insecure) | |
+ | +---+---+ | +------------------------------+ |
+ | | | |
+ +--------------------+ |
+ | | | |
+ v v | |
+ +---+--------+-+ | |
+ | MEMORY | | +----------+ +--------+ +-------+ |
+ | +<-|-----+ IOMMU Rx |<----+ DMA Rx |<----+ | |
+ | | | +----------+ +--------+ | | |
+ | | | | NoC | |
+ | | | +----------+ +--------+ | | |
+ | +--|---->| IOMMU Tx +---->| DMA Tx +---->+ | |
+ | | | +----------+ +--------+ +-------+ |
+ | | +------------------------------------------------+
+ | |
+ | | +--------------+ +------+
+ | |<--->+ IOMMU Rx/Tx +<--->+ PCIe +
+ | | +--------------+ +------+
+ | |
+ | | +--------------+ +------------------------+
+ | |<--->+ IOMMU Rx/Tx +<--->+ master Soc Peripherals |
+ | | +--------------+ +------------------------+
+ +--------------+
+
+
+There is also an IOMMU dedicated to the crypto module but this module will not
+be accessed by the operating system.
+
+We will provide one driver to manage IOMMUs RX/TX. All of them will be
+described in the device tree to be able to get their particularities. See
+the example below that describes the relation between IOMMU, DMA and NoC in
+the cluster.
+
+IOMMU is related to a specific bus like PCIe we will be able to specify that
+all peripherals will go through this IOMMU.
+
+### IOMMU Page table
+
+We need to be able to know which IO virtual addresses (IOVA) are mapped in the
+TLB in order to be able to remove entries when a device finishes a transfer and
+release memory. This information could be extracted when needed by computing all
+sets used by the memory and then reads all sixteen ways and compare them to the
+IOVA but it won't be efficient. We also need to be able to translate an IOVA
+to a physical address as required by the iova_to_phys IOMMU ops that is used
+by DMA. Like previously it can be done by extracting the set from the address
+and comparing the IOVA to each sixteen entries of the given set.
+
+A solution is to keep a page table for the IOMMU. But this method is not
+efficient for reloading an entry of the TLB without the help of an hardware
+page table. So to prevent the need of a refill we will update the TLB when a
+device request access to memory and if there is no more slot available in the
+TLB we will just fail and the device will have to try again later. It is not
+efficient but at least we won't need to manage the refill of the TLB.
+
+This leads to an issue with the memory that can be used for transfer between
+device and memory (see Limitations below). As we only support 4Ko page size we
+can only map 8Mo. To be able to manage bigger transfer we can implement the
+huge page table in the Linux kernel and use a page table that match the size of
+huge page table for a given IOMMU (typically the PCIe IOMMU).
+
+As we won't refill the TLB we know that we won't have more than 128*16 entries.
+In this case we can simply keep a table with all possible entries.
+
+### Maintenance interface
+
+It is possible to have several "maintainers" for the same IOMMU. The driver is
+using two of them. One that writes the TLB and another interface reads TLB. For
+debug purpose it is possible to display the content of the tlb by using the
+following command in gdb:
+
+ gdb> p kvx_iommu_dump_tlb( <iommu addr>, 0)
+
+Since different management interface are used for read and write it is safe to
+execute the above command at any moment.
+
+### Interrupts
+
+IOMMU can have 3 kind of interrupts that corresponds to 3 different types of
+errors (no mapping. protection, parity). When the IOMMU is shared between
+clusters (SoC periph and PCIe) then fifteen IRQs are generated according to the
+configuration of an association table. The association table is indexed by the
+ASN number (9 bits) and the entry of the table is a subscription mask with one
+bit per destination. Currently this is not managed by the driver.
+
+The driver is only managing interrupts for the cluster. The mode used is the
+stall one. So when an interrupt occurs it is managed by the driver. All others
+interrupts that occurs are stored and the IOMMU is stalled. When driver cleans
+the first interrupt others will be managed one by one.
+
+### ASN (Address Space Number)
+
+This is also know as ASID in some other architecture. Each device will have a
+given ASN that will be given through the device tree. As address space is
+managed at the IOMMU domain level we will use one group and one domain per ID.
+ASN are coded on 9 bits.
+
+Device tree
+-----------
+
+Relationships between devices, DMAs and IOMMUs are described in the
+device tree (see Documentation/devicetree/bindings/iommu/kalray,kvx-iommu.txt
+for more details).
+
+Limitations
+-----------
+
+Only supporting 4 KB page size will limit the size of mapped memory to 8 MB
+because the IOMMU TLB can have at most 128*16 entries.
diff --git a/Documentation/kvx/kvx-mmu.txt b/Documentation/kvx/kvx-mmu.txt
new file mode 100644
index 000000000000..a3ebbef36981
--- /dev/null
+++ b/Documentation/kvx/kvx-mmu.txt
@@ -0,0 +1,272 @@
+MMU
+===
+
+Virtual addresses are on 41 bits for kvx when using 64-bit mode.
+To differentiate kernel from user space, we use the high order bit
+(bit 40). When bit 40 is set, then the higher remaining bits must also be set to
+1. The virtual address must be extended with 1 when the bit 40 is set,
+if not the address must be zero extended. Bit 40 is set for kernel space
+mappings and not set for user space mappings.
+
+Memory Map
+==========
+
+In Linux physical memories are arranged into banks according to the cost of an
+access in term of distance to a memory. As we are UMA architecture we only have
+one bank and thus one node.
+
+A node is divided into several kind of zone. For example if DMA can only access
+a specific area in the physical memory we will define a ZONE_DMA for this purpose.
+In our case we are considering that DMA can access all DDR so we don't have a specific
+zone for this. On 64 bit architecture all DDR can be mapped in virtual kernel space
+so there is no need for a ZONE_HIGHMEM. That means that in our case there is
+only one ZONE_NORMAL. This will be updated if DMA cannot access all memory.
+
+Currently, the memory mapping is the following for 4KB page:
+
++-----------------------+-----------------------+------+-------+--------------+
+| Start | End | Attr | Size | Name |
++-----------------------+-----------------------+------+-------+--------------+
+| 0000 0000 0000 0000 | 0000 003F FFFF FFFF | --- | 256GB | User |
+| 0000 0040 0000 0000 | 0000 007F FFFF FFFF | --- | 256GB | MMAP |
+| 0000 0080 0000 0000 | FFFF FF7F FFFF FFFF | --- | --- | Gap |
+| FFFF FF80 0000 0000 | FFFF FFFF FFFF FFFF | --- | 512GB | Kernel |
+| FFFF FF80 0000 0000 | FFFF FF8F FFFF FFFF | RWX | 64GB | Direct Map |
+| FFFF FF90 0000 0000 | FFFF FF90 3FFF FFFF | RWX | 1GB | Vmalloc |
+| FFFF FF90 4000 0000 | FFFF FFFF FFFF FFFF | RW | 447GB | Free area |
++-----------------------+-----------------------+------+-------+--------------+
+
+Enable the MMU
+==============
+
+All kernel functions and symbols are in virtual memory except for kvx_start()
+function which is loaded at 0x0 in physical memory.
+To be able to switch from physical addresses to virtual addresses we choose to
+setup the TLB at the very beginning of the boot process to be able to map both
+pieces of code. For this we added two entries in the LTLB. The first one,
+LTLB[0], contains the mapping between virtual memory and DDR. Its size is 512MB.
+The second entry, LTLB[1], contains a flat mapping of the first 2MB of the SMEM.
+Once those two entries are present we can enable the MMU. LTLB[1] will be
+removed during paging_init() because once we are really running in virtual space
+it will not be used anymore.
+In order to access more than 512MB DDR memory, the remaining memory (> 512MB) is
+refill using a comparison in kernel_perf_refill that does not walk the kernel
+page table, thus having a faster refill time for kernel. These entries are
+inserted into the LTLB for easier computation (4 LTLB entries). The drawback of
+this approach is that mapped entries are using RWX protection attributes,
+leading to no protection at all.
+
+Kernel strict RWX
+=================
+
+CONFIG_STRICT_KERNEL_RWX is enabled by default in default_defconfig.
+Once booted, if CONFIG_STRICT_KERNEL_RWX is enable, the kernel text and memory
+will be mapped in the init_mm page table. Once mapped, the refill routine for
+the kernel is patched to always do a page table walk, bypassing the faster
+comparison but enforcing page protection attributes when refilling.
+Finally, the LTLB[0] entry is replaced by a 4K one, mapping only exceptions with
+RX protection. It allows us to never trigger nomapping on nomapping refill
+routine which would (obviously) not work... Once this is done, we can flush the
+4 LTLB entries for kernel refill in order to be sure there is no stalled
+entries and that new entries inserted in JTLB will apply.
+
+By default, the following policy is applied on vmlinux sections:
+- init_data: RW
+- init_text: RX (or RWX if parameter rodata=off)
+- text: RX (or RWX if parameter rodata=off)
+- rodata: RW before init, RO after init
+- sdata: RW
+
+Kernel RWX mode can then be switched on/off using /sys/kvx/kernel_rwx file.
+
+Privilege Level
+================
+Since we are using privilege levels on kvx, we make use of the virtual
+spaces to be in the same space as the user. The kernel will have the
+$ps.mmup set in kernel (PL1) and unset for user (PL2).
+As said in kvx documentation, we have two cases when the kernel is
+booted:
+- Either we have been booted by someone (bootloader, hypervisor, etc)
+- Or we are alone (boot from flash)
+
+In both cases, we will use the virtual space 0. Indeed, if we are alone
+on the core, then it means nobody is using the MMU and we can take the
+first virtual space. If not alone, then when writing an entry to the tlb
+using writetlb instruction, the hypervisor will catch it and change the
+virtual space accordingly.
+
+Memblock
+========
+
+When the kernel starts there is no memory allocator available. One of the first
+step in the kernel is to detect the amount of DDR available by getting this
+information in the device tree and initialize the low-level "memblock" allocator.
+
+We start by reserving memory for the whole kernel. For instance with a device
+tree containing 512Mo of DDR you could see the following boot messages:
+
+setup_bootmem: Memory : 0x100000000 - 0x120000000
+setup_bootmem: Reserved: 0x10001f000 - 0x1002d1bc0
+
+During the paging init we need to set:
+ - min_low_pfn that is the lowest PFN available in the system
+ - max_low_pfn that indicates the end if NORMAL zone
+ - max_pfn that is the number of pages in the system
+
+This setting is used for dividing memory into pages and for configuring the
+zone. See the memory map section for more information about ZONE.
+
+Zones are configured in free_area_init_core(). During start_kernel() other
+allocations are done for command line, cpu areas, PID hash table, different
+caches for VFS. This allocator is used until mem_init() is called.
+
+mem_init() is provided by the architecture. For MPPA we just call
+free_all_bootmem() that will go through all pages that are not used by the
+low level allocator and mark them as not used. So physical pages that are
+reserved for the kernel are still used and remain in physical memory. All pages
+released will now be used by the buddy allocator.
+
+Peripherals
+===========
+
+Peripherals are mapped using standard ioremap infrastructure, therefore
+mapped addresses are located in the vmalloc space.
+
+LTLB Usage
+==========
+
+LTLB is used to add resident mapping which allows for faster MMU lookup.
+Currently, the LTLB is used to map some mandatory kernel pages and to allow fast
+accesses to l2 cache (mailbox and registers).
+When CONFIG_STRICT_KERNEL_RWX is disabled, 4 entries are reserved for kernel
+TLB refill using 512MB pages. When CONFIG_STRICT_KERNEL_RWX is enabled, these
+entries are unused since kernel is paginated using the same mecanism than for
+user (page walking and entries in JTLB)
+
+Page Table
+==========
+
+We only support three levels for the page table and 4KB for page size.
+
+3 levels page table
+-------------------
+
+...-----+--------+--------+--------+--------+--------+
+ 40|39 32|31 24|23 16|15 8|7 0|
+...-----++-------+--+-----+---+----+----+---+--------+
+ | | | |
+ | | | +---> [11:0] Offset (12 bits)
+ | | +-------------> [20:12] PTE offset (9 bits)
+ | +-----------------------> [29:21] PMD offset (9 bits)
+ +----------------------------------> [39:30] PGD offset (10 bits)
+Bits 40 to 64 are signed extended according to bit 39. If bit 39 is equal to 1
+we are in kernel space.
+
+As 10 bits are used for PGD we need to allocate 2 pages.
+
+PTE format
+==========
+
+About the format of the PTE entry, as we are not forced by hardware for choices,
+we choose to follow the format described in the RiscV implementation as a
+starting point.
+
+ +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+ | 63..23 | 22..13 | 12 | 11..10 | 9 | 8 | 7 | 6 | 5 | 4 | 3..2 | 1 | 0 |
+ +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+ PFN Unused S PageSZ H G X W R D CP A P
+ where:
+ P: Present
+ A: Accessed
+ CP: Cache policy
+ D: Dirty
+ R: Read
+ W: Write
+ X: Executable
+ G: Global
+ H: Huge page
+ PageSZ: Page size as set in TLB format (0:4Ko, 1:64Ko, 2:2Mo, 3:512Mo)
+ S: Soft/Special
+ PFN: Page frame number (depends on page size)
+
+Huge bit must be somewhere in the first 12 bits to be able to detect it
+when reading the PMD entry.
+
+PageSZ must be on bit 10 and 11 because it matches the TEL.PS bits. And
+by doing that it is easier in assembly to set the TEL.PS to PageSZ.
+
+Fast TLB refill
+===============
+
+kvx core does not feature a hardware page walker. This work must be done
+by the core in software. In order to optimize TLB refill, a special fast
+path is taken when entering in kernel space.
+In order to speed up the process, the following actions are taken:
+# Save some registers in a per process scratchpad
+# If the trap is a nomapping then try the fastpath
+# Save some more registers for this fastpath
+# Check if faulting address is a memory direct mapping one.
+ # If entry is a direct mapping one and RWX is not enabled, add an entry into LTLB
+ # If not, continue
+# Try to walk the page table
+ # If entry is not present, take the slowpath (do_page_fault)
+# Refill the tlb properly
+# Exit by restoring only a few registers
+
+ASN Handling
+============
+
+Disclaimer: Some part of this are taken from ARC architecture.
+
+kvx MMU provides 9-bit ASN (Address Space Number) in order to tag TLB entries.
+It allows for multiple process with the same virtual space to cohabit without
+the need to flush TLB everytime we context switch.
+kvx implementation to use them is based on other architectures (such as arc
+or xtensa) and uses a wrapping ASN counter containing both cycle/generation and
+asn.
+
++---------+--------+
+|63 10|9 0|
++---------+--------+
+ Cycle ASN
+
+This ASN counter is incremented monotonously to allocate new ASNs. When the
+counter reaches 511 (9 bit), TLB is completely flushed and a new cycle is
+started. A new allocation cycle, post rollover, could potentially reassign an
+ASN to a different task. Thus the rule is to reassign an ASN when the current
+context cycles does not match the allocation cycle.
+The 64 bit @cpu_asn_cache (and mm->asn) have 9 bits MMU ASN and rest 55 bits
+serve as cycle/generation indicator and natural 64 bit unsigned math
+automagically increments the generation when lower 9 bits rollover.
+When the counter completely wraps, we reset the counter to first cycle value
+(ie cycle = 1). This allows to distinguish context without any ASN and old cycle
+generated value with the same operation (XOR on cycle).
+
+Huge page
+=========
+
+Currently only 3 level page table has been implemented for 4Ko base page size.
+So the page shift is 12 bits, the pmd shift is 21 and the pgdir shift is 30
+bits. This choice implies that for 4Ko base page size if we use a PMD as a huge
+page the size will be 2Mo and if we use a PUD as a huge page it will be 1Go.
+
+To support other huge page sizes (64Ko and 512Mo) we need to use several
+contiguous entries in the page table. For huge page of 64Ko we will need to
+use 16 entries in the PTE and for a huge page of 512Mo it means that 256
+entries in PMD will be used.
+
+Debug
+=====
+
+In order to debug the page table and tlb entries, gdb scripts contains commands
+which allows to dump the page table:
+- lx-kvx-page-table-walk
+ - Display the current process page table by default
+- lx-kvx-tlb-decode
+ - Display the content of $tel and $teh into something readable
+
+Other commands available in kvx-gdb are the following:
+- mppa-dump-tlb
+ - Display the content of TLBs (JTLB and LTLB)
+- mppa-lookup-addr
+ - Find physical address matching a virtual one
diff --git a/Documentation/kvx/kvx-smp.txt b/Documentation/kvx/kvx-smp.txt
new file mode 100644
index 000000000000..1b69d77db8cd
--- /dev/null
+++ b/Documentation/kvx/kvx-smp.txt
@@ -0,0 +1,36 @@
+SMP
+===
+
+On kvx, 5 clusters are organized as groups of 16 processors + 1
+secure core (RM) for each cluster. These 17 processors are L1$ coherent
+for TCM (tightly Coupled Memory). A mixed hw/sw L2$ is present to have
+cache coherency on DDR as well as TCM.
+The RM manager is not meant to run Linux so, 16 processors are available
+for SMP.
+
+Booting
+=======
+
+When booting the kvx processor, only the RM is woken up. This RM will
+execute a portion of code located in a section named .rm_firmware.
+By default, a simple power off code is embedded in this section.
+To avoid embedding the firmware in kernel sources, the section is patched
+using external tools to add the L2$ firmware (and replace the default firmware).
+Before executing this firmware, the RM boots the PE0. PE0 will then enable L2
+coherency and request will be stalled until RM boots the L2$ firmware.
+
+Locking primitives
+==================
+
+spinlock/rwlock are using the kernel standard queued spinlock/rwlocks.
+These primitives are based on cmpxch and xchg. More particularly, it uses xchg16
+which is implemented as a read modify write with acswap on 32 bit word since
+kvx does not have cmpxchg for size < 32bits.
+
+IPI
+===
+
+An IPI controller allows to communicate between CPUs using a simple
+memory mapped register. This register can simply be written using a mask to
+trigger interrupts directly to the cores matching the mask.
+
diff --git a/Documentation/kvx/kvx.txt b/Documentation/kvx/kvx.txt
new file mode 100644
index 000000000000..8ce0703de681
--- /dev/null
+++ b/Documentation/kvx/kvx.txt
@@ -0,0 +1,268 @@
+kvx Core Implementation
+=======================
+
+This documents will try to explain any architecture choice for the kvx
+linux port.
+
+Regarding the peripheral, we MUST use device tree to describe ALL
+peripherals. The bindings should always start with "kalray,kvx" for all
+core related peripherals (watchdog, timer, etc)
+
+System Architecture
+===================
+
+On kvx, we have 4 levels of privilege level starting from 0 (most
+privileged one) to 3 (less privilege one). A system of owners allows
+to delegate ownership of resources by using specials system registers.
+
+The 2 main software stacks for Linux Kernel are the following:
+
++-------------+ +-------------+
+| PL0: Debug | | PL0: Debug |
++-------------+ +-------------+
+| PL1: Linux | | PL1: HyperV |
++-------------+ +-------------+
+| PL2: User | | PL2: Linux |
++-------------+ +-------------+
+| | | PL3: User |
++-------------+ +-------------+
+
+In both cases, the kvx support for privileges has been designed using
+only relative PL and thus should work on both configurations without
+any modifications.
+
+When booting, the CPU is executing in PL0 and owns all the privileges.
+This level is almost dedicated to the debug routines for the debugguer.
+It only needs to own few privileges (breakpoint 0 and watchpoint 0) to
+be able to debug a system executing in PL1 to PL3.
+Debug routines are not always there for instance when the kernel is
+executing alone (booted from flash).
+In order to ease the load of debug routines, software convention is to
+jump directly to PL1 and let PL0 for the debug.
+When the kernel boots, it checks if the current privilege level is 0
+($ps.pl is the only absolute value). If so, then it will delegate
+almost all resources to PL1 and use a RFE to lower its execution
+privilege level (see asm_delegate_pl in head.S).
+If the current PL is already different from 0, then it means somebody
+is above us and we need to request resource to inform it we need them. It will
+then either delegate them to us directly or virtualize the delegation.
+All privileges levels have their set of banked registers (ps, ea, sps,
+sr, etc) which contain privilege level specific values.
+$sr (system reserved) is banked and will hold the current task_struct.
+This register is reserved and should not be touched by any other code.
+For more information, refer to the kvx system level architecture manual.
+
+Boot
+====
+
+On kvx, the RM (Secure Core) of Cluster 0 will boot first. It will then be able
+to boot a firmware. This firmware is stored in the rm_firmware section.
+The first argument ($r0) of this firmware will be a pointer to a function with
+the following prototype: void firmware_init_done(uint64_t features). This
+function is responsible of describing the features supported by the firmware and
+will start the first PE after that.
+By default, the rm_firmware function act as the "default" firmware. This
+function does nothing except calling firmware_init_done and then goes to sleep.
+In order to add another firmware, the rm_firmware section is patched using
+objcopy. The content of this section is then replaced by the provided firmware.
+This firmware will do an init and then call firmware_init_done before running
+the main loop.
+When the PE boots, it will check for the firmware features to enable or disable
+specific core features (L2$ for instance).
+
+When entering the C (kvx_lowlevel_start) the kernel will look for a special
+magic in $r0 (0x494C314B). This magic tells the kernel if there is arguments
+passed by a bootloader.
+Currently, the following values are passed through registers:
+ - r1: pointer to command line setup by bootloader
+ - r2: device tree
+
+If this magic is not set, then, the command line will be the one
+provided in the device tree (see bootargs). The default device tree is
+not builtin but will be patched by the runner used (simulator or jtag) in the
+dtb section.
+
+A default stdout-path is desirable to allow early printk.
+
+Boot Memory Allocator
+=====================
+
+The boot memory allocator is used to allocate memory before paging is enabled.
+It is initialized with DDR and also with the shared memory. This first one is
+initialized during the setup_bootmem() and the second one when calling
+early_init_fdt_scan_reserved_mem().
+
+
+Virtual and physical memory
+===========================
+
+The mapping used and the memory management is described in
+Documentation/kvx/kvx-mmu.txt.
+Our Kernel is compiled using virtual addresses that starts at
+0xffffff0000000000. But when it is started the kernel uses physical addresses.
+Before calling the first function arch_low_level_start() we configure 2 entries
+of the LTLB.
+
+The first entry will map the first 1G of virtual address space to the first
+1G of DDR:
+ - TLB[0]: 0xffffff0000000000 -> 0x100000000 (size 512Mo)
+
+The second entry will be a flat mapping of the first 512 Ko of the SMEM. It
+is required to have this flat mapping because there is still code located at
+this address that needs to be executed:
+ - TLB[1]: 0x0 -> 0x0 (size 512Ko)
+
+Once virtual space reached the second entry is removed.
+
+To be able to set breakpoints when MMU is enabled we added a label called
+gdb_mmu_enabled. If you try to set a breakpoint on a function that is in
+virtual memory before the activation of the MMU this address as no signification
+for GDB. So, for example, if you want to break on the function start_kernel()
+you will need to run:
+
+ kvx-gdb -silent path_to/vmlinux \
+ -ex 'tbreak gdb_mmu_enabled' -ex 'run' \
+ -ex 'break start_kernel' \
+ -ex 'continue'
+
+We will also add an option to kvx-gdb to simplify this step.
+
+Timers
+======
+
+The free-runinng clock (clocksource) is based on the DSU. This clock is
+not interruptible and never stops even if core go into idle.
+
+Regarding the tick (clockevent), we use the timer 0 available on the core.
+This timer allows to set a periodic tick which will be used as the main
+tick for each core. Note that this clock is percpu.
+
+get_cycles implementation is based on performance counter. One of them
+is used to count cycles. Note that since this is used only when the core
+is running, there is no need to worry about core sleeping (which will
+stop the cycle counter)
+
+Context switching
+=================
+
+context switching is done in entry.S. When spawning a fresh thread,
+copy_thread is called. During this call, we setup callee saved register
+r20 and r21 to special values containing the function to call.
+
+The normal path for a kernel thread will be the following:
+
+ 1 - Enter copy_thread_tls and setup callee saved registers which will
+ be restored in __switch_to.
+ 2 - set r20 and r21 (in thread_struct) to function and argument and
+ ra to ret_from_kernel_thread.
+ These callee saved will be restored in switch_to.
+ 3 - Call _switch_to at some point.
+ 4 - Save all callee saved register since switch_to is seen as a
+ standard function call by the caller.
+ 5 - Change stack pointer to the new stack
+ 6 - At the end of switch to, set sr0 to the new task and use ret to
+ jump to ret_from_kernel_thread (address restored from ra).
+ 7 - In ret_from_kernel_thread, execute the function with arguments by
+ using r20, r21 and we are done
+
+For more explanation, you can refer to https://lwn.net/Articles/520227/
+
+User thread creation
+====================
+
+We are using almost the same path as copy_thread to create it.
+The detailed path is the following:
+
+ 1 - Call start_thread which will setup user pc and stack pointer in
+ task regs. We also set sps and clear privilege mode bit.
+ When returning from exception, it will "flip" to user mode.
+ 2 - Enter copy_thread_tls and setup callee saved registers which will
+ be restored in __switch_to. Also, set the "return" function to be
+ ret_from_fork which will be called at end of switch_to
+ 3 - set r20 (in thread_struct) with tracing information.
+ (simply by lazyness to avoid computing it in assembly...)
+ 4 - Call _switch_to at some point.
+ 5 - The current pc will then be restored to be ret_from fork.
+ 6 - Ret from fork calls schedule_tail and then check if tracing is
+ enabled. If so call syscall_trace_exit
+ 7 - finally, instead of returning to kernel, we restore all registers
+ that have been setup by start_thread by restoring regs stored on
+ stack
+
+L2$ handling
+============
+
+On kvx, the L2$ is handled by a firmware running on the RM. This firmware needs
+various information to be aware of its configuration and communicate with the
+kernel. In order to do that, when firmware is starting, the device tree is given
+as parameter along with the "registers" zone. This zone is simply a memory area
+where data are exchanged between kernel <-> L2$. When some commands are written
+to it, the kernel sends an interrupt using a mailbox.
+If the L2$ node is not present in the device tree, then, the RM will directly go
+into sleeping.
+
+Boot diagram:
+
+ RM PE 0
+ +
+ +---------+ |
+ | Boot | |
+ +----+----+ |
+ | |
+ v |
+ +-----+-----+ |
+ | Prepare | |
+ | L2 shared | |
+ | memory | |
+ |(registers)| |
+ +-----+-----+ |
+ | | +-----------+
+ +------------------->+ Boot |
+ | | +-----+-----+
+ v | |
+ +--------+---------+ | |
+ | L2 firmware | | |
+ | parameters: | | |
+ | r0 = registers | | |
+ | r1 = DTB | | |
+ +--------+---------+ | |
+ | | |
+ v | |
+ +-------+--------+ | +------+------+
+ | L2 firmware | | | Wait for L2 |
+ | execution | | | to be ready |
+ +-------+--------+ | +------+------+
+ | | |
+ +------v-------+ | v
+ | L2 requests | | +------+------+
++--->+ handling | | | Enable |
+| +-------+------+ | | L2 caching |
+| | | +------+------+
+| | | |
++------------+ + v
+
+
+Since this driver is started early (before SMP boot), A lot of drivers are not
+yet probed (mailboxes, iommu, etc) and thus can not be used.
+
+Building
+========
+
+In order to build the kernel, you will need a complete kvx toolchain.
+First, setup the config using the following command line
+
+$ make ARCH=kvx O=your_directory default_defconfig
+
+Adjust any configuration option you may need and then, build the kernel:
+
+$ make ARCH=kvx O=your_directory -j12
+
+You will finally have a vmlinux image ready to be run.
+
+$ kvx-mppa -- vmlinux
+
+Additionally, you may want to debug it. To do so, use kvx-gdb:
+
+$ kvx-gdb vmlinux
+
+
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation
2023-01-03 16:43 ` [RFC PATCH 01/25] Documentation: kvx: Add basic documentation Yann Sionneau
@ 2023-01-03 17:50 ` Jonathan Corbet
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
2023-01-18 8:44 ` [RFC PATCH 01/25] Documentation: kvx: Add basic documentation Yann Sionneau
2023-01-05 18:38 ` kernel test robot
2023-01-18 15:09 ` Jeff Xie
2 siblings, 2 replies; 113+ messages in thread
From: Jonathan Corbet @ 2023-01-03 17:50 UTC (permalink / raw)
To: Yann Sionneau
Cc: Yann Sionneau, linux-doc, linux-kernel, Clement Leger,
Guillaume Thouvenin
Yann Sionneau <ysionneau@kalray.eu> writes:
> Add some documentation for kvx arch and its Linux port.
>
> CC: Jonathan Corbet <corbet@lwn.net>
> CC: linux-doc@vger.kernel.org
> CC: linux-kernel@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> Documentation/kvx/kvx-exceptions.txt | 246 ++++++++++++++++++++++++
> Documentation/kvx/kvx-iommu.txt | 183 ++++++++++++++++++
> Documentation/kvx/kvx-mmu.txt | 272 +++++++++++++++++++++++++++
> Documentation/kvx/kvx-smp.txt | 36 ++++
> Documentation/kvx/kvx.txt | 268 ++++++++++++++++++++++++++
> 5 files changed, 1005 insertions(+)
> create mode 100644 Documentation/kvx/kvx-exceptions.txt
> create mode 100644 Documentation/kvx/kvx-iommu.txt
> create mode 100644 Documentation/kvx/kvx-mmu.txt
> create mode 100644 Documentation/kvx/kvx-smp.txt
> create mode 100644 Documentation/kvx/kvx.txt
Please write this documentation in the RST format (you're 95% of the way
there now) and incorporate into the kernel docs build.
Thanks,
jon
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation)
2023-01-03 17:50 ` Jonathan Corbet
@ 2023-01-09 9:51 ` Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 1/8] Documentation: kvx: Convert to reST Bagas Sanjaya
` (9 more replies)
2023-01-18 8:44 ` [RFC PATCH 01/25] Documentation: kvx: Add basic documentation Yann Sionneau
1 sibling, 10 replies; 113+ messages in thread
From: Bagas Sanjaya @ 2023-01-09 9:51 UTC (permalink / raw)
To: Jonathan Corbet, Yann Sionneau
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin,
Bagas Sanjaya
Jonathan Corbet <corbet@lwn.net> writes:
>> Add some documentation for kvx arch and its Linux port.
>>
>> CC: Jonathan Corbet <corbet@lwn.net>
>> CC: linux-doc@vger.kernel.org
>> CC: linux-kernel@vger.kernel.org
>> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
>> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
>> Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
>> Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
>> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
>> ---
>> Documentation/kvx/kvx-exceptions.txt | 246 ++++++++++++++++++++++++
>> Documentation/kvx/kvx-iommu.txt | 183 ++++++++++++++++++
>> Documentation/kvx/kvx-mmu.txt | 272 +++++++++++++++++++++++++++
>> Documentation/kvx/kvx-smp.txt | 36 ++++
>> Documentation/kvx/kvx.txt | 268 ++++++++++++++++++++++++++
>> 5 files changed, 1005 insertions(+)
>> create mode 100644 Documentation/kvx/kvx-exceptions.txt
>> create mode 100644 Documentation/kvx/kvx-iommu.txt
>> create mode 100644 Documentation/kvx/kvx-mmu.txt
>> create mode 100644 Documentation/kvx/kvx-smp.txt
>> create mode 100644 Documentation/kvx/kvx.txt
>
>Please write this documentation in the RST format (you're 95% of the way
>there now) and incorporate into the kernel docs build.
Here is the polished documentation in reST format. Yann, can you please
squash this series into your documentation patch?
Bagas Sanjaya (8):
Documentation: kvx: Convert to reST
Documentation: kvx: Wrap diagrams in literal code block
Documentation: kvx: Fix lists
Documentation: kvx: kvx-iommu: Use reST syntax for subsections
Documentation: kvx: kvx-iommu: monospacize kvx iommu device tree path
Documentation: kvx: Promote title headings
Documentation: kvx: Use literal code block for command-line inputs
Documentation: kvx: reword
Documentation/arch.rst | 1 +
Documentation/kvx/index.rst | 12 ++
Documentation/kvx/kvx-exceptions.rst | 258 +++++++++++++++++++++++
Documentation/kvx/kvx-exceptions.txt | 246 ----------------------
Documentation/kvx/kvx-iommu.rst | 188 +++++++++++++++++
Documentation/kvx/kvx-iommu.txt | 183 -----------------
Documentation/kvx/kvx-mmu.rst | 294 +++++++++++++++++++++++++++
Documentation/kvx/kvx-mmu.txt | 272 -------------------------
Documentation/kvx/kvx-smp.rst | 36 ++++
Documentation/kvx/kvx-smp.txt | 36 ----
Documentation/kvx/kvx.rst | 269 ++++++++++++++++++++++++
Documentation/kvx/kvx.txt | 268 ------------------------
12 files changed, 1058 insertions(+), 1005 deletions(-)
create mode 100644 Documentation/kvx/index.rst
create mode 100644 Documentation/kvx/kvx-exceptions.rst
delete mode 100644 Documentation/kvx/kvx-exceptions.txt
create mode 100644 Documentation/kvx/kvx-iommu.rst
delete mode 100644 Documentation/kvx/kvx-iommu.txt
create mode 100644 Documentation/kvx/kvx-mmu.rst
delete mode 100644 Documentation/kvx/kvx-mmu.txt
create mode 100644 Documentation/kvx/kvx-smp.rst
delete mode 100644 Documentation/kvx/kvx-smp.txt
create mode 100644 Documentation/kvx/kvx.rst
delete mode 100644 Documentation/kvx/kvx.txt
--
An old man doll... just what I always wanted! - Clara
^ permalink raw reply [flat|nested] 113+ messages in thread* [PATCH 1/8] Documentation: kvx: Convert to reST
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
@ 2023-01-09 9:51 ` Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 2/8] Documentation: kvx: Wrap diagrams in literal code block Bagas Sanjaya
` (8 subsequent siblings)
9 siblings, 0 replies; 113+ messages in thread
From: Bagas Sanjaya @ 2023-01-09 9:51 UTC (permalink / raw)
To: Jonathan Corbet, Yann Sionneau
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin,
Bagas Sanjaya
Convert kvx documentation to reST format. While at it, fix constructs
that cause Sphinx to confuse with section markup.
Note that 101 warnings are generated due to markup syntax issues. These
will be addressed in subsequent patches.
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
---
Documentation/arch.rst | 1 +
Documentation/kvx/index.rst | 12 ++++++++
...{kvx-exceptions.txt => kvx-exceptions.rst} | 10 +++----
.../kvx/{kvx-iommu.txt => kvx-iommu.rst} | 0
.../kvx/{kvx-mmu.txt => kvx-mmu.rst} | 28 +++++++++++--------
.../kvx/{kvx-smp.txt => kvx-smp.rst} | 0
Documentation/kvx/{kvx.txt => kvx.rst} | 0
7 files changed, 35 insertions(+), 16 deletions(-)
create mode 100644 Documentation/kvx/index.rst
rename Documentation/kvx/{kvx-exceptions.txt => kvx-exceptions.rst} (98%)
rename Documentation/kvx/{kvx-iommu.txt => kvx-iommu.rst} (100%)
rename Documentation/kvx/{kvx-mmu.txt => kvx-mmu.rst} (88%)
rename Documentation/kvx/{kvx-smp.txt => kvx-smp.rst} (100%)
rename Documentation/kvx/{kvx.txt => kvx.rst} (100%)
diff --git a/Documentation/arch.rst b/Documentation/arch.rst
index 41a66a8b38e424..1ccda8ef6eefb3 100644
--- a/Documentation/arch.rst
+++ b/Documentation/arch.rst
@@ -13,6 +13,7 @@ implementation.
arm/index
arm64/index
ia64/index
+ kvx/index
loongarch/index
m68k/index
mips/index
diff --git a/Documentation/kvx/index.rst b/Documentation/kvx/index.rst
new file mode 100644
index 00000000000000..19a8e0865c82f3
--- /dev/null
+++ b/Documentation/kvx/index.rst
@@ -0,0 +1,12 @@
+================
+KVX Architecture
+================
+
+.. toctree::
+ :maxdepth: 1
+
+ kvx-exceptions
+ kvx-iommu
+ kvx-mmu
+ kvx-smp
+ kvx
diff --git a/Documentation/kvx/kvx-exceptions.txt b/Documentation/kvx/kvx-exceptions.rst
similarity index 98%
rename from Documentation/kvx/kvx-exceptions.txt
rename to Documentation/kvx/kvx-exceptions.rst
index 11368287bd4895..d3e52f30285223 100644
--- a/Documentation/kvx/kvx-exceptions.txt
+++ b/Documentation/kvx/kvx-exceptions.rst
@@ -12,19 +12,19 @@ The offset depends on which exception vector the cpu wants to jump to:
Then, handlers are laid in the following order:
- _____________
+ +-------------+
| |
| Syscall |
- |_____________|
+ +-------------+
| |
| Interrupts |
- |_____________|
+ +-------------+
| |
| Traps |
- |_____________|
+ +-------------+
| | ^
| Debug | | Stride
-BASE -> |_____________| v
+BASE -> +-------------+ v
Interrupts, and traps are serviced similarly, ie:
diff --git a/Documentation/kvx/kvx-iommu.txt b/Documentation/kvx/kvx-iommu.rst
similarity index 100%
rename from Documentation/kvx/kvx-iommu.txt
rename to Documentation/kvx/kvx-iommu.rst
diff --git a/Documentation/kvx/kvx-mmu.txt b/Documentation/kvx/kvx-mmu.rst
similarity index 88%
rename from Documentation/kvx/kvx-mmu.txt
rename to Documentation/kvx/kvx-mmu.rst
index a3ebbef3698129..59bda2afc9abde 100644
--- a/Documentation/kvx/kvx-mmu.txt
+++ b/Documentation/kvx/kvx-mmu.rst
@@ -24,17 +24,23 @@ only one ZONE_NORMAL. This will be updated if DMA cannot access all memory.
Currently, the memory mapping is the following for 4KB page:
-+-----------------------+-----------------------+------+-------+--------------+
-| Start | End | Attr | Size | Name |
-+-----------------------+-----------------------+------+-------+--------------+
-| 0000 0000 0000 0000 | 0000 003F FFFF FFFF | --- | 256GB | User |
-| 0000 0040 0000 0000 | 0000 007F FFFF FFFF | --- | 256GB | MMAP |
-| 0000 0080 0000 0000 | FFFF FF7F FFFF FFFF | --- | --- | Gap |
-| FFFF FF80 0000 0000 | FFFF FFFF FFFF FFFF | --- | 512GB | Kernel |
-| FFFF FF80 0000 0000 | FFFF FF8F FFFF FFFF | RWX | 64GB | Direct Map |
-| FFFF FF90 0000 0000 | FFFF FF90 3FFF FFFF | RWX | 1GB | Vmalloc |
-| FFFF FF90 4000 0000 | FFFF FFFF FFFF FFFF | RW | 447GB | Free area |
-+-----------------------+-----------------------+------+-------+--------------+
++-----------------------+-----------------------+---------+----------+--------------+
+| Start | End | Attr | Size | Name |
++=======================+=======================+=========+==========+==============+
+| 0000 0000 0000 0000 | 0000 003F FFFF FFFF | \-\-\- | 256GB | User |
++-----------------------+-----------------------+---------+----------+--------------+
+| 0000 0040 0000 0000 | 0000 007F FFFF FFFF | \-\-\- | 256GB | MMAP |
++-----------------------+-----------------------+---------+----------+--------------+
+| 0000 0080 0000 0000 | FFFF FF7F FFFF FFFF | \-\-\- | \-\-\- | Gap |
++-----------------------+-----------------------+---------+----------+--------------+
+| FFFF FF80 0000 0000 | FFFF FFFF FFFF FFFF | \-\-\- | 512GB | Kernel |
++-----------------------+-----------------------+---------+----------+--------------+
+| FFFF FF80 0000 0000 | FFFF FF8F FFFF FFFF | RWX | 64GB | Direct Map |
++-----------------------+-----------------------+---------+----------+--------------+
+| FFFF FF90 0000 0000 | FFFF FF90 3FFF FFFF | RWX | 1GB | Vmalloc |
++-----------------------+-----------------------+---------+----------+--------------+
+| FFFF FF90 4000 0000 | FFFF FFFF FFFF FFFF | RW | 447GB | Free area |
++-----------------------+-----------------------+---------+----------+--------------+
Enable the MMU
==============
diff --git a/Documentation/kvx/kvx-smp.txt b/Documentation/kvx/kvx-smp.rst
similarity index 100%
rename from Documentation/kvx/kvx-smp.txt
rename to Documentation/kvx/kvx-smp.rst
diff --git a/Documentation/kvx/kvx.txt b/Documentation/kvx/kvx.rst
similarity index 100%
rename from Documentation/kvx/kvx.txt
rename to Documentation/kvx/kvx.rst
--
An old man doll... just what I always wanted! - Clara
^ permalink raw reply related [flat|nested] 113+ messages in thread* [PATCH 2/8] Documentation: kvx: Wrap diagrams in literal code block
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 1/8] Documentation: kvx: Convert to reST Bagas Sanjaya
@ 2023-01-09 9:51 ` Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 3/8] Documentation: kvx: Fix lists Bagas Sanjaya
` (7 subsequent siblings)
9 siblings, 0 replies; 113+ messages in thread
From: Bagas Sanjaya @ 2023-01-09 9:51 UTC (permalink / raw)
To: Jonathan Corbet, Yann Sionneau
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin,
Bagas Sanjaya
Wrap code path diagrams in literal code block, just like other diagrams
in the kernel documentation. This avoids confusion with other constructs
(tables, block quotes, and inline substitutions).
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
---
Documentation/kvx/kvx-exceptions.rst | 241 ++++++++++++++-------------
Documentation/kvx/kvx-iommu.rst | 64 +++----
Documentation/kvx/kvx-mmu.rst | 41 +++--
Documentation/kvx/kvx.rst | 96 +++++------
4 files changed, 225 insertions(+), 217 deletions(-)
diff --git a/Documentation/kvx/kvx-exceptions.rst b/Documentation/kvx/kvx-exceptions.rst
index d3e52f30285223..bb9010efb14196 100644
--- a/Documentation/kvx/kvx-exceptions.rst
+++ b/Documentation/kvx/kvx-exceptions.rst
@@ -10,21 +10,21 @@ The offset depends on which exception vector the cpu wants to jump to:
* $ev + 0x80 for interrupt
* $ev + 0xc0 for syscall
-Then, handlers are laid in the following order:
+Then, handlers are laid in the following order::
- +-------------+
- | |
- | Syscall |
- +-------------+
- | |
- | Interrupts |
- +-------------+
- | |
- | Traps |
- +-------------+
- | | ^
- | Debug | | Stride
-BASE -> +-------------+ v
+ +-------------+
+ | |
+ | Syscall |
+ +-------------+
+ | |
+ | Interrupts |
+ +-------------+
+ | |
+ | Traps |
+ +-------------+
+ | | ^
+ | Debug | | Stride
+ BASE -> +-------------+ v
Interrupts, and traps are serviced similarly, ie:
@@ -99,65 +99,66 @@ When handling a signal, the path is the following:
7 - User application is restored at the exact point it was interrupted
before.
+ ::
- +----------+
- | 1 |
- | User app | @func
- | (user) |
- +---+------+
- |
- | it/trap/scall
- |
- +---v-------+
- | 2 |
- | exception |
- | handling |
- | (kernel) |
- +---+-------+
- |
- | Check if signal are pending, if so, handle signals
- |
- +---v--------+
- | 3 |
- | do_signal |
- | handling |
- | (kernel) |
- +----+-------+
- |
- | Return to user signal handler
- |
- +----v------+
- | 4 |
- | signal |
- | handler |
- | (user) |
- +----+------+
- |
- | Return to sigreturn trampoline
- |
- +----v-------+
- | 5 |
- | syscall |
- |rt_sigreturn|
- | (user) |
- +----+-------+
- |
- | Syscall to rt_sigreturn
- |
- +----v-------+
- | 6 |
- | sigreturn |
- | handler |
- | (kernel) |
- +----+-------+
- |
- | Modify context to return to original func
- |
- +----v-----+
- | 7 |
- | User app | @func
- | (user) |
- +----------+
+ +----------+
+ | 1 |
+ | User app | @func
+ | (user) |
+ +---+------+
+ |
+ | it/trap/scall
+ |
+ +---v-------+
+ | 2 |
+ | exception |
+ | handling |
+ | (kernel) |
+ +---+-------+
+ |
+ | Check if signal are pending, if so, handle signals
+ |
+ +---v--------+
+ | 3 |
+ | do_signal |
+ | handling |
+ | (kernel) |
+ +----+-------+
+ |
+ | Return to user signal handler
+ |
+ +----v------+
+ | 4 |
+ | signal |
+ | handler |
+ | (user) |
+ +----+------+
+ |
+ | Return to sigreturn trampoline
+ |
+ +----v-------+
+ | 5 |
+ | syscall |
+ |rt_sigreturn|
+ | (user) |
+ +----+-------+
+ |
+ | Syscall to rt_sigreturn
+ |
+ +----v-------+
+ | 6 |
+ | sigreturn |
+ | handler |
+ | (kernel) |
+ +----+-------+
+ |
+ | Modify context to return to original func
+ |
+ +----v-----+
+ | 7 |
+ | User app | @func
+ | (user) |
+ +----------+
Registers handling
==================
@@ -174,62 +175,62 @@ Interrupts and traps
When interrupt and traps are triggered, we only save the caller-saved registers.
Indeed, we rely on the fact that C code will save and restore callee-saved and
-hence, there is no need to save them. This path is the following:
+hence, there is no need to save them. This path is the following::
- +------------+ +-----------+ +---------------+
-IT | Save caller| C Call | Execute C | Ret | Restore caller| Ret from IT
-+--->+ saved +--------->+ handler +------->+ saved +----->
- | registers | +-----------+ | registers |
- +------------+ +---------------+
+ +------------+ +-----------+ +---------------+
+ IT | Save caller| C Call | Execute C | Ret | Restore caller| Ret from IT
+ +--->+ saved +--------->+ handler +------->+ saved +----->
+ | registers | +-----------+ | registers |
+ +------------+ +---------------+
However, when returning to user, we check if there is work_pending. If a signal
is pending and there is a signal handler to be called, then we need all
registers to be saved on the stack in the pt_regs before executing the signal
handler and restored after that. Since we only saved caller-saved registers, we
need to also save callee-saved registers to restore them correctly when
-returning to user. This path is the following (a bit more complicated !):
+returning to user. This path is the following (a bit more complicated !)::
- +------------+
- | Save caller| +-----------+ Ret +------------+
- IT | saved | C Call | Execute C | to asm | Check work |
- +--->+ registers +--------->+ handler +------->+ pending |
- | to pt_regs | +-----------+ +--+---+-----+
- +------------+ | |
- Work pending | | No work pending
- +--------------------------------------------+ |
- | |
- | +------------+
- v |
- +------+------+ v
- | Save callee | +-------+-------+
- | saved | | Restore caller| RFE from IT
- | registers | | saved +------->
- | to pt_regs | | registers |
- +--+-------+--+ | from pt_regs |
- | | +-------+-------+
- | | +---------+ ^
- | | | Execute | |
- | +-------->+ needed +-----------+
- | | work |
- | +---------+
- |Signal handler ?
- v
-+----+----------+ RFE to user +-------------+ +--------------+
-| Copy all | handler | Execute | ret | rt_sigreturn |
-| registers +------------>+ user signal +------>+ trampoline |
-| from pt_regs | | handler | | to kernel |
-| to user stack | +-------------+ +------+-------+
-+---------------+ |
- syscall rt_sigreturn |
- +-------------------------------------------------+
- |
- v
-+--------+-------+ +-------------+
-| Recopy all | | Restore all | RFE
-| registers from +--------------------->+ saved +------->
-| user stack | Return | registers |
-| to pt_regs | from sigreturn |from pt_regs |
-+----------------+ (via ret_from_fork) +-------------+
+ +------------+
+ | Save caller| +-----------+ Ret +------------+
+ IT | saved | C Call | Execute C | to asm | Check work |
+ +--->+ registers +--------->+ handler +------->+ pending |
+ | to pt_regs | +-----------+ +--+---+-----+
+ +------------+ | |
+ Work pending | | No work pending
+ +--------------------------------------------+ |
+ | |
+ | +------------+
+ v |
+ +------+------+ v
+ | Save callee | +-------+-------+
+ | saved | | Restore caller| RFE from IT
+ | registers | | saved +------->
+ | to pt_regs | | registers |
+ +--+-------+--+ | from pt_regs |
+ | | +-------+-------+
+ | | +---------+ ^
+ | | | Execute | |
+ | +-------->+ needed +-----------+
+ | | work |
+ | +---------+
+ |Signal handler ?
+ v
+ +----+----------+ RFE to user +-------------+ +--------------+
+ | Copy all | handler | Execute | ret | rt_sigreturn |
+ | registers +------------>+ user signal +------>+ trampoline |
+ | from pt_regs | | handler | | to kernel |
+ | to user stack | +-------------+ +------+-------+
+ +---------------+ |
+ syscall rt_sigreturn |
+ +-------------------------------------------------+
+ |
+ v
+ +--------+-------+ +-------------+
+ | Recopy all | | Restore all | RFE
+ | registers from +--------------------->+ saved +------->
+ | user stack | Return | registers |
+ | to pt_regs | from sigreturn |from pt_regs |
+ +----------------+ (via ret_from_fork) +-------------+
Syscalls
diff --git a/Documentation/kvx/kvx-iommu.rst b/Documentation/kvx/kvx-iommu.rst
index 96b74ce71acb3e..69eca8d1bc37a1 100644
--- a/Documentation/kvx/kvx-iommu.rst
+++ b/Documentation/kvx/kvx-iommu.rst
@@ -63,39 +63,39 @@ IOMMU implementation
--------------------
The kvx is providing several IOMMUs. Here is a simplified view of all IOMMUs
-and translations that occurs between memory and devices:
+and translations that occurs between memory and devices::
- +---------------------------------------------------------------------+
- | +------------+ +---------+ | CLUSTER X |
- | | Cores 0-15 +---->+ Crypto | +-----------|
- | +-----+------+ +----+----+ |
- | | | |
- | v v |
- | +-------+ +------------------------------+ |
- | | MMU | +----+ IOMMU x4 (secure + insecure) | |
- | +---+---+ | +------------------------------+ |
- | | | |
- +--------------------+ |
- | | | |
- v v | |
- +---+--------+-+ | |
- | MEMORY | | +----------+ +--------+ +-------+ |
- | +<-|-----+ IOMMU Rx |<----+ DMA Rx |<----+ | |
- | | | +----------+ +--------+ | | |
- | | | | NoC | |
- | | | +----------+ +--------+ | | |
- | +--|---->| IOMMU Tx +---->| DMA Tx +---->+ | |
- | | | +----------+ +--------+ +-------+ |
- | | +------------------------------------------------+
- | |
- | | +--------------+ +------+
- | |<--->+ IOMMU Rx/Tx +<--->+ PCIe +
- | | +--------------+ +------+
- | |
- | | +--------------+ +------------------------+
- | |<--->+ IOMMU Rx/Tx +<--->+ master Soc Peripherals |
- | | +--------------+ +------------------------+
- +--------------+
+ +---------------------------------------------------------------------+
+ | +------------+ +---------+ | CLUSTER X |
+ | | Cores 0-15 +---->+ Crypto | +-----------|
+ | +-----+------+ +----+----+ |
+ | | | |
+ | v v |
+ | +-------+ +------------------------------+ |
+ | | MMU | +----+ IOMMU x4 (secure + insecure) | |
+ | +---+---+ | +------------------------------+ |
+ | | | |
+ +--------------------+ |
+ | | | |
+ v v | |
+ +---+--------+-+ | |
+ | MEMORY | | +----------+ +--------+ +-------+ |
+ | +<-|-----+ IOMMU Rx |<----+ DMA Rx |<----+ | |
+ | | | +----------+ +--------+ | | |
+ | | | | NoC | |
+ | | | +----------+ +--------+ | | |
+ | +--|---->| IOMMU Tx +---->| DMA Tx +---->+ | |
+ | | | +----------+ +--------+ +-------+ |
+ | | +------------------------------------------------+
+ | |
+ | | +--------------+ +------+
+ | |<--->+ IOMMU Rx/Tx +<--->+ PCIe +
+ | | +--------------+ +------+
+ | |
+ | | +--------------+ +------------------------+
+ | |<--->+ IOMMU Rx/Tx +<--->+ master Soc Peripherals |
+ | | +--------------+ +------------------------+
+ +--------------+
There is also an IOMMU dedicated to the crypto module but this module will not
diff --git a/Documentation/kvx/kvx-mmu.rst b/Documentation/kvx/kvx-mmu.rst
index 59bda2afc9abde..832fb7c41a49d8 100644
--- a/Documentation/kvx/kvx-mmu.rst
+++ b/Documentation/kvx/kvx-mmu.rst
@@ -157,14 +157,17 @@ We only support three levels for the page table and 4KB for page size.
3 levels page table
-------------------
-...-----+--------+--------+--------+--------+--------+
- 40|39 32|31 24|23 16|15 8|7 0|
-...-----++-------+--+-----+---+----+----+---+--------+
- | | | |
- | | | +---> [11:0] Offset (12 bits)
- | | +-------------> [20:12] PTE offset (9 bits)
- | +-----------------------> [29:21] PMD offset (9 bits)
- +----------------------------------> [39:30] PGD offset (10 bits)
+::
+
+ ...-----+--------+--------+--------+--------+--------+
+ 40|39 32|31 24|23 16|15 8|7 0|
+ ...-----++-------+--+-----+---+----+----+---+--------+
+ | | | |
+ | | | +---> [11:0] Offset (12 bits)
+ | | +-------------> [20:12] PTE offset (9 bits)
+ | +-----------------------> [29:21] PMD offset (9 bits)
+ +----------------------------------> [39:30] PGD offset (10 bits)
+
Bits 40 to 64 are signed extended according to bit 39. If bit 39 is equal to 1
we are in kernel space.
@@ -175,12 +178,14 @@ PTE format
About the format of the PTE entry, as we are not forced by hardware for choices,
we choose to follow the format described in the RiscV implementation as a
-starting point.
+starting point::
+
+ +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+ | 63..23 | 22..13 | 12 | 11..10 | 9 | 8 | 7 | 6 | 5 | 4 | 3..2 | 1 | 0 |
+ +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+ PFN Unused S PageSZ H G X W R D CP A P
+
- +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
- | 63..23 | 22..13 | 12 | 11..10 | 9 | 8 | 7 | 6 | 5 | 4 | 3..2 | 1 | 0 |
- +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
- PFN Unused S PageSZ H G X W R D CP A P
where:
P: Present
A: Accessed
@@ -231,10 +236,12 @@ kvx implementation to use them is based on other architectures (such as arc
or xtensa) and uses a wrapping ASN counter containing both cycle/generation and
asn.
-+---------+--------+
-|63 10|9 0|
-+---------+--------+
- Cycle ASN
+::
+
+ +---------+--------+
+ |63 10|9 0|
+ +---------+--------+
+ Cycle ASN
This ASN counter is incremented monotonously to allocate new ASNs. When the
counter reaches 511 (9 bit), TLB is completely flushed and a new cycle is
diff --git a/Documentation/kvx/kvx.rst b/Documentation/kvx/kvx.rst
index 8ce0703de6813b..20c3666f445e26 100644
--- a/Documentation/kvx/kvx.rst
+++ b/Documentation/kvx/kvx.rst
@@ -15,17 +15,17 @@ On kvx, we have 4 levels of privilege level starting from 0 (most
privileged one) to 3 (less privilege one). A system of owners allows
to delegate ownership of resources by using specials system registers.
-The 2 main software stacks for Linux Kernel are the following:
+The 2 main software stacks for Linux Kernel are the following::
-+-------------+ +-------------+
-| PL0: Debug | | PL0: Debug |
-+-------------+ +-------------+
-| PL1: Linux | | PL1: HyperV |
-+-------------+ +-------------+
-| PL2: User | | PL2: Linux |
-+-------------+ +-------------+
-| | | PL3: User |
-+-------------+ +-------------+
+ +-------------+ +-------------+
+ | PL0: Debug | | PL0: Debug |
+ +-------------+ +-------------+
+ | PL1: Linux | | PL1: HyperV |
+ +-------------+ +-------------+
+ | PL2: User | | PL2: Linux |
+ +-------------+ +-------------+
+ | | | PL3: User |
+ +-------------+ +-------------+
In both cases, the kvx support for privileges has been designed using
only relative PL and thus should work on both configurations without
@@ -201,45 +201,45 @@ to it, the kernel sends an interrupt using a mailbox.
If the L2$ node is not present in the device tree, then, the RM will directly go
into sleeping.
-Boot diagram:
+Boot diagram::
- RM PE 0
- +
- +---------+ |
- | Boot | |
- +----+----+ |
- | |
- v |
- +-----+-----+ |
- | Prepare | |
- | L2 shared | |
- | memory | |
- |(registers)| |
- +-----+-----+ |
- | | +-----------+
- +------------------->+ Boot |
- | | +-----+-----+
- v | |
- +--------+---------+ | |
- | L2 firmware | | |
- | parameters: | | |
- | r0 = registers | | |
- | r1 = DTB | | |
- +--------+---------+ | |
- | | |
- v | |
- +-------+--------+ | +------+------+
- | L2 firmware | | | Wait for L2 |
- | execution | | | to be ready |
- +-------+--------+ | +------+------+
- | | |
- +------v-------+ | v
- | L2 requests | | +------+------+
-+--->+ handling | | | Enable |
-| +-------+------+ | | L2 caching |
-| | | +------+------+
-| | | |
-+------------+ + v
+ RM PE 0
+ +
+ +---------+ |
+ | Boot | |
+ +----+----+ |
+ | |
+ v |
+ +-----+-----+ |
+ | Prepare | |
+ | L2 shared | |
+ | memory | |
+ |(registers)| |
+ +-----+-----+ |
+ | | +-----------+
+ +------------------->+ Boot |
+ | | +-----+-----+
+ v | |
+ +--------+---------+ | |
+ | L2 firmware | | |
+ | parameters: | | |
+ | r0 = registers | | |
+ | r1 = DTB | | |
+ +--------+---------+ | |
+ | | |
+ v | |
+ +-------+--------+ | +------+------+
+ | L2 firmware | | | Wait for L2 |
+ | execution | | | to be ready |
+ +-------+--------+ | +------+------+
+ | | |
+ +------v-------+ | v
+ | L2 requests | | +------+------+
+ +--->+ handling | | | Enable |
+ | +-------+------+ | | L2 caching |
+ | | | +------+------+
+ | | | |
+ +------------+ + v
Since this driver is started early (before SMP boot), A lot of drivers are not
--
An old man doll... just what I always wanted! - Clara
^ permalink raw reply related [flat|nested] 113+ messages in thread* [PATCH 3/8] Documentation: kvx: Fix lists
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 1/8] Documentation: kvx: Convert to reST Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 2/8] Documentation: kvx: Wrap diagrams in literal code block Bagas Sanjaya
@ 2023-01-09 9:51 ` Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 4/8] Documentation: kvx: kvx-iommu: Use reST syntax for subsections Bagas Sanjaya
` (6 subsequent siblings)
9 siblings, 0 replies; 113+ messages in thread
From: Bagas Sanjaya @ 2023-01-09 9:51 UTC (permalink / raw)
To: Jonathan Corbet, Yann Sionneau
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin,
Bagas Sanjaya
Many "unexpected indentation" and block quote warnings are generated due
to errors in lists. Fix them up by:
* Align lists texts just after the lists marker
* Add required blank line between nested lists and between paragraphs
and the lists
* Use appropriate syntax for numbered lists
While at it, also lightly reword.
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
---
Documentation/kvx/kvx-exceptions.rst | 53 ++++++++++++++-----------
Documentation/kvx/kvx-iommu.rst | 3 +-
Documentation/kvx/kvx-mmu.rst | 37 +++++++++--------
Documentation/kvx/kvx.rst | 59 +++++++++++++++-------------
4 files changed, 85 insertions(+), 67 deletions(-)
diff --git a/Documentation/kvx/kvx-exceptions.rst b/Documentation/kvx/kvx-exceptions.rst
index bb9010efb14196..bd485efd2362c1 100644
--- a/Documentation/kvx/kvx-exceptions.rst
+++ b/Documentation/kvx/kvx-exceptions.rst
@@ -5,6 +5,7 @@ specifies a base address.
An offset is added to $ev upon exception and the result is used as
"Next $pc".
The offset depends on which exception vector the cpu wants to jump to:
+
* $ev + 0x00 for debug
* $ev + 0x40 for trap
* $ev + 0x80 for interrupt
@@ -28,6 +29,7 @@ Then, handlers are laid in the following order::
Interrupts, and traps are serviced similarly, ie:
+
- Jump to handler
- Save all registers
- Prepare the call (do_IRQ or trap_handler)
@@ -58,12 +60,15 @@ The following steps are then taken:
- Switch to kernel stack
- Extract syscall number
-- Check that the syscall number is not bogus
- - If so, set syscall func to a not implemented one
-- Check if tracing is enabled
- - If so, jump to trace_syscall_enter
+- Check that the syscall number is not bogus.
+ If so, set syscall func to a not implemented one
+
+- Check if tracing is enabled.
+ If so, jump to trace_syscall_enter, then:
+
- Save syscall arguments (r0 -> r7) on stack in pt_regs
- Call do_trace_syscall_enter function
+
- Restore syscall arguments since they have been modified by C call
- Call the syscall function
- Save $r0 in pt_regs since it can be cloberred afterward
@@ -80,24 +85,28 @@ Signals
Signals are handled when exiting kernel before returning to user.
When handling a signal, the path is the following:
-1 - User application is executing normally
- Then any exception happens (syscall, interrupt, trap)
-2 - The exception handling path is taken
- and before returning to user, pending signals are checked
-3 - Signal are handled by do_signal
- Registers are saved and a special part of the stack is modified
- to create a trampoline to call rt_sigreturn
- $spc is modified to jump to user signal handler
- $ra is modified to jump to sigreturn trampoline directly after
- returning from user signal handler.
-4 - User signal handler is called after rfe from exception
- when returning, $ra is retored to $pc, resulting in a call
- to the syscall trampoline.
-5 - syscall trampoline is executed, leading to rt_sigreturn syscall
-6 - rt_sigreturn syscall is executed
- Previous registers are restored to allow returning to user correctly
-7 - User application is restored at the exact point it was interrupted
- before.
+1. User application is executing normally, then exception occurs (syscall,
+ interrupt, trap)
+2. The exception handling path is taken
+ and before returning to user, pending signals are checked.
+
+3. The signal handling path is as follows:
+
+ * Signals are handled by do_signal.
+ * Registers are saved and a special part of the stack is modified
+ to create a trampoline to call rt_sigreturn.
+ * $spc is modified to jump to user signal handler.
+ * $ra is modified to jump to sigreturn trampoline directly after
+ returning from user signal handler.
+
+4. User signal handler is called after rfe from exception.
+ When returning, $ra is retored to $pc, resulting in a call
+ to the syscall trampoline.
+5. syscall trampoline is executed, leading to rt_sigreturn syscall
+6. rt_sigreturn syscall is executed.
+ Previous registers are restored to allow returning to user correctly
+7. User application is restored at the exact point it was interrupted
+ before.
::
diff --git a/Documentation/kvx/kvx-iommu.rst b/Documentation/kvx/kvx-iommu.rst
index 69eca8d1bc37a1..f7f61777eee21e 100644
--- a/Documentation/kvx/kvx-iommu.rst
+++ b/Documentation/kvx/kvx-iommu.rst
@@ -32,7 +32,8 @@ Cluster IOMMUs
--------------
IOMMUs on cluster are used for DMA and cryptographic accelerators.
-There are six IOMMUs connected to the:
+There are six IOMMUs connected:
+
- cluster DMA tx
- cluster DMA rx
- first non secure cryptographic accelerator
diff --git a/Documentation/kvx/kvx-mmu.rst b/Documentation/kvx/kvx-mmu.rst
index 832fb7c41a49d8..faa6bda2c39959 100644
--- a/Documentation/kvx/kvx-mmu.rst
+++ b/Documentation/kvx/kvx-mmu.rst
@@ -77,6 +77,7 @@ routine which would (obviously) not work... Once this is done, we can flush the
entries and that new entries inserted in JTLB will apply.
By default, the following policy is applied on vmlinux sections:
+
- init_data: RW
- init_text: RX (or RWX if parameter rodata=off)
- text: RX (or RWX if parameter rodata=off)
@@ -92,8 +93,9 @@ spaces to be in the same space as the user. The kernel will have the
$ps.mmup set in kernel (PL1) and unset for user (PL2).
As said in kvx documentation, we have two cases when the kernel is
booted:
-- Either we have been booted by someone (bootloader, hypervisor, etc)
-- Or we are alone (boot from flash)
+
+- Boot via intermediaries (bootloader, hypervisor, etc)
+- Direct boot from flash
In both cases, we will use the virtual space 0. Indeed, if we are alone
on the core, then it means nobody is using the MMU and we can take the
@@ -115,6 +117,7 @@ setup_bootmem: Memory : 0x100000000 - 0x120000000
setup_bootmem: Reserved: 0x10001f000 - 0x1002d1bc0
During the paging init we need to set:
+
- min_low_pfn that is the lowest PFN available in the system
- max_low_pfn that indicates the end if NORMAL zone
- max_pfn that is the number of pages in the system
@@ -213,16 +216,16 @@ kvx core does not feature a hardware page walker. This work must be done
by the core in software. In order to optimize TLB refill, a special fast
path is taken when entering in kernel space.
In order to speed up the process, the following actions are taken:
-# Save some registers in a per process scratchpad
-# If the trap is a nomapping then try the fastpath
-# Save some more registers for this fastpath
-# Check if faulting address is a memory direct mapping one.
- # If entry is a direct mapping one and RWX is not enabled, add an entry into LTLB
- # If not, continue
-# Try to walk the page table
- # If entry is not present, take the slowpath (do_page_fault)
-# Refill the tlb properly
-# Exit by restoring only a few registers
+
+1. Save some registers in a per process scratchpad
+2. If the trap is a nomapping then try the fastpath
+3. Save some more registers for this fastpath
+4. Check if faulting address is a memory direct mapping one. If entry is a
+ direct mapping one and RWX is not enabled, add an entry into LTLB.
+ Otherwise, continue
+5. Try to walk the page table. If entry is not present, take the slowpath (do_page_fault)
+6. Refill the tlb properly
+7. Exit by restoring only a few registers
ASN Handling
============
@@ -273,13 +276,15 @@ Debug
In order to debug the page table and tlb entries, gdb scripts contains commands
which allows to dump the page table:
+
- lx-kvx-page-table-walk
- - Display the current process page table by default
+ Display the current process page table by default
- lx-kvx-tlb-decode
- - Display the content of $tel and $teh into something readable
+ Display the content of $tel and $teh into something readable
Other commands available in kvx-gdb are the following:
+
- mppa-dump-tlb
- - Display the content of TLBs (JTLB and LTLB)
+ Display the content of TLBs (JTLB and LTLB)
- mppa-lookup-addr
- - Find physical address matching a virtual one
+ Find physical address matching a virtual one
diff --git a/Documentation/kvx/kvx.rst b/Documentation/kvx/kvx.rst
index 20c3666f445e26..8982d10f2678df 100644
--- a/Documentation/kvx/kvx.rst
+++ b/Documentation/kvx/kvx.rst
@@ -74,6 +74,7 @@ When entering the C (kvx_lowlevel_start) the kernel will look for a special
magic in $r0 (0x494C314B). This magic tells the kernel if there is arguments
passed by a bootloader.
Currently, the following values are passed through registers:
+
- r1: pointer to command line setup by bootloader
- r2: device tree
@@ -105,11 +106,13 @@ of the LTLB.
The first entry will map the first 1G of virtual address space to the first
1G of DDR:
+
- TLB[0]: 0xffffff0000000000 -> 0x100000000 (size 512Mo)
The second entry will be a flat mapping of the first 512 Ko of the SMEM. It
is required to have this flat mapping because there is still code located at
this address that needs to be executed:
+
- TLB[1]: 0x0 -> 0x0 (size 512Ko)
Once virtual space reached the second entry is removed.
@@ -151,19 +154,19 @@ r20 and r21 to special values containing the function to call.
The normal path for a kernel thread will be the following:
- 1 - Enter copy_thread_tls and setup callee saved registers which will
- be restored in __switch_to.
- 2 - set r20 and r21 (in thread_struct) to function and argument and
- ra to ret_from_kernel_thread.
- These callee saved will be restored in switch_to.
- 3 - Call _switch_to at some point.
- 4 - Save all callee saved register since switch_to is seen as a
- standard function call by the caller.
- 5 - Change stack pointer to the new stack
- 6 - At the end of switch to, set sr0 to the new task and use ret to
- jump to ret_from_kernel_thread (address restored from ra).
- 7 - In ret_from_kernel_thread, execute the function with arguments by
- using r20, r21 and we are done
+ 1. Enter copy_thread_tls and setup callee saved registers which will
+ be restored in __switch_to.
+ 2. set r20 and r21 (in thread_struct) to function and argument and
+ ra to ret_from_kernel_thread.
+ These callee saved will be restored in switch_to.
+ 3. Call _switch_to at some point.
+ 4. Save all callee saved register since switch_to is seen as a
+ standard function call by the caller.
+ 5. Change stack pointer to the new stack
+ 6. At the end of switch to, set sr0 to the new task and use ret to
+ jump to ret_from_kernel_thread (address restored from ra).
+ 7. In ret_from_kernel_thread, execute the function with arguments by
+ using r20, r21 and we are done
For more explanation, you can refer to https://lwn.net/Articles/520227/
@@ -173,21 +176,21 @@ User thread creation
We are using almost the same path as copy_thread to create it.
The detailed path is the following:
- 1 - Call start_thread which will setup user pc and stack pointer in
- task regs. We also set sps and clear privilege mode bit.
- When returning from exception, it will "flip" to user mode.
- 2 - Enter copy_thread_tls and setup callee saved registers which will
- be restored in __switch_to. Also, set the "return" function to be
- ret_from_fork which will be called at end of switch_to
- 3 - set r20 (in thread_struct) with tracing information.
- (simply by lazyness to avoid computing it in assembly...)
- 4 - Call _switch_to at some point.
- 5 - The current pc will then be restored to be ret_from fork.
- 6 - Ret from fork calls schedule_tail and then check if tracing is
- enabled. If so call syscall_trace_exit
- 7 - finally, instead of returning to kernel, we restore all registers
- that have been setup by start_thread by restoring regs stored on
- stack
+ 1. Call start_thread which will setup user pc and stack pointer in
+ task regs. We also set sps and clear privilege mode bit.
+ When returning from exception, it will "flip" to user mode.
+ 2. Enter copy_thread_tls and setup callee saved registers which will
+ be restored in __switch_to. Also, set the "return" function to be
+ ret_from_fork which will be called at end of switch_to
+ 3. set r20 (in thread_struct) with tracing information.
+ (simply by lazyness to avoid computing it in assembly...)
+ 4. Call _switch_to at some point.
+ 5. The current pc will then be restored to be ret_from fork.
+ 6. Ret from fork calls schedule_tail and then check if tracing is
+ enabled. If so call syscall_trace_exit
+ 7. Finally, instead of returning to kernel, we restore all registers
+ that have been setup by start_thread by restoring regs stored on
+ stack
L2$ handling
============
--
An old man doll... just what I always wanted! - Clara
^ permalink raw reply related [flat|nested] 113+ messages in thread* [PATCH 4/8] Documentation: kvx: kvx-iommu: Use reST syntax for subsections
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
` (2 preceding siblings ...)
2023-01-09 9:51 ` [PATCH 3/8] Documentation: kvx: Fix lists Bagas Sanjaya
@ 2023-01-09 9:51 ` Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 5/8] Documentation: kvx: kvx-iommu: monospacize kvx iommu device tree path Bagas Sanjaya
` (5 subsequent siblings)
9 siblings, 0 replies; 113+ messages in thread
From: Bagas Sanjaya @ 2023-01-09 9:51 UTC (permalink / raw)
To: Jonathan Corbet, Yann Sionneau
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin,
Bagas Sanjaya
Subsection headings of "IOMMU implementation" is written using
triple-hash syntax, which is valid only for Markdown. Since the
documentation is written in reST, use appropriate syntax instead
(tilde underline).
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
---
Documentation/kvx/kvx-iommu.rst | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/Documentation/kvx/kvx-iommu.rst b/Documentation/kvx/kvx-iommu.rst
index f7f61777eee21e..5ed34463b8bb1f 100644
--- a/Documentation/kvx/kvx-iommu.rst
+++ b/Documentation/kvx/kvx-iommu.rst
@@ -110,7 +110,8 @@ the cluster.
IOMMU is related to a specific bus like PCIe we will be able to specify that
all peripherals will go through this IOMMU.
-### IOMMU Page table
+IOMMU Page table
+~~~~~~~~~~~~~~~~
We need to be able to know which IO virtual addresses (IOVA) are mapped in the
TLB in order to be able to remove entries when a device finishes a transfer and
@@ -137,7 +138,8 @@ huge page table for a given IOMMU (typically the PCIe IOMMU).
As we won't refill the TLB we know that we won't have more than 128*16 entries.
In this case we can simply keep a table with all possible entries.
-### Maintenance interface
+Maintenance interface
+~~~~~~~~~~~~~~~~~~~~~
It is possible to have several "maintainers" for the same IOMMU. The driver is
using two of them. One that writes the TLB and another interface reads TLB. For
@@ -149,7 +151,8 @@ following command in gdb:
Since different management interface are used for read and write it is safe to
execute the above command at any moment.
-### Interrupts
+Interrupts
+~~~~~~~~~~
IOMMU can have 3 kind of interrupts that corresponds to 3 different types of
errors (no mapping. protection, parity). When the IOMMU is shared between
@@ -163,7 +166,8 @@ stall one. So when an interrupt occurs it is managed by the driver. All others
interrupts that occurs are stored and the IOMMU is stalled. When driver cleans
the first interrupt others will be managed one by one.
-### ASN (Address Space Number)
+ASN (Address Space Number)
+~~~~~~~~~~~~~~~~~~~~~~~~~~
This is also know as ASID in some other architecture. Each device will have a
given ASN that will be given through the device tree. As address space is
--
An old man doll... just what I always wanted! - Clara
^ permalink raw reply related [flat|nested] 113+ messages in thread* [PATCH 5/8] Documentation: kvx: kvx-iommu: monospacize kvx iommu device tree path
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
` (3 preceding siblings ...)
2023-01-09 9:51 ` [PATCH 4/8] Documentation: kvx: kvx-iommu: Use reST syntax for subsections Bagas Sanjaya
@ 2023-01-09 9:51 ` Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 6/8] Documentation: kvx: Promote title headings Bagas Sanjaya
` (4 subsequent siblings)
9 siblings, 0 replies; 113+ messages in thread
From: Bagas Sanjaya @ 2023-01-09 9:51 UTC (permalink / raw)
To: Jonathan Corbet, Yann Sionneau
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin,
Bagas Sanjaya
The device tree path shares the same "kvm-iommu" as the actual
documentation path (kvm-iommu.rst), which make it links to the doc
instead. Monospacize the path by enclosing it in inline code.
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
---
Documentation/kvx/kvx-iommu.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/kvx/kvx-iommu.rst b/Documentation/kvx/kvx-iommu.rst
index 5ed34463b8bb1f..c97035ba624c22 100644
--- a/Documentation/kvx/kvx-iommu.rst
+++ b/Documentation/kvx/kvx-iommu.rst
@@ -178,7 +178,7 @@ Device tree
-----------
Relationships between devices, DMAs and IOMMUs are described in the
-device tree (see Documentation/devicetree/bindings/iommu/kalray,kvx-iommu.txt
+device tree (see ``Documentation/devicetree/bindings/iommu/kalray,kvx-iommu.txt``
for more details).
Limitations
--
An old man doll... just what I always wanted! - Clara
^ permalink raw reply related [flat|nested] 113+ messages in thread* [PATCH 6/8] Documentation: kvx: Promote title headings
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
` (4 preceding siblings ...)
2023-01-09 9:51 ` [PATCH 5/8] Documentation: kvx: kvx-iommu: monospacize kvx iommu device tree path Bagas Sanjaya
@ 2023-01-09 9:51 ` Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 7/8] Documentation: kvx: Use literal code block for command-line inputs Bagas Sanjaya
` (3 subsequent siblings)
9 siblings, 0 replies; 113+ messages in thread
From: Bagas Sanjaya @ 2023-01-09 9:51 UTC (permalink / raw)
To: Jonathan Corbet, Yann Sionneau
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin,
Bagas Sanjaya
The documentation contains headings which all marked up using the same
heading level, which cause doc sections to also appear in index TOC
alongside doc titles.
Promote title headings while expanding their title text to be more
descriptive.
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
---
Documentation/kvx/kvx-exceptions.rst | 6 ++++--
Documentation/kvx/kvx-iommu.rst | 4 ++--
Documentation/kvx/kvx-mmu.rst | 5 +++--
Documentation/kvx/kvx-smp.rst | 5 +++--
Documentation/kvx/kvx.rst | 1 +
5 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/Documentation/kvx/kvx-exceptions.rst b/Documentation/kvx/kvx-exceptions.rst
index bd485efd2362c1..15692f14b9219d 100644
--- a/Documentation/kvx/kvx-exceptions.rst
+++ b/Documentation/kvx/kvx-exceptions.rst
@@ -1,5 +1,7 @@
-Exceptions
-==========
+=========================
+Exception handling in kvx
+=========================
+
On kvx, handlers are set using $ev (exception vector) register which
specifies a base address.
An offset is added to $ev upon exception and the result is used as
diff --git a/Documentation/kvx/kvx-iommu.rst b/Documentation/kvx/kvx-iommu.rst
index c97035ba624c22..5e84dc12adb437 100644
--- a/Documentation/kvx/kvx-iommu.rst
+++ b/Documentation/kvx/kvx-iommu.rst
@@ -1,5 +1,5 @@
-IOMMU
-=====
+IOMMU in kvx
+============
General Overview
----------------
diff --git a/Documentation/kvx/kvx-mmu.rst b/Documentation/kvx/kvx-mmu.rst
index faa6bda2c39959..05b9bc111e02db 100644
--- a/Documentation/kvx/kvx-mmu.rst
+++ b/Documentation/kvx/kvx-mmu.rst
@@ -1,5 +1,6 @@
-MMU
-===
+==========================
+kvx Memory Management Unit
+==========================
Virtual addresses are on 41 bits for kvx when using 64-bit mode.
To differentiate kernel from user space, we use the high order bit
diff --git a/Documentation/kvx/kvx-smp.rst b/Documentation/kvx/kvx-smp.rst
index 1b69d77db8cdaf..f170bc48ea5f7f 100644
--- a/Documentation/kvx/kvx-smp.rst
+++ b/Documentation/kvx/kvx-smp.rst
@@ -1,5 +1,6 @@
-SMP
-===
+===============================================
+Symmetric Multiprocessing Implementation in kvx
+===============================================
On kvx, 5 clusters are organized as groups of 16 processors + 1
secure core (RM) for each cluster. These 17 processors are L1$ coherent
diff --git a/Documentation/kvx/kvx.rst b/Documentation/kvx/kvx.rst
index 8982d10f2678df..4aac7fce4e3d01 100644
--- a/Documentation/kvx/kvx.rst
+++ b/Documentation/kvx/kvx.rst
@@ -1,3 +1,4 @@
+=======================
kvx Core Implementation
=======================
--
An old man doll... just what I always wanted! - Clara
^ permalink raw reply related [flat|nested] 113+ messages in thread* [PATCH 7/8] Documentation: kvx: Use literal code block for command-line inputs
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
` (5 preceding siblings ...)
2023-01-09 9:51 ` [PATCH 6/8] Documentation: kvx: Promote title headings Bagas Sanjaya
@ 2023-01-09 9:51 ` Bagas Sanjaya
2023-01-09 9:51 ` [PATCH 8/8] Documentation: kvx: reword Bagas Sanjaya
` (2 subsequent siblings)
9 siblings, 0 replies; 113+ messages in thread
From: Bagas Sanjaya @ 2023-01-09 9:51 UTC (permalink / raw)
To: Jonathan Corbet, Yann Sionneau
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin,
Bagas Sanjaya
Command-line, like codes, looks ugly when default paragraph block is
used (since proportional font is used), where literal code blocks (which
render as monospace) do the job better.
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
---
Documentation/kvx/kvx-iommu.rst | 2 +-
Documentation/kvx/kvx.rst | 10 +++++-----
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/Documentation/kvx/kvx-iommu.rst b/Documentation/kvx/kvx-iommu.rst
index 5e84dc12adb437..c95d9231d5b665 100644
--- a/Documentation/kvx/kvx-iommu.rst
+++ b/Documentation/kvx/kvx-iommu.rst
@@ -144,7 +144,7 @@ Maintenance interface
It is possible to have several "maintainers" for the same IOMMU. The driver is
using two of them. One that writes the TLB and another interface reads TLB. For
debug purpose it is possible to display the content of the tlb by using the
-following command in gdb:
+following command in gdb::
gdb> p kvx_iommu_dump_tlb( <iommu addr>, 0)
diff --git a/Documentation/kvx/kvx.rst b/Documentation/kvx/kvx.rst
index 4aac7fce4e3d01..5385e1e3d30187 100644
--- a/Documentation/kvx/kvx.rst
+++ b/Documentation/kvx/kvx.rst
@@ -122,7 +122,7 @@ To be able to set breakpoints when MMU is enabled we added a label called
gdb_mmu_enabled. If you try to set a breakpoint on a function that is in
virtual memory before the activation of the MMU this address as no signification
for GDB. So, for example, if you want to break on the function start_kernel()
-you will need to run:
+you will need to run::
kvx-gdb -silent path_to/vmlinux \
-ex 'tbreak gdb_mmu_enabled' -ex 'run' \
@@ -253,19 +253,19 @@ Building
========
In order to build the kernel, you will need a complete kvx toolchain.
-First, setup the config using the following command line
+First, setup the config using the following command line::
$ make ARCH=kvx O=your_directory default_defconfig
-Adjust any configuration option you may need and then, build the kernel:
+Adjust any configuration option you may need and then, build the kernel::
$ make ARCH=kvx O=your_directory -j12
-You will finally have a vmlinux image ready to be run.
+You will finally have a vmlinux image ready to be run::
$ kvx-mppa -- vmlinux
-Additionally, you may want to debug it. To do so, use kvx-gdb:
+Additionally, you may want to debug it. To do so, use kvx-gdb::
$ kvx-gdb vmlinux
--
An old man doll... just what I always wanted! - Clara
^ permalink raw reply related [flat|nested] 113+ messages in thread* [PATCH 8/8] Documentation: kvx: reword
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
` (6 preceding siblings ...)
2023-01-09 9:51 ` [PATCH 7/8] Documentation: kvx: Use literal code block for command-line inputs Bagas Sanjaya
@ 2023-01-09 9:51 ` Bagas Sanjaya
2023-01-09 10:59 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Jules Maselbas
2023-01-10 0:18 ` Randy Dunlap
9 siblings, 0 replies; 113+ messages in thread
From: Bagas Sanjaya @ 2023-01-09 9:51 UTC (permalink / raw)
To: Jonathan Corbet, Yann Sionneau
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin,
Bagas Sanjaya
Improve the documentation wording to be clearer and effective.
In most cases, third-person perspective ("we") is avoided unless
absolutely necessary.
Also, monospacize programming keywords (like variable and function names).
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
---
Documentation/kvx/kvx-exceptions.rst | 114 +++++++-------
Documentation/kvx/kvx-iommu.rst | 124 +++++++--------
Documentation/kvx/kvx-mmu.rst | 227 ++++++++++++++-------------
Documentation/kvx/kvx-smp.rst | 29 ++--
Documentation/kvx/kvx.rst | 209 ++++++++++++------------
5 files changed, 351 insertions(+), 352 deletions(-)
diff --git a/Documentation/kvx/kvx-exceptions.rst b/Documentation/kvx/kvx-exceptions.rst
index 15692f14b9219d..2ce7a62174a40a 100644
--- a/Documentation/kvx/kvx-exceptions.rst
+++ b/Documentation/kvx/kvx-exceptions.rst
@@ -2,11 +2,10 @@
Exception handling in kvx
=========================
-On kvx, handlers are set using $ev (exception vector) register which
-specifies a base address.
-An offset is added to $ev upon exception and the result is used as
-"Next $pc".
-The offset depends on which exception vector the cpu wants to jump to:
+On kvx, handlers are set using $ev (exception vector) register which specifies
+a base address. An offset is added to $ev upon exception and the result is used
+as "Next $pc". The offset depends on which exception vector the cpu wants to
+jump to:
* $ev + 0x00 for debug
* $ev + 0x40 for trap
@@ -30,53 +29,52 @@ Then, handlers are laid in the following order::
BASE -> +-------------+ v
-Interrupts, and traps are serviced similarly, ie:
+Interrupts and traps are serviced similarly, ie:
- Jump to handler
- Save all registers
- Prepare the call (do_IRQ or trap_handler)
-- restore all registers
-- return from exception
+- Restore all registers
+- Return from exception
-entry.S file is (as for other architectures) the entry point into the kernel.
+``entry.S`` is (as for other architectures) the entry point into the kernel.
It contains all assembly routines related to interrupts/traps/syscall.
Syscall handling
================
When executing a syscall, it must be done using "scall $r6"
-where $r6 contains the syscall number. Using this convention allow to
+where $r6 contains the syscall number. This convention allows to
modify and restart a syscall from the kernel.
Syscalls are handled differently than interrupts/exceptions. From an ABI
-point of view, scalls are like function calls: any caller saved register
+point of view, syscalls are like function calls: any caller-saved register
can be clobbered by the syscall. However, syscall parameters are passed
using registers r0 through r7. These registers must be preserved to avoid
-cloberring them before the actual syscall function.
+clobbering them before the actual syscall function.
-On syscall from userspace (scall instruction), the processor will put
+On syscall from userspace (``scall`` instruction), the processor will put
the syscall number in $es.sn and switch from user to kernel privilege
-mode. kvx_syscall_handler will be called in kernel mode.
+mode. ``kvx_syscall_handler`` will then be called in kernel mode.
-The following steps are then taken:
+Below is the path when executing syscall:
-- Switch to kernel stack
-- Extract syscall number
-- Check that the syscall number is not bogus.
- If so, set syscall func to a not implemented one
+- Switch to kernel stack.
+- Extract syscall number.
+- Check that the syscall number is not bogus. If so, set syscall func to the
+ unimplemented one.
-- Check if tracing is enabled.
- If so, jump to trace_syscall_enter, then:
+- Check if tracing is enabled. If so, jump to ``trace_syscall_enter``, then:
- - Save syscall arguments (r0 -> r7) on stack in pt_regs
- - Call do_trace_syscall_enter function
+ - Save syscall arguments (r0 -> r7) on stack in pt_regs.
+ - Call ``do_trace_syscall_enter`` function.
-- Restore syscall arguments since they have been modified by C call
-- Call the syscall function
-- Save $r0 in pt_regs since it can be cloberred afterward
-- If tracing was enabled, call trace_syscall_exit
-- Call work_pending
-- Return to user !
+- Restore syscall arguments since they have been modified by C function call.
+- Call the ``syscall`` function.
+- Save $r0 in ``pt_regs`` since it can be clobbered afterward.
+- If tracing is enabled, call ``trace_syscall_exit``.
+- Call ``work_pending``.
+- Return to user
The trace call is handled out of the fast path. All slow path handling
is done in another part of code to avoid messing with the cache.
@@ -85,18 +83,18 @@ Signals
=======
Signals are handled when exiting kernel before returning to user.
-When handling a signal, the path is the following:
+When handling a signal, the execution path is:
1. User application is executing normally, then exception occurs (syscall,
interrupt, trap)
-2. The exception handling path is taken
- and before returning to user, pending signals are checked.
+2. The exception handling path is taken and before returning to user, pending
+ signals are checked.
3. The signal handling path is as follows:
- * Signals are handled by do_signal.
+ * Signals are handled by ``do_signal``.
* Registers are saved and a special part of the stack is modified
- to create a trampoline to call rt_sigreturn.
+ to create a trampoline to call ``rt_sigreturn``.
* $spc is modified to jump to user signal handler.
* $ra is modified to jump to sigreturn trampoline directly after
returning from user signal handler.
@@ -104,9 +102,9 @@ When handling a signal, the path is the following:
4. User signal handler is called after rfe from exception.
When returning, $ra is retored to $pc, resulting in a call
to the syscall trampoline.
-5. syscall trampoline is executed, leading to rt_sigreturn syscall
-6. rt_sigreturn syscall is executed.
- Previous registers are restored to allow returning to user correctly
+5. syscall trampoline is executed, leading to ``rt_sigreturn`` syscall
+6. ``rt_sigreturn`` syscall is executed. Previous registers are restored to
+ allow returning to user correctly
7. User application is restored at the exact point it was interrupted
before.
@@ -175,18 +173,18 @@ Registers handling
==================
MMU is disabled in all exceptions paths, during register save and restoration.
-This will prevent from triggering MMU fault (such as TLB miss) which could
+This will prevent triggering MMU fault (such as TLB miss) which could
clobber the current register state. Such event can occurs when RWX mode is
enabled and the memory accessed to save register can trigger a TLB miss.
Aside from that which is common for all exceptions path, registers are saved
-differently regarding the type of exception.
+differently according to exception type.
Interrupts and traps
--------------------
-When interrupt and traps are triggered, we only save the caller-saved registers.
+When interrupt and traps are triggered, only caller-saved registers are saved.
Indeed, we rely on the fact that C code will save and restore callee-saved and
-hence, there is no need to save them. This path is the following::
+hence, there is no need to save them. The path is::
+------------+ +-----------+ +---------------+
IT | Save caller| C Call | Execute C | Ret | Restore caller| Ret from IT
@@ -194,12 +192,13 @@ hence, there is no need to save them. This path is the following::
| registers | +-----------+ | registers |
+------------+ +---------------+
-However, when returning to user, we check if there is work_pending. If a signal
-is pending and there is a signal handler to be called, then we need all
-registers to be saved on the stack in the pt_regs before executing the signal
-handler and restored after that. Since we only saved caller-saved registers, we
-need to also save callee-saved registers to restore them correctly when
-returning to user. This path is the following (a bit more complicated !)::
+However, when returning to user, we check if there is ``work_pending``. If a
+signal is pending and there is a signal handler to be called, then all
+registers are needed to be saved on the stack in ``pt_regs`` before executing
+the signal handler and restored after that. Since only caller-saved registers
+are saved, we need to also save callee-saved registers to restore them
+correctly when returning to user. The path will be (note: a bit more
+complicated!)::
+------------+
| Save caller| +-----------+ Ret +------------+
@@ -246,13 +245,14 @@ returning to user. This path is the following (a bit more complicated !)::
Syscalls
--------
-As explained before, for syscalls, we can use whatever callee-saved registers
-we want since syscall are seen as a "classic" call from ABI pov.
-Only different path is the one for clone. For this path, since the child expects
-to find same callee-registers content than his parent, we must save them before
-executing the clone syscall and restore them after that for the child. This is
-done via a redefinition of __sys_clone in assembly which will be called in place
-of the standard sys_clone. This new call will save callee saved registers
-in pt_regs. Parent will return using the syscall standard path. Freshly spawned
-child however will be woken up via ret_from_fork which will restore all
-registers (even if caller saved are not needed).
+As explained before, for syscalls, any arbitrary callee-saved registers can
+besince syscall are seen as a "classic" call from ABI pov. The only different
+path is the one for :manpage:`clone(2)`. For this path, since the child expects
+to find same callee-registers content than its parent, they must be saved
+before executing the :manpage:`clone(2)` syscall and restore them after that
+for the child. This is done via a redefinition of ``__sys_clone`` in assembly
+which will be called in place of the standard ``sys_clone``. This new call will
+save callee-saved registers in ``pt_regs``. Parent will return using the
+syscall standard path. Freshly spawned child however will be woken up via
+``ret_from_fork`` which will restore all registers (even if caller-saved
+registers are not needed).
diff --git a/Documentation/kvx/kvx-iommu.rst b/Documentation/kvx/kvx-iommu.rst
index c95d9231d5b665..dc642ff20d8f67 100644
--- a/Documentation/kvx/kvx-iommu.rst
+++ b/Documentation/kvx/kvx-iommu.rst
@@ -4,28 +4,28 @@ IOMMU in kvx
General Overview
----------------
-To exchange data between device and users through memory, the driver has to
-set up a buffer by doing some kernel allocation. The address of the buffer is
-virtual and the physical one is obtained through the MMU. When the device wants
-to access the same physical memory space it uses a bus address. This address is
-obtained by using the DMA mapping API. The Coolidge SoC includes several IOMMUs for clusters,
-PCIe peripherals, SoC peripherals, and more; that will translate this "bus address"
-into a physical one during DMA operations.
+To exchange data between device and users through memory, the driver has to set
+up a buffer by doing some kernel memory allocation. The address of the buffer
+is virtual and the physical one is obtained through the MMU. When the device
+wants to access the same physical memory space it uses a bus address, which is
+obtained by using the DMA mapping API. The Coolidge SoC includes several IOMMUs
+for clusters, PCIe peripherals, SoC peripherals, and more; that will translate
+this "bus address" into a physical one during DMA operations.
The bus addresses are IOVA (I/O Virtual Address) or DMA addresses. This
addresses can be obtained by calling the allocation functions of the DMA APIs.
-It can also be obtained through classical kernel allocation of physical
+It can also be obtained through classical allocation of physical
contiguous memory and then calling mapping functions of the DMA API.
-In order to be able to use the kvx IOMMU we have implemented the IOMMU DMA
-interface in arch/kvx/mm/dma-mapping.c. DMA functions are registered by
-implementing arch_setup_dma_ops() and generic IOMMU functions. Generic IOMMU
-are calling our specific IOMMU functions that adding or removing mappings
+In order to be able to use the kvx IOMMU the necessary IOMMU DMA interface is
+implemented in ``arch/kvx/mm/dma-mapping.c``. DMA functions are registered by
+implementing ``arch_setup_dma_ops()`` and generic IOMMU functions. Generic
+IOMMU are calling our specific IOMMU functions that adding or removing mappings
between DMA addresses and physical addresses in the IOMMU TLB.
-Specifics IOMMU functions are defined in the kvx IOMMU driver. A kvx IOMMU
-driver is managing two physical hardware IOMMU used for TX and RX. In the next
-section we described the HW IOMMUs.
+Specifics IOMMU functions are defined in the kvx IOMMU driver. It manages two
+physical hardware IOMMU used for TX and RX. In the next section we described
+the HW IOMMUs.
Cluster IOMMUs
@@ -45,9 +45,9 @@ SoC peripherals IOMMUs
----------------------
Since SoC peripherals are connected to an AXI bus, two IOMMUs are used: one for
-each AXI channel (read and write). These two IOMMUs are shared between all master
-devices and DMA. These two IOMMUs will have the same entries but need to be configured
-independently.
+each AXI channel (read and write). These two IOMMUs are shared between all
+master devices and DMA. These two IOMMUs will have the same entries but need to
+be configured independently.
PCIe IOMMUs
-----------
@@ -56,8 +56,8 @@ There is a slave IOMMU (read and write from the MPPA to the PCIe endpoint)
and a master IOMMU (read and write from a PCIe endpoint to system DDR).
The PCIe root complex and the MSI/MSI-X controller have been designed to use
the IOMMU feature when enabled. (For example for supporting endpoint that
-support only 32 bits addresses and allow them to access any memory in a
-64 bits address space). For security reason it is highly recommended to
+support only 32-bit addresses and allow them to access any memory in the
+64-bit address space). For security reason it is highly recommended to
activate the IOMMU for PCIe.
IOMMU implementation
@@ -99,13 +99,12 @@ and translations that occurs between memory and devices::
+--------------+
-There is also an IOMMU dedicated to the crypto module but this module will not
+There is also an IOMMU dedicated to the crypto module but the module will not
be accessed by the operating system.
-We will provide one driver to manage IOMMUs RX/TX. All of them will be
-described in the device tree to be able to get their particularities. See
-the example below that describes the relation between IOMMU, DMA and NoC in
-the cluster.
+The kernel provides one driver to manage IOMMUs RX/TX. All of them are
+described in the device tree in detail. See the example below that describes
+the relation between IOMMU, DMA and NoC in the cluster.
IOMMU is related to a specific bus like PCIe we will be able to specify that
all peripherals will go through this IOMMU.
@@ -113,38 +112,38 @@ all peripherals will go through this IOMMU.
IOMMU Page table
~~~~~~~~~~~~~~~~
-We need to be able to know which IO virtual addresses (IOVA) are mapped in the
-TLB in order to be able to remove entries when a device finishes a transfer and
-release memory. This information could be extracted when needed by computing all
-sets used by the memory and then reads all sixteen ways and compare them to the
-IOVA but it won't be efficient. We also need to be able to translate an IOVA
-to a physical address as required by the iova_to_phys IOMMU ops that is used
-by DMA. Like previously it can be done by extracting the set from the address
-and comparing the IOVA to each sixteen entries of the given set.
+It is necessary to know which IO virtual addresses (IOVA) are mapped in the TLB
+in order to be able to remove entries when a device finishes a transfer and
+release memory. This information could be extracted when needed by computing
+all sets used by the memory and then reads all 16 entries and compare them to
+the IOVA but it won't be efficient. It is also necessary to translate an IOVA
+to a physical address as required by the ``iova_to_phys`` IOMMU ops that is
+used by DMA. Again, it can be done by extracting the set from the address and
+comparing the IOVA to each sixteen entries of the given set.
-A solution is to keep a page table for the IOMMU. But this method is not
-efficient for reloading an entry of the TLB without the help of an hardware
-page table. So to prevent the need of a refill we will update the TLB when a
-device request access to memory and if there is no more slot available in the
-TLB we will just fail and the device will have to try again later. It is not
-efficient but at least we won't need to manage the refill of the TLB.
+A possible solution is to keep a page table for the IOMMU. However, this method
+is not efficient for reloading an entry of the TLB without the help of an
+hardware page table. So to prevent the need to refill the TLB is updated when
+a device requests access to memory and if there is no more slot available in
+the TLB the request will just fail and the device will have to try again later.
+It is not efficient but at least managing TLB refill can be avoided.
This leads to an issue with the memory that can be used for transfer between
-device and memory (see Limitations below). As we only support 4Ko page size we
-can only map 8Mo. To be able to manage bigger transfer we can implement the
-huge page table in the Linux kernel and use a page table that match the size of
-huge page table for a given IOMMU (typically the PCIe IOMMU).
+device and memory (see Limitations below). As the kernel only supports 4KB page
+size only 8MB transfer can be mapped. In order to be able to manage bigger
+transfer size, it is required to implement the huge page table in the Linux
+kernel and use a page table that match the size of huge page table for a given
+IOMMU (typically the PCIe IOMMU).
-As we won't refill the TLB we know that we won't have more than 128*16 entries.
-In this case we can simply keep a table with all possible entries.
+Consequently, the maximum page table entries is 128*16 (2048) and the approach
+chosen to manage IOMMU TLB is is to keep a table with all posible entries.
Maintenance interface
~~~~~~~~~~~~~~~~~~~~~
It is possible to have several "maintainers" for the same IOMMU. The driver is
-using two of them. One that writes the TLB and another interface reads TLB. For
-debug purpose it is possible to display the content of the tlb by using the
-following command in gdb::
+using two of them: one that writes the TLB and another interface that reads it.
+For debug purpose it is possible to display the TLB content in gdb by::
gdb> p kvx_iommu_dump_tlb( <iommu addr>, 0)
@@ -155,34 +154,35 @@ Interrupts
~~~~~~~~~~
IOMMU can have 3 kind of interrupts that corresponds to 3 different types of
-errors (no mapping. protection, parity). When the IOMMU is shared between
-clusters (SoC periph and PCIe) then fifteen IRQs are generated according to the
-configuration of an association table. The association table is indexed by the
-ASN number (9 bits) and the entry of the table is a subscription mask with one
-bit per destination. Currently this is not managed by the driver.
+errors: no mapping, protection, and parity. When the IOMMU is shared between
+clusters (SoC periph and PCIe), 15 IRQs are generated corresponding to
+association table configuration. The association table is indexed by the ASN
+number (9 bits) and the entry of the table is a subscription mask with one bit
+per destination. Currently this is not managed by the driver.
The driver is only managing interrupts for the cluster. The mode used is the
-stall one. So when an interrupt occurs it is managed by the driver. All others
-interrupts that occurs are stored and the IOMMU is stalled. When driver cleans
-the first interrupt others will be managed one by one.
+stall one. Thus, when an interrupt occurs it is managed by the driver. All
+others interrupts that occurs are stored and the IOMMU is stalled. When the
+driver cleans up the first interrupt, other interrupts will be managed
+sequentially.
ASN (Address Space Number)
~~~~~~~~~~~~~~~~~~~~~~~~~~
This is also know as ASID in some other architecture. Each device will have a
given ASN that will be given through the device tree. As address space is
-managed at the IOMMU domain level we will use one group and one domain per ID.
+managed at the IOMMU domain level one group and one domain per ID is used.
ASN are coded on 9 bits.
Device tree
-----------
-Relationships between devices, DMAs and IOMMUs are described in the
-device tree (see ``Documentation/devicetree/bindings/iommu/kalray,kvx-iommu.txt``
-for more details).
+Relationships between devices, DMAs and IOMMUs are described in the device tree
+(see ``Documentation/devicetree/bindings/iommu/kalray,kvx-iommu.txt`` for more
+details).
Limitations
-----------
-Only supporting 4 KB page size will limit the size of mapped memory to 8 MB
-because the IOMMU TLB can have at most 128*16 entries.
+kvx kernel only supports 4 KB page size, which will limit the size of mapped
+memory to 8 MB because the IOMMU TLB can have at most 128*16 (2048) entries.
diff --git a/Documentation/kvx/kvx-mmu.rst b/Documentation/kvx/kvx-mmu.rst
index 05b9bc111e02db..edad3c52caf47f 100644
--- a/Documentation/kvx/kvx-mmu.rst
+++ b/Documentation/kvx/kvx-mmu.rst
@@ -2,26 +2,26 @@
kvx Memory Management Unit
==========================
-Virtual addresses are on 41 bits for kvx when using 64-bit mode.
-To differentiate kernel from user space, we use the high order bit
-(bit 40). When bit 40 is set, then the higher remaining bits must also be set to
-1. The virtual address must be extended with 1 when the bit 40 is set,
-if not the address must be zero extended. Bit 40 is set for kernel space
-mappings and not set for user space mappings.
+Virtual addresses are on 41 bits for kvx when using 64-bit mode. To
+differentiate kernel from user space, the high order bit (bit 40) is used. If
+it is set, then the higher remaining bits must also be set to 1. The virtual
+address must be extended with 1 when the bit 40 is set, if not the address must
+be zero extended. Bit 40 is set for kernel space mappings and not set for user
+space mappings.
Memory Map
==========
-In Linux physical memories are arranged into banks according to the cost of an
-access in term of distance to a memory. As we are UMA architecture we only have
-one bank and thus one node.
+In Linux physical memories are arranged into "banks" according to the cost of
+an access in term of distance to a memory. As kvx is an UMA architecture there
+is only one bank and thus one node.
A node is divided into several kind of zone. For example if DMA can only access
-a specific area in the physical memory we will define a ZONE_DMA for this purpose.
-In our case we are considering that DMA can access all DDR so we don't have a specific
-zone for this. On 64 bit architecture all DDR can be mapped in virtual kernel space
-so there is no need for a ZONE_HIGHMEM. That means that in our case there is
-only one ZONE_NORMAL. This will be updated if DMA cannot access all memory.
+a specific area in the physical memory, the region is called ``ZONE_DMA``. In
+kvx we assume that DMA can access all memory so we don't have a specific zone
+for this purpose. On 64-bit architecture all memory can be mapped in virtual
+kernel space so ``ZONE_HIGHMEM`` is unnecessary. This implies that there is
+only ``ZONE_NORMAL``. This can change if DMA cannot access all memory.
Currently, the memory mapping is the following for 4KB page:
@@ -46,92 +46,94 @@ Currently, the memory mapping is the following for 4KB page:
Enable the MMU
==============
-All kernel functions and symbols are in virtual memory except for kvx_start()
-function which is loaded at 0x0 in physical memory.
-To be able to switch from physical addresses to virtual addresses we choose to
+All kernel functions and symbols are in virtual memory except for
+``kvx_start()`` function which is loaded at 0x0 in physical memory. To be able
+to switch from physical addresses to virtual addresses, the decision is to
setup the TLB at the very beginning of the boot process to be able to map both
-pieces of code. For this we added two entries in the LTLB. The first one,
-LTLB[0], contains the mapping between virtual memory and DDR. Its size is 512MB.
-The second entry, LTLB[1], contains a flat mapping of the first 2MB of the SMEM.
-Once those two entries are present we can enable the MMU. LTLB[1] will be
-removed during paging_init() because once we are really running in virtual space
-it will not be used anymore.
-In order to access more than 512MB DDR memory, the remaining memory (> 512MB) is
-refill using a comparison in kernel_perf_refill that does not walk the kernel
-page table, thus having a faster refill time for kernel. These entries are
-inserted into the LTLB for easier computation (4 LTLB entries). The drawback of
-this approach is that mapped entries are using RWX protection attributes,
-leading to no protection at all.
+pieces of code. For this two entries in the LTLB are added. The first one,
+LTLB[0], contains the mapping between virtual and physical memory. Its size is
+512MB. The second entry, LTLB[1], contains a flat mapping of the first 2MB of
+the SMEM. Once those two entries are present the MMU can be enabled. LTLB[1]
+will be removed during paging_init() because once we are really running in
+virtual space it will not be used anymore.
+
+In order to access more than 512MB of physical memory, the remaining memory (>
+512MB) is refilled using a comparison in ``kernel_perf_refill`` that does not
+walk the kernel page table, thus having a faster refill time for kernel. These
+entries are inserted into the LTLB for easier computation (4 LTLB entries). The
+drawback of this approach is that mapped entries are read-write (RWX), hence
+there is no protection and anything can happen.
Kernel strict RWX
=================
-CONFIG_STRICT_KERNEL_RWX is enabled by default in default_defconfig.
-Once booted, if CONFIG_STRICT_KERNEL_RWX is enable, the kernel text and memory
-will be mapped in the init_mm page table. Once mapped, the refill routine for
-the kernel is patched to always do a page table walk, bypassing the faster
-comparison but enforcing page protection attributes when refilling.
-Finally, the LTLB[0] entry is replaced by a 4K one, mapping only exceptions with
-RX protection. It allows us to never trigger nomapping on nomapping refill
-routine which would (obviously) not work... Once this is done, we can flush the
-4 LTLB entries for kernel refill in order to be sure there is no stalled
-entries and that new entries inserted in JTLB will apply.
+``CONFIG_STRICT_KERNEL_RWX`` is enabled by default. Once booted, if
+``CONFIG_STRICT_KERNEL_RWX`` is enable, the kernel text and memory will be
+mapped in the init_mm page table. Once mapped, the refill routine for the
+kernel is patched to always do walk the page table, bypassing the faster
+comparison but enforcing page protection attributes when refilling. Finally,
+the LTLB[0] entry is replaced by a 4K one, mapping only exceptions read-only
+(RX). It allows us to never trigger nomapping on nomapping refill routine which
+would (obviously) not work. Once this is done, 4 LTLB entries can be flushed
+for kernel refill in order to be sure there is no stalled entries and that new
+entries inserted in JTLB will apply.
By default, the following policy is applied on vmlinux sections:
- init_data: RW
-- init_text: RX (or RWX if parameter rodata=off)
-- text: RX (or RWX if parameter rodata=off)
+- init_text: RX (or RWX if parameter ``rodata=off`` is specified)
+- text: RX (or RWX if parameter ``rodata=off`` is specified)
- rodata: RW before init, RO after init
- sdata: RW
-Kernel RWX mode can then be switched on/off using /sys/kvx/kernel_rwx file.
+Kernel RWX mode can then be switched on/off with ``/sys/kvx/kernel_rwx``.
Privilege Level
================
-Since we are using privilege levels on kvx, we make use of the virtual
-spaces to be in the same space as the user. The kernel will have the
-$ps.mmup set in kernel (PL1) and unset for user (PL2).
-As said in kvx documentation, we have two cases when the kernel is
-booted:
+Since kvx uses privilege levels, the virtual spaces is leveraged so that the
+kernel memory space is same as userspace. The kernel will have the $ps.mmup set
+in kernel (PL1) and unset for user (PL2). As said in kvx documentation, there
+are two cases when the kernel is booted:
- Boot via intermediaries (bootloader, hypervisor, etc)
- Direct boot from flash
-In both cases, we will use the virtual space 0. Indeed, if we are alone
-on the core, then it means nobody is using the MMU and we can take the
-first virtual space. If not alone, then when writing an entry to the tlb
-using writetlb instruction, the hypervisor will catch it and change the
+In both cases, the virtual space 0 is used. Indeed, if there is only kernel
+running on the core, nothing else is using the MMU and the first virtual space
+can be used directly by the kernel. Otherwise, when writing an entry to the tlb
+using ``writetlb`` instruction, the hypervisor will catch it and change the
virtual space accordingly.
Memblock
========
When the kernel starts there is no memory allocator available. One of the first
-step in the kernel is to detect the amount of DDR available by getting this
-information in the device tree and initialize the low-level "memblock" allocator.
+step in the kernel is to detect the amount of available memory by getting this
+information in the device tree and initialize the low-level "memblock"
+allocator.
-We start by reserving memory for the whole kernel. For instance with a device
-tree containing 512Mo of DDR you could see the following boot messages:
+Memory initialization starts by reserving memory for the whole kernel. For
+instance with a 512MB RAM device dmesg will print::
-setup_bootmem: Memory : 0x100000000 - 0x120000000
-setup_bootmem: Reserved: 0x10001f000 - 0x1002d1bc0
+ setup_bootmem: Memory : 0x100000000 - 0x120000000
+ setup_bootmem: Reserved: 0x10001f000 - 0x1002d1bc0
During the paging init we need to set:
- - min_low_pfn that is the lowest PFN available in the system
- - max_low_pfn that indicates the end if NORMAL zone
- - max_pfn that is the number of pages in the system
+ - ``min_low_pfn`` - the lowest PFN available in the system
+ - ``max_low_pfn`` - the end of ``ZONE_NORMAL``
+ - ``max_pfn that`` - the number of pages in the system
-This setting is used for dividing memory into pages and for configuring the
-zone. See the memory map section for more information about ZONE.
+This scheme is used for dividing memory into pages and for configuring the
+zone. See the memory map section for more details.
-Zones are configured in free_area_init_core(). During start_kernel() other
-allocations are done for command line, cpu areas, PID hash table, different
-caches for VFS. This allocator is used until mem_init() is called.
+Zones are configured in ``free_area_init_core()``. During ``start_kernel()``
+other allocations are done for command line, cpu areas, PID hash table,
+different caches for VFS. The memblock allocator is used until ``mem_init()``
+is called.
-mem_init() is provided by the architecture. For MPPA we just call
-free_all_bootmem() that will go through all pages that are not used by the
+``mem_init()`` is provided by the architecture. For MPPA we just call
+``free_all_bootmem()`` that will go through all pages that are not used by the
low level allocator and mark them as not used. So physical pages that are
reserved for the kernel are still used and remain in physical memory. All pages
released will now be used by the buddy allocator.
@@ -146,20 +148,20 @@ LTLB Usage
==========
LTLB is used to add resident mapping which allows for faster MMU lookup.
-Currently, the LTLB is used to map some mandatory kernel pages and to allow fast
-accesses to l2 cache (mailbox and registers).
-When CONFIG_STRICT_KERNEL_RWX is disabled, 4 entries are reserved for kernel
-TLB refill using 512MB pages. When CONFIG_STRICT_KERNEL_RWX is enabled, these
+Currently, the LTLB is used to map some mandatory kernel pages and to allow
+fast accesses to l2 cache (mailbox and registers). When
+``CONFIG_STRICT_KERNEL_RWX`` is disabled, 4 entries are reserved for kernel TLB
+refill using 512MB pages. When ``CONFIG_STRICT_KERNEL_RWX`` is enabled, these
entries are unused since kernel is paginated using the same mecanism than for
user (page walking and entries in JTLB)
Page Table
==========
-We only support three levels for the page table and 4KB for page size.
+Only three-level page table and 4KB page size are supported.
-3 levels page table
--------------------
+3-level page table
+------------------
::
@@ -172,16 +174,16 @@ We only support three levels for the page table and 4KB for page size.
| +-----------------------> [29:21] PMD offset (9 bits)
+----------------------------------> [39:30] PGD offset (10 bits)
-Bits 40 to 64 are signed extended according to bit 39. If bit 39 is equal to 1
-we are in kernel space.
+Bits 40 to 64 are signed extended according to bit 39. If this bit is equal to
+1 the process is in kernel space.
-As 10 bits are used for PGD we need to allocate 2 pages.
+As 10 bits are used for PGD 2 pages are needed to be allocated.
PTE format
==========
-About the format of the PTE entry, as we are not forced by hardware for choices,
-we choose to follow the format described in the RiscV implementation as a
+For PTE entry format, instead of being forced by hardware constraints,
+we choose to follow the format described in the RISC-V implementation as a
starting point::
+---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
@@ -207,16 +209,16 @@ starting point::
Huge bit must be somewhere in the first 12 bits to be able to detect it
when reading the PMD entry.
-PageSZ must be on bit 10 and 11 because it matches the TEL.PS bits. And
-by doing that it is easier in assembly to set the TEL.PS to PageSZ.
+PageSZ must be on bit 10 and 11 because it matches the TEL.PS bits. As such,
+it is easier in assembly to set the TEL.PS to PageSZ.
Fast TLB refill
===============
-kvx core does not feature a hardware page walker. This work must be done
-by the core in software. In order to optimize TLB refill, a special fast
-path is taken when entering in kernel space.
-In order to speed up the process, the following actions are taken:
+kvx core does not feature a hardware page walker. Instead, page walking must be
+done by the core in software. In order to optimize TLB refill, a special fast
+path is utilized when entering in kernel space. In order to speed up the
+process, TLB refill is done by:
1. Save some registers in a per process scratchpad
2. If the trap is a nomapping then try the fastpath
@@ -224,21 +226,22 @@ In order to speed up the process, the following actions are taken:
4. Check if faulting address is a memory direct mapping one. If entry is a
direct mapping one and RWX is not enabled, add an entry into LTLB.
Otherwise, continue
-5. Try to walk the page table. If entry is not present, take the slowpath (do_page_fault)
+5. Try to walk the page table. If entry is not present, take the slowpath
+ (``do_page_fault``)
6. Refill the tlb properly
7. Exit by restoring only a few registers
ASN Handling
============
-Disclaimer: Some part of this are taken from ARC architecture.
+.. note::
+ Some part of ASN handling is inspired from ARC architecture.
kvx MMU provides 9-bit ASN (Address Space Number) in order to tag TLB entries.
It allows for multiple process with the same virtual space to cohabit without
-the need to flush TLB everytime we context switch.
-kvx implementation to use them is based on other architectures (such as arc
-or xtensa) and uses a wrapping ASN counter containing both cycle/generation and
-asn.
+the need to flush TLB everytime we context switch. kvx implementation to use
+them is based on other architectures (such as arc or xtensa) and uses a
+wrapping ASN counter containing both cycle/generation and asn.
::
@@ -250,27 +253,27 @@ asn.
This ASN counter is incremented monotonously to allocate new ASNs. When the
counter reaches 511 (9 bit), TLB is completely flushed and a new cycle is
started. A new allocation cycle, post rollover, could potentially reassign an
-ASN to a different task. Thus the rule is to reassign an ASN when the current
-context cycles does not match the allocation cycle.
-The 64 bit @cpu_asn_cache (and mm->asn) have 9 bits MMU ASN and rest 55 bits
-serve as cycle/generation indicator and natural 64 bit unsigned math
-automagically increments the generation when lower 9 bits rollover.
-When the counter completely wraps, we reset the counter to first cycle value
-(ie cycle = 1). This allows to distinguish context without any ASN and old cycle
-generated value with the same operation (XOR on cycle).
+ASN to a different task, hence the rule is to reassign an ASN when the current
+context cycles does not match the allocation cycle. The 64 bit
+``@cpu_asn_cache`` (and ``mm->asn``) have 9 bits MMU ASN and rest 55 bits serve
+as cycle/generation indicator and natural 64 bit unsigned math automagically
+increments the generation when lower 9 bits rollover. When the counter
+completely wraps, we reset the counter to first cycle value (ie cycle = 1).
+This allows to distinguish context without any ASN and old cycle generated
+value with the same operation (XOR on cycle).
Huge page
=========
-Currently only 3 level page table has been implemented for 4Ko base page size.
-So the page shift is 12 bits, the pmd shift is 21 and the pgdir shift is 30
-bits. This choice implies that for 4Ko base page size if we use a PMD as a huge
-page the size will be 2Mo and if we use a PUD as a huge page it will be 1Go.
+Currently only 3-level page table has been implemented for 4Ko base page size.
+As such, the page shift is 12 bits, the pmd shift is 21 and the pgdir shift is
+30 bits. This also implies that for 4Ko base page size, if PMD is used as a
+huge page the size will be 2Mo and if we use a PUD as a huge page it will be
+1Go.
-To support other huge page sizes (64Ko and 512Mo) we need to use several
-contiguous entries in the page table. For huge page of 64Ko we will need to
-use 16 entries in the PTE and for a huge page of 512Mo it means that 256
-entries in PMD will be used.
+To support other huge page sizes (64KB and 512MB) it is necessary to use
+several contiguous entries in the page table. For 64KB page size 16 entries in
+the PTE are needed whereas for 512MB page size it requires 256 entries in PMD.
Debug
=====
@@ -278,14 +281,14 @@ Debug
In order to debug the page table and tlb entries, gdb scripts contains commands
which allows to dump the page table:
-- lx-kvx-page-table-walk
+- ``lx-kvx-page-table-walk``
Display the current process page table by default
-- lx-kvx-tlb-decode
+- ``lx-kvx-tlb-decode``
Display the content of $tel and $teh into something readable
-Other commands available in kvx-gdb are the following:
+Other commands available in kvx-gdb are:
-- mppa-dump-tlb
+- ``mppa-dump-tlb``
Display the content of TLBs (JTLB and LTLB)
-- mppa-lookup-addr
+- ``mppa-lookup-addr``
Find physical address matching a virtual one
diff --git a/Documentation/kvx/kvx-smp.rst b/Documentation/kvx/kvx-smp.rst
index f170bc48ea5f7f..dbb02207beaff0 100644
--- a/Documentation/kvx/kvx-smp.rst
+++ b/Documentation/kvx/kvx-smp.rst
@@ -2,30 +2,29 @@
Symmetric Multiprocessing Implementation in kvx
===============================================
-On kvx, 5 clusters are organized as groups of 16 processors + 1
-secure core (RM) for each cluster. These 17 processors are L1$ coherent
-for TCM (tightly Coupled Memory). A mixed hw/sw L2$ is present to have
-cache coherency on DDR as well as TCM.
-The RM manager is not meant to run Linux so, 16 processors are available
-for SMP.
+On kvx, 5 clusters are organized as groups of 16 processors + 1 secure core
+(RM) for each cluster. These 17 processors are L1$ coherent for TCM (tightly
+Coupled Memory). There is also a mixed hw/sw L2$ to provide cache coherency on
+DDR as well as TCM. As the secure core (RM) is not meant to run the kernel,
+the rest 16 processors are available for SMP.
Booting
=======
-When booting the kvx processor, only the RM is woken up. This RM will
-execute a portion of code located in a section named .rm_firmware.
-By default, a simple power off code is embedded in this section.
-To avoid embedding the firmware in kernel sources, the section is patched
-using external tools to add the L2$ firmware (and replace the default firmware).
-Before executing this firmware, the RM boots the PE0. PE0 will then enable L2
+When booting the kvx processor, only the RM is woken up. This secure core will
+execute a portion of code located in a section named ``.rm_firmware``. By
+default, a simple power off code is embedded in this section. To avoid
+embedding the firmware in kernel sources, the section is patched using external
+tools to add the L2$ firmware (and replace the default firmware). Before
+executing this firmware, the RM boots the PE0. PE0 will then enable L2
coherency and request will be stalled until RM boots the L2$ firmware.
Locking primitives
==================
-spinlock/rwlock are using the kernel standard queued spinlock/rwlocks.
-These primitives are based on cmpxch and xchg. More particularly, it uses xchg16
-which is implemented as a read modify write with acswap on 32 bit word since
+spinlock/rwlock are using the kernel standard queued spinlock/rwlocks. These
+primitives are based on cmpxch and xchg. More particularly, it uses xchg16
+which is implemented as a read-modify-write with acswap on 32 bit word since
kvx does not have cmpxchg for size < 32bits.
IPI
diff --git a/Documentation/kvx/kvx.rst b/Documentation/kvx/kvx.rst
index 5385e1e3d30187..c2b26ed7b06a8b 100644
--- a/Documentation/kvx/kvx.rst
+++ b/Documentation/kvx/kvx.rst
@@ -5,18 +5,19 @@ kvx Core Implementation
This documents will try to explain any architecture choice for the kvx
linux port.
-Regarding the peripheral, we MUST use device tree to describe ALL
-peripherals. The bindings should always start with "kalray,kvx" for all
-core related peripherals (watchdog, timer, etc)
+Regarding the peripheral, devicetree must be used to describe ALL
+peripherals. The bindings should always start with ``kalray,kvx`` for all
+core-related peripherals (watchdog, timer, etc)
System Architecture
===================
-On kvx, we have 4 levels of privilege level starting from 0 (most
-privileged one) to 3 (less privilege one). A system of owners allows
-to delegate ownership of resources by using specials system registers.
+On kvx, there are 4 levels of privilege level, starting from 0 (most
+privileged) to 3 (least privileged). Ownership system allows to delegate
+ownership of resources by using specials system registers.
-The 2 main software stacks for Linux Kernel are the following::
+The 2 main software stacks for Linux Kernel are bare metal and hypervisor.
+Below is simple privilege level diagrams of both stacks::
+-------------+ +-------------+
| PL0: Debug | | PL0: Debug |
@@ -32,104 +33,100 @@ In both cases, the kvx support for privileges has been designed using
only relative PL and thus should work on both configurations without
any modifications.
-When booting, the CPU is executing in PL0 and owns all the privileges.
-This level is almost dedicated to the debug routines for the debugguer.
-It only needs to own few privileges (breakpoint 0 and watchpoint 0) to
-be able to debug a system executing in PL1 to PL3.
-Debug routines are not always there for instance when the kernel is
-executing alone (booted from flash).
-In order to ease the load of debug routines, software convention is to
-jump directly to PL1 and let PL0 for the debug.
-When the kernel boots, it checks if the current privilege level is 0
-($ps.pl is the only absolute value). If so, then it will delegate
-almost all resources to PL1 and use a RFE to lower its execution
-privilege level (see asm_delegate_pl in head.S).
-If the current PL is already different from 0, then it means somebody
-is above us and we need to request resource to inform it we need them. It will
-then either delegate them to us directly or virtualize the delegation.
-All privileges levels have their set of banked registers (ps, ea, sps,
-sr, etc) which contain privilege level specific values.
-$sr (system reserved) is banked and will hold the current task_struct.
-This register is reserved and should not be touched by any other code.
+When booting, the CPU is executing in PL0 and owns all the privileges. This
+level is almost dedicated to the debug routines for the debugguer. It only
+needs to own few privileges (breakpoint 0 and watchpoint 0) to be able to debug
+a system executing in PL1 to PL3. Debug routines are not always there, for
+instance when the kernel is executing alone (directly booted from flash). In
+order to ease the loading them, the software convention is to jump directly to
+PL1 and let PL0 for the debug. When the kernel boots, it checks if the current
+privilege level is 0 (note that ``$ps.pl`` can only contains positive integer).
+If so, then it will delegate almost all resources to PL1 and use a RFE to lower
+its execution privilege level (see ``asm_delegate_pl`` in ``head.S``). If the
+current PL is already different from 0, then it means that there is something
+in PL 0 and it is necessary to request resource in order to inform it that the
+privilege is needed. It will then either delegate them to the kernel directly
+or virtualize the delegation. All privileges levels have their set of banked
+registers (ps, ea, sps, sr, etc) which contain privilege level specific values.
+$sr (system reserved) is banked and hold the current task_struct. This register
+is reserved and should not be touched by any other code.
+
For more information, refer to the kvx system level architecture manual.
Boot
====
-On kvx, the RM (Secure Core) of Cluster 0 will boot first. It will then be able
-to boot a firmware. This firmware is stored in the rm_firmware section.
-The first argument ($r0) of this firmware will be a pointer to a function with
-the following prototype: void firmware_init_done(uint64_t features). This
-function is responsible of describing the features supported by the firmware and
-will start the first PE after that.
-By default, the rm_firmware function act as the "default" firmware. This
-function does nothing except calling firmware_init_done and then goes to sleep.
-In order to add another firmware, the rm_firmware section is patched using
-objcopy. The content of this section is then replaced by the provided firmware.
-This firmware will do an init and then call firmware_init_done before running
-the main loop.
+On kvx, the RM (Secure Core) of Cluster 0 will boot first. It will be used to
+boot a firmware. This firmware is stored in the ``rm_firmware`` section. The
+first argument ($r0) of this firmware will be a pointer to a function with
+``void firmware_init_done(uint64_t features)`` prototype. This function is
+responsible of describing the features supported by the firmware and will start
+the first PE after that. By default, the ``rm_firmware`` function act as the
+"default" firmware. This function does nothing except calling
+``firmware_init_done`` and then goes to sleep. In order to add another
+firmware, the ``rm_firmware`` section is patched using ``objcopy``. The content
+of this section is then replaced by the provided firmware. This firmware will
+be initialized and then call firmware_init_done before running the main loop.
When the PE boots, it will check for the firmware features to enable or disable
specific core features (L2$ for instance).
-When entering the C (kvx_lowlevel_start) the kernel will look for a special
-magic in $r0 (0x494C314B). This magic tells the kernel if there is arguments
-passed by a bootloader.
-Currently, the following values are passed through registers:
+When entering the C code (``kvx_lowlevel_start``) the kernel will look for a
+special magic in $r0 (0x494C314B). It tells the kernel if there are arguments
+passed by a bootloader. Currently, the following values are passed through
+registers:
- r1: pointer to command line setup by bootloader
- r2: device tree
-If this magic is not set, then, the command line will be the one
-provided in the device tree (see bootargs). The default device tree is
-not builtin but will be patched by the runner used (simulator or jtag) in the
-dtb section.
+If this magic is not set, then, the command line will be the one provided in
+the device tree (see ``bootargs``). The default devicetree is not builtin but
+will be patched by the runner used (simulator or jtag) in the dtb section.
-A default stdout-path is desirable to allow early printk.
+The default stdout path is sufficient to allow early printk.
Boot Memory Allocator
=====================
The boot memory allocator is used to allocate memory before paging is enabled.
It is initialized with DDR and also with the shared memory. This first one is
-initialized during the setup_bootmem() and the second one when calling
-early_init_fdt_scan_reserved_mem().
+initialized during ``setup_bootmem()`` and the second one is initialized when
+calling ``early_init_fdt_scan_reserved_mem()``.
Virtual and physical memory
===========================
The mapping used and the memory management is described in
-Documentation/kvx/kvx-mmu.txt.
-Our Kernel is compiled using virtual addresses that starts at
-0xffffff0000000000. But when it is started the kernel uses physical addresses.
-Before calling the first function arch_low_level_start() we configure 2 entries
-of the LTLB.
+Documentation/kvx/kvx-mmu.rst. The kernel is compiled using virtual addresses
+that starts at 0xffffff0000000000, however when it is started it uses physical
+addresses. Before calling the first function ``arch_low_level_start()``, 2 LTLB
+entries are configured first.
-The first entry will map the first 1G of virtual address space to the first
-1G of DDR:
+The first one maps the first 1G of virtual address space to the first 1G of
+DDR:
- TLB[0]: 0xffffff0000000000 -> 0x100000000 (size 512Mo)
-The second entry will be a flat mapping of the first 512 Ko of the SMEM. It
+The second one is a flat mapping of the first 512 Ko of the SMEM. It
is required to have this flat mapping because there is still code located at
this address that needs to be executed:
- TLB[1]: 0x0 -> 0x0 (size 512Ko)
-Once virtual space reached the second entry is removed.
+Once virtual memory space is reached the second entry is removed.
-To be able to set breakpoints when MMU is enabled we added a label called
-gdb_mmu_enabled. If you try to set a breakpoint on a function that is in
-virtual memory before the activation of the MMU this address as no signification
-for GDB. So, for example, if you want to break on the function start_kernel()
-you will need to run::
+To be able to set breakpoints when MMU is enabled, ``gdb_mmu_enabled`` label is
+added. If you try to set a breakpoint on a function that is in virtual memory
+before the activation of the MMU it will be unhelpful for GDB. Thus, for
+example, if you want to break on the function ``start_kernel()`` you will need
+to do::
kvx-gdb -silent path_to/vmlinux \
-ex 'tbreak gdb_mmu_enabled' -ex 'run' \
-ex 'break start_kernel' \
-ex 'continue'
-We will also add an option to kvx-gdb to simplify this step.
+In the future there will be an option to kvx-gdb to simplify this step.
Timers
======
@@ -137,7 +134,7 @@ Timers
The free-runinng clock (clocksource) is based on the DSU. This clock is
not interruptible and never stops even if core go into idle.
-Regarding the tick (clockevent), we use the timer 0 available on the core.
+Regarding the tick (clockevent), the timer 0 available on the core is used.
This timer allows to set a periodic tick which will be used as the main
tick for each core. Note that this clock is percpu.
@@ -149,60 +146,58 @@ stop the cycle counter)
Context switching
=================
-context switching is done in entry.S. When spawning a fresh thread,
-copy_thread is called. During this call, we setup callee saved register
-r20 and r21 to special values containing the function to call.
+context switching is done in ``entry.S``. When spawning a fresh thread,
+copy_thread is called. During this call, callee-saved registers r20 and r21
+are set up to special values containing the function to call.
-The normal path for a kernel thread will be the following:
+The normal path for a kernel thread is:
- 1. Enter copy_thread_tls and setup callee saved registers which will
- be restored in __switch_to.
- 2. set r20 and r21 (in thread_struct) to function and argument and
- ra to ret_from_kernel_thread.
- These callee saved will be restored in switch_to.
- 3. Call _switch_to at some point.
- 4. Save all callee saved register since switch_to is seen as a
+ 1. Enter ``copy_thread_tls`` and setup callee saved registers which will
+ be restored in ``__switch_to``.
+ 2. Set r20 and r21 (in ``thread_struct``) to function and argument and
+ ra to ``ret_from_kernel_thread``. These callee-saved registers will be
+ restored in switch_to.
+ 3. Call ``_switch_to`` at some point.
+ 4. Save all callee-saved registers since ``switch_to`` is seen as a
standard function call by the caller.
5. Change stack pointer to the new stack
6. At the end of switch to, set sr0 to the new task and use ret to
- jump to ret_from_kernel_thread (address restored from ra).
- 7. In ret_from_kernel_thread, execute the function with arguments by
+ jump to ``ret_from_kernel_thread`` (address restored from ra).
+ 7. In ``ret_from_kernel_thread``, execute the function with arguments by
using r20, r21 and we are done
-For more explanation, you can refer to https://lwn.net/Articles/520227/
+For more explanation, see https://lwn.net/Articles/520227/
User thread creation
====================
-We are using almost the same path as copy_thread to create it.
-The detailed path is the following:
+The similar path as ``copy_thread`` is used to create threads. It consists of:
- 1. Call start_thread which will setup user pc and stack pointer in
- task regs. We also set sps and clear privilege mode bit.
- When returning from exception, it will "flip" to user mode.
- 2. Enter copy_thread_tls and setup callee saved registers which will
- be restored in __switch_to. Also, set the "return" function to be
- ret_from_fork which will be called at end of switch_to
- 3. set r20 (in thread_struct) with tracing information.
- (simply by lazyness to avoid computing it in assembly...)
- 4. Call _switch_to at some point.
- 5. The current pc will then be restored to be ret_from fork.
- 6. Ret from fork calls schedule_tail and then check if tracing is
- enabled. If so call syscall_trace_exit
- 7. Finally, instead of returning to kernel, we restore all registers
- that have been setup by start_thread by restoring regs stored on
- stack
+ 1. Call ``start_thread`` which will set up user pc and stack pointer in
+ task regs. sps and clear privilege mode bits are also set.
+ When return from exception, it will "flip" to user mode.
+ 2. Enter ``copy_thread_tls`` and setup callee-saved registers which will
+ be restored in ``__switch_to``. Also, set the "return" function to be
+ ``ret_from_fork`` which will be called at end of ``switch_to``
+ 3. set r20 (in ``thread_struct``) with tracing information.
+ (this is done to avoid computing it in assembly...)
+ 4. Call ``_switch_to`` at some point.
+ 5. The current pc will then be restored to be ``ret_from`` fork.
+ 6. ``ret_from`` fork calls ``schedule_tail`` and then check if tracing is
+ enabled. If so call ``syscall_trace_exit``.
+ 7. Finally, instead of returning to kernel, all registers that have been
+ setup by ``start_thread`` are restored by restoring regs stored on stack.
L2$ handling
============
On kvx, the L2$ is handled by a firmware running on the RM. This firmware needs
various information to be aware of its configuration and communicate with the
-kernel. In order to do that, when firmware is starting, the device tree is given
-as parameter along with the "registers" zone. This zone is simply a memory area
-where data are exchanged between kernel <-> L2$. When some commands are written
-to it, the kernel sends an interrupt using a mailbox.
-If the L2$ node is not present in the device tree, then, the RM will directly go
+kernel. In order to do that, when firmware is starting, the device tree is
+given as parameter along with the "registers" zone. This zone is simply a
+memory area where data are exchanged between kernel and L2$. When firmware
+commands are written to it, the kernel sends an interrupt using a mailbox. If
+the L2$ node is not present in the device tree, then, the RM will directly go
into sleeping.
Boot diagram::
@@ -246,14 +241,16 @@ Boot diagram::
+------------+ + v
-Since this driver is started early (before SMP boot), A lot of drivers are not
+Since this driver is started early (before SMP boot), a lot of drivers are not
yet probed (mailboxes, iommu, etc) and thus can not be used.
Building
========
-In order to build the kernel, you will need a complete kvx toolchain.
-First, setup the config using the following command line::
+In order to build the kernel, you will need kvx cross toolchain and have it
+somewhere in the ``PATH``.
+
+First, prepare the config by::
$ make ARCH=kvx O=your_directory default_defconfig
@@ -261,11 +258,11 @@ Adjust any configuration option you may need and then, build the kernel::
$ make ARCH=kvx O=your_directory -j12
-You will finally have a vmlinux image ready to be run::
+You will finally have a vmlinux image which can be run by::
$ kvx-mppa -- vmlinux
-Additionally, you may want to debug it. To do so, use kvx-gdb::
+In case you need to debug the kernel, you can simply launch::
$ kvx-gdb vmlinux
--
An old man doll... just what I always wanted! - Clara
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation)
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
` (7 preceding siblings ...)
2023-01-09 9:51 ` [PATCH 8/8] Documentation: kvx: reword Bagas Sanjaya
@ 2023-01-09 10:59 ` Jules Maselbas
2023-01-10 0:18 ` Randy Dunlap
9 siblings, 0 replies; 113+ messages in thread
From: Jules Maselbas @ 2023-01-09 10:59 UTC (permalink / raw)
To: Bagas Sanjaya
Cc: Jonathan Corbet, Yann Sionneau, linux-doc, linux-kernel,
Clement Leger, Guillaume Thouvenin
Hi Bagas,
On Mon, Jan 09, 2023 at 04:51:00PM +0700, Bagas Sanjaya wrote:
> Jonathan Corbet <corbet@lwn.net> writes:
> >> Add some documentation for kvx arch and its Linux port.
> >>
> >> CC: Jonathan Corbet <corbet@lwn.net>
> >> CC: linux-doc@vger.kernel.org
> >> CC: linux-kernel@vger.kernel.org
> >> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> >> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> >> Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> >> Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> >> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> >> ---
> >> Documentation/kvx/kvx-exceptions.txt | 246 ++++++++++++++++++++++++
> >> Documentation/kvx/kvx-iommu.txt | 183 ++++++++++++++++++
> >> Documentation/kvx/kvx-mmu.txt | 272 +++++++++++++++++++++++++++
> >> Documentation/kvx/kvx-smp.txt | 36 ++++
> >> Documentation/kvx/kvx.txt | 268 ++++++++++++++++++++++++++
> >> 5 files changed, 1005 insertions(+)
> >> create mode 100644 Documentation/kvx/kvx-exceptions.txt
> >> create mode 100644 Documentation/kvx/kvx-iommu.txt
> >> create mode 100644 Documentation/kvx/kvx-mmu.txt
> >> create mode 100644 Documentation/kvx/kvx-smp.txt
> >> create mode 100644 Documentation/kvx/kvx.txt
> >
> >Please write this documentation in the RST format (you're 95% of the way
> >there now) and incorporate into the kernel docs build.
>
> Here is the polished documentation in reST format. Yann, can you please
> squash this series into your documentation patch?
Thanks a lot, I've already converted the documentation to reST, but I will
take a look at this series and see what I can apply.
> Bagas Sanjaya (8):
> Documentation: kvx: Convert to reST
> Documentation: kvx: Wrap diagrams in literal code block
> Documentation: kvx: Fix lists
> Documentation: kvx: kvx-iommu: Use reST syntax for subsections
> Documentation: kvx: kvx-iommu: monospacize kvx iommu device tree path
> Documentation: kvx: Promote title headings
> Documentation: kvx: Use literal code block for command-line inputs
> Documentation: kvx: reword
>
> Documentation/arch.rst | 1 +
> Documentation/kvx/index.rst | 12 ++
> Documentation/kvx/kvx-exceptions.rst | 258 +++++++++++++++++++++++
> Documentation/kvx/kvx-exceptions.txt | 246 ----------------------
> Documentation/kvx/kvx-iommu.rst | 188 +++++++++++++++++
> Documentation/kvx/kvx-iommu.txt | 183 -----------------
> Documentation/kvx/kvx-mmu.rst | 294 +++++++++++++++++++++++++++
> Documentation/kvx/kvx-mmu.txt | 272 -------------------------
> Documentation/kvx/kvx-smp.rst | 36 ++++
> Documentation/kvx/kvx-smp.txt | 36 ----
> Documentation/kvx/kvx.rst | 269 ++++++++++++++++++++++++
> Documentation/kvx/kvx.txt | 268 ------------------------
> 12 files changed, 1058 insertions(+), 1005 deletions(-)
> create mode 100644 Documentation/kvx/index.rst
> create mode 100644 Documentation/kvx/kvx-exceptions.rst
> delete mode 100644 Documentation/kvx/kvx-exceptions.txt
> create mode 100644 Documentation/kvx/kvx-iommu.rst
> delete mode 100644 Documentation/kvx/kvx-iommu.txt
> create mode 100644 Documentation/kvx/kvx-mmu.rst
> delete mode 100644 Documentation/kvx/kvx-mmu.txt
> create mode 100644 Documentation/kvx/kvx-smp.rst
> delete mode 100644 Documentation/kvx/kvx-smp.txt
> create mode 100644 Documentation/kvx/kvx.rst
> delete mode 100644 Documentation/kvx/kvx.txt
>
> --
> An old man doll... just what I always wanted! - Clara
>
>
>
>
>
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation)
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
` (8 preceding siblings ...)
2023-01-09 10:59 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Jules Maselbas
@ 2023-01-10 0:18 ` Randy Dunlap
9 siblings, 0 replies; 113+ messages in thread
From: Randy Dunlap @ 2023-01-10 0:18 UTC (permalink / raw)
To: Bagas Sanjaya, Jonathan Corbet, Yann Sionneau
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin
Hi Bagas,
On 1/9/23 01:51, Bagas Sanjaya wrote:
> Jonathan Corbet <corbet@lwn.net> writes:
>>> Add some documentation for kvx arch and its Linux port.
>>>
...
>>
>> Please write this documentation in the RST format (you're 95% of the way
>> there now) and incorporate into the kernel docs build.
>
> Here is the polished documentation in reST format. Yann, can you please
> squash this series into your documentation patch?
JFYI, to me "improv" applies mostly to jazz (music) or comedy. :)
https://www.merriam-webster.com/dictionary/improv
https://en.wikipedia.org/wiki/Improv
Thanks for the patches.
--
~Randy
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation
2023-01-03 17:50 ` Jonathan Corbet
2023-01-09 9:51 ` [PATCH 0/8] kvx documentation improv (was: Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation) Bagas Sanjaya
@ 2023-01-18 8:44 ` Yann Sionneau
1 sibling, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-18 8:44 UTC (permalink / raw)
To: Jonathan Corbet
Cc: linux-doc, linux-kernel, Clement Leger, Guillaume Thouvenin
On 03/01/2023 18:50, Jonathan Corbet wrote:
> Yann Sionneau <ysionneau@kalray.eu> writes:
>
>> Add some documentation for kvx arch and its Linux port.
>>
>> CC: Jonathan Corbet <corbet@lwn.net>
>> CC: linux-doc@vger.kernel.org
>> CC: linux-kernel@vger.kernel.org
>> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
>> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
>> Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
>> Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
>> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
>> ---
>> Documentation/kvx/kvx-exceptions.txt | 246 ++++++++++++++++++++++++
>> Documentation/kvx/kvx-iommu.txt | 183 ++++++++++++++++++
>> Documentation/kvx/kvx-mmu.txt | 272 +++++++++++++++++++++++++++
>> Documentation/kvx/kvx-smp.txt | 36 ++++
>> Documentation/kvx/kvx.txt | 268 ++++++++++++++++++++++++++
>> 5 files changed, 1005 insertions(+)
>> create mode 100644 Documentation/kvx/kvx-exceptions.txt
>> create mode 100644 Documentation/kvx/kvx-iommu.txt
>> create mode 100644 Documentation/kvx/kvx-mmu.txt
>> create mode 100644 Documentation/kvx/kvx-smp.txt
>> create mode 100644 Documentation/kvx/kvx.txt
> Please write this documentation in the RST format (you're 95% of the way
> there now) and incorporate into the kernel docs build.
Ack, it will be part of the V2, thanks for the review.
--
Yann
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation
2023-01-03 16:43 ` [RFC PATCH 01/25] Documentation: kvx: Add basic documentation Yann Sionneau
2023-01-03 17:50 ` Jonathan Corbet
@ 2023-01-05 18:38 ` kernel test robot
2023-01-18 15:09 ` Jeff Xie
2 siblings, 0 replies; 113+ messages in thread
From: kernel test robot @ 2023-01-05 18:38 UTC (permalink / raw)
To: Yann Sionneau; +Cc: oe-kbuild-all
[-- Attachment #1: Type: text/plain, Size: 1569 bytes --]
Hi Yann,
[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on tip/irq/core]
[also build test WARNING on kees/for-next/execve soc/for-next linus/master v6.2-rc2 next-20230105]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Yann-Sionneau/Documentation-kvx-Add-basic-documentation/20230104-010703
patch link: https://lore.kernel.org/r/20230103164359.24347-2-ysionneau%40kalray.eu
patch subject: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation
reproduce:
# https://github.com/intel-lab-lkp/linux/commit/4ac33edac5947a08dadef4d4f4fb2211758220b7
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Yann-Sionneau/Documentation-kvx-Add-basic-documentation/20230104-010703
git checkout 4ac33edac5947a08dadef4d4f4fb2211758220b7
make menuconfig
# enable CONFIG_COMPILE_TEST, CONFIG_WARN_MISSING_DOCUMENTS, CONFIG_WARN_ABI_ERRORS
make htmldocs
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> Warning: Documentation/kvx/kvx-iommu.txt references a file that doesn't exist: Documentation/devicetree/bindings/iommu/kalray,kvx-iommu.txt
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
[-- Attachment #2: config --]
[-- Type: text/plain, Size: 38842 bytes --]
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86_64 6.1.0-rc5 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="gcc-11 (Debian 11.3.0-8) 11.3.0"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=110300
CONFIG_CLANG_VERSION=0
CONFIG_AS_IS_GNU=y
CONFIG_AS_VERSION=23900
CONFIG_LD_IS_BFD=y
CONFIG_LD_VERSION=23900
CONFIG_LLD_VERSION=0
CONFIG_CC_CAN_LINK=y
CONFIG_CC_CAN_LINK_STATIC=y
CONFIG_CC_HAS_ASM_GOTO_OUTPUT=y
CONFIG_CC_HAS_ASM_INLINE=y
CONFIG_CC_HAS_NO_PROFILE_FN_ATTR=y
CONFIG_PAHOLE_VERSION=123
CONFIG_IRQ_WORK=y
CONFIG_BUILDTIME_TABLE_SORT=y
CONFIG_THREAD_INFO_IN_TASK=y
#
# General setup
#
CONFIG_BROKEN_ON_SMP=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_COMPILE_TEST=y
# CONFIG_WERROR is not set
CONFIG_LOCALVERSION=""
CONFIG_BUILD_SALT=""
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_HAVE_KERNEL_LZ4=y
CONFIG_HAVE_KERNEL_ZSTD=y
CONFIG_KERNEL_GZIP=y
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_LZO is not set
# CONFIG_KERNEL_LZ4 is not set
# CONFIG_KERNEL_ZSTD is not set
CONFIG_DEFAULT_INIT=""
CONFIG_DEFAULT_HOSTNAME="(none)"
# CONFIG_SYSVIPC is not set
# CONFIG_WATCH_QUEUE is not set
# CONFIG_CROSS_MEMORY_ATTACH is not set
# CONFIG_USELIB is not set
CONFIG_HAVE_ARCH_AUDITSYSCALL=y
#
# IRQ subsystem
#
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_HARDIRQS_SW_RESEND=y
CONFIG_IRQ_DOMAIN=y
CONFIG_IRQ_DOMAIN_HIERARCHY=y
CONFIG_GENERIC_IRQ_MATRIX_ALLOCATOR=y
CONFIG_GENERIC_IRQ_RESERVATION_MODE=y
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y
# end of IRQ subsystem
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_ARCH_CLOCKSOURCE_INIT=y
CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_HAVE_POSIX_CPU_TIMERS_TASK_WORK=y
CONFIG_POSIX_CPU_TIMERS_TASK_WORK=y
#
# Timers subsystem
#
CONFIG_HZ_PERIODIC=y
# CONFIG_NO_HZ_IDLE is not set
# CONFIG_NO_HZ is not set
# CONFIG_HIGH_RES_TIMERS is not set
CONFIG_CLOCKSOURCE_WATCHDOG_MAX_SKEW_US=100
# end of Timers subsystem
CONFIG_HAVE_EBPF_JIT=y
CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y
#
# BPF subsystem
#
# CONFIG_BPF_SYSCALL is not set
# end of BPF subsystem
CONFIG_PREEMPT_NONE_BUILD=y
CONFIG_PREEMPT_NONE=y
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set
# CONFIG_PREEMPT_DYNAMIC is not set
#
# CPU/Task time and stats accounting
#
CONFIG_TICK_CPU_ACCOUNTING=y
# CONFIG_VIRT_CPU_ACCOUNTING_GEN is not set
# CONFIG_IRQ_TIME_ACCOUNTING is not set
# CONFIG_BSD_PROCESS_ACCT is not set
# CONFIG_PSI is not set
# end of CPU/Task time and stats accounting
CONFIG_CPU_ISOLATION=y
#
# RCU Subsystem
#
CONFIG_TINY_RCU=y
# CONFIG_RCU_EXPERT is not set
CONFIG_SRCU=y
CONFIG_TINY_SRCU=y
# end of RCU Subsystem
# CONFIG_IKCONFIG is not set
# CONFIG_IKHEADERS is not set
CONFIG_LOG_BUF_SHIFT=17
CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=13
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
#
# Scheduler features
#
# end of Scheduler features
CONFIG_ARCH_SUPPORTS_NUMA_BALANCING=y
CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH=y
CONFIG_CC_HAS_INT128=y
CONFIG_CC_IMPLICIT_FALLTHROUGH="-Wimplicit-fallthrough=5"
CONFIG_GCC12_NO_ARRAY_BOUNDS=y
CONFIG_ARCH_SUPPORTS_INT128=y
# CONFIG_CGROUPS is not set
CONFIG_NAMESPACES=y
# CONFIG_UTS_NS is not set
# CONFIG_TIME_NS is not set
# CONFIG_USER_NS is not set
# CONFIG_PID_NS is not set
# CONFIG_CHECKPOINT_RESTORE is not set
# CONFIG_SCHED_AUTOGROUP is not set
# CONFIG_SYSFS_DEPRECATED is not set
# CONFIG_RELAY is not set
# CONFIG_BLK_DEV_INITRD is not set
# CONFIG_BOOT_CONFIG is not set
# CONFIG_INITRAMFS_PRESERVE_MTIME is not set
CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_LD_ORPHAN_WARN=y
CONFIG_SYSCTL=y
CONFIG_SYSCTL_EXCEPTION_TRACE=y
CONFIG_HAVE_PCSPKR_PLATFORM=y
# CONFIG_EXPERT is not set
CONFIG_MULTIUSER=y
CONFIG_SGETMASK_SYSCALL=y
CONFIG_SYSFS_SYSCALL=y
CONFIG_FHANDLE=y
CONFIG_POSIX_TIMERS=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_FUTEX_PI=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_IO_URING=y
CONFIG_ADVISE_SYSCALLS=y
CONFIG_MEMBARRIER=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_BASE_RELATIVE=y
CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE=y
CONFIG_RSEQ=y
# CONFIG_EMBEDDED is not set
CONFIG_HAVE_PERF_EVENTS=y
#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
# end of Kernel Performance Events And Counters
# CONFIG_PROFILING is not set
# end of General setup
CONFIG_64BIT=y
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_MMU=y
CONFIG_ARCH_MMAP_RND_BITS_MIN=28
CONFIG_ARCH_MMAP_RND_BITS_MAX=32
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MIN=8
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MAX=16
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_NR_GPIO=1024
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_AUDIT_ARCH=y
CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_PGTABLE_LEVELS=4
CONFIG_CC_HAS_SANE_STACKPROTECTOR=y
#
# Processor type and features
#
# CONFIG_SMP is not set
CONFIG_X86_FEATURE_NAMES=y
CONFIG_X86_MPPARSE=y
# CONFIG_GOLDFISH is not set
# CONFIG_X86_CPU_RESCTRL is not set
# CONFIG_X86_EXTENDED_PLATFORM is not set
# CONFIG_SCHED_OMIT_FRAME_POINTER is not set
# CONFIG_HYPERVISOR_GUEST is not set
# CONFIG_MK8 is not set
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_MATOM is not set
CONFIG_GENERIC_CPU=y
CONFIG_X86_INTERNODE_CACHE_SHIFT=6
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_IA32_FEAT_CTL=y
CONFIG_X86_VMX_FEATURE_NAMES=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_AMD=y
CONFIG_CPU_SUP_HYGON=y
CONFIG_CPU_SUP_CENTAUR=y
CONFIG_CPU_SUP_ZHAOXIN=y
CONFIG_HPET_TIMER=y
CONFIG_DMI=y
CONFIG_NR_CPUS_RANGE_BEGIN=1
CONFIG_NR_CPUS_RANGE_END=1
CONFIG_NR_CPUS_DEFAULT=1
CONFIG_NR_CPUS=1
CONFIG_UP_LATE_INIT=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
# CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS is not set
# CONFIG_X86_MCE is not set
#
# Performance monitoring
#
# CONFIG_PERF_EVENTS_AMD_POWER is not set
# CONFIG_PERF_EVENTS_AMD_UNCORE is not set
# CONFIG_PERF_EVENTS_AMD_BRS is not set
# end of Performance monitoring
CONFIG_X86_16BIT=y
CONFIG_X86_ESPFIX64=y
CONFIG_X86_VSYSCALL_EMULATION=y
# CONFIG_X86_IOPL_IOPERM is not set
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set
# CONFIG_X86_5LEVEL is not set
CONFIG_X86_DIRECT_GBPAGES=y
# CONFIG_AMD_MEM_ENCRYPT is not set
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
# CONFIG_X86_CHECK_BIOS_CORRUPTION is not set
CONFIG_MTRR=y
# CONFIG_MTRR_SANITIZER is not set
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
CONFIG_X86_UMIP=y
CONFIG_CC_HAS_IBT=y
# CONFIG_X86_KERNEL_IBT is not set
# CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS is not set
CONFIG_X86_INTEL_TSX_MODE_OFF=y
# CONFIG_X86_INTEL_TSX_MODE_ON is not set
# CONFIG_X86_INTEL_TSX_MODE_AUTO is not set
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
# CONFIG_KEXEC is not set
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x1000000
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_ALIGN=0x200000
CONFIG_LEGACY_VSYSCALL_XONLY=y
# CONFIG_LEGACY_VSYSCALL_NONE is not set
# CONFIG_CMDLINE_BOOL is not set
CONFIG_MODIFY_LDT_SYSCALL=y
# CONFIG_STRICT_SIGALTSTACK_SIZE is not set
CONFIG_HAVE_LIVEPATCH=y
# end of Processor type and features
CONFIG_CC_HAS_SLS=y
CONFIG_CC_HAS_RETURN_THUNK=y
# CONFIG_SPECULATION_MITIGATIONS is not set
CONFIG_ARCH_HAS_ADD_PAGES=y
CONFIG_ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE=y
#
# Power management and ACPI options
#
# CONFIG_SUSPEND is not set
# CONFIG_PM is not set
CONFIG_ARCH_SUPPORTS_ACPI=y
# CONFIG_ACPI is not set
#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set
# end of CPU Frequency scaling
#
# CPU Idle
#
# CONFIG_CPU_IDLE is not set
# end of CPU Idle
# end of Power management and ACPI options
#
# Bus options (PCI etc.)
#
CONFIG_ISA_DMA_API=y
# end of Bus options (PCI etc.)
#
# Binary Emulations
#
# CONFIG_IA32_EMULATION is not set
# CONFIG_X86_X32_ABI is not set
# end of Binary Emulations
CONFIG_HAVE_KVM=y
# CONFIG_VIRTUALIZATION is not set
CONFIG_AS_AVX512=y
CONFIG_AS_SHA1_NI=y
CONFIG_AS_SHA256_NI=y
CONFIG_AS_TPAUSE=y
#
# General architecture-dependent options
#
CONFIG_GENERIC_ENTRY=y
# CONFIG_JUMP_LABEL is not set
# CONFIG_STATIC_CALL_SELFTEST is not set
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_ARCH_USE_BUILTIN_BSWAP=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_KPROBES_ON_FTRACE=y
CONFIG_ARCH_CORRECT_STACKTRACE_ON_KRETPROBE=y
CONFIG_HAVE_FUNCTION_ERROR_INJECTION=y
CONFIG_HAVE_NMI=y
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_TRACE_IRQFLAGS_NMI_SUPPORT=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_CONTIGUOUS=y
CONFIG_GENERIC_SMP_IDLE_THREAD=y
CONFIG_ARCH_HAS_FORTIFY_SOURCE=y
CONFIG_ARCH_HAS_SET_MEMORY=y
CONFIG_ARCH_HAS_SET_DIRECT_MAP=y
CONFIG_HAVE_ARCH_THREAD_STRUCT_WHITELIST=y
CONFIG_ARCH_WANTS_DYNAMIC_TASK_STRUCT=y
CONFIG_ARCH_WANTS_NO_INSTR=y
CONFIG_HAVE_ASM_MODVERSIONS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_RSEQ=y
CONFIG_HAVE_RUST=y
CONFIG_HAVE_FUNCTION_ARG_ACCESS_API=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_HARDLOCKUP_DETECTOR_PERF=y
CONFIG_HAVE_PERF_REGS=y
CONFIG_HAVE_PERF_USER_STACK_DUMP=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y
CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE=y
CONFIG_MMU_GATHER_MERGE_VMAS=y
CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG=y
CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y
CONFIG_HAVE_CMPXCHG_LOCAL=y
CONFIG_HAVE_CMPXCHG_DOUBLE=y
CONFIG_HAVE_ARCH_SECCOMP=y
CONFIG_HAVE_ARCH_SECCOMP_FILTER=y
# CONFIG_SECCOMP is not set
CONFIG_HAVE_ARCH_STACKLEAK=y
CONFIG_HAVE_STACKPROTECTOR=y
# CONFIG_STACKPROTECTOR is not set
CONFIG_ARCH_SUPPORTS_LTO_CLANG=y
CONFIG_ARCH_SUPPORTS_LTO_CLANG_THIN=y
CONFIG_LTO_NONE=y
CONFIG_ARCH_SUPPORTS_CFI_CLANG=y
CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES=y
CONFIG_HAVE_CONTEXT_TRACKING_USER=y
CONFIG_HAVE_CONTEXT_TRACKING_USER_OFFSTACK=y
CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_HAVE_IRQ_TIME_ACCOUNTING=y
CONFIG_HAVE_MOVE_PUD=y
CONFIG_HAVE_MOVE_PMD=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD=y
CONFIG_HAVE_ARCH_HUGE_VMAP=y
CONFIG_HAVE_ARCH_HUGE_VMALLOC=y
CONFIG_ARCH_WANT_HUGE_PMD_SHARE=y
CONFIG_HAVE_ARCH_SOFT_DIRTY=y
CONFIG_HAVE_MOD_ARCH_SPECIFIC=y
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK=y
CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK=y
CONFIG_SOFTIRQ_ON_OWN_STACK=y
CONFIG_ARCH_HAS_ELF_RANDOMIZE=y
CONFIG_HAVE_ARCH_MMAP_RND_BITS=y
CONFIG_HAVE_EXIT_THREAD=y
CONFIG_ARCH_MMAP_RND_BITS=28
CONFIG_PAGE_SIZE_LESS_THAN_64KB=y
CONFIG_PAGE_SIZE_LESS_THAN_256KB=y
CONFIG_HAVE_OBJTOOL=y
CONFIG_HAVE_JUMP_LABEL_HACK=y
CONFIG_HAVE_NOINSTR_HACK=y
CONFIG_HAVE_NOINSTR_VALIDATION=y
CONFIG_HAVE_UACCESS_VALIDATION=y
CONFIG_HAVE_STACK_VALIDATION=y
CONFIG_HAVE_RELIABLE_STACKTRACE=y
# CONFIG_COMPAT_32BIT_TIME is not set
CONFIG_HAVE_ARCH_VMAP_STACK=y
# CONFIG_VMAP_STACK is not set
CONFIG_HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET=y
CONFIG_RANDOMIZE_KSTACK_OFFSET=y
# CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT is not set
CONFIG_ARCH_HAS_STRICT_KERNEL_RWX=y
CONFIG_STRICT_KERNEL_RWX=y
CONFIG_ARCH_HAS_STRICT_MODULE_RWX=y
CONFIG_HAVE_ARCH_PREL32_RELOCATIONS=y
CONFIG_ARCH_HAS_MEM_ENCRYPT=y
CONFIG_HAVE_STATIC_CALL=y
CONFIG_HAVE_STATIC_CALL_INLINE=y
CONFIG_HAVE_PREEMPT_DYNAMIC=y
CONFIG_HAVE_PREEMPT_DYNAMIC_CALL=y
CONFIG_ARCH_WANT_LD_ORPHAN_WARN=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_ARCH_SUPPORTS_PAGE_TABLE_CHECK=y
CONFIG_ARCH_HAS_ELFCORE_COMPAT=y
CONFIG_ARCH_HAS_PARANOID_L1D_FLUSH=y
CONFIG_DYNAMIC_SIGFRAME=y
CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG=y
#
# GCOV-based kernel profiling
#
CONFIG_ARCH_HAS_GCOV_PROFILE_ALL=y
# end of GCOV-based kernel profiling
CONFIG_HAVE_GCC_PLUGINS=y
# CONFIG_GCC_PLUGINS is not set
# end of General architecture-dependent options
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
# CONFIG_MODULES is not set
CONFIG_BLOCK=y
# CONFIG_BLOCK_LEGACY_AUTOLOAD is not set
# CONFIG_BLK_DEV_BSGLIB is not set
# CONFIG_BLK_DEV_INTEGRITY is not set
# CONFIG_BLK_DEV_ZONED is not set
# CONFIG_BLK_WBT is not set
# CONFIG_BLK_SED_OPAL is not set
# CONFIG_BLK_INLINE_ENCRYPTION is not set
#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y
CONFIG_EFI_PARTITION=y
# end of Partition Types
#
# IO Schedulers
#
# CONFIG_MQ_IOSCHED_DEADLINE is not set
# CONFIG_MQ_IOSCHED_KYBER is not set
# CONFIG_IOSCHED_BFQ is not set
# end of IO Schedulers
CONFIG_INLINE_SPIN_UNLOCK_IRQ=y
CONFIG_INLINE_READ_UNLOCK=y
CONFIG_INLINE_READ_UNLOCK_IRQ=y
CONFIG_INLINE_WRITE_UNLOCK=y
CONFIG_INLINE_WRITE_UNLOCK_IRQ=y
CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y
CONFIG_ARCH_USE_QUEUED_SPINLOCKS=y
CONFIG_ARCH_USE_QUEUED_RWLOCKS=y
CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE=y
CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE=y
CONFIG_ARCH_HAS_SYSCALL_WRAPPER=y
#
# Executable file formats
#
# CONFIG_BINFMT_ELF is not set
# CONFIG_BINFMT_SCRIPT is not set
# CONFIG_BINFMT_MISC is not set
CONFIG_COREDUMP=y
# end of Executable file formats
#
# Memory Management options
#
# CONFIG_SWAP is not set
#
# SLAB allocator options
#
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLAB_MERGE_DEFAULT is not set
# CONFIG_SLAB_FREELIST_RANDOM is not set
# CONFIG_SLAB_FREELIST_HARDENED is not set
# CONFIG_SLUB_STATS is not set
# end of SLAB allocator options
# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
# CONFIG_COMPAT_BRK is not set
CONFIG_SPARSEMEM=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
# CONFIG_SPARSEMEM_VMEMMAP is not set
CONFIG_HAVE_FAST_GUP=y
CONFIG_EXCLUSIVE_SYSTEM_RAM=y
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
# CONFIG_MEMORY_HOTPLUG is not set
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK=y
# CONFIG_COMPACTION is not set
# CONFIG_PAGE_REPORTING is not set
CONFIG_PHYS_ADDR_T_64BIT=y
# CONFIG_KSM is not set
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
CONFIG_ARCH_WANTS_THP_SWAP=y
# CONFIG_TRANSPARENT_HUGEPAGE is not set
CONFIG_NEED_PER_CPU_KM=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
# CONFIG_CMA is not set
CONFIG_GENERIC_EARLY_IOREMAP=y
# CONFIG_IDLE_PAGE_TRACKING is not set
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_ARCH_HAS_CURRENT_STACK_POINTER=y
CONFIG_ARCH_HAS_PTE_DEVMAP=y
CONFIG_ZONE_DMA=y
CONFIG_ZONE_DMA32=y
CONFIG_VM_EVENT_COUNTERS=y
# CONFIG_PERCPU_STATS is not set
#
# GUP_TEST needs to have DEBUG_FS enabled
#
CONFIG_ARCH_HAS_PTE_SPECIAL=y
CONFIG_SECRETMEM=y
# CONFIG_ANON_VMA_NAME is not set
# CONFIG_USERFAULTFD is not set
# CONFIG_LRU_GEN is not set
#
# Data Access Monitoring
#
# CONFIG_DAMON is not set
# end of Data Access Monitoring
# end of Memory Management options
# CONFIG_NET is not set
#
# Device Drivers
#
CONFIG_HAVE_EISA=y
# CONFIG_EISA is not set
CONFIG_HAVE_PCI=y
# CONFIG_PCI is not set
# CONFIG_PCCARD is not set
#
# Generic Driver Options
#
# CONFIG_UEVENT_HELPER is not set
# CONFIG_DEVTMPFS is not set
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
#
# Firmware loader
#
CONFIG_FW_LOADER=y
CONFIG_EXTRA_FIRMWARE=""
# CONFIG_FW_LOADER_USER_HELPER is not set
# CONFIG_FW_LOADER_COMPRESS is not set
# CONFIG_FW_UPLOAD is not set
# end of Firmware loader
CONFIG_ALLOW_DEV_COREDUMP=y
CONFIG_GENERIC_CPU_AUTOPROBE=y
CONFIG_GENERIC_CPU_VULNERABILITIES=y
# end of Generic Driver Options
#
# Bus devices
#
# CONFIG_ARM_INTEGRATOR_LM is not set
# CONFIG_BT1_APB is not set
# CONFIG_BT1_AXI is not set
# CONFIG_HISILICON_LPC is not set
# CONFIG_INTEL_IXP4XX_EB is not set
# CONFIG_QCOM_EBI2 is not set
# CONFIG_MHI_BUS is not set
# CONFIG_MHI_BUS_EP is not set
# end of Bus devices
#
# Firmware Drivers
#
#
# ARM System Control and Management Interface Protocol
#
# CONFIG_ARM_SCMI_PROTOCOL is not set
# end of ARM System Control and Management Interface Protocol
# CONFIG_EDD is not set
CONFIG_FIRMWARE_MEMMAP=y
# CONFIG_DMIID is not set
# CONFIG_DMI_SYSFS is not set
CONFIG_DMI_SCAN_MACHINE_NON_EFI_FALLBACK=y
# CONFIG_FW_CFG_SYSFS is not set
# CONFIG_SYSFB_SIMPLEFB is not set
# CONFIG_BCM47XX_NVRAM is not set
# CONFIG_GOOGLE_FIRMWARE is not set
#
# Tegra firmware driver
#
# end of Tegra firmware driver
# end of Firmware Drivers
# CONFIG_GNSS is not set
# CONFIG_MTD is not set
# CONFIG_OF is not set
CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y
# CONFIG_PARPORT is not set
# CONFIG_BLK_DEV is not set
#
# NVME Support
#
# CONFIG_NVME_FC is not set
# end of NVME Support
#
# Misc devices
#
# CONFIG_DUMMY_IRQ is not set
# CONFIG_ATMEL_SSC is not set
# CONFIG_ENCLOSURE_SERVICES is not set
# CONFIG_QCOM_COINCELL is not set
# CONFIG_SRAM is not set
# CONFIG_XILINX_SDFEC is not set
# CONFIG_C2PORT is not set
#
# EEPROM support
#
# CONFIG_EEPROM_93CX6 is not set
# end of EEPROM support
#
# Texas Instruments shared transport line discipline
#
# end of Texas Instruments shared transport line discipline
#
# Altera FPGA firmware download module (requires I2C)
#
# CONFIG_ECHO is not set
# CONFIG_PVPANIC is not set
# end of Misc devices
#
# SCSI device support
#
CONFIG_SCSI_MOD=y
# CONFIG_RAID_ATTRS is not set
# CONFIG_SCSI is not set
# end of SCSI device support
# CONFIG_ATA is not set
# CONFIG_MD is not set
# CONFIG_TARGET_CORE is not set
#
# IEEE 1394 (FireWire) support
#
# CONFIG_FIREWIRE is not set
# end of IEEE 1394 (FireWire) support
# CONFIG_MACINTOSH_DRIVERS is not set
#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_FF_MEMLESS is not set
# CONFIG_INPUT_SPARSEKMAP is not set
# CONFIG_INPUT_MATRIXKMAP is not set
#
# Userland interfaces
#
# CONFIG_INPUT_MOUSEDEV is not set
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set
#
# Input Device Drivers
#
# CONFIG_INPUT_KEYBOARD is not set
# CONFIG_INPUT_MOUSE is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set
# CONFIG_RMI4_CORE is not set
#
# Hardware I/O ports
#
# CONFIG_SERIO is not set
CONFIG_ARCH_MIGHT_HAVE_PC_SERIO=y
# CONFIG_GAMEPORT is not set
# end of Hardware I/O ports
# end of Input device support
#
# Character devices
#
CONFIG_TTY=y
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_VT_HW_CONSOLE_BINDING is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_LDISC_AUTOLOAD is not set
#
# Serial drivers
#
# CONFIG_SERIAL_8250 is not set
#
# Non-8250 serial port support
#
# CONFIG_SERIAL_AMBA_PL010 is not set
# CONFIG_SERIAL_MESON is not set
# CONFIG_SERIAL_CLPS711X is not set
# CONFIG_SERIAL_SAMSUNG is not set
# CONFIG_SERIAL_TEGRA is not set
# CONFIG_SERIAL_IMX is not set
# CONFIG_SERIAL_UARTLITE is not set
# CONFIG_SERIAL_SH_SCI is not set
# CONFIG_SERIAL_MSM is not set
# CONFIG_SERIAL_VT8500 is not set
# CONFIG_SERIAL_OMAP is not set
# CONFIG_SERIAL_LANTIQ is not set
# CONFIG_SERIAL_SCCNXP is not set
# CONFIG_SERIAL_TIMBERDALE is not set
# CONFIG_SERIAL_BCM63XX is not set
# CONFIG_SERIAL_ALTERA_JTAGUART is not set
# CONFIG_SERIAL_ALTERA_UART is not set
# CONFIG_SERIAL_MXS_AUART is not set
# CONFIG_SERIAL_MPS2_UART is not set
# CONFIG_SERIAL_ARC is not set
# CONFIG_SERIAL_FSL_LPUART is not set
# CONFIG_SERIAL_FSL_LINFLEXUART is not set
# CONFIG_SERIAL_ST_ASC is not set
# CONFIG_SERIAL_STM32 is not set
# CONFIG_SERIAL_OWL is not set
# CONFIG_SERIAL_RDA is not set
# CONFIG_SERIAL_LITEUART is not set
# CONFIG_SERIAL_SUNPLUS is not set
# end of Serial drivers
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_NULL_TTY is not set
# CONFIG_SERIAL_DEV_BUS is not set
# CONFIG_VIRTIO_CONSOLE is not set
# CONFIG_IPMI_HANDLER is not set
# CONFIG_ASPEED_KCS_IPMI_BMC is not set
# CONFIG_NPCM7XX_KCS_IPMI_BMC is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_MWAVE is not set
# CONFIG_DEVMEM is not set
# CONFIG_NVRAM is not set
# CONFIG_HANGCHECK_TIMER is not set
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
# CONFIG_RANDOM_TRUST_CPU is not set
# CONFIG_RANDOM_TRUST_BOOTLOADER is not set
# end of Character devices
#
# I2C support
#
# CONFIG_I2C is not set
# end of I2C support
# CONFIG_I3C is not set
# CONFIG_SPI is not set
# CONFIG_SPMI is not set
# CONFIG_HSI is not set
# CONFIG_PPS is not set
#
# PTP clock support
#
CONFIG_PTP_1588_CLOCK_OPTIONAL=y
#
# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
#
# end of PTP clock support
# CONFIG_PINCTRL is not set
# CONFIG_GPIOLIB is not set
# CONFIG_W1 is not set
# CONFIG_POWER_RESET is not set
# CONFIG_POWER_SUPPLY is not set
# CONFIG_HWMON is not set
# CONFIG_THERMAL is not set
# CONFIG_WATCHDOG is not set
CONFIG_SSB_POSSIBLE=y
# CONFIG_SSB is not set
CONFIG_BCMA_POSSIBLE=y
# CONFIG_BCMA is not set
#
# Multifunction device drivers
#
# CONFIG_MFD_SUN4I_GPADC is not set
# CONFIG_MFD_AT91_USART is not set
# CONFIG_MFD_MADERA is not set
# CONFIG_MFD_EXYNOS_LPASS is not set
# CONFIG_MFD_MXS_LRADC is not set
# CONFIG_MFD_MX25_TSADC is not set
# CONFIG_HTC_PASIC3 is not set
# CONFIG_MFD_KEMPLD is not set
# CONFIG_MFD_MT6397 is not set
# CONFIG_MFD_PM8XXX is not set
# CONFIG_MFD_SM501 is not set
# CONFIG_ABX500_CORE is not set
# CONFIG_MFD_SUN6I_PRCM is not set
# CONFIG_MFD_SYSCON is not set
# CONFIG_MFD_TI_AM335X_TSCADC is not set
# CONFIG_MFD_TQMX86 is not set
# CONFIG_MFD_STM32_LPTIMER is not set
# CONFIG_MFD_STM32_TIMERS is not set
# end of Multifunction device drivers
# CONFIG_REGULATOR is not set
# CONFIG_RC_CORE is not set
#
# CEC support
#
# CONFIG_MEDIA_CEC_SUPPORT is not set
# end of CEC support
# CONFIG_MEDIA_SUPPORT is not set
#
# Graphics support
#
# CONFIG_IMX_IPUV3_CORE is not set
# CONFIG_DRM is not set
#
# ARM devices
#
# end of ARM devices
#
# Frame buffer Devices
#
# CONFIG_FB is not set
# CONFIG_MMP_DISP is not set
# end of Frame buffer Devices
#
# Backlight & LCD device support
#
# CONFIG_LCD_CLASS_DEVICE is not set
# CONFIG_BACKLIGHT_CLASS_DEVICE is not set
# end of Backlight & LCD device support
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y
CONFIG_DUMMY_CONSOLE_COLUMNS=80
CONFIG_DUMMY_CONSOLE_ROWS=25
# end of Console display driver support
# end of Graphics support
# CONFIG_SOUND is not set
#
# HID support
#
# CONFIG_HID is not set
# end of HID support
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
# CONFIG_USB_SUPPORT is not set
# CONFIG_MMC is not set
# CONFIG_MEMSTICK is not set
# CONFIG_NEW_LEDS is not set
# CONFIG_ACCESSIBILITY is not set
CONFIG_EDAC_ATOMIC_SCRUB=y
CONFIG_EDAC_SUPPORT=y
CONFIG_RTC_LIB=y
CONFIG_RTC_MC146818_LIB=y
# CONFIG_RTC_CLASS is not set
# CONFIG_DMADEVICES is not set
#
# DMABUF options
#
# CONFIG_SYNC_FILE is not set
# CONFIG_DMABUF_HEAPS is not set
# end of DMABUF options
# CONFIG_AUXDISPLAY is not set
# CONFIG_UIO is not set
# CONFIG_VFIO is not set
# CONFIG_VIRT_DRIVERS is not set
# CONFIG_VIRTIO_MENU is not set
# CONFIG_VHOST_MENU is not set
#
# Microsoft Hyper-V guest support
#
# end of Microsoft Hyper-V guest support
# CONFIG_GREYBUS is not set
# CONFIG_COMEDI is not set
# CONFIG_STAGING is not set
# CONFIG_CHROME_PLATFORMS is not set
# CONFIG_MELLANOX_PLATFORM is not set
# CONFIG_OLPC_XO175 is not set
# CONFIG_SURFACE_PLATFORMS is not set
# CONFIG_X86_PLATFORM_DEVICES is not set
# CONFIG_COMMON_CLK is not set
# CONFIG_HWSPINLOCK is not set
#
# Clock Source drivers
#
CONFIG_CLKEVT_I8253=y
CONFIG_I8253_LOCK=y
CONFIG_CLKBLD_I8253=y
# CONFIG_BCM2835_TIMER is not set
# CONFIG_BCM_KONA_TIMER is not set
# CONFIG_DAVINCI_TIMER is not set
# CONFIG_DIGICOLOR_TIMER is not set
# CONFIG_OMAP_DM_TIMER is not set
# CONFIG_DW_APB_TIMER is not set
# CONFIG_FTTMR010_TIMER is not set
# CONFIG_IXP4XX_TIMER is not set
# CONFIG_MESON6_TIMER is not set
# CONFIG_OWL_TIMER is not set
# CONFIG_RDA_TIMER is not set
# CONFIG_SUN4I_TIMER is not set
# CONFIG_TEGRA_TIMER is not set
# CONFIG_VT8500_TIMER is not set
# CONFIG_NPCM7XX_TIMER is not set
# CONFIG_ASM9260_TIMER is not set
# CONFIG_CLKSRC_DBX500_PRCMU is not set
# CONFIG_CLPS711X_TIMER is not set
# CONFIG_MXS_TIMER is not set
# CONFIG_NSPIRE_TIMER is not set
# CONFIG_INTEGRATOR_AP_TIMER is not set
# CONFIG_CLKSRC_PISTACHIO is not set
# CONFIG_CLKSRC_STM32_LP is not set
# CONFIG_ARMV7M_SYSTICK is not set
# CONFIG_ATMEL_PIT is not set
# CONFIG_ATMEL_ST is not set
# CONFIG_CLKSRC_SAMSUNG_PWM is not set
# CONFIG_FSL_FTM_TIMER is not set
# CONFIG_OXNAS_RPS_TIMER is not set
# CONFIG_MTK_TIMER is not set
# CONFIG_SH_TIMER_CMT is not set
# CONFIG_SH_TIMER_MTU2 is not set
# CONFIG_RENESAS_OSTM is not set
# CONFIG_SH_TIMER_TMU is not set
# CONFIG_EM_TIMER_STI is not set
# CONFIG_CLKSRC_PXA is not set
# CONFIG_TIMER_IMX_SYS_CTR is not set
# CONFIG_CLKSRC_ST_LPC is not set
# CONFIG_GXP_TIMER is not set
# CONFIG_MSC313E_TIMER is not set
# CONFIG_MICROCHIP_PIT64B is not set
# end of Clock Source drivers
# CONFIG_MAILBOX is not set
# CONFIG_IOMMU_SUPPORT is not set
#
# Remoteproc drivers
#
# CONFIG_REMOTEPROC is not set
# end of Remoteproc drivers
#
# Rpmsg drivers
#
# CONFIG_RPMSG_VIRTIO is not set
# end of Rpmsg drivers
#
# SOC (System On Chip) specific Drivers
#
#
# Amlogic SoC drivers
#
# CONFIG_MESON_CANVAS is not set
# CONFIG_MESON_CLK_MEASURE is not set
# CONFIG_MESON_GX_SOCINFO is not set
# CONFIG_MESON_MX_SOCINFO is not set
# end of Amlogic SoC drivers
#
# Apple SoC drivers
#
# CONFIG_APPLE_SART is not set
# end of Apple SoC drivers
#
# ASPEED SoC drivers
#
# CONFIG_ASPEED_LPC_CTRL is not set
# CONFIG_ASPEED_LPC_SNOOP is not set
# CONFIG_ASPEED_UART_ROUTING is not set
# CONFIG_ASPEED_P2A_CTRL is not set
# CONFIG_ASPEED_SOCINFO is not set
# end of ASPEED SoC drivers
# CONFIG_AT91_SOC_ID is not set
# CONFIG_AT91_SOC_SFR is not set
#
# Broadcom SoC drivers
#
# CONFIG_SOC_BCM63XX is not set
# CONFIG_SOC_BRCMSTB is not set
# end of Broadcom SoC drivers
#
# NXP/Freescale QorIQ SoC drivers
#
# end of NXP/Freescale QorIQ SoC drivers
#
# fujitsu SoC drivers
#
# end of fujitsu SoC drivers
#
# i.MX SoC drivers
#
# CONFIG_SOC_IMX8M is not set
# CONFIG_SOC_IMX9 is not set
# end of i.MX SoC drivers
#
# IXP4xx SoC drivers
#
# CONFIG_IXP4XX_QMGR is not set
# CONFIG_IXP4XX_NPE is not set
# end of IXP4xx SoC drivers
#
# Enable LiteX SoC Builder specific drivers
#
# CONFIG_LITEX_SOC_CONTROLLER is not set
# end of Enable LiteX SoC Builder specific drivers
#
# MediaTek SoC drivers
#
# CONFIG_MTK_CMDQ is not set
# CONFIG_MTK_DEVAPC is not set
# CONFIG_MTK_INFRACFG is not set
# CONFIG_MTK_MMSYS is not set
# end of MediaTek SoC drivers
#
# Qualcomm SoC drivers
#
# CONFIG_QCOM_GENI_SE is not set
# CONFIG_QCOM_GSBI is not set
# CONFIG_QCOM_LLCC is not set
# CONFIG_QCOM_RPMH is not set
# CONFIG_QCOM_SPM is not set
# CONFIG_QCOM_ICC_BWMON is not set
# end of Qualcomm SoC drivers
# CONFIG_SOC_RENESAS is not set
# CONFIG_ROCKCHIP_GRF is not set
# CONFIG_SOC_SAMSUNG is not set
# CONFIG_SOC_TI is not set
# CONFIG_UX500_SOC_ID is not set
#
# Xilinx SoC drivers
#
# end of Xilinx SoC drivers
# end of SOC (System On Chip) specific Drivers
# CONFIG_PM_DEVFREQ is not set
# CONFIG_EXTCON is not set
# CONFIG_MEMORY is not set
# CONFIG_IIO is not set
# CONFIG_PWM is not set
#
# IRQ chip support
#
# CONFIG_RENESAS_INTC_IRQPIN is not set
# CONFIG_RENESAS_IRQC is not set
# CONFIG_RENESAS_RZA1_IRQC is not set
# CONFIG_RENESAS_RZG2L_IRQC is not set
# CONFIG_SL28CPLD_INTC is not set
# CONFIG_TS4800_IRQ is not set
# CONFIG_INGENIC_TCU_IRQ is not set
# CONFIG_IRQ_UNIPHIER_AIDET is not set
# CONFIG_MESON_IRQ_GPIO is not set
# CONFIG_IMX_IRQSTEER is not set
# CONFIG_IMX_INTMUX is not set
# CONFIG_EXYNOS_IRQ_COMBINER is not set
# CONFIG_MST_IRQ is not set
# CONFIG_MCHP_EIC is not set
# CONFIG_SUNPLUS_SP7021_INTC is not set
# end of IRQ chip support
# CONFIG_IPACK_BUS is not set
# CONFIG_RESET_CONTROLLER is not set
#
# PHY Subsystem
#
# CONFIG_GENERIC_PHY is not set
# CONFIG_PHY_PISTACHIO_USB is not set
# CONFIG_PHY_CAN_TRANSCEIVER is not set
#
# PHY drivers for Broadcom platforms
#
# CONFIG_PHY_BCM63XX_USBH is not set
# CONFIG_BCM_KONA_USB2_PHY is not set
# end of PHY drivers for Broadcom platforms
# CONFIG_PHY_HI6220_USB is not set
# CONFIG_PHY_HI3660_USB is not set
# CONFIG_PHY_HI3670_USB is not set
# CONFIG_PHY_HI3670_PCIE is not set
# CONFIG_PHY_HISTB_COMBPHY is not set
# CONFIG_PHY_HISI_INNO_USB2 is not set
# CONFIG_PHY_PXA_28NM_HSIC is not set
# CONFIG_PHY_PXA_28NM_USB2 is not set
# CONFIG_PHY_PXA_USB is not set
# CONFIG_PHY_MMP3_USB is not set
# CONFIG_PHY_MMP3_HSIC is not set
# CONFIG_PHY_MT7621_PCI is not set
# CONFIG_PHY_RALINK_USB is not set
# CONFIG_PHY_RCAR_GEN3_USB3 is not set
# CONFIG_PHY_ROCKCHIP_DPHY_RX0 is not set
# CONFIG_PHY_ROCKCHIP_PCIE is not set
# CONFIG_PHY_ROCKCHIP_SNPS_PCIE3 is not set
# CONFIG_PHY_EXYNOS_MIPI_VIDEO is not set
# CONFIG_PHY_SAMSUNG_USB2 is not set
# CONFIG_PHY_ST_SPEAR1310_MIPHY is not set
# CONFIG_PHY_ST_SPEAR1340_MIPHY is not set
# CONFIG_PHY_TEGRA194_P2U is not set
# CONFIG_PHY_DA8XX_USB is not set
# CONFIG_OMAP_CONTROL_PHY is not set
# CONFIG_TI_PIPE3 is not set
# CONFIG_PHY_INTEL_KEEMBAY_EMMC is not set
# CONFIG_PHY_INTEL_KEEMBAY_USB is not set
# CONFIG_PHY_INTEL_LGM_EMMC is not set
# CONFIG_PHY_XILINX_ZYNQMP is not set
# end of PHY Subsystem
# CONFIG_POWERCAP is not set
# CONFIG_MCB is not set
#
# Performance monitor support
#
# CONFIG_ARM_CCN is not set
# CONFIG_ARM_CMN is not set
# CONFIG_FSL_IMX8_DDR_PMU is not set
# CONFIG_XGENE_PMU is not set
# CONFIG_ARM_DMC620_PMU is not set
# CONFIG_MARVELL_CN10K_TAD_PMU is not set
# CONFIG_ALIBABA_UNCORE_DRW_PMU is not set
# CONFIG_MARVELL_CN10K_DDR_PMU is not set
# end of Performance monitor support
# CONFIG_RAS is not set
#
# Android
#
# CONFIG_ANDROID_BINDER_IPC is not set
# end of Android
# CONFIG_DAX is not set
# CONFIG_NVMEM is not set
#
# HW tracing support
#
# CONFIG_STM is not set
# CONFIG_INTEL_TH is not set
# end of HW tracing support
# CONFIG_FPGA is not set
# CONFIG_TEE is not set
# CONFIG_SIOX is not set
# CONFIG_SLIMBUS is not set
# CONFIG_INTERCONNECT is not set
# CONFIG_COUNTER is not set
# CONFIG_PECI is not set
# CONFIG_HTE is not set
# end of Device Drivers
#
# File systems
#
CONFIG_DCACHE_WORD_ACCESS=y
# CONFIG_VALIDATE_FS_PARSER is not set
# CONFIG_EXT2_FS is not set
# CONFIG_EXT3_FS is not set
# CONFIG_EXT4_FS is not set
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_BTRFS_FS is not set
# CONFIG_NILFS2_FS is not set
# CONFIG_F2FS_FS is not set
CONFIG_EXPORTFS=y
# CONFIG_EXPORTFS_BLOCK_OPS is not set
CONFIG_FILE_LOCKING=y
# CONFIG_FS_ENCRYPTION is not set
# CONFIG_FS_VERITY is not set
# CONFIG_DNOTIFY is not set
# CONFIG_INOTIFY_USER is not set
# CONFIG_FANOTIFY is not set
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_FUSE_FS is not set
# CONFIG_OVERLAY_FS is not set
#
# Caches
#
# CONFIG_FSCACHE is not set
# end of Caches
#
# CD-ROM/DVD Filesystems
#
# CONFIG_ISO9660_FS is not set
# CONFIG_UDF_FS is not set
# end of CD-ROM/DVD Filesystems
#
# DOS/FAT/EXFAT/NT Filesystems
#
# CONFIG_MSDOS_FS is not set
# CONFIG_VFAT_FS is not set
# CONFIG_EXFAT_FS is not set
# CONFIG_NTFS_FS is not set
# CONFIG_NTFS3_FS is not set
# end of DOS/FAT/EXFAT/NT Filesystems
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
# CONFIG_PROC_KCORE is not set
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
# CONFIG_PROC_CHILDREN is not set
CONFIG_PROC_PID_ARCH_STATUS=y
CONFIG_KERNFS=y
CONFIG_SYSFS=y
# CONFIG_TMPFS is not set
# CONFIG_HUGETLBFS is not set
CONFIG_ARCH_WANT_HUGETLB_PAGE_OPTIMIZE_VMEMMAP=y
CONFIG_ARCH_HAS_GIGANTIC_PAGE=y
# CONFIG_CONFIGFS_FS is not set
# end of Pseudo filesystems
# CONFIG_MISC_FILESYSTEMS is not set
# CONFIG_NLS is not set
# CONFIG_UNICODE is not set
CONFIG_IO_WQ=y
# end of File systems
#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY_DMESG_RESTRICT is not set
# CONFIG_SECURITY is not set
# CONFIG_SECURITYFS is not set
CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
# CONFIG_HARDENED_USERCOPY is not set
# CONFIG_FORTIFY_SOURCE is not set
# CONFIG_STATIC_USERMODEHELPER is not set
CONFIG_DEFAULT_SECURITY_DAC=y
CONFIG_LSM="landlock,lockdown,yama,loadpin,safesetid,integrity,bpf"
#
# Kernel hardening options
#
#
# Memory initialization
#
CONFIG_INIT_STACK_NONE=y
# CONFIG_INIT_ON_ALLOC_DEFAULT_ON is not set
# CONFIG_INIT_ON_FREE_DEFAULT_ON is not set
CONFIG_CC_HAS_ZERO_CALL_USED_REGS=y
# CONFIG_ZERO_CALL_USED_REGS is not set
# end of Memory initialization
CONFIG_RANDSTRUCT_NONE=y
# end of Kernel hardening options
# end of Security options
# CONFIG_CRYPTO is not set
#
# Library routines
#
# CONFIG_PACKING is not set
CONFIG_BITREVERSE=y
CONFIG_GENERIC_STRNCPY_FROM_USER=y
CONFIG_GENERIC_STRNLEN_USER=y
# CONFIG_CORDIC is not set
# CONFIG_PRIME_NUMBERS is not set
CONFIG_GENERIC_PCI_IOMAP=y
CONFIG_GENERIC_IOMAP=y
CONFIG_ARCH_USE_CMPXCHG_LOCKREF=y
CONFIG_ARCH_HAS_FAST_MULTIPLIER=y
CONFIG_ARCH_USE_SYM_ANNOTATIONS=y
#
# Crypto library routines
#
CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC=y
# CONFIG_CRYPTO_LIB_CHACHA is not set
# CONFIG_CRYPTO_LIB_CURVE25519 is not set
CONFIG_CRYPTO_LIB_POLY1305_RSIZE=11
# CONFIG_CRYPTO_LIB_POLY1305 is not set
# end of Crypto library routines
# CONFIG_CRC_CCITT is not set
# CONFIG_CRC16 is not set
# CONFIG_CRC_T10DIF is not set
# CONFIG_CRC64_ROCKSOFT is not set
# CONFIG_CRC_ITU_T is not set
CONFIG_CRC32=y
# CONFIG_CRC32_SELFTEST is not set
CONFIG_CRC32_SLICEBY8=y
# CONFIG_CRC32_SLICEBY4 is not set
# CONFIG_CRC32_SARWATE is not set
# CONFIG_CRC32_BIT is not set
# CONFIG_CRC64 is not set
# CONFIG_CRC4 is not set
# CONFIG_CRC7 is not set
# CONFIG_LIBCRC32C is not set
# CONFIG_CRC8 is not set
# CONFIG_RANDOM32_SELFTEST is not set
# CONFIG_XZ_DEC is not set
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT_MAP=y
CONFIG_HAS_DMA=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_ARCH_DMA_ADDR_T_64BIT=y
CONFIG_SWIOTLB=y
# CONFIG_DMA_API_DEBUG is not set
# CONFIG_IRQ_POLL is not set
CONFIG_HAVE_GENERIC_VDSO=y
CONFIG_GENERIC_GETTIMEOFDAY=y
CONFIG_GENERIC_VDSO_TIME_NS=y
CONFIG_ARCH_HAS_PMEM_API=y
CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE=y
CONFIG_ARCH_HAS_COPY_MC=y
CONFIG_ARCH_STACKWALK=y
CONFIG_STACKDEPOT=y
CONFIG_SBITMAP=y
# CONFIG_PARMAN is not set
# CONFIG_OBJAGG is not set
# end of Library routines
#
# Kernel hacking
#
#
# printk and dmesg options
#
# CONFIG_PRINTK_TIME is not set
# CONFIG_PRINTK_CALLER is not set
# CONFIG_STACKTRACE_BUILD_ID is not set
CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7
CONFIG_CONSOLE_LOGLEVEL_QUIET=4
CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
# CONFIG_DYNAMIC_DEBUG is not set
# CONFIG_DYNAMIC_DEBUG_CORE is not set
# CONFIG_SYMBOLIC_ERRNAME is not set
CONFIG_DEBUG_BUGVERBOSE=y
# end of printk and dmesg options
# CONFIG_DEBUG_KERNEL is not set
#
# Compile-time checks and compiler options
#
CONFIG_AS_HAS_NON_CONST_LEB128=y
CONFIG_FRAME_WARN=2048
# CONFIG_STRIP_ASM_SYMS is not set
# CONFIG_HEADERS_INSTALL is not set
CONFIG_DEBUG_SECTION_MISMATCH=y
CONFIG_SECTION_MISMATCH_WARN_ONLY=y
CONFIG_OBJTOOL=y
# end of Compile-time checks and compiler options
#
# Generic Kernel Debugging Instruments
#
# CONFIG_MAGIC_SYSRQ is not set
# CONFIG_DEBUG_FS is not set
CONFIG_HAVE_ARCH_KGDB=y
CONFIG_ARCH_HAS_UBSAN_SANITIZE_ALL=y
# CONFIG_UBSAN is not set
CONFIG_HAVE_ARCH_KCSAN=y
CONFIG_HAVE_KCSAN_COMPILER=y
# end of Generic Kernel Debugging Instruments
#
# Networking Debugging
#
# end of Networking Debugging
#
# Memory Debugging
#
# CONFIG_PAGE_EXTENSION is not set
CONFIG_SLUB_DEBUG=y
# CONFIG_SLUB_DEBUG_ON is not set
# CONFIG_PAGE_TABLE_CHECK is not set
# CONFIG_PAGE_POISONING is not set
# CONFIG_DEBUG_RODATA_TEST is not set
CONFIG_ARCH_HAS_DEBUG_WX=y
# CONFIG_DEBUG_WX is not set
CONFIG_GENERIC_PTDUMP=y
CONFIG_HAVE_DEBUG_KMEMLEAK=y
CONFIG_ARCH_HAS_DEBUG_VM_PGTABLE=y
# CONFIG_DEBUG_VM_PGTABLE is not set
CONFIG_ARCH_HAS_DEBUG_VIRTUAL=y
CONFIG_DEBUG_MEMORY_INIT=y
CONFIG_ARCH_SUPPORTS_KMAP_LOCAL_FORCE_MAP=y
CONFIG_HAVE_ARCH_KASAN=y
CONFIG_HAVE_ARCH_KASAN_VMALLOC=y
CONFIG_CC_HAS_KASAN_GENERIC=y
CONFIG_CC_HAS_WORKING_NOSANITIZE_ADDRESS=y
# CONFIG_KASAN is not set
CONFIG_HAVE_ARCH_KFENCE=y
# CONFIG_KFENCE is not set
CONFIG_HAVE_ARCH_KMSAN=y
# end of Memory Debugging
#
# Debug Oops, Lockups and Hangs
#
# CONFIG_PANIC_ON_OOPS is not set
CONFIG_PANIC_ON_OOPS_VALUE=0
CONFIG_PANIC_TIMEOUT=0
CONFIG_HARDLOCKUP_CHECK_TIMESTAMP=y
# end of Debug Oops, Lockups and Hangs
#
# Scheduler Debugging
#
# end of Scheduler Debugging
# CONFIG_DEBUG_TIMEKEEPING is not set
#
# Lock Debugging (spinlocks, mutexes, etc...)
#
CONFIG_LOCK_DEBUGGING_SUPPORT=y
# CONFIG_WW_MUTEX_SELFTEST is not set
# end of Lock Debugging (spinlocks, mutexes, etc...)
# CONFIG_DEBUG_IRQFLAGS is not set
CONFIG_STACKTRACE=y
# CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set
#
# Debug kernel data structures
#
# CONFIG_BUG_ON_DATA_CORRUPTION is not set
# end of Debug kernel data structures
#
# RCU Debugging
#
# end of RCU Debugging
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_HAVE_RETHOOK=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS=y
CONFIG_HAVE_DYNAMIC_FTRACE_NO_PATCHABLE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_FENTRY=y
CONFIG_HAVE_OBJTOOL_MCOUNT=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_HAVE_BUILDTIME_MCOUNT_SORT=y
CONFIG_TRACING_SUPPORT=y
# CONFIG_FTRACE is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_SAMPLE_FTRACE_DIRECT=y
CONFIG_HAVE_SAMPLE_FTRACE_DIRECT_MULTI=y
CONFIG_ARCH_HAS_DEVMEM_IS_ALLOWED=y
#
# x86 Debugging
#
# CONFIG_X86_VERBOSE_BOOTUP is not set
CONFIG_EARLY_PRINTK=y
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_UNWINDER_ORC=y
# CONFIG_UNWINDER_FRAME_POINTER is not set
# end of x86 Debugging
#
# Kernel Testing and Coverage
#
# CONFIG_KUNIT is not set
CONFIG_ARCH_HAS_KCOV=y
CONFIG_CC_HAS_SANCOV_TRACE_PC=y
# CONFIG_KCOV is not set
# CONFIG_RUNTIME_TESTING_MENU is not set
CONFIG_ARCH_USE_MEMTEST=y
# CONFIG_MEMTEST is not set
# end of Kernel Testing and Coverage
#
# Rust hacking
#
# end of Rust hacking
CONFIG_WARN_MISSING_DOCUMENTS=y
CONFIG_WARN_ABI_ERRORS=y
# end of Kernel hacking
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 01/25] Documentation: kvx: Add basic documentation
2023-01-03 16:43 ` [RFC PATCH 01/25] Documentation: kvx: Add basic documentation Yann Sionneau
2023-01-03 17:50 ` Jonathan Corbet
2023-01-05 18:38 ` kernel test robot
@ 2023-01-18 15:09 ` Jeff Xie
2 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-18 15:09 UTC (permalink / raw)
To: Yann Sionneau
Cc: Jonathan Corbet, linux-doc, linux-kernel, Clement Leger,
Guillaume Thouvenin
On Wed, Jan 4, 2023 at 1:04 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
>
> Add some documentation for kvx arch and its Linux port.
>
> CC: Jonathan Corbet <corbet@lwn.net>
> CC: linux-doc@vger.kernel.org
> CC: linux-kernel@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> Documentation/kvx/kvx-exceptions.txt | 246 ++++++++++++++++++++++++
> Documentation/kvx/kvx-iommu.txt | 183 ++++++++++++++++++
> Documentation/kvx/kvx-mmu.txt | 272 +++++++++++++++++++++++++++
> Documentation/kvx/kvx-smp.txt | 36 ++++
> Documentation/kvx/kvx.txt | 268 ++++++++++++++++++++++++++
> 5 files changed, 1005 insertions(+)
> create mode 100644 Documentation/kvx/kvx-exceptions.txt
> create mode 100644 Documentation/kvx/kvx-iommu.txt
> create mode 100644 Documentation/kvx/kvx-mmu.txt
> create mode 100644 Documentation/kvx/kvx-smp.txt
> create mode 100644 Documentation/kvx/kvx.txt
[...]
> +Virtual and physical memory
> +===========================
> +
> +The mapping used and the memory management is described in
> +Documentation/kvx/kvx-mmu.txt.
> +Our Kernel is compiled using virtual addresses that starts at
> +0xffffff0000000000. But when it is started the kernel uses physical addresses.
> +Before calling the first function arch_low_level_start() we configure 2 entries
> +of the LTLB.
> +
> +The first entry will map the first 1G of virtual address space to the first
> +1G of DDR:
> + - TLB[0]: 0xffffff0000000000 -> 0x100000000 (size 512Mo)
Maybe should change it to:
- TLB[0]: 0xffffff8000000000 -> 0x100000000 (size 512Mo)
As the PAGE_OFFSET = 0xffffff8000000000
[...]
--
Thanks,
JeffXie
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 02/25] kvx: Add ELF-related definitions
2023-01-03 16:43 ` Yann Sionneau
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Paul Moore, Eric Paris, Eric Biederman, Kees Cook,
linux-audit, linux-kernel, linux-mm, Clement Leger
Add ELF-related definitions for kvx, including: EM_KVX,
AUDIT_ARCH_KVX and NT_KVX_TCA.
CC: Paul Moore <paul@paul-moore.com>
CC: Eric Paris <eparis@redhat.com>
CC: Eric Biederman <ebiederm@xmission.com>
CC: Kees Cook <keescook@chromium.org>
CC: linux-audit@redhat.com
CC: linux-kernel@vger.kernel.org
CC: linux-mm@kvack.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
include/uapi/linux/audit.h | 1 +
include/uapi/linux/elf-em.h | 1 +
include/uapi/linux/elf.h | 1 +
3 files changed, 3 insertions(+)
diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h
index d676ed2b246e..4db7aa3f84c7 100644
--- a/include/uapi/linux/audit.h
+++ b/include/uapi/linux/audit.h
@@ -402,6 +402,7 @@ enum {
#define AUDIT_ARCH_HEXAGON (EM_HEXAGON)
#define AUDIT_ARCH_I386 (EM_386|__AUDIT_ARCH_LE)
#define AUDIT_ARCH_IA64 (EM_IA_64|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE)
+#define AUDIT_ARCH_KVX (EM_KVX|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE)
#define AUDIT_ARCH_M32R (EM_M32R)
#define AUDIT_ARCH_M68K (EM_68K)
#define AUDIT_ARCH_MICROBLAZE (EM_MICROBLAZE)
diff --git a/include/uapi/linux/elf-em.h b/include/uapi/linux/elf-em.h
index ef38c2bc5ab7..9cc348be7f86 100644
--- a/include/uapi/linux/elf-em.h
+++ b/include/uapi/linux/elf-em.h
@@ -51,6 +51,7 @@
#define EM_RISCV 243 /* RISC-V */
#define EM_BPF 247 /* Linux BPF - in-kernel virtual machine */
#define EM_CSKY 252 /* C-SKY */
+#define EM_KVX 256 /* Kalray VLIW Architecture */
#define EM_LOONGARCH 258 /* LoongArch */
#define EM_FRV 0x5441 /* Fujitsu FR-V */
diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
index c7b056af9ef0..49094f3be06c 100644
--- a/include/uapi/linux/elf.h
+++ b/include/uapi/linux/elf.h
@@ -444,6 +444,7 @@ typedef struct elf64_shdr {
#define NT_LOONGARCH_LSX 0xa02 /* LoongArch Loongson SIMD Extension registers */
#define NT_LOONGARCH_LASX 0xa03 /* LoongArch Loongson Advanced SIMD Extension registers */
#define NT_LOONGARCH_LBT 0xa04 /* LoongArch Loongson Binary Translation registers */
+#define NT_KVX_TCA 0x900 /* kvx TCA registers */
/* Note types with note name "GNU" */
#define NT_GNU_PROPERTY_TYPE_0 5
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 02/25] kvx: Add ELF-related definitions
@ 2023-01-03 16:43 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Kees Cook, Yann Sionneau, linux-kernel, Eric Paris, linux-mm,
linux-audit, Eric Biederman, Clement Leger
Add ELF-related definitions for kvx, including: EM_KVX,
AUDIT_ARCH_KVX and NT_KVX_TCA.
CC: Paul Moore <paul@paul-moore.com>
CC: Eric Paris <eparis@redhat.com>
CC: Eric Biederman <ebiederm@xmission.com>
CC: Kees Cook <keescook@chromium.org>
CC: linux-audit@redhat.com
CC: linux-kernel@vger.kernel.org
CC: linux-mm@kvack.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
include/uapi/linux/audit.h | 1 +
include/uapi/linux/elf-em.h | 1 +
include/uapi/linux/elf.h | 1 +
3 files changed, 3 insertions(+)
diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h
index d676ed2b246e..4db7aa3f84c7 100644
--- a/include/uapi/linux/audit.h
+++ b/include/uapi/linux/audit.h
@@ -402,6 +402,7 @@ enum {
#define AUDIT_ARCH_HEXAGON (EM_HEXAGON)
#define AUDIT_ARCH_I386 (EM_386|__AUDIT_ARCH_LE)
#define AUDIT_ARCH_IA64 (EM_IA_64|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE)
+#define AUDIT_ARCH_KVX (EM_KVX|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE)
#define AUDIT_ARCH_M32R (EM_M32R)
#define AUDIT_ARCH_M68K (EM_68K)
#define AUDIT_ARCH_MICROBLAZE (EM_MICROBLAZE)
diff --git a/include/uapi/linux/elf-em.h b/include/uapi/linux/elf-em.h
index ef38c2bc5ab7..9cc348be7f86 100644
--- a/include/uapi/linux/elf-em.h
+++ b/include/uapi/linux/elf-em.h
@@ -51,6 +51,7 @@
#define EM_RISCV 243 /* RISC-V */
#define EM_BPF 247 /* Linux BPF - in-kernel virtual machine */
#define EM_CSKY 252 /* C-SKY */
+#define EM_KVX 256 /* Kalray VLIW Architecture */
#define EM_LOONGARCH 258 /* LoongArch */
#define EM_FRV 0x5441 /* Fujitsu FR-V */
diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
index c7b056af9ef0..49094f3be06c 100644
--- a/include/uapi/linux/elf.h
+++ b/include/uapi/linux/elf.h
@@ -444,6 +444,7 @@ typedef struct elf64_shdr {
#define NT_LOONGARCH_LSX 0xa02 /* LoongArch Loongson SIMD Extension registers */
#define NT_LOONGARCH_LASX 0xa03 /* LoongArch Loongson Advanced SIMD Extension registers */
#define NT_LOONGARCH_LBT 0xa04 /* LoongArch Loongson Binary Translation registers */
+#define NT_KVX_TCA 0x900 /* kvx TCA registers */
/* Note types with note name "GNU" */
#define NT_GNU_PROPERTY_TYPE_0 5
--
2.37.2
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 02/25] kvx: Add ELF-related definitions
@ 2023-01-03 16:43 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Paul Moore, Eric Paris, Eric Biederman, Kees Cook,
linux-audit, linux-kernel, linux-mm, Clement Leger
Add ELF-related definitions for kvx, including: EM_KVX,
AUDIT_ARCH_KVX and NT_KVX_TCA.
CC: Paul Moore <paul@paul-moore.com>
CC: Eric Paris <eparis@redhat.com>
CC: Eric Biederman <ebiederm@xmission.com>
CC: Kees Cook <keescook@chromium.org>
CC: linux-audit@redhat.com
CC: linux-kernel@vger.kernel.org
CC: linux-mm@kvack.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
include/uapi/linux/audit.h | 1 +
include/uapi/linux/elf-em.h | 1 +
include/uapi/linux/elf.h | 1 +
3 files changed, 3 insertions(+)
diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h
index d676ed2b246e..4db7aa3f84c7 100644
--- a/include/uapi/linux/audit.h
+++ b/include/uapi/linux/audit.h
@@ -402,6 +402,7 @@ enum {
#define AUDIT_ARCH_HEXAGON (EM_HEXAGON)
#define AUDIT_ARCH_I386 (EM_386|__AUDIT_ARCH_LE)
#define AUDIT_ARCH_IA64 (EM_IA_64|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE)
+#define AUDIT_ARCH_KVX (EM_KVX|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE)
#define AUDIT_ARCH_M32R (EM_M32R)
#define AUDIT_ARCH_M68K (EM_68K)
#define AUDIT_ARCH_MICROBLAZE (EM_MICROBLAZE)
diff --git a/include/uapi/linux/elf-em.h b/include/uapi/linux/elf-em.h
index ef38c2bc5ab7..9cc348be7f86 100644
--- a/include/uapi/linux/elf-em.h
+++ b/include/uapi/linux/elf-em.h
@@ -51,6 +51,7 @@
#define EM_RISCV 243 /* RISC-V */
#define EM_BPF 247 /* Linux BPF - in-kernel virtual machine */
#define EM_CSKY 252 /* C-SKY */
+#define EM_KVX 256 /* Kalray VLIW Architecture */
#define EM_LOONGARCH 258 /* LoongArch */
#define EM_FRV 0x5441 /* Fujitsu FR-V */
diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
index c7b056af9ef0..49094f3be06c 100644
--- a/include/uapi/linux/elf.h
+++ b/include/uapi/linux/elf.h
@@ -444,6 +444,7 @@ typedef struct elf64_shdr {
#define NT_LOONGARCH_LSX 0xa02 /* LoongArch Loongson SIMD Extension registers */
#define NT_LOONGARCH_LASX 0xa03 /* LoongArch Loongson Advanced SIMD Extension registers */
#define NT_LOONGARCH_LBT 0xa04 /* LoongArch Loongson Binary Translation registers */
+#define NT_KVX_TCA 0x900 /* kvx TCA registers */
/* Note types with note name "GNU" */
#define NT_GNU_PROPERTY_TYPE_0 5
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 02/25] kvx: Add ELF-related definitions
2023-01-03 16:43 ` Yann Sionneau
@ 2023-01-03 21:35 ` Eric W. Biederman
-1 siblings, 0 replies; 113+ messages in thread
From: Eric W. Biederman @ 2023-01-03 21:35 UTC (permalink / raw)
To: Yann Sionneau
Cc: Paul Moore, Eric Paris, Kees Cook, linux-audit, linux-kernel,
linux-mm, Clement Leger
Yann Sionneau <ysionneau@kalray.eu> writes:
> Add ELF-related definitions for kvx, including: EM_KVX,
> AUDIT_ARCH_KVX and NT_KVX_TCA.
Has someone written an SYSVABI architecture specification for
your architecture?
I feel uncomfortable with the linux-kernel headers being the
authoritative place for the ELF abi definitions.
Especially since the linux kernel does not deal with relocations,
and the kernel headers could diverge from the real world and no one
would notice..
I know at least at one point the linux standards base was taking
up the work on collecting up some of these definitions. I would
be happy if there was anything outside of the linux kernel that
people could refer too.
Eric
> CC: Paul Moore <paul@paul-moore.com>
> CC: Eric Paris <eparis@redhat.com>
> CC: Eric Biederman <ebiederm@xmission.com>
> CC: Kees Cook <keescook@chromium.org>
> CC: linux-audit@redhat.com
> CC: linux-kernel@vger.kernel.org
> CC: linux-mm@kvack.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> include/uapi/linux/audit.h | 1 +
> include/uapi/linux/elf-em.h | 1 +
> include/uapi/linux/elf.h | 1 +
> 3 files changed, 3 insertions(+)
>
> diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h
> index d676ed2b246e..4db7aa3f84c7 100644
> --- a/include/uapi/linux/audit.h
> +++ b/include/uapi/linux/audit.h
> @@ -402,6 +402,7 @@ enum {
> #define AUDIT_ARCH_HEXAGON (EM_HEXAGON)
> #define AUDIT_ARCH_I386 (EM_386|__AUDIT_ARCH_LE)
> #define AUDIT_ARCH_IA64 (EM_IA_64|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE)
> +#define AUDIT_ARCH_KVX (EM_KVX|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE)
> #define AUDIT_ARCH_M32R (EM_M32R)
> #define AUDIT_ARCH_M68K (EM_68K)
> #define AUDIT_ARCH_MICROBLAZE (EM_MICROBLAZE)
> diff --git a/include/uapi/linux/elf-em.h b/include/uapi/linux/elf-em.h
> index ef38c2bc5ab7..9cc348be7f86 100644
> --- a/include/uapi/linux/elf-em.h
> +++ b/include/uapi/linux/elf-em.h
> @@ -51,6 +51,7 @@
> #define EM_RISCV 243 /* RISC-V */
> #define EM_BPF 247 /* Linux BPF - in-kernel virtual machine */
> #define EM_CSKY 252 /* C-SKY */
> +#define EM_KVX 256 /* Kalray VLIW Architecture */
> #define EM_LOONGARCH 258 /* LoongArch */
> #define EM_FRV 0x5441 /* Fujitsu FR-V */
>
> diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
> index c7b056af9ef0..49094f3be06c 100644
> --- a/include/uapi/linux/elf.h
> +++ b/include/uapi/linux/elf.h
> @@ -444,6 +444,7 @@ typedef struct elf64_shdr {
> #define NT_LOONGARCH_LSX 0xa02 /* LoongArch Loongson SIMD Extension registers */
> #define NT_LOONGARCH_LASX 0xa03 /* LoongArch Loongson Advanced SIMD Extension registers */
> #define NT_LOONGARCH_LBT 0xa04 /* LoongArch Loongson Binary Translation registers */
> +#define NT_KVX_TCA 0x900 /* kvx TCA registers */
>
> /* Note types with note name "GNU" */
> #define NT_GNU_PROPERTY_TYPE_0 5
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 02/25] kvx: Add ELF-related definitions
@ 2023-01-03 21:35 ` Eric W. Biederman
0 siblings, 0 replies; 113+ messages in thread
From: Eric W. Biederman @ 2023-01-03 21:35 UTC (permalink / raw)
To: Yann Sionneau
Cc: linux-kernel, Eric Paris, linux-mm, linux-audit, Clement Leger,
Kees Cook
Yann Sionneau <ysionneau@kalray.eu> writes:
> Add ELF-related definitions for kvx, including: EM_KVX,
> AUDIT_ARCH_KVX and NT_KVX_TCA.
Has someone written an SYSVABI architecture specification for
your architecture?
I feel uncomfortable with the linux-kernel headers being the
authoritative place for the ELF abi definitions.
Especially since the linux kernel does not deal with relocations,
and the kernel headers could diverge from the real world and no one
would notice..
I know at least at one point the linux standards base was taking
up the work on collecting up some of these definitions. I would
be happy if there was anything outside of the linux kernel that
people could refer too.
Eric
> CC: Paul Moore <paul@paul-moore.com>
> CC: Eric Paris <eparis@redhat.com>
> CC: Eric Biederman <ebiederm@xmission.com>
> CC: Kees Cook <keescook@chromium.org>
> CC: linux-audit@redhat.com
> CC: linux-kernel@vger.kernel.org
> CC: linux-mm@kvack.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> include/uapi/linux/audit.h | 1 +
> include/uapi/linux/elf-em.h | 1 +
> include/uapi/linux/elf.h | 1 +
> 3 files changed, 3 insertions(+)
>
> diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h
> index d676ed2b246e..4db7aa3f84c7 100644
> --- a/include/uapi/linux/audit.h
> +++ b/include/uapi/linux/audit.h
> @@ -402,6 +402,7 @@ enum {
> #define AUDIT_ARCH_HEXAGON (EM_HEXAGON)
> #define AUDIT_ARCH_I386 (EM_386|__AUDIT_ARCH_LE)
> #define AUDIT_ARCH_IA64 (EM_IA_64|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE)
> +#define AUDIT_ARCH_KVX (EM_KVX|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE)
> #define AUDIT_ARCH_M32R (EM_M32R)
> #define AUDIT_ARCH_M68K (EM_68K)
> #define AUDIT_ARCH_MICROBLAZE (EM_MICROBLAZE)
> diff --git a/include/uapi/linux/elf-em.h b/include/uapi/linux/elf-em.h
> index ef38c2bc5ab7..9cc348be7f86 100644
> --- a/include/uapi/linux/elf-em.h
> +++ b/include/uapi/linux/elf-em.h
> @@ -51,6 +51,7 @@
> #define EM_RISCV 243 /* RISC-V */
> #define EM_BPF 247 /* Linux BPF - in-kernel virtual machine */
> #define EM_CSKY 252 /* C-SKY */
> +#define EM_KVX 256 /* Kalray VLIW Architecture */
> #define EM_LOONGARCH 258 /* LoongArch */
> #define EM_FRV 0x5441 /* Fujitsu FR-V */
>
> diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
> index c7b056af9ef0..49094f3be06c 100644
> --- a/include/uapi/linux/elf.h
> +++ b/include/uapi/linux/elf.h
> @@ -444,6 +444,7 @@ typedef struct elf64_shdr {
> #define NT_LOONGARCH_LSX 0xa02 /* LoongArch Loongson SIMD Extension registers */
> #define NT_LOONGARCH_LASX 0xa03 /* LoongArch Loongson Advanced SIMD Extension registers */
> #define NT_LOONGARCH_LBT 0xa04 /* LoongArch Loongson Binary Translation registers */
> +#define NT_KVX_TCA 0x900 /* kvx TCA registers */
>
> /* Note types with note name "GNU" */
> #define NT_GNU_PROPERTY_TYPE_0 5
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 02/25] kvx: Add ELF-related definitions
2023-01-03 21:35 ` Eric W. Biederman
@ 2023-01-18 8:48 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-18 8:48 UTC (permalink / raw)
To: Eric W. Biederman
Cc: Paul Moore, Eric Paris, Kees Cook, linux-audit, linux-kernel,
linux-mm, Clement Leger
On 03/01/2023 22:35, Eric W. Biederman wrote:
> Yann Sionneau <ysionneau@kalray.eu> writes:
>
>> Add ELF-related definitions for kvx, including: EM_KVX,
>> AUDIT_ARCH_KVX and NT_KVX_TCA.
> Has someone written an SYSVABI architecture specification for
> your architecture?
Internally at Kalray we have an ABI document, I started a discussion
about its publication.
It will not be part of the V2, but I'm working on it.
>
> I feel uncomfortable with the linux-kernel headers being the
> authoritative place for the ELF abi definitions.
>
> Especially since the linux kernel does not deal with relocations,
> and the kernel headers could diverge from the real world and no one
> would notice..
>
> I know at least at one point the linux standards base was taking
> up the work on collecting up some of these definitions. I would
> be happy if there was anything outside of the linux kernel that
> people could refer too.
I completely agree with you!
Thanks for the review.
--
Yann
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 02/25] kvx: Add ELF-related definitions
@ 2023-01-18 8:48 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-18 8:48 UTC (permalink / raw)
To: Eric W. Biederman
Cc: linux-kernel, Eric Paris, linux-mm, linux-audit, Clement Leger,
Kees Cook
On 03/01/2023 22:35, Eric W. Biederman wrote:
> Yann Sionneau <ysionneau@kalray.eu> writes:
>
>> Add ELF-related definitions for kvx, including: EM_KVX,
>> AUDIT_ARCH_KVX and NT_KVX_TCA.
> Has someone written an SYSVABI architecture specification for
> your architecture?
Internally at Kalray we have an ABI document, I started a discussion
about its publication.
It will not be part of the V2, but I'm working on it.
>
> I feel uncomfortable with the linux-kernel headers being the
> authoritative place for the ELF abi definitions.
>
> Especially since the linux kernel does not deal with relocations,
> and the kernel headers could diverge from the real world and no one
> would notice..
>
> I know at least at one point the linux standards base was taking
> up the work on collecting up some of these definitions. I would
> be happy if there was anything outside of the linux kernel that
> people could refer too.
I completely agree with you!
Thanks for the review.
--
Yann
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 03/25] kvx: Add build infrastructure
2023-01-03 16:43 ` Yann Sionneau
` (3 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-03 17:29 ` Randy Dunlap
2023-01-06 0:43 ` kernel test robot
-1 siblings, 2 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, bpf, Clement Leger,
Guillaume Thouvenin, Jonathan Borne, Jules Maselbas,
Julian Vetter, Marc Poulhiès, Marius Gligor, Samuel Jones,
Vincent Chardon
Add Kbuild, Makefile, Kconfig and link script for kvx build infrastructure.
CC: linux-kernel@vger.kernel.org
CC: bpf@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Jonathan Borne <jborne@kalray.eu>
Signed-off-by: Jonathan Borne <jborne@kalray.eu>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Marc Poulhiès <dkm@kataplop.net>
Signed-off-by: Marc Poulhiès <dkm@kataplop.net>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Samuel Jones <sjones@kalray.eu>
Signed-off-by: Samuel Jones <sjones@kalray.eu>
Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/Kconfig | 249 +++++++++++++++++++++++++++++++
arch/kvx/Kconfig.debug | 70 +++++++++
arch/kvx/Makefile | 52 +++++++
arch/kvx/include/asm/Kbuild | 20 +++
arch/kvx/include/uapi/asm/Kbuild | 1 +
arch/kvx/kernel/Makefile | 27 ++++
arch/kvx/kernel/kvx_ksyms.c | 24 +++
arch/kvx/kernel/vmlinux.lds.S | 173 +++++++++++++++++++++
arch/kvx/lib/Makefile | 6 +
arch/kvx/mm/Makefile | 10 ++
arch/kvx/platform/Makefile | 7 +
11 files changed, 639 insertions(+)
create mode 100644 arch/kvx/Kconfig
create mode 100644 arch/kvx/Kconfig.debug
create mode 100644 arch/kvx/Makefile
create mode 100644 arch/kvx/include/asm/Kbuild
create mode 100644 arch/kvx/include/uapi/asm/Kbuild
create mode 100644 arch/kvx/kernel/Makefile
create mode 100644 arch/kvx/kernel/kvx_ksyms.c
create mode 100644 arch/kvx/kernel/vmlinux.lds.S
create mode 100644 arch/kvx/lib/Makefile
create mode 100644 arch/kvx/mm/Makefile
create mode 100644 arch/kvx/platform/Makefile
diff --git a/arch/kvx/Kconfig b/arch/kvx/Kconfig
new file mode 100644
index 000000000000..8e8820c5c860
--- /dev/null
+++ b/arch/kvx/Kconfig
@@ -0,0 +1,249 @@
+#
+# For a description of the syntax of this configuration file,
+# see Documentation/kbuild/kconfig-language.txt.
+#
+
+config 64BIT
+ def_bool y
+
+config GENERIC_CALIBRATE_DELAY
+ def_bool y
+
+config FIX_EARLYCON_MEM
+ def_bool y
+
+config MMU
+ def_bool y
+
+config KALLSYMS_BASE_RELATIVE
+ def_bool n
+
+config GENERIC_CSUM
+ def_bool y
+
+config RWSEM_GENERIC_SPINLOCK
+ def_bool y
+
+config GENERIC_HWEIGHT
+ def_bool y
+
+config ARCH_MMAP_RND_BITS_MAX
+ default 24
+
+config ARCH_MMAP_RND_BITS_MIN
+ default 18
+
+config STACKTRACE_SUPPORT
+ def_bool y
+
+config LOCKDEP_SUPPORT
+ def_bool y
+
+config GENERIC_BUG
+ def_bool y
+ depends on BUG
+
+config KVX_4K_PAGES
+ def_bool y
+
+config KVX
+ def_bool y
+ select ARCH_CLOCKSOURCE_DATA
+ select ARCH_DMA_ADDR_T_64BIT
+ select ARCH_HAS_DEVMEM_IS_ALLOWED
+ select ARCH_HAS_DMA_PREP_COHERENT
+ select ARCH_HAS_ELF_RANDOMIZE
+ select ARCH_HAS_PTE_SPECIAL
+ select ARCH_HAS_SETUP_DMA_OPS if IOMMU_SUPPORT
+ select ARCH_HAS_STRICT_KERNEL_RWX
+ select ARCH_HAS_STRICT_MODULE_RWX
+ select ARCH_HAS_SYNC_DMA_FOR_DEVICE
+ select ARCH_HAS_SYNC_DMA_FOR_CPU
+ select ARCH_HAS_TEARDOWN_DMA_OPS if IOMMU_SUPPORT
+ select ARCH_OPTIONAL_KERNEL_RWX_DEFAULT
+ select ARCH_SUPPORTS_HUGETLBFS
+ select ARCH_USE_QUEUED_SPINLOCKS
+ select ARCH_USE_QUEUED_RWLOCKS
+ select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT
+ select ARCH_WANT_FRAME_POINTERS
+ select CLKSRC_OF
+ select COMMON_CLK
+ select DMA_DIRECT_REMAP
+ select GENERIC_ALLOCATOR
+ select GENERIC_CLOCKEVENTS
+ select GENERIC_CLOCKEVENTS
+ select GENERIC_CPU_DEVICES
+ select GENERIC_IOMAP
+ select GENERIC_IOREMAP
+ select GENERIC_IRQ_CHIP
+ select GENERIC_IRQ_PROBE
+ select GENERIC_IRQ_SHOW
+ select GENERIC_SCHED_CLOCK
+ select HAVE_ARCH_AUDITSYSCALL
+ select HAVE_ARCH_BITREVERSE
+ select HAVE_ARCH_JUMP_LABEL
+ select HAVE_ARCH_MMAP_RND_BITS
+ select HAVE_ARCH_TRACEHOOK
+ select HAVE_ARCH_TRANSPARENT_HUGEPAGE
+ select HAVE_ASM_MODVERSIONS
+ select HAVE_DEBUG_KMEMLEAK
+ select HAVE_DYNAMIC_FTRACE
+ select HAVE_EFFICIENT_UNALIGNED_ACCESS
+ select HAVE_FTRACE_MCOUNT_RECORD
+ select HAVE_FUTEX_CMPXCHG if FUTEX
+ select HAVE_FUNCTION_TRACER
+ select HAVE_FUNCTION_GRAPH_TRACER
+ select HAVE_HW_BREAKPOINT
+ select HAVE_IOREMAP_PROT
+ select HAVE_MEMBLOCK_NODE_MAP
+ select HAVE_PCI
+ select HAVE_PERF_EVENTS
+ select HAVE_STACKPROTECTOR
+ select HAVE_SYSCALL_TRACEPOINTS
+ select IOMMU_DMA if IOMMU_SUPPORT
+ select KVX_APIC_GIC
+ select KVX_APIC_MAILBOX
+ select KVX_CORE_INTC
+ select KVX_ITGEN
+ select KVX_WATCHDOG
+ select MODULES_USE_ELF_RELA
+ select OF
+ select OF_EARLY_FLATTREE
+ select OF_RESERVED_MEM
+ select PERF_EVENTS
+ select PCI_DOMAINS_GENERIC if PCI
+ select SPARSE_IRQ
+ select SYSCTL_EXCEPTION_TRACE
+ select THREAD_INFO_IN_TASK
+ select TIMER_OF
+ select TRACE_IRQFLAGS_SUPPORT
+ select WATCHDOG
+ select ZONE_DMA32
+
+config PGTABLE_LEVELS
+ int
+ default 3
+
+config HAVE_KPROBES
+ def_bool n
+
+menu "System setup"
+
+config POISON_INITMEM
+ bool "Enable to poison freed initmem"
+ default y
+ help
+ In order to debug initmem, using poison allows to verify if
+ some data/code is stille using them. Enable this for debug
+ purposes
+
+config KVX_PHYS_OFFSET
+ hex "RAM address of memory base"
+ default 0x100000000
+
+config KVX_PAGE_OFFSET
+ hex "kernel virtual address of memory base"
+ default 0xFFFFFF8000000000
+
+config ARCH_FLATMEM_ENABLE
+ def_bool y
+
+config ARCH_SPARSEMEM_ENABLE
+ def_bool y
+
+config ARCH_SPARSEMEM_DEFAULT
+ def_bool ARCH_SPARSEMEM_ENABLE
+
+config ARCH_SELECT_MEMORY_MODEL
+ def_bool ARCH_SPARSEMEM_ENABLE
+
+config STACK_MAX_DEPTH_TO_PRINT
+ int "Maximum depth of stack to print"
+ range 1 128
+ default "24"
+
+config L2_CACHE
+ bool "Enable l2 cache driver"
+ default y
+ help
+ Enable L2 cache driver in order to handle it. This driver can be
+ disabled to eliminate any overhead related to L2 cache
+ inval/writeback. Note that L2 cache is mandatory for SMP in order
+ to maintain coherency on DDR between all PEs.
+
+config SECURE_DAME_HANDLING
+ bool "Secure DAME handling"
+ default y
+ help
+ In order to securely handle Data Asynchronous Memory Errors, we need
+ to do a barrier upon kernel entry when coming from userspace. This
+ barrier guarantee us that any pending DAME will be serviced right
+ away. We also need to do a barrier when returning from kernel to user.
+ This way, if the kernel or the user triggered a DAME, it will be
+ serviced by knowing we are coming from kernel or user and avoid
+ pulling the wrong lever (panic for kernel or sigfault for user).
+ This can be costly but ensure that user cannot interfere with kernel.
+ /!\ Do not disable unless you want to open a giant breach between
+ user and kernel /!\
+
+config CACHECTL_UNSAFE_PHYS_OPERATIONS
+ bool "Enable cachectl syscall unsafe physical operations"
+ default n
+ help
+ Enable cachectl syscall to allow writebacking/invalidating ranges
+ based on physical addresses. These operations requires the
+ CAP_SYS_ADMIN capability
+
+config ENABLE_TCA
+ bool "Enable TCA coprocessor support"
+ default y
+ help
+ This option enables TCA coprocessor support. It will allow the user to
+ use the coprocessor and save registers on context switch if used.
+ Registers content will also be cleared when switching.
+
+config SMP
+ bool "Symmetric multi-processing support"
+ default n
+ select GENERIC_SMP_IDLE_THREAD
+ select GENERIC_IRQ_IPI
+ select IRQ_DOMAIN_HIERARCHY
+ select IRQ_DOMAIN
+ select L2_CACHE
+ help
+ This enables support for systems with more than one CPU. If you have
+ a system with only one CPU, say N. If you have a system with more
+ than one CPU, say Y.
+
+ If you say N here, the kernel will run on uni- and multiprocessor
+ machines, but will use only one CPU of a multiprocessor machine. If
+ you say Y here, the kernel will run on many, but not all,
+ uniprocessor machines. On a uniprocessor machine, the kernel
+ will run faster if you say N here.
+
+config NR_CPUS
+ int "Maximum number of CPUs"
+ range 1 16
+ default "16"
+ depends on SMP
+ help
+ Kalray support can handle a maximum of 16 CPUs.
+
+config KVX_PAGE_SHIFT
+ int
+ default 12
+
+config CMDLINE
+ string "Default kernel command string"
+ default ""
+ help
+ On some architectures there is currently no way for the boot loader
+ to pass arguments to the kernel. For these architectures, you should
+ supply some command-line options at build time by entering them
+ here.
+
+endmenu
+
+menu "Kernel Features"
+source "kernel/Kconfig.hz"
+endmenu
diff --git a/arch/kvx/Kconfig.debug b/arch/kvx/Kconfig.debug
new file mode 100644
index 000000000000..027e919a1e14
--- /dev/null
+++ b/arch/kvx/Kconfig.debug
@@ -0,0 +1,70 @@
+menu "KVX debugging"
+
+config KVX_DEBUG_ASN
+ bool "Check ASN before writing TLB entry"
+ default n
+ help
+ This option allows to check if the ASN of the current
+ process is matching the ASN found in MMC. If it is not the
+ case an error will be printed.
+
+config KVX_DEBUG_TLB_WRITE
+ bool "Enable TLBs write checks"
+ default n
+ help
+ Enabling this option will enable TLB access checks. This is
+ particularly helpful when modifying the assembly code responsible
+ of TLB refill. If set, mmc.e will be checked each time the tlb are
+ written and a panic will be thrown on error.
+
+config KVX_DEBUG_TLB_ACCESS
+ bool "Enable TLBs accesses logging"
+ default n
+ help
+ Enabling this option will enable TLB entry manipulation logging.
+ Each time an entry will be added to the TLBs, it will be logged in
+ an array readable via gdb scripts. This can be useful to understand
+ strange crashes related to suspicious virtual/physical addresses.
+
+config KVX_DEBUG_TLB_ACCESS_BITS
+ int "Number of bits used as index of entries in log table"
+ default 12
+ depends on KVX_DEBUG_TLB_ACCESS
+ help
+ Set the number of bits used as index of entries that will be logged
+ in a ring buffer called kvx_tlb_access. One entry in the table
+ contains registers TEL, TEH and MMC. It also logs the type of the
+ operations (0:read, 1:write, 2:probe). Buffer is per CPU. For one
+ entry 24 bytes are used. So by default it uses 96Ko of memory per
+ CPU to store 2^12 (4096) entries.
+
+config KVX_MMU_STATS
+ bool "Register mmu stats debugfs entries"
+ default n
+ select DEBUG_FS
+ help
+ Enable debugfs attribute which will allow inspecting various metric
+ regarding MMU:
+ - Number of nomapping traps handled
+ - avg/min/max time for nomapping refill (user/kernel)
+
+config DEBUG_EXCEPTION_STACK
+ bool "Enable exception stack debugging"
+ default n
+ help
+ Enable stack check debugging when entering/exiting
+ exception handlers.
+ This can be particularly helpful after modifying stack
+ handling to see if stack when exiting is the same as the one
+ when entering exception handler.
+
+config DEBUG_SFR_SET_MASK
+ bool "Enable sfr set_mask debugging"
+ default n
+ help
+ Verify that values written using kvx_sfr_set_mask are matching the
+ mask. This ensure that no extra bits of sfr will be overridden by some
+ incorrectly truncated values. This can lead to huge problems by
+ modifying important bits in system registers.
+
+endmenu
diff --git a/arch/kvx/Makefile b/arch/kvx/Makefile
new file mode 100644
index 000000000000..b9da5017cf2a
--- /dev/null
+++ b/arch/kvx/Makefile
@@ -0,0 +1,52 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2018-2023 Kalray Inc.
+
+ifeq ($(CROSS_COMPILE),)
+CROSS_COMPILE := kvx-elf-
+endif
+
+KBUILD_DEFCONFIG := default_defconfig
+
+LDFLAGS_vmlinux := -X
+OBJCOPYFLAGS := -O binary -R .comment -R .note -R .bootloader -S
+
+# Link with libgcc to get __div* builtins.
+LIBGCC := $(shell $(CC) $(KBUILD_CFLAGS) --print-libgcc-file-name)
+
+DEFAULT_OPTS := -nostdlib -fno-builtin -march=kv3-1
+KBUILD_CFLAGS += $(DEFAULT_OPTS)
+KBUILD_AFLAGS += $(DEFAULT_OPTS)
+KBUILD_CFLAGS_MODULE += -mfarcall
+
+KBUILD_LDFLAGS += -m elf64kvx
+
+head-y := arch/kvx/kernel/head.o
+libs-y += $(LIBGCC)
+libs-y += arch/kvx/lib/
+core-y += arch/kvx/kernel/ \
+ arch/kvx/mm/ \
+ arch/kvx/platform/
+# Final targets
+all: vmlinux
+
+BOOT_TARGETS = bImage bImage.bin bImage.bz2 bImage.gz bImage.lzma bImage.lzo
+
+$(BOOT_TARGETS): vmlinux
+ $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
+
+install:
+ $(Q)$(MAKE) $(build)=$(boot) BOOTIMAGE=$(KBUILD_IMAGE) install
+
+define archhelp
+ echo '* bImage - Alias to selected kernel format (bImage.gz by default)'
+ echo ' bImage.bin - Uncompressed Kernel-only image for barebox (arch/$(ARCH)/boot/bImage.bin)'
+ echo ' bImage.bz2 - Kernel-only image for barebox (arch/$(ARCH)/boot/bImage.bz2)'
+ echo '* bImage.gz - Kernel-only image for barebox (arch/$(ARCH)/boot/bImage.gz)'
+ echo ' bImage.lzma - Kernel-only image for barebox (arch/$(ARCH)/boot/bImage.lzma)'
+ echo ' bImage.lzo - Kernel-only image for barebox (arch/$(ARCH)/boot/bImage.lzo)'
+ echo ' install - Install kernel using'
+ echo ' (your) ~/bin/$(INSTALLKERNEL) or'
+ echo ' (distribution) PATH: $(INSTALLKERNEL) or'
+ echo ' install to $$(INSTALL_PATH)'
+endef
diff --git a/arch/kvx/include/asm/Kbuild b/arch/kvx/include/asm/Kbuild
new file mode 100644
index 000000000000..ea73552faa10
--- /dev/null
+++ b/arch/kvx/include/asm/Kbuild
@@ -0,0 +1,20 @@
+generic-y += asm-offsets.h
+generic-y += clkdev.h
+generic-y += auxvec.h
+generic-y += bpf_perf_event.h
+generic-y += cmpxchg-local.h
+generic-y += errno.h
+generic-y += extable.h
+generic-y += export.h
+generic-y += kvm_para.h
+generic-y += mcs_spinlock.h
+generic-y += mman.h
+generic-y += param.h
+generic-y += qrwlock.h
+generic-y += qspinlock.h
+generic-y += rwsem.h
+generic-y += sockios.h
+generic-y += stat.h
+generic-y += statfs.h
+generic-y += ucontext.h
+generic-y += user.h
diff --git a/arch/kvx/include/uapi/asm/Kbuild b/arch/kvx/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..8b137891791f
--- /dev/null
+++ b/arch/kvx/include/uapi/asm/Kbuild
@@ -0,0 +1 @@
+
diff --git a/arch/kvx/kernel/Makefile b/arch/kvx/kernel/Makefile
new file mode 100644
index 000000000000..d155fef62614
--- /dev/null
+++ b/arch/kvx/kernel/Makefile
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2019-2023 Kalray Inc.
+#
+
+ifdef CONFIG_FUNCTION_TRACER
+CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_insns.o = $(CC_FLAGS_FTRACE)
+endif
+
+CFLAGS_REMOVE_return_address.o = $(CC_FLAGS_FTRACE)
+
+obj-y := head.o setup.o process.o traps.o common.o time.o prom.o kvx_ksyms.o \
+ irq.o cpuinfo.o ptrace.o entry.o syscall_table.o signal.o sys_kvx.o \
+ stacktrace.o dame_handler.o vdso.o perf_event.o return_address.o \
+ insns.o break_hook.o reset.o io.o debug.o
+
+obj-$(CONFIG_JUMP_LABEL) += jump_label.o
+obj-$(CONFIG_SMP) += smp.o smpboot.o
+obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
+obj-$(CONFIG_MODULES) += module.o
+obj-$(CONFIG_KGDB) += kgdb.o
+CFLAGS_module.o += -Wstrict-overflow -fstrict-overflow
+obj-$(CONFIG_L2_CACHE) += l2_cache.o
+obj-$(CONFIG_FUNCTION_TRACER) += mcount.o ftrace.o
+
+extra-y += vmlinux.lds
diff --git a/arch/kvx/kernel/kvx_ksyms.c b/arch/kvx/kernel/kvx_ksyms.c
new file mode 100644
index 000000000000..18990aaf259f
--- /dev/null
+++ b/arch/kvx/kernel/kvx_ksyms.c
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/nios2/kernel/nios2_ksyms.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ */
+
+#include <linux/kernel.h>
+#include <linux/export.h>
+
+/*
+ * libgcc functions - functions that are used internally by the
+ * compiler... (prototypes are not correct though, but that
+ * doesn't really matter since they're not versioned).
+ */
+#define DECLARE_EXPORT(name) extern void name(void); EXPORT_SYMBOL(name)
+
+DECLARE_EXPORT(__moddi3);
+DECLARE_EXPORT(__umoddi3);
+DECLARE_EXPORT(__divdi3);
+DECLARE_EXPORT(__udivdi3);
+DECLARE_EXPORT(__multi3);
diff --git a/arch/kvx/kernel/vmlinux.lds.S b/arch/kvx/kernel/vmlinux.lds.S
new file mode 100644
index 000000000000..74013771ff40
--- /dev/null
+++ b/arch/kvx/kernel/vmlinux.lds.S
@@ -0,0 +1,173 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Marius Gligor
+ * Marc Poulhiès
+ * Yann Sionneau
+ */
+
+#include <asm/thread_info.h>
+#include <asm/asm-offsets.h>
+#include <asm/sys_arch.h>
+#include <asm/cache.h>
+#include <asm/rm_fw.h>
+#include <asm/page.h>
+#include <asm/fixmap.h>
+
+#define BOOT_ENTRY 0x0
+#define DTB_DEFAULT_SIZE (64 * 1024)
+#define RM_FIRMWARE_SIZE (64 * 1024)
+#define RM_FIRMWARE_ADDRESS (64 * 1024)
+
+#define LOAD_OFFSET (PAGE_OFFSET - PHYS_OFFSET)
+#include <asm-generic/vmlinux.lds.h>
+
+OUTPUT_FORMAT("elf64-kvx")
+ENTRY(kvx_start)
+
+#define HANDLER_SECTION(__sec, __name) \
+ __sec ## _ ## __name ## _start = .; \
+ KEEP(*(.##__sec ##.## __name)); \
+ . = __sec ## _ ##__name ## _start + EXCEPTION_STRIDE;
+
+/**
+ * Generate correct section positioning for exception handling
+ * Since we need it twice for early exception handler and normal
+ * exception handler, factorize it here.
+ */
+#define EXCEPTION_SECTIONS(__sec) \
+ __ ## __sec ## _start = ABSOLUTE(.); \
+ HANDLER_SECTION(__sec,debug) \
+ HANDLER_SECTION(__sec,trap) \
+ HANDLER_SECTION(__sec,interrupt) \
+ HANDLER_SECTION(__sec,syscall)
+
+jiffies = jiffies_64;
+SECTIONS
+{
+ . = BOOT_ENTRY;
+ .boot :
+ {
+ __kernel_smem_code_start = .;
+ KEEP(*(.boot.startup));
+ KEEP(*(.boot.*));
+ __kernel_smem_code_end = .;
+ }
+
+ /**
+ * Special section to patch rm_firmware binary after compiling
+ * the kernel.
+ */
+ . = RM_FIRMWARE_ADDRESS;
+ .rm_firmware :
+ {
+ __rm_firmware_start = .;
+ KEEP(*(.rm_firmware.default));
+ /* Force minimum size */
+ . = __rm_firmware_start + RM_FIRMWARE_SIZE;
+ __rm_firmware_end = .;
+
+ /* Registers for RM firmware */
+ . = ALIGN(PAGE_SIZE);
+ __rm_firmware_regs_start = .;
+ /* Force minimum size */
+ . = __rm_firmware_regs_start + RM_FIRMWARE_REGS_SIZE;
+ __rm_firmware_regs_end = .;
+ }
+
+ . = PAGE_OFFSET;
+ _start = .;
+
+ _stext = .;
+ __init_begin = .;
+ __inittext_start = .;
+ .exit.text : AT(ADDR(.exit.text) - LOAD_OFFSET)
+ {
+ EXIT_TEXT
+ }
+
+ .early_exception ALIGN(EXCEPTION_ALIGNMENT) :
+ AT(ADDR(.early_exception) - LOAD_OFFSET)
+ {
+ EXCEPTION_SECTIONS(early_exception)
+ }
+
+ HEAD_TEXT_SECTION
+ INIT_TEXT_SECTION(PAGE_SIZE)
+ . = ALIGN(PAGE_SIZE);
+ __inittext_end = .;
+ __initdata_start = .;
+ INIT_DATA_SECTION(16)
+
+ /* we have to discard exit text and such at runtime, not link time */
+ .exit.data : AT(ADDR(.exit.data) - LOAD_OFFSET)
+ {
+ EXIT_DATA
+ }
+
+ PERCPU_SECTION(L1_CACHE_BYTES)
+ . = ALIGN(PAGE_SIZE);
+ __initdata_end = .;
+ __init_end = .;
+
+ /* Everything below this point will be mapped RO EXEC up to _etext */
+ .text ALIGN(PAGE_SIZE) : AT(ADDR(.text) - LOAD_OFFSET)
+ {
+ _text = .;
+ EXCEPTION_SECTIONS(exception)
+ *(.exception.text)
+ . = ALIGN(PAGE_SIZE);
+ __exception_end = .;
+ TEXT_TEXT
+ SCHED_TEXT
+ CPUIDLE_TEXT
+ LOCK_TEXT
+ KPROBES_TEXT
+ ENTRY_TEXT
+ IRQENTRY_TEXT
+ SOFTIRQENTRY_TEXT
+ *(.fixup)
+ }
+ . = ALIGN(PAGE_SIZE);
+ _etext = .;
+
+ /* Everything below this point will be mapped RO NOEXEC up to _sdata */
+ __rodata_start = .;
+ RO_DATA(PAGE_SIZE)
+ EXCEPTION_TABLE(8)
+ . = ALIGN(32);
+ .dtb : AT(ADDR(.dtb) - LOAD_OFFSET)
+ {
+ __dtb_start = .;
+ . += DTB_DEFAULT_SIZE;
+ __dtb_end = .;
+ }
+ . = ALIGN(PAGE_SIZE);
+ __rodata_end = .;
+
+ /* Everything below this point will be mapped RW NOEXEC up to _end */
+ _sdata = .;
+ RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+ _edata = .;
+
+ BSS_SECTION(32, 32, 32)
+ . = ALIGN(PAGE_SIZE);
+ _end = .;
+
+ /* This page will be mapped using a FIXMAP */
+ .gdb_page ALIGN(PAGE_SIZE) : AT(ADDR(.gdb_page) - LOAD_OFFSET)
+ {
+ _debug_start = ADDR(.gdb_page) - LOAD_OFFSET;
+ . += PAGE_SIZE;
+ }
+ _debug_start_lma = ASM_FIX_TO_VIRT(FIX_GDB_MEM_BASE_IDX);
+
+ /* Debugging sections */
+ STABS_DEBUG
+ DWARF_DEBUG
+
+ /* Sections to be discarded -- must be last */
+ DISCARDS
+}
diff --git a/arch/kvx/lib/Makefile b/arch/kvx/lib/Makefile
new file mode 100644
index 000000000000..ddb666c244e1
--- /dev/null
+++ b/arch/kvx/lib/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2017-2023 Kalray Inc.
+#
+
+lib-y := usercopy.o clear_page.o copy_page.o memcpy.o memset.o strlen.o delay.o
diff --git a/arch/kvx/mm/Makefile b/arch/kvx/mm/Makefile
new file mode 100644
index 000000000000..d97ab0e94a6f
--- /dev/null
+++ b/arch/kvx/mm/Makefile
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2017-2023 Kalray Inc.
+#
+
+obj-y := init.o mmu.o fault.o tlb.o extable.o dma-mapping.o cacheflush.o
+obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
+obj-$(CONFIG_KVX_MMU_STATS) += mmu_stats.o
+obj-$(CONFIG_STRICT_KERNEL_RWX) += kernel_rwx.o
+obj-$(CONFIG_STRICT_DEVMEM) += mmap.o
diff --git a/arch/kvx/platform/Makefile b/arch/kvx/platform/Makefile
new file mode 100644
index 000000000000..27f0914e0de5
--- /dev/null
+++ b/arch/kvx/platform/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2018-2023 Kalray Inc.
+#
+
+obj-$(CONFIG_SMP) += pwr_ctrl.o
+obj-$(CONFIG_SMP) += ipi.o
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 03/25] kvx: Add build infrastructure
2023-01-03 16:43 ` [RFC PATCH 03/25] kvx: Add build infrastructure Yann Sionneau
@ 2023-01-03 17:29 ` Randy Dunlap
2023-01-05 13:12 ` Jules Maselbas
2023-01-06 0:43 ` kernel test robot
1 sibling, 1 reply; 113+ messages in thread
From: Randy Dunlap @ 2023-01-03 17:29 UTC (permalink / raw)
To: Yann Sionneau
Cc: linux-kernel, bpf, Clement Leger, Guillaume Thouvenin,
Jonathan Borne, Jules Maselbas, Julian Vetter,
Marc Poulhiès, Marius Gligor, Samuel Jones, Vincent Chardon
Hi,
On 1/3/23 08:43, Yann Sionneau wrote:
> Add Kbuild, Makefile, Kconfig and link script for kvx build infrastructure.
>
> CC: linux-kernel@vger.kernel.org
> CC: bpf@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Co-developed-by: Jonathan Borne <jborne@kalray.eu>
> Signed-off-by: Jonathan Borne <jborne@kalray.eu>
> Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
> Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Co-developed-by: Marc Poulhiès <dkm@kataplop.net>
> Signed-off-by: Marc Poulhiès <dkm@kataplop.net>
> Co-developed-by: Marius Gligor <mgligor@kalray.eu>
> Signed-off-by: Marius Gligor <mgligor@kalray.eu>
> Co-developed-by: Samuel Jones <sjones@kalray.eu>
> Signed-off-by: Samuel Jones <sjones@kalray.eu>
> Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
> Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> arch/kvx/Kconfig | 249 +++++++++++++++++++++++++++++++
> arch/kvx/Kconfig.debug | 70 +++++++++
> arch/kvx/Makefile | 52 +++++++
> arch/kvx/include/asm/Kbuild | 20 +++
> arch/kvx/include/uapi/asm/Kbuild | 1 +
> arch/kvx/kernel/Makefile | 27 ++++
> arch/kvx/kernel/kvx_ksyms.c | 24 +++
> arch/kvx/kernel/vmlinux.lds.S | 173 +++++++++++++++++++++
> arch/kvx/lib/Makefile | 6 +
> arch/kvx/mm/Makefile | 10 ++
> arch/kvx/platform/Makefile | 7 +
> 11 files changed, 639 insertions(+)
> create mode 100644 arch/kvx/Kconfig
> create mode 100644 arch/kvx/Kconfig.debug
> create mode 100644 arch/kvx/Makefile
> create mode 100644 arch/kvx/include/asm/Kbuild
> create mode 100644 arch/kvx/include/uapi/asm/Kbuild
> create mode 100644 arch/kvx/kernel/Makefile
> create mode 100644 arch/kvx/kernel/kvx_ksyms.c
> create mode 100644 arch/kvx/kernel/vmlinux.lds.S
> create mode 100644 arch/kvx/lib/Makefile
> create mode 100644 arch/kvx/mm/Makefile
> create mode 100644 arch/kvx/platform/Makefile
>
> diff --git a/arch/kvx/Kconfig b/arch/kvx/Kconfig
> new file mode 100644
> index 000000000000..8e8820c5c860
> --- /dev/null
> +++ b/arch/kvx/Kconfig
> @@ -0,0 +1,249 @@
> +#
> +# For a description of the syntax of this configuration file,
> +# see Documentation/kbuild/kconfig-language.txt.
> +#
> +
> +config 64BIT
> + def_bool y
> +
> +config GENERIC_CALIBRATE_DELAY
> + def_bool y
> +
> +config FIX_EARLYCON_MEM
> + def_bool y
> +
> +config MMU
> + def_bool y
> +
> +config KALLSYMS_BASE_RELATIVE
> + def_bool n
> +
> +config GENERIC_CSUM
> + def_bool y
> +
> +config RWSEM_GENERIC_SPINLOCK
> + def_bool y
> +
> +config GENERIC_HWEIGHT
> + def_bool y
> +
> +config ARCH_MMAP_RND_BITS_MAX
> + default 24
> +
Do the above and below configs need an "int" line for their type?
I notice that the patch does that below for PGTABLE_LEVELS.
> +config ARCH_MMAP_RND_BITS_MIN
> + default 18
> +
> +config STACKTRACE_SUPPORT
> + def_bool y
> +
> +config LOCKDEP_SUPPORT
> + def_bool y
> +
> +config GENERIC_BUG
> + def_bool y
> + depends on BUG
> +
> +config KVX_4K_PAGES
> + def_bool y
> +
> +config KVX
> + def_bool y
[deletes]
> +
> +config PGTABLE_LEVELS
> + int
> + default 3
^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +config HAVE_KPROBES
> + def_bool n
> +
> +menu "System setup"
> +
> +config POISON_INITMEM
> + bool "Enable to poison freed initmem"
> + default y
> + help
> + In order to debug initmem, using poison allows to verify if
> + some data/code is stille using them. Enable this for debug
still
> + purposes
purposes.
> +
> +config KVX_PHYS_OFFSET
> + hex "RAM address of memory base"
> + default 0x100000000
> +
> +config KVX_PAGE_OFFSET
> + hex "kernel virtual address of memory base"
> + default 0xFFFFFF8000000000
> +
> +config ARCH_FLATMEM_ENABLE
> + def_bool y
> +
> +config ARCH_SPARSEMEM_ENABLE
> + def_bool y
> +
> +config ARCH_SPARSEMEM_DEFAULT
> + def_bool ARCH_SPARSEMEM_ENABLE
> +
> +config ARCH_SELECT_MEMORY_MODEL
> + def_bool ARCH_SPARSEMEM_ENABLE
> +
> +config STACK_MAX_DEPTH_TO_PRINT
> + int "Maximum depth of stack to print"
> + range 1 128
> + default "24"
> +
> +config L2_CACHE
> + bool "Enable l2 cache driver"
L2
> + default y
> + help
> + Enable L2 cache driver in order to handle it. This driver can be
> + disabled to eliminate any overhead related to L2 cache
> + inval/writeback. Note that L2 cache is mandatory for SMP in order
> + to maintain coherency on DDR between all PEs.
> +
> +config SECURE_DAME_HANDLING
> + bool "Secure DAME handling"
> + default y
> + help
> + In order to securely handle Data Asynchronous Memory Errors, we need
> + to do a barrier upon kernel entry when coming from userspace. This
> + barrier guarantee us that any pending DAME will be serviced right
guarantees
> + away. We also need to do a barrier when returning from kernel to user.
> + This way, if the kernel or the user triggered a DAME, it will be
> + serviced by knowing we are coming from kernel or user and avoid
> + pulling the wrong lever (panic for kernel or sigfault for user).
> + This can be costly but ensure that user cannot interfere with kernel.
ensures
> + /!\ Do not disable unless you want to open a giant breach between
> + user and kernel /!\
> +
> +config CACHECTL_UNSAFE_PHYS_OPERATIONS
> + bool "Enable cachectl syscall unsafe physical operations"
> + default n
> + help
> + Enable cachectl syscall to allow writebacking/invalidating ranges
> + based on physical addresses. These operations requires the
> + CAP_SYS_ADMIN capability
capability.
> +
> +config ENABLE_TCA
> + bool "Enable TCA coprocessor support"
> + default y
> + help
> + This option enables TCA coprocessor support. It will allow the user to
> + use the coprocessor and save registers on context switch if used.
> + Registers content will also be cleared when switching.
> +
> +config SMP
> + bool "Symmetric multi-processing support"
> + default n
> + select GENERIC_SMP_IDLE_THREAD
> + select GENERIC_IRQ_IPI
> + select IRQ_DOMAIN_HIERARCHY
> + select IRQ_DOMAIN
> + select L2_CACHE
> + help
> + This enables support for systems with more than one CPU. If you have
> + a system with only one CPU, say N. If you have a system with more
> + than one CPU, say Y.
> +
> + If you say N here, the kernel will run on uni- and multiprocessor
> + machines, but will use only one CPU of a multiprocessor machine. If
> + you say Y here, the kernel will run on many, but not all,
> + uniprocessor machines. On a uniprocessor machine, the kernel
> + will run faster if you say N here.
> +
> +config NR_CPUS
> + int "Maximum number of CPUs"
> + range 1 16
> + default "16"
> + depends on SMP
> + help
> + Kalray support can handle a maximum of 16 CPUs.
> +
> +config KVX_PAGE_SHIFT
> + int
> + default 12
> +
> +config CMDLINE
> + string "Default kernel command string"
> + default ""
> + help
> + On some architectures there is currently no way for the boot loader
> + to pass arguments to the kernel. For these architectures, you should
> + supply some command-line options at build time by entering them
> + here.
> +
> +endmenu
> +
> +menu "Kernel Features"
> +source "kernel/Kconfig.hz"
> +endmenu
> diff --git a/arch/kvx/Kconfig.debug b/arch/kvx/Kconfig.debug
> new file mode 100644
> index 000000000000..027e919a1e14
> --- /dev/null
> +++ b/arch/kvx/Kconfig.debug
> @@ -0,0 +1,70 @@
> +menu "KVX debugging"
> +
> +config KVX_DEBUG_ASN
> + bool "Check ASN before writing TLB entry"
> + default n
> + help
> + This option allows to check if the ASN of the current
> + process is matching the ASN found in MMC. If it is not the
s/is matching/matches/
> + case an error will be printed.
> +
> +config KVX_DEBUG_TLB_WRITE
> + bool "Enable TLBs write checks"
> + default n
> + help
> + Enabling this option will enable TLB access checks. This is
> + particularly helpful when modifying the assembly code responsible
> + of TLB refill. If set, mmc.e will be checked each time the tlb are
for TLB refill. TLB
> + written and a panic will be thrown on error.
> +
> +config KVX_DEBUG_TLB_ACCESS
> + bool "Enable TLBs accesses logging"
> + default n
> + help
> + Enabling this option will enable TLB entry manipulation logging.
> + Each time an entry will be added to the TLBs, it will be logged in
s/will be/is/ s/will be/is/
> + an array readable via gdb scripts. This can be useful to understand
> + strange crashes related to suspicious virtual/physical addresses.
> +
> +config KVX_DEBUG_TLB_ACCESS_BITS
> + int "Number of bits used as index of entries in log table"
> + default 12
> + depends on KVX_DEBUG_TLB_ACCESS
> + help
> + Set the number of bits used as index of entries that will be logged
> + in a ring buffer called kvx_tlb_access. One entry in the table
> + contains registers TEL, TEH and MMC. It also logs the type of the
> + operations (0:read, 1:write, 2:probe). Buffer is per CPU. For one
> + entry 24 bytes are used. So by default it uses 96Ko of memory per
What does "Ko" mean?
> + CPU to store 2^12 (4096) entries.
> +
> +config KVX_MMU_STATS
> + bool "Register mmu stats debugfs entries"
Preferably MMU
> + default n
> + select DEBUG_FS
Preferably:
depends on DEBUG_FS
> + help
> + Enable debugfs attribute which will allow inspecting various metric
metrics
> + regarding MMU:
> + - Number of nomapping traps handled
> + - avg/min/max time for nomapping refill (user/kernel)
> +
> +config DEBUG_EXCEPTION_STACK
> + bool "Enable exception stack debugging"
> + default n
> + help
> + Enable stack check debugging when entering/exiting
> + exception handlers.
> + This can be particularly helpful after modifying stack
> + handling to see if stack when exiting is the same as the one
> + when entering exception handler.
> +
> +config DEBUG_SFR_SET_MASK
> + bool "Enable sfr set_mask debugging"
> + default n
> + help
> + Verify that values written using kvx_sfr_set_mask are matching the
s/are matching/match/
> + mask. This ensure that no extra bits of sfr will be overridden by some
ensures
and preferably s/sfr/SFR/ in 2 places (not in kvx_sfr_set_mask).
> + incorrectly truncated values. This can lead to huge problems by
> + modifying important bits in system registers.
> +
> +endmenu
Thanks.
--
~Randy
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 03/25] kvx: Add build infrastructure
2023-01-03 17:29 ` Randy Dunlap
@ 2023-01-05 13:12 ` Jules Maselbas
0 siblings, 0 replies; 113+ messages in thread
From: Jules Maselbas @ 2023-01-05 13:12 UTC (permalink / raw)
To: Randy Dunlap
Cc: Yann Sionneau, linux-kernel, bpf, Clement Leger,
Guillaume Thouvenin, Jonathan Borne, Julian Vetter,
Marc Poulhiès, Marius Gligor, Samuel Jones, Vincent Chardon
Hi,
On Tue, Jan 03, 2023 at 09:29:21AM -0800, Randy Dunlap wrote:
> Hi,
>
-- snip snip
> > +config ARCH_MMAP_RND_BITS_MAX
> > + default 24
> > +
>
> Do the above and below configs need an "int" line for their type?
> I notice that the patch does that below for PGTABLE_LEVELS.
>
I don't think it needs an "int" line, it is added in arch/Kconfig
(which includes arch/kvx/Kconfig)
> > +config ARCH_MMAP_RND_BITS_MIN
> > + default 18
> > +
> > +config STACKTRACE_SUPPORT
> > + def_bool y
> > +
> > +config LOCKDEP_SUPPORT
> > + def_bool y
> > +
> > +config GENERIC_BUG
> > + def_bool y
> > + depends on BUG
> > +
> > +config KVX_4K_PAGES
> > + def_bool y
> > +
> > +config KVX
> > + def_bool y
> [deletes]
> > +
> > +config PGTABLE_LEVELS
> > + int
> > + default 3
> ^^^^^^^^^^^^^^^^^^^^^^^^
I think the "int" here can be removed, as done for
ARCH_MMAP_RND_BITS_MAX and ARCH_MMAP_RND_BITS_MIN
> > +
> > +config HAVE_KPROBES
> > + def_bool n
> > +
> > +menu "System setup"
> > +
> > +config POISON_INITMEM
> > + bool "Enable to poison freed initmem"
> > + default y
> > + help
> > + In order to debug initmem, using poison allows to verify if
> > + some data/code is stille using them. Enable this for debug
>
> still
>
> > + purposes
>
> purposes.
>
> > +
> > +config KVX_PHYS_OFFSET
> > + hex "RAM address of memory base"
> > + default 0x100000000
> > +
> > +config KVX_PAGE_OFFSET
> > + hex "kernel virtual address of memory base"
> > + default 0xFFFFFF8000000000
> > +
> > +config ARCH_FLATMEM_ENABLE
> > + def_bool y
> > +
> > +config ARCH_SPARSEMEM_ENABLE
> > + def_bool y
> > +
> > +config ARCH_SPARSEMEM_DEFAULT
> > + def_bool ARCH_SPARSEMEM_ENABLE
> > +
> > +config ARCH_SELECT_MEMORY_MODEL
> > + def_bool ARCH_SPARSEMEM_ENABLE
> > +
> > +config STACK_MAX_DEPTH_TO_PRINT
> > + int "Maximum depth of stack to print"
> > + range 1 128
> > + default "24"
> > +
> > +config L2_CACHE
> > + bool "Enable l2 cache driver"
>
> L2
>
> > + default y
> > + help
> > + Enable L2 cache driver in order to handle it. This driver can be
> > + disabled to eliminate any overhead related to L2 cache
> > + inval/writeback. Note that L2 cache is mandatory for SMP in order
> > + to maintain coherency on DDR between all PEs.
> > +
> > +config SECURE_DAME_HANDLING
> > + bool "Secure DAME handling"
> > + default y
> > + help
> > + In order to securely handle Data Asynchronous Memory Errors, we need
> > + to do a barrier upon kernel entry when coming from userspace. This
> > + barrier guarantee us that any pending DAME will be serviced right
>
> guarantees
>
> > + away. We also need to do a barrier when returning from kernel to user.
> > + This way, if the kernel or the user triggered a DAME, it will be
> > + serviced by knowing we are coming from kernel or user and avoid
> > + pulling the wrong lever (panic for kernel or sigfault for user).
> > + This can be costly but ensure that user cannot interfere with kernel.
>
> ensures
>
> > + /!\ Do not disable unless you want to open a giant breach between
> > + user and kernel /!\
> > +
> > +config CACHECTL_UNSAFE_PHYS_OPERATIONS
> > + bool "Enable cachectl syscall unsafe physical operations"
> > + default n
> > + help
> > + Enable cachectl syscall to allow writebacking/invalidating ranges
> > + based on physical addresses. These operations requires the
> > + CAP_SYS_ADMIN capability
>
> capability.
>
> > +
> > +config ENABLE_TCA
> > + bool "Enable TCA coprocessor support"
> > + default y
> > + help
> > + This option enables TCA coprocessor support. It will allow the user to
> > + use the coprocessor and save registers on context switch if used.
> > + Registers content will also be cleared when switching.
> > +
> > +config SMP
> > + bool "Symmetric multi-processing support"
> > + default n
> > + select GENERIC_SMP_IDLE_THREAD
> > + select GENERIC_IRQ_IPI
> > + select IRQ_DOMAIN_HIERARCHY
> > + select IRQ_DOMAIN
> > + select L2_CACHE
> > + help
> > + This enables support for systems with more than one CPU. If you have
> > + a system with only one CPU, say N. If you have a system with more
> > + than one CPU, say Y.
> > +
> > + If you say N here, the kernel will run on uni- and multiprocessor
> > + machines, but will use only one CPU of a multiprocessor machine. If
> > + you say Y here, the kernel will run on many, but not all,
> > + uniprocessor machines. On a uniprocessor machine, the kernel
> > + will run faster if you say N here.
> > +
> > +config NR_CPUS
> > + int "Maximum number of CPUs"
> > + range 1 16
> > + default "16"
> > + depends on SMP
> > + help
> > + Kalray support can handle a maximum of 16 CPUs.
> > +
> > +config KVX_PAGE_SHIFT
> > + int
> > + default 12
> > +
> > +config CMDLINE
> > + string "Default kernel command string"
> > + default ""
> > + help
> > + On some architectures there is currently no way for the boot loader
> > + to pass arguments to the kernel. For these architectures, you should
> > + supply some command-line options at build time by entering them
> > + here.
> > +
> > +endmenu
> > +
> > +menu "Kernel Features"
> > +source "kernel/Kconfig.hz"
> > +endmenu
> > diff --git a/arch/kvx/Kconfig.debug b/arch/kvx/Kconfig.debug
> > new file mode 100644
> > index 000000000000..027e919a1e14
> > --- /dev/null
> > +++ b/arch/kvx/Kconfig.debug
> > @@ -0,0 +1,70 @@
> > +menu "KVX debugging"
> > +
> > +config KVX_DEBUG_ASN
> > + bool "Check ASN before writing TLB entry"
> > + default n
> > + help
> > + This option allows to check if the ASN of the current
> > + process is matching the ASN found in MMC. If it is not the
>
> s/is matching/matches/
>
> > + case an error will be printed.
> > +
> > +config KVX_DEBUG_TLB_WRITE
> > + bool "Enable TLBs write checks"
> > + default n
> > + help
> > + Enabling this option will enable TLB access checks. This is
> > + particularly helpful when modifying the assembly code responsible
> > + of TLB refill. If set, mmc.e will be checked each time the tlb are
>
> for TLB refill. TLB
>
> > + written and a panic will be thrown on error.
> > +
> > +config KVX_DEBUG_TLB_ACCESS
> > + bool "Enable TLBs accesses logging"
> > + default n
> > + help
> > + Enabling this option will enable TLB entry manipulation logging.
> > + Each time an entry will be added to the TLBs, it will be logged in
>
> s/will be/is/ s/will be/is/
>
> > + an array readable via gdb scripts. This can be useful to understand
> > + strange crashes related to suspicious virtual/physical addresses.
> > +
> > +config KVX_DEBUG_TLB_ACCESS_BITS
> > + int "Number of bits used as index of entries in log table"
> > + default 12
> > + depends on KVX_DEBUG_TLB_ACCESS
> > + help
> > + Set the number of bits used as index of entries that will be logged
> > + in a ring buffer called kvx_tlb_access. One entry in the table
> > + contains registers TEL, TEH and MMC. It also logs the type of the
> > + operations (0:read, 1:write, 2:probe). Buffer is per CPU. For one
> > + entry 24 bytes are used. So by default it uses 96Ko of memory per
>
> What does "Ko" mean?
Ko stands for kilo-octet which is the french equivalent for kilo byte.
I've found more typos like this in the documentation and they will be
corrected as well.
> > + CPU to store 2^12 (4096) entries.
> > +
> > +config KVX_MMU_STATS
> > + bool "Register mmu stats debugfs entries"
>
> Preferably MMU
>
> > + default n
> > + select DEBUG_FS
>
> Preferably:
> depends on DEBUG_FS
ack
> > + help
> > + Enable debugfs attribute which will allow inspecting various metric
>
> metrics
>
> > + regarding MMU:
> > + - Number of nomapping traps handled
> > + - avg/min/max time for nomapping refill (user/kernel)
> > +
> > +config DEBUG_EXCEPTION_STACK
> > + bool "Enable exception stack debugging"
> > + default n
> > + help
> > + Enable stack check debugging when entering/exiting
> > + exception handlers.
> > + This can be particularly helpful after modifying stack
> > + handling to see if stack when exiting is the same as the one
> > + when entering exception handler.
> > +
> > +config DEBUG_SFR_SET_MASK
> > + bool "Enable sfr set_mask debugging"
> > + default n
> > + help
> > + Verify that values written using kvx_sfr_set_mask are matching the
>
> s/are matching/match/
>
> > + mask. This ensure that no extra bits of sfr will be overridden by some
>
> ensures
>
> and preferably s/sfr/SFR/ in 2 places (not in kvx_sfr_set_mask).
>
> > + incorrectly truncated values. This can lead to huge problems by
> > + modifying important bits in system registers.
> > +
> > +endmenu
Thanks, all the typos will be corrected :)
Jules
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 03/25] kvx: Add build infrastructure
2023-01-03 16:43 ` [RFC PATCH 03/25] kvx: Add build infrastructure Yann Sionneau
2023-01-03 17:29 ` Randy Dunlap
@ 2023-01-06 0:43 ` kernel test robot
1 sibling, 0 replies; 113+ messages in thread
From: kernel test robot @ 2023-01-06 0:43 UTC (permalink / raw)
To: Yann Sionneau; +Cc: oe-kbuild-all
[-- Attachment #1: Type: text/plain, Size: 1521 bytes --]
Hi Yann,
[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on tip/irq/core]
[also build test WARNING on kees/for-next/execve soc/for-next linus/master v6.2-rc2 next-20230105]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Yann-Sionneau/Documentation-kvx-Add-basic-documentation/20230104-010703
patch link: https://lore.kernel.org/r/20230103164359.24347-4-ysionneau%40kalray.eu
patch subject: [RFC PATCH 03/25] kvx: Add build infrastructure
reproduce:
# https://github.com/intel-lab-lkp/linux/commit/cc2343608420844de50f88e8bd02b4a69a50e353
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Yann-Sionneau/Documentation-kvx-Add-basic-documentation/20230104-010703
git checkout cc2343608420844de50f88e8bd02b4a69a50e353
make menuconfig
# enable CONFIG_COMPILE_TEST, CONFIG_WARN_MISSING_DOCUMENTS, CONFIG_WARN_ABI_ERRORS
make htmldocs
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> Warning: arch/kvx/Kconfig references a file that doesn't exist: Documentation/kbuild/kconfig-language.txt
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
[-- Attachment #2: config --]
[-- Type: text/plain, Size: 38842 bytes --]
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86_64 6.1.0-rc5 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="gcc-11 (Debian 11.3.0-8) 11.3.0"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=110300
CONFIG_CLANG_VERSION=0
CONFIG_AS_IS_GNU=y
CONFIG_AS_VERSION=23900
CONFIG_LD_IS_BFD=y
CONFIG_LD_VERSION=23900
CONFIG_LLD_VERSION=0
CONFIG_CC_CAN_LINK=y
CONFIG_CC_CAN_LINK_STATIC=y
CONFIG_CC_HAS_ASM_GOTO_OUTPUT=y
CONFIG_CC_HAS_ASM_INLINE=y
CONFIG_CC_HAS_NO_PROFILE_FN_ATTR=y
CONFIG_PAHOLE_VERSION=123
CONFIG_IRQ_WORK=y
CONFIG_BUILDTIME_TABLE_SORT=y
CONFIG_THREAD_INFO_IN_TASK=y
#
# General setup
#
CONFIG_BROKEN_ON_SMP=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_COMPILE_TEST=y
# CONFIG_WERROR is not set
CONFIG_LOCALVERSION=""
CONFIG_BUILD_SALT=""
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_HAVE_KERNEL_LZ4=y
CONFIG_HAVE_KERNEL_ZSTD=y
CONFIG_KERNEL_GZIP=y
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_LZO is not set
# CONFIG_KERNEL_LZ4 is not set
# CONFIG_KERNEL_ZSTD is not set
CONFIG_DEFAULT_INIT=""
CONFIG_DEFAULT_HOSTNAME="(none)"
# CONFIG_SYSVIPC is not set
# CONFIG_WATCH_QUEUE is not set
# CONFIG_CROSS_MEMORY_ATTACH is not set
# CONFIG_USELIB is not set
CONFIG_HAVE_ARCH_AUDITSYSCALL=y
#
# IRQ subsystem
#
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_HARDIRQS_SW_RESEND=y
CONFIG_IRQ_DOMAIN=y
CONFIG_IRQ_DOMAIN_HIERARCHY=y
CONFIG_GENERIC_IRQ_MATRIX_ALLOCATOR=y
CONFIG_GENERIC_IRQ_RESERVATION_MODE=y
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y
# end of IRQ subsystem
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_ARCH_CLOCKSOURCE_INIT=y
CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_HAVE_POSIX_CPU_TIMERS_TASK_WORK=y
CONFIG_POSIX_CPU_TIMERS_TASK_WORK=y
#
# Timers subsystem
#
CONFIG_HZ_PERIODIC=y
# CONFIG_NO_HZ_IDLE is not set
# CONFIG_NO_HZ is not set
# CONFIG_HIGH_RES_TIMERS is not set
CONFIG_CLOCKSOURCE_WATCHDOG_MAX_SKEW_US=100
# end of Timers subsystem
CONFIG_HAVE_EBPF_JIT=y
CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y
#
# BPF subsystem
#
# CONFIG_BPF_SYSCALL is not set
# end of BPF subsystem
CONFIG_PREEMPT_NONE_BUILD=y
CONFIG_PREEMPT_NONE=y
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set
# CONFIG_PREEMPT_DYNAMIC is not set
#
# CPU/Task time and stats accounting
#
CONFIG_TICK_CPU_ACCOUNTING=y
# CONFIG_VIRT_CPU_ACCOUNTING_GEN is not set
# CONFIG_IRQ_TIME_ACCOUNTING is not set
# CONFIG_BSD_PROCESS_ACCT is not set
# CONFIG_PSI is not set
# end of CPU/Task time and stats accounting
CONFIG_CPU_ISOLATION=y
#
# RCU Subsystem
#
CONFIG_TINY_RCU=y
# CONFIG_RCU_EXPERT is not set
CONFIG_SRCU=y
CONFIG_TINY_SRCU=y
# end of RCU Subsystem
# CONFIG_IKCONFIG is not set
# CONFIG_IKHEADERS is not set
CONFIG_LOG_BUF_SHIFT=17
CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=13
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
#
# Scheduler features
#
# end of Scheduler features
CONFIG_ARCH_SUPPORTS_NUMA_BALANCING=y
CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH=y
CONFIG_CC_HAS_INT128=y
CONFIG_CC_IMPLICIT_FALLTHROUGH="-Wimplicit-fallthrough=5"
CONFIG_GCC12_NO_ARRAY_BOUNDS=y
CONFIG_ARCH_SUPPORTS_INT128=y
# CONFIG_CGROUPS is not set
CONFIG_NAMESPACES=y
# CONFIG_UTS_NS is not set
# CONFIG_TIME_NS is not set
# CONFIG_USER_NS is not set
# CONFIG_PID_NS is not set
# CONFIG_CHECKPOINT_RESTORE is not set
# CONFIG_SCHED_AUTOGROUP is not set
# CONFIG_SYSFS_DEPRECATED is not set
# CONFIG_RELAY is not set
# CONFIG_BLK_DEV_INITRD is not set
# CONFIG_BOOT_CONFIG is not set
# CONFIG_INITRAMFS_PRESERVE_MTIME is not set
CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_LD_ORPHAN_WARN=y
CONFIG_SYSCTL=y
CONFIG_SYSCTL_EXCEPTION_TRACE=y
CONFIG_HAVE_PCSPKR_PLATFORM=y
# CONFIG_EXPERT is not set
CONFIG_MULTIUSER=y
CONFIG_SGETMASK_SYSCALL=y
CONFIG_SYSFS_SYSCALL=y
CONFIG_FHANDLE=y
CONFIG_POSIX_TIMERS=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_FUTEX_PI=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_IO_URING=y
CONFIG_ADVISE_SYSCALLS=y
CONFIG_MEMBARRIER=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_BASE_RELATIVE=y
CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE=y
CONFIG_RSEQ=y
# CONFIG_EMBEDDED is not set
CONFIG_HAVE_PERF_EVENTS=y
#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
# end of Kernel Performance Events And Counters
# CONFIG_PROFILING is not set
# end of General setup
CONFIG_64BIT=y
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_MMU=y
CONFIG_ARCH_MMAP_RND_BITS_MIN=28
CONFIG_ARCH_MMAP_RND_BITS_MAX=32
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MIN=8
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MAX=16
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_NR_GPIO=1024
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_AUDIT_ARCH=y
CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_PGTABLE_LEVELS=4
CONFIG_CC_HAS_SANE_STACKPROTECTOR=y
#
# Processor type and features
#
# CONFIG_SMP is not set
CONFIG_X86_FEATURE_NAMES=y
CONFIG_X86_MPPARSE=y
# CONFIG_GOLDFISH is not set
# CONFIG_X86_CPU_RESCTRL is not set
# CONFIG_X86_EXTENDED_PLATFORM is not set
# CONFIG_SCHED_OMIT_FRAME_POINTER is not set
# CONFIG_HYPERVISOR_GUEST is not set
# CONFIG_MK8 is not set
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_MATOM is not set
CONFIG_GENERIC_CPU=y
CONFIG_X86_INTERNODE_CACHE_SHIFT=6
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_IA32_FEAT_CTL=y
CONFIG_X86_VMX_FEATURE_NAMES=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_AMD=y
CONFIG_CPU_SUP_HYGON=y
CONFIG_CPU_SUP_CENTAUR=y
CONFIG_CPU_SUP_ZHAOXIN=y
CONFIG_HPET_TIMER=y
CONFIG_DMI=y
CONFIG_NR_CPUS_RANGE_BEGIN=1
CONFIG_NR_CPUS_RANGE_END=1
CONFIG_NR_CPUS_DEFAULT=1
CONFIG_NR_CPUS=1
CONFIG_UP_LATE_INIT=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
# CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS is not set
# CONFIG_X86_MCE is not set
#
# Performance monitoring
#
# CONFIG_PERF_EVENTS_AMD_POWER is not set
# CONFIG_PERF_EVENTS_AMD_UNCORE is not set
# CONFIG_PERF_EVENTS_AMD_BRS is not set
# end of Performance monitoring
CONFIG_X86_16BIT=y
CONFIG_X86_ESPFIX64=y
CONFIG_X86_VSYSCALL_EMULATION=y
# CONFIG_X86_IOPL_IOPERM is not set
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set
# CONFIG_X86_5LEVEL is not set
CONFIG_X86_DIRECT_GBPAGES=y
# CONFIG_AMD_MEM_ENCRYPT is not set
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
# CONFIG_X86_CHECK_BIOS_CORRUPTION is not set
CONFIG_MTRR=y
# CONFIG_MTRR_SANITIZER is not set
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
CONFIG_X86_UMIP=y
CONFIG_CC_HAS_IBT=y
# CONFIG_X86_KERNEL_IBT is not set
# CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS is not set
CONFIG_X86_INTEL_TSX_MODE_OFF=y
# CONFIG_X86_INTEL_TSX_MODE_ON is not set
# CONFIG_X86_INTEL_TSX_MODE_AUTO is not set
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
# CONFIG_KEXEC is not set
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x1000000
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_ALIGN=0x200000
CONFIG_LEGACY_VSYSCALL_XONLY=y
# CONFIG_LEGACY_VSYSCALL_NONE is not set
# CONFIG_CMDLINE_BOOL is not set
CONFIG_MODIFY_LDT_SYSCALL=y
# CONFIG_STRICT_SIGALTSTACK_SIZE is not set
CONFIG_HAVE_LIVEPATCH=y
# end of Processor type and features
CONFIG_CC_HAS_SLS=y
CONFIG_CC_HAS_RETURN_THUNK=y
# CONFIG_SPECULATION_MITIGATIONS is not set
CONFIG_ARCH_HAS_ADD_PAGES=y
CONFIG_ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE=y
#
# Power management and ACPI options
#
# CONFIG_SUSPEND is not set
# CONFIG_PM is not set
CONFIG_ARCH_SUPPORTS_ACPI=y
# CONFIG_ACPI is not set
#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set
# end of CPU Frequency scaling
#
# CPU Idle
#
# CONFIG_CPU_IDLE is not set
# end of CPU Idle
# end of Power management and ACPI options
#
# Bus options (PCI etc.)
#
CONFIG_ISA_DMA_API=y
# end of Bus options (PCI etc.)
#
# Binary Emulations
#
# CONFIG_IA32_EMULATION is not set
# CONFIG_X86_X32_ABI is not set
# end of Binary Emulations
CONFIG_HAVE_KVM=y
# CONFIG_VIRTUALIZATION is not set
CONFIG_AS_AVX512=y
CONFIG_AS_SHA1_NI=y
CONFIG_AS_SHA256_NI=y
CONFIG_AS_TPAUSE=y
#
# General architecture-dependent options
#
CONFIG_GENERIC_ENTRY=y
# CONFIG_JUMP_LABEL is not set
# CONFIG_STATIC_CALL_SELFTEST is not set
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_ARCH_USE_BUILTIN_BSWAP=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_KPROBES_ON_FTRACE=y
CONFIG_ARCH_CORRECT_STACKTRACE_ON_KRETPROBE=y
CONFIG_HAVE_FUNCTION_ERROR_INJECTION=y
CONFIG_HAVE_NMI=y
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_TRACE_IRQFLAGS_NMI_SUPPORT=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_CONTIGUOUS=y
CONFIG_GENERIC_SMP_IDLE_THREAD=y
CONFIG_ARCH_HAS_FORTIFY_SOURCE=y
CONFIG_ARCH_HAS_SET_MEMORY=y
CONFIG_ARCH_HAS_SET_DIRECT_MAP=y
CONFIG_HAVE_ARCH_THREAD_STRUCT_WHITELIST=y
CONFIG_ARCH_WANTS_DYNAMIC_TASK_STRUCT=y
CONFIG_ARCH_WANTS_NO_INSTR=y
CONFIG_HAVE_ASM_MODVERSIONS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_RSEQ=y
CONFIG_HAVE_RUST=y
CONFIG_HAVE_FUNCTION_ARG_ACCESS_API=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_HARDLOCKUP_DETECTOR_PERF=y
CONFIG_HAVE_PERF_REGS=y
CONFIG_HAVE_PERF_USER_STACK_DUMP=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y
CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE=y
CONFIG_MMU_GATHER_MERGE_VMAS=y
CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG=y
CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y
CONFIG_HAVE_CMPXCHG_LOCAL=y
CONFIG_HAVE_CMPXCHG_DOUBLE=y
CONFIG_HAVE_ARCH_SECCOMP=y
CONFIG_HAVE_ARCH_SECCOMP_FILTER=y
# CONFIG_SECCOMP is not set
CONFIG_HAVE_ARCH_STACKLEAK=y
CONFIG_HAVE_STACKPROTECTOR=y
# CONFIG_STACKPROTECTOR is not set
CONFIG_ARCH_SUPPORTS_LTO_CLANG=y
CONFIG_ARCH_SUPPORTS_LTO_CLANG_THIN=y
CONFIG_LTO_NONE=y
CONFIG_ARCH_SUPPORTS_CFI_CLANG=y
CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES=y
CONFIG_HAVE_CONTEXT_TRACKING_USER=y
CONFIG_HAVE_CONTEXT_TRACKING_USER_OFFSTACK=y
CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_HAVE_IRQ_TIME_ACCOUNTING=y
CONFIG_HAVE_MOVE_PUD=y
CONFIG_HAVE_MOVE_PMD=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD=y
CONFIG_HAVE_ARCH_HUGE_VMAP=y
CONFIG_HAVE_ARCH_HUGE_VMALLOC=y
CONFIG_ARCH_WANT_HUGE_PMD_SHARE=y
CONFIG_HAVE_ARCH_SOFT_DIRTY=y
CONFIG_HAVE_MOD_ARCH_SPECIFIC=y
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK=y
CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK=y
CONFIG_SOFTIRQ_ON_OWN_STACK=y
CONFIG_ARCH_HAS_ELF_RANDOMIZE=y
CONFIG_HAVE_ARCH_MMAP_RND_BITS=y
CONFIG_HAVE_EXIT_THREAD=y
CONFIG_ARCH_MMAP_RND_BITS=28
CONFIG_PAGE_SIZE_LESS_THAN_64KB=y
CONFIG_PAGE_SIZE_LESS_THAN_256KB=y
CONFIG_HAVE_OBJTOOL=y
CONFIG_HAVE_JUMP_LABEL_HACK=y
CONFIG_HAVE_NOINSTR_HACK=y
CONFIG_HAVE_NOINSTR_VALIDATION=y
CONFIG_HAVE_UACCESS_VALIDATION=y
CONFIG_HAVE_STACK_VALIDATION=y
CONFIG_HAVE_RELIABLE_STACKTRACE=y
# CONFIG_COMPAT_32BIT_TIME is not set
CONFIG_HAVE_ARCH_VMAP_STACK=y
# CONFIG_VMAP_STACK is not set
CONFIG_HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET=y
CONFIG_RANDOMIZE_KSTACK_OFFSET=y
# CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT is not set
CONFIG_ARCH_HAS_STRICT_KERNEL_RWX=y
CONFIG_STRICT_KERNEL_RWX=y
CONFIG_ARCH_HAS_STRICT_MODULE_RWX=y
CONFIG_HAVE_ARCH_PREL32_RELOCATIONS=y
CONFIG_ARCH_HAS_MEM_ENCRYPT=y
CONFIG_HAVE_STATIC_CALL=y
CONFIG_HAVE_STATIC_CALL_INLINE=y
CONFIG_HAVE_PREEMPT_DYNAMIC=y
CONFIG_HAVE_PREEMPT_DYNAMIC_CALL=y
CONFIG_ARCH_WANT_LD_ORPHAN_WARN=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_ARCH_SUPPORTS_PAGE_TABLE_CHECK=y
CONFIG_ARCH_HAS_ELFCORE_COMPAT=y
CONFIG_ARCH_HAS_PARANOID_L1D_FLUSH=y
CONFIG_DYNAMIC_SIGFRAME=y
CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG=y
#
# GCOV-based kernel profiling
#
CONFIG_ARCH_HAS_GCOV_PROFILE_ALL=y
# end of GCOV-based kernel profiling
CONFIG_HAVE_GCC_PLUGINS=y
# CONFIG_GCC_PLUGINS is not set
# end of General architecture-dependent options
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
# CONFIG_MODULES is not set
CONFIG_BLOCK=y
# CONFIG_BLOCK_LEGACY_AUTOLOAD is not set
# CONFIG_BLK_DEV_BSGLIB is not set
# CONFIG_BLK_DEV_INTEGRITY is not set
# CONFIG_BLK_DEV_ZONED is not set
# CONFIG_BLK_WBT is not set
# CONFIG_BLK_SED_OPAL is not set
# CONFIG_BLK_INLINE_ENCRYPTION is not set
#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y
CONFIG_EFI_PARTITION=y
# end of Partition Types
#
# IO Schedulers
#
# CONFIG_MQ_IOSCHED_DEADLINE is not set
# CONFIG_MQ_IOSCHED_KYBER is not set
# CONFIG_IOSCHED_BFQ is not set
# end of IO Schedulers
CONFIG_INLINE_SPIN_UNLOCK_IRQ=y
CONFIG_INLINE_READ_UNLOCK=y
CONFIG_INLINE_READ_UNLOCK_IRQ=y
CONFIG_INLINE_WRITE_UNLOCK=y
CONFIG_INLINE_WRITE_UNLOCK_IRQ=y
CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y
CONFIG_ARCH_USE_QUEUED_SPINLOCKS=y
CONFIG_ARCH_USE_QUEUED_RWLOCKS=y
CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE=y
CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE=y
CONFIG_ARCH_HAS_SYSCALL_WRAPPER=y
#
# Executable file formats
#
# CONFIG_BINFMT_ELF is not set
# CONFIG_BINFMT_SCRIPT is not set
# CONFIG_BINFMT_MISC is not set
CONFIG_COREDUMP=y
# end of Executable file formats
#
# Memory Management options
#
# CONFIG_SWAP is not set
#
# SLAB allocator options
#
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLAB_MERGE_DEFAULT is not set
# CONFIG_SLAB_FREELIST_RANDOM is not set
# CONFIG_SLAB_FREELIST_HARDENED is not set
# CONFIG_SLUB_STATS is not set
# end of SLAB allocator options
# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
# CONFIG_COMPAT_BRK is not set
CONFIG_SPARSEMEM=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
# CONFIG_SPARSEMEM_VMEMMAP is not set
CONFIG_HAVE_FAST_GUP=y
CONFIG_EXCLUSIVE_SYSTEM_RAM=y
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
# CONFIG_MEMORY_HOTPLUG is not set
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK=y
# CONFIG_COMPACTION is not set
# CONFIG_PAGE_REPORTING is not set
CONFIG_PHYS_ADDR_T_64BIT=y
# CONFIG_KSM is not set
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
CONFIG_ARCH_WANTS_THP_SWAP=y
# CONFIG_TRANSPARENT_HUGEPAGE is not set
CONFIG_NEED_PER_CPU_KM=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
# CONFIG_CMA is not set
CONFIG_GENERIC_EARLY_IOREMAP=y
# CONFIG_IDLE_PAGE_TRACKING is not set
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_ARCH_HAS_CURRENT_STACK_POINTER=y
CONFIG_ARCH_HAS_PTE_DEVMAP=y
CONFIG_ZONE_DMA=y
CONFIG_ZONE_DMA32=y
CONFIG_VM_EVENT_COUNTERS=y
# CONFIG_PERCPU_STATS is not set
#
# GUP_TEST needs to have DEBUG_FS enabled
#
CONFIG_ARCH_HAS_PTE_SPECIAL=y
CONFIG_SECRETMEM=y
# CONFIG_ANON_VMA_NAME is not set
# CONFIG_USERFAULTFD is not set
# CONFIG_LRU_GEN is not set
#
# Data Access Monitoring
#
# CONFIG_DAMON is not set
# end of Data Access Monitoring
# end of Memory Management options
# CONFIG_NET is not set
#
# Device Drivers
#
CONFIG_HAVE_EISA=y
# CONFIG_EISA is not set
CONFIG_HAVE_PCI=y
# CONFIG_PCI is not set
# CONFIG_PCCARD is not set
#
# Generic Driver Options
#
# CONFIG_UEVENT_HELPER is not set
# CONFIG_DEVTMPFS is not set
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
#
# Firmware loader
#
CONFIG_FW_LOADER=y
CONFIG_EXTRA_FIRMWARE=""
# CONFIG_FW_LOADER_USER_HELPER is not set
# CONFIG_FW_LOADER_COMPRESS is not set
# CONFIG_FW_UPLOAD is not set
# end of Firmware loader
CONFIG_ALLOW_DEV_COREDUMP=y
CONFIG_GENERIC_CPU_AUTOPROBE=y
CONFIG_GENERIC_CPU_VULNERABILITIES=y
# end of Generic Driver Options
#
# Bus devices
#
# CONFIG_ARM_INTEGRATOR_LM is not set
# CONFIG_BT1_APB is not set
# CONFIG_BT1_AXI is not set
# CONFIG_HISILICON_LPC is not set
# CONFIG_INTEL_IXP4XX_EB is not set
# CONFIG_QCOM_EBI2 is not set
# CONFIG_MHI_BUS is not set
# CONFIG_MHI_BUS_EP is not set
# end of Bus devices
#
# Firmware Drivers
#
#
# ARM System Control and Management Interface Protocol
#
# CONFIG_ARM_SCMI_PROTOCOL is not set
# end of ARM System Control and Management Interface Protocol
# CONFIG_EDD is not set
CONFIG_FIRMWARE_MEMMAP=y
# CONFIG_DMIID is not set
# CONFIG_DMI_SYSFS is not set
CONFIG_DMI_SCAN_MACHINE_NON_EFI_FALLBACK=y
# CONFIG_FW_CFG_SYSFS is not set
# CONFIG_SYSFB_SIMPLEFB is not set
# CONFIG_BCM47XX_NVRAM is not set
# CONFIG_GOOGLE_FIRMWARE is not set
#
# Tegra firmware driver
#
# end of Tegra firmware driver
# end of Firmware Drivers
# CONFIG_GNSS is not set
# CONFIG_MTD is not set
# CONFIG_OF is not set
CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y
# CONFIG_PARPORT is not set
# CONFIG_BLK_DEV is not set
#
# NVME Support
#
# CONFIG_NVME_FC is not set
# end of NVME Support
#
# Misc devices
#
# CONFIG_DUMMY_IRQ is not set
# CONFIG_ATMEL_SSC is not set
# CONFIG_ENCLOSURE_SERVICES is not set
# CONFIG_QCOM_COINCELL is not set
# CONFIG_SRAM is not set
# CONFIG_XILINX_SDFEC is not set
# CONFIG_C2PORT is not set
#
# EEPROM support
#
# CONFIG_EEPROM_93CX6 is not set
# end of EEPROM support
#
# Texas Instruments shared transport line discipline
#
# end of Texas Instruments shared transport line discipline
#
# Altera FPGA firmware download module (requires I2C)
#
# CONFIG_ECHO is not set
# CONFIG_PVPANIC is not set
# end of Misc devices
#
# SCSI device support
#
CONFIG_SCSI_MOD=y
# CONFIG_RAID_ATTRS is not set
# CONFIG_SCSI is not set
# end of SCSI device support
# CONFIG_ATA is not set
# CONFIG_MD is not set
# CONFIG_TARGET_CORE is not set
#
# IEEE 1394 (FireWire) support
#
# CONFIG_FIREWIRE is not set
# end of IEEE 1394 (FireWire) support
# CONFIG_MACINTOSH_DRIVERS is not set
#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_FF_MEMLESS is not set
# CONFIG_INPUT_SPARSEKMAP is not set
# CONFIG_INPUT_MATRIXKMAP is not set
#
# Userland interfaces
#
# CONFIG_INPUT_MOUSEDEV is not set
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set
#
# Input Device Drivers
#
# CONFIG_INPUT_KEYBOARD is not set
# CONFIG_INPUT_MOUSE is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set
# CONFIG_RMI4_CORE is not set
#
# Hardware I/O ports
#
# CONFIG_SERIO is not set
CONFIG_ARCH_MIGHT_HAVE_PC_SERIO=y
# CONFIG_GAMEPORT is not set
# end of Hardware I/O ports
# end of Input device support
#
# Character devices
#
CONFIG_TTY=y
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_VT_HW_CONSOLE_BINDING is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_LDISC_AUTOLOAD is not set
#
# Serial drivers
#
# CONFIG_SERIAL_8250 is not set
#
# Non-8250 serial port support
#
# CONFIG_SERIAL_AMBA_PL010 is not set
# CONFIG_SERIAL_MESON is not set
# CONFIG_SERIAL_CLPS711X is not set
# CONFIG_SERIAL_SAMSUNG is not set
# CONFIG_SERIAL_TEGRA is not set
# CONFIG_SERIAL_IMX is not set
# CONFIG_SERIAL_UARTLITE is not set
# CONFIG_SERIAL_SH_SCI is not set
# CONFIG_SERIAL_MSM is not set
# CONFIG_SERIAL_VT8500 is not set
# CONFIG_SERIAL_OMAP is not set
# CONFIG_SERIAL_LANTIQ is not set
# CONFIG_SERIAL_SCCNXP is not set
# CONFIG_SERIAL_TIMBERDALE is not set
# CONFIG_SERIAL_BCM63XX is not set
# CONFIG_SERIAL_ALTERA_JTAGUART is not set
# CONFIG_SERIAL_ALTERA_UART is not set
# CONFIG_SERIAL_MXS_AUART is not set
# CONFIG_SERIAL_MPS2_UART is not set
# CONFIG_SERIAL_ARC is not set
# CONFIG_SERIAL_FSL_LPUART is not set
# CONFIG_SERIAL_FSL_LINFLEXUART is not set
# CONFIG_SERIAL_ST_ASC is not set
# CONFIG_SERIAL_STM32 is not set
# CONFIG_SERIAL_OWL is not set
# CONFIG_SERIAL_RDA is not set
# CONFIG_SERIAL_LITEUART is not set
# CONFIG_SERIAL_SUNPLUS is not set
# end of Serial drivers
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_NULL_TTY is not set
# CONFIG_SERIAL_DEV_BUS is not set
# CONFIG_VIRTIO_CONSOLE is not set
# CONFIG_IPMI_HANDLER is not set
# CONFIG_ASPEED_KCS_IPMI_BMC is not set
# CONFIG_NPCM7XX_KCS_IPMI_BMC is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_MWAVE is not set
# CONFIG_DEVMEM is not set
# CONFIG_NVRAM is not set
# CONFIG_HANGCHECK_TIMER is not set
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
# CONFIG_RANDOM_TRUST_CPU is not set
# CONFIG_RANDOM_TRUST_BOOTLOADER is not set
# end of Character devices
#
# I2C support
#
# CONFIG_I2C is not set
# end of I2C support
# CONFIG_I3C is not set
# CONFIG_SPI is not set
# CONFIG_SPMI is not set
# CONFIG_HSI is not set
# CONFIG_PPS is not set
#
# PTP clock support
#
CONFIG_PTP_1588_CLOCK_OPTIONAL=y
#
# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
#
# end of PTP clock support
# CONFIG_PINCTRL is not set
# CONFIG_GPIOLIB is not set
# CONFIG_W1 is not set
# CONFIG_POWER_RESET is not set
# CONFIG_POWER_SUPPLY is not set
# CONFIG_HWMON is not set
# CONFIG_THERMAL is not set
# CONFIG_WATCHDOG is not set
CONFIG_SSB_POSSIBLE=y
# CONFIG_SSB is not set
CONFIG_BCMA_POSSIBLE=y
# CONFIG_BCMA is not set
#
# Multifunction device drivers
#
# CONFIG_MFD_SUN4I_GPADC is not set
# CONFIG_MFD_AT91_USART is not set
# CONFIG_MFD_MADERA is not set
# CONFIG_MFD_EXYNOS_LPASS is not set
# CONFIG_MFD_MXS_LRADC is not set
# CONFIG_MFD_MX25_TSADC is not set
# CONFIG_HTC_PASIC3 is not set
# CONFIG_MFD_KEMPLD is not set
# CONFIG_MFD_MT6397 is not set
# CONFIG_MFD_PM8XXX is not set
# CONFIG_MFD_SM501 is not set
# CONFIG_ABX500_CORE is not set
# CONFIG_MFD_SUN6I_PRCM is not set
# CONFIG_MFD_SYSCON is not set
# CONFIG_MFD_TI_AM335X_TSCADC is not set
# CONFIG_MFD_TQMX86 is not set
# CONFIG_MFD_STM32_LPTIMER is not set
# CONFIG_MFD_STM32_TIMERS is not set
# end of Multifunction device drivers
# CONFIG_REGULATOR is not set
# CONFIG_RC_CORE is not set
#
# CEC support
#
# CONFIG_MEDIA_CEC_SUPPORT is not set
# end of CEC support
# CONFIG_MEDIA_SUPPORT is not set
#
# Graphics support
#
# CONFIG_IMX_IPUV3_CORE is not set
# CONFIG_DRM is not set
#
# ARM devices
#
# end of ARM devices
#
# Frame buffer Devices
#
# CONFIG_FB is not set
# CONFIG_MMP_DISP is not set
# end of Frame buffer Devices
#
# Backlight & LCD device support
#
# CONFIG_LCD_CLASS_DEVICE is not set
# CONFIG_BACKLIGHT_CLASS_DEVICE is not set
# end of Backlight & LCD device support
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y
CONFIG_DUMMY_CONSOLE_COLUMNS=80
CONFIG_DUMMY_CONSOLE_ROWS=25
# end of Console display driver support
# end of Graphics support
# CONFIG_SOUND is not set
#
# HID support
#
# CONFIG_HID is not set
# end of HID support
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
# CONFIG_USB_SUPPORT is not set
# CONFIG_MMC is not set
# CONFIG_MEMSTICK is not set
# CONFIG_NEW_LEDS is not set
# CONFIG_ACCESSIBILITY is not set
CONFIG_EDAC_ATOMIC_SCRUB=y
CONFIG_EDAC_SUPPORT=y
CONFIG_RTC_LIB=y
CONFIG_RTC_MC146818_LIB=y
# CONFIG_RTC_CLASS is not set
# CONFIG_DMADEVICES is not set
#
# DMABUF options
#
# CONFIG_SYNC_FILE is not set
# CONFIG_DMABUF_HEAPS is not set
# end of DMABUF options
# CONFIG_AUXDISPLAY is not set
# CONFIG_UIO is not set
# CONFIG_VFIO is not set
# CONFIG_VIRT_DRIVERS is not set
# CONFIG_VIRTIO_MENU is not set
# CONFIG_VHOST_MENU is not set
#
# Microsoft Hyper-V guest support
#
# end of Microsoft Hyper-V guest support
# CONFIG_GREYBUS is not set
# CONFIG_COMEDI is not set
# CONFIG_STAGING is not set
# CONFIG_CHROME_PLATFORMS is not set
# CONFIG_MELLANOX_PLATFORM is not set
# CONFIG_OLPC_XO175 is not set
# CONFIG_SURFACE_PLATFORMS is not set
# CONFIG_X86_PLATFORM_DEVICES is not set
# CONFIG_COMMON_CLK is not set
# CONFIG_HWSPINLOCK is not set
#
# Clock Source drivers
#
CONFIG_CLKEVT_I8253=y
CONFIG_I8253_LOCK=y
CONFIG_CLKBLD_I8253=y
# CONFIG_BCM2835_TIMER is not set
# CONFIG_BCM_KONA_TIMER is not set
# CONFIG_DAVINCI_TIMER is not set
# CONFIG_DIGICOLOR_TIMER is not set
# CONFIG_OMAP_DM_TIMER is not set
# CONFIG_DW_APB_TIMER is not set
# CONFIG_FTTMR010_TIMER is not set
# CONFIG_IXP4XX_TIMER is not set
# CONFIG_MESON6_TIMER is not set
# CONFIG_OWL_TIMER is not set
# CONFIG_RDA_TIMER is not set
# CONFIG_SUN4I_TIMER is not set
# CONFIG_TEGRA_TIMER is not set
# CONFIG_VT8500_TIMER is not set
# CONFIG_NPCM7XX_TIMER is not set
# CONFIG_ASM9260_TIMER is not set
# CONFIG_CLKSRC_DBX500_PRCMU is not set
# CONFIG_CLPS711X_TIMER is not set
# CONFIG_MXS_TIMER is not set
# CONFIG_NSPIRE_TIMER is not set
# CONFIG_INTEGRATOR_AP_TIMER is not set
# CONFIG_CLKSRC_PISTACHIO is not set
# CONFIG_CLKSRC_STM32_LP is not set
# CONFIG_ARMV7M_SYSTICK is not set
# CONFIG_ATMEL_PIT is not set
# CONFIG_ATMEL_ST is not set
# CONFIG_CLKSRC_SAMSUNG_PWM is not set
# CONFIG_FSL_FTM_TIMER is not set
# CONFIG_OXNAS_RPS_TIMER is not set
# CONFIG_MTK_TIMER is not set
# CONFIG_SH_TIMER_CMT is not set
# CONFIG_SH_TIMER_MTU2 is not set
# CONFIG_RENESAS_OSTM is not set
# CONFIG_SH_TIMER_TMU is not set
# CONFIG_EM_TIMER_STI is not set
# CONFIG_CLKSRC_PXA is not set
# CONFIG_TIMER_IMX_SYS_CTR is not set
# CONFIG_CLKSRC_ST_LPC is not set
# CONFIG_GXP_TIMER is not set
# CONFIG_MSC313E_TIMER is not set
# CONFIG_MICROCHIP_PIT64B is not set
# end of Clock Source drivers
# CONFIG_MAILBOX is not set
# CONFIG_IOMMU_SUPPORT is not set
#
# Remoteproc drivers
#
# CONFIG_REMOTEPROC is not set
# end of Remoteproc drivers
#
# Rpmsg drivers
#
# CONFIG_RPMSG_VIRTIO is not set
# end of Rpmsg drivers
#
# SOC (System On Chip) specific Drivers
#
#
# Amlogic SoC drivers
#
# CONFIG_MESON_CANVAS is not set
# CONFIG_MESON_CLK_MEASURE is not set
# CONFIG_MESON_GX_SOCINFO is not set
# CONFIG_MESON_MX_SOCINFO is not set
# end of Amlogic SoC drivers
#
# Apple SoC drivers
#
# CONFIG_APPLE_SART is not set
# end of Apple SoC drivers
#
# ASPEED SoC drivers
#
# CONFIG_ASPEED_LPC_CTRL is not set
# CONFIG_ASPEED_LPC_SNOOP is not set
# CONFIG_ASPEED_UART_ROUTING is not set
# CONFIG_ASPEED_P2A_CTRL is not set
# CONFIG_ASPEED_SOCINFO is not set
# end of ASPEED SoC drivers
# CONFIG_AT91_SOC_ID is not set
# CONFIG_AT91_SOC_SFR is not set
#
# Broadcom SoC drivers
#
# CONFIG_SOC_BCM63XX is not set
# CONFIG_SOC_BRCMSTB is not set
# end of Broadcom SoC drivers
#
# NXP/Freescale QorIQ SoC drivers
#
# end of NXP/Freescale QorIQ SoC drivers
#
# fujitsu SoC drivers
#
# end of fujitsu SoC drivers
#
# i.MX SoC drivers
#
# CONFIG_SOC_IMX8M is not set
# CONFIG_SOC_IMX9 is not set
# end of i.MX SoC drivers
#
# IXP4xx SoC drivers
#
# CONFIG_IXP4XX_QMGR is not set
# CONFIG_IXP4XX_NPE is not set
# end of IXP4xx SoC drivers
#
# Enable LiteX SoC Builder specific drivers
#
# CONFIG_LITEX_SOC_CONTROLLER is not set
# end of Enable LiteX SoC Builder specific drivers
#
# MediaTek SoC drivers
#
# CONFIG_MTK_CMDQ is not set
# CONFIG_MTK_DEVAPC is not set
# CONFIG_MTK_INFRACFG is not set
# CONFIG_MTK_MMSYS is not set
# end of MediaTek SoC drivers
#
# Qualcomm SoC drivers
#
# CONFIG_QCOM_GENI_SE is not set
# CONFIG_QCOM_GSBI is not set
# CONFIG_QCOM_LLCC is not set
# CONFIG_QCOM_RPMH is not set
# CONFIG_QCOM_SPM is not set
# CONFIG_QCOM_ICC_BWMON is not set
# end of Qualcomm SoC drivers
# CONFIG_SOC_RENESAS is not set
# CONFIG_ROCKCHIP_GRF is not set
# CONFIG_SOC_SAMSUNG is not set
# CONFIG_SOC_TI is not set
# CONFIG_UX500_SOC_ID is not set
#
# Xilinx SoC drivers
#
# end of Xilinx SoC drivers
# end of SOC (System On Chip) specific Drivers
# CONFIG_PM_DEVFREQ is not set
# CONFIG_EXTCON is not set
# CONFIG_MEMORY is not set
# CONFIG_IIO is not set
# CONFIG_PWM is not set
#
# IRQ chip support
#
# CONFIG_RENESAS_INTC_IRQPIN is not set
# CONFIG_RENESAS_IRQC is not set
# CONFIG_RENESAS_RZA1_IRQC is not set
# CONFIG_RENESAS_RZG2L_IRQC is not set
# CONFIG_SL28CPLD_INTC is not set
# CONFIG_TS4800_IRQ is not set
# CONFIG_INGENIC_TCU_IRQ is not set
# CONFIG_IRQ_UNIPHIER_AIDET is not set
# CONFIG_MESON_IRQ_GPIO is not set
# CONFIG_IMX_IRQSTEER is not set
# CONFIG_IMX_INTMUX is not set
# CONFIG_EXYNOS_IRQ_COMBINER is not set
# CONFIG_MST_IRQ is not set
# CONFIG_MCHP_EIC is not set
# CONFIG_SUNPLUS_SP7021_INTC is not set
# end of IRQ chip support
# CONFIG_IPACK_BUS is not set
# CONFIG_RESET_CONTROLLER is not set
#
# PHY Subsystem
#
# CONFIG_GENERIC_PHY is not set
# CONFIG_PHY_PISTACHIO_USB is not set
# CONFIG_PHY_CAN_TRANSCEIVER is not set
#
# PHY drivers for Broadcom platforms
#
# CONFIG_PHY_BCM63XX_USBH is not set
# CONFIG_BCM_KONA_USB2_PHY is not set
# end of PHY drivers for Broadcom platforms
# CONFIG_PHY_HI6220_USB is not set
# CONFIG_PHY_HI3660_USB is not set
# CONFIG_PHY_HI3670_USB is not set
# CONFIG_PHY_HI3670_PCIE is not set
# CONFIG_PHY_HISTB_COMBPHY is not set
# CONFIG_PHY_HISI_INNO_USB2 is not set
# CONFIG_PHY_PXA_28NM_HSIC is not set
# CONFIG_PHY_PXA_28NM_USB2 is not set
# CONFIG_PHY_PXA_USB is not set
# CONFIG_PHY_MMP3_USB is not set
# CONFIG_PHY_MMP3_HSIC is not set
# CONFIG_PHY_MT7621_PCI is not set
# CONFIG_PHY_RALINK_USB is not set
# CONFIG_PHY_RCAR_GEN3_USB3 is not set
# CONFIG_PHY_ROCKCHIP_DPHY_RX0 is not set
# CONFIG_PHY_ROCKCHIP_PCIE is not set
# CONFIG_PHY_ROCKCHIP_SNPS_PCIE3 is not set
# CONFIG_PHY_EXYNOS_MIPI_VIDEO is not set
# CONFIG_PHY_SAMSUNG_USB2 is not set
# CONFIG_PHY_ST_SPEAR1310_MIPHY is not set
# CONFIG_PHY_ST_SPEAR1340_MIPHY is not set
# CONFIG_PHY_TEGRA194_P2U is not set
# CONFIG_PHY_DA8XX_USB is not set
# CONFIG_OMAP_CONTROL_PHY is not set
# CONFIG_TI_PIPE3 is not set
# CONFIG_PHY_INTEL_KEEMBAY_EMMC is not set
# CONFIG_PHY_INTEL_KEEMBAY_USB is not set
# CONFIG_PHY_INTEL_LGM_EMMC is not set
# CONFIG_PHY_XILINX_ZYNQMP is not set
# end of PHY Subsystem
# CONFIG_POWERCAP is not set
# CONFIG_MCB is not set
#
# Performance monitor support
#
# CONFIG_ARM_CCN is not set
# CONFIG_ARM_CMN is not set
# CONFIG_FSL_IMX8_DDR_PMU is not set
# CONFIG_XGENE_PMU is not set
# CONFIG_ARM_DMC620_PMU is not set
# CONFIG_MARVELL_CN10K_TAD_PMU is not set
# CONFIG_ALIBABA_UNCORE_DRW_PMU is not set
# CONFIG_MARVELL_CN10K_DDR_PMU is not set
# end of Performance monitor support
# CONFIG_RAS is not set
#
# Android
#
# CONFIG_ANDROID_BINDER_IPC is not set
# end of Android
# CONFIG_DAX is not set
# CONFIG_NVMEM is not set
#
# HW tracing support
#
# CONFIG_STM is not set
# CONFIG_INTEL_TH is not set
# end of HW tracing support
# CONFIG_FPGA is not set
# CONFIG_TEE is not set
# CONFIG_SIOX is not set
# CONFIG_SLIMBUS is not set
# CONFIG_INTERCONNECT is not set
# CONFIG_COUNTER is not set
# CONFIG_PECI is not set
# CONFIG_HTE is not set
# end of Device Drivers
#
# File systems
#
CONFIG_DCACHE_WORD_ACCESS=y
# CONFIG_VALIDATE_FS_PARSER is not set
# CONFIG_EXT2_FS is not set
# CONFIG_EXT3_FS is not set
# CONFIG_EXT4_FS is not set
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_BTRFS_FS is not set
# CONFIG_NILFS2_FS is not set
# CONFIG_F2FS_FS is not set
CONFIG_EXPORTFS=y
# CONFIG_EXPORTFS_BLOCK_OPS is not set
CONFIG_FILE_LOCKING=y
# CONFIG_FS_ENCRYPTION is not set
# CONFIG_FS_VERITY is not set
# CONFIG_DNOTIFY is not set
# CONFIG_INOTIFY_USER is not set
# CONFIG_FANOTIFY is not set
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_FUSE_FS is not set
# CONFIG_OVERLAY_FS is not set
#
# Caches
#
# CONFIG_FSCACHE is not set
# end of Caches
#
# CD-ROM/DVD Filesystems
#
# CONFIG_ISO9660_FS is not set
# CONFIG_UDF_FS is not set
# end of CD-ROM/DVD Filesystems
#
# DOS/FAT/EXFAT/NT Filesystems
#
# CONFIG_MSDOS_FS is not set
# CONFIG_VFAT_FS is not set
# CONFIG_EXFAT_FS is not set
# CONFIG_NTFS_FS is not set
# CONFIG_NTFS3_FS is not set
# end of DOS/FAT/EXFAT/NT Filesystems
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
# CONFIG_PROC_KCORE is not set
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
# CONFIG_PROC_CHILDREN is not set
CONFIG_PROC_PID_ARCH_STATUS=y
CONFIG_KERNFS=y
CONFIG_SYSFS=y
# CONFIG_TMPFS is not set
# CONFIG_HUGETLBFS is not set
CONFIG_ARCH_WANT_HUGETLB_PAGE_OPTIMIZE_VMEMMAP=y
CONFIG_ARCH_HAS_GIGANTIC_PAGE=y
# CONFIG_CONFIGFS_FS is not set
# end of Pseudo filesystems
# CONFIG_MISC_FILESYSTEMS is not set
# CONFIG_NLS is not set
# CONFIG_UNICODE is not set
CONFIG_IO_WQ=y
# end of File systems
#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY_DMESG_RESTRICT is not set
# CONFIG_SECURITY is not set
# CONFIG_SECURITYFS is not set
CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
# CONFIG_HARDENED_USERCOPY is not set
# CONFIG_FORTIFY_SOURCE is not set
# CONFIG_STATIC_USERMODEHELPER is not set
CONFIG_DEFAULT_SECURITY_DAC=y
CONFIG_LSM="landlock,lockdown,yama,loadpin,safesetid,integrity,bpf"
#
# Kernel hardening options
#
#
# Memory initialization
#
CONFIG_INIT_STACK_NONE=y
# CONFIG_INIT_ON_ALLOC_DEFAULT_ON is not set
# CONFIG_INIT_ON_FREE_DEFAULT_ON is not set
CONFIG_CC_HAS_ZERO_CALL_USED_REGS=y
# CONFIG_ZERO_CALL_USED_REGS is not set
# end of Memory initialization
CONFIG_RANDSTRUCT_NONE=y
# end of Kernel hardening options
# end of Security options
# CONFIG_CRYPTO is not set
#
# Library routines
#
# CONFIG_PACKING is not set
CONFIG_BITREVERSE=y
CONFIG_GENERIC_STRNCPY_FROM_USER=y
CONFIG_GENERIC_STRNLEN_USER=y
# CONFIG_CORDIC is not set
# CONFIG_PRIME_NUMBERS is not set
CONFIG_GENERIC_PCI_IOMAP=y
CONFIG_GENERIC_IOMAP=y
CONFIG_ARCH_USE_CMPXCHG_LOCKREF=y
CONFIG_ARCH_HAS_FAST_MULTIPLIER=y
CONFIG_ARCH_USE_SYM_ANNOTATIONS=y
#
# Crypto library routines
#
CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC=y
# CONFIG_CRYPTO_LIB_CHACHA is not set
# CONFIG_CRYPTO_LIB_CURVE25519 is not set
CONFIG_CRYPTO_LIB_POLY1305_RSIZE=11
# CONFIG_CRYPTO_LIB_POLY1305 is not set
# end of Crypto library routines
# CONFIG_CRC_CCITT is not set
# CONFIG_CRC16 is not set
# CONFIG_CRC_T10DIF is not set
# CONFIG_CRC64_ROCKSOFT is not set
# CONFIG_CRC_ITU_T is not set
CONFIG_CRC32=y
# CONFIG_CRC32_SELFTEST is not set
CONFIG_CRC32_SLICEBY8=y
# CONFIG_CRC32_SLICEBY4 is not set
# CONFIG_CRC32_SARWATE is not set
# CONFIG_CRC32_BIT is not set
# CONFIG_CRC64 is not set
# CONFIG_CRC4 is not set
# CONFIG_CRC7 is not set
# CONFIG_LIBCRC32C is not set
# CONFIG_CRC8 is not set
# CONFIG_RANDOM32_SELFTEST is not set
# CONFIG_XZ_DEC is not set
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT_MAP=y
CONFIG_HAS_DMA=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_ARCH_DMA_ADDR_T_64BIT=y
CONFIG_SWIOTLB=y
# CONFIG_DMA_API_DEBUG is not set
# CONFIG_IRQ_POLL is not set
CONFIG_HAVE_GENERIC_VDSO=y
CONFIG_GENERIC_GETTIMEOFDAY=y
CONFIG_GENERIC_VDSO_TIME_NS=y
CONFIG_ARCH_HAS_PMEM_API=y
CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE=y
CONFIG_ARCH_HAS_COPY_MC=y
CONFIG_ARCH_STACKWALK=y
CONFIG_STACKDEPOT=y
CONFIG_SBITMAP=y
# CONFIG_PARMAN is not set
# CONFIG_OBJAGG is not set
# end of Library routines
#
# Kernel hacking
#
#
# printk and dmesg options
#
# CONFIG_PRINTK_TIME is not set
# CONFIG_PRINTK_CALLER is not set
# CONFIG_STACKTRACE_BUILD_ID is not set
CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7
CONFIG_CONSOLE_LOGLEVEL_QUIET=4
CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
# CONFIG_DYNAMIC_DEBUG is not set
# CONFIG_DYNAMIC_DEBUG_CORE is not set
# CONFIG_SYMBOLIC_ERRNAME is not set
CONFIG_DEBUG_BUGVERBOSE=y
# end of printk and dmesg options
# CONFIG_DEBUG_KERNEL is not set
#
# Compile-time checks and compiler options
#
CONFIG_AS_HAS_NON_CONST_LEB128=y
CONFIG_FRAME_WARN=2048
# CONFIG_STRIP_ASM_SYMS is not set
# CONFIG_HEADERS_INSTALL is not set
CONFIG_DEBUG_SECTION_MISMATCH=y
CONFIG_SECTION_MISMATCH_WARN_ONLY=y
CONFIG_OBJTOOL=y
# end of Compile-time checks and compiler options
#
# Generic Kernel Debugging Instruments
#
# CONFIG_MAGIC_SYSRQ is not set
# CONFIG_DEBUG_FS is not set
CONFIG_HAVE_ARCH_KGDB=y
CONFIG_ARCH_HAS_UBSAN_SANITIZE_ALL=y
# CONFIG_UBSAN is not set
CONFIG_HAVE_ARCH_KCSAN=y
CONFIG_HAVE_KCSAN_COMPILER=y
# end of Generic Kernel Debugging Instruments
#
# Networking Debugging
#
# end of Networking Debugging
#
# Memory Debugging
#
# CONFIG_PAGE_EXTENSION is not set
CONFIG_SLUB_DEBUG=y
# CONFIG_SLUB_DEBUG_ON is not set
# CONFIG_PAGE_TABLE_CHECK is not set
# CONFIG_PAGE_POISONING is not set
# CONFIG_DEBUG_RODATA_TEST is not set
CONFIG_ARCH_HAS_DEBUG_WX=y
# CONFIG_DEBUG_WX is not set
CONFIG_GENERIC_PTDUMP=y
CONFIG_HAVE_DEBUG_KMEMLEAK=y
CONFIG_ARCH_HAS_DEBUG_VM_PGTABLE=y
# CONFIG_DEBUG_VM_PGTABLE is not set
CONFIG_ARCH_HAS_DEBUG_VIRTUAL=y
CONFIG_DEBUG_MEMORY_INIT=y
CONFIG_ARCH_SUPPORTS_KMAP_LOCAL_FORCE_MAP=y
CONFIG_HAVE_ARCH_KASAN=y
CONFIG_HAVE_ARCH_KASAN_VMALLOC=y
CONFIG_CC_HAS_KASAN_GENERIC=y
CONFIG_CC_HAS_WORKING_NOSANITIZE_ADDRESS=y
# CONFIG_KASAN is not set
CONFIG_HAVE_ARCH_KFENCE=y
# CONFIG_KFENCE is not set
CONFIG_HAVE_ARCH_KMSAN=y
# end of Memory Debugging
#
# Debug Oops, Lockups and Hangs
#
# CONFIG_PANIC_ON_OOPS is not set
CONFIG_PANIC_ON_OOPS_VALUE=0
CONFIG_PANIC_TIMEOUT=0
CONFIG_HARDLOCKUP_CHECK_TIMESTAMP=y
# end of Debug Oops, Lockups and Hangs
#
# Scheduler Debugging
#
# end of Scheduler Debugging
# CONFIG_DEBUG_TIMEKEEPING is not set
#
# Lock Debugging (spinlocks, mutexes, etc...)
#
CONFIG_LOCK_DEBUGGING_SUPPORT=y
# CONFIG_WW_MUTEX_SELFTEST is not set
# end of Lock Debugging (spinlocks, mutexes, etc...)
# CONFIG_DEBUG_IRQFLAGS is not set
CONFIG_STACKTRACE=y
# CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set
#
# Debug kernel data structures
#
# CONFIG_BUG_ON_DATA_CORRUPTION is not set
# end of Debug kernel data structures
#
# RCU Debugging
#
# end of RCU Debugging
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_HAVE_RETHOOK=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS=y
CONFIG_HAVE_DYNAMIC_FTRACE_NO_PATCHABLE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_FENTRY=y
CONFIG_HAVE_OBJTOOL_MCOUNT=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_HAVE_BUILDTIME_MCOUNT_SORT=y
CONFIG_TRACING_SUPPORT=y
# CONFIG_FTRACE is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_SAMPLE_FTRACE_DIRECT=y
CONFIG_HAVE_SAMPLE_FTRACE_DIRECT_MULTI=y
CONFIG_ARCH_HAS_DEVMEM_IS_ALLOWED=y
#
# x86 Debugging
#
# CONFIG_X86_VERBOSE_BOOTUP is not set
CONFIG_EARLY_PRINTK=y
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_UNWINDER_ORC=y
# CONFIG_UNWINDER_FRAME_POINTER is not set
# end of x86 Debugging
#
# Kernel Testing and Coverage
#
# CONFIG_KUNIT is not set
CONFIG_ARCH_HAS_KCOV=y
CONFIG_CC_HAS_SANCOV_TRACE_PC=y
# CONFIG_KCOV is not set
# CONFIG_RUNTIME_TESTING_MENU is not set
CONFIG_ARCH_USE_MEMTEST=y
# CONFIG_MEMTEST is not set
# end of Kernel Testing and Coverage
#
# Rust hacking
#
# end of Rust hacking
CONFIG_WARN_MISSING_DOCUMENTS=y
CONFIG_WARN_ABI_ERRORS=y
# end of Kernel hacking
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 04/25] kvx: Add CPU definition headers
2023-01-03 16:43 ` Yann Sionneau
` (4 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, Clement Leger, Guillaume Thouvenin,
Julian Vetter, Marius Gligor
Add common headers for basic kvx support:
- CPU definition
- SFR (System Function Registers)
- Privilege levels owners
- Hw breakpoints
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/hw_breakpoint.h | 72 +
arch/kvx/include/asm/privilege.h | 211 ++
arch/kvx/include/asm/processor.h | 176 +
arch/kvx/include/asm/sfr.h | 107 +
arch/kvx/include/asm/sfr_defs.h | 5028 ++++++++++++++++++++++++++
arch/kvx/include/asm/swab.h | 48 +
arch/kvx/include/asm/sys_arch.h | 51 +
7 files changed, 5693 insertions(+)
create mode 100644 arch/kvx/include/asm/hw_breakpoint.h
create mode 100644 arch/kvx/include/asm/privilege.h
create mode 100644 arch/kvx/include/asm/processor.h
create mode 100644 arch/kvx/include/asm/sfr.h
create mode 100644 arch/kvx/include/asm/sfr_defs.h
create mode 100644 arch/kvx/include/asm/swab.h
create mode 100644 arch/kvx/include/asm/sys_arch.h
diff --git a/arch/kvx/include/asm/hw_breakpoint.h b/arch/kvx/include/asm/hw_breakpoint.h
new file mode 100644
index 000000000000..4d8380c6a905
--- /dev/null
+++ b/arch/kvx/include/asm/hw_breakpoint.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Marius Gligor
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_HW_BREAKPOINT_H
+#define _ASM_KVX_HW_BREAKPOINT_H
+
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+
+#include <linux/types.h>
+
+#define KVX_HW_BREAKPOINT_TYPE 0
+#define KVX_HW_WATCHPOINT_TYPE 1
+
+#define KVX_HW_WP_PER_WP 2
+
+struct arch_hw_breakpoint {
+ u64 addr;
+ u32 len;
+ u32 type;
+ union {
+ struct {
+ u64 hw_addr;
+ u32 hw_range;
+ } bp;
+ struct {
+ u64 hw_addr[KVX_HW_WP_PER_WP];
+ u32 hw_range[KVX_HW_WP_PER_WP];
+ u32 use_wp1;
+ u32 hit_info;
+ } wp;
+ };
+};
+
+struct perf_event_attr;
+struct perf_event;
+struct pt_regs;
+struct task_struct;
+
+int hw_breakpoint_slots(int type);
+int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw);
+int hw_breakpoint_arch_parse(struct perf_event *bp,
+ const struct perf_event_attr *attr,
+ struct arch_hw_breakpoint *hw);
+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
+ unsigned long val, void *data);
+
+int arch_install_hw_breakpoint(struct perf_event *bp);
+void arch_uninstall_hw_breakpoint(struct perf_event *bp);
+void hw_breakpoint_pmu_read(struct perf_event *bp);
+void check_hw_breakpoint(struct pt_regs *regs);
+int check_hw_watchpoint(struct pt_regs *regs, u64 ea);
+int check_hw_watchpoint_stepped(struct pt_regs *regs);
+void clear_ptrace_hw_breakpoint(struct task_struct *tsk);
+int ptrace_request_hw_breakpoint(int idx);
+int ptrace_request_hw_watchpoint(int idx);
+
+#else
+
+struct task_struct;
+
+static inline void clear_ptrace_hw_breakpoint(struct task_struct *tsk)
+{
+}
+
+#endif /* CONFIG_HAVE_HW_BREAKPOINT */
+
+#endif /* _ASM_KVX_HW_BREAKPOINT_H */
+
diff --git a/arch/kvx/include/asm/privilege.h b/arch/kvx/include/asm/privilege.h
new file mode 100644
index 000000000000..e639c4a541c6
--- /dev/null
+++ b/arch/kvx/include/asm/privilege.h
@@ -0,0 +1,211 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_PRIVILEGE_H
+#define _ASM_KVX_PRIVILEGE_H
+
+#include <asm/sys_arch.h>
+
+/**
+ * Privilege level stuff
+ */
+
+/*
+ * When manipulating ring levels, we always use relative values. This means that
+ * settings a resource owner to value 1 means "Current privilege level + 1.
+ * Setting it to 0 means "Current privilege level"
+ */
+#define PL_CUR_PLUS_1 1
+#define PL_CUR 0
+
+/**
+ * Syscall owner configuration
+ */
+#define SYO_WFXL_OWN(__field, __pl) \
+ SFR_SET_VAL_WFXL(SYO, __field, __pl)
+
+#define SYO_WFXL_VALUE(__pl) (SYO_WFXL_OWN(Q0, __pl) | \
+ SYO_WFXL_OWN(Q1, __pl) | \
+ SYO_WFXL_OWN(Q2, __pl) | \
+ SYO_WFXL_OWN(Q3, __pl))
+
+#define SYO_WFXL_VALUE_PL_CUR_PLUS_1 SYO_WFXL_VALUE(PL_CUR_PLUS_1)
+#define SYO_WFXL_VALUE_PL_CUR SYO_WFXL_VALUE(PL_CUR)
+
+/**
+ * hardware trap owner configuration
+ */
+#define HTO_WFXL_OWN(__field, __pl) \
+ SFR_SET_VAL_WFXL(HTO, __field, __pl)
+
+
+#define HTO_WFXL_VALUE_BASE(__pl) (HTO_WFXL_OWN(OPC, __pl) | \
+ HTO_WFXL_OWN(DMIS, __pl) | \
+ HTO_WFXL_OWN(PSYS, __pl) | \
+ HTO_WFXL_OWN(DSYS, __pl) | \
+ HTO_WFXL_OWN(NOMAP, __pl) | \
+ HTO_WFXL_OWN(PROT, __pl) | \
+ HTO_WFXL_OWN(W2CL, __pl) | \
+ HTO_WFXL_OWN(A2CL, __pl) | \
+ HTO_WFXL_OWN(VSFR, __pl) | \
+ HTO_WFXL_OWN(PLO, __pl))
+
+/*
+ * When alone on the processor, we need to request all traps or the processor
+ * will die badly without any information at all by jumping to the more
+ * privilege level even if nobody is there.
+ */
+#define HTO_WFXL_VALUE_PL_CUR_PLUS_1 (HTO_WFXL_VALUE_BASE(PL_CUR_PLUS_1) | \
+ HTO_WFXL_OWN(DECCG, PL_CUR_PLUS_1) | \
+ HTO_WFXL_OWN(SECCG, PL_CUR_PLUS_1) | \
+ HTO_WFXL_OWN(DE, PL_CUR_PLUS_1))
+
+#define HTO_WFXL_VALUE_PL_CUR HTO_WFXL_VALUE_BASE(PL_CUR)
+
+/**
+ * Interrupt owner configuration
+ */
+#define ITO_WFXL_OWN(__field, __pl) \
+ SFR_SET_VAL_WFXL(ITO, __field, __pl)
+
+#define ITO_WFXL_VALUE(__pl) (ITO_WFXL_OWN(IT0, __pl) | \
+ ITO_WFXL_OWN(IT1, __pl) | \
+ ITO_WFXL_OWN(IT2, __pl) | \
+ ITO_WFXL_OWN(IT3, __pl) | \
+ ITO_WFXL_OWN(IT4, __pl) | \
+ ITO_WFXL_OWN(IT5, __pl) | \
+ ITO_WFXL_OWN(IT6, __pl) | \
+ ITO_WFXL_OWN(IT7, __pl) | \
+ ITO_WFXL_OWN(IT8, __pl) | \
+ ITO_WFXL_OWN(IT9, __pl) | \
+ ITO_WFXL_OWN(IT10, __pl) | \
+ ITO_WFXL_OWN(IT11, __pl) | \
+ ITO_WFXL_OWN(IT12, __pl) | \
+ ITO_WFXL_OWN(IT13, __pl) | \
+ ITO_WFXL_OWN(IT14, __pl) | \
+ ITO_WFXL_OWN(IT15, __pl))
+
+#define ITO_WFXL_VALUE_PL_CUR_PLUS_1 ITO_WFXL_VALUE(PL_CUR_PLUS_1)
+#define ITO_WFXL_VALUE_PL_CUR ITO_WFXL_VALUE(PL_CUR)
+
+#define ITO_WFXM_OWN(__field, __pl) \
+ SFR_SET_VAL_WFXM(ITO, __field, __pl)
+
+#define ITO_WFXM_VALUE(__pl) (ITO_WFXM_OWN(IT16, __pl) | \
+ ITO_WFXM_OWN(IT17, __pl) | \
+ ITO_WFXM_OWN(IT18, __pl) | \
+ ITO_WFXM_OWN(IT19, __pl) | \
+ ITO_WFXM_OWN(IT20, __pl) | \
+ ITO_WFXM_OWN(IT21, __pl) | \
+ ITO_WFXM_OWN(IT22, __pl) | \
+ ITO_WFXM_OWN(IT23, __pl) | \
+ ITO_WFXM_OWN(IT24, __pl) | \
+ ITO_WFXM_OWN(IT25, __pl) | \
+ ITO_WFXM_OWN(IT26, __pl) | \
+ ITO_WFXM_OWN(IT27, __pl) | \
+ ITO_WFXM_OWN(IT28, __pl) | \
+ ITO_WFXM_OWN(IT29, __pl) | \
+ ITO_WFXM_OWN(IT30, __pl) | \
+ ITO_WFXM_OWN(IT31, __pl))
+
+#define ITO_WFXM_VALUE_PL_CUR_PLUS_1 ITO_WFXM_VALUE(PL_CUR_PLUS_1)
+#define ITO_WFXM_VALUE_PL_CUR ITO_WFXM_VALUE(PL_CUR)
+
+/**
+ * Debug Owner configuration
+ */
+
+#define DO_WFXL_OWN(__field, __pl) \
+ SFR_SET_VAL_WFXL(DO, __field, __pl)
+
+#define DO_WFXL_VALUE(__pl) (DO_WFXL_OWN(B0, __pl) | \
+ DO_WFXL_OWN(B1, __pl) | \
+ DO_WFXL_OWN(W0, __pl) | \
+ DO_WFXL_OWN(W1, __pl))
+
+#define DO_WFXL_VALUE_PL_CUR_PLUS_1 DO_WFXL_VALUE(PL_CUR_PLUS_1)
+#define DO_WFXL_VALUE_PL_CUR DO_WFXL_VALUE(PL_CUR)
+
+/**
+ * Misc owner configuration
+ */
+#define MO_WFXL_OWN(__field, __pl) \
+ SFR_SET_VAL_WFXL(MO, __field, __pl)
+
+#define MO_WFXL_VALUE(__pl) (MO_WFXL_OWN(MMI, __pl) | \
+ MO_WFXL_OWN(RFE, __pl) | \
+ MO_WFXL_OWN(STOP, __pl) | \
+ MO_WFXL_OWN(SYNC, __pl) | \
+ MO_WFXL_OWN(PCR, __pl) | \
+ MO_WFXL_OWN(MSG, __pl) | \
+ MO_WFXL_OWN(MEN, __pl) | \
+ MO_WFXL_OWN(MES, __pl) | \
+ MO_WFXL_OWN(CSIT, __pl) | \
+ MO_WFXL_OWN(T0, __pl) | \
+ MO_WFXL_OWN(T1, __pl) | \
+ MO_WFXL_OWN(WD, __pl) | \
+ MO_WFXL_OWN(PM0, __pl) | \
+ MO_WFXL_OWN(PM1, __pl) | \
+ MO_WFXL_OWN(PM2, __pl) | \
+ MO_WFXL_OWN(PM3, __pl))
+
+#define MO_WFXL_VALUE_PL_CUR_PLUS_1 MO_WFXL_VALUE(PL_CUR_PLUS_1)
+#define MO_WFXL_VALUE_PL_CUR MO_WFXL_VALUE(PL_CUR)
+
+#define MO_WFXM_OWN(__field, __pl) \
+ SFR_SET_VAL_WFXM(MO, __field, __pl)
+
+#define MO_WFXM_VALUE(__pl) (MO_WFXM_OWN(PMIT, __pl))
+
+#define MO_WFXM_VALUE_PL_CUR_PLUS_1 MO_WFXM_VALUE(PL_CUR_PLUS_1)
+#define MO_WFXM_VALUE_PL_CUR MO_WFXM_VALUE(PL_CUR)
+
+/**
+ * $ps owner configuration
+ */
+#define PSO_WFXL_OWN(__field, __pl) \
+ SFR_SET_VAL_WFXL(PSO, __field, __pl)
+
+#define PSO_WFXL_BASE_VALUE(__pl) (PSO_WFXL_OWN(PL0, __pl) | \
+ PSO_WFXL_OWN(PL1, __pl) | \
+ PSO_WFXL_OWN(ET, __pl) | \
+ PSO_WFXL_OWN(HTD, __pl) | \
+ PSO_WFXL_OWN(IE, __pl) | \
+ PSO_WFXL_OWN(HLE, __pl) | \
+ PSO_WFXL_OWN(SRE, __pl) | \
+ PSO_WFXL_OWN(DAUS, __pl) | \
+ PSO_WFXL_OWN(ICE, __pl) | \
+ PSO_WFXL_OWN(USE, __pl) | \
+ PSO_WFXL_OWN(DCE, __pl) | \
+ PSO_WFXL_OWN(MME, __pl) | \
+ PSO_WFXL_OWN(IL0, __pl) | \
+ PSO_WFXL_OWN(IL1, __pl) | \
+ PSO_WFXL_OWN(VS0, __pl))
+/* Request additionnal VS1 when alone */
+#define PSO_WFXL_VALUE_PL_CUR_PLUS_1 (PSO_WFXL_BASE_VALUE(PL_CUR_PLUS_1) | \
+ PSO_WFXL_OWN(VS1, PL_CUR_PLUS_1))
+#define PSO_WFXL_VALUE_PL_CUR PSO_WFXL_BASE_VALUE(PL_CUR)
+
+#define PSO_WFXM_OWN(__field, __pl) \
+ SFR_SET_VAL_WFXM(PSO, __field, __pl)
+
+#define PSO_WFXM_VALUE(__pl) (PSO_WFXM_OWN(V64, __pl) | \
+ PSO_WFXM_OWN(L2E, __pl) | \
+ PSO_WFXM_OWN(SME, __pl) | \
+ PSO_WFXM_OWN(SMR, __pl) | \
+ PSO_WFXM_OWN(PMJ0, __pl) | \
+ PSO_WFXM_OWN(PMJ1, __pl) | \
+ PSO_WFXM_OWN(PMJ2, __pl) | \
+ PSO_WFXM_OWN(PMJ3, __pl) | \
+ PSO_WFXM_OWN(MMUP, __pl))
+
+/* Request additionnal VS1 */
+#define PSO_WFXM_VALUE_PL_CUR_PLUS_1 PSO_WFXM_VALUE(PL_CUR_PLUS_1)
+#define PSO_WFXM_VALUE_PL_CUR PSO_WFXM_VALUE(PL_CUR)
+
+#endif /* _ASM_KVX_PRIVILEGE_H */
diff --git a/arch/kvx/include/asm/processor.h b/arch/kvx/include/asm/processor.h
new file mode 100644
index 000000000000..e69011c1a913
--- /dev/null
+++ b/arch/kvx/include/asm/processor.h
@@ -0,0 +1,176 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_PROCESSOR_H
+#define _ASM_KVX_PROCESSOR_H
+
+#include <asm/mmu.h>
+#include <asm/types.h>
+#include <asm/ptrace.h>
+#include <asm/sfr_defs.h>
+
+#define ARCH_HAS_PREFETCH
+#define ARCH_HAS_PREFETCHW
+
+static inline void prefetch(const void *x)
+{
+ __builtin_prefetch(x);
+}
+
+static inline void prefetchw(const void *x)
+{
+ __builtin_prefetch(x, 1);
+}
+
+#define TASK_SIZE _BITULL(MMU_USR_ADDR_BITS)
+#define TASK_SIZE_MAX TASK_SIZE
+
+/*
+ * This decides where the kernel will search for a free chunk of vm
+ * space during mmap's.
+ */
+#define TASK_UNMAPPED_BASE PAGE_ALIGN(TASK_SIZE >> 1)
+
+#define STACK_TOP TASK_SIZE
+#define STACK_TOP_MAX STACK_TOP
+
+/* Stack alignment constant */
+#define STACK_ALIGNMENT 32
+#define STACK_ALIGN_MASK (STACK_ALIGNMENT - 1)
+
+#define cpu_relax() barrier()
+
+/* Size for register saving area for refill handler (enough for 3 quad regs) */
+#define SAVE_AREA_SIZE 12
+
+#define TCA_REG_COUNT 48
+
+/* TCA registers are 256 bits wide */
+struct tca_reg {
+ uint64_t x;
+ uint64_t y;
+ uint64_t z;
+ uint64_t t;
+};
+
+/**
+ * According to kvx ABI, the following registers are callee-saved:
+ * fp (r14) r18 r19 r20 r21 r22 r23 r24 r25 r26 r27 r28 r29 r30 r31.
+ * In order to switch from a task to another, we only need to save these
+ * registers + sp (r12) and ra
+ *
+ * WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING
+ *
+ * Do not reorder the following fields !
+ * They are used in asm-offset for store octuples so they must be
+ * all right behind each other
+ */
+struct ctx_switch_regs {
+
+ uint64_t fp;
+
+ uint64_t ra; /* Return address */
+ uint64_t sp;
+ uint64_t r18;
+ uint64_t r19;
+
+ uint64_t r20;
+ uint64_t r21;
+ uint64_t r22;
+ uint64_t r23;
+
+ uint64_t r24;
+ uint64_t r25;
+ uint64_t r26;
+ uint64_t r27;
+
+ uint64_t r28;
+ uint64_t r29;
+ uint64_t r30;
+ uint64_t r31;
+
+#ifdef CONFIG_ENABLE_TCA
+ struct tca_reg tca_regs[TCA_REG_COUNT];
+ uint8_t tca_regs_saved;
+#endif
+};
+
+struct debug_info {
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+ struct perf_event *ptrace_hbp[KVX_HW_BREAKPOINT_COUNT];
+ struct perf_event *ptrace_hwp[KVX_HW_WATCHPOINT_COUNT];
+#endif
+};
+
+struct thread_struct {
+ uint64_t kernel_sp;
+ uint64_t save_area[SAVE_AREA_SIZE]; /* regs save area */
+
+#ifdef CONFIG_KVX_MMU_STATS
+ uint64_t trap_entry_ts;
+#endif
+ /* Context switch related registers */
+ struct ctx_switch_regs ctx_switch;
+
+ /* debugging */
+ struct debug_info debug;
+} __packed;
+
+#define INIT_THREAD { \
+ .ctx_switch.sp = \
+ sizeof(init_stack) + (unsigned long) &init_stack, \
+}
+
+#define KSTK_ESP(tsk) (task_pt_regs(tsk)->sp)
+#define KSTK_EIP(tsk) (task_pt_regs(tsk)->spc)
+
+#define task_pt_regs(p) \
+ ((struct pt_regs *)(task_stack_page(p) + THREAD_SIZE) - 1)
+
+#define thread_saved_reg(__tsk, __reg) \
+ ((unsigned long) ((__tsk)->thread.ctx_switch.__reg))
+
+void release_thread(struct task_struct *t);
+
+void start_thread(struct pt_regs *regs, unsigned long pc, unsigned long sp);
+
+unsigned long __get_wchan(struct task_struct *p);
+
+extern void ret_from_kernel_thread(void);
+
+/* User return function */
+extern void ret_from_fork(void);
+
+static inline void wait_for_interrupt(void)
+{
+ __builtin_kvx_await();
+ kvx_sfr_set_field(WS, WU0, 0);
+}
+
+static inline void local_cpu_stop(void)
+{
+ /* Clear Wake-Up 2 to allow stop instruction to work */
+ kvx_sfr_set_field(WS, WU2, 0);
+ __asm__ __volatile__ (
+ "1: stop\n"
+ ";;\n"
+ "goto 1b\n"
+ ";;\n"
+ );
+}
+
+struct cpuinfo_kvx {
+ u64 freq;
+ u8 arch_rev;
+ u8 uarch_rev;
+ u8 copro_enable;
+};
+
+DECLARE_PER_CPU_READ_MOSTLY(struct cpuinfo_kvx, cpu_info);
+
+#endif /* _ASM_KVX_PROCESSOR_H */
diff --git a/arch/kvx/include/asm/sfr.h b/arch/kvx/include/asm/sfr.h
new file mode 100644
index 000000000000..d91aaa335bb9
--- /dev/null
+++ b/arch/kvx/include/asm/sfr.h
@@ -0,0 +1,107 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ * Guillaume Thouvenin
+ */
+
+#ifndef _ASM_KVX_SFR_H
+#define _ASM_KVX_SFR_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+
+#include <asm/sfr_defs.h>
+
+#define wfxl(_sfr, _val) __builtin_kvx_wfxl(_sfr, _val)
+
+#define wfxm(_sfr, _val) __builtin_kvx_wfxm(_sfr, _val)
+
+static inline void
+__kvx_sfr_set_bit(unsigned char sfr, unsigned char bit)
+{
+ if (bit < 32)
+ wfxl(sfr, (uint64_t) (1 << bit) << 32);
+ else
+ wfxm(sfr, (uint64_t) 1 << bit);
+}
+
+#define kvx_sfr_set_bit(__sfr, __bit) \
+ __kvx_sfr_set_bit(KVX_SFR_ ## __sfr, __bit)
+
+static inline uint64_t make_sfr_val(uint64_t mask, uint64_t value)
+{
+ return ((value & 0xFFFFFFFF) << 32) | (mask & 0xFFFFFFFF);
+}
+
+static inline void
+__kvx_sfr_set_mask(unsigned char sfr, uint64_t mask, uint64_t value)
+{
+ uint64_t wf_val;
+
+ /* Least significant bits */
+ if (mask & 0xFFFFFFFF) {
+ wf_val = make_sfr_val(mask, value);
+ wfxl(sfr, wf_val);
+ }
+
+ /* Most significant bits */
+ if (mask & (0xFFFFFFFFULL << 32)) {
+ value >>= 32;
+ mask >>= 32;
+ wf_val = make_sfr_val(mask, value);
+ wfxm(sfr, wf_val);
+ }
+}
+
+static inline u64 kvx_sfr_iget(unsigned char sfr)
+{
+ u64 res = sfr;
+
+ asm volatile ("iget %0" : "+r"(res) :: );
+ return res;
+}
+
+#ifdef CONFIG_DEBUG_SFR_SET_MASK
+# define kvx_sfr_set_mask(__sfr, __mask, __value) \
+ do { \
+ BUG_ON(((__value) & (__mask)) != (__value)); \
+ __kvx_sfr_set_mask(KVX_SFR_ ## __sfr, __mask, __value); \
+ } while (0)
+
+#else
+# define kvx_sfr_set_mask(__sfr, __mask, __value) \
+ __kvx_sfr_set_mask(KVX_SFR_ ## __sfr, __mask, __value)
+#endif
+
+#define kvx_sfr_set_field(sfr, field, value) \
+ kvx_sfr_set_mask(sfr, KVX_SFR_ ## sfr ## _ ## field ## _MASK, \
+ ((uint64_t) (value) << KVX_SFR_ ## sfr ## _ ## field ## _SHIFT))
+
+static inline void
+__kvx_sfr_clear_bit(unsigned char sfr, unsigned char bit)
+{
+ if (bit < 32)
+ wfxl(sfr, (uint64_t) 1 << bit);
+ else
+ wfxm(sfr, (uint64_t) 1 << (bit - 32));
+}
+
+#define kvx_sfr_clear_bit(__sfr, __bit) \
+ __kvx_sfr_clear_bit(KVX_SFR_ ## __sfr, __bit)
+
+#define kvx_sfr_set(_sfr, _val) __builtin_kvx_set(KVX_SFR_ ## _sfr, _val)
+#define kvx_sfr_get(_sfr) __builtin_kvx_get(KVX_SFR_ ## _sfr)
+
+#define kvx_sfr_field_val(_val, _sfr, _field) \
+ (((_val) & KVX_SFR_ ## _sfr ## _ ## _field ## _MASK) \
+ >> KVX_SFR_ ## _sfr ## _ ## _field ## _SHIFT)
+
+#define kvx_sfr_bit(_sfr, _field) \
+ BIT_ULL(KVX_SFR_ ## _sfr ## _ ## _field ## _SHIFT)
+
+#endif
+
+#endif /* _ASM_KVX_SFR_DEFS_H */
diff --git a/arch/kvx/include/asm/sfr_defs.h b/arch/kvx/include/asm/sfr_defs.h
new file mode 100644
index 000000000000..7e6318348b83
--- /dev/null
+++ b/arch/kvx/include/asm/sfr_defs.h
@@ -0,0 +1,5028 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SFR_DEFS_H
+#define _ASM_KVX_SFR_DEFS_H
+
+#include <linux/const.h>
+
+/* Register file indices */
+#define KVX_SFR_PC 0 /* Program Counter $pc $s0 */
+#define KVX_SFR_PS 1 /* Processor State $ps $s1 */
+#define KVX_SFR_PCR 2 /* Processing Identification $pcr $s2 */
+#define KVX_SFR_RA 3 /* Return Address $ra $s3 */
+#define KVX_SFR_CS 4 /* Compute Status $cs $s4 */
+#define KVX_SFR_CSIT 5 /* Compute Status arithmetic interrupt $csit $s5 */
+#define KVX_SFR_AESPC 6 /* Arithmetic Exception Saved PC $aespc $s6 */
+#define KVX_SFR_LS 7 /* Loop Start Address $ls $s7 */
+#define KVX_SFR_LE 8 /* Loop Exit Address $le $s8 */
+#define KVX_SFR_LC 9 /* Loop Counter $lc $s9 */
+#define KVX_SFR_IPE 10 /* Inter Processor Event $ipe $s10 */
+#define KVX_SFR_MEN 11 /* Misc External Notifications $men $s11 */
+#define KVX_SFR_PMC 12 /* Performance Monitor Control $pmc $s12 */
+#define KVX_SFR_PM0 13 /* Performance Monitor 0 $pm0 $s13 */
+#define KVX_SFR_PM1 14 /* Performance Monitor 1 $pm1 $s14 */
+#define KVX_SFR_PM2 15 /* Performance Monitor 2 $pm2 $s15 */
+#define KVX_SFR_PM3 16 /* Performance Monitor 3 $pm3 $s16 */
+#define KVX_SFR_PMSA 17 /* Performance Monitor Saved Address $pmsa $s17 */
+#define KVX_SFR_TCR 18 /* Timer Control $tcr $s18 */
+#define KVX_SFR_T0V 19 /* Timer 0 value $t0v $s19 */
+#define KVX_SFR_T1V 20 /* Timer 1 value $t1v $s20 */
+#define KVX_SFR_T0R 21 /* Timer 0 reload value $t0r $s21 */
+#define KVX_SFR_T1R 22 /* Timer 1 reload value $t1r $s22 */
+#define KVX_SFR_WDV 23 /* Watchdog Value $wdv $s23 */
+#define KVX_SFR_WDR 24 /* Watchdog Reload $wdr $s24 */
+#define KVX_SFR_ILE 25 /* Interrupt Line Enable $ile $s25 */
+#define KVX_SFR_ILL 26 /* Interrupt Line Level $ill $s26 */
+#define KVX_SFR_ILR 27 /* Interrupt Line Request $ilr $s27 */
+#define KVX_SFR_MMC 28 /* Memory Management Control $mmc $s28 */
+#define KVX_SFR_TEL 29 /* TLB Entry Low $tel $s29 */
+#define KVX_SFR_TEH 30 /* TLB Entry High $teh $s30 */
+#define KVX_SFR_SYO 32 /* Syscall Owners $syo $s32 */
+#define KVX_SFR_HTO 33 /* Hardware Trap Owners $hto $s33 */
+#define KVX_SFR_ITO 34 /* Interrupt Owners $ito $s34 */
+#define KVX_SFR_DO 35 /* Debug Owners $do $s35 */
+#define KVX_SFR_MO 36 /* Miscellaneous Owners $mo $s36 */
+#define KVX_SFR_PSO 37 /* PS register fields Owners $pso $s37 */
+#define KVX_SFR_DC 40 /* OCE (Debug) Control $dc $s40 */
+#define KVX_SFR_DBA0 41 /* Breakpoint Address 0 $dba0 $s41 */
+#define KVX_SFR_DBA1 42 /* Breakpoint Address 1 $dba1 $s42 */
+#define KVX_SFR_DWA0 43 /* Watchpoint Address 0 $dwa0 $s43 */
+#define KVX_SFR_DWA1 44 /* Watchpoint Address 1 $dwa1 $s44 */
+#define KVX_SFR_MES 45 /* Memory Error Status $mes $s45 */
+#define KVX_SFR_WS 46 /* Wake-Up Status $ws $s46 */
+#define KVX_SFR_SPC_PL0 64 /* Shadow PC PL 0 $spc_pl0 $s64 */
+#define KVX_SFR_SPC_PL1 65 /* Shadow PC PL 1 $spc_pl1 $s65 */
+#define KVX_SFR_SPC_PL2 66 /* Shadow PC PL 2 $spc_pl2 $s66 */
+#define KVX_SFR_SPC_PL3 67 /* Shadow PC PL 3 $spc_pl3 $s67 */
+#define KVX_SFR_SPS_PL0 68 /* Shadow PS PL 0 $sps_pl0 $s68 */
+#define KVX_SFR_SPS_PL1 69 /* Shadow PS PL 1 $sps_pl1 $s69 */
+#define KVX_SFR_SPS_PL2 70 /* Shadow PS PL 2 $sps_pl2 $s70 */
+#define KVX_SFR_SPS_PL3 71 /* Shadow PS PL 3 $sps_pl3 $s71 */
+#define KVX_SFR_EA_PL0 72 /* Effective Address PL0 $ea_pl0 $s72 */
+#define KVX_SFR_EA_PL1 73 /* Effective Address PL1 $ea_pl1 $s73 */
+#define KVX_SFR_EA_PL2 74 /* Effective Address PL2 $ea_pl2 $s74 */
+#define KVX_SFR_EA_PL3 75 /* Effective Address PL3 $ea_pl3 $s75 */
+#define KVX_SFR_EV_PL0 76 /* Exception Vector PL 0 $ev_pl0 $s76 */
+#define KVX_SFR_EV_PL1 77 /* Exception Vector PL 1 $ev_pl1 $s77 */
+#define KVX_SFR_EV_PL2 78 /* Exception Vector PL 2 $ev_pl2 $s78 */
+#define KVX_SFR_EV_PL3 79 /* Exception Vector PL 3 $ev_pl3 $s79 */
+#define KVX_SFR_SR_PL0 80 /* System Register PL 0 $sr_pl0 $s80 */
+#define KVX_SFR_SR_PL1 81 /* System Register PL 1 $sr_pl1 $s81 */
+#define KVX_SFR_SR_PL2 82 /* System Register PL 2 $sr_pl2 $s82 */
+#define KVX_SFR_SR_PL3 83 /* System Register PL 3 $sr_pl3 $s83 */
+#define KVX_SFR_ES_PL0 84 /* Exception Syndrome PL 0 $es_pl0 $s84 */
+#define KVX_SFR_ES_PL1 85 /* Exception Syndrome PL 1 $es_pl1 $s85 */
+#define KVX_SFR_ES_PL2 86 /* Exception Syndrome PL 2 $es_pl2 $s86 */
+#define KVX_SFR_ES_PL3 87 /* Exception Syndrome PL 3 $es_pl3 $s87 */
+#define KVX_SFR_SYOW 96 /* Alias to SYO register $syow $s96 */
+#define KVX_SFR_HTOW 97 /* Alias to HTO register $htow $s97 */
+#define KVX_SFR_ITOW 98 /* Alias to ITO register $itow $s98 */
+#define KVX_SFR_DOW 99 /* Alias to DO register $dow $s99 */
+#define KVX_SFR_MOW 100 /* Alias to MO register $mow $s100 */
+#define KVX_SFR_PSOW 101 /* Alias to PSO register $psow $s101 */
+#define KVX_SFR_SPC 128 /* Shadow PC alias on SPC_PL<i> $spc $s128 */
+#define KVX_SFR_SPS 132 /* Shadow PS alias on PS_PL<i> $sps $s132 */
+#define KVX_SFR_EA 136 /* Effective Address alias on EA_PL<i> $ea $s136 */
+#define KVX_SFR_EV 140 /* Exception Vector alias on EV_PL<i> $ev $s140 */
+#define KVX_SFR_SR 144 /* System Register alias on SR_PL<i> $sr $s144 */
+#define KVX_SFR_ES 148 /* Exception Syndrome alias on ES_PL<i> $es $s148 */
+#define KVX_SFR_VSFR0 256 /* Virtual SFR 0 $vsfr0 $s256 */
+#define KVX_SFR_VSFR1 257 /* Virtual SFR 1 $vsfr1 $s257 */
+#define KVX_SFR_VSFR2 258 /* Virtual SFR 2 $vsfr2 $s258 */
+#define KVX_SFR_VSFR3 259 /* Virtual SFR 3 $vsfr3 $s259 */
+#define KVX_SFR_VSFR4 260 /* Virtual SFR 4 $vsfr4 $s260 */
+#define KVX_SFR_VSFR5 261 /* Virtual SFR 5 $vsfr5 $s261 */
+#define KVX_SFR_VSFR6 262 /* Virtual SFR 6 $vsfr6 $s262 */
+#define KVX_SFR_VSFR7 263 /* Virtual SFR 7 $vsfr7 $s263 */
+#define KVX_SFR_VSFR8 264 /* Virtual SFR 8 $vsfr8 $s264 */
+#define KVX_SFR_VSFR9 265 /* Virtual SFR 9 $vsfr9 $s265 */
+#define KVX_SFR_VSFR10 266 /* Virtual SFR 10 $vsfr10 $s266 */
+#define KVX_SFR_VSFR11 267 /* Virtual SFR 11 $vsfr11 $s267 */
+#define KVX_SFR_VSFR12 268 /* Virtual SFR 12 $vsfr12 $s268 */
+#define KVX_SFR_VSFR13 269 /* Virtual SFR 13 $vsfr13 $s269 */
+#define KVX_SFR_VSFR14 270 /* Virtual SFR 14 $vsfr14 $s270 */
+#define KVX_SFR_VSFR15 271 /* Virtual SFR 15 $vsfr15 $s271 */
+#define KVX_SFR_VSFR16 272 /* Virtual SFR 16 $vsfr16 $s272 */
+#define KVX_SFR_VSFR17 273 /* Virtual SFR 17 $vsfr17 $s273 */
+#define KVX_SFR_VSFR18 274 /* Virtual SFR 18 $vsfr18 $s274 */
+#define KVX_SFR_VSFR19 275 /* Virtual SFR 19 $vsfr19 $s275 */
+#define KVX_SFR_VSFR20 276 /* Virtual SFR 20 $vsfr20 $s276 */
+#define KVX_SFR_VSFR21 277 /* Virtual SFR 21 $vsfr21 $s277 */
+#define KVX_SFR_VSFR22 278 /* Virtual SFR 22 $vsfr22 $s278 */
+#define KVX_SFR_VSFR23 279 /* Virtual SFR 23 $vsfr23 $s279 */
+#define KVX_SFR_VSFR24 280 /* Virtual SFR 24 $vsfr24 $s280 */
+#define KVX_SFR_VSFR25 281 /* Virtual SFR 25 $vsfr25 $s281 */
+#define KVX_SFR_VSFR26 282 /* Virtual SFR 26 $vsfr26 $s282 */
+#define KVX_SFR_VSFR27 283 /* Virtual SFR 27 $vsfr27 $s283 */
+#define KVX_SFR_VSFR28 284 /* Virtual SFR 28 $vsfr28 $s284 */
+#define KVX_SFR_VSFR29 285 /* Virtual SFR 29 $vsfr29 $s285 */
+#define KVX_SFR_VSFR30 286 /* Virtual SFR 30 $vsfr30 $s286 */
+#define KVX_SFR_VSFR31 287 /* Virtual SFR 31 $vsfr31 $s287 */
+#define KVX_SFR_VSFR32 288 /* Virtual SFR 32 $vsfr32 $s288 */
+#define KVX_SFR_VSFR33 289 /* Virtual SFR 33 $vsfr33 $s289 */
+#define KVX_SFR_VSFR34 290 /* Virtual SFR 34 $vsfr34 $s290 */
+#define KVX_SFR_VSFR35 291 /* Virtual SFR 35 $vsfr35 $s291 */
+#define KVX_SFR_VSFR36 292 /* Virtual SFR 36 $vsfr36 $s292 */
+#define KVX_SFR_VSFR37 293 /* Virtual SFR 37 $vsfr37 $s293 */
+#define KVX_SFR_VSFR38 294 /* Virtual SFR 38 $vsfr38 $s294 */
+#define KVX_SFR_VSFR39 295 /* Virtual SFR 39 $vsfr39 $s295 */
+#define KVX_SFR_VSFR40 296 /* Virtual SFR 40 $vsfr40 $s296 */
+#define KVX_SFR_VSFR41 297 /* Virtual SFR 41 $vsfr41 $s297 */
+#define KVX_SFR_VSFR42 298 /* Virtual SFR 42 $vsfr42 $s298 */
+#define KVX_SFR_VSFR43 299 /* Virtual SFR 43 $vsfr43 $s299 */
+#define KVX_SFR_VSFR44 300 /* Virtual SFR 44 $vsfr44 $s300 */
+#define KVX_SFR_VSFR45 301 /* Virtual SFR 45 $vsfr45 $s301 */
+#define KVX_SFR_VSFR46 302 /* Virtual SFR 46 $vsfr46 $s302 */
+#define KVX_SFR_VSFR47 303 /* Virtual SFR 47 $vsfr47 $s303 */
+#define KVX_SFR_VSFR48 304 /* Virtual SFR 48 $vsfr48 $s304 */
+#define KVX_SFR_VSFR49 305 /* Virtual SFR 49 $vsfr49 $s305 */
+#define KVX_SFR_VSFR50 306 /* Virtual SFR 50 $vsfr50 $s306 */
+#define KVX_SFR_VSFR51 307 /* Virtual SFR 51 $vsfr51 $s307 */
+#define KVX_SFR_VSFR52 308 /* Virtual SFR 52 $vsfr52 $s308 */
+#define KVX_SFR_VSFR53 309 /* Virtual SFR 53 $vsfr53 $s309 */
+#define KVX_SFR_VSFR54 310 /* Virtual SFR 54 $vsfr54 $s310 */
+#define KVX_SFR_VSFR55 311 /* Virtual SFR 55 $vsfr55 $s311 */
+#define KVX_SFR_VSFR56 312 /* Virtual SFR 56 $vsfr56 $s312 */
+#define KVX_SFR_VSFR57 313 /* Virtual SFR 57 $vsfr57 $s313 */
+#define KVX_SFR_VSFR58 314 /* Virtual SFR 58 $vsfr58 $s314 */
+#define KVX_SFR_VSFR59 315 /* Virtual SFR 59 $vsfr59 $s315 */
+#define KVX_SFR_VSFR60 316 /* Virtual SFR 60 $vsfr60 $s316 */
+#define KVX_SFR_VSFR61 317 /* Virtual SFR 61 $vsfr61 $s317 */
+#define KVX_SFR_VSFR62 318 /* Virtual SFR 62 $vsfr62 $s318 */
+#define KVX_SFR_VSFR63 319 /* Virtual SFR 63 $vsfr63 $s319 */
+#define KVX_SFR_VSFR64 320 /* Virtual SFR 64 $vsfr64 $s320 */
+#define KVX_SFR_VSFR65 321 /* Virtual SFR 65 $vsfr65 $s321 */
+#define KVX_SFR_VSFR66 322 /* Virtual SFR 66 $vsfr66 $s322 */
+#define KVX_SFR_VSFR67 323 /* Virtual SFR 67 $vsfr67 $s323 */
+#define KVX_SFR_VSFR68 324 /* Virtual SFR 68 $vsfr68 $s324 */
+#define KVX_SFR_VSFR69 325 /* Virtual SFR 69 $vsfr69 $s325 */
+#define KVX_SFR_VSFR70 326 /* Virtual SFR 70 $vsfr70 $s326 */
+#define KVX_SFR_VSFR71 327 /* Virtual SFR 71 $vsfr71 $s327 */
+#define KVX_SFR_VSFR72 328 /* Virtual SFR 72 $vsfr72 $s328 */
+#define KVX_SFR_VSFR73 329 /* Virtual SFR 73 $vsfr73 $s329 */
+#define KVX_SFR_VSFR74 330 /* Virtual SFR 74 $vsfr74 $s330 */
+#define KVX_SFR_VSFR75 331 /* Virtual SFR 75 $vsfr75 $s331 */
+#define KVX_SFR_VSFR76 332 /* Virtual SFR 76 $vsfr76 $s332 */
+#define KVX_SFR_VSFR77 333 /* Virtual SFR 77 $vsfr77 $s333 */
+#define KVX_SFR_VSFR78 334 /* Virtual SFR 78 $vsfr78 $s334 */
+#define KVX_SFR_VSFR79 335 /* Virtual SFR 79 $vsfr79 $s335 */
+#define KVX_SFR_VSFR80 336 /* Virtual SFR 80 $vsfr80 $s336 */
+#define KVX_SFR_VSFR81 337 /* Virtual SFR 81 $vsfr81 $s337 */
+#define KVX_SFR_VSFR82 338 /* Virtual SFR 82 $vsfr82 $s338 */
+#define KVX_SFR_VSFR83 339 /* Virtual SFR 83 $vsfr83 $s339 */
+#define KVX_SFR_VSFR84 340 /* Virtual SFR 84 $vsfr84 $s340 */
+#define KVX_SFR_VSFR85 341 /* Virtual SFR 85 $vsfr85 $s341 */
+#define KVX_SFR_VSFR86 342 /* Virtual SFR 86 $vsfr86 $s342 */
+#define KVX_SFR_VSFR87 343 /* Virtual SFR 87 $vsfr87 $s343 */
+#define KVX_SFR_VSFR88 344 /* Virtual SFR 88 $vsfr88 $s344 */
+#define KVX_SFR_VSFR89 345 /* Virtual SFR 89 $vsfr89 $s345 */
+#define KVX_SFR_VSFR90 346 /* Virtual SFR 90 $vsfr90 $s346 */
+#define KVX_SFR_VSFR91 347 /* Virtual SFR 91 $vsfr91 $s347 */
+#define KVX_SFR_VSFR92 348 /* Virtual SFR 92 $vsfr92 $s348 */
+#define KVX_SFR_VSFR93 349 /* Virtual SFR 93 $vsfr93 $s349 */
+#define KVX_SFR_VSFR94 350 /* Virtual SFR 94 $vsfr94 $s350 */
+#define KVX_SFR_VSFR95 351 /* Virtual SFR 95 $vsfr95 $s351 */
+#define KVX_SFR_VSFR96 352 /* Virtual SFR 96 $vsfr96 $s352 */
+#define KVX_SFR_VSFR97 353 /* Virtual SFR 97 $vsfr97 $s353 */
+#define KVX_SFR_VSFR98 354 /* Virtual SFR 98 $vsfr98 $s354 */
+#define KVX_SFR_VSFR99 355 /* Virtual SFR 99 $vsfr99 $s355 */
+#define KVX_SFR_VSFR100 356 /* Virtual SFR 100 $vsfr100 $s356 */
+#define KVX_SFR_VSFR101 357 /* Virtual SFR 101 $vsfr101 $s357 */
+#define KVX_SFR_VSFR102 358 /* Virtual SFR 102 $vsfr102 $s358 */
+#define KVX_SFR_VSFR103 359 /* Virtual SFR 103 $vsfr103 $s359 */
+#define KVX_SFR_VSFR104 360 /* Virtual SFR 104 $vsfr104 $s360 */
+#define KVX_SFR_VSFR105 361 /* Virtual SFR 105 $vsfr105 $s361 */
+#define KVX_SFR_VSFR106 362 /* Virtual SFR 106 $vsfr106 $s362 */
+#define KVX_SFR_VSFR107 363 /* Virtual SFR 107 $vsfr107 $s363 */
+#define KVX_SFR_VSFR108 364 /* Virtual SFR 108 $vsfr108 $s364 */
+#define KVX_SFR_VSFR109 365 /* Virtual SFR 109 $vsfr109 $s365 */
+#define KVX_SFR_VSFR110 366 /* Virtual SFR 110 $vsfr110 $s366 */
+#define KVX_SFR_VSFR111 367 /* Virtual SFR 111 $vsfr111 $s367 */
+#define KVX_SFR_VSFR112 368 /* Virtual SFR 112 $vsfr112 $s368 */
+#define KVX_SFR_VSFR113 369 /* Virtual SFR 113 $vsfr113 $s369 */
+#define KVX_SFR_VSFR114 370 /* Virtual SFR 114 $vsfr114 $s370 */
+#define KVX_SFR_VSFR115 371 /* Virtual SFR 115 $vsfr115 $s371 */
+#define KVX_SFR_VSFR116 372 /* Virtual SFR 116 $vsfr116 $s372 */
+#define KVX_SFR_VSFR117 373 /* Virtual SFR 117 $vsfr117 $s373 */
+#define KVX_SFR_VSFR118 374 /* Virtual SFR 118 $vsfr118 $s374 */
+#define KVX_SFR_VSFR119 375 /* Virtual SFR 119 $vsfr119 $s375 */
+#define KVX_SFR_VSFR120 376 /* Virtual SFR 120 $vsfr120 $s376 */
+#define KVX_SFR_VSFR121 377 /* Virtual SFR 121 $vsfr121 $s377 */
+#define KVX_SFR_VSFR122 378 /* Virtual SFR 122 $vsfr122 $s378 */
+#define KVX_SFR_VSFR123 379 /* Virtual SFR 123 $vsfr123 $s379 */
+#define KVX_SFR_VSFR124 380 /* Virtual SFR 124 $vsfr124 $s380 */
+#define KVX_SFR_VSFR125 381 /* Virtual SFR 125 $vsfr125 $s381 */
+#define KVX_SFR_VSFR126 382 /* Virtual SFR 126 $vsfr126 $s382 */
+#define KVX_SFR_VSFR127 383 /* Virtual SFR 127 $vsfr127 $s383 */
+#define KVX_SFR_VSFR128 384 /* Virtual SFR 128 $vsfr128 $s384 */
+#define KVX_SFR_VSFR129 385 /* Virtual SFR 129 $vsfr129 $s385 */
+#define KVX_SFR_VSFR130 386 /* Virtual SFR 130 $vsfr130 $s386 */
+#define KVX_SFR_VSFR131 387 /* Virtual SFR 131 $vsfr131 $s387 */
+#define KVX_SFR_VSFR132 388 /* Virtual SFR 132 $vsfr132 $s388 */
+#define KVX_SFR_VSFR133 389 /* Virtual SFR 133 $vsfr133 $s389 */
+#define KVX_SFR_VSFR134 390 /* Virtual SFR 134 $vsfr134 $s390 */
+#define KVX_SFR_VSFR135 391 /* Virtual SFR 135 $vsfr135 $s391 */
+#define KVX_SFR_VSFR136 392 /* Virtual SFR 136 $vsfr136 $s392 */
+#define KVX_SFR_VSFR137 393 /* Virtual SFR 137 $vsfr137 $s393 */
+#define KVX_SFR_VSFR138 394 /* Virtual SFR 138 $vsfr138 $s394 */
+#define KVX_SFR_VSFR139 395 /* Virtual SFR 139 $vsfr139 $s395 */
+#define KVX_SFR_VSFR140 396 /* Virtual SFR 140 $vsfr140 $s396 */
+#define KVX_SFR_VSFR141 397 /* Virtual SFR 141 $vsfr141 $s397 */
+#define KVX_SFR_VSFR142 398 /* Virtual SFR 142 $vsfr142 $s398 */
+#define KVX_SFR_VSFR143 399 /* Virtual SFR 143 $vsfr143 $s399 */
+#define KVX_SFR_VSFR144 400 /* Virtual SFR 144 $vsfr144 $s400 */
+#define KVX_SFR_VSFR145 401 /* Virtual SFR 145 $vsfr145 $s401 */
+#define KVX_SFR_VSFR146 402 /* Virtual SFR 146 $vsfr146 $s402 */
+#define KVX_SFR_VSFR147 403 /* Virtual SFR 147 $vsfr147 $s403 */
+#define KVX_SFR_VSFR148 404 /* Virtual SFR 148 $vsfr148 $s404 */
+#define KVX_SFR_VSFR149 405 /* Virtual SFR 149 $vsfr149 $s405 */
+#define KVX_SFR_VSFR150 406 /* Virtual SFR 150 $vsfr150 $s406 */
+#define KVX_SFR_VSFR151 407 /* Virtual SFR 151 $vsfr151 $s407 */
+#define KVX_SFR_VSFR152 408 /* Virtual SFR 152 $vsfr152 $s408 */
+#define KVX_SFR_VSFR153 409 /* Virtual SFR 153 $vsfr153 $s409 */
+#define KVX_SFR_VSFR154 410 /* Virtual SFR 154 $vsfr154 $s410 */
+#define KVX_SFR_VSFR155 411 /* Virtual SFR 155 $vsfr155 $s411 */
+#define KVX_SFR_VSFR156 412 /* Virtual SFR 156 $vsfr156 $s412 */
+#define KVX_SFR_VSFR157 413 /* Virtual SFR 157 $vsfr157 $s413 */
+#define KVX_SFR_VSFR158 414 /* Virtual SFR 158 $vsfr158 $s414 */
+#define KVX_SFR_VSFR159 415 /* Virtual SFR 159 $vsfr159 $s415 */
+#define KVX_SFR_VSFR160 416 /* Virtual SFR 160 $vsfr160 $s416 */
+#define KVX_SFR_VSFR161 417 /* Virtual SFR 161 $vsfr161 $s417 */
+#define KVX_SFR_VSFR162 418 /* Virtual SFR 162 $vsfr162 $s418 */
+#define KVX_SFR_VSFR163 419 /* Virtual SFR 163 $vsfr163 $s419 */
+#define KVX_SFR_VSFR164 420 /* Virtual SFR 164 $vsfr164 $s420 */
+#define KVX_SFR_VSFR165 421 /* Virtual SFR 165 $vsfr165 $s421 */
+#define KVX_SFR_VSFR166 422 /* Virtual SFR 166 $vsfr166 $s422 */
+#define KVX_SFR_VSFR167 423 /* Virtual SFR 167 $vsfr167 $s423 */
+#define KVX_SFR_VSFR168 424 /* Virtual SFR 168 $vsfr168 $s424 */
+#define KVX_SFR_VSFR169 425 /* Virtual SFR 169 $vsfr169 $s425 */
+#define KVX_SFR_VSFR170 426 /* Virtual SFR 170 $vsfr170 $s426 */
+#define KVX_SFR_VSFR171 427 /* Virtual SFR 171 $vsfr171 $s427 */
+#define KVX_SFR_VSFR172 428 /* Virtual SFR 172 $vsfr172 $s428 */
+#define KVX_SFR_VSFR173 429 /* Virtual SFR 173 $vsfr173 $s429 */
+#define KVX_SFR_VSFR174 430 /* Virtual SFR 174 $vsfr174 $s430 */
+#define KVX_SFR_VSFR175 431 /* Virtual SFR 175 $vsfr175 $s431 */
+#define KVX_SFR_VSFR176 432 /* Virtual SFR 176 $vsfr176 $s432 */
+#define KVX_SFR_VSFR177 433 /* Virtual SFR 177 $vsfr177 $s433 */
+#define KVX_SFR_VSFR178 434 /* Virtual SFR 178 $vsfr178 $s434 */
+#define KVX_SFR_VSFR179 435 /* Virtual SFR 179 $vsfr179 $s435 */
+#define KVX_SFR_VSFR180 436 /* Virtual SFR 180 $vsfr180 $s436 */
+#define KVX_SFR_VSFR181 437 /* Virtual SFR 181 $vsfr181 $s437 */
+#define KVX_SFR_VSFR182 438 /* Virtual SFR 182 $vsfr182 $s438 */
+#define KVX_SFR_VSFR183 439 /* Virtual SFR 183 $vsfr183 $s439 */
+#define KVX_SFR_VSFR184 440 /* Virtual SFR 184 $vsfr184 $s440 */
+#define KVX_SFR_VSFR185 441 /* Virtual SFR 185 $vsfr185 $s441 */
+#define KVX_SFR_VSFR186 442 /* Virtual SFR 186 $vsfr186 $s442 */
+#define KVX_SFR_VSFR187 443 /* Virtual SFR 187 $vsfr187 $s443 */
+#define KVX_SFR_VSFR188 444 /* Virtual SFR 188 $vsfr188 $s444 */
+#define KVX_SFR_VSFR189 445 /* Virtual SFR 189 $vsfr189 $s445 */
+#define KVX_SFR_VSFR190 446 /* Virtual SFR 190 $vsfr190 $s446 */
+#define KVX_SFR_VSFR191 447 /* Virtual SFR 191 $vsfr191 $s447 */
+#define KVX_SFR_VSFR192 448 /* Virtual SFR 192 $vsfr192 $s448 */
+#define KVX_SFR_VSFR193 449 /* Virtual SFR 193 $vsfr193 $s449 */
+#define KVX_SFR_VSFR194 450 /* Virtual SFR 194 $vsfr194 $s450 */
+#define KVX_SFR_VSFR195 451 /* Virtual SFR 195 $vsfr195 $s451 */
+#define KVX_SFR_VSFR196 452 /* Virtual SFR 196 $vsfr196 $s452 */
+#define KVX_SFR_VSFR197 453 /* Virtual SFR 197 $vsfr197 $s453 */
+#define KVX_SFR_VSFR198 454 /* Virtual SFR 198 $vsfr198 $s454 */
+#define KVX_SFR_VSFR199 455 /* Virtual SFR 199 $vsfr199 $s455 */
+#define KVX_SFR_VSFR200 456 /* Virtual SFR 200 $vsfr200 $s456 */
+#define KVX_SFR_VSFR201 457 /* Virtual SFR 201 $vsfr201 $s457 */
+#define KVX_SFR_VSFR202 458 /* Virtual SFR 202 $vsfr202 $s458 */
+#define KVX_SFR_VSFR203 459 /* Virtual SFR 203 $vsfr203 $s459 */
+#define KVX_SFR_VSFR204 460 /* Virtual SFR 204 $vsfr204 $s460 */
+#define KVX_SFR_VSFR205 461 /* Virtual SFR 205 $vsfr205 $s461 */
+#define KVX_SFR_VSFR206 462 /* Virtual SFR 206 $vsfr206 $s462 */
+#define KVX_SFR_VSFR207 463 /* Virtual SFR 207 $vsfr207 $s463 */
+#define KVX_SFR_VSFR208 464 /* Virtual SFR 208 $vsfr208 $s464 */
+#define KVX_SFR_VSFR209 465 /* Virtual SFR 209 $vsfr209 $s465 */
+#define KVX_SFR_VSFR210 466 /* Virtual SFR 210 $vsfr210 $s466 */
+#define KVX_SFR_VSFR211 467 /* Virtual SFR 211 $vsfr211 $s467 */
+#define KVX_SFR_VSFR212 468 /* Virtual SFR 212 $vsfr212 $s468 */
+#define KVX_SFR_VSFR213 469 /* Virtual SFR 213 $vsfr213 $s469 */
+#define KVX_SFR_VSFR214 470 /* Virtual SFR 214 $vsfr214 $s470 */
+#define KVX_SFR_VSFR215 471 /* Virtual SFR 215 $vsfr215 $s471 */
+#define KVX_SFR_VSFR216 472 /* Virtual SFR 216 $vsfr216 $s472 */
+#define KVX_SFR_VSFR217 473 /* Virtual SFR 217 $vsfr217 $s473 */
+#define KVX_SFR_VSFR218 474 /* Virtual SFR 218 $vsfr218 $s474 */
+#define KVX_SFR_VSFR219 475 /* Virtual SFR 219 $vsfr219 $s475 */
+#define KVX_SFR_VSFR220 476 /* Virtual SFR 220 $vsfr220 $s476 */
+#define KVX_SFR_VSFR221 477 /* Virtual SFR 221 $vsfr221 $s477 */
+#define KVX_SFR_VSFR222 478 /* Virtual SFR 222 $vsfr222 $s478 */
+#define KVX_SFR_VSFR223 479 /* Virtual SFR 223 $vsfr223 $s479 */
+#define KVX_SFR_VSFR224 480 /* Virtual SFR 224 $vsfr224 $s480 */
+#define KVX_SFR_VSFR225 481 /* Virtual SFR 225 $vsfr225 $s481 */
+#define KVX_SFR_VSFR226 482 /* Virtual SFR 226 $vsfr226 $s482 */
+#define KVX_SFR_VSFR227 483 /* Virtual SFR 227 $vsfr227 $s483 */
+#define KVX_SFR_VSFR228 484 /* Virtual SFR 228 $vsfr228 $s484 */
+#define KVX_SFR_VSFR229 485 /* Virtual SFR 229 $vsfr229 $s485 */
+#define KVX_SFR_VSFR230 486 /* Virtual SFR 230 $vsfr230 $s486 */
+#define KVX_SFR_VSFR231 487 /* Virtual SFR 231 $vsfr231 $s487 */
+#define KVX_SFR_VSFR232 488 /* Virtual SFR 232 $vsfr232 $s488 */
+#define KVX_SFR_VSFR233 489 /* Virtual SFR 233 $vsfr233 $s489 */
+#define KVX_SFR_VSFR234 490 /* Virtual SFR 234 $vsfr234 $s490 */
+#define KVX_SFR_VSFR235 491 /* Virtual SFR 235 $vsfr235 $s491 */
+#define KVX_SFR_VSFR236 492 /* Virtual SFR 236 $vsfr236 $s492 */
+#define KVX_SFR_VSFR237 493 /* Virtual SFR 237 $vsfr237 $s493 */
+#define KVX_SFR_VSFR238 494 /* Virtual SFR 238 $vsfr238 $s494 */
+#define KVX_SFR_VSFR239 495 /* Virtual SFR 239 $vsfr239 $s495 */
+#define KVX_SFR_VSFR240 496 /* Virtual SFR 240 $vsfr240 $s496 */
+#define KVX_SFR_VSFR241 497 /* Virtual SFR 241 $vsfr241 $s497 */
+#define KVX_SFR_VSFR242 498 /* Virtual SFR 242 $vsfr242 $s498 */
+#define KVX_SFR_VSFR243 499 /* Virtual SFR 243 $vsfr243 $s499 */
+#define KVX_SFR_VSFR244 500 /* Virtual SFR 244 $vsfr244 $s500 */
+#define KVX_SFR_VSFR245 501 /* Virtual SFR 245 $vsfr245 $s501 */
+#define KVX_SFR_VSFR246 502 /* Virtual SFR 246 $vsfr246 $s502 */
+#define KVX_SFR_VSFR247 503 /* Virtual SFR 247 $vsfr247 $s503 */
+#define KVX_SFR_VSFR248 504 /* Virtual SFR 248 $vsfr248 $s504 */
+#define KVX_SFR_VSFR249 505 /* Virtual SFR 249 $vsfr249 $s505 */
+#define KVX_SFR_VSFR250 506 /* Virtual SFR 250 $vsfr250 $s506 */
+#define KVX_SFR_VSFR251 507 /* Virtual SFR 251 $vsfr251 $s507 */
+#define KVX_SFR_VSFR252 508 /* Virtual SFR 252 $vsfr252 $s508 */
+#define KVX_SFR_VSFR253 509 /* Virtual SFR 253 $vsfr253 $s509 */
+#define KVX_SFR_VSFR254 510 /* Virtual SFR 254 $vsfr254 $s510 */
+#define KVX_SFR_VSFR255 511 /* Virtual SFR 255 $vsfr255 $s511 */
+
+/* Register field masks */
+
+#define KVX_SFR_MEN_MEN_MASK _ULL(0xffff) /* Miscellaneous External Notifications */
+#define KVX_SFR_MEN_MEN_SHIFT 0
+#define KVX_SFR_MEN_MEN_WIDTH 16
+#define KVX_SFR_MEN_MEN_WFXL_MASK _ULL(0xffff)
+#define KVX_SFR_MEN_MEN_WFXL_CLEAR _ULL(0xffff)
+#define KVX_SFR_MEN_MEN_WFXL_SET _ULL(0xffff00000000)
+
+#define KVX_SFR_SYO_Q0_MASK _ULL(0x3) /* Quarter 0 syscalls 0 to 1023 owner */
+#define KVX_SFR_SYO_Q0_SHIFT 0
+#define KVX_SFR_SYO_Q0_WIDTH 2
+#define KVX_SFR_SYO_Q0_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_SYO_Q0_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_SYO_Q0_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_SYO_Q1_MASK _ULL(0xc) /* Quarter 1 syscalls 1024 to 2047 owner */
+#define KVX_SFR_SYO_Q1_SHIFT 2
+#define KVX_SFR_SYO_Q1_WIDTH 2
+#define KVX_SFR_SYO_Q1_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_SYO_Q1_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_SYO_Q1_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_SYO_Q2_MASK _ULL(0x30) /* Quarter 2 syscalls 2048 to 3071 owner */
+#define KVX_SFR_SYO_Q2_SHIFT 4
+#define KVX_SFR_SYO_Q2_WIDTH 2
+#define KVX_SFR_SYO_Q2_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_SYO_Q2_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_SYO_Q2_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_SYO_Q3_MASK _ULL(0xc0) /* Quarter 3 syscalls 3072 to 4095 owner */
+#define KVX_SFR_SYO_Q3_SHIFT 6
+#define KVX_SFR_SYO_Q3_WIDTH 2
+#define KVX_SFR_SYO_Q3_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_SYO_Q3_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_SYO_Q3_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_SYOW_Q0_MASK _ULL(0x3) /* Quarter 0 syscalls 0 to 1023 owner */
+#define KVX_SFR_SYOW_Q0_SHIFT 0
+#define KVX_SFR_SYOW_Q0_WIDTH 2
+#define KVX_SFR_SYOW_Q0_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_SYOW_Q0_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_SYOW_Q0_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_SYOW_Q1_MASK _ULL(0xc) /* Quarter 1 syscalls 1024 to 2047 owner */
+#define KVX_SFR_SYOW_Q1_SHIFT 2
+#define KVX_SFR_SYOW_Q1_WIDTH 2
+#define KVX_SFR_SYOW_Q1_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_SYOW_Q1_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_SYOW_Q1_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_SYOW_Q2_MASK _ULL(0x30) /* Quarter 2 syscalls 2048 to 3071 owner */
+#define KVX_SFR_SYOW_Q2_SHIFT 4
+#define KVX_SFR_SYOW_Q2_WIDTH 2
+#define KVX_SFR_SYOW_Q2_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_SYOW_Q2_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_SYOW_Q2_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_SYOW_Q3_MASK _ULL(0xc0) /* Quarter 3 syscalls 3072 to 4095 owner */
+#define KVX_SFR_SYOW_Q3_SHIFT 6
+#define KVX_SFR_SYOW_Q3_WIDTH 2
+#define KVX_SFR_SYOW_Q3_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_SYOW_Q3_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_SYOW_Q3_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_HTO_OPC_MASK _ULL(0x3) /* OPCode trap owner */
+#define KVX_SFR_HTO_OPC_SHIFT 0
+#define KVX_SFR_HTO_OPC_WIDTH 2
+#define KVX_SFR_HTO_OPC_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_HTO_OPC_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_HTO_OPC_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_HTO_DMIS_MASK _ULL(0xc) /* Data MISalign access trap owner */
+#define KVX_SFR_HTO_DMIS_SHIFT 2
+#define KVX_SFR_HTO_DMIS_WIDTH 2
+#define KVX_SFR_HTO_DMIS_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_HTO_DMIS_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_HTO_DMIS_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_HTO_PSYS_MASK _ULL(0x30) /* Program System Error trap owner */
+#define KVX_SFR_HTO_PSYS_SHIFT 4
+#define KVX_SFR_HTO_PSYS_WIDTH 2
+#define KVX_SFR_HTO_PSYS_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_HTO_PSYS_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_HTO_PSYS_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_HTO_DSYS_MASK _ULL(0xc0) /* Data System Error trap owner */
+#define KVX_SFR_HTO_DSYS_SHIFT 6
+#define KVX_SFR_HTO_DSYS_WIDTH 2
+#define KVX_SFR_HTO_DSYS_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_HTO_DSYS_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_HTO_DSYS_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_HTO_DECCG_MASK _ULL(0x300) /* Double ECC traps group owner */
+#define KVX_SFR_HTO_DECCG_SHIFT 8
+#define KVX_SFR_HTO_DECCG_WIDTH 2
+#define KVX_SFR_HTO_DECCG_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_HTO_DECCG_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_HTO_DECCG_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_HTO_SECCG_MASK _ULL(0xc00) /* Single ECC traps group owner */
+#define KVX_SFR_HTO_SECCG_SHIFT 10
+#define KVX_SFR_HTO_SECCG_WIDTH 2
+#define KVX_SFR_HTO_SECCG_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_HTO_SECCG_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_HTO_SECCG_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_HTO_NOMAP_MASK _ULL(0x3000) /* No mapping trap owner */
+#define KVX_SFR_HTO_NOMAP_SHIFT 12
+#define KVX_SFR_HTO_NOMAP_WIDTH 2
+#define KVX_SFR_HTO_NOMAP_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_HTO_NOMAP_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_HTO_NOMAP_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_HTO_PROT_MASK _ULL(0xc000) /* PROTection trap owner */
+#define KVX_SFR_HTO_PROT_SHIFT 14
+#define KVX_SFR_HTO_PROT_WIDTH 2
+#define KVX_SFR_HTO_PROT_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_HTO_PROT_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_HTO_PROT_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_HTO_W2CL_MASK _ULL(0x30000) /* Write to clean trap owner */
+#define KVX_SFR_HTO_W2CL_SHIFT 16
+#define KVX_SFR_HTO_W2CL_WIDTH 2
+#define KVX_SFR_HTO_W2CL_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_HTO_W2CL_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_HTO_W2CL_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_HTO_A2CL_MASK _ULL(0xc0000) /* Atomic to clean trap owner */
+#define KVX_SFR_HTO_A2CL_SHIFT 18
+#define KVX_SFR_HTO_A2CL_WIDTH 2
+#define KVX_SFR_HTO_A2CL_WFXL_MASK _ULL(0xc0000)
+#define KVX_SFR_HTO_A2CL_WFXL_CLEAR _ULL(0xc0000)
+#define KVX_SFR_HTO_A2CL_WFXL_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_HTO_DE_MASK _ULL(0x300000) /* Double Exception trap owner */
+#define KVX_SFR_HTO_DE_SHIFT 20
+#define KVX_SFR_HTO_DE_WIDTH 2
+#define KVX_SFR_HTO_DE_WFXL_MASK _ULL(0x300000)
+#define KVX_SFR_HTO_DE_WFXL_CLEAR _ULL(0x300000)
+#define KVX_SFR_HTO_DE_WFXL_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_HTO_VSFR_MASK _ULL(0xc00000) /* Virtual SFR trap owner */
+#define KVX_SFR_HTO_VSFR_SHIFT 22
+#define KVX_SFR_HTO_VSFR_WIDTH 2
+#define KVX_SFR_HTO_VSFR_WFXL_MASK _ULL(0xc00000)
+#define KVX_SFR_HTO_VSFR_WFXL_CLEAR _ULL(0xc00000)
+#define KVX_SFR_HTO_VSFR_WFXL_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_HTO_PLO_MASK _ULL(0x3000000) /* Privilege Level Overflow trap owner */
+#define KVX_SFR_HTO_PLO_SHIFT 24
+#define KVX_SFR_HTO_PLO_WIDTH 2
+#define KVX_SFR_HTO_PLO_WFXL_MASK _ULL(0x3000000)
+#define KVX_SFR_HTO_PLO_WFXL_CLEAR _ULL(0x3000000)
+#define KVX_SFR_HTO_PLO_WFXL_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_HTOW_OPC_MASK _ULL(0x3) /* OPCode trap owner */
+#define KVX_SFR_HTOW_OPC_SHIFT 0
+#define KVX_SFR_HTOW_OPC_WIDTH 2
+#define KVX_SFR_HTOW_OPC_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_HTOW_OPC_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_HTOW_OPC_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_HTOW_DMIS_MASK _ULL(0xc) /* Data MISalign access trap owner */
+#define KVX_SFR_HTOW_DMIS_SHIFT 2
+#define KVX_SFR_HTOW_DMIS_WIDTH 2
+#define KVX_SFR_HTOW_DMIS_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_HTOW_DMIS_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_HTOW_DMIS_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_HTOW_PSYS_MASK _ULL(0x30) /* Program System Error trap owner */
+#define KVX_SFR_HTOW_PSYS_SHIFT 4
+#define KVX_SFR_HTOW_PSYS_WIDTH 2
+#define KVX_SFR_HTOW_PSYS_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_HTOW_PSYS_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_HTOW_PSYS_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_HTOW_DSYS_MASK _ULL(0xc0) /* Data System Error trap owner */
+#define KVX_SFR_HTOW_DSYS_SHIFT 6
+#define KVX_SFR_HTOW_DSYS_WIDTH 2
+#define KVX_SFR_HTOW_DSYS_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_HTOW_DSYS_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_HTOW_DSYS_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_HTOW_DECCG_MASK _ULL(0x300) /* Double ECC traps group owner */
+#define KVX_SFR_HTOW_DECCG_SHIFT 8
+#define KVX_SFR_HTOW_DECCG_WIDTH 2
+#define KVX_SFR_HTOW_DECCG_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_HTOW_DECCG_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_HTOW_DECCG_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_HTOW_SECCG_MASK _ULL(0xc00) /* Single ECC traps group owner */
+#define KVX_SFR_HTOW_SECCG_SHIFT 10
+#define KVX_SFR_HTOW_SECCG_WIDTH 2
+#define KVX_SFR_HTOW_SECCG_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_HTOW_SECCG_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_HTOW_SECCG_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_HTOW_NOMAP_MASK _ULL(0x3000) /* No mapping trap owner */
+#define KVX_SFR_HTOW_NOMAP_SHIFT 12
+#define KVX_SFR_HTOW_NOMAP_WIDTH 2
+#define KVX_SFR_HTOW_NOMAP_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_HTOW_NOMAP_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_HTOW_NOMAP_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_HTOW_PROT_MASK _ULL(0xc000) /* PROTection trap owner */
+#define KVX_SFR_HTOW_PROT_SHIFT 14
+#define KVX_SFR_HTOW_PROT_WIDTH 2
+#define KVX_SFR_HTOW_PROT_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_HTOW_PROT_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_HTOW_PROT_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_HTOW_W2CL_MASK _ULL(0x30000) /* Write to clean trap owner */
+#define KVX_SFR_HTOW_W2CL_SHIFT 16
+#define KVX_SFR_HTOW_W2CL_WIDTH 2
+#define KVX_SFR_HTOW_W2CL_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_HTOW_W2CL_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_HTOW_W2CL_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_HTOW_A2CL_MASK _ULL(0xc0000) /* Atomic to clean trap owner */
+#define KVX_SFR_HTOW_A2CL_SHIFT 18
+#define KVX_SFR_HTOW_A2CL_WIDTH 2
+#define KVX_SFR_HTOW_A2CL_WFXL_MASK _ULL(0xc0000)
+#define KVX_SFR_HTOW_A2CL_WFXL_CLEAR _ULL(0xc0000)
+#define KVX_SFR_HTOW_A2CL_WFXL_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_HTOW_DE_MASK _ULL(0x300000) /* Double Exception trap owner */
+#define KVX_SFR_HTOW_DE_SHIFT 20
+#define KVX_SFR_HTOW_DE_WIDTH 2
+#define KVX_SFR_HTOW_DE_WFXL_MASK _ULL(0x300000)
+#define KVX_SFR_HTOW_DE_WFXL_CLEAR _ULL(0x300000)
+#define KVX_SFR_HTOW_DE_WFXL_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_HTOW_VSFR_MASK _ULL(0xc00000) /* Virtual SFR trap owner */
+#define KVX_SFR_HTOW_VSFR_SHIFT 22
+#define KVX_SFR_HTOW_VSFR_WIDTH 2
+#define KVX_SFR_HTOW_VSFR_WFXL_MASK _ULL(0xc00000)
+#define KVX_SFR_HTOW_VSFR_WFXL_CLEAR _ULL(0xc00000)
+#define KVX_SFR_HTOW_VSFR_WFXL_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_HTOW_PLO_MASK _ULL(0x3000000) /* Privilege Level Overflow trap owner */
+#define KVX_SFR_HTOW_PLO_SHIFT 24
+#define KVX_SFR_HTOW_PLO_WIDTH 2
+#define KVX_SFR_HTOW_PLO_WFXL_MASK _ULL(0x3000000)
+#define KVX_SFR_HTOW_PLO_WFXL_CLEAR _ULL(0x3000000)
+#define KVX_SFR_HTOW_PLO_WFXL_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_ITO_IT0_MASK _ULL(0x3) /* Interrupt 0 owner */
+#define KVX_SFR_ITO_IT0_SHIFT 0
+#define KVX_SFR_ITO_IT0_WIDTH 2
+#define KVX_SFR_ITO_IT0_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_ITO_IT0_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_ITO_IT0_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_ITO_IT1_MASK _ULL(0xc) /* Interrupt 1 owner */
+#define KVX_SFR_ITO_IT1_SHIFT 2
+#define KVX_SFR_ITO_IT1_WIDTH 2
+#define KVX_SFR_ITO_IT1_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_ITO_IT1_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_ITO_IT1_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_ITO_IT2_MASK _ULL(0x30) /* Interrupt 2 owner */
+#define KVX_SFR_ITO_IT2_SHIFT 4
+#define KVX_SFR_ITO_IT2_WIDTH 2
+#define KVX_SFR_ITO_IT2_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_ITO_IT2_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_ITO_IT2_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ITO_IT3_MASK _ULL(0xc0) /* Interrupt 3 owner */
+#define KVX_SFR_ITO_IT3_SHIFT 6
+#define KVX_SFR_ITO_IT3_WIDTH 2
+#define KVX_SFR_ITO_IT3_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_ITO_IT3_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_ITO_IT3_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ITO_IT4_MASK _ULL(0x300) /* Interrupt 4 owner */
+#define KVX_SFR_ITO_IT4_SHIFT 8
+#define KVX_SFR_ITO_IT4_WIDTH 2
+#define KVX_SFR_ITO_IT4_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_ITO_IT4_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_ITO_IT4_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ITO_IT5_MASK _ULL(0xc00) /* Interrupt 5 owner */
+#define KVX_SFR_ITO_IT5_SHIFT 10
+#define KVX_SFR_ITO_IT5_WIDTH 2
+#define KVX_SFR_ITO_IT5_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_ITO_IT5_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_ITO_IT5_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ITO_IT6_MASK _ULL(0x3000) /* Interrupt 6 owner */
+#define KVX_SFR_ITO_IT6_SHIFT 12
+#define KVX_SFR_ITO_IT6_WIDTH 2
+#define KVX_SFR_ITO_IT6_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_ITO_IT6_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_ITO_IT6_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ITO_IT7_MASK _ULL(0xc000) /* Interrupt 7 owner */
+#define KVX_SFR_ITO_IT7_SHIFT 14
+#define KVX_SFR_ITO_IT7_WIDTH 2
+#define KVX_SFR_ITO_IT7_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_ITO_IT7_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_ITO_IT7_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_ITO_IT8_MASK _ULL(0x30000) /* Interrupt 8 owner */
+#define KVX_SFR_ITO_IT8_SHIFT 16
+#define KVX_SFR_ITO_IT8_WIDTH 2
+#define KVX_SFR_ITO_IT8_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_ITO_IT8_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_ITO_IT8_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_ITO_IT9_MASK _ULL(0xc0000) /* Interrupt 9 owner */
+#define KVX_SFR_ITO_IT9_SHIFT 18
+#define KVX_SFR_ITO_IT9_WIDTH 2
+#define KVX_SFR_ITO_IT9_WFXL_MASK _ULL(0xc0000)
+#define KVX_SFR_ITO_IT9_WFXL_CLEAR _ULL(0xc0000)
+#define KVX_SFR_ITO_IT9_WFXL_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_ITO_IT10_MASK _ULL(0x300000) /* Interrupt 10 owner */
+#define KVX_SFR_ITO_IT10_SHIFT 20
+#define KVX_SFR_ITO_IT10_WIDTH 2
+#define KVX_SFR_ITO_IT10_WFXL_MASK _ULL(0x300000)
+#define KVX_SFR_ITO_IT10_WFXL_CLEAR _ULL(0x300000)
+#define KVX_SFR_ITO_IT10_WFXL_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_ITO_IT11_MASK _ULL(0xc00000) /* Interrupt 11 owner */
+#define KVX_SFR_ITO_IT11_SHIFT 22
+#define KVX_SFR_ITO_IT11_WIDTH 2
+#define KVX_SFR_ITO_IT11_WFXL_MASK _ULL(0xc00000)
+#define KVX_SFR_ITO_IT11_WFXL_CLEAR _ULL(0xc00000)
+#define KVX_SFR_ITO_IT11_WFXL_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_ITO_IT12_MASK _ULL(0x3000000) /* Interrupt 12 owner */
+#define KVX_SFR_ITO_IT12_SHIFT 24
+#define KVX_SFR_ITO_IT12_WIDTH 2
+#define KVX_SFR_ITO_IT12_WFXL_MASK _ULL(0x3000000)
+#define KVX_SFR_ITO_IT12_WFXL_CLEAR _ULL(0x3000000)
+#define KVX_SFR_ITO_IT12_WFXL_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_ITO_IT13_MASK _ULL(0xc000000) /* Interrupt 13 owner */
+#define KVX_SFR_ITO_IT13_SHIFT 26
+#define KVX_SFR_ITO_IT13_WIDTH 2
+#define KVX_SFR_ITO_IT13_WFXL_MASK _ULL(0xc000000)
+#define KVX_SFR_ITO_IT13_WFXL_CLEAR _ULL(0xc000000)
+#define KVX_SFR_ITO_IT13_WFXL_SET _ULL(0xc00000000000000)
+
+#define KVX_SFR_ITO_IT14_MASK _ULL(0x30000000) /* Interrupt 14 owner */
+#define KVX_SFR_ITO_IT14_SHIFT 28
+#define KVX_SFR_ITO_IT14_WIDTH 2
+#define KVX_SFR_ITO_IT14_WFXL_MASK _ULL(0x30000000)
+#define KVX_SFR_ITO_IT14_WFXL_CLEAR _ULL(0x30000000)
+#define KVX_SFR_ITO_IT14_WFXL_SET _ULL(0x3000000000000000)
+
+#define KVX_SFR_ITO_IT15_MASK _ULL(0xc0000000) /* Interrupt 15 owner */
+#define KVX_SFR_ITO_IT15_SHIFT 30
+#define KVX_SFR_ITO_IT15_WIDTH 2
+#define KVX_SFR_ITO_IT15_WFXL_MASK _ULL(0xc0000000)
+#define KVX_SFR_ITO_IT15_WFXL_CLEAR _ULL(0xc0000000)
+#define KVX_SFR_ITO_IT15_WFXL_SET _ULL(0xc000000000000000)
+
+#define KVX_SFR_ITO_IT16_MASK _ULL(0x300000000) /* Interrupt 16 owner */
+#define KVX_SFR_ITO_IT16_SHIFT 32
+#define KVX_SFR_ITO_IT16_WIDTH 2
+#define KVX_SFR_ITO_IT16_WFXM_MASK _ULL(0x300000000)
+#define KVX_SFR_ITO_IT16_WFXM_CLEAR _ULL(0x3)
+#define KVX_SFR_ITO_IT16_WFXM_SET _ULL(0x300000000)
+
+#define KVX_SFR_ITO_IT17_MASK _ULL(0xc00000000) /* Interrupt 17 owner */
+#define KVX_SFR_ITO_IT17_SHIFT 34
+#define KVX_SFR_ITO_IT17_WIDTH 2
+#define KVX_SFR_ITO_IT17_WFXM_MASK _ULL(0xc00000000)
+#define KVX_SFR_ITO_IT17_WFXM_CLEAR _ULL(0xc)
+#define KVX_SFR_ITO_IT17_WFXM_SET _ULL(0xc00000000)
+
+#define KVX_SFR_ITO_IT18_MASK _ULL(0x3000000000) /* Interrupt 18 owner */
+#define KVX_SFR_ITO_IT18_SHIFT 36
+#define KVX_SFR_ITO_IT18_WIDTH 2
+#define KVX_SFR_ITO_IT18_WFXM_MASK _ULL(0x3000000000)
+#define KVX_SFR_ITO_IT18_WFXM_CLEAR _ULL(0x30)
+#define KVX_SFR_ITO_IT18_WFXM_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ITO_IT19_MASK _ULL(0xc000000000) /* Interrupt 19 owner */
+#define KVX_SFR_ITO_IT19_SHIFT 38
+#define KVX_SFR_ITO_IT19_WIDTH 2
+#define KVX_SFR_ITO_IT19_WFXM_MASK _ULL(0xc000000000)
+#define KVX_SFR_ITO_IT19_WFXM_CLEAR _ULL(0xc0)
+#define KVX_SFR_ITO_IT19_WFXM_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ITO_IT20_MASK _ULL(0x30000000000) /* Interrupt 20 owner */
+#define KVX_SFR_ITO_IT20_SHIFT 40
+#define KVX_SFR_ITO_IT20_WIDTH 2
+#define KVX_SFR_ITO_IT20_WFXM_MASK _ULL(0x30000000000)
+#define KVX_SFR_ITO_IT20_WFXM_CLEAR _ULL(0x300)
+#define KVX_SFR_ITO_IT20_WFXM_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ITO_IT21_MASK _ULL(0xc0000000000) /* Interrupt 21 owner */
+#define KVX_SFR_ITO_IT21_SHIFT 42
+#define KVX_SFR_ITO_IT21_WIDTH 2
+#define KVX_SFR_ITO_IT21_WFXM_MASK _ULL(0xc0000000000)
+#define KVX_SFR_ITO_IT21_WFXM_CLEAR _ULL(0xc00)
+#define KVX_SFR_ITO_IT21_WFXM_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ITO_IT22_MASK _ULL(0x300000000000) /* Interrupt 22 owner */
+#define KVX_SFR_ITO_IT22_SHIFT 44
+#define KVX_SFR_ITO_IT22_WIDTH 2
+#define KVX_SFR_ITO_IT22_WFXM_MASK _ULL(0x300000000000)
+#define KVX_SFR_ITO_IT22_WFXM_CLEAR _ULL(0x3000)
+#define KVX_SFR_ITO_IT22_WFXM_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ITO_IT23_MASK _ULL(0xc00000000000) /* Interrupt 23 owner */
+#define KVX_SFR_ITO_IT23_SHIFT 46
+#define KVX_SFR_ITO_IT23_WIDTH 2
+#define KVX_SFR_ITO_IT23_WFXM_MASK _ULL(0xc00000000000)
+#define KVX_SFR_ITO_IT23_WFXM_CLEAR _ULL(0xc000)
+#define KVX_SFR_ITO_IT23_WFXM_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_ITO_IT24_MASK _ULL(0x3000000000000) /* Interrupt 24 owner */
+#define KVX_SFR_ITO_IT24_SHIFT 48
+#define KVX_SFR_ITO_IT24_WIDTH 2
+#define KVX_SFR_ITO_IT24_WFXM_MASK _ULL(0x3000000000000)
+#define KVX_SFR_ITO_IT24_WFXM_CLEAR _ULL(0x30000)
+#define KVX_SFR_ITO_IT24_WFXM_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_ITO_IT25_MASK _ULL(0xc000000000000) /* Interrupt 25 owner */
+#define KVX_SFR_ITO_IT25_SHIFT 50
+#define KVX_SFR_ITO_IT25_WIDTH 2
+#define KVX_SFR_ITO_IT25_WFXM_MASK _ULL(0xc000000000000)
+#define KVX_SFR_ITO_IT25_WFXM_CLEAR _ULL(0xc0000)
+#define KVX_SFR_ITO_IT25_WFXM_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_ITO_IT26_MASK _ULL(0x30000000000000) /* Interrupt 26 owner */
+#define KVX_SFR_ITO_IT26_SHIFT 52
+#define KVX_SFR_ITO_IT26_WIDTH 2
+#define KVX_SFR_ITO_IT26_WFXM_MASK _ULL(0x30000000000000)
+#define KVX_SFR_ITO_IT26_WFXM_CLEAR _ULL(0x300000)
+#define KVX_SFR_ITO_IT26_WFXM_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_ITO_IT27_MASK _ULL(0xc0000000000000) /* Interrupt 27 owner */
+#define KVX_SFR_ITO_IT27_SHIFT 54
+#define KVX_SFR_ITO_IT27_WIDTH 2
+#define KVX_SFR_ITO_IT27_WFXM_MASK _ULL(0xc0000000000000)
+#define KVX_SFR_ITO_IT27_WFXM_CLEAR _ULL(0xc00000)
+#define KVX_SFR_ITO_IT27_WFXM_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_ITO_IT28_MASK _ULL(0x300000000000000) /* Interrupt 28 owner */
+#define KVX_SFR_ITO_IT28_SHIFT 56
+#define KVX_SFR_ITO_IT28_WIDTH 2
+#define KVX_SFR_ITO_IT28_WFXM_MASK _ULL(0x300000000000000)
+#define KVX_SFR_ITO_IT28_WFXM_CLEAR _ULL(0x3000000)
+#define KVX_SFR_ITO_IT28_WFXM_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_ITO_IT29_MASK _ULL(0xc00000000000000) /* Interrupt 29 owner */
+#define KVX_SFR_ITO_IT29_SHIFT 58
+#define KVX_SFR_ITO_IT29_WIDTH 2
+#define KVX_SFR_ITO_IT29_WFXM_MASK _ULL(0xc00000000000000)
+#define KVX_SFR_ITO_IT29_WFXM_CLEAR _ULL(0xc000000)
+#define KVX_SFR_ITO_IT29_WFXM_SET _ULL(0xc00000000000000)
+
+#define KVX_SFR_ITO_IT30_MASK _ULL(0x3000000000000000) /* Interrupt 30 owner */
+#define KVX_SFR_ITO_IT30_SHIFT 60
+#define KVX_SFR_ITO_IT30_WIDTH 2
+#define KVX_SFR_ITO_IT30_WFXM_MASK _ULL(0x3000000000000000)
+#define KVX_SFR_ITO_IT30_WFXM_CLEAR _ULL(0x30000000)
+#define KVX_SFR_ITO_IT30_WFXM_SET _ULL(0x3000000000000000)
+
+#define KVX_SFR_ITO_IT31_MASK _ULL(0xc000000000000000) /* Interrupt 31 owner */
+#define KVX_SFR_ITO_IT31_SHIFT 62
+#define KVX_SFR_ITO_IT31_WIDTH 2
+#define KVX_SFR_ITO_IT31_WFXM_MASK _ULL(0xc000000000000000)
+#define KVX_SFR_ITO_IT31_WFXM_CLEAR _ULL(0xc0000000)
+#define KVX_SFR_ITO_IT31_WFXM_SET _ULL(0xc000000000000000)
+
+#define KVX_SFR_ILE_IT0_MASK _ULL(0x1) /* Interrupt 0 owner */
+#define KVX_SFR_ILE_IT0_SHIFT 0
+#define KVX_SFR_ILE_IT0_WIDTH 1
+#define KVX_SFR_ILE_IT0_WFXL_MASK _ULL(0x1)
+#define KVX_SFR_ILE_IT0_WFXL_CLEAR _ULL(0x1)
+#define KVX_SFR_ILE_IT0_WFXL_SET _ULL(0x100000000)
+
+#define KVX_SFR_ILE_IT1_MASK _ULL(0x2) /* Interrupt 1 owner */
+#define KVX_SFR_ILE_IT1_SHIFT 1
+#define KVX_SFR_ILE_IT1_WIDTH 1
+#define KVX_SFR_ILE_IT1_WFXL_MASK _ULL(0x2)
+#define KVX_SFR_ILE_IT1_WFXL_CLEAR _ULL(0x2)
+#define KVX_SFR_ILE_IT1_WFXL_SET _ULL(0x200000000)
+
+#define KVX_SFR_ILE_IT2_MASK _ULL(0x4) /* Interrupt 2 owner */
+#define KVX_SFR_ILE_IT2_SHIFT 2
+#define KVX_SFR_ILE_IT2_WIDTH 1
+#define KVX_SFR_ILE_IT2_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_ILE_IT2_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_ILE_IT2_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_ILE_IT3_MASK _ULL(0x8) /* Interrupt 3 owner */
+#define KVX_SFR_ILE_IT3_SHIFT 3
+#define KVX_SFR_ILE_IT3_WIDTH 1
+#define KVX_SFR_ILE_IT3_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_ILE_IT3_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_ILE_IT3_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_ILE_IT4_MASK _ULL(0x10) /* Interrupt 4 owner */
+#define KVX_SFR_ILE_IT4_SHIFT 4
+#define KVX_SFR_ILE_IT4_WIDTH 1
+#define KVX_SFR_ILE_IT4_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_ILE_IT4_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_ILE_IT4_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_ILE_IT5_MASK _ULL(0x20) /* Interrupt 5 owner */
+#define KVX_SFR_ILE_IT5_SHIFT 5
+#define KVX_SFR_ILE_IT5_WIDTH 1
+#define KVX_SFR_ILE_IT5_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_ILE_IT5_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_ILE_IT5_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_ILE_IT6_MASK _ULL(0x40) /* Interrupt 6 owner */
+#define KVX_SFR_ILE_IT6_SHIFT 6
+#define KVX_SFR_ILE_IT6_WIDTH 1
+#define KVX_SFR_ILE_IT6_WFXL_MASK _ULL(0x40)
+#define KVX_SFR_ILE_IT6_WFXL_CLEAR _ULL(0x40)
+#define KVX_SFR_ILE_IT6_WFXL_SET _ULL(0x4000000000)
+
+#define KVX_SFR_ILE_IT7_MASK _ULL(0x80) /* Interrupt 7 owner */
+#define KVX_SFR_ILE_IT7_SHIFT 7
+#define KVX_SFR_ILE_IT7_WIDTH 1
+#define KVX_SFR_ILE_IT7_WFXL_MASK _ULL(0x80)
+#define KVX_SFR_ILE_IT7_WFXL_CLEAR _ULL(0x80)
+#define KVX_SFR_ILE_IT7_WFXL_SET _ULL(0x8000000000)
+
+#define KVX_SFR_ILE_IT8_MASK _ULL(0x100) /* Interrupt 8 owner */
+#define KVX_SFR_ILE_IT8_SHIFT 8
+#define KVX_SFR_ILE_IT8_WIDTH 1
+#define KVX_SFR_ILE_IT8_WFXL_MASK _ULL(0x100)
+#define KVX_SFR_ILE_IT8_WFXL_CLEAR _ULL(0x100)
+#define KVX_SFR_ILE_IT8_WFXL_SET _ULL(0x10000000000)
+
+#define KVX_SFR_ILE_IT9_MASK _ULL(0x200) /* Interrupt 9 owner */
+#define KVX_SFR_ILE_IT9_SHIFT 9
+#define KVX_SFR_ILE_IT9_WIDTH 1
+#define KVX_SFR_ILE_IT9_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_ILE_IT9_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_ILE_IT9_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_ILE_IT10_MASK _ULL(0x400) /* Interrupt 10 owner */
+#define KVX_SFR_ILE_IT10_SHIFT 10
+#define KVX_SFR_ILE_IT10_WIDTH 1
+#define KVX_SFR_ILE_IT10_WFXL_MASK _ULL(0x400)
+#define KVX_SFR_ILE_IT10_WFXL_CLEAR _ULL(0x400)
+#define KVX_SFR_ILE_IT10_WFXL_SET _ULL(0x40000000000)
+
+#define KVX_SFR_ILE_IT11_MASK _ULL(0x800) /* Interrupt 11 owner */
+#define KVX_SFR_ILE_IT11_SHIFT 11
+#define KVX_SFR_ILE_IT11_WIDTH 1
+#define KVX_SFR_ILE_IT11_WFXL_MASK _ULL(0x800)
+#define KVX_SFR_ILE_IT11_WFXL_CLEAR _ULL(0x800)
+#define KVX_SFR_ILE_IT11_WFXL_SET _ULL(0x80000000000)
+
+#define KVX_SFR_ILE_IT12_MASK _ULL(0x1000) /* Interrupt 12 owner */
+#define KVX_SFR_ILE_IT12_SHIFT 12
+#define KVX_SFR_ILE_IT12_WIDTH 1
+#define KVX_SFR_ILE_IT12_WFXL_MASK _ULL(0x1000)
+#define KVX_SFR_ILE_IT12_WFXL_CLEAR _ULL(0x1000)
+#define KVX_SFR_ILE_IT12_WFXL_SET _ULL(0x100000000000)
+
+#define KVX_SFR_ILE_IT13_MASK _ULL(0x2000) /* Interrupt 13 owner */
+#define KVX_SFR_ILE_IT13_SHIFT 13
+#define KVX_SFR_ILE_IT13_WIDTH 1
+#define KVX_SFR_ILE_IT13_WFXL_MASK _ULL(0x2000)
+#define KVX_SFR_ILE_IT13_WFXL_CLEAR _ULL(0x2000)
+#define KVX_SFR_ILE_IT13_WFXL_SET _ULL(0x200000000000)
+
+#define KVX_SFR_ILE_IT14_MASK _ULL(0x4000) /* Interrupt 14 owner */
+#define KVX_SFR_ILE_IT14_SHIFT 14
+#define KVX_SFR_ILE_IT14_WIDTH 1
+#define KVX_SFR_ILE_IT14_WFXL_MASK _ULL(0x4000)
+#define KVX_SFR_ILE_IT14_WFXL_CLEAR _ULL(0x4000)
+#define KVX_SFR_ILE_IT14_WFXL_SET _ULL(0x400000000000)
+
+#define KVX_SFR_ILE_IT15_MASK _ULL(0x8000) /* Interrupt 15 owner */
+#define KVX_SFR_ILE_IT15_SHIFT 15
+#define KVX_SFR_ILE_IT15_WIDTH 1
+#define KVX_SFR_ILE_IT15_WFXL_MASK _ULL(0x8000)
+#define KVX_SFR_ILE_IT15_WFXL_CLEAR _ULL(0x8000)
+#define KVX_SFR_ILE_IT15_WFXL_SET _ULL(0x800000000000)
+
+#define KVX_SFR_ILE_IT16_MASK _ULL(0x10000) /* Interrupt 16 owner */
+#define KVX_SFR_ILE_IT16_SHIFT 16
+#define KVX_SFR_ILE_IT16_WIDTH 1
+#define KVX_SFR_ILE_IT16_WFXL_MASK _ULL(0x10000)
+#define KVX_SFR_ILE_IT16_WFXL_CLEAR _ULL(0x10000)
+#define KVX_SFR_ILE_IT16_WFXL_SET _ULL(0x1000000000000)
+
+#define KVX_SFR_ILE_IT17_MASK _ULL(0x20000) /* Interrupt 17 owner */
+#define KVX_SFR_ILE_IT17_SHIFT 17
+#define KVX_SFR_ILE_IT17_WIDTH 1
+#define KVX_SFR_ILE_IT17_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_ILE_IT17_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_ILE_IT17_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_ILE_IT18_MASK _ULL(0x40000) /* Interrupt 18 owner */
+#define KVX_SFR_ILE_IT18_SHIFT 18
+#define KVX_SFR_ILE_IT18_WIDTH 1
+#define KVX_SFR_ILE_IT18_WFXL_MASK _ULL(0x40000)
+#define KVX_SFR_ILE_IT18_WFXL_CLEAR _ULL(0x40000)
+#define KVX_SFR_ILE_IT18_WFXL_SET _ULL(0x4000000000000)
+
+#define KVX_SFR_ILE_IT19_MASK _ULL(0x80000) /* Interrupt 19 owner */
+#define KVX_SFR_ILE_IT19_SHIFT 19
+#define KVX_SFR_ILE_IT19_WIDTH 1
+#define KVX_SFR_ILE_IT19_WFXL_MASK _ULL(0x80000)
+#define KVX_SFR_ILE_IT19_WFXL_CLEAR _ULL(0x80000)
+#define KVX_SFR_ILE_IT19_WFXL_SET _ULL(0x8000000000000)
+
+#define KVX_SFR_ILE_IT20_MASK _ULL(0x100000) /* Interrupt 20 owner */
+#define KVX_SFR_ILE_IT20_SHIFT 20
+#define KVX_SFR_ILE_IT20_WIDTH 1
+#define KVX_SFR_ILE_IT20_WFXL_MASK _ULL(0x100000)
+#define KVX_SFR_ILE_IT20_WFXL_CLEAR _ULL(0x100000)
+#define KVX_SFR_ILE_IT20_WFXL_SET _ULL(0x10000000000000)
+
+#define KVX_SFR_ILE_IT21_MASK _ULL(0x200000) /* Interrupt 21 owner */
+#define KVX_SFR_ILE_IT21_SHIFT 21
+#define KVX_SFR_ILE_IT21_WIDTH 1
+#define KVX_SFR_ILE_IT21_WFXL_MASK _ULL(0x200000)
+#define KVX_SFR_ILE_IT21_WFXL_CLEAR _ULL(0x200000)
+#define KVX_SFR_ILE_IT21_WFXL_SET _ULL(0x20000000000000)
+
+#define KVX_SFR_ILE_IT22_MASK _ULL(0x400000) /* Interrupt 22 owner */
+#define KVX_SFR_ILE_IT22_SHIFT 22
+#define KVX_SFR_ILE_IT22_WIDTH 1
+#define KVX_SFR_ILE_IT22_WFXL_MASK _ULL(0x400000)
+#define KVX_SFR_ILE_IT22_WFXL_CLEAR _ULL(0x400000)
+#define KVX_SFR_ILE_IT22_WFXL_SET _ULL(0x40000000000000)
+
+#define KVX_SFR_ILE_IT23_MASK _ULL(0x800000) /* Interrupt 23 owner */
+#define KVX_SFR_ILE_IT23_SHIFT 23
+#define KVX_SFR_ILE_IT23_WIDTH 1
+#define KVX_SFR_ILE_IT23_WFXL_MASK _ULL(0x800000)
+#define KVX_SFR_ILE_IT23_WFXL_CLEAR _ULL(0x800000)
+#define KVX_SFR_ILE_IT23_WFXL_SET _ULL(0x80000000000000)
+
+#define KVX_SFR_ILE_IT24_MASK _ULL(0x1000000) /* Interrupt 24 owner */
+#define KVX_SFR_ILE_IT24_SHIFT 24
+#define KVX_SFR_ILE_IT24_WIDTH 1
+#define KVX_SFR_ILE_IT24_WFXL_MASK _ULL(0x1000000)
+#define KVX_SFR_ILE_IT24_WFXL_CLEAR _ULL(0x1000000)
+#define KVX_SFR_ILE_IT24_WFXL_SET _ULL(0x100000000000000)
+
+#define KVX_SFR_ILE_IT25_MASK _ULL(0x2000000) /* Interrupt 25 owner */
+#define KVX_SFR_ILE_IT25_SHIFT 25
+#define KVX_SFR_ILE_IT25_WIDTH 1
+#define KVX_SFR_ILE_IT25_WFXL_MASK _ULL(0x2000000)
+#define KVX_SFR_ILE_IT25_WFXL_CLEAR _ULL(0x2000000)
+#define KVX_SFR_ILE_IT25_WFXL_SET _ULL(0x200000000000000)
+
+#define KVX_SFR_ILE_IT26_MASK _ULL(0x4000000) /* Interrupt 26 owner */
+#define KVX_SFR_ILE_IT26_SHIFT 26
+#define KVX_SFR_ILE_IT26_WIDTH 1
+#define KVX_SFR_ILE_IT26_WFXL_MASK _ULL(0x4000000)
+#define KVX_SFR_ILE_IT26_WFXL_CLEAR _ULL(0x4000000)
+#define KVX_SFR_ILE_IT26_WFXL_SET _ULL(0x400000000000000)
+
+#define KVX_SFR_ILE_IT27_MASK _ULL(0x8000000) /* Interrupt 27 owner */
+#define KVX_SFR_ILE_IT27_SHIFT 27
+#define KVX_SFR_ILE_IT27_WIDTH 1
+#define KVX_SFR_ILE_IT27_WFXL_MASK _ULL(0x8000000)
+#define KVX_SFR_ILE_IT27_WFXL_CLEAR _ULL(0x8000000)
+#define KVX_SFR_ILE_IT27_WFXL_SET _ULL(0x800000000000000)
+
+#define KVX_SFR_ILE_IT28_MASK _ULL(0x10000000) /* Interrupt 28 owner */
+#define KVX_SFR_ILE_IT28_SHIFT 28
+#define KVX_SFR_ILE_IT28_WIDTH 1
+#define KVX_SFR_ILE_IT28_WFXL_MASK _ULL(0x10000000)
+#define KVX_SFR_ILE_IT28_WFXL_CLEAR _ULL(0x10000000)
+#define KVX_SFR_ILE_IT28_WFXL_SET _ULL(0x1000000000000000)
+
+#define KVX_SFR_ILE_IT29_MASK _ULL(0x20000000) /* Interrupt 29 owner */
+#define KVX_SFR_ILE_IT29_SHIFT 29
+#define KVX_SFR_ILE_IT29_WIDTH 1
+#define KVX_SFR_ILE_IT29_WFXL_MASK _ULL(0x20000000)
+#define KVX_SFR_ILE_IT29_WFXL_CLEAR _ULL(0x20000000)
+#define KVX_SFR_ILE_IT29_WFXL_SET _ULL(0x2000000000000000)
+
+#define KVX_SFR_ILE_IT30_MASK _ULL(0x40000000) /* Interrupt 30 owner */
+#define KVX_SFR_ILE_IT30_SHIFT 30
+#define KVX_SFR_ILE_IT30_WIDTH 1
+#define KVX_SFR_ILE_IT30_WFXL_MASK _ULL(0x40000000)
+#define KVX_SFR_ILE_IT30_WFXL_CLEAR _ULL(0x40000000)
+#define KVX_SFR_ILE_IT30_WFXL_SET _ULL(0x4000000000000000)
+
+#define KVX_SFR_ILE_IT31_MASK _ULL(0x80000000) /* Interrupt 31 owner */
+#define KVX_SFR_ILE_IT31_SHIFT 31
+#define KVX_SFR_ILE_IT31_WIDTH 1
+#define KVX_SFR_ILE_IT31_WFXL_MASK _ULL(0x80000000)
+#define KVX_SFR_ILE_IT31_WFXL_CLEAR _ULL(0x80000000)
+#define KVX_SFR_ILE_IT31_WFXL_SET _ULL(0x8000000000000000)
+
+#define KVX_SFR_ILL_IT0_MASK _ULL(0x3) /* Interrupt 0 owner */
+#define KVX_SFR_ILL_IT0_SHIFT 0
+#define KVX_SFR_ILL_IT0_WIDTH 2
+#define KVX_SFR_ILL_IT0_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_ILL_IT0_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_ILL_IT0_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_ILL_IT1_MASK _ULL(0xc) /* Interrupt 1 owner */
+#define KVX_SFR_ILL_IT1_SHIFT 2
+#define KVX_SFR_ILL_IT1_WIDTH 2
+#define KVX_SFR_ILL_IT1_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_ILL_IT1_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_ILL_IT1_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_ILL_IT2_MASK _ULL(0x30) /* Interrupt 2 owner */
+#define KVX_SFR_ILL_IT2_SHIFT 4
+#define KVX_SFR_ILL_IT2_WIDTH 2
+#define KVX_SFR_ILL_IT2_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_ILL_IT2_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_ILL_IT2_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ILL_IT3_MASK _ULL(0xc0) /* Interrupt 3 owner */
+#define KVX_SFR_ILL_IT3_SHIFT 6
+#define KVX_SFR_ILL_IT3_WIDTH 2
+#define KVX_SFR_ILL_IT3_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_ILL_IT3_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_ILL_IT3_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ILL_IT4_MASK _ULL(0x300) /* Interrupt 4 owner */
+#define KVX_SFR_ILL_IT4_SHIFT 8
+#define KVX_SFR_ILL_IT4_WIDTH 2
+#define KVX_SFR_ILL_IT4_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_ILL_IT4_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_ILL_IT4_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ILL_IT5_MASK _ULL(0xc00) /* Interrupt 5 owner */
+#define KVX_SFR_ILL_IT5_SHIFT 10
+#define KVX_SFR_ILL_IT5_WIDTH 2
+#define KVX_SFR_ILL_IT5_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_ILL_IT5_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_ILL_IT5_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ILL_IT6_MASK _ULL(0x3000) /* Interrupt 6 owner */
+#define KVX_SFR_ILL_IT6_SHIFT 12
+#define KVX_SFR_ILL_IT6_WIDTH 2
+#define KVX_SFR_ILL_IT6_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_ILL_IT6_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_ILL_IT6_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ILL_IT7_MASK _ULL(0xc000) /* Interrupt 7 owner */
+#define KVX_SFR_ILL_IT7_SHIFT 14
+#define KVX_SFR_ILL_IT7_WIDTH 2
+#define KVX_SFR_ILL_IT7_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_ILL_IT7_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_ILL_IT7_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_ILL_IT8_MASK _ULL(0x30000) /* Interrupt 8 owner */
+#define KVX_SFR_ILL_IT8_SHIFT 16
+#define KVX_SFR_ILL_IT8_WIDTH 2
+#define KVX_SFR_ILL_IT8_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_ILL_IT8_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_ILL_IT8_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_ILL_IT9_MASK _ULL(0xc0000) /* Interrupt 9 owner */
+#define KVX_SFR_ILL_IT9_SHIFT 18
+#define KVX_SFR_ILL_IT9_WIDTH 2
+#define KVX_SFR_ILL_IT9_WFXL_MASK _ULL(0xc0000)
+#define KVX_SFR_ILL_IT9_WFXL_CLEAR _ULL(0xc0000)
+#define KVX_SFR_ILL_IT9_WFXL_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_ILL_IT10_MASK _ULL(0x300000) /* Interrupt 10 owner */
+#define KVX_SFR_ILL_IT10_SHIFT 20
+#define KVX_SFR_ILL_IT10_WIDTH 2
+#define KVX_SFR_ILL_IT10_WFXL_MASK _ULL(0x300000)
+#define KVX_SFR_ILL_IT10_WFXL_CLEAR _ULL(0x300000)
+#define KVX_SFR_ILL_IT10_WFXL_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_ILL_IT11_MASK _ULL(0xc00000) /* Interrupt 11 owner */
+#define KVX_SFR_ILL_IT11_SHIFT 22
+#define KVX_SFR_ILL_IT11_WIDTH 2
+#define KVX_SFR_ILL_IT11_WFXL_MASK _ULL(0xc00000)
+#define KVX_SFR_ILL_IT11_WFXL_CLEAR _ULL(0xc00000)
+#define KVX_SFR_ILL_IT11_WFXL_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_ILL_IT12_MASK _ULL(0x3000000) /* Interrupt 12 owner */
+#define KVX_SFR_ILL_IT12_SHIFT 24
+#define KVX_SFR_ILL_IT12_WIDTH 2
+#define KVX_SFR_ILL_IT12_WFXL_MASK _ULL(0x3000000)
+#define KVX_SFR_ILL_IT12_WFXL_CLEAR _ULL(0x3000000)
+#define KVX_SFR_ILL_IT12_WFXL_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_ILL_IT13_MASK _ULL(0xc000000) /* Interrupt 13 owner */
+#define KVX_SFR_ILL_IT13_SHIFT 26
+#define KVX_SFR_ILL_IT13_WIDTH 2
+#define KVX_SFR_ILL_IT13_WFXL_MASK _ULL(0xc000000)
+#define KVX_SFR_ILL_IT13_WFXL_CLEAR _ULL(0xc000000)
+#define KVX_SFR_ILL_IT13_WFXL_SET _ULL(0xc00000000000000)
+
+#define KVX_SFR_ILL_IT14_MASK _ULL(0x30000000) /* Interrupt 14 owner */
+#define KVX_SFR_ILL_IT14_SHIFT 28
+#define KVX_SFR_ILL_IT14_WIDTH 2
+#define KVX_SFR_ILL_IT14_WFXL_MASK _ULL(0x30000000)
+#define KVX_SFR_ILL_IT14_WFXL_CLEAR _ULL(0x30000000)
+#define KVX_SFR_ILL_IT14_WFXL_SET _ULL(0x3000000000000000)
+
+#define KVX_SFR_ILL_IT15_MASK _ULL(0xc0000000) /* Interrupt 15 owner */
+#define KVX_SFR_ILL_IT15_SHIFT 30
+#define KVX_SFR_ILL_IT15_WIDTH 2
+#define KVX_SFR_ILL_IT15_WFXL_MASK _ULL(0xc0000000)
+#define KVX_SFR_ILL_IT15_WFXL_CLEAR _ULL(0xc0000000)
+#define KVX_SFR_ILL_IT15_WFXL_SET _ULL(0xc000000000000000)
+
+#define KVX_SFR_ILL_IT16_MASK _ULL(0x300000000) /* Interrupt 16 owner */
+#define KVX_SFR_ILL_IT16_SHIFT 32
+#define KVX_SFR_ILL_IT16_WIDTH 2
+#define KVX_SFR_ILL_IT16_WFXM_MASK _ULL(0x300000000)
+#define KVX_SFR_ILL_IT16_WFXM_CLEAR _ULL(0x3)
+#define KVX_SFR_ILL_IT16_WFXM_SET _ULL(0x300000000)
+
+#define KVX_SFR_ILL_IT17_MASK _ULL(0xc00000000) /* Interrupt 17 owner */
+#define KVX_SFR_ILL_IT17_SHIFT 34
+#define KVX_SFR_ILL_IT17_WIDTH 2
+#define KVX_SFR_ILL_IT17_WFXM_MASK _ULL(0xc00000000)
+#define KVX_SFR_ILL_IT17_WFXM_CLEAR _ULL(0xc)
+#define KVX_SFR_ILL_IT17_WFXM_SET _ULL(0xc00000000)
+
+#define KVX_SFR_ILL_IT18_MASK _ULL(0x3000000000) /* Interrupt 18 owner */
+#define KVX_SFR_ILL_IT18_SHIFT 36
+#define KVX_SFR_ILL_IT18_WIDTH 2
+#define KVX_SFR_ILL_IT18_WFXM_MASK _ULL(0x3000000000)
+#define KVX_SFR_ILL_IT18_WFXM_CLEAR _ULL(0x30)
+#define KVX_SFR_ILL_IT18_WFXM_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ILL_IT19_MASK _ULL(0xc000000000) /* Interrupt 19 owner */
+#define KVX_SFR_ILL_IT19_SHIFT 38
+#define KVX_SFR_ILL_IT19_WIDTH 2
+#define KVX_SFR_ILL_IT19_WFXM_MASK _ULL(0xc000000000)
+#define KVX_SFR_ILL_IT19_WFXM_CLEAR _ULL(0xc0)
+#define KVX_SFR_ILL_IT19_WFXM_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ILL_IT20_MASK _ULL(0x30000000000) /* Interrupt 20 owner */
+#define KVX_SFR_ILL_IT20_SHIFT 40
+#define KVX_SFR_ILL_IT20_WIDTH 2
+#define KVX_SFR_ILL_IT20_WFXM_MASK _ULL(0x30000000000)
+#define KVX_SFR_ILL_IT20_WFXM_CLEAR _ULL(0x300)
+#define KVX_SFR_ILL_IT20_WFXM_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ILL_IT21_MASK _ULL(0xc0000000000) /* Interrupt 21 owner */
+#define KVX_SFR_ILL_IT21_SHIFT 42
+#define KVX_SFR_ILL_IT21_WIDTH 2
+#define KVX_SFR_ILL_IT21_WFXM_MASK _ULL(0xc0000000000)
+#define KVX_SFR_ILL_IT21_WFXM_CLEAR _ULL(0xc00)
+#define KVX_SFR_ILL_IT21_WFXM_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ILL_IT22_MASK _ULL(0x300000000000) /* Interrupt 22 owner */
+#define KVX_SFR_ILL_IT22_SHIFT 44
+#define KVX_SFR_ILL_IT22_WIDTH 2
+#define KVX_SFR_ILL_IT22_WFXM_MASK _ULL(0x300000000000)
+#define KVX_SFR_ILL_IT22_WFXM_CLEAR _ULL(0x3000)
+#define KVX_SFR_ILL_IT22_WFXM_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ILL_IT23_MASK _ULL(0xc00000000000) /* Interrupt 23 owner */
+#define KVX_SFR_ILL_IT23_SHIFT 46
+#define KVX_SFR_ILL_IT23_WIDTH 2
+#define KVX_SFR_ILL_IT23_WFXM_MASK _ULL(0xc00000000000)
+#define KVX_SFR_ILL_IT23_WFXM_CLEAR _ULL(0xc000)
+#define KVX_SFR_ILL_IT23_WFXM_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_ILL_IT24_MASK _ULL(0x3000000000000) /* Interrupt 24 owner */
+#define KVX_SFR_ILL_IT24_SHIFT 48
+#define KVX_SFR_ILL_IT24_WIDTH 2
+#define KVX_SFR_ILL_IT24_WFXM_MASK _ULL(0x3000000000000)
+#define KVX_SFR_ILL_IT24_WFXM_CLEAR _ULL(0x30000)
+#define KVX_SFR_ILL_IT24_WFXM_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_ILL_IT25_MASK _ULL(0xc000000000000) /* Interrupt 25 owner */
+#define KVX_SFR_ILL_IT25_SHIFT 50
+#define KVX_SFR_ILL_IT25_WIDTH 2
+#define KVX_SFR_ILL_IT25_WFXM_MASK _ULL(0xc000000000000)
+#define KVX_SFR_ILL_IT25_WFXM_CLEAR _ULL(0xc0000)
+#define KVX_SFR_ILL_IT25_WFXM_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_ILL_IT26_MASK _ULL(0x30000000000000) /* Interrupt 26 owner */
+#define KVX_SFR_ILL_IT26_SHIFT 52
+#define KVX_SFR_ILL_IT26_WIDTH 2
+#define KVX_SFR_ILL_IT26_WFXM_MASK _ULL(0x30000000000000)
+#define KVX_SFR_ILL_IT26_WFXM_CLEAR _ULL(0x300000)
+#define KVX_SFR_ILL_IT26_WFXM_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_ILL_IT27_MASK _ULL(0xc0000000000000) /* Interrupt 27 owner */
+#define KVX_SFR_ILL_IT27_SHIFT 54
+#define KVX_SFR_ILL_IT27_WIDTH 2
+#define KVX_SFR_ILL_IT27_WFXM_MASK _ULL(0xc0000000000000)
+#define KVX_SFR_ILL_IT27_WFXM_CLEAR _ULL(0xc00000)
+#define KVX_SFR_ILL_IT27_WFXM_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_ILL_IT28_MASK _ULL(0x300000000000000) /* Interrupt 28 owner */
+#define KVX_SFR_ILL_IT28_SHIFT 56
+#define KVX_SFR_ILL_IT28_WIDTH 2
+#define KVX_SFR_ILL_IT28_WFXM_MASK _ULL(0x300000000000000)
+#define KVX_SFR_ILL_IT28_WFXM_CLEAR _ULL(0x3000000)
+#define KVX_SFR_ILL_IT28_WFXM_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_ILL_IT29_MASK _ULL(0xc00000000000000) /* Interrupt 29 owner */
+#define KVX_SFR_ILL_IT29_SHIFT 58
+#define KVX_SFR_ILL_IT29_WIDTH 2
+#define KVX_SFR_ILL_IT29_WFXM_MASK _ULL(0xc00000000000000)
+#define KVX_SFR_ILL_IT29_WFXM_CLEAR _ULL(0xc000000)
+#define KVX_SFR_ILL_IT29_WFXM_SET _ULL(0xc00000000000000)
+
+#define KVX_SFR_ILL_IT30_MASK _ULL(0x3000000000000000) /* Interrupt 30 owner */
+#define KVX_SFR_ILL_IT30_SHIFT 60
+#define KVX_SFR_ILL_IT30_WIDTH 2
+#define KVX_SFR_ILL_IT30_WFXM_MASK _ULL(0x3000000000000000)
+#define KVX_SFR_ILL_IT30_WFXM_CLEAR _ULL(0x30000000)
+#define KVX_SFR_ILL_IT30_WFXM_SET _ULL(0x3000000000000000)
+
+#define KVX_SFR_ILL_IT31_MASK _ULL(0xc000000000000000) /* Interrupt 31 owner */
+#define KVX_SFR_ILL_IT31_SHIFT 62
+#define KVX_SFR_ILL_IT31_WIDTH 2
+#define KVX_SFR_ILL_IT31_WFXM_MASK _ULL(0xc000000000000000)
+#define KVX_SFR_ILL_IT31_WFXM_CLEAR _ULL(0xc0000000)
+#define KVX_SFR_ILL_IT31_WFXM_SET _ULL(0xc000000000000000)
+
+#define KVX_SFR_ILR_IT0_MASK _ULL(0x1) /* Interrupt 0 owner */
+#define KVX_SFR_ILR_IT0_SHIFT 0
+#define KVX_SFR_ILR_IT0_WIDTH 1
+#define KVX_SFR_ILR_IT0_WFXL_MASK _ULL(0x1)
+#define KVX_SFR_ILR_IT0_WFXL_CLEAR _ULL(0x1)
+#define KVX_SFR_ILR_IT0_WFXL_SET _ULL(0x100000000)
+
+#define KVX_SFR_ILR_IT1_MASK _ULL(0x2) /* Interrupt 1 owner */
+#define KVX_SFR_ILR_IT1_SHIFT 1
+#define KVX_SFR_ILR_IT1_WIDTH 1
+#define KVX_SFR_ILR_IT1_WFXL_MASK _ULL(0x2)
+#define KVX_SFR_ILR_IT1_WFXL_CLEAR _ULL(0x2)
+#define KVX_SFR_ILR_IT1_WFXL_SET _ULL(0x200000000)
+
+#define KVX_SFR_ILR_IT2_MASK _ULL(0x4) /* Interrupt 2 owner */
+#define KVX_SFR_ILR_IT2_SHIFT 2
+#define KVX_SFR_ILR_IT2_WIDTH 1
+#define KVX_SFR_ILR_IT2_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_ILR_IT2_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_ILR_IT2_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_ILR_IT3_MASK _ULL(0x8) /* Interrupt 3 owner */
+#define KVX_SFR_ILR_IT3_SHIFT 3
+#define KVX_SFR_ILR_IT3_WIDTH 1
+#define KVX_SFR_ILR_IT3_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_ILR_IT3_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_ILR_IT3_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_ILR_IT4_MASK _ULL(0x10) /* Interrupt 4 owner */
+#define KVX_SFR_ILR_IT4_SHIFT 4
+#define KVX_SFR_ILR_IT4_WIDTH 1
+#define KVX_SFR_ILR_IT4_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_ILR_IT4_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_ILR_IT4_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_ILR_IT5_MASK _ULL(0x20) /* Interrupt 5 owner */
+#define KVX_SFR_ILR_IT5_SHIFT 5
+#define KVX_SFR_ILR_IT5_WIDTH 1
+#define KVX_SFR_ILR_IT5_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_ILR_IT5_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_ILR_IT5_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_ILR_IT6_MASK _ULL(0x40) /* Interrupt 6 owner */
+#define KVX_SFR_ILR_IT6_SHIFT 6
+#define KVX_SFR_ILR_IT6_WIDTH 1
+#define KVX_SFR_ILR_IT6_WFXL_MASK _ULL(0x40)
+#define KVX_SFR_ILR_IT6_WFXL_CLEAR _ULL(0x40)
+#define KVX_SFR_ILR_IT6_WFXL_SET _ULL(0x4000000000)
+
+#define KVX_SFR_ILR_IT7_MASK _ULL(0x80) /* Interrupt 7 owner */
+#define KVX_SFR_ILR_IT7_SHIFT 7
+#define KVX_SFR_ILR_IT7_WIDTH 1
+#define KVX_SFR_ILR_IT7_WFXL_MASK _ULL(0x80)
+#define KVX_SFR_ILR_IT7_WFXL_CLEAR _ULL(0x80)
+#define KVX_SFR_ILR_IT7_WFXL_SET _ULL(0x8000000000)
+
+#define KVX_SFR_ILR_IT8_MASK _ULL(0x100) /* Interrupt 8 owner */
+#define KVX_SFR_ILR_IT8_SHIFT 8
+#define KVX_SFR_ILR_IT8_WIDTH 1
+#define KVX_SFR_ILR_IT8_WFXL_MASK _ULL(0x100)
+#define KVX_SFR_ILR_IT8_WFXL_CLEAR _ULL(0x100)
+#define KVX_SFR_ILR_IT8_WFXL_SET _ULL(0x10000000000)
+
+#define KVX_SFR_ILR_IT9_MASK _ULL(0x200) /* Interrupt 9 owner */
+#define KVX_SFR_ILR_IT9_SHIFT 9
+#define KVX_SFR_ILR_IT9_WIDTH 1
+#define KVX_SFR_ILR_IT9_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_ILR_IT9_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_ILR_IT9_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_ILR_IT10_MASK _ULL(0x400) /* Interrupt 10 owner */
+#define KVX_SFR_ILR_IT10_SHIFT 10
+#define KVX_SFR_ILR_IT10_WIDTH 1
+#define KVX_SFR_ILR_IT10_WFXL_MASK _ULL(0x400)
+#define KVX_SFR_ILR_IT10_WFXL_CLEAR _ULL(0x400)
+#define KVX_SFR_ILR_IT10_WFXL_SET _ULL(0x40000000000)
+
+#define KVX_SFR_ILR_IT11_MASK _ULL(0x800) /* Interrupt 11 owner */
+#define KVX_SFR_ILR_IT11_SHIFT 11
+#define KVX_SFR_ILR_IT11_WIDTH 1
+#define KVX_SFR_ILR_IT11_WFXL_MASK _ULL(0x800)
+#define KVX_SFR_ILR_IT11_WFXL_CLEAR _ULL(0x800)
+#define KVX_SFR_ILR_IT11_WFXL_SET _ULL(0x80000000000)
+
+#define KVX_SFR_ILR_IT12_MASK _ULL(0x1000) /* Interrupt 12 owner */
+#define KVX_SFR_ILR_IT12_SHIFT 12
+#define KVX_SFR_ILR_IT12_WIDTH 1
+#define KVX_SFR_ILR_IT12_WFXL_MASK _ULL(0x1000)
+#define KVX_SFR_ILR_IT12_WFXL_CLEAR _ULL(0x1000)
+#define KVX_SFR_ILR_IT12_WFXL_SET _ULL(0x100000000000)
+
+#define KVX_SFR_ILR_IT13_MASK _ULL(0x2000) /* Interrupt 13 owner */
+#define KVX_SFR_ILR_IT13_SHIFT 13
+#define KVX_SFR_ILR_IT13_WIDTH 1
+#define KVX_SFR_ILR_IT13_WFXL_MASK _ULL(0x2000)
+#define KVX_SFR_ILR_IT13_WFXL_CLEAR _ULL(0x2000)
+#define KVX_SFR_ILR_IT13_WFXL_SET _ULL(0x200000000000)
+
+#define KVX_SFR_ILR_IT14_MASK _ULL(0x4000) /* Interrupt 14 owner */
+#define KVX_SFR_ILR_IT14_SHIFT 14
+#define KVX_SFR_ILR_IT14_WIDTH 1
+#define KVX_SFR_ILR_IT14_WFXL_MASK _ULL(0x4000)
+#define KVX_SFR_ILR_IT14_WFXL_CLEAR _ULL(0x4000)
+#define KVX_SFR_ILR_IT14_WFXL_SET _ULL(0x400000000000)
+
+#define KVX_SFR_ILR_IT15_MASK _ULL(0x8000) /* Interrupt 15 owner */
+#define KVX_SFR_ILR_IT15_SHIFT 15
+#define KVX_SFR_ILR_IT15_WIDTH 1
+#define KVX_SFR_ILR_IT15_WFXL_MASK _ULL(0x8000)
+#define KVX_SFR_ILR_IT15_WFXL_CLEAR _ULL(0x8000)
+#define KVX_SFR_ILR_IT15_WFXL_SET _ULL(0x800000000000)
+
+#define KVX_SFR_ILR_IT16_MASK _ULL(0x10000) /* Interrupt 16 owner */
+#define KVX_SFR_ILR_IT16_SHIFT 16
+#define KVX_SFR_ILR_IT16_WIDTH 1
+#define KVX_SFR_ILR_IT16_WFXL_MASK _ULL(0x10000)
+#define KVX_SFR_ILR_IT16_WFXL_CLEAR _ULL(0x10000)
+#define KVX_SFR_ILR_IT16_WFXL_SET _ULL(0x1000000000000)
+
+#define KVX_SFR_ILR_IT17_MASK _ULL(0x20000) /* Interrupt 17 owner */
+#define KVX_SFR_ILR_IT17_SHIFT 17
+#define KVX_SFR_ILR_IT17_WIDTH 1
+#define KVX_SFR_ILR_IT17_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_ILR_IT17_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_ILR_IT17_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_ILR_IT18_MASK _ULL(0x40000) /* Interrupt 18 owner */
+#define KVX_SFR_ILR_IT18_SHIFT 18
+#define KVX_SFR_ILR_IT18_WIDTH 1
+#define KVX_SFR_ILR_IT18_WFXL_MASK _ULL(0x40000)
+#define KVX_SFR_ILR_IT18_WFXL_CLEAR _ULL(0x40000)
+#define KVX_SFR_ILR_IT18_WFXL_SET _ULL(0x4000000000000)
+
+#define KVX_SFR_ILR_IT19_MASK _ULL(0x80000) /* Interrupt 19 owner */
+#define KVX_SFR_ILR_IT19_SHIFT 19
+#define KVX_SFR_ILR_IT19_WIDTH 1
+#define KVX_SFR_ILR_IT19_WFXL_MASK _ULL(0x80000)
+#define KVX_SFR_ILR_IT19_WFXL_CLEAR _ULL(0x80000)
+#define KVX_SFR_ILR_IT19_WFXL_SET _ULL(0x8000000000000)
+
+#define KVX_SFR_ILR_IT20_MASK _ULL(0x100000) /* Interrupt 20 owner */
+#define KVX_SFR_ILR_IT20_SHIFT 20
+#define KVX_SFR_ILR_IT20_WIDTH 1
+#define KVX_SFR_ILR_IT20_WFXL_MASK _ULL(0x100000)
+#define KVX_SFR_ILR_IT20_WFXL_CLEAR _ULL(0x100000)
+#define KVX_SFR_ILR_IT20_WFXL_SET _ULL(0x10000000000000)
+
+#define KVX_SFR_ILR_IT21_MASK _ULL(0x200000) /* Interrupt 21 owner */
+#define KVX_SFR_ILR_IT21_SHIFT 21
+#define KVX_SFR_ILR_IT21_WIDTH 1
+#define KVX_SFR_ILR_IT21_WFXL_MASK _ULL(0x200000)
+#define KVX_SFR_ILR_IT21_WFXL_CLEAR _ULL(0x200000)
+#define KVX_SFR_ILR_IT21_WFXL_SET _ULL(0x20000000000000)
+
+#define KVX_SFR_ILR_IT22_MASK _ULL(0x400000) /* Interrupt 22 owner */
+#define KVX_SFR_ILR_IT22_SHIFT 22
+#define KVX_SFR_ILR_IT22_WIDTH 1
+#define KVX_SFR_ILR_IT22_WFXL_MASK _ULL(0x400000)
+#define KVX_SFR_ILR_IT22_WFXL_CLEAR _ULL(0x400000)
+#define KVX_SFR_ILR_IT22_WFXL_SET _ULL(0x40000000000000)
+
+#define KVX_SFR_ILR_IT23_MASK _ULL(0x800000) /* Interrupt 23 owner */
+#define KVX_SFR_ILR_IT23_SHIFT 23
+#define KVX_SFR_ILR_IT23_WIDTH 1
+#define KVX_SFR_ILR_IT23_WFXL_MASK _ULL(0x800000)
+#define KVX_SFR_ILR_IT23_WFXL_CLEAR _ULL(0x800000)
+#define KVX_SFR_ILR_IT23_WFXL_SET _ULL(0x80000000000000)
+
+#define KVX_SFR_ILR_IT24_MASK _ULL(0x1000000) /* Interrupt 24 owner */
+#define KVX_SFR_ILR_IT24_SHIFT 24
+#define KVX_SFR_ILR_IT24_WIDTH 1
+#define KVX_SFR_ILR_IT24_WFXL_MASK _ULL(0x1000000)
+#define KVX_SFR_ILR_IT24_WFXL_CLEAR _ULL(0x1000000)
+#define KVX_SFR_ILR_IT24_WFXL_SET _ULL(0x100000000000000)
+
+#define KVX_SFR_ILR_IT25_MASK _ULL(0x2000000) /* Interrupt 25 owner */
+#define KVX_SFR_ILR_IT25_SHIFT 25
+#define KVX_SFR_ILR_IT25_WIDTH 1
+#define KVX_SFR_ILR_IT25_WFXL_MASK _ULL(0x2000000)
+#define KVX_SFR_ILR_IT25_WFXL_CLEAR _ULL(0x2000000)
+#define KVX_SFR_ILR_IT25_WFXL_SET _ULL(0x200000000000000)
+
+#define KVX_SFR_ILR_IT26_MASK _ULL(0x4000000) /* Interrupt 26 owner */
+#define KVX_SFR_ILR_IT26_SHIFT 26
+#define KVX_SFR_ILR_IT26_WIDTH 1
+#define KVX_SFR_ILR_IT26_WFXL_MASK _ULL(0x4000000)
+#define KVX_SFR_ILR_IT26_WFXL_CLEAR _ULL(0x4000000)
+#define KVX_SFR_ILR_IT26_WFXL_SET _ULL(0x400000000000000)
+
+#define KVX_SFR_ILR_IT27_MASK _ULL(0x8000000) /* Interrupt 27 owner */
+#define KVX_SFR_ILR_IT27_SHIFT 27
+#define KVX_SFR_ILR_IT27_WIDTH 1
+#define KVX_SFR_ILR_IT27_WFXL_MASK _ULL(0x8000000)
+#define KVX_SFR_ILR_IT27_WFXL_CLEAR _ULL(0x8000000)
+#define KVX_SFR_ILR_IT27_WFXL_SET _ULL(0x800000000000000)
+
+#define KVX_SFR_ILR_IT28_MASK _ULL(0x10000000) /* Interrupt 28 owner */
+#define KVX_SFR_ILR_IT28_SHIFT 28
+#define KVX_SFR_ILR_IT28_WIDTH 1
+#define KVX_SFR_ILR_IT28_WFXL_MASK _ULL(0x10000000)
+#define KVX_SFR_ILR_IT28_WFXL_CLEAR _ULL(0x10000000)
+#define KVX_SFR_ILR_IT28_WFXL_SET _ULL(0x1000000000000000)
+
+#define KVX_SFR_ILR_IT29_MASK _ULL(0x20000000) /* Interrupt 29 owner */
+#define KVX_SFR_ILR_IT29_SHIFT 29
+#define KVX_SFR_ILR_IT29_WIDTH 1
+#define KVX_SFR_ILR_IT29_WFXL_MASK _ULL(0x20000000)
+#define KVX_SFR_ILR_IT29_WFXL_CLEAR _ULL(0x20000000)
+#define KVX_SFR_ILR_IT29_WFXL_SET _ULL(0x2000000000000000)
+
+#define KVX_SFR_ILR_IT30_MASK _ULL(0x40000000) /* Interrupt 30 owner */
+#define KVX_SFR_ILR_IT30_SHIFT 30
+#define KVX_SFR_ILR_IT30_WIDTH 1
+#define KVX_SFR_ILR_IT30_WFXL_MASK _ULL(0x40000000)
+#define KVX_SFR_ILR_IT30_WFXL_CLEAR _ULL(0x40000000)
+#define KVX_SFR_ILR_IT30_WFXL_SET _ULL(0x4000000000000000)
+
+#define KVX_SFR_ILR_IT31_MASK _ULL(0x80000000) /* Interrupt 31 owner */
+#define KVX_SFR_ILR_IT31_SHIFT 31
+#define KVX_SFR_ILR_IT31_WIDTH 1
+#define KVX_SFR_ILR_IT31_WFXL_MASK _ULL(0x80000000)
+#define KVX_SFR_ILR_IT31_WFXL_CLEAR _ULL(0x80000000)
+#define KVX_SFR_ILR_IT31_WFXL_SET _ULL(0x8000000000000000)
+
+#define KVX_SFR_ITOW_IT0_MASK _ULL(0x3) /* Interrupt 0 owner */
+#define KVX_SFR_ITOW_IT0_SHIFT 0
+#define KVX_SFR_ITOW_IT0_WIDTH 2
+#define KVX_SFR_ITOW_IT0_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_ITOW_IT0_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_ITOW_IT0_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_ITOW_IT1_MASK _ULL(0xc) /* Interrupt 1 owner */
+#define KVX_SFR_ITOW_IT1_SHIFT 2
+#define KVX_SFR_ITOW_IT1_WIDTH 2
+#define KVX_SFR_ITOW_IT1_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_ITOW_IT1_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_ITOW_IT1_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_ITOW_IT2_MASK _ULL(0x30) /* Interrupt 2 owner */
+#define KVX_SFR_ITOW_IT2_SHIFT 4
+#define KVX_SFR_ITOW_IT2_WIDTH 2
+#define KVX_SFR_ITOW_IT2_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_ITOW_IT2_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_ITOW_IT2_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ITOW_IT3_MASK _ULL(0xc0) /* Interrupt 3 owner */
+#define KVX_SFR_ITOW_IT3_SHIFT 6
+#define KVX_SFR_ITOW_IT3_WIDTH 2
+#define KVX_SFR_ITOW_IT3_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_ITOW_IT3_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_ITOW_IT3_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ITOW_IT4_MASK _ULL(0x300) /* Interrupt 4 owner */
+#define KVX_SFR_ITOW_IT4_SHIFT 8
+#define KVX_SFR_ITOW_IT4_WIDTH 2
+#define KVX_SFR_ITOW_IT4_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_ITOW_IT4_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_ITOW_IT4_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ITOW_IT5_MASK _ULL(0xc00) /* Interrupt 5 owner */
+#define KVX_SFR_ITOW_IT5_SHIFT 10
+#define KVX_SFR_ITOW_IT5_WIDTH 2
+#define KVX_SFR_ITOW_IT5_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_ITOW_IT5_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_ITOW_IT5_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ITOW_IT6_MASK _ULL(0x3000) /* Interrupt 6 owner */
+#define KVX_SFR_ITOW_IT6_SHIFT 12
+#define KVX_SFR_ITOW_IT6_WIDTH 2
+#define KVX_SFR_ITOW_IT6_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_ITOW_IT6_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_ITOW_IT6_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ITOW_IT7_MASK _ULL(0xc000) /* Interrupt 7 owner */
+#define KVX_SFR_ITOW_IT7_SHIFT 14
+#define KVX_SFR_ITOW_IT7_WIDTH 2
+#define KVX_SFR_ITOW_IT7_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_ITOW_IT7_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_ITOW_IT7_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_ITOW_IT8_MASK _ULL(0x30000) /* Interrupt 8 owner */
+#define KVX_SFR_ITOW_IT8_SHIFT 16
+#define KVX_SFR_ITOW_IT8_WIDTH 2
+#define KVX_SFR_ITOW_IT8_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_ITOW_IT8_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_ITOW_IT8_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_ITOW_IT9_MASK _ULL(0xc0000) /* Interrupt 9 owner */
+#define KVX_SFR_ITOW_IT9_SHIFT 18
+#define KVX_SFR_ITOW_IT9_WIDTH 2
+#define KVX_SFR_ITOW_IT9_WFXL_MASK _ULL(0xc0000)
+#define KVX_SFR_ITOW_IT9_WFXL_CLEAR _ULL(0xc0000)
+#define KVX_SFR_ITOW_IT9_WFXL_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_ITOW_IT10_MASK _ULL(0x300000) /* Interrupt 10 owner */
+#define KVX_SFR_ITOW_IT10_SHIFT 20
+#define KVX_SFR_ITOW_IT10_WIDTH 2
+#define KVX_SFR_ITOW_IT10_WFXL_MASK _ULL(0x300000)
+#define KVX_SFR_ITOW_IT10_WFXL_CLEAR _ULL(0x300000)
+#define KVX_SFR_ITOW_IT10_WFXL_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_ITOW_IT11_MASK _ULL(0xc00000) /* Interrupt 11 owner */
+#define KVX_SFR_ITOW_IT11_SHIFT 22
+#define KVX_SFR_ITOW_IT11_WIDTH 2
+#define KVX_SFR_ITOW_IT11_WFXL_MASK _ULL(0xc00000)
+#define KVX_SFR_ITOW_IT11_WFXL_CLEAR _ULL(0xc00000)
+#define KVX_SFR_ITOW_IT11_WFXL_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_ITOW_IT12_MASK _ULL(0x3000000) /* Interrupt 12 owner */
+#define KVX_SFR_ITOW_IT12_SHIFT 24
+#define KVX_SFR_ITOW_IT12_WIDTH 2
+#define KVX_SFR_ITOW_IT12_WFXL_MASK _ULL(0x3000000)
+#define KVX_SFR_ITOW_IT12_WFXL_CLEAR _ULL(0x3000000)
+#define KVX_SFR_ITOW_IT12_WFXL_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_ITOW_IT13_MASK _ULL(0xc000000) /* Interrupt 13 owner */
+#define KVX_SFR_ITOW_IT13_SHIFT 26
+#define KVX_SFR_ITOW_IT13_WIDTH 2
+#define KVX_SFR_ITOW_IT13_WFXL_MASK _ULL(0xc000000)
+#define KVX_SFR_ITOW_IT13_WFXL_CLEAR _ULL(0xc000000)
+#define KVX_SFR_ITOW_IT13_WFXL_SET _ULL(0xc00000000000000)
+
+#define KVX_SFR_ITOW_IT14_MASK _ULL(0x30000000) /* Interrupt 14 owner */
+#define KVX_SFR_ITOW_IT14_SHIFT 28
+#define KVX_SFR_ITOW_IT14_WIDTH 2
+#define KVX_SFR_ITOW_IT14_WFXL_MASK _ULL(0x30000000)
+#define KVX_SFR_ITOW_IT14_WFXL_CLEAR _ULL(0x30000000)
+#define KVX_SFR_ITOW_IT14_WFXL_SET _ULL(0x3000000000000000)
+
+#define KVX_SFR_ITOW_IT15_MASK _ULL(0xc0000000) /* Interrupt 15 owner */
+#define KVX_SFR_ITOW_IT15_SHIFT 30
+#define KVX_SFR_ITOW_IT15_WIDTH 2
+#define KVX_SFR_ITOW_IT15_WFXL_MASK _ULL(0xc0000000)
+#define KVX_SFR_ITOW_IT15_WFXL_CLEAR _ULL(0xc0000000)
+#define KVX_SFR_ITOW_IT15_WFXL_SET _ULL(0xc000000000000000)
+
+#define KVX_SFR_ITOW_IT16_MASK _ULL(0x300000000) /* Interrupt 16 owner */
+#define KVX_SFR_ITOW_IT16_SHIFT 32
+#define KVX_SFR_ITOW_IT16_WIDTH 2
+#define KVX_SFR_ITOW_IT16_WFXM_MASK _ULL(0x300000000)
+#define KVX_SFR_ITOW_IT16_WFXM_CLEAR _ULL(0x3)
+#define KVX_SFR_ITOW_IT16_WFXM_SET _ULL(0x300000000)
+
+#define KVX_SFR_ITOW_IT17_MASK _ULL(0xc00000000) /* Interrupt 17 owner */
+#define KVX_SFR_ITOW_IT17_SHIFT 34
+#define KVX_SFR_ITOW_IT17_WIDTH 2
+#define KVX_SFR_ITOW_IT17_WFXM_MASK _ULL(0xc00000000)
+#define KVX_SFR_ITOW_IT17_WFXM_CLEAR _ULL(0xc)
+#define KVX_SFR_ITOW_IT17_WFXM_SET _ULL(0xc00000000)
+
+#define KVX_SFR_ITOW_IT18_MASK _ULL(0x3000000000) /* Interrupt 18 owner */
+#define KVX_SFR_ITOW_IT18_SHIFT 36
+#define KVX_SFR_ITOW_IT18_WIDTH 2
+#define KVX_SFR_ITOW_IT18_WFXM_MASK _ULL(0x3000000000)
+#define KVX_SFR_ITOW_IT18_WFXM_CLEAR _ULL(0x30)
+#define KVX_SFR_ITOW_IT18_WFXM_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ITOW_IT19_MASK _ULL(0xc000000000) /* Interrupt 19 owner */
+#define KVX_SFR_ITOW_IT19_SHIFT 38
+#define KVX_SFR_ITOW_IT19_WIDTH 2
+#define KVX_SFR_ITOW_IT19_WFXM_MASK _ULL(0xc000000000)
+#define KVX_SFR_ITOW_IT19_WFXM_CLEAR _ULL(0xc0)
+#define KVX_SFR_ITOW_IT19_WFXM_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ITOW_IT20_MASK _ULL(0x30000000000) /* Interrupt 20 owner */
+#define KVX_SFR_ITOW_IT20_SHIFT 40
+#define KVX_SFR_ITOW_IT20_WIDTH 2
+#define KVX_SFR_ITOW_IT20_WFXM_MASK _ULL(0x30000000000)
+#define KVX_SFR_ITOW_IT20_WFXM_CLEAR _ULL(0x300)
+#define KVX_SFR_ITOW_IT20_WFXM_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ITOW_IT21_MASK _ULL(0xc0000000000) /* Interrupt 21 owner */
+#define KVX_SFR_ITOW_IT21_SHIFT 42
+#define KVX_SFR_ITOW_IT21_WIDTH 2
+#define KVX_SFR_ITOW_IT21_WFXM_MASK _ULL(0xc0000000000)
+#define KVX_SFR_ITOW_IT21_WFXM_CLEAR _ULL(0xc00)
+#define KVX_SFR_ITOW_IT21_WFXM_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ITOW_IT22_MASK _ULL(0x300000000000) /* Interrupt 22 owner */
+#define KVX_SFR_ITOW_IT22_SHIFT 44
+#define KVX_SFR_ITOW_IT22_WIDTH 2
+#define KVX_SFR_ITOW_IT22_WFXM_MASK _ULL(0x300000000000)
+#define KVX_SFR_ITOW_IT22_WFXM_CLEAR _ULL(0x3000)
+#define KVX_SFR_ITOW_IT22_WFXM_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ITOW_IT23_MASK _ULL(0xc00000000000) /* Interrupt 23 owner */
+#define KVX_SFR_ITOW_IT23_SHIFT 46
+#define KVX_SFR_ITOW_IT23_WIDTH 2
+#define KVX_SFR_ITOW_IT23_WFXM_MASK _ULL(0xc00000000000)
+#define KVX_SFR_ITOW_IT23_WFXM_CLEAR _ULL(0xc000)
+#define KVX_SFR_ITOW_IT23_WFXM_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_ITOW_IT24_MASK _ULL(0x3000000000000) /* Interrupt 24 owner */
+#define KVX_SFR_ITOW_IT24_SHIFT 48
+#define KVX_SFR_ITOW_IT24_WIDTH 2
+#define KVX_SFR_ITOW_IT24_WFXM_MASK _ULL(0x3000000000000)
+#define KVX_SFR_ITOW_IT24_WFXM_CLEAR _ULL(0x30000)
+#define KVX_SFR_ITOW_IT24_WFXM_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_ITOW_IT25_MASK _ULL(0xc000000000000) /* Interrupt 25 owner */
+#define KVX_SFR_ITOW_IT25_SHIFT 50
+#define KVX_SFR_ITOW_IT25_WIDTH 2
+#define KVX_SFR_ITOW_IT25_WFXM_MASK _ULL(0xc000000000000)
+#define KVX_SFR_ITOW_IT25_WFXM_CLEAR _ULL(0xc0000)
+#define KVX_SFR_ITOW_IT25_WFXM_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_ITOW_IT26_MASK _ULL(0x30000000000000) /* Interrupt 26 owner */
+#define KVX_SFR_ITOW_IT26_SHIFT 52
+#define KVX_SFR_ITOW_IT26_WIDTH 2
+#define KVX_SFR_ITOW_IT26_WFXM_MASK _ULL(0x30000000000000)
+#define KVX_SFR_ITOW_IT26_WFXM_CLEAR _ULL(0x300000)
+#define KVX_SFR_ITOW_IT26_WFXM_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_ITOW_IT27_MASK _ULL(0xc0000000000000) /* Interrupt 27 owner */
+#define KVX_SFR_ITOW_IT27_SHIFT 54
+#define KVX_SFR_ITOW_IT27_WIDTH 2
+#define KVX_SFR_ITOW_IT27_WFXM_MASK _ULL(0xc0000000000000)
+#define KVX_SFR_ITOW_IT27_WFXM_CLEAR _ULL(0xc00000)
+#define KVX_SFR_ITOW_IT27_WFXM_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_ITOW_IT28_MASK _ULL(0x300000000000000) /* Interrupt 28 owner */
+#define KVX_SFR_ITOW_IT28_SHIFT 56
+#define KVX_SFR_ITOW_IT28_WIDTH 2
+#define KVX_SFR_ITOW_IT28_WFXM_MASK _ULL(0x300000000000000)
+#define KVX_SFR_ITOW_IT28_WFXM_CLEAR _ULL(0x3000000)
+#define KVX_SFR_ITOW_IT28_WFXM_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_ITOW_IT29_MASK _ULL(0xc00000000000000) /* Interrupt 29 owner */
+#define KVX_SFR_ITOW_IT29_SHIFT 58
+#define KVX_SFR_ITOW_IT29_WIDTH 2
+#define KVX_SFR_ITOW_IT29_WFXM_MASK _ULL(0xc00000000000000)
+#define KVX_SFR_ITOW_IT29_WFXM_CLEAR _ULL(0xc000000)
+#define KVX_SFR_ITOW_IT29_WFXM_SET _ULL(0xc00000000000000)
+
+#define KVX_SFR_ITOW_IT30_MASK _ULL(0x3000000000000000) /* Interrupt 30 owner */
+#define KVX_SFR_ITOW_IT30_SHIFT 60
+#define KVX_SFR_ITOW_IT30_WIDTH 2
+#define KVX_SFR_ITOW_IT30_WFXM_MASK _ULL(0x3000000000000000)
+#define KVX_SFR_ITOW_IT30_WFXM_CLEAR _ULL(0x30000000)
+#define KVX_SFR_ITOW_IT30_WFXM_SET _ULL(0x3000000000000000)
+
+#define KVX_SFR_ITOW_IT31_MASK _ULL(0xc000000000000000) /* Interrupt 31 owner */
+#define KVX_SFR_ITOW_IT31_SHIFT 62
+#define KVX_SFR_ITOW_IT31_WIDTH 2
+#define KVX_SFR_ITOW_IT31_WFXM_MASK _ULL(0xc000000000000000)
+#define KVX_SFR_ITOW_IT31_WFXM_CLEAR _ULL(0xc0000000)
+#define KVX_SFR_ITOW_IT31_WFXM_SET _ULL(0xc000000000000000)
+
+#define KVX_SFR_DO_B0_MASK _ULL(0x3) /* Breakpoint 0 owner. */
+#define KVX_SFR_DO_B0_SHIFT 0
+#define KVX_SFR_DO_B0_WIDTH 2
+#define KVX_SFR_DO_B0_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_DO_B0_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_DO_B0_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_DO_B1_MASK _ULL(0xc) /* Breakpoint 1 owner. */
+#define KVX_SFR_DO_B1_SHIFT 2
+#define KVX_SFR_DO_B1_WIDTH 2
+#define KVX_SFR_DO_B1_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_DO_B1_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_DO_B1_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_DO_W0_MASK _ULL(0x30) /* Watchpoint 0 owner. */
+#define KVX_SFR_DO_W0_SHIFT 4
+#define KVX_SFR_DO_W0_WIDTH 2
+#define KVX_SFR_DO_W0_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_DO_W0_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_DO_W0_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_DO_W1_MASK _ULL(0xc0) /* Watchpoint 1 owner. */
+#define KVX_SFR_DO_W1_SHIFT 6
+#define KVX_SFR_DO_W1_WIDTH 2
+#define KVX_SFR_DO_W1_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_DO_W1_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_DO_W1_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_DBA0_DBA0_MASK _ULL(0xffffffffffffffff) /* Debug Breakpoint Address 0 */
+#define KVX_SFR_DBA0_DBA0_SHIFT 0
+#define KVX_SFR_DBA0_DBA0_WIDTH 64
+#define KVX_SFR_DBA0_DBA0_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_DBA0_DBA0_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_DBA0_DBA0_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_DBA0_DBA0_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_DBA0_DBA0_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_DBA0_DBA0_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_DBA1_DBA1_MASK _ULL(0xffffffffffffffff) /* Debug Breakpoint Address 1 */
+#define KVX_SFR_DBA1_DBA1_SHIFT 0
+#define KVX_SFR_DBA1_DBA1_WIDTH 64
+#define KVX_SFR_DBA1_DBA1_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_DBA1_DBA1_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_DBA1_DBA1_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_DBA1_DBA1_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_DBA1_DBA1_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_DBA1_DBA1_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_DWA0_DWA0_MASK _ULL(0xffffffffffffffff) /* Debug Breakpoint Address 0 */
+#define KVX_SFR_DWA0_DWA0_SHIFT 0
+#define KVX_SFR_DWA0_DWA0_WIDTH 64
+#define KVX_SFR_DWA0_DWA0_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_DWA0_DWA0_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_DWA0_DWA0_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_DWA0_DWA0_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_DWA0_DWA0_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_DWA0_DWA0_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_DWA1_DWA1_MASK _ULL(0xffffffffffffffff) /* Debug Breakpoint Address 1 */
+#define KVX_SFR_DWA1_DWA1_SHIFT 0
+#define KVX_SFR_DWA1_DWA1_WIDTH 64
+#define KVX_SFR_DWA1_DWA1_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_DWA1_DWA1_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_DWA1_DWA1_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_DWA1_DWA1_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_DWA1_DWA1_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_DWA1_DWA1_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_DOW_B0_MASK _ULL(0x3) /* Breakpoint 0 owner. */
+#define KVX_SFR_DOW_B0_SHIFT 0
+#define KVX_SFR_DOW_B0_WIDTH 2
+#define KVX_SFR_DOW_B0_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_DOW_B0_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_DOW_B0_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_DOW_B1_MASK _ULL(0xc) /* Breakpoint 1 owner. */
+#define KVX_SFR_DOW_B1_SHIFT 2
+#define KVX_SFR_DOW_B1_WIDTH 2
+#define KVX_SFR_DOW_B1_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_DOW_B1_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_DOW_B1_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_DOW_W0_MASK _ULL(0x30) /* Watchpoint 0 owner. */
+#define KVX_SFR_DOW_W0_SHIFT 4
+#define KVX_SFR_DOW_W0_WIDTH 2
+#define KVX_SFR_DOW_W0_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_DOW_W0_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_DOW_W0_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_DOW_W1_MASK _ULL(0xc0) /* Watchpoint 1 owner. */
+#define KVX_SFR_DOW_W1_SHIFT 6
+#define KVX_SFR_DOW_W1_WIDTH 2
+#define KVX_SFR_DOW_W1_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_DOW_W1_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_DOW_W1_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_MO_MMI_MASK _ULL(0x3) /* Memory Management Instructions owner. */
+#define KVX_SFR_MO_MMI_SHIFT 0
+#define KVX_SFR_MO_MMI_WIDTH 2
+#define KVX_SFR_MO_MMI_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_MO_MMI_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_MO_MMI_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_MO_RFE_MASK _ULL(0xc) /* RFE instruction owner. */
+#define KVX_SFR_MO_RFE_SHIFT 2
+#define KVX_SFR_MO_RFE_WIDTH 2
+#define KVX_SFR_MO_RFE_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_MO_RFE_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_MO_RFE_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_MO_STOP_MASK _ULL(0x30) /* STOP instruction owner. */
+#define KVX_SFR_MO_STOP_SHIFT 4
+#define KVX_SFR_MO_STOP_WIDTH 2
+#define KVX_SFR_MO_STOP_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_MO_STOP_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_MO_STOP_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_MO_SYNC_MASK _ULL(0xc0) /* SYNCGROUP instruction owner. */
+#define KVX_SFR_MO_SYNC_SHIFT 6
+#define KVX_SFR_MO_SYNC_WIDTH 2
+#define KVX_SFR_MO_SYNC_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_MO_SYNC_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_MO_SYNC_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_MO_PCR_MASK _ULL(0x300) /* PCR register owner. */
+#define KVX_SFR_MO_PCR_SHIFT 8
+#define KVX_SFR_MO_PCR_WIDTH 2
+#define KVX_SFR_MO_PCR_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_MO_PCR_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_MO_PCR_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_MO_MSG_MASK _ULL(0xc00) /* MMU SFR GROUP registers owner. */
+#define KVX_SFR_MO_MSG_SHIFT 10
+#define KVX_SFR_MO_MSG_WIDTH 2
+#define KVX_SFR_MO_MSG_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_MO_MSG_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_MO_MSG_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_MO_MEN_MASK _ULL(0x3000) /* Miscellaneous External Notifications register owner. */
+#define KVX_SFR_MO_MEN_SHIFT 12
+#define KVX_SFR_MO_MEN_WIDTH 2
+#define KVX_SFR_MO_MEN_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_MO_MEN_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_MO_MEN_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_MO_MES_MASK _ULL(0xc000) /* Memory Error Status register owner. */
+#define KVX_SFR_MO_MES_SHIFT 14
+#define KVX_SFR_MO_MES_WIDTH 2
+#define KVX_SFR_MO_MES_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_MO_MES_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_MO_MES_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_MO_CSIT_MASK _ULL(0x30000) /* Compute Status Artithmetic Interrupt register owner. */
+#define KVX_SFR_MO_CSIT_SHIFT 16
+#define KVX_SFR_MO_CSIT_WIDTH 2
+#define KVX_SFR_MO_CSIT_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_MO_CSIT_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_MO_CSIT_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_MO_T0_MASK _ULL(0xc0000) /* Timer 0 register group owner */
+#define KVX_SFR_MO_T0_SHIFT 18
+#define KVX_SFR_MO_T0_WIDTH 2
+#define KVX_SFR_MO_T0_WFXL_MASK _ULL(0xc0000)
+#define KVX_SFR_MO_T0_WFXL_CLEAR _ULL(0xc0000)
+#define KVX_SFR_MO_T0_WFXL_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_MO_T1_MASK _ULL(0x300000) /* Timer 1 register group owner */
+#define KVX_SFR_MO_T1_SHIFT 20
+#define KVX_SFR_MO_T1_WIDTH 2
+#define KVX_SFR_MO_T1_WFXL_MASK _ULL(0x300000)
+#define KVX_SFR_MO_T1_WFXL_CLEAR _ULL(0x300000)
+#define KVX_SFR_MO_T1_WFXL_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_MO_WD_MASK _ULL(0xc00000) /* Watch Dog register group owner. */
+#define KVX_SFR_MO_WD_SHIFT 22
+#define KVX_SFR_MO_WD_WIDTH 2
+#define KVX_SFR_MO_WD_WFXL_MASK _ULL(0xc00000)
+#define KVX_SFR_MO_WD_WFXL_CLEAR _ULL(0xc00000)
+#define KVX_SFR_MO_WD_WFXL_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_MO_PM0_MASK _ULL(0x3000000) /* Performance Monitor 0 register owner. */
+#define KVX_SFR_MO_PM0_SHIFT 24
+#define KVX_SFR_MO_PM0_WIDTH 2
+#define KVX_SFR_MO_PM0_WFXL_MASK _ULL(0x3000000)
+#define KVX_SFR_MO_PM0_WFXL_CLEAR _ULL(0x3000000)
+#define KVX_SFR_MO_PM0_WFXL_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_MO_PM1_MASK _ULL(0xc000000) /* Performance Monitor 1 register owner. */
+#define KVX_SFR_MO_PM1_SHIFT 26
+#define KVX_SFR_MO_PM1_WIDTH 2
+#define KVX_SFR_MO_PM1_WFXL_MASK _ULL(0xc000000)
+#define KVX_SFR_MO_PM1_WFXL_CLEAR _ULL(0xc000000)
+#define KVX_SFR_MO_PM1_WFXL_SET _ULL(0xc00000000000000)
+
+#define KVX_SFR_MO_PM2_MASK _ULL(0x30000000) /* Performance Monitor 2 register owner. */
+#define KVX_SFR_MO_PM2_SHIFT 28
+#define KVX_SFR_MO_PM2_WIDTH 2
+#define KVX_SFR_MO_PM2_WFXL_MASK _ULL(0x30000000)
+#define KVX_SFR_MO_PM2_WFXL_CLEAR _ULL(0x30000000)
+#define KVX_SFR_MO_PM2_WFXL_SET _ULL(0x3000000000000000)
+
+#define KVX_SFR_MO_PM3_MASK _ULL(0xc0000000) /* Performance Monitor 3 register owner. */
+#define KVX_SFR_MO_PM3_SHIFT 30
+#define KVX_SFR_MO_PM3_WIDTH 2
+#define KVX_SFR_MO_PM3_WFXL_MASK _ULL(0xc0000000)
+#define KVX_SFR_MO_PM3_WFXL_CLEAR _ULL(0xc0000000)
+#define KVX_SFR_MO_PM3_WFXL_SET _ULL(0xc000000000000000)
+
+#define KVX_SFR_MO_PMIT_MASK _ULL(0x300000000) /* Performance Monitor Interrupt register group owner. */
+#define KVX_SFR_MO_PMIT_SHIFT 32
+#define KVX_SFR_MO_PMIT_WIDTH 2
+#define KVX_SFR_MO_PMIT_WFXM_MASK _ULL(0x300000000)
+#define KVX_SFR_MO_PMIT_WFXM_CLEAR _ULL(0x3)
+#define KVX_SFR_MO_PMIT_WFXM_SET _ULL(0x300000000)
+
+#define KVX_SFR_MOW_MMI_MASK _ULL(0x3) /* Memory Management Instructions owner. */
+#define KVX_SFR_MOW_MMI_SHIFT 0
+#define KVX_SFR_MOW_MMI_WIDTH 2
+#define KVX_SFR_MOW_MMI_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_MOW_MMI_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_MOW_MMI_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_MOW_RFE_MASK _ULL(0xc) /* RFE instruction owner. */
+#define KVX_SFR_MOW_RFE_SHIFT 2
+#define KVX_SFR_MOW_RFE_WIDTH 2
+#define KVX_SFR_MOW_RFE_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_MOW_RFE_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_MOW_RFE_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_MOW_STOP_MASK _ULL(0x30) /* STOP instruction owner. */
+#define KVX_SFR_MOW_STOP_SHIFT 4
+#define KVX_SFR_MOW_STOP_WIDTH 2
+#define KVX_SFR_MOW_STOP_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_MOW_STOP_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_MOW_STOP_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_MOW_SYNC_MASK _ULL(0xc0) /* SYNCGROUP instruction owner. */
+#define KVX_SFR_MOW_SYNC_SHIFT 6
+#define KVX_SFR_MOW_SYNC_WIDTH 2
+#define KVX_SFR_MOW_SYNC_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_MOW_SYNC_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_MOW_SYNC_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_MOW_PCR_MASK _ULL(0x300) /* PCR register owner. */
+#define KVX_SFR_MOW_PCR_SHIFT 8
+#define KVX_SFR_MOW_PCR_WIDTH 2
+#define KVX_SFR_MOW_PCR_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_MOW_PCR_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_MOW_PCR_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_MOW_MSG_MASK _ULL(0xc00) /* MMU SFR GROUP registers owner. */
+#define KVX_SFR_MOW_MSG_SHIFT 10
+#define KVX_SFR_MOW_MSG_WIDTH 2
+#define KVX_SFR_MOW_MSG_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_MOW_MSG_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_MOW_MSG_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_MOW_MEN_MASK _ULL(0x3000) /* Miscellaneous External Notifications register owner. */
+#define KVX_SFR_MOW_MEN_SHIFT 12
+#define KVX_SFR_MOW_MEN_WIDTH 2
+#define KVX_SFR_MOW_MEN_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_MOW_MEN_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_MOW_MEN_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_MOW_MES_MASK _ULL(0xc000) /* Memory Error Status register owner. */
+#define KVX_SFR_MOW_MES_SHIFT 14
+#define KVX_SFR_MOW_MES_WIDTH 2
+#define KVX_SFR_MOW_MES_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_MOW_MES_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_MOW_MES_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_MOW_CSIT_MASK _ULL(0x30000) /* Compute Status Artithmetic Interrupt register owner. */
+#define KVX_SFR_MOW_CSIT_SHIFT 16
+#define KVX_SFR_MOW_CSIT_WIDTH 2
+#define KVX_SFR_MOW_CSIT_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_MOW_CSIT_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_MOW_CSIT_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_MOW_T0_MASK _ULL(0xc0000) /* Timer 0 register group owner */
+#define KVX_SFR_MOW_T0_SHIFT 18
+#define KVX_SFR_MOW_T0_WIDTH 2
+#define KVX_SFR_MOW_T0_WFXL_MASK _ULL(0xc0000)
+#define KVX_SFR_MOW_T0_WFXL_CLEAR _ULL(0xc0000)
+#define KVX_SFR_MOW_T0_WFXL_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_MOW_T1_MASK _ULL(0x300000) /* Timer 1 register group owner */
+#define KVX_SFR_MOW_T1_SHIFT 20
+#define KVX_SFR_MOW_T1_WIDTH 2
+#define KVX_SFR_MOW_T1_WFXL_MASK _ULL(0x300000)
+#define KVX_SFR_MOW_T1_WFXL_CLEAR _ULL(0x300000)
+#define KVX_SFR_MOW_T1_WFXL_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_MOW_WD_MASK _ULL(0xc00000) /* Watch Dog register group owner. */
+#define KVX_SFR_MOW_WD_SHIFT 22
+#define KVX_SFR_MOW_WD_WIDTH 2
+#define KVX_SFR_MOW_WD_WFXL_MASK _ULL(0xc00000)
+#define KVX_SFR_MOW_WD_WFXL_CLEAR _ULL(0xc00000)
+#define KVX_SFR_MOW_WD_WFXL_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_MOW_PM0_MASK _ULL(0x3000000) /* Performance Monitor 0 register owner. */
+#define KVX_SFR_MOW_PM0_SHIFT 24
+#define KVX_SFR_MOW_PM0_WIDTH 2
+#define KVX_SFR_MOW_PM0_WFXL_MASK _ULL(0x3000000)
+#define KVX_SFR_MOW_PM0_WFXL_CLEAR _ULL(0x3000000)
+#define KVX_SFR_MOW_PM0_WFXL_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_MOW_PM1_MASK _ULL(0xc000000) /* Performance Monitor 1 register owner. */
+#define KVX_SFR_MOW_PM1_SHIFT 26
+#define KVX_SFR_MOW_PM1_WIDTH 2
+#define KVX_SFR_MOW_PM1_WFXL_MASK _ULL(0xc000000)
+#define KVX_SFR_MOW_PM1_WFXL_CLEAR _ULL(0xc000000)
+#define KVX_SFR_MOW_PM1_WFXL_SET _ULL(0xc00000000000000)
+
+#define KVX_SFR_MOW_PM2_MASK _ULL(0x30000000) /* Performance Monitor 2 register owner. */
+#define KVX_SFR_MOW_PM2_SHIFT 28
+#define KVX_SFR_MOW_PM2_WIDTH 2
+#define KVX_SFR_MOW_PM2_WFXL_MASK _ULL(0x30000000)
+#define KVX_SFR_MOW_PM2_WFXL_CLEAR _ULL(0x30000000)
+#define KVX_SFR_MOW_PM2_WFXL_SET _ULL(0x3000000000000000)
+
+#define KVX_SFR_MOW_PM3_MASK _ULL(0xc0000000) /* Performance Monitor 3 register owner. */
+#define KVX_SFR_MOW_PM3_SHIFT 30
+#define KVX_SFR_MOW_PM3_WIDTH 2
+#define KVX_SFR_MOW_PM3_WFXL_MASK _ULL(0xc0000000)
+#define KVX_SFR_MOW_PM3_WFXL_CLEAR _ULL(0xc0000000)
+#define KVX_SFR_MOW_PM3_WFXL_SET _ULL(0xc000000000000000)
+
+#define KVX_SFR_MOW_PMIT_MASK _ULL(0x300000000) /* Performance Monitor Interrupt register group owner. */
+#define KVX_SFR_MOW_PMIT_SHIFT 32
+#define KVX_SFR_MOW_PMIT_WIDTH 2
+#define KVX_SFR_MOW_PMIT_WFXM_MASK _ULL(0x300000000)
+#define KVX_SFR_MOW_PMIT_WFXM_CLEAR _ULL(0x3)
+#define KVX_SFR_MOW_PMIT_WFXM_SET _ULL(0x300000000)
+
+#define KVX_SFR_PS_PL_MASK _ULL(0x3) /* Current Privilege Level */
+#define KVX_SFR_PS_PL_SHIFT 0
+#define KVX_SFR_PS_PL_WIDTH 2
+#define KVX_SFR_PS_PL_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_PS_PL_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_PS_PL_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_PS_ET_MASK _ULL(0x4) /* Exception Taken */
+#define KVX_SFR_PS_ET_SHIFT 2
+#define KVX_SFR_PS_ET_WIDTH 1
+#define KVX_SFR_PS_ET_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_PS_ET_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_PS_ET_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_PS_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */
+#define KVX_SFR_PS_HTD_SHIFT 3
+#define KVX_SFR_PS_HTD_WIDTH 1
+#define KVX_SFR_PS_HTD_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_PS_HTD_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_PS_HTD_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_PS_IE_MASK _ULL(0x10) /* Interrupt Enable */
+#define KVX_SFR_PS_IE_SHIFT 4
+#define KVX_SFR_PS_IE_WIDTH 1
+#define KVX_SFR_PS_IE_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_PS_IE_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_PS_IE_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_PS_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */
+#define KVX_SFR_PS_HLE_SHIFT 5
+#define KVX_SFR_PS_HLE_WIDTH 1
+#define KVX_SFR_PS_HLE_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_PS_HLE_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_PS_HLE_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_PS_SRE_MASK _ULL(0x40) /* Software REserved */
+#define KVX_SFR_PS_SRE_SHIFT 6
+#define KVX_SFR_PS_SRE_WIDTH 1
+#define KVX_SFR_PS_SRE_WFXL_MASK _ULL(0x40)
+#define KVX_SFR_PS_SRE_WFXL_CLEAR _ULL(0x40)
+#define KVX_SFR_PS_SRE_WFXL_SET _ULL(0x4000000000)
+
+#define KVX_SFR_PS_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */
+#define KVX_SFR_PS_DAUS_SHIFT 7
+#define KVX_SFR_PS_DAUS_WIDTH 1
+#define KVX_SFR_PS_DAUS_WFXL_MASK _ULL(0x80)
+#define KVX_SFR_PS_DAUS_WFXL_CLEAR _ULL(0x80)
+#define KVX_SFR_PS_DAUS_WFXL_SET _ULL(0x8000000000)
+
+#define KVX_SFR_PS_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */
+#define KVX_SFR_PS_ICE_SHIFT 8
+#define KVX_SFR_PS_ICE_WIDTH 1
+#define KVX_SFR_PS_ICE_WFXL_MASK _ULL(0x100)
+#define KVX_SFR_PS_ICE_WFXL_CLEAR _ULL(0x100)
+#define KVX_SFR_PS_ICE_WFXL_SET _ULL(0x10000000000)
+
+#define KVX_SFR_PS_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */
+#define KVX_SFR_PS_USE_SHIFT 9
+#define KVX_SFR_PS_USE_WIDTH 1
+#define KVX_SFR_PS_USE_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_PS_USE_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_PS_USE_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_PS_DCE_MASK _ULL(0x400) /* Data Cache Enable */
+#define KVX_SFR_PS_DCE_SHIFT 10
+#define KVX_SFR_PS_DCE_WIDTH 1
+#define KVX_SFR_PS_DCE_WFXL_MASK _ULL(0x400)
+#define KVX_SFR_PS_DCE_WFXL_CLEAR _ULL(0x400)
+#define KVX_SFR_PS_DCE_WFXL_SET _ULL(0x40000000000)
+
+#define KVX_SFR_PS_MME_MASK _ULL(0x800) /* Memory Management Enable */
+#define KVX_SFR_PS_MME_SHIFT 11
+#define KVX_SFR_PS_MME_WIDTH 1
+#define KVX_SFR_PS_MME_WFXL_MASK _ULL(0x800)
+#define KVX_SFR_PS_MME_WFXL_CLEAR _ULL(0x800)
+#define KVX_SFR_PS_MME_WFXL_SET _ULL(0x80000000000)
+
+#define KVX_SFR_PS_IL_MASK _ULL(0x3000) /* Interrupt Level */
+#define KVX_SFR_PS_IL_SHIFT 12
+#define KVX_SFR_PS_IL_WIDTH 2
+#define KVX_SFR_PS_IL_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_PS_IL_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_PS_IL_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_PS_VS_MASK _ULL(0xc000) /* Virtual Space */
+#define KVX_SFR_PS_VS_SHIFT 14
+#define KVX_SFR_PS_VS_WIDTH 2
+#define KVX_SFR_PS_VS_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_PS_VS_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_PS_VS_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_PS_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */
+#define KVX_SFR_PS_V64_SHIFT 16
+#define KVX_SFR_PS_V64_WIDTH 1
+#define KVX_SFR_PS_V64_WFXL_MASK _ULL(0x10000)
+#define KVX_SFR_PS_V64_WFXL_CLEAR _ULL(0x10000)
+#define KVX_SFR_PS_V64_WFXL_SET _ULL(0x1000000000000)
+
+#define KVX_SFR_PS_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */
+#define KVX_SFR_PS_L2E_SHIFT 17
+#define KVX_SFR_PS_L2E_WIDTH 1
+#define KVX_SFR_PS_L2E_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_PS_L2E_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_PS_L2E_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_PS_SME_MASK _ULL(0x40000) /* Step Mode Enabled */
+#define KVX_SFR_PS_SME_SHIFT 18
+#define KVX_SFR_PS_SME_WIDTH 1
+#define KVX_SFR_PS_SME_WFXL_MASK _ULL(0x40000)
+#define KVX_SFR_PS_SME_WFXL_CLEAR _ULL(0x40000)
+#define KVX_SFR_PS_SME_WFXL_SET _ULL(0x4000000000000)
+
+#define KVX_SFR_PS_SMR_MASK _ULL(0x80000) /* Step Mode Ready */
+#define KVX_SFR_PS_SMR_SHIFT 19
+#define KVX_SFR_PS_SMR_WIDTH 1
+#define KVX_SFR_PS_SMR_WFXL_MASK _ULL(0x80000)
+#define KVX_SFR_PS_SMR_WFXL_CLEAR _ULL(0x80000)
+#define KVX_SFR_PS_SMR_WFXL_SET _ULL(0x8000000000000)
+
+#define KVX_SFR_PS_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */
+#define KVX_SFR_PS_PMJ_SHIFT 20
+#define KVX_SFR_PS_PMJ_WIDTH 4
+#define KVX_SFR_PS_PMJ_WFXL_MASK _ULL(0xf00000)
+#define KVX_SFR_PS_PMJ_WFXL_CLEAR _ULL(0xf00000)
+#define KVX_SFR_PS_PMJ_WFXL_SET _ULL(0xf0000000000000)
+
+#define KVX_SFR_PS_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */
+#define KVX_SFR_PS_MMUP_SHIFT 24
+#define KVX_SFR_PS_MMUP_WIDTH 1
+#define KVX_SFR_PS_MMUP_WFXL_MASK _ULL(0x1000000)
+#define KVX_SFR_PS_MMUP_WFXL_CLEAR _ULL(0x1000000)
+#define KVX_SFR_PS_MMUP_WFXL_SET _ULL(0x100000000000000)
+
+#define KVX_SFR_SPS_PL_MASK _ULL(0x3) /* Current Privilege Level */
+#define KVX_SFR_SPS_PL_SHIFT 0
+#define KVX_SFR_SPS_PL_WIDTH 2
+#define KVX_SFR_SPS_PL_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_SPS_PL_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_SPS_PL_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_SPS_ET_MASK _ULL(0x4) /* Exception Taken */
+#define KVX_SFR_SPS_ET_SHIFT 2
+#define KVX_SFR_SPS_ET_WIDTH 1
+#define KVX_SFR_SPS_ET_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_SPS_ET_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_SPS_ET_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_SPS_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */
+#define KVX_SFR_SPS_HTD_SHIFT 3
+#define KVX_SFR_SPS_HTD_WIDTH 1
+#define KVX_SFR_SPS_HTD_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_SPS_HTD_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_SPS_HTD_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_SPS_IE_MASK _ULL(0x10) /* Interrupt Enable */
+#define KVX_SFR_SPS_IE_SHIFT 4
+#define KVX_SFR_SPS_IE_WIDTH 1
+#define KVX_SFR_SPS_IE_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_SPS_IE_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_SPS_IE_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_SPS_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */
+#define KVX_SFR_SPS_HLE_SHIFT 5
+#define KVX_SFR_SPS_HLE_WIDTH 1
+#define KVX_SFR_SPS_HLE_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_SPS_HLE_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_SPS_HLE_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_SPS_SRE_MASK _ULL(0x40) /* Software REserved */
+#define KVX_SFR_SPS_SRE_SHIFT 6
+#define KVX_SFR_SPS_SRE_WIDTH 1
+#define KVX_SFR_SPS_SRE_WFXL_MASK _ULL(0x40)
+#define KVX_SFR_SPS_SRE_WFXL_CLEAR _ULL(0x40)
+#define KVX_SFR_SPS_SRE_WFXL_SET _ULL(0x4000000000)
+
+#define KVX_SFR_SPS_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */
+#define KVX_SFR_SPS_DAUS_SHIFT 7
+#define KVX_SFR_SPS_DAUS_WIDTH 1
+#define KVX_SFR_SPS_DAUS_WFXL_MASK _ULL(0x80)
+#define KVX_SFR_SPS_DAUS_WFXL_CLEAR _ULL(0x80)
+#define KVX_SFR_SPS_DAUS_WFXL_SET _ULL(0x8000000000)
+
+#define KVX_SFR_SPS_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */
+#define KVX_SFR_SPS_ICE_SHIFT 8
+#define KVX_SFR_SPS_ICE_WIDTH 1
+#define KVX_SFR_SPS_ICE_WFXL_MASK _ULL(0x100)
+#define KVX_SFR_SPS_ICE_WFXL_CLEAR _ULL(0x100)
+#define KVX_SFR_SPS_ICE_WFXL_SET _ULL(0x10000000000)
+
+#define KVX_SFR_SPS_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */
+#define KVX_SFR_SPS_USE_SHIFT 9
+#define KVX_SFR_SPS_USE_WIDTH 1
+#define KVX_SFR_SPS_USE_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_SPS_USE_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_SPS_USE_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_SPS_DCE_MASK _ULL(0x400) /* Data Cache Enable */
+#define KVX_SFR_SPS_DCE_SHIFT 10
+#define KVX_SFR_SPS_DCE_WIDTH 1
+#define KVX_SFR_SPS_DCE_WFXL_MASK _ULL(0x400)
+#define KVX_SFR_SPS_DCE_WFXL_CLEAR _ULL(0x400)
+#define KVX_SFR_SPS_DCE_WFXL_SET _ULL(0x40000000000)
+
+#define KVX_SFR_SPS_MME_MASK _ULL(0x800) /* Memory Management Enable */
+#define KVX_SFR_SPS_MME_SHIFT 11
+#define KVX_SFR_SPS_MME_WIDTH 1
+#define KVX_SFR_SPS_MME_WFXL_MASK _ULL(0x800)
+#define KVX_SFR_SPS_MME_WFXL_CLEAR _ULL(0x800)
+#define KVX_SFR_SPS_MME_WFXL_SET _ULL(0x80000000000)
+
+#define KVX_SFR_SPS_IL_MASK _ULL(0x3000) /* Interrupt Level */
+#define KVX_SFR_SPS_IL_SHIFT 12
+#define KVX_SFR_SPS_IL_WIDTH 2
+#define KVX_SFR_SPS_IL_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_SPS_IL_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_SPS_IL_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_SPS_VS_MASK _ULL(0xc000) /* Virtual Space */
+#define KVX_SFR_SPS_VS_SHIFT 14
+#define KVX_SFR_SPS_VS_WIDTH 2
+#define KVX_SFR_SPS_VS_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_SPS_VS_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_SPS_VS_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_SPS_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */
+#define KVX_SFR_SPS_V64_SHIFT 16
+#define KVX_SFR_SPS_V64_WIDTH 1
+#define KVX_SFR_SPS_V64_WFXL_MASK _ULL(0x10000)
+#define KVX_SFR_SPS_V64_WFXL_CLEAR _ULL(0x10000)
+#define KVX_SFR_SPS_V64_WFXL_SET _ULL(0x1000000000000)
+
+#define KVX_SFR_SPS_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */
+#define KVX_SFR_SPS_L2E_SHIFT 17
+#define KVX_SFR_SPS_L2E_WIDTH 1
+#define KVX_SFR_SPS_L2E_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_SPS_L2E_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_SPS_L2E_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_SPS_SME_MASK _ULL(0x40000) /* Step Mode Enabled */
+#define KVX_SFR_SPS_SME_SHIFT 18
+#define KVX_SFR_SPS_SME_WIDTH 1
+#define KVX_SFR_SPS_SME_WFXL_MASK _ULL(0x40000)
+#define KVX_SFR_SPS_SME_WFXL_CLEAR _ULL(0x40000)
+#define KVX_SFR_SPS_SME_WFXL_SET _ULL(0x4000000000000)
+
+#define KVX_SFR_SPS_SMR_MASK _ULL(0x80000) /* Step Mode Ready */
+#define KVX_SFR_SPS_SMR_SHIFT 19
+#define KVX_SFR_SPS_SMR_WIDTH 1
+#define KVX_SFR_SPS_SMR_WFXL_MASK _ULL(0x80000)
+#define KVX_SFR_SPS_SMR_WFXL_CLEAR _ULL(0x80000)
+#define KVX_SFR_SPS_SMR_WFXL_SET _ULL(0x8000000000000)
+
+#define KVX_SFR_SPS_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */
+#define KVX_SFR_SPS_PMJ_SHIFT 20
+#define KVX_SFR_SPS_PMJ_WIDTH 4
+#define KVX_SFR_SPS_PMJ_WFXL_MASK _ULL(0xf00000)
+#define KVX_SFR_SPS_PMJ_WFXL_CLEAR _ULL(0xf00000)
+#define KVX_SFR_SPS_PMJ_WFXL_SET _ULL(0xf0000000000000)
+
+#define KVX_SFR_SPS_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */
+#define KVX_SFR_SPS_MMUP_SHIFT 24
+#define KVX_SFR_SPS_MMUP_WIDTH 1
+#define KVX_SFR_SPS_MMUP_WFXL_MASK _ULL(0x1000000)
+#define KVX_SFR_SPS_MMUP_WFXL_CLEAR _ULL(0x1000000)
+#define KVX_SFR_SPS_MMUP_WFXL_SET _ULL(0x100000000000000)
+
+#define KVX_SFR_SPS_PL0_PL_MASK _ULL(0x3) /* Current Privilege Level */
+#define KVX_SFR_SPS_PL0_PL_SHIFT 0
+#define KVX_SFR_SPS_PL0_PL_WIDTH 2
+#define KVX_SFR_SPS_PL0_PL_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_SPS_PL0_PL_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_SPS_PL0_PL_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_SPS_PL0_ET_MASK _ULL(0x4) /* Exception Taken */
+#define KVX_SFR_SPS_PL0_ET_SHIFT 2
+#define KVX_SFR_SPS_PL0_ET_WIDTH 1
+#define KVX_SFR_SPS_PL0_ET_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_SPS_PL0_ET_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_SPS_PL0_ET_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_SPS_PL0_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */
+#define KVX_SFR_SPS_PL0_HTD_SHIFT 3
+#define KVX_SFR_SPS_PL0_HTD_WIDTH 1
+#define KVX_SFR_SPS_PL0_HTD_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_SPS_PL0_HTD_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_SPS_PL0_HTD_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_SPS_PL0_IE_MASK _ULL(0x10) /* Interrupt Enable */
+#define KVX_SFR_SPS_PL0_IE_SHIFT 4
+#define KVX_SFR_SPS_PL0_IE_WIDTH 1
+#define KVX_SFR_SPS_PL0_IE_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_SPS_PL0_IE_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_SPS_PL0_IE_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_SPS_PL0_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */
+#define KVX_SFR_SPS_PL0_HLE_SHIFT 5
+#define KVX_SFR_SPS_PL0_HLE_WIDTH 1
+#define KVX_SFR_SPS_PL0_HLE_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_SPS_PL0_HLE_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_SPS_PL0_HLE_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_SPS_PL0_SRE_MASK _ULL(0x40) /* Software REserved */
+#define KVX_SFR_SPS_PL0_SRE_SHIFT 6
+#define KVX_SFR_SPS_PL0_SRE_WIDTH 1
+#define KVX_SFR_SPS_PL0_SRE_WFXL_MASK _ULL(0x40)
+#define KVX_SFR_SPS_PL0_SRE_WFXL_CLEAR _ULL(0x40)
+#define KVX_SFR_SPS_PL0_SRE_WFXL_SET _ULL(0x4000000000)
+
+#define KVX_SFR_SPS_PL0_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */
+#define KVX_SFR_SPS_PL0_DAUS_SHIFT 7
+#define KVX_SFR_SPS_PL0_DAUS_WIDTH 1
+#define KVX_SFR_SPS_PL0_DAUS_WFXL_MASK _ULL(0x80)
+#define KVX_SFR_SPS_PL0_DAUS_WFXL_CLEAR _ULL(0x80)
+#define KVX_SFR_SPS_PL0_DAUS_WFXL_SET _ULL(0x8000000000)
+
+#define KVX_SFR_SPS_PL0_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */
+#define KVX_SFR_SPS_PL0_ICE_SHIFT 8
+#define KVX_SFR_SPS_PL0_ICE_WIDTH 1
+#define KVX_SFR_SPS_PL0_ICE_WFXL_MASK _ULL(0x100)
+#define KVX_SFR_SPS_PL0_ICE_WFXL_CLEAR _ULL(0x100)
+#define KVX_SFR_SPS_PL0_ICE_WFXL_SET _ULL(0x10000000000)
+
+#define KVX_SFR_SPS_PL0_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */
+#define KVX_SFR_SPS_PL0_USE_SHIFT 9
+#define KVX_SFR_SPS_PL0_USE_WIDTH 1
+#define KVX_SFR_SPS_PL0_USE_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_SPS_PL0_USE_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_SPS_PL0_USE_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_SPS_PL0_DCE_MASK _ULL(0x400) /* Data Cache Enable */
+#define KVX_SFR_SPS_PL0_DCE_SHIFT 10
+#define KVX_SFR_SPS_PL0_DCE_WIDTH 1
+#define KVX_SFR_SPS_PL0_DCE_WFXL_MASK _ULL(0x400)
+#define KVX_SFR_SPS_PL0_DCE_WFXL_CLEAR _ULL(0x400)
+#define KVX_SFR_SPS_PL0_DCE_WFXL_SET _ULL(0x40000000000)
+
+#define KVX_SFR_SPS_PL0_MME_MASK _ULL(0x800) /* Memory Management Enable */
+#define KVX_SFR_SPS_PL0_MME_SHIFT 11
+#define KVX_SFR_SPS_PL0_MME_WIDTH 1
+#define KVX_SFR_SPS_PL0_MME_WFXL_MASK _ULL(0x800)
+#define KVX_SFR_SPS_PL0_MME_WFXL_CLEAR _ULL(0x800)
+#define KVX_SFR_SPS_PL0_MME_WFXL_SET _ULL(0x80000000000)
+
+#define KVX_SFR_SPS_PL0_IL_MASK _ULL(0x3000) /* Interrupt Level */
+#define KVX_SFR_SPS_PL0_IL_SHIFT 12
+#define KVX_SFR_SPS_PL0_IL_WIDTH 2
+#define KVX_SFR_SPS_PL0_IL_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_SPS_PL0_IL_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_SPS_PL0_IL_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_SPS_PL0_VS_MASK _ULL(0xc000) /* Virtual Space */
+#define KVX_SFR_SPS_PL0_VS_SHIFT 14
+#define KVX_SFR_SPS_PL0_VS_WIDTH 2
+#define KVX_SFR_SPS_PL0_VS_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_SPS_PL0_VS_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_SPS_PL0_VS_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_SPS_PL0_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */
+#define KVX_SFR_SPS_PL0_V64_SHIFT 16
+#define KVX_SFR_SPS_PL0_V64_WIDTH 1
+#define KVX_SFR_SPS_PL0_V64_WFXL_MASK _ULL(0x10000)
+#define KVX_SFR_SPS_PL0_V64_WFXL_CLEAR _ULL(0x10000)
+#define KVX_SFR_SPS_PL0_V64_WFXL_SET _ULL(0x1000000000000)
+
+#define KVX_SFR_SPS_PL0_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */
+#define KVX_SFR_SPS_PL0_L2E_SHIFT 17
+#define KVX_SFR_SPS_PL0_L2E_WIDTH 1
+#define KVX_SFR_SPS_PL0_L2E_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_SPS_PL0_L2E_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_SPS_PL0_L2E_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_SPS_PL0_SME_MASK _ULL(0x40000) /* Step Mode Enabled */
+#define KVX_SFR_SPS_PL0_SME_SHIFT 18
+#define KVX_SFR_SPS_PL0_SME_WIDTH 1
+#define KVX_SFR_SPS_PL0_SME_WFXL_MASK _ULL(0x40000)
+#define KVX_SFR_SPS_PL0_SME_WFXL_CLEAR _ULL(0x40000)
+#define KVX_SFR_SPS_PL0_SME_WFXL_SET _ULL(0x4000000000000)
+
+#define KVX_SFR_SPS_PL0_SMR_MASK _ULL(0x80000) /* Step Mode Ready */
+#define KVX_SFR_SPS_PL0_SMR_SHIFT 19
+#define KVX_SFR_SPS_PL0_SMR_WIDTH 1
+#define KVX_SFR_SPS_PL0_SMR_WFXL_MASK _ULL(0x80000)
+#define KVX_SFR_SPS_PL0_SMR_WFXL_CLEAR _ULL(0x80000)
+#define KVX_SFR_SPS_PL0_SMR_WFXL_SET _ULL(0x8000000000000)
+
+#define KVX_SFR_SPS_PL0_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */
+#define KVX_SFR_SPS_PL0_PMJ_SHIFT 20
+#define KVX_SFR_SPS_PL0_PMJ_WIDTH 4
+#define KVX_SFR_SPS_PL0_PMJ_WFXL_MASK _ULL(0xf00000)
+#define KVX_SFR_SPS_PL0_PMJ_WFXL_CLEAR _ULL(0xf00000)
+#define KVX_SFR_SPS_PL0_PMJ_WFXL_SET _ULL(0xf0000000000000)
+
+#define KVX_SFR_SPS_PL0_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */
+#define KVX_SFR_SPS_PL0_MMUP_SHIFT 24
+#define KVX_SFR_SPS_PL0_MMUP_WIDTH 1
+#define KVX_SFR_SPS_PL0_MMUP_WFXL_MASK _ULL(0x1000000)
+#define KVX_SFR_SPS_PL0_MMUP_WFXL_CLEAR _ULL(0x1000000)
+#define KVX_SFR_SPS_PL0_MMUP_WFXL_SET _ULL(0x100000000000000)
+
+#define KVX_SFR_SPS_PL1_PL_MASK _ULL(0x3) /* Current Privilege Level */
+#define KVX_SFR_SPS_PL1_PL_SHIFT 0
+#define KVX_SFR_SPS_PL1_PL_WIDTH 2
+#define KVX_SFR_SPS_PL1_PL_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_SPS_PL1_PL_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_SPS_PL1_PL_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_SPS_PL1_ET_MASK _ULL(0x4) /* Exception Taken */
+#define KVX_SFR_SPS_PL1_ET_SHIFT 2
+#define KVX_SFR_SPS_PL1_ET_WIDTH 1
+#define KVX_SFR_SPS_PL1_ET_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_SPS_PL1_ET_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_SPS_PL1_ET_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_SPS_PL1_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */
+#define KVX_SFR_SPS_PL1_HTD_SHIFT 3
+#define KVX_SFR_SPS_PL1_HTD_WIDTH 1
+#define KVX_SFR_SPS_PL1_HTD_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_SPS_PL1_HTD_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_SPS_PL1_HTD_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_SPS_PL1_IE_MASK _ULL(0x10) /* Interrupt Enable */
+#define KVX_SFR_SPS_PL1_IE_SHIFT 4
+#define KVX_SFR_SPS_PL1_IE_WIDTH 1
+#define KVX_SFR_SPS_PL1_IE_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_SPS_PL1_IE_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_SPS_PL1_IE_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_SPS_PL1_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */
+#define KVX_SFR_SPS_PL1_HLE_SHIFT 5
+#define KVX_SFR_SPS_PL1_HLE_WIDTH 1
+#define KVX_SFR_SPS_PL1_HLE_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_SPS_PL1_HLE_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_SPS_PL1_HLE_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_SPS_PL1_SRE_MASK _ULL(0x40) /* Software REserved */
+#define KVX_SFR_SPS_PL1_SRE_SHIFT 6
+#define KVX_SFR_SPS_PL1_SRE_WIDTH 1
+#define KVX_SFR_SPS_PL1_SRE_WFXL_MASK _ULL(0x40)
+#define KVX_SFR_SPS_PL1_SRE_WFXL_CLEAR _ULL(0x40)
+#define KVX_SFR_SPS_PL1_SRE_WFXL_SET _ULL(0x4000000000)
+
+#define KVX_SFR_SPS_PL1_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */
+#define KVX_SFR_SPS_PL1_DAUS_SHIFT 7
+#define KVX_SFR_SPS_PL1_DAUS_WIDTH 1
+#define KVX_SFR_SPS_PL1_DAUS_WFXL_MASK _ULL(0x80)
+#define KVX_SFR_SPS_PL1_DAUS_WFXL_CLEAR _ULL(0x80)
+#define KVX_SFR_SPS_PL1_DAUS_WFXL_SET _ULL(0x8000000000)
+
+#define KVX_SFR_SPS_PL1_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */
+#define KVX_SFR_SPS_PL1_ICE_SHIFT 8
+#define KVX_SFR_SPS_PL1_ICE_WIDTH 1
+#define KVX_SFR_SPS_PL1_ICE_WFXL_MASK _ULL(0x100)
+#define KVX_SFR_SPS_PL1_ICE_WFXL_CLEAR _ULL(0x100)
+#define KVX_SFR_SPS_PL1_ICE_WFXL_SET _ULL(0x10000000000)
+
+#define KVX_SFR_SPS_PL1_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */
+#define KVX_SFR_SPS_PL1_USE_SHIFT 9
+#define KVX_SFR_SPS_PL1_USE_WIDTH 1
+#define KVX_SFR_SPS_PL1_USE_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_SPS_PL1_USE_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_SPS_PL1_USE_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_SPS_PL1_DCE_MASK _ULL(0x400) /* Data Cache Enable */
+#define KVX_SFR_SPS_PL1_DCE_SHIFT 10
+#define KVX_SFR_SPS_PL1_DCE_WIDTH 1
+#define KVX_SFR_SPS_PL1_DCE_WFXL_MASK _ULL(0x400)
+#define KVX_SFR_SPS_PL1_DCE_WFXL_CLEAR _ULL(0x400)
+#define KVX_SFR_SPS_PL1_DCE_WFXL_SET _ULL(0x40000000000)
+
+#define KVX_SFR_SPS_PL1_MME_MASK _ULL(0x800) /* Memory Management Enable */
+#define KVX_SFR_SPS_PL1_MME_SHIFT 11
+#define KVX_SFR_SPS_PL1_MME_WIDTH 1
+#define KVX_SFR_SPS_PL1_MME_WFXL_MASK _ULL(0x800)
+#define KVX_SFR_SPS_PL1_MME_WFXL_CLEAR _ULL(0x800)
+#define KVX_SFR_SPS_PL1_MME_WFXL_SET _ULL(0x80000000000)
+
+#define KVX_SFR_SPS_PL1_IL_MASK _ULL(0x3000) /* Interrupt Level */
+#define KVX_SFR_SPS_PL1_IL_SHIFT 12
+#define KVX_SFR_SPS_PL1_IL_WIDTH 2
+#define KVX_SFR_SPS_PL1_IL_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_SPS_PL1_IL_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_SPS_PL1_IL_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_SPS_PL1_VS_MASK _ULL(0xc000) /* Virtual Space */
+#define KVX_SFR_SPS_PL1_VS_SHIFT 14
+#define KVX_SFR_SPS_PL1_VS_WIDTH 2
+#define KVX_SFR_SPS_PL1_VS_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_SPS_PL1_VS_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_SPS_PL1_VS_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_SPS_PL1_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */
+#define KVX_SFR_SPS_PL1_V64_SHIFT 16
+#define KVX_SFR_SPS_PL1_V64_WIDTH 1
+#define KVX_SFR_SPS_PL1_V64_WFXL_MASK _ULL(0x10000)
+#define KVX_SFR_SPS_PL1_V64_WFXL_CLEAR _ULL(0x10000)
+#define KVX_SFR_SPS_PL1_V64_WFXL_SET _ULL(0x1000000000000)
+
+#define KVX_SFR_SPS_PL1_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */
+#define KVX_SFR_SPS_PL1_L2E_SHIFT 17
+#define KVX_SFR_SPS_PL1_L2E_WIDTH 1
+#define KVX_SFR_SPS_PL1_L2E_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_SPS_PL1_L2E_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_SPS_PL1_L2E_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_SPS_PL1_SME_MASK _ULL(0x40000) /* Step Mode Enabled */
+#define KVX_SFR_SPS_PL1_SME_SHIFT 18
+#define KVX_SFR_SPS_PL1_SME_WIDTH 1
+#define KVX_SFR_SPS_PL1_SME_WFXL_MASK _ULL(0x40000)
+#define KVX_SFR_SPS_PL1_SME_WFXL_CLEAR _ULL(0x40000)
+#define KVX_SFR_SPS_PL1_SME_WFXL_SET _ULL(0x4000000000000)
+
+#define KVX_SFR_SPS_PL1_SMR_MASK _ULL(0x80000) /* Step Mode Ready */
+#define KVX_SFR_SPS_PL1_SMR_SHIFT 19
+#define KVX_SFR_SPS_PL1_SMR_WIDTH 1
+#define KVX_SFR_SPS_PL1_SMR_WFXL_MASK _ULL(0x80000)
+#define KVX_SFR_SPS_PL1_SMR_WFXL_CLEAR _ULL(0x80000)
+#define KVX_SFR_SPS_PL1_SMR_WFXL_SET _ULL(0x8000000000000)
+
+#define KVX_SFR_SPS_PL1_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */
+#define KVX_SFR_SPS_PL1_PMJ_SHIFT 20
+#define KVX_SFR_SPS_PL1_PMJ_WIDTH 4
+#define KVX_SFR_SPS_PL1_PMJ_WFXL_MASK _ULL(0xf00000)
+#define KVX_SFR_SPS_PL1_PMJ_WFXL_CLEAR _ULL(0xf00000)
+#define KVX_SFR_SPS_PL1_PMJ_WFXL_SET _ULL(0xf0000000000000)
+
+#define KVX_SFR_SPS_PL1_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */
+#define KVX_SFR_SPS_PL1_MMUP_SHIFT 24
+#define KVX_SFR_SPS_PL1_MMUP_WIDTH 1
+#define KVX_SFR_SPS_PL1_MMUP_WFXL_MASK _ULL(0x1000000)
+#define KVX_SFR_SPS_PL1_MMUP_WFXL_CLEAR _ULL(0x1000000)
+#define KVX_SFR_SPS_PL1_MMUP_WFXL_SET _ULL(0x100000000000000)
+
+#define KVX_SFR_SPS_PL2_PL_MASK _ULL(0x3) /* Current Privilege Level */
+#define KVX_SFR_SPS_PL2_PL_SHIFT 0
+#define KVX_SFR_SPS_PL2_PL_WIDTH 2
+#define KVX_SFR_SPS_PL2_PL_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_SPS_PL2_PL_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_SPS_PL2_PL_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_SPS_PL2_ET_MASK _ULL(0x4) /* Exception Taken */
+#define KVX_SFR_SPS_PL2_ET_SHIFT 2
+#define KVX_SFR_SPS_PL2_ET_WIDTH 1
+#define KVX_SFR_SPS_PL2_ET_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_SPS_PL2_ET_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_SPS_PL2_ET_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_SPS_PL2_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */
+#define KVX_SFR_SPS_PL2_HTD_SHIFT 3
+#define KVX_SFR_SPS_PL2_HTD_WIDTH 1
+#define KVX_SFR_SPS_PL2_HTD_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_SPS_PL2_HTD_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_SPS_PL2_HTD_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_SPS_PL2_IE_MASK _ULL(0x10) /* Interrupt Enable */
+#define KVX_SFR_SPS_PL2_IE_SHIFT 4
+#define KVX_SFR_SPS_PL2_IE_WIDTH 1
+#define KVX_SFR_SPS_PL2_IE_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_SPS_PL2_IE_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_SPS_PL2_IE_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_SPS_PL2_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */
+#define KVX_SFR_SPS_PL2_HLE_SHIFT 5
+#define KVX_SFR_SPS_PL2_HLE_WIDTH 1
+#define KVX_SFR_SPS_PL2_HLE_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_SPS_PL2_HLE_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_SPS_PL2_HLE_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_SPS_PL2_SRE_MASK _ULL(0x40) /* Software REserved */
+#define KVX_SFR_SPS_PL2_SRE_SHIFT 6
+#define KVX_SFR_SPS_PL2_SRE_WIDTH 1
+#define KVX_SFR_SPS_PL2_SRE_WFXL_MASK _ULL(0x40)
+#define KVX_SFR_SPS_PL2_SRE_WFXL_CLEAR _ULL(0x40)
+#define KVX_SFR_SPS_PL2_SRE_WFXL_SET _ULL(0x4000000000)
+
+#define KVX_SFR_SPS_PL2_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */
+#define KVX_SFR_SPS_PL2_DAUS_SHIFT 7
+#define KVX_SFR_SPS_PL2_DAUS_WIDTH 1
+#define KVX_SFR_SPS_PL2_DAUS_WFXL_MASK _ULL(0x80)
+#define KVX_SFR_SPS_PL2_DAUS_WFXL_CLEAR _ULL(0x80)
+#define KVX_SFR_SPS_PL2_DAUS_WFXL_SET _ULL(0x8000000000)
+
+#define KVX_SFR_SPS_PL2_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */
+#define KVX_SFR_SPS_PL2_ICE_SHIFT 8
+#define KVX_SFR_SPS_PL2_ICE_WIDTH 1
+#define KVX_SFR_SPS_PL2_ICE_WFXL_MASK _ULL(0x100)
+#define KVX_SFR_SPS_PL2_ICE_WFXL_CLEAR _ULL(0x100)
+#define KVX_SFR_SPS_PL2_ICE_WFXL_SET _ULL(0x10000000000)
+
+#define KVX_SFR_SPS_PL2_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */
+#define KVX_SFR_SPS_PL2_USE_SHIFT 9
+#define KVX_SFR_SPS_PL2_USE_WIDTH 1
+#define KVX_SFR_SPS_PL2_USE_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_SPS_PL2_USE_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_SPS_PL2_USE_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_SPS_PL2_DCE_MASK _ULL(0x400) /* Data Cache Enable */
+#define KVX_SFR_SPS_PL2_DCE_SHIFT 10
+#define KVX_SFR_SPS_PL2_DCE_WIDTH 1
+#define KVX_SFR_SPS_PL2_DCE_WFXL_MASK _ULL(0x400)
+#define KVX_SFR_SPS_PL2_DCE_WFXL_CLEAR _ULL(0x400)
+#define KVX_SFR_SPS_PL2_DCE_WFXL_SET _ULL(0x40000000000)
+
+#define KVX_SFR_SPS_PL2_MME_MASK _ULL(0x800) /* Memory Management Enable */
+#define KVX_SFR_SPS_PL2_MME_SHIFT 11
+#define KVX_SFR_SPS_PL2_MME_WIDTH 1
+#define KVX_SFR_SPS_PL2_MME_WFXL_MASK _ULL(0x800)
+#define KVX_SFR_SPS_PL2_MME_WFXL_CLEAR _ULL(0x800)
+#define KVX_SFR_SPS_PL2_MME_WFXL_SET _ULL(0x80000000000)
+
+#define KVX_SFR_SPS_PL2_IL_MASK _ULL(0x3000) /* Interrupt Level */
+#define KVX_SFR_SPS_PL2_IL_SHIFT 12
+#define KVX_SFR_SPS_PL2_IL_WIDTH 2
+#define KVX_SFR_SPS_PL2_IL_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_SPS_PL2_IL_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_SPS_PL2_IL_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_SPS_PL2_VS_MASK _ULL(0xc000) /* Virtual Space */
+#define KVX_SFR_SPS_PL2_VS_SHIFT 14
+#define KVX_SFR_SPS_PL2_VS_WIDTH 2
+#define KVX_SFR_SPS_PL2_VS_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_SPS_PL2_VS_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_SPS_PL2_VS_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_SPS_PL2_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */
+#define KVX_SFR_SPS_PL2_V64_SHIFT 16
+#define KVX_SFR_SPS_PL2_V64_WIDTH 1
+#define KVX_SFR_SPS_PL2_V64_WFXL_MASK _ULL(0x10000)
+#define KVX_SFR_SPS_PL2_V64_WFXL_CLEAR _ULL(0x10000)
+#define KVX_SFR_SPS_PL2_V64_WFXL_SET _ULL(0x1000000000000)
+
+#define KVX_SFR_SPS_PL2_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */
+#define KVX_SFR_SPS_PL2_L2E_SHIFT 17
+#define KVX_SFR_SPS_PL2_L2E_WIDTH 1
+#define KVX_SFR_SPS_PL2_L2E_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_SPS_PL2_L2E_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_SPS_PL2_L2E_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_SPS_PL2_SME_MASK _ULL(0x40000) /* Step Mode Enabled */
+#define KVX_SFR_SPS_PL2_SME_SHIFT 18
+#define KVX_SFR_SPS_PL2_SME_WIDTH 1
+#define KVX_SFR_SPS_PL2_SME_WFXL_MASK _ULL(0x40000)
+#define KVX_SFR_SPS_PL2_SME_WFXL_CLEAR _ULL(0x40000)
+#define KVX_SFR_SPS_PL2_SME_WFXL_SET _ULL(0x4000000000000)
+
+#define KVX_SFR_SPS_PL2_SMR_MASK _ULL(0x80000) /* Step Mode Ready */
+#define KVX_SFR_SPS_PL2_SMR_SHIFT 19
+#define KVX_SFR_SPS_PL2_SMR_WIDTH 1
+#define KVX_SFR_SPS_PL2_SMR_WFXL_MASK _ULL(0x80000)
+#define KVX_SFR_SPS_PL2_SMR_WFXL_CLEAR _ULL(0x80000)
+#define KVX_SFR_SPS_PL2_SMR_WFXL_SET _ULL(0x8000000000000)
+
+#define KVX_SFR_SPS_PL2_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */
+#define KVX_SFR_SPS_PL2_PMJ_SHIFT 20
+#define KVX_SFR_SPS_PL2_PMJ_WIDTH 4
+#define KVX_SFR_SPS_PL2_PMJ_WFXL_MASK _ULL(0xf00000)
+#define KVX_SFR_SPS_PL2_PMJ_WFXL_CLEAR _ULL(0xf00000)
+#define KVX_SFR_SPS_PL2_PMJ_WFXL_SET _ULL(0xf0000000000000)
+
+#define KVX_SFR_SPS_PL2_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */
+#define KVX_SFR_SPS_PL2_MMUP_SHIFT 24
+#define KVX_SFR_SPS_PL2_MMUP_WIDTH 1
+#define KVX_SFR_SPS_PL2_MMUP_WFXL_MASK _ULL(0x1000000)
+#define KVX_SFR_SPS_PL2_MMUP_WFXL_CLEAR _ULL(0x1000000)
+#define KVX_SFR_SPS_PL2_MMUP_WFXL_SET _ULL(0x100000000000000)
+
+#define KVX_SFR_SPS_PL3_PL_MASK _ULL(0x3) /* Current Privilege Level */
+#define KVX_SFR_SPS_PL3_PL_SHIFT 0
+#define KVX_SFR_SPS_PL3_PL_WIDTH 2
+#define KVX_SFR_SPS_PL3_PL_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_SPS_PL3_PL_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_SPS_PL3_PL_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_SPS_PL3_ET_MASK _ULL(0x4) /* Exception Taken */
+#define KVX_SFR_SPS_PL3_ET_SHIFT 2
+#define KVX_SFR_SPS_PL3_ET_WIDTH 1
+#define KVX_SFR_SPS_PL3_ET_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_SPS_PL3_ET_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_SPS_PL3_ET_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_SPS_PL3_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */
+#define KVX_SFR_SPS_PL3_HTD_SHIFT 3
+#define KVX_SFR_SPS_PL3_HTD_WIDTH 1
+#define KVX_SFR_SPS_PL3_HTD_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_SPS_PL3_HTD_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_SPS_PL3_HTD_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_SPS_PL3_IE_MASK _ULL(0x10) /* Interrupt Enable */
+#define KVX_SFR_SPS_PL3_IE_SHIFT 4
+#define KVX_SFR_SPS_PL3_IE_WIDTH 1
+#define KVX_SFR_SPS_PL3_IE_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_SPS_PL3_IE_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_SPS_PL3_IE_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_SPS_PL3_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */
+#define KVX_SFR_SPS_PL3_HLE_SHIFT 5
+#define KVX_SFR_SPS_PL3_HLE_WIDTH 1
+#define KVX_SFR_SPS_PL3_HLE_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_SPS_PL3_HLE_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_SPS_PL3_HLE_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_SPS_PL3_SRE_MASK _ULL(0x40) /* Software REserved */
+#define KVX_SFR_SPS_PL3_SRE_SHIFT 6
+#define KVX_SFR_SPS_PL3_SRE_WIDTH 1
+#define KVX_SFR_SPS_PL3_SRE_WFXL_MASK _ULL(0x40)
+#define KVX_SFR_SPS_PL3_SRE_WFXL_CLEAR _ULL(0x40)
+#define KVX_SFR_SPS_PL3_SRE_WFXL_SET _ULL(0x4000000000)
+
+#define KVX_SFR_SPS_PL3_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */
+#define KVX_SFR_SPS_PL3_DAUS_SHIFT 7
+#define KVX_SFR_SPS_PL3_DAUS_WIDTH 1
+#define KVX_SFR_SPS_PL3_DAUS_WFXL_MASK _ULL(0x80)
+#define KVX_SFR_SPS_PL3_DAUS_WFXL_CLEAR _ULL(0x80)
+#define KVX_SFR_SPS_PL3_DAUS_WFXL_SET _ULL(0x8000000000)
+
+#define KVX_SFR_SPS_PL3_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */
+#define KVX_SFR_SPS_PL3_ICE_SHIFT 8
+#define KVX_SFR_SPS_PL3_ICE_WIDTH 1
+#define KVX_SFR_SPS_PL3_ICE_WFXL_MASK _ULL(0x100)
+#define KVX_SFR_SPS_PL3_ICE_WFXL_CLEAR _ULL(0x100)
+#define KVX_SFR_SPS_PL3_ICE_WFXL_SET _ULL(0x10000000000)
+
+#define KVX_SFR_SPS_PL3_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */
+#define KVX_SFR_SPS_PL3_USE_SHIFT 9
+#define KVX_SFR_SPS_PL3_USE_WIDTH 1
+#define KVX_SFR_SPS_PL3_USE_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_SPS_PL3_USE_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_SPS_PL3_USE_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_SPS_PL3_DCE_MASK _ULL(0x400) /* Data Cache Enable */
+#define KVX_SFR_SPS_PL3_DCE_SHIFT 10
+#define KVX_SFR_SPS_PL3_DCE_WIDTH 1
+#define KVX_SFR_SPS_PL3_DCE_WFXL_MASK _ULL(0x400)
+#define KVX_SFR_SPS_PL3_DCE_WFXL_CLEAR _ULL(0x400)
+#define KVX_SFR_SPS_PL3_DCE_WFXL_SET _ULL(0x40000000000)
+
+#define KVX_SFR_SPS_PL3_MME_MASK _ULL(0x800) /* Memory Management Enable */
+#define KVX_SFR_SPS_PL3_MME_SHIFT 11
+#define KVX_SFR_SPS_PL3_MME_WIDTH 1
+#define KVX_SFR_SPS_PL3_MME_WFXL_MASK _ULL(0x800)
+#define KVX_SFR_SPS_PL3_MME_WFXL_CLEAR _ULL(0x800)
+#define KVX_SFR_SPS_PL3_MME_WFXL_SET _ULL(0x80000000000)
+
+#define KVX_SFR_SPS_PL3_IL_MASK _ULL(0x3000) /* Interrupt Level */
+#define KVX_SFR_SPS_PL3_IL_SHIFT 12
+#define KVX_SFR_SPS_PL3_IL_WIDTH 2
+#define KVX_SFR_SPS_PL3_IL_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_SPS_PL3_IL_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_SPS_PL3_IL_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_SPS_PL3_VS_MASK _ULL(0xc000) /* Virtual Space */
+#define KVX_SFR_SPS_PL3_VS_SHIFT 14
+#define KVX_SFR_SPS_PL3_VS_WIDTH 2
+#define KVX_SFR_SPS_PL3_VS_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_SPS_PL3_VS_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_SPS_PL3_VS_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_SPS_PL3_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */
+#define KVX_SFR_SPS_PL3_V64_SHIFT 16
+#define KVX_SFR_SPS_PL3_V64_WIDTH 1
+#define KVX_SFR_SPS_PL3_V64_WFXL_MASK _ULL(0x10000)
+#define KVX_SFR_SPS_PL3_V64_WFXL_CLEAR _ULL(0x10000)
+#define KVX_SFR_SPS_PL3_V64_WFXL_SET _ULL(0x1000000000000)
+
+#define KVX_SFR_SPS_PL3_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */
+#define KVX_SFR_SPS_PL3_L2E_SHIFT 17
+#define KVX_SFR_SPS_PL3_L2E_WIDTH 1
+#define KVX_SFR_SPS_PL3_L2E_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_SPS_PL3_L2E_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_SPS_PL3_L2E_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_SPS_PL3_SME_MASK _ULL(0x40000) /* Step Mode Enabled */
+#define KVX_SFR_SPS_PL3_SME_SHIFT 18
+#define KVX_SFR_SPS_PL3_SME_WIDTH 1
+#define KVX_SFR_SPS_PL3_SME_WFXL_MASK _ULL(0x40000)
+#define KVX_SFR_SPS_PL3_SME_WFXL_CLEAR _ULL(0x40000)
+#define KVX_SFR_SPS_PL3_SME_WFXL_SET _ULL(0x4000000000000)
+
+#define KVX_SFR_SPS_PL3_SMR_MASK _ULL(0x80000) /* Step Mode Ready */
+#define KVX_SFR_SPS_PL3_SMR_SHIFT 19
+#define KVX_SFR_SPS_PL3_SMR_WIDTH 1
+#define KVX_SFR_SPS_PL3_SMR_WFXL_MASK _ULL(0x80000)
+#define KVX_SFR_SPS_PL3_SMR_WFXL_CLEAR _ULL(0x80000)
+#define KVX_SFR_SPS_PL3_SMR_WFXL_SET _ULL(0x8000000000000)
+
+#define KVX_SFR_SPS_PL3_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */
+#define KVX_SFR_SPS_PL3_PMJ_SHIFT 20
+#define KVX_SFR_SPS_PL3_PMJ_WIDTH 4
+#define KVX_SFR_SPS_PL3_PMJ_WFXL_MASK _ULL(0xf00000)
+#define KVX_SFR_SPS_PL3_PMJ_WFXL_CLEAR _ULL(0xf00000)
+#define KVX_SFR_SPS_PL3_PMJ_WFXL_SET _ULL(0xf0000000000000)
+
+#define KVX_SFR_SPS_PL3_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */
+#define KVX_SFR_SPS_PL3_MMUP_SHIFT 24
+#define KVX_SFR_SPS_PL3_MMUP_WIDTH 1
+#define KVX_SFR_SPS_PL3_MMUP_WFXL_MASK _ULL(0x1000000)
+#define KVX_SFR_SPS_PL3_MMUP_WFXL_CLEAR _ULL(0x1000000)
+#define KVX_SFR_SPS_PL3_MMUP_WFXL_SET _ULL(0x100000000000000)
+
+#define KVX_SFR_PSO_PL0_MASK _ULL(0x3) /* Current Privilege Level bit 0 owner */
+#define KVX_SFR_PSO_PL0_SHIFT 0
+#define KVX_SFR_PSO_PL0_WIDTH 2
+#define KVX_SFR_PSO_PL0_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_PSO_PL0_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_PSO_PL0_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_PSO_PL1_MASK _ULL(0xc) /* Current Privilege Level bit 1 owner */
+#define KVX_SFR_PSO_PL1_SHIFT 2
+#define KVX_SFR_PSO_PL1_WIDTH 2
+#define KVX_SFR_PSO_PL1_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_PSO_PL1_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_PSO_PL1_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_PSO_ET_MASK _ULL(0x30) /* Exception Taken owner */
+#define KVX_SFR_PSO_ET_SHIFT 4
+#define KVX_SFR_PSO_ET_WIDTH 2
+#define KVX_SFR_PSO_ET_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_PSO_ET_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_PSO_ET_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_PSO_HTD_MASK _ULL(0xc0) /* Hardware Trap Disable owner */
+#define KVX_SFR_PSO_HTD_SHIFT 6
+#define KVX_SFR_PSO_HTD_WIDTH 2
+#define KVX_SFR_PSO_HTD_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_PSO_HTD_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_PSO_HTD_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_PSO_IE_MASK _ULL(0x300) /* Interrupt Enable owner */
+#define KVX_SFR_PSO_IE_SHIFT 8
+#define KVX_SFR_PSO_IE_WIDTH 2
+#define KVX_SFR_PSO_IE_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_PSO_IE_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_PSO_IE_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_PSO_HLE_MASK _ULL(0xc00) /* Hardware Loop Enable owner */
+#define KVX_SFR_PSO_HLE_SHIFT 10
+#define KVX_SFR_PSO_HLE_WIDTH 2
+#define KVX_SFR_PSO_HLE_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_PSO_HLE_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_PSO_HLE_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_PSO_SRE_MASK _ULL(0x3000) /* Software REserved owner */
+#define KVX_SFR_PSO_SRE_SHIFT 12
+#define KVX_SFR_PSO_SRE_WIDTH 2
+#define KVX_SFR_PSO_SRE_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_PSO_SRE_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_PSO_SRE_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_PSO_DAUS_MASK _ULL(0xc000) /* Data Accesses Use SPS settings owner */
+#define KVX_SFR_PSO_DAUS_SHIFT 14
+#define KVX_SFR_PSO_DAUS_WIDTH 2
+#define KVX_SFR_PSO_DAUS_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_PSO_DAUS_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_PSO_DAUS_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_PSO_ICE_MASK _ULL(0x30000) /* Instruction Cache Enable owner */
+#define KVX_SFR_PSO_ICE_SHIFT 16
+#define KVX_SFR_PSO_ICE_WIDTH 2
+#define KVX_SFR_PSO_ICE_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_PSO_ICE_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_PSO_ICE_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_PSO_USE_MASK _ULL(0xc0000) /* Uncached Streaming Enable owner */
+#define KVX_SFR_PSO_USE_SHIFT 18
+#define KVX_SFR_PSO_USE_WIDTH 2
+#define KVX_SFR_PSO_USE_WFXL_MASK _ULL(0xc0000)
+#define KVX_SFR_PSO_USE_WFXL_CLEAR _ULL(0xc0000)
+#define KVX_SFR_PSO_USE_WFXL_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_PSO_DCE_MASK _ULL(0x300000) /* Data Cache Enable owner */
+#define KVX_SFR_PSO_DCE_SHIFT 20
+#define KVX_SFR_PSO_DCE_WIDTH 2
+#define KVX_SFR_PSO_DCE_WFXL_MASK _ULL(0x300000)
+#define KVX_SFR_PSO_DCE_WFXL_CLEAR _ULL(0x300000)
+#define KVX_SFR_PSO_DCE_WFXL_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_PSO_MME_MASK _ULL(0xc00000) /* Memory Management Enable owner */
+#define KVX_SFR_PSO_MME_SHIFT 22
+#define KVX_SFR_PSO_MME_WIDTH 2
+#define KVX_SFR_PSO_MME_WFXL_MASK _ULL(0xc00000)
+#define KVX_SFR_PSO_MME_WFXL_CLEAR _ULL(0xc00000)
+#define KVX_SFR_PSO_MME_WFXL_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_PSO_IL0_MASK _ULL(0x3000000) /* Interrupt Level bit 0 owner */
+#define KVX_SFR_PSO_IL0_SHIFT 24
+#define KVX_SFR_PSO_IL0_WIDTH 2
+#define KVX_SFR_PSO_IL0_WFXL_MASK _ULL(0x3000000)
+#define KVX_SFR_PSO_IL0_WFXL_CLEAR _ULL(0x3000000)
+#define KVX_SFR_PSO_IL0_WFXL_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_PSO_IL1_MASK _ULL(0xc000000) /* Interrupt Level bit 1 owner */
+#define KVX_SFR_PSO_IL1_SHIFT 26
+#define KVX_SFR_PSO_IL1_WIDTH 2
+#define KVX_SFR_PSO_IL1_WFXL_MASK _ULL(0xc000000)
+#define KVX_SFR_PSO_IL1_WFXL_CLEAR _ULL(0xc000000)
+#define KVX_SFR_PSO_IL1_WFXL_SET _ULL(0xc00000000000000)
+
+#define KVX_SFR_PSO_VS0_MASK _ULL(0x30000000) /* Virtual Space bit 0 owner */
+#define KVX_SFR_PSO_VS0_SHIFT 28
+#define KVX_SFR_PSO_VS0_WIDTH 2
+#define KVX_SFR_PSO_VS0_WFXL_MASK _ULL(0x30000000)
+#define KVX_SFR_PSO_VS0_WFXL_CLEAR _ULL(0x30000000)
+#define KVX_SFR_PSO_VS0_WFXL_SET _ULL(0x3000000000000000)
+
+#define KVX_SFR_PSO_VS1_MASK _ULL(0xc0000000) /* Virtual Space bit 1 owner */
+#define KVX_SFR_PSO_VS1_SHIFT 30
+#define KVX_SFR_PSO_VS1_WIDTH 2
+#define KVX_SFR_PSO_VS1_WFXL_MASK _ULL(0xc0000000)
+#define KVX_SFR_PSO_VS1_WFXL_CLEAR _ULL(0xc0000000)
+#define KVX_SFR_PSO_VS1_WFXL_SET _ULL(0xc000000000000000)
+
+#define KVX_SFR_PSO_V64_MASK _ULL(0x300000000) /* Virtual 64 bits mode owner */
+#define KVX_SFR_PSO_V64_SHIFT 32
+#define KVX_SFR_PSO_V64_WIDTH 2
+#define KVX_SFR_PSO_V64_WFXM_MASK _ULL(0x300000000)
+#define KVX_SFR_PSO_V64_WFXM_CLEAR _ULL(0x3)
+#define KVX_SFR_PSO_V64_WFXM_SET _ULL(0x300000000)
+
+#define KVX_SFR_PSO_L2E_MASK _ULL(0xc00000000) /* L2 cache Enable owner */
+#define KVX_SFR_PSO_L2E_SHIFT 34
+#define KVX_SFR_PSO_L2E_WIDTH 2
+#define KVX_SFR_PSO_L2E_WFXM_MASK _ULL(0xc00000000)
+#define KVX_SFR_PSO_L2E_WFXM_CLEAR _ULL(0xc)
+#define KVX_SFR_PSO_L2E_WFXM_SET _ULL(0xc00000000)
+
+#define KVX_SFR_PSO_SME_MASK _ULL(0x3000000000) /* Step Mode Enabled owner */
+#define KVX_SFR_PSO_SME_SHIFT 36
+#define KVX_SFR_PSO_SME_WIDTH 2
+#define KVX_SFR_PSO_SME_WFXM_MASK _ULL(0x3000000000)
+#define KVX_SFR_PSO_SME_WFXM_CLEAR _ULL(0x30)
+#define KVX_SFR_PSO_SME_WFXM_SET _ULL(0x3000000000)
+
+#define KVX_SFR_PSO_SMR_MASK _ULL(0xc000000000) /* Step Mode Ready owner */
+#define KVX_SFR_PSO_SMR_SHIFT 38
+#define KVX_SFR_PSO_SMR_WIDTH 2
+#define KVX_SFR_PSO_SMR_WFXM_MASK _ULL(0xc000000000)
+#define KVX_SFR_PSO_SMR_WFXM_CLEAR _ULL(0xc0)
+#define KVX_SFR_PSO_SMR_WFXM_SET _ULL(0xc000000000)
+
+#define KVX_SFR_PSO_PMJ0_MASK _ULL(0x30000000000) /* Page Mask in JTLB bit 0 owner */
+#define KVX_SFR_PSO_PMJ0_SHIFT 40
+#define KVX_SFR_PSO_PMJ0_WIDTH 2
+#define KVX_SFR_PSO_PMJ0_WFXM_MASK _ULL(0x30000000000)
+#define KVX_SFR_PSO_PMJ0_WFXM_CLEAR _ULL(0x300)
+#define KVX_SFR_PSO_PMJ0_WFXM_SET _ULL(0x30000000000)
+
+#define KVX_SFR_PSO_PMJ1_MASK _ULL(0xc0000000000) /* Page Mask in JTLB bit 1 owner */
+#define KVX_SFR_PSO_PMJ1_SHIFT 42
+#define KVX_SFR_PSO_PMJ1_WIDTH 2
+#define KVX_SFR_PSO_PMJ1_WFXM_MASK _ULL(0xc0000000000)
+#define KVX_SFR_PSO_PMJ1_WFXM_CLEAR _ULL(0xc00)
+#define KVX_SFR_PSO_PMJ1_WFXM_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_PSO_PMJ2_MASK _ULL(0x300000000000) /* Page Mask in JTLB bit 2 owner */
+#define KVX_SFR_PSO_PMJ2_SHIFT 44
+#define KVX_SFR_PSO_PMJ2_WIDTH 2
+#define KVX_SFR_PSO_PMJ2_WFXM_MASK _ULL(0x300000000000)
+#define KVX_SFR_PSO_PMJ2_WFXM_CLEAR _ULL(0x3000)
+#define KVX_SFR_PSO_PMJ2_WFXM_SET _ULL(0x300000000000)
+
+#define KVX_SFR_PSO_PMJ3_MASK _ULL(0xc00000000000) /* Page Mask in JTLB bit 3 owner */
+#define KVX_SFR_PSO_PMJ3_SHIFT 46
+#define KVX_SFR_PSO_PMJ3_WIDTH 2
+#define KVX_SFR_PSO_PMJ3_WFXM_MASK _ULL(0xc00000000000)
+#define KVX_SFR_PSO_PMJ3_WFXM_CLEAR _ULL(0xc000)
+#define KVX_SFR_PSO_PMJ3_WFXM_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_PSO_MMUP_MASK _ULL(0x3000000000000) /* Privileged on MMU owner. */
+#define KVX_SFR_PSO_MMUP_SHIFT 48
+#define KVX_SFR_PSO_MMUP_WIDTH 2
+#define KVX_SFR_PSO_MMUP_WFXM_MASK _ULL(0x3000000000000)
+#define KVX_SFR_PSO_MMUP_WFXM_CLEAR _ULL(0x30000)
+#define KVX_SFR_PSO_MMUP_WFXM_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_PSOW_PL0_MASK _ULL(0x3) /* Current Privilege Level bit 0 owner */
+#define KVX_SFR_PSOW_PL0_SHIFT 0
+#define KVX_SFR_PSOW_PL0_WIDTH 2
+#define KVX_SFR_PSOW_PL0_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_PSOW_PL0_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_PSOW_PL0_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_PSOW_PL1_MASK _ULL(0xc) /* Current Privilege Level bit 1 owner */
+#define KVX_SFR_PSOW_PL1_SHIFT 2
+#define KVX_SFR_PSOW_PL1_WIDTH 2
+#define KVX_SFR_PSOW_PL1_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_PSOW_PL1_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_PSOW_PL1_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_PSOW_ET_MASK _ULL(0x30) /* Exception Taken owner */
+#define KVX_SFR_PSOW_ET_SHIFT 4
+#define KVX_SFR_PSOW_ET_WIDTH 2
+#define KVX_SFR_PSOW_ET_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_PSOW_ET_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_PSOW_ET_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_PSOW_HTD_MASK _ULL(0xc0) /* Hardware Trap Disable owner */
+#define KVX_SFR_PSOW_HTD_SHIFT 6
+#define KVX_SFR_PSOW_HTD_WIDTH 2
+#define KVX_SFR_PSOW_HTD_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_PSOW_HTD_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_PSOW_HTD_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_PSOW_IE_MASK _ULL(0x300) /* Interrupt Enable owner */
+#define KVX_SFR_PSOW_IE_SHIFT 8
+#define KVX_SFR_PSOW_IE_WIDTH 2
+#define KVX_SFR_PSOW_IE_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_PSOW_IE_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_PSOW_IE_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_PSOW_HLE_MASK _ULL(0xc00) /* Hardware Loop Enable owner */
+#define KVX_SFR_PSOW_HLE_SHIFT 10
+#define KVX_SFR_PSOW_HLE_WIDTH 2
+#define KVX_SFR_PSOW_HLE_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_PSOW_HLE_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_PSOW_HLE_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_PSOW_SRE_MASK _ULL(0x3000) /* Software REserved owner */
+#define KVX_SFR_PSOW_SRE_SHIFT 12
+#define KVX_SFR_PSOW_SRE_WIDTH 2
+#define KVX_SFR_PSOW_SRE_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_PSOW_SRE_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_PSOW_SRE_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_PSOW_DAUS_MASK _ULL(0xc000) /* Data Accesses Use SPS settings owner */
+#define KVX_SFR_PSOW_DAUS_SHIFT 14
+#define KVX_SFR_PSOW_DAUS_WIDTH 2
+#define KVX_SFR_PSOW_DAUS_WFXL_MASK _ULL(0xc000)
+#define KVX_SFR_PSOW_DAUS_WFXL_CLEAR _ULL(0xc000)
+#define KVX_SFR_PSOW_DAUS_WFXL_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_PSOW_ICE_MASK _ULL(0x30000) /* Instruction Cache Enable owner */
+#define KVX_SFR_PSOW_ICE_SHIFT 16
+#define KVX_SFR_PSOW_ICE_WIDTH 2
+#define KVX_SFR_PSOW_ICE_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_PSOW_ICE_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_PSOW_ICE_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_PSOW_USE_MASK _ULL(0xc0000) /* Uncached Streaming Enable owner */
+#define KVX_SFR_PSOW_USE_SHIFT 18
+#define KVX_SFR_PSOW_USE_WIDTH 2
+#define KVX_SFR_PSOW_USE_WFXL_MASK _ULL(0xc0000)
+#define KVX_SFR_PSOW_USE_WFXL_CLEAR _ULL(0xc0000)
+#define KVX_SFR_PSOW_USE_WFXL_SET _ULL(0xc000000000000)
+
+#define KVX_SFR_PSOW_DCE_MASK _ULL(0x300000) /* Data Cache Enable owner */
+#define KVX_SFR_PSOW_DCE_SHIFT 20
+#define KVX_SFR_PSOW_DCE_WIDTH 2
+#define KVX_SFR_PSOW_DCE_WFXL_MASK _ULL(0x300000)
+#define KVX_SFR_PSOW_DCE_WFXL_CLEAR _ULL(0x300000)
+#define KVX_SFR_PSOW_DCE_WFXL_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_PSOW_MME_MASK _ULL(0xc00000) /* Memory Management Enable owner */
+#define KVX_SFR_PSOW_MME_SHIFT 22
+#define KVX_SFR_PSOW_MME_WIDTH 2
+#define KVX_SFR_PSOW_MME_WFXL_MASK _ULL(0xc00000)
+#define KVX_SFR_PSOW_MME_WFXL_CLEAR _ULL(0xc00000)
+#define KVX_SFR_PSOW_MME_WFXL_SET _ULL(0xc0000000000000)
+
+#define KVX_SFR_PSOW_IL0_MASK _ULL(0x3000000) /* Interrupt Level bit 0 owner */
+#define KVX_SFR_PSOW_IL0_SHIFT 24
+#define KVX_SFR_PSOW_IL0_WIDTH 2
+#define KVX_SFR_PSOW_IL0_WFXL_MASK _ULL(0x3000000)
+#define KVX_SFR_PSOW_IL0_WFXL_CLEAR _ULL(0x3000000)
+#define KVX_SFR_PSOW_IL0_WFXL_SET _ULL(0x300000000000000)
+
+#define KVX_SFR_PSOW_IL1_MASK _ULL(0xc000000) /* Interrupt Level bit 1 owner */
+#define KVX_SFR_PSOW_IL1_SHIFT 26
+#define KVX_SFR_PSOW_IL1_WIDTH 2
+#define KVX_SFR_PSOW_IL1_WFXL_MASK _ULL(0xc000000)
+#define KVX_SFR_PSOW_IL1_WFXL_CLEAR _ULL(0xc000000)
+#define KVX_SFR_PSOW_IL1_WFXL_SET _ULL(0xc00000000000000)
+
+#define KVX_SFR_PSOW_VS0_MASK _ULL(0x30000000) /* Virtual Space bit 0 owner */
+#define KVX_SFR_PSOW_VS0_SHIFT 28
+#define KVX_SFR_PSOW_VS0_WIDTH 2
+#define KVX_SFR_PSOW_VS0_WFXL_MASK _ULL(0x30000000)
+#define KVX_SFR_PSOW_VS0_WFXL_CLEAR _ULL(0x30000000)
+#define KVX_SFR_PSOW_VS0_WFXL_SET _ULL(0x3000000000000000)
+
+#define KVX_SFR_PSOW_VS1_MASK _ULL(0xc0000000) /* Virtual Space bit 1 owner */
+#define KVX_SFR_PSOW_VS1_SHIFT 30
+#define KVX_SFR_PSOW_VS1_WIDTH 2
+#define KVX_SFR_PSOW_VS1_WFXL_MASK _ULL(0xc0000000)
+#define KVX_SFR_PSOW_VS1_WFXL_CLEAR _ULL(0xc0000000)
+#define KVX_SFR_PSOW_VS1_WFXL_SET _ULL(0xc000000000000000)
+
+#define KVX_SFR_PSOW_V64_MASK _ULL(0x300000000) /* Virtual 64 bits mode owner */
+#define KVX_SFR_PSOW_V64_SHIFT 32
+#define KVX_SFR_PSOW_V64_WIDTH 2
+#define KVX_SFR_PSOW_V64_WFXM_MASK _ULL(0x300000000)
+#define KVX_SFR_PSOW_V64_WFXM_CLEAR _ULL(0x3)
+#define KVX_SFR_PSOW_V64_WFXM_SET _ULL(0x300000000)
+
+#define KVX_SFR_PSOW_L2E_MASK _ULL(0xc00000000) /* L2 cache Enable owner */
+#define KVX_SFR_PSOW_L2E_SHIFT 34
+#define KVX_SFR_PSOW_L2E_WIDTH 2
+#define KVX_SFR_PSOW_L2E_WFXM_MASK _ULL(0xc00000000)
+#define KVX_SFR_PSOW_L2E_WFXM_CLEAR _ULL(0xc)
+#define KVX_SFR_PSOW_L2E_WFXM_SET _ULL(0xc00000000)
+
+#define KVX_SFR_PSOW_SME_MASK _ULL(0x3000000000) /* Step Mode Enabled owner */
+#define KVX_SFR_PSOW_SME_SHIFT 36
+#define KVX_SFR_PSOW_SME_WIDTH 2
+#define KVX_SFR_PSOW_SME_WFXM_MASK _ULL(0x3000000000)
+#define KVX_SFR_PSOW_SME_WFXM_CLEAR _ULL(0x30)
+#define KVX_SFR_PSOW_SME_WFXM_SET _ULL(0x3000000000)
+
+#define KVX_SFR_PSOW_SMR_MASK _ULL(0xc000000000) /* Step Mode Ready owner */
+#define KVX_SFR_PSOW_SMR_SHIFT 38
+#define KVX_SFR_PSOW_SMR_WIDTH 2
+#define KVX_SFR_PSOW_SMR_WFXM_MASK _ULL(0xc000000000)
+#define KVX_SFR_PSOW_SMR_WFXM_CLEAR _ULL(0xc0)
+#define KVX_SFR_PSOW_SMR_WFXM_SET _ULL(0xc000000000)
+
+#define KVX_SFR_PSOW_PMJ0_MASK _ULL(0x30000000000) /* Page Mask in JTLB bit 0 owner */
+#define KVX_SFR_PSOW_PMJ0_SHIFT 40
+#define KVX_SFR_PSOW_PMJ0_WIDTH 2
+#define KVX_SFR_PSOW_PMJ0_WFXM_MASK _ULL(0x30000000000)
+#define KVX_SFR_PSOW_PMJ0_WFXM_CLEAR _ULL(0x300)
+#define KVX_SFR_PSOW_PMJ0_WFXM_SET _ULL(0x30000000000)
+
+#define KVX_SFR_PSOW_PMJ1_MASK _ULL(0xc0000000000) /* Page Mask in JTLB bit 1 owner */
+#define KVX_SFR_PSOW_PMJ1_SHIFT 42
+#define KVX_SFR_PSOW_PMJ1_WIDTH 2
+#define KVX_SFR_PSOW_PMJ1_WFXM_MASK _ULL(0xc0000000000)
+#define KVX_SFR_PSOW_PMJ1_WFXM_CLEAR _ULL(0xc00)
+#define KVX_SFR_PSOW_PMJ1_WFXM_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_PSOW_PMJ2_MASK _ULL(0x300000000000) /* Page Mask in JTLB bit 2 owner */
+#define KVX_SFR_PSOW_PMJ2_SHIFT 44
+#define KVX_SFR_PSOW_PMJ2_WIDTH 2
+#define KVX_SFR_PSOW_PMJ2_WFXM_MASK _ULL(0x300000000000)
+#define KVX_SFR_PSOW_PMJ2_WFXM_CLEAR _ULL(0x3000)
+#define KVX_SFR_PSOW_PMJ2_WFXM_SET _ULL(0x300000000000)
+
+#define KVX_SFR_PSOW_PMJ3_MASK _ULL(0xc00000000000) /* Page Mask in JTLB bit 3 owner */
+#define KVX_SFR_PSOW_PMJ3_SHIFT 46
+#define KVX_SFR_PSOW_PMJ3_WIDTH 2
+#define KVX_SFR_PSOW_PMJ3_WFXM_MASK _ULL(0xc00000000000)
+#define KVX_SFR_PSOW_PMJ3_WFXM_CLEAR _ULL(0xc000)
+#define KVX_SFR_PSOW_PMJ3_WFXM_SET _ULL(0xc00000000000)
+
+#define KVX_SFR_PSOW_MMUP_MASK _ULL(0x3000000000000) /* Privileged on MMU owner. */
+#define KVX_SFR_PSOW_MMUP_SHIFT 48
+#define KVX_SFR_PSOW_MMUP_WIDTH 2
+#define KVX_SFR_PSOW_MMUP_WFXM_MASK _ULL(0x3000000000000)
+#define KVX_SFR_PSOW_MMUP_WFXM_CLEAR _ULL(0x30000)
+#define KVX_SFR_PSOW_MMUP_WFXM_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_CS_IC_MASK _ULL(0x1) /* Integer Carry */
+#define KVX_SFR_CS_IC_SHIFT 0
+#define KVX_SFR_CS_IC_WIDTH 1
+#define KVX_SFR_CS_IC_WFXL_MASK _ULL(0x1)
+#define KVX_SFR_CS_IC_WFXL_CLEAR _ULL(0x1)
+#define KVX_SFR_CS_IC_WFXL_SET _ULL(0x100000000)
+
+#define KVX_SFR_CS_IO_MASK _ULL(0x2) /* IEEE 754 Invalid Operation */
+#define KVX_SFR_CS_IO_SHIFT 1
+#define KVX_SFR_CS_IO_WIDTH 1
+#define KVX_SFR_CS_IO_WFXL_MASK _ULL(0x2)
+#define KVX_SFR_CS_IO_WFXL_CLEAR _ULL(0x2)
+#define KVX_SFR_CS_IO_WFXL_SET _ULL(0x200000000)
+
+#define KVX_SFR_CS_DZ_MASK _ULL(0x4) /* IEEE 754 Divide by Zero */
+#define KVX_SFR_CS_DZ_SHIFT 2
+#define KVX_SFR_CS_DZ_WIDTH 1
+#define KVX_SFR_CS_DZ_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_CS_DZ_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_CS_DZ_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_CS_OV_MASK _ULL(0x8) /* IEEE 754 Overflow */
+#define KVX_SFR_CS_OV_SHIFT 3
+#define KVX_SFR_CS_OV_WIDTH 1
+#define KVX_SFR_CS_OV_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_CS_OV_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_CS_OV_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_CS_UN_MASK _ULL(0x10) /* IEEE 754 Underflow */
+#define KVX_SFR_CS_UN_SHIFT 4
+#define KVX_SFR_CS_UN_WIDTH 1
+#define KVX_SFR_CS_UN_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_CS_UN_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_CS_UN_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_CS_IN_MASK _ULL(0x20) /* IEEE 754 Inexact */
+#define KVX_SFR_CS_IN_SHIFT 5
+#define KVX_SFR_CS_IN_WIDTH 1
+#define KVX_SFR_CS_IN_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_CS_IN_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_CS_IN_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_CS_XIO_MASK _ULL(0x200) /* Extension IEEE 754 Invalid Operation */
+#define KVX_SFR_CS_XIO_SHIFT 9
+#define KVX_SFR_CS_XIO_WIDTH 1
+#define KVX_SFR_CS_XIO_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_CS_XIO_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_CS_XIO_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_CS_XDZ_MASK _ULL(0x400) /* Extension IEEE 754 Divide by Zero */
+#define KVX_SFR_CS_XDZ_SHIFT 10
+#define KVX_SFR_CS_XDZ_WIDTH 1
+#define KVX_SFR_CS_XDZ_WFXL_MASK _ULL(0x400)
+#define KVX_SFR_CS_XDZ_WFXL_CLEAR _ULL(0x400)
+#define KVX_SFR_CS_XDZ_WFXL_SET _ULL(0x40000000000)
+
+#define KVX_SFR_CS_XOV_MASK _ULL(0x800) /* Extension IEEE 754 Overflow */
+#define KVX_SFR_CS_XOV_SHIFT 11
+#define KVX_SFR_CS_XOV_WIDTH 1
+#define KVX_SFR_CS_XOV_WFXL_MASK _ULL(0x800)
+#define KVX_SFR_CS_XOV_WFXL_CLEAR _ULL(0x800)
+#define KVX_SFR_CS_XOV_WFXL_SET _ULL(0x80000000000)
+
+#define KVX_SFR_CS_XUN_MASK _ULL(0x1000) /* Extension IEEE 754 Underflow */
+#define KVX_SFR_CS_XUN_SHIFT 12
+#define KVX_SFR_CS_XUN_WIDTH 1
+#define KVX_SFR_CS_XUN_WFXL_MASK _ULL(0x1000)
+#define KVX_SFR_CS_XUN_WFXL_CLEAR _ULL(0x1000)
+#define KVX_SFR_CS_XUN_WFXL_SET _ULL(0x100000000000)
+
+#define KVX_SFR_CS_XIN_MASK _ULL(0x2000) /* Extension IEEE 754 Inexact */
+#define KVX_SFR_CS_XIN_SHIFT 13
+#define KVX_SFR_CS_XIN_WIDTH 1
+#define KVX_SFR_CS_XIN_WFXL_MASK _ULL(0x2000)
+#define KVX_SFR_CS_XIN_WFXL_CLEAR _ULL(0x2000)
+#define KVX_SFR_CS_XIN_WFXL_SET _ULL(0x200000000000)
+
+#define KVX_SFR_CS_RM_MASK _ULL(0x30000) /* IEEE 754 Rounding Mode */
+#define KVX_SFR_CS_RM_SHIFT 16
+#define KVX_SFR_CS_RM_WIDTH 2
+#define KVX_SFR_CS_RM_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_CS_RM_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_CS_RM_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_CS_XRM_MASK _ULL(0x300000) /* Extension IEEE 754 Rounding Mode */
+#define KVX_SFR_CS_XRM_SHIFT 20
+#define KVX_SFR_CS_XRM_WIDTH 2
+#define KVX_SFR_CS_XRM_WFXL_MASK _ULL(0x300000)
+#define KVX_SFR_CS_XRM_WFXL_CLEAR _ULL(0x300000)
+#define KVX_SFR_CS_XRM_WFXL_SET _ULL(0x30000000000000)
+
+#define KVX_SFR_CS_XMF_MASK _ULL(0x1000000) /* eXtension ModiFied */
+#define KVX_SFR_CS_XMF_SHIFT 24
+#define KVX_SFR_CS_XMF_WIDTH 1
+#define KVX_SFR_CS_XMF_WFXL_MASK _ULL(0x1000000)
+#define KVX_SFR_CS_XMF_WFXL_CLEAR _ULL(0x1000000)
+#define KVX_SFR_CS_XMF_WFXL_SET _ULL(0x100000000000000)
+
+#define KVX_SFR_CS_CC_MASK _ULL(0xffff00000000) /* Carry Counter */
+#define KVX_SFR_CS_CC_SHIFT 32
+#define KVX_SFR_CS_CC_WIDTH 16
+#define KVX_SFR_CS_CC_WFXM_MASK _ULL(0xffff00000000)
+#define KVX_SFR_CS_CC_WFXM_CLEAR _ULL(0xffff)
+#define KVX_SFR_CS_CC_WFXM_SET _ULL(0xffff00000000)
+
+#define KVX_SFR_CS_XDROP_MASK _ULL(0x3f000000000000) /* Extension Conversion Drop Bits */
+#define KVX_SFR_CS_XDROP_SHIFT 48
+#define KVX_SFR_CS_XDROP_WIDTH 6
+#define KVX_SFR_CS_XDROP_WFXM_MASK _ULL(0x3f000000000000)
+#define KVX_SFR_CS_XDROP_WFXM_CLEAR _ULL(0x3f0000)
+#define KVX_SFR_CS_XDROP_WFXM_SET _ULL(0x3f000000000000)
+
+#define KVX_SFR_CS_XPOW2_MASK _ULL(0xfc0000000000000) /* Extension FScale Power of Two */
+#define KVX_SFR_CS_XPOW2_SHIFT 54
+#define KVX_SFR_CS_XPOW2_WIDTH 6
+#define KVX_SFR_CS_XPOW2_WFXM_MASK _ULL(0xfc0000000000000)
+#define KVX_SFR_CS_XPOW2_WFXM_CLEAR _ULL(0xfc00000)
+#define KVX_SFR_CS_XPOW2_WFXM_SET _ULL(0xfc0000000000000)
+
+#define KVX_SFR_AESPC_AESPC_MASK _ULL(0xffffffffffffffff) /* Arithmetic Exception Saved PC */
+#define KVX_SFR_AESPC_AESPC_SHIFT 0
+#define KVX_SFR_AESPC_AESPC_WIDTH 64
+#define KVX_SFR_AESPC_AESPC_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_AESPC_AESPC_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_AESPC_AESPC_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_AESPC_AESPC_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_AESPC_AESPC_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_AESPC_AESPC_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_CSIT_ICIE_MASK _ULL(0x1) /* Integer Carry Interrupt Enable */
+#define KVX_SFR_CSIT_ICIE_SHIFT 0
+#define KVX_SFR_CSIT_ICIE_WIDTH 1
+#define KVX_SFR_CSIT_ICIE_WFXL_MASK _ULL(0x1)
+#define KVX_SFR_CSIT_ICIE_WFXL_CLEAR _ULL(0x1)
+#define KVX_SFR_CSIT_ICIE_WFXL_SET _ULL(0x100000000)
+
+#define KVX_SFR_CSIT_IOIE_MASK _ULL(0x2) /* IEEE 754 Invalid Operation Interrupt Enable */
+#define KVX_SFR_CSIT_IOIE_SHIFT 1
+#define KVX_SFR_CSIT_IOIE_WIDTH 1
+#define KVX_SFR_CSIT_IOIE_WFXL_MASK _ULL(0x2)
+#define KVX_SFR_CSIT_IOIE_WFXL_CLEAR _ULL(0x2)
+#define KVX_SFR_CSIT_IOIE_WFXL_SET _ULL(0x200000000)
+
+#define KVX_SFR_CSIT_DZIE_MASK _ULL(0x4) /* IEEE 754 Divide by Zero Interrupt Enable */
+#define KVX_SFR_CSIT_DZIE_SHIFT 2
+#define KVX_SFR_CSIT_DZIE_WIDTH 1
+#define KVX_SFR_CSIT_DZIE_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_CSIT_DZIE_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_CSIT_DZIE_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_CSIT_OVIE_MASK _ULL(0x8) /* IEEE 754 Overflow Interrupt Enable */
+#define KVX_SFR_CSIT_OVIE_SHIFT 3
+#define KVX_SFR_CSIT_OVIE_WIDTH 1
+#define KVX_SFR_CSIT_OVIE_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_CSIT_OVIE_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_CSIT_OVIE_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_CSIT_UNIE_MASK _ULL(0x10) /* IEEE 754 Underflow Interrupt Enable */
+#define KVX_SFR_CSIT_UNIE_SHIFT 4
+#define KVX_SFR_CSIT_UNIE_WIDTH 1
+#define KVX_SFR_CSIT_UNIE_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_CSIT_UNIE_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_CSIT_UNIE_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_CSIT_INIE_MASK _ULL(0x20) /* IEEE 754 Inexact Interrupt Enable */
+#define KVX_SFR_CSIT_INIE_SHIFT 5
+#define KVX_SFR_CSIT_INIE_WIDTH 1
+#define KVX_SFR_CSIT_INIE_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_CSIT_INIE_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_CSIT_INIE_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_CSIT_XIOIE_MASK _ULL(0x200) /* Extension IEEE 754 Invalid Operation Interrupt Enable */
+#define KVX_SFR_CSIT_XIOIE_SHIFT 9
+#define KVX_SFR_CSIT_XIOIE_WIDTH 1
+#define KVX_SFR_CSIT_XIOIE_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_CSIT_XIOIE_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_CSIT_XIOIE_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_CSIT_XDZIE_MASK _ULL(0x400) /* Extension IEEE 754 Divide by Zero Interrupt Enable */
+#define KVX_SFR_CSIT_XDZIE_SHIFT 10
+#define KVX_SFR_CSIT_XDZIE_WIDTH 1
+#define KVX_SFR_CSIT_XDZIE_WFXL_MASK _ULL(0x400)
+#define KVX_SFR_CSIT_XDZIE_WFXL_CLEAR _ULL(0x400)
+#define KVX_SFR_CSIT_XDZIE_WFXL_SET _ULL(0x40000000000)
+
+#define KVX_SFR_CSIT_XOVIE_MASK _ULL(0x800) /* Extension IEEE 754 Overflow Interrupt Enable */
+#define KVX_SFR_CSIT_XOVIE_SHIFT 11
+#define KVX_SFR_CSIT_XOVIE_WIDTH 1
+#define KVX_SFR_CSIT_XOVIE_WFXL_MASK _ULL(0x800)
+#define KVX_SFR_CSIT_XOVIE_WFXL_CLEAR _ULL(0x800)
+#define KVX_SFR_CSIT_XOVIE_WFXL_SET _ULL(0x80000000000)
+
+#define KVX_SFR_CSIT_XUNIE_MASK _ULL(0x1000) /* Extension IEEE 754 Underflow Interrupt Enable */
+#define KVX_SFR_CSIT_XUNIE_SHIFT 12
+#define KVX_SFR_CSIT_XUNIE_WIDTH 1
+#define KVX_SFR_CSIT_XUNIE_WFXL_MASK _ULL(0x1000)
+#define KVX_SFR_CSIT_XUNIE_WFXL_CLEAR _ULL(0x1000)
+#define KVX_SFR_CSIT_XUNIE_WFXL_SET _ULL(0x100000000000)
+
+#define KVX_SFR_CSIT_XINIE_MASK _ULL(0x2000) /* Extension IEEE 754 Inexact Interrupt Enable */
+#define KVX_SFR_CSIT_XINIE_SHIFT 13
+#define KVX_SFR_CSIT_XINIE_WIDTH 1
+#define KVX_SFR_CSIT_XINIE_WFXL_MASK _ULL(0x2000)
+#define KVX_SFR_CSIT_XINIE_WFXL_CLEAR _ULL(0x2000)
+#define KVX_SFR_CSIT_XINIE_WFXL_SET _ULL(0x200000000000)
+
+#define KVX_SFR_CSIT_AEIR_MASK _ULL(0x10000) /* Arithmetic Exception Interrupt Raised */
+#define KVX_SFR_CSIT_AEIR_SHIFT 16
+#define KVX_SFR_CSIT_AEIR_WIDTH 1
+#define KVX_SFR_CSIT_AEIR_WFXL_MASK _ULL(0x10000)
+#define KVX_SFR_CSIT_AEIR_WFXL_CLEAR _ULL(0x10000)
+#define KVX_SFR_CSIT_AEIR_WFXL_SET _ULL(0x1000000000000)
+
+#define KVX_SFR_CSIT_AEC_MASK _ULL(0xe0000) /* Arithmetic Exception Code */
+#define KVX_SFR_CSIT_AEC_SHIFT 17
+#define KVX_SFR_CSIT_AEC_WIDTH 3
+#define KVX_SFR_CSIT_AEC_WFXL_MASK _ULL(0xe0000)
+#define KVX_SFR_CSIT_AEC_WFXL_CLEAR _ULL(0xe0000)
+#define KVX_SFR_CSIT_AEC_WFXL_SET _ULL(0xe000000000000)
+
+#define KVX_SFR_CSIT_SPCV_MASK _ULL(0x100000) /* SPC Valid */
+#define KVX_SFR_CSIT_SPCV_SHIFT 20
+#define KVX_SFR_CSIT_SPCV_WIDTH 1
+#define KVX_SFR_CSIT_SPCV_WFXL_MASK _ULL(0x100000)
+#define KVX_SFR_CSIT_SPCV_WFXL_CLEAR _ULL(0x100000)
+#define KVX_SFR_CSIT_SPCV_WFXL_SET _ULL(0x10000000000000)
+
+#define KVX_SFR_ES_EC_MASK _ULL(0xf) /* Exception Class */
+#define KVX_SFR_ES_EC_SHIFT 0
+#define KVX_SFR_ES_EC_WIDTH 4
+#define KVX_SFR_ES_EC_WFXL_MASK _ULL(0xf)
+#define KVX_SFR_ES_EC_WFXL_CLEAR _ULL(0xf)
+#define KVX_SFR_ES_EC_WFXL_SET _ULL(0xf00000000)
+
+#define KVX_SFR_ES_ED_MASK _ULL(0xfffffffffffffff0) /* Exception Details */
+#define KVX_SFR_ES_ED_SHIFT 4
+#define KVX_SFR_ES_ED_WIDTH 60
+#define KVX_SFR_ES_ED_WFXL_MASK _ULL(0xfffffff0)
+#define KVX_SFR_ES_ED_WFXL_CLEAR _ULL(0xfffffff0)
+#define KVX_SFR_ES_ED_WFXL_SET _ULL(0xfffffff000000000)
+#define KVX_SFR_ES_ED_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_ES_ED_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_ES_ED_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_ES_OAPL_MASK _ULL(0x30) /* Origin Absolute PL */
+#define KVX_SFR_ES_OAPL_SHIFT 4
+#define KVX_SFR_ES_OAPL_WIDTH 2
+#define KVX_SFR_ES_OAPL_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_ES_OAPL_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_ES_OAPL_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ES_ORPL_MASK _ULL(0xc0) /* Origin Relative PL */
+#define KVX_SFR_ES_ORPL_SHIFT 6
+#define KVX_SFR_ES_ORPL_WIDTH 2
+#define KVX_SFR_ES_ORPL_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_ES_ORPL_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_ES_ORPL_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ES_PTAPL_MASK _ULL(0x300) /* Primary Target Absolute PL */
+#define KVX_SFR_ES_PTAPL_SHIFT 8
+#define KVX_SFR_ES_PTAPL_WIDTH 2
+#define KVX_SFR_ES_PTAPL_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_ES_PTAPL_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_ES_PTAPL_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ES_PTRPL_MASK _ULL(0xc00) /* Primary Target Relative PL */
+#define KVX_SFR_ES_PTRPL_SHIFT 10
+#define KVX_SFR_ES_PTRPL_WIDTH 2
+#define KVX_SFR_ES_PTRPL_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_ES_PTRPL_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_ES_PTRPL_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ES_ITN_MASK _ULL(0x1f000) /* InTerrupt Number */
+#define KVX_SFR_ES_ITN_SHIFT 12
+#define KVX_SFR_ES_ITN_WIDTH 5
+#define KVX_SFR_ES_ITN_WFXL_MASK _ULL(0x1f000)
+#define KVX_SFR_ES_ITN_WFXL_CLEAR _ULL(0x1f000)
+#define KVX_SFR_ES_ITN_WFXL_SET _ULL(0x1f00000000000)
+
+#define KVX_SFR_ES_ITL_MASK _ULL(0x60000) /* InTerrupt Level */
+#define KVX_SFR_ES_ITL_SHIFT 17
+#define KVX_SFR_ES_ITL_WIDTH 2
+#define KVX_SFR_ES_ITL_WFXL_MASK _ULL(0x60000)
+#define KVX_SFR_ES_ITL_WFXL_CLEAR _ULL(0x60000)
+#define KVX_SFR_ES_ITL_WFXL_SET _ULL(0x6000000000000)
+
+#define KVX_SFR_ES_ITI_MASK _ULL(0x1ff80000) /* InTerrupt Info */
+#define KVX_SFR_ES_ITI_SHIFT 19
+#define KVX_SFR_ES_ITI_WIDTH 10
+#define KVX_SFR_ES_ITI_WFXL_MASK _ULL(0x1ff80000)
+#define KVX_SFR_ES_ITI_WFXL_CLEAR _ULL(0x1ff80000)
+#define KVX_SFR_ES_ITI_WFXL_SET _ULL(0x1ff8000000000000)
+
+#define KVX_SFR_ES_SN_MASK _ULL(0xfff000) /* Syscall Number */
+#define KVX_SFR_ES_SN_SHIFT 12
+#define KVX_SFR_ES_SN_WIDTH 12
+#define KVX_SFR_ES_SN_WFXL_MASK _ULL(0xfff000)
+#define KVX_SFR_ES_SN_WFXL_CLEAR _ULL(0xfff000)
+#define KVX_SFR_ES_SN_WFXL_SET _ULL(0xfff00000000000)
+
+#define KVX_SFR_ES_HTC_MASK _ULL(0x1f000) /* Hardware Trap Cause */
+#define KVX_SFR_ES_HTC_SHIFT 12
+#define KVX_SFR_ES_HTC_WIDTH 5
+#define KVX_SFR_ES_HTC_WFXL_MASK _ULL(0x1f000)
+#define KVX_SFR_ES_HTC_WFXL_CLEAR _ULL(0x1f000)
+#define KVX_SFR_ES_HTC_WFXL_SET _ULL(0x1f00000000000)
+
+#define KVX_SFR_ES_SFRT_MASK _ULL(0x20000) /* SFR Trap */
+#define KVX_SFR_ES_SFRT_SHIFT 17
+#define KVX_SFR_ES_SFRT_WIDTH 1
+#define KVX_SFR_ES_SFRT_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_ES_SFRT_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_ES_SFRT_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_ES_SFRI_MASK _ULL(0x1c0000) /* SFR Instruction */
+#define KVX_SFR_ES_SFRI_SHIFT 18
+#define KVX_SFR_ES_SFRI_WIDTH 3
+#define KVX_SFR_ES_SFRI_WFXL_MASK _ULL(0x1c0000)
+#define KVX_SFR_ES_SFRI_WFXL_CLEAR _ULL(0x1c0000)
+#define KVX_SFR_ES_SFRI_WFXL_SET _ULL(0x1c000000000000)
+
+#define KVX_SFR_ES_GPRP_MASK _ULL(0x7e00000) /* GPR Pointer */
+#define KVX_SFR_ES_GPRP_SHIFT 21
+#define KVX_SFR_ES_GPRP_WIDTH 6
+#define KVX_SFR_ES_GPRP_WFXL_MASK _ULL(0x7e00000)
+#define KVX_SFR_ES_GPRP_WFXL_CLEAR _ULL(0x7e00000)
+#define KVX_SFR_ES_GPRP_WFXL_SET _ULL(0x7e0000000000000)
+
+#define KVX_SFR_ES_SFRP_MASK _ULL(0xff8000000) /* SFR Pointer */
+#define KVX_SFR_ES_SFRP_SHIFT 27
+#define KVX_SFR_ES_SFRP_WIDTH 9
+#define KVX_SFR_ES_SFRP_WFXL_MASK _ULL(0xf8000000)
+#define KVX_SFR_ES_SFRP_WFXL_CLEAR _ULL(0xf8000000)
+#define KVX_SFR_ES_SFRP_WFXL_SET _ULL(0xf800000000000000)
+#define KVX_SFR_ES_SFRP_WFXM_MASK _ULL(0xf00000000)
+#define KVX_SFR_ES_SFRP_WFXM_CLEAR _ULL(0xf)
+#define KVX_SFR_ES_SFRP_WFXM_SET _ULL(0xf00000000)
+
+#define KVX_SFR_ES_DHT_MASK _ULL(0x1000000000) /* Disabled Hardware Trap */
+#define KVX_SFR_ES_DHT_SHIFT 36
+#define KVX_SFR_ES_DHT_WIDTH 1
+#define KVX_SFR_ES_DHT_WFXM_MASK _ULL(0x1000000000)
+#define KVX_SFR_ES_DHT_WFXM_CLEAR _ULL(0x10)
+#define KVX_SFR_ES_DHT_WFXM_SET _ULL(0x1000000000)
+
+#define KVX_SFR_ES_RWX_MASK _ULL(0x38000000000) /* Read Write Execute */
+#define KVX_SFR_ES_RWX_SHIFT 39
+#define KVX_SFR_ES_RWX_WIDTH 3
+#define KVX_SFR_ES_RWX_WFXM_MASK _ULL(0x38000000000)
+#define KVX_SFR_ES_RWX_WFXM_CLEAR _ULL(0x380)
+#define KVX_SFR_ES_RWX_WFXM_SET _ULL(0x38000000000)
+
+#define KVX_SFR_ES_NTA_MASK _ULL(0x40000000000) /* Non-Trapping Access */
+#define KVX_SFR_ES_NTA_SHIFT 42
+#define KVX_SFR_ES_NTA_WIDTH 1
+#define KVX_SFR_ES_NTA_WFXM_MASK _ULL(0x40000000000)
+#define KVX_SFR_ES_NTA_WFXM_CLEAR _ULL(0x400)
+#define KVX_SFR_ES_NTA_WFXM_SET _ULL(0x40000000000)
+
+#define KVX_SFR_ES_UCA_MASK _ULL(0x80000000000) /* Un-Cached Access */
+#define KVX_SFR_ES_UCA_SHIFT 43
+#define KVX_SFR_ES_UCA_WIDTH 1
+#define KVX_SFR_ES_UCA_WFXM_MASK _ULL(0x80000000000)
+#define KVX_SFR_ES_UCA_WFXM_CLEAR _ULL(0x800)
+#define KVX_SFR_ES_UCA_WFXM_SET _ULL(0x80000000000)
+
+#define KVX_SFR_ES_AS_MASK _ULL(0x3f00000000000) /* Access Size */
+#define KVX_SFR_ES_AS_SHIFT 44
+#define KVX_SFR_ES_AS_WIDTH 6
+#define KVX_SFR_ES_AS_WFXM_MASK _ULL(0x3f00000000000)
+#define KVX_SFR_ES_AS_WFXM_CLEAR _ULL(0x3f000)
+#define KVX_SFR_ES_AS_WFXM_SET _ULL(0x3f00000000000)
+
+#define KVX_SFR_ES_BS_MASK _ULL(0x3c000000000000) /* Bundle Size */
+#define KVX_SFR_ES_BS_SHIFT 50
+#define KVX_SFR_ES_BS_WIDTH 4
+#define KVX_SFR_ES_BS_WFXM_MASK _ULL(0x3c000000000000)
+#define KVX_SFR_ES_BS_WFXM_CLEAR _ULL(0x3c0000)
+#define KVX_SFR_ES_BS_WFXM_SET _ULL(0x3c000000000000)
+
+#define KVX_SFR_ES_DRI_MASK _ULL(0xfc0000000000000) /* Data Register Index */
+#define KVX_SFR_ES_DRI_SHIFT 54
+#define KVX_SFR_ES_DRI_WIDTH 6
+#define KVX_SFR_ES_DRI_WFXM_MASK _ULL(0xfc0000000000000)
+#define KVX_SFR_ES_DRI_WFXM_CLEAR _ULL(0xfc00000)
+#define KVX_SFR_ES_DRI_WFXM_SET _ULL(0xfc0000000000000)
+
+#define KVX_SFR_ES_PIC_MASK _ULL(0xf000000000000000) /* Privileged Instruction Code */
+#define KVX_SFR_ES_PIC_SHIFT 60
+#define KVX_SFR_ES_PIC_WIDTH 4
+#define KVX_SFR_ES_PIC_WFXM_MASK _ULL(0xf000000000000000)
+#define KVX_SFR_ES_PIC_WFXM_CLEAR _ULL(0xf0000000)
+#define KVX_SFR_ES_PIC_WFXM_SET _ULL(0xf000000000000000)
+
+#define KVX_SFR_ES_DC_MASK _ULL(0x3000) /* Debug Cause */
+#define KVX_SFR_ES_DC_SHIFT 12
+#define KVX_SFR_ES_DC_WIDTH 2
+#define KVX_SFR_ES_DC_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_ES_DC_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_ES_DC_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ES_BN_MASK _ULL(0x4000) /* Breakpoint Number */
+#define KVX_SFR_ES_BN_SHIFT 14
+#define KVX_SFR_ES_BN_WIDTH 1
+#define KVX_SFR_ES_BN_WFXL_MASK _ULL(0x4000)
+#define KVX_SFR_ES_BN_WFXL_CLEAR _ULL(0x4000)
+#define KVX_SFR_ES_BN_WFXL_SET _ULL(0x400000000000)
+
+#define KVX_SFR_ES_WN_MASK _ULL(0x8000) /* Watchpoint Number */
+#define KVX_SFR_ES_WN_SHIFT 15
+#define KVX_SFR_ES_WN_WIDTH 1
+#define KVX_SFR_ES_WN_WFXL_MASK _ULL(0x8000)
+#define KVX_SFR_ES_WN_WFXL_CLEAR _ULL(0x8000)
+#define KVX_SFR_ES_WN_WFXL_SET _ULL(0x800000000000)
+
+#define KVX_SFR_ES_PL0_EC_MASK _ULL(0xf) /* Exception Class */
+#define KVX_SFR_ES_PL0_EC_SHIFT 0
+#define KVX_SFR_ES_PL0_EC_WIDTH 4
+#define KVX_SFR_ES_PL0_EC_WFXL_MASK _ULL(0xf)
+#define KVX_SFR_ES_PL0_EC_WFXL_CLEAR _ULL(0xf)
+#define KVX_SFR_ES_PL0_EC_WFXL_SET _ULL(0xf00000000)
+
+#define KVX_SFR_ES_PL0_ED_MASK _ULL(0xfffffffffffffff0) /* Exception Details */
+#define KVX_SFR_ES_PL0_ED_SHIFT 4
+#define KVX_SFR_ES_PL0_ED_WIDTH 60
+#define KVX_SFR_ES_PL0_ED_WFXL_MASK _ULL(0xfffffff0)
+#define KVX_SFR_ES_PL0_ED_WFXL_CLEAR _ULL(0xfffffff0)
+#define KVX_SFR_ES_PL0_ED_WFXL_SET _ULL(0xfffffff000000000)
+#define KVX_SFR_ES_PL0_ED_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_ES_PL0_ED_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_ES_PL0_ED_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_ES_PL0_OAPL_MASK _ULL(0x30) /* Origin Absolute PL */
+#define KVX_SFR_ES_PL0_OAPL_SHIFT 4
+#define KVX_SFR_ES_PL0_OAPL_WIDTH 2
+#define KVX_SFR_ES_PL0_OAPL_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_ES_PL0_OAPL_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_ES_PL0_OAPL_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ES_PL0_ORPL_MASK _ULL(0xc0) /* Origin Relative PL */
+#define KVX_SFR_ES_PL0_ORPL_SHIFT 6
+#define KVX_SFR_ES_PL0_ORPL_WIDTH 2
+#define KVX_SFR_ES_PL0_ORPL_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_ES_PL0_ORPL_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_ES_PL0_ORPL_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ES_PL0_PTAPL_MASK _ULL(0x300) /* Target Absolute PL */
+#define KVX_SFR_ES_PL0_PTAPL_SHIFT 8
+#define KVX_SFR_ES_PL0_PTAPL_WIDTH 2
+#define KVX_SFR_ES_PL0_PTAPL_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_ES_PL0_PTAPL_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_ES_PL0_PTAPL_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ES_PL0_PTRPL_MASK _ULL(0xc00) /* Target Relative PL */
+#define KVX_SFR_ES_PL0_PTRPL_SHIFT 10
+#define KVX_SFR_ES_PL0_PTRPL_WIDTH 2
+#define KVX_SFR_ES_PL0_PTRPL_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_ES_PL0_PTRPL_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_ES_PL0_PTRPL_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ES_PL0_ITN_MASK _ULL(0x1f000) /* InTerrupt Number */
+#define KVX_SFR_ES_PL0_ITN_SHIFT 12
+#define KVX_SFR_ES_PL0_ITN_WIDTH 5
+#define KVX_SFR_ES_PL0_ITN_WFXL_MASK _ULL(0x1f000)
+#define KVX_SFR_ES_PL0_ITN_WFXL_CLEAR _ULL(0x1f000)
+#define KVX_SFR_ES_PL0_ITN_WFXL_SET _ULL(0x1f00000000000)
+
+#define KVX_SFR_ES_PL0_ITL_MASK _ULL(0x60000) /* InTerrupt Level */
+#define KVX_SFR_ES_PL0_ITL_SHIFT 17
+#define KVX_SFR_ES_PL0_ITL_WIDTH 2
+#define KVX_SFR_ES_PL0_ITL_WFXL_MASK _ULL(0x60000)
+#define KVX_SFR_ES_PL0_ITL_WFXL_CLEAR _ULL(0x60000)
+#define KVX_SFR_ES_PL0_ITL_WFXL_SET _ULL(0x6000000000000)
+
+#define KVX_SFR_ES_PL0_ITI_MASK _ULL(0x1ff80000) /* InTerrupt Info */
+#define KVX_SFR_ES_PL0_ITI_SHIFT 19
+#define KVX_SFR_ES_PL0_ITI_WIDTH 10
+#define KVX_SFR_ES_PL0_ITI_WFXL_MASK _ULL(0x1ff80000)
+#define KVX_SFR_ES_PL0_ITI_WFXL_CLEAR _ULL(0x1ff80000)
+#define KVX_SFR_ES_PL0_ITI_WFXL_SET _ULL(0x1ff8000000000000)
+
+#define KVX_SFR_ES_PL0_SN_MASK _ULL(0xfff000) /* Syscall Number */
+#define KVX_SFR_ES_PL0_SN_SHIFT 12
+#define KVX_SFR_ES_PL0_SN_WIDTH 12
+#define KVX_SFR_ES_PL0_SN_WFXL_MASK _ULL(0xfff000)
+#define KVX_SFR_ES_PL0_SN_WFXL_CLEAR _ULL(0xfff000)
+#define KVX_SFR_ES_PL0_SN_WFXL_SET _ULL(0xfff00000000000)
+
+#define KVX_SFR_ES_PL0_HTC_MASK _ULL(0x1f000) /* Hardware Trap Cause */
+#define KVX_SFR_ES_PL0_HTC_SHIFT 12
+#define KVX_SFR_ES_PL0_HTC_WIDTH 5
+#define KVX_SFR_ES_PL0_HTC_WFXL_MASK _ULL(0x1f000)
+#define KVX_SFR_ES_PL0_HTC_WFXL_CLEAR _ULL(0x1f000)
+#define KVX_SFR_ES_PL0_HTC_WFXL_SET _ULL(0x1f00000000000)
+
+#define KVX_SFR_ES_PL0_SFRT_MASK _ULL(0x20000) /* SFR Trap */
+#define KVX_SFR_ES_PL0_SFRT_SHIFT 17
+#define KVX_SFR_ES_PL0_SFRT_WIDTH 1
+#define KVX_SFR_ES_PL0_SFRT_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_ES_PL0_SFRT_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_ES_PL0_SFRT_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_ES_PL0_SFRI_MASK _ULL(0x1c0000) /* SFR Instruction */
+#define KVX_SFR_ES_PL0_SFRI_SHIFT 18
+#define KVX_SFR_ES_PL0_SFRI_WIDTH 3
+#define KVX_SFR_ES_PL0_SFRI_WFXL_MASK _ULL(0x1c0000)
+#define KVX_SFR_ES_PL0_SFRI_WFXL_CLEAR _ULL(0x1c0000)
+#define KVX_SFR_ES_PL0_SFRI_WFXL_SET _ULL(0x1c000000000000)
+
+#define KVX_SFR_ES_PL0_GPRP_MASK _ULL(0x7e00000) /* GPR Pointer */
+#define KVX_SFR_ES_PL0_GPRP_SHIFT 21
+#define KVX_SFR_ES_PL0_GPRP_WIDTH 6
+#define KVX_SFR_ES_PL0_GPRP_WFXL_MASK _ULL(0x7e00000)
+#define KVX_SFR_ES_PL0_GPRP_WFXL_CLEAR _ULL(0x7e00000)
+#define KVX_SFR_ES_PL0_GPRP_WFXL_SET _ULL(0x7e0000000000000)
+
+#define KVX_SFR_ES_PL0_SFRP_MASK _ULL(0xff8000000) /* SFR Pointer */
+#define KVX_SFR_ES_PL0_SFRP_SHIFT 27
+#define KVX_SFR_ES_PL0_SFRP_WIDTH 9
+#define KVX_SFR_ES_PL0_SFRP_WFXL_MASK _ULL(0xf8000000)
+#define KVX_SFR_ES_PL0_SFRP_WFXL_CLEAR _ULL(0xf8000000)
+#define KVX_SFR_ES_PL0_SFRP_WFXL_SET _ULL(0xf800000000000000)
+#define KVX_SFR_ES_PL0_SFRP_WFXM_MASK _ULL(0xf00000000)
+#define KVX_SFR_ES_PL0_SFRP_WFXM_CLEAR _ULL(0xf)
+#define KVX_SFR_ES_PL0_SFRP_WFXM_SET _ULL(0xf00000000)
+
+#define KVX_SFR_ES_PL0_DHT_MASK _ULL(0x1000000000) /* Disabled Hardware Trap */
+#define KVX_SFR_ES_PL0_DHT_SHIFT 36
+#define KVX_SFR_ES_PL0_DHT_WIDTH 1
+#define KVX_SFR_ES_PL0_DHT_WFXM_MASK _ULL(0x1000000000)
+#define KVX_SFR_ES_PL0_DHT_WFXM_CLEAR _ULL(0x10)
+#define KVX_SFR_ES_PL0_DHT_WFXM_SET _ULL(0x1000000000)
+
+#define KVX_SFR_ES_PL0_RWX_MASK _ULL(0x38000000000) /* Read Write Execute */
+#define KVX_SFR_ES_PL0_RWX_SHIFT 39
+#define KVX_SFR_ES_PL0_RWX_WIDTH 3
+#define KVX_SFR_ES_PL0_RWX_WFXM_MASK _ULL(0x38000000000)
+#define KVX_SFR_ES_PL0_RWX_WFXM_CLEAR _ULL(0x380)
+#define KVX_SFR_ES_PL0_RWX_WFXM_SET _ULL(0x38000000000)
+
+#define KVX_SFR_ES_PL0_NTA_MASK _ULL(0x40000000000) /* Non-Trapping Access */
+#define KVX_SFR_ES_PL0_NTA_SHIFT 42
+#define KVX_SFR_ES_PL0_NTA_WIDTH 1
+#define KVX_SFR_ES_PL0_NTA_WFXM_MASK _ULL(0x40000000000)
+#define KVX_SFR_ES_PL0_NTA_WFXM_CLEAR _ULL(0x400)
+#define KVX_SFR_ES_PL0_NTA_WFXM_SET _ULL(0x40000000000)
+
+#define KVX_SFR_ES_PL0_UCA_MASK _ULL(0x80000000000) /* Un-Cached Access */
+#define KVX_SFR_ES_PL0_UCA_SHIFT 43
+#define KVX_SFR_ES_PL0_UCA_WIDTH 1
+#define KVX_SFR_ES_PL0_UCA_WFXM_MASK _ULL(0x80000000000)
+#define KVX_SFR_ES_PL0_UCA_WFXM_CLEAR _ULL(0x800)
+#define KVX_SFR_ES_PL0_UCA_WFXM_SET _ULL(0x80000000000)
+
+#define KVX_SFR_ES_PL0_AS_MASK _ULL(0x3f00000000000) /* Access Size */
+#define KVX_SFR_ES_PL0_AS_SHIFT 44
+#define KVX_SFR_ES_PL0_AS_WIDTH 6
+#define KVX_SFR_ES_PL0_AS_WFXM_MASK _ULL(0x3f00000000000)
+#define KVX_SFR_ES_PL0_AS_WFXM_CLEAR _ULL(0x3f000)
+#define KVX_SFR_ES_PL0_AS_WFXM_SET _ULL(0x3f00000000000)
+
+#define KVX_SFR_ES_PL0_BS_MASK _ULL(0x3c000000000000) /* Bundle Size */
+#define KVX_SFR_ES_PL0_BS_SHIFT 50
+#define KVX_SFR_ES_PL0_BS_WIDTH 4
+#define KVX_SFR_ES_PL0_BS_WFXM_MASK _ULL(0x3c000000000000)
+#define KVX_SFR_ES_PL0_BS_WFXM_CLEAR _ULL(0x3c0000)
+#define KVX_SFR_ES_PL0_BS_WFXM_SET _ULL(0x3c000000000000)
+
+#define KVX_SFR_ES_PL0_DRI_MASK _ULL(0xfc0000000000000) /* Data Register Index */
+#define KVX_SFR_ES_PL0_DRI_SHIFT 54
+#define KVX_SFR_ES_PL0_DRI_WIDTH 6
+#define KVX_SFR_ES_PL0_DRI_WFXM_MASK _ULL(0xfc0000000000000)
+#define KVX_SFR_ES_PL0_DRI_WFXM_CLEAR _ULL(0xfc00000)
+#define KVX_SFR_ES_PL0_DRI_WFXM_SET _ULL(0xfc0000000000000)
+
+#define KVX_SFR_ES_PL0_PIC_MASK _ULL(0xf000000000000000) /* Privileged Instruction Code */
+#define KVX_SFR_ES_PL0_PIC_SHIFT 60
+#define KVX_SFR_ES_PL0_PIC_WIDTH 4
+#define KVX_SFR_ES_PL0_PIC_WFXM_MASK _ULL(0xf000000000000000)
+#define KVX_SFR_ES_PL0_PIC_WFXM_CLEAR _ULL(0xf0000000)
+#define KVX_SFR_ES_PL0_PIC_WFXM_SET _ULL(0xf000000000000000)
+
+#define KVX_SFR_ES_PL0_DC_MASK _ULL(0x3000) /* Debug Cause */
+#define KVX_SFR_ES_PL0_DC_SHIFT 12
+#define KVX_SFR_ES_PL0_DC_WIDTH 2
+#define KVX_SFR_ES_PL0_DC_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_ES_PL0_DC_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_ES_PL0_DC_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ES_PL0_BN_MASK _ULL(0x4000) /* Breakpoint Number */
+#define KVX_SFR_ES_PL0_BN_SHIFT 14
+#define KVX_SFR_ES_PL0_BN_WIDTH 1
+#define KVX_SFR_ES_PL0_BN_WFXL_MASK _ULL(0x4000)
+#define KVX_SFR_ES_PL0_BN_WFXL_CLEAR _ULL(0x4000)
+#define KVX_SFR_ES_PL0_BN_WFXL_SET _ULL(0x400000000000)
+
+#define KVX_SFR_ES_PL0_WN_MASK _ULL(0x8000) /* Watchpoint Number */
+#define KVX_SFR_ES_PL0_WN_SHIFT 15
+#define KVX_SFR_ES_PL0_WN_WIDTH 1
+#define KVX_SFR_ES_PL0_WN_WFXL_MASK _ULL(0x8000)
+#define KVX_SFR_ES_PL0_WN_WFXL_CLEAR _ULL(0x8000)
+#define KVX_SFR_ES_PL0_WN_WFXL_SET _ULL(0x800000000000)
+
+#define KVX_SFR_ES_PL1_EC_MASK _ULL(0xf) /* Exception Class */
+#define KVX_SFR_ES_PL1_EC_SHIFT 0
+#define KVX_SFR_ES_PL1_EC_WIDTH 4
+#define KVX_SFR_ES_PL1_EC_WFXL_MASK _ULL(0xf)
+#define KVX_SFR_ES_PL1_EC_WFXL_CLEAR _ULL(0xf)
+#define KVX_SFR_ES_PL1_EC_WFXL_SET _ULL(0xf00000000)
+
+#define KVX_SFR_ES_PL1_ED_MASK _ULL(0xfffffffffffffff0) /* Exception Details */
+#define KVX_SFR_ES_PL1_ED_SHIFT 4
+#define KVX_SFR_ES_PL1_ED_WIDTH 60
+#define KVX_SFR_ES_PL1_ED_WFXL_MASK _ULL(0xfffffff0)
+#define KVX_SFR_ES_PL1_ED_WFXL_CLEAR _ULL(0xfffffff0)
+#define KVX_SFR_ES_PL1_ED_WFXL_SET _ULL(0xfffffff000000000)
+#define KVX_SFR_ES_PL1_ED_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_ES_PL1_ED_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_ES_PL1_ED_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_ES_PL1_OAPL_MASK _ULL(0x30) /* Origin Absolute PL */
+#define KVX_SFR_ES_PL1_OAPL_SHIFT 4
+#define KVX_SFR_ES_PL1_OAPL_WIDTH 2
+#define KVX_SFR_ES_PL1_OAPL_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_ES_PL1_OAPL_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_ES_PL1_OAPL_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ES_PL1_ORPL_MASK _ULL(0xc0) /* Origin Relative PL */
+#define KVX_SFR_ES_PL1_ORPL_SHIFT 6
+#define KVX_SFR_ES_PL1_ORPL_WIDTH 2
+#define KVX_SFR_ES_PL1_ORPL_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_ES_PL1_ORPL_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_ES_PL1_ORPL_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ES_PL1_PTAPL_MASK _ULL(0x300) /* Target Absolute PL */
+#define KVX_SFR_ES_PL1_PTAPL_SHIFT 8
+#define KVX_SFR_ES_PL1_PTAPL_WIDTH 2
+#define KVX_SFR_ES_PL1_PTAPL_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_ES_PL1_PTAPL_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_ES_PL1_PTAPL_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ES_PL1_PTRPL_MASK _ULL(0xc00) /* Target Relative PL */
+#define KVX_SFR_ES_PL1_PTRPL_SHIFT 10
+#define KVX_SFR_ES_PL1_PTRPL_WIDTH 2
+#define KVX_SFR_ES_PL1_PTRPL_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_ES_PL1_PTRPL_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_ES_PL1_PTRPL_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ES_PL1_ITN_MASK _ULL(0x1f000) /* InTerrupt Number */
+#define KVX_SFR_ES_PL1_ITN_SHIFT 12
+#define KVX_SFR_ES_PL1_ITN_WIDTH 5
+#define KVX_SFR_ES_PL1_ITN_WFXL_MASK _ULL(0x1f000)
+#define KVX_SFR_ES_PL1_ITN_WFXL_CLEAR _ULL(0x1f000)
+#define KVX_SFR_ES_PL1_ITN_WFXL_SET _ULL(0x1f00000000000)
+
+#define KVX_SFR_ES_PL1_ITL_MASK _ULL(0x60000) /* InTerrupt Level */
+#define KVX_SFR_ES_PL1_ITL_SHIFT 17
+#define KVX_SFR_ES_PL1_ITL_WIDTH 2
+#define KVX_SFR_ES_PL1_ITL_WFXL_MASK _ULL(0x60000)
+#define KVX_SFR_ES_PL1_ITL_WFXL_CLEAR _ULL(0x60000)
+#define KVX_SFR_ES_PL1_ITL_WFXL_SET _ULL(0x6000000000000)
+
+#define KVX_SFR_ES_PL1_ITI_MASK _ULL(0x1ff80000) /* InTerrupt Info */
+#define KVX_SFR_ES_PL1_ITI_SHIFT 19
+#define KVX_SFR_ES_PL1_ITI_WIDTH 10
+#define KVX_SFR_ES_PL1_ITI_WFXL_MASK _ULL(0x1ff80000)
+#define KVX_SFR_ES_PL1_ITI_WFXL_CLEAR _ULL(0x1ff80000)
+#define KVX_SFR_ES_PL1_ITI_WFXL_SET _ULL(0x1ff8000000000000)
+
+#define KVX_SFR_ES_PL1_SN_MASK _ULL(0xfff000) /* Syscall Number */
+#define KVX_SFR_ES_PL1_SN_SHIFT 12
+#define KVX_SFR_ES_PL1_SN_WIDTH 12
+#define KVX_SFR_ES_PL1_SN_WFXL_MASK _ULL(0xfff000)
+#define KVX_SFR_ES_PL1_SN_WFXL_CLEAR _ULL(0xfff000)
+#define KVX_SFR_ES_PL1_SN_WFXL_SET _ULL(0xfff00000000000)
+
+#define KVX_SFR_ES_PL1_HTC_MASK _ULL(0x1f000) /* Hardware Trap Cause */
+#define KVX_SFR_ES_PL1_HTC_SHIFT 12
+#define KVX_SFR_ES_PL1_HTC_WIDTH 5
+#define KVX_SFR_ES_PL1_HTC_WFXL_MASK _ULL(0x1f000)
+#define KVX_SFR_ES_PL1_HTC_WFXL_CLEAR _ULL(0x1f000)
+#define KVX_SFR_ES_PL1_HTC_WFXL_SET _ULL(0x1f00000000000)
+
+#define KVX_SFR_ES_PL1_SFRT_MASK _ULL(0x20000) /* SFR Trap */
+#define KVX_SFR_ES_PL1_SFRT_SHIFT 17
+#define KVX_SFR_ES_PL1_SFRT_WIDTH 1
+#define KVX_SFR_ES_PL1_SFRT_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_ES_PL1_SFRT_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_ES_PL1_SFRT_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_ES_PL1_SFRI_MASK _ULL(0x1c0000) /* SFR Instruction */
+#define KVX_SFR_ES_PL1_SFRI_SHIFT 18
+#define KVX_SFR_ES_PL1_SFRI_WIDTH 3
+#define KVX_SFR_ES_PL1_SFRI_WFXL_MASK _ULL(0x1c0000)
+#define KVX_SFR_ES_PL1_SFRI_WFXL_CLEAR _ULL(0x1c0000)
+#define KVX_SFR_ES_PL1_SFRI_WFXL_SET _ULL(0x1c000000000000)
+
+#define KVX_SFR_ES_PL1_GPRP_MASK _ULL(0x7e00000) /* GPR Pointer */
+#define KVX_SFR_ES_PL1_GPRP_SHIFT 21
+#define KVX_SFR_ES_PL1_GPRP_WIDTH 6
+#define KVX_SFR_ES_PL1_GPRP_WFXL_MASK _ULL(0x7e00000)
+#define KVX_SFR_ES_PL1_GPRP_WFXL_CLEAR _ULL(0x7e00000)
+#define KVX_SFR_ES_PL1_GPRP_WFXL_SET _ULL(0x7e0000000000000)
+
+#define KVX_SFR_ES_PL1_SFRP_MASK _ULL(0xff8000000) /* SFR Pointer */
+#define KVX_SFR_ES_PL1_SFRP_SHIFT 27
+#define KVX_SFR_ES_PL1_SFRP_WIDTH 9
+#define KVX_SFR_ES_PL1_SFRP_WFXL_MASK _ULL(0xf8000000)
+#define KVX_SFR_ES_PL1_SFRP_WFXL_CLEAR _ULL(0xf8000000)
+#define KVX_SFR_ES_PL1_SFRP_WFXL_SET _ULL(0xf800000000000000)
+#define KVX_SFR_ES_PL1_SFRP_WFXM_MASK _ULL(0xf00000000)
+#define KVX_SFR_ES_PL1_SFRP_WFXM_CLEAR _ULL(0xf)
+#define KVX_SFR_ES_PL1_SFRP_WFXM_SET _ULL(0xf00000000)
+
+#define KVX_SFR_ES_PL1_DHT_MASK _ULL(0x1000000000) /* Disabled Hardware Trap */
+#define KVX_SFR_ES_PL1_DHT_SHIFT 36
+#define KVX_SFR_ES_PL1_DHT_WIDTH 1
+#define KVX_SFR_ES_PL1_DHT_WFXM_MASK _ULL(0x1000000000)
+#define KVX_SFR_ES_PL1_DHT_WFXM_CLEAR _ULL(0x10)
+#define KVX_SFR_ES_PL1_DHT_WFXM_SET _ULL(0x1000000000)
+
+#define KVX_SFR_ES_PL1_RWX_MASK _ULL(0x38000000000) /* Read Write Execute */
+#define KVX_SFR_ES_PL1_RWX_SHIFT 39
+#define KVX_SFR_ES_PL1_RWX_WIDTH 3
+#define KVX_SFR_ES_PL1_RWX_WFXM_MASK _ULL(0x38000000000)
+#define KVX_SFR_ES_PL1_RWX_WFXM_CLEAR _ULL(0x380)
+#define KVX_SFR_ES_PL1_RWX_WFXM_SET _ULL(0x38000000000)
+
+#define KVX_SFR_ES_PL1_NTA_MASK _ULL(0x40000000000) /* Non-Trapping Access */
+#define KVX_SFR_ES_PL1_NTA_SHIFT 42
+#define KVX_SFR_ES_PL1_NTA_WIDTH 1
+#define KVX_SFR_ES_PL1_NTA_WFXM_MASK _ULL(0x40000000000)
+#define KVX_SFR_ES_PL1_NTA_WFXM_CLEAR _ULL(0x400)
+#define KVX_SFR_ES_PL1_NTA_WFXM_SET _ULL(0x40000000000)
+
+#define KVX_SFR_ES_PL1_UCA_MASK _ULL(0x80000000000) /* Un-Cached Access */
+#define KVX_SFR_ES_PL1_UCA_SHIFT 43
+#define KVX_SFR_ES_PL1_UCA_WIDTH 1
+#define KVX_SFR_ES_PL1_UCA_WFXM_MASK _ULL(0x80000000000)
+#define KVX_SFR_ES_PL1_UCA_WFXM_CLEAR _ULL(0x800)
+#define KVX_SFR_ES_PL1_UCA_WFXM_SET _ULL(0x80000000000)
+
+#define KVX_SFR_ES_PL1_AS_MASK _ULL(0x3f00000000000) /* Access Size */
+#define KVX_SFR_ES_PL1_AS_SHIFT 44
+#define KVX_SFR_ES_PL1_AS_WIDTH 6
+#define KVX_SFR_ES_PL1_AS_WFXM_MASK _ULL(0x3f00000000000)
+#define KVX_SFR_ES_PL1_AS_WFXM_CLEAR _ULL(0x3f000)
+#define KVX_SFR_ES_PL1_AS_WFXM_SET _ULL(0x3f00000000000)
+
+#define KVX_SFR_ES_PL1_BS_MASK _ULL(0x3c000000000000) /* Bundle Size */
+#define KVX_SFR_ES_PL1_BS_SHIFT 50
+#define KVX_SFR_ES_PL1_BS_WIDTH 4
+#define KVX_SFR_ES_PL1_BS_WFXM_MASK _ULL(0x3c000000000000)
+#define KVX_SFR_ES_PL1_BS_WFXM_CLEAR _ULL(0x3c0000)
+#define KVX_SFR_ES_PL1_BS_WFXM_SET _ULL(0x3c000000000000)
+
+#define KVX_SFR_ES_PL1_DRI_MASK _ULL(0xfc0000000000000) /* Data Register Index */
+#define KVX_SFR_ES_PL1_DRI_SHIFT 54
+#define KVX_SFR_ES_PL1_DRI_WIDTH 6
+#define KVX_SFR_ES_PL1_DRI_WFXM_MASK _ULL(0xfc0000000000000)
+#define KVX_SFR_ES_PL1_DRI_WFXM_CLEAR _ULL(0xfc00000)
+#define KVX_SFR_ES_PL1_DRI_WFXM_SET _ULL(0xfc0000000000000)
+
+#define KVX_SFR_ES_PL1_PIC_MASK _ULL(0xf000000000000000) /* Privileged Instruction Code */
+#define KVX_SFR_ES_PL1_PIC_SHIFT 60
+#define KVX_SFR_ES_PL1_PIC_WIDTH 4
+#define KVX_SFR_ES_PL1_PIC_WFXM_MASK _ULL(0xf000000000000000)
+#define KVX_SFR_ES_PL1_PIC_WFXM_CLEAR _ULL(0xf0000000)
+#define KVX_SFR_ES_PL1_PIC_WFXM_SET _ULL(0xf000000000000000)
+
+#define KVX_SFR_ES_PL1_DC_MASK _ULL(0x3000) /* Debug Cause */
+#define KVX_SFR_ES_PL1_DC_SHIFT 12
+#define KVX_SFR_ES_PL1_DC_WIDTH 2
+#define KVX_SFR_ES_PL1_DC_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_ES_PL1_DC_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_ES_PL1_DC_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ES_PL1_BN_MASK _ULL(0x4000) /* Breakpoint Number */
+#define KVX_SFR_ES_PL1_BN_SHIFT 14
+#define KVX_SFR_ES_PL1_BN_WIDTH 1
+#define KVX_SFR_ES_PL1_BN_WFXL_MASK _ULL(0x4000)
+#define KVX_SFR_ES_PL1_BN_WFXL_CLEAR _ULL(0x4000)
+#define KVX_SFR_ES_PL1_BN_WFXL_SET _ULL(0x400000000000)
+
+#define KVX_SFR_ES_PL1_WN_MASK _ULL(0x8000) /* Watchpoint Number */
+#define KVX_SFR_ES_PL1_WN_SHIFT 15
+#define KVX_SFR_ES_PL1_WN_WIDTH 1
+#define KVX_SFR_ES_PL1_WN_WFXL_MASK _ULL(0x8000)
+#define KVX_SFR_ES_PL1_WN_WFXL_CLEAR _ULL(0x8000)
+#define KVX_SFR_ES_PL1_WN_WFXL_SET _ULL(0x800000000000)
+
+#define KVX_SFR_ES_PL2_EC_MASK _ULL(0xf) /* Exception Class */
+#define KVX_SFR_ES_PL2_EC_SHIFT 0
+#define KVX_SFR_ES_PL2_EC_WIDTH 4
+#define KVX_SFR_ES_PL2_EC_WFXL_MASK _ULL(0xf)
+#define KVX_SFR_ES_PL2_EC_WFXL_CLEAR _ULL(0xf)
+#define KVX_SFR_ES_PL2_EC_WFXL_SET _ULL(0xf00000000)
+
+#define KVX_SFR_ES_PL2_ED_MASK _ULL(0xfffffffffffffff0) /* Exception Details */
+#define KVX_SFR_ES_PL2_ED_SHIFT 4
+#define KVX_SFR_ES_PL2_ED_WIDTH 60
+#define KVX_SFR_ES_PL2_ED_WFXL_MASK _ULL(0xfffffff0)
+#define KVX_SFR_ES_PL2_ED_WFXL_CLEAR _ULL(0xfffffff0)
+#define KVX_SFR_ES_PL2_ED_WFXL_SET _ULL(0xfffffff000000000)
+#define KVX_SFR_ES_PL2_ED_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_ES_PL2_ED_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_ES_PL2_ED_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_ES_PL2_OAPL_MASK _ULL(0x30) /* Origin Absolute PL */
+#define KVX_SFR_ES_PL2_OAPL_SHIFT 4
+#define KVX_SFR_ES_PL2_OAPL_WIDTH 2
+#define KVX_SFR_ES_PL2_OAPL_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_ES_PL2_OAPL_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_ES_PL2_OAPL_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ES_PL2_ORPL_MASK _ULL(0xc0) /* Origin Relative PL */
+#define KVX_SFR_ES_PL2_ORPL_SHIFT 6
+#define KVX_SFR_ES_PL2_ORPL_WIDTH 2
+#define KVX_SFR_ES_PL2_ORPL_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_ES_PL2_ORPL_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_ES_PL2_ORPL_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ES_PL2_PTAPL_MASK _ULL(0x300) /* Target Absolute PL */
+#define KVX_SFR_ES_PL2_PTAPL_SHIFT 8
+#define KVX_SFR_ES_PL2_PTAPL_WIDTH 2
+#define KVX_SFR_ES_PL2_PTAPL_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_ES_PL2_PTAPL_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_ES_PL2_PTAPL_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ES_PL2_PTRPL_MASK _ULL(0xc00) /* Target Relative PL */
+#define KVX_SFR_ES_PL2_PTRPL_SHIFT 10
+#define KVX_SFR_ES_PL2_PTRPL_WIDTH 2
+#define KVX_SFR_ES_PL2_PTRPL_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_ES_PL2_PTRPL_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_ES_PL2_PTRPL_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ES_PL2_ITN_MASK _ULL(0x1f000) /* InTerrupt Number */
+#define KVX_SFR_ES_PL2_ITN_SHIFT 12
+#define KVX_SFR_ES_PL2_ITN_WIDTH 5
+#define KVX_SFR_ES_PL2_ITN_WFXL_MASK _ULL(0x1f000)
+#define KVX_SFR_ES_PL2_ITN_WFXL_CLEAR _ULL(0x1f000)
+#define KVX_SFR_ES_PL2_ITN_WFXL_SET _ULL(0x1f00000000000)
+
+#define KVX_SFR_ES_PL2_ITL_MASK _ULL(0x60000) /* InTerrupt Level */
+#define KVX_SFR_ES_PL2_ITL_SHIFT 17
+#define KVX_SFR_ES_PL2_ITL_WIDTH 2
+#define KVX_SFR_ES_PL2_ITL_WFXL_MASK _ULL(0x60000)
+#define KVX_SFR_ES_PL2_ITL_WFXL_CLEAR _ULL(0x60000)
+#define KVX_SFR_ES_PL2_ITL_WFXL_SET _ULL(0x6000000000000)
+
+#define KVX_SFR_ES_PL2_ITI_MASK _ULL(0x1ff80000) /* InTerrupt Info */
+#define KVX_SFR_ES_PL2_ITI_SHIFT 19
+#define KVX_SFR_ES_PL2_ITI_WIDTH 10
+#define KVX_SFR_ES_PL2_ITI_WFXL_MASK _ULL(0x1ff80000)
+#define KVX_SFR_ES_PL2_ITI_WFXL_CLEAR _ULL(0x1ff80000)
+#define KVX_SFR_ES_PL2_ITI_WFXL_SET _ULL(0x1ff8000000000000)
+
+#define KVX_SFR_ES_PL2_SN_MASK _ULL(0xfff000) /* Syscall Number */
+#define KVX_SFR_ES_PL2_SN_SHIFT 12
+#define KVX_SFR_ES_PL2_SN_WIDTH 12
+#define KVX_SFR_ES_PL2_SN_WFXL_MASK _ULL(0xfff000)
+#define KVX_SFR_ES_PL2_SN_WFXL_CLEAR _ULL(0xfff000)
+#define KVX_SFR_ES_PL2_SN_WFXL_SET _ULL(0xfff00000000000)
+
+#define KVX_SFR_ES_PL2_HTC_MASK _ULL(0x1f000) /* Hardware Trap Cause */
+#define KVX_SFR_ES_PL2_HTC_SHIFT 12
+#define KVX_SFR_ES_PL2_HTC_WIDTH 5
+#define KVX_SFR_ES_PL2_HTC_WFXL_MASK _ULL(0x1f000)
+#define KVX_SFR_ES_PL2_HTC_WFXL_CLEAR _ULL(0x1f000)
+#define KVX_SFR_ES_PL2_HTC_WFXL_SET _ULL(0x1f00000000000)
+
+#define KVX_SFR_ES_PL2_SFRT_MASK _ULL(0x20000) /* SFR Trap */
+#define KVX_SFR_ES_PL2_SFRT_SHIFT 17
+#define KVX_SFR_ES_PL2_SFRT_WIDTH 1
+#define KVX_SFR_ES_PL2_SFRT_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_ES_PL2_SFRT_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_ES_PL2_SFRT_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_ES_PL2_SFRI_MASK _ULL(0x1c0000) /* SFR Instruction */
+#define KVX_SFR_ES_PL2_SFRI_SHIFT 18
+#define KVX_SFR_ES_PL2_SFRI_WIDTH 3
+#define KVX_SFR_ES_PL2_SFRI_WFXL_MASK _ULL(0x1c0000)
+#define KVX_SFR_ES_PL2_SFRI_WFXL_CLEAR _ULL(0x1c0000)
+#define KVX_SFR_ES_PL2_SFRI_WFXL_SET _ULL(0x1c000000000000)
+
+#define KVX_SFR_ES_PL2_GPRP_MASK _ULL(0x7e00000) /* GPR Pointer */
+#define KVX_SFR_ES_PL2_GPRP_SHIFT 21
+#define KVX_SFR_ES_PL2_GPRP_WIDTH 6
+#define KVX_SFR_ES_PL2_GPRP_WFXL_MASK _ULL(0x7e00000)
+#define KVX_SFR_ES_PL2_GPRP_WFXL_CLEAR _ULL(0x7e00000)
+#define KVX_SFR_ES_PL2_GPRP_WFXL_SET _ULL(0x7e0000000000000)
+
+#define KVX_SFR_ES_PL2_SFRP_MASK _ULL(0xff8000000) /* SFR Pointer */
+#define KVX_SFR_ES_PL2_SFRP_SHIFT 27
+#define KVX_SFR_ES_PL2_SFRP_WIDTH 9
+#define KVX_SFR_ES_PL2_SFRP_WFXL_MASK _ULL(0xf8000000)
+#define KVX_SFR_ES_PL2_SFRP_WFXL_CLEAR _ULL(0xf8000000)
+#define KVX_SFR_ES_PL2_SFRP_WFXL_SET _ULL(0xf800000000000000)
+#define KVX_SFR_ES_PL2_SFRP_WFXM_MASK _ULL(0xf00000000)
+#define KVX_SFR_ES_PL2_SFRP_WFXM_CLEAR _ULL(0xf)
+#define KVX_SFR_ES_PL2_SFRP_WFXM_SET _ULL(0xf00000000)
+
+#define KVX_SFR_ES_PL2_DHT_MASK _ULL(0x1000000000) /* Disabled Hardware Trap */
+#define KVX_SFR_ES_PL2_DHT_SHIFT 36
+#define KVX_SFR_ES_PL2_DHT_WIDTH 1
+#define KVX_SFR_ES_PL2_DHT_WFXM_MASK _ULL(0x1000000000)
+#define KVX_SFR_ES_PL2_DHT_WFXM_CLEAR _ULL(0x10)
+#define KVX_SFR_ES_PL2_DHT_WFXM_SET _ULL(0x1000000000)
+
+#define KVX_SFR_ES_PL2_RWX_MASK _ULL(0x38000000000) /* Read Write Execute */
+#define KVX_SFR_ES_PL2_RWX_SHIFT 39
+#define KVX_SFR_ES_PL2_RWX_WIDTH 3
+#define KVX_SFR_ES_PL2_RWX_WFXM_MASK _ULL(0x38000000000)
+#define KVX_SFR_ES_PL2_RWX_WFXM_CLEAR _ULL(0x380)
+#define KVX_SFR_ES_PL2_RWX_WFXM_SET _ULL(0x38000000000)
+
+#define KVX_SFR_ES_PL2_NTA_MASK _ULL(0x40000000000) /* Non-Trapping Access */
+#define KVX_SFR_ES_PL2_NTA_SHIFT 42
+#define KVX_SFR_ES_PL2_NTA_WIDTH 1
+#define KVX_SFR_ES_PL2_NTA_WFXM_MASK _ULL(0x40000000000)
+#define KVX_SFR_ES_PL2_NTA_WFXM_CLEAR _ULL(0x400)
+#define KVX_SFR_ES_PL2_NTA_WFXM_SET _ULL(0x40000000000)
+
+#define KVX_SFR_ES_PL2_UCA_MASK _ULL(0x80000000000) /* Un-Cached Access */
+#define KVX_SFR_ES_PL2_UCA_SHIFT 43
+#define KVX_SFR_ES_PL2_UCA_WIDTH 1
+#define KVX_SFR_ES_PL2_UCA_WFXM_MASK _ULL(0x80000000000)
+#define KVX_SFR_ES_PL2_UCA_WFXM_CLEAR _ULL(0x800)
+#define KVX_SFR_ES_PL2_UCA_WFXM_SET _ULL(0x80000000000)
+
+#define KVX_SFR_ES_PL2_AS_MASK _ULL(0x3f00000000000) /* Access Size */
+#define KVX_SFR_ES_PL2_AS_SHIFT 44
+#define KVX_SFR_ES_PL2_AS_WIDTH 6
+#define KVX_SFR_ES_PL2_AS_WFXM_MASK _ULL(0x3f00000000000)
+#define KVX_SFR_ES_PL2_AS_WFXM_CLEAR _ULL(0x3f000)
+#define KVX_SFR_ES_PL2_AS_WFXM_SET _ULL(0x3f00000000000)
+
+#define KVX_SFR_ES_PL2_BS_MASK _ULL(0x3c000000000000) /* Bundle Size */
+#define KVX_SFR_ES_PL2_BS_SHIFT 50
+#define KVX_SFR_ES_PL2_BS_WIDTH 4
+#define KVX_SFR_ES_PL2_BS_WFXM_MASK _ULL(0x3c000000000000)
+#define KVX_SFR_ES_PL2_BS_WFXM_CLEAR _ULL(0x3c0000)
+#define KVX_SFR_ES_PL2_BS_WFXM_SET _ULL(0x3c000000000000)
+
+#define KVX_SFR_ES_PL2_DRI_MASK _ULL(0xfc0000000000000) /* Data Register Index */
+#define KVX_SFR_ES_PL2_DRI_SHIFT 54
+#define KVX_SFR_ES_PL2_DRI_WIDTH 6
+#define KVX_SFR_ES_PL2_DRI_WFXM_MASK _ULL(0xfc0000000000000)
+#define KVX_SFR_ES_PL2_DRI_WFXM_CLEAR _ULL(0xfc00000)
+#define KVX_SFR_ES_PL2_DRI_WFXM_SET _ULL(0xfc0000000000000)
+
+#define KVX_SFR_ES_PL2_PIC_MASK _ULL(0xf000000000000000) /* Privileged Instruction Code */
+#define KVX_SFR_ES_PL2_PIC_SHIFT 60
+#define KVX_SFR_ES_PL2_PIC_WIDTH 4
+#define KVX_SFR_ES_PL2_PIC_WFXM_MASK _ULL(0xf000000000000000)
+#define KVX_SFR_ES_PL2_PIC_WFXM_CLEAR _ULL(0xf0000000)
+#define KVX_SFR_ES_PL2_PIC_WFXM_SET _ULL(0xf000000000000000)
+
+#define KVX_SFR_ES_PL2_DC_MASK _ULL(0x3000) /* Debug Cause */
+#define KVX_SFR_ES_PL2_DC_SHIFT 12
+#define KVX_SFR_ES_PL2_DC_WIDTH 2
+#define KVX_SFR_ES_PL2_DC_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_ES_PL2_DC_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_ES_PL2_DC_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ES_PL2_BN_MASK _ULL(0x4000) /* Breakpoint Number */
+#define KVX_SFR_ES_PL2_BN_SHIFT 14
+#define KVX_SFR_ES_PL2_BN_WIDTH 1
+#define KVX_SFR_ES_PL2_BN_WFXL_MASK _ULL(0x4000)
+#define KVX_SFR_ES_PL2_BN_WFXL_CLEAR _ULL(0x4000)
+#define KVX_SFR_ES_PL2_BN_WFXL_SET _ULL(0x400000000000)
+
+#define KVX_SFR_ES_PL2_WN_MASK _ULL(0x8000) /* Watchpoint Number */
+#define KVX_SFR_ES_PL2_WN_SHIFT 15
+#define KVX_SFR_ES_PL2_WN_WIDTH 1
+#define KVX_SFR_ES_PL2_WN_WFXL_MASK _ULL(0x8000)
+#define KVX_SFR_ES_PL2_WN_WFXL_CLEAR _ULL(0x8000)
+#define KVX_SFR_ES_PL2_WN_WFXL_SET _ULL(0x800000000000)
+
+#define KVX_SFR_ES_PL3_EC_MASK _ULL(0xf) /* Exception Class */
+#define KVX_SFR_ES_PL3_EC_SHIFT 0
+#define KVX_SFR_ES_PL3_EC_WIDTH 4
+#define KVX_SFR_ES_PL3_EC_WFXL_MASK _ULL(0xf)
+#define KVX_SFR_ES_PL3_EC_WFXL_CLEAR _ULL(0xf)
+#define KVX_SFR_ES_PL3_EC_WFXL_SET _ULL(0xf00000000)
+
+#define KVX_SFR_ES_PL3_ED_MASK _ULL(0xfffffffffffffff0) /* Exception Details */
+#define KVX_SFR_ES_PL3_ED_SHIFT 4
+#define KVX_SFR_ES_PL3_ED_WIDTH 60
+#define KVX_SFR_ES_PL3_ED_WFXL_MASK _ULL(0xfffffff0)
+#define KVX_SFR_ES_PL3_ED_WFXL_CLEAR _ULL(0xfffffff0)
+#define KVX_SFR_ES_PL3_ED_WFXL_SET _ULL(0xfffffff000000000)
+#define KVX_SFR_ES_PL3_ED_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_ES_PL3_ED_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_ES_PL3_ED_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_ES_PL3_OAPL_MASK _ULL(0x30) /* Origin Absolute PL */
+#define KVX_SFR_ES_PL3_OAPL_SHIFT 4
+#define KVX_SFR_ES_PL3_OAPL_WIDTH 2
+#define KVX_SFR_ES_PL3_OAPL_WFXL_MASK _ULL(0x30)
+#define KVX_SFR_ES_PL3_OAPL_WFXL_CLEAR _ULL(0x30)
+#define KVX_SFR_ES_PL3_OAPL_WFXL_SET _ULL(0x3000000000)
+
+#define KVX_SFR_ES_PL3_ORPL_MASK _ULL(0xc0) /* Origin Relative PL */
+#define KVX_SFR_ES_PL3_ORPL_SHIFT 6
+#define KVX_SFR_ES_PL3_ORPL_WIDTH 2
+#define KVX_SFR_ES_PL3_ORPL_WFXL_MASK _ULL(0xc0)
+#define KVX_SFR_ES_PL3_ORPL_WFXL_CLEAR _ULL(0xc0)
+#define KVX_SFR_ES_PL3_ORPL_WFXL_SET _ULL(0xc000000000)
+
+#define KVX_SFR_ES_PL3_PTAPL_MASK _ULL(0x300) /* Target Absolute PL */
+#define KVX_SFR_ES_PL3_PTAPL_SHIFT 8
+#define KVX_SFR_ES_PL3_PTAPL_WIDTH 2
+#define KVX_SFR_ES_PL3_PTAPL_WFXL_MASK _ULL(0x300)
+#define KVX_SFR_ES_PL3_PTAPL_WFXL_CLEAR _ULL(0x300)
+#define KVX_SFR_ES_PL3_PTAPL_WFXL_SET _ULL(0x30000000000)
+
+#define KVX_SFR_ES_PL3_PTRPL_MASK _ULL(0xc00) /* Target Relative PL */
+#define KVX_SFR_ES_PL3_PTRPL_SHIFT 10
+#define KVX_SFR_ES_PL3_PTRPL_WIDTH 2
+#define KVX_SFR_ES_PL3_PTRPL_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_ES_PL3_PTRPL_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_ES_PL3_PTRPL_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_ES_PL3_ITN_MASK _ULL(0x1f000) /* InTerrupt Number */
+#define KVX_SFR_ES_PL3_ITN_SHIFT 12
+#define KVX_SFR_ES_PL3_ITN_WIDTH 5
+#define KVX_SFR_ES_PL3_ITN_WFXL_MASK _ULL(0x1f000)
+#define KVX_SFR_ES_PL3_ITN_WFXL_CLEAR _ULL(0x1f000)
+#define KVX_SFR_ES_PL3_ITN_WFXL_SET _ULL(0x1f00000000000)
+
+#define KVX_SFR_ES_PL3_ITL_MASK _ULL(0x60000) /* InTerrupt Level */
+#define KVX_SFR_ES_PL3_ITL_SHIFT 17
+#define KVX_SFR_ES_PL3_ITL_WIDTH 2
+#define KVX_SFR_ES_PL3_ITL_WFXL_MASK _ULL(0x60000)
+#define KVX_SFR_ES_PL3_ITL_WFXL_CLEAR _ULL(0x60000)
+#define KVX_SFR_ES_PL3_ITL_WFXL_SET _ULL(0x6000000000000)
+
+#define KVX_SFR_ES_PL3_ITI_MASK _ULL(0x1ff80000) /* InTerrupt Info */
+#define KVX_SFR_ES_PL3_ITI_SHIFT 19
+#define KVX_SFR_ES_PL3_ITI_WIDTH 10
+#define KVX_SFR_ES_PL3_ITI_WFXL_MASK _ULL(0x1ff80000)
+#define KVX_SFR_ES_PL3_ITI_WFXL_CLEAR _ULL(0x1ff80000)
+#define KVX_SFR_ES_PL3_ITI_WFXL_SET _ULL(0x1ff8000000000000)
+
+#define KVX_SFR_ES_PL3_SN_MASK _ULL(0xfff000) /* Syscall Number */
+#define KVX_SFR_ES_PL3_SN_SHIFT 12
+#define KVX_SFR_ES_PL3_SN_WIDTH 12
+#define KVX_SFR_ES_PL3_SN_WFXL_MASK _ULL(0xfff000)
+#define KVX_SFR_ES_PL3_SN_WFXL_CLEAR _ULL(0xfff000)
+#define KVX_SFR_ES_PL3_SN_WFXL_SET _ULL(0xfff00000000000)
+
+#define KVX_SFR_ES_PL3_HTC_MASK _ULL(0x1f000) /* Hardware Trap Cause */
+#define KVX_SFR_ES_PL3_HTC_SHIFT 12
+#define KVX_SFR_ES_PL3_HTC_WIDTH 5
+#define KVX_SFR_ES_PL3_HTC_WFXL_MASK _ULL(0x1f000)
+#define KVX_SFR_ES_PL3_HTC_WFXL_CLEAR _ULL(0x1f000)
+#define KVX_SFR_ES_PL3_HTC_WFXL_SET _ULL(0x1f00000000000)
+
+#define KVX_SFR_ES_PL3_SFRT_MASK _ULL(0x20000) /* SFR Trap */
+#define KVX_SFR_ES_PL3_SFRT_SHIFT 17
+#define KVX_SFR_ES_PL3_SFRT_WIDTH 1
+#define KVX_SFR_ES_PL3_SFRT_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_ES_PL3_SFRT_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_ES_PL3_SFRT_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_ES_PL3_SFRI_MASK _ULL(0x1c0000) /* SFR Instruction */
+#define KVX_SFR_ES_PL3_SFRI_SHIFT 18
+#define KVX_SFR_ES_PL3_SFRI_WIDTH 3
+#define KVX_SFR_ES_PL3_SFRI_WFXL_MASK _ULL(0x1c0000)
+#define KVX_SFR_ES_PL3_SFRI_WFXL_CLEAR _ULL(0x1c0000)
+#define KVX_SFR_ES_PL3_SFRI_WFXL_SET _ULL(0x1c000000000000)
+
+#define KVX_SFR_ES_PL3_GPRP_MASK _ULL(0x7e00000) /* GPR Pointer */
+#define KVX_SFR_ES_PL3_GPRP_SHIFT 21
+#define KVX_SFR_ES_PL3_GPRP_WIDTH 6
+#define KVX_SFR_ES_PL3_GPRP_WFXL_MASK _ULL(0x7e00000)
+#define KVX_SFR_ES_PL3_GPRP_WFXL_CLEAR _ULL(0x7e00000)
+#define KVX_SFR_ES_PL3_GPRP_WFXL_SET _ULL(0x7e0000000000000)
+
+#define KVX_SFR_ES_PL3_SFRP_MASK _ULL(0xff8000000) /* SFR Pointer */
+#define KVX_SFR_ES_PL3_SFRP_SHIFT 27
+#define KVX_SFR_ES_PL3_SFRP_WIDTH 9
+#define KVX_SFR_ES_PL3_SFRP_WFXL_MASK _ULL(0xf8000000)
+#define KVX_SFR_ES_PL3_SFRP_WFXL_CLEAR _ULL(0xf8000000)
+#define KVX_SFR_ES_PL3_SFRP_WFXL_SET _ULL(0xf800000000000000)
+#define KVX_SFR_ES_PL3_SFRP_WFXM_MASK _ULL(0xf00000000)
+#define KVX_SFR_ES_PL3_SFRP_WFXM_CLEAR _ULL(0xf)
+#define KVX_SFR_ES_PL3_SFRP_WFXM_SET _ULL(0xf00000000)
+
+#define KVX_SFR_ES_PL3_DHT_MASK _ULL(0x1000000000) /* Disabled Hardware Trap */
+#define KVX_SFR_ES_PL3_DHT_SHIFT 36
+#define KVX_SFR_ES_PL3_DHT_WIDTH 1
+#define KVX_SFR_ES_PL3_DHT_WFXM_MASK _ULL(0x1000000000)
+#define KVX_SFR_ES_PL3_DHT_WFXM_CLEAR _ULL(0x10)
+#define KVX_SFR_ES_PL3_DHT_WFXM_SET _ULL(0x1000000000)
+
+#define KVX_SFR_ES_PL3_RWX_MASK _ULL(0x38000000000) /* Read Write Execute */
+#define KVX_SFR_ES_PL3_RWX_SHIFT 39
+#define KVX_SFR_ES_PL3_RWX_WIDTH 3
+#define KVX_SFR_ES_PL3_RWX_WFXM_MASK _ULL(0x38000000000)
+#define KVX_SFR_ES_PL3_RWX_WFXM_CLEAR _ULL(0x380)
+#define KVX_SFR_ES_PL3_RWX_WFXM_SET _ULL(0x38000000000)
+
+#define KVX_SFR_ES_PL3_NTA_MASK _ULL(0x40000000000) /* Non-Trapping Access */
+#define KVX_SFR_ES_PL3_NTA_SHIFT 42
+#define KVX_SFR_ES_PL3_NTA_WIDTH 1
+#define KVX_SFR_ES_PL3_NTA_WFXM_MASK _ULL(0x40000000000)
+#define KVX_SFR_ES_PL3_NTA_WFXM_CLEAR _ULL(0x400)
+#define KVX_SFR_ES_PL3_NTA_WFXM_SET _ULL(0x40000000000)
+
+#define KVX_SFR_ES_PL3_UCA_MASK _ULL(0x80000000000) /* Un-Cached Access */
+#define KVX_SFR_ES_PL3_UCA_SHIFT 43
+#define KVX_SFR_ES_PL3_UCA_WIDTH 1
+#define KVX_SFR_ES_PL3_UCA_WFXM_MASK _ULL(0x80000000000)
+#define KVX_SFR_ES_PL3_UCA_WFXM_CLEAR _ULL(0x800)
+#define KVX_SFR_ES_PL3_UCA_WFXM_SET _ULL(0x80000000000)
+
+#define KVX_SFR_ES_PL3_AS_MASK _ULL(0x3f00000000000) /* Access Size */
+#define KVX_SFR_ES_PL3_AS_SHIFT 44
+#define KVX_SFR_ES_PL3_AS_WIDTH 6
+#define KVX_SFR_ES_PL3_AS_WFXM_MASK _ULL(0x3f00000000000)
+#define KVX_SFR_ES_PL3_AS_WFXM_CLEAR _ULL(0x3f000)
+#define KVX_SFR_ES_PL3_AS_WFXM_SET _ULL(0x3f00000000000)
+
+#define KVX_SFR_ES_PL3_BS_MASK _ULL(0x3c000000000000) /* Bundle Size */
+#define KVX_SFR_ES_PL3_BS_SHIFT 50
+#define KVX_SFR_ES_PL3_BS_WIDTH 4
+#define KVX_SFR_ES_PL3_BS_WFXM_MASK _ULL(0x3c000000000000)
+#define KVX_SFR_ES_PL3_BS_WFXM_CLEAR _ULL(0x3c0000)
+#define KVX_SFR_ES_PL3_BS_WFXM_SET _ULL(0x3c000000000000)
+
+#define KVX_SFR_ES_PL3_DRI_MASK _ULL(0xfc0000000000000) /* Data Register Index */
+#define KVX_SFR_ES_PL3_DRI_SHIFT 54
+#define KVX_SFR_ES_PL3_DRI_WIDTH 6
+#define KVX_SFR_ES_PL3_DRI_WFXM_MASK _ULL(0xfc0000000000000)
+#define KVX_SFR_ES_PL3_DRI_WFXM_CLEAR _ULL(0xfc00000)
+#define KVX_SFR_ES_PL3_DRI_WFXM_SET _ULL(0xfc0000000000000)
+
+#define KVX_SFR_ES_PL3_PIC_MASK _ULL(0xf000000000000000) /* Privileged Instruction Code */
+#define KVX_SFR_ES_PL3_PIC_SHIFT 60
+#define KVX_SFR_ES_PL3_PIC_WIDTH 4
+#define KVX_SFR_ES_PL3_PIC_WFXM_MASK _ULL(0xf000000000000000)
+#define KVX_SFR_ES_PL3_PIC_WFXM_CLEAR _ULL(0xf0000000)
+#define KVX_SFR_ES_PL3_PIC_WFXM_SET _ULL(0xf000000000000000)
+
+#define KVX_SFR_ES_PL3_DC_MASK _ULL(0x3000) /* Debug Cause */
+#define KVX_SFR_ES_PL3_DC_SHIFT 12
+#define KVX_SFR_ES_PL3_DC_WIDTH 2
+#define KVX_SFR_ES_PL3_DC_WFXL_MASK _ULL(0x3000)
+#define KVX_SFR_ES_PL3_DC_WFXL_CLEAR _ULL(0x3000)
+#define KVX_SFR_ES_PL3_DC_WFXL_SET _ULL(0x300000000000)
+
+#define KVX_SFR_ES_PL3_BN_MASK _ULL(0x4000) /* Breakpoint Number */
+#define KVX_SFR_ES_PL3_BN_SHIFT 14
+#define KVX_SFR_ES_PL3_BN_WIDTH 1
+#define KVX_SFR_ES_PL3_BN_WFXL_MASK _ULL(0x4000)
+#define KVX_SFR_ES_PL3_BN_WFXL_CLEAR _ULL(0x4000)
+#define KVX_SFR_ES_PL3_BN_WFXL_SET _ULL(0x400000000000)
+
+#define KVX_SFR_ES_PL3_WN_MASK _ULL(0x8000) /* Watchpoint Number */
+#define KVX_SFR_ES_PL3_WN_SHIFT 15
+#define KVX_SFR_ES_PL3_WN_WIDTH 1
+#define KVX_SFR_ES_PL3_WN_WFXL_MASK _ULL(0x8000)
+#define KVX_SFR_ES_PL3_WN_WFXL_CLEAR _ULL(0x8000)
+#define KVX_SFR_ES_PL3_WN_WFXL_SET _ULL(0x800000000000)
+
+#define KVX_SFR_TCR_T0CE_MASK _ULL(0x10000) /* Timer 0 Count Enable */
+#define KVX_SFR_TCR_T0CE_SHIFT 16
+#define KVX_SFR_TCR_T0CE_WIDTH 1
+#define KVX_SFR_TCR_T0CE_WFXL_MASK _ULL(0x10000)
+#define KVX_SFR_TCR_T0CE_WFXL_CLEAR _ULL(0x10000)
+#define KVX_SFR_TCR_T0CE_WFXL_SET _ULL(0x1000000000000)
+
+#define KVX_SFR_TCR_T1CE_MASK _ULL(0x20000) /* Timer 1 Count Enable */
+#define KVX_SFR_TCR_T1CE_SHIFT 17
+#define KVX_SFR_TCR_T1CE_WIDTH 1
+#define KVX_SFR_TCR_T1CE_WFXL_MASK _ULL(0x20000)
+#define KVX_SFR_TCR_T1CE_WFXL_CLEAR _ULL(0x20000)
+#define KVX_SFR_TCR_T1CE_WFXL_SET _ULL(0x2000000000000)
+
+#define KVX_SFR_TCR_T0IE_MASK _ULL(0x40000) /* Timer 0 Interrupt Enable */
+#define KVX_SFR_TCR_T0IE_SHIFT 18
+#define KVX_SFR_TCR_T0IE_WIDTH 1
+#define KVX_SFR_TCR_T0IE_WFXL_MASK _ULL(0x40000)
+#define KVX_SFR_TCR_T0IE_WFXL_CLEAR _ULL(0x40000)
+#define KVX_SFR_TCR_T0IE_WFXL_SET _ULL(0x4000000000000)
+
+#define KVX_SFR_TCR_T1IE_MASK _ULL(0x80000) /* Timer 1 Interrupt Enable */
+#define KVX_SFR_TCR_T1IE_SHIFT 19
+#define KVX_SFR_TCR_T1IE_WIDTH 1
+#define KVX_SFR_TCR_T1IE_WFXL_MASK _ULL(0x80000)
+#define KVX_SFR_TCR_T1IE_WFXL_CLEAR _ULL(0x80000)
+#define KVX_SFR_TCR_T1IE_WFXL_SET _ULL(0x8000000000000)
+
+#define KVX_SFR_TCR_T0ST_MASK _ULL(0x100000) /* Timer 0 Status */
+#define KVX_SFR_TCR_T0ST_SHIFT 20
+#define KVX_SFR_TCR_T0ST_WIDTH 1
+#define KVX_SFR_TCR_T0ST_WFXL_MASK _ULL(0x100000)
+#define KVX_SFR_TCR_T0ST_WFXL_CLEAR _ULL(0x100000)
+#define KVX_SFR_TCR_T0ST_WFXL_SET _ULL(0x10000000000000)
+
+#define KVX_SFR_TCR_T1ST_MASK _ULL(0x200000) /* Timer 1 Status */
+#define KVX_SFR_TCR_T1ST_SHIFT 21
+#define KVX_SFR_TCR_T1ST_WIDTH 1
+#define KVX_SFR_TCR_T1ST_WFXL_MASK _ULL(0x200000)
+#define KVX_SFR_TCR_T1ST_WFXL_CLEAR _ULL(0x200000)
+#define KVX_SFR_TCR_T1ST_WFXL_SET _ULL(0x20000000000000)
+
+#define KVX_SFR_TCR_T0SI_MASK _ULL(0x400000) /* Stop Timer 0 in Idle */
+#define KVX_SFR_TCR_T0SI_SHIFT 22
+#define KVX_SFR_TCR_T0SI_WIDTH 1
+#define KVX_SFR_TCR_T0SI_WFXL_MASK _ULL(0x400000)
+#define KVX_SFR_TCR_T0SI_WFXL_CLEAR _ULL(0x400000)
+#define KVX_SFR_TCR_T0SI_WFXL_SET _ULL(0x40000000000000)
+
+#define KVX_SFR_TCR_T1SI_MASK _ULL(0x800000) /* Stop Timer 1 in Idle */
+#define KVX_SFR_TCR_T1SI_SHIFT 23
+#define KVX_SFR_TCR_T1SI_WIDTH 1
+#define KVX_SFR_TCR_T1SI_WFXL_MASK _ULL(0x800000)
+#define KVX_SFR_TCR_T1SI_WFXL_CLEAR _ULL(0x800000)
+#define KVX_SFR_TCR_T1SI_WFXL_SET _ULL(0x80000000000000)
+
+#define KVX_SFR_TCR_WCE_MASK _ULL(0x1000000) /* Watchdog Counting Enable */
+#define KVX_SFR_TCR_WCE_SHIFT 24
+#define KVX_SFR_TCR_WCE_WIDTH 1
+#define KVX_SFR_TCR_WCE_WFXL_MASK _ULL(0x1000000)
+#define KVX_SFR_TCR_WCE_WFXL_CLEAR _ULL(0x1000000)
+#define KVX_SFR_TCR_WCE_WFXL_SET _ULL(0x100000000000000)
+
+#define KVX_SFR_TCR_WIE_MASK _ULL(0x2000000) /* Watchdog Interrupt Enable */
+#define KVX_SFR_TCR_WIE_SHIFT 25
+#define KVX_SFR_TCR_WIE_WIDTH 1
+#define KVX_SFR_TCR_WIE_WFXL_MASK _ULL(0x2000000)
+#define KVX_SFR_TCR_WIE_WFXL_CLEAR _ULL(0x2000000)
+#define KVX_SFR_TCR_WIE_WFXL_SET _ULL(0x200000000000000)
+
+#define KVX_SFR_TCR_WUI_MASK _ULL(0x4000000) /* Watchdog Underflow Inform */
+#define KVX_SFR_TCR_WUI_SHIFT 26
+#define KVX_SFR_TCR_WUI_WIDTH 1
+#define KVX_SFR_TCR_WUI_WFXL_MASK _ULL(0x4000000)
+#define KVX_SFR_TCR_WUI_WFXL_CLEAR _ULL(0x4000000)
+#define KVX_SFR_TCR_WUI_WFXL_SET _ULL(0x400000000000000)
+
+#define KVX_SFR_TCR_WUS_MASK _ULL(0x8000000) /* Watchdog Underflow Status */
+#define KVX_SFR_TCR_WUS_SHIFT 27
+#define KVX_SFR_TCR_WUS_WIDTH 1
+#define KVX_SFR_TCR_WUS_WFXL_MASK _ULL(0x8000000)
+#define KVX_SFR_TCR_WUS_WFXL_CLEAR _ULL(0x8000000)
+#define KVX_SFR_TCR_WUS_WFXL_SET _ULL(0x800000000000000)
+
+#define KVX_SFR_TCR_WSI_MASK _ULL(0x10000000) /* Watchdog Stop in Idle */
+#define KVX_SFR_TCR_WSI_SHIFT 28
+#define KVX_SFR_TCR_WSI_WIDTH 1
+#define KVX_SFR_TCR_WSI_WFXL_MASK _ULL(0x10000000)
+#define KVX_SFR_TCR_WSI_WFXL_CLEAR _ULL(0x10000000)
+#define KVX_SFR_TCR_WSI_WFXL_SET _ULL(0x1000000000000000)
+
+#define KVX_SFR_PM0_PM0_MASK _ULL(0xffffffffffffffff) /* Performance Monitor 0 */
+#define KVX_SFR_PM0_PM0_SHIFT 0
+#define KVX_SFR_PM0_PM0_WIDTH 64
+#define KVX_SFR_PM0_PM0_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_PM0_PM0_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_PM0_PM0_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_PM0_PM0_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_PM0_PM0_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_PM0_PM0_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_PM1_PM1_MASK _ULL(0xffffffffffffffff) /* Performance Monitor 1 */
+#define KVX_SFR_PM1_PM1_SHIFT 0
+#define KVX_SFR_PM1_PM1_WIDTH 64
+#define KVX_SFR_PM1_PM1_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_PM1_PM1_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_PM1_PM1_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_PM1_PM1_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_PM1_PM1_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_PM1_PM1_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_PM2_PM2_MASK _ULL(0xffffffffffffffff) /* Performance Monitor 2 */
+#define KVX_SFR_PM2_PM2_SHIFT 0
+#define KVX_SFR_PM2_PM2_WIDTH 64
+#define KVX_SFR_PM2_PM2_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_PM2_PM2_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_PM2_PM2_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_PM2_PM2_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_PM2_PM2_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_PM2_PM2_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_PM3_PM3_MASK _ULL(0xffffffffffffffff) /* Performance Monitor 3 */
+#define KVX_SFR_PM3_PM3_SHIFT 0
+#define KVX_SFR_PM3_PM3_WIDTH 64
+#define KVX_SFR_PM3_PM3_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_PM3_PM3_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_PM3_PM3_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_PM3_PM3_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_PM3_PM3_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_PM3_PM3_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_PMSA_PMSA_MASK _ULL(0xffffffffffffffff) /* Performance Monitor Saved Address */
+#define KVX_SFR_PMSA_PMSA_SHIFT 0
+#define KVX_SFR_PMSA_PMSA_WIDTH 64
+#define KVX_SFR_PMSA_PMSA_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_PMSA_PMSA_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_PMSA_PMSA_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_PMSA_PMSA_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_PMSA_PMSA_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_PMSA_PMSA_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_T0V_T0V_MASK _ULL(0xffffffffffffffff) /* Timer 0 value */
+#define KVX_SFR_T0V_T0V_SHIFT 0
+#define KVX_SFR_T0V_T0V_WIDTH 64
+#define KVX_SFR_T0V_T0V_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_T0V_T0V_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_T0V_T0V_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_T0V_T0V_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_T0V_T0V_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_T0V_T0V_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_T1V_T1V_MASK _ULL(0xffffffffffffffff) /* Timer 1 value */
+#define KVX_SFR_T1V_T1V_SHIFT 0
+#define KVX_SFR_T1V_T1V_WIDTH 64
+#define KVX_SFR_T1V_T1V_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_T1V_T1V_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_T1V_T1V_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_T1V_T1V_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_T1V_T1V_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_T1V_T1V_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_T0R_T0R_MASK _ULL(0xffffffffffffffff) /* Timer 0 reload value */
+#define KVX_SFR_T0R_T0R_SHIFT 0
+#define KVX_SFR_T0R_T0R_WIDTH 64
+#define KVX_SFR_T0R_T0R_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_T0R_T0R_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_T0R_T0R_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_T0R_T0R_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_T0R_T0R_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_T0R_T0R_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_T1R_T1R_MASK _ULL(0xffffffffffffffff) /* Timer 1 reload value */
+#define KVX_SFR_T1R_T1R_SHIFT 0
+#define KVX_SFR_T1R_T1R_WIDTH 64
+#define KVX_SFR_T1R_T1R_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_T1R_T1R_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_T1R_T1R_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_T1R_T1R_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_T1R_T1R_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_T1R_T1R_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_WDV_WDV_MASK _ULL(0xffffffffffffffff) /* Watchdog Value */
+#define KVX_SFR_WDV_WDV_SHIFT 0
+#define KVX_SFR_WDV_WDV_WIDTH 64
+#define KVX_SFR_WDV_WDV_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_WDV_WDV_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_WDV_WDV_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_WDV_WDV_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_WDV_WDV_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_WDV_WDV_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_WDR_WDR_MASK _ULL(0xffffffffffffffff) /* Watchdog Reload Value */
+#define KVX_SFR_WDR_WDR_SHIFT 0
+#define KVX_SFR_WDR_WDR_WIDTH 64
+#define KVX_SFR_WDR_WDR_WFXL_MASK _ULL(0xffffffff)
+#define KVX_SFR_WDR_WDR_WFXL_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_WDR_WDR_WFXL_SET _ULL(0xffffffff00000000)
+#define KVX_SFR_WDR_WDR_WFXM_MASK _ULL(0xffffffff00000000)
+#define KVX_SFR_WDR_WDR_WFXM_CLEAR _ULL(0xffffffff)
+#define KVX_SFR_WDR_WDR_WFXM_SET _ULL(0xffffffff00000000)
+
+#define KVX_SFR_PMC_PM0C_MASK _ULL(0x3f) /* PM0 Configuration */
+#define KVX_SFR_PMC_PM0C_SHIFT 0
+#define KVX_SFR_PMC_PM0C_WIDTH 6
+#define KVX_SFR_PMC_PM0C_WFXL_MASK _ULL(0x3f)
+#define KVX_SFR_PMC_PM0C_WFXL_CLEAR _ULL(0x3f)
+#define KVX_SFR_PMC_PM0C_WFXL_SET _ULL(0x3f00000000)
+
+#define KVX_SFR_PMC_PM1C_MASK _ULL(0x1f80) /* PM1 Configuration */
+#define KVX_SFR_PMC_PM1C_SHIFT 7
+#define KVX_SFR_PMC_PM1C_WIDTH 6
+#define KVX_SFR_PMC_PM1C_WFXL_MASK _ULL(0x1f80)
+#define KVX_SFR_PMC_PM1C_WFXL_CLEAR _ULL(0x1f80)
+#define KVX_SFR_PMC_PM1C_WFXL_SET _ULL(0x1f8000000000)
+
+#define KVX_SFR_PMC_PM2C_MASK _ULL(0xfc000) /* PM2 Configuration */
+#define KVX_SFR_PMC_PM2C_SHIFT 14
+#define KVX_SFR_PMC_PM2C_WIDTH 6
+#define KVX_SFR_PMC_PM2C_WFXL_MASK _ULL(0xfc000)
+#define KVX_SFR_PMC_PM2C_WFXL_CLEAR _ULL(0xfc000)
+#define KVX_SFR_PMC_PM2C_WFXL_SET _ULL(0xfc00000000000)
+
+#define KVX_SFR_PMC_PM3C_MASK _ULL(0x7e00000) /* PM3 Configuration */
+#define KVX_SFR_PMC_PM3C_SHIFT 21
+#define KVX_SFR_PMC_PM3C_WIDTH 6
+#define KVX_SFR_PMC_PM3C_WFXL_MASK _ULL(0x7e00000)
+#define KVX_SFR_PMC_PM3C_WFXL_CLEAR _ULL(0x7e00000)
+#define KVX_SFR_PMC_PM3C_WFXL_SET _ULL(0x7e0000000000000)
+
+#define KVX_SFR_PMC_SAV_MASK _ULL(0x40000000) /* Saved Address Valid */
+#define KVX_SFR_PMC_SAV_SHIFT 30
+#define KVX_SFR_PMC_SAV_WIDTH 1
+#define KVX_SFR_PMC_SAV_WFXL_MASK _ULL(0x40000000)
+#define KVX_SFR_PMC_SAV_WFXL_CLEAR _ULL(0x40000000)
+#define KVX_SFR_PMC_SAV_WFXL_SET _ULL(0x4000000000000000)
+
+#define KVX_SFR_PMC_PM0IE_MASK _ULL(0x100000000) /* PM0 Interrupt Enable */
+#define KVX_SFR_PMC_PM0IE_SHIFT 32
+#define KVX_SFR_PMC_PM0IE_WIDTH 1
+#define KVX_SFR_PMC_PM0IE_WFXM_MASK _ULL(0x100000000)
+#define KVX_SFR_PMC_PM0IE_WFXM_CLEAR _ULL(0x1)
+#define KVX_SFR_PMC_PM0IE_WFXM_SET _ULL(0x100000000)
+
+#define KVX_SFR_PMC_PM1IE_MASK _ULL(0x200000000) /* PM1 Interrupt Enable */
+#define KVX_SFR_PMC_PM1IE_SHIFT 33
+#define KVX_SFR_PMC_PM1IE_WIDTH 1
+#define KVX_SFR_PMC_PM1IE_WFXM_MASK _ULL(0x200000000)
+#define KVX_SFR_PMC_PM1IE_WFXM_CLEAR _ULL(0x2)
+#define KVX_SFR_PMC_PM1IE_WFXM_SET _ULL(0x200000000)
+
+#define KVX_SFR_PMC_PM2IE_MASK _ULL(0x400000000) /* PM2 Interrupt Enable */
+#define KVX_SFR_PMC_PM2IE_SHIFT 34
+#define KVX_SFR_PMC_PM2IE_WIDTH 1
+#define KVX_SFR_PMC_PM2IE_WFXM_MASK _ULL(0x400000000)
+#define KVX_SFR_PMC_PM2IE_WFXM_CLEAR _ULL(0x4)
+#define KVX_SFR_PMC_PM2IE_WFXM_SET _ULL(0x400000000)
+
+#define KVX_SFR_PMC_PM3IE_MASK _ULL(0x800000000) /* PM3 Interrupt Enable */
+#define KVX_SFR_PMC_PM3IE_SHIFT 35
+#define KVX_SFR_PMC_PM3IE_WIDTH 1
+#define KVX_SFR_PMC_PM3IE_WFXM_MASK _ULL(0x800000000)
+#define KVX_SFR_PMC_PM3IE_WFXM_CLEAR _ULL(0x8)
+#define KVX_SFR_PMC_PM3IE_WFXM_SET _ULL(0x800000000)
+
+#define KVX_SFR_PMC_SAT_MASK _ULL(0x3000000000) /* Saved Address Type */
+#define KVX_SFR_PMC_SAT_SHIFT 36
+#define KVX_SFR_PMC_SAT_WIDTH 2
+#define KVX_SFR_PMC_SAT_WFXM_MASK _ULL(0x3000000000)
+#define KVX_SFR_PMC_SAT_WFXM_CLEAR _ULL(0x30)
+#define KVX_SFR_PMC_SAT_WFXM_SET _ULL(0x3000000000)
+
+#define KVX_SFR_PCR_PID_MASK _ULL(0xff) /* Processing Identifier in cluster */
+#define KVX_SFR_PCR_PID_SHIFT 0
+#define KVX_SFR_PCR_PID_WIDTH 8
+#define KVX_SFR_PCR_PID_WFXL_MASK _ULL(0xff)
+#define KVX_SFR_PCR_PID_WFXL_CLEAR _ULL(0xff)
+#define KVX_SFR_PCR_PID_WFXL_SET _ULL(0xff00000000)
+
+#define KVX_SFR_PCR_CID_MASK _ULL(0xff00) /* Cluster Identifier in system */
+#define KVX_SFR_PCR_CID_SHIFT 8
+#define KVX_SFR_PCR_CID_WIDTH 8
+#define KVX_SFR_PCR_CID_WFXL_MASK _ULL(0xff00)
+#define KVX_SFR_PCR_CID_WFXL_CLEAR _ULL(0xff00)
+#define KVX_SFR_PCR_CID_WFXL_SET _ULL(0xff0000000000)
+
+#define KVX_SFR_PCR_MID_MASK _ULL(0xff0000) /* MPPA Identifier */
+#define KVX_SFR_PCR_MID_SHIFT 16
+#define KVX_SFR_PCR_MID_WIDTH 8
+#define KVX_SFR_PCR_MID_WFXL_MASK _ULL(0xff0000)
+#define KVX_SFR_PCR_MID_WFXL_CLEAR _ULL(0xff0000)
+#define KVX_SFR_PCR_MID_WFXL_SET _ULL(0xff000000000000)
+
+#define KVX_SFR_PCR_CAR_MASK _ULL(0xf000000) /* Core Architecture Revision ID */
+#define KVX_SFR_PCR_CAR_SHIFT 24
+#define KVX_SFR_PCR_CAR_WIDTH 4
+#define KVX_SFR_PCR_CAR_WFXL_MASK _ULL(0xf000000)
+#define KVX_SFR_PCR_CAR_WFXL_CLEAR _ULL(0xf000000)
+#define KVX_SFR_PCR_CAR_WFXL_SET _ULL(0xf00000000000000)
+
+#define KVX_SFR_PCR_CMA_MASK _ULL(0xf0000000) /* Core Micro-Architecture Revision ID */
+#define KVX_SFR_PCR_CMA_SHIFT 28
+#define KVX_SFR_PCR_CMA_WIDTH 4
+#define KVX_SFR_PCR_CMA_WFXL_MASK _ULL(0xf0000000)
+#define KVX_SFR_PCR_CMA_WFXL_CLEAR _ULL(0xf0000000)
+#define KVX_SFR_PCR_CMA_WFXL_SET _ULL(0xf000000000000000)
+
+#define KVX_SFR_PCR_SV_MASK _ULL(0xff00000000) /* System-On-Chip Version */
+#define KVX_SFR_PCR_SV_SHIFT 32
+#define KVX_SFR_PCR_SV_WIDTH 8
+#define KVX_SFR_PCR_SV_WFXM_MASK _ULL(0xff00000000)
+#define KVX_SFR_PCR_SV_WFXM_CLEAR _ULL(0xff)
+#define KVX_SFR_PCR_SV_WFXM_SET _ULL(0xff00000000)
+
+#define KVX_SFR_PCR_ST_MASK _ULL(0xf0000000000) /* System-On-Chip Type */
+#define KVX_SFR_PCR_ST_SHIFT 40
+#define KVX_SFR_PCR_ST_WIDTH 4
+#define KVX_SFR_PCR_ST_WFXM_MASK _ULL(0xf0000000000)
+#define KVX_SFR_PCR_ST_WFXM_CLEAR _ULL(0xf00)
+#define KVX_SFR_PCR_ST_WFXM_SET _ULL(0xf0000000000)
+
+#define KVX_SFR_PCR_BM_MASK _ULL(0xff00000000000) /* Boot Mode */
+#define KVX_SFR_PCR_BM_SHIFT 44
+#define KVX_SFR_PCR_BM_WIDTH 8
+#define KVX_SFR_PCR_BM_WFXM_MASK _ULL(0xff00000000000)
+#define KVX_SFR_PCR_BM_WFXM_CLEAR _ULL(0xff000)
+#define KVX_SFR_PCR_BM_WFXM_SET _ULL(0xff00000000000)
+
+#define KVX_SFR_PCR_COE_MASK _ULL(0x10000000000000) /* COprocessor Enable */
+#define KVX_SFR_PCR_COE_SHIFT 52
+#define KVX_SFR_PCR_COE_WIDTH 1
+#define KVX_SFR_PCR_COE_WFXM_MASK _ULL(0x10000000000000)
+#define KVX_SFR_PCR_COE_WFXM_CLEAR _ULL(0x100000)
+#define KVX_SFR_PCR_COE_WFXM_SET _ULL(0x10000000000000)
+
+#define KVX_SFR_PCR_L1CE_MASK _ULL(0x20000000000000) /* L1 cache Coherency Enable */
+#define KVX_SFR_PCR_L1CE_SHIFT 53
+#define KVX_SFR_PCR_L1CE_WIDTH 1
+#define KVX_SFR_PCR_L1CE_WFXM_MASK _ULL(0x20000000000000)
+#define KVX_SFR_PCR_L1CE_WFXM_CLEAR _ULL(0x200000)
+#define KVX_SFR_PCR_L1CE_WFXM_SET _ULL(0x20000000000000)
+
+#define KVX_SFR_PCR_DSEM_MASK _ULL(0x40000000000000) /* Data Simple Ecc exception Mode */
+#define KVX_SFR_PCR_DSEM_SHIFT 54
+#define KVX_SFR_PCR_DSEM_WIDTH 1
+#define KVX_SFR_PCR_DSEM_WFXM_MASK _ULL(0x40000000000000)
+#define KVX_SFR_PCR_DSEM_WFXM_CLEAR _ULL(0x400000)
+#define KVX_SFR_PCR_DSEM_WFXM_SET _ULL(0x40000000000000)
+
+#define KVX_SFR_MMC_ASN_MASK _ULL(0x1ff) /* Address Space Number */
+#define KVX_SFR_MMC_ASN_SHIFT 0
+#define KVX_SFR_MMC_ASN_WIDTH 9
+#define KVX_SFR_MMC_ASN_WFXL_MASK _ULL(0x1ff)
+#define KVX_SFR_MMC_ASN_WFXL_CLEAR _ULL(0x1ff)
+#define KVX_SFR_MMC_ASN_WFXL_SET _ULL(0x1ff00000000)
+
+#define KVX_SFR_MMC_S_MASK _ULL(0x200) /* Speculative */
+#define KVX_SFR_MMC_S_SHIFT 9
+#define KVX_SFR_MMC_S_WIDTH 1
+#define KVX_SFR_MMC_S_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_MMC_S_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_MMC_S_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_MMC_SNE_MASK _ULL(0x4000) /* Speculative NOMAPPING Enable */
+#define KVX_SFR_MMC_SNE_SHIFT 14
+#define KVX_SFR_MMC_SNE_WIDTH 1
+#define KVX_SFR_MMC_SNE_WFXL_MASK _ULL(0x4000)
+#define KVX_SFR_MMC_SNE_WFXL_CLEAR _ULL(0x4000)
+#define KVX_SFR_MMC_SNE_WFXL_SET _ULL(0x400000000000)
+
+#define KVX_SFR_MMC_SPE_MASK _ULL(0x8000) /* Speculative PROTECTION Enable */
+#define KVX_SFR_MMC_SPE_SHIFT 15
+#define KVX_SFR_MMC_SPE_WIDTH 1
+#define KVX_SFR_MMC_SPE_WFXL_MASK _ULL(0x8000)
+#define KVX_SFR_MMC_SPE_WFXL_CLEAR _ULL(0x8000)
+#define KVX_SFR_MMC_SPE_WFXL_SET _ULL(0x800000000000)
+
+#define KVX_SFR_MMC_PTC_MASK _ULL(0x30000) /* Protection Trap Cause */
+#define KVX_SFR_MMC_PTC_SHIFT 16
+#define KVX_SFR_MMC_PTC_WIDTH 2
+#define KVX_SFR_MMC_PTC_WFXL_MASK _ULL(0x30000)
+#define KVX_SFR_MMC_PTC_WFXL_CLEAR _ULL(0x30000)
+#define KVX_SFR_MMC_PTC_WFXL_SET _ULL(0x3000000000000)
+
+#define KVX_SFR_MMC_SW_MASK _ULL(0x3c0000) /* Select Way */
+#define KVX_SFR_MMC_SW_SHIFT 18
+#define KVX_SFR_MMC_SW_WIDTH 4
+#define KVX_SFR_MMC_SW_WFXL_MASK _ULL(0x3c0000)
+#define KVX_SFR_MMC_SW_WFXL_CLEAR _ULL(0x3c0000)
+#define KVX_SFR_MMC_SW_WFXL_SET _ULL(0x3c000000000000)
+
+#define KVX_SFR_MMC_SS_MASK _ULL(0xfc00000) /* Select Set */
+#define KVX_SFR_MMC_SS_SHIFT 22
+#define KVX_SFR_MMC_SS_WIDTH 6
+#define KVX_SFR_MMC_SS_WFXL_MASK _ULL(0xfc00000)
+#define KVX_SFR_MMC_SS_WFXL_CLEAR _ULL(0xfc00000)
+#define KVX_SFR_MMC_SS_WFXL_SET _ULL(0xfc0000000000000)
+
+#define KVX_SFR_MMC_SB_MASK _ULL(0x10000000) /* Select Buffer */
+#define KVX_SFR_MMC_SB_SHIFT 28
+#define KVX_SFR_MMC_SB_WIDTH 1
+#define KVX_SFR_MMC_SB_WFXL_MASK _ULL(0x10000000)
+#define KVX_SFR_MMC_SB_WFXL_CLEAR _ULL(0x10000000)
+#define KVX_SFR_MMC_SB_WFXL_SET _ULL(0x1000000000000000)
+
+#define KVX_SFR_MMC_PAR_MASK _ULL(0x40000000) /* PARity error flag */
+#define KVX_SFR_MMC_PAR_SHIFT 30
+#define KVX_SFR_MMC_PAR_WIDTH 1
+#define KVX_SFR_MMC_PAR_WFXL_MASK _ULL(0x40000000)
+#define KVX_SFR_MMC_PAR_WFXL_CLEAR _ULL(0x40000000)
+#define KVX_SFR_MMC_PAR_WFXL_SET _ULL(0x4000000000000000)
+
+#define KVX_SFR_MMC_E_MASK _ULL(0x80000000) /* Error Flag */
+#define KVX_SFR_MMC_E_SHIFT 31
+#define KVX_SFR_MMC_E_WIDTH 1
+#define KVX_SFR_MMC_E_WFXL_MASK _ULL(0x80000000)
+#define KVX_SFR_MMC_E_WFXL_CLEAR _ULL(0x80000000)
+#define KVX_SFR_MMC_E_WFXL_SET _ULL(0x8000000000000000)
+
+#define KVX_SFR_TEL_ES_MASK _ULL(0x3) /* Entry Status */
+#define KVX_SFR_TEL_ES_SHIFT 0
+#define KVX_SFR_TEL_ES_WIDTH 2
+#define KVX_SFR_TEL_ES_WFXL_MASK _ULL(0x3)
+#define KVX_SFR_TEL_ES_WFXL_CLEAR _ULL(0x3)
+#define KVX_SFR_TEL_ES_WFXL_SET _ULL(0x300000000)
+
+#define KVX_SFR_TEL_CP_MASK _ULL(0xc) /* Cache Policy */
+#define KVX_SFR_TEL_CP_SHIFT 2
+#define KVX_SFR_TEL_CP_WIDTH 2
+#define KVX_SFR_TEL_CP_WFXL_MASK _ULL(0xc)
+#define KVX_SFR_TEL_CP_WFXL_CLEAR _ULL(0xc)
+#define KVX_SFR_TEL_CP_WFXL_SET _ULL(0xc00000000)
+
+#define KVX_SFR_TEL_PA_MASK _ULL(0xf0) /* Protection Attributes */
+#define KVX_SFR_TEL_PA_SHIFT 4
+#define KVX_SFR_TEL_PA_WIDTH 4
+#define KVX_SFR_TEL_PA_WFXL_MASK _ULL(0xf0)
+#define KVX_SFR_TEL_PA_WFXL_CLEAR _ULL(0xf0)
+#define KVX_SFR_TEL_PA_WFXL_SET _ULL(0xf000000000)
+
+#define KVX_SFR_TEL_PS_MASK _ULL(0xc00) /* Page Size */
+#define KVX_SFR_TEL_PS_SHIFT 10
+#define KVX_SFR_TEL_PS_WIDTH 2
+#define KVX_SFR_TEL_PS_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_TEL_PS_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_TEL_PS_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_TEL_FN_MASK _ULL(0xfffffff000) /* Frame Number */
+#define KVX_SFR_TEL_FN_SHIFT 12
+#define KVX_SFR_TEL_FN_WIDTH 28
+#define KVX_SFR_TEL_FN_WFXL_MASK _ULL(0xfffff000)
+#define KVX_SFR_TEL_FN_WFXL_CLEAR _ULL(0xfffff000)
+#define KVX_SFR_TEL_FN_WFXL_SET _ULL(0xfffff00000000000)
+#define KVX_SFR_TEL_FN_WFXM_MASK _ULL(0xff00000000)
+#define KVX_SFR_TEL_FN_WFXM_CLEAR _ULL(0xff)
+#define KVX_SFR_TEL_FN_WFXM_SET _ULL(0xff00000000)
+
+#define KVX_SFR_TEH_ASN_MASK _ULL(0x1ff) /* Address Space Number */
+#define KVX_SFR_TEH_ASN_SHIFT 0
+#define KVX_SFR_TEH_ASN_WIDTH 9
+#define KVX_SFR_TEH_ASN_WFXL_MASK _ULL(0x1ff)
+#define KVX_SFR_TEH_ASN_WFXL_CLEAR _ULL(0x1ff)
+#define KVX_SFR_TEH_ASN_WFXL_SET _ULL(0x1ff00000000)
+
+#define KVX_SFR_TEH_G_MASK _ULL(0x200) /* Global page indicator */
+#define KVX_SFR_TEH_G_SHIFT 9
+#define KVX_SFR_TEH_G_WIDTH 1
+#define KVX_SFR_TEH_G_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_TEH_G_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_TEH_G_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_TEH_VS_MASK _ULL(0xc00) /* Virtual Space */
+#define KVX_SFR_TEH_VS_SHIFT 10
+#define KVX_SFR_TEH_VS_WIDTH 2
+#define KVX_SFR_TEH_VS_WFXL_MASK _ULL(0xc00)
+#define KVX_SFR_TEH_VS_WFXL_CLEAR _ULL(0xc00)
+#define KVX_SFR_TEH_VS_WFXL_SET _ULL(0xc0000000000)
+
+#define KVX_SFR_TEH_PN_MASK _ULL(0x1fffffff000) /* Page Number */
+#define KVX_SFR_TEH_PN_SHIFT 12
+#define KVX_SFR_TEH_PN_WIDTH 29
+#define KVX_SFR_TEH_PN_WFXL_MASK _ULL(0xfffff000)
+#define KVX_SFR_TEH_PN_WFXL_CLEAR _ULL(0xfffff000)
+#define KVX_SFR_TEH_PN_WFXL_SET _ULL(0xfffff00000000000)
+#define KVX_SFR_TEH_PN_WFXM_MASK _ULL(0x1ff00000000)
+#define KVX_SFR_TEH_PN_WFXM_CLEAR _ULL(0x1ff)
+#define KVX_SFR_TEH_PN_WFXM_SET _ULL(0x1ff00000000)
+
+#define KVX_SFR_DC_BE0_MASK _ULL(0x1) /* Breakpoint 0 Enable */
+#define KVX_SFR_DC_BE0_SHIFT 0
+#define KVX_SFR_DC_BE0_WIDTH 1
+#define KVX_SFR_DC_BE0_WFXL_MASK _ULL(0x1)
+#define KVX_SFR_DC_BE0_WFXL_CLEAR _ULL(0x1)
+#define KVX_SFR_DC_BE0_WFXL_SET _ULL(0x100000000)
+
+#define KVX_SFR_DC_BR0_MASK _ULL(0x7e) /* Breakpoint 0 Range */
+#define KVX_SFR_DC_BR0_SHIFT 1
+#define KVX_SFR_DC_BR0_WIDTH 6
+#define KVX_SFR_DC_BR0_WFXL_MASK _ULL(0x7e)
+#define KVX_SFR_DC_BR0_WFXL_CLEAR _ULL(0x7e)
+#define KVX_SFR_DC_BR0_WFXL_SET _ULL(0x7e00000000)
+
+#define KVX_SFR_DC_BE1_MASK _ULL(0x80) /* Breakpoint 1 Enable */
+#define KVX_SFR_DC_BE1_SHIFT 7
+#define KVX_SFR_DC_BE1_WIDTH 1
+#define KVX_SFR_DC_BE1_WFXL_MASK _ULL(0x80)
+#define KVX_SFR_DC_BE1_WFXL_CLEAR _ULL(0x80)
+#define KVX_SFR_DC_BE1_WFXL_SET _ULL(0x8000000000)
+
+#define KVX_SFR_DC_BR1_MASK _ULL(0x3f00) /* Breakpoint 1 Range */
+#define KVX_SFR_DC_BR1_SHIFT 8
+#define KVX_SFR_DC_BR1_WIDTH 6
+#define KVX_SFR_DC_BR1_WFXL_MASK _ULL(0x3f00)
+#define KVX_SFR_DC_BR1_WFXL_CLEAR _ULL(0x3f00)
+#define KVX_SFR_DC_BR1_WFXL_SET _ULL(0x3f0000000000)
+
+#define KVX_SFR_DC_WE0_MASK _ULL(0x4000) /* Watchpoint 0 Enable */
+#define KVX_SFR_DC_WE0_SHIFT 14
+#define KVX_SFR_DC_WE0_WIDTH 1
+#define KVX_SFR_DC_WE0_WFXL_MASK _ULL(0x4000)
+#define KVX_SFR_DC_WE0_WFXL_CLEAR _ULL(0x4000)
+#define KVX_SFR_DC_WE0_WFXL_SET _ULL(0x400000000000)
+
+#define KVX_SFR_DC_WR0_MASK _ULL(0x1f8000) /* Watchpoint 0 Range */
+#define KVX_SFR_DC_WR0_SHIFT 15
+#define KVX_SFR_DC_WR0_WIDTH 6
+#define KVX_SFR_DC_WR0_WFXL_MASK _ULL(0x1f8000)
+#define KVX_SFR_DC_WR0_WFXL_CLEAR _ULL(0x1f8000)
+#define KVX_SFR_DC_WR0_WFXL_SET _ULL(0x1f800000000000)
+
+#define KVX_SFR_DC_WE1_MASK _ULL(0x200000) /* Watchpoint 1 Enable */
+#define KVX_SFR_DC_WE1_SHIFT 21
+#define KVX_SFR_DC_WE1_WIDTH 1
+#define KVX_SFR_DC_WE1_WFXL_MASK _ULL(0x200000)
+#define KVX_SFR_DC_WE1_WFXL_CLEAR _ULL(0x200000)
+#define KVX_SFR_DC_WE1_WFXL_SET _ULL(0x20000000000000)
+
+#define KVX_SFR_DC_WR1_MASK _ULL(0xfc00000) /* Watchpoint 1 Range */
+#define KVX_SFR_DC_WR1_SHIFT 22
+#define KVX_SFR_DC_WR1_WIDTH 6
+#define KVX_SFR_DC_WR1_WFXL_MASK _ULL(0xfc00000)
+#define KVX_SFR_DC_WR1_WFXL_CLEAR _ULL(0xfc00000)
+#define KVX_SFR_DC_WR1_WFXL_SET _ULL(0xfc0000000000000)
+
+#define KVX_SFR_MES_PSE_MASK _ULL(0x1) /* Program Simple Ecc */
+#define KVX_SFR_MES_PSE_SHIFT 0
+#define KVX_SFR_MES_PSE_WIDTH 1
+#define KVX_SFR_MES_PSE_WFXL_MASK _ULL(0x1)
+#define KVX_SFR_MES_PSE_WFXL_CLEAR _ULL(0x1)
+#define KVX_SFR_MES_PSE_WFXL_SET _ULL(0x100000000)
+
+#define KVX_SFR_MES_PILSY_MASK _ULL(0x2) /* Program cache Invalidated Line following pSYs error. */
+#define KVX_SFR_MES_PILSY_SHIFT 1
+#define KVX_SFR_MES_PILSY_WIDTH 1
+#define KVX_SFR_MES_PILSY_WFXL_MASK _ULL(0x2)
+#define KVX_SFR_MES_PILSY_WFXL_CLEAR _ULL(0x2)
+#define KVX_SFR_MES_PILSY_WFXL_SET _ULL(0x200000000)
+
+#define KVX_SFR_MES_PILDE_MASK _ULL(0x4) /* Program cache Invalidated Line following pDEcc error. */
+#define KVX_SFR_MES_PILDE_SHIFT 2
+#define KVX_SFR_MES_PILDE_WIDTH 1
+#define KVX_SFR_MES_PILDE_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_MES_PILDE_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_MES_PILDE_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_MES_PILPA_MASK _ULL(0x8) /* Program cache Invalidated Line following pPArity error. */
+#define KVX_SFR_MES_PILPA_SHIFT 3
+#define KVX_SFR_MES_PILPA_WIDTH 1
+#define KVX_SFR_MES_PILPA_WFXL_MASK _ULL(0x8)
+#define KVX_SFR_MES_PILPA_WFXL_CLEAR _ULL(0x8)
+#define KVX_SFR_MES_PILPA_WFXL_SET _ULL(0x800000000)
+
+#define KVX_SFR_MES_DSE_MASK _ULL(0x10) /* Data Simple Ecc */
+#define KVX_SFR_MES_DSE_SHIFT 4
+#define KVX_SFR_MES_DSE_WIDTH 1
+#define KVX_SFR_MES_DSE_WFXL_MASK _ULL(0x10)
+#define KVX_SFR_MES_DSE_WFXL_CLEAR _ULL(0x10)
+#define KVX_SFR_MES_DSE_WFXL_SET _ULL(0x1000000000)
+
+#define KVX_SFR_MES_DILSY_MASK _ULL(0x20) /* Data cache Invalidated Line following dSYs error. */
+#define KVX_SFR_MES_DILSY_SHIFT 5
+#define KVX_SFR_MES_DILSY_WIDTH 1
+#define KVX_SFR_MES_DILSY_WFXL_MASK _ULL(0x20)
+#define KVX_SFR_MES_DILSY_WFXL_CLEAR _ULL(0x20)
+#define KVX_SFR_MES_DILSY_WFXL_SET _ULL(0x2000000000)
+
+#define KVX_SFR_MES_DILDE_MASK _ULL(0x40) /* Data cache Invalidated Line following dDEcc error. */
+#define KVX_SFR_MES_DILDE_SHIFT 6
+#define KVX_SFR_MES_DILDE_WIDTH 1
+#define KVX_SFR_MES_DILDE_WFXL_MASK _ULL(0x40)
+#define KVX_SFR_MES_DILDE_WFXL_CLEAR _ULL(0x40)
+#define KVX_SFR_MES_DILDE_WFXL_SET _ULL(0x4000000000)
+
+#define KVX_SFR_MES_DILPA_MASK _ULL(0x80) /* Data cache Invalidated Line following dPArity error. */
+#define KVX_SFR_MES_DILPA_SHIFT 7
+#define KVX_SFR_MES_DILPA_WIDTH 1
+#define KVX_SFR_MES_DILPA_WFXL_MASK _ULL(0x80)
+#define KVX_SFR_MES_DILPA_WFXL_CLEAR _ULL(0x80)
+#define KVX_SFR_MES_DILPA_WFXL_SET _ULL(0x8000000000)
+
+#define KVX_SFR_MES_DDEE_MASK _ULL(0x100) /* Data DEcc Error. */
+#define KVX_SFR_MES_DDEE_SHIFT 8
+#define KVX_SFR_MES_DDEE_WIDTH 1
+#define KVX_SFR_MES_DDEE_WFXL_MASK _ULL(0x100)
+#define KVX_SFR_MES_DDEE_WFXL_CLEAR _ULL(0x100)
+#define KVX_SFR_MES_DDEE_WFXL_SET _ULL(0x10000000000)
+
+#define KVX_SFR_MES_DSYE_MASK _ULL(0x200) /* Data dSYs Error. */
+#define KVX_SFR_MES_DSYE_SHIFT 9
+#define KVX_SFR_MES_DSYE_WIDTH 1
+#define KVX_SFR_MES_DSYE_WFXL_MASK _ULL(0x200)
+#define KVX_SFR_MES_DSYE_WFXL_CLEAR _ULL(0x200)
+#define KVX_SFR_MES_DSYE_WFXL_SET _ULL(0x20000000000)
+
+#define KVX_SFR_WS_WU0_MASK _ULL(0x1) /* Wake-Up 0 */
+#define KVX_SFR_WS_WU0_SHIFT 0
+#define KVX_SFR_WS_WU0_WIDTH 1
+#define KVX_SFR_WS_WU0_WFXL_MASK _ULL(0x1)
+#define KVX_SFR_WS_WU0_WFXL_CLEAR _ULL(0x1)
+#define KVX_SFR_WS_WU0_WFXL_SET _ULL(0x100000000)
+
+#define KVX_SFR_WS_WU1_MASK _ULL(0x2) /* Wake-Up 1 */
+#define KVX_SFR_WS_WU1_SHIFT 1
+#define KVX_SFR_WS_WU1_WIDTH 1
+#define KVX_SFR_WS_WU1_WFXL_MASK _ULL(0x2)
+#define KVX_SFR_WS_WU1_WFXL_CLEAR _ULL(0x2)
+#define KVX_SFR_WS_WU1_WFXL_SET _ULL(0x200000000)
+
+#define KVX_SFR_WS_WU2_MASK _ULL(0x4) /* Wake-Up 2 */
+#define KVX_SFR_WS_WU2_SHIFT 2
+#define KVX_SFR_WS_WU2_WIDTH 1
+#define KVX_SFR_WS_WU2_WFXL_MASK _ULL(0x4)
+#define KVX_SFR_WS_WU2_WFXL_CLEAR _ULL(0x4)
+#define KVX_SFR_WS_WU2_WFXL_SET _ULL(0x400000000)
+
+#define KVX_SFR_IPE_FE_MASK _ULL(0xffff) /* Forward Events */
+#define KVX_SFR_IPE_FE_SHIFT 0
+#define KVX_SFR_IPE_FE_WIDTH 16
+#define KVX_SFR_IPE_FE_WFXL_MASK _ULL(0xffff)
+#define KVX_SFR_IPE_FE_WFXL_CLEAR _ULL(0xffff)
+#define KVX_SFR_IPE_FE_WFXL_SET _ULL(0xffff00000000)
+
+#define KVX_SFR_IPE_BE_MASK _ULL(0xffff0000) /* Backward Events */
+#define KVX_SFR_IPE_BE_SHIFT 16
+#define KVX_SFR_IPE_BE_WIDTH 16
+#define KVX_SFR_IPE_BE_WFXL_MASK _ULL(0xffff0000)
+#define KVX_SFR_IPE_BE_WFXL_CLEAR _ULL(0xffff0000)
+#define KVX_SFR_IPE_BE_WFXL_SET _ULL(0xffff000000000000)
+
+#define KVX_SFR_IPE_FM_MASK _ULL(0xffff00000000) /* Forward Mode */
+#define KVX_SFR_IPE_FM_SHIFT 32
+#define KVX_SFR_IPE_FM_WIDTH 16
+#define KVX_SFR_IPE_FM_WFXM_MASK _ULL(0xffff00000000)
+#define KVX_SFR_IPE_FM_WFXM_CLEAR _ULL(0xffff)
+#define KVX_SFR_IPE_FM_WFXM_SET _ULL(0xffff00000000)
+
+#define KVX_SFR_IPE_BM_MASK _ULL(0xffff000000000000) /* Backward Modes */
+#define KVX_SFR_IPE_BM_SHIFT 48
+#define KVX_SFR_IPE_BM_WIDTH 16
+#define KVX_SFR_IPE_BM_WFXM_MASK _ULL(0xffff000000000000)
+#define KVX_SFR_IPE_BM_WFXM_CLEAR _ULL(0xffff0000)
+#define KVX_SFR_IPE_BM_WFXM_SET _ULL(0xffff000000000000)
+
+#endif /*_ASM_KVX_SFR_DEFS_H */
diff --git a/arch/kvx/include/asm/swab.h b/arch/kvx/include/asm/swab.h
new file mode 100644
index 000000000000..83a48871c8fb
--- /dev/null
+++ b/arch/kvx/include/asm/swab.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SWAB_H
+#define _ASM_KVX_SWAB_H
+
+#include <linux/types.h>
+
+#define U64_BYTE_SWAP_MATRIX 0x0102040810204080ULL
+#define U32_BYTE_SWAP_MATRIX 0x0000000001020408ULL
+#define U16_BYTE_SWAP_MATRIX 0x0000000000000102ULL
+#define U32_WORD_SWAP_MATRIX 0x0000000002010804ULL
+#define U32_HL_BYTE_SWAP_MATRIX 0x0000000004080102ULL
+
+static inline __attribute_const__ __u64 __arch_swab64(__u64 val)
+{
+ return __builtin_kvx_sbmm8(val, U64_BYTE_SWAP_MATRIX);
+}
+
+static inline __attribute_const__ __u32 __arch_swab32(__u32 val)
+{
+ return __builtin_kvx_sbmm8(val, U32_BYTE_SWAP_MATRIX);
+}
+
+static inline __attribute_const__ __u16 __arch_swab16(__u16 val)
+{
+ return __builtin_kvx_sbmm8(val, U16_BYTE_SWAP_MATRIX);
+}
+
+static inline __attribute_const__ __u32 __arch_swahw32(__u32 val)
+{
+ return __builtin_kvx_sbmm8(val, U32_WORD_SWAP_MATRIX);
+}
+
+static inline __attribute_const__ __u32 __arch_swahb32(__u32 val)
+{
+ return __builtin_kvx_sbmm8(val, U32_HL_BYTE_SWAP_MATRIX);
+}
+
+#define __arch_swab64 __arch_swab64
+#define __arch_swab32 __arch_swab32
+#define __arch_swab16 __arch_swab16
+#define __arch_swahw32 __arch_swahw32
+#define __arch_swahb32 __arch_swahb32
+#endif
diff --git a/arch/kvx/include/asm/sys_arch.h b/arch/kvx/include/asm/sys_arch.h
new file mode 100644
index 000000000000..ae518106a0a8
--- /dev/null
+++ b/arch/kvx/include/asm/sys_arch.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SYS_ARCH_H
+#define _ASM_KVX_SYS_ARCH_H
+
+#include <asm/sfr_defs.h>
+
+#define EXCEPTION_STRIDE 0x40
+#define EXCEPTION_ALIGNMENT 0x100
+
+#define kvx_cluster_id() ((int) \
+ ((kvx_sfr_get(PCR) & KVX_SFR_PCR_CID_MASK) \
+ >> KVX_SFR_PCR_CID_SHIFT))
+
+#define KVX_SFR_START(__sfr_reg) \
+ (KVX_SFR_## __sfr_reg ## _SHIFT)
+
+#define KVX_SFR_END(__sfr_reg) \
+ (KVX_SFR_## __sfr_reg ## _SHIFT + KVX_SFR_## __sfr_reg ## _WIDTH - 1)
+
+/**
+ * Get the value to clear a sfr
+ */
+#define SFR_CLEAR(__sfr, __field, __lm) \
+ KVX_SFR_## __sfr ## _ ## __field ## _ ## __lm ## _CLEAR
+
+#define SFR_CLEAR_WFXL(__sfr, __field) SFR_CLEAR(__sfr, __field, WFXL)
+#define SFR_CLEAR_WFXM(__sfr, __field) SFR_CLEAR(__sfr, __field, WFXM)
+
+/**
+ * Get the value to set a sfr.
+ */
+#define SFR_SET_WFXL(__sfr, __field, __val) \
+ (__val << (KVX_SFR_ ## __sfr ## _ ## __field ## _SHIFT + 32))
+
+#define SFR_SET_WFXM(__sfr, __field, __val) \
+ (__val << (KVX_SFR_ ## __sfr ## _ ## __field ## _SHIFT))
+
+/**
+ * Generate the mask to clear and set a value using wfx{m|l}.
+ */
+#define SFR_SET_VAL_WFXL(__sfr, __field, __val) \
+ (SFR_SET_WFXL(__sfr, __field, __val) | SFR_CLEAR_WFXL(__sfr, __field))
+#define SFR_SET_VAL_WFXM(__sfr, __field, __val) \
+ (SFR_SET_WFXM(__sfr, __field, __val) | SFR_CLEAR_WFXM(__sfr, __field))
+
+#endif /* _ASM_KVX_SYS_ARCH_H */
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 05/25] kvx: Add atomic/locking headers
2023-01-03 16:43 ` Yann Sionneau
` (5 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-04 9:53 ` Mark Rutland
-1 siblings, 1 reply; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Will Deacon, Peter Zijlstra, Boqun Feng,
Mark Rutland, linux-kernel, Clement Leger, Jules Maselbas,
Julian Vetter, Julien Villette
Add common headers (atomic, bitops, barrier and locking) for basic
kvx support.
CC: Will Deacon <will@kernel.org>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Boqun Feng <boqun.feng@gmail.com>
CC: Mark Rutland <mark.rutland@arm.com>
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Julien Villette <jvillette@kalray.eu>
Signed-off-by: Julien Villette <jvillette@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/atomic.h | 104 +++++++++++++++++
arch/kvx/include/asm/barrier.h | 15 +++
arch/kvx/include/asm/bitops.h | 207 +++++++++++++++++++++++++++++++++
arch/kvx/include/asm/bitrev.h | 32 +++++
arch/kvx/include/asm/cmpxchg.h | 185 +++++++++++++++++++++++++++++
5 files changed, 543 insertions(+)
create mode 100644 arch/kvx/include/asm/atomic.h
create mode 100644 arch/kvx/include/asm/barrier.h
create mode 100644 arch/kvx/include/asm/bitops.h
create mode 100644 arch/kvx/include/asm/bitrev.h
create mode 100644 arch/kvx/include/asm/cmpxchg.h
diff --git a/arch/kvx/include/asm/atomic.h b/arch/kvx/include/asm/atomic.h
new file mode 100644
index 000000000000..eb8acbcbc70d
--- /dev/null
+++ b/arch/kvx/include/asm/atomic.h
@@ -0,0 +1,104 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_ATOMIC_H
+#define _ASM_KVX_ATOMIC_H
+
+#include <linux/types.h>
+
+#include <asm/cmpxchg.h>
+
+#define ATOMIC64_INIT(i) { (i) }
+
+#define arch_atomic64_cmpxchg(v, old, new) (arch_cmpxchg(&((v)->counter), old, new))
+#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), new))
+
+static inline long arch_atomic64_read(const atomic64_t *v)
+{
+ return v->counter;
+}
+
+static inline void arch_atomic64_set(atomic64_t *v, long i)
+{
+ v->counter = i;
+}
+
+#define ATOMIC64_RETURN_OP(op, c_op) \
+static inline long arch_atomic64_##op##_return(long i, atomic64_t *v) \
+{ \
+ long new, old, ret; \
+ \
+ do { \
+ old = v->counter; \
+ new = old c_op i; \
+ ret = arch_cmpxchg(&v->counter, old, new); \
+ } while (ret != old); \
+ \
+ return new; \
+}
+
+#define ATOMIC64_OP(op, c_op) \
+static inline void arch_atomic64_##op(long i, atomic64_t *v) \
+{ \
+ long new, old, ret; \
+ \
+ do { \
+ old = v->counter; \
+ new = old c_op i; \
+ ret = arch_cmpxchg(&v->counter, old, new); \
+ } while (ret != old); \
+}
+
+#define ATOMIC64_FETCH_OP(op, c_op) \
+static inline long arch_atomic64_fetch_##op(long i, atomic64_t *v) \
+{ \
+ long new, old, ret; \
+ \
+ do { \
+ old = v->counter; \
+ new = old c_op i; \
+ ret = arch_cmpxchg(&v->counter, old, new); \
+ } while (ret != old); \
+ \
+ return old; \
+}
+
+#define ATOMIC64_OPS(op, c_op) \
+ ATOMIC64_OP(op, c_op) \
+ ATOMIC64_RETURN_OP(op, c_op) \
+ ATOMIC64_FETCH_OP(op, c_op)
+
+ATOMIC64_OPS(and, &)
+ATOMIC64_OPS(or, |)
+ATOMIC64_OPS(xor, ^)
+ATOMIC64_OPS(add, +)
+ATOMIC64_OPS(sub, -)
+
+#undef ATOMIC64_OPS
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP
+
+static inline int arch_atomic_add_return(int i, atomic_t *v)
+{
+ int new, old, ret;
+
+ do {
+ old = v->counter;
+ new = old + i;
+ ret = arch_cmpxchg(&v->counter, old, new);
+ } while (ret != old);
+
+ return new;
+}
+
+static inline int arch_atomic_sub_return(int i, atomic_t *v)
+{
+ return arch_atomic_add_return(-i, v);
+}
+
+#include <asm-generic/atomic.h>
+
+#endif /* _ASM_KVX_ATOMIC_H */
diff --git a/arch/kvx/include/asm/barrier.h b/arch/kvx/include/asm/barrier.h
new file mode 100644
index 000000000000..371f1c70746d
--- /dev/null
+++ b/arch/kvx/include/asm/barrier.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_BARRIER_H
+#define _ASM_KVX_BARRIER_H
+
+/* fence is sufficient to guarantee write ordering */
+#define mb() __builtin_kvx_fence()
+
+#include <asm-generic/barrier.h>
+
+#endif /* _ASM_KVX_BARRIER_H */
diff --git a/arch/kvx/include/asm/bitops.h b/arch/kvx/include/asm/bitops.h
new file mode 100644
index 000000000000..03b861169ce8
--- /dev/null
+++ b/arch/kvx/include/asm/bitops.h
@@ -0,0 +1,207 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ */
+
+#ifndef _ASM_KVX_BITOPS_H
+#define _ASM_KVX_BITOPS_H
+
+#ifdef __KERNEL__
+
+#ifndef _LINUX_BITOPS_H
+#error only <linux/bitops.h> can be included directly
+#endif
+
+#include <asm/cmpxchg.h>
+
+static inline unsigned long __ffs(unsigned long word);
+
+#include <asm-generic/bitops/const_hweight.h>
+#include <asm-generic/bitops/ffz.h>
+#include <asm-generic/bitops/non-atomic.h>
+#include <asm-generic/bitops/lock.h>
+#include <asm-generic/bitops/sched.h>
+
+static inline int fls(int x)
+{
+ return 32 - __builtin_kvx_clzw(x);
+}
+
+static inline int fls64(__u64 x)
+{
+ return 64 - __builtin_kvx_clzd(x);
+}
+
+/**
+ * __ffs - find first set bit in word
+ * @word: The word to search
+ *
+ * Undefined if no set bit exists, so code should check against 0 first.
+ */
+static inline unsigned long __ffs(unsigned long word)
+{
+ return __builtin_kvx_ctzd(word);
+}
+
+/**
+ * __fls - find last set bit in word
+ * @word: The word to search
+ *
+ * Undefined if no set bit exists, so code should check against 0 first.
+ */
+static inline unsigned long __fls(unsigned long word)
+{
+ return 63 - __builtin_kvx_clzd(word);
+}
+
+
+/**
+ * ffs - find first set bit in word
+ * @x: the word to search
+ *
+ * This is defined the same way as the libc and compiler builtin ffs
+ * routines, therefore differs in spirit from the other bitops.
+ *
+ * ffs(value) returns 0 if value is 0 or the position of the first
+ * set bit if value is nonzero. The first (least significant) bit
+ * is at position 1.
+ */
+static inline int ffs(int x)
+{
+ if (!x)
+ return 0;
+ return __builtin_kvx_ctzw(x) + 1;
+}
+
+
+static inline unsigned int __arch_hweight32(unsigned int w)
+{
+ unsigned int count;
+
+ asm volatile ("cbsw %0 = %1\n\t;;"
+ : "=r" (count)
+ : "r" (w));
+
+ return count;
+}
+
+static inline unsigned int __arch_hweight64(__u64 w)
+{
+ unsigned int count;
+
+ asm volatile ("cbsd %0 = %1\n\t;;"
+ : "=r" (count)
+ : "r" (w));
+
+ return count;
+}
+
+static inline unsigned int __arch_hweight16(unsigned int w)
+{
+ return __arch_hweight32(w & 0xffff);
+}
+
+static inline unsigned int __arch_hweight8(unsigned int w)
+{
+ return __arch_hweight32(w & 0xff);
+}
+
+
+/* Bitmask modifiers */
+#define __NOP(x) (x)
+#define __NOT(x) (~(x))
+
+
+#define __test_and_op_bit(nr, addr, op, mod) \
+({ \
+ unsigned long __mask = BIT_MASK(nr); \
+ unsigned long __new, __old, __ret; \
+ do { \
+ __old = *(&addr[BIT_WORD(nr)]); \
+ __new = __old op mod(__mask); \
+ __ret = cmpxchg(addr, __old, __new); \
+ } while (__ret != __old); \
+ (__old & __mask); \
+})
+
+/**
+ * test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ *
+ * This operation may be reordered on other architectures than x86.
+ */
+static inline int test_and_set_bit(int nr, volatile unsigned long *addr)
+{
+ return __test_and_op_bit(nr, addr, |, __NOP) != 0;
+}
+
+/**
+ * test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ *
+ * This operation can be reordered on other architectures other than x86.
+ */
+static inline int test_and_clear_bit(int nr, volatile unsigned long *addr)
+{
+ return __test_and_op_bit(nr, addr, &, __NOT);
+}
+
+#define __atomic_op(nr, addr, op, mod) \
+({ \
+ unsigned long __new, __old, __ret; \
+ __ret = addr[BIT_WORD(nr)]; \
+ do { \
+ __old = __ret; \
+ __new = __old op mod(BIT_MASK(nr)); \
+ __ret = cmpxchg(&addr[BIT_WORD(nr)], __old, __new); \
+ } while (__ret != __old); \
+})
+
+/**
+ * set_bit - Atomically set a bit in memory
+ * @nr: the bit to set
+ * @addr: the address to start counting from
+ *
+ * Note that @nr may be almost arbitrarily large; this function is not
+ * restricted to acting on a single-word quantity.
+ */
+static inline void set_bit(int nr, volatile unsigned long *addr)
+{
+ __atomic_op(nr, addr, |, __NOP);
+}
+
+/**
+ * clear_bit - Clears a bit in memory
+ * @nr: Bit to clear
+ * @addr: Address to start counting from
+ */
+static inline void clear_bit(int nr, volatile unsigned long *addr)
+{
+ __atomic_op(nr, addr, &, __NOT);
+}
+
+/**
+ * change_bit - Toggle a bit in memory
+ * @nr: Bit to change
+ * @addr: Address to start counting from
+ *
+ * Note that @nr may be almost arbitrarily large; this function is not
+ * restricted to acting on a single-word quantity.
+ */
+static inline void change_bit(int nr, volatile unsigned long *addr)
+{
+ __atomic_op(nr, addr, ^, __NOP);
+}
+
+#include <asm-generic/bitops/lock.h>
+#include <asm-generic/bitops/non-atomic.h>
+#include <asm-generic/bitops/le.h>
+#include <asm-generic/bitops/ext2-atomic.h>
+
+#endif
+
+#endif
diff --git a/arch/kvx/include/asm/bitrev.h b/arch/kvx/include/asm/bitrev.h
new file mode 100644
index 000000000000..79865081905a
--- /dev/null
+++ b/arch/kvx/include/asm/bitrev.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_BITREV_H
+#define _ASM_KVX_BITREV_H
+
+#include <linux/swab.h>
+
+/* Bit reversal constant for matrix multiply */
+#define BIT_REVERSE 0x0102040810204080ULL
+
+static __always_inline __attribute_const__ u32 __arch_bitrev32(u32 x)
+{
+ /* Reverse all bits for each bytes and then byte-reverse the 32 LSB */
+ return swab32(__builtin_kvx_sbmm8(BIT_REVERSE, x));
+}
+
+static __always_inline __attribute_const__ u16 __arch_bitrev16(u16 x)
+{
+ /* Reverse all bits for each bytes and then byte-reverse the 16 LSB */
+ return swab16(__builtin_kvx_sbmm8(BIT_REVERSE, x));
+}
+
+static __always_inline __attribute_const__ u8 __arch_bitrev8(u8 x)
+{
+ return __builtin_kvx_sbmm8(BIT_REVERSE, x);
+}
+
+#endif
diff --git a/arch/kvx/include/asm/cmpxchg.h b/arch/kvx/include/asm/cmpxchg.h
new file mode 100644
index 000000000000..b1d128b060a2
--- /dev/null
+++ b/arch/kvx/include/asm/cmpxchg.h
@@ -0,0 +1,185 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ */
+
+#ifndef _ASM_KVX_CMPXCHG_H
+#define _ASM_KVX_CMPXCHG_H
+
+#include <linux/bits.h>
+#include <linux/types.h>
+#include <linux/build_bug.h>
+
+/*
+ * On kvx, we have a boolean compare and swap which means that the operation
+ * returns only the success of operation.
+ * If operation succeed, this is simple, we just need to return the provided
+ * old value. However, if it fails, we need to load the value to return it for
+ * the caller. If the loaded value is different from the "old" provided by the
+ * caller, we can return it since it will means it failed.
+ * However, if for some reason the value we read is equal to the old value
+ * provided by the caller, we can't simply return it or the caller will think it
+ * succeeded. So if the value we read is the same as the "old" provided by
+ * the caller, we try again until either we succeed or we fail with a different
+ * value than the provided one.
+ */
+#define __cmpxchg(ptr, old, new, op_suffix, load_suffix) \
+({ \
+ register unsigned long __rn asm("r62"); \
+ register unsigned long __ro asm("r63"); \
+ __asm__ __volatile__ ( \
+ /* Fence to guarantee previous store to be committed */ \
+ "fence\n" \
+ /* Init "expect" with previous value */ \
+ "copyd $r63 = %[rOld]\n" \
+ ";;\n" \
+ "1:\n" \
+ /* Init "update" value with new */ \
+ "copyd $r62 = %[rNew]\n" \
+ ";;\n" \
+ "acswap" #op_suffix " 0[%[rPtr]], $r62r63\n" \
+ ";;\n" \
+ /* if acswap succeed, simply return */ \
+ "cb.dnez $r62? 2f\n" \
+ ";;\n" \
+ /* We failed, load old value */ \
+ "l" #op_suffix #load_suffix" $r63 = 0[%[rPtr]]\n" \
+ ";;\n" \
+ /* Check if equal to "old" one */ \
+ "comp" #op_suffix ".ne $r62 = $r63, %[rOld]\n" \
+ ";;\n" \
+ /* If different from "old", return it to caller */ \
+ "cb.deqz $r62? 1b\n" \
+ ";;\n" \
+ "2:\n" \
+ : "+r" (__rn), "+r" (__ro) \
+ : [rPtr] "r" (ptr), [rOld] "r" (old), [rNew] "r" (new) \
+ : "memory"); \
+ (__ro); \
+})
+
+#define arch_cmpxchg(ptr, o, n) \
+({ \
+ unsigned long __ret; \
+ BUILD_BUG_ON(sizeof(*(ptr)) != 4 && sizeof(*(ptr)) != 8); \
+ switch (sizeof(*(ptr))) { \
+ case 4: \
+ __ret = __cmpxchg((ptr), (o), (n), w, s); \
+ break; \
+ case 8: \
+ __ret = __cmpxchg((ptr), (o), (n), d, ); \
+ break; \
+ } \
+ (__typeof__(*(ptr))) (__ret); \
+})
+
+/*
+ * In order to optimize xchg for 16 byte, we can use insf/extfs if we know the
+ * bounds. This way, we only take one more bundle than standard xchg.
+ * We simply do a read modify acswap on a 32 bit word.
+ */
+#define __xchg_small_asm(ptr, new, start, stop) \
+({ \
+ register unsigned long __rn asm("r62"); \
+ register unsigned long __ro asm("r63"); \
+ __asm__ __volatile__ ( \
+ "fence\n" \
+ ";;\n" \
+ "1:\n" \
+ /* Load original old value */ \
+ "lws $r62 = 0[%[rPtr]]\n" \
+ ";;\n" \
+ /* Copy read value into "expect" */ \
+ "copyd $r63 = $r62\n" \
+ /* Prepare new value with insf */ \
+ "insf $r62 = %[rNew], " #stop "," #start "\n" \
+ ";;\n" \
+ /* Try compare & swap with loaded value */ \
+ "acswapw 0[%[rPtr]], $r62r63\n" \
+ ";;\n" \
+ /* Did we succeed ?, if no, try again */ \
+ "cb.deqz $r62? 1b\n" \
+ /* Extract old value for ret value */ \
+ "extfs $r63 = $r63, " #stop "," #start "\n" \
+ ";;\n" \
+ : "+r" (__rn), "+r" (__ro) \
+ : [rPtr] "r" (ptr), [rNew] "r" (new) \
+ : "memory"); \
+ (__ro); \
+})
+
+/* Needed for generic qspinlock implementation */
+static inline unsigned long xchg_u16(volatile void *ptr, unsigned long new,
+ int size)
+{
+ int off = (unsigned long)ptr % sizeof(u32);
+ volatile u32 *p = ptr - off;
+
+ /*
+ * GCC is smart enough to eliminate the dead branches by detecting
+ * the offset statically
+ */
+ if (off == 0)
+ return __xchg_small_asm(p, new, 0, 15);
+ else
+ return __xchg_small_asm(p, new, 16, 31);
+}
+
+#define __xchg_asm(ptr, new, op_suffix, load_suffix) \
+({ \
+ register unsigned long __rn asm("r62") = (unsigned long) (new); \
+ register unsigned long __ro asm("r63"); \
+ __asm__ __volatile__ ( \
+ "fence\n" \
+ ";;\n" \
+ "1:\n" \
+ /* Load original old value */ \
+ "l" #op_suffix #load_suffix " $r63 = 0[%[rPtr]]\n" \
+ ";;\n" \
+ /* Try compare & swap with loaded value */ \
+ "acswap" #op_suffix " 0[%[rPtr]], $r62r63\n" \
+ ";;\n" \
+ /* Did we succeed ?, if no, try again */ \
+ "cb.deqz $r62? 1b\n" \
+ /* $r62 has been cloberred by acswap, restore it */ \
+ "copyd $r62 = %[rNew]\n" \
+ ";;\n" \
+ : "+r" (__rn), "+r" (__ro) \
+ : [rPtr] "r" (ptr), [rNew] "r" (new) \
+ : "memory"); \
+ (__ro); \
+})
+
+/*
+ * This function doesn't exist, so you'll get a linker error if
+ * something tries to do an invalidly-sized xchg().
+ */
+extern unsigned long __xchg_called_with_bad_pointer(void)
+ __compiletime_error("Bad argument size for xchg");
+
+static inline unsigned long __xchg(volatile void *ptr, unsigned long val,
+ int size)
+{
+ switch (size) {
+ case 2:
+ return xchg_u16(ptr, val, size);
+ case 4:
+ return __xchg_asm(ptr, val, w, s);
+ case 8:
+ return __xchg_asm(ptr, val, d, );
+ }
+ __xchg_called_with_bad_pointer();
+
+ return val;
+}
+
+#define arch_xchg(ptr, with) \
+ ({ \
+ (__typeof__(*(ptr))) __xchg((ptr), \
+ (unsigned long)(with), \
+ sizeof(*(ptr))); \
+ })
+
+#endif
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 05/25] kvx: Add atomic/locking headers
2023-01-03 16:43 ` [RFC PATCH 05/25] kvx: Add atomic/locking headers Yann Sionneau
@ 2023-01-04 9:53 ` Mark Rutland
2023-01-06 14:11 ` Jules Maselbas
0 siblings, 1 reply; 113+ messages in thread
From: Mark Rutland @ 2023-01-04 9:53 UTC (permalink / raw)
To: Yann Sionneau
Cc: Will Deacon, Peter Zijlstra, Boqun Feng, linux-kernel,
Clement Leger, Jules Maselbas, Julian Vetter, Julien Villette
On Tue, Jan 03, 2023 at 05:43:39PM +0100, Yann Sionneau wrote:
> Add common headers (atomic, bitops, barrier and locking) for basic
> kvx support.
>
> CC: Will Deacon <will@kernel.org>
> CC: Peter Zijlstra <peterz@infradead.org>
> CC: Boqun Feng <boqun.feng@gmail.com>
> CC: Mark Rutland <mark.rutland@arm.com>
> CC: linux-kernel@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
> Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Co-developed-by: Julien Villette <jvillette@kalray.eu>
> Signed-off-by: Julien Villette <jvillette@kalray.eu>
> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> arch/kvx/include/asm/atomic.h | 104 +++++++++++++++++
> arch/kvx/include/asm/barrier.h | 15 +++
> arch/kvx/include/asm/bitops.h | 207 +++++++++++++++++++++++++++++++++
> arch/kvx/include/asm/bitrev.h | 32 +++++
> arch/kvx/include/asm/cmpxchg.h | 185 +++++++++++++++++++++++++++++
> 5 files changed, 543 insertions(+)
> create mode 100644 arch/kvx/include/asm/atomic.h
> create mode 100644 arch/kvx/include/asm/barrier.h
> create mode 100644 arch/kvx/include/asm/bitops.h
> create mode 100644 arch/kvx/include/asm/bitrev.h
> create mode 100644 arch/kvx/include/asm/cmpxchg.h
>
> diff --git a/arch/kvx/include/asm/atomic.h b/arch/kvx/include/asm/atomic.h
> new file mode 100644
> index 000000000000..eb8acbcbc70d
> --- /dev/null
> +++ b/arch/kvx/include/asm/atomic.h
> @@ -0,0 +1,104 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#ifndef _ASM_KVX_ATOMIC_H
> +#define _ASM_KVX_ATOMIC_H
> +
> +#include <linux/types.h>
> +
> +#include <asm/cmpxchg.h>
> +
> +#define ATOMIC64_INIT(i) { (i) }
> +
> +#define arch_atomic64_cmpxchg(v, old, new) (arch_cmpxchg(&((v)->counter), old, new))
> +#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), new))
> +
> +static inline long arch_atomic64_read(const atomic64_t *v)
> +{
> + return v->counter;
> +}
This is a plain read, and is *not* atomic.
The compiler can replay a plain read an arbitrary number of times, and is
permitted to split it into smaller accesses.
At minimum this needs to be
READ_ONCE(v->counter)
... which will prevent replay. Whether or not that's actually atomic will
depend on the instructions the compiler generates, and how those instructions
are defines in your architecture.
Do you have a single instruction that can read a 64-bit memory location, and is
it guaranteed to result in a single access that cannot be split?
> +static inline void arch_atomic64_set(atomic64_t *v, long i)
> +{
> + v->counter = i;
> +}
Same comments as for arch_atomic64_read(); at minimum this needs to be:
WRITE_ONCE(v->counter, i)
... but that may or may not actually be atomic on your architecture.
> +#define ATOMIC64_RETURN_OP(op, c_op) \
> +static inline long arch_atomic64_##op##_return(long i, atomic64_t *v) \
> +{ \
> + long new, old, ret; \
> + \
> + do { \
> + old = v->counter; \
> + new = old c_op i; \
> + ret = arch_cmpxchg(&v->counter, old, new); \
> + } while (ret != old); \
> + \
> + return new; \
> +}
> +
> +#define ATOMIC64_OP(op, c_op) \
> +static inline void arch_atomic64_##op(long i, atomic64_t *v) \
> +{ \
> + long new, old, ret; \
> + \
> + do { \
> + old = v->counter; \
> + new = old c_op i; \
> + ret = arch_cmpxchg(&v->counter, old, new); \
> + } while (ret != old); \
> +}
> +
> +#define ATOMIC64_FETCH_OP(op, c_op) \
> +static inline long arch_atomic64_fetch_##op(long i, atomic64_t *v) \
> +{ \
> + long new, old, ret; \
> + \
> + do { \
> + old = v->counter; \
> + new = old c_op i; \
> + ret = arch_cmpxchg(&v->counter, old, new); \
> + } while (ret != old); \
> + \
> + return old; \
> +}
These look ok, but it'd be nicer if we could teach the generic atomic64 code to
do this, like the generic atomic code does.
We could rename the existing asm-generic/atomic64 code to atomic64-spinlock,
and add a separate atomic64-cmpxchg (and likewise for the 32-bit code) to make
that clearer and consistent.
> +
> +#define ATOMIC64_OPS(op, c_op) \
> + ATOMIC64_OP(op, c_op) \
> + ATOMIC64_RETURN_OP(op, c_op) \
> + ATOMIC64_FETCH_OP(op, c_op)
> +
> +ATOMIC64_OPS(and, &)
> +ATOMIC64_OPS(or, |)
> +ATOMIC64_OPS(xor, ^)
> +ATOMIC64_OPS(add, +)
> +ATOMIC64_OPS(sub, -)
> +
> +#undef ATOMIC64_OPS
> +#undef ATOMIC64_FETCH_OP
> +#undef ATOMIC64_OP
> +
> +static inline int arch_atomic_add_return(int i, atomic_t *v)
> +{
> + int new, old, ret;
> +
> + do {
> + old = v->counter;
> + new = old + i;
> + ret = arch_cmpxchg(&v->counter, old, new);
> + } while (ret != old);
> +
> + return new;
> +}
> +
> +static inline int arch_atomic_sub_return(int i, atomic_t *v)
> +{
> + return arch_atomic_add_return(-i, v);
> +}
Likewise for these two.
> +
> +#include <asm-generic/atomic.h>
> +
> +#endif /* _ASM_KVX_ATOMIC_H */
> diff --git a/arch/kvx/include/asm/barrier.h b/arch/kvx/include/asm/barrier.h
> new file mode 100644
> index 000000000000..371f1c70746d
> --- /dev/null
> +++ b/arch/kvx/include/asm/barrier.h
> @@ -0,0 +1,15 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#ifndef _ASM_KVX_BARRIER_H
> +#define _ASM_KVX_BARRIER_H
> +/* Bitmask modifiers */
> +#define __NOP(x) (x)
> +#define __NOT(x) (~(x))
> +
> +
> +#define __test_and_op_bit(nr, addr, op, mod) \
> +({ \
> + unsigned long __mask = BIT_MASK(nr); \
> + unsigned long __new, __old, __ret; \
> + do { \
> + __old = *(&addr[BIT_WORD(nr)]); \
> + __new = __old op mod(__mask); \
> + __ret = cmpxchg(addr, __old, __new); \
> + } while (__ret != __old); \
> + (__old & __mask); \
> +})
Please use <asm-generic/bitops/atomic.h> which should give you the common
bit operations "for free" atop your regular atomics.
[...]
> diff --git a/arch/kvx/include/asm/cmpxchg.h b/arch/kvx/include/asm/cmpxchg.h
> new file mode 100644
> index 000000000000..b1d128b060a2
> --- /dev/null
> +++ b/arch/kvx/include/asm/cmpxchg.h
> @@ -0,0 +1,185 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + * Yann Sionneau
> + */
> +
> +#ifndef _ASM_KVX_CMPXCHG_H
> +#define _ASM_KVX_CMPXCHG_H
> +
> +#include <linux/bits.h>
> +#include <linux/types.h>
> +#include <linux/build_bug.h>
> +
> +/*
> + * On kvx, we have a boolean compare and swap which means that the operation
> + * returns only the success of operation.
> + * If operation succeed, this is simple, we just need to return the provided
> + * old value. However, if it fails, we need to load the value to return it for
> + * the caller. If the loaded value is different from the "old" provided by the
> + * caller, we can return it since it will means it failed.
> + * However, if for some reason the value we read is equal to the old value
> + * provided by the caller, we can't simply return it or the caller will think it
> + * succeeded. So if the value we read is the same as the "old" provided by
> + * the caller, we try again until either we succeed or we fail with a different
> + * value than the provided one.
> + */
> +#define __cmpxchg(ptr, old, new, op_suffix, load_suffix) \
> +({ \
> + register unsigned long __rn asm("r62"); \
> + register unsigned long __ro asm("r63"); \
Why do you need to specify the exact registers?
e.g. does some instruction use these implicitly, or do you need two adjacent
register for encoding reasons?
> + __asm__ __volatile__ ( \
> + /* Fence to guarantee previous store to be committed */ \
> + "fence\n" \
This implies you can implement the relaxed form of cmpxchg().
What ordering do you get by default, and do you have any other barriers (e.g.
for acquire/release semantics), or just "fence" ?
Thanks,
Mark.
> + /* Init "expect" with previous value */ \
> + "copyd $r63 = %[rOld]\n" \
> + ";;\n" \
> + "1:\n" \
> + /* Init "update" value with new */ \
> + "copyd $r62 = %[rNew]\n" \
> + ";;\n" \
> + "acswap" #op_suffix " 0[%[rPtr]], $r62r63\n" \
> + ";;\n" \
> + /* if acswap succeed, simply return */ \
> + "cb.dnez $r62? 2f\n" \
> + ";;\n" \
> + /* We failed, load old value */ \
> + "l" #op_suffix #load_suffix" $r63 = 0[%[rPtr]]\n" \
> + ";;\n" \
> + /* Check if equal to "old" one */ \
> + "comp" #op_suffix ".ne $r62 = $r63, %[rOld]\n" \
> + ";;\n" \
> + /* If different from "old", return it to caller */ \
> + "cb.deqz $r62? 1b\n" \
> + ";;\n" \
> + "2:\n" \
> + : "+r" (__rn), "+r" (__ro) \
> + : [rPtr] "r" (ptr), [rOld] "r" (old), [rNew] "r" (new) \
> + : "memory"); \
> + (__ro); \
> +})
> +
> +#define arch_cmpxchg(ptr, o, n) \
> +({ \
> + unsigned long __ret; \
> + BUILD_BUG_ON(sizeof(*(ptr)) != 4 && sizeof(*(ptr)) != 8); \
> + switch (sizeof(*(ptr))) { \
> + case 4: \
> + __ret = __cmpxchg((ptr), (o), (n), w, s); \
> + break; \
> + case 8: \
> + __ret = __cmpxchg((ptr), (o), (n), d, ); \
> + break; \
> + } \
> + (__typeof__(*(ptr))) (__ret); \
> +})
> +
> +/*
> + * In order to optimize xchg for 16 byte, we can use insf/extfs if we know the
> + * bounds. This way, we only take one more bundle than standard xchg.
> + * We simply do a read modify acswap on a 32 bit word.
> + */
> +#define __xchg_small_asm(ptr, new, start, stop) \
> +({ \
> + register unsigned long __rn asm("r62"); \
> + register unsigned long __ro asm("r63"); \
> + __asm__ __volatile__ ( \
> + "fence\n" \
> + ";;\n" \
> + "1:\n" \
> + /* Load original old value */ \
> + "lws $r62 = 0[%[rPtr]]\n" \
> + ";;\n" \
> + /* Copy read value into "expect" */ \
> + "copyd $r63 = $r62\n" \
> + /* Prepare new value with insf */ \
> + "insf $r62 = %[rNew], " #stop "," #start "\n" \
> + ";;\n" \
> + /* Try compare & swap with loaded value */ \
> + "acswapw 0[%[rPtr]], $r62r63\n" \
> + ";;\n" \
> + /* Did we succeed ?, if no, try again */ \
> + "cb.deqz $r62? 1b\n" \
> + /* Extract old value for ret value */ \
> + "extfs $r63 = $r63, " #stop "," #start "\n" \
> + ";;\n" \
> + : "+r" (__rn), "+r" (__ro) \
> + : [rPtr] "r" (ptr), [rNew] "r" (new) \
> + : "memory"); \
> + (__ro); \
> +})
> +
> +/* Needed for generic qspinlock implementation */
> +static inline unsigned long xchg_u16(volatile void *ptr, unsigned long new,
> + int size)
> +{
> + int off = (unsigned long)ptr % sizeof(u32);
> + volatile u32 *p = ptr - off;
> +
> + /*
> + * GCC is smart enough to eliminate the dead branches by detecting
> + * the offset statically
> + */
> + if (off == 0)
> + return __xchg_small_asm(p, new, 0, 15);
> + else
> + return __xchg_small_asm(p, new, 16, 31);
> +}
> +
> +#define __xchg_asm(ptr, new, op_suffix, load_suffix) \
> +({ \
> + register unsigned long __rn asm("r62") = (unsigned long) (new); \
> + register unsigned long __ro asm("r63"); \
> + __asm__ __volatile__ ( \
> + "fence\n" \
> + ";;\n" \
> + "1:\n" \
> + /* Load original old value */ \
> + "l" #op_suffix #load_suffix " $r63 = 0[%[rPtr]]\n" \
> + ";;\n" \
> + /* Try compare & swap with loaded value */ \
> + "acswap" #op_suffix " 0[%[rPtr]], $r62r63\n" \
> + ";;\n" \
> + /* Did we succeed ?, if no, try again */ \
> + "cb.deqz $r62? 1b\n" \
> + /* $r62 has been cloberred by acswap, restore it */ \
> + "copyd $r62 = %[rNew]\n" \
> + ";;\n" \
> + : "+r" (__rn), "+r" (__ro) \
> + : [rPtr] "r" (ptr), [rNew] "r" (new) \
> + : "memory"); \
> + (__ro); \
> +})
> +
> +/*
> + * This function doesn't exist, so you'll get a linker error if
> + * something tries to do an invalidly-sized xchg().
> + */
> +extern unsigned long __xchg_called_with_bad_pointer(void)
> + __compiletime_error("Bad argument size for xchg");
> +
> +static inline unsigned long __xchg(volatile void *ptr, unsigned long val,
> + int size)
> +{
> + switch (size) {
> + case 2:
> + return xchg_u16(ptr, val, size);
> + case 4:
> + return __xchg_asm(ptr, val, w, s);
> + case 8:
> + return __xchg_asm(ptr, val, d, );
> + }
> + __xchg_called_with_bad_pointer();
> +
> + return val;
> +}
> +
> +#define arch_xchg(ptr, with) \
> + ({ \
> + (__typeof__(*(ptr))) __xchg((ptr), \
> + (unsigned long)(with), \
> + sizeof(*(ptr))); \
> + })
> +
> +#endif
> --
> 2.37.2
>
>
>
>
>
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 05/25] kvx: Add atomic/locking headers
2023-01-04 9:53 ` Mark Rutland
@ 2023-01-06 14:11 ` Jules Maselbas
2023-01-10 13:24 ` Mark Rutland
0 siblings, 1 reply; 113+ messages in thread
From: Jules Maselbas @ 2023-01-06 14:11 UTC (permalink / raw)
To: Mark Rutland
Cc: Yann Sionneau, Will Deacon, Peter Zijlstra, Boqun Feng,
linux-kernel, Clement Leger, Julian Vetter, Julien Villette
Hi Mark,
On Wed, Jan 04, 2023 at 09:53:24AM +0000, Mark Rutland wrote:
> On Tue, Jan 03, 2023 at 05:43:39PM +0100, Yann Sionneau wrote:
> > Add common headers (atomic, bitops, barrier and locking) for basic
> > kvx support.
> >
> > CC: Will Deacon <will@kernel.org>
> > CC: Peter Zijlstra <peterz@infradead.org>
> > CC: Boqun Feng <boqun.feng@gmail.com>
> > CC: Mark Rutland <mark.rutland@arm.com>
> > CC: linux-kernel@vger.kernel.org
> > Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> > Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> > Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
> > Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
> > Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> > Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> > Co-developed-by: Julien Villette <jvillette@kalray.eu>
> > Signed-off-by: Julien Villette <jvillette@kalray.eu>
> > Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> > Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> > ---
> > arch/kvx/include/asm/atomic.h | 104 +++++++++++++++++
> > arch/kvx/include/asm/barrier.h | 15 +++
> > arch/kvx/include/asm/bitops.h | 207 +++++++++++++++++++++++++++++++++
> > arch/kvx/include/asm/bitrev.h | 32 +++++
> > arch/kvx/include/asm/cmpxchg.h | 185 +++++++++++++++++++++++++++++
> > 5 files changed, 543 insertions(+)
> > create mode 100644 arch/kvx/include/asm/atomic.h
> > create mode 100644 arch/kvx/include/asm/barrier.h
> > create mode 100644 arch/kvx/include/asm/bitops.h
> > create mode 100644 arch/kvx/include/asm/bitrev.h
> > create mode 100644 arch/kvx/include/asm/cmpxchg.h
> >
> > diff --git a/arch/kvx/include/asm/atomic.h b/arch/kvx/include/asm/atomic.h
> > new file mode 100644
> > index 000000000000..eb8acbcbc70d
> > --- /dev/null
> > +++ b/arch/kvx/include/asm/atomic.h
> > @@ -0,0 +1,104 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Copyright (C) 2017-2023 Kalray Inc.
> > + * Author(s): Clement Leger
> > + */
> > +
> > +#ifndef _ASM_KVX_ATOMIC_H
> > +#define _ASM_KVX_ATOMIC_H
> > +
> > +#include <linux/types.h>
> > +
> > +#include <asm/cmpxchg.h>
> > +
> > +#define ATOMIC64_INIT(i) { (i) }
> > +
> > +#define arch_atomic64_cmpxchg(v, old, new) (arch_cmpxchg(&((v)->counter), old, new))
> > +#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), new))
> > +
> > +static inline long arch_atomic64_read(const atomic64_t *v)
> > +{
> > + return v->counter;
> > +}
>
> This is a plain read, and is *not* atomic.
>
> The compiler can replay a plain read an arbitrary number of times, and is
> permitted to split it into smaller accesses.
>
> At minimum this needs to be
>
> READ_ONCE(v->counter)
>
> ... which will prevent replay. Whether or not that's actually atomic will
> depend on the instructions the compiler generates, and how those instructions
> are defines in your architecture.
Good point, we are going to use {READ,WRITE}_ONCE macros
> Do you have a single instruction that can read a 64-bit memory location, and is
> it guaranteed to result in a single access that cannot be split?
We do have a single instruction that can read a 64-bit memory location
(supported sizes are 8, 16, 32, 64, 128, 256).
All accesses are guaranteed to not be split, unless they are misaligned.
Furthermore, misaligned write accesses crossing a 32-byte boundary may
complete in a non-atomic way.
>
> > +static inline void arch_atomic64_set(atomic64_t *v, long i)
> > +{
> > + v->counter = i;
> > +}
>
> Same comments as for arch_atomic64_read(); at minimum this needs to be:
>
> WRITE_ONCE(v->counter, i)
>
> ... but that may or may not actually be atomic on your architecture.
>
> > +#define ATOMIC64_RETURN_OP(op, c_op) \
> > +static inline long arch_atomic64_##op##_return(long i, atomic64_t *v) \
> > +{ \
> > + long new, old, ret; \
> > + \
> > + do { \
> > + old = v->counter; \
> > + new = old c_op i; \
> > + ret = arch_cmpxchg(&v->counter, old, new); \
> > + } while (ret != old); \
> > + \
> > + return new; \
> > +}
> > +
> > +#define ATOMIC64_OP(op, c_op) \
> > +static inline void arch_atomic64_##op(long i, atomic64_t *v) \
> > +{ \
> > + long new, old, ret; \
> > + \
> > + do { \
> > + old = v->counter; \
> > + new = old c_op i; \
> > + ret = arch_cmpxchg(&v->counter, old, new); \
> > + } while (ret != old); \
> > +}
> > +
> > +#define ATOMIC64_FETCH_OP(op, c_op) \
> > +static inline long arch_atomic64_fetch_##op(long i, atomic64_t *v) \
> > +{ \
> > + long new, old, ret; \
> > + \
> > + do { \
> > + old = v->counter; \
> > + new = old c_op i; \
> > + ret = arch_cmpxchg(&v->counter, old, new); \
> > + } while (ret != old); \
> > + \
> > + return old; \
> > +}
>
> These look ok, but it'd be nicer if we could teach the generic atomic64 code to
> do this, like the generic atomic code does.
>
> We could rename the existing asm-generic/atomic64 code to atomic64-spinlock,
> and add a separate atomic64-cmpxchg (and likewise for the 32-bit code) to make
> that clearer and consistent.
I am not sure what this implies and how big this change might be,
but I'll take a look at this.
> > +
> > +#define ATOMIC64_OPS(op, c_op) \
> > + ATOMIC64_OP(op, c_op) \
> > + ATOMIC64_RETURN_OP(op, c_op) \
> > + ATOMIC64_FETCH_OP(op, c_op)
> > +
> > +ATOMIC64_OPS(and, &)
> > +ATOMIC64_OPS(or, |)
> > +ATOMIC64_OPS(xor, ^)
> > +ATOMIC64_OPS(add, +)
> > +ATOMIC64_OPS(sub, -)
> > +
> > +#undef ATOMIC64_OPS
> > +#undef ATOMIC64_FETCH_OP
> > +#undef ATOMIC64_OP
> > +
> > +static inline int arch_atomic_add_return(int i, atomic_t *v)
> > +{
> > + int new, old, ret;
> > +
> > + do {
> > + old = v->counter;
> > + new = old + i;
> > + ret = arch_cmpxchg(&v->counter, old, new);
> > + } while (ret != old);
> > +
> > + return new;
> > +}
> > +
> > +static inline int arch_atomic_sub_return(int i, atomic_t *v)
> > +{
> > + return arch_atomic_add_return(-i, v);
> > +}
>
> Likewise for these two.
>
> > +
> > +#include <asm-generic/atomic.h>
> > +
> > +#endif /* _ASM_KVX_ATOMIC_H */
> > diff --git a/arch/kvx/include/asm/barrier.h b/arch/kvx/include/asm/barrier.h
> > new file mode 100644
> > index 000000000000..371f1c70746d
> > --- /dev/null
> > +++ b/arch/kvx/include/asm/barrier.h
> > @@ -0,0 +1,15 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Copyright (C) 2017-2023 Kalray Inc.
> > + * Author(s): Clement Leger
> > + */
> > +
> > +#ifndef _ASM_KVX_BARRIER_H
> > +#define _ASM_KVX_BARRIER_H
>
> > +/* Bitmask modifiers */
> > +#define __NOP(x) (x)
> > +#define __NOT(x) (~(x))
> > +
> > +
> > +#define __test_and_op_bit(nr, addr, op, mod) \
> > +({ \
> > + unsigned long __mask = BIT_MASK(nr); \
> > + unsigned long __new, __old, __ret; \
> > + do { \
> > + __old = *(&addr[BIT_WORD(nr)]); \
> > + __new = __old op mod(__mask); \
> > + __ret = cmpxchg(addr, __old, __new); \
> > + } while (__ret != __old); \
> > + (__old & __mask); \
> > +})
>
> Please use <asm-generic/bitops/atomic.h> which should give you the common
> bit operations "for free" atop your regular atomics.
Yes
>
> [...]
>
> > diff --git a/arch/kvx/include/asm/cmpxchg.h b/arch/kvx/include/asm/cmpxchg.h
> > new file mode 100644
> > index 000000000000..b1d128b060a2
> > --- /dev/null
> > +++ b/arch/kvx/include/asm/cmpxchg.h
> > @@ -0,0 +1,185 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Copyright (C) 2017-2023 Kalray Inc.
> > + * Author(s): Clement Leger
> > + * Yann Sionneau
> > + */
> > +
> > +#ifndef _ASM_KVX_CMPXCHG_H
> > +#define _ASM_KVX_CMPXCHG_H
> > +
> > +#include <linux/bits.h>
> > +#include <linux/types.h>
> > +#include <linux/build_bug.h>
> > +
> > +/*
> > + * On kvx, we have a boolean compare and swap which means that the operation
> > + * returns only the success of operation.
> > + * If operation succeed, this is simple, we just need to return the provided
> > + * old value. However, if it fails, we need to load the value to return it for
> > + * the caller. If the loaded value is different from the "old" provided by the
> > + * caller, we can return it since it will means it failed.
> > + * However, if for some reason the value we read is equal to the old value
> > + * provided by the caller, we can't simply return it or the caller will think it
> > + * succeeded. So if the value we read is the same as the "old" provided by
> > + * the caller, we try again until either we succeed or we fail with a different
> > + * value than the provided one.
> > + */
> > +#define __cmpxchg(ptr, old, new, op_suffix, load_suffix) \
> > +({ \
> > + register unsigned long __rn asm("r62"); \
> > + register unsigned long __ro asm("r63"); \
>
> Why do you need to specify the exact registers?
r62 and r63 are hardcoded in the inline assembly, they are caller saved.
I have a C implementation that uses builtins however this is not merged
in our tree yet (but I want to).
> e.g. does some instruction use these implicitly, or do you need two adjacent
> register for encoding reasons?
The atomic compare and swap (acswap) instruction needs a register "pair"
which can only exists with "adjacent" registers: $r0r1, $r2r3 ect.
> > + __asm__ __volatile__ ( \
> > + /* Fence to guarantee previous store to be committed */ \
> > + "fence\n" \
>
> This implies you can implement the relaxed form of cmpxchg().
>
> What ordering do you get by default, and do you have any other barriers (e.g.
> for acquire/release semantics), or just "fence" ?
We have two barrier types:
- fence: ensure that all uncached memory operations are committed to
memory, typically used to ensure a write to memory is visible to
other cores.
- barrier: flush the core instruction pipeline and memory system
Thanks,
-- Jules
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 05/25] kvx: Add atomic/locking headers
2023-01-06 14:11 ` Jules Maselbas
@ 2023-01-10 13:24 ` Mark Rutland
2023-01-18 13:40 ` Yann Sionneau
0 siblings, 1 reply; 113+ messages in thread
From: Mark Rutland @ 2023-01-10 13:24 UTC (permalink / raw)
To: Jules Maselbas
Cc: Yann Sionneau, Will Deacon, Peter Zijlstra, Boqun Feng,
linux-kernel, Clement Leger, Julian Vetter, Julien Villette
On Fri, Jan 06, 2023 at 03:11:58PM +0100, Jules Maselbas wrote:
> Hi Mark,
>
> On Wed, Jan 04, 2023 at 09:53:24AM +0000, Mark Rutland wrote:
> > On Tue, Jan 03, 2023 at 05:43:39PM +0100, Yann Sionneau wrote:
> > > Add common headers (atomic, bitops, barrier and locking) for basic
> > > kvx support.
> > >
> > > CC: Will Deacon <will@kernel.org>
> > > CC: Peter Zijlstra <peterz@infradead.org>
> > > CC: Boqun Feng <boqun.feng@gmail.com>
> > > CC: Mark Rutland <mark.rutland@arm.com>
> > > CC: linux-kernel@vger.kernel.org
> > > Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> > > Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> > > Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
> > > Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
> > > Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> > > Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> > > Co-developed-by: Julien Villette <jvillette@kalray.eu>
> > > Signed-off-by: Julien Villette <jvillette@kalray.eu>
> > > Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> > > Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> > > ---
> > > arch/kvx/include/asm/atomic.h | 104 +++++++++++++++++
> > > arch/kvx/include/asm/barrier.h | 15 +++
> > > arch/kvx/include/asm/bitops.h | 207 +++++++++++++++++++++++++++++++++
> > > arch/kvx/include/asm/bitrev.h | 32 +++++
> > > arch/kvx/include/asm/cmpxchg.h | 185 +++++++++++++++++++++++++++++
> > > 5 files changed, 543 insertions(+)
> > > create mode 100644 arch/kvx/include/asm/atomic.h
> > > create mode 100644 arch/kvx/include/asm/barrier.h
> > > create mode 100644 arch/kvx/include/asm/bitops.h
> > > create mode 100644 arch/kvx/include/asm/bitrev.h
> > > create mode 100644 arch/kvx/include/asm/cmpxchg.h
> > >
> > > diff --git a/arch/kvx/include/asm/atomic.h b/arch/kvx/include/asm/atomic.h
> > > new file mode 100644
> > > index 000000000000..eb8acbcbc70d
> > > --- /dev/null
> > > +++ b/arch/kvx/include/asm/atomic.h
> > > @@ -0,0 +1,104 @@
> > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > +/*
> > > + * Copyright (C) 2017-2023 Kalray Inc.
> > > + * Author(s): Clement Leger
> > > + */
> > > +
> > > +#ifndef _ASM_KVX_ATOMIC_H
> > > +#define _ASM_KVX_ATOMIC_H
> > > +
> > > +#include <linux/types.h>
> > > +
> > > +#include <asm/cmpxchg.h>
> > > +
> > > +#define ATOMIC64_INIT(i) { (i) }
> > > +
> > > +#define arch_atomic64_cmpxchg(v, old, new) (arch_cmpxchg(&((v)->counter), old, new))
> > > +#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), new))
> > > +
> > > +static inline long arch_atomic64_read(const atomic64_t *v)
> > > +{
> > > + return v->counter;
> > > +}
> >
> > This is a plain read, and is *not* atomic.
> >
> > The compiler can replay a plain read an arbitrary number of times, and is
> > permitted to split it into smaller accesses.
> >
> > At minimum this needs to be
> >
> > READ_ONCE(v->counter)
> >
> > ... which will prevent replay. Whether or not that's actually atomic will
> > depend on the instructions the compiler generates, and how those instructions
> > are defines in your architecture.
> Good point, we are going to use {READ,WRITE}_ONCE macros
>
> > Do you have a single instruction that can read a 64-bit memory location, and is
> > it guaranteed to result in a single access that cannot be split?
>
> We do have a single instruction that can read a 64-bit memory location
> (supported sizes are 8, 16, 32, 64, 128, 256).
> All accesses are guaranteed to not be split, unless they are misaligned.
> Furthermore, misaligned write accesses crossing a 32-byte boundary may
> complete in a non-atomic way.
Perfect, thanks for confirming!
[...]
> > > +static inline void arch_atomic64_set(atomic64_t *v, long i)
> > > +{
> > > + v->counter = i;
> > > +}
> >
> > Same comments as for arch_atomic64_read(); at minimum this needs to be:
> >
> > WRITE_ONCE(v->counter, i)
> >
> > ... but that may or may not actually be atomic on your architecture.
> >
> > > +#define ATOMIC64_RETURN_OP(op, c_op) \
> > > +static inline long arch_atomic64_##op##_return(long i, atomic64_t *v) \
> > > +{ \
> > > + long new, old, ret; \
> > > + \
> > > + do { \
> > > + old = v->counter; \
> > > + new = old c_op i; \
> > > + ret = arch_cmpxchg(&v->counter, old, new); \
> > > + } while (ret != old); \
> > > + \
> > > + return new; \
> > > +}
> > > +
> > > +#define ATOMIC64_OP(op, c_op) \
> > > +static inline void arch_atomic64_##op(long i, atomic64_t *v) \
> > > +{ \
> > > + long new, old, ret; \
> > > + \
> > > + do { \
> > > + old = v->counter; \
> > > + new = old c_op i; \
> > > + ret = arch_cmpxchg(&v->counter, old, new); \
> > > + } while (ret != old); \
> > > +}
> > > +
> > > +#define ATOMIC64_FETCH_OP(op, c_op) \
> > > +static inline long arch_atomic64_fetch_##op(long i, atomic64_t *v) \
> > > +{ \
> > > + long new, old, ret; \
> > > + \
> > > + do { \
> > > + old = v->counter; \
> > > + new = old c_op i; \
> > > + ret = arch_cmpxchg(&v->counter, old, new); \
> > > + } while (ret != old); \
> > > + \
> > > + return old; \
> > > +}
> >
> > These look ok, but it'd be nicer if we could teach the generic atomic64 code to
> > do this, like the generic atomic code does.
> >
> > We could rename the existing asm-generic/atomic64 code to atomic64-spinlock,
> > and add a separate atomic64-cmpxchg (and likewise for the 32-bit code) to make
> > that clearer and consistent.
> I am not sure what this implies and how big this change might be,
> but I'll take a look at this.
Hmm... from a quick attempt just now it looks like that will be a bit more
churny than I thought.
We can always factor this out later, so feel free to leave it as-is, thgouh if
we could make this genric (and have it look like asm-generic/atomic.h), it'd be
nice for consistency and maintenance.
> > > +
> > > +#define ATOMIC64_OPS(op, c_op) \
> > > + ATOMIC64_OP(op, c_op) \
> > > + ATOMIC64_RETURN_OP(op, c_op) \
> > > + ATOMIC64_FETCH_OP(op, c_op)
> > > +
> > > +ATOMIC64_OPS(and, &)
> > > +ATOMIC64_OPS(or, |)
> > > +ATOMIC64_OPS(xor, ^)
> > > +ATOMIC64_OPS(add, +)
> > > +ATOMIC64_OPS(sub, -)
> > > +
> > > +#undef ATOMIC64_OPS
> > > +#undef ATOMIC64_FETCH_OP
> > > +#undef ATOMIC64_OP
> > > +
> > > +static inline int arch_atomic_add_return(int i, atomic_t *v)
> > > +{
> > > + int new, old, ret;
> > > +
> > > + do {
> > > + old = v->counter;
> > > + new = old + i;
> > > + ret = arch_cmpxchg(&v->counter, old, new);
> > > + } while (ret != old);
> > > +
> > > + return new;
> > > +}
> > > +
> > > +static inline int arch_atomic_sub_return(int i, atomic_t *v)
> > > +{
> > > + return arch_atomic_add_return(-i, v);
> > > +}
> >
> > Likewise for these two.
The 32-bit atomics should come from asm-generic/atomic.h and not be necesary,
that has both arch_atomic_add_return() and arch_atomic_sub_return().
> > > diff --git a/arch/kvx/include/asm/cmpxchg.h b/arch/kvx/include/asm/cmpxchg.h
> > > new file mode 100644
> > > index 000000000000..b1d128b060a2
> > > --- /dev/null
> > > +++ b/arch/kvx/include/asm/cmpxchg.h
> > > @@ -0,0 +1,185 @@
> > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > +/*
> > > + * Copyright (C) 2017-2023 Kalray Inc.
> > > + * Author(s): Clement Leger
> > > + * Yann Sionneau
> > > + */
> > > +
> > > +#ifndef _ASM_KVX_CMPXCHG_H
> > > +#define _ASM_KVX_CMPXCHG_H
> > > +
> > > +#include <linux/bits.h>
> > > +#include <linux/types.h>
> > > +#include <linux/build_bug.h>
> > > +
> > > +/*
> > > + * On kvx, we have a boolean compare and swap which means that the operation
> > > + * returns only the success of operation.
> > > + * If operation succeed, this is simple, we just need to return the provided
> > > + * old value. However, if it fails, we need to load the value to return it for
> > > + * the caller. If the loaded value is different from the "old" provided by the
> > > + * caller, we can return it since it will means it failed.
> > > + * However, if for some reason the value we read is equal to the old value
> > > + * provided by the caller, we can't simply return it or the caller will think it
> > > + * succeeded. So if the value we read is the same as the "old" provided by
> > > + * the caller, we try again until either we succeed or we fail with a different
> > > + * value than the provided one.
> > > + */
> > > +#define __cmpxchg(ptr, old, new, op_suffix, load_suffix) \
> > > +({ \
> > > + register unsigned long __rn asm("r62"); \
> > > + register unsigned long __ro asm("r63"); \
> >
> > Why do you need to specify the exact registers?
> r62 and r63 are hardcoded in the inline assembly, they are caller saved.
> I have a C implementation that uses builtins however this is not merged
> in our tree yet (but I want to).
>
> > e.g. does some instruction use these implicitly, or do you need two adjacent
> > register for encoding reasons?
>
> The atomic compare and swap (acswap) instruction needs a register "pair"
> which can only exists with "adjacent" registers: $r0r1, $r2r3 ect.
Ok; and you don't have a way to ask GCC for an arbitrary register pair, so you
chose r62+r63 as they can be clobbered?
It might be worth looking into using an __int128_t to give you a register pair,
assuming your calling convention mandares adjacency of the two halves. That
could give you the pair while giving the compiler freedom to chose any suitable
pair (assuming you have a suitable operand modifier to extract the low/high
registers from the asm operand.
> > > + __asm__ __volatile__ ( \
> > > + /* Fence to guarantee previous store to be committed */ \
> > > + "fence\n" \
> >
> > This implies you can implement the relaxed form of cmpxchg().
> >
> > What ordering do you get by default, and do you have any other barriers (e.g.
> > for acquire/release semantics), or just "fence" ?
> We have two barrier types:
> - fence: ensure that all uncached memory operations are committed to
> memory, typically used to ensure a write to memory is visible to
> other cores.
> - barrier: flush the core instruction pipeline and memory system
Ok.
In the absence of barriers do you have any ordering guarantee? e.g. weak like
arm/power, TSO like x86, or something else entirely?
When you say "uncached memory operations", does that mean that you have cached
memory operatins which are not affected by the barriers, or is "uncached memory
operations" just meant to read as "oustanding/incomplete memory operations" ?
Thanks,
Mark.
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 05/25] kvx: Add atomic/locking headers
2023-01-10 13:24 ` Mark Rutland
@ 2023-01-18 13:40 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-18 13:40 UTC (permalink / raw)
To: Mark Rutland, Jules Maselbas
Cc: Will Deacon, Peter Zijlstra, Boqun Feng, linux-kernel,
Clement Leger, Julian Vetter, Julien Villette
Hi Mark,
On 10/01/2023 14:24, Mark Rutland wrote:
> On Fri, Jan 06, 2023 at 03:11:58PM +0100, Jules Maselbas wrote:
>> Hi Mark,
>>
>> On Wed, Jan 04, 2023 at 09:53:24AM +0000, Mark Rutland wrote:
>>> On Tue, Jan 03, 2023 at 05:43:39PM +0100, Yann Sionneau wrote:
>>>> Add common headers (atomic, bitops, barrier and locking) for basic
>>>> kvx support.
>>>>
>>>> CC: Will Deacon <will@kernel.org>
>>>> CC: Peter Zijlstra <peterz@infradead.org>
>>>> CC: Boqun Feng <boqun.feng@gmail.com>
>>>> CC: Mark Rutland <mark.rutland@arm.com>
>>>> CC: linux-kernel@vger.kernel.org
>>>> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
>>>> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
>>>> Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
>>>> Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
>>>> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
>>>> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
>>>> Co-developed-by: Julien Villette <jvillette@kalray.eu>
>>>> Signed-off-by: Julien Villette <jvillette@kalray.eu>
>>>> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
>>>> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
>>>> ---
>>>> arch/kvx/include/asm/atomic.h | 104 +++++++++++++++++
>>>> arch/kvx/include/asm/barrier.h | 15 +++
>>>> arch/kvx/include/asm/bitops.h | 207 +++++++++++++++++++++++++++++++++
>>>> arch/kvx/include/asm/bitrev.h | 32 +++++
>>>> arch/kvx/include/asm/cmpxchg.h | 185 +++++++++++++++++++++++++++++
>>>> 5 files changed, 543 insertions(+)
>>>> create mode 100644 arch/kvx/include/asm/atomic.h
>>>> create mode 100644 arch/kvx/include/asm/barrier.h
>>>> create mode 100644 arch/kvx/include/asm/bitops.h
>>>> create mode 100644 arch/kvx/include/asm/bitrev.h
>>>> create mode 100644 arch/kvx/include/asm/cmpxchg.h
>>>>
>>>> diff --git a/arch/kvx/include/asm/atomic.h b/arch/kvx/include/asm/atomic.h
>>>> new file mode 100644
>>>> index 000000000000..eb8acbcbc70d
>>>> --- /dev/null
>>>> +++ b/arch/kvx/include/asm/atomic.h
>>>> @@ -0,0 +1,104 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +/*
>>>> + * Copyright (C) 2017-2023 Kalray Inc.
>>>> + * Author(s): Clement Leger
>>>> + */
>>>> +
>>>> +#ifndef _ASM_KVX_ATOMIC_H
>>>> +#define _ASM_KVX_ATOMIC_H
>>>> +
>>>> +#include <linux/types.h>
>>>> +
>>>> +#include <asm/cmpxchg.h>
>>>> +
>>>> +#define ATOMIC64_INIT(i) { (i) }
>>>> +
>>>> +#define arch_atomic64_cmpxchg(v, old, new) (arch_cmpxchg(&((v)->counter), old, new))
>>>> +#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), new))
>>>> +
>>>> +static inline long arch_atomic64_read(const atomic64_t *v)
>>>> +{
>>>> + return v->counter;
>>>> +}
>>> This is a plain read, and is *not* atomic.
>>>
>>> The compiler can replay a plain read an arbitrary number of times, and is
>>> permitted to split it into smaller accesses.
>>>
>>> At minimum this needs to be
>>>
>>> READ_ONCE(v->counter)
>>>
>>> ... which will prevent replay. Whether or not that's actually atomic will
>>> depend on the instructions the compiler generates, and how those instructions
>>> are defines in your architecture.
>> Good point, we are going to use {READ,WRITE}_ONCE macros
Done for V2 (which we will send pretty soon).
>>
>>> Do you have a single instruction that can read a 64-bit memory location, and is
>>> it guaranteed to result in a single access that cannot be split?
>> We do have a single instruction that can read a 64-bit memory location
>> (supported sizes are 8, 16, 32, 64, 128, 256).
>> All accesses are guaranteed to not be split, unless they are misaligned.
>> Furthermore, misaligned write accesses crossing a 32-byte boundary may
>> complete in a non-atomic way.
> Perfect, thanks for confirming!
>
> [...]
>
>>>> +static inline void arch_atomic64_set(atomic64_t *v, long i)
>>>> +{
>>>> + v->counter = i;
>>>> +}
>>> Same comments as for arch_atomic64_read(); at minimum this needs to be:
>>>
>>> WRITE_ONCE(v->counter, i)
>>>
>>> ... but that may or may not actually be atomic on your architecture.
>>>
>>>> +#define ATOMIC64_RETURN_OP(op, c_op) \
>>>> +static inline long arch_atomic64_##op##_return(long i, atomic64_t *v) \
>>>> +{ \
>>>> + long new, old, ret; \
>>>> + \
>>>> + do { \
>>>> + old = v->counter; \
>>>> + new = old c_op i; \
>>>> + ret = arch_cmpxchg(&v->counter, old, new); \
>>>> + } while (ret != old); \
>>>> + \
>>>> + return new; \
>>>> +}
>>>> +
>>>> +#define ATOMIC64_OP(op, c_op) \
>>>> +static inline void arch_atomic64_##op(long i, atomic64_t *v) \
>>>> +{ \
>>>> + long new, old, ret; \
>>>> + \
>>>> + do { \
>>>> + old = v->counter; \
>>>> + new = old c_op i; \
>>>> + ret = arch_cmpxchg(&v->counter, old, new); \
>>>> + } while (ret != old); \
>>>> +}
>>>> +
>>>> +#define ATOMIC64_FETCH_OP(op, c_op) \
>>>> +static inline long arch_atomic64_fetch_##op(long i, atomic64_t *v) \
>>>> +{ \
>>>> + long new, old, ret; \
>>>> + \
>>>> + do { \
>>>> + old = v->counter; \
>>>> + new = old c_op i; \
>>>> + ret = arch_cmpxchg(&v->counter, old, new); \
>>>> + } while (ret != old); \
>>>> + \
>>>> + return old; \
>>>> +}
>>> These look ok, but it'd be nicer if we could teach the generic atomic64 code to
>>> do this, like the generic atomic code does.
>>>
>>> We could rename the existing asm-generic/atomic64 code to atomic64-spinlock,
>>> and add a separate atomic64-cmpxchg (and likewise for the 32-bit code) to make
>>> that clearer and consistent.
>> I am not sure what this implies and how big this change might be,
>> but I'll take a look at this.
> Hmm... from a quick attempt just now it looks like that will be a bit more
> churny than I thought.
>
> We can always factor this out later, so feel free to leave it as-is, thgouh if
> we could make this genric (and have it look like asm-generic/atomic.h), it'd be
> nice for consistency and maintenance.
Ack for doing it later.
>
>>>> +
>>>> +#define ATOMIC64_OPS(op, c_op) \
>>>> + ATOMIC64_OP(op, c_op) \
>>>> + ATOMIC64_RETURN_OP(op, c_op) \
>>>> + ATOMIC64_FETCH_OP(op, c_op)
>>>> +
>>>> +ATOMIC64_OPS(and, &)
>>>> +ATOMIC64_OPS(or, |)
>>>> +ATOMIC64_OPS(xor, ^)
>>>> +ATOMIC64_OPS(add, +)
>>>> +ATOMIC64_OPS(sub, -)
>>>> +
>>>> +#undef ATOMIC64_OPS
>>>> +#undef ATOMIC64_FETCH_OP
>>>> +#undef ATOMIC64_OP
>>>> +
>>>> +static inline int arch_atomic_add_return(int i, atomic_t *v)
>>>> +{
>>>> + int new, old, ret;
>>>> +
>>>> + do {
>>>> + old = v->counter;
>>>> + new = old + i;
>>>> + ret = arch_cmpxchg(&v->counter, old, new);
>>>> + } while (ret != old);
>>>> +
>>>> + return new;
>>>> +}
>>>> +
>>>> +static inline int arch_atomic_sub_return(int i, atomic_t *v)
>>>> +{
>>>> + return arch_atomic_add_return(-i, v);
>>>> +}
>>> Likewise for these two.
> The 32-bit atomics should come from asm-generic/atomic.h and not be necesary,
> that has both arch_atomic_add_return() and arch_atomic_sub_return().
>
>>>> diff --git a/arch/kvx/include/asm/cmpxchg.h b/arch/kvx/include/asm/cmpxchg.h
>>>> new file mode 100644
>>>> index 000000000000..b1d128b060a2
>>>> --- /dev/null
>>>> +++ b/arch/kvx/include/asm/cmpxchg.h
>>>> @@ -0,0 +1,185 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +/*
>>>> + * Copyright (C) 2017-2023 Kalray Inc.
>>>> + * Author(s): Clement Leger
>>>> + * Yann Sionneau
>>>> + */
>>>> +
>>>> +#ifndef _ASM_KVX_CMPXCHG_H
>>>> +#define _ASM_KVX_CMPXCHG_H
>>>> +
>>>> +#include <linux/bits.h>
>>>> +#include <linux/types.h>
>>>> +#include <linux/build_bug.h>
>>>> +
>>>> +/*
>>>> + * On kvx, we have a boolean compare and swap which means that the operation
>>>> + * returns only the success of operation.
>>>> + * If operation succeed, this is simple, we just need to return the provided
>>>> + * old value. However, if it fails, we need to load the value to return it for
>>>> + * the caller. If the loaded value is different from the "old" provided by the
>>>> + * caller, we can return it since it will means it failed.
>>>> + * However, if for some reason the value we read is equal to the old value
>>>> + * provided by the caller, we can't simply return it or the caller will think it
>>>> + * succeeded. So if the value we read is the same as the "old" provided by
>>>> + * the caller, we try again until either we succeed or we fail with a different
>>>> + * value than the provided one.
>>>> + */
>>>> +#define __cmpxchg(ptr, old, new, op_suffix, load_suffix) \
>>>> +({ \
>>>> + register unsigned long __rn asm("r62"); \
>>>> + register unsigned long __ro asm("r63"); \
>>> Why do you need to specify the exact registers?
>> r62 and r63 are hardcoded in the inline assembly, they are caller saved.
>> I have a C implementation that uses builtins however this is not merged
>> in our tree yet (but I want to).
>>
>>> e.g. does some instruction use these implicitly, or do you need two adjacent
>>> register for encoding reasons?
>> The atomic compare and swap (acswap) instruction needs a register "pair"
>> which can only exists with "adjacent" registers: $r0r1, $r2r3 ect.
> Ok; and you don't have a way to ask GCC for an arbitrary register pair, so you
> chose r62+r63 as they can be clobbered?
>
> It might be worth looking into using an __int128_t to give you a register pair,
> assuming your calling convention mandares adjacency of the two halves. That
> could give you the pair while giving the compiler freedom to chose any suitable
> pair (assuming you have a suitable operand modifier to extract the low/high
> registers from the asm operand.
In the V2 we replaced the assembly code with C code.
acswap{d,w} instruction is generated by the mean of a compiler builtin.
This way
the code is much more readable and it allows to not hard code some
register pair, letting the compiler allocate them.
>
>>>> + __asm__ __volatile__ ( \
>>>> + /* Fence to guarantee previous store to be committed */ \
>>>> + "fence\n" \
>>> This implies you can implement the relaxed form of cmpxchg().
>>>
>>> What ordering do you get by default, and do you have any other barriers (e.g.
>>> for acquire/release semantics), or just "fence" ?
>> We have two barrier types:
>> - fence: ensure that all uncached memory operations are committed to
>> memory, typically used to ensure a write to memory is visible to
>> other cores.
>> - barrier: flush the core instruction pipeline and memory system
> Ok.
>
> In the absence of barriers do you have any ordering guarantee? e.g. weak like
> arm/power, TSO like x86, or something else entirely?
Our memory ordering model is weak.
```
store @data_addr = new_value
;;
store @data_is_valid_addr = 1
;;
```
By running the previous pseudo-code, there is no guarantee that another
CPU would see the *data_addr == new_value just because it sees
*data_is_valid_addr == 1
The correct code would need an extra bundle containing a `fence`
instruction in-between the 2 other bundles.
> When you say "uncached memory operations", does that mean that you have cached
> memory operatins which are not affected by the barriers, or is "uncached memory
> operations" just meant to read as "oustanding/incomplete memory operations" ?
The fence instruction will force all outstanding memory write operations
to finish and reach their target.
* For an uncached access it means that after the fence the DDR has been
updated (or the target device register).
* For a cached access it means the L2 cache has been updated, and
therefore all the corresponding L1D lines invalidated. (L1I is untouched!)
Bottom line is that after the fence, all other cores of the cluster
running Linux will see the new value. Except of course if the write is
through an uncached MMU mapping and the read is done through a cached
MMU mapping, but this seems normal.
Thanks for the review, it's much appreciated!
--
Yann
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 06/25] kvx: Add other common headers
2023-01-03 16:43 ` Yann Sionneau
` (6 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Peter Zijlstra, Ingo Molnar, Will Deacon,
Waiman Long, Boqun Feng, linux-kernel, Clement Leger,
Julian Vetter, Vincent Chardon
Add some other common headers for basic kvx support.
CC: Peter Zijlstra <peterz@infradead.org>
CC: Ingo Molnar <mingo@redhat.com>
CC: Will Deacon <will@kernel.org>
CC: Waiman Long <longman@redhat.com>
CC: Boqun Feng <boqun.feng@gmail.com>
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/asm-prototypes.h | 14 ++++++++
arch/kvx/include/asm/clocksource.h | 17 +++++++++
arch/kvx/include/asm/linkage.h | 13 +++++++
arch/kvx/include/asm/pci.h | 36 +++++++++++++++++++
arch/kvx/include/asm/sections.h | 18 ++++++++++
arch/kvx/include/asm/spinlock.h | 16 +++++++++
arch/kvx/include/asm/spinlock_types.h | 17 +++++++++
arch/kvx/include/asm/stackprotector.h | 47 +++++++++++++++++++++++++
arch/kvx/include/asm/timex.h | 20 +++++++++++
arch/kvx/include/asm/types.h | 12 +++++++
arch/kvx/include/uapi/asm/bitsperlong.h | 14 ++++++++
arch/kvx/include/uapi/asm/byteorder.h | 12 +++++++
tools/include/uapi/asm/bitsperlong.h | 2 ++
13 files changed, 238 insertions(+)
create mode 100644 arch/kvx/include/asm/asm-prototypes.h
create mode 100644 arch/kvx/include/asm/clocksource.h
create mode 100644 arch/kvx/include/asm/linkage.h
create mode 100644 arch/kvx/include/asm/pci.h
create mode 100644 arch/kvx/include/asm/sections.h
create mode 100644 arch/kvx/include/asm/spinlock.h
create mode 100644 arch/kvx/include/asm/spinlock_types.h
create mode 100644 arch/kvx/include/asm/stackprotector.h
create mode 100644 arch/kvx/include/asm/timex.h
create mode 100644 arch/kvx/include/asm/types.h
create mode 100644 arch/kvx/include/uapi/asm/bitsperlong.h
create mode 100644 arch/kvx/include/uapi/asm/byteorder.h
diff --git a/arch/kvx/include/asm/asm-prototypes.h b/arch/kvx/include/asm/asm-prototypes.h
new file mode 100644
index 000000000000..af032508e30c
--- /dev/null
+++ b/arch/kvx/include/asm/asm-prototypes.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_ASM_PROTOTYPES_H
+#define _ASM_KVX_ASM_PROTOTYPES_H
+
+#include <asm/string.h>
+
+#include <asm-generic/asm-prototypes.h>
+
+#endif /* _ASM_KVX_ASM_PROTOTYPES_H */
diff --git a/arch/kvx/include/asm/clocksource.h b/arch/kvx/include/asm/clocksource.h
new file mode 100644
index 000000000000..4df7c66ffbb5
--- /dev/null
+++ b/arch/kvx/include/asm/clocksource.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Yann Sionneau
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_CLOCKSOURCE_H
+#define _ASM_KVX_CLOCKSOURCE_H
+
+#include <linux/compiler.h>
+
+struct arch_clocksource_data {
+ void __iomem *regs;
+};
+
+#endif /* _ASM_KVX_CLOCKSOURCE_H */
diff --git a/arch/kvx/include/asm/linkage.h b/arch/kvx/include/asm/linkage.h
new file mode 100644
index 000000000000..84e1cacf67c2
--- /dev/null
+++ b/arch/kvx/include/asm/linkage.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Yann Sionneau
+ */
+
+#ifndef __ASM_KVX_LINKAGE_H
+#define __ASM_KVX_LINKAGE_H
+
+#define __ALIGN .align 4
+#define __ALIGN_STR ".align 4"
+
+#endif
diff --git a/arch/kvx/include/asm/pci.h b/arch/kvx/include/asm/pci.h
new file mode 100644
index 000000000000..d5bbaaf041b5
--- /dev/null
+++ b/arch/kvx/include/asm/pci.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Vincent Chardon
+ * Clement Leger
+ */
+
+#ifndef __ASM_KVX_PCI_H_
+#define __ASM_KVX_PCI_H_
+
+#include <linux/dma-mapping.h>
+#include <linux/pci.h>
+#include <linux/of_gpio.h>
+
+#define ARCH_GENERIC_PCI_MMAP_RESOURCE 1
+#define HAVE_PCI_MMAP 1
+
+extern int isa_dma_bridge_buggy;
+
+/* Can be used to override the logic in pci_scan_bus for skipping
+ * already-configured bus numbers - to be used for buggy BIOSes
+ * or architectures with incomplete PCI setup by the loader.
+ */
+#define pcibios_assign_all_busses() 0
+
+#define PCIBIOS_MIN_IO 0UL
+#define PCIBIOS_MIN_MEM 0UL
+
+#ifdef CONFIG_PCI_DOMAINS
+static inline int pci_proc_domain(struct pci_bus *bus)
+{
+ return pci_domain_nr(bus);
+}
+#endif /* CONFIG_PCI_DOMAINS */
+
+#endif /* _ASM_KVX_PCI_H */
diff --git a/arch/kvx/include/asm/sections.h b/arch/kvx/include/asm/sections.h
new file mode 100644
index 000000000000..0777675ef264
--- /dev/null
+++ b/arch/kvx/include/asm/sections.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SECTIONS_H
+#define _ASM_KVX_SECTIONS_H
+
+#include <asm-generic/sections.h>
+
+extern char __rodata_start[], __rodata_end[];
+extern char __initdata_start[], __initdata_end[];
+extern char __inittext_start[], __inittext_end[];
+extern char __exception_start[], __exception_end[];
+extern char __rm_firmware_regs_start[];
+
+#endif
diff --git a/arch/kvx/include/asm/spinlock.h b/arch/kvx/include/asm/spinlock.h
new file mode 100644
index 000000000000..ed32fdba1e19
--- /dev/null
+++ b/arch/kvx/include/asm/spinlock.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SPINLOCK_H
+#define _ASM_KVX_SPINLOCK_H
+
+#include <asm/qspinlock.h>
+#include <asm/qrwlock.h>
+
+/* See include/linux/spinlock.h */
+#define smp_mb__after_spinlock() smp_mb()
+
+#endif
diff --git a/arch/kvx/include/asm/spinlock_types.h b/arch/kvx/include/asm/spinlock_types.h
new file mode 100644
index 000000000000..929a7df16ef3
--- /dev/null
+++ b/arch/kvx/include/asm/spinlock_types.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SPINLOCK_TYPES_H
+#define _ASM_KVX_SPINLOCK_TYPES_H
+
+#if !defined(__LINUX_SPINLOCK_TYPES_RAW_H) && !defined(__ASM_SPINLOCK_H)
+# error "please don't include this file directly"
+#endif
+
+#include <asm-generic/qspinlock_types.h>
+#include <asm-generic/qrwlock_types.h>
+
+#endif /* _ASM_KVX_SPINLOCK_TYPES_H */
diff --git a/arch/kvx/include/asm/stackprotector.h b/arch/kvx/include/asm/stackprotector.h
new file mode 100644
index 000000000000..2c190bbb5efc
--- /dev/null
+++ b/arch/kvx/include/asm/stackprotector.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * derived from arch/mips/include/asm/stackprotector.h
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+/*
+ * GCC stack protector support.
+ *
+ * Stack protector works by putting predefined pattern at the start of
+ * the stack frame and verifying that it hasn't been overwritten when
+ * returning from the function. The pattern is called stack canary
+ * and gcc expects it to be defined by a global variable called
+ * "__stack_chk_guard" on KVX. This unfortunately means that on SMP
+ * we cannot have a different canary value per task.
+ */
+
+#ifndef __ASM_STACKPROTECTOR_H
+#define __ASM_STACKPROTECTOR_H
+
+#include <linux/random.h>
+#include <linux/version.h>
+
+extern unsigned long __stack_chk_guard;
+
+/*
+ * Initialize the stackprotector canary value.
+ *
+ * NOTE: this must only be called from functions that never return,
+ * and it must always be inlined.
+ */
+static __always_inline void boot_init_stack_canary(void)
+{
+ unsigned long canary;
+
+ /* Try to get a semi random initial value. */
+ get_random_bytes(&canary, sizeof(canary));
+ canary ^= LINUX_VERSION_CODE;
+ canary &= CANARY_MASK;
+
+ current->stack_canary = canary;
+ __stack_chk_guard = current->stack_canary;
+}
+
+#endif /* _ASM_STACKPROTECTOR_H */
diff --git a/arch/kvx/include/asm/timex.h b/arch/kvx/include/asm/timex.h
new file mode 100644
index 000000000000..51e346faa887
--- /dev/null
+++ b/arch/kvx/include/asm/timex.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_TIMEX_H
+#define _ASM_KVX_TIMEX_H
+
+#define get_cycles get_cycles
+
+#include <asm/sfr.h>
+#include <asm-generic/timex.h>
+
+static inline cycles_t get_cycles(void)
+{
+ return kvx_sfr_get(PM0);
+}
+
+#endif /* _ASM_KVX_TIMEX_H */
diff --git a/arch/kvx/include/asm/types.h b/arch/kvx/include/asm/types.h
new file mode 100644
index 000000000000..1e6c024ee892
--- /dev/null
+++ b/arch/kvx/include/asm/types.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_TYPES_H
+#define _ASM_KVX_TYPES_H
+
+#include <asm-generic/int-ll64.h>
+
+#endif /* _ASM_KVX_TYPES_H */
diff --git a/arch/kvx/include/uapi/asm/bitsperlong.h b/arch/kvx/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..02a91596d567
--- /dev/null
+++ b/arch/kvx/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _UAPI_ASM_KVX_BITSPERLONG_H
+#define _UAPI_ASM_KVX_BITSPERLONG_H
+
+#define __BITS_PER_LONG 64
+
+#include <asm-generic/bitsperlong.h>
+
+#endif /* _UAPI_ASM_KVX_BITSPERLONG_H */
diff --git a/arch/kvx/include/uapi/asm/byteorder.h b/arch/kvx/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..b7d827daec73
--- /dev/null
+++ b/arch/kvx/include/uapi/asm/byteorder.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_BYTEORDER_H
+#define _ASM_KVX_BYTEORDER_H
+
+#include <linux/byteorder/little_endian.h>
+
+#endif /* _ASM_KVX_BYTEORDER_H */
diff --git a/tools/include/uapi/asm/bitsperlong.h b/tools/include/uapi/asm/bitsperlong.h
index da5206517158..40272ffa9c32 100644
--- a/tools/include/uapi/asm/bitsperlong.h
+++ b/tools/include/uapi/asm/bitsperlong.h
@@ -19,6 +19,8 @@
#include "../../../arch/alpha/include/uapi/asm/bitsperlong.h"
#elif defined(__loongarch__)
#include "../../../arch/loongarch/include/uapi/asm/bitsperlong.h"
+#elif defined(__kvx__)
+#include "../../../arch/kvx/include/uapi/asm/bitsperlong.h"
#else
#include <asm-generic/bitsperlong.h>
#endif
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 07/25] kvx: Add boot and setup routines
2023-01-03 16:43 ` Yann Sionneau
` (7 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, Alex Michon, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette,
Marc Poulhiès, Luc Michel, Marius Gligor
Add basic boot, setup and reset routines for kvx.
CC: linux-kernel@vger.kernel.org
Co-developed-by: Alex Michon <amichon@kalray.eu>
Signed-off-by: Alex Michon <amichon@kalray.eu>
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Missonnier <gmissonnier@kalray.eu>
Signed-off-by: Guillaume Missonnier <gmissonnier@kalray.eu>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Julien Hascoet <jhascoet@kalray.eu>
Signed-off-by: Julien Hascoet <jhascoet@kalray.eu>
Co-developed-by: Julien Villette <jvillette@kalray.eu>
Signed-off-by: Julien Villette <jvillette@kalray.eu>
Co-developed-by: Marc Poulhiès <dkm@kataplop.net>
Signed-off-by: Marc Poulhiès <dkm@kataplop.net>
Co-developed-by: Luc Michel <lmichel@kalray.eu>
Signed-off-by: Luc Michel <lmichel@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/setup.h | 29 ++
arch/kvx/kernel/common.c | 11 +
arch/kvx/kernel/head.S | 612 +++++++++++++++++++++++++++++++++++
arch/kvx/kernel/prom.c | 24 ++
arch/kvx/kernel/reset.c | 37 +++
arch/kvx/kernel/setup.c | 178 ++++++++++
arch/kvx/kernel/time.c | 242 ++++++++++++++
7 files changed, 1133 insertions(+)
create mode 100644 arch/kvx/include/asm/setup.h
create mode 100644 arch/kvx/kernel/common.c
create mode 100644 arch/kvx/kernel/head.S
create mode 100644 arch/kvx/kernel/prom.c
create mode 100644 arch/kvx/kernel/reset.c
create mode 100644 arch/kvx/kernel/setup.c
create mode 100644 arch/kvx/kernel/time.c
diff --git a/arch/kvx/include/asm/setup.h b/arch/kvx/include/asm/setup.h
new file mode 100644
index 000000000000..9c27d5981442
--- /dev/null
+++ b/arch/kvx/include/asm/setup.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SETUP_H
+#define _ASM_KVX_SETUP_H
+
+#include <linux/const.h>
+
+#include <asm-generic/setup.h>
+
+/* Magic is found in r0 when some parameters are given to kernel */
+#define LINUX_BOOT_PARAM_MAGIC ULL(0x31564752414E494C)
+
+#ifndef __ASSEMBLY__
+
+void early_fixmap_init(void);
+
+void setup_device_tree(void);
+
+void setup_arch_memory(void);
+
+void kvx_init_mmu(void);
+
+#endif
+
+#endif /* _ASM_KVX_SETUP_H */
diff --git a/arch/kvx/kernel/common.c b/arch/kvx/kernel/common.c
new file mode 100644
index 000000000000..322498f034fd
--- /dev/null
+++ b/arch/kvx/kernel/common.c
@@ -0,0 +1,11 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/percpu-defs.h>
+#include <linux/sched/task.h>
+
+DEFINE_PER_CPU(int, __preempt_count) = INIT_PREEMPT_COUNT;
+EXPORT_PER_CPU_SYMBOL(__preempt_count);
diff --git a/arch/kvx/kernel/head.S b/arch/kvx/kernel/head.S
new file mode 100644
index 000000000000..ab9848500f39
--- /dev/null
+++ b/arch/kvx/kernel/head.S
@@ -0,0 +1,612 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Marius Gligor
+ * Julian Vetter
+ * Julien Hascoet
+ * Yann Sionneau
+ * Marc Poulhiès
+ */
+#include <asm/thread_info.h>
+#include <asm/page_size.h>
+#include <asm/pwr_ctrl.h>
+#include <asm/sfr_defs.h>
+#include <asm/sys_arch.h>
+#include <asm/privilege.h>
+#include <asm/tlb_defs.h>
+#include <asm/mem_map.h>
+#include <asm/rm_fw.h>
+#include <asm/setup.h>
+#include <asm/page.h>
+
+#include <linux/linkage.h>
+#include <linux/init.h>
+
+#ifdef CONFIG_SMP
+#define SECONDARY_START_ADDR smp_secondary_start
+#else
+#define SECONDARY_START_ADDR proc_power_off
+#endif
+
+#define PS_VAL_WFXL(__field, __val) \
+ SFR_SET_VAL_WFXL(PS, __field, __val)
+
+#define PS_WFXL_VALUE PS_VAL_WFXL(HLE, 1) | \
+ PS_VAL_WFXL(USE, 1) | \
+ PS_VAL_WFXL(DCE, 1) | \
+ PS_VAL_WFXL(ICE, 1) | \
+ PS_VAL_WFXL(MME, 1) | \
+ PS_VAL_WFXL(MMUP, 1) | \
+ PS_VAL_WFXL(ET, 0) | \
+ PS_VAL_WFXL(HTD, 0) | \
+ PS_VAL_WFXL(PMJ, KVX_SUPPORTED_PSIZE)
+
+#define PCR_VAL_WFXM(__field, __val) \
+ SFR_SET_VAL_WFXM(PCR, __field, __val)
+
+#define PCR_WFXM_VALUE PCR_VAL_WFXM(L1CE, 1)
+
+/* 30 sec for primary watchdog timeout */
+#define PRIMARY_WATCHDOG_VALUE (30000000000UL)
+
+#define TCR_WFXL_VALUE SFR_SET_VAL_WFXL(TCR, WUI, 1) | \
+ SFR_SET_VAL_WFXL(TCR, WCE, 1)
+
+/* Enable STOP in WS */
+#define WS_ENABLE_WU2 (KVX_SFR_WS_WU2_MASK)
+/* We only want to clear bits in ws */
+#define WS_WFXL_VALUE (WS_ENABLE_WU2)
+
+/* SMP stuff */
+#define RM_PID_MASK ((KVX_RM_ID) << KVX_SFR_PCR_PID_SHIFT)
+
+#define PWR_CTRL_ADDR 0xA40000
+
+#define PWR_CTRL_GLOBAL_CONFIG_VALUE \
+ (1 << KVX_PWR_CTRL_GLOBAL_SET_PE_EN_SHIFT)
+
+/* Clean error and selected buffer */
+#define MMC_CLEAR_ERROR (KVX_SFR_MMC_E_MASK)
+
+#define TEH_VIRTUAL_MEMORY \
+ TLB_MK_TEH_ENTRY(PAGE_OFFSET, 0, TLB_G_GLOBAL, 0)
+
+#define TEL_VIRTUAL_MEMORY \
+ TLB_MK_TEL_ENTRY(PHYS_OFFSET, TLB_PS_512M, TLB_ES_A_MODIFIED,\
+ TLB_CP_W_C, TLB_PA_NA_RWX)
+
+/* (TEH|TEL)_SHARED_MEMORY are mapping 0x0 to 0x0 */
+#define TEH_SHARED_MEMORY \
+ TLB_MK_TEH_ENTRY(0, 0, TLB_G_GLOBAL, 0)
+
+#define TEL_SHARED_MEMORY \
+ TLB_MK_TEL_ENTRY(0, TLB_PS_2M, TLB_ES_A_MODIFIED,\
+ TLB_CP_W_C, TLB_PA_NA_RWX)
+
+#define TEH_GDB_PAGE_MEMORY \
+ TLB_MK_TEH_ENTRY(0, 0, TLB_G_GLOBAL, 0)
+
+#define TEL_GDB_PAGE_MEMORY \
+ TLB_MK_TEL_ENTRY(0, TLB_PS_4K, TLB_ES_A_MODIFIED,\
+ TLB_CP_U_U, TLB_PA_RWX_RWX)
+
+/**
+ * Macros
+ */
+.altmacro
+
+/* To select the JTLB we clear SB from MMC */
+.macro select_jtlb scratch_reg
+ make \scratch_reg, KVX_SFR_MMC_SB_MASK
+ ;;
+ wfxl $mmc, \scratch_reg
+.endm
+
+/* To select the LTLB we set SB from MMC */
+.macro select_ltlb scratch_reg
+ make \scratch_reg, KVX_SFR_MMC_SB_MASK << 32
+ ;;
+ wfxl $mmc, \scratch_reg
+.endm
+
+/* Set SW of the MMC with number found in the reg register */
+.macro select_way_from_register reg scratch1 scratch2
+ slld \scratch1 = \reg, KVX_SFR_MMC_SW_SHIFT
+ make \scratch2 = KVX_SFR_MMC_SW_MASK
+ ;;
+ slld \scratch1 = \scratch1, 32
+ ;;
+ ord \scratch1 = \scratch1, \scratch2
+ ;;
+ wfxl $mmc = \scratch1
+.endm
+
+/* Set SW of the MMC with the immediate */
+.macro select_way_from_immediate imm scratch1 scratch2
+ make \scratch1 = (\imm << KVX_SFR_MMC_SW_SHIFT) << 32
+ make \scratch2 = KVX_SFR_MMC_SW_MASK
+ ;;
+ ord \scratch1 = \scratch1, \scratch2
+ ;;
+ wfxl $mmc = \scratch1
+.endm
+
+/* write tlb after setting teh and tel registers */
+.macro write_tlb_entry teh tel
+ set $teh = \teh
+ ;;
+ set $tel = \tel
+ ;;
+ tlbwrite
+.endm
+
+/* Boot args */
+#define BOOT_ARGS_COUNT 2
+.align 16
+.section .boot.data, "aw", @progbits
+rm_boot_args:
+.skip BOOT_ARGS_COUNT * 8
+
+/*
+ * This is our entry point. When entering from bootloader,
+ * the following registers are set:
+ * $r0 is a magic (LINUX_BOOT_PARAM_MAGIC)
+ * $r1 device tree pointer
+ *
+ * WARNING WARNING WARNING
+ * ! DO NOT CLOBBER THEM !
+ * WARNING WARNING WARNING
+ *
+ * Try to use register above $r20 to ease parameter adding in future
+ */
+
+__HEAD
+
+.align 8
+.section .boot.startup, "ax", @progbits
+
+ENTRY(kvx_start)
+ /* Setup 64 bit really early to avoid bugs */
+ make $r21 = PS_VAL_WFXL(V64, 1)
+ ;;
+ wfxl $ps = $r21
+ ;;
+ call asm_init_pl
+ ;;
+ get $r20 = $pcr
+ ;;
+ andd $r21 = $r20, RM_PID_MASK
+ ;;
+ cb.dnez $r21 ? asm_rm_cfg_pwr_ctrl
+ ;;
+init_core:
+ /**
+ * Setup watchdog early to catch potential
+ * crash before watchdog driver probe
+ */
+ make $r25 = PRIMARY_WATCHDOG_VALUE
+ make $r26 = TCR_WFXL_VALUE
+ ;;
+ set $wdv = $r25
+ ;;
+ wfxl $tcr, $r26
+ ;;
+ call asm_init_mmu
+ ;;
+ /* Setup default processor status */
+ make $r25 = PS_WFXL_VALUE
+ make $r26 = PCR_WFXM_VALUE
+ ;;
+ /**
+ * There is nothing much we can do if we take a early trap since the
+ * kernel is not yet ready to handle them.
+ * Register this as the early exception handler to at least avoid
+ * going in a black hole.
+ */
+ make $r27 = __early_exception_start
+ ;;
+ set $ev = $r27
+ ;;
+ wfxm $pcr = $r26
+ ;;
+ wfxl $ps = $r25
+ ;;
+ /* Use as break point for debugging purpose.
+ See Documentation/kvx/kvx.txt for more details. */
+gdb_mmu_enabled:
+ /* Extract processor identifier */
+ get $r24 = $pcr
+ ;;
+ extfz $r24 = $r24, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID)
+ ;;
+ /* If proc 0, then go to clear bss and do normal boot */
+ cb.deqz $r24? clear_bss
+ make $r25 = SECONDARY_START_ADDR
+ ;;
+ icall $r25
+ ;;
+clear_bss:
+ /* Copy bootloader arguments before cloberring them */
+ copyd $r20 = $r0
+ copyd $r21 = $r1
+ ;;
+ /* Clear BSS */
+ make $r0 = __bss_start
+ make $r1 = __bss_stop
+ call asm_memzero
+ ;;
+ /* Setup stack */
+ make $r40 = init_thread_union
+ make $r41 = init_task
+ ;;
+ set $sr = $r41
+ copyd $r0 = $r20
+ copyd $r1 = $r21
+ ;;
+ addd $sp = $r40, THREAD_SIZE
+ /* Clear frame pointer */
+ make $fp = 0x0
+ /* Setup the exception handler */
+ make $r27 = __exception_start
+ ;;
+ set $ev = $r27
+ /* Here we go ! start the C stuff */
+ make $r20 = arch_low_level_start
+ ;;
+ icall $r20
+ ;;
+ make $r20 = proc_power_off
+ ;;
+ igoto $r20
+ ;;
+ENDPROC(kvx_start)
+
+/**
+ * When PE 0 is started from the RM, arguments from the bootloaders are copied
+ * into rm_boot_args. It allows to give parameters from RM to PE.
+ * Note that the 4K alignment is required by the reset pc register...
+ */
+.align (4 * 1024)
+ENTRY(pe_start_wrapper)
+ make $r0 = rm_boot_args
+ make $r27 = PWR_CTRL_ADDR
+ make $r28 = kvx_start
+ ;;
+ lq $r0r1 = 0[$r0]
+ ;;
+ /* Set reset PC back to original value for SMP start */
+ sd KVX_PWR_CTRL_RESET_PC_OFFSET[$r27] = $r28
+ ;;
+ fence
+ goto kvx_start
+ ;;
+ENDPROC(pe_start_wrapper)
+
+/**
+ * asm_memzero - Clear a memory zone with zeroes
+ * $r0 is the start of memory zone (must be align on 32 bytes boundary)
+ * $r1 is the end of memory zone (must be align on 32 bytes boundary)
+ */
+ENTRY(asm_memzero)
+ sbfd $r32 = $r0, $r1
+ make $r36 = 0
+ make $r37 = 0
+ ;;
+ make $r38 = 0
+ make $r39 = 0
+ /* Divide by 32 for hardware loop */
+ srld $r32, $r32, 5
+ ;;
+ /* Clear memory with hardware loop */
+ loopdo $r32, clear_mem_done
+ ;;
+ so 0[$r0] = $r36r37r38r39
+ addd $r0 = $r0, 32
+ ;;
+ clear_mem_done:
+ ret
+ ;;
+ENDPROC(asm_memzero)
+
+/**
+ * Configure the power controller to be accessible by PEs and start the
+ * rm firmware
+ */
+ENTRY(asm_rm_cfg_pwr_ctrl)
+ /* Enable hwloop for memzero */
+ make $r32 = PS_VAL_WFXL(HLE, 1)
+ ;;
+ wfxl $ps = $r32
+ ;;
+ make $r26 = PWR_CTRL_ADDR
+ make $r27 = PWR_CTRL_GLOBAL_CONFIG_VALUE
+ ;;
+ /* Set PE enable in power controller */
+ sd PWR_CTRL_GLOBAL_CONFIG_OFFSET[$r26] = $r27
+ make $r28 = rm_boot_args
+ ;;
+ /* Store parameters for PE0 */
+ sq 0[$r28] = $r0r1
+ /* Copy parameters before calling asm_memzero */
+ copyd $r20 = $r0
+ copyd $r21 = $r1
+ make $r29 = pe_start_wrapper
+ ;;
+ /* Set PE reset PC to arguments wrapper */
+ sd KVX_PWR_CTRL_RESET_PC_OFFSET[$r26] = $r29
+ ;;
+ /* Fence to make sure parameters will be visible by PE 0 */
+ fence
+ ;;
+ /* Clear L2 registers */
+ make $r0 = __rm_firmware_regs_start
+ make $r1 = __rm_firmware_regs_end
+ call asm_memzero
+ ;;
+ /* Start PE 0 (1 << cpu) */
+ make $r27 = 1
+ make $r26 = PWR_CTRL_ADDR
+ ;;
+ /* Wake up PE0 */
+ sd PWR_CTRL_WUP_SET_OFFSET[$r26] = $r27
+ ;;
+ /* And clear wakeup to allow PE0 to sleep */
+ sd PWR_CTRL_WUP_CLEAR_OFFSET[$r26] = $r27
+ ;;
+ /* Restore parameters */
+ copyd $r0 = $r20
+ copyd $r1 = $r21
+ ;;
+ /* Load device tree for RM firmware */
+ make $r21 = __PA(__dtb_start)
+ /* Check if magic is in $r0, if so, device tree to use is in $r1 */
+ compd.eq $r20 = $r0, LINUX_BOOT_PARAM_MAGIC
+ ;;
+ cmoved.dnez $r20? $r21 = $r1
+ ;;
+ /* Second argument is regs size */
+ make $r1 = __rm_firmware_regs_end
+ /* First argument is regs address */
+ make $r0 = __rm_firmware_regs_start
+ ;;
+ make $r20 = __rm_firmware_start
+ /* Third argument is dtb address */
+ copyd $r2 = $r21
+ ;;
+ /* Go to firmware */
+ igoto $r20
+ ;;
+ENDPROC(asm_rm_cfg_pwr_ctrl)
+
+#define request_ownership(__pl) ;\
+ make $r21 = SYO_WFXL_VALUE_##__pl ;\
+ ;; ;\
+ wfxl $syow = $r21 ;\
+ ;; ;\
+ make $r21 = HTO_WFXL_VALUE_##__pl ;\
+ ;; ;\
+ wfxl $htow = $r21 ;\
+ ;; ;\
+ make $r21 = MO_WFXL_VALUE_##__pl ;\
+ make $r22 = MO_WFXM_VALUE_##__pl ;\
+ ;; ;\
+ wfxl $mow = $r21 ;\
+ ;; ;\
+ wfxm $mow = $r22 ;\
+ ;; ;\
+ make $r21 = ITO_WFXL_VALUE_##__pl ;\
+ make $r22 = ITO_WFXM_VALUE_##__pl ;\
+ ;; ;\
+ wfxl $itow = $r21 ;\
+ ;; ;\
+ wfxm $itow = $r22 ;\
+ ;; ;\
+ make $r21 = PSO_WFXL_VALUE_##__pl ;\
+ make $r22 = PSO_WFXM_VALUE_##__pl ;\
+ ;; ;\
+ wfxl $psow = $r21 ;\
+ ;; ;\
+ wfxm $psow = $r22 ;\
+ ;; ;\
+ make $r21 = DO_WFXL_VALUE_##__pl ;\
+ ;; ;\
+ wfxl $dow = $r21 ;\
+ ;;
+
+/**
+ * Initialize privilege level for Kernel
+ */
+ENTRY(asm_init_pl)
+ get $r21 = $ps
+ ;;
+ /* Extract privilege level from $ps to check if we need to
+ * lower our privilege level
+ */
+ extfz $r20 = $r21, KVX_SFR_END(PS_PL), KVX_SFR_START(PS_PL)
+ ;;
+ /* If our privilege level is 0, then we need to lower in execution level
+ * to ring 1 in order to let the debug routines be inserted at runtime
+ * by the JTAG. In both case, we will request the resources we need for
+ * linux to run.
+ */
+ cb.deqz $r20? delegate_pl
+ ;;
+ /*
+ * When someone is already above us, request the resources we need to
+ * run the kernel. No need to request double exception or ECC traps for
+ * instance. When doing so, the more privileged level will trap for
+ * permission and delegate us the required resources.
+ */
+ request_ownership(PL_CUR)
+ ;;
+ ret
+ ;;
+delegate_pl:
+ request_ownership(PL_CUR_PLUS_1)
+ ;;
+ /* Copy our $ps into $sps for 1:1 restoration */
+ get $r22 = $ps
+ ;;
+ /* We will return to $ra after rfe */
+ get $r21 = $ra
+ /* Set privilege level to +1 is $sps */
+ addd $r22 = $r22, PL_CUR_PLUS_1
+ ;;
+ set $spc = $r21
+ ;;
+ set $sps = $r22
+ ;;
+ rfe
+ ;;
+ENDPROC(asm_init_pl)
+
+/**
+ * Reset and initialize minimal tlb entries
+ */
+ENTRY(asm_init_mmu)
+ make $r20 = MMC_CLEAR_ERROR
+ ;;
+ wfxl $mmc = $r20
+ ;;
+ /* Reset the JTLB */
+ select_jtlb $r20
+ ;;
+ make $r20 = (MMU_JTLB_SETS - 1) /* Used to select the set */
+ make $r21 = 0 /* Used for shifting and as scratch register */
+ ;;
+ set $tel = $r21 /* tel is always equal to 0 */
+ ;;
+ clear_jtlb:
+ slld $r21 = $r20, KVX_SFR_TEH_PN_SHIFT
+ addd $r20 = $r20, -1
+ ;;
+ set $teh = $r21
+ ;;
+ make $r22 = (MMU_JTLB_WAYS - 1) /* Used to select the way */
+ ;;
+ loop_jtlb_way:
+ select_way_from_register $r22 $r23 $r24
+ ;;
+ tlbwrite
+ ;;
+ addd $r22 = $r22, -1
+ ;;
+ cb.dgez $r22? loop_jtlb_way
+ ;;
+ /* loop_jtlb_way done */
+ cb.dgez $r20? clear_jtlb
+ ;;
+ clear_jtlb_done:
+ /* Reset the LTLB */
+ select_ltlb $r20
+ ;;
+ clear_ltlb:
+ /* There is only one set that is 0 so we can reuse the same
+ values for TEH and TEL. */
+ make $r20 = (MMU_LTLB_WAYS - 1)
+ ;;
+ loop_ltlb_way:
+ select_way_from_register $r20, $r21, $r22
+ ;;
+ tlbwrite
+ ;;
+ addd $r20 = $r20, -1
+ ;;
+ cb.dgez $r20? loop_ltlb_way
+ ;;
+ clear_ltlb_done:
+
+ /* See Documentation/kvx/kvx.txt for details about the settings of
+ the LTLB */
+ select_way_from_immediate LTLB_ENTRY_KERNEL_TEXT, $r20, $r21
+ ;;
+ make $r20 = TEH_VIRTUAL_MEMORY
+ make $r21 = TEL_VIRTUAL_MEMORY
+ ;;
+ write_tlb_entry $r20, $r21
+ ;;
+ select_way_from_immediate LTLB_ENTRY_EARLY_SMEM, $r20, $r21
+ ;;
+ make $r20 = TEH_SHARED_MEMORY
+ make $r21 = TEL_SHARED_MEMORY
+ ;;
+ write_tlb_entry $r20, $r21
+ ;;
+ select_way_from_immediate LTLB_ENTRY_GDB_PAGE, $r20, $r21
+ ;;
+ make $r20 = _debug_start_lma
+ make $r21 = _debug_start
+ ;;
+ andd $r20 = $r20, KVX_SFR_TEH_PN_MASK
+ andd $r21 = $r21, KVX_SFR_TEL_FN_MASK
+ ;;
+ addd $r20 = $r20, TEH_GDB_PAGE_MEMORY
+ addd $r21 = $r21, TEL_GDB_PAGE_MEMORY
+ ;;
+ write_tlb_entry $r20, $r21
+ ;;
+ ret
+ ;;
+ENDPROC(asm_init_mmu)
+
+/**
+ * Entry point for secondary processors
+ * $r24 has been set in caller and is the proc id
+ */
+ENTRY(smp_secondary_start)
+#ifdef CONFIG_SMP
+ dinval
+ ;;
+ iinval
+ ;;
+ barrier
+ ;;
+ /* Enable L2$ in $ps since it is mandatory for SMP */
+ make $r25 = PS_VAL_WFXL(L2E, 1)
+ ;;
+ wfxl $ps = $r25
+ ;;
+ make $r25 = __cpu_up_task_pointer
+ make $r26 = __cpu_up_stack_pointer
+ ;;
+ ld.xs $sp = $r24[$r26]
+ /* Clear frame pointer */
+ make $fp = 0x0
+ ;;
+ ld.xs $r25 = $r24[$r25]
+ ;;
+ set $sr = $r25
+ make $r27 = __exception_start
+ ;;
+ set $ev = $r27
+ make $r26 = start_kernel_secondary
+ ;;
+ icall $r26
+ ;;
+#endif
+ENDPROC(smp_secondary_start)
+
+ENTRY(proc_power_off)
+ make $r1 = WS_WFXL_VALUE
+ ;;
+ /* Enable STOP */
+ wfxl $ws, $r1
+ ;;
+1: stop
+ ;;
+ goto 1b
+ ;;
+ENDPROC(proc_power_off)
+
+/**
+ * This code will be put in rm_firmware section by default.
+ * If the rm_firmware section is overridden using another firmware,
+ * this default code will be overridden.
+ */
+.section .rm_firmware.default, "ax", @progbits
+ENTRY(rm_firmware)
+ goto proc_power_off
+ ;;
+ENDPROC(rm_firmware)
diff --git a/arch/kvx/kernel/prom.c b/arch/kvx/kernel/prom.c
new file mode 100644
index 000000000000..a5241aa66903
--- /dev/null
+++ b/arch/kvx/kernel/prom.c
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/of_platform.h>
+#include <linux/of_fdt.h>
+#include <linux/printk.h>
+#include <linux/init.h>
+
+void __init setup_device_tree(void)
+{
+ const char *name;
+
+ name = of_flat_dt_get_machine_name();
+ if (!name)
+ return;
+
+ pr_info("Machine model: %s\n", name);
+ dump_stack_set_arch_desc("%s (DT)", name);
+
+ unflatten_device_tree();
+}
diff --git a/arch/kvx/kernel/reset.c b/arch/kvx/kernel/reset.c
new file mode 100644
index 000000000000..afa0ceb9d7e9
--- /dev/null
+++ b/arch/kvx/kernel/reset.c
@@ -0,0 +1,37 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/pm.h>
+#include <linux/reboot.h>
+
+#include <asm/processor.h>
+
+static void kvx_default_power_off(void)
+{
+ smp_send_stop();
+ local_cpu_stop();
+}
+
+void (*pm_power_off)(void) = kvx_default_power_off;
+EXPORT_SYMBOL(pm_power_off);
+
+void machine_restart(char *cmd)
+{
+ smp_send_stop();
+ do_kernel_restart(cmd);
+ pr_err("Reboot failed -- System halted\n");
+ local_cpu_stop();
+}
+
+void machine_halt(void)
+{
+ pm_power_off();
+}
+
+void machine_power_off(void)
+{
+ pm_power_off();
+}
diff --git a/arch/kvx/kernel/setup.c b/arch/kvx/kernel/setup.c
new file mode 100644
index 000000000000..6be74e587dca
--- /dev/null
+++ b/arch/kvx/kernel/setup.c
@@ -0,0 +1,178 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/start_kernel.h>
+#include <linux/screen_info.h>
+#include <linux/console.h>
+#include <linux/linkage.h>
+#include <linux/export.h>
+#include <linux/of_fdt.h>
+#include <linux/kernel.h>
+#include <linux/string.h>
+#include <linux/sched.h>
+#include <linux/init.h>
+#include <linux/smp.h>
+
+#include <asm/processor.h>
+#include <asm/l2_cache.h>
+#include <asm/sections.h>
+#include <asm/hw_irq.h>
+#include <asm/setup.h>
+#include <asm/rm_fw.h>
+#include <asm/page.h>
+#include <asm/sfr.h>
+#include <asm/mmu.h>
+#include <asm/smp.h>
+
+struct screen_info screen_info;
+
+unsigned long memory_start;
+EXPORT_SYMBOL(memory_start);
+unsigned long memory_end;
+EXPORT_SYMBOL(memory_end);
+
+DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_kvx, cpu_info);
+EXPORT_PER_CPU_SYMBOL(cpu_info);
+
+static bool use_streaming = true;
+static int __init parse_kvx_streaming(char *arg)
+{
+ strtobool(arg, &use_streaming);
+
+ if (!use_streaming) {
+ pr_info("disabling streaming\n");
+ kvx_sfr_set_field(PS, USE, 0);
+ }
+
+ return 0;
+}
+early_param("kvx.streaming", parse_kvx_streaming);
+
+static void __init setup_user_privilege(void)
+{
+ /*
+ * We want to let the user control various fields of ps:
+ * - hardware loop
+ * - instruction cache enable
+ * - streaming enable
+ */
+ uint64_t mask = KVX_SFR_PSOW_HLE_MASK |
+ KVX_SFR_PSOW_ICE_MASK |
+ KVX_SFR_PSOW_USE_MASK;
+
+ uint64_t value = (1 << KVX_SFR_PSOW_HLE_SHIFT) |
+ (1 << KVX_SFR_PSOW_ICE_SHIFT) |
+ (1 << KVX_SFR_PSOW_USE_SHIFT);
+
+ kvx_sfr_set_mask(PSOW, mask, value);
+}
+
+void __init setup_cpuinfo(void)
+{
+ struct cpuinfo_kvx *n = this_cpu_ptr(&cpu_info);
+ u64 pcr = kvx_sfr_get(PCR);
+
+ n->copro_enable = kvx_sfr_field_val(pcr, PCR, COE);
+ n->arch_rev = kvx_sfr_field_val(pcr, PCR, CAR);
+ n->uarch_rev = kvx_sfr_field_val(pcr, PCR, CMA);
+}
+
+/*
+ * Everything that needs to be setup PER cpu should be put here.
+ * This function will be called by per-cpu setup routine.
+ */
+void __init setup_processor(void)
+{
+ /* Clear performance monitor 0 */
+ kvx_sfr_set_field(PMC, PM0C, 0);
+
+#ifdef CONFIG_ENABLE_TCA
+ /* Enable TCA (COE = Coprocessor Enable) */
+ kvx_sfr_set_field(PCR, COE, 1);
+#else
+ kvx_sfr_set_field(PCR, COE, 0);
+#endif
+
+ /*
+ * On kvx, we have speculative accesses which differ from normal
+ * accesses by the fact their trapping policy is directed by mmc.sne
+ * (speculative no-mapping enable) and mmc.spe (speculative protection
+ * enabled).
+ * To handle these accesses properly, we disable all traps on
+ * speculative accesses while in kernel and user (sne & spe)
+ * in order to silently discard data if fetched.
+ * This allows to do an effective prefetch.
+ */
+ kvx_sfr_set_field(MMC, SNE, 0);
+ kvx_sfr_set_field(MMC, SPE, 0);
+
+ if (!use_streaming)
+ kvx_sfr_set_field(PS, USE, 0);
+
+ kvx_init_core_irq();
+
+ setup_user_privilege();
+
+ setup_cpuinfo();
+}
+
+static char builtin_cmdline[COMMAND_LINE_SIZE] __initdata = CONFIG_CMDLINE;
+
+void __init setup_arch(char **cmdline_p)
+{
+ if (builtin_cmdline[0]) {
+ /* append boot loader cmdline to builtin */
+ strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
+ strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+ strscpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+ }
+
+ *cmdline_p = boot_command_line;
+
+ setup_processor();
+
+ /* Jump labels needs fixmap to be setup for text modifications */
+ early_fixmap_init();
+
+ /* Parameters might set static keys */
+ jump_label_init();
+ /*
+ * Parse early param after setting up arch memory since
+ * we need fixmap for earlycon and fixedmap need to do
+ * memory allocation (fixed_range_init).
+ */
+ parse_early_param();
+
+ setup_arch_memory();
+
+ paging_init();
+
+ setup_device_tree();
+
+ smp_init_cpus();
+
+#ifdef CONFIG_VT
+ conswitchp = &dummy_con;
+#endif
+}
+
+asmlinkage __visible void __init arch_low_level_start(unsigned long r0,
+ void *dtb_ptr)
+{
+ void *dt = __dtb_start;
+
+ kvx_mmu_early_setup();
+
+ if (r0 == LINUX_BOOT_PARAM_MAGIC)
+ dt = __va(dtb_ptr);
+
+ if (!early_init_dt_scan(dt))
+ panic("Missing device tree\n");
+
+ start_kernel();
+}
diff --git a/arch/kvx/kernel/time.c b/arch/kvx/kernel/time.c
new file mode 100644
index 000000000000..f27103a1a6f4
--- /dev/null
+++ b/arch/kvx/kernel/time.c
@@ -0,0 +1,242 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ * Guillaume Thouvenin
+ * Luc Michel
+ * Julian Vetter
+ */
+
+#include <linux/of.h>
+#include <linux/clk.h>
+#include <linux/init.h>
+#include <linux/of_irq.h>
+#include <linux/interrupt.h>
+#include <linux/cpuhotplug.h>
+#include <linux/clockchips.h>
+#include <linux/clocksource.h>
+#include <linux/clk-provider.h>
+#include <linux/of_address.h>
+#include <linux/sched_clock.h>
+
+#include <asm/sfr_defs.h>
+
+#define KVX_TIMER_MIN_DELTA 1
+#define KVX_TIMER_MAX_DELTA 0xFFFFFFFFFFFFFFFFULL
+#define KVX_TIMER_MAX_VALUE 0xFFFFFFFFFFFFFFFFULL
+
+/*
+ * Clockevent
+ */
+static unsigned int kvx_timer_frequency;
+static unsigned int kvx_periodic_timer_value;
+static unsigned int kvx_timer_irq;
+
+static void kvx_timer_set_value(unsigned long value, unsigned long reload_value)
+{
+ kvx_sfr_set(T0R, reload_value);
+ kvx_sfr_set(T0V, value);
+ /* Enable timer */
+ kvx_sfr_set_field(TCR, T0CE, 1);
+}
+
+static int kvx_clkevent_set_next_event(unsigned long cycles,
+ struct clock_event_device *dev)
+{
+ /*
+ * Hardware does not support oneshot mode.
+ * In order to support it, set a really high reload value.
+ * Then, during the interrupt handler, disable the timer if
+ * in oneshot mode
+ */
+ kvx_timer_set_value(cycles - 1, KVX_TIMER_MAX_VALUE);
+
+ return 0;
+}
+
+/*
+ * Configure the rtc to periodically tick HZ times per second
+ */
+static int kvx_clkevent_set_state_periodic(struct clock_event_device *dev)
+{
+ kvx_timer_set_value(kvx_periodic_timer_value,
+ kvx_periodic_timer_value);
+
+ return 0;
+}
+
+static int kvx_clkevent_set_state_oneshot(struct clock_event_device *dev)
+{
+ /* Same as for kvx_clkevent_set_next_event */
+ kvx_clkevent_set_next_event(kvx_periodic_timer_value, dev);
+
+ return 0;
+}
+
+static int kvx_clkevent_set_state_shutdown(struct clock_event_device *dev)
+{
+ kvx_sfr_set_field(TCR, T0CE, 0);
+
+ return 0;
+}
+
+static DEFINE_PER_CPU(struct clock_event_device, kvx_clockevent_device) = {
+ .name = "kvx-timer-0",
+ .features = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_PERIODIC,
+ /* arbitrary rating for this clockevent */
+ .rating = 300,
+ .set_next_event = kvx_clkevent_set_next_event,
+ .set_state_periodic = kvx_clkevent_set_state_periodic,
+ .set_state_oneshot = kvx_clkevent_set_state_oneshot,
+ .set_state_shutdown = kvx_clkevent_set_state_shutdown,
+};
+
+irqreturn_t kvx_timer_irq_handler(int irq, void *dev_id)
+{
+ struct clock_event_device *evt = this_cpu_ptr(&kvx_clockevent_device);
+
+ /* Disable timer if in oneshot mode before reloading */
+ if (likely(clockevent_state_oneshot(evt)))
+ kvx_sfr_set_field(TCR, T0CE, 0);
+
+ evt->event_handler(evt);
+
+ return IRQ_HANDLED;
+}
+
+static int kvx_timer_starting_cpu(unsigned int cpu)
+{
+ struct clock_event_device *evt = this_cpu_ptr(&kvx_clockevent_device);
+
+ evt->cpumask = cpumask_of(cpu);
+ evt->irq = kvx_timer_irq;
+
+ clockevents_config_and_register(evt, kvx_timer_frequency,
+ KVX_TIMER_MIN_DELTA,
+ KVX_TIMER_MAX_DELTA);
+
+ /* Enable timer interrupt */
+ kvx_sfr_set_field(TCR, T0IE, 1);
+
+ enable_percpu_irq(kvx_timer_irq, IRQ_TYPE_NONE);
+
+ return 0;
+}
+
+static int kvx_timer_dying_cpu(unsigned int cpu)
+{
+ disable_percpu_irq(kvx_timer_irq);
+
+ return 0;
+}
+
+static int __init kvx_setup_core_timer(struct device_node *np)
+{
+ struct clock_event_device *evt = this_cpu_ptr(&kvx_clockevent_device);
+ struct clk *clk;
+ int err;
+
+ clk = of_clk_get(np, 0);
+ if (IS_ERR(clk)) {
+ pr_err("kvx_core_timer: Failed to get CPU clock: %ld\n",
+ PTR_ERR(clk));
+ return 1;
+ }
+
+ kvx_timer_frequency = clk_get_rate(clk);
+ clk_put(clk);
+ kvx_periodic_timer_value = kvx_timer_frequency / HZ;
+
+ kvx_timer_irq = irq_of_parse_and_map(np, 0);
+ if (!kvx_timer_irq) {
+ pr_err("kvx_core_timer: Failed to parse irq: %d\n",
+ kvx_timer_irq);
+ return -EINVAL;
+ }
+
+ err = request_percpu_irq(kvx_timer_irq, kvx_timer_irq_handler,
+ "kvx_core_timer", evt);
+ if (err) {
+ pr_err("kvx_core_timer: can't register interrupt %d (%d)\n",
+ kvx_timer_irq, err);
+ return err;
+ }
+
+ err = cpuhp_setup_state(CPUHP_AP_KVX_TIMER_STARTING,
+ "kvx/time:online",
+ kvx_timer_starting_cpu,
+ kvx_timer_dying_cpu);
+ if (err < 0) {
+ pr_err("kvx_core_timer: Failed to setup hotplug state");
+ return err;
+ }
+
+ return 0;
+}
+
+TIMER_OF_DECLARE(kvx_core_timer, "kalray,kvx-core-timer",
+ kvx_setup_core_timer);
+
+/*
+ * Clocksource
+ */
+static u64 kvx_dsu_clocksource_read(struct clocksource *cs)
+{
+ return readq(cs->archdata.regs);
+}
+
+static struct clocksource kvx_dsu_clocksource = {
+ .name = "kvx-dsu-clock",
+ .rating = 400,
+ .read = kvx_dsu_clocksource_read,
+ .mask = CLOCKSOURCE_MASK(64),
+ .flags = CLOCK_SOURCE_IS_CONTINUOUS,
+};
+
+static u64 notrace kvx_dsu_sched_read(void)
+{
+ return readq_relaxed(kvx_dsu_clocksource.archdata.regs);
+}
+
+static int __init kvx_setup_dsu_clock(struct device_node *np)
+{
+ int ret;
+ struct clk *clk;
+ unsigned long kvx_dsu_frequency;
+
+ kvx_dsu_clocksource.archdata.regs = of_iomap(np, 0);
+
+ WARN_ON(!kvx_dsu_clocksource.archdata.regs);
+ if (!kvx_dsu_clocksource.archdata.regs)
+ return -ENXIO;
+
+ clk = of_clk_get(np, 0);
+ if (IS_ERR(clk)) {
+ pr_err("Failed to get CPU clock: %ld\n", PTR_ERR(clk));
+ return PTR_ERR(clk);
+ }
+
+ kvx_dsu_frequency = clk_get_rate(clk);
+ clk_put(clk);
+
+ ret = clocksource_register_hz(&kvx_dsu_clocksource,
+ kvx_dsu_frequency);
+ if (ret) {
+ pr_err("failed to register dsu clocksource");
+ return ret;
+ }
+
+ sched_clock_register(kvx_dsu_sched_read, 64, kvx_dsu_frequency);
+ return 0;
+}
+
+TIMER_OF_DECLARE(kvx_dsu_clock, "kalray,kvx-dsu-clock",
+ kvx_setup_dsu_clock);
+
+void __init time_init(void)
+{
+ of_clk_init(NULL);
+
+ timer_probe();
+}
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 08/25] kvx: Add exception/interrupt handling
2023-01-03 16:43 ` Yann Sionneau
` (8 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-09 20:54 ` Thomas Gleixner
-1 siblings, 1 reply; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, Clement Leger, Guillaume Thouvenin,
Julian Vetter, Luc Michel, Marius Gligor
Add the exception and interrupt handling machanism for basic kvx
support.
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Luc Michel <lmichel@kalray.eu>
Signed-off-by: Luc Michel <lmichel@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/break_hook.h | 69 +++++++++
arch/kvx/include/asm/bug.h | 67 ++++++++
arch/kvx/include/asm/dame.h | 31 ++++
arch/kvx/include/asm/hardirq.h | 14 ++
arch/kvx/include/asm/hw_irq.h | 14 ++
arch/kvx/include/asm/ipi.h | 16 ++
arch/kvx/include/asm/irqflags.h | 58 +++++++
arch/kvx/include/asm/stacktrace.h | 44 ++++++
arch/kvx/include/asm/traps.h | 76 ++++++++++
arch/kvx/kernel/dame_handler.c | 113 ++++++++++++++
arch/kvx/kernel/irq.c | 78 ++++++++++
arch/kvx/kernel/traps.c | 243 ++++++++++++++++++++++++++++++
arch/kvx/platform/ipi.c | 110 ++++++++++++++
13 files changed, 933 insertions(+)
create mode 100644 arch/kvx/include/asm/break_hook.h
create mode 100644 arch/kvx/include/asm/bug.h
create mode 100644 arch/kvx/include/asm/dame.h
create mode 100644 arch/kvx/include/asm/hardirq.h
create mode 100644 arch/kvx/include/asm/hw_irq.h
create mode 100644 arch/kvx/include/asm/ipi.h
create mode 100644 arch/kvx/include/asm/irqflags.h
create mode 100644 arch/kvx/include/asm/stacktrace.h
create mode 100644 arch/kvx/include/asm/traps.h
create mode 100644 arch/kvx/kernel/dame_handler.c
create mode 100644 arch/kvx/kernel/irq.c
create mode 100644 arch/kvx/kernel/traps.c
create mode 100644 arch/kvx/platform/ipi.c
diff --git a/arch/kvx/include/asm/break_hook.h b/arch/kvx/include/asm/break_hook.h
new file mode 100644
index 000000000000..333b2c440c81
--- /dev/null
+++ b/arch/kvx/include/asm/break_hook.h
@@ -0,0 +1,69 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef __ASM_KVX_BREAK_HOOK_H_
+#define __ASM_KVX_BREAK_HOOK_H_
+
+#include <linux/types.h>
+
+#include <asm/sfr_defs.h>
+#include <asm/insns_defs.h>
+
+/*
+ * The following macros define the different causes of break:
+ * We use the `set $vsfr0 = $rXX` instruction which will raise a trap into the
+ * debugger. The trapping instruction is read and decoded to extract the source
+ * register number. The source register number is used to differentiate the
+ * trap cause.
+ */
+#define BREAK_CAUSE_BUG KVX_REG_R1
+#define BREAK_CAUSE_KGDB_DYN KVX_REG_R2
+#define BREAK_CAUSE_KGDB_COMP KVX_REG_R3
+#define BREAK_CAUSE_BKPT KVX_REG_R63
+
+/**
+ * enum break_ret - Break return value
+ * @BREAK_HOOK_HANDLED: Hook handled successfully
+ * @BREAK_HOOK_ERROR: Hook was not handled
+ */
+enum break_ret {
+ BREAK_HOOK_HANDLED = 0,
+ BREAK_HOOK_ERROR = 1,
+};
+
+/*
+ * The following macro assembles a `set` instruction targeting $vsfr0
+ * using the source register whose number is __id.
+ */
+#define KVX_BREAK_INSN(__id) \
+ KVX_INSN_SET_SYLLABLE_0(KVX_INSN_PARALLEL_EOB, KVX_SFR_VSFR0, __id)
+
+#define KVX_BREAK_INSN_SIZE (KVX_INSN_SET_SIZE * KVX_INSN_SYLLABLE_WIDTH)
+
+struct pt_regs;
+
+/**
+ * struct break_hook - Break hook description
+ * @node: List node
+ * @handler: handler called when break matches this hook
+ * @imm: Immediate value expected for break insn
+ * @mode: Hook mode (user/kernel)
+ */
+struct break_hook {
+ struct list_head node;
+ int (*handler)(struct break_hook *brk_hook, struct pt_regs *regs);
+ u8 id;
+ u8 mode;
+};
+
+void kvx_skip_break_insn(struct pt_regs *regs);
+
+void break_hook_register(struct break_hook *brk_hook);
+void break_hook_unregister(struct break_hook *brk_hook);
+
+int break_hook_handler(u64 es, struct pt_regs *regs);
+
+#endif /* __ASM_KVX_BREAK_HOOK_H_ */
diff --git a/arch/kvx/include/asm/bug.h b/arch/kvx/include/asm/bug.h
new file mode 100644
index 000000000000..62f556b00d5a
--- /dev/null
+++ b/arch/kvx/include/asm/bug.h
@@ -0,0 +1,67 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_BUG_H
+#define _ASM_KVX_BUG_H
+
+#include <linux/compiler.h>
+#include <linux/const.h>
+#include <linux/types.h>
+
+#include <asm/break_hook.h>
+
+#ifdef CONFIG_GENERIC_BUG
+
+#define BUG_INSN KVX_BREAK_INSN(BREAK_CAUSE_BUG)
+
+#define __BUG_ENTRY_ADDR ".dword 1b"
+
+#ifdef CONFIG_DEBUG_BUGVERBOSE
+#define __BUG_ENTRY_LAST_MEMBER flags
+#define __BUG_ENTRY \
+ __BUG_ENTRY_ADDR "\n\t" \
+ ".dword %0\n\t" \
+ ".short %1\n\t"
+#else
+#define __BUG_ENTRY_LAST_MEMBER file
+#define __BUG_ENTRY \
+ __BUG_ENTRY_ADDR "\n\t"
+#endif
+
+#define BUG() \
+do { \
+ __asm__ __volatile__ ( \
+ "1:\n\t" \
+ ".word " __stringify(BUG_INSN) "\n" \
+ ".pushsection __bug_table,\"a\"\n\t" \
+ "2:\n\t" \
+ __BUG_ENTRY \
+ ".fill 1, %2, 0\n\t" \
+ ".popsection" \
+ : \
+ : "i" (__FILE__), "i" (__LINE__), \
+ "i" (sizeof(struct bug_entry) - \
+ offsetof(struct bug_entry, __BUG_ENTRY_LAST_MEMBER))); \
+ unreachable(); \
+} while (0)
+
+#else /* CONFIG_GENERIC_BUG */
+#define BUG() \
+do { \
+ __asm__ __volatile__ (".word " __stringify(BUG_INSN) "\n"); \
+ unreachable(); \
+} while (0)
+#endif /* CONFIG_GENERIC_BUG */
+
+#define HAVE_ARCH_BUG
+
+struct pt_regs;
+
+void die(struct pt_regs *regs, unsigned long ea, const char *str);
+
+#include <asm-generic/bug.h>
+
+#endif /* _ASM_KVX_BUG_H */
diff --git a/arch/kvx/include/asm/dame.h b/arch/kvx/include/asm/dame.h
new file mode 100644
index 000000000000..8befd767bbee
--- /dev/null
+++ b/arch/kvx/include/asm/dame.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_DAME_H
+#define _ASM_KVX_DAME_H
+
+#include <asm/sfr.h>
+#include <asm/ptrace.h>
+
+static inline void dame_irq_check(struct pt_regs *regs)
+{
+#ifdef CONFIG_SECURE_DAME_HANDLING
+ unsigned long ilr;
+ /* If we are returning to the kernel, no need to check for DAME */
+ if (!user_mode(regs))
+ return;
+
+ /* Else, make sure we do a barrier to trig any pending DAME IRQ */
+ __builtin_kvx_barrier();
+
+ /* Check if we triggered a DAME */
+ ilr = kvx_sfr_get(ILR);
+ if (ilr & KVX_SFR_ILR_IT16_MASK)
+ panic("DAME error encountered while in kernel !!!!\n");
+#endif
+}
+
+#endif /* _ASM_KVX_DAME_H */
diff --git a/arch/kvx/include/asm/hardirq.h b/arch/kvx/include/asm/hardirq.h
new file mode 100644
index 000000000000..f82630f7e4e2
--- /dev/null
+++ b/arch/kvx/include/asm/hardirq.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_HARDIRQ_H
+#define _ASM_KVX_HARDIRQ_H
+
+#define __ARCH_IRQ_EXIT_IRQS_DISABLED 1
+
+#include <asm-generic/hardirq.h>
+
+#endif /* _ASM_KVX_HARDIRQ_H */
diff --git a/arch/kvx/include/asm/hw_irq.h b/arch/kvx/include/asm/hw_irq.h
new file mode 100644
index 000000000000..f073dba3b1c5
--- /dev/null
+++ b/arch/kvx/include/asm/hw_irq.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * derived from arch/mips/include/asm/ide.h
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_HW_IRQ_H
+#define _ASM_KVX_HW_IRQ_H
+
+void kvx_init_core_irq(void);
+
+#endif /* _ASM_KVX_HW_IRQ_H */
diff --git a/arch/kvx/include/asm/ipi.h b/arch/kvx/include/asm/ipi.h
new file mode 100644
index 000000000000..137407a075e6
--- /dev/null
+++ b/arch/kvx/include/asm/ipi.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_IPI_H
+#define _ASM_KVX_IPI_H
+
+#include <linux/irqreturn.h>
+
+int kvx_ipi_ctrl_probe(irqreturn_t (*ipi_irq_handler)(int, void *));
+
+void kvx_ipi_send(const struct cpumask *mask);
+
+#endif /* _ASM_KVX_IPI_H */
diff --git a/arch/kvx/include/asm/irqflags.h b/arch/kvx/include/asm/irqflags.h
new file mode 100644
index 000000000000..681c890b3fcd
--- /dev/null
+++ b/arch/kvx/include/asm/irqflags.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_IRQFLAGS_H
+#define _ASM_KVX_IRQFLAGS_H
+
+#include <linux/types.h>
+
+#include <asm/sfr.h>
+
+static inline notrace unsigned long arch_local_save_flags(void)
+{
+ return kvx_sfr_get(PS) & (1 << KVX_SFR_PS_IE_SHIFT);
+}
+
+static inline notrace unsigned long arch_local_irq_save(void)
+{
+ unsigned long flags = arch_local_save_flags();
+
+ kvx_sfr_set_field(PS, IE, 0);
+
+ return flags;
+}
+
+static inline notrace void arch_local_irq_restore(unsigned long flags)
+{
+ /* If flags are set, interrupt are enabled), set the IE bit */
+ if (flags)
+ kvx_sfr_set_field(PS, IE, 1);
+ else
+ kvx_sfr_set_field(PS, IE, 0);
+}
+
+static inline notrace void arch_local_irq_enable(void)
+{
+ kvx_sfr_set_field(PS, IE, 1);
+}
+
+static inline notrace void arch_local_irq_disable(void)
+{
+ kvx_sfr_set_field(PS, IE, 0);
+}
+
+static inline notrace bool arch_irqs_disabled_flags(unsigned long flags)
+{
+ return (flags & (1 << KVX_SFR_PS_IE_SHIFT)) == 0;
+}
+
+static inline notrace bool arch_irqs_disabled(void)
+{
+ return arch_irqs_disabled_flags(kvx_sfr_get(PS));
+}
+
+
+#endif /* _ASM_KVX_IRQFLAGS_H */
diff --git a/arch/kvx/include/asm/stacktrace.h b/arch/kvx/include/asm/stacktrace.h
new file mode 100644
index 000000000000..0feed6bd4424
--- /dev/null
+++ b/arch/kvx/include/asm/stacktrace.h
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_STACKTRACE_H
+#define _ASM_KVX_STACKTRACE_H
+
+#include <linux/sched/task_stack.h>
+
+/**
+ * Structure of a frame on the stack
+ */
+struct stackframe {
+ unsigned long fp; /* Next frame pointer */
+ unsigned long ra; /* Return address */
+};
+
+static inline bool on_task_stack(struct task_struct *tsk, unsigned long sp)
+{
+ unsigned long low = (unsigned long) task_stack_page(tsk);
+ unsigned long high = low + THREAD_SIZE;
+
+ if (sp < low || sp >= high)
+ return false;
+
+ return true;
+}
+
+void show_stacktrace(struct task_struct *task, struct pt_regs *regs);
+
+
+void walk_stackframe(struct task_struct *task, struct stackframe *frame,
+ bool (*fn)(unsigned long, void *), void *arg);
+
+static inline void start_stackframe(struct stackframe *frame,
+ unsigned long fp,
+ unsigned long pc)
+{
+ frame->fp = fp;
+ frame->ra = pc;
+}
+#endif /* _ASM_KVX_STACKTRACE_H */
diff --git a/arch/kvx/include/asm/traps.h b/arch/kvx/include/asm/traps.h
new file mode 100644
index 000000000000..77a663968135
--- /dev/null
+++ b/arch/kvx/include/asm/traps.h
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_TRAPS_H
+#define _ASM_KVX_TRAPS_H
+
+#include <asm/sfr.h>
+
+#define KVX_TRAP_RESET 0x0
+#define KVX_TRAP_OPCODE 0x1
+#define KVX_TRAP_PRIVILEGE 0x2
+#define KVX_TRAP_DMISALIGN 0x3
+#define KVX_TRAP_PSYSERROR 0x4
+#define KVX_TRAP_DSYSERROR 0x5
+#define KVX_TRAP_PDECCERROR 0x6
+#define KVX_TRAP_DDECCERROR 0x7
+#define KVX_TRAP_PPARERROR 0x8
+#define KVX_TRAP_DPARERROR 0x9
+#define KVX_TRAP_PSECERROR 0xA
+#define KVX_TRAP_DSECERROR 0xB
+#define KVX_TRAP_NOMAPPING 0xC
+#define KVX_TRAP_PROTECTION 0xD
+#define KVX_TRAP_WRITETOCLEAN 0xE
+#define KVX_TRAP_ATOMICTOCLEAN 0xF
+#define KVX_TRAP_TPAR 0x10
+#define KVX_TRAP_DOUBLE_ECC 0x11
+#define KVX_TRAP_VSFR 0x12
+#define KVX_TRAP_PL_OVERFLOW 0x13
+
+#define KVX_TRAP_COUNT 0x14
+
+#define KVX_TRAP_SFRI_NOT_BCU 0
+#define KVX_TRAP_SFRI_GET 1
+#define KVX_TRAP_SFRI_IGET 2
+#define KVX_TRAP_SFRI_SET 4
+#define KVX_TRAP_SFRI_WFXL 5
+#define KVX_TRAP_SFRI_WFXM 6
+#define KVX_TRAP_SFRI_RSWAP 7
+
+/* Access type on memory trap */
+#define KVX_TRAP_RWX_FETCH 1
+#define KVX_TRAP_RWX_WRITE 2
+#define KVX_TRAP_RWX_READ 4
+#define KVX_TRAP_RWX_ATOMIC 6
+
+#ifndef __ASSEMBLY__
+
+typedef void (*trap_handler_func) (uint64_t es, uint64_t ea,
+ struct pt_regs *regs);
+
+#define trap_cause(__es) kvx_sfr_field_val(__es, ES, HTC)
+
+#define trap_sfri(__es) \
+ kvx_sfr_field_val((__es), ES, SFRI)
+
+#define trap_gprp(__es) \
+ kvx_sfr_field_val((__es), ES, GPRP)
+
+#define trap_sfrp(__es) \
+ kvx_sfr_field_val((__es), ES, SFRP)
+
+#ifdef CONFIG_MMU
+extern void do_page_fault(uint64_t es, uint64_t ea, struct pt_regs *regs);
+extern void do_writetoclean(uint64_t es, uint64_t ea, struct pt_regs *regs);
+#endif
+
+void user_do_sig(struct pt_regs *regs, int signo, int code, unsigned long addr);
+
+#endif /* __ASSEMBLY__ */
+
+#endif
diff --git a/arch/kvx/kernel/dame_handler.c b/arch/kvx/kernel/dame_handler.c
new file mode 100644
index 000000000000..ce190bee8211
--- /dev/null
+++ b/arch/kvx/kernel/dame_handler.c
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/of.h>
+#include <linux/bug.h>
+#include <linux/init.h>
+#include <linux/of_irq.h>
+#include <linux/hardirq.h>
+#include <linux/irqchip.h>
+#include <linux/irqdomain.h>
+#include <linux/interrupt.h>
+#include <linux/cpuhotplug.h>
+#include <linux/sched/signal.h>
+
+static unsigned int kvx_dame_irq;
+
+static const char *error_str[KVX_SFR_ES_ITI_WIDTH] = {
+ "PSE",
+ "PILSY",
+ "PILDE",
+ "PILPA",
+ "DSE",
+ "DILSY",
+ "DILDE",
+ "DILPA",
+ "DDEE",
+ "DSYE"
+};
+
+static irqreturn_t dame_irq_handler(int irq, void *dev_id)
+{
+ int bit;
+ struct pt_regs *regs = get_irq_regs();
+ unsigned long error_status = kvx_sfr_field_val(regs->es, ES, ITI);
+
+ if (error_status) {
+ pr_err("Memory Error:\n");
+ for_each_set_bit(bit, &error_status, KVX_SFR_ES_ITI_WIDTH)
+ pr_err("- %s\n", error_str[bit]);
+ }
+
+ /*
+ * If the DAME happened in user mode, we can handle it properly
+ * by killing the user process.
+ * Otherwise, if we are in kernel, we are fried...
+ */
+ if (user_mode(regs))
+ force_sig_fault(SIGBUS, BUS_ADRERR, (void __user *) NULL);
+ else
+ die(regs, 0, "DAME error encountered while in kernel !!!!\n");
+
+ return IRQ_HANDLED;
+}
+
+static int kvx_dame_starting_cpu(unsigned int cpu)
+{
+ enable_percpu_irq(kvx_dame_irq, IRQ_TYPE_NONE);
+
+ return 0;
+}
+
+static int kvx_dame_dying_cpu(unsigned int cpu)
+{
+ disable_percpu_irq(kvx_dame_irq);
+
+ return 0;
+}
+
+static int __init dame_handler_init(void)
+{
+ struct device_node *dame_node;
+ int ret;
+
+ dame_node = of_find_compatible_node(NULL, NULL,
+ "kalray,kvx-dame-handler");
+ if (!dame_node) {
+ pr_err("Failed to find dame handler device tree node\n");
+ return -ENODEV;
+ }
+
+ kvx_dame_irq = irq_of_parse_and_map(dame_node, 0);
+ of_node_put(dame_node);
+
+ if (!kvx_dame_irq) {
+ pr_err("Failed to parse dame irq\n");
+ return -ENODEV;
+ }
+
+ ret = request_percpu_irq(kvx_dame_irq, dame_irq_handler, "dame",
+ &kvx_dame_irq);
+ if (ret) {
+ pr_err("Failed to request dame irq\n");
+ return -ENODEV;
+ }
+
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+ "kvx/dame_handler:online",
+ kvx_dame_starting_cpu,
+ kvx_dame_dying_cpu);
+ if (ret <= 0) {
+ pr_err("Failed to setup cpuhp\n");
+ return ret;
+ }
+
+ pr_info("DAME handler registered\n");
+
+ return 0;
+}
+
+core_initcall(dame_handler_init);
diff --git a/arch/kvx/kernel/irq.c b/arch/kvx/kernel/irq.c
new file mode 100644
index 000000000000..5d7c8cfba1dd
--- /dev/null
+++ b/arch/kvx/kernel/irq.c
@@ -0,0 +1,78 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/irqdomain.h>
+#include <linux/irqflags.h>
+#include <linux/hardirq.h>
+#include <linux/irqchip.h>
+#include <linux/bitops.h>
+#include <linux/init.h>
+
+#include <asm/dame.h>
+
+#define IT_MASK(__it) (KVX_SFR_ILL_ ## __it ## _MASK)
+#define IT_LEVEL(__it, __level) \
+ (__level##ULL << KVX_SFR_ILL_ ## __it ## _SHIFT)
+
+void do_IRQ(unsigned long hwirq_mask, struct pt_regs *regs)
+{
+ struct pt_regs *old_regs = set_irq_regs(regs);
+ int irq;
+ unsigned int hwirq;
+
+ trace_hardirqs_off();
+
+ irq_enter();
+
+ while (hwirq_mask) {
+ hwirq = __ffs(hwirq_mask);
+ irq = irq_find_mapping(NULL, hwirq);
+ generic_handle_irq(irq);
+ hwirq_mask &= ~BIT_ULL(hwirq);
+ }
+
+ irq_exit();
+ set_irq_regs(old_regs);
+
+ dame_irq_check(regs);
+}
+
+/*
+ * Early Hardware specific Interrupt setup
+ * -Called very early (start_kernel -> setup_arch -> setup_processor)
+ * -Needed for each CPU
+ */
+void kvx_init_core_irq(void)
+{
+ /*
+ * On KVX, Kernel only care about the following IT:
+ * - IT0: Timer 0
+ * - IT2: Watchdog
+ * - IT4: APIC IT 1
+ * - IT24: IPI
+ */
+ uint64_t mask = IT_MASK(IT0) | IT_MASK(IT2) | IT_MASK(IT4) |
+ IT_MASK(IT24);
+
+ /*
+ * Specific priorities for ITs:
+ * - Watchdog has the highest priority: 3
+ * - Timer has priority 2
+ * - APIC entries have lowest priority: 1
+ */
+ uint64_t value = IT_LEVEL(IT0, 0x2) | IT_LEVEL(IT2, 0x3) |
+ IT_LEVEL(IT4, 0x1) | IT_LEVEL(IT24, 0x1);
+
+ kvx_sfr_set_mask(ILL, mask, value);
+
+ /* Set core level to 0 */
+ kvx_sfr_set_field(PS, IL, 0);
+}
+
+void __init init_IRQ(void)
+{
+ irqchip_init();
+}
diff --git a/arch/kvx/kernel/traps.c b/arch/kvx/kernel/traps.c
new file mode 100644
index 000000000000..3a1706b666c2
--- /dev/null
+++ b/arch/kvx/kernel/traps.c
@@ -0,0 +1,243 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Marius Gligor
+ */
+
+#include <linux/context_tracking.h>
+#include <linux/sched/task_stack.h>
+#include <linux/sched/debug.h>
+#include <linux/irqflags.h>
+#include <linux/uaccess.h>
+#include <linux/kdebug.h>
+#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/init.h>
+#include <linux/ptrace.h>
+
+#include <asm/dame.h>
+#include <asm/traps.h>
+#include <asm/ptrace.h>
+#include <asm/break_hook.h>
+#include <asm/stacktrace.h>
+
+int show_unhandled_signals = 1;
+
+static DEFINE_SPINLOCK(die_lock);
+
+static trap_handler_func trap_handler_table[KVX_TRAP_COUNT] = { NULL };
+
+/* Trap names associated to the trap numbers */
+static const char * const trap_name[] = {
+ "RESET",
+ "OPCODE",
+ "PRIVILEGE",
+ "DMISALIGN",
+ "PSYSERROR",
+ "DSYSERROR",
+ "PDECCERROR",
+ "DDECCERROR",
+ "PPARERROR",
+ "DPARERROR",
+ "PSECERROR",
+ "DSECERROR",
+ /* MMU related traps */
+ "NOMAPPING",
+ "PROTECTION",
+ "WRITETOCLEAN",
+ "ATOMICTOCLEAN",
+ "TPAR",
+ "DOUBLE_ECC",
+ "VSFR",
+ "PL_OVERFLOW"
+};
+
+void die(struct pt_regs *regs, unsigned long ea, const char *str)
+{
+ static int die_counter;
+ int ret;
+
+ oops_enter();
+
+ spin_lock_irq(&die_lock);
+ console_verbose();
+ bust_spinlocks(1);
+
+ pr_emerg("%s [#%d]\n", str, ++die_counter);
+ print_modules();
+ show_regs(regs);
+
+ if (!user_mode(regs))
+ show_stacktrace(NULL, regs);
+
+ ret = notify_die(DIE_OOPS, str, regs, ea, 0, SIGSEGV);
+
+ bust_spinlocks(0);
+ add_taint(TAINT_DIE, LOCKDEP_NOW_UNRELIABLE);
+ spin_unlock_irq(&die_lock);
+ oops_exit();
+
+ if (in_interrupt())
+ panic("Fatal exception in interrupt");
+ if (panic_on_oops)
+ panic("Fatal exception");
+ if (ret != NOTIFY_STOP)
+ make_task_dead(SIGSEGV);
+}
+
+void user_do_sig(struct pt_regs *regs, int signo, int code, unsigned long addr)
+{
+ struct task_struct *tsk = current;
+
+ if (show_unhandled_signals && unhandled_signal(tsk, signo)
+ && printk_ratelimit()) {
+ pr_info("%s[%d]: unhandled signal %d code 0x%x at 0x%lx",
+ tsk->comm, task_pid_nr(tsk), signo, code, addr);
+ print_vma_addr(KERN_CONT " in ", instruction_pointer(regs));
+ pr_cont("\n");
+ show_regs(regs);
+ }
+ if (signo == SIGKILL) {
+ force_sig(signo);
+ return;
+ }
+ force_sig_fault(signo, code, (void __user *) addr);
+}
+
+static void panic_or_kill(uint64_t es, uint64_t ea, struct pt_regs *regs,
+ int signo, int sigcode)
+{
+ if (user_mode(regs)) {
+ user_do_sig(regs, signo, sigcode, ea);
+ return;
+ }
+
+ pr_alert(CUT_HERE "ERROR: TRAP %s received at 0x%.16llx\n",
+ trap_name[trap_cause(es)], regs->spc);
+ die(regs, ea, "Oops");
+ make_task_dead(SIGKILL);
+}
+
+int is_valid_bugaddr(unsigned long pc)
+{
+ /*
+ * Since the bug was reported, this means that the break hook handling
+ * already check the faulting instruction so there is no need for
+ * additionnal check here. This is a BUG for sure.
+ */
+ return 1;
+}
+
+static int bug_break_handler(struct break_hook *brk_hook, struct pt_regs *regs)
+{
+ enum bug_trap_type type;
+
+ type = report_bug(regs->spc, regs);
+ switch (type) {
+ case BUG_TRAP_TYPE_NONE:
+ return BREAK_HOOK_ERROR;
+ case BUG_TRAP_TYPE_WARN:
+ break;
+ case BUG_TRAP_TYPE_BUG:
+ die(regs, regs->spc, "Kernel BUG");
+ break;
+ }
+
+ /* Skip over break insn if we survived ! */
+ kvx_skip_break_insn(regs);
+
+ return BREAK_HOOK_HANDLED;
+}
+
+static struct break_hook bug_break_hook = {
+ .handler = bug_break_handler,
+ .id = BREAK_CAUSE_BUG,
+ .mode = MODE_KERNEL,
+};
+
+#define GEN_TRAP_HANDLER(__name, __sig, __code) \
+static void __name ## _trap_handler(uint64_t es, uint64_t ea, \
+ struct pt_regs *regs) \
+{ \
+ panic_or_kill(es, ea, regs, __sig, __code); \
+}
+
+GEN_TRAP_HANDLER(default, SIGKILL, SI_KERNEL);
+GEN_TRAP_HANDLER(privilege, SIGILL, ILL_PRVREG);
+GEN_TRAP_HANDLER(dmisalign, SIGBUS, BUS_ADRALN);
+GEN_TRAP_HANDLER(syserror, SIGBUS, BUS_ADRERR);
+GEN_TRAP_HANDLER(opcode, SIGILL, ILL_ILLOPC);
+
+static void register_trap_handler(unsigned int trap_nb, trap_handler_func fn)
+{
+
+ if (trap_nb >= KVX_TRAP_COUNT || fn == NULL)
+ panic("Failed to register handler #%d\n", trap_nb);
+
+ trap_handler_table[trap_nb] = fn;
+}
+
+static void do_vsfr_fault(uint64_t es, uint64_t ea, struct pt_regs *regs)
+{
+ if (break_hook_handler(es, regs) == BREAK_HOOK_HANDLED)
+ return;
+
+ panic_or_kill(es, ea, regs, SIGILL, ILL_PRVREG);
+}
+
+void __init trap_init(void)
+{
+ int i;
+
+ break_hook_register(&bug_break_hook);
+
+ for (i = 0; i < KVX_TRAP_COUNT; i++)
+ register_trap_handler(i, default_trap_handler);
+#ifdef CONFIG_MMU
+ register_trap_handler(KVX_TRAP_NOMAPPING, do_page_fault);
+ register_trap_handler(KVX_TRAP_PROTECTION, do_page_fault);
+ register_trap_handler(KVX_TRAP_WRITETOCLEAN, do_writetoclean);
+#endif
+
+ register_trap_handler(KVX_TRAP_PSYSERROR, syserror_trap_handler);
+ register_trap_handler(KVX_TRAP_DSYSERROR, syserror_trap_handler);
+ register_trap_handler(KVX_TRAP_PRIVILEGE, privilege_trap_handler);
+ register_trap_handler(KVX_TRAP_OPCODE, opcode_trap_handler);
+ register_trap_handler(KVX_TRAP_DMISALIGN, dmisalign_trap_handler);
+ register_trap_handler(KVX_TRAP_VSFR, do_vsfr_fault);
+}
+
+/**
+ * trap_handler - trap handler called by _trap_handler routine in trap_handler.S
+ * This handler will redirect to other trap handlers if present
+ * If not then it will do a generic action
+ * @es: Exception Syndrome register value
+ * @ea: Exception Address register
+ * @regs: pointer to registers saved when trapping
+ */
+void trap_handler(uint64_t es, uint64_t ea, struct pt_regs *regs)
+{
+ enum ctx_state prev_state = exception_enter();
+ int htc = trap_cause(es);
+ trap_handler_func trap_func = trap_handler_table[htc];
+
+ trace_hardirqs_off();
+
+ /* Normal traps number should and must be between 0 and 15 included */
+ if (unlikely(htc >= KVX_TRAP_COUNT)) {
+ pr_err("Invalid trap %d !\n", htc);
+ goto done;
+ }
+
+ /* If irqs were enabled in the preempted context, reenable them */
+ if (regs->sps & KVX_SFR_PS_IE_MASK)
+ local_irq_enable();
+
+ trap_func(es, ea, regs);
+
+done:
+ dame_irq_check(regs);
+ exception_exit(prev_state);
+}
diff --git a/arch/kvx/platform/ipi.c b/arch/kvx/platform/ipi.c
new file mode 100644
index 000000000000..73d1de27a628
--- /dev/null
+++ b/arch/kvx/platform/ipi.c
@@ -0,0 +1,110 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Luc Michel
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/io.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/module.h>
+#include <linux/of_irq.h>
+#include <linux/cpumask.h>
+#include <linux/interrupt.h>
+#include <linux/cpuhotplug.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+
+#define IPI_INTERRUPT_OFFSET 0x0
+#define IPI_MASK_OFFSET 0x20
+
+/*
+ * IPI controller can signal RM and PE0 -> 15
+ * In order to restrict that to the PE, write the corresponding mask
+ */
+#define KVX_IPI_CPU_MASK (~0xFFFF)
+
+struct kvx_ipi_ctrl {
+ void __iomem *regs;
+ unsigned int ipi_irq;
+};
+
+static struct kvx_ipi_ctrl kvx_ipi_controller;
+
+/**
+ * @kvx_pwr_ctrl_cpu_poweron Wakeup a cpu
+ *
+ * cpu: cpu to wakeup
+ */
+void kvx_ipi_send(const struct cpumask *mask)
+{
+ const unsigned long *maskb = cpumask_bits(mask);
+
+ WARN_ON(*maskb & KVX_IPI_CPU_MASK);
+ writel(*maskb, kvx_ipi_controller.regs + IPI_INTERRUPT_OFFSET);
+}
+
+static int kvx_ipi_starting_cpu(unsigned int cpu)
+{
+ enable_percpu_irq(kvx_ipi_controller.ipi_irq, IRQ_TYPE_NONE);
+
+ return 0;
+}
+
+static int kvx_ipi_dying_cpu(unsigned int cpu)
+{
+ disable_percpu_irq(kvx_ipi_controller.ipi_irq);
+
+ return 0;
+}
+
+int __init kvx_ipi_ctrl_probe(irqreturn_t (*ipi_irq_handler)(int, void *))
+{
+ struct device_node *np;
+ int ret;
+ unsigned int ipi_irq;
+ void __iomem *ipi_base;
+
+ np = of_find_compatible_node(NULL, NULL, "kalray,kvx-ipi-ctrl");
+ BUG_ON(!np);
+
+ ipi_base = of_iomap(np, 0);
+ BUG_ON(!ipi_base);
+
+ kvx_ipi_controller.regs = ipi_base;
+
+ /* Init mask for interrupts to PE0 -> PE15 */
+ writel(KVX_IPI_CPU_MASK, kvx_ipi_controller.regs + IPI_MASK_OFFSET);
+
+ ipi_irq = irq_of_parse_and_map(np, 0);
+ of_node_put(np);
+ if (!ipi_irq) {
+ pr_err("Failed to parse irq: %d\n", ipi_irq);
+ return -EINVAL;
+ }
+
+ ret = request_percpu_irq(ipi_irq, ipi_irq_handler,
+ "kvx_ipi", &kvx_ipi_controller);
+ if (ret) {
+ pr_err("can't register interrupt %d (%d)\n",
+ ipi_irq, ret);
+ return ret;
+ }
+ kvx_ipi_controller.ipi_irq = ipi_irq;
+
+ ret = cpuhp_setup_state(CPUHP_AP_IRQ_KVX_STARTING,
+ "kvx/ipi:online",
+ kvx_ipi_starting_cpu,
+ kvx_ipi_dying_cpu);
+ if (ret < 0) {
+ pr_err("Failed to setup hotplug state");
+ return ret;
+ }
+
+ pr_info("controller probed\n");
+
+ return 0;
+}
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 08/25] kvx: Add exception/interrupt handling
2023-01-03 16:43 ` [RFC PATCH 08/25] kvx: Add exception/interrupt handling Yann Sionneau
@ 2023-01-09 20:54 ` Thomas Gleixner
0 siblings, 0 replies; 113+ messages in thread
From: Thomas Gleixner @ 2023-01-09 20:54 UTC (permalink / raw)
To: Yann Sionneau
Cc: Yann Sionneau, linux-kernel, Clement Leger, Guillaume Thouvenin,
Julian Vetter, Luc Michel, Marius Gligor
On Tue, Jan 03 2023 at 17:43, Yann Sionneau wrote:
> --- /dev/null
> +++ b/arch/kvx/kernel/irq.c
> @@ -0,0 +1,78 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#include <linux/irqdomain.h>
> +#include <linux/irqflags.h>
> +#include <linux/hardirq.h>
> +#include <linux/irqchip.h>
> +#include <linux/bitops.h>
> +#include <linux/init.h>
> +
> +#include <asm/dame.h>
> +
> +#define IT_MASK(__it) (KVX_SFR_ILL_ ## __it ## _MASK)
> +#define IT_LEVEL(__it, __level) \
> + (__level##ULL << KVX_SFR_ILL_ ## __it ## _SHIFT)
> +
> +void do_IRQ(unsigned long hwirq_mask, struct pt_regs *regs)
> +{
> + struct pt_regs *old_regs = set_irq_regs(regs);
> + int irq;
> + unsigned int hwirq;
> +
> + trace_hardirqs_off();
> +
> + irq_enter();
Please use the generic entry code for interrupts to handle tracing,
irq_enter() etc. correctly.
> +/**
> + * trap_handler - trap handler called by _trap_handler routine in trap_handler.S
> + * This handler will redirect to other trap handlers if present
> + * If not then it will do a generic action
> + * @es: Exception Syndrome register value
> + * @ea: Exception Address register
> + * @regs: pointer to registers saved when trapping
> + */
> +void trap_handler(uint64_t es, uint64_t ea, struct pt_regs *regs)
> +{
> + enum ctx_state prev_state = exception_enter();
Please do not implement exception_enter/exit(). Use the generic entry
handling code so your code is correct and extensible.
Thanks,
tglx
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 09/25] kvx: irqchip: Add support for irq controllers
2023-01-03 16:43 ` Yann Sionneau
` (9 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-03 21:28 ` Rob Herring
-1 siblings, 1 reply; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Thomas Gleixner, Marc Zyngier, Rob Herring,
Krzysztof Kozlowski, linux-kernel, devicetree, Clement Leger,
Jules Maselbas, Julian Vetter, Luc Michel, Vincent Chardon
Add support for kvx irq controllers found in Coolidge MPPA SoC
The core-intc:
Each kvx core includes a hardware interrupt controller (core ITC)
with the following features:
* 32 independent interrupt sources
* 4-bit priotity level
* Individual interrupt enable bit
* Interrupt status bit displaying the pending interrupts
* Priority management between the 32 interrupts
Among those 32 interrupt sources, the first are hard-wired to hardware
sources. The remaining interrupt sources can be triggered via software
by directly writing to the ILR SFR.
The hard-wired interrupt sources are the following:
0: Timer 0
1: Timer 1
2: Watchdog
3: Performance Monitors
4: APIC GIC line 0
5: APIC GIC line 1
6: APIC GIC line 2
7: APIC GIC line 3
12: SECC error from memory system
13: Arithmetic exception (carry and IEEE 754 flags)
16: Data Asynchronous Memory Error (DAME), raised for DECC/DSYS errors
17: CLI (Cache Line Invalidation) for L1D or L1I following DECC/DSYS/Parity
errors
The APIC GIC lines will be used to route interrupts coming from SoC peripherals
from outside the Cluster to the kvx core. Those peripherals include USB host
controller, eMMC/SD host controller, i2c, spi, PCIe, IOMMUs etc...
The APIC GIC:
Each Cluster of the Coolidge SoC includes an APIC
(Advanced Programmable Interrupt Controller) GIC (Generic Interrupt Controller).
The APIC GIC acts as an intermediary interrupt controller, muxing/routing
incoming interrupts to output interrupts connected to the kvx core ITC lines.
The first 128 incoming interrupt lines come from the mailbox controller (itself
containing 128 mailboxes).
The remaining 11 interrupt lines come from external interrupt sources (NoC
router, the 5 IOMMUs, L2$ DMA job fifo, watchdog, SECC, DECC, D NoC).
The APIC GIC has 72 output interrupts: 4 per kvx cores in the cluster
(1 RM and 16 PE) connected to the "APIC GIC lines" described above and 1 for the
L2$ controller which makes 69 interrupts lines (rounded up to 72).
The APIC Mailbox:
The APIC includes a mailbox controller, containing 128 mailboxes.
This hardware block is basically a 1 Kb of smart memory space.
Each mailbox is an 8 bytes word memory location which can generate and
interrupt.
Each mailbox has a trigger function and an input function.
When a mailbox is written to, if the condition described by the
trigger function is satisfied, the corresponding interrupt
will fire.
Since this hardware block generates IRQs based on writes
at some memory locations, it is both an interrupt controller
and an MSI controller.
The ITGEN:
The ITGEN (InTerrupt GENerator) is an interrupt controller block.
It's purpose is to convert IRQ lines coming from SoC peripherals
(USB host controller for instance) into writes on the AXI bus.
Those writes are targeting the APIC Mailboxes.
CC: Thomas Gleixner <tglx@linutronix.de>
CC: Marc Zyngier <maz@kernel.org>
CC: Rob Herring <robh+dt@kernel.org>
CC: Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>
CC: linux-kernel@vger.kernel.org
CC: devicetree@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Luc Michel <lmichel@kalray.eu>
Signed-off-by: Luc Michel <lmichel@kalray.eu>
Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
.../kalray,kvx-core-intc.txt | 22 +
drivers/irqchip/Kconfig | 27 +
drivers/irqchip/Makefile | 4 +
drivers/irqchip/irq-kvx-apic-gic.c | 349 +++++++++++++
drivers/irqchip/irq-kvx-apic-mailbox.c | 465 ++++++++++++++++++
drivers/irqchip/irq-kvx-core-intc.c | 82 +++
drivers/irqchip/irq-kvx-itgen.c | 224 +++++++++
include/linux/irqchip/irq-kvx-apic-gic.h | 21 +
include/linux/irqchip/irq-kvx-apic-mailbox.h | 29 ++
include/linux/irqchip/irq-kvx-itgen.h | 24 +
10 files changed, 1247 insertions(+)
create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt
create mode 100644 drivers/irqchip/irq-kvx-apic-gic.c
create mode 100644 drivers/irqchip/irq-kvx-apic-mailbox.c
create mode 100644 drivers/irqchip/irq-kvx-core-intc.c
create mode 100644 drivers/irqchip/irq-kvx-itgen.c
create mode 100644 include/linux/irqchip/irq-kvx-apic-gic.h
create mode 100644 include/linux/irqchip/irq-kvx-apic-mailbox.h
create mode 100644 include/linux/irqchip/irq-kvx-itgen.h
diff --git a/Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt b/Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt
new file mode 100644
index 000000000000..503a661e1e84
--- /dev/null
+++ b/Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt
@@ -0,0 +1,22 @@
+* KVX Core Interrupt controller
+
+Required properties:
+
+- compatible: must to be "kalray,kvx-core-intc".
+- interrupt-controller
+- #interrupt-cells: has to be <1>: an interrupt index
+- regs: Base address of interrupt controller registers.
+
+Optional properties:
+
+- kalray,intc-nr-irqs: Number of irqs handled by the controller.
+ if not given, will default to 32.
+
+Example:
+
+ core_intc: core_intc@0 {
+ compatible = "kalray,kvx-core-intc";
+ #interrupt-cells = <1>;
+ interrupt-controller;
+ interrupt-parent = <&core_intc>;
+ };
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 7ef9f5e696d3..7c7753b33d4e 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -334,6 +334,33 @@ config MIPS_GIC
select IRQ_DOMAIN_HIERARCHY
select MIPS_CM
+config KVX_CORE_INTC
+ bool
+ depends on KVX
+ select IRQ_DOMAIN
+
+config KVX_APIC_GIC
+ bool
+ depends on KVX
+ select IRQ_DOMAIN
+ select IRQ_DOMAIN_HIERARCHY
+
+config KVX_APIC_MAILBOX
+ bool
+ depends on KVX
+ select GENERIC_IRQ_IPI if SMP
+ select GENERIC_MSI_IRQ_DOMAIN
+ select IRQ_DOMAIN
+ select IRQ_DOMAIN_HIERARCHY
+
+config KVX_ITGEN
+ bool
+ depends on KVX
+ select GENERIC_IRQ_IPI if SMP
+ select GENERIC_MSI_IRQ_DOMAIN
+ select IRQ_DOMAIN
+ select IRQ_DOMAIN_HIERARCHY
+
config INGENIC_IRQ
bool
depends on MACH_INGENIC
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 87b49a10962c..4e7e374314c3 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -69,6 +69,10 @@ obj-$(CONFIG_BCM7120_L2_IRQ) += irq-bcm7120-l2.o
obj-$(CONFIG_BRCMSTB_L2_IRQ) += irq-brcmstb-l2.o
obj-$(CONFIG_KEYSTONE_IRQ) += irq-keystone.o
obj-$(CONFIG_MIPS_GIC) += irq-mips-gic.o
+obj-$(CONFIG_KVX_CORE_INTC) += irq-kvx-core-intc.o
+obj-$(CONFIG_KVX_APIC_GIC) += irq-kvx-apic-gic.o
+obj-$(CONFIG_KVX_APIC_MAILBOX) += irq-kvx-apic-mailbox.o
+obj-$(CONFIG_KVX_ITGEN) += irq-kvx-itgen.o
obj-$(CONFIG_ARCH_MEDIATEK) += irq-mtk-sysirq.o irq-mtk-cirq.o
obj-$(CONFIG_ARCH_DIGICOLOR) += irq-digicolor.o
obj-$(CONFIG_ARCH_SA1100) += irq-sa11x0.o
diff --git a/drivers/irqchip/irq-kvx-apic-gic.c b/drivers/irqchip/irq-kvx-apic-gic.c
new file mode 100644
index 000000000000..42d28c8b3322
--- /dev/null
+++ b/drivers/irqchip/irq-kvx-apic-gic.c
@@ -0,0 +1,349 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2017 - 2022 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julian Vetter
+ */
+
+#define pr_fmt(fmt) "kvx_apic_gic: " fmt
+
+#include <linux/irqchip/irq-kvx-apic-gic.h>
+#include <linux/of_address.h>
+#include <linux/cpuhotplug.h>
+#include <linux/interrupt.h>
+#include <linux/irqdomain.h>
+#include <linux/spinlock.h>
+#include <linux/irqchip.h>
+#include <linux/of_irq.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/of.h>
+
+/* APIC is organized in 18 groups of 4 output lines
+ * However, the two upper lines are for Secure RM and DMA engine
+ * Thus, we do not have to use them
+ */
+#define GIC_CPU_OUT_COUNT 16
+#define GIC_PER_CPU_IT_COUNT 4
+
+/**
+ * For each CPU, there is 4 output lines coming from the apic GIC.
+ * We only use 1 line and this structure represent this line.
+ * @base Output line base address
+ * @cpu CPU associated to this line
+ */
+struct gic_out_irq_line {
+ void __iomem *base;
+ unsigned int cpu;
+};
+
+/**
+ * Input irq line.
+ * This structure is used to store the status of the input line and the
+ * associated output line.
+ * @enabled Boolean for line status
+ * @cpu CPU currently receiving this interrupt
+ * @it_num Interrupt number
+ */
+struct gic_in_irq_line {
+ bool enabled;
+ struct gic_out_irq_line *out_line;
+ unsigned int it_num;
+};
+
+/**
+ * struct kvx_apic_gic - kvx apic gic
+ * @base: Base address of the controller
+ * @domain Domain for this controller
+ * @input_nr_irqs: maximum number of supported input interrupts
+ * @cpus: Per cpu interrupt configuration
+ * @output_irq: Array of output irq lines
+ * @input_irq: Array of input irq lines
+ */
+struct kvx_apic_gic {
+ raw_spinlock_t lock;
+ void __iomem *base;
+ struct irq_domain *domain;
+ uint32_t input_nr_irqs;
+ /* For each cpu, there is a output IT line */
+ struct gic_out_irq_line output_irq[GIC_CPU_OUT_COUNT];
+
+ /* Input interrupt status */
+ struct gic_in_irq_line input_irq[KVX_GIC_INPUT_IT_COUNT];
+};
+
+static int gic_parent_irq;
+
+/**
+ * Enable/Disable an output irq line
+ * This function is used by both mask/unmask to disable/enable the line.
+ */
+static void irq_line_set_enable(struct gic_out_irq_line *irq_line,
+ struct gic_in_irq_line *in_irq_line,
+ int enable)
+{
+ void __iomem *enable_line_addr = irq_line->base +
+ KVX_GIC_ENABLE_OFFSET +
+ in_irq_line->it_num * KVX_GIC_ENABLE_ELEM_SIZE;
+
+ writeb((uint8_t) enable ? 1 : 0, enable_line_addr);
+ in_irq_line->enabled = enable;
+}
+
+static void kvx_apic_gic_set_line(struct irq_data *data, int enable)
+{
+ struct kvx_apic_gic *gic = irq_data_get_irq_chip_data(data);
+ unsigned int in_irq = irqd_to_hwirq(data);
+ struct gic_in_irq_line *in_line = &gic->input_irq[in_irq];
+ struct gic_out_irq_line *out_line = in_line->out_line;
+
+ raw_spin_lock(&gic->lock);
+ /* Set line enable on currently assigned cpu */
+ irq_line_set_enable(out_line, in_line, enable);
+ raw_spin_unlock(&gic->lock);
+}
+
+static void kvx_apic_gic_mask(struct irq_data *data)
+{
+ kvx_apic_gic_set_line(data, 0);
+}
+
+static void kvx_apic_gic_unmask(struct irq_data *data)
+{
+ kvx_apic_gic_set_line(data, 1);
+}
+
+#ifdef CONFIG_SMP
+
+static int kvx_apic_gic_set_affinity(struct irq_data *d,
+ const struct cpumask *cpumask,
+ bool force)
+{
+ struct kvx_apic_gic *gic = irq_data_get_irq_chip_data(d);
+ unsigned int new_cpu;
+ unsigned int hw_irq = irqd_to_hwirq(d);
+ struct gic_in_irq_line *input_line = &gic->input_irq[hw_irq];
+ struct gic_out_irq_line *new_out_line;
+
+ /* We assume there is only one cpu in the mask */
+ new_cpu = cpumask_first(cpumask);
+ new_out_line = &gic->output_irq[new_cpu];
+
+ raw_spin_lock(&gic->lock);
+
+ /* Nothing to do, line is the same */
+ if (new_out_line == input_line->out_line)
+ goto out;
+
+ /* If old line was enabled, enable the new one before disabling
+ * the old one
+ */
+ if (input_line->enabled)
+ irq_line_set_enable(new_out_line, input_line, 1);
+
+ /* Disable it on old line */
+ irq_line_set_enable(input_line->out_line, input_line, 0);
+
+ /* Assign new output line to input IRQ */
+ input_line->out_line = new_out_line;
+
+out:
+ raw_spin_unlock(&gic->lock);
+
+ irq_data_update_effective_affinity(d, cpumask_of(new_cpu));
+
+ return IRQ_SET_MASK_OK;
+}
+#endif
+
+static struct irq_chip kvx_apic_gic_chip = {
+ .name = "kvx apic gic",
+ .irq_mask = kvx_apic_gic_mask,
+ .irq_unmask = kvx_apic_gic_unmask,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = kvx_apic_gic_set_affinity,
+#endif
+};
+
+static int kvx_apic_gic_alloc(struct irq_domain *domain, unsigned int virq,
+ unsigned int nr_irqs, void *args)
+{
+ int i;
+ struct irq_fwspec *fwspec = args;
+ int hwirq = fwspec->param[0];
+
+ for (i = 0; i < nr_irqs; i++) {
+ irq_domain_set_info(domain, virq + i, hwirq + i,
+ &kvx_apic_gic_chip,
+ domain->host_data, handle_simple_irq,
+ NULL, NULL);
+ }
+
+ return 0;
+}
+
+static const struct irq_domain_ops kvx_apic_gic_domain_ops = {
+ .alloc = kvx_apic_gic_alloc,
+ .free = irq_domain_free_irqs_common,
+};
+
+static void irq_line_get_status_lac(struct gic_out_irq_line *out_irq_line,
+ uint64_t status[KVX_GIC_STATUS_LAC_ARRAY_SIZE])
+{
+ int i;
+
+ for (i = 0; i < KVX_GIC_STATUS_LAC_ARRAY_SIZE; i++) {
+ status[i] = readq(out_irq_line->base +
+ KVX_GIC_STATUS_LAC_OFFSET +
+ i * KVX_GIC_STATUS_LAC_ELEM_SIZE);
+ }
+}
+
+static void kvx_apic_gic_handle_irq(struct irq_desc *desc)
+{
+ struct kvx_apic_gic *gic_data = irq_desc_get_handler_data(desc);
+ struct gic_out_irq_line *out_line;
+ uint64_t status[KVX_GIC_STATUS_LAC_ARRAY_SIZE];
+ unsigned long irqn, cascade_irq;
+ unsigned long cpu = smp_processor_id();
+
+ out_line = &gic_data->output_irq[cpu];
+
+ irq_line_get_status_lac(out_line, status);
+
+ for_each_set_bit(irqn, (unsigned long *) status,
+ KVX_GIC_STATUS_LAC_ARRAY_SIZE * BITS_PER_LONG) {
+
+ cascade_irq = irq_find_mapping(gic_data->domain, irqn);
+
+ generic_handle_irq(cascade_irq);
+ }
+}
+
+static void __init apic_gic_init(struct kvx_apic_gic *gic)
+{
+ unsigned int cpu, line;
+ struct gic_in_irq_line *input_irq_line;
+ struct gic_out_irq_line *output_irq_line;
+ uint64_t status[KVX_GIC_STATUS_LAC_ARRAY_SIZE];
+
+ /* Initialize all input lines (device -> )*/
+ for (line = 0; line < KVX_GIC_INPUT_IT_COUNT; line++) {
+ input_irq_line = &gic->input_irq[line];
+ input_irq_line->enabled = false;
+ /* All input lines map on output 0 */
+ input_irq_line->out_line = &gic->output_irq[0];
+ input_irq_line->it_num = line;
+ }
+
+ /* Clear all output lines (-> cpus) */
+ for (cpu = 0; cpu < GIC_CPU_OUT_COUNT; cpu++) {
+ output_irq_line = &gic->output_irq[cpu];
+ output_irq_line->cpu = cpu;
+ output_irq_line->base = gic->base +
+ cpu * (KVX_GIC_ELEM_SIZE * GIC_PER_CPU_IT_COUNT);
+
+ /* Disable all external lines on this core */
+ for (line = 0; line < KVX_GIC_INPUT_IT_COUNT; line++)
+ irq_line_set_enable(output_irq_line,
+ &gic->input_irq[line], 0x0);
+
+ irq_line_get_status_lac(output_irq_line, status);
+ }
+}
+
+static int kvx_gic_starting_cpu(unsigned int cpu)
+{
+ enable_percpu_irq(gic_parent_irq, IRQ_TYPE_NONE);
+
+ return 0;
+}
+
+static int kvx_gic_dying_cpu(unsigned int cpu)
+{
+ disable_percpu_irq(gic_parent_irq);
+
+ return 0;
+}
+
+static int __init kvx_init_apic_gic(struct device_node *node,
+ struct device_node *parent)
+{
+ struct kvx_apic_gic *gic;
+ int ret;
+ unsigned int irq;
+
+ if (!parent) {
+ pr_err("kvx apic gic does not have parent\n");
+ return -EINVAL;
+ }
+
+ gic = kzalloc(sizeof(*gic), GFP_KERNEL);
+ if (!gic)
+ return -ENOMEM;
+
+ if (of_property_read_u32(node, "kalray,intc-nr-irqs",
+ &gic->input_nr_irqs))
+ gic->input_nr_irqs = KVX_GIC_INPUT_IT_COUNT;
+
+ if (WARN_ON(gic->input_nr_irqs > KVX_GIC_INPUT_IT_COUNT)) {
+ ret = -EINVAL;
+ goto err_kfree;
+ }
+
+ gic->base = of_io_request_and_map(node, 0, node->name);
+ if (!gic->base) {
+ ret = -EINVAL;
+ goto err_kfree;
+ }
+
+ raw_spin_lock_init(&gic->lock);
+ apic_gic_init(gic);
+
+ gic->domain = irq_domain_add_linear(node,
+ gic->input_nr_irqs,
+ &kvx_apic_gic_domain_ops,
+ gic);
+ if (!gic->domain) {
+ pr_err("Failed to add IRQ domain\n");
+ ret = -EINVAL;
+ goto err_iounmap;
+ }
+
+ irq = irq_of_parse_and_map(node, 0);
+ if (irq <= 0) {
+ pr_err("unable to parse irq\n");
+ ret = -EINVAL;
+ goto err_irq_domain_remove;
+ }
+
+ irq_set_chained_handler_and_data(irq, kvx_apic_gic_handle_irq,
+ gic);
+
+ gic_parent_irq = irq;
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+ "kvx/gic:online",
+ kvx_gic_starting_cpu,
+ kvx_gic_dying_cpu);
+ if (ret < 0) {
+ pr_err("Failed to setup hotplug state");
+ goto err_irq_unmap;
+ }
+
+ pr_info("Initialized interrupt controller with %d interrupts\n",
+ gic->input_nr_irqs);
+ return 0;
+
+err_irq_unmap:
+ irq_dispose_mapping(irq);
+err_irq_domain_remove:
+ irq_domain_remove(gic->domain);
+err_iounmap:
+ iounmap(gic->base);
+err_kfree:
+ kfree(gic);
+
+ return ret;
+}
+
+IRQCHIP_DECLARE(kvx_apic_gic, "kalray,kvx-apic-gic", kvx_init_apic_gic);
diff --git a/drivers/irqchip/irq-kvx-apic-mailbox.c b/drivers/irqchip/irq-kvx-apic-mailbox.c
new file mode 100644
index 000000000000..c279a29ee7d0
--- /dev/null
+++ b/drivers/irqchip/irq-kvx-apic-mailbox.c
@@ -0,0 +1,465 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2017 - 2022 Kalray Inc.
+ * Author(s): Clement Leger
+ * Jules Maselbas
+ */
+
+#define pr_fmt(fmt) "kvx_apic_mailbox: " fmt
+
+#include <linux/irqchip/irq-kvx-apic-mailbox.h>
+#include <linux/irqchip/chained_irq.h>
+#include <linux/of_address.h>
+#include <linux/interrupt.h>
+#include <linux/irqdomain.h>
+#include <linux/iommu.h>
+#include <linux/irqchip.h>
+#include <linux/module.h>
+#include <linux/of_irq.h>
+#include <linux/init.h>
+#include <linux/msi.h>
+#include <linux/of.h>
+
+#define MAILBOXES_MAX_COUNT 128
+
+/* Mailboxes are 64 bits wide */
+#define MAILBOXES_BIT_SIZE 64
+
+/* Maximum number of mailboxes available */
+#define MAILBOXES_MAX_BIT_COUNT (MAILBOXES_MAX_COUNT * MAILBOXES_BIT_SIZE)
+
+/* Mailboxes are grouped by 8 in a single page */
+#define MAILBOXES_BITS_PER_PAGE (8 * MAILBOXES_BIT_SIZE)
+
+/**
+ * struct mb_data - per mailbox data
+ * @cpu: CPU on which the mailbox is routed
+ * @parent_irq: Parent IRQ on the GIC
+ */
+struct mb_data {
+ unsigned int cpu;
+ unsigned int parent_irq;
+};
+
+/**
+ * struct kvx_apic_mailbox - kvx apic mailbox
+ * @base: base address of the controller
+ * @device_domain: IRQ device domain for mailboxes
+ * @msi_domain: platform MSI domain for MSI interface
+ * @domain_info: Domain information needed for the MSI domain
+ * @mb_count: Count of mailboxes we are handling
+ * @available: bitmap of availables bits in mailboxes
+ * @mailboxes_lock: lock for irq migration
+ * @mask_lock: lock for irq masking
+ * @mb_data: data associated to each mailbox
+ */
+struct kvx_apic_mailbox {
+ void __iomem *base;
+ phys_addr_t phys_base;
+ struct irq_domain *device_domain;
+ struct irq_domain *msi_domain;
+ struct msi_domain_info domain_info;
+ /* Start and count of device mailboxes */
+ unsigned int mb_count;
+ /* Bitmap of allocated bits in mailboxes */
+ DECLARE_BITMAP(available, MAILBOXES_MAX_BIT_COUNT);
+ spinlock_t mailboxes_lock;
+ raw_spinlock_t mask_lock;
+ struct mb_data mb_data[MAILBOXES_MAX_COUNT];
+};
+
+/**
+ * struct kvx_irq_data - per irq data
+ * @mb: Mailbox structure
+ */
+struct kvx_irq_data {
+ struct kvx_apic_mailbox *mb;
+};
+
+static void kvx_mailbox_get_from_hwirq(unsigned int hw_irq,
+ unsigned int *mailbox_num,
+ unsigned int *mailbox_bit)
+{
+ *mailbox_num = hw_irq / MAILBOXES_BIT_SIZE;
+ *mailbox_bit = hw_irq % MAILBOXES_BIT_SIZE;
+}
+
+static void __iomem *kvx_mailbox_get_addr(struct kvx_apic_mailbox *mb,
+ unsigned int num)
+{
+ return mb->base + (num * KVX_MAILBOX_ELEM_SIZE);
+}
+
+static phys_addr_t kvx_mailbox_get_phys_addr(struct kvx_apic_mailbox *mb,
+ unsigned int num)
+{
+ return mb->phys_base + (num * KVX_MAILBOX_ELEM_SIZE);
+}
+
+static void kvx_mailbox_msi_compose_msg(struct irq_data *data,
+ struct msi_msg *msg)
+{
+ struct kvx_irq_data *kd = irq_data_get_irq_chip_data(data);
+ struct kvx_apic_mailbox *mb = kd->mb;
+ unsigned int mb_num, mb_bit;
+ phys_addr_t mb_addr;
+
+ kvx_mailbox_get_from_hwirq(irqd_to_hwirq(data), &mb_num, &mb_bit);
+ mb_addr = kvx_mailbox_get_phys_addr(mb, mb_num);
+
+ msg->address_hi = upper_32_bits(mb_addr);
+ msg->address_lo = lower_32_bits(mb_addr);
+ msg->data = mb_bit;
+
+ iommu_dma_compose_msi_msg(irq_data_get_msi_desc(data), msg);
+}
+
+static void kvx_mailbox_set_irq_enable(struct irq_data *data,
+ bool enabled)
+{
+ struct kvx_irq_data *kd = irq_data_get_irq_chip_data(data);
+ struct kvx_apic_mailbox *mb = kd->mb;
+ unsigned int mb_num, mb_bit;
+ void __iomem *mb_addr;
+ u64 mask_value, mb_value;
+
+ kvx_mailbox_get_from_hwirq(irqd_to_hwirq(data), &mb_num, &mb_bit);
+ mb_addr = kvx_mailbox_get_addr(mb, mb_num);
+
+ raw_spin_lock(&mb->mask_lock);
+ mask_value = readq(mb_addr + KVX_MAILBOX_MASK_OFFSET);
+ if (enabled)
+ mask_value |= BIT_ULL(mb_bit);
+ else
+ mask_value &= ~BIT_ULL(mb_bit);
+
+ writeq(mask_value, mb_addr + KVX_MAILBOX_MASK_OFFSET);
+
+ raw_spin_unlock(&mb->mask_lock);
+
+ /**
+ * Since interrupts on mailboxes are edge triggered and are only
+ * triggered when writing the value, we need to trigger it manually
+ * after updating the mask if enabled. If the interrupt was triggered by
+ * the device just after the mask write, we can trigger a spurious
+ * interrupt but that is still better than missing one...
+ * Moreover, the mailbox is configured in OR mode which means that even
+ * if we write a single bit, all other bits will be kept intact.
+ */
+ if (enabled) {
+ mb_value = readq(mb_addr + KVX_MAILBOX_VALUE_OFFSET);
+ if (mb_value & BIT_ULL(mb_bit))
+ writeq(BIT_ULL(mb_bit),
+ mb_addr + KVX_MAILBOX_VALUE_OFFSET);
+ }
+}
+
+static void kvx_mailbox_mask(struct irq_data *data)
+{
+ kvx_mailbox_set_irq_enable(data, false);
+}
+
+static void kvx_mailbox_unmask(struct irq_data *data)
+{
+ kvx_mailbox_set_irq_enable(data, true);
+}
+
+static void kvx_mailbox_set_cpu(struct kvx_apic_mailbox *mb, int mb_id,
+ int new_cpu)
+{
+ irq_set_affinity(mb->mb_data[mb_id].parent_irq, cpumask_of(new_cpu));
+ mb->mb_data[mb_id].cpu = new_cpu;
+}
+
+static void kvx_mailbox_free_bit(struct kvx_apic_mailbox *mb, int hw_irq)
+{
+ unsigned int mb_num, mb_bit;
+
+ kvx_mailbox_get_from_hwirq(hw_irq, &mb_num, &mb_bit);
+ bitmap_clear(mb->available, hw_irq, 1);
+
+ /* If there is no more IRQ on this mailbox, reset it to CPU 0 */
+ if (mb->available[mb_num] == 0)
+ kvx_mailbox_set_cpu(mb, mb_num, 0);
+}
+
+struct irq_chip kvx_apic_mailbox_irq_chip = {
+ .name = "kvx apic mailbox",
+ .irq_compose_msi_msg = kvx_mailbox_msi_compose_msg,
+ .irq_mask = kvx_mailbox_mask,
+ .irq_unmask = kvx_mailbox_unmask,
+};
+
+static int kvx_mailbox_allocate_bits(struct kvx_apic_mailbox *mb, int num_req)
+{
+ int first, align_mask = 0;
+
+ /* This must be a power of 2 for bitmap_find_next_zero_area to work */
+ BUILD_BUG_ON((MAILBOXES_BITS_PER_PAGE & (MAILBOXES_BITS_PER_PAGE - 1)));
+
+ /*
+ * If user requested more than 1 mailbox, we must make sure it will be
+ * aligned on a page size for iommu_dma_prepare_msi to be correctly
+ * mapped in a single page.
+ */
+ if (num_req > 1)
+ align_mask = (MAILBOXES_BITS_PER_PAGE - 1);
+
+ spin_lock(&mb->mailboxes_lock);
+
+ first = bitmap_find_next_zero_area(mb->available,
+ mb->mb_count * MAILBOXES_BIT_SIZE, 0,
+ num_req, align_mask);
+ if (first >= MAILBOXES_MAX_BIT_COUNT) {
+ spin_unlock(&mb->mailboxes_lock);
+ return -ENOSPC;
+ }
+
+ bitmap_set(mb->available, first, num_req);
+
+ spin_unlock(&mb->mailboxes_lock);
+
+ return first;
+}
+
+static int kvx_apic_mailbox_msi_alloc(struct irq_domain *domain,
+ unsigned int virq,
+ unsigned int nr_irqs, void *args)
+{
+ int i, err;
+ int hwirq = 0;
+ u64 mb_addr;
+ struct irq_data *d;
+ struct kvx_irq_data *kd;
+ struct kvx_apic_mailbox *mb = domain->host_data;
+ struct msi_alloc_info *msi_info = (struct msi_alloc_info *)args;
+ struct msi_desc *desc = msi_info->desc;
+ unsigned int mb_num, mb_bit;
+
+ /* We will not be able to guarantee page alignment ! */
+ if (nr_irqs > MAILBOXES_BITS_PER_PAGE)
+ return -EINVAL;
+
+ hwirq = kvx_mailbox_allocate_bits(mb, nr_irqs);
+ if (hwirq < 0)
+ return hwirq;
+
+ kvx_mailbox_get_from_hwirq(hwirq, &mb_num, &mb_bit);
+ mb_addr = (u64) kvx_mailbox_get_phys_addr(mb, mb_num);
+ err = iommu_dma_prepare_msi(desc, mb_addr);
+ if (err)
+ goto free_mb_bits;
+
+ for (i = 0; i < nr_irqs; i++) {
+ kd = kmalloc(sizeof(*kd), GFP_KERNEL);
+ if (!kd) {
+ err = -ENOMEM;
+ goto free_irq_data;
+ }
+
+ kd->mb = mb;
+ irq_domain_set_info(domain, virq + i, hwirq + i,
+ &kvx_apic_mailbox_irq_chip,
+ kd, handle_simple_irq,
+ NULL, NULL);
+ }
+
+ return 0;
+
+free_irq_data:
+ for (i--; i >= 0; i--) {
+ d = irq_domain_get_irq_data(domain, virq + i);
+ kd = irq_data_get_irq_chip_data(d);
+ kfree(kd);
+ }
+
+free_mb_bits:
+ spin_lock(&mb->mailboxes_lock);
+ bitmap_clear(mb->available, hwirq, nr_irqs);
+ spin_unlock(&mb->mailboxes_lock);
+
+ return err;
+}
+
+static void kvx_apic_mailbox_msi_free(struct irq_domain *domain,
+ unsigned int virq,
+ unsigned int nr_irqs)
+{
+ int i;
+ struct irq_data *d;
+ struct kvx_irq_data *kd;
+ struct kvx_apic_mailbox *mb = domain->host_data;
+
+ spin_lock(&mb->mailboxes_lock);
+
+ for (i = 0; i < nr_irqs; i++) {
+ d = irq_domain_get_irq_data(domain, virq + i);
+ kd = irq_data_get_irq_chip_data(d);
+ kfree(kd);
+ kvx_mailbox_free_bit(mb, d->hwirq);
+ }
+
+ spin_unlock(&mb->mailboxes_lock);
+}
+
+static const struct irq_domain_ops kvx_apic_mailbox_domain_ops = {
+ .alloc = kvx_apic_mailbox_msi_alloc,
+ .free = kvx_apic_mailbox_msi_free
+};
+
+static struct irq_chip kvx_msi_irq_chip = {
+ .name = "KVX MSI",
+};
+
+static void kvx_apic_mailbox_handle_irq(struct irq_desc *desc)
+{
+ struct irq_data *data = irq_desc_get_irq_data(desc);
+ struct kvx_apic_mailbox *mb = irq_desc_get_handler_data(desc);
+ void __iomem *mb_addr = kvx_mailbox_get_addr(mb, irqd_to_hwirq(data));
+ unsigned int irqn, cascade_irq, bit;
+ u64 mask_value, masked_its;
+ u64 mb_value;
+ /* Since we allocate 64 interrupts for each mailbox, the scheme
+ * to find the hwirq associated to a mailbox irq is the
+ * following:
+ * hw_irq = mb_num * MAILBOXES_BIT_SIZE + bit
+ */
+ unsigned int mb_hwirq = irqd_to_hwirq(data) * MAILBOXES_BIT_SIZE;
+
+ mb_value = readq(mb_addr + KVX_MAILBOX_LAC_OFFSET);
+ mask_value = readq(mb_addr + KVX_MAILBOX_MASK_OFFSET);
+ /* Mask any disabled interrupts */
+ mb_value &= mask_value;
+
+ /**
+ * Write all pending ITs that are masked to process them later
+ * Since the mailbox is in OR mode, these bits will be merged with any
+ * already set bits and thus avoid losing any interrupts.
+ */
+ masked_its = (~mask_value) & mb_value;
+ if (masked_its)
+ writeq(masked_its, mb_addr + KVX_MAILBOX_LAC_OFFSET);
+
+ for_each_set_bit(bit, (unsigned long *) &mb_value, BITS_PER_LONG) {
+ irqn = bit + mb_hwirq;
+ cascade_irq = irq_find_mapping(mb->device_domain, irqn);
+ generic_handle_irq(cascade_irq);
+ }
+}
+
+static void __init
+apic_mailbox_reset(struct kvx_apic_mailbox *mb)
+{
+ unsigned int i;
+ unsigned int mb_end = mb->mb_count;
+ void __iomem *mb_addr;
+ u64 funct_val = (KVX_MAILBOX_MODE_OR << KVX_MAILBOX_FUNCT_MODE_SHIFT) |
+ (KVX_MAILBOX_TRIG_DOORBELL << KVX_MAILBOX_FUNCT_TRIG_SHIFT);
+
+ for (i = 0; i < mb_end; i++) {
+ mb_addr = kvx_mailbox_get_addr(mb, i);
+ /* Disable all interrupts */
+ writeq(0ULL, mb_addr + KVX_MAILBOX_MASK_OFFSET);
+ /* Set mailbox to OR mode + trigger */
+ writeq(funct_val, mb_addr + KVX_MAILBOX_FUNCT_OFFSET);
+ /* Load & Clear mailbox value */
+ readq(mb_addr + KVX_MAILBOX_LAC_OFFSET);
+ }
+}
+
+static struct msi_domain_ops kvx_msi_domain_ops = {
+};
+
+static struct msi_domain_info kvx_msi_domain_info = {
+ .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS),
+ .ops = &kvx_msi_domain_ops,
+ .chip = &kvx_msi_irq_chip,
+};
+
+static int __init
+kvx_init_apic_mailbox(struct device_node *node,
+ struct device_node *parent)
+{
+ struct kvx_apic_mailbox *mb;
+ unsigned int parent_irq, irq_count;
+ struct resource res;
+ int ret, i;
+
+ mb = kzalloc(sizeof(*mb), GFP_KERNEL);
+ if (!mb)
+ return -ENOMEM;
+
+ ret = of_address_to_resource(node, 0, &res);
+ if (ret)
+ return -EINVAL;
+
+ mb->phys_base = res.start;
+ mb->base = of_io_request_and_map(node, 0, node->name);
+ if (!mb->base) {
+ ret = -EINVAL;
+ goto err_kfree;
+ }
+
+ spin_lock_init(&mb->mailboxes_lock);
+ raw_spin_lock_init(&mb->mask_lock);
+
+ irq_count = of_irq_count(node);
+ if (irq_count == 0 || irq_count > MAILBOXES_MAX_COUNT) {
+ ret = -EINVAL;
+ goto err_kfree;
+ }
+ mb->mb_count = irq_count;
+
+ apic_mailbox_reset(mb);
+
+ mb->device_domain = irq_domain_add_tree(node,
+ &kvx_apic_mailbox_domain_ops,
+ mb);
+ if (!mb->device_domain) {
+ pr_err("Failed to setup device domain\n");
+ ret = -EINVAL;
+ goto err_iounmap;
+ }
+
+ mb->msi_domain = platform_msi_create_irq_domain(of_node_to_fwnode(node),
+ &kvx_msi_domain_info,
+ mb->device_domain);
+ if (!mb->msi_domain) {
+ ret = -EINVAL;
+ goto err_irq_domain_add_tree;
+ }
+
+ /* Chain all interrupts from gic to mailbox */
+ for (i = 0; i < irq_count; i++) {
+ parent_irq = irq_of_parse_and_map(node, i);
+ if (parent_irq == 0) {
+ pr_err("unable to parse irq\n");
+ ret = -EINVAL;
+ goto err_irq_domain_msi_create;
+ }
+ mb->mb_data[i].parent_irq = parent_irq;
+
+ irq_set_chained_handler_and_data(parent_irq,
+ kvx_apic_mailbox_handle_irq,
+ mb);
+ }
+
+ pr_info("Init with %d device interrupt\n",
+ mb->mb_count * MAILBOXES_BIT_SIZE);
+
+ return 0;
+
+err_irq_domain_msi_create:
+ irq_domain_remove(mb->msi_domain);
+err_irq_domain_add_tree:
+ irq_domain_remove(mb->device_domain);
+err_iounmap:
+ iounmap(mb->base);
+err_kfree:
+ kfree(mb);
+
+ return ret;
+}
+
+IRQCHIP_DECLARE(kvx_apic_mailbox, "kalray,kvx-apic-mailbox",
+ kvx_init_apic_mailbox);
diff --git a/drivers/irqchip/irq-kvx-core-intc.c b/drivers/irqchip/irq-kvx-core-intc.c
new file mode 100644
index 000000000000..4315108d563c
--- /dev/null
+++ b/drivers/irqchip/irq-kvx-core-intc.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2017 - 2022 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#define pr_fmt(fmt) "kvx_core_intc: " fmt
+
+#include <linux/interrupt.h>
+#include <linux/irqdomain.h>
+#include <linux/irqchip.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/of.h>
+#include <asm/irq.h>
+
+#define KVX_CORE_INTC_IRQ 32
+
+
+static void kvx_irq_mask(struct irq_data *data)
+{
+ kvx_sfr_clear_bit(ILE, data->hwirq);
+}
+
+static void kvx_irq_unmask(struct irq_data *data)
+{
+ kvx_sfr_set_bit(ILE, data->hwirq);
+}
+
+static struct irq_chip kvx_irq_chip = {
+ .name = "kvx core Intc",
+ .irq_mask = kvx_irq_mask,
+ .irq_unmask = kvx_irq_unmask,
+};
+
+static int kvx_irq_map(struct irq_domain *d, unsigned int irq,
+ irq_hw_number_t hw)
+{
+ /* All interrupts for core are per cpu */
+ irq_set_percpu_devid(irq);
+ irq_set_chip_and_handler(irq, &kvx_irq_chip, handle_percpu_irq);
+
+ return 0;
+}
+
+static const struct irq_domain_ops kvx_irq_ops = {
+ .xlate = irq_domain_xlate_onecell,
+ .map = kvx_irq_map,
+};
+
+static int __init
+kvx_init_core_intc(struct device_node *intc, struct device_node *parent)
+{
+ struct irq_domain *root_domain;
+ uint32_t core_nr_irqs;
+
+ if (parent)
+ panic("DeviceTree core intc not a root irq controller\n");
+
+ if (of_property_read_u32(intc, "kalray,intc-nr-irqs", &core_nr_irqs))
+ core_nr_irqs = KVX_CORE_INTC_IRQ;
+
+ /* We only have up to 32 interrupts, according to IRQ-domain.txt,
+ * linear is likely to be the best choice
+ */
+ root_domain = irq_domain_add_linear(intc, core_nr_irqs,
+ &kvx_irq_ops, NULL);
+ if (!root_domain)
+ panic("root irq domain not avail\n");
+
+ /*
+ * Needed for primary domain lookup to succeed
+ * This is a primary irqchip, and can never have a parent
+ */
+ irq_set_default_host(root_domain);
+
+ pr_info("Initialized with %d interrupts\n", core_nr_irqs);
+
+ return 0;
+}
+
+IRQCHIP_DECLARE(kvx_core_intc, "kalray,kvx-core-intc", kvx_init_core_intc);
diff --git a/drivers/irqchip/irq-kvx-itgen.c b/drivers/irqchip/irq-kvx-itgen.c
new file mode 100644
index 000000000000..1d35c2c772a0
--- /dev/null
+++ b/drivers/irqchip/irq-kvx-itgen.c
@@ -0,0 +1,224 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2017 - 2022 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julian Vetter
+ * Vincent Chardon
+ */
+
+#include <linux/irqchip/irq-kvx-itgen.h>
+#include <linux/platform_device.h>
+#include <linux/of_platform.h>
+#include <linux/of_address.h>
+#include <linux/interrupt.h>
+#include <linux/irqdomain.h>
+#include <linux/irqchip.h>
+#include <linux/module.h>
+#include <linux/msi.h>
+#include <linux/of.h>
+#include <linux/of_irq.h>
+
+#define MB_ADDR_CLUSTER_SHIFT 24
+#define MB_ADDR_MAILBOX_SHIFT 9
+
+/**
+ * struct kvx_itgen - kvx interrupt generator (MSI client)
+ * @base: base address of the itgen controller
+ * @domain: IRQ domain of the controller
+ * @pdev: Platform device associated to the controller
+ */
+struct kvx_itgen {
+ void __iomem *base;
+ struct irq_domain *domain;
+ struct platform_device *pdev;
+};
+
+static void __iomem *get_itgen_cfg_offset(struct kvx_itgen *itgen,
+ irq_hw_number_t hwirq)
+{
+ return itgen->base + KVX_ITGEN_CFG_TARGET_OFFSET +
+ hwirq * KVX_ITGEN_CFG_ELEM_SIZE;
+}
+
+void __iomem *get_itgen_param_offset(struct kvx_itgen *itgen)
+{
+ return itgen->base + KVX_ITGEN_PARAM_OFFSET;
+}
+
+static void kvx_itgen_enable(struct irq_data *data, u32 value)
+{
+ struct kvx_itgen *itgen = irq_data_get_irq_chip_data(data);
+ void __iomem *enable_reg =
+ get_itgen_cfg_offset(itgen, irqd_to_hwirq(data)) +
+ KVX_ITGEN_CFG_ENABLE_OFFSET;
+
+ dev_dbg(&itgen->pdev->dev, "%sabling hwirq %d, addr %p\n",
+ value ? "En" : "Dis",
+ (int) irqd_to_hwirq(data),
+ enable_reg);
+ writel(value, enable_reg);
+}
+
+static void kvx_itgen_mask(struct irq_data *data)
+{
+ kvx_itgen_enable(data, 0x0);
+ irq_chip_mask_parent(data);
+}
+
+static void kvx_itgen_unmask(struct irq_data *data)
+{
+ kvx_itgen_enable(data, 0x1);
+ irq_chip_unmask_parent(data);
+}
+
+#ifdef CONFIG_SMP
+static int kvx_itgen_irq_set_affinity(struct irq_data *data,
+ const struct cpumask *dest, bool force)
+{
+ return -ENOSYS;
+}
+#endif
+
+static struct irq_chip itgen_irq_chip = {
+ .name = "kvx-itgen",
+ .irq_mask = kvx_itgen_mask,
+ .irq_unmask = kvx_itgen_unmask,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = kvx_itgen_irq_set_affinity,
+#endif
+};
+
+#define ITGEN_UNSUPPORTED_TYPES (IRQ_TYPE_LEVEL_LOW | IRQ_TYPE_EDGE_FALLING)
+
+static int kvx_itgen_domain_alloc(struct irq_domain *domain, unsigned int virq,
+ unsigned int nr_irqs, void *args)
+{
+ int i, err;
+ struct irq_fwspec *fwspec = args;
+ int hwirq = fwspec->param[0];
+ int type = IRQ_TYPE_NONE;
+ struct kvx_itgen *itgen;
+
+ if (fwspec->param_count >= 2)
+ type = fwspec->param[1];
+
+ WARN_ON(type & ITGEN_UNSUPPORTED_TYPES);
+
+ err = platform_msi_device_domain_alloc(domain, virq, nr_irqs);
+ if (err)
+ return err;
+
+ itgen = platform_msi_get_host_data(domain);
+
+ for (i = 0; i < nr_irqs; i++) {
+ irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
+ &itgen_irq_chip, itgen);
+ if (type == IRQ_TYPE_LEVEL_HIGH)
+ irq_set_handler(virq + i, handle_level_irq);
+ }
+
+ return 0;
+}
+
+static const struct irq_domain_ops itgen_domain_ops = {
+ .alloc = kvx_itgen_domain_alloc,
+ .free = irq_domain_free_irqs_common,
+};
+
+static void kvx_itgen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
+{
+ struct irq_data *d = irq_get_irq_data(desc->irq);
+ struct kvx_itgen *itgen = irq_data_get_irq_chip_data(d);
+ uint32_t cfg_val = 0;
+ uintptr_t dest_addr = ((uint64_t) msg->address_hi << 32) |
+ msg->address_lo;
+ void __iomem *cfg = get_itgen_cfg_offset(itgen, irqd_to_hwirq(d));
+
+ /**
+ * Address in the msi data is the address of the targeted mailbox.
+ * To save a few cells of hw, itgen configuration expects the
+ * target of the write using mppa id, cluster id and mailbox id instead
+ * of address.
+ * We extract these information from mailbox address.
+ */
+
+ cfg_val |= (((kvx_sfr_get(PCR) & KVX_SFR_PCR_CID_MASK) >>
+ KVX_SFR_PCR_CID_SHIFT)
+ << KVX_ITGEN_CFG_TARGET_CLUSTER_SHIFT);
+ cfg_val |= ((dest_addr >> MB_ADDR_MAILBOX_SHIFT) &
+ KVX_ITGEN_CFG_TARGET_MAILBOX_MASK)
+ << KVX_ITGEN_CFG_TARGET_MAILBOX_SHIFT;
+
+ /**
+ * msg->data contains the bit number to be written and is included in
+ * the itgen config
+ */
+ cfg_val |= ((msg->data << KVX_ITGEN_CFG_TARGET_SELECT_BIT_SHIFT)
+ & KVX_ITGEN_CFG_TARGET_SELECT_BIT_MASK);
+
+ dev_dbg(&itgen->pdev->dev,
+ "Writing dest_addr %lx, value %x to cfg %p\n",
+ dest_addr, cfg_val, cfg);
+
+ writel(cfg_val, cfg);
+}
+
+static int
+kvx_itgen_device_probe(struct platform_device *pdev)
+{
+ struct kvx_itgen *itgen;
+ u32 it_count;
+ struct resource *mem;
+
+ itgen = devm_kzalloc(&pdev->dev, sizeof(*itgen), GFP_KERNEL);
+ if (!itgen)
+ return -ENOMEM;
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ itgen->base = devm_ioremap_resource(&pdev->dev, mem);
+ if (IS_ERR(itgen->base)) {
+ dev_err(&pdev->dev, "Failed to ioremap itgen\n");
+ return PTR_ERR(itgen->base);
+ }
+
+ itgen->pdev = pdev;
+ it_count = readl(get_itgen_param_offset(itgen) +
+ KVX_ITGEN_PARAM_IT_NUM_OFFSET);
+
+ itgen->domain = platform_msi_create_device_domain(&pdev->dev,
+ it_count,
+ kvx_itgen_write_msg,
+ &itgen_domain_ops,
+ itgen);
+ if (!itgen->domain) {
+ dev_err(&pdev->dev, "Failed to create device domain\n");
+ return -ENOMEM;
+ }
+
+ dev_info(&pdev->dev, "Probed with %d interrupts\n", it_count);
+
+ platform_set_drvdata(pdev, itgen);
+
+ return 0;
+}
+
+static const struct of_device_id itgen_of_match[] = {
+ { .compatible = "kalray,kvx-itgen" },
+ { /* END */ }
+};
+MODULE_DEVICE_TABLE(of, itgen_of_match);
+
+static struct platform_driver itgen_platform_driver = {
+ .driver = {
+ .name = "kvx-itgen",
+ .of_match_table = itgen_of_match,
+ },
+ .probe = kvx_itgen_device_probe,
+};
+
+static int __init kvx_itgen_init(void)
+{
+ return platform_driver_register(&itgen_platform_driver);
+}
+
+arch_initcall(kvx_itgen_init);
diff --git a/include/linux/irqchip/irq-kvx-apic-gic.h b/include/linux/irqchip/irq-kvx-apic-gic.h
new file mode 100644
index 000000000000..8efbcd05f3ea
--- /dev/null
+++ b/include/linux/irqchip/irq-kvx-apic-gic.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018-2023 Kalray Inc.
+ * Author: Clement Leger
+ */
+
+#ifndef KVX_APIC_GIC_H
+#define KVX_APIC_GIC_H
+
+/* GIC enable register definitions */
+#define KVX_GIC_ENABLE_OFFSET 0x0
+#define KVX_GIC_ENABLE_ELEM_SIZE 0x1
+#define KVX_GIC_INPUT_IT_COUNT 0x9D
+#define KVX_GIC_ELEM_SIZE 0x400
+
+/* GIC status lac register definitions */
+#define KVX_GIC_STATUS_LAC_OFFSET 0x120
+#define KVX_GIC_STATUS_LAC_ELEM_SIZE 0x8
+#define KVX_GIC_STATUS_LAC_ARRAY_SIZE 0x3
+
+#endif /* KVX_APIC_GIC_H */
diff --git a/include/linux/irqchip/irq-kvx-apic-mailbox.h b/include/linux/irqchip/irq-kvx-apic-mailbox.h
new file mode 100644
index 000000000000..40ec60de155c
--- /dev/null
+++ b/include/linux/irqchip/irq-kvx-apic-mailbox.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018-2023 Kalray Inc.
+ */
+
+#ifndef IRQ_KVX_APIC_MAILBOX_H
+#define IRQ_KVX_APIC_MAILBOX_H
+
+#define KVX_MAILBOX_MODE_WRITE 0x0
+#define KVX_MAILBOX_MODE_OR 0x1
+#define KVX_MAILBOX_MODE_ADD 0x2
+
+#define KVX_MAILBOX_TRIG_NO_TRIG 0x0
+#define KVX_MAILBOX_TRIG_DOORBELL 0x1
+#define KVX_MAILBOX_TRIG_MATCH 0x2
+#define KVX_MAILBOX_TRIG_BARRIER 0x3
+#define KVX_MAILBOX_TRIG_THRESHOLD 0x4
+
+/* Mailbox defines */
+#define KVX_MAILBOX_OFFSET 0x0
+#define KVX_MAILBOX_ELEM_SIZE 0x200
+#define KVX_MAILBOX_MASK_OFFSET 0x10
+#define KVX_MAILBOX_FUNCT_OFFSET 0x18
+#define KVX_MAILBOX_LAC_OFFSET 0x8
+#define KVX_MAILBOX_VALUE_OFFSET 0x0
+#define KVX_MAILBOX_FUNCT_MODE_SHIFT 0x0
+#define KVX_MAILBOX_FUNCT_TRIG_SHIFT 0x8
+
+#endif /* IRQ_KVX_APIC_MAILBOX_H */
diff --git a/include/linux/irqchip/irq-kvx-itgen.h b/include/linux/irqchip/irq-kvx-itgen.h
new file mode 100644
index 000000000000..6845a2509b14
--- /dev/null
+++ b/include/linux/irqchip/irq-kvx-itgen.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018-2023 Kalray Inc.
+ */
+
+#ifndef IRQ_KVX_ITGEN_H
+#define IRQ_KVX_ITGEN_H
+
+/* Parameters */
+#define KVX_ITGEN_PARAM_OFFSET 0x1100
+#define KVX_ITGEN_PARAM_IT_NUM_OFFSET 0x0
+
+/* Target configuration */
+#define KVX_ITGEN_CFG_ENABLE_OFFSET 0x8
+#define KVX_ITGEN_CFG_ELEM_SIZE 0x10
+#define KVX_ITGEN_CFG_TARGET_OFFSET 0x0
+#define KVX_ITGEN_CFG_TARGET_MAILBOX_SHIFT 0x0
+#define KVX_ITGEN_CFG_TARGET_MAILBOX_MASK 0x7FUL
+#define KVX_ITGEN_CFG_TARGET_CLUSTER_SHIFT 0x8
+#define KVX_ITGEN_CFG_TARGET_CLUSTER_MASK 0x700UL
+#define KVX_ITGEN_CFG_TARGET_SELECT_BIT_SHIFT 0x18
+#define KVX_ITGEN_CFG_TARGET_SELECT_BIT_MASK 0x3F000000UL
+
+#endif /* IRQ_KVX_ITGEN_H */
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 09/25] kvx: irqchip: Add support for irq controllers
2023-01-03 16:43 ` [RFC PATCH 09/25] kvx: irqchip: Add support for irq controllers Yann Sionneau
@ 2023-01-03 21:28 ` Rob Herring
0 siblings, 0 replies; 113+ messages in thread
From: Rob Herring @ 2023-01-03 21:28 UTC (permalink / raw)
To: Yann Sionneau
Cc: Thomas Gleixner, Marc Zyngier, Krzysztof Kozlowski, linux-kernel,
devicetree, Clement Leger, Jules Maselbas, Julian Vetter,
Luc Michel, Vincent Chardon
On Tue, Jan 03, 2023 at 05:43:43PM +0100, Yann Sionneau wrote:
> Add support for kvx irq controllers found in Coolidge MPPA SoC
>
> The core-intc:
>
> Each kvx core includes a hardware interrupt controller (core ITC)
> with the following features:
> * 32 independent interrupt sources
> * 4-bit priotity level
> * Individual interrupt enable bit
> * Interrupt status bit displaying the pending interrupts
> * Priority management between the 32 interrupts
>
> Among those 32 interrupt sources, the first are hard-wired to hardware
> sources. The remaining interrupt sources can be triggered via software
> by directly writing to the ILR SFR.
>
> The hard-wired interrupt sources are the following:
> 0: Timer 0
> 1: Timer 1
> 2: Watchdog
> 3: Performance Monitors
> 4: APIC GIC line 0
> 5: APIC GIC line 1
> 6: APIC GIC line 2
> 7: APIC GIC line 3
> 12: SECC error from memory system
> 13: Arithmetic exception (carry and IEEE 754 flags)
> 16: Data Asynchronous Memory Error (DAME), raised for DECC/DSYS errors
> 17: CLI (Cache Line Invalidation) for L1D or L1I following DECC/DSYS/Parity
> errors
>
> The APIC GIC lines will be used to route interrupts coming from SoC peripherals
> from outside the Cluster to the kvx core. Those peripherals include USB host
> controller, eMMC/SD host controller, i2c, spi, PCIe, IOMMUs etc...
>
> The APIC GIC:
>
> Each Cluster of the Coolidge SoC includes an APIC
> (Advanced Programmable Interrupt Controller) GIC (Generic Interrupt Controller).
> The APIC GIC acts as an intermediary interrupt controller, muxing/routing
> incoming interrupts to output interrupts connected to the kvx core ITC lines.
> The first 128 incoming interrupt lines come from the mailbox controller (itself
> containing 128 mailboxes).
> The remaining 11 interrupt lines come from external interrupt sources (NoC
> router, the 5 IOMMUs, L2$ DMA job fifo, watchdog, SECC, DECC, D NoC).
> The APIC GIC has 72 output interrupts: 4 per kvx cores in the cluster
> (1 RM and 16 PE) connected to the "APIC GIC lines" described above and 1 for the
> L2$ controller which makes 69 interrupts lines (rounded up to 72).
>
> The APIC Mailbox:
>
> The APIC includes a mailbox controller, containing 128 mailboxes.
> This hardware block is basically a 1 Kb of smart memory space.
> Each mailbox is an 8 bytes word memory location which can generate and
> interrupt.
> Each mailbox has a trigger function and an input function.
> When a mailbox is written to, if the condition described by the
> trigger function is satisfied, the corresponding interrupt
> will fire.
> Since this hardware block generates IRQs based on writes
> at some memory locations, it is both an interrupt controller
> and an MSI controller.
>
> The ITGEN:
>
> The ITGEN (InTerrupt GENerator) is an interrupt controller block.
> It's purpose is to convert IRQ lines coming from SoC peripherals
> (USB host controller for instance) into writes on the AXI bus.
> Those writes are targeting the APIC Mailboxes.
>
> CC: Thomas Gleixner <tglx@linutronix.de>
> CC: Marc Zyngier <maz@kernel.org>
> CC: Rob Herring <robh+dt@kernel.org>
> CC: Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>
> CC: linux-kernel@vger.kernel.org
> CC: devicetree@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
> Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Co-developed-by: Luc Michel <lmichel@kalray.eu>
> Signed-off-by: Luc Michel <lmichel@kalray.eu>
> Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
> Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> .../kalray,kvx-core-intc.txt | 22 +
Bindings should be a separate patch and must be in schema format now.
You probably should do a patch per driver too.
> diff --git a/Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt b/Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt
> new file mode 100644
> index 000000000000..503a661e1e84
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt
> @@ -0,0 +1,22 @@
> +* KVX Core Interrupt controller
> +
> +Required properties:
> +
> +- compatible: must to be "kalray,kvx-core-intc".
compatible strings should be specific enough to identify bugs/features
of a specific implementation unless there's another way to do that (e.g.
version registers).
> +- interrupt-controller
> +- #interrupt-cells: has to be <1>: an interrupt index
> +- regs: Base address of interrupt controller registers.
> +
> +Optional properties:
> +
> +- kalray,intc-nr-irqs: Number of irqs handled by the controller.
> + if not given, will default to 32.
What's the range?
> +
> +Example:
> +
> + core_intc: core_intc@0 {
interrupt-controller {
> + compatible = "kalray,kvx-core-intc";
> + #interrupt-cells = <1>;
> + interrupt-controller;
> + interrupt-parent = <&core_intc>;
> + };
[...]
> +IRQCHIP_DECLARE(kvx_apic_gic, "kalray,kvx-apic-gic", kvx_init_apic_gic);
> +IRQCHIP_DECLARE(kvx_apic_mailbox, "kalray,kvx-apic-mailbox",
> + kvx_init_apic_mailbox);
> +static const struct of_device_id itgen_of_match[] = {
> + { .compatible = "kalray,kvx-itgen" },
> + { /* END */ }
> +};
All these compatible strings need binding schemas, too.
> diff --git a/include/linux/irqchip/irq-kvx-apic-gic.h b/include/linux/irqchip/irq-kvx-apic-gic.h
> new file mode 100644
> index 000000000000..8efbcd05f3ea
> --- /dev/null
> +++ b/include/linux/irqchip/irq-kvx-apic-gic.h
> @@ -0,0 +1,21 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2018-2023 Kalray Inc.
> + * Author: Clement Leger
> + */
> +
> +#ifndef KVX_APIC_GIC_H
> +#define KVX_APIC_GIC_H
> +
> +/* GIC enable register definitions */
> +#define KVX_GIC_ENABLE_OFFSET 0x0
> +#define KVX_GIC_ENABLE_ELEM_SIZE 0x1
> +#define KVX_GIC_INPUT_IT_COUNT 0x9D
> +#define KVX_GIC_ELEM_SIZE 0x400
> +
> +/* GIC status lac register definitions */
> +#define KVX_GIC_STATUS_LAC_OFFSET 0x120
> +#define KVX_GIC_STATUS_LAC_ELEM_SIZE 0x8
> +#define KVX_GIC_STATUS_LAC_ARRAY_SIZE 0x3
Do these defines need to be public to *all* the kernel? Doesn't look
like it.
Same question on the other headers.
Rob
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 10/25] kvx: Add process management
2023-01-03 16:43 ` Yann Sionneau
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Oleg Nesterov, Christian Brauner, Paul Walmsley,
Palmer Dabbelt, Albert Ou, linux-kernel, linux-riscv,
Clement Leger, Guillaume Thouvenin, Julian Vetter, Marius Gligor,
Vincent Chardon
Add process management support for kvx, including: thread info
definition, context switch and process tracing.
CC: Oleg Nesterov <oleg@redhat.com>
CC: Christian Brauner <brauner@kernel.org>
CC: Paul Walmsley <paul.walmsley@sifive.com>
CC: Palmer Dabbelt <palmer@dabbelt.com>
CC: Albert Ou <aou@eecs.berkeley.edu>
CC: linux-kernel@vger.kernel.org
CC: linux-riscv@lists.infradead.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/current.h | 22 ++
arch/kvx/include/asm/ptrace.h | 217 ++++++++++++++
arch/kvx/include/asm/switch_to.h | 21 ++
arch/kvx/include/asm/thread_info.h | 78 +++++
arch/kvx/include/uapi/asm/ptrace.h | 114 +++++++
arch/kvx/kernel/process.c | 212 +++++++++++++
arch/kvx/kernel/ptrace.c | 461 +++++++++++++++++++++++++++++
arch/kvx/kernel/stacktrace.c | 173 +++++++++++
8 files changed, 1298 insertions(+)
create mode 100644 arch/kvx/include/asm/current.h
create mode 100644 arch/kvx/include/asm/ptrace.h
create mode 100644 arch/kvx/include/asm/switch_to.h
create mode 100644 arch/kvx/include/asm/thread_info.h
create mode 100644 arch/kvx/include/uapi/asm/ptrace.h
create mode 100644 arch/kvx/kernel/process.c
create mode 100644 arch/kvx/kernel/ptrace.c
create mode 100644 arch/kvx/kernel/stacktrace.c
diff --git a/arch/kvx/include/asm/current.h b/arch/kvx/include/asm/current.h
new file mode 100644
index 000000000000..b5fd0f076ec9
--- /dev/null
+++ b/arch/kvx/include/asm/current.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_CURRENT_H
+#define _ASM_KVX_CURRENT_H
+
+#include <asm/percpu.h>
+#include <asm/sfr.h>
+
+struct task_struct;
+
+static __always_inline struct task_struct *get_current(void)
+{
+ return (struct task_struct *) kvx_sfr_get(SR);
+}
+
+#define current get_current()
+
+#endif /* _ASM_KVX_CURRENT_H */
diff --git a/arch/kvx/include/asm/ptrace.h b/arch/kvx/include/asm/ptrace.h
new file mode 100644
index 000000000000..d1b1e0975d9e
--- /dev/null
+++ b/arch/kvx/include/asm/ptrace.h
@@ -0,0 +1,217 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Marius Gligor
+ * Yann Sionneau
+ */
+
+#ifndef _ASM_KVX_PTRACE_H
+#define _ASM_KVX_PTRACE_H
+
+#include <asm/types.h>
+#include <asm/sfr.h>
+#include <uapi/asm/ptrace.h>
+
+#define GPR_COUNT 64
+#define SFR_COUNT 9
+#define VIRT_COUNT 1
+
+#define ES_SYSCALL 0x3
+
+#define KVX_HW_BREAKPOINT_COUNT 2
+#define KVX_HW_WATCHPOINT_COUNT 1
+
+#define REG_SIZE sizeof(u64)
+
+/**
+ * When updating pt_regs structure, this size must be updated.
+ * This is the expected size of the pt_regs struct.
+ * It ensures the structure layout from gcc is the same as the one we
+ * expect in order to do packed load (load/store octuple) in assembly.
+ * Conclusion: never put sizeof(pt_regs) in here or we lose this check
+ * (build time check done in asm-offsets.c via BUILD_BUG_ON)
+ */
+#define PT_REGS_STRUCT_EXPECTED_SIZE \
+ ((GPR_COUNT + SFR_COUNT + VIRT_COUNT) * REG_SIZE + \
+ 2 * REG_SIZE) /* Padding for stack alignment */
+
+/**
+ * Saved register structure. Note that we should save only the necessary
+ * registers.
+ * When you modify it, please read carefully the comment above.
+ * Moreover, you will need to modify user_pt_regs to match the beginning
+ * of this struct 1:1
+ */
+struct pt_regs {
+ union {
+ struct user_pt_regs user_regs;
+ struct {
+ /* GPR */
+ uint64_t r0;
+ uint64_t r1;
+ uint64_t r2;
+ uint64_t r3;
+ uint64_t r4;
+ uint64_t r5;
+ uint64_t r6;
+ uint64_t r7;
+ uint64_t r8;
+ uint64_t r9;
+ uint64_t r10;
+ uint64_t r11;
+ union {
+ uint64_t r12;
+ uint64_t sp;
+ };
+ union {
+ uint64_t r13;
+ uint64_t tp;
+ };
+ union {
+ uint64_t r14;
+ uint64_t fp;
+ };
+ uint64_t r15;
+ uint64_t r16;
+ uint64_t r17;
+ uint64_t r18;
+ uint64_t r19;
+ uint64_t r20;
+ uint64_t r21;
+ uint64_t r22;
+ uint64_t r23;
+ uint64_t r24;
+ uint64_t r25;
+ uint64_t r26;
+ uint64_t r27;
+ uint64_t r28;
+ uint64_t r29;
+ uint64_t r30;
+ uint64_t r31;
+ uint64_t r32;
+ uint64_t r33;
+ uint64_t r34;
+ uint64_t r35;
+ uint64_t r36;
+ uint64_t r37;
+ uint64_t r38;
+ uint64_t r39;
+ uint64_t r40;
+ uint64_t r41;
+ uint64_t r42;
+ uint64_t r43;
+ uint64_t r44;
+ uint64_t r45;
+ uint64_t r46;
+ uint64_t r47;
+ uint64_t r48;
+ uint64_t r49;
+ uint64_t r50;
+ uint64_t r51;
+ uint64_t r52;
+ uint64_t r53;
+ uint64_t r54;
+ uint64_t r55;
+ uint64_t r56;
+ uint64_t r57;
+ uint64_t r58;
+ uint64_t r59;
+ uint64_t r60;
+ uint64_t r61;
+ uint64_t r62;
+ uint64_t r63;
+
+ /* SFR */
+ uint64_t lc;
+ uint64_t le;
+ uint64_t ls;
+ uint64_t ra;
+
+ uint64_t cs;
+ uint64_t spc;
+ };
+ };
+ uint64_t sps;
+ uint64_t es;
+
+ uint64_t ilr;
+
+ /* "Virtual" registers */
+ uint64_t orig_r0;
+
+ /* Padding for stack alignment (see STACK_ALIGN) */
+ uint64_t padding[2];
+
+ /**
+ * If you add some fields, please read carefully the comment for
+ * PT_REGS_STRUCT_EXPECTED_SIZE.
+ */
+};
+
+#define pl(__reg) kvx_sfr_field_val(__reg, PS, PL)
+
+#define MODE_KERNEL 0
+#define MODE_USER 1
+
+/* Privilege level is relative in $sps, so 1 indicates current PL + 1 */
+#define user_mode(regs) (pl((regs)->sps) == MODE_USER)
+#define es_ec(regs) kvx_sfr_field_val(regs->es, ES, EC)
+#define es_sysno(regs) kvx_sfr_field_val(regs->es, ES, SN)
+
+#define debug_dc(es) kvx_sfr_field_val((es), ES, DC)
+
+/* ptrace */
+#define PTRACE_GET_HW_PT_REGS 20
+#define PTRACE_SET_HW_PT_REGS 21
+#define arch_has_single_step() 1
+
+#define DEBUG_CAUSE_BREAKPOINT 0
+#define DEBUG_CAUSE_WATCHPOINT 1
+#define DEBUG_CAUSE_STEPI 2
+#define DEBUG_CAUSE_DSU_BREAK 3
+
+static inline void enable_single_step(struct pt_regs *regs)
+{
+ regs->sps |= KVX_SFR_PS_SME_MASK;
+}
+
+static inline void disable_single_step(struct pt_regs *regs)
+{
+ regs->sps &= ~KVX_SFR_PS_SME_MASK;
+}
+
+static inline bool in_syscall(struct pt_regs const *regs)
+{
+ return es_ec(regs) == ES_SYSCALL;
+}
+
+int do_syscall_trace_enter(struct pt_regs *regs, unsigned long syscall);
+void do_syscall_trace_exit(struct pt_regs *regs);
+
+static inline unsigned long get_current_sp(void)
+{
+ register const unsigned long current_sp __asm__ ("$r12");
+
+ return current_sp;
+}
+
+extern char *user_scall_rt_sigreturn_end;
+extern char *user_scall_rt_sigreturn;
+
+static inline unsigned long instruction_pointer(struct pt_regs *regs)
+{
+ return regs->spc;
+}
+
+static inline long regs_return_value(struct pt_regs *regs)
+{
+ return regs->r0;
+}
+
+static inline unsigned long user_stack_pointer(struct pt_regs *regs)
+{
+ return regs->sp;
+}
+
+#endif /* _ASM_KVX_PTRACE_H */
diff --git a/arch/kvx/include/asm/switch_to.h b/arch/kvx/include/asm/switch_to.h
new file mode 100644
index 000000000000..2b1fda06dea8
--- /dev/null
+++ b/arch/kvx/include/asm/switch_to.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SWITCH_TO_H
+#define _ASM_KVX_SWITCH_TO_H
+
+struct task_struct;
+
+/* context switching is now performed out-of-line in switch_to.S */
+extern struct task_struct *__switch_to(struct task_struct *prev,
+ struct task_struct *next);
+
+#define switch_to(prev, next, last) \
+ do { \
+ ((last) = __switch_to((prev), (next))); \
+ } while (0)
+
+#endif /* _ASM_KVX_SWITCH_TO_H */
diff --git a/arch/kvx/include/asm/thread_info.h b/arch/kvx/include/asm/thread_info.h
new file mode 100644
index 000000000000..4ce0154813ef
--- /dev/null
+++ b/arch/kvx/include/asm/thread_info.h
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef _ASM_KVX_THREAD_INFO_H
+#define _ASM_KVX_THREAD_INFO_H
+
+#include <asm/page.h>
+
+/*
+ * Size of the kernel stack for each process.
+ */
+#define THREAD_SIZE_ORDER 2
+#define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER)
+
+/*
+ * Thread information flags
+ * these are process state flags that various assembly files may need to
+ * access
+ * - pending work-to-be-done flags are in LSW
+ * - other flags in MSW
+ */
+#define TIF_SYSCALL_TRACE 0 /* syscall trace active */
+#define TIF_NOTIFY_RESUME 1 /* resumption notification requested */
+#define TIF_SIGPENDING 2 /* signal pending */
+#define TIF_NEED_RESCHED 3 /* rescheduling necessary */
+#define TIF_SINGLESTEP 4 /* restore singlestep on return to user mode */
+#define TIF_UPROBE 5
+#define TIF_SYSCALL_TRACEPOINT 6 /* syscall tracepoint instrumentation */
+#define TIF_SYSCALL_AUDIT 7 /* syscall auditing active */
+#define TIF_RESTORE_SIGMASK 9
+#define TIF_NOTIFY_SIGNAL 10 /* signal notifications exist */
+#define TIF_POLLING_NRFLAG 16 /* true if poll_idle() is polling TIF_NEED_RESCHED */
+#define TIF_MEMDIE 17
+
+#define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE)
+#define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT)
+#define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT)
+#define _TIF_POLLING_NRFLAG (1 << TIF_POLLING_NRFLAG)
+#define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME)
+#define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
+#define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
+#define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL)
+
+#define _TIF_WORK_MASK \
+ (_TIF_NOTIFY_RESUME | _TIF_SIGPENDING | _TIF_NEED_RESCHED)
+
+#define _TIF_SYSCALL_WORK \
+ (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_TRACEPOINT | _TIF_SYSCALL_AUDIT)
+
+#ifndef __ASSEMBLY__
+/*
+ * We are using THREAD_INFO_IN_TASK so this struct is almost useless
+ * please prefer adding fields in thread_struct (processor.h) rather
+ * than here.
+ * This struct is merely a remnant of distant times where it was placed
+ * on the stack to avoid large task_struct.
+ *
+ * cf https://lwn.net/Articles/700615/
+ */
+struct thread_info {
+ unsigned long flags; /* low level flags */
+ int preempt_count;
+#ifdef CONFIG_SMP
+ u32 cpu; /* current CPU */
+#endif
+};
+
+#define INIT_THREAD_INFO(tsk) \
+{ \
+ .flags = 0, \
+ .preempt_count = INIT_PREEMPT_COUNT, \
+}
+#endif /* __ASSEMBLY__*/
+#endif /* _ASM_KVX_THREAD_INFO_H */
diff --git a/arch/kvx/include/uapi/asm/ptrace.h b/arch/kvx/include/uapi/asm/ptrace.h
new file mode 100644
index 000000000000..f5febe830526
--- /dev/null
+++ b/arch/kvx/include/uapi/asm/ptrace.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ */
+
+#ifndef _UAPI_ASM_KVX_PTRACE_H
+#define _UAPI_ASM_KVX_PTRACE_H
+
+#include <linux/types.h>
+/*
+ * User-mode register state for core dumps, ptrace, sigcontext
+ *
+ * This decouples struct pt_regs from the userspace ABI.
+ * The struct pt_regs must start with the same layout as struct user_pt_regs.
+ */
+struct user_pt_regs {
+ /* GPR */
+ unsigned long r0;
+ unsigned long r1;
+ unsigned long r2;
+ unsigned long r3;
+ unsigned long r4;
+ unsigned long r5;
+ unsigned long r6;
+ unsigned long r7;
+ unsigned long r8;
+ unsigned long r9;
+ unsigned long r10;
+ unsigned long r11;
+ union {
+ unsigned long r12;
+ unsigned long sp;
+ };
+ union {
+ unsigned long r13;
+ unsigned long tp;
+ };
+ union {
+ unsigned long r14;
+ unsigned long fp;
+ };
+ unsigned long r15;
+ unsigned long r16;
+ unsigned long r17;
+ unsigned long r18;
+ unsigned long r19;
+ unsigned long r20;
+ unsigned long r21;
+ unsigned long r22;
+ unsigned long r23;
+ unsigned long r24;
+ unsigned long r25;
+ unsigned long r26;
+ unsigned long r27;
+ unsigned long r28;
+ unsigned long r29;
+ unsigned long r30;
+ unsigned long r31;
+ unsigned long r32;
+ unsigned long r33;
+ unsigned long r34;
+ unsigned long r35;
+ unsigned long r36;
+ unsigned long r37;
+ unsigned long r38;
+ unsigned long r39;
+ unsigned long r40;
+ unsigned long r41;
+ unsigned long r42;
+ unsigned long r43;
+ unsigned long r44;
+ unsigned long r45;
+ unsigned long r46;
+ unsigned long r47;
+ unsigned long r48;
+ unsigned long r49;
+ unsigned long r50;
+ unsigned long r51;
+ unsigned long r52;
+ unsigned long r53;
+ unsigned long r54;
+ unsigned long r55;
+ unsigned long r56;
+ unsigned long r57;
+ unsigned long r58;
+ unsigned long r59;
+ unsigned long r60;
+ unsigned long r61;
+ unsigned long r62;
+ unsigned long r63;
+
+ /* SFR */
+ unsigned long lc;
+ unsigned long le;
+ unsigned long ls;
+ unsigned long ra;
+
+ unsigned long cs;
+ unsigned long spc;
+};
+
+/* TCA registers structure exposed to user */
+struct user_tca_regs {
+ struct {
+ __u64 x;
+ __u64 y;
+ __u64 z;
+ __u64 t;
+ } regs[48];
+};
+
+#endif /* _UAPI_ASM_KVX_PTRACE_H */
diff --git a/arch/kvx/kernel/process.c b/arch/kvx/kernel/process.c
new file mode 100644
index 000000000000..f9ba2571e279
--- /dev/null
+++ b/arch/kvx/kernel/process.c
@@ -0,0 +1,212 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Marius Gligor
+ * Yann Sionneau
+ */
+
+#include <linux/elf.h>
+#include <linux/sched.h>
+#include <linux/ptrace.h>
+#include <linux/printk.h>
+#include <linux/sched/debug.h>
+#include <linux/sched/task_stack.h>
+#include <linux/hw_breakpoint.h>
+
+#include <asm/ptrace.h>
+#include <asm/processor.h>
+#include <asm/ptrace.h>
+#include <asm/uaccess.h>
+#include <asm/stacktrace.h>
+#include <asm/hw_breakpoint.h>
+
+#if defined(CONFIG_STACKPROTECTOR)
+#include <linux/stackprotector.h>
+unsigned long __stack_chk_guard __read_mostly;
+EXPORT_SYMBOL(__stack_chk_guard);
+#endif
+
+#define SCALL_NUM_EXIT "0xfff"
+
+void arch_cpu_idle(void)
+{
+ wait_for_interrupt();
+ local_irq_enable();
+}
+
+void show_regs(struct pt_regs *regs)
+{
+
+ int in_kernel = 1;
+ unsigned short i, reg_offset;
+ void *ptr;
+
+ show_regs_print_info(KERN_DEFAULT);
+
+ if (user_mode(regs))
+ in_kernel = 0;
+
+ pr_info("\nmode: %s\n"
+ " PC: %016llx PS: %016llx\n"
+ " CS: %016llx RA: %016llx\n"
+ " LS: %016llx LE: %016llx\n"
+ " LC: %016llx\n\n",
+ in_kernel ? "kernel" : "user",
+ regs->spc, regs->sps,
+ regs->cs, regs->ra, regs->ls, regs->le, regs->lc);
+
+ /* GPR */
+ ptr = regs;
+ ptr += offsetof(struct pt_regs, r0);
+ reg_offset = offsetof(struct pt_regs, r1) -
+ offsetof(struct pt_regs, r0);
+
+ /**
+ * Display all the 64 GPRs assuming they are ordered correctly
+ * in the pt_regs struct...
+ */
+ for (i = 0; i < GPR_COUNT; i += 2) {
+ pr_info(" R%d: %016llx R%d: %016llx\n",
+ i, *(uint64_t *)ptr,
+ i + 1, *(uint64_t *)(ptr + reg_offset));
+ ptr += reg_offset * 2;
+ }
+
+ pr_info("\n\n");
+}
+
+/**
+ * Prepare a thread to return to userspace
+ */
+void start_thread(struct pt_regs *regs,
+ unsigned long pc, unsigned long sp)
+{
+ /* Remove MMUP bit (user is not privilege in current virtual space) */
+ u64 clear_bit = KVX_SFR_PS_MMUP_MASK | KVX_SFR_PS_SME_MASK |
+ KVX_SFR_PS_SMR_MASK;
+ regs->spc = pc;
+ regs->sp = sp;
+ regs->sps = kvx_sfr_get(PS);
+
+ regs->sps &= ~clear_bit;
+
+ /* Set privilege level to +1 (relative) */
+ regs->sps &= ~KVX_SFR_PS_PL_MASK;
+ regs->sps |= (1 << KVX_SFR_PS_PL_SHIFT);
+}
+
+int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
+{
+ struct pt_regs *regs, *childregs = task_pt_regs(p);
+ unsigned long clone_flags = args->flags;
+ unsigned long usp = args->stack;
+ unsigned long tls = args->tls;
+
+ /* p->thread holds context to be restored by __switch_to() */
+ if (unlikely(args->fn)) {
+ /* Kernel thread */
+ memset(childregs, 0, sizeof(struct pt_regs));
+
+ p->thread.ctx_switch.r20 = (uint64_t)args->fn; /* fn */
+ p->thread.ctx_switch.r21 = (uint64_t)args->fn_arg;
+ p->thread.ctx_switch.ra =
+ (unsigned long) ret_from_kernel_thread;
+ } else {
+ regs = current_pt_regs();
+
+ /* Copy current process registers */
+ *childregs = *regs;
+
+ /* Store tracing status in r20 to avoid computing it
+ * in assembly
+ */
+ p->thread.ctx_switch.r20 =
+ task_thread_info(p)->flags & _TIF_SYSCALL_WORK;
+ p->thread.ctx_switch.ra = (unsigned long) ret_from_fork;
+
+ childregs->r0 = 0; /* Return value of fork() */
+ /* Set stack pointer if any */
+ if (usp)
+ childregs->sp = usp;
+
+ /* Set a new TLS ? */
+ if (clone_flags & CLONE_SETTLS)
+ childregs->r13 = tls;
+ }
+ p->thread.kernel_sp =
+ (unsigned long) (task_stack_page(p) + THREAD_SIZE);
+ p->thread.ctx_switch.sp = (unsigned long) childregs;
+
+ clear_ptrace_hw_breakpoint(p);
+
+ return 0;
+}
+
+void release_thread(struct task_struct *dead_task)
+{
+}
+
+void flush_thread(void)
+{
+ /* This function should clear the values of the floating point
+ * registers and debug registers saved in the TSS segment.
+ */
+
+ flush_ptrace_hw_breakpoint(current);
+}
+
+/* Fill in the fpu structure for a core dump. */
+int dump_fpu(struct pt_regs *regs, elf_fpregset_t *fpu)
+{
+ /*
+ * On kvx, FPU uses standard registers + $cs which is a common register
+ * also needed for non-fpu execution, so there is no additional
+ * register to dump.
+ */
+ return 0;
+}
+
+static bool find_wchan(unsigned long pc, void *arg)
+{
+ unsigned long *p = arg;
+
+ /*
+ * If the pc is in a scheduler function (waiting), then, this is the
+ * address where the process is currently stuck. Note that scheduler
+ * functions also include lock functions. This functions are
+ * materialized using annotation to put them is special text sections.
+ */
+ if (!in_sched_functions(pc)) {
+ *p = pc;
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * __get_wchan is called to obtain "schedule()" caller function address.
+ */
+unsigned long __get_wchan(struct task_struct *p)
+{
+ unsigned long pc = 0;
+ struct stackframe frame;
+
+ /*
+ * We need to obtain the task stack since we don't want the stack to
+ * move under our feet.
+ */
+ if (!try_get_task_stack(p))
+ return 0;
+
+ start_stackframe(&frame, thread_saved_reg(p, fp),
+ thread_saved_reg(p, ra));
+ walk_stackframe(p, &frame, find_wchan, &pc);
+
+ put_task_stack(p);
+
+ return pc;
+}
+
diff --git a/arch/kvx/kernel/ptrace.c b/arch/kvx/kernel/ptrace.c
new file mode 100644
index 000000000000..46096a3ab3e4
--- /dev/null
+++ b/arch/kvx/kernel/ptrace.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/riscv/kernel/ptrace.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Marius Gligor
+ * Clement Leger
+ */
+
+#include <linux/sched.h>
+#include <linux/sched.h>
+#include <linux/audit.h>
+#include <linux/irqflags.h>
+#include <linux/thread_info.h>
+#include <linux/context_tracking.h>
+#include <linux/uaccess.h>
+#include <linux/syscalls.h>
+#include <linux/signal.h>
+#include <linux/regset.h>
+#include <linux/hw_breakpoint.h>
+
+#include <asm/dame.h>
+#include <asm/ptrace.h>
+#include <asm/syscall.h>
+#include <asm/break_hook.h>
+#include <asm/debug.h>
+#include <asm/cacheflush.h>
+#include <asm/hw_breakpoint.h>
+
+#define CREATE_TRACE_POINTS
+#include <trace/events/syscalls.h>
+
+#define HW_PT_CMD_GET_CAPS 0
+#define HW_PT_CMD_GET_PT 1
+#define HW_PT_CMD_SET_RESERVE 0
+#define HW_PT_CMD_SET_ENABLE 1
+
+#define FROM_GDB_CMD_MASK 3
+#define FROM_GDB_HP_TYPE_SHIFT 2
+#define FROM_GDB_HP_TYPE_MASK 4
+#define FROM_GDB_WP_TYPE_SHIFT 3
+#define FROM_GDB_WP_TYPE_MASK 0x18
+#define FROM_GDB_HP_IDX_SHIFT 5
+
+#define hw_pt_cmd(addr) ((addr) & FROM_GDB_CMD_MASK)
+#define hw_pt_is_bkp(addr) ((((addr) & FROM_GDB_HP_TYPE_MASK) >> \
+ FROM_GDB_HP_TYPE_SHIFT) == KVX_HW_BREAKPOINT_TYPE)
+#define get_hw_pt_wp_type(addr) ((((addr) & FROM_GDB_WP_TYPE_MASK)) >> \
+ FROM_GDB_WP_TYPE_SHIFT)
+#define get_hw_pt_idx(addr) ((addr) >> FROM_GDB_HP_IDX_SHIFT)
+#define get_hw_pt_addr(data) ((data)[0])
+#define get_hw_pt_len(data) ((data)[1] >> 1)
+#define hw_pt_is_enabled(data) ((data)[1] & 1)
+
+enum kvx_regset {
+ REGSET_GPR,
+#ifdef CONFIG_ENABLE_TCA
+ REGSET_TCA,
+#endif
+};
+
+static void compute_ptrace_hw_pt_rsp(uint64_t *data, struct perf_event *bp)
+{
+ data[0] = bp->attr.bp_addr;
+ data[1] = bp->attr.bp_len >> 1;
+ if (!bp->attr.disabled)
+ data[1] |= 1;
+}
+
+void ptrace_disable(struct task_struct *child)
+{
+ /* nothing to do */
+}
+
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+static void ptrace_hw_pt_triggered(struct perf_event *bp,
+ struct perf_sample_data *data,
+ struct pt_regs *regs)
+{
+ int i, id = 0;
+ struct arch_hw_breakpoint *bkpt = counter_arch_bp(bp);
+
+ if (bp->attr.bp_type & HW_BREAKPOINT_X) {
+ id = KVX_HW_BREAKPOINT_TYPE;
+ for (i = 0; i < KVX_HW_BREAKPOINT_COUNT; i++)
+ if (current->thread.debug.ptrace_hbp[i] == bp)
+ break;
+ } else {
+ id = KVX_HW_WATCHPOINT_TYPE;
+ for (i = 0; i < KVX_HW_WATCHPOINT_COUNT; i++)
+ if (current->thread.debug.ptrace_hwp[i] == bp)
+ break;
+ }
+
+ id |= i << 1;
+ force_sig_ptrace_errno_trap(id, (void __user *) bkpt->addr);
+}
+
+static struct perf_event *ptrace_hw_pt_create(struct task_struct *tsk, int type)
+{
+ struct perf_event_attr attr;
+
+ ptrace_breakpoint_init(&attr);
+
+ /* Initialise fields to sane defaults. */
+ attr.bp_addr = 0;
+ attr.bp_len = 1;
+ attr.bp_type = type;
+ attr.disabled = 1;
+
+ return register_user_hw_breakpoint(&attr, ptrace_hw_pt_triggered, NULL,
+ tsk);
+}
+
+/*
+ * Address bit 0..1: command id, bit 2: hardware breakpoint (0)
+ * or watchpoint (1), bits 63..3: register number.
+ * Both PTRACE_GET_HW_PT_REGS and PTRACE_SET_HW_PT_REGS transfer two
+ * 64-bit words: for get capabilities: number of breakpoint (0) and
+ * watchpoints (1), for hardware watchpoint/breakpoint enable: address (0)
+ * and enable + length (1)
+ */
+
+static long ptrace_get_hw_pt_pregs(struct task_struct *child, long addr,
+ long __user *datap)
+{
+ struct perf_event *bp;
+ u64 user_data[2];
+ int cmd = hw_pt_cmd(addr);
+
+ if (cmd == HW_PT_CMD_GET_CAPS) {
+ user_data[0] = KVX_HW_BREAKPOINT_COUNT;
+ user_data[1] = KVX_HW_WATCHPOINT_COUNT;
+ } else if (cmd == HW_PT_CMD_GET_PT) {
+ int is_breakpoint = hw_pt_is_bkp(addr);
+ int idx = get_hw_pt_idx(addr);
+
+ if ((is_breakpoint && idx >= KVX_HW_BREAKPOINT_COUNT) ||
+ (!is_breakpoint && idx >= KVX_HW_WATCHPOINT_COUNT))
+ return -EINVAL;
+
+ if (is_breakpoint)
+ bp = child->thread.debug.ptrace_hbp[idx];
+ else
+ bp = child->thread.debug.ptrace_hwp[idx];
+
+ if (bp) {
+ compute_ptrace_hw_pt_rsp(user_data, bp);
+ } else {
+ user_data[0] = 0;
+ user_data[1] = 0;
+ }
+ } else {
+ return -EINVAL;
+ }
+
+ if (copy_to_user(datap, user_data, sizeof(user_data)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static long ptrace_set_hw_pt_regs(struct task_struct *child, long addr,
+ long __user *datap)
+{
+ struct perf_event *bp;
+ struct perf_event_attr attr;
+ u64 user_data[2];
+ int is_breakpoint = hw_pt_is_bkp(addr);
+ int idx = get_hw_pt_idx(addr);
+ int cmd = hw_pt_cmd(addr);
+ int bp_type, ret;
+
+ if ((is_breakpoint && idx >= KVX_HW_BREAKPOINT_COUNT) ||
+ (!is_breakpoint && idx >= KVX_HW_WATCHPOINT_COUNT))
+ return -EINVAL;
+
+ if (copy_from_user(user_data, datap, sizeof(user_data)))
+ return -EFAULT;
+
+ if (cmd == HW_PT_CMD_SET_RESERVE ||
+ (cmd == HW_PT_CMD_SET_ENABLE && hw_pt_is_enabled(user_data))) {
+ if (is_breakpoint)
+ ret = ptrace_request_hw_breakpoint(idx);
+ else
+ ret = ptrace_request_hw_watchpoint(idx);
+
+ if (cmd == HW_PT_CMD_SET_RESERVE || ret != 0)
+ return ret;
+ }
+
+ if (cmd != HW_PT_CMD_SET_ENABLE)
+ return -EINVAL;
+
+ if (is_breakpoint) {
+ bp = child->thread.debug.ptrace_hbp[idx];
+ bp_type = HW_BREAKPOINT_X;
+ } else {
+ bp = child->thread.debug.ptrace_hwp[idx];
+ bp_type = get_hw_pt_wp_type(addr);
+ if (!bp_type)
+ bp_type = HW_BREAKPOINT_W;
+ }
+
+ if (!bp) {
+ bp = ptrace_hw_pt_create(child, bp_type);
+ if (IS_ERR(bp))
+ return PTR_ERR(bp);
+ if (is_breakpoint)
+ child->thread.debug.ptrace_hbp[idx] = bp;
+ else
+ child->thread.debug.ptrace_hwp[idx] = bp;
+ }
+
+ attr = bp->attr;
+ attr.bp_addr = get_hw_pt_addr(user_data);
+ attr.bp_len = get_hw_pt_len(user_data);
+ attr.bp_type = bp_type;
+ attr.disabled = !hw_pt_is_enabled(user_data);
+
+ return modify_user_hw_breakpoint(bp, &attr);
+}
+#endif
+
+static int kvx_gpr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ struct membuf to)
+{
+ struct user_pt_regs *regs = &task_pt_regs(target)->user_regs;
+
+ return membuf_write(&to, regs, sizeof(*regs));
+}
+
+static int kvx_gpr_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ struct user_pt_regs *regs = &task_pt_regs(target)->user_regs;
+
+ return user_regset_copyin(&pos, &count, &kbuf, &ubuf, regs, 0, -1);
+}
+
+#ifdef CONFIG_ENABLE_TCA
+static int kvx_tca_reg_get(struct task_struct *target,
+ const struct user_regset *regset,
+ struct membuf to)
+{
+ struct ctx_switch_regs *ctx_regs = &target->thread.ctx_switch;
+ struct tca_reg *regs = ctx_regs->tca_regs;
+ int ret;
+
+ if (!ctx_regs->tca_regs_saved)
+ ret = membuf_zero(&to, sizeof(*regs));
+ else
+ ret = membuf_write(&to, regs, sizeof(*regs));
+
+ return ret;
+}
+
+static int kvx_tca_reg_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ struct ctx_switch_regs *ctx_regs = &target->thread.ctx_switch;
+ struct tca_reg *regs = ctx_regs->tca_regs;
+ int ret;
+
+ if (!ctx_regs->tca_regs_saved)
+ ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
+ 0, -1);
+ else
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, regs,
+ 0, -1);
+
+ return ret;
+}
+#endif
+
+static const struct user_regset kvx_user_regset[] = {
+ [REGSET_GPR] = {
+ .core_note_type = NT_PRSTATUS,
+ .n = ELF_NGREG,
+ .size = sizeof(elf_greg_t),
+ .align = sizeof(elf_greg_t),
+ .regset_get = &kvx_gpr_get,
+ .set = &kvx_gpr_set,
+ },
+#ifdef CONFIG_ENABLE_TCA
+ [REGSET_TCA] = {
+ .core_note_type = NT_KVX_TCA,
+ .n = TCA_REG_COUNT,
+ .size = sizeof(struct tca_reg),
+ .align = sizeof(struct tca_reg),
+ .regset_get = &kvx_tca_reg_get,
+ .set = &kvx_tca_reg_set,
+ },
+#endif
+};
+
+static const struct user_regset_view user_kvx_view = {
+ .name = "kvx",
+ .e_machine = EM_KVX,
+ .regsets = kvx_user_regset,
+ .n = ARRAY_SIZE(kvx_user_regset)
+};
+
+const struct user_regset_view *task_user_regset_view(struct task_struct *task)
+{
+ return &user_kvx_view;
+}
+
+long arch_ptrace(struct task_struct *child, long request,
+ unsigned long addr, unsigned long data)
+{
+ int ret;
+ unsigned long __user *datap = (unsigned long __user *) data;
+
+ switch (request) {
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+ case PTRACE_GET_HW_PT_REGS:
+ ret = ptrace_get_hw_pt_pregs(child, addr, datap);
+ break;
+ case PTRACE_SET_HW_PT_REGS:
+ ret = ptrace_set_hw_pt_regs(child, addr, datap);
+ break;
+#endif
+ default:
+ ret = ptrace_request(child, request, addr, data);
+ break;
+ }
+
+ return ret;
+}
+
+/*
+ * Allows PTRACE_SYSCALL to work. These are called from entry.S in
+ * {handle,ret_from}_syscall.
+ */
+int do_syscall_trace_enter(struct pt_regs *regs, unsigned long syscall)
+{
+ int ret = 0;
+
+#ifdef CONFIG_CONTEXT_TRACKING_USER
+ user_exit_callable();
+#endif
+ if (test_thread_flag(TIF_SYSCALL_TRACE))
+ ret = ptrace_report_syscall_entry(regs);
+
+#ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS
+ if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
+ trace_sys_enter(regs, syscall_get_nr(current, regs));
+#endif
+
+ audit_syscall_entry(syscall, regs->r0, regs->r1, regs->r2, regs->r3);
+
+ return ret;
+}
+
+void do_syscall_trace_exit(struct pt_regs *regs)
+{
+ if (test_thread_flag(TIF_SYSCALL_TRACE))
+ ptrace_report_syscall_exit(regs, 0);
+
+ audit_syscall_exit(regs);
+
+#ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS
+ if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
+ trace_sys_exit(regs, regs_return_value(regs));
+#endif
+
+#ifdef CONFIG_CONTEXT_TRACKING_USER
+ user_enter_callable();
+#endif
+}
+
+static int kvx_bkpt_handler(struct break_hook *brk_hook, struct pt_regs *regs)
+{
+ /* Unexpected breakpoint */
+ if (!(current->ptrace & PT_PTRACED))
+ return BREAK_HOOK_ERROR;
+
+ /* deliver the signal to userspace */
+ force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *) regs->spc);
+
+ return BREAK_HOOK_HANDLED;
+}
+
+static void kvx_stepi(struct pt_regs *regs)
+{
+ /* deliver the signal to userspace */
+ force_sig_fault(SIGTRAP, TRAP_TRACE, (void __user *) regs->spc);
+}
+
+void user_enable_single_step(struct task_struct *child)
+{
+ struct pt_regs *regs = task_pt_regs(child);
+
+ enable_single_step(regs);
+}
+
+void user_disable_single_step(struct task_struct *child)
+{
+ struct pt_regs *regs = task_pt_regs(child);
+
+ disable_single_step(regs);
+}
+
+/**
+ * Main debug handler called by the _debug_handler routine in entry.S
+ * This handler will perform the required action
+ * @es: Exception Syndrome register value
+ * @ea: Exception Address register
+ * @regs: pointer to registers saved when enter debug
+ */
+int ptrace_debug_handler(u64 ea, struct pt_regs *regs)
+{
+ int debug_cause = debug_dc(regs->es);
+
+ switch (debug_cause) {
+ case DEBUG_CAUSE_STEPI:
+ if (check_hw_watchpoint_stepped(regs))
+ user_disable_single_step(current);
+ else
+ kvx_stepi(regs);
+ break;
+ case DEBUG_CAUSE_BREAKPOINT:
+ check_hw_breakpoint(regs);
+ break;
+ case DEBUG_CAUSE_WATCHPOINT:
+ if (check_hw_watchpoint(regs, ea))
+ user_enable_single_step(current);
+ break;
+ default:
+ break;
+ }
+
+ return DEBUG_HOOK_HANDLED;
+}
+
+static struct debug_hook ptrace_debug_hook = {
+ .handler = ptrace_debug_handler,
+ .mode = MODE_USER,
+};
+
+static struct break_hook bkpt_break_hook = {
+ .id = BREAK_CAUSE_BKPT,
+ .handler = kvx_bkpt_handler,
+ .mode = MODE_USER,
+};
+
+static int __init arch_init_breakpoint(void)
+{
+ break_hook_register(&bkpt_break_hook);
+ debug_hook_register(&ptrace_debug_hook);
+
+ return 0;
+}
+
+postcore_initcall(arch_init_breakpoint);
diff --git a/arch/kvx/kernel/stacktrace.c b/arch/kvx/kernel/stacktrace.c
new file mode 100644
index 000000000000..85d52ba2d082
--- /dev/null
+++ b/arch/kvx/kernel/stacktrace.c
@@ -0,0 +1,173 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Vincent Chardon
+ */
+
+#include <linux/context_tracking.h>
+#include <linux/kallsyms.h>
+#include <linux/printk.h>
+#include <linux/init.h>
+
+#include <asm/stacktrace.h>
+#include <asm/ptrace.h>
+
+#define STACK_SLOT_PER_LINE 4
+#define STACK_MAX_SLOT_PRINT (STACK_SLOT_PER_LINE * 8)
+
+static int notrace unwind_frame(struct task_struct *task,
+ struct stackframe *frame)
+{
+ unsigned long fp = frame->fp;
+
+ /* Frame pointer must be aligned on 8 bytes */
+ if (fp & 0x7)
+ return -EINVAL;
+
+ if (!task)
+ task = current;
+
+ if (!on_task_stack(task, fp))
+ return -EINVAL;
+
+ frame->fp = READ_ONCE_NOCHECK(*(unsigned long *)(fp));
+ frame->ra = READ_ONCE_NOCHECK(*(unsigned long *)(fp + 8));
+
+ /*
+ * When starting, we set the frame pointer to 0, hence end of
+ * frame linked list is signal by that
+ */
+ if (!frame->fp)
+ return -EINVAL;
+
+ return 0;
+}
+
+void notrace walk_stackframe(struct task_struct *task, struct stackframe *frame,
+ bool (*fn)(unsigned long, void *), void *arg)
+{
+ unsigned long addr;
+ int ret;
+
+ while (1) {
+ addr = frame->ra;
+
+ if (fn(addr, arg))
+ break;
+
+ ret = unwind_frame(task, frame);
+ if (ret)
+ break;
+ }
+}
+
+#ifdef CONFIG_STACKTRACE
+bool append_stack_addr(unsigned long pc, void *arg)
+{
+ struct stack_trace *trace;
+
+ trace = (struct stack_trace *)arg;
+ if (trace->skip == 0) {
+ trace->entries[trace->nr_entries++] = pc;
+ if (trace->nr_entries == trace->max_entries)
+ return true;
+ } else {
+ trace->skip--;
+ }
+
+ return false;
+}
+
+/*
+ * Save stack-backtrace addresses into a stack_trace buffer.
+ */
+void save_stack_trace(struct stack_trace *trace)
+{
+ struct stackframe frame;
+
+ trace->nr_entries = 0;
+ /* We want to skip this function and the caller */
+ trace->skip += 2;
+
+ start_stackframe(&frame, (unsigned long) __builtin_frame_address(0),
+ (unsigned long) save_stack_trace);
+ walk_stackframe(current, &frame, append_stack_addr, trace);
+}
+EXPORT_SYMBOL(save_stack_trace);
+#endif /* CONFIG_STACKTRACE */
+
+static bool print_pc(unsigned long pc, void *arg)
+{
+ unsigned long *skip = arg;
+
+ if (*skip == 0)
+ print_ip_sym(KERN_INFO, pc);
+ else
+ (*skip)--;
+
+ return false;
+}
+
+void show_stacktrace(struct task_struct *task, struct pt_regs *regs)
+{
+ struct stackframe frame;
+ unsigned long skip = 0;
+
+ /* Obviously, we can't backtrace on usermode ! */
+ if (regs && user_mode(regs))
+ return;
+
+ if (!task)
+ task = current;
+
+ if (!try_get_task_stack(task))
+ return;
+
+ if (regs) {
+ start_stackframe(&frame, regs->fp, regs->spc);
+ } else if (task == current) {
+ /* Skip current function and caller */
+ skip = 2;
+ start_stackframe(&frame,
+ (unsigned long) __builtin_frame_address(0),
+ (unsigned long) show_stacktrace);
+ } else {
+ /* task blocked in __switch_to */
+ start_stackframe(&frame,
+ thread_saved_reg(task, fp),
+ thread_saved_reg(task, ra));
+ }
+
+ pr_info("Call Trace:\n");
+ walk_stackframe(task, &frame, print_pc, &skip);
+
+ put_task_stack(task);
+}
+
+/*
+ * If show_stack is called with a non-null task, then the task will have been
+ * claimed with try_get_task_stack by the caller. If task is NULL or current
+ * then there is no need to get task stack since it's our current stack...
+ */
+void show_stack(struct task_struct *task, unsigned long *sp)
+{
+ int i = 0;
+
+ if (!sp)
+ sp = (unsigned long *) get_current_sp();
+
+ pr_info("Stack dump (@%p):\n", sp);
+ for (i = 0; i < STACK_MAX_SLOT_PRINT; i++) {
+ if (kstack_end(sp))
+ break;
+
+ if (i && (i % STACK_SLOT_PER_LINE) == 0)
+ pr_cont("\n\t");
+
+ pr_cont("%016lx ", *sp++);
+ }
+ pr_cont("\n");
+
+ show_stacktrace(task, NULL);
+}
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 10/25] kvx: Add process management
@ 2023-01-03 16:43 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Oleg Nesterov, Christian Brauner, Paul Walmsley,
Palmer Dabbelt, Albert Ou, linux-kernel, linux-riscv,
Clement Leger, Guillaume Thouvenin, Julian Vetter, Marius Gligor,
Vincent Chardon
Add process management support for kvx, including: thread info
definition, context switch and process tracing.
CC: Oleg Nesterov <oleg@redhat.com>
CC: Christian Brauner <brauner@kernel.org>
CC: Paul Walmsley <paul.walmsley@sifive.com>
CC: Palmer Dabbelt <palmer@dabbelt.com>
CC: Albert Ou <aou@eecs.berkeley.edu>
CC: linux-kernel@vger.kernel.org
CC: linux-riscv@lists.infradead.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/current.h | 22 ++
arch/kvx/include/asm/ptrace.h | 217 ++++++++++++++
arch/kvx/include/asm/switch_to.h | 21 ++
arch/kvx/include/asm/thread_info.h | 78 +++++
arch/kvx/include/uapi/asm/ptrace.h | 114 +++++++
arch/kvx/kernel/process.c | 212 +++++++++++++
arch/kvx/kernel/ptrace.c | 461 +++++++++++++++++++++++++++++
arch/kvx/kernel/stacktrace.c | 173 +++++++++++
8 files changed, 1298 insertions(+)
create mode 100644 arch/kvx/include/asm/current.h
create mode 100644 arch/kvx/include/asm/ptrace.h
create mode 100644 arch/kvx/include/asm/switch_to.h
create mode 100644 arch/kvx/include/asm/thread_info.h
create mode 100644 arch/kvx/include/uapi/asm/ptrace.h
create mode 100644 arch/kvx/kernel/process.c
create mode 100644 arch/kvx/kernel/ptrace.c
create mode 100644 arch/kvx/kernel/stacktrace.c
diff --git a/arch/kvx/include/asm/current.h b/arch/kvx/include/asm/current.h
new file mode 100644
index 000000000000..b5fd0f076ec9
--- /dev/null
+++ b/arch/kvx/include/asm/current.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_CURRENT_H
+#define _ASM_KVX_CURRENT_H
+
+#include <asm/percpu.h>
+#include <asm/sfr.h>
+
+struct task_struct;
+
+static __always_inline struct task_struct *get_current(void)
+{
+ return (struct task_struct *) kvx_sfr_get(SR);
+}
+
+#define current get_current()
+
+#endif /* _ASM_KVX_CURRENT_H */
diff --git a/arch/kvx/include/asm/ptrace.h b/arch/kvx/include/asm/ptrace.h
new file mode 100644
index 000000000000..d1b1e0975d9e
--- /dev/null
+++ b/arch/kvx/include/asm/ptrace.h
@@ -0,0 +1,217 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Marius Gligor
+ * Yann Sionneau
+ */
+
+#ifndef _ASM_KVX_PTRACE_H
+#define _ASM_KVX_PTRACE_H
+
+#include <asm/types.h>
+#include <asm/sfr.h>
+#include <uapi/asm/ptrace.h>
+
+#define GPR_COUNT 64
+#define SFR_COUNT 9
+#define VIRT_COUNT 1
+
+#define ES_SYSCALL 0x3
+
+#define KVX_HW_BREAKPOINT_COUNT 2
+#define KVX_HW_WATCHPOINT_COUNT 1
+
+#define REG_SIZE sizeof(u64)
+
+/**
+ * When updating pt_regs structure, this size must be updated.
+ * This is the expected size of the pt_regs struct.
+ * It ensures the structure layout from gcc is the same as the one we
+ * expect in order to do packed load (load/store octuple) in assembly.
+ * Conclusion: never put sizeof(pt_regs) in here or we lose this check
+ * (build time check done in asm-offsets.c via BUILD_BUG_ON)
+ */
+#define PT_REGS_STRUCT_EXPECTED_SIZE \
+ ((GPR_COUNT + SFR_COUNT + VIRT_COUNT) * REG_SIZE + \
+ 2 * REG_SIZE) /* Padding for stack alignment */
+
+/**
+ * Saved register structure. Note that we should save only the necessary
+ * registers.
+ * When you modify it, please read carefully the comment above.
+ * Moreover, you will need to modify user_pt_regs to match the beginning
+ * of this struct 1:1
+ */
+struct pt_regs {
+ union {
+ struct user_pt_regs user_regs;
+ struct {
+ /* GPR */
+ uint64_t r0;
+ uint64_t r1;
+ uint64_t r2;
+ uint64_t r3;
+ uint64_t r4;
+ uint64_t r5;
+ uint64_t r6;
+ uint64_t r7;
+ uint64_t r8;
+ uint64_t r9;
+ uint64_t r10;
+ uint64_t r11;
+ union {
+ uint64_t r12;
+ uint64_t sp;
+ };
+ union {
+ uint64_t r13;
+ uint64_t tp;
+ };
+ union {
+ uint64_t r14;
+ uint64_t fp;
+ };
+ uint64_t r15;
+ uint64_t r16;
+ uint64_t r17;
+ uint64_t r18;
+ uint64_t r19;
+ uint64_t r20;
+ uint64_t r21;
+ uint64_t r22;
+ uint64_t r23;
+ uint64_t r24;
+ uint64_t r25;
+ uint64_t r26;
+ uint64_t r27;
+ uint64_t r28;
+ uint64_t r29;
+ uint64_t r30;
+ uint64_t r31;
+ uint64_t r32;
+ uint64_t r33;
+ uint64_t r34;
+ uint64_t r35;
+ uint64_t r36;
+ uint64_t r37;
+ uint64_t r38;
+ uint64_t r39;
+ uint64_t r40;
+ uint64_t r41;
+ uint64_t r42;
+ uint64_t r43;
+ uint64_t r44;
+ uint64_t r45;
+ uint64_t r46;
+ uint64_t r47;
+ uint64_t r48;
+ uint64_t r49;
+ uint64_t r50;
+ uint64_t r51;
+ uint64_t r52;
+ uint64_t r53;
+ uint64_t r54;
+ uint64_t r55;
+ uint64_t r56;
+ uint64_t r57;
+ uint64_t r58;
+ uint64_t r59;
+ uint64_t r60;
+ uint64_t r61;
+ uint64_t r62;
+ uint64_t r63;
+
+ /* SFR */
+ uint64_t lc;
+ uint64_t le;
+ uint64_t ls;
+ uint64_t ra;
+
+ uint64_t cs;
+ uint64_t spc;
+ };
+ };
+ uint64_t sps;
+ uint64_t es;
+
+ uint64_t ilr;
+
+ /* "Virtual" registers */
+ uint64_t orig_r0;
+
+ /* Padding for stack alignment (see STACK_ALIGN) */
+ uint64_t padding[2];
+
+ /**
+ * If you add some fields, please read carefully the comment for
+ * PT_REGS_STRUCT_EXPECTED_SIZE.
+ */
+};
+
+#define pl(__reg) kvx_sfr_field_val(__reg, PS, PL)
+
+#define MODE_KERNEL 0
+#define MODE_USER 1
+
+/* Privilege level is relative in $sps, so 1 indicates current PL + 1 */
+#define user_mode(regs) (pl((regs)->sps) == MODE_USER)
+#define es_ec(regs) kvx_sfr_field_val(regs->es, ES, EC)
+#define es_sysno(regs) kvx_sfr_field_val(regs->es, ES, SN)
+
+#define debug_dc(es) kvx_sfr_field_val((es), ES, DC)
+
+/* ptrace */
+#define PTRACE_GET_HW_PT_REGS 20
+#define PTRACE_SET_HW_PT_REGS 21
+#define arch_has_single_step() 1
+
+#define DEBUG_CAUSE_BREAKPOINT 0
+#define DEBUG_CAUSE_WATCHPOINT 1
+#define DEBUG_CAUSE_STEPI 2
+#define DEBUG_CAUSE_DSU_BREAK 3
+
+static inline void enable_single_step(struct pt_regs *regs)
+{
+ regs->sps |= KVX_SFR_PS_SME_MASK;
+}
+
+static inline void disable_single_step(struct pt_regs *regs)
+{
+ regs->sps &= ~KVX_SFR_PS_SME_MASK;
+}
+
+static inline bool in_syscall(struct pt_regs const *regs)
+{
+ return es_ec(regs) == ES_SYSCALL;
+}
+
+int do_syscall_trace_enter(struct pt_regs *regs, unsigned long syscall);
+void do_syscall_trace_exit(struct pt_regs *regs);
+
+static inline unsigned long get_current_sp(void)
+{
+ register const unsigned long current_sp __asm__ ("$r12");
+
+ return current_sp;
+}
+
+extern char *user_scall_rt_sigreturn_end;
+extern char *user_scall_rt_sigreturn;
+
+static inline unsigned long instruction_pointer(struct pt_regs *regs)
+{
+ return regs->spc;
+}
+
+static inline long regs_return_value(struct pt_regs *regs)
+{
+ return regs->r0;
+}
+
+static inline unsigned long user_stack_pointer(struct pt_regs *regs)
+{
+ return regs->sp;
+}
+
+#endif /* _ASM_KVX_PTRACE_H */
diff --git a/arch/kvx/include/asm/switch_to.h b/arch/kvx/include/asm/switch_to.h
new file mode 100644
index 000000000000..2b1fda06dea8
--- /dev/null
+++ b/arch/kvx/include/asm/switch_to.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SWITCH_TO_H
+#define _ASM_KVX_SWITCH_TO_H
+
+struct task_struct;
+
+/* context switching is now performed out-of-line in switch_to.S */
+extern struct task_struct *__switch_to(struct task_struct *prev,
+ struct task_struct *next);
+
+#define switch_to(prev, next, last) \
+ do { \
+ ((last) = __switch_to((prev), (next))); \
+ } while (0)
+
+#endif /* _ASM_KVX_SWITCH_TO_H */
diff --git a/arch/kvx/include/asm/thread_info.h b/arch/kvx/include/asm/thread_info.h
new file mode 100644
index 000000000000..4ce0154813ef
--- /dev/null
+++ b/arch/kvx/include/asm/thread_info.h
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef _ASM_KVX_THREAD_INFO_H
+#define _ASM_KVX_THREAD_INFO_H
+
+#include <asm/page.h>
+
+/*
+ * Size of the kernel stack for each process.
+ */
+#define THREAD_SIZE_ORDER 2
+#define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER)
+
+/*
+ * Thread information flags
+ * these are process state flags that various assembly files may need to
+ * access
+ * - pending work-to-be-done flags are in LSW
+ * - other flags in MSW
+ */
+#define TIF_SYSCALL_TRACE 0 /* syscall trace active */
+#define TIF_NOTIFY_RESUME 1 /* resumption notification requested */
+#define TIF_SIGPENDING 2 /* signal pending */
+#define TIF_NEED_RESCHED 3 /* rescheduling necessary */
+#define TIF_SINGLESTEP 4 /* restore singlestep on return to user mode */
+#define TIF_UPROBE 5
+#define TIF_SYSCALL_TRACEPOINT 6 /* syscall tracepoint instrumentation */
+#define TIF_SYSCALL_AUDIT 7 /* syscall auditing active */
+#define TIF_RESTORE_SIGMASK 9
+#define TIF_NOTIFY_SIGNAL 10 /* signal notifications exist */
+#define TIF_POLLING_NRFLAG 16 /* true if poll_idle() is polling TIF_NEED_RESCHED */
+#define TIF_MEMDIE 17
+
+#define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE)
+#define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT)
+#define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT)
+#define _TIF_POLLING_NRFLAG (1 << TIF_POLLING_NRFLAG)
+#define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME)
+#define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
+#define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
+#define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL)
+
+#define _TIF_WORK_MASK \
+ (_TIF_NOTIFY_RESUME | _TIF_SIGPENDING | _TIF_NEED_RESCHED)
+
+#define _TIF_SYSCALL_WORK \
+ (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_TRACEPOINT | _TIF_SYSCALL_AUDIT)
+
+#ifndef __ASSEMBLY__
+/*
+ * We are using THREAD_INFO_IN_TASK so this struct is almost useless
+ * please prefer adding fields in thread_struct (processor.h) rather
+ * than here.
+ * This struct is merely a remnant of distant times where it was placed
+ * on the stack to avoid large task_struct.
+ *
+ * cf https://lwn.net/Articles/700615/
+ */
+struct thread_info {
+ unsigned long flags; /* low level flags */
+ int preempt_count;
+#ifdef CONFIG_SMP
+ u32 cpu; /* current CPU */
+#endif
+};
+
+#define INIT_THREAD_INFO(tsk) \
+{ \
+ .flags = 0, \
+ .preempt_count = INIT_PREEMPT_COUNT, \
+}
+#endif /* __ASSEMBLY__*/
+#endif /* _ASM_KVX_THREAD_INFO_H */
diff --git a/arch/kvx/include/uapi/asm/ptrace.h b/arch/kvx/include/uapi/asm/ptrace.h
new file mode 100644
index 000000000000..f5febe830526
--- /dev/null
+++ b/arch/kvx/include/uapi/asm/ptrace.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ */
+
+#ifndef _UAPI_ASM_KVX_PTRACE_H
+#define _UAPI_ASM_KVX_PTRACE_H
+
+#include <linux/types.h>
+/*
+ * User-mode register state for core dumps, ptrace, sigcontext
+ *
+ * This decouples struct pt_regs from the userspace ABI.
+ * The struct pt_regs must start with the same layout as struct user_pt_regs.
+ */
+struct user_pt_regs {
+ /* GPR */
+ unsigned long r0;
+ unsigned long r1;
+ unsigned long r2;
+ unsigned long r3;
+ unsigned long r4;
+ unsigned long r5;
+ unsigned long r6;
+ unsigned long r7;
+ unsigned long r8;
+ unsigned long r9;
+ unsigned long r10;
+ unsigned long r11;
+ union {
+ unsigned long r12;
+ unsigned long sp;
+ };
+ union {
+ unsigned long r13;
+ unsigned long tp;
+ };
+ union {
+ unsigned long r14;
+ unsigned long fp;
+ };
+ unsigned long r15;
+ unsigned long r16;
+ unsigned long r17;
+ unsigned long r18;
+ unsigned long r19;
+ unsigned long r20;
+ unsigned long r21;
+ unsigned long r22;
+ unsigned long r23;
+ unsigned long r24;
+ unsigned long r25;
+ unsigned long r26;
+ unsigned long r27;
+ unsigned long r28;
+ unsigned long r29;
+ unsigned long r30;
+ unsigned long r31;
+ unsigned long r32;
+ unsigned long r33;
+ unsigned long r34;
+ unsigned long r35;
+ unsigned long r36;
+ unsigned long r37;
+ unsigned long r38;
+ unsigned long r39;
+ unsigned long r40;
+ unsigned long r41;
+ unsigned long r42;
+ unsigned long r43;
+ unsigned long r44;
+ unsigned long r45;
+ unsigned long r46;
+ unsigned long r47;
+ unsigned long r48;
+ unsigned long r49;
+ unsigned long r50;
+ unsigned long r51;
+ unsigned long r52;
+ unsigned long r53;
+ unsigned long r54;
+ unsigned long r55;
+ unsigned long r56;
+ unsigned long r57;
+ unsigned long r58;
+ unsigned long r59;
+ unsigned long r60;
+ unsigned long r61;
+ unsigned long r62;
+ unsigned long r63;
+
+ /* SFR */
+ unsigned long lc;
+ unsigned long le;
+ unsigned long ls;
+ unsigned long ra;
+
+ unsigned long cs;
+ unsigned long spc;
+};
+
+/* TCA registers structure exposed to user */
+struct user_tca_regs {
+ struct {
+ __u64 x;
+ __u64 y;
+ __u64 z;
+ __u64 t;
+ } regs[48];
+};
+
+#endif /* _UAPI_ASM_KVX_PTRACE_H */
diff --git a/arch/kvx/kernel/process.c b/arch/kvx/kernel/process.c
new file mode 100644
index 000000000000..f9ba2571e279
--- /dev/null
+++ b/arch/kvx/kernel/process.c
@@ -0,0 +1,212 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Marius Gligor
+ * Yann Sionneau
+ */
+
+#include <linux/elf.h>
+#include <linux/sched.h>
+#include <linux/ptrace.h>
+#include <linux/printk.h>
+#include <linux/sched/debug.h>
+#include <linux/sched/task_stack.h>
+#include <linux/hw_breakpoint.h>
+
+#include <asm/ptrace.h>
+#include <asm/processor.h>
+#include <asm/ptrace.h>
+#include <asm/uaccess.h>
+#include <asm/stacktrace.h>
+#include <asm/hw_breakpoint.h>
+
+#if defined(CONFIG_STACKPROTECTOR)
+#include <linux/stackprotector.h>
+unsigned long __stack_chk_guard __read_mostly;
+EXPORT_SYMBOL(__stack_chk_guard);
+#endif
+
+#define SCALL_NUM_EXIT "0xfff"
+
+void arch_cpu_idle(void)
+{
+ wait_for_interrupt();
+ local_irq_enable();
+}
+
+void show_regs(struct pt_regs *regs)
+{
+
+ int in_kernel = 1;
+ unsigned short i, reg_offset;
+ void *ptr;
+
+ show_regs_print_info(KERN_DEFAULT);
+
+ if (user_mode(regs))
+ in_kernel = 0;
+
+ pr_info("\nmode: %s\n"
+ " PC: %016llx PS: %016llx\n"
+ " CS: %016llx RA: %016llx\n"
+ " LS: %016llx LE: %016llx\n"
+ " LC: %016llx\n\n",
+ in_kernel ? "kernel" : "user",
+ regs->spc, regs->sps,
+ regs->cs, regs->ra, regs->ls, regs->le, regs->lc);
+
+ /* GPR */
+ ptr = regs;
+ ptr += offsetof(struct pt_regs, r0);
+ reg_offset = offsetof(struct pt_regs, r1) -
+ offsetof(struct pt_regs, r0);
+
+ /**
+ * Display all the 64 GPRs assuming they are ordered correctly
+ * in the pt_regs struct...
+ */
+ for (i = 0; i < GPR_COUNT; i += 2) {
+ pr_info(" R%d: %016llx R%d: %016llx\n",
+ i, *(uint64_t *)ptr,
+ i + 1, *(uint64_t *)(ptr + reg_offset));
+ ptr += reg_offset * 2;
+ }
+
+ pr_info("\n\n");
+}
+
+/**
+ * Prepare a thread to return to userspace
+ */
+void start_thread(struct pt_regs *regs,
+ unsigned long pc, unsigned long sp)
+{
+ /* Remove MMUP bit (user is not privilege in current virtual space) */
+ u64 clear_bit = KVX_SFR_PS_MMUP_MASK | KVX_SFR_PS_SME_MASK |
+ KVX_SFR_PS_SMR_MASK;
+ regs->spc = pc;
+ regs->sp = sp;
+ regs->sps = kvx_sfr_get(PS);
+
+ regs->sps &= ~clear_bit;
+
+ /* Set privilege level to +1 (relative) */
+ regs->sps &= ~KVX_SFR_PS_PL_MASK;
+ regs->sps |= (1 << KVX_SFR_PS_PL_SHIFT);
+}
+
+int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
+{
+ struct pt_regs *regs, *childregs = task_pt_regs(p);
+ unsigned long clone_flags = args->flags;
+ unsigned long usp = args->stack;
+ unsigned long tls = args->tls;
+
+ /* p->thread holds context to be restored by __switch_to() */
+ if (unlikely(args->fn)) {
+ /* Kernel thread */
+ memset(childregs, 0, sizeof(struct pt_regs));
+
+ p->thread.ctx_switch.r20 = (uint64_t)args->fn; /* fn */
+ p->thread.ctx_switch.r21 = (uint64_t)args->fn_arg;
+ p->thread.ctx_switch.ra =
+ (unsigned long) ret_from_kernel_thread;
+ } else {
+ regs = current_pt_regs();
+
+ /* Copy current process registers */
+ *childregs = *regs;
+
+ /* Store tracing status in r20 to avoid computing it
+ * in assembly
+ */
+ p->thread.ctx_switch.r20 =
+ task_thread_info(p)->flags & _TIF_SYSCALL_WORK;
+ p->thread.ctx_switch.ra = (unsigned long) ret_from_fork;
+
+ childregs->r0 = 0; /* Return value of fork() */
+ /* Set stack pointer if any */
+ if (usp)
+ childregs->sp = usp;
+
+ /* Set a new TLS ? */
+ if (clone_flags & CLONE_SETTLS)
+ childregs->r13 = tls;
+ }
+ p->thread.kernel_sp =
+ (unsigned long) (task_stack_page(p) + THREAD_SIZE);
+ p->thread.ctx_switch.sp = (unsigned long) childregs;
+
+ clear_ptrace_hw_breakpoint(p);
+
+ return 0;
+}
+
+void release_thread(struct task_struct *dead_task)
+{
+}
+
+void flush_thread(void)
+{
+ /* This function should clear the values of the floating point
+ * registers and debug registers saved in the TSS segment.
+ */
+
+ flush_ptrace_hw_breakpoint(current);
+}
+
+/* Fill in the fpu structure for a core dump. */
+int dump_fpu(struct pt_regs *regs, elf_fpregset_t *fpu)
+{
+ /*
+ * On kvx, FPU uses standard registers + $cs which is a common register
+ * also needed for non-fpu execution, so there is no additional
+ * register to dump.
+ */
+ return 0;
+}
+
+static bool find_wchan(unsigned long pc, void *arg)
+{
+ unsigned long *p = arg;
+
+ /*
+ * If the pc is in a scheduler function (waiting), then, this is the
+ * address where the process is currently stuck. Note that scheduler
+ * functions also include lock functions. This functions are
+ * materialized using annotation to put them is special text sections.
+ */
+ if (!in_sched_functions(pc)) {
+ *p = pc;
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * __get_wchan is called to obtain "schedule()" caller function address.
+ */
+unsigned long __get_wchan(struct task_struct *p)
+{
+ unsigned long pc = 0;
+ struct stackframe frame;
+
+ /*
+ * We need to obtain the task stack since we don't want the stack to
+ * move under our feet.
+ */
+ if (!try_get_task_stack(p))
+ return 0;
+
+ start_stackframe(&frame, thread_saved_reg(p, fp),
+ thread_saved_reg(p, ra));
+ walk_stackframe(p, &frame, find_wchan, &pc);
+
+ put_task_stack(p);
+
+ return pc;
+}
+
diff --git a/arch/kvx/kernel/ptrace.c b/arch/kvx/kernel/ptrace.c
new file mode 100644
index 000000000000..46096a3ab3e4
--- /dev/null
+++ b/arch/kvx/kernel/ptrace.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/riscv/kernel/ptrace.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Marius Gligor
+ * Clement Leger
+ */
+
+#include <linux/sched.h>
+#include <linux/sched.h>
+#include <linux/audit.h>
+#include <linux/irqflags.h>
+#include <linux/thread_info.h>
+#include <linux/context_tracking.h>
+#include <linux/uaccess.h>
+#include <linux/syscalls.h>
+#include <linux/signal.h>
+#include <linux/regset.h>
+#include <linux/hw_breakpoint.h>
+
+#include <asm/dame.h>
+#include <asm/ptrace.h>
+#include <asm/syscall.h>
+#include <asm/break_hook.h>
+#include <asm/debug.h>
+#include <asm/cacheflush.h>
+#include <asm/hw_breakpoint.h>
+
+#define CREATE_TRACE_POINTS
+#include <trace/events/syscalls.h>
+
+#define HW_PT_CMD_GET_CAPS 0
+#define HW_PT_CMD_GET_PT 1
+#define HW_PT_CMD_SET_RESERVE 0
+#define HW_PT_CMD_SET_ENABLE 1
+
+#define FROM_GDB_CMD_MASK 3
+#define FROM_GDB_HP_TYPE_SHIFT 2
+#define FROM_GDB_HP_TYPE_MASK 4
+#define FROM_GDB_WP_TYPE_SHIFT 3
+#define FROM_GDB_WP_TYPE_MASK 0x18
+#define FROM_GDB_HP_IDX_SHIFT 5
+
+#define hw_pt_cmd(addr) ((addr) & FROM_GDB_CMD_MASK)
+#define hw_pt_is_bkp(addr) ((((addr) & FROM_GDB_HP_TYPE_MASK) >> \
+ FROM_GDB_HP_TYPE_SHIFT) == KVX_HW_BREAKPOINT_TYPE)
+#define get_hw_pt_wp_type(addr) ((((addr) & FROM_GDB_WP_TYPE_MASK)) >> \
+ FROM_GDB_WP_TYPE_SHIFT)
+#define get_hw_pt_idx(addr) ((addr) >> FROM_GDB_HP_IDX_SHIFT)
+#define get_hw_pt_addr(data) ((data)[0])
+#define get_hw_pt_len(data) ((data)[1] >> 1)
+#define hw_pt_is_enabled(data) ((data)[1] & 1)
+
+enum kvx_regset {
+ REGSET_GPR,
+#ifdef CONFIG_ENABLE_TCA
+ REGSET_TCA,
+#endif
+};
+
+static void compute_ptrace_hw_pt_rsp(uint64_t *data, struct perf_event *bp)
+{
+ data[0] = bp->attr.bp_addr;
+ data[1] = bp->attr.bp_len >> 1;
+ if (!bp->attr.disabled)
+ data[1] |= 1;
+}
+
+void ptrace_disable(struct task_struct *child)
+{
+ /* nothing to do */
+}
+
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+static void ptrace_hw_pt_triggered(struct perf_event *bp,
+ struct perf_sample_data *data,
+ struct pt_regs *regs)
+{
+ int i, id = 0;
+ struct arch_hw_breakpoint *bkpt = counter_arch_bp(bp);
+
+ if (bp->attr.bp_type & HW_BREAKPOINT_X) {
+ id = KVX_HW_BREAKPOINT_TYPE;
+ for (i = 0; i < KVX_HW_BREAKPOINT_COUNT; i++)
+ if (current->thread.debug.ptrace_hbp[i] == bp)
+ break;
+ } else {
+ id = KVX_HW_WATCHPOINT_TYPE;
+ for (i = 0; i < KVX_HW_WATCHPOINT_COUNT; i++)
+ if (current->thread.debug.ptrace_hwp[i] == bp)
+ break;
+ }
+
+ id |= i << 1;
+ force_sig_ptrace_errno_trap(id, (void __user *) bkpt->addr);
+}
+
+static struct perf_event *ptrace_hw_pt_create(struct task_struct *tsk, int type)
+{
+ struct perf_event_attr attr;
+
+ ptrace_breakpoint_init(&attr);
+
+ /* Initialise fields to sane defaults. */
+ attr.bp_addr = 0;
+ attr.bp_len = 1;
+ attr.bp_type = type;
+ attr.disabled = 1;
+
+ return register_user_hw_breakpoint(&attr, ptrace_hw_pt_triggered, NULL,
+ tsk);
+}
+
+/*
+ * Address bit 0..1: command id, bit 2: hardware breakpoint (0)
+ * or watchpoint (1), bits 63..3: register number.
+ * Both PTRACE_GET_HW_PT_REGS and PTRACE_SET_HW_PT_REGS transfer two
+ * 64-bit words: for get capabilities: number of breakpoint (0) and
+ * watchpoints (1), for hardware watchpoint/breakpoint enable: address (0)
+ * and enable + length (1)
+ */
+
+static long ptrace_get_hw_pt_pregs(struct task_struct *child, long addr,
+ long __user *datap)
+{
+ struct perf_event *bp;
+ u64 user_data[2];
+ int cmd = hw_pt_cmd(addr);
+
+ if (cmd == HW_PT_CMD_GET_CAPS) {
+ user_data[0] = KVX_HW_BREAKPOINT_COUNT;
+ user_data[1] = KVX_HW_WATCHPOINT_COUNT;
+ } else if (cmd == HW_PT_CMD_GET_PT) {
+ int is_breakpoint = hw_pt_is_bkp(addr);
+ int idx = get_hw_pt_idx(addr);
+
+ if ((is_breakpoint && idx >= KVX_HW_BREAKPOINT_COUNT) ||
+ (!is_breakpoint && idx >= KVX_HW_WATCHPOINT_COUNT))
+ return -EINVAL;
+
+ if (is_breakpoint)
+ bp = child->thread.debug.ptrace_hbp[idx];
+ else
+ bp = child->thread.debug.ptrace_hwp[idx];
+
+ if (bp) {
+ compute_ptrace_hw_pt_rsp(user_data, bp);
+ } else {
+ user_data[0] = 0;
+ user_data[1] = 0;
+ }
+ } else {
+ return -EINVAL;
+ }
+
+ if (copy_to_user(datap, user_data, sizeof(user_data)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static long ptrace_set_hw_pt_regs(struct task_struct *child, long addr,
+ long __user *datap)
+{
+ struct perf_event *bp;
+ struct perf_event_attr attr;
+ u64 user_data[2];
+ int is_breakpoint = hw_pt_is_bkp(addr);
+ int idx = get_hw_pt_idx(addr);
+ int cmd = hw_pt_cmd(addr);
+ int bp_type, ret;
+
+ if ((is_breakpoint && idx >= KVX_HW_BREAKPOINT_COUNT) ||
+ (!is_breakpoint && idx >= KVX_HW_WATCHPOINT_COUNT))
+ return -EINVAL;
+
+ if (copy_from_user(user_data, datap, sizeof(user_data)))
+ return -EFAULT;
+
+ if (cmd == HW_PT_CMD_SET_RESERVE ||
+ (cmd == HW_PT_CMD_SET_ENABLE && hw_pt_is_enabled(user_data))) {
+ if (is_breakpoint)
+ ret = ptrace_request_hw_breakpoint(idx);
+ else
+ ret = ptrace_request_hw_watchpoint(idx);
+
+ if (cmd == HW_PT_CMD_SET_RESERVE || ret != 0)
+ return ret;
+ }
+
+ if (cmd != HW_PT_CMD_SET_ENABLE)
+ return -EINVAL;
+
+ if (is_breakpoint) {
+ bp = child->thread.debug.ptrace_hbp[idx];
+ bp_type = HW_BREAKPOINT_X;
+ } else {
+ bp = child->thread.debug.ptrace_hwp[idx];
+ bp_type = get_hw_pt_wp_type(addr);
+ if (!bp_type)
+ bp_type = HW_BREAKPOINT_W;
+ }
+
+ if (!bp) {
+ bp = ptrace_hw_pt_create(child, bp_type);
+ if (IS_ERR(bp))
+ return PTR_ERR(bp);
+ if (is_breakpoint)
+ child->thread.debug.ptrace_hbp[idx] = bp;
+ else
+ child->thread.debug.ptrace_hwp[idx] = bp;
+ }
+
+ attr = bp->attr;
+ attr.bp_addr = get_hw_pt_addr(user_data);
+ attr.bp_len = get_hw_pt_len(user_data);
+ attr.bp_type = bp_type;
+ attr.disabled = !hw_pt_is_enabled(user_data);
+
+ return modify_user_hw_breakpoint(bp, &attr);
+}
+#endif
+
+static int kvx_gpr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ struct membuf to)
+{
+ struct user_pt_regs *regs = &task_pt_regs(target)->user_regs;
+
+ return membuf_write(&to, regs, sizeof(*regs));
+}
+
+static int kvx_gpr_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ struct user_pt_regs *regs = &task_pt_regs(target)->user_regs;
+
+ return user_regset_copyin(&pos, &count, &kbuf, &ubuf, regs, 0, -1);
+}
+
+#ifdef CONFIG_ENABLE_TCA
+static int kvx_tca_reg_get(struct task_struct *target,
+ const struct user_regset *regset,
+ struct membuf to)
+{
+ struct ctx_switch_regs *ctx_regs = &target->thread.ctx_switch;
+ struct tca_reg *regs = ctx_regs->tca_regs;
+ int ret;
+
+ if (!ctx_regs->tca_regs_saved)
+ ret = membuf_zero(&to, sizeof(*regs));
+ else
+ ret = membuf_write(&to, regs, sizeof(*regs));
+
+ return ret;
+}
+
+static int kvx_tca_reg_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ struct ctx_switch_regs *ctx_regs = &target->thread.ctx_switch;
+ struct tca_reg *regs = ctx_regs->tca_regs;
+ int ret;
+
+ if (!ctx_regs->tca_regs_saved)
+ ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
+ 0, -1);
+ else
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, regs,
+ 0, -1);
+
+ return ret;
+}
+#endif
+
+static const struct user_regset kvx_user_regset[] = {
+ [REGSET_GPR] = {
+ .core_note_type = NT_PRSTATUS,
+ .n = ELF_NGREG,
+ .size = sizeof(elf_greg_t),
+ .align = sizeof(elf_greg_t),
+ .regset_get = &kvx_gpr_get,
+ .set = &kvx_gpr_set,
+ },
+#ifdef CONFIG_ENABLE_TCA
+ [REGSET_TCA] = {
+ .core_note_type = NT_KVX_TCA,
+ .n = TCA_REG_COUNT,
+ .size = sizeof(struct tca_reg),
+ .align = sizeof(struct tca_reg),
+ .regset_get = &kvx_tca_reg_get,
+ .set = &kvx_tca_reg_set,
+ },
+#endif
+};
+
+static const struct user_regset_view user_kvx_view = {
+ .name = "kvx",
+ .e_machine = EM_KVX,
+ .regsets = kvx_user_regset,
+ .n = ARRAY_SIZE(kvx_user_regset)
+};
+
+const struct user_regset_view *task_user_regset_view(struct task_struct *task)
+{
+ return &user_kvx_view;
+}
+
+long arch_ptrace(struct task_struct *child, long request,
+ unsigned long addr, unsigned long data)
+{
+ int ret;
+ unsigned long __user *datap = (unsigned long __user *) data;
+
+ switch (request) {
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+ case PTRACE_GET_HW_PT_REGS:
+ ret = ptrace_get_hw_pt_pregs(child, addr, datap);
+ break;
+ case PTRACE_SET_HW_PT_REGS:
+ ret = ptrace_set_hw_pt_regs(child, addr, datap);
+ break;
+#endif
+ default:
+ ret = ptrace_request(child, request, addr, data);
+ break;
+ }
+
+ return ret;
+}
+
+/*
+ * Allows PTRACE_SYSCALL to work. These are called from entry.S in
+ * {handle,ret_from}_syscall.
+ */
+int do_syscall_trace_enter(struct pt_regs *regs, unsigned long syscall)
+{
+ int ret = 0;
+
+#ifdef CONFIG_CONTEXT_TRACKING_USER
+ user_exit_callable();
+#endif
+ if (test_thread_flag(TIF_SYSCALL_TRACE))
+ ret = ptrace_report_syscall_entry(regs);
+
+#ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS
+ if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
+ trace_sys_enter(regs, syscall_get_nr(current, regs));
+#endif
+
+ audit_syscall_entry(syscall, regs->r0, regs->r1, regs->r2, regs->r3);
+
+ return ret;
+}
+
+void do_syscall_trace_exit(struct pt_regs *regs)
+{
+ if (test_thread_flag(TIF_SYSCALL_TRACE))
+ ptrace_report_syscall_exit(regs, 0);
+
+ audit_syscall_exit(regs);
+
+#ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS
+ if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
+ trace_sys_exit(regs, regs_return_value(regs));
+#endif
+
+#ifdef CONFIG_CONTEXT_TRACKING_USER
+ user_enter_callable();
+#endif
+}
+
+static int kvx_bkpt_handler(struct break_hook *brk_hook, struct pt_regs *regs)
+{
+ /* Unexpected breakpoint */
+ if (!(current->ptrace & PT_PTRACED))
+ return BREAK_HOOK_ERROR;
+
+ /* deliver the signal to userspace */
+ force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *) regs->spc);
+
+ return BREAK_HOOK_HANDLED;
+}
+
+static void kvx_stepi(struct pt_regs *regs)
+{
+ /* deliver the signal to userspace */
+ force_sig_fault(SIGTRAP, TRAP_TRACE, (void __user *) regs->spc);
+}
+
+void user_enable_single_step(struct task_struct *child)
+{
+ struct pt_regs *regs = task_pt_regs(child);
+
+ enable_single_step(regs);
+}
+
+void user_disable_single_step(struct task_struct *child)
+{
+ struct pt_regs *regs = task_pt_regs(child);
+
+ disable_single_step(regs);
+}
+
+/**
+ * Main debug handler called by the _debug_handler routine in entry.S
+ * This handler will perform the required action
+ * @es: Exception Syndrome register value
+ * @ea: Exception Address register
+ * @regs: pointer to registers saved when enter debug
+ */
+int ptrace_debug_handler(u64 ea, struct pt_regs *regs)
+{
+ int debug_cause = debug_dc(regs->es);
+
+ switch (debug_cause) {
+ case DEBUG_CAUSE_STEPI:
+ if (check_hw_watchpoint_stepped(regs))
+ user_disable_single_step(current);
+ else
+ kvx_stepi(regs);
+ break;
+ case DEBUG_CAUSE_BREAKPOINT:
+ check_hw_breakpoint(regs);
+ break;
+ case DEBUG_CAUSE_WATCHPOINT:
+ if (check_hw_watchpoint(regs, ea))
+ user_enable_single_step(current);
+ break;
+ default:
+ break;
+ }
+
+ return DEBUG_HOOK_HANDLED;
+}
+
+static struct debug_hook ptrace_debug_hook = {
+ .handler = ptrace_debug_handler,
+ .mode = MODE_USER,
+};
+
+static struct break_hook bkpt_break_hook = {
+ .id = BREAK_CAUSE_BKPT,
+ .handler = kvx_bkpt_handler,
+ .mode = MODE_USER,
+};
+
+static int __init arch_init_breakpoint(void)
+{
+ break_hook_register(&bkpt_break_hook);
+ debug_hook_register(&ptrace_debug_hook);
+
+ return 0;
+}
+
+postcore_initcall(arch_init_breakpoint);
diff --git a/arch/kvx/kernel/stacktrace.c b/arch/kvx/kernel/stacktrace.c
new file mode 100644
index 000000000000..85d52ba2d082
--- /dev/null
+++ b/arch/kvx/kernel/stacktrace.c
@@ -0,0 +1,173 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Vincent Chardon
+ */
+
+#include <linux/context_tracking.h>
+#include <linux/kallsyms.h>
+#include <linux/printk.h>
+#include <linux/init.h>
+
+#include <asm/stacktrace.h>
+#include <asm/ptrace.h>
+
+#define STACK_SLOT_PER_LINE 4
+#define STACK_MAX_SLOT_PRINT (STACK_SLOT_PER_LINE * 8)
+
+static int notrace unwind_frame(struct task_struct *task,
+ struct stackframe *frame)
+{
+ unsigned long fp = frame->fp;
+
+ /* Frame pointer must be aligned on 8 bytes */
+ if (fp & 0x7)
+ return -EINVAL;
+
+ if (!task)
+ task = current;
+
+ if (!on_task_stack(task, fp))
+ return -EINVAL;
+
+ frame->fp = READ_ONCE_NOCHECK(*(unsigned long *)(fp));
+ frame->ra = READ_ONCE_NOCHECK(*(unsigned long *)(fp + 8));
+
+ /*
+ * When starting, we set the frame pointer to 0, hence end of
+ * frame linked list is signal by that
+ */
+ if (!frame->fp)
+ return -EINVAL;
+
+ return 0;
+}
+
+void notrace walk_stackframe(struct task_struct *task, struct stackframe *frame,
+ bool (*fn)(unsigned long, void *), void *arg)
+{
+ unsigned long addr;
+ int ret;
+
+ while (1) {
+ addr = frame->ra;
+
+ if (fn(addr, arg))
+ break;
+
+ ret = unwind_frame(task, frame);
+ if (ret)
+ break;
+ }
+}
+
+#ifdef CONFIG_STACKTRACE
+bool append_stack_addr(unsigned long pc, void *arg)
+{
+ struct stack_trace *trace;
+
+ trace = (struct stack_trace *)arg;
+ if (trace->skip == 0) {
+ trace->entries[trace->nr_entries++] = pc;
+ if (trace->nr_entries == trace->max_entries)
+ return true;
+ } else {
+ trace->skip--;
+ }
+
+ return false;
+}
+
+/*
+ * Save stack-backtrace addresses into a stack_trace buffer.
+ */
+void save_stack_trace(struct stack_trace *trace)
+{
+ struct stackframe frame;
+
+ trace->nr_entries = 0;
+ /* We want to skip this function and the caller */
+ trace->skip += 2;
+
+ start_stackframe(&frame, (unsigned long) __builtin_frame_address(0),
+ (unsigned long) save_stack_trace);
+ walk_stackframe(current, &frame, append_stack_addr, trace);
+}
+EXPORT_SYMBOL(save_stack_trace);
+#endif /* CONFIG_STACKTRACE */
+
+static bool print_pc(unsigned long pc, void *arg)
+{
+ unsigned long *skip = arg;
+
+ if (*skip == 0)
+ print_ip_sym(KERN_INFO, pc);
+ else
+ (*skip)--;
+
+ return false;
+}
+
+void show_stacktrace(struct task_struct *task, struct pt_regs *regs)
+{
+ struct stackframe frame;
+ unsigned long skip = 0;
+
+ /* Obviously, we can't backtrace on usermode ! */
+ if (regs && user_mode(regs))
+ return;
+
+ if (!task)
+ task = current;
+
+ if (!try_get_task_stack(task))
+ return;
+
+ if (regs) {
+ start_stackframe(&frame, regs->fp, regs->spc);
+ } else if (task == current) {
+ /* Skip current function and caller */
+ skip = 2;
+ start_stackframe(&frame,
+ (unsigned long) __builtin_frame_address(0),
+ (unsigned long) show_stacktrace);
+ } else {
+ /* task blocked in __switch_to */
+ start_stackframe(&frame,
+ thread_saved_reg(task, fp),
+ thread_saved_reg(task, ra));
+ }
+
+ pr_info("Call Trace:\n");
+ walk_stackframe(task, &frame, print_pc, &skip);
+
+ put_task_stack(task);
+}
+
+/*
+ * If show_stack is called with a non-null task, then the task will have been
+ * claimed with try_get_task_stack by the caller. If task is NULL or current
+ * then there is no need to get task stack since it's our current stack...
+ */
+void show_stack(struct task_struct *task, unsigned long *sp)
+{
+ int i = 0;
+
+ if (!sp)
+ sp = (unsigned long *) get_current_sp();
+
+ pr_info("Stack dump (@%p):\n", sp);
+ for (i = 0; i < STACK_MAX_SLOT_PRINT; i++) {
+ if (kstack_end(sp))
+ break;
+
+ if (i && (i % STACK_SLOT_PER_LINE) == 0)
+ pr_cont("\n\t");
+
+ pr_cont("%016lx ", *sp++);
+ }
+ pr_cont("\n");
+
+ show_stacktrace(task, NULL);
+}
--
2.37.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 113+ messages in thread
* [RFC PATCH 11/25] kvx: Add memory management
2023-01-03 16:43 ` Yann Sionneau
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Will Deacon, Aneesh Kumar K.V, Andrew Morton,
Nick Piggin, Peter Zijlstra, Paul Walmsley, Palmer Dabbelt,
Albert Ou, linux-kernel, linux-arch, linux-mm, linux-riscv,
Clement Leger, Guillaume Thouvenin, Jean-Christophe Pince,
Jules Maselbas, Julian Vetter, Julien Hascoet, Louis Morhet,
Marc Poulhiès, Marius Gligor, Vincent Chardon
Add memory management support for kvx, including: cache and tlb
management, page fault handling, ioremap/mmap and streaming dma support.
CC: Will Deacon <will@kernel.org>
CC: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: Nick Piggin <npiggin@gmail.com>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Paul Walmsley <paul.walmsley@sifive.com>
CC: Palmer Dabbelt <palmer@dabbelt.com>
CC: Albert Ou <aou@eecs.berkeley.edu>
CC: linux-kernel@vger.kernel.org
CC: linux-arch@vger.kernel.org
CC: linux-mm@kvack.org
CC: linux-riscv@lists.infradead.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Jean-Christophe Pince <jcpince@gmail.com>
Signed-off-by: Jean-Christophe Pince <jcpince@gmail.com>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Julien Hascoet <jhascoet@kalray.eu>
Signed-off-by: Julien Hascoet <jhascoet@kalray.eu>
Co-developed-by: Louis Morhet <lmorhet@kalray.eu>
Signed-off-by: Louis Morhet <lmorhet@kalray.eu>
Co-developed-by: Marc Poulhiès <dkm@kataplop.net>
Signed-off-by: Marc Poulhiès <dkm@kataplop.net>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/cache.h | 46 +++
arch/kvx/include/asm/cacheflush.h | 181 +++++++++
arch/kvx/include/asm/fixmap.h | 47 +++
arch/kvx/include/asm/hugetlb.h | 36 ++
arch/kvx/include/asm/l2_cache.h | 75 ++++
arch/kvx/include/asm/l2_cache_defs.h | 64 ++++
arch/kvx/include/asm/mem_map.h | 44 +++
arch/kvx/include/asm/mmu.h | 296 +++++++++++++++
arch/kvx/include/asm/mmu_context.h | 156 ++++++++
arch/kvx/include/asm/mmu_stats.h | 38 ++
arch/kvx/include/asm/page.h | 187 ++++++++++
arch/kvx/include/asm/page_size.h | 29 ++
arch/kvx/include/asm/pgalloc.h | 101 +++++
arch/kvx/include/asm/pgtable-bits.h | 102 ++++++
arch/kvx/include/asm/pgtable.h | 451 +++++++++++++++++++++++
arch/kvx/include/asm/rm_fw.h | 16 +
arch/kvx/include/asm/sparsemem.h | 15 +
arch/kvx/include/asm/symbols.h | 16 +
arch/kvx/include/asm/tlb.h | 24 ++
arch/kvx/include/asm/tlb_defs.h | 131 +++++++
arch/kvx/include/asm/tlbflush.h | 58 +++
arch/kvx/include/asm/vmalloc.h | 10 +
arch/kvx/mm/cacheflush.c | 154 ++++++++
arch/kvx/mm/dma-mapping.c | 95 +++++
arch/kvx/mm/extable.c | 24 ++
arch/kvx/mm/fault.c | 264 ++++++++++++++
arch/kvx/mm/hugetlbpage.c | 317 ++++++++++++++++
arch/kvx/mm/init.c | 527 +++++++++++++++++++++++++++
arch/kvx/mm/kernel_rwx.c | 228 ++++++++++++
arch/kvx/mm/mmap.c | 31 ++
arch/kvx/mm/mmu.c | 204 +++++++++++
arch/kvx/mm/mmu_stats.c | 94 +++++
arch/kvx/mm/tlb.c | 433 ++++++++++++++++++++++
33 files changed, 4494 insertions(+)
create mode 100644 arch/kvx/include/asm/cache.h
create mode 100644 arch/kvx/include/asm/cacheflush.h
create mode 100644 arch/kvx/include/asm/fixmap.h
create mode 100644 arch/kvx/include/asm/hugetlb.h
create mode 100644 arch/kvx/include/asm/l2_cache.h
create mode 100644 arch/kvx/include/asm/l2_cache_defs.h
create mode 100644 arch/kvx/include/asm/mem_map.h
create mode 100644 arch/kvx/include/asm/mmu.h
create mode 100644 arch/kvx/include/asm/mmu_context.h
create mode 100644 arch/kvx/include/asm/mmu_stats.h
create mode 100644 arch/kvx/include/asm/page.h
create mode 100644 arch/kvx/include/asm/page_size.h
create mode 100644 arch/kvx/include/asm/pgalloc.h
create mode 100644 arch/kvx/include/asm/pgtable-bits.h
create mode 100644 arch/kvx/include/asm/pgtable.h
create mode 100644 arch/kvx/include/asm/rm_fw.h
create mode 100644 arch/kvx/include/asm/sparsemem.h
create mode 100644 arch/kvx/include/asm/symbols.h
create mode 100644 arch/kvx/include/asm/tlb.h
create mode 100644 arch/kvx/include/asm/tlb_defs.h
create mode 100644 arch/kvx/include/asm/tlbflush.h
create mode 100644 arch/kvx/include/asm/vmalloc.h
create mode 100644 arch/kvx/mm/cacheflush.c
create mode 100644 arch/kvx/mm/dma-mapping.c
create mode 100644 arch/kvx/mm/extable.c
create mode 100644 arch/kvx/mm/fault.c
create mode 100644 arch/kvx/mm/hugetlbpage.c
create mode 100644 arch/kvx/mm/init.c
create mode 100644 arch/kvx/mm/kernel_rwx.c
create mode 100644 arch/kvx/mm/mmap.c
create mode 100644 arch/kvx/mm/mmu.c
create mode 100644 arch/kvx/mm/mmu_stats.c
create mode 100644 arch/kvx/mm/tlb.c
diff --git a/arch/kvx/include/asm/cache.h b/arch/kvx/include/asm/cache.h
new file mode 100644
index 000000000000..c53599cb6a6f
--- /dev/null
+++ b/arch/kvx/include/asm/cache.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_CACHE_H
+#define _ASM_KVX_CACHE_H
+
+/**
+ * On kvx I$ and D$ have the same size (16KB).
+ * Caches are 16K bytes big, VIPT 4-way set associative, true LRU, with 64-byte
+ * lines. The D$ is also write-through.
+ */
+#define KVX_ICACHE_WAY_COUNT 4
+#define KVX_ICACHE_SET_COUNT 64
+#define KVX_ICACHE_LINE_SHIFT 6
+#define KVX_ICACHE_LINE_SIZE (1 << KVX_ICACHE_LINE_SHIFT)
+#define KVX_ICACHE_SIZE \
+ (KVX_ICACHE_WAY_COUNT * KVX_ICACHE_SET_COUNT * KVX_ICACHE_LINE_SIZE)
+
+/**
+ * Invalidate the whole I-cache if the size to flush is more than this value
+ */
+#define KVX_ICACHE_INVAL_SIZE (KVX_ICACHE_SIZE)
+
+/* D-Cache */
+#define KVX_DCACHE_WAY_COUNT 4
+#define KVX_DCACHE_SET_COUNT 64
+#define KVX_DCACHE_LINE_SHIFT 6
+#define KVX_DCACHE_LINE_SIZE (1 << KVX_DCACHE_LINE_SHIFT)
+#define KVX_DCACHE_SIZE \
+ (KVX_DCACHE_WAY_COUNT * KVX_DCACHE_SET_COUNT * KVX_DCACHE_LINE_SIZE)
+
+/**
+ * Same for I-cache
+ */
+#define KVX_DCACHE_INVAL_SIZE (KVX_DCACHE_SIZE)
+
+#define L1_CACHE_SHIFT KVX_DCACHE_LINE_SHIFT
+#define L1_CACHE_BYTES KVX_DCACHE_LINE_SIZE
+
+#define L2_CACHE_LINE_SIZE 256
+#define L2_CACHE_LINE_MASK (L2_CACHE_LINE_SIZE - 1)
+
+#endif /* _ASM_KVX_CACHE_H */
diff --git a/arch/kvx/include/asm/cacheflush.h b/arch/kvx/include/asm/cacheflush.h
new file mode 100644
index 000000000000..f501e9956958
--- /dev/null
+++ b/arch/kvx/include/asm/cacheflush.h
@@ -0,0 +1,181 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ * Guillaume Thouvenin
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_CACHEFLUSH_H
+#define _ASM_KVX_CACHEFLUSH_H
+
+#include <linux/mm.h>
+#include <linux/io.h>
+
+#include <asm/l2_cache.h>
+
+#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
+
+#define flush_cache_mm(mm) do { } while (0)
+#define flush_cache_range(vma, start, end) do { } while (0)
+#define flush_cache_dup_mm(mm) do { } while (0)
+#define flush_cache_page(vma, vmaddr, pfn) do { } while (0)
+
+#define flush_cache_vmap(start, end) do { } while (0)
+#define flush_cache_vunmap(start, end) do { } while (0)
+
+#define flush_dcache_page(page) do { } while (0)
+
+#define flush_dcache_mmap_lock(mapping) do { } while (0)
+#define flush_dcache_mmap_unlock(mapping) do { } while (0)
+
+#define l1_inval_dcache_all __builtin_kvx_dinval
+#define kvx_fence __builtin_kvx_fence
+#define l1_inval_icache_all __builtin_kvx_iinval
+
+int dcache_wb_inval_virt_range(unsigned long vaddr, unsigned long len, bool wb,
+ bool inval);
+void dcache_wb_inval_phys_range(phys_addr_t addr, unsigned long len, bool wb,
+ bool inval);
+
+/*
+ * In the following functions, if L2$ is enabled, line invalidations
+ * in the L2$ will also trigger (via hw) invalidations in L1D caches of the
+ * same Cluster in case the lines are present in such L1D caches.
+ */
+
+/*
+ * L1 is indexed by virtual addresses and as such, invalidation takes virtual
+ * addresses as arguments.
+ */
+static inline
+void l1_inval_dcache_range(unsigned long vaddr, unsigned long size)
+{
+ unsigned long end = vaddr + size;
+
+ /* Then inval L1 */
+ if (size >= KVX_DCACHE_INVAL_SIZE) {
+ __builtin_kvx_dinval();
+ return;
+ }
+
+ vaddr = ALIGN_DOWN(vaddr, KVX_DCACHE_LINE_SIZE);
+ for (; vaddr < end; vaddr += KVX_DCACHE_LINE_SIZE)
+ __builtin_kvx_dinvall((void *) vaddr);
+}
+
+static inline
+void inval_dcache_range(phys_addr_t paddr, unsigned long size)
+{
+ /*
+ * Inval L2 first to avoid refilling from cached L2 values.
+ * If L2 cache is not enabled, it will return false and hence, we will
+ * fall back on L1 invalidation.
+ */
+ if (!l2_cache_inval_range(paddr, size))
+ l1_inval_dcache_range((unsigned long) phys_to_virt(paddr),
+ size);
+}
+
+static inline
+void wb_dcache_range(phys_addr_t paddr, unsigned long size)
+{
+ /* Fence to ensure all write are committed */
+ kvx_fence();
+
+ l2_cache_wb_range(paddr, size);
+}
+
+static inline
+void wbinval_dcache_range(phys_addr_t paddr, unsigned long size)
+{
+ /* Fence to ensure all write are committed */
+ kvx_fence();
+
+ if (!l2_cache_wbinval_range(paddr, size))
+ l1_inval_dcache_range((unsigned long) phys_to_virt(paddr),
+ size);
+}
+
+static inline
+void l1_inval_icache_range(unsigned long start, unsigned long end)
+{
+ unsigned long addr;
+ unsigned long size = end - start;
+
+ if (size >= KVX_ICACHE_INVAL_SIZE) {
+ __builtin_kvx_iinval();
+ __builtin_kvx_barrier();
+ return;
+ }
+
+ start = ALIGN_DOWN(start, KVX_ICACHE_LINE_SIZE);
+ for (addr = start; addr < end; addr += KVX_ICACHE_LINE_SIZE)
+ __builtin_kvx_iinvals((void *) addr);
+
+ __builtin_kvx_barrier();
+}
+
+static inline
+void wbinval_icache_range(phys_addr_t paddr, unsigned long size)
+{
+ unsigned long vaddr = (unsigned long) phys_to_virt(paddr);
+
+ /* Fence to ensure all write are committed */
+ kvx_fence();
+
+ l2_cache_wbinval_range(paddr, size);
+ /* invalidating l2 cache will invalidate l1 dcache
+ * but not l1 icache
+ */
+ l1_inval_icache_range(vaddr, vaddr + size);
+}
+
+static inline
+void sync_dcache_icache(unsigned long start, unsigned long end)
+{
+ /* Fence to ensure all write are committed to L2 */
+ kvx_fence();
+ /* Then invalidate the L1 icache to reload from L2 */
+ l1_inval_icache_range(start, end);
+}
+
+static inline
+void local_flush_icache_range(unsigned long start, unsigned long end)
+{
+ sync_dcache_icache(start, end);
+}
+
+#ifdef CONFIG_SMP
+void flush_icache_range(unsigned long start, unsigned long end);
+#else
+#define flush_icache_range local_flush_icache_range
+#endif
+
+static inline
+void flush_icache_page(struct vm_area_struct *vma, struct page *page)
+{
+ unsigned long start = (unsigned long) page_address(page);
+ unsigned long end = start + PAGE_SIZE;
+
+ sync_dcache_icache(start, end);
+}
+
+static inline
+void flush_icache_user_range(struct vm_area_struct *vma, struct page *page,
+ unsigned long vaddr, int len)
+{
+ sync_dcache_icache(vaddr, vaddr + len);
+}
+
+#define copy_to_user_page(vma, page, vaddr, dst, src, len) \
+ do { \
+ memcpy(dst, src, len); \
+ if (vma->vm_flags & VM_EXEC) \
+ flush_icache_user_range(vma, page, vaddr, len); \
+ } while (0)
+#define copy_from_user_page(vma, page, vaddr, dst, src, len) \
+ memcpy(dst, src, len)
+
+#endif /* _ASM_KVX_CACHEFLUSH_H */
diff --git a/arch/kvx/include/asm/fixmap.h b/arch/kvx/include/asm/fixmap.h
new file mode 100644
index 000000000000..3863e410d71d
--- /dev/null
+++ b/arch/kvx/include/asm/fixmap.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_FIXMAP_H
+#define _ASM_KVX_FIXMAP_H
+
+/**
+ * Use the latest available kernel address minus one page.
+ * This is needed since __fix_to_virt returns
+ * (FIXADDR_TOP - ((x) << PAGE_SHIFT))
+ * Due to that, first member will be shifted by 0 and will be equal to
+ * FIXADDR_TOP.
+ * Some other architectures simply add a FIX_HOLE at the beginning of
+ * the fixed_addresses enum (I think ?).
+ */
+#define FIXADDR_TOP (-PAGE_SIZE)
+
+#define ASM_FIX_TO_VIRT(IDX) \
+ (FIXADDR_TOP - ((IDX) << PAGE_SHIFT))
+
+#ifndef __ASSEMBLY__
+#include <asm/page.h>
+#include <asm/pgtable.h>
+
+enum fixed_addresses {
+ FIX_EARLYCON_MEM_BASE,
+ FIX_GDB_BARE_DISPLACED_MEM_BASE,
+ /* Used to access text early in RW mode (jump label) */
+ FIX_TEXT_PATCH,
+ __end_of_fixed_addresses
+};
+
+#define FIXADDR_SIZE (__end_of_fixed_addresses << PAGE_SHIFT)
+#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
+#define FIXMAP_PAGE_IO (PAGE_KERNEL_DEVICE)
+
+void __set_fixmap(enum fixed_addresses idx,
+ phys_addr_t phys, pgprot_t prot);
+
+#include <asm-generic/fixmap.h>
+#endif /* __ASSEMBLY__ */
+
+#endif
diff --git a/arch/kvx/include/asm/hugetlb.h b/arch/kvx/include/asm/hugetlb.h
new file mode 100644
index 000000000000..a5984e8ede7e
--- /dev/null
+++ b/arch/kvx/include/asm/hugetlb.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_HUGETLB_H
+#define _ASM_KVX_HUGETLB_H
+
+#include <asm/pgtable.h>
+
+#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT
+extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte);
+
+#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR
+extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep);
+
+#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS
+extern int huge_ptep_set_access_flags(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep,
+ pte_t pte, int dirty);
+
+#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT
+extern void huge_ptep_set_wrprotect(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep);
+
+#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH
+extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep);
+
+#include <asm-generic/hugetlb.h>
+
+#endif /* _ASM_KVX_HUGETLB_H */
diff --git a/arch/kvx/include/asm/l2_cache.h b/arch/kvx/include/asm/l2_cache.h
new file mode 100644
index 000000000000..437b82bb78ab
--- /dev/null
+++ b/arch/kvx/include/asm/l2_cache.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_L2_CACHE_H
+#define _ASM_KVX_L2_CACHE_H
+
+#include <linux/bits.h>
+#include <linux/jump_label.h>
+
+#ifdef CONFIG_L2_CACHE
+
+#include <asm/cache.h>
+#include <asm/l2_cache_defs.h>
+
+void l2_cache_push_area_cmd(u64 cmd_type, phys_addr_t start,
+ unsigned long size);
+
+extern struct static_key_false l2_enabled;
+
+static bool
+l2_cache_send_cmd(u64 cmd_type, phys_addr_t start, unsigned long size)
+{
+ if (static_branch_likely(&l2_enabled)) {
+ l2_cache_push_area_cmd(cmd_type, start, size);
+ return true;
+ }
+
+ return false;
+}
+
+static inline bool
+l2_cache_wbinval_range(phys_addr_t start, unsigned long size)
+{
+ return l2_cache_send_cmd(L2_CMD_OP_CMD_PURGE_AREA, start, size);
+}
+
+static inline bool
+l2_cache_wb_range(phys_addr_t start, unsigned long size)
+{
+ return l2_cache_send_cmd(L2_CMD_OP_CMD_FLUSH_AREA, start, size);
+}
+
+static inline bool
+l2_cache_inval_range(phys_addr_t start, unsigned long size)
+{
+ /* We don't need invalidation to be synced (ie 0) */
+ return l2_cache_send_cmd(L2_CMD_OP_CMD_INVAL_AREA, start, size);
+}
+
+#else
+
+static inline bool
+l2_cache_wbinval_range(phys_addr_t start, unsigned long size)
+{
+ return false;
+}
+
+static inline bool
+l2_cache_wb_range(phys_addr_t start, unsigned long size)
+{
+ return false;
+}
+
+static inline bool
+l2_cache_inval_range(phys_addr_t start, unsigned long size)
+{
+ return false;
+}
+
+#endif
+
+#endif /* _ASM_KVX_L2_CACHE_H */
diff --git a/arch/kvx/include/asm/l2_cache_defs.h b/arch/kvx/include/asm/l2_cache_defs.h
new file mode 100644
index 000000000000..81f001d9087a
--- /dev/null
+++ b/arch/kvx/include/asm/l2_cache_defs.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julien Hascoet
+ */
+
+#ifndef _ASM_KVX_L2_CACHE_DEFS_H
+#define _ASM_KVX_L2_CACHE_DEFS_H
+
+/* Error registers */
+enum l2_api_error {
+ L2_STA_ERR_CFG = 1,
+ L2_STA_ERR_ADDR,
+ L2_STA_ERR_STOP,
+ L2_STA_ERR_NO_MEM,
+ L2_STA_ERR_DIS,
+ L2_STA_ERR_DT
+};
+
+/* L2 Cache status registers definition */
+#define L2_STATUS_OFFSET 0x0
+#define L2_STATUS_ERROR_MASK 0x4
+#define L2_STATUS_ERROR_SHIFT 0x2
+#define L2_STATUS_READY_MASK 0x2
+#define L2_STATUS_READY_SHIFT 0x1
+#define L2_STATUS_VALID_MASK 0x1
+#define L2_STATUS_VALID_SHIFT 0x0
+#define L2_STATUS_ACK_MASK 0x80
+#define L2_STATUS_ACK_SHIFT 0x7
+
+/* L2 Cache error registers definition */
+#define L2_ERROR_OFFSET 0x20
+#define L2_ERROR_SETUP_ERR_MASK 0x1
+#define L2_ERROR_SETUP_ERR_SHIFT 0x0
+#define L2_ERROR_API_ERR_MASK 0x20
+#define L2_ERROR_API_ERR_SHIFT 0x5
+#define L2_ERROR_ERROR_CODE_MASK 0xFF00
+#define L2_ERROR_ERROR_CODE_SHIFT 0x8
+
+/* L2 Cache instance registers definition */
+#define L2_INSTANCE_OFFSET 0x8
+#define L2_INSTANCE_CMD_QUEUE_SIZE_SHIFT 0x20
+#define L2_INSTANCE_CMD_QUEUE_SIZE_MASK 0x7F00000000UL
+
+/* L2 Cache commands fifo registers definition */
+#define L2_CMD_OFFSET 0x400
+#define L2_CMD_READ_IDX_OFFSET 0x0
+#define L2_CMD_WRITE_IDX_OFFSET 0x8
+#define L2_CMD_DOORBELL_READ_ADDR_OFFSET 0x10
+#define L2_CMD_DOORBELL_WRITE_ADDR_OFFSET 0x18
+#define L2_CMD_FIFO_OFFSET 0x20
+#define L2_CMD_FIFO_ELEM_SIZE 0x20
+#define L2_CMD_OP_CMD_PURGE_LINE 0x4
+#define L2_CMD_OP_CMD_PURGE_AREA 0x8
+#define L2_CMD_OP_CMD_FLUSH_AREA 0x9
+#define L2_CMD_OP_CMD_INVAL_AREA 0xA
+#define L2_CMD_OP_VALID_SHIFT 0x0
+#define L2_CMD_OP_SYNC_SHIFT 0x1
+#define L2_CMD_OP_CMD_SHIFT 0x2
+#define L2_CMD_OP_ARG_COUNT 0x3
+
+
+#endif /* _ASM_KVX_L2_CACHE_DEFS_H */
diff --git a/arch/kvx/include/asm/mem_map.h b/arch/kvx/include/asm/mem_map.h
new file mode 100644
index 000000000000..ea90144209ce
--- /dev/null
+++ b/arch/kvx/include/asm/mem_map.h
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef _ASM_KVX_MEM_MAP_H
+#define _ASM_KVX_MEM_MAP_H
+
+#include <linux/const.h>
+#include <linux/sizes.h>
+
+#include <asm/page.h>
+#include <asm/fixmap.h>
+
+/**
+ * kvx memory mapping defines
+ * For more information on memory mapping, please see
+ * Documentation/kvx/kvx-mmu.txt
+ *
+ * All _BASE defines are relative to PAGE_OFFSET
+ */
+
+/* Guard between various memory map zones */
+#define MAP_GUARD_SIZE SZ_1G
+
+/**
+ * Kernel direct memory mapping
+ */
+#define KERNEL_DIRECT_MEMORY_MAP_BASE PAGE_OFFSET
+#define KERNEL_DIRECT_MEMORY_MAP_SIZE UL(0x1000000000)
+#define KERNEL_DIRECT_MEMORY_MAP_END \
+ (KERNEL_DIRECT_MEMORY_MAP_BASE + KERNEL_DIRECT_MEMORY_MAP_SIZE)
+
+/**
+ * Vmalloc mapping (goes from kernel direct memory map up to fixmap start -
+ * guard size)
+ */
+#define KERNEL_VMALLOC_MAP_BASE (KERNEL_DIRECT_MEMORY_MAP_END + MAP_GUARD_SIZE)
+#define KERNEL_VMALLOC_MAP_SIZE \
+ (FIXADDR_START - KERNEL_VMALLOC_MAP_BASE - MAP_GUARD_SIZE)
+
+#endif
diff --git a/arch/kvx/include/asm/mmu.h b/arch/kvx/include/asm/mmu.h
new file mode 100644
index 000000000000..fbf5ed286e87
--- /dev/null
+++ b/arch/kvx/include/asm/mmu.h
@@ -0,0 +1,296 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Marc Poulhiès
+ */
+
+#ifndef _ASM_KVX_MMU_H
+#define _ASM_KVX_MMU_H
+
+#include <linux/bug.h>
+#include <linux/types.h>
+#include <linux/threads.h>
+
+#include <asm/page.h>
+#include <asm/sfr.h>
+#include <asm/page.h>
+#include <asm/pgtable-bits.h>
+#include <asm/tlb_defs.h>
+
+/* Virtual addresses can use at most 41 bits */
+#define MMU_VIRT_BITS 41
+
+/*
+ * See Documentation/kvx/kvx-mmu.txt for details about the division of the
+ * virtual memory space.
+ */
+#if defined(CONFIG_KVX_4K_PAGES)
+#define MMU_USR_ADDR_BITS 39
+#else
+#error "Only 4Ko page size is supported at this time"
+#endif
+
+typedef struct mm_context {
+ unsigned long end_brk;
+ unsigned long asn[NR_CPUS];
+ unsigned long sigpage;
+} mm_context_t;
+
+struct __packed tlb_entry_low {
+ unsigned int es:2; /* Entry Status */
+ unsigned int cp:2; /* Cache Policy */
+ unsigned int pa:4; /* Protection Attributes */
+ unsigned int r:2; /* Reserved */
+ unsigned int ps:2; /* Page Size */
+ unsigned int fn:28; /* Frame Number */
+};
+
+struct __packed tlb_entry_high {
+ unsigned int asn:9; /* Address Space Number */
+ unsigned int g:1; /* Global Indicator */
+ unsigned int vs:2; /* Virtual Space */
+ unsigned int pn:29; /* Page Number */
+};
+
+struct kvx_tlb_format {
+ union {
+ struct tlb_entry_low tel;
+ uint64_t tel_val;
+ };
+ union {
+ struct tlb_entry_high teh;
+ uint64_t teh_val;
+ };
+};
+
+#define KVX_EMPTY_TLB_ENTRY { .tel_val = 0x0, .teh_val = 0x0 }
+
+/* Bit [0:39] of the TLB format corresponds to TLB Entry low */
+/* Bit [40:80] of the TLB format corresponds to the TLB Entry high */
+#define kvx_mmu_set_tlb_entry(tlbf) do { \
+ kvx_sfr_set(TEL, (uint64_t) tlbf.tel_val); \
+ kvx_sfr_set(TEH, (uint64_t) tlbf.teh_val); \
+} while (0)
+
+#define kvx_mmu_get_tlb_entry(tlbf) do { \
+ tlbf.tel_val = kvx_sfr_get(TEL); \
+ tlbf.teh_val = kvx_sfr_get(TEH); \
+} while (0)
+
+/* Use kvx_mmc_ to read a field from MMC value passed as parameter */
+#define __kvx_mmc(mmc_reg, field) \
+ kvx_sfr_field_val(mmc_reg, MMC, field)
+
+#define kvx_mmc_error(mmc) __kvx_mmc(mmc, E)
+#define kvx_mmc_parity(mmc) __kvx_mmc(mmc, PAR)
+#define kvx_mmc_sb(mmc) __kvx_mmc(mmc, SB)
+#define kvx_mmc_ss(mmc) __kvx_mmc(mmc, SS)
+#define kvx_mmc_sw(mmc) __kvx_mmc(mmc, SW)
+#define kvx_mmc_asn(mmc) __kvx_mmc(mmc, ASN)
+
+#define KVX_TLB_ACCESS_READ 0
+#define KVX_TLB_ACCESS_WRITE 1
+#define KVX_TLB_ACCESS_PROBE 2
+
+#ifdef CONFIG_KVX_DEBUG_TLB_ACCESS
+
+#define KVX_TLB_ACCESS_SIZE (1 << CONFIG_KVX_DEBUG_TLB_ACCESS_BITS)
+#define KVX_TLB_ACCESS_MASK GENMASK((CONFIG_KVX_DEBUG_TLB_ACCESS_BITS - 1), 0)
+#define KVX_TLB_ACCESS_GET_IDX(idx) (idx & KVX_TLB_ACCESS_MASK)
+
+/* This structure is used to make decoding of MMC easier in gdb */
+struct mmc_t {
+ unsigned int asn:9;
+ unsigned int s: 1;
+ unsigned int r1: 4;
+ unsigned int sne: 1;
+ unsigned int spe: 1;
+ unsigned int ptc: 2;
+ unsigned int sw: 4;
+ unsigned int ss: 6;
+ unsigned int sb: 1;
+ unsigned int r2: 1;
+ unsigned int par: 1;
+ unsigned int e: 1;
+};
+
+struct __packed kvx_tlb_access_t {
+ struct kvx_tlb_format entry; /* 128 bits */
+ union {
+ struct mmc_t mmc;
+ uint32_t mmc_val;
+ };
+ uint32_t type;
+};
+
+extern void kvx_update_tlb_access(int type);
+
+#else
+#define kvx_update_tlb_access(type) do {} while (0)
+#endif
+
+static inline void kvx_mmu_readtlb(void)
+{
+ kvx_update_tlb_access(KVX_TLB_ACCESS_READ);
+ asm volatile ("tlbread\n;;");
+}
+
+static inline void kvx_mmu_writetlb(void)
+{
+ kvx_update_tlb_access(KVX_TLB_ACCESS_WRITE);
+ asm volatile ("tlbwrite\n;;");
+}
+
+static inline void kvx_mmu_probetlb(void)
+{
+ kvx_update_tlb_access(KVX_TLB_ACCESS_PROBE);
+ asm volatile ("tlbprobe\n;;");
+}
+
+#define kvx_mmu_add_entry(buffer, way, entry) do { \
+ kvx_sfr_set_field(MMC, SB, buffer); \
+ kvx_sfr_set_field(MMC, SW, way); \
+ kvx_mmu_set_tlb_entry(entry); \
+ kvx_mmu_writetlb(); \
+} while (0)
+
+#define kvx_mmu_remove_ltlb_entry(way) do { \
+ struct kvx_tlb_format __invalid_entry = KVX_EMPTY_TLB_ENTRY; \
+ kvx_mmu_add_entry(MMC_SB_LTLB, way, __invalid_entry); \
+} while (0)
+
+static inline int get_page_size_shift(int ps)
+{
+ /*
+ * Use the same assembly trick using sbmm to directly get the page size
+ * shift using a constant which encodes all page size shifts
+ */
+ return __builtin_kvx_sbmm8(KVX_PS_SHIFT_MATRIX,
+ KVX_SBMM_BYTE_SEL << ps);
+}
+
+/*
+ * 4 bits are used to index the kvx access permissions. Bits are used as
+ * follow:
+ *
+ * +---------------+------------+-------------+------------+
+ * | Bit 3 | Bit 2 | Bit 1 | Bit 0 |
+ * |---------------+------------+-------------+------------|
+ * | _PAGE_GLOBAL | _PAGE_EXEC | _PAGE_WRITE | _PAGE_READ |
+ * +---------------+------------+-------------+------------+
+ *
+ * If _PAGE_GLOBAL is set then the page belongs to the kernel. Otherwise it
+ * belongs to the user. When the page belongs to user-space then give the
+ * same rights to the kernel-space.
+ * In order to quickly compute policy from this value, we use sbmm instruction.
+ * The main interest is to avoid an additionnal load and specifically in the
+ * assembly refill handler.
+ */
+static inline u8 get_page_access_perms(u8 policy)
+{
+ /* If PAGE_READ is unset, there is no permission for this page */
+ if (!(policy & (_PAGE_READ >> _PAGE_PERMS_SHIFT)))
+ return TLB_PA_NA_NA;
+
+ /* Discard _PAGE_READ bit to get a linear number in [0,7] */
+ policy >>= 1;
+
+ /* Use sbmm to directly get the page perms */
+ return __builtin_kvx_sbmm8(KVX_PAGE_PA_MATRIX,
+ KVX_SBMM_BYTE_SEL << policy);
+}
+
+static inline struct kvx_tlb_format tlb_mk_entry(
+ void *paddr,
+ void *vaddr,
+ unsigned int ps,
+ unsigned int global,
+ unsigned int pa,
+ unsigned int cp,
+ unsigned int asn,
+ unsigned int es)
+{
+ struct kvx_tlb_format entry;
+ u64 mask = ULONG_MAX << get_page_size_shift(ps);
+
+ BUG_ON(ps >= (1 << KVX_SFR_TEL_PS_WIDTH));
+
+ /*
+ * 0 matches the virtual space:
+ * - either we are virtualized and the hypervisor will set it
+ * for us when using writetlb
+ * - Or we are native and the virtual space is 0
+ */
+ entry.teh_val = TLB_MK_TEH_ENTRY((uintptr_t)vaddr & mask, 0, global,
+ asn);
+ entry.tel_val = TLB_MK_TEL_ENTRY((uintptr_t)paddr, ps, es, cp, pa);
+
+ return entry;
+}
+
+static inline unsigned long tlb_entry_phys(struct kvx_tlb_format tlbe)
+{
+ return ((unsigned long) tlbe.tel.fn << KVX_SFR_TEL_FN_SHIFT);
+}
+
+static inline unsigned long tlb_entry_virt(struct kvx_tlb_format tlbe)
+{
+ return ((unsigned long) tlbe.teh.pn << KVX_SFR_TEH_PN_SHIFT);
+}
+
+static inline unsigned long tlb_entry_size(struct kvx_tlb_format tlbe)
+{
+ return BIT(get_page_size_shift(tlbe.tel.ps));
+}
+
+static inline int tlb_entry_overlaps(struct kvx_tlb_format tlbe1,
+ struct kvx_tlb_format tlbe2)
+{
+ unsigned long start1, end1;
+ unsigned long start2, end2;
+
+ start1 = tlb_entry_virt(tlbe1);
+ end1 = start1 + tlb_entry_size(tlbe1);
+
+ start2 = tlb_entry_virt(tlbe2);
+ end2 = start2 + tlb_entry_size(tlbe2);
+
+ return start1 <= end2 && end1 >= start2;
+}
+
+static inline int tlb_entry_match_addr(struct kvx_tlb_format tlbe,
+ unsigned long vaddr)
+{
+ /*
+ * TLB entries store up to 41 bits so we must truncate the provided
+ * address to match teh.pn.
+ */
+ vaddr &= GENMASK(MMU_VIRT_BITS - 1, KVX_SFR_TEH_PN_SHIFT);
+
+ return tlb_entry_virt(tlbe) == vaddr;
+}
+
+extern void kvx_mmu_early_setup(void);
+
+
+#if defined(CONFIG_STRICT_KERNEL_RWX)
+void init_kernel_rwx(void);
+void paging_init(void);
+#else
+
+static inline void paging_init(void) {}
+#endif
+
+void kvx_mmu_ltlb_remove_entry(unsigned long vaddr);
+void kvx_mmu_ltlb_add_entry(unsigned long vaddr, phys_addr_t paddr,
+ pgprot_t flags, unsigned long page_shift);
+
+void kvx_mmu_jtlb_add_entry(unsigned long address, pte_t *ptep,
+ unsigned int asn);
+extern void mmu_early_init(void);
+
+struct mm_struct;
+
+#endif /* _ASM_KVX_MMU_H */
diff --git a/arch/kvx/include/asm/mmu_context.h b/arch/kvx/include/asm/mmu_context.h
new file mode 100644
index 000000000000..39fa92f1506b
--- /dev/null
+++ b/arch/kvx/include/asm/mmu_context.h
@@ -0,0 +1,156 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef __ASM_KVX_MMU_CONTEXT_H
+#define __ASM_KVX_MMU_CONTEXT_H
+
+/*
+ * Management of the Address Space Number:
+ * Coolidge architecture provides a 9-bit ASN to tag TLB entries. This can be
+ * used to allow several entries with the same virtual address (so from
+ * different process) to be in the TLB at the same time. That means that won't
+ * necessarily flush the TLB when a context switch occurs and so it will
+ * improve performances.
+ */
+#include <linux/smp.h>
+
+#include <asm/mmu.h>
+#include <asm/sfr_defs.h>
+#include <asm/tlbflush.h>
+
+#include <asm-generic/mm_hooks.h>
+
+#define MM_CTXT_ASN_MASK GENMASK(KVX_SFR_MMC_ASN_WIDTH - 1, 0)
+#define MM_CTXT_CYCLE_MASK (~MM_CTXT_ASN_MASK)
+#define MM_CTXT_NO_ASN UL(0x0)
+#define MM_CTXT_FIRST_CYCLE (MM_CTXT_ASN_MASK + 1)
+
+#define mm_asn(mm, cpu) ((mm)->context.asn[cpu])
+
+DECLARE_PER_CPU(unsigned long, kvx_asn_cache);
+#define cpu_asn_cache(cpu) per_cpu(kvx_asn_cache, cpu)
+
+static inline void get_new_mmu_context(struct mm_struct *mm, unsigned int cpu)
+{
+ unsigned long asn = cpu_asn_cache(cpu);
+
+ asn++;
+ /* Check if we need to start a new cycle */
+ if ((asn & MM_CTXT_ASN_MASK) == 0) {
+ pr_debug("%s: start new cycle, flush all tlb\n", __func__);
+ local_flush_tlb_all();
+
+ /*
+ * Above check for rollover of 9 bit ASN in 64 bit container.
+ * If the container itself wrapped around, set it to a non zero
+ * "generation" to distinguish from no context
+ */
+ if (asn == 0)
+ asn = MM_CTXT_FIRST_CYCLE;
+ }
+
+ cpu_asn_cache(cpu) = asn;
+ mm_asn(mm, cpu) = asn;
+
+ pr_debug("%s: mm = 0x%llx: cpu[%d], cycle: %lu, asn: %lu\n",
+ __func__, (unsigned long long)mm, cpu,
+ (asn & MM_CTXT_CYCLE_MASK) >> KVX_SFR_MMC_ASN_WIDTH,
+ asn & MM_CTXT_ASN_MASK);
+}
+
+static inline void get_mmu_context(struct mm_struct *mm, unsigned int cpu)
+{
+
+ unsigned long asn = mm_asn(mm, cpu);
+
+ /*
+ * Move to new ASN if it was not from current alloc-cycle/generation.
+ * This is done by ensuring that the generation bits in both
+ * mm->context.asn and cpu_asn_cache counter are exactly same.
+ *
+ * NOTE: this also works for checking if mm has a context since the
+ * first alloc-cycle/generation is always '1'. MM_CTXT_NO_ASN value
+ * contains cycle '0', and thus it will match.
+ */
+ if ((asn ^ cpu_asn_cache(cpu)) & MM_CTXT_CYCLE_MASK)
+ get_new_mmu_context(mm, cpu);
+}
+
+static inline void activate_context(struct mm_struct *mm, unsigned int cpu)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ get_mmu_context(mm, cpu);
+
+ kvx_sfr_set_field(MMC, ASN, mm_asn(mm, cpu) & MM_CTXT_ASN_MASK);
+
+ local_irq_restore(flags);
+}
+
+/**
+ * Redefining the generic hooks that are:
+ * - activate_mm
+ * - deactivate_mm
+ * - enter_lazy_tlb
+ * - init_new_context
+ * - destroy_context
+ * - switch_mm
+ */
+
+#define activate_mm(prev, next) switch_mm((prev), (next), NULL)
+#define deactivate_mm(tsk, mm) do { } while (0)
+#define enter_lazy_tlb(mm, tsk) do { } while (0)
+
+static inline int init_new_context(struct task_struct *tsk,
+ struct mm_struct *mm)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu)
+ mm_asn(mm, cpu) = MM_CTXT_NO_ASN;
+
+ return 0;
+}
+
+static inline void destroy_context(struct mm_struct *mm)
+{
+ int cpu = smp_processor_id();
+ unsigned long flags;
+
+ local_irq_save(flags);
+ mm_asn(mm, cpu) = MM_CTXT_NO_ASN;
+ local_irq_restore(flags);
+}
+
+static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next,
+ struct task_struct *tsk)
+{
+ unsigned int cpu = smp_processor_id();
+
+ /**
+ * Comment taken from arc, but logic is the same for us:
+ *
+ * Note that the mm_cpumask is "aggregating" only, we don't clear it
+ * for the switched-out task, unlike some other arches.
+ * It is used to enlist cpus for sending TLB flush IPIs and not sending
+ * it to CPUs where a task once ran-on, could cause stale TLB entry
+ * re-use, specially for a multi-threaded task.
+ * e.g. T1 runs on C1, migrates to C3. T2 running on C2 munmaps.
+ * For a non-aggregating mm_cpumask, IPI not sent C1, and if T1
+ * were to re-migrate to C1, it could access the unmapped region
+ * via any existing stale TLB entries.
+ */
+ cpumask_set_cpu(cpu, mm_cpumask(next));
+
+ if (prev != next)
+ activate_context(next, cpu);
+}
+
+
+#endif /* __ASM_KVX_MMU_CONTEXT_H */
diff --git a/arch/kvx/include/asm/mmu_stats.h b/arch/kvx/include/asm/mmu_stats.h
new file mode 100644
index 000000000000..999352dbc1ce
--- /dev/null
+++ b/arch/kvx/include/asm/mmu_stats.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_MMU_STATS_H
+#define _ASM_KVX_MMU_STATS_H
+
+#ifdef CONFIG_KVX_MMU_STATS
+#include <linux/percpu.h>
+
+struct mmu_refill_stats {
+ unsigned long count;
+ unsigned long total;
+ unsigned long min;
+ unsigned long max;
+};
+
+enum mmu_refill_type {
+ MMU_REFILL_TYPE_USER,
+ MMU_REFILL_TYPE_KERNEL,
+ MMU_REFILL_TYPE_KERNEL_DIRECT,
+ MMU_REFILL_TYPE_COUNT,
+};
+
+struct mmu_stats {
+ struct mmu_refill_stats refill[MMU_REFILL_TYPE_COUNT];
+ /* keep these fields ordered this way for assembly */
+ unsigned long cycles_between_refill;
+ unsigned long last_refill;
+ unsigned long tlb_flush_all;
+};
+
+DECLARE_PER_CPU(struct mmu_stats, mmu_stats);
+#endif
+
+#endif /* _ASM_KVX_MMU_STATS_H */
diff --git a/arch/kvx/include/asm/page.h b/arch/kvx/include/asm/page.h
new file mode 100644
index 000000000000..cea5f02d12bb
--- /dev/null
+++ b/arch/kvx/include/asm/page.h
@@ -0,0 +1,187 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_PAGE_H
+#define _ASM_KVX_PAGE_H
+
+#include <linux/const.h>
+
+#define PAGE_SHIFT CONFIG_KVX_PAGE_SHIFT
+#define PAGE_SIZE _BITUL(PAGE_SHIFT)
+#define PAGE_MASK (~(PAGE_SIZE - 1))
+
+#define PHYS_OFFSET CONFIG_KVX_PHYS_OFFSET
+#define PAGE_OFFSET CONFIG_KVX_PAGE_OFFSET
+
+#define VA_TO_PA_OFFSET (PHYS_OFFSET - PAGE_OFFSET)
+#define PA_TO_VA_OFFSET (PAGE_OFFSET - PHYS_OFFSET)
+
+/*
+ * These macros are specifically written for assembly. They are useful for
+ * converting symbols above PAGE_OFFSET to their physical addresses.
+ */
+#define __PA(x) ((x) + VA_TO_PA_OFFSET)
+#define __VA(x) ((x) + PA_TO_VA_OFFSET)
+
+/*
+ * PFN starts at 0 if physical address starts at 0x0. As it is not the case
+ * for the kvx we need to apply an offset to the calculated PFN.
+ */
+#define ARCH_PFN_OFFSET ((unsigned long)(PHYS_OFFSET >> PAGE_SHIFT))
+
+#if defined(CONFIG_KVX_4K_PAGES)
+/* Maximum usable bit using 4K pages and current page table layout */
+#define VA_MAX_BITS 40
+#define PGDIR_SHIFT 30
+#define PMD_SHIFT 21
+#else
+#error "64K page not supported"
+#endif
+
+/*
+ * Define _SHIFT, _SIZE and _MASK corresponding of the different page
+ * sizes supported by kvx.
+ */
+#define KVX_PAGE_4K_SHIFT 12
+#define KVX_PAGE_4K_SIZE BIT(KVX_PAGE_4K_SHIFT)
+#define KVX_PAGE_4K_MASK (~(KVX_PAGE_4K_SIZE - 1))
+
+#define KVX_PAGE_64K_SHIFT 16
+#define KVX_PAGE_64K_SIZE BIT(KVX_PAGE_64K_SHIFT)
+#define KVX_PAGE_64K_MASK (~(KVX_PAGE_64K_SIZE - 1))
+
+#define KVX_PAGE_2M_SHIFT 21
+#define KVX_PAGE_2M_SIZE BIT(KVX_PAGE_2M_SHIFT)
+#define KVX_PAGE_2M_MASK (~(KVX_PAGE_2M_SIZE - 1))
+
+#define KVX_PAGE_512M_SHIFT 29
+#define KVX_PAGE_512M_SIZE BIT(KVX_PAGE_512M_SHIFT)
+#define KVX_PAGE_512M_MASK (~(KVX_PAGE_512M_SIZE - 1))
+
+/* Encode all page shift into one 32bit constant for sbmm */
+#define KVX_PS_SHIFT_MATRIX ((KVX_PAGE_512M_SHIFT << 24) | \
+ (KVX_PAGE_2M_SHIFT << 16) | \
+ (KVX_PAGE_64K_SHIFT << 8) | \
+ (KVX_PAGE_4K_SHIFT))
+
+/* Encode all page access policy into one 64bit constant for sbmm */
+#define KVX_PAGE_PA_MATRIX ((UL(TLB_PA_NA_RWX) << 56) | \
+ (UL(TLB_PA_NA_RX) << 48) | \
+ (UL(TLB_PA_NA_RW) << 40) | \
+ (UL(TLB_PA_NA_R) << 32) | \
+ (UL(TLB_PA_RWX_RWX) << 24) | \
+ (UL(TLB_PA_RX_RX) << 16) | \
+ (UL(TLB_PA_RW_RW) << 8) | \
+ (UL(TLB_PA_R_R)))
+
+/*
+ * Select a byte using sbmm8. When shifted by one bit left, we get the next
+ * byte.
+ * For instance using this default constant with sbmm yields the value between
+ * first byte of the double word.
+ * If constant is shifted by 1, the value is now 0x0000000000000002ULL and this
+ * yield the second byte and so on, and so on !
+ */
+#define KVX_SBMM_BYTE_SEL 0x01
+
+#ifndef __ASSEMBLY__
+
+#include <linux/string.h>
+
+/* Page Global Directory entry */
+typedef struct {
+ unsigned long pgd;
+} pgd_t;
+
+/* Page Middle Directory entry */
+typedef struct {
+ unsigned long pmd;
+} pmd_t;
+
+/* Page Table entry */
+typedef struct {
+ unsigned long pte;
+} pte_t;
+
+/* Protection bits */
+typedef struct {
+ unsigned long pgprot;
+} pgprot_t;
+
+typedef struct page *pgtable_t;
+
+/**
+ * Macros to access entry values
+ */
+#define pgd_val(x) ((x).pgd)
+#define pmd_val(x) ((x).pmd)
+#define pte_val(x) ((x).pte)
+#define pgprot_val(x) ((x).pgprot)
+
+/**
+ * Macro to create entry from value
+ */
+#define __pgd(x) ((pgd_t) { (x) })
+#define __pmd(x) ((pmd_t) { (x) })
+#define __pte(x) ((pte_t) { (x) })
+#define __pgprot(x) ((pgprot_t) { (x) })
+
+#define pte_pgprot(x) __pgprot(pte_val(x) & ~KVX_PFN_MASK)
+
+#define __pa(x) ((unsigned long)(x) + VA_TO_PA_OFFSET)
+#define __va(x) ((void *)((unsigned long) (x) + PA_TO_VA_OFFSET))
+
+#define phys_to_pfn(phys) (PFN_DOWN(phys))
+#define pfn_to_phys(pfn) (PFN_PHYS(pfn))
+
+#define virt_to_pfn(vaddr) (phys_to_pfn(__pa(vaddr)))
+#define pfn_to_virt(pfn) (__va(pfn_to_phys(pfn)))
+
+#define virt_to_page(vaddr) (pfn_to_page(virt_to_pfn(vaddr)))
+#define page_to_virt(page) (pfn_to_virt(page_to_pfn(page)))
+
+#define page_to_phys(page) virt_to_phys(page_to_virt(page))
+#define phys_to_page(phys) (pfn_to_page(phys_to_pfn(phys)))
+
+#define virt_addr_valid(vaddr) (pfn_valid(virt_to_pfn(vaddr)))
+
+#ifdef CONFIG_FLATMEM
+static inline bool pfn_valid(unsigned long pfn)
+{
+ /* avoid <linux/mm.h> include hell */
+ extern unsigned long max_mapnr;
+
+ return ((pfn >= ARCH_PFN_OFFSET) &&
+ (pfn < (ARCH_PFN_OFFSET + max_mapnr)));
+}
+#endif
+
+extern void clear_page(void *to);
+extern void copy_page(void *to, void *from);
+
+static inline void clear_user_page(void *page, unsigned long vaddr,
+ struct page *pg)
+{
+ clear_page(page);
+}
+
+static inline void copy_user_page(void *to, void *from, unsigned long vaddr,
+ struct page *topage)
+{
+ copy_page(to, from);
+}
+
+#define VM_DATA_DEFAULT_FLAGS (VM_READ | VM_WRITE | \
+ VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
+
+#include <asm-generic/memory_model.h>
+#include <asm-generic/getorder.h>
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_KVX_PAGE_H */
diff --git a/arch/kvx/include/asm/page_size.h b/arch/kvx/include/asm/page_size.h
new file mode 100644
index 000000000000..2c2850205b50
--- /dev/null
+++ b/arch/kvx/include/asm/page_size.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_PAGE_SIZE_H
+#define _ASM_KVX_PAGE_SIZE_H
+
+#include <asm/tlb_defs.h>
+
+#if defined(CONFIG_HUGETLB_PAGE)
+#define HUGE_PAGE_SIZE (MMC_PMJ_64K | MMC_PMJ_2M | MMC_PMJ_512M)
+#else
+#define HUGE_PAGE_SIZE (0)
+#endif
+
+#if defined(CONFIG_KVX_4K_PAGES)
+#define TLB_DEFAULT_PS TLB_PS_4K
+#define KVX_SUPPORTED_PSIZE (MMC_PMJ_4K | HUGE_PAGE_SIZE)
+#elif defined(CONFIG_KVX_64K_PAGES)
+#define TLB_DEFAULT_PS TLB_PS_64K
+#define KVX_SUPPORTED_PSIZE (MMC_PMJ_64K | HUGE_PAGE_SIZE)
+#else
+#error "Unsupported page size"
+#endif
+
+#endif
diff --git a/arch/kvx/include/asm/pgalloc.h b/arch/kvx/include/asm/pgalloc.h
new file mode 100644
index 000000000000..0e654dd1a072
--- /dev/null
+++ b/arch/kvx/include/asm/pgalloc.h
@@ -0,0 +1,101 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_PGALLOC_H
+#define _ASM_KVX_PGALLOC_H
+
+#include <linux/mm.h>
+#include <asm/tlb.h>
+
+#define __HAVE_ARCH_PGD_FREE
+#include <asm-generic/pgalloc.h> /* for pte_{alloc,free}_one */
+
+static inline void check_pgt_cache(void)
+{
+ /*
+ * check_pgt_cache() is called to check watermarks from counters that
+ * computes the number of pages allocated by cached allocation functions
+ * pmd_alloc_one_fast() and pte_alloc_one_fast().
+ * Currently we just skip this test.
+ */
+}
+
+/**
+ * PGD
+ */
+
+static inline void
+pgd_free(struct mm_struct *mm, pgd_t *pgd)
+{
+ free_pages((unsigned long) pgd, PAGES_PER_PGD);
+}
+
+static inline
+pgd_t *pgd_alloc(struct mm_struct *mm)
+{
+ pgd_t *pgd;
+
+ pgd = (pgd_t *) __get_free_pages(GFP_KERNEL, PAGES_PER_PGD);
+ if (unlikely(pgd == NULL))
+ return NULL;
+
+ memset(pgd, 0, USER_PTRS_PER_PGD * sizeof(pgd_t));
+
+ /* Copy kernel mappings */
+ memcpy(pgd + USER_PTRS_PER_PGD,
+ init_mm.pgd + USER_PTRS_PER_PGD,
+ (PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t));
+
+ return pgd;
+}
+
+/**
+ * PUD
+ */
+
+static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
+{
+ unsigned long pfn = virt_to_pfn(pmd);
+
+ set_pud(pud, __pud((unsigned long)pfn << PAGE_SHIFT));
+}
+
+/**
+ * PMD
+ */
+
+static inline void pmd_populate_kernel(struct mm_struct *mm,
+ pmd_t *pmd, pte_t *pte)
+{
+ unsigned long pfn = virt_to_pfn(pte);
+
+ set_pmd(pmd, __pmd((unsigned long)pfn << PAGE_SHIFT));
+}
+
+static inline void pmd_populate(struct mm_struct *mm,
+ pmd_t *pmd, pgtable_t pte)
+{
+ unsigned long pfn = virt_to_pfn(page_address(pte));
+
+ set_pmd(pmd, __pmd((unsigned long)pfn << PAGE_SHIFT));
+}
+
+#if CONFIG_PGTABLE_LEVELS > 2
+#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd)
+#endif /* CONFIG_PGTABLE_LEVELS > 2 */
+
+/**
+ * PTE
+ */
+
+#define __pte_free_tlb(tlb, pte, buf) \
+do { \
+ pgtable_pte_page_dtor(pte); \
+ tlb_remove_page((tlb), pte); \
+} while (0)
+
+#endif /* _ASM_KVX_PGALLOC_H */
diff --git a/arch/kvx/include/asm/pgtable-bits.h b/arch/kvx/include/asm/pgtable-bits.h
new file mode 100644
index 000000000000..98ee7d1fbe63
--- /dev/null
+++ b/arch/kvx/include/asm/pgtable-bits.h
@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Julian Vetter
+ */
+
+#ifndef _ASM_KVX_PGTABLE_BITS_H
+#define _ASM_KVX_PGTABLE_BITS_H
+
+/*
+ * Protection bit definition
+ * As we don't have any HW to handle page table walk, we can define
+ * our own PTE format. In order to make things easier, we are trying to match
+ * some parts of $tel and $teh.
+ *
+ * PageSZ must be on bit 10 and 11 because it matches the TEL.PS bits. And
+ * by doing that it is easier in assembly to set the TEL.PS to PageSZ.
+ * In other words, KVX_PAGE_SZ_SHIFT == KVX_SFR_TEL_PS_SHIFT.
+ * It is checked by using a BUILD_BUG_ON() in arch/kvx/mm/tlb.c.
+ *
+ * Huge bit must be somewhere in the first 12 bits to be able to detect it
+ * when reading the PMD entry.
+ *
+ * KV3-1:
+ * +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+ * | 63..23 | 22..13 | 12 | 11..10 | 9 | 8 | 7 | 6 | 5 | 4 | 3..2 | 1 | 0 |
+ * +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+ * PFN Unused S PageSZ H G X W R D CP A P
+ *
+ * Note: PFN is 40-bits wide. We use 41-bits to ensure that the upper bit is
+ * always set to 0. This is required when shifting PFN to right.
+ */
+
+/* Following shift are used in ASM to easily extract bit */
+#define _PAGE_PERMS_SHIFT 5
+#define _PAGE_GLOBAL_SHIFT 8
+#define _PAGE_HUGE_SHIFT 9
+
+#define _PAGE_PRESENT (1 << 0) /* Present */
+#define _PAGE_ACCESSED (1 << 1) /* Set by tlb refill code on any access */
+/* Bits 2 - 3 reserved for cache policy */
+#define _PAGE_DIRTY (1 << 4) /* Set by tlb refill code on any write */
+#define _PAGE_READ (1 << _PAGE_PERMS_SHIFT) /* Readable */
+#define _PAGE_WRITE (1 << 6) /* Writable */
+#define _PAGE_EXEC (1 << 7) /* Executable */
+#define _PAGE_GLOBAL (1 << _PAGE_GLOBAL_SHIFT) /* Global */
+#define _PAGE_HUGE (1 << _PAGE_HUGE_SHIFT) /* Huge page */
+/* Bits 10 - 11 reserved for page size */
+#define _PAGE_SOFT (1 << 12) /* Reserved for software */
+#define _PAGE_SZ_64K (TLB_PS_64K << KVX_PAGE_SZ_SHIFT)
+#define _PAGE_SZ_2M (TLB_PS_2M << KVX_PAGE_SZ_SHIFT)
+#define _PAGE_SZ_512M (TLB_PS_512M << KVX_PAGE_SZ_SHIFT)
+
+
+#define _PAGE_SPECIAL _PAGE_SOFT
+
+/*
+ * If _PAGE_PRESENT is clear because the user mapped it with PROT_NONE
+ * pte_present still gives true. Bit[15] of the PTE is used since its unused
+ * for a PTE entry for kv3-1 (see above)
+ */
+#define _PAGE_NONE (1 << 15)
+
+/* Note: mask used in assembly cannot be generated with GENMASK */
+#define KVX_PFN_SHIFT 23
+#define KVX_PFN_MASK (~(((1 << KVX_PFN_SHIFT) - 1)))
+
+#define KVX_PAGE_SZ_SHIFT 10
+#define KVX_PAGE_SZ_MASK KVX_SFR_TEL_PS_MASK
+
+/* Huge page of 64K are hold in PTE table */
+#define KVX_PAGE_64K_NR_CONT (1UL << (KVX_PAGE_64K_SHIFT - PAGE_SHIFT))
+/* Huge page of 512M are hold in PMD table */
+#define KVX_PAGE_512M_NR_CONT (1UL << (KVX_PAGE_512M_SHIFT - PMD_SHIFT))
+
+#define KVX_PAGE_CP_SHIFT 2
+#define KVX_PAGE_CP_MASK KVX_SFR_TEL_CP_MASK
+
+
+#define _PAGE_CACHED (TLB_CP_W_C << KVX_PAGE_CP_SHIFT)
+#define _PAGE_UNCACHED (TLB_CP_U_U << KVX_PAGE_CP_SHIFT)
+#define _PAGE_DEVICE (TLB_CP_D_U << KVX_PAGE_CP_SHIFT)
+
+#define KVX_ACCESS_PERMS_BITS 4
+#define KVX_ACCESS_PERMS_OFFSET _PAGE_PERMS_SHIFT
+#define KVX_ACCESS_PERMS_SIZE (1 << KVX_ACCESS_PERMS_BITS)
+
+#define KVX_ACCESS_PERM_START_BIT KVX_ACCESS_PERMS_OFFSET
+#define KVX_ACCESS_PERM_STOP_BIT \
+ (KVX_ACCESS_PERMS_OFFSET + KVX_ACCESS_PERMS_BITS - 1)
+#define KVX_ACCESS_PERMS_MASK \
+ GENMASK(KVX_ACCESS_PERM_STOP_BIT, KVX_ACCESS_PERM_START_BIT)
+#define KVX_ACCESS_PERMS_INDEX(x) \
+ ((unsigned int)(x & KVX_ACCESS_PERMS_MASK) >> KVX_ACCESS_PERMS_OFFSET)
+
+/* Bits read, write, exec and global are not preserved across pte_modify() */
+#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_READ | _PAGE_WRITE | \
+ _PAGE_EXEC | _PAGE_GLOBAL))
+
+#endif
diff --git a/arch/kvx/include/asm/pgtable.h b/arch/kvx/include/asm/pgtable.h
new file mode 100644
index 000000000000..4d3a8af7d174
--- /dev/null
+++ b/arch/kvx/include/asm/pgtable.h
@@ -0,0 +1,451 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Marius Gligor
+ * Yann Sionneau
+ */
+
+#ifndef _ASM_KVX_PGTABLE_H
+#define _ASM_KVX_PGTABLE_H
+
+#include <linux/mmzone.h>
+#include <linux/mm_types.h>
+
+#include <asm/page.h>
+#include <asm/pgtable-bits.h>
+
+#include <asm-generic/pgtable-nopud.h>
+
+#include <asm/mem_map.h>
+
+struct mm_struct;
+struct vm_area_struct;
+
+/*
+ * Hugetlb definitions. All sizes are supported (64 KB, 2 MB and 512 MB).
+ */
+#if defined(CONFIG_KVX_4K_PAGES)
+#define HUGE_MAX_HSTATE 3
+#elif defined(CONFIG_KVX_64K_PAGES)
+#define HUGE_MAX_HSTATE 2
+#else
+#error "Unsupported page size"
+#endif
+
+#define HPAGE_SHIFT PMD_SHIFT
+#define HPAGE_SIZE BIT(HPAGE_SHIFT)
+#define HPAGE_MASK (~(HPAGE_SIZE - 1))
+#define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT)
+
+extern pte_t arch_make_huge_pte(pte_t entry, unsigned int shift,
+ vm_flags_t flags);
+#define arch_make_huge_pte arch_make_huge_pte
+
+/* Vmalloc definitions */
+#define VMALLOC_START KERNEL_VMALLOC_MAP_BASE
+#define VMALLOC_END (VMALLOC_START + KERNEL_VMALLOC_MAP_SIZE - 1)
+
+/* Also used by GDB script to go through the page table */
+#define PGDIR_BITS (VA_MAX_BITS - PGDIR_SHIFT)
+#define PMD_BITS (PGDIR_SHIFT - PMD_SHIFT)
+#define PTE_BITS (PMD_SHIFT - PAGE_SHIFT)
+
+/* Size of region mapped by a page global directory */
+#define PGDIR_SIZE BIT(PGDIR_SHIFT)
+#define PGDIR_MASK (~(PGDIR_SIZE - 1))
+
+/* Size of region mapped by a page middle directory */
+#define PMD_SIZE BIT(PMD_SHIFT)
+#define PMD_MASK (~(PMD_SIZE - 1))
+
+/* Number of entries in the page global directory */
+#define PAGES_PER_PGD 2
+#define PTRS_PER_PGD (PAGES_PER_PGD * PAGE_SIZE / sizeof(pgd_t))
+
+/* Number of entries in the page middle directory */
+#define PTRS_PER_PMD (PAGE_SIZE / sizeof(pmd_t))
+
+/* Number of entries in the page table */
+#define PTRS_PER_PTE (PAGE_SIZE / sizeof(pte_t))
+
+#define USER_PTRS_PER_PGD (TASK_SIZE/PGDIR_SIZE)
+
+extern pgd_t swapper_pg_dir[PTRS_PER_PGD];
+
+/* Page protection bits */
+#define _PAGE_BASE (_PAGE_PRESENT | _PAGE_CACHED)
+#define _PAGE_KERNEL (_PAGE_PRESENT | _PAGE_GLOBAL | \
+ _PAGE_READ | _PAGE_WRITE)
+#define _PAGE_KERNEL_EXEC (_PAGE_BASE | _PAGE_READ | _PAGE_EXEC | \
+ _PAGE_GLOBAL | _PAGE_WRITE)
+#define _PAGE_KERNEL_DEVICE (_PAGE_KERNEL | _PAGE_DEVICE)
+#define _PAGE_KERNEL_NOCACHE (_PAGE_KERNEL | _PAGE_UNCACHED)
+
+#define PAGE_NONE __pgprot(_PAGE_NONE)
+#define PAGE_READ __pgprot(_PAGE_BASE | _PAGE_READ)
+#define PAGE_READ_WRITE __pgprot(_PAGE_BASE | _PAGE_READ | _PAGE_WRITE)
+#define PAGE_READ_EXEC __pgprot(_PAGE_BASE | _PAGE_READ | _PAGE_EXEC)
+#define PAGE_READ_WRITE_EXEC __pgprot(_PAGE_BASE | _PAGE_READ | \
+ _PAGE_EXEC | _PAGE_WRITE)
+
+#define PAGE_KERNEL __pgprot(_PAGE_KERNEL | _PAGE_CACHED)
+#define PAGE_KERNEL_EXEC __pgprot(_PAGE_KERNEL_EXEC)
+#define PAGE_KERNEL_NOCACHE __pgprot(_PAGE_KERNEL | _PAGE_UNCACHED)
+#define PAGE_KERNEL_DEVICE __pgprot(_PAGE_KERNEL_DEVICE)
+#define PAGE_KERNEL_RO __pgprot((_PAGE_KERNEL | _PAGE_CACHED) & ~(_PAGE_WRITE))
+#define PAGE_KERNEL_ROX __pgprot(_PAGE_KERNEL_EXEC & ~(_PAGE_WRITE))
+
+#define pgprot_noncached(prot) (__pgprot((pgprot_val(prot) & ~KVX_PAGE_CP_MASK) | _PAGE_UNCACHED))
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern struct page *empty_zero_page;
+#define ZERO_PAGE(vaddr) (empty_zero_page)
+
+
+/*
+ * Encode and decode a swap entry. Swap entries are encoded in an arch
+ * dependent format as follows:
+ *
+ * +--------+----+-------+------+---+---+
+ * | 63..16 | 15 | 14..7 | 6..2 | 1 | 0 |
+ * +--------+----+-------+------+---+---+
+ * offset 0 0 type 0 0
+ *
+ * This allows for up to 31 swap files and 1PB per swap file.
+ */
+#define __SWP_TYPE_SHIFT 2
+#define __SWP_TYPE_BITS 5
+#define __SWP_TYPE_MASK ((1UL << __SWP_TYPE_BITS) - 1)
+#define __SWP_OFFSET_SHIFT 16
+
+#define MAX_SWAPFILES_CHECK() \
+ BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > __SWP_TYPE_BITS)
+
+#define __swp_type(x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK)
+#define __swp_offset(x) ((x).val >> __SWP_OFFSET_SHIFT)
+#define __swp_entry(type, offset) ((swp_entry_t) \
+ { ((type) << __SWP_TYPE_SHIFT) | ((offset) << __SWP_OFFSET_SHIFT) })
+
+#define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) })
+#define __swp_entry_to_pte(x) ((pte_t) { (x).val })
+
+/*
+ * PGD definitions:
+ * - pgd_ERROR
+ */
+#define pgd_ERROR(e) \
+ pr_err("%s:%d: bad pgd %016lx.\n", __FILE__, __LINE__, pgd_val(e))
+
+/*
+ * PUD
+ *
+ * As we manage a three level page table the call to set_pud is used to fill
+ * PGD.
+ */
+static inline void set_pud(pud_t *pudp, pud_t pmd)
+{
+ *pudp = pmd;
+}
+
+static inline int pud_none(pud_t pud)
+{
+ return pud_val(pud) == 0;
+}
+
+static inline int pud_bad(pud_t pud)
+{
+ return pud_none(pud);
+}
+static inline int pud_present(pud_t pud)
+{
+ return pud_val(pud) != 0;
+}
+
+static inline void pud_clear(pud_t *pud)
+{
+ set_pud(pud, __pud(0));
+}
+
+/*
+ * PMD definitions:
+ * - set_pmd
+ * - pmd_present
+ * - pmd_none
+ * - pmd_bad
+ * - pmd_clear
+ * - pmd_page
+ */
+
+static inline void set_pmd(pmd_t *pmdp, pmd_t pmd)
+{
+ *pmdp = pmd;
+}
+
+/* Returns 1 if entry is present */
+static inline int pmd_present(pmd_t pmd)
+{
+ return pmd_val(pmd) != 0;
+}
+
+/* Returns 1 if the corresponding entry has the value 0 */
+static inline int pmd_none(pmd_t pmd)
+{
+ return pmd_val(pmd) == 0;
+}
+
+/* Used to check that a page middle directory entry is valid */
+static inline int pmd_bad(pmd_t pmd)
+{
+ return pmd_none(pmd);
+}
+
+/* Clears the entry to prevent process to use the linear address that
+ * mapped it.
+ */
+static inline void pmd_clear(pmd_t *pmdp)
+{
+ set_pmd(pmdp, __pmd(0));
+}
+
+/*
+ * Returns the address of the descriptor of the page table referred by the
+ * PMD entry.
+ */
+static inline struct page *pmd_page(pmd_t pmd)
+{
+ if (pmd_val(pmd) & _PAGE_HUGE)
+ return pfn_to_page(
+ (pmd_val(pmd) & KVX_PFN_MASK) >> KVX_PFN_SHIFT);
+
+ return pfn_to_page(pmd_val(pmd) >> PAGE_SHIFT);
+}
+
+#define pmd_ERROR(e) \
+ pr_err("%s:%d: bad pmd %016lx.\n", __FILE__, __LINE__, pmd_val(e))
+
+static inline pmd_t *pud_pgtable(pud_t pud)
+{
+ return (pmd_t *)pfn_to_virt(pud_val(pud) >> PAGE_SHIFT);
+}
+
+static inline struct page *pud_page(pud_t pud)
+{
+ return pfn_to_page(pud_val(pud) >> PAGE_SHIFT);
+}
+
+/*
+ * PTE definitions:
+ * - set_pte
+ * - set_pte_at
+ * - pte_clear
+ * - pte_page
+ * - pte_pfn
+ * - pte_present
+ * - pte_none
+ * - pte_write
+ * - pte_dirty
+ * - pte_young
+ * - pte_special
+ * - pte_mkdirty
+ * - pte_mkwrite
+ * - pte_mkclean
+ * - pte_mkyoung
+ * - pte_mkold
+ * - pte_mkspecial
+ * - pte_wrprotect
+ */
+
+static inline void set_pte(pte_t *ptep, pte_t pteval)
+{
+ *ptep = pteval;
+}
+
+static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pteval)
+{
+ set_pte(ptep, pteval);
+}
+
+#define pte_clear(mm, addr, ptep) set_pte(ptep, __pte(0))
+
+/* Constructs a page table entry */
+static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot)
+{
+ return __pte(((pfn << KVX_PFN_SHIFT) & KVX_PFN_MASK) |
+ pgprot_val(prot));
+}
+
+/* Builds a page table entry by combining a page descriptor and a group of
+ * access rights.
+ */
+#define mk_pte(page, prot) (pfn_pte(page_to_pfn(page), prot))
+
+/* Modifies page access rights */
+static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
+{
+ return __pte((pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot));
+}
+
+#define pte_page(x) pfn_to_page(pte_pfn(x))
+
+static inline unsigned long pmd_page_vaddr(pmd_t pmd)
+{
+ return (unsigned long)pfn_to_virt(pmd_val(pmd) >> PAGE_SHIFT);
+}
+
+/* Yields the page frame number (PFN) of a page table entry */
+static inline unsigned long pte_pfn(pte_t pte)
+{
+ return ((pte_val(pte) & KVX_PFN_MASK) >> KVX_PFN_SHIFT);
+}
+
+static inline int pte_present(pte_t pte)
+{
+ return (pte_val(pte) & (_PAGE_PRESENT | _PAGE_NONE));
+}
+
+static inline int pte_none(pte_t pte)
+{
+ return (pte_val(pte) == 0);
+}
+
+static inline int pte_write(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_WRITE;
+}
+
+static inline int pte_dirty(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_DIRTY;
+}
+
+static inline int pte_young(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_ACCESSED;
+}
+
+static inline int pte_special(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_SPECIAL;
+}
+
+static inline int pte_huge(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_HUGE;
+}
+
+static inline pte_t pte_mkdirty(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_DIRTY);
+}
+
+static inline pte_t pte_mkwrite(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_WRITE);
+}
+
+static inline pte_t pte_mkclean(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~(_PAGE_DIRTY));
+}
+
+static inline pte_t pte_mkyoung(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_ACCESSED);
+}
+
+static inline pte_t pte_mkold(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~(_PAGE_ACCESSED));
+}
+
+static inline pte_t pte_mkspecial(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_SPECIAL);
+}
+
+static inline pte_t pte_wrprotect(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~(_PAGE_WRITE));
+}
+
+static inline pte_t pte_mkhuge(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_HUGE);
+}
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+
+#define pmdp_establish pmdp_establish
+static inline pmd_t pmdp_establish(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp, pmd_t pmd)
+{
+ return __pmd(xchg(&pmd_val(*pmdp), pmd_val(pmd)));
+}
+
+static inline int pmd_trans_huge(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_HUGE);
+}
+
+static inline pte_t pte_of_pmd(pmd_t pmd)
+{
+ return __pte(pmd_val(pmd));
+}
+
+static inline pmd_t pmd_of_pte(pte_t pte)
+{
+ return __pmd(pte_val(pte));
+}
+
+
+#define pmd_mkclean(pmd) pmd_of_pte(pte_mkclean(pte_of_pmd(pmd)))
+#define pmd_mkdirty(pmd) pmd_of_pte(pte_mkdirty(pte_of_pmd(pmd)))
+#define pmd_mkold(pmd) pmd_of_pte(pte_mkold(pte_of_pmd(pmd)))
+#define pmd_mkwrite(pmd) pmd_of_pte(pte_mkwrite(pte_of_pmd(pmd)))
+#define pmd_mkyoung(pmd) pmd_of_pte(pte_mkyoung(pte_of_pmd(pmd)))
+#define pmd_modify(pmd, prot) pmd_of_pte(pte_modify(pte_of_pmd(pmd), prot))
+#define pmd_wrprotect(pmd) pmd_of_pte(pte_wrprotect(pte_of_pmd(pmd)))
+
+static inline pmd_t pmd_mkhuge(pmd_t pmd)
+{
+ /* Create a huge page in PMD implies a size of 2 MB */
+ return __pmd(pmd_val(pmd) |
+ _PAGE_HUGE | (TLB_PS_2M << KVX_PAGE_SZ_SHIFT));
+}
+
+static inline pmd_t pmd_mkinvalid(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~(_PAGE_PRESENT);
+
+ return pmd;
+}
+
+#define pmd_dirty(pmd) pte_dirty(pte_of_pmd(pmd))
+#define pmd_write(pmd) pte_write(pte_of_pmd(pmd))
+#define pmd_young(pmd) pte_young(pte_of_pmd(pmd))
+
+#define mk_pmd(page, prot) pmd_of_pte(mk_pte(page, prot))
+
+#define pmd_pfn(pmd) pte_pfn(pte_of_pmd(pmd))
+
+static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
+{
+ return __pmd(((pfn << KVX_PFN_SHIFT) & KVX_PFN_MASK) |
+ pgprot_val(prot));
+}
+
+static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, pmd_t pmd)
+{
+ *pmdp = pmd;
+}
+
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
+#endif /* _ASM_KVX_PGTABLE_H */
diff --git a/arch/kvx/include/asm/rm_fw.h b/arch/kvx/include/asm/rm_fw.h
new file mode 100644
index 000000000000..f89bdd5915ed
--- /dev/null
+++ b/arch/kvx/include/asm/rm_fw.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_RM_FW_H
+#define _ASM_KVX_RM_FW_H
+
+#include <linux/sizes.h>
+
+#define KVX_RM_ID 16
+
+#define RM_FIRMWARE_REGS_SIZE (SZ_4K)
+
+#endif /* _ASM_KVX_RM_FW_H */
diff --git a/arch/kvx/include/asm/sparsemem.h b/arch/kvx/include/asm/sparsemem.h
new file mode 100644
index 000000000000..2f35743f20fb
--- /dev/null
+++ b/arch/kvx/include/asm/sparsemem.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SPARSEMEM_H
+#define _ASM_KVX_SPARSEMEM_H
+
+#ifdef CONFIG_SPARSEMEM
+#define MAX_PHYSMEM_BITS 40
+#define SECTION_SIZE_BITS 30
+#endif /* CONFIG_SPARSEMEM */
+
+#endif /* _ASM_KVX_SPARSEMEM_H */
diff --git a/arch/kvx/include/asm/symbols.h b/arch/kvx/include/asm/symbols.h
new file mode 100644
index 000000000000..a53c1607979f
--- /dev/null
+++ b/arch/kvx/include/asm/symbols.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SYMBOLS_H
+#define _ASM_KVX_SYMBOLS_H
+
+/* Symbols to patch TLB refill handler */
+extern char kvx_perf_tlb_refill[], kvx_std_tlb_refill[];
+
+/* Entry point of the ELF, used to start other PEs in SMP */
+extern int kvx_start[];
+
+#endif
diff --git a/arch/kvx/include/asm/tlb.h b/arch/kvx/include/asm/tlb.h
new file mode 100644
index 000000000000..190b682e1819
--- /dev/null
+++ b/arch/kvx/include/asm/tlb.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_TLB_H
+#define _ASM_KVX_TLB_H
+
+struct mmu_gather;
+
+static void tlb_flush(struct mmu_gather *tlb);
+
+int clear_ltlb_entry(unsigned long vaddr);
+
+#include <asm-generic/tlb.h>
+
+static inline unsigned int pgprot_cache_policy(unsigned long flags)
+{
+ return (flags & KVX_PAGE_CP_MASK) >> KVX_PAGE_CP_SHIFT;
+}
+
+#endif /* _ASM_KVX_TLB_H */
diff --git a/arch/kvx/include/asm/tlb_defs.h b/arch/kvx/include/asm/tlb_defs.h
new file mode 100644
index 000000000000..3f5b29cd529e
--- /dev/null
+++ b/arch/kvx/include/asm/tlb_defs.h
@@ -0,0 +1,131 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julian Vetter
+ * Guillaume Thouvenin
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_TLB_DEFS_H
+#define _ASM_KVX_TLB_DEFS_H
+
+#include <linux/sizes.h>
+
+#include <asm/sfr.h>
+
+/* Architecture specification */
+#define MMC_SB_JTLB 0
+#define MMC_SB_LTLB 1
+
+#define MMU_LTLB_SETS 1
+#define MMU_LTLB_WAYS 16
+
+#define MMU_JTLB_SETS 64
+#define MMU_JTLB_WAYS_SHIFT 2
+#define MMU_JTLB_WAYS (1 << MMU_JTLB_WAYS_SHIFT)
+
+#define MMU_JTLB_ENTRIES (MMU_JTLB_SETS << MMU_JTLB_WAYS_SHIFT)
+
+/* Set is determined using the 6 lsb of virtual page */
+#define MMU_JTLB_SET_MASK (MMU_JTLB_SETS - 1)
+#define MMU_JTLB_WAY_MASK (MMU_JTLB_WAYS - 1)
+
+/* TLB: Entry Status */
+#define TLB_ES_INVALID 0
+#define TLB_ES_PRESENT 1
+#define TLB_ES_MODIFIED 2
+#define TLB_ES_A_MODIFIED 3
+
+/* TLB: Cache Policy - First value is for data, the second is for instruction
+ * Symbols are
+ * D: device
+ * U: uncached
+ * W: write through
+ * C: cache enabled
+ */
+#define TLB_CP_D_U 0
+#define TLB_CP_U_U 1
+#define TLB_CP_W_C 2
+#define TLB_CP_U_C 3
+
+/* TLB: Protection Attributes: First value is when PM=0, second is when PM=1
+ * Symbols are:
+ * NA: no access
+ * R : read
+ * W : write
+ * X : execute
+ */
+#define TLB_PA_NA_NA 0
+#define TLB_PA_NA_R 1
+#define TLB_PA_NA_RW 2
+#define TLB_PA_NA_RX 3
+#define TLB_PA_NA_RWX 4
+#define TLB_PA_R_R 5
+#define TLB_PA_R_RW 6
+#define TLB_PA_R_RX 7
+#define TLB_PA_R_RWX 8
+#define TLB_PA_RW_RW 9
+#define TLB_PA_RW_RWX 10
+#define TLB_PA_RX_RX 11
+#define TLB_PA_RX_RWX 12
+#define TLB_PA_RWX_RWX 13
+
+/* TLB: Page Size */
+#define TLB_PS_4K 0
+#define TLB_PS_64K 1
+#define TLB_PS_2M 2
+#define TLB_PS_512M 3
+
+#define TLB_G_GLOBAL 1
+#define TLB_G_USE_ASN 0
+
+#define TLB_MK_TEH_ENTRY(_vaddr, _vs, _global, _asn) \
+ (((_vs) << KVX_SFR_TEH_VS_SHIFT) | \
+ ((_global) << KVX_SFR_TEH_G_SHIFT) | \
+ ((_asn) << KVX_SFR_TEH_ASN_SHIFT) | \
+ (((_vaddr) >> KVX_SFR_TEH_PN_SHIFT) << KVX_SFR_TEH_PN_SHIFT))
+
+#define TLB_MK_TEL_ENTRY(_paddr, _ps, _es, _cp, _pa) \
+ (((_es) << KVX_SFR_TEL_ES_SHIFT) | \
+ ((_ps) << KVX_SFR_TEL_PS_SHIFT) | \
+ ((_cp) << KVX_SFR_TEL_CP_SHIFT) | \
+ ((_pa) << KVX_SFR_TEL_PA_SHIFT) | \
+ (((_paddr) >> KVX_SFR_TEL_FN_SHIFT) << KVX_SFR_TEL_FN_SHIFT))
+
+
+/* Refill routine related defines */
+#define REFILL_PERF_ENTRIES 4
+#define REFILL_PERF_PAGE_SIZE SZ_512M
+
+/* paddr will be inserted in assembly code */
+#define REFILL_PERF_TEL_VAL \
+ TLB_MK_TEL_ENTRY(0, TLB_PS_512M, TLB_ES_A_MODIFIED, TLB_CP_W_C, \
+ TLB_PA_NA_RWX)
+/* vaddr will be inserted in assembly code */
+#define REFILL_PERF_TEH_VAL TLB_MK_TEH_ENTRY(0, 0, TLB_G_GLOBAL, 0)
+
+/*
+ * LTLB fixed entry index
+ */
+#define LTLB_ENTRY_KERNEL_TEXT 0
+#define LTLB_ENTRY_GDB_PAGE 1
+/* Reserve entries for kernel pagination */
+#define LTLB_KERNEL_RESERVED 2
+/* This define should reflect the maximum number of fixed LTLB entries */
+#define LTLB_ENTRY_FIXED_COUNT (LTLB_KERNEL_RESERVED + REFILL_PERF_ENTRIES)
+#define LTLB_ENTRY_EARLY_SMEM LTLB_ENTRY_FIXED_COUNT
+
+/* MMC: Protection Trap Cause */
+#define MMC_PTC_RESERVED 0
+#define MMC_PTC_READ 1
+#define MMC_PTC_WRITE 2
+#define MMC_PTC_EXECUTE 3
+
+/* MMC: Page size Mask in JTLB */
+#define MMC_PMJ_4K 1
+#define MMC_PMJ_64K 2
+#define MMC_PMJ_2M 4
+#define MMC_PMJ_512M 8
+
+#endif
diff --git a/arch/kvx/include/asm/tlbflush.h b/arch/kvx/include/asm/tlbflush.h
new file mode 100644
index 000000000000..7d3e50bbc957
--- /dev/null
+++ b/arch/kvx/include/asm/tlbflush.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_TLBFLUSH_H
+#define _ASM_KVX_TLBFLUSH_H
+
+#include <linux/sched.h>
+#include <linux/printk.h>
+#include <linux/mm_types.h>
+
+extern void local_flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long addr);
+extern void local_flush_tlb_all(void);
+extern void local_flush_tlb_mm(struct mm_struct *mm);
+extern void local_flush_tlb_range(struct vm_area_struct *vma,
+ unsigned long start,
+ unsigned long end);
+extern void local_flush_tlb_kernel_range(unsigned long start,
+ unsigned long end);
+
+#ifdef CONFIG_SMP
+extern void smp_flush_tlb_all(void);
+extern void smp_flush_tlb_mm(struct mm_struct *mm);
+extern void smp_flush_tlb_page(struct vm_area_struct *vma, unsigned long addr);
+extern void smp_flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end);
+extern void smp_flush_tlb_kernel_range(unsigned long start, unsigned long end);
+
+static inline void flush_tlb(void)
+{
+ smp_flush_tlb_mm(current->mm);
+}
+
+#define flush_tlb_page smp_flush_tlb_page
+#define flush_tlb_all smp_flush_tlb_all
+#define flush_tlb_mm smp_flush_tlb_mm
+#define flush_tlb_range smp_flush_tlb_range
+#define flush_tlb_kernel_range smp_flush_tlb_kernel_range
+
+#else
+#define flush_tlb_page local_flush_tlb_page
+#define flush_tlb_all local_flush_tlb_all
+#define flush_tlb_mm local_flush_tlb_mm
+#define flush_tlb_range local_flush_tlb_range
+#define flush_tlb_kernel_range local_flush_tlb_kernel_range
+#endif
+
+void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
+ pmd_t *pmd);
+
+void update_mmu_cache(struct vm_area_struct *vma,
+ unsigned long address, pte_t *ptep);
+
+#endif /* _ASM_KVX_TLBFLUSH_H */
diff --git a/arch/kvx/include/asm/vmalloc.h b/arch/kvx/include/asm/vmalloc.h
new file mode 100644
index 000000000000..a07692431108
--- /dev/null
+++ b/arch/kvx/include/asm/vmalloc.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_VMALLOC_H
+#define _ASM_KVX_VMALLOC_H
+
+#endif /* _ASM_KVX_VMALLOC_H */
diff --git a/arch/kvx/mm/cacheflush.c b/arch/kvx/mm/cacheflush.c
new file mode 100644
index 000000000000..2a632645f980
--- /dev/null
+++ b/arch/kvx/mm/cacheflush.c
@@ -0,0 +1,154 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/smp.h>
+#include <linux/hugetlb.h>
+#include <linux/mm_types.h>
+
+#include <asm/cacheflush.h>
+
+#ifdef CONFIG_SMP
+
+struct flush_data {
+ unsigned long start;
+ unsigned long end;
+};
+
+static inline void ipi_flush_icache_range(void *arg)
+{
+ struct flush_data *ta = arg;
+
+ local_flush_icache_range(ta->start, ta->end);
+}
+
+void flush_icache_range(unsigned long start, unsigned long end)
+{
+ struct flush_data data = {
+ .start = start,
+ .end = end
+ };
+
+ /* Then invalidate L1 icache on all cpus */
+ on_each_cpu(ipi_flush_icache_range, &data, 1);
+}
+EXPORT_SYMBOL(flush_icache_range);
+
+#endif /* CONFIG_SMP */
+
+void dcache_wb_inval_phys_range(phys_addr_t addr, unsigned long len, bool wb,
+ bool inval)
+{
+ if (wb && inval) {
+ wbinval_dcache_range(addr, len);
+ } else {
+ if (inval)
+ inval_dcache_range(addr, len);
+ if (wb)
+ wb_dcache_range(addr, len);
+ }
+}
+
+static inline pte_t *get_ptep(struct mm_struct *mm, unsigned long addr)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ pgd = pgd_offset(mm, addr);
+ if (pgd_none(*pgd))
+ return NULL;
+
+ p4d = p4d_offset(pgd, addr);
+ if (p4d_none(*p4d))
+ return NULL;
+
+ pud = pud_offset(p4d, addr);
+ if (pud_none(*pud))
+ return NULL;
+
+ pmd = pmd_offset(pud, addr);
+ if (pmd_none(*pmd))
+ return NULL;
+
+ if (pmd_huge(*pmd)) {
+ pte = (pte_t *) pmd;
+ if (!pte_present(*pte))
+ return NULL;
+
+ return pte;
+ }
+
+ pte = pte_offset_map(pmd, addr);
+ if (!pte_present(*pte))
+ return NULL;
+
+ return pte;
+}
+
+static unsigned long dcache_wb_inval_virt_to_phys(struct vm_area_struct *vma,
+ unsigned long vaddr,
+ unsigned long len,
+ bool wb, bool inval)
+{
+ unsigned long pfn, offset, pgsize;
+ pte_t *ptep;
+
+ ptep = get_ptep(vma->vm_mm, vaddr);
+ if (!ptep) {
+ /*
+ * Since we did not found a matching pte, return needed
+ * length to be aligned on next page boundary
+ */
+ offset = (vaddr & (PAGE_SIZE - 1));
+ return PAGE_SIZE - offset;
+ }
+
+ /* Handle page sizes correctly */
+ pgsize = (pte_val(*ptep) & KVX_PAGE_SZ_MASK) >> KVX_PAGE_SZ_SHIFT;
+ pgsize = (1 << get_page_size_shift(pgsize));
+
+ offset = vaddr & (pgsize - 1);
+ len = min(pgsize - offset, len);
+ pfn = pte_pfn(*ptep);
+
+ dcache_wb_inval_phys_range(PFN_PHYS(pfn) + offset, len, wb, inval);
+
+ return len;
+}
+
+int dcache_wb_inval_virt_range(unsigned long vaddr, unsigned long len, bool wb,
+ bool inval)
+{
+ unsigned long end = vaddr + len;
+ struct vm_area_struct *vma;
+ unsigned long rlen;
+ struct mm_struct *mm = current->mm;
+
+ /* necessary for find_vma */
+ mmap_read_lock(mm);
+
+ /*
+ * Verify that the specified address region actually belongs to this
+ * process.
+ */
+ vma = find_vma(current->mm, vaddr);
+ if (vma == NULL || vaddr < vma->vm_start || vaddr + len > vma->vm_end) {
+ mmap_read_unlock(mm);
+ return -EFAULT;
+ }
+
+ while (vaddr < end) {
+ rlen = dcache_wb_inval_virt_to_phys(vma, vaddr, len, wb, inval);
+ len -= rlen;
+ vaddr += rlen;
+ }
+
+ mmap_read_unlock(mm);
+
+ return 0;
+}
diff --git a/arch/kvx/mm/dma-mapping.c b/arch/kvx/mm/dma-mapping.c
new file mode 100644
index 000000000000..a9829432b974
--- /dev/null
+++ b/arch/kvx/mm/dma-mapping.c
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Jules Maselbas
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/iommu.h>
+
+#include <asm/cacheflush.h>
+
+void arch_dma_prep_coherent(struct page *page, size_t size)
+{
+ unsigned long addr = (unsigned long) page_to_phys(page);
+
+ /* Flush pending data and invalidate pages */
+ wbinval_dcache_range(addr, size);
+}
+
+/**
+ * The implementation of arch should follow the following rules:
+ * map for_cpu for_device unmap
+ * TO_DEV writeback none writeback none
+ * FROM_DEV invalidate invalidate(*) invalidate invalidate(*)
+ * BIDIR writeback invalidate writeback invalidate
+ *
+ * (*) - only necessary if the CPU speculatively prefetches.
+ *
+ * (see https://lkml.org/lkml/2018/5/18/979)
+ */
+void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir)
+{
+ switch (dir) {
+ case DMA_FROM_DEVICE:
+ inval_dcache_range(paddr, size);
+ break;
+
+ case DMA_TO_DEVICE:
+ case DMA_BIDIRECTIONAL:
+ wb_dcache_range(paddr, size);
+ break;
+
+ default:
+ BUG();
+ }
+}
+
+void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir)
+{
+ switch (dir) {
+ case DMA_TO_DEVICE:
+ break;
+ case DMA_FROM_DEVICE:
+#ifdef CONFIG_L2_CACHE
+ /* kvx does not do speculative loads by itself, however
+ * L2 cache lines are bigger than L1 cache lines, which can
+ * cause a "false sharing" situation where two L1 cache lines
+ * share the same L2 cache line. There is a high chance that
+ * the L2 cache line has already been refilled by a previous
+ * memory access.
+ */
+ inval_dcache_range(paddr, size);
+#endif
+ break;
+
+ case DMA_BIDIRECTIONAL:
+ inval_dcache_range(paddr, size);
+ break;
+
+ default:
+ BUG();
+ }
+}
+
+#ifdef CONFIG_IOMMU_DMA
+void arch_teardown_dma_ops(struct device *dev)
+{
+ dev->dma_ops = NULL;
+}
+#endif /* CONFIG_IOMMU_DMA*/
+
+void arch_setup_dma_ops(struct device *dev,
+ u64 dma_base,
+ u64 size,
+ const struct iommu_ops *iommu,
+ bool coherent)
+{
+ dev->dma_coherent = coherent;
+ if (iommu)
+ iommu_setup_dma_ops(dev, dma_base, size);
+}
diff --git a/arch/kvx/mm/extable.c b/arch/kvx/mm/extable.c
new file mode 100644
index 000000000000..8889a1a90a77
--- /dev/null
+++ b/arch/kvx/mm/extable.c
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/riscv/mm/extable.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+
+#include <linux/extable.h>
+#include <linux/module.h>
+#include <linux/uaccess.h>
+
+int fixup_exception(struct pt_regs *regs)
+{
+ const struct exception_table_entry *fixup;
+
+ fixup = search_exception_tables(regs->spc);
+ if (fixup) {
+ regs->spc = fixup->fixup;
+ return 1;
+ }
+ return 0;
+}
diff --git a/arch/kvx/mm/fault.c b/arch/kvx/mm/fault.c
new file mode 100644
index 000000000000..17f6325f3eab
--- /dev/null
+++ b/arch/kvx/mm/fault.c
@@ -0,0 +1,264 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Yann Sionneau
+ */
+
+#include <linux/types.h>
+#include <linux/uaccess.h>
+#include <linux/kernel.h> // only needed to panic
+#include <linux/printk.h>
+#include <linux/perf_event.h>
+#include <linux/sched.h>
+#include <linux/sched/debug.h>
+#include <linux/sched/signal.h>
+#include <linux/mm.h>
+
+#include <asm/mmu.h>
+#include <asm/traps.h>
+#include <asm/ptrace.h>
+#include <asm/pgtable.h>
+#include <asm/sfr_defs.h>
+#include <asm/current.h>
+#include <asm/tlbflush.h>
+
+static int handle_vmalloc_fault(uint64_t ea)
+{
+ /*
+ * Synchronize this task's top level page-table with
+ * the 'reference' page table.
+ * As we only have 2 or 3 level page table we don't need to
+ * deal with other levels.
+ */
+ unsigned long addr = ea & PAGE_MASK;
+ pgd_t *pgd_k, *pgd;
+ p4d_t *p4d_k, *p4d;
+ pud_t *pud_k, *pud;
+ pmd_t *pmd_k, *pmd;
+ pte_t *pte_k;
+
+ pgd = pgd_offset(current->active_mm, ea);
+ pgd_k = pgd_offset_k(ea);
+ if (!pgd_present(*pgd_k))
+ return 1;
+ set_pgd(pgd, *pgd_k);
+
+ p4d = p4d_offset(pgd, ea);
+ p4d_k = p4d_offset(pgd_k, ea);
+ if (!p4d_present(*p4d_k))
+ return 1;
+
+ pud = pud_offset(p4d, ea);
+ pud_k = pud_offset(p4d_k, ea);
+ if (!pud_present(*pud_k))
+ return 1;
+
+ pmd = pmd_offset(pud, ea);
+ pmd_k = pmd_offset(pud_k, ea);
+ if (!pmd_present(*pmd_k))
+ return 1;
+
+ /* Some other architectures set pmd to synchronize them but
+ * as we just synchronized the pgd we don't see how they can
+ * be different. Maybe we miss something so in case we
+ * put a guard here.
+ */
+ if (pmd_val(*pmd) != pmd_val(*pmd_k))
+ pr_err("%s: pmd not synchronized (0x%lx != 0x%lx)\n",
+ __func__, pmd_val(*pmd), pmd_val(*pmd_k));
+
+ pte_k = pte_offset_kernel(pmd_k, ea);
+ if (!pte_present(*pte_k)) {
+ pr_err("%s: PTE not present for 0x%llx\n",
+ __func__, ea);
+ return 1;
+ }
+
+ /* We refill the TLB now to avoid to take another nomapping
+ * trap.
+ */
+ kvx_mmu_jtlb_add_entry(addr, pte_k, 0);
+
+ return 0;
+}
+
+void do_page_fault(uint64_t es, uint64_t ea, struct pt_regs *regs)
+{
+ struct mm_struct *mm;
+ struct vm_area_struct *vma;
+ unsigned long flags, cause, vma_mask;
+ int code = SEGV_MAPERR;
+ vm_fault_t fault;
+
+ cause = kvx_sfr_field_val(es, ES, RWX);
+
+ /* We fault-in kernel-space virtual memory on-demand. The
+ * 'reference' page table is init_mm.pgd.
+ */
+ if (is_vmalloc_addr((void *) ea) && !user_mode(regs)) {
+ if (handle_vmalloc_fault(ea))
+ goto no_context;
+ return;
+ }
+
+ mm = current->mm;
+
+ /*
+ * If we're in an interrupt or have no user
+ * context, we must not take the fault..
+ */
+ if (unlikely(faulthandler_disabled() || !mm))
+ goto no_context;
+
+ /* By default we retry and fault task can be killed */
+ flags = FAULT_FLAG_DEFAULT;
+
+ if (user_mode(regs))
+ flags |= FAULT_FLAG_USER;
+
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, ea);
+
+retry:
+ mmap_read_lock(mm);
+
+ vma = find_vma(mm, ea);
+ if (!vma)
+ goto bad_area;
+ if (likely(vma->vm_start <= ea))
+ goto good_area;
+ if (unlikely(!(vma->vm_flags & VM_GROWSDOWN)))
+ goto bad_area;
+ if (unlikely(expand_stack(vma, ea)))
+ goto bad_area;
+
+good_area:
+ /* Handle access type */
+ switch (cause) {
+ case KVX_TRAP_RWX_FETCH:
+ vma_mask = VM_EXEC;
+ break;
+ case KVX_TRAP_RWX_READ:
+ vma_mask = VM_READ;
+ break;
+ case KVX_TRAP_RWX_WRITE:
+ vma_mask = VM_WRITE;
+ flags |= FAULT_FLAG_WRITE;
+ break;
+ /* Atomic are both read/write */
+ case KVX_TRAP_RWX_ATOMIC:
+ vma_mask = VM_WRITE | VM_READ;
+ flags |= FAULT_FLAG_WRITE;
+ break;
+ default:
+ panic("%s: unhandled cause %lu", __func__, cause);
+ }
+
+ if ((vma->vm_flags & vma_mask) != vma_mask) {
+ code = SEGV_ACCERR;
+ goto bad_area;
+ }
+
+ /*
+ * If for any reason we can not handle the fault we make sure that
+ * we exit gracefully rather then retry endlessly with the same
+ * result.
+ */
+ fault = handle_mm_fault(vma, ea, flags, regs);
+
+ /*
+ * If we need to retry but a fatal signal is pending, handle the
+ * signal first. We do not need to release the mmap_sem because it
+ * would already be released in __lock_page_or_retry in mm/filemap.c.
+ */
+ if (fault_signal_pending(fault, regs))
+ return;
+
+ if (unlikely(fault & VM_FAULT_ERROR)) {
+ if (fault & VM_FAULT_OOM)
+ goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
+ else if (fault & VM_FAULT_SIGBUS)
+ goto do_sigbus;
+ BUG();
+ }
+
+ if (unlikely((fault & VM_FAULT_RETRY) && (flags & FAULT_FLAG_ALLOW_RETRY))) {
+ flags |= FAULT_FLAG_TRIED;
+ /* No need to up_read(&mm->mmap_sem) as we would
+ * have already released it in __lock_page_or_retry().
+ * Look in mm/filemap.c for explanations.
+ */
+ goto retry;
+ }
+
+ /* Fault errors and retry case have been handled nicely */
+ mmap_read_unlock(mm);
+ return;
+
+bad_area:
+ mmap_read_unlock(mm);
+
+ if (user_mode(regs)) {
+ user_do_sig(regs, SIGSEGV, code, ea);
+ return;
+ }
+
+no_context:
+ /* Are we prepared to handle this kernel fault?
+ *
+ * (The kernel has valid exception-points in the source
+ * when it accesses user-memory. When it fails in one
+ * of those points, we find it in a table and do a jump
+ * to some fixup code that loads an appropriate error
+ * code)
+ */
+ if (fixup_exception(regs))
+ return;
+
+ /*
+ * Oops. The kernel tried to access some bad page. We'll have to
+ * terminate things with extreme prejudice.
+ */
+ bust_spinlocks(1);
+ if (kvx_sfr_field_val(es, ES, HTC) == KVX_TRAP_PROTECTION)
+ pr_alert(CUT_HERE "Kernel protection trap at virtual address %016llx\n",
+ ea);
+ else {
+ pr_alert(CUT_HERE "Unable to handle kernel %s at virtual address %016llx\n",
+ (ea < PAGE_SIZE) ? "NULL pointer dereference" :
+ "paging request", ea);
+ }
+ die(regs, ea, "Oops");
+ bust_spinlocks(0);
+ make_task_dead(SIGKILL);
+
+out_of_memory:
+ /*
+ * We ran out of memory, call the OOM killer, and return the userspace
+ * (which will retry the fault, or kill us if we got oom-killed).
+ */
+ mmap_read_unlock(mm);
+ if (!user_mode(regs))
+ goto no_context;
+ pagefault_out_of_memory();
+ return;
+
+do_sigbus:
+ mmap_read_unlock(mm);
+ /* Kernel mode? Handle exceptions or die */
+ if (!user_mode(regs))
+ goto no_context;
+
+ user_do_sig(regs, SIGBUS, BUS_ADRERR, ea);
+
+ return;
+
+}
+
+void do_writetoclean(uint64_t es, uint64_t ea, struct pt_regs *regs)
+{
+ panic("%s not implemented", __func__);
+}
diff --git a/arch/kvx/mm/hugetlbpage.c b/arch/kvx/mm/hugetlbpage.c
new file mode 100644
index 000000000000..c1291fdac708
--- /dev/null
+++ b/arch/kvx/mm/hugetlbpage.c
@@ -0,0 +1,317 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Jean-Christophe Pince
+ */
+
+#include <asm/page.h>
+
+#include <linux/bits.h>
+#include <linux/mm.h>
+#include <linux/hugetlb.h>
+
+/**
+ * get_nr_cont_huge_pages - return the number of contiguous pages
+ * @pte: a PTE
+ *
+ * Return: the number of contiguous entries used by the huge page according to
+ * the huge page size.
+ */
+static unsigned int get_nr_cont_huge_pages(pte_t pte)
+{
+ unsigned int psize, nr_cont;
+ unsigned long ptev = pte_val(pte);
+
+ psize = (ptev & KVX_PAGE_SZ_MASK) >> KVX_PAGE_SZ_SHIFT;
+
+ if (psize == TLB_PS_64K)
+ nr_cont = KVX_PAGE_64K_NR_CONT;
+ else if (psize == TLB_PS_512M)
+ nr_cont = KVX_PAGE_512M_NR_CONT;
+ else
+ /* Only page of 64 KB or 512 MB require more than 1 entry */
+ nr_cont = 1;
+
+ return nr_cont;
+}
+
+void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte)
+{
+ unsigned int nr_cont, i;
+
+ nr_cont = get_nr_cont_huge_pages(pte);
+
+ for (i = 0; i < nr_cont; i++, ptep++)
+ set_pte_at(mm, addr, ptep, pte);
+}
+
+pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+ unsigned long addr,
+ pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+ pte_t pte = huge_ptep_get(ptep);
+
+ nr_cont = get_nr_cont_huge_pages(pte);
+
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ if (pte_dirty(*ptep))
+ pte = pte_mkdirty(pte);
+ if (pte_young(*ptep))
+ pte = pte_mkyoung(pte);
+ pte_clear(mm, addr, ptep);
+ }
+
+ flush_tlb_mm(mm);
+ return pte;
+}
+
+static pte_t get_dirty_young_from_cont(pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+ pte_t pte_orig = huge_ptep_get(ptep);
+
+ nr_cont = get_nr_cont_huge_pages(pte_orig);
+
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ pte_t pte = huge_ptep_get(ptep);
+
+ if (pte_dirty(pte))
+ pte_orig = pte_mkdirty(pte_orig);
+
+ if (pte_young(pte))
+ pte_orig = pte_mkyoung(pte_orig);
+ }
+
+ return pte_orig;
+}
+
+int huge_ptep_set_access_flags(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep,
+ pte_t pte, int dirty)
+{
+ unsigned int nr_cont, i;
+ int flush = 0;
+ pte_t pte_tmp;
+
+ nr_cont = get_nr_cont_huge_pages(huge_ptep_get(ptep));
+
+ /*
+ * As it is done by ARM64, make sure we don't lose the dirty or young
+ * state. So first we read dirty and young bits from all contiguous
+ * PTEs and update the pte given as argument if needed.
+ */
+ pte_tmp = get_dirty_young_from_cont(ptep);
+ if (pte_dirty(pte_tmp))
+ pte = pte_mkdirty(pte);
+
+ if (pte_young(pte_tmp))
+ pte = pte_mkyoung(pte);
+
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ if (!pte_same(*ptep, pte)) {
+ set_pte_at(vma->vm_mm, addr, ptep, pte);
+ flush = 1;
+ }
+ }
+
+ if (flush)
+ flush_tlb_page(vma, addr);
+
+ return flush;
+}
+
+void huge_ptep_set_wrprotect(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+
+ nr_cont = get_nr_cont_huge_pages(huge_ptep_get(ptep));
+
+ for (i = 0; i < nr_cont; i++, ptep++)
+ ptep_set_wrprotect(mm, addr, ptep);
+}
+
+
+pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+ struct mm_struct *mm;
+ int flush = 0;
+ pte_t pte_orig = huge_ptep_get(ptep);
+
+ nr_cont = get_nr_cont_huge_pages(huge_ptep_get(ptep));
+
+ mm = vma->vm_mm;
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ BUG_ON(pte_orig.pte != pte_val(*ptep));
+ pte_clear(mm, address, ptep);
+ if (pte_accessible(mm, *ptep))
+ flush = 1;
+ }
+
+ if (flush)
+ flush_tlb_page(vma, addr);
+
+ return pte_orig;
+}
+
+pte_t arch_make_huge_pte(pte_t entry, unsigned int shift,
+ vm_flags_t flags)
+{
+ unsigned long ptev;
+
+ ptev = pte_val(entry) & ~(KVX_PAGE_SZ_MASK);
+
+ switch (shift) {
+ case KVX_PAGE_64K_SHIFT:
+ ptev |= _PAGE_SZ_64K;
+ break;
+ case KVX_PAGE_2M_SHIFT:
+ ptev |= _PAGE_SZ_2M;
+ break;
+ case KVX_PAGE_512M_SHIFT:
+ ptev |= _PAGE_SZ_512M;
+ break;
+ default:
+ ptev = 0;
+ pr_err("huge page shift %d not supported\n", shift);
+ BUG();
+ }
+
+ return __pte(ptev);
+}
+
+pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
+ unsigned long addr,
+ unsigned long size)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd = NULL;
+
+ pgd = pgd_offset(mm, addr);
+ if (pgd_present(*pgd)) {
+ p4d = p4d_offset(pgd, addr);
+ if (p4d_present(*p4d)) {
+ pud = pud_offset(p4d, addr);
+ if (pud_present(*pud))
+ pmd = pmd_alloc(mm, pud, addr);
+ }
+ }
+
+ if (size > KVX_PAGE_64K_SIZE)
+ return (pte_t *)pmd;
+
+ return pmd ? pte_alloc_map(mm, pmd, addr) : NULL;
+}
+
+/**
+ * huge_pte_offset - get the offset of the huge page
+ * @mm: the memory structure
+ * @addr: the address
+ * @size: size of the memory area
+ *
+ * On kvx the huge page are backed bye PMD or PTE depending of the size of the
+ * huge page. Huge pages larger or equal to 2 MB are backed by PMD, smaller are
+ * backed by PTE.
+ *
+ * Return: pointer to the huge page if it exists, NULL otherwise.
+ */
+pte_t *huge_pte_offset(struct mm_struct *mm,
+ unsigned long addr,
+ unsigned long size)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ /*
+ * 64 KB and 512 MB huge pages require the usage of contiguous page so
+ * we need to be aligned for this pages in order to get the right
+ * offset (ie the offset of the first index of contiguous pages).
+ */
+ if (size == KVX_PAGE_64K_SIZE)
+ addr &= KVX_PAGE_64K_MASK;
+ else if (size == KVX_PAGE_512M_SIZE)
+ addr &= KVX_PAGE_512M_MASK;
+
+ pgd = pgd_offset(mm, addr);
+ if (!pgd_present(*pgd))
+ return NULL;
+
+ p4d = p4d_offset(pgd, addr);
+ if (!p4d_present(*p4d))
+ return NULL;
+
+ pud = pud_offset(p4d, addr);
+ if (!pud_present(*pud))
+ return NULL;
+
+ pmd = pmd_offset(pud, addr);
+
+ if (size == KVX_PAGE_64K_SIZE) {
+ /* we need to go deeper in the page table */
+ if (pmd_present(*pmd))
+ pte = pte_offset_kernel(pmd, addr);
+ else
+ pte = NULL;
+ } else {
+ pte = (pte_t *)pmd;
+ }
+
+ return pte;
+}
+
+int pmd_huge(pmd_t pmd)
+{
+ return pmd_val(pmd) & _PAGE_HUGE;
+}
+
+int pud_huge(pud_t pud)
+{
+ return 0;
+}
+
+static void __init add_huge_page_size(unsigned long size)
+{
+ if (size_to_hstate(size)) {
+ WARN(1, "Failed to add huge page size %lu\n", size);
+ return;
+ }
+
+ hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
+}
+
+static int __init hugetlbpage_init(void)
+{
+#if defined(CONFIG_KVX_4K_PAGES)
+ add_huge_page_size(KVX_PAGE_64K_SIZE);
+ add_huge_page_size(KVX_PAGE_2M_SIZE);
+ add_huge_page_size(KVX_PAGE_512M_SIZE);
+#endif
+
+ return 0;
+}
+arch_initcall(hugetlbpage_init);
+
+bool __init arch_hugetlb_valid_size(unsigned long size)
+{
+#if defined(CONFIG_KVX_4K_PAGES)
+ switch (size) {
+ case KVX_PAGE_64K_SIZE:
+ case KVX_PAGE_2M_SIZE:
+ case KVX_PAGE_512M_SIZE:
+ return true;
+ }
+#endif
+
+ return false;
+}
diff --git a/arch/kvx/mm/init.c b/arch/kvx/mm/init.c
new file mode 100644
index 000000000000..abc4620d9cf3
--- /dev/null
+++ b/arch/kvx/mm/init.c
@@ -0,0 +1,527 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+/* Memblock header depends on types.h but does not include it ! */
+#include <linux/types.h>
+#include <linux/memblock.h>
+#include <linux/mmzone.h>
+#include <linux/of_fdt.h>
+#include <linux/sched.h>
+#include <linux/sizes.h>
+#include <linux/init.h>
+#include <linux/initrd.h>
+#include <linux/pfn.h>
+#include <linux/mm.h>
+
+#include <asm/sections.h>
+#include <asm/tlb_defs.h>
+#include <asm/tlbflush.h>
+#include <asm/fixmap.h>
+#include <asm/page.h>
+
+/*
+ * On kvx, memory map contains the first 2G of DDR being aliased.
+ * Full contiguous DDR is located at @[4G - 68G].
+ * However, to access this DDR in 32bit mode, the first 2G of DDR are
+ * mirrored from 4G to 2G.
+ * These first 2G are accessible from all DMAs (included 32 bits one).
+ *
+ * Hence, the memory map is the following:
+ *
+ * (68G) 0x1100000000-> +-------------+
+ * | |
+ * 66G |(ZONE_NORMAL)|
+ * | |
+ * (6G) 0x180000000-> +-------------+
+ * | |
+ * 2G |(ZONE_DMA32) |
+ * | |
+ * (4G) 0x100000000-> +-------------+ +--+
+ * | | |
+ * 2G | (Alias) | | 2G Alias
+ * | | |
+ * (2G) 0x80000000-> +-------------+ <--+
+ *
+ * The translation of 64 bits -> 32 bits can then be done using dma-ranges property
+ * in device-trees.
+ */
+
+#define DDR_64BIT_START (4ULL * SZ_1G)
+#define DDR_32BIT_ALIAS_SIZE (2ULL * SZ_1G)
+
+#define MAX_DMA32_PFN PHYS_PFN(DDR_64BIT_START + DDR_32BIT_ALIAS_SIZE)
+
+pgd_t swapper_pg_dir[PTRS_PER_PGD] __page_aligned_bss;
+
+/*
+ * empty_zero_page is a special page that is used for zero-initialized data and
+ * COW.
+ */
+struct page *empty_zero_page;
+EXPORT_SYMBOL(empty_zero_page);
+
+extern char _start[];
+extern char __kernel_smem_code_start[];
+extern char __kernel_smem_code_end[];
+
+struct kernel_section {
+ phys_addr_t start;
+ phys_addr_t end;
+};
+
+struct kernel_section kernel_sections[] = {
+ {
+ .start = (phys_addr_t)__kernel_smem_code_start,
+ .end = (phys_addr_t)__kernel_smem_code_end
+ },
+ {
+ .start = __pa(_start),
+ .end = __pa(_end)
+ }
+};
+
+static void __init zone_sizes_init(void)
+{
+ unsigned long zones_size[MAX_NR_ZONES];
+
+ memset(zones_size, 0, sizeof(zones_size));
+
+ zones_size[ZONE_DMA32] = min(MAX_DMA32_PFN, max_low_pfn);
+ zones_size[ZONE_NORMAL] = max_low_pfn;
+
+ free_area_init(zones_size);
+}
+
+#ifdef CONFIG_BLK_DEV_INITRD
+static void __init setup_initrd(void)
+{
+ u64 base = phys_initrd_start;
+ u64 end = phys_initrd_start + phys_initrd_size;
+
+ if (phys_initrd_size == 0) {
+ pr_info("initrd not found or empty");
+ return;
+ }
+
+ if (base < memblock_start_of_DRAM() || end > memblock_end_of_DRAM()) {
+ pr_err("initrd not in accessible memory, disabling it");
+ phys_initrd_size = 0;
+ return;
+ }
+
+ pr_info("initrd: 0x%llx - 0x%llx\n", base, end);
+
+ memblock_reserve(phys_initrd_start, phys_initrd_size);
+
+ /* the generic initrd code expects virtual addresses */
+ initrd_start = (unsigned long) __va(base);
+ initrd_end = initrd_start + phys_initrd_size;
+}
+#endif
+
+static phys_addr_t memory_limit = PHYS_ADDR_MAX;
+
+static int __init early_mem(char *p)
+{
+ if (!p)
+ return 1;
+
+ memory_limit = memparse(p, &p) & PAGE_MASK;
+ pr_notice("Memory limited to %lldMB\n", memory_limit >> 20);
+
+ return 0;
+}
+early_param("mem", early_mem);
+
+static void __init setup_bootmem(void)
+{
+ phys_addr_t kernel_start, kernel_end;
+ phys_addr_t start, end = 0;
+ u64 i;
+
+ init_mm.start_code = (unsigned long)_stext;
+ init_mm.end_code = (unsigned long)_etext;
+ init_mm.end_data = (unsigned long)_edata;
+ init_mm.brk = (unsigned long)_end;
+
+ for (i = 0; i < ARRAY_SIZE(kernel_sections); i++) {
+ kernel_start = kernel_sections[i].start;
+ kernel_end = kernel_sections[i].end;
+
+ memblock_reserve(kernel_start, kernel_end - kernel_start);
+ }
+
+ for_each_mem_range(i, &start, &end) {
+ pr_info("%15s: memory : 0x%lx - 0x%lx\n", __func__,
+ (unsigned long)start,
+ (unsigned long)end);
+ }
+
+ /* min_low_pfn is the lowest PFN available in the system */
+ min_low_pfn = PFN_UP(memblock_start_of_DRAM());
+
+ /* max_low_pfn indicates the end if NORMAL zone */
+ max_low_pfn = PFN_DOWN(memblock_end_of_DRAM());
+
+ /* Set the maximum number of pages in the system */
+ set_max_mapnr(max_low_pfn - min_low_pfn);
+
+#ifdef CONFIG_BLK_DEV_INITRD
+ setup_initrd();
+#endif
+
+ if (memory_limit != PHYS_ADDR_MAX)
+ memblock_mem_limit_remove_map(memory_limit);
+
+ /* Don't reserve the device tree if its builtin */
+ if (!is_kernel_rodata((unsigned long) initial_boot_params))
+ early_init_fdt_reserve_self();
+ early_init_fdt_scan_reserved_mem();
+
+ memblock_allow_resize();
+ memblock_dump_all();
+}
+
+static void * __init alloc_page_table(void)
+{
+ void *pgt;
+
+ pgt = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+ BUG_ON(!pgt);
+
+ return pgt;
+}
+
+static pmd_t fixmap_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused;
+static pte_t fixmap_pte[PTRS_PER_PTE] __page_aligned_bss __maybe_unused;
+
+void __init early_fixmap_init(void)
+{
+ unsigned long vaddr;
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+
+ /*
+ * Fixed mappings:
+ */
+ vaddr = __fix_to_virt(__end_of_fixed_addresses - 1);
+ pgd = pgd_offset_pgd(swapper_pg_dir, vaddr);
+ set_pgd(pgd, __pgd(__pa_symbol(fixmap_pmd)));
+
+ p4d = p4d_offset(pgd, vaddr);
+ pud = pud_offset(p4d, vaddr);
+ pmd = pmd_offset(pud, vaddr);
+ set_pmd(pmd, __pmd(__pa_symbol(fixmap_pte)));
+}
+
+#ifdef CONFIG_STRICT_KERNEL_RWX
+
+static bool use_huge_page(unsigned long start, unsigned long end,
+ phys_addr_t phys, unsigned long page_size)
+{
+ unsigned long size = end - start;
+
+ return IS_ALIGNED(start | phys, page_size) && (size >= page_size);
+}
+
+static void create_pte_mapping(pmd_t *pmdp, unsigned long va_start,
+ unsigned long va_end, phys_addr_t phys,
+ pgprot_t prot, bool alloc_pgtable)
+{
+ pmd_t pmd = READ_ONCE(*pmdp);
+ unsigned long nr_cont, i;
+ pte_t *ptep = NULL;
+ pgprot_t pte_prot;
+
+ if (pmd_none(pmd)) {
+ BUG_ON(!alloc_pgtable);
+ ptep = alloc_page_table();
+ set_pmd(pmdp, __pmd(__pa(ptep)));
+ pmd = READ_ONCE(*pmdp);
+ }
+ BUG_ON(pmd_bad(pmd));
+
+ ptep = pte_offset_kernel(pmdp, va_start);
+
+ do {
+ /* Try to use 64K page whenever it is possible */
+ if (use_huge_page(va_start, va_end, phys, KVX_PAGE_64K_SIZE)) {
+ pte_prot = __pgprot(pgprot_val(prot) | _PAGE_SZ_64K |
+ _PAGE_HUGE);
+ nr_cont = KVX_PAGE_64K_NR_CONT;
+ } else {
+ pte_prot = prot;
+ nr_cont = 1;
+ }
+
+ for (i = 0; i < nr_cont; i++) {
+ set_pte(ptep, pfn_pte(phys_to_pfn(phys), pte_prot));
+ ptep++;
+ }
+
+ phys += nr_cont * PAGE_SIZE;
+ va_start += nr_cont * PAGE_SIZE;
+ } while (va_start != va_end);
+}
+
+static void create_pmd_mapping(pgd_t *pgdp, unsigned long va_start,
+ unsigned long va_end, phys_addr_t phys,
+ pgprot_t prot, bool alloc_pgtable)
+{
+ unsigned long next, huge_size, i, nr_cont;
+ pgprot_t pmd_prot;
+ bool use_huge;
+ pmd_t *pmdp;
+ pud_t *pudp;
+ p4d_t *p4dp;
+ pud_t pud;
+ pte_t pte;
+
+ p4dp = p4d_offset(pgdp, va_start);
+ pudp = pud_offset(p4dp, va_start);
+ pud = READ_ONCE(*pudp);
+
+ if (pud_none(pud)) {
+ BUG_ON(!alloc_pgtable);
+ pmdp = alloc_page_table();
+ set_pud(pudp, __pud(__pa(pmdp)));
+ pud = READ_ONCE(*pudp);
+ }
+ BUG_ON(pud_bad(pud));
+
+ pmdp = pmd_offset(pudp, va_start);
+
+ do {
+ next = pmd_addr_end(va_start, va_end);
+
+ /* Try to use huge pages (2M, 512M) whenever it is possible */
+ if (use_huge_page(va_start, next, phys, KVX_PAGE_2M_SIZE)) {
+ pmd_prot = __pgprot(pgprot_val(prot) | _PAGE_SZ_2M);
+ nr_cont = 1;
+ huge_size = KVX_PAGE_2M_SIZE;
+ use_huge = true;
+ } else if (use_huge_page(va_start, next, phys,
+ KVX_PAGE_512M_SIZE)) {
+ pmd_prot = __pgprot(pgprot_val(prot) | _PAGE_SZ_512M);
+ nr_cont = KVX_PAGE_512M_NR_CONT;
+ huge_size = KVX_PAGE_512M_SIZE;
+ use_huge = true;
+ } else {
+ use_huge = false;
+ }
+
+ if (use_huge) {
+ pmd_prot = __pgprot(pgprot_val(pmd_prot) | _PAGE_HUGE);
+ pte = pfn_pte(phys_to_pfn(phys), pmd_prot);
+ for (i = 0; i < nr_cont; i++) {
+ set_pmd(pmdp, __pmd(pte_val(pte)));
+ pmdp++;
+ }
+ } else {
+ create_pte_mapping(pmdp, va_start, next, phys, prot,
+ alloc_pgtable);
+ pmdp++;
+ }
+
+ phys += next - va_start;
+ } while (va_start = next, va_start != va_end);
+}
+
+static void create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys,
+ unsigned long va_start, unsigned long va_end,
+ pgprot_t prot, bool alloc_pgtable)
+{
+ unsigned long next;
+ pgd_t *pgdp = pgd_offset_pgd(pgdir, va_start);
+
+ BUG_ON(!PAGE_ALIGNED(phys));
+ BUG_ON(!PAGE_ALIGNED(va_start));
+ BUG_ON(!PAGE_ALIGNED(va_end));
+
+ do {
+ next = pgd_addr_end(va_start, va_end);
+ create_pmd_mapping(pgdp, va_start, next, phys, prot,
+ alloc_pgtable);
+ phys += next - va_start;
+ } while (pgdp++, va_start = next, va_start != va_end);
+}
+
+static void __init map_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end,
+ pgprot_t prot)
+{
+ phys_addr_t pa_start = __pa(va_start);
+
+ create_pgd_mapping(pgdp, pa_start, (unsigned long) va_start,
+ (unsigned long) va_end, prot, true);
+}
+
+static void remap_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end,
+ pgprot_t prot)
+{
+ phys_addr_t pa_start = __pa(va_start);
+
+ create_pgd_mapping(pgdp, pa_start, (unsigned long) va_start,
+ (unsigned long) va_end, prot, false);
+
+ flush_tlb_kernel_range((unsigned long) va_start,
+ (unsigned long) va_end);
+}
+
+/*
+ * Create fine-grained mappings for the kernel.
+ */
+static void __init map_kernel(void)
+{
+ pgprot_t text_prot = PAGE_KERNEL_ROX;
+
+ /*
+ * External debuggers may need to write directly to the text
+ * mapping to install SW breakpoints. Allow this (only) when
+ * explicitly requested with rodata=off.
+ */
+ if (!rodata_enabled)
+ text_prot = PAGE_KERNEL_EXEC;
+
+ map_kernel_segment(swapper_pg_dir, __inittext_start, __inittext_end,
+ text_prot);
+ map_kernel_segment(swapper_pg_dir, __initdata_start, __initdata_end,
+ PAGE_KERNEL);
+ map_kernel_segment(swapper_pg_dir, __rodata_start, __rodata_end,
+ PAGE_KERNEL);
+ map_kernel_segment(swapper_pg_dir, _sdata, _end, PAGE_KERNEL);
+ /* We skip exceptions mapping to avoid multimappings */
+ map_kernel_segment(swapper_pg_dir, __exception_end, _etext, text_prot);
+}
+
+static void __init map_memory(void)
+{
+ phys_addr_t start, end;
+ phys_addr_t kernel_start = __pa(__inittext_start);
+ phys_addr_t kernel_end = __pa_symbol(_end);
+ u64 i;
+
+ /**
+ * Mark the full kernel text/data as nomap to avoid remapping all
+ * section as RW.
+ */
+ memblock_mark_nomap(kernel_start, kernel_end - kernel_start);
+
+ /* Map all memory banks */
+ for_each_mem_range(i, &start, &end) {
+
+ if (start >= end)
+ break;
+
+ create_pgd_mapping(swapper_pg_dir, start,
+ (unsigned long) __va(start),
+ (unsigned long) __va(end), PAGE_KERNEL_EXEC,
+ true);
+ }
+ memblock_clear_nomap(kernel_start, kernel_end - kernel_start);
+}
+
+void mark_rodata_ro(void)
+{
+ remap_kernel_segment(swapper_pg_dir, __rodata_start, __rodata_end,
+ PAGE_KERNEL_RO);
+}
+
+void __init paging_init(void)
+{
+ map_kernel();
+ map_memory();
+ init_kernel_rwx();
+}
+
+static int __init parse_rodata(char *arg)
+{
+ strtobool(arg, &rodata_enabled);
+
+ return 0;
+}
+early_param("rodata", parse_rodata);
+
+#else
+
+static void remap_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end,
+ pgprot_t prot)
+{
+}
+
+#endif
+
+void __init setup_arch_memory(void)
+{
+ setup_bootmem();
+ sparse_init();
+ zone_sizes_init();
+}
+
+void __init mem_init(void)
+{
+ memblock_free_all();
+
+ /* allocate the zero page */
+ empty_zero_page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+ if (!empty_zero_page)
+ panic("Failed to allocate the empty_zero_page");
+}
+
+void free_initmem(void)
+{
+ /* Remap init text as RW to allow writing to it */
+ remap_kernel_segment(swapper_pg_dir, __inittext_start, __inittext_end,
+ PAGE_KERNEL);
+
+#ifdef CONFIG_POISON_INITMEM
+ free_initmem_default(0x0);
+#else
+ free_initmem_default(-1);
+#endif
+}
+
+void __set_fixmap(enum fixed_addresses idx,
+ phys_addr_t phys, pgprot_t flags)
+{
+ unsigned long addr = __fix_to_virt(idx);
+ pte_t *pte;
+
+
+ BUG_ON(idx >= __end_of_fixed_addresses);
+
+ pte = &fixmap_pte[pte_index(addr)];
+
+ if (pgprot_val(flags)) {
+ set_pte(pte, pfn_pte(phys_to_pfn(phys), flags));
+ } else {
+ /* Remove the fixmap */
+ pte_clear(&init_mm, addr, pte);
+ }
+ local_flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
+}
+
+static const pgprot_t protection_map[16] = {
+ [VM_NONE] = PAGE_NONE,
+ [VM_READ] = PAGE_READ,
+ [VM_WRITE] = PAGE_READ,
+ [VM_WRITE | VM_READ] = PAGE_READ,
+ [VM_EXEC] = PAGE_READ_EXEC,
+ [VM_EXEC | VM_READ] = PAGE_READ_EXEC,
+ [VM_EXEC | VM_WRITE] = PAGE_READ_EXEC,
+ [VM_EXEC | VM_WRITE | VM_READ] = PAGE_READ_EXEC,
+ [VM_SHARED] = PAGE_NONE,
+ [VM_SHARED | VM_READ] = PAGE_READ,
+ [VM_SHARED | VM_WRITE] = PAGE_READ_WRITE,
+ [VM_SHARED | VM_WRITE | VM_READ] = PAGE_READ_WRITE,
+ [VM_SHARED | VM_EXEC] = PAGE_READ_EXEC,
+ [VM_SHARED | VM_EXEC | VM_READ] = PAGE_READ_EXEC,
+ [VM_SHARED | VM_EXEC | VM_WRITE] = PAGE_READ_WRITE_EXEC,
+ [VM_SHARED | VM_EXEC | VM_WRITE | VM_READ] = PAGE_READ_WRITE_EXEC
+};
+DECLARE_VM_GET_PAGE_PROT
diff --git a/arch/kvx/mm/kernel_rwx.c b/arch/kvx/mm/kernel_rwx.c
new file mode 100644
index 000000000000..efa502bab336
--- /dev/null
+++ b/arch/kvx/mm/kernel_rwx.c
@@ -0,0 +1,228 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julian Vetter
+ */
+
+#include <linux/init.h>
+#include <linux/sysfs.h>
+#include <linux/kernel.h>
+#include <linux/device.h>
+#include <linux/kobject.h>
+
+#include <asm/mmu.h>
+#include <asm/tlb.h>
+#include <asm/insns.h>
+#include <asm/sections.h>
+#include <asm/symbols.h>
+#include <asm/insns_defs.h>
+
+#define PERF_REFILL_INSN_SIZE (KVX_INSN_GOTO_SIZE * KVX_INSN_SYLLABLE_WIDTH)
+
+struct kobject *kvx_kobj;
+
+static bool kernel_rwx = true;
+static u32 perf_refill_insn;
+
+static DEFINE_MUTEX(kernel_rwx_mutex);
+
+static int __init parse_kernel_rwx(char *arg)
+{
+ strtobool(arg, &kernel_rwx);
+
+ return 0;
+}
+early_param("kvx.kernel_rwx", parse_kernel_rwx);
+
+static void map_exception_only_in_ltlb(void)
+{
+ struct kvx_tlb_format tlbe;
+
+ tlbe = tlb_mk_entry(
+ (void *)__pa(__exception_start),
+ (void *)__exception_start,
+ TLB_PS_4K,
+ TLB_G_GLOBAL,
+ TLB_PA_NA_RX,
+ TLB_CP_W_C,
+ 0,
+ TLB_ES_A_MODIFIED);
+
+ BUG_ON((__exception_end - __exception_start) > PAGE_SIZE);
+
+ kvx_mmu_add_entry(MMC_SB_LTLB, LTLB_ENTRY_KERNEL_TEXT, tlbe);
+}
+
+static void map_kernel_in_ltlb(void)
+{
+ struct kvx_tlb_format tlbe;
+
+ tlbe = tlb_mk_entry(
+ (void *) PHYS_OFFSET,
+ (void *) PAGE_OFFSET,
+ TLB_PS_512M,
+ TLB_G_GLOBAL,
+ TLB_PA_NA_RWX,
+ TLB_CP_W_C,
+ 0,
+ TLB_ES_A_MODIFIED);
+
+ kvx_mmu_add_entry(MMC_SB_LTLB, LTLB_ENTRY_KERNEL_TEXT, tlbe);
+}
+
+static void mmu_disable_kernel_perf_refill(void)
+{
+ unsigned int off = kvx_std_tlb_refill - kvx_perf_tlb_refill;
+ u32 goto_insn;
+ int ret;
+
+ BUG_ON(KVX_INSN_GOTO_PCREL27_CHECK(off));
+
+ KVX_INSN_GOTO(&goto_insn, KVX_INSN_PARALLEL_EOB, off);
+
+ ret = kvx_insns_read(&perf_refill_insn, PERF_REFILL_INSN_SIZE,
+ (u32 *) kvx_perf_tlb_refill);
+ BUG_ON(ret);
+ ret = kvx_insns_write(&goto_insn, PERF_REFILL_INSN_SIZE,
+ (u32 *) kvx_perf_tlb_refill);
+ BUG_ON(ret);
+}
+
+static void enable_kernel_perf_refill(void)
+{
+ int ret;
+
+ ret = kvx_insns_write(&perf_refill_insn, PERF_REFILL_INSN_SIZE,
+ (u32 *) kvx_perf_tlb_refill);
+ BUG_ON(ret);
+}
+
+static void local_mmu_enable_kernel_rwx(void)
+{
+ int i;
+ struct kvx_tlb_format tlbe;
+
+ tlbe = tlb_mk_entry(0, (void *) 0, 0, 0, 0, 0, 0,
+ TLB_ES_INVALID);
+
+ /* Map exceptions handlers in LTLB entry instead of full kernel */
+ map_exception_only_in_ltlb();
+
+ /* Invalidate previously added LTLB entries */
+ for (i = 0; i < REFILL_PERF_ENTRIES; i++)
+ kvx_mmu_add_entry(MMC_SB_LTLB, LTLB_KERNEL_RESERVED + i, tlbe);
+
+}
+
+/**
+ * init_kernel_rwx - Initialize kernel struct RWX at boot time
+ * This function MUST be used only at boot time to setup kernel strict RWX mode
+ * Once done, kernel rwx mode can be enabled/disabled using sysfs entry.
+ */
+void init_kernel_rwx(void)
+{
+ /* Kernel strict RWX mode disabled */
+ if (!kernel_rwx)
+ return;
+
+ /* First processor only will disable perf refill by patching code */
+ if (raw_smp_processor_id() == 0)
+ mmu_disable_kernel_perf_refill();
+
+ local_mmu_enable_kernel_rwx();
+}
+
+static void ipi_enable_kernel_rwx(void *arg)
+{
+ local_mmu_enable_kernel_rwx();
+}
+
+static void local_mmu_disable_kernel_rwx(void)
+{
+ /* Map full kernel in LTLB entry instead of only exceptions */
+ map_kernel_in_ltlb();
+
+ /* Flush JTLB completely to force refill and avoid stalled entries */
+ local_flush_tlb_all();
+}
+
+static void ipi_disable_kernel_rwx(void *arg)
+{
+ local_mmu_disable_kernel_rwx();
+}
+
+static void smp_set_kernel_rwx(bool kernel_rwx)
+{
+ smp_call_func_t fn;
+
+ pr_info("%sabling kernel rwx mode\n", kernel_rwx ? "En" : "Dis");
+
+ if (kernel_rwx) {
+ mmu_disable_kernel_perf_refill();
+ fn = ipi_enable_kernel_rwx;
+ } else {
+ enable_kernel_perf_refill();
+ fn = ipi_disable_kernel_rwx;
+ }
+
+ on_each_cpu(fn, NULL, 1);
+}
+
+static ssize_t kernel_rwx_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return sprintf(buf, "%s\n", kernel_rwx ? "enabled" : "disabled");
+}
+
+static ssize_t kernel_rwx_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t len)
+{
+ bool value;
+ int ret;
+
+ ret = strtobool(buf, &value);
+ if (ret)
+ return ret;
+
+ mutex_lock(&kernel_rwx_mutex);
+ /* Switch only if necessary */
+ if (value != kernel_rwx) {
+ kernel_rwx = value;
+ smp_set_kernel_rwx(kernel_rwx);
+ }
+ mutex_unlock(&kernel_rwx_mutex);
+
+ return len;
+}
+
+static struct kobj_attribute kernel_rwx_attr =
+ __ATTR(kernel_rwx, 0644, kernel_rwx_show, kernel_rwx_store);
+
+static struct attribute *default_attrs[] = {
+ &kernel_rwx_attr.attr,
+ NULL
+};
+
+static struct attribute_group kernel_rwx_attr_group = {
+ .attrs = default_attrs,
+};
+
+static int __init kvx_kernel_rwx_init(void)
+{
+ int ret;
+
+ kvx_kobj = kobject_create_and_add("kvx", NULL);
+ if (!kvx_kobj)
+ return -ENOMEM;
+
+ ret = sysfs_create_group(kvx_kobj, &kernel_rwx_attr_group);
+ if (ret) {
+ kobject_put(kvx_kobj);
+ return ret;
+ }
+
+ return 0;
+}
+postcore_initcall(kvx_kernel_rwx_init);
diff --git a/arch/kvx/mm/mmap.c b/arch/kvx/mm/mmap.c
new file mode 100644
index 000000000000..a2225db64438
--- /dev/null
+++ b/arch/kvx/mm/mmap.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/arm64/mm/mmap.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifdef CONFIG_STRICT_DEVMEM
+
+#include <linux/mm.h>
+#include <linux/ioport.h>
+
+#include <asm/page.h>
+
+/*
+ * devmem_is_allowed() checks to see if /dev/mem access to a certain address
+ * is valid. The argument is a physical page number. We mimic x86 here by
+ * disallowing access to system RAM as well as device-exclusive MMIO regions.
+ * This effectively disable read()/write() on /dev/mem.
+ */
+int devmem_is_allowed(unsigned long pfn)
+{
+ if (iomem_is_exclusive(pfn << PAGE_SHIFT))
+ return 0;
+ if (!page_is_ram(pfn))
+ return 1;
+ return 0;
+}
+
+#endif
diff --git a/arch/kvx/mm/mmu.c b/arch/kvx/mm/mmu.c
new file mode 100644
index 000000000000..a1eadececb6a
--- /dev/null
+++ b/arch/kvx/mm/mmu.c
@@ -0,0 +1,204 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Vincent Chardon
+ * Jules Maselbas
+ */
+
+#include <linux/cache.h>
+#include <linux/types.h>
+#include <linux/irqflags.h>
+#include <linux/printk.h>
+#include <linux/percpu.h>
+#include <linux/kernel.h>
+#include <linux/spinlock.h>
+#include <linux/spinlock_types.h>
+
+#include <asm/mmu.h>
+#include <asm/tlb.h>
+#include <asm/page_size.h>
+#include <asm/mmu_context.h>
+
+#define DUMP_LTLB 0
+#define DUMP_JTLB 1
+
+DEFINE_PER_CPU_ALIGNED(uint8_t[MMU_JTLB_SETS], jtlb_current_set_way);
+static struct kvx_tlb_format ltlb_entries[MMU_LTLB_WAYS];
+static unsigned long ltlb_entries_bmp;
+
+static int kvx_mmu_ltlb_overlaps(struct kvx_tlb_format tlbe)
+{
+ int bit = LTLB_ENTRY_FIXED_COUNT;
+
+ for_each_set_bit_from(bit, <lb_entries_bmp, MMU_LTLB_WAYS) {
+ if (tlb_entry_overlaps(tlbe, ltlb_entries[bit]))
+ return 1;
+ }
+
+ return 0;
+}
+
+/**
+ * kvx_mmu_ltlb_add_entry - Add a kernel entry in the LTLB
+ *
+ * In order to lock some entries in the LTLB and be always mapped, this
+ * function can be called with a physical address, a virtual address and
+ * protection attribute to add an entry into the LTLB. This is mainly for
+ * performances since there won't be any NOMAPPING traps for these pages.
+ *
+ * @vaddr: Virtual address for the entry (must be aligned according to tlb_ps)
+ * @paddr: Physical address for the entry (must be aligned according to tlb_ps)
+ * @flags: Protection attributes
+ * @tlb_ps: Page size attribute for TLB (TLB_PS_*)
+ */
+void kvx_mmu_ltlb_add_entry(unsigned long vaddr, phys_addr_t paddr,
+ pgprot_t flags, unsigned long tlb_ps)
+{
+ unsigned int cp;
+ unsigned int idx;
+ unsigned long irqflags;
+ struct kvx_tlb_format tlbe;
+ u64 page_size = BIT(get_page_size_shift(tlb_ps));
+
+ BUG_ON(!IS_ALIGNED(vaddr, page_size) || !IS_ALIGNED(paddr, page_size));
+
+ cp = pgprot_cache_policy(pgprot_val(flags));
+
+ tlbe = tlb_mk_entry(
+ (void *) paddr,
+ (void *) vaddr,
+ tlb_ps,
+ TLB_G_GLOBAL,
+ TLB_PA_NA_RW,
+ cp,
+ 0,
+ TLB_ES_A_MODIFIED);
+
+ local_irq_save(irqflags);
+
+ if (IS_ENABLED(CONFIG_KVX_DEBUG_TLB_WRITE) &&
+ kvx_mmu_ltlb_overlaps(tlbe))
+ panic("VA %lx overlaps with an existing LTLB mapping", vaddr);
+
+ idx = find_next_zero_bit(<lb_entries_bmp, MMU_LTLB_WAYS,
+ LTLB_ENTRY_FIXED_COUNT);
+ /* This should never happen */
+ BUG_ON(idx >= MMU_LTLB_WAYS);
+ __set_bit(idx, <lb_entries_bmp);
+ ltlb_entries[idx] = tlbe;
+ kvx_mmu_add_entry(MMC_SB_LTLB, idx, tlbe);
+
+ if (kvx_mmc_error(kvx_sfr_get(MMC)))
+ panic("Failed to write entry to the LTLB");
+
+ local_irq_restore(irqflags);
+}
+
+void kvx_mmu_ltlb_remove_entry(unsigned long vaddr)
+{
+ int ret, bit = LTLB_ENTRY_FIXED_COUNT;
+ struct kvx_tlb_format tlbe;
+
+ for_each_set_bit_from(bit, <lb_entries_bmp, MMU_LTLB_WAYS) {
+ tlbe = ltlb_entries[bit];
+ if (tlb_entry_match_addr(tlbe, vaddr)) {
+ __clear_bit(bit, <lb_entries_bmp);
+ break;
+ }
+ }
+
+ if (bit == MMU_LTLB_WAYS)
+ panic("Trying to remove non-existent LTLB entry for addr %lx\n",
+ vaddr);
+
+ ret = clear_ltlb_entry(vaddr);
+ if (ret)
+ panic("Failed to remove LTLB entry for addr %lx\n", vaddr);
+}
+
+/**
+ * kvx_mmu_jtlb_add_entry - Add an entry into JTLB
+ *
+ * JTLB is used both for kernel and user entries.
+ *
+ * @address: Virtual address for the entry (must be aligned according to tlb_ps)
+ * @ptep: pte entry pointer
+ * @asn: ASN (if pte entry is not global)
+ */
+void kvx_mmu_jtlb_add_entry(unsigned long address, pte_t *ptep,
+ unsigned int asn)
+{
+ unsigned int shifted_addr, set, way;
+ unsigned long flags, pte_val;
+ struct kvx_tlb_format tlbe;
+ unsigned int pa, cp, ps;
+ phys_addr_t pfn;
+
+ pte_val = pte_val(*ptep);
+
+ pfn = (phys_addr_t)pte_pfn(*ptep);
+
+ asn &= MM_CTXT_ASN_MASK;
+
+ /* Set page as accessed */
+ pte_val(*ptep) |= _PAGE_ACCESSED;
+
+ BUILD_BUG_ON(KVX_PAGE_SZ_SHIFT != KVX_SFR_TEL_PS_SHIFT);
+
+ ps = (pte_val & KVX_PAGE_SZ_MASK) >> KVX_PAGE_SZ_SHIFT;
+ pa = get_page_access_perms(KVX_ACCESS_PERMS_INDEX(pte_val));
+ cp = pgprot_cache_policy(pte_val);
+
+ tlbe = tlb_mk_entry(
+ (void *)pfn_to_phys(pfn),
+ (void *)address,
+ ps,
+ (pte_val & _PAGE_GLOBAL) ? TLB_G_GLOBAL : TLB_G_USE_ASN,
+ pa,
+ cp,
+ asn,
+ TLB_ES_A_MODIFIED);
+
+ shifted_addr = address >> get_page_size_shift(ps);
+ set = shifted_addr & MMU_JTLB_SET_MASK;
+
+ local_irq_save(flags);
+
+ if (IS_ENABLED(CONFIG_KVX_DEBUG_TLB_WRITE) &&
+ kvx_mmu_ltlb_overlaps(tlbe))
+ panic("VA %lx overlaps with an existing LTLB mapping", address);
+
+ way = get_cpu_var(jtlb_current_set_way)[set]++;
+ put_cpu_var(jtlb_current_set_way);
+
+ way &= MMU_JTLB_WAY_MASK;
+
+ kvx_mmu_add_entry(MMC_SB_JTLB, way, tlbe);
+
+ if (IS_ENABLED(CONFIG_KVX_DEBUG_TLB_WRITE) &&
+ kvx_mmc_error(kvx_sfr_get(MMC)))
+ panic("Failed to write entry to the JTLB (in update_mmu_cache)");
+
+ local_irq_restore(flags);
+}
+
+void __init kvx_mmu_early_setup(void)
+{
+ int bit;
+ struct kvx_tlb_format tlbe;
+
+ kvx_mmu_remove_ltlb_entry(LTLB_ENTRY_EARLY_SMEM);
+
+ if (raw_smp_processor_id() != 0) {
+ /* Apply existing ltlb entries starting from first one free */
+ bit = LTLB_ENTRY_FIXED_COUNT;
+ for_each_set_bit_from(bit, <lb_entries_bmp, MMU_LTLB_WAYS) {
+ tlbe = ltlb_entries[bit];
+ kvx_mmu_add_entry(MMC_SB_LTLB, bit, tlbe);
+ }
+
+ init_kernel_rwx();
+ }
+}
diff --git a/arch/kvx/mm/mmu_stats.c b/arch/kvx/mm/mmu_stats.c
new file mode 100644
index 000000000000..e70b7e2dcab1
--- /dev/null
+++ b/arch/kvx/mm/mmu_stats.c
@@ -0,0 +1,94 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/seq_file.h>
+#include <linux/debugfs.h>
+
+#include <asm/mmu_stats.h>
+
+static struct dentry *mmu_stats_debufs;
+
+static const char *mmu_refill_types_name[MMU_REFILL_TYPE_COUNT] = {
+ [MMU_REFILL_TYPE_USER] = "User",
+ [MMU_REFILL_TYPE_KERNEL] = "Kernel",
+ [MMU_REFILL_TYPE_KERNEL_DIRECT] = "Kernel Direct"
+};
+
+DEFINE_PER_CPU(struct mmu_stats, mmu_stats);
+
+static int mmu_stats_show(struct seq_file *m, void *v)
+{
+ int cpu, type;
+ unsigned long avg = 0, total_refill, efficiency, total_cycles;
+ struct mmu_stats *stats;
+ struct mmu_refill_stats *ref_stat;
+
+ total_cycles = get_cycles();
+ for_each_present_cpu(cpu) {
+ stats = &per_cpu(mmu_stats, cpu);
+ total_refill = 0;
+
+ seq_printf(m, " - CPU %d\n", cpu);
+ for (type = 0; type < MMU_REFILL_TYPE_COUNT; type++) {
+ ref_stat = &stats->refill[type];
+ total_refill += ref_stat->count;
+ if (ref_stat->count)
+ avg = ref_stat->total / ref_stat->count;
+ else
+ avg = 0;
+
+ seq_printf(m,
+ " - %s refill stats:\n"
+ " - count: %lu\n"
+ " - min: %lu\n"
+ " - avg: %lu\n"
+ " - max: %lu\n",
+ mmu_refill_types_name[type],
+ ref_stat->count,
+ ref_stat->min,
+ avg,
+ ref_stat->max
+ );
+ }
+
+ if (total_refill)
+ avg = stats->cycles_between_refill / total_refill;
+ else
+ avg = 0;
+
+ seq_printf(m, " - Average cycles between refill: %lu\n", avg);
+ seq_printf(m, " - tlb_flush_all calls: %lu\n",
+ stats->tlb_flush_all);
+ efficiency = stats->cycles_between_refill * 100 /
+ stats->last_refill;
+ seq_printf(m, " - Efficiency: %lu%%\n", efficiency);
+ }
+
+ return 0;
+}
+
+static int mmu_stats_open(struct inode *inode, struct file *filp)
+{
+ return single_open(filp, mmu_stats_show, NULL);
+}
+
+static const struct file_operations mmu_stats_fops = {
+ .open = mmu_stats_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static int __init mmu_stats_debufs_init(void)
+{
+ mmu_stats_debufs = debugfs_create_dir("kvx_mmu_debug", NULL);
+
+ debugfs_create_file("mmu_stats", 0444, mmu_stats_debufs, NULL,
+ &mmu_stats_fops);
+
+ return 0;
+}
+subsys_initcall(mmu_stats_debufs_init);
diff --git a/arch/kvx/mm/tlb.c b/arch/kvx/mm/tlb.c
new file mode 100644
index 000000000000..9542332fe4d6
--- /dev/null
+++ b/arch/kvx/mm/tlb.c
@@ -0,0 +1,433 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#include <linux/mmu_context.h>
+#include <linux/sched.h>
+
+#include <asm/tlbflush.h>
+#include <asm/tlb_defs.h>
+#include <asm/page_size.h>
+#include <asm/mmu_stats.h>
+#include <asm/pgtable.h>
+#include <asm/tlb.h>
+
+/*
+ * When in kernel, use dummy ASN 42 to be able to catch any problem easily if
+ * ASN is not restored properly.
+ */
+#define KERNEL_DUMMY_ASN 42
+
+/* Threshold of page count above which we will regenerate a new ASN */
+#define ASN_FLUSH_PAGE_THRESHOLD (MMU_JTLB_ENTRIES)
+
+/* Threshold of page count above which we will flush the whole JTLB */
+#define FLUSH_ALL_PAGE_THRESHOLD (MMU_JTLB_ENTRIES)
+
+DEFINE_PER_CPU(unsigned long, kvx_asn_cache) = MM_CTXT_FIRST_CYCLE;
+
+#ifdef CONFIG_KVX_DEBUG_TLB_ACCESS
+
+static DEFINE_PER_CPU_ALIGNED(struct kvx_tlb_access_t[KVX_TLB_ACCESS_SIZE],
+ kvx_tlb_access_rb);
+/* Lower bits hold the index and upper ones hold the number of wrapped */
+static DEFINE_PER_CPU(unsigned int, kvx_tlb_access_idx);
+
+void kvx_update_tlb_access(int type)
+{
+ unsigned int *idx_ptr = &get_cpu_var(kvx_tlb_access_idx);
+ unsigned int idx;
+ struct kvx_tlb_access_t *tab = get_cpu_var(kvx_tlb_access_rb);
+
+ idx = KVX_TLB_ACCESS_GET_IDX(*idx_ptr);
+
+ kvx_mmu_get_tlb_entry(tab[idx].entry);
+ tab[idx].mmc_val = kvx_sfr_get(MMC);
+ tab[idx].type = type;
+
+ (*idx_ptr)++;
+ put_cpu_var(kvx_tlb_access_rb);
+ put_cpu_var(kvx_tlb_access_idx);
+};
+
+#endif
+
+/**
+ * clear_tlb_entry() - clear an entry in TLB if it exists
+ * @addr: the address used to set TEH.PN
+ * @global: is page global or not
+ * @asn: ASN used if page is not global
+ * @tlb_type: tlb type (MMC_SB_LTLB or MMC_SB_JTLB)
+ *
+ * Preemption must be disabled when calling this function. There is no need to
+ * invalidate micro TLB because it is invalidated when we write TLB.
+ *
+ * Return: 0 if TLB entry was found and deleted properly, -ENOENT if not found
+ * -EINVAL if found but in incorrect TLB.
+ *
+ */
+static int clear_tlb_entry(unsigned long addr,
+ unsigned int global,
+ unsigned int asn,
+ unsigned int tlb_type)
+{
+ struct kvx_tlb_format entry;
+ unsigned long mmc_val;
+ int saved_asn, ret = 0;
+
+ /* Sanitize ASN */
+ asn &= MM_CTXT_ASN_MASK;
+
+ /* Before probing we need to save the current ASN */
+ mmc_val = kvx_sfr_get(MMC);
+ saved_asn = kvx_sfr_field_val(mmc_val, MMC, ASN);
+ kvx_sfr_set_field(MMC, ASN, asn);
+
+ /* Probe is based on PN and ASN. So ES can be anything */
+ entry = tlb_mk_entry(0, (void *)addr, 0, global, 0, 0, 0,
+ TLB_ES_INVALID);
+ kvx_mmu_set_tlb_entry(entry);
+
+ kvx_mmu_probetlb();
+
+ mmc_val = kvx_sfr_get(MMC);
+
+ if (kvx_mmc_error(mmc_val)) {
+ if (kvx_mmc_parity(mmc_val)) {
+ /*
+ * This should never happens unless you are bombared by
+ * streams of charged particules. If it happens just
+ * flush the JTLB and let's continue (but check your
+ * environment you are probably not in a safe place).
+ */
+ WARN(1, "%s: parity error during lookup (addr 0x%lx, asn %u). JTLB will be flushed\n",
+ __func__, addr, asn);
+ kvx_sfr_set_field(MMC, PAR, 0);
+ local_flush_tlb_all();
+ }
+
+ /*
+ * else there is no matching entry so just clean the error and
+ * restore the ASN before returning.
+ */
+ kvx_sfr_set_field(MMC, E, 0);
+ ret = -ENOENT;
+ goto restore_asn;
+ }
+
+ /* We surely don't want to flush another TLB type or we are fried */
+ if (kvx_mmc_sb(mmc_val) != tlb_type) {
+ ret = -EINVAL;
+ goto restore_asn;
+ }
+
+ /*
+ * At this point the probe found an entry. TEL and TEH have correct
+ * values so we just need to set the entry status to invalid to clear
+ * the entry.
+ */
+ kvx_sfr_set_field(TEL, ES, TLB_ES_INVALID);
+
+ kvx_mmu_writetlb();
+
+ /* Need to read MMC SFR again */
+ mmc_val = kvx_sfr_get(MMC);
+ if (kvx_mmc_error(mmc_val))
+ panic("%s: Failed to clear entry (addr 0x%lx, asn %u)",
+ __func__, addr, asn);
+ else
+ pr_debug("%s: Entry (addr 0x%lx, asn %u) cleared\n",
+ __func__, addr, asn);
+
+restore_asn:
+ kvx_sfr_set_field(MMC, ASN, saved_asn);
+
+ return ret;
+}
+
+static void clear_jtlb_entry(unsigned long addr,
+ unsigned int global,
+ unsigned int asn)
+{
+ clear_tlb_entry(addr, global, asn, MMC_SB_JTLB);
+}
+
+/**
+ * clear_ltlb_entry() - Remove a LTLB entry
+ * @vaddr: Virtual address to be matched against LTLB entries
+ *
+ * Return: Same value as clear_tlb_entry
+ */
+int clear_ltlb_entry(unsigned long vaddr)
+{
+ return clear_tlb_entry(vaddr, TLB_G_GLOBAL, KERNEL_DUMMY_ASN,
+ MMC_SB_LTLB);
+}
+
+/* If mm is current we just need to assign the current task a new ASN. By
+ * doing this all previous TLB entries with the former ASN will be invalidated.
+ * if mm is not the current one we invalidate the context and when this other
+ * mm will be swapped in then a new context will be generated.
+ */
+void local_flush_tlb_mm(struct mm_struct *mm)
+{
+ int cpu = smp_processor_id();
+
+ destroy_context(mm);
+ if (current->active_mm == mm)
+ activate_context(mm, cpu);
+}
+
+void local_flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long addr)
+{
+ int cpu = smp_processor_id();
+ unsigned int current_asn;
+ struct mm_struct *mm;
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ mm = vma->vm_mm;
+ current_asn = mm_asn(mm, cpu);
+
+ /* If mm has no context there is nothing to do */
+ if (current_asn != MM_CTXT_NO_ASN)
+ clear_jtlb_entry(addr, TLB_G_USE_ASN, current_asn);
+
+ local_irq_restore(flags);
+}
+
+void local_flush_tlb_all(void)
+{
+ struct kvx_tlb_format tlbe = KVX_EMPTY_TLB_ENTRY;
+ int set, way;
+ unsigned long flags;
+#ifdef CONFIG_KVX_MMU_STATS
+ struct mmu_stats *stats = &per_cpu(mmu_stats, smp_processor_id());
+
+ stats->tlb_flush_all++;
+#endif
+
+ local_irq_save(flags);
+
+ /* Select JTLB and prepare TEL (constant) */
+ kvx_sfr_set(TEL, (uint64_t) tlbe.tel_val);
+ kvx_sfr_set_field(MMC, SB, MMC_SB_JTLB);
+
+ for (set = 0; set < MMU_JTLB_SETS; set++) {
+ tlbe.teh.pn = set;
+ for (way = 0; way < MMU_JTLB_WAYS; way++) {
+ /* Set is selected automatically according to the
+ * virtual address.
+ * With 4K pages the set is the value of the 6 lower
+ * significant bits of the page number.
+ */
+ kvx_sfr_set(TEH, (uint64_t) tlbe.teh_val);
+ kvx_sfr_set_field(MMC, SW, way);
+ kvx_mmu_writetlb();
+
+ if (kvx_mmc_error(kvx_sfr_get(MMC)))
+ panic("Failed to initialize JTLB[s:%02d w:%d]",
+ set, way);
+ }
+ }
+
+ local_irq_restore(flags);
+}
+
+void local_flush_tlb_range(struct vm_area_struct *vma,
+ unsigned long start,
+ unsigned long end)
+{
+ const unsigned int cpu = smp_processor_id();
+ unsigned long flags;
+ unsigned int current_asn;
+ unsigned long pages = (end - start) >> PAGE_SHIFT;
+
+ if (pages > ASN_FLUSH_PAGE_THRESHOLD) {
+ local_flush_tlb_mm(vma->vm_mm);
+ return;
+ }
+
+ start &= PAGE_MASK;
+
+ local_irq_save(flags);
+
+ current_asn = mm_asn(vma->vm_mm, cpu);
+ if (current_asn != MM_CTXT_NO_ASN) {
+ while (start < end) {
+ clear_jtlb_entry(start, TLB_G_USE_ASN, current_asn);
+ start += PAGE_SIZE;
+ }
+ }
+
+ local_irq_restore(flags);
+}
+
+/**
+ * local_flush_tlb_kernel_range() - flush kernel TLB entries
+ * @start: start kernel virtual address
+ * @end: end kernel virtual address
+ *
+ * Return: nothing
+ */
+void local_flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ unsigned long flags;
+ unsigned long pages = (end - start) >> PAGE_SHIFT;
+
+ if (pages > FLUSH_ALL_PAGE_THRESHOLD) {
+ local_flush_tlb_all();
+ return;
+ }
+
+ start &= PAGE_MASK;
+
+ local_irq_save(flags);
+
+ while (start < end) {
+ clear_jtlb_entry(start, TLB_G_GLOBAL, KERNEL_DUMMY_ASN);
+ start += PAGE_SIZE;
+ }
+
+ local_irq_restore(flags);
+}
+
+void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
+ pmd_t *pmd)
+{
+ pte_t pte = __pte(pmd_val(*pmd));
+
+ /* THP PMD accessors are implemented in terms of PTE */
+ update_mmu_cache(vma, addr, &pte);
+}
+
+void update_mmu_cache(struct vm_area_struct *vma,
+ unsigned long address, pte_t *ptep)
+{
+ struct mm_struct *mm;
+ unsigned long asn;
+ int cpu = smp_processor_id();
+
+ if (unlikely(ptep == NULL))
+ panic("pte should not be NULL\n");
+
+ /* Flush any previous TLB entries */
+ flush_tlb_page(vma, address);
+
+ /* No need to add the TLB entry until the process that owns the memory
+ * is running.
+ */
+ mm = current->active_mm;
+ if (vma && (mm != vma->vm_mm))
+ return;
+
+ /*
+ * Get the ASN:
+ * ASN can have no context if address belongs to kernel space. In
+ * fact as pages are global for kernel space the ASN is ignored
+ * and can be equal to any value.
+ */
+ asn = mm_asn(mm, cpu);
+
+#ifdef CONFIG_KVX_DEBUG_ASN
+ /*
+ * For addresses that belong to user space, the value of the ASN
+ * must match the mmc.asn and be non zero
+ */
+ if (address < PAGE_OFFSET) {
+ unsigned int mmc_asn = kvx_mmc_asn(kvx_sfr_get(MMC));
+
+ if (asn == MM_CTXT_NO_ASN)
+ panic("%s: ASN [%lu] is not properly set for address 0x%lx on CPU %d\n",
+ __func__, asn, address, cpu);
+
+ if ((asn & MM_CTXT_ASN_MASK) != mmc_asn)
+ panic("%s: ASN not synchronized with MMC: asn:%lu != mmc.asn:%u\n",
+ __func__, (asn & MM_CTXT_ASN_MASK), mmc_asn);
+ }
+#endif
+
+ kvx_mmu_jtlb_add_entry(address, ptep, asn);
+}
+
+#ifdef CONFIG_SMP
+
+struct tlb_args {
+ struct vm_area_struct *ta_vma;
+ unsigned long ta_start;
+ unsigned long ta_end;
+};
+
+static inline void ipi_flush_tlb_page(void *arg)
+{
+ struct tlb_args *ta = arg;
+
+ local_flush_tlb_page(ta->ta_vma, ta->ta_start);
+}
+
+void
+smp_flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
+{
+ struct tlb_args ta = {
+ .ta_vma = vma,
+ .ta_start = addr
+ };
+
+ on_each_cpu_mask(mm_cpumask(vma->vm_mm), ipi_flush_tlb_page, &ta, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_page);
+
+void
+smp_flush_tlb_mm(struct mm_struct *mm)
+{
+ on_each_cpu_mask(mm_cpumask(mm), (smp_call_func_t)local_flush_tlb_mm,
+ mm, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_mm);
+
+static inline void ipi_flush_tlb_range(void *arg)
+{
+ struct tlb_args *ta = arg;
+
+ local_flush_tlb_range(ta->ta_vma, ta->ta_start, ta->ta_end);
+}
+
+void
+smp_flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end)
+{
+ struct tlb_args ta = {
+ .ta_vma = vma,
+ .ta_start = start,
+ .ta_end = end
+ };
+
+ on_each_cpu_mask(mm_cpumask(vma->vm_mm), ipi_flush_tlb_range, &ta, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_range);
+
+static inline void ipi_flush_tlb_kernel_range(void *arg)
+{
+ struct tlb_args *ta = arg;
+
+ local_flush_tlb_kernel_range(ta->ta_start, ta->ta_end);
+}
+
+void
+smp_flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ struct tlb_args ta = {
+ .ta_start = start,
+ .ta_end = end
+ };
+
+ on_each_cpu(ipi_flush_tlb_kernel_range, &ta, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_kernel_range);
+
+#endif
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 11/25] kvx: Add memory management
@ 2023-01-03 16:43 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Will Deacon, Aneesh Kumar K.V, Andrew Morton,
Nick Piggin, Peter Zijlstra, Paul Walmsley, Palmer Dabbelt,
Albert Ou, linux-kernel, linux-arch, linux-mm, linux-riscv,
Clement Leger, Guillaume Thouvenin, Jean-Christophe Pince,
Jules Maselbas, Julian Vetter, Julien Hascoet, Louis Morhet,
Marc Poulhiès, Marius Gligor, Vincent Chardon
Add memory management support for kvx, including: cache and tlb
management, page fault handling, ioremap/mmap and streaming dma support.
CC: Will Deacon <will@kernel.org>
CC: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: Nick Piggin <npiggin@gmail.com>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Paul Walmsley <paul.walmsley@sifive.com>
CC: Palmer Dabbelt <palmer@dabbelt.com>
CC: Albert Ou <aou@eecs.berkeley.edu>
CC: linux-kernel@vger.kernel.org
CC: linux-arch@vger.kernel.org
CC: linux-mm@kvack.org
CC: linux-riscv@lists.infradead.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Jean-Christophe Pince <jcpince@gmail.com>
Signed-off-by: Jean-Christophe Pince <jcpince@gmail.com>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Julien Hascoet <jhascoet@kalray.eu>
Signed-off-by: Julien Hascoet <jhascoet@kalray.eu>
Co-developed-by: Louis Morhet <lmorhet@kalray.eu>
Signed-off-by: Louis Morhet <lmorhet@kalray.eu>
Co-developed-by: Marc Poulhiès <dkm@kataplop.net>
Signed-off-by: Marc Poulhiès <dkm@kataplop.net>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/cache.h | 46 +++
arch/kvx/include/asm/cacheflush.h | 181 +++++++++
arch/kvx/include/asm/fixmap.h | 47 +++
arch/kvx/include/asm/hugetlb.h | 36 ++
arch/kvx/include/asm/l2_cache.h | 75 ++++
arch/kvx/include/asm/l2_cache_defs.h | 64 ++++
arch/kvx/include/asm/mem_map.h | 44 +++
arch/kvx/include/asm/mmu.h | 296 +++++++++++++++
arch/kvx/include/asm/mmu_context.h | 156 ++++++++
arch/kvx/include/asm/mmu_stats.h | 38 ++
arch/kvx/include/asm/page.h | 187 ++++++++++
arch/kvx/include/asm/page_size.h | 29 ++
arch/kvx/include/asm/pgalloc.h | 101 +++++
arch/kvx/include/asm/pgtable-bits.h | 102 ++++++
arch/kvx/include/asm/pgtable.h | 451 +++++++++++++++++++++++
arch/kvx/include/asm/rm_fw.h | 16 +
arch/kvx/include/asm/sparsemem.h | 15 +
arch/kvx/include/asm/symbols.h | 16 +
arch/kvx/include/asm/tlb.h | 24 ++
arch/kvx/include/asm/tlb_defs.h | 131 +++++++
arch/kvx/include/asm/tlbflush.h | 58 +++
arch/kvx/include/asm/vmalloc.h | 10 +
arch/kvx/mm/cacheflush.c | 154 ++++++++
arch/kvx/mm/dma-mapping.c | 95 +++++
arch/kvx/mm/extable.c | 24 ++
arch/kvx/mm/fault.c | 264 ++++++++++++++
arch/kvx/mm/hugetlbpage.c | 317 ++++++++++++++++
arch/kvx/mm/init.c | 527 +++++++++++++++++++++++++++
arch/kvx/mm/kernel_rwx.c | 228 ++++++++++++
arch/kvx/mm/mmap.c | 31 ++
arch/kvx/mm/mmu.c | 204 +++++++++++
arch/kvx/mm/mmu_stats.c | 94 +++++
arch/kvx/mm/tlb.c | 433 ++++++++++++++++++++++
33 files changed, 4494 insertions(+)
create mode 100644 arch/kvx/include/asm/cache.h
create mode 100644 arch/kvx/include/asm/cacheflush.h
create mode 100644 arch/kvx/include/asm/fixmap.h
create mode 100644 arch/kvx/include/asm/hugetlb.h
create mode 100644 arch/kvx/include/asm/l2_cache.h
create mode 100644 arch/kvx/include/asm/l2_cache_defs.h
create mode 100644 arch/kvx/include/asm/mem_map.h
create mode 100644 arch/kvx/include/asm/mmu.h
create mode 100644 arch/kvx/include/asm/mmu_context.h
create mode 100644 arch/kvx/include/asm/mmu_stats.h
create mode 100644 arch/kvx/include/asm/page.h
create mode 100644 arch/kvx/include/asm/page_size.h
create mode 100644 arch/kvx/include/asm/pgalloc.h
create mode 100644 arch/kvx/include/asm/pgtable-bits.h
create mode 100644 arch/kvx/include/asm/pgtable.h
create mode 100644 arch/kvx/include/asm/rm_fw.h
create mode 100644 arch/kvx/include/asm/sparsemem.h
create mode 100644 arch/kvx/include/asm/symbols.h
create mode 100644 arch/kvx/include/asm/tlb.h
create mode 100644 arch/kvx/include/asm/tlb_defs.h
create mode 100644 arch/kvx/include/asm/tlbflush.h
create mode 100644 arch/kvx/include/asm/vmalloc.h
create mode 100644 arch/kvx/mm/cacheflush.c
create mode 100644 arch/kvx/mm/dma-mapping.c
create mode 100644 arch/kvx/mm/extable.c
create mode 100644 arch/kvx/mm/fault.c
create mode 100644 arch/kvx/mm/hugetlbpage.c
create mode 100644 arch/kvx/mm/init.c
create mode 100644 arch/kvx/mm/kernel_rwx.c
create mode 100644 arch/kvx/mm/mmap.c
create mode 100644 arch/kvx/mm/mmu.c
create mode 100644 arch/kvx/mm/mmu_stats.c
create mode 100644 arch/kvx/mm/tlb.c
diff --git a/arch/kvx/include/asm/cache.h b/arch/kvx/include/asm/cache.h
new file mode 100644
index 000000000000..c53599cb6a6f
--- /dev/null
+++ b/arch/kvx/include/asm/cache.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_CACHE_H
+#define _ASM_KVX_CACHE_H
+
+/**
+ * On kvx I$ and D$ have the same size (16KB).
+ * Caches are 16K bytes big, VIPT 4-way set associative, true LRU, with 64-byte
+ * lines. The D$ is also write-through.
+ */
+#define KVX_ICACHE_WAY_COUNT 4
+#define KVX_ICACHE_SET_COUNT 64
+#define KVX_ICACHE_LINE_SHIFT 6
+#define KVX_ICACHE_LINE_SIZE (1 << KVX_ICACHE_LINE_SHIFT)
+#define KVX_ICACHE_SIZE \
+ (KVX_ICACHE_WAY_COUNT * KVX_ICACHE_SET_COUNT * KVX_ICACHE_LINE_SIZE)
+
+/**
+ * Invalidate the whole I-cache if the size to flush is more than this value
+ */
+#define KVX_ICACHE_INVAL_SIZE (KVX_ICACHE_SIZE)
+
+/* D-Cache */
+#define KVX_DCACHE_WAY_COUNT 4
+#define KVX_DCACHE_SET_COUNT 64
+#define KVX_DCACHE_LINE_SHIFT 6
+#define KVX_DCACHE_LINE_SIZE (1 << KVX_DCACHE_LINE_SHIFT)
+#define KVX_DCACHE_SIZE \
+ (KVX_DCACHE_WAY_COUNT * KVX_DCACHE_SET_COUNT * KVX_DCACHE_LINE_SIZE)
+
+/**
+ * Same for I-cache
+ */
+#define KVX_DCACHE_INVAL_SIZE (KVX_DCACHE_SIZE)
+
+#define L1_CACHE_SHIFT KVX_DCACHE_LINE_SHIFT
+#define L1_CACHE_BYTES KVX_DCACHE_LINE_SIZE
+
+#define L2_CACHE_LINE_SIZE 256
+#define L2_CACHE_LINE_MASK (L2_CACHE_LINE_SIZE - 1)
+
+#endif /* _ASM_KVX_CACHE_H */
diff --git a/arch/kvx/include/asm/cacheflush.h b/arch/kvx/include/asm/cacheflush.h
new file mode 100644
index 000000000000..f501e9956958
--- /dev/null
+++ b/arch/kvx/include/asm/cacheflush.h
@@ -0,0 +1,181 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ * Guillaume Thouvenin
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_CACHEFLUSH_H
+#define _ASM_KVX_CACHEFLUSH_H
+
+#include <linux/mm.h>
+#include <linux/io.h>
+
+#include <asm/l2_cache.h>
+
+#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
+
+#define flush_cache_mm(mm) do { } while (0)
+#define flush_cache_range(vma, start, end) do { } while (0)
+#define flush_cache_dup_mm(mm) do { } while (0)
+#define flush_cache_page(vma, vmaddr, pfn) do { } while (0)
+
+#define flush_cache_vmap(start, end) do { } while (0)
+#define flush_cache_vunmap(start, end) do { } while (0)
+
+#define flush_dcache_page(page) do { } while (0)
+
+#define flush_dcache_mmap_lock(mapping) do { } while (0)
+#define flush_dcache_mmap_unlock(mapping) do { } while (0)
+
+#define l1_inval_dcache_all __builtin_kvx_dinval
+#define kvx_fence __builtin_kvx_fence
+#define l1_inval_icache_all __builtin_kvx_iinval
+
+int dcache_wb_inval_virt_range(unsigned long vaddr, unsigned long len, bool wb,
+ bool inval);
+void dcache_wb_inval_phys_range(phys_addr_t addr, unsigned long len, bool wb,
+ bool inval);
+
+/*
+ * In the following functions, if L2$ is enabled, line invalidations
+ * in the L2$ will also trigger (via hw) invalidations in L1D caches of the
+ * same Cluster in case the lines are present in such L1D caches.
+ */
+
+/*
+ * L1 is indexed by virtual addresses and as such, invalidation takes virtual
+ * addresses as arguments.
+ */
+static inline
+void l1_inval_dcache_range(unsigned long vaddr, unsigned long size)
+{
+ unsigned long end = vaddr + size;
+
+ /* Then inval L1 */
+ if (size >= KVX_DCACHE_INVAL_SIZE) {
+ __builtin_kvx_dinval();
+ return;
+ }
+
+ vaddr = ALIGN_DOWN(vaddr, KVX_DCACHE_LINE_SIZE);
+ for (; vaddr < end; vaddr += KVX_DCACHE_LINE_SIZE)
+ __builtin_kvx_dinvall((void *) vaddr);
+}
+
+static inline
+void inval_dcache_range(phys_addr_t paddr, unsigned long size)
+{
+ /*
+ * Inval L2 first to avoid refilling from cached L2 values.
+ * If L2 cache is not enabled, it will return false and hence, we will
+ * fall back on L1 invalidation.
+ */
+ if (!l2_cache_inval_range(paddr, size))
+ l1_inval_dcache_range((unsigned long) phys_to_virt(paddr),
+ size);
+}
+
+static inline
+void wb_dcache_range(phys_addr_t paddr, unsigned long size)
+{
+ /* Fence to ensure all write are committed */
+ kvx_fence();
+
+ l2_cache_wb_range(paddr, size);
+}
+
+static inline
+void wbinval_dcache_range(phys_addr_t paddr, unsigned long size)
+{
+ /* Fence to ensure all write are committed */
+ kvx_fence();
+
+ if (!l2_cache_wbinval_range(paddr, size))
+ l1_inval_dcache_range((unsigned long) phys_to_virt(paddr),
+ size);
+}
+
+static inline
+void l1_inval_icache_range(unsigned long start, unsigned long end)
+{
+ unsigned long addr;
+ unsigned long size = end - start;
+
+ if (size >= KVX_ICACHE_INVAL_SIZE) {
+ __builtin_kvx_iinval();
+ __builtin_kvx_barrier();
+ return;
+ }
+
+ start = ALIGN_DOWN(start, KVX_ICACHE_LINE_SIZE);
+ for (addr = start; addr < end; addr += KVX_ICACHE_LINE_SIZE)
+ __builtin_kvx_iinvals((void *) addr);
+
+ __builtin_kvx_barrier();
+}
+
+static inline
+void wbinval_icache_range(phys_addr_t paddr, unsigned long size)
+{
+ unsigned long vaddr = (unsigned long) phys_to_virt(paddr);
+
+ /* Fence to ensure all write are committed */
+ kvx_fence();
+
+ l2_cache_wbinval_range(paddr, size);
+ /* invalidating l2 cache will invalidate l1 dcache
+ * but not l1 icache
+ */
+ l1_inval_icache_range(vaddr, vaddr + size);
+}
+
+static inline
+void sync_dcache_icache(unsigned long start, unsigned long end)
+{
+ /* Fence to ensure all write are committed to L2 */
+ kvx_fence();
+ /* Then invalidate the L1 icache to reload from L2 */
+ l1_inval_icache_range(start, end);
+}
+
+static inline
+void local_flush_icache_range(unsigned long start, unsigned long end)
+{
+ sync_dcache_icache(start, end);
+}
+
+#ifdef CONFIG_SMP
+void flush_icache_range(unsigned long start, unsigned long end);
+#else
+#define flush_icache_range local_flush_icache_range
+#endif
+
+static inline
+void flush_icache_page(struct vm_area_struct *vma, struct page *page)
+{
+ unsigned long start = (unsigned long) page_address(page);
+ unsigned long end = start + PAGE_SIZE;
+
+ sync_dcache_icache(start, end);
+}
+
+static inline
+void flush_icache_user_range(struct vm_area_struct *vma, struct page *page,
+ unsigned long vaddr, int len)
+{
+ sync_dcache_icache(vaddr, vaddr + len);
+}
+
+#define copy_to_user_page(vma, page, vaddr, dst, src, len) \
+ do { \
+ memcpy(dst, src, len); \
+ if (vma->vm_flags & VM_EXEC) \
+ flush_icache_user_range(vma, page, vaddr, len); \
+ } while (0)
+#define copy_from_user_page(vma, page, vaddr, dst, src, len) \
+ memcpy(dst, src, len)
+
+#endif /* _ASM_KVX_CACHEFLUSH_H */
diff --git a/arch/kvx/include/asm/fixmap.h b/arch/kvx/include/asm/fixmap.h
new file mode 100644
index 000000000000..3863e410d71d
--- /dev/null
+++ b/arch/kvx/include/asm/fixmap.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_FIXMAP_H
+#define _ASM_KVX_FIXMAP_H
+
+/**
+ * Use the latest available kernel address minus one page.
+ * This is needed since __fix_to_virt returns
+ * (FIXADDR_TOP - ((x) << PAGE_SHIFT))
+ * Due to that, first member will be shifted by 0 and will be equal to
+ * FIXADDR_TOP.
+ * Some other architectures simply add a FIX_HOLE at the beginning of
+ * the fixed_addresses enum (I think ?).
+ */
+#define FIXADDR_TOP (-PAGE_SIZE)
+
+#define ASM_FIX_TO_VIRT(IDX) \
+ (FIXADDR_TOP - ((IDX) << PAGE_SHIFT))
+
+#ifndef __ASSEMBLY__
+#include <asm/page.h>
+#include <asm/pgtable.h>
+
+enum fixed_addresses {
+ FIX_EARLYCON_MEM_BASE,
+ FIX_GDB_BARE_DISPLACED_MEM_BASE,
+ /* Used to access text early in RW mode (jump label) */
+ FIX_TEXT_PATCH,
+ __end_of_fixed_addresses
+};
+
+#define FIXADDR_SIZE (__end_of_fixed_addresses << PAGE_SHIFT)
+#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
+#define FIXMAP_PAGE_IO (PAGE_KERNEL_DEVICE)
+
+void __set_fixmap(enum fixed_addresses idx,
+ phys_addr_t phys, pgprot_t prot);
+
+#include <asm-generic/fixmap.h>
+#endif /* __ASSEMBLY__ */
+
+#endif
diff --git a/arch/kvx/include/asm/hugetlb.h b/arch/kvx/include/asm/hugetlb.h
new file mode 100644
index 000000000000..a5984e8ede7e
--- /dev/null
+++ b/arch/kvx/include/asm/hugetlb.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_HUGETLB_H
+#define _ASM_KVX_HUGETLB_H
+
+#include <asm/pgtable.h>
+
+#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT
+extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte);
+
+#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR
+extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep);
+
+#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS
+extern int huge_ptep_set_access_flags(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep,
+ pte_t pte, int dirty);
+
+#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT
+extern void huge_ptep_set_wrprotect(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep);
+
+#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH
+extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep);
+
+#include <asm-generic/hugetlb.h>
+
+#endif /* _ASM_KVX_HUGETLB_H */
diff --git a/arch/kvx/include/asm/l2_cache.h b/arch/kvx/include/asm/l2_cache.h
new file mode 100644
index 000000000000..437b82bb78ab
--- /dev/null
+++ b/arch/kvx/include/asm/l2_cache.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_L2_CACHE_H
+#define _ASM_KVX_L2_CACHE_H
+
+#include <linux/bits.h>
+#include <linux/jump_label.h>
+
+#ifdef CONFIG_L2_CACHE
+
+#include <asm/cache.h>
+#include <asm/l2_cache_defs.h>
+
+void l2_cache_push_area_cmd(u64 cmd_type, phys_addr_t start,
+ unsigned long size);
+
+extern struct static_key_false l2_enabled;
+
+static bool
+l2_cache_send_cmd(u64 cmd_type, phys_addr_t start, unsigned long size)
+{
+ if (static_branch_likely(&l2_enabled)) {
+ l2_cache_push_area_cmd(cmd_type, start, size);
+ return true;
+ }
+
+ return false;
+}
+
+static inline bool
+l2_cache_wbinval_range(phys_addr_t start, unsigned long size)
+{
+ return l2_cache_send_cmd(L2_CMD_OP_CMD_PURGE_AREA, start, size);
+}
+
+static inline bool
+l2_cache_wb_range(phys_addr_t start, unsigned long size)
+{
+ return l2_cache_send_cmd(L2_CMD_OP_CMD_FLUSH_AREA, start, size);
+}
+
+static inline bool
+l2_cache_inval_range(phys_addr_t start, unsigned long size)
+{
+ /* We don't need invalidation to be synced (ie 0) */
+ return l2_cache_send_cmd(L2_CMD_OP_CMD_INVAL_AREA, start, size);
+}
+
+#else
+
+static inline bool
+l2_cache_wbinval_range(phys_addr_t start, unsigned long size)
+{
+ return false;
+}
+
+static inline bool
+l2_cache_wb_range(phys_addr_t start, unsigned long size)
+{
+ return false;
+}
+
+static inline bool
+l2_cache_inval_range(phys_addr_t start, unsigned long size)
+{
+ return false;
+}
+
+#endif
+
+#endif /* _ASM_KVX_L2_CACHE_H */
diff --git a/arch/kvx/include/asm/l2_cache_defs.h b/arch/kvx/include/asm/l2_cache_defs.h
new file mode 100644
index 000000000000..81f001d9087a
--- /dev/null
+++ b/arch/kvx/include/asm/l2_cache_defs.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julien Hascoet
+ */
+
+#ifndef _ASM_KVX_L2_CACHE_DEFS_H
+#define _ASM_KVX_L2_CACHE_DEFS_H
+
+/* Error registers */
+enum l2_api_error {
+ L2_STA_ERR_CFG = 1,
+ L2_STA_ERR_ADDR,
+ L2_STA_ERR_STOP,
+ L2_STA_ERR_NO_MEM,
+ L2_STA_ERR_DIS,
+ L2_STA_ERR_DT
+};
+
+/* L2 Cache status registers definition */
+#define L2_STATUS_OFFSET 0x0
+#define L2_STATUS_ERROR_MASK 0x4
+#define L2_STATUS_ERROR_SHIFT 0x2
+#define L2_STATUS_READY_MASK 0x2
+#define L2_STATUS_READY_SHIFT 0x1
+#define L2_STATUS_VALID_MASK 0x1
+#define L2_STATUS_VALID_SHIFT 0x0
+#define L2_STATUS_ACK_MASK 0x80
+#define L2_STATUS_ACK_SHIFT 0x7
+
+/* L2 Cache error registers definition */
+#define L2_ERROR_OFFSET 0x20
+#define L2_ERROR_SETUP_ERR_MASK 0x1
+#define L2_ERROR_SETUP_ERR_SHIFT 0x0
+#define L2_ERROR_API_ERR_MASK 0x20
+#define L2_ERROR_API_ERR_SHIFT 0x5
+#define L2_ERROR_ERROR_CODE_MASK 0xFF00
+#define L2_ERROR_ERROR_CODE_SHIFT 0x8
+
+/* L2 Cache instance registers definition */
+#define L2_INSTANCE_OFFSET 0x8
+#define L2_INSTANCE_CMD_QUEUE_SIZE_SHIFT 0x20
+#define L2_INSTANCE_CMD_QUEUE_SIZE_MASK 0x7F00000000UL
+
+/* L2 Cache commands fifo registers definition */
+#define L2_CMD_OFFSET 0x400
+#define L2_CMD_READ_IDX_OFFSET 0x0
+#define L2_CMD_WRITE_IDX_OFFSET 0x8
+#define L2_CMD_DOORBELL_READ_ADDR_OFFSET 0x10
+#define L2_CMD_DOORBELL_WRITE_ADDR_OFFSET 0x18
+#define L2_CMD_FIFO_OFFSET 0x20
+#define L2_CMD_FIFO_ELEM_SIZE 0x20
+#define L2_CMD_OP_CMD_PURGE_LINE 0x4
+#define L2_CMD_OP_CMD_PURGE_AREA 0x8
+#define L2_CMD_OP_CMD_FLUSH_AREA 0x9
+#define L2_CMD_OP_CMD_INVAL_AREA 0xA
+#define L2_CMD_OP_VALID_SHIFT 0x0
+#define L2_CMD_OP_SYNC_SHIFT 0x1
+#define L2_CMD_OP_CMD_SHIFT 0x2
+#define L2_CMD_OP_ARG_COUNT 0x3
+
+
+#endif /* _ASM_KVX_L2_CACHE_DEFS_H */
diff --git a/arch/kvx/include/asm/mem_map.h b/arch/kvx/include/asm/mem_map.h
new file mode 100644
index 000000000000..ea90144209ce
--- /dev/null
+++ b/arch/kvx/include/asm/mem_map.h
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef _ASM_KVX_MEM_MAP_H
+#define _ASM_KVX_MEM_MAP_H
+
+#include <linux/const.h>
+#include <linux/sizes.h>
+
+#include <asm/page.h>
+#include <asm/fixmap.h>
+
+/**
+ * kvx memory mapping defines
+ * For more information on memory mapping, please see
+ * Documentation/kvx/kvx-mmu.txt
+ *
+ * All _BASE defines are relative to PAGE_OFFSET
+ */
+
+/* Guard between various memory map zones */
+#define MAP_GUARD_SIZE SZ_1G
+
+/**
+ * Kernel direct memory mapping
+ */
+#define KERNEL_DIRECT_MEMORY_MAP_BASE PAGE_OFFSET
+#define KERNEL_DIRECT_MEMORY_MAP_SIZE UL(0x1000000000)
+#define KERNEL_DIRECT_MEMORY_MAP_END \
+ (KERNEL_DIRECT_MEMORY_MAP_BASE + KERNEL_DIRECT_MEMORY_MAP_SIZE)
+
+/**
+ * Vmalloc mapping (goes from kernel direct memory map up to fixmap start -
+ * guard size)
+ */
+#define KERNEL_VMALLOC_MAP_BASE (KERNEL_DIRECT_MEMORY_MAP_END + MAP_GUARD_SIZE)
+#define KERNEL_VMALLOC_MAP_SIZE \
+ (FIXADDR_START - KERNEL_VMALLOC_MAP_BASE - MAP_GUARD_SIZE)
+
+#endif
diff --git a/arch/kvx/include/asm/mmu.h b/arch/kvx/include/asm/mmu.h
new file mode 100644
index 000000000000..fbf5ed286e87
--- /dev/null
+++ b/arch/kvx/include/asm/mmu.h
@@ -0,0 +1,296 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Marc Poulhiès
+ */
+
+#ifndef _ASM_KVX_MMU_H
+#define _ASM_KVX_MMU_H
+
+#include <linux/bug.h>
+#include <linux/types.h>
+#include <linux/threads.h>
+
+#include <asm/page.h>
+#include <asm/sfr.h>
+#include <asm/page.h>
+#include <asm/pgtable-bits.h>
+#include <asm/tlb_defs.h>
+
+/* Virtual addresses can use at most 41 bits */
+#define MMU_VIRT_BITS 41
+
+/*
+ * See Documentation/kvx/kvx-mmu.txt for details about the division of the
+ * virtual memory space.
+ */
+#if defined(CONFIG_KVX_4K_PAGES)
+#define MMU_USR_ADDR_BITS 39
+#else
+#error "Only 4Ko page size is supported at this time"
+#endif
+
+typedef struct mm_context {
+ unsigned long end_brk;
+ unsigned long asn[NR_CPUS];
+ unsigned long sigpage;
+} mm_context_t;
+
+struct __packed tlb_entry_low {
+ unsigned int es:2; /* Entry Status */
+ unsigned int cp:2; /* Cache Policy */
+ unsigned int pa:4; /* Protection Attributes */
+ unsigned int r:2; /* Reserved */
+ unsigned int ps:2; /* Page Size */
+ unsigned int fn:28; /* Frame Number */
+};
+
+struct __packed tlb_entry_high {
+ unsigned int asn:9; /* Address Space Number */
+ unsigned int g:1; /* Global Indicator */
+ unsigned int vs:2; /* Virtual Space */
+ unsigned int pn:29; /* Page Number */
+};
+
+struct kvx_tlb_format {
+ union {
+ struct tlb_entry_low tel;
+ uint64_t tel_val;
+ };
+ union {
+ struct tlb_entry_high teh;
+ uint64_t teh_val;
+ };
+};
+
+#define KVX_EMPTY_TLB_ENTRY { .tel_val = 0x0, .teh_val = 0x0 }
+
+/* Bit [0:39] of the TLB format corresponds to TLB Entry low */
+/* Bit [40:80] of the TLB format corresponds to the TLB Entry high */
+#define kvx_mmu_set_tlb_entry(tlbf) do { \
+ kvx_sfr_set(TEL, (uint64_t) tlbf.tel_val); \
+ kvx_sfr_set(TEH, (uint64_t) tlbf.teh_val); \
+} while (0)
+
+#define kvx_mmu_get_tlb_entry(tlbf) do { \
+ tlbf.tel_val = kvx_sfr_get(TEL); \
+ tlbf.teh_val = kvx_sfr_get(TEH); \
+} while (0)
+
+/* Use kvx_mmc_ to read a field from MMC value passed as parameter */
+#define __kvx_mmc(mmc_reg, field) \
+ kvx_sfr_field_val(mmc_reg, MMC, field)
+
+#define kvx_mmc_error(mmc) __kvx_mmc(mmc, E)
+#define kvx_mmc_parity(mmc) __kvx_mmc(mmc, PAR)
+#define kvx_mmc_sb(mmc) __kvx_mmc(mmc, SB)
+#define kvx_mmc_ss(mmc) __kvx_mmc(mmc, SS)
+#define kvx_mmc_sw(mmc) __kvx_mmc(mmc, SW)
+#define kvx_mmc_asn(mmc) __kvx_mmc(mmc, ASN)
+
+#define KVX_TLB_ACCESS_READ 0
+#define KVX_TLB_ACCESS_WRITE 1
+#define KVX_TLB_ACCESS_PROBE 2
+
+#ifdef CONFIG_KVX_DEBUG_TLB_ACCESS
+
+#define KVX_TLB_ACCESS_SIZE (1 << CONFIG_KVX_DEBUG_TLB_ACCESS_BITS)
+#define KVX_TLB_ACCESS_MASK GENMASK((CONFIG_KVX_DEBUG_TLB_ACCESS_BITS - 1), 0)
+#define KVX_TLB_ACCESS_GET_IDX(idx) (idx & KVX_TLB_ACCESS_MASK)
+
+/* This structure is used to make decoding of MMC easier in gdb */
+struct mmc_t {
+ unsigned int asn:9;
+ unsigned int s: 1;
+ unsigned int r1: 4;
+ unsigned int sne: 1;
+ unsigned int spe: 1;
+ unsigned int ptc: 2;
+ unsigned int sw: 4;
+ unsigned int ss: 6;
+ unsigned int sb: 1;
+ unsigned int r2: 1;
+ unsigned int par: 1;
+ unsigned int e: 1;
+};
+
+struct __packed kvx_tlb_access_t {
+ struct kvx_tlb_format entry; /* 128 bits */
+ union {
+ struct mmc_t mmc;
+ uint32_t mmc_val;
+ };
+ uint32_t type;
+};
+
+extern void kvx_update_tlb_access(int type);
+
+#else
+#define kvx_update_tlb_access(type) do {} while (0)
+#endif
+
+static inline void kvx_mmu_readtlb(void)
+{
+ kvx_update_tlb_access(KVX_TLB_ACCESS_READ);
+ asm volatile ("tlbread\n;;");
+}
+
+static inline void kvx_mmu_writetlb(void)
+{
+ kvx_update_tlb_access(KVX_TLB_ACCESS_WRITE);
+ asm volatile ("tlbwrite\n;;");
+}
+
+static inline void kvx_mmu_probetlb(void)
+{
+ kvx_update_tlb_access(KVX_TLB_ACCESS_PROBE);
+ asm volatile ("tlbprobe\n;;");
+}
+
+#define kvx_mmu_add_entry(buffer, way, entry) do { \
+ kvx_sfr_set_field(MMC, SB, buffer); \
+ kvx_sfr_set_field(MMC, SW, way); \
+ kvx_mmu_set_tlb_entry(entry); \
+ kvx_mmu_writetlb(); \
+} while (0)
+
+#define kvx_mmu_remove_ltlb_entry(way) do { \
+ struct kvx_tlb_format __invalid_entry = KVX_EMPTY_TLB_ENTRY; \
+ kvx_mmu_add_entry(MMC_SB_LTLB, way, __invalid_entry); \
+} while (0)
+
+static inline int get_page_size_shift(int ps)
+{
+ /*
+ * Use the same assembly trick using sbmm to directly get the page size
+ * shift using a constant which encodes all page size shifts
+ */
+ return __builtin_kvx_sbmm8(KVX_PS_SHIFT_MATRIX,
+ KVX_SBMM_BYTE_SEL << ps);
+}
+
+/*
+ * 4 bits are used to index the kvx access permissions. Bits are used as
+ * follow:
+ *
+ * +---------------+------------+-------------+------------+
+ * | Bit 3 | Bit 2 | Bit 1 | Bit 0 |
+ * |---------------+------------+-------------+------------|
+ * | _PAGE_GLOBAL | _PAGE_EXEC | _PAGE_WRITE | _PAGE_READ |
+ * +---------------+------------+-------------+------------+
+ *
+ * If _PAGE_GLOBAL is set then the page belongs to the kernel. Otherwise it
+ * belongs to the user. When the page belongs to user-space then give the
+ * same rights to the kernel-space.
+ * In order to quickly compute policy from this value, we use sbmm instruction.
+ * The main interest is to avoid an additionnal load and specifically in the
+ * assembly refill handler.
+ */
+static inline u8 get_page_access_perms(u8 policy)
+{
+ /* If PAGE_READ is unset, there is no permission for this page */
+ if (!(policy & (_PAGE_READ >> _PAGE_PERMS_SHIFT)))
+ return TLB_PA_NA_NA;
+
+ /* Discard _PAGE_READ bit to get a linear number in [0,7] */
+ policy >>= 1;
+
+ /* Use sbmm to directly get the page perms */
+ return __builtin_kvx_sbmm8(KVX_PAGE_PA_MATRIX,
+ KVX_SBMM_BYTE_SEL << policy);
+}
+
+static inline struct kvx_tlb_format tlb_mk_entry(
+ void *paddr,
+ void *vaddr,
+ unsigned int ps,
+ unsigned int global,
+ unsigned int pa,
+ unsigned int cp,
+ unsigned int asn,
+ unsigned int es)
+{
+ struct kvx_tlb_format entry;
+ u64 mask = ULONG_MAX << get_page_size_shift(ps);
+
+ BUG_ON(ps >= (1 << KVX_SFR_TEL_PS_WIDTH));
+
+ /*
+ * 0 matches the virtual space:
+ * - either we are virtualized and the hypervisor will set it
+ * for us when using writetlb
+ * - Or we are native and the virtual space is 0
+ */
+ entry.teh_val = TLB_MK_TEH_ENTRY((uintptr_t)vaddr & mask, 0, global,
+ asn);
+ entry.tel_val = TLB_MK_TEL_ENTRY((uintptr_t)paddr, ps, es, cp, pa);
+
+ return entry;
+}
+
+static inline unsigned long tlb_entry_phys(struct kvx_tlb_format tlbe)
+{
+ return ((unsigned long) tlbe.tel.fn << KVX_SFR_TEL_FN_SHIFT);
+}
+
+static inline unsigned long tlb_entry_virt(struct kvx_tlb_format tlbe)
+{
+ return ((unsigned long) tlbe.teh.pn << KVX_SFR_TEH_PN_SHIFT);
+}
+
+static inline unsigned long tlb_entry_size(struct kvx_tlb_format tlbe)
+{
+ return BIT(get_page_size_shift(tlbe.tel.ps));
+}
+
+static inline int tlb_entry_overlaps(struct kvx_tlb_format tlbe1,
+ struct kvx_tlb_format tlbe2)
+{
+ unsigned long start1, end1;
+ unsigned long start2, end2;
+
+ start1 = tlb_entry_virt(tlbe1);
+ end1 = start1 + tlb_entry_size(tlbe1);
+
+ start2 = tlb_entry_virt(tlbe2);
+ end2 = start2 + tlb_entry_size(tlbe2);
+
+ return start1 <= end2 && end1 >= start2;
+}
+
+static inline int tlb_entry_match_addr(struct kvx_tlb_format tlbe,
+ unsigned long vaddr)
+{
+ /*
+ * TLB entries store up to 41 bits so we must truncate the provided
+ * address to match teh.pn.
+ */
+ vaddr &= GENMASK(MMU_VIRT_BITS - 1, KVX_SFR_TEH_PN_SHIFT);
+
+ return tlb_entry_virt(tlbe) == vaddr;
+}
+
+extern void kvx_mmu_early_setup(void);
+
+
+#if defined(CONFIG_STRICT_KERNEL_RWX)
+void init_kernel_rwx(void);
+void paging_init(void);
+#else
+
+static inline void paging_init(void) {}
+#endif
+
+void kvx_mmu_ltlb_remove_entry(unsigned long vaddr);
+void kvx_mmu_ltlb_add_entry(unsigned long vaddr, phys_addr_t paddr,
+ pgprot_t flags, unsigned long page_shift);
+
+void kvx_mmu_jtlb_add_entry(unsigned long address, pte_t *ptep,
+ unsigned int asn);
+extern void mmu_early_init(void);
+
+struct mm_struct;
+
+#endif /* _ASM_KVX_MMU_H */
diff --git a/arch/kvx/include/asm/mmu_context.h b/arch/kvx/include/asm/mmu_context.h
new file mode 100644
index 000000000000..39fa92f1506b
--- /dev/null
+++ b/arch/kvx/include/asm/mmu_context.h
@@ -0,0 +1,156 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef __ASM_KVX_MMU_CONTEXT_H
+#define __ASM_KVX_MMU_CONTEXT_H
+
+/*
+ * Management of the Address Space Number:
+ * Coolidge architecture provides a 9-bit ASN to tag TLB entries. This can be
+ * used to allow several entries with the same virtual address (so from
+ * different process) to be in the TLB at the same time. That means that won't
+ * necessarily flush the TLB when a context switch occurs and so it will
+ * improve performances.
+ */
+#include <linux/smp.h>
+
+#include <asm/mmu.h>
+#include <asm/sfr_defs.h>
+#include <asm/tlbflush.h>
+
+#include <asm-generic/mm_hooks.h>
+
+#define MM_CTXT_ASN_MASK GENMASK(KVX_SFR_MMC_ASN_WIDTH - 1, 0)
+#define MM_CTXT_CYCLE_MASK (~MM_CTXT_ASN_MASK)
+#define MM_CTXT_NO_ASN UL(0x0)
+#define MM_CTXT_FIRST_CYCLE (MM_CTXT_ASN_MASK + 1)
+
+#define mm_asn(mm, cpu) ((mm)->context.asn[cpu])
+
+DECLARE_PER_CPU(unsigned long, kvx_asn_cache);
+#define cpu_asn_cache(cpu) per_cpu(kvx_asn_cache, cpu)
+
+static inline void get_new_mmu_context(struct mm_struct *mm, unsigned int cpu)
+{
+ unsigned long asn = cpu_asn_cache(cpu);
+
+ asn++;
+ /* Check if we need to start a new cycle */
+ if ((asn & MM_CTXT_ASN_MASK) == 0) {
+ pr_debug("%s: start new cycle, flush all tlb\n", __func__);
+ local_flush_tlb_all();
+
+ /*
+ * Above check for rollover of 9 bit ASN in 64 bit container.
+ * If the container itself wrapped around, set it to a non zero
+ * "generation" to distinguish from no context
+ */
+ if (asn == 0)
+ asn = MM_CTXT_FIRST_CYCLE;
+ }
+
+ cpu_asn_cache(cpu) = asn;
+ mm_asn(mm, cpu) = asn;
+
+ pr_debug("%s: mm = 0x%llx: cpu[%d], cycle: %lu, asn: %lu\n",
+ __func__, (unsigned long long)mm, cpu,
+ (asn & MM_CTXT_CYCLE_MASK) >> KVX_SFR_MMC_ASN_WIDTH,
+ asn & MM_CTXT_ASN_MASK);
+}
+
+static inline void get_mmu_context(struct mm_struct *mm, unsigned int cpu)
+{
+
+ unsigned long asn = mm_asn(mm, cpu);
+
+ /*
+ * Move to new ASN if it was not from current alloc-cycle/generation.
+ * This is done by ensuring that the generation bits in both
+ * mm->context.asn and cpu_asn_cache counter are exactly same.
+ *
+ * NOTE: this also works for checking if mm has a context since the
+ * first alloc-cycle/generation is always '1'. MM_CTXT_NO_ASN value
+ * contains cycle '0', and thus it will match.
+ */
+ if ((asn ^ cpu_asn_cache(cpu)) & MM_CTXT_CYCLE_MASK)
+ get_new_mmu_context(mm, cpu);
+}
+
+static inline void activate_context(struct mm_struct *mm, unsigned int cpu)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ get_mmu_context(mm, cpu);
+
+ kvx_sfr_set_field(MMC, ASN, mm_asn(mm, cpu) & MM_CTXT_ASN_MASK);
+
+ local_irq_restore(flags);
+}
+
+/**
+ * Redefining the generic hooks that are:
+ * - activate_mm
+ * - deactivate_mm
+ * - enter_lazy_tlb
+ * - init_new_context
+ * - destroy_context
+ * - switch_mm
+ */
+
+#define activate_mm(prev, next) switch_mm((prev), (next), NULL)
+#define deactivate_mm(tsk, mm) do { } while (0)
+#define enter_lazy_tlb(mm, tsk) do { } while (0)
+
+static inline int init_new_context(struct task_struct *tsk,
+ struct mm_struct *mm)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu)
+ mm_asn(mm, cpu) = MM_CTXT_NO_ASN;
+
+ return 0;
+}
+
+static inline void destroy_context(struct mm_struct *mm)
+{
+ int cpu = smp_processor_id();
+ unsigned long flags;
+
+ local_irq_save(flags);
+ mm_asn(mm, cpu) = MM_CTXT_NO_ASN;
+ local_irq_restore(flags);
+}
+
+static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next,
+ struct task_struct *tsk)
+{
+ unsigned int cpu = smp_processor_id();
+
+ /**
+ * Comment taken from arc, but logic is the same for us:
+ *
+ * Note that the mm_cpumask is "aggregating" only, we don't clear it
+ * for the switched-out task, unlike some other arches.
+ * It is used to enlist cpus for sending TLB flush IPIs and not sending
+ * it to CPUs where a task once ran-on, could cause stale TLB entry
+ * re-use, specially for a multi-threaded task.
+ * e.g. T1 runs on C1, migrates to C3. T2 running on C2 munmaps.
+ * For a non-aggregating mm_cpumask, IPI not sent C1, and if T1
+ * were to re-migrate to C1, it could access the unmapped region
+ * via any existing stale TLB entries.
+ */
+ cpumask_set_cpu(cpu, mm_cpumask(next));
+
+ if (prev != next)
+ activate_context(next, cpu);
+}
+
+
+#endif /* __ASM_KVX_MMU_CONTEXT_H */
diff --git a/arch/kvx/include/asm/mmu_stats.h b/arch/kvx/include/asm/mmu_stats.h
new file mode 100644
index 000000000000..999352dbc1ce
--- /dev/null
+++ b/arch/kvx/include/asm/mmu_stats.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_MMU_STATS_H
+#define _ASM_KVX_MMU_STATS_H
+
+#ifdef CONFIG_KVX_MMU_STATS
+#include <linux/percpu.h>
+
+struct mmu_refill_stats {
+ unsigned long count;
+ unsigned long total;
+ unsigned long min;
+ unsigned long max;
+};
+
+enum mmu_refill_type {
+ MMU_REFILL_TYPE_USER,
+ MMU_REFILL_TYPE_KERNEL,
+ MMU_REFILL_TYPE_KERNEL_DIRECT,
+ MMU_REFILL_TYPE_COUNT,
+};
+
+struct mmu_stats {
+ struct mmu_refill_stats refill[MMU_REFILL_TYPE_COUNT];
+ /* keep these fields ordered this way for assembly */
+ unsigned long cycles_between_refill;
+ unsigned long last_refill;
+ unsigned long tlb_flush_all;
+};
+
+DECLARE_PER_CPU(struct mmu_stats, mmu_stats);
+#endif
+
+#endif /* _ASM_KVX_MMU_STATS_H */
diff --git a/arch/kvx/include/asm/page.h b/arch/kvx/include/asm/page.h
new file mode 100644
index 000000000000..cea5f02d12bb
--- /dev/null
+++ b/arch/kvx/include/asm/page.h
@@ -0,0 +1,187 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_PAGE_H
+#define _ASM_KVX_PAGE_H
+
+#include <linux/const.h>
+
+#define PAGE_SHIFT CONFIG_KVX_PAGE_SHIFT
+#define PAGE_SIZE _BITUL(PAGE_SHIFT)
+#define PAGE_MASK (~(PAGE_SIZE - 1))
+
+#define PHYS_OFFSET CONFIG_KVX_PHYS_OFFSET
+#define PAGE_OFFSET CONFIG_KVX_PAGE_OFFSET
+
+#define VA_TO_PA_OFFSET (PHYS_OFFSET - PAGE_OFFSET)
+#define PA_TO_VA_OFFSET (PAGE_OFFSET - PHYS_OFFSET)
+
+/*
+ * These macros are specifically written for assembly. They are useful for
+ * converting symbols above PAGE_OFFSET to their physical addresses.
+ */
+#define __PA(x) ((x) + VA_TO_PA_OFFSET)
+#define __VA(x) ((x) + PA_TO_VA_OFFSET)
+
+/*
+ * PFN starts at 0 if physical address starts at 0x0. As it is not the case
+ * for the kvx we need to apply an offset to the calculated PFN.
+ */
+#define ARCH_PFN_OFFSET ((unsigned long)(PHYS_OFFSET >> PAGE_SHIFT))
+
+#if defined(CONFIG_KVX_4K_PAGES)
+/* Maximum usable bit using 4K pages and current page table layout */
+#define VA_MAX_BITS 40
+#define PGDIR_SHIFT 30
+#define PMD_SHIFT 21
+#else
+#error "64K page not supported"
+#endif
+
+/*
+ * Define _SHIFT, _SIZE and _MASK corresponding of the different page
+ * sizes supported by kvx.
+ */
+#define KVX_PAGE_4K_SHIFT 12
+#define KVX_PAGE_4K_SIZE BIT(KVX_PAGE_4K_SHIFT)
+#define KVX_PAGE_4K_MASK (~(KVX_PAGE_4K_SIZE - 1))
+
+#define KVX_PAGE_64K_SHIFT 16
+#define KVX_PAGE_64K_SIZE BIT(KVX_PAGE_64K_SHIFT)
+#define KVX_PAGE_64K_MASK (~(KVX_PAGE_64K_SIZE - 1))
+
+#define KVX_PAGE_2M_SHIFT 21
+#define KVX_PAGE_2M_SIZE BIT(KVX_PAGE_2M_SHIFT)
+#define KVX_PAGE_2M_MASK (~(KVX_PAGE_2M_SIZE - 1))
+
+#define KVX_PAGE_512M_SHIFT 29
+#define KVX_PAGE_512M_SIZE BIT(KVX_PAGE_512M_SHIFT)
+#define KVX_PAGE_512M_MASK (~(KVX_PAGE_512M_SIZE - 1))
+
+/* Encode all page shift into one 32bit constant for sbmm */
+#define KVX_PS_SHIFT_MATRIX ((KVX_PAGE_512M_SHIFT << 24) | \
+ (KVX_PAGE_2M_SHIFT << 16) | \
+ (KVX_PAGE_64K_SHIFT << 8) | \
+ (KVX_PAGE_4K_SHIFT))
+
+/* Encode all page access policy into one 64bit constant for sbmm */
+#define KVX_PAGE_PA_MATRIX ((UL(TLB_PA_NA_RWX) << 56) | \
+ (UL(TLB_PA_NA_RX) << 48) | \
+ (UL(TLB_PA_NA_RW) << 40) | \
+ (UL(TLB_PA_NA_R) << 32) | \
+ (UL(TLB_PA_RWX_RWX) << 24) | \
+ (UL(TLB_PA_RX_RX) << 16) | \
+ (UL(TLB_PA_RW_RW) << 8) | \
+ (UL(TLB_PA_R_R)))
+
+/*
+ * Select a byte using sbmm8. When shifted by one bit left, we get the next
+ * byte.
+ * For instance using this default constant with sbmm yields the value between
+ * first byte of the double word.
+ * If constant is shifted by 1, the value is now 0x0000000000000002ULL and this
+ * yield the second byte and so on, and so on !
+ */
+#define KVX_SBMM_BYTE_SEL 0x01
+
+#ifndef __ASSEMBLY__
+
+#include <linux/string.h>
+
+/* Page Global Directory entry */
+typedef struct {
+ unsigned long pgd;
+} pgd_t;
+
+/* Page Middle Directory entry */
+typedef struct {
+ unsigned long pmd;
+} pmd_t;
+
+/* Page Table entry */
+typedef struct {
+ unsigned long pte;
+} pte_t;
+
+/* Protection bits */
+typedef struct {
+ unsigned long pgprot;
+} pgprot_t;
+
+typedef struct page *pgtable_t;
+
+/**
+ * Macros to access entry values
+ */
+#define pgd_val(x) ((x).pgd)
+#define pmd_val(x) ((x).pmd)
+#define pte_val(x) ((x).pte)
+#define pgprot_val(x) ((x).pgprot)
+
+/**
+ * Macro to create entry from value
+ */
+#define __pgd(x) ((pgd_t) { (x) })
+#define __pmd(x) ((pmd_t) { (x) })
+#define __pte(x) ((pte_t) { (x) })
+#define __pgprot(x) ((pgprot_t) { (x) })
+
+#define pte_pgprot(x) __pgprot(pte_val(x) & ~KVX_PFN_MASK)
+
+#define __pa(x) ((unsigned long)(x) + VA_TO_PA_OFFSET)
+#define __va(x) ((void *)((unsigned long) (x) + PA_TO_VA_OFFSET))
+
+#define phys_to_pfn(phys) (PFN_DOWN(phys))
+#define pfn_to_phys(pfn) (PFN_PHYS(pfn))
+
+#define virt_to_pfn(vaddr) (phys_to_pfn(__pa(vaddr)))
+#define pfn_to_virt(pfn) (__va(pfn_to_phys(pfn)))
+
+#define virt_to_page(vaddr) (pfn_to_page(virt_to_pfn(vaddr)))
+#define page_to_virt(page) (pfn_to_virt(page_to_pfn(page)))
+
+#define page_to_phys(page) virt_to_phys(page_to_virt(page))
+#define phys_to_page(phys) (pfn_to_page(phys_to_pfn(phys)))
+
+#define virt_addr_valid(vaddr) (pfn_valid(virt_to_pfn(vaddr)))
+
+#ifdef CONFIG_FLATMEM
+static inline bool pfn_valid(unsigned long pfn)
+{
+ /* avoid <linux/mm.h> include hell */
+ extern unsigned long max_mapnr;
+
+ return ((pfn >= ARCH_PFN_OFFSET) &&
+ (pfn < (ARCH_PFN_OFFSET + max_mapnr)));
+}
+#endif
+
+extern void clear_page(void *to);
+extern void copy_page(void *to, void *from);
+
+static inline void clear_user_page(void *page, unsigned long vaddr,
+ struct page *pg)
+{
+ clear_page(page);
+}
+
+static inline void copy_user_page(void *to, void *from, unsigned long vaddr,
+ struct page *topage)
+{
+ copy_page(to, from);
+}
+
+#define VM_DATA_DEFAULT_FLAGS (VM_READ | VM_WRITE | \
+ VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
+
+#include <asm-generic/memory_model.h>
+#include <asm-generic/getorder.h>
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_KVX_PAGE_H */
diff --git a/arch/kvx/include/asm/page_size.h b/arch/kvx/include/asm/page_size.h
new file mode 100644
index 000000000000..2c2850205b50
--- /dev/null
+++ b/arch/kvx/include/asm/page_size.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_PAGE_SIZE_H
+#define _ASM_KVX_PAGE_SIZE_H
+
+#include <asm/tlb_defs.h>
+
+#if defined(CONFIG_HUGETLB_PAGE)
+#define HUGE_PAGE_SIZE (MMC_PMJ_64K | MMC_PMJ_2M | MMC_PMJ_512M)
+#else
+#define HUGE_PAGE_SIZE (0)
+#endif
+
+#if defined(CONFIG_KVX_4K_PAGES)
+#define TLB_DEFAULT_PS TLB_PS_4K
+#define KVX_SUPPORTED_PSIZE (MMC_PMJ_4K | HUGE_PAGE_SIZE)
+#elif defined(CONFIG_KVX_64K_PAGES)
+#define TLB_DEFAULT_PS TLB_PS_64K
+#define KVX_SUPPORTED_PSIZE (MMC_PMJ_64K | HUGE_PAGE_SIZE)
+#else
+#error "Unsupported page size"
+#endif
+
+#endif
diff --git a/arch/kvx/include/asm/pgalloc.h b/arch/kvx/include/asm/pgalloc.h
new file mode 100644
index 000000000000..0e654dd1a072
--- /dev/null
+++ b/arch/kvx/include/asm/pgalloc.h
@@ -0,0 +1,101 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_PGALLOC_H
+#define _ASM_KVX_PGALLOC_H
+
+#include <linux/mm.h>
+#include <asm/tlb.h>
+
+#define __HAVE_ARCH_PGD_FREE
+#include <asm-generic/pgalloc.h> /* for pte_{alloc,free}_one */
+
+static inline void check_pgt_cache(void)
+{
+ /*
+ * check_pgt_cache() is called to check watermarks from counters that
+ * computes the number of pages allocated by cached allocation functions
+ * pmd_alloc_one_fast() and pte_alloc_one_fast().
+ * Currently we just skip this test.
+ */
+}
+
+/**
+ * PGD
+ */
+
+static inline void
+pgd_free(struct mm_struct *mm, pgd_t *pgd)
+{
+ free_pages((unsigned long) pgd, PAGES_PER_PGD);
+}
+
+static inline
+pgd_t *pgd_alloc(struct mm_struct *mm)
+{
+ pgd_t *pgd;
+
+ pgd = (pgd_t *) __get_free_pages(GFP_KERNEL, PAGES_PER_PGD);
+ if (unlikely(pgd == NULL))
+ return NULL;
+
+ memset(pgd, 0, USER_PTRS_PER_PGD * sizeof(pgd_t));
+
+ /* Copy kernel mappings */
+ memcpy(pgd + USER_PTRS_PER_PGD,
+ init_mm.pgd + USER_PTRS_PER_PGD,
+ (PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t));
+
+ return pgd;
+}
+
+/**
+ * PUD
+ */
+
+static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
+{
+ unsigned long pfn = virt_to_pfn(pmd);
+
+ set_pud(pud, __pud((unsigned long)pfn << PAGE_SHIFT));
+}
+
+/**
+ * PMD
+ */
+
+static inline void pmd_populate_kernel(struct mm_struct *mm,
+ pmd_t *pmd, pte_t *pte)
+{
+ unsigned long pfn = virt_to_pfn(pte);
+
+ set_pmd(pmd, __pmd((unsigned long)pfn << PAGE_SHIFT));
+}
+
+static inline void pmd_populate(struct mm_struct *mm,
+ pmd_t *pmd, pgtable_t pte)
+{
+ unsigned long pfn = virt_to_pfn(page_address(pte));
+
+ set_pmd(pmd, __pmd((unsigned long)pfn << PAGE_SHIFT));
+}
+
+#if CONFIG_PGTABLE_LEVELS > 2
+#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd)
+#endif /* CONFIG_PGTABLE_LEVELS > 2 */
+
+/**
+ * PTE
+ */
+
+#define __pte_free_tlb(tlb, pte, buf) \
+do { \
+ pgtable_pte_page_dtor(pte); \
+ tlb_remove_page((tlb), pte); \
+} while (0)
+
+#endif /* _ASM_KVX_PGALLOC_H */
diff --git a/arch/kvx/include/asm/pgtable-bits.h b/arch/kvx/include/asm/pgtable-bits.h
new file mode 100644
index 000000000000..98ee7d1fbe63
--- /dev/null
+++ b/arch/kvx/include/asm/pgtable-bits.h
@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Julian Vetter
+ */
+
+#ifndef _ASM_KVX_PGTABLE_BITS_H
+#define _ASM_KVX_PGTABLE_BITS_H
+
+/*
+ * Protection bit definition
+ * As we don't have any HW to handle page table walk, we can define
+ * our own PTE format. In order to make things easier, we are trying to match
+ * some parts of $tel and $teh.
+ *
+ * PageSZ must be on bit 10 and 11 because it matches the TEL.PS bits. And
+ * by doing that it is easier in assembly to set the TEL.PS to PageSZ.
+ * In other words, KVX_PAGE_SZ_SHIFT == KVX_SFR_TEL_PS_SHIFT.
+ * It is checked by using a BUILD_BUG_ON() in arch/kvx/mm/tlb.c.
+ *
+ * Huge bit must be somewhere in the first 12 bits to be able to detect it
+ * when reading the PMD entry.
+ *
+ * KV3-1:
+ * +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+ * | 63..23 | 22..13 | 12 | 11..10 | 9 | 8 | 7 | 6 | 5 | 4 | 3..2 | 1 | 0 |
+ * +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+ * PFN Unused S PageSZ H G X W R D CP A P
+ *
+ * Note: PFN is 40-bits wide. We use 41-bits to ensure that the upper bit is
+ * always set to 0. This is required when shifting PFN to right.
+ */
+
+/* Following shift are used in ASM to easily extract bit */
+#define _PAGE_PERMS_SHIFT 5
+#define _PAGE_GLOBAL_SHIFT 8
+#define _PAGE_HUGE_SHIFT 9
+
+#define _PAGE_PRESENT (1 << 0) /* Present */
+#define _PAGE_ACCESSED (1 << 1) /* Set by tlb refill code on any access */
+/* Bits 2 - 3 reserved for cache policy */
+#define _PAGE_DIRTY (1 << 4) /* Set by tlb refill code on any write */
+#define _PAGE_READ (1 << _PAGE_PERMS_SHIFT) /* Readable */
+#define _PAGE_WRITE (1 << 6) /* Writable */
+#define _PAGE_EXEC (1 << 7) /* Executable */
+#define _PAGE_GLOBAL (1 << _PAGE_GLOBAL_SHIFT) /* Global */
+#define _PAGE_HUGE (1 << _PAGE_HUGE_SHIFT) /* Huge page */
+/* Bits 10 - 11 reserved for page size */
+#define _PAGE_SOFT (1 << 12) /* Reserved for software */
+#define _PAGE_SZ_64K (TLB_PS_64K << KVX_PAGE_SZ_SHIFT)
+#define _PAGE_SZ_2M (TLB_PS_2M << KVX_PAGE_SZ_SHIFT)
+#define _PAGE_SZ_512M (TLB_PS_512M << KVX_PAGE_SZ_SHIFT)
+
+
+#define _PAGE_SPECIAL _PAGE_SOFT
+
+/*
+ * If _PAGE_PRESENT is clear because the user mapped it with PROT_NONE
+ * pte_present still gives true. Bit[15] of the PTE is used since its unused
+ * for a PTE entry for kv3-1 (see above)
+ */
+#define _PAGE_NONE (1 << 15)
+
+/* Note: mask used in assembly cannot be generated with GENMASK */
+#define KVX_PFN_SHIFT 23
+#define KVX_PFN_MASK (~(((1 << KVX_PFN_SHIFT) - 1)))
+
+#define KVX_PAGE_SZ_SHIFT 10
+#define KVX_PAGE_SZ_MASK KVX_SFR_TEL_PS_MASK
+
+/* Huge page of 64K are hold in PTE table */
+#define KVX_PAGE_64K_NR_CONT (1UL << (KVX_PAGE_64K_SHIFT - PAGE_SHIFT))
+/* Huge page of 512M are hold in PMD table */
+#define KVX_PAGE_512M_NR_CONT (1UL << (KVX_PAGE_512M_SHIFT - PMD_SHIFT))
+
+#define KVX_PAGE_CP_SHIFT 2
+#define KVX_PAGE_CP_MASK KVX_SFR_TEL_CP_MASK
+
+
+#define _PAGE_CACHED (TLB_CP_W_C << KVX_PAGE_CP_SHIFT)
+#define _PAGE_UNCACHED (TLB_CP_U_U << KVX_PAGE_CP_SHIFT)
+#define _PAGE_DEVICE (TLB_CP_D_U << KVX_PAGE_CP_SHIFT)
+
+#define KVX_ACCESS_PERMS_BITS 4
+#define KVX_ACCESS_PERMS_OFFSET _PAGE_PERMS_SHIFT
+#define KVX_ACCESS_PERMS_SIZE (1 << KVX_ACCESS_PERMS_BITS)
+
+#define KVX_ACCESS_PERM_START_BIT KVX_ACCESS_PERMS_OFFSET
+#define KVX_ACCESS_PERM_STOP_BIT \
+ (KVX_ACCESS_PERMS_OFFSET + KVX_ACCESS_PERMS_BITS - 1)
+#define KVX_ACCESS_PERMS_MASK \
+ GENMASK(KVX_ACCESS_PERM_STOP_BIT, KVX_ACCESS_PERM_START_BIT)
+#define KVX_ACCESS_PERMS_INDEX(x) \
+ ((unsigned int)(x & KVX_ACCESS_PERMS_MASK) >> KVX_ACCESS_PERMS_OFFSET)
+
+/* Bits read, write, exec and global are not preserved across pte_modify() */
+#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_READ | _PAGE_WRITE | \
+ _PAGE_EXEC | _PAGE_GLOBAL))
+
+#endif
diff --git a/arch/kvx/include/asm/pgtable.h b/arch/kvx/include/asm/pgtable.h
new file mode 100644
index 000000000000..4d3a8af7d174
--- /dev/null
+++ b/arch/kvx/include/asm/pgtable.h
@@ -0,0 +1,451 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Marius Gligor
+ * Yann Sionneau
+ */
+
+#ifndef _ASM_KVX_PGTABLE_H
+#define _ASM_KVX_PGTABLE_H
+
+#include <linux/mmzone.h>
+#include <linux/mm_types.h>
+
+#include <asm/page.h>
+#include <asm/pgtable-bits.h>
+
+#include <asm-generic/pgtable-nopud.h>
+
+#include <asm/mem_map.h>
+
+struct mm_struct;
+struct vm_area_struct;
+
+/*
+ * Hugetlb definitions. All sizes are supported (64 KB, 2 MB and 512 MB).
+ */
+#if defined(CONFIG_KVX_4K_PAGES)
+#define HUGE_MAX_HSTATE 3
+#elif defined(CONFIG_KVX_64K_PAGES)
+#define HUGE_MAX_HSTATE 2
+#else
+#error "Unsupported page size"
+#endif
+
+#define HPAGE_SHIFT PMD_SHIFT
+#define HPAGE_SIZE BIT(HPAGE_SHIFT)
+#define HPAGE_MASK (~(HPAGE_SIZE - 1))
+#define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT)
+
+extern pte_t arch_make_huge_pte(pte_t entry, unsigned int shift,
+ vm_flags_t flags);
+#define arch_make_huge_pte arch_make_huge_pte
+
+/* Vmalloc definitions */
+#define VMALLOC_START KERNEL_VMALLOC_MAP_BASE
+#define VMALLOC_END (VMALLOC_START + KERNEL_VMALLOC_MAP_SIZE - 1)
+
+/* Also used by GDB script to go through the page table */
+#define PGDIR_BITS (VA_MAX_BITS - PGDIR_SHIFT)
+#define PMD_BITS (PGDIR_SHIFT - PMD_SHIFT)
+#define PTE_BITS (PMD_SHIFT - PAGE_SHIFT)
+
+/* Size of region mapped by a page global directory */
+#define PGDIR_SIZE BIT(PGDIR_SHIFT)
+#define PGDIR_MASK (~(PGDIR_SIZE - 1))
+
+/* Size of region mapped by a page middle directory */
+#define PMD_SIZE BIT(PMD_SHIFT)
+#define PMD_MASK (~(PMD_SIZE - 1))
+
+/* Number of entries in the page global directory */
+#define PAGES_PER_PGD 2
+#define PTRS_PER_PGD (PAGES_PER_PGD * PAGE_SIZE / sizeof(pgd_t))
+
+/* Number of entries in the page middle directory */
+#define PTRS_PER_PMD (PAGE_SIZE / sizeof(pmd_t))
+
+/* Number of entries in the page table */
+#define PTRS_PER_PTE (PAGE_SIZE / sizeof(pte_t))
+
+#define USER_PTRS_PER_PGD (TASK_SIZE/PGDIR_SIZE)
+
+extern pgd_t swapper_pg_dir[PTRS_PER_PGD];
+
+/* Page protection bits */
+#define _PAGE_BASE (_PAGE_PRESENT | _PAGE_CACHED)
+#define _PAGE_KERNEL (_PAGE_PRESENT | _PAGE_GLOBAL | \
+ _PAGE_READ | _PAGE_WRITE)
+#define _PAGE_KERNEL_EXEC (_PAGE_BASE | _PAGE_READ | _PAGE_EXEC | \
+ _PAGE_GLOBAL | _PAGE_WRITE)
+#define _PAGE_KERNEL_DEVICE (_PAGE_KERNEL | _PAGE_DEVICE)
+#define _PAGE_KERNEL_NOCACHE (_PAGE_KERNEL | _PAGE_UNCACHED)
+
+#define PAGE_NONE __pgprot(_PAGE_NONE)
+#define PAGE_READ __pgprot(_PAGE_BASE | _PAGE_READ)
+#define PAGE_READ_WRITE __pgprot(_PAGE_BASE | _PAGE_READ | _PAGE_WRITE)
+#define PAGE_READ_EXEC __pgprot(_PAGE_BASE | _PAGE_READ | _PAGE_EXEC)
+#define PAGE_READ_WRITE_EXEC __pgprot(_PAGE_BASE | _PAGE_READ | \
+ _PAGE_EXEC | _PAGE_WRITE)
+
+#define PAGE_KERNEL __pgprot(_PAGE_KERNEL | _PAGE_CACHED)
+#define PAGE_KERNEL_EXEC __pgprot(_PAGE_KERNEL_EXEC)
+#define PAGE_KERNEL_NOCACHE __pgprot(_PAGE_KERNEL | _PAGE_UNCACHED)
+#define PAGE_KERNEL_DEVICE __pgprot(_PAGE_KERNEL_DEVICE)
+#define PAGE_KERNEL_RO __pgprot((_PAGE_KERNEL | _PAGE_CACHED) & ~(_PAGE_WRITE))
+#define PAGE_KERNEL_ROX __pgprot(_PAGE_KERNEL_EXEC & ~(_PAGE_WRITE))
+
+#define pgprot_noncached(prot) (__pgprot((pgprot_val(prot) & ~KVX_PAGE_CP_MASK) | _PAGE_UNCACHED))
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern struct page *empty_zero_page;
+#define ZERO_PAGE(vaddr) (empty_zero_page)
+
+
+/*
+ * Encode and decode a swap entry. Swap entries are encoded in an arch
+ * dependent format as follows:
+ *
+ * +--------+----+-------+------+---+---+
+ * | 63..16 | 15 | 14..7 | 6..2 | 1 | 0 |
+ * +--------+----+-------+------+---+---+
+ * offset 0 0 type 0 0
+ *
+ * This allows for up to 31 swap files and 1PB per swap file.
+ */
+#define __SWP_TYPE_SHIFT 2
+#define __SWP_TYPE_BITS 5
+#define __SWP_TYPE_MASK ((1UL << __SWP_TYPE_BITS) - 1)
+#define __SWP_OFFSET_SHIFT 16
+
+#define MAX_SWAPFILES_CHECK() \
+ BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > __SWP_TYPE_BITS)
+
+#define __swp_type(x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK)
+#define __swp_offset(x) ((x).val >> __SWP_OFFSET_SHIFT)
+#define __swp_entry(type, offset) ((swp_entry_t) \
+ { ((type) << __SWP_TYPE_SHIFT) | ((offset) << __SWP_OFFSET_SHIFT) })
+
+#define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) })
+#define __swp_entry_to_pte(x) ((pte_t) { (x).val })
+
+/*
+ * PGD definitions:
+ * - pgd_ERROR
+ */
+#define pgd_ERROR(e) \
+ pr_err("%s:%d: bad pgd %016lx.\n", __FILE__, __LINE__, pgd_val(e))
+
+/*
+ * PUD
+ *
+ * As we manage a three level page table the call to set_pud is used to fill
+ * PGD.
+ */
+static inline void set_pud(pud_t *pudp, pud_t pmd)
+{
+ *pudp = pmd;
+}
+
+static inline int pud_none(pud_t pud)
+{
+ return pud_val(pud) == 0;
+}
+
+static inline int pud_bad(pud_t pud)
+{
+ return pud_none(pud);
+}
+static inline int pud_present(pud_t pud)
+{
+ return pud_val(pud) != 0;
+}
+
+static inline void pud_clear(pud_t *pud)
+{
+ set_pud(pud, __pud(0));
+}
+
+/*
+ * PMD definitions:
+ * - set_pmd
+ * - pmd_present
+ * - pmd_none
+ * - pmd_bad
+ * - pmd_clear
+ * - pmd_page
+ */
+
+static inline void set_pmd(pmd_t *pmdp, pmd_t pmd)
+{
+ *pmdp = pmd;
+}
+
+/* Returns 1 if entry is present */
+static inline int pmd_present(pmd_t pmd)
+{
+ return pmd_val(pmd) != 0;
+}
+
+/* Returns 1 if the corresponding entry has the value 0 */
+static inline int pmd_none(pmd_t pmd)
+{
+ return pmd_val(pmd) == 0;
+}
+
+/* Used to check that a page middle directory entry is valid */
+static inline int pmd_bad(pmd_t pmd)
+{
+ return pmd_none(pmd);
+}
+
+/* Clears the entry to prevent process to use the linear address that
+ * mapped it.
+ */
+static inline void pmd_clear(pmd_t *pmdp)
+{
+ set_pmd(pmdp, __pmd(0));
+}
+
+/*
+ * Returns the address of the descriptor of the page table referred by the
+ * PMD entry.
+ */
+static inline struct page *pmd_page(pmd_t pmd)
+{
+ if (pmd_val(pmd) & _PAGE_HUGE)
+ return pfn_to_page(
+ (pmd_val(pmd) & KVX_PFN_MASK) >> KVX_PFN_SHIFT);
+
+ return pfn_to_page(pmd_val(pmd) >> PAGE_SHIFT);
+}
+
+#define pmd_ERROR(e) \
+ pr_err("%s:%d: bad pmd %016lx.\n", __FILE__, __LINE__, pmd_val(e))
+
+static inline pmd_t *pud_pgtable(pud_t pud)
+{
+ return (pmd_t *)pfn_to_virt(pud_val(pud) >> PAGE_SHIFT);
+}
+
+static inline struct page *pud_page(pud_t pud)
+{
+ return pfn_to_page(pud_val(pud) >> PAGE_SHIFT);
+}
+
+/*
+ * PTE definitions:
+ * - set_pte
+ * - set_pte_at
+ * - pte_clear
+ * - pte_page
+ * - pte_pfn
+ * - pte_present
+ * - pte_none
+ * - pte_write
+ * - pte_dirty
+ * - pte_young
+ * - pte_special
+ * - pte_mkdirty
+ * - pte_mkwrite
+ * - pte_mkclean
+ * - pte_mkyoung
+ * - pte_mkold
+ * - pte_mkspecial
+ * - pte_wrprotect
+ */
+
+static inline void set_pte(pte_t *ptep, pte_t pteval)
+{
+ *ptep = pteval;
+}
+
+static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pteval)
+{
+ set_pte(ptep, pteval);
+}
+
+#define pte_clear(mm, addr, ptep) set_pte(ptep, __pte(0))
+
+/* Constructs a page table entry */
+static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot)
+{
+ return __pte(((pfn << KVX_PFN_SHIFT) & KVX_PFN_MASK) |
+ pgprot_val(prot));
+}
+
+/* Builds a page table entry by combining a page descriptor and a group of
+ * access rights.
+ */
+#define mk_pte(page, prot) (pfn_pte(page_to_pfn(page), prot))
+
+/* Modifies page access rights */
+static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
+{
+ return __pte((pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot));
+}
+
+#define pte_page(x) pfn_to_page(pte_pfn(x))
+
+static inline unsigned long pmd_page_vaddr(pmd_t pmd)
+{
+ return (unsigned long)pfn_to_virt(pmd_val(pmd) >> PAGE_SHIFT);
+}
+
+/* Yields the page frame number (PFN) of a page table entry */
+static inline unsigned long pte_pfn(pte_t pte)
+{
+ return ((pte_val(pte) & KVX_PFN_MASK) >> KVX_PFN_SHIFT);
+}
+
+static inline int pte_present(pte_t pte)
+{
+ return (pte_val(pte) & (_PAGE_PRESENT | _PAGE_NONE));
+}
+
+static inline int pte_none(pte_t pte)
+{
+ return (pte_val(pte) == 0);
+}
+
+static inline int pte_write(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_WRITE;
+}
+
+static inline int pte_dirty(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_DIRTY;
+}
+
+static inline int pte_young(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_ACCESSED;
+}
+
+static inline int pte_special(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_SPECIAL;
+}
+
+static inline int pte_huge(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_HUGE;
+}
+
+static inline pte_t pte_mkdirty(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_DIRTY);
+}
+
+static inline pte_t pte_mkwrite(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_WRITE);
+}
+
+static inline pte_t pte_mkclean(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~(_PAGE_DIRTY));
+}
+
+static inline pte_t pte_mkyoung(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_ACCESSED);
+}
+
+static inline pte_t pte_mkold(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~(_PAGE_ACCESSED));
+}
+
+static inline pte_t pte_mkspecial(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_SPECIAL);
+}
+
+static inline pte_t pte_wrprotect(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~(_PAGE_WRITE));
+}
+
+static inline pte_t pte_mkhuge(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_HUGE);
+}
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+
+#define pmdp_establish pmdp_establish
+static inline pmd_t pmdp_establish(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp, pmd_t pmd)
+{
+ return __pmd(xchg(&pmd_val(*pmdp), pmd_val(pmd)));
+}
+
+static inline int pmd_trans_huge(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_HUGE);
+}
+
+static inline pte_t pte_of_pmd(pmd_t pmd)
+{
+ return __pte(pmd_val(pmd));
+}
+
+static inline pmd_t pmd_of_pte(pte_t pte)
+{
+ return __pmd(pte_val(pte));
+}
+
+
+#define pmd_mkclean(pmd) pmd_of_pte(pte_mkclean(pte_of_pmd(pmd)))
+#define pmd_mkdirty(pmd) pmd_of_pte(pte_mkdirty(pte_of_pmd(pmd)))
+#define pmd_mkold(pmd) pmd_of_pte(pte_mkold(pte_of_pmd(pmd)))
+#define pmd_mkwrite(pmd) pmd_of_pte(pte_mkwrite(pte_of_pmd(pmd)))
+#define pmd_mkyoung(pmd) pmd_of_pte(pte_mkyoung(pte_of_pmd(pmd)))
+#define pmd_modify(pmd, prot) pmd_of_pte(pte_modify(pte_of_pmd(pmd), prot))
+#define pmd_wrprotect(pmd) pmd_of_pte(pte_wrprotect(pte_of_pmd(pmd)))
+
+static inline pmd_t pmd_mkhuge(pmd_t pmd)
+{
+ /* Create a huge page in PMD implies a size of 2 MB */
+ return __pmd(pmd_val(pmd) |
+ _PAGE_HUGE | (TLB_PS_2M << KVX_PAGE_SZ_SHIFT));
+}
+
+static inline pmd_t pmd_mkinvalid(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~(_PAGE_PRESENT);
+
+ return pmd;
+}
+
+#define pmd_dirty(pmd) pte_dirty(pte_of_pmd(pmd))
+#define pmd_write(pmd) pte_write(pte_of_pmd(pmd))
+#define pmd_young(pmd) pte_young(pte_of_pmd(pmd))
+
+#define mk_pmd(page, prot) pmd_of_pte(mk_pte(page, prot))
+
+#define pmd_pfn(pmd) pte_pfn(pte_of_pmd(pmd))
+
+static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
+{
+ return __pmd(((pfn << KVX_PFN_SHIFT) & KVX_PFN_MASK) |
+ pgprot_val(prot));
+}
+
+static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, pmd_t pmd)
+{
+ *pmdp = pmd;
+}
+
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
+#endif /* _ASM_KVX_PGTABLE_H */
diff --git a/arch/kvx/include/asm/rm_fw.h b/arch/kvx/include/asm/rm_fw.h
new file mode 100644
index 000000000000..f89bdd5915ed
--- /dev/null
+++ b/arch/kvx/include/asm/rm_fw.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_RM_FW_H
+#define _ASM_KVX_RM_FW_H
+
+#include <linux/sizes.h>
+
+#define KVX_RM_ID 16
+
+#define RM_FIRMWARE_REGS_SIZE (SZ_4K)
+
+#endif /* _ASM_KVX_RM_FW_H */
diff --git a/arch/kvx/include/asm/sparsemem.h b/arch/kvx/include/asm/sparsemem.h
new file mode 100644
index 000000000000..2f35743f20fb
--- /dev/null
+++ b/arch/kvx/include/asm/sparsemem.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SPARSEMEM_H
+#define _ASM_KVX_SPARSEMEM_H
+
+#ifdef CONFIG_SPARSEMEM
+#define MAX_PHYSMEM_BITS 40
+#define SECTION_SIZE_BITS 30
+#endif /* CONFIG_SPARSEMEM */
+
+#endif /* _ASM_KVX_SPARSEMEM_H */
diff --git a/arch/kvx/include/asm/symbols.h b/arch/kvx/include/asm/symbols.h
new file mode 100644
index 000000000000..a53c1607979f
--- /dev/null
+++ b/arch/kvx/include/asm/symbols.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SYMBOLS_H
+#define _ASM_KVX_SYMBOLS_H
+
+/* Symbols to patch TLB refill handler */
+extern char kvx_perf_tlb_refill[], kvx_std_tlb_refill[];
+
+/* Entry point of the ELF, used to start other PEs in SMP */
+extern int kvx_start[];
+
+#endif
diff --git a/arch/kvx/include/asm/tlb.h b/arch/kvx/include/asm/tlb.h
new file mode 100644
index 000000000000..190b682e1819
--- /dev/null
+++ b/arch/kvx/include/asm/tlb.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_TLB_H
+#define _ASM_KVX_TLB_H
+
+struct mmu_gather;
+
+static void tlb_flush(struct mmu_gather *tlb);
+
+int clear_ltlb_entry(unsigned long vaddr);
+
+#include <asm-generic/tlb.h>
+
+static inline unsigned int pgprot_cache_policy(unsigned long flags)
+{
+ return (flags & KVX_PAGE_CP_MASK) >> KVX_PAGE_CP_SHIFT;
+}
+
+#endif /* _ASM_KVX_TLB_H */
diff --git a/arch/kvx/include/asm/tlb_defs.h b/arch/kvx/include/asm/tlb_defs.h
new file mode 100644
index 000000000000..3f5b29cd529e
--- /dev/null
+++ b/arch/kvx/include/asm/tlb_defs.h
@@ -0,0 +1,131 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julian Vetter
+ * Guillaume Thouvenin
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_TLB_DEFS_H
+#define _ASM_KVX_TLB_DEFS_H
+
+#include <linux/sizes.h>
+
+#include <asm/sfr.h>
+
+/* Architecture specification */
+#define MMC_SB_JTLB 0
+#define MMC_SB_LTLB 1
+
+#define MMU_LTLB_SETS 1
+#define MMU_LTLB_WAYS 16
+
+#define MMU_JTLB_SETS 64
+#define MMU_JTLB_WAYS_SHIFT 2
+#define MMU_JTLB_WAYS (1 << MMU_JTLB_WAYS_SHIFT)
+
+#define MMU_JTLB_ENTRIES (MMU_JTLB_SETS << MMU_JTLB_WAYS_SHIFT)
+
+/* Set is determined using the 6 lsb of virtual page */
+#define MMU_JTLB_SET_MASK (MMU_JTLB_SETS - 1)
+#define MMU_JTLB_WAY_MASK (MMU_JTLB_WAYS - 1)
+
+/* TLB: Entry Status */
+#define TLB_ES_INVALID 0
+#define TLB_ES_PRESENT 1
+#define TLB_ES_MODIFIED 2
+#define TLB_ES_A_MODIFIED 3
+
+/* TLB: Cache Policy - First value is for data, the second is for instruction
+ * Symbols are
+ * D: device
+ * U: uncached
+ * W: write through
+ * C: cache enabled
+ */
+#define TLB_CP_D_U 0
+#define TLB_CP_U_U 1
+#define TLB_CP_W_C 2
+#define TLB_CP_U_C 3
+
+/* TLB: Protection Attributes: First value is when PM=0, second is when PM=1
+ * Symbols are:
+ * NA: no access
+ * R : read
+ * W : write
+ * X : execute
+ */
+#define TLB_PA_NA_NA 0
+#define TLB_PA_NA_R 1
+#define TLB_PA_NA_RW 2
+#define TLB_PA_NA_RX 3
+#define TLB_PA_NA_RWX 4
+#define TLB_PA_R_R 5
+#define TLB_PA_R_RW 6
+#define TLB_PA_R_RX 7
+#define TLB_PA_R_RWX 8
+#define TLB_PA_RW_RW 9
+#define TLB_PA_RW_RWX 10
+#define TLB_PA_RX_RX 11
+#define TLB_PA_RX_RWX 12
+#define TLB_PA_RWX_RWX 13
+
+/* TLB: Page Size */
+#define TLB_PS_4K 0
+#define TLB_PS_64K 1
+#define TLB_PS_2M 2
+#define TLB_PS_512M 3
+
+#define TLB_G_GLOBAL 1
+#define TLB_G_USE_ASN 0
+
+#define TLB_MK_TEH_ENTRY(_vaddr, _vs, _global, _asn) \
+ (((_vs) << KVX_SFR_TEH_VS_SHIFT) | \
+ ((_global) << KVX_SFR_TEH_G_SHIFT) | \
+ ((_asn) << KVX_SFR_TEH_ASN_SHIFT) | \
+ (((_vaddr) >> KVX_SFR_TEH_PN_SHIFT) << KVX_SFR_TEH_PN_SHIFT))
+
+#define TLB_MK_TEL_ENTRY(_paddr, _ps, _es, _cp, _pa) \
+ (((_es) << KVX_SFR_TEL_ES_SHIFT) | \
+ ((_ps) << KVX_SFR_TEL_PS_SHIFT) | \
+ ((_cp) << KVX_SFR_TEL_CP_SHIFT) | \
+ ((_pa) << KVX_SFR_TEL_PA_SHIFT) | \
+ (((_paddr) >> KVX_SFR_TEL_FN_SHIFT) << KVX_SFR_TEL_FN_SHIFT))
+
+
+/* Refill routine related defines */
+#define REFILL_PERF_ENTRIES 4
+#define REFILL_PERF_PAGE_SIZE SZ_512M
+
+/* paddr will be inserted in assembly code */
+#define REFILL_PERF_TEL_VAL \
+ TLB_MK_TEL_ENTRY(0, TLB_PS_512M, TLB_ES_A_MODIFIED, TLB_CP_W_C, \
+ TLB_PA_NA_RWX)
+/* vaddr will be inserted in assembly code */
+#define REFILL_PERF_TEH_VAL TLB_MK_TEH_ENTRY(0, 0, TLB_G_GLOBAL, 0)
+
+/*
+ * LTLB fixed entry index
+ */
+#define LTLB_ENTRY_KERNEL_TEXT 0
+#define LTLB_ENTRY_GDB_PAGE 1
+/* Reserve entries for kernel pagination */
+#define LTLB_KERNEL_RESERVED 2
+/* This define should reflect the maximum number of fixed LTLB entries */
+#define LTLB_ENTRY_FIXED_COUNT (LTLB_KERNEL_RESERVED + REFILL_PERF_ENTRIES)
+#define LTLB_ENTRY_EARLY_SMEM LTLB_ENTRY_FIXED_COUNT
+
+/* MMC: Protection Trap Cause */
+#define MMC_PTC_RESERVED 0
+#define MMC_PTC_READ 1
+#define MMC_PTC_WRITE 2
+#define MMC_PTC_EXECUTE 3
+
+/* MMC: Page size Mask in JTLB */
+#define MMC_PMJ_4K 1
+#define MMC_PMJ_64K 2
+#define MMC_PMJ_2M 4
+#define MMC_PMJ_512M 8
+
+#endif
diff --git a/arch/kvx/include/asm/tlbflush.h b/arch/kvx/include/asm/tlbflush.h
new file mode 100644
index 000000000000..7d3e50bbc957
--- /dev/null
+++ b/arch/kvx/include/asm/tlbflush.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_TLBFLUSH_H
+#define _ASM_KVX_TLBFLUSH_H
+
+#include <linux/sched.h>
+#include <linux/printk.h>
+#include <linux/mm_types.h>
+
+extern void local_flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long addr);
+extern void local_flush_tlb_all(void);
+extern void local_flush_tlb_mm(struct mm_struct *mm);
+extern void local_flush_tlb_range(struct vm_area_struct *vma,
+ unsigned long start,
+ unsigned long end);
+extern void local_flush_tlb_kernel_range(unsigned long start,
+ unsigned long end);
+
+#ifdef CONFIG_SMP
+extern void smp_flush_tlb_all(void);
+extern void smp_flush_tlb_mm(struct mm_struct *mm);
+extern void smp_flush_tlb_page(struct vm_area_struct *vma, unsigned long addr);
+extern void smp_flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end);
+extern void smp_flush_tlb_kernel_range(unsigned long start, unsigned long end);
+
+static inline void flush_tlb(void)
+{
+ smp_flush_tlb_mm(current->mm);
+}
+
+#define flush_tlb_page smp_flush_tlb_page
+#define flush_tlb_all smp_flush_tlb_all
+#define flush_tlb_mm smp_flush_tlb_mm
+#define flush_tlb_range smp_flush_tlb_range
+#define flush_tlb_kernel_range smp_flush_tlb_kernel_range
+
+#else
+#define flush_tlb_page local_flush_tlb_page
+#define flush_tlb_all local_flush_tlb_all
+#define flush_tlb_mm local_flush_tlb_mm
+#define flush_tlb_range local_flush_tlb_range
+#define flush_tlb_kernel_range local_flush_tlb_kernel_range
+#endif
+
+void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
+ pmd_t *pmd);
+
+void update_mmu_cache(struct vm_area_struct *vma,
+ unsigned long address, pte_t *ptep);
+
+#endif /* _ASM_KVX_TLBFLUSH_H */
diff --git a/arch/kvx/include/asm/vmalloc.h b/arch/kvx/include/asm/vmalloc.h
new file mode 100644
index 000000000000..a07692431108
--- /dev/null
+++ b/arch/kvx/include/asm/vmalloc.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_VMALLOC_H
+#define _ASM_KVX_VMALLOC_H
+
+#endif /* _ASM_KVX_VMALLOC_H */
diff --git a/arch/kvx/mm/cacheflush.c b/arch/kvx/mm/cacheflush.c
new file mode 100644
index 000000000000..2a632645f980
--- /dev/null
+++ b/arch/kvx/mm/cacheflush.c
@@ -0,0 +1,154 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/smp.h>
+#include <linux/hugetlb.h>
+#include <linux/mm_types.h>
+
+#include <asm/cacheflush.h>
+
+#ifdef CONFIG_SMP
+
+struct flush_data {
+ unsigned long start;
+ unsigned long end;
+};
+
+static inline void ipi_flush_icache_range(void *arg)
+{
+ struct flush_data *ta = arg;
+
+ local_flush_icache_range(ta->start, ta->end);
+}
+
+void flush_icache_range(unsigned long start, unsigned long end)
+{
+ struct flush_data data = {
+ .start = start,
+ .end = end
+ };
+
+ /* Then invalidate L1 icache on all cpus */
+ on_each_cpu(ipi_flush_icache_range, &data, 1);
+}
+EXPORT_SYMBOL(flush_icache_range);
+
+#endif /* CONFIG_SMP */
+
+void dcache_wb_inval_phys_range(phys_addr_t addr, unsigned long len, bool wb,
+ bool inval)
+{
+ if (wb && inval) {
+ wbinval_dcache_range(addr, len);
+ } else {
+ if (inval)
+ inval_dcache_range(addr, len);
+ if (wb)
+ wb_dcache_range(addr, len);
+ }
+}
+
+static inline pte_t *get_ptep(struct mm_struct *mm, unsigned long addr)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ pgd = pgd_offset(mm, addr);
+ if (pgd_none(*pgd))
+ return NULL;
+
+ p4d = p4d_offset(pgd, addr);
+ if (p4d_none(*p4d))
+ return NULL;
+
+ pud = pud_offset(p4d, addr);
+ if (pud_none(*pud))
+ return NULL;
+
+ pmd = pmd_offset(pud, addr);
+ if (pmd_none(*pmd))
+ return NULL;
+
+ if (pmd_huge(*pmd)) {
+ pte = (pte_t *) pmd;
+ if (!pte_present(*pte))
+ return NULL;
+
+ return pte;
+ }
+
+ pte = pte_offset_map(pmd, addr);
+ if (!pte_present(*pte))
+ return NULL;
+
+ return pte;
+}
+
+static unsigned long dcache_wb_inval_virt_to_phys(struct vm_area_struct *vma,
+ unsigned long vaddr,
+ unsigned long len,
+ bool wb, bool inval)
+{
+ unsigned long pfn, offset, pgsize;
+ pte_t *ptep;
+
+ ptep = get_ptep(vma->vm_mm, vaddr);
+ if (!ptep) {
+ /*
+ * Since we did not found a matching pte, return needed
+ * length to be aligned on next page boundary
+ */
+ offset = (vaddr & (PAGE_SIZE - 1));
+ return PAGE_SIZE - offset;
+ }
+
+ /* Handle page sizes correctly */
+ pgsize = (pte_val(*ptep) & KVX_PAGE_SZ_MASK) >> KVX_PAGE_SZ_SHIFT;
+ pgsize = (1 << get_page_size_shift(pgsize));
+
+ offset = vaddr & (pgsize - 1);
+ len = min(pgsize - offset, len);
+ pfn = pte_pfn(*ptep);
+
+ dcache_wb_inval_phys_range(PFN_PHYS(pfn) + offset, len, wb, inval);
+
+ return len;
+}
+
+int dcache_wb_inval_virt_range(unsigned long vaddr, unsigned long len, bool wb,
+ bool inval)
+{
+ unsigned long end = vaddr + len;
+ struct vm_area_struct *vma;
+ unsigned long rlen;
+ struct mm_struct *mm = current->mm;
+
+ /* necessary for find_vma */
+ mmap_read_lock(mm);
+
+ /*
+ * Verify that the specified address region actually belongs to this
+ * process.
+ */
+ vma = find_vma(current->mm, vaddr);
+ if (vma == NULL || vaddr < vma->vm_start || vaddr + len > vma->vm_end) {
+ mmap_read_unlock(mm);
+ return -EFAULT;
+ }
+
+ while (vaddr < end) {
+ rlen = dcache_wb_inval_virt_to_phys(vma, vaddr, len, wb, inval);
+ len -= rlen;
+ vaddr += rlen;
+ }
+
+ mmap_read_unlock(mm);
+
+ return 0;
+}
diff --git a/arch/kvx/mm/dma-mapping.c b/arch/kvx/mm/dma-mapping.c
new file mode 100644
index 000000000000..a9829432b974
--- /dev/null
+++ b/arch/kvx/mm/dma-mapping.c
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Jules Maselbas
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/iommu.h>
+
+#include <asm/cacheflush.h>
+
+void arch_dma_prep_coherent(struct page *page, size_t size)
+{
+ unsigned long addr = (unsigned long) page_to_phys(page);
+
+ /* Flush pending data and invalidate pages */
+ wbinval_dcache_range(addr, size);
+}
+
+/**
+ * The implementation of arch should follow the following rules:
+ * map for_cpu for_device unmap
+ * TO_DEV writeback none writeback none
+ * FROM_DEV invalidate invalidate(*) invalidate invalidate(*)
+ * BIDIR writeback invalidate writeback invalidate
+ *
+ * (*) - only necessary if the CPU speculatively prefetches.
+ *
+ * (see https://lkml.org/lkml/2018/5/18/979)
+ */
+void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir)
+{
+ switch (dir) {
+ case DMA_FROM_DEVICE:
+ inval_dcache_range(paddr, size);
+ break;
+
+ case DMA_TO_DEVICE:
+ case DMA_BIDIRECTIONAL:
+ wb_dcache_range(paddr, size);
+ break;
+
+ default:
+ BUG();
+ }
+}
+
+void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir)
+{
+ switch (dir) {
+ case DMA_TO_DEVICE:
+ break;
+ case DMA_FROM_DEVICE:
+#ifdef CONFIG_L2_CACHE
+ /* kvx does not do speculative loads by itself, however
+ * L2 cache lines are bigger than L1 cache lines, which can
+ * cause a "false sharing" situation where two L1 cache lines
+ * share the same L2 cache line. There is a high chance that
+ * the L2 cache line has already been refilled by a previous
+ * memory access.
+ */
+ inval_dcache_range(paddr, size);
+#endif
+ break;
+
+ case DMA_BIDIRECTIONAL:
+ inval_dcache_range(paddr, size);
+ break;
+
+ default:
+ BUG();
+ }
+}
+
+#ifdef CONFIG_IOMMU_DMA
+void arch_teardown_dma_ops(struct device *dev)
+{
+ dev->dma_ops = NULL;
+}
+#endif /* CONFIG_IOMMU_DMA*/
+
+void arch_setup_dma_ops(struct device *dev,
+ u64 dma_base,
+ u64 size,
+ const struct iommu_ops *iommu,
+ bool coherent)
+{
+ dev->dma_coherent = coherent;
+ if (iommu)
+ iommu_setup_dma_ops(dev, dma_base, size);
+}
diff --git a/arch/kvx/mm/extable.c b/arch/kvx/mm/extable.c
new file mode 100644
index 000000000000..8889a1a90a77
--- /dev/null
+++ b/arch/kvx/mm/extable.c
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/riscv/mm/extable.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+
+#include <linux/extable.h>
+#include <linux/module.h>
+#include <linux/uaccess.h>
+
+int fixup_exception(struct pt_regs *regs)
+{
+ const struct exception_table_entry *fixup;
+
+ fixup = search_exception_tables(regs->spc);
+ if (fixup) {
+ regs->spc = fixup->fixup;
+ return 1;
+ }
+ return 0;
+}
diff --git a/arch/kvx/mm/fault.c b/arch/kvx/mm/fault.c
new file mode 100644
index 000000000000..17f6325f3eab
--- /dev/null
+++ b/arch/kvx/mm/fault.c
@@ -0,0 +1,264 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Yann Sionneau
+ */
+
+#include <linux/types.h>
+#include <linux/uaccess.h>
+#include <linux/kernel.h> // only needed to panic
+#include <linux/printk.h>
+#include <linux/perf_event.h>
+#include <linux/sched.h>
+#include <linux/sched/debug.h>
+#include <linux/sched/signal.h>
+#include <linux/mm.h>
+
+#include <asm/mmu.h>
+#include <asm/traps.h>
+#include <asm/ptrace.h>
+#include <asm/pgtable.h>
+#include <asm/sfr_defs.h>
+#include <asm/current.h>
+#include <asm/tlbflush.h>
+
+static int handle_vmalloc_fault(uint64_t ea)
+{
+ /*
+ * Synchronize this task's top level page-table with
+ * the 'reference' page table.
+ * As we only have 2 or 3 level page table we don't need to
+ * deal with other levels.
+ */
+ unsigned long addr = ea & PAGE_MASK;
+ pgd_t *pgd_k, *pgd;
+ p4d_t *p4d_k, *p4d;
+ pud_t *pud_k, *pud;
+ pmd_t *pmd_k, *pmd;
+ pte_t *pte_k;
+
+ pgd = pgd_offset(current->active_mm, ea);
+ pgd_k = pgd_offset_k(ea);
+ if (!pgd_present(*pgd_k))
+ return 1;
+ set_pgd(pgd, *pgd_k);
+
+ p4d = p4d_offset(pgd, ea);
+ p4d_k = p4d_offset(pgd_k, ea);
+ if (!p4d_present(*p4d_k))
+ return 1;
+
+ pud = pud_offset(p4d, ea);
+ pud_k = pud_offset(p4d_k, ea);
+ if (!pud_present(*pud_k))
+ return 1;
+
+ pmd = pmd_offset(pud, ea);
+ pmd_k = pmd_offset(pud_k, ea);
+ if (!pmd_present(*pmd_k))
+ return 1;
+
+ /* Some other architectures set pmd to synchronize them but
+ * as we just synchronized the pgd we don't see how they can
+ * be different. Maybe we miss something so in case we
+ * put a guard here.
+ */
+ if (pmd_val(*pmd) != pmd_val(*pmd_k))
+ pr_err("%s: pmd not synchronized (0x%lx != 0x%lx)\n",
+ __func__, pmd_val(*pmd), pmd_val(*pmd_k));
+
+ pte_k = pte_offset_kernel(pmd_k, ea);
+ if (!pte_present(*pte_k)) {
+ pr_err("%s: PTE not present for 0x%llx\n",
+ __func__, ea);
+ return 1;
+ }
+
+ /* We refill the TLB now to avoid to take another nomapping
+ * trap.
+ */
+ kvx_mmu_jtlb_add_entry(addr, pte_k, 0);
+
+ return 0;
+}
+
+void do_page_fault(uint64_t es, uint64_t ea, struct pt_regs *regs)
+{
+ struct mm_struct *mm;
+ struct vm_area_struct *vma;
+ unsigned long flags, cause, vma_mask;
+ int code = SEGV_MAPERR;
+ vm_fault_t fault;
+
+ cause = kvx_sfr_field_val(es, ES, RWX);
+
+ /* We fault-in kernel-space virtual memory on-demand. The
+ * 'reference' page table is init_mm.pgd.
+ */
+ if (is_vmalloc_addr((void *) ea) && !user_mode(regs)) {
+ if (handle_vmalloc_fault(ea))
+ goto no_context;
+ return;
+ }
+
+ mm = current->mm;
+
+ /*
+ * If we're in an interrupt or have no user
+ * context, we must not take the fault..
+ */
+ if (unlikely(faulthandler_disabled() || !mm))
+ goto no_context;
+
+ /* By default we retry and fault task can be killed */
+ flags = FAULT_FLAG_DEFAULT;
+
+ if (user_mode(regs))
+ flags |= FAULT_FLAG_USER;
+
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, ea);
+
+retry:
+ mmap_read_lock(mm);
+
+ vma = find_vma(mm, ea);
+ if (!vma)
+ goto bad_area;
+ if (likely(vma->vm_start <= ea))
+ goto good_area;
+ if (unlikely(!(vma->vm_flags & VM_GROWSDOWN)))
+ goto bad_area;
+ if (unlikely(expand_stack(vma, ea)))
+ goto bad_area;
+
+good_area:
+ /* Handle access type */
+ switch (cause) {
+ case KVX_TRAP_RWX_FETCH:
+ vma_mask = VM_EXEC;
+ break;
+ case KVX_TRAP_RWX_READ:
+ vma_mask = VM_READ;
+ break;
+ case KVX_TRAP_RWX_WRITE:
+ vma_mask = VM_WRITE;
+ flags |= FAULT_FLAG_WRITE;
+ break;
+ /* Atomic are both read/write */
+ case KVX_TRAP_RWX_ATOMIC:
+ vma_mask = VM_WRITE | VM_READ;
+ flags |= FAULT_FLAG_WRITE;
+ break;
+ default:
+ panic("%s: unhandled cause %lu", __func__, cause);
+ }
+
+ if ((vma->vm_flags & vma_mask) != vma_mask) {
+ code = SEGV_ACCERR;
+ goto bad_area;
+ }
+
+ /*
+ * If for any reason we can not handle the fault we make sure that
+ * we exit gracefully rather then retry endlessly with the same
+ * result.
+ */
+ fault = handle_mm_fault(vma, ea, flags, regs);
+
+ /*
+ * If we need to retry but a fatal signal is pending, handle the
+ * signal first. We do not need to release the mmap_sem because it
+ * would already be released in __lock_page_or_retry in mm/filemap.c.
+ */
+ if (fault_signal_pending(fault, regs))
+ return;
+
+ if (unlikely(fault & VM_FAULT_ERROR)) {
+ if (fault & VM_FAULT_OOM)
+ goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
+ else if (fault & VM_FAULT_SIGBUS)
+ goto do_sigbus;
+ BUG();
+ }
+
+ if (unlikely((fault & VM_FAULT_RETRY) && (flags & FAULT_FLAG_ALLOW_RETRY))) {
+ flags |= FAULT_FLAG_TRIED;
+ /* No need to up_read(&mm->mmap_sem) as we would
+ * have already released it in __lock_page_or_retry().
+ * Look in mm/filemap.c for explanations.
+ */
+ goto retry;
+ }
+
+ /* Fault errors and retry case have been handled nicely */
+ mmap_read_unlock(mm);
+ return;
+
+bad_area:
+ mmap_read_unlock(mm);
+
+ if (user_mode(regs)) {
+ user_do_sig(regs, SIGSEGV, code, ea);
+ return;
+ }
+
+no_context:
+ /* Are we prepared to handle this kernel fault?
+ *
+ * (The kernel has valid exception-points in the source
+ * when it accesses user-memory. When it fails in one
+ * of those points, we find it in a table and do a jump
+ * to some fixup code that loads an appropriate error
+ * code)
+ */
+ if (fixup_exception(regs))
+ return;
+
+ /*
+ * Oops. The kernel tried to access some bad page. We'll have to
+ * terminate things with extreme prejudice.
+ */
+ bust_spinlocks(1);
+ if (kvx_sfr_field_val(es, ES, HTC) == KVX_TRAP_PROTECTION)
+ pr_alert(CUT_HERE "Kernel protection trap at virtual address %016llx\n",
+ ea);
+ else {
+ pr_alert(CUT_HERE "Unable to handle kernel %s at virtual address %016llx\n",
+ (ea < PAGE_SIZE) ? "NULL pointer dereference" :
+ "paging request", ea);
+ }
+ die(regs, ea, "Oops");
+ bust_spinlocks(0);
+ make_task_dead(SIGKILL);
+
+out_of_memory:
+ /*
+ * We ran out of memory, call the OOM killer, and return the userspace
+ * (which will retry the fault, or kill us if we got oom-killed).
+ */
+ mmap_read_unlock(mm);
+ if (!user_mode(regs))
+ goto no_context;
+ pagefault_out_of_memory();
+ return;
+
+do_sigbus:
+ mmap_read_unlock(mm);
+ /* Kernel mode? Handle exceptions or die */
+ if (!user_mode(regs))
+ goto no_context;
+
+ user_do_sig(regs, SIGBUS, BUS_ADRERR, ea);
+
+ return;
+
+}
+
+void do_writetoclean(uint64_t es, uint64_t ea, struct pt_regs *regs)
+{
+ panic("%s not implemented", __func__);
+}
diff --git a/arch/kvx/mm/hugetlbpage.c b/arch/kvx/mm/hugetlbpage.c
new file mode 100644
index 000000000000..c1291fdac708
--- /dev/null
+++ b/arch/kvx/mm/hugetlbpage.c
@@ -0,0 +1,317 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Jean-Christophe Pince
+ */
+
+#include <asm/page.h>
+
+#include <linux/bits.h>
+#include <linux/mm.h>
+#include <linux/hugetlb.h>
+
+/**
+ * get_nr_cont_huge_pages - return the number of contiguous pages
+ * @pte: a PTE
+ *
+ * Return: the number of contiguous entries used by the huge page according to
+ * the huge page size.
+ */
+static unsigned int get_nr_cont_huge_pages(pte_t pte)
+{
+ unsigned int psize, nr_cont;
+ unsigned long ptev = pte_val(pte);
+
+ psize = (ptev & KVX_PAGE_SZ_MASK) >> KVX_PAGE_SZ_SHIFT;
+
+ if (psize == TLB_PS_64K)
+ nr_cont = KVX_PAGE_64K_NR_CONT;
+ else if (psize == TLB_PS_512M)
+ nr_cont = KVX_PAGE_512M_NR_CONT;
+ else
+ /* Only page of 64 KB or 512 MB require more than 1 entry */
+ nr_cont = 1;
+
+ return nr_cont;
+}
+
+void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte)
+{
+ unsigned int nr_cont, i;
+
+ nr_cont = get_nr_cont_huge_pages(pte);
+
+ for (i = 0; i < nr_cont; i++, ptep++)
+ set_pte_at(mm, addr, ptep, pte);
+}
+
+pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+ unsigned long addr,
+ pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+ pte_t pte = huge_ptep_get(ptep);
+
+ nr_cont = get_nr_cont_huge_pages(pte);
+
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ if (pte_dirty(*ptep))
+ pte = pte_mkdirty(pte);
+ if (pte_young(*ptep))
+ pte = pte_mkyoung(pte);
+ pte_clear(mm, addr, ptep);
+ }
+
+ flush_tlb_mm(mm);
+ return pte;
+}
+
+static pte_t get_dirty_young_from_cont(pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+ pte_t pte_orig = huge_ptep_get(ptep);
+
+ nr_cont = get_nr_cont_huge_pages(pte_orig);
+
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ pte_t pte = huge_ptep_get(ptep);
+
+ if (pte_dirty(pte))
+ pte_orig = pte_mkdirty(pte_orig);
+
+ if (pte_young(pte))
+ pte_orig = pte_mkyoung(pte_orig);
+ }
+
+ return pte_orig;
+}
+
+int huge_ptep_set_access_flags(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep,
+ pte_t pte, int dirty)
+{
+ unsigned int nr_cont, i;
+ int flush = 0;
+ pte_t pte_tmp;
+
+ nr_cont = get_nr_cont_huge_pages(huge_ptep_get(ptep));
+
+ /*
+ * As it is done by ARM64, make sure we don't lose the dirty or young
+ * state. So first we read dirty and young bits from all contiguous
+ * PTEs and update the pte given as argument if needed.
+ */
+ pte_tmp = get_dirty_young_from_cont(ptep);
+ if (pte_dirty(pte_tmp))
+ pte = pte_mkdirty(pte);
+
+ if (pte_young(pte_tmp))
+ pte = pte_mkyoung(pte);
+
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ if (!pte_same(*ptep, pte)) {
+ set_pte_at(vma->vm_mm, addr, ptep, pte);
+ flush = 1;
+ }
+ }
+
+ if (flush)
+ flush_tlb_page(vma, addr);
+
+ return flush;
+}
+
+void huge_ptep_set_wrprotect(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+
+ nr_cont = get_nr_cont_huge_pages(huge_ptep_get(ptep));
+
+ for (i = 0; i < nr_cont; i++, ptep++)
+ ptep_set_wrprotect(mm, addr, ptep);
+}
+
+
+pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+ struct mm_struct *mm;
+ int flush = 0;
+ pte_t pte_orig = huge_ptep_get(ptep);
+
+ nr_cont = get_nr_cont_huge_pages(huge_ptep_get(ptep));
+
+ mm = vma->vm_mm;
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ BUG_ON(pte_orig.pte != pte_val(*ptep));
+ pte_clear(mm, address, ptep);
+ if (pte_accessible(mm, *ptep))
+ flush = 1;
+ }
+
+ if (flush)
+ flush_tlb_page(vma, addr);
+
+ return pte_orig;
+}
+
+pte_t arch_make_huge_pte(pte_t entry, unsigned int shift,
+ vm_flags_t flags)
+{
+ unsigned long ptev;
+
+ ptev = pte_val(entry) & ~(KVX_PAGE_SZ_MASK);
+
+ switch (shift) {
+ case KVX_PAGE_64K_SHIFT:
+ ptev |= _PAGE_SZ_64K;
+ break;
+ case KVX_PAGE_2M_SHIFT:
+ ptev |= _PAGE_SZ_2M;
+ break;
+ case KVX_PAGE_512M_SHIFT:
+ ptev |= _PAGE_SZ_512M;
+ break;
+ default:
+ ptev = 0;
+ pr_err("huge page shift %d not supported\n", shift);
+ BUG();
+ }
+
+ return __pte(ptev);
+}
+
+pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
+ unsigned long addr,
+ unsigned long size)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd = NULL;
+
+ pgd = pgd_offset(mm, addr);
+ if (pgd_present(*pgd)) {
+ p4d = p4d_offset(pgd, addr);
+ if (p4d_present(*p4d)) {
+ pud = pud_offset(p4d, addr);
+ if (pud_present(*pud))
+ pmd = pmd_alloc(mm, pud, addr);
+ }
+ }
+
+ if (size > KVX_PAGE_64K_SIZE)
+ return (pte_t *)pmd;
+
+ return pmd ? pte_alloc_map(mm, pmd, addr) : NULL;
+}
+
+/**
+ * huge_pte_offset - get the offset of the huge page
+ * @mm: the memory structure
+ * @addr: the address
+ * @size: size of the memory area
+ *
+ * On kvx the huge page are backed bye PMD or PTE depending of the size of the
+ * huge page. Huge pages larger or equal to 2 MB are backed by PMD, smaller are
+ * backed by PTE.
+ *
+ * Return: pointer to the huge page if it exists, NULL otherwise.
+ */
+pte_t *huge_pte_offset(struct mm_struct *mm,
+ unsigned long addr,
+ unsigned long size)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ /*
+ * 64 KB and 512 MB huge pages require the usage of contiguous page so
+ * we need to be aligned for this pages in order to get the right
+ * offset (ie the offset of the first index of contiguous pages).
+ */
+ if (size == KVX_PAGE_64K_SIZE)
+ addr &= KVX_PAGE_64K_MASK;
+ else if (size == KVX_PAGE_512M_SIZE)
+ addr &= KVX_PAGE_512M_MASK;
+
+ pgd = pgd_offset(mm, addr);
+ if (!pgd_present(*pgd))
+ return NULL;
+
+ p4d = p4d_offset(pgd, addr);
+ if (!p4d_present(*p4d))
+ return NULL;
+
+ pud = pud_offset(p4d, addr);
+ if (!pud_present(*pud))
+ return NULL;
+
+ pmd = pmd_offset(pud, addr);
+
+ if (size == KVX_PAGE_64K_SIZE) {
+ /* we need to go deeper in the page table */
+ if (pmd_present(*pmd))
+ pte = pte_offset_kernel(pmd, addr);
+ else
+ pte = NULL;
+ } else {
+ pte = (pte_t *)pmd;
+ }
+
+ return pte;
+}
+
+int pmd_huge(pmd_t pmd)
+{
+ return pmd_val(pmd) & _PAGE_HUGE;
+}
+
+int pud_huge(pud_t pud)
+{
+ return 0;
+}
+
+static void __init add_huge_page_size(unsigned long size)
+{
+ if (size_to_hstate(size)) {
+ WARN(1, "Failed to add huge page size %lu\n", size);
+ return;
+ }
+
+ hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
+}
+
+static int __init hugetlbpage_init(void)
+{
+#if defined(CONFIG_KVX_4K_PAGES)
+ add_huge_page_size(KVX_PAGE_64K_SIZE);
+ add_huge_page_size(KVX_PAGE_2M_SIZE);
+ add_huge_page_size(KVX_PAGE_512M_SIZE);
+#endif
+
+ return 0;
+}
+arch_initcall(hugetlbpage_init);
+
+bool __init arch_hugetlb_valid_size(unsigned long size)
+{
+#if defined(CONFIG_KVX_4K_PAGES)
+ switch (size) {
+ case KVX_PAGE_64K_SIZE:
+ case KVX_PAGE_2M_SIZE:
+ case KVX_PAGE_512M_SIZE:
+ return true;
+ }
+#endif
+
+ return false;
+}
diff --git a/arch/kvx/mm/init.c b/arch/kvx/mm/init.c
new file mode 100644
index 000000000000..abc4620d9cf3
--- /dev/null
+++ b/arch/kvx/mm/init.c
@@ -0,0 +1,527 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+/* Memblock header depends on types.h but does not include it ! */
+#include <linux/types.h>
+#include <linux/memblock.h>
+#include <linux/mmzone.h>
+#include <linux/of_fdt.h>
+#include <linux/sched.h>
+#include <linux/sizes.h>
+#include <linux/init.h>
+#include <linux/initrd.h>
+#include <linux/pfn.h>
+#include <linux/mm.h>
+
+#include <asm/sections.h>
+#include <asm/tlb_defs.h>
+#include <asm/tlbflush.h>
+#include <asm/fixmap.h>
+#include <asm/page.h>
+
+/*
+ * On kvx, memory map contains the first 2G of DDR being aliased.
+ * Full contiguous DDR is located at @[4G - 68G].
+ * However, to access this DDR in 32bit mode, the first 2G of DDR are
+ * mirrored from 4G to 2G.
+ * These first 2G are accessible from all DMAs (included 32 bits one).
+ *
+ * Hence, the memory map is the following:
+ *
+ * (68G) 0x1100000000-> +-------------+
+ * | |
+ * 66G |(ZONE_NORMAL)|
+ * | |
+ * (6G) 0x180000000-> +-------------+
+ * | |
+ * 2G |(ZONE_DMA32) |
+ * | |
+ * (4G) 0x100000000-> +-------------+ +--+
+ * | | |
+ * 2G | (Alias) | | 2G Alias
+ * | | |
+ * (2G) 0x80000000-> +-------------+ <--+
+ *
+ * The translation of 64 bits -> 32 bits can then be done using dma-ranges property
+ * in device-trees.
+ */
+
+#define DDR_64BIT_START (4ULL * SZ_1G)
+#define DDR_32BIT_ALIAS_SIZE (2ULL * SZ_1G)
+
+#define MAX_DMA32_PFN PHYS_PFN(DDR_64BIT_START + DDR_32BIT_ALIAS_SIZE)
+
+pgd_t swapper_pg_dir[PTRS_PER_PGD] __page_aligned_bss;
+
+/*
+ * empty_zero_page is a special page that is used for zero-initialized data and
+ * COW.
+ */
+struct page *empty_zero_page;
+EXPORT_SYMBOL(empty_zero_page);
+
+extern char _start[];
+extern char __kernel_smem_code_start[];
+extern char __kernel_smem_code_end[];
+
+struct kernel_section {
+ phys_addr_t start;
+ phys_addr_t end;
+};
+
+struct kernel_section kernel_sections[] = {
+ {
+ .start = (phys_addr_t)__kernel_smem_code_start,
+ .end = (phys_addr_t)__kernel_smem_code_end
+ },
+ {
+ .start = __pa(_start),
+ .end = __pa(_end)
+ }
+};
+
+static void __init zone_sizes_init(void)
+{
+ unsigned long zones_size[MAX_NR_ZONES];
+
+ memset(zones_size, 0, sizeof(zones_size));
+
+ zones_size[ZONE_DMA32] = min(MAX_DMA32_PFN, max_low_pfn);
+ zones_size[ZONE_NORMAL] = max_low_pfn;
+
+ free_area_init(zones_size);
+}
+
+#ifdef CONFIG_BLK_DEV_INITRD
+static void __init setup_initrd(void)
+{
+ u64 base = phys_initrd_start;
+ u64 end = phys_initrd_start + phys_initrd_size;
+
+ if (phys_initrd_size == 0) {
+ pr_info("initrd not found or empty");
+ return;
+ }
+
+ if (base < memblock_start_of_DRAM() || end > memblock_end_of_DRAM()) {
+ pr_err("initrd not in accessible memory, disabling it");
+ phys_initrd_size = 0;
+ return;
+ }
+
+ pr_info("initrd: 0x%llx - 0x%llx\n", base, end);
+
+ memblock_reserve(phys_initrd_start, phys_initrd_size);
+
+ /* the generic initrd code expects virtual addresses */
+ initrd_start = (unsigned long) __va(base);
+ initrd_end = initrd_start + phys_initrd_size;
+}
+#endif
+
+static phys_addr_t memory_limit = PHYS_ADDR_MAX;
+
+static int __init early_mem(char *p)
+{
+ if (!p)
+ return 1;
+
+ memory_limit = memparse(p, &p) & PAGE_MASK;
+ pr_notice("Memory limited to %lldMB\n", memory_limit >> 20);
+
+ return 0;
+}
+early_param("mem", early_mem);
+
+static void __init setup_bootmem(void)
+{
+ phys_addr_t kernel_start, kernel_end;
+ phys_addr_t start, end = 0;
+ u64 i;
+
+ init_mm.start_code = (unsigned long)_stext;
+ init_mm.end_code = (unsigned long)_etext;
+ init_mm.end_data = (unsigned long)_edata;
+ init_mm.brk = (unsigned long)_end;
+
+ for (i = 0; i < ARRAY_SIZE(kernel_sections); i++) {
+ kernel_start = kernel_sections[i].start;
+ kernel_end = kernel_sections[i].end;
+
+ memblock_reserve(kernel_start, kernel_end - kernel_start);
+ }
+
+ for_each_mem_range(i, &start, &end) {
+ pr_info("%15s: memory : 0x%lx - 0x%lx\n", __func__,
+ (unsigned long)start,
+ (unsigned long)end);
+ }
+
+ /* min_low_pfn is the lowest PFN available in the system */
+ min_low_pfn = PFN_UP(memblock_start_of_DRAM());
+
+ /* max_low_pfn indicates the end if NORMAL zone */
+ max_low_pfn = PFN_DOWN(memblock_end_of_DRAM());
+
+ /* Set the maximum number of pages in the system */
+ set_max_mapnr(max_low_pfn - min_low_pfn);
+
+#ifdef CONFIG_BLK_DEV_INITRD
+ setup_initrd();
+#endif
+
+ if (memory_limit != PHYS_ADDR_MAX)
+ memblock_mem_limit_remove_map(memory_limit);
+
+ /* Don't reserve the device tree if its builtin */
+ if (!is_kernel_rodata((unsigned long) initial_boot_params))
+ early_init_fdt_reserve_self();
+ early_init_fdt_scan_reserved_mem();
+
+ memblock_allow_resize();
+ memblock_dump_all();
+}
+
+static void * __init alloc_page_table(void)
+{
+ void *pgt;
+
+ pgt = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+ BUG_ON(!pgt);
+
+ return pgt;
+}
+
+static pmd_t fixmap_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused;
+static pte_t fixmap_pte[PTRS_PER_PTE] __page_aligned_bss __maybe_unused;
+
+void __init early_fixmap_init(void)
+{
+ unsigned long vaddr;
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+
+ /*
+ * Fixed mappings:
+ */
+ vaddr = __fix_to_virt(__end_of_fixed_addresses - 1);
+ pgd = pgd_offset_pgd(swapper_pg_dir, vaddr);
+ set_pgd(pgd, __pgd(__pa_symbol(fixmap_pmd)));
+
+ p4d = p4d_offset(pgd, vaddr);
+ pud = pud_offset(p4d, vaddr);
+ pmd = pmd_offset(pud, vaddr);
+ set_pmd(pmd, __pmd(__pa_symbol(fixmap_pte)));
+}
+
+#ifdef CONFIG_STRICT_KERNEL_RWX
+
+static bool use_huge_page(unsigned long start, unsigned long end,
+ phys_addr_t phys, unsigned long page_size)
+{
+ unsigned long size = end - start;
+
+ return IS_ALIGNED(start | phys, page_size) && (size >= page_size);
+}
+
+static void create_pte_mapping(pmd_t *pmdp, unsigned long va_start,
+ unsigned long va_end, phys_addr_t phys,
+ pgprot_t prot, bool alloc_pgtable)
+{
+ pmd_t pmd = READ_ONCE(*pmdp);
+ unsigned long nr_cont, i;
+ pte_t *ptep = NULL;
+ pgprot_t pte_prot;
+
+ if (pmd_none(pmd)) {
+ BUG_ON(!alloc_pgtable);
+ ptep = alloc_page_table();
+ set_pmd(pmdp, __pmd(__pa(ptep)));
+ pmd = READ_ONCE(*pmdp);
+ }
+ BUG_ON(pmd_bad(pmd));
+
+ ptep = pte_offset_kernel(pmdp, va_start);
+
+ do {
+ /* Try to use 64K page whenever it is possible */
+ if (use_huge_page(va_start, va_end, phys, KVX_PAGE_64K_SIZE)) {
+ pte_prot = __pgprot(pgprot_val(prot) | _PAGE_SZ_64K |
+ _PAGE_HUGE);
+ nr_cont = KVX_PAGE_64K_NR_CONT;
+ } else {
+ pte_prot = prot;
+ nr_cont = 1;
+ }
+
+ for (i = 0; i < nr_cont; i++) {
+ set_pte(ptep, pfn_pte(phys_to_pfn(phys), pte_prot));
+ ptep++;
+ }
+
+ phys += nr_cont * PAGE_SIZE;
+ va_start += nr_cont * PAGE_SIZE;
+ } while (va_start != va_end);
+}
+
+static void create_pmd_mapping(pgd_t *pgdp, unsigned long va_start,
+ unsigned long va_end, phys_addr_t phys,
+ pgprot_t prot, bool alloc_pgtable)
+{
+ unsigned long next, huge_size, i, nr_cont;
+ pgprot_t pmd_prot;
+ bool use_huge;
+ pmd_t *pmdp;
+ pud_t *pudp;
+ p4d_t *p4dp;
+ pud_t pud;
+ pte_t pte;
+
+ p4dp = p4d_offset(pgdp, va_start);
+ pudp = pud_offset(p4dp, va_start);
+ pud = READ_ONCE(*pudp);
+
+ if (pud_none(pud)) {
+ BUG_ON(!alloc_pgtable);
+ pmdp = alloc_page_table();
+ set_pud(pudp, __pud(__pa(pmdp)));
+ pud = READ_ONCE(*pudp);
+ }
+ BUG_ON(pud_bad(pud));
+
+ pmdp = pmd_offset(pudp, va_start);
+
+ do {
+ next = pmd_addr_end(va_start, va_end);
+
+ /* Try to use huge pages (2M, 512M) whenever it is possible */
+ if (use_huge_page(va_start, next, phys, KVX_PAGE_2M_SIZE)) {
+ pmd_prot = __pgprot(pgprot_val(prot) | _PAGE_SZ_2M);
+ nr_cont = 1;
+ huge_size = KVX_PAGE_2M_SIZE;
+ use_huge = true;
+ } else if (use_huge_page(va_start, next, phys,
+ KVX_PAGE_512M_SIZE)) {
+ pmd_prot = __pgprot(pgprot_val(prot) | _PAGE_SZ_512M);
+ nr_cont = KVX_PAGE_512M_NR_CONT;
+ huge_size = KVX_PAGE_512M_SIZE;
+ use_huge = true;
+ } else {
+ use_huge = false;
+ }
+
+ if (use_huge) {
+ pmd_prot = __pgprot(pgprot_val(pmd_prot) | _PAGE_HUGE);
+ pte = pfn_pte(phys_to_pfn(phys), pmd_prot);
+ for (i = 0; i < nr_cont; i++) {
+ set_pmd(pmdp, __pmd(pte_val(pte)));
+ pmdp++;
+ }
+ } else {
+ create_pte_mapping(pmdp, va_start, next, phys, prot,
+ alloc_pgtable);
+ pmdp++;
+ }
+
+ phys += next - va_start;
+ } while (va_start = next, va_start != va_end);
+}
+
+static void create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys,
+ unsigned long va_start, unsigned long va_end,
+ pgprot_t prot, bool alloc_pgtable)
+{
+ unsigned long next;
+ pgd_t *pgdp = pgd_offset_pgd(pgdir, va_start);
+
+ BUG_ON(!PAGE_ALIGNED(phys));
+ BUG_ON(!PAGE_ALIGNED(va_start));
+ BUG_ON(!PAGE_ALIGNED(va_end));
+
+ do {
+ next = pgd_addr_end(va_start, va_end);
+ create_pmd_mapping(pgdp, va_start, next, phys, prot,
+ alloc_pgtable);
+ phys += next - va_start;
+ } while (pgdp++, va_start = next, va_start != va_end);
+}
+
+static void __init map_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end,
+ pgprot_t prot)
+{
+ phys_addr_t pa_start = __pa(va_start);
+
+ create_pgd_mapping(pgdp, pa_start, (unsigned long) va_start,
+ (unsigned long) va_end, prot, true);
+}
+
+static void remap_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end,
+ pgprot_t prot)
+{
+ phys_addr_t pa_start = __pa(va_start);
+
+ create_pgd_mapping(pgdp, pa_start, (unsigned long) va_start,
+ (unsigned long) va_end, prot, false);
+
+ flush_tlb_kernel_range((unsigned long) va_start,
+ (unsigned long) va_end);
+}
+
+/*
+ * Create fine-grained mappings for the kernel.
+ */
+static void __init map_kernel(void)
+{
+ pgprot_t text_prot = PAGE_KERNEL_ROX;
+
+ /*
+ * External debuggers may need to write directly to the text
+ * mapping to install SW breakpoints. Allow this (only) when
+ * explicitly requested with rodata=off.
+ */
+ if (!rodata_enabled)
+ text_prot = PAGE_KERNEL_EXEC;
+
+ map_kernel_segment(swapper_pg_dir, __inittext_start, __inittext_end,
+ text_prot);
+ map_kernel_segment(swapper_pg_dir, __initdata_start, __initdata_end,
+ PAGE_KERNEL);
+ map_kernel_segment(swapper_pg_dir, __rodata_start, __rodata_end,
+ PAGE_KERNEL);
+ map_kernel_segment(swapper_pg_dir, _sdata, _end, PAGE_KERNEL);
+ /* We skip exceptions mapping to avoid multimappings */
+ map_kernel_segment(swapper_pg_dir, __exception_end, _etext, text_prot);
+}
+
+static void __init map_memory(void)
+{
+ phys_addr_t start, end;
+ phys_addr_t kernel_start = __pa(__inittext_start);
+ phys_addr_t kernel_end = __pa_symbol(_end);
+ u64 i;
+
+ /**
+ * Mark the full kernel text/data as nomap to avoid remapping all
+ * section as RW.
+ */
+ memblock_mark_nomap(kernel_start, kernel_end - kernel_start);
+
+ /* Map all memory banks */
+ for_each_mem_range(i, &start, &end) {
+
+ if (start >= end)
+ break;
+
+ create_pgd_mapping(swapper_pg_dir, start,
+ (unsigned long) __va(start),
+ (unsigned long) __va(end), PAGE_KERNEL_EXEC,
+ true);
+ }
+ memblock_clear_nomap(kernel_start, kernel_end - kernel_start);
+}
+
+void mark_rodata_ro(void)
+{
+ remap_kernel_segment(swapper_pg_dir, __rodata_start, __rodata_end,
+ PAGE_KERNEL_RO);
+}
+
+void __init paging_init(void)
+{
+ map_kernel();
+ map_memory();
+ init_kernel_rwx();
+}
+
+static int __init parse_rodata(char *arg)
+{
+ strtobool(arg, &rodata_enabled);
+
+ return 0;
+}
+early_param("rodata", parse_rodata);
+
+#else
+
+static void remap_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end,
+ pgprot_t prot)
+{
+}
+
+#endif
+
+void __init setup_arch_memory(void)
+{
+ setup_bootmem();
+ sparse_init();
+ zone_sizes_init();
+}
+
+void __init mem_init(void)
+{
+ memblock_free_all();
+
+ /* allocate the zero page */
+ empty_zero_page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+ if (!empty_zero_page)
+ panic("Failed to allocate the empty_zero_page");
+}
+
+void free_initmem(void)
+{
+ /* Remap init text as RW to allow writing to it */
+ remap_kernel_segment(swapper_pg_dir, __inittext_start, __inittext_end,
+ PAGE_KERNEL);
+
+#ifdef CONFIG_POISON_INITMEM
+ free_initmem_default(0x0);
+#else
+ free_initmem_default(-1);
+#endif
+}
+
+void __set_fixmap(enum fixed_addresses idx,
+ phys_addr_t phys, pgprot_t flags)
+{
+ unsigned long addr = __fix_to_virt(idx);
+ pte_t *pte;
+
+
+ BUG_ON(idx >= __end_of_fixed_addresses);
+
+ pte = &fixmap_pte[pte_index(addr)];
+
+ if (pgprot_val(flags)) {
+ set_pte(pte, pfn_pte(phys_to_pfn(phys), flags));
+ } else {
+ /* Remove the fixmap */
+ pte_clear(&init_mm, addr, pte);
+ }
+ local_flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
+}
+
+static const pgprot_t protection_map[16] = {
+ [VM_NONE] = PAGE_NONE,
+ [VM_READ] = PAGE_READ,
+ [VM_WRITE] = PAGE_READ,
+ [VM_WRITE | VM_READ] = PAGE_READ,
+ [VM_EXEC] = PAGE_READ_EXEC,
+ [VM_EXEC | VM_READ] = PAGE_READ_EXEC,
+ [VM_EXEC | VM_WRITE] = PAGE_READ_EXEC,
+ [VM_EXEC | VM_WRITE | VM_READ] = PAGE_READ_EXEC,
+ [VM_SHARED] = PAGE_NONE,
+ [VM_SHARED | VM_READ] = PAGE_READ,
+ [VM_SHARED | VM_WRITE] = PAGE_READ_WRITE,
+ [VM_SHARED | VM_WRITE | VM_READ] = PAGE_READ_WRITE,
+ [VM_SHARED | VM_EXEC] = PAGE_READ_EXEC,
+ [VM_SHARED | VM_EXEC | VM_READ] = PAGE_READ_EXEC,
+ [VM_SHARED | VM_EXEC | VM_WRITE] = PAGE_READ_WRITE_EXEC,
+ [VM_SHARED | VM_EXEC | VM_WRITE | VM_READ] = PAGE_READ_WRITE_EXEC
+};
+DECLARE_VM_GET_PAGE_PROT
diff --git a/arch/kvx/mm/kernel_rwx.c b/arch/kvx/mm/kernel_rwx.c
new file mode 100644
index 000000000000..efa502bab336
--- /dev/null
+++ b/arch/kvx/mm/kernel_rwx.c
@@ -0,0 +1,228 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julian Vetter
+ */
+
+#include <linux/init.h>
+#include <linux/sysfs.h>
+#include <linux/kernel.h>
+#include <linux/device.h>
+#include <linux/kobject.h>
+
+#include <asm/mmu.h>
+#include <asm/tlb.h>
+#include <asm/insns.h>
+#include <asm/sections.h>
+#include <asm/symbols.h>
+#include <asm/insns_defs.h>
+
+#define PERF_REFILL_INSN_SIZE (KVX_INSN_GOTO_SIZE * KVX_INSN_SYLLABLE_WIDTH)
+
+struct kobject *kvx_kobj;
+
+static bool kernel_rwx = true;
+static u32 perf_refill_insn;
+
+static DEFINE_MUTEX(kernel_rwx_mutex);
+
+static int __init parse_kernel_rwx(char *arg)
+{
+ strtobool(arg, &kernel_rwx);
+
+ return 0;
+}
+early_param("kvx.kernel_rwx", parse_kernel_rwx);
+
+static void map_exception_only_in_ltlb(void)
+{
+ struct kvx_tlb_format tlbe;
+
+ tlbe = tlb_mk_entry(
+ (void *)__pa(__exception_start),
+ (void *)__exception_start,
+ TLB_PS_4K,
+ TLB_G_GLOBAL,
+ TLB_PA_NA_RX,
+ TLB_CP_W_C,
+ 0,
+ TLB_ES_A_MODIFIED);
+
+ BUG_ON((__exception_end - __exception_start) > PAGE_SIZE);
+
+ kvx_mmu_add_entry(MMC_SB_LTLB, LTLB_ENTRY_KERNEL_TEXT, tlbe);
+}
+
+static void map_kernel_in_ltlb(void)
+{
+ struct kvx_tlb_format tlbe;
+
+ tlbe = tlb_mk_entry(
+ (void *) PHYS_OFFSET,
+ (void *) PAGE_OFFSET,
+ TLB_PS_512M,
+ TLB_G_GLOBAL,
+ TLB_PA_NA_RWX,
+ TLB_CP_W_C,
+ 0,
+ TLB_ES_A_MODIFIED);
+
+ kvx_mmu_add_entry(MMC_SB_LTLB, LTLB_ENTRY_KERNEL_TEXT, tlbe);
+}
+
+static void mmu_disable_kernel_perf_refill(void)
+{
+ unsigned int off = kvx_std_tlb_refill - kvx_perf_tlb_refill;
+ u32 goto_insn;
+ int ret;
+
+ BUG_ON(KVX_INSN_GOTO_PCREL27_CHECK(off));
+
+ KVX_INSN_GOTO(&goto_insn, KVX_INSN_PARALLEL_EOB, off);
+
+ ret = kvx_insns_read(&perf_refill_insn, PERF_REFILL_INSN_SIZE,
+ (u32 *) kvx_perf_tlb_refill);
+ BUG_ON(ret);
+ ret = kvx_insns_write(&goto_insn, PERF_REFILL_INSN_SIZE,
+ (u32 *) kvx_perf_tlb_refill);
+ BUG_ON(ret);
+}
+
+static void enable_kernel_perf_refill(void)
+{
+ int ret;
+
+ ret = kvx_insns_write(&perf_refill_insn, PERF_REFILL_INSN_SIZE,
+ (u32 *) kvx_perf_tlb_refill);
+ BUG_ON(ret);
+}
+
+static void local_mmu_enable_kernel_rwx(void)
+{
+ int i;
+ struct kvx_tlb_format tlbe;
+
+ tlbe = tlb_mk_entry(0, (void *) 0, 0, 0, 0, 0, 0,
+ TLB_ES_INVALID);
+
+ /* Map exceptions handlers in LTLB entry instead of full kernel */
+ map_exception_only_in_ltlb();
+
+ /* Invalidate previously added LTLB entries */
+ for (i = 0; i < REFILL_PERF_ENTRIES; i++)
+ kvx_mmu_add_entry(MMC_SB_LTLB, LTLB_KERNEL_RESERVED + i, tlbe);
+
+}
+
+/**
+ * init_kernel_rwx - Initialize kernel struct RWX at boot time
+ * This function MUST be used only at boot time to setup kernel strict RWX mode
+ * Once done, kernel rwx mode can be enabled/disabled using sysfs entry.
+ */
+void init_kernel_rwx(void)
+{
+ /* Kernel strict RWX mode disabled */
+ if (!kernel_rwx)
+ return;
+
+ /* First processor only will disable perf refill by patching code */
+ if (raw_smp_processor_id() == 0)
+ mmu_disable_kernel_perf_refill();
+
+ local_mmu_enable_kernel_rwx();
+}
+
+static void ipi_enable_kernel_rwx(void *arg)
+{
+ local_mmu_enable_kernel_rwx();
+}
+
+static void local_mmu_disable_kernel_rwx(void)
+{
+ /* Map full kernel in LTLB entry instead of only exceptions */
+ map_kernel_in_ltlb();
+
+ /* Flush JTLB completely to force refill and avoid stalled entries */
+ local_flush_tlb_all();
+}
+
+static void ipi_disable_kernel_rwx(void *arg)
+{
+ local_mmu_disable_kernel_rwx();
+}
+
+static void smp_set_kernel_rwx(bool kernel_rwx)
+{
+ smp_call_func_t fn;
+
+ pr_info("%sabling kernel rwx mode\n", kernel_rwx ? "En" : "Dis");
+
+ if (kernel_rwx) {
+ mmu_disable_kernel_perf_refill();
+ fn = ipi_enable_kernel_rwx;
+ } else {
+ enable_kernel_perf_refill();
+ fn = ipi_disable_kernel_rwx;
+ }
+
+ on_each_cpu(fn, NULL, 1);
+}
+
+static ssize_t kernel_rwx_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return sprintf(buf, "%s\n", kernel_rwx ? "enabled" : "disabled");
+}
+
+static ssize_t kernel_rwx_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t len)
+{
+ bool value;
+ int ret;
+
+ ret = strtobool(buf, &value);
+ if (ret)
+ return ret;
+
+ mutex_lock(&kernel_rwx_mutex);
+ /* Switch only if necessary */
+ if (value != kernel_rwx) {
+ kernel_rwx = value;
+ smp_set_kernel_rwx(kernel_rwx);
+ }
+ mutex_unlock(&kernel_rwx_mutex);
+
+ return len;
+}
+
+static struct kobj_attribute kernel_rwx_attr =
+ __ATTR(kernel_rwx, 0644, kernel_rwx_show, kernel_rwx_store);
+
+static struct attribute *default_attrs[] = {
+ &kernel_rwx_attr.attr,
+ NULL
+};
+
+static struct attribute_group kernel_rwx_attr_group = {
+ .attrs = default_attrs,
+};
+
+static int __init kvx_kernel_rwx_init(void)
+{
+ int ret;
+
+ kvx_kobj = kobject_create_and_add("kvx", NULL);
+ if (!kvx_kobj)
+ return -ENOMEM;
+
+ ret = sysfs_create_group(kvx_kobj, &kernel_rwx_attr_group);
+ if (ret) {
+ kobject_put(kvx_kobj);
+ return ret;
+ }
+
+ return 0;
+}
+postcore_initcall(kvx_kernel_rwx_init);
diff --git a/arch/kvx/mm/mmap.c b/arch/kvx/mm/mmap.c
new file mode 100644
index 000000000000..a2225db64438
--- /dev/null
+++ b/arch/kvx/mm/mmap.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/arm64/mm/mmap.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifdef CONFIG_STRICT_DEVMEM
+
+#include <linux/mm.h>
+#include <linux/ioport.h>
+
+#include <asm/page.h>
+
+/*
+ * devmem_is_allowed() checks to see if /dev/mem access to a certain address
+ * is valid. The argument is a physical page number. We mimic x86 here by
+ * disallowing access to system RAM as well as device-exclusive MMIO regions.
+ * This effectively disable read()/write() on /dev/mem.
+ */
+int devmem_is_allowed(unsigned long pfn)
+{
+ if (iomem_is_exclusive(pfn << PAGE_SHIFT))
+ return 0;
+ if (!page_is_ram(pfn))
+ return 1;
+ return 0;
+}
+
+#endif
diff --git a/arch/kvx/mm/mmu.c b/arch/kvx/mm/mmu.c
new file mode 100644
index 000000000000..a1eadececb6a
--- /dev/null
+++ b/arch/kvx/mm/mmu.c
@@ -0,0 +1,204 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Vincent Chardon
+ * Jules Maselbas
+ */
+
+#include <linux/cache.h>
+#include <linux/types.h>
+#include <linux/irqflags.h>
+#include <linux/printk.h>
+#include <linux/percpu.h>
+#include <linux/kernel.h>
+#include <linux/spinlock.h>
+#include <linux/spinlock_types.h>
+
+#include <asm/mmu.h>
+#include <asm/tlb.h>
+#include <asm/page_size.h>
+#include <asm/mmu_context.h>
+
+#define DUMP_LTLB 0
+#define DUMP_JTLB 1
+
+DEFINE_PER_CPU_ALIGNED(uint8_t[MMU_JTLB_SETS], jtlb_current_set_way);
+static struct kvx_tlb_format ltlb_entries[MMU_LTLB_WAYS];
+static unsigned long ltlb_entries_bmp;
+
+static int kvx_mmu_ltlb_overlaps(struct kvx_tlb_format tlbe)
+{
+ int bit = LTLB_ENTRY_FIXED_COUNT;
+
+ for_each_set_bit_from(bit, <lb_entries_bmp, MMU_LTLB_WAYS) {
+ if (tlb_entry_overlaps(tlbe, ltlb_entries[bit]))
+ return 1;
+ }
+
+ return 0;
+}
+
+/**
+ * kvx_mmu_ltlb_add_entry - Add a kernel entry in the LTLB
+ *
+ * In order to lock some entries in the LTLB and be always mapped, this
+ * function can be called with a physical address, a virtual address and
+ * protection attribute to add an entry into the LTLB. This is mainly for
+ * performances since there won't be any NOMAPPING traps for these pages.
+ *
+ * @vaddr: Virtual address for the entry (must be aligned according to tlb_ps)
+ * @paddr: Physical address for the entry (must be aligned according to tlb_ps)
+ * @flags: Protection attributes
+ * @tlb_ps: Page size attribute for TLB (TLB_PS_*)
+ */
+void kvx_mmu_ltlb_add_entry(unsigned long vaddr, phys_addr_t paddr,
+ pgprot_t flags, unsigned long tlb_ps)
+{
+ unsigned int cp;
+ unsigned int idx;
+ unsigned long irqflags;
+ struct kvx_tlb_format tlbe;
+ u64 page_size = BIT(get_page_size_shift(tlb_ps));
+
+ BUG_ON(!IS_ALIGNED(vaddr, page_size) || !IS_ALIGNED(paddr, page_size));
+
+ cp = pgprot_cache_policy(pgprot_val(flags));
+
+ tlbe = tlb_mk_entry(
+ (void *) paddr,
+ (void *) vaddr,
+ tlb_ps,
+ TLB_G_GLOBAL,
+ TLB_PA_NA_RW,
+ cp,
+ 0,
+ TLB_ES_A_MODIFIED);
+
+ local_irq_save(irqflags);
+
+ if (IS_ENABLED(CONFIG_KVX_DEBUG_TLB_WRITE) &&
+ kvx_mmu_ltlb_overlaps(tlbe))
+ panic("VA %lx overlaps with an existing LTLB mapping", vaddr);
+
+ idx = find_next_zero_bit(<lb_entries_bmp, MMU_LTLB_WAYS,
+ LTLB_ENTRY_FIXED_COUNT);
+ /* This should never happen */
+ BUG_ON(idx >= MMU_LTLB_WAYS);
+ __set_bit(idx, <lb_entries_bmp);
+ ltlb_entries[idx] = tlbe;
+ kvx_mmu_add_entry(MMC_SB_LTLB, idx, tlbe);
+
+ if (kvx_mmc_error(kvx_sfr_get(MMC)))
+ panic("Failed to write entry to the LTLB");
+
+ local_irq_restore(irqflags);
+}
+
+void kvx_mmu_ltlb_remove_entry(unsigned long vaddr)
+{
+ int ret, bit = LTLB_ENTRY_FIXED_COUNT;
+ struct kvx_tlb_format tlbe;
+
+ for_each_set_bit_from(bit, <lb_entries_bmp, MMU_LTLB_WAYS) {
+ tlbe = ltlb_entries[bit];
+ if (tlb_entry_match_addr(tlbe, vaddr)) {
+ __clear_bit(bit, <lb_entries_bmp);
+ break;
+ }
+ }
+
+ if (bit == MMU_LTLB_WAYS)
+ panic("Trying to remove non-existent LTLB entry for addr %lx\n",
+ vaddr);
+
+ ret = clear_ltlb_entry(vaddr);
+ if (ret)
+ panic("Failed to remove LTLB entry for addr %lx\n", vaddr);
+}
+
+/**
+ * kvx_mmu_jtlb_add_entry - Add an entry into JTLB
+ *
+ * JTLB is used both for kernel and user entries.
+ *
+ * @address: Virtual address for the entry (must be aligned according to tlb_ps)
+ * @ptep: pte entry pointer
+ * @asn: ASN (if pte entry is not global)
+ */
+void kvx_mmu_jtlb_add_entry(unsigned long address, pte_t *ptep,
+ unsigned int asn)
+{
+ unsigned int shifted_addr, set, way;
+ unsigned long flags, pte_val;
+ struct kvx_tlb_format tlbe;
+ unsigned int pa, cp, ps;
+ phys_addr_t pfn;
+
+ pte_val = pte_val(*ptep);
+
+ pfn = (phys_addr_t)pte_pfn(*ptep);
+
+ asn &= MM_CTXT_ASN_MASK;
+
+ /* Set page as accessed */
+ pte_val(*ptep) |= _PAGE_ACCESSED;
+
+ BUILD_BUG_ON(KVX_PAGE_SZ_SHIFT != KVX_SFR_TEL_PS_SHIFT);
+
+ ps = (pte_val & KVX_PAGE_SZ_MASK) >> KVX_PAGE_SZ_SHIFT;
+ pa = get_page_access_perms(KVX_ACCESS_PERMS_INDEX(pte_val));
+ cp = pgprot_cache_policy(pte_val);
+
+ tlbe = tlb_mk_entry(
+ (void *)pfn_to_phys(pfn),
+ (void *)address,
+ ps,
+ (pte_val & _PAGE_GLOBAL) ? TLB_G_GLOBAL : TLB_G_USE_ASN,
+ pa,
+ cp,
+ asn,
+ TLB_ES_A_MODIFIED);
+
+ shifted_addr = address >> get_page_size_shift(ps);
+ set = shifted_addr & MMU_JTLB_SET_MASK;
+
+ local_irq_save(flags);
+
+ if (IS_ENABLED(CONFIG_KVX_DEBUG_TLB_WRITE) &&
+ kvx_mmu_ltlb_overlaps(tlbe))
+ panic("VA %lx overlaps with an existing LTLB mapping", address);
+
+ way = get_cpu_var(jtlb_current_set_way)[set]++;
+ put_cpu_var(jtlb_current_set_way);
+
+ way &= MMU_JTLB_WAY_MASK;
+
+ kvx_mmu_add_entry(MMC_SB_JTLB, way, tlbe);
+
+ if (IS_ENABLED(CONFIG_KVX_DEBUG_TLB_WRITE) &&
+ kvx_mmc_error(kvx_sfr_get(MMC)))
+ panic("Failed to write entry to the JTLB (in update_mmu_cache)");
+
+ local_irq_restore(flags);
+}
+
+void __init kvx_mmu_early_setup(void)
+{
+ int bit;
+ struct kvx_tlb_format tlbe;
+
+ kvx_mmu_remove_ltlb_entry(LTLB_ENTRY_EARLY_SMEM);
+
+ if (raw_smp_processor_id() != 0) {
+ /* Apply existing ltlb entries starting from first one free */
+ bit = LTLB_ENTRY_FIXED_COUNT;
+ for_each_set_bit_from(bit, <lb_entries_bmp, MMU_LTLB_WAYS) {
+ tlbe = ltlb_entries[bit];
+ kvx_mmu_add_entry(MMC_SB_LTLB, bit, tlbe);
+ }
+
+ init_kernel_rwx();
+ }
+}
diff --git a/arch/kvx/mm/mmu_stats.c b/arch/kvx/mm/mmu_stats.c
new file mode 100644
index 000000000000..e70b7e2dcab1
--- /dev/null
+++ b/arch/kvx/mm/mmu_stats.c
@@ -0,0 +1,94 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/seq_file.h>
+#include <linux/debugfs.h>
+
+#include <asm/mmu_stats.h>
+
+static struct dentry *mmu_stats_debufs;
+
+static const char *mmu_refill_types_name[MMU_REFILL_TYPE_COUNT] = {
+ [MMU_REFILL_TYPE_USER] = "User",
+ [MMU_REFILL_TYPE_KERNEL] = "Kernel",
+ [MMU_REFILL_TYPE_KERNEL_DIRECT] = "Kernel Direct"
+};
+
+DEFINE_PER_CPU(struct mmu_stats, mmu_stats);
+
+static int mmu_stats_show(struct seq_file *m, void *v)
+{
+ int cpu, type;
+ unsigned long avg = 0, total_refill, efficiency, total_cycles;
+ struct mmu_stats *stats;
+ struct mmu_refill_stats *ref_stat;
+
+ total_cycles = get_cycles();
+ for_each_present_cpu(cpu) {
+ stats = &per_cpu(mmu_stats, cpu);
+ total_refill = 0;
+
+ seq_printf(m, " - CPU %d\n", cpu);
+ for (type = 0; type < MMU_REFILL_TYPE_COUNT; type++) {
+ ref_stat = &stats->refill[type];
+ total_refill += ref_stat->count;
+ if (ref_stat->count)
+ avg = ref_stat->total / ref_stat->count;
+ else
+ avg = 0;
+
+ seq_printf(m,
+ " - %s refill stats:\n"
+ " - count: %lu\n"
+ " - min: %lu\n"
+ " - avg: %lu\n"
+ " - max: %lu\n",
+ mmu_refill_types_name[type],
+ ref_stat->count,
+ ref_stat->min,
+ avg,
+ ref_stat->max
+ );
+ }
+
+ if (total_refill)
+ avg = stats->cycles_between_refill / total_refill;
+ else
+ avg = 0;
+
+ seq_printf(m, " - Average cycles between refill: %lu\n", avg);
+ seq_printf(m, " - tlb_flush_all calls: %lu\n",
+ stats->tlb_flush_all);
+ efficiency = stats->cycles_between_refill * 100 /
+ stats->last_refill;
+ seq_printf(m, " - Efficiency: %lu%%\n", efficiency);
+ }
+
+ return 0;
+}
+
+static int mmu_stats_open(struct inode *inode, struct file *filp)
+{
+ return single_open(filp, mmu_stats_show, NULL);
+}
+
+static const struct file_operations mmu_stats_fops = {
+ .open = mmu_stats_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static int __init mmu_stats_debufs_init(void)
+{
+ mmu_stats_debufs = debugfs_create_dir("kvx_mmu_debug", NULL);
+
+ debugfs_create_file("mmu_stats", 0444, mmu_stats_debufs, NULL,
+ &mmu_stats_fops);
+
+ return 0;
+}
+subsys_initcall(mmu_stats_debufs_init);
diff --git a/arch/kvx/mm/tlb.c b/arch/kvx/mm/tlb.c
new file mode 100644
index 000000000000..9542332fe4d6
--- /dev/null
+++ b/arch/kvx/mm/tlb.c
@@ -0,0 +1,433 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#include <linux/mmu_context.h>
+#include <linux/sched.h>
+
+#include <asm/tlbflush.h>
+#include <asm/tlb_defs.h>
+#include <asm/page_size.h>
+#include <asm/mmu_stats.h>
+#include <asm/pgtable.h>
+#include <asm/tlb.h>
+
+/*
+ * When in kernel, use dummy ASN 42 to be able to catch any problem easily if
+ * ASN is not restored properly.
+ */
+#define KERNEL_DUMMY_ASN 42
+
+/* Threshold of page count above which we will regenerate a new ASN */
+#define ASN_FLUSH_PAGE_THRESHOLD (MMU_JTLB_ENTRIES)
+
+/* Threshold of page count above which we will flush the whole JTLB */
+#define FLUSH_ALL_PAGE_THRESHOLD (MMU_JTLB_ENTRIES)
+
+DEFINE_PER_CPU(unsigned long, kvx_asn_cache) = MM_CTXT_FIRST_CYCLE;
+
+#ifdef CONFIG_KVX_DEBUG_TLB_ACCESS
+
+static DEFINE_PER_CPU_ALIGNED(struct kvx_tlb_access_t[KVX_TLB_ACCESS_SIZE],
+ kvx_tlb_access_rb);
+/* Lower bits hold the index and upper ones hold the number of wrapped */
+static DEFINE_PER_CPU(unsigned int, kvx_tlb_access_idx);
+
+void kvx_update_tlb_access(int type)
+{
+ unsigned int *idx_ptr = &get_cpu_var(kvx_tlb_access_idx);
+ unsigned int idx;
+ struct kvx_tlb_access_t *tab = get_cpu_var(kvx_tlb_access_rb);
+
+ idx = KVX_TLB_ACCESS_GET_IDX(*idx_ptr);
+
+ kvx_mmu_get_tlb_entry(tab[idx].entry);
+ tab[idx].mmc_val = kvx_sfr_get(MMC);
+ tab[idx].type = type;
+
+ (*idx_ptr)++;
+ put_cpu_var(kvx_tlb_access_rb);
+ put_cpu_var(kvx_tlb_access_idx);
+};
+
+#endif
+
+/**
+ * clear_tlb_entry() - clear an entry in TLB if it exists
+ * @addr: the address used to set TEH.PN
+ * @global: is page global or not
+ * @asn: ASN used if page is not global
+ * @tlb_type: tlb type (MMC_SB_LTLB or MMC_SB_JTLB)
+ *
+ * Preemption must be disabled when calling this function. There is no need to
+ * invalidate micro TLB because it is invalidated when we write TLB.
+ *
+ * Return: 0 if TLB entry was found and deleted properly, -ENOENT if not found
+ * -EINVAL if found but in incorrect TLB.
+ *
+ */
+static int clear_tlb_entry(unsigned long addr,
+ unsigned int global,
+ unsigned int asn,
+ unsigned int tlb_type)
+{
+ struct kvx_tlb_format entry;
+ unsigned long mmc_val;
+ int saved_asn, ret = 0;
+
+ /* Sanitize ASN */
+ asn &= MM_CTXT_ASN_MASK;
+
+ /* Before probing we need to save the current ASN */
+ mmc_val = kvx_sfr_get(MMC);
+ saved_asn = kvx_sfr_field_val(mmc_val, MMC, ASN);
+ kvx_sfr_set_field(MMC, ASN, asn);
+
+ /* Probe is based on PN and ASN. So ES can be anything */
+ entry = tlb_mk_entry(0, (void *)addr, 0, global, 0, 0, 0,
+ TLB_ES_INVALID);
+ kvx_mmu_set_tlb_entry(entry);
+
+ kvx_mmu_probetlb();
+
+ mmc_val = kvx_sfr_get(MMC);
+
+ if (kvx_mmc_error(mmc_val)) {
+ if (kvx_mmc_parity(mmc_val)) {
+ /*
+ * This should never happens unless you are bombared by
+ * streams of charged particules. If it happens just
+ * flush the JTLB and let's continue (but check your
+ * environment you are probably not in a safe place).
+ */
+ WARN(1, "%s: parity error during lookup (addr 0x%lx, asn %u). JTLB will be flushed\n",
+ __func__, addr, asn);
+ kvx_sfr_set_field(MMC, PAR, 0);
+ local_flush_tlb_all();
+ }
+
+ /*
+ * else there is no matching entry so just clean the error and
+ * restore the ASN before returning.
+ */
+ kvx_sfr_set_field(MMC, E, 0);
+ ret = -ENOENT;
+ goto restore_asn;
+ }
+
+ /* We surely don't want to flush another TLB type or we are fried */
+ if (kvx_mmc_sb(mmc_val) != tlb_type) {
+ ret = -EINVAL;
+ goto restore_asn;
+ }
+
+ /*
+ * At this point the probe found an entry. TEL and TEH have correct
+ * values so we just need to set the entry status to invalid to clear
+ * the entry.
+ */
+ kvx_sfr_set_field(TEL, ES, TLB_ES_INVALID);
+
+ kvx_mmu_writetlb();
+
+ /* Need to read MMC SFR again */
+ mmc_val = kvx_sfr_get(MMC);
+ if (kvx_mmc_error(mmc_val))
+ panic("%s: Failed to clear entry (addr 0x%lx, asn %u)",
+ __func__, addr, asn);
+ else
+ pr_debug("%s: Entry (addr 0x%lx, asn %u) cleared\n",
+ __func__, addr, asn);
+
+restore_asn:
+ kvx_sfr_set_field(MMC, ASN, saved_asn);
+
+ return ret;
+}
+
+static void clear_jtlb_entry(unsigned long addr,
+ unsigned int global,
+ unsigned int asn)
+{
+ clear_tlb_entry(addr, global, asn, MMC_SB_JTLB);
+}
+
+/**
+ * clear_ltlb_entry() - Remove a LTLB entry
+ * @vaddr: Virtual address to be matched against LTLB entries
+ *
+ * Return: Same value as clear_tlb_entry
+ */
+int clear_ltlb_entry(unsigned long vaddr)
+{
+ return clear_tlb_entry(vaddr, TLB_G_GLOBAL, KERNEL_DUMMY_ASN,
+ MMC_SB_LTLB);
+}
+
+/* If mm is current we just need to assign the current task a new ASN. By
+ * doing this all previous TLB entries with the former ASN will be invalidated.
+ * if mm is not the current one we invalidate the context and when this other
+ * mm will be swapped in then a new context will be generated.
+ */
+void local_flush_tlb_mm(struct mm_struct *mm)
+{
+ int cpu = smp_processor_id();
+
+ destroy_context(mm);
+ if (current->active_mm == mm)
+ activate_context(mm, cpu);
+}
+
+void local_flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long addr)
+{
+ int cpu = smp_processor_id();
+ unsigned int current_asn;
+ struct mm_struct *mm;
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ mm = vma->vm_mm;
+ current_asn = mm_asn(mm, cpu);
+
+ /* If mm has no context there is nothing to do */
+ if (current_asn != MM_CTXT_NO_ASN)
+ clear_jtlb_entry(addr, TLB_G_USE_ASN, current_asn);
+
+ local_irq_restore(flags);
+}
+
+void local_flush_tlb_all(void)
+{
+ struct kvx_tlb_format tlbe = KVX_EMPTY_TLB_ENTRY;
+ int set, way;
+ unsigned long flags;
+#ifdef CONFIG_KVX_MMU_STATS
+ struct mmu_stats *stats = &per_cpu(mmu_stats, smp_processor_id());
+
+ stats->tlb_flush_all++;
+#endif
+
+ local_irq_save(flags);
+
+ /* Select JTLB and prepare TEL (constant) */
+ kvx_sfr_set(TEL, (uint64_t) tlbe.tel_val);
+ kvx_sfr_set_field(MMC, SB, MMC_SB_JTLB);
+
+ for (set = 0; set < MMU_JTLB_SETS; set++) {
+ tlbe.teh.pn = set;
+ for (way = 0; way < MMU_JTLB_WAYS; way++) {
+ /* Set is selected automatically according to the
+ * virtual address.
+ * With 4K pages the set is the value of the 6 lower
+ * significant bits of the page number.
+ */
+ kvx_sfr_set(TEH, (uint64_t) tlbe.teh_val);
+ kvx_sfr_set_field(MMC, SW, way);
+ kvx_mmu_writetlb();
+
+ if (kvx_mmc_error(kvx_sfr_get(MMC)))
+ panic("Failed to initialize JTLB[s:%02d w:%d]",
+ set, way);
+ }
+ }
+
+ local_irq_restore(flags);
+}
+
+void local_flush_tlb_range(struct vm_area_struct *vma,
+ unsigned long start,
+ unsigned long end)
+{
+ const unsigned int cpu = smp_processor_id();
+ unsigned long flags;
+ unsigned int current_asn;
+ unsigned long pages = (end - start) >> PAGE_SHIFT;
+
+ if (pages > ASN_FLUSH_PAGE_THRESHOLD) {
+ local_flush_tlb_mm(vma->vm_mm);
+ return;
+ }
+
+ start &= PAGE_MASK;
+
+ local_irq_save(flags);
+
+ current_asn = mm_asn(vma->vm_mm, cpu);
+ if (current_asn != MM_CTXT_NO_ASN) {
+ while (start < end) {
+ clear_jtlb_entry(start, TLB_G_USE_ASN, current_asn);
+ start += PAGE_SIZE;
+ }
+ }
+
+ local_irq_restore(flags);
+}
+
+/**
+ * local_flush_tlb_kernel_range() - flush kernel TLB entries
+ * @start: start kernel virtual address
+ * @end: end kernel virtual address
+ *
+ * Return: nothing
+ */
+void local_flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ unsigned long flags;
+ unsigned long pages = (end - start) >> PAGE_SHIFT;
+
+ if (pages > FLUSH_ALL_PAGE_THRESHOLD) {
+ local_flush_tlb_all();
+ return;
+ }
+
+ start &= PAGE_MASK;
+
+ local_irq_save(flags);
+
+ while (start < end) {
+ clear_jtlb_entry(start, TLB_G_GLOBAL, KERNEL_DUMMY_ASN);
+ start += PAGE_SIZE;
+ }
+
+ local_irq_restore(flags);
+}
+
+void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
+ pmd_t *pmd)
+{
+ pte_t pte = __pte(pmd_val(*pmd));
+
+ /* THP PMD accessors are implemented in terms of PTE */
+ update_mmu_cache(vma, addr, &pte);
+}
+
+void update_mmu_cache(struct vm_area_struct *vma,
+ unsigned long address, pte_t *ptep)
+{
+ struct mm_struct *mm;
+ unsigned long asn;
+ int cpu = smp_processor_id();
+
+ if (unlikely(ptep == NULL))
+ panic("pte should not be NULL\n");
+
+ /* Flush any previous TLB entries */
+ flush_tlb_page(vma, address);
+
+ /* No need to add the TLB entry until the process that owns the memory
+ * is running.
+ */
+ mm = current->active_mm;
+ if (vma && (mm != vma->vm_mm))
+ return;
+
+ /*
+ * Get the ASN:
+ * ASN can have no context if address belongs to kernel space. In
+ * fact as pages are global for kernel space the ASN is ignored
+ * and can be equal to any value.
+ */
+ asn = mm_asn(mm, cpu);
+
+#ifdef CONFIG_KVX_DEBUG_ASN
+ /*
+ * For addresses that belong to user space, the value of the ASN
+ * must match the mmc.asn and be non zero
+ */
+ if (address < PAGE_OFFSET) {
+ unsigned int mmc_asn = kvx_mmc_asn(kvx_sfr_get(MMC));
+
+ if (asn == MM_CTXT_NO_ASN)
+ panic("%s: ASN [%lu] is not properly set for address 0x%lx on CPU %d\n",
+ __func__, asn, address, cpu);
+
+ if ((asn & MM_CTXT_ASN_MASK) != mmc_asn)
+ panic("%s: ASN not synchronized with MMC: asn:%lu != mmc.asn:%u\n",
+ __func__, (asn & MM_CTXT_ASN_MASK), mmc_asn);
+ }
+#endif
+
+ kvx_mmu_jtlb_add_entry(address, ptep, asn);
+}
+
+#ifdef CONFIG_SMP
+
+struct tlb_args {
+ struct vm_area_struct *ta_vma;
+ unsigned long ta_start;
+ unsigned long ta_end;
+};
+
+static inline void ipi_flush_tlb_page(void *arg)
+{
+ struct tlb_args *ta = arg;
+
+ local_flush_tlb_page(ta->ta_vma, ta->ta_start);
+}
+
+void
+smp_flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
+{
+ struct tlb_args ta = {
+ .ta_vma = vma,
+ .ta_start = addr
+ };
+
+ on_each_cpu_mask(mm_cpumask(vma->vm_mm), ipi_flush_tlb_page, &ta, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_page);
+
+void
+smp_flush_tlb_mm(struct mm_struct *mm)
+{
+ on_each_cpu_mask(mm_cpumask(mm), (smp_call_func_t)local_flush_tlb_mm,
+ mm, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_mm);
+
+static inline void ipi_flush_tlb_range(void *arg)
+{
+ struct tlb_args *ta = arg;
+
+ local_flush_tlb_range(ta->ta_vma, ta->ta_start, ta->ta_end);
+}
+
+void
+smp_flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end)
+{
+ struct tlb_args ta = {
+ .ta_vma = vma,
+ .ta_start = start,
+ .ta_end = end
+ };
+
+ on_each_cpu_mask(mm_cpumask(vma->vm_mm), ipi_flush_tlb_range, &ta, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_range);
+
+static inline void ipi_flush_tlb_kernel_range(void *arg)
+{
+ struct tlb_args *ta = arg;
+
+ local_flush_tlb_kernel_range(ta->ta_start, ta->ta_end);
+}
+
+void
+smp_flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ struct tlb_args ta = {
+ .ta_start = start,
+ .ta_end = end
+ };
+
+ on_each_cpu(ipi_flush_tlb_kernel_range, &ta, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_kernel_range);
+
+#endif
--
2.37.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 11/25] kvx: Add memory management
@ 2023-01-03 16:43 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Will Deacon, Aneesh Kumar K.V, Andrew Morton,
Nick Piggin, Peter Zijlstra, Paul Walmsley, Palmer Dabbelt,
Albert Ou, linux-kernel, linux-arch, linux-mm, linux-riscv,
Clement Leger, Guillaume Thouvenin, Jean-Christophe Pince,
Jules Maselbas, Julian Vetter, Julien Hascoet, Louis Morhet,
Marc Poulhiès, Marius Gligor, Vincent Chardon
Add memory management support for kvx, including: cache and tlb
management, page fault handling, ioremap/mmap and streaming dma support.
CC: Will Deacon <will@kernel.org>
CC: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: Nick Piggin <npiggin@gmail.com>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Paul Walmsley <paul.walmsley@sifive.com>
CC: Palmer Dabbelt <palmer@dabbelt.com>
CC: Albert Ou <aou@eecs.berkeley.edu>
CC: linux-kernel@vger.kernel.org
CC: linux-arch@vger.kernel.org
CC: linux-mm@kvack.org
CC: linux-riscv@lists.infradead.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Jean-Christophe Pince <jcpince@gmail.com>
Signed-off-by: Jean-Christophe Pince <jcpince@gmail.com>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Julien Hascoet <jhascoet@kalray.eu>
Signed-off-by: Julien Hascoet <jhascoet@kalray.eu>
Co-developed-by: Louis Morhet <lmorhet@kalray.eu>
Signed-off-by: Louis Morhet <lmorhet@kalray.eu>
Co-developed-by: Marc Poulhiès <dkm@kataplop.net>
Signed-off-by: Marc Poulhiès <dkm@kataplop.net>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/cache.h | 46 +++
arch/kvx/include/asm/cacheflush.h | 181 +++++++++
arch/kvx/include/asm/fixmap.h | 47 +++
arch/kvx/include/asm/hugetlb.h | 36 ++
arch/kvx/include/asm/l2_cache.h | 75 ++++
arch/kvx/include/asm/l2_cache_defs.h | 64 ++++
arch/kvx/include/asm/mem_map.h | 44 +++
arch/kvx/include/asm/mmu.h | 296 +++++++++++++++
arch/kvx/include/asm/mmu_context.h | 156 ++++++++
arch/kvx/include/asm/mmu_stats.h | 38 ++
arch/kvx/include/asm/page.h | 187 ++++++++++
arch/kvx/include/asm/page_size.h | 29 ++
arch/kvx/include/asm/pgalloc.h | 101 +++++
arch/kvx/include/asm/pgtable-bits.h | 102 ++++++
arch/kvx/include/asm/pgtable.h | 451 +++++++++++++++++++++++
arch/kvx/include/asm/rm_fw.h | 16 +
arch/kvx/include/asm/sparsemem.h | 15 +
arch/kvx/include/asm/symbols.h | 16 +
arch/kvx/include/asm/tlb.h | 24 ++
arch/kvx/include/asm/tlb_defs.h | 131 +++++++
arch/kvx/include/asm/tlbflush.h | 58 +++
arch/kvx/include/asm/vmalloc.h | 10 +
arch/kvx/mm/cacheflush.c | 154 ++++++++
arch/kvx/mm/dma-mapping.c | 95 +++++
arch/kvx/mm/extable.c | 24 ++
arch/kvx/mm/fault.c | 264 ++++++++++++++
arch/kvx/mm/hugetlbpage.c | 317 ++++++++++++++++
arch/kvx/mm/init.c | 527 +++++++++++++++++++++++++++
arch/kvx/mm/kernel_rwx.c | 228 ++++++++++++
arch/kvx/mm/mmap.c | 31 ++
arch/kvx/mm/mmu.c | 204 +++++++++++
arch/kvx/mm/mmu_stats.c | 94 +++++
arch/kvx/mm/tlb.c | 433 ++++++++++++++++++++++
33 files changed, 4494 insertions(+)
create mode 100644 arch/kvx/include/asm/cache.h
create mode 100644 arch/kvx/include/asm/cacheflush.h
create mode 100644 arch/kvx/include/asm/fixmap.h
create mode 100644 arch/kvx/include/asm/hugetlb.h
create mode 100644 arch/kvx/include/asm/l2_cache.h
create mode 100644 arch/kvx/include/asm/l2_cache_defs.h
create mode 100644 arch/kvx/include/asm/mem_map.h
create mode 100644 arch/kvx/include/asm/mmu.h
create mode 100644 arch/kvx/include/asm/mmu_context.h
create mode 100644 arch/kvx/include/asm/mmu_stats.h
create mode 100644 arch/kvx/include/asm/page.h
create mode 100644 arch/kvx/include/asm/page_size.h
create mode 100644 arch/kvx/include/asm/pgalloc.h
create mode 100644 arch/kvx/include/asm/pgtable-bits.h
create mode 100644 arch/kvx/include/asm/pgtable.h
create mode 100644 arch/kvx/include/asm/rm_fw.h
create mode 100644 arch/kvx/include/asm/sparsemem.h
create mode 100644 arch/kvx/include/asm/symbols.h
create mode 100644 arch/kvx/include/asm/tlb.h
create mode 100644 arch/kvx/include/asm/tlb_defs.h
create mode 100644 arch/kvx/include/asm/tlbflush.h
create mode 100644 arch/kvx/include/asm/vmalloc.h
create mode 100644 arch/kvx/mm/cacheflush.c
create mode 100644 arch/kvx/mm/dma-mapping.c
create mode 100644 arch/kvx/mm/extable.c
create mode 100644 arch/kvx/mm/fault.c
create mode 100644 arch/kvx/mm/hugetlbpage.c
create mode 100644 arch/kvx/mm/init.c
create mode 100644 arch/kvx/mm/kernel_rwx.c
create mode 100644 arch/kvx/mm/mmap.c
create mode 100644 arch/kvx/mm/mmu.c
create mode 100644 arch/kvx/mm/mmu_stats.c
create mode 100644 arch/kvx/mm/tlb.c
diff --git a/arch/kvx/include/asm/cache.h b/arch/kvx/include/asm/cache.h
new file mode 100644
index 000000000000..c53599cb6a6f
--- /dev/null
+++ b/arch/kvx/include/asm/cache.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_CACHE_H
+#define _ASM_KVX_CACHE_H
+
+/**
+ * On kvx I$ and D$ have the same size (16KB).
+ * Caches are 16K bytes big, VIPT 4-way set associative, true LRU, with 64-byte
+ * lines. The D$ is also write-through.
+ */
+#define KVX_ICACHE_WAY_COUNT 4
+#define KVX_ICACHE_SET_COUNT 64
+#define KVX_ICACHE_LINE_SHIFT 6
+#define KVX_ICACHE_LINE_SIZE (1 << KVX_ICACHE_LINE_SHIFT)
+#define KVX_ICACHE_SIZE \
+ (KVX_ICACHE_WAY_COUNT * KVX_ICACHE_SET_COUNT * KVX_ICACHE_LINE_SIZE)
+
+/**
+ * Invalidate the whole I-cache if the size to flush is more than this value
+ */
+#define KVX_ICACHE_INVAL_SIZE (KVX_ICACHE_SIZE)
+
+/* D-Cache */
+#define KVX_DCACHE_WAY_COUNT 4
+#define KVX_DCACHE_SET_COUNT 64
+#define KVX_DCACHE_LINE_SHIFT 6
+#define KVX_DCACHE_LINE_SIZE (1 << KVX_DCACHE_LINE_SHIFT)
+#define KVX_DCACHE_SIZE \
+ (KVX_DCACHE_WAY_COUNT * KVX_DCACHE_SET_COUNT * KVX_DCACHE_LINE_SIZE)
+
+/**
+ * Same for I-cache
+ */
+#define KVX_DCACHE_INVAL_SIZE (KVX_DCACHE_SIZE)
+
+#define L1_CACHE_SHIFT KVX_DCACHE_LINE_SHIFT
+#define L1_CACHE_BYTES KVX_DCACHE_LINE_SIZE
+
+#define L2_CACHE_LINE_SIZE 256
+#define L2_CACHE_LINE_MASK (L2_CACHE_LINE_SIZE - 1)
+
+#endif /* _ASM_KVX_CACHE_H */
diff --git a/arch/kvx/include/asm/cacheflush.h b/arch/kvx/include/asm/cacheflush.h
new file mode 100644
index 000000000000..f501e9956958
--- /dev/null
+++ b/arch/kvx/include/asm/cacheflush.h
@@ -0,0 +1,181 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ * Guillaume Thouvenin
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_CACHEFLUSH_H
+#define _ASM_KVX_CACHEFLUSH_H
+
+#include <linux/mm.h>
+#include <linux/io.h>
+
+#include <asm/l2_cache.h>
+
+#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
+
+#define flush_cache_mm(mm) do { } while (0)
+#define flush_cache_range(vma, start, end) do { } while (0)
+#define flush_cache_dup_mm(mm) do { } while (0)
+#define flush_cache_page(vma, vmaddr, pfn) do { } while (0)
+
+#define flush_cache_vmap(start, end) do { } while (0)
+#define flush_cache_vunmap(start, end) do { } while (0)
+
+#define flush_dcache_page(page) do { } while (0)
+
+#define flush_dcache_mmap_lock(mapping) do { } while (0)
+#define flush_dcache_mmap_unlock(mapping) do { } while (0)
+
+#define l1_inval_dcache_all __builtin_kvx_dinval
+#define kvx_fence __builtin_kvx_fence
+#define l1_inval_icache_all __builtin_kvx_iinval
+
+int dcache_wb_inval_virt_range(unsigned long vaddr, unsigned long len, bool wb,
+ bool inval);
+void dcache_wb_inval_phys_range(phys_addr_t addr, unsigned long len, bool wb,
+ bool inval);
+
+/*
+ * In the following functions, if L2$ is enabled, line invalidations
+ * in the L2$ will also trigger (via hw) invalidations in L1D caches of the
+ * same Cluster in case the lines are present in such L1D caches.
+ */
+
+/*
+ * L1 is indexed by virtual addresses and as such, invalidation takes virtual
+ * addresses as arguments.
+ */
+static inline
+void l1_inval_dcache_range(unsigned long vaddr, unsigned long size)
+{
+ unsigned long end = vaddr + size;
+
+ /* Then inval L1 */
+ if (size >= KVX_DCACHE_INVAL_SIZE) {
+ __builtin_kvx_dinval();
+ return;
+ }
+
+ vaddr = ALIGN_DOWN(vaddr, KVX_DCACHE_LINE_SIZE);
+ for (; vaddr < end; vaddr += KVX_DCACHE_LINE_SIZE)
+ __builtin_kvx_dinvall((void *) vaddr);
+}
+
+static inline
+void inval_dcache_range(phys_addr_t paddr, unsigned long size)
+{
+ /*
+ * Inval L2 first to avoid refilling from cached L2 values.
+ * If L2 cache is not enabled, it will return false and hence, we will
+ * fall back on L1 invalidation.
+ */
+ if (!l2_cache_inval_range(paddr, size))
+ l1_inval_dcache_range((unsigned long) phys_to_virt(paddr),
+ size);
+}
+
+static inline
+void wb_dcache_range(phys_addr_t paddr, unsigned long size)
+{
+ /* Fence to ensure all write are committed */
+ kvx_fence();
+
+ l2_cache_wb_range(paddr, size);
+}
+
+static inline
+void wbinval_dcache_range(phys_addr_t paddr, unsigned long size)
+{
+ /* Fence to ensure all write are committed */
+ kvx_fence();
+
+ if (!l2_cache_wbinval_range(paddr, size))
+ l1_inval_dcache_range((unsigned long) phys_to_virt(paddr),
+ size);
+}
+
+static inline
+void l1_inval_icache_range(unsigned long start, unsigned long end)
+{
+ unsigned long addr;
+ unsigned long size = end - start;
+
+ if (size >= KVX_ICACHE_INVAL_SIZE) {
+ __builtin_kvx_iinval();
+ __builtin_kvx_barrier();
+ return;
+ }
+
+ start = ALIGN_DOWN(start, KVX_ICACHE_LINE_SIZE);
+ for (addr = start; addr < end; addr += KVX_ICACHE_LINE_SIZE)
+ __builtin_kvx_iinvals((void *) addr);
+
+ __builtin_kvx_barrier();
+}
+
+static inline
+void wbinval_icache_range(phys_addr_t paddr, unsigned long size)
+{
+ unsigned long vaddr = (unsigned long) phys_to_virt(paddr);
+
+ /* Fence to ensure all write are committed */
+ kvx_fence();
+
+ l2_cache_wbinval_range(paddr, size);
+ /* invalidating l2 cache will invalidate l1 dcache
+ * but not l1 icache
+ */
+ l1_inval_icache_range(vaddr, vaddr + size);
+}
+
+static inline
+void sync_dcache_icache(unsigned long start, unsigned long end)
+{
+ /* Fence to ensure all write are committed to L2 */
+ kvx_fence();
+ /* Then invalidate the L1 icache to reload from L2 */
+ l1_inval_icache_range(start, end);
+}
+
+static inline
+void local_flush_icache_range(unsigned long start, unsigned long end)
+{
+ sync_dcache_icache(start, end);
+}
+
+#ifdef CONFIG_SMP
+void flush_icache_range(unsigned long start, unsigned long end);
+#else
+#define flush_icache_range local_flush_icache_range
+#endif
+
+static inline
+void flush_icache_page(struct vm_area_struct *vma, struct page *page)
+{
+ unsigned long start = (unsigned long) page_address(page);
+ unsigned long end = start + PAGE_SIZE;
+
+ sync_dcache_icache(start, end);
+}
+
+static inline
+void flush_icache_user_range(struct vm_area_struct *vma, struct page *page,
+ unsigned long vaddr, int len)
+{
+ sync_dcache_icache(vaddr, vaddr + len);
+}
+
+#define copy_to_user_page(vma, page, vaddr, dst, src, len) \
+ do { \
+ memcpy(dst, src, len); \
+ if (vma->vm_flags & VM_EXEC) \
+ flush_icache_user_range(vma, page, vaddr, len); \
+ } while (0)
+#define copy_from_user_page(vma, page, vaddr, dst, src, len) \
+ memcpy(dst, src, len)
+
+#endif /* _ASM_KVX_CACHEFLUSH_H */
diff --git a/arch/kvx/include/asm/fixmap.h b/arch/kvx/include/asm/fixmap.h
new file mode 100644
index 000000000000..3863e410d71d
--- /dev/null
+++ b/arch/kvx/include/asm/fixmap.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_FIXMAP_H
+#define _ASM_KVX_FIXMAP_H
+
+/**
+ * Use the latest available kernel address minus one page.
+ * This is needed since __fix_to_virt returns
+ * (FIXADDR_TOP - ((x) << PAGE_SHIFT))
+ * Due to that, first member will be shifted by 0 and will be equal to
+ * FIXADDR_TOP.
+ * Some other architectures simply add a FIX_HOLE at the beginning of
+ * the fixed_addresses enum (I think ?).
+ */
+#define FIXADDR_TOP (-PAGE_SIZE)
+
+#define ASM_FIX_TO_VIRT(IDX) \
+ (FIXADDR_TOP - ((IDX) << PAGE_SHIFT))
+
+#ifndef __ASSEMBLY__
+#include <asm/page.h>
+#include <asm/pgtable.h>
+
+enum fixed_addresses {
+ FIX_EARLYCON_MEM_BASE,
+ FIX_GDB_BARE_DISPLACED_MEM_BASE,
+ /* Used to access text early in RW mode (jump label) */
+ FIX_TEXT_PATCH,
+ __end_of_fixed_addresses
+};
+
+#define FIXADDR_SIZE (__end_of_fixed_addresses << PAGE_SHIFT)
+#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
+#define FIXMAP_PAGE_IO (PAGE_KERNEL_DEVICE)
+
+void __set_fixmap(enum fixed_addresses idx,
+ phys_addr_t phys, pgprot_t prot);
+
+#include <asm-generic/fixmap.h>
+#endif /* __ASSEMBLY__ */
+
+#endif
diff --git a/arch/kvx/include/asm/hugetlb.h b/arch/kvx/include/asm/hugetlb.h
new file mode 100644
index 000000000000..a5984e8ede7e
--- /dev/null
+++ b/arch/kvx/include/asm/hugetlb.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_HUGETLB_H
+#define _ASM_KVX_HUGETLB_H
+
+#include <asm/pgtable.h>
+
+#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT
+extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte);
+
+#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR
+extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep);
+
+#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS
+extern int huge_ptep_set_access_flags(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep,
+ pte_t pte, int dirty);
+
+#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT
+extern void huge_ptep_set_wrprotect(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep);
+
+#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH
+extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep);
+
+#include <asm-generic/hugetlb.h>
+
+#endif /* _ASM_KVX_HUGETLB_H */
diff --git a/arch/kvx/include/asm/l2_cache.h b/arch/kvx/include/asm/l2_cache.h
new file mode 100644
index 000000000000..437b82bb78ab
--- /dev/null
+++ b/arch/kvx/include/asm/l2_cache.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_L2_CACHE_H
+#define _ASM_KVX_L2_CACHE_H
+
+#include <linux/bits.h>
+#include <linux/jump_label.h>
+
+#ifdef CONFIG_L2_CACHE
+
+#include <asm/cache.h>
+#include <asm/l2_cache_defs.h>
+
+void l2_cache_push_area_cmd(u64 cmd_type, phys_addr_t start,
+ unsigned long size);
+
+extern struct static_key_false l2_enabled;
+
+static bool
+l2_cache_send_cmd(u64 cmd_type, phys_addr_t start, unsigned long size)
+{
+ if (static_branch_likely(&l2_enabled)) {
+ l2_cache_push_area_cmd(cmd_type, start, size);
+ return true;
+ }
+
+ return false;
+}
+
+static inline bool
+l2_cache_wbinval_range(phys_addr_t start, unsigned long size)
+{
+ return l2_cache_send_cmd(L2_CMD_OP_CMD_PURGE_AREA, start, size);
+}
+
+static inline bool
+l2_cache_wb_range(phys_addr_t start, unsigned long size)
+{
+ return l2_cache_send_cmd(L2_CMD_OP_CMD_FLUSH_AREA, start, size);
+}
+
+static inline bool
+l2_cache_inval_range(phys_addr_t start, unsigned long size)
+{
+ /* We don't need invalidation to be synced (ie 0) */
+ return l2_cache_send_cmd(L2_CMD_OP_CMD_INVAL_AREA, start, size);
+}
+
+#else
+
+static inline bool
+l2_cache_wbinval_range(phys_addr_t start, unsigned long size)
+{
+ return false;
+}
+
+static inline bool
+l2_cache_wb_range(phys_addr_t start, unsigned long size)
+{
+ return false;
+}
+
+static inline bool
+l2_cache_inval_range(phys_addr_t start, unsigned long size)
+{
+ return false;
+}
+
+#endif
+
+#endif /* _ASM_KVX_L2_CACHE_H */
diff --git a/arch/kvx/include/asm/l2_cache_defs.h b/arch/kvx/include/asm/l2_cache_defs.h
new file mode 100644
index 000000000000..81f001d9087a
--- /dev/null
+++ b/arch/kvx/include/asm/l2_cache_defs.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julien Hascoet
+ */
+
+#ifndef _ASM_KVX_L2_CACHE_DEFS_H
+#define _ASM_KVX_L2_CACHE_DEFS_H
+
+/* Error registers */
+enum l2_api_error {
+ L2_STA_ERR_CFG = 1,
+ L2_STA_ERR_ADDR,
+ L2_STA_ERR_STOP,
+ L2_STA_ERR_NO_MEM,
+ L2_STA_ERR_DIS,
+ L2_STA_ERR_DT
+};
+
+/* L2 Cache status registers definition */
+#define L2_STATUS_OFFSET 0x0
+#define L2_STATUS_ERROR_MASK 0x4
+#define L2_STATUS_ERROR_SHIFT 0x2
+#define L2_STATUS_READY_MASK 0x2
+#define L2_STATUS_READY_SHIFT 0x1
+#define L2_STATUS_VALID_MASK 0x1
+#define L2_STATUS_VALID_SHIFT 0x0
+#define L2_STATUS_ACK_MASK 0x80
+#define L2_STATUS_ACK_SHIFT 0x7
+
+/* L2 Cache error registers definition */
+#define L2_ERROR_OFFSET 0x20
+#define L2_ERROR_SETUP_ERR_MASK 0x1
+#define L2_ERROR_SETUP_ERR_SHIFT 0x0
+#define L2_ERROR_API_ERR_MASK 0x20
+#define L2_ERROR_API_ERR_SHIFT 0x5
+#define L2_ERROR_ERROR_CODE_MASK 0xFF00
+#define L2_ERROR_ERROR_CODE_SHIFT 0x8
+
+/* L2 Cache instance registers definition */
+#define L2_INSTANCE_OFFSET 0x8
+#define L2_INSTANCE_CMD_QUEUE_SIZE_SHIFT 0x20
+#define L2_INSTANCE_CMD_QUEUE_SIZE_MASK 0x7F00000000UL
+
+/* L2 Cache commands fifo registers definition */
+#define L2_CMD_OFFSET 0x400
+#define L2_CMD_READ_IDX_OFFSET 0x0
+#define L2_CMD_WRITE_IDX_OFFSET 0x8
+#define L2_CMD_DOORBELL_READ_ADDR_OFFSET 0x10
+#define L2_CMD_DOORBELL_WRITE_ADDR_OFFSET 0x18
+#define L2_CMD_FIFO_OFFSET 0x20
+#define L2_CMD_FIFO_ELEM_SIZE 0x20
+#define L2_CMD_OP_CMD_PURGE_LINE 0x4
+#define L2_CMD_OP_CMD_PURGE_AREA 0x8
+#define L2_CMD_OP_CMD_FLUSH_AREA 0x9
+#define L2_CMD_OP_CMD_INVAL_AREA 0xA
+#define L2_CMD_OP_VALID_SHIFT 0x0
+#define L2_CMD_OP_SYNC_SHIFT 0x1
+#define L2_CMD_OP_CMD_SHIFT 0x2
+#define L2_CMD_OP_ARG_COUNT 0x3
+
+
+#endif /* _ASM_KVX_L2_CACHE_DEFS_H */
diff --git a/arch/kvx/include/asm/mem_map.h b/arch/kvx/include/asm/mem_map.h
new file mode 100644
index 000000000000..ea90144209ce
--- /dev/null
+++ b/arch/kvx/include/asm/mem_map.h
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef _ASM_KVX_MEM_MAP_H
+#define _ASM_KVX_MEM_MAP_H
+
+#include <linux/const.h>
+#include <linux/sizes.h>
+
+#include <asm/page.h>
+#include <asm/fixmap.h>
+
+/**
+ * kvx memory mapping defines
+ * For more information on memory mapping, please see
+ * Documentation/kvx/kvx-mmu.txt
+ *
+ * All _BASE defines are relative to PAGE_OFFSET
+ */
+
+/* Guard between various memory map zones */
+#define MAP_GUARD_SIZE SZ_1G
+
+/**
+ * Kernel direct memory mapping
+ */
+#define KERNEL_DIRECT_MEMORY_MAP_BASE PAGE_OFFSET
+#define KERNEL_DIRECT_MEMORY_MAP_SIZE UL(0x1000000000)
+#define KERNEL_DIRECT_MEMORY_MAP_END \
+ (KERNEL_DIRECT_MEMORY_MAP_BASE + KERNEL_DIRECT_MEMORY_MAP_SIZE)
+
+/**
+ * Vmalloc mapping (goes from kernel direct memory map up to fixmap start -
+ * guard size)
+ */
+#define KERNEL_VMALLOC_MAP_BASE (KERNEL_DIRECT_MEMORY_MAP_END + MAP_GUARD_SIZE)
+#define KERNEL_VMALLOC_MAP_SIZE \
+ (FIXADDR_START - KERNEL_VMALLOC_MAP_BASE - MAP_GUARD_SIZE)
+
+#endif
diff --git a/arch/kvx/include/asm/mmu.h b/arch/kvx/include/asm/mmu.h
new file mode 100644
index 000000000000..fbf5ed286e87
--- /dev/null
+++ b/arch/kvx/include/asm/mmu.h
@@ -0,0 +1,296 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Marc Poulhiès
+ */
+
+#ifndef _ASM_KVX_MMU_H
+#define _ASM_KVX_MMU_H
+
+#include <linux/bug.h>
+#include <linux/types.h>
+#include <linux/threads.h>
+
+#include <asm/page.h>
+#include <asm/sfr.h>
+#include <asm/page.h>
+#include <asm/pgtable-bits.h>
+#include <asm/tlb_defs.h>
+
+/* Virtual addresses can use at most 41 bits */
+#define MMU_VIRT_BITS 41
+
+/*
+ * See Documentation/kvx/kvx-mmu.txt for details about the division of the
+ * virtual memory space.
+ */
+#if defined(CONFIG_KVX_4K_PAGES)
+#define MMU_USR_ADDR_BITS 39
+#else
+#error "Only 4Ko page size is supported at this time"
+#endif
+
+typedef struct mm_context {
+ unsigned long end_brk;
+ unsigned long asn[NR_CPUS];
+ unsigned long sigpage;
+} mm_context_t;
+
+struct __packed tlb_entry_low {
+ unsigned int es:2; /* Entry Status */
+ unsigned int cp:2; /* Cache Policy */
+ unsigned int pa:4; /* Protection Attributes */
+ unsigned int r:2; /* Reserved */
+ unsigned int ps:2; /* Page Size */
+ unsigned int fn:28; /* Frame Number */
+};
+
+struct __packed tlb_entry_high {
+ unsigned int asn:9; /* Address Space Number */
+ unsigned int g:1; /* Global Indicator */
+ unsigned int vs:2; /* Virtual Space */
+ unsigned int pn:29; /* Page Number */
+};
+
+struct kvx_tlb_format {
+ union {
+ struct tlb_entry_low tel;
+ uint64_t tel_val;
+ };
+ union {
+ struct tlb_entry_high teh;
+ uint64_t teh_val;
+ };
+};
+
+#define KVX_EMPTY_TLB_ENTRY { .tel_val = 0x0, .teh_val = 0x0 }
+
+/* Bit [0:39] of the TLB format corresponds to TLB Entry low */
+/* Bit [40:80] of the TLB format corresponds to the TLB Entry high */
+#define kvx_mmu_set_tlb_entry(tlbf) do { \
+ kvx_sfr_set(TEL, (uint64_t) tlbf.tel_val); \
+ kvx_sfr_set(TEH, (uint64_t) tlbf.teh_val); \
+} while (0)
+
+#define kvx_mmu_get_tlb_entry(tlbf) do { \
+ tlbf.tel_val = kvx_sfr_get(TEL); \
+ tlbf.teh_val = kvx_sfr_get(TEH); \
+} while (0)
+
+/* Use kvx_mmc_ to read a field from MMC value passed as parameter */
+#define __kvx_mmc(mmc_reg, field) \
+ kvx_sfr_field_val(mmc_reg, MMC, field)
+
+#define kvx_mmc_error(mmc) __kvx_mmc(mmc, E)
+#define kvx_mmc_parity(mmc) __kvx_mmc(mmc, PAR)
+#define kvx_mmc_sb(mmc) __kvx_mmc(mmc, SB)
+#define kvx_mmc_ss(mmc) __kvx_mmc(mmc, SS)
+#define kvx_mmc_sw(mmc) __kvx_mmc(mmc, SW)
+#define kvx_mmc_asn(mmc) __kvx_mmc(mmc, ASN)
+
+#define KVX_TLB_ACCESS_READ 0
+#define KVX_TLB_ACCESS_WRITE 1
+#define KVX_TLB_ACCESS_PROBE 2
+
+#ifdef CONFIG_KVX_DEBUG_TLB_ACCESS
+
+#define KVX_TLB_ACCESS_SIZE (1 << CONFIG_KVX_DEBUG_TLB_ACCESS_BITS)
+#define KVX_TLB_ACCESS_MASK GENMASK((CONFIG_KVX_DEBUG_TLB_ACCESS_BITS - 1), 0)
+#define KVX_TLB_ACCESS_GET_IDX(idx) (idx & KVX_TLB_ACCESS_MASK)
+
+/* This structure is used to make decoding of MMC easier in gdb */
+struct mmc_t {
+ unsigned int asn:9;
+ unsigned int s: 1;
+ unsigned int r1: 4;
+ unsigned int sne: 1;
+ unsigned int spe: 1;
+ unsigned int ptc: 2;
+ unsigned int sw: 4;
+ unsigned int ss: 6;
+ unsigned int sb: 1;
+ unsigned int r2: 1;
+ unsigned int par: 1;
+ unsigned int e: 1;
+};
+
+struct __packed kvx_tlb_access_t {
+ struct kvx_tlb_format entry; /* 128 bits */
+ union {
+ struct mmc_t mmc;
+ uint32_t mmc_val;
+ };
+ uint32_t type;
+};
+
+extern void kvx_update_tlb_access(int type);
+
+#else
+#define kvx_update_tlb_access(type) do {} while (0)
+#endif
+
+static inline void kvx_mmu_readtlb(void)
+{
+ kvx_update_tlb_access(KVX_TLB_ACCESS_READ);
+ asm volatile ("tlbread\n;;");
+}
+
+static inline void kvx_mmu_writetlb(void)
+{
+ kvx_update_tlb_access(KVX_TLB_ACCESS_WRITE);
+ asm volatile ("tlbwrite\n;;");
+}
+
+static inline void kvx_mmu_probetlb(void)
+{
+ kvx_update_tlb_access(KVX_TLB_ACCESS_PROBE);
+ asm volatile ("tlbprobe\n;;");
+}
+
+#define kvx_mmu_add_entry(buffer, way, entry) do { \
+ kvx_sfr_set_field(MMC, SB, buffer); \
+ kvx_sfr_set_field(MMC, SW, way); \
+ kvx_mmu_set_tlb_entry(entry); \
+ kvx_mmu_writetlb(); \
+} while (0)
+
+#define kvx_mmu_remove_ltlb_entry(way) do { \
+ struct kvx_tlb_format __invalid_entry = KVX_EMPTY_TLB_ENTRY; \
+ kvx_mmu_add_entry(MMC_SB_LTLB, way, __invalid_entry); \
+} while (0)
+
+static inline int get_page_size_shift(int ps)
+{
+ /*
+ * Use the same assembly trick using sbmm to directly get the page size
+ * shift using a constant which encodes all page size shifts
+ */
+ return __builtin_kvx_sbmm8(KVX_PS_SHIFT_MATRIX,
+ KVX_SBMM_BYTE_SEL << ps);
+}
+
+/*
+ * 4 bits are used to index the kvx access permissions. Bits are used as
+ * follow:
+ *
+ * +---------------+------------+-------------+------------+
+ * | Bit 3 | Bit 2 | Bit 1 | Bit 0 |
+ * |---------------+------------+-------------+------------|
+ * | _PAGE_GLOBAL | _PAGE_EXEC | _PAGE_WRITE | _PAGE_READ |
+ * +---------------+------------+-------------+------------+
+ *
+ * If _PAGE_GLOBAL is set then the page belongs to the kernel. Otherwise it
+ * belongs to the user. When the page belongs to user-space then give the
+ * same rights to the kernel-space.
+ * In order to quickly compute policy from this value, we use sbmm instruction.
+ * The main interest is to avoid an additionnal load and specifically in the
+ * assembly refill handler.
+ */
+static inline u8 get_page_access_perms(u8 policy)
+{
+ /* If PAGE_READ is unset, there is no permission for this page */
+ if (!(policy & (_PAGE_READ >> _PAGE_PERMS_SHIFT)))
+ return TLB_PA_NA_NA;
+
+ /* Discard _PAGE_READ bit to get a linear number in [0,7] */
+ policy >>= 1;
+
+ /* Use sbmm to directly get the page perms */
+ return __builtin_kvx_sbmm8(KVX_PAGE_PA_MATRIX,
+ KVX_SBMM_BYTE_SEL << policy);
+}
+
+static inline struct kvx_tlb_format tlb_mk_entry(
+ void *paddr,
+ void *vaddr,
+ unsigned int ps,
+ unsigned int global,
+ unsigned int pa,
+ unsigned int cp,
+ unsigned int asn,
+ unsigned int es)
+{
+ struct kvx_tlb_format entry;
+ u64 mask = ULONG_MAX << get_page_size_shift(ps);
+
+ BUG_ON(ps >= (1 << KVX_SFR_TEL_PS_WIDTH));
+
+ /*
+ * 0 matches the virtual space:
+ * - either we are virtualized and the hypervisor will set it
+ * for us when using writetlb
+ * - Or we are native and the virtual space is 0
+ */
+ entry.teh_val = TLB_MK_TEH_ENTRY((uintptr_t)vaddr & mask, 0, global,
+ asn);
+ entry.tel_val = TLB_MK_TEL_ENTRY((uintptr_t)paddr, ps, es, cp, pa);
+
+ return entry;
+}
+
+static inline unsigned long tlb_entry_phys(struct kvx_tlb_format tlbe)
+{
+ return ((unsigned long) tlbe.tel.fn << KVX_SFR_TEL_FN_SHIFT);
+}
+
+static inline unsigned long tlb_entry_virt(struct kvx_tlb_format tlbe)
+{
+ return ((unsigned long) tlbe.teh.pn << KVX_SFR_TEH_PN_SHIFT);
+}
+
+static inline unsigned long tlb_entry_size(struct kvx_tlb_format tlbe)
+{
+ return BIT(get_page_size_shift(tlbe.tel.ps));
+}
+
+static inline int tlb_entry_overlaps(struct kvx_tlb_format tlbe1,
+ struct kvx_tlb_format tlbe2)
+{
+ unsigned long start1, end1;
+ unsigned long start2, end2;
+
+ start1 = tlb_entry_virt(tlbe1);
+ end1 = start1 + tlb_entry_size(tlbe1);
+
+ start2 = tlb_entry_virt(tlbe2);
+ end2 = start2 + tlb_entry_size(tlbe2);
+
+ return start1 <= end2 && end1 >= start2;
+}
+
+static inline int tlb_entry_match_addr(struct kvx_tlb_format tlbe,
+ unsigned long vaddr)
+{
+ /*
+ * TLB entries store up to 41 bits so we must truncate the provided
+ * address to match teh.pn.
+ */
+ vaddr &= GENMASK(MMU_VIRT_BITS - 1, KVX_SFR_TEH_PN_SHIFT);
+
+ return tlb_entry_virt(tlbe) == vaddr;
+}
+
+extern void kvx_mmu_early_setup(void);
+
+
+#if defined(CONFIG_STRICT_KERNEL_RWX)
+void init_kernel_rwx(void);
+void paging_init(void);
+#else
+
+static inline void paging_init(void) {}
+#endif
+
+void kvx_mmu_ltlb_remove_entry(unsigned long vaddr);
+void kvx_mmu_ltlb_add_entry(unsigned long vaddr, phys_addr_t paddr,
+ pgprot_t flags, unsigned long page_shift);
+
+void kvx_mmu_jtlb_add_entry(unsigned long address, pte_t *ptep,
+ unsigned int asn);
+extern void mmu_early_init(void);
+
+struct mm_struct;
+
+#endif /* _ASM_KVX_MMU_H */
diff --git a/arch/kvx/include/asm/mmu_context.h b/arch/kvx/include/asm/mmu_context.h
new file mode 100644
index 000000000000..39fa92f1506b
--- /dev/null
+++ b/arch/kvx/include/asm/mmu_context.h
@@ -0,0 +1,156 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef __ASM_KVX_MMU_CONTEXT_H
+#define __ASM_KVX_MMU_CONTEXT_H
+
+/*
+ * Management of the Address Space Number:
+ * Coolidge architecture provides a 9-bit ASN to tag TLB entries. This can be
+ * used to allow several entries with the same virtual address (so from
+ * different process) to be in the TLB at the same time. That means that won't
+ * necessarily flush the TLB when a context switch occurs and so it will
+ * improve performances.
+ */
+#include <linux/smp.h>
+
+#include <asm/mmu.h>
+#include <asm/sfr_defs.h>
+#include <asm/tlbflush.h>
+
+#include <asm-generic/mm_hooks.h>
+
+#define MM_CTXT_ASN_MASK GENMASK(KVX_SFR_MMC_ASN_WIDTH - 1, 0)
+#define MM_CTXT_CYCLE_MASK (~MM_CTXT_ASN_MASK)
+#define MM_CTXT_NO_ASN UL(0x0)
+#define MM_CTXT_FIRST_CYCLE (MM_CTXT_ASN_MASK + 1)
+
+#define mm_asn(mm, cpu) ((mm)->context.asn[cpu])
+
+DECLARE_PER_CPU(unsigned long, kvx_asn_cache);
+#define cpu_asn_cache(cpu) per_cpu(kvx_asn_cache, cpu)
+
+static inline void get_new_mmu_context(struct mm_struct *mm, unsigned int cpu)
+{
+ unsigned long asn = cpu_asn_cache(cpu);
+
+ asn++;
+ /* Check if we need to start a new cycle */
+ if ((asn & MM_CTXT_ASN_MASK) == 0) {
+ pr_debug("%s: start new cycle, flush all tlb\n", __func__);
+ local_flush_tlb_all();
+
+ /*
+ * Above check for rollover of 9 bit ASN in 64 bit container.
+ * If the container itself wrapped around, set it to a non zero
+ * "generation" to distinguish from no context
+ */
+ if (asn == 0)
+ asn = MM_CTXT_FIRST_CYCLE;
+ }
+
+ cpu_asn_cache(cpu) = asn;
+ mm_asn(mm, cpu) = asn;
+
+ pr_debug("%s: mm = 0x%llx: cpu[%d], cycle: %lu, asn: %lu\n",
+ __func__, (unsigned long long)mm, cpu,
+ (asn & MM_CTXT_CYCLE_MASK) >> KVX_SFR_MMC_ASN_WIDTH,
+ asn & MM_CTXT_ASN_MASK);
+}
+
+static inline void get_mmu_context(struct mm_struct *mm, unsigned int cpu)
+{
+
+ unsigned long asn = mm_asn(mm, cpu);
+
+ /*
+ * Move to new ASN if it was not from current alloc-cycle/generation.
+ * This is done by ensuring that the generation bits in both
+ * mm->context.asn and cpu_asn_cache counter are exactly same.
+ *
+ * NOTE: this also works for checking if mm has a context since the
+ * first alloc-cycle/generation is always '1'. MM_CTXT_NO_ASN value
+ * contains cycle '0', and thus it will match.
+ */
+ if ((asn ^ cpu_asn_cache(cpu)) & MM_CTXT_CYCLE_MASK)
+ get_new_mmu_context(mm, cpu);
+}
+
+static inline void activate_context(struct mm_struct *mm, unsigned int cpu)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ get_mmu_context(mm, cpu);
+
+ kvx_sfr_set_field(MMC, ASN, mm_asn(mm, cpu) & MM_CTXT_ASN_MASK);
+
+ local_irq_restore(flags);
+}
+
+/**
+ * Redefining the generic hooks that are:
+ * - activate_mm
+ * - deactivate_mm
+ * - enter_lazy_tlb
+ * - init_new_context
+ * - destroy_context
+ * - switch_mm
+ */
+
+#define activate_mm(prev, next) switch_mm((prev), (next), NULL)
+#define deactivate_mm(tsk, mm) do { } while (0)
+#define enter_lazy_tlb(mm, tsk) do { } while (0)
+
+static inline int init_new_context(struct task_struct *tsk,
+ struct mm_struct *mm)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu)
+ mm_asn(mm, cpu) = MM_CTXT_NO_ASN;
+
+ return 0;
+}
+
+static inline void destroy_context(struct mm_struct *mm)
+{
+ int cpu = smp_processor_id();
+ unsigned long flags;
+
+ local_irq_save(flags);
+ mm_asn(mm, cpu) = MM_CTXT_NO_ASN;
+ local_irq_restore(flags);
+}
+
+static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next,
+ struct task_struct *tsk)
+{
+ unsigned int cpu = smp_processor_id();
+
+ /**
+ * Comment taken from arc, but logic is the same for us:
+ *
+ * Note that the mm_cpumask is "aggregating" only, we don't clear it
+ * for the switched-out task, unlike some other arches.
+ * It is used to enlist cpus for sending TLB flush IPIs and not sending
+ * it to CPUs where a task once ran-on, could cause stale TLB entry
+ * re-use, specially for a multi-threaded task.
+ * e.g. T1 runs on C1, migrates to C3. T2 running on C2 munmaps.
+ * For a non-aggregating mm_cpumask, IPI not sent C1, and if T1
+ * were to re-migrate to C1, it could access the unmapped region
+ * via any existing stale TLB entries.
+ */
+ cpumask_set_cpu(cpu, mm_cpumask(next));
+
+ if (prev != next)
+ activate_context(next, cpu);
+}
+
+
+#endif /* __ASM_KVX_MMU_CONTEXT_H */
diff --git a/arch/kvx/include/asm/mmu_stats.h b/arch/kvx/include/asm/mmu_stats.h
new file mode 100644
index 000000000000..999352dbc1ce
--- /dev/null
+++ b/arch/kvx/include/asm/mmu_stats.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_MMU_STATS_H
+#define _ASM_KVX_MMU_STATS_H
+
+#ifdef CONFIG_KVX_MMU_STATS
+#include <linux/percpu.h>
+
+struct mmu_refill_stats {
+ unsigned long count;
+ unsigned long total;
+ unsigned long min;
+ unsigned long max;
+};
+
+enum mmu_refill_type {
+ MMU_REFILL_TYPE_USER,
+ MMU_REFILL_TYPE_KERNEL,
+ MMU_REFILL_TYPE_KERNEL_DIRECT,
+ MMU_REFILL_TYPE_COUNT,
+};
+
+struct mmu_stats {
+ struct mmu_refill_stats refill[MMU_REFILL_TYPE_COUNT];
+ /* keep these fields ordered this way for assembly */
+ unsigned long cycles_between_refill;
+ unsigned long last_refill;
+ unsigned long tlb_flush_all;
+};
+
+DECLARE_PER_CPU(struct mmu_stats, mmu_stats);
+#endif
+
+#endif /* _ASM_KVX_MMU_STATS_H */
diff --git a/arch/kvx/include/asm/page.h b/arch/kvx/include/asm/page.h
new file mode 100644
index 000000000000..cea5f02d12bb
--- /dev/null
+++ b/arch/kvx/include/asm/page.h
@@ -0,0 +1,187 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_PAGE_H
+#define _ASM_KVX_PAGE_H
+
+#include <linux/const.h>
+
+#define PAGE_SHIFT CONFIG_KVX_PAGE_SHIFT
+#define PAGE_SIZE _BITUL(PAGE_SHIFT)
+#define PAGE_MASK (~(PAGE_SIZE - 1))
+
+#define PHYS_OFFSET CONFIG_KVX_PHYS_OFFSET
+#define PAGE_OFFSET CONFIG_KVX_PAGE_OFFSET
+
+#define VA_TO_PA_OFFSET (PHYS_OFFSET - PAGE_OFFSET)
+#define PA_TO_VA_OFFSET (PAGE_OFFSET - PHYS_OFFSET)
+
+/*
+ * These macros are specifically written for assembly. They are useful for
+ * converting symbols above PAGE_OFFSET to their physical addresses.
+ */
+#define __PA(x) ((x) + VA_TO_PA_OFFSET)
+#define __VA(x) ((x) + PA_TO_VA_OFFSET)
+
+/*
+ * PFN starts at 0 if physical address starts at 0x0. As it is not the case
+ * for the kvx we need to apply an offset to the calculated PFN.
+ */
+#define ARCH_PFN_OFFSET ((unsigned long)(PHYS_OFFSET >> PAGE_SHIFT))
+
+#if defined(CONFIG_KVX_4K_PAGES)
+/* Maximum usable bit using 4K pages and current page table layout */
+#define VA_MAX_BITS 40
+#define PGDIR_SHIFT 30
+#define PMD_SHIFT 21
+#else
+#error "64K page not supported"
+#endif
+
+/*
+ * Define _SHIFT, _SIZE and _MASK corresponding of the different page
+ * sizes supported by kvx.
+ */
+#define KVX_PAGE_4K_SHIFT 12
+#define KVX_PAGE_4K_SIZE BIT(KVX_PAGE_4K_SHIFT)
+#define KVX_PAGE_4K_MASK (~(KVX_PAGE_4K_SIZE - 1))
+
+#define KVX_PAGE_64K_SHIFT 16
+#define KVX_PAGE_64K_SIZE BIT(KVX_PAGE_64K_SHIFT)
+#define KVX_PAGE_64K_MASK (~(KVX_PAGE_64K_SIZE - 1))
+
+#define KVX_PAGE_2M_SHIFT 21
+#define KVX_PAGE_2M_SIZE BIT(KVX_PAGE_2M_SHIFT)
+#define KVX_PAGE_2M_MASK (~(KVX_PAGE_2M_SIZE - 1))
+
+#define KVX_PAGE_512M_SHIFT 29
+#define KVX_PAGE_512M_SIZE BIT(KVX_PAGE_512M_SHIFT)
+#define KVX_PAGE_512M_MASK (~(KVX_PAGE_512M_SIZE - 1))
+
+/* Encode all page shift into one 32bit constant for sbmm */
+#define KVX_PS_SHIFT_MATRIX ((KVX_PAGE_512M_SHIFT << 24) | \
+ (KVX_PAGE_2M_SHIFT << 16) | \
+ (KVX_PAGE_64K_SHIFT << 8) | \
+ (KVX_PAGE_4K_SHIFT))
+
+/* Encode all page access policy into one 64bit constant for sbmm */
+#define KVX_PAGE_PA_MATRIX ((UL(TLB_PA_NA_RWX) << 56) | \
+ (UL(TLB_PA_NA_RX) << 48) | \
+ (UL(TLB_PA_NA_RW) << 40) | \
+ (UL(TLB_PA_NA_R) << 32) | \
+ (UL(TLB_PA_RWX_RWX) << 24) | \
+ (UL(TLB_PA_RX_RX) << 16) | \
+ (UL(TLB_PA_RW_RW) << 8) | \
+ (UL(TLB_PA_R_R)))
+
+/*
+ * Select a byte using sbmm8. When shifted by one bit left, we get the next
+ * byte.
+ * For instance using this default constant with sbmm yields the value between
+ * first byte of the double word.
+ * If constant is shifted by 1, the value is now 0x0000000000000002ULL and this
+ * yield the second byte and so on, and so on !
+ */
+#define KVX_SBMM_BYTE_SEL 0x01
+
+#ifndef __ASSEMBLY__
+
+#include <linux/string.h>
+
+/* Page Global Directory entry */
+typedef struct {
+ unsigned long pgd;
+} pgd_t;
+
+/* Page Middle Directory entry */
+typedef struct {
+ unsigned long pmd;
+} pmd_t;
+
+/* Page Table entry */
+typedef struct {
+ unsigned long pte;
+} pte_t;
+
+/* Protection bits */
+typedef struct {
+ unsigned long pgprot;
+} pgprot_t;
+
+typedef struct page *pgtable_t;
+
+/**
+ * Macros to access entry values
+ */
+#define pgd_val(x) ((x).pgd)
+#define pmd_val(x) ((x).pmd)
+#define pte_val(x) ((x).pte)
+#define pgprot_val(x) ((x).pgprot)
+
+/**
+ * Macro to create entry from value
+ */
+#define __pgd(x) ((pgd_t) { (x) })
+#define __pmd(x) ((pmd_t) { (x) })
+#define __pte(x) ((pte_t) { (x) })
+#define __pgprot(x) ((pgprot_t) { (x) })
+
+#define pte_pgprot(x) __pgprot(pte_val(x) & ~KVX_PFN_MASK)
+
+#define __pa(x) ((unsigned long)(x) + VA_TO_PA_OFFSET)
+#define __va(x) ((void *)((unsigned long) (x) + PA_TO_VA_OFFSET))
+
+#define phys_to_pfn(phys) (PFN_DOWN(phys))
+#define pfn_to_phys(pfn) (PFN_PHYS(pfn))
+
+#define virt_to_pfn(vaddr) (phys_to_pfn(__pa(vaddr)))
+#define pfn_to_virt(pfn) (__va(pfn_to_phys(pfn)))
+
+#define virt_to_page(vaddr) (pfn_to_page(virt_to_pfn(vaddr)))
+#define page_to_virt(page) (pfn_to_virt(page_to_pfn(page)))
+
+#define page_to_phys(page) virt_to_phys(page_to_virt(page))
+#define phys_to_page(phys) (pfn_to_page(phys_to_pfn(phys)))
+
+#define virt_addr_valid(vaddr) (pfn_valid(virt_to_pfn(vaddr)))
+
+#ifdef CONFIG_FLATMEM
+static inline bool pfn_valid(unsigned long pfn)
+{
+ /* avoid <linux/mm.h> include hell */
+ extern unsigned long max_mapnr;
+
+ return ((pfn >= ARCH_PFN_OFFSET) &&
+ (pfn < (ARCH_PFN_OFFSET + max_mapnr)));
+}
+#endif
+
+extern void clear_page(void *to);
+extern void copy_page(void *to, void *from);
+
+static inline void clear_user_page(void *page, unsigned long vaddr,
+ struct page *pg)
+{
+ clear_page(page);
+}
+
+static inline void copy_user_page(void *to, void *from, unsigned long vaddr,
+ struct page *topage)
+{
+ copy_page(to, from);
+}
+
+#define VM_DATA_DEFAULT_FLAGS (VM_READ | VM_WRITE | \
+ VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
+
+#include <asm-generic/memory_model.h>
+#include <asm-generic/getorder.h>
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_KVX_PAGE_H */
diff --git a/arch/kvx/include/asm/page_size.h b/arch/kvx/include/asm/page_size.h
new file mode 100644
index 000000000000..2c2850205b50
--- /dev/null
+++ b/arch/kvx/include/asm/page_size.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_PAGE_SIZE_H
+#define _ASM_KVX_PAGE_SIZE_H
+
+#include <asm/tlb_defs.h>
+
+#if defined(CONFIG_HUGETLB_PAGE)
+#define HUGE_PAGE_SIZE (MMC_PMJ_64K | MMC_PMJ_2M | MMC_PMJ_512M)
+#else
+#define HUGE_PAGE_SIZE (0)
+#endif
+
+#if defined(CONFIG_KVX_4K_PAGES)
+#define TLB_DEFAULT_PS TLB_PS_4K
+#define KVX_SUPPORTED_PSIZE (MMC_PMJ_4K | HUGE_PAGE_SIZE)
+#elif defined(CONFIG_KVX_64K_PAGES)
+#define TLB_DEFAULT_PS TLB_PS_64K
+#define KVX_SUPPORTED_PSIZE (MMC_PMJ_64K | HUGE_PAGE_SIZE)
+#else
+#error "Unsupported page size"
+#endif
+
+#endif
diff --git a/arch/kvx/include/asm/pgalloc.h b/arch/kvx/include/asm/pgalloc.h
new file mode 100644
index 000000000000..0e654dd1a072
--- /dev/null
+++ b/arch/kvx/include/asm/pgalloc.h
@@ -0,0 +1,101 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_PGALLOC_H
+#define _ASM_KVX_PGALLOC_H
+
+#include <linux/mm.h>
+#include <asm/tlb.h>
+
+#define __HAVE_ARCH_PGD_FREE
+#include <asm-generic/pgalloc.h> /* for pte_{alloc,free}_one */
+
+static inline void check_pgt_cache(void)
+{
+ /*
+ * check_pgt_cache() is called to check watermarks from counters that
+ * computes the number of pages allocated by cached allocation functions
+ * pmd_alloc_one_fast() and pte_alloc_one_fast().
+ * Currently we just skip this test.
+ */
+}
+
+/**
+ * PGD
+ */
+
+static inline void
+pgd_free(struct mm_struct *mm, pgd_t *pgd)
+{
+ free_pages((unsigned long) pgd, PAGES_PER_PGD);
+}
+
+static inline
+pgd_t *pgd_alloc(struct mm_struct *mm)
+{
+ pgd_t *pgd;
+
+ pgd = (pgd_t *) __get_free_pages(GFP_KERNEL, PAGES_PER_PGD);
+ if (unlikely(pgd == NULL))
+ return NULL;
+
+ memset(pgd, 0, USER_PTRS_PER_PGD * sizeof(pgd_t));
+
+ /* Copy kernel mappings */
+ memcpy(pgd + USER_PTRS_PER_PGD,
+ init_mm.pgd + USER_PTRS_PER_PGD,
+ (PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t));
+
+ return pgd;
+}
+
+/**
+ * PUD
+ */
+
+static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
+{
+ unsigned long pfn = virt_to_pfn(pmd);
+
+ set_pud(pud, __pud((unsigned long)pfn << PAGE_SHIFT));
+}
+
+/**
+ * PMD
+ */
+
+static inline void pmd_populate_kernel(struct mm_struct *mm,
+ pmd_t *pmd, pte_t *pte)
+{
+ unsigned long pfn = virt_to_pfn(pte);
+
+ set_pmd(pmd, __pmd((unsigned long)pfn << PAGE_SHIFT));
+}
+
+static inline void pmd_populate(struct mm_struct *mm,
+ pmd_t *pmd, pgtable_t pte)
+{
+ unsigned long pfn = virt_to_pfn(page_address(pte));
+
+ set_pmd(pmd, __pmd((unsigned long)pfn << PAGE_SHIFT));
+}
+
+#if CONFIG_PGTABLE_LEVELS > 2
+#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd)
+#endif /* CONFIG_PGTABLE_LEVELS > 2 */
+
+/**
+ * PTE
+ */
+
+#define __pte_free_tlb(tlb, pte, buf) \
+do { \
+ pgtable_pte_page_dtor(pte); \
+ tlb_remove_page((tlb), pte); \
+} while (0)
+
+#endif /* _ASM_KVX_PGALLOC_H */
diff --git a/arch/kvx/include/asm/pgtable-bits.h b/arch/kvx/include/asm/pgtable-bits.h
new file mode 100644
index 000000000000..98ee7d1fbe63
--- /dev/null
+++ b/arch/kvx/include/asm/pgtable-bits.h
@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Julian Vetter
+ */
+
+#ifndef _ASM_KVX_PGTABLE_BITS_H
+#define _ASM_KVX_PGTABLE_BITS_H
+
+/*
+ * Protection bit definition
+ * As we don't have any HW to handle page table walk, we can define
+ * our own PTE format. In order to make things easier, we are trying to match
+ * some parts of $tel and $teh.
+ *
+ * PageSZ must be on bit 10 and 11 because it matches the TEL.PS bits. And
+ * by doing that it is easier in assembly to set the TEL.PS to PageSZ.
+ * In other words, KVX_PAGE_SZ_SHIFT == KVX_SFR_TEL_PS_SHIFT.
+ * It is checked by using a BUILD_BUG_ON() in arch/kvx/mm/tlb.c.
+ *
+ * Huge bit must be somewhere in the first 12 bits to be able to detect it
+ * when reading the PMD entry.
+ *
+ * KV3-1:
+ * +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+ * | 63..23 | 22..13 | 12 | 11..10 | 9 | 8 | 7 | 6 | 5 | 4 | 3..2 | 1 | 0 |
+ * +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+ * PFN Unused S PageSZ H G X W R D CP A P
+ *
+ * Note: PFN is 40-bits wide. We use 41-bits to ensure that the upper bit is
+ * always set to 0. This is required when shifting PFN to right.
+ */
+
+/* Following shift are used in ASM to easily extract bit */
+#define _PAGE_PERMS_SHIFT 5
+#define _PAGE_GLOBAL_SHIFT 8
+#define _PAGE_HUGE_SHIFT 9
+
+#define _PAGE_PRESENT (1 << 0) /* Present */
+#define _PAGE_ACCESSED (1 << 1) /* Set by tlb refill code on any access */
+/* Bits 2 - 3 reserved for cache policy */
+#define _PAGE_DIRTY (1 << 4) /* Set by tlb refill code on any write */
+#define _PAGE_READ (1 << _PAGE_PERMS_SHIFT) /* Readable */
+#define _PAGE_WRITE (1 << 6) /* Writable */
+#define _PAGE_EXEC (1 << 7) /* Executable */
+#define _PAGE_GLOBAL (1 << _PAGE_GLOBAL_SHIFT) /* Global */
+#define _PAGE_HUGE (1 << _PAGE_HUGE_SHIFT) /* Huge page */
+/* Bits 10 - 11 reserved for page size */
+#define _PAGE_SOFT (1 << 12) /* Reserved for software */
+#define _PAGE_SZ_64K (TLB_PS_64K << KVX_PAGE_SZ_SHIFT)
+#define _PAGE_SZ_2M (TLB_PS_2M << KVX_PAGE_SZ_SHIFT)
+#define _PAGE_SZ_512M (TLB_PS_512M << KVX_PAGE_SZ_SHIFT)
+
+
+#define _PAGE_SPECIAL _PAGE_SOFT
+
+/*
+ * If _PAGE_PRESENT is clear because the user mapped it with PROT_NONE
+ * pte_present still gives true. Bit[15] of the PTE is used since its unused
+ * for a PTE entry for kv3-1 (see above)
+ */
+#define _PAGE_NONE (1 << 15)
+
+/* Note: mask used in assembly cannot be generated with GENMASK */
+#define KVX_PFN_SHIFT 23
+#define KVX_PFN_MASK (~(((1 << KVX_PFN_SHIFT) - 1)))
+
+#define KVX_PAGE_SZ_SHIFT 10
+#define KVX_PAGE_SZ_MASK KVX_SFR_TEL_PS_MASK
+
+/* Huge page of 64K are hold in PTE table */
+#define KVX_PAGE_64K_NR_CONT (1UL << (KVX_PAGE_64K_SHIFT - PAGE_SHIFT))
+/* Huge page of 512M are hold in PMD table */
+#define KVX_PAGE_512M_NR_CONT (1UL << (KVX_PAGE_512M_SHIFT - PMD_SHIFT))
+
+#define KVX_PAGE_CP_SHIFT 2
+#define KVX_PAGE_CP_MASK KVX_SFR_TEL_CP_MASK
+
+
+#define _PAGE_CACHED (TLB_CP_W_C << KVX_PAGE_CP_SHIFT)
+#define _PAGE_UNCACHED (TLB_CP_U_U << KVX_PAGE_CP_SHIFT)
+#define _PAGE_DEVICE (TLB_CP_D_U << KVX_PAGE_CP_SHIFT)
+
+#define KVX_ACCESS_PERMS_BITS 4
+#define KVX_ACCESS_PERMS_OFFSET _PAGE_PERMS_SHIFT
+#define KVX_ACCESS_PERMS_SIZE (1 << KVX_ACCESS_PERMS_BITS)
+
+#define KVX_ACCESS_PERM_START_BIT KVX_ACCESS_PERMS_OFFSET
+#define KVX_ACCESS_PERM_STOP_BIT \
+ (KVX_ACCESS_PERMS_OFFSET + KVX_ACCESS_PERMS_BITS - 1)
+#define KVX_ACCESS_PERMS_MASK \
+ GENMASK(KVX_ACCESS_PERM_STOP_BIT, KVX_ACCESS_PERM_START_BIT)
+#define KVX_ACCESS_PERMS_INDEX(x) \
+ ((unsigned int)(x & KVX_ACCESS_PERMS_MASK) >> KVX_ACCESS_PERMS_OFFSET)
+
+/* Bits read, write, exec and global are not preserved across pte_modify() */
+#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_READ | _PAGE_WRITE | \
+ _PAGE_EXEC | _PAGE_GLOBAL))
+
+#endif
diff --git a/arch/kvx/include/asm/pgtable.h b/arch/kvx/include/asm/pgtable.h
new file mode 100644
index 000000000000..4d3a8af7d174
--- /dev/null
+++ b/arch/kvx/include/asm/pgtable.h
@@ -0,0 +1,451 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Marius Gligor
+ * Yann Sionneau
+ */
+
+#ifndef _ASM_KVX_PGTABLE_H
+#define _ASM_KVX_PGTABLE_H
+
+#include <linux/mmzone.h>
+#include <linux/mm_types.h>
+
+#include <asm/page.h>
+#include <asm/pgtable-bits.h>
+
+#include <asm-generic/pgtable-nopud.h>
+
+#include <asm/mem_map.h>
+
+struct mm_struct;
+struct vm_area_struct;
+
+/*
+ * Hugetlb definitions. All sizes are supported (64 KB, 2 MB and 512 MB).
+ */
+#if defined(CONFIG_KVX_4K_PAGES)
+#define HUGE_MAX_HSTATE 3
+#elif defined(CONFIG_KVX_64K_PAGES)
+#define HUGE_MAX_HSTATE 2
+#else
+#error "Unsupported page size"
+#endif
+
+#define HPAGE_SHIFT PMD_SHIFT
+#define HPAGE_SIZE BIT(HPAGE_SHIFT)
+#define HPAGE_MASK (~(HPAGE_SIZE - 1))
+#define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT)
+
+extern pte_t arch_make_huge_pte(pte_t entry, unsigned int shift,
+ vm_flags_t flags);
+#define arch_make_huge_pte arch_make_huge_pte
+
+/* Vmalloc definitions */
+#define VMALLOC_START KERNEL_VMALLOC_MAP_BASE
+#define VMALLOC_END (VMALLOC_START + KERNEL_VMALLOC_MAP_SIZE - 1)
+
+/* Also used by GDB script to go through the page table */
+#define PGDIR_BITS (VA_MAX_BITS - PGDIR_SHIFT)
+#define PMD_BITS (PGDIR_SHIFT - PMD_SHIFT)
+#define PTE_BITS (PMD_SHIFT - PAGE_SHIFT)
+
+/* Size of region mapped by a page global directory */
+#define PGDIR_SIZE BIT(PGDIR_SHIFT)
+#define PGDIR_MASK (~(PGDIR_SIZE - 1))
+
+/* Size of region mapped by a page middle directory */
+#define PMD_SIZE BIT(PMD_SHIFT)
+#define PMD_MASK (~(PMD_SIZE - 1))
+
+/* Number of entries in the page global directory */
+#define PAGES_PER_PGD 2
+#define PTRS_PER_PGD (PAGES_PER_PGD * PAGE_SIZE / sizeof(pgd_t))
+
+/* Number of entries in the page middle directory */
+#define PTRS_PER_PMD (PAGE_SIZE / sizeof(pmd_t))
+
+/* Number of entries in the page table */
+#define PTRS_PER_PTE (PAGE_SIZE / sizeof(pte_t))
+
+#define USER_PTRS_PER_PGD (TASK_SIZE/PGDIR_SIZE)
+
+extern pgd_t swapper_pg_dir[PTRS_PER_PGD];
+
+/* Page protection bits */
+#define _PAGE_BASE (_PAGE_PRESENT | _PAGE_CACHED)
+#define _PAGE_KERNEL (_PAGE_PRESENT | _PAGE_GLOBAL | \
+ _PAGE_READ | _PAGE_WRITE)
+#define _PAGE_KERNEL_EXEC (_PAGE_BASE | _PAGE_READ | _PAGE_EXEC | \
+ _PAGE_GLOBAL | _PAGE_WRITE)
+#define _PAGE_KERNEL_DEVICE (_PAGE_KERNEL | _PAGE_DEVICE)
+#define _PAGE_KERNEL_NOCACHE (_PAGE_KERNEL | _PAGE_UNCACHED)
+
+#define PAGE_NONE __pgprot(_PAGE_NONE)
+#define PAGE_READ __pgprot(_PAGE_BASE | _PAGE_READ)
+#define PAGE_READ_WRITE __pgprot(_PAGE_BASE | _PAGE_READ | _PAGE_WRITE)
+#define PAGE_READ_EXEC __pgprot(_PAGE_BASE | _PAGE_READ | _PAGE_EXEC)
+#define PAGE_READ_WRITE_EXEC __pgprot(_PAGE_BASE | _PAGE_READ | \
+ _PAGE_EXEC | _PAGE_WRITE)
+
+#define PAGE_KERNEL __pgprot(_PAGE_KERNEL | _PAGE_CACHED)
+#define PAGE_KERNEL_EXEC __pgprot(_PAGE_KERNEL_EXEC)
+#define PAGE_KERNEL_NOCACHE __pgprot(_PAGE_KERNEL | _PAGE_UNCACHED)
+#define PAGE_KERNEL_DEVICE __pgprot(_PAGE_KERNEL_DEVICE)
+#define PAGE_KERNEL_RO __pgprot((_PAGE_KERNEL | _PAGE_CACHED) & ~(_PAGE_WRITE))
+#define PAGE_KERNEL_ROX __pgprot(_PAGE_KERNEL_EXEC & ~(_PAGE_WRITE))
+
+#define pgprot_noncached(prot) (__pgprot((pgprot_val(prot) & ~KVX_PAGE_CP_MASK) | _PAGE_UNCACHED))
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern struct page *empty_zero_page;
+#define ZERO_PAGE(vaddr) (empty_zero_page)
+
+
+/*
+ * Encode and decode a swap entry. Swap entries are encoded in an arch
+ * dependent format as follows:
+ *
+ * +--------+----+-------+------+---+---+
+ * | 63..16 | 15 | 14..7 | 6..2 | 1 | 0 |
+ * +--------+----+-------+------+---+---+
+ * offset 0 0 type 0 0
+ *
+ * This allows for up to 31 swap files and 1PB per swap file.
+ */
+#define __SWP_TYPE_SHIFT 2
+#define __SWP_TYPE_BITS 5
+#define __SWP_TYPE_MASK ((1UL << __SWP_TYPE_BITS) - 1)
+#define __SWP_OFFSET_SHIFT 16
+
+#define MAX_SWAPFILES_CHECK() \
+ BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > __SWP_TYPE_BITS)
+
+#define __swp_type(x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK)
+#define __swp_offset(x) ((x).val >> __SWP_OFFSET_SHIFT)
+#define __swp_entry(type, offset) ((swp_entry_t) \
+ { ((type) << __SWP_TYPE_SHIFT) | ((offset) << __SWP_OFFSET_SHIFT) })
+
+#define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) })
+#define __swp_entry_to_pte(x) ((pte_t) { (x).val })
+
+/*
+ * PGD definitions:
+ * - pgd_ERROR
+ */
+#define pgd_ERROR(e) \
+ pr_err("%s:%d: bad pgd %016lx.\n", __FILE__, __LINE__, pgd_val(e))
+
+/*
+ * PUD
+ *
+ * As we manage a three level page table the call to set_pud is used to fill
+ * PGD.
+ */
+static inline void set_pud(pud_t *pudp, pud_t pmd)
+{
+ *pudp = pmd;
+}
+
+static inline int pud_none(pud_t pud)
+{
+ return pud_val(pud) == 0;
+}
+
+static inline int pud_bad(pud_t pud)
+{
+ return pud_none(pud);
+}
+static inline int pud_present(pud_t pud)
+{
+ return pud_val(pud) != 0;
+}
+
+static inline void pud_clear(pud_t *pud)
+{
+ set_pud(pud, __pud(0));
+}
+
+/*
+ * PMD definitions:
+ * - set_pmd
+ * - pmd_present
+ * - pmd_none
+ * - pmd_bad
+ * - pmd_clear
+ * - pmd_page
+ */
+
+static inline void set_pmd(pmd_t *pmdp, pmd_t pmd)
+{
+ *pmdp = pmd;
+}
+
+/* Returns 1 if entry is present */
+static inline int pmd_present(pmd_t pmd)
+{
+ return pmd_val(pmd) != 0;
+}
+
+/* Returns 1 if the corresponding entry has the value 0 */
+static inline int pmd_none(pmd_t pmd)
+{
+ return pmd_val(pmd) == 0;
+}
+
+/* Used to check that a page middle directory entry is valid */
+static inline int pmd_bad(pmd_t pmd)
+{
+ return pmd_none(pmd);
+}
+
+/* Clears the entry to prevent process to use the linear address that
+ * mapped it.
+ */
+static inline void pmd_clear(pmd_t *pmdp)
+{
+ set_pmd(pmdp, __pmd(0));
+}
+
+/*
+ * Returns the address of the descriptor of the page table referred by the
+ * PMD entry.
+ */
+static inline struct page *pmd_page(pmd_t pmd)
+{
+ if (pmd_val(pmd) & _PAGE_HUGE)
+ return pfn_to_page(
+ (pmd_val(pmd) & KVX_PFN_MASK) >> KVX_PFN_SHIFT);
+
+ return pfn_to_page(pmd_val(pmd) >> PAGE_SHIFT);
+}
+
+#define pmd_ERROR(e) \
+ pr_err("%s:%d: bad pmd %016lx.\n", __FILE__, __LINE__, pmd_val(e))
+
+static inline pmd_t *pud_pgtable(pud_t pud)
+{
+ return (pmd_t *)pfn_to_virt(pud_val(pud) >> PAGE_SHIFT);
+}
+
+static inline struct page *pud_page(pud_t pud)
+{
+ return pfn_to_page(pud_val(pud) >> PAGE_SHIFT);
+}
+
+/*
+ * PTE definitions:
+ * - set_pte
+ * - set_pte_at
+ * - pte_clear
+ * - pte_page
+ * - pte_pfn
+ * - pte_present
+ * - pte_none
+ * - pte_write
+ * - pte_dirty
+ * - pte_young
+ * - pte_special
+ * - pte_mkdirty
+ * - pte_mkwrite
+ * - pte_mkclean
+ * - pte_mkyoung
+ * - pte_mkold
+ * - pte_mkspecial
+ * - pte_wrprotect
+ */
+
+static inline void set_pte(pte_t *ptep, pte_t pteval)
+{
+ *ptep = pteval;
+}
+
+static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pteval)
+{
+ set_pte(ptep, pteval);
+}
+
+#define pte_clear(mm, addr, ptep) set_pte(ptep, __pte(0))
+
+/* Constructs a page table entry */
+static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot)
+{
+ return __pte(((pfn << KVX_PFN_SHIFT) & KVX_PFN_MASK) |
+ pgprot_val(prot));
+}
+
+/* Builds a page table entry by combining a page descriptor and a group of
+ * access rights.
+ */
+#define mk_pte(page, prot) (pfn_pte(page_to_pfn(page), prot))
+
+/* Modifies page access rights */
+static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
+{
+ return __pte((pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot));
+}
+
+#define pte_page(x) pfn_to_page(pte_pfn(x))
+
+static inline unsigned long pmd_page_vaddr(pmd_t pmd)
+{
+ return (unsigned long)pfn_to_virt(pmd_val(pmd) >> PAGE_SHIFT);
+}
+
+/* Yields the page frame number (PFN) of a page table entry */
+static inline unsigned long pte_pfn(pte_t pte)
+{
+ return ((pte_val(pte) & KVX_PFN_MASK) >> KVX_PFN_SHIFT);
+}
+
+static inline int pte_present(pte_t pte)
+{
+ return (pte_val(pte) & (_PAGE_PRESENT | _PAGE_NONE));
+}
+
+static inline int pte_none(pte_t pte)
+{
+ return (pte_val(pte) == 0);
+}
+
+static inline int pte_write(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_WRITE;
+}
+
+static inline int pte_dirty(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_DIRTY;
+}
+
+static inline int pte_young(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_ACCESSED;
+}
+
+static inline int pte_special(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_SPECIAL;
+}
+
+static inline int pte_huge(pte_t pte)
+{
+ return pte_val(pte) & _PAGE_HUGE;
+}
+
+static inline pte_t pte_mkdirty(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_DIRTY);
+}
+
+static inline pte_t pte_mkwrite(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_WRITE);
+}
+
+static inline pte_t pte_mkclean(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~(_PAGE_DIRTY));
+}
+
+static inline pte_t pte_mkyoung(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_ACCESSED);
+}
+
+static inline pte_t pte_mkold(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~(_PAGE_ACCESSED));
+}
+
+static inline pte_t pte_mkspecial(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_SPECIAL);
+}
+
+static inline pte_t pte_wrprotect(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~(_PAGE_WRITE));
+}
+
+static inline pte_t pte_mkhuge(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_HUGE);
+}
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+
+#define pmdp_establish pmdp_establish
+static inline pmd_t pmdp_establish(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp, pmd_t pmd)
+{
+ return __pmd(xchg(&pmd_val(*pmdp), pmd_val(pmd)));
+}
+
+static inline int pmd_trans_huge(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_HUGE);
+}
+
+static inline pte_t pte_of_pmd(pmd_t pmd)
+{
+ return __pte(pmd_val(pmd));
+}
+
+static inline pmd_t pmd_of_pte(pte_t pte)
+{
+ return __pmd(pte_val(pte));
+}
+
+
+#define pmd_mkclean(pmd) pmd_of_pte(pte_mkclean(pte_of_pmd(pmd)))
+#define pmd_mkdirty(pmd) pmd_of_pte(pte_mkdirty(pte_of_pmd(pmd)))
+#define pmd_mkold(pmd) pmd_of_pte(pte_mkold(pte_of_pmd(pmd)))
+#define pmd_mkwrite(pmd) pmd_of_pte(pte_mkwrite(pte_of_pmd(pmd)))
+#define pmd_mkyoung(pmd) pmd_of_pte(pte_mkyoung(pte_of_pmd(pmd)))
+#define pmd_modify(pmd, prot) pmd_of_pte(pte_modify(pte_of_pmd(pmd), prot))
+#define pmd_wrprotect(pmd) pmd_of_pte(pte_wrprotect(pte_of_pmd(pmd)))
+
+static inline pmd_t pmd_mkhuge(pmd_t pmd)
+{
+ /* Create a huge page in PMD implies a size of 2 MB */
+ return __pmd(pmd_val(pmd) |
+ _PAGE_HUGE | (TLB_PS_2M << KVX_PAGE_SZ_SHIFT));
+}
+
+static inline pmd_t pmd_mkinvalid(pmd_t pmd)
+{
+ pmd_val(pmd) &= ~(_PAGE_PRESENT);
+
+ return pmd;
+}
+
+#define pmd_dirty(pmd) pte_dirty(pte_of_pmd(pmd))
+#define pmd_write(pmd) pte_write(pte_of_pmd(pmd))
+#define pmd_young(pmd) pte_young(pte_of_pmd(pmd))
+
+#define mk_pmd(page, prot) pmd_of_pte(mk_pte(page, prot))
+
+#define pmd_pfn(pmd) pte_pfn(pte_of_pmd(pmd))
+
+static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
+{
+ return __pmd(((pfn << KVX_PFN_SHIFT) & KVX_PFN_MASK) |
+ pgprot_val(prot));
+}
+
+static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, pmd_t pmd)
+{
+ *pmdp = pmd;
+}
+
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
+#endif /* _ASM_KVX_PGTABLE_H */
diff --git a/arch/kvx/include/asm/rm_fw.h b/arch/kvx/include/asm/rm_fw.h
new file mode 100644
index 000000000000..f89bdd5915ed
--- /dev/null
+++ b/arch/kvx/include/asm/rm_fw.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_RM_FW_H
+#define _ASM_KVX_RM_FW_H
+
+#include <linux/sizes.h>
+
+#define KVX_RM_ID 16
+
+#define RM_FIRMWARE_REGS_SIZE (SZ_4K)
+
+#endif /* _ASM_KVX_RM_FW_H */
diff --git a/arch/kvx/include/asm/sparsemem.h b/arch/kvx/include/asm/sparsemem.h
new file mode 100644
index 000000000000..2f35743f20fb
--- /dev/null
+++ b/arch/kvx/include/asm/sparsemem.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SPARSEMEM_H
+#define _ASM_KVX_SPARSEMEM_H
+
+#ifdef CONFIG_SPARSEMEM
+#define MAX_PHYSMEM_BITS 40
+#define SECTION_SIZE_BITS 30
+#endif /* CONFIG_SPARSEMEM */
+
+#endif /* _ASM_KVX_SPARSEMEM_H */
diff --git a/arch/kvx/include/asm/symbols.h b/arch/kvx/include/asm/symbols.h
new file mode 100644
index 000000000000..a53c1607979f
--- /dev/null
+++ b/arch/kvx/include/asm/symbols.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SYMBOLS_H
+#define _ASM_KVX_SYMBOLS_H
+
+/* Symbols to patch TLB refill handler */
+extern char kvx_perf_tlb_refill[], kvx_std_tlb_refill[];
+
+/* Entry point of the ELF, used to start other PEs in SMP */
+extern int kvx_start[];
+
+#endif
diff --git a/arch/kvx/include/asm/tlb.h b/arch/kvx/include/asm/tlb.h
new file mode 100644
index 000000000000..190b682e1819
--- /dev/null
+++ b/arch/kvx/include/asm/tlb.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_TLB_H
+#define _ASM_KVX_TLB_H
+
+struct mmu_gather;
+
+static void tlb_flush(struct mmu_gather *tlb);
+
+int clear_ltlb_entry(unsigned long vaddr);
+
+#include <asm-generic/tlb.h>
+
+static inline unsigned int pgprot_cache_policy(unsigned long flags)
+{
+ return (flags & KVX_PAGE_CP_MASK) >> KVX_PAGE_CP_SHIFT;
+}
+
+#endif /* _ASM_KVX_TLB_H */
diff --git a/arch/kvx/include/asm/tlb_defs.h b/arch/kvx/include/asm/tlb_defs.h
new file mode 100644
index 000000000000..3f5b29cd529e
--- /dev/null
+++ b/arch/kvx/include/asm/tlb_defs.h
@@ -0,0 +1,131 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julian Vetter
+ * Guillaume Thouvenin
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_TLB_DEFS_H
+#define _ASM_KVX_TLB_DEFS_H
+
+#include <linux/sizes.h>
+
+#include <asm/sfr.h>
+
+/* Architecture specification */
+#define MMC_SB_JTLB 0
+#define MMC_SB_LTLB 1
+
+#define MMU_LTLB_SETS 1
+#define MMU_LTLB_WAYS 16
+
+#define MMU_JTLB_SETS 64
+#define MMU_JTLB_WAYS_SHIFT 2
+#define MMU_JTLB_WAYS (1 << MMU_JTLB_WAYS_SHIFT)
+
+#define MMU_JTLB_ENTRIES (MMU_JTLB_SETS << MMU_JTLB_WAYS_SHIFT)
+
+/* Set is determined using the 6 lsb of virtual page */
+#define MMU_JTLB_SET_MASK (MMU_JTLB_SETS - 1)
+#define MMU_JTLB_WAY_MASK (MMU_JTLB_WAYS - 1)
+
+/* TLB: Entry Status */
+#define TLB_ES_INVALID 0
+#define TLB_ES_PRESENT 1
+#define TLB_ES_MODIFIED 2
+#define TLB_ES_A_MODIFIED 3
+
+/* TLB: Cache Policy - First value is for data, the second is for instruction
+ * Symbols are
+ * D: device
+ * U: uncached
+ * W: write through
+ * C: cache enabled
+ */
+#define TLB_CP_D_U 0
+#define TLB_CP_U_U 1
+#define TLB_CP_W_C 2
+#define TLB_CP_U_C 3
+
+/* TLB: Protection Attributes: First value is when PM=0, second is when PM=1
+ * Symbols are:
+ * NA: no access
+ * R : read
+ * W : write
+ * X : execute
+ */
+#define TLB_PA_NA_NA 0
+#define TLB_PA_NA_R 1
+#define TLB_PA_NA_RW 2
+#define TLB_PA_NA_RX 3
+#define TLB_PA_NA_RWX 4
+#define TLB_PA_R_R 5
+#define TLB_PA_R_RW 6
+#define TLB_PA_R_RX 7
+#define TLB_PA_R_RWX 8
+#define TLB_PA_RW_RW 9
+#define TLB_PA_RW_RWX 10
+#define TLB_PA_RX_RX 11
+#define TLB_PA_RX_RWX 12
+#define TLB_PA_RWX_RWX 13
+
+/* TLB: Page Size */
+#define TLB_PS_4K 0
+#define TLB_PS_64K 1
+#define TLB_PS_2M 2
+#define TLB_PS_512M 3
+
+#define TLB_G_GLOBAL 1
+#define TLB_G_USE_ASN 0
+
+#define TLB_MK_TEH_ENTRY(_vaddr, _vs, _global, _asn) \
+ (((_vs) << KVX_SFR_TEH_VS_SHIFT) | \
+ ((_global) << KVX_SFR_TEH_G_SHIFT) | \
+ ((_asn) << KVX_SFR_TEH_ASN_SHIFT) | \
+ (((_vaddr) >> KVX_SFR_TEH_PN_SHIFT) << KVX_SFR_TEH_PN_SHIFT))
+
+#define TLB_MK_TEL_ENTRY(_paddr, _ps, _es, _cp, _pa) \
+ (((_es) << KVX_SFR_TEL_ES_SHIFT) | \
+ ((_ps) << KVX_SFR_TEL_PS_SHIFT) | \
+ ((_cp) << KVX_SFR_TEL_CP_SHIFT) | \
+ ((_pa) << KVX_SFR_TEL_PA_SHIFT) | \
+ (((_paddr) >> KVX_SFR_TEL_FN_SHIFT) << KVX_SFR_TEL_FN_SHIFT))
+
+
+/* Refill routine related defines */
+#define REFILL_PERF_ENTRIES 4
+#define REFILL_PERF_PAGE_SIZE SZ_512M
+
+/* paddr will be inserted in assembly code */
+#define REFILL_PERF_TEL_VAL \
+ TLB_MK_TEL_ENTRY(0, TLB_PS_512M, TLB_ES_A_MODIFIED, TLB_CP_W_C, \
+ TLB_PA_NA_RWX)
+/* vaddr will be inserted in assembly code */
+#define REFILL_PERF_TEH_VAL TLB_MK_TEH_ENTRY(0, 0, TLB_G_GLOBAL, 0)
+
+/*
+ * LTLB fixed entry index
+ */
+#define LTLB_ENTRY_KERNEL_TEXT 0
+#define LTLB_ENTRY_GDB_PAGE 1
+/* Reserve entries for kernel pagination */
+#define LTLB_KERNEL_RESERVED 2
+/* This define should reflect the maximum number of fixed LTLB entries */
+#define LTLB_ENTRY_FIXED_COUNT (LTLB_KERNEL_RESERVED + REFILL_PERF_ENTRIES)
+#define LTLB_ENTRY_EARLY_SMEM LTLB_ENTRY_FIXED_COUNT
+
+/* MMC: Protection Trap Cause */
+#define MMC_PTC_RESERVED 0
+#define MMC_PTC_READ 1
+#define MMC_PTC_WRITE 2
+#define MMC_PTC_EXECUTE 3
+
+/* MMC: Page size Mask in JTLB */
+#define MMC_PMJ_4K 1
+#define MMC_PMJ_64K 2
+#define MMC_PMJ_2M 4
+#define MMC_PMJ_512M 8
+
+#endif
diff --git a/arch/kvx/include/asm/tlbflush.h b/arch/kvx/include/asm/tlbflush.h
new file mode 100644
index 000000000000..7d3e50bbc957
--- /dev/null
+++ b/arch/kvx/include/asm/tlbflush.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_TLBFLUSH_H
+#define _ASM_KVX_TLBFLUSH_H
+
+#include <linux/sched.h>
+#include <linux/printk.h>
+#include <linux/mm_types.h>
+
+extern void local_flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long addr);
+extern void local_flush_tlb_all(void);
+extern void local_flush_tlb_mm(struct mm_struct *mm);
+extern void local_flush_tlb_range(struct vm_area_struct *vma,
+ unsigned long start,
+ unsigned long end);
+extern void local_flush_tlb_kernel_range(unsigned long start,
+ unsigned long end);
+
+#ifdef CONFIG_SMP
+extern void smp_flush_tlb_all(void);
+extern void smp_flush_tlb_mm(struct mm_struct *mm);
+extern void smp_flush_tlb_page(struct vm_area_struct *vma, unsigned long addr);
+extern void smp_flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end);
+extern void smp_flush_tlb_kernel_range(unsigned long start, unsigned long end);
+
+static inline void flush_tlb(void)
+{
+ smp_flush_tlb_mm(current->mm);
+}
+
+#define flush_tlb_page smp_flush_tlb_page
+#define flush_tlb_all smp_flush_tlb_all
+#define flush_tlb_mm smp_flush_tlb_mm
+#define flush_tlb_range smp_flush_tlb_range
+#define flush_tlb_kernel_range smp_flush_tlb_kernel_range
+
+#else
+#define flush_tlb_page local_flush_tlb_page
+#define flush_tlb_all local_flush_tlb_all
+#define flush_tlb_mm local_flush_tlb_mm
+#define flush_tlb_range local_flush_tlb_range
+#define flush_tlb_kernel_range local_flush_tlb_kernel_range
+#endif
+
+void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
+ pmd_t *pmd);
+
+void update_mmu_cache(struct vm_area_struct *vma,
+ unsigned long address, pte_t *ptep);
+
+#endif /* _ASM_KVX_TLBFLUSH_H */
diff --git a/arch/kvx/include/asm/vmalloc.h b/arch/kvx/include/asm/vmalloc.h
new file mode 100644
index 000000000000..a07692431108
--- /dev/null
+++ b/arch/kvx/include/asm/vmalloc.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_VMALLOC_H
+#define _ASM_KVX_VMALLOC_H
+
+#endif /* _ASM_KVX_VMALLOC_H */
diff --git a/arch/kvx/mm/cacheflush.c b/arch/kvx/mm/cacheflush.c
new file mode 100644
index 000000000000..2a632645f980
--- /dev/null
+++ b/arch/kvx/mm/cacheflush.c
@@ -0,0 +1,154 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/smp.h>
+#include <linux/hugetlb.h>
+#include <linux/mm_types.h>
+
+#include <asm/cacheflush.h>
+
+#ifdef CONFIG_SMP
+
+struct flush_data {
+ unsigned long start;
+ unsigned long end;
+};
+
+static inline void ipi_flush_icache_range(void *arg)
+{
+ struct flush_data *ta = arg;
+
+ local_flush_icache_range(ta->start, ta->end);
+}
+
+void flush_icache_range(unsigned long start, unsigned long end)
+{
+ struct flush_data data = {
+ .start = start,
+ .end = end
+ };
+
+ /* Then invalidate L1 icache on all cpus */
+ on_each_cpu(ipi_flush_icache_range, &data, 1);
+}
+EXPORT_SYMBOL(flush_icache_range);
+
+#endif /* CONFIG_SMP */
+
+void dcache_wb_inval_phys_range(phys_addr_t addr, unsigned long len, bool wb,
+ bool inval)
+{
+ if (wb && inval) {
+ wbinval_dcache_range(addr, len);
+ } else {
+ if (inval)
+ inval_dcache_range(addr, len);
+ if (wb)
+ wb_dcache_range(addr, len);
+ }
+}
+
+static inline pte_t *get_ptep(struct mm_struct *mm, unsigned long addr)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ pgd = pgd_offset(mm, addr);
+ if (pgd_none(*pgd))
+ return NULL;
+
+ p4d = p4d_offset(pgd, addr);
+ if (p4d_none(*p4d))
+ return NULL;
+
+ pud = pud_offset(p4d, addr);
+ if (pud_none(*pud))
+ return NULL;
+
+ pmd = pmd_offset(pud, addr);
+ if (pmd_none(*pmd))
+ return NULL;
+
+ if (pmd_huge(*pmd)) {
+ pte = (pte_t *) pmd;
+ if (!pte_present(*pte))
+ return NULL;
+
+ return pte;
+ }
+
+ pte = pte_offset_map(pmd, addr);
+ if (!pte_present(*pte))
+ return NULL;
+
+ return pte;
+}
+
+static unsigned long dcache_wb_inval_virt_to_phys(struct vm_area_struct *vma,
+ unsigned long vaddr,
+ unsigned long len,
+ bool wb, bool inval)
+{
+ unsigned long pfn, offset, pgsize;
+ pte_t *ptep;
+
+ ptep = get_ptep(vma->vm_mm, vaddr);
+ if (!ptep) {
+ /*
+ * Since we did not found a matching pte, return needed
+ * length to be aligned on next page boundary
+ */
+ offset = (vaddr & (PAGE_SIZE - 1));
+ return PAGE_SIZE - offset;
+ }
+
+ /* Handle page sizes correctly */
+ pgsize = (pte_val(*ptep) & KVX_PAGE_SZ_MASK) >> KVX_PAGE_SZ_SHIFT;
+ pgsize = (1 << get_page_size_shift(pgsize));
+
+ offset = vaddr & (pgsize - 1);
+ len = min(pgsize - offset, len);
+ pfn = pte_pfn(*ptep);
+
+ dcache_wb_inval_phys_range(PFN_PHYS(pfn) + offset, len, wb, inval);
+
+ return len;
+}
+
+int dcache_wb_inval_virt_range(unsigned long vaddr, unsigned long len, bool wb,
+ bool inval)
+{
+ unsigned long end = vaddr + len;
+ struct vm_area_struct *vma;
+ unsigned long rlen;
+ struct mm_struct *mm = current->mm;
+
+ /* necessary for find_vma */
+ mmap_read_lock(mm);
+
+ /*
+ * Verify that the specified address region actually belongs to this
+ * process.
+ */
+ vma = find_vma(current->mm, vaddr);
+ if (vma == NULL || vaddr < vma->vm_start || vaddr + len > vma->vm_end) {
+ mmap_read_unlock(mm);
+ return -EFAULT;
+ }
+
+ while (vaddr < end) {
+ rlen = dcache_wb_inval_virt_to_phys(vma, vaddr, len, wb, inval);
+ len -= rlen;
+ vaddr += rlen;
+ }
+
+ mmap_read_unlock(mm);
+
+ return 0;
+}
diff --git a/arch/kvx/mm/dma-mapping.c b/arch/kvx/mm/dma-mapping.c
new file mode 100644
index 000000000000..a9829432b974
--- /dev/null
+++ b/arch/kvx/mm/dma-mapping.c
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Jules Maselbas
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/iommu.h>
+
+#include <asm/cacheflush.h>
+
+void arch_dma_prep_coherent(struct page *page, size_t size)
+{
+ unsigned long addr = (unsigned long) page_to_phys(page);
+
+ /* Flush pending data and invalidate pages */
+ wbinval_dcache_range(addr, size);
+}
+
+/**
+ * The implementation of arch should follow the following rules:
+ * map for_cpu for_device unmap
+ * TO_DEV writeback none writeback none
+ * FROM_DEV invalidate invalidate(*) invalidate invalidate(*)
+ * BIDIR writeback invalidate writeback invalidate
+ *
+ * (*) - only necessary if the CPU speculatively prefetches.
+ *
+ * (see https://lkml.org/lkml/2018/5/18/979)
+ */
+void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir)
+{
+ switch (dir) {
+ case DMA_FROM_DEVICE:
+ inval_dcache_range(paddr, size);
+ break;
+
+ case DMA_TO_DEVICE:
+ case DMA_BIDIRECTIONAL:
+ wb_dcache_range(paddr, size);
+ break;
+
+ default:
+ BUG();
+ }
+}
+
+void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
+ enum dma_data_direction dir)
+{
+ switch (dir) {
+ case DMA_TO_DEVICE:
+ break;
+ case DMA_FROM_DEVICE:
+#ifdef CONFIG_L2_CACHE
+ /* kvx does not do speculative loads by itself, however
+ * L2 cache lines are bigger than L1 cache lines, which can
+ * cause a "false sharing" situation where two L1 cache lines
+ * share the same L2 cache line. There is a high chance that
+ * the L2 cache line has already been refilled by a previous
+ * memory access.
+ */
+ inval_dcache_range(paddr, size);
+#endif
+ break;
+
+ case DMA_BIDIRECTIONAL:
+ inval_dcache_range(paddr, size);
+ break;
+
+ default:
+ BUG();
+ }
+}
+
+#ifdef CONFIG_IOMMU_DMA
+void arch_teardown_dma_ops(struct device *dev)
+{
+ dev->dma_ops = NULL;
+}
+#endif /* CONFIG_IOMMU_DMA*/
+
+void arch_setup_dma_ops(struct device *dev,
+ u64 dma_base,
+ u64 size,
+ const struct iommu_ops *iommu,
+ bool coherent)
+{
+ dev->dma_coherent = coherent;
+ if (iommu)
+ iommu_setup_dma_ops(dev, dma_base, size);
+}
diff --git a/arch/kvx/mm/extable.c b/arch/kvx/mm/extable.c
new file mode 100644
index 000000000000..8889a1a90a77
--- /dev/null
+++ b/arch/kvx/mm/extable.c
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/riscv/mm/extable.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+
+#include <linux/extable.h>
+#include <linux/module.h>
+#include <linux/uaccess.h>
+
+int fixup_exception(struct pt_regs *regs)
+{
+ const struct exception_table_entry *fixup;
+
+ fixup = search_exception_tables(regs->spc);
+ if (fixup) {
+ regs->spc = fixup->fixup;
+ return 1;
+ }
+ return 0;
+}
diff --git a/arch/kvx/mm/fault.c b/arch/kvx/mm/fault.c
new file mode 100644
index 000000000000..17f6325f3eab
--- /dev/null
+++ b/arch/kvx/mm/fault.c
@@ -0,0 +1,264 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Yann Sionneau
+ */
+
+#include <linux/types.h>
+#include <linux/uaccess.h>
+#include <linux/kernel.h> // only needed to panic
+#include <linux/printk.h>
+#include <linux/perf_event.h>
+#include <linux/sched.h>
+#include <linux/sched/debug.h>
+#include <linux/sched/signal.h>
+#include <linux/mm.h>
+
+#include <asm/mmu.h>
+#include <asm/traps.h>
+#include <asm/ptrace.h>
+#include <asm/pgtable.h>
+#include <asm/sfr_defs.h>
+#include <asm/current.h>
+#include <asm/tlbflush.h>
+
+static int handle_vmalloc_fault(uint64_t ea)
+{
+ /*
+ * Synchronize this task's top level page-table with
+ * the 'reference' page table.
+ * As we only have 2 or 3 level page table we don't need to
+ * deal with other levels.
+ */
+ unsigned long addr = ea & PAGE_MASK;
+ pgd_t *pgd_k, *pgd;
+ p4d_t *p4d_k, *p4d;
+ pud_t *pud_k, *pud;
+ pmd_t *pmd_k, *pmd;
+ pte_t *pte_k;
+
+ pgd = pgd_offset(current->active_mm, ea);
+ pgd_k = pgd_offset_k(ea);
+ if (!pgd_present(*pgd_k))
+ return 1;
+ set_pgd(pgd, *pgd_k);
+
+ p4d = p4d_offset(pgd, ea);
+ p4d_k = p4d_offset(pgd_k, ea);
+ if (!p4d_present(*p4d_k))
+ return 1;
+
+ pud = pud_offset(p4d, ea);
+ pud_k = pud_offset(p4d_k, ea);
+ if (!pud_present(*pud_k))
+ return 1;
+
+ pmd = pmd_offset(pud, ea);
+ pmd_k = pmd_offset(pud_k, ea);
+ if (!pmd_present(*pmd_k))
+ return 1;
+
+ /* Some other architectures set pmd to synchronize them but
+ * as we just synchronized the pgd we don't see how they can
+ * be different. Maybe we miss something so in case we
+ * put a guard here.
+ */
+ if (pmd_val(*pmd) != pmd_val(*pmd_k))
+ pr_err("%s: pmd not synchronized (0x%lx != 0x%lx)\n",
+ __func__, pmd_val(*pmd), pmd_val(*pmd_k));
+
+ pte_k = pte_offset_kernel(pmd_k, ea);
+ if (!pte_present(*pte_k)) {
+ pr_err("%s: PTE not present for 0x%llx\n",
+ __func__, ea);
+ return 1;
+ }
+
+ /* We refill the TLB now to avoid to take another nomapping
+ * trap.
+ */
+ kvx_mmu_jtlb_add_entry(addr, pte_k, 0);
+
+ return 0;
+}
+
+void do_page_fault(uint64_t es, uint64_t ea, struct pt_regs *regs)
+{
+ struct mm_struct *mm;
+ struct vm_area_struct *vma;
+ unsigned long flags, cause, vma_mask;
+ int code = SEGV_MAPERR;
+ vm_fault_t fault;
+
+ cause = kvx_sfr_field_val(es, ES, RWX);
+
+ /* We fault-in kernel-space virtual memory on-demand. The
+ * 'reference' page table is init_mm.pgd.
+ */
+ if (is_vmalloc_addr((void *) ea) && !user_mode(regs)) {
+ if (handle_vmalloc_fault(ea))
+ goto no_context;
+ return;
+ }
+
+ mm = current->mm;
+
+ /*
+ * If we're in an interrupt or have no user
+ * context, we must not take the fault..
+ */
+ if (unlikely(faulthandler_disabled() || !mm))
+ goto no_context;
+
+ /* By default we retry and fault task can be killed */
+ flags = FAULT_FLAG_DEFAULT;
+
+ if (user_mode(regs))
+ flags |= FAULT_FLAG_USER;
+
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, ea);
+
+retry:
+ mmap_read_lock(mm);
+
+ vma = find_vma(mm, ea);
+ if (!vma)
+ goto bad_area;
+ if (likely(vma->vm_start <= ea))
+ goto good_area;
+ if (unlikely(!(vma->vm_flags & VM_GROWSDOWN)))
+ goto bad_area;
+ if (unlikely(expand_stack(vma, ea)))
+ goto bad_area;
+
+good_area:
+ /* Handle access type */
+ switch (cause) {
+ case KVX_TRAP_RWX_FETCH:
+ vma_mask = VM_EXEC;
+ break;
+ case KVX_TRAP_RWX_READ:
+ vma_mask = VM_READ;
+ break;
+ case KVX_TRAP_RWX_WRITE:
+ vma_mask = VM_WRITE;
+ flags |= FAULT_FLAG_WRITE;
+ break;
+ /* Atomic are both read/write */
+ case KVX_TRAP_RWX_ATOMIC:
+ vma_mask = VM_WRITE | VM_READ;
+ flags |= FAULT_FLAG_WRITE;
+ break;
+ default:
+ panic("%s: unhandled cause %lu", __func__, cause);
+ }
+
+ if ((vma->vm_flags & vma_mask) != vma_mask) {
+ code = SEGV_ACCERR;
+ goto bad_area;
+ }
+
+ /*
+ * If for any reason we can not handle the fault we make sure that
+ * we exit gracefully rather then retry endlessly with the same
+ * result.
+ */
+ fault = handle_mm_fault(vma, ea, flags, regs);
+
+ /*
+ * If we need to retry but a fatal signal is pending, handle the
+ * signal first. We do not need to release the mmap_sem because it
+ * would already be released in __lock_page_or_retry in mm/filemap.c.
+ */
+ if (fault_signal_pending(fault, regs))
+ return;
+
+ if (unlikely(fault & VM_FAULT_ERROR)) {
+ if (fault & VM_FAULT_OOM)
+ goto out_of_memory;
+ else if (fault & VM_FAULT_SIGSEGV)
+ goto bad_area;
+ else if (fault & VM_FAULT_SIGBUS)
+ goto do_sigbus;
+ BUG();
+ }
+
+ if (unlikely((fault & VM_FAULT_RETRY) && (flags & FAULT_FLAG_ALLOW_RETRY))) {
+ flags |= FAULT_FLAG_TRIED;
+ /* No need to up_read(&mm->mmap_sem) as we would
+ * have already released it in __lock_page_or_retry().
+ * Look in mm/filemap.c for explanations.
+ */
+ goto retry;
+ }
+
+ /* Fault errors and retry case have been handled nicely */
+ mmap_read_unlock(mm);
+ return;
+
+bad_area:
+ mmap_read_unlock(mm);
+
+ if (user_mode(regs)) {
+ user_do_sig(regs, SIGSEGV, code, ea);
+ return;
+ }
+
+no_context:
+ /* Are we prepared to handle this kernel fault?
+ *
+ * (The kernel has valid exception-points in the source
+ * when it accesses user-memory. When it fails in one
+ * of those points, we find it in a table and do a jump
+ * to some fixup code that loads an appropriate error
+ * code)
+ */
+ if (fixup_exception(regs))
+ return;
+
+ /*
+ * Oops. The kernel tried to access some bad page. We'll have to
+ * terminate things with extreme prejudice.
+ */
+ bust_spinlocks(1);
+ if (kvx_sfr_field_val(es, ES, HTC) == KVX_TRAP_PROTECTION)
+ pr_alert(CUT_HERE "Kernel protection trap at virtual address %016llx\n",
+ ea);
+ else {
+ pr_alert(CUT_HERE "Unable to handle kernel %s at virtual address %016llx\n",
+ (ea < PAGE_SIZE) ? "NULL pointer dereference" :
+ "paging request", ea);
+ }
+ die(regs, ea, "Oops");
+ bust_spinlocks(0);
+ make_task_dead(SIGKILL);
+
+out_of_memory:
+ /*
+ * We ran out of memory, call the OOM killer, and return the userspace
+ * (which will retry the fault, or kill us if we got oom-killed).
+ */
+ mmap_read_unlock(mm);
+ if (!user_mode(regs))
+ goto no_context;
+ pagefault_out_of_memory();
+ return;
+
+do_sigbus:
+ mmap_read_unlock(mm);
+ /* Kernel mode? Handle exceptions or die */
+ if (!user_mode(regs))
+ goto no_context;
+
+ user_do_sig(regs, SIGBUS, BUS_ADRERR, ea);
+
+ return;
+
+}
+
+void do_writetoclean(uint64_t es, uint64_t ea, struct pt_regs *regs)
+{
+ panic("%s not implemented", __func__);
+}
diff --git a/arch/kvx/mm/hugetlbpage.c b/arch/kvx/mm/hugetlbpage.c
new file mode 100644
index 000000000000..c1291fdac708
--- /dev/null
+++ b/arch/kvx/mm/hugetlbpage.c
@@ -0,0 +1,317 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ * Jean-Christophe Pince
+ */
+
+#include <asm/page.h>
+
+#include <linux/bits.h>
+#include <linux/mm.h>
+#include <linux/hugetlb.h>
+
+/**
+ * get_nr_cont_huge_pages - return the number of contiguous pages
+ * @pte: a PTE
+ *
+ * Return: the number of contiguous entries used by the huge page according to
+ * the huge page size.
+ */
+static unsigned int get_nr_cont_huge_pages(pte_t pte)
+{
+ unsigned int psize, nr_cont;
+ unsigned long ptev = pte_val(pte);
+
+ psize = (ptev & KVX_PAGE_SZ_MASK) >> KVX_PAGE_SZ_SHIFT;
+
+ if (psize == TLB_PS_64K)
+ nr_cont = KVX_PAGE_64K_NR_CONT;
+ else if (psize == TLB_PS_512M)
+ nr_cont = KVX_PAGE_512M_NR_CONT;
+ else
+ /* Only page of 64 KB or 512 MB require more than 1 entry */
+ nr_cont = 1;
+
+ return nr_cont;
+}
+
+void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte)
+{
+ unsigned int nr_cont, i;
+
+ nr_cont = get_nr_cont_huge_pages(pte);
+
+ for (i = 0; i < nr_cont; i++, ptep++)
+ set_pte_at(mm, addr, ptep, pte);
+}
+
+pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+ unsigned long addr,
+ pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+ pte_t pte = huge_ptep_get(ptep);
+
+ nr_cont = get_nr_cont_huge_pages(pte);
+
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ if (pte_dirty(*ptep))
+ pte = pte_mkdirty(pte);
+ if (pte_young(*ptep))
+ pte = pte_mkyoung(pte);
+ pte_clear(mm, addr, ptep);
+ }
+
+ flush_tlb_mm(mm);
+ return pte;
+}
+
+static pte_t get_dirty_young_from_cont(pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+ pte_t pte_orig = huge_ptep_get(ptep);
+
+ nr_cont = get_nr_cont_huge_pages(pte_orig);
+
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ pte_t pte = huge_ptep_get(ptep);
+
+ if (pte_dirty(pte))
+ pte_orig = pte_mkdirty(pte_orig);
+
+ if (pte_young(pte))
+ pte_orig = pte_mkyoung(pte_orig);
+ }
+
+ return pte_orig;
+}
+
+int huge_ptep_set_access_flags(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep,
+ pte_t pte, int dirty)
+{
+ unsigned int nr_cont, i;
+ int flush = 0;
+ pte_t pte_tmp;
+
+ nr_cont = get_nr_cont_huge_pages(huge_ptep_get(ptep));
+
+ /*
+ * As it is done by ARM64, make sure we don't lose the dirty or young
+ * state. So first we read dirty and young bits from all contiguous
+ * PTEs and update the pte given as argument if needed.
+ */
+ pte_tmp = get_dirty_young_from_cont(ptep);
+ if (pte_dirty(pte_tmp))
+ pte = pte_mkdirty(pte);
+
+ if (pte_young(pte_tmp))
+ pte = pte_mkyoung(pte);
+
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ if (!pte_same(*ptep, pte)) {
+ set_pte_at(vma->vm_mm, addr, ptep, pte);
+ flush = 1;
+ }
+ }
+
+ if (flush)
+ flush_tlb_page(vma, addr);
+
+ return flush;
+}
+
+void huge_ptep_set_wrprotect(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+
+ nr_cont = get_nr_cont_huge_pages(huge_ptep_get(ptep));
+
+ for (i = 0; i < nr_cont; i++, ptep++)
+ ptep_set_wrprotect(mm, addr, ptep);
+}
+
+
+pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep)
+{
+ unsigned int nr_cont, i;
+ struct mm_struct *mm;
+ int flush = 0;
+ pte_t pte_orig = huge_ptep_get(ptep);
+
+ nr_cont = get_nr_cont_huge_pages(huge_ptep_get(ptep));
+
+ mm = vma->vm_mm;
+ for (i = 0; i < nr_cont; i++, ptep++) {
+ BUG_ON(pte_orig.pte != pte_val(*ptep));
+ pte_clear(mm, address, ptep);
+ if (pte_accessible(mm, *ptep))
+ flush = 1;
+ }
+
+ if (flush)
+ flush_tlb_page(vma, addr);
+
+ return pte_orig;
+}
+
+pte_t arch_make_huge_pte(pte_t entry, unsigned int shift,
+ vm_flags_t flags)
+{
+ unsigned long ptev;
+
+ ptev = pte_val(entry) & ~(KVX_PAGE_SZ_MASK);
+
+ switch (shift) {
+ case KVX_PAGE_64K_SHIFT:
+ ptev |= _PAGE_SZ_64K;
+ break;
+ case KVX_PAGE_2M_SHIFT:
+ ptev |= _PAGE_SZ_2M;
+ break;
+ case KVX_PAGE_512M_SHIFT:
+ ptev |= _PAGE_SZ_512M;
+ break;
+ default:
+ ptev = 0;
+ pr_err("huge page shift %d not supported\n", shift);
+ BUG();
+ }
+
+ return __pte(ptev);
+}
+
+pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
+ unsigned long addr,
+ unsigned long size)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd = NULL;
+
+ pgd = pgd_offset(mm, addr);
+ if (pgd_present(*pgd)) {
+ p4d = p4d_offset(pgd, addr);
+ if (p4d_present(*p4d)) {
+ pud = pud_offset(p4d, addr);
+ if (pud_present(*pud))
+ pmd = pmd_alloc(mm, pud, addr);
+ }
+ }
+
+ if (size > KVX_PAGE_64K_SIZE)
+ return (pte_t *)pmd;
+
+ return pmd ? pte_alloc_map(mm, pmd, addr) : NULL;
+}
+
+/**
+ * huge_pte_offset - get the offset of the huge page
+ * @mm: the memory structure
+ * @addr: the address
+ * @size: size of the memory area
+ *
+ * On kvx the huge page are backed bye PMD or PTE depending of the size of the
+ * huge page. Huge pages larger or equal to 2 MB are backed by PMD, smaller are
+ * backed by PTE.
+ *
+ * Return: pointer to the huge page if it exists, NULL otherwise.
+ */
+pte_t *huge_pte_offset(struct mm_struct *mm,
+ unsigned long addr,
+ unsigned long size)
+{
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ /*
+ * 64 KB and 512 MB huge pages require the usage of contiguous page so
+ * we need to be aligned for this pages in order to get the right
+ * offset (ie the offset of the first index of contiguous pages).
+ */
+ if (size == KVX_PAGE_64K_SIZE)
+ addr &= KVX_PAGE_64K_MASK;
+ else if (size == KVX_PAGE_512M_SIZE)
+ addr &= KVX_PAGE_512M_MASK;
+
+ pgd = pgd_offset(mm, addr);
+ if (!pgd_present(*pgd))
+ return NULL;
+
+ p4d = p4d_offset(pgd, addr);
+ if (!p4d_present(*p4d))
+ return NULL;
+
+ pud = pud_offset(p4d, addr);
+ if (!pud_present(*pud))
+ return NULL;
+
+ pmd = pmd_offset(pud, addr);
+
+ if (size == KVX_PAGE_64K_SIZE) {
+ /* we need to go deeper in the page table */
+ if (pmd_present(*pmd))
+ pte = pte_offset_kernel(pmd, addr);
+ else
+ pte = NULL;
+ } else {
+ pte = (pte_t *)pmd;
+ }
+
+ return pte;
+}
+
+int pmd_huge(pmd_t pmd)
+{
+ return pmd_val(pmd) & _PAGE_HUGE;
+}
+
+int pud_huge(pud_t pud)
+{
+ return 0;
+}
+
+static void __init add_huge_page_size(unsigned long size)
+{
+ if (size_to_hstate(size)) {
+ WARN(1, "Failed to add huge page size %lu\n", size);
+ return;
+ }
+
+ hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
+}
+
+static int __init hugetlbpage_init(void)
+{
+#if defined(CONFIG_KVX_4K_PAGES)
+ add_huge_page_size(KVX_PAGE_64K_SIZE);
+ add_huge_page_size(KVX_PAGE_2M_SIZE);
+ add_huge_page_size(KVX_PAGE_512M_SIZE);
+#endif
+
+ return 0;
+}
+arch_initcall(hugetlbpage_init);
+
+bool __init arch_hugetlb_valid_size(unsigned long size)
+{
+#if defined(CONFIG_KVX_4K_PAGES)
+ switch (size) {
+ case KVX_PAGE_64K_SIZE:
+ case KVX_PAGE_2M_SIZE:
+ case KVX_PAGE_512M_SIZE:
+ return true;
+ }
+#endif
+
+ return false;
+}
diff --git a/arch/kvx/mm/init.c b/arch/kvx/mm/init.c
new file mode 100644
index 000000000000..abc4620d9cf3
--- /dev/null
+++ b/arch/kvx/mm/init.c
@@ -0,0 +1,527 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+/* Memblock header depends on types.h but does not include it ! */
+#include <linux/types.h>
+#include <linux/memblock.h>
+#include <linux/mmzone.h>
+#include <linux/of_fdt.h>
+#include <linux/sched.h>
+#include <linux/sizes.h>
+#include <linux/init.h>
+#include <linux/initrd.h>
+#include <linux/pfn.h>
+#include <linux/mm.h>
+
+#include <asm/sections.h>
+#include <asm/tlb_defs.h>
+#include <asm/tlbflush.h>
+#include <asm/fixmap.h>
+#include <asm/page.h>
+
+/*
+ * On kvx, memory map contains the first 2G of DDR being aliased.
+ * Full contiguous DDR is located at @[4G - 68G].
+ * However, to access this DDR in 32bit mode, the first 2G of DDR are
+ * mirrored from 4G to 2G.
+ * These first 2G are accessible from all DMAs (included 32 bits one).
+ *
+ * Hence, the memory map is the following:
+ *
+ * (68G) 0x1100000000-> +-------------+
+ * | |
+ * 66G |(ZONE_NORMAL)|
+ * | |
+ * (6G) 0x180000000-> +-------------+
+ * | |
+ * 2G |(ZONE_DMA32) |
+ * | |
+ * (4G) 0x100000000-> +-------------+ +--+
+ * | | |
+ * 2G | (Alias) | | 2G Alias
+ * | | |
+ * (2G) 0x80000000-> +-------------+ <--+
+ *
+ * The translation of 64 bits -> 32 bits can then be done using dma-ranges property
+ * in device-trees.
+ */
+
+#define DDR_64BIT_START (4ULL * SZ_1G)
+#define DDR_32BIT_ALIAS_SIZE (2ULL * SZ_1G)
+
+#define MAX_DMA32_PFN PHYS_PFN(DDR_64BIT_START + DDR_32BIT_ALIAS_SIZE)
+
+pgd_t swapper_pg_dir[PTRS_PER_PGD] __page_aligned_bss;
+
+/*
+ * empty_zero_page is a special page that is used for zero-initialized data and
+ * COW.
+ */
+struct page *empty_zero_page;
+EXPORT_SYMBOL(empty_zero_page);
+
+extern char _start[];
+extern char __kernel_smem_code_start[];
+extern char __kernel_smem_code_end[];
+
+struct kernel_section {
+ phys_addr_t start;
+ phys_addr_t end;
+};
+
+struct kernel_section kernel_sections[] = {
+ {
+ .start = (phys_addr_t)__kernel_smem_code_start,
+ .end = (phys_addr_t)__kernel_smem_code_end
+ },
+ {
+ .start = __pa(_start),
+ .end = __pa(_end)
+ }
+};
+
+static void __init zone_sizes_init(void)
+{
+ unsigned long zones_size[MAX_NR_ZONES];
+
+ memset(zones_size, 0, sizeof(zones_size));
+
+ zones_size[ZONE_DMA32] = min(MAX_DMA32_PFN, max_low_pfn);
+ zones_size[ZONE_NORMAL] = max_low_pfn;
+
+ free_area_init(zones_size);
+}
+
+#ifdef CONFIG_BLK_DEV_INITRD
+static void __init setup_initrd(void)
+{
+ u64 base = phys_initrd_start;
+ u64 end = phys_initrd_start + phys_initrd_size;
+
+ if (phys_initrd_size == 0) {
+ pr_info("initrd not found or empty");
+ return;
+ }
+
+ if (base < memblock_start_of_DRAM() || end > memblock_end_of_DRAM()) {
+ pr_err("initrd not in accessible memory, disabling it");
+ phys_initrd_size = 0;
+ return;
+ }
+
+ pr_info("initrd: 0x%llx - 0x%llx\n", base, end);
+
+ memblock_reserve(phys_initrd_start, phys_initrd_size);
+
+ /* the generic initrd code expects virtual addresses */
+ initrd_start = (unsigned long) __va(base);
+ initrd_end = initrd_start + phys_initrd_size;
+}
+#endif
+
+static phys_addr_t memory_limit = PHYS_ADDR_MAX;
+
+static int __init early_mem(char *p)
+{
+ if (!p)
+ return 1;
+
+ memory_limit = memparse(p, &p) & PAGE_MASK;
+ pr_notice("Memory limited to %lldMB\n", memory_limit >> 20);
+
+ return 0;
+}
+early_param("mem", early_mem);
+
+static void __init setup_bootmem(void)
+{
+ phys_addr_t kernel_start, kernel_end;
+ phys_addr_t start, end = 0;
+ u64 i;
+
+ init_mm.start_code = (unsigned long)_stext;
+ init_mm.end_code = (unsigned long)_etext;
+ init_mm.end_data = (unsigned long)_edata;
+ init_mm.brk = (unsigned long)_end;
+
+ for (i = 0; i < ARRAY_SIZE(kernel_sections); i++) {
+ kernel_start = kernel_sections[i].start;
+ kernel_end = kernel_sections[i].end;
+
+ memblock_reserve(kernel_start, kernel_end - kernel_start);
+ }
+
+ for_each_mem_range(i, &start, &end) {
+ pr_info("%15s: memory : 0x%lx - 0x%lx\n", __func__,
+ (unsigned long)start,
+ (unsigned long)end);
+ }
+
+ /* min_low_pfn is the lowest PFN available in the system */
+ min_low_pfn = PFN_UP(memblock_start_of_DRAM());
+
+ /* max_low_pfn indicates the end if NORMAL zone */
+ max_low_pfn = PFN_DOWN(memblock_end_of_DRAM());
+
+ /* Set the maximum number of pages in the system */
+ set_max_mapnr(max_low_pfn - min_low_pfn);
+
+#ifdef CONFIG_BLK_DEV_INITRD
+ setup_initrd();
+#endif
+
+ if (memory_limit != PHYS_ADDR_MAX)
+ memblock_mem_limit_remove_map(memory_limit);
+
+ /* Don't reserve the device tree if its builtin */
+ if (!is_kernel_rodata((unsigned long) initial_boot_params))
+ early_init_fdt_reserve_self();
+ early_init_fdt_scan_reserved_mem();
+
+ memblock_allow_resize();
+ memblock_dump_all();
+}
+
+static void * __init alloc_page_table(void)
+{
+ void *pgt;
+
+ pgt = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+ BUG_ON(!pgt);
+
+ return pgt;
+}
+
+static pmd_t fixmap_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused;
+static pte_t fixmap_pte[PTRS_PER_PTE] __page_aligned_bss __maybe_unused;
+
+void __init early_fixmap_init(void)
+{
+ unsigned long vaddr;
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+
+ /*
+ * Fixed mappings:
+ */
+ vaddr = __fix_to_virt(__end_of_fixed_addresses - 1);
+ pgd = pgd_offset_pgd(swapper_pg_dir, vaddr);
+ set_pgd(pgd, __pgd(__pa_symbol(fixmap_pmd)));
+
+ p4d = p4d_offset(pgd, vaddr);
+ pud = pud_offset(p4d, vaddr);
+ pmd = pmd_offset(pud, vaddr);
+ set_pmd(pmd, __pmd(__pa_symbol(fixmap_pte)));
+}
+
+#ifdef CONFIG_STRICT_KERNEL_RWX
+
+static bool use_huge_page(unsigned long start, unsigned long end,
+ phys_addr_t phys, unsigned long page_size)
+{
+ unsigned long size = end - start;
+
+ return IS_ALIGNED(start | phys, page_size) && (size >= page_size);
+}
+
+static void create_pte_mapping(pmd_t *pmdp, unsigned long va_start,
+ unsigned long va_end, phys_addr_t phys,
+ pgprot_t prot, bool alloc_pgtable)
+{
+ pmd_t pmd = READ_ONCE(*pmdp);
+ unsigned long nr_cont, i;
+ pte_t *ptep = NULL;
+ pgprot_t pte_prot;
+
+ if (pmd_none(pmd)) {
+ BUG_ON(!alloc_pgtable);
+ ptep = alloc_page_table();
+ set_pmd(pmdp, __pmd(__pa(ptep)));
+ pmd = READ_ONCE(*pmdp);
+ }
+ BUG_ON(pmd_bad(pmd));
+
+ ptep = pte_offset_kernel(pmdp, va_start);
+
+ do {
+ /* Try to use 64K page whenever it is possible */
+ if (use_huge_page(va_start, va_end, phys, KVX_PAGE_64K_SIZE)) {
+ pte_prot = __pgprot(pgprot_val(prot) | _PAGE_SZ_64K |
+ _PAGE_HUGE);
+ nr_cont = KVX_PAGE_64K_NR_CONT;
+ } else {
+ pte_prot = prot;
+ nr_cont = 1;
+ }
+
+ for (i = 0; i < nr_cont; i++) {
+ set_pte(ptep, pfn_pte(phys_to_pfn(phys), pte_prot));
+ ptep++;
+ }
+
+ phys += nr_cont * PAGE_SIZE;
+ va_start += nr_cont * PAGE_SIZE;
+ } while (va_start != va_end);
+}
+
+static void create_pmd_mapping(pgd_t *pgdp, unsigned long va_start,
+ unsigned long va_end, phys_addr_t phys,
+ pgprot_t prot, bool alloc_pgtable)
+{
+ unsigned long next, huge_size, i, nr_cont;
+ pgprot_t pmd_prot;
+ bool use_huge;
+ pmd_t *pmdp;
+ pud_t *pudp;
+ p4d_t *p4dp;
+ pud_t pud;
+ pte_t pte;
+
+ p4dp = p4d_offset(pgdp, va_start);
+ pudp = pud_offset(p4dp, va_start);
+ pud = READ_ONCE(*pudp);
+
+ if (pud_none(pud)) {
+ BUG_ON(!alloc_pgtable);
+ pmdp = alloc_page_table();
+ set_pud(pudp, __pud(__pa(pmdp)));
+ pud = READ_ONCE(*pudp);
+ }
+ BUG_ON(pud_bad(pud));
+
+ pmdp = pmd_offset(pudp, va_start);
+
+ do {
+ next = pmd_addr_end(va_start, va_end);
+
+ /* Try to use huge pages (2M, 512M) whenever it is possible */
+ if (use_huge_page(va_start, next, phys, KVX_PAGE_2M_SIZE)) {
+ pmd_prot = __pgprot(pgprot_val(prot) | _PAGE_SZ_2M);
+ nr_cont = 1;
+ huge_size = KVX_PAGE_2M_SIZE;
+ use_huge = true;
+ } else if (use_huge_page(va_start, next, phys,
+ KVX_PAGE_512M_SIZE)) {
+ pmd_prot = __pgprot(pgprot_val(prot) | _PAGE_SZ_512M);
+ nr_cont = KVX_PAGE_512M_NR_CONT;
+ huge_size = KVX_PAGE_512M_SIZE;
+ use_huge = true;
+ } else {
+ use_huge = false;
+ }
+
+ if (use_huge) {
+ pmd_prot = __pgprot(pgprot_val(pmd_prot) | _PAGE_HUGE);
+ pte = pfn_pte(phys_to_pfn(phys), pmd_prot);
+ for (i = 0; i < nr_cont; i++) {
+ set_pmd(pmdp, __pmd(pte_val(pte)));
+ pmdp++;
+ }
+ } else {
+ create_pte_mapping(pmdp, va_start, next, phys, prot,
+ alloc_pgtable);
+ pmdp++;
+ }
+
+ phys += next - va_start;
+ } while (va_start = next, va_start != va_end);
+}
+
+static void create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys,
+ unsigned long va_start, unsigned long va_end,
+ pgprot_t prot, bool alloc_pgtable)
+{
+ unsigned long next;
+ pgd_t *pgdp = pgd_offset_pgd(pgdir, va_start);
+
+ BUG_ON(!PAGE_ALIGNED(phys));
+ BUG_ON(!PAGE_ALIGNED(va_start));
+ BUG_ON(!PAGE_ALIGNED(va_end));
+
+ do {
+ next = pgd_addr_end(va_start, va_end);
+ create_pmd_mapping(pgdp, va_start, next, phys, prot,
+ alloc_pgtable);
+ phys += next - va_start;
+ } while (pgdp++, va_start = next, va_start != va_end);
+}
+
+static void __init map_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end,
+ pgprot_t prot)
+{
+ phys_addr_t pa_start = __pa(va_start);
+
+ create_pgd_mapping(pgdp, pa_start, (unsigned long) va_start,
+ (unsigned long) va_end, prot, true);
+}
+
+static void remap_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end,
+ pgprot_t prot)
+{
+ phys_addr_t pa_start = __pa(va_start);
+
+ create_pgd_mapping(pgdp, pa_start, (unsigned long) va_start,
+ (unsigned long) va_end, prot, false);
+
+ flush_tlb_kernel_range((unsigned long) va_start,
+ (unsigned long) va_end);
+}
+
+/*
+ * Create fine-grained mappings for the kernel.
+ */
+static void __init map_kernel(void)
+{
+ pgprot_t text_prot = PAGE_KERNEL_ROX;
+
+ /*
+ * External debuggers may need to write directly to the text
+ * mapping to install SW breakpoints. Allow this (only) when
+ * explicitly requested with rodata=off.
+ */
+ if (!rodata_enabled)
+ text_prot = PAGE_KERNEL_EXEC;
+
+ map_kernel_segment(swapper_pg_dir, __inittext_start, __inittext_end,
+ text_prot);
+ map_kernel_segment(swapper_pg_dir, __initdata_start, __initdata_end,
+ PAGE_KERNEL);
+ map_kernel_segment(swapper_pg_dir, __rodata_start, __rodata_end,
+ PAGE_KERNEL);
+ map_kernel_segment(swapper_pg_dir, _sdata, _end, PAGE_KERNEL);
+ /* We skip exceptions mapping to avoid multimappings */
+ map_kernel_segment(swapper_pg_dir, __exception_end, _etext, text_prot);
+}
+
+static void __init map_memory(void)
+{
+ phys_addr_t start, end;
+ phys_addr_t kernel_start = __pa(__inittext_start);
+ phys_addr_t kernel_end = __pa_symbol(_end);
+ u64 i;
+
+ /**
+ * Mark the full kernel text/data as nomap to avoid remapping all
+ * section as RW.
+ */
+ memblock_mark_nomap(kernel_start, kernel_end - kernel_start);
+
+ /* Map all memory banks */
+ for_each_mem_range(i, &start, &end) {
+
+ if (start >= end)
+ break;
+
+ create_pgd_mapping(swapper_pg_dir, start,
+ (unsigned long) __va(start),
+ (unsigned long) __va(end), PAGE_KERNEL_EXEC,
+ true);
+ }
+ memblock_clear_nomap(kernel_start, kernel_end - kernel_start);
+}
+
+void mark_rodata_ro(void)
+{
+ remap_kernel_segment(swapper_pg_dir, __rodata_start, __rodata_end,
+ PAGE_KERNEL_RO);
+}
+
+void __init paging_init(void)
+{
+ map_kernel();
+ map_memory();
+ init_kernel_rwx();
+}
+
+static int __init parse_rodata(char *arg)
+{
+ strtobool(arg, &rodata_enabled);
+
+ return 0;
+}
+early_param("rodata", parse_rodata);
+
+#else
+
+static void remap_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end,
+ pgprot_t prot)
+{
+}
+
+#endif
+
+void __init setup_arch_memory(void)
+{
+ setup_bootmem();
+ sparse_init();
+ zone_sizes_init();
+}
+
+void __init mem_init(void)
+{
+ memblock_free_all();
+
+ /* allocate the zero page */
+ empty_zero_page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+ if (!empty_zero_page)
+ panic("Failed to allocate the empty_zero_page");
+}
+
+void free_initmem(void)
+{
+ /* Remap init text as RW to allow writing to it */
+ remap_kernel_segment(swapper_pg_dir, __inittext_start, __inittext_end,
+ PAGE_KERNEL);
+
+#ifdef CONFIG_POISON_INITMEM
+ free_initmem_default(0x0);
+#else
+ free_initmem_default(-1);
+#endif
+}
+
+void __set_fixmap(enum fixed_addresses idx,
+ phys_addr_t phys, pgprot_t flags)
+{
+ unsigned long addr = __fix_to_virt(idx);
+ pte_t *pte;
+
+
+ BUG_ON(idx >= __end_of_fixed_addresses);
+
+ pte = &fixmap_pte[pte_index(addr)];
+
+ if (pgprot_val(flags)) {
+ set_pte(pte, pfn_pte(phys_to_pfn(phys), flags));
+ } else {
+ /* Remove the fixmap */
+ pte_clear(&init_mm, addr, pte);
+ }
+ local_flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
+}
+
+static const pgprot_t protection_map[16] = {
+ [VM_NONE] = PAGE_NONE,
+ [VM_READ] = PAGE_READ,
+ [VM_WRITE] = PAGE_READ,
+ [VM_WRITE | VM_READ] = PAGE_READ,
+ [VM_EXEC] = PAGE_READ_EXEC,
+ [VM_EXEC | VM_READ] = PAGE_READ_EXEC,
+ [VM_EXEC | VM_WRITE] = PAGE_READ_EXEC,
+ [VM_EXEC | VM_WRITE | VM_READ] = PAGE_READ_EXEC,
+ [VM_SHARED] = PAGE_NONE,
+ [VM_SHARED | VM_READ] = PAGE_READ,
+ [VM_SHARED | VM_WRITE] = PAGE_READ_WRITE,
+ [VM_SHARED | VM_WRITE | VM_READ] = PAGE_READ_WRITE,
+ [VM_SHARED | VM_EXEC] = PAGE_READ_EXEC,
+ [VM_SHARED | VM_EXEC | VM_READ] = PAGE_READ_EXEC,
+ [VM_SHARED | VM_EXEC | VM_WRITE] = PAGE_READ_WRITE_EXEC,
+ [VM_SHARED | VM_EXEC | VM_WRITE | VM_READ] = PAGE_READ_WRITE_EXEC
+};
+DECLARE_VM_GET_PAGE_PROT
diff --git a/arch/kvx/mm/kernel_rwx.c b/arch/kvx/mm/kernel_rwx.c
new file mode 100644
index 000000000000..efa502bab336
--- /dev/null
+++ b/arch/kvx/mm/kernel_rwx.c
@@ -0,0 +1,228 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julian Vetter
+ */
+
+#include <linux/init.h>
+#include <linux/sysfs.h>
+#include <linux/kernel.h>
+#include <linux/device.h>
+#include <linux/kobject.h>
+
+#include <asm/mmu.h>
+#include <asm/tlb.h>
+#include <asm/insns.h>
+#include <asm/sections.h>
+#include <asm/symbols.h>
+#include <asm/insns_defs.h>
+
+#define PERF_REFILL_INSN_SIZE (KVX_INSN_GOTO_SIZE * KVX_INSN_SYLLABLE_WIDTH)
+
+struct kobject *kvx_kobj;
+
+static bool kernel_rwx = true;
+static u32 perf_refill_insn;
+
+static DEFINE_MUTEX(kernel_rwx_mutex);
+
+static int __init parse_kernel_rwx(char *arg)
+{
+ strtobool(arg, &kernel_rwx);
+
+ return 0;
+}
+early_param("kvx.kernel_rwx", parse_kernel_rwx);
+
+static void map_exception_only_in_ltlb(void)
+{
+ struct kvx_tlb_format tlbe;
+
+ tlbe = tlb_mk_entry(
+ (void *)__pa(__exception_start),
+ (void *)__exception_start,
+ TLB_PS_4K,
+ TLB_G_GLOBAL,
+ TLB_PA_NA_RX,
+ TLB_CP_W_C,
+ 0,
+ TLB_ES_A_MODIFIED);
+
+ BUG_ON((__exception_end - __exception_start) > PAGE_SIZE);
+
+ kvx_mmu_add_entry(MMC_SB_LTLB, LTLB_ENTRY_KERNEL_TEXT, tlbe);
+}
+
+static void map_kernel_in_ltlb(void)
+{
+ struct kvx_tlb_format tlbe;
+
+ tlbe = tlb_mk_entry(
+ (void *) PHYS_OFFSET,
+ (void *) PAGE_OFFSET,
+ TLB_PS_512M,
+ TLB_G_GLOBAL,
+ TLB_PA_NA_RWX,
+ TLB_CP_W_C,
+ 0,
+ TLB_ES_A_MODIFIED);
+
+ kvx_mmu_add_entry(MMC_SB_LTLB, LTLB_ENTRY_KERNEL_TEXT, tlbe);
+}
+
+static void mmu_disable_kernel_perf_refill(void)
+{
+ unsigned int off = kvx_std_tlb_refill - kvx_perf_tlb_refill;
+ u32 goto_insn;
+ int ret;
+
+ BUG_ON(KVX_INSN_GOTO_PCREL27_CHECK(off));
+
+ KVX_INSN_GOTO(&goto_insn, KVX_INSN_PARALLEL_EOB, off);
+
+ ret = kvx_insns_read(&perf_refill_insn, PERF_REFILL_INSN_SIZE,
+ (u32 *) kvx_perf_tlb_refill);
+ BUG_ON(ret);
+ ret = kvx_insns_write(&goto_insn, PERF_REFILL_INSN_SIZE,
+ (u32 *) kvx_perf_tlb_refill);
+ BUG_ON(ret);
+}
+
+static void enable_kernel_perf_refill(void)
+{
+ int ret;
+
+ ret = kvx_insns_write(&perf_refill_insn, PERF_REFILL_INSN_SIZE,
+ (u32 *) kvx_perf_tlb_refill);
+ BUG_ON(ret);
+}
+
+static void local_mmu_enable_kernel_rwx(void)
+{
+ int i;
+ struct kvx_tlb_format tlbe;
+
+ tlbe = tlb_mk_entry(0, (void *) 0, 0, 0, 0, 0, 0,
+ TLB_ES_INVALID);
+
+ /* Map exceptions handlers in LTLB entry instead of full kernel */
+ map_exception_only_in_ltlb();
+
+ /* Invalidate previously added LTLB entries */
+ for (i = 0; i < REFILL_PERF_ENTRIES; i++)
+ kvx_mmu_add_entry(MMC_SB_LTLB, LTLB_KERNEL_RESERVED + i, tlbe);
+
+}
+
+/**
+ * init_kernel_rwx - Initialize kernel struct RWX at boot time
+ * This function MUST be used only at boot time to setup kernel strict RWX mode
+ * Once done, kernel rwx mode can be enabled/disabled using sysfs entry.
+ */
+void init_kernel_rwx(void)
+{
+ /* Kernel strict RWX mode disabled */
+ if (!kernel_rwx)
+ return;
+
+ /* First processor only will disable perf refill by patching code */
+ if (raw_smp_processor_id() == 0)
+ mmu_disable_kernel_perf_refill();
+
+ local_mmu_enable_kernel_rwx();
+}
+
+static void ipi_enable_kernel_rwx(void *arg)
+{
+ local_mmu_enable_kernel_rwx();
+}
+
+static void local_mmu_disable_kernel_rwx(void)
+{
+ /* Map full kernel in LTLB entry instead of only exceptions */
+ map_kernel_in_ltlb();
+
+ /* Flush JTLB completely to force refill and avoid stalled entries */
+ local_flush_tlb_all();
+}
+
+static void ipi_disable_kernel_rwx(void *arg)
+{
+ local_mmu_disable_kernel_rwx();
+}
+
+static void smp_set_kernel_rwx(bool kernel_rwx)
+{
+ smp_call_func_t fn;
+
+ pr_info("%sabling kernel rwx mode\n", kernel_rwx ? "En" : "Dis");
+
+ if (kernel_rwx) {
+ mmu_disable_kernel_perf_refill();
+ fn = ipi_enable_kernel_rwx;
+ } else {
+ enable_kernel_perf_refill();
+ fn = ipi_disable_kernel_rwx;
+ }
+
+ on_each_cpu(fn, NULL, 1);
+}
+
+static ssize_t kernel_rwx_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return sprintf(buf, "%s\n", kernel_rwx ? "enabled" : "disabled");
+}
+
+static ssize_t kernel_rwx_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t len)
+{
+ bool value;
+ int ret;
+
+ ret = strtobool(buf, &value);
+ if (ret)
+ return ret;
+
+ mutex_lock(&kernel_rwx_mutex);
+ /* Switch only if necessary */
+ if (value != kernel_rwx) {
+ kernel_rwx = value;
+ smp_set_kernel_rwx(kernel_rwx);
+ }
+ mutex_unlock(&kernel_rwx_mutex);
+
+ return len;
+}
+
+static struct kobj_attribute kernel_rwx_attr =
+ __ATTR(kernel_rwx, 0644, kernel_rwx_show, kernel_rwx_store);
+
+static struct attribute *default_attrs[] = {
+ &kernel_rwx_attr.attr,
+ NULL
+};
+
+static struct attribute_group kernel_rwx_attr_group = {
+ .attrs = default_attrs,
+};
+
+static int __init kvx_kernel_rwx_init(void)
+{
+ int ret;
+
+ kvx_kobj = kobject_create_and_add("kvx", NULL);
+ if (!kvx_kobj)
+ return -ENOMEM;
+
+ ret = sysfs_create_group(kvx_kobj, &kernel_rwx_attr_group);
+ if (ret) {
+ kobject_put(kvx_kobj);
+ return ret;
+ }
+
+ return 0;
+}
+postcore_initcall(kvx_kernel_rwx_init);
diff --git a/arch/kvx/mm/mmap.c b/arch/kvx/mm/mmap.c
new file mode 100644
index 000000000000..a2225db64438
--- /dev/null
+++ b/arch/kvx/mm/mmap.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/arm64/mm/mmap.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifdef CONFIG_STRICT_DEVMEM
+
+#include <linux/mm.h>
+#include <linux/ioport.h>
+
+#include <asm/page.h>
+
+/*
+ * devmem_is_allowed() checks to see if /dev/mem access to a certain address
+ * is valid. The argument is a physical page number. We mimic x86 here by
+ * disallowing access to system RAM as well as device-exclusive MMIO regions.
+ * This effectively disable read()/write() on /dev/mem.
+ */
+int devmem_is_allowed(unsigned long pfn)
+{
+ if (iomem_is_exclusive(pfn << PAGE_SHIFT))
+ return 0;
+ if (!page_is_ram(pfn))
+ return 1;
+ return 0;
+}
+
+#endif
diff --git a/arch/kvx/mm/mmu.c b/arch/kvx/mm/mmu.c
new file mode 100644
index 000000000000..a1eadececb6a
--- /dev/null
+++ b/arch/kvx/mm/mmu.c
@@ -0,0 +1,204 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Vincent Chardon
+ * Jules Maselbas
+ */
+
+#include <linux/cache.h>
+#include <linux/types.h>
+#include <linux/irqflags.h>
+#include <linux/printk.h>
+#include <linux/percpu.h>
+#include <linux/kernel.h>
+#include <linux/spinlock.h>
+#include <linux/spinlock_types.h>
+
+#include <asm/mmu.h>
+#include <asm/tlb.h>
+#include <asm/page_size.h>
+#include <asm/mmu_context.h>
+
+#define DUMP_LTLB 0
+#define DUMP_JTLB 1
+
+DEFINE_PER_CPU_ALIGNED(uint8_t[MMU_JTLB_SETS], jtlb_current_set_way);
+static struct kvx_tlb_format ltlb_entries[MMU_LTLB_WAYS];
+static unsigned long ltlb_entries_bmp;
+
+static int kvx_mmu_ltlb_overlaps(struct kvx_tlb_format tlbe)
+{
+ int bit = LTLB_ENTRY_FIXED_COUNT;
+
+ for_each_set_bit_from(bit, <lb_entries_bmp, MMU_LTLB_WAYS) {
+ if (tlb_entry_overlaps(tlbe, ltlb_entries[bit]))
+ return 1;
+ }
+
+ return 0;
+}
+
+/**
+ * kvx_mmu_ltlb_add_entry - Add a kernel entry in the LTLB
+ *
+ * In order to lock some entries in the LTLB and be always mapped, this
+ * function can be called with a physical address, a virtual address and
+ * protection attribute to add an entry into the LTLB. This is mainly for
+ * performances since there won't be any NOMAPPING traps for these pages.
+ *
+ * @vaddr: Virtual address for the entry (must be aligned according to tlb_ps)
+ * @paddr: Physical address for the entry (must be aligned according to tlb_ps)
+ * @flags: Protection attributes
+ * @tlb_ps: Page size attribute for TLB (TLB_PS_*)
+ */
+void kvx_mmu_ltlb_add_entry(unsigned long vaddr, phys_addr_t paddr,
+ pgprot_t flags, unsigned long tlb_ps)
+{
+ unsigned int cp;
+ unsigned int idx;
+ unsigned long irqflags;
+ struct kvx_tlb_format tlbe;
+ u64 page_size = BIT(get_page_size_shift(tlb_ps));
+
+ BUG_ON(!IS_ALIGNED(vaddr, page_size) || !IS_ALIGNED(paddr, page_size));
+
+ cp = pgprot_cache_policy(pgprot_val(flags));
+
+ tlbe = tlb_mk_entry(
+ (void *) paddr,
+ (void *) vaddr,
+ tlb_ps,
+ TLB_G_GLOBAL,
+ TLB_PA_NA_RW,
+ cp,
+ 0,
+ TLB_ES_A_MODIFIED);
+
+ local_irq_save(irqflags);
+
+ if (IS_ENABLED(CONFIG_KVX_DEBUG_TLB_WRITE) &&
+ kvx_mmu_ltlb_overlaps(tlbe))
+ panic("VA %lx overlaps with an existing LTLB mapping", vaddr);
+
+ idx = find_next_zero_bit(<lb_entries_bmp, MMU_LTLB_WAYS,
+ LTLB_ENTRY_FIXED_COUNT);
+ /* This should never happen */
+ BUG_ON(idx >= MMU_LTLB_WAYS);
+ __set_bit(idx, <lb_entries_bmp);
+ ltlb_entries[idx] = tlbe;
+ kvx_mmu_add_entry(MMC_SB_LTLB, idx, tlbe);
+
+ if (kvx_mmc_error(kvx_sfr_get(MMC)))
+ panic("Failed to write entry to the LTLB");
+
+ local_irq_restore(irqflags);
+}
+
+void kvx_mmu_ltlb_remove_entry(unsigned long vaddr)
+{
+ int ret, bit = LTLB_ENTRY_FIXED_COUNT;
+ struct kvx_tlb_format tlbe;
+
+ for_each_set_bit_from(bit, <lb_entries_bmp, MMU_LTLB_WAYS) {
+ tlbe = ltlb_entries[bit];
+ if (tlb_entry_match_addr(tlbe, vaddr)) {
+ __clear_bit(bit, <lb_entries_bmp);
+ break;
+ }
+ }
+
+ if (bit == MMU_LTLB_WAYS)
+ panic("Trying to remove non-existent LTLB entry for addr %lx\n",
+ vaddr);
+
+ ret = clear_ltlb_entry(vaddr);
+ if (ret)
+ panic("Failed to remove LTLB entry for addr %lx\n", vaddr);
+}
+
+/**
+ * kvx_mmu_jtlb_add_entry - Add an entry into JTLB
+ *
+ * JTLB is used both for kernel and user entries.
+ *
+ * @address: Virtual address for the entry (must be aligned according to tlb_ps)
+ * @ptep: pte entry pointer
+ * @asn: ASN (if pte entry is not global)
+ */
+void kvx_mmu_jtlb_add_entry(unsigned long address, pte_t *ptep,
+ unsigned int asn)
+{
+ unsigned int shifted_addr, set, way;
+ unsigned long flags, pte_val;
+ struct kvx_tlb_format tlbe;
+ unsigned int pa, cp, ps;
+ phys_addr_t pfn;
+
+ pte_val = pte_val(*ptep);
+
+ pfn = (phys_addr_t)pte_pfn(*ptep);
+
+ asn &= MM_CTXT_ASN_MASK;
+
+ /* Set page as accessed */
+ pte_val(*ptep) |= _PAGE_ACCESSED;
+
+ BUILD_BUG_ON(KVX_PAGE_SZ_SHIFT != KVX_SFR_TEL_PS_SHIFT);
+
+ ps = (pte_val & KVX_PAGE_SZ_MASK) >> KVX_PAGE_SZ_SHIFT;
+ pa = get_page_access_perms(KVX_ACCESS_PERMS_INDEX(pte_val));
+ cp = pgprot_cache_policy(pte_val);
+
+ tlbe = tlb_mk_entry(
+ (void *)pfn_to_phys(pfn),
+ (void *)address,
+ ps,
+ (pte_val & _PAGE_GLOBAL) ? TLB_G_GLOBAL : TLB_G_USE_ASN,
+ pa,
+ cp,
+ asn,
+ TLB_ES_A_MODIFIED);
+
+ shifted_addr = address >> get_page_size_shift(ps);
+ set = shifted_addr & MMU_JTLB_SET_MASK;
+
+ local_irq_save(flags);
+
+ if (IS_ENABLED(CONFIG_KVX_DEBUG_TLB_WRITE) &&
+ kvx_mmu_ltlb_overlaps(tlbe))
+ panic("VA %lx overlaps with an existing LTLB mapping", address);
+
+ way = get_cpu_var(jtlb_current_set_way)[set]++;
+ put_cpu_var(jtlb_current_set_way);
+
+ way &= MMU_JTLB_WAY_MASK;
+
+ kvx_mmu_add_entry(MMC_SB_JTLB, way, tlbe);
+
+ if (IS_ENABLED(CONFIG_KVX_DEBUG_TLB_WRITE) &&
+ kvx_mmc_error(kvx_sfr_get(MMC)))
+ panic("Failed to write entry to the JTLB (in update_mmu_cache)");
+
+ local_irq_restore(flags);
+}
+
+void __init kvx_mmu_early_setup(void)
+{
+ int bit;
+ struct kvx_tlb_format tlbe;
+
+ kvx_mmu_remove_ltlb_entry(LTLB_ENTRY_EARLY_SMEM);
+
+ if (raw_smp_processor_id() != 0) {
+ /* Apply existing ltlb entries starting from first one free */
+ bit = LTLB_ENTRY_FIXED_COUNT;
+ for_each_set_bit_from(bit, <lb_entries_bmp, MMU_LTLB_WAYS) {
+ tlbe = ltlb_entries[bit];
+ kvx_mmu_add_entry(MMC_SB_LTLB, bit, tlbe);
+ }
+
+ init_kernel_rwx();
+ }
+}
diff --git a/arch/kvx/mm/mmu_stats.c b/arch/kvx/mm/mmu_stats.c
new file mode 100644
index 000000000000..e70b7e2dcab1
--- /dev/null
+++ b/arch/kvx/mm/mmu_stats.c
@@ -0,0 +1,94 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/seq_file.h>
+#include <linux/debugfs.h>
+
+#include <asm/mmu_stats.h>
+
+static struct dentry *mmu_stats_debufs;
+
+static const char *mmu_refill_types_name[MMU_REFILL_TYPE_COUNT] = {
+ [MMU_REFILL_TYPE_USER] = "User",
+ [MMU_REFILL_TYPE_KERNEL] = "Kernel",
+ [MMU_REFILL_TYPE_KERNEL_DIRECT] = "Kernel Direct"
+};
+
+DEFINE_PER_CPU(struct mmu_stats, mmu_stats);
+
+static int mmu_stats_show(struct seq_file *m, void *v)
+{
+ int cpu, type;
+ unsigned long avg = 0, total_refill, efficiency, total_cycles;
+ struct mmu_stats *stats;
+ struct mmu_refill_stats *ref_stat;
+
+ total_cycles = get_cycles();
+ for_each_present_cpu(cpu) {
+ stats = &per_cpu(mmu_stats, cpu);
+ total_refill = 0;
+
+ seq_printf(m, " - CPU %d\n", cpu);
+ for (type = 0; type < MMU_REFILL_TYPE_COUNT; type++) {
+ ref_stat = &stats->refill[type];
+ total_refill += ref_stat->count;
+ if (ref_stat->count)
+ avg = ref_stat->total / ref_stat->count;
+ else
+ avg = 0;
+
+ seq_printf(m,
+ " - %s refill stats:\n"
+ " - count: %lu\n"
+ " - min: %lu\n"
+ " - avg: %lu\n"
+ " - max: %lu\n",
+ mmu_refill_types_name[type],
+ ref_stat->count,
+ ref_stat->min,
+ avg,
+ ref_stat->max
+ );
+ }
+
+ if (total_refill)
+ avg = stats->cycles_between_refill / total_refill;
+ else
+ avg = 0;
+
+ seq_printf(m, " - Average cycles between refill: %lu\n", avg);
+ seq_printf(m, " - tlb_flush_all calls: %lu\n",
+ stats->tlb_flush_all);
+ efficiency = stats->cycles_between_refill * 100 /
+ stats->last_refill;
+ seq_printf(m, " - Efficiency: %lu%%\n", efficiency);
+ }
+
+ return 0;
+}
+
+static int mmu_stats_open(struct inode *inode, struct file *filp)
+{
+ return single_open(filp, mmu_stats_show, NULL);
+}
+
+static const struct file_operations mmu_stats_fops = {
+ .open = mmu_stats_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static int __init mmu_stats_debufs_init(void)
+{
+ mmu_stats_debufs = debugfs_create_dir("kvx_mmu_debug", NULL);
+
+ debugfs_create_file("mmu_stats", 0444, mmu_stats_debufs, NULL,
+ &mmu_stats_fops);
+
+ return 0;
+}
+subsys_initcall(mmu_stats_debufs_init);
diff --git a/arch/kvx/mm/tlb.c b/arch/kvx/mm/tlb.c
new file mode 100644
index 000000000000..9542332fe4d6
--- /dev/null
+++ b/arch/kvx/mm/tlb.c
@@ -0,0 +1,433 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#include <linux/mmu_context.h>
+#include <linux/sched.h>
+
+#include <asm/tlbflush.h>
+#include <asm/tlb_defs.h>
+#include <asm/page_size.h>
+#include <asm/mmu_stats.h>
+#include <asm/pgtable.h>
+#include <asm/tlb.h>
+
+/*
+ * When in kernel, use dummy ASN 42 to be able to catch any problem easily if
+ * ASN is not restored properly.
+ */
+#define KERNEL_DUMMY_ASN 42
+
+/* Threshold of page count above which we will regenerate a new ASN */
+#define ASN_FLUSH_PAGE_THRESHOLD (MMU_JTLB_ENTRIES)
+
+/* Threshold of page count above which we will flush the whole JTLB */
+#define FLUSH_ALL_PAGE_THRESHOLD (MMU_JTLB_ENTRIES)
+
+DEFINE_PER_CPU(unsigned long, kvx_asn_cache) = MM_CTXT_FIRST_CYCLE;
+
+#ifdef CONFIG_KVX_DEBUG_TLB_ACCESS
+
+static DEFINE_PER_CPU_ALIGNED(struct kvx_tlb_access_t[KVX_TLB_ACCESS_SIZE],
+ kvx_tlb_access_rb);
+/* Lower bits hold the index and upper ones hold the number of wrapped */
+static DEFINE_PER_CPU(unsigned int, kvx_tlb_access_idx);
+
+void kvx_update_tlb_access(int type)
+{
+ unsigned int *idx_ptr = &get_cpu_var(kvx_tlb_access_idx);
+ unsigned int idx;
+ struct kvx_tlb_access_t *tab = get_cpu_var(kvx_tlb_access_rb);
+
+ idx = KVX_TLB_ACCESS_GET_IDX(*idx_ptr);
+
+ kvx_mmu_get_tlb_entry(tab[idx].entry);
+ tab[idx].mmc_val = kvx_sfr_get(MMC);
+ tab[idx].type = type;
+
+ (*idx_ptr)++;
+ put_cpu_var(kvx_tlb_access_rb);
+ put_cpu_var(kvx_tlb_access_idx);
+};
+
+#endif
+
+/**
+ * clear_tlb_entry() - clear an entry in TLB if it exists
+ * @addr: the address used to set TEH.PN
+ * @global: is page global or not
+ * @asn: ASN used if page is not global
+ * @tlb_type: tlb type (MMC_SB_LTLB or MMC_SB_JTLB)
+ *
+ * Preemption must be disabled when calling this function. There is no need to
+ * invalidate micro TLB because it is invalidated when we write TLB.
+ *
+ * Return: 0 if TLB entry was found and deleted properly, -ENOENT if not found
+ * -EINVAL if found but in incorrect TLB.
+ *
+ */
+static int clear_tlb_entry(unsigned long addr,
+ unsigned int global,
+ unsigned int asn,
+ unsigned int tlb_type)
+{
+ struct kvx_tlb_format entry;
+ unsigned long mmc_val;
+ int saved_asn, ret = 0;
+
+ /* Sanitize ASN */
+ asn &= MM_CTXT_ASN_MASK;
+
+ /* Before probing we need to save the current ASN */
+ mmc_val = kvx_sfr_get(MMC);
+ saved_asn = kvx_sfr_field_val(mmc_val, MMC, ASN);
+ kvx_sfr_set_field(MMC, ASN, asn);
+
+ /* Probe is based on PN and ASN. So ES can be anything */
+ entry = tlb_mk_entry(0, (void *)addr, 0, global, 0, 0, 0,
+ TLB_ES_INVALID);
+ kvx_mmu_set_tlb_entry(entry);
+
+ kvx_mmu_probetlb();
+
+ mmc_val = kvx_sfr_get(MMC);
+
+ if (kvx_mmc_error(mmc_val)) {
+ if (kvx_mmc_parity(mmc_val)) {
+ /*
+ * This should never happens unless you are bombared by
+ * streams of charged particules. If it happens just
+ * flush the JTLB and let's continue (but check your
+ * environment you are probably not in a safe place).
+ */
+ WARN(1, "%s: parity error during lookup (addr 0x%lx, asn %u). JTLB will be flushed\n",
+ __func__, addr, asn);
+ kvx_sfr_set_field(MMC, PAR, 0);
+ local_flush_tlb_all();
+ }
+
+ /*
+ * else there is no matching entry so just clean the error and
+ * restore the ASN before returning.
+ */
+ kvx_sfr_set_field(MMC, E, 0);
+ ret = -ENOENT;
+ goto restore_asn;
+ }
+
+ /* We surely don't want to flush another TLB type or we are fried */
+ if (kvx_mmc_sb(mmc_val) != tlb_type) {
+ ret = -EINVAL;
+ goto restore_asn;
+ }
+
+ /*
+ * At this point the probe found an entry. TEL and TEH have correct
+ * values so we just need to set the entry status to invalid to clear
+ * the entry.
+ */
+ kvx_sfr_set_field(TEL, ES, TLB_ES_INVALID);
+
+ kvx_mmu_writetlb();
+
+ /* Need to read MMC SFR again */
+ mmc_val = kvx_sfr_get(MMC);
+ if (kvx_mmc_error(mmc_val))
+ panic("%s: Failed to clear entry (addr 0x%lx, asn %u)",
+ __func__, addr, asn);
+ else
+ pr_debug("%s: Entry (addr 0x%lx, asn %u) cleared\n",
+ __func__, addr, asn);
+
+restore_asn:
+ kvx_sfr_set_field(MMC, ASN, saved_asn);
+
+ return ret;
+}
+
+static void clear_jtlb_entry(unsigned long addr,
+ unsigned int global,
+ unsigned int asn)
+{
+ clear_tlb_entry(addr, global, asn, MMC_SB_JTLB);
+}
+
+/**
+ * clear_ltlb_entry() - Remove a LTLB entry
+ * @vaddr: Virtual address to be matched against LTLB entries
+ *
+ * Return: Same value as clear_tlb_entry
+ */
+int clear_ltlb_entry(unsigned long vaddr)
+{
+ return clear_tlb_entry(vaddr, TLB_G_GLOBAL, KERNEL_DUMMY_ASN,
+ MMC_SB_LTLB);
+}
+
+/* If mm is current we just need to assign the current task a new ASN. By
+ * doing this all previous TLB entries with the former ASN will be invalidated.
+ * if mm is not the current one we invalidate the context and when this other
+ * mm will be swapped in then a new context will be generated.
+ */
+void local_flush_tlb_mm(struct mm_struct *mm)
+{
+ int cpu = smp_processor_id();
+
+ destroy_context(mm);
+ if (current->active_mm == mm)
+ activate_context(mm, cpu);
+}
+
+void local_flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long addr)
+{
+ int cpu = smp_processor_id();
+ unsigned int current_asn;
+ struct mm_struct *mm;
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ mm = vma->vm_mm;
+ current_asn = mm_asn(mm, cpu);
+
+ /* If mm has no context there is nothing to do */
+ if (current_asn != MM_CTXT_NO_ASN)
+ clear_jtlb_entry(addr, TLB_G_USE_ASN, current_asn);
+
+ local_irq_restore(flags);
+}
+
+void local_flush_tlb_all(void)
+{
+ struct kvx_tlb_format tlbe = KVX_EMPTY_TLB_ENTRY;
+ int set, way;
+ unsigned long flags;
+#ifdef CONFIG_KVX_MMU_STATS
+ struct mmu_stats *stats = &per_cpu(mmu_stats, smp_processor_id());
+
+ stats->tlb_flush_all++;
+#endif
+
+ local_irq_save(flags);
+
+ /* Select JTLB and prepare TEL (constant) */
+ kvx_sfr_set(TEL, (uint64_t) tlbe.tel_val);
+ kvx_sfr_set_field(MMC, SB, MMC_SB_JTLB);
+
+ for (set = 0; set < MMU_JTLB_SETS; set++) {
+ tlbe.teh.pn = set;
+ for (way = 0; way < MMU_JTLB_WAYS; way++) {
+ /* Set is selected automatically according to the
+ * virtual address.
+ * With 4K pages the set is the value of the 6 lower
+ * significant bits of the page number.
+ */
+ kvx_sfr_set(TEH, (uint64_t) tlbe.teh_val);
+ kvx_sfr_set_field(MMC, SW, way);
+ kvx_mmu_writetlb();
+
+ if (kvx_mmc_error(kvx_sfr_get(MMC)))
+ panic("Failed to initialize JTLB[s:%02d w:%d]",
+ set, way);
+ }
+ }
+
+ local_irq_restore(flags);
+}
+
+void local_flush_tlb_range(struct vm_area_struct *vma,
+ unsigned long start,
+ unsigned long end)
+{
+ const unsigned int cpu = smp_processor_id();
+ unsigned long flags;
+ unsigned int current_asn;
+ unsigned long pages = (end - start) >> PAGE_SHIFT;
+
+ if (pages > ASN_FLUSH_PAGE_THRESHOLD) {
+ local_flush_tlb_mm(vma->vm_mm);
+ return;
+ }
+
+ start &= PAGE_MASK;
+
+ local_irq_save(flags);
+
+ current_asn = mm_asn(vma->vm_mm, cpu);
+ if (current_asn != MM_CTXT_NO_ASN) {
+ while (start < end) {
+ clear_jtlb_entry(start, TLB_G_USE_ASN, current_asn);
+ start += PAGE_SIZE;
+ }
+ }
+
+ local_irq_restore(flags);
+}
+
+/**
+ * local_flush_tlb_kernel_range() - flush kernel TLB entries
+ * @start: start kernel virtual address
+ * @end: end kernel virtual address
+ *
+ * Return: nothing
+ */
+void local_flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ unsigned long flags;
+ unsigned long pages = (end - start) >> PAGE_SHIFT;
+
+ if (pages > FLUSH_ALL_PAGE_THRESHOLD) {
+ local_flush_tlb_all();
+ return;
+ }
+
+ start &= PAGE_MASK;
+
+ local_irq_save(flags);
+
+ while (start < end) {
+ clear_jtlb_entry(start, TLB_G_GLOBAL, KERNEL_DUMMY_ASN);
+ start += PAGE_SIZE;
+ }
+
+ local_irq_restore(flags);
+}
+
+void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
+ pmd_t *pmd)
+{
+ pte_t pte = __pte(pmd_val(*pmd));
+
+ /* THP PMD accessors are implemented in terms of PTE */
+ update_mmu_cache(vma, addr, &pte);
+}
+
+void update_mmu_cache(struct vm_area_struct *vma,
+ unsigned long address, pte_t *ptep)
+{
+ struct mm_struct *mm;
+ unsigned long asn;
+ int cpu = smp_processor_id();
+
+ if (unlikely(ptep == NULL))
+ panic("pte should not be NULL\n");
+
+ /* Flush any previous TLB entries */
+ flush_tlb_page(vma, address);
+
+ /* No need to add the TLB entry until the process that owns the memory
+ * is running.
+ */
+ mm = current->active_mm;
+ if (vma && (mm != vma->vm_mm))
+ return;
+
+ /*
+ * Get the ASN:
+ * ASN can have no context if address belongs to kernel space. In
+ * fact as pages are global for kernel space the ASN is ignored
+ * and can be equal to any value.
+ */
+ asn = mm_asn(mm, cpu);
+
+#ifdef CONFIG_KVX_DEBUG_ASN
+ /*
+ * For addresses that belong to user space, the value of the ASN
+ * must match the mmc.asn and be non zero
+ */
+ if (address < PAGE_OFFSET) {
+ unsigned int mmc_asn = kvx_mmc_asn(kvx_sfr_get(MMC));
+
+ if (asn == MM_CTXT_NO_ASN)
+ panic("%s: ASN [%lu] is not properly set for address 0x%lx on CPU %d\n",
+ __func__, asn, address, cpu);
+
+ if ((asn & MM_CTXT_ASN_MASK) != mmc_asn)
+ panic("%s: ASN not synchronized with MMC: asn:%lu != mmc.asn:%u\n",
+ __func__, (asn & MM_CTXT_ASN_MASK), mmc_asn);
+ }
+#endif
+
+ kvx_mmu_jtlb_add_entry(address, ptep, asn);
+}
+
+#ifdef CONFIG_SMP
+
+struct tlb_args {
+ struct vm_area_struct *ta_vma;
+ unsigned long ta_start;
+ unsigned long ta_end;
+};
+
+static inline void ipi_flush_tlb_page(void *arg)
+{
+ struct tlb_args *ta = arg;
+
+ local_flush_tlb_page(ta->ta_vma, ta->ta_start);
+}
+
+void
+smp_flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
+{
+ struct tlb_args ta = {
+ .ta_vma = vma,
+ .ta_start = addr
+ };
+
+ on_each_cpu_mask(mm_cpumask(vma->vm_mm), ipi_flush_tlb_page, &ta, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_page);
+
+void
+smp_flush_tlb_mm(struct mm_struct *mm)
+{
+ on_each_cpu_mask(mm_cpumask(mm), (smp_call_func_t)local_flush_tlb_mm,
+ mm, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_mm);
+
+static inline void ipi_flush_tlb_range(void *arg)
+{
+ struct tlb_args *ta = arg;
+
+ local_flush_tlb_range(ta->ta_vma, ta->ta_start, ta->ta_end);
+}
+
+void
+smp_flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end)
+{
+ struct tlb_args ta = {
+ .ta_vma = vma,
+ .ta_start = start,
+ .ta_end = end
+ };
+
+ on_each_cpu_mask(mm_cpumask(vma->vm_mm), ipi_flush_tlb_range, &ta, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_range);
+
+static inline void ipi_flush_tlb_kernel_range(void *arg)
+{
+ struct tlb_args *ta = arg;
+
+ local_flush_tlb_kernel_range(ta->ta_start, ta->ta_end);
+}
+
+void
+smp_flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ struct tlb_args ta = {
+ .ta_start = start,
+ .ta_end = end
+ };
+
+ on_each_cpu(ipi_flush_tlb_kernel_range, &ta, 1);
+}
+EXPORT_SYMBOL(smp_flush_tlb_kernel_range);
+
+#endif
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 11/25] kvx: Add memory management
2023-01-03 16:43 ` Yann Sionneau
@ 2023-01-04 11:37 ` Mike Rapoport
-1 siblings, 0 replies; 113+ messages in thread
From: Mike Rapoport @ 2023-01-04 11:37 UTC (permalink / raw)
To: Yann Sionneau
Cc: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
linux-kernel, linux-arch, linux-mm, linux-riscv, Clement Leger,
Guillaume Thouvenin, Jean-Christophe Pince, Jules Maselbas,
Julian Vetter, Julien Hascoet, Louis Morhet, Marc Poulhiès,
Marius Gligor, Vincent Chardon
On Tue, Jan 03, 2023 at 05:43:45PM +0100, Yann Sionneau wrote:
> Add memory management support for kvx, including: cache and tlb
> management, page fault handling, ioremap/mmap and streaming dma support.
>
> CC: Will Deacon <will@kernel.org>
> CC: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
> CC: Andrew Morton <akpm@linux-foundation.org>
> CC: Nick Piggin <npiggin@gmail.com>
> CC: Peter Zijlstra <peterz@infradead.org>
> CC: Paul Walmsley <paul.walmsley@sifive.com>
> CC: Palmer Dabbelt <palmer@dabbelt.com>
> CC: Albert Ou <aou@eecs.berkeley.edu>
> CC: linux-kernel@vger.kernel.org
> CC: linux-arch@vger.kernel.org
> CC: linux-mm@kvack.org
> CC: linux-riscv@lists.infradead.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Co-developed-by: Jean-Christophe Pince <jcpince@gmail.com>
> Signed-off-by: Jean-Christophe Pince <jcpince@gmail.com>
> Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
> Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Co-developed-by: Julien Hascoet <jhascoet@kalray.eu>
> Signed-off-by: Julien Hascoet <jhascoet@kalray.eu>
> Co-developed-by: Louis Morhet <lmorhet@kalray.eu>
> Signed-off-by: Louis Morhet <lmorhet@kalray.eu>
> Co-developed-by: Marc Poulhiès <dkm@kataplop.net>
> Signed-off-by: Marc Poulhiès <dkm@kataplop.net>
> Co-developed-by: Marius Gligor <mgligor@kalray.eu>
> Signed-off-by: Marius Gligor <mgligor@kalray.eu>
> Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
> Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> arch/kvx/include/asm/cache.h | 46 +++
> arch/kvx/include/asm/cacheflush.h | 181 +++++++++
> arch/kvx/include/asm/fixmap.h | 47 +++
> arch/kvx/include/asm/hugetlb.h | 36 ++
> arch/kvx/include/asm/l2_cache.h | 75 ++++
> arch/kvx/include/asm/l2_cache_defs.h | 64 ++++
> arch/kvx/include/asm/mem_map.h | 44 +++
> arch/kvx/include/asm/mmu.h | 296 +++++++++++++++
> arch/kvx/include/asm/mmu_context.h | 156 ++++++++
> arch/kvx/include/asm/mmu_stats.h | 38 ++
> arch/kvx/include/asm/page.h | 187 ++++++++++
> arch/kvx/include/asm/page_size.h | 29 ++
> arch/kvx/include/asm/pgalloc.h | 101 +++++
> arch/kvx/include/asm/pgtable-bits.h | 102 ++++++
> arch/kvx/include/asm/pgtable.h | 451 +++++++++++++++++++++++
> arch/kvx/include/asm/rm_fw.h | 16 +
> arch/kvx/include/asm/sparsemem.h | 15 +
> arch/kvx/include/asm/symbols.h | 16 +
> arch/kvx/include/asm/tlb.h | 24 ++
> arch/kvx/include/asm/tlb_defs.h | 131 +++++++
> arch/kvx/include/asm/tlbflush.h | 58 +++
> arch/kvx/include/asm/vmalloc.h | 10 +
> arch/kvx/mm/cacheflush.c | 154 ++++++++
> arch/kvx/mm/dma-mapping.c | 95 +++++
> arch/kvx/mm/extable.c | 24 ++
> arch/kvx/mm/fault.c | 264 ++++++++++++++
> arch/kvx/mm/hugetlbpage.c | 317 ++++++++++++++++
> arch/kvx/mm/init.c | 527 +++++++++++++++++++++++++++
> arch/kvx/mm/kernel_rwx.c | 228 ++++++++++++
> arch/kvx/mm/mmap.c | 31 ++
> arch/kvx/mm/mmu.c | 204 +++++++++++
> arch/kvx/mm/mmu_stats.c | 94 +++++
> arch/kvx/mm/tlb.c | 433 ++++++++++++++++++++++
> 33 files changed, 4494 insertions(+)
Please strip functionality that's not essential for boot and the re-add it
after the very core code is merged. I'd say multiple page size, hugetlb and
strict RWX are not necessary to start with.
I'd also suggest to have separate patches for different parts, like cache
and tlb management, page tables and page fault fault handling, ioremap/mmap
and streaming dma support.
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 11/25] kvx: Add memory management
@ 2023-01-04 11:37 ` Mike Rapoport
0 siblings, 0 replies; 113+ messages in thread
From: Mike Rapoport @ 2023-01-04 11:37 UTC (permalink / raw)
To: Yann Sionneau
Cc: Will Deacon, Aneesh Kumar K.V, Andrew Morton, Nick Piggin,
Peter Zijlstra, Paul Walmsley, Palmer Dabbelt, Albert Ou,
linux-kernel, linux-arch, linux-mm, linux-riscv, Clement Leger,
Guillaume Thouvenin, Jean-Christophe Pince, Jules Maselbas,
Julian Vetter, Julien Hascoet, Louis Morhet, Marc Poulhiès,
Marius Gligor, Vincent Chardon
On Tue, Jan 03, 2023 at 05:43:45PM +0100, Yann Sionneau wrote:
> Add memory management support for kvx, including: cache and tlb
> management, page fault handling, ioremap/mmap and streaming dma support.
>
> CC: Will Deacon <will@kernel.org>
> CC: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
> CC: Andrew Morton <akpm@linux-foundation.org>
> CC: Nick Piggin <npiggin@gmail.com>
> CC: Peter Zijlstra <peterz@infradead.org>
> CC: Paul Walmsley <paul.walmsley@sifive.com>
> CC: Palmer Dabbelt <palmer@dabbelt.com>
> CC: Albert Ou <aou@eecs.berkeley.edu>
> CC: linux-kernel@vger.kernel.org
> CC: linux-arch@vger.kernel.org
> CC: linux-mm@kvack.org
> CC: linux-riscv@lists.infradead.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Co-developed-by: Jean-Christophe Pince <jcpince@gmail.com>
> Signed-off-by: Jean-Christophe Pince <jcpince@gmail.com>
> Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
> Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Co-developed-by: Julien Hascoet <jhascoet@kalray.eu>
> Signed-off-by: Julien Hascoet <jhascoet@kalray.eu>
> Co-developed-by: Louis Morhet <lmorhet@kalray.eu>
> Signed-off-by: Louis Morhet <lmorhet@kalray.eu>
> Co-developed-by: Marc Poulhiès <dkm@kataplop.net>
> Signed-off-by: Marc Poulhiès <dkm@kataplop.net>
> Co-developed-by: Marius Gligor <mgligor@kalray.eu>
> Signed-off-by: Marius Gligor <mgligor@kalray.eu>
> Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
> Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> arch/kvx/include/asm/cache.h | 46 +++
> arch/kvx/include/asm/cacheflush.h | 181 +++++++++
> arch/kvx/include/asm/fixmap.h | 47 +++
> arch/kvx/include/asm/hugetlb.h | 36 ++
> arch/kvx/include/asm/l2_cache.h | 75 ++++
> arch/kvx/include/asm/l2_cache_defs.h | 64 ++++
> arch/kvx/include/asm/mem_map.h | 44 +++
> arch/kvx/include/asm/mmu.h | 296 +++++++++++++++
> arch/kvx/include/asm/mmu_context.h | 156 ++++++++
> arch/kvx/include/asm/mmu_stats.h | 38 ++
> arch/kvx/include/asm/page.h | 187 ++++++++++
> arch/kvx/include/asm/page_size.h | 29 ++
> arch/kvx/include/asm/pgalloc.h | 101 +++++
> arch/kvx/include/asm/pgtable-bits.h | 102 ++++++
> arch/kvx/include/asm/pgtable.h | 451 +++++++++++++++++++++++
> arch/kvx/include/asm/rm_fw.h | 16 +
> arch/kvx/include/asm/sparsemem.h | 15 +
> arch/kvx/include/asm/symbols.h | 16 +
> arch/kvx/include/asm/tlb.h | 24 ++
> arch/kvx/include/asm/tlb_defs.h | 131 +++++++
> arch/kvx/include/asm/tlbflush.h | 58 +++
> arch/kvx/include/asm/vmalloc.h | 10 +
> arch/kvx/mm/cacheflush.c | 154 ++++++++
> arch/kvx/mm/dma-mapping.c | 95 +++++
> arch/kvx/mm/extable.c | 24 ++
> arch/kvx/mm/fault.c | 264 ++++++++++++++
> arch/kvx/mm/hugetlbpage.c | 317 ++++++++++++++++
> arch/kvx/mm/init.c | 527 +++++++++++++++++++++++++++
> arch/kvx/mm/kernel_rwx.c | 228 ++++++++++++
> arch/kvx/mm/mmap.c | 31 ++
> arch/kvx/mm/mmu.c | 204 +++++++++++
> arch/kvx/mm/mmu_stats.c | 94 +++++
> arch/kvx/mm/tlb.c | 433 ++++++++++++++++++++++
> 33 files changed, 4494 insertions(+)
Please strip functionality that's not essential for boot and the re-add it
after the very core code is merged. I'd say multiple page size, hugetlb and
strict RWX are not necessary to start with.
I'd also suggest to have separate patches for different parts, like cache
and tlb management, page tables and page fault fault handling, ioremap/mmap
and streaming dma support.
--
Sincerely yours,
Mike.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 12/25] kvx: Add system call support
2023-01-03 16:43 ` Yann Sionneau
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Christian Brauner, Paul Walmsley, Palmer Dabbelt,
Albert Ou, linux-kernel, linux-riscv, Clement Leger,
Guillaume Thouvenin, Julian Vetter, Julien Villette,
Marius Gligor
Add system call support and related uaccess.h for kvx.
CC: Christian Brauner <brauner@kernel.org>
CC: Paul Walmsley <paul.walmsley@sifive.com>
CC: Palmer Dabbelt <palmer@dabbelt.com>
CC: Albert Ou <aou@eecs.berkeley.edu>
CC: linux-kernel@vger.kernel.org
CC: linux-riscv@lists.infradead.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Julien Villette <jvillette@kalray.eu>
Signed-off-by: Julien Villette <jvillette@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/syscall.h | 73 ++
arch/kvx/include/asm/syscalls.h | 21 +
arch/kvx/include/asm/uaccess.h | 324 +++++
arch/kvx/include/asm/unistd.h | 11 +
arch/kvx/include/uapi/asm/cachectl.h | 25 +
arch/kvx/include/uapi/asm/unistd.h | 16 +
arch/kvx/kernel/entry.S | 1759 ++++++++++++++++++++++++++
arch/kvx/kernel/sys_kvx.c | 58 +
arch/kvx/kernel/syscall_table.c | 19 +
9 files changed, 2306 insertions(+)
create mode 100644 arch/kvx/include/asm/syscall.h
create mode 100644 arch/kvx/include/asm/syscalls.h
create mode 100644 arch/kvx/include/asm/uaccess.h
create mode 100644 arch/kvx/include/asm/unistd.h
create mode 100644 arch/kvx/include/uapi/asm/cachectl.h
create mode 100644 arch/kvx/include/uapi/asm/unistd.h
create mode 100644 arch/kvx/kernel/entry.S
create mode 100644 arch/kvx/kernel/sys_kvx.c
create mode 100644 arch/kvx/kernel/syscall_table.c
diff --git a/arch/kvx/include/asm/syscall.h b/arch/kvx/include/asm/syscall.h
new file mode 100644
index 000000000000..a3f6cef73e4a
--- /dev/null
+++ b/arch/kvx/include/asm/syscall.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SYSCALL_H
+#define _ASM_KVX_SYSCALL_H
+
+#include <linux/err.h>
+#include <linux/audit.h>
+
+#include <asm/ptrace.h>
+
+/* The array of function pointers for syscalls. */
+extern void *sys_call_table[];
+
+void scall_machine_exit(unsigned char value);
+
+/**
+ * syscall_get_nr - find which system call a task is executing
+ * @task: task of interest, must be blocked
+ * @regs: task_pt_regs() of @task
+ *
+ * If @task is executing a system call or is at system call
+ * tracing about to attempt one, returns the system call number.
+ * If @task is not executing a system call, i.e. it's blocked
+ * inside the kernel for a fault or signal, returns -1.
+ *
+ * Note this returns int even on 64-bit machines. Only 32 bits of
+ * system call number can be meaningful. If the actual arch value
+ * is 64 bits, this truncates to 32 bits so 0xffffffff means -1.
+ *
+ * It's only valid to call this when @task is known to be blocked.
+ */
+static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs)
+{
+ if (!in_syscall(regs))
+ return -1;
+
+ return es_sysno(regs);
+}
+
+static inline long syscall_get_error(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ /* 0 if syscall succeeded, otherwise -Errorcode */
+ return IS_ERR_VALUE(regs->r0) ? regs->r0 : 0;
+}
+
+static inline long syscall_get_return_value(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ return regs->r0;
+}
+
+static inline int syscall_get_arch(struct task_struct *task)
+{
+ return AUDIT_ARCH_KVX;
+}
+
+static inline void syscall_get_arguments(struct task_struct *task,
+ struct pt_regs *regs,
+ unsigned long *args)
+{
+ args[0] = regs->orig_r0;
+ args++;
+ memcpy(args, ®s->r1, 5 * sizeof(args[0]));
+}
+
+int __init setup_syscall_sigreturn_page(void *sigpage_addr);
+
+#endif
diff --git a/arch/kvx/include/asm/syscalls.h b/arch/kvx/include/asm/syscalls.h
new file mode 100644
index 000000000000..beec95ebb97a
--- /dev/null
+++ b/arch/kvx/include/asm/syscalls.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SYSCALLS_H
+#define _ASM_KVX_SYSCALLS_H
+
+#include <asm-generic/syscalls.h>
+
+/* We redefine clone in assembly for special slowpath */
+asmlinkage long __sys_clone(unsigned long clone_flags, unsigned long newsp,
+ int __user *parent_tid, int __user *child_tid, int tls);
+
+#define sys_clone __sys_clone
+
+long sys_cachectl(unsigned long addr, unsigned long len, unsigned long cache,
+ unsigned long flags);
+
+#endif
diff --git a/arch/kvx/include/asm/uaccess.h b/arch/kvx/include/asm/uaccess.h
new file mode 100644
index 000000000000..c72677800763
--- /dev/null
+++ b/arch/kvx/include/asm/uaccess.h
@@ -0,0 +1,324 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * derived from arch/riscv/include/asm/uaccess.h
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef _ASM_KVX_UACCESS_H
+#define _ASM_KVX_UACCESS_H
+
+#include <linux/sched.h>
+#include <linux/types.h>
+
+/**
+ * access_ok: - Checks if a user space pointer is valid
+ * @addr: User space pointer to start of block to check
+ * @size: Size of block to check
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * Checks if a pointer to a block of memory in user space is valid.
+ *
+ * Returns true (nonzero) if the memory block may be valid, false (zero)
+ * if it is definitely invalid.
+ *
+ * Note that, depending on architecture, this function probably just
+ * checks that the pointer is in the user space range - after calling
+ * this function, memory access functions may still return -EFAULT.
+ */
+#define access_ok(addr, size) ({ \
+ __chk_user_ptr(addr); \
+ likely(__access_ok((unsigned long __force)(addr), (size))); \
+})
+
+/*
+ * Ensure that the range [addr, addr+size) is within the process's
+ * address space
+ */
+static inline int __access_ok(unsigned long addr, unsigned long size)
+{
+ return size <= TASK_SIZE && addr <= TASK_SIZE - size;
+}
+
+/*
+ * The exception table consists of pairs of addresses: the first is the
+ * address of an instruction that is allowed to fault, and the second is
+ * the address at which the program should continue. No registers are
+ * modified, so it is entirely up to the continuation code to figure out
+ * what to do.
+ *
+ * All the routines below use bits of fixup code that are out of line
+ * with the main instruction path. This means when everything is well,
+ * we don't even have to jump over them. Further, they do not intrude
+ * on our cache or TLB entries.
+ */
+
+struct exception_table_entry {
+ unsigned long insn, fixup;
+};
+
+extern int fixup_exception(struct pt_regs *regs);
+
+/**
+ * Assembly defined function (usercopy.S)
+ */
+extern unsigned long
+raw_copy_from_user(void *to, const void __user *from, unsigned long n);
+
+extern unsigned long
+raw_copy_to_user(void __user *to, const void *from, unsigned long n);
+
+extern unsigned long
+asm_clear_user(void __user *to, unsigned long n);
+
+#define __clear_user asm_clear_user
+
+static inline __must_check unsigned long
+clear_user(void __user *to, unsigned long n)
+{
+ might_fault();
+ if (!access_ok(to, n))
+ return n;
+
+ return asm_clear_user(to, n);
+}
+
+extern __must_check long strnlen_user(const char __user *str, long n);
+extern long strncpy_from_user(char *dest, const char __user *src, long count);
+
+#define __enable_user_access()
+#define __disable_user_access()
+
+/**
+ * get_user: - Get a simple variable from user space.
+ * @x: Variable to store result.
+ * @ptr: Source address, in user space.
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * This macro copies a single simple variable from user space to kernel
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and the result of
+ * dereferencing @ptr must be assignable to @x without a cast.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ * On error, the variable @x is set to zero.
+ */
+#define get_user(x, ptr) \
+({ \
+ long __e = -EFAULT; \
+ const __typeof__(*(ptr)) __user *__p = (ptr); \
+ might_fault(); \
+ if (likely(access_ok(__p, sizeof(*__p)))) { \
+ __e = __get_user(x, __p); \
+ } else { \
+ x = 0; \
+ } \
+ __e; \
+})
+
+/**
+ * __get_user: - Get a simple variable from user space, with less checking.
+ * @x: Variable to store result.
+ * @ptr: Source address, in user space.
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * This macro copies a single simple variable from user space to kernel
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and the result of
+ * dereferencing @ptr must be assignable to @x without a cast.
+ *
+ * Caller must check the pointer with access_ok() before calling this
+ * function.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ * On error, the variable @x is set to zero.
+ */
+#define __get_user(x, ptr) \
+({ \
+ unsigned long __err = 0; \
+ __chk_user_ptr(ptr); \
+ \
+ __enable_user_access(); \
+ __get_user_nocheck(x, ptr, __err); \
+ __disable_user_access(); \
+ \
+ __err; \
+})
+
+#define __get_user_nocheck(x, ptr, err) \
+do { \
+ unsigned long __gu_addr = (unsigned long)(ptr); \
+ unsigned long __gu_val; \
+ switch (sizeof(*(ptr))) { \
+ case 1: \
+ __get_user_asm("lbz", __gu_val, __gu_addr, err); \
+ break; \
+ case 2: \
+ __get_user_asm("lhz", __gu_val, __gu_addr, err); \
+ break; \
+ case 4: \
+ __get_user_asm("lwz", __gu_val, __gu_addr, err); \
+ break; \
+ case 8: \
+ __get_user_asm("ld", __gu_val, __gu_addr, err); \
+ break; \
+ default: \
+ BUILD_BUG(); \
+ } \
+ (x) = (__typeof__(*(ptr)))__gu_val; \
+} while (0)
+
+#define __get_user_asm(op, x, addr, err) \
+({ \
+ __asm__ __volatile__( \
+ "1: "op" %1 = 0[%2]\n" \
+ " ;;\n" \
+ "2:\n" \
+ ".section .fixup,\"ax\"\n" \
+ "3: make %0 = 2b\n" \
+ " make %1 = 0\n" \
+ " ;;\n" \
+ " make %0 = %3\n" \
+ " igoto %0\n" \
+ " ;;\n" \
+ ".previous\n" \
+ ".section __ex_table,\"a\"\n" \
+ " .align 8\n" \
+ " .dword 1b,3b\n" \
+ ".previous" \
+ : "=r"(err), "=r"(x) \
+ : "r"(addr), "i"(-EFAULT), "0"(err)); \
+})
+
+/**
+ * put_user: - Write a simple value into user space.
+ * @x: Value to copy to user space.
+ * @ptr: Destination address, in user space.
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * This macro copies a single simple value from kernel space to user
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and @x must be assignable
+ * to the result of dereferencing @ptr.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ */
+#define put_user(x, ptr) \
+({ \
+ long __e = -EFAULT; \
+ __typeof__(*(ptr)) __user *__p = (ptr); \
+ might_fault(); \
+ if (likely(access_ok(__p, sizeof(*__p)))) { \
+ __e = __put_user(x, __p); \
+ } \
+ __e; \
+})
+
+/**
+ * __put_user: - Write a simple value into user space, with less checking.
+ * @x: Value to copy to user space.
+ * @ptr: Destination address, in user space.
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * This macro copies a single simple value from kernel space to user
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and @x must be assignable
+ * to the result of dereferencing @ptr.
+ *
+ * Caller must check the pointer with access_ok() before calling this
+ * function.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ */
+#define __put_user(x, ptr) \
+({ \
+ unsigned long __err = 0; \
+ __chk_user_ptr(ptr); \
+ \
+ __enable_user_access(); \
+ __put_user_nocheck(x, ptr, __err); \
+ __disable_user_access(); \
+ \
+ __err; \
+})
+
+#define __put_user_nocheck(x, ptr, err) \
+do { \
+ unsigned long __pu_addr = (unsigned long)(ptr); \
+ __typeof__(*(ptr)) __pu_val = (x); \
+ switch (sizeof(*(ptr))) { \
+ case 1: \
+ __put_user_asm("sb", __pu_val, __pu_addr, err); \
+ break; \
+ case 2: \
+ __put_user_asm("sh", __pu_val, __pu_addr, err); \
+ break; \
+ case 4: \
+ __put_user_asm("sw", __pu_val, __pu_addr, err); \
+ break; \
+ case 8: \
+ __put_user_asm("sd", __pu_val, __pu_addr, err); \
+ break; \
+ default: \
+ BUILD_BUG(); \
+ } \
+} while (0)
+
+#define __put_user_asm(op, x, addr, err) \
+({ \
+ __asm__ __volatile__( \
+ "1: "op" 0[%2], %1\n" \
+ " ;;\n" \
+ "2:\n" \
+ ".section .fixup,\"ax\"\n" \
+ "3: make %0 = 2b\n" \
+ " ;;\n" \
+ " make %0 = %3\n" \
+ " igoto %0\n" \
+ " ;;\n" \
+ ".previous\n" \
+ ".section __ex_table,\"a\"\n" \
+ " .align 8\n" \
+ " .dword 1b,3b\n" \
+ ".previous" \
+ : "=r"(err) \
+ : "r"(x), "r"(addr), "i"(-EFAULT), "0"(err)); \
+})
+
+#define HAVE_GET_KERNEL_NOFAULT
+
+#define __get_kernel_nofault(dst, src, type, err_label) \
+do { \
+ long __kr_err; \
+ \
+ __get_user_nocheck(*((type *)(dst)), (type *)(src), __kr_err); \
+ if (unlikely(__kr_err)) \
+ goto err_label; \
+} while (0)
+
+#define __put_kernel_nofault(dst, src, type, err_label) \
+do { \
+ long __kr_err; \
+ \
+ __put_user_nocheck(*((type *)(src)), (type *)(dst), __kr_err); \
+ if (unlikely(__kr_err)) \
+ goto err_label; \
+} while (0)
+
+
+#endif /* _ASM_KVX_UACCESS_H */
diff --git a/arch/kvx/include/asm/unistd.h b/arch/kvx/include/asm/unistd.h
new file mode 100644
index 000000000000..6cd093dbf2d8
--- /dev/null
+++ b/arch/kvx/include/asm/unistd.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#define __ARCH_WANT_SYS_CLONE
+
+#include <uapi/asm/unistd.h>
+
+#define NR_syscalls (__NR_syscalls)
diff --git a/arch/kvx/include/uapi/asm/cachectl.h b/arch/kvx/include/uapi/asm/cachectl.h
new file mode 100644
index 000000000000..be0a1aa23cf6
--- /dev/null
+++ b/arch/kvx/include/uapi/asm/cachectl.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _UAPI_ASM_KVX_CACHECTL_H
+#define _UAPI_ASM_KVX_CACHECTL_H
+
+/*
+ * Cache type for cachectl system call
+ */
+#define CACHECTL_CACHE_DCACHE (1 << 0)
+
+/*
+ * Flags for cachectl system call
+ */
+#define CACHECTL_FLAG_OP_INVAL (1 << 0)
+#define CACHECTL_FLAG_OP_WB (1 << 1)
+#define CACHECTL_FLAG_OP_MASK (CACHECTL_FLAG_OP_INVAL | \
+ CACHECTL_FLAG_OP_WB)
+
+#define CACHECTL_FLAG_ADDR_PHYS (1 << 2)
+
+#endif /* _UAPI_ASM_KVX_CACHECTL_H */
diff --git a/arch/kvx/include/uapi/asm/unistd.h b/arch/kvx/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..5f86d81dbb76
--- /dev/null
+++ b/arch/kvx/include/uapi/asm/unistd.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#define __ARCH_WANT_RENAMEAT
+#define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+#define __ARCH_WANT_SYS_CLONE3
+
+#include <asm-generic/unistd.h>
+
+/* Additional KVX specific syscalls */
+#define __NR_cachectl (__NR_arch_specific_syscall)
+__SYSCALL(__NR_cachectl, sys_cachectl)
diff --git a/arch/kvx/kernel/entry.S b/arch/kvx/kernel/entry.S
new file mode 100644
index 000000000000..2d16086d29dc
--- /dev/null
+++ b/arch/kvx/kernel/entry.S
@@ -0,0 +1,1759 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Marius Gligor
+ * Yann Sionneau
+ * Julien Villette
+ */
+
+#include <linux/linkage.h>
+
+#include <asm/thread_info.h>
+#include <asm/asm-offsets.h>
+#include <asm/cache.h>
+#include <asm/page.h>
+#include <asm/pgtable-bits.h>
+#include <asm/sys_arch.h>
+#include <asm/sfr_defs.h>
+#include <asm/tlb_defs.h>
+#include <asm/mem_map.h>
+#include <asm/traps.h>
+#include <asm/unistd.h>
+
+#define MMU_SET_MASK ((1 << KVX_SFR_MMC_SS_WIDTH) - 1)
+
+/* Mask to replicate data from 32bits LSB to MSBs */
+#define REPLICATE_32_MASK 0x0804020108040201
+
+#define PS_HWLOOP_ENABLE (KVX_SFR_PS_HLE_MASK << 32)
+#define PS_HWLOOP_DISABLE (KVX_SFR_PS_HLE_MASK)
+#define PS_HWLOOP_EN_ET_CLEAR_DAUS_DIS \
+ (PS_HWLOOP_ENABLE | KVX_SFR_PS_ET_MASK | SFR_SET_VAL_WFXL(PS, DAUS, 0))
+#define PS_ET_CLEAR KVX_SFR_PS_ET_MASK
+#define PS_HLE_EN_IT_EN_ET_CLEAR_DAUS_DIS \
+ (SFR_SET_VAL_WFXL(PS, HLE, 1) | \
+ SFR_SET_VAL_WFXL(PS, IE, 1) | \
+ SFR_SET_VAL_WFXL(PS, ET, 0) | \
+ SFR_SET_VAL_WFXL(PS, DAUS, 0))
+
+#define PS_IT_DIS KVX_SFR_PS_IE_MASK
+#define PS_IL_CLEAR KVX_SFR_PS_IL_WFXL_CLEAR
+
+#define MMC_CLEAR_TLB_CLEAR_WAY \
+ (SFR_CLEAR_WFXL(MMC, SB) | SFR_CLEAR_WFXL(MMC, SW))
+
+#define MMC_SEL_TLB_CLEAR_WAY(__tlb) \
+ (SFR_SET_WFXL(MMC, SB, __tlb) | MMC_CLEAR_TLB_CLEAR_WAY)
+#define MMC_SEL_JTLB_CLEAR_WAY MMC_SEL_TLB_CLEAR_WAY(MMC_SB_JTLB)
+#define MMC_SEL_LTLB_CLEAR_WAY MMC_SEL_TLB_CLEAR_WAY(MMC_SB_LTLB)
+
+#define MME_WFXL(_enable) SFR_SET_VAL_WFXL(PS, MME, _enable)
+
+/* Temporary scract system register for trap handling */
+#define TMP_SCRATCH_SR $sr_pl3
+
+.altmacro
+
+#define TEL_DEFAULT_VALUE (TLB_ES_A_MODIFIED << KVX_SFR_TEL_ES_SHIFT)
+
+#define TASK_THREAD_SAVE_AREA_QUAD(__q) \
+ (TASK_THREAD_SAVE_AREA + (__q) * QUAD_REG_SIZE)
+
+#ifdef CONFIG_DEBUG_EXCEPTION_STACK
+.section .rodata
+stack_error_panic_str_label:
+ .string "Stack has been messed up !"
+#endif
+
+#ifdef CONFIG_KVX_DEBUG_TLB_WRITE
+.section .rodata
+mmc_error_panic_str_label:
+ .string "Failed to write entry to the JTLB (in assembly refill)"
+#endif
+
+#ifdef CONFIG_KVX_DEBUG_ASN
+.section .rodata
+asn_error_panic_str_label:
+ .string "ASN mismatch !"
+#endif
+
+/**
+ * call_trace_hardirqs: hardirqs tracing call
+ * state: State of hardirqs to be reported (on/off)
+ */
+.macro call_trace_hardirqs state
+#ifdef CONFIG_TRACE_IRQFLAGS
+ call trace_hardirqs_\state
+ ;;
+#endif
+.endm
+
+/**
+ * disable_interrupts: Disable interrupts
+ * tmp_reg: Temporary register to use for interrupt disabling
+ */
+.macro disable_interrupt tmp_reg
+ make \tmp_reg = KVX_SFR_PS_IE_MASK
+ ;;
+ wfxl $ps, \tmp_reg
+ ;;
+.endm
+
+/**
+ * call_do_work_pending: Call do_work_pending and set stack argument ($r0)
+ * NOTE: Since do_work_pending requires thread_flags in $r1, they must
+ * be provided in $r1 before calling this macro.
+ * Moreover, do_work_pending expects interrupts to be disabled.
+ */
+.macro call_do_work_pending
+ copyd $r0 = $sp
+ call do_work_pending
+ ;;
+.endm
+
+/**
+ * save_quad_regs: Save quad registers in temporary thread area
+ * After call, the quad is saved in task_struct.thread.save_area.
+ * sr_swap_reg is only used as a temporary value and will be
+ * restored after returning from this macro
+ *
+ * quad: Quad to saved
+ * sr_swap_reg: Register used for sr swap and quad saving.
+ */
+.macro save_quad_regs quad sr_swap_reg
+ rswap \sr_swap_reg = $sr
+ ;;
+ /* Save registers in save_area */
+ so __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[\sr_swap_reg] = \quad
+ /* Restore register */
+ rswap \sr_swap_reg = $sr
+ ;;
+.endm
+
+/**
+ * Switch task when entering kernel if needed.
+ * sp_reg: register which will be used to store original stack pointer when
+ * entering the kernel
+ * task_reg: is a register containing the current task pointer
+ * save_regs_label: label to jump to save register immediately. This label is
+ * used when we are already in kernel execution context.
+ * NOTE: when returning from this macro, we consider that the assembly following
+ * it will be executed only when coming from user space
+ */
+.macro switch_stack sp_reg task_reg save_regs_label
+ get \sp_reg = $sps
+ ;;
+ /* Check if $sps.pl bit is 0 (ie in kernel mode)
+ * if so, then we dont have to switch stack */
+ cb.even \sp_reg? \save_regs_label
+ /* Copy original $sp in a scratch reg */
+ copyd \sp_reg = $sp
+ ;;
+ /* restore sp from kernel stack pointer */
+ ld $sp = __PA(TASK_THREAD_KERNEL_SP)[\task_reg]
+ ;;
+.endm
+
+/**
+ * Disable MMU using a specified register
+ * scratch_reg: Scratch register to be used for $ps modification (clobbered)
+ * ps_val: Additionnal $ps modifications after returning disabling MMU
+ */
+.macro disable_mmu scratch_reg ps_val=0
+ /* Disable MME in $sps */
+ make \scratch_reg = MME_WFXL(0)
+ ;;
+ wfxl $sps = \scratch_reg
+ ;;
+ /* Enable DAUS in $ps */
+ make \scratch_reg = SFR_SET_VAL_WFXL(PS, DAUS, 1) | \ps_val
+ ;;
+ wfxl $ps = \scratch_reg
+ /* We are now accessing data with MMU disabled */
+ ;;
+.endm
+
+/**
+ * Disable MMU when entering exception path
+ * This macro does not require any register since used register will be
+ * restored after use. This can safely be used directly after entering
+ * exceptions.
+ */
+.macro exception_entry_disable_mmu
+ set TMP_SCRATCH_SR = $r0
+ disable_mmu $r0
+ ;;
+ get $r0 = TMP_SCRATCH_SR
+ ;;
+.endm
+
+/* Save callee registers on stack */
+.macro save_callee
+ sq PT_R18R19[$sp] = $r18r19
+ ;;
+ so PT_Q20[$sp] = $r20r21r22r23
+ ;;
+ so PT_Q24[$sp] = $r24r25r26r27
+ ;;
+ so PT_Q28[$sp] = $r28r29r30r31
+ ;;
+.endm
+
+/**
+ * Save registers for entry in kernel space.
+ * sp_reg should point to the original stack pointer when entering kernel space
+ * task_reg is a register containing the current task pointer
+ * both task_reg and sp_reg must be in saved_quad since they have been cloberred
+ * and will be restored when restoring saved_quad.
+ * pt_regs_sp is a single register which will be filled by pt_regs addr.
+ * When "returning" from this macro, hardware loop are enabled and exception
+ * is cleared, allowing to call kernel function without any risk.
+ */
+.macro save_regs_for_exception sp_reg task_reg saved_quad pt_regs_sp
+.if (\saved_quad==$r4r5r6r7 || \saved_quad==$r60r61r62r63)
+.error "saved_quad must not be $r4r5r6r7 or $r60r61r62r63 !"
+.endif
+ /* make some room on stack to save registers */
+ addd $sp = $sp, -(PT_SIZE_ON_STACK)
+ so __PA(PT_Q4-PT_SIZE_ON_STACK)[$sp] = $r4r5r6r7
+ /* Compute physical address of stack to avoid using 64bits immediate */
+ addd $r6 = $sp, VA_TO_PA_OFFSET - (PT_SIZE_ON_STACK)
+ ;;
+ so PT_Q60[$r6] = $r60r61r62r63
+ /* Now that $r60r61r62r63 is saved, we can use it for saving
+ * original stack stored in scratch_reg. Note that we can not
+ * use the r12r13r14r15 quad to do that because it would
+ * modify the current $r12/sp ! This is if course not what we
+ * want and hence we use the freshly saved quad $r60r61r62r63.
+ *
+ * Note that we must use scratch_reg before reloading the saved
+ * quad since the scratch reg is contained in it, so reloading
+ * it before copying it would overwrite it.
+ */
+ copyd $r60 = \sp_reg
+ ;;
+ /* Reload the saved quad registers to save correct values
+ * Since we use the scratch reg before that */
+ lo \saved_quad = __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[\task_reg]
+ ;;
+ so PT_Q8[$r6] = $r8r9r10r11
+ ;;
+ so PT_Q0[$r6] = $r0r1r2r3
+ copyd $r61 = $r13
+ get $r5 = $le
+ ;;
+ sq PT_R16R17[$r6] = $r16r17
+ make $r10 = 0x0
+ copyd $r62 = $r14
+ ;;
+ so PT_Q32[$r6] = $r32r33r34r35
+ /* Since we are going to enable hardware loop, we must be careful
+ * and reset le (loop exit) to avoid any exploit and return to
+ * user with kernel mode */
+ set $le = $r10
+ copyd $r63 = $r15
+ ;;
+ so PT_Q36[$r6] = $r36r37r38r39
+ get $r0 = $cs
+ ;;
+ so PT_Q40[$r6] = $r40r41r42r43
+ get $r1 = $spc
+ ;;
+ so PT_Q44[$r6] = $r44r45r46r47
+ get $r2 = $sps
+ ;;
+ so PT_Q48[$r6] = $r48r49r50r51
+ get $r3 = $es
+ ;;
+ so PT_Q52[$r6] = $r52r53r54r55
+ get $r7 = $ra
+ ;;
+ so PT_Q56[$r6] = $r56r57r58r59
+ get $r4 = $lc
+ ;;
+ so PT_Q12[$r6] = $r60r61r62r63
+ copyd $r63 = $r6
+ get $r6 = $ls
+ ;;
+ so PT_CS_SPC_SPS_ES[$r63] = $r0r1r2r3
+ ;;
+ so PT_LC_LE_LS_RA[$r63] = $r4r5r6r7
+ /* Clear frame pointer */
+ make $fp = 0
+ ;;
+ /* Copy regs stack pointer for macro caller */
+ copyd \pt_regs_sp = $sp
+#ifdef CONFIG_DEBUG_EXCEPTION_STACK
+ addd $sp = $sp, -STACK_REG_SIZE
+ ;;
+ sd VA_TO_PA_OFFSET[$sp] = $sp
+ ;;
+#endif
+ /* Reenable hwloop, MMU and clear exception taken */
+ make $r8 = PS_HWLOOP_EN_ET_CLEAR_DAUS_DIS
+ ;;
+ wfxl $ps, $r8
+ ;;
+.endm
+
+/***********************************************************************
+* Exception vectors trampolines
+***********************************************************************/
+#define exception_trampoline(__type) \
+.section .exception.## __type, "ax", @progbits ;\
+ENTRY(kvx_##__type ##_handler_trampoline): ;\
+ goto kvx_## __type ##_handler ;\
+ ;; ;\
+ENDPROC(kvx_ ## __type ## _handler_trampoline) ;\
+.section .early_exception.## __type, "ax", @progbits ;\
+ENTRY(kvx_## __type ##_early_handler): ;\
+1: nop ;\
+ ;; ;\
+ goto 1b ;\
+ ;; ;\
+ENDPROC(kvx_ ## __type ## _early_handler)
+
+exception_trampoline(debug)
+exception_trampoline(trap)
+exception_trampoline(interrupt)
+exception_trampoline(syscall)
+
+#define EXCEPTION_ENTRY(__name) \
+.section .exception.text, "ax", @progbits ;\
+ENTRY(__name)
+
+
+/***********************************************************************
+* Common exception return path
+***********************************************************************/
+/**
+ * Restore registers after exception
+ * When entering this macro, $sp must be located right before regs
+ * storage.
+ */
+EXCEPTION_ENTRY(return_from_exception)
+#ifdef CONFIG_DEBUG_EXCEPTION_STACK
+ ld $r1 = 0[$sp]
+ ;;
+ sbfd $r1 = $r1, $sp
+ ;;
+ cb.deqz $r1, _check_ok
+ ;;
+ make $r2 = panic
+ make $r0 = stack_error_panic_str_label
+ ;;
+ icall $r2
+ ;;
+_check_ok:
+ addd $sp = $sp, STACK_REG_SIZE
+ ;;
+#endif
+ get $r11 = $sr
+ /* Load sps value from saved registers */
+ ld $r6 = PT_SPS[$sp]
+ ;;
+ /* Disable interrupt to check task flags atomically */
+ disable_interrupt $r60
+ ;;
+ /* Check PL bit of sps, if set, then it means we are returning
+ * to a lower privilege level (ie to user), if so, we need to
+ * check work pending. If coming from kernel, directly go to
+ * register restoration */
+ cb.even $r6? _restore_regs
+ ld $r1 = TASK_TI_FLAGS[$r11]
+ ;;
+ /* Do we have work pending ? */
+ andd $r5 = $r1, _TIF_WORK_MASK
+ ;;
+ /**
+ * If we do not have work pending (ie $r5 == 0) then we can
+ * directly jump to _restore_regs without calling do_work_pending
+ */
+ cb.deqz $r5? _restore_regs
+ ;;
+ /*
+ * Work pending can potentially call a signal handler and then return
+ * via rt_sigreturn. Return path will be different (restore all regs)
+ * and hence all registers are needed to be saved.
+ */
+ save_callee
+ ;;
+ call_do_work_pending
+ ;;
+#ifdef CONFIG_TRACE_IRQFLAGS
+ /* reload sps value from saved registers */
+ ld $r6 = PT_SPS[$sp]
+ ;;
+#endif
+_restore_regs:
+#ifdef CONFIG_TRACE_IRQFLAGS
+ /* Check if IRQs are going to be reenable in next context */
+ andd $r6 = $r6, KVX_SFR_SPS_IE_MASK
+ ;;
+ cb.deqz $r6? 1f
+ ;;
+ call trace_hardirqs_on
+ ;;
+1:
+#endif
+ disable_mmu $r0, PS_HWLOOP_DISABLE
+ ;;
+ lo $r0r1r2r3 = __PA(PT_CS_SPC_SPS_ES)[$sp]
+ /* Compute physical address of stack to avoid using 64bits immediate */
+ addd $r11 = $sp, VA_TO_PA_OFFSET
+ ;;
+ lo $r4r5r6r7 = PT_LC_LE_LS_RA[$r11]
+ ;;
+ lo $r60r61r62r63 = PT_Q60[$r11]
+ ;;
+ lo $r56r57r58r59 = PT_Q56[$r11]
+ ;;
+ lo $r52r53r54r55 = PT_Q52[$r11]
+ get $r14 = $sps
+ ;;
+ lo $r48r49r50r51 = PT_Q48[$r11]
+ /* Generate a mask of ones at each bit where the current $sps
+ * differs from the $sps to be restored
+ */
+ xord $r14 = $r2, $r14
+ /* prepare wfxl clear mask on LSBs */
+ notd $r15 = $r2
+ /* prepare wfxl set mask on MSBs */
+ slld $r13 = $r2, 32
+ ;;
+ lo $r44r45r46r47 = PT_Q44[$r11]
+ /* Replicate mask of ones on the 32 MSBs */
+ sbmm8 $r14 = $r14, REPLICATE_32_MASK
+ /* Combine the set and clear mask for wfxl */
+ insf $r13 = $r15, 31, 0
+ ;;
+ lo $r40r41r42r43 = PT_Q40[$r11]
+ set $lc = $r4
+ /* Mask to drop identical bits in order to avoid useless
+ * privilege traps
+ */
+ andd $r13 = $r13, $r14
+ ;;
+ lq $r16r17 = PT_R16R17[$r11]
+ set $le = $r5
+ ;;
+ lo $r32r33r34r35 = PT_Q32[$r11]
+ set $ls = $r6
+ ;;
+ lo $r36r37r38r39 = PT_Q36[$r11]
+ copyd $r14 = $r11
+ set $ra = $r7
+ ;;
+ lo $r8r9r10r11 = PT_Q8[$r14]
+ set $cs = $r0
+ /* MME was disabled by disable_mmu, reenable it before leaving */
+ ord $r13 = $r13, SFR_SET_VAL_WFXL(SPS, MME, 1)
+ ;;
+ lo $r4r5r6r7 = PT_Q4[$r14]
+ set $spc = $r1
+ ;;
+ lo $r0r1r2r3 = PT_Q0[$r14]
+ /* Store $sps wfxl value in scratch system register */
+ set TMP_SCRATCH_SR = $r13
+ ;;
+ lo $r12r13r14r15 = PT_Q12[$r14]
+ rswap $r0 = TMP_SCRATCH_SR
+ ;;
+ wfxl $sps = $r0
+ ;;
+ /* Finally, restore $r0 value */
+ rswap $r0 = TMP_SCRATCH_SR
+ ;;
+ rfe
+ ;;
+ENDPROC(return_from_exception)
+
+/***********************************************************************
+* Debug handling
+***********************************************************************/
+EXCEPTION_ENTRY(kvx_debug_handler):
+ exception_entry_disable_mmu
+ ;;
+ save_quad_regs $r0r1r2r3 $r4
+ ;;
+ get $r2 = $sr
+ ;;
+ switch_stack $r1 $r2 debug_save_regs
+ ;;
+debug_save_regs:
+ save_regs_for_exception $r1 $r2 $r0r1r2r3 $r1
+ ;;
+ get $r0 = $ea
+ /* tail-call for return_from_exception */
+ make $r3 = return_from_exception
+ ;;
+ set $ra = $r3
+ ;;
+ goto debug_handler
+ ;;
+ENDPROC(kvx_debug_handler)
+
+/***********************************************************************
+* Traps handling
+***********************************************************************/
+/* These labels will be used for instruction patching */
+.global kvx_perf_tlb_refill, kvx_std_tlb_refill
+EXCEPTION_ENTRY(kvx_trap_handler):
+ /* Enable DAUS for physical data access */
+ exception_entry_disable_mmu
+ ;;
+ /* Save r3 in a temporary system register to check if the trap is a
+ * nomapping or not */
+ set TMP_SCRATCH_SR = $r3
+ ;;
+ get $r3 = $es
+ ;;
+ /* Hardware trap cause */
+ extfz $r3 = $r3, KVX_SFR_END(ES_HTC), KVX_SFR_START(ES_HTC)
+ ;;
+ /* Is this a nomapping trap ? */
+ compd.eq $r3 = $r3, KVX_TRAP_NOMAPPING
+ ;;
+ /* if nomapping trap, try fast_refill */
+ cb.even $r3? trap_slow_path
+ ;;
+ /*
+ * Fast TLB refill routine
+ *
+ * On kvx, we do not have hardware page walking, hence, TLB refill is
+ * done using the core on no-mapping traps.
+ * This routine must be as fast as possible to avoid wasting CPU time.
+ * For that purpose, it is called directly from trap_handle after saving
+ * only 8 registers ($r0 -> $r7) in a dedicated buffer.
+ * To avoid taking nomapping while accessing page tables inside this
+ * refill handler we switch to physical accesses using DAUS.
+ * Once the switch is done, we save up to 8 registers to compute
+ * the refill data.
+ * This allows to avoid computing a complete task switching in order
+ * to greatly reduce the refill time.
+ *
+ * We refill the JTLB which contains 128 sets with 4 way each.
+ * Currently, the way selection is done using a round robin algorithm.
+ *
+ * The refill is using the basic flow:
+ * 1 - Enable physical access using DAUS.
+ * 2 - Save necessary registers
+ * 3 - Walk the page table to find the TLB entry to add (virtual to
+ * physical)
+ * 4 - Compute the TLB entry to be written (convert PTE to TLB entry)
+ * 5 - Compute the target set (0 -> 127) for the new TLB entry
+ * This is done by extracting the 6 lsb of page number
+ * 6 - Get the current way to be used which is selected using a
+ simple round robin
+ * 7 - Mark PTE entry as _PAGE_ACCESSED (and optionally PAGE_DIRTY)
+ * 8 - Commit the new tlb entry
+ * 9 - Restore the virtual memory by disabling DAUS.
+ *
+ */
+ /* Get current task */
+ rswap $r63 = $sr
+ ;;
+#ifdef CONFIG_KVX_MMU_STATS
+ get $r3 = $pm0
+ ;;
+ sd __PA(TASK_THREAD_ENTRY_TS)[$r63] = $r3
+ ;;
+#endif
+ /* Restore $r3 from temporary system scratch register */
+ get $r3 = TMP_SCRATCH_SR
+ /* Save registers to save $tel, $teh and $mmc */
+ so __PA(TASK_THREAD_SAVE_AREA_QUAD(2))[$r63] = $r8r9r10r11
+ ;;
+ get $r8 = $tel
+ /* Save register for refill handler */
+ so __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r63] = $r4r5r6r7
+ ;;
+ /* Get exception address */
+ get $r0 = $ea
+ /* Save more registers to be comfy */
+ so __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[$r63] = $r0r1r2r3
+ ;;
+ get $r9 = $teh
+ ;;
+ get $r10 = $mmc
+ ;;
+ /* Restore $r63 value */
+ rswap $r63 = $sr
+ /* Check kernel address range */
+ addd $r4 = $r0, -KERNEL_DIRECT_MEMORY_MAP_BASE
+ /* Load task active mm for pgd loading in macro_tlb_refill */
+ ld $r1 = __PA(TASK_ACTIVE_MM)[$r63]
+ ;;
+kvx_perf_tlb_refill:
+ /* Check if the address is in the kernel direct memory mapping */
+ compd.ltu $r3 = $r4, KERNEL_DIRECT_MEMORY_MAP_SIZE
+ /* Clear low bits of virtual address to align on page size */
+ andd $r5 = $r0, ~(REFILL_PERF_PAGE_SIZE - 1)
+ /* Create corresponding physical address */
+ addd $r2 = $r4, PHYS_OFFSET
+ ;;
+ /* If address is not a kernel one, take the standard path */
+ cb.deqz $r3? kvx_std_tlb_refill
+ /* Prepare $teh value with virtual address and kernel value */
+ ord $r7 = $r5, REFILL_PERF_TEH_VAL
+ ;;
+ /* Get $pm0 value as a pseudo random value for LTLB way to use */
+ get $r4 = $pm0
+ /* Clear low bits of physical address to align on page size */
+ andd $r2 = $r2, ~(REFILL_PERF_PAGE_SIZE - 1)
+ /* Prepare value for $mmc wfxl to select LTLB and correct way */
+ make $r5 = MMC_SEL_LTLB_CLEAR_WAY
+ ;;
+ /* Keep low bits of timer value */
+ andw $r4 = $r4, (REFILL_PERF_ENTRIES - 1)
+ /* Get current task pointer for register restoration */
+ get $r11 = $sr
+ ;;
+ /* Add LTLB base way number for kernel refill way */
+ addw $r4 = $r4, LTLB_KERNEL_RESERVED
+ /* Prepare $tel value with physical address and kernel value */
+ ord $r6 = $r2, REFILL_PERF_TEL_VAL
+ set $teh = $r7
+ ;;
+ /* insert way in $mmc wfxl value */
+ insf $r5 = $r4, KVX_SFR_END(MMC_SW) + 32, KVX_SFR_START(MMC_SW) + 32
+ set $tel = $r6
+ ;;
+ wfxl $mmc = $r5
+ ;;
+ goto do_tlb_write
+ ;;
+kvx_std_tlb_refill:
+ /* extract PGD offset */
+ extfz $r3 = $r0, (ASM_PGDIR_SHIFT + ASM_PGDIR_BITS - 1), ASM_PGDIR_SHIFT
+ /* is mm NULL ? if so, use init_mm */
+ cmoved.deqz $r1? $r1 = init_mm
+ ;;
+ get $r7 = $pcr
+ /* Load pgd base address into $r1 */
+ ld $r1 = __PA(MM_PGD)[$r1]
+ ;;
+ /* Add offset for physical address */
+ addd $r1 = $r1, VA_TO_PA_OFFSET
+ /* Extract processor ID to compute cpu_offset*/
+ extfz $r7 = $r7, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID)
+ ;;
+ /* Load PGD entry offset */
+ ld.xs $r1 = $r3[$r1]
+ /* Load per_cpu_offset */
+#if defined(CONFIG_SMP)
+ make $r5 = __PA(__per_cpu_offset)
+#endif
+ ;;
+ /* extract PMD offset*/
+ extfz $r3 = $r0, (ASM_PMD_SHIFT + ASM_PMD_BITS - 1), ASM_PMD_SHIFT
+ /* If pgd entry is null -> out */
+ cb.deqz $r1? refill_err_out
+#if defined(CONFIG_SMP)
+ /* Load cpu offset */
+ ld.xs $r7 = $r7[$r5]
+#else
+ /* Force cpu offset to 0 */
+ make $r7 = 0
+#endif
+ ;;
+ /* Load PMD entry offset and keep pointer to the entry for huge page */
+ addx8d $r2 = $r3, $r1
+ ld.xs $r1 = $r3[$r1]
+ ;;
+ /* Check if it is a huge page (2Mb or 512Mb in PMD table)*/
+ andd $r6 = $r1, _PAGE_HUGE
+ /* If pmd entry is null -> out */
+ cb.deqz $r1? refill_err_out
+ /* extract PTE offset */
+ extfz $r3 = $r0, (PAGE_SHIFT + 8), PAGE_SHIFT
+ ;;
+ /*
+ * If the page is a huge one we already have set the PTE and the
+ * pointer to the PTE.
+ */
+ cb.dnez $r6? is_huge_page
+ ;;
+ /* Load PTE entry */
+ ld.xs $r1 = $r3[$r1]
+ addx8d $r2 = $r3, $r1
+ ;;
+ /* Check if it is a huge page (64Kb in PTE table) */
+ andd $r6 = $r1, _PAGE_HUGE
+ ;;
+ /* Check if PTE entry is for a huge page */
+ cb.dnez $r6? is_huge_page
+ ;;
+ /* 4K: Extract set value */
+ extfz $r0 = $r1, (PAGE_SHIFT + KVX_SFR_MMC_SS_WIDTH - 1), PAGE_SHIFT
+ /* 4K: Extract virt page from ea */
+ andd $r4 = $r0, PAGE_MASK
+ ;;
+/*
+ * This path expects the following:
+ * - $r0 = set index
+ * - $r1 = pte entry
+ * - $r2 = pte entry address
+ * - $r4 = virtual page address
+ */
+pte_prepared:
+ /* Compute per_cpu_offset + current way of set address */
+ addd $r5 = $r0, $r7
+ /* Get exception cause for access type handling (page dirtying) */
+ get $r7 = $es
+ /* Clear way and select JTLB */
+ make $r6 = MMC_SEL_JTLB_CLEAR_WAY
+ ;;
+ /* Load current way to use for current set */
+ lbz $r0 = __PA(jtlb_current_set_way)[$r5]
+ /* Check if the access was a "write" access */
+ andd $r7 = $r7, (KVX_TRAP_RWX_WRITE << KVX_SFR_ES_RWX_SHIFT)
+ ;;
+ /* If bit PRESENT of pte entry is 0, then entry is not present */
+ cb.even $r1? refill_err_out
+ /*
+ * Set the JTLB way in $mmc value, add 32 bits to be in the set part.
+ * Since we are refilling JTLB, we must make sure we insert only
+ * relevant bits (ie 2 bits for ways) to avoid using nonexistent ways.
+ */
+ insf $r6 = $r0, KVX_SFR_START(MMC_SW) + 32 + (MMU_JTLB_WAYS_SHIFT - 1),\
+ KVX_SFR_START(MMC_SW) + 32
+ /* Extract page global bit */
+ extfz $r3 = $r1, _PAGE_GLOBAL_SHIFT, _PAGE_GLOBAL_SHIFT
+ /* Increment way value, note that we do not care about overflow since
+ * we only use the two lower byte */
+ addd $r0 = $r0, 1
+ ;;
+ /* Prepare MMC */
+ wfxl $mmc, $r6
+ ;;
+ /* Insert global bit (if any) to its position into $teh value */
+ insf $r4 = $r3, KVX_SFR_TEH_G_SHIFT, KVX_SFR_TEH_G_SHIFT
+ /* If "write" access ($r7 != 0), then set it as dirty */
+ cmoved.dnez $r7? $r7 = _PAGE_DIRTY
+ /* Clear bits not related to FN in the pte entry for TEL writing */
+ andd $r6 = $r1, KVX_PFN_MASK
+ /* Store new way */
+ sb __PA(jtlb_current_set_way)[$r5] = $r0
+ ;;
+ /* Extract access perms from pte entry (discard PAGE_READ bit +1) */
+ extfz $r3 = $r1, KVX_ACCESS_PERM_STOP_BIT, KVX_ACCESS_PERM_START_BIT + 1
+ /* Move FN bits to their place */
+ srld $r6 = $r6, KVX_PFN_SHIFT - PAGE_SHIFT
+ /* Extract the page size + cache policy */
+ andd $r0 = $r1, (KVX_PAGE_SZ_MASK | KVX_PAGE_CP_MASK)
+ /* Prepare SBMM value */
+ make $r5 = KVX_SBMM_BYTE_SEL
+ ;;
+ /* Add page size + cache policy to $tel value */
+ ord $r6 = $r6, $r0
+ /* Get $mmc to get current ASN */
+ get $r0 = $mmc
+ /* Add _PAGE_ACCESSED bit to PTE entry for writeback */
+ ord $r7 = $r7, _PAGE_ACCESSED
+ ;;
+ /* OR PTE value with accessed/dirty flags */
+ ord $r1 = $r1, $r7
+ /* Generate the byte selection for sbmm */
+ slld $r5 = $r5, $r3
+ /* Compute the mask to extract set and mask exception address */
+ make $r7 = KVX_PAGE_PA_MATRIX
+ ;;
+ ord $r0 = $r6, TEL_DEFAULT_VALUE
+ /* Add ASN from mmc into future $teh value */
+ insf $r4 = $r0, KVX_SFR_END(MMC_ASN), KVX_SFR_START(MMC_ASN)
+ /* Get the page permission value */
+ sbmm8 $r6 = $r7, $r5
+ /* Check PAGE_READ bit in PTE entry */
+ andd $r3 = $r1, _PAGE_READ
+ ;;
+ /* If PAGE_READ bit is not set, set policy as NA_NA */
+ cmoved.deqz $r3? $r6 = TLB_PA_NA_NA
+ ;;
+ /* Shift PA to correct position */
+ slld $r6 = $r6, KVX_SFR_TEL_PA_SHIFT
+ set $teh = $r4
+ ;;
+ /* Store updated pte entry */
+ sd 0[$r2] = $r1
+ /* Prepare tel */
+ ord $r6 = $r0, $r6
+ /* Get current task pointer for register restoration */
+ get $r11 = $sr
+ ;;
+ set $tel = $r6
+ ;;
+do_tlb_write:
+ tlbwrite
+ ;;
+#ifdef CONFIG_KVX_DEBUG_TLB_WRITE
+ goto mmc_error_check
+ ;;
+mmc_error_check_ok:
+#endif
+#ifdef CONFIG_KVX_DEBUG_ASN
+ goto asn_check
+ ;;
+asn_check_ok:
+#endif
+ set $tel = $r8
+ /* Restore registers */
+ lo $r4r5r6r7 = __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r11]
+ ;;
+ set $teh = $r9
+ lo $r0r1r2r3 = __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[$r11]
+ ;;
+ set $mmc = $r10
+ ;;
+ lo $r8r9r10r11 = __PA(TASK_THREAD_SAVE_AREA_QUAD(2))[$r11]
+ ;;
+#ifdef CONFIG_KVX_MMU_STATS
+ /*
+ * Fence to simulate a direct data dependency after returning from trap
+ * nomapping handling. This is the worst case that can happen and the
+ * processor will be stalled waiting for previous loads to complete.
+ */
+ fence
+ ;;
+ get $r4 = $pm0
+ ;;
+ get $r0 = $sr
+ ;;
+ /* Get cycles measured on trap entry */
+ ld $r1 = __PA(TASK_THREAD_ENTRY_TS)[$r0]
+ ;;
+ /* Compute refill time */
+ sbfd $r0 = $r1, $r4
+ ;;
+#ifdef CONFIG_SMP
+ get $r1 = $pcr
+ ;;
+ /* Extract processor ID to compute cpu_offset */
+ extfz $r1 = $r1, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID)
+ make $r2 = __PA(__per_cpu_offset)
+ ;;
+ /* Load cpu offset */
+ ld.xs $r1 = $r1[$r2]
+ ;;
+ addd $r1 = $r1, __PA(mmu_stats)
+ ;;
+#else
+ make $r1 = __PA(mmu_stats)
+ ;;
+#endif
+ /* Load time between refill + last refill cycle */
+ lq $r2r3 = MMU_STATS_CYCLES_BETWEEN_REFILL_OFF[$r1]
+ ;;
+ /* Set last update time to current if 0 */
+ cmoved.deqz $r3? $r3 = $r4
+ /* remove current refill time to current cycle */
+ sbfd $r4 = $r0, $r4
+ ;;
+ /* Compute time between last refill and current refill */
+ sbfd $r5 = $r3, $r4
+ /* Update last cycle time */
+ copyd $r3 = $r4
+ ;;
+ /* Increment total time between refill */
+ addd $r2 = $r2, $r5
+ ;;
+ sq MMU_STATS_CYCLES_BETWEEN_REFILL_OFF[$r1] = $r2r3
+ /* Get exception address */
+ get $r4 = $ea
+ ;;
+ /* $r2 holds refill type (user/kernel/kernel_direct) */
+ make $r2 = MMU_STATS_REFILL_KERNEL_OFF
+ /* Check if address is a kernel direct mapping one */
+ compd.ltu $r3 = $r4, (KERNEL_DIRECT_MEMORY_MAP_BASE + \
+ KERNEL_DIRECT_MEMORY_MAP_SIZE)
+ ;;
+ cmoved.dnez $r3? $r2 = MMU_STATS_REFILL_KERNEL_DIRECT_OFF
+ /* Check if address is a user (ie below kernel) */
+ compd.ltu $r3 = $r4, KERNEL_DIRECT_MEMORY_MAP_BASE
+ ;;
+ cmoved.dnez $r3? $r2 = MMU_STATS_REFILL_USER_OFF
+ ;;
+ /* Compute refill struct addr into one reg */
+ addd $r1 = $r2, $r1
+ /* Load refill_struct values */
+ lo $r4r5r6r7 = $r2[$r1]
+ ;;
+ /* Increment count */
+ addd $r4 = $r4, 1
+ /* Increment total cycles count */
+ addd $r5 = $r5, $r0
+ ;;
+ /* Set min to ~0 if 0 */
+ cmoved.deqz $r6? $r6 = ~0
+ ;;
+ /* Compare min and max */
+ compd.ltu $r2 = $r0, $r6
+ compd.gtu $r3 = $r0, $r7
+ ;;
+ /* Update min and max*/
+ cmoved.dnez $r2? $r6 = $r0
+ cmoved.dnez $r3? $r7 = $r0
+ ;;
+ /* store back all values */
+ so 0[$r1] = $r4r5r6r7
+ ;;
+ get $r0 = $sr
+ ;;
+ /* Restore cloberred registers */
+ lo $r4r5r6r7 = __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r0]
+ ;;
+ lo $r0r1r2r3 = __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[$r0]
+ ;;
+#endif
+ /* Save $r4 for reenabling mmu and data cache in sps */
+ set TMP_SCRATCH_SR = $r4
+ /* Enable MME in $sps */
+ make $r4 = MME_WFXL(1)
+ ;;
+ /* Reenable $mme in $sps */
+ wfxl $sps = $r4
+ ;;
+ get $r4 = TMP_SCRATCH_SR
+ ;;
+ rfe
+ ;;
+
+is_huge_page:
+ /*
+ * When entering this path:
+ * - $r0 = $ea
+ * - $r1 = pte entry
+ * - $r7 = cpu offset for tlb_current_set_way
+ *
+ * From now on, we have the pte value in $r1 so we can extract the page
+ * size. This value is stored as it is expected by the MMU (ie between
+ * 0 and 3).
+ * Note that page size value is located at the same place as in $tel
+ * and this is checked at build time so we can use TEL_PS defines.
+ * In this codepath, we will extract the set and mask exception address
+ * and align virt and phys address with what the hardware expect.
+ * Indeed, MMU expect lsb of the virtual and physycal address to be 0
+ * according to page size.
+ * This means that for 4K pages, the 12 lsb must be 0, for 64K
+ * pages, the 16 lsb must be 0 and so on.
+ */
+ extfz $r5 = $r1, KVX_SFR_END(TEL_PS), KVX_SFR_START(TEL_PS)
+ /* Compute the mask to extract set and mask exception address */
+ make $r4 = KVX_PS_SHIFT_MATRIX
+ make $r6 = KVX_SBMM_BYTE_SEL
+ ;;
+ /* Generate the byte selection for sbmm */
+ slld $r6 = $r6, $r5
+ ;;
+ /* Get the shift value */
+ sbmm8 $r5 = $r4, $r6
+ make $r4 = 0xFFFFFFFFFFFFFFFF
+ ;;
+ /* extract TLB set from ea (6 lsb of virtual page) */
+ srld $r5 = $r0, $r5
+ /* Generate ea masking according to page shift */
+ slld $r4 = $r4, $r5
+ ;;
+ /* Mask to get the set value */
+ andd $r0 = $r5, MMU_SET_MASK
+ /* Extract virt page from ea */
+ andd $r4 = $r0, $r4
+ ;;
+ /* Returned to fast path */
+ goto pte_prepared
+ ;;
+
+#ifdef CONFIG_KVX_DEBUG_TLB_WRITE
+mmc_error_check:
+ get $r1 = $mmc
+ ;;
+ andd $r1 = $r1, KVX_SFR_MMC_E_MASK
+ ;;
+ cb.deqz $r1? mmc_error_check_ok
+ ;;
+ make $r0 = mmc_error_panic_str_label
+ goto asm_panic
+ ;;
+#endif
+#ifdef CONFIG_KVX_DEBUG_ASN
+/*
+ * When entering this path $r11 = $sr.
+ * WARNING: Do not clobber it here if you don't want to mess up with registers
+ * restoration above.
+ */
+asn_check:
+ get $r1 = $ea
+ ;;
+ /* Check if kernel address, if so, there is no ASN */
+ compd.geu $r2 = $r1, PAGE_OFFSET
+ ;;
+ cb.dnez $r2? asn_check_ok
+ ;;
+ get $r2 = $pcr
+ /* Load active mm addr */
+ ld $r3 = __PA(TASK_ACTIVE_MM)[$r11]
+ ;;
+ get $r5 = $mmc
+ /* Extract processor ID to compute cpu_offset*/
+ extfz $r2 = $r2, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID)
+ addd $r3 = $r3, MM_CTXT_ASN
+ ;;
+ extfz $r4 = $r5, KVX_SFR_END(MMC_ASN), KVX_SFR_START(MMC_ASN)
+ addd $r3 = $r3, VA_TO_PA_OFFSET
+ ;;
+ /* Load current asn from active_mm */
+ ld.xs $r3 = $r2[$r3]
+ ;;
+ /* Error if ASN is not set */
+ cb.deqz $r3? asn_check_err
+ /* Mask $r3 asn cycle part */
+ andd $r5 = $r3, ((1 << KVX_SFR_MMC_ASN_WIDTH) - 1)
+ ;;
+ /* Compare asn in $mmc and asn in current task mm */
+ compd.eq $r3 = $r5, $r4
+ ;;
+ cb.dnez $r3? asn_check_ok
+ ;;
+asn_check_err:
+ /* We are fried, die peacefully */
+ make $r0 = asn_error_panic_str_label
+ goto asm_panic
+ ;;
+#endif
+
+#if defined(CONFIG_KVX_DEBUG_ASN) || defined(CONFIG_KVX_DEBUG_TLB_WRITE)
+
+/**
+ * This routine calls panic from assembly after setting appropriate things
+ * $r0 = panic string
+ */
+asm_panic:
+ /*
+ * Reenable hardware loop and traps (for nomapping) since some functions
+ * might need it. Moreover, disable DAUS to reenable MMU accesses.
+ */
+ make $r32 = PS_HWLOOP_EN_ET_CLEAR_DAUS_DIS
+ make $r33 = 0
+ get $r34 = $sr
+ ;;
+ /* Clear hw loop exit to disable current loop */
+ set $le = $r33
+ ;;
+ wfxl $ps = $r32
+ ;;
+ /* Restore kernel stack */
+ ld $r12 = TASK_THREAD_KERNEL_SP[$r34]
+ ;;
+ call panic
+ ;;
+#endif
+
+/* Error path for TLB refill */
+refill_err_out:
+ get $r2 = $sr
+ ;;
+ /* Restore clobbered registers */
+ lo $r8r9r10r11 = __PA(TASK_THREAD_SAVE_AREA_QUAD(2))[$r2]
+ ;;
+ lo $r4r5r6r7 = __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r2]
+ goto trap_switch_stack
+ ;;
+
+/* This path is entered only when the trap is NOT a NOMAPPING */
+trap_slow_path:
+ /* Restore $r3 from temporary scratch system register */
+ get $r3 = TMP_SCRATCH_SR
+ ;;
+ save_quad_regs $r0r1r2r3 $r4
+ ;;
+ get $r2 = $sr
+ ;;
+trap_switch_stack:
+ switch_stack $r1 $r2 trap_save_regs
+ ;;
+trap_save_regs:
+ save_regs_for_exception $r1 $r2 $r0r1r2r3 $r2
+ ;;
+ get $r1 = $ea
+ /* tail-call for return_from_exception */
+ make $r3 = return_from_exception
+ ;;
+ set $ra = $r3
+ ;;
+ /* Handler call */
+ get $r0 = $es
+ ;;
+ goto trap_handler
+ ;;
+ENDPROC(kvx_trap_handler)
+
+/***********************************************************************
+* Interrupts handling
+***********************************************************************/
+EXCEPTION_ENTRY(kvx_interrupt_handler):
+ exception_entry_disable_mmu
+ ;;
+ save_quad_regs $r0r1r2r3 $r4
+ ;;
+ get $r0 = $sr
+ ;;
+ switch_stack $r1 $r0 it_save_regs
+ ;;
+#ifdef CONFIG_SECURE_DAME_HANDLING
+ /**
+ * In order to securely Handle Data Asynchronous Memory Error,
+ * we need to have a correct entry point. This means we do not
+ * want to handle a user induced DAME interrupt when entering
+ * kernel.
+ * In order to do that, we need to do a barrier, which will
+ * reflect the DAME status in $ilr (if any).
+ */
+ barrier
+ ;;
+#endif
+it_save_regs:
+ save_regs_for_exception $r1 $r0 $r0r1r2r3 $r1
+ ;;
+ get $r0 = $ilr
+ ;;
+ get $r2 = $ile
+ ;;
+ /**
+ * When an interrupt happens, the processor automatically clears the
+ * corresponding bit in $ilr. However, as we are using $ilr to get the
+ * list of irqs we want to handle, we need to add the automatically
+ * cleared interrupt bit. This is done by getting the interrupt number
+ * from $es.
+ */
+ get $r3 = $es
+ make $r4 = 1
+ ;;
+ /* Extract interrupt number from $es */
+ extfz $r3 = $r3, KVX_SFR_END(ES_ITN), KVX_SFR_START(ES_ITN)
+ /**
+ * Mask requests with enabled line since ILR will also contain disabled
+ * interrupt lines (ie not enabled in $ile) and we want to respect the
+ * current state of interrupt lines.
+ */
+ andd $r0 = $r0, $r2
+ ;;
+ /* Clear $ilr with bits we are going to handle */
+ wfxl $ilr = $r0
+ slld $r4 = $r4, $r3
+ ;;
+ /* OR the irq mask with the current pending irq */
+ ord $r0 = $r0, $r4
+ call do_IRQ
+ ;;
+ /* From now on, lower the interrupt level (IL) to allow IT nesting.
+ * If returning to user, we will call schedule which will reenable
+ * interrupts by itself when ready.
+ * If returning to kernel and with CONFIG_PREEMPT, we will call
+ * preempt_schedule_irq which will do the same.
+ */
+ make $r0 = PS_IL_CLEAR
+ ;;
+ wfxl $ps = $r0
+ ;;
+ goto return_from_exception
+ ;;
+ENDPROC(kvx_interrupt_handler)
+
+/***********************************************************************
+* Syscall handling
+***********************************************************************/
+EXCEPTION_ENTRY(kvx_syscall_handler):
+ /**
+ * Syscalls are seen as standard func call from ABI POV
+ * We may use all caller saved register whithout causing havoc
+ * in the userspace. We need to save callee registers because they
+ * will be restored when returning from fork.
+ * Note that r0 -> r11 MUST not be used since they are
+ * containing syscall parameters !
+ * During this function, $r38 is the syscall handler address. Hence,
+ * this register must not be cloberred during function calls (tracing
+ * for instance.
+ */
+ disable_mmu $r43
+ ;;
+ get $r43 = $es
+ copyd $r52 = $sp
+ copyd $r53 = $tp
+ ;;
+ get $r36 = $sr
+ copyd $r54 = $fp
+ copyd $r55 = $r15
+ ;;
+ /* Extract syscall number */
+ extfz $r32 = $r43, KVX_SFR_END(ES_SN), KVX_SFR_START(ES_SN)
+ /* Get regs to save on stack */
+ get $r63 = $ra
+ addd $r36 = $r36, VA_TO_PA_OFFSET
+ ;;
+ ld $r39 = TASK_TI_FLAGS[$r36]
+ get $r41 = $spc
+ make $r42 = __PA(sys_call_table)
+ ;;
+ /* Check for out-of-bound syscall number */
+ sbfd $r50 = $r32, __NR_syscalls
+ /* Compute syscall func addr (ie sys_call_table[$r32])*/
+ ld.xs $r38 = $r32[$r42]
+ get $r42 = $sps
+ ;;
+ /* True if trace syscall enable */
+ andd $r37 = $r39, _TIF_SYSCALL_WORK
+ /* Restore kernel stack pointer */
+ ld $sp = TASK_THREAD_KERNEL_SP[$r36]
+ /* If the syscall number is invalid, set invalid handler */
+ cmoved.dlez $r50? $r38 = sys_ni_syscall
+ ;;
+ /* Prepare space on stack */
+ addd $sp = $sp, -PT_SIZE_ON_STACK
+ get $r40 = $cs
+ /* Save regs r0 -> r3 in pt_regs for restart & trace if needed */
+ so __PA(PT_Q0 - PT_SIZE_ON_STACK)[$sp] = $r0r1r2r3
+ ;;
+ /* Store user stack pointer, frame pointer, thread pointer and r15 */
+ so __PA(PT_Q12)[$sp] = $r52r53r54r55
+ addd $r36 = $sp, VA_TO_PA_OFFSET
+ get $r60 = $lc
+ ;;
+#ifdef CONFIG_SECURE_DAME_HANDLING
+ get $r44 = $ilr
+ make $r45 = SFR_CLEAR_WFXL(ILR, IT16);
+ ;;
+ /* Extract $ilr.dame bit */
+ extfz $r44 = $r44, KVX_SFR_END(ILR_IT16), KVX_SFR_START(ILR_IT16)
+ /* Save $ilr value */
+ sd PT_ILR[$r36] = $r44
+ /* Clear $ilr.dame */
+ wfxl $ilr = $r45
+ ;;
+#endif
+ so PT_CS_SPC_SPS_ES[$r36] = $r40r41r42r43
+ get $r61 = $le
+ make $r43 = 0x0
+ ;;
+ /* Reenable hardware loops, IT, exceptions and disable DAUS */
+ make $r44 = PS_HLE_EN_IT_EN_ET_CLEAR_DAUS_DIS
+ get $r62 = $ls
+ /* Save regs r4 -> r7 in pt_regs for restart & trace if needed */
+ so PT_Q4[$r36] = $r4r5r6r7
+ ;;
+ /* Clear $le on entry */
+ set $le = $r43
+ /* Save hw loop stuff */
+ so PT_LC_LE_LS_RA[$r36] = $r60r61r62r63
+ /* Clear frame pointer for kernel */
+ make $fp = 0
+ ;;
+ /* Enable hwloop and interrupts and MMU
+ * Note that we have to reenable interrupts after saving context
+ * to avoid losing registers content */
+ wfxl $ps, $r44
+ ;;
+ /* Do we have to trace the syscall ? */
+ cb.dnez $r37? trace_syscall_enter
+ /* Stroe original r0 value */
+ sd PT_ORIG_R0[$sp] = $r0
+ ;;
+do_syscall:
+ /* Call the syscall handler */
+ icall $r38
+ ;;
+ /*
+ * rt_sigreturn syscall will not take this exit path (see
+ * sys_rt_sigreturn for more information).
+ */
+ /* Reload task flags since the syscall might have generated a signal*/
+ get $r36 = $sr
+ ;;
+#ifdef CONFIG_SECURE_DAME_HANDLING
+ /* Barrier to force DAME interrupt to be generated if any pending */
+ barrier
+ ;;
+#endif
+ /* Disable interrupt to check task flags atomically */
+ disable_interrupt $r60
+ ;;
+ ld $r38 = TASK_TI_FLAGS[$r36]
+ ;;
+ andd $r37 = $r38, _TIF_SYSCALL_WORK
+ ;;
+ /* Save r0 which was returned from do_scall previously and
+ * will be cloberred by do_work_pending(and potentially
+ * do_syscall_trace_exit if tracing is enabled)
+ * If do_signal is called and that syscall is restarted,
+ * it will be modified by handle_restart to restore original
+ * r0 value
+ */
+ sd PT_Q0[$sp] = $r0
+ /* used to store if trace system */
+ cb.dnez $r37? trace_syscall_exit
+ ;;
+check_work_pending:
+ /* Do we have work pending ? */
+ andd $r1 = $r38, _TIF_WORK_MASK
+ ;;
+ /* If no work pending, directly continue to ret_to_user */
+ cb.dnez $r1? call_work_pending
+ ;;
+ret_to_user:
+ disable_mmu $r60, PS_HWLOOP_DISABLE
+ ;;
+ /* Compute $sp virtual address to avoid using 64 bits offset */
+ addd $r15 = $sp, VA_TO_PA_OFFSET
+ ;;
+ /* Restore registers */
+ lo $r60r61r62r63 = PT_LC_LE_LS_RA[$r15]
+ get $r33 = $sps
+ ;;
+ lo $r40r41r42r43 = PT_CS_SPC_SPS_ES[$r15];
+ set $ra = $r63
+ ;;
+ /* Restore syscall arguments since they might be needed for
+ * syscall restart
+ */
+ lo $r0r1r2r3 = PT_Q0[$r15]
+ set $cs = $r40
+ /* Generate a mask of ones at each bit where the current $sps
+ * differs from the $sps to be restored
+ */
+ xord $r33 = $r42, $r33
+ /* prepare wfxl clear mask on LSBs */
+ notd $r34 = $r42
+ /* prepare wfxl set mask on MSBs */
+ slld $r35 = $r42, 32
+ ;;
+ set $lc = $r60
+ /* Replicate mask of ones on the 32 MSBs */
+ sbmm8 $r33 = $r33, REPLICATE_32_MASK
+ /* Combine the set and clear mask for wfxl */
+ insf $r35 = $r34, 31, 0
+ ;;
+ lo $r4r5r6r7 = PT_Q4[$r15]
+ set $le = $r61
+ /* Mask to drop identical bits in order to avoid useless
+ * privilege traps
+ */
+ andd $r35 = $r35, $r33
+ ;;
+ /* Restore $ilr */
+ ld $r44 = PT_ILR[$r15]
+ set $ls = $r62
+ /* Reenable MMU in $sps */
+ ord $r35 = $r35, SFR_SET_VAL_WFXL(SPS, MME, 1)
+ ;;
+ /* Restore user pointer */
+ lo $r12r13r14r15 = PT_Q12[$r15]
+ /* Prepare $r44 as a set mask for $ilr wfxl */
+ slld $r44 = $r44, 32
+ ;;
+ /**
+ * wfxl on $ilr to avoid privilege traps when restoring with set
+ * Note that we do that after disabling interrupt since we explicitly
+ * want to serve DAME itnerrupt to the user (ie not in kernel mode).
+ */
+ wfxl $ilr = $r44
+ ;;
+ wfxl $sps = $r35
+ ;;
+ set $spc = $r41
+ ;;
+ /* TODO: we might have to clear some registers to avoid leaking
+ * information to user space ! callee saved regs have been
+ * restored by called function but caller saved regs might
+ * have been used without being cleared */
+ rfe
+ ;;
+
+/* Slow paths handling */
+call_work_pending:
+ /*
+ * Work pending can potentially call a signal handler and then return
+ * via rt_sigreturn. Return path will be different (restore all regs)
+ * and hence all registers are needed to be saved.
+ */
+ save_callee
+ ;;
+ call_do_work_pending
+ ;;
+ /* Since we are returning to user, interrupts will be reenabled */
+ call_trace_hardirqs on
+ ;;
+ goto ret_to_user
+ ;;
+
+trace_syscall_enter:
+ /* Save $r38 (syscall handler) which was computed above */
+ sd PT_R38[$sp] = $r38
+ ;;
+ /* do_syscall_trace_enter expect pt_regs and syscall number
+ * as argument */
+ copyd $r0 = $sp
+ copyd $r1 = $r32
+ ;;
+ call do_syscall_trace_enter
+ ;;
+ /* Restore r38 (syscall handler) which might has been clobbered by
+ * do_syscall_trace_enter */
+ ld $r38 = PT_R38[$sp]
+ ;;
+ /* if the trace system requested to abort syscall, set $r38 to
+ * non implemented syscall */
+ cmoved.dnez $r0? $r38 = sys_ni_syscall
+ ;;
+ /* Restore registers since the do_syscall_trace_enter call might
+ * have clobbered them and we need them for the actual syscall
+ * call */
+ lo $r0r1r2r3 = PT_Q0[$sp]
+ ;;
+ lo $r4r5r6r7 = PT_Q4[$sp]
+ ;;
+ goto do_syscall
+ ;;
+trace_syscall_exit:
+ copyd $r0 = $sp
+ call do_syscall_trace_exit
+ ;;
+ /* ptrace_notify might re-enable interrupts, disable them to be sure
+ * it will not mess up context restoration path */
+ disable_interrupt $r36
+ ;;
+ get $r36 = $sr
+ ;;
+ ld $r38 = TASK_TI_FLAGS[$r36]
+ goto check_work_pending
+ ;;
+ENDPROC(kvx_syscall_handler)
+
+.text
+/**
+ * sys_rt_sigreturn: Special handler for sigreturn
+ * rt_sigreturn is called after invoking a user signal handler (see
+ * user_scall_rt_sigreturn). Since this signal handler can be invoked after
+ * interrupts have been handled, this means we must restore absolutely all user
+ * registers. Normally, this has been done by setup_sigcontext which saved all
+ * user registers on the user stack.
+ * In restore sigcontext, they have been restored onto entry stack (stack to be
+ * restored). However, the standard syscall path do not restore it completely
+ * since only callee-saved registers are restored for fork and al.
+ * Here, we will restore all registers which might have been cloberred.
+ */
+ENTRY(sys_rt_sigreturn)
+ /*
+ * If syscall tracing is enable (stored in $r37 during syscall
+ * fastpath), tail-call to trace_exit. If not, tail-call to
+ * ret_from_kernel.
+ */
+ cmoved.dnez $r37? $r38 = scall_trace_exit
+ cmoved.deqz $r37? $r38 = ret_from_kernel
+ ;;
+ set $ra = $r38
+ ;;
+ goto _sys_rt_sigreturn
+ ;;
+scall_trace_exit:
+ copyd $r0 = $sp
+ call do_syscall_trace_exit
+ ;;
+ goto ret_from_kernel
+ ;;
+ENDPROC(sys_rt_sigreturn)
+
+/**
+ * __sys_clone: slow path handler for clone
+ *
+ * Save callee registers (from $r18 to $31) since they are needed for
+ * child process when restoring it.
+ * Indeed, when forking we want it to have the same registers contents
+ * as its parent. These registers will then be restored in
+ * ret_from_fork.
+ * This slow path saves them out of the fast path to avoid bloating all syscall
+ * just for one special case.
+ */
+ENTRY(__sys_clone)
+ save_callee
+ ;;
+ /*
+ * Use goto since we want sys_clone to "ret" to the previous caller.
+ * This allows to simply go back in the normal syscall fastpath
+ */
+ goto sys_clone
+ ;;
+ENDPROC(__sys_clone)
+/***********************************************************************
+* Context switch
+***********************************************************************/
+
+#define SAVE_TCA_REG(__reg_num, __task_reg, __zero_reg1, __zero_reg2) \
+ xso (CTX_SWITCH_TCA_REGS + (__reg_num * TCA_REG_SIZE))[__task_reg] = \
+ $a##__reg_num ;\
+ movetq $a##__reg_num##.lo = __zero_reg1, __zero_reg2 ;\
+ movetq $a##__reg_num##.hi = __zero_reg1, __zero_reg2 ;\
+ ;;
+
+.macro save_tca_regs task_reg zero_reg1 zero_reg2
+ SAVE_TCA_REG(0, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(1, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(2, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(3, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(4, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(5, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(6, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(7, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(8, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(9, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(10, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(11, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(12, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(13, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(14, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(15, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(16, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(17, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(18, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(19, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(20, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(21, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(22, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(23, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(24, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(25, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(26, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(27, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(28, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(29, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(30, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(31, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(32, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(33, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(34, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(35, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(36, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(37, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(38, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(39, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(40, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(41, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(42, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(43, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(44, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(45, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(46, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(47, \task_reg, \zero_reg1, \zero_reg2)
+.endm
+
+#define RESTORE_TCA_REG(__reg_num, __task_reg) \
+ xlo.u $a##__reg_num = (CTX_SWITCH_TCA_REGS + (__reg_num * TCA_REG_SIZE)) \
+ [__task_reg] ;\
+ ;;
+
+.macro restore_tca_regs task_reg
+ RESTORE_TCA_REG(0, \task_reg)
+ RESTORE_TCA_REG(1, \task_reg)
+ RESTORE_TCA_REG(2, \task_reg)
+ RESTORE_TCA_REG(3, \task_reg)
+ RESTORE_TCA_REG(4, \task_reg)
+ RESTORE_TCA_REG(5, \task_reg)
+ RESTORE_TCA_REG(6, \task_reg)
+ RESTORE_TCA_REG(7, \task_reg)
+ RESTORE_TCA_REG(8, \task_reg)
+ RESTORE_TCA_REG(9, \task_reg)
+ RESTORE_TCA_REG(10, \task_reg)
+ RESTORE_TCA_REG(11, \task_reg)
+ RESTORE_TCA_REG(12, \task_reg)
+ RESTORE_TCA_REG(13, \task_reg)
+ RESTORE_TCA_REG(14, \task_reg)
+ RESTORE_TCA_REG(15, \task_reg)
+ RESTORE_TCA_REG(16, \task_reg)
+ RESTORE_TCA_REG(17, \task_reg)
+ RESTORE_TCA_REG(18, \task_reg)
+ RESTORE_TCA_REG(19, \task_reg)
+ RESTORE_TCA_REG(20, \task_reg)
+ RESTORE_TCA_REG(21, \task_reg)
+ RESTORE_TCA_REG(22, \task_reg)
+ RESTORE_TCA_REG(23, \task_reg)
+ RESTORE_TCA_REG(24, \task_reg)
+ RESTORE_TCA_REG(25, \task_reg)
+ RESTORE_TCA_REG(26, \task_reg)
+ RESTORE_TCA_REG(27, \task_reg)
+ RESTORE_TCA_REG(28, \task_reg)
+ RESTORE_TCA_REG(29, \task_reg)
+ RESTORE_TCA_REG(30, \task_reg)
+ RESTORE_TCA_REG(31, \task_reg)
+ RESTORE_TCA_REG(32, \task_reg)
+ RESTORE_TCA_REG(33, \task_reg)
+ RESTORE_TCA_REG(34, \task_reg)
+ RESTORE_TCA_REG(35, \task_reg)
+ RESTORE_TCA_REG(36, \task_reg)
+ RESTORE_TCA_REG(37, \task_reg)
+ RESTORE_TCA_REG(38, \task_reg)
+ RESTORE_TCA_REG(39, \task_reg)
+ RESTORE_TCA_REG(40, \task_reg)
+ RESTORE_TCA_REG(41, \task_reg)
+ RESTORE_TCA_REG(42, \task_reg)
+ RESTORE_TCA_REG(43, \task_reg)
+ RESTORE_TCA_REG(44, \task_reg)
+ RESTORE_TCA_REG(45, \task_reg)
+ RESTORE_TCA_REG(46, \task_reg)
+ RESTORE_TCA_REG(47, \task_reg)
+.endm
+
+.text
+/*
+ * When entering in ret_from_kernel_thread, r20 and r21 where set by
+ * copy_thread and have been restored in switch_to.
+ * These registers contains the values needed to call a function
+ * specified by the switch_to caller (or where set by copy_thread).
+ */
+ENTRY(ret_from_kernel_thread)
+ call schedule_tail
+ ;;
+ /* Call fn(arg) */
+ copyd $r0 = $r21
+ ;;
+ icall $r20
+ ;;
+ goto ret_from_kernel
+ ;;
+ENDPROC(ret_from_kernel_thread)
+
+/**
+ * Return from fork.
+ * start_thread will set return stack and pc. Then copy_thread will
+ * take care of the copying logic.
+ * $r20 will then contains 0 if tracing disabled (set by copy_thread)
+ * The mechanism is almost the same as for ret_from_kernel_thread.
+ */
+ENTRY(ret_from_fork)
+ call schedule_tail
+ ;;
+ /* $r20 contains 0 if tracing disable */
+ cb.deqz $r20? ret_from_kernel
+ ;;
+ copyd $r0 = $sp
+ call do_syscall_trace_exit
+ ;;
+ret_from_kernel:
+ /*
+ * When returning from a fork, the child will take this path.
+ * Since we did not restore callee in return_from_exception, we
+ * must do it before.
+ */
+ lo $r28r29r30r31 = PT_Q28[$sp]
+ ;;
+ lo $r24r25r26r27 = PT_Q24[$sp]
+ ;;
+ lo $r20r21r22r23 = PT_Q20[$sp]
+ ;;
+ lq $r18r19 = PT_R18R19[$sp]
+ ;;
+#ifdef CONFIG_DEBUG_EXCEPTION_STACK
+ /**
+ * Debug code expect entry stack to be stored at current $sp.
+ * Make some room and store current $sp to avoid triggering false alarm.
+ */
+ addd $sp = $sp, -STACK_REG_SIZE
+ ;;
+ sd 0[$sp] = $sp
+#endif
+ ;;
+ goto return_from_exception
+ ;;
+ENDPROC(ret_from_fork)
+
+/*
+ * The callee-saved registers must be saved and restored.
+ * When entering:
+ * - r0 = previous task struct
+ * - r1 = next task struct
+ * Moreover, the parameters for function call (given by copy_thread)
+ * are stored in:
+ * - r20 = Func to call
+ * - r21 = Argument for function
+ */
+ENTRY(__switch_to)
+ sd CTX_SWITCH_FP[$r0] = $fp
+ ;;
+ /* Save previous task context */
+ so CTX_SWITCH_Q20[$r0] = $r20r21r22r23
+ ;;
+ so CTX_SWITCH_Q24[$r0] = $r24r25r26r27
+ get $r16 = $ra
+ ;;
+ so CTX_SWITCH_Q28[$r0] = $r28r29r30r31
+ copyd $r17 = $sp
+ get $r2 = $cs
+ ;;
+ so CTX_SWITCH_RA_SP_R18_R19[$r0] = $r16r17r18r19
+ /* Extract XMF bit which means coprocessor was used by user */
+ andd $r3 = $r2, KVX_SFR_CS_XMF_MASK
+ ;;
+#ifdef CONFIG_ENABLE_TCA
+ make $r4 = 0
+ make $r5 = 0
+ make $r6 = 1
+ cb.dnez $r3? save_tca_registers
+ /* Check if next task needs TCA registers to be restored */
+ ld $r7 = CTX_SWITCH_TCA_REGS_SAVED[$r1]
+ ;;
+check_restore_tca:
+ cb.dnez $r7? restore_tca_registers
+ ;;
+restore_fast_path:
+#endif
+ /* Restore next task context */
+ lo $r16r17r18r19 = CTX_SWITCH_RA_SP_R18_R19[$r1]
+ ;;
+ lo $r20r21r22r23 = CTX_SWITCH_Q20[$r1]
+ ;;
+ lo $r24r25r26r27 = CTX_SWITCH_Q24[$r1]
+ copyd $sp = $r17
+ set $ra = $r16
+ ;;
+ lo $r28r29r30r31 = CTX_SWITCH_Q28[$r1]
+ set $sr = $r1
+ ;;
+ ld $fp = CTX_SWITCH_FP[$r1]
+ ;;
+ ret
+ ;;
+#ifdef CONFIG_ENABLE_TCA
+save_tca_registers:
+ save_tca_regs $r0 $r4 $r5
+ ;;
+ /* Indicates that we saved the TCA context */
+ sb CTX_SWITCH_TCA_REGS_SAVED[$r0] = $r6
+ goto check_restore_tca
+ ;;
+restore_tca_registers:
+ restore_tca_regs $r1
+ ;;
+ /* Clear TCA registers saved hint */
+ sb CTX_SWITCH_TCA_REGS_SAVED[$r1] = $r5
+ goto restore_fast_path
+ ;;
+#endif
+ENDPROC(__switch_to)
+
+/***********************************************************************
+* Misc functions
+***********************************************************************/
+
+/**
+ * Avoid hardcoding trampoline for rt_sigreturn by using this code and
+ * copying it on user trampoline
+ */
+.pushsection .text
+.global user_scall_rt_sigreturn_end
+ENTRY(user_scall_rt_sigreturn)
+ make $r6 = __NR_rt_sigreturn
+ ;;
+ scall $r6
+ ;;
+user_scall_rt_sigreturn_end:
+ENDPROC(user_scall_rt_sigreturn)
+.popsection
diff --git a/arch/kvx/kernel/sys_kvx.c b/arch/kvx/kernel/sys_kvx.c
new file mode 100644
index 000000000000..b32a821d76c9
--- /dev/null
+++ b/arch/kvx/kernel/sys_kvx.c
@@ -0,0 +1,58 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#include <linux/syscalls.h>
+
+#include <asm/cacheflush.h>
+#include <asm/cachectl.h>
+
+SYSCALL_DEFINE6(mmap, unsigned long, addr, unsigned long, len,
+ unsigned long, prot, unsigned long, flags,
+ unsigned long, fd, off_t, off)
+{
+ /* offset must be a multiple of the page size */
+ if (unlikely(offset_in_page(off) != 0))
+ return -EINVAL;
+
+ /* Unlike mmap2 where the offset is in PAGE_SIZE-byte units, here it
+ * is in bytes. So we need to use PAGE_SHIFT.
+ */
+ return ksys_mmap_pgoff(addr, len, prot, flags, fd, off >> PAGE_SHIFT);
+}
+
+SYSCALL_DEFINE4(cachectl, unsigned long, addr, unsigned long, len,
+ unsigned long, cache, unsigned long, flags)
+{
+ bool wb = !!(flags & CACHECTL_FLAG_OP_WB);
+ bool inval = !!(flags & CACHECTL_FLAG_OP_INVAL);
+
+ if (len == 0)
+ return 0;
+
+ /* Check for overflow */
+ if (addr + len < addr)
+ return -EFAULT;
+
+ if (cache != CACHECTL_CACHE_DCACHE)
+ return -EINVAL;
+
+ if ((flags & CACHECTL_FLAG_OP_MASK) == 0)
+ return -EINVAL;
+
+ if (flags & CACHECTL_FLAG_ADDR_PHYS) {
+ if (!IS_ENABLED(CONFIG_CACHECTL_UNSAFE_PHYS_OPERATIONS))
+ return -EINVAL;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ dcache_wb_inval_phys_range(addr, len, wb, inval);
+ return 0;
+ }
+
+ return dcache_wb_inval_virt_range(addr, len, wb, inval);
+}
diff --git a/arch/kvx/kernel/syscall_table.c b/arch/kvx/kernel/syscall_table.c
new file mode 100644
index 000000000000..7859d25e728d
--- /dev/null
+++ b/arch/kvx/kernel/syscall_table.c
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/riscv/kernel/syscall_table.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/syscalls.h>
+
+#include <asm/syscalls.h>
+
+#undef __SYSCALL
+#define __SYSCALL(nr, call) [nr] = (call),
+
+void *sys_call_table[__NR_syscalls] = {
+ [0 ... __NR_syscalls - 1] = sys_ni_syscall,
+#include <asm/unistd.h>
+};
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 12/25] kvx: Add system call support
@ 2023-01-03 16:43 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Christian Brauner, Paul Walmsley, Palmer Dabbelt,
Albert Ou, linux-kernel, linux-riscv, Clement Leger,
Guillaume Thouvenin, Julian Vetter, Julien Villette,
Marius Gligor
Add system call support and related uaccess.h for kvx.
CC: Christian Brauner <brauner@kernel.org>
CC: Paul Walmsley <paul.walmsley@sifive.com>
CC: Palmer Dabbelt <palmer@dabbelt.com>
CC: Albert Ou <aou@eecs.berkeley.edu>
CC: linux-kernel@vger.kernel.org
CC: linux-riscv@lists.infradead.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Julien Villette <jvillette@kalray.eu>
Signed-off-by: Julien Villette <jvillette@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/syscall.h | 73 ++
arch/kvx/include/asm/syscalls.h | 21 +
arch/kvx/include/asm/uaccess.h | 324 +++++
arch/kvx/include/asm/unistd.h | 11 +
arch/kvx/include/uapi/asm/cachectl.h | 25 +
arch/kvx/include/uapi/asm/unistd.h | 16 +
arch/kvx/kernel/entry.S | 1759 ++++++++++++++++++++++++++
arch/kvx/kernel/sys_kvx.c | 58 +
arch/kvx/kernel/syscall_table.c | 19 +
9 files changed, 2306 insertions(+)
create mode 100644 arch/kvx/include/asm/syscall.h
create mode 100644 arch/kvx/include/asm/syscalls.h
create mode 100644 arch/kvx/include/asm/uaccess.h
create mode 100644 arch/kvx/include/asm/unistd.h
create mode 100644 arch/kvx/include/uapi/asm/cachectl.h
create mode 100644 arch/kvx/include/uapi/asm/unistd.h
create mode 100644 arch/kvx/kernel/entry.S
create mode 100644 arch/kvx/kernel/sys_kvx.c
create mode 100644 arch/kvx/kernel/syscall_table.c
diff --git a/arch/kvx/include/asm/syscall.h b/arch/kvx/include/asm/syscall.h
new file mode 100644
index 000000000000..a3f6cef73e4a
--- /dev/null
+++ b/arch/kvx/include/asm/syscall.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SYSCALL_H
+#define _ASM_KVX_SYSCALL_H
+
+#include <linux/err.h>
+#include <linux/audit.h>
+
+#include <asm/ptrace.h>
+
+/* The array of function pointers for syscalls. */
+extern void *sys_call_table[];
+
+void scall_machine_exit(unsigned char value);
+
+/**
+ * syscall_get_nr - find which system call a task is executing
+ * @task: task of interest, must be blocked
+ * @regs: task_pt_regs() of @task
+ *
+ * If @task is executing a system call or is at system call
+ * tracing about to attempt one, returns the system call number.
+ * If @task is not executing a system call, i.e. it's blocked
+ * inside the kernel for a fault or signal, returns -1.
+ *
+ * Note this returns int even on 64-bit machines. Only 32 bits of
+ * system call number can be meaningful. If the actual arch value
+ * is 64 bits, this truncates to 32 bits so 0xffffffff means -1.
+ *
+ * It's only valid to call this when @task is known to be blocked.
+ */
+static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs)
+{
+ if (!in_syscall(regs))
+ return -1;
+
+ return es_sysno(regs);
+}
+
+static inline long syscall_get_error(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ /* 0 if syscall succeeded, otherwise -Errorcode */
+ return IS_ERR_VALUE(regs->r0) ? regs->r0 : 0;
+}
+
+static inline long syscall_get_return_value(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ return regs->r0;
+}
+
+static inline int syscall_get_arch(struct task_struct *task)
+{
+ return AUDIT_ARCH_KVX;
+}
+
+static inline void syscall_get_arguments(struct task_struct *task,
+ struct pt_regs *regs,
+ unsigned long *args)
+{
+ args[0] = regs->orig_r0;
+ args++;
+ memcpy(args, ®s->r1, 5 * sizeof(args[0]));
+}
+
+int __init setup_syscall_sigreturn_page(void *sigpage_addr);
+
+#endif
diff --git a/arch/kvx/include/asm/syscalls.h b/arch/kvx/include/asm/syscalls.h
new file mode 100644
index 000000000000..beec95ebb97a
--- /dev/null
+++ b/arch/kvx/include/asm/syscalls.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SYSCALLS_H
+#define _ASM_KVX_SYSCALLS_H
+
+#include <asm-generic/syscalls.h>
+
+/* We redefine clone in assembly for special slowpath */
+asmlinkage long __sys_clone(unsigned long clone_flags, unsigned long newsp,
+ int __user *parent_tid, int __user *child_tid, int tls);
+
+#define sys_clone __sys_clone
+
+long sys_cachectl(unsigned long addr, unsigned long len, unsigned long cache,
+ unsigned long flags);
+
+#endif
diff --git a/arch/kvx/include/asm/uaccess.h b/arch/kvx/include/asm/uaccess.h
new file mode 100644
index 000000000000..c72677800763
--- /dev/null
+++ b/arch/kvx/include/asm/uaccess.h
@@ -0,0 +1,324 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * derived from arch/riscv/include/asm/uaccess.h
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef _ASM_KVX_UACCESS_H
+#define _ASM_KVX_UACCESS_H
+
+#include <linux/sched.h>
+#include <linux/types.h>
+
+/**
+ * access_ok: - Checks if a user space pointer is valid
+ * @addr: User space pointer to start of block to check
+ * @size: Size of block to check
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * Checks if a pointer to a block of memory in user space is valid.
+ *
+ * Returns true (nonzero) if the memory block may be valid, false (zero)
+ * if it is definitely invalid.
+ *
+ * Note that, depending on architecture, this function probably just
+ * checks that the pointer is in the user space range - after calling
+ * this function, memory access functions may still return -EFAULT.
+ */
+#define access_ok(addr, size) ({ \
+ __chk_user_ptr(addr); \
+ likely(__access_ok((unsigned long __force)(addr), (size))); \
+})
+
+/*
+ * Ensure that the range [addr, addr+size) is within the process's
+ * address space
+ */
+static inline int __access_ok(unsigned long addr, unsigned long size)
+{
+ return size <= TASK_SIZE && addr <= TASK_SIZE - size;
+}
+
+/*
+ * The exception table consists of pairs of addresses: the first is the
+ * address of an instruction that is allowed to fault, and the second is
+ * the address at which the program should continue. No registers are
+ * modified, so it is entirely up to the continuation code to figure out
+ * what to do.
+ *
+ * All the routines below use bits of fixup code that are out of line
+ * with the main instruction path. This means when everything is well,
+ * we don't even have to jump over them. Further, they do not intrude
+ * on our cache or TLB entries.
+ */
+
+struct exception_table_entry {
+ unsigned long insn, fixup;
+};
+
+extern int fixup_exception(struct pt_regs *regs);
+
+/**
+ * Assembly defined function (usercopy.S)
+ */
+extern unsigned long
+raw_copy_from_user(void *to, const void __user *from, unsigned long n);
+
+extern unsigned long
+raw_copy_to_user(void __user *to, const void *from, unsigned long n);
+
+extern unsigned long
+asm_clear_user(void __user *to, unsigned long n);
+
+#define __clear_user asm_clear_user
+
+static inline __must_check unsigned long
+clear_user(void __user *to, unsigned long n)
+{
+ might_fault();
+ if (!access_ok(to, n))
+ return n;
+
+ return asm_clear_user(to, n);
+}
+
+extern __must_check long strnlen_user(const char __user *str, long n);
+extern long strncpy_from_user(char *dest, const char __user *src, long count);
+
+#define __enable_user_access()
+#define __disable_user_access()
+
+/**
+ * get_user: - Get a simple variable from user space.
+ * @x: Variable to store result.
+ * @ptr: Source address, in user space.
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * This macro copies a single simple variable from user space to kernel
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and the result of
+ * dereferencing @ptr must be assignable to @x without a cast.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ * On error, the variable @x is set to zero.
+ */
+#define get_user(x, ptr) \
+({ \
+ long __e = -EFAULT; \
+ const __typeof__(*(ptr)) __user *__p = (ptr); \
+ might_fault(); \
+ if (likely(access_ok(__p, sizeof(*__p)))) { \
+ __e = __get_user(x, __p); \
+ } else { \
+ x = 0; \
+ } \
+ __e; \
+})
+
+/**
+ * __get_user: - Get a simple variable from user space, with less checking.
+ * @x: Variable to store result.
+ * @ptr: Source address, in user space.
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * This macro copies a single simple variable from user space to kernel
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and the result of
+ * dereferencing @ptr must be assignable to @x without a cast.
+ *
+ * Caller must check the pointer with access_ok() before calling this
+ * function.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ * On error, the variable @x is set to zero.
+ */
+#define __get_user(x, ptr) \
+({ \
+ unsigned long __err = 0; \
+ __chk_user_ptr(ptr); \
+ \
+ __enable_user_access(); \
+ __get_user_nocheck(x, ptr, __err); \
+ __disable_user_access(); \
+ \
+ __err; \
+})
+
+#define __get_user_nocheck(x, ptr, err) \
+do { \
+ unsigned long __gu_addr = (unsigned long)(ptr); \
+ unsigned long __gu_val; \
+ switch (sizeof(*(ptr))) { \
+ case 1: \
+ __get_user_asm("lbz", __gu_val, __gu_addr, err); \
+ break; \
+ case 2: \
+ __get_user_asm("lhz", __gu_val, __gu_addr, err); \
+ break; \
+ case 4: \
+ __get_user_asm("lwz", __gu_val, __gu_addr, err); \
+ break; \
+ case 8: \
+ __get_user_asm("ld", __gu_val, __gu_addr, err); \
+ break; \
+ default: \
+ BUILD_BUG(); \
+ } \
+ (x) = (__typeof__(*(ptr)))__gu_val; \
+} while (0)
+
+#define __get_user_asm(op, x, addr, err) \
+({ \
+ __asm__ __volatile__( \
+ "1: "op" %1 = 0[%2]\n" \
+ " ;;\n" \
+ "2:\n" \
+ ".section .fixup,\"ax\"\n" \
+ "3: make %0 = 2b\n" \
+ " make %1 = 0\n" \
+ " ;;\n" \
+ " make %0 = %3\n" \
+ " igoto %0\n" \
+ " ;;\n" \
+ ".previous\n" \
+ ".section __ex_table,\"a\"\n" \
+ " .align 8\n" \
+ " .dword 1b,3b\n" \
+ ".previous" \
+ : "=r"(err), "=r"(x) \
+ : "r"(addr), "i"(-EFAULT), "0"(err)); \
+})
+
+/**
+ * put_user: - Write a simple value into user space.
+ * @x: Value to copy to user space.
+ * @ptr: Destination address, in user space.
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * This macro copies a single simple value from kernel space to user
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and @x must be assignable
+ * to the result of dereferencing @ptr.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ */
+#define put_user(x, ptr) \
+({ \
+ long __e = -EFAULT; \
+ __typeof__(*(ptr)) __user *__p = (ptr); \
+ might_fault(); \
+ if (likely(access_ok(__p, sizeof(*__p)))) { \
+ __e = __put_user(x, __p); \
+ } \
+ __e; \
+})
+
+/**
+ * __put_user: - Write a simple value into user space, with less checking.
+ * @x: Value to copy to user space.
+ * @ptr: Destination address, in user space.
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * This macro copies a single simple value from kernel space to user
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and @x must be assignable
+ * to the result of dereferencing @ptr.
+ *
+ * Caller must check the pointer with access_ok() before calling this
+ * function.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ */
+#define __put_user(x, ptr) \
+({ \
+ unsigned long __err = 0; \
+ __chk_user_ptr(ptr); \
+ \
+ __enable_user_access(); \
+ __put_user_nocheck(x, ptr, __err); \
+ __disable_user_access(); \
+ \
+ __err; \
+})
+
+#define __put_user_nocheck(x, ptr, err) \
+do { \
+ unsigned long __pu_addr = (unsigned long)(ptr); \
+ __typeof__(*(ptr)) __pu_val = (x); \
+ switch (sizeof(*(ptr))) { \
+ case 1: \
+ __put_user_asm("sb", __pu_val, __pu_addr, err); \
+ break; \
+ case 2: \
+ __put_user_asm("sh", __pu_val, __pu_addr, err); \
+ break; \
+ case 4: \
+ __put_user_asm("sw", __pu_val, __pu_addr, err); \
+ break; \
+ case 8: \
+ __put_user_asm("sd", __pu_val, __pu_addr, err); \
+ break; \
+ default: \
+ BUILD_BUG(); \
+ } \
+} while (0)
+
+#define __put_user_asm(op, x, addr, err) \
+({ \
+ __asm__ __volatile__( \
+ "1: "op" 0[%2], %1\n" \
+ " ;;\n" \
+ "2:\n" \
+ ".section .fixup,\"ax\"\n" \
+ "3: make %0 = 2b\n" \
+ " ;;\n" \
+ " make %0 = %3\n" \
+ " igoto %0\n" \
+ " ;;\n" \
+ ".previous\n" \
+ ".section __ex_table,\"a\"\n" \
+ " .align 8\n" \
+ " .dword 1b,3b\n" \
+ ".previous" \
+ : "=r"(err) \
+ : "r"(x), "r"(addr), "i"(-EFAULT), "0"(err)); \
+})
+
+#define HAVE_GET_KERNEL_NOFAULT
+
+#define __get_kernel_nofault(dst, src, type, err_label) \
+do { \
+ long __kr_err; \
+ \
+ __get_user_nocheck(*((type *)(dst)), (type *)(src), __kr_err); \
+ if (unlikely(__kr_err)) \
+ goto err_label; \
+} while (0)
+
+#define __put_kernel_nofault(dst, src, type, err_label) \
+do { \
+ long __kr_err; \
+ \
+ __put_user_nocheck(*((type *)(src)), (type *)(dst), __kr_err); \
+ if (unlikely(__kr_err)) \
+ goto err_label; \
+} while (0)
+
+
+#endif /* _ASM_KVX_UACCESS_H */
diff --git a/arch/kvx/include/asm/unistd.h b/arch/kvx/include/asm/unistd.h
new file mode 100644
index 000000000000..6cd093dbf2d8
--- /dev/null
+++ b/arch/kvx/include/asm/unistd.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#define __ARCH_WANT_SYS_CLONE
+
+#include <uapi/asm/unistd.h>
+
+#define NR_syscalls (__NR_syscalls)
diff --git a/arch/kvx/include/uapi/asm/cachectl.h b/arch/kvx/include/uapi/asm/cachectl.h
new file mode 100644
index 000000000000..be0a1aa23cf6
--- /dev/null
+++ b/arch/kvx/include/uapi/asm/cachectl.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _UAPI_ASM_KVX_CACHECTL_H
+#define _UAPI_ASM_KVX_CACHECTL_H
+
+/*
+ * Cache type for cachectl system call
+ */
+#define CACHECTL_CACHE_DCACHE (1 << 0)
+
+/*
+ * Flags for cachectl system call
+ */
+#define CACHECTL_FLAG_OP_INVAL (1 << 0)
+#define CACHECTL_FLAG_OP_WB (1 << 1)
+#define CACHECTL_FLAG_OP_MASK (CACHECTL_FLAG_OP_INVAL | \
+ CACHECTL_FLAG_OP_WB)
+
+#define CACHECTL_FLAG_ADDR_PHYS (1 << 2)
+
+#endif /* _UAPI_ASM_KVX_CACHECTL_H */
diff --git a/arch/kvx/include/uapi/asm/unistd.h b/arch/kvx/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..5f86d81dbb76
--- /dev/null
+++ b/arch/kvx/include/uapi/asm/unistd.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#define __ARCH_WANT_RENAMEAT
+#define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+#define __ARCH_WANT_SYS_CLONE3
+
+#include <asm-generic/unistd.h>
+
+/* Additional KVX specific syscalls */
+#define __NR_cachectl (__NR_arch_specific_syscall)
+__SYSCALL(__NR_cachectl, sys_cachectl)
diff --git a/arch/kvx/kernel/entry.S b/arch/kvx/kernel/entry.S
new file mode 100644
index 000000000000..2d16086d29dc
--- /dev/null
+++ b/arch/kvx/kernel/entry.S
@@ -0,0 +1,1759 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Marius Gligor
+ * Yann Sionneau
+ * Julien Villette
+ */
+
+#include <linux/linkage.h>
+
+#include <asm/thread_info.h>
+#include <asm/asm-offsets.h>
+#include <asm/cache.h>
+#include <asm/page.h>
+#include <asm/pgtable-bits.h>
+#include <asm/sys_arch.h>
+#include <asm/sfr_defs.h>
+#include <asm/tlb_defs.h>
+#include <asm/mem_map.h>
+#include <asm/traps.h>
+#include <asm/unistd.h>
+
+#define MMU_SET_MASK ((1 << KVX_SFR_MMC_SS_WIDTH) - 1)
+
+/* Mask to replicate data from 32bits LSB to MSBs */
+#define REPLICATE_32_MASK 0x0804020108040201
+
+#define PS_HWLOOP_ENABLE (KVX_SFR_PS_HLE_MASK << 32)
+#define PS_HWLOOP_DISABLE (KVX_SFR_PS_HLE_MASK)
+#define PS_HWLOOP_EN_ET_CLEAR_DAUS_DIS \
+ (PS_HWLOOP_ENABLE | KVX_SFR_PS_ET_MASK | SFR_SET_VAL_WFXL(PS, DAUS, 0))
+#define PS_ET_CLEAR KVX_SFR_PS_ET_MASK
+#define PS_HLE_EN_IT_EN_ET_CLEAR_DAUS_DIS \
+ (SFR_SET_VAL_WFXL(PS, HLE, 1) | \
+ SFR_SET_VAL_WFXL(PS, IE, 1) | \
+ SFR_SET_VAL_WFXL(PS, ET, 0) | \
+ SFR_SET_VAL_WFXL(PS, DAUS, 0))
+
+#define PS_IT_DIS KVX_SFR_PS_IE_MASK
+#define PS_IL_CLEAR KVX_SFR_PS_IL_WFXL_CLEAR
+
+#define MMC_CLEAR_TLB_CLEAR_WAY \
+ (SFR_CLEAR_WFXL(MMC, SB) | SFR_CLEAR_WFXL(MMC, SW))
+
+#define MMC_SEL_TLB_CLEAR_WAY(__tlb) \
+ (SFR_SET_WFXL(MMC, SB, __tlb) | MMC_CLEAR_TLB_CLEAR_WAY)
+#define MMC_SEL_JTLB_CLEAR_WAY MMC_SEL_TLB_CLEAR_WAY(MMC_SB_JTLB)
+#define MMC_SEL_LTLB_CLEAR_WAY MMC_SEL_TLB_CLEAR_WAY(MMC_SB_LTLB)
+
+#define MME_WFXL(_enable) SFR_SET_VAL_WFXL(PS, MME, _enable)
+
+/* Temporary scract system register for trap handling */
+#define TMP_SCRATCH_SR $sr_pl3
+
+.altmacro
+
+#define TEL_DEFAULT_VALUE (TLB_ES_A_MODIFIED << KVX_SFR_TEL_ES_SHIFT)
+
+#define TASK_THREAD_SAVE_AREA_QUAD(__q) \
+ (TASK_THREAD_SAVE_AREA + (__q) * QUAD_REG_SIZE)
+
+#ifdef CONFIG_DEBUG_EXCEPTION_STACK
+.section .rodata
+stack_error_panic_str_label:
+ .string "Stack has been messed up !"
+#endif
+
+#ifdef CONFIG_KVX_DEBUG_TLB_WRITE
+.section .rodata
+mmc_error_panic_str_label:
+ .string "Failed to write entry to the JTLB (in assembly refill)"
+#endif
+
+#ifdef CONFIG_KVX_DEBUG_ASN
+.section .rodata
+asn_error_panic_str_label:
+ .string "ASN mismatch !"
+#endif
+
+/**
+ * call_trace_hardirqs: hardirqs tracing call
+ * state: State of hardirqs to be reported (on/off)
+ */
+.macro call_trace_hardirqs state
+#ifdef CONFIG_TRACE_IRQFLAGS
+ call trace_hardirqs_\state
+ ;;
+#endif
+.endm
+
+/**
+ * disable_interrupts: Disable interrupts
+ * tmp_reg: Temporary register to use for interrupt disabling
+ */
+.macro disable_interrupt tmp_reg
+ make \tmp_reg = KVX_SFR_PS_IE_MASK
+ ;;
+ wfxl $ps, \tmp_reg
+ ;;
+.endm
+
+/**
+ * call_do_work_pending: Call do_work_pending and set stack argument ($r0)
+ * NOTE: Since do_work_pending requires thread_flags in $r1, they must
+ * be provided in $r1 before calling this macro.
+ * Moreover, do_work_pending expects interrupts to be disabled.
+ */
+.macro call_do_work_pending
+ copyd $r0 = $sp
+ call do_work_pending
+ ;;
+.endm
+
+/**
+ * save_quad_regs: Save quad registers in temporary thread area
+ * After call, the quad is saved in task_struct.thread.save_area.
+ * sr_swap_reg is only used as a temporary value and will be
+ * restored after returning from this macro
+ *
+ * quad: Quad to saved
+ * sr_swap_reg: Register used for sr swap and quad saving.
+ */
+.macro save_quad_regs quad sr_swap_reg
+ rswap \sr_swap_reg = $sr
+ ;;
+ /* Save registers in save_area */
+ so __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[\sr_swap_reg] = \quad
+ /* Restore register */
+ rswap \sr_swap_reg = $sr
+ ;;
+.endm
+
+/**
+ * Switch task when entering kernel if needed.
+ * sp_reg: register which will be used to store original stack pointer when
+ * entering the kernel
+ * task_reg: is a register containing the current task pointer
+ * save_regs_label: label to jump to save register immediately. This label is
+ * used when we are already in kernel execution context.
+ * NOTE: when returning from this macro, we consider that the assembly following
+ * it will be executed only when coming from user space
+ */
+.macro switch_stack sp_reg task_reg save_regs_label
+ get \sp_reg = $sps
+ ;;
+ /* Check if $sps.pl bit is 0 (ie in kernel mode)
+ * if so, then we dont have to switch stack */
+ cb.even \sp_reg? \save_regs_label
+ /* Copy original $sp in a scratch reg */
+ copyd \sp_reg = $sp
+ ;;
+ /* restore sp from kernel stack pointer */
+ ld $sp = __PA(TASK_THREAD_KERNEL_SP)[\task_reg]
+ ;;
+.endm
+
+/**
+ * Disable MMU using a specified register
+ * scratch_reg: Scratch register to be used for $ps modification (clobbered)
+ * ps_val: Additionnal $ps modifications after returning disabling MMU
+ */
+.macro disable_mmu scratch_reg ps_val=0
+ /* Disable MME in $sps */
+ make \scratch_reg = MME_WFXL(0)
+ ;;
+ wfxl $sps = \scratch_reg
+ ;;
+ /* Enable DAUS in $ps */
+ make \scratch_reg = SFR_SET_VAL_WFXL(PS, DAUS, 1) | \ps_val
+ ;;
+ wfxl $ps = \scratch_reg
+ /* We are now accessing data with MMU disabled */
+ ;;
+.endm
+
+/**
+ * Disable MMU when entering exception path
+ * This macro does not require any register since used register will be
+ * restored after use. This can safely be used directly after entering
+ * exceptions.
+ */
+.macro exception_entry_disable_mmu
+ set TMP_SCRATCH_SR = $r0
+ disable_mmu $r0
+ ;;
+ get $r0 = TMP_SCRATCH_SR
+ ;;
+.endm
+
+/* Save callee registers on stack */
+.macro save_callee
+ sq PT_R18R19[$sp] = $r18r19
+ ;;
+ so PT_Q20[$sp] = $r20r21r22r23
+ ;;
+ so PT_Q24[$sp] = $r24r25r26r27
+ ;;
+ so PT_Q28[$sp] = $r28r29r30r31
+ ;;
+.endm
+
+/**
+ * Save registers for entry in kernel space.
+ * sp_reg should point to the original stack pointer when entering kernel space
+ * task_reg is a register containing the current task pointer
+ * both task_reg and sp_reg must be in saved_quad since they have been cloberred
+ * and will be restored when restoring saved_quad.
+ * pt_regs_sp is a single register which will be filled by pt_regs addr.
+ * When "returning" from this macro, hardware loop are enabled and exception
+ * is cleared, allowing to call kernel function without any risk.
+ */
+.macro save_regs_for_exception sp_reg task_reg saved_quad pt_regs_sp
+.if (\saved_quad==$r4r5r6r7 || \saved_quad==$r60r61r62r63)
+.error "saved_quad must not be $r4r5r6r7 or $r60r61r62r63 !"
+.endif
+ /* make some room on stack to save registers */
+ addd $sp = $sp, -(PT_SIZE_ON_STACK)
+ so __PA(PT_Q4-PT_SIZE_ON_STACK)[$sp] = $r4r5r6r7
+ /* Compute physical address of stack to avoid using 64bits immediate */
+ addd $r6 = $sp, VA_TO_PA_OFFSET - (PT_SIZE_ON_STACK)
+ ;;
+ so PT_Q60[$r6] = $r60r61r62r63
+ /* Now that $r60r61r62r63 is saved, we can use it for saving
+ * original stack stored in scratch_reg. Note that we can not
+ * use the r12r13r14r15 quad to do that because it would
+ * modify the current $r12/sp ! This is if course not what we
+ * want and hence we use the freshly saved quad $r60r61r62r63.
+ *
+ * Note that we must use scratch_reg before reloading the saved
+ * quad since the scratch reg is contained in it, so reloading
+ * it before copying it would overwrite it.
+ */
+ copyd $r60 = \sp_reg
+ ;;
+ /* Reload the saved quad registers to save correct values
+ * Since we use the scratch reg before that */
+ lo \saved_quad = __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[\task_reg]
+ ;;
+ so PT_Q8[$r6] = $r8r9r10r11
+ ;;
+ so PT_Q0[$r6] = $r0r1r2r3
+ copyd $r61 = $r13
+ get $r5 = $le
+ ;;
+ sq PT_R16R17[$r6] = $r16r17
+ make $r10 = 0x0
+ copyd $r62 = $r14
+ ;;
+ so PT_Q32[$r6] = $r32r33r34r35
+ /* Since we are going to enable hardware loop, we must be careful
+ * and reset le (loop exit) to avoid any exploit and return to
+ * user with kernel mode */
+ set $le = $r10
+ copyd $r63 = $r15
+ ;;
+ so PT_Q36[$r6] = $r36r37r38r39
+ get $r0 = $cs
+ ;;
+ so PT_Q40[$r6] = $r40r41r42r43
+ get $r1 = $spc
+ ;;
+ so PT_Q44[$r6] = $r44r45r46r47
+ get $r2 = $sps
+ ;;
+ so PT_Q48[$r6] = $r48r49r50r51
+ get $r3 = $es
+ ;;
+ so PT_Q52[$r6] = $r52r53r54r55
+ get $r7 = $ra
+ ;;
+ so PT_Q56[$r6] = $r56r57r58r59
+ get $r4 = $lc
+ ;;
+ so PT_Q12[$r6] = $r60r61r62r63
+ copyd $r63 = $r6
+ get $r6 = $ls
+ ;;
+ so PT_CS_SPC_SPS_ES[$r63] = $r0r1r2r3
+ ;;
+ so PT_LC_LE_LS_RA[$r63] = $r4r5r6r7
+ /* Clear frame pointer */
+ make $fp = 0
+ ;;
+ /* Copy regs stack pointer for macro caller */
+ copyd \pt_regs_sp = $sp
+#ifdef CONFIG_DEBUG_EXCEPTION_STACK
+ addd $sp = $sp, -STACK_REG_SIZE
+ ;;
+ sd VA_TO_PA_OFFSET[$sp] = $sp
+ ;;
+#endif
+ /* Reenable hwloop, MMU and clear exception taken */
+ make $r8 = PS_HWLOOP_EN_ET_CLEAR_DAUS_DIS
+ ;;
+ wfxl $ps, $r8
+ ;;
+.endm
+
+/***********************************************************************
+* Exception vectors trampolines
+***********************************************************************/
+#define exception_trampoline(__type) \
+.section .exception.## __type, "ax", @progbits ;\
+ENTRY(kvx_##__type ##_handler_trampoline): ;\
+ goto kvx_## __type ##_handler ;\
+ ;; ;\
+ENDPROC(kvx_ ## __type ## _handler_trampoline) ;\
+.section .early_exception.## __type, "ax", @progbits ;\
+ENTRY(kvx_## __type ##_early_handler): ;\
+1: nop ;\
+ ;; ;\
+ goto 1b ;\
+ ;; ;\
+ENDPROC(kvx_ ## __type ## _early_handler)
+
+exception_trampoline(debug)
+exception_trampoline(trap)
+exception_trampoline(interrupt)
+exception_trampoline(syscall)
+
+#define EXCEPTION_ENTRY(__name) \
+.section .exception.text, "ax", @progbits ;\
+ENTRY(__name)
+
+
+/***********************************************************************
+* Common exception return path
+***********************************************************************/
+/**
+ * Restore registers after exception
+ * When entering this macro, $sp must be located right before regs
+ * storage.
+ */
+EXCEPTION_ENTRY(return_from_exception)
+#ifdef CONFIG_DEBUG_EXCEPTION_STACK
+ ld $r1 = 0[$sp]
+ ;;
+ sbfd $r1 = $r1, $sp
+ ;;
+ cb.deqz $r1, _check_ok
+ ;;
+ make $r2 = panic
+ make $r0 = stack_error_panic_str_label
+ ;;
+ icall $r2
+ ;;
+_check_ok:
+ addd $sp = $sp, STACK_REG_SIZE
+ ;;
+#endif
+ get $r11 = $sr
+ /* Load sps value from saved registers */
+ ld $r6 = PT_SPS[$sp]
+ ;;
+ /* Disable interrupt to check task flags atomically */
+ disable_interrupt $r60
+ ;;
+ /* Check PL bit of sps, if set, then it means we are returning
+ * to a lower privilege level (ie to user), if so, we need to
+ * check work pending. If coming from kernel, directly go to
+ * register restoration */
+ cb.even $r6? _restore_regs
+ ld $r1 = TASK_TI_FLAGS[$r11]
+ ;;
+ /* Do we have work pending ? */
+ andd $r5 = $r1, _TIF_WORK_MASK
+ ;;
+ /**
+ * If we do not have work pending (ie $r5 == 0) then we can
+ * directly jump to _restore_regs without calling do_work_pending
+ */
+ cb.deqz $r5? _restore_regs
+ ;;
+ /*
+ * Work pending can potentially call a signal handler and then return
+ * via rt_sigreturn. Return path will be different (restore all regs)
+ * and hence all registers are needed to be saved.
+ */
+ save_callee
+ ;;
+ call_do_work_pending
+ ;;
+#ifdef CONFIG_TRACE_IRQFLAGS
+ /* reload sps value from saved registers */
+ ld $r6 = PT_SPS[$sp]
+ ;;
+#endif
+_restore_regs:
+#ifdef CONFIG_TRACE_IRQFLAGS
+ /* Check if IRQs are going to be reenable in next context */
+ andd $r6 = $r6, KVX_SFR_SPS_IE_MASK
+ ;;
+ cb.deqz $r6? 1f
+ ;;
+ call trace_hardirqs_on
+ ;;
+1:
+#endif
+ disable_mmu $r0, PS_HWLOOP_DISABLE
+ ;;
+ lo $r0r1r2r3 = __PA(PT_CS_SPC_SPS_ES)[$sp]
+ /* Compute physical address of stack to avoid using 64bits immediate */
+ addd $r11 = $sp, VA_TO_PA_OFFSET
+ ;;
+ lo $r4r5r6r7 = PT_LC_LE_LS_RA[$r11]
+ ;;
+ lo $r60r61r62r63 = PT_Q60[$r11]
+ ;;
+ lo $r56r57r58r59 = PT_Q56[$r11]
+ ;;
+ lo $r52r53r54r55 = PT_Q52[$r11]
+ get $r14 = $sps
+ ;;
+ lo $r48r49r50r51 = PT_Q48[$r11]
+ /* Generate a mask of ones at each bit where the current $sps
+ * differs from the $sps to be restored
+ */
+ xord $r14 = $r2, $r14
+ /* prepare wfxl clear mask on LSBs */
+ notd $r15 = $r2
+ /* prepare wfxl set mask on MSBs */
+ slld $r13 = $r2, 32
+ ;;
+ lo $r44r45r46r47 = PT_Q44[$r11]
+ /* Replicate mask of ones on the 32 MSBs */
+ sbmm8 $r14 = $r14, REPLICATE_32_MASK
+ /* Combine the set and clear mask for wfxl */
+ insf $r13 = $r15, 31, 0
+ ;;
+ lo $r40r41r42r43 = PT_Q40[$r11]
+ set $lc = $r4
+ /* Mask to drop identical bits in order to avoid useless
+ * privilege traps
+ */
+ andd $r13 = $r13, $r14
+ ;;
+ lq $r16r17 = PT_R16R17[$r11]
+ set $le = $r5
+ ;;
+ lo $r32r33r34r35 = PT_Q32[$r11]
+ set $ls = $r6
+ ;;
+ lo $r36r37r38r39 = PT_Q36[$r11]
+ copyd $r14 = $r11
+ set $ra = $r7
+ ;;
+ lo $r8r9r10r11 = PT_Q8[$r14]
+ set $cs = $r0
+ /* MME was disabled by disable_mmu, reenable it before leaving */
+ ord $r13 = $r13, SFR_SET_VAL_WFXL(SPS, MME, 1)
+ ;;
+ lo $r4r5r6r7 = PT_Q4[$r14]
+ set $spc = $r1
+ ;;
+ lo $r0r1r2r3 = PT_Q0[$r14]
+ /* Store $sps wfxl value in scratch system register */
+ set TMP_SCRATCH_SR = $r13
+ ;;
+ lo $r12r13r14r15 = PT_Q12[$r14]
+ rswap $r0 = TMP_SCRATCH_SR
+ ;;
+ wfxl $sps = $r0
+ ;;
+ /* Finally, restore $r0 value */
+ rswap $r0 = TMP_SCRATCH_SR
+ ;;
+ rfe
+ ;;
+ENDPROC(return_from_exception)
+
+/***********************************************************************
+* Debug handling
+***********************************************************************/
+EXCEPTION_ENTRY(kvx_debug_handler):
+ exception_entry_disable_mmu
+ ;;
+ save_quad_regs $r0r1r2r3 $r4
+ ;;
+ get $r2 = $sr
+ ;;
+ switch_stack $r1 $r2 debug_save_regs
+ ;;
+debug_save_regs:
+ save_regs_for_exception $r1 $r2 $r0r1r2r3 $r1
+ ;;
+ get $r0 = $ea
+ /* tail-call for return_from_exception */
+ make $r3 = return_from_exception
+ ;;
+ set $ra = $r3
+ ;;
+ goto debug_handler
+ ;;
+ENDPROC(kvx_debug_handler)
+
+/***********************************************************************
+* Traps handling
+***********************************************************************/
+/* These labels will be used for instruction patching */
+.global kvx_perf_tlb_refill, kvx_std_tlb_refill
+EXCEPTION_ENTRY(kvx_trap_handler):
+ /* Enable DAUS for physical data access */
+ exception_entry_disable_mmu
+ ;;
+ /* Save r3 in a temporary system register to check if the trap is a
+ * nomapping or not */
+ set TMP_SCRATCH_SR = $r3
+ ;;
+ get $r3 = $es
+ ;;
+ /* Hardware trap cause */
+ extfz $r3 = $r3, KVX_SFR_END(ES_HTC), KVX_SFR_START(ES_HTC)
+ ;;
+ /* Is this a nomapping trap ? */
+ compd.eq $r3 = $r3, KVX_TRAP_NOMAPPING
+ ;;
+ /* if nomapping trap, try fast_refill */
+ cb.even $r3? trap_slow_path
+ ;;
+ /*
+ * Fast TLB refill routine
+ *
+ * On kvx, we do not have hardware page walking, hence, TLB refill is
+ * done using the core on no-mapping traps.
+ * This routine must be as fast as possible to avoid wasting CPU time.
+ * For that purpose, it is called directly from trap_handle after saving
+ * only 8 registers ($r0 -> $r7) in a dedicated buffer.
+ * To avoid taking nomapping while accessing page tables inside this
+ * refill handler we switch to physical accesses using DAUS.
+ * Once the switch is done, we save up to 8 registers to compute
+ * the refill data.
+ * This allows to avoid computing a complete task switching in order
+ * to greatly reduce the refill time.
+ *
+ * We refill the JTLB which contains 128 sets with 4 way each.
+ * Currently, the way selection is done using a round robin algorithm.
+ *
+ * The refill is using the basic flow:
+ * 1 - Enable physical access using DAUS.
+ * 2 - Save necessary registers
+ * 3 - Walk the page table to find the TLB entry to add (virtual to
+ * physical)
+ * 4 - Compute the TLB entry to be written (convert PTE to TLB entry)
+ * 5 - Compute the target set (0 -> 127) for the new TLB entry
+ * This is done by extracting the 6 lsb of page number
+ * 6 - Get the current way to be used which is selected using a
+ simple round robin
+ * 7 - Mark PTE entry as _PAGE_ACCESSED (and optionally PAGE_DIRTY)
+ * 8 - Commit the new tlb entry
+ * 9 - Restore the virtual memory by disabling DAUS.
+ *
+ */
+ /* Get current task */
+ rswap $r63 = $sr
+ ;;
+#ifdef CONFIG_KVX_MMU_STATS
+ get $r3 = $pm0
+ ;;
+ sd __PA(TASK_THREAD_ENTRY_TS)[$r63] = $r3
+ ;;
+#endif
+ /* Restore $r3 from temporary system scratch register */
+ get $r3 = TMP_SCRATCH_SR
+ /* Save registers to save $tel, $teh and $mmc */
+ so __PA(TASK_THREAD_SAVE_AREA_QUAD(2))[$r63] = $r8r9r10r11
+ ;;
+ get $r8 = $tel
+ /* Save register for refill handler */
+ so __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r63] = $r4r5r6r7
+ ;;
+ /* Get exception address */
+ get $r0 = $ea
+ /* Save more registers to be comfy */
+ so __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[$r63] = $r0r1r2r3
+ ;;
+ get $r9 = $teh
+ ;;
+ get $r10 = $mmc
+ ;;
+ /* Restore $r63 value */
+ rswap $r63 = $sr
+ /* Check kernel address range */
+ addd $r4 = $r0, -KERNEL_DIRECT_MEMORY_MAP_BASE
+ /* Load task active mm for pgd loading in macro_tlb_refill */
+ ld $r1 = __PA(TASK_ACTIVE_MM)[$r63]
+ ;;
+kvx_perf_tlb_refill:
+ /* Check if the address is in the kernel direct memory mapping */
+ compd.ltu $r3 = $r4, KERNEL_DIRECT_MEMORY_MAP_SIZE
+ /* Clear low bits of virtual address to align on page size */
+ andd $r5 = $r0, ~(REFILL_PERF_PAGE_SIZE - 1)
+ /* Create corresponding physical address */
+ addd $r2 = $r4, PHYS_OFFSET
+ ;;
+ /* If address is not a kernel one, take the standard path */
+ cb.deqz $r3? kvx_std_tlb_refill
+ /* Prepare $teh value with virtual address and kernel value */
+ ord $r7 = $r5, REFILL_PERF_TEH_VAL
+ ;;
+ /* Get $pm0 value as a pseudo random value for LTLB way to use */
+ get $r4 = $pm0
+ /* Clear low bits of physical address to align on page size */
+ andd $r2 = $r2, ~(REFILL_PERF_PAGE_SIZE - 1)
+ /* Prepare value for $mmc wfxl to select LTLB and correct way */
+ make $r5 = MMC_SEL_LTLB_CLEAR_WAY
+ ;;
+ /* Keep low bits of timer value */
+ andw $r4 = $r4, (REFILL_PERF_ENTRIES - 1)
+ /* Get current task pointer for register restoration */
+ get $r11 = $sr
+ ;;
+ /* Add LTLB base way number for kernel refill way */
+ addw $r4 = $r4, LTLB_KERNEL_RESERVED
+ /* Prepare $tel value with physical address and kernel value */
+ ord $r6 = $r2, REFILL_PERF_TEL_VAL
+ set $teh = $r7
+ ;;
+ /* insert way in $mmc wfxl value */
+ insf $r5 = $r4, KVX_SFR_END(MMC_SW) + 32, KVX_SFR_START(MMC_SW) + 32
+ set $tel = $r6
+ ;;
+ wfxl $mmc = $r5
+ ;;
+ goto do_tlb_write
+ ;;
+kvx_std_tlb_refill:
+ /* extract PGD offset */
+ extfz $r3 = $r0, (ASM_PGDIR_SHIFT + ASM_PGDIR_BITS - 1), ASM_PGDIR_SHIFT
+ /* is mm NULL ? if so, use init_mm */
+ cmoved.deqz $r1? $r1 = init_mm
+ ;;
+ get $r7 = $pcr
+ /* Load pgd base address into $r1 */
+ ld $r1 = __PA(MM_PGD)[$r1]
+ ;;
+ /* Add offset for physical address */
+ addd $r1 = $r1, VA_TO_PA_OFFSET
+ /* Extract processor ID to compute cpu_offset*/
+ extfz $r7 = $r7, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID)
+ ;;
+ /* Load PGD entry offset */
+ ld.xs $r1 = $r3[$r1]
+ /* Load per_cpu_offset */
+#if defined(CONFIG_SMP)
+ make $r5 = __PA(__per_cpu_offset)
+#endif
+ ;;
+ /* extract PMD offset*/
+ extfz $r3 = $r0, (ASM_PMD_SHIFT + ASM_PMD_BITS - 1), ASM_PMD_SHIFT
+ /* If pgd entry is null -> out */
+ cb.deqz $r1? refill_err_out
+#if defined(CONFIG_SMP)
+ /* Load cpu offset */
+ ld.xs $r7 = $r7[$r5]
+#else
+ /* Force cpu offset to 0 */
+ make $r7 = 0
+#endif
+ ;;
+ /* Load PMD entry offset and keep pointer to the entry for huge page */
+ addx8d $r2 = $r3, $r1
+ ld.xs $r1 = $r3[$r1]
+ ;;
+ /* Check if it is a huge page (2Mb or 512Mb in PMD table)*/
+ andd $r6 = $r1, _PAGE_HUGE
+ /* If pmd entry is null -> out */
+ cb.deqz $r1? refill_err_out
+ /* extract PTE offset */
+ extfz $r3 = $r0, (PAGE_SHIFT + 8), PAGE_SHIFT
+ ;;
+ /*
+ * If the page is a huge one we already have set the PTE and the
+ * pointer to the PTE.
+ */
+ cb.dnez $r6? is_huge_page
+ ;;
+ /* Load PTE entry */
+ ld.xs $r1 = $r3[$r1]
+ addx8d $r2 = $r3, $r1
+ ;;
+ /* Check if it is a huge page (64Kb in PTE table) */
+ andd $r6 = $r1, _PAGE_HUGE
+ ;;
+ /* Check if PTE entry is for a huge page */
+ cb.dnez $r6? is_huge_page
+ ;;
+ /* 4K: Extract set value */
+ extfz $r0 = $r1, (PAGE_SHIFT + KVX_SFR_MMC_SS_WIDTH - 1), PAGE_SHIFT
+ /* 4K: Extract virt page from ea */
+ andd $r4 = $r0, PAGE_MASK
+ ;;
+/*
+ * This path expects the following:
+ * - $r0 = set index
+ * - $r1 = pte entry
+ * - $r2 = pte entry address
+ * - $r4 = virtual page address
+ */
+pte_prepared:
+ /* Compute per_cpu_offset + current way of set address */
+ addd $r5 = $r0, $r7
+ /* Get exception cause for access type handling (page dirtying) */
+ get $r7 = $es
+ /* Clear way and select JTLB */
+ make $r6 = MMC_SEL_JTLB_CLEAR_WAY
+ ;;
+ /* Load current way to use for current set */
+ lbz $r0 = __PA(jtlb_current_set_way)[$r5]
+ /* Check if the access was a "write" access */
+ andd $r7 = $r7, (KVX_TRAP_RWX_WRITE << KVX_SFR_ES_RWX_SHIFT)
+ ;;
+ /* If bit PRESENT of pte entry is 0, then entry is not present */
+ cb.even $r1? refill_err_out
+ /*
+ * Set the JTLB way in $mmc value, add 32 bits to be in the set part.
+ * Since we are refilling JTLB, we must make sure we insert only
+ * relevant bits (ie 2 bits for ways) to avoid using nonexistent ways.
+ */
+ insf $r6 = $r0, KVX_SFR_START(MMC_SW) + 32 + (MMU_JTLB_WAYS_SHIFT - 1),\
+ KVX_SFR_START(MMC_SW) + 32
+ /* Extract page global bit */
+ extfz $r3 = $r1, _PAGE_GLOBAL_SHIFT, _PAGE_GLOBAL_SHIFT
+ /* Increment way value, note that we do not care about overflow since
+ * we only use the two lower byte */
+ addd $r0 = $r0, 1
+ ;;
+ /* Prepare MMC */
+ wfxl $mmc, $r6
+ ;;
+ /* Insert global bit (if any) to its position into $teh value */
+ insf $r4 = $r3, KVX_SFR_TEH_G_SHIFT, KVX_SFR_TEH_G_SHIFT
+ /* If "write" access ($r7 != 0), then set it as dirty */
+ cmoved.dnez $r7? $r7 = _PAGE_DIRTY
+ /* Clear bits not related to FN in the pte entry for TEL writing */
+ andd $r6 = $r1, KVX_PFN_MASK
+ /* Store new way */
+ sb __PA(jtlb_current_set_way)[$r5] = $r0
+ ;;
+ /* Extract access perms from pte entry (discard PAGE_READ bit +1) */
+ extfz $r3 = $r1, KVX_ACCESS_PERM_STOP_BIT, KVX_ACCESS_PERM_START_BIT + 1
+ /* Move FN bits to their place */
+ srld $r6 = $r6, KVX_PFN_SHIFT - PAGE_SHIFT
+ /* Extract the page size + cache policy */
+ andd $r0 = $r1, (KVX_PAGE_SZ_MASK | KVX_PAGE_CP_MASK)
+ /* Prepare SBMM value */
+ make $r5 = KVX_SBMM_BYTE_SEL
+ ;;
+ /* Add page size + cache policy to $tel value */
+ ord $r6 = $r6, $r0
+ /* Get $mmc to get current ASN */
+ get $r0 = $mmc
+ /* Add _PAGE_ACCESSED bit to PTE entry for writeback */
+ ord $r7 = $r7, _PAGE_ACCESSED
+ ;;
+ /* OR PTE value with accessed/dirty flags */
+ ord $r1 = $r1, $r7
+ /* Generate the byte selection for sbmm */
+ slld $r5 = $r5, $r3
+ /* Compute the mask to extract set and mask exception address */
+ make $r7 = KVX_PAGE_PA_MATRIX
+ ;;
+ ord $r0 = $r6, TEL_DEFAULT_VALUE
+ /* Add ASN from mmc into future $teh value */
+ insf $r4 = $r0, KVX_SFR_END(MMC_ASN), KVX_SFR_START(MMC_ASN)
+ /* Get the page permission value */
+ sbmm8 $r6 = $r7, $r5
+ /* Check PAGE_READ bit in PTE entry */
+ andd $r3 = $r1, _PAGE_READ
+ ;;
+ /* If PAGE_READ bit is not set, set policy as NA_NA */
+ cmoved.deqz $r3? $r6 = TLB_PA_NA_NA
+ ;;
+ /* Shift PA to correct position */
+ slld $r6 = $r6, KVX_SFR_TEL_PA_SHIFT
+ set $teh = $r4
+ ;;
+ /* Store updated pte entry */
+ sd 0[$r2] = $r1
+ /* Prepare tel */
+ ord $r6 = $r0, $r6
+ /* Get current task pointer for register restoration */
+ get $r11 = $sr
+ ;;
+ set $tel = $r6
+ ;;
+do_tlb_write:
+ tlbwrite
+ ;;
+#ifdef CONFIG_KVX_DEBUG_TLB_WRITE
+ goto mmc_error_check
+ ;;
+mmc_error_check_ok:
+#endif
+#ifdef CONFIG_KVX_DEBUG_ASN
+ goto asn_check
+ ;;
+asn_check_ok:
+#endif
+ set $tel = $r8
+ /* Restore registers */
+ lo $r4r5r6r7 = __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r11]
+ ;;
+ set $teh = $r9
+ lo $r0r1r2r3 = __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[$r11]
+ ;;
+ set $mmc = $r10
+ ;;
+ lo $r8r9r10r11 = __PA(TASK_THREAD_SAVE_AREA_QUAD(2))[$r11]
+ ;;
+#ifdef CONFIG_KVX_MMU_STATS
+ /*
+ * Fence to simulate a direct data dependency after returning from trap
+ * nomapping handling. This is the worst case that can happen and the
+ * processor will be stalled waiting for previous loads to complete.
+ */
+ fence
+ ;;
+ get $r4 = $pm0
+ ;;
+ get $r0 = $sr
+ ;;
+ /* Get cycles measured on trap entry */
+ ld $r1 = __PA(TASK_THREAD_ENTRY_TS)[$r0]
+ ;;
+ /* Compute refill time */
+ sbfd $r0 = $r1, $r4
+ ;;
+#ifdef CONFIG_SMP
+ get $r1 = $pcr
+ ;;
+ /* Extract processor ID to compute cpu_offset */
+ extfz $r1 = $r1, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID)
+ make $r2 = __PA(__per_cpu_offset)
+ ;;
+ /* Load cpu offset */
+ ld.xs $r1 = $r1[$r2]
+ ;;
+ addd $r1 = $r1, __PA(mmu_stats)
+ ;;
+#else
+ make $r1 = __PA(mmu_stats)
+ ;;
+#endif
+ /* Load time between refill + last refill cycle */
+ lq $r2r3 = MMU_STATS_CYCLES_BETWEEN_REFILL_OFF[$r1]
+ ;;
+ /* Set last update time to current if 0 */
+ cmoved.deqz $r3? $r3 = $r4
+ /* remove current refill time to current cycle */
+ sbfd $r4 = $r0, $r4
+ ;;
+ /* Compute time between last refill and current refill */
+ sbfd $r5 = $r3, $r4
+ /* Update last cycle time */
+ copyd $r3 = $r4
+ ;;
+ /* Increment total time between refill */
+ addd $r2 = $r2, $r5
+ ;;
+ sq MMU_STATS_CYCLES_BETWEEN_REFILL_OFF[$r1] = $r2r3
+ /* Get exception address */
+ get $r4 = $ea
+ ;;
+ /* $r2 holds refill type (user/kernel/kernel_direct) */
+ make $r2 = MMU_STATS_REFILL_KERNEL_OFF
+ /* Check if address is a kernel direct mapping one */
+ compd.ltu $r3 = $r4, (KERNEL_DIRECT_MEMORY_MAP_BASE + \
+ KERNEL_DIRECT_MEMORY_MAP_SIZE)
+ ;;
+ cmoved.dnez $r3? $r2 = MMU_STATS_REFILL_KERNEL_DIRECT_OFF
+ /* Check if address is a user (ie below kernel) */
+ compd.ltu $r3 = $r4, KERNEL_DIRECT_MEMORY_MAP_BASE
+ ;;
+ cmoved.dnez $r3? $r2 = MMU_STATS_REFILL_USER_OFF
+ ;;
+ /* Compute refill struct addr into one reg */
+ addd $r1 = $r2, $r1
+ /* Load refill_struct values */
+ lo $r4r5r6r7 = $r2[$r1]
+ ;;
+ /* Increment count */
+ addd $r4 = $r4, 1
+ /* Increment total cycles count */
+ addd $r5 = $r5, $r0
+ ;;
+ /* Set min to ~0 if 0 */
+ cmoved.deqz $r6? $r6 = ~0
+ ;;
+ /* Compare min and max */
+ compd.ltu $r2 = $r0, $r6
+ compd.gtu $r3 = $r0, $r7
+ ;;
+ /* Update min and max*/
+ cmoved.dnez $r2? $r6 = $r0
+ cmoved.dnez $r3? $r7 = $r0
+ ;;
+ /* store back all values */
+ so 0[$r1] = $r4r5r6r7
+ ;;
+ get $r0 = $sr
+ ;;
+ /* Restore cloberred registers */
+ lo $r4r5r6r7 = __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r0]
+ ;;
+ lo $r0r1r2r3 = __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[$r0]
+ ;;
+#endif
+ /* Save $r4 for reenabling mmu and data cache in sps */
+ set TMP_SCRATCH_SR = $r4
+ /* Enable MME in $sps */
+ make $r4 = MME_WFXL(1)
+ ;;
+ /* Reenable $mme in $sps */
+ wfxl $sps = $r4
+ ;;
+ get $r4 = TMP_SCRATCH_SR
+ ;;
+ rfe
+ ;;
+
+is_huge_page:
+ /*
+ * When entering this path:
+ * - $r0 = $ea
+ * - $r1 = pte entry
+ * - $r7 = cpu offset for tlb_current_set_way
+ *
+ * From now on, we have the pte value in $r1 so we can extract the page
+ * size. This value is stored as it is expected by the MMU (ie between
+ * 0 and 3).
+ * Note that page size value is located at the same place as in $tel
+ * and this is checked at build time so we can use TEL_PS defines.
+ * In this codepath, we will extract the set and mask exception address
+ * and align virt and phys address with what the hardware expect.
+ * Indeed, MMU expect lsb of the virtual and physycal address to be 0
+ * according to page size.
+ * This means that for 4K pages, the 12 lsb must be 0, for 64K
+ * pages, the 16 lsb must be 0 and so on.
+ */
+ extfz $r5 = $r1, KVX_SFR_END(TEL_PS), KVX_SFR_START(TEL_PS)
+ /* Compute the mask to extract set and mask exception address */
+ make $r4 = KVX_PS_SHIFT_MATRIX
+ make $r6 = KVX_SBMM_BYTE_SEL
+ ;;
+ /* Generate the byte selection for sbmm */
+ slld $r6 = $r6, $r5
+ ;;
+ /* Get the shift value */
+ sbmm8 $r5 = $r4, $r6
+ make $r4 = 0xFFFFFFFFFFFFFFFF
+ ;;
+ /* extract TLB set from ea (6 lsb of virtual page) */
+ srld $r5 = $r0, $r5
+ /* Generate ea masking according to page shift */
+ slld $r4 = $r4, $r5
+ ;;
+ /* Mask to get the set value */
+ andd $r0 = $r5, MMU_SET_MASK
+ /* Extract virt page from ea */
+ andd $r4 = $r0, $r4
+ ;;
+ /* Returned to fast path */
+ goto pte_prepared
+ ;;
+
+#ifdef CONFIG_KVX_DEBUG_TLB_WRITE
+mmc_error_check:
+ get $r1 = $mmc
+ ;;
+ andd $r1 = $r1, KVX_SFR_MMC_E_MASK
+ ;;
+ cb.deqz $r1? mmc_error_check_ok
+ ;;
+ make $r0 = mmc_error_panic_str_label
+ goto asm_panic
+ ;;
+#endif
+#ifdef CONFIG_KVX_DEBUG_ASN
+/*
+ * When entering this path $r11 = $sr.
+ * WARNING: Do not clobber it here if you don't want to mess up with registers
+ * restoration above.
+ */
+asn_check:
+ get $r1 = $ea
+ ;;
+ /* Check if kernel address, if so, there is no ASN */
+ compd.geu $r2 = $r1, PAGE_OFFSET
+ ;;
+ cb.dnez $r2? asn_check_ok
+ ;;
+ get $r2 = $pcr
+ /* Load active mm addr */
+ ld $r3 = __PA(TASK_ACTIVE_MM)[$r11]
+ ;;
+ get $r5 = $mmc
+ /* Extract processor ID to compute cpu_offset*/
+ extfz $r2 = $r2, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID)
+ addd $r3 = $r3, MM_CTXT_ASN
+ ;;
+ extfz $r4 = $r5, KVX_SFR_END(MMC_ASN), KVX_SFR_START(MMC_ASN)
+ addd $r3 = $r3, VA_TO_PA_OFFSET
+ ;;
+ /* Load current asn from active_mm */
+ ld.xs $r3 = $r2[$r3]
+ ;;
+ /* Error if ASN is not set */
+ cb.deqz $r3? asn_check_err
+ /* Mask $r3 asn cycle part */
+ andd $r5 = $r3, ((1 << KVX_SFR_MMC_ASN_WIDTH) - 1)
+ ;;
+ /* Compare asn in $mmc and asn in current task mm */
+ compd.eq $r3 = $r5, $r4
+ ;;
+ cb.dnez $r3? asn_check_ok
+ ;;
+asn_check_err:
+ /* We are fried, die peacefully */
+ make $r0 = asn_error_panic_str_label
+ goto asm_panic
+ ;;
+#endif
+
+#if defined(CONFIG_KVX_DEBUG_ASN) || defined(CONFIG_KVX_DEBUG_TLB_WRITE)
+
+/**
+ * This routine calls panic from assembly after setting appropriate things
+ * $r0 = panic string
+ */
+asm_panic:
+ /*
+ * Reenable hardware loop and traps (for nomapping) since some functions
+ * might need it. Moreover, disable DAUS to reenable MMU accesses.
+ */
+ make $r32 = PS_HWLOOP_EN_ET_CLEAR_DAUS_DIS
+ make $r33 = 0
+ get $r34 = $sr
+ ;;
+ /* Clear hw loop exit to disable current loop */
+ set $le = $r33
+ ;;
+ wfxl $ps = $r32
+ ;;
+ /* Restore kernel stack */
+ ld $r12 = TASK_THREAD_KERNEL_SP[$r34]
+ ;;
+ call panic
+ ;;
+#endif
+
+/* Error path for TLB refill */
+refill_err_out:
+ get $r2 = $sr
+ ;;
+ /* Restore clobbered registers */
+ lo $r8r9r10r11 = __PA(TASK_THREAD_SAVE_AREA_QUAD(2))[$r2]
+ ;;
+ lo $r4r5r6r7 = __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r2]
+ goto trap_switch_stack
+ ;;
+
+/* This path is entered only when the trap is NOT a NOMAPPING */
+trap_slow_path:
+ /* Restore $r3 from temporary scratch system register */
+ get $r3 = TMP_SCRATCH_SR
+ ;;
+ save_quad_regs $r0r1r2r3 $r4
+ ;;
+ get $r2 = $sr
+ ;;
+trap_switch_stack:
+ switch_stack $r1 $r2 trap_save_regs
+ ;;
+trap_save_regs:
+ save_regs_for_exception $r1 $r2 $r0r1r2r3 $r2
+ ;;
+ get $r1 = $ea
+ /* tail-call for return_from_exception */
+ make $r3 = return_from_exception
+ ;;
+ set $ra = $r3
+ ;;
+ /* Handler call */
+ get $r0 = $es
+ ;;
+ goto trap_handler
+ ;;
+ENDPROC(kvx_trap_handler)
+
+/***********************************************************************
+* Interrupts handling
+***********************************************************************/
+EXCEPTION_ENTRY(kvx_interrupt_handler):
+ exception_entry_disable_mmu
+ ;;
+ save_quad_regs $r0r1r2r3 $r4
+ ;;
+ get $r0 = $sr
+ ;;
+ switch_stack $r1 $r0 it_save_regs
+ ;;
+#ifdef CONFIG_SECURE_DAME_HANDLING
+ /**
+ * In order to securely Handle Data Asynchronous Memory Error,
+ * we need to have a correct entry point. This means we do not
+ * want to handle a user induced DAME interrupt when entering
+ * kernel.
+ * In order to do that, we need to do a barrier, which will
+ * reflect the DAME status in $ilr (if any).
+ */
+ barrier
+ ;;
+#endif
+it_save_regs:
+ save_regs_for_exception $r1 $r0 $r0r1r2r3 $r1
+ ;;
+ get $r0 = $ilr
+ ;;
+ get $r2 = $ile
+ ;;
+ /**
+ * When an interrupt happens, the processor automatically clears the
+ * corresponding bit in $ilr. However, as we are using $ilr to get the
+ * list of irqs we want to handle, we need to add the automatically
+ * cleared interrupt bit. This is done by getting the interrupt number
+ * from $es.
+ */
+ get $r3 = $es
+ make $r4 = 1
+ ;;
+ /* Extract interrupt number from $es */
+ extfz $r3 = $r3, KVX_SFR_END(ES_ITN), KVX_SFR_START(ES_ITN)
+ /**
+ * Mask requests with enabled line since ILR will also contain disabled
+ * interrupt lines (ie not enabled in $ile) and we want to respect the
+ * current state of interrupt lines.
+ */
+ andd $r0 = $r0, $r2
+ ;;
+ /* Clear $ilr with bits we are going to handle */
+ wfxl $ilr = $r0
+ slld $r4 = $r4, $r3
+ ;;
+ /* OR the irq mask with the current pending irq */
+ ord $r0 = $r0, $r4
+ call do_IRQ
+ ;;
+ /* From now on, lower the interrupt level (IL) to allow IT nesting.
+ * If returning to user, we will call schedule which will reenable
+ * interrupts by itself when ready.
+ * If returning to kernel and with CONFIG_PREEMPT, we will call
+ * preempt_schedule_irq which will do the same.
+ */
+ make $r0 = PS_IL_CLEAR
+ ;;
+ wfxl $ps = $r0
+ ;;
+ goto return_from_exception
+ ;;
+ENDPROC(kvx_interrupt_handler)
+
+/***********************************************************************
+* Syscall handling
+***********************************************************************/
+EXCEPTION_ENTRY(kvx_syscall_handler):
+ /**
+ * Syscalls are seen as standard func call from ABI POV
+ * We may use all caller saved register whithout causing havoc
+ * in the userspace. We need to save callee registers because they
+ * will be restored when returning from fork.
+ * Note that r0 -> r11 MUST not be used since they are
+ * containing syscall parameters !
+ * During this function, $r38 is the syscall handler address. Hence,
+ * this register must not be cloberred during function calls (tracing
+ * for instance.
+ */
+ disable_mmu $r43
+ ;;
+ get $r43 = $es
+ copyd $r52 = $sp
+ copyd $r53 = $tp
+ ;;
+ get $r36 = $sr
+ copyd $r54 = $fp
+ copyd $r55 = $r15
+ ;;
+ /* Extract syscall number */
+ extfz $r32 = $r43, KVX_SFR_END(ES_SN), KVX_SFR_START(ES_SN)
+ /* Get regs to save on stack */
+ get $r63 = $ra
+ addd $r36 = $r36, VA_TO_PA_OFFSET
+ ;;
+ ld $r39 = TASK_TI_FLAGS[$r36]
+ get $r41 = $spc
+ make $r42 = __PA(sys_call_table)
+ ;;
+ /* Check for out-of-bound syscall number */
+ sbfd $r50 = $r32, __NR_syscalls
+ /* Compute syscall func addr (ie sys_call_table[$r32])*/
+ ld.xs $r38 = $r32[$r42]
+ get $r42 = $sps
+ ;;
+ /* True if trace syscall enable */
+ andd $r37 = $r39, _TIF_SYSCALL_WORK
+ /* Restore kernel stack pointer */
+ ld $sp = TASK_THREAD_KERNEL_SP[$r36]
+ /* If the syscall number is invalid, set invalid handler */
+ cmoved.dlez $r50? $r38 = sys_ni_syscall
+ ;;
+ /* Prepare space on stack */
+ addd $sp = $sp, -PT_SIZE_ON_STACK
+ get $r40 = $cs
+ /* Save regs r0 -> r3 in pt_regs for restart & trace if needed */
+ so __PA(PT_Q0 - PT_SIZE_ON_STACK)[$sp] = $r0r1r2r3
+ ;;
+ /* Store user stack pointer, frame pointer, thread pointer and r15 */
+ so __PA(PT_Q12)[$sp] = $r52r53r54r55
+ addd $r36 = $sp, VA_TO_PA_OFFSET
+ get $r60 = $lc
+ ;;
+#ifdef CONFIG_SECURE_DAME_HANDLING
+ get $r44 = $ilr
+ make $r45 = SFR_CLEAR_WFXL(ILR, IT16);
+ ;;
+ /* Extract $ilr.dame bit */
+ extfz $r44 = $r44, KVX_SFR_END(ILR_IT16), KVX_SFR_START(ILR_IT16)
+ /* Save $ilr value */
+ sd PT_ILR[$r36] = $r44
+ /* Clear $ilr.dame */
+ wfxl $ilr = $r45
+ ;;
+#endif
+ so PT_CS_SPC_SPS_ES[$r36] = $r40r41r42r43
+ get $r61 = $le
+ make $r43 = 0x0
+ ;;
+ /* Reenable hardware loops, IT, exceptions and disable DAUS */
+ make $r44 = PS_HLE_EN_IT_EN_ET_CLEAR_DAUS_DIS
+ get $r62 = $ls
+ /* Save regs r4 -> r7 in pt_regs for restart & trace if needed */
+ so PT_Q4[$r36] = $r4r5r6r7
+ ;;
+ /* Clear $le on entry */
+ set $le = $r43
+ /* Save hw loop stuff */
+ so PT_LC_LE_LS_RA[$r36] = $r60r61r62r63
+ /* Clear frame pointer for kernel */
+ make $fp = 0
+ ;;
+ /* Enable hwloop and interrupts and MMU
+ * Note that we have to reenable interrupts after saving context
+ * to avoid losing registers content */
+ wfxl $ps, $r44
+ ;;
+ /* Do we have to trace the syscall ? */
+ cb.dnez $r37? trace_syscall_enter
+ /* Stroe original r0 value */
+ sd PT_ORIG_R0[$sp] = $r0
+ ;;
+do_syscall:
+ /* Call the syscall handler */
+ icall $r38
+ ;;
+ /*
+ * rt_sigreturn syscall will not take this exit path (see
+ * sys_rt_sigreturn for more information).
+ */
+ /* Reload task flags since the syscall might have generated a signal*/
+ get $r36 = $sr
+ ;;
+#ifdef CONFIG_SECURE_DAME_HANDLING
+ /* Barrier to force DAME interrupt to be generated if any pending */
+ barrier
+ ;;
+#endif
+ /* Disable interrupt to check task flags atomically */
+ disable_interrupt $r60
+ ;;
+ ld $r38 = TASK_TI_FLAGS[$r36]
+ ;;
+ andd $r37 = $r38, _TIF_SYSCALL_WORK
+ ;;
+ /* Save r0 which was returned from do_scall previously and
+ * will be cloberred by do_work_pending(and potentially
+ * do_syscall_trace_exit if tracing is enabled)
+ * If do_signal is called and that syscall is restarted,
+ * it will be modified by handle_restart to restore original
+ * r0 value
+ */
+ sd PT_Q0[$sp] = $r0
+ /* used to store if trace system */
+ cb.dnez $r37? trace_syscall_exit
+ ;;
+check_work_pending:
+ /* Do we have work pending ? */
+ andd $r1 = $r38, _TIF_WORK_MASK
+ ;;
+ /* If no work pending, directly continue to ret_to_user */
+ cb.dnez $r1? call_work_pending
+ ;;
+ret_to_user:
+ disable_mmu $r60, PS_HWLOOP_DISABLE
+ ;;
+ /* Compute $sp virtual address to avoid using 64 bits offset */
+ addd $r15 = $sp, VA_TO_PA_OFFSET
+ ;;
+ /* Restore registers */
+ lo $r60r61r62r63 = PT_LC_LE_LS_RA[$r15]
+ get $r33 = $sps
+ ;;
+ lo $r40r41r42r43 = PT_CS_SPC_SPS_ES[$r15];
+ set $ra = $r63
+ ;;
+ /* Restore syscall arguments since they might be needed for
+ * syscall restart
+ */
+ lo $r0r1r2r3 = PT_Q0[$r15]
+ set $cs = $r40
+ /* Generate a mask of ones at each bit where the current $sps
+ * differs from the $sps to be restored
+ */
+ xord $r33 = $r42, $r33
+ /* prepare wfxl clear mask on LSBs */
+ notd $r34 = $r42
+ /* prepare wfxl set mask on MSBs */
+ slld $r35 = $r42, 32
+ ;;
+ set $lc = $r60
+ /* Replicate mask of ones on the 32 MSBs */
+ sbmm8 $r33 = $r33, REPLICATE_32_MASK
+ /* Combine the set and clear mask for wfxl */
+ insf $r35 = $r34, 31, 0
+ ;;
+ lo $r4r5r6r7 = PT_Q4[$r15]
+ set $le = $r61
+ /* Mask to drop identical bits in order to avoid useless
+ * privilege traps
+ */
+ andd $r35 = $r35, $r33
+ ;;
+ /* Restore $ilr */
+ ld $r44 = PT_ILR[$r15]
+ set $ls = $r62
+ /* Reenable MMU in $sps */
+ ord $r35 = $r35, SFR_SET_VAL_WFXL(SPS, MME, 1)
+ ;;
+ /* Restore user pointer */
+ lo $r12r13r14r15 = PT_Q12[$r15]
+ /* Prepare $r44 as a set mask for $ilr wfxl */
+ slld $r44 = $r44, 32
+ ;;
+ /**
+ * wfxl on $ilr to avoid privilege traps when restoring with set
+ * Note that we do that after disabling interrupt since we explicitly
+ * want to serve DAME itnerrupt to the user (ie not in kernel mode).
+ */
+ wfxl $ilr = $r44
+ ;;
+ wfxl $sps = $r35
+ ;;
+ set $spc = $r41
+ ;;
+ /* TODO: we might have to clear some registers to avoid leaking
+ * information to user space ! callee saved regs have been
+ * restored by called function but caller saved regs might
+ * have been used without being cleared */
+ rfe
+ ;;
+
+/* Slow paths handling */
+call_work_pending:
+ /*
+ * Work pending can potentially call a signal handler and then return
+ * via rt_sigreturn. Return path will be different (restore all regs)
+ * and hence all registers are needed to be saved.
+ */
+ save_callee
+ ;;
+ call_do_work_pending
+ ;;
+ /* Since we are returning to user, interrupts will be reenabled */
+ call_trace_hardirqs on
+ ;;
+ goto ret_to_user
+ ;;
+
+trace_syscall_enter:
+ /* Save $r38 (syscall handler) which was computed above */
+ sd PT_R38[$sp] = $r38
+ ;;
+ /* do_syscall_trace_enter expect pt_regs and syscall number
+ * as argument */
+ copyd $r0 = $sp
+ copyd $r1 = $r32
+ ;;
+ call do_syscall_trace_enter
+ ;;
+ /* Restore r38 (syscall handler) which might has been clobbered by
+ * do_syscall_trace_enter */
+ ld $r38 = PT_R38[$sp]
+ ;;
+ /* if the trace system requested to abort syscall, set $r38 to
+ * non implemented syscall */
+ cmoved.dnez $r0? $r38 = sys_ni_syscall
+ ;;
+ /* Restore registers since the do_syscall_trace_enter call might
+ * have clobbered them and we need them for the actual syscall
+ * call */
+ lo $r0r1r2r3 = PT_Q0[$sp]
+ ;;
+ lo $r4r5r6r7 = PT_Q4[$sp]
+ ;;
+ goto do_syscall
+ ;;
+trace_syscall_exit:
+ copyd $r0 = $sp
+ call do_syscall_trace_exit
+ ;;
+ /* ptrace_notify might re-enable interrupts, disable them to be sure
+ * it will not mess up context restoration path */
+ disable_interrupt $r36
+ ;;
+ get $r36 = $sr
+ ;;
+ ld $r38 = TASK_TI_FLAGS[$r36]
+ goto check_work_pending
+ ;;
+ENDPROC(kvx_syscall_handler)
+
+.text
+/**
+ * sys_rt_sigreturn: Special handler for sigreturn
+ * rt_sigreturn is called after invoking a user signal handler (see
+ * user_scall_rt_sigreturn). Since this signal handler can be invoked after
+ * interrupts have been handled, this means we must restore absolutely all user
+ * registers. Normally, this has been done by setup_sigcontext which saved all
+ * user registers on the user stack.
+ * In restore sigcontext, they have been restored onto entry stack (stack to be
+ * restored). However, the standard syscall path do not restore it completely
+ * since only callee-saved registers are restored for fork and al.
+ * Here, we will restore all registers which might have been cloberred.
+ */
+ENTRY(sys_rt_sigreturn)
+ /*
+ * If syscall tracing is enable (stored in $r37 during syscall
+ * fastpath), tail-call to trace_exit. If not, tail-call to
+ * ret_from_kernel.
+ */
+ cmoved.dnez $r37? $r38 = scall_trace_exit
+ cmoved.deqz $r37? $r38 = ret_from_kernel
+ ;;
+ set $ra = $r38
+ ;;
+ goto _sys_rt_sigreturn
+ ;;
+scall_trace_exit:
+ copyd $r0 = $sp
+ call do_syscall_trace_exit
+ ;;
+ goto ret_from_kernel
+ ;;
+ENDPROC(sys_rt_sigreturn)
+
+/**
+ * __sys_clone: slow path handler for clone
+ *
+ * Save callee registers (from $r18 to $31) since they are needed for
+ * child process when restoring it.
+ * Indeed, when forking we want it to have the same registers contents
+ * as its parent. These registers will then be restored in
+ * ret_from_fork.
+ * This slow path saves them out of the fast path to avoid bloating all syscall
+ * just for one special case.
+ */
+ENTRY(__sys_clone)
+ save_callee
+ ;;
+ /*
+ * Use goto since we want sys_clone to "ret" to the previous caller.
+ * This allows to simply go back in the normal syscall fastpath
+ */
+ goto sys_clone
+ ;;
+ENDPROC(__sys_clone)
+/***********************************************************************
+* Context switch
+***********************************************************************/
+
+#define SAVE_TCA_REG(__reg_num, __task_reg, __zero_reg1, __zero_reg2) \
+ xso (CTX_SWITCH_TCA_REGS + (__reg_num * TCA_REG_SIZE))[__task_reg] = \
+ $a##__reg_num ;\
+ movetq $a##__reg_num##.lo = __zero_reg1, __zero_reg2 ;\
+ movetq $a##__reg_num##.hi = __zero_reg1, __zero_reg2 ;\
+ ;;
+
+.macro save_tca_regs task_reg zero_reg1 zero_reg2
+ SAVE_TCA_REG(0, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(1, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(2, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(3, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(4, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(5, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(6, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(7, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(8, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(9, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(10, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(11, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(12, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(13, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(14, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(15, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(16, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(17, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(18, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(19, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(20, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(21, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(22, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(23, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(24, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(25, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(26, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(27, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(28, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(29, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(30, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(31, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(32, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(33, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(34, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(35, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(36, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(37, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(38, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(39, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(40, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(41, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(42, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(43, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(44, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(45, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(46, \task_reg, \zero_reg1, \zero_reg2)
+ SAVE_TCA_REG(47, \task_reg, \zero_reg1, \zero_reg2)
+.endm
+
+#define RESTORE_TCA_REG(__reg_num, __task_reg) \
+ xlo.u $a##__reg_num = (CTX_SWITCH_TCA_REGS + (__reg_num * TCA_REG_SIZE)) \
+ [__task_reg] ;\
+ ;;
+
+.macro restore_tca_regs task_reg
+ RESTORE_TCA_REG(0, \task_reg)
+ RESTORE_TCA_REG(1, \task_reg)
+ RESTORE_TCA_REG(2, \task_reg)
+ RESTORE_TCA_REG(3, \task_reg)
+ RESTORE_TCA_REG(4, \task_reg)
+ RESTORE_TCA_REG(5, \task_reg)
+ RESTORE_TCA_REG(6, \task_reg)
+ RESTORE_TCA_REG(7, \task_reg)
+ RESTORE_TCA_REG(8, \task_reg)
+ RESTORE_TCA_REG(9, \task_reg)
+ RESTORE_TCA_REG(10, \task_reg)
+ RESTORE_TCA_REG(11, \task_reg)
+ RESTORE_TCA_REG(12, \task_reg)
+ RESTORE_TCA_REG(13, \task_reg)
+ RESTORE_TCA_REG(14, \task_reg)
+ RESTORE_TCA_REG(15, \task_reg)
+ RESTORE_TCA_REG(16, \task_reg)
+ RESTORE_TCA_REG(17, \task_reg)
+ RESTORE_TCA_REG(18, \task_reg)
+ RESTORE_TCA_REG(19, \task_reg)
+ RESTORE_TCA_REG(20, \task_reg)
+ RESTORE_TCA_REG(21, \task_reg)
+ RESTORE_TCA_REG(22, \task_reg)
+ RESTORE_TCA_REG(23, \task_reg)
+ RESTORE_TCA_REG(24, \task_reg)
+ RESTORE_TCA_REG(25, \task_reg)
+ RESTORE_TCA_REG(26, \task_reg)
+ RESTORE_TCA_REG(27, \task_reg)
+ RESTORE_TCA_REG(28, \task_reg)
+ RESTORE_TCA_REG(29, \task_reg)
+ RESTORE_TCA_REG(30, \task_reg)
+ RESTORE_TCA_REG(31, \task_reg)
+ RESTORE_TCA_REG(32, \task_reg)
+ RESTORE_TCA_REG(33, \task_reg)
+ RESTORE_TCA_REG(34, \task_reg)
+ RESTORE_TCA_REG(35, \task_reg)
+ RESTORE_TCA_REG(36, \task_reg)
+ RESTORE_TCA_REG(37, \task_reg)
+ RESTORE_TCA_REG(38, \task_reg)
+ RESTORE_TCA_REG(39, \task_reg)
+ RESTORE_TCA_REG(40, \task_reg)
+ RESTORE_TCA_REG(41, \task_reg)
+ RESTORE_TCA_REG(42, \task_reg)
+ RESTORE_TCA_REG(43, \task_reg)
+ RESTORE_TCA_REG(44, \task_reg)
+ RESTORE_TCA_REG(45, \task_reg)
+ RESTORE_TCA_REG(46, \task_reg)
+ RESTORE_TCA_REG(47, \task_reg)
+.endm
+
+.text
+/*
+ * When entering in ret_from_kernel_thread, r20 and r21 where set by
+ * copy_thread and have been restored in switch_to.
+ * These registers contains the values needed to call a function
+ * specified by the switch_to caller (or where set by copy_thread).
+ */
+ENTRY(ret_from_kernel_thread)
+ call schedule_tail
+ ;;
+ /* Call fn(arg) */
+ copyd $r0 = $r21
+ ;;
+ icall $r20
+ ;;
+ goto ret_from_kernel
+ ;;
+ENDPROC(ret_from_kernel_thread)
+
+/**
+ * Return from fork.
+ * start_thread will set return stack and pc. Then copy_thread will
+ * take care of the copying logic.
+ * $r20 will then contains 0 if tracing disabled (set by copy_thread)
+ * The mechanism is almost the same as for ret_from_kernel_thread.
+ */
+ENTRY(ret_from_fork)
+ call schedule_tail
+ ;;
+ /* $r20 contains 0 if tracing disable */
+ cb.deqz $r20? ret_from_kernel
+ ;;
+ copyd $r0 = $sp
+ call do_syscall_trace_exit
+ ;;
+ret_from_kernel:
+ /*
+ * When returning from a fork, the child will take this path.
+ * Since we did not restore callee in return_from_exception, we
+ * must do it before.
+ */
+ lo $r28r29r30r31 = PT_Q28[$sp]
+ ;;
+ lo $r24r25r26r27 = PT_Q24[$sp]
+ ;;
+ lo $r20r21r22r23 = PT_Q20[$sp]
+ ;;
+ lq $r18r19 = PT_R18R19[$sp]
+ ;;
+#ifdef CONFIG_DEBUG_EXCEPTION_STACK
+ /**
+ * Debug code expect entry stack to be stored at current $sp.
+ * Make some room and store current $sp to avoid triggering false alarm.
+ */
+ addd $sp = $sp, -STACK_REG_SIZE
+ ;;
+ sd 0[$sp] = $sp
+#endif
+ ;;
+ goto return_from_exception
+ ;;
+ENDPROC(ret_from_fork)
+
+/*
+ * The callee-saved registers must be saved and restored.
+ * When entering:
+ * - r0 = previous task struct
+ * - r1 = next task struct
+ * Moreover, the parameters for function call (given by copy_thread)
+ * are stored in:
+ * - r20 = Func to call
+ * - r21 = Argument for function
+ */
+ENTRY(__switch_to)
+ sd CTX_SWITCH_FP[$r0] = $fp
+ ;;
+ /* Save previous task context */
+ so CTX_SWITCH_Q20[$r0] = $r20r21r22r23
+ ;;
+ so CTX_SWITCH_Q24[$r0] = $r24r25r26r27
+ get $r16 = $ra
+ ;;
+ so CTX_SWITCH_Q28[$r0] = $r28r29r30r31
+ copyd $r17 = $sp
+ get $r2 = $cs
+ ;;
+ so CTX_SWITCH_RA_SP_R18_R19[$r0] = $r16r17r18r19
+ /* Extract XMF bit which means coprocessor was used by user */
+ andd $r3 = $r2, KVX_SFR_CS_XMF_MASK
+ ;;
+#ifdef CONFIG_ENABLE_TCA
+ make $r4 = 0
+ make $r5 = 0
+ make $r6 = 1
+ cb.dnez $r3? save_tca_registers
+ /* Check if next task needs TCA registers to be restored */
+ ld $r7 = CTX_SWITCH_TCA_REGS_SAVED[$r1]
+ ;;
+check_restore_tca:
+ cb.dnez $r7? restore_tca_registers
+ ;;
+restore_fast_path:
+#endif
+ /* Restore next task context */
+ lo $r16r17r18r19 = CTX_SWITCH_RA_SP_R18_R19[$r1]
+ ;;
+ lo $r20r21r22r23 = CTX_SWITCH_Q20[$r1]
+ ;;
+ lo $r24r25r26r27 = CTX_SWITCH_Q24[$r1]
+ copyd $sp = $r17
+ set $ra = $r16
+ ;;
+ lo $r28r29r30r31 = CTX_SWITCH_Q28[$r1]
+ set $sr = $r1
+ ;;
+ ld $fp = CTX_SWITCH_FP[$r1]
+ ;;
+ ret
+ ;;
+#ifdef CONFIG_ENABLE_TCA
+save_tca_registers:
+ save_tca_regs $r0 $r4 $r5
+ ;;
+ /* Indicates that we saved the TCA context */
+ sb CTX_SWITCH_TCA_REGS_SAVED[$r0] = $r6
+ goto check_restore_tca
+ ;;
+restore_tca_registers:
+ restore_tca_regs $r1
+ ;;
+ /* Clear TCA registers saved hint */
+ sb CTX_SWITCH_TCA_REGS_SAVED[$r1] = $r5
+ goto restore_fast_path
+ ;;
+#endif
+ENDPROC(__switch_to)
+
+/***********************************************************************
+* Misc functions
+***********************************************************************/
+
+/**
+ * Avoid hardcoding trampoline for rt_sigreturn by using this code and
+ * copying it on user trampoline
+ */
+.pushsection .text
+.global user_scall_rt_sigreturn_end
+ENTRY(user_scall_rt_sigreturn)
+ make $r6 = __NR_rt_sigreturn
+ ;;
+ scall $r6
+ ;;
+user_scall_rt_sigreturn_end:
+ENDPROC(user_scall_rt_sigreturn)
+.popsection
diff --git a/arch/kvx/kernel/sys_kvx.c b/arch/kvx/kernel/sys_kvx.c
new file mode 100644
index 000000000000..b32a821d76c9
--- /dev/null
+++ b/arch/kvx/kernel/sys_kvx.c
@@ -0,0 +1,58 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#include <linux/syscalls.h>
+
+#include <asm/cacheflush.h>
+#include <asm/cachectl.h>
+
+SYSCALL_DEFINE6(mmap, unsigned long, addr, unsigned long, len,
+ unsigned long, prot, unsigned long, flags,
+ unsigned long, fd, off_t, off)
+{
+ /* offset must be a multiple of the page size */
+ if (unlikely(offset_in_page(off) != 0))
+ return -EINVAL;
+
+ /* Unlike mmap2 where the offset is in PAGE_SIZE-byte units, here it
+ * is in bytes. So we need to use PAGE_SHIFT.
+ */
+ return ksys_mmap_pgoff(addr, len, prot, flags, fd, off >> PAGE_SHIFT);
+}
+
+SYSCALL_DEFINE4(cachectl, unsigned long, addr, unsigned long, len,
+ unsigned long, cache, unsigned long, flags)
+{
+ bool wb = !!(flags & CACHECTL_FLAG_OP_WB);
+ bool inval = !!(flags & CACHECTL_FLAG_OP_INVAL);
+
+ if (len == 0)
+ return 0;
+
+ /* Check for overflow */
+ if (addr + len < addr)
+ return -EFAULT;
+
+ if (cache != CACHECTL_CACHE_DCACHE)
+ return -EINVAL;
+
+ if ((flags & CACHECTL_FLAG_OP_MASK) == 0)
+ return -EINVAL;
+
+ if (flags & CACHECTL_FLAG_ADDR_PHYS) {
+ if (!IS_ENABLED(CONFIG_CACHECTL_UNSAFE_PHYS_OPERATIONS))
+ return -EINVAL;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ dcache_wb_inval_phys_range(addr, len, wb, inval);
+ return 0;
+ }
+
+ return dcache_wb_inval_virt_range(addr, len, wb, inval);
+}
diff --git a/arch/kvx/kernel/syscall_table.c b/arch/kvx/kernel/syscall_table.c
new file mode 100644
index 000000000000..7859d25e728d
--- /dev/null
+++ b/arch/kvx/kernel/syscall_table.c
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/riscv/kernel/syscall_table.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/syscalls.h>
+
+#include <asm/syscalls.h>
+
+#undef __SYSCALL
+#define __SYSCALL(nr, call) [nr] = (call),
+
+void *sys_call_table[__NR_syscalls] = {
+ [0 ... __NR_syscalls - 1] = sys_ni_syscall,
+#include <asm/unistd.h>
+};
--
2.37.2
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 12/25] kvx: Add system call support
2023-01-03 16:43 ` Yann Sionneau
@ 2023-01-04 15:07 ` Arnd Bergmann
-1 siblings, 0 replies; 113+ messages in thread
From: Arnd Bergmann @ 2023-01-04 15:07 UTC (permalink / raw)
To: Yann Sionneau
Cc: Christian Brauner, Paul Walmsley, Palmer Dabbelt, Albert Ou,
linux-kernel, linux-riscv, Clement Leger, Guillaume Thouvenin,
Julian Vetter, Julien Villette, Marius Gligor
On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
> +
> +/*
> + * Ensure that the range [addr, addr+size) is within the process's
> + * address space
> + */
> +static inline int __access_ok(unsigned long addr, unsigned long size)
> +{
> + return size <= TASK_SIZE && addr <= TASK_SIZE - size;
> +}
This is the same as the generic version, so just use that instead.
> +#define HAVE_GET_KERNEL_NOFAULT
> +
> +#define __get_kernel_nofault(dst, src, type, err_label) \
> +do { \
> + long __kr_err; \
> + \
> + __get_user_nocheck(*((type *)(dst)), (type *)(src), __kr_err); \
> + if (unlikely(__kr_err)) \
> + goto err_label; \
> +} while (0)
> +
> +#define __put_kernel_nofault(dst, src, type, err_label) \
> +do { \
> + long __kr_err; \
> + \
> + __put_user_nocheck(*((type *)(src)), (type *)(dst), __kr_err); \
> + if (unlikely(__kr_err)) \
> + goto err_label; \
> +} while (0)
The wrapper around __get_user_nocheck/__put_user_nocheck
is not ideal here. Since I think you only support new
compilers anyway, you can use the asm-goto-with-output feature
to define the asm to branch to the label directly, and use
the same thing to build __get_user()/__put_user().
> +++ b/arch/kvx/include/uapi/asm/unistd.h
> @@ -0,0 +1,16 @@
> +
> +#define __ARCH_WANT_RENAMEAT
> +#define __ARCH_WANT_NEW_STAT
> +#define __ARCH_WANT_SET_GET_RLIMIT
> +#define __ARCH_WANT_SYS_CLONE3
It's good to have clone3() but the other three sets
of syscalls should no longer be defined for new architectures,
so please remove those.
Arnd
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 12/25] kvx: Add system call support
@ 2023-01-04 15:07 ` Arnd Bergmann
0 siblings, 0 replies; 113+ messages in thread
From: Arnd Bergmann @ 2023-01-04 15:07 UTC (permalink / raw)
To: Yann Sionneau
Cc: Christian Brauner, Paul Walmsley, Palmer Dabbelt, Albert Ou,
linux-kernel, linux-riscv, Clement Leger, Guillaume Thouvenin,
Julian Vetter, Julien Villette, Marius Gligor
On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
> +
> +/*
> + * Ensure that the range [addr, addr+size) is within the process's
> + * address space
> + */
> +static inline int __access_ok(unsigned long addr, unsigned long size)
> +{
> + return size <= TASK_SIZE && addr <= TASK_SIZE - size;
> +}
This is the same as the generic version, so just use that instead.
> +#define HAVE_GET_KERNEL_NOFAULT
> +
> +#define __get_kernel_nofault(dst, src, type, err_label) \
> +do { \
> + long __kr_err; \
> + \
> + __get_user_nocheck(*((type *)(dst)), (type *)(src), __kr_err); \
> + if (unlikely(__kr_err)) \
> + goto err_label; \
> +} while (0)
> +
> +#define __put_kernel_nofault(dst, src, type, err_label) \
> +do { \
> + long __kr_err; \
> + \
> + __put_user_nocheck(*((type *)(src)), (type *)(dst), __kr_err); \
> + if (unlikely(__kr_err)) \
> + goto err_label; \
> +} while (0)
The wrapper around __get_user_nocheck/__put_user_nocheck
is not ideal here. Since I think you only support new
compilers anyway, you can use the asm-goto-with-output feature
to define the asm to branch to the label directly, and use
the same thing to build __get_user()/__put_user().
> +++ b/arch/kvx/include/uapi/asm/unistd.h
> @@ -0,0 +1,16 @@
> +
> +#define __ARCH_WANT_RENAMEAT
> +#define __ARCH_WANT_NEW_STAT
> +#define __ARCH_WANT_SET_GET_RLIMIT
> +#define __ARCH_WANT_SYS_CLONE3
It's good to have clone3() but the other three sets
of syscalls should no longer be defined for new architectures,
so please remove those.
Arnd
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 12/25] kvx: Add system call support
2023-01-03 16:43 ` Yann Sionneau
@ 2023-01-09 20:55 ` Thomas Gleixner
-1 siblings, 0 replies; 113+ messages in thread
From: Thomas Gleixner @ 2023-01-09 20:55 UTC (permalink / raw)
To: Yann Sionneau
Cc: Yann Sionneau, Christian Brauner, Paul Walmsley, Palmer Dabbelt,
Albert Ou, linux-kernel, linux-riscv, Clement Leger,
Guillaume Thouvenin, Julian Vetter, Julien Villette,
Marius Gligor
On Tue, Jan 03 2023 at 17:43, Yann Sionneau wrote:
> +/***********************************************************************
> +* Common exception return path
> +***********************************************************************/
> +/**
> + * Restore registers after exception
> + * When entering this macro, $sp must be located right before regs
> + * storage.
> + */
> +EXCEPTION_ENTRY(return_from_exception)
> +#ifdef CONFIG_DEBUG_EXCEPTION_STACK
> + ld $r1 = 0[$sp]
> + ;;
> + sbfd $r1 = $r1, $sp
> + ;;
> + cb.deqz $r1, _check_ok
> + ;;
> + make $r2 = panic
> + make $r0 = stack_error_panic_str_label
> + ;;
> + icall $r2
> + ;;
> +_check_ok:
> + addd $sp = $sp, STACK_REG_SIZE
> + ;;
> +#endif
> + get $r11 = $sr
> + /* Load sps value from saved registers */
> + ld $r6 = PT_SPS[$sp]
> + ;;
> + /* Disable interrupt to check task flags atomically */
> + disable_interrupt $r60
> + ;;
> + /* Check PL bit of sps, if set, then it means we are returning
> + * to a lower privilege level (ie to user), if so, we need to
> + * check work pending. If coming from kernel, directly go to
> + * register restoration */
> + cb.even $r6? _restore_regs
> + ld $r1 = TASK_TI_FLAGS[$r11]
> + ;;
> + /* Do we have work pending ? */
> + andd $r5 = $r1, _TIF_WORK_MASK
Please use the generic entry code to handle pending work, tracing etc.
All over the place....
Thanks,
tglx
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 12/25] kvx: Add system call support
@ 2023-01-09 20:55 ` Thomas Gleixner
0 siblings, 0 replies; 113+ messages in thread
From: Thomas Gleixner @ 2023-01-09 20:55 UTC (permalink / raw)
To: Yann Sionneau
Cc: Yann Sionneau, Christian Brauner, Paul Walmsley, Palmer Dabbelt,
Albert Ou, linux-kernel, linux-riscv, Clement Leger,
Guillaume Thouvenin, Julian Vetter, Julien Villette,
Marius Gligor
On Tue, Jan 03 2023 at 17:43, Yann Sionneau wrote:
> +/***********************************************************************
> +* Common exception return path
> +***********************************************************************/
> +/**
> + * Restore registers after exception
> + * When entering this macro, $sp must be located right before regs
> + * storage.
> + */
> +EXCEPTION_ENTRY(return_from_exception)
> +#ifdef CONFIG_DEBUG_EXCEPTION_STACK
> + ld $r1 = 0[$sp]
> + ;;
> + sbfd $r1 = $r1, $sp
> + ;;
> + cb.deqz $r1, _check_ok
> + ;;
> + make $r2 = panic
> + make $r0 = stack_error_panic_str_label
> + ;;
> + icall $r2
> + ;;
> +_check_ok:
> + addd $sp = $sp, STACK_REG_SIZE
> + ;;
> +#endif
> + get $r11 = $sr
> + /* Load sps value from saved registers */
> + ld $r6 = PT_SPS[$sp]
> + ;;
> + /* Disable interrupt to check task flags atomically */
> + disable_interrupt $r60
> + ;;
> + /* Check PL bit of sps, if set, then it means we are returning
> + * to a lower privilege level (ie to user), if so, we need to
> + * check work pending. If coming from kernel, directly go to
> + * register restoration */
> + cb.even $r6? _restore_regs
> + ld $r1 = TASK_TI_FLAGS[$r11]
> + ;;
> + /* Do we have work pending ? */
> + andd $r5 = $r1, _TIF_WORK_MASK
Please use the generic entry code to handle pending work, tracing etc.
All over the place....
Thanks,
tglx
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 13/25] kvx: Add signal handling support
2023-01-03 16:43 ` Yann Sionneau
` (13 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-04 11:28 ` Mark Rutland
-1 siblings, 1 reply; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, Clement Leger, Jules Maselbas,
Julian Vetter
Add sigcontext definition and signal handling support for
kvx.
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/uapi/asm/sigcontext.h | 16 ++
arch/kvx/kernel/signal.c | 266 +++++++++++++++++++++++++
arch/kvx/kernel/vdso.c | 87 ++++++++
3 files changed, 369 insertions(+)
create mode 100644 arch/kvx/include/uapi/asm/sigcontext.h
create mode 100644 arch/kvx/kernel/signal.c
create mode 100644 arch/kvx/kernel/vdso.c
diff --git a/arch/kvx/include/uapi/asm/sigcontext.h b/arch/kvx/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..97ab4f78152a
--- /dev/null
+++ b/arch/kvx/include/uapi/asm/sigcontext.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _UAPI_ASM_KVX_SIGCONTEXT_H
+#define _UAPI_ASM_KVX_SIGCONTEXT_H
+
+#include <asm/ptrace.h>
+
+struct sigcontext {
+ struct user_pt_regs sc_regs;
+};
+
+#endif /* _UAPI_ASM_KVX_SIGCONTEXT_H */
diff --git a/arch/kvx/kernel/signal.c b/arch/kvx/kernel/signal.c
new file mode 100644
index 000000000000..e52bbb30c336
--- /dev/null
+++ b/arch/kvx/kernel/signal.c
@@ -0,0 +1,266 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/bug.h>
+#include <linux/syscalls.h>
+
+#include <asm/ucontext.h>
+#include <asm/processor.h>
+#include <asm/cacheflush.h>
+#include <linux/resume_user_mode.h>
+
+struct rt_sigframe {
+ struct siginfo info;
+ struct ucontext uc;
+};
+
+int __init setup_syscall_sigreturn_page(void *sigpage_addr)
+{
+ unsigned int frame_size = (uintptr_t) &user_scall_rt_sigreturn_end -
+ (uintptr_t) &user_scall_rt_sigreturn;
+
+ /* Copy the sigreturn scall implementation */
+ memcpy(sigpage_addr, &user_scall_rt_sigreturn, frame_size);
+
+ flush_icache_range((unsigned long) sigpage_addr,
+ (unsigned long) sigpage_addr + frame_size);
+
+ return 0;
+}
+
+static long restore_sigcontext(struct pt_regs *regs,
+ struct sigcontext __user *sc)
+{
+ long err;
+
+ /* sc_regs is structured the same as the start of pt_regs */
+ err = __copy_from_user(regs, &sc->sc_regs, sizeof(sc->sc_regs));
+
+ return err;
+}
+
+long _sys_rt_sigreturn(void)
+{
+ struct pt_regs *regs = current_pt_regs();
+ struct rt_sigframe __user *frame;
+ struct task_struct *task;
+ sigset_t set;
+
+ current->restart_block.fn = do_no_restart_syscall;
+
+ frame = (struct rt_sigframe __user *) user_stack_pointer(regs);
+
+ /*
+ * Stack is not aligned but should be !
+ * User probably did some malicious things.
+ */
+ if (user_stack_pointer(regs) & STACK_ALIGN_MASK)
+ goto badframe;
+
+ if (!access_ok(frame, sizeof(*frame)))
+ goto badframe;
+
+ /* Restore sigmask */
+ if (__copy_from_user(&set, &frame->uc.uc_sigmask, sizeof(set)))
+ goto badframe;
+
+ set_current_blocked(&set);
+
+ if (restore_sigcontext(regs, &frame->uc.uc_mcontext))
+ goto badframe;
+
+ if (restore_altstack(&frame->uc.uc_stack))
+ goto badframe;
+
+ return regs->r0;
+
+badframe:
+ task = current;
+ if (show_unhandled_signals) {
+ pr_info_ratelimited(
+ "%s[%d]: bad frame in %s: frame=%p pc=%p sp=%p\n",
+ task->comm, task_pid_nr(task), __func__,
+ frame, (void *) instruction_pointer(regs),
+ (void *) user_stack_pointer(regs));
+ }
+ force_sig(SIGSEGV);
+ return 0;
+}
+
+
+static long setup_sigcontext(struct rt_sigframe __user *frame,
+ struct pt_regs *regs)
+{
+ struct sigcontext __user *sc = &frame->uc.uc_mcontext;
+ long err;
+
+ /* sc_regs is structured the same as the start of pt_regs */
+ err = __copy_to_user(&sc->sc_regs, regs, sizeof(sc->sc_regs));
+
+ return err;
+}
+
+static inline void __user *get_sigframe(struct ksignal *ksig,
+ struct pt_regs *regs, size_t framesize)
+{
+ unsigned long sp;
+ /* Default to using normal stack */
+ sp = regs->sp;
+
+ /*
+ * If we are on the alternate signal stack and would overflow it, don't.
+ * Return an always-bogus address instead so we will die with SIGSEGV.
+ */
+ if (on_sig_stack(sp) && !likely(on_sig_stack(sp - framesize)))
+ return (void __user __force *)(-1UL);
+
+ /* This is the X/Open sanctioned signal stack switching. */
+ sp = sigsp(sp, ksig) - framesize;
+
+ /* Align the stack frame on 16bytes */
+ sp &= ~STACK_ALIGN_MASK;
+
+ return (void __user *)sp;
+}
+
+/* TODO: Use VDSO when ready ! */
+static int setup_rt_frame(struct ksignal *ksig, sigset_t *set,
+ struct pt_regs *regs)
+{
+ unsigned long sigpage = current->mm->context.sigpage;
+ struct rt_sigframe __user *frame;
+ long err = 0;
+
+ frame = get_sigframe(ksig, regs, sizeof(*frame));
+ if (!access_ok(frame, sizeof(*frame)))
+ return -EFAULT;
+
+ err |= copy_siginfo_to_user(&frame->info, &ksig->info);
+
+ /* Create the ucontext. */
+ err |= __put_user(0, &frame->uc.uc_flags);
+ err |= __put_user(NULL, &frame->uc.uc_link);
+ err |= __save_altstack(&frame->uc.uc_stack, user_stack_pointer(regs));
+ err |= setup_sigcontext(frame, regs);
+ err |= __copy_to_user(&frame->uc.uc_sigmask, set, sizeof(*set));
+ if (err)
+ return -EFAULT;
+
+ /*
+ * When returning from the handler, we want to jump to the
+ * sigpage which will execute the sigreturn scall.
+ */
+ regs->ra = sigpage;
+ /* Return to signal handler */
+ regs->spc = (unsigned long)ksig->ka.sa.sa_handler;
+ regs->sp = (unsigned long) frame;
+
+ /* Parameters for signal handler */
+ regs->r0 = ksig->sig; /* r0: signal number */
+ regs->r1 = (unsigned long)(&frame->info); /* r1: siginfo pointer */
+ regs->r2 = (unsigned long)(&frame->uc); /* r2: ucontext pointer */
+
+ return 0;
+}
+
+static void handle_signal(struct ksignal *ksig, struct pt_regs *regs)
+{
+ sigset_t *oldset = sigmask_to_save();
+ int ret;
+
+ /* Are we from a system call? */
+ if (in_syscall(regs)) {
+ /* If so, check system call restarting.. */
+ switch (regs->r0) {
+ case -ERESTART_RESTARTBLOCK:
+ case -ERESTARTNOHAND:
+ regs->r0 = -EINTR;
+ break;
+ case -ERESTARTSYS:
+ if (!(ksig->ka.sa.sa_flags & SA_RESTART)) {
+ regs->r0 = -EINTR;
+ break;
+ }
+ fallthrough;
+ case -ERESTARTNOINTR:
+ regs->r0 = regs->orig_r0;
+ regs->spc -= 0x4;
+ break;
+ }
+ }
+
+ ret = setup_rt_frame(ksig, oldset, regs);
+
+ signal_setup_done(ret, ksig, 0);
+}
+
+asmlinkage void do_signal(struct pt_regs *regs)
+{
+ struct ksignal ksig;
+
+ if (get_signal(&ksig)) {
+ handle_signal(&ksig, regs);
+ return;
+ }
+
+ /* Are we from a system call? */
+ if (in_syscall(regs)) {
+ /*
+ * If we are here, this means there is no handler
+ * present and we must restart the syscall.
+ */
+ switch (regs->r0) {
+ case -ERESTART_RESTARTBLOCK:
+ /* Modify the syscall number in order to restart it */
+ regs->r6 = __NR_restart_syscall;
+ fallthrough;
+ case -ERESTARTNOHAND:
+ case -ERESTARTSYS:
+ case -ERESTARTNOINTR:
+ /* We are restarting the syscall */
+ regs->r0 = regs->orig_r0;
+ /*
+ * scall instruction isn't bundled with anything else,
+ * so we can just revert the spc to restart the syscall.
+ */
+ regs->spc -= 0x4;
+ break;
+ }
+ }
+
+ /*
+ * If there's no signal to deliver, we just put the saved sigmask
+ * back.
+ */
+ restore_saved_sigmask();
+}
+
+
+asmlinkage void do_work_pending(struct pt_regs *regs,
+ unsigned long thread_flags)
+{
+ /* We are called with IRQs disabled */
+ trace_hardirqs_off();
+
+ do {
+ if (thread_flags & _TIF_NEED_RESCHED) {
+ schedule();
+ } else {
+ local_irq_enable();
+ if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL))
+ do_signal(regs);
+
+ if (thread_flags & _TIF_NOTIFY_RESUME) {
+ clear_thread_flag(TIF_NOTIFY_RESUME);
+ resume_user_mode_work(regs);
+ }
+ }
+ /* Guarantee task flag atomic read */
+ local_irq_disable();
+ thread_flags = READ_ONCE(current_thread_info()->flags);
+ } while (thread_flags & _TIF_WORK_MASK);
+}
+
diff --git a/arch/kvx/kernel/vdso.c b/arch/kvx/kernel/vdso.c
new file mode 100644
index 000000000000..1515de15eb31
--- /dev/null
+++ b/arch/kvx/kernel/vdso.c
@@ -0,0 +1,87 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/mm.h>
+#include <linux/vmalloc.h>
+#include <linux/binfmts.h>
+
+#include <asm/syscall.h>
+
+static struct page *signal_page;
+
+static int __init init_sigreturn(void)
+{
+ struct page *sigpage;
+ void *mapped_sigpage;
+ int err;
+
+ sigpage = alloc_page(GFP_KERNEL);
+ if (!sigpage)
+ panic("Cannot allocate sigreturn page");
+
+ mapped_sigpage = vmap(&sigpage, 1, 0, PAGE_KERNEL);
+ if (!mapped_sigpage)
+ panic("Cannot map sigreturn page");
+
+ clear_page(mapped_sigpage);
+
+ err = setup_syscall_sigreturn_page(mapped_sigpage);
+ if (err)
+ panic("Cannot set signal return syscall, err: %x.", err);
+
+ vunmap(mapped_sigpage);
+
+ signal_page = sigpage;
+
+ return 0;
+}
+arch_initcall(init_sigreturn);
+
+static int sigpage_mremap(const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ current->mm->context.sigpage = new_vma->vm_start;
+ return 0;
+}
+
+static const struct vm_special_mapping sigpage_mapping = {
+ .name = "[sigpage]",
+ .pages = &signal_page,
+ .mremap = sigpage_mremap,
+};
+
+int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ int ret = 0;
+ unsigned long addr;
+ struct mm_struct *mm = current->mm;
+ struct vm_area_struct *vma;
+
+ mmap_write_lock(mm);
+
+ addr = get_unmapped_area(NULL, STACK_TOP, PAGE_SIZE, 0, 0);
+ if (IS_ERR_VALUE(addr)) {
+ ret = addr;
+ goto up_fail;
+ }
+
+ vma = _install_special_mapping(
+ mm,
+ addr,
+ PAGE_SIZE,
+ VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
+ &sigpage_mapping);
+ if (IS_ERR(vma)) {
+ ret = PTR_ERR(vma);
+ goto up_fail;
+ }
+
+ mm->context.sigpage = addr;
+
+up_fail:
+ mmap_write_unlock(mm);
+ return ret;
+}
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 13/25] kvx: Add signal handling support
2023-01-03 16:43 ` [RFC PATCH 13/25] kvx: Add signal handling support Yann Sionneau
@ 2023-01-04 11:28 ` Mark Rutland
0 siblings, 0 replies; 113+ messages in thread
From: Mark Rutland @ 2023-01-04 11:28 UTC (permalink / raw)
To: Yann Sionneau; +Cc: linux-kernel, Clement Leger, Jules Maselbas, Julian Vetter
On Tue, Jan 03, 2023 at 05:43:47PM +0100, Yann Sionneau wrote:
> Add sigcontext definition and signal handling support for
> kvx.
[...]
> +asmlinkage void do_work_pending(struct pt_regs *regs,
> + unsigned long thread_flags)
> +{
> + /* We are called with IRQs disabled */
> + trace_hardirqs_off();
> +
> + do {
> + if (thread_flags & _TIF_NEED_RESCHED) {
> + schedule();
> + } else {
> + local_irq_enable();
> + if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL))
> + do_signal(regs);
> +
> + if (thread_flags & _TIF_NOTIFY_RESUME) {
> + clear_thread_flag(TIF_NOTIFY_RESUME);
> + resume_user_mode_work(regs);
> + }
> + }
> + /* Guarantee task flag atomic read */
> + local_irq_disable();
> + thread_flags = READ_ONCE(current_thread_info()->flags);
The comment here is a bit misleading; disabling IRQs doesn't affect the
atomicity of the read (and flags can be set by remote threads), so I'd
recommend removing it.
Please use read_thread_flags() to read the flags.
> + } while (thread_flags & _TIF_WORK_MASK);
> +}
Thanks,
Mark.
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 14/25] kvx: Add ELF relocations and module support
2023-01-03 16:43 ` Yann Sionneau
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Eric Biederman, Kees Cook, linux-mm, linux-kernel,
Clement Leger, Julian Vetter, Marius Gligor
Add ELF-related definition and module relocation code for basic
kvx support.
CC: Eric Biederman <ebiederm@xmission.com>
CC: Kees Cook <keescook@chromium.org>
CC: linux-mm@kvack.org
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/elf.h | 155 ++++++++++++++++++++++++++++++++
arch/kvx/include/asm/vermagic.h | 12 +++
arch/kvx/kernel/module.c | 148 ++++++++++++++++++++++++++++++
3 files changed, 315 insertions(+)
create mode 100644 arch/kvx/include/asm/elf.h
create mode 100644 arch/kvx/include/asm/vermagic.h
create mode 100644 arch/kvx/kernel/module.c
diff --git a/arch/kvx/include/asm/elf.h b/arch/kvx/include/asm/elf.h
new file mode 100644
index 000000000000..38978d48221e
--- /dev/null
+++ b/arch/kvx/include/asm/elf.h
@@ -0,0 +1,155 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Yann Sionneau
+ * Clement Leger
+ * Marius Gligor
+ * Guillaume Thouvenin
+ */
+
+#ifndef _ASM_KVX_ELF_H
+#define _ASM_KVX_ELF_H
+
+#include <linux/types.h>
+
+#include <asm/ptrace.h>
+
+/*
+ * These are used to set parameters in the core dumps.
+ */
+#define ELF_CLASS ELFCLASS64
+#define ELF_DATA ELFDATA2LSB
+#define ELF_ARCH EM_KVX
+
+typedef uint64_t elf_greg_t;
+typedef uint64_t elf_fpregset_t;
+
+#define ELF_NGREG (sizeof(struct user_pt_regs) / sizeof(elf_greg_t))
+typedef elf_greg_t elf_gregset_t[ELF_NGREG];
+
+/* Copy user_pt_regs from pt_regs into the elf_gregset_t */
+#define ELF_CORE_COPY_REGS(dest, regs) \
+ *(struct user_pt_regs *)&(dest) = (regs)->user_regs;
+
+/*
+ * This is used to ensure we don't load something for the wrong architecture.
+ */
+#define elf_check_arch(x) ((x)->e_machine == EM_KVX)
+
+#define ELF_CORE_EFLAGS 0x1308
+
+#define ELF_EXEC_PAGESIZE (PAGE_SIZE)
+
+/*
+ * This is the location that an ET_DYN program is loaded if exec'ed. Typical
+ * use of this is to invoke "./ld.so someprog" to test out a new version of
+ * the loader. We need to make sure that it is out of the way of the program
+ * that it will "exec", and that there is sufficient room for the brk.
+ */
+#define ELF_ET_DYN_BASE ((TASK_SIZE / 3) * 2)
+
+/*
+ * This yields a mask that user programs can use to figure out what
+ * instruction set this CPU supports. This could be done in user space,
+ * but it's not easy, and we've already done it here.
+ */
+#define ELF_HWCAP (elf_hwcap)
+extern unsigned long elf_hwcap;
+
+/*
+ * This yields a string that ld.so will use to load implementation
+ * specific libraries for optimization. This is more specific in
+ * intent than poking at uname or /proc/cpuinfo.
+ */
+#define ELF_PLATFORM (NULL)
+
+#define ARCH_HAS_SETUP_ADDITIONAL_PAGES 1
+struct linux_binprm;
+extern int arch_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
+
+/* KVX relocs */
+#define R_KVX_NONE 0
+#define R_KVX_16 1
+#define R_KVX_32 2
+#define R_KVX_64 3
+#define R_KVX_S16_PCREL 4
+#define R_KVX_PCREL17 5
+#define R_KVX_PCREL27 6
+#define R_KVX_32_PCREL 7
+#define R_KVX_S37_PCREL_LO10 8
+#define R_KVX_S37_PCREL_UP27 9
+#define R_KVX_S43_PCREL_LO10 10
+#define R_KVX_S43_PCREL_UP27 11
+#define R_KVX_S43_PCREL_EX6 12
+#define R_KVX_S64_PCREL_LO10 13
+#define R_KVX_S64_PCREL_UP27 14
+#define R_KVX_S64_PCREL_EX27 15
+#define R_KVX_64_PCREL 16
+#define R_KVX_S16 17
+#define R_KVX_S32_LO5 18
+#define R_KVX_S32_UP27 19
+#define R_KVX_S37_LO10 20
+#define R_KVX_S37_UP27 21
+#define R_KVX_S37_GOTOFF_LO10 22
+#define R_KVX_S37_GOTOFF_UP27 23
+#define R_KVX_S43_GOTOFF_LO10 24
+#define R_KVX_S43_GOTOFF_UP27 25
+#define R_KVX_S43_GOTOFF_EX6 26
+#define R_KVX_32_GOTOFF 27
+#define R_KVX_64_GOTOFF 28
+#define R_KVX_32_GOT 29
+#define R_KVX_S37_GOT_LO10 30
+#define R_KVX_S37_GOT_UP27 31
+#define R_KVX_S43_GOT_LO10 32
+#define R_KVX_S43_GOT_UP27 33
+#define R_KVX_S43_GOT_EX6 34
+#define R_KVX_64_GOT 35
+#define R_KVX_GLOB_DAT 36
+#define R_KVX_COPY 37
+#define R_KVX_JMP_SLOT 38
+#define R_KVX_RELATIVE 39
+#define R_KVX_S43_LO10 40
+#define R_KVX_S43_UP27 41
+#define R_KVX_S43_EX6 42
+#define R_KVX_S64_LO10 43
+#define R_KVX_S64_UP27 44
+#define R_KVX_S64_EX27 45
+#define R_KVX_S37_GOTADDR_LO10 46
+#define R_KVX_S37_GOTADDR_UP27 47
+#define R_KVX_S43_GOTADDR_LO10 48
+#define R_KVX_S43_GOTADDR_UP27 49
+#define R_KVX_S43_GOTADDR_EX6 50
+#define R_KVX_S64_GOTADDR_LO10 51
+#define R_KVX_S64_GOTADDR_UP27 52
+#define R_KVX_S64_GOTADDR_EX27 53
+#define R_KVX_64_DTPMOD 54
+#define R_KVX_64_DTPOFF 55
+#define R_KVX_S37_TLS_DTPOFF_LO10 56
+#define R_KVX_S37_TLS_DTPOFF_UP27 57
+#define R_KVX_S43_TLS_DTPOFF_LO10 58
+#define R_KVX_S43_TLS_DTPOFF_UP27 59
+#define R_KVX_S43_TLS_DTPOFF_EX6 60
+#define R_KVX_S37_TLS_GD_LO10 61
+#define R_KVX_S37_TLS_GD_UP27 62
+#define R_KVX_S43_TLS_GD_LO10 63
+#define R_KVX_S43_TLS_GD_UP27 64
+#define R_KVX_S43_TLS_GD_EX6 65
+#define R_KVX_S37_TLS_LD_LO10 66
+#define R_KVX_S37_TLS_LD_UP27 67
+#define R_KVX_S43_TLS_LD_LO10 68
+#define R_KVX_S43_TLS_LD_UP27 69
+#define R_KVX_S43_TLS_LD_EX6 70
+#define R_KVX_64_TPOFF 71
+#define R_KVX_S37_TLS_IE_LO10 72
+#define R_KVX_S37_TLS_IE_UP27 73
+#define R_KVX_S43_TLS_IE_LO10 74
+#define R_KVX_S43_TLS_IE_UP27 75
+#define R_KVX_S43_TLS_IE_EX6 76
+#define R_KVX_S37_TLS_LE_LO10 77
+#define R_KVX_S37_TLS_LE_UP27 78
+#define R_KVX_S43_TLS_LE_LO10 79
+#define R_KVX_S43_TLS_LE_UP27 80
+#define R_KVX_S43_TLS_LE_EX6 81
+
+#endif /* _ASM_KVX_ELF_H */
diff --git a/arch/kvx/include/asm/vermagic.h b/arch/kvx/include/asm/vermagic.h
new file mode 100644
index 000000000000..fef9a33065df
--- /dev/null
+++ b/arch/kvx/include/asm/vermagic.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_VERMAGIC_H
+#define _ASM_KVX_VERMAGIC_H
+
+#define MODULE_ARCH_VERMAGIC "kvx"
+
+#endif /* _ASM_KVX_VERMAGIC_H */
diff --git a/arch/kvx/kernel/module.c b/arch/kvx/kernel/module.c
new file mode 100644
index 000000000000..b9383792ae45
--- /dev/null
+++ b/arch/kvx/kernel/module.c
@@ -0,0 +1,148 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Yann Sionneau
+ * Clement Leger
+ */
+
+#include <linux/elf.h>
+#include <linux/moduleloader.h>
+#include <linux/overflow.h>
+
+
+static int apply_rela_bits(Elf64_Addr loc, Elf64_Addr val,
+ int sign, int immsize, int bits, int rshift,
+ int lshift, unsigned int relocnum,
+ struct module *me)
+{
+ unsigned long long umax;
+ long long min, max;
+ unsigned long long mask = GENMASK_ULL(bits + lshift - 1, lshift);
+
+ if (sign) {
+ min = -(1ULL << (immsize - 1));
+ max = (1ULL << (immsize - 1)) - 1;
+ if ((long long) val < min || (long long) val > max)
+ goto too_big;
+ val = (Elf64_Addr)(((long) val) >> rshift);
+ } else {
+ if (immsize < 64)
+ umax = (1ULL << immsize) - 1;
+ else
+ umax = -1ULL;
+ if ((unsigned long long) val > umax)
+ goto too_big;
+ val >>= rshift;
+ }
+
+ val <<= lshift;
+ val &= mask;
+ if (bits <= 32)
+ *(u32 *) loc = (*(u32 *)loc & ~mask) | val;
+ else
+ *(u64 *) loc = (*(u64 *)loc & ~mask) | val;
+
+ return 0;
+too_big:
+ pr_err("%s: value %llx does not fit in %d bits for reloc %u",
+ me->name, val, bits, relocnum);
+ return -ENOEXEC;
+}
+
+int apply_relocate_add(Elf64_Shdr *sechdrs,
+ const char *strtab,
+ unsigned int symindex,
+ unsigned int relsec,
+ struct module *me)
+{
+ unsigned int i;
+ Elf64_Addr loc;
+ u64 val;
+ s64 sval;
+ Elf64_Sym *sym;
+ Elf64_Rela *rel = (void *)sechdrs[relsec].sh_addr;
+ int ret = 0;
+
+ pr_debug("Applying relocate section %u to %u\n",
+ relsec, sechdrs[relsec].sh_info);
+
+ for (i = 0; i < sechdrs[relsec].sh_size / sizeof(*rel); i++) {
+ /* This is where to make the change */
+ loc = (Elf64_Addr)sechdrs[sechdrs[relsec].sh_info].sh_addr
+ + rel[i].r_offset;
+ /* This is the symbol it is referring to. Note that all
+ * undefined symbols have been resolved.
+ */
+ sym = (Elf64_Sym *)sechdrs[symindex].sh_addr
+ + ELF64_R_SYM(rel[i].r_info);
+
+ pr_debug("type %d st_value %llx r_addend %llx loc %llx offset %llx\n",
+ (int)ELF64_R_TYPE(rel[i].r_info),
+ sym->st_value, rel[i].r_addend, (uint64_t)loc,
+ rel[i].r_offset);
+
+ val = sym->st_value + rel[i].r_addend;
+ switch (ELF64_R_TYPE(rel[i].r_info)) {
+ case R_KVX_NONE:
+ break;
+ case R_KVX_32:
+ ret = apply_rela_bits(loc, val, 0, 32, 32, 0, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_64:
+ ret = apply_rela_bits(loc, val, 0, 64, 64, 0, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S43_LO10:
+ ret = apply_rela_bits(loc, val, 1, 43, 10, 0, 6,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S64_LO10:
+ ret = apply_rela_bits(loc, val, 1, 64, 10, 0, 6,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S43_UP27:
+ ret = apply_rela_bits(loc, val, 1, 43, 27, 10, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S64_UP27:
+ ret = apply_rela_bits(loc, val, 1, 64, 27, 10, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S43_EX6:
+ ret = apply_rela_bits(loc, val, 1, 43, 6, 37, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S64_EX27:
+ ret = apply_rela_bits(loc, val, 1, 64, 27, 37, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_PCREL27:
+ if (__builtin_sub_overflow(val, loc, &sval)) {
+ pr_err("%s: Signed integer overflow, this should not happen\n",
+ me->name);
+ return -ENOEXEC;
+ }
+ sval >>= 2;
+ ret = apply_rela_bits(loc, (Elf64_Addr)sval, 1, 27, 27,
+ 0, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ default:
+ pr_err("%s: Unknown relocation: %llu\n",
+ me->name, ELF64_R_TYPE(rel[i].r_info));
+ ret = -ENOEXEC;
+ }
+ }
+ return ret;
+}
+
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 14/25] kvx: Add ELF relocations and module support
@ 2023-01-03 16:43 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Eric Biederman, Kees Cook, linux-mm, linux-kernel,
Clement Leger, Julian Vetter, Marius Gligor
Add ELF-related definition and module relocation code for basic
kvx support.
CC: Eric Biederman <ebiederm@xmission.com>
CC: Kees Cook <keescook@chromium.org>
CC: linux-mm@kvack.org
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/elf.h | 155 ++++++++++++++++++++++++++++++++
arch/kvx/include/asm/vermagic.h | 12 +++
arch/kvx/kernel/module.c | 148 ++++++++++++++++++++++++++++++
3 files changed, 315 insertions(+)
create mode 100644 arch/kvx/include/asm/elf.h
create mode 100644 arch/kvx/include/asm/vermagic.h
create mode 100644 arch/kvx/kernel/module.c
diff --git a/arch/kvx/include/asm/elf.h b/arch/kvx/include/asm/elf.h
new file mode 100644
index 000000000000..38978d48221e
--- /dev/null
+++ b/arch/kvx/include/asm/elf.h
@@ -0,0 +1,155 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Yann Sionneau
+ * Clement Leger
+ * Marius Gligor
+ * Guillaume Thouvenin
+ */
+
+#ifndef _ASM_KVX_ELF_H
+#define _ASM_KVX_ELF_H
+
+#include <linux/types.h>
+
+#include <asm/ptrace.h>
+
+/*
+ * These are used to set parameters in the core dumps.
+ */
+#define ELF_CLASS ELFCLASS64
+#define ELF_DATA ELFDATA2LSB
+#define ELF_ARCH EM_KVX
+
+typedef uint64_t elf_greg_t;
+typedef uint64_t elf_fpregset_t;
+
+#define ELF_NGREG (sizeof(struct user_pt_regs) / sizeof(elf_greg_t))
+typedef elf_greg_t elf_gregset_t[ELF_NGREG];
+
+/* Copy user_pt_regs from pt_regs into the elf_gregset_t */
+#define ELF_CORE_COPY_REGS(dest, regs) \
+ *(struct user_pt_regs *)&(dest) = (regs)->user_regs;
+
+/*
+ * This is used to ensure we don't load something for the wrong architecture.
+ */
+#define elf_check_arch(x) ((x)->e_machine == EM_KVX)
+
+#define ELF_CORE_EFLAGS 0x1308
+
+#define ELF_EXEC_PAGESIZE (PAGE_SIZE)
+
+/*
+ * This is the location that an ET_DYN program is loaded if exec'ed. Typical
+ * use of this is to invoke "./ld.so someprog" to test out a new version of
+ * the loader. We need to make sure that it is out of the way of the program
+ * that it will "exec", and that there is sufficient room for the brk.
+ */
+#define ELF_ET_DYN_BASE ((TASK_SIZE / 3) * 2)
+
+/*
+ * This yields a mask that user programs can use to figure out what
+ * instruction set this CPU supports. This could be done in user space,
+ * but it's not easy, and we've already done it here.
+ */
+#define ELF_HWCAP (elf_hwcap)
+extern unsigned long elf_hwcap;
+
+/*
+ * This yields a string that ld.so will use to load implementation
+ * specific libraries for optimization. This is more specific in
+ * intent than poking at uname or /proc/cpuinfo.
+ */
+#define ELF_PLATFORM (NULL)
+
+#define ARCH_HAS_SETUP_ADDITIONAL_PAGES 1
+struct linux_binprm;
+extern int arch_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
+
+/* KVX relocs */
+#define R_KVX_NONE 0
+#define R_KVX_16 1
+#define R_KVX_32 2
+#define R_KVX_64 3
+#define R_KVX_S16_PCREL 4
+#define R_KVX_PCREL17 5
+#define R_KVX_PCREL27 6
+#define R_KVX_32_PCREL 7
+#define R_KVX_S37_PCREL_LO10 8
+#define R_KVX_S37_PCREL_UP27 9
+#define R_KVX_S43_PCREL_LO10 10
+#define R_KVX_S43_PCREL_UP27 11
+#define R_KVX_S43_PCREL_EX6 12
+#define R_KVX_S64_PCREL_LO10 13
+#define R_KVX_S64_PCREL_UP27 14
+#define R_KVX_S64_PCREL_EX27 15
+#define R_KVX_64_PCREL 16
+#define R_KVX_S16 17
+#define R_KVX_S32_LO5 18
+#define R_KVX_S32_UP27 19
+#define R_KVX_S37_LO10 20
+#define R_KVX_S37_UP27 21
+#define R_KVX_S37_GOTOFF_LO10 22
+#define R_KVX_S37_GOTOFF_UP27 23
+#define R_KVX_S43_GOTOFF_LO10 24
+#define R_KVX_S43_GOTOFF_UP27 25
+#define R_KVX_S43_GOTOFF_EX6 26
+#define R_KVX_32_GOTOFF 27
+#define R_KVX_64_GOTOFF 28
+#define R_KVX_32_GOT 29
+#define R_KVX_S37_GOT_LO10 30
+#define R_KVX_S37_GOT_UP27 31
+#define R_KVX_S43_GOT_LO10 32
+#define R_KVX_S43_GOT_UP27 33
+#define R_KVX_S43_GOT_EX6 34
+#define R_KVX_64_GOT 35
+#define R_KVX_GLOB_DAT 36
+#define R_KVX_COPY 37
+#define R_KVX_JMP_SLOT 38
+#define R_KVX_RELATIVE 39
+#define R_KVX_S43_LO10 40
+#define R_KVX_S43_UP27 41
+#define R_KVX_S43_EX6 42
+#define R_KVX_S64_LO10 43
+#define R_KVX_S64_UP27 44
+#define R_KVX_S64_EX27 45
+#define R_KVX_S37_GOTADDR_LO10 46
+#define R_KVX_S37_GOTADDR_UP27 47
+#define R_KVX_S43_GOTADDR_LO10 48
+#define R_KVX_S43_GOTADDR_UP27 49
+#define R_KVX_S43_GOTADDR_EX6 50
+#define R_KVX_S64_GOTADDR_LO10 51
+#define R_KVX_S64_GOTADDR_UP27 52
+#define R_KVX_S64_GOTADDR_EX27 53
+#define R_KVX_64_DTPMOD 54
+#define R_KVX_64_DTPOFF 55
+#define R_KVX_S37_TLS_DTPOFF_LO10 56
+#define R_KVX_S37_TLS_DTPOFF_UP27 57
+#define R_KVX_S43_TLS_DTPOFF_LO10 58
+#define R_KVX_S43_TLS_DTPOFF_UP27 59
+#define R_KVX_S43_TLS_DTPOFF_EX6 60
+#define R_KVX_S37_TLS_GD_LO10 61
+#define R_KVX_S37_TLS_GD_UP27 62
+#define R_KVX_S43_TLS_GD_LO10 63
+#define R_KVX_S43_TLS_GD_UP27 64
+#define R_KVX_S43_TLS_GD_EX6 65
+#define R_KVX_S37_TLS_LD_LO10 66
+#define R_KVX_S37_TLS_LD_UP27 67
+#define R_KVX_S43_TLS_LD_LO10 68
+#define R_KVX_S43_TLS_LD_UP27 69
+#define R_KVX_S43_TLS_LD_EX6 70
+#define R_KVX_64_TPOFF 71
+#define R_KVX_S37_TLS_IE_LO10 72
+#define R_KVX_S37_TLS_IE_UP27 73
+#define R_KVX_S43_TLS_IE_LO10 74
+#define R_KVX_S43_TLS_IE_UP27 75
+#define R_KVX_S43_TLS_IE_EX6 76
+#define R_KVX_S37_TLS_LE_LO10 77
+#define R_KVX_S37_TLS_LE_UP27 78
+#define R_KVX_S43_TLS_LE_LO10 79
+#define R_KVX_S43_TLS_LE_UP27 80
+#define R_KVX_S43_TLS_LE_EX6 81
+
+#endif /* _ASM_KVX_ELF_H */
diff --git a/arch/kvx/include/asm/vermagic.h b/arch/kvx/include/asm/vermagic.h
new file mode 100644
index 000000000000..fef9a33065df
--- /dev/null
+++ b/arch/kvx/include/asm/vermagic.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_VERMAGIC_H
+#define _ASM_KVX_VERMAGIC_H
+
+#define MODULE_ARCH_VERMAGIC "kvx"
+
+#endif /* _ASM_KVX_VERMAGIC_H */
diff --git a/arch/kvx/kernel/module.c b/arch/kvx/kernel/module.c
new file mode 100644
index 000000000000..b9383792ae45
--- /dev/null
+++ b/arch/kvx/kernel/module.c
@@ -0,0 +1,148 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Yann Sionneau
+ * Clement Leger
+ */
+
+#include <linux/elf.h>
+#include <linux/moduleloader.h>
+#include <linux/overflow.h>
+
+
+static int apply_rela_bits(Elf64_Addr loc, Elf64_Addr val,
+ int sign, int immsize, int bits, int rshift,
+ int lshift, unsigned int relocnum,
+ struct module *me)
+{
+ unsigned long long umax;
+ long long min, max;
+ unsigned long long mask = GENMASK_ULL(bits + lshift - 1, lshift);
+
+ if (sign) {
+ min = -(1ULL << (immsize - 1));
+ max = (1ULL << (immsize - 1)) - 1;
+ if ((long long) val < min || (long long) val > max)
+ goto too_big;
+ val = (Elf64_Addr)(((long) val) >> rshift);
+ } else {
+ if (immsize < 64)
+ umax = (1ULL << immsize) - 1;
+ else
+ umax = -1ULL;
+ if ((unsigned long long) val > umax)
+ goto too_big;
+ val >>= rshift;
+ }
+
+ val <<= lshift;
+ val &= mask;
+ if (bits <= 32)
+ *(u32 *) loc = (*(u32 *)loc & ~mask) | val;
+ else
+ *(u64 *) loc = (*(u64 *)loc & ~mask) | val;
+
+ return 0;
+too_big:
+ pr_err("%s: value %llx does not fit in %d bits for reloc %u",
+ me->name, val, bits, relocnum);
+ return -ENOEXEC;
+}
+
+int apply_relocate_add(Elf64_Shdr *sechdrs,
+ const char *strtab,
+ unsigned int symindex,
+ unsigned int relsec,
+ struct module *me)
+{
+ unsigned int i;
+ Elf64_Addr loc;
+ u64 val;
+ s64 sval;
+ Elf64_Sym *sym;
+ Elf64_Rela *rel = (void *)sechdrs[relsec].sh_addr;
+ int ret = 0;
+
+ pr_debug("Applying relocate section %u to %u\n",
+ relsec, sechdrs[relsec].sh_info);
+
+ for (i = 0; i < sechdrs[relsec].sh_size / sizeof(*rel); i++) {
+ /* This is where to make the change */
+ loc = (Elf64_Addr)sechdrs[sechdrs[relsec].sh_info].sh_addr
+ + rel[i].r_offset;
+ /* This is the symbol it is referring to. Note that all
+ * undefined symbols have been resolved.
+ */
+ sym = (Elf64_Sym *)sechdrs[symindex].sh_addr
+ + ELF64_R_SYM(rel[i].r_info);
+
+ pr_debug("type %d st_value %llx r_addend %llx loc %llx offset %llx\n",
+ (int)ELF64_R_TYPE(rel[i].r_info),
+ sym->st_value, rel[i].r_addend, (uint64_t)loc,
+ rel[i].r_offset);
+
+ val = sym->st_value + rel[i].r_addend;
+ switch (ELF64_R_TYPE(rel[i].r_info)) {
+ case R_KVX_NONE:
+ break;
+ case R_KVX_32:
+ ret = apply_rela_bits(loc, val, 0, 32, 32, 0, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_64:
+ ret = apply_rela_bits(loc, val, 0, 64, 64, 0, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S43_LO10:
+ ret = apply_rela_bits(loc, val, 1, 43, 10, 0, 6,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S64_LO10:
+ ret = apply_rela_bits(loc, val, 1, 64, 10, 0, 6,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S43_UP27:
+ ret = apply_rela_bits(loc, val, 1, 43, 27, 10, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S64_UP27:
+ ret = apply_rela_bits(loc, val, 1, 64, 27, 10, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S43_EX6:
+ ret = apply_rela_bits(loc, val, 1, 43, 6, 37, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_S64_EX27:
+ ret = apply_rela_bits(loc, val, 1, 64, 27, 37, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ case R_KVX_PCREL27:
+ if (__builtin_sub_overflow(val, loc, &sval)) {
+ pr_err("%s: Signed integer overflow, this should not happen\n",
+ me->name);
+ return -ENOEXEC;
+ }
+ sval >>= 2;
+ ret = apply_rela_bits(loc, (Elf64_Addr)sval, 1, 27, 27,
+ 0, 0,
+ ELF64_R_TYPE(rel[i].r_info),
+ me);
+ break;
+ default:
+ pr_err("%s: Unknown relocation: %llu\n",
+ me->name, ELF64_R_TYPE(rel[i].r_info));
+ ret = -ENOEXEC;
+ }
+ }
+ return ret;
+}
+
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread
* [RFC PATCH 15/25] kvx: Add misc common routines
2023-01-03 16:43 ` Yann Sionneau
` (15 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, Clement Leger, Guillaume Thouvenin,
Jonathan Borne, Julian Vetter, Julien Villette
Add some misc common routines for kvx, including: asm-offsets
routines, futex functions, i/o memory access functions.
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Jonathan Borne <jborne@kalray.eu>
Signed-off-by: Jonathan Borne <jborne@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Julien Villette <jvillette@kalray.eu>
Signed-off-by: Julien Villette <jvillette@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/futex.h | 141 ++++++++++++++++++++++++++++++
arch/kvx/include/asm/io.h | 34 ++++++++
arch/kvx/kernel/asm-offsets.c | 157 ++++++++++++++++++++++++++++++++++
arch/kvx/kernel/io.c | 96 +++++++++++++++++++++
4 files changed, 428 insertions(+)
create mode 100644 arch/kvx/include/asm/futex.h
create mode 100644 arch/kvx/include/asm/io.h
create mode 100644 arch/kvx/kernel/asm-offsets.c
create mode 100644 arch/kvx/kernel/io.c
diff --git a/arch/kvx/include/asm/futex.h b/arch/kvx/include/asm/futex.h
new file mode 100644
index 000000000000..b71b52339729
--- /dev/null
+++ b/arch/kvx/include/asm/futex.h
@@ -0,0 +1,141 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2018-2023 Kalray Inc.
+ * Authors:
+ * Clement Leger <cleger@kalray.eu>
+ * Yann Sionneau <ysionneau@kalray.eu>
+ * Jonathan Borne <jborne@kalray.eu>
+ *
+ * Part of code is taken from RiscV port
+ */
+
+#ifndef _ASM_KVX_FUTEX_H
+#define _ASM_KVX_FUTEX_H
+
+#ifdef __KERNEL__
+
+#include <linux/futex.h>
+#include <linux/uaccess.h>
+
+#define __futex_atomic_op(insn, ret, oldval, uaddr, oparg) \
+{ \
+ __enable_user_access(); \
+ __asm__ __volatile__ ( \
+ " fence \n" \
+ " ;;\n " \
+ "1: lwz $r63 = 0[%[u]] \n" \
+ " ;;\n " \
+ " " insn " \n" \
+ " ;;\n " \
+ " acswapw 0[%[u]], $r62r63 \n" \
+ " ;;\n " \
+ " cb.deqz $r62? 1b \n" \
+ " ;;\n " \
+ " copyd %[ov] = $r63 \n" \
+ " ;;\n " \
+ "2: \n" \
+ " .section .fixup,\"ax\" \n" \
+ "3: make %[r] = 2b \n" \
+ " ;;\n " \
+ " make %[r] = %[e] \n" \
+ " igoto %[r] \n" \
+ " ;;\n " \
+ " .previous \n" \
+ " .section __ex_table,\"a\" \n" \
+ " .align 8 \n" \
+ " .dword 1b,3b \n" \
+ " .dword 2b,3b \n" \
+ " .previous \n" \
+ : [r] "+r" (ret), [ov] "+r" (oldval) \
+ : [u] "r" (uaddr), \
+ [op] "r" (oparg), [e] "i" (-EFAULT) \
+ : "r62", "r63", "memory"); \
+ __disable_user_access(); \
+}
+
+
+static inline int
+arch_futex_atomic_op_inuser(int op, u32 oparg, int *oval, u32 __user *uaddr)
+{
+ int oldval = 0, ret = 0;
+
+ if (!access_ok(uaddr, sizeof(u32)))
+ return -EFAULT;
+ switch (op) {
+ case FUTEX_OP_SET: /* *(int *)UADDR = OPARG; */
+ __futex_atomic_op("copyd $r62 = %[op]",
+ ret, oldval, uaddr, oparg);
+ break;
+ case FUTEX_OP_ADD: /* *(int *)UADDR += OPARG; */
+ __futex_atomic_op("addw $r62 = $r63, %[op]",
+ ret, oldval, uaddr, oparg);
+ break;
+ case FUTEX_OP_OR: /* *(int *)UADDR |= OPARG; */
+ __futex_atomic_op("orw $r62 = $r63, %[op]",
+ ret, oldval, uaddr, oparg);
+ break;
+ case FUTEX_OP_ANDN: /* *(int *)UADDR &= ~OPARG; */
+ __futex_atomic_op("andnw $r62 = %[op], $r63",
+ ret, oldval, uaddr, oparg);
+ break;
+ case FUTEX_OP_XOR:
+ __futex_atomic_op("xorw $r62 = $r63, %[op]",
+ ret, oldval, uaddr, oparg);
+ break;
+ default:
+ ret = -ENOSYS;
+ }
+
+ if (!ret)
+ *oval = oldval;
+
+ return ret;
+}
+
+static inline int futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr,
+ u32 oldval, u32 newval)
+{
+ int ret = 0;
+
+ if (!access_ok(uaddr, sizeof(u32)))
+ return -EFAULT;
+ __enable_user_access();
+ __asm__ __volatile__ (
+ " fence \n"/* commit previous stores */
+ " copyd $r63 = %[ov] \n"/* init "expect" with ov */
+ " copyd $r62 = %[nv] \n"/* init "update" with nv */
+ " ;;\n "
+ "1: acswapw 0[%[u]], $r62r63 \n"
+ " ;;\n "
+ " cb.dnez $r62? 3f \n"/* if acswap ok -> return */
+ " ;;\n "
+ "2: lws $r63 = 0[%[u]] \n"/* fail -> load old value */
+ " ;;\n "
+ " compw.ne $r62 = $r63, %[ov] \n"/* check if equal to "old" */
+ " ;;\n "
+ " cb.deqz $r62? 1b \n"/* if not equal, try again */
+ " ;;\n "
+ "3: \n"
+ " .section .fixup,\"ax\" \n"
+ "4: make %[r] = 3b \n"
+ " ;;\n "
+ " make %[r] = %[e] \n"
+ " igoto %[r] \n"/* goto 3b */
+ " ;;\n "
+ " .previous \n"
+ " .section __ex_table,\"a\" \n"
+ " .align 8 \n"
+ " .dword 1b,4b \n"
+ " .dword 2b,4b \n"
+ ".previous \n"
+ : [r] "+r" (ret)
+ : [ov] "r" (oldval), [nv] "r" (newval),
+ [e] "i" (-EFAULT), [u] "r" (uaddr)
+ : "r62", "r63", "memory");
+ __disable_user_access();
+ *uval = oldval;
+ return ret;
+}
+
+#endif
+#endif /* _ASM_KVX_FUTEX_H */
diff --git a/arch/kvx/include/asm/io.h b/arch/kvx/include/asm/io.h
new file mode 100644
index 000000000000..c5e458c59bbb
--- /dev/null
+++ b/arch/kvx/include/asm/io.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_IO_H
+#define _ASM_KVX_IO_H
+
+#include <linux/types.h>
+
+#include <asm/page.h>
+#include <asm/pgtable.h>
+
+#define _PAGE_IOREMAP _PAGE_KERNEL_DEVICE
+
+/*
+ * String version of I/O memory access operations.
+ */
+extern void __memcpy_fromio(void *to, const volatile void __iomem *from,
+ size_t count);
+extern void __memcpy_toio(volatile void __iomem *to, const void *from,
+ size_t count);
+extern void __memset_io(volatile void __iomem *dst, int c, size_t count);
+
+#define memset_io(c, v, l) __memset_io((c), (v), (l))
+#define memcpy_fromio(a, c, l) __memcpy_fromio((a), (c), (l))
+#define memcpy_toio(c, a, l) __memcpy_toio((c), (a), (l))
+
+#include <asm-generic/io.h>
+
+extern int devmem_is_allowed(unsigned long pfn);
+
+#endif /* _ASM_KVX_IO_H */
diff --git a/arch/kvx/kernel/asm-offsets.c b/arch/kvx/kernel/asm-offsets.c
new file mode 100644
index 000000000000..3e79b6dd13bd
--- /dev/null
+++ b/arch/kvx/kernel/asm-offsets.c
@@ -0,0 +1,157 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ * Yann Sionneau
+ */
+
+#include <linux/preempt.h>
+#include <linux/thread_info.h>
+#include <linux/kbuild.h>
+#include <linux/stddef.h>
+#include <linux/sched.h>
+#include <linux/bug.h>
+
+#include <asm/processor.h>
+#include <asm/ptrace.h>
+#include <asm/page.h>
+#include <asm/fixmap.h>
+#include <asm/page_size.h>
+#include <asm/pgtable.h>
+#include <asm/ptrace.h>
+#include <asm/tlb_defs.h>
+#include <asm/mmu_stats.h>
+#include <asm/stacktrace.h>
+
+int foo(void)
+{
+ BUILD_BUG_ON(sizeof(struct pt_regs) != PT_REGS_STRUCT_EXPECTED_SIZE);
+ /*
+ * For stack alignment purposes we must make sure the pt_regs size is
+ * a mutliple of stack_align
+ */
+ BUILD_BUG_ON(!IS_ALIGNED(sizeof(struct pt_regs), STACK_ALIGNMENT));
+
+ /* Check that user_pt_regs size matches the beginning of pt_regs */
+ BUILD_BUG_ON((offsetof(struct user_pt_regs, spc) + sizeof(uint64_t)) !=
+ sizeof(struct user_pt_regs));
+
+ DEFINE(FIX_GDB_MEM_BASE_IDX, FIX_GDB_BARE_DISPLACED_MEM_BASE);
+
+#ifdef CONFIG_DEBUG_EXCEPTION_STACK
+ DEFINE(STACK_REG_SIZE, ALIGN(sizeof(uint64_t), STACK_ALIGNMENT));
+#endif
+
+ /*
+ * We allocate a pt_regs on the stack when entering the kernel. This
+ * ensures the alignment is sane.
+ */
+ DEFINE(PT_SIZE_ON_STACK, sizeof(struct pt_regs));
+ DEFINE(TI_FLAGS_SIZE, sizeof(unsigned long));
+ DEFINE(QUAD_REG_SIZE, 4 * sizeof(uint64_t));
+
+ /*
+ * When restoring registers, we do not want to restore r12
+ * right now since this is our stack pointer. Allow to save
+ * only $r13 by using this offset.
+ */
+ OFFSET(PT_R12, pt_regs, r12);
+ OFFSET(PT_R13, pt_regs, r13);
+ OFFSET(PT_TP, pt_regs, tp);
+ OFFSET(PT_R14R15, pt_regs, r14);
+ OFFSET(PT_R16R17, pt_regs, r16);
+ OFFSET(PT_R18R19, pt_regs, r18);
+ OFFSET(PT_FP, pt_regs, fp);
+ OFFSET(PT_SPS, pt_regs, sps);
+
+ /* Quad description */
+ OFFSET(PT_Q0, pt_regs, r0);
+ OFFSET(PT_Q4, pt_regs, r4);
+ OFFSET(PT_Q8, pt_regs, r8);
+ OFFSET(PT_Q12, pt_regs, r12);
+ OFFSET(PT_Q16, pt_regs, r16);
+ OFFSET(PT_Q20, pt_regs, r20);
+ OFFSET(PT_Q24, pt_regs, r24);
+ OFFSET(PT_Q28, pt_regs, r28);
+ OFFSET(PT_Q32, pt_regs, r32);
+ OFFSET(PT_Q36, pt_regs, r36);
+ OFFSET(PT_R38, pt_regs, r38);
+ OFFSET(PT_Q40, pt_regs, r40);
+ OFFSET(PT_Q44, pt_regs, r44);
+ OFFSET(PT_Q48, pt_regs, r48);
+ OFFSET(PT_Q52, pt_regs, r52);
+ OFFSET(PT_Q56, pt_regs, r56);
+ OFFSET(PT_Q60, pt_regs, r60);
+ OFFSET(PT_CS_SPC_SPS_ES, pt_regs, cs);
+ OFFSET(PT_LC_LE_LS_RA, pt_regs, lc);
+ OFFSET(PT_ILR, pt_regs, ilr);
+ OFFSET(PT_ORIG_R0, pt_regs, orig_r0);
+
+ /*
+ * Flags in thread info
+ */
+ OFFSET(TASK_TI_FLAGS, task_struct, thread_info.flags);
+
+ /*
+ * Stack pointers
+ */
+ OFFSET(TASK_THREAD_KERNEL_SP, task_struct, thread.kernel_sp);
+
+ /*
+ * Offsets to save registers in switch_to using quads
+ */
+ OFFSET(CTX_SWITCH_RA_SP_R18_R19, task_struct, thread.ctx_switch.ra);
+ OFFSET(CTX_SWITCH_Q20, task_struct, thread.ctx_switch.r20);
+ OFFSET(CTX_SWITCH_Q24, task_struct, thread.ctx_switch.r24);
+ OFFSET(CTX_SWITCH_Q28, task_struct, thread.ctx_switch.r28);
+ OFFSET(CTX_SWITCH_FP, task_struct, thread.ctx_switch.fp);
+
+#ifdef CONFIG_ENABLE_TCA
+ OFFSET(CTX_SWITCH_TCA_REGS, task_struct, thread.ctx_switch.tca_regs[0]);
+ OFFSET(CTX_SWITCH_TCA_REGS_SAVED, task_struct,
+ thread.ctx_switch.tca_regs_saved);
+ DEFINE(TCA_REG_SIZE, sizeof(struct tca_reg));
+#endif
+
+ /* Save area offset */
+ OFFSET(TASK_THREAD_SAVE_AREA, task_struct, thread.save_area);
+
+ /* Fast tlb refill defines */
+ OFFSET(TASK_ACTIVE_MM, task_struct, active_mm);
+ OFFSET(MM_PGD, mm_struct, pgd);
+#ifdef CONFIG_KVX_DEBUG_ASN
+ OFFSET(MM_CTXT_ASN, mm_struct, context.asn);
+#endif
+
+#ifdef CONFIG_KVX_MMU_STATS
+ DEFINE(MMU_REFILL_SIZE, sizeof(struct mmu_refill_stats));
+
+ OFFSET(MMU_STATS_REFILL_USER_OFF, mmu_stats,
+ refill[MMU_REFILL_TYPE_USER]);
+ OFFSET(MMU_STATS_REFILL_KERNEL_OFF, mmu_stats,
+ refill[MMU_REFILL_TYPE_KERNEL]);
+ OFFSET(MMU_STATS_REFILL_KERNEL_DIRECT_OFF, mmu_stats,
+ refill[MMU_REFILL_TYPE_KERNEL_DIRECT]);
+ OFFSET(MMU_STATS_CYCLES_BETWEEN_REFILL_OFF, mmu_stats,
+ cycles_between_refill);
+ OFFSET(MMU_STATS_LAST_REFILL, mmu_stats, last_refill);
+
+ OFFSET(TASK_THREAD_ENTRY_TS, task_struct, thread.trap_entry_ts);
+#endif
+
+ DEFINE(ASM_PGDIR_SHIFT, PGDIR_SHIFT);
+ DEFINE(ASM_PMD_SHIFT, PMD_SHIFT);
+
+ DEFINE(ASM_PGDIR_BITS, PGDIR_BITS);
+ DEFINE(ASM_PMD_BITS, PMD_BITS);
+ DEFINE(ASM_PTE_BITS, PTE_BITS);
+
+ DEFINE(ASM_PTRS_PER_PGD, PTRS_PER_PGD);
+ DEFINE(ASM_PTRS_PER_PMD, PTRS_PER_PMD);
+ DEFINE(ASM_PTRS_PER_PTE, PTRS_PER_PTE);
+
+ DEFINE(ASM_TLB_PS, TLB_DEFAULT_PS);
+
+ return 0;
+}
diff --git a/arch/kvx/kernel/io.c b/arch/kvx/kernel/io.c
new file mode 100644
index 000000000000..0922c1d6d0f7
--- /dev/null
+++ b/arch/kvx/kernel/io.c
@@ -0,0 +1,96 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/arm/kernel/io.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/export.h>
+#include <linux/types.h>
+#include <linux/io.h>
+
+#define REPLICATE_BYTE_MASK 0x0101010101010101
+
+/*
+ * Copy data from IO memory space to "real" memory space.
+ */
+void __memcpy_fromio(void *to, const volatile void __iomem *from, size_t count)
+{
+ while (count && !IS_ALIGNED((unsigned long)from, 8)) {
+ *(u8 *)to = __raw_readb(from);
+ from++;
+ to++;
+ count--;
+ }
+
+ while (count >= 8) {
+ *(u64 *)to = __raw_readq(from);
+ from += 8;
+ to += 8;
+ count -= 8;
+ }
+
+ while (count) {
+ *(u8 *)to = __raw_readb(from);
+ from++;
+ to++;
+ count--;
+ }
+}
+EXPORT_SYMBOL(__memcpy_fromio);
+
+/*
+ * Copy data from "real" memory space to IO memory space.
+ */
+void __memcpy_toio(volatile void __iomem *to, const void *from, size_t count)
+{
+ while (count && !IS_ALIGNED((unsigned long)to, 8)) {
+ __raw_writeb(*(u8 *)from, to);
+ from++;
+ to++;
+ count--;
+ }
+
+ while (count >= 8) {
+ __raw_writeq(*(u64 *)from, to);
+ from += 8;
+ to += 8;
+ count -= 8;
+ }
+
+ while (count) {
+ __raw_writeb(*(u8 *)from, to);
+ from++;
+ to++;
+ count--;
+ }
+}
+EXPORT_SYMBOL(__memcpy_toio);
+
+/*
+ * "memset" on IO memory space.
+ */
+void __memset_io(volatile void __iomem *dst, int c, size_t count)
+{
+ u64 qc = __builtin_kvx_sbmm8(c, REPLICATE_BYTE_MASK);
+
+ while (count && !IS_ALIGNED((unsigned long)dst, 8)) {
+ __raw_writeb(c, dst);
+ dst++;
+ count--;
+ }
+
+ while (count >= 8) {
+ __raw_writeq(qc, dst);
+ dst += 8;
+ count -= 8;
+ }
+
+ while (count) {
+ __raw_writeb(c, dst);
+ dst++;
+ count--;
+ }
+}
+EXPORT_SYMBOL(__memset_io);
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 16/25] kvx: Add some library functions
2023-01-03 16:43 ` Yann Sionneau
` (16 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-05 13:05 ` Clément Léger
-1 siblings, 1 reply; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, Clement Leger, Jules Maselbas,
Julian Vetter, Marius Gligor
Add some library functions for kvx, including: delay, memset,
memcpy, strlen, clear_page, copy_page, raw_copy_from/to_user,
asm_clear_user.
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/string.h | 20 ++
arch/kvx/kernel/kvx_ksyms.c | 5 +
arch/kvx/lib/clear_page.S | 40 ++++
arch/kvx/lib/copy_page.S | 90 +++++++++
arch/kvx/lib/delay.c | 39 ++++
arch/kvx/lib/memcpy.c | 70 +++++++
arch/kvx/lib/memset.S | 351 ++++++++++++++++++++++++++++++++++
arch/kvx/lib/strlen.S | 122 ++++++++++++
arch/kvx/lib/usercopy.S | 90 +++++++++
9 files changed, 827 insertions(+)
create mode 100644 arch/kvx/include/asm/string.h
create mode 100644 arch/kvx/lib/clear_page.S
create mode 100644 arch/kvx/lib/copy_page.S
create mode 100644 arch/kvx/lib/delay.c
create mode 100644 arch/kvx/lib/memcpy.c
create mode 100644 arch/kvx/lib/memset.S
create mode 100644 arch/kvx/lib/strlen.S
create mode 100644 arch/kvx/lib/usercopy.S
diff --git a/arch/kvx/include/asm/string.h b/arch/kvx/include/asm/string.h
new file mode 100644
index 000000000000..677c1393a5cd
--- /dev/null
+++ b/arch/kvx/include/asm/string.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Jules Maselbas
+ */
+
+#ifndef _ASM_KVX_STRING_H
+#define _ASM_KVX_STRING_H
+
+#define __HAVE_ARCH_MEMSET
+extern void *memset(void *s, int c, size_t n);
+
+#define __HAVE_ARCH_MEMCPY
+extern void *memcpy(void *dest, const void *src, size_t n);
+
+#define __HAVE_ARCH_STRLEN
+extern size_t strlen(const char *s);
+
+#endif /* _ASM_KVX_STRING_H */
diff --git a/arch/kvx/kernel/kvx_ksyms.c b/arch/kvx/kernel/kvx_ksyms.c
index 18990aaf259f..678f81716dea 100644
--- a/arch/kvx/kernel/kvx_ksyms.c
+++ b/arch/kvx/kernel/kvx_ksyms.c
@@ -22,3 +22,8 @@ DECLARE_EXPORT(__umoddi3);
DECLARE_EXPORT(__divdi3);
DECLARE_EXPORT(__udivdi3);
DECLARE_EXPORT(__multi3);
+
+DECLARE_EXPORT(clear_page);
+DECLARE_EXPORT(copy_page);
+DECLARE_EXPORT(memset);
+DECLARE_EXPORT(asm_clear_user);
diff --git a/arch/kvx/lib/clear_page.S b/arch/kvx/lib/clear_page.S
new file mode 100644
index 000000000000..364fe0663ca2
--- /dev/null
+++ b/arch/kvx/lib/clear_page.S
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Marius Gligor
+ * Clement Leger
+ */
+
+#include <linux/linkage.h>
+#include <linux/export.h>
+#include <linux/const.h>
+
+#include <asm/cache.h>
+#include <asm/page.h>
+
+#define CLEAR_PAGE_LOOP_COUNT (PAGE_SIZE / 32)
+
+/*
+ * Clear page @dest.
+ *
+ * Parameters:
+ * r0 - dest page
+ */
+ENTRY(clear_page)
+ make $r1 = CLEAR_PAGE_LOOP_COUNT
+ ;;
+ make $r4 = 0
+ make $r5 = 0
+ make $r6 = 0
+ make $r7 = 0
+ ;;
+
+ loopdo $r1, clear_page_done
+ ;;
+ so 0[$r0] = $r4r5r6r7
+ addd $r0 = $r0, 32
+ ;;
+ clear_page_done:
+ ret
+ ;;
+ENDPROC(clear_page)
diff --git a/arch/kvx/lib/copy_page.S b/arch/kvx/lib/copy_page.S
new file mode 100644
index 000000000000..4bb82d1c964c
--- /dev/null
+++ b/arch/kvx/lib/copy_page.S
@@ -0,0 +1,90 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/linkage.h>
+#include <linux/const.h>
+
+#include <asm/page.h>
+
+/* We have 8 load/store octuple (32 bytes) per hardware loop */
+#define COPY_SIZE_PER_LOOP (32 * 8)
+#define COPY_PAGE_LOOP_COUNT (PAGE_SIZE / COPY_SIZE_PER_LOOP)
+
+/*
+ * Copy a page from src to dest (both are page aligned)
+ * In order to recover from smem latency, unroll the loop to trigger multiple
+ * onfly loads and avoid waiting too much for them to return.
+ * We use 8 * 32 load even though we could use more (up to 10 loads) to simplify
+ * the handling using a single hardware loop
+ *
+ * Parameters:
+ * r0 - dest
+ * r1 - src
+ */
+ENTRY(copy_page)
+ make $r2 = COPY_PAGE_LOOP_COUNT
+ make $r3 = 0
+ ;;
+ loopdo $r2, copy_page_done
+ ;;
+ /*
+ * Load 8 * 32 bytes using uncached access to avoid hitting
+ * the cache
+ */
+ lo.xs $r32r33r34r35 = $r3[$r1]
+ /* Copy current copy index for store */
+ copyd $r2 = $r3
+ addd $r3 = $r3, 1
+ ;;
+ lo.xs $r36r37r38r39 = $r3[$r1]
+ addd $r3 = $r3, 1
+ ;;
+ lo.xs $r40r41r42r43 = $r3[$r1]
+ addd $r3 = $r3, 1
+ ;;
+ lo.xs $r44r45r46r47 = $r3[$r1]
+ addd $r3 = $r3, 1
+ ;;
+ lo.xs $r48r49r50r51 = $r3[$r1]
+ addd $r3 = $r3, 1
+ ;;
+ lo.xs $r52r53r54r55 = $r3[$r1]
+ addd $r3 = $r3, 1
+ ;;
+ lo.xs $r56r57r58r59 = $r3[$r1]
+ addd $r3 = $r3, 1
+ ;;
+ lo.xs $r60r61r62r63 = $r3[$r1]
+ addd $r3 = $r3, 1
+ ;;
+ /* And then store all of them */
+ so.xs $r2[$r0] = $r32r33r34r35
+ addd $r2 = $r2, 1
+ ;;
+ so.xs $r2[$r0] = $r36r37r38r39
+ addd $r2 = $r2, 1
+ ;;
+ so.xs $r2[$r0] = $r40r41r42r43
+ addd $r2 = $r2, 1
+ ;;
+ so.xs $r2[$r0] = $r44r45r46r47
+ addd $r2 = $r2, 1
+ ;;
+ so.xs $r2[$r0] = $r48r49r50r51
+ addd $r2 = $r2, 1
+ ;;
+ so.xs $r2[$r0] = $r52r53r54r55
+ addd $r2 = $r2, 1
+ ;;
+ so.xs $r2[$r0] = $r56r57r58r59
+ addd $r2 = $r2, 1
+ ;;
+ so.xs $r2[$r0] = $r60r61r62r63
+ ;;
+ copy_page_done:
+ ret
+ ;;
+ENDPROC(copy_page)
diff --git a/arch/kvx/lib/delay.c b/arch/kvx/lib/delay.c
new file mode 100644
index 000000000000..11295eedc3f5
--- /dev/null
+++ b/arch/kvx/lib/delay.c
@@ -0,0 +1,39 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/export.h>
+#include <linux/delay.h>
+
+#include <asm/param.h>
+#include <asm/timex.h>
+
+void __delay(unsigned long loops)
+{
+ cycles_t target_cycle = get_cycles() + loops;
+
+ while (get_cycles() < target_cycle);
+}
+EXPORT_SYMBOL(__delay);
+
+inline void __const_udelay(unsigned long xloops)
+{
+ u64 loops = (u64)xloops * (u64)loops_per_jiffy * HZ;
+
+ __delay(loops >> 32);
+}
+EXPORT_SYMBOL(__const_udelay);
+
+void __udelay(unsigned long usecs)
+{
+ __const_udelay(usecs * 0x10C7UL); /* 2**32 / 1000000 (rounded up) */
+}
+EXPORT_SYMBOL(__udelay);
+
+void __ndelay(unsigned long nsecs)
+{
+ __const_udelay(nsecs * 0x5UL); /* 2**32 / 1000000000 (rounded up) */
+}
+EXPORT_SYMBOL(__ndelay);
diff --git a/arch/kvx/lib/memcpy.c b/arch/kvx/lib/memcpy.c
new file mode 100644
index 000000000000..b81f746a80ee
--- /dev/null
+++ b/arch/kvx/lib/memcpy.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ */
+
+#include <linux/export.h>
+#include <linux/types.h>
+
+void *memcpy(void *dest, const void *src, size_t n)
+{
+ __uint128_t *tmp128_d = dest;
+ const __uint128_t *tmp128_s = src;
+ uint64_t *tmp64_d;
+ const uint64_t *tmp64_s;
+ uint32_t *tmp32_d;
+ const uint32_t *tmp32_s;
+ uint16_t *tmp16_d;
+ const uint16_t *tmp16_s;
+ uint8_t *tmp8_d;
+ const uint8_t *tmp8_s;
+
+ while (n >= 16) {
+ *tmp128_d = *tmp128_s;
+ tmp128_d++;
+ tmp128_s++;
+ n -= 16;
+ }
+
+ tmp64_d = (uint64_t *) tmp128_d;
+ tmp64_s = (uint64_t *) tmp128_s;
+ while (n >= 8) {
+ *tmp64_d = *tmp64_s;
+ tmp64_d++;
+ tmp64_s++;
+ n -= 8;
+ }
+
+ tmp32_d = (uint32_t *) tmp64_d;
+ tmp32_s = (uint32_t *) tmp64_s;
+ while (n >= 4) {
+ *tmp32_d = *tmp32_s;
+ tmp32_d++;
+ tmp32_s++;
+ n -= 4;
+ }
+
+ tmp16_d = (uint16_t *) tmp32_d;
+ tmp16_s = (uint16_t *) tmp32_s;
+ while (n >= 2) {
+ *tmp16_d = *tmp16_s;
+ tmp16_d++;
+ tmp16_s++;
+ n -= 2;
+ }
+
+ tmp8_d = (uint8_t *) tmp16_d;
+ tmp8_s = (uint8_t *) tmp16_s;
+ while (n >= 1) {
+ *tmp8_d = *tmp8_s;
+ tmp8_d++;
+ tmp8_s++;
+ n--;
+ }
+
+ return dest;
+}
+EXPORT_SYMBOL(memcpy);
+
diff --git a/arch/kvx/lib/memset.S b/arch/kvx/lib/memset.S
new file mode 100644
index 000000000000..9eebc28da2be
--- /dev/null
+++ b/arch/kvx/lib/memset.S
@@ -0,0 +1,351 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Marius Gligor
+ */
+
+#include <linux/linkage.h>
+
+#include <asm/cache.h>
+
+#define REPLICATE_BYTE_MASK 0x0101010101010101
+#define MIN_SIZE_FOR_ALIGN 128
+
+/*
+ * Optimized memset for kvx architecture
+ *
+ * In order to optimize memset on kvx, we can use various things:
+ * - conditionnal store which avoid branch penalty
+ * - store half/word/double/quad/octuple to store up to 16 bytes at a time
+ * - dzerol to zero a cacheline when the pattern is '0' (often the case)
+ * - hardware loop for steady cases.
+ *
+ * First, we assume that memset is mainly used for zeroing areas. In order
+ * to optimize this case, we consider it to be the fast path of the algorithm.
+ * In both cases (0 and non 0 pattern), we start by checking if the size is
+ * below a minimum size. If so, we skip the alignment part. Indeed, the kvx
+ * supports misalignment and the penalty for letting it do unaligned accesses is
+ * lower than trying to realigning us. So for small sizes, we don't even bother
+ * to realign. Minor difference is that in the memset with 0, we skip after the
+ * dzerol loop since dzerol must be cache-line aligned (no misalignment of
+ * course).
+ * Regarding the non 0 pattern memset, we use sbmm to replicate the pattern on
+ * all bits on a register in one call.
+ * Once alignment has been reached, we can do the hardware loop for both cases(
+ * store octuple/dzerol) in order to optimize throughput. Care must be taken to
+ * align hardware loops on at least 8 bytes for performances.
+ * Once the main loop has been done, we finish the copy by checking length to do
+ * the necessary calls to store remaining bytes.
+ *
+ * Pseudo code (applies for non 0 pattern):
+ *
+ * int memset(void *dest, char pattern, long length)
+ * {
+ * long dest_align = -((long) dest);
+ * long copy;
+ * long orig_dest = dest;
+ *
+ * uint64_t pattern = sbmm8(pattern, 0x0101010101010101);
+ * uint128_t pattern128 = pattern << 64 | pattern;
+ * uint256_t pattern128 = pattern128 << 128 | pattern128;
+ *
+ * // Keep only low bits
+ * dest_align &= 0x1F;
+ * length -= dest_align;
+ *
+ * // Byte align
+ * copy = align & (1 << 0);
+ * if (copy)
+ * *((u8 *) dest) = pattern;
+ * dest += copy;
+ * // Half align
+ * copy = align & (1 << 1);
+ * if (copy)
+ * *((u16 *) dest) = pattern;
+ * dest += copy;
+ * // Word align
+ * copy = align & (1 << 2);
+ * if (copy)
+ * *((u32 *) dest) = pattern;
+ * dest += copy;
+ * // Double align
+ * copy = align & (1 << 3);
+ * if (copy)
+ * *((u64 *) dest) = pattern;
+ * dest += copy;
+ * // Quad align
+ * copy = align & (1 << 4);
+ * if (copy)
+ * *((u128 *) dest) = pattern128;
+ * dest += copy;
+ *
+ * // We are now aligned on 256 bits
+ * loop_octuple_count = size >> 5;
+ * for (i = 0; i < loop_octuple_count; i++) {
+ * *((u256 *) dest) = pattern256;
+ * dest += 32;
+ * }
+ *
+ * if (length == 0)
+ * return orig_dest;
+ *
+ * // Copy remaining part
+ * remain = length & (1 << 4);
+ * if (copy)
+ * *((u128 *) dest) = pattern128;
+ * dest += remain;
+ * remain = length & (1 << 3);
+ * if (copy)
+ * *((u64 *) dest) = pattern;
+ * dest += remain;
+ * remain = length & (1 << 2);
+ * if (copy)
+ * *((u32 *) dest) = pattern;
+ * dest += remain;
+ * remain = length & (1 << 1);
+ * if (copy)
+ * *((u16 *) dest) = pattern;
+ * dest += remain;
+ * remain = length & (1 << 0);
+ * if (copy)
+ * *((u8 *) dest) = pattern;
+ * dest += remain;
+ *
+ * return orig_dest;
+ * }
+ */
+
+.text
+.align 16
+ENTRY(memset):
+ make $r32 = 0
+ make $r33 = 0
+ /* Check if length < KVX_DCACHE_LINE_SIZE */
+ compd.ltu $r7 = $r2, KVX_DCACHE_LINE_SIZE
+ /* Jump to generic memset if pattern is != 0 */
+ cb.dnez $r1? memset_non_0_pattern
+ ;;
+ /* Preserve return value */
+ copyd $r3 = $r0
+ /* Invert address to compute size to copy to be aligned on 32 bytes */
+ negd $r5 = $r0
+ /* Remaining bytes for 16 bytes store (for alignment on 64 bytes) */
+ andd $r8 = $r2, (1 << 5)
+ copyq $r34r35 = $r32, $r33
+ /* Skip loopdo with dzerol if length < KVX_DCACHE_LINE_SIZE */
+ cb.dnez $r7? .Ldzerol_done
+ ;;
+ /* Compute the size that will be copied to align on 64 bytes boundary */
+ andw $r6 = $r5, 0x3F
+ /* Check if address is aligned on 64 bytes */
+ andw $r9 = $r0, 0x3F
+ /* Alignment */
+ nop
+ ;;
+ /* If address already aligned on 64 bytes, jump to dzerol loop */
+ cb.deqz $r9? .Laligned_64
+ /* Remove unaligned part from length */
+ sbfd $r2 = $r6, $r2
+ /* Check if we need to copy 1 byte */
+ andw $r4 = $r5, (1 << 0)
+ ;;
+ /* If we are not aligned, store byte */
+ sb.dnez $r4? [$r0] = $r32
+ /* Check if we need to copy 2 bytes */
+ andw $r4 = $r5, (1 << 1)
+ /* Add potentially copied part for next store offset */
+ addd $r0 = $r0, $r4
+ ;;
+ sh.dnez $r4? [$r0] = $r32
+ /* Check if we need to copy 4 bytes */
+ andw $r4 = $r5, (1 << 2)
+ addd $r0 = $r0, $r4
+ ;;
+ sw.dnez $r4? [$r0] = $r32
+ /* Check if we need to copy 8 bytes */
+ andw $r4 = $r5, (1 << 3)
+ addd $r0 = $r0, $r4
+ ;;
+ sd.dnez $r4? [$r0] = $r32
+ /* Check if we need to copy 16 bytes */
+ andw $r4 = $r5, (1 << 4)
+ addd $r0 = $r0, $r4
+ ;;
+ sq.dnez $r4? [$r0] = $r32r33
+ /* Check if we need to copy 32 bytes */
+ andw $r4 = $r5, (1 << 5)
+ addd $r0 = $r0, $r4
+ ;;
+ so.dnez $r4? [$r0] = $r32r33r34r35
+ addd $r0 = $r0, $r4
+ ;;
+.Laligned_64:
+ /* Prepare amount of data for dzerol */
+ srld $r10 = $r2, 6
+ /* Size to be handled in loopdo */
+ andd $r4 = $r2, ~0x3F
+ make $r11 = 64
+ cb.deqz $r2? .Lmemset_done
+ ;;
+ /* Remaining bytes for 16 bytes store */
+ andw $r8 = $r2, (1 << 5)
+ /* Skip dzerol if there are not enough data for 64 bytes store */
+ cb.deqz $r10? .Ldzerol_done
+ /* Update length to copy */
+ sbfd $r2 = $r4, $r2
+ ;;
+ loopdo $r10, .Ldzerol_done
+ ;;
+ so 0[$r0], $r32r33r34r35
+ ;;
+ so 32[$r0], $r32r33r34r35
+ addd $r0 = $r0, $r11
+ ;;
+ .Ldzerol_done:
+ /*
+ * Now that we have handled every aligned bytes using 'dzerol', we can
+ * handled the remainder of length using store by decrementing size
+ * We also exploit the fact we are aligned to simply check remaining
+ * size */
+ so.dnez $r8? [$r0] = $r32r33r34r35
+ addd $r0 = $r0, $r8
+ /* Remaining bytes for 16 bytes store */
+ andw $r8 = $r2, (1 << 4)
+ cb.deqz $r2? .Lmemset_done
+ ;;
+ sq.dnez $r8? [$r0] = $r32r33
+ addd $r0 = $r0, $r8
+ /* Remaining bytes for 8 bytes store */
+ andw $r8 = $r2, (1 << 3)
+ ;;
+ sd.dnez $r8? [$r0] = $r32
+ addd $r0 = $r0, $r8
+ /* Remaining bytes for 4 bytes store */
+ andw $r8 = $r2, (1 << 2)
+ ;;
+ sw.dnez $r8? [$r0] = $r32
+ addd $r0 = $r0, $r8
+ /* Remaining bytes for 2 bytes store */
+ andw $r8 = $r2, (1 << 1)
+ ;;
+ sh.dnez $r8? [$r0] = $r32
+ addd $r0 = $r0, $r8
+ ;;
+ sb.odd $r2? [$r0] = $r32
+ /* Restore original value */
+ copyd $r0 = $r3
+ ret
+ ;;
+
+.align 16
+memset_non_0_pattern:
+ /* Preserve return value */
+ copyd $r3 = $r0
+ /* Replicate the first pattern byte on all bytes */
+ sbmm8 $r32 = $r1, REPLICATE_BYTE_MASK
+ /* Check if length < MIN_SIZE_FOR_ALIGN */
+ compd.geu $r7 = $r2, MIN_SIZE_FOR_ALIGN
+ /* Invert address to compute size to copy to be aligned on 32 bytes */
+ negd $r5 = $r0
+ ;;
+ /* Check if we are aligned on 32 bytes */
+ andw $r9 = $r0, 0x1F
+ /* Compute the size that will be copied to align on 32 bytes boundary */
+ andw $r6 = $r5, 0x1F
+ /*
+ * If size < MIN_SIZE_FOR_ALIGN bits, directly go to so, it will be done
+ * unaligned but that is still better that what we can do with sb
+ */
+ cb.deqz $r7? .Laligned_32
+ ;;
+ /* Remove unaligned part from length */
+ sbfd $r2 = $r6, $r2
+ /* If we are already aligned on 32 bytes, jump to main "so" loop */
+ cb.deqz $r9? .Laligned_32
+ /* Check if we need to copy 1 byte */
+ andw $r4 = $r5, (1 << 0)
+ ;;
+ /* If we are not aligned, store byte */
+ sb.dnez $r4? [$r0] = $r32
+ /* Check if we need to copy 2 bytes */
+ andw $r4 = $r5, (1 << 1)
+ /* Add potentially copied part for next store offset */
+ addd $r0 = $r0, $r4
+ ;;
+ sh.dnez $r4? [$r0] = $r32
+ /* Check if we need to copy 4 bytes */
+ andw $r4 = $r5, (1 << 2)
+ addd $r0 = $r0, $r4
+ ;;
+ sw.dnez $r4? [$r0] = $r32
+ /* Check if we need to copy 8 bytes */
+ andw $r4 = $r5, (1 << 3)
+ addd $r0 = $r0, $r4
+ /* Copy second part of pattern for sq */
+ copyd $r33 = $r32
+ ;;
+ sd.dnez $r4? [$r0] = $r32
+ /* Check if we need to copy 16 bytes */
+ andw $r4 = $r5, (1 << 4)
+ addd $r0 = $r0, $r4
+ ;;
+ sq.dnez $r4? [$r0] = $r32r33
+ addd $r0 = $r0, $r4
+ ;;
+.Laligned_32:
+ /* Copy second part of pattern for sq */
+ copyd $r33 = $r32
+ /* Prepare amount of data for 32 bytes store */
+ srld $r10 = $r2, 5
+ nop
+ nop
+ ;;
+ copyq $r34r35 = $r32, $r33
+ /* Remaining bytes for 16 bytes store */
+ andw $r8 = $r2, (1 << 4)
+ make $r11 = 32
+ /* Check if there are enough data for 32 bytes store */
+ cb.deqz $r10? .Laligned_32_done
+ ;;
+ loopdo $r10, .Laligned_32_done
+ ;;
+ so 0[$r0] = $r32r33r34r35
+ addd $r0 = $r0, $r11
+ ;;
+ .Laligned_32_done:
+ /*
+ * Now that we have handled every aligned bytes using 'so', we can
+ * handled the remainder of length using store by decrementing size
+ * We also exploit the fact we are aligned to simply check remaining
+ * size */
+ sq.dnez $r8? [$r0] = $r32r33
+ addd $r0 = $r0, $r8
+ /* Remaining bytes for 8 bytes store */
+ andw $r8 = $r2, (1 << 3)
+ cb.deqz $r2? .Lmemset_done
+ ;;
+ sd.dnez $r8? [$r0] = $r32
+ addd $r0 = $r0, $r8
+ /* Remaining bytes for 4 bytes store */
+ andw $r8 = $r2, (1 << 2)
+ ;;
+ sw.dnez $r8? [$r0] = $r32
+ addd $r0 = $r0, $r8
+ /* Remaining bytes for 2 bytes store */
+ andw $r8 = $r2, (1 << 1)
+ ;;
+ sh.dnez $r8? [$r0] = $r32
+ addd $r0 = $r0, $r8
+ ;;
+ sb.odd $r2? [$r0] = $r32
+ /* Restore original value */
+ copyd $r0 = $r3
+ ret
+ ;;
+.Lmemset_done:
+ /* Restore original value */
+ copyd $r0 = $r3
+ ret
+ ;;
+ENDPROC(memset)
diff --git a/arch/kvx/lib/strlen.S b/arch/kvx/lib/strlen.S
new file mode 100644
index 000000000000..8298402a7898
--- /dev/null
+++ b/arch/kvx/lib/strlen.S
@@ -0,0 +1,122 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Jules Maselbas
+ */
+#include <linux/linkage.h>
+#include <asm/export.h>
+
+/*
+ * kvx optimized strlen
+ *
+ * This implementation of strlen only does aligned memory accesses.
+ * Since we don't know the total length the idea is to do double word
+ * load and stop on the first null byte found. As it's always safe to
+ * read more up to a lower 8-bytes boundary.
+ *
+ * This implementation of strlen uses a trick to detect if a double
+ * word contains a null byte [1]:
+ *
+ * > #define haszero(v) (((v) - 0x01010101UL) & ~(v) & 0x80808080UL)
+ * > The sub-expression (v - 0x01010101UL), evaluates to a high bit set
+ * > in any byte whenever the corresponding byte in v is zero or greater
+ * > than 0x80. The sub-expression ~v & 0x80808080UL evaluates to high
+ * > bits set in bytes where the byte of v doesn't have its high bit set
+ * > (so the byte was less than 0x80). Finally, by ANDing these two sub-
+ * > expressions the result is the high bits set where the bytes in v
+ * > were zero, since the high bits set due to a value greater than 0x80
+ * > in the first sub-expression are masked off by the second.
+ *
+ * [1] http://graphics.stanford.edu/~seander/bithacks.html#ZeroInWord
+ *
+ * A second trick is used to get the exact number of characters before
+ * the first null byte in a double word:
+ *
+ * clz(sbmmt(zero, 0x0102040810204080))
+ *
+ * This trick uses the haszero result which maps null byte to 0x80 and
+ * others value to 0x00. The idea is to count the number of consecutive
+ * null byte in the double word (counting from less significant byte
+ * to most significant byte). To do so, using the bit matrix transpose
+ * will "pack" all high bit (0x80) to the most significant byte (MSB).
+ * It is not possible to count the trailing zeros in this MSB, however
+ * if a byte swap is done before the bit matrix transpose we still have
+ * all the information in the MSB but now we can count the leading zeros.
+ * The instruction sbmmt with the matrix 0x0102040810204080 does exactly
+ * what we need a byte swap followed by a bit transpose.
+ *
+ * A last trick is used to handle the first double word misalignment.
+ * This is done by masking off the N lower bytes (excess read) with N
+ * between 0 and 7. The mask is applied on haszero results and will
+ * force the N lower bytes to be considered not null.
+ *
+ * This is a C implementation of the algorithm described above:
+ *
+ * size_t strlen(char *s) {
+ * uint64_t *p = (uint64_t *)((uintptr_t)s) & ~0x7;
+ * uint64_t rem = ((uintptr_t)s) % 8;
+ * uint64_t low = -0x0101010101010101;
+ * uint64_t high = 0x8080808080808080;
+ * uint64_t dword, zero;
+ * uint64_t msk, len;
+ *
+ * dword = *p++;
+ * zero = (dword + low) & ~dword & high;
+ * msk = 0xffffffffffffffff << (rem * 8);
+ * zero &= msk;
+ *
+ * while (!zero) {
+ * dword = *p++;
+ * zero = (dword + low) & ~dword & high;
+ * }
+ *
+ * zero = __builtin_kvx_sbmmt8(zero, 0x0102040810204080);
+ * len = ((void *)p - (void *)s) - 8;
+ * len += __builtin_kvx_clzd(zero);
+ *
+ * return len;
+ * }
+ */
+
+.text
+.align 16
+ENTRY(strlen)
+ andd $r1 = $r0, ~0x7
+ andd $r2 = $r0, 0x7
+ make $r10 = -0x0101010101010101
+ make $r11 = 0x8080808080808080
+ ;;
+ ld $r4 = 0[$r1]
+ sllw $r2 = $r2, 3
+ make $r3 = 0xffffffffffffffff
+ ;;
+ slld $r2 = $r3, $r2
+ addd $r5 = $r4, $r10
+ andnd $r6 = $r4, $r11
+ ;;
+ andd $r6 = $r6, $r2
+ make $r3 = 0
+ ;;
+.loop:
+ andd $r4 = $r5, $r6
+ addd $r1 = $r1, 0x8
+ ;;
+ cb.dnez $r4? .end
+ ld.deqz $r4? $r4 = [$r1]
+ ;;
+ addd $r5 = $r4, $r10
+ andnd $r6 = $r4, $r11
+ goto .loop
+ ;;
+.end:
+ addd $r1 = $r1, -0x8
+ sbmmt8 $r4 = $r4, 0x0102040810204080
+ ;;
+ clzd $r4 = $r4
+ sbfd $r1 = $r0, $r1
+ ;;
+ addd $r0 = $r4, $r1
+ ret
+ ;;
+ENDPROC(strlen)
+EXPORT_SYMBOL(strlen)
diff --git a/arch/kvx/lib/usercopy.S b/arch/kvx/lib/usercopy.S
new file mode 100644
index 000000000000..bc7e1a45e1c7
--- /dev/null
+++ b/arch/kvx/lib/usercopy.S
@@ -0,0 +1,90 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+#include <linux/linkage.h>
+
+/**
+ * Copy from/to a user buffer
+ * r0 = to buffer
+ * r1 = from buffer
+ * r2 = size to copy
+ * This function can trapped when hitting a non-mapped page.
+ * It will trigger a trap NOMAPPING and the trap handler will interpret
+ * it and check if instruction pointer is inside __ex_table.
+ * The next step are described later !
+ */
+.text
+ENTRY(raw_copy_from_user)
+ENTRY(raw_copy_to_user)
+ /**
+ * naive implementation byte per byte
+ */
+ make $r33 = 0x0;
+ /* If size == 0, exit directly */
+ cb.deqz $r2? copy_exit
+ ;;
+ loopdo $r2, copy_exit
+ ;;
+0: lbz $r34 = $r33[$r1]
+ ;;
+1: sb $r33[$r0] = $r34
+ addd $r33 = $r33, 1 /* Ptr increment */
+ addd $r2 = $r2, -1 /* Remaining bytes to copy */
+ ;;
+ copy_exit:
+ copyd $r0 = $r2
+ ret
+ ;;
+ENDPROC(raw_copy_to_user)
+ENDPROC(raw_copy_from_user)
+
+/**
+ * Exception table
+ * each entry correspond to the following:
+ * .dword trapping_addr, restore_addr
+ *
+ * On trap, the handler will try to locate if $spc is matching a
+ * trapping address in the exception table. If so, the restore addr
+ * will be put in the return address of the trap handler, allowing
+ * to properly finish the copy and return only the bytes copied/cleared
+ */
+.pushsection __ex_table,"a"
+.balign 8
+.dword 0b, copy_exit
+.dword 1b, copy_exit
+.popsection
+
+/**
+ * Clear a user buffer
+ * r0 = buffer to clear
+ * r1 = size to clear
+ */
+.text
+ENTRY(asm_clear_user)
+ /**
+ * naive implementation byte per byte
+ */
+ make $r33 = 0x0;
+ make $r34 = 0x0;
+ /* If size == 0, exit directly */
+ cb.deqz $r1? clear_exit
+ ;;
+ loopdo $r1, clear_exit
+ ;;
+40: sb $r33[$r0] = $r34
+ addd $r33 = $r33, 1 /* Ptr increment */
+ addd $r1 = $r1, -1 /* Remaining bytes to copy */
+ ;;
+ clear_exit:
+ copyd $r0 = $r1
+ ret
+ ;;
+ENDPROC(asm_clear_user)
+
+.pushsection __ex_table,"a"
+.balign 8
+.dword 40b, clear_exit
+.popsection
+
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 16/25] kvx: Add some library functions
2023-01-03 16:43 ` [RFC PATCH 16/25] kvx: Add some library functions Yann Sionneau
@ 2023-01-05 13:05 ` Clément Léger
0 siblings, 0 replies; 113+ messages in thread
From: Clément Léger @ 2023-01-05 13:05 UTC (permalink / raw)
To: Yann Sionneau; +Cc: linux-kernel, Jules Maselbas, Julian Vetter, Marius Gligor
Le Tue, 3 Jan 2023 17:43:50 +0100,
Yann Sionneau <ysionneau@kalray.eu> a écrit :
> Add some library functions for kvx, including: delay, memset,
> memcpy, strlen, clear_page, copy_page, raw_copy_from/to_user,
> asm_clear_user.
>
> CC: linux-kernel@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
> Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Co-developed-by: Marius Gligor <mgligor@kalray.eu>
> Signed-off-by: Marius Gligor <mgligor@kalray.eu>
> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> arch/kvx/include/asm/string.h | 20 ++
> arch/kvx/kernel/kvx_ksyms.c | 5 +
> arch/kvx/lib/clear_page.S | 40 ++++
> arch/kvx/lib/copy_page.S | 90 +++++++++
> arch/kvx/lib/delay.c | 39 ++++
> arch/kvx/lib/memcpy.c | 70 +++++++
> arch/kvx/lib/memset.S | 351 ++++++++++++++++++++++++++++++++++
> arch/kvx/lib/strlen.S | 122 ++++++++++++
> arch/kvx/lib/usercopy.S | 90 +++++++++
> 9 files changed, 827 insertions(+)
> create mode 100644 arch/kvx/include/asm/string.h
> create mode 100644 arch/kvx/lib/clear_page.S
> create mode 100644 arch/kvx/lib/copy_page.S
> create mode 100644 arch/kvx/lib/delay.c
> create mode 100644 arch/kvx/lib/memcpy.c
> create mode 100644 arch/kvx/lib/memset.S
> create mode 100644 arch/kvx/lib/strlen.S
> create mode 100644 arch/kvx/lib/usercopy.S
>
> diff --git a/arch/kvx/include/asm/string.h b/arch/kvx/include/asm/string.h
> new file mode 100644
> index 000000000000..677c1393a5cd
> --- /dev/null
> +++ b/arch/kvx/include/asm/string.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + * Jules Maselbas
> + */
> +
> +#ifndef _ASM_KVX_STRING_H
> +#define _ASM_KVX_STRING_H
> +
> +#define __HAVE_ARCH_MEMSET
> +extern void *memset(void *s, int c, size_t n);
> +
> +#define __HAVE_ARCH_MEMCPY
> +extern void *memcpy(void *dest, const void *src, size_t n);
> +
> +#define __HAVE_ARCH_STRLEN
> +extern size_t strlen(const char *s);
> +
> +#endif /* _ASM_KVX_STRING_H */
> diff --git a/arch/kvx/kernel/kvx_ksyms.c b/arch/kvx/kernel/kvx_ksyms.c
> index 18990aaf259f..678f81716dea 100644
> --- a/arch/kvx/kernel/kvx_ksyms.c
> +++ b/arch/kvx/kernel/kvx_ksyms.c
> @@ -22,3 +22,8 @@ DECLARE_EXPORT(__umoddi3);
> DECLARE_EXPORT(__divdi3);
> DECLARE_EXPORT(__udivdi3);
> DECLARE_EXPORT(__multi3);
> +
> +DECLARE_EXPORT(clear_page);
> +DECLARE_EXPORT(copy_page);
> +DECLARE_EXPORT(memset);
> +DECLARE_EXPORT(asm_clear_user);
> diff --git a/arch/kvx/lib/clear_page.S b/arch/kvx/lib/clear_page.S
> new file mode 100644
> index 000000000000..364fe0663ca2
> --- /dev/null
> +++ b/arch/kvx/lib/clear_page.S
> @@ -0,0 +1,40 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Marius Gligor
> + * Clement Leger
> + */
> +
> +#include <linux/linkage.h>
> +#include <linux/export.h>
> +#include <linux/const.h>
> +
> +#include <asm/cache.h>
> +#include <asm/page.h>
> +
> +#define CLEAR_PAGE_LOOP_COUNT (PAGE_SIZE / 32)
> +
> +/*
> + * Clear page @dest.
> + *
> + * Parameters:
> + * r0 - dest page
> + */
> +ENTRY(clear_page)
> + make $r1 = CLEAR_PAGE_LOOP_COUNT
> + ;;
> + make $r4 = 0
> + make $r5 = 0
> + make $r6 = 0
> + make $r7 = 0
> + ;;
> +
> + loopdo $r1, clear_page_done
> + ;;
> + so 0[$r0] = $r4r5r6r7
> + addd $r0 = $r0, 32
> + ;;
> + clear_page_done:
> + ret
> + ;;
> +ENDPROC(clear_page)
> diff --git a/arch/kvx/lib/copy_page.S b/arch/kvx/lib/copy_page.S
> new file mode 100644
> index 000000000000..4bb82d1c964c
> --- /dev/null
> +++ b/arch/kvx/lib/copy_page.S
> @@ -0,0 +1,90 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#include <linux/linkage.h>
> +#include <linux/const.h>
> +
> +#include <asm/page.h>
> +
> +/* We have 8 load/store octuple (32 bytes) per hardware loop */
> +#define COPY_SIZE_PER_LOOP (32 * 8)
> +#define COPY_PAGE_LOOP_COUNT (PAGE_SIZE / COPY_SIZE_PER_LOOP)
> +
> +/*
> + * Copy a page from src to dest (both are page aligned)
> + * In order to recover from smem latency, unroll the loop to trigger multiple
> + * onfly loads and avoid waiting too much for them to return.
> + * We use 8 * 32 load even though we could use more (up to 10 loads) to simplify
> + * the handling using a single hardware loop
> + *
> + * Parameters:
> + * r0 - dest
> + * r1 - src
> + */
> +ENTRY(copy_page)
> + make $r2 = COPY_PAGE_LOOP_COUNT
> + make $r3 = 0
> + ;;
> + loopdo $r2, copy_page_done
> + ;;
> + /*
> + * Load 8 * 32 bytes using uncached access to avoid hitting
> + * the cache
> + */
> + lo.xs $r32r33r34r35 = $r3[$r1]
> + /* Copy current copy index for store */
> + copyd $r2 = $r3
> + addd $r3 = $r3, 1
> + ;;
> + lo.xs $r36r37r38r39 = $r3[$r1]
> + addd $r3 = $r3, 1
> + ;;
> + lo.xs $r40r41r42r43 = $r3[$r1]
> + addd $r3 = $r3, 1
> + ;;
> + lo.xs $r44r45r46r47 = $r3[$r1]
> + addd $r3 = $r3, 1
> + ;;
> + lo.xs $r48r49r50r51 = $r3[$r1]
> + addd $r3 = $r3, 1
> + ;;
> + lo.xs $r52r53r54r55 = $r3[$r1]
> + addd $r3 = $r3, 1
> + ;;
> + lo.xs $r56r57r58r59 = $r3[$r1]
> + addd $r3 = $r3, 1
> + ;;
> + lo.xs $r60r61r62r63 = $r3[$r1]
> + addd $r3 = $r3, 1
> + ;;
> + /* And then store all of them */
> + so.xs $r2[$r0] = $r32r33r34r35
> + addd $r2 = $r2, 1
> + ;;
> + so.xs $r2[$r0] = $r36r37r38r39
> + addd $r2 = $r2, 1
> + ;;
> + so.xs $r2[$r0] = $r40r41r42r43
> + addd $r2 = $r2, 1
> + ;;
> + so.xs $r2[$r0] = $r44r45r46r47
> + addd $r2 = $r2, 1
> + ;;
> + so.xs $r2[$r0] = $r48r49r50r51
> + addd $r2 = $r2, 1
> + ;;
> + so.xs $r2[$r0] = $r52r53r54r55
> + addd $r2 = $r2, 1
> + ;;
> + so.xs $r2[$r0] = $r56r57r58r59
> + addd $r2 = $r2, 1
> + ;;
> + so.xs $r2[$r0] = $r60r61r62r63
> + ;;
> + copy_page_done:
> + ret
> + ;;
> +ENDPROC(copy_page)
> diff --git a/arch/kvx/lib/delay.c b/arch/kvx/lib/delay.c
> new file mode 100644
> index 000000000000..11295eedc3f5
> --- /dev/null
> +++ b/arch/kvx/lib/delay.c
> @@ -0,0 +1,39 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#include <linux/export.h>
> +#include <linux/delay.h>
> +
> +#include <asm/param.h>
> +#include <asm/timex.h>
> +
> +void __delay(unsigned long loops)
> +{
> + cycles_t target_cycle = get_cycles() + loops;
> +
> + while (get_cycles() < target_cycle);
> +}
> +EXPORT_SYMBOL(__delay);
> +
> +inline void __const_udelay(unsigned long xloops)
> +{
> + u64 loops = (u64)xloops * (u64)loops_per_jiffy * HZ;
> +
> + __delay(loops >> 32);
> +}
> +EXPORT_SYMBOL(__const_udelay);
> +
> +void __udelay(unsigned long usecs)
> +{
> + __const_udelay(usecs * 0x10C7UL); /* 2**32 / 1000000 (rounded up) */
> +}
> +EXPORT_SYMBOL(__udelay);
This could probably be factorized with arm64, csky, nios2, etc.
> +
> +void __ndelay(unsigned long nsecs)
> +{
> + __const_udelay(nsecs * 0x5UL); /* 2**32 / 1000000000 (rounded up) */
> +}
> +EXPORT_SYMBOL(__ndelay);
Same goes for __ndelay I think
> diff --git a/arch/kvx/lib/memcpy.c b/arch/kvx/lib/memcpy.c
> new file mode 100644
> index 000000000000..b81f746a80ee
> --- /dev/null
> +++ b/arch/kvx/lib/memcpy.c
> @@ -0,0 +1,70 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + * Yann Sionneau
> + */
> +
> +#include <linux/export.h>
> +#include <linux/types.h>
> +
> +void *memcpy(void *dest, const void *src, size_t n)
> +{
> + __uint128_t *tmp128_d = dest;
> + const __uint128_t *tmp128_s = src;
> + uint64_t *tmp64_d;
> + const uint64_t *tmp64_s;
> + uint32_t *tmp32_d;
> + const uint32_t *tmp32_s;
> + uint16_t *tmp16_d;
> + const uint16_t *tmp16_s;
> + uint8_t *tmp8_d;
> + const uint8_t *tmp8_s;
> +
> + while (n >= 16) {
> + *tmp128_d = *tmp128_s;
> + tmp128_d++;
> + tmp128_s++;
> + n -= 16;
> + }
> +
> + tmp64_d = (uint64_t *) tmp128_d;
> + tmp64_s = (uint64_t *) tmp128_s;
> + while (n >= 8) {
> + *tmp64_d = *tmp64_s;
> + tmp64_d++;
> + tmp64_s++;
> + n -= 8;
> + }
> +
> + tmp32_d = (uint32_t *) tmp64_d;
> + tmp32_s = (uint32_t *) tmp64_s;
> + while (n >= 4) {
> + *tmp32_d = *tmp32_s;
> + tmp32_d++;
> + tmp32_s++;
> + n -= 4;
> + }
> +
> + tmp16_d = (uint16_t *) tmp32_d;
> + tmp16_s = (uint16_t *) tmp32_s;
> + while (n >= 2) {
> + *tmp16_d = *tmp16_s;
> + tmp16_d++;
> + tmp16_s++;
> + n -= 2;
> + }
> +
> + tmp8_d = (uint8_t *) tmp16_d;
> + tmp8_s = (uint8_t *) tmp16_s;
> + while (n >= 1) {
> + *tmp8_d = *tmp8_s;
> + tmp8_d++;
> + tmp8_s++;
> + n--;
> + }
> +
> + return dest;
> +}
> +EXPORT_SYMBOL(memcpy);
> +
> diff --git a/arch/kvx/lib/memset.S b/arch/kvx/lib/memset.S
> new file mode 100644
> index 000000000000..9eebc28da2be
> --- /dev/null
> +++ b/arch/kvx/lib/memset.S
> @@ -0,0 +1,351 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + * Marius Gligor
> + */
> +
> +#include <linux/linkage.h>
> +
> +#include <asm/cache.h>
> +
> +#define REPLICATE_BYTE_MASK 0x0101010101010101
> +#define MIN_SIZE_FOR_ALIGN 128
> +
> +/*
> + * Optimized memset for kvx architecture
> + *
> + * In order to optimize memset on kvx, we can use various things:
> + * - conditionnal store which avoid branch penalty
> + * - store half/word/double/quad/octuple to store up to 16 bytes at a time
> + * - dzerol to zero a cacheline when the pattern is '0' (often the case)
> + * - hardware loop for steady cases.
> + *
> + * First, we assume that memset is mainly used for zeroing areas. In order
> + * to optimize this case, we consider it to be the fast path of the algorithm.
> + * In both cases (0 and non 0 pattern), we start by checking if the size is
> + * below a minimum size. If so, we skip the alignment part. Indeed, the kvx
> + * supports misalignment and the penalty for letting it do unaligned accesses is
> + * lower than trying to realigning us. So for small sizes, we don't even bother
> + * to realign. Minor difference is that in the memset with 0, we skip after the
> + * dzerol loop since dzerol must be cache-line aligned (no misalignment of
> + * course).
> + * Regarding the non 0 pattern memset, we use sbmm to replicate the pattern on
> + * all bits on a register in one call.
> + * Once alignment has been reached, we can do the hardware loop for both cases(
> + * store octuple/dzerol) in order to optimize throughput. Care must be taken to
> + * align hardware loops on at least 8 bytes for performances.
> + * Once the main loop has been done, we finish the copy by checking length to do
> + * the necessary calls to store remaining bytes.
> + *
> + * Pseudo code (applies for non 0 pattern):
> + *
> + * int memset(void *dest, char pattern, long length)
> + * {
> + * long dest_align = -((long) dest);
> + * long copy;
> + * long orig_dest = dest;
> + *
> + * uint64_t pattern = sbmm8(pattern, 0x0101010101010101);
> + * uint128_t pattern128 = pattern << 64 | pattern;
> + * uint256_t pattern128 = pattern128 << 128 | pattern128;
> + *
> + * // Keep only low bits
> + * dest_align &= 0x1F;
> + * length -= dest_align;
> + *
> + * // Byte align
> + * copy = align & (1 << 0);
> + * if (copy)
> + * *((u8 *) dest) = pattern;
> + * dest += copy;
> + * // Half align
> + * copy = align & (1 << 1);
> + * if (copy)
> + * *((u16 *) dest) = pattern;
> + * dest += copy;
> + * // Word align
> + * copy = align & (1 << 2);
> + * if (copy)
> + * *((u32 *) dest) = pattern;
> + * dest += copy;
> + * // Double align
> + * copy = align & (1 << 3);
> + * if (copy)
> + * *((u64 *) dest) = pattern;
> + * dest += copy;
> + * // Quad align
> + * copy = align & (1 << 4);
> + * if (copy)
> + * *((u128 *) dest) = pattern128;
> + * dest += copy;
> + *
> + * // We are now aligned on 256 bits
> + * loop_octuple_count = size >> 5;
> + * for (i = 0; i < loop_octuple_count; i++) {
> + * *((u256 *) dest) = pattern256;
> + * dest += 32;
> + * }
> + *
> + * if (length == 0)
> + * return orig_dest;
> + *
> + * // Copy remaining part
> + * remain = length & (1 << 4);
> + * if (copy)
> + * *((u128 *) dest) = pattern128;
> + * dest += remain;
> + * remain = length & (1 << 3);
> + * if (copy)
> + * *((u64 *) dest) = pattern;
> + * dest += remain;
> + * remain = length & (1 << 2);
> + * if (copy)
> + * *((u32 *) dest) = pattern;
> + * dest += remain;
> + * remain = length & (1 << 1);
> + * if (copy)
> + * *((u16 *) dest) = pattern;
> + * dest += remain;
> + * remain = length & (1 << 0);
> + * if (copy)
> + * *((u8 *) dest) = pattern;
> + * dest += remain;
> + *
> + * return orig_dest;
> + * }
> + */
> +
> +.text
> +.align 16
> +ENTRY(memset):
> + make $r32 = 0
> + make $r33 = 0
> + /* Check if length < KVX_DCACHE_LINE_SIZE */
> + compd.ltu $r7 = $r2, KVX_DCACHE_LINE_SIZE
> + /* Jump to generic memset if pattern is != 0 */
> + cb.dnez $r1? memset_non_0_pattern
> + ;;
> + /* Preserve return value */
> + copyd $r3 = $r0
> + /* Invert address to compute size to copy to be aligned on 32 bytes */
> + negd $r5 = $r0
> + /* Remaining bytes for 16 bytes store (for alignment on 64 bytes) */
> + andd $r8 = $r2, (1 << 5)
> + copyq $r34r35 = $r32, $r33
> + /* Skip loopdo with dzerol if length < KVX_DCACHE_LINE_SIZE */
> + cb.dnez $r7? .Ldzerol_done
> + ;;
> + /* Compute the size that will be copied to align on 64 bytes boundary */
> + andw $r6 = $r5, 0x3F
> + /* Check if address is aligned on 64 bytes */
> + andw $r9 = $r0, 0x3F
> + /* Alignment */
> + nop
> + ;;
> + /* If address already aligned on 64 bytes, jump to dzerol loop */
> + cb.deqz $r9? .Laligned_64
> + /* Remove unaligned part from length */
> + sbfd $r2 = $r6, $r2
> + /* Check if we need to copy 1 byte */
> + andw $r4 = $r5, (1 << 0)
> + ;;
> + /* If we are not aligned, store byte */
> + sb.dnez $r4? [$r0] = $r32
> + /* Check if we need to copy 2 bytes */
> + andw $r4 = $r5, (1 << 1)
> + /* Add potentially copied part for next store offset */
> + addd $r0 = $r0, $r4
> + ;;
> + sh.dnez $r4? [$r0] = $r32
> + /* Check if we need to copy 4 bytes */
> + andw $r4 = $r5, (1 << 2)
> + addd $r0 = $r0, $r4
> + ;;
> + sw.dnez $r4? [$r0] = $r32
> + /* Check if we need to copy 8 bytes */
> + andw $r4 = $r5, (1 << 3)
> + addd $r0 = $r0, $r4
> + ;;
> + sd.dnez $r4? [$r0] = $r32
> + /* Check if we need to copy 16 bytes */
> + andw $r4 = $r5, (1 << 4)
> + addd $r0 = $r0, $r4
> + ;;
> + sq.dnez $r4? [$r0] = $r32r33
> + /* Check if we need to copy 32 bytes */
> + andw $r4 = $r5, (1 << 5)
> + addd $r0 = $r0, $r4
> + ;;
> + so.dnez $r4? [$r0] = $r32r33r34r35
> + addd $r0 = $r0, $r4
> + ;;
> +.Laligned_64:
> + /* Prepare amount of data for dzerol */
> + srld $r10 = $r2, 6
> + /* Size to be handled in loopdo */
> + andd $r4 = $r2, ~0x3F
> + make $r11 = 64
> + cb.deqz $r2? .Lmemset_done
> + ;;
> + /* Remaining bytes for 16 bytes store */
> + andw $r8 = $r2, (1 << 5)
> + /* Skip dzerol if there are not enough data for 64 bytes store */
> + cb.deqz $r10? .Ldzerol_done
> + /* Update length to copy */
> + sbfd $r2 = $r4, $r2
> + ;;
> + loopdo $r10, .Ldzerol_done
> + ;;
> + so 0[$r0], $r32r33r34r35
> + ;;
> + so 32[$r0], $r32r33r34r35
> + addd $r0 = $r0, $r11
> + ;;
> + .Ldzerol_done:
> + /*
> + * Now that we have handled every aligned bytes using 'dzerol', we can
> + * handled the remainder of length using store by decrementing size
> + * We also exploit the fact we are aligned to simply check remaining
> + * size */
> + so.dnez $r8? [$r0] = $r32r33r34r35
> + addd $r0 = $r0, $r8
> + /* Remaining bytes for 16 bytes store */
> + andw $r8 = $r2, (1 << 4)
> + cb.deqz $r2? .Lmemset_done
> + ;;
> + sq.dnez $r8? [$r0] = $r32r33
> + addd $r0 = $r0, $r8
> + /* Remaining bytes for 8 bytes store */
> + andw $r8 = $r2, (1 << 3)
> + ;;
> + sd.dnez $r8? [$r0] = $r32
> + addd $r0 = $r0, $r8
> + /* Remaining bytes for 4 bytes store */
> + andw $r8 = $r2, (1 << 2)
> + ;;
> + sw.dnez $r8? [$r0] = $r32
> + addd $r0 = $r0, $r8
> + /* Remaining bytes for 2 bytes store */
> + andw $r8 = $r2, (1 << 1)
> + ;;
> + sh.dnez $r8? [$r0] = $r32
> + addd $r0 = $r0, $r8
> + ;;
> + sb.odd $r2? [$r0] = $r32
> + /* Restore original value */
> + copyd $r0 = $r3
> + ret
> + ;;
> +
> +.align 16
> +memset_non_0_pattern:
> + /* Preserve return value */
> + copyd $r3 = $r0
> + /* Replicate the first pattern byte on all bytes */
> + sbmm8 $r32 = $r1, REPLICATE_BYTE_MASK
> + /* Check if length < MIN_SIZE_FOR_ALIGN */
> + compd.geu $r7 = $r2, MIN_SIZE_FOR_ALIGN
> + /* Invert address to compute size to copy to be aligned on 32 bytes */
> + negd $r5 = $r0
> + ;;
> + /* Check if we are aligned on 32 bytes */
> + andw $r9 = $r0, 0x1F
> + /* Compute the size that will be copied to align on 32 bytes boundary */
> + andw $r6 = $r5, 0x1F
> + /*
> + * If size < MIN_SIZE_FOR_ALIGN bits, directly go to so, it will be done
> + * unaligned but that is still better that what we can do with sb
> + */
> + cb.deqz $r7? .Laligned_32
> + ;;
> + /* Remove unaligned part from length */
> + sbfd $r2 = $r6, $r2
> + /* If we are already aligned on 32 bytes, jump to main "so" loop */
> + cb.deqz $r9? .Laligned_32
> + /* Check if we need to copy 1 byte */
> + andw $r4 = $r5, (1 << 0)
> + ;;
> + /* If we are not aligned, store byte */
> + sb.dnez $r4? [$r0] = $r32
> + /* Check if we need to copy 2 bytes */
> + andw $r4 = $r5, (1 << 1)
> + /* Add potentially copied part for next store offset */
> + addd $r0 = $r0, $r4
> + ;;
> + sh.dnez $r4? [$r0] = $r32
> + /* Check if we need to copy 4 bytes */
> + andw $r4 = $r5, (1 << 2)
> + addd $r0 = $r0, $r4
> + ;;
> + sw.dnez $r4? [$r0] = $r32
> + /* Check if we need to copy 8 bytes */
> + andw $r4 = $r5, (1 << 3)
> + addd $r0 = $r0, $r4
> + /* Copy second part of pattern for sq */
> + copyd $r33 = $r32
> + ;;
> + sd.dnez $r4? [$r0] = $r32
> + /* Check if we need to copy 16 bytes */
> + andw $r4 = $r5, (1 << 4)
> + addd $r0 = $r0, $r4
> + ;;
> + sq.dnez $r4? [$r0] = $r32r33
> + addd $r0 = $r0, $r4
> + ;;
> +.Laligned_32:
> + /* Copy second part of pattern for sq */
> + copyd $r33 = $r32
> + /* Prepare amount of data for 32 bytes store */
> + srld $r10 = $r2, 5
> + nop
> + nop
> + ;;
> + copyq $r34r35 = $r32, $r33
> + /* Remaining bytes for 16 bytes store */
> + andw $r8 = $r2, (1 << 4)
> + make $r11 = 32
> + /* Check if there are enough data for 32 bytes store */
> + cb.deqz $r10? .Laligned_32_done
> + ;;
> + loopdo $r10, .Laligned_32_done
> + ;;
> + so 0[$r0] = $r32r33r34r35
> + addd $r0 = $r0, $r11
> + ;;
> + .Laligned_32_done:
> + /*
> + * Now that we have handled every aligned bytes using 'so', we can
> + * handled the remainder of length using store by decrementing size
> + * We also exploit the fact we are aligned to simply check remaining
> + * size */
> + sq.dnez $r8? [$r0] = $r32r33
> + addd $r0 = $r0, $r8
> + /* Remaining bytes for 8 bytes store */
> + andw $r8 = $r2, (1 << 3)
> + cb.deqz $r2? .Lmemset_done
> + ;;
> + sd.dnez $r8? [$r0] = $r32
> + addd $r0 = $r0, $r8
> + /* Remaining bytes for 4 bytes store */
> + andw $r8 = $r2, (1 << 2)
> + ;;
> + sw.dnez $r8? [$r0] = $r32
> + addd $r0 = $r0, $r8
> + /* Remaining bytes for 2 bytes store */
> + andw $r8 = $r2, (1 << 1)
> + ;;
> + sh.dnez $r8? [$r0] = $r32
> + addd $r0 = $r0, $r8
> + ;;
> + sb.odd $r2? [$r0] = $r32
> + /* Restore original value */
> + copyd $r0 = $r3
> + ret
> + ;;
> +.Lmemset_done:
> + /* Restore original value */
> + copyd $r0 = $r3
> + ret
> + ;;
> +ENDPROC(memset)
> diff --git a/arch/kvx/lib/strlen.S b/arch/kvx/lib/strlen.S
> new file mode 100644
> index 000000000000..8298402a7898
> --- /dev/null
> +++ b/arch/kvx/lib/strlen.S
> @@ -0,0 +1,122 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Jules Maselbas
> + */
> +#include <linux/linkage.h>
> +#include <asm/export.h>
> +
> +/*
> + * kvx optimized strlen
> + *
> + * This implementation of strlen only does aligned memory accesses.
> + * Since we don't know the total length the idea is to do double word
> + * load and stop on the first null byte found. As it's always safe to
> + * read more up to a lower 8-bytes boundary.
> + *
> + * This implementation of strlen uses a trick to detect if a double
> + * word contains a null byte [1]:
> + *
> + * > #define haszero(v) (((v) - 0x01010101UL) & ~(v) & 0x80808080UL)
> + * > The sub-expression (v - 0x01010101UL), evaluates to a high bit set
> + * > in any byte whenever the corresponding byte in v is zero or greater
> + * > than 0x80. The sub-expression ~v & 0x80808080UL evaluates to high
> + * > bits set in bytes where the byte of v doesn't have its high bit set
> + * > (so the byte was less than 0x80). Finally, by ANDing these two sub-
> + * > expressions the result is the high bits set where the bytes in v
> + * > were zero, since the high bits set due to a value greater than 0x80
> + * > in the first sub-expression are masked off by the second.
> + *
> + * [1] http://graphics.stanford.edu/~seander/bithacks.html#ZeroInWord
> + *
> + * A second trick is used to get the exact number of characters before
> + * the first null byte in a double word:
> + *
> + * clz(sbmmt(zero, 0x0102040810204080))
> + *
> + * This trick uses the haszero result which maps null byte to 0x80 and
> + * others value to 0x00. The idea is to count the number of consecutive
> + * null byte in the double word (counting from less significant byte
> + * to most significant byte). To do so, using the bit matrix transpose
> + * will "pack" all high bit (0x80) to the most significant byte (MSB).
> + * It is not possible to count the trailing zeros in this MSB, however
> + * if a byte swap is done before the bit matrix transpose we still have
> + * all the information in the MSB but now we can count the leading zeros.
> + * The instruction sbmmt with the matrix 0x0102040810204080 does exactly
> + * what we need a byte swap followed by a bit transpose.
> + *
> + * A last trick is used to handle the first double word misalignment.
> + * This is done by masking off the N lower bytes (excess read) with N
> + * between 0 and 7. The mask is applied on haszero results and will
> + * force the N lower bytes to be considered not null.
> + *
> + * This is a C implementation of the algorithm described above:
> + *
> + * size_t strlen(char *s) {
> + * uint64_t *p = (uint64_t *)((uintptr_t)s) & ~0x7;
> + * uint64_t rem = ((uintptr_t)s) % 8;
> + * uint64_t low = -0x0101010101010101;
> + * uint64_t high = 0x8080808080808080;
> + * uint64_t dword, zero;
> + * uint64_t msk, len;
> + *
> + * dword = *p++;
> + * zero = (dword + low) & ~dword & high;
> + * msk = 0xffffffffffffffff << (rem * 8);
> + * zero &= msk;
> + *
> + * while (!zero) {
> + * dword = *p++;
> + * zero = (dword + low) & ~dword & high;
> + * }
> + *
> + * zero = __builtin_kvx_sbmmt8(zero, 0x0102040810204080);
> + * len = ((void *)p - (void *)s) - 8;
> + * len += __builtin_kvx_clzd(zero);
> + *
> + * return len;
> + * }
> + */
> +
> +.text
> +.align 16
> +ENTRY(strlen)
> + andd $r1 = $r0, ~0x7
> + andd $r2 = $r0, 0x7
> + make $r10 = -0x0101010101010101
> + make $r11 = 0x8080808080808080
> + ;;
> + ld $r4 = 0[$r1]
> + sllw $r2 = $r2, 3
> + make $r3 = 0xffffffffffffffff
> + ;;
> + slld $r2 = $r3, $r2
> + addd $r5 = $r4, $r10
> + andnd $r6 = $r4, $r11
> + ;;
> + andd $r6 = $r6, $r2
> + make $r3 = 0
> + ;;
> +.loop:
> + andd $r4 = $r5, $r6
> + addd $r1 = $r1, 0x8
> + ;;
> + cb.dnez $r4? .end
> + ld.deqz $r4? $r4 = [$r1]
> + ;;
> + addd $r5 = $r4, $r10
> + andnd $r6 = $r4, $r11
> + goto .loop
> + ;;
> +.end:
> + addd $r1 = $r1, -0x8
> + sbmmt8 $r4 = $r4, 0x0102040810204080
> + ;;
> + clzd $r4 = $r4
> + sbfd $r1 = $r0, $r1
> + ;;
> + addd $r0 = $r4, $r1
> + ret
> + ;;
> +ENDPROC(strlen)
> +EXPORT_SYMBOL(strlen)
> diff --git a/arch/kvx/lib/usercopy.S b/arch/kvx/lib/usercopy.S
> new file mode 100644
> index 000000000000..bc7e1a45e1c7
> --- /dev/null
> +++ b/arch/kvx/lib/usercopy.S
> @@ -0,0 +1,90 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +#include <linux/linkage.h>
> +
> +/**
> + * Copy from/to a user buffer
> + * r0 = to buffer
> + * r1 = from buffer
> + * r2 = size to copy
> + * This function can trapped when hitting a non-mapped page.
> + * It will trigger a trap NOMAPPING and the trap handler will interpret
> + * it and check if instruction pointer is inside __ex_table.
> + * The next step are described later !
> + */
> +.text
> +ENTRY(raw_copy_from_user)
> +ENTRY(raw_copy_to_user)
> + /**
> + * naive implementation byte per byte
> + */
> + make $r33 = 0x0;
> + /* If size == 0, exit directly */
> + cb.deqz $r2? copy_exit
> + ;;
> + loopdo $r2, copy_exit
> + ;;
> +0: lbz $r34 = $r33[$r1]
> + ;;
> +1: sb $r33[$r0] = $r34
> + addd $r33 = $r33, 1 /* Ptr increment */
> + addd $r2 = $r2, -1 /* Remaining bytes to copy */
> + ;;
> + copy_exit:
> + copyd $r0 = $r2
> + ret
> + ;;
> +ENDPROC(raw_copy_to_user)
> +ENDPROC(raw_copy_from_user)
> +
> +/**
> + * Exception table
> + * each entry correspond to the following:
> + * .dword trapping_addr, restore_addr
> + *
> + * On trap, the handler will try to locate if $spc is matching a
> + * trapping address in the exception table. If so, the restore addr
> + * will be put in the return address of the trap handler, allowing
> + * to properly finish the copy and return only the bytes copied/cleared
> + */
> +.pushsection __ex_table,"a"
> +.balign 8
> +.dword 0b, copy_exit
> +.dword 1b, copy_exit
> +.popsection
> +
> +/**
> + * Clear a user buffer
> + * r0 = buffer to clear
> + * r1 = size to clear
> + */
> +.text
> +ENTRY(asm_clear_user)
> + /**
> + * naive implementation byte per byte
> + */
> + make $r33 = 0x0;
> + make $r34 = 0x0;
> + /* If size == 0, exit directly */
> + cb.deqz $r1? clear_exit
> + ;;
> + loopdo $r1, clear_exit
> + ;;
> +40: sb $r33[$r0] = $r34
> + addd $r33 = $r33, 1 /* Ptr increment */
> + addd $r1 = $r1, -1 /* Remaining bytes to copy */
> + ;;
> + clear_exit:
> + copyd $r0 = $r1
> + ret
> + ;;
> +ENDPROC(asm_clear_user)
> +
> +.pushsection __ex_table,"a"
> +.balign 8
> +.dword 40b, clear_exit
> +.popsection
> +
--
Clément Léger,
Embedded Linux and Kernel engineer at Bootlin
https://bootlin.com
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 17/25] kvx: Add multi-processor (SMP) support
2023-01-03 16:43 ` Yann Sionneau
` (17 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-03 21:22 ` Rob Herring
2023-01-05 8:12 ` Clément Léger
-1 siblings, 2 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, Clement Leger, Julian Vetter,
Julien Hascoet, Louis Morhet, Luc Michel, Marius Gligor
Coolidge v1 SoC has 5 clusters of 16 kvx cores
(+1 special Resource Manager (RM) core).
Linux can run in SMP config on the 16 cores of
a Cluster.
Cache coherency is done by the L2 cache.
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Julien Hascoet <jhascoet@kalray.eu>
Signed-off-by: Julien Hascoet <jhascoet@kalray.eu>
Co-developed-by: Louis Morhet <lmorhet@kalray.eu>
Signed-off-by: Louis Morhet <lmorhet@kalray.eu>
Co-developed-by: Luc Michel <lmichel@kalray.eu>
Signed-off-by: Luc Michel <lmichel@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/pwr_ctrl.h | 45 ++++
arch/kvx/include/asm/smp.h | 42 +++
arch/kvx/kernel/l2_cache.c | 448 ++++++++++++++++++++++++++++++++
arch/kvx/kernel/smp.c | 110 ++++++++
arch/kvx/kernel/smpboot.c | 127 +++++++++
arch/kvx/platform/pwr_ctrl.c | 93 +++++++
include/linux/cpuhotplug.h | 2 +
7 files changed, 867 insertions(+)
create mode 100644 arch/kvx/include/asm/pwr_ctrl.h
create mode 100644 arch/kvx/include/asm/smp.h
create mode 100644 arch/kvx/kernel/l2_cache.c
create mode 100644 arch/kvx/kernel/smp.c
create mode 100644 arch/kvx/kernel/smpboot.c
create mode 100644 arch/kvx/platform/pwr_ctrl.c
diff --git a/arch/kvx/include/asm/pwr_ctrl.h b/arch/kvx/include/asm/pwr_ctrl.h
new file mode 100644
index 000000000000..25f403ba935a
--- /dev/null
+++ b/arch/kvx/include/asm/pwr_ctrl.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Marius Gligor
+ */
+
+#ifndef _ASM_KVX_PWR_CTRL_H
+#define _ASM_KVX_PWR_CTRL_H
+
+#ifndef __ASSEMBLY__
+
+int kvx_pwr_ctrl_probe(void);
+
+void kvx_pwr_ctrl_cpu_poweron(unsigned int cpu);
+
+#endif
+
+/* Power controller vector register definitions */
+#define KVX_PWR_CTRL_VEC_OFFSET 0x1000
+#define KVX_PWR_CTRL_VEC_WUP_SET_OFFSET 0x10
+#define KVX_PWR_CTRL_VEC_WUP_CLEAR_OFFSET 0x20
+
+/* Power controller PE reset PC register definitions */
+#define KVX_PWR_CTRL_RESET_PC_OFFSET 0x2000
+
+/* Power controller global register definitions */
+#define KVX_PWR_CTRL_GLOBAL_OFFSET 0x4040
+
+#define KVX_PWR_CTRL_GLOBAL_SET_OFFSET 0x10
+#define KVX_PWR_CTRL_GLOBAL_SET_PE_EN_SHIFT 0x1
+
+#define PWR_CTRL_WUP_SET_OFFSET \
+ (KVX_PWR_CTRL_VEC_OFFSET + \
+ KVX_PWR_CTRL_VEC_WUP_SET_OFFSET)
+
+#define PWR_CTRL_WUP_CLEAR_OFFSET \
+ (KVX_PWR_CTRL_VEC_OFFSET + \
+ KVX_PWR_CTRL_VEC_WUP_CLEAR_OFFSET)
+
+#define PWR_CTRL_GLOBAL_CONFIG_OFFSET \
+ (KVX_PWR_CTRL_GLOBAL_OFFSET + \
+ KVX_PWR_CTRL_GLOBAL_SET_OFFSET)
+
+#endif /* _ASM_KVX_PWR_CTRL_H */
diff --git a/arch/kvx/include/asm/smp.h b/arch/kvx/include/asm/smp.h
new file mode 100644
index 000000000000..e4fd4d001b2c
--- /dev/null
+++ b/arch/kvx/include/asm/smp.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_SMP_H
+#define _ASM_KVX_SMP_H
+
+#include <linux/cpumask.h>
+#include <linux/irqreturn.h>
+
+#include <asm/sfr.h>
+
+#ifdef CONFIG_SMP
+
+/* Hook for the generic smp_call_function_many() routine. */
+void arch_send_call_function_ipi_mask(struct cpumask *mask);
+
+/* Hook for the generic smp_call_function_single() routine. */
+void arch_send_call_function_single_ipi(int cpu);
+
+void __init setup_processor(void);
+
+void smp_init_cpus(void);
+
+irqreturn_t ipi_call_interrupt(int irq, void *dev_id);
+
+#define raw_smp_processor_id() ((int) \
+ ((kvx_sfr_get(PCR) & KVX_SFR_PCR_PID_MASK) \
+ >> KVX_SFR_PCR_PID_SHIFT))
+
+#define flush_cache_vmap(start, end) do { } while (0)
+#define flush_cache_vunmap(start, end) do { } while (0)
+
+#else
+
+void smp_init_cpus(void) {}
+
+#endif /* CONFIG_SMP */
+
+#endif
diff --git a/arch/kvx/kernel/l2_cache.c b/arch/kvx/kernel/l2_cache.c
new file mode 100644
index 000000000000..94a4d9159471
--- /dev/null
+++ b/arch/kvx/kernel/l2_cache.c
@@ -0,0 +1,448 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Luc Michel
+ * Julien Hascoet
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/jiffies.h>
+#include <linux/of_address.h>
+#include <linux/irqchip/irq-kvx-apic-mailbox.h>
+
+#include <asm/rm_fw.h>
+#include <asm/l2_cache.h>
+#include <asm/sections.h>
+#include <asm/cacheflush.h>
+
+#define L2_START_TIMEOUT_MS 10
+#define L2_CMD_TIMEOUT_MS 200
+
+#define L2_MK_OP(__cmd, __sync) (BIT(L2_CMD_OP_VALID_SHIFT) | \
+ ((u64) __sync << L2_CMD_OP_SYNC_SHIFT) | \
+ ((u64) __cmd << L2_CMD_OP_CMD_SHIFT))
+
+#define L2_ERROR(_err) \
+ ((error & L2_ERROR_ERROR_CODE_MASK) >> L2_ERROR_ERROR_CODE_SHIFT)
+
+struct l2_cache_hw_cmd {
+ u64 op;
+ u64 args[L2_CMD_OP_ARG_COUNT];
+} __packed;
+
+struct l2_cache_cmd {
+ int sync;
+ int cmd_type;
+ unsigned int arg_count;
+ u64 args[L2_CMD_OP_ARG_COUNT];
+};
+
+/**
+ * struct l2_cached_data - Data associated to the l2-cache
+ * @regs: base of L2 registers
+ * @mbox_regs: Mailbox registers for L2 signaling
+ */
+struct l2_cache_data {
+ void __iomem *regs;
+ void __iomem *mbox_regs;
+ u64 fifo_cmd_count;
+};
+
+DEFINE_STATIC_KEY_FALSE(l2_enabled);
+static struct l2_cache_data l2c_ctrl;
+
+static void *l2_cmd_regs_addr(void)
+{
+ return l2c_ctrl.regs + L2_CMD_OFFSET;
+}
+
+static struct l2_cache_hw_cmd *l2_cache_hw_cmd_addr(u64 idx)
+{
+ void *cmd_regs = l2_cmd_regs_addr();
+
+ /* Wrap index */
+ idx &= (l2c_ctrl.fifo_cmd_count - 1);
+ return cmd_regs + L2_CMD_FIFO_OFFSET + idx * L2_CMD_FIFO_ELEM_SIZE;
+}
+
+static u64 l2_cache_get_cmd_idx(unsigned int cmd_count)
+{
+ u64 cmd_idx;
+ void *cmd_regs = l2_cmd_regs_addr();
+ u64 *write_idx_ptr = cmd_regs + L2_CMD_WRITE_IDX_OFFSET;
+ u64 *read_idx_ptr = cmd_regs + L2_CMD_READ_IDX_OFFSET;
+
+ /* Grab a commands tickets */
+ cmd_idx = __builtin_kvx_aladdd(write_idx_ptr, cmd_count);
+
+ /* Wait until there is room in command fifo to enqueue commands */
+ while ((cmd_idx + cmd_count) >=
+ (readq(read_idx_ptr) + l2c_ctrl.fifo_cmd_count))
+ cpu_relax();
+
+ return cmd_idx;
+}
+
+static void l2_wait_completion(u64 cmd_idx)
+{
+ u64 *read_idx_ptr = l2_cmd_regs_addr() + L2_CMD_READ_IDX_OFFSET;
+ unsigned long timeout = jiffies + msecs_to_jiffies(L2_CMD_TIMEOUT_MS);
+
+ /* Wait for completion */
+ while (cmd_idx >= readq(read_idx_ptr)) {
+ cpu_relax();
+ if (time_after(jiffies, timeout))
+ panic("L2 cache completion timeout\n");
+ }
+}
+
+static u64 l2_cache_push_cmds(struct l2_cache_cmd *cmds, int cmd_count)
+{
+ int i, arg;
+ u64 cmd_op;
+ struct l2_cache_hw_cmd *cmd;
+ struct l2_cache_cmd *soft_cmd;
+ u64 cmd_idx = l2_cache_get_cmd_idx(cmd_count);
+
+ for (i = 0; i < cmd_count; i++) {
+ soft_cmd = &cmds[i];
+ cmd = l2_cache_hw_cmd_addr(cmd_idx);
+ cmd_idx++;
+
+ for (arg = 0; arg < soft_cmd->arg_count; arg++)
+ writeq_relaxed(soft_cmd->args[arg], &cmd->args[arg]);
+
+ cmd_op = L2_MK_OP(soft_cmd->cmd_type, soft_cmd->sync);
+ writeq(cmd_op, &cmd->op);
+ }
+
+ return cmd_idx - 1;
+}
+
+static void l2_cache_create_line_cmd(struct l2_cache_cmd *cmd, int cmd_type,
+ int sync, u64 addr)
+{
+ cmd->cmd_type = cmd_type;
+ cmd->sync = sync;
+ cmd->arg_count = 1;
+ cmd->args[0] = addr;
+}
+
+static void l2_cache_create_area_cmd(struct l2_cache_cmd *cmd, int cmd_type,
+ int sync, u64 addr, u64 size)
+{
+ l2_cache_create_line_cmd(cmd, cmd_type, sync, addr);
+ cmd->arg_count = 2;
+ cmd->args[1] = size;
+}
+
+static void l2_cache_push_inval_cmd(phys_addr_t start,
+ unsigned long size)
+{
+ phys_addr_t end = start + size;
+ struct l2_cache_cmd cmds[3];
+ unsigned long irq_flags;
+ int cmd_count = 0;
+ u64 cmd_idx;
+
+ /*
+ * In case of invalidation, we must make sure we do not invalidate
+ * unwanted area and thus discard legit data. In case we are not aligned
+ * send a purge line command (writeback + inval) to unaligned lines
+ * (which can be the end line or the start line)
+ */
+ if (!IS_ALIGNED(end, L2_CACHE_LINE_SIZE)) {
+ end &= ~L2_CACHE_LINE_MASK;
+ l2_cache_create_line_cmd(&cmds[cmd_count],
+ L2_CMD_OP_CMD_PURGE_LINE, 1, end);
+ cmd_count++;
+ }
+
+ if (!IS_ALIGNED(start, L2_CACHE_LINE_SIZE)) {
+ start &= ~L2_CACHE_LINE_MASK;
+ /* If there is at least another line to clear */
+ if (end != start) {
+ l2_cache_create_line_cmd(&cmds[cmd_count],
+ L2_CMD_OP_CMD_PURGE_LINE, 1,
+ start);
+ cmd_count++;
+ start += L2_CACHE_LINE_SIZE;
+ }
+ }
+
+ BUG_ON(end < start);
+
+ size = (end - start);
+ if (size > 0) {
+ l2_cache_create_area_cmd(&cmds[cmd_count],
+ L2_CMD_OP_CMD_INVAL_AREA, 1, start,
+ size);
+ cmd_count++;
+ }
+
+ BUG_ON(cmd_count == 0);
+
+ local_irq_save(irq_flags);
+
+ cmd_idx = l2_cache_push_cmds(cmds, cmd_count);
+
+ /* Finally, ping the L2 cache controller */
+ writeq(1, l2c_ctrl.mbox_regs);
+
+ local_irq_restore(irq_flags);
+
+ l2_wait_completion(cmd_idx);
+}
+
+static void l2_cache_push_generic_cmd(u64 cmd_type, phys_addr_t start,
+ unsigned long size)
+{
+ unsigned long irq_flags;
+ struct l2_cache_cmd cmd;
+ u64 cmd_idx;
+
+ /* Align the start address and size on cache line */
+ size += start & (L2_CACHE_LINE_SIZE - 1);
+ size = ALIGN(size, L2_CACHE_LINE_SIZE);
+ start = ALIGN_DOWN(start, L2_CACHE_LINE_SIZE);
+
+ local_irq_save(irq_flags);
+
+ l2_cache_create_area_cmd(&cmd, cmd_type, 1, start, size);
+ cmd_idx = l2_cache_push_cmds(&cmd, 1);
+
+ /* Finally, ping the L2 cache controller */
+ writeq(1, l2c_ctrl.mbox_regs);
+
+ local_irq_restore(irq_flags);
+
+ l2_wait_completion(cmd_idx);
+}
+
+void l2_cache_push_area_cmd(u64 cmd_type, phys_addr_t start,
+ unsigned long size)
+{
+ if (WARN_ON(size == 0))
+ return;
+
+ if (cmd_type == L2_CMD_OP_CMD_INVAL_AREA)
+ l2_cache_push_inval_cmd(start, size);
+ else
+ l2_cache_push_generic_cmd(cmd_type, start, size);
+}
+
+static void __init l2_disp_error(u64 error)
+{
+ const char *err_type;
+
+ if (error & L2_ERROR_API_ERR_MASK)
+ err_type = "API";
+ else if (error & L2_ERROR_SETUP_ERR_MASK)
+ err_type = "SETUP";
+ else
+ err_type = "UNKNOWN";
+
+ pr_err("%s error: 0x%llx\n", err_type, L2_ERROR(error));
+}
+
+static int __init l2_cache_configure_mailboxes(void)
+{
+ phys_addr_t l2_mbox_addr = 0;
+ void *cmd_regs = l2_cmd_regs_addr();
+
+ /* We do not use mailbox to wait for completion, set it to 0 */
+ writeq(0, cmd_regs + L2_CMD_DOORBELL_READ_ADDR_OFFSET);
+
+ /* Read mailbox address from L2 registers */
+ l2_mbox_addr = readq(cmd_regs + L2_CMD_DOORBELL_WRITE_ADDR_OFFSET);
+
+ /* Then map the mailbox */
+ l2c_ctrl.mbox_regs = ioremap(l2_mbox_addr, PAGE_SIZE);
+ if (!l2c_ctrl.mbox_regs) {
+ pr_err("Failed to map mailbox\n");
+ return 1;
+ }
+
+ /* Lock this entry into the LTLB */
+ kvx_mmu_ltlb_add_entry((unsigned long) l2c_ctrl.mbox_regs & PAGE_MASK,
+ l2_mbox_addr & PAGE_MASK,
+ PAGE_KERNEL_DEVICE, TLB_PS_4K);
+
+ return 0;
+}
+
+static int __init l2_cache_read_queue_size(void)
+{
+ u64 inst;
+
+ /* Read command queue size */
+ inst = readq(l2c_ctrl.regs + L2_INSTANCE_OFFSET);
+ l2c_ctrl.fifo_cmd_count = (inst & L2_INSTANCE_CMD_QUEUE_SIZE_MASK)
+ >> L2_INSTANCE_CMD_QUEUE_SIZE_SHIFT;
+
+ /* Check if value is a power of two */
+ if (hweight64(l2c_ctrl.fifo_cmd_count) != 1) {
+ pr_err("Command queue size is not a power of two\n");
+ return 1;
+ }
+
+ return 0;
+}
+
+static int __init l2_cache_init_hw(void)
+{
+ unsigned long timeout = jiffies + msecs_to_jiffies(L2_START_TIMEOUT_MS);
+ unsigned long flags;
+ u64 status, error;
+ int ret;
+
+ /* Wait for L2 to be up */
+ do {
+ status = readq(l2c_ctrl.regs + L2_STATUS_OFFSET);
+ if (status & (L2_STATUS_READY_MASK | L2_STATUS_ERROR_MASK))
+ break;
+ } while (time_before(jiffies, timeout));
+
+ if (!status) {
+ pr_err("Timeout while waiting for firmware status\n");
+ return -ENODEV;
+ }
+
+ if (status & L2_STATUS_ERROR_MASK) {
+ error = readq(l2c_ctrl.regs + L2_ERROR_OFFSET);
+ l2_disp_error(error);
+ return -EINVAL;
+ }
+
+ /* Now write ack to L2 firmware */
+ writeq(status | L2_STATUS_ACK_MASK, l2c_ctrl.regs + L2_STATUS_OFFSET);
+
+ ret = l2_cache_read_queue_size();
+ if (ret)
+ return ret;
+
+ ret = l2_cache_configure_mailboxes();
+ if (ret)
+ return ret;
+
+ /* Enable the L2 atomically */
+ local_irq_save(flags);
+
+ /* Fence data accesses */
+ kvx_fence();
+ /* Purge L1 */
+ l1_inval_dcache_all();
+ l1_inval_icache_all();
+ __builtin_kvx_barrier();
+
+ local_irq_restore(flags);
+
+ /* Enable L2$ */
+ kvx_sfr_set_field(PS, L2E, 1);
+
+ return 0;
+}
+
+static phys_addr_t __init l2_get_regs_addr(struct device_node *np)
+{
+ const void *reg;
+ struct resource res;
+ phys_addr_t l2_regs_addr;
+ int ret;
+
+ /*
+ * If regs is specified in device tree, then the L2$ has been loaded by
+ * someone else and not by ourself.
+ */
+ reg = of_get_property(np, "reg", NULL);
+ if (reg) {
+ ret = of_address_to_resource(np, 0, &res);
+ if (ret) {
+ pr_err("Address translation error\n");
+ return 0;
+ }
+ if ((res.end - res.start) > PAGE_SIZE) {
+ pr_err("L2 reg size > PAGE_SIZE\n");
+ return 0;
+ }
+
+ l2_regs_addr = res.start;
+ } else {
+ l2_regs_addr = (phys_addr_t) __rm_firmware_regs_start;
+ }
+
+ if (!IS_ALIGNED(l2_regs_addr, PAGE_SIZE)) {
+ pr_err("Registers not aligned on PAGE_SIZE\n");
+ return 0;
+ }
+
+ return l2_regs_addr;
+}
+
+static int __init l2_cache_init(void)
+{
+ int ret = -ENODEV;
+ struct device_node *np;
+ phys_addr_t l2_regs_addr;
+
+ np = of_find_compatible_node(NULL, NULL, "kalray,kvx-l2-cache");
+ if (!np || !of_device_is_available(np)) {
+ if (!IS_ENABLED(CONFIG_SMP)) {
+ pr_info("controller disabled\n");
+ return 0;
+ }
+
+ if (np && of_get_property(np, "kalray,is-qemu", NULL)) {
+ /*
+ * QEMU is always full cache coherent. The L2 cache controller is
+ * not strictly necessary to ensure coherency in SMP.
+ */
+ pr_info("controller disabled (QEMU detected)\n");
+ return 0;
+ }
+
+ /* Else, SMP is enabled and L2 is mandatory for it */
+ goto err;
+ }
+
+ l2_regs_addr = l2_get_regs_addr(np);
+ if (!l2_regs_addr)
+ goto err;
+
+ /* Map the L2 registers */
+ l2c_ctrl.regs = ioremap(l2_regs_addr, PAGE_SIZE);
+ if (!l2c_ctrl.regs)
+ goto err;
+
+ /* Lock this entry into the LTLB */
+ kvx_mmu_ltlb_add_entry((unsigned long) l2c_ctrl.regs, l2_regs_addr,
+ PAGE_KERNEL_NOCACHE, TLB_PS_4K);
+
+ ret = l2_cache_init_hw();
+ if (ret) {
+ pr_err("Failed to init L2 cache controller");
+ goto err_unmap_l2;
+ }
+
+ static_branch_enable(&l2_enabled);
+
+ pr_info("controller enabled\n");
+
+ return 0;
+
+err_unmap_l2:
+ kvx_mmu_ltlb_remove_entry((unsigned long) l2c_ctrl.regs);
+ iounmap(l2c_ctrl.regs);
+err:
+ if (IS_ENABLED(CONFIG_SMP))
+ panic("L2$ controller is mandatory for SMP");
+
+ return ret;
+}
+
+
+early_initcall(l2_cache_init);
diff --git a/arch/kvx/kernel/smp.c b/arch/kvx/kernel/smp.c
new file mode 100644
index 000000000000..ed4c35a8c4bc
--- /dev/null
+++ b/arch/kvx/kernel/smp.c
@@ -0,0 +1,110 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/smp.h>
+#include <linux/cpu.h>
+#include <linux/of_irq.h>
+#include <linux/cpumask.h>
+#include <linux/irq_work.h>
+#include <linux/mm_types.h>
+#include <linux/interrupt.h>
+
+#include <asm/ipi.h>
+#include <asm/tlbflush.h>
+
+enum ipi_message_type {
+ IPI_RESCHEDULE,
+ IPI_CALL_FUNC,
+ IPI_IRQ_WORK,
+ IPI_MAX
+};
+
+/* A collection of single bit ipi messages. */
+static struct {
+ unsigned long bits ____cacheline_aligned;
+} ipi_data[NR_CPUS] __cacheline_aligned;
+
+static void send_ipi_message(const struct cpumask *mask,
+ enum ipi_message_type operation)
+{
+ unsigned long flags;
+ int cpu;
+
+ /* Set operation that must be done by receiver */
+ for_each_cpu(cpu, mask)
+ set_bit(operation, &ipi_data[cpu].bits);
+
+ /* Commit the write before sending IPI */
+ smp_wmb();
+
+ local_irq_save(flags);
+
+ kvx_ipi_send(mask);
+
+ local_irq_restore(flags);
+}
+
+void arch_send_call_function_ipi_mask(struct cpumask *mask)
+{
+ send_ipi_message(mask, IPI_CALL_FUNC);
+}
+
+void arch_send_call_function_single_ipi(int cpu)
+{
+ send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC);
+}
+
+#ifdef CONFIG_IRQ_WORK
+void arch_irq_work_raise(void)
+{
+ send_ipi_message(cpumask_of(smp_processor_id()), IPI_IRQ_WORK);
+}
+#endif
+
+static void ipi_stop(void *unused)
+{
+ local_cpu_stop();
+}
+
+void smp_send_stop(void)
+{
+ struct cpumask targets;
+
+ cpumask_copy(&targets, cpu_online_mask);
+ cpumask_clear_cpu(smp_processor_id(), &targets);
+
+ smp_call_function_many(&targets, ipi_stop, NULL, 0);
+}
+
+void smp_send_reschedule(int cpu)
+{
+ send_ipi_message(cpumask_of(cpu), IPI_RESCHEDULE);
+}
+
+irqreturn_t ipi_call_interrupt(int irq, void *dev_id)
+{
+ unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
+
+ while (true) {
+ unsigned long ops = xchg(pending_ipis, 0);
+
+ if (ops == 0)
+ return IRQ_HANDLED;
+
+ if (ops & (1 << IPI_RESCHEDULE))
+ scheduler_ipi();
+
+ if (ops & (1 << IPI_CALL_FUNC))
+ generic_smp_call_function_interrupt();
+
+ if (ops & (1 << IPI_IRQ_WORK))
+ irq_work_run();
+
+ BUG_ON((ops >> IPI_MAX) != 0);
+ }
+
+ return IRQ_HANDLED;
+}
diff --git a/arch/kvx/kernel/smpboot.c b/arch/kvx/kernel/smpboot.c
new file mode 100644
index 000000000000..987a6f014163
--- /dev/null
+++ b/arch/kvx/kernel/smpboot.c
@@ -0,0 +1,127 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Julian Vetter
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/smp.h>
+#include <linux/cpu.h>
+#include <linux/sched.h>
+#include <linux/cpumask.h>
+#include <linux/sched/mm.h>
+#include <linux/mm_types.h>
+#include <linux/of_platform.h>
+#include <linux/sched/task_stack.h>
+
+#include <asm/pwr_ctrl.h>
+#include <asm/tlbflush.h>
+#include <asm/ipi.h>
+
+void *__cpu_up_stack_pointer[NR_CPUS];
+void *__cpu_up_task_pointer[NR_CPUS];
+
+void __init smp_prepare_boot_cpu(void)
+{
+}
+
+int __cpu_up(unsigned int cpu, struct task_struct *tidle)
+{
+ __cpu_up_stack_pointer[cpu] = task_stack_page(tidle) + THREAD_SIZE;
+ __cpu_up_task_pointer[cpu] = tidle;
+ /* We need to be sure writes are committed */
+ smp_mb();
+
+ kvx_pwr_ctrl_cpu_poweron(cpu);
+ while (!cpu_online(cpu))
+ cpu_relax();
+
+ return 0;
+}
+
+void __init smp_cpus_done(unsigned int max_cpus)
+{
+}
+
+void __init smp_init_cpus(void)
+{
+ struct cpumask cpumask;
+ struct device_node *cpu;
+ const __be32 *reg;
+ u32 cpu_num;
+ unsigned int nr_cpus = 0;
+
+ cpumask_clear(&cpumask);
+
+ for_each_of_cpu_node(cpu) {
+ if (!of_device_is_available(cpu))
+ continue;
+
+ reg = of_get_property(cpu, "reg", NULL);
+ if (!reg)
+ continue;
+
+ cpu_num = be32_to_cpup(reg);
+ if (cpu_num >= nr_cpu_ids)
+ continue;
+
+ nr_cpus++;
+ cpumask_set_cpu(cpu_num, &cpumask);
+ }
+
+ pr_info("%d possible cpus\n", nr_cpus);
+ init_cpu_possible(&cpumask);
+}
+
+void __init smp_prepare_cpus(unsigned int max_cpus)
+{
+ if (num_present_cpus() <= 1)
+ init_cpu_present(cpu_possible_mask);
+}
+
+int __init setup_smp(void)
+{
+ int ret;
+
+ ret = kvx_pwr_ctrl_probe();
+ if (ret)
+ panic("Failed to probe power controller !");
+
+ ret = kvx_ipi_ctrl_probe(ipi_call_interrupt);
+ if (ret)
+ panic("Failed to probe IPI controller !");
+
+ return 0;
+}
+
+early_initcall(setup_smp);
+
+/*
+ * C entry point for a secondary processor.
+ */
+void __init start_kernel_secondary(void)
+{
+ struct mm_struct *mm = &init_mm;
+ unsigned int cpu = smp_processor_id();
+
+ setup_processor();
+ kvx_mmu_early_setup();
+
+ /* All kernel threads share the same mm context. */
+ mmgrab(mm);
+ current->active_mm = mm;
+ cpumask_set_cpu(cpu, mm_cpumask(mm));
+
+ notify_cpu_starting(cpu);
+ set_cpu_online(cpu, true);
+ trace_hardirqs_off();
+
+ local_flush_tlb_all();
+
+ local_irq_enable();
+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
+}
diff --git a/arch/kvx/platform/pwr_ctrl.c b/arch/kvx/platform/pwr_ctrl.c
new file mode 100644
index 000000000000..64c86cd18695
--- /dev/null
+++ b/arch/kvx/platform/pwr_ctrl.c
@@ -0,0 +1,93 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/io.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+
+#include <asm/pwr_ctrl.h>
+#include <asm/symbols.h>
+
+struct kvx_pwr_ctrl {
+ void __iomem *regs;
+};
+
+static struct kvx_pwr_ctrl kvx_pwr_controller;
+
+/**
+ * kvx_pwr_ctrl_cpu_poweron() - Wakeup a cpu
+ * @cpu: cpu to wakeup
+ */
+void kvx_pwr_ctrl_cpu_poweron(unsigned int cpu)
+{
+ /* Set PE boot address */
+ writeq((unsigned long long)kvx_start,
+ kvx_pwr_controller.regs + KVX_PWR_CTRL_RESET_PC_OFFSET);
+ /* Wake up processor ! */
+ writeq(1ULL << cpu,
+ kvx_pwr_controller.regs + PWR_CTRL_WUP_SET_OFFSET);
+ /* Then clear wakeup to allow processor to sleep */
+ writeq(1ULL << cpu,
+ kvx_pwr_controller.regs + PWR_CTRL_WUP_CLEAR_OFFSET);
+}
+
+static struct device_node * __init get_pwr_ctrl_node(void)
+{
+ const phandle *ph;
+ struct device_node *cpu;
+ struct device_node *node;
+
+ cpu = of_get_cpu_node(raw_smp_processor_id(), NULL);
+ if (!cpu) {
+ pr_err("Failed to get CPU node\n");
+ return NULL;
+ }
+
+ ph = of_get_property(cpu, "power-controller", NULL);
+ if (!ph) {
+ pr_err("Failed to get power-controller phandle\n");
+ return NULL;
+ }
+
+ node = of_find_node_by_phandle(be32_to_cpup(ph));
+ if (!node) {
+ pr_err("Failed to get power-controller node\n");
+ return NULL;
+ }
+
+ return node;
+}
+
+int __init kvx_pwr_ctrl_probe(void)
+{
+ struct device_node *ctrl;
+
+ ctrl = get_pwr_ctrl_node();
+ if (!ctrl) {
+ pr_err("Failed to get power controller node\n");
+ return -EINVAL;
+ }
+
+ if (!of_device_is_compatible(ctrl, "kalray,kvx-pwr-ctrl")) {
+ pr_err("Failed to get power controller node\n");
+ return -EINVAL;
+ }
+
+ kvx_pwr_controller.regs = of_iomap(ctrl, 0);
+ if (!kvx_pwr_controller.regs) {
+ pr_err("Failed ioremap\n");
+ return -EINVAL;
+ }
+
+ pr_info("kvx power controller probed\n");
+
+ return 0;
+}
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index f61447913db9..f5a484547b15 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -152,6 +152,7 @@ enum cpuhp_state {
CPUHP_AP_IRQ_RISCV_STARTING,
CPUHP_AP_IRQ_LOONGARCH_STARTING,
CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
+ CPUHP_AP_IRQ_KVX_STARTING,
CPUHP_AP_ARM_MVEBU_COHERENCY,
CPUHP_AP_MICROCODE_LOADER,
CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING,
@@ -189,6 +190,7 @@ enum cpuhp_state {
CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING,
CPUHP_AP_KVM_ARM_VGIC_STARTING,
CPUHP_AP_KVM_ARM_TIMER_STARTING,
+ CPUHP_AP_KVX_TIMER_STARTING,
/* Must be the last timer callback */
CPUHP_AP_DUMMY_TIMER_STARTING,
CPUHP_AP_ARM_XEN_STARTING,
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 17/25] kvx: Add multi-processor (SMP) support
2023-01-03 16:43 ` [RFC PATCH 17/25] kvx: Add multi-processor (SMP) support Yann Sionneau
@ 2023-01-03 21:22 ` Rob Herring
2023-01-05 8:12 ` Clément Léger
1 sibling, 0 replies; 113+ messages in thread
From: Rob Herring @ 2023-01-03 21:22 UTC (permalink / raw)
To: Yann Sionneau
Cc: linux-kernel, Clement Leger, Julian Vetter, Julien Hascoet,
Louis Morhet, Luc Michel, Marius Gligor
On Tue, Jan 03, 2023 at 05:43:51PM +0100, Yann Sionneau wrote:
> Coolidge v1 SoC has 5 clusters of 16 kvx cores
> (+1 special Resource Manager (RM) core).
>
> Linux can run in SMP config on the 16 cores of
> a Cluster.
>
> Cache coherency is done by the L2 cache.
>
> CC: linux-kernel@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Co-developed-by: Julien Hascoet <jhascoet@kalray.eu>
> Signed-off-by: Julien Hascoet <jhascoet@kalray.eu>
> Co-developed-by: Louis Morhet <lmorhet@kalray.eu>
> Signed-off-by: Louis Morhet <lmorhet@kalray.eu>
> Co-developed-by: Luc Michel <lmichel@kalray.eu>
> Signed-off-by: Luc Michel <lmichel@kalray.eu>
> Co-developed-by: Marius Gligor <mgligor@kalray.eu>
> Signed-off-by: Marius Gligor <mgligor@kalray.eu>
> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> arch/kvx/include/asm/pwr_ctrl.h | 45 ++++
> arch/kvx/include/asm/smp.h | 42 +++
> arch/kvx/kernel/l2_cache.c | 448 ++++++++++++++++++++++++++++++++
> arch/kvx/kernel/smp.c | 110 ++++++++
> arch/kvx/kernel/smpboot.c | 127 +++++++++
> arch/kvx/platform/pwr_ctrl.c | 93 +++++++
> include/linux/cpuhotplug.h | 2 +
> 7 files changed, 867 insertions(+)
> create mode 100644 arch/kvx/include/asm/pwr_ctrl.h
> create mode 100644 arch/kvx/include/asm/smp.h
> create mode 100644 arch/kvx/kernel/l2_cache.c
> create mode 100644 arch/kvx/kernel/smp.c
> create mode 100644 arch/kvx/kernel/smpboot.c
> create mode 100644 arch/kvx/platform/pwr_ctrl.c
>
> diff --git a/arch/kvx/include/asm/pwr_ctrl.h b/arch/kvx/include/asm/pwr_ctrl.h
> new file mode 100644
> index 000000000000..25f403ba935a
> --- /dev/null
> +++ b/arch/kvx/include/asm/pwr_ctrl.h
> @@ -0,0 +1,45 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + * Marius Gligor
> + */
> +
> +#ifndef _ASM_KVX_PWR_CTRL_H
> +#define _ASM_KVX_PWR_CTRL_H
> +
> +#ifndef __ASSEMBLY__
> +
> +int kvx_pwr_ctrl_probe(void);
> +
> +void kvx_pwr_ctrl_cpu_poweron(unsigned int cpu);
> +
> +#endif
> +
> +/* Power controller vector register definitions */
> +#define KVX_PWR_CTRL_VEC_OFFSET 0x1000
> +#define KVX_PWR_CTRL_VEC_WUP_SET_OFFSET 0x10
> +#define KVX_PWR_CTRL_VEC_WUP_CLEAR_OFFSET 0x20
> +
> +/* Power controller PE reset PC register definitions */
> +#define KVX_PWR_CTRL_RESET_PC_OFFSET 0x2000
> +
> +/* Power controller global register definitions */
> +#define KVX_PWR_CTRL_GLOBAL_OFFSET 0x4040
> +
> +#define KVX_PWR_CTRL_GLOBAL_SET_OFFSET 0x10
> +#define KVX_PWR_CTRL_GLOBAL_SET_PE_EN_SHIFT 0x1
> +
> +#define PWR_CTRL_WUP_SET_OFFSET \
> + (KVX_PWR_CTRL_VEC_OFFSET + \
> + KVX_PWR_CTRL_VEC_WUP_SET_OFFSET)
> +
> +#define PWR_CTRL_WUP_CLEAR_OFFSET \
> + (KVX_PWR_CTRL_VEC_OFFSET + \
> + KVX_PWR_CTRL_VEC_WUP_CLEAR_OFFSET)
> +
> +#define PWR_CTRL_GLOBAL_CONFIG_OFFSET \
> + (KVX_PWR_CTRL_GLOBAL_OFFSET + \
> + KVX_PWR_CTRL_GLOBAL_SET_OFFSET)
> +
> +#endif /* _ASM_KVX_PWR_CTRL_H */
> diff --git a/arch/kvx/include/asm/smp.h b/arch/kvx/include/asm/smp.h
> new file mode 100644
> index 000000000000..e4fd4d001b2c
> --- /dev/null
> +++ b/arch/kvx/include/asm/smp.h
> @@ -0,0 +1,42 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#ifndef _ASM_KVX_SMP_H
> +#define _ASM_KVX_SMP_H
> +
> +#include <linux/cpumask.h>
> +#include <linux/irqreturn.h>
> +
> +#include <asm/sfr.h>
> +
> +#ifdef CONFIG_SMP
> +
> +/* Hook for the generic smp_call_function_many() routine. */
> +void arch_send_call_function_ipi_mask(struct cpumask *mask);
> +
> +/* Hook for the generic smp_call_function_single() routine. */
> +void arch_send_call_function_single_ipi(int cpu);
> +
> +void __init setup_processor(void);
> +
> +void smp_init_cpus(void);
> +
> +irqreturn_t ipi_call_interrupt(int irq, void *dev_id);
> +
> +#define raw_smp_processor_id() ((int) \
> + ((kvx_sfr_get(PCR) & KVX_SFR_PCR_PID_MASK) \
> + >> KVX_SFR_PCR_PID_SHIFT))
> +
> +#define flush_cache_vmap(start, end) do { } while (0)
> +#define flush_cache_vunmap(start, end) do { } while (0)
> +
> +#else
> +
> +void smp_init_cpus(void) {}
> +
> +#endif /* CONFIG_SMP */
> +
> +#endif
> diff --git a/arch/kvx/kernel/l2_cache.c b/arch/kvx/kernel/l2_cache.c
> new file mode 100644
> index 000000000000..94a4d9159471
> --- /dev/null
> +++ b/arch/kvx/kernel/l2_cache.c
> @@ -0,0 +1,448 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + * Luc Michel
> + * Julien Hascoet
> + */
> +
> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +#include <linux/io.h>
> +#include <linux/of.h>
> +#include <linux/jiffies.h>
> +#include <linux/of_address.h>
> +#include <linux/irqchip/irq-kvx-apic-mailbox.h>
> +
> +#include <asm/rm_fw.h>
> +#include <asm/l2_cache.h>
> +#include <asm/sections.h>
> +#include <asm/cacheflush.h>
> +
> +#define L2_START_TIMEOUT_MS 10
> +#define L2_CMD_TIMEOUT_MS 200
> +
> +#define L2_MK_OP(__cmd, __sync) (BIT(L2_CMD_OP_VALID_SHIFT) | \
> + ((u64) __sync << L2_CMD_OP_SYNC_SHIFT) | \
> + ((u64) __cmd << L2_CMD_OP_CMD_SHIFT))
> +
> +#define L2_ERROR(_err) \
> + ((error & L2_ERROR_ERROR_CODE_MASK) >> L2_ERROR_ERROR_CODE_SHIFT)
> +
> +struct l2_cache_hw_cmd {
> + u64 op;
> + u64 args[L2_CMD_OP_ARG_COUNT];
> +} __packed;
> +
> +struct l2_cache_cmd {
> + int sync;
> + int cmd_type;
> + unsigned int arg_count;
> + u64 args[L2_CMD_OP_ARG_COUNT];
> +};
> +
> +/**
> + * struct l2_cached_data - Data associated to the l2-cache
> + * @regs: base of L2 registers
> + * @mbox_regs: Mailbox registers for L2 signaling
> + */
> +struct l2_cache_data {
> + void __iomem *regs;
> + void __iomem *mbox_regs;
> + u64 fifo_cmd_count;
> +};
> +
> +DEFINE_STATIC_KEY_FALSE(l2_enabled);
> +static struct l2_cache_data l2c_ctrl;
> +
> +static void *l2_cmd_regs_addr(void)
> +{
> + return l2c_ctrl.regs + L2_CMD_OFFSET;
> +}
> +
> +static struct l2_cache_hw_cmd *l2_cache_hw_cmd_addr(u64 idx)
> +{
> + void *cmd_regs = l2_cmd_regs_addr();
> +
> + /* Wrap index */
> + idx &= (l2c_ctrl.fifo_cmd_count - 1);
> + return cmd_regs + L2_CMD_FIFO_OFFSET + idx * L2_CMD_FIFO_ELEM_SIZE;
> +}
> +
> +static u64 l2_cache_get_cmd_idx(unsigned int cmd_count)
> +{
> + u64 cmd_idx;
> + void *cmd_regs = l2_cmd_regs_addr();
> + u64 *write_idx_ptr = cmd_regs + L2_CMD_WRITE_IDX_OFFSET;
> + u64 *read_idx_ptr = cmd_regs + L2_CMD_READ_IDX_OFFSET;
> +
> + /* Grab a commands tickets */
> + cmd_idx = __builtin_kvx_aladdd(write_idx_ptr, cmd_count);
> +
> + /* Wait until there is room in command fifo to enqueue commands */
> + while ((cmd_idx + cmd_count) >=
> + (readq(read_idx_ptr) + l2c_ctrl.fifo_cmd_count))
> + cpu_relax();
> +
> + return cmd_idx;
> +}
> +
> +static void l2_wait_completion(u64 cmd_idx)
> +{
> + u64 *read_idx_ptr = l2_cmd_regs_addr() + L2_CMD_READ_IDX_OFFSET;
> + unsigned long timeout = jiffies + msecs_to_jiffies(L2_CMD_TIMEOUT_MS);
> +
> + /* Wait for completion */
> + while (cmd_idx >= readq(read_idx_ptr)) {
> + cpu_relax();
> + if (time_after(jiffies, timeout))
> + panic("L2 cache completion timeout\n");
> + }
> +}
> +
> +static u64 l2_cache_push_cmds(struct l2_cache_cmd *cmds, int cmd_count)
> +{
> + int i, arg;
> + u64 cmd_op;
> + struct l2_cache_hw_cmd *cmd;
> + struct l2_cache_cmd *soft_cmd;
> + u64 cmd_idx = l2_cache_get_cmd_idx(cmd_count);
> +
> + for (i = 0; i < cmd_count; i++) {
> + soft_cmd = &cmds[i];
> + cmd = l2_cache_hw_cmd_addr(cmd_idx);
> + cmd_idx++;
> +
> + for (arg = 0; arg < soft_cmd->arg_count; arg++)
> + writeq_relaxed(soft_cmd->args[arg], &cmd->args[arg]);
> +
> + cmd_op = L2_MK_OP(soft_cmd->cmd_type, soft_cmd->sync);
> + writeq(cmd_op, &cmd->op);
> + }
> +
> + return cmd_idx - 1;
> +}
> +
> +static void l2_cache_create_line_cmd(struct l2_cache_cmd *cmd, int cmd_type,
> + int sync, u64 addr)
> +{
> + cmd->cmd_type = cmd_type;
> + cmd->sync = sync;
> + cmd->arg_count = 1;
> + cmd->args[0] = addr;
> +}
> +
> +static void l2_cache_create_area_cmd(struct l2_cache_cmd *cmd, int cmd_type,
> + int sync, u64 addr, u64 size)
> +{
> + l2_cache_create_line_cmd(cmd, cmd_type, sync, addr);
> + cmd->arg_count = 2;
> + cmd->args[1] = size;
> +}
> +
> +static void l2_cache_push_inval_cmd(phys_addr_t start,
> + unsigned long size)
> +{
> + phys_addr_t end = start + size;
> + struct l2_cache_cmd cmds[3];
> + unsigned long irq_flags;
> + int cmd_count = 0;
> + u64 cmd_idx;
> +
> + /*
> + * In case of invalidation, we must make sure we do not invalidate
> + * unwanted area and thus discard legit data. In case we are not aligned
> + * send a purge line command (writeback + inval) to unaligned lines
> + * (which can be the end line or the start line)
> + */
> + if (!IS_ALIGNED(end, L2_CACHE_LINE_SIZE)) {
> + end &= ~L2_CACHE_LINE_MASK;
> + l2_cache_create_line_cmd(&cmds[cmd_count],
> + L2_CMD_OP_CMD_PURGE_LINE, 1, end);
> + cmd_count++;
> + }
> +
> + if (!IS_ALIGNED(start, L2_CACHE_LINE_SIZE)) {
> + start &= ~L2_CACHE_LINE_MASK;
> + /* If there is at least another line to clear */
> + if (end != start) {
> + l2_cache_create_line_cmd(&cmds[cmd_count],
> + L2_CMD_OP_CMD_PURGE_LINE, 1,
> + start);
> + cmd_count++;
> + start += L2_CACHE_LINE_SIZE;
> + }
> + }
> +
> + BUG_ON(end < start);
> +
> + size = (end - start);
> + if (size > 0) {
> + l2_cache_create_area_cmd(&cmds[cmd_count],
> + L2_CMD_OP_CMD_INVAL_AREA, 1, start,
> + size);
> + cmd_count++;
> + }
> +
> + BUG_ON(cmd_count == 0);
> +
> + local_irq_save(irq_flags);
> +
> + cmd_idx = l2_cache_push_cmds(cmds, cmd_count);
> +
> + /* Finally, ping the L2 cache controller */
> + writeq(1, l2c_ctrl.mbox_regs);
> +
> + local_irq_restore(irq_flags);
> +
> + l2_wait_completion(cmd_idx);
> +}
> +
> +static void l2_cache_push_generic_cmd(u64 cmd_type, phys_addr_t start,
> + unsigned long size)
> +{
> + unsigned long irq_flags;
> + struct l2_cache_cmd cmd;
> + u64 cmd_idx;
> +
> + /* Align the start address and size on cache line */
> + size += start & (L2_CACHE_LINE_SIZE - 1);
> + size = ALIGN(size, L2_CACHE_LINE_SIZE);
> + start = ALIGN_DOWN(start, L2_CACHE_LINE_SIZE);
> +
> + local_irq_save(irq_flags);
> +
> + l2_cache_create_area_cmd(&cmd, cmd_type, 1, start, size);
> + cmd_idx = l2_cache_push_cmds(&cmd, 1);
> +
> + /* Finally, ping the L2 cache controller */
> + writeq(1, l2c_ctrl.mbox_regs);
> +
> + local_irq_restore(irq_flags);
> +
> + l2_wait_completion(cmd_idx);
> +}
> +
> +void l2_cache_push_area_cmd(u64 cmd_type, phys_addr_t start,
> + unsigned long size)
> +{
> + if (WARN_ON(size == 0))
> + return;
> +
> + if (cmd_type == L2_CMD_OP_CMD_INVAL_AREA)
> + l2_cache_push_inval_cmd(start, size);
> + else
> + l2_cache_push_generic_cmd(cmd_type, start, size);
> +}
> +
> +static void __init l2_disp_error(u64 error)
> +{
> + const char *err_type;
> +
> + if (error & L2_ERROR_API_ERR_MASK)
> + err_type = "API";
> + else if (error & L2_ERROR_SETUP_ERR_MASK)
> + err_type = "SETUP";
> + else
> + err_type = "UNKNOWN";
> +
> + pr_err("%s error: 0x%llx\n", err_type, L2_ERROR(error));
> +}
> +
> +static int __init l2_cache_configure_mailboxes(void)
> +{
> + phys_addr_t l2_mbox_addr = 0;
> + void *cmd_regs = l2_cmd_regs_addr();
> +
> + /* We do not use mailbox to wait for completion, set it to 0 */
> + writeq(0, cmd_regs + L2_CMD_DOORBELL_READ_ADDR_OFFSET);
> +
> + /* Read mailbox address from L2 registers */
> + l2_mbox_addr = readq(cmd_regs + L2_CMD_DOORBELL_WRITE_ADDR_OFFSET);
> +
> + /* Then map the mailbox */
> + l2c_ctrl.mbox_regs = ioremap(l2_mbox_addr, PAGE_SIZE);
> + if (!l2c_ctrl.mbox_regs) {
> + pr_err("Failed to map mailbox\n");
> + return 1;
> + }
> +
> + /* Lock this entry into the LTLB */
> + kvx_mmu_ltlb_add_entry((unsigned long) l2c_ctrl.mbox_regs & PAGE_MASK,
> + l2_mbox_addr & PAGE_MASK,
> + PAGE_KERNEL_DEVICE, TLB_PS_4K);
> +
> + return 0;
> +}
> +
> +static int __init l2_cache_read_queue_size(void)
> +{
> + u64 inst;
> +
> + /* Read command queue size */
> + inst = readq(l2c_ctrl.regs + L2_INSTANCE_OFFSET);
> + l2c_ctrl.fifo_cmd_count = (inst & L2_INSTANCE_CMD_QUEUE_SIZE_MASK)
> + >> L2_INSTANCE_CMD_QUEUE_SIZE_SHIFT;
> +
> + /* Check if value is a power of two */
> + if (hweight64(l2c_ctrl.fifo_cmd_count) != 1) {
> + pr_err("Command queue size is not a power of two\n");
> + return 1;
> + }
> +
> + return 0;
> +}
> +
> +static int __init l2_cache_init_hw(void)
> +{
> + unsigned long timeout = jiffies + msecs_to_jiffies(L2_START_TIMEOUT_MS);
> + unsigned long flags;
> + u64 status, error;
> + int ret;
> +
> + /* Wait for L2 to be up */
> + do {
> + status = readq(l2c_ctrl.regs + L2_STATUS_OFFSET);
> + if (status & (L2_STATUS_READY_MASK | L2_STATUS_ERROR_MASK))
> + break;
> + } while (time_before(jiffies, timeout));
> +
> + if (!status) {
> + pr_err("Timeout while waiting for firmware status\n");
> + return -ENODEV;
> + }
> +
> + if (status & L2_STATUS_ERROR_MASK) {
> + error = readq(l2c_ctrl.regs + L2_ERROR_OFFSET);
> + l2_disp_error(error);
> + return -EINVAL;
> + }
> +
> + /* Now write ack to L2 firmware */
> + writeq(status | L2_STATUS_ACK_MASK, l2c_ctrl.regs + L2_STATUS_OFFSET);
> +
> + ret = l2_cache_read_queue_size();
> + if (ret)
> + return ret;
> +
> + ret = l2_cache_configure_mailboxes();
> + if (ret)
> + return ret;
> +
> + /* Enable the L2 atomically */
> + local_irq_save(flags);
> +
> + /* Fence data accesses */
> + kvx_fence();
> + /* Purge L1 */
> + l1_inval_dcache_all();
> + l1_inval_icache_all();
> + __builtin_kvx_barrier();
> +
> + local_irq_restore(flags);
> +
> + /* Enable L2$ */
> + kvx_sfr_set_field(PS, L2E, 1);
> +
> + return 0;
> +}
> +
> +static phys_addr_t __init l2_get_regs_addr(struct device_node *np)
> +{
> + const void *reg;
> + struct resource res;
> + phys_addr_t l2_regs_addr;
> + int ret;
> +
> + /*
> + * If regs is specified in device tree, then the L2$ has been loaded by
> + * someone else and not by ourself.
> + */
> + reg = of_get_property(np, "reg", NULL);
Why do you read 'reg' and then...
> + if (reg) {
> + ret = of_address_to_resource(np, 0, &res);
...read it again?
> + if (ret) {
> + pr_err("Address translation error\n");
> + return 0;
> + }
> + if ((res.end - res.start) > PAGE_SIZE) {
> + pr_err("L2 reg size > PAGE_SIZE\n");
Why do you care?
> + return 0;
> + }
> +
> + l2_regs_addr = res.start;
> + } else {
> + l2_regs_addr = (phys_addr_t) __rm_firmware_regs_start;
> + }
> +
> + if (!IS_ALIGNED(l2_regs_addr, PAGE_SIZE)) {
> + pr_err("Registers not aligned on PAGE_SIZE\n");
So?
If you have constraints, put those in the DT schema for this device and
validate your DT. The kernel doesn't need to validate the DT.
> + return 0;
> + }
> +
> + return l2_regs_addr;
> +}
> +
> +static int __init l2_cache_init(void)
> +{
> + int ret = -ENODEV;
> + struct device_node *np;
> + phys_addr_t l2_regs_addr;
> +
> + np = of_find_compatible_node(NULL, NULL, "kalray,kvx-l2-cache");
Needs to be documented.
> + if (!np || !of_device_is_available(np)) {
> + if (!IS_ENABLED(CONFIG_SMP)) {
> + pr_info("controller disabled\n");
> + return 0;
> + }
> +
> + if (np && of_get_property(np, "kalray,is-qemu", NULL)) {
Make your compatible strings specific enough to detect this rather than
a flag.
> + /*
> + * QEMU is always full cache coherent. The L2 cache controller is
> + * not strictly necessary to ensure coherency in SMP.
> + */
> + pr_info("controller disabled (QEMU detected)\n");
> + return 0;
> + }
> +
> + /* Else, SMP is enabled and L2 is mandatory for it */
> + goto err;
> + }
> +
> + l2_regs_addr = l2_get_regs_addr(np);
> + if (!l2_regs_addr)
> + goto err;
> +
> + /* Map the L2 registers */
> + l2c_ctrl.regs = ioremap(l2_regs_addr, PAGE_SIZE);
> + if (!l2c_ctrl.regs)
> + goto err;
> +
> + /* Lock this entry into the LTLB */
> + kvx_mmu_ltlb_add_entry((unsigned long) l2c_ctrl.regs, l2_regs_addr,
> + PAGE_KERNEL_NOCACHE, TLB_PS_4K);
> +
> + ret = l2_cache_init_hw();
> + if (ret) {
> + pr_err("Failed to init L2 cache controller");
> + goto err_unmap_l2;
> + }
> +
> + static_branch_enable(&l2_enabled);
> +
> + pr_info("controller enabled\n");
> +
> + return 0;
> +
> +err_unmap_l2:
> + kvx_mmu_ltlb_remove_entry((unsigned long) l2c_ctrl.regs);
> + iounmap(l2c_ctrl.regs);
> +err:
> + if (IS_ENABLED(CONFIG_SMP))
> + panic("L2$ controller is mandatory for SMP");
> +
> + return ret;
> +}
> +
> +
> +early_initcall(l2_cache_init);
> diff --git a/arch/kvx/kernel/smp.c b/arch/kvx/kernel/smp.c
> new file mode 100644
> index 000000000000..ed4c35a8c4bc
> --- /dev/null
> +++ b/arch/kvx/kernel/smp.c
> @@ -0,0 +1,110 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#include <linux/smp.h>
> +#include <linux/cpu.h>
> +#include <linux/of_irq.h>
> +#include <linux/cpumask.h>
> +#include <linux/irq_work.h>
> +#include <linux/mm_types.h>
> +#include <linux/interrupt.h>
> +
> +#include <asm/ipi.h>
> +#include <asm/tlbflush.h>
> +
> +enum ipi_message_type {
> + IPI_RESCHEDULE,
> + IPI_CALL_FUNC,
> + IPI_IRQ_WORK,
> + IPI_MAX
> +};
> +
> +/* A collection of single bit ipi messages. */
> +static struct {
> + unsigned long bits ____cacheline_aligned;
> +} ipi_data[NR_CPUS] __cacheline_aligned;
> +
> +static void send_ipi_message(const struct cpumask *mask,
> + enum ipi_message_type operation)
> +{
> + unsigned long flags;
> + int cpu;
> +
> + /* Set operation that must be done by receiver */
> + for_each_cpu(cpu, mask)
> + set_bit(operation, &ipi_data[cpu].bits);
> +
> + /* Commit the write before sending IPI */
> + smp_wmb();
> +
> + local_irq_save(flags);
> +
> + kvx_ipi_send(mask);
> +
> + local_irq_restore(flags);
> +}
> +
> +void arch_send_call_function_ipi_mask(struct cpumask *mask)
> +{
> + send_ipi_message(mask, IPI_CALL_FUNC);
> +}
> +
> +void arch_send_call_function_single_ipi(int cpu)
> +{
> + send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC);
> +}
> +
> +#ifdef CONFIG_IRQ_WORK
> +void arch_irq_work_raise(void)
> +{
> + send_ipi_message(cpumask_of(smp_processor_id()), IPI_IRQ_WORK);
> +}
> +#endif
> +
> +static void ipi_stop(void *unused)
> +{
> + local_cpu_stop();
> +}
> +
> +void smp_send_stop(void)
> +{
> + struct cpumask targets;
> +
> + cpumask_copy(&targets, cpu_online_mask);
> + cpumask_clear_cpu(smp_processor_id(), &targets);
> +
> + smp_call_function_many(&targets, ipi_stop, NULL, 0);
> +}
> +
> +void smp_send_reschedule(int cpu)
> +{
> + send_ipi_message(cpumask_of(cpu), IPI_RESCHEDULE);
> +}
> +
> +irqreturn_t ipi_call_interrupt(int irq, void *dev_id)
> +{
> + unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
> +
> + while (true) {
> + unsigned long ops = xchg(pending_ipis, 0);
> +
> + if (ops == 0)
> + return IRQ_HANDLED;
> +
> + if (ops & (1 << IPI_RESCHEDULE))
> + scheduler_ipi();
> +
> + if (ops & (1 << IPI_CALL_FUNC))
> + generic_smp_call_function_interrupt();
> +
> + if (ops & (1 << IPI_IRQ_WORK))
> + irq_work_run();
> +
> + BUG_ON((ops >> IPI_MAX) != 0);
> + }
> +
> + return IRQ_HANDLED;
> +}
> diff --git a/arch/kvx/kernel/smpboot.c b/arch/kvx/kernel/smpboot.c
> new file mode 100644
> index 000000000000..987a6f014163
> --- /dev/null
> +++ b/arch/kvx/kernel/smpboot.c
> @@ -0,0 +1,127 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + * Julian Vetter
> + */
> +
> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +#include <linux/io.h>
> +#include <linux/of.h>
> +#include <linux/smp.h>
> +#include <linux/cpu.h>
> +#include <linux/sched.h>
> +#include <linux/cpumask.h>
> +#include <linux/sched/mm.h>
> +#include <linux/mm_types.h>
> +#include <linux/of_platform.h>
Don't think this is needed.
> +#include <linux/sched/task_stack.h>
> +
> +#include <asm/pwr_ctrl.h>
> +#include <asm/tlbflush.h>
> +#include <asm/ipi.h>
> +
> +void *__cpu_up_stack_pointer[NR_CPUS];
> +void *__cpu_up_task_pointer[NR_CPUS];
> +
> +void __init smp_prepare_boot_cpu(void)
> +{
> +}
> +
> +int __cpu_up(unsigned int cpu, struct task_struct *tidle)
> +{
> + __cpu_up_stack_pointer[cpu] = task_stack_page(tidle) + THREAD_SIZE;
> + __cpu_up_task_pointer[cpu] = tidle;
> + /* We need to be sure writes are committed */
> + smp_mb();
> +
> + kvx_pwr_ctrl_cpu_poweron(cpu);
> + while (!cpu_online(cpu))
> + cpu_relax();
> +
> + return 0;
> +}
> +
> +void __init smp_cpus_done(unsigned int max_cpus)
> +{
> +}
> +
> +void __init smp_init_cpus(void)
> +{
> + struct cpumask cpumask;
> + struct device_node *cpu;
> + const __be32 *reg;
> + u32 cpu_num;
> + unsigned int nr_cpus = 0;
> +
> + cpumask_clear(&cpumask);
> +
> + for_each_of_cpu_node(cpu) {
> + if (!of_device_is_available(cpu))
> + continue;
> +
> + reg = of_get_property(cpu, "reg", NULL);
Use of_get_cpu_hwid()
> + if (!reg)
> + continue;
> +
> + cpu_num = be32_to_cpup(reg);
> + if (cpu_num >= nr_cpu_ids)
> + continue;
> +
> + nr_cpus++;
> + cpumask_set_cpu(cpu_num, &cpumask);
Why not just do set_cpu_possible() directly instead of the temp cpumask.
> + }
> +
> + pr_info("%d possible cpus\n", nr_cpus);
> + init_cpu_possible(&cpumask);
> +}
> +
> +void __init smp_prepare_cpus(unsigned int max_cpus)
> +{
> + if (num_present_cpus() <= 1)
> + init_cpu_present(cpu_possible_mask);
> +}
> +
> +int __init setup_smp(void)
> +{
> + int ret;
> +
> + ret = kvx_pwr_ctrl_probe();
> + if (ret)
> + panic("Failed to probe power controller !");
> +
> + ret = kvx_ipi_ctrl_probe(ipi_call_interrupt);
> + if (ret)
> + panic("Failed to probe IPI controller !");
> +
> + return 0;
> +}
> +
> +early_initcall(setup_smp);
> +
> +/*
> + * C entry point for a secondary processor.
> + */
> +void __init start_kernel_secondary(void)
> +{
> + struct mm_struct *mm = &init_mm;
> + unsigned int cpu = smp_processor_id();
> +
> + setup_processor();
> + kvx_mmu_early_setup();
> +
> + /* All kernel threads share the same mm context. */
> + mmgrab(mm);
> + current->active_mm = mm;
> + cpumask_set_cpu(cpu, mm_cpumask(mm));
> +
> + notify_cpu_starting(cpu);
> + set_cpu_online(cpu, true);
> + trace_hardirqs_off();
> +
> + local_flush_tlb_all();
> +
> + local_irq_enable();
> + cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
> +}
> diff --git a/arch/kvx/platform/pwr_ctrl.c b/arch/kvx/platform/pwr_ctrl.c
> new file mode 100644
> index 000000000000..64c86cd18695
> --- /dev/null
> +++ b/arch/kvx/platform/pwr_ctrl.c
> @@ -0,0 +1,93 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +#include <linux/io.h>
> +#include <linux/slab.h>
> +#include <linux/types.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_platform.h>
> +
> +#include <asm/pwr_ctrl.h>
> +#include <asm/symbols.h>
> +
> +struct kvx_pwr_ctrl {
> + void __iomem *regs;
> +};
> +
> +static struct kvx_pwr_ctrl kvx_pwr_controller;
> +
> +/**
> + * kvx_pwr_ctrl_cpu_poweron() - Wakeup a cpu
> + * @cpu: cpu to wakeup
> + */
> +void kvx_pwr_ctrl_cpu_poweron(unsigned int cpu)
> +{
> + /* Set PE boot address */
> + writeq((unsigned long long)kvx_start,
> + kvx_pwr_controller.regs + KVX_PWR_CTRL_RESET_PC_OFFSET);
> + /* Wake up processor ! */
> + writeq(1ULL << cpu,
> + kvx_pwr_controller.regs + PWR_CTRL_WUP_SET_OFFSET);
> + /* Then clear wakeup to allow processor to sleep */
> + writeq(1ULL << cpu,
> + kvx_pwr_controller.regs + PWR_CTRL_WUP_CLEAR_OFFSET);
> +}
> +
> +static struct device_node * __init get_pwr_ctrl_node(void)
> +{
> + const phandle *ph;
> + struct device_node *cpu;
> + struct device_node *node;
> +
> + cpu = of_get_cpu_node(raw_smp_processor_id(), NULL);
> + if (!cpu) {
> + pr_err("Failed to get CPU node\n");
> + return NULL;
> + }
> +
> + ph = of_get_property(cpu, "power-controller", NULL);
Not a standard property.
> + if (!ph) {
> + pr_err("Failed to get power-controller phandle\n");
> + return NULL;
> + }
> +
> + node = of_find_node_by_phandle(be32_to_cpup(ph));
> + if (!node) {
> + pr_err("Failed to get power-controller node\n");
> + return NULL;
> + }
> +
> + return node;
> +}
> +
> +int __init kvx_pwr_ctrl_probe(void)
> +{
> + struct device_node *ctrl;
> +
> + ctrl = get_pwr_ctrl_node();
> + if (!ctrl) {
> + pr_err("Failed to get power controller node\n");
> + return -EINVAL;
> + }
> +
> + if (!of_device_is_compatible(ctrl, "kalray,kvx-pwr-ctrl")) {
Needs to be documented...
> + pr_err("Failed to get power controller node\n");
> + return -EINVAL;
> + }
> +
> + kvx_pwr_controller.regs = of_iomap(ctrl, 0);
> + if (!kvx_pwr_controller.regs) {
> + pr_err("Failed ioremap\n");
> + return -EINVAL;
> + }
> +
> + pr_info("kvx power controller probed\n");
Drop success messages.
> +
> + return 0;
> +}
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index f61447913db9..f5a484547b15 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -152,6 +152,7 @@ enum cpuhp_state {
> CPUHP_AP_IRQ_RISCV_STARTING,
> CPUHP_AP_IRQ_LOONGARCH_STARTING,
> CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
> + CPUHP_AP_IRQ_KVX_STARTING,
> CPUHP_AP_ARM_MVEBU_COHERENCY,
> CPUHP_AP_MICROCODE_LOADER,
> CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING,
> @@ -189,6 +190,7 @@ enum cpuhp_state {
> CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING,
> CPUHP_AP_KVM_ARM_VGIC_STARTING,
> CPUHP_AP_KVM_ARM_TIMER_STARTING,
> + CPUHP_AP_KVX_TIMER_STARTING,
> /* Must be the last timer callback */
> CPUHP_AP_DUMMY_TIMER_STARTING,
> CPUHP_AP_ARM_XEN_STARTING,
> --
> 2.37.2
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 17/25] kvx: Add multi-processor (SMP) support
2023-01-03 16:43 ` [RFC PATCH 17/25] kvx: Add multi-processor (SMP) support Yann Sionneau
2023-01-03 21:22 ` Rob Herring
@ 2023-01-05 8:12 ` Clément Léger
1 sibling, 0 replies; 113+ messages in thread
From: Clément Léger @ 2023-01-05 8:12 UTC (permalink / raw)
To: Yann Sionneau
Cc: linux-kernel, Julian Vetter, Julien Hascoet, Louis Morhet,
Luc Michel, Marius Gligor
Le Tue, 3 Jan 2023 17:43:51 +0100,
Yann Sionneau <ysionneau@kalray.eu> a écrit :
> Coolidge v1 SoC has 5 clusters of 16 kvx cores
> (+1 special Resource Manager (RM) core).
>
> Linux can run in SMP config on the 16 cores of
> a Cluster.
>
> Cache coherency is done by the L2 cache.
>
> CC: linux-kernel@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Co-developed-by: Julien Hascoet <jhascoet@kalray.eu>
> Signed-off-by: Julien Hascoet <jhascoet@kalray.eu>
> Co-developed-by: Louis Morhet <lmorhet@kalray.eu>
> Signed-off-by: Louis Morhet <lmorhet@kalray.eu>
> Co-developed-by: Luc Michel <lmichel@kalray.eu>
> Signed-off-by: Luc Michel <lmichel@kalray.eu>
> Co-developed-by: Marius Gligor <mgligor@kalray.eu>
> Signed-off-by: Marius Gligor <mgligor@kalray.eu>
> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> arch/kvx/include/asm/pwr_ctrl.h | 45 ++++
> arch/kvx/include/asm/smp.h | 42 +++
> arch/kvx/kernel/l2_cache.c | 448 ++++++++++++++++++++++++++++++++
> arch/kvx/kernel/smp.c | 110 ++++++++
> arch/kvx/kernel/smpboot.c | 127 +++++++++
> arch/kvx/platform/pwr_ctrl.c | 93 +++++++
> include/linux/cpuhotplug.h | 2 +
> 7 files changed, 867 insertions(+)
> create mode 100644 arch/kvx/include/asm/pwr_ctrl.h
> create mode 100644 arch/kvx/include/asm/smp.h
> create mode 100644 arch/kvx/kernel/l2_cache.c
> create mode 100644 arch/kvx/kernel/smp.c
> create mode 100644 arch/kvx/kernel/smpboot.c
> create mode 100644 arch/kvx/platform/pwr_ctrl.c
>
> diff --git a/arch/kvx/include/asm/pwr_ctrl.h b/arch/kvx/include/asm/pwr_ctrl.h
> new file mode 100644
> index 000000000000..25f403ba935a
> --- /dev/null
> +++ b/arch/kvx/include/asm/pwr_ctrl.h
> @@ -0,0 +1,45 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + * Marius Gligor
> + */
> +
> +#ifndef _ASM_KVX_PWR_CTRL_H
> +#define _ASM_KVX_PWR_CTRL_H
> +
> +#ifndef __ASSEMBLY__
> +
> +int kvx_pwr_ctrl_probe(void);
> +
> +void kvx_pwr_ctrl_cpu_poweron(unsigned int cpu);
> +
> +#endif
> +
> +/* Power controller vector register definitions */
> +#define KVX_PWR_CTRL_VEC_OFFSET 0x1000
> +#define KVX_PWR_CTRL_VEC_WUP_SET_OFFSET 0x10
> +#define KVX_PWR_CTRL_VEC_WUP_CLEAR_OFFSET 0x20
> +
> +/* Power controller PE reset PC register definitions */
> +#define KVX_PWR_CTRL_RESET_PC_OFFSET 0x2000
> +
> +/* Power controller global register definitions */
> +#define KVX_PWR_CTRL_GLOBAL_OFFSET 0x4040
> +
> +#define KVX_PWR_CTRL_GLOBAL_SET_OFFSET 0x10
> +#define KVX_PWR_CTRL_GLOBAL_SET_PE_EN_SHIFT 0x1
> +
> +#define PWR_CTRL_WUP_SET_OFFSET \
> + (KVX_PWR_CTRL_VEC_OFFSET + \
> + KVX_PWR_CTRL_VEC_WUP_SET_OFFSET)
> +
> +#define PWR_CTRL_WUP_CLEAR_OFFSET \
> + (KVX_PWR_CTRL_VEC_OFFSET + \
> + KVX_PWR_CTRL_VEC_WUP_CLEAR_OFFSET)
> +
> +#define PWR_CTRL_GLOBAL_CONFIG_OFFSET \
> + (KVX_PWR_CTRL_GLOBAL_OFFSET + \
> + KVX_PWR_CTRL_GLOBAL_SET_OFFSET)
> +
> +#endif /* _ASM_KVX_PWR_CTRL_H */
> diff --git a/arch/kvx/include/asm/smp.h b/arch/kvx/include/asm/smp.h
> new file mode 100644
> index 000000000000..e4fd4d001b2c
> --- /dev/null
> +++ b/arch/kvx/include/asm/smp.h
> @@ -0,0 +1,42 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#ifndef _ASM_KVX_SMP_H
> +#define _ASM_KVX_SMP_H
> +
> +#include <linux/cpumask.h>
> +#include <linux/irqreturn.h>
> +
> +#include <asm/sfr.h>
> +
> +#ifdef CONFIG_SMP
> +
> +/* Hook for the generic smp_call_function_many() routine. */
> +void arch_send_call_function_ipi_mask(struct cpumask *mask);
> +
> +/* Hook for the generic smp_call_function_single() routine. */
> +void arch_send_call_function_single_ipi(int cpu);
> +
> +void __init setup_processor(void);
> +
> +void smp_init_cpus(void);
> +
> +irqreturn_t ipi_call_interrupt(int irq, void *dev_id);
> +
> +#define raw_smp_processor_id() ((int) \
> + ((kvx_sfr_get(PCR) & KVX_SFR_PCR_PID_MASK) \
> + >> KVX_SFR_PCR_PID_SHIFT))
> +
> +#define flush_cache_vmap(start, end) do { } while (0)
> +#define flush_cache_vunmap(start, end) do { } while (0)
> +
> +#else
> +
> +void smp_init_cpus(void) {}
> +
> +#endif /* CONFIG_SMP */
> +
> +#endif
> diff --git a/arch/kvx/kernel/l2_cache.c b/arch/kvx/kernel/l2_cache.c
> new file mode 100644
> index 000000000000..94a4d9159471
> --- /dev/null
> +++ b/arch/kvx/kernel/l2_cache.c
> @@ -0,0 +1,448 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + * Luc Michel
> + * Julien Hascoet
> + */
> +
> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +#include <linux/io.h>
> +#include <linux/of.h>
> +#include <linux/jiffies.h>
> +#include <linux/of_address.h>
> +#include <linux/irqchip/irq-kvx-apic-mailbox.h>
> +
> +#include <asm/rm_fw.h>
> +#include <asm/l2_cache.h>
> +#include <asm/sections.h>
> +#include <asm/cacheflush.h>
> +
> +#define L2_START_TIMEOUT_MS 10
> +#define L2_CMD_TIMEOUT_MS 200
> +
> +#define L2_MK_OP(__cmd, __sync) (BIT(L2_CMD_OP_VALID_SHIFT) | \
> + ((u64) __sync << L2_CMD_OP_SYNC_SHIFT) | \
> + ((u64) __cmd << L2_CMD_OP_CMD_SHIFT))
> +
> +#define L2_ERROR(_err) \
> + ((error & L2_ERROR_ERROR_CODE_MASK) >> L2_ERROR_ERROR_CODE_SHIFT)
> +
> +struct l2_cache_hw_cmd {
> + u64 op;
> + u64 args[L2_CMD_OP_ARG_COUNT];
> +} __packed;
> +
> +struct l2_cache_cmd {
> + int sync;
> + int cmd_type;
> + unsigned int arg_count;
> + u64 args[L2_CMD_OP_ARG_COUNT];
> +};
> +
> +/**
> + * struct l2_cached_data - Data associated to the l2-cache
> + * @regs: base of L2 registers
> + * @mbox_regs: Mailbox registers for L2 signaling
> + */
> +struct l2_cache_data {
> + void __iomem *regs;
> + void __iomem *mbox_regs;
> + u64 fifo_cmd_count;
> +};
> +
> +DEFINE_STATIC_KEY_FALSE(l2_enabled);
> +static struct l2_cache_data l2c_ctrl;
> +
> +static void *l2_cmd_regs_addr(void)
> +{
> + return l2c_ctrl.regs + L2_CMD_OFFSET;
> +}
> +
> +static struct l2_cache_hw_cmd *l2_cache_hw_cmd_addr(u64 idx)
> +{
> + void *cmd_regs = l2_cmd_regs_addr();
> +
> + /* Wrap index */
> + idx &= (l2c_ctrl.fifo_cmd_count - 1);
> + return cmd_regs + L2_CMD_FIFO_OFFSET + idx * L2_CMD_FIFO_ELEM_SIZE;
> +}
> +
> +static u64 l2_cache_get_cmd_idx(unsigned int cmd_count)
> +{
> + u64 cmd_idx;
> + void *cmd_regs = l2_cmd_regs_addr();
> + u64 *write_idx_ptr = cmd_regs + L2_CMD_WRITE_IDX_OFFSET;
> + u64 *read_idx_ptr = cmd_regs + L2_CMD_READ_IDX_OFFSET;
> +
> + /* Grab a commands tickets */
> + cmd_idx = __builtin_kvx_aladdd(write_idx_ptr, cmd_count);
> +
> + /* Wait until there is room in command fifo to enqueue commands */
> + while ((cmd_idx + cmd_count) >=
> + (readq(read_idx_ptr) + l2c_ctrl.fifo_cmd_count))
> + cpu_relax();
> +
> + return cmd_idx;
> +}
> +
> +static void l2_wait_completion(u64 cmd_idx)
> +{
> + u64 *read_idx_ptr = l2_cmd_regs_addr() + L2_CMD_READ_IDX_OFFSET;
> + unsigned long timeout = jiffies + msecs_to_jiffies(L2_CMD_TIMEOUT_MS);
> +
> + /* Wait for completion */
> + while (cmd_idx >= readq(read_idx_ptr)) {
> + cpu_relax();
> + if (time_after(jiffies, timeout))
> + panic("L2 cache completion timeout\n");
> + }
> +}
> +
> +static u64 l2_cache_push_cmds(struct l2_cache_cmd *cmds, int cmd_count)
> +{
> + int i, arg;
> + u64 cmd_op;
> + struct l2_cache_hw_cmd *cmd;
> + struct l2_cache_cmd *soft_cmd;
> + u64 cmd_idx = l2_cache_get_cmd_idx(cmd_count);
> +
> + for (i = 0; i < cmd_count; i++) {
> + soft_cmd = &cmds[i];
> + cmd = l2_cache_hw_cmd_addr(cmd_idx);
> + cmd_idx++;
> +
> + for (arg = 0; arg < soft_cmd->arg_count; arg++)
> + writeq_relaxed(soft_cmd->args[arg], &cmd->args[arg]);
> +
> + cmd_op = L2_MK_OP(soft_cmd->cmd_type, soft_cmd->sync);
> + writeq(cmd_op, &cmd->op);
> + }
> +
> + return cmd_idx - 1;
> +}
> +
> +static void l2_cache_create_line_cmd(struct l2_cache_cmd *cmd, int cmd_type,
> + int sync, u64 addr)
> +{
> + cmd->cmd_type = cmd_type;
> + cmd->sync = sync;
> + cmd->arg_count = 1;
> + cmd->args[0] = addr;
> +}
> +
> +static void l2_cache_create_area_cmd(struct l2_cache_cmd *cmd, int cmd_type,
> + int sync, u64 addr, u64 size)
> +{
> + l2_cache_create_line_cmd(cmd, cmd_type, sync, addr);
> + cmd->arg_count = 2;
> + cmd->args[1] = size;
> +}
> +
> +static void l2_cache_push_inval_cmd(phys_addr_t start,
> + unsigned long size)
> +{
> + phys_addr_t end = start + size;
> + struct l2_cache_cmd cmds[3];
> + unsigned long irq_flags;
> + int cmd_count = 0;
> + u64 cmd_idx;
> +
> + /*
> + * In case of invalidation, we must make sure we do not invalidate
> + * unwanted area and thus discard legit data. In case we are not aligned
> + * send a purge line command (writeback + inval) to unaligned lines
> + * (which can be the end line or the start line)
> + */
> + if (!IS_ALIGNED(end, L2_CACHE_LINE_SIZE)) {
> + end &= ~L2_CACHE_LINE_MASK;
> + l2_cache_create_line_cmd(&cmds[cmd_count],
> + L2_CMD_OP_CMD_PURGE_LINE, 1, end);
> + cmd_count++;
> + }
> +
> + if (!IS_ALIGNED(start, L2_CACHE_LINE_SIZE)) {
> + start &= ~L2_CACHE_LINE_MASK;
> + /* If there is at least another line to clear */
> + if (end != start) {
> + l2_cache_create_line_cmd(&cmds[cmd_count],
> + L2_CMD_OP_CMD_PURGE_LINE, 1,
> + start);
> + cmd_count++;
> + start += L2_CACHE_LINE_SIZE;
> + }
> + }
> +
> + BUG_ON(end < start);
> +
> + size = (end - start);
> + if (size > 0) {
> + l2_cache_create_area_cmd(&cmds[cmd_count],
> + L2_CMD_OP_CMD_INVAL_AREA, 1, start,
> + size);
> + cmd_count++;
> + }
> +
> + BUG_ON(cmd_count == 0);
> +
> + local_irq_save(irq_flags);
> +
> + cmd_idx = l2_cache_push_cmds(cmds, cmd_count);
> +
> + /* Finally, ping the L2 cache controller */
> + writeq(1, l2c_ctrl.mbox_regs);
> +
> + local_irq_restore(irq_flags);
> +
> + l2_wait_completion(cmd_idx);
> +}
> +
> +static void l2_cache_push_generic_cmd(u64 cmd_type, phys_addr_t start,
> + unsigned long size)
> +{
> + unsigned long irq_flags;
> + struct l2_cache_cmd cmd;
> + u64 cmd_idx;
> +
> + /* Align the start address and size on cache line */
> + size += start & (L2_CACHE_LINE_SIZE - 1);
> + size = ALIGN(size, L2_CACHE_LINE_SIZE);
> + start = ALIGN_DOWN(start, L2_CACHE_LINE_SIZE);
> +
> + local_irq_save(irq_flags);
> +
> + l2_cache_create_area_cmd(&cmd, cmd_type, 1, start, size);
> + cmd_idx = l2_cache_push_cmds(&cmd, 1);
> +
> + /* Finally, ping the L2 cache controller */
> + writeq(1, l2c_ctrl.mbox_regs);
> +
> + local_irq_restore(irq_flags);
> +
> + l2_wait_completion(cmd_idx);
> +}
> +
> +void l2_cache_push_area_cmd(u64 cmd_type, phys_addr_t start,
> + unsigned long size)
> +{
> + if (WARN_ON(size == 0))
> + return;
> +
> + if (cmd_type == L2_CMD_OP_CMD_INVAL_AREA)
> + l2_cache_push_inval_cmd(start, size);
> + else
> + l2_cache_push_generic_cmd(cmd_type, start, size);
> +}
> +
> +static void __init l2_disp_error(u64 error)
> +{
> + const char *err_type;
> +
> + if (error & L2_ERROR_API_ERR_MASK)
> + err_type = "API";
> + else if (error & L2_ERROR_SETUP_ERR_MASK)
> + err_type = "SETUP";
> + else
> + err_type = "UNKNOWN";
> +
> + pr_err("%s error: 0x%llx\n", err_type, L2_ERROR(error));
> +}
> +
> +static int __init l2_cache_configure_mailboxes(void)
> +{
> + phys_addr_t l2_mbox_addr = 0;
> + void *cmd_regs = l2_cmd_regs_addr();
> +
> + /* We do not use mailbox to wait for completion, set it to 0 */
> + writeq(0, cmd_regs + L2_CMD_DOORBELL_READ_ADDR_OFFSET);
> +
> + /* Read mailbox address from L2 registers */
> + l2_mbox_addr = readq(cmd_regs + L2_CMD_DOORBELL_WRITE_ADDR_OFFSET);
> +
> + /* Then map the mailbox */
> + l2c_ctrl.mbox_regs = ioremap(l2_mbox_addr, PAGE_SIZE);
> + if (!l2c_ctrl.mbox_regs) {
> + pr_err("Failed to map mailbox\n");
> + return 1;
> + }
> +
> + /* Lock this entry into the LTLB */
> + kvx_mmu_ltlb_add_entry((unsigned long) l2c_ctrl.mbox_regs & PAGE_MASK,
> + l2_mbox_addr & PAGE_MASK,
> + PAGE_KERNEL_DEVICE, TLB_PS_4K);
> +
> + return 0;
> +}
> +
> +static int __init l2_cache_read_queue_size(void)
> +{
> + u64 inst;
> +
> + /* Read command queue size */
> + inst = readq(l2c_ctrl.regs + L2_INSTANCE_OFFSET);
> + l2c_ctrl.fifo_cmd_count = (inst & L2_INSTANCE_CMD_QUEUE_SIZE_MASK)
> + >> L2_INSTANCE_CMD_QUEUE_SIZE_SHIFT;
> +
> + /* Check if value is a power of two */
> + if (hweight64(l2c_ctrl.fifo_cmd_count) != 1) {
> + pr_err("Command queue size is not a power of two\n");
> + return 1;
> + }
> +
> + return 0;
> +}
> +
> +static int __init l2_cache_init_hw(void)
> +{
> + unsigned long timeout = jiffies + msecs_to_jiffies(L2_START_TIMEOUT_MS);
> + unsigned long flags;
> + u64 status, error;
> + int ret;
> +
> + /* Wait for L2 to be up */
> + do {
> + status = readq(l2c_ctrl.regs + L2_STATUS_OFFSET);
> + if (status & (L2_STATUS_READY_MASK | L2_STATUS_ERROR_MASK))
> + break;
> + } while (time_before(jiffies, timeout));
> +
> + if (!status) {
> + pr_err("Timeout while waiting for firmware status\n");
> + return -ENODEV;
> + }
> +
> + if (status & L2_STATUS_ERROR_MASK) {
> + error = readq(l2c_ctrl.regs + L2_ERROR_OFFSET);
> + l2_disp_error(error);
> + return -EINVAL;
> + }
> +
> + /* Now write ack to L2 firmware */
> + writeq(status | L2_STATUS_ACK_MASK, l2c_ctrl.regs + L2_STATUS_OFFSET);
> +
> + ret = l2_cache_read_queue_size();
> + if (ret)
> + return ret;
> +
> + ret = l2_cache_configure_mailboxes();
> + if (ret)
> + return ret;
> +
> + /* Enable the L2 atomically */
> + local_irq_save(flags);
> +
> + /* Fence data accesses */
> + kvx_fence();
> + /* Purge L1 */
> + l1_inval_dcache_all();
> + l1_inval_icache_all();
> + __builtin_kvx_barrier();
> +
> + local_irq_restore(flags);
> +
> + /* Enable L2$ */
> + kvx_sfr_set_field(PS, L2E, 1);
> +
> + return 0;
> +}
> +
> +static phys_addr_t __init l2_get_regs_addr(struct device_node *np)
> +{
> + const void *reg;
> + struct resource res;
> + phys_addr_t l2_regs_addr;
> + int ret;
> +
> + /*
> + * If regs is specified in device tree, then the L2$ has been loaded by
> + * someone else and not by ourself.
> + */
> + reg = of_get_property(np, "reg", NULL);
> + if (reg) {
> + ret = of_address_to_resource(np, 0, &res);
> + if (ret) {
> + pr_err("Address translation error\n");
> + return 0;
> + }
> + if ((res.end - res.start) > PAGE_SIZE) {
> + pr_err("L2 reg size > PAGE_SIZE\n");
> + return 0;
> + }
> +
> + l2_regs_addr = res.start;
> + } else {
> + l2_regs_addr = (phys_addr_t)
> __rm_firmware_regs_start;
Hi Yann !
Regarding the L2 cache support, since for the moment, only the QEMU
support is available and that KVX QEMU emulates a coherent L2 cache, I
think it would be preferable to remove this L2 cache driver for
multiple reasons:
- The firmware is still proprietary-sourced (at least to my knowledge)
and can not be obtained easily.
- L2 cache firmware needs to be linked with the kernel at compile time
if not already loaded by the RM.
If the RM is now already loaded with the L2 cache firmware at linux boot
time, then maybe this support can be kept but the "rm_firmware_regs"
section support can then be removed.
Clément
> + }
> +
> + if (!IS_ALIGNED(l2_regs_addr, PAGE_SIZE)) {
> + pr_err("Registers not aligned on PAGE_SIZE\n");
> + return 0;
> + }
> +
> + return l2_regs_addr;
> +}
> +
> +static int __init l2_cache_init(void)
> +{
> + int ret = -ENODEV;
> + struct device_node *np;
> + phys_addr_t l2_regs_addr;
> +
> + np = of_find_compatible_node(NULL, NULL, "kalray,kvx-l2-cache");
> + if (!np || !of_device_is_available(np)) {
> + if (!IS_ENABLED(CONFIG_SMP)) {
> + pr_info("controller disabled\n");
> + return 0;
> + }
> +
> + if (np && of_get_property(np, "kalray,is-qemu", NULL)) {
> + /*
> + * QEMU is always full cache coherent. The L2 cache controller is
> + * not strictly necessary to ensure coherency in SMP.
> + */
> + pr_info("controller disabled (QEMU detected)\n");
> + return 0;
> + }
> +
> + /* Else, SMP is enabled and L2 is mandatory for it */
> + goto err;
> + }
> +
> + l2_regs_addr = l2_get_regs_addr(np);
> + if (!l2_regs_addr)
> + goto err;
> +
> + /* Map the L2 registers */
> + l2c_ctrl.regs = ioremap(l2_regs_addr, PAGE_SIZE);
> + if (!l2c_ctrl.regs)
> + goto err;
> +
> + /* Lock this entry into the LTLB */
> + kvx_mmu_ltlb_add_entry((unsigned long) l2c_ctrl.regs, l2_regs_addr,
> + PAGE_KERNEL_NOCACHE, TLB_PS_4K);
> +
> + ret = l2_cache_init_hw();
> + if (ret) {
> + pr_err("Failed to init L2 cache controller");
> + goto err_unmap_l2;
> + }
> +
> + static_branch_enable(&l2_enabled);
> +
> + pr_info("controller enabled\n");
> +
> + return 0;
> +
> +err_unmap_l2:
> + kvx_mmu_ltlb_remove_entry((unsigned long) l2c_ctrl.regs);
> + iounmap(l2c_ctrl.regs);
> +err:
> + if (IS_ENABLED(CONFIG_SMP))
> + panic("L2$ controller is mandatory for SMP");
> +
> + return ret;
> +}
> +
> +
> +early_initcall(l2_cache_init);
> diff --git a/arch/kvx/kernel/smp.c b/arch/kvx/kernel/smp.c
> new file mode 100644
> index 000000000000..ed4c35a8c4bc
> --- /dev/null
> +++ b/arch/kvx/kernel/smp.c
> @@ -0,0 +1,110 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#include <linux/smp.h>
> +#include <linux/cpu.h>
> +#include <linux/of_irq.h>
> +#include <linux/cpumask.h>
> +#include <linux/irq_work.h>
> +#include <linux/mm_types.h>
> +#include <linux/interrupt.h>
> +
> +#include <asm/ipi.h>
> +#include <asm/tlbflush.h>
> +
> +enum ipi_message_type {
> + IPI_RESCHEDULE,
> + IPI_CALL_FUNC,
> + IPI_IRQ_WORK,
> + IPI_MAX
> +};
> +
> +/* A collection of single bit ipi messages. */
> +static struct {
> + unsigned long bits ____cacheline_aligned;
> +} ipi_data[NR_CPUS] __cacheline_aligned;
> +
> +static void send_ipi_message(const struct cpumask *mask,
> + enum ipi_message_type operation)
> +{
> + unsigned long flags;
> + int cpu;
> +
> + /* Set operation that must be done by receiver */
> + for_each_cpu(cpu, mask)
> + set_bit(operation, &ipi_data[cpu].bits);
> +
> + /* Commit the write before sending IPI */
> + smp_wmb();
> +
> + local_irq_save(flags);
> +
> + kvx_ipi_send(mask);
> +
> + local_irq_restore(flags);
> +}
> +
> +void arch_send_call_function_ipi_mask(struct cpumask *mask)
> +{
> + send_ipi_message(mask, IPI_CALL_FUNC);
> +}
> +
> +void arch_send_call_function_single_ipi(int cpu)
> +{
> + send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC);
> +}
> +
> +#ifdef CONFIG_IRQ_WORK
> +void arch_irq_work_raise(void)
> +{
> + send_ipi_message(cpumask_of(smp_processor_id()), IPI_IRQ_WORK);
> +}
> +#endif
> +
> +static void ipi_stop(void *unused)
> +{
> + local_cpu_stop();
> +}
> +
> +void smp_send_stop(void)
> +{
> + struct cpumask targets;
> +
> + cpumask_copy(&targets, cpu_online_mask);
> + cpumask_clear_cpu(smp_processor_id(), &targets);
> +
> + smp_call_function_many(&targets, ipi_stop, NULL, 0);
> +}
> +
> +void smp_send_reschedule(int cpu)
> +{
> + send_ipi_message(cpumask_of(cpu), IPI_RESCHEDULE);
> +}
> +
> +irqreturn_t ipi_call_interrupt(int irq, void *dev_id)
> +{
> + unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
> +
> + while (true) {
> + unsigned long ops = xchg(pending_ipis, 0);
> +
> + if (ops == 0)
> + return IRQ_HANDLED;
> +
> + if (ops & (1 << IPI_RESCHEDULE))
> + scheduler_ipi();
> +
> + if (ops & (1 << IPI_CALL_FUNC))
> + generic_smp_call_function_interrupt();
> +
> + if (ops & (1 << IPI_IRQ_WORK))
> + irq_work_run();
> +
> + BUG_ON((ops >> IPI_MAX) != 0);
> + }
> +
> + return IRQ_HANDLED;
> +}
> diff --git a/arch/kvx/kernel/smpboot.c b/arch/kvx/kernel/smpboot.c
> new file mode 100644
> index 000000000000..987a6f014163
> --- /dev/null
> +++ b/arch/kvx/kernel/smpboot.c
> @@ -0,0 +1,127 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + * Julian Vetter
> + */
> +
> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +#include <linux/io.h>
> +#include <linux/of.h>
> +#include <linux/smp.h>
> +#include <linux/cpu.h>
> +#include <linux/sched.h>
> +#include <linux/cpumask.h>
> +#include <linux/sched/mm.h>
> +#include <linux/mm_types.h>
> +#include <linux/of_platform.h>
> +#include <linux/sched/task_stack.h>
> +
> +#include <asm/pwr_ctrl.h>
> +#include <asm/tlbflush.h>
> +#include <asm/ipi.h>
> +
> +void *__cpu_up_stack_pointer[NR_CPUS];
> +void *__cpu_up_task_pointer[NR_CPUS];
> +
> +void __init smp_prepare_boot_cpu(void)
> +{
> +}
> +
> +int __cpu_up(unsigned int cpu, struct task_struct *tidle)
> +{
> + __cpu_up_stack_pointer[cpu] = task_stack_page(tidle) + THREAD_SIZE;
> + __cpu_up_task_pointer[cpu] = tidle;
> + /* We need to be sure writes are committed */
> + smp_mb();
> +
> + kvx_pwr_ctrl_cpu_poweron(cpu);
> + while (!cpu_online(cpu))
> + cpu_relax();
> +
> + return 0;
> +}
> +
> +void __init smp_cpus_done(unsigned int max_cpus)
> +{
> +}
> +
> +void __init smp_init_cpus(void)
> +{
> + struct cpumask cpumask;
> + struct device_node *cpu;
> + const __be32 *reg;
> + u32 cpu_num;
> + unsigned int nr_cpus = 0;
> +
> + cpumask_clear(&cpumask);
> +
> + for_each_of_cpu_node(cpu) {
> + if (!of_device_is_available(cpu))
> + continue;
> +
> + reg = of_get_property(cpu, "reg", NULL);
> + if (!reg)
> + continue;
> +
> + cpu_num = be32_to_cpup(reg);
> + if (cpu_num >= nr_cpu_ids)
> + continue;
> +
> + nr_cpus++;
> + cpumask_set_cpu(cpu_num, &cpumask);
> + }
> +
> + pr_info("%d possible cpus\n", nr_cpus);
> + init_cpu_possible(&cpumask);
> +}
> +
> +void __init smp_prepare_cpus(unsigned int max_cpus)
> +{
> + if (num_present_cpus() <= 1)
> + init_cpu_present(cpu_possible_mask);
> +}
> +
> +int __init setup_smp(void)
> +{
> + int ret;
> +
> + ret = kvx_pwr_ctrl_probe();
> + if (ret)
> + panic("Failed to probe power controller !");
> +
> + ret = kvx_ipi_ctrl_probe(ipi_call_interrupt);
> + if (ret)
> + panic("Failed to probe IPI controller !");
> +
> + return 0;
> +}
> +
> +early_initcall(setup_smp);
> +
> +/*
> + * C entry point for a secondary processor.
> + */
> +void __init start_kernel_secondary(void)
> +{
> + struct mm_struct *mm = &init_mm;
> + unsigned int cpu = smp_processor_id();
> +
> + setup_processor();
> + kvx_mmu_early_setup();
> +
> + /* All kernel threads share the same mm context. */
> + mmgrab(mm);
> + current->active_mm = mm;
> + cpumask_set_cpu(cpu, mm_cpumask(mm));
> +
> + notify_cpu_starting(cpu);
> + set_cpu_online(cpu, true);
> + trace_hardirqs_off();
> +
> + local_flush_tlb_all();
> +
> + local_irq_enable();
> + cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
> +}
> diff --git a/arch/kvx/platform/pwr_ctrl.c b/arch/kvx/platform/pwr_ctrl.c
> new file mode 100644
> index 000000000000..64c86cd18695
> --- /dev/null
> +++ b/arch/kvx/platform/pwr_ctrl.c
> @@ -0,0 +1,93 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +#include <linux/io.h>
> +#include <linux/slab.h>
> +#include <linux/types.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_platform.h>
> +
> +#include <asm/pwr_ctrl.h>
> +#include <asm/symbols.h>
> +
> +struct kvx_pwr_ctrl {
> + void __iomem *regs;
> +};
> +
> +static struct kvx_pwr_ctrl kvx_pwr_controller;
> +
> +/**
> + * kvx_pwr_ctrl_cpu_poweron() - Wakeup a cpu
> + * @cpu: cpu to wakeup
> + */
> +void kvx_pwr_ctrl_cpu_poweron(unsigned int cpu)
> +{
> + /* Set PE boot address */
> + writeq((unsigned long long)kvx_start,
> + kvx_pwr_controller.regs + KVX_PWR_CTRL_RESET_PC_OFFSET);
> + /* Wake up processor ! */
> + writeq(1ULL << cpu,
> + kvx_pwr_controller.regs + PWR_CTRL_WUP_SET_OFFSET);
> + /* Then clear wakeup to allow processor to sleep */
> + writeq(1ULL << cpu,
> + kvx_pwr_controller.regs + PWR_CTRL_WUP_CLEAR_OFFSET);
> +}
> +
> +static struct device_node * __init get_pwr_ctrl_node(void)
> +{
> + const phandle *ph;
> + struct device_node *cpu;
> + struct device_node *node;
> +
> + cpu = of_get_cpu_node(raw_smp_processor_id(), NULL);
> + if (!cpu) {
> + pr_err("Failed to get CPU node\n");
> + return NULL;
> + }
> +
> + ph = of_get_property(cpu, "power-controller", NULL);
> + if (!ph) {
> + pr_err("Failed to get power-controller phandle\n");
> + return NULL;
> + }
> +
> + node = of_find_node_by_phandle(be32_to_cpup(ph));
> + if (!node) {
> + pr_err("Failed to get power-controller node\n");
> + return NULL;
> + }
> +
> + return node;
> +}
> +
> +int __init kvx_pwr_ctrl_probe(void)
> +{
> + struct device_node *ctrl;
> +
> + ctrl = get_pwr_ctrl_node();
> + if (!ctrl) {
> + pr_err("Failed to get power controller node\n");
> + return -EINVAL;
> + }
> +
> + if (!of_device_is_compatible(ctrl, "kalray,kvx-pwr-ctrl")) {
> + pr_err("Failed to get power controller node\n");
> + return -EINVAL;
> + }
> +
> + kvx_pwr_controller.regs = of_iomap(ctrl, 0);
> + if (!kvx_pwr_controller.regs) {
> + pr_err("Failed ioremap\n");
> + return -EINVAL;
> + }
> +
> + pr_info("kvx power controller probed\n");
> +
> + return 0;
> +}
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index f61447913db9..f5a484547b15 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -152,6 +152,7 @@ enum cpuhp_state {
> CPUHP_AP_IRQ_RISCV_STARTING,
> CPUHP_AP_IRQ_LOONGARCH_STARTING,
> CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
> + CPUHP_AP_IRQ_KVX_STARTING,
> CPUHP_AP_ARM_MVEBU_COHERENCY,
> CPUHP_AP_MICROCODE_LOADER,
> CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING,
> @@ -189,6 +190,7 @@ enum cpuhp_state {
> CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING,
> CPUHP_AP_KVM_ARM_VGIC_STARTING,
> CPUHP_AP_KVM_ARM_TIMER_STARTING,
> + CPUHP_AP_KVX_TIMER_STARTING,
> /* Must be the last timer callback */
> CPUHP_AP_DUMMY_TIMER_STARTING,
> CPUHP_AP_ARM_XEN_STARTING,
--
Clément Léger,
Embedded Linux and Kernel engineer at Bootlin
https://bootlin.com
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 18/25] kvx: Add kvx default config file
2023-01-03 16:43 ` Yann Sionneau
` (18 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-04 13:02 ` Bagas Sanjaya
-1 siblings, 1 reply; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Thouvenin, Jules Maselbas,
Julian Vetter, Samuel Jones, Thomas Costis, Vincent Chardon
Add a default config file for kvx based Coolidge SoC.
CC: linux-kernel@vger.kernel.org
Co-developed-by: Ashley Lesdalons <alesdalons@kalray.eu>
Signed-off-by: Ashley Lesdalons <alesdalons@kalray.eu>
Co-developed-by: Benjamin Mugnier <mugnier.benjamin@gmail.com>
Signed-off-by: Benjamin Mugnier <mugnier.benjamin@gmail.com>
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Samuel Jones <sjones@kalray.eu>
Signed-off-by: Samuel Jones <sjones@kalray.eu>
Co-developed-by: Thomas Costis <tcostis@kalray.eu>
Signed-off-by: Thomas Costis <tcostis@kalray.eu>
Co-developed-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Signed-off-by: Vincent Chardon <vincent.chardon@elsys-design.com>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/configs/default_defconfig | 130 +++++++++++++++++++++++++++++
1 file changed, 130 insertions(+)
create mode 100644 arch/kvx/configs/default_defconfig
diff --git a/arch/kvx/configs/default_defconfig b/arch/kvx/configs/default_defconfig
new file mode 100644
index 000000000000..d05f1f3ccb8d
--- /dev/null
+++ b/arch/kvx/configs/default_defconfig
@@ -0,0 +1,130 @@
+CONFIG_DEFAULT_HOSTNAME="KVXlinux"
+CONFIG_EMBEDDED=y
+CONFIG_SERIAL_KVX_SCALL_COMM=y
+CONFIG_CONFIGFS_FS=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_DEBUG_INFO=y
+CONFIG_DEBUG_INFO_DWARF4=y
+CONFIG_PRINTK_TIME=y
+CONFIG_CONSOLE_LOGLEVEL_DEFAULT=15
+CONFIG_MESSAGE_LOGLEVEL_DEFAULT=7
+CONFIG_PANIC_TIMEOUT=-1
+CONFIG_BLK_DEV_INITRD=y
+CONFIG_GDB_SCRIPTS=y
+CONFIG_FRAME_POINTER=y
+CONFIG_HZ_100=y
+CONFIG_SERIAL_EARLYCON=y
+CONFIG_HOTPLUG_PCI_PCIE=y
+CONFIG_PCIEAER=y
+CONFIG_PCIE_DPC=y
+CONFIG_HOTPLUG_PCI=y
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SERIAL_8250_DW=y
+CONFIG_SERIAL_8250_NR_UARTS=8
+CONFIG_SERIAL_8250_RUNTIME_UARTS=8
+CONFIG_PINCTRL=y
+CONFIG_PINCTRL_SINGLE=y
+CONFIG_JUMP_LABEL=y
+CONFIG_POWER_RESET_KVX_SCALL_POWEROFF=y
+CONFIG_PCI=y
+CONFIG_PCI_MSI=y
+CONFIG_PCIE_KVX_NWL=y
+CONFIG_PCIEPORTBUS=y
+# CONFIG_PCIEASPM is not set
+CONFIG_PCIEAER_INJECT=y
+CONFIG_TMPFS=y
+CONFIG_DMADEVICES=y
+CONFIG_KVX_DMA_NOC=m
+CONFIG_KVX_IOMMU=y
+CONFIG_KVX_OTP_NV=y
+CONFIG_PACKET=y
+CONFIG_NET=y
+# CONFIG_WLAN is not set
+CONFIG_INET=y
+CONFIG_IPV6=y
+CONFIG_NETDEVICES=y
+CONFIG_NET_CORE=y
+CONFIG_E1000E=y
+CONFIG_BLK_DEV_NVME=y
+CONFIG_VFAT_FS=y
+CONFIG_NLS_DEFAULT="iso8859-1"
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_WATCHDOG=y
+CONFIG_KVX_WATCHDOG=y
+CONFIG_HUGETLBFS=y
+CONFIG_MAILBOX=y
+CONFIG_KVX_MBOX=y
+CONFIG_REMOTEPROC=y
+CONFIG_KVX_REMOTEPROC=y
+CONFIG_VIRTIO_NET=y
+CONFIG_VIRTIO_MMIO=y
+CONFIG_RPMSG_VIRTIO=y
+CONFIG_RPMSG_CHAR=y
+CONFIG_MODULES=y
+CONFIG_MODULE_UNLOAD=y
+CONFIG_MODVERSIONS=y
+CONFIG_MODULE_SRCVERSION_ALL=y
+CONFIG_BLK_DEV=y
+CONFIG_BLK_DEV_LOOP=m
+CONFIG_BLK_DEV_LOOP_MIN_COUNT=8
+CONFIG_EXT4_FS=m
+CONFIG_EXT4_USE_FOR_EXT2=y
+CONFIG_SYSVIPC=y
+CONFIG_UNIX=y
+CONFIG_NET_VENDOR_KALRAY=y
+CONFIG_NET_KVX_SOC=m
+CONFIG_STACKPROTECTOR=y
+CONFIG_TRANSPARENT_HUGEPAGE=y
+CONFIG_GPIO_DWAPB=y
+CONFIG_I2C=y
+CONFIG_I2C_SLAVE=y
+CONFIG_I2C_CHARDEV=y
+CONFIG_I2C_DESIGNWARE_PLATFORM=y
+CONFIG_I2C_DESIGNWARE_CORE=y
+CONFIG_I2C_DESIGNWARE_SLAVE=y
+CONFIG_I2C_SLAVE_USPACE=y
+CONFIG_POWER_RESET=y
+CONFIG_POWER_RESET_SYSCON=y
+CONFIG_SPI=y
+CONFIG_SPI_DESIGNWARE=y
+CONFIG_SPI_DW_MMIO=y
+CONFIG_SPI_DW_KVX=y
+CONFIG_MTD=y
+CONFIG_MTD_SPI_NOR=y
+# CONFIG_MTD_SPI_NOR_USE_4K_SECTORS is not set
+CONFIG_SQUASHFS=m
+CONFIG_USB=y
+CONFIG_USB_CONFIGFS=m
+CONFIG_USB_CONFIGFS_ACM=y
+CONFIG_USB_CONFIGFS_ECM=y
+CONFIG_USB_DWC2=y
+CONFIG_USB_DWC2_DUAL_ROLE=y
+CONFIG_USB_GADGET=y
+CONFIG_U_SERIAL_CONSOLE=y
+CONFIG_USB_USBNET=m
+CONFIG_USB_NET_SMSC95XX=m
+# CONFIG_NOP_USB_XCEIV is not set
+CONFIG_USB_PHY=y
+CONFIG_GENERIC_PHY=y
+CONFIG_GENERIC_PHY_USB=y
+CONFIG_MMC=y
+CONFIG_MMC_SDHCI=y
+CONFIG_MMC_SDHCI_PLTFM=y
+CONFIG_MMC_SDHCI_OF_DWCMSHC=y
+CONFIG_MDIO_BITBANG=m
+CONFIG_MDIO_GPIO=m
+CONFIG_MARVELL_PHY=m
+CONFIG_GPIO_PCA953X=y
+CONFIG_NETFILTER=y
+CONFIG_NF_CONNTRACK=m
+CONFIG_NF_NAT=m
+CONFIG_IP_NF_IPTABLES=m
+CONFIG_IP_NF_NAT=m
+CONFIG_LEDS_GPIO=y
+CONFIG_LEDS_CLASS=y
+CONFIG_LEDS_TRIGGERS=y
+CONFIG_LEDS_TRIGGER_NETDEV=y
+CONFIG_LEDS_TRIGGER_PATTERN=y
+CONFIG_DCB=y
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 18/25] kvx: Add kvx default config file
2023-01-03 16:43 ` [RFC PATCH 18/25] kvx: Add kvx default config file Yann Sionneau
@ 2023-01-04 13:02 ` Bagas Sanjaya
2023-01-06 14:52 ` Jules Maselbas
0 siblings, 1 reply; 113+ messages in thread
From: Bagas Sanjaya @ 2023-01-04 13:02 UTC (permalink / raw)
To: Yann Sionneau
Cc: linux-kernel, Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Thouvenin, Jules Maselbas, Julian Vetter, Samuel Jones,
Thomas Costis, Vincent Chardon
[-- Attachment #1: Type: text/plain, Size: 383 bytes --]
On Tue, Jan 03, 2023 at 05:43:52PM +0100, Yann Sionneau wrote:
> arch/kvx/configs/default_defconfig | 130 +++++++++++++++++++++++++++++
> 1 file changed, 130 insertions(+)
> create mode 100644 arch/kvx/configs/default_defconfig
>
Shouldn't it be arch/kvx/configs/defconfig to be consistent with other
archs?
--
An old man doll... just what I always wanted! - Clara
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 18/25] kvx: Add kvx default config file
2023-01-04 13:02 ` Bagas Sanjaya
@ 2023-01-06 14:52 ` Jules Maselbas
0 siblings, 0 replies; 113+ messages in thread
From: Jules Maselbas @ 2023-01-06 14:52 UTC (permalink / raw)
To: Bagas Sanjaya
Cc: Yann Sionneau, linux-kernel, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Thouvenin, Julian Vetter, Samuel Jones,
Thomas Costis, Vincent Chardon
On Wed, Jan 04, 2023 at 08:02:43PM +0700, Bagas Sanjaya wrote:
> On Tue, Jan 03, 2023 at 05:43:52PM +0100, Yann Sionneau wrote:
> > arch/kvx/configs/default_defconfig | 130 +++++++++++++++++++++++++++++
> > 1 file changed, 130 insertions(+)
> > create mode 100644 arch/kvx/configs/default_defconfig
> >
>
> Shouldn't it be arch/kvx/configs/defconfig to be consistent with other
> archs?
Yes, we are going to use defconfig instead of default_defconfig (which
doesn't make much sens)
Thanks,
-- Jules
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 19/25] kvx: power: scall poweroff driver
2023-01-03 16:43 ` Yann Sionneau
` (19 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-04 17:08 ` Sebastian Reichel
-1 siblings, 1 reply; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Sebastian Reichel, linux-kernel, linux-pm,
Clement Leger, Julian Vetter
Add a driver to poweroff the Coolidge SoC
when running under Qemu, ISS or when
the debugger (jtag-runner) runs on PL0
to catch the scall.
CC: Sebastian Reichel <sre@kernel.org>
CC: linux-kernel@vger.kernel.org
CC: linux-pm@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
drivers/power/reset/kvx-scall-poweroff.c | 53 ++++++++++++++++++++++++
1 file changed, 53 insertions(+)
create mode 100644 drivers/power/reset/kvx-scall-poweroff.c
diff --git a/drivers/power/reset/kvx-scall-poweroff.c b/drivers/power/reset/kvx-scall-poweroff.c
new file mode 100644
index 000000000000..586d93fbcaed
--- /dev/null
+++ b/drivers/power/reset/kvx-scall-poweroff.c
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2017 - 2022 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/pm.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+
+#define SCALL_NUM_EXIT "0xfff"
+
+static void kvx_scall_poweroff(void)
+{
+ register int status asm("r0") = 0;
+
+ asm volatile ("scall " SCALL_NUM_EXIT "\n\t;;"
+ : /* out */
+ : "r"(status));
+
+ unreachable();
+}
+
+static int kvx_scall_poweroff_probe(struct platform_device *pdev)
+{
+ pm_power_off = kvx_scall_poweroff;
+
+ return 0;
+}
+
+static int kvx_scall_poweroff_remove(struct platform_device *pdev)
+{
+ if (pm_power_off == kvx_scall_poweroff)
+ pm_power_off = NULL;
+
+ return 0;
+}
+
+static const struct of_device_id kvx_scall_poweroff_of_match[] = {
+ { .compatible = "kalray,kvx-scall-poweroff" },
+ {}
+};
+
+static struct platform_driver kvx_scall_poweroff_driver = {
+ .probe = kvx_scall_poweroff_probe,
+ .remove = kvx_scall_poweroff_remove,
+ .driver = {
+ .name = "kvx-scall-poweroff",
+ .of_match_table = kvx_scall_poweroff_of_match,
+ },
+};
+module_platform_driver(kvx_scall_poweroff_driver);
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 19/25] kvx: power: scall poweroff driver
2023-01-03 16:43 ` [RFC PATCH 19/25] kvx: power: scall poweroff driver Yann Sionneau
@ 2023-01-04 17:08 ` Sebastian Reichel
0 siblings, 0 replies; 113+ messages in thread
From: Sebastian Reichel @ 2023-01-04 17:08 UTC (permalink / raw)
To: Yann Sionneau; +Cc: linux-kernel, linux-pm, Clement Leger, Julian Vetter
[-- Attachment #1: Type: text/plain, Size: 2539 bytes --]
Hi,
On Tue, Jan 03, 2023 at 05:43:53PM +0100, Yann Sionneau wrote:
> Add a driver to poweroff the Coolidge SoC
> when running under Qemu, ISS or when
> the debugger (jtag-runner) runs on PL0
> to catch the scall.
line length of 41 characters is quite short.
> CC: Sebastian Reichel <sre@kernel.org>
> CC: linux-kernel@vger.kernel.org
> CC: linux-pm@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> drivers/power/reset/kvx-scall-poweroff.c | 53 ++++++++++++++++++++++++
> 1 file changed, 53 insertions(+)
> create mode 100644 drivers/power/reset/kvx-scall-poweroff.c
>
> diff --git a/drivers/power/reset/kvx-scall-poweroff.c b/drivers/power/reset/kvx-scall-poweroff.c
> new file mode 100644
> index 000000000000..586d93fbcaed
> --- /dev/null
> +++ b/drivers/power/reset/kvx-scall-poweroff.c
> @@ -0,0 +1,53 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (c) 2017 - 2022 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#include <linux/pm.h>
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +
> +#define SCALL_NUM_EXIT "0xfff"
> +
> +static void kvx_scall_poweroff(void)
> +{
> + register int status asm("r0") = 0;
> +
> + asm volatile ("scall " SCALL_NUM_EXIT "\n\t;;"
> + : /* out */
> + : "r"(status));
> +
> + unreachable();
> +}
> +
> +static int kvx_scall_poweroff_probe(struct platform_device *pdev)
> +{
> + pm_power_off = kvx_scall_poweroff;
> +
> + return 0;
> +}
Please use devm_register_power_off_handler().
-- Sebastian
> +static int kvx_scall_poweroff_remove(struct platform_device *pdev)
> +{
> + if (pm_power_off == kvx_scall_poweroff)
> + pm_power_off = NULL;
> +
> + return 0;
> +}
> +
> +static const struct of_device_id kvx_scall_poweroff_of_match[] = {
> + { .compatible = "kalray,kvx-scall-poweroff" },
> + {}
> +};
> +
> +static struct platform_driver kvx_scall_poweroff_driver = {
> + .probe = kvx_scall_poweroff_probe,
> + .remove = kvx_scall_poweroff_remove,
> + .driver = {
> + .name = "kvx-scall-poweroff",
> + .of_match_table = kvx_scall_poweroff_of_match,
> + },
> +};
> +module_platform_driver(kvx_scall_poweroff_driver);
> --
> 2.37.2
>
>
>
>
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 20/25] kvx: gdb: add kvx related gdb helpers
2023-01-03 16:43 ` Yann Sionneau
` (20 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-04 7:41 ` Jan Kiszka
-1 siblings, 1 reply; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Jan Kiszka, Kieran Bingham, linux-kernel,
Clement Leger, Guillaume Thouvenin
Add kvx related gdb helpers:
* lx-kvx-tlb-access
* lx-kvx-tlb-decode
* lx-kvx-page-table-walk
* lx-kvx-virt-to-phys
CC: Jan Kiszka <jan.kiszka@siemens.com>
CC: Kieran Bingham <kbingham@kernel.org>
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
scripts/gdb/arch/Makefile | 11 ++
scripts/gdb/arch/__init__.py | 1 +
scripts/gdb/arch/kvx/Makefile | 25 +++
scripts/gdb/arch/kvx/__init__.py | 1 +
scripts/gdb/arch/kvx/constants.py.in | 74 +++++++++
scripts/gdb/arch/kvx/mmu.py | 199 +++++++++++++++++++++++
scripts/gdb/arch/kvx/page_table_walk.py | 207 ++++++++++++++++++++++++
7 files changed, 518 insertions(+)
create mode 100644 scripts/gdb/arch/Makefile
create mode 100644 scripts/gdb/arch/__init__.py
create mode 100644 scripts/gdb/arch/kvx/Makefile
create mode 100644 scripts/gdb/arch/kvx/__init__.py
create mode 100644 scripts/gdb/arch/kvx/constants.py.in
create mode 100644 scripts/gdb/arch/kvx/mmu.py
create mode 100644 scripts/gdb/arch/kvx/page_table_walk.py
diff --git a/scripts/gdb/arch/Makefile b/scripts/gdb/arch/Makefile
new file mode 100644
index 000000000000..25e59accb884
--- /dev/null
+++ b/scripts/gdb/arch/Makefile
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0
+subdir-y := kvx
+
+always-y := gdb-scripts
+
+SRCTREE := $(abspath $(srctree))
+
+$(obj)/gdb-scripts:
+ifdef building_out_of_srctree
+ $(Q)ln -fsn $(SRCTREE)/$(obj)/*.py $(objtree)/$(obj)
+endif
diff --git a/scripts/gdb/arch/__init__.py b/scripts/gdb/arch/__init__.py
new file mode 100644
index 000000000000..4680fb176337
--- /dev/null
+++ b/scripts/gdb/arch/__init__.py
@@ -0,0 +1 @@
+# nothing to do for the initialization of this package
diff --git a/scripts/gdb/arch/kvx/Makefile b/scripts/gdb/arch/kvx/Makefile
new file mode 100644
index 000000000000..124755087510
--- /dev/null
+++ b/scripts/gdb/arch/kvx/Makefile
@@ -0,0 +1,25 @@
+# SPDX-License-Identifier: GPL-2.0
+
+ifdef building_out_of_srctree
+
+symlinks := $(patsubst $(srctree)/$(src)/%,%,$(wildcard $(srctree)/$(src)/*.py))
+
+quiet_cmd_symlink = SYMLINK $@
+ cmd_symlink = ln -fsn $(patsubst $(obj)/%,$(abspath $(srctree))/$(src)/%,$@) $@
+
+extra-y += $(symlinks)
+$(addprefix $(obj)/, $(symlinks)): FORCE
+ $(call if_changed,symlink)
+
+endif
+
+quiet_cmd_gen_constants_py = GEN $@
+ cmd_gen_constants_py = \
+ $(CPP) -E -x c -P $(c_flags) $< > $@ ;\
+ sed -i '1,/<!-- end-c-headers -->/d;' $@
+
+extra-y += constants.py
+$(obj)/constants.py: $(src)/constants.py.in FORCE
+ $(call if_changed_dep,gen_constants_py)
+
+clean-files := *.pyc *.pyo
diff --git a/scripts/gdb/arch/kvx/__init__.py b/scripts/gdb/arch/kvx/__init__.py
new file mode 100644
index 000000000000..4680fb176337
--- /dev/null
+++ b/scripts/gdb/arch/kvx/__init__.py
@@ -0,0 +1 @@
+# nothing to do for the initialization of this package
diff --git a/scripts/gdb/arch/kvx/constants.py.in b/scripts/gdb/arch/kvx/constants.py.in
new file mode 100644
index 000000000000..861f630c3a79
--- /dev/null
+++ b/scripts/gdb/arch/kvx/constants.py.in
@@ -0,0 +1,74 @@
+/*
+ * gdb helper commands and functions for Linux kernel debugging
+ *
+ * Kernel constants derived from include files.
+ *
+ * Copyright (c) 2016 Linaro Ltd
+ *
+ * Authors:
+ * Kieran Bingham <kieran.bingham@linaro.org>
+ *
+ * This work is licensed under the terms of the GNU GPL version 2.
+ *
+ */
+
+#include <asm/tlb_defs.h>
+#include <asm/page.h>
+#include <asm/mmu.h>
+#include <asm/pgtable.h>
+
+#define LX_VALUE(x) LX_##x = x
+
+/* The build system will take care of deleting everything above this marker */
+<!-- end-c-headers -->
+
+LX_VALUE(TLB_ES_INVALID)
+LX_VALUE(TLB_ES_PRESENT)
+LX_VALUE(TLB_ES_MODIFIED)
+LX_VALUE(TLB_ES_A_MODIFIED)
+
+LX_VALUE(TLB_CP_D_U)
+LX_VALUE(TLB_CP_U_U)
+LX_VALUE(TLB_CP_W_C)
+LX_VALUE(TLB_CP_U_C)
+
+LX_VALUE(TLB_PA_NA_NA)
+LX_VALUE(TLB_PA_NA_R)
+LX_VALUE(TLB_PA_NA_RW)
+LX_VALUE(TLB_PA_NA_RX)
+LX_VALUE(TLB_PA_NA_RWX)
+LX_VALUE(TLB_PA_R_R)
+LX_VALUE(TLB_PA_R_RW)
+LX_VALUE(TLB_PA_R_RX)
+LX_VALUE(TLB_PA_R_RWX)
+LX_VALUE(TLB_PA_RW_RW)
+LX_VALUE(TLB_PA_RW_RWX)
+LX_VALUE(TLB_PA_RX_RX)
+LX_VALUE(TLB_PA_RX_RWX)
+LX_VALUE(TLB_PA_RWX_RWX)
+
+LX_VALUE(TLB_PS_4K)
+LX_VALUE(TLB_PS_64K)
+LX_VALUE(TLB_PS_2M)
+LX_VALUE(TLB_PS_512M)
+
+LX_VALUE(TLB_G_GLOBAL)
+LX_VALUE(TLB_G_USE_ASN)
+
+#ifdef CONFIG_KVX_DEBUG_TLB_ACCESS_BITS
+LX_VALUE(CONFIG_KVX_DEBUG_TLB_ACCESS_BITS)
+#endif
+LX_VALUE(KVX_TLB_ACCESS_READ)
+LX_VALUE(KVX_TLB_ACCESS_WRITE)
+LX_VALUE(KVX_TLB_ACCESS_PROBE)
+
+/* asm/page.h */
+LX_VALUE(PAGE_SHIFT)
+
+/* asm/pgtable.h */
+LX_VALUE(PGDIR_BITS)
+LX_VALUE(PMD_BITS)
+LX_VALUE(PTE_BITS)
+LX_VALUE(_PAGE_HUGE)
+
+LX_VALUE(PA_TO_VA_OFFSET)
diff --git a/scripts/gdb/arch/kvx/mmu.py b/scripts/gdb/arch/kvx/mmu.py
new file mode 100644
index 000000000000..0f2f2523a207
--- /dev/null
+++ b/scripts/gdb/arch/kvx/mmu.py
@@ -0,0 +1,199 @@
+import gdb
+import re
+import ctypes
+from arch.kvx import constants
+from linux import cpus
+
+ps_value = {
+ constants.LX_TLB_PS_4K: '4K',
+ constants.LX_TLB_PS_64K: '64K',
+ constants.LX_TLB_PS_2M: '2M',
+ constants.LX_TLB_PS_512M: '512M',
+}
+
+ps_shift = {
+ constants.LX_TLB_PS_4K: 12,
+ constants.LX_TLB_PS_64K: 16,
+ constants.LX_TLB_PS_2M: 21,
+ constants.LX_TLB_PS_512M: 29,
+}
+
+g_value = {
+ constants.LX_TLB_G_USE_ASN: 'Use ASN',
+ constants.LX_TLB_G_GLOBAL: 'Global',
+}
+
+es_value = {
+ constants.LX_TLB_ES_INVALID: 'Invalid',
+ constants.LX_TLB_ES_PRESENT: 'Present',
+ constants.LX_TLB_ES_MODIFIED: 'Modified',
+ constants.LX_TLB_ES_A_MODIFIED: 'A-Modified',
+}
+
+cp_value = {
+ constants.LX_TLB_CP_D_U: 'Device/Uncached',
+ constants.LX_TLB_CP_U_U: 'Uncached/Uncached',
+ constants.LX_TLB_CP_W_C: 'WriteThrough/Cached',
+ constants.LX_TLB_CP_U_C: 'Unchached/Cached',
+}
+
+pa_value = {
+ constants.LX_TLB_PA_NA_NA: 'NA_NA',
+ constants.LX_TLB_PA_NA_R: 'NA_R',
+ constants.LX_TLB_PA_NA_RW: 'NA_RW',
+ constants.LX_TLB_PA_NA_RX: 'NA_RX',
+ constants.LX_TLB_PA_NA_RWX: 'NA_RWX',
+ constants.LX_TLB_PA_R_R: 'R_R',
+ constants.LX_TLB_PA_R_RW: 'R_RW',
+ constants.LX_TLB_PA_R_RX: 'R_RX',
+ constants.LX_TLB_PA_R_RWX: 'R_RWX',
+ constants.LX_TLB_PA_RW_RW: 'RW_RW',
+ constants.LX_TLB_PA_RW_RWX: 'RW_RWX',
+ constants.LX_TLB_PA_RX_RX: 'RX_RX',
+ constants.LX_TLB_PA_RX_RWX: 'RX_RWX',
+ constants.LX_TLB_PA_RWX_RWX: 'RWX_RWX',
+}
+
+access_type = {
+ constants.LX_KVX_TLB_ACCESS_READ: "READ",
+ constants.LX_KVX_TLB_ACCESS_WRITE: "WRITE",
+ constants.LX_KVX_TLB_ACCESS_PROBE: "PROBE",
+}
+
+def tlb_access_get_idx(idx):
+ return idx & ((1 << constants.LX_CONFIG_KVX_DEBUG_TLB_ACCESS_BITS) - 1)
+
+def computed_set(tel, teh):
+ # For a page of size 2^n (with n >= 12), the index of the set is
+ # determined by the value of the bit slice comprising bits n to n+5 of
+ # the virtual address corresponding to the start of the page
+ page_size = int(tel.cast(gdb.lookup_type("struct tlb_entry_low"))['ps'])
+ teh_val = long(teh.cast(gdb.lookup_type("uint64_t")))
+ return (teh_val >> ps_shift[page_size]) & ((1 << 5) -1)
+
+def tlb_access_objdump(obj, name):
+ gdb.write(" {}\t|".format(name))
+ for f in obj.type.fields():
+ try:
+ val = int(obj[f.name])
+ gdb.write('\t{}:0x{:x}'.format(f.name, val))
+ except:
+ gdb.write('\t{}:{}'.format(f.name, obj[f.name]))
+
+ gdb.write("\n")
+
+# This return the tuple (type, mmc, tel, teh)
+def tlb_access_get_val(cpu, idx):
+ var_ptr = gdb.parse_and_eval("&kvx_tlb_access_rb")
+ return (
+ cpus.per_cpu(var_ptr, cpu)[idx]['type'],
+ cpus.per_cpu(var_ptr, cpu)[idx]['mmc'],
+ cpus.per_cpu(var_ptr, cpu)[idx]['entry']['tel'],
+ cpus.per_cpu(var_ptr, cpu)[idx]['entry']['teh']
+ )
+
+class TLBaccess(gdb.Command):
+ """Dump the last commands exectued when accessing the TLB. The number of
+ commands to dump is passed as the first argument and the CPU is given
+ as the second argument. By default the number of entries dump is 10 and
+ CPU is 0.
+ """
+
+ def __init__(self):
+ super(TLBaccess, self).__init__("lx-kvx-tlb-access", gdb.COMMAND_DATA)
+
+ def invoke(self, arg, from_tty):
+
+ dump = 10
+ cpu = 0
+ argv = gdb.string_to_argv(arg)
+
+ try:
+ constants.LX_CONFIG_KVX_DEBUG_TLB_ACCESS_BITS
+ except AttributeError:
+ gdb.write("Kernel was not compiled with KVX_DEBUG_TLB_ACCESS_BITS.\n")
+ gdb.write("This is required to get information about access to TLB.\n")
+ return
+
+ return
+
+ if (len(argv) >= 1):
+ try:
+ dump = int(argv[0])
+ except:
+ gdb.write("WARNING: dump given as argument is not an integer, 10 is used\n")
+
+ if (len(argv) >= 2):
+ try:
+ cpu = int(argv[1])
+ except:
+ gdb.write("WARNING: cpu given as argument is not an integer, 0 is used\n")
+
+ # access_idx is the next id that will be written. There is no value at
+ # this index.
+ try:
+ var_ptr = gdb.parse_and_eval("&kvx_tlb_access_idx")
+ access_idx = int(cpus.per_cpu(var_ptr, cpu))
+ except:
+ gdb.write("Failed to get current index for the buffer. Abort\n")
+ return
+
+ for i in range(0, dump):
+ current_idx = access_idx - i
+
+ # When idx 0 is reach we need to check if we wrapped or not. If we
+ # wrapped then there is no more operations logged.
+ if (current_idx == 0) and (current_idx == tlb_access_get_idx(current_idx)):
+ gdb.write("no more operations occurred in TLB for cpu {}\n".format(cpu))
+ break
+
+ real_idx = tlb_access_get_idx(current_idx - 1)
+
+ e_type, e_mmc, e_tel, e_teh = tlb_access_get_val(cpu, real_idx)
+ access_type_str = access_type.get(int(e_type), "UNKNOWN")
+
+ gdb.write('CPU: {} \n'.format(cpu))
+ if (access_type_str == "WRITE"):
+ gdb.write(" TYPE\t|\tWRITE - computed set {}\n".format(computed_set(e_tel, e_teh)))
+ else:
+ gdb.write(" TYPE\t|\t{}\n".format(access_type_str))
+
+ mmc = e_mmc.cast(gdb.lookup_type("struct mmc_t"))
+ tlb_access_objdump(mmc, "MMC")
+
+ tel = e_tel.cast(gdb.lookup_type("struct tlb_entry_low"))
+ tlb_access_objdump(tel, "TEL")
+
+ teh = e_teh.cast(gdb.lookup_type("struct tlb_entry_high"))
+ tlb_access_objdump(teh, "TEH")
+
+ gdb.write('\n')
+
+TLBaccess()
+
+class LxDecodetlb(gdb.Command):
+ """Decode $tel and $teh values
+ """
+
+ def __init__(self):
+ super(LxDecodetlb, self).__init__("lx-kvx-tlb-decode", gdb.COMMAND_DATA)
+
+ def invoke(self, argument, from_tty):
+ tel_val = gdb.parse_and_eval("$tel")
+ gdb.write('tel:\n\tes:\t{es}\n\tcp:\t{cp}\n\tpa:\t{pa}\n\tps:\t{ps}\n\tfn:\t0x{fn:016x}\n'.format(
+ es=es_value[int(tel_val["es"])],
+ cp=cp_value[int(tel_val["cp"])],
+ pa=pa_value[int(tel_val["pa"])],
+ ps=ps_value[int(tel_val["ps"])],
+ fn=int(tel_val["fn"]))
+ )
+
+ teh_val = gdb.parse_and_eval("$teh")
+ gdb.write('teh:\n\tasn:\t{asn}\n\tg:\t{g}\n\tvs:\t{vs}\n\tpn:\t0x{pn:016x}\n'.format(
+ asn=teh_val["asn"],
+ g=g_value[int(teh_val["g"])],
+ vs=teh_val["vs"],
+ pn=int(teh_val["pn"]))
+ )
+
+LxDecodetlb()
diff --git a/scripts/gdb/arch/kvx/page_table_walk.py b/scripts/gdb/arch/kvx/page_table_walk.py
new file mode 100644
index 000000000000..c2f4a21907d1
--- /dev/null
+++ b/scripts/gdb/arch/kvx/page_table_walk.py
@@ -0,0 +1,207 @@
+import gdb
+import re
+import ctypes
+from arch.kvx import constants
+#
+# PTE format:
+# +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+# | 63..23 | 22..13 | 12 | 11..10 | 9 | 8 | 7 | 6 | 5 | 4 | 3..2 | 1 | 0 |
+# +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+
+# PFN Unused S PageSZ H G X W R D CP A P
+
+class pte_bits(ctypes.LittleEndianStructure):
+ _fields_ = [
+ ("P", ctypes.c_uint8, 1),
+ ("A", ctypes.c_uint8, 1),
+ ("CP", ctypes.c_uint8, 2),
+ ("D", ctypes.c_uint8, 1),
+ ("R", ctypes.c_uint8, 1),
+ ("W", ctypes.c_uint8, 1),
+ ("X", ctypes.c_uint8, 1),
+ ("G", ctypes.c_uint8, 1),
+ ("H", ctypes.c_uint8, 1),
+ ("PageSZ", ctypes.c_uint8, 2),
+ ("S", ctypes.c_uint8, 1),
+ ("Unused", ctypes.c_uint16, 10),
+ ("PFN", ctypes.c_uint64, 41),
+ ]
+
+class Pte(ctypes.Union):
+ _fields_ = [("bf", pte_bits),
+ ("value", ctypes.c_uint64)]
+
+ def __init__(self, pteValue):
+ self.value = pteValue
+
+ def phys_addr(self):
+ return self.bf.PFN << constants.LX_PAGE_SHIFT
+
+ def __str__(self):
+ pte_str = "PFN: 0x{:016x}, ".format(self.phys_addr())
+ pte_str += "PS: " + page_sz_str[self.bf.PageSZ] + ", bits: "
+ ignore_fields = ["PFN", "Unused", "PageSZ"]
+
+ for field in reversed(self.bf._fields_):
+ if field[0] in ignore_fields:
+ continue
+
+ if long(getattr(self.bf, field[0])) != 0:
+ pte_str += field[0] + " "
+
+ return pte_str
+
+SIGN_EXT_BITS = 64 - constants.LX_PAGE_SHIFT - constants.LX_PTE_BITS - constants.LX_PMD_BITS - constants.LX_PGDIR_BITS
+
+class virt_addr_bits(ctypes.LittleEndianStructure):
+ _fields_ = [
+ ("page_off", ctypes.c_uint64, constants.LX_PAGE_SHIFT),
+ ("pte_idx", ctypes.c_uint64, constants.LX_PTE_BITS),
+ ("pmd_idx", ctypes.c_uint64, constants.LX_PMD_BITS),
+ ("pgd_idx", ctypes.c_uint64, constants.LX_PGDIR_BITS),
+ ("sign_extension", ctypes.c_uint64, SIGN_EXT_BITS),
+ ]
+
+class VirtAddr(ctypes.Union):
+ _fields_ = [("bf", virt_addr_bits),
+ ("value", ctypes.c_uint64)]
+
+ def __init__(self, virtAddr):
+ self.value = virtAddr
+
+PGD_ENTRIES = 1 << constants.LX_PGDIR_BITS
+PMD_ENTRIES = 1 << constants.LX_PMD_BITS
+PTE_ENTRIES = 1 << constants.LX_PTE_BITS
+
+page_sz_str = {
+ constants.LX_TLB_PS_4K : "4 Ko",
+ constants.LX_TLB_PS_64K: "64 Ko",
+ constants.LX_TLB_PS_2M: "2 Mo",
+ constants.LX_TLB_PS_512M: "512 Mo"
+}
+
+unsigned_long_ptr = gdb.lookup_type('unsigned long').pointer()
+
+def unsigned_long(gdb_val):
+ """Convert a unsigned long gdb.Value to a displayable python value
+ """
+ return long(gdb_val) & 0xFFFFFFFFFFFFFFFF
+
+def do_lookup(base, size=512):
+ """Lookup a page table and return an array of pair with index:entry_value
+ """
+ base_ptr = base.cast(unsigned_long_ptr)
+
+ res = []
+
+ for entry in range(size):
+ entry_val = base_ptr.dereference()
+ base_ptr += 1
+ if entry_val != 0:
+ res.append((entry, entry_val))
+
+ return res
+
+def phys_to_virt(virt):
+ return virt + constants.LX_PA_TO_VA_OFFSET
+
+class LxPageTableWalk(gdb.Command):
+ """Looks for entries in the page table. The base address of the PGD is
+ the argument.
+ """
+
+ def __init__(self):
+ super(LxPageTableWalk, self).__init__("lx-kvx-page-table-walk", gdb.COMMAND_DATA)
+
+ def invoke(self, argument, from_tty):
+ argv = gdb.string_to_argv(argument)
+
+ if (len(argv) == 1):
+ pgd = argv[0]
+ else:
+ pgd = "$lx_current().active_mm.pgd"
+
+ val = gdb.parse_and_eval(pgd)
+ pgd = unsigned_long(val..cast(unsigned_long_ptr))
+
+ print "> Looking for PGD base 0x{:016x}\n".format(pgd)
+
+ for pgd_pair in do_lookup(val, PGD_ENTRIES):
+ gdb.write("[{}] -> Entry[0x{:016x}]\n".format(
+ pgd_pair[0], unsigned_long(pgd_pair[1])))
+ gdb.write("\t> Looking for PMD base 0x{:016x}\n".format(unsigned_long(pgd_pair[1])))
+ for pmd_pair in do_lookup(phys_to_virt(pgd_pair[1]), PMD_ENTRIES):
+ gdb.write("\t[{}] -> Entry[0x{:016x}]\n".format(
+ pmd_pair[0], unsigned_long(pmd_pair[1])))
+ # Check if the PMD value read is a huge page or a pointer to a
+ # PTE base.
+ if (unsigned_long(pmd_pair[1]) & constants.LX__PAGE_HUGE):
+ gdb.write("\t\t> Huge PTE: ")
+ gdb.write(str(Pte(pmd_pair[1])) + "\n")
+ else:
+ gdb.write("\t\t> Looking for PTE base 0x{:016x}\n".format(unsigned_long(pmd_pair[1])))
+ for pte_pair in do_lookup(phys_to_virt(pmd_pair[1]), PTE_ENTRIES):
+ gdb.write("\t\t[{}] -> PTE [0x{:016x}]\n".format(
+ pte_pair[0], unsigned_long(pte_pair[1])))
+ gdb.write("\t\t" + str(Pte(pte_pair[1]))+ "\n")
+
+LxPageTableWalk()
+
+ps_shift = {
+ constants.LX_TLB_PS_4K: 12,
+ constants.LX_TLB_PS_64K: 16,
+ constants.LX_TLB_PS_2M: 21,
+ constants.LX_TLB_PS_512M: 29,
+}
+
+def mask_addr(addr, shift):
+ return addr & ((1 << shift) - 1)
+
+def lookup_entry(base, idx):
+ entry_addr = base.cast(unsigned_long_ptr) + idx
+ res_addr = entry_addr.dereference()
+ if res_addr == 0:
+ raise
+
+ return res_addr
+
+class LxVirtToPhys(gdb.Command):
+ """Translate a virtual address to a physical one by walking the page table
+ """
+
+ def __init__(self):
+ super(LxVirtToPhys, self).__init__("lx-kvx-virt-to-phys", gdb.COMMAND_DATA)
+
+ def invoke(self, argument, from_tty):
+ argv = gdb.string_to_argv(argument)
+
+ if (len(argv) == 0):
+ gdb.write("Missing arguments for command: <virt_addr> [<pgd_addr>]\n")
+ return
+
+ addr = gdb.parse_and_eval(argv[0])
+ if (len(argv) == 2):
+ pgd = argv[1]
+ else:
+ pgd = "$lx_current().active_mm.pgd"
+
+ pgd = gdb.parse_and_eval(pgd)
+
+ addr = unsigned_long(addr.cast(unsigned_long_ptr))
+ gdb.write("Trying to find phys_address for 0x{:016x} \n".format(addr))
+
+ virt_split = VirtAddr(addr)
+
+ try:
+ pmd_addr = lookup_entry(pgd, virt_split.bf.pgd_idx)
+ pte_addr = lookup_entry(phys_to_virt(pmd_addr), virt_split.bf.pmd_idx)
+ pte_value = lookup_entry(phys_to_virt(pte_addr), virt_split.bf.pte_idx)
+ except:
+ raise gdb.GdbError("No page table entry for address 0x{:016x}".format(addr))
+
+ pte_val = Pte(pte_value)
+ gdb.write(str(pte_val)+ "\n")
+
+ low_addr_bits = mask_addr(addr, ps_shift[pte_val.bf.PageSZ])
+ gdb.write("Physical address: 0x{:016x}\n".format(pte_val.phys_addr() + low_addr_bits))
+
+LxVirtToPhys()
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 20/25] kvx: gdb: add kvx related gdb helpers
2023-01-03 16:43 ` [RFC PATCH 20/25] kvx: gdb: add kvx related gdb helpers Yann Sionneau
@ 2023-01-04 7:41 ` Jan Kiszka
2023-01-05 15:19 ` Dmitrii Bundin
0 siblings, 1 reply; 113+ messages in thread
From: Jan Kiszka @ 2023-01-04 7:41 UTC (permalink / raw)
To: Yann Sionneau
Cc: Kieran Bingham, linux-kernel, Clement Leger, Guillaume Thouvenin,
Dmitrii Bundin
On 03.01.23 17:43, Yann Sionneau wrote:
> Add kvx related gdb helpers:
> * lx-kvx-tlb-access
> * lx-kvx-tlb-decode
> * lx-kvx-page-table-walk
> * lx-kvx-virt-to-phys
Dmitrii just proposed translate-vm [1], just x86-only so far. Would be
good to use that scheme, maybe even watch out for chances to factor out
arch-independent logic if there is any.
In general, it would be good to define helpers that can be implemented
by other archs as well and stay away from "lx-<my-special-arch>-*"
unless there is really something valuable to add that only makes sense
for that arch.
Jan
[1]
https://lore.kernel.org/lkml/20230102171014.31408-1-dmitrii.bundin.a@gmail.com/
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 20/25] kvx: gdb: add kvx related gdb helpers
2023-01-04 7:41 ` Jan Kiszka
@ 2023-01-05 15:19 ` Dmitrii Bundin
0 siblings, 0 replies; 113+ messages in thread
From: Dmitrii Bundin @ 2023-01-05 15:19 UTC (permalink / raw)
To: Jan Kiszka
Cc: Yann Sionneau, Kieran Bingham, linux-kernel, Clement Leger,
Guillaume Thouvenin
On 03.01.23 17:43, Yann Sionneau wrote:
> * lx-kvx-virt-to-phys
I think it might be a good idea to implement lx-kvx-virt-to-phys as
the translate-vm command for the kvx arch. The details of the page
walk used for translation of a particular address and page's
attributes values might be useful for debugging purposes.
On 03.01.23 17:43, Yann Sionneau wrote:
> * lx-kvx-page-table-walk
It seems to be a good candidate to be implemented as an
arch-independent command.
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 21/25] kvx: Add support for ftrace
2023-01-03 16:43 ` Yann Sionneau
` (21 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
2023-01-05 12:55 ` Clément Léger
2023-01-05 14:50 ` Mark Rutland
-1 siblings, 2 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Steven Rostedt, Masami Hiramatsu, Mark Rutland,
linux-kernel, Clement Leger, Guillaume Thouvenin, Julian Vetter,
Marius Gligor
Add support for ftrace to kvx arch.
CC: Steven Rostedt <rostedt@goodmis.org>
CC: Masami Hiramatsu <mhiramat@kernel.org>
CC: Mark Rutland <mark.rutland@arm.com>
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/ftrace.h | 41 ++++
arch/kvx/kernel/ftrace.c | 339 ++++++++++++++++++++++++++++++
arch/kvx/kernel/mcount.S | 340 +++++++++++++++++++++++++++++++
arch/kvx/kernel/return_address.c | 55 +++++
4 files changed, 775 insertions(+)
create mode 100644 arch/kvx/include/asm/ftrace.h
create mode 100644 arch/kvx/kernel/ftrace.c
create mode 100644 arch/kvx/kernel/mcount.S
create mode 100644 arch/kvx/kernel/return_address.c
diff --git a/arch/kvx/include/asm/ftrace.h b/arch/kvx/include/asm/ftrace.h
new file mode 100644
index 000000000000..f7bcb325d255
--- /dev/null
+++ b/arch/kvx/include/asm/ftrace.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_FTRACE_H
+#define _ASM_KVX_FTRACE_H
+
+#include <asm/insns_defs.h>
+
+#define INSN_MAKE_IMM64_SYLLABLE_SIZE INSN_SIZE(MAKE_IMM64)
+#define INSN_ICALL_SYLLABLE_SIZE INSN_SIZE(ICALL)
+#define INSN_IGOTO_SYLLABLE_SIZE INSN_SIZE(IGOTO)
+#define INSN_CALL_SYLLABLE_SIZE INSN_SIZE(CALL)
+#define INSN_NOP_SYLLABLE_SIZE INSN_SIZE(NOP)
+
+#define INSN_ICALL_REG_MASK 0x3f
+
+#define MCOUNT_ADDR ((unsigned long)(__mcount))
+#define MCOUNT_INSN_SIZE INSN_CALL_SYLLABLE_SIZE /* sizeof mcount call */
+
+extern void ftrace_graph_call(void);
+
+#ifdef CONFIG_DYNAMIC_FTRACE
+extern unsigned long ftrace_call_adjust(unsigned long addr);
+struct dyn_arch_ftrace {
+ unsigned int insn;
+};
+#endif
+
+extern void *return_address(unsigned int level);
+#define ftrace_return_address(n) return_address(n)
+
+#ifdef CONFIG_FUNCTION_TRACER
+extern void __mcount(void);
+#define mcount __mcount
+#endif /* CONFIG_FUNCTION_TRACER */
+
+#endif /* _ASM_KVX_FTRACE_H */
diff --git a/arch/kvx/kernel/ftrace.c b/arch/kvx/kernel/ftrace.c
new file mode 100644
index 000000000000..4c9e3ef62714
--- /dev/null
+++ b/arch/kvx/kernel/ftrace.c
@@ -0,0 +1,339 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Marius Gligor
+ * Clement Leger
+ */
+
+#include <linux/ftrace.h>
+#include <linux/atomic.h>
+#include <linux/stop_machine.h>
+#include <asm/insns.h>
+#include <asm/insns_defs.h>
+#include <asm/cacheflush.h>
+
+/* The longest insns we check are for the far call: make + icall */
+#define MAX_SYLLABLES_TO_CHECK (KVX_INSN_MAKE_IMM64_SIZE + INSN_ICALL_SYLLABLE_SIZE)
+
+static int read_insns_and_check(u32 *insns, u8 insns_len, u32 *addr)
+{
+ u32 insns_read[MAX_SYLLABLES_TO_CHECK];
+ int syllables = insns_len / KVX_INSN_SYLLABLE_WIDTH;
+ int i;
+
+ if (syllables > MAX_SYLLABLES_TO_CHECK) {
+ pr_err("%s: shouldn't have more than %d syllables to check\n",
+ __func__, MAX_SYLLABLES_TO_CHECK);
+ return -EINVAL;
+ }
+
+ if (kvx_insns_read(insns_read, insns_len, addr)) {
+ pr_err("%s: error when trying to read syllable\n", __func__);
+ return -EFAULT;
+ }
+
+ for (i = 0; i < syllables; i++) {
+ if (insns[i] != insns_read[i]) {
+ pr_err("%s: Instruction verification failed at PC 0x%lx\n",
+ __func__,
+ (unsigned long)addr + i * KVX_INSN_SYLLABLE_WIDTH);
+ pr_err("%s: \tExpect 0x%x\n", __func__, insns[i]);
+ pr_err("%s: \tRead 0x%x\n", __func__, insns_read[i]);
+ return -EFAULT;
+ }
+ }
+
+ return 0;
+}
+
+static int write_insns_and_check(u32 *insns, u8 insns_len, u32 *insn_addr)
+{
+ int ret;
+
+ ret = kvx_insns_write_nostop(insns, insns_len, insn_addr);
+ if (ret)
+ return ret;
+
+ /* Check that what have been written is correct. */
+ return read_insns_and_check(insns, insns_len, insn_addr);
+}
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
+ unsigned long frame_pointer)
+{
+ unsigned long return_hooker = (unsigned long)&return_to_handler;
+ unsigned long old;
+
+ if (unlikely(atomic_read(¤t->tracing_graph_pause)))
+ return;
+
+ old = *parent;
+ *parent = return_hooker;
+
+ if (function_graph_enter(old, self_addr, frame_pointer, NULL))
+ *parent = old;
+}
+
+#ifdef CONFIG_DYNAMIC_FTRACE
+int ftrace_enable_ftrace_graph_caller(void)
+{
+ unsigned int insn[KVX_INSN_MAKE_IMM64_SIZE + KVX_INSN_IGOTO_SIZE];
+ void *ip = (void *)ftrace_call;
+
+ KVX_INSN_MAKE_IMM64(insn, KVX_INSN_PARALLEL_EOB, KVX_REG_R32,
+ (unsigned long)&ftrace_graph_call);
+ KVX_INSN_IGOTO(&insn[KVX_INSN_MAKE_IMM64_SIZE],
+ KVX_INSN_PARALLEL_EOB,
+ KVX_REG_R32);
+
+ return write_insns_and_check(insn,
+ INSN_MAKE_IMM64_SYLLABLE_SIZE
+ + INSN_IGOTO_SYLLABLE_SIZE,
+ ip);
+}
+
+int ftrace_disable_ftrace_graph_caller(void)
+{
+ unsigned int nop;
+ void *ip = (void *)ftrace_call;
+
+ KVX_INSN_NOP(&nop, KVX_INSN_PARALLEL_EOB);
+ return write_insns_and_check(&nop,
+ INSN_NOP_SYLLABLE_SIZE,
+ ip + INSN_MAKE_IMM64_SYLLABLE_SIZE);
+}
+#endif /* CONFIG_DYNAMIC_FTRACE */
+
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+
+#ifdef CONFIG_DYNAMIC_FTRACE
+struct kvx_ftrace_modify_param {
+ atomic_t cpu_ack;
+ int cpu_master;
+ int cmd;
+};
+
+static int __ftrace_modify_code_kvx(void *data)
+{
+ struct kvx_ftrace_modify_param *mp = data;
+ int no_cpus = num_online_cpus();
+ int cpu = smp_processor_id();
+
+ if (cpu == mp->cpu_master) {
+ ftrace_modify_all_code(mp->cmd);
+
+ /* Inform the other cpus that they can invalidate I-cache */
+ atomic_inc(&mp->cpu_ack);
+
+ /* param (which contains the cpu_ack counter) is allocated on the
+ * master stack: the master must wait for all other CPUs to leave
+ * this function before returning and releasing the stack
+ * allocation.
+ */
+ while (atomic_read(&mp->cpu_ack) < no_cpus)
+ cpu_relax();
+ } else {
+ /* Wait for the master cpu to finish the code modification */
+ while (atomic_read(&mp->cpu_ack) == 0)
+ cpu_relax();
+ atomic_inc(&mp->cpu_ack);
+
+ l1_inval_icache_all();
+ }
+
+ return 0;
+}
+
+void arch_ftrace_update_code(int command)
+{
+ const struct cpumask *cpumask = cpu_online_mask;
+ struct kvx_ftrace_modify_param mp = {
+ .cpu_ack = ATOMIC_INIT(0),
+ .cpu_master = smp_processor_id(),
+ .cmd = command,
+ };
+
+ stop_machine(__ftrace_modify_code_kvx, &mp, cpu_online_mask);
+}
+
+unsigned long ftrace_call_adjust(unsigned long addr)
+{
+ /*
+ * Module are using far call and kernel functions are using
+ * pcrel. If it is a call we don't need to adjust the address but
+ * if it is an icall the address is on the make. The generated code
+ * looks like:
+ *
+ * 1c: e0 00 c4 8f get $r32 = $ra
+ * 20: 00 00 84 e0 00 00 00 80 00 00 00 00 make $r33 = 0 (0x0);;
+ *
+ * 20: R_KVX_S64_LO10 __mcount
+ * 24: R_KVX_S64_UP27 __mcount
+ * 28: R_KVX_S64_EX27 __mcount
+ * 2c: 21 00 dc 0f icall $r33;;
+ *
+ * So we just need to add INSN_MAKE_IMM64_SYLLABLE_SIZE (0xc) to the
+ * address.
+ */
+ unsigned int insn;
+
+ /*
+ * The CALL is 1 syllable while the MAKE IMM64 is 3. But we just
+ * need to check that the first syllable of the MAKE IMM64 is the
+ * LO10. So finally we just need to read one syllable to adjust the
+ * call.
+ */
+ if (kvx_insns_read(&insn, KVX_INSN_SYLLABLE_WIDTH, (void *)addr)) {
+ pr_err("%s: error when trying to read syllable\n", __func__);
+ return 0;
+ }
+
+ if (IS_INSN(insn, CALL))
+ return addr;
+
+ if (IS_INSN(insn, MAKE_IMM64))
+ return addr + INSN_MAKE_IMM64_SYLLABLE_SIZE;
+
+ pr_err("%s: syllable is neither a CALL nor a MAKE\n", __func__);
+ return 0;
+}
+
+/*
+ * Do runtime patching of the active tracer.
+ * This will be modifying the assembly code at the location of the
+ * ftrace_call symbol inside of the ftrace_caller() function.
+ */
+int ftrace_update_ftrace_func(ftrace_func_t func)
+{
+ void *ip;
+ unsigned int insn[KVX_INSN_MAKE_IMM64_SIZE + KVX_INSN_ICALL_SIZE];
+
+ ip = (void *)ftrace_call;
+ KVX_INSN_MAKE_IMM64(insn, KVX_INSN_PARALLEL_EOB, KVX_REG_R32,
+ (unsigned long)func);
+ KVX_INSN_ICALL(&insn[KVX_INSN_MAKE_IMM64_SIZE],
+ KVX_INSN_PARALLEL_EOB,
+ KVX_REG_R32);
+ return write_insns_and_check(insn,
+ INSN_MAKE_IMM64_SYLLABLE_SIZE
+ + INSN_ICALL_SYLLABLE_SIZE,
+ ip);
+}
+
+/*
+ * Turn the mcount call site into a call to an arbitrary location (but
+ * typically that is ftrace_caller()) at runtime.
+ */
+int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+ void *ip = (void *)rec->ip;
+ unsigned int insn;
+ int ret;
+
+ /* Ensure that a NOP will be replaced */
+ if (kvx_insns_read(&insn, KVX_INSN_SYLLABLE_WIDTH, ip)) {
+ pr_err("%s: failed to read insn\n", __func__);
+ return -EFAULT;
+ }
+
+ if (!IS_INSN(insn, NOP)) {
+ pr_err("%s: insn 0x%x is not a NOP\n", __func__, insn);
+ return -EINVAL;
+ }
+
+ /*
+ * Now we can replace the instruction depending of what has been
+ * nopified (call or icall)
+ */
+ insn = rec->arch.insn;
+
+ if (IS_INSN(insn, CALL)) {
+ s32 pcrel = addr - (unsigned long) ip;
+ u32 insn_call;
+
+ BUG_ON(KVX_INSN_GOTO_PCREL27_CHECK(pcrel));
+ KVX_INSN_CALL(&insn_call, KVX_INSN_PARALLEL_EOB, pcrel);
+
+ return write_insns_and_check(&insn_call,
+ INSN_CALL_SYLLABLE_SIZE, ip);
+ }
+
+ if (IS_INSN(insn, ICALL)) {
+ u32 insn_make[KVX_INSN_MAKE_IMM64_SIZE];
+ u32 r = insn & INSN_ICALL_REG_MASK;
+
+ KVX_INSN_MAKE_IMM64(insn_make, KVX_INSN_PARALLEL_EOB, r, addr);
+ ret = write_insns_and_check(insn_make,
+ INSN_MAKE_IMM64_SYLLABLE_SIZE,
+ ip - INSN_MAKE_IMM64_SYLLABLE_SIZE);
+ if (ret)
+ return ret;
+
+ return write_insns_and_check(&insn,
+ INSN_ICALL_SYLLABLE_SIZE, ip);
+ }
+
+ /* It is neither a call nor an icall */
+ pr_err("%s: insn 0x%x is neither a CALL nor ICALL\n", __func__, insn);
+ return -EINVAL;
+}
+
+/*
+ * Turn the mcount call site into a nop at runtime
+ */
+int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
+ unsigned long addr)
+{
+ unsigned long ip = rec->ip;
+ unsigned int insn;
+ unsigned int nop;
+
+ /*
+ * Ensure that the instruction that will be replaced is a call or an
+ * icall to addr.
+ */
+ if (kvx_insns_read(&insn, KVX_INSN_SYLLABLE_WIDTH, (void *)ip)) {
+ pr_err("%s: error when trying to read syllable\n", __func__);
+ return -EFAULT;
+ }
+
+ if (IS_INSN(insn, CALL)) {
+ int pcrel = ((int)(insn & 0x7ffffff) << 5) >> 3;
+
+ if ((ip + pcrel != addr)) {
+ pr_err("%s: failed to check call addr 0x%lx != 0x%lx\n",
+ __func__, ip + pcrel, addr);
+ return -EINVAL;
+ }
+ } else if (IS_INSN(insn, ICALL)) {
+ unsigned int insn_make[KVX_INSN_MAKE_IMM64_SIZE];
+ unsigned int reg = insn & INSN_ICALL_REG_MASK;
+ int ret;
+
+ KVX_INSN_MAKE_IMM64(insn_make,
+ KVX_INSN_PARALLEL_EOB, reg,
+ addr);
+
+ ret = read_insns_and_check(insn_make,
+ INSN_MAKE_IMM64_SYLLABLE_SIZE,
+ (void *)ip -
+ INSN_MAKE_IMM64_SYLLABLE_SIZE);
+ if (ret)
+ return ret;
+ } else {
+ pr_err("%s: insn 0x%x is neither a CALL nor an ICALL\n",
+ __func__, insn);
+ return -EINVAL;
+ }
+
+ rec->arch.insn = insn;
+ KVX_INSN_NOP(&nop, KVX_INSN_PARALLEL_EOB);
+ return write_insns_and_check(&nop, INSN_NOP_SYLLABLE_SIZE, (void *)ip);
+}
+
+#endif /* CONFIG_DYNAMIC_FTRACE */
+
+/* __mcount is defined in mcount.S */
+EXPORT_SYMBOL(__mcount);
diff --git a/arch/kvx/kernel/mcount.S b/arch/kvx/kernel/mcount.S
new file mode 100644
index 000000000000..317797aeed42
--- /dev/null
+++ b/arch/kvx/kernel/mcount.S
@@ -0,0 +1,340 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Guillaume Thouvenin
+ * Marius Gligor
+ * Clement Leger
+ */
+
+#include <linux/linkage.h>
+#include <asm/insns_defs.h>
+#include <asm/sfr_defs.h>
+
+.altmacro
+
+/*
+ * !!! The call to __mcount is special and breaks the ABI !!!
+ * It is because with dynamic ftrace the call can be replaced by any
+ * tracer and GCC cannot assume which caller registers must be saved.
+ * It's because the call is being inserted in the prologue of functions
+ * that is out of GCC knowledge.
+ * We could make this known to GCC, but then we would hit the problem of not
+ * being able to easily decide which insns are emitted and how they are
+ * bundled. So we choose to save nothing when calling __mcount and we will
+ * save all caller registers once __mcount called.
+ *
+ * Another specificity is that parameter is passed in $r32 instead of $r0.
+ *
+ * Our stack will be:
+ * 0 | $FP $RA $r15 -
+ * 32 | $r0 $r1 $r2 $r3
+ * 64 | $r4 $r5 $r6 $r7
+ * 96 | $r8 $r9 $r10 $r11
+ * 128| $r32 $r33 $r34 $r35
+ * 160| $r36 $r37 $r38 $r39
+ * 192| $r40 $r41 $r42 $r43
+ * 224| $r44 $r45 $r46 $r47
+ * 256| $r48 $r49 $r50 $r51
+ * 288| $r52 $r53 $r54 $r55
+ * 320| $r56 $r57 $r58 $r59
+ * 352| $r60 $r61 $r62 $r63
+ * 384| $r16 $r17
+ */
+
+#define STACK_OFFSET_FP 0
+#define STACK_OFFSET_RA (STACK_OFFSET_FP + 8)
+#define STACK_OFFSET_R15 (STACK_OFFSET_RA + 8)
+#define STACK_OFFSET_Q0 (STACK_OFFSET_FP + 32)
+#define STACK_OFFSET_Q4 (STACK_OFFSET_Q0 + 32)
+#define STACK_OFFSET_Q8 (STACK_OFFSET_Q4 + 32)
+#define STACK_OFFSET_Q32 (STACK_OFFSET_Q8 + 32)
+#define STACK_OFFSET_Q36 (STACK_OFFSET_Q32 + 32)
+#define STACK_OFFSET_Q40 (STACK_OFFSET_Q36 + 32)
+#define STACK_OFFSET_Q44 (STACK_OFFSET_Q40 + 32)
+#define STACK_OFFSET_Q48 (STACK_OFFSET_Q44 + 32)
+#define STACK_OFFSET_Q52 (STACK_OFFSET_Q48 + 32)
+#define STACK_OFFSET_Q56 (STACK_OFFSET_Q52 + 32)
+#define STACK_OFFSET_Q60 (STACK_OFFSET_Q56 + 32)
+#define STACK_OFFSET_P16 (STACK_OFFSET_Q60 + 32)
+#define STACK_SIZE (STACK_OFFSET_P16 + 16)
+
+#define RA_DWARF_REGNO (64 + KVX_SFR_RA)
+#define R14_DWARF_REGNO 14
+
+.macro save_all_context_but_P16
+ /*
+ * r16 and r17 are already saved. r16 is already used to store the
+ * address of the function tracer if any.
+ * so !!! USE r17 as scratch register !!!
+ */
+ get $r17 = $ra
+ sd STACK_OFFSET_FP[$r12] = $r14
+ ;;
+ .cfi_offset R14_DWARF_REGNO, -STACK_SIZE + STACK_OFFSET_FP
+ sd STACK_OFFSET_RA[$r12] = $r17
+ ;;
+ .cfi_offset RA_DWARF_REGNO, -STACK_SIZE + STACK_OFFSET_RA
+ sd STACK_OFFSET_R15[$r12] = $r15
+ ;;
+ so STACK_OFFSET_Q0[$r12] = $r0r1r2r3
+ ;;
+ so STACK_OFFSET_Q4[$r12] = $r4r5r6r7
+ ;;
+ so STACK_OFFSET_Q8[$r12] = $r8r9r10r11
+ ;;
+ so STACK_OFFSET_Q32[$r12] = $r32r33r34r35
+ ;;
+ so STACK_OFFSET_Q36[$r12] = $r36r37r38r39
+ ;;
+ so STACK_OFFSET_Q40[$r12] = $r40r41r42r43
+ ;;
+ so STACK_OFFSET_Q44[$r12] = $r44r45r46r47
+ ;;
+ so STACK_OFFSET_Q48[$r12] = $r48r49r50r51
+ ;;
+ so STACK_OFFSET_Q52[$r12] = $r52r53r54r55
+ ;;
+ so STACK_OFFSET_Q56[$r12] = $r56r57r58r59
+ ;;
+ so STACK_OFFSET_Q60[$r12] = $r60r61r62r63
+ ;;
+.endm
+
+.macro restore_all_context
+ ld $r15 = STACK_OFFSET_RA[$r12]
+ ;;
+ ld $r14 = STACK_OFFSET_FP[$r12]
+ set $ra = $r15
+ ;;
+ .cfi_restore RA_DWARF_REGNO
+ .cfi_restore R14_DWARF_REGNO
+ ld $r15 = STACK_OFFSET_R15[$r12]
+ ;;
+ lq $r16r17 = STACK_OFFSET_P16[$r12]
+ ;;
+ lo $r0r1r2r3 = STACK_OFFSET_Q0[$r12]
+ ;;
+ lo $r4r5r6r7 = STACK_OFFSET_Q4[$r12]
+ ;;
+ lo $r8r9r10r11 = STACK_OFFSET_Q8[$r12]
+ ;;
+ lo $r32r33r34r35 = STACK_OFFSET_Q32[$r12]
+ ;;
+ lo $r36r37r38r39 = STACK_OFFSET_Q36[$r12]
+ ;;
+ lo $r40r41r42r43 = STACK_OFFSET_Q40[$r12]
+ ;;
+ lo $r44r45r46r47 = STACK_OFFSET_Q44[$r12]
+ ;;
+ lo $r48r49r50r51 = STACK_OFFSET_Q48[$r12]
+ ;;
+ lo $r52r53r54r55 = STACK_OFFSET_Q52[$r12]
+ ;;
+ lo $r56r57r58r59 = STACK_OFFSET_Q56[$r12]
+ ;;
+ lo $r60r61r62r63 = STACK_OFFSET_Q60[$r12]
+ ;;
+ addd $r12 = $r12, STACK_SIZE
+ ;;
+ .cfi_def_cfa_offset 0
+.endm
+
+ENTRY(__mcount)
+.cfi_startproc
+#ifdef CONFIG_DYNAMIC_FTRACE
+ /*
+ * With dynamic ftrace mcount() is only used during boot and then all
+ * references to it will be patched out never to return. ftrace_caller
+ * is used.
+ */
+ ret
+ ;;
+#endif
+ addd $r12 = $r12, -STACK_SIZE
+ ;;
+ .cfi_def_cfa_offset STACK_SIZE
+ sq STACK_OFFSET_P16[$r12] = $r16r17
+ ;;
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+ /*
+ * Check the function pointers ftrace_graph_return
+ * (compare to ftrace_stub) and ftrace_graph_entry (compare to
+ * ftrace_graph_entry_stub). If either of those is not set to the
+ * relevant stub function, call the arch-specific function
+ * ftrace_graph_caller.
+ */
+ make $r16 = ftrace_graph_return
+ make $r17 = ftrace_graph_entry
+ ;;
+ ld $r16 = 0[$r16]
+ ;;
+ compd.eq $r16 = $r16, ftrace_stub
+ ld $r17 = 0[$r17]
+ ;;
+ /*
+ * If ftrace_graph_return is not equal to ftrace_stub ($r16 == 0) then
+ * jump to ftrace_graph_caller. Otherwise compare ftrace_graph_entry.
+ */
+ cb.deqz $r16? ftrace_graph_caller
+ compd.eq $r17 = $r17, ftrace_graph_entry_stub
+ ;;
+ /*
+ * If ftrace_graph_entry is equal to ftrace_graph_entry_stub ($r17 == 1)
+ * then jump to ftrace_graph_end. If not we continue.
+ */
+ cb.dnez $r17? ftrace_graph_end
+ ;;
+ftrace_graph_caller:
+ save_all_context_but_P16
+ ;;
+GLOBAL(ftrace_graph_call)
+ ld $r16 = 0[$r12]
+ ;;
+ addd $r0 = $r16, 0x8 /* a pointer to the frompc */
+ get $r1 = $ra /* selfpc */
+ copyd $r2 = $r16 /* FP of the function that called __mcount */
+ ;;
+ /* get the address of the bundle */
+ addd $r1 = $r1, -KVX_INSN_SYLLABLE_WIDTH
+ ;;
+ call prepare_ftrace_return
+ ;;
+ restore_all_context
+ ;;
+ ret
+ ;;
+ftrace_graph_end:
+ .cfi_def_cfa_offset STACK_SIZE
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+ /*
+ * Check if function pointer ftrace_trace_function is set to
+ * ftrace_stub.
+ * If it isn't, then call that function in the same way the mcount
+ * function normally calls __mcount_internal
+ * - the first argument is the "frompc" ($r0)
+ * - the second argument is the "selfpc" ($ra - mcount size adjustement)
+ * As we don't use any callee saved registers we don't need to save
+ * them.
+ */
+ make $r16 = ftrace_trace_function
+ ;;
+ ld $r16 = 0[$r16]
+ ;;
+ /*
+ * NOTE: keep value in $r16 (the function we may call if set). See
+ * comment in save_all_context_but_P16.
+ */
+ compd.eq $r17 = $r16, ftrace_stub
+ ;;
+ /*
+ * If ftrace_trace_function is equal to ftrace_stub ($r17 == 1), there
+ * is nothing to do, so return immediately otherwise call the function
+ * in $r16 with correct parameters.
+ */
+ cb.deqz $r17? 1f
+ ;;
+ /* Only need to restore P16 */
+ lq $r16r17 = STACK_OFFSET_P16[$r12]
+ ;;
+ addd $r12 = $r12, STACK_SIZE
+ ;;
+ .cfi_def_cfa_offset 0
+ ret
+ ;;
+
+1:
+ save_all_context_but_P16
+ ;;
+ copyd $r0 = $r32
+ get $r1 = $ra
+ ;;
+ addd $r1 = $r1, -KVX_INSN_SYLLABLE_WIDTH
+ ;;
+ icall $r16
+ ;;
+ restore_all_context
+ ;;
+ ret
+ ;;
+.cfi_endproc
+ENDPROC(__mcount)
+
+/* Required by ftrace_graph_ret_addr */
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+ENTRY(return_to_handler)
+ addd $r12 = $r12, -128
+ ;;
+ so 0[$r12] = $r0r1r2r3
+ ;;
+ so 32[$r12] = $r4r5r6r7
+ ;;
+ so 64[$r12] = $r8r9r10r11
+ ;;
+ call ftrace_return_to_handler
+ ;;
+ set $ra = $r0
+ lo $r0r1r2r3 = 0[$r12]
+ ;;
+ lo $r4r5r6r7 = 32[$r12]
+ ;;
+ lo $r8r9r10r11 = 64[$r12]
+ addd $r12 = $r12, 128
+ ;;
+ ret
+ ;;
+ENDPROC(return_to_handler)
+#endif
+
+#ifdef CONFIG_DYNAMIC_FTRACE
+ENTRY(ftrace_caller)
+.cfi_startproc
+
+ addd $r12 = $r12, -STACK_SIZE
+ ;;
+ .cfi_def_cfa_offset STACK_SIZE
+ sq STACK_OFFSET_P16[$r12] = $r16r17
+ ;;
+ save_all_context_but_P16
+ ;;
+ copyd $r0 = $r32
+ get $r1 = $ra
+ ;;
+ addd $r1 = $r1, -KVX_INSN_SYLLABLE_WIDTH
+ ;;
+ goto ftrace_call
+ ;;
+.align 64
+GLOBAL(ftrace_call)
+ /*
+ * To work with module we need to use far call. So prepare the
+ * space for them:
+ * - make: 3 syllables
+ * - icall/igoto: 1 syllable
+ *
+ * We are using igoto in ftrace_enable_ftrace_graph_caller(). This is
+ * because this path is a tweak to enable function graph tracer with
+ * dynamic ftrace. We need to jump to ftrace_graph_call without updating
+ * RA and FP. Once in ftrace_graph_call we will return to the correct
+ * address from the classical path of the function graph tracer.
+ */
+ nop
+ ;;
+ nop
+ ;;
+ nop
+ ;;
+ nop
+ ;;
+ restore_all_context
+ ;;
+ ret
+ ;;
+.cfi_endproc
+ENDPROC(ftrace_caller)
+#endif // CONFIG_DYNAMIC_FTRACE
+
+ENTRY(ftrace_stub)
+ ret
+ ;;
+ENDPROC(ftrace_stub)
+
diff --git a/arch/kvx/kernel/return_address.c b/arch/kvx/kernel/return_address.c
new file mode 100644
index 000000000000..3ddf7b79662c
--- /dev/null
+++ b/arch/kvx/kernel/return_address.c
@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * derived from arch/arm64/kernel/return_address.c
+ *
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/export.h>
+#include <linux/ftrace.h>
+#include <linux/kprobes.h>
+
+#include <asm/stacktrace.h>
+
+struct return_address_data {
+ unsigned int level;
+ void *addr;
+};
+
+static bool save_return_addr(unsigned long pc, void *d)
+{
+ struct return_address_data *data = d;
+
+ /* We hit the desired level, return the address */
+ if (data->level == 0) {
+ data->addr = (void *) pc;
+ return true;
+ }
+
+ data->level--;
+ return false;
+}
+NOKPROBE_SYMBOL(save_return_addr);
+
+void *return_address(unsigned int level)
+{
+ struct return_address_data data;
+ struct stackframe frame;
+
+ /* Skip this function + caller */
+ data.level = level + 2;
+ data.addr = NULL;
+
+ start_stackframe(&frame,
+ (unsigned long) __builtin_frame_address(0),
+ (unsigned long) return_address);
+ walk_stackframe(current, &frame, save_return_addr, &data);
+
+ if (!data.level)
+ return data.addr;
+ else
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(return_address);
+NOKPROBE_SYMBOL(return_address);
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 21/25] kvx: Add support for ftrace
2023-01-03 16:43 ` [RFC PATCH 21/25] kvx: Add support for ftrace Yann Sionneau
@ 2023-01-05 12:55 ` Clément Léger
2023-01-05 14:20 ` Steven Rostedt
2023-01-05 14:50 ` Mark Rutland
1 sibling, 1 reply; 113+ messages in thread
From: Clément Léger @ 2023-01-05 12:55 UTC (permalink / raw)
To: Yann Sionneau
Cc: Steven Rostedt, Masami Hiramatsu, Mark Rutland, linux-kernel,
Guillaume Thouvenin, Julian Vetter, Marius Gligor
Le Tue, 3 Jan 2023 17:43:55 +0100,
Yann Sionneau <ysionneau@kalray.eu> a écrit :
> Add support for ftrace to kvx arch.
>
> CC: Steven Rostedt <rostedt@goodmis.org>
> CC: Masami Hiramatsu <mhiramat@kernel.org>
> CC: Mark Rutland <mark.rutland@arm.com>
> CC: linux-kernel@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Co-developed-by: Marius Gligor <mgligor@kalray.eu>
> Signed-off-by: Marius Gligor <mgligor@kalray.eu>
> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> arch/kvx/include/asm/ftrace.h | 41 ++++
> arch/kvx/kernel/ftrace.c | 339 ++++++++++++++++++++++++++++++
> arch/kvx/kernel/mcount.S | 340 +++++++++++++++++++++++++++++++
> arch/kvx/kernel/return_address.c | 55 +++++
> 4 files changed, 775 insertions(+)
> create mode 100644 arch/kvx/include/asm/ftrace.h
> create mode 100644 arch/kvx/kernel/ftrace.c
> create mode 100644 arch/kvx/kernel/mcount.S
> create mode 100644 arch/kvx/kernel/return_address.c
>
> diff --git a/arch/kvx/include/asm/ftrace.h b/arch/kvx/include/asm/ftrace.h
> new file mode 100644
> index 000000000000..f7bcb325d255
> --- /dev/null
> +++ b/arch/kvx/include/asm/ftrace.h
> @@ -0,0 +1,41 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Guillaume Thouvenin
> + * Clement Leger
> + */
> +
> +#ifndef _ASM_KVX_FTRACE_H
> +#define _ASM_KVX_FTRACE_H
> +
> +#include <asm/insns_defs.h>
> +
> +#define INSN_MAKE_IMM64_SYLLABLE_SIZE INSN_SIZE(MAKE_IMM64)
> +#define INSN_ICALL_SYLLABLE_SIZE INSN_SIZE(ICALL)
> +#define INSN_IGOTO_SYLLABLE_SIZE INSN_SIZE(IGOTO)
> +#define INSN_CALL_SYLLABLE_SIZE INSN_SIZE(CALL)
> +#define INSN_NOP_SYLLABLE_SIZE INSN_SIZE(NOP)
> +
> +#define INSN_ICALL_REG_MASK 0x3f
> +
> +#define MCOUNT_ADDR ((unsigned long)(__mcount))
> +#define MCOUNT_INSN_SIZE INSN_CALL_SYLLABLE_SIZE /* sizeof mcount call */
> +
> +extern void ftrace_graph_call(void);
> +
> +#ifdef CONFIG_DYNAMIC_FTRACE
> +extern unsigned long ftrace_call_adjust(unsigned long addr);
> +struct dyn_arch_ftrace {
> + unsigned int insn;
> +};
> +#endif
> +
> +extern void *return_address(unsigned int level);
> +#define ftrace_return_address(n) return_address(n)
> +
> +#ifdef CONFIG_FUNCTION_TRACER
> +extern void __mcount(void);
> +#define mcount __mcount
> +#endif /* CONFIG_FUNCTION_TRACER */
> +
> +#endif /* _ASM_KVX_FTRACE_H */
> diff --git a/arch/kvx/kernel/ftrace.c b/arch/kvx/kernel/ftrace.c
> new file mode 100644
> index 000000000000..4c9e3ef62714
> --- /dev/null
> +++ b/arch/kvx/kernel/ftrace.c
> @@ -0,0 +1,339 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Guillaume Thouvenin
> + * Marius Gligor
> + * Clement Leger
> + */
> +
> +#include <linux/ftrace.h>
> +#include <linux/atomic.h>
> +#include <linux/stop_machine.h>
> +#include <asm/insns.h>
> +#include <asm/insns_defs.h>
> +#include <asm/cacheflush.h>
> +
> +/* The longest insns we check are for the far call: make + icall */
> +#define MAX_SYLLABLES_TO_CHECK (KVX_INSN_MAKE_IMM64_SIZE + INSN_ICALL_SYLLABLE_SIZE)
> +
> +static int read_insns_and_check(u32 *insns, u8 insns_len, u32 *addr)
> +{
> + u32 insns_read[MAX_SYLLABLES_TO_CHECK];
> + int syllables = insns_len / KVX_INSN_SYLLABLE_WIDTH;
> + int i;
> +
> + if (syllables > MAX_SYLLABLES_TO_CHECK) {
> + pr_err("%s: shouldn't have more than %d syllables to check\n",
> + __func__, MAX_SYLLABLES_TO_CHECK);
> + return -EINVAL;
> + }
> +
> + if (kvx_insns_read(insns_read, insns_len, addr)) {
> + pr_err("%s: error when trying to read syllable\n", __func__);
> + return -EFAULT;
> + }
> +
> + for (i = 0; i < syllables; i++) {
> + if (insns[i] != insns_read[i]) {
> + pr_err("%s: Instruction verification failed at PC 0x%lx\n",
> + __func__,
> + (unsigned long)addr + i * KVX_INSN_SYLLABLE_WIDTH);
> + pr_err("%s: \tExpect 0x%x\n", __func__, insns[i]);
> + pr_err("%s: \tRead 0x%x\n", __func__, insns_read[i]);
> + return -EFAULT;
> + }
> + }
> +
> + return 0;
> +}
Hi Yann,
Is this still needed ? I'm guessing the instructions should always be
correctly written no ? If not, something probably went horribly wrong ;)
> +
> +static int write_insns_and_check(u32 *insns, u8 insns_len, u32 *insn_addr)
> +{
> + int ret;
> +
> + ret = kvx_insns_write_nostop(insns, insns_len, insn_addr);
> + if (ret)
> + return ret;
> +
> + /* Check that what have been written is correct. */
> + return read_insns_and_check(insns, insns_len, insn_addr);
> +}
> +
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> +void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
> + unsigned long frame_pointer)
> +{
> + unsigned long return_hooker = (unsigned long)&return_to_handler;
> + unsigned long old;
> +
> + if (unlikely(atomic_read(¤t->tracing_graph_pause)))
> + return;
> +
> + old = *parent;
> + *parent = return_hooker;
> +
> + if (function_graph_enter(old, self_addr, frame_pointer, NULL))
> + *parent = old;
> +}
> +
> +#ifdef CONFIG_DYNAMIC_FTRACE
> +int ftrace_enable_ftrace_graph_caller(void)
> +{
> + unsigned int insn[KVX_INSN_MAKE_IMM64_SIZE + KVX_INSN_IGOTO_SIZE];
> + void *ip = (void *)ftrace_call;
> +
> + KVX_INSN_MAKE_IMM64(insn, KVX_INSN_PARALLEL_EOB, KVX_REG_R32,
> + (unsigned long)&ftrace_graph_call);
> + KVX_INSN_IGOTO(&insn[KVX_INSN_MAKE_IMM64_SIZE],
> + KVX_INSN_PARALLEL_EOB,
> + KVX_REG_R32);
> +
> + return write_insns_and_check(insn,
> + INSN_MAKE_IMM64_SYLLABLE_SIZE
> + + INSN_IGOTO_SYLLABLE_SIZE,
> + ip);
> +}
> +
> +int ftrace_disable_ftrace_graph_caller(void)
> +{
> + unsigned int nop;
> + void *ip = (void *)ftrace_call;
> +
> + KVX_INSN_NOP(&nop, KVX_INSN_PARALLEL_EOB);
> + return write_insns_and_check(&nop,
> + INSN_NOP_SYLLABLE_SIZE,
> + ip + INSN_MAKE_IMM64_SYLLABLE_SIZE);
> +}
> +#endif /* CONFIG_DYNAMIC_FTRACE */
> +
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> +
> +#ifdef CONFIG_DYNAMIC_FTRACE
> +struct kvx_ftrace_modify_param {
> + atomic_t cpu_ack;
> + int cpu_master;
> + int cmd;
> +};
> +
> +static int __ftrace_modify_code_kvx(void *data)
> +{
> + struct kvx_ftrace_modify_param *mp = data;
> + int no_cpus = num_online_cpus();
> + int cpu = smp_processor_id();
> +
> + if (cpu == mp->cpu_master) {
> + ftrace_modify_all_code(mp->cmd);
> +
> + /* Inform the other cpus that they can invalidate I-cache */
> + atomic_inc(&mp->cpu_ack);
> +
> + /* param (which contains the cpu_ack counter) is allocated on the
> + * master stack: the master must wait for all other CPUs to leave
> + * this function before returning and releasing the stack
> + * allocation.
> + */
> + while (atomic_read(&mp->cpu_ack) < no_cpus)
> + cpu_relax();
> + } else {
> + /* Wait for the master cpu to finish the code modification */
> + while (atomic_read(&mp->cpu_ack) == 0)
> + cpu_relax();
> + atomic_inc(&mp->cpu_ack);
> +
> + l1_inval_icache_all();
> + }
> +
> + return 0;
> +}
> +
> +void arch_ftrace_update_code(int command)
> +{
> + const struct cpumask *cpumask = cpu_online_mask;
> + struct kvx_ftrace_modify_param mp = {
> + .cpu_ack = ATOMIC_INIT(0),
> + .cpu_master = smp_processor_id(),
> + .cmd = command,
> + };
> +
> + stop_machine(__ftrace_modify_code_kvx, &mp, cpu_online_mask);
> +}
> +
> +unsigned long ftrace_call_adjust(unsigned long addr)
> +{
> + /*
> + * Module are using far call and kernel functions are using
> + * pcrel. If it is a call we don't need to adjust the address but
> + * if it is an icall the address is on the make. The generated code
> + * looks like:
> + *
> + * 1c: e0 00 c4 8f get $r32 = $ra
> + * 20: 00 00 84 e0 00 00 00 80 00 00 00 00 make $r33 = 0 (0x0);;
> + *
> + * 20: R_KVX_S64_LO10 __mcount
> + * 24: R_KVX_S64_UP27 __mcount
> + * 28: R_KVX_S64_EX27 __mcount
> + * 2c: 21 00 dc 0f icall $r33;;
> + *
> + * So we just need to add INSN_MAKE_IMM64_SYLLABLE_SIZE (0xc) to the
> + * address.
> + */
> + unsigned int insn;
> +
> + /*
> + * The CALL is 1 syllable while the MAKE IMM64 is 3. But we just
> + * need to check that the first syllable of the MAKE IMM64 is the
> + * LO10. So finally we just need to read one syllable to adjust the
> + * call.
> + */
> + if (kvx_insns_read(&insn, KVX_INSN_SYLLABLE_WIDTH, (void *)addr)) {
> + pr_err("%s: error when trying to read syllable\n", __func__);
> + return 0;
> + }
> +
> + if (IS_INSN(insn, CALL))
> + return addr;
> +
> + if (IS_INSN(insn, MAKE_IMM64))
> + return addr + INSN_MAKE_IMM64_SYLLABLE_SIZE;
> +
> + pr_err("%s: syllable is neither a CALL nor a MAKE\n", __func__);
> + return 0;
> +}
> +
> +/*
> + * Do runtime patching of the active tracer.
> + * This will be modifying the assembly code at the location of the
> + * ftrace_call symbol inside of the ftrace_caller() function.
> + */
> +int ftrace_update_ftrace_func(ftrace_func_t func)
> +{
> + void *ip;
> + unsigned int insn[KVX_INSN_MAKE_IMM64_SIZE + KVX_INSN_ICALL_SIZE];
> +
> + ip = (void *)ftrace_call;
> + KVX_INSN_MAKE_IMM64(insn, KVX_INSN_PARALLEL_EOB, KVX_REG_R32,
> + (unsigned long)func);
> + KVX_INSN_ICALL(&insn[KVX_INSN_MAKE_IMM64_SIZE],
> + KVX_INSN_PARALLEL_EOB,
> + KVX_REG_R32);
> + return write_insns_and_check(insn,
> + INSN_MAKE_IMM64_SYLLABLE_SIZE
> + + INSN_ICALL_SYLLABLE_SIZE,
> + ip);
> +}
> +
> +/*
> + * Turn the mcount call site into a call to an arbitrary location (but
> + * typically that is ftrace_caller()) at runtime.
> + */
> +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> +{
> + void *ip = (void *)rec->ip;
> + unsigned int insn;
> + int ret;
> +
> + /* Ensure that a NOP will be replaced */
> + if (kvx_insns_read(&insn, KVX_INSN_SYLLABLE_WIDTH, ip)) {
> + pr_err("%s: failed to read insn\n", __func__);
> + return -EFAULT;
> + }
> +
> + if (!IS_INSN(insn, NOP)) {
> + pr_err("%s: insn 0x%x is not a NOP\n", __func__, insn);
> + return -EINVAL;
> + }
> +
> + /*
> + * Now we can replace the instruction depending of what has been
> + * nopified (call or icall)
> + */
> + insn = rec->arch.insn;
> +
> + if (IS_INSN(insn, CALL)) {
> + s32 pcrel = addr - (unsigned long) ip;
> + u32 insn_call;
> +
> + BUG_ON(KVX_INSN_GOTO_PCREL27_CHECK(pcrel));
> + KVX_INSN_CALL(&insn_call, KVX_INSN_PARALLEL_EOB, pcrel);
> +
> + return write_insns_and_check(&insn_call,
> + INSN_CALL_SYLLABLE_SIZE, ip);
> + }
> +
> + if (IS_INSN(insn, ICALL)) {
> + u32 insn_make[KVX_INSN_MAKE_IMM64_SIZE];
> + u32 r = insn & INSN_ICALL_REG_MASK;
> +
> + KVX_INSN_MAKE_IMM64(insn_make, KVX_INSN_PARALLEL_EOB, r, addr);
> + ret = write_insns_and_check(insn_make,
> + INSN_MAKE_IMM64_SYLLABLE_SIZE,
> + ip - INSN_MAKE_IMM64_SYLLABLE_SIZE);
> + if (ret)
> + return ret;
> +
> + return write_insns_and_check(&insn,
> + INSN_ICALL_SYLLABLE_SIZE, ip);
> + }
> +
> + /* It is neither a call nor an icall */
> + pr_err("%s: insn 0x%x is neither a CALL nor ICALL\n", __func__, insn);
> + return -EINVAL;
> +}
> +
> +/*
> + * Turn the mcount call site into a nop at runtime
> + */
> +int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
> + unsigned long addr)
> +{
> + unsigned long ip = rec->ip;
> + unsigned int insn;
> + unsigned int nop;
> +
> + /*
> + * Ensure that the instruction that will be replaced is a call or an
> + * icall to addr.
> + */
> + if (kvx_insns_read(&insn, KVX_INSN_SYLLABLE_WIDTH, (void *)ip)) {
> + pr_err("%s: error when trying to read syllable\n", __func__);
> + return -EFAULT;
> + }
> +
> + if (IS_INSN(insn, CALL)) {
> + int pcrel = ((int)(insn & 0x7ffffff) << 5) >> 3;
Hardcoded values
> +
> + if ((ip + pcrel != addr)) {
> + pr_err("%s: failed to check call addr 0x%lx != 0x%lx\n",
> + __func__, ip + pcrel, addr);
> + return -EINVAL;
> + }
> + } else if (IS_INSN(insn, ICALL)) {
> + unsigned int insn_make[KVX_INSN_MAKE_IMM64_SIZE];
> + unsigned int reg = insn & INSN_ICALL_REG_MASK;
> + int ret;
> +
> + KVX_INSN_MAKE_IMM64(insn_make,
> + KVX_INSN_PARALLEL_EOB, reg,
> + addr);
> +
> + ret = read_insns_and_check(insn_make,
> + INSN_MAKE_IMM64_SYLLABLE_SIZE,
> + (void *)ip -
> + INSN_MAKE_IMM64_SYLLABLE_SIZE);
> + if (ret)
> + return ret;
> + } else {
> + pr_err("%s: insn 0x%x is neither a CALL nor an ICALL\n",
> + __func__, insn);
> + return -EINVAL;
> + }
> +
> + rec->arch.insn = insn;
> + KVX_INSN_NOP(&nop, KVX_INSN_PARALLEL_EOB);
> + return write_insns_and_check(&nop, INSN_NOP_SYLLABLE_SIZE, (void *)ip);
> +}
> +
> +#endif /* CONFIG_DYNAMIC_FTRACE */
> +
> +/* __mcount is defined in mcount.S */
> +EXPORT_SYMBOL(__mcount);
> diff --git a/arch/kvx/kernel/mcount.S b/arch/kvx/kernel/mcount.S
> new file mode 100644
> index 000000000000..317797aeed42
> --- /dev/null
> +++ b/arch/kvx/kernel/mcount.S
> @@ -0,0 +1,340 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Guillaume Thouvenin
> + * Marius Gligor
> + * Clement Leger
> + */
> +
> +#include <linux/linkage.h>
> +#include <asm/insns_defs.h>
> +#include <asm/sfr_defs.h>
> +
> +.altmacro
> +
> +/*
> + * !!! The call to __mcount is special and breaks the ABI !!!
> + * It is because with dynamic ftrace the call can be replaced by any
> + * tracer and GCC cannot assume which caller registers must be saved.
> + * It's because the call is being inserted in the prologue of functions
> + * that is out of GCC knowledge.
> + * We could make this known to GCC, but then we would hit the problem of not
> + * being able to easily decide which insns are emitted and how they are
> + * bundled. So we choose to save nothing when calling __mcount and we will
> + * save all caller registers once __mcount called.
> + *
> + * Another specificity is that parameter is passed in $r32 instead of $r0.
> + *
> + * Our stack will be:
> + * 0 | $FP $RA $r15 -
> + * 32 | $r0 $r1 $r2 $r3
> + * 64 | $r4 $r5 $r6 $r7
> + * 96 | $r8 $r9 $r10 $r11
> + * 128| $r32 $r33 $r34 $r35
> + * 160| $r36 $r37 $r38 $r39
> + * 192| $r40 $r41 $r42 $r43
> + * 224| $r44 $r45 $r46 $r47
> + * 256| $r48 $r49 $r50 $r51
> + * 288| $r52 $r53 $r54 $r55
> + * 320| $r56 $r57 $r58 $r59
> + * 352| $r60 $r61 $r62 $r63
> + * 384| $r16 $r17
> + */
> +
> +#define STACK_OFFSET_FP 0
> +#define STACK_OFFSET_RA (STACK_OFFSET_FP + 8)
> +#define STACK_OFFSET_R15 (STACK_OFFSET_RA + 8)
> +#define STACK_OFFSET_Q0 (STACK_OFFSET_FP + 32)
> +#define STACK_OFFSET_Q4 (STACK_OFFSET_Q0 + 32)
> +#define STACK_OFFSET_Q8 (STACK_OFFSET_Q4 + 32)
> +#define STACK_OFFSET_Q32 (STACK_OFFSET_Q8 + 32)
> +#define STACK_OFFSET_Q36 (STACK_OFFSET_Q32 + 32)
> +#define STACK_OFFSET_Q40 (STACK_OFFSET_Q36 + 32)
> +#define STACK_OFFSET_Q44 (STACK_OFFSET_Q40 + 32)
> +#define STACK_OFFSET_Q48 (STACK_OFFSET_Q44 + 32)
> +#define STACK_OFFSET_Q52 (STACK_OFFSET_Q48 + 32)
> +#define STACK_OFFSET_Q56 (STACK_OFFSET_Q52 + 32)
> +#define STACK_OFFSET_Q60 (STACK_OFFSET_Q56 + 32)
> +#define STACK_OFFSET_P16 (STACK_OFFSET_Q60 + 32)
> +#define STACK_SIZE (STACK_OFFSET_P16 + 16)
> +
> +#define RA_DWARF_REGNO (64 + KVX_SFR_RA)
> +#define R14_DWARF_REGNO 14
> +
> +.macro save_all_context_but_P16
> + /*
> + * r16 and r17 are already saved. r16 is already used to store the
> + * address of the function tracer if any.
> + * so !!! USE r17 as scratch register !!!
> + */
> + get $r17 = $ra
> + sd STACK_OFFSET_FP[$r12] = $r14
> + ;;
> + .cfi_offset R14_DWARF_REGNO, -STACK_SIZE + STACK_OFFSET_FP
> + sd STACK_OFFSET_RA[$r12] = $r17
> + ;;
> + .cfi_offset RA_DWARF_REGNO, -STACK_SIZE + STACK_OFFSET_RA
> + sd STACK_OFFSET_R15[$r12] = $r15
> + ;;
> + so STACK_OFFSET_Q0[$r12] = $r0r1r2r3
> + ;;
> + so STACK_OFFSET_Q4[$r12] = $r4r5r6r7
> + ;;
> + so STACK_OFFSET_Q8[$r12] = $r8r9r10r11
> + ;;
> + so STACK_OFFSET_Q32[$r12] = $r32r33r34r35
> + ;;
> + so STACK_OFFSET_Q36[$r12] = $r36r37r38r39
> + ;;
> + so STACK_OFFSET_Q40[$r12] = $r40r41r42r43
> + ;;
> + so STACK_OFFSET_Q44[$r12] = $r44r45r46r47
> + ;;
> + so STACK_OFFSET_Q48[$r12] = $r48r49r50r51
> + ;;
> + so STACK_OFFSET_Q52[$r12] = $r52r53r54r55
> + ;;
> + so STACK_OFFSET_Q56[$r12] = $r56r57r58r59
> + ;;
> + so STACK_OFFSET_Q60[$r12] = $r60r61r62r63
> + ;;
> +.endm
> +
> +.macro restore_all_context
> + ld $r15 = STACK_OFFSET_RA[$r12]
> + ;;
> + ld $r14 = STACK_OFFSET_FP[$r12]
> + set $ra = $r15
> + ;;
> + .cfi_restore RA_DWARF_REGNO
> + .cfi_restore R14_DWARF_REGNO
> + ld $r15 = STACK_OFFSET_R15[$r12]
> + ;;
> + lq $r16r17 = STACK_OFFSET_P16[$r12]
> + ;;
> + lo $r0r1r2r3 = STACK_OFFSET_Q0[$r12]
> + ;;
> + lo $r4r5r6r7 = STACK_OFFSET_Q4[$r12]
> + ;;
> + lo $r8r9r10r11 = STACK_OFFSET_Q8[$r12]
> + ;;
> + lo $r32r33r34r35 = STACK_OFFSET_Q32[$r12]
> + ;;
> + lo $r36r37r38r39 = STACK_OFFSET_Q36[$r12]
> + ;;
> + lo $r40r41r42r43 = STACK_OFFSET_Q40[$r12]
> + ;;
> + lo $r44r45r46r47 = STACK_OFFSET_Q44[$r12]
> + ;;
> + lo $r48r49r50r51 = STACK_OFFSET_Q48[$r12]
> + ;;
> + lo $r52r53r54r55 = STACK_OFFSET_Q52[$r12]
> + ;;
> + lo $r56r57r58r59 = STACK_OFFSET_Q56[$r12]
> + ;;
> + lo $r60r61r62r63 = STACK_OFFSET_Q60[$r12]
> + ;;
> + addd $r12 = $r12, STACK_SIZE
> + ;;
> + .cfi_def_cfa_offset 0
> +.endm
> +
> +ENTRY(__mcount)
> +.cfi_startproc
> +#ifdef CONFIG_DYNAMIC_FTRACE
> + /*
> + * With dynamic ftrace mcount() is only used during boot and then all
> + * references to it will be patched out never to return. ftrace_caller
> + * is used.
> + */
> + ret
> + ;;
> +#endif
> + addd $r12 = $r12, -STACK_SIZE
> + ;;
> + .cfi_def_cfa_offset STACK_SIZE
> + sq STACK_OFFSET_P16[$r12] = $r16r17
> + ;;
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> + /*
> + * Check the function pointers ftrace_graph_return
> + * (compare to ftrace_stub) and ftrace_graph_entry (compare to
> + * ftrace_graph_entry_stub). If either of those is not set to the
> + * relevant stub function, call the arch-specific function
> + * ftrace_graph_caller.
> + */
> + make $r16 = ftrace_graph_return
> + make $r17 = ftrace_graph_entry
> + ;;
> + ld $r16 = 0[$r16]
> + ;;
> + compd.eq $r16 = $r16, ftrace_stub
> + ld $r17 = 0[$r17]
> + ;;
> + /*
> + * If ftrace_graph_return is not equal to ftrace_stub ($r16 == 0) then
> + * jump to ftrace_graph_caller. Otherwise compare ftrace_graph_entry.
> + */
> + cb.deqz $r16? ftrace_graph_caller
> + compd.eq $r17 = $r17, ftrace_graph_entry_stub
> + ;;
> + /*
> + * If ftrace_graph_entry is equal to ftrace_graph_entry_stub ($r17 == 1)
> + * then jump to ftrace_graph_end. If not we continue.
> + */
> + cb.dnez $r17? ftrace_graph_end
> + ;;
> +ftrace_graph_caller:
> + save_all_context_but_P16
> + ;;
> +GLOBAL(ftrace_graph_call)
> + ld $r16 = 0[$r12]
> + ;;
> + addd $r0 = $r16, 0x8 /* a pointer to the frompc */
> + get $r1 = $ra /* selfpc */
> + copyd $r2 = $r16 /* FP of the function that called __mcount */
> + ;;
> + /* get the address of the bundle */
> + addd $r1 = $r1, -KVX_INSN_SYLLABLE_WIDTH
> + ;;
> + call prepare_ftrace_return
> + ;;
> + restore_all_context
> + ;;
> + ret
> + ;;
> +ftrace_graph_end:
> + .cfi_def_cfa_offset STACK_SIZE
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> + /*
> + * Check if function pointer ftrace_trace_function is set to
> + * ftrace_stub.
> + * If it isn't, then call that function in the same way the mcount
> + * function normally calls __mcount_internal
> + * - the first argument is the "frompc" ($r0)
> + * - the second argument is the "selfpc" ($ra - mcount size adjustement)
> + * As we don't use any callee saved registers we don't need to save
> + * them.
> + */
> + make $r16 = ftrace_trace_function
> + ;;
> + ld $r16 = 0[$r16]
> + ;;
> + /*
> + * NOTE: keep value in $r16 (the function we may call if set). See
> + * comment in save_all_context_but_P16.
> + */
> + compd.eq $r17 = $r16, ftrace_stub
> + ;;
> + /*
> + * If ftrace_trace_function is equal to ftrace_stub ($r17 == 1), there
> + * is nothing to do, so return immediately otherwise call the function
> + * in $r16 with correct parameters.
> + */
> + cb.deqz $r17? 1f
> + ;;
> + /* Only need to restore P16 */
> + lq $r16r17 = STACK_OFFSET_P16[$r12]
> + ;;
> + addd $r12 = $r12, STACK_SIZE
> + ;;
> + .cfi_def_cfa_offset 0
> + ret
> + ;;
> +
> +1:
> + save_all_context_but_P16
> + ;;
> + copyd $r0 = $r32
> + get $r1 = $ra
> + ;;
> + addd $r1 = $r1, -KVX_INSN_SYLLABLE_WIDTH
> + ;;
> + icall $r16
> + ;;
> + restore_all_context
> + ;;
> + ret
> + ;;
> +.cfi_endproc
> +ENDPROC(__mcount)
> +
> +/* Required by ftrace_graph_ret_addr */
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> +ENTRY(return_to_handler)
> + addd $r12 = $r12, -128
> + ;;
> + so 0[$r12] = $r0r1r2r3
> + ;;
> + so 32[$r12] = $r4r5r6r7
> + ;;
> + so 64[$r12] = $r8r9r10r11
> + ;;
> + call ftrace_return_to_handler
> + ;;
> + set $ra = $r0
> + lo $r0r1r2r3 = 0[$r12]
> + ;;
> + lo $r4r5r6r7 = 32[$r12]
> + ;;
> + lo $r8r9r10r11 = 64[$r12]
> + addd $r12 = $r12, 128
> + ;;
> + ret
> + ;;
> +ENDPROC(return_to_handler)
> +#endif
> +
> +#ifdef CONFIG_DYNAMIC_FTRACE
> +ENTRY(ftrace_caller)
> +.cfi_startproc
> +
> + addd $r12 = $r12, -STACK_SIZE
> + ;;
> + .cfi_def_cfa_offset STACK_SIZE
> + sq STACK_OFFSET_P16[$r12] = $r16r17
> + ;;
> + save_all_context_but_P16
> + ;;
> + copyd $r0 = $r32
> + get $r1 = $ra
> + ;;
> + addd $r1 = $r1, -KVX_INSN_SYLLABLE_WIDTH
> + ;;
> + goto ftrace_call
> + ;;
> +.align 64
> +GLOBAL(ftrace_call)
> + /*
> + * To work with module we need to use far call. So prepare the
> + * space for them:
> + * - make: 3 syllables
> + * - icall/igoto: 1 syllable
> + *
> + * We are using igoto in ftrace_enable_ftrace_graph_caller(). This is
> + * because this path is a tweak to enable function graph tracer with
> + * dynamic ftrace. We need to jump to ftrace_graph_call without updating
> + * RA and FP. Once in ftrace_graph_call we will return to the correct
> + * address from the classical path of the function graph tracer.
> + */
> + nop
> + ;;
> + nop
> + ;;
> + nop
> + ;;
> + nop
> + ;;
> + restore_all_context
> + ;;
> + ret
> + ;;
> +.cfi_endproc
> +ENDPROC(ftrace_caller)
> +#endif // CONFIG_DYNAMIC_FTRACE
> +
> +ENTRY(ftrace_stub)
> + ret
> + ;;
> +ENDPROC(ftrace_stub)
> +
> diff --git a/arch/kvx/kernel/return_address.c b/arch/kvx/kernel/return_address.c
> new file mode 100644
> index 000000000000..3ddf7b79662c
> --- /dev/null
> +++ b/arch/kvx/kernel/return_address.c
> @@ -0,0 +1,55 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * derived from arch/arm64/kernel/return_address.c
> + *
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Clement Leger
> + */
> +
> +#include <linux/export.h>
> +#include <linux/ftrace.h>
> +#include <linux/kprobes.h>
> +
> +#include <asm/stacktrace.h>
> +
> +struct return_address_data {
> + unsigned int level;
> + void *addr;
> +};
> +
> +static bool save_return_addr(unsigned long pc, void *d)
> +{
> + struct return_address_data *data = d;
> +
> + /* We hit the desired level, return the address */
> + if (data->level == 0) {
> + data->addr = (void *) pc;
> + return true;
> + }
> +
> + data->level--;
> + return false;
> +}
> +NOKPROBE_SYMBOL(save_return_addr);
> +
> +void *return_address(unsigned int level)
> +{
> + struct return_address_data data;
> + struct stackframe frame;
> +
> + /* Skip this function + caller */
> + data.level = level + 2;
> + data.addr = NULL;
> +
> + start_stackframe(&frame,
> + (unsigned long) __builtin_frame_address(0),
> + (unsigned long) return_address);
> + walk_stackframe(current, &frame, save_return_addr, &data);
> +
> + if (!data.level)
> + return data.addr;
> + else
> + return NULL;
> +}
> +EXPORT_SYMBOL_GPL(return_address);
> +NOKPROBE_SYMBOL(return_address);
--
Clément Léger,
Embedded Linux and Kernel engineer at Bootlin
https://bootlin.com
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 21/25] kvx: Add support for ftrace
2023-01-05 12:55 ` Clément Léger
@ 2023-01-05 14:20 ` Steven Rostedt
0 siblings, 0 replies; 113+ messages in thread
From: Steven Rostedt @ 2023-01-05 14:20 UTC (permalink / raw)
To: Clément Léger
Cc: Yann Sionneau, Masami Hiramatsu, Mark Rutland, linux-kernel,
Guillaume Thouvenin, Julian Vetter, Marius Gligor
On Thu, 5 Jan 2023 13:55:26 +0100
Clément Léger <clement.leger@bootlin.com> wrote:
> > +/* The longest insns we check are for the far call: make + icall */
> > +#define MAX_SYLLABLES_TO_CHECK (KVX_INSN_MAKE_IMM64_SIZE + INSN_ICALL_SYLLABLE_SIZE)
> > +
> > +static int read_insns_and_check(u32 *insns, u8 insns_len, u32 *addr)
> > +{
> > + u32 insns_read[MAX_SYLLABLES_TO_CHECK];
> > + int syllables = insns_len / KVX_INSN_SYLLABLE_WIDTH;
> > + int i;
> > +
> > + if (syllables > MAX_SYLLABLES_TO_CHECK) {
> > + pr_err("%s: shouldn't have more than %d syllables to check\n",
> > + __func__, MAX_SYLLABLES_TO_CHECK);
> > + return -EINVAL;
> > + }
> > +
> > + if (kvx_insns_read(insns_read, insns_len, addr)) {
> > + pr_err("%s: error when trying to read syllable\n", __func__);
> > + return -EFAULT;
> > + }
> > +
> > + for (i = 0; i < syllables; i++) {
> > + if (insns[i] != insns_read[i]) {
> > + pr_err("%s: Instruction verification failed at PC 0x%lx\n",
> > + __func__,
> > + (unsigned long)addr + i * KVX_INSN_SYLLABLE_WIDTH);
> > + pr_err("%s: \tExpect 0x%x\n", __func__, insns[i]);
> > + pr_err("%s: \tRead 0x%x\n", __func__, insns_read[i]);
> > + return -EFAULT;
> > + }
> > + }
> > +
> > + return 0;
> > +}
>
> Hi Yann,
>
> Is this still needed ? I'm guessing the instructions should always be
> correctly written no ? If not, something probably went horribly wrong ;)
I would definitely keep it. Code modifications can be quite fragile. Most
of the time things don't go wrong, but when they do, having these checks
makes it obvious to where the problem happened. Problems here is usually
some code mapping that got incorrectly flagged to be traced, when it
shouldn't be. Reporting these errors helps find what that was.
-- Steve
>
> > +
> > +static int write_insns_and_check(u32 *insns, u8 insns_len, u32 *insn_addr)
> > +{
> > + int ret;
> > +
> > + ret = kvx_insns_write_nostop(insns, insns_len, insn_addr);
> > + if (ret)
> > + return ret;
> > +
> > + /* Check that what have been written is correct. */
> > + return read_insns_and_check(insns, insns_len, insn_addr);
> > +}
> > +
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 21/25] kvx: Add support for ftrace
2023-01-03 16:43 ` [RFC PATCH 21/25] kvx: Add support for ftrace Yann Sionneau
2023-01-05 12:55 ` Clément Léger
@ 2023-01-05 14:50 ` Mark Rutland
1 sibling, 0 replies; 113+ messages in thread
From: Mark Rutland @ 2023-01-05 14:50 UTC (permalink / raw)
To: Yann Sionneau
Cc: Steven Rostedt, Masami Hiramatsu, linux-kernel, Clement Leger,
Guillaume Thouvenin, Julian Vetter, Marius Gligor
Hi,
As others have said, I think it'd be worth splitting this series up, with this
being part of a follow-up, since it's not vital to boot and use the kernel.
That'll aid review of the smaller initial series, and later we can focus on the
bits and pieces here.
That said, I have a few comments below.
On Tue, Jan 03, 2023 at 05:43:55PM +0100, Yann Sionneau wrote:
> Add support for ftrace to kvx arch.
>
> CC: Steven Rostedt <rostedt@goodmis.org>
> CC: Masami Hiramatsu <mhiramat@kernel.org>
> CC: Mark Rutland <mark.rutland@arm.com>
> CC: linux-kernel@vger.kernel.org
> Co-developed-by: Clement Leger <clement.leger@bootlin.com>
> Signed-off-by: Clement Leger <clement.leger@bootlin.com>
> Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
> Co-developed-by: Julian Vetter <jvetter@kalray.eu>
> Signed-off-by: Julian Vetter <jvetter@kalray.eu>
> Co-developed-by: Marius Gligor <mgligor@kalray.eu>
> Signed-off-by: Marius Gligor <mgligor@kalray.eu>
> Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
> Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
> ---
> arch/kvx/include/asm/ftrace.h | 41 ++++
> arch/kvx/kernel/ftrace.c | 339 ++++++++++++++++++++++++++++++
> arch/kvx/kernel/mcount.S | 340 +++++++++++++++++++++++++++++++
> arch/kvx/kernel/return_address.c | 55 +++++
> 4 files changed, 775 insertions(+)
> create mode 100644 arch/kvx/include/asm/ftrace.h
> create mode 100644 arch/kvx/kernel/ftrace.c
> create mode 100644 arch/kvx/kernel/mcount.S
> create mode 100644 arch/kvx/kernel/return_address.c
>
> diff --git a/arch/kvx/include/asm/ftrace.h b/arch/kvx/include/asm/ftrace.h
> new file mode 100644
> index 000000000000..f7bcb325d255
> --- /dev/null
> +++ b/arch/kvx/include/asm/ftrace.h
> @@ -0,0 +1,41 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Guillaume Thouvenin
> + * Clement Leger
> + */
> +
> +#ifndef _ASM_KVX_FTRACE_H
> +#define _ASM_KVX_FTRACE_H
> +
> +#include <asm/insns_defs.h>
The <asm/insns_defs.h> header file gets added by a subsequent patch, in:
"[RFC PATCH 23/25] kvx: Add debugging related support"
Obviously that means it won't build as-is, but it's also confusing for
reviewers.
> +
> +#define INSN_MAKE_IMM64_SYLLABLE_SIZE INSN_SIZE(MAKE_IMM64)
IIUC this is 12 bytes.
> +#define INSN_ICALL_SYLLABLE_SIZE INSN_SIZE(ICALL)
> +#define INSN_IGOTO_SYLLABLE_SIZE INSN_SIZE(IGOTO)
> +#define INSN_CALL_SYLLABLE_SIZE INSN_SIZE(CALL)
> +#define INSN_NOP_SYLLABLE_SIZE INSN_SIZE(NOP)
IIUC all these are all 4 bytes.
> +
> +#define INSN_ICALL_REG_MASK 0x3f
> +
> +#define MCOUNT_ADDR ((unsigned long)(__mcount))
> +#define MCOUNT_INSN_SIZE INSN_CALL_SYLLABLE_SIZE /* sizeof mcount call */
Does your GCC port support -fpatchable-function-entry?
If it does, I would strongly recommend moving over to using that, as it'll
allow you to implement FTRACE_WITH_ARGS and all the fancy features dependent
upon that.
If it doesn't, I'd strongly recommend implementing that on the GCC side now so
that you can make use of it in future without having to retain support for an
mcount based ftrace. That'll make things easier to maintain going forwards, and
it'll give you a lot of flexibility w.r.t. your ftrace implementation.
> +extern void ftrace_graph_call(void);
> +
> +#ifdef CONFIG_DYNAMIC_FTRACE
> +extern unsigned long ftrace_call_adjust(unsigned long addr);
> +struct dyn_arch_ftrace {
> + unsigned int insn;
> +};
> +#endif
> +
> +extern void *return_address(unsigned int level);
> +#define ftrace_return_address(n) return_address(n)
> +
> +#ifdef CONFIG_FUNCTION_TRACER
> +extern void __mcount(void);
> +#define mcount __mcount
> +#endif /* CONFIG_FUNCTION_TRACER */
> +
> +#endif /* _ASM_KVX_FTRACE_H */
> diff --git a/arch/kvx/kernel/ftrace.c b/arch/kvx/kernel/ftrace.c
> new file mode 100644
> index 000000000000..4c9e3ef62714
> --- /dev/null
> +++ b/arch/kvx/kernel/ftrace.c
> @@ -0,0 +1,339 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Guillaume Thouvenin
> + * Marius Gligor
> + * Clement Leger
> + */
> +
> +#include <linux/ftrace.h>
> +#include <linux/atomic.h>
> +#include <linux/stop_machine.h>
> +#include <asm/insns.h>
> +#include <asm/insns_defs.h>
> +#include <asm/cacheflush.h>
> +
> +/* The longest insns we check are for the far call: make + icall */
> +#define MAX_SYLLABLES_TO_CHECK (KVX_INSN_MAKE_IMM64_SIZE + INSN_ICALL_SYLLABLE_SIZE)
What's the range on your CALL instruction? Is that smaller than your kernel
image size, or is this just for modules?
> +
> +static int read_insns_and_check(u32 *insns, u8 insns_len, u32 *addr)
> +{
> + u32 insns_read[MAX_SYLLABLES_TO_CHECK];
> + int syllables = insns_len / KVX_INSN_SYLLABLE_WIDTH;
> + int i;
> +
> + if (syllables > MAX_SYLLABLES_TO_CHECK) {
> + pr_err("%s: shouldn't have more than %d syllables to check\n",
> + __func__, MAX_SYLLABLES_TO_CHECK);
> + return -EINVAL;
> + }
> +
> + if (kvx_insns_read(insns_read, insns_len, addr)) {
> + pr_err("%s: error when trying to read syllable\n", __func__);
> + return -EFAULT;
> + }
> +
> + for (i = 0; i < syllables; i++) {
> + if (insns[i] != insns_read[i]) {
> + pr_err("%s: Instruction verification failed at PC 0x%lx\n",
> + __func__,
> + (unsigned long)addr + i * KVX_INSN_SYLLABLE_WIDTH);
> + pr_err("%s: \tExpect 0x%x\n", __func__, insns[i]);
> + pr_err("%s: \tRead 0x%x\n", __func__, insns_read[i]);
> + return -EFAULT;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int write_insns_and_check(u32 *insns, u8 insns_len, u32 *insn_addr)
> +{
> + int ret;
> +
> + ret = kvx_insns_write_nostop(insns, insns_len, insn_addr);
> + if (ret)
> + return ret;
Again, kvx_insns_write_nostop() is added by a subsequenbt patch, which makes
this painful to review.
AFAICT this assumees that you can concurrently modify instructions while those
are being executed. Does your architecture provide a strong guanratee there?
For example, on arm64 we can only do this safely for a defined small set of
instructions (including B/BL/BLR/NOP), and even then it doesn't gurantee that
updates to multiple instructions will be seen in order.
I see that ftrace_enable_ftrace_graph_caller() writes multiple instructions,
which strikes me as likely to be unsafe depending on the ordering and atomicity
of both those writes and the instruction fetches which will consume them.
I see that for arch_ftrace_update_code() you use stop_machine(), and I'd expect
similar here, but maybe I've missed something that inhibits concurrent
exxecution?
> +
> + /* Check that what have been written is correct. */
> + return read_insns_and_check(insns, insns_len, insn_addr);
> +}
> +
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> +void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr,
> + unsigned long frame_pointer)
> +{
> + unsigned long return_hooker = (unsigned long)&return_to_handler;
> + unsigned long old;
> +
> + if (unlikely(atomic_read(¤t->tracing_graph_pause)))
> + return;
> +
> + old = *parent;
> + *parent = return_hooker;
> +
> + if (function_graph_enter(old, self_addr, frame_pointer, NULL))
> + *parent = old;
> +}
> +
> +#ifdef CONFIG_DYNAMIC_FTRACE
> +int ftrace_enable_ftrace_graph_caller(void)
> +{
> + unsigned int insn[KVX_INSN_MAKE_IMM64_SIZE + KVX_INSN_IGOTO_SIZE];
> + void *ip = (void *)ftrace_call;
> +
> + KVX_INSN_MAKE_IMM64(insn, KVX_INSN_PARALLEL_EOB, KVX_REG_R32,
> + (unsigned long)&ftrace_graph_call);
> + KVX_INSN_IGOTO(&insn[KVX_INSN_MAKE_IMM64_SIZE],
> + KVX_INSN_PARALLEL_EOB,
> + KVX_REG_R32);
> +
> + return write_insns_and_check(insn,
> + INSN_MAKE_IMM64_SYLLABLE_SIZE
> + + INSN_IGOTO_SYLLABLE_SIZE,
> + ip);
> +}
> +
> +int ftrace_disable_ftrace_graph_caller(void)
> +{
> + unsigned int nop;
> + void *ip = (void *)ftrace_call;
> +
> + KVX_INSN_NOP(&nop, KVX_INSN_PARALLEL_EOB);
> + return write_insns_and_check(&nop,
> + INSN_NOP_SYLLABLE_SIZE,
> + ip + INSN_MAKE_IMM64_SYLLABLE_SIZE);
> +}
> +#endif /* CONFIG_DYNAMIC_FTRACE */
> +
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> +
> +#ifdef CONFIG_DYNAMIC_FTRACE
> +struct kvx_ftrace_modify_param {
> + atomic_t cpu_ack;
> + int cpu_master;
> + int cmd;
> +};
> +
> +static int __ftrace_modify_code_kvx(void *data)
> +{
> + struct kvx_ftrace_modify_param *mp = data;
> + int no_cpus = num_online_cpus();
> + int cpu = smp_processor_id();
> +
> + if (cpu == mp->cpu_master) {
> + ftrace_modify_all_code(mp->cmd);
> +
> + /* Inform the other cpus that they can invalidate I-cache */
> + atomic_inc(&mp->cpu_ack);
> +
> + /* param (which contains the cpu_ack counter) is allocated on the
> + * master stack: the master must wait for all other CPUs to leave
> + * this function before returning and releasing the stack
> + * allocation.
> + */
> + while (atomic_read(&mp->cpu_ack) < no_cpus)
> + cpu_relax();
> + } else {
> + /* Wait for the master cpu to finish the code modification */
> + while (atomic_read(&mp->cpu_ack) == 0)
> + cpu_relax();
> + atomic_inc(&mp->cpu_ack);
> +
> + l1_inval_icache_all();
Wouldn't it make more sense to do the icache invalidate *before* signalling the
master?
This function (and its callees) will need to be noinstr to be safe. There's
some latent issues there on arm64 which I'm slowly working through at the
moment, but there's no need for you to start with the same problems.
> + }
> +
> + return 0;
> +}
> +
> +void arch_ftrace_update_code(int command)
> +{
> + const struct cpumask *cpumask = cpu_online_mask;
Below you use cpu_online_mask directly, so this isn't necessary.
> + struct kvx_ftrace_modify_param mp = {
> + .cpu_ack = ATOMIC_INIT(0),
> + .cpu_master = smp_processor_id(),
> + .cmd = command,
> + };
> +
> + stop_machine(__ftrace_modify_code_kvx, &mp, cpu_online_mask);
> +}
> +
> +unsigned long ftrace_call_adjust(unsigned long addr)
> +{
> + /*
> + * Module are using far call and kernel functions are using
> + * pcrel. If it is a call we don't need to adjust the address but
> + * if it is an icall the address is on the make. The generated code
> + * looks like:
> + *
> + * 1c: e0 00 c4 8f get $r32 = $ra
> + * 20: 00 00 84 e0 00 00 00 80 00 00 00 00 make $r33 = 0 (0x0);;
> + *
> + * 20: R_KVX_S64_LO10 __mcount
> + * 24: R_KVX_S64_UP27 __mcount
> + * 28: R_KVX_S64_EX27 __mcount
> + * 2c: 21 00 dc 0f icall $r33;;
> + *
> + * So we just need to add INSN_MAKE_IMM64_SYLLABLE_SIZE (0xc) to the
> + * address.
> + */
You could cheat here, and place a small veneer/PLT next to each module to do
the indirect branch to ftrace. That way, each call site would only need a pcrel
rather than a far call, which might simplfiy your runtime patching logic.
On arm64 we a special ftrace PLT for this.
> + unsigned int insn;
> +
> + /*
> + * The CALL is 1 syllable while the MAKE IMM64 is 3. But we just
> + * need to check that the first syllable of the MAKE IMM64 is the
> + * LO10. So finally we just need to read one syllable to adjust the
> + * call.
> + */
> + if (kvx_insns_read(&insn, KVX_INSN_SYLLABLE_WIDTH, (void *)addr)) {
> + pr_err("%s: error when trying to read syllable\n", __func__);
> + return 0;
> + }
> +
> + if (IS_INSN(insn, CALL))
> + return addr;
> +
> + if (IS_INSN(insn, MAKE_IMM64))
> + return addr + INSN_MAKE_IMM64_SYLLABLE_SIZE;
> +
> + pr_err("%s: syllable is neither a CALL nor a MAKE\n", __func__);
> + return 0;
> +}
> +
> +/*
> + * Do runtime patching of the active tracer.
> + * This will be modifying the assembly code at the location of the
> + * ftrace_call symbol inside of the ftrace_caller() function.
> + */
> +int ftrace_update_ftrace_func(ftrace_func_t func)
> +{
> + void *ip;
> + unsigned int insn[KVX_INSN_MAKE_IMM64_SIZE + KVX_INSN_ICALL_SIZE];
> +
> + ip = (void *)ftrace_call;
> + KVX_INSN_MAKE_IMM64(insn, KVX_INSN_PARALLEL_EOB, KVX_REG_R32,
> + (unsigned long)func);
> + KVX_INSN_ICALL(&insn[KVX_INSN_MAKE_IMM64_SIZE],
> + KVX_INSN_PARALLEL_EOB,
> + KVX_REG_R32);
> + return write_insns_and_check(insn,
> + INSN_MAKE_IMM64_SYLLABLE_SIZE
> + + INSN_ICALL_SYLLABLE_SIZE,
> + ip);
> +}
> +
> +/*
> + * Turn the mcount call site into a call to an arbitrary location (but
> + * typically that is ftrace_caller()) at runtime.
> + */
> +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
> +{
> + void *ip = (void *)rec->ip;
> + unsigned int insn;
> + int ret;
> +
> + /* Ensure that a NOP will be replaced */
> + if (kvx_insns_read(&insn, KVX_INSN_SYLLABLE_WIDTH, ip)) {
> + pr_err("%s: failed to read insn\n", __func__);
> + return -EFAULT;
> + }
> +
> + if (!IS_INSN(insn, NOP)) {
> + pr_err("%s: insn 0x%x is not a NOP\n", __func__, insn);
> + return -EINVAL;
> + }
> +
> + /*
> + * Now we can replace the instruction depending of what has been
> + * nopified (call or icall)
> + */
> + insn = rec->arch.insn;
> +
> + if (IS_INSN(insn, CALL)) {
> + s32 pcrel = addr - (unsigned long) ip;
> + u32 insn_call;
> +
> + BUG_ON(KVX_INSN_GOTO_PCREL27_CHECK(pcrel));
> + KVX_INSN_CALL(&insn_call, KVX_INSN_PARALLEL_EOB, pcrel);
> +
> + return write_insns_and_check(&insn_call,
> + INSN_CALL_SYLLABLE_SIZE, ip);
> + }
> +
> + if (IS_INSN(insn, ICALL)) {
> + u32 insn_make[KVX_INSN_MAKE_IMM64_SIZE];
> + u32 r = insn & INSN_ICALL_REG_MASK;
> +
> + KVX_INSN_MAKE_IMM64(insn_make, KVX_INSN_PARALLEL_EOB, r, addr);
> + ret = write_insns_and_check(insn_make,
> + INSN_MAKE_IMM64_SYLLABLE_SIZE,
> + ip - INSN_MAKE_IMM64_SYLLABLE_SIZE);
> + if (ret)
> + return ret;
> +
> + return write_insns_and_check(&insn,
> + INSN_ICALL_SYLLABLE_SIZE, ip);
> + }
> +
> + /* It is neither a call nor an icall */
> + pr_err("%s: insn 0x%x is neither a CALL nor ICALL\n", __func__, insn);
> + return -EINVAL;
> +}
> +
> +/*
> + * Turn the mcount call site into a nop at runtime
> + */
> +int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec,
> + unsigned long addr)
> +{
> + unsigned long ip = rec->ip;
> + unsigned int insn;
> + unsigned int nop;
> +
> + /*
> + * Ensure that the instruction that will be replaced is a call or an
> + * icall to addr.
> + */
> + if (kvx_insns_read(&insn, KVX_INSN_SYLLABLE_WIDTH, (void *)ip)) {
> + pr_err("%s: error when trying to read syllable\n", __func__);
> + return -EFAULT;
> + }
> +
> + if (IS_INSN(insn, CALL)) {
> + int pcrel = ((int)(insn & 0x7ffffff) << 5) >> 3;
> +
> + if ((ip + pcrel != addr)) {
> + pr_err("%s: failed to check call addr 0x%lx != 0x%lx\n",
> + __func__, ip + pcrel, addr);
> + return -EINVAL;
> + }
> + } else if (IS_INSN(insn, ICALL)) {
> + unsigned int insn_make[KVX_INSN_MAKE_IMM64_SIZE];
> + unsigned int reg = insn & INSN_ICALL_REG_MASK;
> + int ret;
> +
> + KVX_INSN_MAKE_IMM64(insn_make,
> + KVX_INSN_PARALLEL_EOB, reg,
> + addr);
> +
> + ret = read_insns_and_check(insn_make,
> + INSN_MAKE_IMM64_SYLLABLE_SIZE,
> + (void *)ip -
> + INSN_MAKE_IMM64_SYLLABLE_SIZE);
> + if (ret)
> + return ret;
> + } else {
> + pr_err("%s: insn 0x%x is neither a CALL nor an ICALL\n",
> + __func__, insn);
> + return -EINVAL;
> + }
> +
> + rec->arch.insn = insn;
> + KVX_INSN_NOP(&nop, KVX_INSN_PARALLEL_EOB);
> + return write_insns_and_check(&nop, INSN_NOP_SYLLABLE_SIZE, (void *)ip);
> +}
> +
> +#endif /* CONFIG_DYNAMIC_FTRACE */
> +
> +/* __mcount is defined in mcount.S */
> +EXPORT_SYMBOL(__mcount);
> diff --git a/arch/kvx/kernel/mcount.S b/arch/kvx/kernel/mcount.S
> new file mode 100644
> index 000000000000..317797aeed42
> --- /dev/null
> +++ b/arch/kvx/kernel/mcount.S
> @@ -0,0 +1,340 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017-2023 Kalray Inc.
> + * Author(s): Guillaume Thouvenin
> + * Marius Gligor
> + * Clement Leger
> + */
> +
> +#include <linux/linkage.h>
> +#include <asm/insns_defs.h>
> +#include <asm/sfr_defs.h>
> +
> +.altmacro
> +
> +/*
> + * !!! The call to __mcount is special and breaks the ABI !!!
> + * It is because with dynamic ftrace the call can be replaced by any
> + * tracer and GCC cannot assume which caller registers must be saved.
> + * It's because the call is being inserted in the prologue of functions
> + * that is out of GCC knowledge.
> + * We could make this known to GCC, but then we would hit the problem of not
> + * being able to easily decide which insns are emitted and how they are
> + * bundled. So we choose to save nothing when calling __mcount and we will
> + * save all caller registers once __mcount called.
What about the return address? I assume that this is a branch-and-link
architecture, and CALL/ICALL clobber that?
Thanks,
Mark.
> + *
> + * Another specificity is that parameter is passed in $r32 instead of $r0.
> + *
> + * Our stack will be:
> + * 0 | $FP $RA $r15 -
> + * 32 | $r0 $r1 $r2 $r3
> + * 64 | $r4 $r5 $r6 $r7
> + * 96 | $r8 $r9 $r10 $r11
> + * 128| $r32 $r33 $r34 $r35
> + * 160| $r36 $r37 $r38 $r39
> + * 192| $r40 $r41 $r42 $r43
> + * 224| $r44 $r45 $r46 $r47
> + * 256| $r48 $r49 $r50 $r51
> + * 288| $r52 $r53 $r54 $r55
> + * 320| $r56 $r57 $r58 $r59
> + * 352| $r60 $r61 $r62 $r63
> + * 384| $r16 $r17
> + */
> +
> +#define STACK_OFFSET_FP 0
> +#define STACK_OFFSET_RA (STACK_OFFSET_FP + 8)
> +#define STACK_OFFSET_R15 (STACK_OFFSET_RA + 8)
> +#define STACK_OFFSET_Q0 (STACK_OFFSET_FP + 32)
> +#define STACK_OFFSET_Q4 (STACK_OFFSET_Q0 + 32)
> +#define STACK_OFFSET_Q8 (STACK_OFFSET_Q4 + 32)
> +#define STACK_OFFSET_Q32 (STACK_OFFSET_Q8 + 32)
> +#define STACK_OFFSET_Q36 (STACK_OFFSET_Q32 + 32)
> +#define STACK_OFFSET_Q40 (STACK_OFFSET_Q36 + 32)
> +#define STACK_OFFSET_Q44 (STACK_OFFSET_Q40 + 32)
> +#define STACK_OFFSET_Q48 (STACK_OFFSET_Q44 + 32)
> +#define STACK_OFFSET_Q52 (STACK_OFFSET_Q48 + 32)
> +#define STACK_OFFSET_Q56 (STACK_OFFSET_Q52 + 32)
> +#define STACK_OFFSET_Q60 (STACK_OFFSET_Q56 + 32)
> +#define STACK_OFFSET_P16 (STACK_OFFSET_Q60 + 32)
> +#define STACK_SIZE (STACK_OFFSET_P16 + 16)
> +
> +#define RA_DWARF_REGNO (64 + KVX_SFR_RA)
> +#define R14_DWARF_REGNO 14
> +
> +.macro save_all_context_but_P16
> + /*
> + * r16 and r17 are already saved. r16 is already used to store the
> + * address of the function tracer if any.
> + * so !!! USE r17 as scratch register !!!
> + */
> + get $r17 = $ra
> + sd STACK_OFFSET_FP[$r12] = $r14
> + ;;
> + .cfi_offset R14_DWARF_REGNO, -STACK_SIZE + STACK_OFFSET_FP
> + sd STACK_OFFSET_RA[$r12] = $r17
> + ;;
> + .cfi_offset RA_DWARF_REGNO, -STACK_SIZE + STACK_OFFSET_RA
> + sd STACK_OFFSET_R15[$r12] = $r15
> + ;;
> + so STACK_OFFSET_Q0[$r12] = $r0r1r2r3
> + ;;
> + so STACK_OFFSET_Q4[$r12] = $r4r5r6r7
> + ;;
> + so STACK_OFFSET_Q8[$r12] = $r8r9r10r11
> + ;;
> + so STACK_OFFSET_Q32[$r12] = $r32r33r34r35
> + ;;
> + so STACK_OFFSET_Q36[$r12] = $r36r37r38r39
> + ;;
> + so STACK_OFFSET_Q40[$r12] = $r40r41r42r43
> + ;;
> + so STACK_OFFSET_Q44[$r12] = $r44r45r46r47
> + ;;
> + so STACK_OFFSET_Q48[$r12] = $r48r49r50r51
> + ;;
> + so STACK_OFFSET_Q52[$r12] = $r52r53r54r55
> + ;;
> + so STACK_OFFSET_Q56[$r12] = $r56r57r58r59
> + ;;
> + so STACK_OFFSET_Q60[$r12] = $r60r61r62r63
> + ;;
> +.endm
> +
> +.macro restore_all_context
> + ld $r15 = STACK_OFFSET_RA[$r12]
> + ;;
> + ld $r14 = STACK_OFFSET_FP[$r12]
> + set $ra = $r15
> + ;;
> + .cfi_restore RA_DWARF_REGNO
> + .cfi_restore R14_DWARF_REGNO
> + ld $r15 = STACK_OFFSET_R15[$r12]
> + ;;
> + lq $r16r17 = STACK_OFFSET_P16[$r12]
> + ;;
> + lo $r0r1r2r3 = STACK_OFFSET_Q0[$r12]
> + ;;
> + lo $r4r5r6r7 = STACK_OFFSET_Q4[$r12]
> + ;;
> + lo $r8r9r10r11 = STACK_OFFSET_Q8[$r12]
> + ;;
> + lo $r32r33r34r35 = STACK_OFFSET_Q32[$r12]
> + ;;
> + lo $r36r37r38r39 = STACK_OFFSET_Q36[$r12]
> + ;;
> + lo $r40r41r42r43 = STACK_OFFSET_Q40[$r12]
> + ;;
> + lo $r44r45r46r47 = STACK_OFFSET_Q44[$r12]
> + ;;
> + lo $r48r49r50r51 = STACK_OFFSET_Q48[$r12]
> + ;;
> + lo $r52r53r54r55 = STACK_OFFSET_Q52[$r12]
> + ;;
> + lo $r56r57r58r59 = STACK_OFFSET_Q56[$r12]
> + ;;
> + lo $r60r61r62r63 = STACK_OFFSET_Q60[$r12]
> + ;;
> + addd $r12 = $r12, STACK_SIZE
> + ;;
> + .cfi_def_cfa_offset 0
> +.endm
> +
> +ENTRY(__mcount)
> +.cfi_startproc
> +#ifdef CONFIG_DYNAMIC_FTRACE
> + /*
> + * With dynamic ftrace mcount() is only used during boot and then all
> + * references to it will be patched out never to return. ftrace_caller
> + * is used.
> + */
> + ret
> + ;;
> +#endif
> + addd $r12 = $r12, -STACK_SIZE
> + ;;
> + .cfi_def_cfa_offset STACK_SIZE
> + sq STACK_OFFSET_P16[$r12] = $r16r17
> + ;;
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> + /*
> + * Check the function pointers ftrace_graph_return
> + * (compare to ftrace_stub) and ftrace_graph_entry (compare to
> + * ftrace_graph_entry_stub). If either of those is not set to the
> + * relevant stub function, call the arch-specific function
> + * ftrace_graph_caller.
> + */
> + make $r16 = ftrace_graph_return
> + make $r17 = ftrace_graph_entry
> + ;;
> + ld $r16 = 0[$r16]
> + ;;
> + compd.eq $r16 = $r16, ftrace_stub
> + ld $r17 = 0[$r17]
> + ;;
> + /*
> + * If ftrace_graph_return is not equal to ftrace_stub ($r16 == 0) then
> + * jump to ftrace_graph_caller. Otherwise compare ftrace_graph_entry.
> + */
> + cb.deqz $r16? ftrace_graph_caller
> + compd.eq $r17 = $r17, ftrace_graph_entry_stub
> + ;;
> + /*
> + * If ftrace_graph_entry is equal to ftrace_graph_entry_stub ($r17 == 1)
> + * then jump to ftrace_graph_end. If not we continue.
> + */
> + cb.dnez $r17? ftrace_graph_end
> + ;;
> +ftrace_graph_caller:
> + save_all_context_but_P16
> + ;;
> +GLOBAL(ftrace_graph_call)
> + ld $r16 = 0[$r12]
> + ;;
> + addd $r0 = $r16, 0x8 /* a pointer to the frompc */
> + get $r1 = $ra /* selfpc */
> + copyd $r2 = $r16 /* FP of the function that called __mcount */
> + ;;
> + /* get the address of the bundle */
> + addd $r1 = $r1, -KVX_INSN_SYLLABLE_WIDTH
> + ;;
> + call prepare_ftrace_return
> + ;;
> + restore_all_context
> + ;;
> + ret
> + ;;
> +ftrace_graph_end:
> + .cfi_def_cfa_offset STACK_SIZE
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> + /*
> + * Check if function pointer ftrace_trace_function is set to
> + * ftrace_stub.
> + * If it isn't, then call that function in the same way the mcount
> + * function normally calls __mcount_internal
> + * - the first argument is the "frompc" ($r0)
> + * - the second argument is the "selfpc" ($ra - mcount size adjustement)
> + * As we don't use any callee saved registers we don't need to save
> + * them.
> + */
> + make $r16 = ftrace_trace_function
> + ;;
> + ld $r16 = 0[$r16]
> + ;;
> + /*
> + * NOTE: keep value in $r16 (the function we may call if set). See
> + * comment in save_all_context_but_P16.
> + */
> + compd.eq $r17 = $r16, ftrace_stub
> + ;;
> + /*
> + * If ftrace_trace_function is equal to ftrace_stub ($r17 == 1), there
> + * is nothing to do, so return immediately otherwise call the function
> + * in $r16 with correct parameters.
> + */
> + cb.deqz $r17? 1f
> + ;;
> + /* Only need to restore P16 */
> + lq $r16r17 = STACK_OFFSET_P16[$r12]
> + ;;
> + addd $r12 = $r12, STACK_SIZE
> + ;;
> + .cfi_def_cfa_offset 0
> + ret
> + ;;
> +
> +1:
> + save_all_context_but_P16
> + ;;
> + copyd $r0 = $r32
> + get $r1 = $ra
> + ;;
> + addd $r1 = $r1, -KVX_INSN_SYLLABLE_WIDTH
> + ;;
> + icall $r16
> + ;;
> + restore_all_context
> + ;;
> + ret
> + ;;
> +.cfi_endproc
> +ENDPROC(__mcount)
> +
> +/* Required by ftrace_graph_ret_addr */
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> +ENTRY(return_to_handler)
> + addd $r12 = $r12, -128
> + ;;
> + so 0[$r12] = $r0r1r2r3
> + ;;
> + so 32[$r12] = $r4r5r6r7
> + ;;
> + so 64[$r12] = $r8r9r10r11
> + ;;
> + call ftrace_return_to_handler
> + ;;
> + set $ra = $r0
> + lo $r0r1r2r3 = 0[$r12]
> + ;;
> + lo $r4r5r6r7 = 32[$r12]
> + ;;
> + lo $r8r9r10r11 = 64[$r12]
> + addd $r12 = $r12, 128
> + ;;
> + ret
> + ;;
> +ENDPROC(return_to_handler)
> +#endif
> +
> +#ifdef CONFIG_DYNAMIC_FTRACE
> +ENTRY(ftrace_caller)
> +.cfi_startproc
> +
> + addd $r12 = $r12, -STACK_SIZE
> + ;;
> + .cfi_def_cfa_offset STACK_SIZE
> + sq STACK_OFFSET_P16[$r12] = $r16r17
> + ;;
> + save_all_context_but_P16
> + ;;
> + copyd $r0 = $r32
> + get $r1 = $ra
> + ;;
> + addd $r1 = $r1, -KVX_INSN_SYLLABLE_WIDTH
> + ;;
> + goto ftrace_call
> + ;;
> +.align 64
> +GLOBAL(ftrace_call)
> + /*
> + * To work with module we need to use far call. So prepare the
> + * space for them:
> + * - make: 3 syllables
> + * - icall/igoto: 1 syllable
> + *
> + * We are using igoto in ftrace_enable_ftrace_graph_caller(). This is
> + * because this path is a tweak to enable function graph tracer with
> + * dynamic ftrace. We need to jump to ftrace_graph_call without updating
> + * RA and FP. Once in ftrace_graph_call we will return to the correct
> + * address from the classical path of the function graph tracer.
> + */
> + nop
> + ;;
> + nop
> + ;;
> + nop
> + ;;
> + nop
> + ;;
> + restore_all_context
> + ;;
> + ret
> + ;;
> +.cfi_endproc
> +ENDPROC(ftrace_caller)
> +#endif // CONFIG_DYNAMIC_FTRACE
> +
> +ENTRY(ftrace_stub)
> + ret
> + ;;
> +ENDPROC(ftrace_stub)
> +
^ permalink raw reply [flat|nested] 113+ messages in thread
* [RFC PATCH 22/25] kvx: Add support for jump labels
2023-01-03 16:43 ` Yann Sionneau
` (22 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Peter Zijlstra, Josh Poimboeuf, Jason Baron,
Steven Rostedt, Ard Biesheuvel, linux-kernel, Clement Leger,
Julian Vetter
Add support for jump labels to kvx arch.
CC: Peter Zijlstra <peterz@infradead.org>
CC: Josh Poimboeuf <jpoimboe@kernel.org>
CC: Jason Baron <jbaron@akamai.com>
CC: Steven Rostedt <rostedt@goodmis.org>
CC: Ard Biesheuvel <ardb@kernel.org>
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/jump_label.h | 59 +++++++++++++++++++++++++++++++
arch/kvx/kernel/jump_label.c | 34 ++++++++++++++++++
2 files changed, 93 insertions(+)
create mode 100644 arch/kvx/include/asm/jump_label.h
create mode 100644 arch/kvx/kernel/jump_label.c
diff --git a/arch/kvx/include/asm/jump_label.h b/arch/kvx/include/asm/jump_label.h
new file mode 100644
index 000000000000..5becccaad20c
--- /dev/null
+++ b/arch/kvx/include/asm/jump_label.h
@@ -0,0 +1,59 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_JUMP_LABEL_H
+#define _ASM_KVX_JUMP_LABEL_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+
+#include <asm/insns_defs.h>
+
+#define JUMP_LABEL_NOP_SIZE (KVX_INSN_NOP_SIZE * KVX_INSN_SYLLABLE_WIDTH)
+
+static __always_inline bool arch_static_branch(struct static_key *key,
+ bool branch)
+{
+ asm_volatile_goto("1:\n\t"
+ "nop\n\t"
+ ";;\n\t"
+ ".pushsection __jump_table, \"aw\"\n\t"
+ ".dword 1b, %l[l_yes], %c0\n\t"
+ ".popsection\n\t"
+ : : "i" (&((char *)key)[branch]) : : l_yes);
+
+ return false;
+l_yes:
+ return true;
+}
+
+static __always_inline bool arch_static_branch_jump(struct static_key *key,
+ bool branch)
+{
+ asm_volatile_goto("1:\n\t"
+ "goto %l[l_yes]\n\t"
+ ";;\n\t"
+ ".pushsection __jump_table, \"aw\"\n\t"
+ ".dword 1b, %l[l_yes], %c0\n\t"
+ ".popsection\n\t"
+ : : "i" (&((char *)key)[branch]) : : l_yes);
+
+ return false;
+l_yes:
+ return true;
+}
+
+typedef u64 jump_label_t;
+
+struct jump_entry {
+ jump_label_t code;
+ jump_label_t target;
+ jump_label_t key;
+};
+
+#endif /* __ASSEMBLY__ */
+#endif
diff --git a/arch/kvx/kernel/jump_label.c b/arch/kvx/kernel/jump_label.c
new file mode 100644
index 000000000000..5a602dbdede0
--- /dev/null
+++ b/arch/kvx/kernel/jump_label.c
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/cpu.h>
+#include <linux/jump_label.h>
+#include <linux/kernel.h>
+#include <linux/memory.h>
+#include <linux/stop_machine.h>
+#include <linux/types.h>
+
+#include <asm/cacheflush.h>
+#include <asm/insns.h>
+#include <asm/insns_defs.h>
+
+void arch_jump_label_transform(struct jump_entry *e,
+ enum jump_label_type type)
+{
+ s32 off = (jump_entry_target(e) - jump_entry_code(e));
+ u32 insn_code;
+ u32 *insn_addr = (u32 *) jump_entry_code(e);
+
+ if (type == JUMP_LABEL_JMP) {
+ BUG_ON(KVX_INSN_GOTO_PCREL27_CHECK(off));
+
+ KVX_INSN_GOTO(&insn_code, KVX_INSN_PARALLEL_EOB, off);
+ } else {
+ KVX_INSN_NOP(&insn_code, KVX_INSN_PARALLEL_EOB);
+ }
+
+ kvx_insns_write(&insn_code, JUMP_LABEL_NOP_SIZE, insn_addr);
+}
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 23/25] kvx: Add debugging related support
2023-01-03 16:43 ` Yann Sionneau
` (23 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, Clement Leger, Guillaume Thouvenin,
Julian Vetter, Marius Gligor
Add kvx support for ptrace and hw breakpoints.
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Marius Gligor <mgligor@kalray.eu>
Signed-off-by: Marius Gligor <mgligor@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/include/asm/debug.h | 35 ++
arch/kvx/include/asm/insns.h | 16 +
arch/kvx/include/asm/insns_defs.h | 197 +++++++++++
arch/kvx/kernel/break_hook.c | 77 +++++
arch/kvx/kernel/debug.c | 64 ++++
arch/kvx/kernel/hw_breakpoint.c | 556 ++++++++++++++++++++++++++++++
arch/kvx/kernel/insns.c | 146 ++++++++
7 files changed, 1091 insertions(+)
create mode 100644 arch/kvx/include/asm/debug.h
create mode 100644 arch/kvx/include/asm/insns.h
create mode 100644 arch/kvx/include/asm/insns_defs.h
create mode 100644 arch/kvx/kernel/break_hook.c
create mode 100644 arch/kvx/kernel/debug.c
create mode 100644 arch/kvx/kernel/hw_breakpoint.c
create mode 100644 arch/kvx/kernel/insns.c
diff --git a/arch/kvx/include/asm/debug.h b/arch/kvx/include/asm/debug.h
new file mode 100644
index 000000000000..f60c632e6697
--- /dev/null
+++ b/arch/kvx/include/asm/debug.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef __ASM_KVX_DEBUG_HOOK_H_
+#define __ASM_KVX_DEBUG_HOOK_H_
+
+/**
+ * enum debug_ret - Break return value
+ * @DEBUG_HOOK_HANDLED: Hook handled successfully
+ * @DEBUG_HOOK_IGNORED: Hook call has been ignored
+ */
+enum debug_ret {
+ DEBUG_HOOK_HANDLED = 0,
+ DEBUG_HOOK_IGNORED = 1,
+};
+
+/**
+ * struct debug_hook - Debug hook description
+ * @node: List node
+ * @handler: handler called on debug entry
+ * @mode: Hook mode (user/kernel)
+ */
+struct debug_hook {
+ struct list_head node;
+ int (*handler)(u64 ea, struct pt_regs *regs);
+ u8 mode;
+};
+
+void debug_hook_register(struct debug_hook *dbg_hook);
+void debug_hook_unregister(struct debug_hook *dbg_hook);
+
+#endif /* __ASM_KVX_DEBUG_HOOK_H_ */
diff --git a/arch/kvx/include/asm/insns.h b/arch/kvx/include/asm/insns.h
new file mode 100644
index 000000000000..36a9e8335ce8
--- /dev/null
+++ b/arch/kvx/include/asm/insns.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#ifndef _ASM_KVX_INSNS_H
+#define _ASM_KVX_INSNS_H
+
+int kvx_insns_write_nostop(u32 *insns, u8 insns_len, u32 *insn_addr);
+
+int kvx_insns_write(u32 *insns, unsigned long insns_len, u32 *addr);
+
+int kvx_insns_read(u32 *insns, unsigned long insns_len, u32 *addr);
+
+#endif
diff --git a/arch/kvx/include/asm/insns_defs.h b/arch/kvx/include/asm/insns_defs.h
new file mode 100644
index 000000000000..ed8d9d6f0817
--- /dev/null
+++ b/arch/kvx/include/asm/insns_defs.h
@@ -0,0 +1,197 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#ifndef __ASM_KVX_INSNS_DEFS_H_
+#define __ASM_KVX_INSNS_DEFS_H_
+
+#include <linux/bits.h>
+
+#ifndef __ASSEMBLY__
+static inline int check_signed_imm(long long imm, int bits)
+{
+ long long min, max;
+
+ min = -BIT_ULL(bits - 1);
+ max = BIT_ULL(bits - 1) - 1;
+ if (imm < min || imm > max)
+ return 1;
+
+ return 0;
+}
+#endif /* __ASSEMBLY__ */
+
+#define BITMASK(bits) (BIT_ULL(bits) - 1)
+
+#define KVX_INSN_SYLLABLE_WIDTH 4
+
+#define IS_INSN(__insn, __mnemo) \
+ ((__insn & KVX_INSN_ ## __mnemo ## _MASK_0) == \
+ KVX_INSN_ ## __mnemo ## _OPCODE_0)
+
+#define INSN_SIZE(__insn) \
+ (KVX_INSN_ ## __insn ## _SIZE * KVX_INSN_SYLLABLE_WIDTH)
+
+/* Values for general registers */
+#define KVX_REG_R0 0
+#define KVX_REG_R1 1
+#define KVX_REG_R2 2
+#define KVX_REG_R3 3
+#define KVX_REG_R4 4
+#define KVX_REG_R5 5
+#define KVX_REG_R6 6
+#define KVX_REG_R7 7
+#define KVX_REG_R8 8
+#define KVX_REG_R9 9
+#define KVX_REG_R10 10
+#define KVX_REG_R11 11
+#define KVX_REG_R12 12
+#define KVX_REG_SP 12
+#define KVX_REG_R13 13
+#define KVX_REG_TP 13
+#define KVX_REG_R14 14
+#define KVX_REG_FP 14
+#define KVX_REG_R15 15
+#define KVX_REG_R16 16
+#define KVX_REG_R17 17
+#define KVX_REG_R18 18
+#define KVX_REG_R19 19
+#define KVX_REG_R20 20
+#define KVX_REG_R21 21
+#define KVX_REG_R22 22
+#define KVX_REG_R23 23
+#define KVX_REG_R24 24
+#define KVX_REG_R25 25
+#define KVX_REG_R26 26
+#define KVX_REG_R27 27
+#define KVX_REG_R28 28
+#define KVX_REG_R29 29
+#define KVX_REG_R30 30
+#define KVX_REG_R31 31
+#define KVX_REG_R32 32
+#define KVX_REG_R33 33
+#define KVX_REG_R34 34
+#define KVX_REG_R35 35
+#define KVX_REG_R36 36
+#define KVX_REG_R37 37
+#define KVX_REG_R38 38
+#define KVX_REG_R39 39
+#define KVX_REG_R40 40
+#define KVX_REG_R41 41
+#define KVX_REG_R42 42
+#define KVX_REG_R43 43
+#define KVX_REG_R44 44
+#define KVX_REG_R45 45
+#define KVX_REG_R46 46
+#define KVX_REG_R47 47
+#define KVX_REG_R48 48
+#define KVX_REG_R49 49
+#define KVX_REG_R50 50
+#define KVX_REG_R51 51
+#define KVX_REG_R52 52
+#define KVX_REG_R53 53
+#define KVX_REG_R54 54
+#define KVX_REG_R55 55
+#define KVX_REG_R56 56
+#define KVX_REG_R57 57
+#define KVX_REG_R58 58
+#define KVX_REG_R59 59
+#define KVX_REG_R60 60
+#define KVX_REG_R61 61
+#define KVX_REG_R62 62
+#define KVX_REG_R63 63
+
+/* Value for bitfield parallel */
+#define KVX_INSN_PARALLEL_EOB 0x0
+#define KVX_INSN_PARALLEL_NONE 0x1
+
+#define KVX_INSN_PARALLEL(__insn) (((__insn) >> 31) & 0x1)
+
+#define KVX_INSN_MAKE_IMM64_SIZE 3
+#define KVX_INSN_MAKE_IMM64_W64_CHECK(__val) \
+ (check_signed_imm(__val, 64))
+#define KVX_INSN_MAKE_IMM64_MASK_0 0xff030000
+#define KVX_INSN_MAKE_IMM64_OPCODE_0 0xe0000000
+#define KVX_INSN_MAKE_IMM64_SYLLABLE_0(__rw, __w64) \
+ (KVX_INSN_MAKE_IMM64_OPCODE_0 | (((__rw) & 0x3f) << 18) | (((__w64) & 0x3ff) << 6))
+#define KVX_INSN_MAKE_IMM64_MASK_1 0xe0000000
+#define KVX_INSN_MAKE_IMM64_OPCODE_1 0x80000000
+#define KVX_INSN_MAKE_IMM64_SYLLABLE_1(__w64) \
+ (KVX_INSN_MAKE_IMM64_OPCODE_1 | (((__w64) >> 10) & 0x7ffffff))
+#define KVX_INSN_MAKE_IMM64_SYLLABLE_2(__p, __w64) \
+ (((__p) << 31) | (((__w64) >> 37) & 0x7ffffff))
+#define KVX_INSN_MAKE_IMM64(__buf, __p, __rw, __w64) \
+do { \
+ (__buf)[0] = KVX_INSN_MAKE_IMM64_SYLLABLE_0(__rw, __w64); \
+ (__buf)[1] = KVX_INSN_MAKE_IMM64_SYLLABLE_1(__w64); \
+ (__buf)[2] = KVX_INSN_MAKE_IMM64_SYLLABLE_2(__p, __w64); \
+} while (0)
+
+#define KVX_INSN_ICALL_SIZE 1
+#define KVX_INSN_ICALL_MASK_0 0x7ffc0000
+#define KVX_INSN_ICALL_OPCODE_0 0xfdc0000
+#define KVX_INSN_ICALL_SYLLABLE_0(__p, __rz) \
+ (KVX_INSN_ICALL_OPCODE_0 | ((__p) << 31) | ((__rz) & 0x3f))
+#define KVX_INSN_ICALL(__buf, __p, __rz) \
+do { \
+ (__buf)[0] = KVX_INSN_ICALL_SYLLABLE_0(__p, __rz); \
+} while (0)
+
+#define KVX_INSN_IGOTO_SIZE 1
+#define KVX_INSN_IGOTO_MASK_0 0x7ffc0000
+#define KVX_INSN_IGOTO_OPCODE_0 0xfd80000
+#define KVX_INSN_IGOTO_SYLLABLE_0(__p, __rz) \
+ (KVX_INSN_IGOTO_OPCODE_0 | ((__p) << 31) | ((__rz) & 0x3f))
+#define KVX_INSN_IGOTO(__buf, __p, __rz) \
+do { \
+ (__buf)[0] = KVX_INSN_IGOTO_SYLLABLE_0(__p, __rz); \
+} while (0)
+
+#define KVX_INSN_CALL_SIZE 1
+#define KVX_INSN_CALL_PCREL27_CHECK(__val) \
+ (((__val) & BITMASK(2)) || check_signed_imm((__val) >> 2, 27))
+#define KVX_INSN_CALL_MASK_0 0x78000000
+#define KVX_INSN_CALL_OPCODE_0 0x18000000
+#define KVX_INSN_CALL_SYLLABLE_0(__p, __pcrel27) \
+ (KVX_INSN_CALL_OPCODE_0 | ((__p) << 31) | (((__pcrel27) >> 2) & 0x7ffffff))
+#define KVX_INSN_CALL(__buf, __p, __pcrel27) \
+do { \
+ (__buf)[0] = KVX_INSN_CALL_SYLLABLE_0(__p, __pcrel27); \
+} while (0)
+
+#define KVX_INSN_GOTO_SIZE 1
+#define KVX_INSN_GOTO_PCREL27_CHECK(__val) \
+ (((__val) & BITMASK(2)) || check_signed_imm((__val) >> 2, 27))
+#define KVX_INSN_GOTO_MASK_0 0x78000000
+#define KVX_INSN_GOTO_OPCODE_0 0x10000000
+#define KVX_INSN_GOTO_SYLLABLE_0(__p, __pcrel27) \
+ (KVX_INSN_GOTO_OPCODE_0 | ((__p) << 31) | (((__pcrel27) >> 2) & 0x7ffffff))
+#define KVX_INSN_GOTO(__buf, __p, __pcrel27) \
+do { \
+ (__buf)[0] = KVX_INSN_GOTO_SYLLABLE_0(__p, __pcrel27); \
+} while (0)
+
+#define KVX_INSN_NOP_SIZE 1
+#define KVX_INSN_NOP_MASK_0 0x7f03f000
+#define KVX_INSN_NOP_OPCODE_0 0x7f03f000
+#define KVX_INSN_NOP_SYLLABLE_0(__p) \
+ (KVX_INSN_NOP_OPCODE_0 | ((__p) << 31))
+#define KVX_INSN_NOP(__buf, __p) \
+do { \
+ (__buf)[0] = KVX_INSN_NOP_SYLLABLE_0(__p); \
+} while (0)
+
+#define KVX_INSN_SET_SIZE 1
+#define KVX_INSN_SET_MASK_0 0x7ffc0000
+#define KVX_INSN_SET_OPCODE_0 0xfc00000
+#define KVX_INSN_SET_SYLLABLE_0(__p, __systemT3, __rz) \
+ (KVX_INSN_SET_OPCODE_0 | ((__p) << 31) | (((__systemT3) & 0x1ff) << 6) | ((__rz) & 0x3f))
+#define KVX_INSN_SET(__buf, __p, __systemT3, __rz) \
+do { \
+ (__buf)[0] = KVX_INSN_SET_SYLLABLE_0(__p, __systemT3, __rz); \
+} while (0)
+
+#endif /* __ASM_KVX_INSNS_DEFS_H_ */
diff --git a/arch/kvx/kernel/break_hook.c b/arch/kvx/kernel/break_hook.c
new file mode 100644
index 000000000000..2c02287c04a6
--- /dev/null
+++ b/arch/kvx/kernel/break_hook.c
@@ -0,0 +1,77 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+
+#include <linux/list.h>
+#include <linux/rculist.h>
+#include <linux/spinlock.h>
+#include <linux/rcupdate.h>
+
+#include <asm/insns.h>
+#include <asm/traps.h>
+#include <asm/processor.h>
+#include <asm/break_hook.h>
+
+static DEFINE_SPINLOCK(debug_hook_lock);
+static LIST_HEAD(user_break_hook);
+static LIST_HEAD(kernel_break_hook);
+
+void kvx_skip_break_insn(struct pt_regs *regs)
+{
+ regs->spc += KVX_BREAK_INSN_SIZE;
+}
+
+int break_hook_handler(uint64_t es, struct pt_regs *regs)
+{
+ int (*fn)(struct break_hook *brk_hook, struct pt_regs *regs) = NULL;
+ struct break_hook *tmp_hook, *hook = NULL;
+ struct list_head *list;
+ unsigned long flags;
+ u32 idx;
+
+ if (trap_sfri(es) != KVX_TRAP_SFRI_SET ||
+ trap_sfrp(es) != KVX_SFR_VSFR0)
+ return BREAK_HOOK_ERROR;
+
+ idx = trap_gprp(es);
+ list = user_mode(regs) ? &user_break_hook : &kernel_break_hook;
+
+ local_irq_save(flags);
+ list_for_each_entry_rcu(tmp_hook, list, node) {
+ if (idx == tmp_hook->id) {
+ hook = tmp_hook;
+ break;
+ }
+ }
+ local_irq_restore(flags);
+
+ if (!hook)
+ return BREAK_HOOK_ERROR;
+
+ fn = hook->handler;
+ return fn(hook, regs);
+}
+
+void break_hook_register(struct break_hook *brk_hook)
+{
+ struct list_head *list;
+
+ if (brk_hook->mode == MODE_USER)
+ list = &user_break_hook;
+ else
+ list = &kernel_break_hook;
+
+ spin_lock(&debug_hook_lock);
+ list_add_rcu(&brk_hook->node, list);
+ spin_unlock(&debug_hook_lock);
+}
+
+void break_hook_unregister(struct break_hook *brk_hook)
+{
+ spin_lock(&debug_hook_lock);
+ list_del_rcu(&brk_hook->node);
+ spin_unlock(&debug_hook_lock);
+ synchronize_rcu();
+}
diff --git a/arch/kvx/kernel/debug.c b/arch/kvx/kernel/debug.c
new file mode 100644
index 000000000000..d4cde403bca9
--- /dev/null
+++ b/arch/kvx/kernel/debug.c
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ */
+#include <linux/list.h>
+#include <linux/rculist.h>
+#include <linux/spinlock.h>
+#include <linux/rcupdate.h>
+
+#include <asm/dame.h>
+#include <asm/debug.h>
+
+static DEFINE_SPINLOCK(debug_hook_lock);
+static LIST_HEAD(user_debug_hook);
+static LIST_HEAD(kernel_debug_hook);
+
+static struct list_head *debug_hook_list(bool user_mode)
+{
+ return user_mode ? &user_debug_hook : &kernel_debug_hook;
+}
+
+static void call_debug_hook(u64 ea, struct pt_regs *regs)
+{
+ int ret;
+ struct debug_hook *hook;
+ struct list_head *list = debug_hook_list(user_mode(regs));
+
+ list_for_each_entry_rcu(hook, list, node) {
+ ret = hook->handler(ea, regs);
+ if (ret == DEBUG_HOOK_HANDLED)
+ return;
+ }
+
+ panic("Entered debug but no requester !");
+}
+
+void debug_hook_register(struct debug_hook *dbg_hook)
+{
+ struct list_head *list = debug_hook_list(dbg_hook->mode == MODE_USER);
+
+ spin_lock(&debug_hook_lock);
+ list_add_rcu(&dbg_hook->node, list);
+ spin_unlock(&debug_hook_lock);
+}
+
+void debug_hook_unregister(struct debug_hook *dbg_hook)
+{
+ spin_lock(&debug_hook_lock);
+ list_del_rcu(&dbg_hook->node);
+ spin_unlock(&debug_hook_lock);
+ synchronize_rcu();
+}
+
+/**
+ * Main debug handler called by the _debug_handler routine in entry.S
+ * This handler will perform the required action
+ */
+void debug_handler(u64 ea, struct pt_regs *regs)
+{
+ trace_hardirqs_off();
+ call_debug_hook(ea, regs);
+ dame_irq_check(regs);
+}
diff --git a/arch/kvx/kernel/hw_breakpoint.c b/arch/kvx/kernel/hw_breakpoint.c
new file mode 100644
index 000000000000..f28ed1a99bf2
--- /dev/null
+++ b/arch/kvx/kernel/hw_breakpoint.c
@@ -0,0 +1,556 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Marius Gligor
+ * Clement Leger
+ */
+
+#include <linux/hw_breakpoint.h>
+#include <linux/percpu.h>
+#include <linux/perf_event.h>
+#include <linux/bitops.h>
+#include <linux/cache.h>
+
+#include <asm/hw_breakpoint.h>
+#include <asm/sfr.h>
+
+#define HW_BREAKPOINT_SIZE 4
+#define HW_BREAKPOINT_RANGE 2
+#define MAX_STORE_LENGTH 32
+#define L1_LINE_MASK ((u64) KVX_DCACHE_LINE_SIZE - 1)
+
+#define ES_AS_DZEROL_CODE 0x3F
+#define ES_AS_MAINT_CODE 0x21
+
+#define WATCHPOINT_STEPPED 1
+#define WATCHPOINT_GDB_HIT 2
+
+#define hw_breakpoint_remap(idx) (KVX_HW_BREAKPOINT_COUNT - 1 - (idx))
+#define hw_watchpoint_remap(idx) (idx)
+
+/* Breakpoint currently in use */
+static DEFINE_PER_CPU(struct perf_event *, hbp_on_reg[KVX_HW_BREAKPOINT_COUNT]);
+
+/* Watchpoint currently in use */
+static DEFINE_PER_CPU(struct perf_event *, hwp_on_reg[KVX_HW_WATCHPOINT_COUNT]);
+
+/* Get and set function of the debug hardware registers */
+#define gen_set_hw_sfr(__name, __sfr) \
+static inline void set_hw_ ## __name(int idx, u64 addr) \
+{ \
+ if (idx == 0) \
+ kvx_sfr_set(__sfr ## 0, addr); \
+ else \
+ kvx_sfr_set(__sfr ## 1, addr); \
+}
+
+#define gen_set_hw_sfr_field(__name, __sfr, __field) \
+static inline void set_hw_ ## __name(int idx, u32 value) \
+{ \
+ if (idx == 0) \
+ kvx_sfr_set_field(__sfr, __field ## 0, value); \
+ else \
+ kvx_sfr_set_field(__sfr, __field ## 1, value); \
+}
+
+#define gen_get_hw_sfr_field(__name, __sfr, __field) \
+static inline u32 get_hw_ ## __name(int idx) \
+{ \
+ if (idx == 0) \
+ return kvx_sfr_field_val(kvx_sfr_get(__sfr), \
+ __sfr, __field ## 0); \
+ return kvx_sfr_field_val(kvx_sfr_get(__sfr), __sfr, \
+ __field ## 1); \
+}
+
+gen_set_hw_sfr_field(bp_owner, DOW, B);
+gen_set_hw_sfr_field(wp_owner, DOW, W);
+gen_get_hw_sfr_field(bp_owner, DO, B);
+gen_get_hw_sfr_field(wp_owner, DO, W);
+gen_set_hw_sfr(bp_addr, DBA);
+gen_set_hw_sfr(wp_addr, DWA);
+gen_set_hw_sfr_field(bp_range, DC, BR);
+gen_set_hw_sfr_field(wp_range, DC, WR);
+gen_set_hw_sfr_field(bp_enable, DC, BE);
+gen_set_hw_sfr_field(wp_enable, DC, WE);
+
+/**
+ * hw_breakpoint_slots() - obtain the maximum available number of hardware
+ * resources for the specified type
+ * @type: Requested hardware watchpoint/breakpoint type
+ * Return: The maximum number of hardware resources for the given type
+ * or 0 if type is invalide
+ */
+int hw_breakpoint_slots(int type)
+{
+ switch (type) {
+ case TYPE_INST:
+ return KVX_HW_BREAKPOINT_COUNT;
+ case TYPE_DATA:
+ return KVX_HW_WATCHPOINT_COUNT;
+ default:
+ pr_warn("unknown slot type: %d\n", type);
+ return 0;
+ }
+}
+
+/**
+ * arch_check_bp_in_kernelspace() - verify if the specified
+ * watchpoint/breakpoint address is inside the kernel
+ * @hw: The arch hardware watchpoint/breakpoint whose address should be checked
+ * Return: The result of the verification
+ */
+int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw)
+{
+ return hw->addr >= PAGE_OFFSET;
+}
+
+/*
+ * compute_hw_watchpoint_range() - compute the watchpoint hardware registers to
+ * cover attr->bp_len bytes beginning from attr->bp_addr
+ * @attr: Provides the address and length for which the hardware registers
+ * should be computed
+ * @hw: Address of the arch_hw_breakpoint where the computed values should be
+ * stored
+ * Observation: Because of the kvx watchpoint range problem, both hardware
+ * watchpoints may be used for index 0
+ */
+static void compute_hw_watchpoint_range(const struct perf_event_attr *attr,
+ struct arch_hw_breakpoint *hw)
+{
+ u64 addr = attr->bp_addr;
+ u32 size = attr->bp_len;
+ u64 begin = (addr >= MAX_STORE_LENGTH - 1) ?
+ (addr - (MAX_STORE_LENGTH - 1)) : 0;
+ u64 end = addr + size - 1;
+ u64 addr_l1_aligned = addr & ~L1_LINE_MASK;
+ u64 end_l1_aligned = end & ~L1_LINE_MASK;
+
+ /* The maximum range of a store instruction is 32 bytes (store octuple).
+ * The stores may be unaligned. The dzerol instruction fills the
+ * specified line cache with 0, so its range is the L1 cache line size
+ * (64 bytes for Coolidge). So, the range that should be covered is:
+ * MIN(MAX(addr - 31, 0), (addr & ~L1_LINE_MASK)) to (addr + len -1)
+ * We can have MAX(addr - 31, 0) less (addr & ~L1_LINE_MASK) only for an
+ * unaligned access. In this case, we have a store that modifies data in
+ * 2 L1 cache lines. If the addresses of the 2 consecutive L1 cache
+ * lines have many bits different (e.g. 0x10000000 and 0xfffffc0), the
+ * watchpoint range will be very big (29 bits (512 MB) in the example)
+ * and each time a byte in this range will change, the watchpoint will
+ * be triggered, so, in fact, almost all stores will trigger the
+ * watchpoint and the execution will be very very slow. To avoid this,
+ * in this case, we use 2 hardware watchpoints, one for each L1 cache
+ * line, each covering only a few bytes in their cache line.
+ * A similar case, requiring 2 hardware watchpoints, happens when
+ * (addr + len -1) is in next cache line
+ */
+ if (begin < addr_l1_aligned) {
+ hw->wp.hw_addr[0] = begin;
+ hw->wp.hw_range[0] = fls64(begin ^ (addr_l1_aligned - 1));
+
+ hw->wp.use_wp1 = 1;
+ hw->wp.hw_addr[1] = addr_l1_aligned;
+ hw->wp.hw_range[1] = fls64(addr_l1_aligned ^ end);
+ } else if (addr_l1_aligned != end_l1_aligned) {
+ hw->wp.hw_addr[0] = addr_l1_aligned;
+ hw->wp.hw_range[0] =
+ fls64(addr_l1_aligned ^ (end_l1_aligned - 1));
+
+ hw->wp.use_wp1 = 1;
+ hw->wp.hw_addr[1] = end_l1_aligned;
+ hw->wp.hw_range[1] = fls64(end_l1_aligned ^ end);
+ } else {
+ hw->wp.use_wp1 = 0;
+ hw->wp.hw_addr[0] = addr_l1_aligned;
+ hw->wp.hw_range[0] = fls64(addr_l1_aligned ^ end);
+ }
+
+ if (!hw->wp.use_wp1) {
+ hw->wp.hw_addr[1] = 0;
+ hw->wp.hw_range[1] = 0;
+ }
+ hw->wp.hit_info = 0;
+}
+
+/*
+ * hw_breakpoint_arch_parse() - Construct an arch_hw_breakpoint from
+ * a perf_event
+ * @bp: The source perf event
+ * @attr: Attributes of the perf_event
+ * @hw: Address of the arch_hw_breakpoint to be constructed
+ * Return: 0 for success, or negative error code
+ */
+int hw_breakpoint_arch_parse(struct perf_event *bp,
+ const struct perf_event_attr *attr,
+ struct arch_hw_breakpoint *hw)
+{
+ /* Type */
+ if (attr->bp_type == HW_BREAKPOINT_X) {
+ if (!attr->disabled) {
+ if (!IS_ALIGNED(attr->bp_addr, HW_BREAKPOINT_SIZE) ||
+ attr->bp_len != HW_BREAKPOINT_SIZE)
+ return -EINVAL;
+ }
+
+ hw->type = KVX_HW_BREAKPOINT_TYPE;
+ hw->bp.hw_addr = attr->bp_addr;
+ hw->bp.hw_range = HW_BREAKPOINT_RANGE;
+ } else {
+ if (attr->bp_type == HW_BREAKPOINT_W) {
+ hw->type = KVX_HW_WATCHPOINT_TYPE;
+ if (!attr->disabled)
+ compute_hw_watchpoint_range(attr, hw);
+ } else
+ return -EINVAL;
+ }
+
+ hw->addr = attr->bp_addr;
+ hw->len = attr->bp_len;
+
+ return 0;
+}
+
+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
+ unsigned long val, void *data)
+{
+ return NOTIFY_DONE;
+}
+
+static int alloc_slot(struct perf_event **slot, size_t n,
+ struct perf_event *bp)
+{
+ int idx;
+
+ for (idx = 0; idx < n; idx++) {
+ if (!slot[idx]) {
+ slot[idx] = bp;
+ return idx;
+ }
+ }
+ return -EBUSY;
+}
+
+static void enable_hw_breakpoint(int idx, int enable, struct perf_event *bp)
+{
+ struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+
+ idx = hw_breakpoint_remap(idx);
+ if (enable) {
+ set_hw_bp_addr(idx, info->bp.hw_addr);
+ set_hw_bp_range(idx, info->bp.hw_range);
+ }
+
+ set_hw_bp_enable(idx, enable);
+}
+
+/*
+ * ptrace_request_hw_breakpoint() - tries to obtain the ownership for the
+ * requested hardware breakpoint
+ * @idx: The index of the requested hardware breakpoint
+ * Return: 0 for success, or negative error code
+ */
+int ptrace_request_hw_breakpoint(int idx)
+{
+ int linux_pl, pl;
+
+ if (idx < 0 || idx >= KVX_HW_BREAKPOINT_COUNT)
+ return -EINVAL;
+
+ linux_pl = kvx_sfr_field_val(kvx_sfr_get(PS), PS, PL);
+
+ /* Remap the indexes: request first the last hw breakpoint */
+ idx = hw_breakpoint_remap(idx);
+ pl = get_hw_bp_owner(idx);
+ if (pl < linux_pl) {
+ set_hw_bp_owner(idx, 0);
+ pl = get_hw_bp_owner(idx);
+ }
+
+ return (pl == linux_pl) ? 0 : -EPERM;
+}
+
+static int reserve_one_hw_watchpoint(int idx)
+{
+ int linux_pl = kvx_sfr_field_val(kvx_sfr_get(PS), PS, PL);
+ int pl = get_hw_wp_owner(idx);
+
+ if (pl < linux_pl) {
+ set_hw_wp_owner(idx, 0);
+ pl = get_hw_wp_owner(idx);
+ }
+
+ return (pl == linux_pl) ? 0 : -EPERM;
+}
+
+/*
+ * ptrace_request_hw_watchpoint() - tries to obtain the ownership for the
+ * requested hardware watchpoint
+ * @idx: The index of the requested hardware watchpoint
+ * Return: 0 for success, or negative error code
+ *
+ * Observation: Because of the kvx watchpoint range limitation, both hardware
+ * watchpoints are used for index 0 for Coolidge V1
+ */
+int ptrace_request_hw_watchpoint(int idx)
+{
+ int res;
+
+ if (idx < 0 || idx >= KVX_HW_WATCHPOINT_COUNT)
+ return -EINVAL;
+
+ idx = hw_watchpoint_remap(idx);
+ res = reserve_one_hw_watchpoint(idx);
+ if (res)
+ return res;
+
+ /* Request the both watchpoints for Coolidge V1. W0 is ours, now W1 */
+ return reserve_one_hw_watchpoint(idx + 1);
+}
+
+static void enable_one_watchpoint(int idx, int sub_idx, int enable,
+ struct arch_hw_breakpoint *info)
+{
+ if (enable) {
+ set_hw_wp_addr(idx, info->wp.hw_addr[sub_idx]);
+ set_hw_wp_range(idx, info->wp.hw_range[sub_idx]);
+ }
+
+ set_hw_wp_enable(idx, enable);
+}
+
+static void enable_hw_watchpoint(int idx, int enable, struct perf_event *bp)
+{
+ struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+
+ if (idx < 0 || idx >= KVX_HW_WATCHPOINT_COUNT)
+ return;
+
+ idx = hw_watchpoint_remap(idx);
+ enable_one_watchpoint(idx, 0, enable, info);
+ if (info->wp.use_wp1)
+ enable_one_watchpoint(idx + 1, 1, enable, info);
+}
+
+static void get_hw_pt_list(int type, struct perf_event ***p, int *count)
+{
+ if (type == KVX_HW_BREAKPOINT_TYPE) {
+ /* Breakpoint */
+ *p = this_cpu_ptr(hbp_on_reg);
+ *count = KVX_HW_BREAKPOINT_COUNT;
+ } else {
+ /* Watchpoint */
+ *p = this_cpu_ptr(hwp_on_reg);
+ *count = KVX_HW_WATCHPOINT_COUNT;
+ }
+}
+
+static void enable_hw_pt(int idx, int enable, struct perf_event *bp)
+{
+ int type = counter_arch_bp(bp)->type;
+
+ if (type == KVX_HW_BREAKPOINT_TYPE)
+ enable_hw_breakpoint(idx, enable, bp);
+ else
+ enable_hw_watchpoint(idx, enable, bp);
+}
+
+int arch_install_hw_breakpoint(struct perf_event *bp)
+{
+ struct perf_event **p;
+ int idx, count;
+
+ get_hw_pt_list(counter_arch_bp(bp)->type, &p, &count);
+ idx = alloc_slot(p, count, bp);
+ if (idx < 0)
+ return idx;
+
+ enable_hw_pt(idx, 1, bp);
+
+ return 0;
+}
+
+static int free_slot(struct perf_event **slot, size_t n,
+ struct perf_event *bp)
+{
+ int idx;
+
+ for (idx = 0; idx < n; idx++) {
+ if (slot[idx] == bp) {
+ slot[idx] = NULL;
+ return idx;
+ }
+ }
+ return -EBUSY;
+}
+
+void arch_uninstall_hw_breakpoint(struct perf_event *bp)
+{
+ struct perf_event **p;
+ int idx, count;
+
+ get_hw_pt_list(counter_arch_bp(bp)->type, &p, &count);
+ idx = free_slot(p, count, bp);
+ if (idx < 0)
+ return;
+
+ enable_hw_pt(idx, 0, bp);
+}
+
+void hw_breakpoint_pmu_read(struct perf_event *bp)
+{
+}
+
+void flush_ptrace_hw_breakpoint(struct task_struct *tsk)
+{
+ int i;
+ struct thread_struct *t = &tsk->thread;
+
+ for (i = 0; i < KVX_HW_BREAKPOINT_COUNT; i++) {
+ if (t->debug.ptrace_hbp[i]) {
+ unregister_hw_breakpoint(t->debug.ptrace_hbp[i]);
+ t->debug.ptrace_hbp[i] = NULL;
+ }
+ }
+ for (i = 0; i < KVX_HW_WATCHPOINT_COUNT; i++) {
+ if (t->debug.ptrace_hwp[i]) {
+ unregister_hw_breakpoint(t->debug.ptrace_hwp[i]);
+ t->debug.ptrace_hwp[i] = NULL;
+ }
+ }
+}
+
+/*
+ * Set ptrace breakpoint pointers to zero for this task.
+ * This is required in order to prevent child processes from unregistering
+ * breakpoints held by their parent.
+ */
+void clear_ptrace_hw_breakpoint(struct task_struct *tsk)
+{
+ struct debug_info *d = &tsk->thread.debug;
+
+ memset(d->ptrace_hbp, 0, sizeof(d->ptrace_hbp));
+ memset(d->ptrace_hwp, 0, sizeof(d->ptrace_hwp));
+}
+
+/**
+ * check_hw_breakpoint() - called from debug_handler for each hardware
+ * breakpoint exception
+ * @regs: Pointer to registers saved when trapping
+ *
+ * This function informs the debugger if a hardware breakpoint hit
+ */
+void check_hw_breakpoint(struct pt_regs *regs)
+{
+ int i;
+ struct perf_event **bp = this_cpu_ptr(hbp_on_reg);
+
+ for (i = 0; i < KVX_HW_BREAKPOINT_COUNT; i++) {
+ if (bp[i] && !bp[i]->attr.disabled &&
+ regs->spc == bp[i]->attr.bp_addr)
+ perf_bp_event(bp[i], regs);
+ }
+}
+
+static void watchpoint_triggered(struct perf_event *wp, struct pt_regs *regs,
+ int idx, u64 ea)
+{
+ struct arch_hw_breakpoint *info = counter_arch_bp(wp);
+ int as = kvx_sfr_field_val(regs->es, ES, AS);
+
+ /* Disable the watchpoint in order to be able to stepi */
+ info->wp.hit_info = WATCHPOINT_STEPPED;
+ enable_hw_watchpoint(idx, 0, wp);
+
+ if (as == ES_AS_MAINT_CODE)
+ return;
+
+ if (as == ES_AS_DZEROL_CODE) {
+ as = KVX_DCACHE_LINE_SIZE;
+ ea &= ~L1_LINE_MASK;
+ }
+
+ /* Check if the user watchpoint range was written */
+ if (ea < info->addr + info->len && ea + as >= info->addr)
+ info->wp.hit_info |= WATCHPOINT_GDB_HIT;
+}
+
+/**
+ * check_hw_watchpoint() - called from debug_handler for each hardware
+ * watchpoint exception
+ * @regs: Pointer to registers saved when trapping
+ * @ea: Exception Address register
+ * Return: 1 if this exception was caused by a registered user watchpoint,
+ * otherwise returns 0
+ */
+int check_hw_watchpoint(struct pt_regs *regs, u64 ea)
+{
+ struct perf_event **wp;
+ struct arch_hw_breakpoint *info;
+ u64 mask;
+ int i, ret = 0;
+
+ wp = this_cpu_ptr(hwp_on_reg);
+ for (i = 0; i < KVX_HW_WATCHPOINT_COUNT; i++) {
+ if (!wp[i] || wp[i]->attr.disabled)
+ continue;
+
+ info = counter_arch_bp(wp[i]);
+ mask = ~(BIT_ULL(info->wp.hw_range[0]) - 1);
+ if ((info->wp.hw_addr[0] & mask) == (ea & mask)) {
+ ret = 1;
+ watchpoint_triggered(wp[i], regs, i, ea);
+ }
+
+ if (info->wp.use_wp1) {
+ mask = ~(BIT_ULL(info->wp.hw_range[1]) - 1);
+ if ((info->wp.hw_addr[1] & mask) == (ea & mask)) {
+ ret = 1;
+ watchpoint_triggered(wp[i], regs, i, ea);
+ }
+ }
+ }
+
+ return ret;
+}
+
+/**
+ * check_hw_watchpoint_stepped() - called from debug_handler for each
+ * stepi exception
+ * @regs: Pointer to registers saved when trapping
+ * Return: 1 if this stepi event was caused by stepping a watchpoint,
+ * otherwise returns 0
+ *
+ * This function verifies if this stepi event was caused by stepping a
+ * watchpoint, restores the watchpoints disabled before stepping and informs
+ * the debugger about the wathcpoint hit
+ */
+int check_hw_watchpoint_stepped(struct pt_regs *regs)
+{
+ struct perf_event **wp;
+ struct arch_hw_breakpoint *info;
+ int i, ret = 0;
+
+ wp = this_cpu_ptr(hwp_on_reg);
+ for (i = 0; i < KVX_HW_WATCHPOINT_COUNT; i++) {
+ if (!wp[i] || wp[i]->attr.disabled)
+ continue;
+
+ info = counter_arch_bp(wp[i]);
+ if (info->wp.hit_info & WATCHPOINT_STEPPED) {
+ ret = 1;
+ enable_hw_watchpoint(i, 1, wp[i]);
+ }
+
+ /* Inform the debugger about the watchpoint only if the
+ * requested watched range was written to
+ */
+ if (info->wp.hit_info & WATCHPOINT_GDB_HIT) {
+ if (user_mode(regs))
+ perf_bp_event(wp[i], regs);
+ }
+
+ info->wp.hit_info = 0;
+ }
+
+ return ret;
+}
diff --git a/arch/kvx/kernel/insns.c b/arch/kvx/kernel/insns.c
new file mode 100644
index 000000000000..d25d8ad2a48c
--- /dev/null
+++ b/arch/kvx/kernel/insns.c
@@ -0,0 +1,146 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Yann Sionneau
+ * Marius Gligor
+ * Guillaume Thouvenin
+ */
+
+#include <linux/types.h>
+#include <linux/string.h>
+#include <linux/atomic.h>
+#include <linux/cpumask.h>
+#include <linux/uaccess.h>
+#include <linux/stop_machine.h>
+
+#include <asm/cacheflush.h>
+#include <asm/insns_defs.h>
+#include <asm/fixmap.h>
+
+struct insns_patch {
+ atomic_t cpu_count;
+ u32 *addr;
+ u32 *insns;
+ unsigned long insns_len;
+};
+
+static void *insn_patch_map(void *addr)
+{
+ unsigned long uintaddr = (uintptr_t) addr;
+ bool module = !core_kernel_text(uintaddr);
+ struct page *page;
+
+ if (module && IS_ENABLED(CONFIG_STRICT_MODULE_RWX))
+ page = vmalloc_to_page(addr);
+ else if (!module && IS_ENABLED(CONFIG_STRICT_KERNEL_RWX))
+ page = phys_to_page(__pa_symbol(addr));
+ else
+ return addr;
+
+ BUG_ON(!page);
+ return (void *)set_fixmap_offset(FIX_TEXT_PATCH, page_to_phys(page) +
+ (uintaddr & ~PAGE_MASK));
+}
+
+static void insn_patch_unmap(void)
+{
+ clear_fixmap(FIX_TEXT_PATCH);
+}
+
+int kvx_insns_write_nostop(u32 *insns, u8 insns_len, u32 *insn_addr)
+{
+ unsigned long current_insn_addr = (unsigned long) insn_addr;
+ unsigned long len_remain = insns_len;
+ unsigned long next_insn_page, patch_len;
+ void *map_patch_addr;
+ int ret = 0;
+
+ do {
+ /* Compute next upper page boundary */
+ next_insn_page = (current_insn_addr + PAGE_SIZE) & PAGE_MASK;
+
+ patch_len = min(next_insn_page - current_insn_addr, len_remain);
+ len_remain -= patch_len;
+
+ /* Map & patch insns */
+ map_patch_addr = insn_patch_map((void *) current_insn_addr);
+ ret = copy_to_kernel_nofault(map_patch_addr, insns, patch_len);
+ if (ret)
+ break;
+
+ insns = (void *) insns + patch_len;
+ current_insn_addr = next_insn_page;
+
+ } while (len_remain);
+
+ insn_patch_unmap();
+
+ /*
+ * Flush & invalidate L2 + L1 icache to reload instructions from memory
+ * L2 wbinval is necessary because we write through DEVICE cache policy
+ * mapping which is uncached therefore L2 is bypassed
+ */
+ wbinval_icache_range(virt_to_phys(insn_addr), insns_len);
+
+ return ret;
+}
+
+static int patch_insns_percpu(void *data)
+{
+ struct insns_patch *ip = data;
+ unsigned long insn_addr = (unsigned long) ip->addr;
+ int ret;
+
+ if (atomic_inc_return(&ip->cpu_count) == 1) {
+ ret = kvx_insns_write_nostop(ip->insns, ip->insns_len,
+ ip->addr);
+ /* Additionnal up to release other processors */
+ atomic_inc(&ip->cpu_count);
+
+ return ret;
+ }
+
+ /* Wait for first processor to update instructions */
+ while (atomic_read(&ip->cpu_count) <= num_online_cpus())
+ cpu_relax();
+
+ /* Simply invalidate L1 I-cache to reload from L2 or memory */
+ l1_inval_icache_range(insn_addr, insn_addr + ip->insns_len);
+ return 0;
+}
+
+/**
+ * kvx_insns_write() Patch instructions at a specified address
+ * @insns: Instructions to be written at @addr
+ * @insns_len: Size of instructions to patch
+ * @addr: Address of the first instruction to patch
+ */
+int kvx_insns_write(u32 *insns, unsigned long insns_len, u32 *addr)
+{
+ struct insns_patch ip = {
+ .cpu_count = ATOMIC_INIT(0),
+ .addr = addr,
+ .insns = insns,
+ .insns_len = insns_len
+ };
+
+ if (!insns_len)
+ return -EINVAL;
+
+ if (!IS_ALIGNED((unsigned long) addr, KVX_INSN_SYLLABLE_WIDTH))
+ return -EINVAL;
+
+ /*
+ * Function name is a "bit" misleading. while being named
+ * stop_machine, this function does not stop the machine per se
+ * but execute the provided function on all CPU in a safe state.
+ */
+ return stop_machine(patch_insns_percpu, &ip, cpu_online_mask);
+}
+
+int kvx_insns_read(u32 *insns, unsigned long insns_len, u32 *addr)
+{
+ l1_inval_dcache_range((unsigned long)addr, insns_len);
+ return copy_from_kernel_nofault(insns, addr, insns_len);
+}
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 24/25] kvx: Add support for CPU Perf Monitors
2023-01-03 16:43 ` Yann Sionneau
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Will Deacon, Mark Rutland, Rob Herring,
Krzysztof Kozlowski, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Alexander Shishkin, Jiri Olsa,
Namhyung Kim, linux-arm-kernel, devicetree, linux-kernel,
linux-perf-users, Clement Leger, Jules Maselbas, Julian Vetter
Each kvx core includes several Perf Monitors.
This commit adds the driver that handles those core PM.
CC: Will Deacon <will@kernel.org>
CC: Mark Rutland <mark.rutland@arm.com>
CC: Rob Herring <robh+dt@kernel.org>
CC: Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Ingo Molnar <mingo@redhat.com>
CC: Arnaldo Carvalho de Melo <acme@kernel.org>
CC: Alexander Shishkin <alexander.shishkin@linux.intel.com>
CC: Jiri Olsa <jolsa@kernel.org>
CC: Namhyung Kim <namhyung@kernel.org>
CC: linux-arm-kernel@lists.infradead.org
CC: devicetree@vger.kernel.org
CC: linux-kernel@vger.kernel.org
CC: linux-perf-users@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
.../devicetree/bindings/perf/kalray-pm.txt | 21 +
arch/kvx/include/asm/perf_event.h | 90 +++
arch/kvx/kernel/perf_event.c | 609 ++++++++++++++++++
3 files changed, 720 insertions(+)
create mode 100644 Documentation/devicetree/bindings/perf/kalray-pm.txt
create mode 100644 arch/kvx/include/asm/perf_event.h
create mode 100644 arch/kvx/kernel/perf_event.c
diff --git a/Documentation/devicetree/bindings/perf/kalray-pm.txt b/Documentation/devicetree/bindings/perf/kalray-pm.txt
new file mode 100644
index 000000000000..9ce00d703941
--- /dev/null
+++ b/Documentation/devicetree/bindings/perf/kalray-pm.txt
@@ -0,0 +1,21 @@
+* Kalray kvx Performance Monitors
+
+KVX core has several Performance Monitors for counting cpu and cache events.
+The KVX PM representation in the device tree should be done as under:
+
+Required properties:
+
+- compatible :
+ "kalray,kvx-core-pm"
+
+- interrupts : The interrupt number for kvx PM is 3.
+- interrupt-parent : The kvx core interrupt controller.
+- kalray,pm-num : Number of Performance Monitors the kvx core has.
+
+Example:
+core_pm {
+ compatible = "kalray,kvx-core-pm";
+ interrupts = <3>;
+ interrupt-parent = <&core_intc>;
+ kalray,pm-num = <4>;
+}
diff --git a/arch/kvx/include/asm/perf_event.h b/arch/kvx/include/asm/perf_event.h
new file mode 100644
index 000000000000..bb0147dfdf47
--- /dev/null
+++ b/arch/kvx/include/asm/perf_event.h
@@ -0,0 +1,90 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Yann Sionneau
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_PERF_EVENT_H
+#define _ASM_KVX_PERF_EVENT_H
+
+#include <linux/perf_event.h>
+
+/**
+ * struct cpu_hw_events - per-cpu structure to describe PM resource usage
+ * @n_events: number of events currently existing
+ * @events: events[i] is the event using PMi. NULL if PMi is not used.
+ */
+struct cpu_hw_events {
+ unsigned int n_events;
+ struct perf_event **events;
+};
+
+enum kvx_pmc_ie {
+ PMC_IE_DISABLED,
+ PMC_IE_ENABLED
+};
+
+enum kvx_pm_idx {
+ KVX_PM_1,
+ KVX_PM_2,
+ KVX_PM_3
+};
+
+enum kvx_pm_event_code {
+ KVX_PM_PCC,
+ KVX_PM_ICC,
+ KVX_PM_EBE,
+ KVX_PM_ENIE,
+ KVX_PM_ENSE,
+ KVX_PM_ICHE,
+ KVX_PM_ICME,
+ KVX_PM_ICMABE,
+ KVX_PM_MNGIC,
+ KVX_PM_MIMHE,
+ KVX_PM_MIMME,
+ KVX_PM_IATSC,
+ KVX_PM_FE,
+ KVX_PM_PBSC,
+ KVX_PM_PNVC,
+ KVX_PM_PSC,
+ KVX_PM_TADBE,
+ KVX_PM_TABE,
+ KVX_PM_TBE,
+ KVX_PM_MDMHE,
+ KVX_PM_MDMME,
+ KVX_PM_DATSC,
+ KVX_PM_DCLHE,
+ KVX_PM_DCHE,
+ KVX_PM_DCLME,
+ KVX_PM_DCME,
+ KVX_PM_DARSC,
+ KVX_PM_LDSC,
+ KVX_PM_DCNGC,
+ KVX_PM_DMAE,
+ KVX_PM_LCFSC,
+ KVX_PM_MNGDC,
+ KVX_PM_MACC,
+ KVX_PM_TACC,
+ KVX_PM_IWC,
+ KVX_PM_WISC,
+ KVX_PM_SISC,
+ KVX_PM_DDSC,
+ KVX_PM_SC,
+ KVX_PM_ELE,
+ KVX_PM_ELNBE,
+ KVX_PM_ELUE,
+ KVX_PM_ELUNBE,
+ KVX_PM_ESE,
+ KVX_PM_ESNBE,
+ KVX_PM_EAE,
+ KVX_PM_CIRE,
+ KVX_PM_CIE,
+ KVX_PM_SE,
+ KVX_PM_RE,
+ KVX_PM_FSC,
+ KVX_PM_MAX,
+ KVX_PM_UNSUPPORTED = KVX_PM_MAX,
+};
+
+#endif /* _ASM_KVX_PERF_EVENT_H */
diff --git a/arch/kvx/kernel/perf_event.c b/arch/kvx/kernel/perf_event.c
new file mode 100644
index 000000000000..bfc547e78aba
--- /dev/null
+++ b/arch/kvx/kernel/perf_event.c
@@ -0,0 +1,609 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Yann Sionneau
+ * Clement Leger
+ */
+
+#include <linux/of.h>
+#include <linux/of_irq.h>
+#include <linux/kernel.h>
+#include <linux/perf_event.h>
+#include <linux/errno.h>
+#include <linux/platform_device.h>
+#include <asm/perf_event.h>
+
+static unsigned int pm_num;
+static unsigned int kvx_pm_irq;
+static DEFINE_PER_CPU(struct cpu_hw_events, cpu_hw_events);
+
+static const enum kvx_pm_event_code kvx_hw_event_map[] = {
+ [PERF_COUNT_HW_CPU_CYCLES] = KVX_PM_PCC,
+ [PERF_COUNT_HW_INSTRUCTIONS] = KVX_PM_ENIE,
+ [PERF_COUNT_HW_CACHE_REFERENCES] = KVX_PM_UNSUPPORTED,
+ [PERF_COUNT_HW_CACHE_MISSES] = KVX_PM_UNSUPPORTED,
+ [PERF_COUNT_HW_BRANCH_INSTRUCTIONS] = KVX_PM_TABE,
+ [PERF_COUNT_HW_BRANCH_MISSES] = KVX_PM_UNSUPPORTED,
+ [PERF_COUNT_HW_BUS_CYCLES] = KVX_PM_PCC,
+ [PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] = KVX_PM_PSC,
+ [PERF_COUNT_HW_STALLED_CYCLES_BACKEND] = KVX_PM_UNSUPPORTED,
+ [PERF_COUNT_HW_REF_CPU_CYCLES] = KVX_PM_UNSUPPORTED,
+};
+
+#define C(_x) PERF_COUNT_HW_CACHE_##_x
+
+static const enum kvx_pm_event_code
+ kvx_cache_map[C(MAX)][C(OP_MAX)][C(RESULT_MAX)] = {
+[C(L1D)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_DCLME,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(L1I)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_ICME,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(LL)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(DTLB)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(ITLB)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_MIMME,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(BPU)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(NODE)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+};
+
+static u64 read_counter(struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ unsigned int pm = hwc->idx;
+
+ if (pm > pm_num) {
+ WARN_ONCE(1, "This PM (%u) does not exist!\n", pm);
+ return 0;
+ }
+ return kvx_sfr_iget(KVX_SFR_PM1 + pm);
+}
+
+static void kvx_pmu_read(struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ u64 delta, prev_raw_count, new_raw_count;
+
+ do {
+ prev_raw_count = local64_read(&hwc->prev_count);
+ new_raw_count = read_counter(event);
+ } while (local64_cmpxchg(&hwc->prev_count, prev_raw_count,
+ new_raw_count) != prev_raw_count);
+ /*
+ * delta is the value to update the counter we maintain in the kernel.
+ */
+ delta = (new_raw_count - prev_raw_count);
+ local64_add(delta, &event->count);
+}
+
+static void kvx_set_pmc_ie(unsigned int pm_num, enum kvx_pmc_ie ievalue)
+{
+ u64 shifted_value = ((u64)ievalue << KVX_SFR_PMC_PM1IE_SHIFT)
+ & KVX_SFR_PMC_PM1IE_MASK;
+ u64 clr_mask = KVX_SFR_PMC_PM1IE_MASK << pm_num;
+ u64 set_mask = shifted_value << pm_num;
+
+ kvx_sfr_set_mask(PMC, clr_mask, set_mask);
+}
+
+static void kvx_set_pmc(unsigned int pm_num, enum kvx_pm_event_code pmc_value)
+{
+ u64 pm_shift = (pm_num + 1) * KVX_SFR_PMC_PM1C_SHIFT;
+ u64 clr_mask = KVX_SFR_PMC_PM0C_MASK << pm_shift;
+ u64 set_mask = pmc_value << pm_shift;
+
+ kvx_sfr_set_mask(PMC, clr_mask, set_mask);
+}
+
+static void give_pm_to_user(unsigned int pm)
+{
+ int pl_value = 1 << (KVX_SFR_MOW_PM0_SHIFT
+ + KVX_SFR_MOW_PM0_WIDTH * (pm + 1));
+ int pl_clr_mask = 3 << (KVX_SFR_MOW_PM0_SHIFT
+ + KVX_SFR_MOW_PM0_WIDTH * (pm + 1));
+ kvx_sfr_set_mask(MOW, pl_clr_mask, pl_value);
+}
+
+static void get_pm_back_to_kernel(unsigned int pm)
+{
+ int pl_value = 0;
+ int pl_clr_mask = 3 << (KVX_SFR_MOW_PM0_SHIFT
+ + KVX_SFR_MOW_PM0_WIDTH * (pm + 1));
+ kvx_sfr_set_mask(MOW, pl_clr_mask, pl_value);
+}
+
+static void kvx_set_pm(enum kvx_pm_idx pm, u64 value)
+{
+ switch (pm) {
+ case KVX_PM_1:
+ kvx_sfr_set(PM1, value);
+ break;
+ case KVX_PM_2:
+ kvx_sfr_set(PM2, value);
+ break;
+ case KVX_PM_3:
+ kvx_sfr_set(PM3, value);
+ break;
+ default:
+ WARN_ONCE(1, "This PM (%u) does not exist!\n", pm);
+ }
+}
+
+static void kvx_stop_sampling_event(unsigned int pm)
+{
+ kvx_set_pmc_ie(pm, PMC_IE_DISABLED);
+}
+
+static u64 kvx_start_sampling_event(struct perf_event *event, unsigned int pm)
+{
+ u64 start_value;
+
+ if (event->attr.freq) {
+ pr_err_once("kvx_pm: Frequency sampling is not supported\n");
+ return 0;
+ }
+
+ /* PM counter will overflow after "sample_period" ticks */
+ start_value = (u64)(-event->attr.sample_period);
+
+ kvx_set_pmc(pm, KVX_PM_SE);
+ kvx_set_pm(pm, start_value);
+ kvx_set_pmc_ie(pm, PMC_IE_ENABLED);
+ return start_value;
+}
+
+static void kvx_pmu_start(struct perf_event *event, int flags)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct perf_event_attr *attr = &event->attr;
+ u64 pm_config = hwc->config;
+ unsigned int pm = hwc->idx;
+ u64 start_value = 0;
+
+ if (WARN_ON_ONCE(!(event->hw.state & PERF_HES_STOPPED)))
+ return;
+
+ if (flags & PERF_EF_RELOAD)
+ WARN_ON_ONCE(!(event->hw.state & PERF_HES_UPTODATE));
+
+ hwc->state = 0;
+ perf_event_update_userpage(event);
+
+ if (is_sampling_event(event))
+ start_value = kvx_start_sampling_event(event, pm);
+
+ local64_set(&hwc->prev_count, start_value);
+ if (attr->exclude_kernel)
+ give_pm_to_user(pm);
+ if (!is_sampling_event(event))
+ kvx_set_pmc(pm, KVX_PM_RE);
+ kvx_set_pmc(pm, pm_config);
+}
+
+static void kvx_pmu_stop(struct perf_event *event, int flags)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct perf_event_attr *attr = &event->attr;
+ unsigned int pm = hwc->idx;
+
+ if (is_sampling_event(event))
+ kvx_stop_sampling_event(pm);
+
+ kvx_set_pmc(pm, KVX_PM_SE);
+ if (attr->exclude_kernel)
+ get_pm_back_to_kernel(pm);
+
+ WARN_ON_ONCE(hwc->state & PERF_HES_STOPPED);
+ hwc->state |= PERF_HES_STOPPED;
+
+ if ((flags & PERF_EF_UPDATE) && !(hwc->state & PERF_HES_UPTODATE)) {
+ kvx_pmu_read(event);
+ hwc->state |= PERF_HES_UPTODATE;
+ }
+}
+
+static void kvx_pmu_del(struct perf_event *event, int flags)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct cpu_hw_events *cpuc = &get_cpu_var(cpu_hw_events);
+
+ cpuc->events[hwc->idx] = NULL;
+ cpuc->n_events--;
+ put_cpu_var(cpu_hw_events);
+ kvx_pmu_stop(event, PERF_EF_UPDATE);
+ perf_event_update_userpage(event);
+}
+
+static int kvx_pmu_add(struct perf_event *event, int flags)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct cpu_hw_events *cpuc = &get_cpu_var(cpu_hw_events);
+ unsigned int i, idx = -1;
+
+ if (cpuc->n_events >= pm_num) {
+ put_cpu_var(cpu_hw_events);
+ return -ENOSPC;
+ }
+
+ for (i = 0; i < pm_num; i++)
+ if (cpuc->events[i] == NULL)
+ idx = i;
+
+ BUG_ON(idx == -1);
+
+ hwc->idx = idx;
+ cpuc->events[idx] = event;
+ cpuc->n_events++;
+ put_cpu_var(cpu_hw_events);
+ hwc->state = PERF_HES_UPTODATE | PERF_HES_STOPPED;
+
+ if (flags & PERF_EF_START)
+ kvx_pmu_start(event, PERF_EF_RELOAD);
+
+ return 0;
+}
+
+static enum kvx_pm_event_code kvx_pmu_cache_event(u64 config)
+{
+ unsigned int type, op, result;
+ enum kvx_pm_event_code code;
+
+ type = (config >> 0) & 0xff;
+ op = (config >> 8) & 0xff;
+ result = (config >> 16) & 0xff;
+ if (type >= PERF_COUNT_HW_CACHE_MAX ||
+ op >= PERF_COUNT_HW_CACHE_OP_MAX ||
+ result >= PERF_COUNT_HW_CACHE_RESULT_MAX)
+ return KVX_PM_UNSUPPORTED;
+
+ code = kvx_cache_map[type][op][result];
+
+ return code;
+}
+
+static int kvx_pm_starting_cpu(unsigned int cpu)
+{
+ struct cpu_hw_events *cpuc = &per_cpu(cpu_hw_events, cpu);
+
+ cpuc->events = kmalloc_array(pm_num, sizeof(struct perf_event *),
+ GFP_ATOMIC);
+ if (!cpuc->events)
+ return -ENOMEM;
+
+ memset(cpuc->events, 0, pm_num * sizeof(struct perf_event *));
+
+ enable_percpu_irq(kvx_pm_irq, IRQ_TYPE_NONE);
+ return 0;
+}
+
+static int kvx_pm_dying_cpu(unsigned int cpu)
+{
+ struct cpu_hw_events *cpuc = &get_cpu_var(cpu_hw_events);
+
+ disable_percpu_irq(kvx_pm_irq);
+ kfree(cpuc->events);
+ put_cpu_var(cpu_hw_events);
+ return 0;
+}
+
+static enum kvx_pm_event_code kvx_pmu_raw_events(u64 config)
+{
+ if (config >= KVX_PM_MAX)
+ return KVX_PM_UNSUPPORTED;
+
+ if (config == KVX_PM_SE || config == KVX_PM_RE)
+ return KVX_PM_UNSUPPORTED;
+
+ return config;
+}
+
+static int kvx_pmu_event_init(struct perf_event *event)
+{
+ struct perf_event_attr *attr = &event->attr;
+ struct hw_perf_event *hwc = &event->hw;
+ enum kvx_pm_event_code code;
+
+ if (attr->exclude_user && !attr->exclude_kernel) {
+ attr->exclude_user = 0;
+ pr_err_once("kvx_pm: Cannot exclude userspace from perf events and not kernelspace\n");
+ }
+
+ switch (attr->type) {
+ case PERF_TYPE_HARDWARE:
+ code = kvx_hw_event_map[attr->config];
+ break;
+ case PERF_TYPE_HW_CACHE:
+ code = kvx_pmu_cache_event(attr->config);
+ break;
+ case PERF_TYPE_RAW:
+ code = kvx_pmu_raw_events(attr->config);
+ break;
+ default:
+ return -ENOENT;
+ }
+
+ if (code == KVX_PM_UNSUPPORTED)
+ return -EOPNOTSUPP;
+
+ hwc->config = code;
+ hwc->idx = -1;
+
+ if (event->cpu >= 0 && !cpu_online(event->cpu))
+ return -ENODEV;
+
+ return 0;
+}
+
+static struct pmu pmu = {
+ .event_init = kvx_pmu_event_init,
+ .add = kvx_pmu_add,
+ .del = kvx_pmu_del,
+ .start = kvx_pmu_start,
+ .stop = kvx_pmu_stop,
+ .read = kvx_pmu_read,
+};
+
+static void kvx_pm_clear_sav(void)
+{
+ u64 clr_mask = KVX_SFR_PMC_SAV_MASK;
+ u64 set_mask = 0;
+
+ kvx_sfr_set_mask(PMC, clr_mask, set_mask);
+}
+
+static void kvx_pm_reload(struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ u64 pm = hwc->idx;
+ u64 start_value = (u64)(-event->attr.sample_period);
+
+ kvx_set_pmc(pm, KVX_PM_SE);
+ kvx_set_pm(pm, start_value);
+}
+
+static bool kvx_pm_is_sav_set(void)
+{
+ return kvx_sfr_get(PMC) & KVX_SFR_PMC_SAV_MASK;
+}
+
+static int handle_pm_overflow(u8 pm_id, struct perf_event *event, u64 pmc,
+ struct pt_regs *regs)
+{
+ u64 pm_ie_mask = KVX_SFR_PMC_PM0IE_MASK << (pm_id + 1);
+ u64 pmc_event_code_mask = KVX_SFR_PMC_PM0C_MASK <<
+ ((pm_id + 1)
+ * KVX_SFR_PMC_PM1C_SHIFT);
+ struct hw_perf_event *hwc = &event->hw;
+ struct perf_sample_data data;
+ u64 sample_period;
+ u64 pm;
+
+ sample_period = event->attr.sample_period;
+ pm = kvx_sfr_iget(pm_id + KVX_SFR_PM1);
+
+ /*
+ * check if this pm has just overflowed
+ * ie: pm value is 0, pm interrupt is enabled
+ * and pm is not stopped.
+ */
+ if ((pm < local64_read(&hwc->prev_count)) && (pmc & pm_ie_mask)
+ && ((pmc & pmc_event_code_mask) != KVX_PM_SE)) {
+ perf_sample_data_init(&data, 0, sample_period);
+ if (perf_event_overflow(event, &data, regs))
+ pmu.stop(event, 0);
+ else {
+ kvx_pmu_read(event);
+ if (is_sampling_event(event))
+ kvx_pm_reload(event);
+ }
+ return 1;
+ }
+
+ return 0;
+}
+
+irqreturn_t pm_irq_handler(int irq, void *dev_id)
+{
+ struct cpu_hw_events *cpuc = &get_cpu_var(cpu_hw_events);
+ struct pt_regs *regs;
+ enum kvx_pm_idx pm_id;
+ u64 pmc;
+ bool a_pm_overflowed = false;
+ irqreturn_t ret = IRQ_NONE;
+
+ regs = get_irq_regs();
+ pmc = kvx_sfr_get(PMC);
+
+ for (pm_id = KVX_PM_1; pm_id <= KVX_PM_3; pm_id++) {
+ struct perf_event *event = cpuc->events[pm_id];
+
+ if (!event)
+ continue;
+
+ if (handle_pm_overflow(pm_id, event, pmc, regs)) {
+ ret = IRQ_HANDLED;
+ a_pm_overflowed = true;
+ }
+ }
+
+ put_cpu_var(cpu_hw_events);
+
+ if (likely(kvx_pm_is_sav_set()))
+ kvx_pm_clear_sav();
+ else
+ pr_err_once("kvx_pm: PM triggered an IRQ but did not set pmc.sav\n");
+
+ if (unlikely(!a_pm_overflowed))
+ pr_err_once("kvx_pm: PM triggered an IRQ but no PM seemed to have overflowed\n");
+
+ if (ret == IRQ_HANDLED)
+ irq_work_run();
+ return ret;
+}
+
+static int kvx_pmu_device_probe(struct platform_device *pdev)
+{
+ int ret;
+ int statenum;
+ struct device *dev = &pdev->dev;
+
+ ret = of_property_read_u32(dev->of_node, "kalray,pm-num", &pm_num);
+ if (ret < 0) {
+ dev_err(dev, "Cannot read kalray,pm-num from device tree\n");
+ return -ENODEV;
+ }
+
+ /*
+ * PM0 is reserved for cycle counting, that's why pm_num is
+ * decremented.
+ */
+ if (pm_num-- < 2) {
+ dev_err(dev, "Not enough PM to handle perf events, at least 2 are needed\n");
+ return -ENODEV;
+ }
+
+ kvx_pm_irq = platform_get_irq(pdev, 0);
+ if (!kvx_pm_irq) {
+ dev_err(dev, "Failed to parse pm irq\n");
+ return -ENODEV;
+ }
+
+ ret = request_percpu_irq(kvx_pm_irq, pm_irq_handler, "pm",
+ this_cpu_ptr(&cpu_hw_events));
+ if (ret) {
+ dev_err(dev, "Failed to request pm irq\n");
+ return -ENODEV;
+ }
+
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+ "kvx/pm_handler:online",
+ kvx_pm_starting_cpu,
+ kvx_pm_dying_cpu);
+
+ if (ret <= 0) {
+ dev_err(dev, "Failed to setup cpuhp\n");
+ goto free_irq;
+ }
+ statenum = ret;
+
+ ret = perf_pmu_register(&pmu, "cpu", PERF_TYPE_RAW);
+ if (ret) {
+ dev_err(dev, "Failed to register CPU PM as PMU\n");
+ goto free_cpuhp_state;
+ }
+
+ return ret;
+
+free_cpuhp_state:
+ cpuhp_remove_state(statenum);
+free_irq:
+ free_percpu_irq(kvx_pm_irq, this_cpu_ptr(&cpu_hw_events));
+ return ret;
+}
+
+static const struct of_device_id kvx_pmu_of_device_ids[] = {
+ {.compatible = "kalray,kvx-core-pm"},
+ {},
+};
+
+static struct platform_driver kvx_pmu_driver = {
+ .driver = {
+ .name = "pmu",
+ .of_match_table = kvx_pmu_of_device_ids,
+ },
+ .probe = kvx_pmu_device_probe,
+};
+
+static int __init kvx_pmu_driver_init(void)
+{
+ return platform_driver_register(&kvx_pmu_driver);
+}
+
+device_initcall(kvx_pmu_driver_init);
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* [RFC PATCH 24/25] kvx: Add support for CPU Perf Monitors
@ 2023-01-03 16:43 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, Will Deacon, Mark Rutland, Rob Herring,
Krzysztof Kozlowski, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Alexander Shishkin, Jiri Olsa,
Namhyung Kim, linux-arm-kernel, devicetree, linux-kernel,
linux-perf-users, Clement Leger, Jules Maselbas, Julian Vetter
Each kvx core includes several Perf Monitors.
This commit adds the driver that handles those core PM.
CC: Will Deacon <will@kernel.org>
CC: Mark Rutland <mark.rutland@arm.com>
CC: Rob Herring <robh+dt@kernel.org>
CC: Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Ingo Molnar <mingo@redhat.com>
CC: Arnaldo Carvalho de Melo <acme@kernel.org>
CC: Alexander Shishkin <alexander.shishkin@linux.intel.com>
CC: Jiri Olsa <jolsa@kernel.org>
CC: Namhyung Kim <namhyung@kernel.org>
CC: linux-arm-kernel@lists.infradead.org
CC: devicetree@vger.kernel.org
CC: linux-kernel@vger.kernel.org
CC: linux-perf-users@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Jules Maselbas <jmaselbas@kalray.eu>
Signed-off-by: Jules Maselbas <jmaselbas@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Co-developed-by: Yann Sionneau <ysionneau@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
.../devicetree/bindings/perf/kalray-pm.txt | 21 +
arch/kvx/include/asm/perf_event.h | 90 +++
arch/kvx/kernel/perf_event.c | 609 ++++++++++++++++++
3 files changed, 720 insertions(+)
create mode 100644 Documentation/devicetree/bindings/perf/kalray-pm.txt
create mode 100644 arch/kvx/include/asm/perf_event.h
create mode 100644 arch/kvx/kernel/perf_event.c
diff --git a/Documentation/devicetree/bindings/perf/kalray-pm.txt b/Documentation/devicetree/bindings/perf/kalray-pm.txt
new file mode 100644
index 000000000000..9ce00d703941
--- /dev/null
+++ b/Documentation/devicetree/bindings/perf/kalray-pm.txt
@@ -0,0 +1,21 @@
+* Kalray kvx Performance Monitors
+
+KVX core has several Performance Monitors for counting cpu and cache events.
+The KVX PM representation in the device tree should be done as under:
+
+Required properties:
+
+- compatible :
+ "kalray,kvx-core-pm"
+
+- interrupts : The interrupt number for kvx PM is 3.
+- interrupt-parent : The kvx core interrupt controller.
+- kalray,pm-num : Number of Performance Monitors the kvx core has.
+
+Example:
+core_pm {
+ compatible = "kalray,kvx-core-pm";
+ interrupts = <3>;
+ interrupt-parent = <&core_intc>;
+ kalray,pm-num = <4>;
+}
diff --git a/arch/kvx/include/asm/perf_event.h b/arch/kvx/include/asm/perf_event.h
new file mode 100644
index 000000000000..bb0147dfdf47
--- /dev/null
+++ b/arch/kvx/include/asm/perf_event.h
@@ -0,0 +1,90 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Yann Sionneau
+ * Clement Leger
+ */
+
+#ifndef _ASM_KVX_PERF_EVENT_H
+#define _ASM_KVX_PERF_EVENT_H
+
+#include <linux/perf_event.h>
+
+/**
+ * struct cpu_hw_events - per-cpu structure to describe PM resource usage
+ * @n_events: number of events currently existing
+ * @events: events[i] is the event using PMi. NULL if PMi is not used.
+ */
+struct cpu_hw_events {
+ unsigned int n_events;
+ struct perf_event **events;
+};
+
+enum kvx_pmc_ie {
+ PMC_IE_DISABLED,
+ PMC_IE_ENABLED
+};
+
+enum kvx_pm_idx {
+ KVX_PM_1,
+ KVX_PM_2,
+ KVX_PM_3
+};
+
+enum kvx_pm_event_code {
+ KVX_PM_PCC,
+ KVX_PM_ICC,
+ KVX_PM_EBE,
+ KVX_PM_ENIE,
+ KVX_PM_ENSE,
+ KVX_PM_ICHE,
+ KVX_PM_ICME,
+ KVX_PM_ICMABE,
+ KVX_PM_MNGIC,
+ KVX_PM_MIMHE,
+ KVX_PM_MIMME,
+ KVX_PM_IATSC,
+ KVX_PM_FE,
+ KVX_PM_PBSC,
+ KVX_PM_PNVC,
+ KVX_PM_PSC,
+ KVX_PM_TADBE,
+ KVX_PM_TABE,
+ KVX_PM_TBE,
+ KVX_PM_MDMHE,
+ KVX_PM_MDMME,
+ KVX_PM_DATSC,
+ KVX_PM_DCLHE,
+ KVX_PM_DCHE,
+ KVX_PM_DCLME,
+ KVX_PM_DCME,
+ KVX_PM_DARSC,
+ KVX_PM_LDSC,
+ KVX_PM_DCNGC,
+ KVX_PM_DMAE,
+ KVX_PM_LCFSC,
+ KVX_PM_MNGDC,
+ KVX_PM_MACC,
+ KVX_PM_TACC,
+ KVX_PM_IWC,
+ KVX_PM_WISC,
+ KVX_PM_SISC,
+ KVX_PM_DDSC,
+ KVX_PM_SC,
+ KVX_PM_ELE,
+ KVX_PM_ELNBE,
+ KVX_PM_ELUE,
+ KVX_PM_ELUNBE,
+ KVX_PM_ESE,
+ KVX_PM_ESNBE,
+ KVX_PM_EAE,
+ KVX_PM_CIRE,
+ KVX_PM_CIE,
+ KVX_PM_SE,
+ KVX_PM_RE,
+ KVX_PM_FSC,
+ KVX_PM_MAX,
+ KVX_PM_UNSUPPORTED = KVX_PM_MAX,
+};
+
+#endif /* _ASM_KVX_PERF_EVENT_H */
diff --git a/arch/kvx/kernel/perf_event.c b/arch/kvx/kernel/perf_event.c
new file mode 100644
index 000000000000..bfc547e78aba
--- /dev/null
+++ b/arch/kvx/kernel/perf_event.c
@@ -0,0 +1,609 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Yann Sionneau
+ * Clement Leger
+ */
+
+#include <linux/of.h>
+#include <linux/of_irq.h>
+#include <linux/kernel.h>
+#include <linux/perf_event.h>
+#include <linux/errno.h>
+#include <linux/platform_device.h>
+#include <asm/perf_event.h>
+
+static unsigned int pm_num;
+static unsigned int kvx_pm_irq;
+static DEFINE_PER_CPU(struct cpu_hw_events, cpu_hw_events);
+
+static const enum kvx_pm_event_code kvx_hw_event_map[] = {
+ [PERF_COUNT_HW_CPU_CYCLES] = KVX_PM_PCC,
+ [PERF_COUNT_HW_INSTRUCTIONS] = KVX_PM_ENIE,
+ [PERF_COUNT_HW_CACHE_REFERENCES] = KVX_PM_UNSUPPORTED,
+ [PERF_COUNT_HW_CACHE_MISSES] = KVX_PM_UNSUPPORTED,
+ [PERF_COUNT_HW_BRANCH_INSTRUCTIONS] = KVX_PM_TABE,
+ [PERF_COUNT_HW_BRANCH_MISSES] = KVX_PM_UNSUPPORTED,
+ [PERF_COUNT_HW_BUS_CYCLES] = KVX_PM_PCC,
+ [PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] = KVX_PM_PSC,
+ [PERF_COUNT_HW_STALLED_CYCLES_BACKEND] = KVX_PM_UNSUPPORTED,
+ [PERF_COUNT_HW_REF_CPU_CYCLES] = KVX_PM_UNSUPPORTED,
+};
+
+#define C(_x) PERF_COUNT_HW_CACHE_##_x
+
+static const enum kvx_pm_event_code
+ kvx_cache_map[C(MAX)][C(OP_MAX)][C(RESULT_MAX)] = {
+[C(L1D)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_DCLME,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(L1I)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_ICME,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(LL)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(DTLB)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(ITLB)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_MIMME,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(BPU)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+[C(NODE)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = KVX_PM_UNSUPPORTED,
+ [C(RESULT_MISS)] = KVX_PM_UNSUPPORTED,
+ },
+},
+};
+
+static u64 read_counter(struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ unsigned int pm = hwc->idx;
+
+ if (pm > pm_num) {
+ WARN_ONCE(1, "This PM (%u) does not exist!\n", pm);
+ return 0;
+ }
+ return kvx_sfr_iget(KVX_SFR_PM1 + pm);
+}
+
+static void kvx_pmu_read(struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ u64 delta, prev_raw_count, new_raw_count;
+
+ do {
+ prev_raw_count = local64_read(&hwc->prev_count);
+ new_raw_count = read_counter(event);
+ } while (local64_cmpxchg(&hwc->prev_count, prev_raw_count,
+ new_raw_count) != prev_raw_count);
+ /*
+ * delta is the value to update the counter we maintain in the kernel.
+ */
+ delta = (new_raw_count - prev_raw_count);
+ local64_add(delta, &event->count);
+}
+
+static void kvx_set_pmc_ie(unsigned int pm_num, enum kvx_pmc_ie ievalue)
+{
+ u64 shifted_value = ((u64)ievalue << KVX_SFR_PMC_PM1IE_SHIFT)
+ & KVX_SFR_PMC_PM1IE_MASK;
+ u64 clr_mask = KVX_SFR_PMC_PM1IE_MASK << pm_num;
+ u64 set_mask = shifted_value << pm_num;
+
+ kvx_sfr_set_mask(PMC, clr_mask, set_mask);
+}
+
+static void kvx_set_pmc(unsigned int pm_num, enum kvx_pm_event_code pmc_value)
+{
+ u64 pm_shift = (pm_num + 1) * KVX_SFR_PMC_PM1C_SHIFT;
+ u64 clr_mask = KVX_SFR_PMC_PM0C_MASK << pm_shift;
+ u64 set_mask = pmc_value << pm_shift;
+
+ kvx_sfr_set_mask(PMC, clr_mask, set_mask);
+}
+
+static void give_pm_to_user(unsigned int pm)
+{
+ int pl_value = 1 << (KVX_SFR_MOW_PM0_SHIFT
+ + KVX_SFR_MOW_PM0_WIDTH * (pm + 1));
+ int pl_clr_mask = 3 << (KVX_SFR_MOW_PM0_SHIFT
+ + KVX_SFR_MOW_PM0_WIDTH * (pm + 1));
+ kvx_sfr_set_mask(MOW, pl_clr_mask, pl_value);
+}
+
+static void get_pm_back_to_kernel(unsigned int pm)
+{
+ int pl_value = 0;
+ int pl_clr_mask = 3 << (KVX_SFR_MOW_PM0_SHIFT
+ + KVX_SFR_MOW_PM0_WIDTH * (pm + 1));
+ kvx_sfr_set_mask(MOW, pl_clr_mask, pl_value);
+}
+
+static void kvx_set_pm(enum kvx_pm_idx pm, u64 value)
+{
+ switch (pm) {
+ case KVX_PM_1:
+ kvx_sfr_set(PM1, value);
+ break;
+ case KVX_PM_2:
+ kvx_sfr_set(PM2, value);
+ break;
+ case KVX_PM_3:
+ kvx_sfr_set(PM3, value);
+ break;
+ default:
+ WARN_ONCE(1, "This PM (%u) does not exist!\n", pm);
+ }
+}
+
+static void kvx_stop_sampling_event(unsigned int pm)
+{
+ kvx_set_pmc_ie(pm, PMC_IE_DISABLED);
+}
+
+static u64 kvx_start_sampling_event(struct perf_event *event, unsigned int pm)
+{
+ u64 start_value;
+
+ if (event->attr.freq) {
+ pr_err_once("kvx_pm: Frequency sampling is not supported\n");
+ return 0;
+ }
+
+ /* PM counter will overflow after "sample_period" ticks */
+ start_value = (u64)(-event->attr.sample_period);
+
+ kvx_set_pmc(pm, KVX_PM_SE);
+ kvx_set_pm(pm, start_value);
+ kvx_set_pmc_ie(pm, PMC_IE_ENABLED);
+ return start_value;
+}
+
+static void kvx_pmu_start(struct perf_event *event, int flags)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct perf_event_attr *attr = &event->attr;
+ u64 pm_config = hwc->config;
+ unsigned int pm = hwc->idx;
+ u64 start_value = 0;
+
+ if (WARN_ON_ONCE(!(event->hw.state & PERF_HES_STOPPED)))
+ return;
+
+ if (flags & PERF_EF_RELOAD)
+ WARN_ON_ONCE(!(event->hw.state & PERF_HES_UPTODATE));
+
+ hwc->state = 0;
+ perf_event_update_userpage(event);
+
+ if (is_sampling_event(event))
+ start_value = kvx_start_sampling_event(event, pm);
+
+ local64_set(&hwc->prev_count, start_value);
+ if (attr->exclude_kernel)
+ give_pm_to_user(pm);
+ if (!is_sampling_event(event))
+ kvx_set_pmc(pm, KVX_PM_RE);
+ kvx_set_pmc(pm, pm_config);
+}
+
+static void kvx_pmu_stop(struct perf_event *event, int flags)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct perf_event_attr *attr = &event->attr;
+ unsigned int pm = hwc->idx;
+
+ if (is_sampling_event(event))
+ kvx_stop_sampling_event(pm);
+
+ kvx_set_pmc(pm, KVX_PM_SE);
+ if (attr->exclude_kernel)
+ get_pm_back_to_kernel(pm);
+
+ WARN_ON_ONCE(hwc->state & PERF_HES_STOPPED);
+ hwc->state |= PERF_HES_STOPPED;
+
+ if ((flags & PERF_EF_UPDATE) && !(hwc->state & PERF_HES_UPTODATE)) {
+ kvx_pmu_read(event);
+ hwc->state |= PERF_HES_UPTODATE;
+ }
+}
+
+static void kvx_pmu_del(struct perf_event *event, int flags)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct cpu_hw_events *cpuc = &get_cpu_var(cpu_hw_events);
+
+ cpuc->events[hwc->idx] = NULL;
+ cpuc->n_events--;
+ put_cpu_var(cpu_hw_events);
+ kvx_pmu_stop(event, PERF_EF_UPDATE);
+ perf_event_update_userpage(event);
+}
+
+static int kvx_pmu_add(struct perf_event *event, int flags)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct cpu_hw_events *cpuc = &get_cpu_var(cpu_hw_events);
+ unsigned int i, idx = -1;
+
+ if (cpuc->n_events >= pm_num) {
+ put_cpu_var(cpu_hw_events);
+ return -ENOSPC;
+ }
+
+ for (i = 0; i < pm_num; i++)
+ if (cpuc->events[i] == NULL)
+ idx = i;
+
+ BUG_ON(idx == -1);
+
+ hwc->idx = idx;
+ cpuc->events[idx] = event;
+ cpuc->n_events++;
+ put_cpu_var(cpu_hw_events);
+ hwc->state = PERF_HES_UPTODATE | PERF_HES_STOPPED;
+
+ if (flags & PERF_EF_START)
+ kvx_pmu_start(event, PERF_EF_RELOAD);
+
+ return 0;
+}
+
+static enum kvx_pm_event_code kvx_pmu_cache_event(u64 config)
+{
+ unsigned int type, op, result;
+ enum kvx_pm_event_code code;
+
+ type = (config >> 0) & 0xff;
+ op = (config >> 8) & 0xff;
+ result = (config >> 16) & 0xff;
+ if (type >= PERF_COUNT_HW_CACHE_MAX ||
+ op >= PERF_COUNT_HW_CACHE_OP_MAX ||
+ result >= PERF_COUNT_HW_CACHE_RESULT_MAX)
+ return KVX_PM_UNSUPPORTED;
+
+ code = kvx_cache_map[type][op][result];
+
+ return code;
+}
+
+static int kvx_pm_starting_cpu(unsigned int cpu)
+{
+ struct cpu_hw_events *cpuc = &per_cpu(cpu_hw_events, cpu);
+
+ cpuc->events = kmalloc_array(pm_num, sizeof(struct perf_event *),
+ GFP_ATOMIC);
+ if (!cpuc->events)
+ return -ENOMEM;
+
+ memset(cpuc->events, 0, pm_num * sizeof(struct perf_event *));
+
+ enable_percpu_irq(kvx_pm_irq, IRQ_TYPE_NONE);
+ return 0;
+}
+
+static int kvx_pm_dying_cpu(unsigned int cpu)
+{
+ struct cpu_hw_events *cpuc = &get_cpu_var(cpu_hw_events);
+
+ disable_percpu_irq(kvx_pm_irq);
+ kfree(cpuc->events);
+ put_cpu_var(cpu_hw_events);
+ return 0;
+}
+
+static enum kvx_pm_event_code kvx_pmu_raw_events(u64 config)
+{
+ if (config >= KVX_PM_MAX)
+ return KVX_PM_UNSUPPORTED;
+
+ if (config == KVX_PM_SE || config == KVX_PM_RE)
+ return KVX_PM_UNSUPPORTED;
+
+ return config;
+}
+
+static int kvx_pmu_event_init(struct perf_event *event)
+{
+ struct perf_event_attr *attr = &event->attr;
+ struct hw_perf_event *hwc = &event->hw;
+ enum kvx_pm_event_code code;
+
+ if (attr->exclude_user && !attr->exclude_kernel) {
+ attr->exclude_user = 0;
+ pr_err_once("kvx_pm: Cannot exclude userspace from perf events and not kernelspace\n");
+ }
+
+ switch (attr->type) {
+ case PERF_TYPE_HARDWARE:
+ code = kvx_hw_event_map[attr->config];
+ break;
+ case PERF_TYPE_HW_CACHE:
+ code = kvx_pmu_cache_event(attr->config);
+ break;
+ case PERF_TYPE_RAW:
+ code = kvx_pmu_raw_events(attr->config);
+ break;
+ default:
+ return -ENOENT;
+ }
+
+ if (code == KVX_PM_UNSUPPORTED)
+ return -EOPNOTSUPP;
+
+ hwc->config = code;
+ hwc->idx = -1;
+
+ if (event->cpu >= 0 && !cpu_online(event->cpu))
+ return -ENODEV;
+
+ return 0;
+}
+
+static struct pmu pmu = {
+ .event_init = kvx_pmu_event_init,
+ .add = kvx_pmu_add,
+ .del = kvx_pmu_del,
+ .start = kvx_pmu_start,
+ .stop = kvx_pmu_stop,
+ .read = kvx_pmu_read,
+};
+
+static void kvx_pm_clear_sav(void)
+{
+ u64 clr_mask = KVX_SFR_PMC_SAV_MASK;
+ u64 set_mask = 0;
+
+ kvx_sfr_set_mask(PMC, clr_mask, set_mask);
+}
+
+static void kvx_pm_reload(struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ u64 pm = hwc->idx;
+ u64 start_value = (u64)(-event->attr.sample_period);
+
+ kvx_set_pmc(pm, KVX_PM_SE);
+ kvx_set_pm(pm, start_value);
+}
+
+static bool kvx_pm_is_sav_set(void)
+{
+ return kvx_sfr_get(PMC) & KVX_SFR_PMC_SAV_MASK;
+}
+
+static int handle_pm_overflow(u8 pm_id, struct perf_event *event, u64 pmc,
+ struct pt_regs *regs)
+{
+ u64 pm_ie_mask = KVX_SFR_PMC_PM0IE_MASK << (pm_id + 1);
+ u64 pmc_event_code_mask = KVX_SFR_PMC_PM0C_MASK <<
+ ((pm_id + 1)
+ * KVX_SFR_PMC_PM1C_SHIFT);
+ struct hw_perf_event *hwc = &event->hw;
+ struct perf_sample_data data;
+ u64 sample_period;
+ u64 pm;
+
+ sample_period = event->attr.sample_period;
+ pm = kvx_sfr_iget(pm_id + KVX_SFR_PM1);
+
+ /*
+ * check if this pm has just overflowed
+ * ie: pm value is 0, pm interrupt is enabled
+ * and pm is not stopped.
+ */
+ if ((pm < local64_read(&hwc->prev_count)) && (pmc & pm_ie_mask)
+ && ((pmc & pmc_event_code_mask) != KVX_PM_SE)) {
+ perf_sample_data_init(&data, 0, sample_period);
+ if (perf_event_overflow(event, &data, regs))
+ pmu.stop(event, 0);
+ else {
+ kvx_pmu_read(event);
+ if (is_sampling_event(event))
+ kvx_pm_reload(event);
+ }
+ return 1;
+ }
+
+ return 0;
+}
+
+irqreturn_t pm_irq_handler(int irq, void *dev_id)
+{
+ struct cpu_hw_events *cpuc = &get_cpu_var(cpu_hw_events);
+ struct pt_regs *regs;
+ enum kvx_pm_idx pm_id;
+ u64 pmc;
+ bool a_pm_overflowed = false;
+ irqreturn_t ret = IRQ_NONE;
+
+ regs = get_irq_regs();
+ pmc = kvx_sfr_get(PMC);
+
+ for (pm_id = KVX_PM_1; pm_id <= KVX_PM_3; pm_id++) {
+ struct perf_event *event = cpuc->events[pm_id];
+
+ if (!event)
+ continue;
+
+ if (handle_pm_overflow(pm_id, event, pmc, regs)) {
+ ret = IRQ_HANDLED;
+ a_pm_overflowed = true;
+ }
+ }
+
+ put_cpu_var(cpu_hw_events);
+
+ if (likely(kvx_pm_is_sav_set()))
+ kvx_pm_clear_sav();
+ else
+ pr_err_once("kvx_pm: PM triggered an IRQ but did not set pmc.sav\n");
+
+ if (unlikely(!a_pm_overflowed))
+ pr_err_once("kvx_pm: PM triggered an IRQ but no PM seemed to have overflowed\n");
+
+ if (ret == IRQ_HANDLED)
+ irq_work_run();
+ return ret;
+}
+
+static int kvx_pmu_device_probe(struct platform_device *pdev)
+{
+ int ret;
+ int statenum;
+ struct device *dev = &pdev->dev;
+
+ ret = of_property_read_u32(dev->of_node, "kalray,pm-num", &pm_num);
+ if (ret < 0) {
+ dev_err(dev, "Cannot read kalray,pm-num from device tree\n");
+ return -ENODEV;
+ }
+
+ /*
+ * PM0 is reserved for cycle counting, that's why pm_num is
+ * decremented.
+ */
+ if (pm_num-- < 2) {
+ dev_err(dev, "Not enough PM to handle perf events, at least 2 are needed\n");
+ return -ENODEV;
+ }
+
+ kvx_pm_irq = platform_get_irq(pdev, 0);
+ if (!kvx_pm_irq) {
+ dev_err(dev, "Failed to parse pm irq\n");
+ return -ENODEV;
+ }
+
+ ret = request_percpu_irq(kvx_pm_irq, pm_irq_handler, "pm",
+ this_cpu_ptr(&cpu_hw_events));
+ if (ret) {
+ dev_err(dev, "Failed to request pm irq\n");
+ return -ENODEV;
+ }
+
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+ "kvx/pm_handler:online",
+ kvx_pm_starting_cpu,
+ kvx_pm_dying_cpu);
+
+ if (ret <= 0) {
+ dev_err(dev, "Failed to setup cpuhp\n");
+ goto free_irq;
+ }
+ statenum = ret;
+
+ ret = perf_pmu_register(&pmu, "cpu", PERF_TYPE_RAW);
+ if (ret) {
+ dev_err(dev, "Failed to register CPU PM as PMU\n");
+ goto free_cpuhp_state;
+ }
+
+ return ret;
+
+free_cpuhp_state:
+ cpuhp_remove_state(statenum);
+free_irq:
+ free_percpu_irq(kvx_pm_irq, this_cpu_ptr(&cpu_hw_events));
+ return ret;
+}
+
+static const struct of_device_id kvx_pmu_of_device_ids[] = {
+ {.compatible = "kalray,kvx-core-pm"},
+ {},
+};
+
+static struct platform_driver kvx_pmu_driver = {
+ .driver = {
+ .name = "pmu",
+ .of_match_table = kvx_pmu_of_device_ids,
+ },
+ .probe = kvx_pmu_device_probe,
+};
+
+static int __init kvx_pmu_driver_init(void)
+{
+ return platform_driver_register(&kvx_pmu_driver);
+}
+
+device_initcall(kvx_pmu_driver_init);
--
2.37.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 113+ messages in thread
* [RFC PATCH 25/25] kvx: Add support for cpuinfo
2023-01-03 16:43 ` Yann Sionneau
` (25 preceding siblings ...)
(?)
@ 2023-01-03 16:43 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-03 16:43 UTC (permalink / raw)
Cc: Yann Sionneau, linux-kernel, Clement Leger, Guillaume Thouvenin,
Julian Vetter
Add support for cpuinfo on kvx arch.
CC: linux-kernel@vger.kernel.org
Co-developed-by: Clement Leger <clement.leger@bootlin.com>
Signed-off-by: Clement Leger <clement.leger@bootlin.com>
Co-developed-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Signed-off-by: Guillaume Thouvenin <gthouvenin@kalray.eu>
Co-developed-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Julian Vetter <jvetter@kalray.eu>
Signed-off-by: Yann Sionneau <ysionneau@kalray.eu>
---
arch/kvx/kernel/cpuinfo.c | 96 +++++++++++++++++++++++++++++++++++++++
1 file changed, 96 insertions(+)
create mode 100644 arch/kvx/kernel/cpuinfo.c
diff --git a/arch/kvx/kernel/cpuinfo.c b/arch/kvx/kernel/cpuinfo.c
new file mode 100644
index 000000000000..f44c46c1e4ba
--- /dev/null
+++ b/arch/kvx/kernel/cpuinfo.c
@@ -0,0 +1,96 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017-2023 Kalray Inc.
+ * Author(s): Clement Leger
+ * Guillaume Thouvenin
+ */
+
+#include <linux/seq_file.h>
+#include <linux/delay.h>
+#include <linux/clk.h>
+#include <linux/cpu.h>
+#include <linux/of.h>
+
+unsigned long elf_hwcap __read_mostly;
+
+static int show_cpuinfo(struct seq_file *m, void *v)
+{
+ int cpu_num = *(unsigned int *)v;
+ struct cpuinfo_kvx *n = per_cpu_ptr(&cpu_info, cpu_num);
+
+ seq_printf(m, "processor\t: %d\nvendor_id\t: Kalray\n", cpu_num);
+
+ seq_printf(m,
+ "copro enabled\t: %s\n"
+ "arch revision\t: %d\n"
+ "uarch revision\t: %d\n",
+ n->copro_enable ? "yes" : "no",
+ n->arch_rev,
+ n->uarch_rev);
+
+ seq_printf(m,
+ "bogomips\t: %lu.%02lu\n"
+ "cpu MHz\t\t: %llu.%03llu\n\n",
+ (loops_per_jiffy * HZ) / 500000,
+ ((loops_per_jiffy * HZ) / 5000) % 100,
+ n->freq / 1000000, (n->freq / 10000) % 100);
+
+ return 0;
+}
+
+static void *c_start(struct seq_file *m, loff_t *pos)
+{
+ if (*pos == 0)
+ *pos = cpumask_first(cpu_online_mask);
+ if (*pos >= num_online_cpus())
+ return NULL;
+
+ return pos;
+}
+
+static void *c_next(struct seq_file *m, void *v, loff_t *pos)
+{
+ *pos = cpumask_next(*pos, cpu_online_mask);
+
+ return c_start(m, pos);
+}
+
+static void c_stop(struct seq_file *m, void *v)
+{
+}
+
+const struct seq_operations cpuinfo_op = {
+ .start = c_start,
+ .next = c_next,
+ .stop = c_stop,
+ .show = show_cpuinfo,
+};
+
+static int __init setup_cpuinfo(void)
+{
+ int cpu;
+ struct clk *clk;
+ unsigned long cpu_freq = 1000000000;
+ struct device_node *node = of_get_cpu_node(0, NULL);
+
+ clk = of_clk_get(node, 0);
+ if (IS_ERR(clk)) {
+ printk(KERN_WARNING
+ "Device tree missing CPU 'clock' parameter. Assuming frequency is 1GHZ");
+ goto setup_cpu_freq;
+ }
+
+ cpu_freq = clk_get_rate(clk);
+
+ clk_put(clk);
+
+setup_cpu_freq:
+ of_node_put(node);
+
+ for_each_possible_cpu(cpu)
+ per_cpu_ptr(&cpu_info, cpu)->freq = cpu_freq;
+
+ return 0;
+}
+
+late_initcall(setup_cpuinfo);
--
2.37.2
^ permalink raw reply related [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-03 16:43 ` Yann Sionneau
(?)
@ 2023-01-03 20:52 ` Rob Herring
-1 siblings, 0 replies; 113+ messages in thread
From: Rob Herring @ 2023-01-03 20:52 UTC (permalink / raw)
To: Yann Sionneau
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Sebastian Reichel,
Steven Rostedt, Thomas Gleixner, Waiman Long, Will Deacon,
Alex Michon, Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jean-Christophe Pince,
Jonathan Borne, Jules Maselbas, Julian Vetter, Julien Hascoet,
Julien Villette, Louis Morhet, Luc Michel, Marc Poulhiès,
Marius Gligor, Samuel Jones, Thomas Costis, Vincent Chardon
On Tue, Jan 03, 2023 at 05:43:34PM +0100, Yann Sionneau wrote:
> This patch series adds support for the kv3-1 CPU architecture of the kvx family
> found in the Coolidge (aka MPPA3-80) SoC of Kalray.
>
> This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> therefore this patch series cannot be merged into Linux for now.
>
> The goal is to have preliminary reviews and to fix problems early.
>
> The Kalray VLIW processor family (kvx) has the following features:
> * 32/64 bits execution mode
> * 6-issue VLIW architecture
> * 64 x 64bits general purpose registers
> * SIMD instructions
> * little-endian
> * deep learning co-processor
>
> Kalray kv3-1 core which is the third of the kvx family is embedded in Kalray
> Coolidge SoC currently used on K200 and K200-LP boards.
>
> The Coolidge SoC contains 5 clusters each of which is made of:
> * 4MiB of on-chip memory (SMEM)
> * 1 dedicated safety/security core (kv3-1 core).
> * 16 PEs (Processing Elements) (kv3-1 cores).
> * 16 Co-processors (one per PE)
> * 2 Crypto accelerators
>
> The Coolidge SoC contains the following features:
> * 5 Clusters
> * 2 100G Ethernet controllers
> * 8 PCIe GEN4 controllers (Root Complex and Endpoint capable)
> * 2 USB 2.0 controllers
> * 1 Octal SPI-NOR flash controller
> * 1 eMMC controller
> * 3 Quad SPI controllers
> * 6 UART
> * 5 I2C controllers (3 of which are SMBus capable)
> * 4 CAN controllers
> * 1 OTP memory
>
> A kvx toolchain can be built using:
> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> $ git clone https://github.com/kalray/build-scripts
> $ cd build-scripts
> $ source last.refs
> $ ./build-kvx-xgcc.sh output
>
> The kvx toolchain will be installed in the "output" directory.
>
> A buildroot image (kernel+rootfs) and toolchain can be built using:
> $ git clone -b coolidge-for-upstream https://github.com/kalray/buildroot
> $ cd buildroot
> $ make O=build_kvx kvx_defconfig
> $ make O=build_kvx
>
> The vmlinux image can be found in buildroot/build_kvx/images/vmlinux.
>
> If you are just interested in building the Linux kernel with no rootfs you can
> just do this with the kvx-elf- toolchain:
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- default_defconfig
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- -j$(($(nproc) + 1))
>
> The vmlinux ELF can be run with qemu by doing:
> # install dependencies: ninja pkg-config libglib-2.0-dev cmake libfdt-dev libpixman-1-dev zlib1g-dev
> $ git clone https://github.com/kalray/qemu-builder
> $ cd qemu-builder
> $ git submodule update --init
> $ make -j$(($(nproc) + 1))
> $ ./qemu-system-kvx -m 1024 -nographic -kernel <path/to/vmlinux>
>
> Yann Sionneau (25):
> Documentation: kvx: Add basic documentation
> kvx: Add ELF-related definitions
> kvx: Add build infrastructure
> kvx: Add CPU definition headers
> kvx: Add atomic/locking headers
> kvx: Add other common headers
> kvx: Add boot and setup routines
> kvx: Add exception/interrupt handling
> kvx: irqchip: Add support for irq controllers
> kvx: Add process management
> kvx: Add memory management
> kvx: Add system call support
> kvx: Add signal handling support
> kvx: Add ELF relocations and module support
> kvx: Add misc common routines
> kvx: Add some library functions
> kvx: Add multi-processor (SMP) support
> kvx: Add kvx default config file
> kvx: power: scall poweroff driver
> kvx: gdb: add kvx related gdb helpers
> kvx: Add support for ftrace
> kvx: Add support for jump labels
> kvx: Add debugging related support
> kvx: Add support for CPU Perf Monitors
> kvx: Add support for cpuinfo
You should strip this series down to just what's needed to boot. You
don't need the last 7 patches at least.
Rob
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-03 20:52 ` Rob Herring
0 siblings, 0 replies; 113+ messages in thread
From: Rob Herring @ 2023-01-03 20:52 UTC (permalink / raw)
To: Yann Sionneau
Cc: Mark Rutland, Marc Poulhiès, linux-doc, Alexander Shishkin,
Jan Kiszka, Sebastian Reichel, Marius Gligor, Oleg Nesterov,
linux-mm, Louis Morhet, Krzysztof Kozlowski, Luc Michel,
linux-riscv, Ashley Lesdalons, Will Deacon, Ard Biesheuvel,
linux-arch, Marc Zyngier, Thomas Costis, Jonathan Corbet,
Jonathan Borne, Aneesh Kumar K.V, Peter Zijlstra, Ingo Molnar,
Steven Rostedt, Jean-Christophe Pince, Waiman Long,
Clement Leger, Masami Hiramatsu, devicetree, Albert Ou,
Kees Cook, Arnd Bergmann, linux-pm, Samuel Jones, Boqun Feng,
Guillaume Thouvenin, Julian Vetter, Arnaldo Carvalho de Melo,
Julien Hascoet, Jason Baron, Nick Piggin, Paul Walmsley,
Namhyung Kim, Thomas Gleixner, Josh Poimboeuf, linux-arm-kernel,
Julien Villette, Jules Maselbas, Christian Brauner,
Andrew Morton, Benjamin Mugnier, Vincent Chardon,
Guillaume Missonnier, linux-kernel, Eric Paris, Alex Michon,
linux-perf-users, linux-audit, Palmer Dabbelt, Eric Biederman,
Jiri Olsa, Kieran Bingham, bpf
On Tue, Jan 03, 2023 at 05:43:34PM +0100, Yann Sionneau wrote:
> This patch series adds support for the kv3-1 CPU architecture of the kvx family
> found in the Coolidge (aka MPPA3-80) SoC of Kalray.
>
> This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> therefore this patch series cannot be merged into Linux for now.
>
> The goal is to have preliminary reviews and to fix problems early.
>
> The Kalray VLIW processor family (kvx) has the following features:
> * 32/64 bits execution mode
> * 6-issue VLIW architecture
> * 64 x 64bits general purpose registers
> * SIMD instructions
> * little-endian
> * deep learning co-processor
>
> Kalray kv3-1 core which is the third of the kvx family is embedded in Kalray
> Coolidge SoC currently used on K200 and K200-LP boards.
>
> The Coolidge SoC contains 5 clusters each of which is made of:
> * 4MiB of on-chip memory (SMEM)
> * 1 dedicated safety/security core (kv3-1 core).
> * 16 PEs (Processing Elements) (kv3-1 cores).
> * 16 Co-processors (one per PE)
> * 2 Crypto accelerators
>
> The Coolidge SoC contains the following features:
> * 5 Clusters
> * 2 100G Ethernet controllers
> * 8 PCIe GEN4 controllers (Root Complex and Endpoint capable)
> * 2 USB 2.0 controllers
> * 1 Octal SPI-NOR flash controller
> * 1 eMMC controller
> * 3 Quad SPI controllers
> * 6 UART
> * 5 I2C controllers (3 of which are SMBus capable)
> * 4 CAN controllers
> * 1 OTP memory
>
> A kvx toolchain can be built using:
> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> $ git clone https://github.com/kalray/build-scripts
> $ cd build-scripts
> $ source last.refs
> $ ./build-kvx-xgcc.sh output
>
> The kvx toolchain will be installed in the "output" directory.
>
> A buildroot image (kernel+rootfs) and toolchain can be built using:
> $ git clone -b coolidge-for-upstream https://github.com/kalray/buildroot
> $ cd buildroot
> $ make O=build_kvx kvx_defconfig
> $ make O=build_kvx
>
> The vmlinux image can be found in buildroot/build_kvx/images/vmlinux.
>
> If you are just interested in building the Linux kernel with no rootfs you can
> just do this with the kvx-elf- toolchain:
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- default_defconfig
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- -j$(($(nproc) + 1))
>
> The vmlinux ELF can be run with qemu by doing:
> # install dependencies: ninja pkg-config libglib-2.0-dev cmake libfdt-dev libpixman-1-dev zlib1g-dev
> $ git clone https://github.com/kalray/qemu-builder
> $ cd qemu-builder
> $ git submodule update --init
> $ make -j$(($(nproc) + 1))
> $ ./qemu-system-kvx -m 1024 -nographic -kernel <path/to/vmlinux>
>
> Yann Sionneau (25):
> Documentation: kvx: Add basic documentation
> kvx: Add ELF-related definitions
> kvx: Add build infrastructure
> kvx: Add CPU definition headers
> kvx: Add atomic/locking headers
> kvx: Add other common headers
> kvx: Add boot and setup routines
> kvx: Add exception/interrupt handling
> kvx: irqchip: Add support for irq controllers
> kvx: Add process management
> kvx: Add memory management
> kvx: Add system call support
> kvx: Add signal handling support
> kvx: Add ELF relocations and module support
> kvx: Add misc common routines
> kvx: Add some library functions
> kvx: Add multi-processor (SMP) support
> kvx: Add kvx default config file
> kvx: power: scall poweroff driver
> kvx: gdb: add kvx related gdb helpers
> kvx: Add support for ftrace
> kvx: Add support for jump labels
> kvx: Add debugging related support
> kvx: Add support for CPU Perf Monitors
> kvx: Add support for cpuinfo
You should strip this series down to just what's needed to boot. You
don't need the last 7 patches at least.
Rob
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-03 20:52 ` Rob Herring
0 siblings, 0 replies; 113+ messages in thread
From: Rob Herring @ 2023-01-03 20:52 UTC (permalink / raw)
To: Yann Sionneau
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Sebastian Reichel,
Steven Rostedt, Thomas Gleixner, Waiman Long, Will Deacon,
Alex Michon, Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jean-Christophe Pince,
Jonathan Borne, Jules Maselbas, Julian Vetter, Julien Hascoet,
Julien Villette, Louis Morhet, Luc Michel, Marc Poulhiès,
Marius Gligor, Samuel Jones, Thomas Costis, Vincent Chardon
On Tue, Jan 03, 2023 at 05:43:34PM +0100, Yann Sionneau wrote:
> This patch series adds support for the kv3-1 CPU architecture of the kvx family
> found in the Coolidge (aka MPPA3-80) SoC of Kalray.
>
> This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> therefore this patch series cannot be merged into Linux for now.
>
> The goal is to have preliminary reviews and to fix problems early.
>
> The Kalray VLIW processor family (kvx) has the following features:
> * 32/64 bits execution mode
> * 6-issue VLIW architecture
> * 64 x 64bits general purpose registers
> * SIMD instructions
> * little-endian
> * deep learning co-processor
>
> Kalray kv3-1 core which is the third of the kvx family is embedded in Kalray
> Coolidge SoC currently used on K200 and K200-LP boards.
>
> The Coolidge SoC contains 5 clusters each of which is made of:
> * 4MiB of on-chip memory (SMEM)
> * 1 dedicated safety/security core (kv3-1 core).
> * 16 PEs (Processing Elements) (kv3-1 cores).
> * 16 Co-processors (one per PE)
> * 2 Crypto accelerators
>
> The Coolidge SoC contains the following features:
> * 5 Clusters
> * 2 100G Ethernet controllers
> * 8 PCIe GEN4 controllers (Root Complex and Endpoint capable)
> * 2 USB 2.0 controllers
> * 1 Octal SPI-NOR flash controller
> * 1 eMMC controller
> * 3 Quad SPI controllers
> * 6 UART
> * 5 I2C controllers (3 of which are SMBus capable)
> * 4 CAN controllers
> * 1 OTP memory
>
> A kvx toolchain can be built using:
> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> $ git clone https://github.com/kalray/build-scripts
> $ cd build-scripts
> $ source last.refs
> $ ./build-kvx-xgcc.sh output
>
> The kvx toolchain will be installed in the "output" directory.
>
> A buildroot image (kernel+rootfs) and toolchain can be built using:
> $ git clone -b coolidge-for-upstream https://github.com/kalray/buildroot
> $ cd buildroot
> $ make O=build_kvx kvx_defconfig
> $ make O=build_kvx
>
> The vmlinux image can be found in buildroot/build_kvx/images/vmlinux.
>
> If you are just interested in building the Linux kernel with no rootfs you can
> just do this with the kvx-elf- toolchain:
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- default_defconfig
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- -j$(($(nproc) + 1))
>
> The vmlinux ELF can be run with qemu by doing:
> # install dependencies: ninja pkg-config libglib-2.0-dev cmake libfdt-dev libpixman-1-dev zlib1g-dev
> $ git clone https://github.com/kalray/qemu-builder
> $ cd qemu-builder
> $ git submodule update --init
> $ make -j$(($(nproc) + 1))
> $ ./qemu-system-kvx -m 1024 -nographic -kernel <path/to/vmlinux>
>
> Yann Sionneau (25):
> Documentation: kvx: Add basic documentation
> kvx: Add ELF-related definitions
> kvx: Add build infrastructure
> kvx: Add CPU definition headers
> kvx: Add atomic/locking headers
> kvx: Add other common headers
> kvx: Add boot and setup routines
> kvx: Add exception/interrupt handling
> kvx: irqchip: Add support for irq controllers
> kvx: Add process management
> kvx: Add memory management
> kvx: Add system call support
> kvx: Add signal handling support
> kvx: Add ELF relocations and module support
> kvx: Add misc common routines
> kvx: Add some library functions
> kvx: Add multi-processor (SMP) support
> kvx: Add kvx default config file
> kvx: power: scall poweroff driver
> kvx: gdb: add kvx related gdb helpers
> kvx: Add support for ftrace
> kvx: Add support for jump labels
> kvx: Add debugging related support
> kvx: Add support for CPU Perf Monitors
> kvx: Add support for cpuinfo
You should strip this series down to just what's needed to boot. You
don't need the last 7 patches at least.
Rob
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-03 16:43 ` Yann Sionneau
(?)
@ 2023-01-04 15:58 ` Arnd Bergmann
-1 siblings, 0 replies; 113+ messages in thread
From: Arnd Bergmann @ 2023-01-04 15:58 UTC (permalink / raw)
To: Yann Sionneau
Cc: Albert Ou, Alexander Shishkin, Andrew Morton, Aneesh Kumar,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Boqun Feng, bpf,
Christian Brauner, devicetree, Eric W. Biederman, Eric Paris,
Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa, Jonathan Corbet,
Josh Poimboeuf, Kees Cook, Kieran Bingham, Krzysztof Kozlowski,
Linux-Arch, linux-arm-kernel, linux-audit, linux-doc,
linux-kernel, linux-mm, linux-perf-users, linux-pm, linux-riscv,
Marc Zyngier, Mark Rutland, Masami Hiramatsu, Namhyung Kim,
Nicholas Piggin, Oleg Nesterov, Palmer Dabbelt, Paul Moore,
Paul Walmsley, Peter Zijlstra, Rob Herring, Sebastian Reichel,
Steven Rostedt, Thomas Gleixner, Waiman Long, Will Deacon,
Alex Michon, Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jean-Christophe Pince,
Jonathan Borne, Jules Maselbas, Julian Vetter, Julien Hascoet,
Julien Villette, Louis Morhet, Luc Michel, Marc Poulhiès,
Marius Gligor, Samuel Jones, Thomas Costis, Vincent Chardon
On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
> This patch series adds support for the kv3-1 CPU architecture of the kvx family
> found in the Coolidge (aka MPPA3-80) SoC of Kalray.
>
> This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> therefore this patch series cannot be merged into Linux for now.
>
> The goal is to have preliminary reviews and to fix problems early.
>
> The Kalray VLIW processor family (kvx) has the following features:
> * 32/64 bits execution mode
> * 6-issue VLIW architecture
> * 64 x 64bits general purpose registers
> * SIMD instructions
> * little-endian
> * deep learning co-processor
Thanks for posting these, I had been wondering about the
state of the port. Overall this looks really nice, I can
see that you and the team have looked at other ports
and generally made the right decisions.
I commented on the syscall patch directly, I think it's
important to stop using the deprecated syscalls as soon
as possible to avoid having dependencies in too many
libc binaries. Almost everything else can be changed
easily as you get closer to upstream inclusion.
I did not receive most of the other patches as I'm
not subscribed to all the mainline lists. For future
submissions, can you add the linux-arch list to Cc for
all patches?
Reading the rest of the series through lore.kernel.org,
most of the comments I have are for improvements that
you may find valuable rather than serious mistakes:
- the {copy_to,copy_from,clear}_user functions are
well worth optimizing better than the byte-at-a-time
version you have, even just a C version built around
your __get_user/__put_user inline asm should help, and
could be added to lib/usercopy.c.
- The __raw_{read,write}{b,w,l,q} helpers should
normally be defined as inline asm instead of
volatile pointer dereferences, I've seen cases where
the compiler ends up splitting the access or does
other things you may not want on MMIO areas.
- I would recomment implementing HAVE_ARCH_VMAP_STACK
as well as IRQ stacks, both of these help to
avoid data corruption from stack overflow that you
will eventually run into.
- You use qspinlock as the only available spinlock
implementation, but only support running on a
single cluster of 16 cores. It may help to use
the generic ticket spinlock instead, or leave it
as a Kconfig option, in particular since you only
have the emulated xchg16() atomic for qspinlock.
- Your defconfig file enables CONFIG_EMBEDDED, which
in turn enables CONFIG_EXPERT. This is probably
not what you want, so better turn off both of these.
- The GENERIC_CALIBRATE_DELAY should not be necessary
since you have a get_cycles() based delay loop.
Just set loops_per_jiffy to the correct value based
on the frequency of the cycle counter, to save
a little time during boot and get a more accurate
delay loop.
Arnd
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-04 15:58 ` Arnd Bergmann
0 siblings, 0 replies; 113+ messages in thread
From: Arnd Bergmann @ 2023-01-04 15:58 UTC (permalink / raw)
To: Yann Sionneau
Cc: Mark Rutland, Marc Poulhiès, linux-doc, Alexander Shishkin,
Jan Kiszka, Sebastian Reichel, Marius Gligor, Oleg Nesterov,
linux-mm, Louis Morhet, Krzysztof Kozlowski, Luc Michel,
linux-riscv, Ashley Lesdalons, Will Deacon, Ard Biesheuvel,
Linux-Arch, Marc Zyngier, Thomas Costis, Jonathan Corbet,
Jonathan Borne, Aneesh Kumar, Peter Zijlstra, Ingo Molnar,
Steven Rostedt, Jean-Christophe Pince, Waiman Long,
Clement Leger, Masami Hiramatsu, devicetree, Albert Ou,
Kees Cook, linux-pm, Samuel Jones, Boqun Feng,
Guillaume Thouvenin, Julian Vetter, Arnaldo Carvalho de Melo,
Julien Hascoet, Jason Baron, Rob Herring, Nicholas Piggin,
Paul Walmsley, Namhyung Kim, Thomas Gleixner, Josh Poimboeuf,
linux-arm-kernel, Julien Villette, Jules Maselbas,
Christian Brauner, Andrew Morton, Benjamin Mugnier,
Vincent Chardon, Guillaume Missonnier, linux-kernel, Eric Paris,
Alex Michon, linux-perf-users, linux-audit, Palmer Dabbelt,
Eric W. Biederman, Jiri Olsa, Kieran Bingham, bpf
On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
> This patch series adds support for the kv3-1 CPU architecture of the kvx family
> found in the Coolidge (aka MPPA3-80) SoC of Kalray.
>
> This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> therefore this patch series cannot be merged into Linux for now.
>
> The goal is to have preliminary reviews and to fix problems early.
>
> The Kalray VLIW processor family (kvx) has the following features:
> * 32/64 bits execution mode
> * 6-issue VLIW architecture
> * 64 x 64bits general purpose registers
> * SIMD instructions
> * little-endian
> * deep learning co-processor
Thanks for posting these, I had been wondering about the
state of the port. Overall this looks really nice, I can
see that you and the team have looked at other ports
and generally made the right decisions.
I commented on the syscall patch directly, I think it's
important to stop using the deprecated syscalls as soon
as possible to avoid having dependencies in too many
libc binaries. Almost everything else can be changed
easily as you get closer to upstream inclusion.
I did not receive most of the other patches as I'm
not subscribed to all the mainline lists. For future
submissions, can you add the linux-arch list to Cc for
all patches?
Reading the rest of the series through lore.kernel.org,
most of the comments I have are for improvements that
you may find valuable rather than serious mistakes:
- the {copy_to,copy_from,clear}_user functions are
well worth optimizing better than the byte-at-a-time
version you have, even just a C version built around
your __get_user/__put_user inline asm should help, and
could be added to lib/usercopy.c.
- The __raw_{read,write}{b,w,l,q} helpers should
normally be defined as inline asm instead of
volatile pointer dereferences, I've seen cases where
the compiler ends up splitting the access or does
other things you may not want on MMIO areas.
- I would recomment implementing HAVE_ARCH_VMAP_STACK
as well as IRQ stacks, both of these help to
avoid data corruption from stack overflow that you
will eventually run into.
- You use qspinlock as the only available spinlock
implementation, but only support running on a
single cluster of 16 cores. It may help to use
the generic ticket spinlock instead, or leave it
as a Kconfig option, in particular since you only
have the emulated xchg16() atomic for qspinlock.
- Your defconfig file enables CONFIG_EMBEDDED, which
in turn enables CONFIG_EXPERT. This is probably
not what you want, so better turn off both of these.
- The GENERIC_CALIBRATE_DELAY should not be necessary
since you have a get_cycles() based delay loop.
Just set loops_per_jiffy to the correct value based
on the frequency of the cycle counter, to save
a little time during boot and get a more accurate
delay loop.
Arnd
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-04 15:58 ` Arnd Bergmann
0 siblings, 0 replies; 113+ messages in thread
From: Arnd Bergmann @ 2023-01-04 15:58 UTC (permalink / raw)
To: Yann Sionneau
Cc: Albert Ou, Alexander Shishkin, Andrew Morton, Aneesh Kumar,
Ard Biesheuvel, Arnaldo Carvalho de Melo, Boqun Feng, bpf,
Christian Brauner, devicetree, Eric W. Biederman, Eric Paris,
Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa, Jonathan Corbet,
Josh Poimboeuf, Kees Cook, Kieran Bingham, Krzysztof Kozlowski,
Linux-Arch, linux-arm-kernel, linux-audit, linux-doc,
linux-kernel, linux-mm, linux-perf-users, linux-pm, linux-riscv,
Marc Zyngier, Mark Rutland, Masami Hiramatsu, Namhyung Kim,
Nicholas Piggin, Oleg Nesterov, Palmer Dabbelt, Paul Moore,
Paul Walmsley, Peter Zijlstra, Rob Herring, Sebastian Reichel,
Steven Rostedt, Thomas Gleixner, Waiman Long, Will Deacon,
Alex Michon, Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jean-Christophe Pince,
Jonathan Borne, Jules Maselbas, Julian Vetter, Julien Hascoet,
Julien Villette, Louis Morhet, Luc Michel, Marc Poulhiès,
Marius Gligor, Samuel Jones, Thomas Costis, Vincent Chardon
On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
> This patch series adds support for the kv3-1 CPU architecture of the kvx family
> found in the Coolidge (aka MPPA3-80) SoC of Kalray.
>
> This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> therefore this patch series cannot be merged into Linux for now.
>
> The goal is to have preliminary reviews and to fix problems early.
>
> The Kalray VLIW processor family (kvx) has the following features:
> * 32/64 bits execution mode
> * 6-issue VLIW architecture
> * 64 x 64bits general purpose registers
> * SIMD instructions
> * little-endian
> * deep learning co-processor
Thanks for posting these, I had been wondering about the
state of the port. Overall this looks really nice, I can
see that you and the team have looked at other ports
and generally made the right decisions.
I commented on the syscall patch directly, I think it's
important to stop using the deprecated syscalls as soon
as possible to avoid having dependencies in too many
libc binaries. Almost everything else can be changed
easily as you get closer to upstream inclusion.
I did not receive most of the other patches as I'm
not subscribed to all the mainline lists. For future
submissions, can you add the linux-arch list to Cc for
all patches?
Reading the rest of the series through lore.kernel.org,
most of the comments I have are for improvements that
you may find valuable rather than serious mistakes:
- the {copy_to,copy_from,clear}_user functions are
well worth optimizing better than the byte-at-a-time
version you have, even just a C version built around
your __get_user/__put_user inline asm should help, and
could be added to lib/usercopy.c.
- The __raw_{read,write}{b,w,l,q} helpers should
normally be defined as inline asm instead of
volatile pointer dereferences, I've seen cases where
the compiler ends up splitting the access or does
other things you may not want on MMIO areas.
- I would recomment implementing HAVE_ARCH_VMAP_STACK
as well as IRQ stacks, both of these help to
avoid data corruption from stack overflow that you
will eventually run into.
- You use qspinlock as the only available spinlock
implementation, but only support running on a
single cluster of 16 cores. It may help to use
the generic ticket spinlock instead, or leave it
as a Kconfig option, in particular since you only
have the emulated xchg16() atomic for qspinlock.
- Your defconfig file enables CONFIG_EMBEDDED, which
in turn enables CONFIG_EXPERT. This is probably
not what you want, so better turn off both of these.
- The GENERIC_CALIBRATE_DELAY should not be necessary
since you have a get_cycles() based delay loop.
Just set loops_per_jiffy to the correct value based
on the frequency of the cycle counter, to save
a little time during boot and get a more accurate
delay loop.
Arnd
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-04 15:58 ` Arnd Bergmann
(?)
@ 2023-01-05 10:40 ` Jules Maselbas
-1 siblings, 0 replies; 113+ messages in thread
From: Jules Maselbas @ 2023-01-05 10:40 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Yann Sionneau, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree,
Eric W. Biederman, Eric Paris, Ingo Molnar, Jan Kiszka,
Jason Baron, Jiri Olsa, Jonathan Corbet, Josh Poimboeuf,
Kees Cook, Kieran Bingham, Krzysztof Kozlowski, Linux-Arch,
linux-arm-kernel, linux-audit, linux-doc, linux-kernel, linux-mm,
linux-perf-users, linux-pm, linux-riscv, Marc Zyngier,
Mark Rutland, Masami Hiramatsu, Namhyung Kim, Nicholas Piggin,
Oleg Nesterov, Palmer Dabbelt, Paul Moore, Paul Walmsley,
Peter Zijlstra, Rob Herring, Sebastian Reichel, Steven Rostedt,
Thomas Gleixner, Waiman Long, Will Deacon, Alex Michon,
Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jean-Christophe Pince,
Jonathan Borne, Julian Vetter, Julien Hascoet, Julien Villette,
Louis Morhet, Luc Michel, Marc Poulhiès, Marius Gligor,
Samuel Jones, Thomas Costis, Vincent Chardon
Hi,
On Wed, Jan 04, 2023 at 04:58:25PM +0100, Arnd Bergmann wrote:
> On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
> > This patch series adds support for the kv3-1 CPU architecture of the kvx family
> > found in the Coolidge (aka MPPA3-80) SoC of Kalray.
> >
> > This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> > therefore this patch series cannot be merged into Linux for now.
> >
> > The goal is to have preliminary reviews and to fix problems early.
> >
> > The Kalray VLIW processor family (kvx) has the following features:
> > * 32/64 bits execution mode
> > * 6-issue VLIW architecture
> > * 64 x 64bits general purpose registers
> > * SIMD instructions
> > * little-endian
> > * deep learning co-processor
>
> Thanks for posting these, I had been wondering about the
> state of the port. Overall this looks really nice, I can
> see that you and the team have looked at other ports
> and generally made the right decisions.
Thank you and all for the reviews. We are currently going
through every remarks and we are trying to do our best to
send a new patch series with everything addressed.
> I commented on the syscall patch directly, I think it's
> important to stop using the deprecated syscalls as soon
> as possible to avoid having dependencies in too many
> libc binaries. Almost everything else can be changed
> easily as you get closer to upstream inclusion.
>
> I did not receive most of the other patches as I'm
> not subscribed to all the mainline lists. For future
> submissions, can you add the linux-arch list to Cc for
> all patches?
We misused get_maintainers.pl, running it on each patch instead
of using it on the whole series. next time every one will be in
copy of every patch in the series and including linux-arch.
> Reading the rest of the series through lore.kernel.org,
> most of the comments I have are for improvements that
> you may find valuable rather than serious mistakes:
>
> - the {copy_to,copy_from,clear}_user functions are
> well worth optimizing better than the byte-at-a-time
> version you have, even just a C version built around
> your __get_user/__put_user inline asm should help, and
> could be added to lib/usercopy.c.
right, we are using memcpy for {copy_to,copy_from}_user_page
which has a simple optimized version introduced in
(kvx: Add some library functions).
I wonder if it is possible to do the same for copy_*_user functions.
> - The __raw_{read,write}{b,w,l,q} helpers should
> normally be defined as inline asm instead of
> volatile pointer dereferences, I've seen cases where
> the compiler ends up splitting the access or does
> other things you may not want on MMIO areas.
>
> - I would recomment implementing HAVE_ARCH_VMAP_STACK
> as well as IRQ stacks, both of these help to
> avoid data corruption from stack overflow that you
> will eventually run into.
>
> - You use qspinlock as the only available spinlock
> implementation, but only support running on a
> single cluster of 16 cores. It may help to use
> the generic ticket spinlock instead, or leave it
> as a Kconfig option, in particular since you only
> have the emulated xchg16() atomic for qspinlock.
>
> - Your defconfig file enables CONFIG_EMBEDDED, which
> in turn enables CONFIG_EXPERT. This is probably
> not what you want, so better turn off both of these.
>
> - The GENERIC_CALIBRATE_DELAY should not be necessary
> since you have a get_cycles() based delay loop.
> Just set loops_per_jiffy to the correct value based
> on the frequency of the cycle counter, to save
> a little time during boot and get a more accurate
> delay loop.
>
Ack !
Jules
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-05 10:40 ` Jules Maselbas
0 siblings, 0 replies; 113+ messages in thread
From: Jules Maselbas @ 2023-01-05 10:40 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Yann Sionneau, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree,
Eric W. Biederman, Eric Paris, Ingo Molnar, Jan Kiszka,
Jason Baron, Jiri Olsa, Jonathan Corbet, Josh Poimboeuf,
Kees Cook, Kieran Bingham, Krzysztof Kozlowski, Linux-Arch,
linux-arm-kernel, linux-audit, linux-doc, linux-kernel, linux-mm,
linux-perf-users, linux-pm, linux-riscv, Marc Zyngier,
Mark Rutland, Masami Hiramatsu, Namhyung Kim, Nicholas Piggin,
Oleg Nesterov, Palmer Dabbelt, Paul Moore, Paul Walmsley,
Peter Zijlstra, Rob Herring, Sebastian Reichel, Steven Rostedt,
Thomas Gleixner, Waiman Long, Will Deacon, Alex Michon,
Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jean-Christophe Pince,
Jonathan Borne, Julian Vetter, Julien Hascoet, Julien Villette,
Louis Morhet, Luc Michel, Marc Poulhiès, Marius Gligor,
Samuel Jones, Thomas Costis, Vincent Chardon
Hi,
On Wed, Jan 04, 2023 at 04:58:25PM +0100, Arnd Bergmann wrote:
> On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
> > This patch series adds support for the kv3-1 CPU architecture of the kvx family
> > found in the Coolidge (aka MPPA3-80) SoC of Kalray.
> >
> > This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> > therefore this patch series cannot be merged into Linux for now.
> >
> > The goal is to have preliminary reviews and to fix problems early.
> >
> > The Kalray VLIW processor family (kvx) has the following features:
> > * 32/64 bits execution mode
> > * 6-issue VLIW architecture
> > * 64 x 64bits general purpose registers
> > * SIMD instructions
> > * little-endian
> > * deep learning co-processor
>
> Thanks for posting these, I had been wondering about the
> state of the port. Overall this looks really nice, I can
> see that you and the team have looked at other ports
> and generally made the right decisions.
Thank you and all for the reviews. We are currently going
through every remarks and we are trying to do our best to
send a new patch series with everything addressed.
> I commented on the syscall patch directly, I think it's
> important to stop using the deprecated syscalls as soon
> as possible to avoid having dependencies in too many
> libc binaries. Almost everything else can be changed
> easily as you get closer to upstream inclusion.
>
> I did not receive most of the other patches as I'm
> not subscribed to all the mainline lists. For future
> submissions, can you add the linux-arch list to Cc for
> all patches?
We misused get_maintainers.pl, running it on each patch instead
of using it on the whole series. next time every one will be in
copy of every patch in the series and including linux-arch.
> Reading the rest of the series through lore.kernel.org,
> most of the comments I have are for improvements that
> you may find valuable rather than serious mistakes:
>
> - the {copy_to,copy_from,clear}_user functions are
> well worth optimizing better than the byte-at-a-time
> version you have, even just a C version built around
> your __get_user/__put_user inline asm should help, and
> could be added to lib/usercopy.c.
right, we are using memcpy for {copy_to,copy_from}_user_page
which has a simple optimized version introduced in
(kvx: Add some library functions).
I wonder if it is possible to do the same for copy_*_user functions.
> - The __raw_{read,write}{b,w,l,q} helpers should
> normally be defined as inline asm instead of
> volatile pointer dereferences, I've seen cases where
> the compiler ends up splitting the access or does
> other things you may not want on MMIO areas.
>
> - I would recomment implementing HAVE_ARCH_VMAP_STACK
> as well as IRQ stacks, both of these help to
> avoid data corruption from stack overflow that you
> will eventually run into.
>
> - You use qspinlock as the only available spinlock
> implementation, but only support running on a
> single cluster of 16 cores. It may help to use
> the generic ticket spinlock instead, or leave it
> as a Kconfig option, in particular since you only
> have the emulated xchg16() atomic for qspinlock.
>
> - Your defconfig file enables CONFIG_EMBEDDED, which
> in turn enables CONFIG_EXPERT. This is probably
> not what you want, so better turn off both of these.
>
> - The GENERIC_CALIBRATE_DELAY should not be necessary
> since you have a get_cycles() based delay loop.
> Just set loops_per_jiffy to the correct value based
> on the frequency of the cycle counter, to save
> a little time during boot and get a more accurate
> delay loop.
>
Ack !
Jules
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-05 10:40 ` Jules Maselbas
0 siblings, 0 replies; 113+ messages in thread
From: Jules Maselbas @ 2023-01-05 10:40 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Mark Rutland, Marc Poulhiès, linux-doc, Alexander Shishkin,
Jan Kiszka, Sebastian Reichel, Marius Gligor, Oleg Nesterov,
linux-mm, Louis Morhet, Krzysztof Kozlowski, Luc Michel,
linux-riscv, Ashley Lesdalons, Will Deacon, Ard Biesheuvel,
Linux-Arch, Marc Zyngier, Thomas Costis, Jonathan Corbet,
Jonathan Borne, Aneesh Kumar, Samuel Jones, Peter Zijlstra,
Ingo Molnar, Steven Rostedt, Jean-Christophe Pince, Waiman Long,
Clement Leger, Masami Hiramatsu, devicetree, Albert Ou,
Kees Cook, linux-pm, Yann Sionneau, Boqun Feng,
Guillaume Thouvenin, Julian Vetter, Arnaldo Carvalho de Melo,
Julien Hascoet, Jason Baron, Rob Herring, Nicholas Piggin,
Paul Walmsley, Namhyung Kim, Thomas Gleixner, Josh Poimboeuf,
linux-arm-kernel, Julien Villette, Christian Brauner,
Andrew Morton, Benjamin Mugnier, Vincent Chardon,
Guillaume Missonnier, linux-kernel, Eric Paris, Alex Michon,
linux-perf-users, linux-audit, Palmer Dabbelt, Eric W. Biederman,
Jiri Olsa, Kieran Bingham, bpf
Hi,
On Wed, Jan 04, 2023 at 04:58:25PM +0100, Arnd Bergmann wrote:
> On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
> > This patch series adds support for the kv3-1 CPU architecture of the kvx family
> > found in the Coolidge (aka MPPA3-80) SoC of Kalray.
> >
> > This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> > therefore this patch series cannot be merged into Linux for now.
> >
> > The goal is to have preliminary reviews and to fix problems early.
> >
> > The Kalray VLIW processor family (kvx) has the following features:
> > * 32/64 bits execution mode
> > * 6-issue VLIW architecture
> > * 64 x 64bits general purpose registers
> > * SIMD instructions
> > * little-endian
> > * deep learning co-processor
>
> Thanks for posting these, I had been wondering about the
> state of the port. Overall this looks really nice, I can
> see that you and the team have looked at other ports
> and generally made the right decisions.
Thank you and all for the reviews. We are currently going
through every remarks and we are trying to do our best to
send a new patch series with everything addressed.
> I commented on the syscall patch directly, I think it's
> important to stop using the deprecated syscalls as soon
> as possible to avoid having dependencies in too many
> libc binaries. Almost everything else can be changed
> easily as you get closer to upstream inclusion.
>
> I did not receive most of the other patches as I'm
> not subscribed to all the mainline lists. For future
> submissions, can you add the linux-arch list to Cc for
> all patches?
We misused get_maintainers.pl, running it on each patch instead
of using it on the whole series. next time every one will be in
copy of every patch in the series and including linux-arch.
> Reading the rest of the series through lore.kernel.org,
> most of the comments I have are for improvements that
> you may find valuable rather than serious mistakes:
>
> - the {copy_to,copy_from,clear}_user functions are
> well worth optimizing better than the byte-at-a-time
> version you have, even just a C version built around
> your __get_user/__put_user inline asm should help, and
> could be added to lib/usercopy.c.
right, we are using memcpy for {copy_to,copy_from}_user_page
which has a simple optimized version introduced in
(kvx: Add some library functions).
I wonder if it is possible to do the same for copy_*_user functions.
> - The __raw_{read,write}{b,w,l,q} helpers should
> normally be defined as inline asm instead of
> volatile pointer dereferences, I've seen cases where
> the compiler ends up splitting the access or does
> other things you may not want on MMIO areas.
>
> - I would recomment implementing HAVE_ARCH_VMAP_STACK
> as well as IRQ stacks, both of these help to
> avoid data corruption from stack overflow that you
> will eventually run into.
>
> - You use qspinlock as the only available spinlock
> implementation, but only support running on a
> single cluster of 16 cores. It may help to use
> the generic ticket spinlock instead, or leave it
> as a Kconfig option, in particular since you only
> have the emulated xchg16() atomic for qspinlock.
>
> - Your defconfig file enables CONFIG_EMBEDDED, which
> in turn enables CONFIG_EXPERT. This is probably
> not what you want, so better turn off both of these.
>
> - The GENERIC_CALIBRATE_DELAY should not be necessary
> since you have a get_cycles() based delay loop.
> Just set loops_per_jiffy to the correct value based
> on the frequency of the cycle counter, to save
> a little time during boot and get a more accurate
> delay loop.
>
Ack !
Jules
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-05 10:40 ` Jules Maselbas
(?)
@ 2023-01-05 12:05 ` Arnd Bergmann
-1 siblings, 0 replies; 113+ messages in thread
From: Arnd Bergmann @ 2023-01-05 12:05 UTC (permalink / raw)
To: Jules Maselbas
Cc: Yann Sionneau, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree,
Eric W. Biederman, Eric Paris, Ingo Molnar, Jan Kiszka,
Jason Baron, Jiri Olsa, Jonathan Corbet, Josh Poimboeuf,
Kees Cook, Kieran Bingham, Krzysztof Kozlowski, Linux-Arch,
linux-arm-kernel, linux-audit, linux-doc, linux-kernel, linux-mm,
linux-perf-users, linux-pm, linux-riscv, Marc Zyngier,
Mark Rutland, Masami Hiramatsu, Namhyung Kim, Nicholas Piggin,
Oleg Nesterov, Palmer Dabbelt, Paul Moore, Paul Walmsley,
Peter Zijlstra, Rob Herring, Sebastian Reichel, Steven Rostedt,
Thomas Gleixner, Waiman Long, Will Deacon, Alex Michon,
Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jean-Christophe Pince,
Jonathan Borne, Julian Vetter, Julien Hascoet, Julien Villette,
Louis Morhet, Luc Michel, Marc Poulhiès, Marius Gligor,
Samuel Jones, Thomas Costis, Vincent Chardon
On Thu, Jan 5, 2023, at 11:40, Jules Maselbas wrote:
> On Wed, Jan 04, 2023 at 04:58:25PM +0100, Arnd Bergmann wrote:
>> On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
>> I commented on the syscall patch directly, I think it's
>> important to stop using the deprecated syscalls as soon
>> as possible to avoid having dependencies in too many
>> libc binaries. Almost everything else can be changed
>> easily as you get closer to upstream inclusion.
>>
>> I did not receive most of the other patches as I'm
>> not subscribed to all the mainline lists. For future
>> submissions, can you add the linux-arch list to Cc for
>> all patches?
>
> We misused get_maintainers.pl, running it on each patch instead
> of using it on the whole series. next time every one will be in
> copy of every patch in the series and including linux-arch.
Be careful not to make the list too long though, there is
usually a limit of 1024 characters for the entire Cc list,
above this your mails may get dropped by the mailing lists.
>> Reading the rest of the series through lore.kernel.org,
>> most of the comments I have are for improvements that
>> you may find valuable rather than serious mistakes:
>>
>> - the {copy_to,copy_from,clear}_user functions are
>> well worth optimizing better than the byte-at-a-time
>> version you have, even just a C version built around
>> your __get_user/__put_user inline asm should help, and
>> could be added to lib/usercopy.c.
>
> right, we are using memcpy for {copy_to,copy_from}_user_page
> which has a simple optimized version introduced in
> (kvx: Add some library functions).
> I wonder if it is possible to do the same for copy_*_user functions.
copy_from_user_page() is only about managing cache aliases,
not user access, and it's not used anywhere in the fast
path, so it's a bit different.
arch/arm/lib/copy_template.S has an example for how
you can share the same assembler implementation between
memcpy() and copy_from_user(), but it's not very
intuitive.
The tricky bit with copy_from_user() is the partial copy
that relies on copying the exact amount of data that can
be accessed including the last byte before the end of
the mapping, and returning the correct count of non-copied
bytes.
If you want a C version, you can probably use the
copy_from_kernel_nofault implementation mm/maccess.c
as a template, but have to add code for the correct
return value.
Arnd
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-05 12:05 ` Arnd Bergmann
0 siblings, 0 replies; 113+ messages in thread
From: Arnd Bergmann @ 2023-01-05 12:05 UTC (permalink / raw)
To: Jules Maselbas
Cc: Yann Sionneau, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree,
Eric W. Biederman, Eric Paris, Ingo Molnar, Jan Kiszka,
Jason Baron, Jiri Olsa, Jonathan Corbet, Josh Poimboeuf,
Kees Cook, Kieran Bingham, Krzysztof Kozlowski, Linux-Arch,
linux-arm-kernel, linux-audit, linux-doc, linux-kernel, linux-mm,
linux-perf-users, linux-pm, linux-riscv, Marc Zyngier,
Mark Rutland, Masami Hiramatsu, Namhyung Kim, Nicholas Piggin,
Oleg Nesterov, Palmer Dabbelt, Paul Moore, Paul Walmsley,
Peter Zijlstra, Rob Herring, Sebastian Reichel, Steven Rostedt,
Thomas Gleixner, Waiman Long, Will Deacon, Alex Michon,
Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jean-Christophe Pince,
Jonathan Borne, Julian Vetter, Julien Hascoet, Julien Villette,
Louis Morhet, Luc Michel, Marc Poulhiès, Marius Gligor,
Samuel Jones, Thomas Costis, Vincent Chardon
On Thu, Jan 5, 2023, at 11:40, Jules Maselbas wrote:
> On Wed, Jan 04, 2023 at 04:58:25PM +0100, Arnd Bergmann wrote:
>> On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
>> I commented on the syscall patch directly, I think it's
>> important to stop using the deprecated syscalls as soon
>> as possible to avoid having dependencies in too many
>> libc binaries. Almost everything else can be changed
>> easily as you get closer to upstream inclusion.
>>
>> I did not receive most of the other patches as I'm
>> not subscribed to all the mainline lists. For future
>> submissions, can you add the linux-arch list to Cc for
>> all patches?
>
> We misused get_maintainers.pl, running it on each patch instead
> of using it on the whole series. next time every one will be in
> copy of every patch in the series and including linux-arch.
Be careful not to make the list too long though, there is
usually a limit of 1024 characters for the entire Cc list,
above this your mails may get dropped by the mailing lists.
>> Reading the rest of the series through lore.kernel.org,
>> most of the comments I have are for improvements that
>> you may find valuable rather than serious mistakes:
>>
>> - the {copy_to,copy_from,clear}_user functions are
>> well worth optimizing better than the byte-at-a-time
>> version you have, even just a C version built around
>> your __get_user/__put_user inline asm should help, and
>> could be added to lib/usercopy.c.
>
> right, we are using memcpy for {copy_to,copy_from}_user_page
> which has a simple optimized version introduced in
> (kvx: Add some library functions).
> I wonder if it is possible to do the same for copy_*_user functions.
copy_from_user_page() is only about managing cache aliases,
not user access, and it's not used anywhere in the fast
path, so it's a bit different.
arch/arm/lib/copy_template.S has an example for how
you can share the same assembler implementation between
memcpy() and copy_from_user(), but it's not very
intuitive.
The tricky bit with copy_from_user() is the partial copy
that relies on copying the exact amount of data that can
be accessed including the last byte before the end of
the mapping, and returning the correct count of non-copied
bytes.
If you want a C version, you can probably use the
copy_from_kernel_nofault implementation mm/maccess.c
as a template, but have to add code for the correct
return value.
Arnd
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-05 12:05 ` Arnd Bergmann
0 siblings, 0 replies; 113+ messages in thread
From: Arnd Bergmann @ 2023-01-05 12:05 UTC (permalink / raw)
To: Jules Maselbas
Cc: Mark Rutland, Marc Poulhiès, linux-doc, Alexander Shishkin,
Jan Kiszka, Sebastian Reichel, Marius Gligor, Oleg Nesterov,
linux-mm, Louis Morhet, Krzysztof Kozlowski, Luc Michel,
linux-riscv, Ashley Lesdalons, Will Deacon, Ard Biesheuvel,
Linux-Arch, Marc Zyngier, Thomas Costis, Jonathan Corbet,
Jonathan Borne, Aneesh Kumar, Samuel Jones, Peter Zijlstra,
Ingo Molnar, Steven Rostedt, Jean-Christophe Pince, Waiman Long,
Clement Leger, Masami Hiramatsu, devicetree, Albert Ou,
Kees Cook, linux-pm, Yann Sionneau, Boqun Feng,
Guillaume Thouvenin, Julian Vetter, Arnaldo Carvalho de Melo,
Julien Hascoet, Jason Baron, Rob Herring, Nicholas Piggin,
Paul Walmsley, Namhyung Kim, Thomas Gleixner, Josh Poimboeuf,
linux-arm-kernel, Julien Villette, Christian Brauner,
Andrew Morton, Benjamin Mugnier, Vincent Chardon,
Guillaume Missonnier, linux-kernel, Eric Paris, Alex Michon,
linux-perf-users, linux-audit, Palmer Dabbelt, Eric W. Biederman,
Jiri Olsa, Kieran Bingham, bpf
On Thu, Jan 5, 2023, at 11:40, Jules Maselbas wrote:
> On Wed, Jan 04, 2023 at 04:58:25PM +0100, Arnd Bergmann wrote:
>> On Tue, Jan 3, 2023, at 17:43, Yann Sionneau wrote:
>> I commented on the syscall patch directly, I think it's
>> important to stop using the deprecated syscalls as soon
>> as possible to avoid having dependencies in too many
>> libc binaries. Almost everything else can be changed
>> easily as you get closer to upstream inclusion.
>>
>> I did not receive most of the other patches as I'm
>> not subscribed to all the mainline lists. For future
>> submissions, can you add the linux-arch list to Cc for
>> all patches?
>
> We misused get_maintainers.pl, running it on each patch instead
> of using it on the whole series. next time every one will be in
> copy of every patch in the series and including linux-arch.
Be careful not to make the list too long though, there is
usually a limit of 1024 characters for the entire Cc list,
above this your mails may get dropped by the mailing lists.
>> Reading the rest of the series through lore.kernel.org,
>> most of the comments I have are for improvements that
>> you may find valuable rather than serious mistakes:
>>
>> - the {copy_to,copy_from,clear}_user functions are
>> well worth optimizing better than the byte-at-a-time
>> version you have, even just a C version built around
>> your __get_user/__put_user inline asm should help, and
>> could be added to lib/usercopy.c.
>
> right, we are using memcpy for {copy_to,copy_from}_user_page
> which has a simple optimized version introduced in
> (kvx: Add some library functions).
> I wonder if it is possible to do the same for copy_*_user functions.
copy_from_user_page() is only about managing cache aliases,
not user access, and it's not used anywhere in the fast
path, so it's a bit different.
arch/arm/lib/copy_template.S has an example for how
you can share the same assembler implementation between
memcpy() and copy_from_user(), but it's not very
intuitive.
The tricky bit with copy_from_user() is the partial copy
that relies on copying the exact amount of data that can
be accessed including the last byte before the end of
the mapping, and returning the correct count of non-copied
bytes.
If you want a C version, you can probably use the
copy_from_kernel_nofault implementation mm/maccess.c
as a template, but have to add code for the correct
return value.
Arnd
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-05 12:05 ` Arnd Bergmann
(?)
@ 2023-01-05 14:12 ` Steven Rostedt
-1 siblings, 0 replies; 113+ messages in thread
From: Steven Rostedt @ 2023-01-05 14:12 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Jules Maselbas, Yann Sionneau, Albert Ou, Alexander Shishkin,
Andrew Morton, Aneesh Kumar, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Boqun Feng, bpf, Christian Brauner,
devicetree, Eric W. Biederman, Eric Paris, Ingo Molnar,
Jan Kiszka, Jason Baron, Jiri Olsa, Jonathan Corbet,
Josh Poimboeuf, Kees Cook, Kieran Bingham, Krzysztof Kozlowski,
Linux-Arch, linux-arm-kernel, linux-audit, linux-doc,
linux-kernel, linux-mm, linux-perf-users, linux-pm, linux-riscv,
Marc Zyngier, Mark Rutland, Masami Hiramatsu, Namhyung Kim,
Nicholas Piggin, Oleg Nesterov, Palmer Dabbelt, Paul Moore,
Paul Walmsley, Peter Zijlstra, Rob Herring, Sebastian Reichel,
Thomas Gleixner, Waiman Long, Will Deacon, Alex Michon,
Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jean-Christophe Pince,
Jonathan Borne, Julian Vetter, Julien Hascoet, Julien Villette,
Louis Morhet, Luc Michel, Marc Poulhiès, Marius Gligor,
Samuel Jones, Thomas Costis, Vincent Chardon
On Thu, 05 Jan 2023 13:05:32 +0100
"Arnd Bergmann" <arnd@arndb.de> wrote:
> >> I did not receive most of the other patches as I'm
> >> not subscribed to all the mainline lists. For future
> >> submissions, can you add the linux-arch list to Cc for
> >> all patches?
> >
> > We misused get_maintainers.pl, running it on each patch instead
> > of using it on the whole series. next time every one will be in
> > copy of every patch in the series and including linux-arch.
>
> Be careful not to make the list too long though, there is
> usually a limit of 1024 characters for the entire Cc list,
> above this your mails may get dropped by the mailing lists.
It's best to include mailing lists for the entire series, and perhaps
individuals for each patch. As I don't want the entire series just to see
the tracing portion.
-- Steve
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-05 14:12 ` Steven Rostedt
0 siblings, 0 replies; 113+ messages in thread
From: Steven Rostedt @ 2023-01-05 14:12 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Mark Rutland, Marc Poulhiès, linux-doc, Alexander Shishkin,
Jan Kiszka, Sebastian Reichel, Marius Gligor, Oleg Nesterov,
linux-mm, Louis Morhet, Krzysztof Kozlowski, Luc Michel,
linux-riscv, Ashley Lesdalons, Will Deacon, Ard Biesheuvel,
Linux-Arch, Marc Zyngier, Thomas Costis, Jonathan Corbet,
Jonathan Borne, Aneesh Kumar, Samuel Jones, Peter Zijlstra,
Ingo Molnar, Jean-Christophe Pince, Waiman Long, Clement Leger,
Masami Hiramatsu, devicetree, Albert Ou, Kees Cook, linux-pm,
Yann Sionneau, Boqun Feng, Guillaume Thouvenin, Julian Vetter,
Arnaldo Carvalho de Melo, Julien Hascoet, Jason Baron,
Rob Herring, Nicholas Piggin, Paul Walmsley, Namhyung Kim,
Thomas Gleixner, Josh Poimboeuf, linux-arm-kernel,
Julien Villette, Jules Maselbas, Christian Brauner,
Andrew Morton, Benjamin Mugnier, Vincent Chardon,
Guillaume Missonnier, linux-kernel, Eric Paris, Alex Michon,
linux-perf-users, linux-audit, Palmer Dabbelt, Eric W. Biederman,
Jiri Olsa, Kieran Bingham, bpf
On Thu, 05 Jan 2023 13:05:32 +0100
"Arnd Bergmann" <arnd@arndb.de> wrote:
> >> I did not receive most of the other patches as I'm
> >> not subscribed to all the mainline lists. For future
> >> submissions, can you add the linux-arch list to Cc for
> >> all patches?
> >
> > We misused get_maintainers.pl, running it on each patch instead
> > of using it on the whole series. next time every one will be in
> > copy of every patch in the series and including linux-arch.
>
> Be careful not to make the list too long though, there is
> usually a limit of 1024 characters for the entire Cc list,
> above this your mails may get dropped by the mailing lists.
It's best to include mailing lists for the entire series, and perhaps
individuals for each patch. As I don't want the entire series just to see
the tracing portion.
-- Steve
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-05 14:12 ` Steven Rostedt
0 siblings, 0 replies; 113+ messages in thread
From: Steven Rostedt @ 2023-01-05 14:12 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Jules Maselbas, Yann Sionneau, Albert Ou, Alexander Shishkin,
Andrew Morton, Aneesh Kumar, Ard Biesheuvel,
Arnaldo Carvalho de Melo, Boqun Feng, bpf, Christian Brauner,
devicetree, Eric W. Biederman, Eric Paris, Ingo Molnar,
Jan Kiszka, Jason Baron, Jiri Olsa, Jonathan Corbet,
Josh Poimboeuf, Kees Cook, Kieran Bingham, Krzysztof Kozlowski,
Linux-Arch, linux-arm-kernel, linux-audit, linux-doc,
linux-kernel, linux-mm, linux-perf-users, linux-pm, linux-riscv,
Marc Zyngier, Mark Rutland, Masami Hiramatsu, Namhyung Kim,
Nicholas Piggin, Oleg Nesterov, Palmer Dabbelt, Paul Moore,
Paul Walmsley, Peter Zijlstra, Rob Herring, Sebastian Reichel,
Thomas Gleixner, Waiman Long, Will Deacon, Alex Michon,
Ashley Lesdalons, Benjamin Mugnier, Clement Leger,
Guillaume Missonnier, Guillaume Thouvenin, Jean-Christophe Pince,
Jonathan Borne, Julian Vetter, Julien Hascoet, Julien Villette,
Louis Morhet, Luc Michel, Marc Poulhiès, Marius Gligor,
Samuel Jones, Thomas Costis, Vincent Chardon
On Thu, 05 Jan 2023 13:05:32 +0100
"Arnd Bergmann" <arnd@arndb.de> wrote:
> >> I did not receive most of the other patches as I'm
> >> not subscribed to all the mainline lists. For future
> >> submissions, can you add the linux-arch list to Cc for
> >> all patches?
> >
> > We misused get_maintainers.pl, running it on each patch instead
> > of using it on the whole series. next time every one will be in
> > copy of every patch in the series and including linux-arch.
>
> Be careful not to make the list too long though, there is
> usually a limit of 1024 characters for the entire Cc list,
> above this your mails may get dropped by the mailing lists.
It's best to include mailing lists for the entire series, and perhaps
individuals for each patch. As I don't want the entire series just to see
the tracing portion.
-- Steve
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-03 16:43 ` Yann Sionneau
(?)
@ 2023-01-07 6:25 ` Jeff Xie
-1 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-07 6:25 UTC (permalink / raw)
To: Yann Sionneau
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
Hi,
On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
>
> This patch series adds support for the kv3-1 CPU architecture of the kvx family
> found in the Coolidge (aka MPPA3-80) SoC of Kalray.
>
> This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> therefore this patch series cannot be merged into Linux for now.
>
> The goal is to have preliminary reviews and to fix problems early.
>
> The Kalray VLIW processor family (kvx) has the following features:
> * 32/64 bits execution mode
> * 6-issue VLIW architecture
> * 64 x 64bits general purpose registers
> * SIMD instructions
> * little-endian
> * deep learning co-processor
>
> Kalray kv3-1 core which is the third of the kvx family is embedded in Kalray
> Coolidge SoC currently used on K200 and K200-LP boards.
>
> The Coolidge SoC contains 5 clusters each of which is made of:
> * 4MiB of on-chip memory (SMEM)
> * 1 dedicated safety/security core (kv3-1 core).
> * 16 PEs (Processing Elements) (kv3-1 cores).
> * 16 Co-processors (one per PE)
> * 2 Crypto accelerators
>
> The Coolidge SoC contains the following features:
> * 5 Clusters
> * 2 100G Ethernet controllers
> * 8 PCIe GEN4 controllers (Root Complex and Endpoint capable)
> * 2 USB 2.0 controllers
> * 1 Octal SPI-NOR flash controller
> * 1 eMMC controller
> * 3 Quad SPI controllers
> * 6 UART
> * 5 I2C controllers (3 of which are SMBus capable)
> * 4 CAN controllers
> * 1 OTP memory
>
> A kvx toolchain can be built using:
> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> $ git clone https://github.com/kalray/build-scripts
> $ cd build-scripts
> $ source last.refs
> $ ./build-kvx-xgcc.sh output
I would like to build the kvx-xgcc to compile and test the linux
kernel, but it reported a compile error.
I wonder what version of gcc you are using.
My build environment:
VERSION="20.04.2 LTS (Focal Fossa)"
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
Compile error:
$ ./build-kvx-xgcc.sh output
../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
(first use in this function)
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:220:30: note: in expansion of macro
‘FIBHEAPKEY_MIN’
220 | if (okey == key && okey != FIBHEAPKEY_MIN)
| ^~~~~~~~~~~~~~
../../binutils/libiberty/fibheap.c:36:1: note: ‘LONG_MIN’ is defined
in header ‘<limits.h>’; did you forget to ‘#include <limits.h>’?
35 | #include "fibheap.h"
+++ |+#include <limits.h>
36 |
../../binutils/libiberty/fibheap.c:38:24: note: each undeclared
identifier is reported only once for each function it appears in
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:220:30: note: in expansion of macro
‘FIBHEAPKEY_MIN’
220 | if (okey == key && okey != FIBHEAPKEY_MIN)
| ^~~~~~~~~~~~~~
../../binutils/libiberty/fibheap.c: In function ‘fibheap_delete_node’:
../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
(first use in this function)
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:261:36: note: in expansion of macro
‘FIBHEAPKEY_MIN’
261 | fibheap_replace_key (heap, node, FIBHEAPKEY_MIN);
| ^~~~~~~~~~~~~~
../../binutils/libiberty/fibheap.c:38:24: note: ‘LONG_MIN’ is defined
in header ‘<limits.h>’; did you forget to ‘#include <limits.h>’?
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:261:36: note: in expansion of macro
‘FIBHEAPKEY_MIN’
261 | fibheap_replace_key (heap, node, FIBHEAPKEY_MIN);
| ^~~~~~~~~~~~~~
> The kvx toolchain will be installed in the "output" directory.
>
> A buildroot image (kernel+rootfs) and toolchain can be built using:
> $ git clone -b coolidge-for-upstream https://github.com/kalray/buildroot
> $ cd buildroot
> $ make O=build_kvx kvx_defconfig
> $ make O=build_kvx
>
> The vmlinux image can be found in buildroot/build_kvx/images/vmlinux.
>
> If you are just interested in building the Linux kernel with no rootfs you can
> just do this with the kvx-elf- toolchain:
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- default_defconfig
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- -j$(($(nproc) + 1))
>
> The vmlinux ELF can be run with qemu by doing:
> # install dependencies: ninja pkg-config libglib-2.0-dev cmake libfdt-dev libpixman-1-dev zlib1g-dev
> $ git clone https://github.com/kalray/qemu-builder
> $ cd qemu-builder
> $ git submodule update --init
> $ make -j$(($(nproc) + 1))
> $ ./qemu-system-kvx -m 1024 -nographic -kernel <path/to/vmlinux>
>
> Yann Sionneau (25):
> Documentation: kvx: Add basic documentation
> kvx: Add ELF-related definitions
> kvx: Add build infrastructure
> kvx: Add CPU definition headers
> kvx: Add atomic/locking headers
> kvx: Add other common headers
> kvx: Add boot and setup routines
> kvx: Add exception/interrupt handling
> kvx: irqchip: Add support for irq controllers
> kvx: Add process management
> kvx: Add memory management
> kvx: Add system call support
> kvx: Add signal handling support
> kvx: Add ELF relocations and module support
> kvx: Add misc common routines
> kvx: Add some library functions
> kvx: Add multi-processor (SMP) support
> kvx: Add kvx default config file
> kvx: power: scall poweroff driver
> kvx: gdb: add kvx related gdb helpers
> kvx: Add support for ftrace
> kvx: Add support for jump labels
> kvx: Add debugging related support
> kvx: Add support for CPU Perf Monitors
> kvx: Add support for cpuinfo
>
> .../kalray,kvx-core-intc.txt | 22 +
> .../devicetree/bindings/perf/kalray-pm.txt | 21 +
> Documentation/kvx/kvx-exceptions.txt | 246 +
> Documentation/kvx/kvx-iommu.txt | 183 +
> Documentation/kvx/kvx-mmu.txt | 272 +
> Documentation/kvx/kvx-smp.txt | 36 +
> Documentation/kvx/kvx.txt | 268 +
> arch/kvx/Kconfig | 249 +
> arch/kvx/Kconfig.debug | 70 +
> arch/kvx/Makefile | 52 +
> arch/kvx/configs/default_defconfig | 130 +
> arch/kvx/include/asm/Kbuild | 20 +
> arch/kvx/include/asm/asm-prototypes.h | 14 +
> arch/kvx/include/asm/atomic.h | 104 +
> arch/kvx/include/asm/barrier.h | 15 +
> arch/kvx/include/asm/bitops.h | 207 +
> arch/kvx/include/asm/bitrev.h | 32 +
> arch/kvx/include/asm/break_hook.h | 69 +
> arch/kvx/include/asm/bug.h | 67 +
> arch/kvx/include/asm/cache.h | 46 +
> arch/kvx/include/asm/cacheflush.h | 181 +
> arch/kvx/include/asm/clocksource.h | 17 +
> arch/kvx/include/asm/cmpxchg.h | 185 +
> arch/kvx/include/asm/current.h | 22 +
> arch/kvx/include/asm/dame.h | 31 +
> arch/kvx/include/asm/debug.h | 35 +
> arch/kvx/include/asm/elf.h | 155 +
> arch/kvx/include/asm/fixmap.h | 47 +
> arch/kvx/include/asm/ftrace.h | 41 +
> arch/kvx/include/asm/futex.h | 141 +
> arch/kvx/include/asm/hardirq.h | 14 +
> arch/kvx/include/asm/hugetlb.h | 36 +
> arch/kvx/include/asm/hw_breakpoint.h | 72 +
> arch/kvx/include/asm/hw_irq.h | 14 +
> arch/kvx/include/asm/insns.h | 16 +
> arch/kvx/include/asm/insns_defs.h | 197 +
> arch/kvx/include/asm/io.h | 34 +
> arch/kvx/include/asm/ipi.h | 16 +
> arch/kvx/include/asm/irqflags.h | 58 +
> arch/kvx/include/asm/jump_label.h | 59 +
> arch/kvx/include/asm/l2_cache.h | 75 +
> arch/kvx/include/asm/l2_cache_defs.h | 64 +
> arch/kvx/include/asm/linkage.h | 13 +
> arch/kvx/include/asm/mem_map.h | 44 +
> arch/kvx/include/asm/mmu.h | 296 +
> arch/kvx/include/asm/mmu_context.h | 156 +
> arch/kvx/include/asm/mmu_stats.h | 38 +
> arch/kvx/include/asm/page.h | 187 +
> arch/kvx/include/asm/page_size.h | 29 +
> arch/kvx/include/asm/pci.h | 36 +
> arch/kvx/include/asm/perf_event.h | 90 +
> arch/kvx/include/asm/pgalloc.h | 101 +
> arch/kvx/include/asm/pgtable-bits.h | 102 +
> arch/kvx/include/asm/pgtable.h | 451 ++
> arch/kvx/include/asm/privilege.h | 211 +
> arch/kvx/include/asm/processor.h | 176 +
> arch/kvx/include/asm/ptrace.h | 217 +
> arch/kvx/include/asm/pwr_ctrl.h | 45 +
> arch/kvx/include/asm/rm_fw.h | 16 +
> arch/kvx/include/asm/sections.h | 18 +
> arch/kvx/include/asm/setup.h | 29 +
> arch/kvx/include/asm/sfr.h | 107 +
> arch/kvx/include/asm/sfr_defs.h | 5028 +++++++++++++++++
> arch/kvx/include/asm/smp.h | 42 +
> arch/kvx/include/asm/sparsemem.h | 15 +
> arch/kvx/include/asm/spinlock.h | 16 +
> arch/kvx/include/asm/spinlock_types.h | 17 +
> arch/kvx/include/asm/stackprotector.h | 47 +
> arch/kvx/include/asm/stacktrace.h | 44 +
> arch/kvx/include/asm/string.h | 20 +
> arch/kvx/include/asm/swab.h | 48 +
> arch/kvx/include/asm/switch_to.h | 21 +
> arch/kvx/include/asm/symbols.h | 16 +
> arch/kvx/include/asm/sys_arch.h | 51 +
> arch/kvx/include/asm/syscall.h | 73 +
> arch/kvx/include/asm/syscalls.h | 21 +
> arch/kvx/include/asm/thread_info.h | 78 +
> arch/kvx/include/asm/timex.h | 20 +
> arch/kvx/include/asm/tlb.h | 24 +
> arch/kvx/include/asm/tlb_defs.h | 131 +
> arch/kvx/include/asm/tlbflush.h | 58 +
> arch/kvx/include/asm/traps.h | 76 +
> arch/kvx/include/asm/types.h | 12 +
> arch/kvx/include/asm/uaccess.h | 324 ++
> arch/kvx/include/asm/unistd.h | 11 +
> arch/kvx/include/asm/vermagic.h | 12 +
> arch/kvx/include/asm/vmalloc.h | 10 +
> arch/kvx/include/uapi/asm/Kbuild | 1 +
> arch/kvx/include/uapi/asm/bitsperlong.h | 14 +
> arch/kvx/include/uapi/asm/byteorder.h | 12 +
> arch/kvx/include/uapi/asm/cachectl.h | 25 +
> arch/kvx/include/uapi/asm/ptrace.h | 114 +
> arch/kvx/include/uapi/asm/sigcontext.h | 16 +
> arch/kvx/include/uapi/asm/unistd.h | 16 +
> arch/kvx/kernel/Makefile | 27 +
> arch/kvx/kernel/asm-offsets.c | 157 +
> arch/kvx/kernel/break_hook.c | 77 +
> arch/kvx/kernel/common.c | 11 +
> arch/kvx/kernel/cpuinfo.c | 96 +
> arch/kvx/kernel/dame_handler.c | 113 +
> arch/kvx/kernel/debug.c | 64 +
> arch/kvx/kernel/entry.S | 1759 ++++++
> arch/kvx/kernel/ftrace.c | 339 ++
> arch/kvx/kernel/head.S | 612 ++
> arch/kvx/kernel/hw_breakpoint.c | 556 ++
> arch/kvx/kernel/insns.c | 146 +
> arch/kvx/kernel/io.c | 96 +
> arch/kvx/kernel/irq.c | 78 +
> arch/kvx/kernel/jump_label.c | 34 +
> arch/kvx/kernel/kvx_ksyms.c | 29 +
> arch/kvx/kernel/l2_cache.c | 448 ++
> arch/kvx/kernel/mcount.S | 340 ++
> arch/kvx/kernel/module.c | 148 +
> arch/kvx/kernel/perf_event.c | 609 ++
> arch/kvx/kernel/process.c | 212 +
> arch/kvx/kernel/prom.c | 24 +
> arch/kvx/kernel/ptrace.c | 461 ++
> arch/kvx/kernel/reset.c | 37 +
> arch/kvx/kernel/return_address.c | 55 +
> arch/kvx/kernel/setup.c | 178 +
> arch/kvx/kernel/signal.c | 266 +
> arch/kvx/kernel/smp.c | 110 +
> arch/kvx/kernel/smpboot.c | 127 +
> arch/kvx/kernel/stacktrace.c | 173 +
> arch/kvx/kernel/sys_kvx.c | 58 +
> arch/kvx/kernel/syscall_table.c | 19 +
> arch/kvx/kernel/time.c | 242 +
> arch/kvx/kernel/traps.c | 243 +
> arch/kvx/kernel/vdso.c | 87 +
> arch/kvx/kernel/vmlinux.lds.S | 173 +
> arch/kvx/lib/Makefile | 6 +
> arch/kvx/lib/clear_page.S | 40 +
> arch/kvx/lib/copy_page.S | 90 +
> arch/kvx/lib/delay.c | 39 +
> arch/kvx/lib/memcpy.c | 70 +
> arch/kvx/lib/memset.S | 351 ++
> arch/kvx/lib/strlen.S | 122 +
> arch/kvx/lib/usercopy.S | 90 +
> arch/kvx/mm/Makefile | 10 +
> arch/kvx/mm/cacheflush.c | 154 +
> arch/kvx/mm/dma-mapping.c | 95 +
> arch/kvx/mm/extable.c | 24 +
> arch/kvx/mm/fault.c | 264 +
> arch/kvx/mm/hugetlbpage.c | 317 ++
> arch/kvx/mm/init.c | 527 ++
> arch/kvx/mm/kernel_rwx.c | 228 +
> arch/kvx/mm/mmap.c | 31 +
> arch/kvx/mm/mmu.c | 204 +
> arch/kvx/mm/mmu_stats.c | 94 +
> arch/kvx/mm/tlb.c | 433 ++
> arch/kvx/platform/Makefile | 7 +
> arch/kvx/platform/ipi.c | 110 +
> arch/kvx/platform/pwr_ctrl.c | 93 +
> drivers/irqchip/Kconfig | 27 +
> drivers/irqchip/Makefile | 4 +
> drivers/irqchip/irq-kvx-apic-gic.c | 349 ++
> drivers/irqchip/irq-kvx-apic-mailbox.c | 465 ++
> drivers/irqchip/irq-kvx-core-intc.c | 82 +
> drivers/irqchip/irq-kvx-itgen.c | 224 +
> drivers/power/reset/kvx-scall-poweroff.c | 53 +
> include/linux/cpuhotplug.h | 2 +
> include/linux/irqchip/irq-kvx-apic-gic.h | 21 +
> include/linux/irqchip/irq-kvx-apic-mailbox.h | 29 +
> include/linux/irqchip/irq-kvx-itgen.h | 24 +
> include/uapi/linux/audit.h | 1 +
> include/uapi/linux/elf-em.h | 1 +
> include/uapi/linux/elf.h | 1 +
> scripts/gdb/arch/Makefile | 11 +
> scripts/gdb/arch/__init__.py | 1 +
> scripts/gdb/arch/kvx/Makefile | 25 +
> scripts/gdb/arch/kvx/__init__.py | 1 +
> scripts/gdb/arch/kvx/constants.py.in | 74 +
> scripts/gdb/arch/kvx/mmu.py | 199 +
> scripts/gdb/arch/kvx/page_table_walk.py | 207 +
> tools/include/uapi/asm/bitsperlong.h | 2 +
> 175 files changed, 25814 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt
> create mode 100644 Documentation/devicetree/bindings/perf/kalray-pm.txt
> create mode 100644 Documentation/kvx/kvx-exceptions.txt
> create mode 100644 Documentation/kvx/kvx-iommu.txt
> create mode 100644 Documentation/kvx/kvx-mmu.txt
> create mode 100644 Documentation/kvx/kvx-smp.txt
> create mode 100644 Documentation/kvx/kvx.txt
> create mode 100644 arch/kvx/Kconfig
> create mode 100644 arch/kvx/Kconfig.debug
> create mode 100644 arch/kvx/Makefile
> create mode 100644 arch/kvx/configs/default_defconfig
> create mode 100644 arch/kvx/include/asm/Kbuild
> create mode 100644 arch/kvx/include/asm/asm-prototypes.h
> create mode 100644 arch/kvx/include/asm/atomic.h
> create mode 100644 arch/kvx/include/asm/barrier.h
> create mode 100644 arch/kvx/include/asm/bitops.h
> create mode 100644 arch/kvx/include/asm/bitrev.h
> create mode 100644 arch/kvx/include/asm/break_hook.h
> create mode 100644 arch/kvx/include/asm/bug.h
> create mode 100644 arch/kvx/include/asm/cache.h
> create mode 100644 arch/kvx/include/asm/cacheflush.h
> create mode 100644 arch/kvx/include/asm/clocksource.h
> create mode 100644 arch/kvx/include/asm/cmpxchg.h
> create mode 100644 arch/kvx/include/asm/current.h
> create mode 100644 arch/kvx/include/asm/dame.h
> create mode 100644 arch/kvx/include/asm/debug.h
> create mode 100644 arch/kvx/include/asm/elf.h
> create mode 100644 arch/kvx/include/asm/fixmap.h
> create mode 100644 arch/kvx/include/asm/ftrace.h
> create mode 100644 arch/kvx/include/asm/futex.h
> create mode 100644 arch/kvx/include/asm/hardirq.h
> create mode 100644 arch/kvx/include/asm/hugetlb.h
> create mode 100644 arch/kvx/include/asm/hw_breakpoint.h
> create mode 100644 arch/kvx/include/asm/hw_irq.h
> create mode 100644 arch/kvx/include/asm/insns.h
> create mode 100644 arch/kvx/include/asm/insns_defs.h
> create mode 100644 arch/kvx/include/asm/io.h
> create mode 100644 arch/kvx/include/asm/ipi.h
> create mode 100644 arch/kvx/include/asm/irqflags.h
> create mode 100644 arch/kvx/include/asm/jump_label.h
> create mode 100644 arch/kvx/include/asm/l2_cache.h
> create mode 100644 arch/kvx/include/asm/l2_cache_defs.h
> create mode 100644 arch/kvx/include/asm/linkage.h
> create mode 100644 arch/kvx/include/asm/mem_map.h
> create mode 100644 arch/kvx/include/asm/mmu.h
> create mode 100644 arch/kvx/include/asm/mmu_context.h
> create mode 100644 arch/kvx/include/asm/mmu_stats.h
> create mode 100644 arch/kvx/include/asm/page.h
> create mode 100644 arch/kvx/include/asm/page_size.h
> create mode 100644 arch/kvx/include/asm/pci.h
> create mode 100644 arch/kvx/include/asm/perf_event.h
> create mode 100644 arch/kvx/include/asm/pgalloc.h
> create mode 100644 arch/kvx/include/asm/pgtable-bits.h
> create mode 100644 arch/kvx/include/asm/pgtable.h
> create mode 100644 arch/kvx/include/asm/privilege.h
> create mode 100644 arch/kvx/include/asm/processor.h
> create mode 100644 arch/kvx/include/asm/ptrace.h
> create mode 100644 arch/kvx/include/asm/pwr_ctrl.h
> create mode 100644 arch/kvx/include/asm/rm_fw.h
> create mode 100644 arch/kvx/include/asm/sections.h
> create mode 100644 arch/kvx/include/asm/setup.h
> create mode 100644 arch/kvx/include/asm/sfr.h
> create mode 100644 arch/kvx/include/asm/sfr_defs.h
> create mode 100644 arch/kvx/include/asm/smp.h
> create mode 100644 arch/kvx/include/asm/sparsemem.h
> create mode 100644 arch/kvx/include/asm/spinlock.h
> create mode 100644 arch/kvx/include/asm/spinlock_types.h
> create mode 100644 arch/kvx/include/asm/stackprotector.h
> create mode 100644 arch/kvx/include/asm/stacktrace.h
> create mode 100644 arch/kvx/include/asm/string.h
> create mode 100644 arch/kvx/include/asm/swab.h
> create mode 100644 arch/kvx/include/asm/switch_to.h
> create mode 100644 arch/kvx/include/asm/symbols.h
> create mode 100644 arch/kvx/include/asm/sys_arch.h
> create mode 100644 arch/kvx/include/asm/syscall.h
> create mode 100644 arch/kvx/include/asm/syscalls.h
> create mode 100644 arch/kvx/include/asm/thread_info.h
> create mode 100644 arch/kvx/include/asm/timex.h
> create mode 100644 arch/kvx/include/asm/tlb.h
> create mode 100644 arch/kvx/include/asm/tlb_defs.h
> create mode 100644 arch/kvx/include/asm/tlbflush.h
> create mode 100644 arch/kvx/include/asm/traps.h
> create mode 100644 arch/kvx/include/asm/types.h
> create mode 100644 arch/kvx/include/asm/uaccess.h
> create mode 100644 arch/kvx/include/asm/unistd.h
> create mode 100644 arch/kvx/include/asm/vermagic.h
> create mode 100644 arch/kvx/include/asm/vmalloc.h
> create mode 100644 arch/kvx/include/uapi/asm/Kbuild
> create mode 100644 arch/kvx/include/uapi/asm/bitsperlong.h
> create mode 100644 arch/kvx/include/uapi/asm/byteorder.h
> create mode 100644 arch/kvx/include/uapi/asm/cachectl.h
> create mode 100644 arch/kvx/include/uapi/asm/ptrace.h
> create mode 100644 arch/kvx/include/uapi/asm/sigcontext.h
> create mode 100644 arch/kvx/include/uapi/asm/unistd.h
> create mode 100644 arch/kvx/kernel/Makefile
> create mode 100644 arch/kvx/kernel/asm-offsets.c
> create mode 100644 arch/kvx/kernel/break_hook.c
> create mode 100644 arch/kvx/kernel/common.c
> create mode 100644 arch/kvx/kernel/cpuinfo.c
> create mode 100644 arch/kvx/kernel/dame_handler.c
> create mode 100644 arch/kvx/kernel/debug.c
> create mode 100644 arch/kvx/kernel/entry.S
> create mode 100644 arch/kvx/kernel/ftrace.c
> create mode 100644 arch/kvx/kernel/head.S
> create mode 100644 arch/kvx/kernel/hw_breakpoint.c
> create mode 100644 arch/kvx/kernel/insns.c
> create mode 100644 arch/kvx/kernel/io.c
> create mode 100644 arch/kvx/kernel/irq.c
> create mode 100644 arch/kvx/kernel/jump_label.c
> create mode 100644 arch/kvx/kernel/kvx_ksyms.c
> create mode 100644 arch/kvx/kernel/l2_cache.c
> create mode 100644 arch/kvx/kernel/mcount.S
> create mode 100644 arch/kvx/kernel/module.c
> create mode 100644 arch/kvx/kernel/perf_event.c
> create mode 100644 arch/kvx/kernel/process.c
> create mode 100644 arch/kvx/kernel/prom.c
> create mode 100644 arch/kvx/kernel/ptrace.c
> create mode 100644 arch/kvx/kernel/reset.c
> create mode 100644 arch/kvx/kernel/return_address.c
> create mode 100644 arch/kvx/kernel/setup.c
> create mode 100644 arch/kvx/kernel/signal.c
> create mode 100644 arch/kvx/kernel/smp.c
> create mode 100644 arch/kvx/kernel/smpboot.c
> create mode 100644 arch/kvx/kernel/stacktrace.c
> create mode 100644 arch/kvx/kernel/sys_kvx.c
> create mode 100644 arch/kvx/kernel/syscall_table.c
> create mode 100644 arch/kvx/kernel/time.c
> create mode 100644 arch/kvx/kernel/traps.c
> create mode 100644 arch/kvx/kernel/vdso.c
> create mode 100644 arch/kvx/kernel/vmlinux.lds.S
> create mode 100644 arch/kvx/lib/Makefile
> create mode 100644 arch/kvx/lib/clear_page.S
> create mode 100644 arch/kvx/lib/copy_page.S
> create mode 100644 arch/kvx/lib/delay.c
> create mode 100644 arch/kvx/lib/memcpy.c
> create mode 100644 arch/kvx/lib/memset.S
> create mode 100644 arch/kvx/lib/strlen.S
> create mode 100644 arch/kvx/lib/usercopy.S
> create mode 100644 arch/kvx/mm/Makefile
> create mode 100644 arch/kvx/mm/cacheflush.c
> create mode 100644 arch/kvx/mm/dma-mapping.c
> create mode 100644 arch/kvx/mm/extable.c
> create mode 100644 arch/kvx/mm/fault.c
> create mode 100644 arch/kvx/mm/hugetlbpage.c
> create mode 100644 arch/kvx/mm/init.c
> create mode 100644 arch/kvx/mm/kernel_rwx.c
> create mode 100644 arch/kvx/mm/mmap.c
> create mode 100644 arch/kvx/mm/mmu.c
> create mode 100644 arch/kvx/mm/mmu_stats.c
> create mode 100644 arch/kvx/mm/tlb.c
> create mode 100644 arch/kvx/platform/Makefile
> create mode 100644 arch/kvx/platform/ipi.c
> create mode 100644 arch/kvx/platform/pwr_ctrl.c
> create mode 100644 drivers/irqchip/irq-kvx-apic-gic.c
> create mode 100644 drivers/irqchip/irq-kvx-apic-mailbox.c
> create mode 100644 drivers/irqchip/irq-kvx-core-intc.c
> create mode 100644 drivers/irqchip/irq-kvx-itgen.c
> create mode 100644 drivers/power/reset/kvx-scall-poweroff.c
> create mode 100644 include/linux/irqchip/irq-kvx-apic-gic.h
> create mode 100644 include/linux/irqchip/irq-kvx-apic-mailbox.h
> create mode 100644 include/linux/irqchip/irq-kvx-itgen.h
> create mode 100644 scripts/gdb/arch/Makefile
> create mode 100644 scripts/gdb/arch/__init__.py
> create mode 100644 scripts/gdb/arch/kvx/Makefile
> create mode 100644 scripts/gdb/arch/kvx/__init__.py
> create mode 100644 scripts/gdb/arch/kvx/constants.py.in
> create mode 100644 scripts/gdb/arch/kvx/mmu.py
> create mode 100644 scripts/gdb/arch/kvx/page_table_walk.py
>
> --
> 2.37.2
>
>
>
>
>
--
Thanks,
JeffXie
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-07 6:25 ` Jeff Xie
0 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-07 6:25 UTC (permalink / raw)
To: Yann Sionneau
Cc: Mark Rutland, Marc Poulhiès, linux-doc, Alexander Shishkin,
Jan Kiszka, Sebastian Reichel, Marius Gligor, Oleg Nesterov,
linux-mm, Louis Morhet, Krzysztof Kozlowski, Luc Michel,
linux-riscv, Ashley Lesdalons, Will Deacon, Ard Biesheuvel,
linux-arch, Marc Zyngier, Thomas Costis, Jonathan Corbet,
Jonathan Borne, Aneesh Kumar K.V, Peter Zijlstra, Ingo Molnar,
Steven Rostedt, Jean-Christophe Pince, Waiman Long,
Clement Leger, Masami Hiramatsu, devicetree, Albert Ou,
Kees Cook, Arnd Bergmann, linux-pm, Samuel Jones, Boqun Feng,
Guillaume Thouvenin, Julian Vetter, Arnaldo Carvalho de Melo,
Julien Hascoet, Jason Baron, Rob Herring, Nick Piggin,
Paul Walmsley, Namhyung Kim, Thomas Gleixner, Josh Poimboeuf,
linux-arm-kernel, Julien Villette, Jules Maselbas,
Christian Brauner, Andrew Morton, Benjamin Mugnier,
Vincent Chardon, Guillaume Missonnier, linux-kernel, Eric Paris,
Alex Michon, linux-perf-users, linux-audit, Palmer Dabbelt,
Eric Biederman, Jiri Olsa, Kieran Bingham, bpf
Hi,
On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
>
> This patch series adds support for the kv3-1 CPU architecture of the kvx family
> found in the Coolidge (aka MPPA3-80) SoC of Kalray.
>
> This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> therefore this patch series cannot be merged into Linux for now.
>
> The goal is to have preliminary reviews and to fix problems early.
>
> The Kalray VLIW processor family (kvx) has the following features:
> * 32/64 bits execution mode
> * 6-issue VLIW architecture
> * 64 x 64bits general purpose registers
> * SIMD instructions
> * little-endian
> * deep learning co-processor
>
> Kalray kv3-1 core which is the third of the kvx family is embedded in Kalray
> Coolidge SoC currently used on K200 and K200-LP boards.
>
> The Coolidge SoC contains 5 clusters each of which is made of:
> * 4MiB of on-chip memory (SMEM)
> * 1 dedicated safety/security core (kv3-1 core).
> * 16 PEs (Processing Elements) (kv3-1 cores).
> * 16 Co-processors (one per PE)
> * 2 Crypto accelerators
>
> The Coolidge SoC contains the following features:
> * 5 Clusters
> * 2 100G Ethernet controllers
> * 8 PCIe GEN4 controllers (Root Complex and Endpoint capable)
> * 2 USB 2.0 controllers
> * 1 Octal SPI-NOR flash controller
> * 1 eMMC controller
> * 3 Quad SPI controllers
> * 6 UART
> * 5 I2C controllers (3 of which are SMBus capable)
> * 4 CAN controllers
> * 1 OTP memory
>
> A kvx toolchain can be built using:
> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> $ git clone https://github.com/kalray/build-scripts
> $ cd build-scripts
> $ source last.refs
> $ ./build-kvx-xgcc.sh output
I would like to build the kvx-xgcc to compile and test the linux
kernel, but it reported a compile error.
I wonder what version of gcc you are using.
My build environment:
VERSION="20.04.2 LTS (Focal Fossa)"
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
Compile error:
$ ./build-kvx-xgcc.sh output
../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
(first use in this function)
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:220:30: note: in expansion of macro
‘FIBHEAPKEY_MIN’
220 | if (okey == key && okey != FIBHEAPKEY_MIN)
| ^~~~~~~~~~~~~~
../../binutils/libiberty/fibheap.c:36:1: note: ‘LONG_MIN’ is defined
in header ‘<limits.h>’; did you forget to ‘#include <limits.h>’?
35 | #include "fibheap.h"
+++ |+#include <limits.h>
36 |
../../binutils/libiberty/fibheap.c:38:24: note: each undeclared
identifier is reported only once for each function it appears in
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:220:30: note: in expansion of macro
‘FIBHEAPKEY_MIN’
220 | if (okey == key && okey != FIBHEAPKEY_MIN)
| ^~~~~~~~~~~~~~
../../binutils/libiberty/fibheap.c: In function ‘fibheap_delete_node’:
../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
(first use in this function)
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:261:36: note: in expansion of macro
‘FIBHEAPKEY_MIN’
261 | fibheap_replace_key (heap, node, FIBHEAPKEY_MIN);
| ^~~~~~~~~~~~~~
../../binutils/libiberty/fibheap.c:38:24: note: ‘LONG_MIN’ is defined
in header ‘<limits.h>’; did you forget to ‘#include <limits.h>’?
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:261:36: note: in expansion of macro
‘FIBHEAPKEY_MIN’
261 | fibheap_replace_key (heap, node, FIBHEAPKEY_MIN);
| ^~~~~~~~~~~~~~
> The kvx toolchain will be installed in the "output" directory.
>
> A buildroot image (kernel+rootfs) and toolchain can be built using:
> $ git clone -b coolidge-for-upstream https://github.com/kalray/buildroot
> $ cd buildroot
> $ make O=build_kvx kvx_defconfig
> $ make O=build_kvx
>
> The vmlinux image can be found in buildroot/build_kvx/images/vmlinux.
>
> If you are just interested in building the Linux kernel with no rootfs you can
> just do this with the kvx-elf- toolchain:
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- default_defconfig
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- -j$(($(nproc) + 1))
>
> The vmlinux ELF can be run with qemu by doing:
> # install dependencies: ninja pkg-config libglib-2.0-dev cmake libfdt-dev libpixman-1-dev zlib1g-dev
> $ git clone https://github.com/kalray/qemu-builder
> $ cd qemu-builder
> $ git submodule update --init
> $ make -j$(($(nproc) + 1))
> $ ./qemu-system-kvx -m 1024 -nographic -kernel <path/to/vmlinux>
>
> Yann Sionneau (25):
> Documentation: kvx: Add basic documentation
> kvx: Add ELF-related definitions
> kvx: Add build infrastructure
> kvx: Add CPU definition headers
> kvx: Add atomic/locking headers
> kvx: Add other common headers
> kvx: Add boot and setup routines
> kvx: Add exception/interrupt handling
> kvx: irqchip: Add support for irq controllers
> kvx: Add process management
> kvx: Add memory management
> kvx: Add system call support
> kvx: Add signal handling support
> kvx: Add ELF relocations and module support
> kvx: Add misc common routines
> kvx: Add some library functions
> kvx: Add multi-processor (SMP) support
> kvx: Add kvx default config file
> kvx: power: scall poweroff driver
> kvx: gdb: add kvx related gdb helpers
> kvx: Add support for ftrace
> kvx: Add support for jump labels
> kvx: Add debugging related support
> kvx: Add support for CPU Perf Monitors
> kvx: Add support for cpuinfo
>
> .../kalray,kvx-core-intc.txt | 22 +
> .../devicetree/bindings/perf/kalray-pm.txt | 21 +
> Documentation/kvx/kvx-exceptions.txt | 246 +
> Documentation/kvx/kvx-iommu.txt | 183 +
> Documentation/kvx/kvx-mmu.txt | 272 +
> Documentation/kvx/kvx-smp.txt | 36 +
> Documentation/kvx/kvx.txt | 268 +
> arch/kvx/Kconfig | 249 +
> arch/kvx/Kconfig.debug | 70 +
> arch/kvx/Makefile | 52 +
> arch/kvx/configs/default_defconfig | 130 +
> arch/kvx/include/asm/Kbuild | 20 +
> arch/kvx/include/asm/asm-prototypes.h | 14 +
> arch/kvx/include/asm/atomic.h | 104 +
> arch/kvx/include/asm/barrier.h | 15 +
> arch/kvx/include/asm/bitops.h | 207 +
> arch/kvx/include/asm/bitrev.h | 32 +
> arch/kvx/include/asm/break_hook.h | 69 +
> arch/kvx/include/asm/bug.h | 67 +
> arch/kvx/include/asm/cache.h | 46 +
> arch/kvx/include/asm/cacheflush.h | 181 +
> arch/kvx/include/asm/clocksource.h | 17 +
> arch/kvx/include/asm/cmpxchg.h | 185 +
> arch/kvx/include/asm/current.h | 22 +
> arch/kvx/include/asm/dame.h | 31 +
> arch/kvx/include/asm/debug.h | 35 +
> arch/kvx/include/asm/elf.h | 155 +
> arch/kvx/include/asm/fixmap.h | 47 +
> arch/kvx/include/asm/ftrace.h | 41 +
> arch/kvx/include/asm/futex.h | 141 +
> arch/kvx/include/asm/hardirq.h | 14 +
> arch/kvx/include/asm/hugetlb.h | 36 +
> arch/kvx/include/asm/hw_breakpoint.h | 72 +
> arch/kvx/include/asm/hw_irq.h | 14 +
> arch/kvx/include/asm/insns.h | 16 +
> arch/kvx/include/asm/insns_defs.h | 197 +
> arch/kvx/include/asm/io.h | 34 +
> arch/kvx/include/asm/ipi.h | 16 +
> arch/kvx/include/asm/irqflags.h | 58 +
> arch/kvx/include/asm/jump_label.h | 59 +
> arch/kvx/include/asm/l2_cache.h | 75 +
> arch/kvx/include/asm/l2_cache_defs.h | 64 +
> arch/kvx/include/asm/linkage.h | 13 +
> arch/kvx/include/asm/mem_map.h | 44 +
> arch/kvx/include/asm/mmu.h | 296 +
> arch/kvx/include/asm/mmu_context.h | 156 +
> arch/kvx/include/asm/mmu_stats.h | 38 +
> arch/kvx/include/asm/page.h | 187 +
> arch/kvx/include/asm/page_size.h | 29 +
> arch/kvx/include/asm/pci.h | 36 +
> arch/kvx/include/asm/perf_event.h | 90 +
> arch/kvx/include/asm/pgalloc.h | 101 +
> arch/kvx/include/asm/pgtable-bits.h | 102 +
> arch/kvx/include/asm/pgtable.h | 451 ++
> arch/kvx/include/asm/privilege.h | 211 +
> arch/kvx/include/asm/processor.h | 176 +
> arch/kvx/include/asm/ptrace.h | 217 +
> arch/kvx/include/asm/pwr_ctrl.h | 45 +
> arch/kvx/include/asm/rm_fw.h | 16 +
> arch/kvx/include/asm/sections.h | 18 +
> arch/kvx/include/asm/setup.h | 29 +
> arch/kvx/include/asm/sfr.h | 107 +
> arch/kvx/include/asm/sfr_defs.h | 5028 +++++++++++++++++
> arch/kvx/include/asm/smp.h | 42 +
> arch/kvx/include/asm/sparsemem.h | 15 +
> arch/kvx/include/asm/spinlock.h | 16 +
> arch/kvx/include/asm/spinlock_types.h | 17 +
> arch/kvx/include/asm/stackprotector.h | 47 +
> arch/kvx/include/asm/stacktrace.h | 44 +
> arch/kvx/include/asm/string.h | 20 +
> arch/kvx/include/asm/swab.h | 48 +
> arch/kvx/include/asm/switch_to.h | 21 +
> arch/kvx/include/asm/symbols.h | 16 +
> arch/kvx/include/asm/sys_arch.h | 51 +
> arch/kvx/include/asm/syscall.h | 73 +
> arch/kvx/include/asm/syscalls.h | 21 +
> arch/kvx/include/asm/thread_info.h | 78 +
> arch/kvx/include/asm/timex.h | 20 +
> arch/kvx/include/asm/tlb.h | 24 +
> arch/kvx/include/asm/tlb_defs.h | 131 +
> arch/kvx/include/asm/tlbflush.h | 58 +
> arch/kvx/include/asm/traps.h | 76 +
> arch/kvx/include/asm/types.h | 12 +
> arch/kvx/include/asm/uaccess.h | 324 ++
> arch/kvx/include/asm/unistd.h | 11 +
> arch/kvx/include/asm/vermagic.h | 12 +
> arch/kvx/include/asm/vmalloc.h | 10 +
> arch/kvx/include/uapi/asm/Kbuild | 1 +
> arch/kvx/include/uapi/asm/bitsperlong.h | 14 +
> arch/kvx/include/uapi/asm/byteorder.h | 12 +
> arch/kvx/include/uapi/asm/cachectl.h | 25 +
> arch/kvx/include/uapi/asm/ptrace.h | 114 +
> arch/kvx/include/uapi/asm/sigcontext.h | 16 +
> arch/kvx/include/uapi/asm/unistd.h | 16 +
> arch/kvx/kernel/Makefile | 27 +
> arch/kvx/kernel/asm-offsets.c | 157 +
> arch/kvx/kernel/break_hook.c | 77 +
> arch/kvx/kernel/common.c | 11 +
> arch/kvx/kernel/cpuinfo.c | 96 +
> arch/kvx/kernel/dame_handler.c | 113 +
> arch/kvx/kernel/debug.c | 64 +
> arch/kvx/kernel/entry.S | 1759 ++++++
> arch/kvx/kernel/ftrace.c | 339 ++
> arch/kvx/kernel/head.S | 612 ++
> arch/kvx/kernel/hw_breakpoint.c | 556 ++
> arch/kvx/kernel/insns.c | 146 +
> arch/kvx/kernel/io.c | 96 +
> arch/kvx/kernel/irq.c | 78 +
> arch/kvx/kernel/jump_label.c | 34 +
> arch/kvx/kernel/kvx_ksyms.c | 29 +
> arch/kvx/kernel/l2_cache.c | 448 ++
> arch/kvx/kernel/mcount.S | 340 ++
> arch/kvx/kernel/module.c | 148 +
> arch/kvx/kernel/perf_event.c | 609 ++
> arch/kvx/kernel/process.c | 212 +
> arch/kvx/kernel/prom.c | 24 +
> arch/kvx/kernel/ptrace.c | 461 ++
> arch/kvx/kernel/reset.c | 37 +
> arch/kvx/kernel/return_address.c | 55 +
> arch/kvx/kernel/setup.c | 178 +
> arch/kvx/kernel/signal.c | 266 +
> arch/kvx/kernel/smp.c | 110 +
> arch/kvx/kernel/smpboot.c | 127 +
> arch/kvx/kernel/stacktrace.c | 173 +
> arch/kvx/kernel/sys_kvx.c | 58 +
> arch/kvx/kernel/syscall_table.c | 19 +
> arch/kvx/kernel/time.c | 242 +
> arch/kvx/kernel/traps.c | 243 +
> arch/kvx/kernel/vdso.c | 87 +
> arch/kvx/kernel/vmlinux.lds.S | 173 +
> arch/kvx/lib/Makefile | 6 +
> arch/kvx/lib/clear_page.S | 40 +
> arch/kvx/lib/copy_page.S | 90 +
> arch/kvx/lib/delay.c | 39 +
> arch/kvx/lib/memcpy.c | 70 +
> arch/kvx/lib/memset.S | 351 ++
> arch/kvx/lib/strlen.S | 122 +
> arch/kvx/lib/usercopy.S | 90 +
> arch/kvx/mm/Makefile | 10 +
> arch/kvx/mm/cacheflush.c | 154 +
> arch/kvx/mm/dma-mapping.c | 95 +
> arch/kvx/mm/extable.c | 24 +
> arch/kvx/mm/fault.c | 264 +
> arch/kvx/mm/hugetlbpage.c | 317 ++
> arch/kvx/mm/init.c | 527 ++
> arch/kvx/mm/kernel_rwx.c | 228 +
> arch/kvx/mm/mmap.c | 31 +
> arch/kvx/mm/mmu.c | 204 +
> arch/kvx/mm/mmu_stats.c | 94 +
> arch/kvx/mm/tlb.c | 433 ++
> arch/kvx/platform/Makefile | 7 +
> arch/kvx/platform/ipi.c | 110 +
> arch/kvx/platform/pwr_ctrl.c | 93 +
> drivers/irqchip/Kconfig | 27 +
> drivers/irqchip/Makefile | 4 +
> drivers/irqchip/irq-kvx-apic-gic.c | 349 ++
> drivers/irqchip/irq-kvx-apic-mailbox.c | 465 ++
> drivers/irqchip/irq-kvx-core-intc.c | 82 +
> drivers/irqchip/irq-kvx-itgen.c | 224 +
> drivers/power/reset/kvx-scall-poweroff.c | 53 +
> include/linux/cpuhotplug.h | 2 +
> include/linux/irqchip/irq-kvx-apic-gic.h | 21 +
> include/linux/irqchip/irq-kvx-apic-mailbox.h | 29 +
> include/linux/irqchip/irq-kvx-itgen.h | 24 +
> include/uapi/linux/audit.h | 1 +
> include/uapi/linux/elf-em.h | 1 +
> include/uapi/linux/elf.h | 1 +
> scripts/gdb/arch/Makefile | 11 +
> scripts/gdb/arch/__init__.py | 1 +
> scripts/gdb/arch/kvx/Makefile | 25 +
> scripts/gdb/arch/kvx/__init__.py | 1 +
> scripts/gdb/arch/kvx/constants.py.in | 74 +
> scripts/gdb/arch/kvx/mmu.py | 199 +
> scripts/gdb/arch/kvx/page_table_walk.py | 207 +
> tools/include/uapi/asm/bitsperlong.h | 2 +
> 175 files changed, 25814 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt
> create mode 100644 Documentation/devicetree/bindings/perf/kalray-pm.txt
> create mode 100644 Documentation/kvx/kvx-exceptions.txt
> create mode 100644 Documentation/kvx/kvx-iommu.txt
> create mode 100644 Documentation/kvx/kvx-mmu.txt
> create mode 100644 Documentation/kvx/kvx-smp.txt
> create mode 100644 Documentation/kvx/kvx.txt
> create mode 100644 arch/kvx/Kconfig
> create mode 100644 arch/kvx/Kconfig.debug
> create mode 100644 arch/kvx/Makefile
> create mode 100644 arch/kvx/configs/default_defconfig
> create mode 100644 arch/kvx/include/asm/Kbuild
> create mode 100644 arch/kvx/include/asm/asm-prototypes.h
> create mode 100644 arch/kvx/include/asm/atomic.h
> create mode 100644 arch/kvx/include/asm/barrier.h
> create mode 100644 arch/kvx/include/asm/bitops.h
> create mode 100644 arch/kvx/include/asm/bitrev.h
> create mode 100644 arch/kvx/include/asm/break_hook.h
> create mode 100644 arch/kvx/include/asm/bug.h
> create mode 100644 arch/kvx/include/asm/cache.h
> create mode 100644 arch/kvx/include/asm/cacheflush.h
> create mode 100644 arch/kvx/include/asm/clocksource.h
> create mode 100644 arch/kvx/include/asm/cmpxchg.h
> create mode 100644 arch/kvx/include/asm/current.h
> create mode 100644 arch/kvx/include/asm/dame.h
> create mode 100644 arch/kvx/include/asm/debug.h
> create mode 100644 arch/kvx/include/asm/elf.h
> create mode 100644 arch/kvx/include/asm/fixmap.h
> create mode 100644 arch/kvx/include/asm/ftrace.h
> create mode 100644 arch/kvx/include/asm/futex.h
> create mode 100644 arch/kvx/include/asm/hardirq.h
> create mode 100644 arch/kvx/include/asm/hugetlb.h
> create mode 100644 arch/kvx/include/asm/hw_breakpoint.h
> create mode 100644 arch/kvx/include/asm/hw_irq.h
> create mode 100644 arch/kvx/include/asm/insns.h
> create mode 100644 arch/kvx/include/asm/insns_defs.h
> create mode 100644 arch/kvx/include/asm/io.h
> create mode 100644 arch/kvx/include/asm/ipi.h
> create mode 100644 arch/kvx/include/asm/irqflags.h
> create mode 100644 arch/kvx/include/asm/jump_label.h
> create mode 100644 arch/kvx/include/asm/l2_cache.h
> create mode 100644 arch/kvx/include/asm/l2_cache_defs.h
> create mode 100644 arch/kvx/include/asm/linkage.h
> create mode 100644 arch/kvx/include/asm/mem_map.h
> create mode 100644 arch/kvx/include/asm/mmu.h
> create mode 100644 arch/kvx/include/asm/mmu_context.h
> create mode 100644 arch/kvx/include/asm/mmu_stats.h
> create mode 100644 arch/kvx/include/asm/page.h
> create mode 100644 arch/kvx/include/asm/page_size.h
> create mode 100644 arch/kvx/include/asm/pci.h
> create mode 100644 arch/kvx/include/asm/perf_event.h
> create mode 100644 arch/kvx/include/asm/pgalloc.h
> create mode 100644 arch/kvx/include/asm/pgtable-bits.h
> create mode 100644 arch/kvx/include/asm/pgtable.h
> create mode 100644 arch/kvx/include/asm/privilege.h
> create mode 100644 arch/kvx/include/asm/processor.h
> create mode 100644 arch/kvx/include/asm/ptrace.h
> create mode 100644 arch/kvx/include/asm/pwr_ctrl.h
> create mode 100644 arch/kvx/include/asm/rm_fw.h
> create mode 100644 arch/kvx/include/asm/sections.h
> create mode 100644 arch/kvx/include/asm/setup.h
> create mode 100644 arch/kvx/include/asm/sfr.h
> create mode 100644 arch/kvx/include/asm/sfr_defs.h
> create mode 100644 arch/kvx/include/asm/smp.h
> create mode 100644 arch/kvx/include/asm/sparsemem.h
> create mode 100644 arch/kvx/include/asm/spinlock.h
> create mode 100644 arch/kvx/include/asm/spinlock_types.h
> create mode 100644 arch/kvx/include/asm/stackprotector.h
> create mode 100644 arch/kvx/include/asm/stacktrace.h
> create mode 100644 arch/kvx/include/asm/string.h
> create mode 100644 arch/kvx/include/asm/swab.h
> create mode 100644 arch/kvx/include/asm/switch_to.h
> create mode 100644 arch/kvx/include/asm/symbols.h
> create mode 100644 arch/kvx/include/asm/sys_arch.h
> create mode 100644 arch/kvx/include/asm/syscall.h
> create mode 100644 arch/kvx/include/asm/syscalls.h
> create mode 100644 arch/kvx/include/asm/thread_info.h
> create mode 100644 arch/kvx/include/asm/timex.h
> create mode 100644 arch/kvx/include/asm/tlb.h
> create mode 100644 arch/kvx/include/asm/tlb_defs.h
> create mode 100644 arch/kvx/include/asm/tlbflush.h
> create mode 100644 arch/kvx/include/asm/traps.h
> create mode 100644 arch/kvx/include/asm/types.h
> create mode 100644 arch/kvx/include/asm/uaccess.h
> create mode 100644 arch/kvx/include/asm/unistd.h
> create mode 100644 arch/kvx/include/asm/vermagic.h
> create mode 100644 arch/kvx/include/asm/vmalloc.h
> create mode 100644 arch/kvx/include/uapi/asm/Kbuild
> create mode 100644 arch/kvx/include/uapi/asm/bitsperlong.h
> create mode 100644 arch/kvx/include/uapi/asm/byteorder.h
> create mode 100644 arch/kvx/include/uapi/asm/cachectl.h
> create mode 100644 arch/kvx/include/uapi/asm/ptrace.h
> create mode 100644 arch/kvx/include/uapi/asm/sigcontext.h
> create mode 100644 arch/kvx/include/uapi/asm/unistd.h
> create mode 100644 arch/kvx/kernel/Makefile
> create mode 100644 arch/kvx/kernel/asm-offsets.c
> create mode 100644 arch/kvx/kernel/break_hook.c
> create mode 100644 arch/kvx/kernel/common.c
> create mode 100644 arch/kvx/kernel/cpuinfo.c
> create mode 100644 arch/kvx/kernel/dame_handler.c
> create mode 100644 arch/kvx/kernel/debug.c
> create mode 100644 arch/kvx/kernel/entry.S
> create mode 100644 arch/kvx/kernel/ftrace.c
> create mode 100644 arch/kvx/kernel/head.S
> create mode 100644 arch/kvx/kernel/hw_breakpoint.c
> create mode 100644 arch/kvx/kernel/insns.c
> create mode 100644 arch/kvx/kernel/io.c
> create mode 100644 arch/kvx/kernel/irq.c
> create mode 100644 arch/kvx/kernel/jump_label.c
> create mode 100644 arch/kvx/kernel/kvx_ksyms.c
> create mode 100644 arch/kvx/kernel/l2_cache.c
> create mode 100644 arch/kvx/kernel/mcount.S
> create mode 100644 arch/kvx/kernel/module.c
> create mode 100644 arch/kvx/kernel/perf_event.c
> create mode 100644 arch/kvx/kernel/process.c
> create mode 100644 arch/kvx/kernel/prom.c
> create mode 100644 arch/kvx/kernel/ptrace.c
> create mode 100644 arch/kvx/kernel/reset.c
> create mode 100644 arch/kvx/kernel/return_address.c
> create mode 100644 arch/kvx/kernel/setup.c
> create mode 100644 arch/kvx/kernel/signal.c
> create mode 100644 arch/kvx/kernel/smp.c
> create mode 100644 arch/kvx/kernel/smpboot.c
> create mode 100644 arch/kvx/kernel/stacktrace.c
> create mode 100644 arch/kvx/kernel/sys_kvx.c
> create mode 100644 arch/kvx/kernel/syscall_table.c
> create mode 100644 arch/kvx/kernel/time.c
> create mode 100644 arch/kvx/kernel/traps.c
> create mode 100644 arch/kvx/kernel/vdso.c
> create mode 100644 arch/kvx/kernel/vmlinux.lds.S
> create mode 100644 arch/kvx/lib/Makefile
> create mode 100644 arch/kvx/lib/clear_page.S
> create mode 100644 arch/kvx/lib/copy_page.S
> create mode 100644 arch/kvx/lib/delay.c
> create mode 100644 arch/kvx/lib/memcpy.c
> create mode 100644 arch/kvx/lib/memset.S
> create mode 100644 arch/kvx/lib/strlen.S
> create mode 100644 arch/kvx/lib/usercopy.S
> create mode 100644 arch/kvx/mm/Makefile
> create mode 100644 arch/kvx/mm/cacheflush.c
> create mode 100644 arch/kvx/mm/dma-mapping.c
> create mode 100644 arch/kvx/mm/extable.c
> create mode 100644 arch/kvx/mm/fault.c
> create mode 100644 arch/kvx/mm/hugetlbpage.c
> create mode 100644 arch/kvx/mm/init.c
> create mode 100644 arch/kvx/mm/kernel_rwx.c
> create mode 100644 arch/kvx/mm/mmap.c
> create mode 100644 arch/kvx/mm/mmu.c
> create mode 100644 arch/kvx/mm/mmu_stats.c
> create mode 100644 arch/kvx/mm/tlb.c
> create mode 100644 arch/kvx/platform/Makefile
> create mode 100644 arch/kvx/platform/ipi.c
> create mode 100644 arch/kvx/platform/pwr_ctrl.c
> create mode 100644 drivers/irqchip/irq-kvx-apic-gic.c
> create mode 100644 drivers/irqchip/irq-kvx-apic-mailbox.c
> create mode 100644 drivers/irqchip/irq-kvx-core-intc.c
> create mode 100644 drivers/irqchip/irq-kvx-itgen.c
> create mode 100644 drivers/power/reset/kvx-scall-poweroff.c
> create mode 100644 include/linux/irqchip/irq-kvx-apic-gic.h
> create mode 100644 include/linux/irqchip/irq-kvx-apic-mailbox.h
> create mode 100644 include/linux/irqchip/irq-kvx-itgen.h
> create mode 100644 scripts/gdb/arch/Makefile
> create mode 100644 scripts/gdb/arch/__init__.py
> create mode 100644 scripts/gdb/arch/kvx/Makefile
> create mode 100644 scripts/gdb/arch/kvx/__init__.py
> create mode 100644 scripts/gdb/arch/kvx/constants.py.in
> create mode 100644 scripts/gdb/arch/kvx/mmu.py
> create mode 100644 scripts/gdb/arch/kvx/page_table_walk.py
>
> --
> 2.37.2
>
>
>
>
>
--
Thanks,
JeffXie
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-07 6:25 ` Jeff Xie
0 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-07 6:25 UTC (permalink / raw)
To: Yann Sionneau
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
Hi,
On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
>
> This patch series adds support for the kv3-1 CPU architecture of the kvx family
> found in the Coolidge (aka MPPA3-80) SoC of Kalray.
>
> This is an RFC, since kvx support is not yet upstreamed into gcc/binutils,
> therefore this patch series cannot be merged into Linux for now.
>
> The goal is to have preliminary reviews and to fix problems early.
>
> The Kalray VLIW processor family (kvx) has the following features:
> * 32/64 bits execution mode
> * 6-issue VLIW architecture
> * 64 x 64bits general purpose registers
> * SIMD instructions
> * little-endian
> * deep learning co-processor
>
> Kalray kv3-1 core which is the third of the kvx family is embedded in Kalray
> Coolidge SoC currently used on K200 and K200-LP boards.
>
> The Coolidge SoC contains 5 clusters each of which is made of:
> * 4MiB of on-chip memory (SMEM)
> * 1 dedicated safety/security core (kv3-1 core).
> * 16 PEs (Processing Elements) (kv3-1 cores).
> * 16 Co-processors (one per PE)
> * 2 Crypto accelerators
>
> The Coolidge SoC contains the following features:
> * 5 Clusters
> * 2 100G Ethernet controllers
> * 8 PCIe GEN4 controllers (Root Complex and Endpoint capable)
> * 2 USB 2.0 controllers
> * 1 Octal SPI-NOR flash controller
> * 1 eMMC controller
> * 3 Quad SPI controllers
> * 6 UART
> * 5 I2C controllers (3 of which are SMBus capable)
> * 4 CAN controllers
> * 1 OTP memory
>
> A kvx toolchain can be built using:
> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> $ git clone https://github.com/kalray/build-scripts
> $ cd build-scripts
> $ source last.refs
> $ ./build-kvx-xgcc.sh output
I would like to build the kvx-xgcc to compile and test the linux
kernel, but it reported a compile error.
I wonder what version of gcc you are using.
My build environment:
VERSION="20.04.2 LTS (Focal Fossa)"
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
Compile error:
$ ./build-kvx-xgcc.sh output
../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
(first use in this function)
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:220:30: note: in expansion of macro
‘FIBHEAPKEY_MIN’
220 | if (okey == key && okey != FIBHEAPKEY_MIN)
| ^~~~~~~~~~~~~~
../../binutils/libiberty/fibheap.c:36:1: note: ‘LONG_MIN’ is defined
in header ‘<limits.h>’; did you forget to ‘#include <limits.h>’?
35 | #include "fibheap.h"
+++ |+#include <limits.h>
36 |
../../binutils/libiberty/fibheap.c:38:24: note: each undeclared
identifier is reported only once for each function it appears in
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:220:30: note: in expansion of macro
‘FIBHEAPKEY_MIN’
220 | if (okey == key && okey != FIBHEAPKEY_MIN)
| ^~~~~~~~~~~~~~
../../binutils/libiberty/fibheap.c: In function ‘fibheap_delete_node’:
../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
(first use in this function)
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:261:36: note: in expansion of macro
‘FIBHEAPKEY_MIN’
261 | fibheap_replace_key (heap, node, FIBHEAPKEY_MIN);
| ^~~~~~~~~~~~~~
../../binutils/libiberty/fibheap.c:38:24: note: ‘LONG_MIN’ is defined
in header ‘<limits.h>’; did you forget to ‘#include <limits.h>’?
38 | #define FIBHEAPKEY_MIN LONG_MIN
| ^~~~~~~~
../../binutils/libiberty/fibheap.c:261:36: note: in expansion of macro
‘FIBHEAPKEY_MIN’
261 | fibheap_replace_key (heap, node, FIBHEAPKEY_MIN);
| ^~~~~~~~~~~~~~
> The kvx toolchain will be installed in the "output" directory.
>
> A buildroot image (kernel+rootfs) and toolchain can be built using:
> $ git clone -b coolidge-for-upstream https://github.com/kalray/buildroot
> $ cd buildroot
> $ make O=build_kvx kvx_defconfig
> $ make O=build_kvx
>
> The vmlinux image can be found in buildroot/build_kvx/images/vmlinux.
>
> If you are just interested in building the Linux kernel with no rootfs you can
> just do this with the kvx-elf- toolchain:
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- default_defconfig
> $ make ARCH=kvx O=build_kvx CROSS_COMPILE=kvx-elf- -j$(($(nproc) + 1))
>
> The vmlinux ELF can be run with qemu by doing:
> # install dependencies: ninja pkg-config libglib-2.0-dev cmake libfdt-dev libpixman-1-dev zlib1g-dev
> $ git clone https://github.com/kalray/qemu-builder
> $ cd qemu-builder
> $ git submodule update --init
> $ make -j$(($(nproc) + 1))
> $ ./qemu-system-kvx -m 1024 -nographic -kernel <path/to/vmlinux>
>
> Yann Sionneau (25):
> Documentation: kvx: Add basic documentation
> kvx: Add ELF-related definitions
> kvx: Add build infrastructure
> kvx: Add CPU definition headers
> kvx: Add atomic/locking headers
> kvx: Add other common headers
> kvx: Add boot and setup routines
> kvx: Add exception/interrupt handling
> kvx: irqchip: Add support for irq controllers
> kvx: Add process management
> kvx: Add memory management
> kvx: Add system call support
> kvx: Add signal handling support
> kvx: Add ELF relocations and module support
> kvx: Add misc common routines
> kvx: Add some library functions
> kvx: Add multi-processor (SMP) support
> kvx: Add kvx default config file
> kvx: power: scall poweroff driver
> kvx: gdb: add kvx related gdb helpers
> kvx: Add support for ftrace
> kvx: Add support for jump labels
> kvx: Add debugging related support
> kvx: Add support for CPU Perf Monitors
> kvx: Add support for cpuinfo
>
> .../kalray,kvx-core-intc.txt | 22 +
> .../devicetree/bindings/perf/kalray-pm.txt | 21 +
> Documentation/kvx/kvx-exceptions.txt | 246 +
> Documentation/kvx/kvx-iommu.txt | 183 +
> Documentation/kvx/kvx-mmu.txt | 272 +
> Documentation/kvx/kvx-smp.txt | 36 +
> Documentation/kvx/kvx.txt | 268 +
> arch/kvx/Kconfig | 249 +
> arch/kvx/Kconfig.debug | 70 +
> arch/kvx/Makefile | 52 +
> arch/kvx/configs/default_defconfig | 130 +
> arch/kvx/include/asm/Kbuild | 20 +
> arch/kvx/include/asm/asm-prototypes.h | 14 +
> arch/kvx/include/asm/atomic.h | 104 +
> arch/kvx/include/asm/barrier.h | 15 +
> arch/kvx/include/asm/bitops.h | 207 +
> arch/kvx/include/asm/bitrev.h | 32 +
> arch/kvx/include/asm/break_hook.h | 69 +
> arch/kvx/include/asm/bug.h | 67 +
> arch/kvx/include/asm/cache.h | 46 +
> arch/kvx/include/asm/cacheflush.h | 181 +
> arch/kvx/include/asm/clocksource.h | 17 +
> arch/kvx/include/asm/cmpxchg.h | 185 +
> arch/kvx/include/asm/current.h | 22 +
> arch/kvx/include/asm/dame.h | 31 +
> arch/kvx/include/asm/debug.h | 35 +
> arch/kvx/include/asm/elf.h | 155 +
> arch/kvx/include/asm/fixmap.h | 47 +
> arch/kvx/include/asm/ftrace.h | 41 +
> arch/kvx/include/asm/futex.h | 141 +
> arch/kvx/include/asm/hardirq.h | 14 +
> arch/kvx/include/asm/hugetlb.h | 36 +
> arch/kvx/include/asm/hw_breakpoint.h | 72 +
> arch/kvx/include/asm/hw_irq.h | 14 +
> arch/kvx/include/asm/insns.h | 16 +
> arch/kvx/include/asm/insns_defs.h | 197 +
> arch/kvx/include/asm/io.h | 34 +
> arch/kvx/include/asm/ipi.h | 16 +
> arch/kvx/include/asm/irqflags.h | 58 +
> arch/kvx/include/asm/jump_label.h | 59 +
> arch/kvx/include/asm/l2_cache.h | 75 +
> arch/kvx/include/asm/l2_cache_defs.h | 64 +
> arch/kvx/include/asm/linkage.h | 13 +
> arch/kvx/include/asm/mem_map.h | 44 +
> arch/kvx/include/asm/mmu.h | 296 +
> arch/kvx/include/asm/mmu_context.h | 156 +
> arch/kvx/include/asm/mmu_stats.h | 38 +
> arch/kvx/include/asm/page.h | 187 +
> arch/kvx/include/asm/page_size.h | 29 +
> arch/kvx/include/asm/pci.h | 36 +
> arch/kvx/include/asm/perf_event.h | 90 +
> arch/kvx/include/asm/pgalloc.h | 101 +
> arch/kvx/include/asm/pgtable-bits.h | 102 +
> arch/kvx/include/asm/pgtable.h | 451 ++
> arch/kvx/include/asm/privilege.h | 211 +
> arch/kvx/include/asm/processor.h | 176 +
> arch/kvx/include/asm/ptrace.h | 217 +
> arch/kvx/include/asm/pwr_ctrl.h | 45 +
> arch/kvx/include/asm/rm_fw.h | 16 +
> arch/kvx/include/asm/sections.h | 18 +
> arch/kvx/include/asm/setup.h | 29 +
> arch/kvx/include/asm/sfr.h | 107 +
> arch/kvx/include/asm/sfr_defs.h | 5028 +++++++++++++++++
> arch/kvx/include/asm/smp.h | 42 +
> arch/kvx/include/asm/sparsemem.h | 15 +
> arch/kvx/include/asm/spinlock.h | 16 +
> arch/kvx/include/asm/spinlock_types.h | 17 +
> arch/kvx/include/asm/stackprotector.h | 47 +
> arch/kvx/include/asm/stacktrace.h | 44 +
> arch/kvx/include/asm/string.h | 20 +
> arch/kvx/include/asm/swab.h | 48 +
> arch/kvx/include/asm/switch_to.h | 21 +
> arch/kvx/include/asm/symbols.h | 16 +
> arch/kvx/include/asm/sys_arch.h | 51 +
> arch/kvx/include/asm/syscall.h | 73 +
> arch/kvx/include/asm/syscalls.h | 21 +
> arch/kvx/include/asm/thread_info.h | 78 +
> arch/kvx/include/asm/timex.h | 20 +
> arch/kvx/include/asm/tlb.h | 24 +
> arch/kvx/include/asm/tlb_defs.h | 131 +
> arch/kvx/include/asm/tlbflush.h | 58 +
> arch/kvx/include/asm/traps.h | 76 +
> arch/kvx/include/asm/types.h | 12 +
> arch/kvx/include/asm/uaccess.h | 324 ++
> arch/kvx/include/asm/unistd.h | 11 +
> arch/kvx/include/asm/vermagic.h | 12 +
> arch/kvx/include/asm/vmalloc.h | 10 +
> arch/kvx/include/uapi/asm/Kbuild | 1 +
> arch/kvx/include/uapi/asm/bitsperlong.h | 14 +
> arch/kvx/include/uapi/asm/byteorder.h | 12 +
> arch/kvx/include/uapi/asm/cachectl.h | 25 +
> arch/kvx/include/uapi/asm/ptrace.h | 114 +
> arch/kvx/include/uapi/asm/sigcontext.h | 16 +
> arch/kvx/include/uapi/asm/unistd.h | 16 +
> arch/kvx/kernel/Makefile | 27 +
> arch/kvx/kernel/asm-offsets.c | 157 +
> arch/kvx/kernel/break_hook.c | 77 +
> arch/kvx/kernel/common.c | 11 +
> arch/kvx/kernel/cpuinfo.c | 96 +
> arch/kvx/kernel/dame_handler.c | 113 +
> arch/kvx/kernel/debug.c | 64 +
> arch/kvx/kernel/entry.S | 1759 ++++++
> arch/kvx/kernel/ftrace.c | 339 ++
> arch/kvx/kernel/head.S | 612 ++
> arch/kvx/kernel/hw_breakpoint.c | 556 ++
> arch/kvx/kernel/insns.c | 146 +
> arch/kvx/kernel/io.c | 96 +
> arch/kvx/kernel/irq.c | 78 +
> arch/kvx/kernel/jump_label.c | 34 +
> arch/kvx/kernel/kvx_ksyms.c | 29 +
> arch/kvx/kernel/l2_cache.c | 448 ++
> arch/kvx/kernel/mcount.S | 340 ++
> arch/kvx/kernel/module.c | 148 +
> arch/kvx/kernel/perf_event.c | 609 ++
> arch/kvx/kernel/process.c | 212 +
> arch/kvx/kernel/prom.c | 24 +
> arch/kvx/kernel/ptrace.c | 461 ++
> arch/kvx/kernel/reset.c | 37 +
> arch/kvx/kernel/return_address.c | 55 +
> arch/kvx/kernel/setup.c | 178 +
> arch/kvx/kernel/signal.c | 266 +
> arch/kvx/kernel/smp.c | 110 +
> arch/kvx/kernel/smpboot.c | 127 +
> arch/kvx/kernel/stacktrace.c | 173 +
> arch/kvx/kernel/sys_kvx.c | 58 +
> arch/kvx/kernel/syscall_table.c | 19 +
> arch/kvx/kernel/time.c | 242 +
> arch/kvx/kernel/traps.c | 243 +
> arch/kvx/kernel/vdso.c | 87 +
> arch/kvx/kernel/vmlinux.lds.S | 173 +
> arch/kvx/lib/Makefile | 6 +
> arch/kvx/lib/clear_page.S | 40 +
> arch/kvx/lib/copy_page.S | 90 +
> arch/kvx/lib/delay.c | 39 +
> arch/kvx/lib/memcpy.c | 70 +
> arch/kvx/lib/memset.S | 351 ++
> arch/kvx/lib/strlen.S | 122 +
> arch/kvx/lib/usercopy.S | 90 +
> arch/kvx/mm/Makefile | 10 +
> arch/kvx/mm/cacheflush.c | 154 +
> arch/kvx/mm/dma-mapping.c | 95 +
> arch/kvx/mm/extable.c | 24 +
> arch/kvx/mm/fault.c | 264 +
> arch/kvx/mm/hugetlbpage.c | 317 ++
> arch/kvx/mm/init.c | 527 ++
> arch/kvx/mm/kernel_rwx.c | 228 +
> arch/kvx/mm/mmap.c | 31 +
> arch/kvx/mm/mmu.c | 204 +
> arch/kvx/mm/mmu_stats.c | 94 +
> arch/kvx/mm/tlb.c | 433 ++
> arch/kvx/platform/Makefile | 7 +
> arch/kvx/platform/ipi.c | 110 +
> arch/kvx/platform/pwr_ctrl.c | 93 +
> drivers/irqchip/Kconfig | 27 +
> drivers/irqchip/Makefile | 4 +
> drivers/irqchip/irq-kvx-apic-gic.c | 349 ++
> drivers/irqchip/irq-kvx-apic-mailbox.c | 465 ++
> drivers/irqchip/irq-kvx-core-intc.c | 82 +
> drivers/irqchip/irq-kvx-itgen.c | 224 +
> drivers/power/reset/kvx-scall-poweroff.c | 53 +
> include/linux/cpuhotplug.h | 2 +
> include/linux/irqchip/irq-kvx-apic-gic.h | 21 +
> include/linux/irqchip/irq-kvx-apic-mailbox.h | 29 +
> include/linux/irqchip/irq-kvx-itgen.h | 24 +
> include/uapi/linux/audit.h | 1 +
> include/uapi/linux/elf-em.h | 1 +
> include/uapi/linux/elf.h | 1 +
> scripts/gdb/arch/Makefile | 11 +
> scripts/gdb/arch/__init__.py | 1 +
> scripts/gdb/arch/kvx/Makefile | 25 +
> scripts/gdb/arch/kvx/__init__.py | 1 +
> scripts/gdb/arch/kvx/constants.py.in | 74 +
> scripts/gdb/arch/kvx/mmu.py | 199 +
> scripts/gdb/arch/kvx/page_table_walk.py | 207 +
> tools/include/uapi/asm/bitsperlong.h | 2 +
> 175 files changed, 25814 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kvx-core-intc.txt
> create mode 100644 Documentation/devicetree/bindings/perf/kalray-pm.txt
> create mode 100644 Documentation/kvx/kvx-exceptions.txt
> create mode 100644 Documentation/kvx/kvx-iommu.txt
> create mode 100644 Documentation/kvx/kvx-mmu.txt
> create mode 100644 Documentation/kvx/kvx-smp.txt
> create mode 100644 Documentation/kvx/kvx.txt
> create mode 100644 arch/kvx/Kconfig
> create mode 100644 arch/kvx/Kconfig.debug
> create mode 100644 arch/kvx/Makefile
> create mode 100644 arch/kvx/configs/default_defconfig
> create mode 100644 arch/kvx/include/asm/Kbuild
> create mode 100644 arch/kvx/include/asm/asm-prototypes.h
> create mode 100644 arch/kvx/include/asm/atomic.h
> create mode 100644 arch/kvx/include/asm/barrier.h
> create mode 100644 arch/kvx/include/asm/bitops.h
> create mode 100644 arch/kvx/include/asm/bitrev.h
> create mode 100644 arch/kvx/include/asm/break_hook.h
> create mode 100644 arch/kvx/include/asm/bug.h
> create mode 100644 arch/kvx/include/asm/cache.h
> create mode 100644 arch/kvx/include/asm/cacheflush.h
> create mode 100644 arch/kvx/include/asm/clocksource.h
> create mode 100644 arch/kvx/include/asm/cmpxchg.h
> create mode 100644 arch/kvx/include/asm/current.h
> create mode 100644 arch/kvx/include/asm/dame.h
> create mode 100644 arch/kvx/include/asm/debug.h
> create mode 100644 arch/kvx/include/asm/elf.h
> create mode 100644 arch/kvx/include/asm/fixmap.h
> create mode 100644 arch/kvx/include/asm/ftrace.h
> create mode 100644 arch/kvx/include/asm/futex.h
> create mode 100644 arch/kvx/include/asm/hardirq.h
> create mode 100644 arch/kvx/include/asm/hugetlb.h
> create mode 100644 arch/kvx/include/asm/hw_breakpoint.h
> create mode 100644 arch/kvx/include/asm/hw_irq.h
> create mode 100644 arch/kvx/include/asm/insns.h
> create mode 100644 arch/kvx/include/asm/insns_defs.h
> create mode 100644 arch/kvx/include/asm/io.h
> create mode 100644 arch/kvx/include/asm/ipi.h
> create mode 100644 arch/kvx/include/asm/irqflags.h
> create mode 100644 arch/kvx/include/asm/jump_label.h
> create mode 100644 arch/kvx/include/asm/l2_cache.h
> create mode 100644 arch/kvx/include/asm/l2_cache_defs.h
> create mode 100644 arch/kvx/include/asm/linkage.h
> create mode 100644 arch/kvx/include/asm/mem_map.h
> create mode 100644 arch/kvx/include/asm/mmu.h
> create mode 100644 arch/kvx/include/asm/mmu_context.h
> create mode 100644 arch/kvx/include/asm/mmu_stats.h
> create mode 100644 arch/kvx/include/asm/page.h
> create mode 100644 arch/kvx/include/asm/page_size.h
> create mode 100644 arch/kvx/include/asm/pci.h
> create mode 100644 arch/kvx/include/asm/perf_event.h
> create mode 100644 arch/kvx/include/asm/pgalloc.h
> create mode 100644 arch/kvx/include/asm/pgtable-bits.h
> create mode 100644 arch/kvx/include/asm/pgtable.h
> create mode 100644 arch/kvx/include/asm/privilege.h
> create mode 100644 arch/kvx/include/asm/processor.h
> create mode 100644 arch/kvx/include/asm/ptrace.h
> create mode 100644 arch/kvx/include/asm/pwr_ctrl.h
> create mode 100644 arch/kvx/include/asm/rm_fw.h
> create mode 100644 arch/kvx/include/asm/sections.h
> create mode 100644 arch/kvx/include/asm/setup.h
> create mode 100644 arch/kvx/include/asm/sfr.h
> create mode 100644 arch/kvx/include/asm/sfr_defs.h
> create mode 100644 arch/kvx/include/asm/smp.h
> create mode 100644 arch/kvx/include/asm/sparsemem.h
> create mode 100644 arch/kvx/include/asm/spinlock.h
> create mode 100644 arch/kvx/include/asm/spinlock_types.h
> create mode 100644 arch/kvx/include/asm/stackprotector.h
> create mode 100644 arch/kvx/include/asm/stacktrace.h
> create mode 100644 arch/kvx/include/asm/string.h
> create mode 100644 arch/kvx/include/asm/swab.h
> create mode 100644 arch/kvx/include/asm/switch_to.h
> create mode 100644 arch/kvx/include/asm/symbols.h
> create mode 100644 arch/kvx/include/asm/sys_arch.h
> create mode 100644 arch/kvx/include/asm/syscall.h
> create mode 100644 arch/kvx/include/asm/syscalls.h
> create mode 100644 arch/kvx/include/asm/thread_info.h
> create mode 100644 arch/kvx/include/asm/timex.h
> create mode 100644 arch/kvx/include/asm/tlb.h
> create mode 100644 arch/kvx/include/asm/tlb_defs.h
> create mode 100644 arch/kvx/include/asm/tlbflush.h
> create mode 100644 arch/kvx/include/asm/traps.h
> create mode 100644 arch/kvx/include/asm/types.h
> create mode 100644 arch/kvx/include/asm/uaccess.h
> create mode 100644 arch/kvx/include/asm/unistd.h
> create mode 100644 arch/kvx/include/asm/vermagic.h
> create mode 100644 arch/kvx/include/asm/vmalloc.h
> create mode 100644 arch/kvx/include/uapi/asm/Kbuild
> create mode 100644 arch/kvx/include/uapi/asm/bitsperlong.h
> create mode 100644 arch/kvx/include/uapi/asm/byteorder.h
> create mode 100644 arch/kvx/include/uapi/asm/cachectl.h
> create mode 100644 arch/kvx/include/uapi/asm/ptrace.h
> create mode 100644 arch/kvx/include/uapi/asm/sigcontext.h
> create mode 100644 arch/kvx/include/uapi/asm/unistd.h
> create mode 100644 arch/kvx/kernel/Makefile
> create mode 100644 arch/kvx/kernel/asm-offsets.c
> create mode 100644 arch/kvx/kernel/break_hook.c
> create mode 100644 arch/kvx/kernel/common.c
> create mode 100644 arch/kvx/kernel/cpuinfo.c
> create mode 100644 arch/kvx/kernel/dame_handler.c
> create mode 100644 arch/kvx/kernel/debug.c
> create mode 100644 arch/kvx/kernel/entry.S
> create mode 100644 arch/kvx/kernel/ftrace.c
> create mode 100644 arch/kvx/kernel/head.S
> create mode 100644 arch/kvx/kernel/hw_breakpoint.c
> create mode 100644 arch/kvx/kernel/insns.c
> create mode 100644 arch/kvx/kernel/io.c
> create mode 100644 arch/kvx/kernel/irq.c
> create mode 100644 arch/kvx/kernel/jump_label.c
> create mode 100644 arch/kvx/kernel/kvx_ksyms.c
> create mode 100644 arch/kvx/kernel/l2_cache.c
> create mode 100644 arch/kvx/kernel/mcount.S
> create mode 100644 arch/kvx/kernel/module.c
> create mode 100644 arch/kvx/kernel/perf_event.c
> create mode 100644 arch/kvx/kernel/process.c
> create mode 100644 arch/kvx/kernel/prom.c
> create mode 100644 arch/kvx/kernel/ptrace.c
> create mode 100644 arch/kvx/kernel/reset.c
> create mode 100644 arch/kvx/kernel/return_address.c
> create mode 100644 arch/kvx/kernel/setup.c
> create mode 100644 arch/kvx/kernel/signal.c
> create mode 100644 arch/kvx/kernel/smp.c
> create mode 100644 arch/kvx/kernel/smpboot.c
> create mode 100644 arch/kvx/kernel/stacktrace.c
> create mode 100644 arch/kvx/kernel/sys_kvx.c
> create mode 100644 arch/kvx/kernel/syscall_table.c
> create mode 100644 arch/kvx/kernel/time.c
> create mode 100644 arch/kvx/kernel/traps.c
> create mode 100644 arch/kvx/kernel/vdso.c
> create mode 100644 arch/kvx/kernel/vmlinux.lds.S
> create mode 100644 arch/kvx/lib/Makefile
> create mode 100644 arch/kvx/lib/clear_page.S
> create mode 100644 arch/kvx/lib/copy_page.S
> create mode 100644 arch/kvx/lib/delay.c
> create mode 100644 arch/kvx/lib/memcpy.c
> create mode 100644 arch/kvx/lib/memset.S
> create mode 100644 arch/kvx/lib/strlen.S
> create mode 100644 arch/kvx/lib/usercopy.S
> create mode 100644 arch/kvx/mm/Makefile
> create mode 100644 arch/kvx/mm/cacheflush.c
> create mode 100644 arch/kvx/mm/dma-mapping.c
> create mode 100644 arch/kvx/mm/extable.c
> create mode 100644 arch/kvx/mm/fault.c
> create mode 100644 arch/kvx/mm/hugetlbpage.c
> create mode 100644 arch/kvx/mm/init.c
> create mode 100644 arch/kvx/mm/kernel_rwx.c
> create mode 100644 arch/kvx/mm/mmap.c
> create mode 100644 arch/kvx/mm/mmu.c
> create mode 100644 arch/kvx/mm/mmu_stats.c
> create mode 100644 arch/kvx/mm/tlb.c
> create mode 100644 arch/kvx/platform/Makefile
> create mode 100644 arch/kvx/platform/ipi.c
> create mode 100644 arch/kvx/platform/pwr_ctrl.c
> create mode 100644 drivers/irqchip/irq-kvx-apic-gic.c
> create mode 100644 drivers/irqchip/irq-kvx-apic-mailbox.c
> create mode 100644 drivers/irqchip/irq-kvx-core-intc.c
> create mode 100644 drivers/irqchip/irq-kvx-itgen.c
> create mode 100644 drivers/power/reset/kvx-scall-poweroff.c
> create mode 100644 include/linux/irqchip/irq-kvx-apic-gic.h
> create mode 100644 include/linux/irqchip/irq-kvx-apic-mailbox.h
> create mode 100644 include/linux/irqchip/irq-kvx-itgen.h
> create mode 100644 scripts/gdb/arch/Makefile
> create mode 100644 scripts/gdb/arch/__init__.py
> create mode 100644 scripts/gdb/arch/kvx/Makefile
> create mode 100644 scripts/gdb/arch/kvx/__init__.py
> create mode 100644 scripts/gdb/arch/kvx/constants.py.in
> create mode 100644 scripts/gdb/arch/kvx/mmu.py
> create mode 100644 scripts/gdb/arch/kvx/page_table_walk.py
>
> --
> 2.37.2
>
>
>
>
>
--
Thanks,
JeffXie
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-07 6:25 ` Jeff Xie
(?)
@ 2023-01-09 13:21 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-09 13:21 UTC (permalink / raw)
To: Jeff Xie
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
Hi Jeff,
On 1/7/23 07:25, Jeff Xie wrote:
> Hi,
>
> On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
>> [snip]
>>
>> A kvx toolchain can be built using:
>> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
>> $ git clone https://github.com/kalray/build-scripts
>> $ cd build-scripts
>> $ source last.refs
>> $ ./build-kvx-xgcc.sh output
> I would like to build the kvx-xgcc to compile and test the linux
> kernel, but it reported a compile error.
> I wonder what version of gcc you are using.
>
> My build environment:
> VERSION="20.04.2 LTS (Focal Fossa)"
> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
>
>
> Compile error:
> $ ./build-kvx-xgcc.sh output
>
> ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> (first use in this function)
> 38 | #define FIBHEAPKEY_MIN LONG_MIN
> | ^~~~~~~~
> [snip]
What SHA1 of https://github.com/kalray/build-scripts are you using?
We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
issues, I don't understand why it does not work for you, although indeed
the error log you are having pops out on my search engine and seems to
be some well known issue.
If the build-script does not work for you, you can still use the
pre-built toolchains generated by the GitHub automated actions:
https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
means 22.04)
I hope it will work for you.
Regards,
--
Yann
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-09 13:21 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-09 13:21 UTC (permalink / raw)
To: Jeff Xie
Cc: Mark Rutland, Marc Poulhiès, linux-doc, Alexander Shishkin,
Jan Kiszka, Sebastian Reichel, Marius Gligor, Oleg Nesterov,
linux-mm, Louis Morhet, Krzysztof Kozlowski, Luc Michel,
linux-riscv, Ashley Lesdalons, Will Deacon, Ard Biesheuvel,
linux-arch, Marc Zyngier, Thomas Costis, Jonathan Corbet,
Jonathan Borne, Aneesh Kumar K.V, Peter Zijlstra, Ingo Molnar,
Steven Rostedt, Jean-Christophe Pince, Waiman Long,
Clement Leger, Masami Hiramatsu, devicetree, Albert Ou,
Kees Cook, Arnd Bergmann, linux-pm, Samuel Jones, Boqun Feng,
Guillaume Thouvenin, Julian Vetter, Arnaldo Carvalho de Melo,
Julien Hascoet, Jason Baron, Rob Herring, Nick Piggin,
Paul Walmsley, Namhyung Kim, Thomas Gleixner, Josh Poimboeuf,
linux-arm-kernel, Julien Villette, Jules Maselbas,
Christian Brauner, Andrew Morton, Benjamin Mugnier,
Vincent Chardon, Guillaume Missonnier, linux-kernel, Eric Paris,
Alex Michon, linux-perf-users, linux-audit, Palmer Dabbelt,
Eric Biederman, Jiri Olsa, Kieran Bingham, bpf
Hi Jeff,
On 1/7/23 07:25, Jeff Xie wrote:
> Hi,
>
> On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
>> [snip]
>>
>> A kvx toolchain can be built using:
>> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
>> $ git clone https://github.com/kalray/build-scripts
>> $ cd build-scripts
>> $ source last.refs
>> $ ./build-kvx-xgcc.sh output
> I would like to build the kvx-xgcc to compile and test the linux
> kernel, but it reported a compile error.
> I wonder what version of gcc you are using.
>
> My build environment:
> VERSION="20.04.2 LTS (Focal Fossa)"
> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
>
>
> Compile error:
> $ ./build-kvx-xgcc.sh output
>
> ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> (first use in this function)
> 38 | #define FIBHEAPKEY_MIN LONG_MIN
> | ^~~~~~~~
> [snip]
What SHA1 of https://github.com/kalray/build-scripts are you using?
We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
issues, I don't understand why it does not work for you, although indeed
the error log you are having pops out on my search engine and seems to
be some well known issue.
If the build-script does not work for you, you can still use the
pre-built toolchains generated by the GitHub automated actions:
https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
means 22.04)
I hope it will work for you.
Regards,
--
Yann
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-09 13:21 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-09 13:21 UTC (permalink / raw)
To: Jeff Xie
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
Hi Jeff,
On 1/7/23 07:25, Jeff Xie wrote:
> Hi,
>
> On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
>> [snip]
>>
>> A kvx toolchain can be built using:
>> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
>> $ git clone https://github.com/kalray/build-scripts
>> $ cd build-scripts
>> $ source last.refs
>> $ ./build-kvx-xgcc.sh output
> I would like to build the kvx-xgcc to compile and test the linux
> kernel, but it reported a compile error.
> I wonder what version of gcc you are using.
>
> My build environment:
> VERSION="20.04.2 LTS (Focal Fossa)"
> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
>
>
> Compile error:
> $ ./build-kvx-xgcc.sh output
>
> ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> (first use in this function)
> 38 | #define FIBHEAPKEY_MIN LONG_MIN
> | ^~~~~~~~
> [snip]
What SHA1 of https://github.com/kalray/build-scripts are you using?
We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
issues, I don't understand why it does not work for you, although indeed
the error log you are having pops out on my search engine and seems to
be some well known issue.
If the build-script does not work for you, you can still use the
pre-built toolchains generated by the GitHub automated actions:
https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
means 22.04)
I hope it will work for you.
Regards,
--
Yann
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-09 13:21 ` Yann Sionneau
(?)
@ 2023-01-09 15:11 ` Jeff Xie
-1 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-09 15:11 UTC (permalink / raw)
To: Yann Sionneau
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
On Mon, Jan 9, 2023 at 9:21 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
>
> Hi Jeff,
>
> On 1/7/23 07:25, Jeff Xie wrote:
> > Hi,
> >
> > On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
> >> [snip]
> >>
> >> A kvx toolchain can be built using:
> >> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> >> $ git clone https://github.com/kalray/build-scripts
> >> $ cd build-scripts
> >> $ source last.refs
> >> $ ./build-kvx-xgcc.sh output
> > I would like to build the kvx-xgcc to compile and test the linux
> > kernel, but it reported a compile error.
> > I wonder what version of gcc you are using.
> >
> > My build environment:
> > VERSION="20.04.2 LTS (Focal Fossa)"
> > gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
> >
> >
> > Compile error:
> > $ ./build-kvx-xgcc.sh output
> >
> > ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> > ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> > (first use in this function)
> > 38 | #define FIBHEAPKEY_MIN LONG_MIN
> > | ^~~~~~~~
> > [snip]
>
> What SHA1 of https://github.com/kalray/build-scripts are you using?
I have executed the "source last.refs"
> We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
> issues, I don't understand why it does not work for you, although indeed
> the error log you are having pops out on my search engine and seems to
> be some well known issue.
Yes, there are many answers on the web, but none of them solve this problem.
> If the build-script does not work for you, you can still use the
> pre-built toolchains generated by the GitHub automated actions:
> https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
> means 22.04)
Thanks, this is the final solution ;-)
>
> I hope it will work for you.
>
> Regards,
>
> --
>
> Yann
>
>
>
>
>
--
Thanks,
JeffXie
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-09 15:11 ` Jeff Xie
0 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-09 15:11 UTC (permalink / raw)
To: Yann Sionneau
Cc: Mark Rutland, Marc Poulhiès, linux-doc, Alexander Shishkin,
Jan Kiszka, Sebastian Reichel, Marius Gligor, Oleg Nesterov,
linux-mm, Louis Morhet, Krzysztof Kozlowski, Luc Michel,
linux-riscv, Ashley Lesdalons, Will Deacon, Ard Biesheuvel,
linux-arch, Marc Zyngier, Thomas Costis, Jonathan Corbet,
Jonathan Borne, Aneesh Kumar K.V, Peter Zijlstra, Ingo Molnar,
Steven Rostedt, Jean-Christophe Pince, Waiman Long,
Clement Leger, Masami Hiramatsu, devicetree, Albert Ou,
Kees Cook, Arnd Bergmann, linux-pm, Samuel Jones, Boqun Feng,
Guillaume Thouvenin, Julian Vetter, Arnaldo Carvalho de Melo,
Julien Hascoet, Jason Baron, Rob Herring, Nick Piggin,
Paul Walmsley, Namhyung Kim, Thomas Gleixner, Josh Poimboeuf,
linux-arm-kernel, Julien Villette, Jules Maselbas,
Christian Brauner, Andrew Morton, Benjamin Mugnier,
Vincent Chardon, Guillaume Missonnier, linux-kernel, Eric Paris,
Alex Michon, linux-perf-users, linux-audit, Palmer Dabbelt,
Eric Biederman, Jiri Olsa, Kieran Bingham, bpf
On Mon, Jan 9, 2023 at 9:21 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
>
> Hi Jeff,
>
> On 1/7/23 07:25, Jeff Xie wrote:
> > Hi,
> >
> > On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
> >> [snip]
> >>
> >> A kvx toolchain can be built using:
> >> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> >> $ git clone https://github.com/kalray/build-scripts
> >> $ cd build-scripts
> >> $ source last.refs
> >> $ ./build-kvx-xgcc.sh output
> > I would like to build the kvx-xgcc to compile and test the linux
> > kernel, but it reported a compile error.
> > I wonder what version of gcc you are using.
> >
> > My build environment:
> > VERSION="20.04.2 LTS (Focal Fossa)"
> > gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
> >
> >
> > Compile error:
> > $ ./build-kvx-xgcc.sh output
> >
> > ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> > ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> > (first use in this function)
> > 38 | #define FIBHEAPKEY_MIN LONG_MIN
> > | ^~~~~~~~
> > [snip]
>
> What SHA1 of https://github.com/kalray/build-scripts are you using?
I have executed the "source last.refs"
> We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
> issues, I don't understand why it does not work for you, although indeed
> the error log you are having pops out on my search engine and seems to
> be some well known issue.
Yes, there are many answers on the web, but none of them solve this problem.
> If the build-script does not work for you, you can still use the
> pre-built toolchains generated by the GitHub automated actions:
> https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
> means 22.04)
Thanks, this is the final solution ;-)
>
> I hope it will work for you.
>
> Regards,
>
> --
>
> Yann
>
>
>
>
>
--
Thanks,
JeffXie
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-09 15:11 ` Jeff Xie
0 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-09 15:11 UTC (permalink / raw)
To: Yann Sionneau
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
On Mon, Jan 9, 2023 at 9:21 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
>
> Hi Jeff,
>
> On 1/7/23 07:25, Jeff Xie wrote:
> > Hi,
> >
> > On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
> >> [snip]
> >>
> >> A kvx toolchain can be built using:
> >> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> >> $ git clone https://github.com/kalray/build-scripts
> >> $ cd build-scripts
> >> $ source last.refs
> >> $ ./build-kvx-xgcc.sh output
> > I would like to build the kvx-xgcc to compile and test the linux
> > kernel, but it reported a compile error.
> > I wonder what version of gcc you are using.
> >
> > My build environment:
> > VERSION="20.04.2 LTS (Focal Fossa)"
> > gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
> >
> >
> > Compile error:
> > $ ./build-kvx-xgcc.sh output
> >
> > ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> > ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> > (first use in this function)
> > 38 | #define FIBHEAPKEY_MIN LONG_MIN
> > | ^~~~~~~~
> > [snip]
>
> What SHA1 of https://github.com/kalray/build-scripts are you using?
I have executed the "source last.refs"
> We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
> issues, I don't understand why it does not work for you, although indeed
> the error log you are having pops out on my search engine and seems to
> be some well known issue.
Yes, there are many answers on the web, but none of them solve this problem.
> If the build-script does not work for you, you can still use the
> pre-built toolchains generated by the GitHub automated actions:
> https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
> means 22.04)
Thanks, this is the final solution ;-)
>
> I hope it will work for you.
>
> Regards,
>
> --
>
> Yann
>
>
>
>
>
--
Thanks,
JeffXie
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-09 15:11 ` Jeff Xie
@ 2023-01-09 15:30 ` Yann Sionneau
-1 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-09 15:30 UTC (permalink / raw)
To: Jeff Xie
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
Hi Jeff,
On 1/9/23 16:11, Jeff Xie wrote:
> On Mon, Jan 9, 2023 at 9:21 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
>> Hi Jeff,
>>
>> On 1/7/23 07:25, Jeff Xie wrote:
>>> Hi,
>>>
>>> On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
>>>> [snip]
>>>>
>>>> A kvx toolchain can be built using:
>>>> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
>>>> $ git clone https://github.com/kalray/build-scripts
>>>> $ cd build-scripts
>>>> $ source last.refs
>>>> $ ./build-kvx-xgcc.sh output
>>> I would like to build the kvx-xgcc to compile and test the linux
>>> kernel, but it reported a compile error.
>>> I wonder what version of gcc you are using.
>>>
>>> My build environment:
>>> VERSION="20.04.2 LTS (Focal Fossa)"
>>> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
>>>
>>>
>>> Compile error:
>>> $ ./build-kvx-xgcc.sh output
>>>
>>> ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
>>> ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
>>> (first use in this function)
>>> 38 | #define FIBHEAPKEY_MIN LONG_MIN
>>> | ^~~~~~~~
>>> [snip]
>> What SHA1 of https://github.com/kalray/build-scripts are you using?
> I have executed the "source last.refs"
I was referring to the SHA1 of the repo itself (build-scripts).
`last.refs` is a symbolic link which can point to several releases,
depending on "when" you did the clone.
I am asking this because we recently published new toolchains.
I want to make sure which one you are trying to build.
>> We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
>> issues, I don't understand why it does not work for you, although indeed
>> the error log you are having pops out on my search engine and seems to
>> be some well known issue.
> Yes, there are many answers on the web, but none of them solve this problem.
>
>> If the build-script does not work for you, you can still use the
>> pre-built toolchains generated by the GitHub automated actions:
>> https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
>> means 22.04)
> Thanks, this is the final solution ;-)
Good to see it helped :)
Regards,
--
Yann
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-09 15:30 ` Yann Sionneau
0 siblings, 0 replies; 113+ messages in thread
From: Yann Sionneau @ 2023-01-09 15:30 UTC (permalink / raw)
To: Jeff Xie
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
Hi Jeff,
On 1/9/23 16:11, Jeff Xie wrote:
> On Mon, Jan 9, 2023 at 9:21 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
>> Hi Jeff,
>>
>> On 1/7/23 07:25, Jeff Xie wrote:
>>> Hi,
>>>
>>> On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
>>>> [snip]
>>>>
>>>> A kvx toolchain can be built using:
>>>> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
>>>> $ git clone https://github.com/kalray/build-scripts
>>>> $ cd build-scripts
>>>> $ source last.refs
>>>> $ ./build-kvx-xgcc.sh output
>>> I would like to build the kvx-xgcc to compile and test the linux
>>> kernel, but it reported a compile error.
>>> I wonder what version of gcc you are using.
>>>
>>> My build environment:
>>> VERSION="20.04.2 LTS (Focal Fossa)"
>>> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
>>>
>>>
>>> Compile error:
>>> $ ./build-kvx-xgcc.sh output
>>>
>>> ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
>>> ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
>>> (first use in this function)
>>> 38 | #define FIBHEAPKEY_MIN LONG_MIN
>>> | ^~~~~~~~
>>> [snip]
>> What SHA1 of https://github.com/kalray/build-scripts are you using?
> I have executed the "source last.refs"
I was referring to the SHA1 of the repo itself (build-scripts).
`last.refs` is a symbolic link which can point to several releases,
depending on "when" you did the clone.
I am asking this because we recently published new toolchains.
I want to make sure which one you are trying to build.
>> We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
>> issues, I don't understand why it does not work for you, although indeed
>> the error log you are having pops out on my search engine and seems to
>> be some well known issue.
> Yes, there are many answers on the web, but none of them solve this problem.
>
>> If the build-script does not work for you, you can still use the
>> pre-built toolchains generated by the GitHub automated actions:
>> https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
>> means 22.04)
> Thanks, this is the final solution ;-)
Good to see it helped :)
Regards,
--
Yann
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-09 15:30 ` Yann Sionneau
@ 2023-01-09 15:53 ` Jeff Xie
-1 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-09 15:53 UTC (permalink / raw)
To: Yann Sionneau
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
On Mon, Jan 9, 2023 at 11:30 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
>
> Hi Jeff,
>
> On 1/9/23 16:11, Jeff Xie wrote:
> > On Mon, Jan 9, 2023 at 9:21 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
> >> Hi Jeff,
> >>
> >> On 1/7/23 07:25, Jeff Xie wrote:
> >>> Hi,
> >>>
> >>> On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
> >>>> [snip]
> >>>>
> >>>> A kvx toolchain can be built using:
> >>>> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> >>>> $ git clone https://github.com/kalray/build-scripts
> >>>> $ cd build-scripts
> >>>> $ source last.refs
> >>>> $ ./build-kvx-xgcc.sh output
> >>> I would like to build the kvx-xgcc to compile and test the linux
> >>> kernel, but it reported a compile error.
> >>> I wonder what version of gcc you are using.
> >>>
> >>> My build environment:
> >>> VERSION="20.04.2 LTS (Focal Fossa)"
> >>> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
> >>>
> >>>
> >>> Compile error:
> >>> $ ./build-kvx-xgcc.sh output
> >>>
> >>> ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> >>> ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> >>> (first use in this function)
> >>> 38 | #define FIBHEAPKEY_MIN LONG_MIN
> >>> | ^~~~~~~~
> >>> [snip]
> >> What SHA1 of https://github.com/kalray/build-scripts are you using?
> > I have executed the "source last.refs"
>
> I was referring to the SHA1 of the repo itself (build-scripts).
>
> `last.refs` is a symbolic link which can point to several releases,
> depending on "when" you did the clone.
>
> I am asking this because we recently published new toolchains.
>
> I want to make sure which one you are trying to build.
Unfortunately I deleted this repo a few minutes before you asked me ;-(
But I remember that I cloned this repo two days ago.
it should be: last.refs -> refs/4.11.0.refs
> >> We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
> >> issues, I don't understand why it does not work for you, although indeed
> >> the error log you are having pops out on my search engine and seems to
> >> be some well known issue.
> > Yes, there are many answers on the web, but none of them solve this problem.
> >
> >> If the build-script does not work for you, you can still use the
> >> pre-built toolchains generated by the GitHub automated actions:
> >> https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
> >> means 22.04)
> > Thanks, this is the final solution ;-)
> Good to see it helped :)
>
> Regards,
>
> --
>
> Yann
>
>
>
>
>
--
Thanks,
JeffXie
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-09 15:53 ` Jeff Xie
0 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-09 15:53 UTC (permalink / raw)
To: Yann Sionneau
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
On Mon, Jan 9, 2023 at 11:30 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
>
> Hi Jeff,
>
> On 1/9/23 16:11, Jeff Xie wrote:
> > On Mon, Jan 9, 2023 at 9:21 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
> >> Hi Jeff,
> >>
> >> On 1/7/23 07:25, Jeff Xie wrote:
> >>> Hi,
> >>>
> >>> On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
> >>>> [snip]
> >>>>
> >>>> A kvx toolchain can be built using:
> >>>> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> >>>> $ git clone https://github.com/kalray/build-scripts
> >>>> $ cd build-scripts
> >>>> $ source last.refs
> >>>> $ ./build-kvx-xgcc.sh output
> >>> I would like to build the kvx-xgcc to compile and test the linux
> >>> kernel, but it reported a compile error.
> >>> I wonder what version of gcc you are using.
> >>>
> >>> My build environment:
> >>> VERSION="20.04.2 LTS (Focal Fossa)"
> >>> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
> >>>
> >>>
> >>> Compile error:
> >>> $ ./build-kvx-xgcc.sh output
> >>>
> >>> ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> >>> ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> >>> (first use in this function)
> >>> 38 | #define FIBHEAPKEY_MIN LONG_MIN
> >>> | ^~~~~~~~
> >>> [snip]
> >> What SHA1 of https://github.com/kalray/build-scripts are you using?
> > I have executed the "source last.refs"
>
> I was referring to the SHA1 of the repo itself (build-scripts).
>
> `last.refs` is a symbolic link which can point to several releases,
> depending on "when" you did the clone.
>
> I am asking this because we recently published new toolchains.
>
> I want to make sure which one you are trying to build.
Unfortunately I deleted this repo a few minutes before you asked me ;-(
But I remember that I cloned this repo two days ago.
it should be: last.refs -> refs/4.11.0.refs
> >> We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
> >> issues, I don't understand why it does not work for you, although indeed
> >> the error log you are having pops out on my search engine and seems to
> >> be some well known issue.
> > Yes, there are many answers on the web, but none of them solve this problem.
> >
> >> If the build-script does not work for you, you can still use the
> >> pre-built toolchains generated by the GitHub automated actions:
> >> https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
> >> means 22.04)
> > Thanks, this is the final solution ;-)
> Good to see it helped :)
>
> Regards,
>
> --
>
> Yann
>
>
>
>
>
--
Thanks,
JeffXie
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
2023-01-09 15:53 ` Jeff Xie
(?)
@ 2023-01-16 7:31 ` Jeff Xie
-1 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-16 7:31 UTC (permalink / raw)
To: Yann Sionneau
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
On Mon, Jan 9, 2023 at 11:53 PM Jeff Xie <xiehuan09@gmail.com> wrote:
>
> On Mon, Jan 9, 2023 at 11:30 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
> >
> > Hi Jeff,
> >
> > On 1/9/23 16:11, Jeff Xie wrote:
> > > On Mon, Jan 9, 2023 at 9:21 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
> > >> Hi Jeff,
> > >>
> > >> On 1/7/23 07:25, Jeff Xie wrote:
> > >>> Hi,
> > >>>
> > >>> On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
> > >>>> [snip]
> > >>>>
> > >>>> A kvx toolchain can be built using:
> > >>>> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> > >>>> $ git clone https://github.com/kalray/build-scripts
> > >>>> $ cd build-scripts
> > >>>> $ source last.refs
> > >>>> $ ./build-kvx-xgcc.sh output
> > >>> I would like to build the kvx-xgcc to compile and test the linux
> > >>> kernel, but it reported a compile error.
> > >>> I wonder what version of gcc you are using.
> > >>>
> > >>> My build environment:
> > >>> VERSION="20.04.2 LTS (Focal Fossa)"
> > >>> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
> > >>>
> > >>>
> > >>> Compile error:
> > >>> $ ./build-kvx-xgcc.sh output
> > >>>
> > >>> ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> > >>> ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> > >>> (first use in this function)
> > >>> 38 | #define FIBHEAPKEY_MIN LONG_MIN
> > >>> | ^~~~~~~~
> > >>> [snip]
> > >> What SHA1 of https://github.com/kalray/build-scripts are you using?
> > > I have executed the "source last.refs"
> >
> > I was referring to the SHA1 of the repo itself (build-scripts).
> >
> > `last.refs` is a symbolic link which can point to several releases,
> > depending on "when" you did the clone.
> >
> > I am asking this because we recently published new toolchains.
> >
> > I want to make sure which one you are trying to build.
>
> Unfortunately I deleted this repo a few minutes before you asked me ;-(
> But I remember that I cloned this repo two days ago.
> it should be: last.refs -> refs/4.11.0.refs
It should be my own environmental problem.
I reinstalled the system once and it has been able to compile normally ;-)
In the past few days, I have reviewed almost all the codes,
which is very meaningful for me to learn, thank you team.
>
> > >> We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
> > >> issues, I don't understand why it does not work for you, although indeed
> > >> the error log you are having pops out on my search engine and seems to
> > >> be some well known issue.
> > > Yes, there are many answers on the web, but none of them solve this problem.
> > >
> > >> If the build-script does not work for you, you can still use the
> > >> pre-built toolchains generated by the GitHub automated actions:
> > >> https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
> > >> means 22.04)
> > > Thanks, this is the final solution ;-)
> > Good to see it helped :)
> >
> > Regards,
> >
> > --
> >
> > Yann
> >
> >
> >
> >
> >
>
>
> --
> Thanks,
> JeffXie
--
Thanks,
JeffXie
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-16 7:31 ` Jeff Xie
0 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-16 7:31 UTC (permalink / raw)
To: Yann Sionneau
Cc: Mark Rutland, Marc Poulhiès, linux-doc, Alexander Shishkin,
Jan Kiszka, Sebastian Reichel, Marius Gligor, Oleg Nesterov,
linux-mm, Louis Morhet, Krzysztof Kozlowski, Luc Michel,
linux-riscv, Ashley Lesdalons, Will Deacon, Ard Biesheuvel,
linux-arch, Marc Zyngier, Thomas Costis, Jonathan Corbet,
Jonathan Borne, Aneesh Kumar K.V, Peter Zijlstra, Ingo Molnar,
Steven Rostedt, Jean-Christophe Pince, Waiman Long,
Clement Leger, Masami Hiramatsu, devicetree, Albert Ou,
Kees Cook, Arnd Bergmann, linux-pm, Samuel Jones, Boqun Feng,
Guillaume Thouvenin, Julian Vetter, Arnaldo Carvalho de Melo,
Julien Hascoet, Jason Baron, Rob Herring, Nick Piggin,
Paul Walmsley, Namhyung Kim, Thomas Gleixner, Josh Poimboeuf,
linux-arm-kernel, Julien Villette, Jules Maselbas,
Christian Brauner, Andrew Morton, Benjamin Mugnier,
Vincent Chardon, Guillaume Missonnier, linux-kernel, Eric Paris,
Alex Michon, linux-perf-users, linux-audit, Palmer Dabbelt,
Eric Biederman, Jiri Olsa, Kieran Bingham, bpf
On Mon, Jan 9, 2023 at 11:53 PM Jeff Xie <xiehuan09@gmail.com> wrote:
>
> On Mon, Jan 9, 2023 at 11:30 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
> >
> > Hi Jeff,
> >
> > On 1/9/23 16:11, Jeff Xie wrote:
> > > On Mon, Jan 9, 2023 at 9:21 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
> > >> Hi Jeff,
> > >>
> > >> On 1/7/23 07:25, Jeff Xie wrote:
> > >>> Hi,
> > >>>
> > >>> On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
> > >>>> [snip]
> > >>>>
> > >>>> A kvx toolchain can be built using:
> > >>>> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> > >>>> $ git clone https://github.com/kalray/build-scripts
> > >>>> $ cd build-scripts
> > >>>> $ source last.refs
> > >>>> $ ./build-kvx-xgcc.sh output
> > >>> I would like to build the kvx-xgcc to compile and test the linux
> > >>> kernel, but it reported a compile error.
> > >>> I wonder what version of gcc you are using.
> > >>>
> > >>> My build environment:
> > >>> VERSION="20.04.2 LTS (Focal Fossa)"
> > >>> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
> > >>>
> > >>>
> > >>> Compile error:
> > >>> $ ./build-kvx-xgcc.sh output
> > >>>
> > >>> ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> > >>> ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> > >>> (first use in this function)
> > >>> 38 | #define FIBHEAPKEY_MIN LONG_MIN
> > >>> | ^~~~~~~~
> > >>> [snip]
> > >> What SHA1 of https://github.com/kalray/build-scripts are you using?
> > > I have executed the "source last.refs"
> >
> > I was referring to the SHA1 of the repo itself (build-scripts).
> >
> > `last.refs` is a symbolic link which can point to several releases,
> > depending on "when" you did the clone.
> >
> > I am asking this because we recently published new toolchains.
> >
> > I want to make sure which one you are trying to build.
>
> Unfortunately I deleted this repo a few minutes before you asked me ;-(
> But I remember that I cloned this repo two days ago.
> it should be: last.refs -> refs/4.11.0.refs
It should be my own environmental problem.
I reinstalled the system once and it has been able to compile normally ;-)
In the past few days, I have reviewed almost all the codes,
which is very meaningful for me to learn, thank you team.
>
> > >> We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
> > >> issues, I don't understand why it does not work for you, although indeed
> > >> the error log you are having pops out on my search engine and seems to
> > >> be some well known issue.
> > > Yes, there are many answers on the web, but none of them solve this problem.
> > >
> > >> If the build-script does not work for you, you can still use the
> > >> pre-built toolchains generated by the GitHub automated actions:
> > >> https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
> > >> means 22.04)
> > > Thanks, this is the final solution ;-)
> > Good to see it helped :)
> >
> > Regards,
> >
> > --
> >
> > Yann
> >
> >
> >
> >
> >
>
>
> --
> Thanks,
> JeffXie
--
Thanks,
JeffXie
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply [flat|nested] 113+ messages in thread* Re: [RFC PATCH 00/25] Upstream kvx Linux port
@ 2023-01-16 7:31 ` Jeff Xie
0 siblings, 0 replies; 113+ messages in thread
From: Jeff Xie @ 2023-01-16 7:31 UTC (permalink / raw)
To: Yann Sionneau
Cc: Arnd Bergmann, Albert Ou, Alexander Shishkin, Andrew Morton,
Aneesh Kumar K.V, Ard Biesheuvel, Arnaldo Carvalho de Melo,
Boqun Feng, bpf, Christian Brauner, devicetree, Eric Biederman,
Eric Paris, Ingo Molnar, Jan Kiszka, Jason Baron, Jiri Olsa,
Jonathan Corbet, Josh Poimboeuf, Kees Cook, Kieran Bingham,
Krzysztof Kozlowski, linux-arch, linux-arm-kernel, linux-audit,
linux-doc, linux-kernel, linux-mm, linux-perf-users, linux-pm,
linux-riscv, Marc Zyngier, Mark Rutland, Masami Hiramatsu,
Namhyung Kim, Nick Piggin, Oleg Nesterov, Palmer Dabbelt,
Paul Moore, Paul Walmsley, Peter Zijlstra, Rob Herring,
Sebastian Reichel, Steven Rostedt, Thomas Gleixner, Waiman Long,
Will Deacon, Alex Michon, Ashley Lesdalons, Benjamin Mugnier,
Clement Leger, Guillaume Missonnier, Guillaume Thouvenin,
Jean-Christophe Pince, Jonathan Borne, Jules Maselbas,
Julian Vetter, Julien Hascoet, Julien Villette, Louis Morhet,
Luc Michel, Marc Poulhiès, Marius Gligor, Samuel Jones,
Thomas Costis, Vincent Chardon
On Mon, Jan 9, 2023 at 11:53 PM Jeff Xie <xiehuan09@gmail.com> wrote:
>
> On Mon, Jan 9, 2023 at 11:30 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
> >
> > Hi Jeff,
> >
> > On 1/9/23 16:11, Jeff Xie wrote:
> > > On Mon, Jan 9, 2023 at 9:21 PM Yann Sionneau <ysionneau@kalray.eu> wrote:
> > >> Hi Jeff,
> > >>
> > >> On 1/7/23 07:25, Jeff Xie wrote:
> > >>> Hi,
> > >>>
> > >>> On Wed, Jan 4, 2023 at 1:01 AM Yann Sionneau <ysionneau@kalray.eu> wrote:
> > >>>> [snip]
> > >>>>
> > >>>> A kvx toolchain can be built using:
> > >>>> # install dependencies: texinfo bison flex libgmp-dev libmpc-dev libmpfr-dev
> > >>>> $ git clone https://github.com/kalray/build-scripts
> > >>>> $ cd build-scripts
> > >>>> $ source last.refs
> > >>>> $ ./build-kvx-xgcc.sh output
> > >>> I would like to build the kvx-xgcc to compile and test the linux
> > >>> kernel, but it reported a compile error.
> > >>> I wonder what version of gcc you are using.
> > >>>
> > >>> My build environment:
> > >>> VERSION="20.04.2 LTS (Focal Fossa)"
> > >>> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
> > >>>
> > >>>
> > >>> Compile error:
> > >>> $ ./build-kvx-xgcc.sh output
> > >>>
> > >>> ../../binutils/libiberty/fibheap.c: In function ‘fibheap_replace_key_data’:
> > >>> ../../binutils/libiberty/fibheap.c:38:24: error: ‘LONG_MIN’ undeclared
> > >>> (first use in this function)
> > >>> 38 | #define FIBHEAPKEY_MIN LONG_MIN
> > >>> | ^~~~~~~~
> > >>> [snip]
> > >> What SHA1 of https://github.com/kalray/build-scripts are you using?
> > > I have executed the "source last.refs"
> >
> > I was referring to the SHA1 of the repo itself (build-scripts).
> >
> > `last.refs` is a symbolic link which can point to several releases,
> > depending on "when" you did the clone.
> >
> > I am asking this because we recently published new toolchains.
> >
> > I want to make sure which one you are trying to build.
>
> Unfortunately I deleted this repo a few minutes before you asked me ;-(
> But I remember that I cloned this repo two days ago.
> it should be: last.refs -> refs/4.11.0.refs
It should be my own environmental problem.
I reinstalled the system once and it has been able to compile normally ;-)
In the past few days, I have reviewed almost all the codes,
which is very meaningful for me to learn, thank you team.
>
> > >> We are building our toolchain on Ubuntu 18.04 / 20.04 and 22.04 without
> > >> issues, I don't understand why it does not work for you, although indeed
> > >> the error log you are having pops out on my search engine and seems to
> > >> be some well known issue.
> > > Yes, there are many answers on the web, but none of them solve this problem.
> > >
> > >> If the build-script does not work for you, you can still use the
> > >> pre-built toolchains generated by the GitHub automated actions:
> > >> https://github.com/kalray/build-scripts/releases/tag/v4.11.1 ("latest"
> > >> means 22.04)
> > > Thanks, this is the final solution ;-)
> > Good to see it helped :)
> >
> > Regards,
> >
> > --
> >
> > Yann
> >
> >
> >
> >
> >
>
>
> --
> Thanks,
> JeffXie
--
Thanks,
JeffXie
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 113+ messages in thread