* [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
@ 2023-05-18 13:09 guoren
2023-05-18 13:09 ` [RFC PATCH 01/22] riscv: vdso: Unify vdso32 & compat_vdso into vdso/Makefile guoren
` (23 more replies)
0 siblings, 24 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:09 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
This patch series adds s64ilp32 support to riscv. The term s64ilp32
means smode-xlen=64 and -mabi=ilp32 (ints, longs, and pointers are all
32-bit), i.e., running 32-bit Linux kernel on pure 64-bit supervisor
mode. There have been many 64ilp32 abis existing, such as mips-n32 [1],
arm-aarch64ilp32 [2], and x86-x32 [3], but they are all about userspace.
Thus, this should be the first time running a 32-bit Linux kernel with
the 64ilp32 ABI at supervisor mode (If not, correct me).
Why 32-bit Linux?
=================
The motivation for using a 32-bit Linux kernel is to reduce memory
footprint and meet the small capacity of DDR & cache requirement
(e.g., 64/128MB SIP SoC).
Here are the 32-bit v.s. 64-bit Linux kernel data type comparison
summary:
32-bit 64-bit
sizeof(page): 32bytes 64bytes
sizeof(list_head): 8bytes 16bytes
sizeof(hlist_head): 8bytes 16bytes
sizeof(vm_area): 68bytes 136bytes
...
The size of ilp32's long & pointer is just half of lp64's (rv64 default
abi - longs and pointers are all 64-bit). This significant difference
in data type causes different memory & cache footprint costs. Here is
the comparison measurement between s32ilp32, s64ilp32, and s64lp64 in
the same 128MB qemu system environment:
Rootfs:
u32ilp32 - Using the same 32-bit userspace rootfs.ext2 (UXL=32) binary
from buildroot 2023.02-rc3, qemu_riscv32_virt_defconfig
Linux:
s32ilp32 - Linux version 6.3.0-rc1 (124MB)
rv32_defconfig: $(Q)$(MAKE) -f $(srctree)/Makefile
defconfig 32-bit.config
s64lp64 - Linux version 6.3.0-rc1 (126MB)
defconfig: $(Q)$(MAKE) -f $(srctree)/Makefile defconfig
s64ilp32 - Linux version 6.3.0-rc1 (126MB)
rv64ilp32_defconfig: $(Q)$(MAKE) -f $(srctree)/Makefile
defconfig 64ilp32.config
Opensbi:
m64lp64 - (2MB) OpenSBI v1.2-80-g4b28afc98bbe
m32ilp32 - (4MB) OpenSBI v1.2-80-g4b28afc98bbe
+----------------------------------------+--------
| u32ilp32 |
| UXL=32 | Rootfs
+----------------------------------------+--------
| +----------+ +---------+ | +---------+ |
| | s64ilp32 | | s64lp64 | | | s32ilp32| |
| | SXL=64 | | SXL=64 | | | SXL=32 | | Linux
| +----------+ +---------+ | +---------+ |
+----------------------------------------+--------
| +----------------------+ | +---------+ |
| | m64lp64 | | | m32ilp32| |
| | MXL=64 | | | MXL=32 | | Opensbi
| +----------------------+ | +---------+ |
+----------------------------------------+--------
| +----------------------+ | +---------+ |
| | qemu-rv64 | | |qemu-rv32| | HW
| +----------------------+ | +---------+ |
+----------------------------------------+--------
Mem-usage:
(s32ilp32) # free
total used free shared buff/cache available
Mem: 100040 8380 88244 44 3416 88080
(s64lp64) # free
total used free shared buff/cache available
Mem: 91568 11848 75796 44 3924 75952
(s64ilp32) # free
total used free shared buff/cache available
Mem: 101952 8528 90004 44 3420 89816
^^^^^
It's a rough measurement based on the current default config without any
modification, and 32-bit (s32ilp32, s64ilp32) saved more than 16% memory
to 64-bit (s64lp64). But s32ilp32 & s64ilp32 have a similar memory
footprint (about 0.33% difference), meaning s64ilp32 has a big chance to
replace s32ilp32 on the 64-bit machine.
Why s64ilp32?
=============
The current RISC-V has the profiles of RVA20S64, RVA22S64, and RVA23S64
(ongoing) [4], but no RVA**S32 profile exists or any ongoing plan. That
means when a vendor wants to produce a 32-bit s-mode RISC-V Application
Processor, they have no shape to follow. Therefore, many cheap riscv
chips have come out but follow the RVA2xS64 profiles, such as Allwinner
D1/D1s/F133 [5], SOPHGO CV1800B [6], Canaan Kendryte k230 [7], and
Bouffalo Lab BL808 which are typically cortex a7/a35/a53 product
scenarios. The D1 & CV1800B & BL808 didn't support UXL=32 (32-bit U-mode),
so they need a new u64ilp32 userspace ABI which has no software ecosystem
for the current. Thus, the first landing of s64ilp32 would be on Canaan
Kendryte k230, which has c908 with rv64gcv and compat user mode
(sstatus.uxl=32/64), which could support the existing rv32 userspace
software ecosystem.
Another reason for inventing s64ilp32 is performance benefits and
simplify 64-bit CPU hardware design (v.s. s32ilp32).
Why s64ilp32 has better performance?
====================================
Generally speaking, we should build a 32-bit hardware s-mode to run
32-bit Linux on a 64-bit processor (such as Linux-arm32 on cortex-a53).
Or only use old 32ilp32-abi on a 64-bit machine (such as mips
SYS_SUPPORTS_32BIT_KERNEL). These can't reuse performance-related
features and instructions of the 64-bit hardware, such as 64-bit ALU,
AMO, and LD/SD, which would cause significant performance gaps on many
Linux features:
- memcpy/memset/strcmp (s64ilp32 has half of the instructions count
and double the bandwidth of load/store instructions than s32ilp32.)
- ebpf JIT is a 64-bit virtual ISA, which is not suitable
for mapping to s32ilp32.
- Atomic64 (s64ilp32 has the exact native instructions mapping as
s64lp64, but s32ilp32 only uses generic_atomic64, a tradeoff &
limited software solution.)
- 64-bit native arithmetic instructions for "long long" type
- Support cmxchg_double for slub (The 2nd 32-bit Linux
supports the feature, the 1st is i386.)
- ...
Compared with the user space ecosystem, the 32-bit Linux kernel is more
eager to need 64ilp32 to improve performance because the Linux kernel
can't utilize float-point/vector features of the ISA.
Let's look at performance from another perspective (s64ilp32 v.s.
s64lp64). Just as the first chapter said, the pointer size of ilp32 is
half of the lp64, and it reduces the size of the critical data structs
(e.g., page, list, ...). That means the cache of using ilp32 could
contain double data that lp64 with the same cache capacity, which is a
natural advantage of 32-bit.
Why s64ilp32 simplifies CPU design?
===================================
Yes, there are a lot of runing 32-bit Linux on 64-bit hardware examples
in history, such as arm cortex a35/a53/a55, which implements the 32-bit
EL1/EL2/EL3 hardware mode to support 32-bit Linux. We could follow Arm's
style, but riscv could choose another better way. Compared to UXL=32,
the MXL=SXL=32 has many CSR-related hardware functionalities, which
causes a lot of effort to mix them into 64-bit hardware. The s64ilp32
works on MXL=SXL=64 mode, so the CPU vendors needn't implement 32-bit
machine and supervisor modes.
How does s64ilp32 work?
=======================
The s64ilp32 is the same as the s64lp64 compat mode from a hardware
view, i.e., MXL=SXL=64 + UXL=32. Because the s64ilp32 uses CONFIG_32BIT
of Linux, it only supports u32ilp32 abi user space, the current standard
rv32 software ecosystem, and it can't work with u64lp64 abi (I don't
want that complex and useless stuff). But it may work with u64ilp32 in the
future; now, the s64ilp32 depends on the UXL=32 feature of the hardware.
The 64ilp32 gcc still uses sign-extend lw & auipc to generate address
variables because inserting zero-extend instructions to mask the highest
32-bit would cause significant code size and performance problems. Thus,
we invented an OS approach to solve the problem:
- When satp=bare and start physical address < 2GB, there is no sign-extend
address problem.
- When satp=bare and start physical address > 2GB, we need zjpm liked
hardware extensions to mask high 32bit.
(Fortunately, all existed SoCs' (D1/D1s/F133, CV1800B, k230, BL808)
start physical address < 2GB.)
- When satp=sv39, we invent double mapping to make the sign-extended
virtual address the same as the zero-extended virtual address.
+--------+ +---------+ +--------+
| | +--| 511:PUD1| | |
| | | +---------+ | |
| | | | 510:PUD0|--+ | |
| | | +---------+ | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | INVALID | | | |
| | | | | | | |
| .... | | | | | | .... |
| | | | | | | |
| | | +---------+ | | |
| | +--| 3:PUD1 | | | |
| | | +---------+ | | |
| | | | 2:PUD0 |--+ | |
| | | +---------+ | | |
| | | |1:USR_PUD| | | |
| | | +---------+ | | |
| | | |0:USR_PUD| | | |
+--------+<--+ +---------+ +-->+--------+
PUD1 ^ PGD PUD0
1GB | 4GB 1GB
|
+----------+
| Sv39 PGDP|
+----------+
SATP
The size of xlen was always equal to the pointer/long size before
s64ilp32 emerged. So we need to introduce a new type of data - xlen_t,
which could deal with CSR-related and callee-save/restore operations.
Some kernel features use 32BIT/64BIT to determine the exact ISA, such as
ebpf JIT would map to rv32 ISA when CONFIG_32BIT=y. But s64ilp32 needs
the ebpf JIT map to rv64 ISA when CONFIG_32BIT=y and we need to use
another config to distinguish the difference.
More detials, please review the path series.
How to run s64ilp32?
====================
GNU toolchain
-------------
git clone https://github.com/Liaoshihua/riscv-gnu-toolchain.git
cd riscv-gnu-toolchain
./configure --prefix="$PWD/opt-rv64-ilp32/" --with-arch=rv64imac --with-abi=ilp32
make linux
export PATH=$PATH:$PWD/opt-rv64-ilp32/bin/
Opensbi
-------
git clone https://github.com/riscv-software-src/opensbi.git
CROSS_COMPILE=riscv64-unknown-linux-gnu- make PLATFORM=generic
Linux kernel
------------
git clone https://github.com/guoren83/linux.git -b s64ilp32
cd linux
make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- rv64ilp32_defconfig
make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- all
Rootfs
------
git clone git://git.busybox.net/buildroot
cd buildroot
make qemu_riscv32_virt_defconfig
make
Qemu
----
git clone https://github.com/plctlab/plct-qemu.git -b plct-s64ilp32-dev
cd plct-qemu
mkdir build
cd build
../qemu/configure --target-list="riscv64-softmmu riscv32-softmmu"
make
Run
---
./qemu-system-riscv64 -cpu rv64 -M virt -m 128m -nographic -bios fw_dynamic.bin -kernel Image -drive file=rootfs.ext2,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 -append "rootwait root=/dev/vda ro console=ttyS0 earlycon=sbi" -netdev user,id=net0 -device virtio-net-device,netdev=net0
OpenSBI v1.2-119-gdc1c7db05e07
____ _____ ____ _____
/ __ \ / ____| _ \_ _|
| | | |_ __ ___ _ __ | (___ | |_) || |
| | | | '_ \ / _ \ '_ \ \___ \| _ < | |
| |__| | |_) | __/ | | |____) | |_) || |_
\____/| .__/ \___|_| |_|_____/|___/_____|
| |
|_|
Platform Name : riscv-virtio,qemu
Platform Features : medeleg
Platform HART Count : 1
Platform IPI Device : aclint-mswi
Platform Timer Device : aclint-mtimer @ 10000000Hz
Platform Console Device : uart8250
Platform HSM Device : ---
Platform PMU Device : ---
Platform Reboot Device : sifive_test
Platform Shutdown Device : sifive_test
Platform Suspend Device : ---
Platform CPPC Device : ---
Firmware Base : 0x60000000
Firmware Size : 360 KB
Firmware RW Offset : 0x40000
Runtime SBI Version : 1.0
Domain0 Name : root
Domain0 Boot HART : 0
Domain0 HARTs : 0*
Domain0 Region00 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
Domain0 Region01 : 0x0000000060040000-0x000000006005ffff M: (R,W) S/U: ()
Domain0 Region02 : 0x0000000060000000-0x000000006003ffff M: (R,X) S/U: ()
Domain0 Region03 : 0x0000000000000000-0xffffffffffffffff M: (R,W,X) S/U: (R,W,X)
Domain0 Next Address : 0x0000000060200000
Domain0 Next Arg1 : 0x0000000067e00000
Domain0 Next Mode : S-mode
Domain0 SysReset : yes
Domain0 SysSuspend : yes
Boot HART ID : 0
Boot HART Domain : root
Boot HART Priv Version : v1.12
Boot HART Base ISA : rv64imafdch
Boot HART ISA Extensions : time,sstc
Boot HART PMP Count : 16
Boot HART PMP Granularity : 4
Boot HART PMP Address Bits: 54
Boot HART MHPM Count : 16
Boot HART MIDELEG : 0x0000000000001666
Boot HART MEDELEG : 0x0000000000f0b509
[ 0.000000] Linux version 6.3.0-rc1-00086-gc8d2fedb997a (guoren@fedora) (riscv64-unknown-linux-gnu-gcc (g5e578a16201f) 13.0.1 20230206 (experimental), GNU ld (GNU Binutils) 2.40.50.20230205) #1 SMP Sun May 14 10:46:42 EDT 2023
[ 0.000000] random: crng init done
[ 0.000000] OF: fdt: Ignoring memory range 0x60000000 - 0x60200000
[ 0.000000] Machine model: riscv-virtio,qemu
[ 0.000000] efi: UEFI not found.
[ 0.000000] OF: reserved mem: 0x60000000..0x6003ffff (256 KiB) map non-reusable mmode_resv1@60000000
[ 0.000000] OF: reserved mem: 0x60040000..0x6005ffff (128 KiB) map non-reusable mmode_resv0@60040000
[ 0.000000] Zone ranges:
[ 0.000000] Normal [mem 0x0000000060200000-0x0000000067ffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000060200000-0x0000000067ffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000060200000-0x0000000067ffffff]
[ 0.000000] On node 0, zone Normal: 512 pages in unavailable ranges
[ 0.000000] SBI specification v1.0 detected
[ 0.000000] SBI implementation ID=0x1 Version=0x10002
[ 0.000000] SBI TIME extension detected
[ 0.000000] SBI IPI extension detected
[ 0.000000] SBI RFENCE extension detected
[ 0.000000] SBI SRST extension detected
[ 0.000000] SBI HSM extension detected
[ 0.000000] riscv: base ISA extensions acdfhim
[ 0.000000] riscv: ELF capabilities acdfim
[ 0.000000] percpu: Embedded 13 pages/cpu s24352 r8192 d20704 u53248
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 31941
[ 0.000000] Kernel command line: rootwait root=/dev/vda ro console=ttyS0 earlycon=sbi norandmaps
[ 0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes, linear)
[ 0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes, linear)
[ 0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] fixmap : 0x9ce00000 - 0x9d000000 (2048 kB)
[ 0.000000] pci io : 0x9d000000 - 0x9e000000 ( 16 MB)
[ 0.000000] vmemmap : 0x9e000000 - 0xa0000000 ( 32 MB)
[ 0.000000] vmalloc : 0xa0000000 - 0xc0000000 ( 512 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xc7e00000 ( 126 MB)
[ 0.000000] Memory: 97748K/129024K available (8699K kernel code, 8867K rwdata, 4096K rodata, 4204K init, 361K bss, 31276K reserved, 0K cma-reserved)
...
Starting network: udhcpc: started, v1.36.0
udhcpc: broadcasting discover
udhcpc: broadcasting select for 10.0.2.15, server 10.0.2.2
udhcpc: lease of 10.0.2.15 obtained from 10.0.2.2, lease time 86400
deleting routers
adding dns 10.0.2.3
OK
Welcome to Buildroot
buildroot login: root
# cat /proc/cpuinfo
processor : 0
hart : 0
isa : rv64imafdch_zihintpause_zbb_sstc
mmu : sv39
mvendorid : 0x0
marchid : 0x70232
mimpid : 0x70232
# uname -a
Linux buildroot 6.3.0-rc1-00086-gc8d2fedb997a #1 SMP Sun May 14 10:46:42 EDT 2023 riscv32 GNU/Linux
# ls /lib/
ld-linux-riscv32-ilp32d.so.1 libgcc_s.so.1
libanl.so.1 libm.so.6
libatomic.so libnss_dns.so.2
libatomic.so.1 libnss_files.so.2
libatomic.so.1.2.0 libpthread.so.0
libc.so.6 libresolv.so.2
libcrypt.so.1 librt.so.1
libdl.so.2 libutil.so.1
libgcc_s.so modules
# cat /proc/99/maps
0000000055554000-0000000055634000 r-xp 00000000 00000000fe:00 17 /bin/busybox
0000000055634000-0000000055636000 r--p 00000000df000 00000000fe:00 17 /bin/busybox
0000000055636000-0000000055637000 rw-p 00000000e1000 00000000fe:00 17 /bin/busybox
0000000055637000-0000000055659000 rw-p 00000000 00:00 0 [heap]
0000000077e8d000-0000000077fbe000 r-xp 00000000 00000000fe:00 137 /lib/libc.so.6
0000000077fbe000-0000000077fbf000 ---p 00000000131000 00000000fe:00 137 /lib/libc.so.6
0000000077fbf000-0000000077fc1000 r--p 00000000131000 00000000fe:00 137 /lib/libc.so.6
0000000077fc1000-0000000077fc2000 rw-p 00000000133000 00000000fe:00 137 /lib/libc.so.6
0000000077fc2000-0000000077fcc000 rw-p 00000000 00:00 0
0000000077fcc000-0000000077fd4000 r-xp 00000000 00000000fe:00 146 /lib/libresolv.so.2
0000000077fd4000-0000000077fd5000 ---p 000000008000 00000000fe:00 146 /lib/libresolv.so.2
0000000077fd5000-0000000077fd6000 r--p 000000008000 00000000fe:00 146 /lib/libresolv.so.2
0000000077fd6000-0000000077fd7000 rw-p 000000009000 00000000fe:00 146 /lib/libresolv.so.2
0000000077fd7000-0000000077fd9000 rw-p 00000000 00:00 0
0000000077fd9000-0000000077fdb000 r--p 00000000 00:00 0 [vvar]
0000000077fdb000-0000000077fdd000 r-xp 00000000 00:00 0 [vdso]
0000000077fdd000-0000000077ffc000 r-xp 00000000 00000000fe:00 132 /lib/ld-linux-riscv32-ilp32d.so.1
0000000077ffd000-0000000077ffe000 r--p 000000001f000 00000000fe:00 132 /lib/ld-linux-riscv32-ilp32d.so.1
0000000077ffe000-0000000077fff000 rw-p 0000000020000 00000000fe:00 132 /lib/ld-linux-riscv32-ilp32d.so.1
000000007ffde000-000000007ffff000 rw-p 00000000 00:00 0 [stack]
Other resources
===============
OpenEuler riscv32 rootfs
------------------------
The OpenEuler riscv32 rootfs you can download from here:
https://repo.tarsier-infra.com/openEuler-RISC-V/obs/archive/rv32/openeuler-image-qemu-riscv32-20221111070036.rootfs.ext4
(Made by Junqiang Wang)
Debain riscv32 rootfs
---------------------
The Debian riscv32 rootfs you can download from here:
https://github.com/yuzibo/riscv32
(Made by Bo YU and Han Gao)
Fedora riscv32 rootfs
---------------------
https://fedoraproject.org/wiki/Architectures/RISC-V/RV32
(Made by Wei Fu)
LLVM 64ilp32
------------
git clone https://github.com/luxufan/llvm-project.git -b rv64-ilp32
cd llvm-project
mkdir build && cd build
cmake ../llvm -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=“X86;RISCV" -DLLVM_ENABLE_PROJECTS="clang;lld"
ninja all
(LLVM development status is that CC=clang can compile the kernel with
LLVM=1 but has not yet booted successfully.)
Patch organization
==================
This series depends on 64ilp32 toolchain patches that are not upstream
yet.
PATCH [0-1] unify vdso32 & compat_vdso
PATCH [2] adds time-related vDSO common flow for vdso32
PATCH [3] adds s64ilp32 support of clocksource driver
PATCH [5] adds s64ilp32 support of irqchip driver
PATCH [4,6-12] add basic data types and compiling framework
PATCH [13] adds MMU_SV39 support
PATCH [14] adds native atomic64
PATCH [15] adds TImode
PATCH [16] adds cmpxchg_double
PATCH [17-19] cleanup kconfig & add defconfig
PATCH [20-21] fix temporary compiler problems
Open issues
===========
Callee saved the register width
-------------------------------
For 64-bit ISA (including 64lp64, 64ilp32), callee can't determine the
correct width used in the register, so they saved the maximum width of
the ISA register, i.e., xlen size. We also found this rule in x86-x32,
mips-n32, and aarch64ilp32, which comes from 64lp64. See PATCH [20]
Here are two downsides of this:
- It would cause a difference with 32ilp32's stack frame, and s64ilp32
reuses 32ilp32 software stack. Thus, many additional compatible
problems would happen during the porting of 64ilp32 software.
- It also increases the budget of the stack usage.
<setup_vm>:
auipc a3,0xff3fb
add a3,a3,1234 # c0000000
li a5,-1
lui a4,0xc0000
addw sp,sp,-96
srl a5,a5,0x20
subw a4,a4,a3
auipc a2,0x111a
add a2,a2,1212 # c1d1f000
sd s0,80(sp)----+
sd s1,72(sp) |
sd s2,64(sp) |
sd s7,24(sp) |
sd s8,16(sp) |
sd s9,8(sp) |-> All <= 32b widths, but occupy 64b
sd ra,88(sp) | stack space.
sd s3,56(sp) | Affect memory footprint & cache
sd s4,48(sp) | performance.
sd s5,40(sp) |
sd s6,32(sp) |
sd s10,0(sp)----+
sll a1,a4,0x20
subw a2,a2,a3
and a4,a4,a5
So here is a proposal to riscv 64ilp32 ABI:
- Let the compiler prevent callee saving ">32b variables" in
callee-registers. (Q: We need to measure, how the influence of
64b variables cross function call?)
EF_RISCV_X32
------------
We add an e_flag (EF_RISCV_X32) to distinguish the 32-bit ELF, which
occupies BIT[6] of the e_flags layout.
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: REL (Relocatable file)
Machine: RISC-V
Version: 0x1
Entry point address: 0x0
Start of program headers: 0 (bytes into file)
Start of section headers: 24620 (bytes into file)
Flags: 0x21, RVC, X32, soft-float ABI
^^^
64-bit Optimization problem
---------------------------
There is an existing problem in 64ilp32 gcc that combines two pointers
in one register. Liao is solving that problem. Before he finishes the
job, we could prevent it with a simple noinline attribute, fortunately.
struct path {
struct vfsmount *mnt;
struct dentry *dentry;
} __randomize_layout;
struct nameidata {
struct path path;
...
struct path root;
...
} __randomize_layout;
struct nameidata *nd
...
nd->path = nd->root;
6c88 ld a0,24(s1)
^^ // a0 contains two pointers
e088 sd a0,0(s1)
mntget(path->mnt);
// Need "lw a0,0(s1)" or "a0 << 32; a0 >> 32"
2a6150ef jal c01ce946 <mntget> // bug!
Acknowledge
===========
The s64ilp32 needs many other projects' cooperation. Thx, all guys
involved:
- GNU: LiaoShihua <shihua@iscas.ac.cn>,
Jiawe Chen<jiawei@iscas.ac.cn>
- Qemu: Weiwei Li <liweiwei@iscas.ac.cn>
- LLVM: luxufan <luxufan@iscas.ac.cn>,
Chunyu Liao<chunyu@iscas.ac.cn>
- OpenEuler rv32: Junqiang Wang <wangjunqiang@iscas.ac.cn>
- Debian rv32: Bo YU <tsu.yubo@gmail.com>
Han Gao <gaohan@iscas.ac.cn>
- Fedora rv32: Wei Fu <wefu@redhat.com>
References
==========
[1] https://techpubs.jurassic.nl/manuals/0630/developer/Mpro_n32_ABI/sgi_html/index.html
[2] https://wiki.debian.org/Arm64ilp32Port
[3] https://lwn.net/Articles/456731/
[4] https://github.com/riscv/riscv-profiles/releases
[5] https://www.cnx-software.com/2021/10/25/allwinner-d1s-f133-risc-v-processor-64mb-ddr2/
[6] https://milkv.io/duo/
[7] https://twitter.com/tphuang/status/1631308330256801793
[8] https://www.cnx-software.com/2022/12/02/pine64-ox64-sbc-bl808-risc-v-multi-protocol-wisoc-64mb-ram/
Guo Ren (22):
riscv: vdso: Unify vdso32 & compat_vdso into vdso/Makefile
riscv: vdso: Remove compat_vdso/
riscv: vdso: Add time-related vDSO common flow for vdso32
clocksource: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT
riscv: s64ilp32: Introduce xlen_t
irqchip: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT
riscv: s64ilp32: Add sbi support
riscv: s64ilp32: Add asid support
riscv: s64ilp32: Introduce PTR_L and PTR_S
riscv: s64ilp32: Enable user space runtime environment
riscv: s64ilp32: Add ebpf jit support
riscv: s64ilp32: Add ELF32 support
riscv: s64ilp32: Add ARCH RV64 ILP32 compiling framework
riscv: s64ilp32: Add MMU_SV39 mode support for 32BIT
riscv: s64ilp32: Enable native atomic64
riscv: s64ilp32: Add TImode (128 int) support
riscv: s64ilp32: Implement cmpxchg_double
riscv: s64ilp32: Disable KVM
riscv: Cleanup rv32_defconfig
riscv: s64ilp32: Add rv64ilp32_defconfig
riscv: s64ilp32: Correct the rv64ilp32 stackframe layout
riscv: s64ilp32: Temporary workaround solution to gcc problem
arch/riscv/Kconfig | 36 +++-
arch/riscv/Makefile | 24 ++-
arch/riscv/configs/32-bit.config | 2 -
arch/riscv/configs/64ilp32.config | 2 +
arch/riscv/include/asm/asm.h | 5 +
arch/riscv/include/asm/atomic.h | 6 +
arch/riscv/include/asm/cmpxchg.h | 53 ++++++
arch/riscv/include/asm/cpu_ops_sbi.h | 4 +-
arch/riscv/include/asm/csr.h | 58 +++---
arch/riscv/include/asm/extable.h | 2 +-
arch/riscv/include/asm/page.h | 24 ++-
arch/riscv/include/asm/pgtable-64.h | 42 ++---
arch/riscv/include/asm/pgtable.h | 26 ++-
arch/riscv/include/asm/processor.h | 8 +-
arch/riscv/include/asm/ptrace.h | 96 +++++-----
arch/riscv/include/asm/sbi.h | 24 +--
arch/riscv/include/asm/stacktrace.h | 6 +
arch/riscv/include/asm/timex.h | 10 +-
arch/riscv/include/asm/vdso.h | 34 +++-
arch/riscv/include/asm/vdso/gettimeofday.h | 84 +++++++++
arch/riscv/include/uapi/asm/elf.h | 2 +-
arch/riscv/include/uapi/asm/unistd.h | 1 +
arch/riscv/kernel/Makefile | 3 +-
arch/riscv/kernel/compat_signal.c | 2 +-
arch/riscv/kernel/compat_vdso/.gitignore | 2 -
arch/riscv/kernel/compat_vdso/compat_vdso.S | 8 -
.../kernel/compat_vdso/compat_vdso.lds.S | 3 -
arch/riscv/kernel/compat_vdso/flush_icache.S | 3 -
arch/riscv/kernel/compat_vdso/getcpu.S | 3 -
arch/riscv/kernel/compat_vdso/note.S | 3 -
arch/riscv/kernel/compat_vdso/rt_sigreturn.S | 3 -
arch/riscv/kernel/cpu.c | 4 +-
arch/riscv/kernel/cpu_ops_sbi.c | 4 +-
arch/riscv/kernel/cpufeature.c | 4 +-
arch/riscv/kernel/entry.S | 24 +--
arch/riscv/kernel/head.S | 8 +-
arch/riscv/kernel/process.c | 8 +-
arch/riscv/kernel/sbi.c | 24 +--
arch/riscv/kernel/signal.c | 6 +-
arch/riscv/kernel/traps.c | 4 +-
arch/riscv/kernel/vdso.c | 4 +-
arch/riscv/kernel/vdso/Makefile | 176 ++++++++++++------
..._vdso_offsets.sh => gen_vdso32_offsets.sh} | 2 +-
.../gen_vdso64_offsets.sh} | 2 +-
arch/riscv/kernel/vdso/vgettimeofday.c | 39 +++-
arch/riscv/kernel/vdso32.S | 8 +
arch/riscv/kernel/{vdso/vdso.S => vdso64.S} | 8 +-
arch/riscv/kvm/Kconfig | 1 +
arch/riscv/lib/Makefile | 1 +
arch/riscv/lib/memset.S | 4 +-
arch/riscv/mm/context.c | 16 +-
arch/riscv/mm/fault.c | 13 +-
arch/riscv/mm/init.c | 29 ++-
arch/riscv/net/Makefile | 6 +-
arch/riscv/net/bpf_jit_comp64.c | 10 +-
drivers/clocksource/timer-riscv.c | 2 +-
drivers/irqchip/irq-riscv-intc.c | 4 +-
fs/namei.c | 2 +-
58 files changed, 675 insertions(+), 317 deletions(-)
create mode 100644 arch/riscv/configs/64ilp32.config
delete mode 100644 arch/riscv/kernel/compat_vdso/.gitignore
delete mode 100644 arch/riscv/kernel/compat_vdso/compat_vdso.S
delete mode 100644 arch/riscv/kernel/compat_vdso/compat_vdso.lds.S
delete mode 100644 arch/riscv/kernel/compat_vdso/flush_icache.S
delete mode 100644 arch/riscv/kernel/compat_vdso/getcpu.S
delete mode 100644 arch/riscv/kernel/compat_vdso/note.S
delete mode 100644 arch/riscv/kernel/compat_vdso/rt_sigreturn.S
rename arch/riscv/kernel/vdso/{gen_vdso_offsets.sh => gen_vdso32_offsets.sh} (78%)
rename arch/riscv/kernel/{compat_vdso/gen_compat_vdso_offsets.sh => vdso/gen_vdso64_offsets.sh} (77%)
create mode 100644 arch/riscv/kernel/vdso32.S
rename arch/riscv/kernel/{vdso/vdso.S => vdso64.S} (73%)
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* [RFC PATCH 01/22] riscv: vdso: Unify vdso32 & compat_vdso into vdso/Makefile
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
@ 2023-05-18 13:09 ` guoren
2023-05-18 13:09 ` [RFC PATCH 02/22] riscv: vdso: Remove compat_vdso/ guoren
` (22 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:09 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Linux kernel abi and vdso abi could be different, and current
vdso/Makefile force vdso use the same abi as the kernel. It isn't
suitable for the next s64ilp32 patch series because s64ilp32 uses
64ilp32 abi in the kernel but still uses 32ilp32 in the userspace. This
patch unifies vdso32 & compat_vdso into vdso/Makefile to solve this
problem, similar to Powerpc's vdso framework.
Before this:
- vdso/
- vdso/vdso.S
- vdso/gen_vdso_offsets.sh
- compat_vdso/compat_vdso.S
- compat_vdso/gen_compat_vdso_offsets.sh
- vdso.c
After this:
- vdso/
- vdso64.S
- vdso/gen_vdso64_offsets.sh
- vdso32.S
- vdso/gen_vdso32_offsets.sh
- vdso.c
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 11 ++
arch/riscv/Makefile | 15 +-
arch/riscv/include/asm/vdso.h | 34 +++-
arch/riscv/include/asm/vdso/gettimeofday.h | 7 +-
arch/riscv/kernel/Makefile | 3 +-
arch/riscv/kernel/compat_signal.c | 2 +-
arch/riscv/kernel/vdso.c | 4 +-
arch/riscv/kernel/vdso/Makefile | 179 ++++++++++++------
..._vdso_offsets.sh => gen_vdso32_offsets.sh} | 2 +-
arch/riscv/kernel/vdso/gen_vdso64_offsets.sh | 5 +
arch/riscv/kernel/vdso32.S | 8 +
arch/riscv/kernel/{vdso/vdso.S => vdso64.S} | 8 +-
12 files changed, 197 insertions(+), 81 deletions(-)
rename arch/riscv/kernel/vdso/{gen_vdso_offsets.sh => gen_vdso32_offsets.sh} (78%)
create mode 100755 arch/riscv/kernel/vdso/gen_vdso64_offsets.sh
create mode 100644 arch/riscv/kernel/vdso32.S
rename arch/riscv/kernel/{vdso/vdso.S => vdso64.S} (73%)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index c58d74235363..643884620cd6 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -5,9 +5,11 @@
#
config 64BIT
+ select VDSO64
bool
config 32BIT
+ select VDSO32
bool
config RISCV
@@ -248,6 +250,14 @@ config RISCV_DMA_NONCOHERENT
config AS_HAS_INSN
def_bool $(as-instr,.insn r 51$(comma) 0$(comma) 0$(comma) t0$(comma) t0$(comma) zero)
+config VDSO32
+ bool
+ depends on MMU
+
+config VDSO64
+ bool
+ depends on MMU
+
source "arch/riscv/Kconfig.socs"
source "arch/riscv/Kconfig.errata"
@@ -594,6 +604,7 @@ config CRASH_DUMP
config COMPAT
bool "Kernel support for 32-bit U-mode"
default 64BIT
+ select VDSO32
depends on 64BIT && MMU
help
This option enables support for a 32-bit U-mode running under a 64-bit
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index e859e1721a8f..dafe958c4217 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -122,18 +122,19 @@ libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
PHONY += vdso_install
vdso_install:
- $(Q)$(MAKE) $(build)=arch/riscv/kernel/vdso $@
- $(if $(CONFIG_COMPAT),$(Q)$(MAKE) \
- $(build)=arch/riscv/kernel/compat_vdso compat_$@)
+ $(if $(CONFIG_VDSO32),$(Q)$(MAKE) \
+ $(build)=arch/riscv/kernel/vdso vdso32_install
+ $(if $(CONFIG_VDSO64),$(Q)$(MAKE) \
+ $(build)=arch/riscv/kernel/vdso vdso64_install
ifeq ($(KBUILD_EXTMOD),)
ifeq ($(CONFIG_MMU),y)
prepare: vdso_prepare
vdso_prepare: prepare0
- $(Q)$(MAKE) $(build)=arch/riscv/kernel/vdso include/generated/vdso-offsets.h
- $(if $(CONFIG_COMPAT),$(Q)$(MAKE) \
- $(build)=arch/riscv/kernel/compat_vdso include/generated/compat_vdso-offsets.h)
-
+ $(if $(CONFIG_VDSO32),$(Q)$(MAKE) \
+ $(build)=arch/riscv/kernel/vdso include/generated/vdso32-offsets.h)
+ $(if $(CONFIG_VDSO64),$(Q)$(MAKE) \
+ $(build)=arch/riscv/kernel/vdso include/generated/vdso64-offsets.h)
endif
endif
diff --git a/arch/riscv/include/asm/vdso.h b/arch/riscv/include/asm/vdso.h
index f891478829a5..305ddc6de21c 100644
--- a/arch/riscv/include/asm/vdso.h
+++ b/arch/riscv/include/asm/vdso.h
@@ -17,22 +17,36 @@
#define __VVAR_PAGES 2
#ifndef __ASSEMBLY__
-#include <generated/vdso-offsets.h>
-#define VDSO_SYMBOL(base, name) \
- (void __user *)((unsigned long)(base) + __vdso_##name##_offset)
+#ifdef CONFIG_VDSO64
+#include <generated/vdso64-offsets.h>
-#ifdef CONFIG_COMPAT
-#include <generated/compat_vdso-offsets.h>
+#define VDSO64_SYMBOL(base, name) \
+ (void __user *)((unsigned long)(base) + rv64__vdso_##name##_offset)
-#define COMPAT_VDSO_SYMBOL(base, name) \
- (void __user *)((unsigned long)(base) + compat__vdso_##name##_offset)
+extern char vdso64_start[], vdso64_end[];
-extern char compat_vdso_start[], compat_vdso_end[];
+#endif /* CONFIG_VDSO64 */
-#endif /* CONFIG_COMPAT */
+#ifdef CONFIG_VDSO32
+#include <generated/vdso32-offsets.h>
-extern char vdso_start[], vdso_end[];
+#define VDSO32_SYMBOL(base, name) \
+ (void __user *)((unsigned long)(base) + rv32__vdso_##name##_offset)
+
+extern char vdso32_start[], vdso32_end[];
+
+#endif /* CONFIG_VDSO32 */
+
+#ifdef CONFIG_64BIT
+#define vdso_start vdso64_start
+#define vdso_end vdso64_end
+#define VDSO_SYMBOL VDSO64_SYMBOL
+#else /* CONFIG_64BIT */
+#define vdso_start vdso32_start
+#define vdso_end vdso32_end
+#define VDSO_SYMBOL VDSO32_SYMBOL
+#endif /* CONFIG_64BIT */
#endif /* !__ASSEMBLY__ */
diff --git a/arch/riscv/include/asm/vdso/gettimeofday.h b/arch/riscv/include/asm/vdso/gettimeofday.h
index ba3283cf7acc..a7ae8576797b 100644
--- a/arch/riscv/include/asm/vdso/gettimeofday.h
+++ b/arch/riscv/include/asm/vdso/gettimeofday.h
@@ -17,6 +17,7 @@
#define VDSO_HAS_CLOCK_GETRES 1
+#ifdef __NR_gettimeofday
static __always_inline
int gettimeofday_fallback(struct __kernel_old_timeval *_tv,
struct timezone *_tz)
@@ -33,7 +34,9 @@ int gettimeofday_fallback(struct __kernel_old_timeval *_tv,
return ret;
}
+#endif
+#ifdef __NR_clock_gettime
static __always_inline
long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
@@ -49,7 +52,9 @@ long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
return ret;
}
+#endif
+#ifdef __NR_clock_getres
static __always_inline
int clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
@@ -65,7 +70,7 @@ int clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
return ret;
}
-
+#endif
#endif /* CONFIG_GENERIC_TIME_VSYSCALL */
static __always_inline u64 __arch_get_hw_counter(s32 clock_mode,
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 392fa6e35d4a..93de624f7fcd 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -52,6 +52,8 @@ obj-y += cacheinfo.o
obj-y += patch.o
obj-y += probes/
obj-$(CONFIG_MMU) += vdso.o vdso/
+obj-$(CONFIG_VDSO64) += vdso64.o
+obj-$(CONFIG_VDSO32) += vdso32.o
obj-$(CONFIG_RISCV_M_MODE) += traps_misaligned.o
obj-$(CONFIG_FPU) += fpu.o
@@ -86,4 +88,3 @@ obj-$(CONFIG_JUMP_LABEL) += jump_label.o
obj-$(CONFIG_EFI) += efi.o
obj-$(CONFIG_COMPAT) += compat_syscall_table.o
obj-$(CONFIG_COMPAT) += compat_signal.o
-obj-$(CONFIG_COMPAT) += compat_vdso/
diff --git a/arch/riscv/kernel/compat_signal.c b/arch/riscv/kernel/compat_signal.c
index 6ec4e34255a9..8dea2012836e 100644
--- a/arch/riscv/kernel/compat_signal.c
+++ b/arch/riscv/kernel/compat_signal.c
@@ -217,7 +217,7 @@ int compat_setup_rt_frame(struct ksignal *ksig, sigset_t *set,
if (err)
return -EFAULT;
- regs->ra = (unsigned long)COMPAT_VDSO_SYMBOL(
+ regs->ra = (unsigned long)VDSO32_SYMBOL(
current->mm->context.vdso, rt_sigreturn);
/*
diff --git a/arch/riscv/kernel/vdso.c b/arch/riscv/kernel/vdso.c
index 9a68e7eaae4d..497cde87dc69 100644
--- a/arch/riscv/kernel/vdso.c
+++ b/arch/riscv/kernel/vdso.c
@@ -193,8 +193,8 @@ static struct vm_special_mapping rv_compat_vdso_maps[] __ro_after_init = {
static struct __vdso_info compat_vdso_info __ro_after_init = {
.name = "compat_vdso",
- .vdso_code_start = compat_vdso_start,
- .vdso_code_end = compat_vdso_end,
+ .vdso_code_start = vdso32_start,
+ .vdso_code_end = vdso32_end,
.dm = &rv_compat_vdso_maps[RV_VDSO_MAP_VVAR],
.cm = &rv_compat_vdso_maps[RV_VDSO_MAP_VDSO],
};
diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile
index 022258426050..e3c3bf669c7b 100644
--- a/arch/riscv/kernel/vdso/Makefile
+++ b/arch/riscv/kernel/vdso/Makefile
@@ -1,87 +1,158 @@
# SPDX-License-Identifier: GPL-2.0-only
-# Copied from arch/tile/kernel/vdso/Makefile
# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
# the inclusion of generic Makefile.
ARCH_REL_TYPE_ABS := R_RISCV_32|R_RISCV_64|R_RISCV_JUMP_SLOT
include $(srctree)/lib/vdso/Makefile
-# Symbols present in the vdso
-vdso-syms = rt_sigreturn
-ifdef CONFIG_64BIT
-vdso-syms += vgettimeofday
-endif
-vdso-syms += getcpu
-vdso-syms += flush_icache
-vdso-syms += hwprobe
-vdso-syms += sys_hwprobe
-# Files to link into the vdso
-obj-vdso = $(patsubst %, %.o, $(vdso-syms)) note.o
+VDSO_CC := $(CC)
+VDSO_LD := $(LD)
+
+# Disable profiling and instrumentation for VDSO code
+GCOV_PROFILE := n
+KCOV_INSTRUMENT := n
+KASAN_SANITIZE := n
+UBSAN_SANITIZE := n
ccflags-y := -fno-stack-protector
ccflags-y += -DDISABLE_BRANCH_PROFILING
-ifneq ($(c-gettimeofday-y),)
- CFLAGS_vgettimeofday.o += -fPIC -include $(c-gettimeofday-y)
+CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+ifneq ($(filter vgettimeofday, $(vdso-cc-syms)),)
+ CPPFLAGS_vdso.lds += -DHAS_VGETTIMEOFDAY
endif
-CFLAGS_hwprobe.o += -fPIC
+# strip rule for the .so file
+$(obj)/%.so: OBJCOPYFLAGS := -S
+$(obj)/%.so: $(obj)/%.so.dbg FORCE
+ $(call if_changed,objcopy)
-# Build rules
-targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.lds
-obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
+# install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-obj-y += vdso.o
-CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
-ifneq ($(filter vgettimeofday, $(vdso-syms)),)
-CPPFLAGS_vdso.lds += -DHAS_VGETTIMEOFDAY
-endif
+# Symbols present in the vdso
+ifdef CONFIG_VDSO64
+vdso64-as-syms = rt_sigreturn
+vdso64-as-syms += getcpu
+vdso64-as-syms += flush_icache
+vdso64-as-syms += sys_hwprobe
-# Disable -pg to prevent insert call site
-CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE)
+vdso64-cc-syms = vgettimeofday
+vdso64-cc-syms += hwprobe
-# Disable profiling and instrumentation for VDSO code
-GCOV_PROFILE := n
-KCOV_INSTRUMENT := n
-KASAN_SANITIZE := n
-UBSAN_SANITIZE := n
+obj-as-vdso64 = $(patsubst %, %-64.o, $(vdso64-as-syms)) note-64.o
+obj-as-vdso64 := $(addprefix $(obj)/, $(obj-as-vdso64))
-# Force dependency
-$(obj)/vdso.o: $(obj)/vdso.so
+obj-cc-vdso64 = $(patsubst %, %-64.o, $(vdso64-cc-syms))
+obj-cc-vdso64 := $(addprefix $(obj)/, $(obj-cc-vdso64))
-# link rule for the .so file, .lds has to be first
-$(obj)/vdso.so.dbg: $(obj)/vdso.lds $(obj-vdso) FORCE
- $(call if_changed,vdsold)
-LDFLAGS_vdso.so.dbg = -shared -S -soname=linux-vdso.so.1 \
+targets += $(obj-as-vdso64) $(obj-cc-vdso64) vdso64.so vdso64.so.dbg vdso64.lds
+
+$(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(obj-as-vdso64) $(obj-cc-vdso64) FORCE
+ $(call if_changed,vdso64ld)
+LDFLAGS_vdso64.so.dbg = -shared -S -soname=linux-vdso64.so.1 \
--build-id=sha1 --hash-style=both --eh-frame-hdr
+# The DSO images are built using a special linker script
+# Make sure only to export the intended __vdso_xxx symbol offsets.
+quiet_cmd_vdso64ld = VDSO64LD $@
+ cmd_vdso64ld = $(VDSO_LD) $(ld_flags) $(VDSO64_LD_FLAGS) -T $(filter-out FORCE,$^) -o $@.tmp && \
+ $(OBJCOPY) $(patsubst %, -G __vdso_%, $(vdso64-as-syms) $(vdso64-cc-syms)) $@.tmp $@ && \
+ rm $@.tmp
-# strip rule for the .so file
-$(obj)/%.so: OBJCOPYFLAGS := -S
-$(obj)/%.so: $(obj)/%.so.dbg FORCE
- $(call if_changed,objcopy)
+# actual build commands
+quiet_cmd_vdso64as = VDSO64AS $@
+ cmd_vdso64as = $(VDSO_CC) $(a_flags) $(VDSO64_CC_FLAGS) -c -o $@ $<
+quiet_cmd_vdso64cc = VDSO64CC $@
+ cmd_vdso64cc = $(VDSO_CC) $(c_flags) $(VDSO64_CC_FLAGS) -c -o $@ $<
+
+# Force dependency
+$(obj)/vdso64.o: $(obj)/vdso64.so
+
+$(obj-as-vdso64): %-64.o: %.S FORCE
+ $(call if_changed_dep,vdso64as)
+$(obj-cc-vdso64): %-64.o: %.c FORCE
+ $(call if_changed_dep,vdso64cc)
+
+CFLAGS_vgettimeofday-64.o += -fPIC -include $(c-gettimeofday-y)
+# Disable -pg to prevent insert call site
+CFLAGS_REMOVE_vgettimeofday-64.o = $(CC_FLAGS_FTRACE)
+
+CFLAGS_hwprobe-64.o += -fPIC
# Generate VDSO offsets using helper script
-gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
-quiet_cmd_vdsosym = VDSOSYM $@
- cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+gen-vdso64sym := $(srctree)/$(src)/gen_vdso64_offsets.sh
+quiet_cmd_vdso64sym = VDSO64SYM $@
+ cmd_vdso64sym = $(NM) $< | $(gen-vdso64sym) | LC_ALL=C sort > $@
-include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
- $(call if_changed,vdsosym)
+include/generated/vdso64-offsets.h: $(obj)/vdso64.so.dbg $(obj)/vdso64.so FORCE
+ $(call if_changed,vdso64sym)
+
+vdso64.so: $(obj)/vdso64.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso64_install: vdso64.so
+endif
+
+ifdef CONFIG_VDSO32
+vdso32-as-syms = rt_sigreturn
+vdso32-as-syms += getcpu
+vdso32-as-syms += flush_icache
+vdso32-as-syms += sys_hwprobe
+
+vdso32-cc-syms += hwprobe
+
+VDSO32_CC_FLAGS := -march=rv32g -mabi=ilp32
+VDSO32_LD_FLAGS := -melf32lriscv
+
+obj-as-vdso32 = $(patsubst %, %-32.o, $(vdso32-as-syms)) note-32.o
+obj-as-vdso32 := $(addprefix $(obj)/, $(obj-as-vdso32))
+
+obj-cc-vdso32 = $(patsubst %, %-32.o, $(vdso32-cc-syms))
+obj-cc-vdso32 := $(addprefix $(obj)/, $(obj-cc-vdso32))
+
+targets += $(obj-as-vdso32) $(obj-cc-vdso32) vdso32.so vdso32.so.dbg vdso32.lds
+
+$(obj)/vdso32.so.dbg: $(obj)/vdso.lds $(obj-as-vdso32) $(obj-cc-vdso32) FORCE
+ $(call if_changed,vdso32ld)
+LDFLAGS_vdso32.so.dbg = -shared -S -soname=linux-vdso32.so.1 \
+ --build-id=sha1 --hash-style=both --eh-frame-hdr
-# actual build commands
# The DSO images are built using a special linker script
# Make sure only to export the intended __vdso_xxx symbol offsets.
-quiet_cmd_vdsold = VDSOLD $@
- cmd_vdsold = $(LD) $(ld_flags) -T $(filter-out FORCE,$^) -o $@.tmp && \
- $(OBJCOPY) $(patsubst %, -G __vdso_%, $(vdso-syms)) $@.tmp $@ && \
+quiet_cmd_vdso32ld = VDSO32LD $@
+ cmd_vdso32ld = $(VDSO_LD) $(ld_flags) $(VDSO32_LD_FLAGS) -T $(filter-out FORCE,$^) -o $@.tmp && \
+ $(OBJCOPY) $(patsubst %, -G __vdso_%, $(vdso32-as-syms) $(vdso32-cc-syms)) $@.tmp $@ && \
rm $@.tmp
-# install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
- cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
+# actual build commands
+quiet_cmd_vdso32as = VDSO32AS $@
+ cmd_vdso32as = $(VDSO_CC) $(a_flags) $(VDSO32_CC_FLAGS) -c -o $@ $<
+quiet_cmd_vdso32cc = VDSO32CC $@
+ cmd_vdso32cc = $(VDSO_CC) $(c_flags) $(VDSO32_CC_FLAGS) -c -o $@ $<
-vdso.so: $(obj)/vdso.so.dbg
+# Force dependency
+$(obj)/vdso32.o: $(obj)/vdso32.so
+
+$(obj-as-vdso32): %-32.o: %.S FORCE
+ $(call if_changed_dep,vdso32as)
+$(obj-cc-vdso32): %-32.o: %.c FORCE
+ $(call if_changed_dep,vdso32cc)
+
+CFLAGS_hwprobe-32.o += -fPIC
+
+# Generate VDSO offsets using helper script
+gen-vdso32sym := $(srctree)/$(src)/gen_vdso32_offsets.sh
+quiet_cmd_vdso32sym = VDSO32SYM $@
+ cmd_vdso32sym = $(NM) $< | $(gen-vdso32sym) | LC_ALL=C sort > $@
+
+include/generated/vdso32-offsets.h: $(obj)/vdso32.so.dbg $(obj)/vdso32.so FORCE
+ $(call if_changed,vdso32sym)
+
+vdso32.so: $(obj)/vdso32.so.dbg
@mkdir -p $(MODLIB)/vdso
$(call cmd,vdso_install)
-vdso_install: vdso.so
+vdso32_install: vdso32.so
+endif
diff --git a/arch/riscv/kernel/vdso/gen_vdso_offsets.sh b/arch/riscv/kernel/vdso/gen_vdso32_offsets.sh
similarity index 78%
rename from arch/riscv/kernel/vdso/gen_vdso_offsets.sh
rename to arch/riscv/kernel/vdso/gen_vdso32_offsets.sh
index c2e5613f3495..c0dee7361530 100755
--- a/arch/riscv/kernel/vdso/gen_vdso_offsets.sh
+++ b/arch/riscv/kernel/vdso/gen_vdso32_offsets.sh
@@ -2,4 +2,4 @@
# SPDX-License-Identifier: GPL-2.0
LC_ALL=C
-sed -n -e 's/^[0]\+\(0[0-9a-fA-F]*\) . \(__vdso_[a-zA-Z0-9_]*\)$/\#define \2_offset\t0x\1/p'
+sed -n -e 's/^[0]\+\(0[0-9a-fA-F]*\) . \(__vdso_[a-zA-Z0-9_]*\)$/\#define rv32\2_offset\t0x\1/p'
diff --git a/arch/riscv/kernel/vdso/gen_vdso64_offsets.sh b/arch/riscv/kernel/vdso/gen_vdso64_offsets.sh
new file mode 100755
index 000000000000..ac265ed49eaf
--- /dev/null
+++ b/arch/riscv/kernel/vdso/gen_vdso64_offsets.sh
@@ -0,0 +1,5 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+
+LC_ALL=C
+sed -n -e 's/^[0]\+\(0[0-9a-fA-F]*\) . \(__vdso_[a-zA-Z0-9_]*\)$/\#define rv64\2_offset\t0x\1/p'
diff --git a/arch/riscv/kernel/vdso32.S b/arch/riscv/kernel/vdso32.S
new file mode 100644
index 000000000000..9bdf3cb20ccb
--- /dev/null
+++ b/arch/riscv/kernel/vdso32.S
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#define vdso64_start vdso32_start
+#define vdso64_end vdso32_end
+
+#define __VDSO_PATH "arch/riscv/kernel/vdso/vdso32.so"
+
+#include "vdso64.S"
diff --git a/arch/riscv/kernel/vdso/vdso.S b/arch/riscv/kernel/vdso64.S
similarity index 73%
rename from arch/riscv/kernel/vdso/vdso.S
rename to arch/riscv/kernel/vdso64.S
index 83f1c899e8d8..498b73da0dbf 100644
--- a/arch/riscv/kernel/vdso/vdso.S
+++ b/arch/riscv/kernel/vdso64.S
@@ -8,16 +8,16 @@
#include <asm/page.h>
#ifndef __VDSO_PATH
-#define __VDSO_PATH "arch/riscv/kernel/vdso/vdso.so"
+#define __VDSO_PATH "arch/riscv/kernel/vdso/vdso64.so"
#endif
__PAGE_ALIGNED_DATA
- .globl vdso_start, vdso_end
+ .globl vdso64_start, vdso64_end
.balign PAGE_SIZE
-vdso_start:
+vdso64_start:
.incbin __VDSO_PATH
.balign PAGE_SIZE
-vdso_end:
+vdso64_end:
.previous
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 02/22] riscv: vdso: Remove compat_vdso/
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
2023-05-18 13:09 ` [RFC PATCH 01/22] riscv: vdso: Unify vdso32 & compat_vdso into vdso/Makefile guoren
@ 2023-05-18 13:09 ` guoren
2023-05-18 13:09 ` [RFC PATCH 03/22] riscv: vdso: Add time-related vDSO common flow for vdso32 guoren
` (21 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:09 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
After unifying vdso32 & vdso64 into vdso/, we ever needn't compat_vdso
directory. This commit removes the whole compat_vdso/.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/kernel/compat_vdso/.gitignore | 2 --
arch/riscv/kernel/compat_vdso/compat_vdso.S | 8 --------
arch/riscv/kernel/compat_vdso/compat_vdso.lds.S | 3 ---
arch/riscv/kernel/compat_vdso/flush_icache.S | 3 ---
arch/riscv/kernel/compat_vdso/gen_compat_vdso_offsets.sh | 5 -----
arch/riscv/kernel/compat_vdso/getcpu.S | 3 ---
arch/riscv/kernel/compat_vdso/note.S | 3 ---
arch/riscv/kernel/compat_vdso/rt_sigreturn.S | 3 ---
8 files changed, 30 deletions(-)
delete mode 100644 arch/riscv/kernel/compat_vdso/.gitignore
delete mode 100644 arch/riscv/kernel/compat_vdso/compat_vdso.S
delete mode 100644 arch/riscv/kernel/compat_vdso/compat_vdso.lds.S
delete mode 100644 arch/riscv/kernel/compat_vdso/flush_icache.S
delete mode 100755 arch/riscv/kernel/compat_vdso/gen_compat_vdso_offsets.sh
delete mode 100644 arch/riscv/kernel/compat_vdso/getcpu.S
delete mode 100644 arch/riscv/kernel/compat_vdso/note.S
delete mode 100644 arch/riscv/kernel/compat_vdso/rt_sigreturn.S
diff --git a/arch/riscv/kernel/compat_vdso/.gitignore b/arch/riscv/kernel/compat_vdso/.gitignore
deleted file mode 100644
index 19d83d846c1e..000000000000
--- a/arch/riscv/kernel/compat_vdso/.gitignore
+++ /dev/null
@@ -1,2 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-compat_vdso.lds
diff --git a/arch/riscv/kernel/compat_vdso/compat_vdso.S b/arch/riscv/kernel/compat_vdso/compat_vdso.S
deleted file mode 100644
index ffd66237e091..000000000000
--- a/arch/riscv/kernel/compat_vdso/compat_vdso.S
+++ /dev/null
@@ -1,8 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-
-#define vdso_start compat_vdso_start
-#define vdso_end compat_vdso_end
-
-#define __VDSO_PATH "arch/riscv/kernel/compat_vdso/compat_vdso.so"
-
-#include "../vdso/vdso.S"
diff --git a/arch/riscv/kernel/compat_vdso/compat_vdso.lds.S b/arch/riscv/kernel/compat_vdso/compat_vdso.lds.S
deleted file mode 100644
index c7c9355d311e..000000000000
--- a/arch/riscv/kernel/compat_vdso/compat_vdso.lds.S
+++ /dev/null
@@ -1,3 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-
-#include "../vdso/vdso.lds.S"
diff --git a/arch/riscv/kernel/compat_vdso/flush_icache.S b/arch/riscv/kernel/compat_vdso/flush_icache.S
deleted file mode 100644
index 523dd8b96045..000000000000
--- a/arch/riscv/kernel/compat_vdso/flush_icache.S
+++ /dev/null
@@ -1,3 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-
-#include "../vdso/flush_icache.S"
diff --git a/arch/riscv/kernel/compat_vdso/gen_compat_vdso_offsets.sh b/arch/riscv/kernel/compat_vdso/gen_compat_vdso_offsets.sh
deleted file mode 100755
index 8ac070c783b3..000000000000
--- a/arch/riscv/kernel/compat_vdso/gen_compat_vdso_offsets.sh
+++ /dev/null
@@ -1,5 +0,0 @@
-#!/bin/sh
-# SPDX-License-Identifier: GPL-2.0
-
-LC_ALL=C
-sed -n -e 's/^[0]\+\(0[0-9a-fA-F]*\) . \(__vdso_[a-zA-Z0-9_]*\)$/\#define compat\2_offset\t0x\1/p'
diff --git a/arch/riscv/kernel/compat_vdso/getcpu.S b/arch/riscv/kernel/compat_vdso/getcpu.S
deleted file mode 100644
index 10f463efe271..000000000000
--- a/arch/riscv/kernel/compat_vdso/getcpu.S
+++ /dev/null
@@ -1,3 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-
-#include "../vdso/getcpu.S"
diff --git a/arch/riscv/kernel/compat_vdso/note.S b/arch/riscv/kernel/compat_vdso/note.S
deleted file mode 100644
index b10312907542..000000000000
--- a/arch/riscv/kernel/compat_vdso/note.S
+++ /dev/null
@@ -1,3 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-
-#include "../vdso/note.S"
diff --git a/arch/riscv/kernel/compat_vdso/rt_sigreturn.S b/arch/riscv/kernel/compat_vdso/rt_sigreturn.S
deleted file mode 100644
index 884aada4facc..000000000000
--- a/arch/riscv/kernel/compat_vdso/rt_sigreturn.S
+++ /dev/null
@@ -1,3 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-
-#include "../vdso/rt_sigreturn.S"
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 03/22] riscv: vdso: Add time-related vDSO common flow for vdso32
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
2023-05-18 13:09 ` [RFC PATCH 01/22] riscv: vdso: Unify vdso32 & compat_vdso into vdso/Makefile guoren
2023-05-18 13:09 ` [RFC PATCH 02/22] riscv: vdso: Remove compat_vdso/ guoren
@ 2023-05-18 13:09 ` guoren
2023-05-18 13:09 ` [RFC PATCH 04/22] clocksource: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT guoren
` (20 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:09 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
This patch adds time-related vDSO common flow for vdso32, and it's an
addition to commit: ad5d1122b82f ("riscv: use vDSO common flow to reduce
the latency of the time-related functions"). Then we could reduce the
latency of collecting clock information for u32ilp32 (native 32-bit
userspace ecosystem), just like what we've done for u64lp64.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 4 +-
arch/riscv/include/asm/vdso/gettimeofday.h | 79 ++++++++++++++++++++++
arch/riscv/include/uapi/asm/unistd.h | 1 +
arch/riscv/kernel/vdso/Makefile | 39 +++++------
arch/riscv/kernel/vdso/vgettimeofday.c | 39 +++++++++--
5 files changed, 134 insertions(+), 28 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 643884620cd6..4d4fac81390f 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -75,7 +75,7 @@ config RISCV
select GENERIC_PTDUMP if MMU
select GENERIC_SCHED_CLOCK
select GENERIC_SMP_IDLE_THREAD
- select GENERIC_TIME_VSYSCALL if MMU && 64BIT
+ select GENERIC_TIME_VSYSCALL if MMU
select GENERIC_VDSO_TIME_NS if HAVE_GENERIC_VDSO
select HARDIRQS_SW_RESEND
select HAVE_ARCH_AUDITSYSCALL
@@ -103,7 +103,7 @@ config RISCV
select HAVE_FUNCTION_ARG_ACCESS_API
select HAVE_FUNCTION_ERROR_INJECTION
select HAVE_GCC_PLUGINS
- select HAVE_GENERIC_VDSO if MMU && 64BIT
+ select HAVE_GENERIC_VDSO if MMU
select HAVE_IRQ_TIME_ACCOUNTING
select HAVE_KPROBES if !XIP_KERNEL
select HAVE_KPROBES_ON_FTRACE if !XIP_KERNEL
diff --git a/arch/riscv/include/asm/vdso/gettimeofday.h b/arch/riscv/include/asm/vdso/gettimeofday.h
index a7ae8576797b..0dcb6c5c4111 100644
--- a/arch/riscv/include/asm/vdso/gettimeofday.h
+++ b/arch/riscv/include/asm/vdso/gettimeofday.h
@@ -36,6 +36,7 @@ int gettimeofday_fallback(struct __kernel_old_timeval *_tv,
}
#endif
+#if __SIZEOF_POINTER__ == 8
#ifdef __NR_clock_gettime
static __always_inline
long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
@@ -71,6 +72,84 @@ int clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
return ret;
}
#endif
+
+#elif __SIZEOF_POINTER__ == 4
+
+#define BUILD_VDSO32 1
+
+#ifdef __NR_clock_gettime64
+static __always_inline
+long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
+{
+ register clockid_t clkid asm("a0") = _clkid;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
+ register long ret asm("a0");
+ register long nr asm("a7") = __NR_clock_gettime64;
+
+ asm volatile ("ecall\n"
+ : "=r" (ret)
+ : "r"(clkid), "r"(ts), "r"(nr)
+ : "memory");
+
+ return ret;
+}
+#endif
+
+#ifdef __NR_clock_getres_time64
+static __always_inline
+int clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
+{
+ register clockid_t clkid asm("a0") = _clkid;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
+ register long ret asm("a0");
+ register long nr asm("a7") = __NR_clock_getres_time64;
+
+ asm volatile ("ecall\n"
+ : "=r" (ret)
+ : "r"(clkid), "r"(ts), "r"(nr)
+ : "memory");
+
+ return ret;
+}
+#endif
+
+#ifdef __NR_clock_gettime
+static __always_inline
+int clock_gettime32_fallback(clockid_t _clkid, struct old_timespec32 *_ts)
+{
+ register clockid_t clkid asm("a0") = _clkid;
+ register struct old_timespec32 *ts asm("a1") = _ts;
+ register long ret asm("a0");
+ register long nr asm("a7") = __NR_clock_gettime;
+
+ asm volatile ("ecall\n"
+ : "=r" (ret)
+ : "r"(clkid), "r"(ts), "r"(nr)
+ : "memory");
+
+ return ret;
+}
+#endif
+
+#ifdef __NR_clock_getres
+static __always_inline
+int clock_getres32_fallback(clockid_t _clkid, struct old_timespec32 *_ts)
+{
+ register clockid_t clkid asm("a0") = _clkid;
+ register struct old_timespec32 *ts asm("a1") = _ts;
+ register long ret asm("a0");
+ register long nr asm("a7") = __NR_clock_getres;
+
+ asm volatile ("ecall\n"
+ : "=r" (ret)
+ : "r"(clkid), "r"(ts), "r"(nr)
+ : "memory");
+
+ return ret;
+}
+#endif
+
+#endif /* __SIZEOF_POINTER__ */
#endif /* CONFIG_GENERIC_TIME_VSYSCALL */
static __always_inline u64 __arch_get_hw_counter(s32 clock_mode,
diff --git a/arch/riscv/include/uapi/asm/unistd.h b/arch/riscv/include/uapi/asm/unistd.h
index 950ab3fd4409..0ee86ef907e4 100644
--- a/arch/riscv/include/uapi/asm/unistd.h
+++ b/arch/riscv/include/uapi/asm/unistd.h
@@ -22,6 +22,7 @@
#define __ARCH_WANT_SYS_CLONE3
#define __ARCH_WANT_MEMFD_SECRET
+#define __ARCH_WANT_TIME32_SYSCALLS
#include <asm-generic/unistd.h>
diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile
index e3c3bf669c7b..42338a2c26a7 100644
--- a/arch/riscv/kernel/vdso/Makefile
+++ b/arch/riscv/kernel/vdso/Makefile
@@ -31,20 +31,20 @@ $(obj)/%.so: $(obj)/%.so.dbg FORCE
quiet_cmd_vdso_install = INSTALL $@
cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-# Symbols present in the vdso
-ifdef CONFIG_VDSO64
-vdso64-as-syms = rt_sigreturn
-vdso64-as-syms += getcpu
-vdso64-as-syms += flush_icache
-vdso64-as-syms += sys_hwprobe
+vdso-as-syms = rt_sigreturn
+vdso-as-syms += getcpu
+vdso-as-syms += flush_icache
+vdso-as-syms += sys_hwprobe
-vdso64-cc-syms = vgettimeofday
-vdso64-cc-syms += hwprobe
+vdso-cc-syms = vgettimeofday
+vdso-cc-syms += hwprobe
-obj-as-vdso64 = $(patsubst %, %-64.o, $(vdso64-as-syms)) note-64.o
+# Symbols present in the vdso
+ifdef CONFIG_VDSO64
+obj-as-vdso64 = $(patsubst %, %-64.o, $(vdso-as-syms)) note-64.o
obj-as-vdso64 := $(addprefix $(obj)/, $(obj-as-vdso64))
-obj-cc-vdso64 = $(patsubst %, %-64.o, $(vdso64-cc-syms))
+obj-cc-vdso64 = $(patsubst %, %-64.o, $(vdso-cc-syms))
obj-cc-vdso64 := $(addprefix $(obj)/, $(obj-cc-vdso64))
targets += $(obj-as-vdso64) $(obj-cc-vdso64) vdso64.so vdso64.so.dbg vdso64.lds
@@ -57,7 +57,7 @@ LDFLAGS_vdso64.so.dbg = -shared -S -soname=linux-vdso64.so.1 \
# Make sure only to export the intended __vdso_xxx symbol offsets.
quiet_cmd_vdso64ld = VDSO64LD $@
cmd_vdso64ld = $(VDSO_LD) $(ld_flags) $(VDSO64_LD_FLAGS) -T $(filter-out FORCE,$^) -o $@.tmp && \
- $(OBJCOPY) $(patsubst %, -G __vdso_%, $(vdso64-as-syms) $(vdso64-cc-syms)) $@.tmp $@ && \
+ $(OBJCOPY) $(patsubst %, -G __vdso_%, $(vdso-as-syms) $(vdso-cc-syms)) $@.tmp $@ && \
rm $@.tmp
# actual build commands
@@ -96,20 +96,13 @@ vdso64_install: vdso64.so
endif
ifdef CONFIG_VDSO32
-vdso32-as-syms = rt_sigreturn
-vdso32-as-syms += getcpu
-vdso32-as-syms += flush_icache
-vdso32-as-syms += sys_hwprobe
-
-vdso32-cc-syms += hwprobe
-
VDSO32_CC_FLAGS := -march=rv32g -mabi=ilp32
VDSO32_LD_FLAGS := -melf32lriscv
-obj-as-vdso32 = $(patsubst %, %-32.o, $(vdso32-as-syms)) note-32.o
+obj-as-vdso32 = $(patsubst %, %-32.o, $(vdso-as-syms)) note-32.o
obj-as-vdso32 := $(addprefix $(obj)/, $(obj-as-vdso32))
-obj-cc-vdso32 = $(patsubst %, %-32.o, $(vdso32-cc-syms))
+obj-cc-vdso32 = $(patsubst %, %-32.o, $(vdso-cc-syms))
obj-cc-vdso32 := $(addprefix $(obj)/, $(obj-cc-vdso32))
targets += $(obj-as-vdso32) $(obj-cc-vdso32) vdso32.so vdso32.so.dbg vdso32.lds
@@ -123,7 +116,7 @@ LDFLAGS_vdso32.so.dbg = -shared -S -soname=linux-vdso32.so.1 \
# Make sure only to export the intended __vdso_xxx symbol offsets.
quiet_cmd_vdso32ld = VDSO32LD $@
cmd_vdso32ld = $(VDSO_LD) $(ld_flags) $(VDSO32_LD_FLAGS) -T $(filter-out FORCE,$^) -o $@.tmp && \
- $(OBJCOPY) $(patsubst %, -G __vdso_%, $(vdso32-as-syms) $(vdso32-cc-syms)) $@.tmp $@ && \
+ $(OBJCOPY) $(patsubst %, -G __vdso_%, $(vdso-as-syms) $(vdso-cc-syms)) $@.tmp $@ && \
rm $@.tmp
# actual build commands
@@ -142,6 +135,10 @@ $(obj-cc-vdso32): %-32.o: %.c FORCE
CFLAGS_hwprobe-32.o += -fPIC
+CFLAGS_vgettimeofday-32.o += -fPIC -include $(c-gettimeofday-y)
+# Disable -pg to prevent insert call site
+CFLAGS_REMOVE_vgettimeofday-32.o = $(CC_FLAGS_FTRACE)
+
# Generate VDSO offsets using helper script
gen-vdso32sym := $(srctree)/$(src)/gen_vdso32_offsets.sh
quiet_cmd_vdso32sym = VDSO32SYM $@
diff --git a/arch/riscv/kernel/vdso/vgettimeofday.c b/arch/riscv/kernel/vdso/vgettimeofday.c
index cc0d80699c31..056b465eb161 100644
--- a/arch/riscv/kernel/vdso/vgettimeofday.c
+++ b/arch/riscv/kernel/vdso/vgettimeofday.c
@@ -9,6 +9,14 @@
#include <linux/time.h>
#include <linux/types.h>
+extern
+int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz);
+int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+#if __SIZEOF_POINTER__ == 8
extern
int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts);
int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
@@ -17,15 +25,36 @@ int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
}
extern
-int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz);
-int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
+int __vdso_clock_getres(clockid_t clock_id, struct __kernel_timespec *res);
+int __vdso_clock_getres(clockid_t clock_id, struct __kernel_timespec *res)
{
- return __cvdso_gettimeofday(tv, tz);
+ return __cvdso_clock_getres(clock_id, res);
+}
+#elif __SIZEOF_POINTER__ == 4
+extern
+int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
+int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
+{
+ return __cvdso_clock_gettime32(clock, ts);
+}
+
+int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts);
+int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
}
extern
-int __vdso_clock_getres(clockid_t clock_id, struct __kernel_timespec *res);
-int __vdso_clock_getres(clockid_t clock_id, struct __kernel_timespec *res)
+int __vdso_clock_getres(clockid_t clock_id, struct old_timespec32 *res);
+int __vdso_clock_getres(clockid_t clock_id, struct old_timespec32 *res)
+{
+ return __cvdso_clock_getres_time32(clock_id, res);
+}
+
+extern
+int __vdso_clock_getres64(clockid_t clock_id, struct __kernel_timespec *res);
+int __vdso_clock_getres64(clockid_t clock_id, struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock_id, res);
}
+#endif
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 04/22] clocksource: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (2 preceding siblings ...)
2023-05-18 13:09 ` [RFC PATCH 03/22] riscv: vdso: Add time-related vDSO common flow for vdso32 guoren
@ 2023-05-18 13:09 ` guoren
2023-05-18 13:09 ` [RFC PATCH 05/22] riscv: s64ilp32: Introduce xlen_t guoren
` (19 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:09 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
When s64ilp32 enabled, CONFIG_32BIT=y but __riscv_xlen=64. So we
must use __riscv_xlen to detect real machine XLEN for CSR access.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
drivers/clocksource/timer-riscv.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
index 5f0f10c7e222..459a634012ce 100644
--- a/drivers/clocksource/timer-riscv.c
+++ b/drivers/clocksource/timer-riscv.c
@@ -37,7 +37,7 @@ static int riscv_clock_next_event(unsigned long delta,
csr_set(CSR_IE, IE_TIE);
if (static_branch_likely(&riscv_sstc_available)) {
-#if defined(CONFIG_32BIT)
+#if __riscv_xlen == 32
csr_write(CSR_STIMECMP, next_tval & 0xFFFFFFFF);
csr_write(CSR_STIMECMPH, next_tval >> 32);
#else
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 05/22] riscv: s64ilp32: Introduce xlen_t
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (3 preceding siblings ...)
2023-05-18 13:09 ` [RFC PATCH 04/22] clocksource: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT guoren
@ 2023-05-18 13:09 ` guoren
2023-05-18 13:09 ` [RFC PATCH 06/22] irqchip: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT guoren
` (18 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:09 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
When s64ilp32 landed, we couldn't use CONFIG_64/32BIT to distingue XLEN
data types. Because the xlen is 64, but the long & pointer is 32 for
s64ilp32, and s64ilp32 is a 32BIT from the software view. So introduce a
new data type - "xlen_t" and use __riscv_xlen instead of CONFIG_64/32BIT
ifdef macro.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/csr.h | 58 ++++++++++--------
arch/riscv/include/asm/processor.h | 8 +--
arch/riscv/include/asm/ptrace.h | 96 +++++++++++++++---------------
arch/riscv/include/asm/timex.h | 10 ++--
arch/riscv/kernel/cpufeature.c | 4 +-
arch/riscv/kernel/process.c | 4 +-
arch/riscv/kernel/signal.c | 6 +-
arch/riscv/kernel/traps.c | 4 +-
arch/riscv/lib/memset.S | 4 +-
arch/riscv/mm/fault.c | 2 +-
10 files changed, 104 insertions(+), 92 deletions(-)
diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h
index 7c2b8cdb7b77..adc3c866d353 100644
--- a/arch/riscv/include/asm/csr.h
+++ b/arch/riscv/include/asm/csr.h
@@ -30,37 +30,41 @@
#define SR_XS_CLEAN _AC(0x00010000, UL)
#define SR_XS_DIRTY _AC(0x00018000, UL)
-#ifndef CONFIG_64BIT
+#if __riscv_xlen == 32
#define SR_SD _AC(0x80000000, UL) /* FS/XS dirty */
#else
-#define SR_SD _AC(0x8000000000000000, UL) /* FS/XS dirty */
+#define SR_SD _AC(0x8000000000000000, ULL) /* FS/XS dirty */
#endif
-#ifdef CONFIG_64BIT
-#define SR_UXL _AC(0x300000000, UL) /* XLEN mask for U-mode */
-#define SR_UXL_32 _AC(0x100000000, UL) /* XLEN = 32 for U-mode */
-#define SR_UXL_64 _AC(0x200000000, UL) /* XLEN = 64 for U-mode */
+#if __riscv_xlen == 64
+#define SR_UXL _AC(0x300000000, ULL) /* XLEN mask for U-mode */
+#define SR_UXL_32 _AC(0x100000000, ULL) /* XLEN = 32 for U-mode */
+#define SR_UXL_64 _AC(0x200000000, ULL) /* XLEN = 64 for U-mode */
#endif
/* SATP flags */
-#ifndef CONFIG_64BIT
+#if __riscv_xlen == 32
#define SATP_PPN _AC(0x003FFFFF, UL)
#define SATP_MODE_32 _AC(0x80000000, UL)
#define SATP_ASID_BITS 9
#define SATP_ASID_SHIFT 22
#define SATP_ASID_MASK _AC(0x1FF, UL)
#else
-#define SATP_PPN _AC(0x00000FFFFFFFFFFF, UL)
-#define SATP_MODE_39 _AC(0x8000000000000000, UL)
-#define SATP_MODE_48 _AC(0x9000000000000000, UL)
-#define SATP_MODE_57 _AC(0xa000000000000000, UL)
+#define SATP_PPN _AC(0x00000FFFFFFFFFFF, ULL)
+#define SATP_MODE_39 _AC(0x8000000000000000, ULL)
+#define SATP_MODE_48 _AC(0x9000000000000000, ULL)
+#define SATP_MODE_57 _AC(0xa000000000000000, ULL)
#define SATP_ASID_BITS 16
#define SATP_ASID_SHIFT 44
-#define SATP_ASID_MASK _AC(0xFFFF, UL)
+#define SATP_ASID_MASK _AC(0xFFFF, ULL)
#endif
/* Exception cause high bit - is an interrupt if set */
+#if __riscv_xlen == 32
#define CAUSE_IRQ_FLAG (_AC(1, UL) << (__riscv_xlen - 1))
+#else
+#define CAUSE_IRQ_FLAG (_AC(1, ULL) << (__riscv_xlen - 1))
+#endif
/* Interrupt causes (minus the high bit) */
#define IRQ_S_SOFT 1
@@ -103,8 +107,8 @@
#define PMP_L 0x80
/* HSTATUS flags */
-#ifdef CONFIG_64BIT
-#define HSTATUS_VSXL _AC(0x300000000, UL)
+#if __riscv_xlen == 64
+#define HSTATUS_VSXL _AC(0x300000000, ULL)
#define HSTATUS_VSXL_SHIFT 32
#endif
#define HSTATUS_VTSR _AC(0x00400000, UL)
@@ -132,12 +136,12 @@
#define HGATP64_MODE_SHIFT 60
#define HGATP64_VMID_SHIFT 44
-#define HGATP64_VMID_MASK _AC(0x03FFF00000000000, UL)
-#define HGATP64_PPN _AC(0x00000FFFFFFFFFFF, UL)
+#define HGATP64_VMID_MASK _AC(0x03FFF00000000000, ULL)
+#define HGATP64_PPN _AC(0x00000FFFFFFFFFFF, ULL)
#define HGATP_PAGE_SHIFT 12
-#ifdef CONFIG_64BIT
+#if __riscv_xlen == 64
#define HGATP_PPN HGATP64_PPN
#define HGATP_VMID_SHIFT HGATP64_VMID_SHIFT
#define HGATP_VMID_MASK HGATP64_VMID_MASK
@@ -342,9 +346,15 @@
#ifndef __ASSEMBLY__
+#if __riscv_xlen == 64
+typedef u64 xlen_t;
+#else
+typedef u32 xlen_t;
+#endif
+
#define csr_swap(csr, val) \
({ \
- unsigned long __v = (unsigned long)(val); \
+ xlen_t __v = (xlen_t)(val); \
__asm__ __volatile__ ("csrrw %0, " __ASM_STR(csr) ", %1"\
: "=r" (__v) : "rK" (__v) \
: "memory"); \
@@ -353,7 +363,7 @@
#define csr_read(csr) \
({ \
- register unsigned long __v; \
+ register xlen_t __v; \
__asm__ __volatile__ ("csrr %0, " __ASM_STR(csr) \
: "=r" (__v) : \
: "memory"); \
@@ -362,7 +372,7 @@
#define csr_write(csr, val) \
({ \
- unsigned long __v = (unsigned long)(val); \
+ xlen_t __v = (xlen_t)(val); \
__asm__ __volatile__ ("csrw " __ASM_STR(csr) ", %0" \
: : "rK" (__v) \
: "memory"); \
@@ -370,7 +380,7 @@
#define csr_read_set(csr, val) \
({ \
- unsigned long __v = (unsigned long)(val); \
+ xlen_t __v = (xlen_t)(val); \
__asm__ __volatile__ ("csrrs %0, " __ASM_STR(csr) ", %1"\
: "=r" (__v) : "rK" (__v) \
: "memory"); \
@@ -379,7 +389,7 @@
#define csr_set(csr, val) \
({ \
- unsigned long __v = (unsigned long)(val); \
+ xlen_t __v = (xlen_t)(val); \
__asm__ __volatile__ ("csrs " __ASM_STR(csr) ", %0" \
: : "rK" (__v) \
: "memory"); \
@@ -387,7 +397,7 @@
#define csr_read_clear(csr, val) \
({ \
- unsigned long __v = (unsigned long)(val); \
+ xlen_t __v = (xlen_t)(val); \
__asm__ __volatile__ ("csrrc %0, " __ASM_STR(csr) ", %1"\
: "=r" (__v) : "rK" (__v) \
: "memory"); \
@@ -396,7 +406,7 @@
#define csr_clear(csr, val) \
({ \
- unsigned long __v = (unsigned long)(val); \
+ xlen_t __v = (xlen_t)(val); \
__asm__ __volatile__ ("csrc " __ASM_STR(csr) ", %0" \
: : "rK" (__v) \
: "memory"); \
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index 94a0590c6971..829000d6de94 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -36,10 +36,10 @@ struct thread_struct {
/* Callee-saved registers */
unsigned long ra;
unsigned long sp; /* Kernel mode stack */
- unsigned long s[12]; /* s[0]: frame pointer */
+ xlen_t s[12]; /* s[0]: frame pointer */
struct __riscv_d_ext_state fstate;
unsigned long bad_cause;
-};
+} __attribute__((__aligned__(sizeof(xlen_t))));
/* Whitelist the fstate from the task_struct for hardened usercopy */
static inline void arch_thread_struct_whitelist(unsigned long *offset,
@@ -57,8 +57,8 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
((struct pt_regs *)(task_stack_page(tsk) + THREAD_SIZE \
- ALIGN(sizeof(struct pt_regs), STACK_ALIGN)))
-#define KSTK_EIP(tsk) (task_pt_regs(tsk)->epc)
-#define KSTK_ESP(tsk) (task_pt_regs(tsk)->sp)
+#define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
+#define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
/* Do necessary setup to start up a newly executed thread. */
diff --git a/arch/riscv/include/asm/ptrace.h b/arch/riscv/include/asm/ptrace.h
index b5b0adcc85c1..54cdeec8ee79 100644
--- a/arch/riscv/include/asm/ptrace.h
+++ b/arch/riscv/include/asm/ptrace.h
@@ -13,53 +13,53 @@
#ifndef __ASSEMBLY__
struct pt_regs {
- unsigned long epc;
- unsigned long ra;
- unsigned long sp;
- unsigned long gp;
- unsigned long tp;
- unsigned long t0;
- unsigned long t1;
- unsigned long t2;
- unsigned long s0;
- unsigned long s1;
- unsigned long a0;
- unsigned long a1;
- unsigned long a2;
- unsigned long a3;
- unsigned long a4;
- unsigned long a5;
- unsigned long a6;
- unsigned long a7;
- unsigned long s2;
- unsigned long s3;
- unsigned long s4;
- unsigned long s5;
- unsigned long s6;
- unsigned long s7;
- unsigned long s8;
- unsigned long s9;
- unsigned long s10;
- unsigned long s11;
- unsigned long t3;
- unsigned long t4;
- unsigned long t5;
- unsigned long t6;
+ xlen_t epc;
+ xlen_t ra;
+ xlen_t sp;
+ xlen_t gp;
+ xlen_t tp;
+ xlen_t t0;
+ xlen_t t1;
+ xlen_t t2;
+ xlen_t s0;
+ xlen_t s1;
+ xlen_t a0;
+ xlen_t a1;
+ xlen_t a2;
+ xlen_t a3;
+ xlen_t a4;
+ xlen_t a5;
+ xlen_t a6;
+ xlen_t a7;
+ xlen_t s2;
+ xlen_t s3;
+ xlen_t s4;
+ xlen_t s5;
+ xlen_t s6;
+ xlen_t s7;
+ xlen_t s8;
+ xlen_t s9;
+ xlen_t s10;
+ xlen_t s11;
+ xlen_t t3;
+ xlen_t t4;
+ xlen_t t5;
+ xlen_t t6;
/* Supervisor/Machine CSRs */
- unsigned long status;
- unsigned long badaddr;
- unsigned long cause;
+ xlen_t status;
+ xlen_t badaddr;
+ xlen_t cause;
/* a0 value before the syscall */
- unsigned long orig_a0;
+ xlen_t orig_a0;
};
#define PTRACE_SYSEMU 0x1f
#define PTRACE_SYSEMU_SINGLESTEP 0x20
-#ifdef CONFIG_64BIT
-#define REG_FMT "%016lx"
+#if __riscv_xlen == 64
+#define REG_FMT "%016llx"
#else
-#define REG_FMT "%08lx"
+#define REG_FMT "%08x"
#endif
#define user_mode(regs) (((regs)->status & SR_PP) == 0)
@@ -69,12 +69,12 @@ struct pt_regs {
/* Helpers for working with the instruction pointer */
static inline unsigned long instruction_pointer(struct pt_regs *regs)
{
- return regs->epc;
+ return (unsigned long)regs->epc;
}
static inline void instruction_pointer_set(struct pt_regs *regs,
unsigned long val)
{
- regs->epc = val;
+ regs->epc = (xlen_t)val;
}
#define profile_pc(regs) instruction_pointer(regs)
@@ -82,40 +82,40 @@ static inline void instruction_pointer_set(struct pt_regs *regs,
/* Helpers for working with the user stack pointer */
static inline unsigned long user_stack_pointer(struct pt_regs *regs)
{
- return regs->sp;
+ return (unsigned long)regs->sp;
}
static inline void user_stack_pointer_set(struct pt_regs *regs,
unsigned long val)
{
- regs->sp = val;
+ regs->sp = (xlen_t)val;
}
/* Valid only for Kernel mode traps. */
static inline unsigned long kernel_stack_pointer(struct pt_regs *regs)
{
- return regs->sp;
+ return (unsigned long)regs->sp;
}
/* Helpers for working with the frame pointer */
static inline unsigned long frame_pointer(struct pt_regs *regs)
{
- return regs->s0;
+ return (unsigned long)regs->s0;
}
static inline void frame_pointer_set(struct pt_regs *regs,
unsigned long val)
{
- regs->s0 = val;
+ regs->s0 = (xlen_t)val;
}
static inline unsigned long regs_return_value(struct pt_regs *regs)
{
- return regs->a0;
+ return (unsigned long)regs->a0;
}
static inline void regs_set_return_value(struct pt_regs *regs,
unsigned long val)
{
- regs->a0 = val;
+ regs->a0 = (xlen_t)val;
}
extern int regs_query_register_offset(const char *name);
diff --git a/arch/riscv/include/asm/timex.h b/arch/riscv/include/asm/timex.h
index d6a7428f6248..b610819c7ff4 100644
--- a/arch/riscv/include/asm/timex.h
+++ b/arch/riscv/include/asm/timex.h
@@ -8,7 +8,7 @@
#include <asm/csr.h>
-typedef unsigned long cycles_t;
+typedef xlen_t cycles_t;
#ifdef CONFIG_RISCV_M_MODE
@@ -62,12 +62,12 @@ static inline u32 get_cycles_hi(void)
#endif /* !CONFIG_RISCV_M_MODE */
-#ifdef CONFIG_64BIT
+#if __riscv_xlen == 64
static inline u64 get_cycles64(void)
{
return get_cycles();
}
-#else /* CONFIG_64BIT */
+#else /* __riscv_xlen == 64 */
static inline u64 get_cycles64(void)
{
u32 hi, lo;
@@ -79,12 +79,12 @@ static inline u64 get_cycles64(void)
return ((u64)hi << 32) | lo;
}
-#endif /* CONFIG_64BIT */
+#endif /* __riscv_xlen == 64 */
#define ARCH_HAS_READ_CURRENT_TIMER
static inline int read_current_timer(unsigned long *timer_val)
{
- *timer_val = get_cycles();
+ *timer_val = (unsigned long)get_cycles();
return 0;
}
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index 52585e088873..5de6418b7588 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -126,10 +126,10 @@ void __init riscv_fill_hwcap(void)
}
temp = isa;
-#if IS_ENABLED(CONFIG_32BIT)
+#if IS_ENABLED(CONFIG_32BIT) && !IS_ENABLED(CONFIG_ARCH_RV64ILP32)
if (!strncmp(isa, "rv32", 4))
isa += 4;
-#elif IS_ENABLED(CONFIG_64BIT)
+#else
if (!strncmp(isa, "rv64", 4))
isa += 4;
#endif
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index e2a060066730..ab01b51a5bb9 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -45,8 +45,8 @@ void __show_regs(struct pt_regs *regs)
show_regs_print_info(KERN_DEFAULT);
if (!user_mode(regs)) {
- pr_cont("epc : %pS\n", (void *)regs->epc);
- pr_cont(" ra : %pS\n", (void *)regs->ra);
+ pr_cont("epc : %pS\n", (void *)(ulong)regs->epc);
+ pr_cont(" ra : %pS\n", (void *)(ulong)regs->ra);
}
pr_cont("epc : " REG_FMT " ra : " REG_FMT " sp : " REG_FMT "\n",
diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c
index 2e365084417e..f142a9eeec36 100644
--- a/arch/riscv/kernel/signal.c
+++ b/arch/riscv/kernel/signal.c
@@ -109,7 +109,7 @@ SYSCALL_DEFINE0(rt_sigreturn)
/* Always make any pending restarted system calls return -EINTR */
current->restart_block.fn = do_no_restart_syscall;
- frame = (struct rt_sigframe __user *)regs->sp;
+ frame = (struct rt_sigframe __user *)kernel_stack_pointer(regs);
if (!access_ok(frame, sizeof(*frame)))
goto badframe;
@@ -135,7 +135,9 @@ SYSCALL_DEFINE0(rt_sigreturn)
pr_info_ratelimited(
"%s[%d]: bad frame in %s: frame=%p pc=%p sp=%p\n",
task->comm, task_pid_nr(task), __func__,
- frame, (void *)regs->epc, (void *)regs->sp);
+ frame,
+ (void *)instruction_pointer(regs),
+ (void *)kernel_stack_pointer(regs));
}
force_sig(SIGSEGV);
return 0;
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index 8c258b78c925..efc6b649985a 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -98,7 +98,7 @@ void do_trap(struct pt_regs *regs, int signo, int code, unsigned long addr)
if (show_unhandled_signals && unhandled_signal(tsk, signo)
&& printk_ratelimit()) {
pr_info("%s[%d]: unhandled signal %d code 0x%x at 0x" REG_FMT,
- tsk->comm, task_pid_nr(tsk), signo, code, addr);
+ tsk->comm, task_pid_nr(tsk), signo, code, (xlen_t)addr);
print_vma_addr(KERN_CONT " in ", instruction_pointer(regs));
pr_cont("\n");
__show_regs(regs);
@@ -236,7 +236,7 @@ void handle_break(struct pt_regs *regs)
current->thread.bad_cause = regs->cause;
if (user_mode(regs))
- force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *)regs->epc);
+ force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *)instruction_pointer(regs));
#ifdef CONFIG_KGDB
else if (notify_die(DIE_TRAP, "EBREAK", regs, 0, regs->cause, SIGTRAP)
== NOTIFY_STOP)
diff --git a/arch/riscv/lib/memset.S b/arch/riscv/lib/memset.S
index 34c5360c6705..34be7bf51731 100644
--- a/arch/riscv/lib/memset.S
+++ b/arch/riscv/lib/memset.S
@@ -38,7 +38,7 @@ WEAK(memset)
or a1, a3, a1
slli a3, a1, 16
or a1, a3, a1
-#ifdef CONFIG_64BIT
+#if __riscv_xlen == 64
slli a3, a1, 32
or a1, a3, a1
#endif
@@ -58,7 +58,7 @@ WEAK(memset)
/* Jump into loop body */
/* Assumes 32-bit instruction lengths */
la a5, 3f
-#ifdef CONFIG_64BIT
+#if __riscv_xlen == 64
srli a4, a4, 1
#endif
add a5, a5, a4
diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c
index 3aba72ec1742..1b0bd0683766 100644
--- a/arch/riscv/mm/fault.c
+++ b/arch/riscv/mm/fault.c
@@ -28,7 +28,7 @@ static void die_kernel_fault(const char *msg, unsigned long addr,
bust_spinlocks(1);
pr_alert("Unable to handle kernel %s at virtual address " REG_FMT "\n", msg,
- addr);
+ (xlen_t)addr);
bust_spinlocks(0);
die(regs, "Oops");
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 06/22] irqchip: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (4 preceding siblings ...)
2023-05-18 13:09 ` [RFC PATCH 05/22] riscv: s64ilp32: Introduce xlen_t guoren
@ 2023-05-18 13:09 ` guoren
2023-05-18 13:09 ` [RFC PATCH 07/22] riscv: s64ilp32: Add sbi support guoren
` (17 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:09 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
When s64ilp32 enabled, CONFIG_32BIT=y but __riscv_xlen=64. So we
must use __riscv_xlen to detect real machine XLEN for CSR access.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
drivers/irqchip/irq-riscv-intc.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
index 499e5f81b3fe..18f3c837e488 100644
--- a/drivers/irqchip/irq-riscv-intc.c
+++ b/drivers/irqchip/irq-riscv-intc.c
@@ -21,7 +21,7 @@ static struct irq_domain *intc_domain;
static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
{
- unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
+ xlen_t cause = regs->cause & ~CAUSE_IRQ_FLAG;
if (unlikely(cause >= BITS_PER_LONG))
panic("unexpected interrupt cause");
@@ -113,7 +113,7 @@ static int __init riscv_intc_init(struct device_node *node,
if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
return 0;
- intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
+ intc_domain = irq_domain_add_linear(node, __riscv_xlen,
&riscv_intc_domain_ops, NULL);
if (!intc_domain) {
pr_err("unable to add IRQ domain\n");
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 07/22] riscv: s64ilp32: Add sbi support
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (5 preceding siblings ...)
2023-05-18 13:09 ` [RFC PATCH 06/22] irqchip: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT guoren
@ 2023-05-18 13:09 ` guoren
2023-05-18 13:09 ` [RFC PATCH 08/22] riscv: s64ilp32: Add asid support guoren
` (16 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:09 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The sbi uses xlen as base argument elements to connect m-mode and
s-mode. The previous implementation assumes sizeof(xlen_t) =
sizeof(long), but the s64ilp32's are different. So modify the sbi code
suitable with the s64ilp32 change.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/cpu_ops_sbi.h | 4 ++--
arch/riscv/include/asm/sbi.h | 24 ++++++++++++------------
arch/riscv/kernel/cpu_ops_sbi.c | 4 ++--
arch/riscv/kernel/sbi.c | 24 ++++++++++++------------
4 files changed, 28 insertions(+), 28 deletions(-)
diff --git a/arch/riscv/include/asm/cpu_ops_sbi.h b/arch/riscv/include/asm/cpu_ops_sbi.h
index d6e4665b3195..d967adad6b48 100644
--- a/arch/riscv/include/asm/cpu_ops_sbi.h
+++ b/arch/riscv/include/asm/cpu_ops_sbi.h
@@ -19,8 +19,8 @@ extern const struct cpu_operations cpu_ops_sbi;
* @stack_ptr: A pointer to the hart specific sp
*/
struct sbi_hart_boot_data {
- void *task_ptr;
- void *stack_ptr;
+ xlen_t task_ptr;
+ xlen_t stack_ptr;
};
#endif
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 945b7be249c1..d31135715f0e 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -123,16 +123,16 @@ enum sbi_ext_pmu_fid {
};
union sbi_pmu_ctr_info {
- unsigned long value;
+ xlen_t value;
struct {
- unsigned long csr:12;
- unsigned long width:6;
+ xlen_t csr:12;
+ xlen_t width:6;
#if __riscv_xlen == 32
- unsigned long reserved:13;
+ xlen_t reserved:13;
#else
- unsigned long reserved:45;
+ xlen_t reserved:45;
#endif
- unsigned long type:1;
+ xlen_t type:1;
};
};
@@ -254,15 +254,15 @@ enum sbi_pmu_ctr_type {
extern unsigned long sbi_spec_version;
struct sbiret {
- long error;
- long value;
+ xlen_t error;
+ xlen_t value;
};
void sbi_init(void);
-struct sbiret sbi_ecall(int ext, int fid, unsigned long arg0,
- unsigned long arg1, unsigned long arg2,
- unsigned long arg3, unsigned long arg4,
- unsigned long arg5);
+struct sbiret sbi_ecall(int ext, int fid, xlen_t arg0,
+ xlen_t arg1, xlen_t arg2,
+ xlen_t arg3, xlen_t arg4,
+ xlen_t arg5);
void sbi_console_putchar(int ch);
int sbi_console_getchar(void);
diff --git a/arch/riscv/kernel/cpu_ops_sbi.c b/arch/riscv/kernel/cpu_ops_sbi.c
index efa0f0816634..01a1e270ec1d 100644
--- a/arch/riscv/kernel/cpu_ops_sbi.c
+++ b/arch/riscv/kernel/cpu_ops_sbi.c
@@ -71,8 +71,8 @@ static int sbi_cpu_start(unsigned int cpuid, struct task_struct *tidle)
/* Make sure tidle is updated */
smp_mb();
- bdata->task_ptr = tidle;
- bdata->stack_ptr = task_stack_page(tidle) + THREAD_SIZE;
+ bdata->task_ptr = (ulong)tidle;
+ bdata->stack_ptr = (ulong)task_stack_page(tidle) + THREAD_SIZE;
/* Make sure boot data is updated */
smp_mb();
hsm_data = __pa(bdata);
diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
index 5c87db8fdff2..b649562aff61 100644
--- a/arch/riscv/kernel/sbi.c
+++ b/arch/riscv/kernel/sbi.c
@@ -22,21 +22,21 @@ static int (*__sbi_rfence)(int fid, const struct cpumask *cpu_mask,
unsigned long start, unsigned long size,
unsigned long arg4, unsigned long arg5) __ro_after_init;
-struct sbiret sbi_ecall(int ext, int fid, unsigned long arg0,
- unsigned long arg1, unsigned long arg2,
- unsigned long arg3, unsigned long arg4,
- unsigned long arg5)
+struct sbiret sbi_ecall(int ext, int fid, xlen_t arg0,
+ xlen_t arg1, xlen_t arg2,
+ xlen_t arg3, xlen_t arg4,
+ xlen_t arg5)
{
struct sbiret ret;
- register uintptr_t a0 asm ("a0") = (uintptr_t)(arg0);
- register uintptr_t a1 asm ("a1") = (uintptr_t)(arg1);
- register uintptr_t a2 asm ("a2") = (uintptr_t)(arg2);
- register uintptr_t a3 asm ("a3") = (uintptr_t)(arg3);
- register uintptr_t a4 asm ("a4") = (uintptr_t)(arg4);
- register uintptr_t a5 asm ("a5") = (uintptr_t)(arg5);
- register uintptr_t a6 asm ("a6") = (uintptr_t)(fid);
- register uintptr_t a7 asm ("a7") = (uintptr_t)(ext);
+ register xlen_t a0 asm ("a0") = arg0;
+ register xlen_t a1 asm ("a1") = arg1;
+ register xlen_t a2 asm ("a2") = arg2;
+ register xlen_t a3 asm ("a3") = arg3;
+ register xlen_t a4 asm ("a4") = arg4;
+ register xlen_t a5 asm ("a5") = arg5;
+ register xlen_t a6 asm ("a6") = fid;
+ register xlen_t a7 asm ("a7") = ext;
asm volatile ("ecall"
: "+r" (a0), "+r" (a1)
: "r" (a2), "r" (a3), "r" (a4), "r" (a5), "r" (a6), "r" (a7)
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 08/22] riscv: s64ilp32: Add asid support
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (6 preceding siblings ...)
2023-05-18 13:09 ` [RFC PATCH 07/22] riscv: s64ilp32: Add sbi support guoren
@ 2023-05-18 13:09 ` guoren
2023-05-18 13:10 ` [RFC PATCH 09/22] riscv: s64ilp32: Introduce PTR_L and PTR_S guoren
` (15 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:09 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The s32ilp32 uses 9 bits as asid_bits because of the xlen=32
limitation of CSR. The xlen of s64ilp32 is 64 bits in width, and
the SATP CSR format is the same for Sv32, Sv39, Sv48, and Sv57. So
this patch makes asid mechanism support s64ilp32 with maximum
num_asids.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/mm/context.c | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c
index 80ce9caba8d2..696a7ca41f55 100644
--- a/arch/riscv/mm/context.c
+++ b/arch/riscv/mm/context.c
@@ -20,9 +20,9 @@
DEFINE_STATIC_KEY_FALSE(use_asid_allocator);
-static unsigned long asid_bits;
+static xlen_t asid_bits;
static unsigned long num_asids;
-static unsigned long asid_mask;
+static xlen_t asid_mask;
static atomic_long_t current_version;
@@ -225,14 +225,18 @@ static inline void set_mm(struct mm_struct *mm, unsigned int cpu)
static int __init asids_init(void)
{
- unsigned long old;
+ xlen_t old;
/* Figure-out number of ASID bits in HW */
old = csr_read(CSR_SATP);
asid_bits = old | (SATP_ASID_MASK << SATP_ASID_SHIFT);
csr_write(CSR_SATP, asid_bits);
asid_bits = (csr_read(CSR_SATP) >> SATP_ASID_SHIFT) & SATP_ASID_MASK;
- asid_bits = fls_long(asid_bits);
+#if __riscv_xlen == 64
+ asid_bits = fls64(asid_bits);
+#else
+ asid_bits = fls(asid_bits);
+#endif
csr_write(CSR_SATP, old);
/*
@@ -265,9 +269,9 @@ static int __init asids_init(void)
static_branch_enable(&use_asid_allocator);
pr_info("ASID allocator using %lu bits (%lu entries)\n",
- asid_bits, num_asids);
+ (ulong)asid_bits, num_asids);
} else {
- pr_info("ASID allocator disabled (%lu bits)\n", asid_bits);
+ pr_info("ASID allocator disabled (%lu bits)\n", (ulong)asid_bits);
}
return 0;
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 09/22] riscv: s64ilp32: Introduce PTR_L and PTR_S
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (7 preceding siblings ...)
2023-05-18 13:09 ` [RFC PATCH 08/22] riscv: s64ilp32: Add asid support guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 10/22] riscv: s64ilp32: Enable user space runtime environment guoren
` (14 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
REG_L and REG_S can't satisfy s64ilp32 situation, because its
__SIZEOF_POINTER__*8 != __riscv_xlen. So we introduce new PTR_L
and PTR_S macro to help head.S and entry.S deal with the pointer
data type and replace all REG_L/S by PTR_L/S to fit the current
algorithm in memcpy, memove, memset, strcmp, strlen and strncmp.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/asm.h | 5 +++++
arch/riscv/kernel/entry.S | 24 ++++++++++++------------
arch/riscv/kernel/head.S | 8 ++++----
3 files changed, 21 insertions(+), 16 deletions(-)
diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index 114bbadaef41..1cf20027bdbd 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -38,6 +38,7 @@
#define RISCV_SZPTR "8"
#define RISCV_LGPTR "3"
#endif
+#define __PTR_SEL(a, b) __ASM_STR(a)
#elif __SIZEOF_POINTER__ == 4
#ifdef __ASSEMBLY__
#define RISCV_PTR .word
@@ -48,10 +49,14 @@
#define RISCV_SZPTR "4"
#define RISCV_LGPTR "2"
#endif
+#define __PTR_SEL(a, b) __ASM_STR(b)
#else
#error "Unexpected __SIZEOF_POINTER__"
#endif
+#define PTR_L __PTR_SEL(ld, lw)
+#define PTR_S __PTR_SEL(sd, sw)
+
#if (__SIZEOF_INT__ == 4)
#define RISCV_INT __ASM_STR(.word)
#define RISCV_SZINT __ASM_STR(4)
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 3fbb100bc9e4..9d8a94fec097 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -25,19 +25,19 @@ SYM_CODE_START(handle_exception)
_restore_kernel_tpsp:
csrr tp, CSR_SCRATCH
- REG_S sp, TASK_TI_KERNEL_SP(tp)
+ PTR_S sp, TASK_TI_KERNEL_SP(tp)
#ifdef CONFIG_VMAP_STACK
addi sp, sp, -(PT_SIZE_ON_STACK)
srli sp, sp, THREAD_SHIFT
andi sp, sp, 0x1
bnez sp, handle_kernel_stack_overflow
- REG_L sp, TASK_TI_KERNEL_SP(tp)
+ PTR_L sp, TASK_TI_KERNEL_SP(tp)
#endif
_save_context:
- REG_S sp, TASK_TI_USER_SP(tp)
- REG_L sp, TASK_TI_KERNEL_SP(tp)
+ PTR_S sp, TASK_TI_USER_SP(tp)
+ PTR_L sp, TASK_TI_KERNEL_SP(tp)
addi sp, sp, -(PT_SIZE_ON_STACK)
REG_S x1, PT_RA(sp)
REG_S x3, PT_GP(sp)
@@ -53,7 +53,7 @@ _save_context:
*/
li t0, SR_SUM | SR_FS
- REG_L s0, TASK_TI_USER_SP(tp)
+ PTR_L s0, TASK_TI_USER_SP(tp)
csrrc s1, CSR_STATUS, t0
csrr s2, CSR_EPC
csrr s3, CSR_TVAL
@@ -96,7 +96,7 @@ _save_context:
add t0, t1, t0
/* Check if exception code lies within bounds */
bgeu t0, t2, 1f
- REG_L t0, 0(t0)
+ PTR_L t0, 0(t0)
jr t0
1:
tail do_trap_unknown
@@ -121,7 +121,7 @@ SYM_CODE_START_NOALIGN(ret_from_exception)
/* Save unwound kernel stack pointer in thread_info */
addi s0, sp, PT_SIZE_ON_STACK
- REG_S s0, TASK_TI_KERNEL_SP(tp)
+ PTR_S s0, TASK_TI_KERNEL_SP(tp)
/*
* Save TP into the scratch register , so we can find the kernel data
@@ -239,7 +239,7 @@ restore_caller_reg:
REG_S x5, PT_T0(sp)
save_from_x6_to_x31
- REG_L s0, TASK_TI_KERNEL_SP(tp)
+ PTR_L s0, TASK_TI_KERNEL_SP(tp)
csrr s1, CSR_STATUS
csrr s2, CSR_EPC
csrr s3, CSR_TVAL
@@ -283,8 +283,8 @@ SYM_FUNC_START(__switch_to)
li a4, TASK_THREAD_RA
add a3, a0, a4
add a4, a1, a4
- REG_S ra, TASK_THREAD_RA_RA(a3)
- REG_S sp, TASK_THREAD_SP_RA(a3)
+ PTR_S ra, TASK_THREAD_RA_RA(a3)
+ PTR_S sp, TASK_THREAD_SP_RA(a3)
REG_S s0, TASK_THREAD_S0_RA(a3)
REG_S s1, TASK_THREAD_S1_RA(a3)
REG_S s2, TASK_THREAD_S2_RA(a3)
@@ -298,8 +298,8 @@ SYM_FUNC_START(__switch_to)
REG_S s10, TASK_THREAD_S10_RA(a3)
REG_S s11, TASK_THREAD_S11_RA(a3)
/* Restore context from next->thread */
- REG_L ra, TASK_THREAD_RA_RA(a4)
- REG_L sp, TASK_THREAD_SP_RA(a4)
+ PTR_L ra, TASK_THREAD_RA_RA(a4)
+ PTR_L sp, TASK_THREAD_SP_RA(a4)
REG_L s0, TASK_THREAD_S0_RA(a4)
REG_L s1, TASK_THREAD_S1_RA(a4)
REG_L s2, TASK_THREAD_S2_RA(a4)
diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
index 4bf6c449d78b..27d134ee754f 100644
--- a/arch/riscv/kernel/head.S
+++ b/arch/riscv/kernel/head.S
@@ -42,7 +42,7 @@ ENTRY(_start)
/* Image load offset (0MB) from start of RAM for M-mode */
.dword 0
#else
-#if __riscv_xlen == 64
+#ifdef CONFIG_64BIT
/* Image load offset(2MB) from start of RAM */
.dword 0x200000
#else
@@ -75,7 +75,7 @@ relocate_enable_mmu:
/* Relocate return address */
la a1, kernel_map
XIP_FIXUP_OFFSET a1
- REG_L a1, KERNEL_MAP_VIRT_ADDR(a1)
+ PTR_L a1, KERNEL_MAP_VIRT_ADDR(a1)
la a2, _start
sub a1, a1, a2
add ra, ra, a1
@@ -346,8 +346,8 @@ clear_bss_done:
*/
.Lwait_for_cpu_up:
/* FIXME: We should WFI to save some energy here. */
- REG_L sp, (a1)
- REG_L tp, (a2)
+ PTR_L sp, (a1)
+ PTR_L tp, (a2)
beqz sp, .Lwait_for_cpu_up
beqz tp, .Lwait_for_cpu_up
fence
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 10/22] riscv: s64ilp32: Enable user space runtime environment
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (8 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 09/22] riscv: s64ilp32: Introduce PTR_L and PTR_S guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 11/22] riscv: s64ilp32: Add ebpf jit support guoren
` (13 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
RV64ILP32 uses the same setting from COMPAT mode for the user
space runtime environment. They all have the same 2GB TASK_SIZE.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/csr.h | 2 --
arch/riscv/include/asm/pgtable.h | 8 +++++++-
arch/riscv/kernel/process.c | 4 +++-
3 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h
index adc3c866d353..7558e0808af4 100644
--- a/arch/riscv/include/asm/csr.h
+++ b/arch/riscv/include/asm/csr.h
@@ -36,11 +36,9 @@
#define SR_SD _AC(0x8000000000000000, ULL) /* FS/XS dirty */
#endif
-#if __riscv_xlen == 64
#define SR_UXL _AC(0x300000000, ULL) /* XLEN mask for U-mode */
#define SR_UXL_32 _AC(0x100000000, ULL) /* XLEN = 32 for U-mode */
#define SR_UXL_64 _AC(0x200000000, ULL) /* XLEN = 64 for U-mode */
-#endif
/* SATP flags */
#if __riscv_xlen == 32
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 3303fc03d724..d7b8eff0ade9 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -830,26 +830,32 @@ static inline pte_t pte_swp_clear_exclusive(pte_t pte)
* - 0x9fc00000 (~2.5GB) for RV32.
* - 0x4000000000 ( 256GB) for RV64 using SV39 mmu
* - 0x800000000000 ( 128TB) for RV64 using SV48 mmu
+ * - 0x80000000 ( 2GB) for COMPAT and RV64ILP32
*
* Note that PGDIR_SIZE must evenly divide TASK_SIZE since "RISC-V
* Instruction Set Manual Volume II: Privileged Architecture" states that
* "load and store effective addresses, which are 64bits, must have bits
* 63–48 all equal to bit 47, or else a page-fault exception will occur."
*/
+#define TASK_SIZE_32 (_AC(0x80000000, UL) - PAGE_SIZE)
+
#ifdef CONFIG_64BIT
#define TASK_SIZE_64 (PGDIR_SIZE * PTRS_PER_PGD / 2)
#define TASK_SIZE_MIN (PGDIR_SIZE_L3 * PTRS_PER_PGD / 2)
#ifdef CONFIG_COMPAT
-#define TASK_SIZE_32 (_AC(0x80000000, UL) - PAGE_SIZE)
#define TASK_SIZE (test_thread_flag(TIF_32BIT) ? \
TASK_SIZE_32 : TASK_SIZE_64)
#else
#define TASK_SIZE TASK_SIZE_64
#endif
+#else
+#ifdef CONFIG_ARCH_RV64ILP32
+#define TASK_SIZE TASK_SIZE_32
#else
#define TASK_SIZE FIXADDR_START
+#endif
#define TASK_SIZE_MIN TASK_SIZE
#endif
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index ab01b51a5bb9..e033dbe5b5eb 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -125,13 +125,15 @@ void start_thread(struct pt_regs *regs, unsigned long pc,
regs->epc = pc;
regs->sp = sp;
-#ifdef CONFIG_64BIT
regs->status &= ~SR_UXL;
+#ifdef CONFIG_64BIT
if (is_compat_task())
regs->status |= SR_UXL_32;
else
regs->status |= SR_UXL_64;
+#else
+ regs->status |= SR_UXL_32;
#endif
}
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 11/22] riscv: s64ilp32: Add ebpf jit support
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (9 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 10/22] riscv: s64ilp32: Enable user space runtime environment guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 12/22] riscv: s64ilp32: Add ELF32 support guoren
` (12 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The s64ilp32 uses the rv64 ISA instruction set, not the rv32 ISA. So
bpf_jit_comp32.c can't be used for s64ilp32, and we use bpf_jit_comp64.c
instead. This patch makes s64ilp32 ebpf jit correct and improves the
performance because bpf_jit_comp32.c has significant gaps in mapping
ebpf 64-bit ISA.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/extable.h | 2 +-
arch/riscv/net/Makefile | 6 +++---
arch/riscv/net/bpf_jit_comp64.c | 10 +++++-----
3 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/arch/riscv/include/asm/extable.h b/arch/riscv/include/asm/extable.h
index 512012d193dc..3ad79a7989e2 100644
--- a/arch/riscv/include/asm/extable.h
+++ b/arch/riscv/include/asm/extable.h
@@ -34,7 +34,7 @@ do { \
bool fixup_exception(struct pt_regs *regs);
-#if defined(CONFIG_BPF_JIT) && defined(CONFIG_ARCH_RV64I)
+#if defined(CONFIG_BPF_JIT) && !defined(CONFIG_ARCH_RV32I)
bool ex_handler_bpf(const struct exception_table_entry *ex, struct pt_regs *regs);
#else
static inline bool
diff --git a/arch/riscv/net/Makefile b/arch/riscv/net/Makefile
index 9a1e5f0a94e5..907edce21acc 100644
--- a/arch/riscv/net/Makefile
+++ b/arch/riscv/net/Makefile
@@ -2,8 +2,8 @@
obj-$(CONFIG_BPF_JIT) += bpf_jit_core.o
-ifeq ($(CONFIG_ARCH_RV64I),y)
- obj-$(CONFIG_BPF_JIT) += bpf_jit_comp64.o
-else
+ifeq ($(CONFIG_ARCH_RV32I),y)
obj-$(CONFIG_BPF_JIT) += bpf_jit_comp32.o
+else
+ obj-$(CONFIG_BPF_JIT) += bpf_jit_comp64.o
endif
diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index f5a668736c79..5a65cd01c73c 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -125,7 +125,7 @@ static u8 rv_tail_call_reg(struct rv_jit_context *ctx)
static bool is_32b_int(s64 val)
{
- return -(1L << 31) <= val && val < (1L << 31);
+ return -(1LL << 31) <= val && val < (1LL << 31);
}
static bool in_auipc_jalr_range(s64 val)
@@ -134,15 +134,15 @@ static bool in_auipc_jalr_range(s64 val)
* auipc+jalr can reach any signed PC-relative offset in the range
* [-2^31 - 2^11, 2^31 - 2^11).
*/
- return (-(1L << 31) - (1L << 11)) <= val &&
- val < ((1L << 31) - (1L << 11));
+ return (-(1LL << 31) - (1LL << 11)) <= val &&
+ val < ((1LL << 31) - (1LL << 11));
}
/* Emit fixed-length instructions for address */
static int emit_addr(u8 rd, u64 addr, bool extra_pass, struct rv_jit_context *ctx)
{
- u64 ip = (u64)(ctx->insns + ctx->ninsns);
- s64 off = addr - ip;
+ ulong ip = (ulong)(ctx->insns + ctx->ninsns);
+ s64 off = (ulong)addr - ip;
s64 upper = (off + (1 << 11)) >> 12;
s64 lower = off & 0xfff;
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 12/22] riscv: s64ilp32: Add ELF32 support
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (10 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 11/22] riscv: s64ilp32: Add ebpf jit support guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 13/22] riscv: s64ilp32: Add ARCH RV64 ILP32 compiling framework guoren
` (11 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Use abi_len to distinct ELF32 and ELF64 because s64ilp32 is xlen=64 and
abi_len=32 (__SIZEOF_POINTER__=4). And s64ilp32 is an ELF32 based the
same as s32ilp32.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kerenl.org>
---
arch/riscv/include/uapi/asm/elf.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/riscv/include/uapi/asm/elf.h b/arch/riscv/include/uapi/asm/elf.h
index d696d6610231..962e8ec8fe05 100644
--- a/arch/riscv/include/uapi/asm/elf.h
+++ b/arch/riscv/include/uapi/asm/elf.h
@@ -24,7 +24,7 @@ typedef __u64 elf_fpreg_t;
typedef union __riscv_fp_state elf_fpregset_t;
#define ELF_NFPREG (sizeof(struct __riscv_d_ext_state) / sizeof(elf_fpreg_t))
-#if __riscv_xlen == 64
+#if __SIZEOF_POINTER__ == 8
#define ELF_RISCV_R_SYM(r_info) ELF64_R_SYM(r_info)
#define ELF_RISCV_R_TYPE(r_info) ELF64_R_TYPE(r_info)
#else
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 13/22] riscv: s64ilp32: Add ARCH RV64 ILP32 compiling framework
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (11 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 12/22] riscv: s64ilp32: Add ELF32 support guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 14/22] riscv: s64ilp32: Add MMU_SV39 mode support for 32BIT guoren
` (10 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Just the same as ARCH_RV64I & ARCH_RV32I, add ARCH_RV64ILP32 config
for s64ilp32 and turn on the s64ilp32 compile switch in the
arch/riscv/Makefile.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 6 ++++++
arch/riscv/Makefile | 5 +++++
2 files changed, 11 insertions(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 4d4fac81390f..d824fcf3cc1c 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -300,6 +300,12 @@ config ARCH_RV64I
select ARCH_SUPPORTS_INT128 if CC_HAS_INT128
select SWIOTLB if MMU
+config ARCH_RV64ILP32
+ bool "RV64ILP32"
+ depends on NONPORTABLE
+ select 32BIT
+ select MMU
+
endchoice
# We must be able to map all physical memory into the kernel, but the compiler
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index dafe958c4217..d47ba6b09b41 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -57,6 +57,7 @@ endif
# ISA string setting
riscv-march-$(CONFIG_ARCH_RV32I) := rv32ima
riscv-march-$(CONFIG_ARCH_RV64I) := rv64ima
+riscv-march-$(CONFIG_ARCH_RV64ILP32) := rv64ima
riscv-march-$(CONFIG_FPU) := $(riscv-march-y)fd
riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c
@@ -107,7 +108,11 @@ stack_protector_prepare: prepare0
endif
# arch specific predefines for sparse
+ifeq ($(CONFIG_ARCH_RV64ILP32),y)
+CHECKFLAGS += -D__riscv
+else
CHECKFLAGS += -D__riscv -D__riscv_xlen=$(BITS)
+endif
# Default target when executing plain make
boot := arch/riscv/boot
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 14/22] riscv: s64ilp32: Add MMU_SV39 mode support for 32BIT
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (12 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 13/22] riscv: s64ilp32: Add ARCH RV64 ILP32 compiling framework guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 15/22] riscv: s64ilp32: Enable native atomic64 guoren
` (9 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
There is no MMU_SV32 support in xlen=64 ISA generally, but s64ilp32
selects 32BIT, which uses MMU_SV32 default. This commit enables MMU_SV39
for 32BIT to satisfy the 4GB mapping requirement. The Sv39 is the
mandatory MMU mode in RVA20S64 and RVA22S64, so we needn't care about
Sv48 & Sv57.
We use duplicate remapping to solve the address sign extension problem
from the compiler. Make the address of 0xffffffff80000000 equal to
0x80000000 by pg_dir[2] = pg_dir[510] and pg_dir[3] = pg_dir[511] of the
page table.
Why didn't we prevent address sign extension in the compiler?
- Additional zero extension reduces the performance
- Prevent complex and unnecessary work for compiler guys.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 10 ++++++-
arch/riscv/include/asm/page.h | 24 ++++++++++++-----
arch/riscv/include/asm/pgtable-64.h | 42 ++++++++++++++---------------
arch/riscv/include/asm/pgtable.h | 18 ++++++++-----
arch/riscv/kernel/cpu.c | 4 +--
arch/riscv/mm/fault.c | 11 ++++++++
arch/riscv/mm/init.c | 29 ++++++++++++++++----
7 files changed, 96 insertions(+), 42 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index d824fcf3cc1c..9c458496ec3a 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -233,7 +233,7 @@ config FIX_EARLYCON_MEM
config PGTABLE_LEVELS
int
- default 5 if 64BIT
+ default 5 if !MMU_SV32
default 2
config LOCKDEP_SUPPORT
@@ -293,6 +293,8 @@ config ARCH_RV32I
select GENERIC_LIB_ASHRDI3
select GENERIC_LIB_LSHRDI3
select GENERIC_LIB_UCMPDI2
+ select MMU
+ select MMU_SV32
config ARCH_RV64I
bool "RV64I"
@@ -531,6 +533,12 @@ config FPU
If you don't know what to do here, say Y.
+config MMU_SV32
+ bool "MMU using Sv32 mode"
+ depends on ARCH_RV32I
+ help
+ Say N here if you have a 64-bit processor without satp-sv32 mode support.
+
endmenu # "Platform type"
menu "Kernel features"
diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h
index b55ba20903ec..7c535e88cf91 100644
--- a/arch/riscv/include/asm/page.h
+++ b/arch/riscv/include/asm/page.h
@@ -61,16 +61,28 @@ void clear_page(void *page);
/* Page Global Directory entry */
typedef struct {
- unsigned long pgd;
+#ifndef CONFIG_MMU_SV32
+ u64 pgd;
+#else
+ u32 pgd;
+#endif
} pgd_t;
/* Page Table entry */
typedef struct {
- unsigned long pte;
+#ifndef CONFIG_MMU_SV32
+ u64 pte;
+#else
+ u32 pte;
+#endif
} pte_t;
typedef struct {
- unsigned long pgprot;
+#ifndef CONFIG_MMU_SV32
+ u64 pgprot;
+#else
+ u32 pgprot;
+#endif
} pgprot_t;
typedef struct page *pgtable_t;
@@ -83,10 +95,10 @@ typedef struct page *pgtable_t;
#define __pgd(x) ((pgd_t) { (x) })
#define __pgprot(x) ((pgprot_t) { (x) })
-#ifdef CONFIG_64BIT
-#define PTE_FMT "%016lx"
+#ifndef CONFIG_MMU_SV32
+#define PTE_FMT "%016llx"
#else
-#define PTE_FMT "%08lx"
+#define PTE_FMT "%08x"
#endif
#ifdef CONFIG_64BIT
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 7a5097202e15..3da589de28a0 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -45,7 +45,7 @@ extern bool pgtable_l5_enabled;
/* Page 4th Directory entry */
typedef struct {
- unsigned long p4d;
+ u64 p4d;
} p4d_t;
#define p4d_val(x) ((x).p4d)
@@ -54,7 +54,7 @@ typedef struct {
/* Page Upper Directory entry */
typedef struct {
- unsigned long pud;
+ u64 pud;
} pud_t;
#define pud_val(x) ((x).pud)
@@ -63,7 +63,7 @@ typedef struct {
/* Page Middle Directory entry */
typedef struct {
- unsigned long pmd;
+ u64 pmd;
} pmd_t;
#define pmd_val(x) ((x).pmd)
@@ -76,7 +76,7 @@ typedef struct {
* | 63 | 62 61 | 60 54 | 53 10 | 9 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
* N MT RSV PFN reserved for SW D A G U X W R V
*/
-#define _PAGE_PFN_MASK GENMASK(53, 10)
+#define _PAGE_PFN_MASK GENMASK_ULL(53, 10)
/*
* [63] Svnapot definitions:
@@ -120,8 +120,8 @@ enum napot_cont_order {
* 10 - IO Non-cacheable, non-idempotent, strongly-ordered I/O memory
* 11 - Rsvd Reserved for future standard use
*/
-#define _PAGE_NOCACHE_SVPBMT (1UL << 61)
-#define _PAGE_IO_SVPBMT (1UL << 62)
+#define _PAGE_NOCACHE_SVPBMT (1ULL << 61)
+#define _PAGE_IO_SVPBMT (1ULL << 62)
#define _PAGE_MTMASK_SVPBMT (_PAGE_NOCACHE_SVPBMT | _PAGE_IO_SVPBMT)
/*
@@ -131,10 +131,10 @@ enum napot_cont_order {
* 01110 - PMA Weakly-ordered, Cacheable, Bufferable, Shareable, Non-trustable
* 10000 - IO Strongly-ordered, Non-cacheable, Non-bufferable, Non-shareable, Non-trustable
*/
-#define _PAGE_PMA_THEAD ((1UL << 62) | (1UL << 61) | (1UL << 60))
-#define _PAGE_NOCACHE_THEAD 0UL
-#define _PAGE_IO_THEAD (1UL << 63)
-#define _PAGE_MTMASK_THEAD (_PAGE_PMA_THEAD | _PAGE_IO_THEAD | (1UL << 59))
+#define _PAGE_PMA_THEAD ((1ULL << 62) | (1ULL << 61) | (1ULL << 60))
+#define _PAGE_NOCACHE_THEAD 0ULL
+#define _PAGE_IO_THEAD (1ULL << 63)
+#define _PAGE_MTMASK_THEAD (_PAGE_PMA_THEAD | _PAGE_IO_THEAD | (1ULL << 59))
static inline u64 riscv_page_mtmask(void)
{
@@ -165,7 +165,7 @@ static inline u64 riscv_page_io(void)
#define _PAGE_MTMASK riscv_page_mtmask()
/* Set of bits to preserve across pte_modify() */
-#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ | \
+#define _PAGE_CHG_MASK (~(u64)(_PAGE_PRESENT | _PAGE_READ | \
_PAGE_WRITE | _PAGE_EXEC | \
_PAGE_USER | _PAGE_GLOBAL | \
_PAGE_MTMASK))
@@ -206,12 +206,12 @@ static inline void pud_clear(pud_t *pudp)
set_pud(pudp, __pud(0));
}
-static inline pud_t pfn_pud(unsigned long pfn, pgprot_t prot)
+static inline pud_t pfn_pud(u64 pfn, pgprot_t prot)
{
return __pud((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
}
-static inline unsigned long _pud_pfn(pud_t pud)
+static inline u64 _pud_pfn(pud_t pud)
{
return __page_val_to_pfn(pud_val(pud));
}
@@ -246,16 +246,16 @@ static inline bool mm_pud_folded(struct mm_struct *mm)
#define pmd_index(addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1))
-static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
+static inline pmd_t pfn_pmd(u64 pfn, pgprot_t prot)
{
- unsigned long prot_val = pgprot_val(prot);
+ u64 prot_val = pgprot_val(prot);
ALT_THEAD_PMA(prot_val);
return __pmd((pfn << _PAGE_PFN_SHIFT) | prot_val);
}
-static inline unsigned long _pmd_pfn(pmd_t pmd)
+static inline u64 _pmd_pfn(pmd_t pmd)
{
return __page_val_to_pfn(pmd_val(pmd));
}
@@ -263,13 +263,13 @@ static inline unsigned long _pmd_pfn(pmd_t pmd)
#define mk_pmd(page, prot) pfn_pmd(page_to_pfn(page), prot)
#define pmd_ERROR(e) \
- pr_err("%s:%d: bad pmd %016lx.\n", __FILE__, __LINE__, pmd_val(e))
+ pr_err("%s:%d: bad pmd " PTE_FMT ".\n", __FILE__, __LINE__, pmd_val(e))
#define pud_ERROR(e) \
- pr_err("%s:%d: bad pud %016lx.\n", __FILE__, __LINE__, pud_val(e))
+ pr_err("%s:%d: bad pud " PTE_FMT ".\n", __FILE__, __LINE__, pud_val(e))
#define p4d_ERROR(e) \
- pr_err("%s:%d: bad p4d %016lx.\n", __FILE__, __LINE__, p4d_val(e))
+ pr_err("%s:%d: bad p4d " PTE_FMT ".\n", __FILE__, __LINE__, p4d_val(e))
static inline void set_p4d(p4d_t *p4dp, p4d_t p4d)
{
@@ -309,12 +309,12 @@ static inline void p4d_clear(p4d_t *p4d)
set_p4d(p4d, __p4d(0));
}
-static inline p4d_t pfn_p4d(unsigned long pfn, pgprot_t prot)
+static inline p4d_t pfn_p4d(u64 pfn, pgprot_t prot)
{
return __p4d((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
}
-static inline unsigned long _p4d_pfn(p4d_t p4d)
+static inline u64 _p4d_pfn(p4d_t p4d)
{
return __page_val_to_pfn(p4d_val(p4d));
}
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index d7b8eff0ade9..c5e915f21354 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -34,7 +34,11 @@
* Half of the kernel address space (1/4 of the entries of the page global
* directory) is for the direct mapping.
*/
+#if IS_ENABLED(CONFIG_ARCH_RV64ILP32) && !IS_ENABLED(CONFIG_MMU_SV32)
+#define KERN_VIRT_SIZE (PTRS_PER_PGD * PMD_SIZE)
+#else
#define KERN_VIRT_SIZE ((PTRS_PER_PGD / 2 * PGDIR_SIZE) / 2)
+#endif
#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1)
#define VMALLOC_END PAGE_OFFSET
@@ -86,11 +90,7 @@
#define PCI_IO_START (PCI_IO_END - PCI_IO_SIZE)
#define FIXADDR_TOP PCI_IO_START
-#ifdef CONFIG_64BIT
#define FIXADDR_SIZE PMD_SIZE
-#else
-#define FIXADDR_SIZE PGDIR_SIZE
-#endif
#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
#endif
@@ -110,11 +110,11 @@
#define __page_val_to_pfn(_val) (((_val) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT)
-#ifdef CONFIG_64BIT
+#ifndef CONFIG_MMU_SV32
#include <asm/pgtable-64.h>
#else
#include <asm/pgtable-32.h>
-#endif /* CONFIG_64BIT */
+#endif /* !CONFIG_MMU_SV32 */
#include <linux/page_table_check.h>
@@ -524,7 +524,11 @@ static inline int ptep_set_access_flags(struct vm_area_struct *vma,
static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
unsigned long address, pte_t *ptep)
{
+#ifndef CONFIG_MMU_SV32
+ pte_t pte = __pte(atomic64_xchg((atomic64_t *)ptep, 0));
+#else
pte_t pte = __pte(atomic_long_xchg((atomic_long_t *)ptep, 0));
+#endif
page_table_check_pte_clear(mm, address, pte);
@@ -538,7 +542,7 @@ static inline int ptep_test_and_clear_young(struct vm_area_struct *vma,
{
if (!pte_young(*ptep))
return 0;
- return test_and_clear_bit(_PAGE_ACCESSED_OFFSET, &pte_val(*ptep));
+ return test_and_clear_bit(_PAGE_ACCESSED_OFFSET, (unsigned long *)&pte_val(*ptep));
}
#define __HAVE_ARCH_PTEP_SET_WRPROTECT
diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
index 3df38052dcbd..1c8d880a5c7e 100644
--- a/arch/riscv/kernel/cpu.c
+++ b/arch/riscv/kernel/cpu.c
@@ -241,9 +241,9 @@ static void print_mmu(struct seq_file *f)
char sv_type[16];
#ifdef CONFIG_MMU
-#if defined(CONFIG_32BIT)
+#if defined(CONFIG_MMU_SV32)
strncpy(sv_type, "sv32", 5);
-#elif defined(CONFIG_64BIT)
+#else
if (pgtable_l5_enabled)
strncpy(sv_type, "sv57", 5);
else if (pgtable_l4_enabled)
diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c
index 1b0bd0683766..56eae0deefd2 100644
--- a/arch/riscv/mm/fault.c
+++ b/arch/riscv/mm/fault.c
@@ -131,7 +131,18 @@ static inline void vmalloc_fault(struct pt_regs *regs, int code, unsigned long a
no_context(regs, addr);
return;
}
+#if !IS_ENABLED(CONFIG_MMU_SV32) && IS_ENABLED(CONFIG_ARCH_RV64ILP32)
+ /*
+ * The pg_dir[2,510,3,511] has been set during early
+ * boot, so we only make a check here.
+ */
+ if (pgd_val(*pgd) != pgd_val(*pgd_k)) {
+ no_context(regs, addr);
+ return;
+ }
+#else
set_pgd(pgd, *pgd_k);
+#endif
p4d_k = p4d_offset(pgd_k, addr);
if (!p4d_present(*p4d_k)) {
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index bce899b180cd..3ee5b80affce 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -43,8 +43,12 @@ EXPORT_SYMBOL(kernel_map);
#ifdef CONFIG_64BIT
u64 satp_mode __ro_after_init = !IS_ENABLED(CONFIG_XIP_KERNEL) ? SATP_MODE_57 : SATP_MODE_39;
#else
+#ifndef CONFIG_MMU_SV32
+u64 satp_mode __ro_after_init = SATP_MODE_39;
+#else
u64 satp_mode __ro_after_init = SATP_MODE_32;
#endif
+#endif
EXPORT_SYMBOL(satp_mode);
bool pgtable_l4_enabled = IS_ENABLED(CONFIG_64BIT) && !IS_ENABLED(CONFIG_XIP_KERNEL);
@@ -60,7 +64,12 @@ unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)]
EXPORT_SYMBOL(empty_zero_page);
extern char _start[];
+#if IS_ENABLED(CONFIG_ARCH_RV64ILP32) && !IS_ENABLED(CONFIG_MMU_SV32)
+#define DTB_EARLY_BASE_VA PGDIR_SIZE
+#else
#define DTB_EARLY_BASE_VA (ADDRESS_SPACE_END - (PTRS_PER_PGD / 2 * PGDIR_SIZE) + 1)
+#endif
+
void *_dtb_early_va __initdata;
uintptr_t _dtb_early_pa __initdata;
@@ -656,16 +665,26 @@ void __init create_pgd_mapping(pgd_t *pgdp,
pgd_next_t *nextp;
phys_addr_t next_phys;
uintptr_t pgd_idx = pgd_index(va);
+#if !IS_ENABLED(CONFIG_MMU_SV32) && IS_ENABLED(CONFIG_ARCH_RV64ILP32)
+ uintptr_t pgd_idh = pgd_index(sign_extend64((u64)va, 31));
+#endif
if (sz == PGDIR_SIZE) {
- if (pgd_val(pgdp[pgd_idx]) == 0)
+ if (pgd_val(pgdp[pgd_idx]) == 0) {
pgdp[pgd_idx] = pfn_pgd(PFN_DOWN(pa), prot);
+#if !IS_ENABLED(CONFIG_MMU_SV32) && IS_ENABLED(CONFIG_ARCH_RV64ILP32)
+ pgdp[pgd_idh] = pfn_pgd(PFN_DOWN(pa), prot);
+#endif
+ }
return;
}
if (pgd_val(pgdp[pgd_idx]) == 0) {
next_phys = alloc_pgd_next(va);
pgdp[pgd_idx] = pfn_pgd(PFN_DOWN(next_phys), PAGE_TABLE);
+#if !IS_ENABLED(CONFIG_MMU_SV32) && IS_ENABLED(CONFIG_ARCH_RV64ILP32)
+ pgdp[pgd_idh] = pfn_pgd(PFN_DOWN(next_phys), PAGE_TABLE);
+#endif
nextp = get_pgd_next_virt(next_phys);
memset(nextp, 0, PAGE_SIZE);
} else {
@@ -918,9 +937,9 @@ static void __init create_fdt_early_page_table(pgd_t *pgdir, uintptr_t dtb_pa)
uintptr_t pa = dtb_pa & ~(PMD_SIZE - 1);
create_pgd_mapping(early_pg_dir, DTB_EARLY_BASE_VA,
- IS_ENABLED(CONFIG_64BIT) ? early_dtb_pgd_next : pa,
+ !IS_ENABLED(CONFIG_MMU_SV32) ? early_dtb_pgd_next : pa,
PGDIR_SIZE,
- IS_ENABLED(CONFIG_64BIT) ? PAGE_TABLE : PAGE_KERNEL);
+ !IS_ENABLED(CONFIG_MMU_SV32) ? PAGE_TABLE : PAGE_KERNEL);
if (pgtable_l5_enabled)
create_p4d_mapping(early_dtb_p4d, DTB_EARLY_BASE_VA,
@@ -930,7 +949,7 @@ static void __init create_fdt_early_page_table(pgd_t *pgdir, uintptr_t dtb_pa)
create_pud_mapping(early_dtb_pud, DTB_EARLY_BASE_VA,
(uintptr_t)early_dtb_pmd, PUD_SIZE, PAGE_TABLE);
- if (IS_ENABLED(CONFIG_64BIT)) {
+ if (!IS_ENABLED(CONFIG_MMU_SV32)) {
create_pmd_mapping(early_dtb_pmd, DTB_EARLY_BASE_VA,
pa, PMD_SIZE, PAGE_KERNEL);
create_pmd_mapping(early_dtb_pmd, DTB_EARLY_BASE_VA + PMD_SIZE,
@@ -1149,7 +1168,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa)
fix_bmap_epmd = fixmap_pmd[pmd_index(__fix_to_virt(FIX_BTMAP_END))];
if (pmd_val(fix_bmap_spmd) != pmd_val(fix_bmap_epmd)) {
WARN_ON(1);
- pr_warn("fixmap btmap start [%08lx] != end [%08lx]\n",
+ pr_warn("fixmap btmap start [" PTE_FMT "] != end [" PTE_FMT "]\n",
pmd_val(fix_bmap_spmd), pmd_val(fix_bmap_epmd));
pr_warn("fix_to_virt(FIX_BTMAP_BEGIN): %08lx\n",
fix_to_virt(FIX_BTMAP_BEGIN));
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 15/22] riscv: s64ilp32: Enable native atomic64
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (13 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 14/22] riscv: s64ilp32: Add MMU_SV39 mode support for 32BIT guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 16/22] riscv: s64ilp32: Add TImode (128 int) support guoren
` (8 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The traditional rv32 Linux (s32ilp32) uses a generic version of the
lib/atomic64.c, which are inaccurate atomic64 primitives and couldn't
co-work with READ_ONCE/WRITE_ONCE, atomic_8/16/32. The s64ilp32 could
use native AMO instructions to implement accurate atomic64 primitives.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 2 +-
arch/riscv/include/asm/atomic.h | 6 ++++++
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 9c458496ec3a..33fe624ef6d3 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -60,7 +60,7 @@ config RISCV
select CPU_PM if CPU_IDLE
select EDAC_SUPPORT
select GENERIC_ARCH_TOPOLOGY
- select GENERIC_ATOMIC64 if !64BIT
+ select GENERIC_ATOMIC64 if ARCH_RV32I
select GENERIC_CLOCKEVENTS_BROADCAST if SMP
select GENERIC_EARLY_IOREMAP
select GENERIC_ENTRY
diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomic.h
index 0dfe9d857a76..edfa6a74fe04 100644
--- a/arch/riscv/include/asm/atomic.h
+++ b/arch/riscv/include/asm/atomic.h
@@ -16,6 +16,12 @@
# endif
#endif
+#ifdef CONFIG_ARCH_RV64ILP32
+typedef struct {
+ s64 counter;
+} atomic64_t;
+#endif
+
#include <asm/cmpxchg.h>
#include <asm/barrier.h>
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 16/22] riscv: s64ilp32: Add TImode (128 int) support
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (14 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 15/22] riscv: s64ilp32: Enable native atomic64 guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 17/22] riscv: s64ilp32: Implement cmpxchg_double guoren
` (7 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The s64ilp32 uses 64bit compiler, so it could support “Tetra
Integer” mode, which represents a sixteen-byte (128) integer.
It's the first 32BIT linux support TImode :)
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 1 +
arch/riscv/lib/Makefile | 1 +
2 files changed, 2 insertions(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 33fe624ef6d3..e0c3dee68510 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -305,6 +305,7 @@ config ARCH_RV64I
config ARCH_RV64ILP32
bool "RV64ILP32"
depends on NONPORTABLE
+ select ARCH_SUPPORTS_INT128 if !$(cc-option,$(m64-flag) -D__SIZEOF_INT128__=0)
select 32BIT
select MMU
diff --git a/arch/riscv/lib/Makefile b/arch/riscv/lib/Makefile
index 26cb2502ecf8..68af463795e1 100644
--- a/arch/riscv/lib/Makefile
+++ b/arch/riscv/lib/Makefile
@@ -9,5 +9,6 @@ lib-y += strncmp.o
lib-$(CONFIG_MMU) += uaccess.o
lib-$(CONFIG_64BIT) += tishift.o
lib-$(CONFIG_RISCV_ISA_ZICBOZ) += clear_page.o
+lib-$(CONFIG_ARCH_RV64ILP32) += tishift.o
obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 17/22] riscv: s64ilp32: Implement cmpxchg_double
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (15 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 16/22] riscv: s64ilp32: Add TImode (128 int) support guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 18/22] riscv: s64ilp32: Disable KVM guoren
` (6 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The s64ilp32 has the ability to exclusively load and store (ld/sd)
a pair of words from an address. Then the SLUB can take advantage
of a cmpxchg_double implementation to avoid taking some locks.
This patch provides an implementation of cmpxchg_double for 64-bit
pairs, and activates the logic required for the SLUB to use these
functions (HAVE_ALIGNED_STRUCT_PAGE and HAVE_CMPXCHG_DOUBLE).
Similar commit: 5284e1b4bc8a ("arm64: xchg: Implement
cmpxchg_double")
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 2 ++
arch/riscv/include/asm/cmpxchg.h | 53 ++++++++++++++++++++++++++++++++
2 files changed, 55 insertions(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index e0c3dee68510..51853f883fc5 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -78,6 +78,7 @@ config RISCV
select GENERIC_TIME_VSYSCALL if MMU
select GENERIC_VDSO_TIME_NS if HAVE_GENERIC_VDSO
select HARDIRQS_SW_RESEND
+ select HAVE_ALIGNED_STRUCT_PAGE if SLUB && ARCH_RV64ILP32
select HAVE_ARCH_AUDITSYSCALL
select HAVE_ARCH_HUGE_VMALLOC if HAVE_ARCH_HUGE_VMAP
select HAVE_ARCH_HUGE_VMAP if MMU && 64BIT && !XIP_KERNEL
@@ -96,6 +97,7 @@ config RISCV
select HAVE_ARCH_TRANSPARENT_HUGEPAGE if 64BIT && MMU
select HAVE_ARCH_VMAP_STACK if MMU && 64BIT
select HAVE_ASM_MODVERSIONS
+ select HAVE_CMPXCHG_DOUBLE if ARCH_RV64ILP32
select HAVE_CONTEXT_TRACKING_USER
select HAVE_DEBUG_KMEMLEAK
select HAVE_DMA_CONTIGUOUS if MMU
diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
index 12debce235e5..808730d151e7 100644
--- a/arch/riscv/include/asm/cmpxchg.h
+++ b/arch/riscv/include/asm/cmpxchg.h
@@ -7,6 +7,7 @@
#define _ASM_RISCV_CMPXCHG_H
#include <linux/bug.h>
+#include <linux/mmdebug.h>
#include <asm/barrier.h>
#include <asm/fence.h>
@@ -360,4 +361,56 @@
arch_cmpxchg_relaxed((ptr), (o), (n)); \
})
+#ifdef CONFIG_ARCH_RV64ILP32
+#define system_has_cmpxchg_double() 1
+
+#define __cmpxchg_double_check(ptr1, ptr2) \
+({ \
+ if (sizeof(*(ptr1)) != 4) \
+ BUILD_BUG(); \
+ if (sizeof(*(ptr2)) != 4) \
+ BUILD_BUG(); \
+ VM_BUG_ON((ulong *)(ptr2) - (ulong *)(ptr1) != 1); \
+ VM_BUG_ON(((ulong)ptr1 & 0x7) != 0); \
+})
+
+#define __cmpxchg_double(old1, old2, new1, new2, ptr) \
+({ \
+ __typeof__(ptr) __ptr = (ptr); \
+ register unsigned int __ret; \
+ u64 __old; \
+ u64 __new; \
+ u64 __tmp; \
+ switch (sizeof(*(ptr))) { \
+ case 4: \
+ __old = ((u64)old2 << 32) | (u64)old1; \
+ __new = ((u64)new2 << 32) | (u64)new1; \
+ __asm__ __volatile__ ( \
+ "0: lr.d %0, %2\n" \
+ " bne %0, %z3, 1f\n" \
+ " sc.d %1, %z4, %2\n" \
+ " bnez %1, 0b\n" \
+ "1:\n" \
+ : "=&r" (__tmp), "=&r" (__ret), "+A" (*__ptr) \
+ : "rJ" (__old), "rJ" (__new) \
+ : "memory"); \
+ __ret = (__old == __tmp); \
+ break; \
+ default: \
+ BUILD_BUG(); \
+ } \
+ __ret; \
+})
+
+#define arch_cmpxchg_double(ptr1, ptr2, o1, o2, n1, n2) \
+({ \
+ int __ret; \
+ __cmpxchg_double_check(ptr1, ptr2); \
+ __ret = __cmpxchg_double((ulong)(o1), (ulong)(o2), \
+ (ulong)(n1), (ulong)(n2), \
+ ptr1); \
+ __ret; \
+})
+#endif
+
#endif /* _ASM_RISCV_CMPXCHG_H */
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 18/22] riscv: s64ilp32: Disable KVM
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (16 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 17/22] riscv: s64ilp32: Implement cmpxchg_double guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 19/22] riscv: Cleanup rv32_defconfig guoren
` (5 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Disable the KVM host feature for s64ilp32 first, and let's work on this
feature after the s64ilp32 main feature is merged.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
Cc: Anup Patel <anup@brainfault.org>
---
arch/riscv/kvm/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/riscv/kvm/Kconfig b/arch/riscv/kvm/Kconfig
index d5a658a047a7..a91ce7e0b754 100644
--- a/arch/riscv/kvm/Kconfig
+++ b/arch/riscv/kvm/Kconfig
@@ -20,6 +20,7 @@ if VIRTUALIZATION
config KVM
tristate "Kernel-based Virtual Machine (KVM) support (EXPERIMENTAL)"
depends on RISCV_SBI && MMU
+ depends on !ARCH_RV64ILP32
select KVM_GENERIC_HARDWARE_ENABLING
select MMU_NOTIFIER
select PREEMPT_NOTIFIERS
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 19/22] riscv: Cleanup rv32_defconfig
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (17 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 18/22] riscv: s64ilp32: Disable KVM guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 20/22] riscv: s64ilp32: Add rv64ilp32_defconfig guoren
` (4 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren, Randy Dunlap
From: Guo Ren <guoren@linux.alibaba.com>
Remove unnecessary configs to make rv32_defconfig have a minimal
difference from the defconfig. CONFIG_ARCH_RV32I selects the
CONFIG_32BIT, so putting it in the file is unnecessary. Also, there is
no need to comment on CONFIG_PORTABLE; it should come from carelessness.
Next rv64ilp32_defconfig would like:
CONFIG_ARCH_RV64ILP32=y
CONFIG_NONPORTABLE=y
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Palmer Dabbelt <palmer@rivosinc.com>
---
arch/riscv/configs/32-bit.config | 2 --
1 file changed, 2 deletions(-)
diff --git a/arch/riscv/configs/32-bit.config b/arch/riscv/configs/32-bit.config
index f6af0f708df4..eb87885c8640 100644
--- a/arch/riscv/configs/32-bit.config
+++ b/arch/riscv/configs/32-bit.config
@@ -1,4 +1,2 @@
CONFIG_ARCH_RV32I=y
-CONFIG_32BIT=y
-# CONFIG_PORTABLE is not set
CONFIG_NONPORTABLE=y
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 20/22] riscv: s64ilp32: Add rv64ilp32_defconfig
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (18 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 19/22] riscv: Cleanup rv32_defconfig guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 21/22] riscv: s64ilp32: Correct the rv64ilp32 stackframe layout guoren
` (3 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Follow the rv32_defconfig rule to add rv64ilp32_defconfig; the only
difference is:
-CONFIG_ARCH_RV32I=y
+CONFIG_ARCH_RV64ILP32=y
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Makefile | 4 ++++
arch/riscv/configs/64ilp32.config | 2 ++
2 files changed, 6 insertions(+)
create mode 100644 arch/riscv/configs/64ilp32.config
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index d47ba6b09b41..22b62de62192 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -187,3 +187,7 @@ rv32_defconfig:
PHONY += rv32_nommu_virt_defconfig
rv32_nommu_virt_defconfig:
$(Q)$(MAKE) -f $(srctree)/Makefile nommu_virt_defconfig 32-bit.config
+
+PHONY += rv64ilp32_defconfig
+rv64ilp32_defconfig:
+ $(Q)$(MAKE) -f $(srctree)/Makefile defconfig 64ilp32.config
diff --git a/arch/riscv/configs/64ilp32.config b/arch/riscv/configs/64ilp32.config
new file mode 100644
index 000000000000..7d836aa2fae7
--- /dev/null
+++ b/arch/riscv/configs/64ilp32.config
@@ -0,0 +1,2 @@
+CONFIG_ARCH_RV64ILP32=y
+CONFIG_NONPORTABLE=y
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 21/22] riscv: s64ilp32: Correct the rv64ilp32 stackframe layout
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (19 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 20/22] riscv: s64ilp32: Add rv64ilp32_defconfig guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 13:10 ` [RFC PATCH 22/22] riscv: s64ilp32: Temporary workaround solution to gcc problem guoren
` (2 subsequent siblings)
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The callee saved fp & ra are xlen size, not long size. This patch
corrects the layout for the struct stackframe.
echo c > /proc/sysrq-trigger
Before the patch:
sysrq: Trigger a crash
Kernel panic - not syncing: sysrq triggered crash
CPU: 0 PID: 102 Comm: sh Not tainted 6.3.0-rc1-00084-g9e2ba938797e-dirty #2
Hardware name: riscv-virtio,qemu (DT)
Call Trace:
---[ end Kernel panic - not syncing: sysrq triggered crash ]---
After the patch:
sysrq: Trigger a crash
Kernel panic - not syncing: sysrq triggered crash
CPU: 0 PID: 102 Comm: sh Not tainted 6.3.0-rc1-00084-g9e2ba938797e-dirty #1
Hardware name: riscv-virtio,qemu (DT)
Call Trace:
[<c00050c8>] dump_backtrace+0x1e/0x26
[<c086dcae>] show_stack+0x2e/0x3c
[<c0878e00>] dump_stack_lvl+0x40/0x5a
[<c0878e30>] dump_stack+0x16/0x1e
[<c086df7c>] panic+0x10c/0x2a8
[<c04f4c1e>] sysrq_reset_seq_param_set+0x0/0x76
[<c04f52cc>] __handle_sysrq+0x9c/0x19c
[<c04f5946>] write_sysrq_trigger+0x64/0x78
[<c020c7f6>] proc_reg_write+0x4a/0xa2
[<c01acf0a>] vfs_write+0xac/0x308
[<c01ad2b8>] ksys_write+0x62/0xda
[<c01ad33e>] sys_write+0xe/0x16
[<c0879860>] do_trap_ecall_u+0xd8/0xda
[<c00037de>] ret_from_exception+0x0/0x66
---[ end Kernel panic - not syncing: sysrq triggered crash ]---
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/stacktrace.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/arch/riscv/include/asm/stacktrace.h b/arch/riscv/include/asm/stacktrace.h
index f7e8ef2418b9..cea8aafbecca 100644
--- a/arch/riscv/include/asm/stacktrace.h
+++ b/arch/riscv/include/asm/stacktrace.h
@@ -8,7 +8,13 @@
struct stackframe {
unsigned long fp;
+#ifdef CONFIG_ARCH_RV64ILP32
+ unsigned long pad1;
+#endif
unsigned long ra;
+#ifdef CONFIG_ARCH_RV64ILP32
+ unsigned long pad2;
+#endif
};
extern void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs,
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 22/22] riscv: s64ilp32: Temporary workaround solution to gcc problem
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (20 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 21/22] riscv: s64ilp32: Correct the rv64ilp32 stackframe layout guoren
@ 2023-05-18 13:10 ` guoren
2023-05-18 15:38 ` [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode Palmer Dabbelt
2023-05-19 20:20 ` Arnd Bergmann
23 siblings, 0 replies; 37+ messages in thread
From: guoren @ 2023-05-18 13:10 UTC (permalink / raw)
To: arnd, guoren, palmer, tglx, peterz, luto, conor.dooley, heiko,
jszhang, chenhuacai, apatel, atishp, mark.rutland, bjorn,
paul.walmsley, catalin.marinas, will, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27
Cc: linux-arch, linux-kernel, linux-riscv, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
There is an existing problem in 64ilp32 gcc that combines two pointers
in one register. Liao is solving that problem. Before he finishes the
job, we could prevent it with a simple noinline attribute, fortunately.
struct path {
struct vfsmount *mnt;
struct dentry *dentry;
} __randomize_layout;
struct nameidata {
struct path path;
struct qstr last;
struct path root;
...
} __randomize_layout;
struct nameidata *nd
...
nd->path = nd->root;
6c88 ld a0,24(s1)
^^ // Wrong arg of mntget
e088 sd a0,0(s1)
// Need inserting "lw a0,0(s1)" here
mntget(path->mnt);
2a6150ef jal c01ce946 <mntget>
Any gcc helps are welcome :)
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
Cc: LiaoShihua <shihua@iscas.ac.cn>
---
fs/namei.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/fs/namei.c b/fs/namei.c
index edfedfbccaef..22a2ef77e074 100644
--- a/fs/namei.c
+++ b/fs/namei.c
@@ -541,7 +541,7 @@ EXPORT_SYMBOL(inode_permission);
*
* Given a path increment the reference count to the dentry and the vfsmount.
*/
-void path_get(const struct path *path)
+void noinline path_get(const struct path *path)
{
mntget(path->mnt);
dget(path->dentry);
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (21 preceding siblings ...)
2023-05-18 13:10 ` [RFC PATCH 22/22] riscv: s64ilp32: Temporary workaround solution to gcc problem guoren
@ 2023-05-18 15:38 ` Palmer Dabbelt
2023-05-18 18:29 ` Arnd Bergmann
2023-05-19 0:14 ` Paul Walmsley
2023-05-19 20:20 ` Arnd Bergmann
23 siblings, 2 replies; 37+ messages in thread
From: Palmer Dabbelt @ 2023-05-18 15:38 UTC (permalink / raw)
To: guoren
Cc: Arnd Bergmann, guoren, tglx, peterz, luto, Conor Dooley, heiko,
jszhang, chenhuacai, apatel, atishp, Mark Rutland, bjorn,
Paul Walmsley, Catalin Marinas, Will Deacon, rppt, anup, shihua,
jiawei, liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27, linux-arch, linux-kernel, linux-riscv, guoren
On Thu, 18 May 2023 06:09:51 PDT (-0700), guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> This patch series adds s64ilp32 support to riscv. The term s64ilp32
> means smode-xlen=64 and -mabi=ilp32 (ints, longs, and pointers are all
> 32-bit), i.e., running 32-bit Linux kernel on pure 64-bit supervisor
> mode. There have been many 64ilp32 abis existing, such as mips-n32 [1],
> arm-aarch64ilp32 [2], and x86-x32 [3], but they are all about userspace.
> Thus, this should be the first time running a 32-bit Linux kernel with
> the 64ilp32 ABI at supervisor mode (If not, correct me).
Does anyone actually want this? At a bare minimum we'd need to add it
to the psABI, which would presumably also be required on the compiler
side of things.
It's not even clear anyone wants rv64/ilp32 in userspace, the kernel
seems like it'd be even less widely used.
> Why 32-bit Linux?
> =================
> The motivation for using a 32-bit Linux kernel is to reduce memory
> footprint and meet the small capacity of DDR & cache requirement
> (e.g., 64/128MB SIP SoC).
>
> Here are the 32-bit v.s. 64-bit Linux kernel data type comparison
> summary:
> 32-bit 64-bit
> sizeof(page): 32bytes 64bytes
> sizeof(list_head): 8bytes 16bytes
> sizeof(hlist_head): 8bytes 16bytes
> sizeof(vm_area): 68bytes 136bytes
> ...
>
> The size of ilp32's long & pointer is just half of lp64's (rv64 default
> abi - longs and pointers are all 64-bit). This significant difference
> in data type causes different memory & cache footprint costs. Here is
> the comparison measurement between s32ilp32, s64ilp32, and s64lp64 in
> the same 128MB qemu system environment:
>
> Rootfs:
> u32ilp32 - Using the same 32-bit userspace rootfs.ext2 (UXL=32) binary
> from buildroot 2023.02-rc3, qemu_riscv32_virt_defconfig
>
> Linux:
> s32ilp32 - Linux version 6.3.0-rc1 (124MB)
> rv32_defconfig: $(Q)$(MAKE) -f $(srctree)/Makefile
> defconfig 32-bit.config
>
> s64lp64 - Linux version 6.3.0-rc1 (126MB)
> defconfig: $(Q)$(MAKE) -f $(srctree)/Makefile defconfig
>
> s64ilp32 - Linux version 6.3.0-rc1 (126MB)
> rv64ilp32_defconfig: $(Q)$(MAKE) -f $(srctree)/Makefile
> defconfig 64ilp32.config
>
> Opensbi:
> m64lp64 - (2MB) OpenSBI v1.2-80-g4b28afc98bbe
> m32ilp32 - (4MB) OpenSBI v1.2-80-g4b28afc98bbe
>
> +----------------------------------------+--------
> | u32ilp32 |
> | UXL=32 | Rootfs
> +----------------------------------------+--------
> | +----------+ +---------+ | +---------+ |
> | | s64ilp32 | | s64lp64 | | | s32ilp32| |
> | | SXL=64 | | SXL=64 | | | SXL=32 | | Linux
> | +----------+ +---------+ | +---------+ |
> +----------------------------------------+--------
> | +----------------------+ | +---------+ |
> | | m64lp64 | | | m32ilp32| |
> | | MXL=64 | | | MXL=32 | | Opensbi
> | +----------------------+ | +---------+ |
> +----------------------------------------+--------
> | +----------------------+ | +---------+ |
> | | qemu-rv64 | | |qemu-rv32| | HW
> | +----------------------+ | +---------+ |
> +----------------------------------------+--------
>
> Mem-usage:
> (s32ilp32) # free
> total used free shared buff/cache available
> Mem: 100040 8380 88244 44 3416 88080
>
> (s64lp64) # free
> total used free shared buff/cache available
> Mem: 91568 11848 75796 44 3924 75952
>
> (s64ilp32) # free
> total used free shared buff/cache available
> Mem: 101952 8528 90004 44 3420 89816
> ^^^^^
>
> It's a rough measurement based on the current default config without any
> modification, and 32-bit (s32ilp32, s64ilp32) saved more than 16% memory
> to 64-bit (s64lp64). But s32ilp32 & s64ilp32 have a similar memory
> footprint (about 0.33% difference), meaning s64ilp32 has a big chance to
> replace s32ilp32 on the 64-bit machine.
>
> Why s64ilp32?
> =============
> The current RISC-V has the profiles of RVA20S64, RVA22S64, and RVA23S64
> (ongoing) [4], but no RVA**S32 profile exists or any ongoing plan. That
> means when a vendor wants to produce a 32-bit s-mode RISC-V Application
> Processor, they have no shape to follow. Therefore, many cheap riscv
> chips have come out but follow the RVA2xS64 profiles, such as Allwinner
> D1/D1s/F133 [5], SOPHGO CV1800B [6], Canaan Kendryte k230 [7], and
> Bouffalo Lab BL808 which are typically cortex a7/a35/a53 product
> scenarios. The D1 & CV1800B & BL808 didn't support UXL=32 (32-bit U-mode),
> so they need a new u64ilp32 userspace ABI which has no software ecosystem
> for the current. Thus, the first landing of s64ilp32 would be on Canaan
> Kendryte k230, which has c908 with rv64gcv and compat user mode
> (sstatus.uxl=32/64), which could support the existing rv32 userspace
> software ecosystem.
>
> Another reason for inventing s64ilp32 is performance benefits and
> simplify 64-bit CPU hardware design (v.s. s32ilp32).
>
> Why s64ilp32 has better performance?
> ====================================
> Generally speaking, we should build a 32-bit hardware s-mode to run
> 32-bit Linux on a 64-bit processor (such as Linux-arm32 on cortex-a53).
> Or only use old 32ilp32-abi on a 64-bit machine (such as mips
> SYS_SUPPORTS_32BIT_KERNEL). These can't reuse performance-related
> features and instructions of the 64-bit hardware, such as 64-bit ALU,
> AMO, and LD/SD, which would cause significant performance gaps on many
> Linux features:
>
> - memcpy/memset/strcmp (s64ilp32 has half of the instructions count
> and double the bandwidth of load/store instructions than s32ilp32.)
>
> - ebpf JIT is a 64-bit virtual ISA, which is not suitable
> for mapping to s32ilp32.
>
> - Atomic64 (s64ilp32 has the exact native instructions mapping as
> s64lp64, but s32ilp32 only uses generic_atomic64, a tradeoff &
> limited software solution.)
>
> - 64-bit native arithmetic instructions for "long long" type
>
> - Support cmxchg_double for slub (The 2nd 32-bit Linux
> supports the feature, the 1st is i386.)
>
> - ...
>
> Compared with the user space ecosystem, the 32-bit Linux kernel is more
> eager to need 64ilp32 to improve performance because the Linux kernel
> can't utilize float-point/vector features of the ISA.
>
> Let's look at performance from another perspective (s64ilp32 v.s.
> s64lp64). Just as the first chapter said, the pointer size of ilp32 is
> half of the lp64, and it reduces the size of the critical data structs
> (e.g., page, list, ...). That means the cache of using ilp32 could
> contain double data that lp64 with the same cache capacity, which is a
> natural advantage of 32-bit.
>
> Why s64ilp32 simplifies CPU design?
> ===================================
> Yes, there are a lot of runing 32-bit Linux on 64-bit hardware examples
> in history, such as arm cortex a35/a53/a55, which implements the 32-bit
> EL1/EL2/EL3 hardware mode to support 32-bit Linux. We could follow Arm's
> style, but riscv could choose another better way. Compared to UXL=32,
> the MXL=SXL=32 has many CSR-related hardware functionalities, which
> causes a lot of effort to mix them into 64-bit hardware. The s64ilp32
> works on MXL=SXL=64 mode, so the CPU vendors needn't implement 32-bit
> machine and supervisor modes.
>
> How does s64ilp32 work?
> =======================
> The s64ilp32 is the same as the s64lp64 compat mode from a hardware
> view, i.e., MXL=SXL=64 + UXL=32. Because the s64ilp32 uses CONFIG_32BIT
> of Linux, it only supports u32ilp32 abi user space, the current standard
> rv32 software ecosystem, and it can't work with u64lp64 abi (I don't
> want that complex and useless stuff). But it may work with u64ilp32 in the
> future; now, the s64ilp32 depends on the UXL=32 feature of the hardware.
>
> The 64ilp32 gcc still uses sign-extend lw & auipc to generate address
> variables because inserting zero-extend instructions to mask the highest
> 32-bit would cause significant code size and performance problems. Thus,
> we invented an OS approach to solve the problem:
> - When satp=bare and start physical address < 2GB, there is no sign-extend
> address problem.
> - When satp=bare and start physical address > 2GB, we need zjpm liked
> hardware extensions to mask high 32bit.
> (Fortunately, all existed SoCs' (D1/D1s/F133, CV1800B, k230, BL808)
> start physical address < 2GB.)
> - When satp=sv39, we invent double mapping to make the sign-extended
> virtual address the same as the zero-extended virtual address.
>
> +--------+ +---------+ +--------+
> | | +--| 511:PUD1| | |
> | | | +---------+ | |
> | | | | 510:PUD0|--+ | |
> | | | +---------+ | | |
> | | | | | | | |
> | | | | | | | |
> | | | | | | | |
> | | | | INVALID | | | |
> | | | | | | | |
> | .... | | | | | | .... |
> | | | | | | | |
> | | | +---------+ | | |
> | | +--| 3:PUD1 | | | |
> | | | +---------+ | | |
> | | | | 2:PUD0 |--+ | |
> | | | +---------+ | | |
> | | | |1:USR_PUD| | | |
> | | | +---------+ | | |
> | | | |0:USR_PUD| | | |
> +--------+<--+ +---------+ +-->+--------+
> PUD1 ^ PGD PUD0
> 1GB | 4GB 1GB
> |
> +----------+
> | Sv39 PGDP|
> +----------+
> SATP
>
> The size of xlen was always equal to the pointer/long size before
> s64ilp32 emerged. So we need to introduce a new type of data - xlen_t,
> which could deal with CSR-related and callee-save/restore operations.
>
> Some kernel features use 32BIT/64BIT to determine the exact ISA, such as
> ebpf JIT would map to rv32 ISA when CONFIG_32BIT=y. But s64ilp32 needs
> the ebpf JIT map to rv64 ISA when CONFIG_32BIT=y and we need to use
> another config to distinguish the difference.
>
> More detials, please review the path series.
>
> How to run s64ilp32?
> ====================
>
> GNU toolchain
> -------------
> git clone https://github.com/Liaoshihua/riscv-gnu-toolchain.git
> cd riscv-gnu-toolchain
> ./configure --prefix="$PWD/opt-rv64-ilp32/" --with-arch=rv64imac --with-abi=ilp32
> make linux
> export PATH=$PATH:$PWD/opt-rv64-ilp32/bin/
>
> Opensbi
> -------
> git clone https://github.com/riscv-software-src/opensbi.git
> CROSS_COMPILE=riscv64-unknown-linux-gnu- make PLATFORM=generic
>
> Linux kernel
> ------------
> git clone https://github.com/guoren83/linux.git -b s64ilp32
> cd linux
> make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- rv64ilp32_defconfig
> make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- all
>
> Rootfs
> ------
> git clone git://git.busybox.net/buildroot
> cd buildroot
> make qemu_riscv32_virt_defconfig
> make
>
> Qemu
> ----
> git clone https://github.com/plctlab/plct-qemu.git -b plct-s64ilp32-dev
> cd plct-qemu
> mkdir build
> cd build
> ../qemu/configure --target-list="riscv64-softmmu riscv32-softmmu"
> make
>
> Run
> ---
> ./qemu-system-riscv64 -cpu rv64 -M virt -m 128m -nographic -bios fw_dynamic.bin -kernel Image -drive file=rootfs.ext2,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 -append "rootwait root=/dev/vda ro console=ttyS0 earlycon=sbi" -netdev user,id=net0 -device virtio-net-device,netdev=net0
>
> OpenSBI v1.2-119-gdc1c7db05e07
> ____ _____ ____ _____
> / __ \ / ____| _ \_ _|
> | | | |_ __ ___ _ __ | (___ | |_) || |
> | | | | '_ \ / _ \ '_ \ \___ \| _ < | |
> | |__| | |_) | __/ | | |____) | |_) || |_
> \____/| .__/ \___|_| |_|_____/|___/_____|
> | |
> |_|
>
> Platform Name : riscv-virtio,qemu
> Platform Features : medeleg
> Platform HART Count : 1
> Platform IPI Device : aclint-mswi
> Platform Timer Device : aclint-mtimer @ 10000000Hz
> Platform Console Device : uart8250
> Platform HSM Device : ---
> Platform PMU Device : ---
> Platform Reboot Device : sifive_test
> Platform Shutdown Device : sifive_test
> Platform Suspend Device : ---
> Platform CPPC Device : ---
> Firmware Base : 0x60000000
> Firmware Size : 360 KB
> Firmware RW Offset : 0x40000
> Runtime SBI Version : 1.0
>
> Domain0 Name : root
> Domain0 Boot HART : 0
> Domain0 HARTs : 0*
> Domain0 Region00 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
> Domain0 Region01 : 0x0000000060040000-0x000000006005ffff M: (R,W) S/U: ()
> Domain0 Region02 : 0x0000000060000000-0x000000006003ffff M: (R,X) S/U: ()
> Domain0 Region03 : 0x0000000000000000-0xffffffffffffffff M: (R,W,X) S/U: (R,W,X)
> Domain0 Next Address : 0x0000000060200000
> Domain0 Next Arg1 : 0x0000000067e00000
> Domain0 Next Mode : S-mode
> Domain0 SysReset : yes
> Domain0 SysSuspend : yes
>
> Boot HART ID : 0
> Boot HART Domain : root
> Boot HART Priv Version : v1.12
> Boot HART Base ISA : rv64imafdch
> Boot HART ISA Extensions : time,sstc
> Boot HART PMP Count : 16
> Boot HART PMP Granularity : 4
> Boot HART PMP Address Bits: 54
> Boot HART MHPM Count : 16
> Boot HART MIDELEG : 0x0000000000001666
> Boot HART MEDELEG : 0x0000000000f0b509
> [ 0.000000] Linux version 6.3.0-rc1-00086-gc8d2fedb997a (guoren@fedora) (riscv64-unknown-linux-gnu-gcc (g5e578a16201f) 13.0.1 20230206 (experimental), GNU ld (GNU Binutils) 2.40.50.20230205) #1 SMP Sun May 14 10:46:42 EDT 2023
> [ 0.000000] random: crng init done
> [ 0.000000] OF: fdt: Ignoring memory range 0x60000000 - 0x60200000
> [ 0.000000] Machine model: riscv-virtio,qemu
> [ 0.000000] efi: UEFI not found.
> [ 0.000000] OF: reserved mem: 0x60000000..0x6003ffff (256 KiB) map non-reusable mmode_resv1@60000000
> [ 0.000000] OF: reserved mem: 0x60040000..0x6005ffff (128 KiB) map non-reusable mmode_resv0@60040000
> [ 0.000000] Zone ranges:
> [ 0.000000] Normal [mem 0x0000000060200000-0x0000000067ffffff]
> [ 0.000000] Movable zone start for each node
> [ 0.000000] Early memory node ranges
> [ 0.000000] node 0: [mem 0x0000000060200000-0x0000000067ffffff]
> [ 0.000000] Initmem setup node 0 [mem 0x0000000060200000-0x0000000067ffffff]
> [ 0.000000] On node 0, zone Normal: 512 pages in unavailable ranges
> [ 0.000000] SBI specification v1.0 detected
> [ 0.000000] SBI implementation ID=0x1 Version=0x10002
> [ 0.000000] SBI TIME extension detected
> [ 0.000000] SBI IPI extension detected
> [ 0.000000] SBI RFENCE extension detected
> [ 0.000000] SBI SRST extension detected
> [ 0.000000] SBI HSM extension detected
> [ 0.000000] riscv: base ISA extensions acdfhim
> [ 0.000000] riscv: ELF capabilities acdfim
> [ 0.000000] percpu: Embedded 13 pages/cpu s24352 r8192 d20704 u53248
> [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 31941
> [ 0.000000] Kernel command line: rootwait root=/dev/vda ro console=ttyS0 earlycon=sbi norandmaps
> [ 0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes, linear)
> [ 0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes, linear)
> [ 0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off
> [ 0.000000] Virtual kernel memory layout:
> [ 0.000000] fixmap : 0x9ce00000 - 0x9d000000 (2048 kB)
> [ 0.000000] pci io : 0x9d000000 - 0x9e000000 ( 16 MB)
> [ 0.000000] vmemmap : 0x9e000000 - 0xa0000000 ( 32 MB)
> [ 0.000000] vmalloc : 0xa0000000 - 0xc0000000 ( 512 MB)
> [ 0.000000] lowmem : 0xc0000000 - 0xc7e00000 ( 126 MB)
> [ 0.000000] Memory: 97748K/129024K available (8699K kernel code, 8867K rwdata, 4096K rodata, 4204K init, 361K bss, 31276K reserved, 0K cma-reserved)
> ...
> Starting network: udhcpc: started, v1.36.0
> udhcpc: broadcasting discover
> udhcpc: broadcasting select for 10.0.2.15, server 10.0.2.2
> udhcpc: lease of 10.0.2.15 obtained from 10.0.2.2, lease time 86400
> deleting routers
> adding dns 10.0.2.3
> OK
>
> Welcome to Buildroot
> buildroot login: root
> # cat /proc/cpuinfo
> processor : 0
> hart : 0
> isa : rv64imafdch_zihintpause_zbb_sstc
> mmu : sv39
> mvendorid : 0x0
> marchid : 0x70232
> mimpid : 0x70232
>
> # uname -a
> Linux buildroot 6.3.0-rc1-00086-gc8d2fedb997a #1 SMP Sun May 14 10:46:42 EDT 2023 riscv32 GNU/Linux
> # ls /lib/
> ld-linux-riscv32-ilp32d.so.1 libgcc_s.so.1
> libanl.so.1 libm.so.6
> libatomic.so libnss_dns.so.2
> libatomic.so.1 libnss_files.so.2
> libatomic.so.1.2.0 libpthread.so.0
> libc.so.6 libresolv.so.2
> libcrypt.so.1 librt.so.1
> libdl.so.2 libutil.so.1
> libgcc_s.so modules
>
> # cat /proc/99/maps
> 0000000055554000-0000000055634000 r-xp 00000000 00000000fe:00 17 /bin/busybox
> 0000000055634000-0000000055636000 r--p 00000000df000 00000000fe:00 17 /bin/busybox
> 0000000055636000-0000000055637000 rw-p 00000000e1000 00000000fe:00 17 /bin/busybox
> 0000000055637000-0000000055659000 rw-p 00000000 00:00 0 [heap]
> 0000000077e8d000-0000000077fbe000 r-xp 00000000 00000000fe:00 137 /lib/libc.so.6
> 0000000077fbe000-0000000077fbf000 ---p 00000000131000 00000000fe:00 137 /lib/libc.so.6
> 0000000077fbf000-0000000077fc1000 r--p 00000000131000 00000000fe:00 137 /lib/libc.so.6
> 0000000077fc1000-0000000077fc2000 rw-p 00000000133000 00000000fe:00 137 /lib/libc.so.6
> 0000000077fc2000-0000000077fcc000 rw-p 00000000 00:00 0
> 0000000077fcc000-0000000077fd4000 r-xp 00000000 00000000fe:00 146 /lib/libresolv.so.2
> 0000000077fd4000-0000000077fd5000 ---p 000000008000 00000000fe:00 146 /lib/libresolv.so.2
> 0000000077fd5000-0000000077fd6000 r--p 000000008000 00000000fe:00 146 /lib/libresolv.so.2
> 0000000077fd6000-0000000077fd7000 rw-p 000000009000 00000000fe:00 146 /lib/libresolv.so.2
> 0000000077fd7000-0000000077fd9000 rw-p 00000000 00:00 0
> 0000000077fd9000-0000000077fdb000 r--p 00000000 00:00 0 [vvar]
> 0000000077fdb000-0000000077fdd000 r-xp 00000000 00:00 0 [vdso]
> 0000000077fdd000-0000000077ffc000 r-xp 00000000 00000000fe:00 132 /lib/ld-linux-riscv32-ilp32d.so.1
> 0000000077ffd000-0000000077ffe000 r--p 000000001f000 00000000fe:00 132 /lib/ld-linux-riscv32-ilp32d.so.1
> 0000000077ffe000-0000000077fff000 rw-p 0000000020000 00000000fe:00 132 /lib/ld-linux-riscv32-ilp32d.so.1
> 000000007ffde000-000000007ffff000 rw-p 00000000 00:00 0 [stack]
>
> Other resources
> ===============
>
> OpenEuler riscv32 rootfs
> ------------------------
> The OpenEuler riscv32 rootfs you can download from here:
> https://repo.tarsier-infra.com/openEuler-RISC-V/obs/archive/rv32/openeuler-image-qemu-riscv32-20221111070036.rootfs.ext4
> (Made by Junqiang Wang)
>
> Debain riscv32 rootfs
> ---------------------
> The Debian riscv32 rootfs you can download from here:
> https://github.com/yuzibo/riscv32
> (Made by Bo YU and Han Gao)
>
> Fedora riscv32 rootfs
> ---------------------
> https://fedoraproject.org/wiki/Architectures/RISC-V/RV32
> (Made by Wei Fu)
>
> LLVM 64ilp32
> ------------
> git clone https://github.com/luxufan/llvm-project.git -b rv64-ilp32
> cd llvm-project
> mkdir build && cd build
> cmake ../llvm -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=“X86;RISCV" -DLLVM_ENABLE_PROJECTS="clang;lld"
> ninja all
>
> (LLVM development status is that CC=clang can compile the kernel with
> LLVM=1 but has not yet booted successfully.)
>
> Patch organization
> ==================
> This series depends on 64ilp32 toolchain patches that are not upstream
> yet.
>
> PATCH [0-1] unify vdso32 & compat_vdso
> PATCH [2] adds time-related vDSO common flow for vdso32
> PATCH [3] adds s64ilp32 support of clocksource driver
> PATCH [5] adds s64ilp32 support of irqchip driver
> PATCH [4,6-12] add basic data types and compiling framework
> PATCH [13] adds MMU_SV39 support
> PATCH [14] adds native atomic64
> PATCH [15] adds TImode
> PATCH [16] adds cmpxchg_double
> PATCH [17-19] cleanup kconfig & add defconfig
> PATCH [20-21] fix temporary compiler problems
>
> Open issues
> ===========
>
> Callee saved the register width
> -------------------------------
> For 64-bit ISA (including 64lp64, 64ilp32), callee can't determine the
> correct width used in the register, so they saved the maximum width of
> the ISA register, i.e., xlen size. We also found this rule in x86-x32,
> mips-n32, and aarch64ilp32, which comes from 64lp64. See PATCH [20]
>
> Here are two downsides of this:
> - It would cause a difference with 32ilp32's stack frame, and s64ilp32
> reuses 32ilp32 software stack. Thus, many additional compatible
> problems would happen during the porting of 64ilp32 software.
> - It also increases the budget of the stack usage.
> <setup_vm>:
> auipc a3,0xff3fb
> add a3,a3,1234 # c0000000
> li a5,-1
> lui a4,0xc0000
> addw sp,sp,-96
> srl a5,a5,0x20
> subw a4,a4,a3
> auipc a2,0x111a
> add a2,a2,1212 # c1d1f000
> sd s0,80(sp)----+
> sd s1,72(sp) |
> sd s2,64(sp) |
> sd s7,24(sp) |
> sd s8,16(sp) |
> sd s9,8(sp) |-> All <= 32b widths, but occupy 64b
> sd ra,88(sp) | stack space.
> sd s3,56(sp) | Affect memory footprint & cache
> sd s4,48(sp) | performance.
> sd s5,40(sp) |
> sd s6,32(sp) |
> sd s10,0(sp)----+
> sll a1,a4,0x20
> subw a2,a2,a3
> and a4,a4,a5
>
> So here is a proposal to riscv 64ilp32 ABI:
> - Let the compiler prevent callee saving ">32b variables" in
> callee-registers. (Q: We need to measure, how the influence of
> 64b variables cross function call?)
>
> EF_RISCV_X32
> ------------
> We add an e_flag (EF_RISCV_X32) to distinguish the 32-bit ELF, which
> occupies BIT[6] of the e_flags layout.
>
> ELF Header:
> Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
> Class: ELF32
> Data: 2's complement, little endian
> Version: 1 (current)
> OS/ABI: UNIX - System V
> ABI Version: 0
> Type: REL (Relocatable file)
> Machine: RISC-V
> Version: 0x1
> Entry point address: 0x0
> Start of program headers: 0 (bytes into file)
> Start of section headers: 24620 (bytes into file)
> Flags: 0x21, RVC, X32, soft-float ABI
> ^^^
> 64-bit Optimization problem
> ---------------------------
> There is an existing problem in 64ilp32 gcc that combines two pointers
> in one register. Liao is solving that problem. Before he finishes the
> job, we could prevent it with a simple noinline attribute, fortunately.
> struct path {
> struct vfsmount *mnt;
> struct dentry *dentry;
> } __randomize_layout;
>
> struct nameidata {
> struct path path;
> ...
> struct path root;
> ...
> } __randomize_layout;
>
> struct nameidata *nd
> ...
> nd->path = nd->root;
> 6c88 ld a0,24(s1)
> ^^ // a0 contains two pointers
> e088 sd a0,0(s1)
> mntget(path->mnt);
> // Need "lw a0,0(s1)" or "a0 << 32; a0 >> 32"
> 2a6150ef jal c01ce946 <mntget> // bug!
>
> Acknowledge
> ===========
> The s64ilp32 needs many other projects' cooperation. Thx, all guys
> involved:
> - GNU: LiaoShihua <shihua@iscas.ac.cn>,
> Jiawe Chen<jiawei@iscas.ac.cn>
> - Qemu: Weiwei Li <liweiwei@iscas.ac.cn>
> - LLVM: luxufan <luxufan@iscas.ac.cn>,
> Chunyu Liao<chunyu@iscas.ac.cn>
> - OpenEuler rv32: Junqiang Wang <wangjunqiang@iscas.ac.cn>
> - Debian rv32: Bo YU <tsu.yubo@gmail.com>
> Han Gao <gaohan@iscas.ac.cn>
> - Fedora rv32: Wei Fu <wefu@redhat.com>
>
> References
> ==========
> [1] https://techpubs.jurassic.nl/manuals/0630/developer/Mpro_n32_ABI/sgi_html/index.html
> [2] https://wiki.debian.org/Arm64ilp32Port
> [3] https://lwn.net/Articles/456731/
> [4] https://github.com/riscv/riscv-profiles/releases
> [5] https://www.cnx-software.com/2021/10/25/allwinner-d1s-f133-risc-v-processor-64mb-ddr2/
> [6] https://milkv.io/duo/
> [7] https://twitter.com/tphuang/status/1631308330256801793
> [8] https://www.cnx-software.com/2022/12/02/pine64-ox64-sbc-bl808-risc-v-multi-protocol-wisoc-64mb-ram/
>
> Guo Ren (22):
> riscv: vdso: Unify vdso32 & compat_vdso into vdso/Makefile
> riscv: vdso: Remove compat_vdso/
> riscv: vdso: Add time-related vDSO common flow for vdso32
> clocksource: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT
> riscv: s64ilp32: Introduce xlen_t
> irqchip: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT
> riscv: s64ilp32: Add sbi support
> riscv: s64ilp32: Add asid support
> riscv: s64ilp32: Introduce PTR_L and PTR_S
> riscv: s64ilp32: Enable user space runtime environment
> riscv: s64ilp32: Add ebpf jit support
> riscv: s64ilp32: Add ELF32 support
> riscv: s64ilp32: Add ARCH RV64 ILP32 compiling framework
> riscv: s64ilp32: Add MMU_SV39 mode support for 32BIT
> riscv: s64ilp32: Enable native atomic64
> riscv: s64ilp32: Add TImode (128 int) support
> riscv: s64ilp32: Implement cmpxchg_double
> riscv: s64ilp32: Disable KVM
> riscv: Cleanup rv32_defconfig
> riscv: s64ilp32: Add rv64ilp32_defconfig
> riscv: s64ilp32: Correct the rv64ilp32 stackframe layout
> riscv: s64ilp32: Temporary workaround solution to gcc problem
>
> arch/riscv/Kconfig | 36 +++-
> arch/riscv/Makefile | 24 ++-
> arch/riscv/configs/32-bit.config | 2 -
> arch/riscv/configs/64ilp32.config | 2 +
> arch/riscv/include/asm/asm.h | 5 +
> arch/riscv/include/asm/atomic.h | 6 +
> arch/riscv/include/asm/cmpxchg.h | 53 ++++++
> arch/riscv/include/asm/cpu_ops_sbi.h | 4 +-
> arch/riscv/include/asm/csr.h | 58 +++---
> arch/riscv/include/asm/extable.h | 2 +-
> arch/riscv/include/asm/page.h | 24 ++-
> arch/riscv/include/asm/pgtable-64.h | 42 ++---
> arch/riscv/include/asm/pgtable.h | 26 ++-
> arch/riscv/include/asm/processor.h | 8 +-
> arch/riscv/include/asm/ptrace.h | 96 +++++-----
> arch/riscv/include/asm/sbi.h | 24 +--
> arch/riscv/include/asm/stacktrace.h | 6 +
> arch/riscv/include/asm/timex.h | 10 +-
> arch/riscv/include/asm/vdso.h | 34 +++-
> arch/riscv/include/asm/vdso/gettimeofday.h | 84 +++++++++
> arch/riscv/include/uapi/asm/elf.h | 2 +-
> arch/riscv/include/uapi/asm/unistd.h | 1 +
> arch/riscv/kernel/Makefile | 3 +-
> arch/riscv/kernel/compat_signal.c | 2 +-
> arch/riscv/kernel/compat_vdso/.gitignore | 2 -
> arch/riscv/kernel/compat_vdso/compat_vdso.S | 8 -
> .../kernel/compat_vdso/compat_vdso.lds.S | 3 -
> arch/riscv/kernel/compat_vdso/flush_icache.S | 3 -
> arch/riscv/kernel/compat_vdso/getcpu.S | 3 -
> arch/riscv/kernel/compat_vdso/note.S | 3 -
> arch/riscv/kernel/compat_vdso/rt_sigreturn.S | 3 -
> arch/riscv/kernel/cpu.c | 4 +-
> arch/riscv/kernel/cpu_ops_sbi.c | 4 +-
> arch/riscv/kernel/cpufeature.c | 4 +-
> arch/riscv/kernel/entry.S | 24 +--
> arch/riscv/kernel/head.S | 8 +-
> arch/riscv/kernel/process.c | 8 +-
> arch/riscv/kernel/sbi.c | 24 +--
> arch/riscv/kernel/signal.c | 6 +-
> arch/riscv/kernel/traps.c | 4 +-
> arch/riscv/kernel/vdso.c | 4 +-
> arch/riscv/kernel/vdso/Makefile | 176 ++++++++++++------
> ..._vdso_offsets.sh => gen_vdso32_offsets.sh} | 2 +-
> .../gen_vdso64_offsets.sh} | 2 +-
> arch/riscv/kernel/vdso/vgettimeofday.c | 39 +++-
> arch/riscv/kernel/vdso32.S | 8 +
> arch/riscv/kernel/{vdso/vdso.S => vdso64.S} | 8 +-
> arch/riscv/kvm/Kconfig | 1 +
> arch/riscv/lib/Makefile | 1 +
> arch/riscv/lib/memset.S | 4 +-
> arch/riscv/mm/context.c | 16 +-
> arch/riscv/mm/fault.c | 13 +-
> arch/riscv/mm/init.c | 29 ++-
> arch/riscv/net/Makefile | 6 +-
> arch/riscv/net/bpf_jit_comp64.c | 10 +-
> drivers/clocksource/timer-riscv.c | 2 +-
> drivers/irqchip/irq-riscv-intc.c | 4 +-
> fs/namei.c | 2 +-
> 58 files changed, 675 insertions(+), 317 deletions(-)
> create mode 100644 arch/riscv/configs/64ilp32.config
> delete mode 100644 arch/riscv/kernel/compat_vdso/.gitignore
> delete mode 100644 arch/riscv/kernel/compat_vdso/compat_vdso.S
> delete mode 100644 arch/riscv/kernel/compat_vdso/compat_vdso.lds.S
> delete mode 100644 arch/riscv/kernel/compat_vdso/flush_icache.S
> delete mode 100644 arch/riscv/kernel/compat_vdso/getcpu.S
> delete mode 100644 arch/riscv/kernel/compat_vdso/note.S
> delete mode 100644 arch/riscv/kernel/compat_vdso/rt_sigreturn.S
> rename arch/riscv/kernel/vdso/{gen_vdso_offsets.sh => gen_vdso32_offsets.sh} (78%)
> rename arch/riscv/kernel/{compat_vdso/gen_compat_vdso_offsets.sh => vdso/gen_vdso64_offsets.sh} (77%)
> create mode 100644 arch/riscv/kernel/vdso32.S
> rename arch/riscv/kernel/{vdso/vdso.S => vdso64.S} (73%)
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-18 15:38 ` [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode Palmer Dabbelt
@ 2023-05-18 18:29 ` Arnd Bergmann
2023-05-19 0:38 ` Paul Walmsley
2023-05-19 15:31 ` Guo Ren
2023-05-19 0:14 ` Paul Walmsley
1 sibling, 2 replies; 37+ messages in thread
From: Arnd Bergmann @ 2023-05-18 18:29 UTC (permalink / raw)
To: Palmer Dabbelt, guoren
Cc: Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Conor.Dooley,
Heiko Stübner, Jisheng Zhang, Huacai Chen, Anup Patel,
Atish Patra, Mark Rutland, Björn Töpel, Paul Walmsley,
Catalin Marinas, Will Deacon, Mike Rapoport, Anup Patel, shihua,
jiawei, liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, Andy Chiu, Vincent Chen, Greentime Hu,
Jonathan Corbet, wuwei2016, Jessica Clarke, Linux-Arch,
linux-kernel, linux-riscv, Guo Ren
On Thu, May 18, 2023, at 17:38, Palmer Dabbelt wrote:
> On Thu, 18 May 2023 06:09:51 PDT (-0700), guoren@kernel.org wrote:
>> From: Guo Ren <guoren@linux.alibaba.com>
>>
>> This patch series adds s64ilp32 support to riscv. The term s64ilp32
>> means smode-xlen=64 and -mabi=ilp32 (ints, longs, and pointers are all
>> 32-bit), i.e., running 32-bit Linux kernel on pure 64-bit supervisor
>> mode. There have been many 64ilp32 abis existing, such as mips-n32 [1],
>> arm-aarch64ilp32 [2], and x86-x32 [3], but they are all about userspace.
>> Thus, this should be the first time running a 32-bit Linux kernel with
>> the 64ilp32 ABI at supervisor mode (If not, correct me).
>
> Does anyone actually want this? At a bare minimum we'd need to add it
> to the psABI, which would presumably also be required on the compiler
> side of things.
>
> It's not even clear anyone wants rv64/ilp32 in userspace, the kernel
> seems like it'd be even less widely used.
We have had long discussions about supporting ilp32 userspace on
arm64, and I think almost everyone is glad we never merged it into
the mainline kernel, so we don't have to worry about supporting it
in the future. The cost of supporting an extra user space ABI
is huge, and I'm sure you don't want to go there. The other two
cited examples (mips-n32 and x86-x32) are pretty much unused now
as well, but still have a maintenance burden until they can finally
get removed.
If for some crazy reason you'd still want the 64ilp32 ABI in user
space, running the kernel this way is probably still a bad idea,
but that one is less clear. There is clearly a small memory
penalty of running a 64-bit kernel for larger data structures
(page, inode, task_struct, ...) and vmlinux, and there is no
huge additional maintenance cost on top of the ABI itself
that you'd need either way, but using a 64-bit address space
in the kernel has some important advantages even when running
32-bit userland: processes can use the entire 4GB virtual
space, while the kernel can address more than 768MB of lowmem,
and KASLR has more bits to work with for randomization. On
RISCV, some additional features (VMAP_STACK, KASAN, KFENCE,
...) depend on 64-bit kernels even though they don't
strictly need that.
Arnd
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-18 15:38 ` [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode Palmer Dabbelt
2023-05-18 18:29 ` Arnd Bergmann
@ 2023-05-19 0:14 ` Paul Walmsley
2023-05-21 12:37 ` Guo Ren
1 sibling, 1 reply; 37+ messages in thread
From: Paul Walmsley @ 2023-05-19 0:14 UTC (permalink / raw)
To: Palmer Dabbelt
Cc: guoren, Arnd Bergmann, tglx, peterz, luto, Conor Dooley, heiko,
jszhang, chenhuacai, apatel, atishp, Mark Rutland, bjorn,
Catalin Marinas, Will Deacon, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27, linux-arch, linux-kernel, linux-riscv, guoren
On Thu, 18 May 2023, Palmer Dabbelt wrote:
> On Thu, 18 May 2023 06:09:51 PDT (-0700), guoren@kernel.org wrote:
>
> > This patch series adds s64ilp32 support to riscv. The term s64ilp32
> > means smode-xlen=64 and -mabi=ilp32 (ints, longs, and pointers are all
> > 32-bit), i.e., running 32-bit Linux kernel on pure 64-bit supervisor
> > mode. There have been many 64ilp32 abis existing, such as mips-n32 [1],
> > arm-aarch64ilp32 [2], and x86-x32 [3], but they are all about userspace.
> > Thus, this should be the first time running a 32-bit Linux kernel with
> > the 64ilp32 ABI at supervisor mode (If not, correct me).
>
> Does anyone actually want this? At a bare minimum we'd need to add it to the
> psABI, which would presumably also be required on the compiler side of things.
>
> It's not even clear anyone wants rv64/ilp32 in userspace, the kernel seems
> like it'd be even less widely used.
We've certainly talked to folks who are interested in RV64 ILP32 userspace
with an LP64 kernel. The motivation is the usual one: to reduce data size
and therefore (ideally) BOM cost. I think this work, if it goes forward,
would need to go hand in hand with the RVIA psABI group.
The RV64 ILP32 kernel and ILP32 userspace approach implemented by this
patch is intriguing, but I guess for me, the question is whether it's
worth the extra hassle vs. a pure RV32 kernel & userspace.
- Paul
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-18 18:29 ` Arnd Bergmann
@ 2023-05-19 0:38 ` Paul Walmsley
2023-05-19 8:55 ` Arnd Bergmann
2023-05-19 15:31 ` Guo Ren
1 sibling, 1 reply; 37+ messages in thread
From: Paul Walmsley @ 2023-05-19 0:38 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Palmer Dabbelt, guoren, Thomas Gleixner, Peter Zijlstra,
Andy Lutomirski, Conor.Dooley, Heiko Stübner, Jisheng Zhang,
Huacai Chen, Anup Patel, Atish Patra, Mark Rutland,
Björn Töpel, Catalin Marinas, Will Deacon,
Mike Rapoport, Anup Patel, shihua, jiawei, liweiwei, luxufan,
chunyu, tsu.yubo, wefu, wangjunqiang, kito.cheng, Andy Chiu,
Vincent Chen, Greentime Hu, Jonathan Corbet, wuwei2016,
Jessica Clarke, Linux-Arch, linux-kernel, linux-riscv, Guo Ren
On Thu, 18 May 2023, Arnd Bergmann wrote:
> We have had long discussions about supporting ilp32 userspace on
> arm64, and I think almost everyone is glad we never merged it into
> the mainline kernel, so we don't have to worry about supporting it
> in the future. The cost of supporting an extra user space ABI
> is huge, and I'm sure you don't want to go there. The other two
> cited examples (mips-n32 and x86-x32) are pretty much unused now
> as well, but still have a maintenance burden until they can finally
> get removed.
There probably hasn't been much pressure to support Aarch64 ILP32 since
ARM still has hardware support for Aarch32. Will be interesting to see if
that's still the case after ARM drops Aarch32 support for future designs.
- Paul
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-19 0:38 ` Paul Walmsley
@ 2023-05-19 8:55 ` Arnd Bergmann
0 siblings, 0 replies; 37+ messages in thread
From: Arnd Bergmann @ 2023-05-19 8:55 UTC (permalink / raw)
To: Paul Walmsley
Cc: Palmer Dabbelt, guoren, Thomas Gleixner, Peter Zijlstra,
Andy Lutomirski, Conor.Dooley, Heiko Stübner, Jisheng Zhang,
Huacai Chen, Anup Patel, Atish Patra, Mark Rutland,
Björn Töpel, Catalin Marinas, Will Deacon,
Mike Rapoport, Anup Patel, shihua, jiawei, liweiwei, luxufan,
chunyu, tsu.yubo, wefu, wangjunqiang, kito.cheng, Andy Chiu,
Vincent Chen, Greentime Hu, Jonathan Corbet, wuwei2016,
Jessica Clarke, Linux-Arch, linux-kernel, linux-riscv, Guo Ren
On Fri, May 19, 2023, at 02:38, Paul Walmsley wrote:
> On Thu, 18 May 2023, Arnd Bergmann wrote:
>
>> We have had long discussions about supporting ilp32 userspace on
>> arm64, and I think almost everyone is glad we never merged it into
>> the mainline kernel, so we don't have to worry about supporting it
>> in the future. The cost of supporting an extra user space ABI
>> is huge, and I'm sure you don't want to go there. The other two
>> cited examples (mips-n32 and x86-x32) are pretty much unused now
>> as well, but still have a maintenance burden until they can finally
>> get removed.
>
> There probably hasn't been much pressure to support Aarch64 ILP32 since
> ARM still has hardware support for Aarch32. Will be interesting to see if
> that's still the case after ARM drops Aarch32 support for future designs.
I think there was a some pressure for 64ilp32 from Arm when aarch64 support
was originally added, as they always planned to drop aarch32 support
eventually, but I don't see that coming back now. I think the situation
is quite different as well:
On aarch64, there is a significant cost in supporting aarch32 userspace
because of the complexity of that particular instruction set, but at
the same time there is also a huge amount of software that is compiled
for or written to support aarch32 software, and nobody wants to
replace that. There are also a lot of existing arm32 chips with
guaranteed availability well into the 2030s, new 32-bit-only chips
based on Cortex-A7 (originally released in 2011) coming out constantly,
and even the latest low-end core (Cortex-A510 r1). It's probably
going to be several years before that core even shows up in low-memory
systems, and then decades before this stops being available in SoCs,
even in the unlikely case that no future low-end cores support
aarch32-el0 mode (it's already been announced that there are no
plans for future high-end cores with aarch32 mode, but those won't
be used in low-memory configurations anyway).
For RISC-V, I have not seen much interest in Linux userspace for
the existing rv32 mode, so you could argue that there is not much
to lose in abandoning it. On the other hand, the cost of adding
rv32 support to an rv64 core should be very small as all the
instructions are already present in some other encoding, and
developers have already spent a significant amount of work on
bringing up rv32 userspace that would all have to be done again
for a new ABI, and you'd end up splitting the already tiny
developer base for 32-bit riscv in two for the existing rv32 side
and a new rv64ilp32 side.
I suppose the answer in both cases is the same though: if a
SoC maker wants to sell a product to users with low memory,
they should pick a CPU core that implements standard 32-bit
user space support rather than making a mess of it and
expecting software to work around it.
Arnd
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-18 18:29 ` Arnd Bergmann
2023-05-19 0:38 ` Paul Walmsley
@ 2023-05-19 15:31 ` Guo Ren
2023-05-19 16:53 ` Arnd Bergmann
1 sibling, 1 reply; 37+ messages in thread
From: Guo Ren @ 2023-05-19 15:31 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Palmer Dabbelt, Thomas Gleixner, Peter Zijlstra, Andy Lutomirski,
Conor.Dooley, Heiko Stübner, Jisheng Zhang, Huacai Chen,
Anup Patel, Atish Patra, Mark Rutland, Björn Töpel,
Paul Walmsley, Catalin Marinas, Will Deacon, Mike Rapoport,
Anup Patel, shihua, jiawei, liweiwei, luxufan, chunyu, tsu.yubo,
wefu, wangjunqiang, kito.cheng, Andy Chiu, Vincent Chen,
Greentime Hu, Jonathan Corbet, wuwei2016, Jessica Clarke,
Linux-Arch, linux-kernel, linux-riscv, Guo Ren
On Fri, May 19, 2023 at 2:29 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Thu, May 18, 2023, at 17:38, Palmer Dabbelt wrote:
> > On Thu, 18 May 2023 06:09:51 PDT (-0700), guoren@kernel.org wrote:
> >> From: Guo Ren <guoren@linux.alibaba.com>
> >>
> >> This patch series adds s64ilp32 support to riscv. The term s64ilp32
> >> means smode-xlen=64 and -mabi=ilp32 (ints, longs, and pointers are all
> >> 32-bit), i.e., running 32-bit Linux kernel on pure 64-bit supervisor
> >> mode. There have been many 64ilp32 abis existing, such as mips-n32 [1],
> >> arm-aarch64ilp32 [2], and x86-x32 [3], but they are all about userspace.
> >> Thus, this should be the first time running a 32-bit Linux kernel with
> >> the 64ilp32 ABI at supervisor mode (If not, correct me).
> >
> > Does anyone actually want this? At a bare minimum we'd need to add it
> > to the psABI, which would presumably also be required on the compiler
> > side of things.
> >
> > It's not even clear anyone wants rv64/ilp32 in userspace, the kernel
> > seems like it'd be even less widely used.
>
> We have had long discussions about supporting ilp32 userspace on
> arm64, and I think almost everyone is glad we never merged it into
> the mainline kernel, so we don't have to worry about supporting it
> in the future. The cost of supporting an extra user space ABI
> is huge, and I'm sure you don't want to go there. The other two
> cited examples (mips-n32 and x86-x32) are pretty much unused now
> as well, but still have a maintenance burden until they can finally
> get removed.
>
> If for some crazy reason you'd still want the 64ilp32 ABI in user
> space, running the kernel this way is probably still a bad idea,
> but that one is less clear. There is clearly a small memory
> penalty of running a 64-bit kernel for larger data structures
> (page, inode, task_struct, ...) and vmlinux, and there is no
I don't think it's a small memory penalty, our measurement is about
16% with defconfig, see "Why 32-bit Linux?" section.
This patch series doesn't add 64ilp32 userspace abi, but it seems you
also don't like to run 32-bit Linux kernel on 64-bit hardware, right?
The motivation of s64ilp32 (running 32-bit Linux kernel on 64-bit s-mode):
- The target hardware (Canaan Kendryte k230) only supports MXL=64,
SXL=64, UXL=64/32.
- The 64-bit Linux + compat 32-bit app can't satisfy the 64/128MB scenarios.
> huge additional maintenance cost on top of the ABI itself
> that you'd need either way, but using a 64-bit address space
> in the kernel has some important advantages even when running
> 32-bit userland: processes can use the entire 4GB virtual
> space, while the kernel can address more than 768MB of lowmem,
> and KASLR has more bits to work with for randomization. On
> RISCV, some additional features (VMAP_STACK, KASAN, KFENCE,
> ...) depend on 64-bit kernels even though they don't
> strictly need that.
I agree that the 64-bit linux kernel has more functionalities, but:
- What do you think about linux on a 64/128MB SoC? Could it be
affordable to VMAP_STACK, KASAN, KFENCE?
- I think 32-bit Linux & RTOS have monopolized this market (64/128MB
scenarios), right?
>
> Arnd
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-19 15:31 ` Guo Ren
@ 2023-05-19 16:53 ` Arnd Bergmann
2023-05-19 17:18 ` Palmer Dabbelt
2023-05-20 1:43 ` Guo Ren
0 siblings, 2 replies; 37+ messages in thread
From: Arnd Bergmann @ 2023-05-19 16:53 UTC (permalink / raw)
To: guoren
Cc: Palmer Dabbelt, Thomas Gleixner, Peter Zijlstra, Andy Lutomirski,
Conor.Dooley, Heiko Stübner, Jisheng Zhang, Huacai Chen,
Anup Patel, Atish Patra, Mark Rutland, Björn Töpel,
Paul Walmsley, Catalin Marinas, Will Deacon, Mike Rapoport,
Anup Patel, shihua, jiawei, liweiwei, luxufan, chunyu, tsu.yubo,
wefu, wangjunqiang, kito.cheng, Andy Chiu, Vincent Chen,
Greentime Hu, Jonathan Corbet, wuwei2016, Jessica Clarke,
Linux-Arch, linux-kernel, linux-riscv, Guo Ren
On Fri, May 19, 2023, at 17:31, Guo Ren wrote:
> On Fri, May 19, 2023 at 2:29 AM Arnd Bergmann <arnd@arndb.de> wrote:
>> On Thu, May 18, 2023, at 17:38, Palmer Dabbelt wrote:
>> > On Thu, 18 May 2023 06:09:51 PDT (-0700), guoren@kernel.org wrote:
>>
>> If for some crazy reason you'd still want the 64ilp32 ABI in user
>> space, running the kernel this way is probably still a bad idea,
>> but that one is less clear. There is clearly a small memory
>> penalty of running a 64-bit kernel for larger data structures
>> (page, inode, task_struct, ...) and vmlinux, and there is no
> I don't think it's a small memory penalty, our measurement is about
> 16% with defconfig, see "Why 32-bit Linux?" section.
>
> This patch series doesn't add 64ilp32 userspace abi, but it seems you
> also don't like to run 32-bit Linux kernel on 64-bit hardware, right?
Ok, I'm sorry for missing the important bit here. So if this can
still use the normal 32-bit user space, the cost of this patch set
is not huge, and it's something that can be beneficial in a few
cases, though I suspect most users are still better off running
64-bit kernels.
> The motivation of s64ilp32 (running 32-bit Linux kernel on 64-bit s-mode):
> - The target hardware (Canaan Kendryte k230) only supports MXL=64,
> SXL=64, UXL=64/32.
> - The 64-bit Linux + compat 32-bit app can't satisfy the 64/128MB scenarios.
>
>> huge additional maintenance cost on top of the ABI itself
>> that you'd need either way, but using a 64-bit address space
>> in the kernel has some important advantages even when running
>> 32-bit userland: processes can use the entire 4GB virtual
>> space, while the kernel can address more than 768MB of lowmem,
>> and KASLR has more bits to work with for randomization. On
>> RISCV, some additional features (VMAP_STACK, KASAN, KFENCE,
>> ...) depend on 64-bit kernels even though they don't
>> strictly need that.
>
> I agree that the 64-bit linux kernel has more functionalities, but:
> - What do you think about linux on a 64/128MB SoC? Could it be
> affordable to VMAP_STACK, KASAN, KFENCE?
I would definitely recommend VMAP_STACK, but that can be implemented
and is used on other 32-bit architectures (ppc32, arm32) without a
huge cost. The larger virtual user address space can help even on
machines with 128MB, though most applications probably don't care at
that point.
> - I think 32-bit Linux & RTOS have monopolized this market (64/128MB
> scenarios), right?
The minimum amount of RAM that makes a system usable for Linux is
constantly going up, so I think with 64MB, most new projects are
already better off running some RTOS kernel instead of Linux.
The ones that are still usable today probably won't last a lot
of distro upgrades before the bloat catches up with them, but I
can see how your patch set can give them a few extra years of
updates.
For the 256MB+ systems, I would expect the sensitive kernel
allocations to be small enough that the series makes little
difference. The 128MB systems are the most interesting ones
here, and I'm curious to see where you spot most of the
memory usage differences, I'll also reply to your initial
mail for that.
Arnd
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-19 16:53 ` Arnd Bergmann
@ 2023-05-19 17:18 ` Palmer Dabbelt
2023-05-20 1:43 ` Guo Ren
1 sibling, 0 replies; 37+ messages in thread
From: Palmer Dabbelt @ 2023-05-19 17:18 UTC (permalink / raw)
To: Arnd Bergmann
Cc: guoren, tglx, peterz, luto, Conor Dooley, heiko, jszhang,
chenhuacai, apatel, atishp, Mark Rutland, bjorn, Paul Walmsley,
Catalin Marinas, Will Deacon, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27, linux-arch, linux-kernel, linux-riscv, guoren
On Fri, 19 May 2023 09:53:35 PDT (-0700), Arnd Bergmann wrote:
> On Fri, May 19, 2023, at 17:31, Guo Ren wrote:
>> On Fri, May 19, 2023 at 2:29 AM Arnd Bergmann <arnd@arndb.de> wrote:
>>> On Thu, May 18, 2023, at 17:38, Palmer Dabbelt wrote:
>>> > On Thu, 18 May 2023 06:09:51 PDT (-0700), guoren@kernel.org wrote:
>>>
>>> If for some crazy reason you'd still want the 64ilp32 ABI in user
>>> space, running the kernel this way is probably still a bad idea,
>>> but that one is less clear. There is clearly a small memory
>>> penalty of running a 64-bit kernel for larger data structures
>>> (page, inode, task_struct, ...) and vmlinux, and there is no
>> I don't think it's a small memory penalty, our measurement is about
>> 16% with defconfig, see "Why 32-bit Linux?" section.
>>
>> This patch series doesn't add 64ilp32 userspace abi, but it seems you
>> also don't like to run 32-bit Linux kernel on 64-bit hardware, right?
>
> Ok, I'm sorry for missing the important bit here. So if this can
> still use the normal 32-bit user space, the cost of this patch set
> is not huge, and it's something that can be beneficial in a few
> cases, though I suspect most users are still better off running
> 64-bit kernels.
Running a normal 32-bit userspace would require HW support for the
32-bit mode switch for userspace, though (rv32 isn't a subset of rv64,
so there's nothing we can do to make those binaries function correctly
with uABI). The userspace-only mode switch is a bit simpler than the
user+supervisor switch, but it seems like vendors who really want the
memory savings would just implement both mode switches.
>> The motivation of s64ilp32 (running 32-bit Linux kernel on 64-bit s-mode):
>> - The target hardware (Canaan Kendryte k230) only supports MXL=64,
>> SXL=64, UXL=64/32.
>> - The 64-bit Linux + compat 32-bit app can't satisfy the 64/128MB scenarios.
>>
>>> huge additional maintenance cost on top of the ABI itself
>>> that you'd need either way, but using a 64-bit address space
>>> in the kernel has some important advantages even when running
>>> 32-bit userland: processes can use the entire 4GB virtual
>>> space, while the kernel can address more than 768MB of lowmem,
>>> and KASLR has more bits to work with for randomization. On
>>> RISCV, some additional features (VMAP_STACK, KASAN, KFENCE,
>>> ...) depend on 64-bit kernels even though they don't
>>> strictly need that.
>>
>> I agree that the 64-bit linux kernel has more functionalities, but:
>> - What do you think about linux on a 64/128MB SoC? Could it be
>> affordable to VMAP_STACK, KASAN, KFENCE?
>
> I would definitely recommend VMAP_STACK, but that can be implemented
> and is used on other 32-bit architectures (ppc32, arm32) without a
> huge cost. The larger virtual user address space can help even on
> machines with 128MB, though most applications probably don't care at
> that point.
At least having them as an option seems reasonable. Historically we
haven't gated new base systems on having every feature the others do,
though (!MMU, rv32, etc).
>> - I think 32-bit Linux & RTOS have monopolized this market (64/128MB
>> scenarios), right?
>
> The minimum amount of RAM that makes a system usable for Linux is
> constantly going up, so I think with 64MB, most new projects are
> already better off running some RTOS kernel instead of Linux.
> The ones that are still usable today probably won't last a lot
> of distro upgrades before the bloat catches up with them, but I
> can see how your patch set can give them a few extra years of
> updates.
We also have 32-bit kernel support. Systems that have tens of MB of RAM
tend to end up with some memory technology that doesn't scale to
gigabytes these days, and since that's fixed when the chip is built it
seems like those folks would be better off just having HW support for
32-bit kernels (and maybe not even bothering with HW support for 64-bit
kernels).
> For the 256MB+ systems, I would expect the sensitive kernel
> allocations to be small enough that the series makes little
> difference. The 128MB systems are the most interesting ones
> here, and I'm curious to see where you spot most of the
> memory usage differences, I'll also reply to your initial
> mail for that.
Thanks. I agree we need to see some real systems that benefit from
this, as it's a pretty big support cost. Just defconfig sizes doesn't
mean a whole lot, as users on these very constrained systems aren't
likely to run defconfig anyway.
If someone's going to use it then I'm fine taking the code, it just
seems like a very thin set of possible use cases. We've already got
almost no users in RISC-V land, I've got a feeling this is esoteric
enough to actually have zero.
>
> Arnd
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
` (22 preceding siblings ...)
2023-05-18 15:38 ` [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode Palmer Dabbelt
@ 2023-05-19 20:20 ` Arnd Bergmann
2023-05-20 2:53 ` Guo Ren
23 siblings, 1 reply; 37+ messages in thread
From: Arnd Bergmann @ 2023-05-19 20:20 UTC (permalink / raw)
To: guoren, Palmer Dabbelt, Thomas Gleixner, Peter Zijlstra,
Andy Lutomirski, Conor.Dooley, Heiko Stübner, Jisheng Zhang,
Huacai Chen, Anup Patel, Atish Patra, Mark Rutland,
Björn Töpel, Paul Walmsley, Catalin Marinas,
Will Deacon, Mike Rapoport, Anup Patel, shihua, jiawei, liweiwei,
luxufan, chunyu, tsu.yubo, wefu, wangjunqiang, kito.cheng,
Andy Chiu, Vincent Chen, Greentime Hu, Jonathan Corbet,
wuwei2016, Jessica Clarke
Cc: Linux-Arch, linux-kernel, linux-riscv, Guo Ren
On Thu, May 18, 2023, at 15:09, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
> Why 32-bit Linux?
> =================
> The motivation for using a 32-bit Linux kernel is to reduce memory
> footprint and meet the small capacity of DDR & cache requirement
> (e.g., 64/128MB SIP SoC).
>
> Here are the 32-bit v.s. 64-bit Linux kernel data type comparison
> summary:
> 32-bit 64-bit
> sizeof(page): 32bytes 64bytes
> sizeof(list_head): 8bytes 16bytes
> sizeof(hlist_head): 8bytes 16bytes
> sizeof(vm_area): 68bytes 136bytes
> ...
> Mem-usage:
> (s32ilp32) # free
> total used free shared buff/cache available
> Mem: 100040 8380 88244 44 3416 88080
>
> (s64lp64) # free
> total used free shared buff/cache available
> Mem: 91568 11848 75796 44 3924 75952
>
> (s64ilp32) # free
> total used free shared buff/cache available
> Mem: 101952 8528 90004 44 3420 89816
> ^^^^^
>
> It's a rough measurement based on the current default config without any
> modification, and 32-bit (s32ilp32, s64ilp32) saved more than 16% memory
> to 64-bit (s64lp64). But s32ilp32 & s64ilp32 have a similar memory
> footprint (about 0.33% difference), meaning s64ilp32 has a big chance to
> replace s32ilp32 on the 64-bit machine.
I've tried to run the same numbers for the debate about running
32-bit vs 64-bit arm kernels in the past, but focused mostly on
slightly larger systems, but I looked mainly at the 512MB case,
as that is the most cost-efficient DDR3 memory configuration
and fairly common.
What I'd like to understand better in your example is where
the 14MB of memory went. I assume this is for 128MB of total
RAM, so we know that 1MB went into additional 'struct page'
objects (32 bytes * 32768 pages). It would be good to know
where the dynamic allocations went and if they are reclaimable
(e.g. inodes) or non-reclaimable (e.g. kmalloc-128).
For the vmlinux size, is this already a minimal config
that one would run on a board with 128MB of RAM, or a
defconfig that includes a lot of stuff that is only relevant
for other platforms but also grows on 64-bit?
What do you see in /proc/slabinfo, /proc/meminfo/, and
'size vmlinux' for the s64ilp32 and s64lp64 kernels here?
Arnd
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-19 16:53 ` Arnd Bergmann
2023-05-19 17:18 ` Palmer Dabbelt
@ 2023-05-20 1:43 ` Guo Ren
1 sibling, 0 replies; 37+ messages in thread
From: Guo Ren @ 2023-05-20 1:43 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Palmer Dabbelt, Thomas Gleixner, Peter Zijlstra, Andy Lutomirski,
Conor.Dooley, Heiko Stübner, Jisheng Zhang, Huacai Chen,
Anup Patel, Atish Patra, Mark Rutland, Björn Töpel,
Paul Walmsley, Catalin Marinas, Will Deacon, Mike Rapoport,
Anup Patel, shihua, jiawei, liweiwei, luxufan, chunyu, tsu.yubo,
wefu, wangjunqiang, kito.cheng, Andy Chiu, Vincent Chen,
Greentime Hu, Jonathan Corbet, wuwei2016, Jessica Clarke,
Linux-Arch, linux-kernel, linux-riscv, Guo Ren
On Sat, May 20, 2023 at 12:54 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Fri, May 19, 2023, at 17:31, Guo Ren wrote:
> > On Fri, May 19, 2023 at 2:29 AM Arnd Bergmann <arnd@arndb.de> wrote:
> >> On Thu, May 18, 2023, at 17:38, Palmer Dabbelt wrote:
> >> > On Thu, 18 May 2023 06:09:51 PDT (-0700), guoren@kernel.org wrote:
> >>
> >> If for some crazy reason you'd still want the 64ilp32 ABI in user
> >> space, running the kernel this way is probably still a bad idea,
> >> but that one is less clear. There is clearly a small memory
> >> penalty of running a 64-bit kernel for larger data structures
> >> (page, inode, task_struct, ...) and vmlinux, and there is no
> > I don't think it's a small memory penalty, our measurement is about
> > 16% with defconfig, see "Why 32-bit Linux?" section.
> >
> > This patch series doesn't add 64ilp32 userspace abi, but it seems you
> > also don't like to run 32-bit Linux kernel on 64-bit hardware, right?
>
> Ok, I'm sorry for missing the important bit here. So if this can
> still use the normal 32-bit user space, the cost of this patch set
> is not huge, and it's something that can be beneficial in a few
> cases, though I suspect most users are still better off running
> 64-bit kernels.
>
> > The motivation of s64ilp32 (running 32-bit Linux kernel on 64-bit s-mode):
> > - The target hardware (Canaan Kendryte k230) only supports MXL=64,
> > SXL=64, UXL=64/32.
> > - The 64-bit Linux + compat 32-bit app can't satisfy the 64/128MB scenarios.
> >
> >> huge additional maintenance cost on top of the ABI itself
> >> that you'd need either way, but using a 64-bit address space
> >> in the kernel has some important advantages even when running
> >> 32-bit userland: processes can use the entire 4GB virtual
> >> space, while the kernel can address more than 768MB of lowmem,
> >> and KASLR has more bits to work with for randomization. On
> >> RISCV, some additional features (VMAP_STACK, KASAN, KFENCE,
> >> ...) depend on 64-bit kernels even though they don't
> >> strictly need that.
> >
> > I agree that the 64-bit linux kernel has more functionalities, but:
> > - What do you think about linux on a 64/128MB SoC? Could it be
> > affordable to VMAP_STACK, KASAN, KFENCE?
>
> I would definitely recommend VMAP_STACK, but that can be implemented
> and is used on other 32-bit architectures (ppc32, arm32) without a
> huge cost. The larger virtual user address space can help even on
> machines with 128MB, though most applications probably don't care at
> that point.
Good point, I would support VMAP_STACK in ARCH_RV64ILP32.
>
> > - I think 32-bit Linux & RTOS have monopolized this market (64/128MB
> > scenarios), right?
>
> The minimum amount of RAM that makes a system usable for Linux is
> constantly going up, so I think with 64MB, most new projects are
> already better off running some RTOS kernel instead of Linux.
> The ones that are still usable today probably won't last a lot
> of distro upgrades before the bloat catches up with them, but I
> can see how your patch set can give them a few extra years of
> updates.
Linux development costs much cheaper than RTOS, so the vendors would
first develop a Linux version. If it succeeds in the market, the
vendors will create a cost-down solution. So their first choice is to
cut down the memory footprint of the first Linux version instead of
moving to RTOS.
With the price of 128MB-DDR3 & 64MB-DDR2 being more and more similar,
32bit-Linux has more opportunities to instead of RTOS.
>
> For the 256MB+ systems, I would expect the sensitive kernel
> allocations to be small enough that the series makes little
> difference. The 128MB systems are the most interesting ones
> here, and I'm curious to see where you spot most of the
> memory usage differences, I'll also reply to your initial
> mail for that.
Thx, I aslo recommand you read about "Why s64ilp32 has better
performance?" section :)
How do you think running arm32-Linux on coretex-A35/A53/A55?
>
> Arnd
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-19 20:20 ` Arnd Bergmann
@ 2023-05-20 2:53 ` Guo Ren
2023-05-20 10:13 ` Arnd Bergmann
0 siblings, 1 reply; 37+ messages in thread
From: Guo Ren @ 2023-05-20 2:53 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Palmer Dabbelt, Thomas Gleixner, Peter Zijlstra, Andy Lutomirski,
Conor.Dooley, Heiko Stübner, Jisheng Zhang, Huacai Chen,
Anup Patel, Atish Patra, Mark Rutland, Björn Töpel,
Paul Walmsley, Catalin Marinas, Will Deacon, Mike Rapoport,
Anup Patel, shihua, jiawei, liweiwei, luxufan, chunyu, tsu.yubo,
wefu, wangjunqiang, kito.cheng, Andy Chiu, Vincent Chen,
Greentime Hu, Jonathan Corbet, wuwei2016, Jessica Clarke,
Linux-Arch, linux-kernel, linux-riscv, Guo Ren
On Sat, May 20, 2023 at 4:20 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Thu, May 18, 2023, at 15:09, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> > Why 32-bit Linux?
> > =================
> > The motivation for using a 32-bit Linux kernel is to reduce memory
> > footprint and meet the small capacity of DDR & cache requirement
> > (e.g., 64/128MB SIP SoC).
> >
> > Here are the 32-bit v.s. 64-bit Linux kernel data type comparison
> > summary:
> > 32-bit 64-bit
> > sizeof(page): 32bytes 64bytes
> > sizeof(list_head): 8bytes 16bytes
> > sizeof(hlist_head): 8bytes 16bytes
> > sizeof(vm_area): 68bytes 136bytes
> > ...
>
> > Mem-usage:
> > (s32ilp32) # free
> > total used free shared buff/cache available
> > Mem: 100040 8380 88244 44 3416 88080
> >
> > (s64lp64) # free
> > total used free shared buff/cache available
> > Mem: 91568 11848 75796 44 3924 75952
> >
> > (s64ilp32) # free
> > total used free shared buff/cache available
> > Mem: 101952 8528 90004 44 3420 89816
> > ^^^^^
> >
> > It's a rough measurement based on the current default config without any
> > modification, and 32-bit (s32ilp32, s64ilp32) saved more than 16% memory
> > to 64-bit (s64lp64). But s32ilp32 & s64ilp32 have a similar memory
> > footprint (about 0.33% difference), meaning s64ilp32 has a big chance to
> > replace s32ilp32 on the 64-bit machine.
>
> I've tried to run the same numbers for the debate about running
> 32-bit vs 64-bit arm kernels in the past, but focused mostly on
> slightly larger systems, but I looked mainly at the 512MB case,
> as that is the most cost-efficient DDR3 memory configuration
> and fairly common.
512MB is extravagant, in my opinion. In the IPC market, 32/64MB is for
480P/720P/1080p, 128/256MB is for 1080p/2k, and 512/1024MB is for 4K.
> 512MB chips is less than 5% of the total (I guess). Even in 512MB
chips, the additional memory is for the frame buffer, not the Linux
system.
I agree for the > 512MB scenarios would make it less sensitive on a
32/64-bit Linux kernel.
>
> What I'd like to understand better in your example is where
> the 14MB of memory went. I assume this is for 128MB of total
> RAM, so we know that 1MB went into additional 'struct page'
> objects (32 bytes * 32768 pages). It would be good to know
> where the dynamic allocations went and if they are reclaimable
> (e.g. inodes) or non-reclaimable (e.g. kmalloc-128).
>
> For the vmlinux size, is this already a minimal config
> that one would run on a board with 128MB of RAM, or a
> defconfig that includes a lot of stuff that is only relevant
> for other platforms but also grows on 64-bit?
It's not minimal config, it's defconfig. So I say it's a roungh measurement :)
I admit I wanted a little bit to exaggerate it, but that's the
starting point for cutting down memory usage for most people, right?
During the past year, we have been convincing our customers to use the
s64lp64 + u32ilp32, but they can't tolerate even 1% memory additional
cost in 64MB/128MB scenarios and then chose cortex-a7/a35, which could
run 32-bit Linux. I think it's too early to talk about throwing 32-bit
Linux into the garbage, not only for the reason of memory footprint
but also for the ingrained opinion of the people. Changing their mind
needs a long time.
>
> What do you see in /proc/slabinfo, /proc/meminfo/, and
> 'size vmlinux' for the s64ilp32 and s64lp64 kernels here?
Both s64ilp32 & s64lp64 use the same u32ilp32_rootfs.ext2 binary and
the same opensbi binary.
All are opensbi(2MB) + Linux(126MB) memory layout.
Here is the result:
s64ilp32:
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] fixmap : 0x9ce00000 - 0x9d000000 (2048 kB)
[ 0.000000] pci io : 0x9d000000 - 0x9e000000 ( 16 MB)
[ 0.000000] vmemmap : 0x9e000000 - 0xa0000000 ( 32 MB)
[ 0.000000] vmalloc : 0xa0000000 - 0xc0000000 ( 512 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xc7e00000 ( 126 MB)
[ 0.000000] Memory: 97748K/129024K available (8699K kernel code,
8867K rwdata, 4096K rodata, 4204K init, 361K bss, 31276K reserved, 0K
cma-reserved)
...
# free
total used free shared buff/cache available
Mem: 101952 8516 90016 44 3420 89828
Swap: 0 0 0
# cat /proc/meminfo
MemTotal: 101952 kB
MemFree: 90016 kB
MemAvailable: 89836 kB
Buffers: 292 kB
Cached: 2484 kB
SwapCached: 0 kB
Active: 2556 kB
Inactive: 656 kB
Active(anon): 40 kB
Inactive(anon): 440 kB
Active(file): 2516 kB
Inactive(file): 216 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 32 kB
Writeback: 0 kB
AnonPages: 480 kB
Mapped: 1804 kB
Shmem: 44 kB
KReclaimable: 644 kB
Slab: 4536 kB
SReclaimable: 644 kB
SUnreclaim: 3892 kB
KernelStack: 344 kB
PageTables: 112 kB
SecPageTables: 0 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 50976 kB
Committed_AS: 2040 kB
VmallocTotal: 524288 kB
VmallocUsed: 112 kB
VmallocChunk: 0 kB
Percpu: 64 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
Hugetlb: 0 kB
# cat /proc/slabinfo
[68/1691]
slabinfo - version: 2.1
# name <active_objs> <num_objs> <objsize> <objperslab>
<pagesperslab> : tunables <limit> <batchcount> <sharedfactor> :
slabdata <active_slabs> <num_slabs> <sharedavail>
ext4_groupinfo_1k 28 28 144 28 1 : tunables 0 0
0 : slabdata 1 1 0
p9_req_t 0 0 104 39 1 : tunables 0 0
0 : slabdata 0 0 0
UDPv6 0 0 1088 15 4 : tunables 0 0
0 : slabdata 0 0 0
tw_sock_TCPv6 0 0 200 20 1 : tunables 0 0
0 : slabdata 0 0 0
request_sock_TCPv6 0 0 240 17 1 : tunables 0 0
0 : slabdata 0 0 0
TCPv6 0 0 2048 8 4 : tunables 0 0
0 : slabdata 0 0 0
bio-72 32 32 128 32 1 : tunables 0 0
0 : slabdata 1 1 0
bfq_io_cq 0 0 1000 8 2 : tunables 0 0
0 : slabdata 0 0 0
bio-184 21 21 192 21 1 : tunables 0 0
0 : slabdata 1 1 0
mqueue_inode_cache 10 10 768 10 2 : tunables 0 0
0 : slabdata 1 1 0
v9fs_inode_cache 0 0 576 14 2 : tunables 0 0
0 : slabdata 0 0 0
nfs4_xattr_cache_cache 0 0 1848 17 8 : tunables 0
0 0 : slabdata 0 0 0
nfs_direct_cache 0 0 152 26 1 : tunables 0 0
0 : slabdata 0 0 0
nfs_read_data 36 36 640 12 2 : tunables 0 0
0 : slabdata 3 3 0
nfs_inode_cache 0 0 832 19 4 : tunables 0 0
0 : slabdata 0 0 0
isofs_inode_cache 0 0 528 15 2 : tunables 0 0
0 : slabdata 0 0 0
fat_inode_cache 0 0 632 25 4 : tunables 0 0
0 : slabdata 0 0 0
fat_cache 0 0 24 170 1 : tunables 0 0
0 : slabdata 0 0 0
jbd2_journal_handle 0 0 48 85 1 : tunables 0
0 0 : slabdata 0 0 0
jbd2_journal_head 0 0 80 51 1 : tunables 0 0
0 : slabdata 0 0 0
ext4_fc_dentry_update 0 0 88 46 1 : tunables 0
0 0 : slabdata 0 0 0
ext4_inode_cache 88 88 984 8 2 : tunables 0 0
0 : slabdata 11 11 0
ext4_allocation_context 36 36 112 36 1 : tunables 0
0 0 : slabdata 1 1 0
ext4_io_end_vec 0 0 24 170 1 : tunables 0 0
0 : slabdata 0 0 0
pending_reservation 0 0 16 256 1 : tunables 0
0 0 : slabdata 0 0 0
extent_status 256 256 32 128 1 : tunables 0 0
0 : slabdata 2 2 0
mbcache 102 102 40 102 1 : tunables 0 0
0 : slabdata 1 1 0
dio 0 0 384 10 1 : tunables 0 0
0 : slabdata 0 0 0
audit_tree_mark 0 0 64 64 1 : tunables 0 0
0 : slabdata 0 0 0
rpc_inode_cache 0 0 576 14 2 : tunables 0 0
0 : slabdata 0 0 0
ip4-frags 0 0 152 26 1 : tunables 0 0
0 : slabdata 0 0 0
RAW 9 9 896 9 2 : tunables 0 0
0 : slabdata 1 1 0
UDP 8 8 960 8 2 : tunables 0 0
0 : slabdata 1 1 0
tw_sock_TCP 0 0 200 20 1 : tunables 0 0
0 : slabdata 0 0 0
request_sock_TCP 0 0 240 17 1 : tunables 0 0
0 : slabdata 0 0 0
TCP 0 0 1920 8 4 : tunables 0 0
0 : slabdata 0 0 0
hugetlbfs_inode_cache 8 8 504 8 1 : tunables 0
0 0 : slabdata 1 1 0
bio-164 42 42 192 21 1 : tunables 0 0
0 : slabdata 2 2 0
ep_head 0 0 8 512 1 : tunables 0 0
0 : slabdata 0 0 0
dax_cache 14 14 576 14 2 : tunables 0 0
0 : slabdata 1 1 0
sgpool-128 16 16 2048 8 4 : tunables 0 0
0 : slabdata 2 2 0
sgpool-64 8 8 1024 8 2 : tunables 0 0
0 : slabdata 1 1 0
request_queue 13 13 616 13 2 : tunables 0 0
0 : slabdata 1 1 0
blkdev_ioc 0 0 80 51 1 : tunables 0 0
0 : slabdata 0 0 0
bio-120 64 64 128 32 1 : tunables 0 0
0 : slabdata 2 2 0
biovec-max 40 40 3072 10 8 : tunables 0 0
0 : slabdata 4 4 0
biovec-128 0 0 1536 10 4 : tunables 0 0
0 : slabdata 0 0 0
[19/1691]
biovec-64 10 10 768 10 2 : tunables 0 0
0 : slabdata 1 1 0
dmaengine-unmap-2 128 128 32 128 1 : tunables 0 0
0 : slabdata 1 1 0
sock_inode_cache 22 22 704 11 2 : tunables 0 0
0 : slabdata 2 2 0
skbuff_small_head 14 14 576 14 2 : tunables 0 0
0 : slabdata 1 1 0
skbuff_fclone_cache 0 0 448 9 1 : tunables 0
0 0 : slabdata 0 0 0
file_lock_cache 28 28 144 28 1 : tunables 0 0
0 : slabdata 1 1 0
buffer_head 357 357 80 51 1 : tunables 0 0
0 : slabdata 7 7 0
proc_dir_entry 256 256 128 32 1 : tunables 0 0
0 : slabdata 8 8 0
pde_opener 0 0 24 170 1 : tunables 0 0
0 : slabdata 0 0 0
proc_inode_cache 60 60 536 15 2 : tunables 0 0
0 : slabdata 4 4 0
seq_file 42 42 96 42 1 : tunables 0 0
0 : slabdata 1 1 0
sigqueue 85 85 48 85 1 : tunables 0 0
0 : slabdata 1 1 0
bdev_cache 14 14 1152 14 4 : tunables 0 0
0 : slabdata 1 1 0
shmem_inode_cache 637 637 600 13 2 : tunables 0 0
0 : slabdata 49 49 0
kernfs_node_cache 13938 13938 88 46 1 : tunables 0 0
0 : slabdata 303 303 0
inode_cache 360 360 496 8 1 : tunables 0 0
0 : slabdata 45 45 0
dentry 1196 1196 152 26 1 : tunables 0 0
0 : slabdata 46 46 0
names_cache 8 8 4096 8 8 : tunables 0 0
0 : slabdata 1 1 0
net_namespace 0 0 2944 11 8 : tunables 0 0
0 : slabdata 0 0 0
iint_cache 0 0 96 42 1 : tunables 0 0
0 : slabdata 0 0 0
key_jar 105 105 192 21 1 : tunables 0 0
0 : slabdata 5 5 0
uts_namespace 0 0 416 19 2 : tunables 0 0
0 : slabdata 0 0 0
nsproxy 102 102 40 102 1 : tunables 0 0
0 : slabdata 1 1 0
vm_area_struct 255 255 80 51 1 : tunables 0 0
0 : slabdata 5 5 0
signal_cache 55 55 704 11 2 : tunables 0 0
0 : slabdata 5 5 0
sighand_cache 60 60 1088 15 4 : tunables 0 0
0 : slabdata 4 4 0
anon_vma_chain 384 384 32 128 1 : tunables 0 0
0 : slabdata 3 3 0
anon_vma 168 168 72 56 1 : tunables 0 0
0 : slabdata 3 3 0
perf_event 0 0 816 10 2 : tunables 0 0
0 : slabdata 0 0 0
maple_node 32 32 256 16 1 : tunables 0 0
0 : slabdata 2 2 0
radix_tree_node 338 338 304 13 1 : tunables 0 0
0 : slabdata 26 26 0
task_group 8 8 512 8 1 : tunables 0 0
0 : slabdata 1 1 0
mm_struct 20 20 768 10 2 : tunables 0 0
0 : slabdata 2 2 0
vmap_area 102 102 40 102 1 : tunables 0 0
0 : slabdata 1 1 0
page->ptl 256 256 16 256 1 : tunables 0 0
0 : slabdata 1 1 0
kmalloc-cg-8k 0 0 8192 4 8 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-cg-4k 8 8 4096 8 8 : tunables 0 0
0 : slabdata 1 1 0
kmalloc-cg-2k 72 72 2048 8 4 : tunables 0 0
0 : slabdata 9 9 0
kmalloc-cg-1k 32 32 1024 8 2 : tunables 0 0
0 : slabdata 4 4 0
kmalloc-cg-512 32 32 512 8 1 : tunables 0 0
0 : slabdata 4 4 0
kmalloc-cg-256 96 96 256 16 1 : tunables 0 0
0 : slabdata 6 6 0
kmalloc-cg-192 63 63 192 21 1 : tunables 0 0
0 : slabdata 3 3 0
kmalloc-cg-128 160 160 128 32 1 : tunables 0 0
0 : slabdata 5 5 0
kmalloc-cg-64 128 128 64 64 1 : tunables 0 0
0 : slabdata 2 2 0
kmalloc-rcl-8k 0 0 8192 4 8 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-4k 0 0 4096 8 8 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-2k 0 0 2048 8 4 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-1k 0 0 1024 8 2 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-512 0 0 512 8 1 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-256 0 0 256 16 1 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-192 0 0 192 21 1 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-128 0 0 128 32 1 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-64 0 0 64 64 1 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-8k 12 12 8192 4 8 : tunables 0 0
0 : slabdata 3 3 0
kmalloc-4k 16 16 4096 8 8 : tunables 0 0
0 : slabdata 2 2 0
kmalloc-2k 40 40 2048 8 4 : tunables 0 0
0 : slabdata 5 5 0
kmalloc-1k 88 88 1024 8 2 : tunables 0 0
0 : slabdata 11 11 0
kmalloc-512 856 856 512 8 1 : tunables 0 0
0 : slabdata 107 107 0
kmalloc-256 64 64 256 16 1 : tunables 0 0
0 : slabdata 4 4 0
kmalloc-192 126 126 192 21 1 : tunables 0 0
0 : slabdata 6 6 0
kmalloc-128 1056 1056 128 32 1 : tunables 0 0
0 : slabdata 33 33 0
kmalloc-64 5302 5312 64 64 1 : tunables 0 0
0 : slabdata 83 83 0
kmem_cache_node 128 128 64 64 1 : tunables 0 0
0 : slabdata 2 2 0
kmem_cache 128 128 128 32 1 : tunables 0 0
0 : slabdata 4 4 0
s64lp64:
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] fixmap : 0xff1bfffffee00000 - 0xff1bffffff000000
(2048 kB)
[ 0.000000] pci io : 0xff1bffffff000000 - 0xff1c000000000000
( 16 MB)
[ 0.000000] vmemmap : 0xff1c000000000000 - 0xff20000000000000
(1024 TB)
[ 0.000000] vmalloc : 0xff20000000000000 - 0xff60000000000000
(16384 TB)
[ 0.000000] modules : 0xffffffff01579000 - 0xffffffff80000000
(2026 MB)
[ 0.000000] lowmem : 0xff60000000000000 - 0xff60000008000000
( 128 MB)
[ 0.000000] kernel : 0xffffffff80000000 - 0xffffffffffffffff
(2047 MB)
[ 0.000000] Memory: 89380K/131072K available (8638K kernel code,
4979K rwdata, 4096K rodata, 2191K init, 477K bss, 41692K reserved, 0K
cma-reserved)
...
# free
total used free shared buff/cache available
Mem: 91568 11472 76264 48 3832 76376
Swap: 0 0 0
# cat /proc/meminfo
MemTotal: 91568 kB
MemFree: 76220 kB
MemAvailable: 76352 kB
Buffers: 292 kB
Cached: 2488 kB
SwapCached: 0 kB
Active: 2560 kB
Inactive: 656 kB
Active(anon): 44 kB
Inactive(anon): 440 kB
Active(file): 2516 kB
Inactive(file): 216 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 16 kB
Writeback: 0 kB
AnonPages: 480 kB
Mapped: 1804 kB
Shmem: 48 kB
KReclaimable: 1092 kB
Slab: 6900 kB
SReclaimable: 1092 kB
SUnreclaim: 5808 kB
KernelStack: 688 kB
PageTables: 120 kB
SecPageTables: 0 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 45784 kB
Committed_AS: 2044 kB
VmallocTotal: 17592186044416 kB
VmallocUsed: 904 kB
VmallocChunk: 0 kB
Percpu: 88 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
Hugetlb: 0 kB
# cat /proc/slabinfo
slabinfo - version: 2.1
# name <active_objs> <num_objs> <objsize> <objperslab>
<pagesperslab> : tunables <limit> <batchcount> <sharedfactor> :
slabdata <active_slabs> <num_slabs> <sharedavail>
ext4_groupinfo_1k 19 19 208 19 1 : tunables 0 0
0 : slabdata 1 1 0
p9_req_t 0 0 176 23 1 : tunables 0 0
0 : slabdata 0 0 0
ip6-frags 0 0 208 19 1 : tunables 0 0
0 : slabdata 0 0 0
UDPv6 0 0 1472 11 4 : tunables 0 0
0 : slabdata 0 0 0
tw_sock_TCPv6 0 0 264 15 1 : tunables 0 0
0 : slabdata 0 0 0
request_sock_TCPv6 0 0 312 13 1 : tunables 0 0
0 : slabdata 0 0 0
TCPv6 0 0 2560 12 8 : tunables 0 0
0 : slabdata 0 0 0
bio-96 32 32 128 32 1 : tunables 0 0
0 : slabdata 1 1 0
bfq_io_cq 0 0 1352 12 4 : tunables 0 0
0 : slabdata 0 0 0
bfq_queue 0 0 576 14 2 : tunables 0 0
0 : slabdata 0 0 0
mqueue_inode_cache 14 14 1152 14 4 : tunables 0 0
0 : slabdata 1 1 0
v9fs_inode_cache 0 0 888 9 2 : tunables 0 0
0 : slabdata 0 0 0
nfs4_xattr_cache_cache 0 0 3168 10 8 : tunables 0
0 0 : slabdata 0 0 0
nfs_direct_cache 0 0 264 15 1 : tunables 0 0
0 : slabdata 0 0 0
nfs_commit_data 11 11 704 11 2 : tunables 0 0
0 : slabdata 1 1 0
nfs_read_data 36 36 896 9 2 : tunables 0 0
0 : slabdata 4 4 0
nfs_inode_cache 0 0 1272 25 8 : tunables 0 0
0 : slabdata 0 0 0
isofs_inode_cache 0 0 824 19 4 : tunables 0 0
0 : slabdata 0 0 0
fat_inode_cache 0 0 976 8 2 : tunables 0 0
0 : slabdata 0 0 0
fat_cache 0 0 40 102 1 : tunables 0 0
0 : slabdata 0 0 0
jbd2_journal_head 0 0 144 28 1 : tunables 0 0
0 : slabdata 0 0 0
jbd2_revoke_table_s 0 0 16 256 1 : tunables 0
0 0 : slabdata 0 0 0
ext4_fc_dentry_update 0 0 96 42 1 : tunables 0
0 0 : slabdata 0 0 0
ext4_inode_cache 105 105 1496 21 8 : tunables 0 0
0 : slabdata 5 5 0
ext4_allocation_context 30 30 136 30 1 : tunables 0
0 0 : slabdata 1 1 0
ext4_prealloc_space 34 34 120 34 1 : tunables 0
0 0 : slabdata 1 1 0
ext4_system_zone 102 102 40 102 1 : tunables 0 0
0 : slabdata 1 1 0
ext4_io_end_vec 0 0 32 128 1 : tunables 0 0
0 : slabdata 0 0 0
bio_post_read_ctx 170 170 48 85 1 : tunables 0 0
0 : slabdata 2 2 0
pending_reservation 0 0 32 128 1 : tunables 0
0 0 : slabdata 0 0 0
extent_status 102 102 40 102 1 : tunables 0 0
0 : slabdata 1 1 0
mbcache 0 0 56 73 1 : tunables 0 0
0 : slabdata 0 0 0
dnotify_struct 0 0 32 128 1 : tunables 0 0
0 : slabdata 0 0 0
pid_namespace 0 0 160 25 1 : tunables 0 0
0 : slabdata 0 0 0
posix_timers_cache 0 0 272 15 1 : tunables 0 0
0 : slabdata 0 0 0
rpc_inode_cache 0 0 832 19 4 : tunables 0 0
0 : slabdata 0 0 0
UNIX 12 12 1344 12 4 : tunables 0 0
0 : slabdata 1 1 0
ip4-frags 0 0 224 18 1 : tunables 0 0
0 : slabdata 0 0 0
xfrm_dst_cache 0 0 320 12 1 : tunables 0 0
0 : slabdata 0 0 0
ip_fib_trie 85 85 48 85 1 : tunables 0 0
0 : slabdata 1 1 0
ip_fib_alias 73 73 56 73 1 : tunables 0 0
0 : slabdata 1 1 0
UDP 12 12 1280 12 4 : tunables 0 0
0 : slabdata 1 1 0
[35/1689]
tw_sock_TCP 0 0 264 15 1 : tunables 0 0
0 : slabdata 0 0 0
request_sock_TCP 0 0 312 13 1 : tunables 0 0
0 : slabdata 0 0 0
TCP 0 0 2432 13 8 : tunables 0 0
0 : slabdata 0 0 0
hugetlbfs_inode_cache 10 10 784 10 2 : tunables 0
0 0 : slabdata 1 1 0
bio-224 48 48 256 16 1 : tunables 0 0
0 : slabdata 3 3 0
ep_head 0 0 16 256 1 : tunables 0 0
0 : slabdata 0 0 0
inotify_inode_mark 0 0 96 42 1 : tunables 0 0
0 : slabdata 0 0 0
dax_cache 8 8 960 8 2 : tunables 0 0
0 : slabdata 1 1 0
sgpool-128 10 10 3072 10 8 : tunables 0 0
0 : slabdata 1 1 0
sgpool-64 10 10 1536 10 4 : tunables 0 0
0 : slabdata 1 1 0
sgpool-16 10 10 384 10 1 : tunables 0 0
0 : slabdata 1 1 0
request_queue 15 15 1040 15 4 : tunables 0 0
0 : slabdata 1 1 0
bio-160 42 42 192 21 1 : tunables 0 0
0 : slabdata 2 2 0
biovec-128 8 8 2048 8 4 : tunables 0 0
0 : slabdata 1 1 0
biovec-64 8 8 1024 8 2 : tunables 0 0
0 : slabdata 1 1 0
user_namespace 0 0 632 25 4 : tunables 0 0
0 : slabdata 0 0 0
uid_cache 84 84 192 21 1 : tunables 0 0
0 : slabdata 4 4 0
dmaengine-unmap-2 64 64 64 64 1 : tunables 0 0
0 : slabdata 1 1 0
sock_inode_cache 24 24 1024 8 2 : tunables 0 0
0 : slabdata 3 3 0
skbuff_small_head 12 12 640 12 2 : tunables 0 0
0 : slabdata 1 1 0
skbuff_fclone_cache 0 0 512 8 1 : tunables 0
0 0 : slabdata 0 0 0
file_lock_cache 17 17 232 17 1 : tunables 0 0
0 : slabdata 1 1 0
fsnotify_mark_connector 0 0 56 73 1 : tunables 0
0 0 : slabdata 0 0 0
pde_opener 0 0 40 102 1 : tunables 0 0
0 : slabdata 0 0 0
proc_inode_cache 57 57 848 19 4 : tunables 0 0
0 : slabdata 3 3 0
seq_file 26 26 152 26 1 : tunables 0 0
0 : slabdata 1 1 0
sigqueue 51 51 80 51 1 : tunables 0 0
0 : slabdata 1 1 0
bdev_cache 18 18 1792 9 4 : tunables 0 0
0 : slabdata 2 2 0
shmem_inode_cache 646 646 936 17 4 : tunables 0 0
0 : slabdata 38 38 0
kernfs_iattrs_cache 0 0 96 42 1 : tunables 0
0 0 : slabdata 0 0 0
kernfs_node_cache 14304 14304 128 32 1 : tunables 0 0
0 : slabdata 447 447 0
filp 84 84 320 12 1 : tunables 0 0
0 : slabdata 7 7 0
inode_cache 360 360 776 10 2 : tunables 0 0
0 : slabdata 36 36 0
dentry 1188 1188 216 18 1 : tunables 0 0
0 : slabdata 66 66 0
names_cache 48 48 4096 8 8 : tunables 0 0
0 : slabdata 6 6 0
net_namespace 0 0 3840 8 8 : tunables 0 0
0 : slabdata 0 0 0
iint_cache 0 0 152 26 1 : tunables 0 0
0 : slabdata 0 0 0
uts_namespace 0 0 432 9 1 : tunables 0 0
0 : slabdata 0 0 0
nsproxy 56 56 72 56 1 : tunables 0 0
0 : slabdata 1 1 0
vm_area_struct 240 240 136 30 1 : tunables 0 0
0 : slabdata 8 8 0
files_cache 22 22 704 11 2 : tunables 0 0
0 : slabdata 2 2 0
signal_cache 56 56 1152 14 4 : tunables 0 0
0 : slabdata 4 4 0
sighand_cache 57 57 1664 19 8 : tunables 0 0
0 : slabdata 3 3 0
task_struct 55 55 2880 11 8 : tunables 0 0
0 : slabdata 5 5 0
anon_vma 120 120 136 30 1 : tunables 0 0
0 : slabdata 4 4 0
perf_event 0 0 1152 14 4 : tunables 0 0
0 : slabdata 0 0 0
maple_node 304 304 256 16 1 : tunables 0 0
0 : slabdata 19 19 0
radix_tree_node 350 350 584 14 2 : tunables 0 0
0 : slabdata 25 25 0
task_group 10 10 768 10 2 : tunables 0 0
0 : slabdata 1 1 0
mm_struct 22 22 1408 11 4 : tunables 0 0
0 : slabdata 2 2 0
vmap_area 168 168 72 56 1 : tunables 0 0
0 : slabdata 3 3 0
page->ptl 170 170 24 170 1 : tunables 0 0
0 : slabdata 1 1 0
kmalloc-cg-8k 0 0 8192 4 8 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-cg-4k 24 24 4096 8 8 : tunables 0 0
0 : slabdata 3 3 0
kmalloc-cg-2k 32 32 2048 8 4 : tunables 0 0
0 : slabdata 4 4 0
kmalloc-cg-1k 24 24 1024 8 2 : tunables 0 0
0 : slabdata 3 3 0
kmalloc-cg-512 32 32 512 8 1 : tunables 0 0
0 : slabdata 4 4 0
kmalloc-cg-256 16 16 256 16 1 : tunables 0 0
0 : slabdata 1 1 0
kmalloc-cg-192 147 147 192 21 1 : tunables 0 0
0 : slabdata 7 7 0
kmalloc-cg-128 64 64 128 32 1 : tunables 0 0
0 : slabdata 2 2 0
kmalloc-cg-64 320 320 64 64 1 : tunables 0 0
0 : slabdata 5 5 0
kmalloc-rcl-8k 0 0 8192 4 8 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-4k 0 0 4096 8 8 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-2k 0 0 2048 8 4 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-1k 0 0 1024 8 2 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-512 0 0 512 8 1 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-256 0 0 256 16 1 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-192 0 0 192 21 1 : tunables 0 0
0 : slabdata 0 0 0
kmalloc-rcl-128 320 320 128 32 1 : tunables 0 0
0 : slabdata 10 10 0
kmalloc-rcl-64 64 64 64 64 1 : tunables 0 0
0 : slabdata 1 1 0
kmalloc-8k 12 12 8192 4 8 : tunables 0 0
0 : slabdata 3 3 0
kmalloc-4k 16 16 4096 8 8 : tunables 0 0
0 : slabdata 2 2 0
kmalloc-2k 64 64 2048 8 4 : tunables 0 0
0 : slabdata 8 8 0
kmalloc-1k 840 840 1024 8 2 : tunables 0 0
0 : slabdata 105 105 0
kmalloc-512 144 144 512 8 1 : tunables 0 0
0 : slabdata 18 18 0
kmalloc-256 816 816 256 16 1 : tunables 0 0
0 : slabdata 51 51 0
kmalloc-192 252 252 192 21 1 : tunables 0 0
0 : slabdata 12 12 0
kmalloc-128 480 480 128 32 1 : tunables 0 0
0 : slabdata 15 15 0
kmalloc-64 4912 4928 64 64 1 : tunables 0 0
0 : slabdata 77 77 0
kmem_cache_node 128 128 128 32 1 : tunables 0 0
0 : slabdata 4 4 0
kmem_cache 126 126 192 21 1 : tunables 0 0
0 : slabdata 6 6 0
>
> Arnd
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-20 2:53 ` Guo Ren
@ 2023-05-20 10:13 ` Arnd Bergmann
2023-05-20 15:57 ` Guo Ren
0 siblings, 1 reply; 37+ messages in thread
From: Arnd Bergmann @ 2023-05-20 10:13 UTC (permalink / raw)
To: guoren
Cc: Palmer Dabbelt, Thomas Gleixner, Peter Zijlstra, Andy Lutomirski,
Conor.Dooley, Heiko Stübner, Jisheng Zhang, Huacai Chen,
Anup Patel, Atish Patra, Mark Rutland, Björn Töpel,
Paul Walmsley, Catalin Marinas, Will Deacon, Mike Rapoport,
Anup Patel, shihua, jiawei, liweiwei, luxufan, chunyu, tsu.yubo,
wefu, wangjunqiang, kito.cheng, Andy Chiu, Vincent Chen,
Greentime Hu, Jonathan Corbet, wuwei2016, Jessica Clarke,
Linux-Arch, linux-kernel, linux-riscv, Guo Ren
On Sat, May 20, 2023, at 04:53, Guo Ren wrote:
> On Sat, May 20, 2023 at 4:20 AM Arnd Bergmann <arnd@arndb.de> wrote:
>> On Thu, May 18, 2023, at 15:09, guoren@kernel.org wrote:
>>
>> I've tried to run the same numbers for the debate about running
>> 32-bit vs 64-bit arm kernels in the past, but focused mostly on
>> slightly larger systems, but I looked mainly at the 512MB case,
>> as that is the most cost-efficient DDR3 memory configuration
>> and fairly common.
> 512MB is extravagant, in my opinion. In the IPC market, 32/64MB is for
> 480P/720P/1080p, 128/256MB is for 1080p/2k, and 512/1024MB is for 4K.
>> 512MB chips is less than 5% of the total (I guess). Even in 512MB
> chips, the additional memory is for the frame buffer, not the Linux
> system.
This depends a lot on the target application of course. For
a phone or NAS box, 512MB is probably the lower limit.
What I observe in arch/arm/ devicetree submissions, in board-db.org,
and when looking at industrial Arm board vendor websites is that
512MB is the most common configuration, and I think 1GB is still
more common than 256MB even for 32-bit machines. There is of course
a difference between number of individual products, and number of
machines shipped in a given configuration, and I guess you have
a good point that the cheapest ones are also the ones that ship
in the highest volume.
>> What I'd like to understand better in your example is where
>> the 14MB of memory went. I assume this is for 128MB of total
>> RAM, so we know that 1MB went into additional 'struct page'
>> objects (32 bytes * 32768 pages). It would be good to know
>> where the dynamic allocations went and if they are reclaimable
>> (e.g. inodes) or non-reclaimable (e.g. kmalloc-128).
>>
>> For the vmlinux size, is this already a minimal config
>> that one would run on a board with 128MB of RAM, or a
>> defconfig that includes a lot of stuff that is only relevant
>> for other platforms but also grows on 64-bit?
> It's not minimal config, it's defconfig. So I say it's a roungh
> measurement :)
>
> I admit I wanted a little bit to exaggerate it, but that's the
> starting point for cutting down memory usage for most people, right?
> During the past year, we have been convincing our customers to use the
> s64lp64 + u32ilp32, but they can't tolerate even 1% memory additional
> cost in 64MB/128MB scenarios and then chose cortex-a7/a35, which could
> run 32-bit Linux. I think it's too early to talk about throwing 32-bit
> Linux into the garbage, not only for the reason of memory footprint
> but also for the ingrained opinion of the people. Changing their mind
> needs a long time.
>
>>
>> What do you see in /proc/slabinfo, /proc/meminfo/, and
>> 'size vmlinux' for the s64ilp32 and s64lp64 kernels here?
> Both s64ilp32 & s64lp64 use the same u32ilp32_rootfs.ext2 binary and
> the same opensbi binary.
> All are opensbi(2MB) + Linux(126MB) memory layout.
>
> Here is the result:
>
> s64ilp32:
> [ 0.000000] Virtual kernel memory layout:
> [ 0.000000] fixmap : 0x9ce00000 - 0x9d000000 (2048 kB)
> [ 0.000000] pci io : 0x9d000000 - 0x9e000000 ( 16 MB)
> [ 0.000000] vmemmap : 0x9e000000 - 0xa0000000 ( 32 MB)
> [ 0.000000] vmalloc : 0xa0000000 - 0xc0000000 ( 512 MB)
> [ 0.000000] lowmem : 0xc0000000 - 0xc7e00000 ( 126 MB)
> [ 0.000000] Memory: 97748K/129024K available (8699K kernel code,
> 8867K rwdata, 4096K rodata, 4204K init, 361K bss, 31276K reserved, 0K
> cma-reserved)
Ok, so it saves only a little bit on .text/.init/.bss/.rodata, but
there is a 4MB difference in rwdata, and a total of 10.4MB difference
in "reserved" size, which I think includes all of the above plus
the mem_map[] array.
89380K/131072K available (8638K kernel code, 4979K rwdata, 4096K rodata, 2191K init, 477K bss, 41692K reserved, 0K cma-reserved)
Oddly, I don't see anywhere close to 8KB in a riscv64 defconfig
build (linux-next, gcc-13), so I don't know where that comes
from:
$ size -A build/tmp/vmlinux | sort -k2 -nr | head
Total 13518684
.text 8896058 18446744071562076160
.rodata 2219008 18446744071576748032
.data 933760 18446744071583039488
.bss 476080 18446744071584092160
.init.text 264718 18446744071572553728
__ksymtab_strings 183986 18446744071579214312
__ksymtab_gpl 122928 18446744071579091384
__ksymtab 109080 18446744071578982304
__bug_table 98352 18446744071583973248
> KReclaimable: 644 kB
> Slab: 4536 kB
> SReclaimable: 644 kB
> SUnreclaim: 3892 kB
> KernelStack: 344 kB
These look like the only notable differences in meminfo:
KReclaimable: 1092 kB
Slab: 6900 kB
SReclaimable: 1092 kB
SUnreclaim: 5808 kB
KernelStack: 688 kB
The largest chunk here is 2MB in non-reclaimable slab allocations,
or a 50% growth of those.
The kernel stacks are doubled as expected, but that's only 344KB,
similarly for reclaimable slabs.
> # cat /proc/slabinfo
>
> [68/1691]
> slabinfo - version: 2.1
> # name <active_objs> <num_objs> <objsize> <objperslab>
> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> :
> slabdata <active_slabs> <num_slabs> <sharedavail>
> ext4_groupinfo_1k 28 28 144 28 1 : tunables 0 0
> 0 : slabdata 1 1 0
> p9_req_t 0 0 104 39 1 : tunables 0 0
Did you perhaps miss a few lines while pasting these? It seems
odd that some caches only show up in the ilp32 case (proc_dir_entry,
bd2_journa_handle, buffer_head, biovec_max, anon_vma_chain, ...) and
some others are only in the lp64 case (UNIX, ext4_prealloc_space,
files_cache, filp, ip_fib_alias, task_struct, uid_cache, ...).
Looking at the ones that are in both and have the largest size
increase, I see
# lp64
1788 kernfs_node_cache 14304 128
590 shmem_inode_cache 646 936
272 inode_cache 360 776
153 ext4_inode_cache 105 1496
250 dentry 1188 216
192 names_cache 48 4096
199 radix_tree_node 350 584
307 kmalloc-64 4912 64
60 kmalloc-128 480 128
47 kmalloc-192 252 192
204 kmalloc-256 816 256
72 kmalloc-512 144 512
840 kmalloc-1k 840 1024
# ilp32
1197 kernfs_node_cache 13938 88
373 shmem_inode_cache 637 600
174 inode_cache 360 496
84 ext4_inode_cache 88 984
177 dentry 1196 152
32 names_cache 8 4096
100 radix_tree_node 338 304
331 kmalloc-64 5302 64
132 kmalloc-128 1056 128
23 kmalloc-192 126 192
16 kmalloc-256 64 256
428 kmalloc-512 856 512
88 kmalloc-1k 88 1024
So sysfs (kernfs_node_cache) has the largest chunk of the
2MB non-reclaimable slab, grown 50% from 1.2MB to 1.8MB.
In some cases, this could be avoided entirely by turning
off sysfs, but most users can't do that.
shmem_inode_cache is probably mostly devtmpfs, the
other inode caches ones are smaller and likely reclaimable.
It's interesting how the largest slab cache ends up
being the kmalloc-1k cache (840 1K objects) on lp64,
but the kmalloc-512 cache (856 512B objects) on ilp32.
My guess is that the majority of this is from a single
callsite that has an allocation groing just beyond 512B.
This alone seems significant enough to need further
investigation, I would hope we can completely avoid
these by adding a custom slab cache. I don't see this
effect on an arm64 boot though, for me the 512B allocations
are much higher the 1K ones.
Maybe you can identify the culprit using the boot-time traces
as listed in https://elinux.org/Kernel_dynamic_memory_analysis#Dynamic
That might help everyone running a 64-bit kernel on
low-memory configurations, though it would of course slightly
weaken your argument for an ilp32 kernel ;-)
Arnd
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-20 10:13 ` Arnd Bergmann
@ 2023-05-20 15:57 ` Guo Ren
0 siblings, 0 replies; 37+ messages in thread
From: Guo Ren @ 2023-05-20 15:57 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Palmer Dabbelt, Thomas Gleixner, Peter Zijlstra, Andy Lutomirski,
Conor.Dooley, Heiko Stübner, Jisheng Zhang, Huacai Chen,
Anup Patel, Atish Patra, Mark Rutland, Björn Töpel,
Paul Walmsley, Catalin Marinas, Will Deacon, Mike Rapoport,
Anup Patel, shihua, jiawei, liweiwei, luxufan, chunyu, tsu.yubo,
wefu, wangjunqiang, kito.cheng, Andy Chiu, Vincent Chen,
Greentime Hu, Jonathan Corbet, wuwei2016, Jessica Clarke,
Linux-Arch, linux-kernel, linux-riscv, Guo Ren
On Sat, May 20, 2023 at 6:13 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Sat, May 20, 2023, at 04:53, Guo Ren wrote:
> > On Sat, May 20, 2023 at 4:20 AM Arnd Bergmann <arnd@arndb.de> wrote:
> >> On Thu, May 18, 2023, at 15:09, guoren@kernel.org wrote:
> >>
> >> I've tried to run the same numbers for the debate about running
> >> 32-bit vs 64-bit arm kernels in the past, but focused mostly on
> >> slightly larger systems, but I looked mainly at the 512MB case,
> >> as that is the most cost-efficient DDR3 memory configuration
> >> and fairly common.
> > 512MB is extravagant, in my opinion. In the IPC market, 32/64MB is for
> > 480P/720P/1080p, 128/256MB is for 1080p/2k, and 512/1024MB is for 4K.
> >> 512MB chips is less than 5% of the total (I guess). Even in 512MB
> > chips, the additional memory is for the frame buffer, not the Linux
> > system.
>
> This depends a lot on the target application of course. For
> a phone or NAS box, 512MB is probably the lower limit.
>
> What I observe in arch/arm/ devicetree submissions, in board-db.org,
> and when looking at industrial Arm board vendor websites is that
> 512MB is the most common configuration, and I think 1GB is still
> more common than 256MB even for 32-bit machines. There is of course
> a difference between number of individual products, and number of
> machines shipped in a given configuration, and I guess you have
> a good point that the cheapest ones are also the ones that ship
> in the highest volume.
>
> >> What I'd like to understand better in your example is where
> >> the 14MB of memory went. I assume this is for 128MB of total
> >> RAM, so we know that 1MB went into additional 'struct page'
> >> objects (32 bytes * 32768 pages). It would be good to know
> >> where the dynamic allocations went and if they are reclaimable
> >> (e.g. inodes) or non-reclaimable (e.g. kmalloc-128).
> >>
> >> For the vmlinux size, is this already a minimal config
> >> that one would run on a board with 128MB of RAM, or a
> >> defconfig that includes a lot of stuff that is only relevant
> >> for other platforms but also grows on 64-bit?
> > It's not minimal config, it's defconfig. So I say it's a roungh
> > measurement :)
> >
> > I admit I wanted a little bit to exaggerate it, but that's the
> > starting point for cutting down memory usage for most people, right?
> > During the past year, we have been convincing our customers to use the
> > s64lp64 + u32ilp32, but they can't tolerate even 1% memory additional
> > cost in 64MB/128MB scenarios and then chose cortex-a7/a35, which could
> > run 32-bit Linux. I think it's too early to talk about throwing 32-bit
> > Linux into the garbage, not only for the reason of memory footprint
> > but also for the ingrained opinion of the people. Changing their mind
> > needs a long time.
> >
> >>
> >> What do you see in /proc/slabinfo, /proc/meminfo/, and
> >> 'size vmlinux' for the s64ilp32 and s64lp64 kernels here?
> > Both s64ilp32 & s64lp64 use the same u32ilp32_rootfs.ext2 binary and
> > the same opensbi binary.
> > All are opensbi(2MB) + Linux(126MB) memory layout.
> >
> > Here is the result:
> >
> > s64ilp32:
> > [ 0.000000] Virtual kernel memory layout:
> > [ 0.000000] fixmap : 0x9ce00000 - 0x9d000000 (2048 kB)
> > [ 0.000000] pci io : 0x9d000000 - 0x9e000000 ( 16 MB)
> > [ 0.000000] vmemmap : 0x9e000000 - 0xa0000000 ( 32 MB)
> > [ 0.000000] vmalloc : 0xa0000000 - 0xc0000000 ( 512 MB)
> > [ 0.000000] lowmem : 0xc0000000 - 0xc7e00000 ( 126 MB)
> > [ 0.000000] Memory: 97748K/129024K available (8699K kernel code,
> > 8867K rwdata, 4096K rodata, 4204K init, 361K bss, 31276K reserved, 0K
> > cma-reserved)
>
> Ok, so it saves only a little bit on .text/.init/.bss/.rodata, but
> there is a 4MB difference in rwdata, and a total of 10.4MB difference
> in "reserved" size, which I think includes all of the above plus
> the mem_map[] array.
>
> 89380K/131072K available (8638K kernel code, 4979K rwdata, 4096K rodata, 2191K init, 477K bss, 41692K reserved, 0K cma-reserved)
>
> Oddly, I don't see anywhere close to 8KB in a riscv64 defconfig
> build (linux-next, gcc-13), so I don't know where that comes
> from:
>
> $ size -A build/tmp/vmlinux | sort -k2 -nr | head
> Total 13518684
> .text 8896058 18446744071562076160
> .rodata 2219008 18446744071576748032
> .data 933760 18446744071583039488
> .bss 476080 18446744071584092160
> .init.text 264718 18446744071572553728
> __ksymtab_strings 183986 18446744071579214312
> __ksymtab_gpl 122928 18446744071579091384
> __ksymtab 109080 18446744071578982304
> __bug_table 98352 18446744071583973248
>
>
>
> > KReclaimable: 644 kB
> > Slab: 4536 kB
> > SReclaimable: 644 kB
> > SUnreclaim: 3892 kB
> > KernelStack: 344 kB
>
> These look like the only notable differences in meminfo:
>
> KReclaimable: 1092 kB
> Slab: 6900 kB
> SReclaimable: 1092 kB
> SUnreclaim: 5808 kB
> KernelStack: 688 kB
>
> The largest chunk here is 2MB in non-reclaimable slab allocations,
> or a 50% growth of those.
>
> The kernel stacks are doubled as expected, but that's only 344KB,
> similarly for reclaimable slabs.
>
> > # cat /proc/slabinfo
> >
> > [68/1691]
> > slabinfo - version: 2.1
> > # name <active_objs> <num_objs> <objsize> <objperslab>
> > <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> :
> > slabdata <active_slabs> <num_slabs> <sharedavail>
> > ext4_groupinfo_1k 28 28 144 28 1 : tunables 0 0
> > 0 : slabdata 1 1 0
> > p9_req_t 0 0 104 39 1 : tunables 0 0
>
> Did you perhaps miss a few lines while pasting these? It seems
> odd that some caches only show up in the ilp32 case (proc_dir_entry,
> bd2_journa_handle, buffer_head, biovec_max, anon_vma_chain, ...) and
> some others are only in the lp64 case (UNIX, ext4_prealloc_space,
> files_cache, filp, ip_fib_alias, task_struct, uid_cache, ...).
>
> Looking at the ones that are in both and have the largest size
> increase, I see
>
> # lp64
> 1788 kernfs_node_cache 14304 128
> 590 shmem_inode_cache 646 936
> 272 inode_cache 360 776
> 153 ext4_inode_cache 105 1496
> 250 dentry 1188 216
> 192 names_cache 48 4096
> 199 radix_tree_node 350 584
> 307 kmalloc-64 4912 64
> 60 kmalloc-128 480 128
> 47 kmalloc-192 252 192
> 204 kmalloc-256 816 256
> 72 kmalloc-512 144 512
> 840 kmalloc-1k 840 1024
>
> # ilp32
> 1197 kernfs_node_cache 13938 88
> 373 shmem_inode_cache 637 600
> 174 inode_cache 360 496
> 84 ext4_inode_cache 88 984
> 177 dentry 1196 152
> 32 names_cache 8 4096
> 100 radix_tree_node 338 304
> 331 kmalloc-64 5302 64
> 132 kmalloc-128 1056 128
> 23 kmalloc-192 126 192
> 16 kmalloc-256 64 256
> 428 kmalloc-512 856 512
> 88 kmalloc-1k 88 1024
>
> So sysfs (kernfs_node_cache) has the largest chunk of the
> 2MB non-reclaimable slab, grown 50% from 1.2MB to 1.8MB.
> In some cases, this could be avoided entirely by turning
> off sysfs, but most users can't do that.
> shmem_inode_cache is probably mostly devtmpfs, the
> other inode caches ones are smaller and likely reclaimable.
>
> It's interesting how the largest slab cache ends up
> being the kmalloc-1k cache (840 1K objects) on lp64,
> but the kmalloc-512 cache (856 512B objects) on ilp32.
> My guess is that the majority of this is from a single
> callsite that has an allocation groing just beyond 512B.
> This alone seems significant enough to need further
> investigation, I would hope we can completely avoid
> these by adding a custom slab cache. I don't see this
> effect on an arm64 boot though, for me the 512B allocations
> are much higher the 1K ones.
>
> Maybe you can identify the culprit using the boot-time traces
> as listed in https://elinux.org/Kernel_dynamic_memory_analysis#Dynamic
> That might help everyone running a 64-bit kernel on
> low-memory configurations, though it would of course slightly
> weaken your argument for an ilp32 kernel ;-)
Thx for the detailed reply, I would try your approches mentioned
lately. But these about traditional CONFIG_32BIT v.s. CONFIG_64BIT
comparation.
Besides the detailed analysis data, we also would meet the people's
concept problem. Such as struct page, struct list_head, and some
variables containing pointers, ilp32's would be significantly smaller
than lp64. That means ilp32 is smaller than lp64 in people's minds.
This concept would prevent vendors from accepting lp64 as a cost-down
solution. They even won't try, which I've met these years. I was an
lp64 kernel supporter last year, but I met a lot of arguments on
s64lp64 + u32ilp32. Some guys are using arm32 Linux; they want to stay
on 32-bit Linux to ensure their complex C code can work. So our
argument about "ilp32 v.s. lp64" won't have a result.
Let's change another view, cache utilization. These 64/128MB SoCs also
have limited cache capacities (L1-32KB+L2-128KB/only L1-64KB). Such as
List walk and stack saving/restoring are very common in Linux. What do
you think about "32-bit v.s. 64-bit" cache utilization?
>
> Arnd
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode
2023-05-19 0:14 ` Paul Walmsley
@ 2023-05-21 12:37 ` Guo Ren
0 siblings, 0 replies; 37+ messages in thread
From: Guo Ren @ 2023-05-21 12:37 UTC (permalink / raw)
To: Paul Walmsley
Cc: Palmer Dabbelt, Arnd Bergmann, tglx, peterz, luto, Conor Dooley,
heiko, jszhang, chenhuacai, apatel, atishp, Mark Rutland, bjorn,
Catalin Marinas, Will Deacon, rppt, anup, shihua, jiawei,
liweiwei, luxufan, chunyu, tsu.yubo, wefu, wangjunqiang,
kito.cheng, andy.chiu, vincent.chen, greentime.hu, corbet,
wuwei2016, jrtc27, linux-arch, linux-kernel, linux-riscv, guoren
On Fri, May 19, 2023 at 8:14 AM Paul Walmsley <paul.walmsley@sifive.com> wrote:
>
> On Thu, 18 May 2023, Palmer Dabbelt wrote:
>
> > On Thu, 18 May 2023 06:09:51 PDT (-0700), guoren@kernel.org wrote:
> >
> > > This patch series adds s64ilp32 support to riscv. The term s64ilp32
> > > means smode-xlen=64 and -mabi=ilp32 (ints, longs, and pointers are all
> > > 32-bit), i.e., running 32-bit Linux kernel on pure 64-bit supervisor
> > > mode. There have been many 64ilp32 abis existing, such as mips-n32 [1],
> > > arm-aarch64ilp32 [2], and x86-x32 [3], but they are all about userspace.
> > > Thus, this should be the first time running a 32-bit Linux kernel with
> > > the 64ilp32 ABI at supervisor mode (If not, correct me).
> >
> > Does anyone actually want this? At a bare minimum we'd need to add it to the
> > psABI, which would presumably also be required on the compiler side of things.
> >
> > It's not even clear anyone wants rv64/ilp32 in userspace, the kernel seems
> > like it'd be even less widely used.
>
> We've certainly talked to folks who are interested in RV64 ILP32 userspace
> with an LP64 kernel. The motivation is the usual one: to reduce data size
> and therefore (ideally) BOM cost. I think this work, if it goes forward,
> would need to go hand in hand with the RVIA psABI group.
>
> The RV64 ILP32 kernel and ILP32 userspace approach implemented by this
> patch is intriguing, but I guess for me, the question is whether it's
> worth the extra hassle vs. a pure RV32 kernel & userspace.
Running pure RV32 kernel on 64-bit hardware is not an intelligent
choice (such as cortex-a35/a53/a55), because they wasted 64-bit hw
capabilities, and the hardware designer would waste additional
resources & time on 32-bit machine & supervisor modes (In arm it is
called EL3/EL2/EL1 modes). Think about too many PMP CSRs, PMU CSRs,
and mode switch ... it's definitely wrong to follow the
cortex-a35/a53/a55 way to deal with riscv32 on a 64-bit hardware. The
chapter "Why s64ilp32 has better performance?" give out the
improvement v.s. pure 32-bit, I repeat it here:
- memcpy/memset/strcmp (s64ilp32 has half of the number of
instructions and double the bandwidth per load/store instruction than
s32ilp32.)
- ebpf JIT is a 64-bit virtual ISA, which couldn't be sufficient
mapping by s32ilp32, but s64ilp32 could (just like s64lp64).
- Atomic64 (s64ilp32 has the exact native instructions mapping as
s64lp64, but s32ilp32 only uses generic_atomic64, a tradeoff & limited
software solution.)
- 64-bit native arithmetic instructions for "long long" type
- riscv s64ilp32 could support cmxchg_double for slub (The 2nd 32-bit
Linux supports the feature, the 1st is i386.)
>
>
> - Paul
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2023-05-21 12:38 UTC | newest]
Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-18 13:09 [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode guoren
2023-05-18 13:09 ` [RFC PATCH 01/22] riscv: vdso: Unify vdso32 & compat_vdso into vdso/Makefile guoren
2023-05-18 13:09 ` [RFC PATCH 02/22] riscv: vdso: Remove compat_vdso/ guoren
2023-05-18 13:09 ` [RFC PATCH 03/22] riscv: vdso: Add time-related vDSO common flow for vdso32 guoren
2023-05-18 13:09 ` [RFC PATCH 04/22] clocksource: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT guoren
2023-05-18 13:09 ` [RFC PATCH 05/22] riscv: s64ilp32: Introduce xlen_t guoren
2023-05-18 13:09 ` [RFC PATCH 06/22] irqchip: riscv: s64ilp32: Use __riscv_xlen instead of CONFIG_32BIT guoren
2023-05-18 13:09 ` [RFC PATCH 07/22] riscv: s64ilp32: Add sbi support guoren
2023-05-18 13:09 ` [RFC PATCH 08/22] riscv: s64ilp32: Add asid support guoren
2023-05-18 13:10 ` [RFC PATCH 09/22] riscv: s64ilp32: Introduce PTR_L and PTR_S guoren
2023-05-18 13:10 ` [RFC PATCH 10/22] riscv: s64ilp32: Enable user space runtime environment guoren
2023-05-18 13:10 ` [RFC PATCH 11/22] riscv: s64ilp32: Add ebpf jit support guoren
2023-05-18 13:10 ` [RFC PATCH 12/22] riscv: s64ilp32: Add ELF32 support guoren
2023-05-18 13:10 ` [RFC PATCH 13/22] riscv: s64ilp32: Add ARCH RV64 ILP32 compiling framework guoren
2023-05-18 13:10 ` [RFC PATCH 14/22] riscv: s64ilp32: Add MMU_SV39 mode support for 32BIT guoren
2023-05-18 13:10 ` [RFC PATCH 15/22] riscv: s64ilp32: Enable native atomic64 guoren
2023-05-18 13:10 ` [RFC PATCH 16/22] riscv: s64ilp32: Add TImode (128 int) support guoren
2023-05-18 13:10 ` [RFC PATCH 17/22] riscv: s64ilp32: Implement cmpxchg_double guoren
2023-05-18 13:10 ` [RFC PATCH 18/22] riscv: s64ilp32: Disable KVM guoren
2023-05-18 13:10 ` [RFC PATCH 19/22] riscv: Cleanup rv32_defconfig guoren
2023-05-18 13:10 ` [RFC PATCH 20/22] riscv: s64ilp32: Add rv64ilp32_defconfig guoren
2023-05-18 13:10 ` [RFC PATCH 21/22] riscv: s64ilp32: Correct the rv64ilp32 stackframe layout guoren
2023-05-18 13:10 ` [RFC PATCH 22/22] riscv: s64ilp32: Temporary workaround solution to gcc problem guoren
2023-05-18 15:38 ` [RFC PATCH 00/22] riscv: s64ilp32: Running 32-bit Linux kernel on 64-bit supervisor mode Palmer Dabbelt
2023-05-18 18:29 ` Arnd Bergmann
2023-05-19 0:38 ` Paul Walmsley
2023-05-19 8:55 ` Arnd Bergmann
2023-05-19 15:31 ` Guo Ren
2023-05-19 16:53 ` Arnd Bergmann
2023-05-19 17:18 ` Palmer Dabbelt
2023-05-20 1:43 ` Guo Ren
2023-05-19 0:14 ` Paul Walmsley
2023-05-21 12:37 ` Guo Ren
2023-05-19 20:20 ` Arnd Bergmann
2023-05-20 2:53 ` Guo Ren
2023-05-20 10:13 ` Arnd Bergmann
2023-05-20 15:57 ` Guo Ren
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).