* [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-10 8:28 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
patch[1 - 10]: Native qspinlock
patch[11 -17]: Paravirt qspinlock
patch[4]: Add prefetchw in qspinlock's xchg_tail when cpus >= 16k
This series based on:
- [RFC PATCH v5 0/5] Rework & improve riscv cmpxchg.h and atomic.h
https://lore.kernel.org/linux-riscv/20230810040349.92279-2-leobras@redhat.com/
- [PATCH V3] asm-generic: ticket-lock: Optimize arch_spin_value_unlocked
https://lore.kernel.org/linux-riscv/20230908154339.3250567-1-guoren@kernel.org/
I merge them into sg2042-master branch, then you could directly try it on
sg2042 hardware platform:
https://github.com/guoren83/linux/tree/sg2042-master-qspinlock-64ilp32_v5
Use sophgo_mango_ubuntu_defconfig for sg2042 64/128 cores hardware
platform.
Native qspinlock
================
This time we've proved the qspinlock on th1520 [1] & sg2042 [2], which
gives stability and performance improvement. All T-HEAD processors have
a strong LR/SC forward progress guarantee than the requirements of the
ISA, which could satisfy the xchg_tail of native_qspinlock. Now,
qspinlock has been run with us for more than 1 year, and we have enough
confidence to enable it for all the T-HEAD processors. Of causes, we
found a livelock problem with the qspinlock lock torture test from the
CPU store merge buffer delay mechanism, which caused the queued spinlock
becomes a dead ring and RCU warning to come out. We introduce a custom
WRITE_ONCE to solve this. Do we need explicit ISA instruction to signal
it? Or let hardware handle this.
We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
test on Fedora & Ubuntu & OpenEuler ... Here is the performance
comparison between qspinlock and ticket_lock on sg2042 (64 cores):
sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
queued_spinlock 0.5109/0.00
ticket_spinlock 0.5814/0.00
perf futex/hash (+6.7%):
queued_spinlock 1444393 operations/sec (+- 0.09%)
ticket_spinlock 1353215 operations/sec (+- 0.15%)
perf futex/wake-parallel (+8.6%):
queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
perf futex/requeue (+4.2%):
queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
System Benchmarks (+6.4%)
queued_spinlock:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
Double-Precision Whetstone 55.0 182422.8 33167.8
Execl Throughput 43.0 13116.6 3050.4
File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
Pipe Throughput 12440.0 23058600.5 18535.9
Pipe-based Context Switching 4000.0 2835617.7 7089.0
Process Creation 126.0 12537.3 995.0
Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
System Call Overhead 15000.0 33308301.3 22205.5
========
System Benchmarks Index Score 12426.1
ticket_spinlock:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
Double-Precision Whetstone 55.0 181921.0 33076.5
Execl Throughput 43.0 12625.1 2936.1
File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
Pipe Throughput 12440.0 20594018.7 16554.7
Pipe-based Context Switching 4000.0 2571117.7 6427.8
Process Creation 126.0 10798.4 857.0
Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
System Call Overhead 15000.0 30766778.4 20511.2
========
System Benchmarks Index Score 11670.7
The qspinlock has a significant improvement on SOPHGO SG2042 64
cores platform than the ticket_lock.
Paravirt qspinlock
==================
We implemented kvm_kick_cpu/kvm_wait_cpu and add tracepoints to observe the
behaviors. Also, introduce a new SBI extension SBI_EXT_PVLOCK (0xAB0401). If the
name and number are approved, I will send a formal proposal to the SBI spec.
Changlog:
V11:
- Based on Leonardo Bras's cmpxchg_small patches v5.
- Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
- Remove abusing alternative framework and use jump_label instead.
- Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
guarantee.
- Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
V10:
https://lore.kernel.org/linux-riscv/20230802164701.192791-1-guoren@kernel.org/
- Using an alternative framework instead of static_key_branch in the
asm/spinlock.h.
- Fixup store merge buffer problem, which causes qspinlock lock
torture test livelock.
- Add paravirt qspinlock support, include KVM backend
- Add Compact NUMA-awared qspinlock support
V9:
https://lore.kernel.org/linux-riscv/20220808071318.3335746-1-guoren@kernel.org/
- Cleanup generic ticket-lock code, (Using smp_mb__after_spinlock as
RCsc)
- Add qspinlock and combo-lock for riscv
- Add qspinlock to openrisc
- Use generic header in csky
- Optimize cmpxchg & atomic code
V8:
https://lore.kernel.org/linux-riscv/20220724122517.1019187-1-guoren@kernel.org/
- Coding convention ticket fixup
- Move combo spinlock into riscv and simply asm-generic/spinlock.h
- Fixup xchg16 with wrong return value
- Add csky qspinlock
- Add combo & qspinlock & ticket-lock comparison
- Clean up unnecessary riscv acquire and release definitions
- Enable ARCH_INLINE_READ*/WRITE*/SPIN* for riscv & csky
V7:
https://lore.kernel.org/linux-riscv/20220628081946.1999419-1-guoren@kernel.org/
- Add combo spinlock (ticket & queued) support
- Rename ticket_spinlock.h
- Remove unnecessary atomic_read in ticket_spin_value_unlocked
V6:
https://lore.kernel.org/linux-riscv/20220621144920.2945595-1-guoren@kernel.org/
- Fixup Clang compile problem Reported-by: kernel test robot
- Cleanup asm-generic/spinlock.h
- Remove changelog in patch main comment part, suggested by
Conor.Dooley
- Remove "default y if NUMA" in Kconfig
V5:
https://lore.kernel.org/linux-riscv/20220620155404.1968739-1-guoren@kernel.org/
- Update comment with RISC-V forward guarantee feature.
- Back to V3 direction and optimize asm code.
V4:
https://lore.kernel.org/linux-riscv/1616868399-82848-4-git-send-email-guoren@kernel.org/
- Remove custom sub-word xchg implementation
- Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 in locking/qspinlock
V3:
https://lore.kernel.org/linux-riscv/1616658937-82063-1-git-send-email-guoren@kernel.org/
- Coding convention by Peter Zijlstra's advices
V2:
https://lore.kernel.org/linux-riscv/1606225437-22948-2-git-send-email-guoren@kernel.org/
- Coding convention in cmpxchg.h
- Re-implement short xchg
- Remove char & cmpxchg implementations
V1:
https://lore.kernel.org/linux-riscv/20190211043829.30096-1-michaeljclark@mac.com/
- Using cmpxchg loop to implement sub-word atomic
Guo Ren (17):
asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
asm-generic: ticket-lock: Move into ticket_spinlock.h
riscv: Use Zicbop in arch_xchg when available
locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
riscv: qspinlock: Add basic queued_spinlock support
riscv: qspinlock: Introduce combo spinlock
riscv: qspinlock: Introduce qspinlock param for command line
riscv: qspinlock: Add virt_spin_lock() support for KVM guest
riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
RISC-V: paravirt: pvqspinlock: Add SBI implementation
RISC-V: paravirt: pvqspinlock: Add kconfig entry
RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
RISC-V: paravirt: pvqspinlock: KVM: Implement
kvm_sbi_ext_pvlock_kick_cpu()
.../admin-guide/kernel-parameters.txt | 8 +-
arch/riscv/Kconfig | 50 ++++++++
arch/riscv/Kconfig.errata | 19 +++
arch/riscv/errata/thead/errata.c | 29 +++++
arch/riscv/include/asm/Kbuild | 2 +-
arch/riscv/include/asm/cmpxchg.h | 4 +-
arch/riscv/include/asm/errata_list.h | 13 --
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/insn-def.h | 5 +
arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
arch/riscv/include/asm/processor.h | 13 ++
arch/riscv/include/asm/qspinlock.h | 35 ++++++
arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++
arch/riscv/include/asm/rwonce.h | 24 ++++
arch/riscv/include/asm/sbi.h | 14 +++
arch/riscv/include/asm/spinlock.h | 113 ++++++++++++++++++
arch/riscv/include/asm/vendorid_list.h | 14 +++
arch/riscv/include/uapi/asm/kvm.h | 1 +
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/cpufeature.c | 1 +
arch/riscv/kernel/qspinlock_paravirt.c | 83 +++++++++++++
arch/riscv/kernel/sbi.c | 2 +-
arch/riscv/kernel/setup.c | 60 ++++++++++
.../kernel/trace_events_filter_paravirt.h | 60 ++++++++++
arch/riscv/kvm/Makefile | 1 +
arch/riscv/kvm/vcpu_sbi.c | 4 +
arch/riscv/kvm/vcpu_sbi_pvlock.c | 57 +++++++++
include/asm-generic/rwonce.h | 2 +
include/asm-generic/spinlock.h | 87 +-------------
include/asm-generic/spinlock_types.h | 12 +-
include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++
kernel/locking/qspinlock.c | 5 +-
32 files changed, 739 insertions(+), 114 deletions(-)
create mode 100644 arch/riscv/include/asm/qspinlock.h
create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
create mode 100644 arch/riscv/include/asm/rwonce.h
create mode 100644 arch/riscv/include/asm/spinlock.h
create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
create mode 100644 include/asm-generic/ticket_spinlock.h
--
2.36.1
^ permalink raw reply [flat|nested] 215+ messages in thread
* [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-10 8:28 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
patch[1 - 10]: Native qspinlock
patch[11 -17]: Paravirt qspinlock
patch[4]: Add prefetchw in qspinlock's xchg_tail when cpus >= 16k
This series based on:
- [RFC PATCH v5 0/5] Rework & improve riscv cmpxchg.h and atomic.h
https://lore.kernel.org/linux-riscv/20230810040349.92279-2-leobras@redhat.com/
- [PATCH V3] asm-generic: ticket-lock: Optimize arch_spin_value_unlocked
https://lore.kernel.org/linux-riscv/20230908154339.3250567-1-guoren@kernel.org/
I merge them into sg2042-master branch, then you could directly try it on
sg2042 hardware platform:
https://github.com/guoren83/linux/tree/sg2042-master-qspinlock-64ilp32_v5
Use sophgo_mango_ubuntu_defconfig for sg2042 64/128 cores hardware
platform.
Native qspinlock
================
This time we've proved the qspinlock on th1520 [1] & sg2042 [2], which
gives stability and performance improvement. All T-HEAD processors have
a strong LR/SC forward progress guarantee than the requirements of the
ISA, which could satisfy the xchg_tail of native_qspinlock. Now,
qspinlock has been run with us for more than 1 year, and we have enough
confidence to enable it for all the T-HEAD processors. Of causes, we
found a livelock problem with the qspinlock lock torture test from the
CPU store merge buffer delay mechanism, which caused the queued spinlock
becomes a dead ring and RCU warning to come out. We introduce a custom
WRITE_ONCE to solve this. Do we need explicit ISA instruction to signal
it? Or let hardware handle this.
We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
test on Fedora & Ubuntu & OpenEuler ... Here is the performance
comparison between qspinlock and ticket_lock on sg2042 (64 cores):
sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
queued_spinlock 0.5109/0.00
ticket_spinlock 0.5814/0.00
perf futex/hash (+6.7%):
queued_spinlock 1444393 operations/sec (+- 0.09%)
ticket_spinlock 1353215 operations/sec (+- 0.15%)
perf futex/wake-parallel (+8.6%):
queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
perf futex/requeue (+4.2%):
queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
System Benchmarks (+6.4%)
queued_spinlock:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
Double-Precision Whetstone 55.0 182422.8 33167.8
Execl Throughput 43.0 13116.6 3050.4
File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
Pipe Throughput 12440.0 23058600.5 18535.9
Pipe-based Context Switching 4000.0 2835617.7 7089.0
Process Creation 126.0 12537.3 995.0
Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
System Call Overhead 15000.0 33308301.3 22205.5
========
System Benchmarks Index Score 12426.1
ticket_spinlock:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
Double-Precision Whetstone 55.0 181921.0 33076.5
Execl Throughput 43.0 12625.1 2936.1
File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
Pipe Throughput 12440.0 20594018.7 16554.7
Pipe-based Context Switching 4000.0 2571117.7 6427.8
Process Creation 126.0 10798.4 857.0
Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
System Call Overhead 15000.0 30766778.4 20511.2
========
System Benchmarks Index Score 11670.7
The qspinlock has a significant improvement on SOPHGO SG2042 64
cores platform than the ticket_lock.
Paravirt qspinlock
==================
We implemented kvm_kick_cpu/kvm_wait_cpu and add tracepoints to observe the
behaviors. Also, introduce a new SBI extension SBI_EXT_PVLOCK (0xAB0401). If the
name and number are approved, I will send a formal proposal to the SBI spec.
Changlog:
V11:
- Based on Leonardo Bras's cmpxchg_small patches v5.
- Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
- Remove abusing alternative framework and use jump_label instead.
- Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
guarantee.
- Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
V10:
https://lore.kernel.org/linux-riscv/20230802164701.192791-1-guoren@kernel.org/
- Using an alternative framework instead of static_key_branch in the
asm/spinlock.h.
- Fixup store merge buffer problem, which causes qspinlock lock
torture test livelock.
- Add paravirt qspinlock support, include KVM backend
- Add Compact NUMA-awared qspinlock support
V9:
https://lore.kernel.org/linux-riscv/20220808071318.3335746-1-guoren@kernel.org/
- Cleanup generic ticket-lock code, (Using smp_mb__after_spinlock as
RCsc)
- Add qspinlock and combo-lock for riscv
- Add qspinlock to openrisc
- Use generic header in csky
- Optimize cmpxchg & atomic code
V8:
https://lore.kernel.org/linux-riscv/20220724122517.1019187-1-guoren@kernel.org/
- Coding convention ticket fixup
- Move combo spinlock into riscv and simply asm-generic/spinlock.h
- Fixup xchg16 with wrong return value
- Add csky qspinlock
- Add combo & qspinlock & ticket-lock comparison
- Clean up unnecessary riscv acquire and release definitions
- Enable ARCH_INLINE_READ*/WRITE*/SPIN* for riscv & csky
V7:
https://lore.kernel.org/linux-riscv/20220628081946.1999419-1-guoren@kernel.org/
- Add combo spinlock (ticket & queued) support
- Rename ticket_spinlock.h
- Remove unnecessary atomic_read in ticket_spin_value_unlocked
V6:
https://lore.kernel.org/linux-riscv/20220621144920.2945595-1-guoren@kernel.org/
- Fixup Clang compile problem Reported-by: kernel test robot
- Cleanup asm-generic/spinlock.h
- Remove changelog in patch main comment part, suggested by
Conor.Dooley
- Remove "default y if NUMA" in Kconfig
V5:
https://lore.kernel.org/linux-riscv/20220620155404.1968739-1-guoren@kernel.org/
- Update comment with RISC-V forward guarantee feature.
- Back to V3 direction and optimize asm code.
V4:
https://lore.kernel.org/linux-riscv/1616868399-82848-4-git-send-email-guoren@kernel.org/
- Remove custom sub-word xchg implementation
- Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 in locking/qspinlock
V3:
https://lore.kernel.org/linux-riscv/1616658937-82063-1-git-send-email-guoren@kernel.org/
- Coding convention by Peter Zijlstra's advices
V2:
https://lore.kernel.org/linux-riscv/1606225437-22948-2-git-send-email-guoren@kernel.org/
- Coding convention in cmpxchg.h
- Re-implement short xchg
- Remove char & cmpxchg implementations
V1:
https://lore.kernel.org/linux-riscv/20190211043829.30096-1-michaeljclark@mac.com/
- Using cmpxchg loop to implement sub-word atomic
Guo Ren (17):
asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
asm-generic: ticket-lock: Move into ticket_spinlock.h
riscv: Use Zicbop in arch_xchg when available
locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
riscv: qspinlock: Add basic queued_spinlock support
riscv: qspinlock: Introduce combo spinlock
riscv: qspinlock: Introduce qspinlock param for command line
riscv: qspinlock: Add virt_spin_lock() support for KVM guest
riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
RISC-V: paravirt: pvqspinlock: Add SBI implementation
RISC-V: paravirt: pvqspinlock: Add kconfig entry
RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
RISC-V: paravirt: pvqspinlock: KVM: Implement
kvm_sbi_ext_pvlock_kick_cpu()
.../admin-guide/kernel-parameters.txt | 8 +-
arch/riscv/Kconfig | 50 ++++++++
arch/riscv/Kconfig.errata | 19 +++
arch/riscv/errata/thead/errata.c | 29 +++++
arch/riscv/include/asm/Kbuild | 2 +-
arch/riscv/include/asm/cmpxchg.h | 4 +-
arch/riscv/include/asm/errata_list.h | 13 --
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/insn-def.h | 5 +
arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
arch/riscv/include/asm/processor.h | 13 ++
arch/riscv/include/asm/qspinlock.h | 35 ++++++
arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++
arch/riscv/include/asm/rwonce.h | 24 ++++
arch/riscv/include/asm/sbi.h | 14 +++
arch/riscv/include/asm/spinlock.h | 113 ++++++++++++++++++
arch/riscv/include/asm/vendorid_list.h | 14 +++
arch/riscv/include/uapi/asm/kvm.h | 1 +
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/cpufeature.c | 1 +
arch/riscv/kernel/qspinlock_paravirt.c | 83 +++++++++++++
arch/riscv/kernel/sbi.c | 2 +-
arch/riscv/kernel/setup.c | 60 ++++++++++
.../kernel/trace_events_filter_paravirt.h | 60 ++++++++++
arch/riscv/kvm/Makefile | 1 +
arch/riscv/kvm/vcpu_sbi.c | 4 +
arch/riscv/kvm/vcpu_sbi_pvlock.c | 57 +++++++++
include/asm-generic/rwonce.h | 2 +
include/asm-generic/spinlock.h | 87 +-------------
include/asm-generic/spinlock_types.h | 12 +-
include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++
kernel/locking/qspinlock.c | 5 +-
32 files changed, 739 insertions(+), 114 deletions(-)
create mode 100644 arch/riscv/include/asm/qspinlock.h
create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
create mode 100644 arch/riscv/include/asm/rwonce.h
create mode 100644 arch/riscv/include/asm/spinlock.h
create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
create mode 100644 include/asm-generic/ticket_spinlock.h
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* [PATCH V11 01/17] asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:28 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The arch_spinlock_t of qspinlock has contained the atomic_t val, which
satisfies the ticket-lock requirement. Thus, unify the arch_spinlock_t
into qspinlock_types.h. This is the preparation for the next combo
spinlock.
Signed-off-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
---
include/asm-generic/spinlock.h | 14 +++++++-------
include/asm-generic/spinlock_types.h | 12 ++----------
2 files changed, 9 insertions(+), 17 deletions(-)
diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
index 90803a826ba0..4773334ee638 100644
--- a/include/asm-generic/spinlock.h
+++ b/include/asm-generic/spinlock.h
@@ -32,7 +32,7 @@
static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
{
- u32 val = atomic_fetch_add(1<<16, lock);
+ u32 val = atomic_fetch_add(1<<16, &lock->val);
u16 ticket = val >> 16;
if (ticket == (u16)val)
@@ -46,31 +46,31 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
* have no outstanding writes due to the atomic_fetch_add() the extra
* orderings are free.
*/
- atomic_cond_read_acquire(lock, ticket == (u16)VAL);
+ atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
smp_mb();
}
static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
{
- u32 old = atomic_read(lock);
+ u32 old = atomic_read(&lock->val);
if ((old >> 16) != (old & 0xffff))
return false;
- return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */
+ return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
}
static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
{
u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
- u32 val = atomic_read(lock);
+ u32 val = atomic_read(&lock->val);
smp_store_release(ptr, (u16)val + 1);
}
static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
{
- u32 val = lock.counter;
+ u32 val = lock.val.counter;
return ((val >> 16) == (val & 0xffff));
}
@@ -84,7 +84,7 @@ static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
{
- u32 val = atomic_read(lock);
+ u32 val = atomic_read(&lock->val);
return (s16)((val >> 16) - (val & 0xffff)) > 1;
}
diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spinlock_types.h
index 8962bb730945..f534aa5de394 100644
--- a/include/asm-generic/spinlock_types.h
+++ b/include/asm-generic/spinlock_types.h
@@ -3,15 +3,7 @@
#ifndef __ASM_GENERIC_SPINLOCK_TYPES_H
#define __ASM_GENERIC_SPINLOCK_TYPES_H
-#include <linux/types.h>
-typedef atomic_t arch_spinlock_t;
-
-/*
- * qrwlock_types depends on arch_spinlock_t, so we must typedef that before the
- * include.
- */
-#include <asm/qrwlock_types.h>
-
-#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0)
+#include <asm-generic/qspinlock_types.h>
+#include <asm-generic/qrwlock_types.h>
#endif /* __ASM_GENERIC_SPINLOCK_TYPES_H */
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 01/17] asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
@ 2023-09-10 8:28 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The arch_spinlock_t of qspinlock has contained the atomic_t val, which
satisfies the ticket-lock requirement. Thus, unify the arch_spinlock_t
into qspinlock_types.h. This is the preparation for the next combo
spinlock.
Signed-off-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
---
include/asm-generic/spinlock.h | 14 +++++++-------
include/asm-generic/spinlock_types.h | 12 ++----------
2 files changed, 9 insertions(+), 17 deletions(-)
diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
index 90803a826ba0..4773334ee638 100644
--- a/include/asm-generic/spinlock.h
+++ b/include/asm-generic/spinlock.h
@@ -32,7 +32,7 @@
static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
{
- u32 val = atomic_fetch_add(1<<16, lock);
+ u32 val = atomic_fetch_add(1<<16, &lock->val);
u16 ticket = val >> 16;
if (ticket == (u16)val)
@@ -46,31 +46,31 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
* have no outstanding writes due to the atomic_fetch_add() the extra
* orderings are free.
*/
- atomic_cond_read_acquire(lock, ticket == (u16)VAL);
+ atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
smp_mb();
}
static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
{
- u32 old = atomic_read(lock);
+ u32 old = atomic_read(&lock->val);
if ((old >> 16) != (old & 0xffff))
return false;
- return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */
+ return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
}
static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
{
u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
- u32 val = atomic_read(lock);
+ u32 val = atomic_read(&lock->val);
smp_store_release(ptr, (u16)val + 1);
}
static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
{
- u32 val = lock.counter;
+ u32 val = lock.val.counter;
return ((val >> 16) == (val & 0xffff));
}
@@ -84,7 +84,7 @@ static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
{
- u32 val = atomic_read(lock);
+ u32 val = atomic_read(&lock->val);
return (s16)((val >> 16) - (val & 0xffff)) > 1;
}
diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spinlock_types.h
index 8962bb730945..f534aa5de394 100644
--- a/include/asm-generic/spinlock_types.h
+++ b/include/asm-generic/spinlock_types.h
@@ -3,15 +3,7 @@
#ifndef __ASM_GENERIC_SPINLOCK_TYPES_H
#define __ASM_GENERIC_SPINLOCK_TYPES_H
-#include <linux/types.h>
-typedef atomic_t arch_spinlock_t;
-
-/*
- * qrwlock_types depends on arch_spinlock_t, so we must typedef that before the
- * include.
- */
-#include <asm/qrwlock_types.h>
-
-#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0)
+#include <asm-generic/qspinlock_types.h>
+#include <asm-generic/qrwlock_types.h>
#endif /* __ASM_GENERIC_SPINLOCK_TYPES_H */
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 02/17] asm-generic: ticket-lock: Move into ticket_spinlock.h
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:28 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Move ticket-lock definition into an independent file. This is the
preparation for the next combo spinlock of riscv.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
include/asm-generic/spinlock.h | 87 +---------------------
include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++++++++++++
2 files changed, 104 insertions(+), 86 deletions(-)
create mode 100644 include/asm-generic/ticket_spinlock.h
diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
index 4773334ee638..970590baf61b 100644
--- a/include/asm-generic/spinlock.h
+++ b/include/asm-generic/spinlock.h
@@ -1,94 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * 'Generic' ticket-lock implementation.
- *
- * It relies on atomic_fetch_add() having well defined forward progress
- * guarantees under contention. If your architecture cannot provide this, stick
- * to a test-and-set lock.
- *
- * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a
- * sub-word of the value. This is generally true for anything LL/SC although
- * you'd be hard pressed to find anything useful in architecture specifications
- * about this. If your architecture cannot do this you might be better off with
- * a test-and-set.
- *
- * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence
- * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along with
- * a full fence after the spin to upgrade the otherwise-RCpc
- * atomic_cond_read_acquire().
- *
- * The implementation uses smp_cond_load_acquire() to spin, so if the
- * architecture has WFE like instructions to sleep instead of poll for word
- * modifications be sure to implement that (see ARM64 for example).
- *
- */
-
#ifndef __ASM_GENERIC_SPINLOCK_H
#define __ASM_GENERIC_SPINLOCK_H
-#include <linux/atomic.h>
-#include <asm-generic/spinlock_types.h>
-
-static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
-{
- u32 val = atomic_fetch_add(1<<16, &lock->val);
- u16 ticket = val >> 16;
-
- if (ticket == (u16)val)
- return;
-
- /*
- * atomic_cond_read_acquire() is RCpc, but rather than defining a
- * custom cond_read_rcsc() here we just emit a full fence. We only
- * need the prior reads before subsequent writes ordering from
- * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we
- * have no outstanding writes due to the atomic_fetch_add() the extra
- * orderings are free.
- */
- atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
- smp_mb();
-}
-
-static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
-{
- u32 old = atomic_read(&lock->val);
-
- if ((old >> 16) != (old & 0xffff))
- return false;
-
- return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
-}
-
-static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
-{
- u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
- u32 val = atomic_read(&lock->val);
-
- smp_store_release(ptr, (u16)val + 1);
-}
-
-static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
-{
- u32 val = lock.val.counter;
-
- return ((val >> 16) == (val & 0xffff));
-}
-
-static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
-{
- arch_spinlock_t val = READ_ONCE(*lock);
-
- return !arch_spin_value_unlocked(val);
-}
-
-static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
-{
- u32 val = atomic_read(&lock->val);
-
- return (s16)((val >> 16) - (val & 0xffff)) > 1;
-}
-
+#include <asm-generic/ticket_spinlock.h>
#include <asm/qrwlock.h>
#endif /* __ASM_GENERIC_SPINLOCK_H */
diff --git a/include/asm-generic/ticket_spinlock.h b/include/asm-generic/ticket_spinlock.h
new file mode 100644
index 000000000000..cfcff22b37b3
--- /dev/null
+++ b/include/asm-generic/ticket_spinlock.h
@@ -0,0 +1,103 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * 'Generic' ticket-lock implementation.
+ *
+ * It relies on atomic_fetch_add() having well defined forward progress
+ * guarantees under contention. If your architecture cannot provide this, stick
+ * to a test-and-set lock.
+ *
+ * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a
+ * sub-word of the value. This is generally true for anything LL/SC although
+ * you'd be hard pressed to find anything useful in architecture specifications
+ * about this. If your architecture cannot do this you might be better off with
+ * a test-and-set.
+ *
+ * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence
+ * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along with
+ * a full fence after the spin to upgrade the otherwise-RCpc
+ * atomic_cond_read_acquire().
+ *
+ * The implementation uses smp_cond_load_acquire() to spin, so if the
+ * architecture has WFE like instructions to sleep instead of poll for word
+ * modifications be sure to implement that (see ARM64 for example).
+ *
+ */
+
+#ifndef __ASM_GENERIC_TICKET_SPINLOCK_H
+#define __ASM_GENERIC_TICKET_SPINLOCK_H
+
+#include <linux/atomic.h>
+#include <asm-generic/spinlock_types.h>
+
+static __always_inline void ticket_spin_lock(arch_spinlock_t *lock)
+{
+ u32 val = atomic_fetch_add(1<<16, &lock->val);
+ u16 ticket = val >> 16;
+
+ if (ticket == (u16)val)
+ return;
+
+ /*
+ * atomic_cond_read_acquire() is RCpc, but rather than defining a
+ * custom cond_read_rcsc() here we just emit a full fence. We only
+ * need the prior reads before subsequent writes ordering from
+ * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we
+ * have no outstanding writes due to the atomic_fetch_add() the extra
+ * orderings are free.
+ */
+ atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
+ smp_mb();
+}
+
+static __always_inline bool ticket_spin_trylock(arch_spinlock_t *lock)
+{
+ u32 old = atomic_read(&lock->val);
+
+ if ((old >> 16) != (old & 0xffff))
+ return false;
+
+ return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
+}
+
+static __always_inline void ticket_spin_unlock(arch_spinlock_t *lock)
+{
+ u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
+ u32 val = atomic_read(&lock->val);
+
+ smp_store_release(ptr, (u16)val + 1);
+}
+
+static __always_inline int ticket_spin_value_unlocked(arch_spinlock_t lock)
+{
+ u32 val = lock.val.counter;
+
+ return ((val >> 16) == (val & 0xffff));
+}
+
+static __always_inline int ticket_spin_is_locked(arch_spinlock_t *lock)
+{
+ arch_spinlock_t val = READ_ONCE(*lock);
+
+ return !ticket_spin_value_unlocked(val);
+}
+
+static __always_inline int ticket_spin_is_contended(arch_spinlock_t *lock)
+{
+ u32 val = atomic_read(&lock->val);
+
+ return (s16)((val >> 16) - (val & 0xffff)) > 1;
+}
+
+/*
+ * Remapping spinlock architecture specific functions to the corresponding
+ * ticket spinlock functions.
+ */
+#define arch_spin_is_locked(l) ticket_spin_is_locked(l)
+#define arch_spin_is_contended(l) ticket_spin_is_contended(l)
+#define arch_spin_value_unlocked(l) ticket_spin_value_unlocked(l)
+#define arch_spin_lock(l) ticket_spin_lock(l)
+#define arch_spin_trylock(l) ticket_spin_trylock(l)
+#define arch_spin_unlock(l) ticket_spin_unlock(l)
+
+#endif /* __ASM_GENERIC_TICKET_SPINLOCK_H */
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 02/17] asm-generic: ticket-lock: Move into ticket_spinlock.h
@ 2023-09-10 8:28 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Move ticket-lock definition into an independent file. This is the
preparation for the next combo spinlock of riscv.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
include/asm-generic/spinlock.h | 87 +---------------------
include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++++++++++++
2 files changed, 104 insertions(+), 86 deletions(-)
create mode 100644 include/asm-generic/ticket_spinlock.h
diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
index 4773334ee638..970590baf61b 100644
--- a/include/asm-generic/spinlock.h
+++ b/include/asm-generic/spinlock.h
@@ -1,94 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * 'Generic' ticket-lock implementation.
- *
- * It relies on atomic_fetch_add() having well defined forward progress
- * guarantees under contention. If your architecture cannot provide this, stick
- * to a test-and-set lock.
- *
- * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a
- * sub-word of the value. This is generally true for anything LL/SC although
- * you'd be hard pressed to find anything useful in architecture specifications
- * about this. If your architecture cannot do this you might be better off with
- * a test-and-set.
- *
- * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence
- * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along with
- * a full fence after the spin to upgrade the otherwise-RCpc
- * atomic_cond_read_acquire().
- *
- * The implementation uses smp_cond_load_acquire() to spin, so if the
- * architecture has WFE like instructions to sleep instead of poll for word
- * modifications be sure to implement that (see ARM64 for example).
- *
- */
-
#ifndef __ASM_GENERIC_SPINLOCK_H
#define __ASM_GENERIC_SPINLOCK_H
-#include <linux/atomic.h>
-#include <asm-generic/spinlock_types.h>
-
-static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
-{
- u32 val = atomic_fetch_add(1<<16, &lock->val);
- u16 ticket = val >> 16;
-
- if (ticket == (u16)val)
- return;
-
- /*
- * atomic_cond_read_acquire() is RCpc, but rather than defining a
- * custom cond_read_rcsc() here we just emit a full fence. We only
- * need the prior reads before subsequent writes ordering from
- * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we
- * have no outstanding writes due to the atomic_fetch_add() the extra
- * orderings are free.
- */
- atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
- smp_mb();
-}
-
-static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
-{
- u32 old = atomic_read(&lock->val);
-
- if ((old >> 16) != (old & 0xffff))
- return false;
-
- return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
-}
-
-static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
-{
- u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
- u32 val = atomic_read(&lock->val);
-
- smp_store_release(ptr, (u16)val + 1);
-}
-
-static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
-{
- u32 val = lock.val.counter;
-
- return ((val >> 16) == (val & 0xffff));
-}
-
-static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
-{
- arch_spinlock_t val = READ_ONCE(*lock);
-
- return !arch_spin_value_unlocked(val);
-}
-
-static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
-{
- u32 val = atomic_read(&lock->val);
-
- return (s16)((val >> 16) - (val & 0xffff)) > 1;
-}
-
+#include <asm-generic/ticket_spinlock.h>
#include <asm/qrwlock.h>
#endif /* __ASM_GENERIC_SPINLOCK_H */
diff --git a/include/asm-generic/ticket_spinlock.h b/include/asm-generic/ticket_spinlock.h
new file mode 100644
index 000000000000..cfcff22b37b3
--- /dev/null
+++ b/include/asm-generic/ticket_spinlock.h
@@ -0,0 +1,103 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * 'Generic' ticket-lock implementation.
+ *
+ * It relies on atomic_fetch_add() having well defined forward progress
+ * guarantees under contention. If your architecture cannot provide this, stick
+ * to a test-and-set lock.
+ *
+ * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a
+ * sub-word of the value. This is generally true for anything LL/SC although
+ * you'd be hard pressed to find anything useful in architecture specifications
+ * about this. If your architecture cannot do this you might be better off with
+ * a test-and-set.
+ *
+ * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence
+ * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along with
+ * a full fence after the spin to upgrade the otherwise-RCpc
+ * atomic_cond_read_acquire().
+ *
+ * The implementation uses smp_cond_load_acquire() to spin, so if the
+ * architecture has WFE like instructions to sleep instead of poll for word
+ * modifications be sure to implement that (see ARM64 for example).
+ *
+ */
+
+#ifndef __ASM_GENERIC_TICKET_SPINLOCK_H
+#define __ASM_GENERIC_TICKET_SPINLOCK_H
+
+#include <linux/atomic.h>
+#include <asm-generic/spinlock_types.h>
+
+static __always_inline void ticket_spin_lock(arch_spinlock_t *lock)
+{
+ u32 val = atomic_fetch_add(1<<16, &lock->val);
+ u16 ticket = val >> 16;
+
+ if (ticket == (u16)val)
+ return;
+
+ /*
+ * atomic_cond_read_acquire() is RCpc, but rather than defining a
+ * custom cond_read_rcsc() here we just emit a full fence. We only
+ * need the prior reads before subsequent writes ordering from
+ * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we
+ * have no outstanding writes due to the atomic_fetch_add() the extra
+ * orderings are free.
+ */
+ atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
+ smp_mb();
+}
+
+static __always_inline bool ticket_spin_trylock(arch_spinlock_t *lock)
+{
+ u32 old = atomic_read(&lock->val);
+
+ if ((old >> 16) != (old & 0xffff))
+ return false;
+
+ return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
+}
+
+static __always_inline void ticket_spin_unlock(arch_spinlock_t *lock)
+{
+ u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
+ u32 val = atomic_read(&lock->val);
+
+ smp_store_release(ptr, (u16)val + 1);
+}
+
+static __always_inline int ticket_spin_value_unlocked(arch_spinlock_t lock)
+{
+ u32 val = lock.val.counter;
+
+ return ((val >> 16) == (val & 0xffff));
+}
+
+static __always_inline int ticket_spin_is_locked(arch_spinlock_t *lock)
+{
+ arch_spinlock_t val = READ_ONCE(*lock);
+
+ return !ticket_spin_value_unlocked(val);
+}
+
+static __always_inline int ticket_spin_is_contended(arch_spinlock_t *lock)
+{
+ u32 val = atomic_read(&lock->val);
+
+ return (s16)((val >> 16) - (val & 0xffff)) > 1;
+}
+
+/*
+ * Remapping spinlock architecture specific functions to the corresponding
+ * ticket spinlock functions.
+ */
+#define arch_spin_is_locked(l) ticket_spin_is_locked(l)
+#define arch_spin_is_contended(l) ticket_spin_is_contended(l)
+#define arch_spin_value_unlocked(l) ticket_spin_value_unlocked(l)
+#define arch_spin_lock(l) ticket_spin_lock(l)
+#define arch_spin_trylock(l) ticket_spin_trylock(l)
+#define arch_spin_unlock(l) ticket_spin_unlock(l)
+
+#endif /* __ASM_GENERIC_TICKET_SPINLOCK_H */
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-10 8:28 ` guoren
(?)
@ 2023-09-10 8:28 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Cache-block prefetch instructions are HINTs to the hardware to
indicate that software intends to perform a particular type of
memory access in the near future. Enable ARCH_HAS_PREFETCHW and
improve the arch_xchg for qspinlock xchg_tail.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 15 +++++++++++++++
arch/riscv/include/asm/cmpxchg.h | 4 +++-
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/insn-def.h | 5 +++++
arch/riscv/include/asm/processor.h | 13 +++++++++++++
arch/riscv/kernel/cpufeature.c | 1 +
6 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index e9ae6fa232c3..2c346fe169c1 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
If you don't know what to do here, say Y.
+config RISCV_ISA_ZICBOP
+ bool "Zicbop extension support for cache block prefetch"
+ depends on MMU
+ depends on RISCV_ALTERNATIVE
+ default y
+ help
+ Adds support to dynamically detect the presence of the ZICBOP
+ extension (Cache Block Prefetch Operations) and enable its
+ usage.
+
+ The Zicbop extension can be used to prefetch cache block for
+ read/write/instruction fetch.
+
+ If you don't know what to do here, say Y.
+
config TOOLCHAIN_HAS_ZIHINTPAUSE
bool
default y
diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
index 702725727671..56eff7a9d2d2 100644
--- a/arch/riscv/include/asm/cmpxchg.h
+++ b/arch/riscv/include/asm/cmpxchg.h
@@ -11,6 +11,7 @@
#include <asm/barrier.h>
#include <asm/fence.h>
+#include <asm/processor.h>
#define __arch_xchg_masked(prepend, append, r, p, n) \
({ \
@@ -25,6 +26,7 @@
\
__asm__ __volatile__ ( \
prepend \
+ PREFETCHW_ASM(%5) \
"0: lr.w %0, %2\n" \
" and %1, %0, %z4\n" \
" or %1, %1, %z3\n" \
@@ -32,7 +34,7 @@
" bnez %1, 0b\n" \
append \
: "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
- : "rJ" (__newx), "rJ" (~__mask) \
+ : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
: "memory"); \
\
r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
index b7b58258f6c7..78b7b8b53778 100644
--- a/arch/riscv/include/asm/hwcap.h
+++ b/arch/riscv/include/asm/hwcap.h
@@ -58,6 +58,7 @@
#define RISCV_ISA_EXT_ZICSR 40
#define RISCV_ISA_EXT_ZIFENCEI 41
#define RISCV_ISA_EXT_ZIHPM 42
+#define RISCV_ISA_EXT_ZICBOP 43
#define RISCV_ISA_EXT_MAX 64
diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
index 6960beb75f32..dc590d331894 100644
--- a/arch/riscv/include/asm/insn-def.h
+++ b/arch/riscv/include/asm/insn-def.h
@@ -134,6 +134,7 @@
#define RV_OPCODE_MISC_MEM RV_OPCODE(15)
#define RV_OPCODE_SYSTEM RV_OPCODE(115)
+#define RV_OPCODE_PREFETCH RV_OPCODE(19)
#define HFENCE_VVMA(vaddr, asid) \
INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
@@ -196,4 +197,8 @@
INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
RS1(base), SIMM12(4))
+#define CBO_prefetchw(base) \
+ INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
+ RD(x0), RS1(base), RS2(x0))
+
#endif /* __ASM_INSN_DEF_H */
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index de9da852f78d..7ad3a24212e8 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -12,6 +12,8 @@
#include <vdso/processor.h>
#include <asm/ptrace.h>
+#include <asm/insn-def.h>
+#include <asm/hwcap.h>
#ifdef CONFIG_64BIT
#define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
@@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
#define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
#define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
+#define ARCH_HAS_PREFETCHW
+#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
+ CBO_prefetchw(base), \
+ 0, \
+ RISCV_ISA_EXT_ZICBOP, \
+ CONFIG_RISCV_ISA_ZICBOP)
+static inline void prefetchw(const void *ptr)
+{
+ asm volatile(PREFETCHW_ASM(%0)
+ : : "r" (ptr) : "memory");
+}
/* Do necessary setup to start up a newly executed thread. */
extern void start_thread(struct pt_regs *regs,
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index ef7b4fd9e876..e0b897db0b97 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
__RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
__RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
__RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
+ __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
__RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
__RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
__RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-10 8:28 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Cache-block prefetch instructions are HINTs to the hardware to
indicate that software intends to perform a particular type of
memory access in the near future. Enable ARCH_HAS_PREFETCHW and
improve the arch_xchg for qspinlock xchg_tail.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 15 +++++++++++++++
arch/riscv/include/asm/cmpxchg.h | 4 +++-
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/insn-def.h | 5 +++++
arch/riscv/include/asm/processor.h | 13 +++++++++++++
arch/riscv/kernel/cpufeature.c | 1 +
6 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index e9ae6fa232c3..2c346fe169c1 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
If you don't know what to do here, say Y.
+config RISCV_ISA_ZICBOP
+ bool "Zicbop extension support for cache block prefetch"
+ depends on MMU
+ depends on RISCV_ALTERNATIVE
+ default y
+ help
+ Adds support to dynamically detect the presence of the ZICBOP
+ extension (Cache Block Prefetch Operations) and enable its
+ usage.
+
+ The Zicbop extension can be used to prefetch cache block for
+ read/write/instruction fetch.
+
+ If you don't know what to do here, say Y.
+
config TOOLCHAIN_HAS_ZIHINTPAUSE
bool
default y
diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
index 702725727671..56eff7a9d2d2 100644
--- a/arch/riscv/include/asm/cmpxchg.h
+++ b/arch/riscv/include/asm/cmpxchg.h
@@ -11,6 +11,7 @@
#include <asm/barrier.h>
#include <asm/fence.h>
+#include <asm/processor.h>
#define __arch_xchg_masked(prepend, append, r, p, n) \
({ \
@@ -25,6 +26,7 @@
\
__asm__ __volatile__ ( \
prepend \
+ PREFETCHW_ASM(%5) \
"0: lr.w %0, %2\n" \
" and %1, %0, %z4\n" \
" or %1, %1, %z3\n" \
@@ -32,7 +34,7 @@
" bnez %1, 0b\n" \
append \
: "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
- : "rJ" (__newx), "rJ" (~__mask) \
+ : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
: "memory"); \
\
r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
index b7b58258f6c7..78b7b8b53778 100644
--- a/arch/riscv/include/asm/hwcap.h
+++ b/arch/riscv/include/asm/hwcap.h
@@ -58,6 +58,7 @@
#define RISCV_ISA_EXT_ZICSR 40
#define RISCV_ISA_EXT_ZIFENCEI 41
#define RISCV_ISA_EXT_ZIHPM 42
+#define RISCV_ISA_EXT_ZICBOP 43
#define RISCV_ISA_EXT_MAX 64
diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
index 6960beb75f32..dc590d331894 100644
--- a/arch/riscv/include/asm/insn-def.h
+++ b/arch/riscv/include/asm/insn-def.h
@@ -134,6 +134,7 @@
#define RV_OPCODE_MISC_MEM RV_OPCODE(15)
#define RV_OPCODE_SYSTEM RV_OPCODE(115)
+#define RV_OPCODE_PREFETCH RV_OPCODE(19)
#define HFENCE_VVMA(vaddr, asid) \
INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
@@ -196,4 +197,8 @@
INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
RS1(base), SIMM12(4))
+#define CBO_prefetchw(base) \
+ INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
+ RD(x0), RS1(base), RS2(x0))
+
#endif /* __ASM_INSN_DEF_H */
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index de9da852f78d..7ad3a24212e8 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -12,6 +12,8 @@
#include <vdso/processor.h>
#include <asm/ptrace.h>
+#include <asm/insn-def.h>
+#include <asm/hwcap.h>
#ifdef CONFIG_64BIT
#define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
@@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
#define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
#define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
+#define ARCH_HAS_PREFETCHW
+#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
+ CBO_prefetchw(base), \
+ 0, \
+ RISCV_ISA_EXT_ZICBOP, \
+ CONFIG_RISCV_ISA_ZICBOP)
+static inline void prefetchw(const void *ptr)
+{
+ asm volatile(PREFETCHW_ASM(%0)
+ : : "r" (ptr) : "memory");
+}
/* Do necessary setup to start up a newly executed thread. */
extern void start_thread(struct pt_regs *regs,
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index ef7b4fd9e876..e0b897db0b97 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
__RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
__RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
__RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
+ __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
__RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
__RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
__RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:28 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The target of xchg_tail is to write the tail to the lock value, so
adding prefetchw could help the next cmpxchg step, which may
decrease the cmpxchg retry loops of xchg_tail. Some processors may
utilize this feature to give a forward guarantee, e.g., RISC-V
XuanTie processors would block the snoop channel & irq for several
cycles when prefetch.w instruction (from Zicbop extension) retired,
which guarantees the next cmpxchg succeeds.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
kernel/locking/qspinlock.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
index d3f99060b60f..96b54e2ade86 100644
--- a/kernel/locking/qspinlock.c
+++ b/kernel/locking/qspinlock.c
@@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
*/
static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
{
- u32 old, new, val = atomic_read(&lock->val);
+ u32 old, new, val;
+
+ prefetchw(&lock->val);
+ val = atomic_read(&lock->val);
for (;;) {
new = (val & _Q_LOCKED_PENDING_MASK) | tail;
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-10 8:28 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The target of xchg_tail is to write the tail to the lock value, so
adding prefetchw could help the next cmpxchg step, which may
decrease the cmpxchg retry loops of xchg_tail. Some processors may
utilize this feature to give a forward guarantee, e.g., RISC-V
XuanTie processors would block the snoop channel & irq for several
cycles when prefetch.w instruction (from Zicbop extension) retired,
which guarantees the next cmpxchg succeeds.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
kernel/locking/qspinlock.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
index d3f99060b60f..96b54e2ade86 100644
--- a/kernel/locking/qspinlock.c
+++ b/kernel/locking/qspinlock.c
@@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
*/
static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
{
- u32 old, new, val = atomic_read(&lock->val);
+ u32 old, new, val;
+
+ prefetchw(&lock->val);
+ val = atomic_read(&lock->val);
for (;;) {
new = (val & _Q_LOCKED_PENDING_MASK) | tail;
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:28 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The requirements of qspinlock have been documented by commit:
a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
atomics").
Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
doesn't satisfy the requirements of qspinlock above, it won't prevent
some riscv vendors from implementing a strong fwd guarantee LR/SC in
microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
is the one.
We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
test on Fedora & Ubuntu & OpenEuler ... Here is the performance
comparison between qspinlock and ticket_lock on sg2042 (64 cores):
sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
queued_spinlock 0.5109/0.00
ticket_spinlock 0.5814/0.00
perf futex/hash (+6.7%):
queued_spinlock 1444393 operations/sec (+- 0.09%)
ticket_spinlock 1353215 operations/sec (+- 0.15%)
perf futex/wake-parallel (+8.6%):
queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
perf futex/requeue (+4.2%):
queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
System Benchmarks (+6.4%)
queued_spinlock:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
Double-Precision Whetstone 55.0 182422.8 33167.8
Execl Throughput 43.0 13116.6 3050.4
File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
Pipe Throughput 12440.0 23058600.5 18535.9
Pipe-based Context Switching 4000.0 2835617.7 7089.0
Process Creation 126.0 12537.3 995.0
Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
System Call Overhead 15000.0 33308301.3 22205.5
========
System Benchmarks Index Score 12426.1
ticket_spinlock:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
Double-Precision Whetstone 55.0 181921.0 33076.5
Execl Throughput 43.0 12625.1 2936.1
File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
Pipe Throughput 12440.0 20594018.7 16554.7
Pipe-based Context Switching 4000.0 2571117.7 6427.8
Process Creation 126.0 10798.4 857.0
Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
System Call Overhead 15000.0 30766778.4 20511.2
========
System Benchmarks Index Score 11670.7
The qspinlock has a significant improvement on SOPHGO SG2042 64
cores platform than the ticket_lock.
Signed-off-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
---
arch/riscv/Kconfig | 16 ++++++++++++++++
arch/riscv/include/asm/Kbuild | 3 ++-
arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
3 files changed, 35 insertions(+), 1 deletion(-)
create mode 100644 arch/riscv/include/asm/spinlock.h
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 2c346fe169c1..7f39bfc75744 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -471,6 +471,22 @@ config NODES_SHIFT
Specify the maximum number of NUMA Nodes available on the target
system. Increases memory reserved to accommodate various tables.
+choice
+ prompt "RISC-V spinlock type"
+ default RISCV_TICKET_SPINLOCKS
+
+config RISCV_TICKET_SPINLOCKS
+ bool "Using ticket spinlock"
+
+config RISCV_QUEUED_SPINLOCKS
+ bool "Using queued spinlock"
+ depends on SMP && MMU
+ select ARCH_USE_QUEUED_SPINLOCKS
+ help
+ Make sure your micro arch LL/SC has a strong forward progress guarantee.
+ Otherwise, stay at ticket-lock.
+endchoice
+
config RISCV_ALTERNATIVE
bool
depends on !XIP_KERNEL
diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
index 504f8b7e72d4..a0dc85e4a754 100644
--- a/arch/riscv/include/asm/Kbuild
+++ b/arch/riscv/include/asm/Kbuild
@@ -2,10 +2,11 @@
generic-y += early_ioremap.h
generic-y += flat.h
generic-y += kvm_para.h
+generic-y += mcs_spinlock.h
generic-y += parport.h
-generic-y += spinlock.h
generic-y += spinlock_types.h
generic-y += qrwlock.h
generic-y += qrwlock_types.h
+generic-y += qspinlock.h
generic-y += user.h
generic-y += vmlinux.lds.h
diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
new file mode 100644
index 000000000000..c644a92d4548
--- /dev/null
+++ b/arch/riscv/include/asm/spinlock.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ASM_RISCV_SPINLOCK_H
+#define __ASM_RISCV_SPINLOCK_H
+
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#define _Q_PENDING_LOOPS (1 << 9)
+#endif
+
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm/qspinlock.h>
+#include <asm/qrwlock.h>
+#else
+#include <asm-generic/spinlock.h>
+#endif
+
+#endif /* __ASM_RISCV_SPINLOCK_H */
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
@ 2023-09-10 8:28 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:28 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The requirements of qspinlock have been documented by commit:
a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
atomics").
Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
doesn't satisfy the requirements of qspinlock above, it won't prevent
some riscv vendors from implementing a strong fwd guarantee LR/SC in
microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
is the one.
We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
test on Fedora & Ubuntu & OpenEuler ... Here is the performance
comparison between qspinlock and ticket_lock on sg2042 (64 cores):
sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
queued_spinlock 0.5109/0.00
ticket_spinlock 0.5814/0.00
perf futex/hash (+6.7%):
queued_spinlock 1444393 operations/sec (+- 0.09%)
ticket_spinlock 1353215 operations/sec (+- 0.15%)
perf futex/wake-parallel (+8.6%):
queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
perf futex/requeue (+4.2%):
queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
System Benchmarks (+6.4%)
queued_spinlock:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
Double-Precision Whetstone 55.0 182422.8 33167.8
Execl Throughput 43.0 13116.6 3050.4
File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
Pipe Throughput 12440.0 23058600.5 18535.9
Pipe-based Context Switching 4000.0 2835617.7 7089.0
Process Creation 126.0 12537.3 995.0
Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
System Call Overhead 15000.0 33308301.3 22205.5
========
System Benchmarks Index Score 12426.1
ticket_spinlock:
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
Double-Precision Whetstone 55.0 181921.0 33076.5
Execl Throughput 43.0 12625.1 2936.1
File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
Pipe Throughput 12440.0 20594018.7 16554.7
Pipe-based Context Switching 4000.0 2571117.7 6427.8
Process Creation 126.0 10798.4 857.0
Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
System Call Overhead 15000.0 30766778.4 20511.2
========
System Benchmarks Index Score 11670.7
The qspinlock has a significant improvement on SOPHGO SG2042 64
cores platform than the ticket_lock.
Signed-off-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
---
arch/riscv/Kconfig | 16 ++++++++++++++++
arch/riscv/include/asm/Kbuild | 3 ++-
arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
3 files changed, 35 insertions(+), 1 deletion(-)
create mode 100644 arch/riscv/include/asm/spinlock.h
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 2c346fe169c1..7f39bfc75744 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -471,6 +471,22 @@ config NODES_SHIFT
Specify the maximum number of NUMA Nodes available on the target
system. Increases memory reserved to accommodate various tables.
+choice
+ prompt "RISC-V spinlock type"
+ default RISCV_TICKET_SPINLOCKS
+
+config RISCV_TICKET_SPINLOCKS
+ bool "Using ticket spinlock"
+
+config RISCV_QUEUED_SPINLOCKS
+ bool "Using queued spinlock"
+ depends on SMP && MMU
+ select ARCH_USE_QUEUED_SPINLOCKS
+ help
+ Make sure your micro arch LL/SC has a strong forward progress guarantee.
+ Otherwise, stay at ticket-lock.
+endchoice
+
config RISCV_ALTERNATIVE
bool
depends on !XIP_KERNEL
diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
index 504f8b7e72d4..a0dc85e4a754 100644
--- a/arch/riscv/include/asm/Kbuild
+++ b/arch/riscv/include/asm/Kbuild
@@ -2,10 +2,11 @@
generic-y += early_ioremap.h
generic-y += flat.h
generic-y += kvm_para.h
+generic-y += mcs_spinlock.h
generic-y += parport.h
-generic-y += spinlock.h
generic-y += spinlock_types.h
generic-y += qrwlock.h
generic-y += qrwlock_types.h
+generic-y += qspinlock.h
generic-y += user.h
generic-y += vmlinux.lds.h
diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
new file mode 100644
index 000000000000..c644a92d4548
--- /dev/null
+++ b/arch/riscv/include/asm/spinlock.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ASM_RISCV_SPINLOCK_H
+#define __ASM_RISCV_SPINLOCK_H
+
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#define _Q_PENDING_LOOPS (1 << 9)
+#endif
+
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm/qspinlock.h>
+#include <asm/qrwlock.h>
+#else
+#include <asm-generic/spinlock.h>
+#endif
+
+#endif /* __ASM_RISCV_SPINLOCK_H */
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Combo spinlock could support queued and ticket in one Linux Image and
select them during boot time via errata mechanism. Here is the func
size (Bytes) comparison table below:
TYPE : COMBO | TICKET | QUEUED
arch_spin_lock : 106 | 60 | 50
arch_spin_unlock : 54 | 36 | 26
arch_spin_trylock : 110 | 72 | 54
arch_spin_is_locked : 48 | 34 | 20
arch_spin_is_contended : 56 | 40 | 24
rch_spin_value_unlocked : 48 | 34 | 24
One example of disassemble combo arch_spin_unlock:
0xffffffff8000409c <+14>: nop # detour slot
0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
0xffffffff800040a8 <+26>: ld s0,8(sp)
0xffffffff800040aa <+28>: addi sp,sp,16
0xffffffff800040ac <+30>: ret
0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
0xffffffff800040b0 <+34>: sext.w a5,a5
0xffffffff800040b2 <+36>: fence rw,w
0xffffffff800040b6 <+40>: addiw a5,a5,1
0xffffffff800040b8 <+42>: slli a5,a5,0x30
0xffffffff800040ba <+44>: srli a5,a5,0x30
0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
0xffffffff800040c0 <+50>: ld s0,8(sp)
0xffffffff800040c2 <+52>: addi sp,sp,16
0xffffffff800040c4 <+54>: ret
The qspinlock is smaller and faster than ticket-lock when all are in
fast-path, and combo spinlock could provide a compatible Linux Image
for different micro-arch design (weak/strict fwd guarantee LR/SC)
processors.
Signed-off-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
---
arch/riscv/Kconfig | 9 +++-
arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
arch/riscv/kernel/setup.c | 14 ++++++
3 files changed, 98 insertions(+), 3 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 7f39bfc75744..4bcff2860f48 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -473,7 +473,7 @@ config NODES_SHIFT
choice
prompt "RISC-V spinlock type"
- default RISCV_TICKET_SPINLOCKS
+ default RISCV_COMBO_SPINLOCKS
config RISCV_TICKET_SPINLOCKS
bool "Using ticket spinlock"
@@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
help
Make sure your micro arch LL/SC has a strong forward progress guarantee.
Otherwise, stay at ticket-lock.
+
+config RISCV_COMBO_SPINLOCKS
+ bool "Using combo spinlock"
+ depends on SMP && MMU
+ select ARCH_USE_QUEUED_SPINLOCKS
+ help
+ Select queued spinlock or ticket-lock via errata.
endchoice
config RISCV_ALTERNATIVE
diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
index c644a92d4548..8ea0fee80652 100644
--- a/arch/riscv/include/asm/spinlock.h
+++ b/arch/riscv/include/asm/spinlock.h
@@ -7,11 +7,85 @@
#define _Q_PENDING_LOOPS (1 << 9)
#endif
+#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
+#include <asm-generic/ticket_spinlock.h>
+
+#undef arch_spin_is_locked
+#undef arch_spin_is_contended
+#undef arch_spin_value_unlocked
+#undef arch_spin_lock
+#undef arch_spin_trylock
+#undef arch_spin_unlock
+
+#include <asm-generic/qspinlock.h>
+#include <linux/jump_label.h>
+
+#undef arch_spin_is_locked
+#undef arch_spin_is_contended
+#undef arch_spin_value_unlocked
+#undef arch_spin_lock
+#undef arch_spin_trylock
+#undef arch_spin_unlock
+
+DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
+
+static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ queued_spin_lock(lock);
+ else
+ ticket_spin_lock(lock);
+}
+
+static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ return queued_spin_trylock(lock);
+ else
+ return ticket_spin_trylock(lock);
+}
+
+static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ queued_spin_unlock(lock);
+ else
+ ticket_spin_unlock(lock);
+}
+
+static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ return queued_spin_value_unlocked(lock);
+ else
+ return ticket_spin_value_unlocked(lock);
+}
+
+static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ return queued_spin_is_locked(lock);
+ else
+ return ticket_spin_is_locked(lock);
+}
+
+static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ return queued_spin_is_contended(lock);
+ else
+ return ticket_spin_is_contended(lock);
+}
+#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
+
#ifdef CONFIG_QUEUED_SPINLOCKS
#include <asm/qspinlock.h>
-#include <asm/qrwlock.h>
#else
-#include <asm-generic/spinlock.h>
+#include <asm-generic/ticket_spinlock.h>
#endif
+#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
+
+#include <asm/qrwlock.h>
+
#endif /* __ASM_RISCV_SPINLOCK_H */
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 32c2e1eb71bd..a447cf360a18 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -269,6 +269,18 @@ static void __init parse_dtb(void)
#endif
}
+#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
+DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
+EXPORT_SYMBOL(combo_qspinlock_key);
+#endif
+
+static void __init riscv_spinlock_init(void)
+{
+#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
+ static_branch_disable(&combo_qspinlock_key);
+#endif
+}
+
extern void __init init_rt_signal_env(void);
void __init setup_arch(char **cmdline_p)
@@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
riscv_isa_extension_available(NULL, ZICBOM))
riscv_noncoherent_supported();
riscv_set_dma_cache_alignment();
+
+ riscv_spinlock_init();
}
static int __init topology_init(void)
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Combo spinlock could support queued and ticket in one Linux Image and
select them during boot time via errata mechanism. Here is the func
size (Bytes) comparison table below:
TYPE : COMBO | TICKET | QUEUED
arch_spin_lock : 106 | 60 | 50
arch_spin_unlock : 54 | 36 | 26
arch_spin_trylock : 110 | 72 | 54
arch_spin_is_locked : 48 | 34 | 20
arch_spin_is_contended : 56 | 40 | 24
rch_spin_value_unlocked : 48 | 34 | 24
One example of disassemble combo arch_spin_unlock:
0xffffffff8000409c <+14>: nop # detour slot
0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
0xffffffff800040a8 <+26>: ld s0,8(sp)
0xffffffff800040aa <+28>: addi sp,sp,16
0xffffffff800040ac <+30>: ret
0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
0xffffffff800040b0 <+34>: sext.w a5,a5
0xffffffff800040b2 <+36>: fence rw,w
0xffffffff800040b6 <+40>: addiw a5,a5,1
0xffffffff800040b8 <+42>: slli a5,a5,0x30
0xffffffff800040ba <+44>: srli a5,a5,0x30
0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
0xffffffff800040c0 <+50>: ld s0,8(sp)
0xffffffff800040c2 <+52>: addi sp,sp,16
0xffffffff800040c4 <+54>: ret
The qspinlock is smaller and faster than ticket-lock when all are in
fast-path, and combo spinlock could provide a compatible Linux Image
for different micro-arch design (weak/strict fwd guarantee LR/SC)
processors.
Signed-off-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
---
arch/riscv/Kconfig | 9 +++-
arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
arch/riscv/kernel/setup.c | 14 ++++++
3 files changed, 98 insertions(+), 3 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 7f39bfc75744..4bcff2860f48 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -473,7 +473,7 @@ config NODES_SHIFT
choice
prompt "RISC-V spinlock type"
- default RISCV_TICKET_SPINLOCKS
+ default RISCV_COMBO_SPINLOCKS
config RISCV_TICKET_SPINLOCKS
bool "Using ticket spinlock"
@@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
help
Make sure your micro arch LL/SC has a strong forward progress guarantee.
Otherwise, stay at ticket-lock.
+
+config RISCV_COMBO_SPINLOCKS
+ bool "Using combo spinlock"
+ depends on SMP && MMU
+ select ARCH_USE_QUEUED_SPINLOCKS
+ help
+ Select queued spinlock or ticket-lock via errata.
endchoice
config RISCV_ALTERNATIVE
diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
index c644a92d4548..8ea0fee80652 100644
--- a/arch/riscv/include/asm/spinlock.h
+++ b/arch/riscv/include/asm/spinlock.h
@@ -7,11 +7,85 @@
#define _Q_PENDING_LOOPS (1 << 9)
#endif
+#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
+#include <asm-generic/ticket_spinlock.h>
+
+#undef arch_spin_is_locked
+#undef arch_spin_is_contended
+#undef arch_spin_value_unlocked
+#undef arch_spin_lock
+#undef arch_spin_trylock
+#undef arch_spin_unlock
+
+#include <asm-generic/qspinlock.h>
+#include <linux/jump_label.h>
+
+#undef arch_spin_is_locked
+#undef arch_spin_is_contended
+#undef arch_spin_value_unlocked
+#undef arch_spin_lock
+#undef arch_spin_trylock
+#undef arch_spin_unlock
+
+DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
+
+static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ queued_spin_lock(lock);
+ else
+ ticket_spin_lock(lock);
+}
+
+static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ return queued_spin_trylock(lock);
+ else
+ return ticket_spin_trylock(lock);
+}
+
+static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ queued_spin_unlock(lock);
+ else
+ ticket_spin_unlock(lock);
+}
+
+static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ return queued_spin_value_unlocked(lock);
+ else
+ return ticket_spin_value_unlocked(lock);
+}
+
+static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ return queued_spin_is_locked(lock);
+ else
+ return ticket_spin_is_locked(lock);
+}
+
+static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
+{
+ if (static_branch_likely(&combo_qspinlock_key))
+ return queued_spin_is_contended(lock);
+ else
+ return ticket_spin_is_contended(lock);
+}
+#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
+
#ifdef CONFIG_QUEUED_SPINLOCKS
#include <asm/qspinlock.h>
-#include <asm/qrwlock.h>
#else
-#include <asm-generic/spinlock.h>
+#include <asm-generic/ticket_spinlock.h>
#endif
+#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
+
+#include <asm/qrwlock.h>
+
#endif /* __ASM_RISCV_SPINLOCK_H */
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 32c2e1eb71bd..a447cf360a18 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -269,6 +269,18 @@ static void __init parse_dtb(void)
#endif
}
+#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
+DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
+EXPORT_SYMBOL(combo_qspinlock_key);
+#endif
+
+static void __init riscv_spinlock_init(void)
+{
+#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
+ static_branch_disable(&combo_qspinlock_key);
+#endif
+}
+
extern void __init init_rt_signal_env(void);
void __init setup_arch(char **cmdline_p)
@@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
riscv_isa_extension_available(NULL, ZICBOM))
riscv_noncoherent_supported();
riscv_set_dma_cache_alignment();
+
+ riscv_spinlock_init();
}
static int __init topology_init(void)
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Allow cmdline to force the kernel to use queued_spinlock when
CONFIG_RISCV_COMBO_SPINLOCKS=y.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
Documentation/admin-guide/kernel-parameters.txt | 2 ++
arch/riscv/kernel/setup.c | 16 +++++++++++++++-
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 7dfb540c4f6c..61cacb8dfd0e 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -4693,6 +4693,8 @@
[KNL] Number of legacy pty's. Overwrites compiled-in
default number.
+ qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
+
qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
Set the time threshold in nanoseconds for the
number of intra-node lock hand-offs before the
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index a447cf360a18..0f084f037651 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -270,6 +270,15 @@ static void __init parse_dtb(void)
}
#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
+bool enable_qspinlock_key = false;
+static int __init queued_spinlock_setup(char *p)
+{
+ enable_qspinlock_key = true;
+
+ return 0;
+}
+early_param("qspinlock", queued_spinlock_setup);
+
DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
EXPORT_SYMBOL(combo_qspinlock_key);
#endif
@@ -277,7 +286,12 @@ EXPORT_SYMBOL(combo_qspinlock_key);
static void __init riscv_spinlock_init(void)
{
#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
- static_branch_disable(&combo_qspinlock_key);
+ if (!enable_qspinlock_key) {
+ static_branch_disable(&combo_qspinlock_key);
+ pr_info("Ticket spinlock: enabled\n");
+ } else {
+ pr_info("Queued spinlock: enabled\n");
+ }
#endif
}
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Allow cmdline to force the kernel to use queued_spinlock when
CONFIG_RISCV_COMBO_SPINLOCKS=y.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
Documentation/admin-guide/kernel-parameters.txt | 2 ++
arch/riscv/kernel/setup.c | 16 +++++++++++++++-
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 7dfb540c4f6c..61cacb8dfd0e 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -4693,6 +4693,8 @@
[KNL] Number of legacy pty's. Overwrites compiled-in
default number.
+ qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
+
qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
Set the time threshold in nanoseconds for the
number of intra-node lock hand-offs before the
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index a447cf360a18..0f084f037651 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -270,6 +270,15 @@ static void __init parse_dtb(void)
}
#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
+bool enable_qspinlock_key = false;
+static int __init queued_spinlock_setup(char *p)
+{
+ enable_qspinlock_key = true;
+
+ return 0;
+}
+early_param("qspinlock", queued_spinlock_setup);
+
DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
EXPORT_SYMBOL(combo_qspinlock_key);
#endif
@@ -277,7 +286,12 @@ EXPORT_SYMBOL(combo_qspinlock_key);
static void __init riscv_spinlock_init(void)
{
#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
- static_branch_disable(&combo_qspinlock_key);
+ if (!enable_qspinlock_key) {
+ static_branch_disable(&combo_qspinlock_key);
+ pr_info("Ticket spinlock: enabled\n");
+ } else {
+ pr_info("Queued spinlock: enabled\n");
+ }
#endif
}
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Add a static key controlling whether virt_spin_lock() should be
called or not. When running on bare metal set the new key to
false.
The KVM guests fall back to a Test-and-Set spinlock, because fair
locks have horrible lock 'holder' preemption issues. The
virt_spin_lock_key would shortcut for the
queued_spin_lock_slowpath() function that allow virt_spin_lock to
hijack it.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
.../admin-guide/kernel-parameters.txt | 4 +++
arch/riscv/include/asm/sbi.h | 8 +++++
arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
arch/riscv/kernel/sbi.c | 2 +-
arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
5 files changed, 64 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 61cacb8dfd0e..f75bedc50e00 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3927,6 +3927,10 @@
no_uaccess_flush
[PPC] Don't flush the L1-D cache after accessing user data.
+ no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
+ native_queued_spinlock when the nopvspin option is enabled.
+ This would help vcpu=pcpu scenarios.
+
novmcoredd [KNL,KDUMP]
Disable device dump. Device dump allows drivers to
append dump data to vmcore so you can collect driver
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 501e06e52078..e0233b3d7a5f 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
SBI_EXT_BASE_GET_MIMPID,
};
+enum sbi_ext_base_impl_id {
+ SBI_EXT_BASE_IMPL_ID_BBL = 0,
+ SBI_EXT_BASE_IMPL_ID_OPENSBI,
+ SBI_EXT_BASE_IMPL_ID_XVISOR,
+ SBI_EXT_BASE_IMPL_ID_KVM,
+};
+
enum sbi_ext_time_fid {
SBI_EXT_TIME_SET_TIMER = 0,
};
@@ -269,6 +276,7 @@ int sbi_console_getchar(void);
long sbi_get_mvendorid(void);
long sbi_get_marchid(void);
long sbi_get_mimpid(void);
+long sbi_get_firmware_id(void);
void sbi_set_timer(uint64_t stime_value);
void sbi_shutdown(void);
void sbi_send_ipi(unsigned int cpu);
diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
index 8ea0fee80652..6b38d6616f14 100644
--- a/arch/riscv/include/asm/spinlock.h
+++ b/arch/riscv/include/asm/spinlock.h
@@ -4,6 +4,28 @@
#define __ASM_RISCV_SPINLOCK_H
#ifdef CONFIG_QUEUED_SPINLOCKS
+/*
+ * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
+ * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
+ * would shortcut for the queued_spin_lock_slowpath() function that allow
+ * virt_spin_lock to hijack it.
+ */
+DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
+
+#define virt_spin_lock virt_spin_lock
+static inline bool virt_spin_lock(struct qspinlock *lock)
+{
+ if (!static_branch_likely(&virt_spin_lock_key))
+ return false;
+
+ do {
+ while (atomic_read(&lock->val) != 0)
+ cpu_relax();
+ } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
+
+ return true;
+}
+
#define _Q_PENDING_LOOPS (1 << 9)
#endif
diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
index 88eea3a99ee0..cdd45edc8db4 100644
--- a/arch/riscv/kernel/sbi.c
+++ b/arch/riscv/kernel/sbi.c
@@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
}
-static inline long sbi_get_firmware_id(void)
+long sbi_get_firmware_id(void)
{
return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
}
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 0f084f037651..c57d15b05160 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -26,6 +26,7 @@
#include <asm/alternative.h>
#include <asm/cacheflush.h>
#include <asm/cpu_ops.h>
+#include <asm/cpufeature.h>
#include <asm/early_ioremap.h>
#include <asm/pgtable.h>
#include <asm/setup.h>
@@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
EXPORT_SYMBOL(combo_qspinlock_key);
#endif
+#ifdef CONFIG_QUEUED_SPINLOCKS
+static bool no_virt_spin_key = false;
+DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
+
+static int __init no_virt_spin_setup(char *p)
+{
+ no_virt_spin_key = true;
+
+ return 0;
+}
+early_param("no_virt_spin", no_virt_spin_setup);
+
+static void __init virt_spin_lock_init(void)
+{
+ if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
+ no_virt_spin_key)
+ static_branch_disable(&virt_spin_lock_key);
+ else
+ pr_info("Enable virt_spin_lock\n");
+}
+#endif
+
static void __init riscv_spinlock_init(void)
{
#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
- if (!enable_qspinlock_key) {
+ if (!enable_qspinlock_key &&
+ (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
static_branch_disable(&combo_qspinlock_key);
pr_info("Ticket spinlock: enabled\n");
} else {
pr_info("Queued spinlock: enabled\n");
}
#endif
+
+#ifdef CONFIG_QUEUED_SPINLOCKS
+ virt_spin_lock_init();
+#endif
}
extern void __init init_rt_signal_env(void);
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Add a static key controlling whether virt_spin_lock() should be
called or not. When running on bare metal set the new key to
false.
The KVM guests fall back to a Test-and-Set spinlock, because fair
locks have horrible lock 'holder' preemption issues. The
virt_spin_lock_key would shortcut for the
queued_spin_lock_slowpath() function that allow virt_spin_lock to
hijack it.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
.../admin-guide/kernel-parameters.txt | 4 +++
arch/riscv/include/asm/sbi.h | 8 +++++
arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
arch/riscv/kernel/sbi.c | 2 +-
arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
5 files changed, 64 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 61cacb8dfd0e..f75bedc50e00 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3927,6 +3927,10 @@
no_uaccess_flush
[PPC] Don't flush the L1-D cache after accessing user data.
+ no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
+ native_queued_spinlock when the nopvspin option is enabled.
+ This would help vcpu=pcpu scenarios.
+
novmcoredd [KNL,KDUMP]
Disable device dump. Device dump allows drivers to
append dump data to vmcore so you can collect driver
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 501e06e52078..e0233b3d7a5f 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
SBI_EXT_BASE_GET_MIMPID,
};
+enum sbi_ext_base_impl_id {
+ SBI_EXT_BASE_IMPL_ID_BBL = 0,
+ SBI_EXT_BASE_IMPL_ID_OPENSBI,
+ SBI_EXT_BASE_IMPL_ID_XVISOR,
+ SBI_EXT_BASE_IMPL_ID_KVM,
+};
+
enum sbi_ext_time_fid {
SBI_EXT_TIME_SET_TIMER = 0,
};
@@ -269,6 +276,7 @@ int sbi_console_getchar(void);
long sbi_get_mvendorid(void);
long sbi_get_marchid(void);
long sbi_get_mimpid(void);
+long sbi_get_firmware_id(void);
void sbi_set_timer(uint64_t stime_value);
void sbi_shutdown(void);
void sbi_send_ipi(unsigned int cpu);
diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
index 8ea0fee80652..6b38d6616f14 100644
--- a/arch/riscv/include/asm/spinlock.h
+++ b/arch/riscv/include/asm/spinlock.h
@@ -4,6 +4,28 @@
#define __ASM_RISCV_SPINLOCK_H
#ifdef CONFIG_QUEUED_SPINLOCKS
+/*
+ * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
+ * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
+ * would shortcut for the queued_spin_lock_slowpath() function that allow
+ * virt_spin_lock to hijack it.
+ */
+DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
+
+#define virt_spin_lock virt_spin_lock
+static inline bool virt_spin_lock(struct qspinlock *lock)
+{
+ if (!static_branch_likely(&virt_spin_lock_key))
+ return false;
+
+ do {
+ while (atomic_read(&lock->val) != 0)
+ cpu_relax();
+ } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
+
+ return true;
+}
+
#define _Q_PENDING_LOOPS (1 << 9)
#endif
diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
index 88eea3a99ee0..cdd45edc8db4 100644
--- a/arch/riscv/kernel/sbi.c
+++ b/arch/riscv/kernel/sbi.c
@@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
}
-static inline long sbi_get_firmware_id(void)
+long sbi_get_firmware_id(void)
{
return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
}
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 0f084f037651..c57d15b05160 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -26,6 +26,7 @@
#include <asm/alternative.h>
#include <asm/cacheflush.h>
#include <asm/cpu_ops.h>
+#include <asm/cpufeature.h>
#include <asm/early_ioremap.h>
#include <asm/pgtable.h>
#include <asm/setup.h>
@@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
EXPORT_SYMBOL(combo_qspinlock_key);
#endif
+#ifdef CONFIG_QUEUED_SPINLOCKS
+static bool no_virt_spin_key = false;
+DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
+
+static int __init no_virt_spin_setup(char *p)
+{
+ no_virt_spin_key = true;
+
+ return 0;
+}
+early_param("no_virt_spin", no_virt_spin_setup);
+
+static void __init virt_spin_lock_init(void)
+{
+ if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
+ no_virt_spin_key)
+ static_branch_disable(&virt_spin_lock_key);
+ else
+ pr_info("Enable virt_spin_lock\n");
+}
+#endif
+
static void __init riscv_spinlock_init(void)
{
#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
- if (!enable_qspinlock_key) {
+ if (!enable_qspinlock_key &&
+ (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
static_branch_disable(&combo_qspinlock_key);
pr_info("Ticket spinlock: enabled\n");
} else {
pr_info("Queued spinlock: enabled\n");
}
#endif
+
+#ifdef CONFIG_QUEUED_SPINLOCKS
+ virt_spin_lock_init();
+#endif
}
extern void __init init_rt_signal_env(void);
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 09/17] riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The early version of T-Head C9xx cores has a store merge buffer
delay problem. The store merge buffer could improve the store queue
performance by merging multi-store requests, but when there are not
continued store requests, the prior single store request would be
waiting in the store queue for a long time. That would cause
significant problems for communication between multi-cores. This
problem was found on sg2042 & th1520 platforms with the qspinlock
lock torture test.
So appending a fence w.o could immediately flush the store merge
buffer and let other cores see the write result.
This will apply the WRITE_ONCE errata to handle the non-standard
behavior via appending a fence w.o instruction for WRITE_ONCE().
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig.errata | 19 +++++++++++++++++++
arch/riscv/errata/thead/errata.c | 20 ++++++++++++++++++++
arch/riscv/include/asm/errata_list.h | 13 -------------
arch/riscv/include/asm/rwonce.h | 24 ++++++++++++++++++++++++
arch/riscv/include/asm/vendorid_list.h | 14 ++++++++++++++
include/asm-generic/rwonce.h | 2 ++
6 files changed, 79 insertions(+), 13 deletions(-)
create mode 100644 arch/riscv/include/asm/rwonce.h
diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata
index 1aa85a427ff3..c919cc3f1a3a 100644
--- a/arch/riscv/Kconfig.errata
+++ b/arch/riscv/Kconfig.errata
@@ -77,4 +77,23 @@ config ERRATA_THEAD_PMU
If you don't know what to do here, say "Y".
+config ERRATA_THEAD_WRITE_ONCE
+ bool "Apply T-Head WRITE_ONCE errata"
+ depends on ERRATA_THEAD
+ default y
+ help
+ The early version of T-Head C9xx cores has a store merge buffer
+ delay problem. The store merge buffer could improve the store queue
+ performance by merging multi-store requests, but when there are no
+ continued store requests, the prior single store request would be
+ waiting in the store queue for a long time. That would cause
+ significant problems for communication between multi-cores. Appending
+ a fence w.o could immediately flush the store merge buffer and let
+ other cores see the write result.
+
+ This will apply the WRITE_ONCE errata to handle the non-standard
+ behavior via appending a fence w.o instruction for WRITE_ONCE().
+
+ If you don't know what to do here, say "Y".
+
endmenu # "CPU errata selection"
diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
index be84b14f0118..751eb5a7f614 100644
--- a/arch/riscv/errata/thead/errata.c
+++ b/arch/riscv/errata/thead/errata.c
@@ -69,6 +69,23 @@ static bool errata_probe_pmu(unsigned int stage,
return true;
}
+static bool errata_probe_write_once(unsigned int stage,
+ unsigned long arch_id, unsigned long impid)
+{
+ if (!IS_ENABLED(CONFIG_ERRATA_THEAD_WRITE_ONCE))
+ return false;
+
+ /* target-c9xx cores report arch_id and impid as 0 */
+ if (arch_id != 0 || impid != 0)
+ return false;
+
+ if (stage == RISCV_ALTERNATIVES_BOOT ||
+ stage == RISCV_ALTERNATIVES_MODULE)
+ return true;
+
+ return false;
+}
+
static u32 thead_errata_probe(unsigned int stage,
unsigned long archid, unsigned long impid)
{
@@ -83,6 +100,9 @@ static u32 thead_errata_probe(unsigned int stage,
if (errata_probe_pmu(stage, archid, impid))
cpu_req_errata |= BIT(ERRATA_THEAD_PMU);
+ if (errata_probe_write_once(stage, archid, impid))
+ cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
+
return cpu_req_errata;
}
diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
index 712cab7adffe..fbb2b8d39321 100644
--- a/arch/riscv/include/asm/errata_list.h
+++ b/arch/riscv/include/asm/errata_list.h
@@ -11,19 +11,6 @@
#include <asm/hwcap.h>
#include <asm/vendorid_list.h>
-#ifdef CONFIG_ERRATA_SIFIVE
-#define ERRATA_SIFIVE_CIP_453 0
-#define ERRATA_SIFIVE_CIP_1200 1
-#define ERRATA_SIFIVE_NUMBER 2
-#endif
-
-#ifdef CONFIG_ERRATA_THEAD
-#define ERRATA_THEAD_PBMT 0
-#define ERRATA_THEAD_CMO 1
-#define ERRATA_THEAD_PMU 2
-#define ERRATA_THEAD_NUMBER 3
-#endif
-
#ifdef __ASSEMBLY__
#define ALT_INSN_FAULT(x) \
diff --git a/arch/riscv/include/asm/rwonce.h b/arch/riscv/include/asm/rwonce.h
new file mode 100644
index 000000000000..be0b8864969d
--- /dev/null
+++ b/arch/riscv/include/asm/rwonce.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ASM_RWONCE_H
+#define __ASM_RWONCE_H
+
+#include <linux/compiler_types.h>
+#include <asm/alternative-macros.h>
+#include <asm/vendorid_list.h>
+
+#define __WRITE_ONCE(x, val) \
+do { \
+ *(volatile typeof(x) *)&(x) = (val); \
+ asm volatile(ALTERNATIVE( \
+ __nops(1), \
+ "fence w, o\n\t", \
+ THEAD_VENDOR_ID, \
+ ERRATA_THEAD_WRITE_ONCE, \
+ CONFIG_ERRATA_THEAD_WRITE_ONCE) \
+ : : : "memory"); \
+} while (0)
+
+#include <asm-generic/rwonce.h>
+
+#endif /* __ASM_RWONCE_H */
diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
index cb89af3f0704..73078cfe4029 100644
--- a/arch/riscv/include/asm/vendorid_list.h
+++ b/arch/riscv/include/asm/vendorid_list.h
@@ -8,4 +8,18 @@
#define SIFIVE_VENDOR_ID 0x489
#define THEAD_VENDOR_ID 0x5b7
+#ifdef CONFIG_ERRATA_SIFIVE
+#define ERRATA_SIFIVE_CIP_453 0
+#define ERRATA_SIFIVE_CIP_1200 1
+#define ERRATA_SIFIVE_NUMBER 2
+#endif
+
+#ifdef CONFIG_ERRATA_THEAD
+#define ERRATA_THEAD_PBMT 0
+#define ERRATA_THEAD_CMO 1
+#define ERRATA_THEAD_PMU 2
+#define ERRATA_THEAD_WRITE_ONCE 3
+#define ERRATA_THEAD_NUMBER 4
+#endif
+
#endif
diff --git a/include/asm-generic/rwonce.h b/include/asm-generic/rwonce.h
index 8d0a6280e982..fb07fe8c6e45 100644
--- a/include/asm-generic/rwonce.h
+++ b/include/asm-generic/rwonce.h
@@ -50,10 +50,12 @@
__READ_ONCE(x); \
})
+#ifndef __WRITE_ONCE
#define __WRITE_ONCE(x, val) \
do { \
*(volatile typeof(x) *)&(x) = (val); \
} while (0)
+#endif
#define WRITE_ONCE(x, val) \
do { \
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 09/17] riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
The early version of T-Head C9xx cores has a store merge buffer
delay problem. The store merge buffer could improve the store queue
performance by merging multi-store requests, but when there are not
continued store requests, the prior single store request would be
waiting in the store queue for a long time. That would cause
significant problems for communication between multi-cores. This
problem was found on sg2042 & th1520 platforms with the qspinlock
lock torture test.
So appending a fence w.o could immediately flush the store merge
buffer and let other cores see the write result.
This will apply the WRITE_ONCE errata to handle the non-standard
behavior via appending a fence w.o instruction for WRITE_ONCE().
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig.errata | 19 +++++++++++++++++++
arch/riscv/errata/thead/errata.c | 20 ++++++++++++++++++++
arch/riscv/include/asm/errata_list.h | 13 -------------
arch/riscv/include/asm/rwonce.h | 24 ++++++++++++++++++++++++
arch/riscv/include/asm/vendorid_list.h | 14 ++++++++++++++
include/asm-generic/rwonce.h | 2 ++
6 files changed, 79 insertions(+), 13 deletions(-)
create mode 100644 arch/riscv/include/asm/rwonce.h
diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata
index 1aa85a427ff3..c919cc3f1a3a 100644
--- a/arch/riscv/Kconfig.errata
+++ b/arch/riscv/Kconfig.errata
@@ -77,4 +77,23 @@ config ERRATA_THEAD_PMU
If you don't know what to do here, say "Y".
+config ERRATA_THEAD_WRITE_ONCE
+ bool "Apply T-Head WRITE_ONCE errata"
+ depends on ERRATA_THEAD
+ default y
+ help
+ The early version of T-Head C9xx cores has a store merge buffer
+ delay problem. The store merge buffer could improve the store queue
+ performance by merging multi-store requests, but when there are no
+ continued store requests, the prior single store request would be
+ waiting in the store queue for a long time. That would cause
+ significant problems for communication between multi-cores. Appending
+ a fence w.o could immediately flush the store merge buffer and let
+ other cores see the write result.
+
+ This will apply the WRITE_ONCE errata to handle the non-standard
+ behavior via appending a fence w.o instruction for WRITE_ONCE().
+
+ If you don't know what to do here, say "Y".
+
endmenu # "CPU errata selection"
diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
index be84b14f0118..751eb5a7f614 100644
--- a/arch/riscv/errata/thead/errata.c
+++ b/arch/riscv/errata/thead/errata.c
@@ -69,6 +69,23 @@ static bool errata_probe_pmu(unsigned int stage,
return true;
}
+static bool errata_probe_write_once(unsigned int stage,
+ unsigned long arch_id, unsigned long impid)
+{
+ if (!IS_ENABLED(CONFIG_ERRATA_THEAD_WRITE_ONCE))
+ return false;
+
+ /* target-c9xx cores report arch_id and impid as 0 */
+ if (arch_id != 0 || impid != 0)
+ return false;
+
+ if (stage == RISCV_ALTERNATIVES_BOOT ||
+ stage == RISCV_ALTERNATIVES_MODULE)
+ return true;
+
+ return false;
+}
+
static u32 thead_errata_probe(unsigned int stage,
unsigned long archid, unsigned long impid)
{
@@ -83,6 +100,9 @@ static u32 thead_errata_probe(unsigned int stage,
if (errata_probe_pmu(stage, archid, impid))
cpu_req_errata |= BIT(ERRATA_THEAD_PMU);
+ if (errata_probe_write_once(stage, archid, impid))
+ cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
+
return cpu_req_errata;
}
diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
index 712cab7adffe..fbb2b8d39321 100644
--- a/arch/riscv/include/asm/errata_list.h
+++ b/arch/riscv/include/asm/errata_list.h
@@ -11,19 +11,6 @@
#include <asm/hwcap.h>
#include <asm/vendorid_list.h>
-#ifdef CONFIG_ERRATA_SIFIVE
-#define ERRATA_SIFIVE_CIP_453 0
-#define ERRATA_SIFIVE_CIP_1200 1
-#define ERRATA_SIFIVE_NUMBER 2
-#endif
-
-#ifdef CONFIG_ERRATA_THEAD
-#define ERRATA_THEAD_PBMT 0
-#define ERRATA_THEAD_CMO 1
-#define ERRATA_THEAD_PMU 2
-#define ERRATA_THEAD_NUMBER 3
-#endif
-
#ifdef __ASSEMBLY__
#define ALT_INSN_FAULT(x) \
diff --git a/arch/riscv/include/asm/rwonce.h b/arch/riscv/include/asm/rwonce.h
new file mode 100644
index 000000000000..be0b8864969d
--- /dev/null
+++ b/arch/riscv/include/asm/rwonce.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ASM_RWONCE_H
+#define __ASM_RWONCE_H
+
+#include <linux/compiler_types.h>
+#include <asm/alternative-macros.h>
+#include <asm/vendorid_list.h>
+
+#define __WRITE_ONCE(x, val) \
+do { \
+ *(volatile typeof(x) *)&(x) = (val); \
+ asm volatile(ALTERNATIVE( \
+ __nops(1), \
+ "fence w, o\n\t", \
+ THEAD_VENDOR_ID, \
+ ERRATA_THEAD_WRITE_ONCE, \
+ CONFIG_ERRATA_THEAD_WRITE_ONCE) \
+ : : : "memory"); \
+} while (0)
+
+#include <asm-generic/rwonce.h>
+
+#endif /* __ASM_RWONCE_H */
diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
index cb89af3f0704..73078cfe4029 100644
--- a/arch/riscv/include/asm/vendorid_list.h
+++ b/arch/riscv/include/asm/vendorid_list.h
@@ -8,4 +8,18 @@
#define SIFIVE_VENDOR_ID 0x489
#define THEAD_VENDOR_ID 0x5b7
+#ifdef CONFIG_ERRATA_SIFIVE
+#define ERRATA_SIFIVE_CIP_453 0
+#define ERRATA_SIFIVE_CIP_1200 1
+#define ERRATA_SIFIVE_NUMBER 2
+#endif
+
+#ifdef CONFIG_ERRATA_THEAD
+#define ERRATA_THEAD_PBMT 0
+#define ERRATA_THEAD_CMO 1
+#define ERRATA_THEAD_PMU 2
+#define ERRATA_THEAD_WRITE_ONCE 3
+#define ERRATA_THEAD_NUMBER 4
+#endif
+
#endif
diff --git a/include/asm-generic/rwonce.h b/include/asm-generic/rwonce.h
index 8d0a6280e982..fb07fe8c6e45 100644
--- a/include/asm-generic/rwonce.h
+++ b/include/asm-generic/rwonce.h
@@ -50,10 +50,12 @@
__READ_ONCE(x); \
})
+#ifndef __WRITE_ONCE
#define __WRITE_ONCE(x, val) \
do { \
*(volatile typeof(x) *)&(x) = (val); \
} while (0)
+#endif
#define WRITE_ONCE(x, val) \
do { \
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 10/17] riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
According to qspinlock requirements, RISC-V gives out a weak LR/SC
forward progress guarantee which does not satisfy qspinlock. But
many vendors could produce stronger forward guarantee LR/SC to
ensure the xchg_tail could be finished in time on any kind of
hart. T-HEAD is the vendor which implements strong forward
guarantee LR/SC instruction pairs, so enable qspinlock for T-HEAD
with errata init help.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/errata/thead/errata.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
index 751eb5a7f614..0df6a67302c0 100644
--- a/arch/riscv/errata/thead/errata.c
+++ b/arch/riscv/errata/thead/errata.c
@@ -86,6 +86,13 @@ static bool errata_probe_write_once(unsigned int stage,
return false;
}
+extern bool enable_qspinlock_key;
+static void errata_probe_qspinlock(unsigned int stage)
+{
+ if (stage == RISCV_ALTERNATIVES_BOOT)
+ enable_qspinlock_key = true;
+}
+
static u32 thead_errata_probe(unsigned int stage,
unsigned long archid, unsigned long impid)
{
@@ -103,6 +110,8 @@ static u32 thead_errata_probe(unsigned int stage,
if (errata_probe_write_once(stage, archid, impid))
cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
+ errata_probe_qspinlock(stage);
+
return cpu_req_errata;
}
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 10/17] riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
According to qspinlock requirements, RISC-V gives out a weak LR/SC
forward progress guarantee which does not satisfy qspinlock. But
many vendors could produce stronger forward guarantee LR/SC to
ensure the xchg_tail could be finished in time on any kind of
hart. T-HEAD is the vendor which implements strong forward
guarantee LR/SC instruction pairs, so enable qspinlock for T-HEAD
with errata init help.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/errata/thead/errata.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
index 751eb5a7f614..0df6a67302c0 100644
--- a/arch/riscv/errata/thead/errata.c
+++ b/arch/riscv/errata/thead/errata.c
@@ -86,6 +86,13 @@ static bool errata_probe_write_once(unsigned int stage,
return false;
}
+extern bool enable_qspinlock_key;
+static void errata_probe_qspinlock(unsigned int stage)
+{
+ if (stage == RISCV_ALTERNATIVES_BOOT)
+ enable_qspinlock_key = true;
+}
+
static u32 thead_errata_probe(unsigned int stage,
unsigned long archid, unsigned long impid)
{
@@ -103,6 +110,8 @@ static u32 thead_errata_probe(unsigned int stage,
if (errata_probe_write_once(stage, archid, impid))
cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
+ errata_probe_qspinlock(stage);
+
return cpu_req_errata;
}
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 11/17] RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Using static_call to switch between:
native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath()
native_queued_spin_unlock() __pv_queued_spin_unlock()
Finish the pv_wait implementation, but pv_kick needs the SBI
definition of the next patches.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/Kbuild | 1 -
arch/riscv/include/asm/qspinlock.h | 35 +++++++++++++
arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++++++++
arch/riscv/include/asm/spinlock.h | 2 +-
arch/riscv/kernel/qspinlock_paravirt.c | 57 +++++++++++++++++++++
arch/riscv/kernel/setup.c | 4 ++
6 files changed, 126 insertions(+), 2 deletions(-)
create mode 100644 arch/riscv/include/asm/qspinlock.h
create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
index a0dc85e4a754..b89cb3b73c13 100644
--- a/arch/riscv/include/asm/Kbuild
+++ b/arch/riscv/include/asm/Kbuild
@@ -7,6 +7,5 @@ generic-y += parport.h
generic-y += spinlock_types.h
generic-y += qrwlock.h
generic-y += qrwlock_types.h
-generic-y += qspinlock.h
generic-y += user.h
generic-y += vmlinux.lds.h
diff --git a/arch/riscv/include/asm/qspinlock.h b/arch/riscv/include/asm/qspinlock.h
new file mode 100644
index 000000000000..7d4f416c908c
--- /dev/null
+++ b/arch/riscv/include/asm/qspinlock.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c), 2023 Alibaba Cloud
+ * Authors:
+ * Guo Ren <guoren@linux.alibaba.com>
+ */
+
+#ifndef _ASM_RISCV_QSPINLOCK_H
+#define _ASM_RISCV_QSPINLOCK_H
+
+#ifdef CONFIG_PARAVIRT_SPINLOCKS
+#include <asm/qspinlock_paravirt.h>
+
+/* How long a lock should spin before we consider blocking */
+#define SPIN_THRESHOLD (1 << 15)
+
+void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
+void __pv_init_lock_hash(void);
+void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
+
+static inline void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
+{
+ static_call(pv_queued_spin_lock_slowpath)(lock, val);
+}
+
+#define queued_spin_unlock queued_spin_unlock
+static inline void queued_spin_unlock(struct qspinlock *lock)
+{
+ static_call(pv_queued_spin_unlock)(lock);
+}
+#endif /* CONFIG_PARAVIRT_SPINLOCKS */
+
+#include <asm-generic/qspinlock.h>
+
+#endif /* _ASM_RISCV_QSPINLOCK_H */
diff --git a/arch/riscv/include/asm/qspinlock_paravirt.h b/arch/riscv/include/asm/qspinlock_paravirt.h
new file mode 100644
index 000000000000..9681e851f69d
--- /dev/null
+++ b/arch/riscv/include/asm/qspinlock_paravirt.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c), 2023 Alibaba Cloud
+ * Authors:
+ * Guo Ren <guoren@linux.alibaba.com>
+ */
+
+#ifndef _ASM_RISCV_QSPINLOCK_PARAVIRT_H
+#define _ASM_RISCV_QSPINLOCK_PARAVIRT_H
+
+void pv_wait(u8 *ptr, u8 val);
+void pv_kick(int cpu);
+
+void dummy_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
+void dummy_queued_spin_unlock(struct qspinlock *lock);
+
+DECLARE_STATIC_CALL(pv_queued_spin_lock_slowpath, dummy_queued_spin_lock_slowpath);
+DECLARE_STATIC_CALL(pv_queued_spin_unlock, dummy_queued_spin_unlock);
+
+void __init pv_qspinlock_init(void);
+
+static inline bool pv_is_native_spin_unlock(void)
+{
+ return false;
+}
+
+void __pv_queued_spin_unlock(struct qspinlock *lock);
+
+#endif /* _ASM_RISCV_QSPINLOCK_PARAVIRT_H */
diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
index 6b38d6616f14..ed4253f491fe 100644
--- a/arch/riscv/include/asm/spinlock.h
+++ b/arch/riscv/include/asm/spinlock.h
@@ -39,7 +39,7 @@ static inline bool virt_spin_lock(struct qspinlock *lock)
#undef arch_spin_trylock
#undef arch_spin_unlock
-#include <asm-generic/qspinlock.h>
+#include <asm/qspinlock.h>
#include <linux/jump_label.h>
#undef arch_spin_is_locked
diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
new file mode 100644
index 000000000000..85ff5a3ec234
--- /dev/null
+++ b/arch/riscv/kernel/qspinlock_paravirt.c
@@ -0,0 +1,57 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c), 2023 Alibaba Cloud
+ * Authors:
+ * Guo Ren <guoren@linux.alibaba.com>
+ */
+
+#include <linux/static_call.h>
+#include <asm/qspinlock_paravirt.h>
+#include <asm/sbi.h>
+
+void pv_kick(int cpu)
+{
+ return;
+}
+
+void pv_wait(u8 *ptr, u8 val)
+{
+ unsigned long flags;
+
+ if (in_nmi())
+ return;
+
+ local_irq_save(flags);
+ if (READ_ONCE(*ptr) != val)
+ goto out;
+
+ /* wait_for_interrupt(); */
+out:
+ local_irq_restore(flags);
+}
+
+static void native_queued_spin_unlock(struct qspinlock *lock)
+{
+ smp_store_release(&lock->locked, 0);
+}
+
+DEFINE_STATIC_CALL(pv_queued_spin_lock_slowpath, native_queued_spin_lock_slowpath);
+EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
+
+DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
+EXPORT_STATIC_CALL(pv_queued_spin_unlock);
+
+void __init pv_qspinlock_init(void)
+{
+ if (num_possible_cpus() == 1)
+ return;
+
+ if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
+ return;
+
+ pr_info("PV qspinlocks enabled\n");
+ __pv_init_lock_hash();
+
+ static_call_update(pv_queued_spin_lock_slowpath, __pv_queued_spin_lock_slowpath);
+ static_call_update(pv_queued_spin_unlock, __pv_queued_spin_unlock);
+}
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index c57d15b05160..88690751f2ee 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -321,6 +321,10 @@ static void __init riscv_spinlock_init(void)
#ifdef CONFIG_QUEUED_SPINLOCKS
virt_spin_lock_init();
#endif
+
+#ifdef CONFIG_PARAVIRT_SPINLOCKS
+ pv_qspinlock_init();
+#endif
}
extern void __init init_rt_signal_env(void);
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 11/17] RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Using static_call to switch between:
native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath()
native_queued_spin_unlock() __pv_queued_spin_unlock()
Finish the pv_wait implementation, but pv_kick needs the SBI
definition of the next patches.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/Kbuild | 1 -
arch/riscv/include/asm/qspinlock.h | 35 +++++++++++++
arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++++++++
arch/riscv/include/asm/spinlock.h | 2 +-
arch/riscv/kernel/qspinlock_paravirt.c | 57 +++++++++++++++++++++
arch/riscv/kernel/setup.c | 4 ++
6 files changed, 126 insertions(+), 2 deletions(-)
create mode 100644 arch/riscv/include/asm/qspinlock.h
create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
index a0dc85e4a754..b89cb3b73c13 100644
--- a/arch/riscv/include/asm/Kbuild
+++ b/arch/riscv/include/asm/Kbuild
@@ -7,6 +7,5 @@ generic-y += parport.h
generic-y += spinlock_types.h
generic-y += qrwlock.h
generic-y += qrwlock_types.h
-generic-y += qspinlock.h
generic-y += user.h
generic-y += vmlinux.lds.h
diff --git a/arch/riscv/include/asm/qspinlock.h b/arch/riscv/include/asm/qspinlock.h
new file mode 100644
index 000000000000..7d4f416c908c
--- /dev/null
+++ b/arch/riscv/include/asm/qspinlock.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c), 2023 Alibaba Cloud
+ * Authors:
+ * Guo Ren <guoren@linux.alibaba.com>
+ */
+
+#ifndef _ASM_RISCV_QSPINLOCK_H
+#define _ASM_RISCV_QSPINLOCK_H
+
+#ifdef CONFIG_PARAVIRT_SPINLOCKS
+#include <asm/qspinlock_paravirt.h>
+
+/* How long a lock should spin before we consider blocking */
+#define SPIN_THRESHOLD (1 << 15)
+
+void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
+void __pv_init_lock_hash(void);
+void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
+
+static inline void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
+{
+ static_call(pv_queued_spin_lock_slowpath)(lock, val);
+}
+
+#define queued_spin_unlock queued_spin_unlock
+static inline void queued_spin_unlock(struct qspinlock *lock)
+{
+ static_call(pv_queued_spin_unlock)(lock);
+}
+#endif /* CONFIG_PARAVIRT_SPINLOCKS */
+
+#include <asm-generic/qspinlock.h>
+
+#endif /* _ASM_RISCV_QSPINLOCK_H */
diff --git a/arch/riscv/include/asm/qspinlock_paravirt.h b/arch/riscv/include/asm/qspinlock_paravirt.h
new file mode 100644
index 000000000000..9681e851f69d
--- /dev/null
+++ b/arch/riscv/include/asm/qspinlock_paravirt.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c), 2023 Alibaba Cloud
+ * Authors:
+ * Guo Ren <guoren@linux.alibaba.com>
+ */
+
+#ifndef _ASM_RISCV_QSPINLOCK_PARAVIRT_H
+#define _ASM_RISCV_QSPINLOCK_PARAVIRT_H
+
+void pv_wait(u8 *ptr, u8 val);
+void pv_kick(int cpu);
+
+void dummy_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
+void dummy_queued_spin_unlock(struct qspinlock *lock);
+
+DECLARE_STATIC_CALL(pv_queued_spin_lock_slowpath, dummy_queued_spin_lock_slowpath);
+DECLARE_STATIC_CALL(pv_queued_spin_unlock, dummy_queued_spin_unlock);
+
+void __init pv_qspinlock_init(void);
+
+static inline bool pv_is_native_spin_unlock(void)
+{
+ return false;
+}
+
+void __pv_queued_spin_unlock(struct qspinlock *lock);
+
+#endif /* _ASM_RISCV_QSPINLOCK_PARAVIRT_H */
diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
index 6b38d6616f14..ed4253f491fe 100644
--- a/arch/riscv/include/asm/spinlock.h
+++ b/arch/riscv/include/asm/spinlock.h
@@ -39,7 +39,7 @@ static inline bool virt_spin_lock(struct qspinlock *lock)
#undef arch_spin_trylock
#undef arch_spin_unlock
-#include <asm-generic/qspinlock.h>
+#include <asm/qspinlock.h>
#include <linux/jump_label.h>
#undef arch_spin_is_locked
diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
new file mode 100644
index 000000000000..85ff5a3ec234
--- /dev/null
+++ b/arch/riscv/kernel/qspinlock_paravirt.c
@@ -0,0 +1,57 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c), 2023 Alibaba Cloud
+ * Authors:
+ * Guo Ren <guoren@linux.alibaba.com>
+ */
+
+#include <linux/static_call.h>
+#include <asm/qspinlock_paravirt.h>
+#include <asm/sbi.h>
+
+void pv_kick(int cpu)
+{
+ return;
+}
+
+void pv_wait(u8 *ptr, u8 val)
+{
+ unsigned long flags;
+
+ if (in_nmi())
+ return;
+
+ local_irq_save(flags);
+ if (READ_ONCE(*ptr) != val)
+ goto out;
+
+ /* wait_for_interrupt(); */
+out:
+ local_irq_restore(flags);
+}
+
+static void native_queued_spin_unlock(struct qspinlock *lock)
+{
+ smp_store_release(&lock->locked, 0);
+}
+
+DEFINE_STATIC_CALL(pv_queued_spin_lock_slowpath, native_queued_spin_lock_slowpath);
+EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
+
+DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
+EXPORT_STATIC_CALL(pv_queued_spin_unlock);
+
+void __init pv_qspinlock_init(void)
+{
+ if (num_possible_cpus() == 1)
+ return;
+
+ if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
+ return;
+
+ pr_info("PV qspinlocks enabled\n");
+ __pv_init_lock_hash();
+
+ static_call_update(pv_queued_spin_lock_slowpath, __pv_queued_spin_lock_slowpath);
+ static_call_update(pv_queued_spin_unlock, __pv_queued_spin_unlock);
+}
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index c57d15b05160..88690751f2ee 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -321,6 +321,10 @@ static void __init riscv_spinlock_init(void)
#ifdef CONFIG_QUEUED_SPINLOCKS
virt_spin_lock_init();
#endif
+
+#ifdef CONFIG_PARAVIRT_SPINLOCKS
+ pv_qspinlock_init();
+#endif
}
extern void __init init_rt_signal_env(void);
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 12/17] RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Disables the qspinlock slow path using PV optimizations which
allow the hypervisor to 'idle' the guest on lock contention.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
Documentation/admin-guide/kernel-parameters.txt | 2 +-
arch/riscv/kernel/qspinlock_paravirt.c | 13 +++++++++++++
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index f75bedc50e00..e74aed631573 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3857,7 +3857,7 @@
as generic guest with no PV drivers. Currently support
XEN HVM, KVM, HYPER_V and VMWARE guest.
- nopvspin [X86,XEN,KVM]
+ nopvspin [X86,XEN,KVM,RISC-V]
Disables the qspinlock slow path using PV optimizations
which allow the hypervisor to 'idle' the guest on lock
contention.
diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
index 85ff5a3ec234..a0ad4657f437 100644
--- a/arch/riscv/kernel/qspinlock_paravirt.c
+++ b/arch/riscv/kernel/qspinlock_paravirt.c
@@ -41,8 +41,21 @@ EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
EXPORT_STATIC_CALL(pv_queued_spin_unlock);
+static bool nopvspin;
+static __init int parse_nopvspin(char *arg)
+{
+ nopvspin = true;
+ return 0;
+}
+early_param("nopvspin", parse_nopvspin);
+
void __init pv_qspinlock_init(void)
{
+ if (nopvspin) {
+ pr_info("PV qspinlocks disabled\n");
+ return;
+ }
+
if (num_possible_cpus() == 1)
return;
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 12/17] RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Disables the qspinlock slow path using PV optimizations which
allow the hypervisor to 'idle' the guest on lock contention.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
Documentation/admin-guide/kernel-parameters.txt | 2 +-
arch/riscv/kernel/qspinlock_paravirt.c | 13 +++++++++++++
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index f75bedc50e00..e74aed631573 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3857,7 +3857,7 @@
as generic guest with no PV drivers. Currently support
XEN HVM, KVM, HYPER_V and VMWARE guest.
- nopvspin [X86,XEN,KVM]
+ nopvspin [X86,XEN,KVM,RISC-V]
Disables the qspinlock slow path using PV optimizations
which allow the hypervisor to 'idle' the guest on lock
contention.
diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
index 85ff5a3ec234..a0ad4657f437 100644
--- a/arch/riscv/kernel/qspinlock_paravirt.c
+++ b/arch/riscv/kernel/qspinlock_paravirt.c
@@ -41,8 +41,21 @@ EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
EXPORT_STATIC_CALL(pv_queued_spin_unlock);
+static bool nopvspin;
+static __init int parse_nopvspin(char *arg)
+{
+ nopvspin = true;
+ return 0;
+}
+early_param("nopvspin", parse_nopvspin);
+
void __init pv_qspinlock_init(void)
{
+ if (nopvspin) {
+ pr_info("PV qspinlocks disabled\n");
+ return;
+ }
+
if (num_possible_cpus() == 1)
return;
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 13/17] RISC-V: paravirt: pvqspinlock: Add SBI implementation
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Implement pv_kick with SBI implementation, and add SBI_EXT_PVLOCK
extension detection.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/sbi.h | 6 ++++++
arch/riscv/kernel/qspinlock_paravirt.c | 7 ++++++-
2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index e0233b3d7a5f..3533f8d4f3e2 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -30,6 +30,7 @@ enum sbi_ext_id {
SBI_EXT_HSM = 0x48534D,
SBI_EXT_SRST = 0x53525354,
SBI_EXT_PMU = 0x504D55,
+ SBI_EXT_PVLOCK = 0xAB0401,
/* Experimentals extensions must lie within this range */
SBI_EXT_EXPERIMENTAL_START = 0x08000000,
@@ -243,6 +244,11 @@ enum sbi_pmu_ctr_type {
/* Flags defined for counter stop function */
#define SBI_PMU_STOP_FLAG_RESET (1 << 0)
+/* SBI PVLOCK (kick cpu out of wfi) */
+enum sbi_ext_pvlock_fid {
+ SBI_EXT_PVLOCK_KICK_CPU = 0,
+};
+
#define SBI_SPEC_VERSION_DEFAULT 0x1
#define SBI_SPEC_VERSION_MAJOR_SHIFT 24
#define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
index a0ad4657f437..571626f350be 100644
--- a/arch/riscv/kernel/qspinlock_paravirt.c
+++ b/arch/riscv/kernel/qspinlock_paravirt.c
@@ -11,6 +11,8 @@
void pv_kick(int cpu)
{
+ sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
+ cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
return;
}
@@ -25,7 +27,7 @@ void pv_wait(u8 *ptr, u8 val)
if (READ_ONCE(*ptr) != val)
goto out;
- /* wait_for_interrupt(); */
+ wait_for_interrupt();
out:
local_irq_restore(flags);
}
@@ -62,6 +64,9 @@ void __init pv_qspinlock_init(void)
if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
return;
+ if (!sbi_probe_extension(SBI_EXT_PVLOCK))
+ return;
+
pr_info("PV qspinlocks enabled\n");
__pv_init_lock_hash();
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 13/17] RISC-V: paravirt: pvqspinlock: Add SBI implementation
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Implement pv_kick with SBI implementation, and add SBI_EXT_PVLOCK
extension detection.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/sbi.h | 6 ++++++
arch/riscv/kernel/qspinlock_paravirt.c | 7 ++++++-
2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index e0233b3d7a5f..3533f8d4f3e2 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -30,6 +30,7 @@ enum sbi_ext_id {
SBI_EXT_HSM = 0x48534D,
SBI_EXT_SRST = 0x53525354,
SBI_EXT_PMU = 0x504D55,
+ SBI_EXT_PVLOCK = 0xAB0401,
/* Experimentals extensions must lie within this range */
SBI_EXT_EXPERIMENTAL_START = 0x08000000,
@@ -243,6 +244,11 @@ enum sbi_pmu_ctr_type {
/* Flags defined for counter stop function */
#define SBI_PMU_STOP_FLAG_RESET (1 << 0)
+/* SBI PVLOCK (kick cpu out of wfi) */
+enum sbi_ext_pvlock_fid {
+ SBI_EXT_PVLOCK_KICK_CPU = 0,
+};
+
#define SBI_SPEC_VERSION_DEFAULT 0x1
#define SBI_SPEC_VERSION_MAJOR_SHIFT 24
#define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
index a0ad4657f437..571626f350be 100644
--- a/arch/riscv/kernel/qspinlock_paravirt.c
+++ b/arch/riscv/kernel/qspinlock_paravirt.c
@@ -11,6 +11,8 @@
void pv_kick(int cpu)
{
+ sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
+ cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
return;
}
@@ -25,7 +27,7 @@ void pv_wait(u8 *ptr, u8 val)
if (READ_ONCE(*ptr) != val)
goto out;
- /* wait_for_interrupt(); */
+ wait_for_interrupt();
out:
local_irq_restore(flags);
}
@@ -62,6 +64,9 @@ void __init pv_qspinlock_init(void)
if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
return;
+ if (!sbi_probe_extension(SBI_EXT_PVLOCK))
+ return;
+
pr_info("PV qspinlocks enabled\n");
__pv_init_lock_hash();
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 14/17] RISC-V: paravirt: pvqspinlock: Add kconfig entry
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Add kconfig entry for paravirt_spinlock, an unfair qspinlock
virtualization-friendly backend, by halting the virtual CPU rather
than spinning.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 12 ++++++++++++
arch/riscv/kernel/Makefile | 1 +
2 files changed, 13 insertions(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 4bcff2860f48..ec0da24ed6fb 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -813,6 +813,18 @@ config RELOCATABLE
If unsure, say N.
+config PARAVIRT_SPINLOCKS
+ bool "Paravirtualization layer for spinlocks"
+ depends on QUEUED_SPINLOCKS
+ default y
+ help
+ Paravirtualized spinlocks allow a unfair qspinlock to replace the
+ test-set kvm-guest virt spinlock implementation with something
+ virtualization-friendly, for example, halt the virtual CPU rather
+ than spinning.
+
+ If you are unsure how to answer this question, answer Y.
+
endmenu # "Kernel features"
menu "Boot options"
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 671ad85f28f2..114b29234c46 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -103,3 +103,4 @@ obj-$(CONFIG_ARCH_RV64ILP32) += compat_signal.o
obj-$(CONFIG_64BIT) += pi/
obj-$(CONFIG_ACPI) += acpi.o
+obj-$(CONFIG_PARAVIRT_SPINLOCKS) += qspinlock_paravirt.o
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 14/17] RISC-V: paravirt: pvqspinlock: Add kconfig entry
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Add kconfig entry for paravirt_spinlock, an unfair qspinlock
virtualization-friendly backend, by halting the virtual CPU rather
than spinning.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 12 ++++++++++++
arch/riscv/kernel/Makefile | 1 +
2 files changed, 13 insertions(+)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 4bcff2860f48..ec0da24ed6fb 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -813,6 +813,18 @@ config RELOCATABLE
If unsure, say N.
+config PARAVIRT_SPINLOCKS
+ bool "Paravirtualization layer for spinlocks"
+ depends on QUEUED_SPINLOCKS
+ default y
+ help
+ Paravirtualized spinlocks allow a unfair qspinlock to replace the
+ test-set kvm-guest virt spinlock implementation with something
+ virtualization-friendly, for example, halt the virtual CPU rather
+ than spinning.
+
+ If you are unsure how to answer this question, answer Y.
+
endmenu # "Kernel features"
menu "Boot options"
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 671ad85f28f2..114b29234c46 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -103,3 +103,4 @@ obj-$(CONFIG_ARCH_RV64ILP32) += compat_signal.o
obj-$(CONFIG_64BIT) += pi/
obj-$(CONFIG_ACPI) += acpi.o
+obj-$(CONFIG_PARAVIRT_SPINLOCKS) += qspinlock_paravirt.o
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 15/17] RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Add trace point for pv_kick/wait, here is the output:
entries-in-buffer/entries-written: 33927/33927 #P:12
_-----=> irqs-off/BH-disabled
/ _----=> need-resched
| / _---=> hardirq/softirq
|| / _--=> preempt-depth
||| / _-=> migrate-disable
|||| / delay
TASK-PID CPU# ||||| TIMESTAMP FUNCTION
| | | ||||| | |
sh-100 [001] d..2. 28.312294: pv_wait: cpu 1 out of wfi
<idle>-0 [000] d.h4. 28.322030: pv_kick: cpu 0 kick target cpu 1
sh-100 [001] d..2. 30.982631: pv_wait: cpu 1 out of wfi
<idle>-0 [000] d.h4. 30.993289: pv_kick: cpu 0 kick target cpu 1
sh-100 [002] d..2. 44.987573: pv_wait: cpu 2 out of wfi
<idle>-0 [000] d.h4. 44.989000: pv_kick: cpu 0 kick target cpu 2
<idle>-0 [003] d.s3. 51.593978: pv_kick: cpu 3 kick target cpu 4
rcu_sched-15 [004] d..2. 51.595192: pv_wait: cpu 4 out of wfi
lock_torture_wr-115 [004] ...2. 52.656482: pv_kick: cpu 4 kick target cpu 2
lock_torture_wr-113 [002] d..2. 52.659146: pv_wait: cpu 2 out of wfi
lock_torture_wr-114 [008] d..2. 52.659507: pv_wait: cpu 8 out of wfi
lock_torture_wr-114 [008] d..2. 52.663503: pv_wait: cpu 8 out of wfi
lock_torture_wr-113 [002] ...2. 52.666128: pv_kick: cpu 2 kick target cpu 8
lock_torture_wr-114 [008] d..2. 52.667261: pv_wait: cpu 8 out of wfi
lock_torture_wr-114 [009] .n.2. 53.141515: pv_kick: cpu 9 kick target cpu 11
lock_torture_wr-113 [002] d..2. 53.143339: pv_wait: cpu 2 out of wfi
lock_torture_wr-116 [007] d..2. 53.143412: pv_wait: cpu 7 out of wfi
lock_torture_wr-118 [000] d..2. 53.143457: pv_wait: cpu 0 out of wfi
lock_torture_wr-115 [008] d..2. 53.143481: pv_wait: cpu 8 out of wfi
lock_torture_wr-117 [011] d..2. 53.143522: pv_wait: cpu 11 out of wfi
lock_torture_wr-117 [011] ...2. 53.143987: pv_kick: cpu 11 kick target cpu 8
lock_torture_wr-115 [008] ...2. 53.144269: pv_kick: cpu 8 kick target cpu 7
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/kernel/qspinlock_paravirt.c | 8 +++
.../kernel/trace_events_filter_paravirt.h | 60 +++++++++++++++++++
2 files changed, 68 insertions(+)
create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
index 571626f350be..5d298e989b99 100644
--- a/arch/riscv/kernel/qspinlock_paravirt.c
+++ b/arch/riscv/kernel/qspinlock_paravirt.c
@@ -9,10 +9,16 @@
#include <asm/qspinlock_paravirt.h>
#include <asm/sbi.h>
+#define CREATE_TRACE_POINTS
+#include "trace_events_filter_paravirt.h"
+
void pv_kick(int cpu)
{
sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
+
+ trace_pv_kick(smp_processor_id(), cpu);
+
return;
}
@@ -28,6 +34,8 @@ void pv_wait(u8 *ptr, u8 val)
goto out;
wait_for_interrupt();
+
+ trace_pv_wait(smp_processor_id());
out:
local_irq_restore(flags);
}
diff --git a/arch/riscv/kernel/trace_events_filter_paravirt.h b/arch/riscv/kernel/trace_events_filter_paravirt.h
new file mode 100644
index 000000000000..9ff5aa451b12
--- /dev/null
+++ b/arch/riscv/kernel/trace_events_filter_paravirt.h
@@ -0,0 +1,60 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c), 2023 Alibaba Cloud
+ * Authors:
+ * Guo Ren <guoren@linux.alibaba.com>
+ */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM paravirt
+
+#if !defined(_TRACE_PARAVIRT_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_PARAVIRT_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(pv_kick,
+ TP_PROTO(int cpu, int target),
+ TP_ARGS(cpu, target),
+
+ TP_STRUCT__entry(
+ __field(int, cpu)
+ __field(int, target)
+ ),
+
+ TP_fast_assign(
+ __entry->cpu = cpu;
+ __entry->target = target;
+ ),
+
+ TP_printk("cpu %d kick target cpu %d",
+ __entry->cpu,
+ __entry->target
+ )
+);
+
+TRACE_EVENT(pv_wait,
+ TP_PROTO(int cpu),
+ TP_ARGS(cpu),
+
+ TP_STRUCT__entry(
+ __field(int, cpu)
+ ),
+
+ TP_fast_assign(
+ __entry->cpu = cpu;
+ ),
+
+ TP_printk("cpu %d out of wfi",
+ __entry->cpu
+ )
+);
+
+#endif /* _TRACE_PARAVIRT_H || TRACE_HEADER_MULTI_READ */
+
+#undef TRACE_INCLUDE_PATH
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_PATH ../../../arch/riscv/kernel/
+#define TRACE_INCLUDE_FILE trace_events_filter_paravirt
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 15/17] RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Add trace point for pv_kick/wait, here is the output:
entries-in-buffer/entries-written: 33927/33927 #P:12
_-----=> irqs-off/BH-disabled
/ _----=> need-resched
| / _---=> hardirq/softirq
|| / _--=> preempt-depth
||| / _-=> migrate-disable
|||| / delay
TASK-PID CPU# ||||| TIMESTAMP FUNCTION
| | | ||||| | |
sh-100 [001] d..2. 28.312294: pv_wait: cpu 1 out of wfi
<idle>-0 [000] d.h4. 28.322030: pv_kick: cpu 0 kick target cpu 1
sh-100 [001] d..2. 30.982631: pv_wait: cpu 1 out of wfi
<idle>-0 [000] d.h4. 30.993289: pv_kick: cpu 0 kick target cpu 1
sh-100 [002] d..2. 44.987573: pv_wait: cpu 2 out of wfi
<idle>-0 [000] d.h4. 44.989000: pv_kick: cpu 0 kick target cpu 2
<idle>-0 [003] d.s3. 51.593978: pv_kick: cpu 3 kick target cpu 4
rcu_sched-15 [004] d..2. 51.595192: pv_wait: cpu 4 out of wfi
lock_torture_wr-115 [004] ...2. 52.656482: pv_kick: cpu 4 kick target cpu 2
lock_torture_wr-113 [002] d..2. 52.659146: pv_wait: cpu 2 out of wfi
lock_torture_wr-114 [008] d..2. 52.659507: pv_wait: cpu 8 out of wfi
lock_torture_wr-114 [008] d..2. 52.663503: pv_wait: cpu 8 out of wfi
lock_torture_wr-113 [002] ...2. 52.666128: pv_kick: cpu 2 kick target cpu 8
lock_torture_wr-114 [008] d..2. 52.667261: pv_wait: cpu 8 out of wfi
lock_torture_wr-114 [009] .n.2. 53.141515: pv_kick: cpu 9 kick target cpu 11
lock_torture_wr-113 [002] d..2. 53.143339: pv_wait: cpu 2 out of wfi
lock_torture_wr-116 [007] d..2. 53.143412: pv_wait: cpu 7 out of wfi
lock_torture_wr-118 [000] d..2. 53.143457: pv_wait: cpu 0 out of wfi
lock_torture_wr-115 [008] d..2. 53.143481: pv_wait: cpu 8 out of wfi
lock_torture_wr-117 [011] d..2. 53.143522: pv_wait: cpu 11 out of wfi
lock_torture_wr-117 [011] ...2. 53.143987: pv_kick: cpu 11 kick target cpu 8
lock_torture_wr-115 [008] ...2. 53.144269: pv_kick: cpu 8 kick target cpu 7
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/kernel/qspinlock_paravirt.c | 8 +++
.../kernel/trace_events_filter_paravirt.h | 60 +++++++++++++++++++
2 files changed, 68 insertions(+)
create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
index 571626f350be..5d298e989b99 100644
--- a/arch/riscv/kernel/qspinlock_paravirt.c
+++ b/arch/riscv/kernel/qspinlock_paravirt.c
@@ -9,10 +9,16 @@
#include <asm/qspinlock_paravirt.h>
#include <asm/sbi.h>
+#define CREATE_TRACE_POINTS
+#include "trace_events_filter_paravirt.h"
+
void pv_kick(int cpu)
{
sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
+
+ trace_pv_kick(smp_processor_id(), cpu);
+
return;
}
@@ -28,6 +34,8 @@ void pv_wait(u8 *ptr, u8 val)
goto out;
wait_for_interrupt();
+
+ trace_pv_wait(smp_processor_id());
out:
local_irq_restore(flags);
}
diff --git a/arch/riscv/kernel/trace_events_filter_paravirt.h b/arch/riscv/kernel/trace_events_filter_paravirt.h
new file mode 100644
index 000000000000..9ff5aa451b12
--- /dev/null
+++ b/arch/riscv/kernel/trace_events_filter_paravirt.h
@@ -0,0 +1,60 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c), 2023 Alibaba Cloud
+ * Authors:
+ * Guo Ren <guoren@linux.alibaba.com>
+ */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM paravirt
+
+#if !defined(_TRACE_PARAVIRT_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_PARAVIRT_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(pv_kick,
+ TP_PROTO(int cpu, int target),
+ TP_ARGS(cpu, target),
+
+ TP_STRUCT__entry(
+ __field(int, cpu)
+ __field(int, target)
+ ),
+
+ TP_fast_assign(
+ __entry->cpu = cpu;
+ __entry->target = target;
+ ),
+
+ TP_printk("cpu %d kick target cpu %d",
+ __entry->cpu,
+ __entry->target
+ )
+);
+
+TRACE_EVENT(pv_wait,
+ TP_PROTO(int cpu),
+ TP_ARGS(cpu),
+
+ TP_STRUCT__entry(
+ __field(int, cpu)
+ ),
+
+ TP_fast_assign(
+ __entry->cpu = cpu;
+ ),
+
+ TP_printk("cpu %d out of wfi",
+ __entry->cpu
+ )
+);
+
+#endif /* _TRACE_PARAVIRT_H || TRACE_HEADER_MULTI_READ */
+
+#undef TRACE_INCLUDE_PATH
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_PATH ../../../arch/riscv/kernel/
+#define TRACE_INCLUDE_FILE trace_events_filter_paravirt
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 16/17] RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Add the files functions needed to support the SBI PVLOCK (paravirt
qspinlock kick_cpu) extension. This is a preparation for the next
core implementation of kick_cpu.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
arch/riscv/include/uapi/asm/kvm.h | 1 +
arch/riscv/kvm/Makefile | 1 +
arch/riscv/kvm/vcpu_sbi.c | 4 +++
arch/riscv/kvm/vcpu_sbi_pvlock.c | 38 +++++++++++++++++++++++++++
5 files changed, 45 insertions(+)
create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
index cdcf0ff07be7..7b4d60b54d7e 100644
--- a/arch/riscv/include/asm/kvm_vcpu_sbi.h
+++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
@@ -71,6 +71,7 @@ extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_srst;
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_hsm;
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_experimental;
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_vendor;
+extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pvlock;
#ifdef CONFIG_RISCV_PMU_SBI
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pmu;
diff --git a/arch/riscv/include/uapi/asm/kvm.h b/arch/riscv/include/uapi/asm/kvm.h
index 992c5e407104..d005c229f2da 100644
--- a/arch/riscv/include/uapi/asm/kvm.h
+++ b/arch/riscv/include/uapi/asm/kvm.h
@@ -148,6 +148,7 @@ enum KVM_RISCV_SBI_EXT_ID {
KVM_RISCV_SBI_EXT_PMU,
KVM_RISCV_SBI_EXT_EXPERIMENTAL,
KVM_RISCV_SBI_EXT_VENDOR,
+ KVM_RISCV_SBI_EXT_PVLOCK,
KVM_RISCV_SBI_EXT_MAX,
};
diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
index 4c2067fc59fc..6112750a3a0c 100644
--- a/arch/riscv/kvm/Makefile
+++ b/arch/riscv/kvm/Makefile
@@ -26,6 +26,7 @@ kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
kvm-y += vcpu_sbi_base.o
kvm-y += vcpu_sbi_replace.o
kvm-y += vcpu_sbi_hsm.o
+kvm-y += vcpu_sbi_pvlock.o
kvm-y += vcpu_timer.o
kvm-$(CONFIG_RISCV_PMU_SBI) += vcpu_pmu.o vcpu_sbi_pmu.o
kvm-y += aia.o
diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
index 9cd97091c723..c03c3d489b2b 100644
--- a/arch/riscv/kvm/vcpu_sbi.c
+++ b/arch/riscv/kvm/vcpu_sbi.c
@@ -74,6 +74,10 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ext[] = {
.ext_idx = KVM_RISCV_SBI_EXT_VENDOR,
.ext_ptr = &vcpu_sbi_ext_vendor,
},
+ {
+ .ext_idx = KVM_RISCV_SBI_EXT_PVLOCK,
+ .ext_ptr = &vcpu_sbi_ext_pvlock,
+ },
};
void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
diff --git a/arch/riscv/kvm/vcpu_sbi_pvlock.c b/arch/riscv/kvm/vcpu_sbi_pvlock.c
new file mode 100644
index 000000000000..544a456c5041
--- /dev/null
+++ b/arch/riscv/kvm/vcpu_sbi_pvlock.c
@@ -0,0 +1,38 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c), 2023 Alibaba Cloud
+ *
+ * Authors:
+ * Guo Ren <guoren@linux.alibaba.com>
+ */
+
+#include <linux/errno.h>
+#include <linux/err.h>
+#include <linux/kvm_host.h>
+#include <asm/sbi.h>
+#include <asm/kvm_vcpu_sbi.h>
+
+static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ struct kvm_vcpu_sbi_return *retdata)
+{
+ int ret = 0;
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ unsigned long funcid = cp->a6;
+
+ switch (funcid) {
+ case SBI_EXT_PVLOCK_KICK_CPU:
+ break;
+ default:
+ ret = SBI_ERR_NOT_SUPPORTED;
+ }
+
+ retdata->err_val = ret;
+
+ return 0;
+}
+
+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pvlock = {
+ .extid_start = SBI_EXT_PVLOCK,
+ .extid_end = SBI_EXT_PVLOCK,
+ .handler = kvm_sbi_ext_pvlock_handler,
+};
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 16/17] RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Add the files functions needed to support the SBI PVLOCK (paravirt
qspinlock kick_cpu) extension. This is a preparation for the next
core implementation of kick_cpu.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
arch/riscv/include/uapi/asm/kvm.h | 1 +
arch/riscv/kvm/Makefile | 1 +
arch/riscv/kvm/vcpu_sbi.c | 4 +++
arch/riscv/kvm/vcpu_sbi_pvlock.c | 38 +++++++++++++++++++++++++++
5 files changed, 45 insertions(+)
create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
index cdcf0ff07be7..7b4d60b54d7e 100644
--- a/arch/riscv/include/asm/kvm_vcpu_sbi.h
+++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
@@ -71,6 +71,7 @@ extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_srst;
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_hsm;
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_experimental;
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_vendor;
+extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pvlock;
#ifdef CONFIG_RISCV_PMU_SBI
extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pmu;
diff --git a/arch/riscv/include/uapi/asm/kvm.h b/arch/riscv/include/uapi/asm/kvm.h
index 992c5e407104..d005c229f2da 100644
--- a/arch/riscv/include/uapi/asm/kvm.h
+++ b/arch/riscv/include/uapi/asm/kvm.h
@@ -148,6 +148,7 @@ enum KVM_RISCV_SBI_EXT_ID {
KVM_RISCV_SBI_EXT_PMU,
KVM_RISCV_SBI_EXT_EXPERIMENTAL,
KVM_RISCV_SBI_EXT_VENDOR,
+ KVM_RISCV_SBI_EXT_PVLOCK,
KVM_RISCV_SBI_EXT_MAX,
};
diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
index 4c2067fc59fc..6112750a3a0c 100644
--- a/arch/riscv/kvm/Makefile
+++ b/arch/riscv/kvm/Makefile
@@ -26,6 +26,7 @@ kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
kvm-y += vcpu_sbi_base.o
kvm-y += vcpu_sbi_replace.o
kvm-y += vcpu_sbi_hsm.o
+kvm-y += vcpu_sbi_pvlock.o
kvm-y += vcpu_timer.o
kvm-$(CONFIG_RISCV_PMU_SBI) += vcpu_pmu.o vcpu_sbi_pmu.o
kvm-y += aia.o
diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
index 9cd97091c723..c03c3d489b2b 100644
--- a/arch/riscv/kvm/vcpu_sbi.c
+++ b/arch/riscv/kvm/vcpu_sbi.c
@@ -74,6 +74,10 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ext[] = {
.ext_idx = KVM_RISCV_SBI_EXT_VENDOR,
.ext_ptr = &vcpu_sbi_ext_vendor,
},
+ {
+ .ext_idx = KVM_RISCV_SBI_EXT_PVLOCK,
+ .ext_ptr = &vcpu_sbi_ext_pvlock,
+ },
};
void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
diff --git a/arch/riscv/kvm/vcpu_sbi_pvlock.c b/arch/riscv/kvm/vcpu_sbi_pvlock.c
new file mode 100644
index 000000000000..544a456c5041
--- /dev/null
+++ b/arch/riscv/kvm/vcpu_sbi_pvlock.c
@@ -0,0 +1,38 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c), 2023 Alibaba Cloud
+ *
+ * Authors:
+ * Guo Ren <guoren@linux.alibaba.com>
+ */
+
+#include <linux/errno.h>
+#include <linux/err.h>
+#include <linux/kvm_host.h>
+#include <asm/sbi.h>
+#include <asm/kvm_vcpu_sbi.h>
+
+static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ struct kvm_vcpu_sbi_return *retdata)
+{
+ int ret = 0;
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ unsigned long funcid = cp->a6;
+
+ switch (funcid) {
+ case SBI_EXT_PVLOCK_KICK_CPU:
+ break;
+ default:
+ ret = SBI_ERR_NOT_SUPPORTED;
+ }
+
+ retdata->err_val = ret;
+
+ return 0;
+}
+
+const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pvlock = {
+ .extid_start = SBI_EXT_PVLOCK,
+ .extid_end = SBI_EXT_PVLOCK,
+ .handler = kvm_sbi_ext_pvlock_handler,
+};
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 17/17] RISC-V: paravirt: pvqspinlock: KVM: Implement kvm_sbi_ext_pvlock_kick_cpu()
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:29 ` guoren
-1 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
We only need to call the kvm_vcpu_kick() and bring target_vcpu
from the halt state. No irq raised, no other request, just a pure
vcpu_kick.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/kvm/vcpu_sbi_pvlock.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/arch/riscv/kvm/vcpu_sbi_pvlock.c b/arch/riscv/kvm/vcpu_sbi_pvlock.c
index 544a456c5041..914fc58aedfe 100644
--- a/arch/riscv/kvm/vcpu_sbi_pvlock.c
+++ b/arch/riscv/kvm/vcpu_sbi_pvlock.c
@@ -12,6 +12,24 @@
#include <asm/sbi.h>
#include <asm/kvm_vcpu_sbi.h>
+static int kvm_sbi_ext_pvlock_kick_cpu(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ struct kvm *kvm = vcpu->kvm;
+ struct kvm_vcpu *target;
+
+ target = kvm_get_vcpu_by_id(kvm, cp->a0);
+ if (!target)
+ return SBI_ERR_INVALID_PARAM;
+
+ kvm_vcpu_kick(target);
+
+ if (READ_ONCE(target->ready))
+ kvm_vcpu_yield_to(target);
+
+ return SBI_SUCCESS;
+}
+
static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
struct kvm_vcpu_sbi_return *retdata)
{
@@ -21,6 +39,7 @@ static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run
switch (funcid) {
case SBI_EXT_PVLOCK_KICK_CPU:
+ ret = kvm_sbi_ext_pvlock_kick_cpu(vcpu);
break;
default:
ret = SBI_ERR_NOT_SUPPORTED;
--
2.36.1
^ permalink raw reply related [flat|nested] 215+ messages in thread
* [PATCH V11 17/17] RISC-V: paravirt: pvqspinlock: KVM: Implement kvm_sbi_ext_pvlock_kick_cpu()
@ 2023-09-10 8:29 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-09-10 8:29 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
We only need to call the kvm_vcpu_kick() and bring target_vcpu
from the halt state. No irq raised, no other request, just a pure
vcpu_kick.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/kvm/vcpu_sbi_pvlock.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/arch/riscv/kvm/vcpu_sbi_pvlock.c b/arch/riscv/kvm/vcpu_sbi_pvlock.c
index 544a456c5041..914fc58aedfe 100644
--- a/arch/riscv/kvm/vcpu_sbi_pvlock.c
+++ b/arch/riscv/kvm/vcpu_sbi_pvlock.c
@@ -12,6 +12,24 @@
#include <asm/sbi.h>
#include <asm/kvm_vcpu_sbi.h>
+static int kvm_sbi_ext_pvlock_kick_cpu(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
+ struct kvm *kvm = vcpu->kvm;
+ struct kvm_vcpu *target;
+
+ target = kvm_get_vcpu_by_id(kvm, cp->a0);
+ if (!target)
+ return SBI_ERR_INVALID_PARAM;
+
+ kvm_vcpu_kick(target);
+
+ if (READ_ONCE(target->ready))
+ kvm_vcpu_yield_to(target);
+
+ return SBI_SUCCESS;
+}
+
static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
struct kvm_vcpu_sbi_return *retdata)
{
@@ -21,6 +39,7 @@ static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run
switch (funcid) {
case SBI_EXT_PVLOCK_KICK_CPU:
+ ret = kvm_sbi_ext_pvlock_kick_cpu(vcpu);
break;
default:
ret = SBI_ERR_NOT_SUPPORTED;
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-10 8:28 ` guoren
@ 2023-09-10 8:58 ` Conor Dooley
-1 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-10 8:58 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
[-- Attachment #1: Type: text/plain, Size: 677 bytes --]
On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> Changlog:
> V11:
> - Based on Leonardo Bras's cmpxchg_small patches v5.
> - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> - Remove abusing alternative framework and use jump_label instead.
btw, I didn't say that using alternatives was the problem, it was
abusing the errata framework to perform feature detection that I had
a problem with. That's not changed in v11.
A stronger forward progress guarantee is not an erratum, AFAICT.
> - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> guarantee.
> - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-10 8:58 ` Conor Dooley
0 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-10 8:58 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
[-- Attachment #1.1: Type: text/plain, Size: 677 bytes --]
On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> Changlog:
> V11:
> - Based on Leonardo Bras's cmpxchg_small patches v5.
> - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> - Remove abusing alternative framework and use jump_label instead.
btw, I didn't say that using alternatives was the problem, it was
abusing the errata framework to perform feature detection that I had
a problem with. That's not changed in v11.
A stronger forward progress guarantee is not an erratum, AFAICT.
> - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> guarantee.
> - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-10 8:58 ` Conor Dooley
@ 2023-09-10 9:16 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-10 9:16 UTC (permalink / raw)
To: Conor Dooley
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
>
> On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
>
> > Changlog:
> > V11:
> > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > - Remove abusing alternative framework and use jump_label instead.
>
> btw, I didn't say that using alternatives was the problem, it was
> abusing the errata framework to perform feature detection that I had
> a problem with. That's not changed in v11.
I've removed errata feature detection. The only related patches are:
- riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
- riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
Which one is your concern? Could you reply on the exact patch thread? Thx.
>
> A stronger forward progress guarantee is not an erratum, AFAICT.
Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
>
> > - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> > guarantee.
> > - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-10 9:16 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-10 9:16 UTC (permalink / raw)
To: Conor Dooley
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
>
> On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
>
> > Changlog:
> > V11:
> > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > - Remove abusing alternative framework and use jump_label instead.
>
> btw, I didn't say that using alternatives was the problem, it was
> abusing the errata framework to perform feature detection that I had
> a problem with. That's not changed in v11.
I've removed errata feature detection. The only related patches are:
- riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
- riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
Which one is your concern? Could you reply on the exact patch thread? Thx.
>
> A stronger forward progress guarantee is not an erratum, AFAICT.
Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
>
> > - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> > guarantee.
> > - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-10 9:16 ` Guo Ren
@ 2023-09-10 9:20 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-10 9:20 UTC (permalink / raw)
To: Conor Dooley
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 5:16 PM Guo Ren <guoren@kernel.org> wrote:
>
> On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> >
> > > Changlog:
> > > V11:
> > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > - Remove abusing alternative framework and use jump_label instead.
> >
> > btw, I didn't say that using alternatives was the problem, it was
> > abusing the errata framework to perform feature detection that I had
> > a problem with. That's not changed in v11.
> I've removed errata feature detection. The only related patches are:
> - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
And this one:
- riscv: Use Zicbop in arch_xchg when available
>
> Which one is your concern? Could you reply on the exact patch thread? Thx.
>
> >
> > A stronger forward progress guarantee is not an erratum, AFAICT.
> Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
>
> >
> > > - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> > > guarantee.
> > > - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
>
>
>
> --
> Best Regards
> Guo Ren
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-10 9:20 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-10 9:20 UTC (permalink / raw)
To: Conor Dooley
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 5:16 PM Guo Ren <guoren@kernel.org> wrote:
>
> On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> >
> > > Changlog:
> > > V11:
> > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > - Remove abusing alternative framework and use jump_label instead.
> >
> > btw, I didn't say that using alternatives was the problem, it was
> > abusing the errata framework to perform feature detection that I had
> > a problem with. That's not changed in v11.
> I've removed errata feature detection. The only related patches are:
> - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
And this one:
- riscv: Use Zicbop in arch_xchg when available
>
> Which one is your concern? Could you reply on the exact patch thread? Thx.
>
> >
> > A stronger forward progress guarantee is not an erratum, AFAICT.
> Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
>
> >
> > > - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> > > guarantee.
> > > - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
>
>
>
> --
> Best Regards
> Guo Ren
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-10 9:16 ` Guo Ren
@ 2023-09-10 9:31 ` Conor Dooley
-1 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-10 9:31 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
[-- Attachment #1: Type: text/plain, Size: 1455 bytes --]
On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> >
> > > Changlog:
> > > V11:
> > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > - Remove abusing alternative framework and use jump_label instead.
> >
> > btw, I didn't say that using alternatives was the problem, it was
> > abusing the errata framework to perform feature detection that I had
> > a problem with. That's not changed in v11.
> I've removed errata feature detection. The only related patches are:
> - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
>
> Which one is your concern? Could you reply on the exact patch thread? Thx.
riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
Please go back and re-read the comments I left on v11 about using the
errata code for feature detection.
> > A stronger forward progress guarantee is not an erratum, AFAICT.
> Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
"riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
uses the errata framework to detect the presence of the stronger forward
progress guarantee in v11.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-10 9:31 ` Conor Dooley
0 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-10 9:31 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
[-- Attachment #1.1: Type: text/plain, Size: 1455 bytes --]
On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> >
> > > Changlog:
> > > V11:
> > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > - Remove abusing alternative framework and use jump_label instead.
> >
> > btw, I didn't say that using alternatives was the problem, it was
> > abusing the errata framework to perform feature detection that I had
> > a problem with. That's not changed in v11.
> I've removed errata feature detection. The only related patches are:
> - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
>
> Which one is your concern? Could you reply on the exact patch thread? Thx.
riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
Please go back and re-read the comments I left on v11 about using the
errata code for feature detection.
> > A stronger forward progress guarantee is not an erratum, AFAICT.
> Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
"riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
uses the errata framework to detect the presence of the stronger forward
progress guarantee in v11.
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-10 9:31 ` Conor Dooley
@ 2023-09-10 9:49 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-10 9:49 UTC (permalink / raw)
To: Conor Dooley
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 5:32 PM Conor Dooley <conor@kernel.org> wrote:
>
> On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> > On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > >
> > > > Changlog:
> > > > V11:
> > > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > > - Remove abusing alternative framework and use jump_label instead.
> > >
> > > btw, I didn't say that using alternatives was the problem, it was
> > > abusing the errata framework to perform feature detection that I had
> > > a problem with. That's not changed in v11.
> > I've removed errata feature detection. The only related patches are:
> > - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> >
> > Which one is your concern? Could you reply on the exact patch thread? Thx.
>
> riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
>
> Please go back and re-read the comments I left on v11 about using the
> errata code for feature detection.
>
> > > A stronger forward progress guarantee is not an erratum, AFAICT.
>
> > Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
>
> "riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
> uses the errata framework to detect the presence of the stronger forward
> progress guarantee in v11.
Oh, thx for pointing it out. I could replace it with this:
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 88690751f2ee..4be92766d3e3 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -310,7 +310,8 @@ static void __init riscv_spinlock_init(void)
{
#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
if (!enable_qspinlock_key &&
- (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
+ (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM) &&
+ (sbi_get_mvendorid() != THEAD_VENDOR_ID)) {
static_branch_disable(&combo_qspinlock_key);
pr_info("Ticket spinlock: enabled\n");
} else {
--
Best Regards
Guo Ren
^ permalink raw reply related [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-10 9:49 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-10 9:49 UTC (permalink / raw)
To: Conor Dooley
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 5:32 PM Conor Dooley <conor@kernel.org> wrote:
>
> On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> > On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > >
> > > > Changlog:
> > > > V11:
> > > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > > - Remove abusing alternative framework and use jump_label instead.
> > >
> > > btw, I didn't say that using alternatives was the problem, it was
> > > abusing the errata framework to perform feature detection that I had
> > > a problem with. That's not changed in v11.
> > I've removed errata feature detection. The only related patches are:
> > - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> >
> > Which one is your concern? Could you reply on the exact patch thread? Thx.
>
> riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
>
> Please go back and re-read the comments I left on v11 about using the
> errata code for feature detection.
>
> > > A stronger forward progress guarantee is not an erratum, AFAICT.
>
> > Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
>
> "riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
> uses the errata framework to detect the presence of the stronger forward
> progress guarantee in v11.
Oh, thx for pointing it out. I could replace it with this:
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 88690751f2ee..4be92766d3e3 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -310,7 +310,8 @@ static void __init riscv_spinlock_init(void)
{
#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
if (!enable_qspinlock_key &&
- (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
+ (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM) &&
+ (sbi_get_mvendorid() != THEAD_VENDOR_ID)) {
static_branch_disable(&combo_qspinlock_key);
pr_info("Ticket spinlock: enabled\n");
} else {
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
* Re: [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
2023-09-10 8:29 ` guoren
@ 2023-09-10 11:06 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-10 11:06 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:00AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Combo spinlock could support queued and ticket in one Linux Image and
> select them during boot time via errata mechanism. Here is the func
> size (Bytes) comparison table below:
>
> TYPE : COMBO | TICKET | QUEUED
> arch_spin_lock : 106 | 60 | 50
> arch_spin_unlock : 54 | 36 | 26
> arch_spin_trylock : 110 | 72 | 54
> arch_spin_is_locked : 48 | 34 | 20
> arch_spin_is_contended : 56 | 40 | 24
> rch_spin_value_unlocked : 48 | 34 | 24
>
> One example of disassemble combo arch_spin_unlock:
> 0xffffffff8000409c <+14>: nop # detour slot
> 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
> 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
> 0xffffffff800040a8 <+26>: ld s0,8(sp)
> 0xffffffff800040aa <+28>: addi sp,sp,16
> 0xffffffff800040ac <+30>: ret
> 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
> 0xffffffff800040b0 <+34>: sext.w a5,a5
> 0xffffffff800040b2 <+36>: fence rw,w
> 0xffffffff800040b6 <+40>: addiw a5,a5,1
> 0xffffffff800040b8 <+42>: slli a5,a5,0x30
> 0xffffffff800040ba <+44>: srli a5,a5,0x30
> 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
> 0xffffffff800040c0 <+50>: ld s0,8(sp)
> 0xffffffff800040c2 <+52>: addi sp,sp,16
> 0xffffffff800040c4 <+54>: ret
>
> The qspinlock is smaller and faster than ticket-lock when all are in
> fast-path, and combo spinlock could provide a compatible Linux Image
> for different micro-arch design (weak/strict fwd guarantee LR/SC)
> processors.
>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> ---
> arch/riscv/Kconfig | 9 +++-
> arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
> arch/riscv/kernel/setup.c | 14 ++++++
> 3 files changed, 98 insertions(+), 3 deletions(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 7f39bfc75744..4bcff2860f48 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -473,7 +473,7 @@ config NODES_SHIFT
>
> choice
> prompt "RISC-V spinlock type"
> - default RISCV_TICKET_SPINLOCKS
> + default RISCV_COMBO_SPINLOCKS
>
> config RISCV_TICKET_SPINLOCKS
> bool "Using ticket spinlock"
> @@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
> help
> Make sure your micro arch LL/SC has a strong forward progress guarantee.
> Otherwise, stay at ticket-lock.
> +
> +config RISCV_COMBO_SPINLOCKS
> + bool "Using combo spinlock"
> + depends on SMP && MMU
> + select ARCH_USE_QUEUED_SPINLOCKS
> + help
> + Select queued spinlock or ticket-lock via errata.
> endchoice
>
> config RISCV_ALTERNATIVE
> diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> index c644a92d4548..8ea0fee80652 100644
> --- a/arch/riscv/include/asm/spinlock.h
> +++ b/arch/riscv/include/asm/spinlock.h
> @@ -7,11 +7,85 @@
> #define _Q_PENDING_LOOPS (1 << 9)
> #endif
>
> +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> +#include <asm-generic/ticket_spinlock.h>
> +
> +#undef arch_spin_is_locked
> +#undef arch_spin_is_contended
> +#undef arch_spin_value_unlocked
> +#undef arch_spin_lock
> +#undef arch_spin_trylock
> +#undef arch_spin_unlock
> +
> +#include <asm-generic/qspinlock.h>
> +#include <linux/jump_label.h>
> +
> +#undef arch_spin_is_locked
> +#undef arch_spin_is_contended
> +#undef arch_spin_value_unlocked
> +#undef arch_spin_lock
> +#undef arch_spin_trylock
> +#undef arch_spin_unlock
Sorry, I forgot __no_arch_spinlock_redefine advice here. I would add it in v12.
https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> +
> +DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
> +
> +static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + queued_spin_lock(lock);
> + else
> + ticket_spin_lock(lock);
> +}
> +
> +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + return queued_spin_trylock(lock);
> + else
> + return ticket_spin_trylock(lock);
> +}
> +
> +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + queued_spin_unlock(lock);
> + else
> + ticket_spin_unlock(lock);
> +}
> +
> +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + return queued_spin_value_unlocked(lock);
> + else
> + return ticket_spin_value_unlocked(lock);
> +}
> +
> +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + return queued_spin_is_locked(lock);
> + else
> + return ticket_spin_is_locked(lock);
> +}
> +
> +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + return queued_spin_is_contended(lock);
> + else
> + return ticket_spin_is_contended(lock);
> +}
> +#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
> +
> #ifdef CONFIG_QUEUED_SPINLOCKS
> #include <asm/qspinlock.h>
> -#include <asm/qrwlock.h>
> #else
> -#include <asm-generic/spinlock.h>
> +#include <asm-generic/ticket_spinlock.h>
> #endif
>
> +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
> +
> +#include <asm/qrwlock.h>
> +
> #endif /* __ASM_RISCV_SPINLOCK_H */
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index 32c2e1eb71bd..a447cf360a18 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -269,6 +269,18 @@ static void __init parse_dtb(void)
> #endif
> }
>
> +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> +DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> +EXPORT_SYMBOL(combo_qspinlock_key);
> +#endif
> +
> +static void __init riscv_spinlock_init(void)
> +{
> +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> + static_branch_disable(&combo_qspinlock_key);
> +#endif
> +}
> +
> extern void __init init_rt_signal_env(void);
>
> void __init setup_arch(char **cmdline_p)
> @@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
> riscv_isa_extension_available(NULL, ZICBOM))
> riscv_noncoherent_supported();
> riscv_set_dma_cache_alignment();
> +
> + riscv_spinlock_init();
> }
>
> static int __init topology_init(void)
> --
> 2.36.1
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
@ 2023-09-10 11:06 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-10 11:06 UTC (permalink / raw)
To: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:00AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Combo spinlock could support queued and ticket in one Linux Image and
> select them during boot time via errata mechanism. Here is the func
> size (Bytes) comparison table below:
>
> TYPE : COMBO | TICKET | QUEUED
> arch_spin_lock : 106 | 60 | 50
> arch_spin_unlock : 54 | 36 | 26
> arch_spin_trylock : 110 | 72 | 54
> arch_spin_is_locked : 48 | 34 | 20
> arch_spin_is_contended : 56 | 40 | 24
> rch_spin_value_unlocked : 48 | 34 | 24
>
> One example of disassemble combo arch_spin_unlock:
> 0xffffffff8000409c <+14>: nop # detour slot
> 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
> 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
> 0xffffffff800040a8 <+26>: ld s0,8(sp)
> 0xffffffff800040aa <+28>: addi sp,sp,16
> 0xffffffff800040ac <+30>: ret
> 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
> 0xffffffff800040b0 <+34>: sext.w a5,a5
> 0xffffffff800040b2 <+36>: fence rw,w
> 0xffffffff800040b6 <+40>: addiw a5,a5,1
> 0xffffffff800040b8 <+42>: slli a5,a5,0x30
> 0xffffffff800040ba <+44>: srli a5,a5,0x30
> 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
> 0xffffffff800040c0 <+50>: ld s0,8(sp)
> 0xffffffff800040c2 <+52>: addi sp,sp,16
> 0xffffffff800040c4 <+54>: ret
>
> The qspinlock is smaller and faster than ticket-lock when all are in
> fast-path, and combo spinlock could provide a compatible Linux Image
> for different micro-arch design (weak/strict fwd guarantee LR/SC)
> processors.
>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> ---
> arch/riscv/Kconfig | 9 +++-
> arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
> arch/riscv/kernel/setup.c | 14 ++++++
> 3 files changed, 98 insertions(+), 3 deletions(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 7f39bfc75744..4bcff2860f48 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -473,7 +473,7 @@ config NODES_SHIFT
>
> choice
> prompt "RISC-V spinlock type"
> - default RISCV_TICKET_SPINLOCKS
> + default RISCV_COMBO_SPINLOCKS
>
> config RISCV_TICKET_SPINLOCKS
> bool "Using ticket spinlock"
> @@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
> help
> Make sure your micro arch LL/SC has a strong forward progress guarantee.
> Otherwise, stay at ticket-lock.
> +
> +config RISCV_COMBO_SPINLOCKS
> + bool "Using combo spinlock"
> + depends on SMP && MMU
> + select ARCH_USE_QUEUED_SPINLOCKS
> + help
> + Select queued spinlock or ticket-lock via errata.
> endchoice
>
> config RISCV_ALTERNATIVE
> diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> index c644a92d4548..8ea0fee80652 100644
> --- a/arch/riscv/include/asm/spinlock.h
> +++ b/arch/riscv/include/asm/spinlock.h
> @@ -7,11 +7,85 @@
> #define _Q_PENDING_LOOPS (1 << 9)
> #endif
>
> +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> +#include <asm-generic/ticket_spinlock.h>
> +
> +#undef arch_spin_is_locked
> +#undef arch_spin_is_contended
> +#undef arch_spin_value_unlocked
> +#undef arch_spin_lock
> +#undef arch_spin_trylock
> +#undef arch_spin_unlock
> +
> +#include <asm-generic/qspinlock.h>
> +#include <linux/jump_label.h>
> +
> +#undef arch_spin_is_locked
> +#undef arch_spin_is_contended
> +#undef arch_spin_value_unlocked
> +#undef arch_spin_lock
> +#undef arch_spin_trylock
> +#undef arch_spin_unlock
Sorry, I forgot __no_arch_spinlock_redefine advice here. I would add it in v12.
https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> +
> +DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
> +
> +static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + queued_spin_lock(lock);
> + else
> + ticket_spin_lock(lock);
> +}
> +
> +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + return queued_spin_trylock(lock);
> + else
> + return ticket_spin_trylock(lock);
> +}
> +
> +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + queued_spin_unlock(lock);
> + else
> + ticket_spin_unlock(lock);
> +}
> +
> +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + return queued_spin_value_unlocked(lock);
> + else
> + return ticket_spin_value_unlocked(lock);
> +}
> +
> +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + return queued_spin_is_locked(lock);
> + else
> + return ticket_spin_is_locked(lock);
> +}
> +
> +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> +{
> + if (static_branch_likely(&combo_qspinlock_key))
> + return queued_spin_is_contended(lock);
> + else
> + return ticket_spin_is_contended(lock);
> +}
> +#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
> +
> #ifdef CONFIG_QUEUED_SPINLOCKS
> #include <asm/qspinlock.h>
> -#include <asm/qrwlock.h>
> #else
> -#include <asm-generic/spinlock.h>
> +#include <asm-generic/ticket_spinlock.h>
> #endif
>
> +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
> +
> +#include <asm/qrwlock.h>
> +
> #endif /* __ASM_RISCV_SPINLOCK_H */
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index 32c2e1eb71bd..a447cf360a18 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -269,6 +269,18 @@ static void __init parse_dtb(void)
> #endif
> }
>
> +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> +DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> +EXPORT_SYMBOL(combo_qspinlock_key);
> +#endif
> +
> +static void __init riscv_spinlock_init(void)
> +{
> +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> + static_branch_disable(&combo_qspinlock_key);
> +#endif
> +}
> +
> extern void __init init_rt_signal_env(void);
>
> void __init setup_arch(char **cmdline_p)
> @@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
> riscv_isa_extension_available(NULL, ZICBOM))
> riscv_noncoherent_supported();
> riscv_set_dma_cache_alignment();
> +
> + riscv_spinlock_init();
> }
>
> static int __init topology_init(void)
> --
> 2.36.1
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-10 9:49 ` Guo Ren
@ 2023-09-10 19:45 ` Conor Dooley
-1 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-10 19:45 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
[-- Attachment #1: Type: text/plain, Size: 2861 bytes --]
On Sun, Sep 10, 2023 at 05:49:13PM +0800, Guo Ren wrote:
> On Sun, Sep 10, 2023 at 5:32 PM Conor Dooley <conor@kernel.org> wrote:
> >
> > On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> > > On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> > > >
> > > > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > > >
> > > > > Changlog:
> > > > > V11:
> > > > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > > > - Remove abusing alternative framework and use jump_label instead.
> > > >
> > > > btw, I didn't say that using alternatives was the problem, it was
> > > > abusing the errata framework to perform feature detection that I had
> > > > a problem with. That's not changed in v11.
> > > I've removed errata feature detection. The only related patches are:
> > > - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > > - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > >
> > > Which one is your concern? Could you reply on the exact patch thread? Thx.
> >
> > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> >
> > Please go back and re-read the comments I left on v11 about using the
> > errata code for feature detection.
> >
> > > > A stronger forward progress guarantee is not an erratum, AFAICT.
> >
> > > Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
> >
> > "riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
> > uses the errata framework to detect the presence of the stronger forward
> > progress guarantee in v11.
> Oh, thx for pointing it out. I could replace it with this:
>
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index 88690751f2ee..4be92766d3e3 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -310,7 +310,8 @@ static void __init riscv_spinlock_init(void)
> {
> #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> if (!enable_qspinlock_key &&
> - (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM) &&
> + (sbi_get_mvendorid() != THEAD_VENDOR_ID)) {
> static_branch_disable(&combo_qspinlock_key);
> pr_info("Ticket spinlock: enabled\n");
> } else {
As I said on v11, I am opposed to feature probing using mvendorid & Co,
partially due to the exact sort of check here to see if the kernel is
running as a KVM guest. IMO, whether a platform has this stronger
guarantee needs to be communicated by firmware, using ACPI or DT.
I made some comments on v11, referring similar discussion about the
thead vector stuff. Please go take a look at that.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-10 19:45 ` Conor Dooley
0 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-10 19:45 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
[-- Attachment #1.1: Type: text/plain, Size: 2861 bytes --]
On Sun, Sep 10, 2023 at 05:49:13PM +0800, Guo Ren wrote:
> On Sun, Sep 10, 2023 at 5:32 PM Conor Dooley <conor@kernel.org> wrote:
> >
> > On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> > > On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> > > >
> > > > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > > >
> > > > > Changlog:
> > > > > V11:
> > > > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > > > - Remove abusing alternative framework and use jump_label instead.
> > > >
> > > > btw, I didn't say that using alternatives was the problem, it was
> > > > abusing the errata framework to perform feature detection that I had
> > > > a problem with. That's not changed in v11.
> > > I've removed errata feature detection. The only related patches are:
> > > - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > > - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > >
> > > Which one is your concern? Could you reply on the exact patch thread? Thx.
> >
> > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> >
> > Please go back and re-read the comments I left on v11 about using the
> > errata code for feature detection.
> >
> > > > A stronger forward progress guarantee is not an erratum, AFAICT.
> >
> > > Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
> >
> > "riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
> > uses the errata framework to detect the presence of the stronger forward
> > progress guarantee in v11.
> Oh, thx for pointing it out. I could replace it with this:
>
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index 88690751f2ee..4be92766d3e3 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -310,7 +310,8 @@ static void __init riscv_spinlock_init(void)
> {
> #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> if (!enable_qspinlock_key &&
> - (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM) &&
> + (sbi_get_mvendorid() != THEAD_VENDOR_ID)) {
> static_branch_disable(&combo_qspinlock_key);
> pr_info("Ticket spinlock: enabled\n");
> } else {
As I said on v11, I am opposed to feature probing using mvendorid & Co,
partially due to the exact sort of check here to see if the kernel is
running as a KVM guest. IMO, whether a platform has this stronger
guarantee needs to be communicated by firmware, using ACPI or DT.
I made some comments on v11, referring similar discussion about the
thead vector stuff. Please go take a look at that.
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
2023-09-10 8:28 ` guoren
(?)
@ 2023-09-11 2:35 ` Waiman Long
-1 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 2:35 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, Guo Ren, kvm, linux-doc, linux-csky, virtualization,
linux-riscv
On 9/10/23 04:28, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The target of xchg_tail is to write the tail to the lock value, so
> adding prefetchw could help the next cmpxchg step, which may
> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> utilize this feature to give a forward guarantee, e.g., RISC-V
> XuanTie processors would block the snoop channel & irq for several
> cycles when prefetch.w instruction (from Zicbop extension) retired,
> which guarantees the next cmpxchg succeeds.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> kernel/locking/qspinlock.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> index d3f99060b60f..96b54e2ade86 100644
> --- a/kernel/locking/qspinlock.c
> +++ b/kernel/locking/qspinlock.c
> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> */
> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> {
> - u32 old, new, val = atomic_read(&lock->val);
> + u32 old, new, val;
> +
> + prefetchw(&lock->val);
> + val = atomic_read(&lock->val);
>
> for (;;) {
> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
That looks a bit weird. You pre-fetch and then immediately read it. How
much performance gain you get by this change alone?
Maybe you can define an arch specific primitive that default back to
atomic_read() if not defined.
Cheers,
Longman
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-11 2:35 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 2:35 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On 9/10/23 04:28, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The target of xchg_tail is to write the tail to the lock value, so
> adding prefetchw could help the next cmpxchg step, which may
> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> utilize this feature to give a forward guarantee, e.g., RISC-V
> XuanTie processors would block the snoop channel & irq for several
> cycles when prefetch.w instruction (from Zicbop extension) retired,
> which guarantees the next cmpxchg succeeds.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> kernel/locking/qspinlock.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> index d3f99060b60f..96b54e2ade86 100644
> --- a/kernel/locking/qspinlock.c
> +++ b/kernel/locking/qspinlock.c
> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> */
> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> {
> - u32 old, new, val = atomic_read(&lock->val);
> + u32 old, new, val;
> +
> + prefetchw(&lock->val);
> + val = atomic_read(&lock->val);
>
> for (;;) {
> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
That looks a bit weird. You pre-fetch and then immediately read it. How
much performance gain you get by this change alone?
Maybe you can define an arch specific primitive that default back to
atomic_read() if not defined.
Cheers,
Longman
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-11 2:35 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 2:35 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On 9/10/23 04:28, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The target of xchg_tail is to write the tail to the lock value, so
> adding prefetchw could help the next cmpxchg step, which may
> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> utilize this feature to give a forward guarantee, e.g., RISC-V
> XuanTie processors would block the snoop channel & irq for several
> cycles when prefetch.w instruction (from Zicbop extension) retired,
> which guarantees the next cmpxchg succeeds.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> kernel/locking/qspinlock.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> index d3f99060b60f..96b54e2ade86 100644
> --- a/kernel/locking/qspinlock.c
> +++ b/kernel/locking/qspinlock.c
> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> */
> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> {
> - u32 old, new, val = atomic_read(&lock->val);
> + u32 old, new, val;
> +
> + prefetchw(&lock->val);
> + val = atomic_read(&lock->val);
>
> for (;;) {
> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
That looks a bit weird. You pre-fetch and then immediately read it. How
much performance gain you get by this change alone?
Maybe you can define an arch specific primitive that default back to
atomic_read() if not defined.
Cheers,
Longman
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
2023-09-11 2:35 ` Waiman Long
@ 2023-09-11 3:09 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-11 3:09 UTC (permalink / raw)
To: Waiman Long
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, leobras, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
>
>
> On 9/10/23 04:28, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The target of xchg_tail is to write the tail to the lock value, so
> > adding prefetchw could help the next cmpxchg step, which may
> > decrease the cmpxchg retry loops of xchg_tail. Some processors may
> > utilize this feature to give a forward guarantee, e.g., RISC-V
> > XuanTie processors would block the snoop channel & irq for several
> > cycles when prefetch.w instruction (from Zicbop extension) retired,
> > which guarantees the next cmpxchg succeeds.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > kernel/locking/qspinlock.c | 5 ++++-
> > 1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> > index d3f99060b60f..96b54e2ade86 100644
> > --- a/kernel/locking/qspinlock.c
> > +++ b/kernel/locking/qspinlock.c
> > @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> > */
> > static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> > {
> > - u32 old, new, val = atomic_read(&lock->val);
> > + u32 old, new, val;
> > +
> > + prefetchw(&lock->val);
> > + val = atomic_read(&lock->val);
> >
> > for (;;) {
> > new = (val & _Q_LOCKED_PENDING_MASK) | tail;
>
> That looks a bit weird. You pre-fetch and then immediately read it. How
> much performance gain you get by this change alone?
>
> Maybe you can define an arch specific primitive that default back to
> atomic_read() if not defined.
Thx for the reply. This is a generic optimization point I would like
to talk about with you.
First, prefetchw() makes cacheline an exclusive state and serves for
the next cmpxchg loop semantic, which writes the idx_tail part of
arch_spin_lock. The atomic_read only makes cacheline in the shared
state, which couldn't give any guarantee for the next cmpxchg loop
semantic. Micro-architecture could utilize prefetchw() to provide a
strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
XuanTie processor would hold the exclusive cacheline state until the
next cmpxchg write success.
In the end, Let's go back to the principle: the xchg_tail is an atomic
swap operation that contains write eventually, so giving a prefetchw()
at the beginning is acceptable for all architectures..
>
> Cheers,
> Longman
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-11 3:09 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-11 3:09 UTC (permalink / raw)
To: Waiman Long
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, leobras, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
>
>
> On 9/10/23 04:28, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The target of xchg_tail is to write the tail to the lock value, so
> > adding prefetchw could help the next cmpxchg step, which may
> > decrease the cmpxchg retry loops of xchg_tail. Some processors may
> > utilize this feature to give a forward guarantee, e.g., RISC-V
> > XuanTie processors would block the snoop channel & irq for several
> > cycles when prefetch.w instruction (from Zicbop extension) retired,
> > which guarantees the next cmpxchg succeeds.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > kernel/locking/qspinlock.c | 5 ++++-
> > 1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> > index d3f99060b60f..96b54e2ade86 100644
> > --- a/kernel/locking/qspinlock.c
> > +++ b/kernel/locking/qspinlock.c
> > @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> > */
> > static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> > {
> > - u32 old, new, val = atomic_read(&lock->val);
> > + u32 old, new, val;
> > +
> > + prefetchw(&lock->val);
> > + val = atomic_read(&lock->val);
> >
> > for (;;) {
> > new = (val & _Q_LOCKED_PENDING_MASK) | tail;
>
> That looks a bit weird. You pre-fetch and then immediately read it. How
> much performance gain you get by this change alone?
>
> Maybe you can define an arch specific primitive that default back to
> atomic_read() if not defined.
Thx for the reply. This is a generic optimization point I would like
to talk about with you.
First, prefetchw() makes cacheline an exclusive state and serves for
the next cmpxchg loop semantic, which writes the idx_tail part of
arch_spin_lock. The atomic_read only makes cacheline in the shared
state, which couldn't give any guarantee for the next cmpxchg loop
semantic. Micro-architecture could utilize prefetchw() to provide a
strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
XuanTie processor would hold the exclusive cacheline state until the
next cmpxchg write success.
In the end, Let's go back to the principle: the xchg_tail is an atomic
swap operation that contains write eventually, so giving a prefetchw()
at the beginning is acceptable for all architectures..
>
> Cheers,
> Longman
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-10 19:45 ` Conor Dooley
@ 2023-09-11 3:36 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-11 3:36 UTC (permalink / raw)
To: Conor Dooley
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 3:45 AM Conor Dooley <conor@kernel.org> wrote:
>
> On Sun, Sep 10, 2023 at 05:49:13PM +0800, Guo Ren wrote:
> > On Sun, Sep 10, 2023 at 5:32 PM Conor Dooley <conor@kernel.org> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> > > > On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> > > > >
> > > > > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > > > >
> > > > > > Changlog:
> > > > > > V11:
> > > > > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > > > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > > > > - Remove abusing alternative framework and use jump_label instead.
> > > > >
> > > > > btw, I didn't say that using alternatives was the problem, it was
> > > > > abusing the errata framework to perform feature detection that I had
> > > > > a problem with. That's not changed in v11.
> > > > I've removed errata feature detection. The only related patches are:
> > > > - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > > > - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > >
> > > > Which one is your concern? Could you reply on the exact patch thread? Thx.
> > >
> > > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > >
> > > Please go back and re-read the comments I left on v11 about using the
> > > errata code for feature detection.
> > >
> > > > > A stronger forward progress guarantee is not an erratum, AFAICT.
> > >
> > > > Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
> > >
> > > "riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
> > > uses the errata framework to detect the presence of the stronger forward
> > > progress guarantee in v11.
> > Oh, thx for pointing it out. I could replace it with this:
> >
> > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > index 88690751f2ee..4be92766d3e3 100644
> > --- a/arch/riscv/kernel/setup.c
> > +++ b/arch/riscv/kernel/setup.c
> > @@ -310,7 +310,8 @@ static void __init riscv_spinlock_init(void)
> > {
> > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > if (!enable_qspinlock_key &&
> > - (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM) &&
> > + (sbi_get_mvendorid() != THEAD_VENDOR_ID)) {
> > static_branch_disable(&combo_qspinlock_key);
> > pr_info("Ticket spinlock: enabled\n");
> > } else {
>
> As I said on v11, I am opposed to feature probing using mvendorid & Co,
> partially due to the exact sort of check here to see if the kernel is
> running as a KVM guest. IMO, whether a platform has this stronger
KVM can't use any fairness lock, so forcing it using a Test-Set lock
or paravirt qspinlock is the right way. KVM is not a vendor platform.
> guarantee needs to be communicated by firmware, using ACPI or DT.
> I made some comments on v11, referring similar discussion about the
> thead vector stuff. Please go take a look at that.
I prefer forcing T-HEAD processors using qspinlock, but if all people
thought it must be in the ACPI or DT, I would compromise. Then, I
would delete the qspinlock cmdline param patch and move it into DT.
By the way, what's the kind of DT format? How about:
cpus {
#address-cells = <1>;
#size-cells = <0>;
+ qspinlock;
cpu0: cpu@0 {
compatible = "sifive,bullet0", "riscv";
device_type = "cpu";
i-cache-block-size = <64>;
i-cache-sets = <128>;
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-11 3:36 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-11 3:36 UTC (permalink / raw)
To: Conor Dooley
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 3:45 AM Conor Dooley <conor@kernel.org> wrote:
>
> On Sun, Sep 10, 2023 at 05:49:13PM +0800, Guo Ren wrote:
> > On Sun, Sep 10, 2023 at 5:32 PM Conor Dooley <conor@kernel.org> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> > > > On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> > > > >
> > > > > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > > > >
> > > > > > Changlog:
> > > > > > V11:
> > > > > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > > > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > > > > - Remove abusing alternative framework and use jump_label instead.
> > > > >
> > > > > btw, I didn't say that using alternatives was the problem, it was
> > > > > abusing the errata framework to perform feature detection that I had
> > > > > a problem with. That's not changed in v11.
> > > > I've removed errata feature detection. The only related patches are:
> > > > - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > > > - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > >
> > > > Which one is your concern? Could you reply on the exact patch thread? Thx.
> > >
> > > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > >
> > > Please go back and re-read the comments I left on v11 about using the
> > > errata code for feature detection.
> > >
> > > > > A stronger forward progress guarantee is not an erratum, AFAICT.
> > >
> > > > Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
> > >
> > > "riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
> > > uses the errata framework to detect the presence of the stronger forward
> > > progress guarantee in v11.
> > Oh, thx for pointing it out. I could replace it with this:
> >
> > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > index 88690751f2ee..4be92766d3e3 100644
> > --- a/arch/riscv/kernel/setup.c
> > +++ b/arch/riscv/kernel/setup.c
> > @@ -310,7 +310,8 @@ static void __init riscv_spinlock_init(void)
> > {
> > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > if (!enable_qspinlock_key &&
> > - (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM) &&
> > + (sbi_get_mvendorid() != THEAD_VENDOR_ID)) {
> > static_branch_disable(&combo_qspinlock_key);
> > pr_info("Ticket spinlock: enabled\n");
> > } else {
>
> As I said on v11, I am opposed to feature probing using mvendorid & Co,
> partially due to the exact sort of check here to see if the kernel is
> running as a KVM guest. IMO, whether a platform has this stronger
KVM can't use any fairness lock, so forcing it using a Test-Set lock
or paravirt qspinlock is the right way. KVM is not a vendor platform.
> guarantee needs to be communicated by firmware, using ACPI or DT.
> I made some comments on v11, referring similar discussion about the
> thead vector stuff. Please go take a look at that.
I prefer forcing T-HEAD processors using qspinlock, but if all people
thought it must be in the ACPI or DT, I would compromise. Then, I
would delete the qspinlock cmdline param patch and move it into DT.
By the way, what's the kind of DT format? How about:
cpus {
#address-cells = <1>;
#size-cells = <0>;
+ qspinlock;
cpu0: cpu@0 {
compatible = "sifive,bullet0", "riscv";
device_type = "cpu";
i-cache-block-size = <64>;
i-cache-sets = <128>;
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-11 3:36 ` Guo Ren
@ 2023-09-11 12:52 ` Conor Dooley
-1 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-11 12:52 UTC (permalink / raw)
To: Guo Ren
Cc: Conor Dooley, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
[-- Attachment #1.1: Type: text/plain, Size: 5010 bytes --]
On Mon, Sep 11, 2023 at 11:36:27AM +0800, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 3:45 AM Conor Dooley <conor@kernel.org> wrote:
> >
> > On Sun, Sep 10, 2023 at 05:49:13PM +0800, Guo Ren wrote:
> > > On Sun, Sep 10, 2023 at 5:32 PM Conor Dooley <conor@kernel.org> wrote:
> > > >
> > > > On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> > > > > On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> > > > > >
> > > > > > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > > > > >
> > > > > > > Changlog:
> > > > > > > V11:
> > > > > > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > > > > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > > > > > - Remove abusing alternative framework and use jump_label instead.
> > > > > >
> > > > > > btw, I didn't say that using alternatives was the problem, it was
> > > > > > abusing the errata framework to perform feature detection that I had
> > > > > > a problem with. That's not changed in v11.
> > > > > I've removed errata feature detection. The only related patches are:
> > > > > - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > > > > - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > > >
> > > > > Which one is your concern? Could you reply on the exact patch thread? Thx.
> > > >
> > > > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > >
> > > > Please go back and re-read the comments I left on v11 about using the
> > > > errata code for feature detection.
> > > >
> > > > > > A stronger forward progress guarantee is not an erratum, AFAICT.
> > > >
> > > > > Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
> > > >
> > > > "riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
> > > > uses the errata framework to detect the presence of the stronger forward
> > > > progress guarantee in v11.
> > > Oh, thx for pointing it out. I could replace it with this:
> > >
> > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > index 88690751f2ee..4be92766d3e3 100644
> > > --- a/arch/riscv/kernel/setup.c
> > > +++ b/arch/riscv/kernel/setup.c
> > > @@ -310,7 +310,8 @@ static void __init riscv_spinlock_init(void)
> > > {
> > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > if (!enable_qspinlock_key &&
> > > - (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM) &&
> > > + (sbi_get_mvendorid() != THEAD_VENDOR_ID)) {
> > > static_branch_disable(&combo_qspinlock_key);
> > > pr_info("Ticket spinlock: enabled\n");
> > > } else {
> >
> > As I said on v11, I am opposed to feature probing using mvendorid & Co,
> > partially due to the exact sort of check here to see if the kernel is
> > running as a KVM guest. IMO, whether a platform has this stronger
> KVM can't use any fairness lock, so forcing it using a Test-Set lock
> or paravirt qspinlock is the right way. KVM is not a vendor platform.
My point is that KVM should be telling the guest what additional features
it is capable of using, rather than the kernel making some assumptions
based on$vendorid etc that are invalid when the kernel is running as a
KVM guest.
If the mvendorid etc related assumptions were dropped, the kernel would
then default away from your qspinlock & there'd not be a need to
special-case KVM AFAICT.
> > guarantee needs to be communicated by firmware, using ACPI or DT.
> > I made some comments on v11, referring similar discussion about the
> > thead vector stuff. Please go take a look at that.
> I prefer forcing T-HEAD processors using qspinlock, but if all people
> thought it must be in the ACPI or DT, I would compromise. Then, I
> would delete the qspinlock cmdline param patch and move it into DT.
>
> By the way, what's the kind of DT format? How about:
I added the new "riscv,isa-extensions" property in part to make
communicating vendor extensions like this easier. Please try to use
that. "qspinlock" is software configuration though, the vendor extension
should focus on the guarantee of strong forward progress, since that is
the non-standard aspect of your IP.
A commandline property may still be desirable, to control the locking
method used, since the DT should be a description of the hardware, not
for configuring software policy in your operating system.
Thanks,
Conor.
> cpus {
> #address-cells = <1>;
> #size-cells = <0>;
> + qspinlock;
> cpu0: cpu@0 {
> compatible = "sifive,bullet0", "riscv";
> device_type = "cpu";
> i-cache-block-size = <64>;
> i-cache-sets = <128>;
>
> --
> Best Regards
> Guo Ren
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-11 12:52 ` Conor Dooley
0 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-11 12:52 UTC (permalink / raw)
To: Guo Ren
Cc: Conor Dooley, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
[-- Attachment #1: Type: text/plain, Size: 5010 bytes --]
On Mon, Sep 11, 2023 at 11:36:27AM +0800, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 3:45 AM Conor Dooley <conor@kernel.org> wrote:
> >
> > On Sun, Sep 10, 2023 at 05:49:13PM +0800, Guo Ren wrote:
> > > On Sun, Sep 10, 2023 at 5:32 PM Conor Dooley <conor@kernel.org> wrote:
> > > >
> > > > On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> > > > > On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> > > > > >
> > > > > > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > > > > >
> > > > > > > Changlog:
> > > > > > > V11:
> > > > > > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > > > > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > > > > > - Remove abusing alternative framework and use jump_label instead.
> > > > > >
> > > > > > btw, I didn't say that using alternatives was the problem, it was
> > > > > > abusing the errata framework to perform feature detection that I had
> > > > > > a problem with. That's not changed in v11.
> > > > > I've removed errata feature detection. The only related patches are:
> > > > > - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > > > > - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > > >
> > > > > Which one is your concern? Could you reply on the exact patch thread? Thx.
> > > >
> > > > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > >
> > > > Please go back and re-read the comments I left on v11 about using the
> > > > errata code for feature detection.
> > > >
> > > > > > A stronger forward progress guarantee is not an erratum, AFAICT.
> > > >
> > > > > Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
> > > >
> > > > "riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
> > > > uses the errata framework to detect the presence of the stronger forward
> > > > progress guarantee in v11.
> > > Oh, thx for pointing it out. I could replace it with this:
> > >
> > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > index 88690751f2ee..4be92766d3e3 100644
> > > --- a/arch/riscv/kernel/setup.c
> > > +++ b/arch/riscv/kernel/setup.c
> > > @@ -310,7 +310,8 @@ static void __init riscv_spinlock_init(void)
> > > {
> > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > if (!enable_qspinlock_key &&
> > > - (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM) &&
> > > + (sbi_get_mvendorid() != THEAD_VENDOR_ID)) {
> > > static_branch_disable(&combo_qspinlock_key);
> > > pr_info("Ticket spinlock: enabled\n");
> > > } else {
> >
> > As I said on v11, I am opposed to feature probing using mvendorid & Co,
> > partially due to the exact sort of check here to see if the kernel is
> > running as a KVM guest. IMO, whether a platform has this stronger
> KVM can't use any fairness lock, so forcing it using a Test-Set lock
> or paravirt qspinlock is the right way. KVM is not a vendor platform.
My point is that KVM should be telling the guest what additional features
it is capable of using, rather than the kernel making some assumptions
based on$vendorid etc that are invalid when the kernel is running as a
KVM guest.
If the mvendorid etc related assumptions were dropped, the kernel would
then default away from your qspinlock & there'd not be a need to
special-case KVM AFAICT.
> > guarantee needs to be communicated by firmware, using ACPI or DT.
> > I made some comments on v11, referring similar discussion about the
> > thead vector stuff. Please go take a look at that.
> I prefer forcing T-HEAD processors using qspinlock, but if all people
> thought it must be in the ACPI or DT, I would compromise. Then, I
> would delete the qspinlock cmdline param patch and move it into DT.
>
> By the way, what's the kind of DT format? How about:
I added the new "riscv,isa-extensions" property in part to make
communicating vendor extensions like this easier. Please try to use
that. "qspinlock" is software configuration though, the vendor extension
should focus on the guarantee of strong forward progress, since that is
the non-standard aspect of your IP.
A commandline property may still be desirable, to control the locking
method used, since the DT should be a description of the hardware, not
for configuring software policy in your operating system.
Thanks,
Conor.
> cpus {
> #address-cells = <1>;
> #size-cells = <0>;
> + qspinlock;
> cpu0: cpu@0 {
> compatible = "sifive,bullet0", "riscv";
> device_type = "cpu";
> i-cache-block-size = <64>;
> i-cache-sets = <128>;
>
> --
> Best Regards
> Guo Ren
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
2023-09-11 3:09 ` Guo Ren
(?)
@ 2023-09-11 13:03 ` Waiman Long
-1 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 13:03 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, leobras, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On 9/10/23 23:09, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
>>
>> On 9/10/23 04:28, guoren@kernel.org wrote:
>>> From: Guo Ren <guoren@linux.alibaba.com>
>>>
>>> The target of xchg_tail is to write the tail to the lock value, so
>>> adding prefetchw could help the next cmpxchg step, which may
>>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
>>> utilize this feature to give a forward guarantee, e.g., RISC-V
>>> XuanTie processors would block the snoop channel & irq for several
>>> cycles when prefetch.w instruction (from Zicbop extension) retired,
>>> which guarantees the next cmpxchg succeeds.
>>>
>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>> ---
>>> kernel/locking/qspinlock.c | 5 ++++-
>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
>>> index d3f99060b60f..96b54e2ade86 100644
>>> --- a/kernel/locking/qspinlock.c
>>> +++ b/kernel/locking/qspinlock.c
>>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
>>> */
>>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
>>> {
>>> - u32 old, new, val = atomic_read(&lock->val);
>>> + u32 old, new, val;
>>> +
>>> + prefetchw(&lock->val);
>>> + val = atomic_read(&lock->val);
>>>
>>> for (;;) {
>>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
>> That looks a bit weird. You pre-fetch and then immediately read it. How
>> much performance gain you get by this change alone?
>>
>> Maybe you can define an arch specific primitive that default back to
>> atomic_read() if not defined.
> Thx for the reply. This is a generic optimization point I would like
> to talk about with you.
>
> First, prefetchw() makes cacheline an exclusive state and serves for
> the next cmpxchg loop semantic, which writes the idx_tail part of
> arch_spin_lock. The atomic_read only makes cacheline in the shared
> state, which couldn't give any guarantee for the next cmpxchg loop
> semantic. Micro-architecture could utilize prefetchw() to provide a
> strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> XuanTie processor would hold the exclusive cacheline state until the
> next cmpxchg write success.
>
> In the end, Let's go back to the principle: the xchg_tail is an atomic
> swap operation that contains write eventually, so giving a prefetchw()
> at the beginning is acceptable for all architectures..
> ••••••••••••
I did realize afterward that prefetchw gets the cacheline in exclusive
state. I will suggest you mention that in your commit log as well as
adding a comment about its purpose in the code.
Thanks,
Longman
>> Cheers,
>> Longman
>>
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-11 13:03 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 13:03 UTC (permalink / raw)
To: Guo Ren
Cc: Guo Ren, kvm, linux-doc, peterz, catalin.marinas, bjorn, palmer,
virtualization, conor.dooley, jszhang, linux-riscv, will,
keescook, linux-arch, anup, linux-csky, xiaoguang.xing, mingo,
greentime.hu, ajones, alexghiti, paulmck, boqun.feng, rostedt,
leobras, paul.walmsley, tglx, rdunlap, wuwei2016, wefu
On 9/10/23 23:09, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
>>
>> On 9/10/23 04:28, guoren@kernel.org wrote:
>>> From: Guo Ren <guoren@linux.alibaba.com>
>>>
>>> The target of xchg_tail is to write the tail to the lock value, so
>>> adding prefetchw could help the next cmpxchg step, which may
>>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
>>> utilize this feature to give a forward guarantee, e.g., RISC-V
>>> XuanTie processors would block the snoop channel & irq for several
>>> cycles when prefetch.w instruction (from Zicbop extension) retired,
>>> which guarantees the next cmpxchg succeeds.
>>>
>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>> ---
>>> kernel/locking/qspinlock.c | 5 ++++-
>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
>>> index d3f99060b60f..96b54e2ade86 100644
>>> --- a/kernel/locking/qspinlock.c
>>> +++ b/kernel/locking/qspinlock.c
>>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
>>> */
>>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
>>> {
>>> - u32 old, new, val = atomic_read(&lock->val);
>>> + u32 old, new, val;
>>> +
>>> + prefetchw(&lock->val);
>>> + val = atomic_read(&lock->val);
>>>
>>> for (;;) {
>>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
>> That looks a bit weird. You pre-fetch and then immediately read it. How
>> much performance gain you get by this change alone?
>>
>> Maybe you can define an arch specific primitive that default back to
>> atomic_read() if not defined.
> Thx for the reply. This is a generic optimization point I would like
> to talk about with you.
>
> First, prefetchw() makes cacheline an exclusive state and serves for
> the next cmpxchg loop semantic, which writes the idx_tail part of
> arch_spin_lock. The atomic_read only makes cacheline in the shared
> state, which couldn't give any guarantee for the next cmpxchg loop
> semantic. Micro-architecture could utilize prefetchw() to provide a
> strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> XuanTie processor would hold the exclusive cacheline state until the
> next cmpxchg write success.
>
> In the end, Let's go back to the principle: the xchg_tail is an atomic
> swap operation that contains write eventually, so giving a prefetchw()
> at the beginning is acceptable for all architectures..
> ••••••••••••
I did realize afterward that prefetchw gets the cacheline in exclusive
state. I will suggest you mention that in your commit log as well as
adding a comment about its purpose in the code.
Thanks,
Longman
>> Cheers,
>> Longman
>>
>
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-11 13:03 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 13:03 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, leobras, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On 9/10/23 23:09, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
>>
>> On 9/10/23 04:28, guoren@kernel.org wrote:
>>> From: Guo Ren <guoren@linux.alibaba.com>
>>>
>>> The target of xchg_tail is to write the tail to the lock value, so
>>> adding prefetchw could help the next cmpxchg step, which may
>>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
>>> utilize this feature to give a forward guarantee, e.g., RISC-V
>>> XuanTie processors would block the snoop channel & irq for several
>>> cycles when prefetch.w instruction (from Zicbop extension) retired,
>>> which guarantees the next cmpxchg succeeds.
>>>
>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>> ---
>>> kernel/locking/qspinlock.c | 5 ++++-
>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
>>> index d3f99060b60f..96b54e2ade86 100644
>>> --- a/kernel/locking/qspinlock.c
>>> +++ b/kernel/locking/qspinlock.c
>>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
>>> */
>>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
>>> {
>>> - u32 old, new, val = atomic_read(&lock->val);
>>> + u32 old, new, val;
>>> +
>>> + prefetchw(&lock->val);
>>> + val = atomic_read(&lock->val);
>>>
>>> for (;;) {
>>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
>> That looks a bit weird. You pre-fetch and then immediately read it. How
>> much performance gain you get by this change alone?
>>
>> Maybe you can define an arch specific primitive that default back to
>> atomic_read() if not defined.
> Thx for the reply. This is a generic optimization point I would like
> to talk about with you.
>
> First, prefetchw() makes cacheline an exclusive state and serves for
> the next cmpxchg loop semantic, which writes the idx_tail part of
> arch_spin_lock. The atomic_read only makes cacheline in the shared
> state, which couldn't give any guarantee for the next cmpxchg loop
> semantic. Micro-architecture could utilize prefetchw() to provide a
> strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> XuanTie processor would hold the exclusive cacheline state until the
> next cmpxchg write success.
>
> In the end, Let's go back to the principle: the xchg_tail is an atomic
> swap operation that contains write eventually, so giving a prefetchw()
> at the beginning is acceptable for all architectures..
> ••••••••••••
I did realize afterward that prefetchw gets the cacheline in exclusive
state. I will suggest you mention that in your commit log as well as
adding a comment about its purpose in the code.
Thanks,
Longman
>> Cheers,
>> Longman
>>
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
2023-09-10 8:29 ` guoren
(?)
@ 2023-09-11 15:22 ` Waiman Long
-1 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 15:22 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On 9/10/23 04:29, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Allow cmdline to force the kernel to use queued_spinlock when
> CONFIG_RISCV_COMBO_SPINLOCKS=y.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 2 ++
> arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> 2 files changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 7dfb540c4f6c..61cacb8dfd0e 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -4693,6 +4693,8 @@
> [KNL] Number of legacy pty's. Overwrites compiled-in
> default number.
>
> + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> +
> qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> Set the time threshold in nanoseconds for the
> number of intra-node lock hand-offs before the
Your patch series is still based on top of numa-aware qspinlock patchset
which isn't upstream yet. Please rebase it without that as that will
cause merge conflict during upstream merge.
Cheers,
Longman
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
@ 2023-09-11 15:22 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 15:22 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, Guo Ren, kvm, linux-doc, linux-csky, virtualization,
linux-riscv
On 9/10/23 04:29, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Allow cmdline to force the kernel to use queued_spinlock when
> CONFIG_RISCV_COMBO_SPINLOCKS=y.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 2 ++
> arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> 2 files changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 7dfb540c4f6c..61cacb8dfd0e 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -4693,6 +4693,8 @@
> [KNL] Number of legacy pty's. Overwrites compiled-in
> default number.
>
> + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> +
> qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> Set the time threshold in nanoseconds for the
> number of intra-node lock hand-offs before the
Your patch series is still based on top of numa-aware qspinlock patchset
which isn't upstream yet. Please rebase it without that as that will
cause merge conflict during upstream merge.
Cheers,
Longman
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
@ 2023-09-11 15:22 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 15:22 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On 9/10/23 04:29, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Allow cmdline to force the kernel to use queued_spinlock when
> CONFIG_RISCV_COMBO_SPINLOCKS=y.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 2 ++
> arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> 2 files changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 7dfb540c4f6c..61cacb8dfd0e 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -4693,6 +4693,8 @@
> [KNL] Number of legacy pty's. Overwrites compiled-in
> default number.
>
> + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> +
> qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> Set the time threshold in nanoseconds for the
> number of intra-node lock hand-offs before the
Your patch series is still based on top of numa-aware qspinlock patchset
which isn't upstream yet. Please rebase it without that as that will
cause merge conflict during upstream merge.
Cheers,
Longman
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
2023-09-10 8:29 ` guoren
(?)
@ 2023-09-11 15:34 ` Waiman Long
-1 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 15:34 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On 9/10/23 04:29, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Allow cmdline to force the kernel to use queued_spinlock when
> CONFIG_RISCV_COMBO_SPINLOCKS=y.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 2 ++
> arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> 2 files changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 7dfb540c4f6c..61cacb8dfd0e 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -4693,6 +4693,8 @@
> [KNL] Number of legacy pty's. Overwrites compiled-in
> default number.
>
> + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> +
> qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> Set the time threshold in nanoseconds for the
> number of intra-node lock hand-offs before the
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index a447cf360a18..0f084f037651 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -270,6 +270,15 @@ static void __init parse_dtb(void)
> }
>
> #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> +bool enable_qspinlock_key = false;
You can use __ro_after_init qualifier for enable_qspinlock_key. BTW,
this is not a static key, just a simple flag. So what is the point of
the _key suffix?
Cheers,
Longman
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
@ 2023-09-11 15:34 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 15:34 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, Guo Ren, kvm, linux-doc, linux-csky, virtualization,
linux-riscv
On 9/10/23 04:29, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Allow cmdline to force the kernel to use queued_spinlock when
> CONFIG_RISCV_COMBO_SPINLOCKS=y.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 2 ++
> arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> 2 files changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 7dfb540c4f6c..61cacb8dfd0e 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -4693,6 +4693,8 @@
> [KNL] Number of legacy pty's. Overwrites compiled-in
> default number.
>
> + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> +
> qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> Set the time threshold in nanoseconds for the
> number of intra-node lock hand-offs before the
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index a447cf360a18..0f084f037651 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -270,6 +270,15 @@ static void __init parse_dtb(void)
> }
>
> #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> +bool enable_qspinlock_key = false;
You can use __ro_after_init qualifier for enable_qspinlock_key. BTW,
this is not a static key, just a simple flag. So what is the point of
the _key suffix?
Cheers,
Longman
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
@ 2023-09-11 15:34 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-11 15:34 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On 9/10/23 04:29, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Allow cmdline to force the kernel to use queued_spinlock when
> CONFIG_RISCV_COMBO_SPINLOCKS=y.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 2 ++
> arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> 2 files changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 7dfb540c4f6c..61cacb8dfd0e 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -4693,6 +4693,8 @@
> [KNL] Number of legacy pty's. Overwrites compiled-in
> default number.
>
> + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> +
> qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> Set the time threshold in nanoseconds for the
> number of intra-node lock hand-offs before the
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index a447cf360a18..0f084f037651 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -270,6 +270,15 @@ static void __init parse_dtb(void)
> }
>
> #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> +bool enable_qspinlock_key = false;
You can use __ro_after_init qualifier for enable_qspinlock_key. BTW,
this is not a static key, just a simple flag. So what is the point of
the _key suffix?
Cheers,
Longman
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 01/17] asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
2023-09-10 8:28 ` guoren
@ 2023-09-11 19:05 ` Leonardo Brás
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Brás @ 2023-09-11 19:05 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, conor.dooley, xiaoguang.xing, bjorn, alexghiti,
keescook, greentime.hu, ajones, jszhang, wefu, wuwei2016
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Sun, 2023-09-10 at 04:28 -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The arch_spinlock_t of qspinlock has contained the atomic_t val, which
> satisfies the ticket-lock requirement. Thus, unify the arch_spinlock_t
> into qspinlock_types.h. This is the preparation for the next combo
> spinlock.
>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> ---
> include/asm-generic/spinlock.h | 14 +++++++-------
> include/asm-generic/spinlock_types.h | 12 ++----------
> 2 files changed, 9 insertions(+), 17 deletions(-)
>
> diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
> index 90803a826ba0..4773334ee638 100644
> --- a/include/asm-generic/spinlock.h
> +++ b/include/asm-generic/spinlock.h
> @@ -32,7 +32,7 @@
>
> static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> {
> - u32 val = atomic_fetch_add(1<<16, lock);
> + u32 val = atomic_fetch_add(1<<16, &lock->val);
> u16 ticket = val >> 16;
>
> if (ticket == (u16)val)
> @@ -46,31 +46,31 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> * have no outstanding writes due to the atomic_fetch_add() the extra
> * orderings are free.
> */
> - atomic_cond_read_acquire(lock, ticket == (u16)VAL);
> + atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
> smp_mb();
> }
>
> static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> {
> - u32 old = atomic_read(lock);
> + u32 old = atomic_read(&lock->val);
>
> if ((old >> 16) != (old & 0xffff))
> return false;
>
> - return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */
> + return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
> }
>
> static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> {
> u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
> - u32 val = atomic_read(lock);
> + u32 val = atomic_read(&lock->val);
>
> smp_store_release(ptr, (u16)val + 1);
> }
>
> static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> {
> - u32 val = lock.counter;
> + u32 val = lock.val.counter;
>
> return ((val >> 16) == (val & 0xffff));
> }
This one seems to be different in torvalds/master, but I suppose it's because of
the requirement patches I have not merged.
> @@ -84,7 +84,7 @@ static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
>
> static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> {
> - u32 val = atomic_read(lock);
> + u32 val = atomic_read(&lock->val);
>
> return (s16)((val >> 16) - (val & 0xffff)) > 1;
> }
> diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spinlock_types.h
> index 8962bb730945..f534aa5de394 100644
> --- a/include/asm-generic/spinlock_types.h
> +++ b/include/asm-generic/spinlock_types.h
> @@ -3,15 +3,7 @@
> #ifndef __ASM_GENERIC_SPINLOCK_TYPES_H
> #define __ASM_GENERIC_SPINLOCK_TYPES_H
>
> -#include <linux/types.h>
> -typedef atomic_t arch_spinlock_t;
> -
> -/*
> - * qrwlock_types depends on arch_spinlock_t, so we must typedef that before the
> - * include.
> - */
> -#include <asm/qrwlock_types.h>
> -
> -#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0)
> +#include <asm-generic/qspinlock_types.h>
> +#include <asm-generic/qrwlock_types.h>
>
> #endif /* __ASM_GENERIC_SPINLOCK_TYPES_H */
FWIW, LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Just a suggestion: In this patch I could see a lot of usage changes to
arch_spinlock_t, and only at the end I could see the actual change in the .h
file.
In cases like this, it looks nicer to see the .h file first.
I recently found out about this git diff.orderFile option, which helps to
achieve exactly this.
I use the following git.orderfile, adapted from qemu:
############################################################################
#
# order file for git, to produce patches which are easier to review
# by diffing the important stuff like interface changes first.
#
# one-off usage:
# git diff -O scripts/git.orderfile ...
#
# add to git config:
# git config diff.orderFile scripts/git.orderfile
#
MAINTAINERS
# Documentation
Documentation/*
*.rst
*.rst.inc
# build system
Kbuild
Makefile*
*.mak
# semantic patches
*.cocci
# headers
*.h
*.h.inc
# code
*.c
*.c.inc
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 01/17] asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
@ 2023-09-11 19:05 ` Leonardo Brás
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Brás @ 2023-09-11 19:05 UTC (permalink / raw)
To: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, conor.dooley, xiaoguang.xing, bjorn, alexghiti,
keescook, greentime.hu, ajones, jszhang, wefu, wuwei2016
Cc: linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Sun, 2023-09-10 at 04:28 -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The arch_spinlock_t of qspinlock has contained the atomic_t val, which
> satisfies the ticket-lock requirement. Thus, unify the arch_spinlock_t
> into qspinlock_types.h. This is the preparation for the next combo
> spinlock.
>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> ---
> include/asm-generic/spinlock.h | 14 +++++++-------
> include/asm-generic/spinlock_types.h | 12 ++----------
> 2 files changed, 9 insertions(+), 17 deletions(-)
>
> diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
> index 90803a826ba0..4773334ee638 100644
> --- a/include/asm-generic/spinlock.h
> +++ b/include/asm-generic/spinlock.h
> @@ -32,7 +32,7 @@
>
> static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> {
> - u32 val = atomic_fetch_add(1<<16, lock);
> + u32 val = atomic_fetch_add(1<<16, &lock->val);
> u16 ticket = val >> 16;
>
> if (ticket == (u16)val)
> @@ -46,31 +46,31 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> * have no outstanding writes due to the atomic_fetch_add() the extra
> * orderings are free.
> */
> - atomic_cond_read_acquire(lock, ticket == (u16)VAL);
> + atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
> smp_mb();
> }
>
> static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> {
> - u32 old = atomic_read(lock);
> + u32 old = atomic_read(&lock->val);
>
> if ((old >> 16) != (old & 0xffff))
> return false;
>
> - return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */
> + return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
> }
>
> static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> {
> u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
> - u32 val = atomic_read(lock);
> + u32 val = atomic_read(&lock->val);
>
> smp_store_release(ptr, (u16)val + 1);
> }
>
> static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> {
> - u32 val = lock.counter;
> + u32 val = lock.val.counter;
>
> return ((val >> 16) == (val & 0xffff));
> }
This one seems to be different in torvalds/master, but I suppose it's because of
the requirement patches I have not merged.
> @@ -84,7 +84,7 @@ static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
>
> static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> {
> - u32 val = atomic_read(lock);
> + u32 val = atomic_read(&lock->val);
>
> return (s16)((val >> 16) - (val & 0xffff)) > 1;
> }
> diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spinlock_types.h
> index 8962bb730945..f534aa5de394 100644
> --- a/include/asm-generic/spinlock_types.h
> +++ b/include/asm-generic/spinlock_types.h
> @@ -3,15 +3,7 @@
> #ifndef __ASM_GENERIC_SPINLOCK_TYPES_H
> #define __ASM_GENERIC_SPINLOCK_TYPES_H
>
> -#include <linux/types.h>
> -typedef atomic_t arch_spinlock_t;
> -
> -/*
> - * qrwlock_types depends on arch_spinlock_t, so we must typedef that before the
> - * include.
> - */
> -#include <asm/qrwlock_types.h>
> -
> -#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0)
> +#include <asm-generic/qspinlock_types.h>
> +#include <asm-generic/qrwlock_types.h>
>
> #endif /* __ASM_GENERIC_SPINLOCK_TYPES_H */
FWIW, LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Just a suggestion: In this patch I could see a lot of usage changes to
arch_spinlock_t, and only at the end I could see the actual change in the .h
file.
In cases like this, it looks nicer to see the .h file first.
I recently found out about this git diff.orderFile option, which helps to
achieve exactly this.
I use the following git.orderfile, adapted from qemu:
############################################################################
#
# order file for git, to produce patches which are easier to review
# by diffing the important stuff like interface changes first.
#
# one-off usage:
# git diff -O scripts/git.orderfile ...
#
# add to git config:
# git config diff.orderFile scripts/git.orderfile
#
MAINTAINERS
# Documentation
Documentation/*
*.rst
*.rst.inc
# build system
Kbuild
Makefile*
*.mak
# semantic patches
*.cocci
# headers
*.h
*.h.inc
# code
*.c
*.c.inc
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
2023-09-11 15:22 ` Waiman Long
@ 2023-09-12 1:06 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-12 1:06 UTC (permalink / raw)
To: Waiman Long
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, leobras, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 11:22 PM Waiman Long <longman@redhat.com> wrote:
>
> On 9/10/23 04:29, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Allow cmdline to force the kernel to use queued_spinlock when
> > CONFIG_RISCV_COMBO_SPINLOCKS=y.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > Documentation/admin-guide/kernel-parameters.txt | 2 ++
> > arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> > 2 files changed, 17 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 7dfb540c4f6c..61cacb8dfd0e 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -4693,6 +4693,8 @@
> > [KNL] Number of legacy pty's. Overwrites compiled-in
> > default number.
> >
> > + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> > +
> > qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> > Set the time threshold in nanoseconds for the
> > number of intra-node lock hand-offs before the
>
> Your patch series is still based on top of numa-aware qspinlock patchset
> which isn't upstream yet. Please rebase it without that as that will
> cause merge conflict during upstream merge.
Okay, thx for pointing it out.
>
> Cheers,
> Longman
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
@ 2023-09-12 1:06 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-12 1:06 UTC (permalink / raw)
To: Waiman Long
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, leobras, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 11:22 PM Waiman Long <longman@redhat.com> wrote:
>
> On 9/10/23 04:29, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Allow cmdline to force the kernel to use queued_spinlock when
> > CONFIG_RISCV_COMBO_SPINLOCKS=y.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > Documentation/admin-guide/kernel-parameters.txt | 2 ++
> > arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> > 2 files changed, 17 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 7dfb540c4f6c..61cacb8dfd0e 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -4693,6 +4693,8 @@
> > [KNL] Number of legacy pty's. Overwrites compiled-in
> > default number.
> >
> > + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> > +
> > qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> > Set the time threshold in nanoseconds for the
> > number of intra-node lock hand-offs before the
>
> Your patch series is still based on top of numa-aware qspinlock patchset
> which isn't upstream yet. Please rebase it without that as that will
> cause merge conflict during upstream merge.
Okay, thx for pointing it out.
>
> Cheers,
> Longman
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
2023-09-11 15:34 ` Waiman Long
@ 2023-09-12 1:08 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-12 1:08 UTC (permalink / raw)
To: Waiman Long
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, leobras, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 11:34 PM Waiman Long <longman@redhat.com> wrote:
>
> On 9/10/23 04:29, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Allow cmdline to force the kernel to use queued_spinlock when
> > CONFIG_RISCV_COMBO_SPINLOCKS=y.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > Documentation/admin-guide/kernel-parameters.txt | 2 ++
> > arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> > 2 files changed, 17 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 7dfb540c4f6c..61cacb8dfd0e 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -4693,6 +4693,8 @@
> > [KNL] Number of legacy pty's. Overwrites compiled-in
> > default number.
> >
> > + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> > +
> > qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> > Set the time threshold in nanoseconds for the
> > number of intra-node lock hand-offs before the
> > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > index a447cf360a18..0f084f037651 100644
> > --- a/arch/riscv/kernel/setup.c
> > +++ b/arch/riscv/kernel/setup.c
> > @@ -270,6 +270,15 @@ static void __init parse_dtb(void)
> > }
> >
> > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > +bool enable_qspinlock_key = false;
>
> You can use __ro_after_init qualifier for enable_qspinlock_key. BTW,
> this is not a static key, just a simple flag. So what is the point of
> the _key suffix?
Okay, I would change it to:
bool enable_qspinlock_flag __ro_after_init = false;
>
> Cheers,
> Longman
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
@ 2023-09-12 1:08 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-12 1:08 UTC (permalink / raw)
To: Waiman Long
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, leobras, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 11:34 PM Waiman Long <longman@redhat.com> wrote:
>
> On 9/10/23 04:29, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Allow cmdline to force the kernel to use queued_spinlock when
> > CONFIG_RISCV_COMBO_SPINLOCKS=y.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > Documentation/admin-guide/kernel-parameters.txt | 2 ++
> > arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> > 2 files changed, 17 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 7dfb540c4f6c..61cacb8dfd0e 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -4693,6 +4693,8 @@
> > [KNL] Number of legacy pty's. Overwrites compiled-in
> > default number.
> >
> > + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> > +
> > qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> > Set the time threshold in nanoseconds for the
> > number of intra-node lock hand-offs before the
> > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > index a447cf360a18..0f084f037651 100644
> > --- a/arch/riscv/kernel/setup.c
> > +++ b/arch/riscv/kernel/setup.c
> > @@ -270,6 +270,15 @@ static void __init parse_dtb(void)
> > }
> >
> > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > +bool enable_qspinlock_key = false;
>
> You can use __ro_after_init qualifier for enable_qspinlock_key. BTW,
> this is not a static key, just a simple flag. So what is the point of
> the _key suffix?
Okay, I would change it to:
bool enable_qspinlock_flag __ro_after_init = false;
>
> Cheers,
> Longman
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
2023-09-11 13:03 ` Waiman Long
@ 2023-09-12 1:10 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-12 1:10 UTC (permalink / raw)
To: Waiman Long
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, leobras, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
>
> On 9/10/23 23:09, Guo Ren wrote:
> > On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
> >>
> >> On 9/10/23 04:28, guoren@kernel.org wrote:
> >>> From: Guo Ren <guoren@linux.alibaba.com>
> >>>
> >>> The target of xchg_tail is to write the tail to the lock value, so
> >>> adding prefetchw could help the next cmpxchg step, which may
> >>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> >>> utilize this feature to give a forward guarantee, e.g., RISC-V
> >>> XuanTie processors would block the snoop channel & irq for several
> >>> cycles when prefetch.w instruction (from Zicbop extension) retired,
> >>> which guarantees the next cmpxchg succeeds.
> >>>
> >>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> >>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >>> ---
> >>> kernel/locking/qspinlock.c | 5 ++++-
> >>> 1 file changed, 4 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> >>> index d3f99060b60f..96b54e2ade86 100644
> >>> --- a/kernel/locking/qspinlock.c
> >>> +++ b/kernel/locking/qspinlock.c
> >>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> >>> */
> >>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> >>> {
> >>> - u32 old, new, val = atomic_read(&lock->val);
> >>> + u32 old, new, val;
> >>> +
> >>> + prefetchw(&lock->val);
> >>> + val = atomic_read(&lock->val);
> >>>
> >>> for (;;) {
> >>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
> >> That looks a bit weird. You pre-fetch and then immediately read it. How
> >> much performance gain you get by this change alone?
> >>
> >> Maybe you can define an arch specific primitive that default back to
> >> atomic_read() if not defined.
> > Thx for the reply. This is a generic optimization point I would like
> > to talk about with you.
> >
> > First, prefetchw() makes cacheline an exclusive state and serves for
> > the next cmpxchg loop semantic, which writes the idx_tail part of
> > arch_spin_lock. The atomic_read only makes cacheline in the shared
> > state, which couldn't give any guarantee for the next cmpxchg loop
> > semantic. Micro-architecture could utilize prefetchw() to provide a
> > strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> > XuanTie processor would hold the exclusive cacheline state until the
> > next cmpxchg write success.
> >
> > In the end, Let's go back to the principle: the xchg_tail is an atomic
> > swap operation that contains write eventually, so giving a prefetchw()
> > at the beginning is acceptable for all architectures..
> > ••••••••••••
>
> I did realize afterward that prefetchw gets the cacheline in exclusive
> state. I will suggest you mention that in your commit log as well as
> adding a comment about its purpose in the code.
Okay, I would do that in v12, thx.
>
> Thanks,
> Longman
>
> >> Cheers,
> >> Longman
> >>
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-12 1:10 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-12 1:10 UTC (permalink / raw)
To: Waiman Long
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, leobras, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
>
> On 9/10/23 23:09, Guo Ren wrote:
> > On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
> >>
> >> On 9/10/23 04:28, guoren@kernel.org wrote:
> >>> From: Guo Ren <guoren@linux.alibaba.com>
> >>>
> >>> The target of xchg_tail is to write the tail to the lock value, so
> >>> adding prefetchw could help the next cmpxchg step, which may
> >>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> >>> utilize this feature to give a forward guarantee, e.g., RISC-V
> >>> XuanTie processors would block the snoop channel & irq for several
> >>> cycles when prefetch.w instruction (from Zicbop extension) retired,
> >>> which guarantees the next cmpxchg succeeds.
> >>>
> >>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> >>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >>> ---
> >>> kernel/locking/qspinlock.c | 5 ++++-
> >>> 1 file changed, 4 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> >>> index d3f99060b60f..96b54e2ade86 100644
> >>> --- a/kernel/locking/qspinlock.c
> >>> +++ b/kernel/locking/qspinlock.c
> >>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> >>> */
> >>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> >>> {
> >>> - u32 old, new, val = atomic_read(&lock->val);
> >>> + u32 old, new, val;
> >>> +
> >>> + prefetchw(&lock->val);
> >>> + val = atomic_read(&lock->val);
> >>>
> >>> for (;;) {
> >>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
> >> That looks a bit weird. You pre-fetch and then immediately read it. How
> >> much performance gain you get by this change alone?
> >>
> >> Maybe you can define an arch specific primitive that default back to
> >> atomic_read() if not defined.
> > Thx for the reply. This is a generic optimization point I would like
> > to talk about with you.
> >
> > First, prefetchw() makes cacheline an exclusive state and serves for
> > the next cmpxchg loop semantic, which writes the idx_tail part of
> > arch_spin_lock. The atomic_read only makes cacheline in the shared
> > state, which couldn't give any guarantee for the next cmpxchg loop
> > semantic. Micro-architecture could utilize prefetchw() to provide a
> > strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> > XuanTie processor would hold the exclusive cacheline state until the
> > next cmpxchg write success.
> >
> > In the end, Let's go back to the principle: the xchg_tail is an atomic
> > swap operation that contains write eventually, so giving a prefetchw()
> > at the beginning is acceptable for all architectures..
> > ••••••••••••
>
> I did realize afterward that prefetchw gets the cacheline in exclusive
> state. I will suggest you mention that in your commit log as well as
> adding a comment about its purpose in the code.
Okay, I would do that in v12, thx.
>
> Thanks,
> Longman
>
> >> Cheers,
> >> Longman
> >>
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-11 12:52 ` Conor Dooley
@ 2023-09-12 1:33 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-12 1:33 UTC (permalink / raw)
To: Conor Dooley
Cc: Conor Dooley, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 8:53 PM Conor Dooley <conor.dooley@microchip.com> wrote:
>
> On Mon, Sep 11, 2023 at 11:36:27AM +0800, Guo Ren wrote:
> > On Mon, Sep 11, 2023 at 3:45 AM Conor Dooley <conor@kernel.org> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 05:49:13PM +0800, Guo Ren wrote:
> > > > On Sun, Sep 10, 2023 at 5:32 PM Conor Dooley <conor@kernel.org> wrote:
> > > > >
> > > > > On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> > > > > > On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> > > > > > >
> > > > > > > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > > > > > >
> > > > > > > > Changlog:
> > > > > > > > V11:
> > > > > > > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > > > > > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > > > > > > - Remove abusing alternative framework and use jump_label instead.
> > > > > > >
> > > > > > > btw, I didn't say that using alternatives was the problem, it was
> > > > > > > abusing the errata framework to perform feature detection that I had
> > > > > > > a problem with. That's not changed in v11.
> > > > > > I've removed errata feature detection. The only related patches are:
> > > > > > - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > > > > > - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > > > >
> > > > > > Which one is your concern? Could you reply on the exact patch thread? Thx.
> > > > >
> > > > > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > > >
> > > > > Please go back and re-read the comments I left on v11 about using the
> > > > > errata code for feature detection.
> > > > >
> > > > > > > A stronger forward progress guarantee is not an erratum, AFAICT.
> > > > >
> > > > > > Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
> > > > >
> > > > > "riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
> > > > > uses the errata framework to detect the presence of the stronger forward
> > > > > progress guarantee in v11.
> > > > Oh, thx for pointing it out. I could replace it with this:
> > > >
> > > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > > index 88690751f2ee..4be92766d3e3 100644
> > > > --- a/arch/riscv/kernel/setup.c
> > > > +++ b/arch/riscv/kernel/setup.c
> > > > @@ -310,7 +310,8 @@ static void __init riscv_spinlock_init(void)
> > > > {
> > > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > if (!enable_qspinlock_key &&
> > > > - (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > > > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM) &&
> > > > + (sbi_get_mvendorid() != THEAD_VENDOR_ID)) {
> > > > static_branch_disable(&combo_qspinlock_key);
> > > > pr_info("Ticket spinlock: enabled\n");
> > > > } else {
> > >
> > > As I said on v11, I am opposed to feature probing using mvendorid & Co,
> > > partially due to the exact sort of check here to see if the kernel is
> > > running as a KVM guest. IMO, whether a platform has this stronger
>
> > KVM can't use any fairness lock, so forcing it using a Test-Set lock
> > or paravirt qspinlock is the right way. KVM is not a vendor platform.
>
> My point is that KVM should be telling the guest what additional features
> it is capable of using, rather than the kernel making some assumptions
> based on$vendorid etc that are invalid when the kernel is running as a
> KVM guest.
> If the mvendorid etc related assumptions were dropped, the kernel would
> then default away from your qspinlock & there'd not be a need to
> special-case KVM AFAICT.
>
> > > guarantee needs to be communicated by firmware, using ACPI or DT.
> > > I made some comments on v11, referring similar discussion about the
> > > thead vector stuff. Please go take a look at that.
> > I prefer forcing T-HEAD processors using qspinlock, but if all people
> > thought it must be in the ACPI or DT, I would compromise. Then, I
> > would delete the qspinlock cmdline param patch and move it into DT.
> >
> > By the way, what's the kind of DT format? How about:
>
> I added the new "riscv,isa-extensions" property in part to make
> communicating vendor extensions like this easier. Please try to use
> that. "qspinlock" is software configuration though, the vendor extension
> should focus on the guarantee of strong forward progress, since that is
> the non-standard aspect of your IP.
The qspinlock contains three paths:
- Native qspinlock, this is your strong forward progress.
- virt_spin_lock, for KVM guest when paravirt qspinlock disabled.
https://lore.kernel.org/linux-riscv/20230910082911.3378782-9-guoren@kernel.org/
- paravirt qspinlock, for KVM guest
So, we need a software configuration here, "riscv,isa-extensions" is
all about vendor extension.
>
> A commandline property may still be desirable, to control the locking
> method used, since the DT should be a description of the hardware, not
> for configuring software policy in your operating system.
Okay, I would keep the cmdline property.
>
> Thanks,
> Conor.
>
> > cpus {
> > #address-cells = <1>;
> > #size-cells = <0>;
> > + qspinlock;
> > cpu0: cpu@0 {
> > compatible = "sifive,bullet0", "riscv";
> > device_type = "cpu";
> > i-cache-block-size = <64>;
> > i-cache-sets = <128>;
> >
> > --
> > Best Regards
> > Guo Ren
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-12 1:33 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-12 1:33 UTC (permalink / raw)
To: Conor Dooley
Cc: Conor Dooley, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Mon, Sep 11, 2023 at 8:53 PM Conor Dooley <conor.dooley@microchip.com> wrote:
>
> On Mon, Sep 11, 2023 at 11:36:27AM +0800, Guo Ren wrote:
> > On Mon, Sep 11, 2023 at 3:45 AM Conor Dooley <conor@kernel.org> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 05:49:13PM +0800, Guo Ren wrote:
> > > > On Sun, Sep 10, 2023 at 5:32 PM Conor Dooley <conor@kernel.org> wrote:
> > > > >
> > > > > On Sun, Sep 10, 2023 at 05:16:46PM +0800, Guo Ren wrote:
> > > > > > On Sun, Sep 10, 2023 at 4:58 PM Conor Dooley <conor@kernel.org> wrote:
> > > > > > >
> > > > > > > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > > > > > >
> > > > > > > > Changlog:
> > > > > > > > V11:
> > > > > > > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > > > > > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > > > > > > - Remove abusing alternative framework and use jump_label instead.
> > > > > > >
> > > > > > > btw, I didn't say that using alternatives was the problem, it was
> > > > > > > abusing the errata framework to perform feature detection that I had
> > > > > > > a problem with. That's not changed in v11.
> > > > > > I've removed errata feature detection. The only related patches are:
> > > > > > - riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > > > > > - riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > > > >
> > > > > > Which one is your concern? Could you reply on the exact patch thread? Thx.
> > > > >
> > > > > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > > >
> > > > > Please go back and re-read the comments I left on v11 about using the
> > > > > errata code for feature detection.
> > > > >
> > > > > > > A stronger forward progress guarantee is not an erratum, AFAICT.
> > > > >
> > > > > > Sorry, there is no erratum of "stronger forward progress guarantee" in the V11.
> > > > >
> > > > > "riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors" still
> > > > > uses the errata framework to detect the presence of the stronger forward
> > > > > progress guarantee in v11.
> > > > Oh, thx for pointing it out. I could replace it with this:
> > > >
> > > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > > index 88690751f2ee..4be92766d3e3 100644
> > > > --- a/arch/riscv/kernel/setup.c
> > > > +++ b/arch/riscv/kernel/setup.c
> > > > @@ -310,7 +310,8 @@ static void __init riscv_spinlock_init(void)
> > > > {
> > > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > if (!enable_qspinlock_key &&
> > > > - (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > > > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM) &&
> > > > + (sbi_get_mvendorid() != THEAD_VENDOR_ID)) {
> > > > static_branch_disable(&combo_qspinlock_key);
> > > > pr_info("Ticket spinlock: enabled\n");
> > > > } else {
> > >
> > > As I said on v11, I am opposed to feature probing using mvendorid & Co,
> > > partially due to the exact sort of check here to see if the kernel is
> > > running as a KVM guest. IMO, whether a platform has this stronger
>
> > KVM can't use any fairness lock, so forcing it using a Test-Set lock
> > or paravirt qspinlock is the right way. KVM is not a vendor platform.
>
> My point is that KVM should be telling the guest what additional features
> it is capable of using, rather than the kernel making some assumptions
> based on$vendorid etc that are invalid when the kernel is running as a
> KVM guest.
> If the mvendorid etc related assumptions were dropped, the kernel would
> then default away from your qspinlock & there'd not be a need to
> special-case KVM AFAICT.
>
> > > guarantee needs to be communicated by firmware, using ACPI or DT.
> > > I made some comments on v11, referring similar discussion about the
> > > thead vector stuff. Please go take a look at that.
> > I prefer forcing T-HEAD processors using qspinlock, but if all people
> > thought it must be in the ACPI or DT, I would compromise. Then, I
> > would delete the qspinlock cmdline param patch and move it into DT.
> >
> > By the way, what's the kind of DT format? How about:
>
> I added the new "riscv,isa-extensions" property in part to make
> communicating vendor extensions like this easier. Please try to use
> that. "qspinlock" is software configuration though, the vendor extension
> should focus on the guarantee of strong forward progress, since that is
> the non-standard aspect of your IP.
The qspinlock contains three paths:
- Native qspinlock, this is your strong forward progress.
- virt_spin_lock, for KVM guest when paravirt qspinlock disabled.
https://lore.kernel.org/linux-riscv/20230910082911.3378782-9-guoren@kernel.org/
- paravirt qspinlock, for KVM guest
So, we need a software configuration here, "riscv,isa-extensions" is
all about vendor extension.
>
> A commandline property may still be desirable, to control the locking
> method used, since the DT should be a description of the hardware, not
> for configuring software policy in your operating system.
Okay, I would keep the cmdline property.
>
> Thanks,
> Conor.
>
> > cpus {
> > #address-cells = <1>;
> > #size-cells = <0>;
> > + qspinlock;
> > cpu0: cpu@0 {
> > compatible = "sifive,bullet0", "riscv";
> > device_type = "cpu";
> > i-cache-block-size = <64>;
> > i-cache-sets = <128>;
> >
> > --
> > Best Regards
> > Guo Ren
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-12 1:33 ` Guo Ren
@ 2023-09-12 8:07 ` Conor Dooley
-1 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-12 8:07 UTC (permalink / raw)
To: Guo Ren
Cc: Conor Dooley, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
[-- Attachment #1: Type: text/plain, Size: 1327 bytes --]
On Tue, Sep 12, 2023 at 09:33:57AM +0800, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 8:53 PM Conor Dooley <conor.dooley@microchip.com> wrote:
> > I added the new "riscv,isa-extensions" property in part to make
> > communicating vendor extensions like this easier. Please try to use
> > that. "qspinlock" is software configuration though, the vendor extension
> > should focus on the guarantee of strong forward progress, since that is
> > the non-standard aspect of your IP.
>
> The qspinlock contains three paths:
> - Native qspinlock, this is your strong forward progress.
> - virt_spin_lock, for KVM guest when paravirt qspinlock disabled.
> https://lore.kernel.org/linux-riscv/20230910082911.3378782-9-guoren@kernel.org/
> - paravirt qspinlock, for KVM guest
>
> So, we need a software configuration here, "riscv,isa-extensions" is
> all about vendor extension.
Ah right, yes it would only be able to be used to determine whether or
not the platform is capable of supporting these spinlocks, not whether or
not the kernel is a guest. I think I misinterpreted that snippet you posted,
thinking you were trying to disable your new spinlock for KVM, sorry.
On that note though, what about other sorts of guests? Will non-KVM
guests not also want to use this virt spinlock?
Thanks,
Conor.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-12 8:07 ` Conor Dooley
0 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-12 8:07 UTC (permalink / raw)
To: Guo Ren
Cc: Conor Dooley, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
[-- Attachment #1.1: Type: text/plain, Size: 1327 bytes --]
On Tue, Sep 12, 2023 at 09:33:57AM +0800, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 8:53 PM Conor Dooley <conor.dooley@microchip.com> wrote:
> > I added the new "riscv,isa-extensions" property in part to make
> > communicating vendor extensions like this easier. Please try to use
> > that. "qspinlock" is software configuration though, the vendor extension
> > should focus on the guarantee of strong forward progress, since that is
> > the non-standard aspect of your IP.
>
> The qspinlock contains three paths:
> - Native qspinlock, this is your strong forward progress.
> - virt_spin_lock, for KVM guest when paravirt qspinlock disabled.
> https://lore.kernel.org/linux-riscv/20230910082911.3378782-9-guoren@kernel.org/
> - paravirt qspinlock, for KVM guest
>
> So, we need a software configuration here, "riscv,isa-extensions" is
> all about vendor extension.
Ah right, yes it would only be able to be used to determine whether or
not the platform is capable of supporting these spinlocks, not whether or
not the kernel is a guest. I think I misinterpreted that snippet you posted,
thinking you were trying to disable your new spinlock for KVM, sorry.
On that note though, what about other sorts of guests? Will non-KVM
guests not also want to use this virt spinlock?
Thanks,
Conor.
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-12 8:07 ` Conor Dooley
@ 2023-09-12 10:58 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-12 10:58 UTC (permalink / raw)
To: Conor Dooley
Cc: Conor Dooley, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Tue, Sep 12, 2023 at 4:08 PM Conor Dooley <conor.dooley@microchip.com> wrote:
>
> On Tue, Sep 12, 2023 at 09:33:57AM +0800, Guo Ren wrote:
> > On Mon, Sep 11, 2023 at 8:53 PM Conor Dooley <conor.dooley@microchip.com> wrote:
>
> > > I added the new "riscv,isa-extensions" property in part to make
> > > communicating vendor extensions like this easier. Please try to use
> > > that. "qspinlock" is software configuration though, the vendor extension
> > > should focus on the guarantee of strong forward progress, since that is
> > > the non-standard aspect of your IP.
> >
> > The qspinlock contains three paths:
> > - Native qspinlock, this is your strong forward progress.
> > - virt_spin_lock, for KVM guest when paravirt qspinlock disabled.
> > https://lore.kernel.org/linux-riscv/20230910082911.3378782-9-guoren@kernel.org/
> > - paravirt qspinlock, for KVM guest
> >
> > So, we need a software configuration here, "riscv,isa-extensions" is
> > all about vendor extension.
>
> Ah right, yes it would only be able to be used to determine whether or
> not the platform is capable of supporting these spinlocks, not whether or
> not the kernel is a guest. I think I misinterpreted that snippet you posted,
> thinking you were trying to disable your new spinlock for KVM, sorry.
> On that note though, what about other sorts of guests? Will non-KVM
> guests not also want to use this virt spinlock?
I only put KVM guests here, and I can't answer other hypervisor that
is another topic.
>
> Thanks,
> Conor.
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-09-12 10:58 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-12 10:58 UTC (permalink / raw)
To: Conor Dooley
Cc: Conor Dooley, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, leobras,
linux-arch, linux-riscv, linux-doc, kvm, virtualization,
linux-csky, Guo Ren
On Tue, Sep 12, 2023 at 4:08 PM Conor Dooley <conor.dooley@microchip.com> wrote:
>
> On Tue, Sep 12, 2023 at 09:33:57AM +0800, Guo Ren wrote:
> > On Mon, Sep 11, 2023 at 8:53 PM Conor Dooley <conor.dooley@microchip.com> wrote:
>
> > > I added the new "riscv,isa-extensions" property in part to make
> > > communicating vendor extensions like this easier. Please try to use
> > > that. "qspinlock" is software configuration though, the vendor extension
> > > should focus on the guarantee of strong forward progress, since that is
> > > the non-standard aspect of your IP.
> >
> > The qspinlock contains three paths:
> > - Native qspinlock, this is your strong forward progress.
> > - virt_spin_lock, for KVM guest when paravirt qspinlock disabled.
> > https://lore.kernel.org/linux-riscv/20230910082911.3378782-9-guoren@kernel.org/
> > - paravirt qspinlock, for KVM guest
> >
> > So, we need a software configuration here, "riscv,isa-extensions" is
> > all about vendor extension.
>
> Ah right, yes it would only be able to be used to determine whether or
> not the platform is capable of supporting these spinlocks, not whether or
> not the kernel is a guest. I think I misinterpreted that snippet you posted,
> thinking you were trying to disable your new spinlock for KVM, sorry.
> On that note though, what about other sorts of guests? Will non-KVM
> guests not also want to use this virt spinlock?
I only put KVM guests here, and I can't answer other hypervisor that
is another topic.
>
> Thanks,
> Conor.
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 01/17] asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
2023-09-11 19:05 ` Leonardo Brás
@ 2023-09-13 1:55 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-13 1:55 UTC (permalink / raw)
To: Leonardo Brás
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 12, 2023 at 3:05 AM Leonardo Brás <leobras@redhat.com> wrote:
>
> On Sun, 2023-09-10 at 04:28 -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The arch_spinlock_t of qspinlock has contained the atomic_t val, which
> > satisfies the ticket-lock requirement. Thus, unify the arch_spinlock_t
> > into qspinlock_types.h. This is the preparation for the next combo
> > spinlock.
> >
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > ---
> > include/asm-generic/spinlock.h | 14 +++++++-------
> > include/asm-generic/spinlock_types.h | 12 ++----------
> > 2 files changed, 9 insertions(+), 17 deletions(-)
> >
> > diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
> > index 90803a826ba0..4773334ee638 100644
> > --- a/include/asm-generic/spinlock.h
> > +++ b/include/asm-generic/spinlock.h
> > @@ -32,7 +32,7 @@
> >
> > static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > {
> > - u32 val = atomic_fetch_add(1<<16, lock);
> > + u32 val = atomic_fetch_add(1<<16, &lock->val);
> > u16 ticket = val >> 16;
> >
> > if (ticket == (u16)val)
> > @@ -46,31 +46,31 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > * have no outstanding writes due to the atomic_fetch_add() the extra
> > * orderings are free.
> > */
> > - atomic_cond_read_acquire(lock, ticket == (u16)VAL);
> > + atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
> > smp_mb();
> > }
> >
> > static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > {
> > - u32 old = atomic_read(lock);
> > + u32 old = atomic_read(&lock->val);
> >
> > if ((old >> 16) != (old & 0xffff))
> > return false;
> >
> > - return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */
> > + return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
> > }
> >
> > static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > {
> > u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
> > - u32 val = atomic_read(lock);
> > + u32 val = atomic_read(&lock->val);
> >
> > smp_store_release(ptr, (u16)val + 1);
> > }
> >
> > static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > {
> > - u32 val = lock.counter;
> > + u32 val = lock.val.counter;
> >
> > return ((val >> 16) == (val & 0xffff));
> > }
>
> This one seems to be different in torvalds/master, but I suppose it's because of
> the requirement patches I have not merged.
>
> > @@ -84,7 +84,7 @@ static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> >
> > static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > {
> > - u32 val = atomic_read(lock);
> > + u32 val = atomic_read(&lock->val);
> >
> > return (s16)((val >> 16) - (val & 0xffff)) > 1;
> > }
> > diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spinlock_types.h
> > index 8962bb730945..f534aa5de394 100644
> > --- a/include/asm-generic/spinlock_types.h
> > +++ b/include/asm-generic/spinlock_types.h
> > @@ -3,15 +3,7 @@
> > #ifndef __ASM_GENERIC_SPINLOCK_TYPES_H
> > #define __ASM_GENERIC_SPINLOCK_TYPES_H
> >
> > -#include <linux/types.h>
> > -typedef atomic_t arch_spinlock_t;
> > -
> > -/*
> > - * qrwlock_types depends on arch_spinlock_t, so we must typedef that before the
> > - * include.
> > - */
> > -#include <asm/qrwlock_types.h>
> > -
> > -#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0)
> > +#include <asm-generic/qspinlock_types.h>
> > +#include <asm-generic/qrwlock_types.h>
> >
> > #endif /* __ASM_GENERIC_SPINLOCK_TYPES_H */
>
> FWIW, LGTM:
>
> Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
>
> Just a suggestion: In this patch I could see a lot of usage changes to
> arch_spinlock_t, and only at the end I could see the actual change in the .h
> file.
include/asm-generic/spinlock.h | 14 +++++++-------
include/asm-generic/spinlock_types.h | 12 ++----------
All are .h files. So, how to use git.orderfile?
>
> In cases like this, it looks nicer to see the .h file first.
>
> I recently found out about this git diff.orderFile option, which helps to
> achieve exactly this.
>
> I use the following git.orderfile, adapted from qemu:
>
> ############################################################################
> #
> # order file for git, to produce patches which are easier to review
> # by diffing the important stuff like interface changes first.
> #
> # one-off usage:
> # git diff -O scripts/git.orderfile ...
> #
> # add to git config:
> # git config diff.orderFile scripts/git.orderfile
> #
>
> MAINTAINERS
>
> # Documentation
> Documentation/*
> *.rst
> *.rst.inc
>
> # build system
> Kbuild
> Makefile*
> *.mak
>
> # semantic patches
> *.cocci
>
> # headers
> *.h
> *.h.inc
>
> # code
> *.c
> *.c.inc
>
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 01/17] asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
@ 2023-09-13 1:55 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-13 1:55 UTC (permalink / raw)
To: Leonardo Brás
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 12, 2023 at 3:05 AM Leonardo Brás <leobras@redhat.com> wrote:
>
> On Sun, 2023-09-10 at 04:28 -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The arch_spinlock_t of qspinlock has contained the atomic_t val, which
> > satisfies the ticket-lock requirement. Thus, unify the arch_spinlock_t
> > into qspinlock_types.h. This is the preparation for the next combo
> > spinlock.
> >
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > ---
> > include/asm-generic/spinlock.h | 14 +++++++-------
> > include/asm-generic/spinlock_types.h | 12 ++----------
> > 2 files changed, 9 insertions(+), 17 deletions(-)
> >
> > diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
> > index 90803a826ba0..4773334ee638 100644
> > --- a/include/asm-generic/spinlock.h
> > +++ b/include/asm-generic/spinlock.h
> > @@ -32,7 +32,7 @@
> >
> > static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > {
> > - u32 val = atomic_fetch_add(1<<16, lock);
> > + u32 val = atomic_fetch_add(1<<16, &lock->val);
> > u16 ticket = val >> 16;
> >
> > if (ticket == (u16)val)
> > @@ -46,31 +46,31 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > * have no outstanding writes due to the atomic_fetch_add() the extra
> > * orderings are free.
> > */
> > - atomic_cond_read_acquire(lock, ticket == (u16)VAL);
> > + atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
> > smp_mb();
> > }
> >
> > static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > {
> > - u32 old = atomic_read(lock);
> > + u32 old = atomic_read(&lock->val);
> >
> > if ((old >> 16) != (old & 0xffff))
> > return false;
> >
> > - return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */
> > + return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
> > }
> >
> > static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > {
> > u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
> > - u32 val = atomic_read(lock);
> > + u32 val = atomic_read(&lock->val);
> >
> > smp_store_release(ptr, (u16)val + 1);
> > }
> >
> > static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > {
> > - u32 val = lock.counter;
> > + u32 val = lock.val.counter;
> >
> > return ((val >> 16) == (val & 0xffff));
> > }
>
> This one seems to be different in torvalds/master, but I suppose it's because of
> the requirement patches I have not merged.
>
> > @@ -84,7 +84,7 @@ static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> >
> > static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > {
> > - u32 val = atomic_read(lock);
> > + u32 val = atomic_read(&lock->val);
> >
> > return (s16)((val >> 16) - (val & 0xffff)) > 1;
> > }
> > diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spinlock_types.h
> > index 8962bb730945..f534aa5de394 100644
> > --- a/include/asm-generic/spinlock_types.h
> > +++ b/include/asm-generic/spinlock_types.h
> > @@ -3,15 +3,7 @@
> > #ifndef __ASM_GENERIC_SPINLOCK_TYPES_H
> > #define __ASM_GENERIC_SPINLOCK_TYPES_H
> >
> > -#include <linux/types.h>
> > -typedef atomic_t arch_spinlock_t;
> > -
> > -/*
> > - * qrwlock_types depends on arch_spinlock_t, so we must typedef that before the
> > - * include.
> > - */
> > -#include <asm/qrwlock_types.h>
> > -
> > -#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0)
> > +#include <asm-generic/qspinlock_types.h>
> > +#include <asm-generic/qrwlock_types.h>
> >
> > #endif /* __ASM_GENERIC_SPINLOCK_TYPES_H */
>
> FWIW, LGTM:
>
> Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
>
> Just a suggestion: In this patch I could see a lot of usage changes to
> arch_spinlock_t, and only at the end I could see the actual change in the .h
> file.
include/asm-generic/spinlock.h | 14 +++++++-------
include/asm-generic/spinlock_types.h | 12 ++----------
All are .h files. So, how to use git.orderfile?
>
> In cases like this, it looks nicer to see the .h file first.
>
> I recently found out about this git diff.orderFile option, which helps to
> achieve exactly this.
>
> I use the following git.orderfile, adapted from qemu:
>
> ############################################################################
> #
> # order file for git, to produce patches which are easier to review
> # by diffing the important stuff like interface changes first.
> #
> # one-off usage:
> # git diff -O scripts/git.orderfile ...
> #
> # add to git config:
> # git config diff.orderFile scripts/git.orderfile
> #
>
> MAINTAINERS
>
> # Documentation
> Documentation/*
> *.rst
> *.rst.inc
>
> # build system
> Kbuild
> Makefile*
> *.mak
>
> # semantic patches
> *.cocci
>
> # headers
> *.h
> *.h.inc
>
> # code
> *.c
> *.c.inc
>
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 01/17] asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
2023-09-13 1:55 ` Guo Ren
@ 2023-09-13 7:59 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 7:59 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Wed, Sep 13, 2023 at 09:55:31AM +0800, Guo Ren wrote:
> On Tue, Sep 12, 2023 at 3:05 AM Leonardo Brás <leobras@redhat.com> wrote:
> >
> > On Sun, 2023-09-10 at 04:28 -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > The arch_spinlock_t of qspinlock has contained the atomic_t val, which
> > > satisfies the ticket-lock requirement. Thus, unify the arch_spinlock_t
> > > into qspinlock_types.h. This is the preparation for the next combo
> > > spinlock.
> > >
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > ---
> > > include/asm-generic/spinlock.h | 14 +++++++-------
> > > include/asm-generic/spinlock_types.h | 12 ++----------
> > > 2 files changed, 9 insertions(+), 17 deletions(-)
> > >
> > > diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
> > > index 90803a826ba0..4773334ee638 100644
> > > --- a/include/asm-generic/spinlock.h
> > > +++ b/include/asm-generic/spinlock.h
> > > @@ -32,7 +32,7 @@
> > >
> > > static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > > {
> > > - u32 val = atomic_fetch_add(1<<16, lock);
> > > + u32 val = atomic_fetch_add(1<<16, &lock->val);
> > > u16 ticket = val >> 16;
> > >
> > > if (ticket == (u16)val)
> > > @@ -46,31 +46,31 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > > * have no outstanding writes due to the atomic_fetch_add() the extra
> > > * orderings are free.
> > > */
> > > - atomic_cond_read_acquire(lock, ticket == (u16)VAL);
> > > + atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
> > > smp_mb();
> > > }
> > >
> > > static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > > {
> > > - u32 old = atomic_read(lock);
> > > + u32 old = atomic_read(&lock->val);
> > >
> > > if ((old >> 16) != (old & 0xffff))
> > > return false;
> > >
> > > - return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */
> > > + return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
> > > }
> > >
> > > static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > > {
> > > u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
> > > - u32 val = atomic_read(lock);
> > > + u32 val = atomic_read(&lock->val);
> > >
> > > smp_store_release(ptr, (u16)val + 1);
> > > }
> > >
> > > static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > > {
> > > - u32 val = lock.counter;
> > > + u32 val = lock.val.counter;
> > >
> > > return ((val >> 16) == (val & 0xffff));
> > > }
> >
> > This one seems to be different in torvalds/master, but I suppose it's because of
> > the requirement patches I have not merged.
> >
> > > @@ -84,7 +84,7 @@ static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> > >
> > > static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > > {
> > > - u32 val = atomic_read(lock);
> > > + u32 val = atomic_read(&lock->val);
> > >
> > > return (s16)((val >> 16) - (val & 0xffff)) > 1;
> > > }
> > > diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spinlock_types.h
> > > index 8962bb730945..f534aa5de394 100644
> > > --- a/include/asm-generic/spinlock_types.h
> > > +++ b/include/asm-generic/spinlock_types.h
> > > @@ -3,15 +3,7 @@
> > > #ifndef __ASM_GENERIC_SPINLOCK_TYPES_H
> > > #define __ASM_GENERIC_SPINLOCK_TYPES_H
> > >
> > > -#include <linux/types.h>
> > > -typedef atomic_t arch_spinlock_t;
> > > -
> > > -/*
> > > - * qrwlock_types depends on arch_spinlock_t, so we must typedef that before the
> > > - * include.
> > > - */
> > > -#include <asm/qrwlock_types.h>
> > > -
> > > -#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0)
> > > +#include <asm-generic/qspinlock_types.h>
> > > +#include <asm-generic/qrwlock_types.h>
> > >
> > > #endif /* __ASM_GENERIC_SPINLOCK_TYPES_H */
> >
> > FWIW, LGTM:
> >
> > Reviewed-by: Leonardo Bras <leobras@redhat.com>
> >
> >
> > Just a suggestion: In this patch I could see a lot of usage changes to
> > arch_spinlock_t, and only at the end I could see the actual change in the .h
> > file.
> include/asm-generic/spinlock.h | 14 +++++++-------
> include/asm-generic/spinlock_types.h | 12 ++----------
>
> All are .h files. So, how to use git.orderfile?
Yeap, you are right.
For some reason I got confused about seeing functions before type definition.
But in any way, we can get the same result with:
*types.h
*.h
*.c
Meaning 'spinlock_types.h' will appear before 'spinlock.h'.
After first suggesting this, I also sent a patch providing a default
orderFile for the kernel, and I also added this to the latest version:
https://lore.kernel.org/all/20230913075550.90934-2-leobras@redhat.com/
>
> >
> > In cases like this, it looks nicer to see the .h file first.
> >
> > I recently found out about this git diff.orderFile option, which helps to
> > achieve exactly this.
> >
> > I use the following git.orderfile, adapted from qemu:
> >
> > ############################################################################
> > #
> > # order file for git, to produce patches which are easier to review
> > # by diffing the important stuff like interface changes first.
> > #
> > # one-off usage:
> > # git diff -O scripts/git.orderfile ...
> > #
> > # add to git config:
> > # git config diff.orderFile scripts/git.orderfile
> > #
> >
> > MAINTAINERS
> >
> > # Documentation
> > Documentation/*
> > *.rst
> > *.rst.inc
> >
> > # build system
> > Kbuild
> > Makefile*
> > *.mak
> >
> > # semantic patches
> > *.cocci
> >
> > # headers
> > *.h
> > *.h.inc
> >
> > # code
> > *.c
> > *.c.inc
> >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 01/17] asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
@ 2023-09-13 7:59 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 7:59 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Wed, Sep 13, 2023 at 09:55:31AM +0800, Guo Ren wrote:
> On Tue, Sep 12, 2023 at 3:05 AM Leonardo Brás <leobras@redhat.com> wrote:
> >
> > On Sun, 2023-09-10 at 04:28 -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > The arch_spinlock_t of qspinlock has contained the atomic_t val, which
> > > satisfies the ticket-lock requirement. Thus, unify the arch_spinlock_t
> > > into qspinlock_types.h. This is the preparation for the next combo
> > > spinlock.
> > >
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > ---
> > > include/asm-generic/spinlock.h | 14 +++++++-------
> > > include/asm-generic/spinlock_types.h | 12 ++----------
> > > 2 files changed, 9 insertions(+), 17 deletions(-)
> > >
> > > diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
> > > index 90803a826ba0..4773334ee638 100644
> > > --- a/include/asm-generic/spinlock.h
> > > +++ b/include/asm-generic/spinlock.h
> > > @@ -32,7 +32,7 @@
> > >
> > > static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > > {
> > > - u32 val = atomic_fetch_add(1<<16, lock);
> > > + u32 val = atomic_fetch_add(1<<16, &lock->val);
> > > u16 ticket = val >> 16;
> > >
> > > if (ticket == (u16)val)
> > > @@ -46,31 +46,31 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > > * have no outstanding writes due to the atomic_fetch_add() the extra
> > > * orderings are free.
> > > */
> > > - atomic_cond_read_acquire(lock, ticket == (u16)VAL);
> > > + atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
> > > smp_mb();
> > > }
> > >
> > > static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > > {
> > > - u32 old = atomic_read(lock);
> > > + u32 old = atomic_read(&lock->val);
> > >
> > > if ((old >> 16) != (old & 0xffff))
> > > return false;
> > >
> > > - return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */
> > > + return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
> > > }
> > >
> > > static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > > {
> > > u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
> > > - u32 val = atomic_read(lock);
> > > + u32 val = atomic_read(&lock->val);
> > >
> > > smp_store_release(ptr, (u16)val + 1);
> > > }
> > >
> > > static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > > {
> > > - u32 val = lock.counter;
> > > + u32 val = lock.val.counter;
> > >
> > > return ((val >> 16) == (val & 0xffff));
> > > }
> >
> > This one seems to be different in torvalds/master, but I suppose it's because of
> > the requirement patches I have not merged.
> >
> > > @@ -84,7 +84,7 @@ static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> > >
> > > static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > > {
> > > - u32 val = atomic_read(lock);
> > > + u32 val = atomic_read(&lock->val);
> > >
> > > return (s16)((val >> 16) - (val & 0xffff)) > 1;
> > > }
> > > diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spinlock_types.h
> > > index 8962bb730945..f534aa5de394 100644
> > > --- a/include/asm-generic/spinlock_types.h
> > > +++ b/include/asm-generic/spinlock_types.h
> > > @@ -3,15 +3,7 @@
> > > #ifndef __ASM_GENERIC_SPINLOCK_TYPES_H
> > > #define __ASM_GENERIC_SPINLOCK_TYPES_H
> > >
> > > -#include <linux/types.h>
> > > -typedef atomic_t arch_spinlock_t;
> > > -
> > > -/*
> > > - * qrwlock_types depends on arch_spinlock_t, so we must typedef that before the
> > > - * include.
> > > - */
> > > -#include <asm/qrwlock_types.h>
> > > -
> > > -#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0)
> > > +#include <asm-generic/qspinlock_types.h>
> > > +#include <asm-generic/qrwlock_types.h>
> > >
> > > #endif /* __ASM_GENERIC_SPINLOCK_TYPES_H */
> >
> > FWIW, LGTM:
> >
> > Reviewed-by: Leonardo Bras <leobras@redhat.com>
> >
> >
> > Just a suggestion: In this patch I could see a lot of usage changes to
> > arch_spinlock_t, and only at the end I could see the actual change in the .h
> > file.
> include/asm-generic/spinlock.h | 14 +++++++-------
> include/asm-generic/spinlock_types.h | 12 ++----------
>
> All are .h files. So, how to use git.orderfile?
Yeap, you are right.
For some reason I got confused about seeing functions before type definition.
But in any way, we can get the same result with:
*types.h
*.h
*.c
Meaning 'spinlock_types.h' will appear before 'spinlock.h'.
After first suggesting this, I also sent a patch providing a default
orderFile for the kernel, and I also added this to the latest version:
https://lore.kernel.org/all/20230913075550.90934-2-leobras@redhat.com/
>
> >
> > In cases like this, it looks nicer to see the .h file first.
> >
> > I recently found out about this git diff.orderFile option, which helps to
> > achieve exactly this.
> >
> > I use the following git.orderfile, adapted from qemu:
> >
> > ############################################################################
> > #
> > # order file for git, to produce patches which are easier to review
> > # by diffing the important stuff like interface changes first.
> > #
> > # one-off usage:
> > # git diff -O scripts/git.orderfile ...
> > #
> > # add to git config:
> > # git config diff.orderFile scripts/git.orderfile
> > #
> >
> > MAINTAINERS
> >
> > # Documentation
> > Documentation/*
> > *.rst
> > *.rst.inc
> >
> > # build system
> > Kbuild
> > Makefile*
> > *.mak
> >
> > # semantic patches
> > *.cocci
> >
> > # headers
> > *.h
> > *.h.inc
> >
> > # code
> > *.c
> > *.c.inc
> >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 02/17] asm-generic: ticket-lock: Move into ticket_spinlock.h
2023-09-10 8:28 ` guoren
@ 2023-09-13 8:15 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 8:15 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:56AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Move ticket-lock definition into an independent file. This is the
> preparation for the next combo spinlock of riscv.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> include/asm-generic/spinlock.h | 87 +---------------------
> include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++++++++++++
> 2 files changed, 104 insertions(+), 86 deletions(-)
> create mode 100644 include/asm-generic/ticket_spinlock.h
>
> diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
> index 4773334ee638..970590baf61b 100644
> --- a/include/asm-generic/spinlock.h
> +++ b/include/asm-generic/spinlock.h
> @@ -1,94 +1,9 @@
> /* SPDX-License-Identifier: GPL-2.0 */
>
> -/*
> - * 'Generic' ticket-lock implementation.
> - *
> - * It relies on atomic_fetch_add() having well defined forward progress
> - * guarantees under contention. If your architecture cannot provide this, stick
> - * to a test-and-set lock.
> - *
> - * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a
> - * sub-word of the value. This is generally true for anything LL/SC although
> - * you'd be hard pressed to find anything useful in architecture specifications
> - * about this. If your architecture cannot do this you might be better off with
> - * a test-and-set.
> - *
> - * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence
> - * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along with
> - * a full fence after the spin to upgrade the otherwise-RCpc
> - * atomic_cond_read_acquire().
> - *
> - * The implementation uses smp_cond_load_acquire() to spin, so if the
> - * architecture has WFE like instructions to sleep instead of poll for word
> - * modifications be sure to implement that (see ARM64 for example).
> - *
> - */
> -
> #ifndef __ASM_GENERIC_SPINLOCK_H
> #define __ASM_GENERIC_SPINLOCK_H
>
> -#include <linux/atomic.h>
> -#include <asm-generic/spinlock_types.h>
> -
> -static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> -{
> - u32 val = atomic_fetch_add(1<<16, &lock->val);
> - u16 ticket = val >> 16;
> -
> - if (ticket == (u16)val)
> - return;
> -
> - /*
> - * atomic_cond_read_acquire() is RCpc, but rather than defining a
> - * custom cond_read_rcsc() here we just emit a full fence. We only
> - * need the prior reads before subsequent writes ordering from
> - * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we
> - * have no outstanding writes due to the atomic_fetch_add() the extra
> - * orderings are free.
> - */
> - atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
> - smp_mb();
> -}
> -
> -static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> -{
> - u32 old = atomic_read(&lock->val);
> -
> - if ((old >> 16) != (old & 0xffff))
> - return false;
> -
> - return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
> -}
> -
> -static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> -{
> - u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
> - u32 val = atomic_read(&lock->val);
> -
> - smp_store_release(ptr, (u16)val + 1);
> -}
> -
> -static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> -{
> - u32 val = lock.val.counter;
> -
> - return ((val >> 16) == (val & 0xffff));
> -}
> -
> -static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> -{
> - arch_spinlock_t val = READ_ONCE(*lock);
> -
> - return !arch_spin_value_unlocked(val);
> -}
> -
> -static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> -{
> - u32 val = atomic_read(&lock->val);
> -
> - return (s16)((val >> 16) - (val & 0xffff)) > 1;
> -}
> -
> +#include <asm-generic/ticket_spinlock.h>
> #include <asm/qrwlock.h>
>
> #endif /* __ASM_GENERIC_SPINLOCK_H */
> diff --git a/include/asm-generic/ticket_spinlock.h b/include/asm-generic/ticket_spinlock.h
> new file mode 100644
> index 000000000000..cfcff22b37b3
> --- /dev/null
> +++ b/include/asm-generic/ticket_spinlock.h
> @@ -0,0 +1,103 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * 'Generic' ticket-lock implementation.
> + *
> + * It relies on atomic_fetch_add() having well defined forward progress
> + * guarantees under contention. If your architecture cannot provide this, stick
> + * to a test-and-set lock.
> + *
> + * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a
> + * sub-word of the value. This is generally true for anything LL/SC although
> + * you'd be hard pressed to find anything useful in architecture specifications
> + * about this. If your architecture cannot do this you might be better off with
> + * a test-and-set.
> + *
> + * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence
> + * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along with
> + * a full fence after the spin to upgrade the otherwise-RCpc
> + * atomic_cond_read_acquire().
> + *
> + * The implementation uses smp_cond_load_acquire() to spin, so if the
> + * architecture has WFE like instructions to sleep instead of poll for word
> + * modifications be sure to implement that (see ARM64 for example).
> + *
> + */
> +
> +#ifndef __ASM_GENERIC_TICKET_SPINLOCK_H
> +#define __ASM_GENERIC_TICKET_SPINLOCK_H
> +
> +#include <linux/atomic.h>
> +#include <asm-generic/spinlock_types.h>
> +
> +static __always_inline void ticket_spin_lock(arch_spinlock_t *lock)
> +{
> + u32 val = atomic_fetch_add(1<<16, &lock->val);
> + u16 ticket = val >> 16;
> +
> + if (ticket == (u16)val)
> + return;
> +
> + /*
> + * atomic_cond_read_acquire() is RCpc, but rather than defining a
> + * custom cond_read_rcsc() here we just emit a full fence. We only
> + * need the prior reads before subsequent writes ordering from
> + * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we
> + * have no outstanding writes due to the atomic_fetch_add() the extra
> + * orderings are free.
> + */
> + atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
> + smp_mb();
> +}
> +
> +static __always_inline bool ticket_spin_trylock(arch_spinlock_t *lock)
> +{
> + u32 old = atomic_read(&lock->val);
> +
> + if ((old >> 16) != (old & 0xffff))
> + return false;
> +
> + return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
> +}
> +
> +static __always_inline void ticket_spin_unlock(arch_spinlock_t *lock)
> +{
> + u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
> + u32 val = atomic_read(&lock->val);
> +
> + smp_store_release(ptr, (u16)val + 1);
> +}
> +
> +static __always_inline int ticket_spin_value_unlocked(arch_spinlock_t lock)
> +{
> + u32 val = lock.val.counter;
> +
> + return ((val >> 16) == (val & 0xffff));
> +}
> +
> +static __always_inline int ticket_spin_is_locked(arch_spinlock_t *lock)
> +{
> + arch_spinlock_t val = READ_ONCE(*lock);
> +
> + return !ticket_spin_value_unlocked(val);
> +}
> +
> +static __always_inline int ticket_spin_is_contended(arch_spinlock_t *lock)
> +{
> + u32 val = atomic_read(&lock->val);
> +
> + return (s16)((val >> 16) - (val & 0xffff)) > 1;
> +}
> +
> +/*
> + * Remapping spinlock architecture specific functions to the corresponding
> + * ticket spinlock functions.
> + */
> +#define arch_spin_is_locked(l) ticket_spin_is_locked(l)
> +#define arch_spin_is_contended(l) ticket_spin_is_contended(l)
> +#define arch_spin_value_unlocked(l) ticket_spin_value_unlocked(l)
> +#define arch_spin_lock(l) ticket_spin_lock(l)
> +#define arch_spin_trylock(l) ticket_spin_trylock(l)
> +#define arch_spin_unlock(l) ticket_spin_unlock(l)
> +
> +#endif /* __ASM_GENERIC_TICKET_SPINLOCK_H */
IIUC here most of the file was moved, and the above defines are introduced.
I understand this pattern of creating the defines at the end of the file is
the same used in "asm-generic/qspinlock.h" but I don't actually think this
is a good way of doing this.
Instead of having those defines here (and similarly on
"asm-generic/qspinlock.h", I think it would be better to have those defines
in the arch-specific header including them, which would allow the arch to
include multiple spinlock versions and decide (compile-time, even run-time)
which version to use. It gives decision power to the arch code.
(And it would remove the need of undefining them on a later patch)
There are only 3 archs which use this arch-generic qspinlock, so should
not be a huge deal to have the defines copied there:
# git grep asm-generic/qspinlock.h
arch/loongarch/include/asm/qspinlock.h:16:#include <asm-generic/qspinlock.h>
arch/sparc/include/asm/qspinlock.h:6:#include <asm-generic/qspinlock.h>
arch/x86/include/asm/qspinlock.h:107:#include <asm-generic/qspinlock.h>
Other than that:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 02/17] asm-generic: ticket-lock: Move into ticket_spinlock.h
@ 2023-09-13 8:15 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 8:15 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:56AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Move ticket-lock definition into an independent file. This is the
> preparation for the next combo spinlock of riscv.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> include/asm-generic/spinlock.h | 87 +---------------------
> include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++++++++++++
> 2 files changed, 104 insertions(+), 86 deletions(-)
> create mode 100644 include/asm-generic/ticket_spinlock.h
>
> diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h
> index 4773334ee638..970590baf61b 100644
> --- a/include/asm-generic/spinlock.h
> +++ b/include/asm-generic/spinlock.h
> @@ -1,94 +1,9 @@
> /* SPDX-License-Identifier: GPL-2.0 */
>
> -/*
> - * 'Generic' ticket-lock implementation.
> - *
> - * It relies on atomic_fetch_add() having well defined forward progress
> - * guarantees under contention. If your architecture cannot provide this, stick
> - * to a test-and-set lock.
> - *
> - * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a
> - * sub-word of the value. This is generally true for anything LL/SC although
> - * you'd be hard pressed to find anything useful in architecture specifications
> - * about this. If your architecture cannot do this you might be better off with
> - * a test-and-set.
> - *
> - * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence
> - * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along with
> - * a full fence after the spin to upgrade the otherwise-RCpc
> - * atomic_cond_read_acquire().
> - *
> - * The implementation uses smp_cond_load_acquire() to spin, so if the
> - * architecture has WFE like instructions to sleep instead of poll for word
> - * modifications be sure to implement that (see ARM64 for example).
> - *
> - */
> -
> #ifndef __ASM_GENERIC_SPINLOCK_H
> #define __ASM_GENERIC_SPINLOCK_H
>
> -#include <linux/atomic.h>
> -#include <asm-generic/spinlock_types.h>
> -
> -static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> -{
> - u32 val = atomic_fetch_add(1<<16, &lock->val);
> - u16 ticket = val >> 16;
> -
> - if (ticket == (u16)val)
> - return;
> -
> - /*
> - * atomic_cond_read_acquire() is RCpc, but rather than defining a
> - * custom cond_read_rcsc() here we just emit a full fence. We only
> - * need the prior reads before subsequent writes ordering from
> - * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we
> - * have no outstanding writes due to the atomic_fetch_add() the extra
> - * orderings are free.
> - */
> - atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
> - smp_mb();
> -}
> -
> -static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> -{
> - u32 old = atomic_read(&lock->val);
> -
> - if ((old >> 16) != (old & 0xffff))
> - return false;
> -
> - return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
> -}
> -
> -static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> -{
> - u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
> - u32 val = atomic_read(&lock->val);
> -
> - smp_store_release(ptr, (u16)val + 1);
> -}
> -
> -static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> -{
> - u32 val = lock.val.counter;
> -
> - return ((val >> 16) == (val & 0xffff));
> -}
> -
> -static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> -{
> - arch_spinlock_t val = READ_ONCE(*lock);
> -
> - return !arch_spin_value_unlocked(val);
> -}
> -
> -static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> -{
> - u32 val = atomic_read(&lock->val);
> -
> - return (s16)((val >> 16) - (val & 0xffff)) > 1;
> -}
> -
> +#include <asm-generic/ticket_spinlock.h>
> #include <asm/qrwlock.h>
>
> #endif /* __ASM_GENERIC_SPINLOCK_H */
> diff --git a/include/asm-generic/ticket_spinlock.h b/include/asm-generic/ticket_spinlock.h
> new file mode 100644
> index 000000000000..cfcff22b37b3
> --- /dev/null
> +++ b/include/asm-generic/ticket_spinlock.h
> @@ -0,0 +1,103 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * 'Generic' ticket-lock implementation.
> + *
> + * It relies on atomic_fetch_add() having well defined forward progress
> + * guarantees under contention. If your architecture cannot provide this, stick
> + * to a test-and-set lock.
> + *
> + * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a
> + * sub-word of the value. This is generally true for anything LL/SC although
> + * you'd be hard pressed to find anything useful in architecture specifications
> + * about this. If your architecture cannot do this you might be better off with
> + * a test-and-set.
> + *
> + * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence
> + * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along with
> + * a full fence after the spin to upgrade the otherwise-RCpc
> + * atomic_cond_read_acquire().
> + *
> + * The implementation uses smp_cond_load_acquire() to spin, so if the
> + * architecture has WFE like instructions to sleep instead of poll for word
> + * modifications be sure to implement that (see ARM64 for example).
> + *
> + */
> +
> +#ifndef __ASM_GENERIC_TICKET_SPINLOCK_H
> +#define __ASM_GENERIC_TICKET_SPINLOCK_H
> +
> +#include <linux/atomic.h>
> +#include <asm-generic/spinlock_types.h>
> +
> +static __always_inline void ticket_spin_lock(arch_spinlock_t *lock)
> +{
> + u32 val = atomic_fetch_add(1<<16, &lock->val);
> + u16 ticket = val >> 16;
> +
> + if (ticket == (u16)val)
> + return;
> +
> + /*
> + * atomic_cond_read_acquire() is RCpc, but rather than defining a
> + * custom cond_read_rcsc() here we just emit a full fence. We only
> + * need the prior reads before subsequent writes ordering from
> + * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we
> + * have no outstanding writes due to the atomic_fetch_add() the extra
> + * orderings are free.
> + */
> + atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL);
> + smp_mb();
> +}
> +
> +static __always_inline bool ticket_spin_trylock(arch_spinlock_t *lock)
> +{
> + u32 old = atomic_read(&lock->val);
> +
> + if ((old >> 16) != (old & 0xffff))
> + return false;
> +
> + return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */
> +}
> +
> +static __always_inline void ticket_spin_unlock(arch_spinlock_t *lock)
> +{
> + u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN);
> + u32 val = atomic_read(&lock->val);
> +
> + smp_store_release(ptr, (u16)val + 1);
> +}
> +
> +static __always_inline int ticket_spin_value_unlocked(arch_spinlock_t lock)
> +{
> + u32 val = lock.val.counter;
> +
> + return ((val >> 16) == (val & 0xffff));
> +}
> +
> +static __always_inline int ticket_spin_is_locked(arch_spinlock_t *lock)
> +{
> + arch_spinlock_t val = READ_ONCE(*lock);
> +
> + return !ticket_spin_value_unlocked(val);
> +}
> +
> +static __always_inline int ticket_spin_is_contended(arch_spinlock_t *lock)
> +{
> + u32 val = atomic_read(&lock->val);
> +
> + return (s16)((val >> 16) - (val & 0xffff)) > 1;
> +}
> +
> +/*
> + * Remapping spinlock architecture specific functions to the corresponding
> + * ticket spinlock functions.
> + */
> +#define arch_spin_is_locked(l) ticket_spin_is_locked(l)
> +#define arch_spin_is_contended(l) ticket_spin_is_contended(l)
> +#define arch_spin_value_unlocked(l) ticket_spin_value_unlocked(l)
> +#define arch_spin_lock(l) ticket_spin_lock(l)
> +#define arch_spin_trylock(l) ticket_spin_trylock(l)
> +#define arch_spin_unlock(l) ticket_spin_unlock(l)
> +
> +#endif /* __ASM_GENERIC_TICKET_SPINLOCK_H */
IIUC here most of the file was moved, and the above defines are introduced.
I understand this pattern of creating the defines at the end of the file is
the same used in "asm-generic/qspinlock.h" but I don't actually think this
is a good way of doing this.
Instead of having those defines here (and similarly on
"asm-generic/qspinlock.h", I think it would be better to have those defines
in the arch-specific header including them, which would allow the arch to
include multiple spinlock versions and decide (compile-time, even run-time)
which version to use. It gives decision power to the arch code.
(And it would remove the need of undefining them on a later patch)
There are only 3 archs which use this arch-generic qspinlock, so should
not be a huge deal to have the defines copied there:
# git grep asm-generic/qspinlock.h
arch/loongarch/include/asm/qspinlock.h:16:#include <asm-generic/qspinlock.h>
arch/sparc/include/asm/qspinlock.h:6:#include <asm-generic/qspinlock.h>
arch/x86/include/asm/qspinlock.h:107:#include <asm-generic/qspinlock.h>
Other than that:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-10 8:28 ` guoren
@ 2023-09-13 8:49 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 8:49 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Cache-block prefetch instructions are HINTs to the hardware to
> indicate that software intends to perform a particular type of
> memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> improve the arch_xchg for qspinlock xchg_tail.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 15 +++++++++++++++
> arch/riscv/include/asm/cmpxchg.h | 4 +++-
> arch/riscv/include/asm/hwcap.h | 1 +
> arch/riscv/include/asm/insn-def.h | 5 +++++
> arch/riscv/include/asm/processor.h | 13 +++++++++++++
> arch/riscv/kernel/cpufeature.c | 1 +
> 6 files changed, 38 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index e9ae6fa232c3..2c346fe169c1 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
>
> If you don't know what to do here, say Y.
>
> +config RISCV_ISA_ZICBOP
> + bool "Zicbop extension support for cache block prefetch"
> + depends on MMU
> + depends on RISCV_ALTERNATIVE
> + default y
> + help
> + Adds support to dynamically detect the presence of the ZICBOP
> + extension (Cache Block Prefetch Operations) and enable its
> + usage.
> +
> + The Zicbop extension can be used to prefetch cache block for
> + read/write/instruction fetch.
> +
> + If you don't know what to do here, say Y.
> +
> config TOOLCHAIN_HAS_ZIHINTPAUSE
> bool
> default y
> diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> index 702725727671..56eff7a9d2d2 100644
> --- a/arch/riscv/include/asm/cmpxchg.h
> +++ b/arch/riscv/include/asm/cmpxchg.h
> @@ -11,6 +11,7 @@
>
> #include <asm/barrier.h>
> #include <asm/fence.h>
> +#include <asm/processor.h>
>
> #define __arch_xchg_masked(prepend, append, r, p, n) \
> ({ \
> @@ -25,6 +26,7 @@
> \
> __asm__ __volatile__ ( \
> prepend \
> + PREFETCHW_ASM(%5) \
> "0: lr.w %0, %2\n" \
> " and %1, %0, %z4\n" \
> " or %1, %1, %z3\n" \
> @@ -32,7 +34,7 @@
> " bnez %1, 0b\n" \
> append \
> : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> - : "rJ" (__newx), "rJ" (~__mask) \
> + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> : "memory"); \
> \
> r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> index b7b58258f6c7..78b7b8b53778 100644
> --- a/arch/riscv/include/asm/hwcap.h
> +++ b/arch/riscv/include/asm/hwcap.h
> @@ -58,6 +58,7 @@
> #define RISCV_ISA_EXT_ZICSR 40
> #define RISCV_ISA_EXT_ZIFENCEI 41
> #define RISCV_ISA_EXT_ZIHPM 42
> +#define RISCV_ISA_EXT_ZICBOP 43
>
> #define RISCV_ISA_EXT_MAX 64
>
> diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> index 6960beb75f32..dc590d331894 100644
> --- a/arch/riscv/include/asm/insn-def.h
> +++ b/arch/riscv/include/asm/insn-def.h
> @@ -134,6 +134,7 @@
>
> #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
>
> #define HFENCE_VVMA(vaddr, asid) \
> INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> @@ -196,4 +197,8 @@
> INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> RS1(base), SIMM12(4))
>
> +#define CBO_prefetchw(base) \
> + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> + RD(x0), RS1(base), RS2(x0))
> +
I understand that here you create the instruction via bitfield, following
the ISA, and this enables using instructions not available on the
toolchain.
It took me some time to find the document with this instruction, so please
add this to the commit msg:
https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
Page 23.
IIUC, the instruction is "prefetch.w".
Maybe I am missing something, but in the document the rs2 field
(PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
Is the above correct, or am I missing something?
Thanks!
Leo
> #endif /* __ASM_INSN_DEF_H */
> diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> index de9da852f78d..7ad3a24212e8 100644
> --- a/arch/riscv/include/asm/processor.h
> +++ b/arch/riscv/include/asm/processor.h
> @@ -12,6 +12,8 @@
> #include <vdso/processor.h>
>
> #include <asm/ptrace.h>
> +#include <asm/insn-def.h>
> +#include <asm/hwcap.h>
>
> #ifdef CONFIG_64BIT
> #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
>
> +#define ARCH_HAS_PREFETCHW
> +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> + CBO_prefetchw(base), \
> + 0, \
> + RISCV_ISA_EXT_ZICBOP, \
> + CONFIG_RISCV_ISA_ZICBOP)
> +static inline void prefetchw(const void *ptr)
> +{
> + asm volatile(PREFETCHW_ASM(%0)
> + : : "r" (ptr) : "memory");
> +}
>
> /* Do necessary setup to start up a newly executed thread. */
> extern void start_thread(struct pt_regs *regs,
> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> index ef7b4fd9e876..e0b897db0b97 100644
> --- a/arch/riscv/kernel/cpufeature.c
> +++ b/arch/riscv/kernel/cpufeature.c
> @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> --
> 2.36.1
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-13 8:49 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 8:49 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Cache-block prefetch instructions are HINTs to the hardware to
> indicate that software intends to perform a particular type of
> memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> improve the arch_xchg for qspinlock xchg_tail.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 15 +++++++++++++++
> arch/riscv/include/asm/cmpxchg.h | 4 +++-
> arch/riscv/include/asm/hwcap.h | 1 +
> arch/riscv/include/asm/insn-def.h | 5 +++++
> arch/riscv/include/asm/processor.h | 13 +++++++++++++
> arch/riscv/kernel/cpufeature.c | 1 +
> 6 files changed, 38 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index e9ae6fa232c3..2c346fe169c1 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
>
> If you don't know what to do here, say Y.
>
> +config RISCV_ISA_ZICBOP
> + bool "Zicbop extension support for cache block prefetch"
> + depends on MMU
> + depends on RISCV_ALTERNATIVE
> + default y
> + help
> + Adds support to dynamically detect the presence of the ZICBOP
> + extension (Cache Block Prefetch Operations) and enable its
> + usage.
> +
> + The Zicbop extension can be used to prefetch cache block for
> + read/write/instruction fetch.
> +
> + If you don't know what to do here, say Y.
> +
> config TOOLCHAIN_HAS_ZIHINTPAUSE
> bool
> default y
> diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> index 702725727671..56eff7a9d2d2 100644
> --- a/arch/riscv/include/asm/cmpxchg.h
> +++ b/arch/riscv/include/asm/cmpxchg.h
> @@ -11,6 +11,7 @@
>
> #include <asm/barrier.h>
> #include <asm/fence.h>
> +#include <asm/processor.h>
>
> #define __arch_xchg_masked(prepend, append, r, p, n) \
> ({ \
> @@ -25,6 +26,7 @@
> \
> __asm__ __volatile__ ( \
> prepend \
> + PREFETCHW_ASM(%5) \
> "0: lr.w %0, %2\n" \
> " and %1, %0, %z4\n" \
> " or %1, %1, %z3\n" \
> @@ -32,7 +34,7 @@
> " bnez %1, 0b\n" \
> append \
> : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> - : "rJ" (__newx), "rJ" (~__mask) \
> + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> : "memory"); \
> \
> r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> index b7b58258f6c7..78b7b8b53778 100644
> --- a/arch/riscv/include/asm/hwcap.h
> +++ b/arch/riscv/include/asm/hwcap.h
> @@ -58,6 +58,7 @@
> #define RISCV_ISA_EXT_ZICSR 40
> #define RISCV_ISA_EXT_ZIFENCEI 41
> #define RISCV_ISA_EXT_ZIHPM 42
> +#define RISCV_ISA_EXT_ZICBOP 43
>
> #define RISCV_ISA_EXT_MAX 64
>
> diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> index 6960beb75f32..dc590d331894 100644
> --- a/arch/riscv/include/asm/insn-def.h
> +++ b/arch/riscv/include/asm/insn-def.h
> @@ -134,6 +134,7 @@
>
> #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
>
> #define HFENCE_VVMA(vaddr, asid) \
> INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> @@ -196,4 +197,8 @@
> INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> RS1(base), SIMM12(4))
>
> +#define CBO_prefetchw(base) \
> + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> + RD(x0), RS1(base), RS2(x0))
> +
I understand that here you create the instruction via bitfield, following
the ISA, and this enables using instructions not available on the
toolchain.
It took me some time to find the document with this instruction, so please
add this to the commit msg:
https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
Page 23.
IIUC, the instruction is "prefetch.w".
Maybe I am missing something, but in the document the rs2 field
(PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
Is the above correct, or am I missing something?
Thanks!
Leo
> #endif /* __ASM_INSN_DEF_H */
> diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> index de9da852f78d..7ad3a24212e8 100644
> --- a/arch/riscv/include/asm/processor.h
> +++ b/arch/riscv/include/asm/processor.h
> @@ -12,6 +12,8 @@
> #include <vdso/processor.h>
>
> #include <asm/ptrace.h>
> +#include <asm/insn-def.h>
> +#include <asm/hwcap.h>
>
> #ifdef CONFIG_64BIT
> #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
>
> +#define ARCH_HAS_PREFETCHW
> +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> + CBO_prefetchw(base), \
> + 0, \
> + RISCV_ISA_EXT_ZICBOP, \
> + CONFIG_RISCV_ISA_ZICBOP)
> +static inline void prefetchw(const void *ptr)
> +{
> + asm volatile(PREFETCHW_ASM(%0)
> + : : "r" (ptr) : "memory");
> +}
>
> /* Do necessary setup to start up a newly executed thread. */
> extern void start_thread(struct pt_regs *regs,
> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> index ef7b4fd9e876..e0b897db0b97 100644
> --- a/arch/riscv/kernel/cpufeature.c
> +++ b/arch/riscv/kernel/cpufeature.c
> @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> --
> 2.36.1
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
2023-09-12 1:10 ` Guo Ren
@ 2023-09-13 8:55 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 8:55 UTC (permalink / raw)
To: Guo Ren
Cc: Waiman Long, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 12, 2023 at 09:10:08AM +0800, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
> >
> > On 9/10/23 23:09, Guo Ren wrote:
> > > On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
> > >>
> > >> On 9/10/23 04:28, guoren@kernel.org wrote:
> > >>> From: Guo Ren <guoren@linux.alibaba.com>
> > >>>
> > >>> The target of xchg_tail is to write the tail to the lock value, so
> > >>> adding prefetchw could help the next cmpxchg step, which may
> > >>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> > >>> utilize this feature to give a forward guarantee, e.g., RISC-V
> > >>> XuanTie processors would block the snoop channel & irq for several
> > >>> cycles when prefetch.w instruction (from Zicbop extension) retired,
> > >>> which guarantees the next cmpxchg succeeds.
> > >>>
> > >>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > >>> Signed-off-by: Guo Ren <guoren@kernel.org>
> > >>> ---
> > >>> kernel/locking/qspinlock.c | 5 ++++-
> > >>> 1 file changed, 4 insertions(+), 1 deletion(-)
> > >>>
> > >>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> > >>> index d3f99060b60f..96b54e2ade86 100644
> > >>> --- a/kernel/locking/qspinlock.c
> > >>> +++ b/kernel/locking/qspinlock.c
> > >>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> > >>> */
> > >>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> > >>> {
> > >>> - u32 old, new, val = atomic_read(&lock->val);
> > >>> + u32 old, new, val;
> > >>> +
> > >>> + prefetchw(&lock->val);
> > >>> + val = atomic_read(&lock->val);
> > >>>
> > >>> for (;;) {
> > >>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
> > >> That looks a bit weird. You pre-fetch and then immediately read it. How
> > >> much performance gain you get by this change alone?
> > >>
> > >> Maybe you can define an arch specific primitive that default back to
> > >> atomic_read() if not defined.
> > > Thx for the reply. This is a generic optimization point I would like
> > > to talk about with you.
> > >
> > > First, prefetchw() makes cacheline an exclusive state and serves for
> > > the next cmpxchg loop semantic, which writes the idx_tail part of
> > > arch_spin_lock. The atomic_read only makes cacheline in the shared
> > > state, which couldn't give any guarantee for the next cmpxchg loop
> > > semantic. Micro-architecture could utilize prefetchw() to provide a
> > > strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> > > XuanTie processor would hold the exclusive cacheline state until the
> > > next cmpxchg write success.
> > >
> > > In the end, Let's go back to the principle: the xchg_tail is an atomic
> > > swap operation that contains write eventually, so giving a prefetchw()
> > > at the beginning is acceptable for all architectures..
> > > ••••••••••••
> >
> > I did realize afterward that prefetchw gets the cacheline in exclusive
> > state. I will suggest you mention that in your commit log as well as
> > adding a comment about its purpose in the code.
> Okay, I would do that in v12, thx.
I would suggest adding a snippet from the ISA Extenstion doc:
"A prefetch.w instruction indicates to hardware that the cache block whose
effective address is the sum of the base address specified in rs1 and the
sign-extended offset encoded in imm[11:0], where imm[4:0] equals 0b00000,
is likely to be accessed by a data write (i.e. store) in the near future."
Other than that,
Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
> >
> > Thanks,
> > Longman
> >
> > >> Cheers,
> > >> Longman
> > >>
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-13 8:55 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 8:55 UTC (permalink / raw)
To: Guo Ren
Cc: Waiman Long, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 12, 2023 at 09:10:08AM +0800, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
> >
> > On 9/10/23 23:09, Guo Ren wrote:
> > > On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
> > >>
> > >> On 9/10/23 04:28, guoren@kernel.org wrote:
> > >>> From: Guo Ren <guoren@linux.alibaba.com>
> > >>>
> > >>> The target of xchg_tail is to write the tail to the lock value, so
> > >>> adding prefetchw could help the next cmpxchg step, which may
> > >>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> > >>> utilize this feature to give a forward guarantee, e.g., RISC-V
> > >>> XuanTie processors would block the snoop channel & irq for several
> > >>> cycles when prefetch.w instruction (from Zicbop extension) retired,
> > >>> which guarantees the next cmpxchg succeeds.
> > >>>
> > >>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > >>> Signed-off-by: Guo Ren <guoren@kernel.org>
> > >>> ---
> > >>> kernel/locking/qspinlock.c | 5 ++++-
> > >>> 1 file changed, 4 insertions(+), 1 deletion(-)
> > >>>
> > >>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> > >>> index d3f99060b60f..96b54e2ade86 100644
> > >>> --- a/kernel/locking/qspinlock.c
> > >>> +++ b/kernel/locking/qspinlock.c
> > >>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> > >>> */
> > >>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> > >>> {
> > >>> - u32 old, new, val = atomic_read(&lock->val);
> > >>> + u32 old, new, val;
> > >>> +
> > >>> + prefetchw(&lock->val);
> > >>> + val = atomic_read(&lock->val);
> > >>>
> > >>> for (;;) {
> > >>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
> > >> That looks a bit weird. You pre-fetch and then immediately read it. How
> > >> much performance gain you get by this change alone?
> > >>
> > >> Maybe you can define an arch specific primitive that default back to
> > >> atomic_read() if not defined.
> > > Thx for the reply. This is a generic optimization point I would like
> > > to talk about with you.
> > >
> > > First, prefetchw() makes cacheline an exclusive state and serves for
> > > the next cmpxchg loop semantic, which writes the idx_tail part of
> > > arch_spin_lock. The atomic_read only makes cacheline in the shared
> > > state, which couldn't give any guarantee for the next cmpxchg loop
> > > semantic. Micro-architecture could utilize prefetchw() to provide a
> > > strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> > > XuanTie processor would hold the exclusive cacheline state until the
> > > next cmpxchg write success.
> > >
> > > In the end, Let's go back to the principle: the xchg_tail is an atomic
> > > swap operation that contains write eventually, so giving a prefetchw()
> > > at the beginning is acceptable for all architectures..
> > > ••••••••••••
> >
> > I did realize afterward that prefetchw gets the cacheline in exclusive
> > state. I will suggest you mention that in your commit log as well as
> > adding a comment about its purpose in the code.
> Okay, I would do that in v12, thx.
I would suggest adding a snippet from the ISA Extenstion doc:
"A prefetch.w instruction indicates to hardware that the cache block whose
effective address is the sum of the base address specified in rs1 and the
sign-extended offset encoded in imm[11:0], where imm[4:0] equals 0b00000,
is likely to be accessed by a data write (i.e. store) in the near future."
Other than that,
Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
> >
> > Thanks,
> > Longman
> >
> > >> Cheers,
> > >> Longman
> > >>
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
2023-09-13 8:55 ` Leonardo Bras
@ 2023-09-13 12:52 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-13 12:52 UTC (permalink / raw)
To: Leonardo Bras
Cc: Waiman Long, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Wed, Sep 13, 2023 at 4:55 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Tue, Sep 12, 2023 at 09:10:08AM +0800, Guo Ren wrote:
> > On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
> > >
> > > On 9/10/23 23:09, Guo Ren wrote:
> > > > On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
> > > >>
> > > >> On 9/10/23 04:28, guoren@kernel.org wrote:
> > > >>> From: Guo Ren <guoren@linux.alibaba.com>
> > > >>>
> > > >>> The target of xchg_tail is to write the tail to the lock value, so
> > > >>> adding prefetchw could help the next cmpxchg step, which may
> > > >>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> > > >>> utilize this feature to give a forward guarantee, e.g., RISC-V
> > > >>> XuanTie processors would block the snoop channel & irq for several
> > > >>> cycles when prefetch.w instruction (from Zicbop extension) retired,
> > > >>> which guarantees the next cmpxchg succeeds.
> > > >>>
> > > >>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > >>> Signed-off-by: Guo Ren <guoren@kernel.org>
> > > >>> ---
> > > >>> kernel/locking/qspinlock.c | 5 ++++-
> > > >>> 1 file changed, 4 insertions(+), 1 deletion(-)
> > > >>>
> > > >>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> > > >>> index d3f99060b60f..96b54e2ade86 100644
> > > >>> --- a/kernel/locking/qspinlock.c
> > > >>> +++ b/kernel/locking/qspinlock.c
> > > >>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> > > >>> */
> > > >>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> > > >>> {
> > > >>> - u32 old, new, val = atomic_read(&lock->val);
> > > >>> + u32 old, new, val;
> > > >>> +
> > > >>> + prefetchw(&lock->val);
> > > >>> + val = atomic_read(&lock->val);
> > > >>>
> > > >>> for (;;) {
> > > >>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
> > > >> That looks a bit weird. You pre-fetch and then immediately read it. How
> > > >> much performance gain you get by this change alone?
> > > >>
> > > >> Maybe you can define an arch specific primitive that default back to
> > > >> atomic_read() if not defined.
> > > > Thx for the reply. This is a generic optimization point I would like
> > > > to talk about with you.
> > > >
> > > > First, prefetchw() makes cacheline an exclusive state and serves for
> > > > the next cmpxchg loop semantic, which writes the idx_tail part of
> > > > arch_spin_lock. The atomic_read only makes cacheline in the shared
> > > > state, which couldn't give any guarantee for the next cmpxchg loop
> > > > semantic. Micro-architecture could utilize prefetchw() to provide a
> > > > strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> > > > XuanTie processor would hold the exclusive cacheline state until the
> > > > next cmpxchg write success.
> > > >
> > > > In the end, Let's go back to the principle: the xchg_tail is an atomic
> > > > swap operation that contains write eventually, so giving a prefetchw()
> > > > at the beginning is acceptable for all architectures..
> > > > ••••••••••••
> > >
> > > I did realize afterward that prefetchw gets the cacheline in exclusive
> > > state. I will suggest you mention that in your commit log as well as
> > > adding a comment about its purpose in the code.
> > Okay, I would do that in v12, thx.
>
> I would suggest adding a snippet from the ISA Extenstion doc:
>
> "A prefetch.w instruction indicates to hardware that the cache block whose
> effective address is the sum of the base address specified in rs1 and the
> sign-extended offset encoded in imm[11:0], where imm[4:0] equals 0b00000,
> is likely to be accessed by a data write (i.e. store) in the near future."
Good point, thx.
>
> Other than that,
> Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
>
> >
> > >
> > > Thanks,
> > > Longman
> > >
> > > >> Cheers,
> > > >> Longman
> > > >>
> > > >
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-13 12:52 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-13 12:52 UTC (permalink / raw)
To: Leonardo Bras
Cc: Waiman Long, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Wed, Sep 13, 2023 at 4:55 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Tue, Sep 12, 2023 at 09:10:08AM +0800, Guo Ren wrote:
> > On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
> > >
> > > On 9/10/23 23:09, Guo Ren wrote:
> > > > On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
> > > >>
> > > >> On 9/10/23 04:28, guoren@kernel.org wrote:
> > > >>> From: Guo Ren <guoren@linux.alibaba.com>
> > > >>>
> > > >>> The target of xchg_tail is to write the tail to the lock value, so
> > > >>> adding prefetchw could help the next cmpxchg step, which may
> > > >>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> > > >>> utilize this feature to give a forward guarantee, e.g., RISC-V
> > > >>> XuanTie processors would block the snoop channel & irq for several
> > > >>> cycles when prefetch.w instruction (from Zicbop extension) retired,
> > > >>> which guarantees the next cmpxchg succeeds.
> > > >>>
> > > >>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > >>> Signed-off-by: Guo Ren <guoren@kernel.org>
> > > >>> ---
> > > >>> kernel/locking/qspinlock.c | 5 ++++-
> > > >>> 1 file changed, 4 insertions(+), 1 deletion(-)
> > > >>>
> > > >>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> > > >>> index d3f99060b60f..96b54e2ade86 100644
> > > >>> --- a/kernel/locking/qspinlock.c
> > > >>> +++ b/kernel/locking/qspinlock.c
> > > >>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> > > >>> */
> > > >>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> > > >>> {
> > > >>> - u32 old, new, val = atomic_read(&lock->val);
> > > >>> + u32 old, new, val;
> > > >>> +
> > > >>> + prefetchw(&lock->val);
> > > >>> + val = atomic_read(&lock->val);
> > > >>>
> > > >>> for (;;) {
> > > >>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
> > > >> That looks a bit weird. You pre-fetch and then immediately read it. How
> > > >> much performance gain you get by this change alone?
> > > >>
> > > >> Maybe you can define an arch specific primitive that default back to
> > > >> atomic_read() if not defined.
> > > > Thx for the reply. This is a generic optimization point I would like
> > > > to talk about with you.
> > > >
> > > > First, prefetchw() makes cacheline an exclusive state and serves for
> > > > the next cmpxchg loop semantic, which writes the idx_tail part of
> > > > arch_spin_lock. The atomic_read only makes cacheline in the shared
> > > > state, which couldn't give any guarantee for the next cmpxchg loop
> > > > semantic. Micro-architecture could utilize prefetchw() to provide a
> > > > strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> > > > XuanTie processor would hold the exclusive cacheline state until the
> > > > next cmpxchg write success.
> > > >
> > > > In the end, Let's go back to the principle: the xchg_tail is an atomic
> > > > swap operation that contains write eventually, so giving a prefetchw()
> > > > at the beginning is acceptable for all architectures..
> > > > ••••••••••••
> > >
> > > I did realize afterward that prefetchw gets the cacheline in exclusive
> > > state. I will suggest you mention that in your commit log as well as
> > > adding a comment about its purpose in the code.
> > Okay, I would do that in v12, thx.
>
> I would suggest adding a snippet from the ISA Extenstion doc:
>
> "A prefetch.w instruction indicates to hardware that the cache block whose
> effective address is the sum of the base address specified in rs1 and the
> sign-extended offset encoded in imm[11:0], where imm[4:0] equals 0b00000,
> is likely to be accessed by a data write (i.e. store) in the near future."
Good point, thx.
>
> Other than that,
> Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
>
> >
> > >
> > > Thanks,
> > > Longman
> > >
> > > >> Cheers,
> > > >> Longman
> > > >>
> > > >
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
2023-09-13 12:52 ` Guo Ren
(?)
@ 2023-09-13 13:06 ` Waiman Long
-1 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-13 13:06 UTC (permalink / raw)
To: Guo Ren, Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, linux-arch, linux-riscv, linux-doc,
kvm, virtualization, linux-csky, Guo Ren
On 9/13/23 08:52, Guo Ren wrote:
> On Wed, Sep 13, 2023 at 4:55 PM Leonardo Bras <leobras@redhat.com> wrote:
>> On Tue, Sep 12, 2023 at 09:10:08AM +0800, Guo Ren wrote:
>>> On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
>>>> On 9/10/23 23:09, Guo Ren wrote:
>>>>> On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
>>>>>> On 9/10/23 04:28, guoren@kernel.org wrote:
>>>>>>> From: Guo Ren <guoren@linux.alibaba.com>
>>>>>>>
>>>>>>> The target of xchg_tail is to write the tail to the lock value, so
>>>>>>> adding prefetchw could help the next cmpxchg step, which may
>>>>>>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
>>>>>>> utilize this feature to give a forward guarantee, e.g., RISC-V
>>>>>>> XuanTie processors would block the snoop channel & irq for several
>>>>>>> cycles when prefetch.w instruction (from Zicbop extension) retired,
>>>>>>> which guarantees the next cmpxchg succeeds.
>>>>>>>
>>>>>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
>>>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>>>>>> ---
>>>>>>> kernel/locking/qspinlock.c | 5 ++++-
>>>>>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>>>>>
>>>>>>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
>>>>>>> index d3f99060b60f..96b54e2ade86 100644
>>>>>>> --- a/kernel/locking/qspinlock.c
>>>>>>> +++ b/kernel/locking/qspinlock.c
>>>>>>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
>>>>>>> */
>>>>>>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
>>>>>>> {
>>>>>>> - u32 old, new, val = atomic_read(&lock->val);
>>>>>>> + u32 old, new, val;
>>>>>>> +
>>>>>>> + prefetchw(&lock->val);
>>>>>>> + val = atomic_read(&lock->val);
>>>>>>>
>>>>>>> for (;;) {
>>>>>>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
>>>>>> That looks a bit weird. You pre-fetch and then immediately read it. How
>>>>>> much performance gain you get by this change alone?
>>>>>>
>>>>>> Maybe you can define an arch specific primitive that default back to
>>>>>> atomic_read() if not defined.
>>>>> Thx for the reply. This is a generic optimization point I would like
>>>>> to talk about with you.
>>>>>
>>>>> First, prefetchw() makes cacheline an exclusive state and serves for
>>>>> the next cmpxchg loop semantic, which writes the idx_tail part of
>>>>> arch_spin_lock. The atomic_read only makes cacheline in the shared
>>>>> state, which couldn't give any guarantee for the next cmpxchg loop
>>>>> semantic. Micro-architecture could utilize prefetchw() to provide a
>>>>> strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
>>>>> XuanTie processor would hold the exclusive cacheline state until the
>>>>> next cmpxchg write success.
>>>>>
>>>>> In the end, Let's go back to the principle: the xchg_tail is an atomic
>>>>> swap operation that contains write eventually, so giving a prefetchw()
>>>>> at the beginning is acceptable for all architectures..
>>>>> ••••••••••••
>>>> I did realize afterward that prefetchw gets the cacheline in exclusive
>>>> state. I will suggest you mention that in your commit log as well as
>>>> adding a comment about its purpose in the code.
>>> Okay, I would do that in v12, thx.
>> I would suggest adding a snippet from the ISA Extenstion doc:
>>
>> "A prefetch.w instruction indicates to hardware that the cache block whose
>> effective address is the sum of the base address specified in rs1 and the
>> sign-extended offset encoded in imm[11:0], where imm[4:0] equals 0b00000,
>> is likely to be accessed by a data write (i.e. store) in the near future."
> Good point, thx.
qspinlock is generic code. I suppose this is for the RISCV architecture.
You can mention that in the commit log as an example, but I prefer more
generic comment especially in the code.
Cheers,
Longman
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-13 13:06 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-13 13:06 UTC (permalink / raw)
To: Guo Ren, Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, linux-arch, linux-riscv, linux-doc,
kvm, virtualization, linux-csky, Guo Ren
On 9/13/23 08:52, Guo Ren wrote:
> On Wed, Sep 13, 2023 at 4:55 PM Leonardo Bras <leobras@redhat.com> wrote:
>> On Tue, Sep 12, 2023 at 09:10:08AM +0800, Guo Ren wrote:
>>> On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
>>>> On 9/10/23 23:09, Guo Ren wrote:
>>>>> On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
>>>>>> On 9/10/23 04:28, guoren@kernel.org wrote:
>>>>>>> From: Guo Ren <guoren@linux.alibaba.com>
>>>>>>>
>>>>>>> The target of xchg_tail is to write the tail to the lock value, so
>>>>>>> adding prefetchw could help the next cmpxchg step, which may
>>>>>>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
>>>>>>> utilize this feature to give a forward guarantee, e.g., RISC-V
>>>>>>> XuanTie processors would block the snoop channel & irq for several
>>>>>>> cycles when prefetch.w instruction (from Zicbop extension) retired,
>>>>>>> which guarantees the next cmpxchg succeeds.
>>>>>>>
>>>>>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
>>>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>>>>>> ---
>>>>>>> kernel/locking/qspinlock.c | 5 ++++-
>>>>>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>>>>>
>>>>>>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
>>>>>>> index d3f99060b60f..96b54e2ade86 100644
>>>>>>> --- a/kernel/locking/qspinlock.c
>>>>>>> +++ b/kernel/locking/qspinlock.c
>>>>>>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
>>>>>>> */
>>>>>>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
>>>>>>> {
>>>>>>> - u32 old, new, val = atomic_read(&lock->val);
>>>>>>> + u32 old, new, val;
>>>>>>> +
>>>>>>> + prefetchw(&lock->val);
>>>>>>> + val = atomic_read(&lock->val);
>>>>>>>
>>>>>>> for (;;) {
>>>>>>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
>>>>>> That looks a bit weird. You pre-fetch and then immediately read it. How
>>>>>> much performance gain you get by this change alone?
>>>>>>
>>>>>> Maybe you can define an arch specific primitive that default back to
>>>>>> atomic_read() if not defined.
>>>>> Thx for the reply. This is a generic optimization point I would like
>>>>> to talk about with you.
>>>>>
>>>>> First, prefetchw() makes cacheline an exclusive state and serves for
>>>>> the next cmpxchg loop semantic, which writes the idx_tail part of
>>>>> arch_spin_lock. The atomic_read only makes cacheline in the shared
>>>>> state, which couldn't give any guarantee for the next cmpxchg loop
>>>>> semantic. Micro-architecture could utilize prefetchw() to provide a
>>>>> strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
>>>>> XuanTie processor would hold the exclusive cacheline state until the
>>>>> next cmpxchg write success.
>>>>>
>>>>> In the end, Let's go back to the principle: the xchg_tail is an atomic
>>>>> swap operation that contains write eventually, so giving a prefetchw()
>>>>> at the beginning is acceptable for all architectures..
>>>>> ••••••••••••
>>>> I did realize afterward that prefetchw gets the cacheline in exclusive
>>>> state. I will suggest you mention that in your commit log as well as
>>>> adding a comment about its purpose in the code.
>>> Okay, I would do that in v12, thx.
>> I would suggest adding a snippet from the ISA Extenstion doc:
>>
>> "A prefetch.w instruction indicates to hardware that the cache block whose
>> effective address is the sum of the base address specified in rs1 and the
>> sign-extended offset encoded in imm[11:0], where imm[4:0] equals 0b00000,
>> is likely to be accessed by a data write (i.e. store) in the near future."
> Good point, thx.
qspinlock is generic code. I suppose this is for the RISCV architecture.
You can mention that in the commit log as an example, but I prefer more
generic comment especially in the code.
Cheers,
Longman
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-13 13:06 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-13 13:06 UTC (permalink / raw)
To: Guo Ren, Leonardo Bras
Cc: Guo Ren, kvm, linux-doc, peterz, catalin.marinas, bjorn, palmer,
virtualization, conor.dooley, jszhang, linux-riscv, will,
keescook, linux-arch, anup, linux-csky, xiaoguang.xing, mingo,
greentime.hu, ajones, alexghiti, paulmck, boqun.feng, rostedt,
paul.walmsley, tglx, rdunlap, wuwei2016, wefu
On 9/13/23 08:52, Guo Ren wrote:
> On Wed, Sep 13, 2023 at 4:55 PM Leonardo Bras <leobras@redhat.com> wrote:
>> On Tue, Sep 12, 2023 at 09:10:08AM +0800, Guo Ren wrote:
>>> On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
>>>> On 9/10/23 23:09, Guo Ren wrote:
>>>>> On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
>>>>>> On 9/10/23 04:28, guoren@kernel.org wrote:
>>>>>>> From: Guo Ren <guoren@linux.alibaba.com>
>>>>>>>
>>>>>>> The target of xchg_tail is to write the tail to the lock value, so
>>>>>>> adding prefetchw could help the next cmpxchg step, which may
>>>>>>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
>>>>>>> utilize this feature to give a forward guarantee, e.g., RISC-V
>>>>>>> XuanTie processors would block the snoop channel & irq for several
>>>>>>> cycles when prefetch.w instruction (from Zicbop extension) retired,
>>>>>>> which guarantees the next cmpxchg succeeds.
>>>>>>>
>>>>>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
>>>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>>>>>> ---
>>>>>>> kernel/locking/qspinlock.c | 5 ++++-
>>>>>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>>>>>
>>>>>>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
>>>>>>> index d3f99060b60f..96b54e2ade86 100644
>>>>>>> --- a/kernel/locking/qspinlock.c
>>>>>>> +++ b/kernel/locking/qspinlock.c
>>>>>>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
>>>>>>> */
>>>>>>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
>>>>>>> {
>>>>>>> - u32 old, new, val = atomic_read(&lock->val);
>>>>>>> + u32 old, new, val;
>>>>>>> +
>>>>>>> + prefetchw(&lock->val);
>>>>>>> + val = atomic_read(&lock->val);
>>>>>>>
>>>>>>> for (;;) {
>>>>>>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
>>>>>> That looks a bit weird. You pre-fetch and then immediately read it. How
>>>>>> much performance gain you get by this change alone?
>>>>>>
>>>>>> Maybe you can define an arch specific primitive that default back to
>>>>>> atomic_read() if not defined.
>>>>> Thx for the reply. This is a generic optimization point I would like
>>>>> to talk about with you.
>>>>>
>>>>> First, prefetchw() makes cacheline an exclusive state and serves for
>>>>> the next cmpxchg loop semantic, which writes the idx_tail part of
>>>>> arch_spin_lock. The atomic_read only makes cacheline in the shared
>>>>> state, which couldn't give any guarantee for the next cmpxchg loop
>>>>> semantic. Micro-architecture could utilize prefetchw() to provide a
>>>>> strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
>>>>> XuanTie processor would hold the exclusive cacheline state until the
>>>>> next cmpxchg write success.
>>>>>
>>>>> In the end, Let's go back to the principle: the xchg_tail is an atomic
>>>>> swap operation that contains write eventually, so giving a prefetchw()
>>>>> at the beginning is acceptable for all architectures..
>>>>> ••••••••••••
>>>> I did realize afterward that prefetchw gets the cacheline in exclusive
>>>> state. I will suggest you mention that in your commit log as well as
>>>> adding a comment about its purpose in the code.
>>> Okay, I would do that in v12, thx.
>> I would suggest adding a snippet from the ISA Extenstion doc:
>>
>> "A prefetch.w instruction indicates to hardware that the cache block whose
>> effective address is the sum of the base address specified in rs1 and the
>> sign-extended offset encoded in imm[11:0], where imm[4:0] equals 0b00000,
>> is likely to be accessed by a data write (i.e. store) in the near future."
> Good point, thx.
qspinlock is generic code. I suppose this is for the RISCV architecture.
You can mention that in the commit log as an example, but I prefer more
generic comment especially in the code.
Cheers,
Longman
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
2023-09-10 8:28 ` guoren
@ 2023-09-13 20:28 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 20:28 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The requirements of qspinlock have been documented by commit:
> a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> atomics").
>
> Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> doesn't satisfy the requirements of qspinlock above, it won't prevent
> some riscv vendors from implementing a strong fwd guarantee LR/SC in
> microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> is the one.
>
> We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> comparison between qspinlock and ticket_lock on sg2042 (64 cores):
>
> sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> queued_spinlock 0.5109/0.00
> ticket_spinlock 0.5814/0.00
>
> perf futex/hash (+6.7%):
> queued_spinlock 1444393 operations/sec (+- 0.09%)
> ticket_spinlock 1353215 operations/sec (+- 0.15%)
>
> perf futex/wake-parallel (+8.6%):
> queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
>
> perf futex/requeue (+4.2%):
> queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
>
> System Benchmarks (+6.4%)
> queued_spinlock:
> System Benchmarks Index Values BASELINE RESULT INDEX
> Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> Double-Precision Whetstone 55.0 182422.8 33167.8
> Execl Throughput 43.0 13116.6 3050.4
> File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> Pipe Throughput 12440.0 23058600.5 18535.9
> Pipe-based Context Switching 4000.0 2835617.7 7089.0
> Process Creation 126.0 12537.3 995.0
> Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> System Call Overhead 15000.0 33308301.3 22205.5
> ========
> System Benchmarks Index Score 12426.1
>
> ticket_spinlock:
> System Benchmarks Index Values BASELINE RESULT INDEX
> Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> Double-Precision Whetstone 55.0 181921.0 33076.5
> Execl Throughput 43.0 12625.1 2936.1
> File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> Pipe Throughput 12440.0 20594018.7 16554.7
> Pipe-based Context Switching 4000.0 2571117.7 6427.8
> Process Creation 126.0 10798.4 857.0
> Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> System Call Overhead 15000.0 30766778.4 20511.2
> ========
> System Benchmarks Index Score 11670.7
>
> The qspinlock has a significant improvement on SOPHGO SG2042 64
> cores platform than the ticket_lock.
>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> ---
> arch/riscv/Kconfig | 16 ++++++++++++++++
> arch/riscv/include/asm/Kbuild | 3 ++-
> arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> 3 files changed, 35 insertions(+), 1 deletion(-)
> create mode 100644 arch/riscv/include/asm/spinlock.h
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 2c346fe169c1..7f39bfc75744 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -471,6 +471,22 @@ config NODES_SHIFT
> Specify the maximum number of NUMA Nodes available on the target
> system. Increases memory reserved to accommodate various tables.
>
> +choice
> + prompt "RISC-V spinlock type"
> + default RISCV_TICKET_SPINLOCKS
> +
> +config RISCV_TICKET_SPINLOCKS
> + bool "Using ticket spinlock"
> +
> +config RISCV_QUEUED_SPINLOCKS
> + bool "Using queued spinlock"
> + depends on SMP && MMU
> + select ARCH_USE_QUEUED_SPINLOCKS
> + help
> + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> + Otherwise, stay at ticket-lock.
> +endchoice
> +
> config RISCV_ALTERNATIVE
> bool
> depends on !XIP_KERNEL
> diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> index 504f8b7e72d4..a0dc85e4a754 100644
> --- a/arch/riscv/include/asm/Kbuild
> +++ b/arch/riscv/include/asm/Kbuild
> @@ -2,10 +2,11 @@
> generic-y += early_ioremap.h
> generic-y += flat.h
> generic-y += kvm_para.h
> +generic-y += mcs_spinlock.h
> generic-y += parport.h
> -generic-y += spinlock.h
IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
and include the asm-generic headers of mcs_spinlock and qspinlock.
In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
which seems the oposite of the above description (ticket spinlocks being
the standard).
Shouldn't ticket-spinlock.h also get included here?
(Also, I am probably missing something, as I dont' see the use of
mcs_spinlock here.)
> generic-y += spinlock_types.h
> generic-y += qrwlock.h
> generic-y += qrwlock_types.h
> +generic-y += qspinlock.h
> generic-y += user.h
> generic-y += vmlinux.lds.h
> diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> new file mode 100644
> index 000000000000..c644a92d4548
> --- /dev/null
> +++ b/arch/riscv/include/asm/spinlock.h
> @@ -0,0 +1,17 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#ifndef __ASM_RISCV_SPINLOCK_H
> +#define __ASM_RISCV_SPINLOCK_H
> +
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#define _Q_PENDING_LOOPS (1 << 9)
> +#endif
Any reason the above define couldn't be merged on the ifdef below?
> +
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#include <asm/qspinlock.h>
> +#include <asm/qrwlock.h>
> +#else
> +#include <asm-generic/spinlock.h>
> +#endif
> +
> +#endif /* __ASM_RISCV_SPINLOCK_H */
> --
> 2.36.1
>
Thanks!
Leo
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
@ 2023-09-13 20:28 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 20:28 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The requirements of qspinlock have been documented by commit:
> a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> atomics").
>
> Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> doesn't satisfy the requirements of qspinlock above, it won't prevent
> some riscv vendors from implementing a strong fwd guarantee LR/SC in
> microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> is the one.
>
> We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> comparison between qspinlock and ticket_lock on sg2042 (64 cores):
>
> sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> queued_spinlock 0.5109/0.00
> ticket_spinlock 0.5814/0.00
>
> perf futex/hash (+6.7%):
> queued_spinlock 1444393 operations/sec (+- 0.09%)
> ticket_spinlock 1353215 operations/sec (+- 0.15%)
>
> perf futex/wake-parallel (+8.6%):
> queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
>
> perf futex/requeue (+4.2%):
> queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
>
> System Benchmarks (+6.4%)
> queued_spinlock:
> System Benchmarks Index Values BASELINE RESULT INDEX
> Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> Double-Precision Whetstone 55.0 182422.8 33167.8
> Execl Throughput 43.0 13116.6 3050.4
> File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> Pipe Throughput 12440.0 23058600.5 18535.9
> Pipe-based Context Switching 4000.0 2835617.7 7089.0
> Process Creation 126.0 12537.3 995.0
> Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> System Call Overhead 15000.0 33308301.3 22205.5
> ========
> System Benchmarks Index Score 12426.1
>
> ticket_spinlock:
> System Benchmarks Index Values BASELINE RESULT INDEX
> Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> Double-Precision Whetstone 55.0 181921.0 33076.5
> Execl Throughput 43.0 12625.1 2936.1
> File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> Pipe Throughput 12440.0 20594018.7 16554.7
> Pipe-based Context Switching 4000.0 2571117.7 6427.8
> Process Creation 126.0 10798.4 857.0
> Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> System Call Overhead 15000.0 30766778.4 20511.2
> ========
> System Benchmarks Index Score 11670.7
>
> The qspinlock has a significant improvement on SOPHGO SG2042 64
> cores platform than the ticket_lock.
>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> ---
> arch/riscv/Kconfig | 16 ++++++++++++++++
> arch/riscv/include/asm/Kbuild | 3 ++-
> arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> 3 files changed, 35 insertions(+), 1 deletion(-)
> create mode 100644 arch/riscv/include/asm/spinlock.h
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 2c346fe169c1..7f39bfc75744 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -471,6 +471,22 @@ config NODES_SHIFT
> Specify the maximum number of NUMA Nodes available on the target
> system. Increases memory reserved to accommodate various tables.
>
> +choice
> + prompt "RISC-V spinlock type"
> + default RISCV_TICKET_SPINLOCKS
> +
> +config RISCV_TICKET_SPINLOCKS
> + bool "Using ticket spinlock"
> +
> +config RISCV_QUEUED_SPINLOCKS
> + bool "Using queued spinlock"
> + depends on SMP && MMU
> + select ARCH_USE_QUEUED_SPINLOCKS
> + help
> + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> + Otherwise, stay at ticket-lock.
> +endchoice
> +
> config RISCV_ALTERNATIVE
> bool
> depends on !XIP_KERNEL
> diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> index 504f8b7e72d4..a0dc85e4a754 100644
> --- a/arch/riscv/include/asm/Kbuild
> +++ b/arch/riscv/include/asm/Kbuild
> @@ -2,10 +2,11 @@
> generic-y += early_ioremap.h
> generic-y += flat.h
> generic-y += kvm_para.h
> +generic-y += mcs_spinlock.h
> generic-y += parport.h
> -generic-y += spinlock.h
IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
and include the asm-generic headers of mcs_spinlock and qspinlock.
In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
which seems the oposite of the above description (ticket spinlocks being
the standard).
Shouldn't ticket-spinlock.h also get included here?
(Also, I am probably missing something, as I dont' see the use of
mcs_spinlock here.)
> generic-y += spinlock_types.h
> generic-y += qrwlock.h
> generic-y += qrwlock_types.h
> +generic-y += qspinlock.h
> generic-y += user.h
> generic-y += vmlinux.lds.h
> diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> new file mode 100644
> index 000000000000..c644a92d4548
> --- /dev/null
> +++ b/arch/riscv/include/asm/spinlock.h
> @@ -0,0 +1,17 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#ifndef __ASM_RISCV_SPINLOCK_H
> +#define __ASM_RISCV_SPINLOCK_H
> +
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#define _Q_PENDING_LOOPS (1 << 9)
> +#endif
Any reason the above define couldn't be merged on the ifdef below?
> +
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#include <asm/qspinlock.h>
> +#include <asm/qrwlock.h>
> +#else
> +#include <asm-generic/spinlock.h>
> +#endif
> +
> +#endif /* __ASM_RISCV_SPINLOCK_H */
> --
> 2.36.1
>
Thanks!
Leo
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
2023-09-10 11:06 ` Guo Ren
@ 2023-09-13 20:37 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 20:37 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 07:06:23AM -0400, Guo Ren wrote:
> On Sun, Sep 10, 2023 at 04:29:00AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Combo spinlock could support queued and ticket in one Linux Image and
> > select them during boot time via errata mechanism. Here is the func
> > size (Bytes) comparison table below:
> >
> > TYPE : COMBO | TICKET | QUEUED
> > arch_spin_lock : 106 | 60 | 50
> > arch_spin_unlock : 54 | 36 | 26
> > arch_spin_trylock : 110 | 72 | 54
> > arch_spin_is_locked : 48 | 34 | 20
> > arch_spin_is_contended : 56 | 40 | 24
> > rch_spin_value_unlocked : 48 | 34 | 24
> >
> > One example of disassemble combo arch_spin_unlock:
> > 0xffffffff8000409c <+14>: nop # detour slot
> > 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
> > 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
> > 0xffffffff800040a8 <+26>: ld s0,8(sp)
> > 0xffffffff800040aa <+28>: addi sp,sp,16
> > 0xffffffff800040ac <+30>: ret
> > 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
> > 0xffffffff800040b0 <+34>: sext.w a5,a5
> > 0xffffffff800040b2 <+36>: fence rw,w
> > 0xffffffff800040b6 <+40>: addiw a5,a5,1
> > 0xffffffff800040b8 <+42>: slli a5,a5,0x30
> > 0xffffffff800040ba <+44>: srli a5,a5,0x30
> > 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
> > 0xffffffff800040c0 <+50>: ld s0,8(sp)
> > 0xffffffff800040c2 <+52>: addi sp,sp,16
> > 0xffffffff800040c4 <+54>: ret
> >
> > The qspinlock is smaller and faster than ticket-lock when all are in
> > fast-path, and combo spinlock could provide a compatible Linux Image
> > for different micro-arch design (weak/strict fwd guarantee LR/SC)
> > processors.
> >
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > ---
> > arch/riscv/Kconfig | 9 +++-
> > arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
> > arch/riscv/kernel/setup.c | 14 ++++++
> > 3 files changed, 98 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 7f39bfc75744..4bcff2860f48 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -473,7 +473,7 @@ config NODES_SHIFT
> >
> > choice
> > prompt "RISC-V spinlock type"
> > - default RISCV_TICKET_SPINLOCKS
> > + default RISCV_COMBO_SPINLOCKS
> >
> > config RISCV_TICKET_SPINLOCKS
> > bool "Using ticket spinlock"
> > @@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
> > help
> > Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > Otherwise, stay at ticket-lock.
> > +
> > +config RISCV_COMBO_SPINLOCKS
> > + bool "Using combo spinlock"
> > + depends on SMP && MMU
> > + select ARCH_USE_QUEUED_SPINLOCKS
> > + help
> > + Select queued spinlock or ticket-lock via errata.
> > endchoice
> >
> > config RISCV_ALTERNATIVE
> > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > index c644a92d4548..8ea0fee80652 100644
> > --- a/arch/riscv/include/asm/spinlock.h
> > +++ b/arch/riscv/include/asm/spinlock.h
> > @@ -7,11 +7,85 @@
> > #define _Q_PENDING_LOOPS (1 << 9)
> > #endif
> >
> > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > +#include <asm-generic/ticket_spinlock.h>
> > +
> > +#undef arch_spin_is_locked
> > +#undef arch_spin_is_contended
> > +#undef arch_spin_value_unlocked
> > +#undef arch_spin_lock
> > +#undef arch_spin_trylock
> > +#undef arch_spin_unlock
> > +
> > +#include <asm-generic/qspinlock.h>
> > +#include <linux/jump_label.h>
> > +
> > +#undef arch_spin_is_locked
> > +#undef arch_spin_is_contended
> > +#undef arch_spin_value_unlocked
> > +#undef arch_spin_lock
> > +#undef arch_spin_trylock
> > +#undef arch_spin_unlock
> Sorry, I forgot __no_arch_spinlock_redefine advice here. I would add it in v12.
> https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
>
Please check a reply to a previous patch I sent earlier: I think these
#undef can be avoided.
> > +
> > +DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > +
> > +static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + queued_spin_lock(lock);
> > + else
> > + ticket_spin_lock(lock);
> > +}
> > +
> > +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + return queued_spin_trylock(lock);
> > + else
> > + return ticket_spin_trylock(lock);
> > +}
> > +
> > +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + queued_spin_unlock(lock);
> > + else
> > + ticket_spin_unlock(lock);
> > +}
> > +
> > +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + return queued_spin_value_unlocked(lock);
> > + else
> > + return ticket_spin_value_unlocked(lock);
> > +}
> > +
> > +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + return queued_spin_is_locked(lock);
> > + else
> > + return ticket_spin_is_locked(lock);
> > +}
> > +
> > +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + return queued_spin_is_contended(lock);
> > + else
> > + return ticket_spin_is_contended(lock);
> > +}
> > +#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > +
> > #ifdef CONFIG_QUEUED_SPINLOCKS
> > #include <asm/qspinlock.h>
> > -#include <asm/qrwlock.h>
> > #else
> > -#include <asm-generic/spinlock.h>
> > +#include <asm-generic/ticket_spinlock.h>
> > #endif
> >
> > +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > +
> > +#include <asm/qrwlock.h>
> > +
> > #endif /* __ASM_RISCV_SPINLOCK_H */
> > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > index 32c2e1eb71bd..a447cf360a18 100644
> > --- a/arch/riscv/kernel/setup.c
> > +++ b/arch/riscv/kernel/setup.c
> > @@ -269,6 +269,18 @@ static void __init parse_dtb(void)
> > #endif
> > }
> >
> > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > +DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > +EXPORT_SYMBOL(combo_qspinlock_key);
> > +#endif
> > +
> > +static void __init riscv_spinlock_init(void)
> > +{
> > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > + static_branch_disable(&combo_qspinlock_key);
> > +#endif
> > +}
> > +
> > extern void __init init_rt_signal_env(void);
> >
> > void __init setup_arch(char **cmdline_p)
> > @@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
> > riscv_isa_extension_available(NULL, ZICBOM))
> > riscv_noncoherent_supported();
> > riscv_set_dma_cache_alignment();
> > +
> > + riscv_spinlock_init();
> > }
> >
> > static int __init topology_init(void)
> > --
> > 2.36.1
> >
> >
> > _______________________________________________
> > linux-riscv mailing list
> > linux-riscv@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-riscv
> >
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
@ 2023-09-13 20:37 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 20:37 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 07:06:23AM -0400, Guo Ren wrote:
> On Sun, Sep 10, 2023 at 04:29:00AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Combo spinlock could support queued and ticket in one Linux Image and
> > select them during boot time via errata mechanism. Here is the func
> > size (Bytes) comparison table below:
> >
> > TYPE : COMBO | TICKET | QUEUED
> > arch_spin_lock : 106 | 60 | 50
> > arch_spin_unlock : 54 | 36 | 26
> > arch_spin_trylock : 110 | 72 | 54
> > arch_spin_is_locked : 48 | 34 | 20
> > arch_spin_is_contended : 56 | 40 | 24
> > rch_spin_value_unlocked : 48 | 34 | 24
> >
> > One example of disassemble combo arch_spin_unlock:
> > 0xffffffff8000409c <+14>: nop # detour slot
> > 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
> > 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
> > 0xffffffff800040a8 <+26>: ld s0,8(sp)
> > 0xffffffff800040aa <+28>: addi sp,sp,16
> > 0xffffffff800040ac <+30>: ret
> > 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
> > 0xffffffff800040b0 <+34>: sext.w a5,a5
> > 0xffffffff800040b2 <+36>: fence rw,w
> > 0xffffffff800040b6 <+40>: addiw a5,a5,1
> > 0xffffffff800040b8 <+42>: slli a5,a5,0x30
> > 0xffffffff800040ba <+44>: srli a5,a5,0x30
> > 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
> > 0xffffffff800040c0 <+50>: ld s0,8(sp)
> > 0xffffffff800040c2 <+52>: addi sp,sp,16
> > 0xffffffff800040c4 <+54>: ret
> >
> > The qspinlock is smaller and faster than ticket-lock when all are in
> > fast-path, and combo spinlock could provide a compatible Linux Image
> > for different micro-arch design (weak/strict fwd guarantee LR/SC)
> > processors.
> >
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > ---
> > arch/riscv/Kconfig | 9 +++-
> > arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
> > arch/riscv/kernel/setup.c | 14 ++++++
> > 3 files changed, 98 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 7f39bfc75744..4bcff2860f48 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -473,7 +473,7 @@ config NODES_SHIFT
> >
> > choice
> > prompt "RISC-V spinlock type"
> > - default RISCV_TICKET_SPINLOCKS
> > + default RISCV_COMBO_SPINLOCKS
> >
> > config RISCV_TICKET_SPINLOCKS
> > bool "Using ticket spinlock"
> > @@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
> > help
> > Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > Otherwise, stay at ticket-lock.
> > +
> > +config RISCV_COMBO_SPINLOCKS
> > + bool "Using combo spinlock"
> > + depends on SMP && MMU
> > + select ARCH_USE_QUEUED_SPINLOCKS
> > + help
> > + Select queued spinlock or ticket-lock via errata.
> > endchoice
> >
> > config RISCV_ALTERNATIVE
> > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > index c644a92d4548..8ea0fee80652 100644
> > --- a/arch/riscv/include/asm/spinlock.h
> > +++ b/arch/riscv/include/asm/spinlock.h
> > @@ -7,11 +7,85 @@
> > #define _Q_PENDING_LOOPS (1 << 9)
> > #endif
> >
> > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > +#include <asm-generic/ticket_spinlock.h>
> > +
> > +#undef arch_spin_is_locked
> > +#undef arch_spin_is_contended
> > +#undef arch_spin_value_unlocked
> > +#undef arch_spin_lock
> > +#undef arch_spin_trylock
> > +#undef arch_spin_unlock
> > +
> > +#include <asm-generic/qspinlock.h>
> > +#include <linux/jump_label.h>
> > +
> > +#undef arch_spin_is_locked
> > +#undef arch_spin_is_contended
> > +#undef arch_spin_value_unlocked
> > +#undef arch_spin_lock
> > +#undef arch_spin_trylock
> > +#undef arch_spin_unlock
> Sorry, I forgot __no_arch_spinlock_redefine advice here. I would add it in v12.
> https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
>
Please check a reply to a previous patch I sent earlier: I think these
#undef can be avoided.
> > +
> > +DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > +
> > +static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + queued_spin_lock(lock);
> > + else
> > + ticket_spin_lock(lock);
> > +}
> > +
> > +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + return queued_spin_trylock(lock);
> > + else
> > + return ticket_spin_trylock(lock);
> > +}
> > +
> > +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + queued_spin_unlock(lock);
> > + else
> > + ticket_spin_unlock(lock);
> > +}
> > +
> > +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + return queued_spin_value_unlocked(lock);
> > + else
> > + return ticket_spin_value_unlocked(lock);
> > +}
> > +
> > +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + return queued_spin_is_locked(lock);
> > + else
> > + return ticket_spin_is_locked(lock);
> > +}
> > +
> > +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > +{
> > + if (static_branch_likely(&combo_qspinlock_key))
> > + return queued_spin_is_contended(lock);
> > + else
> > + return ticket_spin_is_contended(lock);
> > +}
> > +#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > +
> > #ifdef CONFIG_QUEUED_SPINLOCKS
> > #include <asm/qspinlock.h>
> > -#include <asm/qrwlock.h>
> > #else
> > -#include <asm-generic/spinlock.h>
> > +#include <asm-generic/ticket_spinlock.h>
> > #endif
> >
> > +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > +
> > +#include <asm/qrwlock.h>
> > +
> > #endif /* __ASM_RISCV_SPINLOCK_H */
> > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > index 32c2e1eb71bd..a447cf360a18 100644
> > --- a/arch/riscv/kernel/setup.c
> > +++ b/arch/riscv/kernel/setup.c
> > @@ -269,6 +269,18 @@ static void __init parse_dtb(void)
> > #endif
> > }
> >
> > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > +DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > +EXPORT_SYMBOL(combo_qspinlock_key);
> > +#endif
> > +
> > +static void __init riscv_spinlock_init(void)
> > +{
> > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > + static_branch_disable(&combo_qspinlock_key);
> > +#endif
> > +}
> > +
> > extern void __init init_rt_signal_env(void);
> >
> > void __init setup_arch(char **cmdline_p)
> > @@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
> > riscv_isa_extension_available(NULL, ZICBOM))
> > riscv_noncoherent_supported();
> > riscv_set_dma_cache_alignment();
> > +
> > + riscv_spinlock_init();
> > }
> >
> > static int __init topology_init(void)
> > --
> > 2.36.1
> >
> >
> > _______________________________________________
> > linux-riscv mailing list
> > linux-riscv@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-riscv
> >
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
2023-09-13 20:37 ` Leonardo Bras
@ 2023-09-13 20:49 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 20:49 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Wed, Sep 13, 2023 at 05:37:01PM -0300, Leonardo Bras wrote:
> On Sun, Sep 10, 2023 at 07:06:23AM -0400, Guo Ren wrote:
> > On Sun, Sep 10, 2023 at 04:29:00AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Combo spinlock could support queued and ticket in one Linux Image and
> > > select them during boot time via errata mechanism. Here is the func
> > > size (Bytes) comparison table below:
> > >
> > > TYPE : COMBO | TICKET | QUEUED
> > > arch_spin_lock : 106 | 60 | 50
> > > arch_spin_unlock : 54 | 36 | 26
> > > arch_spin_trylock : 110 | 72 | 54
> > > arch_spin_is_locked : 48 | 34 | 20
> > > arch_spin_is_contended : 56 | 40 | 24
> > > rch_spin_value_unlocked : 48 | 34 | 24
> > >
> > > One example of disassemble combo arch_spin_unlock:
> > > 0xffffffff8000409c <+14>: nop # detour slot
> > > 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
> > > 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
> > > 0xffffffff800040a8 <+26>: ld s0,8(sp)
> > > 0xffffffff800040aa <+28>: addi sp,sp,16
> > > 0xffffffff800040ac <+30>: ret
> > > 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
> > > 0xffffffff800040b0 <+34>: sext.w a5,a5
> > > 0xffffffff800040b2 <+36>: fence rw,w
> > > 0xffffffff800040b6 <+40>: addiw a5,a5,1
> > > 0xffffffff800040b8 <+42>: slli a5,a5,0x30
> > > 0xffffffff800040ba <+44>: srli a5,a5,0x30
> > > 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
> > > 0xffffffff800040c0 <+50>: ld s0,8(sp)
> > > 0xffffffff800040c2 <+52>: addi sp,sp,16
> > > 0xffffffff800040c4 <+54>: ret
> > >
> > > The qspinlock is smaller and faster than ticket-lock when all are in
> > > fast-path, and combo spinlock could provide a compatible Linux Image
> > > for different micro-arch design (weak/strict fwd guarantee LR/SC)
> > > processors.
> > >
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > ---
> > > arch/riscv/Kconfig | 9 +++-
> > > arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
> > > arch/riscv/kernel/setup.c | 14 ++++++
> > > 3 files changed, 98 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index 7f39bfc75744..4bcff2860f48 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -473,7 +473,7 @@ config NODES_SHIFT
> > >
> > > choice
> > > prompt "RISC-V spinlock type"
> > > - default RISCV_TICKET_SPINLOCKS
> > > + default RISCV_COMBO_SPINLOCKS
> > >
> > > config RISCV_TICKET_SPINLOCKS
> > > bool "Using ticket spinlock"
> > > @@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
> > > help
> > > Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > Otherwise, stay at ticket-lock.
> > > +
> > > +config RISCV_COMBO_SPINLOCKS
> > > + bool "Using combo spinlock"
> > > + depends on SMP && MMU
> > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > + help
> > > + Select queued spinlock or ticket-lock via errata.
> > > endchoice
> > >
> > > config RISCV_ALTERNATIVE
> > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > index c644a92d4548..8ea0fee80652 100644
> > > --- a/arch/riscv/include/asm/spinlock.h
> > > +++ b/arch/riscv/include/asm/spinlock.h
> > > @@ -7,11 +7,85 @@
> > > #define _Q_PENDING_LOOPS (1 << 9)
> > > #endif
> > >
> > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > +#include <asm-generic/ticket_spinlock.h>
> > > +
> > > +#undef arch_spin_is_locked
> > > +#undef arch_spin_is_contended
> > > +#undef arch_spin_value_unlocked
> > > +#undef arch_spin_lock
> > > +#undef arch_spin_trylock
> > > +#undef arch_spin_unlock
> > > +
> > > +#include <asm-generic/qspinlock.h>
> > > +#include <linux/jump_label.h>
> > > +
> > > +#undef arch_spin_is_locked
> > > +#undef arch_spin_is_contended
> > > +#undef arch_spin_value_unlocked
> > > +#undef arch_spin_lock
> > > +#undef arch_spin_trylock
> > > +#undef arch_spin_unlock
> > Sorry, I forgot __no_arch_spinlock_redefine advice here. I would add it in v12.
> > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> >
>
> Please check a reply to a previous patch I sent earlier: I think these
> #undef can be avoided.
>
> > > +
> > > +DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > +
> > > +static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + queued_spin_lock(lock);
> > > + else
> > > + ticket_spin_lock(lock);
> > > +}
> > > +
> > > +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + return queued_spin_trylock(lock);
> > > + else
> > > + return ticket_spin_trylock(lock);
> > > +}
> > > +
> > > +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + queued_spin_unlock(lock);
> > > + else
> > > + ticket_spin_unlock(lock);
> > > +}
> > > +
> > > +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + return queued_spin_value_unlocked(lock);
> > > + else
> > > + return ticket_spin_value_unlocked(lock);
> > > +}
> > > +
> > > +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + return queued_spin_is_locked(lock);
> > > + else
> > > + return ticket_spin_is_locked(lock);
> > > +}
> > > +
> > > +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + return queued_spin_is_contended(lock);
> > > + else
> > > + return ticket_spin_is_contended(lock);
> > > +}
> > > +#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > +
Also, those functions all reproduce the same behavior, so maybe it would be
better to keep that behavior in a macro such as:
#define COMBO_SPINLOCK_DECLARE(f) \
static __always_inline int arch_spin_ ## f(arch_spinlock_t *lock) \
{ \
if (static_branch_likely(&combo_qspinlock_key)) \
return queued_spin_ ## f(lock); \
else \
return ticket_spin_ ## f(lock); \
}
COMBO_SPINLOCK_DECLARE(lock)
COMBO_SPINLOCK_DECLARE(trylock)
COMBO_SPINLOCK_DECLARE(unlock)
COMBO_SPINLOCK_DECLARE(value_unlocked)
COMBO_SPINLOCK_DECLARE(is_locked)
COMBO_SPINLOCK_DECLARE(is_contended)
Does that make sense?
Thanks!
Leo
> > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > #include <asm/qspinlock.h>
> > > -#include <asm/qrwlock.h>
> > > #else
> > > -#include <asm-generic/spinlock.h>
> > > +#include <asm-generic/ticket_spinlock.h>
> > > #endif
> > >
> > > +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > +
> > > +#include <asm/qrwlock.h>
> > > +
> > > #endif /* __ASM_RISCV_SPINLOCK_H */
> > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > index 32c2e1eb71bd..a447cf360a18 100644
> > > --- a/arch/riscv/kernel/setup.c
> > > +++ b/arch/riscv/kernel/setup.c
> > > @@ -269,6 +269,18 @@ static void __init parse_dtb(void)
> > > #endif
> > > }
> > >
> > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > +DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > +EXPORT_SYMBOL(combo_qspinlock_key);
> > > +#endif
> > > +
> > > +static void __init riscv_spinlock_init(void)
> > > +{
> > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > + static_branch_disable(&combo_qspinlock_key);
> > > +#endif
> > > +}
> > > +
> > > extern void __init init_rt_signal_env(void);
> > >
> > > void __init setup_arch(char **cmdline_p)
> > > @@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
> > > riscv_isa_extension_available(NULL, ZICBOM))
> > > riscv_noncoherent_supported();
> > > riscv_set_dma_cache_alignment();
> > > +
> > > + riscv_spinlock_init();
> > > }
> > >
> > > static int __init topology_init(void)
> > > --
> > > 2.36.1
> > >
> > >
> > > _______________________________________________
> > > linux-riscv mailing list
> > > linux-riscv@lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/linux-riscv
> > >
> >
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
@ 2023-09-13 20:49 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-13 20:49 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Wed, Sep 13, 2023 at 05:37:01PM -0300, Leonardo Bras wrote:
> On Sun, Sep 10, 2023 at 07:06:23AM -0400, Guo Ren wrote:
> > On Sun, Sep 10, 2023 at 04:29:00AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Combo spinlock could support queued and ticket in one Linux Image and
> > > select them during boot time via errata mechanism. Here is the func
> > > size (Bytes) comparison table below:
> > >
> > > TYPE : COMBO | TICKET | QUEUED
> > > arch_spin_lock : 106 | 60 | 50
> > > arch_spin_unlock : 54 | 36 | 26
> > > arch_spin_trylock : 110 | 72 | 54
> > > arch_spin_is_locked : 48 | 34 | 20
> > > arch_spin_is_contended : 56 | 40 | 24
> > > rch_spin_value_unlocked : 48 | 34 | 24
> > >
> > > One example of disassemble combo arch_spin_unlock:
> > > 0xffffffff8000409c <+14>: nop # detour slot
> > > 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
> > > 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
> > > 0xffffffff800040a8 <+26>: ld s0,8(sp)
> > > 0xffffffff800040aa <+28>: addi sp,sp,16
> > > 0xffffffff800040ac <+30>: ret
> > > 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
> > > 0xffffffff800040b0 <+34>: sext.w a5,a5
> > > 0xffffffff800040b2 <+36>: fence rw,w
> > > 0xffffffff800040b6 <+40>: addiw a5,a5,1
> > > 0xffffffff800040b8 <+42>: slli a5,a5,0x30
> > > 0xffffffff800040ba <+44>: srli a5,a5,0x30
> > > 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
> > > 0xffffffff800040c0 <+50>: ld s0,8(sp)
> > > 0xffffffff800040c2 <+52>: addi sp,sp,16
> > > 0xffffffff800040c4 <+54>: ret
> > >
> > > The qspinlock is smaller and faster than ticket-lock when all are in
> > > fast-path, and combo spinlock could provide a compatible Linux Image
> > > for different micro-arch design (weak/strict fwd guarantee LR/SC)
> > > processors.
> > >
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > ---
> > > arch/riscv/Kconfig | 9 +++-
> > > arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
> > > arch/riscv/kernel/setup.c | 14 ++++++
> > > 3 files changed, 98 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index 7f39bfc75744..4bcff2860f48 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -473,7 +473,7 @@ config NODES_SHIFT
> > >
> > > choice
> > > prompt "RISC-V spinlock type"
> > > - default RISCV_TICKET_SPINLOCKS
> > > + default RISCV_COMBO_SPINLOCKS
> > >
> > > config RISCV_TICKET_SPINLOCKS
> > > bool "Using ticket spinlock"
> > > @@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
> > > help
> > > Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > Otherwise, stay at ticket-lock.
> > > +
> > > +config RISCV_COMBO_SPINLOCKS
> > > + bool "Using combo spinlock"
> > > + depends on SMP && MMU
> > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > + help
> > > + Select queued spinlock or ticket-lock via errata.
> > > endchoice
> > >
> > > config RISCV_ALTERNATIVE
> > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > index c644a92d4548..8ea0fee80652 100644
> > > --- a/arch/riscv/include/asm/spinlock.h
> > > +++ b/arch/riscv/include/asm/spinlock.h
> > > @@ -7,11 +7,85 @@
> > > #define _Q_PENDING_LOOPS (1 << 9)
> > > #endif
> > >
> > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > +#include <asm-generic/ticket_spinlock.h>
> > > +
> > > +#undef arch_spin_is_locked
> > > +#undef arch_spin_is_contended
> > > +#undef arch_spin_value_unlocked
> > > +#undef arch_spin_lock
> > > +#undef arch_spin_trylock
> > > +#undef arch_spin_unlock
> > > +
> > > +#include <asm-generic/qspinlock.h>
> > > +#include <linux/jump_label.h>
> > > +
> > > +#undef arch_spin_is_locked
> > > +#undef arch_spin_is_contended
> > > +#undef arch_spin_value_unlocked
> > > +#undef arch_spin_lock
> > > +#undef arch_spin_trylock
> > > +#undef arch_spin_unlock
> > Sorry, I forgot __no_arch_spinlock_redefine advice here. I would add it in v12.
> > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> >
>
> Please check a reply to a previous patch I sent earlier: I think these
> #undef can be avoided.
>
> > > +
> > > +DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > +
> > > +static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + queued_spin_lock(lock);
> > > + else
> > > + ticket_spin_lock(lock);
> > > +}
> > > +
> > > +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + return queued_spin_trylock(lock);
> > > + else
> > > + return ticket_spin_trylock(lock);
> > > +}
> > > +
> > > +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + queued_spin_unlock(lock);
> > > + else
> > > + ticket_spin_unlock(lock);
> > > +}
> > > +
> > > +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + return queued_spin_value_unlocked(lock);
> > > + else
> > > + return ticket_spin_value_unlocked(lock);
> > > +}
> > > +
> > > +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + return queued_spin_is_locked(lock);
> > > + else
> > > + return ticket_spin_is_locked(lock);
> > > +}
> > > +
> > > +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > > +{
> > > + if (static_branch_likely(&combo_qspinlock_key))
> > > + return queued_spin_is_contended(lock);
> > > + else
> > > + return ticket_spin_is_contended(lock);
> > > +}
> > > +#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > +
Also, those functions all reproduce the same behavior, so maybe it would be
better to keep that behavior in a macro such as:
#define COMBO_SPINLOCK_DECLARE(f) \
static __always_inline int arch_spin_ ## f(arch_spinlock_t *lock) \
{ \
if (static_branch_likely(&combo_qspinlock_key)) \
return queued_spin_ ## f(lock); \
else \
return ticket_spin_ ## f(lock); \
}
COMBO_SPINLOCK_DECLARE(lock)
COMBO_SPINLOCK_DECLARE(trylock)
COMBO_SPINLOCK_DECLARE(unlock)
COMBO_SPINLOCK_DECLARE(value_unlocked)
COMBO_SPINLOCK_DECLARE(is_locked)
COMBO_SPINLOCK_DECLARE(is_contended)
Does that make sense?
Thanks!
Leo
> > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > #include <asm/qspinlock.h>
> > > -#include <asm/qrwlock.h>
> > > #else
> > > -#include <asm-generic/spinlock.h>
> > > +#include <asm-generic/ticket_spinlock.h>
> > > #endif
> > >
> > > +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > +
> > > +#include <asm/qrwlock.h>
> > > +
> > > #endif /* __ASM_RISCV_SPINLOCK_H */
> > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > index 32c2e1eb71bd..a447cf360a18 100644
> > > --- a/arch/riscv/kernel/setup.c
> > > +++ b/arch/riscv/kernel/setup.c
> > > @@ -269,6 +269,18 @@ static void __init parse_dtb(void)
> > > #endif
> > > }
> > >
> > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > +DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > +EXPORT_SYMBOL(combo_qspinlock_key);
> > > +#endif
> > > +
> > > +static void __init riscv_spinlock_init(void)
> > > +{
> > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > + static_branch_disable(&combo_qspinlock_key);
> > > +#endif
> > > +}
> > > +
> > > extern void __init init_rt_signal_env(void);
> > >
> > > void __init setup_arch(char **cmdline_p)
> > > @@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
> > > riscv_isa_extension_available(NULL, ZICBOM))
> > > riscv_noncoherent_supported();
> > > riscv_set_dma_cache_alignment();
> > > +
> > > + riscv_spinlock_init();
> > > }
> > >
> > > static int __init topology_init(void)
> > > --
> > > 2.36.1
> > >
> > >
> > > _______________________________________________
> > > linux-riscv mailing list
> > > linux-riscv@lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/linux-riscv
> > >
> >
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
2023-09-13 13:06 ` Waiman Long
@ 2023-09-14 3:45 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-14 3:45 UTC (permalink / raw)
To: Waiman Long
Cc: Leonardo Bras, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Wed, Sep 13, 2023 at 9:06 PM Waiman Long <longman@redhat.com> wrote:
>
> On 9/13/23 08:52, Guo Ren wrote:
> > On Wed, Sep 13, 2023 at 4:55 PM Leonardo Bras <leobras@redhat.com> wrote:
> >> On Tue, Sep 12, 2023 at 09:10:08AM +0800, Guo Ren wrote:
> >>> On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
> >>>> On 9/10/23 23:09, Guo Ren wrote:
> >>>>> On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
> >>>>>> On 9/10/23 04:28, guoren@kernel.org wrote:
> >>>>>>> From: Guo Ren <guoren@linux.alibaba.com>
> >>>>>>>
> >>>>>>> The target of xchg_tail is to write the tail to the lock value, so
> >>>>>>> adding prefetchw could help the next cmpxchg step, which may
> >>>>>>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> >>>>>>> utilize this feature to give a forward guarantee, e.g., RISC-V
> >>>>>>> XuanTie processors would block the snoop channel & irq for several
> >>>>>>> cycles when prefetch.w instruction (from Zicbop extension) retired,
> >>>>>>> which guarantees the next cmpxchg succeeds.
> >>>>>>>
> >>>>>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> >>>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >>>>>>> ---
> >>>>>>> kernel/locking/qspinlock.c | 5 ++++-
> >>>>>>> 1 file changed, 4 insertions(+), 1 deletion(-)
> >>>>>>>
> >>>>>>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> >>>>>>> index d3f99060b60f..96b54e2ade86 100644
> >>>>>>> --- a/kernel/locking/qspinlock.c
> >>>>>>> +++ b/kernel/locking/qspinlock.c
> >>>>>>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> >>>>>>> */
> >>>>>>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> >>>>>>> {
> >>>>>>> - u32 old, new, val = atomic_read(&lock->val);
> >>>>>>> + u32 old, new, val;
> >>>>>>> +
> >>>>>>> + prefetchw(&lock->val);
> >>>>>>> + val = atomic_read(&lock->val);
> >>>>>>>
> >>>>>>> for (;;) {
> >>>>>>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
> >>>>>> That looks a bit weird. You pre-fetch and then immediately read it. How
> >>>>>> much performance gain you get by this change alone?
> >>>>>>
> >>>>>> Maybe you can define an arch specific primitive that default back to
> >>>>>> atomic_read() if not defined.
> >>>>> Thx for the reply. This is a generic optimization point I would like
> >>>>> to talk about with you.
> >>>>>
> >>>>> First, prefetchw() makes cacheline an exclusive state and serves for
> >>>>> the next cmpxchg loop semantic, which writes the idx_tail part of
> >>>>> arch_spin_lock. The atomic_read only makes cacheline in the shared
> >>>>> state, which couldn't give any guarantee for the next cmpxchg loop
> >>>>> semantic. Micro-architecture could utilize prefetchw() to provide a
> >>>>> strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> >>>>> XuanTie processor would hold the exclusive cacheline state until the
> >>>>> next cmpxchg write success.
> >>>>>
> >>>>> In the end, Let's go back to the principle: the xchg_tail is an atomic
> >>>>> swap operation that contains write eventually, so giving a prefetchw()
> >>>>> at the beginning is acceptable for all architectures..
> >>>>> ••••••••••••
> >>>> I did realize afterward that prefetchw gets the cacheline in exclusive
> >>>> state. I will suggest you mention that in your commit log as well as
> >>>> adding a comment about its purpose in the code.
> >>> Okay, I would do that in v12, thx.
> >> I would suggest adding a snippet from the ISA Extenstion doc:
> >>
> >> "A prefetch.w instruction indicates to hardware that the cache block whose
> >> effective address is the sum of the base address specified in rs1 and the
> >> sign-extended offset encoded in imm[11:0], where imm[4:0] equals 0b00000,
> >> is likely to be accessed by a data write (i.e. store) in the near future."
> > Good point, thx.
>
> qspinlock is generic code. I suppose this is for the RISCV architecture.
> You can mention that in the commit log as an example, but I prefer more
> generic comment especially in the code.
Okay, I would only leave a generic comment on it and move Leonardo's
advice into this patch:
https://lore.kernel.org/linux-riscv/ZQF3qS1KRYAt3coC@redhat.com/
>
> Cheers,
> Longman
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
@ 2023-09-14 3:45 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-14 3:45 UTC (permalink / raw)
To: Waiman Long
Cc: Leonardo Bras, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Wed, Sep 13, 2023 at 9:06 PM Waiman Long <longman@redhat.com> wrote:
>
> On 9/13/23 08:52, Guo Ren wrote:
> > On Wed, Sep 13, 2023 at 4:55 PM Leonardo Bras <leobras@redhat.com> wrote:
> >> On Tue, Sep 12, 2023 at 09:10:08AM +0800, Guo Ren wrote:
> >>> On Mon, Sep 11, 2023 at 9:03 PM Waiman Long <longman@redhat.com> wrote:
> >>>> On 9/10/23 23:09, Guo Ren wrote:
> >>>>> On Mon, Sep 11, 2023 at 10:35 AM Waiman Long <longman@redhat.com> wrote:
> >>>>>> On 9/10/23 04:28, guoren@kernel.org wrote:
> >>>>>>> From: Guo Ren <guoren@linux.alibaba.com>
> >>>>>>>
> >>>>>>> The target of xchg_tail is to write the tail to the lock value, so
> >>>>>>> adding prefetchw could help the next cmpxchg step, which may
> >>>>>>> decrease the cmpxchg retry loops of xchg_tail. Some processors may
> >>>>>>> utilize this feature to give a forward guarantee, e.g., RISC-V
> >>>>>>> XuanTie processors would block the snoop channel & irq for several
> >>>>>>> cycles when prefetch.w instruction (from Zicbop extension) retired,
> >>>>>>> which guarantees the next cmpxchg succeeds.
> >>>>>>>
> >>>>>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> >>>>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
> >>>>>>> ---
> >>>>>>> kernel/locking/qspinlock.c | 5 ++++-
> >>>>>>> 1 file changed, 4 insertions(+), 1 deletion(-)
> >>>>>>>
> >>>>>>> diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
> >>>>>>> index d3f99060b60f..96b54e2ade86 100644
> >>>>>>> --- a/kernel/locking/qspinlock.c
> >>>>>>> +++ b/kernel/locking/qspinlock.c
> >>>>>>> @@ -223,7 +223,10 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock)
> >>>>>>> */
> >>>>>>> static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail)
> >>>>>>> {
> >>>>>>> - u32 old, new, val = atomic_read(&lock->val);
> >>>>>>> + u32 old, new, val;
> >>>>>>> +
> >>>>>>> + prefetchw(&lock->val);
> >>>>>>> + val = atomic_read(&lock->val);
> >>>>>>>
> >>>>>>> for (;;) {
> >>>>>>> new = (val & _Q_LOCKED_PENDING_MASK) | tail;
> >>>>>> That looks a bit weird. You pre-fetch and then immediately read it. How
> >>>>>> much performance gain you get by this change alone?
> >>>>>>
> >>>>>> Maybe you can define an arch specific primitive that default back to
> >>>>>> atomic_read() if not defined.
> >>>>> Thx for the reply. This is a generic optimization point I would like
> >>>>> to talk about with you.
> >>>>>
> >>>>> First, prefetchw() makes cacheline an exclusive state and serves for
> >>>>> the next cmpxchg loop semantic, which writes the idx_tail part of
> >>>>> arch_spin_lock. The atomic_read only makes cacheline in the shared
> >>>>> state, which couldn't give any guarantee for the next cmpxchg loop
> >>>>> semantic. Micro-architecture could utilize prefetchw() to provide a
> >>>>> strong forward progress guarantee for the xchg_tail, e.g., the T-HEAD
> >>>>> XuanTie processor would hold the exclusive cacheline state until the
> >>>>> next cmpxchg write success.
> >>>>>
> >>>>> In the end, Let's go back to the principle: the xchg_tail is an atomic
> >>>>> swap operation that contains write eventually, so giving a prefetchw()
> >>>>> at the beginning is acceptable for all architectures..
> >>>>> ••••••••••••
> >>>> I did realize afterward that prefetchw gets the cacheline in exclusive
> >>>> state. I will suggest you mention that in your commit log as well as
> >>>> adding a comment about its purpose in the code.
> >>> Okay, I would do that in v12, thx.
> >> I would suggest adding a snippet from the ISA Extenstion doc:
> >>
> >> "A prefetch.w instruction indicates to hardware that the cache block whose
> >> effective address is the sum of the base address specified in rs1 and the
> >> sign-extended offset encoded in imm[11:0], where imm[4:0] equals 0b00000,
> >> is likely to be accessed by a data write (i.e. store) in the near future."
> > Good point, thx.
>
> qspinlock is generic code. I suppose this is for the RISCV architecture.
> You can mention that in the commit log as an example, but I prefer more
> generic comment especially in the code.
Okay, I would only leave a generic comment on it and move Leonardo's
advice into this patch:
https://lore.kernel.org/linux-riscv/ZQF3qS1KRYAt3coC@redhat.com/
>
> Cheers,
> Longman
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
2023-09-13 20:28 ` Leonardo Bras
@ 2023-09-14 4:46 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-14 4:46 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The requirements of qspinlock have been documented by commit:
> > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > atomics").
> >
> > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > is the one.
> >
> > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> >
> > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > queued_spinlock 0.5109/0.00
> > ticket_spinlock 0.5814/0.00
> >
> > perf futex/hash (+6.7%):
> > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> >
> > perf futex/wake-parallel (+8.6%):
> > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> >
> > perf futex/requeue (+4.2%):
> > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> >
> > System Benchmarks (+6.4%)
> > queued_spinlock:
> > System Benchmarks Index Values BASELINE RESULT INDEX
> > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > Double-Precision Whetstone 55.0 182422.8 33167.8
> > Execl Throughput 43.0 13116.6 3050.4
> > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > Pipe Throughput 12440.0 23058600.5 18535.9
> > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > Process Creation 126.0 12537.3 995.0
> > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > System Call Overhead 15000.0 33308301.3 22205.5
> > ========
> > System Benchmarks Index Score 12426.1
> >
> > ticket_spinlock:
> > System Benchmarks Index Values BASELINE RESULT INDEX
> > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > Double-Precision Whetstone 55.0 181921.0 33076.5
> > Execl Throughput 43.0 12625.1 2936.1
> > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > Pipe Throughput 12440.0 20594018.7 16554.7
> > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > Process Creation 126.0 10798.4 857.0
> > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > System Call Overhead 15000.0 30766778.4 20511.2
> > ========
> > System Benchmarks Index Score 11670.7
> >
> > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > cores platform than the ticket_lock.
> >
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > ---
> > arch/riscv/Kconfig | 16 ++++++++++++++++
> > arch/riscv/include/asm/Kbuild | 3 ++-
> > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > 3 files changed, 35 insertions(+), 1 deletion(-)
> > create mode 100644 arch/riscv/include/asm/spinlock.h
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 2c346fe169c1..7f39bfc75744 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > Specify the maximum number of NUMA Nodes available on the target
> > system. Increases memory reserved to accommodate various tables.
> >
> > +choice
> > + prompt "RISC-V spinlock type"
> > + default RISCV_TICKET_SPINLOCKS
> > +
> > +config RISCV_TICKET_SPINLOCKS
> > + bool "Using ticket spinlock"
> > +
> > +config RISCV_QUEUED_SPINLOCKS
> > + bool "Using queued spinlock"
> > + depends on SMP && MMU
> > + select ARCH_USE_QUEUED_SPINLOCKS
> > + help
> > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > + Otherwise, stay at ticket-lock.
> > +endchoice
> > +
> > config RISCV_ALTERNATIVE
> > bool
> > depends on !XIP_KERNEL
> > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > index 504f8b7e72d4..a0dc85e4a754 100644
> > --- a/arch/riscv/include/asm/Kbuild
> > +++ b/arch/riscv/include/asm/Kbuild
> > @@ -2,10 +2,11 @@
> > generic-y += early_ioremap.h
> > generic-y += flat.h
> > generic-y += kvm_para.h
> > +generic-y += mcs_spinlock.h
> > generic-y += parport.h
> > -generic-y += spinlock.h
>
> IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> and include the asm-generic headers of mcs_spinlock and qspinlock.
>
> In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> which seems the oposite of the above description (ticket spinlocks being
> the standard).
>
> Shouldn't ticket-spinlock.h also get included here?
> (Also, I am probably missing something, as I dont' see the use of
> mcs_spinlock here.)
No, because asm-generic/spinlock.h:
...
#include <asm-generic/ticket_spinlock.h>
...
>
> > generic-y += spinlock_types.h
> > generic-y += qrwlock.h
> > generic-y += qrwlock_types.h
> > +generic-y += qspinlock.h
> > generic-y += user.h
> > generic-y += vmlinux.lds.h
> > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > new file mode 100644
> > index 000000000000..c644a92d4548
> > --- /dev/null
> > +++ b/arch/riscv/include/asm/spinlock.h
> > @@ -0,0 +1,17 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +
> > +#ifndef __ASM_RISCV_SPINLOCK_H
> > +#define __ASM_RISCV_SPINLOCK_H
> > +
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > +#define _Q_PENDING_LOOPS (1 << 9)
> > +#endif
>
> Any reason the above define couldn't be merged on the ifdef below?
Easy for the next patch to modify. See Waiman's comment:
https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> index c644a92d4548..9eb3ad31e564 100644
> --- a/arch/riscv/include/asm/spinlock.h
> +++ b/arch/riscv/include/asm/spinlock.h
> @@ -7,11 +7,94 @@
> #define _Q_PENDING_LOOPS (1 << 9)
> #endif
>
I see why you separated the _Q_PENDING_LOOPS out.
>
> > +
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > +#include <asm/qspinlock.h>
> > +#include <asm/qrwlock.h>
> > +#else
> > +#include <asm-generic/spinlock.h>
> > +#endif
> > +
> > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > --
> > 2.36.1
> >
>
> Thanks!
> Leo
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
@ 2023-09-14 4:46 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-14 4:46 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The requirements of qspinlock have been documented by commit:
> > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > atomics").
> >
> > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > is the one.
> >
> > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> >
> > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > queued_spinlock 0.5109/0.00
> > ticket_spinlock 0.5814/0.00
> >
> > perf futex/hash (+6.7%):
> > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> >
> > perf futex/wake-parallel (+8.6%):
> > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> >
> > perf futex/requeue (+4.2%):
> > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> >
> > System Benchmarks (+6.4%)
> > queued_spinlock:
> > System Benchmarks Index Values BASELINE RESULT INDEX
> > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > Double-Precision Whetstone 55.0 182422.8 33167.8
> > Execl Throughput 43.0 13116.6 3050.4
> > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > Pipe Throughput 12440.0 23058600.5 18535.9
> > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > Process Creation 126.0 12537.3 995.0
> > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > System Call Overhead 15000.0 33308301.3 22205.5
> > ========
> > System Benchmarks Index Score 12426.1
> >
> > ticket_spinlock:
> > System Benchmarks Index Values BASELINE RESULT INDEX
> > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > Double-Precision Whetstone 55.0 181921.0 33076.5
> > Execl Throughput 43.0 12625.1 2936.1
> > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > Pipe Throughput 12440.0 20594018.7 16554.7
> > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > Process Creation 126.0 10798.4 857.0
> > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > System Call Overhead 15000.0 30766778.4 20511.2
> > ========
> > System Benchmarks Index Score 11670.7
> >
> > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > cores platform than the ticket_lock.
> >
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > ---
> > arch/riscv/Kconfig | 16 ++++++++++++++++
> > arch/riscv/include/asm/Kbuild | 3 ++-
> > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > 3 files changed, 35 insertions(+), 1 deletion(-)
> > create mode 100644 arch/riscv/include/asm/spinlock.h
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 2c346fe169c1..7f39bfc75744 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > Specify the maximum number of NUMA Nodes available on the target
> > system. Increases memory reserved to accommodate various tables.
> >
> > +choice
> > + prompt "RISC-V spinlock type"
> > + default RISCV_TICKET_SPINLOCKS
> > +
> > +config RISCV_TICKET_SPINLOCKS
> > + bool "Using ticket spinlock"
> > +
> > +config RISCV_QUEUED_SPINLOCKS
> > + bool "Using queued spinlock"
> > + depends on SMP && MMU
> > + select ARCH_USE_QUEUED_SPINLOCKS
> > + help
> > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > + Otherwise, stay at ticket-lock.
> > +endchoice
> > +
> > config RISCV_ALTERNATIVE
> > bool
> > depends on !XIP_KERNEL
> > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > index 504f8b7e72d4..a0dc85e4a754 100644
> > --- a/arch/riscv/include/asm/Kbuild
> > +++ b/arch/riscv/include/asm/Kbuild
> > @@ -2,10 +2,11 @@
> > generic-y += early_ioremap.h
> > generic-y += flat.h
> > generic-y += kvm_para.h
> > +generic-y += mcs_spinlock.h
> > generic-y += parport.h
> > -generic-y += spinlock.h
>
> IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> and include the asm-generic headers of mcs_spinlock and qspinlock.
>
> In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> which seems the oposite of the above description (ticket spinlocks being
> the standard).
>
> Shouldn't ticket-spinlock.h also get included here?
> (Also, I am probably missing something, as I dont' see the use of
> mcs_spinlock here.)
No, because asm-generic/spinlock.h:
...
#include <asm-generic/ticket_spinlock.h>
...
>
> > generic-y += spinlock_types.h
> > generic-y += qrwlock.h
> > generic-y += qrwlock_types.h
> > +generic-y += qspinlock.h
> > generic-y += user.h
> > generic-y += vmlinux.lds.h
> > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > new file mode 100644
> > index 000000000000..c644a92d4548
> > --- /dev/null
> > +++ b/arch/riscv/include/asm/spinlock.h
> > @@ -0,0 +1,17 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +
> > +#ifndef __ASM_RISCV_SPINLOCK_H
> > +#define __ASM_RISCV_SPINLOCK_H
> > +
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > +#define _Q_PENDING_LOOPS (1 << 9)
> > +#endif
>
> Any reason the above define couldn't be merged on the ifdef below?
Easy for the next patch to modify. See Waiman's comment:
https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> index c644a92d4548..9eb3ad31e564 100644
> --- a/arch/riscv/include/asm/spinlock.h
> +++ b/arch/riscv/include/asm/spinlock.h
> @@ -7,11 +7,94 @@
> #define _Q_PENDING_LOOPS (1 << 9)
> #endif
>
I see why you separated the _Q_PENDING_LOOPS out.
>
> > +
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > +#include <asm/qspinlock.h>
> > +#include <asm/qrwlock.h>
> > +#else
> > +#include <asm-generic/spinlock.h>
> > +#endif
> > +
> > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > --
> > 2.36.1
> >
>
> Thanks!
> Leo
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
2023-09-13 20:49 ` Leonardo Bras
@ 2023-09-14 4:49 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-14 4:49 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Wed, Sep 13, 2023 at 05:37:01PM -0300, Leonardo Bras wrote:
> > On Sun, Sep 10, 2023 at 07:06:23AM -0400, Guo Ren wrote:
> > > On Sun, Sep 10, 2023 at 04:29:00AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > Combo spinlock could support queued and ticket in one Linux Image and
> > > > select them during boot time via errata mechanism. Here is the func
> > > > size (Bytes) comparison table below:
> > > >
> > > > TYPE : COMBO | TICKET | QUEUED
> > > > arch_spin_lock : 106 | 60 | 50
> > > > arch_spin_unlock : 54 | 36 | 26
> > > > arch_spin_trylock : 110 | 72 | 54
> > > > arch_spin_is_locked : 48 | 34 | 20
> > > > arch_spin_is_contended : 56 | 40 | 24
> > > > rch_spin_value_unlocked : 48 | 34 | 24
> > > >
> > > > One example of disassemble combo arch_spin_unlock:
> > > > 0xffffffff8000409c <+14>: nop # detour slot
> > > > 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
> > > > 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
> > > > 0xffffffff800040a8 <+26>: ld s0,8(sp)
> > > > 0xffffffff800040aa <+28>: addi sp,sp,16
> > > > 0xffffffff800040ac <+30>: ret
> > > > 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
> > > > 0xffffffff800040b0 <+34>: sext.w a5,a5
> > > > 0xffffffff800040b2 <+36>: fence rw,w
> > > > 0xffffffff800040b6 <+40>: addiw a5,a5,1
> > > > 0xffffffff800040b8 <+42>: slli a5,a5,0x30
> > > > 0xffffffff800040ba <+44>: srli a5,a5,0x30
> > > > 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
> > > > 0xffffffff800040c0 <+50>: ld s0,8(sp)
> > > > 0xffffffff800040c2 <+52>: addi sp,sp,16
> > > > 0xffffffff800040c4 <+54>: ret
> > > >
> > > > The qspinlock is smaller and faster than ticket-lock when all are in
> > > > fast-path, and combo spinlock could provide a compatible Linux Image
> > > > for different micro-arch design (weak/strict fwd guarantee LR/SC)
> > > > processors.
> > > >
> > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > ---
> > > > arch/riscv/Kconfig | 9 +++-
> > > > arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
> > > > arch/riscv/kernel/setup.c | 14 ++++++
> > > > 3 files changed, 98 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > index 7f39bfc75744..4bcff2860f48 100644
> > > > --- a/arch/riscv/Kconfig
> > > > +++ b/arch/riscv/Kconfig
> > > > @@ -473,7 +473,7 @@ config NODES_SHIFT
> > > >
> > > > choice
> > > > prompt "RISC-V spinlock type"
> > > > - default RISCV_TICKET_SPINLOCKS
> > > > + default RISCV_COMBO_SPINLOCKS
> > > >
> > > > config RISCV_TICKET_SPINLOCKS
> > > > bool "Using ticket spinlock"
> > > > @@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
> > > > help
> > > > Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > Otherwise, stay at ticket-lock.
> > > > +
> > > > +config RISCV_COMBO_SPINLOCKS
> > > > + bool "Using combo spinlock"
> > > > + depends on SMP && MMU
> > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > + help
> > > > + Select queued spinlock or ticket-lock via errata.
> > > > endchoice
> > > >
> > > > config RISCV_ALTERNATIVE
> > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > index c644a92d4548..8ea0fee80652 100644
> > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > @@ -7,11 +7,85 @@
> > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > #endif
> > > >
> > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > +#include <asm-generic/ticket_spinlock.h>
> > > > +
> > > > +#undef arch_spin_is_locked
> > > > +#undef arch_spin_is_contended
> > > > +#undef arch_spin_value_unlocked
> > > > +#undef arch_spin_lock
> > > > +#undef arch_spin_trylock
> > > > +#undef arch_spin_unlock
> > > > +
> > > > +#include <asm-generic/qspinlock.h>
> > > > +#include <linux/jump_label.h>
> > > > +
> > > > +#undef arch_spin_is_locked
> > > > +#undef arch_spin_is_contended
> > > > +#undef arch_spin_value_unlocked
> > > > +#undef arch_spin_lock
> > > > +#undef arch_spin_trylock
> > > > +#undef arch_spin_unlock
> > > Sorry, I forgot __no_arch_spinlock_redefine advice here. I would add it in v12.
> > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> > >
> >
> > Please check a reply to a previous patch I sent earlier: I think these
> > #undef can be avoided.
> >
> > > > +
> > > > +DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > +
> > > > +static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + queued_spin_lock(lock);
> > > > + else
> > > > + ticket_spin_lock(lock);
> > > > +}
> > > > +
> > > > +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + return queued_spin_trylock(lock);
> > > > + else
> > > > + return ticket_spin_trylock(lock);
> > > > +}
> > > > +
> > > > +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + queued_spin_unlock(lock);
> > > > + else
> > > > + ticket_spin_unlock(lock);
> > > > +}
> > > > +
> > > > +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + return queued_spin_value_unlocked(lock);
> > > > + else
> > > > + return ticket_spin_value_unlocked(lock);
> > > > +}
> > > > +
> > > > +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + return queued_spin_is_locked(lock);
> > > > + else
> > > > + return ticket_spin_is_locked(lock);
> > > > +}
> > > > +
> > > > +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + return queued_spin_is_contended(lock);
> > > > + else
> > > > + return ticket_spin_is_contended(lock);
> > > > +}
> > > > +#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > > +
>
> Also, those functions all reproduce the same behavior, so maybe it would be
> better to keep that behavior in a macro such as:
>
> #define COMBO_SPINLOCK_DECLARE(f) \
> static __always_inline int arch_spin_ ## f(arch_spinlock_t *lock) \
> { \
> if (static_branch_likely(&combo_qspinlock_key)) \
> return queued_spin_ ## f(lock); \
> else \
> return ticket_spin_ ## f(lock); \
> }
>
> COMBO_SPINLOCK_DECLARE(lock)
> COMBO_SPINLOCK_DECLARE(trylock)
> COMBO_SPINLOCK_DECLARE(unlock)
> COMBO_SPINLOCK_DECLARE(value_unlocked)
> COMBO_SPINLOCK_DECLARE(is_locked)
> COMBO_SPINLOCK_DECLARE(is_contended)
>
> Does that make sense?
Yeah, thanks. I would try. You improved my macro skills. :)
>
> Thanks!
> Leo
>
>
> > > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > > #include <asm/qspinlock.h>
> > > > -#include <asm/qrwlock.h>
> > > > #else
> > > > -#include <asm-generic/spinlock.h>
> > > > +#include <asm-generic/ticket_spinlock.h>
> > > > #endif
> > > >
> > > > +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > > +
> > > > +#include <asm/qrwlock.h>
> > > > +
> > > > #endif /* __ASM_RISCV_SPINLOCK_H */
> > > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > > index 32c2e1eb71bd..a447cf360a18 100644
> > > > --- a/arch/riscv/kernel/setup.c
> > > > +++ b/arch/riscv/kernel/setup.c
> > > > @@ -269,6 +269,18 @@ static void __init parse_dtb(void)
> > > > #endif
> > > > }
> > > >
> > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > +DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > +EXPORT_SYMBOL(combo_qspinlock_key);
> > > > +#endif
> > > > +
> > > > +static void __init riscv_spinlock_init(void)
> > > > +{
> > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > + static_branch_disable(&combo_qspinlock_key);
> > > > +#endif
> > > > +}
> > > > +
> > > > extern void __init init_rt_signal_env(void);
> > > >
> > > > void __init setup_arch(char **cmdline_p)
> > > > @@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
> > > > riscv_isa_extension_available(NULL, ZICBOM))
> > > > riscv_noncoherent_supported();
> > > > riscv_set_dma_cache_alignment();
> > > > +
> > > > + riscv_spinlock_init();
> > > > }
> > > >
> > > > static int __init topology_init(void)
> > > > --
> > > > 2.36.1
> > > >
> > > >
> > > > _______________________________________________
> > > > linux-riscv mailing list
> > > > linux-riscv@lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/linux-riscv
> > > >
> > >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
@ 2023-09-14 4:49 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-14 4:49 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Wed, Sep 13, 2023 at 05:37:01PM -0300, Leonardo Bras wrote:
> > On Sun, Sep 10, 2023 at 07:06:23AM -0400, Guo Ren wrote:
> > > On Sun, Sep 10, 2023 at 04:29:00AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > Combo spinlock could support queued and ticket in one Linux Image and
> > > > select them during boot time via errata mechanism. Here is the func
> > > > size (Bytes) comparison table below:
> > > >
> > > > TYPE : COMBO | TICKET | QUEUED
> > > > arch_spin_lock : 106 | 60 | 50
> > > > arch_spin_unlock : 54 | 36 | 26
> > > > arch_spin_trylock : 110 | 72 | 54
> > > > arch_spin_is_locked : 48 | 34 | 20
> > > > arch_spin_is_contended : 56 | 40 | 24
> > > > rch_spin_value_unlocked : 48 | 34 | 24
> > > >
> > > > One example of disassemble combo arch_spin_unlock:
> > > > 0xffffffff8000409c <+14>: nop # detour slot
> > > > 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
> > > > 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
> > > > 0xffffffff800040a8 <+26>: ld s0,8(sp)
> > > > 0xffffffff800040aa <+28>: addi sp,sp,16
> > > > 0xffffffff800040ac <+30>: ret
> > > > 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
> > > > 0xffffffff800040b0 <+34>: sext.w a5,a5
> > > > 0xffffffff800040b2 <+36>: fence rw,w
> > > > 0xffffffff800040b6 <+40>: addiw a5,a5,1
> > > > 0xffffffff800040b8 <+42>: slli a5,a5,0x30
> > > > 0xffffffff800040ba <+44>: srli a5,a5,0x30
> > > > 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
> > > > 0xffffffff800040c0 <+50>: ld s0,8(sp)
> > > > 0xffffffff800040c2 <+52>: addi sp,sp,16
> > > > 0xffffffff800040c4 <+54>: ret
> > > >
> > > > The qspinlock is smaller and faster than ticket-lock when all are in
> > > > fast-path, and combo spinlock could provide a compatible Linux Image
> > > > for different micro-arch design (weak/strict fwd guarantee LR/SC)
> > > > processors.
> > > >
> > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > ---
> > > > arch/riscv/Kconfig | 9 +++-
> > > > arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
> > > > arch/riscv/kernel/setup.c | 14 ++++++
> > > > 3 files changed, 98 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > index 7f39bfc75744..4bcff2860f48 100644
> > > > --- a/arch/riscv/Kconfig
> > > > +++ b/arch/riscv/Kconfig
> > > > @@ -473,7 +473,7 @@ config NODES_SHIFT
> > > >
> > > > choice
> > > > prompt "RISC-V spinlock type"
> > > > - default RISCV_TICKET_SPINLOCKS
> > > > + default RISCV_COMBO_SPINLOCKS
> > > >
> > > > config RISCV_TICKET_SPINLOCKS
> > > > bool "Using ticket spinlock"
> > > > @@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
> > > > help
> > > > Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > Otherwise, stay at ticket-lock.
> > > > +
> > > > +config RISCV_COMBO_SPINLOCKS
> > > > + bool "Using combo spinlock"
> > > > + depends on SMP && MMU
> > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > + help
> > > > + Select queued spinlock or ticket-lock via errata.
> > > > endchoice
> > > >
> > > > config RISCV_ALTERNATIVE
> > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > index c644a92d4548..8ea0fee80652 100644
> > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > @@ -7,11 +7,85 @@
> > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > #endif
> > > >
> > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > +#include <asm-generic/ticket_spinlock.h>
> > > > +
> > > > +#undef arch_spin_is_locked
> > > > +#undef arch_spin_is_contended
> > > > +#undef arch_spin_value_unlocked
> > > > +#undef arch_spin_lock
> > > > +#undef arch_spin_trylock
> > > > +#undef arch_spin_unlock
> > > > +
> > > > +#include <asm-generic/qspinlock.h>
> > > > +#include <linux/jump_label.h>
> > > > +
> > > > +#undef arch_spin_is_locked
> > > > +#undef arch_spin_is_contended
> > > > +#undef arch_spin_value_unlocked
> > > > +#undef arch_spin_lock
> > > > +#undef arch_spin_trylock
> > > > +#undef arch_spin_unlock
> > > Sorry, I forgot __no_arch_spinlock_redefine advice here. I would add it in v12.
> > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> > >
> >
> > Please check a reply to a previous patch I sent earlier: I think these
> > #undef can be avoided.
> >
> > > > +
> > > > +DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > +
> > > > +static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + queued_spin_lock(lock);
> > > > + else
> > > > + ticket_spin_lock(lock);
> > > > +}
> > > > +
> > > > +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + return queued_spin_trylock(lock);
> > > > + else
> > > > + return ticket_spin_trylock(lock);
> > > > +}
> > > > +
> > > > +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + queued_spin_unlock(lock);
> > > > + else
> > > > + ticket_spin_unlock(lock);
> > > > +}
> > > > +
> > > > +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + return queued_spin_value_unlocked(lock);
> > > > + else
> > > > + return ticket_spin_value_unlocked(lock);
> > > > +}
> > > > +
> > > > +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + return queued_spin_is_locked(lock);
> > > > + else
> > > > + return ticket_spin_is_locked(lock);
> > > > +}
> > > > +
> > > > +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > > > +{
> > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > + return queued_spin_is_contended(lock);
> > > > + else
> > > > + return ticket_spin_is_contended(lock);
> > > > +}
> > > > +#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > > +
>
> Also, those functions all reproduce the same behavior, so maybe it would be
> better to keep that behavior in a macro such as:
>
> #define COMBO_SPINLOCK_DECLARE(f) \
> static __always_inline int arch_spin_ ## f(arch_spinlock_t *lock) \
> { \
> if (static_branch_likely(&combo_qspinlock_key)) \
> return queued_spin_ ## f(lock); \
> else \
> return ticket_spin_ ## f(lock); \
> }
>
> COMBO_SPINLOCK_DECLARE(lock)
> COMBO_SPINLOCK_DECLARE(trylock)
> COMBO_SPINLOCK_DECLARE(unlock)
> COMBO_SPINLOCK_DECLARE(value_unlocked)
> COMBO_SPINLOCK_DECLARE(is_locked)
> COMBO_SPINLOCK_DECLARE(is_contended)
>
> Does that make sense?
Yeah, thanks. I would try. You improved my macro skills. :)
>
> Thanks!
> Leo
>
>
> > > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > > #include <asm/qspinlock.h>
> > > > -#include <asm/qrwlock.h>
> > > > #else
> > > > -#include <asm-generic/spinlock.h>
> > > > +#include <asm-generic/ticket_spinlock.h>
> > > > #endif
> > > >
> > > > +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > > +
> > > > +#include <asm/qrwlock.h>
> > > > +
> > > > #endif /* __ASM_RISCV_SPINLOCK_H */
> > > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > > index 32c2e1eb71bd..a447cf360a18 100644
> > > > --- a/arch/riscv/kernel/setup.c
> > > > +++ b/arch/riscv/kernel/setup.c
> > > > @@ -269,6 +269,18 @@ static void __init parse_dtb(void)
> > > > #endif
> > > > }
> > > >
> > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > +DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > +EXPORT_SYMBOL(combo_qspinlock_key);
> > > > +#endif
> > > > +
> > > > +static void __init riscv_spinlock_init(void)
> > > > +{
> > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > + static_branch_disable(&combo_qspinlock_key);
> > > > +#endif
> > > > +}
> > > > +
> > > > extern void __init init_rt_signal_env(void);
> > > >
> > > > void __init setup_arch(char **cmdline_p)
> > > > @@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
> > > > riscv_isa_extension_available(NULL, ZICBOM))
> > > > riscv_noncoherent_supported();
> > > > riscv_set_dma_cache_alignment();
> > > > +
> > > > + riscv_spinlock_init();
> > > > }
> > > >
> > > > static int __init topology_init(void)
> > > > --
> > > > 2.36.1
> > > >
> > > >
> > > > _______________________________________________
> > > > linux-riscv mailing list
> > > > linux-riscv@lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/linux-riscv
> > > >
> > >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
2023-09-14 4:49 ` Guo Ren
@ 2023-09-14 7:17 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 7:17 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 12:49:45PM +0800, Guo Ren wrote:
> On Thu, Sep 14, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Wed, Sep 13, 2023 at 05:37:01PM -0300, Leonardo Bras wrote:
> > > On Sun, Sep 10, 2023 at 07:06:23AM -0400, Guo Ren wrote:
> > > > On Sun, Sep 10, 2023 at 04:29:00AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > >
> > > > > Combo spinlock could support queued and ticket in one Linux Image and
> > > > > select them during boot time via errata mechanism. Here is the func
> > > > > size (Bytes) comparison table below:
> > > > >
> > > > > TYPE : COMBO | TICKET | QUEUED
> > > > > arch_spin_lock : 106 | 60 | 50
> > > > > arch_spin_unlock : 54 | 36 | 26
> > > > > arch_spin_trylock : 110 | 72 | 54
> > > > > arch_spin_is_locked : 48 | 34 | 20
> > > > > arch_spin_is_contended : 56 | 40 | 24
> > > > > rch_spin_value_unlocked : 48 | 34 | 24
> > > > >
> > > > > One example of disassemble combo arch_spin_unlock:
> > > > > 0xffffffff8000409c <+14>: nop # detour slot
> > > > > 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
> > > > > 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
> > > > > 0xffffffff800040a8 <+26>: ld s0,8(sp)
> > > > > 0xffffffff800040aa <+28>: addi sp,sp,16
> > > > > 0xffffffff800040ac <+30>: ret
> > > > > 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
> > > > > 0xffffffff800040b0 <+34>: sext.w a5,a5
> > > > > 0xffffffff800040b2 <+36>: fence rw,w
> > > > > 0xffffffff800040b6 <+40>: addiw a5,a5,1
> > > > > 0xffffffff800040b8 <+42>: slli a5,a5,0x30
> > > > > 0xffffffff800040ba <+44>: srli a5,a5,0x30
> > > > > 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
> > > > > 0xffffffff800040c0 <+50>: ld s0,8(sp)
> > > > > 0xffffffff800040c2 <+52>: addi sp,sp,16
> > > > > 0xffffffff800040c4 <+54>: ret
> > > > >
> > > > > The qspinlock is smaller and faster than ticket-lock when all are in
> > > > > fast-path, and combo spinlock could provide a compatible Linux Image
> > > > > for different micro-arch design (weak/strict fwd guarantee LR/SC)
> > > > > processors.
> > > > >
> > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > ---
> > > > > arch/riscv/Kconfig | 9 +++-
> > > > > arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
> > > > > arch/riscv/kernel/setup.c | 14 ++++++
> > > > > 3 files changed, 98 insertions(+), 3 deletions(-)
> > > > >
> > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > index 7f39bfc75744..4bcff2860f48 100644
> > > > > --- a/arch/riscv/Kconfig
> > > > > +++ b/arch/riscv/Kconfig
> > > > > @@ -473,7 +473,7 @@ config NODES_SHIFT
> > > > >
> > > > > choice
> > > > > prompt "RISC-V spinlock type"
> > > > > - default RISCV_TICKET_SPINLOCKS
> > > > > + default RISCV_COMBO_SPINLOCKS
> > > > >
> > > > > config RISCV_TICKET_SPINLOCKS
> > > > > bool "Using ticket spinlock"
> > > > > @@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
> > > > > help
> > > > > Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > > Otherwise, stay at ticket-lock.
> > > > > +
> > > > > +config RISCV_COMBO_SPINLOCKS
> > > > > + bool "Using combo spinlock"
> > > > > + depends on SMP && MMU
> > > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > > + help
> > > > > + Select queued spinlock or ticket-lock via errata.
> > > > > endchoice
> > > > >
> > > > > config RISCV_ALTERNATIVE
> > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > index c644a92d4548..8ea0fee80652 100644
> > > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > @@ -7,11 +7,85 @@
> > > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > > #endif
> > > > >
> > > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > > +#include <asm-generic/ticket_spinlock.h>
> > > > > +
> > > > > +#undef arch_spin_is_locked
> > > > > +#undef arch_spin_is_contended
> > > > > +#undef arch_spin_value_unlocked
> > > > > +#undef arch_spin_lock
> > > > > +#undef arch_spin_trylock
> > > > > +#undef arch_spin_unlock
> > > > > +
> > > > > +#include <asm-generic/qspinlock.h>
> > > > > +#include <linux/jump_label.h>
> > > > > +
> > > > > +#undef arch_spin_is_locked
> > > > > +#undef arch_spin_is_contended
> > > > > +#undef arch_spin_value_unlocked
> > > > > +#undef arch_spin_lock
> > > > > +#undef arch_spin_trylock
> > > > > +#undef arch_spin_unlock
> > > > Sorry, I forgot __no_arch_spinlock_redefine advice here. I would add it in v12.
> > > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> > > >
> > >
> > > Please check a reply to a previous patch I sent earlier: I think these
> > > #undef can be avoided.
> > >
> > > > > +
> > > > > +DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > > +
> > > > > +static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + queued_spin_lock(lock);
> > > > > + else
> > > > > + ticket_spin_lock(lock);
> > > > > +}
> > > > > +
> > > > > +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + return queued_spin_trylock(lock);
> > > > > + else
> > > > > + return ticket_spin_trylock(lock);
> > > > > +}
> > > > > +
> > > > > +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + queued_spin_unlock(lock);
> > > > > + else
> > > > > + ticket_spin_unlock(lock);
> > > > > +}
> > > > > +
> > > > > +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + return queued_spin_value_unlocked(lock);
> > > > > + else
> > > > > + return ticket_spin_value_unlocked(lock);
> > > > > +}
> > > > > +
> > > > > +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + return queued_spin_is_locked(lock);
> > > > > + else
> > > > > + return ticket_spin_is_locked(lock);
> > > > > +}
> > > > > +
> > > > > +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + return queued_spin_is_contended(lock);
> > > > > + else
> > > > > + return ticket_spin_is_contended(lock);
> > > > > +}
> > > > > +#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > > > +
> >
> > Also, those functions all reproduce the same behavior, so maybe it would be
> > better to keep that behavior in a macro such as:
> >
> > #define COMBO_SPINLOCK_DECLARE(f) \
> > static __always_inline int arch_spin_ ## f(arch_spinlock_t *lock) \
> > { \
> > if (static_branch_likely(&combo_qspinlock_key)) \
> > return queued_spin_ ## f(lock); \
> > else \
> > return ticket_spin_ ## f(lock); \
> > }
> >
> > COMBO_SPINLOCK_DECLARE(lock)
> > COMBO_SPINLOCK_DECLARE(trylock)
> > COMBO_SPINLOCK_DECLARE(unlock)
> > COMBO_SPINLOCK_DECLARE(value_unlocked)
> > COMBO_SPINLOCK_DECLARE(is_locked)
> > COMBO_SPINLOCK_DECLARE(is_contended)
> >
> > Does that make sense?
> Yeah, thanks. I would try. You improved my macro skills. :)
Glad to be of any help :)
>
> >
> > Thanks!
> > Leo
> >
> >
> > > > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > #include <asm/qspinlock.h>
> > > > > -#include <asm/qrwlock.h>
> > > > > #else
> > > > > -#include <asm-generic/spinlock.h>
> > > > > +#include <asm-generic/ticket_spinlock.h>
> > > > > #endif
> > > > >
> > > > > +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > > > +
> > > > > +#include <asm/qrwlock.h>
> > > > > +
> > > > > #endif /* __ASM_RISCV_SPINLOCK_H */
> > > > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > > > index 32c2e1eb71bd..a447cf360a18 100644
> > > > > --- a/arch/riscv/kernel/setup.c
> > > > > +++ b/arch/riscv/kernel/setup.c
> > > > > @@ -269,6 +269,18 @@ static void __init parse_dtb(void)
> > > > > #endif
> > > > > }
> > > > >
> > > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > > +DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > > +EXPORT_SYMBOL(combo_qspinlock_key);
> > > > > +#endif
> > > > > +
> > > > > +static void __init riscv_spinlock_init(void)
> > > > > +{
> > > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > > + static_branch_disable(&combo_qspinlock_key);
> > > > > +#endif
> > > > > +}
> > > > > +
> > > > > extern void __init init_rt_signal_env(void);
> > > > >
> > > > > void __init setup_arch(char **cmdline_p)
> > > > > @@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
> > > > > riscv_isa_extension_available(NULL, ZICBOM))
> > > > > riscv_noncoherent_supported();
> > > > > riscv_set_dma_cache_alignment();
> > > > > +
> > > > > + riscv_spinlock_init();
> > > > > }
> > > > >
> > > > > static int __init topology_init(void)
> > > > > --
> > > > > 2.36.1
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > linux-riscv mailing list
> > > > > linux-riscv@lists.infradead.org
> > > > > http://lists.infradead.org/mailman/listinfo/linux-riscv
> > > > >
> > > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock
@ 2023-09-14 7:17 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 7:17 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 12:49:45PM +0800, Guo Ren wrote:
> On Thu, Sep 14, 2023 at 4:49 AM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Wed, Sep 13, 2023 at 05:37:01PM -0300, Leonardo Bras wrote:
> > > On Sun, Sep 10, 2023 at 07:06:23AM -0400, Guo Ren wrote:
> > > > On Sun, Sep 10, 2023 at 04:29:00AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > >
> > > > > Combo spinlock could support queued and ticket in one Linux Image and
> > > > > select them during boot time via errata mechanism. Here is the func
> > > > > size (Bytes) comparison table below:
> > > > >
> > > > > TYPE : COMBO | TICKET | QUEUED
> > > > > arch_spin_lock : 106 | 60 | 50
> > > > > arch_spin_unlock : 54 | 36 | 26
> > > > > arch_spin_trylock : 110 | 72 | 54
> > > > > arch_spin_is_locked : 48 | 34 | 20
> > > > > arch_spin_is_contended : 56 | 40 | 24
> > > > > rch_spin_value_unlocked : 48 | 34 | 24
> > > > >
> > > > > One example of disassemble combo arch_spin_unlock:
> > > > > 0xffffffff8000409c <+14>: nop # detour slot
> > > > > 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start
> > > > > 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end
> > > > > 0xffffffff800040a8 <+26>: ld s0,8(sp)
> > > > > 0xffffffff800040aa <+28>: addi sp,sp,16
> > > > > 0xffffffff800040ac <+30>: ret
> > > > > 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start
> > > > > 0xffffffff800040b0 <+34>: sext.w a5,a5
> > > > > 0xffffffff800040b2 <+36>: fence rw,w
> > > > > 0xffffffff800040b6 <+40>: addiw a5,a5,1
> > > > > 0xffffffff800040b8 <+42>: slli a5,a5,0x30
> > > > > 0xffffffff800040ba <+44>: srli a5,a5,0x30
> > > > > 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end
> > > > > 0xffffffff800040c0 <+50>: ld s0,8(sp)
> > > > > 0xffffffff800040c2 <+52>: addi sp,sp,16
> > > > > 0xffffffff800040c4 <+54>: ret
> > > > >
> > > > > The qspinlock is smaller and faster than ticket-lock when all are in
> > > > > fast-path, and combo spinlock could provide a compatible Linux Image
> > > > > for different micro-arch design (weak/strict fwd guarantee LR/SC)
> > > > > processors.
> > > > >
> > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > ---
> > > > > arch/riscv/Kconfig | 9 +++-
> > > > > arch/riscv/include/asm/spinlock.h | 78 ++++++++++++++++++++++++++++++-
> > > > > arch/riscv/kernel/setup.c | 14 ++++++
> > > > > 3 files changed, 98 insertions(+), 3 deletions(-)
> > > > >
> > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > index 7f39bfc75744..4bcff2860f48 100644
> > > > > --- a/arch/riscv/Kconfig
> > > > > +++ b/arch/riscv/Kconfig
> > > > > @@ -473,7 +473,7 @@ config NODES_SHIFT
> > > > >
> > > > > choice
> > > > > prompt "RISC-V spinlock type"
> > > > > - default RISCV_TICKET_SPINLOCKS
> > > > > + default RISCV_COMBO_SPINLOCKS
> > > > >
> > > > > config RISCV_TICKET_SPINLOCKS
> > > > > bool "Using ticket spinlock"
> > > > > @@ -485,6 +485,13 @@ config RISCV_QUEUED_SPINLOCKS
> > > > > help
> > > > > Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > > Otherwise, stay at ticket-lock.
> > > > > +
> > > > > +config RISCV_COMBO_SPINLOCKS
> > > > > + bool "Using combo spinlock"
> > > > > + depends on SMP && MMU
> > > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > > + help
> > > > > + Select queued spinlock or ticket-lock via errata.
> > > > > endchoice
> > > > >
> > > > > config RISCV_ALTERNATIVE
> > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > index c644a92d4548..8ea0fee80652 100644
> > > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > @@ -7,11 +7,85 @@
> > > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > > #endif
> > > > >
> > > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > > +#include <asm-generic/ticket_spinlock.h>
> > > > > +
> > > > > +#undef arch_spin_is_locked
> > > > > +#undef arch_spin_is_contended
> > > > > +#undef arch_spin_value_unlocked
> > > > > +#undef arch_spin_lock
> > > > > +#undef arch_spin_trylock
> > > > > +#undef arch_spin_unlock
> > > > > +
> > > > > +#include <asm-generic/qspinlock.h>
> > > > > +#include <linux/jump_label.h>
> > > > > +
> > > > > +#undef arch_spin_is_locked
> > > > > +#undef arch_spin_is_contended
> > > > > +#undef arch_spin_value_unlocked
> > > > > +#undef arch_spin_lock
> > > > > +#undef arch_spin_trylock
> > > > > +#undef arch_spin_unlock
> > > > Sorry, I forgot __no_arch_spinlock_redefine advice here. I would add it in v12.
> > > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> > > >
> > >
> > > Please check a reply to a previous patch I sent earlier: I think these
> > > #undef can be avoided.
> > >
> > > > > +
> > > > > +DECLARE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > > +
> > > > > +static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + queued_spin_lock(lock);
> > > > > + else
> > > > > + ticket_spin_lock(lock);
> > > > > +}
> > > > > +
> > > > > +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + return queued_spin_trylock(lock);
> > > > > + else
> > > > > + return ticket_spin_trylock(lock);
> > > > > +}
> > > > > +
> > > > > +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + queued_spin_unlock(lock);
> > > > > + else
> > > > > + ticket_spin_unlock(lock);
> > > > > +}
> > > > > +
> > > > > +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + return queued_spin_value_unlocked(lock);
> > > > > + else
> > > > > + return ticket_spin_value_unlocked(lock);
> > > > > +}
> > > > > +
> > > > > +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + return queued_spin_is_locked(lock);
> > > > > + else
> > > > > + return ticket_spin_is_locked(lock);
> > > > > +}
> > > > > +
> > > > > +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock)
> > > > > +{
> > > > > + if (static_branch_likely(&combo_qspinlock_key))
> > > > > + return queued_spin_is_contended(lock);
> > > > > + else
> > > > > + return ticket_spin_is_contended(lock);
> > > > > +}
> > > > > +#else /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > > > +
> >
> > Also, those functions all reproduce the same behavior, so maybe it would be
> > better to keep that behavior in a macro such as:
> >
> > #define COMBO_SPINLOCK_DECLARE(f) \
> > static __always_inline int arch_spin_ ## f(arch_spinlock_t *lock) \
> > { \
> > if (static_branch_likely(&combo_qspinlock_key)) \
> > return queued_spin_ ## f(lock); \
> > else \
> > return ticket_spin_ ## f(lock); \
> > }
> >
> > COMBO_SPINLOCK_DECLARE(lock)
> > COMBO_SPINLOCK_DECLARE(trylock)
> > COMBO_SPINLOCK_DECLARE(unlock)
> > COMBO_SPINLOCK_DECLARE(value_unlocked)
> > COMBO_SPINLOCK_DECLARE(is_locked)
> > COMBO_SPINLOCK_DECLARE(is_contended)
> >
> > Does that make sense?
> Yeah, thanks. I would try. You improved my macro skills. :)
Glad to be of any help :)
>
> >
> > Thanks!
> > Leo
> >
> >
> > > > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > #include <asm/qspinlock.h>
> > > > > -#include <asm/qrwlock.h>
> > > > > #else
> > > > > -#include <asm-generic/spinlock.h>
> > > > > +#include <asm-generic/ticket_spinlock.h>
> > > > > #endif
> > > > >
> > > > > +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */
> > > > > +
> > > > > +#include <asm/qrwlock.h>
> > > > > +
> > > > > #endif /* __ASM_RISCV_SPINLOCK_H */
> > > > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > > > index 32c2e1eb71bd..a447cf360a18 100644
> > > > > --- a/arch/riscv/kernel/setup.c
> > > > > +++ b/arch/riscv/kernel/setup.c
> > > > > @@ -269,6 +269,18 @@ static void __init parse_dtb(void)
> > > > > #endif
> > > > > }
> > > > >
> > > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > > +DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > > +EXPORT_SYMBOL(combo_qspinlock_key);
> > > > > +#endif
> > > > > +
> > > > > +static void __init riscv_spinlock_init(void)
> > > > > +{
> > > > > +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > > + static_branch_disable(&combo_qspinlock_key);
> > > > > +#endif
> > > > > +}
> > > > > +
> > > > > extern void __init init_rt_signal_env(void);
> > > > >
> > > > > void __init setup_arch(char **cmdline_p)
> > > > > @@ -317,6 +329,8 @@ void __init setup_arch(char **cmdline_p)
> > > > > riscv_isa_extension_available(NULL, ZICBOM))
> > > > > riscv_noncoherent_supported();
> > > > > riscv_set_dma_cache_alignment();
> > > > > +
> > > > > + riscv_spinlock_init();
> > > > > }
> > > > >
> > > > > static int __init topology_init(void)
> > > > > --
> > > > > 2.36.1
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > linux-riscv mailing list
> > > > > linux-riscv@lists.infradead.org
> > > > > http://lists.infradead.org/mailman/listinfo/linux-riscv
> > > > >
> > > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
2023-09-12 1:08 ` Guo Ren
@ 2023-09-14 7:32 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 7:32 UTC (permalink / raw)
To: Guo Ren
Cc: Waiman Long, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 12, 2023 at 09:08:34AM +0800, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 11:34 PM Waiman Long <longman@redhat.com> wrote:
> >
> > On 9/10/23 04:29, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Allow cmdline to force the kernel to use queued_spinlock when
> > > CONFIG_RISCV_COMBO_SPINLOCKS=y.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > Documentation/admin-guide/kernel-parameters.txt | 2 ++
> > > arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> > > 2 files changed, 17 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > index 7dfb540c4f6c..61cacb8dfd0e 100644
> > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > @@ -4693,6 +4693,8 @@
> > > [KNL] Number of legacy pty's. Overwrites compiled-in
> > > default number.
> > >
> > > + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> > > +
> > > qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> > > Set the time threshold in nanoseconds for the
> > > number of intra-node lock hand-offs before the
> > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > index a447cf360a18..0f084f037651 100644
> > > --- a/arch/riscv/kernel/setup.c
> > > +++ b/arch/riscv/kernel/setup.c
> > > @@ -270,6 +270,15 @@ static void __init parse_dtb(void)
> > > }
> > >
> > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > +bool enable_qspinlock_key = false;
> >
> > You can use __ro_after_init qualifier for enable_qspinlock_key. BTW,
> > this is not a static key, just a simple flag. So what is the point of
> > the _key suffix?
> Okay, I would change it to:
> bool enable_qspinlock_flag __ro_after_init = false;
IIUC, this bool / flag is used in a single file, so it makes sense for it
to be static. Being static means it does not need to be initialized to
false, as it's standard to zero-fill this areas.
Also, since it's a bool, it does not need to be called _flag.
I would go with:
static bool enable_qspinlock __ro_after_init;
>
> >
> > Cheers,
> > Longman
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
@ 2023-09-14 7:32 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 7:32 UTC (permalink / raw)
To: Guo Ren
Cc: Waiman Long, paul.walmsley, anup, peterz, mingo, will, palmer,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 12, 2023 at 09:08:34AM +0800, Guo Ren wrote:
> On Mon, Sep 11, 2023 at 11:34 PM Waiman Long <longman@redhat.com> wrote:
> >
> > On 9/10/23 04:29, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Allow cmdline to force the kernel to use queued_spinlock when
> > > CONFIG_RISCV_COMBO_SPINLOCKS=y.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > Documentation/admin-guide/kernel-parameters.txt | 2 ++
> > > arch/riscv/kernel/setup.c | 16 +++++++++++++++-
> > > 2 files changed, 17 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > index 7dfb540c4f6c..61cacb8dfd0e 100644
> > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > @@ -4693,6 +4693,8 @@
> > > [KNL] Number of legacy pty's. Overwrites compiled-in
> > > default number.
> > >
> > > + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
> > > +
> > > qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
> > > Set the time threshold in nanoseconds for the
> > > number of intra-node lock hand-offs before the
> > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > index a447cf360a18..0f084f037651 100644
> > > --- a/arch/riscv/kernel/setup.c
> > > +++ b/arch/riscv/kernel/setup.c
> > > @@ -270,6 +270,15 @@ static void __init parse_dtb(void)
> > > }
> > >
> > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > +bool enable_qspinlock_key = false;
> >
> > You can use __ro_after_init qualifier for enable_qspinlock_key. BTW,
> > this is not a static key, just a simple flag. So what is the point of
> > the _key suffix?
> Okay, I would change it to:
> bool enable_qspinlock_flag __ro_after_init = false;
IIUC, this bool / flag is used in a single file, so it makes sense for it
to be static. Being static means it does not need to be initialized to
false, as it's standard to zero-fill this areas.
Also, since it's a bool, it does not need to be called _flag.
I would go with:
static bool enable_qspinlock __ro_after_init;
>
> >
> > Cheers,
> > Longman
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
2023-09-10 8:29 ` guoren
@ 2023-09-14 8:02 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 8:02 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:02AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Add a static key controlling whether virt_spin_lock() should be
> called or not. When running on bare metal set the new key to
> false.
>
> The KVM guests fall back to a Test-and-Set spinlock, because fair
> locks have horrible lock 'holder' preemption issues. The
> virt_spin_lock_key would shortcut for the
> queued_spin_lock_slowpath() function that allow virt_spin_lock to
> hijack it.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> .../admin-guide/kernel-parameters.txt | 4 +++
> arch/riscv/include/asm/sbi.h | 8 +++++
> arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
> arch/riscv/kernel/sbi.c | 2 +-
> arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
> 5 files changed, 64 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 61cacb8dfd0e..f75bedc50e00 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -3927,6 +3927,10 @@
> no_uaccess_flush
> [PPC] Don't flush the L1-D cache after accessing user data.
>
> + no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
> + native_queued_spinlock when the nopvspin option is enabled.
> + This would help vcpu=pcpu scenarios.
> +
> novmcoredd [KNL,KDUMP]
> Disable device dump. Device dump allows drivers to
> append dump data to vmcore so you can collect driver
> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index 501e06e52078..e0233b3d7a5f 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
> SBI_EXT_BASE_GET_MIMPID,
> };
>
> +enum sbi_ext_base_impl_id {
> + SBI_EXT_BASE_IMPL_ID_BBL = 0,
> + SBI_EXT_BASE_IMPL_ID_OPENSBI,
> + SBI_EXT_BASE_IMPL_ID_XVISOR,
> + SBI_EXT_BASE_IMPL_ID_KVM,
> +};
> +
> enum sbi_ext_time_fid {
> SBI_EXT_TIME_SET_TIMER = 0,
> };
> @@ -269,6 +276,7 @@ int sbi_console_getchar(void);
> long sbi_get_mvendorid(void);
> long sbi_get_marchid(void);
> long sbi_get_mimpid(void);
> +long sbi_get_firmware_id(void);
> void sbi_set_timer(uint64_t stime_value);
> void sbi_shutdown(void);
> void sbi_send_ipi(unsigned int cpu);
> diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> index 8ea0fee80652..6b38d6616f14 100644
> --- a/arch/riscv/include/asm/spinlock.h
> +++ b/arch/riscv/include/asm/spinlock.h
> @@ -4,6 +4,28 @@
> #define __ASM_RISCV_SPINLOCK_H
>
> #ifdef CONFIG_QUEUED_SPINLOCKS
> +/*
> + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
> + * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
> + * would shortcut for the queued_spin_lock_slowpath() function that allow
> + * virt_spin_lock to hijack it.
> + */
> +DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
> +
> +#define virt_spin_lock virt_spin_lock
> +static inline bool virt_spin_lock(struct qspinlock *lock)
> +{
> + if (!static_branch_likely(&virt_spin_lock_key))
> + return false;
> +
> + do {
> + while (atomic_read(&lock->val) != 0)
> + cpu_relax();
> + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
> +
> + return true;
> +}
> +
> #define _Q_PENDING_LOOPS (1 << 9)
> #endif
>
> diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> index 88eea3a99ee0..cdd45edc8db4 100644
> --- a/arch/riscv/kernel/sbi.c
> +++ b/arch/riscv/kernel/sbi.c
> @@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
> return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
> }
>
> -static inline long sbi_get_firmware_id(void)
> +long sbi_get_firmware_id(void)
> {
> return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
> }
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index 0f084f037651..c57d15b05160 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -26,6 +26,7 @@
> #include <asm/alternative.h>
> #include <asm/cacheflush.h>
> #include <asm/cpu_ops.h>
> +#include <asm/cpufeature.h>
> #include <asm/early_ioremap.h>
> #include <asm/pgtable.h>
> #include <asm/setup.h>
> @@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> EXPORT_SYMBOL(combo_qspinlock_key);
> #endif
>
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +static bool no_virt_spin_key = false;
I suggest no _key, also there is no need for "= false".
To be consistent with enable_qspinlock, I also suggest
adding __ro_after_init:
static bool no_virt_spin __ro_after_init;
> +DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
> +
> +static int __init no_virt_spin_setup(char *p)
> +{
> + no_virt_spin_key = true;
> +
> + return 0;
> +}
> +early_param("no_virt_spin", no_virt_spin_setup);
> +
> +static void __init virt_spin_lock_init(void)
> +{
> + if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
> + no_virt_spin_key)
> + static_branch_disable(&virt_spin_lock_key);
> + else
> + pr_info("Enable virt_spin_lock\n");
> +}
> +#endif
> +
A new virt_no_spin kernel parameter was introduced, but without
CONFIG_QUEUED_SPINLOCKS it will silently fail.
I would suggest an #else clause here with a function to print an error /
warning message about no_virt_spin being invalid in this scenario.
It will probably help future debugging.
> static void __init riscv_spinlock_init(void)
> {
> #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> - if (!enable_qspinlock_key) {
> + if (!enable_qspinlock_key &&
> + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> static_branch_disable(&combo_qspinlock_key);
> pr_info("Ticket spinlock: enabled\n");
> } else {
> pr_info("Queued spinlock: enabled\n");
> }
> #endif
> +
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> + virt_spin_lock_init();
> +#endif
> }
>
> extern void __init init_rt_signal_env(void);
> --
> 2.36.1
>
I am probably missing something out, but it looks to me that this patch is
causing 2 different changes:
1 - Enabling no_virt_spin parameter
2 - Disabling queued spinlocks for some firmware_id
Wouldn't be better to split those changes in multiple patches?
Or am I missing the point on why they need to be together?
Thanks!
Leo
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
@ 2023-09-14 8:02 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 8:02 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:02AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Add a static key controlling whether virt_spin_lock() should be
> called or not. When running on bare metal set the new key to
> false.
>
> The KVM guests fall back to a Test-and-Set spinlock, because fair
> locks have horrible lock 'holder' preemption issues. The
> virt_spin_lock_key would shortcut for the
> queued_spin_lock_slowpath() function that allow virt_spin_lock to
> hijack it.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> .../admin-guide/kernel-parameters.txt | 4 +++
> arch/riscv/include/asm/sbi.h | 8 +++++
> arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
> arch/riscv/kernel/sbi.c | 2 +-
> arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
> 5 files changed, 64 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 61cacb8dfd0e..f75bedc50e00 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -3927,6 +3927,10 @@
> no_uaccess_flush
> [PPC] Don't flush the L1-D cache after accessing user data.
>
> + no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
> + native_queued_spinlock when the nopvspin option is enabled.
> + This would help vcpu=pcpu scenarios.
> +
> novmcoredd [KNL,KDUMP]
> Disable device dump. Device dump allows drivers to
> append dump data to vmcore so you can collect driver
> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index 501e06e52078..e0233b3d7a5f 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
> SBI_EXT_BASE_GET_MIMPID,
> };
>
> +enum sbi_ext_base_impl_id {
> + SBI_EXT_BASE_IMPL_ID_BBL = 0,
> + SBI_EXT_BASE_IMPL_ID_OPENSBI,
> + SBI_EXT_BASE_IMPL_ID_XVISOR,
> + SBI_EXT_BASE_IMPL_ID_KVM,
> +};
> +
> enum sbi_ext_time_fid {
> SBI_EXT_TIME_SET_TIMER = 0,
> };
> @@ -269,6 +276,7 @@ int sbi_console_getchar(void);
> long sbi_get_mvendorid(void);
> long sbi_get_marchid(void);
> long sbi_get_mimpid(void);
> +long sbi_get_firmware_id(void);
> void sbi_set_timer(uint64_t stime_value);
> void sbi_shutdown(void);
> void sbi_send_ipi(unsigned int cpu);
> diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> index 8ea0fee80652..6b38d6616f14 100644
> --- a/arch/riscv/include/asm/spinlock.h
> +++ b/arch/riscv/include/asm/spinlock.h
> @@ -4,6 +4,28 @@
> #define __ASM_RISCV_SPINLOCK_H
>
> #ifdef CONFIG_QUEUED_SPINLOCKS
> +/*
> + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
> + * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
> + * would shortcut for the queued_spin_lock_slowpath() function that allow
> + * virt_spin_lock to hijack it.
> + */
> +DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
> +
> +#define virt_spin_lock virt_spin_lock
> +static inline bool virt_spin_lock(struct qspinlock *lock)
> +{
> + if (!static_branch_likely(&virt_spin_lock_key))
> + return false;
> +
> + do {
> + while (atomic_read(&lock->val) != 0)
> + cpu_relax();
> + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
> +
> + return true;
> +}
> +
> #define _Q_PENDING_LOOPS (1 << 9)
> #endif
>
> diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> index 88eea3a99ee0..cdd45edc8db4 100644
> --- a/arch/riscv/kernel/sbi.c
> +++ b/arch/riscv/kernel/sbi.c
> @@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
> return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
> }
>
> -static inline long sbi_get_firmware_id(void)
> +long sbi_get_firmware_id(void)
> {
> return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
> }
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index 0f084f037651..c57d15b05160 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -26,6 +26,7 @@
> #include <asm/alternative.h>
> #include <asm/cacheflush.h>
> #include <asm/cpu_ops.h>
> +#include <asm/cpufeature.h>
> #include <asm/early_ioremap.h>
> #include <asm/pgtable.h>
> #include <asm/setup.h>
> @@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> EXPORT_SYMBOL(combo_qspinlock_key);
> #endif
>
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +static bool no_virt_spin_key = false;
I suggest no _key, also there is no need for "= false".
To be consistent with enable_qspinlock, I also suggest
adding __ro_after_init:
static bool no_virt_spin __ro_after_init;
> +DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
> +
> +static int __init no_virt_spin_setup(char *p)
> +{
> + no_virt_spin_key = true;
> +
> + return 0;
> +}
> +early_param("no_virt_spin", no_virt_spin_setup);
> +
> +static void __init virt_spin_lock_init(void)
> +{
> + if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
> + no_virt_spin_key)
> + static_branch_disable(&virt_spin_lock_key);
> + else
> + pr_info("Enable virt_spin_lock\n");
> +}
> +#endif
> +
A new virt_no_spin kernel parameter was introduced, but without
CONFIG_QUEUED_SPINLOCKS it will silently fail.
I would suggest an #else clause here with a function to print an error /
warning message about no_virt_spin being invalid in this scenario.
It will probably help future debugging.
> static void __init riscv_spinlock_init(void)
> {
> #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> - if (!enable_qspinlock_key) {
> + if (!enable_qspinlock_key &&
> + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> static_branch_disable(&combo_qspinlock_key);
> pr_info("Ticket spinlock: enabled\n");
> } else {
> pr_info("Queued spinlock: enabled\n");
> }
> #endif
> +
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> + virt_spin_lock_init();
> +#endif
> }
>
> extern void __init init_rt_signal_env(void);
> --
> 2.36.1
>
I am probably missing something out, but it looks to me that this patch is
causing 2 different changes:
1 - Enabling no_virt_spin parameter
2 - Disabling queued spinlocks for some firmware_id
Wouldn't be better to split those changes in multiple patches?
Or am I missing the point on why they need to be together?
Thanks!
Leo
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 09/17] riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
2023-09-10 8:29 ` guoren
@ 2023-09-14 8:32 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 8:32 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:03AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The early version of T-Head C9xx cores has a store merge buffer
> delay problem. The store merge buffer could improve the store queue
> performance by merging multi-store requests, but when there are not
> continued store requests, the prior single store request would be
> waiting in the store queue for a long time. That would cause
> significant problems for communication between multi-cores. This
> problem was found on sg2042 & th1520 platforms with the qspinlock
> lock torture test.
>
> So appending a fence w.o could immediately flush the store merge
> buffer and let other cores see the write result.
>
> This will apply the WRITE_ONCE errata to handle the non-standard
> behavior via appending a fence w.o instruction for WRITE_ONCE().
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig.errata | 19 +++++++++++++++++++
> arch/riscv/errata/thead/errata.c | 20 ++++++++++++++++++++
> arch/riscv/include/asm/errata_list.h | 13 -------------
> arch/riscv/include/asm/rwonce.h | 24 ++++++++++++++++++++++++
> arch/riscv/include/asm/vendorid_list.h | 14 ++++++++++++++
> include/asm-generic/rwonce.h | 2 ++
> 6 files changed, 79 insertions(+), 13 deletions(-)
> create mode 100644 arch/riscv/include/asm/rwonce.h
>
> diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata
> index 1aa85a427ff3..c919cc3f1a3a 100644
> --- a/arch/riscv/Kconfig.errata
> +++ b/arch/riscv/Kconfig.errata
> @@ -77,4 +77,23 @@ config ERRATA_THEAD_PMU
>
> If you don't know what to do here, say "Y".
>
> +config ERRATA_THEAD_WRITE_ONCE
> + bool "Apply T-Head WRITE_ONCE errata"
> + depends on ERRATA_THEAD
> + default y
> + help
> + The early version of T-Head C9xx cores has a store merge buffer
> + delay problem. The store merge buffer could improve the store queue
> + performance by merging multi-store requests, but when there are no
> + continued store requests, the prior single store request would be
> + waiting in the store queue for a long time. That would cause
> + significant problems for communication between multi-cores. Appending
> + a fence w.o could immediately flush the store merge buffer and let
> + other cores see the write result.
> +
> + This will apply the WRITE_ONCE errata to handle the non-standard
> + behavior via appending a fence w.o instruction for WRITE_ONCE().
> +
> + If you don't know what to do here, say "Y".
> +
> endmenu # "CPU errata selection"
> diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
> index be84b14f0118..751eb5a7f614 100644
> --- a/arch/riscv/errata/thead/errata.c
> +++ b/arch/riscv/errata/thead/errata.c
> @@ -69,6 +69,23 @@ static bool errata_probe_pmu(unsigned int stage,
> return true;
> }
>
> +static bool errata_probe_write_once(unsigned int stage,
> + unsigned long arch_id, unsigned long impid)
> +{
> + if (!IS_ENABLED(CONFIG_ERRATA_THEAD_WRITE_ONCE))
> + return false;
> +
> + /* target-c9xx cores report arch_id and impid as 0 */
> + if (arch_id != 0 || impid != 0)
> + return false;
> +
> + if (stage == RISCV_ALTERNATIVES_BOOT ||
> + stage == RISCV_ALTERNATIVES_MODULE)
> + return true;
> +
> + return false;
> +}
> +
> static u32 thead_errata_probe(unsigned int stage,
> unsigned long archid, unsigned long impid)
> {
> @@ -83,6 +100,9 @@ static u32 thead_errata_probe(unsigned int stage,
> if (errata_probe_pmu(stage, archid, impid))
> cpu_req_errata |= BIT(ERRATA_THEAD_PMU);
>
> + if (errata_probe_write_once(stage, archid, impid))
> + cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
> +
> return cpu_req_errata;
> }
>
> diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
> index 712cab7adffe..fbb2b8d39321 100644
> --- a/arch/riscv/include/asm/errata_list.h
> +++ b/arch/riscv/include/asm/errata_list.h
> @@ -11,19 +11,6 @@
> #include <asm/hwcap.h>
> #include <asm/vendorid_list.h>
>
> -#ifdef CONFIG_ERRATA_SIFIVE
> -#define ERRATA_SIFIVE_CIP_453 0
> -#define ERRATA_SIFIVE_CIP_1200 1
> -#define ERRATA_SIFIVE_NUMBER 2
> -#endif
> -
> -#ifdef CONFIG_ERRATA_THEAD
> -#define ERRATA_THEAD_PBMT 0
> -#define ERRATA_THEAD_CMO 1
> -#define ERRATA_THEAD_PMU 2
> -#define ERRATA_THEAD_NUMBER 3
> -#endif
> -
Here I understand you are moving stuff from errata_list.h to
vendorid_list.h. Wouldn't it be better to do this on a separated patch
before this one?
I understand this is used here, but it looks like it's unrelated.
> #ifdef __ASSEMBLY__
>
> #define ALT_INSN_FAULT(x) \
> diff --git a/arch/riscv/include/asm/rwonce.h b/arch/riscv/include/asm/rwonce.h
> new file mode 100644
> index 000000000000..be0b8864969d
> --- /dev/null
> +++ b/arch/riscv/include/asm/rwonce.h
> @@ -0,0 +1,24 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#ifndef __ASM_RWONCE_H
> +#define __ASM_RWONCE_H
> +
> +#include <linux/compiler_types.h>
> +#include <asm/alternative-macros.h>
> +#include <asm/vendorid_list.h>
> +
> +#define __WRITE_ONCE(x, val) \
> +do { \
> + *(volatile typeof(x) *)&(x) = (val); \
> + asm volatile(ALTERNATIVE( \
> + __nops(1), \
> + "fence w, o\n\t", \
> + THEAD_VENDOR_ID, \
> + ERRATA_THEAD_WRITE_ONCE, \
> + CONFIG_ERRATA_THEAD_WRITE_ONCE) \
> + : : : "memory"); \
> +} while (0)
> +
> +#include <asm-generic/rwonce.h>
> +
> +#endif /* __ASM_RWONCE_H */
IIUC the idea here is to have an alternative __WRITE_ONCE that replaces the
asm-generic one.
Honestly, this asm alternative here seems too much information, and too
cryptic. I mean, yeah in the patch it all makes sense, but I imagine myself
in the future looking at all this and trying to understand what is going
on.
Wouldn't it look better to have something like:
#####
/* Some explanation like the one on Kconfig */
#define write_once_flush() \
do { \
asm volatile(ALTERNATIVE( \
__nops(1), \
"fence w, o\n\t", \
THEAD_VENDOR_ID, \
ERRATA_THEAD_WRITE_ONCE, \
CONFIG_ERRATA_THEAD_WRITE_ONCE) \
: : : "memory"); \
} while(0)
#define __WRITE_ONCE(x, val) \
do { \
*(volatile typeof(x) *)&(x) = (val); \
write_once_flush(); \
} while(0)
#####
This way I could quickly see there is a flush after the writting of
WRITE_ONCE(), and this flush is the above "complicated" asm.
What do you think?
> diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
> index cb89af3f0704..73078cfe4029 100644
> --- a/arch/riscv/include/asm/vendorid_list.h
> +++ b/arch/riscv/include/asm/vendorid_list.h
> @@ -8,4 +8,18 @@
> #define SIFIVE_VENDOR_ID 0x489
> #define THEAD_VENDOR_ID 0x5b7
>
> +#ifdef CONFIG_ERRATA_SIFIVE
> +#define ERRATA_SIFIVE_CIP_453 0
> +#define ERRATA_SIFIVE_CIP_1200 1
> +#define ERRATA_SIFIVE_NUMBER 2
> +#endif
> +
> +#ifdef CONFIG_ERRATA_THEAD
> +#define ERRATA_THEAD_PBMT 0
> +#define ERRATA_THEAD_CMO 1
> +#define ERRATA_THEAD_PMU 2
> +#define ERRATA_THEAD_WRITE_ONCE 3
> +#define ERRATA_THEAD_NUMBER 4
> +#endif
> +
> #endif
> diff --git a/include/asm-generic/rwonce.h b/include/asm-generic/rwonce.h
> index 8d0a6280e982..fb07fe8c6e45 100644
> --- a/include/asm-generic/rwonce.h
> +++ b/include/asm-generic/rwonce.h
> @@ -50,10 +50,12 @@
> __READ_ONCE(x); \
> })
>
> +#ifndef __WRITE_ONCE
> #define __WRITE_ONCE(x, val) \
> do { \
> *(volatile typeof(x) *)&(x) = (val); \
> } while (0)
> +#endif
>
> #define WRITE_ONCE(x, val) \
> do { \
> --
> 2.36.1
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 09/17] riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
@ 2023-09-14 8:32 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 8:32 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:03AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> The early version of T-Head C9xx cores has a store merge buffer
> delay problem. The store merge buffer could improve the store queue
> performance by merging multi-store requests, but when there are not
> continued store requests, the prior single store request would be
> waiting in the store queue for a long time. That would cause
> significant problems for communication between multi-cores. This
> problem was found on sg2042 & th1520 platforms with the qspinlock
> lock torture test.
>
> So appending a fence w.o could immediately flush the store merge
> buffer and let other cores see the write result.
>
> This will apply the WRITE_ONCE errata to handle the non-standard
> behavior via appending a fence w.o instruction for WRITE_ONCE().
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig.errata | 19 +++++++++++++++++++
> arch/riscv/errata/thead/errata.c | 20 ++++++++++++++++++++
> arch/riscv/include/asm/errata_list.h | 13 -------------
> arch/riscv/include/asm/rwonce.h | 24 ++++++++++++++++++++++++
> arch/riscv/include/asm/vendorid_list.h | 14 ++++++++++++++
> include/asm-generic/rwonce.h | 2 ++
> 6 files changed, 79 insertions(+), 13 deletions(-)
> create mode 100644 arch/riscv/include/asm/rwonce.h
>
> diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata
> index 1aa85a427ff3..c919cc3f1a3a 100644
> --- a/arch/riscv/Kconfig.errata
> +++ b/arch/riscv/Kconfig.errata
> @@ -77,4 +77,23 @@ config ERRATA_THEAD_PMU
>
> If you don't know what to do here, say "Y".
>
> +config ERRATA_THEAD_WRITE_ONCE
> + bool "Apply T-Head WRITE_ONCE errata"
> + depends on ERRATA_THEAD
> + default y
> + help
> + The early version of T-Head C9xx cores has a store merge buffer
> + delay problem. The store merge buffer could improve the store queue
> + performance by merging multi-store requests, but when there are no
> + continued store requests, the prior single store request would be
> + waiting in the store queue for a long time. That would cause
> + significant problems for communication between multi-cores. Appending
> + a fence w.o could immediately flush the store merge buffer and let
> + other cores see the write result.
> +
> + This will apply the WRITE_ONCE errata to handle the non-standard
> + behavior via appending a fence w.o instruction for WRITE_ONCE().
> +
> + If you don't know what to do here, say "Y".
> +
> endmenu # "CPU errata selection"
> diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
> index be84b14f0118..751eb5a7f614 100644
> --- a/arch/riscv/errata/thead/errata.c
> +++ b/arch/riscv/errata/thead/errata.c
> @@ -69,6 +69,23 @@ static bool errata_probe_pmu(unsigned int stage,
> return true;
> }
>
> +static bool errata_probe_write_once(unsigned int stage,
> + unsigned long arch_id, unsigned long impid)
> +{
> + if (!IS_ENABLED(CONFIG_ERRATA_THEAD_WRITE_ONCE))
> + return false;
> +
> + /* target-c9xx cores report arch_id and impid as 0 */
> + if (arch_id != 0 || impid != 0)
> + return false;
> +
> + if (stage == RISCV_ALTERNATIVES_BOOT ||
> + stage == RISCV_ALTERNATIVES_MODULE)
> + return true;
> +
> + return false;
> +}
> +
> static u32 thead_errata_probe(unsigned int stage,
> unsigned long archid, unsigned long impid)
> {
> @@ -83,6 +100,9 @@ static u32 thead_errata_probe(unsigned int stage,
> if (errata_probe_pmu(stage, archid, impid))
> cpu_req_errata |= BIT(ERRATA_THEAD_PMU);
>
> + if (errata_probe_write_once(stage, archid, impid))
> + cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
> +
> return cpu_req_errata;
> }
>
> diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
> index 712cab7adffe..fbb2b8d39321 100644
> --- a/arch/riscv/include/asm/errata_list.h
> +++ b/arch/riscv/include/asm/errata_list.h
> @@ -11,19 +11,6 @@
> #include <asm/hwcap.h>
> #include <asm/vendorid_list.h>
>
> -#ifdef CONFIG_ERRATA_SIFIVE
> -#define ERRATA_SIFIVE_CIP_453 0
> -#define ERRATA_SIFIVE_CIP_1200 1
> -#define ERRATA_SIFIVE_NUMBER 2
> -#endif
> -
> -#ifdef CONFIG_ERRATA_THEAD
> -#define ERRATA_THEAD_PBMT 0
> -#define ERRATA_THEAD_CMO 1
> -#define ERRATA_THEAD_PMU 2
> -#define ERRATA_THEAD_NUMBER 3
> -#endif
> -
Here I understand you are moving stuff from errata_list.h to
vendorid_list.h. Wouldn't it be better to do this on a separated patch
before this one?
I understand this is used here, but it looks like it's unrelated.
> #ifdef __ASSEMBLY__
>
> #define ALT_INSN_FAULT(x) \
> diff --git a/arch/riscv/include/asm/rwonce.h b/arch/riscv/include/asm/rwonce.h
> new file mode 100644
> index 000000000000..be0b8864969d
> --- /dev/null
> +++ b/arch/riscv/include/asm/rwonce.h
> @@ -0,0 +1,24 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#ifndef __ASM_RWONCE_H
> +#define __ASM_RWONCE_H
> +
> +#include <linux/compiler_types.h>
> +#include <asm/alternative-macros.h>
> +#include <asm/vendorid_list.h>
> +
> +#define __WRITE_ONCE(x, val) \
> +do { \
> + *(volatile typeof(x) *)&(x) = (val); \
> + asm volatile(ALTERNATIVE( \
> + __nops(1), \
> + "fence w, o\n\t", \
> + THEAD_VENDOR_ID, \
> + ERRATA_THEAD_WRITE_ONCE, \
> + CONFIG_ERRATA_THEAD_WRITE_ONCE) \
> + : : : "memory"); \
> +} while (0)
> +
> +#include <asm-generic/rwonce.h>
> +
> +#endif /* __ASM_RWONCE_H */
IIUC the idea here is to have an alternative __WRITE_ONCE that replaces the
asm-generic one.
Honestly, this asm alternative here seems too much information, and too
cryptic. I mean, yeah in the patch it all makes sense, but I imagine myself
in the future looking at all this and trying to understand what is going
on.
Wouldn't it look better to have something like:
#####
/* Some explanation like the one on Kconfig */
#define write_once_flush() \
do { \
asm volatile(ALTERNATIVE( \
__nops(1), \
"fence w, o\n\t", \
THEAD_VENDOR_ID, \
ERRATA_THEAD_WRITE_ONCE, \
CONFIG_ERRATA_THEAD_WRITE_ONCE) \
: : : "memory"); \
} while(0)
#define __WRITE_ONCE(x, val) \
do { \
*(volatile typeof(x) *)&(x) = (val); \
write_once_flush(); \
} while(0)
#####
This way I could quickly see there is a flush after the writting of
WRITE_ONCE(), and this flush is the above "complicated" asm.
What do you think?
> diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
> index cb89af3f0704..73078cfe4029 100644
> --- a/arch/riscv/include/asm/vendorid_list.h
> +++ b/arch/riscv/include/asm/vendorid_list.h
> @@ -8,4 +8,18 @@
> #define SIFIVE_VENDOR_ID 0x489
> #define THEAD_VENDOR_ID 0x5b7
>
> +#ifdef CONFIG_ERRATA_SIFIVE
> +#define ERRATA_SIFIVE_CIP_453 0
> +#define ERRATA_SIFIVE_CIP_1200 1
> +#define ERRATA_SIFIVE_NUMBER 2
> +#endif
> +
> +#ifdef CONFIG_ERRATA_THEAD
> +#define ERRATA_THEAD_PBMT 0
> +#define ERRATA_THEAD_CMO 1
> +#define ERRATA_THEAD_PMU 2
> +#define ERRATA_THEAD_WRITE_ONCE 3
> +#define ERRATA_THEAD_NUMBER 4
> +#endif
> +
> #endif
> diff --git a/include/asm-generic/rwonce.h b/include/asm-generic/rwonce.h
> index 8d0a6280e982..fb07fe8c6e45 100644
> --- a/include/asm-generic/rwonce.h
> +++ b/include/asm-generic/rwonce.h
> @@ -50,10 +50,12 @@
> __READ_ONCE(x); \
> })
>
> +#ifndef __WRITE_ONCE
> #define __WRITE_ONCE(x, val) \
> do { \
> *(volatile typeof(x) *)&(x) = (val); \
> } while (0)
> +#endif
>
> #define WRITE_ONCE(x, val) \
> do { \
> --
> 2.36.1
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 10/17] riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
2023-09-10 8:29 ` guoren
@ 2023-09-14 9:36 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 9:36 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:04AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> According to qspinlock requirements, RISC-V gives out a weak LR/SC
> forward progress guarantee which does not satisfy qspinlock. But
> many vendors could produce stronger forward guarantee LR/SC to
> ensure the xchg_tail could be finished in time on any kind of
> hart. T-HEAD is the vendor
nit: "a vendor"
> which implements strong forward
> guarantee LR/SC instruction pairs, so enable qspinlock for T-HEAD
> with errata init help.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/errata/thead/errata.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
> index 751eb5a7f614..0df6a67302c0 100644
> --- a/arch/riscv/errata/thead/errata.c
> +++ b/arch/riscv/errata/thead/errata.c
> @@ -86,6 +86,13 @@ static bool errata_probe_write_once(unsigned int stage,
> return false;
> }
>
> +extern bool enable_qspinlock_key;
Oh, now I understand why it was not static before. :)
> +static void errata_probe_qspinlock(unsigned int stage)
> +{
> + if (stage == RISCV_ALTERNATIVES_BOOT)
> + enable_qspinlock_key = true;
> +}
> +
> static u32 thead_errata_probe(unsigned int stage,
> unsigned long archid, unsigned long impid)
> {
> @@ -103,6 +110,8 @@ static u32 thead_errata_probe(unsigned int stage,
> if (errata_probe_write_once(stage, archid, impid))
> cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
>
> + errata_probe_qspinlock(stage);
> +
> return cpu_req_errata;
> }
>
So, if THEAD then enable qspinlock().
LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
> --
> 2.36.1
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 10/17] riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
@ 2023-09-14 9:36 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 9:36 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:04AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> According to qspinlock requirements, RISC-V gives out a weak LR/SC
> forward progress guarantee which does not satisfy qspinlock. But
> many vendors could produce stronger forward guarantee LR/SC to
> ensure the xchg_tail could be finished in time on any kind of
> hart. T-HEAD is the vendor
nit: "a vendor"
> which implements strong forward
> guarantee LR/SC instruction pairs, so enable qspinlock for T-HEAD
> with errata init help.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/errata/thead/errata.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
> index 751eb5a7f614..0df6a67302c0 100644
> --- a/arch/riscv/errata/thead/errata.c
> +++ b/arch/riscv/errata/thead/errata.c
> @@ -86,6 +86,13 @@ static bool errata_probe_write_once(unsigned int stage,
> return false;
> }
>
> +extern bool enable_qspinlock_key;
Oh, now I understand why it was not static before. :)
> +static void errata_probe_qspinlock(unsigned int stage)
> +{
> + if (stage == RISCV_ALTERNATIVES_BOOT)
> + enable_qspinlock_key = true;
> +}
> +
> static u32 thead_errata_probe(unsigned int stage,
> unsigned long archid, unsigned long impid)
> {
> @@ -103,6 +110,8 @@ static u32 thead_errata_probe(unsigned int stage,
> if (errata_probe_write_once(stage, archid, impid))
> cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
>
> + errata_probe_qspinlock(stage);
> +
> return cpu_req_errata;
> }
>
So, if THEAD then enable qspinlock().
LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
> --
> 2.36.1
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
2023-09-14 4:46 ` Guo Ren
@ 2023-09-14 9:43 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 9:43 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote:
> On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > The requirements of qspinlock have been documented by commit:
> > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > > atomics").
> > >
> > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > > is the one.
> > >
> > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > >
> > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > queued_spinlock 0.5109/0.00
> > > ticket_spinlock 0.5814/0.00
> > >
> > > perf futex/hash (+6.7%):
> > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > >
> > > perf futex/wake-parallel (+8.6%):
> > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > >
> > > perf futex/requeue (+4.2%):
> > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > >
> > > System Benchmarks (+6.4%)
> > > queued_spinlock:
> > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > Execl Throughput 43.0 13116.6 3050.4
> > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > Process Creation 126.0 12537.3 995.0
> > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > System Call Overhead 15000.0 33308301.3 22205.5
> > > ========
> > > System Benchmarks Index Score 12426.1
> > >
> > > ticket_spinlock:
> > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > Execl Throughput 43.0 12625.1 2936.1
> > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > Process Creation 126.0 10798.4 857.0
> > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > System Call Overhead 15000.0 30766778.4 20511.2
> > > ========
> > > System Benchmarks Index Score 11670.7
> > >
> > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > cores platform than the ticket_lock.
> > >
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > ---
> > > arch/riscv/Kconfig | 16 ++++++++++++++++
> > > arch/riscv/include/asm/Kbuild | 3 ++-
> > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > > 3 files changed, 35 insertions(+), 1 deletion(-)
> > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > >
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index 2c346fe169c1..7f39bfc75744 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > > Specify the maximum number of NUMA Nodes available on the target
> > > system. Increases memory reserved to accommodate various tables.
> > >
> > > +choice
> > > + prompt "RISC-V spinlock type"
> > > + default RISCV_TICKET_SPINLOCKS
> > > +
> > > +config RISCV_TICKET_SPINLOCKS
> > > + bool "Using ticket spinlock"
> > > +
> > > +config RISCV_QUEUED_SPINLOCKS
> > > + bool "Using queued spinlock"
> > > + depends on SMP && MMU
> > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > + help
> > > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > + Otherwise, stay at ticket-lock.
> > > +endchoice
> > > +
> > > config RISCV_ALTERNATIVE
> > > bool
> > > depends on !XIP_KERNEL
> > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > index 504f8b7e72d4..a0dc85e4a754 100644
> > > --- a/arch/riscv/include/asm/Kbuild
> > > +++ b/arch/riscv/include/asm/Kbuild
> > > @@ -2,10 +2,11 @@
> > > generic-y += early_ioremap.h
> > > generic-y += flat.h
> > > generic-y += kvm_para.h
> > > +generic-y += mcs_spinlock.h
> > > generic-y += parport.h
> > > -generic-y += spinlock.h
> >
> > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> > and include the asm-generic headers of mcs_spinlock and qspinlock.
> >
> > In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> > which seems the oposite of the above description (ticket spinlocks being
> > the standard).
> >
> > Shouldn't ticket-spinlock.h also get included here?
> > (Also, I am probably missing something, as I dont' see the use of
> > mcs_spinlock here.)
> No, because asm-generic/spinlock.h:
> ...
> #include <asm-generic/ticket_spinlock.h>
> ...
>
But aren't you removing asm-generic/spinlock.h below ?
-generic-y += spinlock.h
> >
> > > generic-y += spinlock_types.h
> > > generic-y += qrwlock.h
> > > generic-y += qrwlock_types.h
> > > +generic-y += qspinlock.h
> > > generic-y += user.h
> > > generic-y += vmlinux.lds.h
> > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > new file mode 100644
> > > index 000000000000..c644a92d4548
> > > --- /dev/null
> > > +++ b/arch/riscv/include/asm/spinlock.h
> > > @@ -0,0 +1,17 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +
> > > +#ifndef __ASM_RISCV_SPINLOCK_H
> > > +#define __ASM_RISCV_SPINLOCK_H
> > > +
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > +#define _Q_PENDING_LOOPS (1 << 9)
> > > +#endif
> >
> > Any reason the above define couldn't be merged on the ifdef below?
> Easy for the next patch to modify. See Waiman's comment:
>
> https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
>
> > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > index c644a92d4548..9eb3ad31e564 100644
> > --- a/arch/riscv/include/asm/spinlock.h
> > +++ b/arch/riscv/include/asm/spinlock.h
> > @@ -7,11 +7,94 @@
> > #define _Q_PENDING_LOOPS (1 << 9)
> > #endif
> >
>
> I see why you separated the _Q_PENDING_LOOPS out.
>
I see, should be fine then.
Thanks!
Leo
>
> >
> > > +
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > +#include <asm/qspinlock.h>
> > > +#include <asm/qrwlock.h>
> > > +#else
> > > +#include <asm-generic/spinlock.h>
> > > +#endif
> > > +
> > > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > > --
> > > 2.36.1
> > >
> >
> > Thanks!
> > Leo
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
@ 2023-09-14 9:43 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-14 9:43 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote:
> On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > The requirements of qspinlock have been documented by commit:
> > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > > atomics").
> > >
> > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > > is the one.
> > >
> > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > >
> > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > queued_spinlock 0.5109/0.00
> > > ticket_spinlock 0.5814/0.00
> > >
> > > perf futex/hash (+6.7%):
> > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > >
> > > perf futex/wake-parallel (+8.6%):
> > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > >
> > > perf futex/requeue (+4.2%):
> > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > >
> > > System Benchmarks (+6.4%)
> > > queued_spinlock:
> > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > Execl Throughput 43.0 13116.6 3050.4
> > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > Process Creation 126.0 12537.3 995.0
> > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > System Call Overhead 15000.0 33308301.3 22205.5
> > > ========
> > > System Benchmarks Index Score 12426.1
> > >
> > > ticket_spinlock:
> > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > Execl Throughput 43.0 12625.1 2936.1
> > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > Process Creation 126.0 10798.4 857.0
> > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > System Call Overhead 15000.0 30766778.4 20511.2
> > > ========
> > > System Benchmarks Index Score 11670.7
> > >
> > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > cores platform than the ticket_lock.
> > >
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > ---
> > > arch/riscv/Kconfig | 16 ++++++++++++++++
> > > arch/riscv/include/asm/Kbuild | 3 ++-
> > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > > 3 files changed, 35 insertions(+), 1 deletion(-)
> > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > >
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index 2c346fe169c1..7f39bfc75744 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > > Specify the maximum number of NUMA Nodes available on the target
> > > system. Increases memory reserved to accommodate various tables.
> > >
> > > +choice
> > > + prompt "RISC-V spinlock type"
> > > + default RISCV_TICKET_SPINLOCKS
> > > +
> > > +config RISCV_TICKET_SPINLOCKS
> > > + bool "Using ticket spinlock"
> > > +
> > > +config RISCV_QUEUED_SPINLOCKS
> > > + bool "Using queued spinlock"
> > > + depends on SMP && MMU
> > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > + help
> > > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > + Otherwise, stay at ticket-lock.
> > > +endchoice
> > > +
> > > config RISCV_ALTERNATIVE
> > > bool
> > > depends on !XIP_KERNEL
> > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > index 504f8b7e72d4..a0dc85e4a754 100644
> > > --- a/arch/riscv/include/asm/Kbuild
> > > +++ b/arch/riscv/include/asm/Kbuild
> > > @@ -2,10 +2,11 @@
> > > generic-y += early_ioremap.h
> > > generic-y += flat.h
> > > generic-y += kvm_para.h
> > > +generic-y += mcs_spinlock.h
> > > generic-y += parport.h
> > > -generic-y += spinlock.h
> >
> > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> > and include the asm-generic headers of mcs_spinlock and qspinlock.
> >
> > In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> > which seems the oposite of the above description (ticket spinlocks being
> > the standard).
> >
> > Shouldn't ticket-spinlock.h also get included here?
> > (Also, I am probably missing something, as I dont' see the use of
> > mcs_spinlock here.)
> No, because asm-generic/spinlock.h:
> ...
> #include <asm-generic/ticket_spinlock.h>
> ...
>
But aren't you removing asm-generic/spinlock.h below ?
-generic-y += spinlock.h
> >
> > > generic-y += spinlock_types.h
> > > generic-y += qrwlock.h
> > > generic-y += qrwlock_types.h
> > > +generic-y += qspinlock.h
> > > generic-y += user.h
> > > generic-y += vmlinux.lds.h
> > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > new file mode 100644
> > > index 000000000000..c644a92d4548
> > > --- /dev/null
> > > +++ b/arch/riscv/include/asm/spinlock.h
> > > @@ -0,0 +1,17 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +
> > > +#ifndef __ASM_RISCV_SPINLOCK_H
> > > +#define __ASM_RISCV_SPINLOCK_H
> > > +
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > +#define _Q_PENDING_LOOPS (1 << 9)
> > > +#endif
> >
> > Any reason the above define couldn't be merged on the ifdef below?
> Easy for the next patch to modify. See Waiman's comment:
>
> https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
>
> > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > index c644a92d4548..9eb3ad31e564 100644
> > --- a/arch/riscv/include/asm/spinlock.h
> > +++ b/arch/riscv/include/asm/spinlock.h
> > @@ -7,11 +7,94 @@
> > #define _Q_PENDING_LOOPS (1 << 9)
> > #endif
> >
>
> I see why you separated the _Q_PENDING_LOOPS out.
>
I see, should be fine then.
Thanks!
Leo
>
> >
> > > +
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > +#include <asm/qspinlock.h>
> > > +#include <asm/qrwlock.h>
> > > +#else
> > > +#include <asm-generic/spinlock.h>
> > > +#endif
> > > +
> > > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > > --
> > > 2.36.1
> > >
> >
> > Thanks!
> > Leo
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-10 8:28 ` guoren
@ 2023-09-14 13:47 ` Andrew Jones
-1 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-14 13:47 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Cache-block prefetch instructions are HINTs to the hardware to
> indicate that software intends to perform a particular type of
> memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> improve the arch_xchg for qspinlock xchg_tail.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 15 +++++++++++++++
> arch/riscv/include/asm/cmpxchg.h | 4 +++-
> arch/riscv/include/asm/hwcap.h | 1 +
> arch/riscv/include/asm/insn-def.h | 5 +++++
> arch/riscv/include/asm/processor.h | 13 +++++++++++++
> arch/riscv/kernel/cpufeature.c | 1 +
> 6 files changed, 38 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index e9ae6fa232c3..2c346fe169c1 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
>
> If you don't know what to do here, say Y.
>
> +config RISCV_ISA_ZICBOP
> + bool "Zicbop extension support for cache block prefetch"
> + depends on MMU
> + depends on RISCV_ALTERNATIVE
> + default y
> + help
> + Adds support to dynamically detect the presence of the ZICBOP
> + extension (Cache Block Prefetch Operations) and enable its
> + usage.
> +
> + The Zicbop extension can be used to prefetch cache block for
> + read/write/instruction fetch.
> +
> + If you don't know what to do here, say Y.
> +
> config TOOLCHAIN_HAS_ZIHINTPAUSE
> bool
> default y
> diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> index 702725727671..56eff7a9d2d2 100644
> --- a/arch/riscv/include/asm/cmpxchg.h
> +++ b/arch/riscv/include/asm/cmpxchg.h
> @@ -11,6 +11,7 @@
>
> #include <asm/barrier.h>
> #include <asm/fence.h>
> +#include <asm/processor.h>
>
> #define __arch_xchg_masked(prepend, append, r, p, n) \
> ({ \
> @@ -25,6 +26,7 @@
> \
> __asm__ __volatile__ ( \
> prepend \
> + PREFETCHW_ASM(%5) \
> "0: lr.w %0, %2\n" \
> " and %1, %0, %z4\n" \
> " or %1, %1, %z3\n" \
> @@ -32,7 +34,7 @@
> " bnez %1, 0b\n" \
> append \
> : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> - : "rJ" (__newx), "rJ" (~__mask) \
> + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> : "memory"); \
> \
> r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> index b7b58258f6c7..78b7b8b53778 100644
> --- a/arch/riscv/include/asm/hwcap.h
> +++ b/arch/riscv/include/asm/hwcap.h
> @@ -58,6 +58,7 @@
> #define RISCV_ISA_EXT_ZICSR 40
> #define RISCV_ISA_EXT_ZIFENCEI 41
> #define RISCV_ISA_EXT_ZIHPM 42
> +#define RISCV_ISA_EXT_ZICBOP 43
>
> #define RISCV_ISA_EXT_MAX 64
>
> diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> index 6960beb75f32..dc590d331894 100644
> --- a/arch/riscv/include/asm/insn-def.h
> +++ b/arch/riscv/include/asm/insn-def.h
> @@ -134,6 +134,7 @@
>
> #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
This should be named RV_OPCODE_OP_IMM and be placed in
numerical order with the others, i.e. above SYSTEM.
>
> #define HFENCE_VVMA(vaddr, asid) \
> INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> @@ -196,4 +197,8 @@
> INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> RS1(base), SIMM12(4))
>
> +#define CBO_prefetchw(base) \
Please name this 'PREFETCH_w' and it should take an immediate parameter,
even if we intend to pass 0 for it.
> + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> + RD(x0), RS1(base), RS2(x0))
prefetch.w is not an R-type instruction, it's an S-type. While the bit
shifts are the same, the names are different. We need to add S-type
names while defining this instruction. Then, this define would be
#define PREFETCH_w(base, imm) \
INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
RS1(base), __RS2(3))
When the assembler as insn_r I hope it will validate that
(imm & 0xfe0) == imm
> +
> #endif /* __ASM_INSN_DEF_H */
> diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> index de9da852f78d..7ad3a24212e8 100644
> --- a/arch/riscv/include/asm/processor.h
> +++ b/arch/riscv/include/asm/processor.h
> @@ -12,6 +12,8 @@
> #include <vdso/processor.h>
>
> #include <asm/ptrace.h>
> +#include <asm/insn-def.h>
> +#include <asm/hwcap.h>
>
> #ifdef CONFIG_64BIT
> #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
>
> +#define ARCH_HAS_PREFETCHW
> +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> + CBO_prefetchw(base), \
> + 0, \
> + RISCV_ISA_EXT_ZICBOP, \
> + CONFIG_RISCV_ISA_ZICBOP)
> +static inline void prefetchw(const void *ptr)
> +{
> + asm volatile(PREFETCHW_ASM(%0)
> + : : "r" (ptr) : "memory");
> +}
>
> /* Do necessary setup to start up a newly executed thread. */
> extern void start_thread(struct pt_regs *regs,
> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> index ef7b4fd9e876..e0b897db0b97 100644
> --- a/arch/riscv/kernel/cpufeature.c
> +++ b/arch/riscv/kernel/cpufeature.c
> @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
zicbop should be above zicboz (extensions alphabetical within their
category).
> __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> --
> 2.36.1
>
Thanks,
drew
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-14 13:47 ` Andrew Jones
0 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-14 13:47 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Cache-block prefetch instructions are HINTs to the hardware to
> indicate that software intends to perform a particular type of
> memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> improve the arch_xchg for qspinlock xchg_tail.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 15 +++++++++++++++
> arch/riscv/include/asm/cmpxchg.h | 4 +++-
> arch/riscv/include/asm/hwcap.h | 1 +
> arch/riscv/include/asm/insn-def.h | 5 +++++
> arch/riscv/include/asm/processor.h | 13 +++++++++++++
> arch/riscv/kernel/cpufeature.c | 1 +
> 6 files changed, 38 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index e9ae6fa232c3..2c346fe169c1 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
>
> If you don't know what to do here, say Y.
>
> +config RISCV_ISA_ZICBOP
> + bool "Zicbop extension support for cache block prefetch"
> + depends on MMU
> + depends on RISCV_ALTERNATIVE
> + default y
> + help
> + Adds support to dynamically detect the presence of the ZICBOP
> + extension (Cache Block Prefetch Operations) and enable its
> + usage.
> +
> + The Zicbop extension can be used to prefetch cache block for
> + read/write/instruction fetch.
> +
> + If you don't know what to do here, say Y.
> +
> config TOOLCHAIN_HAS_ZIHINTPAUSE
> bool
> default y
> diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> index 702725727671..56eff7a9d2d2 100644
> --- a/arch/riscv/include/asm/cmpxchg.h
> +++ b/arch/riscv/include/asm/cmpxchg.h
> @@ -11,6 +11,7 @@
>
> #include <asm/barrier.h>
> #include <asm/fence.h>
> +#include <asm/processor.h>
>
> #define __arch_xchg_masked(prepend, append, r, p, n) \
> ({ \
> @@ -25,6 +26,7 @@
> \
> __asm__ __volatile__ ( \
> prepend \
> + PREFETCHW_ASM(%5) \
> "0: lr.w %0, %2\n" \
> " and %1, %0, %z4\n" \
> " or %1, %1, %z3\n" \
> @@ -32,7 +34,7 @@
> " bnez %1, 0b\n" \
> append \
> : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> - : "rJ" (__newx), "rJ" (~__mask) \
> + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> : "memory"); \
> \
> r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> index b7b58258f6c7..78b7b8b53778 100644
> --- a/arch/riscv/include/asm/hwcap.h
> +++ b/arch/riscv/include/asm/hwcap.h
> @@ -58,6 +58,7 @@
> #define RISCV_ISA_EXT_ZICSR 40
> #define RISCV_ISA_EXT_ZIFENCEI 41
> #define RISCV_ISA_EXT_ZIHPM 42
> +#define RISCV_ISA_EXT_ZICBOP 43
>
> #define RISCV_ISA_EXT_MAX 64
>
> diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> index 6960beb75f32..dc590d331894 100644
> --- a/arch/riscv/include/asm/insn-def.h
> +++ b/arch/riscv/include/asm/insn-def.h
> @@ -134,6 +134,7 @@
>
> #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
This should be named RV_OPCODE_OP_IMM and be placed in
numerical order with the others, i.e. above SYSTEM.
>
> #define HFENCE_VVMA(vaddr, asid) \
> INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> @@ -196,4 +197,8 @@
> INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> RS1(base), SIMM12(4))
>
> +#define CBO_prefetchw(base) \
Please name this 'PREFETCH_w' and it should take an immediate parameter,
even if we intend to pass 0 for it.
> + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> + RD(x0), RS1(base), RS2(x0))
prefetch.w is not an R-type instruction, it's an S-type. While the bit
shifts are the same, the names are different. We need to add S-type
names while defining this instruction. Then, this define would be
#define PREFETCH_w(base, imm) \
INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
RS1(base), __RS2(3))
When the assembler as insn_r I hope it will validate that
(imm & 0xfe0) == imm
> +
> #endif /* __ASM_INSN_DEF_H */
> diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> index de9da852f78d..7ad3a24212e8 100644
> --- a/arch/riscv/include/asm/processor.h
> +++ b/arch/riscv/include/asm/processor.h
> @@ -12,6 +12,8 @@
> #include <vdso/processor.h>
>
> #include <asm/ptrace.h>
> +#include <asm/insn-def.h>
> +#include <asm/hwcap.h>
>
> #ifdef CONFIG_64BIT
> #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
>
> +#define ARCH_HAS_PREFETCHW
> +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> + CBO_prefetchw(base), \
> + 0, \
> + RISCV_ISA_EXT_ZICBOP, \
> + CONFIG_RISCV_ISA_ZICBOP)
> +static inline void prefetchw(const void *ptr)
> +{
> + asm volatile(PREFETCHW_ASM(%0)
> + : : "r" (ptr) : "memory");
> +}
>
> /* Do necessary setup to start up a newly executed thread. */
> extern void start_thread(struct pt_regs *regs,
> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> index ef7b4fd9e876..e0b897db0b97 100644
> --- a/arch/riscv/kernel/cpufeature.c
> +++ b/arch/riscv/kernel/cpufeature.c
> @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
zicbop should be above zicboz (extensions alphabetical within their
category).
> __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> --
> 2.36.1
>
Thanks,
drew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-10 8:28 ` guoren
@ 2023-09-14 14:25 ` Andrew Jones
-1 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-14 14:25 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Cache-block prefetch instructions are HINTs to the hardware to
> indicate that software intends to perform a particular type of
> memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> improve the arch_xchg for qspinlock xchg_tail.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 15 +++++++++++++++
> arch/riscv/include/asm/cmpxchg.h | 4 +++-
> arch/riscv/include/asm/hwcap.h | 1 +
> arch/riscv/include/asm/insn-def.h | 5 +++++
> arch/riscv/include/asm/processor.h | 13 +++++++++++++
> arch/riscv/kernel/cpufeature.c | 1 +
> 6 files changed, 38 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index e9ae6fa232c3..2c346fe169c1 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
>
> If you don't know what to do here, say Y.
>
> +config RISCV_ISA_ZICBOP
Even if we're not concerned with looping over blocks yet, I think we
should introduce zicbop block size DT parsing at the same time we bring
zicbop support to the kernel (it's just more copy+paste from zicbom and
zicboz). It's a bit annoying that the CMO spec doesn't state that block
sizes should be the same for m/z/p. And, the fact that m/z/p are all
separate extensions leads us to needing to parse block sizes for all
three, despite the fact that in practice they'll probably be the same.
Thanks,
drew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-14 14:25 ` Andrew Jones
0 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-14 14:25 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Cache-block prefetch instructions are HINTs to the hardware to
> indicate that software intends to perform a particular type of
> memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> improve the arch_xchg for qspinlock xchg_tail.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 15 +++++++++++++++
> arch/riscv/include/asm/cmpxchg.h | 4 +++-
> arch/riscv/include/asm/hwcap.h | 1 +
> arch/riscv/include/asm/insn-def.h | 5 +++++
> arch/riscv/include/asm/processor.h | 13 +++++++++++++
> arch/riscv/kernel/cpufeature.c | 1 +
> 6 files changed, 38 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index e9ae6fa232c3..2c346fe169c1 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
>
> If you don't know what to do here, say Y.
>
> +config RISCV_ISA_ZICBOP
Even if we're not concerned with looping over blocks yet, I think we
should introduce zicbop block size DT parsing at the same time we bring
zicbop support to the kernel (it's just more copy+paste from zicbom and
zicboz). It's a bit annoying that the CMO spec doesn't state that block
sizes should be the same for m/z/p. And, the fact that m/z/p are all
separate extensions leads us to needing to parse block sizes for all
three, despite the fact that in practice they'll probably be the same.
Thanks,
drew
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-14 14:25 ` Andrew Jones
@ 2023-09-14 14:47 ` Andrew Jones
-1 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-14 14:47 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 04:25:53PM +0200, Andrew Jones wrote:
> On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Cache-block prefetch instructions are HINTs to the hardware to
> > indicate that software intends to perform a particular type of
> > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > improve the arch_xchg for qspinlock xchg_tail.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig | 15 +++++++++++++++
> > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > arch/riscv/include/asm/hwcap.h | 1 +
> > arch/riscv/include/asm/insn-def.h | 5 +++++
> > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > arch/riscv/kernel/cpufeature.c | 1 +
> > 6 files changed, 38 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index e9ae6fa232c3..2c346fe169c1 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> >
> > If you don't know what to do here, say Y.
> >
> > +config RISCV_ISA_ZICBOP
>
> Even if we're not concerned with looping over blocks yet, I think we
> should introduce zicbop block size DT parsing at the same time we bring
> zicbop support to the kernel (it's just more copy+paste from zicbom and
> zicboz). It's a bit annoying that the CMO spec doesn't state that block
> sizes should be the same for m/z/p. And, the fact that m/z/p are all
> separate extensions leads us to needing to parse block sizes for all
> three, despite the fact that in practice they'll probably be the same.
Although, I saw on a different mailing list that Andrei Warkentin
interpreted section 2.7 "Software Discovery" of the spec, which states
"""
The initial set of CMO extensions requires the following information to be
discovered by software:
* The size of the cache block for management and prefetch instructions
* The size of the cache block for zero instructions
* CBIE support at each privilege level
Other general cache characteristics may also be specified in the discovery
mechanism.
"""
as management and prefetch having the same block size and only zero
potentially having a different size. That looks like a reasonable
interpretation to me, too. So, we could maybe proceed with assuming we
can use zicbom_block_size for prefetch, for now. If a platform comes along
that interpreted the spec differently, requiring prefetch block size to
be specified separately, then we'll cross that bridge when we get there.
Thanks,
drew
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-14 14:47 ` Andrew Jones
0 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-14 14:47 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 04:25:53PM +0200, Andrew Jones wrote:
> On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Cache-block prefetch instructions are HINTs to the hardware to
> > indicate that software intends to perform a particular type of
> > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > improve the arch_xchg for qspinlock xchg_tail.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig | 15 +++++++++++++++
> > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > arch/riscv/include/asm/hwcap.h | 1 +
> > arch/riscv/include/asm/insn-def.h | 5 +++++
> > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > arch/riscv/kernel/cpufeature.c | 1 +
> > 6 files changed, 38 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index e9ae6fa232c3..2c346fe169c1 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> >
> > If you don't know what to do here, say Y.
> >
> > +config RISCV_ISA_ZICBOP
>
> Even if we're not concerned with looping over blocks yet, I think we
> should introduce zicbop block size DT parsing at the same time we bring
> zicbop support to the kernel (it's just more copy+paste from zicbom and
> zicboz). It's a bit annoying that the CMO spec doesn't state that block
> sizes should be the same for m/z/p. And, the fact that m/z/p are all
> separate extensions leads us to needing to parse block sizes for all
> three, despite the fact that in practice they'll probably be the same.
Although, I saw on a different mailing list that Andrei Warkentin
interpreted section 2.7 "Software Discovery" of the spec, which states
"""
The initial set of CMO extensions requires the following information to be
discovered by software:
* The size of the cache block for management and prefetch instructions
* The size of the cache block for zero instructions
* CBIE support at each privilege level
Other general cache characteristics may also be specified in the discovery
mechanism.
"""
as management and prefetch having the same block size and only zero
potentially having a different size. That looks like a reasonable
interpretation to me, too. So, we could maybe proceed with assuming we
can use zicbom_block_size for prefetch, for now. If a platform comes along
that interpreted the spec differently, requiring prefetch block size to
be specified separately, then we'll cross that bridge when we get there.
Thanks,
drew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
2023-09-14 7:32 ` Leonardo Bras
(?)
@ 2023-09-14 17:23 ` Waiman Long
-1 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-14 17:23 UTC (permalink / raw)
To: Leonardo Bras, Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, linux-arch, linux-riscv, linux-doc,
kvm, virtualization, linux-csky, Guo Ren
On 9/14/23 03:32, Leonardo Bras wrote:
> On Tue, Sep 12, 2023 at 09:08:34AM +0800, Guo Ren wrote:
>> On Mon, Sep 11, 2023 at 11:34 PM Waiman Long <longman@redhat.com> wrote:
>>> On 9/10/23 04:29, guoren@kernel.org wrote:
>>>> From: Guo Ren <guoren@linux.alibaba.com>
>>>>
>>>> Allow cmdline to force the kernel to use queued_spinlock when
>>>> CONFIG_RISCV_COMBO_SPINLOCKS=y.
>>>>
>>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>>> ---
>>>> Documentation/admin-guide/kernel-parameters.txt | 2 ++
>>>> arch/riscv/kernel/setup.c | 16 +++++++++++++++-
>>>> 2 files changed, 17 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>>>> index 7dfb540c4f6c..61cacb8dfd0e 100644
>>>> --- a/Documentation/admin-guide/kernel-parameters.txt
>>>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>>>> @@ -4693,6 +4693,8 @@
>>>> [KNL] Number of legacy pty's. Overwrites compiled-in
>>>> default number.
>>>>
>>>> + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
>>>> +
>>>> qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
>>>> Set the time threshold in nanoseconds for the
>>>> number of intra-node lock hand-offs before the
>>>> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
>>>> index a447cf360a18..0f084f037651 100644
>>>> --- a/arch/riscv/kernel/setup.c
>>>> +++ b/arch/riscv/kernel/setup.c
>>>> @@ -270,6 +270,15 @@ static void __init parse_dtb(void)
>>>> }
>>>>
>>>> #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
>>>> +bool enable_qspinlock_key = false;
>>> You can use __ro_after_init qualifier for enable_qspinlock_key. BTW,
>>> this is not a static key, just a simple flag. So what is the point of
>>> the _key suffix?
>> Okay, I would change it to:
>> bool enable_qspinlock_flag __ro_after_init = false;
> IIUC, this bool / flag is used in a single file, so it makes sense for it
> to be static. Being static means it does not need to be initialized to
> false, as it's standard to zero-fill this areas.
>
> Also, since it's a bool, it does not need to be called _flag.
>
> I would go with:
>
> static bool enable_qspinlock __ro_after_init;
I actually was thinking about the same suggestion to add static. Then I
realized that the flag was also used in another file in a later patch.
Of course, if it turns out that this flag is no longer needed outside of
this file, it should be static.
Cheers,
Longman
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
@ 2023-09-14 17:23 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-14 17:23 UTC (permalink / raw)
To: Leonardo Bras, Guo Ren
Cc: Guo Ren, kvm, linux-doc, peterz, catalin.marinas, bjorn, palmer,
virtualization, conor.dooley, jszhang, linux-riscv, will,
keescook, linux-arch, anup, linux-csky, xiaoguang.xing, mingo,
greentime.hu, ajones, alexghiti, paulmck, boqun.feng, rostedt,
paul.walmsley, tglx, rdunlap, wuwei2016, wefu
On 9/14/23 03:32, Leonardo Bras wrote:
> On Tue, Sep 12, 2023 at 09:08:34AM +0800, Guo Ren wrote:
>> On Mon, Sep 11, 2023 at 11:34 PM Waiman Long <longman@redhat.com> wrote:
>>> On 9/10/23 04:29, guoren@kernel.org wrote:
>>>> From: Guo Ren <guoren@linux.alibaba.com>
>>>>
>>>> Allow cmdline to force the kernel to use queued_spinlock when
>>>> CONFIG_RISCV_COMBO_SPINLOCKS=y.
>>>>
>>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>>> ---
>>>> Documentation/admin-guide/kernel-parameters.txt | 2 ++
>>>> arch/riscv/kernel/setup.c | 16 +++++++++++++++-
>>>> 2 files changed, 17 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>>>> index 7dfb540c4f6c..61cacb8dfd0e 100644
>>>> --- a/Documentation/admin-guide/kernel-parameters.txt
>>>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>>>> @@ -4693,6 +4693,8 @@
>>>> [KNL] Number of legacy pty's. Overwrites compiled-in
>>>> default number.
>>>>
>>>> + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
>>>> +
>>>> qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
>>>> Set the time threshold in nanoseconds for the
>>>> number of intra-node lock hand-offs before the
>>>> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
>>>> index a447cf360a18..0f084f037651 100644
>>>> --- a/arch/riscv/kernel/setup.c
>>>> +++ b/arch/riscv/kernel/setup.c
>>>> @@ -270,6 +270,15 @@ static void __init parse_dtb(void)
>>>> }
>>>>
>>>> #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
>>>> +bool enable_qspinlock_key = false;
>>> You can use __ro_after_init qualifier for enable_qspinlock_key. BTW,
>>> this is not a static key, just a simple flag. So what is the point of
>>> the _key suffix?
>> Okay, I would change it to:
>> bool enable_qspinlock_flag __ro_after_init = false;
> IIUC, this bool / flag is used in a single file, so it makes sense for it
> to be static. Being static means it does not need to be initialized to
> false, as it's standard to zero-fill this areas.
>
> Also, since it's a bool, it does not need to be called _flag.
>
> I would go with:
>
> static bool enable_qspinlock __ro_after_init;
I actually was thinking about the same suggestion to add static. Then I
realized that the flag was also used in another file in a later patch.
Of course, if it turns out that this flag is no longer needed outside of
this file, it should be static.
Cheers,
Longman
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line
@ 2023-09-14 17:23 ` Waiman Long
0 siblings, 0 replies; 215+ messages in thread
From: Waiman Long @ 2023-09-14 17:23 UTC (permalink / raw)
To: Leonardo Bras, Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, conor.dooley,
xiaoguang.xing, bjorn, alexghiti, keescook, greentime.hu, ajones,
jszhang, wefu, wuwei2016, linux-arch, linux-riscv, linux-doc,
kvm, virtualization, linux-csky, Guo Ren
On 9/14/23 03:32, Leonardo Bras wrote:
> On Tue, Sep 12, 2023 at 09:08:34AM +0800, Guo Ren wrote:
>> On Mon, Sep 11, 2023 at 11:34 PM Waiman Long <longman@redhat.com> wrote:
>>> On 9/10/23 04:29, guoren@kernel.org wrote:
>>>> From: Guo Ren <guoren@linux.alibaba.com>
>>>>
>>>> Allow cmdline to force the kernel to use queued_spinlock when
>>>> CONFIG_RISCV_COMBO_SPINLOCKS=y.
>>>>
>>>> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
>>>> Signed-off-by: Guo Ren <guoren@kernel.org>
>>>> ---
>>>> Documentation/admin-guide/kernel-parameters.txt | 2 ++
>>>> arch/riscv/kernel/setup.c | 16 +++++++++++++++-
>>>> 2 files changed, 17 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>>>> index 7dfb540c4f6c..61cacb8dfd0e 100644
>>>> --- a/Documentation/admin-guide/kernel-parameters.txt
>>>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>>>> @@ -4693,6 +4693,8 @@
>>>> [KNL] Number of legacy pty's. Overwrites compiled-in
>>>> default number.
>>>>
>>>> + qspinlock [RISCV] Force to use qspinlock or auto-detect spinlock.
>>>> +
>>>> qspinlock.numa_spinlock_threshold_ns= [NUMA, PV_OPS]
>>>> Set the time threshold in nanoseconds for the
>>>> number of intra-node lock hand-offs before the
>>>> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
>>>> index a447cf360a18..0f084f037651 100644
>>>> --- a/arch/riscv/kernel/setup.c
>>>> +++ b/arch/riscv/kernel/setup.c
>>>> @@ -270,6 +270,15 @@ static void __init parse_dtb(void)
>>>> }
>>>>
>>>> #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
>>>> +bool enable_qspinlock_key = false;
>>> You can use __ro_after_init qualifier for enable_qspinlock_key. BTW,
>>> this is not a static key, just a simple flag. So what is the point of
>>> the _key suffix?
>> Okay, I would change it to:
>> bool enable_qspinlock_flag __ro_after_init = false;
> IIUC, this bool / flag is used in a single file, so it makes sense for it
> to be static. Being static means it does not need to be initialized to
> false, as it's standard to zero-fill this areas.
>
> Also, since it's a bool, it does not need to be called _flag.
>
> I would go with:
>
> static bool enable_qspinlock __ro_after_init;
I actually was thinking about the same suggestion to add static. Then I
realized that the flag was also used in another file in a later patch.
Of course, if it turns out that this flag is no longer needed outside of
this file, it should be static.
Cheers,
Longman
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
2023-09-14 9:43 ` Leonardo Bras
@ 2023-09-15 2:10 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-15 2:10 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 5:43 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote:
> > On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > The requirements of qspinlock have been documented by commit:
> > > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > > > atomics").
> > > >
> > > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > > > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > > > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > > > is the one.
> > > >
> > > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > > >
> > > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > > queued_spinlock 0.5109/0.00
> > > > ticket_spinlock 0.5814/0.00
> > > >
> > > > perf futex/hash (+6.7%):
> > > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > > >
> > > > perf futex/wake-parallel (+8.6%):
> > > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > > >
> > > > perf futex/requeue (+4.2%):
> > > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > > >
> > > > System Benchmarks (+6.4%)
> > > > queued_spinlock:
> > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > > Execl Throughput 43.0 13116.6 3050.4
> > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > > Process Creation 126.0 12537.3 995.0
> > > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > > System Call Overhead 15000.0 33308301.3 22205.5
> > > > ========
> > > > System Benchmarks Index Score 12426.1
> > > >
> > > > ticket_spinlock:
> > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > > Execl Throughput 43.0 12625.1 2936.1
> > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > > Process Creation 126.0 10798.4 857.0
> > > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > > System Call Overhead 15000.0 30766778.4 20511.2
> > > > ========
> > > > System Benchmarks Index Score 11670.7
> > > >
> > > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > > cores platform than the ticket_lock.
> > > >
> > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > ---
> > > > arch/riscv/Kconfig | 16 ++++++++++++++++
> > > > arch/riscv/include/asm/Kbuild | 3 ++-
> > > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > > > 3 files changed, 35 insertions(+), 1 deletion(-)
> > > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > > >
> > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > index 2c346fe169c1..7f39bfc75744 100644
> > > > --- a/arch/riscv/Kconfig
> > > > +++ b/arch/riscv/Kconfig
> > > > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > > > Specify the maximum number of NUMA Nodes available on the target
> > > > system. Increases memory reserved to accommodate various tables.
> > > >
> > > > +choice
> > > > + prompt "RISC-V spinlock type"
> > > > + default RISCV_TICKET_SPINLOCKS
> > > > +
> > > > +config RISCV_TICKET_SPINLOCKS
> > > > + bool "Using ticket spinlock"
> > > > +
> > > > +config RISCV_QUEUED_SPINLOCKS
> > > > + bool "Using queued spinlock"
> > > > + depends on SMP && MMU
> > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > + help
> > > > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > + Otherwise, stay at ticket-lock.
> > > > +endchoice
> > > > +
> > > > config RISCV_ALTERNATIVE
> > > > bool
> > > > depends on !XIP_KERNEL
> > > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > > index 504f8b7e72d4..a0dc85e4a754 100644
> > > > --- a/arch/riscv/include/asm/Kbuild
> > > > +++ b/arch/riscv/include/asm/Kbuild
> > > > @@ -2,10 +2,11 @@
> > > > generic-y += early_ioremap.h
> > > > generic-y += flat.h
> > > > generic-y += kvm_para.h
> > > > +generic-y += mcs_spinlock.h
> > > > generic-y += parport.h
> > > > -generic-y += spinlock.h
> > >
> > > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> > > and include the asm-generic headers of mcs_spinlock and qspinlock.
> > >
> > > In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> > > which seems the oposite of the above description (ticket spinlocks being
> > > the standard).
> > >
> > > Shouldn't ticket-spinlock.h also get included here?
> > > (Also, I am probably missing something, as I dont' see the use of
> > > mcs_spinlock here.)
> > No, because asm-generic/spinlock.h:
> > ...
> > #include <asm-generic/ticket_spinlock.h>
> > ...
> >
>
> But aren't you removing asm-generic/spinlock.h below ?
> -generic-y += spinlock.h
Yes, current is:
arch/riscv/include/asm/spinlock.h -> include/asm-generic/spinlock.h ->
include/asm-generic/ticket_spinlock.h
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm/qspinlock.h>
+#include <asm/qrwlock.h>
+#else
+#include <asm-generic/spinlock.h>
+#endif
So, you want me:
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm/qspinlock.h>
+#else
+#include <asm-generic/ticket_spinlock.h>
+#endif
+#include <asm/qrwlock.h>
Right?
>
> > >
> > > > generic-y += spinlock_types.h
> > > > generic-y += qrwlock.h
> > > > generic-y += qrwlock_types.h
> > > > +generic-y += qspinlock.h
> > > > generic-y += user.h
> > > > generic-y += vmlinux.lds.h
> > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > new file mode 100644
> > > > index 000000000000..c644a92d4548
> > > > --- /dev/null
> > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > @@ -0,0 +1,17 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > +
> > > > +#ifndef __ASM_RISCV_SPINLOCK_H
> > > > +#define __ASM_RISCV_SPINLOCK_H
> > > > +
> > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > +#define _Q_PENDING_LOOPS (1 << 9)
> > > > +#endif
> > >
> > > Any reason the above define couldn't be merged on the ifdef below?
> > Easy for the next patch to modify. See Waiman's comment:
> >
> > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> >
> > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > index c644a92d4548..9eb3ad31e564 100644
> > > --- a/arch/riscv/include/asm/spinlock.h
> > > +++ b/arch/riscv/include/asm/spinlock.h
> > > @@ -7,11 +7,94 @@
> > > #define _Q_PENDING_LOOPS (1 << 9)
> > > #endif
> > >
> >
> > I see why you separated the _Q_PENDING_LOOPS out.
> >
>
> I see, should be fine then.
>
> Thanks!
> Leo
>
> >
> > >
> > > > +
> > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > +#include <asm/qspinlock.h>
> > > > +#include <asm/qrwlock.h>
> > > > +#else
> > > > +#include <asm-generic/spinlock.h>
> > > > +#endif
> > > > +
> > > > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > > > --
> > > > 2.36.1
> > > >
> > >
> > > Thanks!
> > > Leo
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
@ 2023-09-15 2:10 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-15 2:10 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 5:43 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote:
> > On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > The requirements of qspinlock have been documented by commit:
> > > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > > > atomics").
> > > >
> > > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > > > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > > > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > > > is the one.
> > > >
> > > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > > >
> > > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > > queued_spinlock 0.5109/0.00
> > > > ticket_spinlock 0.5814/0.00
> > > >
> > > > perf futex/hash (+6.7%):
> > > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > > >
> > > > perf futex/wake-parallel (+8.6%):
> > > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > > >
> > > > perf futex/requeue (+4.2%):
> > > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > > >
> > > > System Benchmarks (+6.4%)
> > > > queued_spinlock:
> > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > > Execl Throughput 43.0 13116.6 3050.4
> > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > > Process Creation 126.0 12537.3 995.0
> > > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > > System Call Overhead 15000.0 33308301.3 22205.5
> > > > ========
> > > > System Benchmarks Index Score 12426.1
> > > >
> > > > ticket_spinlock:
> > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > > Execl Throughput 43.0 12625.1 2936.1
> > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > > Process Creation 126.0 10798.4 857.0
> > > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > > System Call Overhead 15000.0 30766778.4 20511.2
> > > > ========
> > > > System Benchmarks Index Score 11670.7
> > > >
> > > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > > cores platform than the ticket_lock.
> > > >
> > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > ---
> > > > arch/riscv/Kconfig | 16 ++++++++++++++++
> > > > arch/riscv/include/asm/Kbuild | 3 ++-
> > > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > > > 3 files changed, 35 insertions(+), 1 deletion(-)
> > > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > > >
> > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > index 2c346fe169c1..7f39bfc75744 100644
> > > > --- a/arch/riscv/Kconfig
> > > > +++ b/arch/riscv/Kconfig
> > > > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > > > Specify the maximum number of NUMA Nodes available on the target
> > > > system. Increases memory reserved to accommodate various tables.
> > > >
> > > > +choice
> > > > + prompt "RISC-V spinlock type"
> > > > + default RISCV_TICKET_SPINLOCKS
> > > > +
> > > > +config RISCV_TICKET_SPINLOCKS
> > > > + bool "Using ticket spinlock"
> > > > +
> > > > +config RISCV_QUEUED_SPINLOCKS
> > > > + bool "Using queued spinlock"
> > > > + depends on SMP && MMU
> > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > + help
> > > > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > + Otherwise, stay at ticket-lock.
> > > > +endchoice
> > > > +
> > > > config RISCV_ALTERNATIVE
> > > > bool
> > > > depends on !XIP_KERNEL
> > > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > > index 504f8b7e72d4..a0dc85e4a754 100644
> > > > --- a/arch/riscv/include/asm/Kbuild
> > > > +++ b/arch/riscv/include/asm/Kbuild
> > > > @@ -2,10 +2,11 @@
> > > > generic-y += early_ioremap.h
> > > > generic-y += flat.h
> > > > generic-y += kvm_para.h
> > > > +generic-y += mcs_spinlock.h
> > > > generic-y += parport.h
> > > > -generic-y += spinlock.h
> > >
> > > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> > > and include the asm-generic headers of mcs_spinlock and qspinlock.
> > >
> > > In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> > > which seems the oposite of the above description (ticket spinlocks being
> > > the standard).
> > >
> > > Shouldn't ticket-spinlock.h also get included here?
> > > (Also, I am probably missing something, as I dont' see the use of
> > > mcs_spinlock here.)
> > No, because asm-generic/spinlock.h:
> > ...
> > #include <asm-generic/ticket_spinlock.h>
> > ...
> >
>
> But aren't you removing asm-generic/spinlock.h below ?
> -generic-y += spinlock.h
Yes, current is:
arch/riscv/include/asm/spinlock.h -> include/asm-generic/spinlock.h ->
include/asm-generic/ticket_spinlock.h
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm/qspinlock.h>
+#include <asm/qrwlock.h>
+#else
+#include <asm-generic/spinlock.h>
+#endif
So, you want me:
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm/qspinlock.h>
+#else
+#include <asm-generic/ticket_spinlock.h>
+#endif
+#include <asm/qrwlock.h>
Right?
>
> > >
> > > > generic-y += spinlock_types.h
> > > > generic-y += qrwlock.h
> > > > generic-y += qrwlock_types.h
> > > > +generic-y += qspinlock.h
> > > > generic-y += user.h
> > > > generic-y += vmlinux.lds.h
> > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > new file mode 100644
> > > > index 000000000000..c644a92d4548
> > > > --- /dev/null
> > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > @@ -0,0 +1,17 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > +
> > > > +#ifndef __ASM_RISCV_SPINLOCK_H
> > > > +#define __ASM_RISCV_SPINLOCK_H
> > > > +
> > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > +#define _Q_PENDING_LOOPS (1 << 9)
> > > > +#endif
> > >
> > > Any reason the above define couldn't be merged on the ifdef below?
> > Easy for the next patch to modify. See Waiman's comment:
> >
> > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> >
> > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > index c644a92d4548..9eb3ad31e564 100644
> > > --- a/arch/riscv/include/asm/spinlock.h
> > > +++ b/arch/riscv/include/asm/spinlock.h
> > > @@ -7,11 +7,94 @@
> > > #define _Q_PENDING_LOOPS (1 << 9)
> > > #endif
> > >
> >
> > I see why you separated the _Q_PENDING_LOOPS out.
> >
>
> I see, should be fine then.
>
> Thanks!
> Leo
>
> >
> > >
> > > > +
> > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > +#include <asm/qspinlock.h>
> > > > +#include <asm/qrwlock.h>
> > > > +#else
> > > > +#include <asm-generic/spinlock.h>
> > > > +#endif
> > > > +
> > > > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > > > --
> > > > 2.36.1
> > > >
> > >
> > > Thanks!
> > > Leo
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 11/17] RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
2023-09-10 8:29 ` guoren
@ 2023-09-15 5:42 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 5:42 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:05AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Using static_call to switch between:
> native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath()
> native_queued_spin_unlock() __pv_queued_spin_unlock()
>
> Finish the pv_wait implementation, but pv_kick needs the SBI
> definition of the next patches.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/include/asm/Kbuild | 1 -
> arch/riscv/include/asm/qspinlock.h | 35 +++++++++++++
> arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++++++++
> arch/riscv/include/asm/spinlock.h | 2 +-
> arch/riscv/kernel/qspinlock_paravirt.c | 57 +++++++++++++++++++++
> arch/riscv/kernel/setup.c | 4 ++
> 6 files changed, 126 insertions(+), 2 deletions(-)
> create mode 100644 arch/riscv/include/asm/qspinlock.h
> create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
>
> diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> index a0dc85e4a754..b89cb3b73c13 100644
> --- a/arch/riscv/include/asm/Kbuild
> +++ b/arch/riscv/include/asm/Kbuild
> @@ -7,6 +7,5 @@ generic-y += parport.h
> generic-y += spinlock_types.h
> generic-y += qrwlock.h
> generic-y += qrwlock_types.h
> -generic-y += qspinlock.h
> generic-y += user.h
> generic-y += vmlinux.lds.h
> diff --git a/arch/riscv/include/asm/qspinlock.h b/arch/riscv/include/asm/qspinlock.h
> new file mode 100644
> index 000000000000..7d4f416c908c
> --- /dev/null
> +++ b/arch/riscv/include/asm/qspinlock.h
> @@ -0,0 +1,35 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c), 2023 Alibaba Cloud
> + * Authors:
> + * Guo Ren <guoren@linux.alibaba.com>
> + */
> +
> +#ifndef _ASM_RISCV_QSPINLOCK_H
> +#define _ASM_RISCV_QSPINLOCK_H
> +
> +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> +#include <asm/qspinlock_paravirt.h>
> +
> +/* How long a lock should spin before we consider blocking */
> +#define SPIN_THRESHOLD (1 << 15)
> +
> +void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> +void __pv_init_lock_hash(void);
> +void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> +
> +static inline void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
> +{
> + static_call(pv_queued_spin_lock_slowpath)(lock, val);
> +}
> +
> +#define queued_spin_unlock queued_spin_unlock
> +static inline void queued_spin_unlock(struct qspinlock *lock)
> +{
> + static_call(pv_queued_spin_unlock)(lock);
> +}
> +#endif /* CONFIG_PARAVIRT_SPINLOCKS */
> +
> +#include <asm-generic/qspinlock.h>
> +
> +#endif /* _ASM_RISCV_QSPINLOCK_H */
> diff --git a/arch/riscv/include/asm/qspinlock_paravirt.h b/arch/riscv/include/asm/qspinlock_paravirt.h
> new file mode 100644
> index 000000000000..9681e851f69d
> --- /dev/null
> +++ b/arch/riscv/include/asm/qspinlock_paravirt.h
> @@ -0,0 +1,29 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c), 2023 Alibaba Cloud
> + * Authors:
> + * Guo Ren <guoren@linux.alibaba.com>
> + */
> +
> +#ifndef _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> +#define _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> +
> +void pv_wait(u8 *ptr, u8 val);
> +void pv_kick(int cpu);
> +
> +void dummy_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> +void dummy_queued_spin_unlock(struct qspinlock *lock);
> +
> +DECLARE_STATIC_CALL(pv_queued_spin_lock_slowpath, dummy_queued_spin_lock_slowpath);
> +DECLARE_STATIC_CALL(pv_queued_spin_unlock, dummy_queued_spin_unlock);
> +
> +void __init pv_qspinlock_init(void);
> +
> +static inline bool pv_is_native_spin_unlock(void)
> +{
> + return false;
> +}
> +
> +void __pv_queued_spin_unlock(struct qspinlock *lock);
> +
> +#endif /* _ASM_RISCV_QSPINLOCK_PARAVIRT_H */
> diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> index 6b38d6616f14..ed4253f491fe 100644
> --- a/arch/riscv/include/asm/spinlock.h
> +++ b/arch/riscv/include/asm/spinlock.h
> @@ -39,7 +39,7 @@ static inline bool virt_spin_lock(struct qspinlock *lock)
> #undef arch_spin_trylock
> #undef arch_spin_unlock
>
> -#include <asm-generic/qspinlock.h>
> +#include <asm/qspinlock.h>
> #include <linux/jump_label.h>
>
> #undef arch_spin_is_locked
> diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> new file mode 100644
> index 000000000000..85ff5a3ec234
> --- /dev/null
> +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> @@ -0,0 +1,57 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c), 2023 Alibaba Cloud
> + * Authors:
> + * Guo Ren <guoren@linux.alibaba.com>
> + */
> +
> +#include <linux/static_call.h>
> +#include <asm/qspinlock_paravirt.h>
> +#include <asm/sbi.h>
> +
> +void pv_kick(int cpu)
> +{
> + return;
> +}
> +
> +void pv_wait(u8 *ptr, u8 val)
> +{
> + unsigned long flags;
> +
> + if (in_nmi())
> + return;
> +
> + local_irq_save(flags);
> + if (READ_ONCE(*ptr) != val)
> + goto out;
> +
> + /* wait_for_interrupt(); */
> +out:
> + local_irq_restore(flags);
> +}
> +
> +static void native_queued_spin_unlock(struct qspinlock *lock)
> +{
> + smp_store_release(&lock->locked, 0);
> +}
> +
> +DEFINE_STATIC_CALL(pv_queued_spin_lock_slowpath, native_queued_spin_lock_slowpath);
> +EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> +
> +DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> +EXPORT_STATIC_CALL(pv_queued_spin_unlock);
> +
> +void __init pv_qspinlock_init(void)
> +{
> + if (num_possible_cpus() == 1)
> + return;
> +
> + if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
Checks like this seem to be very common on this patchset.
For someone not much familiar with this, it can be hard to
understand.
I mean, on patch 8/17 you introduce those IDs, which look to be
incremental ( ID == N includes stuff from ID < N ), but I am not sure as I
couln't find much documentation on that.
Then above you test for the id being different than
SBI_EXT_BASE_IMPL_ID_KVM, but if they are actually incremental and a new
version lands, the new version will also return early because it passes the
test.
I am no sure if above is right, but it's all I could understand without
documentation.
Well, my point is: this seems hard to understand & review, so it would be
nice to have a macro like this to be used instead:
#define sbi_fw_implements_kvm() \
(sbi_get_firmware_id() >= SBI_EXT_BASE_IMPL_ID_KVM)
if(!sbi_fw_implements_kvm())
return;
What do you think?
Other than that, LGTM.
Thanks!
Leo
> + return;
> +
> + pr_info("PV qspinlocks enabled\n");
> + __pv_init_lock_hash();
> +
> + static_call_update(pv_queued_spin_lock_slowpath, __pv_queued_spin_lock_slowpath);
> + static_call_update(pv_queued_spin_unlock, __pv_queued_spin_unlock);
> +}
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index c57d15b05160..88690751f2ee 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -321,6 +321,10 @@ static void __init riscv_spinlock_init(void)
> #ifdef CONFIG_QUEUED_SPINLOCKS
> virt_spin_lock_init();
> #endif
> +
> +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> + pv_qspinlock_init();
> +#endif
> }
>
> extern void __init init_rt_signal_env(void);
> --
> 2.36.1
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 11/17] RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
@ 2023-09-15 5:42 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 5:42 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:05AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Using static_call to switch between:
> native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath()
> native_queued_spin_unlock() __pv_queued_spin_unlock()
>
> Finish the pv_wait implementation, but pv_kick needs the SBI
> definition of the next patches.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/include/asm/Kbuild | 1 -
> arch/riscv/include/asm/qspinlock.h | 35 +++++++++++++
> arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++++++++
> arch/riscv/include/asm/spinlock.h | 2 +-
> arch/riscv/kernel/qspinlock_paravirt.c | 57 +++++++++++++++++++++
> arch/riscv/kernel/setup.c | 4 ++
> 6 files changed, 126 insertions(+), 2 deletions(-)
> create mode 100644 arch/riscv/include/asm/qspinlock.h
> create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
>
> diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> index a0dc85e4a754..b89cb3b73c13 100644
> --- a/arch/riscv/include/asm/Kbuild
> +++ b/arch/riscv/include/asm/Kbuild
> @@ -7,6 +7,5 @@ generic-y += parport.h
> generic-y += spinlock_types.h
> generic-y += qrwlock.h
> generic-y += qrwlock_types.h
> -generic-y += qspinlock.h
> generic-y += user.h
> generic-y += vmlinux.lds.h
> diff --git a/arch/riscv/include/asm/qspinlock.h b/arch/riscv/include/asm/qspinlock.h
> new file mode 100644
> index 000000000000..7d4f416c908c
> --- /dev/null
> +++ b/arch/riscv/include/asm/qspinlock.h
> @@ -0,0 +1,35 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c), 2023 Alibaba Cloud
> + * Authors:
> + * Guo Ren <guoren@linux.alibaba.com>
> + */
> +
> +#ifndef _ASM_RISCV_QSPINLOCK_H
> +#define _ASM_RISCV_QSPINLOCK_H
> +
> +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> +#include <asm/qspinlock_paravirt.h>
> +
> +/* How long a lock should spin before we consider blocking */
> +#define SPIN_THRESHOLD (1 << 15)
> +
> +void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> +void __pv_init_lock_hash(void);
> +void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> +
> +static inline void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
> +{
> + static_call(pv_queued_spin_lock_slowpath)(lock, val);
> +}
> +
> +#define queued_spin_unlock queued_spin_unlock
> +static inline void queued_spin_unlock(struct qspinlock *lock)
> +{
> + static_call(pv_queued_spin_unlock)(lock);
> +}
> +#endif /* CONFIG_PARAVIRT_SPINLOCKS */
> +
> +#include <asm-generic/qspinlock.h>
> +
> +#endif /* _ASM_RISCV_QSPINLOCK_H */
> diff --git a/arch/riscv/include/asm/qspinlock_paravirt.h b/arch/riscv/include/asm/qspinlock_paravirt.h
> new file mode 100644
> index 000000000000..9681e851f69d
> --- /dev/null
> +++ b/arch/riscv/include/asm/qspinlock_paravirt.h
> @@ -0,0 +1,29 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c), 2023 Alibaba Cloud
> + * Authors:
> + * Guo Ren <guoren@linux.alibaba.com>
> + */
> +
> +#ifndef _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> +#define _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> +
> +void pv_wait(u8 *ptr, u8 val);
> +void pv_kick(int cpu);
> +
> +void dummy_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> +void dummy_queued_spin_unlock(struct qspinlock *lock);
> +
> +DECLARE_STATIC_CALL(pv_queued_spin_lock_slowpath, dummy_queued_spin_lock_slowpath);
> +DECLARE_STATIC_CALL(pv_queued_spin_unlock, dummy_queued_spin_unlock);
> +
> +void __init pv_qspinlock_init(void);
> +
> +static inline bool pv_is_native_spin_unlock(void)
> +{
> + return false;
> +}
> +
> +void __pv_queued_spin_unlock(struct qspinlock *lock);
> +
> +#endif /* _ASM_RISCV_QSPINLOCK_PARAVIRT_H */
> diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> index 6b38d6616f14..ed4253f491fe 100644
> --- a/arch/riscv/include/asm/spinlock.h
> +++ b/arch/riscv/include/asm/spinlock.h
> @@ -39,7 +39,7 @@ static inline bool virt_spin_lock(struct qspinlock *lock)
> #undef arch_spin_trylock
> #undef arch_spin_unlock
>
> -#include <asm-generic/qspinlock.h>
> +#include <asm/qspinlock.h>
> #include <linux/jump_label.h>
>
> #undef arch_spin_is_locked
> diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> new file mode 100644
> index 000000000000..85ff5a3ec234
> --- /dev/null
> +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> @@ -0,0 +1,57 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c), 2023 Alibaba Cloud
> + * Authors:
> + * Guo Ren <guoren@linux.alibaba.com>
> + */
> +
> +#include <linux/static_call.h>
> +#include <asm/qspinlock_paravirt.h>
> +#include <asm/sbi.h>
> +
> +void pv_kick(int cpu)
> +{
> + return;
> +}
> +
> +void pv_wait(u8 *ptr, u8 val)
> +{
> + unsigned long flags;
> +
> + if (in_nmi())
> + return;
> +
> + local_irq_save(flags);
> + if (READ_ONCE(*ptr) != val)
> + goto out;
> +
> + /* wait_for_interrupt(); */
> +out:
> + local_irq_restore(flags);
> +}
> +
> +static void native_queued_spin_unlock(struct qspinlock *lock)
> +{
> + smp_store_release(&lock->locked, 0);
> +}
> +
> +DEFINE_STATIC_CALL(pv_queued_spin_lock_slowpath, native_queued_spin_lock_slowpath);
> +EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> +
> +DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> +EXPORT_STATIC_CALL(pv_queued_spin_unlock);
> +
> +void __init pv_qspinlock_init(void)
> +{
> + if (num_possible_cpus() == 1)
> + return;
> +
> + if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
Checks like this seem to be very common on this patchset.
For someone not much familiar with this, it can be hard to
understand.
I mean, on patch 8/17 you introduce those IDs, which look to be
incremental ( ID == N includes stuff from ID < N ), but I am not sure as I
couln't find much documentation on that.
Then above you test for the id being different than
SBI_EXT_BASE_IMPL_ID_KVM, but if they are actually incremental and a new
version lands, the new version will also return early because it passes the
test.
I am no sure if above is right, but it's all I could understand without
documentation.
Well, my point is: this seems hard to understand & review, so it would be
nice to have a macro like this to be used instead:
#define sbi_fw_implements_kvm() \
(sbi_get_firmware_id() >= SBI_EXT_BASE_IMPL_ID_KVM)
if(!sbi_fw_implements_kvm())
return;
What do you think?
Other than that, LGTM.
Thanks!
Leo
> + return;
> +
> + pr_info("PV qspinlocks enabled\n");
> + __pv_init_lock_hash();
> +
> + static_call_update(pv_queued_spin_lock_slowpath, __pv_queued_spin_lock_slowpath);
> + static_call_update(pv_queued_spin_unlock, __pv_queued_spin_unlock);
> +}
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index c57d15b05160..88690751f2ee 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -321,6 +321,10 @@ static void __init riscv_spinlock_init(void)
> #ifdef CONFIG_QUEUED_SPINLOCKS
> virt_spin_lock_init();
> #endif
> +
> +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> + pv_qspinlock_init();
> +#endif
> }
>
> extern void __init init_rt_signal_env(void);
> --
> 2.36.1
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 12/17] RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
2023-09-10 8:29 ` guoren
@ 2023-09-15 6:05 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:05 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:06AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Disables the qspinlock slow path using PV optimizations which
> allow the hypervisor to 'idle' the guest on lock contention.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 2 +-
> arch/riscv/kernel/qspinlock_paravirt.c | 13 +++++++++++++
> 2 files changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index f75bedc50e00..e74aed631573 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -3857,7 +3857,7 @@
> as generic guest with no PV drivers. Currently support
> XEN HVM, KVM, HYPER_V and VMWARE guest.
>
> - nopvspin [X86,XEN,KVM]
> + nopvspin [X86,XEN,KVM,RISC-V]
> Disables the qspinlock slow path using PV optimizations
> which allow the hypervisor to 'idle' the guest on lock
> contention.
> diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> index 85ff5a3ec234..a0ad4657f437 100644
> --- a/arch/riscv/kernel/qspinlock_paravirt.c
> +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> @@ -41,8 +41,21 @@ EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> EXPORT_STATIC_CALL(pv_queued_spin_unlock);
>
> +static bool nopvspin;
It is only used in init, so it makes sense to add __initdata.
static bool nopvspin __initdata;
Other than that, LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
> +static __init int parse_nopvspin(char *arg)
> +{
> + nopvspin = true;
> + return 0;
> +}
> +early_param("nopvspin", parse_nopvspin);
> +
> void __init pv_qspinlock_init(void)
> {
> + if (nopvspin) {
> + pr_info("PV qspinlocks disabled\n");
> + return;
> + }
> +
> if (num_possible_cpus() == 1)
> return;
>
> --
> 2.36.1
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 12/17] RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
@ 2023-09-15 6:05 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:05 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:06AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Disables the qspinlock slow path using PV optimizations which
> allow the hypervisor to 'idle' the guest on lock contention.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 2 +-
> arch/riscv/kernel/qspinlock_paravirt.c | 13 +++++++++++++
> 2 files changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index f75bedc50e00..e74aed631573 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -3857,7 +3857,7 @@
> as generic guest with no PV drivers. Currently support
> XEN HVM, KVM, HYPER_V and VMWARE guest.
>
> - nopvspin [X86,XEN,KVM]
> + nopvspin [X86,XEN,KVM,RISC-V]
> Disables the qspinlock slow path using PV optimizations
> which allow the hypervisor to 'idle' the guest on lock
> contention.
> diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> index 85ff5a3ec234..a0ad4657f437 100644
> --- a/arch/riscv/kernel/qspinlock_paravirt.c
> +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> @@ -41,8 +41,21 @@ EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> EXPORT_STATIC_CALL(pv_queued_spin_unlock);
>
> +static bool nopvspin;
It is only used in init, so it makes sense to add __initdata.
static bool nopvspin __initdata;
Other than that, LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
> +static __init int parse_nopvspin(char *arg)
> +{
> + nopvspin = true;
> + return 0;
> +}
> +early_param("nopvspin", parse_nopvspin);
> +
> void __init pv_qspinlock_init(void)
> {
> + if (nopvspin) {
> + pr_info("PV qspinlocks disabled\n");
> + return;
> + }
> +
> if (num_possible_cpus() == 1)
> return;
>
> --
> 2.36.1
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 13/17] RISC-V: paravirt: pvqspinlock: Add SBI implementation
2023-09-10 8:29 ` guoren
@ 2023-09-15 6:23 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:23 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:07AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Implement pv_kick with SBI implementation, and add SBI_EXT_PVLOCK
> extension detection.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/include/asm/sbi.h | 6 ++++++
> arch/riscv/kernel/qspinlock_paravirt.c | 7 ++++++-
> 2 files changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index e0233b3d7a5f..3533f8d4f3e2 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -30,6 +30,7 @@ enum sbi_ext_id {
> SBI_EXT_HSM = 0x48534D,
> SBI_EXT_SRST = 0x53525354,
> SBI_EXT_PMU = 0x504D55,
> + SBI_EXT_PVLOCK = 0xAB0401,
>
> /* Experimentals extensions must lie within this range */
> SBI_EXT_EXPERIMENTAL_START = 0x08000000,
> @@ -243,6 +244,11 @@ enum sbi_pmu_ctr_type {
> /* Flags defined for counter stop function */
> #define SBI_PMU_STOP_FLAG_RESET (1 << 0)
>
> +/* SBI PVLOCK (kick cpu out of wfi) */
> +enum sbi_ext_pvlock_fid {
> + SBI_EXT_PVLOCK_KICK_CPU = 0,
> +};
> +
> #define SBI_SPEC_VERSION_DEFAULT 0x1
> #define SBI_SPEC_VERSION_MAJOR_SHIFT 24
> #define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
> diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> index a0ad4657f437..571626f350be 100644
> --- a/arch/riscv/kernel/qspinlock_paravirt.c
> +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> @@ -11,6 +11,8 @@
>
> void pv_kick(int cpu)
> {
> + sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
> + cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
> return;
> }
>
> @@ -25,7 +27,7 @@ void pv_wait(u8 *ptr, u8 val)
> if (READ_ONCE(*ptr) != val)
> goto out;
>
> - /* wait_for_interrupt(); */
> + wait_for_interrupt();
> out:
> local_irq_restore(flags);
> }
> @@ -62,6 +64,9 @@ void __init pv_qspinlock_init(void)
> if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
> return;
>
> + if (!sbi_probe_extension(SBI_EXT_PVLOCK))
> + return;
> +
> pr_info("PV qspinlocks enabled\n");
> __pv_init_lock_hash();
>
> --
> 2.36.1
>
IIUC this PVLOCK extension is now a requirement to use pv_qspinlock(), and
it allows a cpu to use an instruction to wait for interrupt in pv_wait(),
and kicks it out of this wait using a new sbi_ecall() on pv_kick().
Overall it LGTM, but would be nice to have the reference doc in the commit
msg. I end up inferring some of the inner workings by your implementation,
which is not ideal for reviewing.
If understanding above is right,
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 13/17] RISC-V: paravirt: pvqspinlock: Add SBI implementation
@ 2023-09-15 6:23 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:23 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:07AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Implement pv_kick with SBI implementation, and add SBI_EXT_PVLOCK
> extension detection.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/include/asm/sbi.h | 6 ++++++
> arch/riscv/kernel/qspinlock_paravirt.c | 7 ++++++-
> 2 files changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index e0233b3d7a5f..3533f8d4f3e2 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -30,6 +30,7 @@ enum sbi_ext_id {
> SBI_EXT_HSM = 0x48534D,
> SBI_EXT_SRST = 0x53525354,
> SBI_EXT_PMU = 0x504D55,
> + SBI_EXT_PVLOCK = 0xAB0401,
>
> /* Experimentals extensions must lie within this range */
> SBI_EXT_EXPERIMENTAL_START = 0x08000000,
> @@ -243,6 +244,11 @@ enum sbi_pmu_ctr_type {
> /* Flags defined for counter stop function */
> #define SBI_PMU_STOP_FLAG_RESET (1 << 0)
>
> +/* SBI PVLOCK (kick cpu out of wfi) */
> +enum sbi_ext_pvlock_fid {
> + SBI_EXT_PVLOCK_KICK_CPU = 0,
> +};
> +
> #define SBI_SPEC_VERSION_DEFAULT 0x1
> #define SBI_SPEC_VERSION_MAJOR_SHIFT 24
> #define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
> diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> index a0ad4657f437..571626f350be 100644
> --- a/arch/riscv/kernel/qspinlock_paravirt.c
> +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> @@ -11,6 +11,8 @@
>
> void pv_kick(int cpu)
> {
> + sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
> + cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
> return;
> }
>
> @@ -25,7 +27,7 @@ void pv_wait(u8 *ptr, u8 val)
> if (READ_ONCE(*ptr) != val)
> goto out;
>
> - /* wait_for_interrupt(); */
> + wait_for_interrupt();
> out:
> local_irq_restore(flags);
> }
> @@ -62,6 +64,9 @@ void __init pv_qspinlock_init(void)
> if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
> return;
>
> + if (!sbi_probe_extension(SBI_EXT_PVLOCK))
> + return;
> +
> pr_info("PV qspinlocks enabled\n");
> __pv_init_lock_hash();
>
> --
> 2.36.1
>
IIUC this PVLOCK extension is now a requirement to use pv_qspinlock(), and
it allows a cpu to use an instruction to wait for interrupt in pv_wait(),
and kicks it out of this wait using a new sbi_ecall() on pv_kick().
Overall it LGTM, but would be nice to have the reference doc in the commit
msg. I end up inferring some of the inner workings by your implementation,
which is not ideal for reviewing.
If understanding above is right,
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 14/17] RISC-V: paravirt: pvqspinlock: Add kconfig entry
2023-09-10 8:29 ` guoren
@ 2023-09-15 6:25 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:25 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:08AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Add kconfig entry for paravirt_spinlock, an unfair qspinlock
> virtualization-friendly backend, by halting the virtual CPU rather
> than spinning.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 12 ++++++++++++
> arch/riscv/kernel/Makefile | 1 +
> 2 files changed, 13 insertions(+)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 4bcff2860f48..ec0da24ed6fb 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -813,6 +813,18 @@ config RELOCATABLE
>
> If unsure, say N.
>
> +config PARAVIRT_SPINLOCKS
> + bool "Paravirtualization layer for spinlocks"
> + depends on QUEUED_SPINLOCKS
> + default y
> + help
> + Paravirtualized spinlocks allow a unfair qspinlock to replace the
> + test-set kvm-guest virt spinlock implementation with something
> + virtualization-friendly, for example, halt the virtual CPU rather
> + than spinning.
> +
> + If you are unsure how to answer this question, answer Y.
> +
> endmenu # "Kernel features"
>
> menu "Boot options"
> diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
> index 671ad85f28f2..114b29234c46 100644
> --- a/arch/riscv/kernel/Makefile
> +++ b/arch/riscv/kernel/Makefile
> @@ -103,3 +103,4 @@ obj-$(CONFIG_ARCH_RV64ILP32) += compat_signal.o
>
> obj-$(CONFIG_64BIT) += pi/
> obj-$(CONFIG_ACPI) += acpi.o
> +obj-$(CONFIG_PARAVIRT_SPINLOCKS) += qspinlock_paravirt.o
> --
> 2.36.1
>
LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 14/17] RISC-V: paravirt: pvqspinlock: Add kconfig entry
@ 2023-09-15 6:25 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:25 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:08AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Add kconfig entry for paravirt_spinlock, an unfair qspinlock
> virtualization-friendly backend, by halting the virtual CPU rather
> than spinning.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/Kconfig | 12 ++++++++++++
> arch/riscv/kernel/Makefile | 1 +
> 2 files changed, 13 insertions(+)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 4bcff2860f48..ec0da24ed6fb 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -813,6 +813,18 @@ config RELOCATABLE
>
> If unsure, say N.
>
> +config PARAVIRT_SPINLOCKS
> + bool "Paravirtualization layer for spinlocks"
> + depends on QUEUED_SPINLOCKS
> + default y
> + help
> + Paravirtualized spinlocks allow a unfair qspinlock to replace the
> + test-set kvm-guest virt spinlock implementation with something
> + virtualization-friendly, for example, halt the virtual CPU rather
> + than spinning.
> +
> + If you are unsure how to answer this question, answer Y.
> +
> endmenu # "Kernel features"
>
> menu "Boot options"
> diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
> index 671ad85f28f2..114b29234c46 100644
> --- a/arch/riscv/kernel/Makefile
> +++ b/arch/riscv/kernel/Makefile
> @@ -103,3 +103,4 @@ obj-$(CONFIG_ARCH_RV64ILP32) += compat_signal.o
>
> obj-$(CONFIG_64BIT) += pi/
> obj-$(CONFIG_ACPI) += acpi.o
> +obj-$(CONFIG_PARAVIRT_SPINLOCKS) += qspinlock_paravirt.o
> --
> 2.36.1
>
LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 15/17] RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
2023-09-10 8:29 ` guoren
@ 2023-09-15 6:33 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:33 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:09AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Add trace point for pv_kick/wait, here is the output:
>
> entries-in-buffer/entries-written: 33927/33927 #P:12
>
> _-----=> irqs-off/BH-disabled
> / _----=> need-resched
> | / _---=> hardirq/softirq
> || / _--=> preempt-depth
> ||| / _-=> migrate-disable
> |||| / delay
> TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> | | | ||||| | |
> sh-100 [001] d..2. 28.312294: pv_wait: cpu 1 out of wfi
> <idle>-0 [000] d.h4. 28.322030: pv_kick: cpu 0 kick target cpu 1
> sh-100 [001] d..2. 30.982631: pv_wait: cpu 1 out of wfi
> <idle>-0 [000] d.h4. 30.993289: pv_kick: cpu 0 kick target cpu 1
> sh-100 [002] d..2. 44.987573: pv_wait: cpu 2 out of wfi
> <idle>-0 [000] d.h4. 44.989000: pv_kick: cpu 0 kick target cpu 2
> <idle>-0 [003] d.s3. 51.593978: pv_kick: cpu 3 kick target cpu 4
> rcu_sched-15 [004] d..2. 51.595192: pv_wait: cpu 4 out of wfi
> lock_torture_wr-115 [004] ...2. 52.656482: pv_kick: cpu 4 kick target cpu 2
> lock_torture_wr-113 [002] d..2. 52.659146: pv_wait: cpu 2 out of wfi
> lock_torture_wr-114 [008] d..2. 52.659507: pv_wait: cpu 8 out of wfi
> lock_torture_wr-114 [008] d..2. 52.663503: pv_wait: cpu 8 out of wfi
> lock_torture_wr-113 [002] ...2. 52.666128: pv_kick: cpu 2 kick target cpu 8
> lock_torture_wr-114 [008] d..2. 52.667261: pv_wait: cpu 8 out of wfi
> lock_torture_wr-114 [009] .n.2. 53.141515: pv_kick: cpu 9 kick target cpu 11
> lock_torture_wr-113 [002] d..2. 53.143339: pv_wait: cpu 2 out of wfi
> lock_torture_wr-116 [007] d..2. 53.143412: pv_wait: cpu 7 out of wfi
> lock_torture_wr-118 [000] d..2. 53.143457: pv_wait: cpu 0 out of wfi
> lock_torture_wr-115 [008] d..2. 53.143481: pv_wait: cpu 8 out of wfi
> lock_torture_wr-117 [011] d..2. 53.143522: pv_wait: cpu 11 out of wfi
> lock_torture_wr-117 [011] ...2. 53.143987: pv_kick: cpu 11 kick target cpu 8
> lock_torture_wr-115 [008] ...2. 53.144269: pv_kick: cpu 8 kick target cpu 7
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/kernel/qspinlock_paravirt.c | 8 +++
> .../kernel/trace_events_filter_paravirt.h | 60 +++++++++++++++++++
> 2 files changed, 68 insertions(+)
> create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
>
> diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> index 571626f350be..5d298e989b99 100644
> --- a/arch/riscv/kernel/qspinlock_paravirt.c
> +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> @@ -9,10 +9,16 @@
> #include <asm/qspinlock_paravirt.h>
> #include <asm/sbi.h>
>
> +#define CREATE_TRACE_POINTS
> +#include "trace_events_filter_paravirt.h"
> +
> void pv_kick(int cpu)
> {
> sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
> cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
> +
> + trace_pv_kick(smp_processor_id(), cpu);
> +
> return;
> }
>
> @@ -28,6 +34,8 @@ void pv_wait(u8 *ptr, u8 val)
> goto out;
>
> wait_for_interrupt();
> +
> + trace_pv_wait(smp_processor_id());
> out:
> local_irq_restore(flags);
> }
> diff --git a/arch/riscv/kernel/trace_events_filter_paravirt.h b/arch/riscv/kernel/trace_events_filter_paravirt.h
> new file mode 100644
> index 000000000000..9ff5aa451b12
> --- /dev/null
> +++ b/arch/riscv/kernel/trace_events_filter_paravirt.h
> @@ -0,0 +1,60 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c), 2023 Alibaba Cloud
> + * Authors:
> + * Guo Ren <guoren@linux.alibaba.com>
> + */
> +#undef TRACE_SYSTEM
> +#define TRACE_SYSTEM paravirt
> +
> +#if !defined(_TRACE_PARAVIRT_H) || defined(TRACE_HEADER_MULTI_READ)
> +#define _TRACE_PARAVIRT_H
> +
> +#include <linux/tracepoint.h>
> +
> +TRACE_EVENT(pv_kick,
> + TP_PROTO(int cpu, int target),
> + TP_ARGS(cpu, target),
> +
> + TP_STRUCT__entry(
> + __field(int, cpu)
> + __field(int, target)
> + ),
> +
> + TP_fast_assign(
> + __entry->cpu = cpu;
> + __entry->target = target;
> + ),
> +
> + TP_printk("cpu %d kick target cpu %d",
> + __entry->cpu,
> + __entry->target
> + )
> +);
> +
> +TRACE_EVENT(pv_wait,
> + TP_PROTO(int cpu),
> + TP_ARGS(cpu),
> +
> + TP_STRUCT__entry(
> + __field(int, cpu)
> + ),
> +
> + TP_fast_assign(
> + __entry->cpu = cpu;
> + ),
> +
> + TP_printk("cpu %d out of wfi",
> + __entry->cpu
> + )
> +);
> +
> +#endif /* _TRACE_PARAVIRT_H || TRACE_HEADER_MULTI_READ */
> +
> +#undef TRACE_INCLUDE_PATH
> +#undef TRACE_INCLUDE_FILE
> +#define TRACE_INCLUDE_PATH ../../../arch/riscv/kernel/
> +#define TRACE_INCLUDE_FILE trace_events_filter_paravirt
> +
> +/* This part must be outside protection */
> +#include <trace/define_trace.h>
> --
> 2.36.1
>
LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 15/17] RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
@ 2023-09-15 6:33 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:33 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:09AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Add trace point for pv_kick/wait, here is the output:
>
> entries-in-buffer/entries-written: 33927/33927 #P:12
>
> _-----=> irqs-off/BH-disabled
> / _----=> need-resched
> | / _---=> hardirq/softirq
> || / _--=> preempt-depth
> ||| / _-=> migrate-disable
> |||| / delay
> TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> | | | ||||| | |
> sh-100 [001] d..2. 28.312294: pv_wait: cpu 1 out of wfi
> <idle>-0 [000] d.h4. 28.322030: pv_kick: cpu 0 kick target cpu 1
> sh-100 [001] d..2. 30.982631: pv_wait: cpu 1 out of wfi
> <idle>-0 [000] d.h4. 30.993289: pv_kick: cpu 0 kick target cpu 1
> sh-100 [002] d..2. 44.987573: pv_wait: cpu 2 out of wfi
> <idle>-0 [000] d.h4. 44.989000: pv_kick: cpu 0 kick target cpu 2
> <idle>-0 [003] d.s3. 51.593978: pv_kick: cpu 3 kick target cpu 4
> rcu_sched-15 [004] d..2. 51.595192: pv_wait: cpu 4 out of wfi
> lock_torture_wr-115 [004] ...2. 52.656482: pv_kick: cpu 4 kick target cpu 2
> lock_torture_wr-113 [002] d..2. 52.659146: pv_wait: cpu 2 out of wfi
> lock_torture_wr-114 [008] d..2. 52.659507: pv_wait: cpu 8 out of wfi
> lock_torture_wr-114 [008] d..2. 52.663503: pv_wait: cpu 8 out of wfi
> lock_torture_wr-113 [002] ...2. 52.666128: pv_kick: cpu 2 kick target cpu 8
> lock_torture_wr-114 [008] d..2. 52.667261: pv_wait: cpu 8 out of wfi
> lock_torture_wr-114 [009] .n.2. 53.141515: pv_kick: cpu 9 kick target cpu 11
> lock_torture_wr-113 [002] d..2. 53.143339: pv_wait: cpu 2 out of wfi
> lock_torture_wr-116 [007] d..2. 53.143412: pv_wait: cpu 7 out of wfi
> lock_torture_wr-118 [000] d..2. 53.143457: pv_wait: cpu 0 out of wfi
> lock_torture_wr-115 [008] d..2. 53.143481: pv_wait: cpu 8 out of wfi
> lock_torture_wr-117 [011] d..2. 53.143522: pv_wait: cpu 11 out of wfi
> lock_torture_wr-117 [011] ...2. 53.143987: pv_kick: cpu 11 kick target cpu 8
> lock_torture_wr-115 [008] ...2. 53.144269: pv_kick: cpu 8 kick target cpu 7
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/kernel/qspinlock_paravirt.c | 8 +++
> .../kernel/trace_events_filter_paravirt.h | 60 +++++++++++++++++++
> 2 files changed, 68 insertions(+)
> create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
>
> diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> index 571626f350be..5d298e989b99 100644
> --- a/arch/riscv/kernel/qspinlock_paravirt.c
> +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> @@ -9,10 +9,16 @@
> #include <asm/qspinlock_paravirt.h>
> #include <asm/sbi.h>
>
> +#define CREATE_TRACE_POINTS
> +#include "trace_events_filter_paravirt.h"
> +
> void pv_kick(int cpu)
> {
> sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
> cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
> +
> + trace_pv_kick(smp_processor_id(), cpu);
> +
> return;
> }
>
> @@ -28,6 +34,8 @@ void pv_wait(u8 *ptr, u8 val)
> goto out;
>
> wait_for_interrupt();
> +
> + trace_pv_wait(smp_processor_id());
> out:
> local_irq_restore(flags);
> }
> diff --git a/arch/riscv/kernel/trace_events_filter_paravirt.h b/arch/riscv/kernel/trace_events_filter_paravirt.h
> new file mode 100644
> index 000000000000..9ff5aa451b12
> --- /dev/null
> +++ b/arch/riscv/kernel/trace_events_filter_paravirt.h
> @@ -0,0 +1,60 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c), 2023 Alibaba Cloud
> + * Authors:
> + * Guo Ren <guoren@linux.alibaba.com>
> + */
> +#undef TRACE_SYSTEM
> +#define TRACE_SYSTEM paravirt
> +
> +#if !defined(_TRACE_PARAVIRT_H) || defined(TRACE_HEADER_MULTI_READ)
> +#define _TRACE_PARAVIRT_H
> +
> +#include <linux/tracepoint.h>
> +
> +TRACE_EVENT(pv_kick,
> + TP_PROTO(int cpu, int target),
> + TP_ARGS(cpu, target),
> +
> + TP_STRUCT__entry(
> + __field(int, cpu)
> + __field(int, target)
> + ),
> +
> + TP_fast_assign(
> + __entry->cpu = cpu;
> + __entry->target = target;
> + ),
> +
> + TP_printk("cpu %d kick target cpu %d",
> + __entry->cpu,
> + __entry->target
> + )
> +);
> +
> +TRACE_EVENT(pv_wait,
> + TP_PROTO(int cpu),
> + TP_ARGS(cpu),
> +
> + TP_STRUCT__entry(
> + __field(int, cpu)
> + ),
> +
> + TP_fast_assign(
> + __entry->cpu = cpu;
> + ),
> +
> + TP_printk("cpu %d out of wfi",
> + __entry->cpu
> + )
> +);
> +
> +#endif /* _TRACE_PARAVIRT_H || TRACE_HEADER_MULTI_READ */
> +
> +#undef TRACE_INCLUDE_PATH
> +#undef TRACE_INCLUDE_FILE
> +#define TRACE_INCLUDE_PATH ../../../arch/riscv/kernel/
> +#define TRACE_INCLUDE_FILE trace_events_filter_paravirt
> +
> +/* This part must be outside protection */
> +#include <trace/define_trace.h>
> --
> 2.36.1
>
LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 16/17] RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
2023-09-10 8:29 ` guoren
@ 2023-09-15 6:46 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:46 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:10AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Add the files functions needed to support the SBI PVLOCK (paravirt
> qspinlock kick_cpu) extension. This is a preparation for the next
> core implementation of kick_cpu.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
> arch/riscv/include/uapi/asm/kvm.h | 1 +
> arch/riscv/kvm/Makefile | 1 +
> arch/riscv/kvm/vcpu_sbi.c | 4 +++
> arch/riscv/kvm/vcpu_sbi_pvlock.c | 38 +++++++++++++++++++++++++++
> 5 files changed, 45 insertions(+)
> create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
>
> diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> index cdcf0ff07be7..7b4d60b54d7e 100644
> --- a/arch/riscv/include/asm/kvm_vcpu_sbi.h
> +++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> @@ -71,6 +71,7 @@ extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_srst;
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_hsm;
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_experimental;
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_vendor;
> +extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pvlock;
>
> #ifdef CONFIG_RISCV_PMU_SBI
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pmu;
> diff --git a/arch/riscv/include/uapi/asm/kvm.h b/arch/riscv/include/uapi/asm/kvm.h
> index 992c5e407104..d005c229f2da 100644
> --- a/arch/riscv/include/uapi/asm/kvm.h
> +++ b/arch/riscv/include/uapi/asm/kvm.h
> @@ -148,6 +148,7 @@ enum KVM_RISCV_SBI_EXT_ID {
> KVM_RISCV_SBI_EXT_PMU,
> KVM_RISCV_SBI_EXT_EXPERIMENTAL,
> KVM_RISCV_SBI_EXT_VENDOR,
> + KVM_RISCV_SBI_EXT_PVLOCK,
> KVM_RISCV_SBI_EXT_MAX,
> };
>
> diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
> index 4c2067fc59fc..6112750a3a0c 100644
> --- a/arch/riscv/kvm/Makefile
> +++ b/arch/riscv/kvm/Makefile
> @@ -26,6 +26,7 @@ kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
> kvm-y += vcpu_sbi_base.o
> kvm-y += vcpu_sbi_replace.o
> kvm-y += vcpu_sbi_hsm.o
> +kvm-y += vcpu_sbi_pvlock.o
> kvm-y += vcpu_timer.o
> kvm-$(CONFIG_RISCV_PMU_SBI) += vcpu_pmu.o vcpu_sbi_pmu.o
> kvm-y += aia.o
> diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
> index 9cd97091c723..c03c3d489b2b 100644
> --- a/arch/riscv/kvm/vcpu_sbi.c
> +++ b/arch/riscv/kvm/vcpu_sbi.c
> @@ -74,6 +74,10 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ext[] = {
> .ext_idx = KVM_RISCV_SBI_EXT_VENDOR,
> .ext_ptr = &vcpu_sbi_ext_vendor,
> },
> + {
> + .ext_idx = KVM_RISCV_SBI_EXT_PVLOCK,
> + .ext_ptr = &vcpu_sbi_ext_pvlock,
> + },
> };
>
> void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
> diff --git a/arch/riscv/kvm/vcpu_sbi_pvlock.c b/arch/riscv/kvm/vcpu_sbi_pvlock.c
> new file mode 100644
> index 000000000000..544a456c5041
> --- /dev/null
> +++ b/arch/riscv/kvm/vcpu_sbi_pvlock.c
> @@ -0,0 +1,38 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c), 2023 Alibaba Cloud
> + *
> + * Authors:
> + * Guo Ren <guoren@linux.alibaba.com>
> + */
> +
> +#include <linux/errno.h>
> +#include <linux/err.h>
> +#include <linux/kvm_host.h>
> +#include <asm/sbi.h>
> +#include <asm/kvm_vcpu_sbi.h>
> +
> +static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> + struct kvm_vcpu_sbi_return *retdata)
> +{
> + int ret = 0;
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + unsigned long funcid = cp->a6;
> +
> + switch (funcid) {
> + case SBI_EXT_PVLOCK_KICK_CPU:
> + break;
IIUC, the kick implementation comes in the next patch but here it becomes a
no-op. Is there any chance this may break a future bisect?
I don't understand a lot, but I would suggest either removing this no-op
case SBI_EXT_PVLOCK_KICK_CPU, or merging this patch with the next one.
Other than that, LGTM.
Thanks,
Leo
> + default:
> + ret = SBI_ERR_NOT_SUPPORTED;
> + }
> +
> + retdata->err_val = ret;
> +
> + return 0;
> +}
> +
> +const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pvlock = {
> + .extid_start = SBI_EXT_PVLOCK,
> + .extid_end = SBI_EXT_PVLOCK,
> + .handler = kvm_sbi_ext_pvlock_handler,
> +};
> --
> 2.36.1
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 16/17] RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
@ 2023-09-15 6:46 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:46 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:10AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> Add the files functions needed to support the SBI PVLOCK (paravirt
> qspinlock kick_cpu) extension. This is a preparation for the next
> core implementation of kick_cpu.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
> arch/riscv/include/uapi/asm/kvm.h | 1 +
> arch/riscv/kvm/Makefile | 1 +
> arch/riscv/kvm/vcpu_sbi.c | 4 +++
> arch/riscv/kvm/vcpu_sbi_pvlock.c | 38 +++++++++++++++++++++++++++
> 5 files changed, 45 insertions(+)
> create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
>
> diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> index cdcf0ff07be7..7b4d60b54d7e 100644
> --- a/arch/riscv/include/asm/kvm_vcpu_sbi.h
> +++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> @@ -71,6 +71,7 @@ extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_srst;
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_hsm;
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_experimental;
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_vendor;
> +extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pvlock;
>
> #ifdef CONFIG_RISCV_PMU_SBI
> extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pmu;
> diff --git a/arch/riscv/include/uapi/asm/kvm.h b/arch/riscv/include/uapi/asm/kvm.h
> index 992c5e407104..d005c229f2da 100644
> --- a/arch/riscv/include/uapi/asm/kvm.h
> +++ b/arch/riscv/include/uapi/asm/kvm.h
> @@ -148,6 +148,7 @@ enum KVM_RISCV_SBI_EXT_ID {
> KVM_RISCV_SBI_EXT_PMU,
> KVM_RISCV_SBI_EXT_EXPERIMENTAL,
> KVM_RISCV_SBI_EXT_VENDOR,
> + KVM_RISCV_SBI_EXT_PVLOCK,
> KVM_RISCV_SBI_EXT_MAX,
> };
>
> diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile
> index 4c2067fc59fc..6112750a3a0c 100644
> --- a/arch/riscv/kvm/Makefile
> +++ b/arch/riscv/kvm/Makefile
> @@ -26,6 +26,7 @@ kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
> kvm-y += vcpu_sbi_base.o
> kvm-y += vcpu_sbi_replace.o
> kvm-y += vcpu_sbi_hsm.o
> +kvm-y += vcpu_sbi_pvlock.o
> kvm-y += vcpu_timer.o
> kvm-$(CONFIG_RISCV_PMU_SBI) += vcpu_pmu.o vcpu_sbi_pmu.o
> kvm-y += aia.o
> diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c
> index 9cd97091c723..c03c3d489b2b 100644
> --- a/arch/riscv/kvm/vcpu_sbi.c
> +++ b/arch/riscv/kvm/vcpu_sbi.c
> @@ -74,6 +74,10 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ext[] = {
> .ext_idx = KVM_RISCV_SBI_EXT_VENDOR,
> .ext_ptr = &vcpu_sbi_ext_vendor,
> },
> + {
> + .ext_idx = KVM_RISCV_SBI_EXT_PVLOCK,
> + .ext_ptr = &vcpu_sbi_ext_pvlock,
> + },
> };
>
> void kvm_riscv_vcpu_sbi_forward(struct kvm_vcpu *vcpu, struct kvm_run *run)
> diff --git a/arch/riscv/kvm/vcpu_sbi_pvlock.c b/arch/riscv/kvm/vcpu_sbi_pvlock.c
> new file mode 100644
> index 000000000000..544a456c5041
> --- /dev/null
> +++ b/arch/riscv/kvm/vcpu_sbi_pvlock.c
> @@ -0,0 +1,38 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c), 2023 Alibaba Cloud
> + *
> + * Authors:
> + * Guo Ren <guoren@linux.alibaba.com>
> + */
> +
> +#include <linux/errno.h>
> +#include <linux/err.h>
> +#include <linux/kvm_host.h>
> +#include <asm/sbi.h>
> +#include <asm/kvm_vcpu_sbi.h>
> +
> +static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> + struct kvm_vcpu_sbi_return *retdata)
> +{
> + int ret = 0;
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + unsigned long funcid = cp->a6;
> +
> + switch (funcid) {
> + case SBI_EXT_PVLOCK_KICK_CPU:
> + break;
IIUC, the kick implementation comes in the next patch but here it becomes a
no-op. Is there any chance this may break a future bisect?
I don't understand a lot, but I would suggest either removing this no-op
case SBI_EXT_PVLOCK_KICK_CPU, or merging this patch with the next one.
Other than that, LGTM.
Thanks,
Leo
> + default:
> + ret = SBI_ERR_NOT_SUPPORTED;
> + }
> +
> + retdata->err_val = ret;
> +
> + return 0;
> +}
> +
> +const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pvlock = {
> + .extid_start = SBI_EXT_PVLOCK,
> + .extid_end = SBI_EXT_PVLOCK,
> + .handler = kvm_sbi_ext_pvlock_handler,
> +};
> --
> 2.36.1
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 17/17] RISC-V: paravirt: pvqspinlock: KVM: Implement kvm_sbi_ext_pvlock_kick_cpu()
2023-09-10 8:29 ` guoren
@ 2023-09-15 6:52 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:52 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:11AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> We only need to call the kvm_vcpu_kick() and bring target_vcpu
> from the halt state. No irq raised, no other request, just a pure
> vcpu_kick.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/kvm/vcpu_sbi_pvlock.c | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/arch/riscv/kvm/vcpu_sbi_pvlock.c b/arch/riscv/kvm/vcpu_sbi_pvlock.c
> index 544a456c5041..914fc58aedfe 100644
> --- a/arch/riscv/kvm/vcpu_sbi_pvlock.c
> +++ b/arch/riscv/kvm/vcpu_sbi_pvlock.c
> @@ -12,6 +12,24 @@
> #include <asm/sbi.h>
> #include <asm/kvm_vcpu_sbi.h>
>
> +static int kvm_sbi_ext_pvlock_kick_cpu(struct kvm_vcpu *vcpu)
> +{
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + struct kvm *kvm = vcpu->kvm;
> + struct kvm_vcpu *target;
> +
> + target = kvm_get_vcpu_by_id(kvm, cp->a0);
> + if (!target)
> + return SBI_ERR_INVALID_PARAM;
> +
> + kvm_vcpu_kick(target);
> +
> + if (READ_ONCE(target->ready))
> + kvm_vcpu_yield_to(target);
> +
> + return SBI_SUCCESS;
> +}
> +
> static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> struct kvm_vcpu_sbi_return *retdata)
> {
> @@ -21,6 +39,7 @@ static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run
>
> switch (funcid) {
> case SBI_EXT_PVLOCK_KICK_CPU:
> + ret = kvm_sbi_ext_pvlock_kick_cpu(vcpu);
> break;
> default:
> ret = SBI_ERR_NOT_SUPPORTED;
> --
> 2.36.1
>
LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 17/17] RISC-V: paravirt: pvqspinlock: KVM: Implement kvm_sbi_ext_pvlock_kick_cpu()
@ 2023-09-15 6:52 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 6:52 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:29:11AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> We only need to call the kvm_vcpu_kick() and bring target_vcpu
> from the halt state. No irq raised, no other request, just a pure
> vcpu_kick.
>
> Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren@kernel.org>
> ---
> arch/riscv/kvm/vcpu_sbi_pvlock.c | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/arch/riscv/kvm/vcpu_sbi_pvlock.c b/arch/riscv/kvm/vcpu_sbi_pvlock.c
> index 544a456c5041..914fc58aedfe 100644
> --- a/arch/riscv/kvm/vcpu_sbi_pvlock.c
> +++ b/arch/riscv/kvm/vcpu_sbi_pvlock.c
> @@ -12,6 +12,24 @@
> #include <asm/sbi.h>
> #include <asm/kvm_vcpu_sbi.h>
>
> +static int kvm_sbi_ext_pvlock_kick_cpu(struct kvm_vcpu *vcpu)
> +{
> + struct kvm_cpu_context *cp = &vcpu->arch.guest_context;
> + struct kvm *kvm = vcpu->kvm;
> + struct kvm_vcpu *target;
> +
> + target = kvm_get_vcpu_by_id(kvm, cp->a0);
> + if (!target)
> + return SBI_ERR_INVALID_PARAM;
> +
> + kvm_vcpu_kick(target);
> +
> + if (READ_ONCE(target->ready))
> + kvm_vcpu_yield_to(target);
> +
> + return SBI_SUCCESS;
> +}
> +
> static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
> struct kvm_vcpu_sbi_return *retdata)
> {
> @@ -21,6 +39,7 @@ static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_run *run
>
> switch (funcid) {
> case SBI_EXT_PVLOCK_KICK_CPU:
> + ret = kvm_sbi_ext_pvlock_kick_cpu(vcpu);
> break;
> default:
> ret = SBI_ERR_NOT_SUPPORTED;
> --
> 2.36.1
>
LGTM:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-14 13:47 ` Andrew Jones
@ 2023-09-15 8:22 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 8:22 UTC (permalink / raw)
To: Andrew Jones
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, conor.dooley, xiaoguang.xing, bjorn, alexghiti,
keescook, greentime.hu, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 03:47:59PM +0200, Andrew Jones wrote:
> On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Cache-block prefetch instructions are HINTs to the hardware to
> > indicate that software intends to perform a particular type of
> > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > improve the arch_xchg for qspinlock xchg_tail.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig | 15 +++++++++++++++
> > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > arch/riscv/include/asm/hwcap.h | 1 +
> > arch/riscv/include/asm/insn-def.h | 5 +++++
> > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > arch/riscv/kernel/cpufeature.c | 1 +
> > 6 files changed, 38 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index e9ae6fa232c3..2c346fe169c1 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> >
> > If you don't know what to do here, say Y.
> >
> > +config RISCV_ISA_ZICBOP
> > + bool "Zicbop extension support for cache block prefetch"
> > + depends on MMU
> > + depends on RISCV_ALTERNATIVE
> > + default y
> > + help
> > + Adds support to dynamically detect the presence of the ZICBOP
> > + extension (Cache Block Prefetch Operations) and enable its
> > + usage.
> > +
> > + The Zicbop extension can be used to prefetch cache block for
> > + read/write/instruction fetch.
> > +
> > + If you don't know what to do here, say Y.
> > +
> > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > bool
> > default y
> > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > index 702725727671..56eff7a9d2d2 100644
> > --- a/arch/riscv/include/asm/cmpxchg.h
> > +++ b/arch/riscv/include/asm/cmpxchg.h
> > @@ -11,6 +11,7 @@
> >
> > #include <asm/barrier.h>
> > #include <asm/fence.h>
> > +#include <asm/processor.h>
> >
> > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > ({ \
> > @@ -25,6 +26,7 @@
> > \
> > __asm__ __volatile__ ( \
> > prepend \
> > + PREFETCHW_ASM(%5) \
> > "0: lr.w %0, %2\n" \
> > " and %1, %0, %z4\n" \
> > " or %1, %1, %z3\n" \
> > @@ -32,7 +34,7 @@
> > " bnez %1, 0b\n" \
> > append \
> > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > - : "rJ" (__newx), "rJ" (~__mask) \
> > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > : "memory"); \
> > \
> > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > index b7b58258f6c7..78b7b8b53778 100644
> > --- a/arch/riscv/include/asm/hwcap.h
> > +++ b/arch/riscv/include/asm/hwcap.h
> > @@ -58,6 +58,7 @@
> > #define RISCV_ISA_EXT_ZICSR 40
> > #define RISCV_ISA_EXT_ZIFENCEI 41
> > #define RISCV_ISA_EXT_ZIHPM 42
> > +#define RISCV_ISA_EXT_ZICBOP 43
> >
> > #define RISCV_ISA_EXT_MAX 64
> >
> > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > index 6960beb75f32..dc590d331894 100644
> > --- a/arch/riscv/include/asm/insn-def.h
> > +++ b/arch/riscv/include/asm/insn-def.h
> > @@ -134,6 +134,7 @@
> >
> > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
>
> This should be named RV_OPCODE_OP_IMM and be placed in
> numerical order with the others, i.e. above SYSTEM.
>
> >
> > #define HFENCE_VVMA(vaddr, asid) \
> > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > @@ -196,4 +197,8 @@
> > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > RS1(base), SIMM12(4))
> >
> > +#define CBO_prefetchw(base) \
>
> Please name this 'PREFETCH_w' and it should take an immediate parameter,
> even if we intend to pass 0 for it.
It makes sense.
The mnemonic in the previously mentioned documentation is:
prefetch.w offset(base)
So yeah, makes sense to have both offset and base as parameters for
CBO_prefetchw (or PREFETCH_w, I have no strong preference).
>
> > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > + RD(x0), RS1(base), RS2(x0))
>
> prefetch.w is not an R-type instruction, it's an S-type. While the bit
> shifts are the same, the names are different. We need to add S-type
> names while defining this instruction.
That is correct, it is supposed to look like a store instruction (S-type),
even though documentation don't explicitly state that.
Even though it works fine with the R-type definition, code documentation
would be wrong, and future changes could break it.
> Then, this define would be
>
> #define PREFETCH_w(base, imm) \
> INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
> RS1(base), __RS2(3))
s/OPCODE_OP_IMM/OPCODE_PREFETCH
0x4 vs 0x13
RS2 == 0x3 is correct (PREFETCH.W instead of PREFETCH.I)
So IIUC, it should be:
INSN_S(OPCODE_PREFETCH, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
RS1(base), __RS2(3)
Thanks,
Leo
>
> When the assembler as insn_r I hope it will validate that
> (imm & 0xfe0) == imm
>
> > +
> > #endif /* __ASM_INSN_DEF_H */
> > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > index de9da852f78d..7ad3a24212e8 100644
> > --- a/arch/riscv/include/asm/processor.h
> > +++ b/arch/riscv/include/asm/processor.h
> > @@ -12,6 +12,8 @@
> > #include <vdso/processor.h>
> >
> > #include <asm/ptrace.h>
> > +#include <asm/insn-def.h>
> > +#include <asm/hwcap.h>
> >
> > #ifdef CONFIG_64BIT
> > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> >
> > +#define ARCH_HAS_PREFETCHW
> > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > + CBO_prefetchw(base), \
> > + 0, \
> > + RISCV_ISA_EXT_ZICBOP, \
> > + CONFIG_RISCV_ISA_ZICBOP)
> > +static inline void prefetchw(const void *ptr)
> > +{
> > + asm volatile(PREFETCHW_ASM(%0)
> > + : : "r" (ptr) : "memory");
> > +}
> >
> > /* Do necessary setup to start up a newly executed thread. */
> > extern void start_thread(struct pt_regs *regs,
> > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > index ef7b4fd9e876..e0b897db0b97 100644
> > --- a/arch/riscv/kernel/cpufeature.c
> > +++ b/arch/riscv/kernel/cpufeature.c
> > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
>
> zicbop should be above zicboz (extensions alphabetical within their
> category).
>
> > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > --
> > 2.36.1
> >
>
> Thanks,
> drew
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-15 8:22 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 8:22 UTC (permalink / raw)
To: Andrew Jones
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, conor.dooley, xiaoguang.xing, bjorn, alexghiti,
keescook, greentime.hu, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 03:47:59PM +0200, Andrew Jones wrote:
> On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Cache-block prefetch instructions are HINTs to the hardware to
> > indicate that software intends to perform a particular type of
> > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > improve the arch_xchg for qspinlock xchg_tail.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig | 15 +++++++++++++++
> > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > arch/riscv/include/asm/hwcap.h | 1 +
> > arch/riscv/include/asm/insn-def.h | 5 +++++
> > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > arch/riscv/kernel/cpufeature.c | 1 +
> > 6 files changed, 38 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index e9ae6fa232c3..2c346fe169c1 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> >
> > If you don't know what to do here, say Y.
> >
> > +config RISCV_ISA_ZICBOP
> > + bool "Zicbop extension support for cache block prefetch"
> > + depends on MMU
> > + depends on RISCV_ALTERNATIVE
> > + default y
> > + help
> > + Adds support to dynamically detect the presence of the ZICBOP
> > + extension (Cache Block Prefetch Operations) and enable its
> > + usage.
> > +
> > + The Zicbop extension can be used to prefetch cache block for
> > + read/write/instruction fetch.
> > +
> > + If you don't know what to do here, say Y.
> > +
> > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > bool
> > default y
> > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > index 702725727671..56eff7a9d2d2 100644
> > --- a/arch/riscv/include/asm/cmpxchg.h
> > +++ b/arch/riscv/include/asm/cmpxchg.h
> > @@ -11,6 +11,7 @@
> >
> > #include <asm/barrier.h>
> > #include <asm/fence.h>
> > +#include <asm/processor.h>
> >
> > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > ({ \
> > @@ -25,6 +26,7 @@
> > \
> > __asm__ __volatile__ ( \
> > prepend \
> > + PREFETCHW_ASM(%5) \
> > "0: lr.w %0, %2\n" \
> > " and %1, %0, %z4\n" \
> > " or %1, %1, %z3\n" \
> > @@ -32,7 +34,7 @@
> > " bnez %1, 0b\n" \
> > append \
> > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > - : "rJ" (__newx), "rJ" (~__mask) \
> > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > : "memory"); \
> > \
> > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > index b7b58258f6c7..78b7b8b53778 100644
> > --- a/arch/riscv/include/asm/hwcap.h
> > +++ b/arch/riscv/include/asm/hwcap.h
> > @@ -58,6 +58,7 @@
> > #define RISCV_ISA_EXT_ZICSR 40
> > #define RISCV_ISA_EXT_ZIFENCEI 41
> > #define RISCV_ISA_EXT_ZIHPM 42
> > +#define RISCV_ISA_EXT_ZICBOP 43
> >
> > #define RISCV_ISA_EXT_MAX 64
> >
> > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > index 6960beb75f32..dc590d331894 100644
> > --- a/arch/riscv/include/asm/insn-def.h
> > +++ b/arch/riscv/include/asm/insn-def.h
> > @@ -134,6 +134,7 @@
> >
> > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
>
> This should be named RV_OPCODE_OP_IMM and be placed in
> numerical order with the others, i.e. above SYSTEM.
>
> >
> > #define HFENCE_VVMA(vaddr, asid) \
> > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > @@ -196,4 +197,8 @@
> > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > RS1(base), SIMM12(4))
> >
> > +#define CBO_prefetchw(base) \
>
> Please name this 'PREFETCH_w' and it should take an immediate parameter,
> even if we intend to pass 0 for it.
It makes sense.
The mnemonic in the previously mentioned documentation is:
prefetch.w offset(base)
So yeah, makes sense to have both offset and base as parameters for
CBO_prefetchw (or PREFETCH_w, I have no strong preference).
>
> > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > + RD(x0), RS1(base), RS2(x0))
>
> prefetch.w is not an R-type instruction, it's an S-type. While the bit
> shifts are the same, the names are different. We need to add S-type
> names while defining this instruction.
That is correct, it is supposed to look like a store instruction (S-type),
even though documentation don't explicitly state that.
Even though it works fine with the R-type definition, code documentation
would be wrong, and future changes could break it.
> Then, this define would be
>
> #define PREFETCH_w(base, imm) \
> INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
> RS1(base), __RS2(3))
s/OPCODE_OP_IMM/OPCODE_PREFETCH
0x4 vs 0x13
RS2 == 0x3 is correct (PREFETCH.W instead of PREFETCH.I)
So IIUC, it should be:
INSN_S(OPCODE_PREFETCH, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
RS1(base), __RS2(3)
Thanks,
Leo
>
> When the assembler as insn_r I hope it will validate that
> (imm & 0xfe0) == imm
>
> > +
> > #endif /* __ASM_INSN_DEF_H */
> > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > index de9da852f78d..7ad3a24212e8 100644
> > --- a/arch/riscv/include/asm/processor.h
> > +++ b/arch/riscv/include/asm/processor.h
> > @@ -12,6 +12,8 @@
> > #include <vdso/processor.h>
> >
> > #include <asm/ptrace.h>
> > +#include <asm/insn-def.h>
> > +#include <asm/hwcap.h>
> >
> > #ifdef CONFIG_64BIT
> > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> >
> > +#define ARCH_HAS_PREFETCHW
> > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > + CBO_prefetchw(base), \
> > + 0, \
> > + RISCV_ISA_EXT_ZICBOP, \
> > + CONFIG_RISCV_ISA_ZICBOP)
> > +static inline void prefetchw(const void *ptr)
> > +{
> > + asm volatile(PREFETCHW_ASM(%0)
> > + : : "r" (ptr) : "memory");
> > +}
> >
> > /* Do necessary setup to start up a newly executed thread. */
> > extern void start_thread(struct pt_regs *regs,
> > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > index ef7b4fd9e876..e0b897db0b97 100644
> > --- a/arch/riscv/kernel/cpufeature.c
> > +++ b/arch/riscv/kernel/cpufeature.c
> > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
>
> zicbop should be above zicboz (extensions alphabetical within their
> category).
>
> > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > --
> > 2.36.1
> >
>
> Thanks,
> drew
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
2023-09-15 2:10 ` Guo Ren
@ 2023-09-15 9:08 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 9:08 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 10:10:25AM +0800, Guo Ren wrote:
> On Thu, Sep 14, 2023 at 5:43 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote:
> > > On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > >
> > > > > The requirements of qspinlock have been documented by commit:
> > > > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > > > > atomics").
> > > > >
> > > > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > > > > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > > > > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > > > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > > > > is the one.
> > > > >
> > > > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > > > >
> > > > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > > > queued_spinlock 0.5109/0.00
> > > > > ticket_spinlock 0.5814/0.00
> > > > >
> > > > > perf futex/hash (+6.7%):
> > > > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > > > >
> > > > > perf futex/wake-parallel (+8.6%):
> > > > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > > > >
> > > > > perf futex/requeue (+4.2%):
> > > > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > > > >
> > > > > System Benchmarks (+6.4%)
> > > > > queued_spinlock:
> > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > > > Execl Throughput 43.0 13116.6 3050.4
> > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > > > Process Creation 126.0 12537.3 995.0
> > > > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > > > System Call Overhead 15000.0 33308301.3 22205.5
> > > > > ========
> > > > > System Benchmarks Index Score 12426.1
> > > > >
> > > > > ticket_spinlock:
> > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > > > Execl Throughput 43.0 12625.1 2936.1
> > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > > > Process Creation 126.0 10798.4 857.0
> > > > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > > > System Call Overhead 15000.0 30766778.4 20511.2
> > > > > ========
> > > > > System Benchmarks Index Score 11670.7
> > > > >
> > > > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > > > cores platform than the ticket_lock.
> > > > >
> > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > ---
> > > > > arch/riscv/Kconfig | 16 ++++++++++++++++
> > > > > arch/riscv/include/asm/Kbuild | 3 ++-
> > > > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > > > > 3 files changed, 35 insertions(+), 1 deletion(-)
> > > > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > > > >
> > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > index 2c346fe169c1..7f39bfc75744 100644
> > > > > --- a/arch/riscv/Kconfig
> > > > > +++ b/arch/riscv/Kconfig
> > > > > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > > > > Specify the maximum number of NUMA Nodes available on the target
> > > > > system. Increases memory reserved to accommodate various tables.
> > > > >
> > > > > +choice
> > > > > + prompt "RISC-V spinlock type"
> > > > > + default RISCV_TICKET_SPINLOCKS
> > > > > +
> > > > > +config RISCV_TICKET_SPINLOCKS
> > > > > + bool "Using ticket spinlock"
> > > > > +
> > > > > +config RISCV_QUEUED_SPINLOCKS
> > > > > + bool "Using queued spinlock"
> > > > > + depends on SMP && MMU
> > > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > > + help
> > > > > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > > + Otherwise, stay at ticket-lock.
> > > > > +endchoice
> > > > > +
> > > > > config RISCV_ALTERNATIVE
> > > > > bool
> > > > > depends on !XIP_KERNEL
> > > > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > > > index 504f8b7e72d4..a0dc85e4a754 100644
> > > > > --- a/arch/riscv/include/asm/Kbuild
> > > > > +++ b/arch/riscv/include/asm/Kbuild
> > > > > @@ -2,10 +2,11 @@
> > > > > generic-y += early_ioremap.h
> > > > > generic-y += flat.h
> > > > > generic-y += kvm_para.h
> > > > > +generic-y += mcs_spinlock.h
> > > > > generic-y += parport.h
> > > > > -generic-y += spinlock.h
> > > >
> > > > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> > > > and include the asm-generic headers of mcs_spinlock and qspinlock.
> > > >
> > > > In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> > > > which seems the oposite of the above description (ticket spinlocks being
> > > > the standard).
> > > >
> > > > Shouldn't ticket-spinlock.h also get included here?
> > > > (Also, I am probably missing something, as I dont' see the use of
> > > > mcs_spinlock here.)
> > > No, because asm-generic/spinlock.h:
> > > ...
> > > #include <asm-generic/ticket_spinlock.h>
> > > ...
> > >
> >
> > But aren't you removing asm-generic/spinlock.h below ?
> > -generic-y += spinlock.h
> Yes, current is:
>
> arch/riscv/include/asm/spinlock.h -> include/asm-generic/spinlock.h ->
> include/asm-generic/ticket_spinlock.h
I did a little reading on how generic-y works (which I was unaware):
"If an architecture uses a verbatim copy of a header from
include/asm-generic then this is listed in the file
arch/$(SRCARCH)/include/asm/Kbuild [...] During the prepare phase of the
build a wrapper include file is generated in the directory [...]"
Oh, so you are removing the asm-generic/spinlock.h because it's link
was replaced by a new asm/spinlock.h.
You add qspinlock.h to generic-y because it's new in riscv, and add
mcs_spinlock.h because it's needed by qspinlock.h.
Ok, it makes sense now.
Sorry about this noise.
I was unaware of how generic-y worked, and (wrongly)
assumed it was about including headers automatically in the build.
>
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#include <asm/qspinlock.h>
> +#include <asm/qrwlock.h>
> +#else
> +#include <asm-generic/spinlock.h>
> +#endif
>
> So, you want me:
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#include <asm/qspinlock.h>
> +#else
> +#include <asm-generic/ticket_spinlock.h>
> +#endif
>
> +#include <asm/qrwlock.h>
>
> Right?
No, I didn't mean that.
I was just worried about the arch_spin_*() interfaces, but they should be
fine.
BTW, according to kernel doc on generic-y, shouldn't be a better idea to
add 'ticket_spinlock.h' to generic-y, and include above as
asm/ticket_spinlock.h?
Or is generic-y reserved only for stuff which is indirectly included by
other headers?
Thanks!
Leo
>
> >
> > > >
> > > > > generic-y += spinlock_types.h
> > > > > generic-y += qrwlock.h
> > > > > generic-y += qrwlock_types.h
> > > > > +generic-y += qspinlock.h
> > > > > generic-y += user.h
> > > > > generic-y += vmlinux.lds.h
> > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > new file mode 100644
> > > > > index 000000000000..c644a92d4548
> > > > > --- /dev/null
> > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > @@ -0,0 +1,17 @@
> > > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > > +
> > > > > +#ifndef __ASM_RISCV_SPINLOCK_H
> > > > > +#define __ASM_RISCV_SPINLOCK_H
> > > > > +
> > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > +#define _Q_PENDING_LOOPS (1 << 9)
> > > > > +#endif
> > > >
> > > > Any reason the above define couldn't be merged on the ifdef below?
> > > Easy for the next patch to modify. See Waiman's comment:
> > >
> > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> > >
> > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > index c644a92d4548..9eb3ad31e564 100644
> > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > @@ -7,11 +7,94 @@
> > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > #endif
> > > >
> > >
> > > I see why you separated the _Q_PENDING_LOOPS out.
> > >
> >
> > I see, should be fine then.
> >
> > Thanks!
> > Leo
> >
> > >
> > > >
> > > > > +
> > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > +#include <asm/qspinlock.h>
> > > > > +#include <asm/qrwlock.h>
> > > > > +#else
> > > > > +#include <asm-generic/spinlock.h>
> > > > > +#endif
> > > > > +
> > > > > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > > > > --
> > > > > 2.36.1
> > > > >
> > > >
> > > > Thanks!
> > > > Leo
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
@ 2023-09-15 9:08 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 9:08 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 10:10:25AM +0800, Guo Ren wrote:
> On Thu, Sep 14, 2023 at 5:43 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote:
> > > On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > >
> > > > > The requirements of qspinlock have been documented by commit:
> > > > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > > > > atomics").
> > > > >
> > > > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > > > > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > > > > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > > > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > > > > is the one.
> > > > >
> > > > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > > > >
> > > > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > > > queued_spinlock 0.5109/0.00
> > > > > ticket_spinlock 0.5814/0.00
> > > > >
> > > > > perf futex/hash (+6.7%):
> > > > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > > > >
> > > > > perf futex/wake-parallel (+8.6%):
> > > > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > > > >
> > > > > perf futex/requeue (+4.2%):
> > > > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > > > >
> > > > > System Benchmarks (+6.4%)
> > > > > queued_spinlock:
> > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > > > Execl Throughput 43.0 13116.6 3050.4
> > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > > > Process Creation 126.0 12537.3 995.0
> > > > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > > > System Call Overhead 15000.0 33308301.3 22205.5
> > > > > ========
> > > > > System Benchmarks Index Score 12426.1
> > > > >
> > > > > ticket_spinlock:
> > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > > > Execl Throughput 43.0 12625.1 2936.1
> > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > > > Process Creation 126.0 10798.4 857.0
> > > > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > > > System Call Overhead 15000.0 30766778.4 20511.2
> > > > > ========
> > > > > System Benchmarks Index Score 11670.7
> > > > >
> > > > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > > > cores platform than the ticket_lock.
> > > > >
> > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > ---
> > > > > arch/riscv/Kconfig | 16 ++++++++++++++++
> > > > > arch/riscv/include/asm/Kbuild | 3 ++-
> > > > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > > > > 3 files changed, 35 insertions(+), 1 deletion(-)
> > > > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > > > >
> > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > index 2c346fe169c1..7f39bfc75744 100644
> > > > > --- a/arch/riscv/Kconfig
> > > > > +++ b/arch/riscv/Kconfig
> > > > > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > > > > Specify the maximum number of NUMA Nodes available on the target
> > > > > system. Increases memory reserved to accommodate various tables.
> > > > >
> > > > > +choice
> > > > > + prompt "RISC-V spinlock type"
> > > > > + default RISCV_TICKET_SPINLOCKS
> > > > > +
> > > > > +config RISCV_TICKET_SPINLOCKS
> > > > > + bool "Using ticket spinlock"
> > > > > +
> > > > > +config RISCV_QUEUED_SPINLOCKS
> > > > > + bool "Using queued spinlock"
> > > > > + depends on SMP && MMU
> > > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > > + help
> > > > > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > > + Otherwise, stay at ticket-lock.
> > > > > +endchoice
> > > > > +
> > > > > config RISCV_ALTERNATIVE
> > > > > bool
> > > > > depends on !XIP_KERNEL
> > > > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > > > index 504f8b7e72d4..a0dc85e4a754 100644
> > > > > --- a/arch/riscv/include/asm/Kbuild
> > > > > +++ b/arch/riscv/include/asm/Kbuild
> > > > > @@ -2,10 +2,11 @@
> > > > > generic-y += early_ioremap.h
> > > > > generic-y += flat.h
> > > > > generic-y += kvm_para.h
> > > > > +generic-y += mcs_spinlock.h
> > > > > generic-y += parport.h
> > > > > -generic-y += spinlock.h
> > > >
> > > > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> > > > and include the asm-generic headers of mcs_spinlock and qspinlock.
> > > >
> > > > In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> > > > which seems the oposite of the above description (ticket spinlocks being
> > > > the standard).
> > > >
> > > > Shouldn't ticket-spinlock.h also get included here?
> > > > (Also, I am probably missing something, as I dont' see the use of
> > > > mcs_spinlock here.)
> > > No, because asm-generic/spinlock.h:
> > > ...
> > > #include <asm-generic/ticket_spinlock.h>
> > > ...
> > >
> >
> > But aren't you removing asm-generic/spinlock.h below ?
> > -generic-y += spinlock.h
> Yes, current is:
>
> arch/riscv/include/asm/spinlock.h -> include/asm-generic/spinlock.h ->
> include/asm-generic/ticket_spinlock.h
I did a little reading on how generic-y works (which I was unaware):
"If an architecture uses a verbatim copy of a header from
include/asm-generic then this is listed in the file
arch/$(SRCARCH)/include/asm/Kbuild [...] During the prepare phase of the
build a wrapper include file is generated in the directory [...]"
Oh, so you are removing the asm-generic/spinlock.h because it's link
was replaced by a new asm/spinlock.h.
You add qspinlock.h to generic-y because it's new in riscv, and add
mcs_spinlock.h because it's needed by qspinlock.h.
Ok, it makes sense now.
Sorry about this noise.
I was unaware of how generic-y worked, and (wrongly)
assumed it was about including headers automatically in the build.
>
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#include <asm/qspinlock.h>
> +#include <asm/qrwlock.h>
> +#else
> +#include <asm-generic/spinlock.h>
> +#endif
>
> So, you want me:
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#include <asm/qspinlock.h>
> +#else
> +#include <asm-generic/ticket_spinlock.h>
> +#endif
>
> +#include <asm/qrwlock.h>
>
> Right?
No, I didn't mean that.
I was just worried about the arch_spin_*() interfaces, but they should be
fine.
BTW, according to kernel doc on generic-y, shouldn't be a better idea to
add 'ticket_spinlock.h' to generic-y, and include above as
asm/ticket_spinlock.h?
Or is generic-y reserved only for stuff which is indirectly included by
other headers?
Thanks!
Leo
>
> >
> > > >
> > > > > generic-y += spinlock_types.h
> > > > > generic-y += qrwlock.h
> > > > > generic-y += qrwlock_types.h
> > > > > +generic-y += qspinlock.h
> > > > > generic-y += user.h
> > > > > generic-y += vmlinux.lds.h
> > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > new file mode 100644
> > > > > index 000000000000..c644a92d4548
> > > > > --- /dev/null
> > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > @@ -0,0 +1,17 @@
> > > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > > +
> > > > > +#ifndef __ASM_RISCV_SPINLOCK_H
> > > > > +#define __ASM_RISCV_SPINLOCK_H
> > > > > +
> > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > +#define _Q_PENDING_LOOPS (1 << 9)
> > > > > +#endif
> > > >
> > > > Any reason the above define couldn't be merged on the ifdef below?
> > > Easy for the next patch to modify. See Waiman's comment:
> > >
> > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> > >
> > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > index c644a92d4548..9eb3ad31e564 100644
> > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > @@ -7,11 +7,94 @@
> > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > #endif
> > > >
> > >
> > > I see why you separated the _Q_PENDING_LOOPS out.
> > >
> >
> > I see, should be fine then.
> >
> > Thanks!
> > Leo
> >
> > >
> > > >
> > > > > +
> > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > +#include <asm/qspinlock.h>
> > > > > +#include <asm/qrwlock.h>
> > > > > +#else
> > > > > +#include <asm-generic/spinlock.h>
> > > > > +#endif
> > > > > +
> > > > > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > > > > --
> > > > > 2.36.1
> > > > >
> > > >
> > > > Thanks!
> > > > Leo
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-15 8:22 ` Leonardo Bras
@ 2023-09-15 11:07 ` Andrew Jones
-1 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-15 11:07 UTC (permalink / raw)
To: Leonardo Bras
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, conor.dooley, xiaoguang.xing, bjorn, alexghiti,
keescook, greentime.hu, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 05:22:26AM -0300, Leonardo Bras wrote:
> On Thu, Sep 14, 2023 at 03:47:59PM +0200, Andrew Jones wrote:
> > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
...
> > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > index 6960beb75f32..dc590d331894 100644
> > > --- a/arch/riscv/include/asm/insn-def.h
> > > +++ b/arch/riscv/include/asm/insn-def.h
> > > @@ -134,6 +134,7 @@
> > >
> > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> >
> > This should be named RV_OPCODE_OP_IMM and be placed in
> > numerical order with the others, i.e. above SYSTEM.
> >
> > >
> > > #define HFENCE_VVMA(vaddr, asid) \
> > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > @@ -196,4 +197,8 @@
> > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > RS1(base), SIMM12(4))
> > >
> > > +#define CBO_prefetchw(base) \
> >
> > Please name this 'PREFETCH_w' and it should take an immediate parameter,
> > even if we intend to pass 0 for it.
>
> It makes sense.
>
> The mnemonic in the previously mentioned documentation is:
>
> prefetch.w offset(base)
>
> So yeah, makes sense to have both offset and base as parameters for
> CBO_prefetchw (or PREFETCH_w, I have no strong preference).
I have a strong preference :-)
PREFETCH_w is consistent with the naming we already have for e.g.
cbo.clean, which is CBO_clean. The instruction we're picking a name
for now is prefetch.w, not cbo.prefetchw.
>
> >
> > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > + RD(x0), RS1(base), RS2(x0))
> >
> > prefetch.w is not an R-type instruction, it's an S-type. While the bit
> > shifts are the same, the names are different. We need to add S-type
> > names while defining this instruction.
>
> That is correct, it is supposed to look like a store instruction (S-type),
> even though documentation don't explicitly state that.
>
> Even though it works fine with the R-type definition, code documentation
> would be wrong, and future changes could break it.
>
> > Then, this define would be
> >
> > #define PREFETCH_w(base, imm) \
I should have suggested 'offset' instead of 'imm' for the second parameter
name.
> > INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
> > RS1(base), __RS2(3))
>
> s/OPCODE_OP_IMM/OPCODE_PREFETCH
> 0x4 vs 0x13
There's no major opcode named "PREFETCH" and the spec says that the major
opcode used for prefetch instructions is OP-IMM. That's why we want to
name this OPCODE_OP_IMM. I'm not sure where the 0x4 you're referring to
comes from. A 32-bit instruction has the lowest two bits set (figure 1.1
of the unpriv spec) and table 27.1 of the unpriv spec shows OP-IMM is
0b00100xx, so we have 0b0010011. Keeping the naming of the opcode macros
consistent with the spec also keeps them consistent with the .insn
directive where we could even use the names directly, i.e.
.insn s OP_IMM, 6, x3, 0(a0)
> >
> > When the assembler as insn_r I hope it will validate that
I meant insn_s here, which would be the macro for '.insn s'
> > (imm & 0xfe0) == imm
I played with it. It won't do what we want for prefetch, only
what works for s-type instructions in general, i.e. it allows
+/-2047 offsets and fails for everything else. That's good enough.
We can just mask off the low 5 bits here in our macro
#define PREFETCH_w(base, offset) \
INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5((offset) & ~0x1f), \
__IMM_4_0(0), RS1(base), __RS2(3))
Thanks,
drew
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-15 11:07 ` Andrew Jones
0 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-15 11:07 UTC (permalink / raw)
To: Leonardo Bras
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, conor.dooley, xiaoguang.xing, bjorn, alexghiti,
keescook, greentime.hu, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 05:22:26AM -0300, Leonardo Bras wrote:
> On Thu, Sep 14, 2023 at 03:47:59PM +0200, Andrew Jones wrote:
> > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
...
> > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > index 6960beb75f32..dc590d331894 100644
> > > --- a/arch/riscv/include/asm/insn-def.h
> > > +++ b/arch/riscv/include/asm/insn-def.h
> > > @@ -134,6 +134,7 @@
> > >
> > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> >
> > This should be named RV_OPCODE_OP_IMM and be placed in
> > numerical order with the others, i.e. above SYSTEM.
> >
> > >
> > > #define HFENCE_VVMA(vaddr, asid) \
> > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > @@ -196,4 +197,8 @@
> > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > RS1(base), SIMM12(4))
> > >
> > > +#define CBO_prefetchw(base) \
> >
> > Please name this 'PREFETCH_w' and it should take an immediate parameter,
> > even if we intend to pass 0 for it.
>
> It makes sense.
>
> The mnemonic in the previously mentioned documentation is:
>
> prefetch.w offset(base)
>
> So yeah, makes sense to have both offset and base as parameters for
> CBO_prefetchw (or PREFETCH_w, I have no strong preference).
I have a strong preference :-)
PREFETCH_w is consistent with the naming we already have for e.g.
cbo.clean, which is CBO_clean. The instruction we're picking a name
for now is prefetch.w, not cbo.prefetchw.
>
> >
> > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > + RD(x0), RS1(base), RS2(x0))
> >
> > prefetch.w is not an R-type instruction, it's an S-type. While the bit
> > shifts are the same, the names are different. We need to add S-type
> > names while defining this instruction.
>
> That is correct, it is supposed to look like a store instruction (S-type),
> even though documentation don't explicitly state that.
>
> Even though it works fine with the R-type definition, code documentation
> would be wrong, and future changes could break it.
>
> > Then, this define would be
> >
> > #define PREFETCH_w(base, imm) \
I should have suggested 'offset' instead of 'imm' for the second parameter
name.
> > INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
> > RS1(base), __RS2(3))
>
> s/OPCODE_OP_IMM/OPCODE_PREFETCH
> 0x4 vs 0x13
There's no major opcode named "PREFETCH" and the spec says that the major
opcode used for prefetch instructions is OP-IMM. That's why we want to
name this OPCODE_OP_IMM. I'm not sure where the 0x4 you're referring to
comes from. A 32-bit instruction has the lowest two bits set (figure 1.1
of the unpriv spec) and table 27.1 of the unpriv spec shows OP-IMM is
0b00100xx, so we have 0b0010011. Keeping the naming of the opcode macros
consistent with the spec also keeps them consistent with the .insn
directive where we could even use the names directly, i.e.
.insn s OP_IMM, 6, x3, 0(a0)
> >
> > When the assembler as insn_r I hope it will validate that
I meant insn_s here, which would be the macro for '.insn s'
> > (imm & 0xfe0) == imm
I played with it. It won't do what we want for prefetch, only
what works for s-type instructions in general, i.e. it allows
+/-2047 offsets and fails for everything else. That's good enough.
We can just mask off the low 5 bits here in our macro
#define PREFETCH_w(base, offset) \
INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5((offset) & ~0x1f), \
__IMM_4_0(0), RS1(base), __RS2(3))
Thanks,
drew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-15 11:07 ` Andrew Jones
@ 2023-09-15 11:26 ` Conor Dooley
-1 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-15 11:26 UTC (permalink / raw)
To: Andrew Jones
Cc: Leonardo Bras, guoren, paul.walmsley, anup, peterz, mingo, will,
palmer, longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
[-- Attachment #1: Type: text/plain, Size: 4601 bytes --]
On Fri, Sep 15, 2023 at 01:07:40PM +0200, Andrew Jones wrote:
> On Fri, Sep 15, 2023 at 05:22:26AM -0300, Leonardo Bras wrote:
> > On Thu, Sep 14, 2023 at 03:47:59PM +0200, Andrew Jones wrote:
> > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> ...
> > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > index 6960beb75f32..dc590d331894 100644
> > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > @@ -134,6 +134,7 @@
> > > >
> > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > >
> > > This should be named RV_OPCODE_OP_IMM and be placed in
> > > numerical order with the others, i.e. above SYSTEM.
> > >
> > > >
> > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > @@ -196,4 +197,8 @@
> > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > RS1(base), SIMM12(4))
> > > >
> > > > +#define CBO_prefetchw(base) \
> > >
> > > Please name this 'PREFETCH_w' and it should take an immediate parameter,
> > > even if we intend to pass 0 for it.
> >
> > It makes sense.
> >
> > The mnemonic in the previously mentioned documentation is:
> >
> > prefetch.w offset(base)
> >
> > So yeah, makes sense to have both offset and base as parameters for
> > CBO_prefetchw (or PREFETCH_w, I have no strong preference).
>
> I have a strong preference :-)
>
> PREFETCH_w is consistent with the naming we already have for e.g.
> cbo.clean, which is CBO_clean. The instruction we're picking a name
> for now is prefetch.w, not cbo.prefetchw.
btw, the CBO_foo stuff was named that way as we were using them in
alternatives originally as an argument, that manifested as:
"cbo." __stringify(_op) " (a0)\n\t"
That was later changed to
CBO_##_op(a0)
but the then un-needed (AFAICT) capitalisation was kept to avoid
touching the callsites of the alternative. Maybe you remember better
than I do drew, since the idea was yours & I forgot I even wrote that
pattch.
If this isn't being used in a similar manner, then the w has no reason
to be in the odd lowercase form.
Cheers,
Conor.
>
> >
> > >
> > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > + RD(x0), RS1(base), RS2(x0))
> > >
> > > prefetch.w is not an R-type instruction, it's an S-type. While the bit
> > > shifts are the same, the names are different. We need to add S-type
> > > names while defining this instruction.
> >
> > That is correct, it is supposed to look like a store instruction (S-type),
> > even though documentation don't explicitly state that.
> >
> > Even though it works fine with the R-type definition, code documentation
> > would be wrong, and future changes could break it.
> >
> > > Then, this define would be
> > >
> > > #define PREFETCH_w(base, imm) \
>
> I should have suggested 'offset' instead of 'imm' for the second parameter
> name.
>
> > > INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
> > > RS1(base), __RS2(3))
> >
> > s/OPCODE_OP_IMM/OPCODE_PREFETCH
> > 0x4 vs 0x13
>
> There's no major opcode named "PREFETCH" and the spec says that the major
> opcode used for prefetch instructions is OP-IMM. That's why we want to
> name this OPCODE_OP_IMM. I'm not sure where the 0x4 you're referring to
> comes from. A 32-bit instruction has the lowest two bits set (figure 1.1
> of the unpriv spec) and table 27.1 of the unpriv spec shows OP-IMM is
> 0b00100xx, so we have 0b0010011. Keeping the naming of the opcode macros
> consistent with the spec also keeps them consistent with the .insn
> directive where we could even use the names directly, i.e.
>
> .insn s OP_IMM, 6, x3, 0(a0)
>
> > >
> > > When the assembler as insn_r I hope it will validate that
>
> I meant insn_s here, which would be the macro for '.insn s'
>
> > > (imm & 0xfe0) == imm
>
> I played with it. It won't do what we want for prefetch, only
> what works for s-type instructions in general, i.e. it allows
> +/-2047 offsets and fails for everything else. That's good enough.
> We can just mask off the low 5 bits here in our macro
>
> #define PREFETCH_w(base, offset) \
> INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5((offset) & ~0x1f), \
> __IMM_4_0(0), RS1(base), __RS2(3))
>
> Thanks,
> drew
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-15 11:26 ` Conor Dooley
0 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-15 11:26 UTC (permalink / raw)
To: Andrew Jones
Cc: Leonardo Bras, guoren, paul.walmsley, anup, peterz, mingo, will,
palmer, longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
[-- Attachment #1.1: Type: text/plain, Size: 4601 bytes --]
On Fri, Sep 15, 2023 at 01:07:40PM +0200, Andrew Jones wrote:
> On Fri, Sep 15, 2023 at 05:22:26AM -0300, Leonardo Bras wrote:
> > On Thu, Sep 14, 2023 at 03:47:59PM +0200, Andrew Jones wrote:
> > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> ...
> > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > index 6960beb75f32..dc590d331894 100644
> > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > @@ -134,6 +134,7 @@
> > > >
> > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > >
> > > This should be named RV_OPCODE_OP_IMM and be placed in
> > > numerical order with the others, i.e. above SYSTEM.
> > >
> > > >
> > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > @@ -196,4 +197,8 @@
> > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > RS1(base), SIMM12(4))
> > > >
> > > > +#define CBO_prefetchw(base) \
> > >
> > > Please name this 'PREFETCH_w' and it should take an immediate parameter,
> > > even if we intend to pass 0 for it.
> >
> > It makes sense.
> >
> > The mnemonic in the previously mentioned documentation is:
> >
> > prefetch.w offset(base)
> >
> > So yeah, makes sense to have both offset and base as parameters for
> > CBO_prefetchw (or PREFETCH_w, I have no strong preference).
>
> I have a strong preference :-)
>
> PREFETCH_w is consistent with the naming we already have for e.g.
> cbo.clean, which is CBO_clean. The instruction we're picking a name
> for now is prefetch.w, not cbo.prefetchw.
btw, the CBO_foo stuff was named that way as we were using them in
alternatives originally as an argument, that manifested as:
"cbo." __stringify(_op) " (a0)\n\t"
That was later changed to
CBO_##_op(a0)
but the then un-needed (AFAICT) capitalisation was kept to avoid
touching the callsites of the alternative. Maybe you remember better
than I do drew, since the idea was yours & I forgot I even wrote that
pattch.
If this isn't being used in a similar manner, then the w has no reason
to be in the odd lowercase form.
Cheers,
Conor.
>
> >
> > >
> > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > + RD(x0), RS1(base), RS2(x0))
> > >
> > > prefetch.w is not an R-type instruction, it's an S-type. While the bit
> > > shifts are the same, the names are different. We need to add S-type
> > > names while defining this instruction.
> >
> > That is correct, it is supposed to look like a store instruction (S-type),
> > even though documentation don't explicitly state that.
> >
> > Even though it works fine with the R-type definition, code documentation
> > would be wrong, and future changes could break it.
> >
> > > Then, this define would be
> > >
> > > #define PREFETCH_w(base, imm) \
>
> I should have suggested 'offset' instead of 'imm' for the second parameter
> name.
>
> > > INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
> > > RS1(base), __RS2(3))
> >
> > s/OPCODE_OP_IMM/OPCODE_PREFETCH
> > 0x4 vs 0x13
>
> There's no major opcode named "PREFETCH" and the spec says that the major
> opcode used for prefetch instructions is OP-IMM. That's why we want to
> name this OPCODE_OP_IMM. I'm not sure where the 0x4 you're referring to
> comes from. A 32-bit instruction has the lowest two bits set (figure 1.1
> of the unpriv spec) and table 27.1 of the unpriv spec shows OP-IMM is
> 0b00100xx, so we have 0b0010011. Keeping the naming of the opcode macros
> consistent with the spec also keeps them consistent with the .insn
> directive where we could even use the names directly, i.e.
>
> .insn s OP_IMM, 6, x3, 0(a0)
>
> > >
> > > When the assembler as insn_r I hope it will validate that
>
> I meant insn_s here, which would be the macro for '.insn s'
>
> > > (imm & 0xfe0) == imm
>
> I played with it. It won't do what we want for prefetch, only
> what works for s-type instructions in general, i.e. it allows
> +/-2047 offsets and fails for everything else. That's good enough.
> We can just mask off the low 5 bits here in our macro
>
> #define PREFETCH_w(base, offset) \
> INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5((offset) & ~0x1f), \
> __IMM_4_0(0), RS1(base), __RS2(3))
>
> Thanks,
> drew
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-14 14:47 ` Andrew Jones
@ 2023-09-15 11:37 ` Conor Dooley
-1 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-15 11:37 UTC (permalink / raw)
To: Andrew Jones
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
[-- Attachment #1: Type: text/plain, Size: 4091 bytes --]
Yo,
On Thu, Sep 14, 2023 at 04:47:18PM +0200, Andrew Jones wrote:
> On Thu, Sep 14, 2023 at 04:25:53PM +0200, Andrew Jones wrote:
> > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Cache-block prefetch instructions are HINTs to the hardware to
> > > indicate that software intends to perform a particular type of
> > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > improve the arch_xchg for qspinlock xchg_tail.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > arch/riscv/include/asm/hwcap.h | 1 +
> > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > arch/riscv/kernel/cpufeature.c | 1 +
> > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index e9ae6fa232c3..2c346fe169c1 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > >
> > > If you don't know what to do here, say Y.
> > >
> > > +config RISCV_ISA_ZICBOP
> >
> > Even if we're not concerned with looping over blocks yet, I think we
> > should introduce zicbop block size DT parsing at the same time we bring
> > zicbop support to the kernel (it's just more copy+paste from zicbom and
> > zicboz). It's a bit annoying that the CMO spec doesn't state that block
> > sizes should be the same for m/z/p. And, the fact that m/z/p are all
> > separate extensions leads us to needing to parse block sizes for all
> > three, despite the fact that in practice they'll probably be the same.
>
> Although, I saw on a different mailing list that Andrei Warkentin
> interpreted section 2.7 "Software Discovery" of the spec, which states
>
> """
> The initial set of CMO extensions requires the following information to be
> discovered by software:
>
> * The size of the cache block for management and prefetch instructions
> * The size of the cache block for zero instructions
> * CBIE support at each privilege level
>
> Other general cache characteristics may also be specified in the discovery
> mechanism.
> """
>
> as management and prefetch having the same block size and only zero
> potentially having a different size. That looks like a reasonable
> interpretation to me, too.
TBH, I don't really care what ambiguous wording the spec has used, we
have the opportunity to make better decisions if we please. I hate the
fact that the specs are often not abundantly clear about things like this.
> So, we could maybe proceed with assuming we
> can use zicbom_block_size for prefetch, for now. If a platform comes along
> that interpreted the spec differently, requiring prefetch block size to
> be specified separately, then we'll cross that bridge when we get there.
That said, I think I suggested originally having the zicboz stuff default
to the zicbom size too, so I'd be happy with prefetch stuff working
exclusively that way until someone comes along looking for different sizes.
The binding should be updated though since
riscv,cbom-block-size:
$ref: /schemas/types.yaml#/definitions/uint32
description:
The blocksize in bytes for the Zicbom cache operations.
would no longer be a complete description.
While thinking about new wording though, it feels really clunky to describe
it like:
The block size in bytes for the Zicbom cache operations, Zicbop
cache operations will default to this block size where not
explicitly defined.
since there's then no way to actually define the block size if it is
different. Unless you've got some magic wording, I'd rather document
riscv,cbop-block-size, even if we are going to use riscv,cbom-block-size
as the default.
Cheers,
Conor.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-15 11:37 ` Conor Dooley
0 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-15 11:37 UTC (permalink / raw)
To: Andrew Jones
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
[-- Attachment #1.1: Type: text/plain, Size: 4091 bytes --]
Yo,
On Thu, Sep 14, 2023 at 04:47:18PM +0200, Andrew Jones wrote:
> On Thu, Sep 14, 2023 at 04:25:53PM +0200, Andrew Jones wrote:
> > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Cache-block prefetch instructions are HINTs to the hardware to
> > > indicate that software intends to perform a particular type of
> > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > improve the arch_xchg for qspinlock xchg_tail.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > arch/riscv/include/asm/hwcap.h | 1 +
> > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > arch/riscv/kernel/cpufeature.c | 1 +
> > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index e9ae6fa232c3..2c346fe169c1 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > >
> > > If you don't know what to do here, say Y.
> > >
> > > +config RISCV_ISA_ZICBOP
> >
> > Even if we're not concerned with looping over blocks yet, I think we
> > should introduce zicbop block size DT parsing at the same time we bring
> > zicbop support to the kernel (it's just more copy+paste from zicbom and
> > zicboz). It's a bit annoying that the CMO spec doesn't state that block
> > sizes should be the same for m/z/p. And, the fact that m/z/p are all
> > separate extensions leads us to needing to parse block sizes for all
> > three, despite the fact that in practice they'll probably be the same.
>
> Although, I saw on a different mailing list that Andrei Warkentin
> interpreted section 2.7 "Software Discovery" of the spec, which states
>
> """
> The initial set of CMO extensions requires the following information to be
> discovered by software:
>
> * The size of the cache block for management and prefetch instructions
> * The size of the cache block for zero instructions
> * CBIE support at each privilege level
>
> Other general cache characteristics may also be specified in the discovery
> mechanism.
> """
>
> as management and prefetch having the same block size and only zero
> potentially having a different size. That looks like a reasonable
> interpretation to me, too.
TBH, I don't really care what ambiguous wording the spec has used, we
have the opportunity to make better decisions if we please. I hate the
fact that the specs are often not abundantly clear about things like this.
> So, we could maybe proceed with assuming we
> can use zicbom_block_size for prefetch, for now. If a platform comes along
> that interpreted the spec differently, requiring prefetch block size to
> be specified separately, then we'll cross that bridge when we get there.
That said, I think I suggested originally having the zicboz stuff default
to the zicbom size too, so I'd be happy with prefetch stuff working
exclusively that way until someone comes along looking for different sizes.
The binding should be updated though since
riscv,cbom-block-size:
$ref: /schemas/types.yaml#/definitions/uint32
description:
The blocksize in bytes for the Zicbom cache operations.
would no longer be a complete description.
While thinking about new wording though, it feels really clunky to describe
it like:
The block size in bytes for the Zicbom cache operations, Zicbop
cache operations will default to this block size where not
explicitly defined.
since there's then no way to actually define the block size if it is
different. Unless you've got some magic wording, I'd rather document
riscv,cbop-block-size, even if we are going to use riscv,cbom-block-size
as the default.
Cheers,
Conor.
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-15 11:37 ` Conor Dooley
@ 2023-09-15 12:14 ` Andrew Jones
-1 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-15 12:14 UTC (permalink / raw)
To: Conor Dooley
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 12:37:50PM +0100, Conor Dooley wrote:
> Yo,
>
> On Thu, Sep 14, 2023 at 04:47:18PM +0200, Andrew Jones wrote:
> > On Thu, Sep 14, 2023 at 04:25:53PM +0200, Andrew Jones wrote:
> > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > Cache-block prefetch instructions are HINTs to the hardware to
> > > > indicate that software intends to perform a particular type of
> > > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > > improve the arch_xchg for qspinlock xchg_tail.
> > > >
> > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > ---
> > > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > > arch/riscv/include/asm/hwcap.h | 1 +
> > > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > > arch/riscv/kernel/cpufeature.c | 1 +
> > > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > index e9ae6fa232c3..2c346fe169c1 100644
> > > > --- a/arch/riscv/Kconfig
> > > > +++ b/arch/riscv/Kconfig
> > > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > > >
> > > > If you don't know what to do here, say Y.
> > > >
> > > > +config RISCV_ISA_ZICBOP
> > >
> > > Even if we're not concerned with looping over blocks yet, I think we
> > > should introduce zicbop block size DT parsing at the same time we bring
> > > zicbop support to the kernel (it's just more copy+paste from zicbom and
> > > zicboz). It's a bit annoying that the CMO spec doesn't state that block
> > > sizes should be the same for m/z/p. And, the fact that m/z/p are all
> > > separate extensions leads us to needing to parse block sizes for all
> > > three, despite the fact that in practice they'll probably be the same.
> >
> > Although, I saw on a different mailing list that Andrei Warkentin
> > interpreted section 2.7 "Software Discovery" of the spec, which states
> >
> > """
> > The initial set of CMO extensions requires the following information to be
> > discovered by software:
> >
> > * The size of the cache block for management and prefetch instructions
> > * The size of the cache block for zero instructions
> > * CBIE support at each privilege level
> >
> > Other general cache characteristics may also be specified in the discovery
> > mechanism.
> > """
> >
> > as management and prefetch having the same block size and only zero
> > potentially having a different size. That looks like a reasonable
> > interpretation to me, too.
>
> TBH, I don't really care what ambiguous wording the spec has used, we
> have the opportunity to make better decisions if we please. I hate the
> fact that the specs are often not abundantly clear about things like this.
>
> > So, we could maybe proceed with assuming we
> > can use zicbom_block_size for prefetch, for now. If a platform comes along
> > that interpreted the spec differently, requiring prefetch block size to
> > be specified separately, then we'll cross that bridge when we get there.
>
> That said, I think I suggested originally having the zicboz stuff default
> to the zicbom size too, so I'd be happy with prefetch stuff working
> exclusively that way until someone comes along looking for different sizes.
> The binding should be updated though since
>
> riscv,cbom-block-size:
> $ref: /schemas/types.yaml#/definitions/uint32
> description:
> The blocksize in bytes for the Zicbom cache operations.
>
> would no longer be a complete description.
>
> While thinking about new wording though, it feels really clunky to describe
> it like:
> The block size in bytes for the Zicbom cache operations, Zicbop
> cache operations will default to this block size where not
> explicitly defined.
>
> since there's then no way to actually define the block size if it is
> different. Unless you've got some magic wording, I'd rather document
> riscv,cbop-block-size, even if we are going to use riscv,cbom-block-size
> as the default.
>
Sounds good to me, but if it's documented, then we should probably
implement its parsing. Then, at that point, I wonder if it makes sense to
have the fallback at all, or if it's not better just to require all the
DTs to be explicit (even if redundant).
Thanks,
drew
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-15 12:14 ` Andrew Jones
0 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-15 12:14 UTC (permalink / raw)
To: Conor Dooley
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 12:37:50PM +0100, Conor Dooley wrote:
> Yo,
>
> On Thu, Sep 14, 2023 at 04:47:18PM +0200, Andrew Jones wrote:
> > On Thu, Sep 14, 2023 at 04:25:53PM +0200, Andrew Jones wrote:
> > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > Cache-block prefetch instructions are HINTs to the hardware to
> > > > indicate that software intends to perform a particular type of
> > > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > > improve the arch_xchg for qspinlock xchg_tail.
> > > >
> > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > ---
> > > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > > arch/riscv/include/asm/hwcap.h | 1 +
> > > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > > arch/riscv/kernel/cpufeature.c | 1 +
> > > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > index e9ae6fa232c3..2c346fe169c1 100644
> > > > --- a/arch/riscv/Kconfig
> > > > +++ b/arch/riscv/Kconfig
> > > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > > >
> > > > If you don't know what to do here, say Y.
> > > >
> > > > +config RISCV_ISA_ZICBOP
> > >
> > > Even if we're not concerned with looping over blocks yet, I think we
> > > should introduce zicbop block size DT parsing at the same time we bring
> > > zicbop support to the kernel (it's just more copy+paste from zicbom and
> > > zicboz). It's a bit annoying that the CMO spec doesn't state that block
> > > sizes should be the same for m/z/p. And, the fact that m/z/p are all
> > > separate extensions leads us to needing to parse block sizes for all
> > > three, despite the fact that in practice they'll probably be the same.
> >
> > Although, I saw on a different mailing list that Andrei Warkentin
> > interpreted section 2.7 "Software Discovery" of the spec, which states
> >
> > """
> > The initial set of CMO extensions requires the following information to be
> > discovered by software:
> >
> > * The size of the cache block for management and prefetch instructions
> > * The size of the cache block for zero instructions
> > * CBIE support at each privilege level
> >
> > Other general cache characteristics may also be specified in the discovery
> > mechanism.
> > """
> >
> > as management and prefetch having the same block size and only zero
> > potentially having a different size. That looks like a reasonable
> > interpretation to me, too.
>
> TBH, I don't really care what ambiguous wording the spec has used, we
> have the opportunity to make better decisions if we please. I hate the
> fact that the specs are often not abundantly clear about things like this.
>
> > So, we could maybe proceed with assuming we
> > can use zicbom_block_size for prefetch, for now. If a platform comes along
> > that interpreted the spec differently, requiring prefetch block size to
> > be specified separately, then we'll cross that bridge when we get there.
>
> That said, I think I suggested originally having the zicboz stuff default
> to the zicbom size too, so I'd be happy with prefetch stuff working
> exclusively that way until someone comes along looking for different sizes.
> The binding should be updated though since
>
> riscv,cbom-block-size:
> $ref: /schemas/types.yaml#/definitions/uint32
> description:
> The blocksize in bytes for the Zicbom cache operations.
>
> would no longer be a complete description.
>
> While thinking about new wording though, it feels really clunky to describe
> it like:
> The block size in bytes for the Zicbom cache operations, Zicbop
> cache operations will default to this block size where not
> explicitly defined.
>
> since there's then no way to actually define the block size if it is
> different. Unless you've got some magic wording, I'd rather document
> riscv,cbop-block-size, even if we are going to use riscv,cbom-block-size
> as the default.
>
Sounds good to me, but if it's documented, then we should probably
implement its parsing. Then, at that point, I wonder if it makes sense to
have the fallback at all, or if it's not better just to require all the
DTs to be explicit (even if redundant).
Thanks,
drew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-15 11:26 ` Conor Dooley
@ 2023-09-15 12:22 ` Andrew Jones
-1 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-15 12:22 UTC (permalink / raw)
To: Conor Dooley
Cc: Leonardo Bras, guoren, paul.walmsley, anup, peterz, mingo, will,
palmer, longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 12:26:20PM +0100, Conor Dooley wrote:
> On Fri, Sep 15, 2023 at 01:07:40PM +0200, Andrew Jones wrote:
> > On Fri, Sep 15, 2023 at 05:22:26AM -0300, Leonardo Bras wrote:
> > > On Thu, Sep 14, 2023 at 03:47:59PM +0200, Andrew Jones wrote:
> > > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > ...
> > > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > > index 6960beb75f32..dc590d331894 100644
> > > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > > @@ -134,6 +134,7 @@
> > > > >
> > > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > > >
> > > > This should be named RV_OPCODE_OP_IMM and be placed in
> > > > numerical order with the others, i.e. above SYSTEM.
> > > >
> > > > >
> > > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > > @@ -196,4 +197,8 @@
> > > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > > RS1(base), SIMM12(4))
> > > > >
> > > > > +#define CBO_prefetchw(base) \
> > > >
> > > > Please name this 'PREFETCH_w' and it should take an immediate parameter,
> > > > even if we intend to pass 0 for it.
> > >
> > > It makes sense.
> > >
> > > The mnemonic in the previously mentioned documentation is:
> > >
> > > prefetch.w offset(base)
> > >
> > > So yeah, makes sense to have both offset and base as parameters for
> > > CBO_prefetchw (or PREFETCH_w, I have no strong preference).
> >
> > I have a strong preference :-)
> >
> > PREFETCH_w is consistent with the naming we already have for e.g.
> > cbo.clean, which is CBO_clean. The instruction we're picking a name
> > for now is prefetch.w, not cbo.prefetchw.
>
> btw, the CBO_foo stuff was named that way as we were using them in
> alternatives originally as an argument, that manifested as:
> "cbo." __stringify(_op) " (a0)\n\t"
> That was later changed to
> CBO_##_op(a0)
> but the then un-needed (AFAICT) capitalisation was kept to avoid
> touching the callsites of the alternative. Maybe you remember better
> than I do drew, since the idea was yours & I forgot I even wrote that
> pattch.
And I forgot anything I may have suggested about it :-)
> If this isn't being used in a similar manner, then the w has no reason
> to be in the odd lowercase form.
Other than to be consistent... However, the CBO_* instructions are not
consistent with the rest of macros. If we don't need lowercase for any
reason, then my preference would be to bite the bullet and change all the
callsites of CBO_* macros and then introduce this new instruction as
PREFETCH_W
Thanks,
drew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-15 12:22 ` Andrew Jones
0 siblings, 0 replies; 215+ messages in thread
From: Andrew Jones @ 2023-09-15 12:22 UTC (permalink / raw)
To: Conor Dooley
Cc: Leonardo Bras, guoren, paul.walmsley, anup, peterz, mingo, will,
palmer, longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 12:26:20PM +0100, Conor Dooley wrote:
> On Fri, Sep 15, 2023 at 01:07:40PM +0200, Andrew Jones wrote:
> > On Fri, Sep 15, 2023 at 05:22:26AM -0300, Leonardo Bras wrote:
> > > On Thu, Sep 14, 2023 at 03:47:59PM +0200, Andrew Jones wrote:
> > > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > ...
> > > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > > index 6960beb75f32..dc590d331894 100644
> > > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > > @@ -134,6 +134,7 @@
> > > > >
> > > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > > >
> > > > This should be named RV_OPCODE_OP_IMM and be placed in
> > > > numerical order with the others, i.e. above SYSTEM.
> > > >
> > > > >
> > > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > > @@ -196,4 +197,8 @@
> > > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > > RS1(base), SIMM12(4))
> > > > >
> > > > > +#define CBO_prefetchw(base) \
> > > >
> > > > Please name this 'PREFETCH_w' and it should take an immediate parameter,
> > > > even if we intend to pass 0 for it.
> > >
> > > It makes sense.
> > >
> > > The mnemonic in the previously mentioned documentation is:
> > >
> > > prefetch.w offset(base)
> > >
> > > So yeah, makes sense to have both offset and base as parameters for
> > > CBO_prefetchw (or PREFETCH_w, I have no strong preference).
> >
> > I have a strong preference :-)
> >
> > PREFETCH_w is consistent with the naming we already have for e.g.
> > cbo.clean, which is CBO_clean. The instruction we're picking a name
> > for now is prefetch.w, not cbo.prefetchw.
>
> btw, the CBO_foo stuff was named that way as we were using them in
> alternatives originally as an argument, that manifested as:
> "cbo." __stringify(_op) " (a0)\n\t"
> That was later changed to
> CBO_##_op(a0)
> but the then un-needed (AFAICT) capitalisation was kept to avoid
> touching the callsites of the alternative. Maybe you remember better
> than I do drew, since the idea was yours & I forgot I even wrote that
> pattch.
And I forgot anything I may have suggested about it :-)
> If this isn't being used in a similar manner, then the w has no reason
> to be in the odd lowercase form.
Other than to be consistent... However, the CBO_* instructions are not
consistent with the rest of macros. If we don't need lowercase for any
reason, then my preference would be to bite the bullet and change all the
callsites of CBO_* macros and then introduce this new instruction as
PREFETCH_W
Thanks,
drew
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-13 8:49 ` Leonardo Bras
@ 2023-09-15 12:36 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-15 12:36 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Cache-block prefetch instructions are HINTs to the hardware to
> > indicate that software intends to perform a particular type of
> > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > improve the arch_xchg for qspinlock xchg_tail.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig | 15 +++++++++++++++
> > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > arch/riscv/include/asm/hwcap.h | 1 +
> > arch/riscv/include/asm/insn-def.h | 5 +++++
> > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > arch/riscv/kernel/cpufeature.c | 1 +
> > 6 files changed, 38 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index e9ae6fa232c3..2c346fe169c1 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> >
> > If you don't know what to do here, say Y.
> >
> > +config RISCV_ISA_ZICBOP
> > + bool "Zicbop extension support for cache block prefetch"
> > + depends on MMU
> > + depends on RISCV_ALTERNATIVE
> > + default y
> > + help
> > + Adds support to dynamically detect the presence of the ZICBOP
> > + extension (Cache Block Prefetch Operations) and enable its
> > + usage.
> > +
> > + The Zicbop extension can be used to prefetch cache block for
> > + read/write/instruction fetch.
> > +
> > + If you don't know what to do here, say Y.
> > +
> > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > bool
> > default y
> > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > index 702725727671..56eff7a9d2d2 100644
> > --- a/arch/riscv/include/asm/cmpxchg.h
> > +++ b/arch/riscv/include/asm/cmpxchg.h
> > @@ -11,6 +11,7 @@
> >
> > #include <asm/barrier.h>
> > #include <asm/fence.h>
> > +#include <asm/processor.h>
> >
> > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > ({ \
> > @@ -25,6 +26,7 @@
> > \
> > __asm__ __volatile__ ( \
> > prepend \
> > + PREFETCHW_ASM(%5) \
> > "0: lr.w %0, %2\n" \
> > " and %1, %0, %z4\n" \
> > " or %1, %1, %z3\n" \
> > @@ -32,7 +34,7 @@
> > " bnez %1, 0b\n" \
> > append \
> > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > - : "rJ" (__newx), "rJ" (~__mask) \
> > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > : "memory"); \
> > \
> > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > index b7b58258f6c7..78b7b8b53778 100644
> > --- a/arch/riscv/include/asm/hwcap.h
> > +++ b/arch/riscv/include/asm/hwcap.h
> > @@ -58,6 +58,7 @@
> > #define RISCV_ISA_EXT_ZICSR 40
> > #define RISCV_ISA_EXT_ZIFENCEI 41
> > #define RISCV_ISA_EXT_ZIHPM 42
> > +#define RISCV_ISA_EXT_ZICBOP 43
> >
> > #define RISCV_ISA_EXT_MAX 64
> >
> > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > index 6960beb75f32..dc590d331894 100644
> > --- a/arch/riscv/include/asm/insn-def.h
> > +++ b/arch/riscv/include/asm/insn-def.h
> > @@ -134,6 +134,7 @@
> >
> > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> >
> > #define HFENCE_VVMA(vaddr, asid) \
> > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > @@ -196,4 +197,8 @@
> > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > RS1(base), SIMM12(4))
> >
> > +#define CBO_prefetchw(base) \
> > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > + RD(x0), RS1(base), RS2(x0))
> > +
>
> I understand that here you create the instruction via bitfield, following
> the ISA, and this enables using instructions not available on the
> toolchain.
>
> It took me some time to find the document with this instruction, so please
> add this to the commit msg:
>
> https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> Page 23.
>
> IIUC, the instruction is "prefetch.w".
>
> Maybe I am missing something, but in the document the rs2 field
> (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
>
> rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
>
> Is the above correct, or am I missing something?
Oh, you are right. My fault, thx for pointing out. It should be:
+ INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
+ RD(x0), RS1(base), RS2(x3))
>
>
> Thanks!
> Leo
>
> > #endif /* __ASM_INSN_DEF_H */
> > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > index de9da852f78d..7ad3a24212e8 100644
> > --- a/arch/riscv/include/asm/processor.h
> > +++ b/arch/riscv/include/asm/processor.h
> > @@ -12,6 +12,8 @@
> > #include <vdso/processor.h>
> >
> > #include <asm/ptrace.h>
> > +#include <asm/insn-def.h>
> > +#include <asm/hwcap.h>
> >
> > #ifdef CONFIG_64BIT
> > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> >
> > +#define ARCH_HAS_PREFETCHW
> > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > + CBO_prefetchw(base), \
> > + 0, \
> > + RISCV_ISA_EXT_ZICBOP, \
> > + CONFIG_RISCV_ISA_ZICBOP)
> > +static inline void prefetchw(const void *ptr)
> > +{
> > + asm volatile(PREFETCHW_ASM(%0)
> > + : : "r" (ptr) : "memory");
> > +}
> >
> > /* Do necessary setup to start up a newly executed thread. */
> > extern void start_thread(struct pt_regs *regs,
> > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > index ef7b4fd9e876..e0b897db0b97 100644
> > --- a/arch/riscv/kernel/cpufeature.c
> > +++ b/arch/riscv/kernel/cpufeature.c
> > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > --
> > 2.36.1
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-15 12:36 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-15 12:36 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Cache-block prefetch instructions are HINTs to the hardware to
> > indicate that software intends to perform a particular type of
> > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > improve the arch_xchg for qspinlock xchg_tail.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig | 15 +++++++++++++++
> > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > arch/riscv/include/asm/hwcap.h | 1 +
> > arch/riscv/include/asm/insn-def.h | 5 +++++
> > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > arch/riscv/kernel/cpufeature.c | 1 +
> > 6 files changed, 38 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index e9ae6fa232c3..2c346fe169c1 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> >
> > If you don't know what to do here, say Y.
> >
> > +config RISCV_ISA_ZICBOP
> > + bool "Zicbop extension support for cache block prefetch"
> > + depends on MMU
> > + depends on RISCV_ALTERNATIVE
> > + default y
> > + help
> > + Adds support to dynamically detect the presence of the ZICBOP
> > + extension (Cache Block Prefetch Operations) and enable its
> > + usage.
> > +
> > + The Zicbop extension can be used to prefetch cache block for
> > + read/write/instruction fetch.
> > +
> > + If you don't know what to do here, say Y.
> > +
> > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > bool
> > default y
> > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > index 702725727671..56eff7a9d2d2 100644
> > --- a/arch/riscv/include/asm/cmpxchg.h
> > +++ b/arch/riscv/include/asm/cmpxchg.h
> > @@ -11,6 +11,7 @@
> >
> > #include <asm/barrier.h>
> > #include <asm/fence.h>
> > +#include <asm/processor.h>
> >
> > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > ({ \
> > @@ -25,6 +26,7 @@
> > \
> > __asm__ __volatile__ ( \
> > prepend \
> > + PREFETCHW_ASM(%5) \
> > "0: lr.w %0, %2\n" \
> > " and %1, %0, %z4\n" \
> > " or %1, %1, %z3\n" \
> > @@ -32,7 +34,7 @@
> > " bnez %1, 0b\n" \
> > append \
> > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > - : "rJ" (__newx), "rJ" (~__mask) \
> > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > : "memory"); \
> > \
> > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > index b7b58258f6c7..78b7b8b53778 100644
> > --- a/arch/riscv/include/asm/hwcap.h
> > +++ b/arch/riscv/include/asm/hwcap.h
> > @@ -58,6 +58,7 @@
> > #define RISCV_ISA_EXT_ZICSR 40
> > #define RISCV_ISA_EXT_ZIFENCEI 41
> > #define RISCV_ISA_EXT_ZIHPM 42
> > +#define RISCV_ISA_EXT_ZICBOP 43
> >
> > #define RISCV_ISA_EXT_MAX 64
> >
> > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > index 6960beb75f32..dc590d331894 100644
> > --- a/arch/riscv/include/asm/insn-def.h
> > +++ b/arch/riscv/include/asm/insn-def.h
> > @@ -134,6 +134,7 @@
> >
> > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> >
> > #define HFENCE_VVMA(vaddr, asid) \
> > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > @@ -196,4 +197,8 @@
> > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > RS1(base), SIMM12(4))
> >
> > +#define CBO_prefetchw(base) \
> > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > + RD(x0), RS1(base), RS2(x0))
> > +
>
> I understand that here you create the instruction via bitfield, following
> the ISA, and this enables using instructions not available on the
> toolchain.
>
> It took me some time to find the document with this instruction, so please
> add this to the commit msg:
>
> https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> Page 23.
>
> IIUC, the instruction is "prefetch.w".
>
> Maybe I am missing something, but in the document the rs2 field
> (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
>
> rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
>
> Is the above correct, or am I missing something?
Oh, you are right. My fault, thx for pointing out. It should be:
+ INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
+ RD(x0), RS1(base), RS2(x3))
>
>
> Thanks!
> Leo
>
> > #endif /* __ASM_INSN_DEF_H */
> > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > index de9da852f78d..7ad3a24212e8 100644
> > --- a/arch/riscv/include/asm/processor.h
> > +++ b/arch/riscv/include/asm/processor.h
> > @@ -12,6 +12,8 @@
> > #include <vdso/processor.h>
> >
> > #include <asm/ptrace.h>
> > +#include <asm/insn-def.h>
> > +#include <asm/hwcap.h>
> >
> > #ifdef CONFIG_64BIT
> > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> >
> > +#define ARCH_HAS_PREFETCHW
> > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > + CBO_prefetchw(base), \
> > + 0, \
> > + RISCV_ISA_EXT_ZICBOP, \
> > + CONFIG_RISCV_ISA_ZICBOP)
> > +static inline void prefetchw(const void *ptr)
> > +{
> > + asm volatile(PREFETCHW_ASM(%0)
> > + : : "r" (ptr) : "memory");
> > +}
> >
> > /* Do necessary setup to start up a newly executed thread. */
> > extern void start_thread(struct pt_regs *regs,
> > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > index ef7b4fd9e876..e0b897db0b97 100644
> > --- a/arch/riscv/kernel/cpufeature.c
> > +++ b/arch/riscv/kernel/cpufeature.c
> > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > --
> > 2.36.1
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-15 12:22 ` Andrew Jones
@ 2023-09-15 12:42 ` Conor Dooley
-1 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-15 12:42 UTC (permalink / raw)
To: Andrew Jones
Cc: Leonardo Bras, guoren, paul.walmsley, anup, peterz, mingo, will,
palmer, longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
[-- Attachment #1: Type: text/plain, Size: 538 bytes --]
> > If this isn't being used in a similar manner, then the w has no reason
> > to be in the odd lowercase form.
>
> Other than to be consistent... However, the CBO_* instructions are not
> consistent with the rest of macros. If we don't need lowercase for any
> reason, then my preference would be to bite the bullet and change all the
> callsites of CBO_* macros and then introduce this new instruction as
> PREFETCH_W
Aye, I probably should've done it to begin with. Maybe there was some
other consideration at the time.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-15 12:42 ` Conor Dooley
0 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-15 12:42 UTC (permalink / raw)
To: Andrew Jones
Cc: Leonardo Bras, guoren, paul.walmsley, anup, peterz, mingo, will,
palmer, longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, linux-arch, linux-riscv,
linux-doc, kvm, virtualization, linux-csky, Guo Ren
[-- Attachment #1.1: Type: text/plain, Size: 538 bytes --]
> > If this isn't being used in a similar manner, then the w has no reason
> > to be in the odd lowercase form.
>
> Other than to be consistent... However, the CBO_* instructions are not
> consistent with the rest of macros. If we don't need lowercase for any
> reason, then my preference would be to bite the bullet and change all the
> callsites of CBO_* macros and then introduce this new instruction as
> PREFETCH_W
Aye, I probably should've done it to begin with. Maybe there was some
other consideration at the time.
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-15 12:14 ` Andrew Jones
@ 2023-09-15 12:53 ` Conor Dooley
-1 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-15 12:53 UTC (permalink / raw)
To: Andrew Jones
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
[-- Attachment #1: Type: text/plain, Size: 2022 bytes --]
On Fri, Sep 15, 2023 at 02:14:40PM +0200, Andrew Jones wrote:
> On Fri, Sep 15, 2023 at 12:37:50PM +0100, Conor Dooley wrote:
> > On Thu, Sep 14, 2023 at 04:47:18PM +0200, Andrew Jones wrote:
> > > On Thu, Sep 14, 2023 at 04:25:53PM +0200, Andrew Jones wrote:
> > > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > So, we could maybe proceed with assuming we
> > > can use zicbom_block_size for prefetch, for now. If a platform comes along
> > > that interpreted the spec differently, requiring prefetch block size to
> > > be specified separately, then we'll cross that bridge when we get there.
> >
> > That said, I think I suggested originally having the zicboz stuff default
> > to the zicbom size too, so I'd be happy with prefetch stuff working
> > exclusively that way until someone comes along looking for different sizes.
> > The binding should be updated though since
> >
> > riscv,cbom-block-size:
> > $ref: /schemas/types.yaml#/definitions/uint32
> > description:
> > The blocksize in bytes for the Zicbom cache operations.
> >
> > would no longer be a complete description.
> >
> > While thinking about new wording though, it feels really clunky to describe
> > it like:
> > The block size in bytes for the Zicbom cache operations, Zicbop
> > cache operations will default to this block size where not
> > explicitly defined.
> >
> > since there's then no way to actually define the block size if it is
> > different. Unless you've got some magic wording, I'd rather document
> > riscv,cbop-block-size, even if we are going to use riscv,cbom-block-size
> > as the default.
> >
>
> Sounds good to me, but if it's documented, then we should probably
> implement its parsing. Then, at that point, I wonder if it makes sense to
> have the fallback at all, or if it's not better just to require all the
> DTs to be explicit (even if redundant).
Sure, why not I guess.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-15 12:53 ` Conor Dooley
0 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-15 12:53 UTC (permalink / raw)
To: Andrew Jones
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, jszhang, wefu, wuwei2016, leobras, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
[-- Attachment #1.1: Type: text/plain, Size: 2022 bytes --]
On Fri, Sep 15, 2023 at 02:14:40PM +0200, Andrew Jones wrote:
> On Fri, Sep 15, 2023 at 12:37:50PM +0100, Conor Dooley wrote:
> > On Thu, Sep 14, 2023 at 04:47:18PM +0200, Andrew Jones wrote:
> > > On Thu, Sep 14, 2023 at 04:25:53PM +0200, Andrew Jones wrote:
> > > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > So, we could maybe proceed with assuming we
> > > can use zicbom_block_size for prefetch, for now. If a platform comes along
> > > that interpreted the spec differently, requiring prefetch block size to
> > > be specified separately, then we'll cross that bridge when we get there.
> >
> > That said, I think I suggested originally having the zicboz stuff default
> > to the zicbom size too, so I'd be happy with prefetch stuff working
> > exclusively that way until someone comes along looking for different sizes.
> > The binding should be updated though since
> >
> > riscv,cbom-block-size:
> > $ref: /schemas/types.yaml#/definitions/uint32
> > description:
> > The blocksize in bytes for the Zicbom cache operations.
> >
> > would no longer be a complete description.
> >
> > While thinking about new wording though, it feels really clunky to describe
> > it like:
> > The block size in bytes for the Zicbom cache operations, Zicbop
> > cache operations will default to this block size where not
> > explicitly defined.
> >
> > since there's then no way to actually define the block size if it is
> > different. Unless you've got some magic wording, I'd rather document
> > riscv,cbop-block-size, even if we are going to use riscv,cbom-block-size
> > as the default.
> >
>
> Sounds good to me, but if it's documented, then we should probably
> implement its parsing. Then, at that point, I wonder if it makes sense to
> have the fallback at all, or if it's not better just to require all the
> DTs to be explicit (even if redundant).
Sure, why not I guess.
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-15 11:07 ` Andrew Jones
@ 2023-09-15 20:32 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 20:32 UTC (permalink / raw)
To: Andrew Jones
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, conor.dooley, xiaoguang.xing, bjorn, alexghiti,
keescook, greentime.hu, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 01:07:40PM +0200, Andrew Jones wrote:
> On Fri, Sep 15, 2023 at 05:22:26AM -0300, Leonardo Bras wrote:
> > On Thu, Sep 14, 2023 at 03:47:59PM +0200, Andrew Jones wrote:
> > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> ...
> > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > index 6960beb75f32..dc590d331894 100644
> > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > @@ -134,6 +134,7 @@
> > > >
> > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > >
> > > This should be named RV_OPCODE_OP_IMM and be placed in
> > > numerical order with the others, i.e. above SYSTEM.
> > >
> > > >
> > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > @@ -196,4 +197,8 @@
> > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > RS1(base), SIMM12(4))
> > > >
> > > > +#define CBO_prefetchw(base) \
> > >
> > > Please name this 'PREFETCH_w' and it should take an immediate parameter,
> > > even if we intend to pass 0 for it.
> >
> > It makes sense.
> >
> > The mnemonic in the previously mentioned documentation is:
> >
> > prefetch.w offset(base)
> >
> > So yeah, makes sense to have both offset and base as parameters for
> > CBO_prefetchw (or PREFETCH_w, I have no strong preference).
>
> I have a strong preference :-)
>
> PREFETCH_w is consistent with the naming we already have for e.g.
> cbo.clean, which is CBO_clean. The instruction we're picking a name
> for now is prefetch.w, not cbo.prefetchw.
>
> >
> > >
> > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > + RD(x0), RS1(base), RS2(x0))
> > >
> > > prefetch.w is not an R-type instruction, it's an S-type. While the bit
> > > shifts are the same, the names are different. We need to add S-type
> > > names while defining this instruction.
> >
> > That is correct, it is supposed to look like a store instruction (S-type),
> > even though documentation don't explicitly state that.
> >
> > Even though it works fine with the R-type definition, code documentation
> > would be wrong, and future changes could break it.
> >
> > > Then, this define would be
> > >
> > > #define PREFETCH_w(base, imm) \
>
> I should have suggested 'offset' instead of 'imm' for the second parameter
> name.
>
> > > INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
> > > RS1(base), __RS2(3))
> >
> > s/OPCODE_OP_IMM/OPCODE_PREFETCH
> > 0x4 vs 0x13
>
> There's no major opcode named "PREFETCH" and the spec says that the major
> opcode used for prefetch instructions is OP-IMM. That's why we want to
> name this OPCODE_OP_IMM. I'm not sure where the 0x4 you're referring to
> comes from
Oh, you are right.
Sorry about this, I misinterpreted table 24.1 from the
Unprivileged ISA (20191213).
Yeap, everything make sense now, and the define below is not actually
needed:
> > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
Thanks!
Leo
> . A 32-bit instruction has the lowest two bits set (figure 1.1
> of the unpriv spec) and table 27.1 of the unpriv spec shows OP-IMM is
> 0b00100xx, so we have 0b0010011. Keeping the naming of the opcode macros
> consistent with the spec also keeps them consistent with the .insn
> directive where we could even use the names directly, i.e.
>
> .insn s OP_IMM, 6, x3, 0(a0)
>
> > >
> > > When the assembler as insn_r I hope it will validate that
>
> I meant insn_s here, which would be the macro for '.insn s'
>
> > > (imm & 0xfe0) == imm
>
> I played with it. It won't do what we want for prefetch, only
> what works for s-type instructions in general, i.e. it allows
> +/-2047 offsets and fails for everything else. That's good enough.
> We can just mask off the low 5 bits here in our macro
>
> #define PREFETCH_w(base, offset) \
> INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5((offset) & ~0x1f), \
> __IMM_4_0(0), RS1(base), __RS2(3))
>
> Thanks,
> drew
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-15 20:32 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-15 20:32 UTC (permalink / raw)
To: Andrew Jones
Cc: guoren, paul.walmsley, anup, peterz, mingo, will, palmer,
longman, boqun.feng, tglx, paulmck, rostedt, rdunlap,
catalin.marinas, conor.dooley, xiaoguang.xing, bjorn, alexghiti,
keescook, greentime.hu, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 01:07:40PM +0200, Andrew Jones wrote:
> On Fri, Sep 15, 2023 at 05:22:26AM -0300, Leonardo Bras wrote:
> > On Thu, Sep 14, 2023 at 03:47:59PM +0200, Andrew Jones wrote:
> > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> ...
> > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > index 6960beb75f32..dc590d331894 100644
> > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > @@ -134,6 +134,7 @@
> > > >
> > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > >
> > > This should be named RV_OPCODE_OP_IMM and be placed in
> > > numerical order with the others, i.e. above SYSTEM.
> > >
> > > >
> > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > @@ -196,4 +197,8 @@
> > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > RS1(base), SIMM12(4))
> > > >
> > > > +#define CBO_prefetchw(base) \
> > >
> > > Please name this 'PREFETCH_w' and it should take an immediate parameter,
> > > even if we intend to pass 0 for it.
> >
> > It makes sense.
> >
> > The mnemonic in the previously mentioned documentation is:
> >
> > prefetch.w offset(base)
> >
> > So yeah, makes sense to have both offset and base as parameters for
> > CBO_prefetchw (or PREFETCH_w, I have no strong preference).
>
> I have a strong preference :-)
>
> PREFETCH_w is consistent with the naming we already have for e.g.
> cbo.clean, which is CBO_clean. The instruction we're picking a name
> for now is prefetch.w, not cbo.prefetchw.
>
> >
> > >
> > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > + RD(x0), RS1(base), RS2(x0))
> > >
> > > prefetch.w is not an R-type instruction, it's an S-type. While the bit
> > > shifts are the same, the names are different. We need to add S-type
> > > names while defining this instruction.
> >
> > That is correct, it is supposed to look like a store instruction (S-type),
> > even though documentation don't explicitly state that.
> >
> > Even though it works fine with the R-type definition, code documentation
> > would be wrong, and future changes could break it.
> >
> > > Then, this define would be
> > >
> > > #define PREFETCH_w(base, imm) \
>
> I should have suggested 'offset' instead of 'imm' for the second parameter
> name.
>
> > > INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5(imm), __IMM_4_0(0), \
> > > RS1(base), __RS2(3))
> >
> > s/OPCODE_OP_IMM/OPCODE_PREFETCH
> > 0x4 vs 0x13
>
> There's no major opcode named "PREFETCH" and the spec says that the major
> opcode used for prefetch instructions is OP-IMM. That's why we want to
> name this OPCODE_OP_IMM. I'm not sure where the 0x4 you're referring to
> comes from
Oh, you are right.
Sorry about this, I misinterpreted table 24.1 from the
Unprivileged ISA (20191213).
Yeap, everything make sense now, and the define below is not actually
needed:
> > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
Thanks!
Leo
> . A 32-bit instruction has the lowest two bits set (figure 1.1
> of the unpriv spec) and table 27.1 of the unpriv spec shows OP-IMM is
> 0b00100xx, so we have 0b0010011. Keeping the naming of the opcode macros
> consistent with the spec also keeps them consistent with the .insn
> directive where we could even use the names directly, i.e.
>
> .insn s OP_IMM, 6, x3, 0(a0)
>
> > >
> > > When the assembler as insn_r I hope it will validate that
>
> I meant insn_s here, which would be the macro for '.insn s'
>
> > > (imm & 0xfe0) == imm
>
> I played with it. It won't do what we want for prefetch, only
> what works for s-type instructions in general, i.e. it allows
> +/-2047 offsets and fails for everything else. That's good enough.
> We can just mask off the low 5 bits here in our macro
>
> #define PREFETCH_w(base, offset) \
> INSN_S(OPCODE_OP_IMM, FUNC3(6), IMM_11_5((offset) & ~0x1f), \
> __IMM_4_0(0), RS1(base), __RS2(3))
>
> Thanks,
> drew
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-15 12:42 ` Conor Dooley
@ 2023-09-16 0:05 ` Conor Dooley
-1 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-16 0:05 UTC (permalink / raw)
To: Conor Dooley
Cc: Andrew Jones, Leonardo Bras, guoren, paul.walmsley, anup, peterz,
mingo, will, palmer, longman, boqun.feng, tglx, paulmck, rostedt,
rdunlap, catalin.marinas, xiaoguang.xing, bjorn, alexghiti,
keescook, greentime.hu, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
[-- Attachment #1.1: Type: text/plain, Size: 786 bytes --]
On Fri, Sep 15, 2023 at 01:42:56PM +0100, Conor Dooley wrote:
>
> > > If this isn't being used in a similar manner, then the w has no reason
> > > to be in the odd lowercase form.
> >
> > Other than to be consistent... However, the CBO_* instructions are not
> > consistent with the rest of macros. If we don't need lowercase for any
> > reason, then my preference would be to bite the bullet and change all the
> > callsites of CBO_* macros and then introduce this new instruction as
> > PREFETCH_W
>
> Aye, I probably should've done it to begin with. Maybe there was some
> other consideration at the time.
FWIW, I sent a patch for this earlier today. I figure you saw it Drew,
but nonetheless:
https://lore.kernel.org/all/20230915-aloe-dollar-994937477776@spud/
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-16 0:05 ` Conor Dooley
0 siblings, 0 replies; 215+ messages in thread
From: Conor Dooley @ 2023-09-16 0:05 UTC (permalink / raw)
To: Conor Dooley
Cc: Andrew Jones, Leonardo Bras, guoren, paul.walmsley, anup, peterz,
mingo, will, palmer, longman, boqun.feng, tglx, paulmck, rostedt,
rdunlap, catalin.marinas, xiaoguang.xing, bjorn, alexghiti,
keescook, greentime.hu, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
[-- Attachment #1: Type: text/plain, Size: 786 bytes --]
On Fri, Sep 15, 2023 at 01:42:56PM +0100, Conor Dooley wrote:
>
> > > If this isn't being used in a similar manner, then the w has no reason
> > > to be in the odd lowercase form.
> >
> > Other than to be consistent... However, the CBO_* instructions are not
> > consistent with the rest of macros. If we don't need lowercase for any
> > reason, then my preference would be to bite the bullet and change all the
> > callsites of CBO_* macros and then introduce this new instruction as
> > PREFETCH_W
>
> Aye, I probably should've done it to begin with. Maybe there was some
> other consideration at the time.
FWIW, I sent a patch for this earlier today. I figure you saw it Drew,
but nonetheless:
https://lore.kernel.org/all/20230915-aloe-dollar-994937477776@spud/
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-15 12:36 ` Guo Ren
@ 2023-09-16 1:25 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-16 1:25 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 08:36:31PM +0800, Guo Ren wrote:
> On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Cache-block prefetch instructions are HINTs to the hardware to
> > > indicate that software intends to perform a particular type of
> > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > improve the arch_xchg for qspinlock xchg_tail.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > arch/riscv/include/asm/hwcap.h | 1 +
> > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > arch/riscv/kernel/cpufeature.c | 1 +
> > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index e9ae6fa232c3..2c346fe169c1 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > >
> > > If you don't know what to do here, say Y.
> > >
> > > +config RISCV_ISA_ZICBOP
> > > + bool "Zicbop extension support for cache block prefetch"
> > > + depends on MMU
> > > + depends on RISCV_ALTERNATIVE
> > > + default y
> > > + help
> > > + Adds support to dynamically detect the presence of the ZICBOP
> > > + extension (Cache Block Prefetch Operations) and enable its
> > > + usage.
> > > +
> > > + The Zicbop extension can be used to prefetch cache block for
> > > + read/write/instruction fetch.
> > > +
> > > + If you don't know what to do here, say Y.
> > > +
> > > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > > bool
> > > default y
> > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > index 702725727671..56eff7a9d2d2 100644
> > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > @@ -11,6 +11,7 @@
> > >
> > > #include <asm/barrier.h>
> > > #include <asm/fence.h>
> > > +#include <asm/processor.h>
> > >
> > > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > > ({ \
> > > @@ -25,6 +26,7 @@
> > > \
> > > __asm__ __volatile__ ( \
> > > prepend \
> > > + PREFETCHW_ASM(%5) \
> > > "0: lr.w %0, %2\n" \
> > > " and %1, %0, %z4\n" \
> > > " or %1, %1, %z3\n" \
> > > @@ -32,7 +34,7 @@
> > > " bnez %1, 0b\n" \
> > > append \
> > > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > > - : "rJ" (__newx), "rJ" (~__mask) \
> > > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > > : "memory"); \
> > > \
> > > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > > index b7b58258f6c7..78b7b8b53778 100644
> > > --- a/arch/riscv/include/asm/hwcap.h
> > > +++ b/arch/riscv/include/asm/hwcap.h
> > > @@ -58,6 +58,7 @@
> > > #define RISCV_ISA_EXT_ZICSR 40
> > > #define RISCV_ISA_EXT_ZIFENCEI 41
> > > #define RISCV_ISA_EXT_ZIHPM 42
> > > +#define RISCV_ISA_EXT_ZICBOP 43
> > >
> > > #define RISCV_ISA_EXT_MAX 64
> > >
> > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > index 6960beb75f32..dc590d331894 100644
> > > --- a/arch/riscv/include/asm/insn-def.h
> > > +++ b/arch/riscv/include/asm/insn-def.h
> > > @@ -134,6 +134,7 @@
> > >
> > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > >
> > > #define HFENCE_VVMA(vaddr, asid) \
> > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > @@ -196,4 +197,8 @@
> > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > RS1(base), SIMM12(4))
> > >
> > > +#define CBO_prefetchw(base) \
> > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > + RD(x0), RS1(base), RS2(x0))
> > > +
> >
> > I understand that here you create the instruction via bitfield, following
> > the ISA, and this enables using instructions not available on the
> > toolchain.
> >
> > It took me some time to find the document with this instruction, so please
> > add this to the commit msg:
> >
> > https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> > Page 23.
> >
> > IIUC, the instruction is "prefetch.w".
> >
> > Maybe I am missing something, but in the document the rs2 field
> > (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
> >
> > rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
> >
> > Is the above correct, or am I missing something?
> Oh, you are right. My fault, thx for pointing out. It should be:
> + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> + RD(x0), RS1(base), RS2(x3))
Now I am curious to check if / how will this impact performance. :)
(Please let me know)
>
> >
> >
> > Thanks!
> > Leo
> >
> > > #endif /* __ASM_INSN_DEF_H */
> > > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > > index de9da852f78d..7ad3a24212e8 100644
> > > --- a/arch/riscv/include/asm/processor.h
> > > +++ b/arch/riscv/include/asm/processor.h
> > > @@ -12,6 +12,8 @@
> > > #include <vdso/processor.h>
> > >
> > > #include <asm/ptrace.h>
> > > +#include <asm/insn-def.h>
> > > +#include <asm/hwcap.h>
> > >
> > > #ifdef CONFIG_64BIT
> > > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> > >
> > > +#define ARCH_HAS_PREFETCHW
> > > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > > + CBO_prefetchw(base), \
> > > + 0, \
> > > + RISCV_ISA_EXT_ZICBOP, \
> > > + CONFIG_RISCV_ISA_ZICBOP)
> > > +static inline void prefetchw(const void *ptr)
> > > +{
> > > + asm volatile(PREFETCHW_ASM(%0)
> > > + : : "r" (ptr) : "memory");
> > > +}
> > >
> > > /* Do necessary setup to start up a newly executed thread. */
> > > extern void start_thread(struct pt_regs *regs,
> > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > index ef7b4fd9e876..e0b897db0b97 100644
> > > --- a/arch/riscv/kernel/cpufeature.c
> > > +++ b/arch/riscv/kernel/cpufeature.c
> > > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-16 1:25 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-16 1:25 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 08:36:31PM +0800, Guo Ren wrote:
> On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Cache-block prefetch instructions are HINTs to the hardware to
> > > indicate that software intends to perform a particular type of
> > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > improve the arch_xchg for qspinlock xchg_tail.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > arch/riscv/include/asm/hwcap.h | 1 +
> > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > arch/riscv/kernel/cpufeature.c | 1 +
> > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > index e9ae6fa232c3..2c346fe169c1 100644
> > > --- a/arch/riscv/Kconfig
> > > +++ b/arch/riscv/Kconfig
> > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > >
> > > If you don't know what to do here, say Y.
> > >
> > > +config RISCV_ISA_ZICBOP
> > > + bool "Zicbop extension support for cache block prefetch"
> > > + depends on MMU
> > > + depends on RISCV_ALTERNATIVE
> > > + default y
> > > + help
> > > + Adds support to dynamically detect the presence of the ZICBOP
> > > + extension (Cache Block Prefetch Operations) and enable its
> > > + usage.
> > > +
> > > + The Zicbop extension can be used to prefetch cache block for
> > > + read/write/instruction fetch.
> > > +
> > > + If you don't know what to do here, say Y.
> > > +
> > > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > > bool
> > > default y
> > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > index 702725727671..56eff7a9d2d2 100644
> > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > @@ -11,6 +11,7 @@
> > >
> > > #include <asm/barrier.h>
> > > #include <asm/fence.h>
> > > +#include <asm/processor.h>
> > >
> > > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > > ({ \
> > > @@ -25,6 +26,7 @@
> > > \
> > > __asm__ __volatile__ ( \
> > > prepend \
> > > + PREFETCHW_ASM(%5) \
> > > "0: lr.w %0, %2\n" \
> > > " and %1, %0, %z4\n" \
> > > " or %1, %1, %z3\n" \
> > > @@ -32,7 +34,7 @@
> > > " bnez %1, 0b\n" \
> > > append \
> > > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > > - : "rJ" (__newx), "rJ" (~__mask) \
> > > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > > : "memory"); \
> > > \
> > > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > > index b7b58258f6c7..78b7b8b53778 100644
> > > --- a/arch/riscv/include/asm/hwcap.h
> > > +++ b/arch/riscv/include/asm/hwcap.h
> > > @@ -58,6 +58,7 @@
> > > #define RISCV_ISA_EXT_ZICSR 40
> > > #define RISCV_ISA_EXT_ZIFENCEI 41
> > > #define RISCV_ISA_EXT_ZIHPM 42
> > > +#define RISCV_ISA_EXT_ZICBOP 43
> > >
> > > #define RISCV_ISA_EXT_MAX 64
> > >
> > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > index 6960beb75f32..dc590d331894 100644
> > > --- a/arch/riscv/include/asm/insn-def.h
> > > +++ b/arch/riscv/include/asm/insn-def.h
> > > @@ -134,6 +134,7 @@
> > >
> > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > >
> > > #define HFENCE_VVMA(vaddr, asid) \
> > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > @@ -196,4 +197,8 @@
> > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > RS1(base), SIMM12(4))
> > >
> > > +#define CBO_prefetchw(base) \
> > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > + RD(x0), RS1(base), RS2(x0))
> > > +
> >
> > I understand that here you create the instruction via bitfield, following
> > the ISA, and this enables using instructions not available on the
> > toolchain.
> >
> > It took me some time to find the document with this instruction, so please
> > add this to the commit msg:
> >
> > https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> > Page 23.
> >
> > IIUC, the instruction is "prefetch.w".
> >
> > Maybe I am missing something, but in the document the rs2 field
> > (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
> >
> > rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
> >
> > Is the above correct, or am I missing something?
> Oh, you are right. My fault, thx for pointing out. It should be:
> + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> + RD(x0), RS1(base), RS2(x3))
Now I am curious to check if / how will this impact performance. :)
(Please let me know)
>
> >
> >
> > Thanks!
> > Leo
> >
> > > #endif /* __ASM_INSN_DEF_H */
> > > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > > index de9da852f78d..7ad3a24212e8 100644
> > > --- a/arch/riscv/include/asm/processor.h
> > > +++ b/arch/riscv/include/asm/processor.h
> > > @@ -12,6 +12,8 @@
> > > #include <vdso/processor.h>
> > >
> > > #include <asm/ptrace.h>
> > > +#include <asm/insn-def.h>
> > > +#include <asm/hwcap.h>
> > >
> > > #ifdef CONFIG_64BIT
> > > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> > >
> > > +#define ARCH_HAS_PREFETCHW
> > > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > > + CBO_prefetchw(base), \
> > > + 0, \
> > > + RISCV_ISA_EXT_ZICBOP, \
> > > + CONFIG_RISCV_ISA_ZICBOP)
> > > +static inline void prefetchw(const void *ptr)
> > > +{
> > > + asm volatile(PREFETCHW_ASM(%0)
> > > + : : "r" (ptr) : "memory");
> > > +}
> > >
> > > /* Do necessary setup to start up a newly executed thread. */
> > > extern void start_thread(struct pt_regs *regs,
> > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > index ef7b4fd9e876..e0b897db0b97 100644
> > > --- a/arch/riscv/kernel/cpufeature.c
> > > +++ b/arch/riscv/kernel/cpufeature.c
> > > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-16 1:25 ` Leonardo Bras
@ 2023-09-17 14:34 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 14:34 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sat, Sep 16, 2023 at 9:25 AM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Fri, Sep 15, 2023 at 08:36:31PM +0800, Guo Ren wrote:
> > On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > Cache-block prefetch instructions are HINTs to the hardware to
> > > > indicate that software intends to perform a particular type of
> > > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > > improve the arch_xchg for qspinlock xchg_tail.
> > > >
> > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > ---
> > > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > > arch/riscv/include/asm/hwcap.h | 1 +
> > > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > > arch/riscv/kernel/cpufeature.c | 1 +
> > > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > index e9ae6fa232c3..2c346fe169c1 100644
> > > > --- a/arch/riscv/Kconfig
> > > > +++ b/arch/riscv/Kconfig
> > > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > > >
> > > > If you don't know what to do here, say Y.
> > > >
> > > > +config RISCV_ISA_ZICBOP
> > > > + bool "Zicbop extension support for cache block prefetch"
> > > > + depends on MMU
> > > > + depends on RISCV_ALTERNATIVE
> > > > + default y
> > > > + help
> > > > + Adds support to dynamically detect the presence of the ZICBOP
> > > > + extension (Cache Block Prefetch Operations) and enable its
> > > > + usage.
> > > > +
> > > > + The Zicbop extension can be used to prefetch cache block for
> > > > + read/write/instruction fetch.
> > > > +
> > > > + If you don't know what to do here, say Y.
> > > > +
> > > > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > > > bool
> > > > default y
> > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > index 702725727671..56eff7a9d2d2 100644
> > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > @@ -11,6 +11,7 @@
> > > >
> > > > #include <asm/barrier.h>
> > > > #include <asm/fence.h>
> > > > +#include <asm/processor.h>
> > > >
> > > > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > > > ({ \
> > > > @@ -25,6 +26,7 @@
> > > > \
> > > > __asm__ __volatile__ ( \
> > > > prepend \
> > > > + PREFETCHW_ASM(%5) \
> > > > "0: lr.w %0, %2\n" \
> > > > " and %1, %0, %z4\n" \
> > > > " or %1, %1, %z3\n" \
> > > > @@ -32,7 +34,7 @@
> > > > " bnez %1, 0b\n" \
> > > > append \
> > > > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > > > - : "rJ" (__newx), "rJ" (~__mask) \
> > > > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > > > : "memory"); \
> > > > \
> > > > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > > > index b7b58258f6c7..78b7b8b53778 100644
> > > > --- a/arch/riscv/include/asm/hwcap.h
> > > > +++ b/arch/riscv/include/asm/hwcap.h
> > > > @@ -58,6 +58,7 @@
> > > > #define RISCV_ISA_EXT_ZICSR 40
> > > > #define RISCV_ISA_EXT_ZIFENCEI 41
> > > > #define RISCV_ISA_EXT_ZIHPM 42
> > > > +#define RISCV_ISA_EXT_ZICBOP 43
> > > >
> > > > #define RISCV_ISA_EXT_MAX 64
> > > >
> > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > index 6960beb75f32..dc590d331894 100644
> > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > @@ -134,6 +134,7 @@
> > > >
> > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > > >
> > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > @@ -196,4 +197,8 @@
> > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > RS1(base), SIMM12(4))
> > > >
> > > > +#define CBO_prefetchw(base) \
> > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > + RD(x0), RS1(base), RS2(x0))
> > > > +
> > >
> > > I understand that here you create the instruction via bitfield, following
> > > the ISA, and this enables using instructions not available on the
> > > toolchain.
> > >
> > > It took me some time to find the document with this instruction, so please
> > > add this to the commit msg:
> > >
> > > https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> > > Page 23.
> > >
> > > IIUC, the instruction is "prefetch.w".
> > >
> > > Maybe I am missing something, but in the document the rs2 field
> > > (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
> > >
> > > rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
> > >
> > > Is the above correct, or am I missing something?
> > Oh, you are right. My fault, thx for pointing out. It should be:
> > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > + RD(x0), RS1(base), RS2(x3))
>
> Now I am curious to check if / how will this impact performance. :)
> (Please let me know)
Ref:
commit 0ea366f5e1b6 ("arm64: atomics: prefetch the destination word
for write prior to stxr")
commit 86d231459d6d ("bpf: cpumap memory prefetchw optimizations for
struct page")
>
>
> >
> > >
> > >
> > > Thanks!
> > > Leo
> > >
> > > > #endif /* __ASM_INSN_DEF_H */
> > > > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > > > index de9da852f78d..7ad3a24212e8 100644
> > > > --- a/arch/riscv/include/asm/processor.h
> > > > +++ b/arch/riscv/include/asm/processor.h
> > > > @@ -12,6 +12,8 @@
> > > > #include <vdso/processor.h>
> > > >
> > > > #include <asm/ptrace.h>
> > > > +#include <asm/insn-def.h>
> > > > +#include <asm/hwcap.h>
> > > >
> > > > #ifdef CONFIG_64BIT
> > > > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > > > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > > > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > > > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> > > >
> > > > +#define ARCH_HAS_PREFETCHW
> > > > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > > > + CBO_prefetchw(base), \
> > > > + 0, \
> > > > + RISCV_ISA_EXT_ZICBOP, \
> > > > + CONFIG_RISCV_ISA_ZICBOP)
> > > > +static inline void prefetchw(const void *ptr)
> > > > +{
> > > > + asm volatile(PREFETCHW_ASM(%0)
> > > > + : : "r" (ptr) : "memory");
> > > > +}
> > > >
> > > > /* Do necessary setup to start up a newly executed thread. */
> > > > extern void start_thread(struct pt_regs *regs,
> > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > > index ef7b4fd9e876..e0b897db0b97 100644
> > > > --- a/arch/riscv/kernel/cpufeature.c
> > > > +++ b/arch/riscv/kernel/cpufeature.c
> > > > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > > > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > > > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > > > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > > > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > > > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > > > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > > > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > > > --
> > > > 2.36.1
> > > >
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-17 14:34 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 14:34 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sat, Sep 16, 2023 at 9:25 AM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Fri, Sep 15, 2023 at 08:36:31PM +0800, Guo Ren wrote:
> > On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > Cache-block prefetch instructions are HINTs to the hardware to
> > > > indicate that software intends to perform a particular type of
> > > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > > improve the arch_xchg for qspinlock xchg_tail.
> > > >
> > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > ---
> > > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > > arch/riscv/include/asm/hwcap.h | 1 +
> > > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > > arch/riscv/kernel/cpufeature.c | 1 +
> > > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > index e9ae6fa232c3..2c346fe169c1 100644
> > > > --- a/arch/riscv/Kconfig
> > > > +++ b/arch/riscv/Kconfig
> > > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > > >
> > > > If you don't know what to do here, say Y.
> > > >
> > > > +config RISCV_ISA_ZICBOP
> > > > + bool "Zicbop extension support for cache block prefetch"
> > > > + depends on MMU
> > > > + depends on RISCV_ALTERNATIVE
> > > > + default y
> > > > + help
> > > > + Adds support to dynamically detect the presence of the ZICBOP
> > > > + extension (Cache Block Prefetch Operations) and enable its
> > > > + usage.
> > > > +
> > > > + The Zicbop extension can be used to prefetch cache block for
> > > > + read/write/instruction fetch.
> > > > +
> > > > + If you don't know what to do here, say Y.
> > > > +
> > > > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > > > bool
> > > > default y
> > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > index 702725727671..56eff7a9d2d2 100644
> > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > @@ -11,6 +11,7 @@
> > > >
> > > > #include <asm/barrier.h>
> > > > #include <asm/fence.h>
> > > > +#include <asm/processor.h>
> > > >
> > > > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > > > ({ \
> > > > @@ -25,6 +26,7 @@
> > > > \
> > > > __asm__ __volatile__ ( \
> > > > prepend \
> > > > + PREFETCHW_ASM(%5) \
> > > > "0: lr.w %0, %2\n" \
> > > > " and %1, %0, %z4\n" \
> > > > " or %1, %1, %z3\n" \
> > > > @@ -32,7 +34,7 @@
> > > > " bnez %1, 0b\n" \
> > > > append \
> > > > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > > > - : "rJ" (__newx), "rJ" (~__mask) \
> > > > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > > > : "memory"); \
> > > > \
> > > > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > > > index b7b58258f6c7..78b7b8b53778 100644
> > > > --- a/arch/riscv/include/asm/hwcap.h
> > > > +++ b/arch/riscv/include/asm/hwcap.h
> > > > @@ -58,6 +58,7 @@
> > > > #define RISCV_ISA_EXT_ZICSR 40
> > > > #define RISCV_ISA_EXT_ZIFENCEI 41
> > > > #define RISCV_ISA_EXT_ZIHPM 42
> > > > +#define RISCV_ISA_EXT_ZICBOP 43
> > > >
> > > > #define RISCV_ISA_EXT_MAX 64
> > > >
> > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > index 6960beb75f32..dc590d331894 100644
> > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > @@ -134,6 +134,7 @@
> > > >
> > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > > >
> > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > @@ -196,4 +197,8 @@
> > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > RS1(base), SIMM12(4))
> > > >
> > > > +#define CBO_prefetchw(base) \
> > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > + RD(x0), RS1(base), RS2(x0))
> > > > +
> > >
> > > I understand that here you create the instruction via bitfield, following
> > > the ISA, and this enables using instructions not available on the
> > > toolchain.
> > >
> > > It took me some time to find the document with this instruction, so please
> > > add this to the commit msg:
> > >
> > > https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> > > Page 23.
> > >
> > > IIUC, the instruction is "prefetch.w".
> > >
> > > Maybe I am missing something, but in the document the rs2 field
> > > (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
> > >
> > > rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
> > >
> > > Is the above correct, or am I missing something?
> > Oh, you are right. My fault, thx for pointing out. It should be:
> > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > + RD(x0), RS1(base), RS2(x3))
>
> Now I am curious to check if / how will this impact performance. :)
> (Please let me know)
Ref:
commit 0ea366f5e1b6 ("arm64: atomics: prefetch the destination word
for write prior to stxr")
commit 86d231459d6d ("bpf: cpumap memory prefetchw optimizations for
struct page")
>
>
> >
> > >
> > >
> > > Thanks!
> > > Leo
> > >
> > > > #endif /* __ASM_INSN_DEF_H */
> > > > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > > > index de9da852f78d..7ad3a24212e8 100644
> > > > --- a/arch/riscv/include/asm/processor.h
> > > > +++ b/arch/riscv/include/asm/processor.h
> > > > @@ -12,6 +12,8 @@
> > > > #include <vdso/processor.h>
> > > >
> > > > #include <asm/ptrace.h>
> > > > +#include <asm/insn-def.h>
> > > > +#include <asm/hwcap.h>
> > > >
> > > > #ifdef CONFIG_64BIT
> > > > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > > > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > > > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > > > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> > > >
> > > > +#define ARCH_HAS_PREFETCHW
> > > > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > > > + CBO_prefetchw(base), \
> > > > + 0, \
> > > > + RISCV_ISA_EXT_ZICBOP, \
> > > > + CONFIG_RISCV_ISA_ZICBOP)
> > > > +static inline void prefetchw(const void *ptr)
> > > > +{
> > > > + asm volatile(PREFETCHW_ASM(%0)
> > > > + : : "r" (ptr) : "memory");
> > > > +}
> > > >
> > > > /* Do necessary setup to start up a newly executed thread. */
> > > > extern void start_thread(struct pt_regs *regs,
> > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > > index ef7b4fd9e876..e0b897db0b97 100644
> > > > --- a/arch/riscv/kernel/cpufeature.c
> > > > +++ b/arch/riscv/kernel/cpufeature.c
> > > > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > > > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > > > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > > > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > > > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > > > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > > > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > > > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > > > --
> > > > 2.36.1
> > > >
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 11/17] RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
2023-09-15 5:42 ` Leonardo Bras
@ 2023-09-17 14:58 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 14:58 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 1:42 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:29:05AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Using static_call to switch between:
> > native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath()
> > native_queued_spin_unlock() __pv_queued_spin_unlock()
> >
> > Finish the pv_wait implementation, but pv_kick needs the SBI
> > definition of the next patches.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/include/asm/Kbuild | 1 -
> > arch/riscv/include/asm/qspinlock.h | 35 +++++++++++++
> > arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++++++++
> > arch/riscv/include/asm/spinlock.h | 2 +-
> > arch/riscv/kernel/qspinlock_paravirt.c | 57 +++++++++++++++++++++
> > arch/riscv/kernel/setup.c | 4 ++
> > 6 files changed, 126 insertions(+), 2 deletions(-)
> > create mode 100644 arch/riscv/include/asm/qspinlock.h
> > create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> > create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> >
> > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > index a0dc85e4a754..b89cb3b73c13 100644
> > --- a/arch/riscv/include/asm/Kbuild
> > +++ b/arch/riscv/include/asm/Kbuild
> > @@ -7,6 +7,5 @@ generic-y += parport.h
> > generic-y += spinlock_types.h
> > generic-y += qrwlock.h
> > generic-y += qrwlock_types.h
> > -generic-y += qspinlock.h
> > generic-y += user.h
> > generic-y += vmlinux.lds.h
> > diff --git a/arch/riscv/include/asm/qspinlock.h b/arch/riscv/include/asm/qspinlock.h
> > new file mode 100644
> > index 000000000000..7d4f416c908c
> > --- /dev/null
> > +++ b/arch/riscv/include/asm/qspinlock.h
> > @@ -0,0 +1,35 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c), 2023 Alibaba Cloud
> > + * Authors:
> > + * Guo Ren <guoren@linux.alibaba.com>
> > + */
> > +
> > +#ifndef _ASM_RISCV_QSPINLOCK_H
> > +#define _ASM_RISCV_QSPINLOCK_H
> > +
> > +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> > +#include <asm/qspinlock_paravirt.h>
> > +
> > +/* How long a lock should spin before we consider blocking */
> > +#define SPIN_THRESHOLD (1 << 15)
> > +
> > +void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > +void __pv_init_lock_hash(void);
> > +void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > +
> > +static inline void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
> > +{
> > + static_call(pv_queued_spin_lock_slowpath)(lock, val);
> > +}
> > +
> > +#define queued_spin_unlock queued_spin_unlock
> > +static inline void queued_spin_unlock(struct qspinlock *lock)
> > +{
> > + static_call(pv_queued_spin_unlock)(lock);
> > +}
> > +#endif /* CONFIG_PARAVIRT_SPINLOCKS */
> > +
> > +#include <asm-generic/qspinlock.h>
> > +
> > +#endif /* _ASM_RISCV_QSPINLOCK_H */
> > diff --git a/arch/riscv/include/asm/qspinlock_paravirt.h b/arch/riscv/include/asm/qspinlock_paravirt.h
> > new file mode 100644
> > index 000000000000..9681e851f69d
> > --- /dev/null
> > +++ b/arch/riscv/include/asm/qspinlock_paravirt.h
> > @@ -0,0 +1,29 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c), 2023 Alibaba Cloud
> > + * Authors:
> > + * Guo Ren <guoren@linux.alibaba.com>
> > + */
> > +
> > +#ifndef _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> > +#define _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> > +
> > +void pv_wait(u8 *ptr, u8 val);
> > +void pv_kick(int cpu);
> > +
> > +void dummy_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > +void dummy_queued_spin_unlock(struct qspinlock *lock);
> > +
> > +DECLARE_STATIC_CALL(pv_queued_spin_lock_slowpath, dummy_queued_spin_lock_slowpath);
> > +DECLARE_STATIC_CALL(pv_queued_spin_unlock, dummy_queued_spin_unlock);
> > +
> > +void __init pv_qspinlock_init(void);
> > +
> > +static inline bool pv_is_native_spin_unlock(void)
> > +{
> > + return false;
> > +}
> > +
> > +void __pv_queued_spin_unlock(struct qspinlock *lock);
> > +
> > +#endif /* _ASM_RISCV_QSPINLOCK_PARAVIRT_H */
> > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > index 6b38d6616f14..ed4253f491fe 100644
> > --- a/arch/riscv/include/asm/spinlock.h
> > +++ b/arch/riscv/include/asm/spinlock.h
> > @@ -39,7 +39,7 @@ static inline bool virt_spin_lock(struct qspinlock *lock)
> > #undef arch_spin_trylock
> > #undef arch_spin_unlock
> >
> > -#include <asm-generic/qspinlock.h>
> > +#include <asm/qspinlock.h>
> > #include <linux/jump_label.h>
> >
> > #undef arch_spin_is_locked
> > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > new file mode 100644
> > index 000000000000..85ff5a3ec234
> > --- /dev/null
> > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > @@ -0,0 +1,57 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c), 2023 Alibaba Cloud
> > + * Authors:
> > + * Guo Ren <guoren@linux.alibaba.com>
> > + */
> > +
> > +#include <linux/static_call.h>
> > +#include <asm/qspinlock_paravirt.h>
> > +#include <asm/sbi.h>
> > +
> > +void pv_kick(int cpu)
> > +{
> > + return;
> > +}
> > +
> > +void pv_wait(u8 *ptr, u8 val)
> > +{
> > + unsigned long flags;
> > +
> > + if (in_nmi())
> > + return;
> > +
> > + local_irq_save(flags);
> > + if (READ_ONCE(*ptr) != val)
> > + goto out;
> > +
> > + /* wait_for_interrupt(); */
> > +out:
> > + local_irq_restore(flags);
> > +}
> > +
> > +static void native_queued_spin_unlock(struct qspinlock *lock)
> > +{
> > + smp_store_release(&lock->locked, 0);
> > +}
> > +
> > +DEFINE_STATIC_CALL(pv_queued_spin_lock_slowpath, native_queued_spin_lock_slowpath);
> > +EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> > +
> > +DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> > +EXPORT_STATIC_CALL(pv_queued_spin_unlock);
> > +
> > +void __init pv_qspinlock_init(void)
> > +{
> > + if (num_possible_cpus() == 1)
> > + return;
> > +
> > + if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
>
> Checks like this seem to be very common on this patchset.
> For someone not much familiar with this, it can be hard to
> understand.
>
> I mean, on patch 8/17 you introduce those IDs, which look to be
> incremental ( ID == N includes stuff from ID < N ), but I am not sure as I
> couln't find much documentation on that.
It's from sbi spec:
https://github.com/riscv-non-isa/riscv-sbi-doc/releases
0 Berkeley Boot Loader (BBL)
1 OpenSBI
2 Xvisor
3 KVM
4 RustSBI
5 Diosix
6 Coffer
7 Xen Project
8 PolarFire Hart Software Service
>
> Then above you test for the id being different than
> SBI_EXT_BASE_IMPL_ID_KVM, but if they are actually incremental and a new
> version lands, the new version will also return early because it passes the
> test.
>
> I am no sure if above is right, but it's all I could understand without
> documentation.
>
> Well, my point is: this seems hard to understand & review, so it would be
> nice to have a macro like this to be used instead:
>
> #define sbi_fw_implements_kvm() \
> (sbi_get_firmware_id() >= SBI_EXT_BASE_IMPL_ID_KVM)
No, it's not correct. It must be:
(sbi_get_firmware_id() == SBI_EXT_BASE_IMPL_ID_KVM)
>
> if(!sbi_fw_implements_kvm())
I'm okay with sbi_fw_implements_kvm().
> return;
>
> What do you think?
>
> Other than that, LGTM.
>
> Thanks!
> Leo
>
> > + return;
> > +
> > + pr_info("PV qspinlocks enabled\n");
> > + __pv_init_lock_hash();
> > +
> > + static_call_update(pv_queued_spin_lock_slowpath, __pv_queued_spin_lock_slowpath);
> > + static_call_update(pv_queued_spin_unlock, __pv_queued_spin_unlock);
> > +}
> > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > index c57d15b05160..88690751f2ee 100644
> > --- a/arch/riscv/kernel/setup.c
> > +++ b/arch/riscv/kernel/setup.c
> > @@ -321,6 +321,10 @@ static void __init riscv_spinlock_init(void)
> > #ifdef CONFIG_QUEUED_SPINLOCKS
> > virt_spin_lock_init();
> > #endif
> > +
> > +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> > + pv_qspinlock_init();
> > +#endif
> > }
> >
> > extern void __init init_rt_signal_env(void);
> > --
> > 2.36.1
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 11/17] RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
@ 2023-09-17 14:58 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 14:58 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 1:42 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:29:05AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Using static_call to switch between:
> > native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath()
> > native_queued_spin_unlock() __pv_queued_spin_unlock()
> >
> > Finish the pv_wait implementation, but pv_kick needs the SBI
> > definition of the next patches.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/include/asm/Kbuild | 1 -
> > arch/riscv/include/asm/qspinlock.h | 35 +++++++++++++
> > arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++++++++
> > arch/riscv/include/asm/spinlock.h | 2 +-
> > arch/riscv/kernel/qspinlock_paravirt.c | 57 +++++++++++++++++++++
> > arch/riscv/kernel/setup.c | 4 ++
> > 6 files changed, 126 insertions(+), 2 deletions(-)
> > create mode 100644 arch/riscv/include/asm/qspinlock.h
> > create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> > create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> >
> > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > index a0dc85e4a754..b89cb3b73c13 100644
> > --- a/arch/riscv/include/asm/Kbuild
> > +++ b/arch/riscv/include/asm/Kbuild
> > @@ -7,6 +7,5 @@ generic-y += parport.h
> > generic-y += spinlock_types.h
> > generic-y += qrwlock.h
> > generic-y += qrwlock_types.h
> > -generic-y += qspinlock.h
> > generic-y += user.h
> > generic-y += vmlinux.lds.h
> > diff --git a/arch/riscv/include/asm/qspinlock.h b/arch/riscv/include/asm/qspinlock.h
> > new file mode 100644
> > index 000000000000..7d4f416c908c
> > --- /dev/null
> > +++ b/arch/riscv/include/asm/qspinlock.h
> > @@ -0,0 +1,35 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c), 2023 Alibaba Cloud
> > + * Authors:
> > + * Guo Ren <guoren@linux.alibaba.com>
> > + */
> > +
> > +#ifndef _ASM_RISCV_QSPINLOCK_H
> > +#define _ASM_RISCV_QSPINLOCK_H
> > +
> > +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> > +#include <asm/qspinlock_paravirt.h>
> > +
> > +/* How long a lock should spin before we consider blocking */
> > +#define SPIN_THRESHOLD (1 << 15)
> > +
> > +void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > +void __pv_init_lock_hash(void);
> > +void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > +
> > +static inline void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
> > +{
> > + static_call(pv_queued_spin_lock_slowpath)(lock, val);
> > +}
> > +
> > +#define queued_spin_unlock queued_spin_unlock
> > +static inline void queued_spin_unlock(struct qspinlock *lock)
> > +{
> > + static_call(pv_queued_spin_unlock)(lock);
> > +}
> > +#endif /* CONFIG_PARAVIRT_SPINLOCKS */
> > +
> > +#include <asm-generic/qspinlock.h>
> > +
> > +#endif /* _ASM_RISCV_QSPINLOCK_H */
> > diff --git a/arch/riscv/include/asm/qspinlock_paravirt.h b/arch/riscv/include/asm/qspinlock_paravirt.h
> > new file mode 100644
> > index 000000000000..9681e851f69d
> > --- /dev/null
> > +++ b/arch/riscv/include/asm/qspinlock_paravirt.h
> > @@ -0,0 +1,29 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Copyright (c), 2023 Alibaba Cloud
> > + * Authors:
> > + * Guo Ren <guoren@linux.alibaba.com>
> > + */
> > +
> > +#ifndef _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> > +#define _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> > +
> > +void pv_wait(u8 *ptr, u8 val);
> > +void pv_kick(int cpu);
> > +
> > +void dummy_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > +void dummy_queued_spin_unlock(struct qspinlock *lock);
> > +
> > +DECLARE_STATIC_CALL(pv_queued_spin_lock_slowpath, dummy_queued_spin_lock_slowpath);
> > +DECLARE_STATIC_CALL(pv_queued_spin_unlock, dummy_queued_spin_unlock);
> > +
> > +void __init pv_qspinlock_init(void);
> > +
> > +static inline bool pv_is_native_spin_unlock(void)
> > +{
> > + return false;
> > +}
> > +
> > +void __pv_queued_spin_unlock(struct qspinlock *lock);
> > +
> > +#endif /* _ASM_RISCV_QSPINLOCK_PARAVIRT_H */
> > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > index 6b38d6616f14..ed4253f491fe 100644
> > --- a/arch/riscv/include/asm/spinlock.h
> > +++ b/arch/riscv/include/asm/spinlock.h
> > @@ -39,7 +39,7 @@ static inline bool virt_spin_lock(struct qspinlock *lock)
> > #undef arch_spin_trylock
> > #undef arch_spin_unlock
> >
> > -#include <asm-generic/qspinlock.h>
> > +#include <asm/qspinlock.h>
> > #include <linux/jump_label.h>
> >
> > #undef arch_spin_is_locked
> > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > new file mode 100644
> > index 000000000000..85ff5a3ec234
> > --- /dev/null
> > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > @@ -0,0 +1,57 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c), 2023 Alibaba Cloud
> > + * Authors:
> > + * Guo Ren <guoren@linux.alibaba.com>
> > + */
> > +
> > +#include <linux/static_call.h>
> > +#include <asm/qspinlock_paravirt.h>
> > +#include <asm/sbi.h>
> > +
> > +void pv_kick(int cpu)
> > +{
> > + return;
> > +}
> > +
> > +void pv_wait(u8 *ptr, u8 val)
> > +{
> > + unsigned long flags;
> > +
> > + if (in_nmi())
> > + return;
> > +
> > + local_irq_save(flags);
> > + if (READ_ONCE(*ptr) != val)
> > + goto out;
> > +
> > + /* wait_for_interrupt(); */
> > +out:
> > + local_irq_restore(flags);
> > +}
> > +
> > +static void native_queued_spin_unlock(struct qspinlock *lock)
> > +{
> > + smp_store_release(&lock->locked, 0);
> > +}
> > +
> > +DEFINE_STATIC_CALL(pv_queued_spin_lock_slowpath, native_queued_spin_lock_slowpath);
> > +EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> > +
> > +DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> > +EXPORT_STATIC_CALL(pv_queued_spin_unlock);
> > +
> > +void __init pv_qspinlock_init(void)
> > +{
> > + if (num_possible_cpus() == 1)
> > + return;
> > +
> > + if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
>
> Checks like this seem to be very common on this patchset.
> For someone not much familiar with this, it can be hard to
> understand.
>
> I mean, on patch 8/17 you introduce those IDs, which look to be
> incremental ( ID == N includes stuff from ID < N ), but I am not sure as I
> couln't find much documentation on that.
It's from sbi spec:
https://github.com/riscv-non-isa/riscv-sbi-doc/releases
0 Berkeley Boot Loader (BBL)
1 OpenSBI
2 Xvisor
3 KVM
4 RustSBI
5 Diosix
6 Coffer
7 Xen Project
8 PolarFire Hart Software Service
>
> Then above you test for the id being different than
> SBI_EXT_BASE_IMPL_ID_KVM, but if they are actually incremental and a new
> version lands, the new version will also return early because it passes the
> test.
>
> I am no sure if above is right, but it's all I could understand without
> documentation.
>
> Well, my point is: this seems hard to understand & review, so it would be
> nice to have a macro like this to be used instead:
>
> #define sbi_fw_implements_kvm() \
> (sbi_get_firmware_id() >= SBI_EXT_BASE_IMPL_ID_KVM)
No, it's not correct. It must be:
(sbi_get_firmware_id() == SBI_EXT_BASE_IMPL_ID_KVM)
>
> if(!sbi_fw_implements_kvm())
I'm okay with sbi_fw_implements_kvm().
> return;
>
> What do you think?
>
> Other than that, LGTM.
>
> Thanks!
> Leo
>
> > + return;
> > +
> > + pr_info("PV qspinlocks enabled\n");
> > + __pv_init_lock_hash();
> > +
> > + static_call_update(pv_queued_spin_lock_slowpath, __pv_queued_spin_lock_slowpath);
> > + static_call_update(pv_queued_spin_unlock, __pv_queued_spin_unlock);
> > +}
> > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > index c57d15b05160..88690751f2ee 100644
> > --- a/arch/riscv/kernel/setup.c
> > +++ b/arch/riscv/kernel/setup.c
> > @@ -321,6 +321,10 @@ static void __init riscv_spinlock_init(void)
> > #ifdef CONFIG_QUEUED_SPINLOCKS
> > virt_spin_lock_init();
> > #endif
> > +
> > +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> > + pv_qspinlock_init();
> > +#endif
> > }
> >
> > extern void __init init_rt_signal_env(void);
> > --
> > 2.36.1
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
2023-09-15 9:08 ` Leonardo Bras
@ 2023-09-17 15:02 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 15:02 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 5:08 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Fri, Sep 15, 2023 at 10:10:25AM +0800, Guo Ren wrote:
> > On Thu, Sep 14, 2023 at 5:43 PM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote:
> > > > On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > > >
> > > > > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > > >
> > > > > > The requirements of qspinlock have been documented by commit:
> > > > > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > > > > > atomics").
> > > > > >
> > > > > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > > > > > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > > > > > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > > > > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > > > > > is the one.
> > > > > >
> > > > > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > > > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > > > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > > > > >
> > > > > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > > > > queued_spinlock 0.5109/0.00
> > > > > > ticket_spinlock 0.5814/0.00
> > > > > >
> > > > > > perf futex/hash (+6.7%):
> > > > > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > > > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > > > > >
> > > > > > perf futex/wake-parallel (+8.6%):
> > > > > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > > > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > > > > >
> > > > > > perf futex/requeue (+4.2%):
> > > > > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > > > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > > > > >
> > > > > > System Benchmarks (+6.4%)
> > > > > > queued_spinlock:
> > > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > > > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > > > > Execl Throughput 43.0 13116.6 3050.4
> > > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > > > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > > > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > > > > Process Creation 126.0 12537.3 995.0
> > > > > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > > > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > > > > System Call Overhead 15000.0 33308301.3 22205.5
> > > > > > ========
> > > > > > System Benchmarks Index Score 12426.1
> > > > > >
> > > > > > ticket_spinlock:
> > > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > > > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > > > > Execl Throughput 43.0 12625.1 2936.1
> > > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > > > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > > > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > > > > Process Creation 126.0 10798.4 857.0
> > > > > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > > > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > > > > System Call Overhead 15000.0 30766778.4 20511.2
> > > > > > ========
> > > > > > System Benchmarks Index Score 11670.7
> > > > > >
> > > > > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > > > > cores platform than the ticket_lock.
> > > > > >
> > > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > > ---
> > > > > > arch/riscv/Kconfig | 16 ++++++++++++++++
> > > > > > arch/riscv/include/asm/Kbuild | 3 ++-
> > > > > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > > > > > 3 files changed, 35 insertions(+), 1 deletion(-)
> > > > > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > > > > >
> > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > > index 2c346fe169c1..7f39bfc75744 100644
> > > > > > --- a/arch/riscv/Kconfig
> > > > > > +++ b/arch/riscv/Kconfig
> > > > > > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > > > > > Specify the maximum number of NUMA Nodes available on the target
> > > > > > system. Increases memory reserved to accommodate various tables.
> > > > > >
> > > > > > +choice
> > > > > > + prompt "RISC-V spinlock type"
> > > > > > + default RISCV_TICKET_SPINLOCKS
> > > > > > +
> > > > > > +config RISCV_TICKET_SPINLOCKS
> > > > > > + bool "Using ticket spinlock"
> > > > > > +
> > > > > > +config RISCV_QUEUED_SPINLOCKS
> > > > > > + bool "Using queued spinlock"
> > > > > > + depends on SMP && MMU
> > > > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > > > + help
> > > > > > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > > > + Otherwise, stay at ticket-lock.
> > > > > > +endchoice
> > > > > > +
> > > > > > config RISCV_ALTERNATIVE
> > > > > > bool
> > > > > > depends on !XIP_KERNEL
> > > > > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > > > > index 504f8b7e72d4..a0dc85e4a754 100644
> > > > > > --- a/arch/riscv/include/asm/Kbuild
> > > > > > +++ b/arch/riscv/include/asm/Kbuild
> > > > > > @@ -2,10 +2,11 @@
> > > > > > generic-y += early_ioremap.h
> > > > > > generic-y += flat.h
> > > > > > generic-y += kvm_para.h
> > > > > > +generic-y += mcs_spinlock.h
> > > > > > generic-y += parport.h
> > > > > > -generic-y += spinlock.h
> > > > >
> > > > > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> > > > > and include the asm-generic headers of mcs_spinlock and qspinlock.
> > > > >
> > > > > In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> > > > > which seems the oposite of the above description (ticket spinlocks being
> > > > > the standard).
> > > > >
> > > > > Shouldn't ticket-spinlock.h also get included here?
> > > > > (Also, I am probably missing something, as I dont' see the use of
> > > > > mcs_spinlock here.)
> > > > No, because asm-generic/spinlock.h:
> > > > ...
> > > > #include <asm-generic/ticket_spinlock.h>
> > > > ...
> > > >
> > >
> > > But aren't you removing asm-generic/spinlock.h below ?
> > > -generic-y += spinlock.h
> > Yes, current is:
> >
> > arch/riscv/include/asm/spinlock.h -> include/asm-generic/spinlock.h ->
> > include/asm-generic/ticket_spinlock.h
>
> I did a little reading on how generic-y works (which I was unaware):
>
> "If an architecture uses a verbatim copy of a header from
> include/asm-generic then this is listed in the file
> arch/$(SRCARCH)/include/asm/Kbuild [...] During the prepare phase of the
> build a wrapper include file is generated in the directory [...]"
>
> Oh, so you are removing the asm-generic/spinlock.h because it's link
> was replaced by a new asm/spinlock.h.
>
> You add qspinlock.h to generic-y because it's new in riscv, and add
> mcs_spinlock.h because it's needed by qspinlock.h.
>
> Ok, it makes sense now.
>
> Sorry about this noise.
> I was unaware of how generic-y worked, and (wrongly)
> assumed it was about including headers automatically in the build.
>
>
> >
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > +#include <asm/qspinlock.h>
> > +#include <asm/qrwlock.h>
> > +#else
> > +#include <asm-generic/spinlock.h>
> > +#endif
> >
> > So, you want me:
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > +#include <asm/qspinlock.h>
> > +#else
> > +#include <asm-generic/ticket_spinlock.h>
> > +#endif
> >
> > +#include <asm/qrwlock.h>
> >
> > Right?
>
> No, I didn't mean that.
> I was just worried about the arch_spin_*() interfaces, but they should be
> fine.
>
> BTW, according to kernel doc on generic-y, shouldn't be a better idea to
> add 'ticket_spinlock.h' to generic-y, and include above as
> asm/ticket_spinlock.h?
>
> Or is generic-y reserved only for stuff which is indirectly included by
> other headers?
It's okay to add generic-y for ticket_spinlock.h, and I'm okay with
the following:
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm/qspinlock.h>
+#else
+#include <asm/ticket_spinlock.h>
+#endif
+#include <asm/qrwlock.h>
>
> Thanks!
> Leo
>
> >
> > >
> > > > >
> > > > > > generic-y += spinlock_types.h
> > > > > > generic-y += qrwlock.h
> > > > > > generic-y += qrwlock_types.h
> > > > > > +generic-y += qspinlock.h
> > > > > > generic-y += user.h
> > > > > > generic-y += vmlinux.lds.h
> > > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > > new file mode 100644
> > > > > > index 000000000000..c644a92d4548
> > > > > > --- /dev/null
> > > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > > @@ -0,0 +1,17 @@
> > > > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > > > +
> > > > > > +#ifndef __ASM_RISCV_SPINLOCK_H
> > > > > > +#define __ASM_RISCV_SPINLOCK_H
> > > > > > +
> > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > > +#define _Q_PENDING_LOOPS (1 << 9)
> > > > > > +#endif
> > > > >
> > > > > Any reason the above define couldn't be merged on the ifdef below?
> > > > Easy for the next patch to modify. See Waiman's comment:
> > > >
> > > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> > > >
> > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > index c644a92d4548..9eb3ad31e564 100644
> > > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > @@ -7,11 +7,94 @@
> > > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > > #endif
> > > > >
> > > >
> > > > I see why you separated the _Q_PENDING_LOOPS out.
> > > >
> > >
> > > I see, should be fine then.
> > >
> > > Thanks!
> > > Leo
> > >
> > > >
> > > > >
> > > > > > +
> > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > > +#include <asm/qspinlock.h>
> > > > > > +#include <asm/qrwlock.h>
> > > > > > +#else
> > > > > > +#include <asm-generic/spinlock.h>
> > > > > > +#endif
> > > > > > +
> > > > > > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > > > > > --
> > > > > > 2.36.1
> > > > > >
> > > > >
> > > > > Thanks!
> > > > > Leo
> > > > >
> > > >
> > > >
> > > > --
> > > > Best Regards
> > > > Guo Ren
> > > >
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
@ 2023-09-17 15:02 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 15:02 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 5:08 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Fri, Sep 15, 2023 at 10:10:25AM +0800, Guo Ren wrote:
> > On Thu, Sep 14, 2023 at 5:43 PM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote:
> > > > On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > > >
> > > > > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > > >
> > > > > > The requirements of qspinlock have been documented by commit:
> > > > > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > > > > > atomics").
> > > > > >
> > > > > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > > > > > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > > > > > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > > > > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > > > > > is the one.
> > > > > >
> > > > > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > > > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > > > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > > > > >
> > > > > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > > > > queued_spinlock 0.5109/0.00
> > > > > > ticket_spinlock 0.5814/0.00
> > > > > >
> > > > > > perf futex/hash (+6.7%):
> > > > > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > > > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > > > > >
> > > > > > perf futex/wake-parallel (+8.6%):
> > > > > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > > > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > > > > >
> > > > > > perf futex/requeue (+4.2%):
> > > > > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > > > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > > > > >
> > > > > > System Benchmarks (+6.4%)
> > > > > > queued_spinlock:
> > > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > > > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > > > > Execl Throughput 43.0 13116.6 3050.4
> > > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > > > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > > > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > > > > Process Creation 126.0 12537.3 995.0
> > > > > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > > > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > > > > System Call Overhead 15000.0 33308301.3 22205.5
> > > > > > ========
> > > > > > System Benchmarks Index Score 12426.1
> > > > > >
> > > > > > ticket_spinlock:
> > > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > > > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > > > > Execl Throughput 43.0 12625.1 2936.1
> > > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > > > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > > > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > > > > Process Creation 126.0 10798.4 857.0
> > > > > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > > > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > > > > System Call Overhead 15000.0 30766778.4 20511.2
> > > > > > ========
> > > > > > System Benchmarks Index Score 11670.7
> > > > > >
> > > > > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > > > > cores platform than the ticket_lock.
> > > > > >
> > > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > > ---
> > > > > > arch/riscv/Kconfig | 16 ++++++++++++++++
> > > > > > arch/riscv/include/asm/Kbuild | 3 ++-
> > > > > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > > > > > 3 files changed, 35 insertions(+), 1 deletion(-)
> > > > > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > > > > >
> > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > > index 2c346fe169c1..7f39bfc75744 100644
> > > > > > --- a/arch/riscv/Kconfig
> > > > > > +++ b/arch/riscv/Kconfig
> > > > > > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > > > > > Specify the maximum number of NUMA Nodes available on the target
> > > > > > system. Increases memory reserved to accommodate various tables.
> > > > > >
> > > > > > +choice
> > > > > > + prompt "RISC-V spinlock type"
> > > > > > + default RISCV_TICKET_SPINLOCKS
> > > > > > +
> > > > > > +config RISCV_TICKET_SPINLOCKS
> > > > > > + bool "Using ticket spinlock"
> > > > > > +
> > > > > > +config RISCV_QUEUED_SPINLOCKS
> > > > > > + bool "Using queued spinlock"
> > > > > > + depends on SMP && MMU
> > > > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > > > + help
> > > > > > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > > > + Otherwise, stay at ticket-lock.
> > > > > > +endchoice
> > > > > > +
> > > > > > config RISCV_ALTERNATIVE
> > > > > > bool
> > > > > > depends on !XIP_KERNEL
> > > > > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > > > > index 504f8b7e72d4..a0dc85e4a754 100644
> > > > > > --- a/arch/riscv/include/asm/Kbuild
> > > > > > +++ b/arch/riscv/include/asm/Kbuild
> > > > > > @@ -2,10 +2,11 @@
> > > > > > generic-y += early_ioremap.h
> > > > > > generic-y += flat.h
> > > > > > generic-y += kvm_para.h
> > > > > > +generic-y += mcs_spinlock.h
> > > > > > generic-y += parport.h
> > > > > > -generic-y += spinlock.h
> > > > >
> > > > > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> > > > > and include the asm-generic headers of mcs_spinlock and qspinlock.
> > > > >
> > > > > In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> > > > > which seems the oposite of the above description (ticket spinlocks being
> > > > > the standard).
> > > > >
> > > > > Shouldn't ticket-spinlock.h also get included here?
> > > > > (Also, I am probably missing something, as I dont' see the use of
> > > > > mcs_spinlock here.)
> > > > No, because asm-generic/spinlock.h:
> > > > ...
> > > > #include <asm-generic/ticket_spinlock.h>
> > > > ...
> > > >
> > >
> > > But aren't you removing asm-generic/spinlock.h below ?
> > > -generic-y += spinlock.h
> > Yes, current is:
> >
> > arch/riscv/include/asm/spinlock.h -> include/asm-generic/spinlock.h ->
> > include/asm-generic/ticket_spinlock.h
>
> I did a little reading on how generic-y works (which I was unaware):
>
> "If an architecture uses a verbatim copy of a header from
> include/asm-generic then this is listed in the file
> arch/$(SRCARCH)/include/asm/Kbuild [...] During the prepare phase of the
> build a wrapper include file is generated in the directory [...]"
>
> Oh, so you are removing the asm-generic/spinlock.h because it's link
> was replaced by a new asm/spinlock.h.
>
> You add qspinlock.h to generic-y because it's new in riscv, and add
> mcs_spinlock.h because it's needed by qspinlock.h.
>
> Ok, it makes sense now.
>
> Sorry about this noise.
> I was unaware of how generic-y worked, and (wrongly)
> assumed it was about including headers automatically in the build.
>
>
> >
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > +#include <asm/qspinlock.h>
> > +#include <asm/qrwlock.h>
> > +#else
> > +#include <asm-generic/spinlock.h>
> > +#endif
> >
> > So, you want me:
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > +#include <asm/qspinlock.h>
> > +#else
> > +#include <asm-generic/ticket_spinlock.h>
> > +#endif
> >
> > +#include <asm/qrwlock.h>
> >
> > Right?
>
> No, I didn't mean that.
> I was just worried about the arch_spin_*() interfaces, but they should be
> fine.
>
> BTW, according to kernel doc on generic-y, shouldn't be a better idea to
> add 'ticket_spinlock.h' to generic-y, and include above as
> asm/ticket_spinlock.h?
>
> Or is generic-y reserved only for stuff which is indirectly included by
> other headers?
It's okay to add generic-y for ticket_spinlock.h, and I'm okay with
the following:
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm/qspinlock.h>
+#else
+#include <asm/ticket_spinlock.h>
+#endif
+#include <asm/qrwlock.h>
>
> Thanks!
> Leo
>
> >
> > >
> > > > >
> > > > > > generic-y += spinlock_types.h
> > > > > > generic-y += qrwlock.h
> > > > > > generic-y += qrwlock_types.h
> > > > > > +generic-y += qspinlock.h
> > > > > > generic-y += user.h
> > > > > > generic-y += vmlinux.lds.h
> > > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > > new file mode 100644
> > > > > > index 000000000000..c644a92d4548
> > > > > > --- /dev/null
> > > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > > @@ -0,0 +1,17 @@
> > > > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > > > +
> > > > > > +#ifndef __ASM_RISCV_SPINLOCK_H
> > > > > > +#define __ASM_RISCV_SPINLOCK_H
> > > > > > +
> > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > > +#define _Q_PENDING_LOOPS (1 << 9)
> > > > > > +#endif
> > > > >
> > > > > Any reason the above define couldn't be merged on the ifdef below?
> > > > Easy for the next patch to modify. See Waiman's comment:
> > > >
> > > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> > > >
> > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > index c644a92d4548..9eb3ad31e564 100644
> > > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > @@ -7,11 +7,94 @@
> > > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > > #endif
> > > > >
> > > >
> > > > I see why you separated the _Q_PENDING_LOOPS out.
> > > >
> > >
> > > I see, should be fine then.
> > >
> > > Thanks!
> > > Leo
> > >
> > > >
> > > > >
> > > > > > +
> > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > > +#include <asm/qspinlock.h>
> > > > > > +#include <asm/qrwlock.h>
> > > > > > +#else
> > > > > > +#include <asm-generic/spinlock.h>
> > > > > > +#endif
> > > > > > +
> > > > > > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > > > > > --
> > > > > > 2.36.1
> > > > > >
> > > > >
> > > > > Thanks!
> > > > > Leo
> > > > >
> > > >
> > > >
> > > > --
> > > > Best Regards
> > > > Guo Ren
> > > >
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 12/17] RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
2023-09-15 6:05 ` Leonardo Bras
@ 2023-09-17 15:03 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 15:03 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 2:05 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:29:06AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Disables the qspinlock slow path using PV optimizations which
> > allow the hypervisor to 'idle' the guest on lock contention.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > Documentation/admin-guide/kernel-parameters.txt | 2 +-
> > arch/riscv/kernel/qspinlock_paravirt.c | 13 +++++++++++++
> > 2 files changed, 14 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index f75bedc50e00..e74aed631573 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -3857,7 +3857,7 @@
> > as generic guest with no PV drivers. Currently support
> > XEN HVM, KVM, HYPER_V and VMWARE guest.
> >
> > - nopvspin [X86,XEN,KVM]
> > + nopvspin [X86,XEN,KVM,RISC-V]
> > Disables the qspinlock slow path using PV optimizations
> > which allow the hypervisor to 'idle' the guest on lock
> > contention.
> > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > index 85ff5a3ec234..a0ad4657f437 100644
> > --- a/arch/riscv/kernel/qspinlock_paravirt.c
> > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > @@ -41,8 +41,21 @@ EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> > DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> > EXPORT_STATIC_CALL(pv_queued_spin_unlock);
> >
> > +static bool nopvspin;
>
> It is only used in init, so it makes sense to add __initdata.
>
> static bool nopvspin __initdata;
Okay.
>
> Other than that, LGTM:
> Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
> Thanks!
> Leo
>
> > +static __init int parse_nopvspin(char *arg)
> > +{
> > + nopvspin = true;
> > + return 0;
> > +}
> > +early_param("nopvspin", parse_nopvspin);
> > +
> > void __init pv_qspinlock_init(void)
> > {
> > + if (nopvspin) {
> > + pr_info("PV qspinlocks disabled\n");
> > + return;
> > + }
> > +
> > if (num_possible_cpus() == 1)
> > return;
> >
> > --
> > 2.36.1
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 12/17] RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
@ 2023-09-17 15:03 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 15:03 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 2:05 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:29:06AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Disables the qspinlock slow path using PV optimizations which
> > allow the hypervisor to 'idle' the guest on lock contention.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > Documentation/admin-guide/kernel-parameters.txt | 2 +-
> > arch/riscv/kernel/qspinlock_paravirt.c | 13 +++++++++++++
> > 2 files changed, 14 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index f75bedc50e00..e74aed631573 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -3857,7 +3857,7 @@
> > as generic guest with no PV drivers. Currently support
> > XEN HVM, KVM, HYPER_V and VMWARE guest.
> >
> > - nopvspin [X86,XEN,KVM]
> > + nopvspin [X86,XEN,KVM,RISC-V]
> > Disables the qspinlock slow path using PV optimizations
> > which allow the hypervisor to 'idle' the guest on lock
> > contention.
> > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > index 85ff5a3ec234..a0ad4657f437 100644
> > --- a/arch/riscv/kernel/qspinlock_paravirt.c
> > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > @@ -41,8 +41,21 @@ EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> > DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> > EXPORT_STATIC_CALL(pv_queued_spin_unlock);
> >
> > +static bool nopvspin;
>
> It is only used in init, so it makes sense to add __initdata.
>
> static bool nopvspin __initdata;
Okay.
>
> Other than that, LGTM:
> Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
> Thanks!
> Leo
>
> > +static __init int parse_nopvspin(char *arg)
> > +{
> > + nopvspin = true;
> > + return 0;
> > +}
> > +early_param("nopvspin", parse_nopvspin);
> > +
> > void __init pv_qspinlock_init(void)
> > {
> > + if (nopvspin) {
> > + pr_info("PV qspinlocks disabled\n");
> > + return;
> > + }
> > +
> > if (num_possible_cpus() == 1)
> > return;
> >
> > --
> > 2.36.1
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 13/17] RISC-V: paravirt: pvqspinlock: Add SBI implementation
2023-09-15 6:23 ` Leonardo Bras
@ 2023-09-17 15:06 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 15:06 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 2:23 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:29:07AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Implement pv_kick with SBI implementation, and add SBI_EXT_PVLOCK
> > extension detection.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/include/asm/sbi.h | 6 ++++++
> > arch/riscv/kernel/qspinlock_paravirt.c | 7 ++++++-
> > 2 files changed, 12 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > index e0233b3d7a5f..3533f8d4f3e2 100644
> > --- a/arch/riscv/include/asm/sbi.h
> > +++ b/arch/riscv/include/asm/sbi.h
> > @@ -30,6 +30,7 @@ enum sbi_ext_id {
> > SBI_EXT_HSM = 0x48534D,
> > SBI_EXT_SRST = 0x53525354,
> > SBI_EXT_PMU = 0x504D55,
> > + SBI_EXT_PVLOCK = 0xAB0401,
> >
> > /* Experimentals extensions must lie within this range */
> > SBI_EXT_EXPERIMENTAL_START = 0x08000000,
> > @@ -243,6 +244,11 @@ enum sbi_pmu_ctr_type {
> > /* Flags defined for counter stop function */
> > #define SBI_PMU_STOP_FLAG_RESET (1 << 0)
> >
> > +/* SBI PVLOCK (kick cpu out of wfi) */
> > +enum sbi_ext_pvlock_fid {
> > + SBI_EXT_PVLOCK_KICK_CPU = 0,
> > +};
> > +
> > #define SBI_SPEC_VERSION_DEFAULT 0x1
> > #define SBI_SPEC_VERSION_MAJOR_SHIFT 24
> > #define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
> > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > index a0ad4657f437..571626f350be 100644
> > --- a/arch/riscv/kernel/qspinlock_paravirt.c
> > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > @@ -11,6 +11,8 @@
> >
> > void pv_kick(int cpu)
> > {
> > + sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
> > + cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
> > return;
> > }
> >
> > @@ -25,7 +27,7 @@ void pv_wait(u8 *ptr, u8 val)
> > if (READ_ONCE(*ptr) != val)
> > goto out;
> >
> > - /* wait_for_interrupt(); */
> > + wait_for_interrupt();
> > out:
> > local_irq_restore(flags);
> > }
> > @@ -62,6 +64,9 @@ void __init pv_qspinlock_init(void)
> > if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
> > return;
> >
> > + if (!sbi_probe_extension(SBI_EXT_PVLOCK))
> > + return;
> > +
> > pr_info("PV qspinlocks enabled\n");
> > __pv_init_lock_hash();
> >
> > --
> > 2.36.1
> >
>
> IIUC this PVLOCK extension is now a requirement to use pv_qspinlock(), and
> it allows a cpu to use an instruction to wait for interrupt in pv_wait(),
> and kicks it out of this wait using a new sbi_ecall() on pv_kick().
Yes.
>
> Overall it LGTM, but would be nice to have the reference doc in the commit
> msg. I end up inferring some of the inner workings by your implementation,
> which is not ideal for reviewing.
I would improve the commit msg in the next version of patch.
>
> If understanding above is right,
> Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
> Thanks!
> Leo
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 13/17] RISC-V: paravirt: pvqspinlock: Add SBI implementation
@ 2023-09-17 15:06 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 15:06 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Fri, Sep 15, 2023 at 2:23 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:29:07AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Implement pv_kick with SBI implementation, and add SBI_EXT_PVLOCK
> > extension detection.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/include/asm/sbi.h | 6 ++++++
> > arch/riscv/kernel/qspinlock_paravirt.c | 7 ++++++-
> > 2 files changed, 12 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > index e0233b3d7a5f..3533f8d4f3e2 100644
> > --- a/arch/riscv/include/asm/sbi.h
> > +++ b/arch/riscv/include/asm/sbi.h
> > @@ -30,6 +30,7 @@ enum sbi_ext_id {
> > SBI_EXT_HSM = 0x48534D,
> > SBI_EXT_SRST = 0x53525354,
> > SBI_EXT_PMU = 0x504D55,
> > + SBI_EXT_PVLOCK = 0xAB0401,
> >
> > /* Experimentals extensions must lie within this range */
> > SBI_EXT_EXPERIMENTAL_START = 0x08000000,
> > @@ -243,6 +244,11 @@ enum sbi_pmu_ctr_type {
> > /* Flags defined for counter stop function */
> > #define SBI_PMU_STOP_FLAG_RESET (1 << 0)
> >
> > +/* SBI PVLOCK (kick cpu out of wfi) */
> > +enum sbi_ext_pvlock_fid {
> > + SBI_EXT_PVLOCK_KICK_CPU = 0,
> > +};
> > +
> > #define SBI_SPEC_VERSION_DEFAULT 0x1
> > #define SBI_SPEC_VERSION_MAJOR_SHIFT 24
> > #define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
> > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > index a0ad4657f437..571626f350be 100644
> > --- a/arch/riscv/kernel/qspinlock_paravirt.c
> > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > @@ -11,6 +11,8 @@
> >
> > void pv_kick(int cpu)
> > {
> > + sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
> > + cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
> > return;
> > }
> >
> > @@ -25,7 +27,7 @@ void pv_wait(u8 *ptr, u8 val)
> > if (READ_ONCE(*ptr) != val)
> > goto out;
> >
> > - /* wait_for_interrupt(); */
> > + wait_for_interrupt();
> > out:
> > local_irq_restore(flags);
> > }
> > @@ -62,6 +64,9 @@ void __init pv_qspinlock_init(void)
> > if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
> > return;
> >
> > + if (!sbi_probe_extension(SBI_EXT_PVLOCK))
> > + return;
> > +
> > pr_info("PV qspinlocks enabled\n");
> > __pv_init_lock_hash();
> >
> > --
> > 2.36.1
> >
>
> IIUC this PVLOCK extension is now a requirement to use pv_qspinlock(), and
> it allows a cpu to use an instruction to wait for interrupt in pv_wait(),
> and kicks it out of this wait using a new sbi_ecall() on pv_kick().
Yes.
>
> Overall it LGTM, but would be nice to have the reference doc in the commit
> msg. I end up inferring some of the inner workings by your implementation,
> which is not ideal for reviewing.
I would improve the commit msg in the next version of patch.
>
> If understanding above is right,
> Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
> Thanks!
> Leo
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
2023-09-14 8:02 ` Leonardo Bras
@ 2023-09-17 15:12 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 15:12 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 4:02 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:29:02AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Add a static key controlling whether virt_spin_lock() should be
> > called or not. When running on bare metal set the new key to
> > false.
> >
> > The KVM guests fall back to a Test-and-Set spinlock, because fair
> > locks have horrible lock 'holder' preemption issues. The
> > virt_spin_lock_key would shortcut for the
> > queued_spin_lock_slowpath() function that allow virt_spin_lock to
> > hijack it.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > .../admin-guide/kernel-parameters.txt | 4 +++
> > arch/riscv/include/asm/sbi.h | 8 +++++
> > arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
> > arch/riscv/kernel/sbi.c | 2 +-
> > arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
> > 5 files changed, 64 insertions(+), 2 deletions(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 61cacb8dfd0e..f75bedc50e00 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -3927,6 +3927,10 @@
> > no_uaccess_flush
> > [PPC] Don't flush the L1-D cache after accessing user data.
> >
> > + no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
> > + native_queued_spinlock when the nopvspin option is enabled.
> > + This would help vcpu=pcpu scenarios.
> > +
> > novmcoredd [KNL,KDUMP]
> > Disable device dump. Device dump allows drivers to
> > append dump data to vmcore so you can collect driver
> > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > index 501e06e52078..e0233b3d7a5f 100644
> > --- a/arch/riscv/include/asm/sbi.h
> > +++ b/arch/riscv/include/asm/sbi.h
> > @@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
> > SBI_EXT_BASE_GET_MIMPID,
> > };
> >
> > +enum sbi_ext_base_impl_id {
> > + SBI_EXT_BASE_IMPL_ID_BBL = 0,
> > + SBI_EXT_BASE_IMPL_ID_OPENSBI,
> > + SBI_EXT_BASE_IMPL_ID_XVISOR,
> > + SBI_EXT_BASE_IMPL_ID_KVM,
> > +};
> > +
> > enum sbi_ext_time_fid {
> > SBI_EXT_TIME_SET_TIMER = 0,
> > };
> > @@ -269,6 +276,7 @@ int sbi_console_getchar(void);
> > long sbi_get_mvendorid(void);
> > long sbi_get_marchid(void);
> > long sbi_get_mimpid(void);
> > +long sbi_get_firmware_id(void);
> > void sbi_set_timer(uint64_t stime_value);
> > void sbi_shutdown(void);
> > void sbi_send_ipi(unsigned int cpu);
> > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > index 8ea0fee80652..6b38d6616f14 100644
> > --- a/arch/riscv/include/asm/spinlock.h
> > +++ b/arch/riscv/include/asm/spinlock.h
> > @@ -4,6 +4,28 @@
> > #define __ASM_RISCV_SPINLOCK_H
> >
> > #ifdef CONFIG_QUEUED_SPINLOCKS
> > +/*
> > + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
> > + * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
> > + * would shortcut for the queued_spin_lock_slowpath() function that allow
> > + * virt_spin_lock to hijack it.
> > + */
> > +DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > +
> > +#define virt_spin_lock virt_spin_lock
> > +static inline bool virt_spin_lock(struct qspinlock *lock)
> > +{
> > + if (!static_branch_likely(&virt_spin_lock_key))
> > + return false;
> > +
> > + do {
> > + while (atomic_read(&lock->val) != 0)
> > + cpu_relax();
> > + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
> > +
> > + return true;
> > +}
> > +
> > #define _Q_PENDING_LOOPS (1 << 9)
> > #endif
> >
> > diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> > index 88eea3a99ee0..cdd45edc8db4 100644
> > --- a/arch/riscv/kernel/sbi.c
> > +++ b/arch/riscv/kernel/sbi.c
> > @@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
> > return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
> > }
> >
> > -static inline long sbi_get_firmware_id(void)
> > +long sbi_get_firmware_id(void)
> > {
> > return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
> > }
> > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > index 0f084f037651..c57d15b05160 100644
> > --- a/arch/riscv/kernel/setup.c
> > +++ b/arch/riscv/kernel/setup.c
> > @@ -26,6 +26,7 @@
> > #include <asm/alternative.h>
> > #include <asm/cacheflush.h>
> > #include <asm/cpu_ops.h>
> > +#include <asm/cpufeature.h>
> > #include <asm/early_ioremap.h>
> > #include <asm/pgtable.h>
> > #include <asm/setup.h>
> > @@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > EXPORT_SYMBOL(combo_qspinlock_key);
> > #endif
> >
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > +static bool no_virt_spin_key = false;
>
> I suggest no _key, also there is no need for "= false".
> To be consistent with enable_qspinlock, I also suggest
> adding __ro_after_init:
>
> static bool no_virt_spin __ro_after_init;
okay.
>
>
>
> > +DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > +
> > +static int __init no_virt_spin_setup(char *p)
> > +{
> > + no_virt_spin_key = true;
> > +
> > + return 0;
> > +}
> > +early_param("no_virt_spin", no_virt_spin_setup);
> > +
> > +static void __init virt_spin_lock_init(void)
> > +{
> > + if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
> > + no_virt_spin_key)
> > + static_branch_disable(&virt_spin_lock_key);
> > + else
> > + pr_info("Enable virt_spin_lock\n");
> > +}
> > +#endif
> > +
>
> A new virt_no_spin kernel parameter was introduced, but without
> CONFIG_QUEUED_SPINLOCKS it will silently fail.
>
> I would suggest an #else clause here with a function to print an error /
> warning message about no_virt_spin being invalid in this scenario.
> It will probably help future debugging.
If CONFIG_QUEUED_SPINLOCKS=n, no_virt_spin should be quiet. The
no_virt_spin is one path of qspinlock.
>
>
> > static void __init riscv_spinlock_init(void)
> > {
> > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > - if (!enable_qspinlock_key) {
> > + if (!enable_qspinlock_key &&
> > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > static_branch_disable(&combo_qspinlock_key);
> > pr_info("Ticket spinlock: enabled\n");
> > } else {
> > pr_info("Queued spinlock: enabled\n");
> > }
> > #endif
> > +
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > + virt_spin_lock_init();
> > +#endif
> > }
> >
> > extern void __init init_rt_signal_env(void);
> > --
> > 2.36.1
> >
>
> I am probably missing something out, but it looks to me that this patch is
> causing 2 different changes:
> 1 - Enabling no_virt_spin parameter
> 2 - Disabling queued spinlocks for some firmware_id
>
> Wouldn't be better to split those changes in multiple patches?
> Or am I missing the point on why they need to be together?
>
> Thanks!
> Leo
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
@ 2023-09-17 15:12 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 15:12 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 4:02 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:29:02AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > Add a static key controlling whether virt_spin_lock() should be
> > called or not. When running on bare metal set the new key to
> > false.
> >
> > The KVM guests fall back to a Test-and-Set spinlock, because fair
> > locks have horrible lock 'holder' preemption issues. The
> > virt_spin_lock_key would shortcut for the
> > queued_spin_lock_slowpath() function that allow virt_spin_lock to
> > hijack it.
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > .../admin-guide/kernel-parameters.txt | 4 +++
> > arch/riscv/include/asm/sbi.h | 8 +++++
> > arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
> > arch/riscv/kernel/sbi.c | 2 +-
> > arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
> > 5 files changed, 64 insertions(+), 2 deletions(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 61cacb8dfd0e..f75bedc50e00 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -3927,6 +3927,10 @@
> > no_uaccess_flush
> > [PPC] Don't flush the L1-D cache after accessing user data.
> >
> > + no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
> > + native_queued_spinlock when the nopvspin option is enabled.
> > + This would help vcpu=pcpu scenarios.
> > +
> > novmcoredd [KNL,KDUMP]
> > Disable device dump. Device dump allows drivers to
> > append dump data to vmcore so you can collect driver
> > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > index 501e06e52078..e0233b3d7a5f 100644
> > --- a/arch/riscv/include/asm/sbi.h
> > +++ b/arch/riscv/include/asm/sbi.h
> > @@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
> > SBI_EXT_BASE_GET_MIMPID,
> > };
> >
> > +enum sbi_ext_base_impl_id {
> > + SBI_EXT_BASE_IMPL_ID_BBL = 0,
> > + SBI_EXT_BASE_IMPL_ID_OPENSBI,
> > + SBI_EXT_BASE_IMPL_ID_XVISOR,
> > + SBI_EXT_BASE_IMPL_ID_KVM,
> > +};
> > +
> > enum sbi_ext_time_fid {
> > SBI_EXT_TIME_SET_TIMER = 0,
> > };
> > @@ -269,6 +276,7 @@ int sbi_console_getchar(void);
> > long sbi_get_mvendorid(void);
> > long sbi_get_marchid(void);
> > long sbi_get_mimpid(void);
> > +long sbi_get_firmware_id(void);
> > void sbi_set_timer(uint64_t stime_value);
> > void sbi_shutdown(void);
> > void sbi_send_ipi(unsigned int cpu);
> > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > index 8ea0fee80652..6b38d6616f14 100644
> > --- a/arch/riscv/include/asm/spinlock.h
> > +++ b/arch/riscv/include/asm/spinlock.h
> > @@ -4,6 +4,28 @@
> > #define __ASM_RISCV_SPINLOCK_H
> >
> > #ifdef CONFIG_QUEUED_SPINLOCKS
> > +/*
> > + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
> > + * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
> > + * would shortcut for the queued_spin_lock_slowpath() function that allow
> > + * virt_spin_lock to hijack it.
> > + */
> > +DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > +
> > +#define virt_spin_lock virt_spin_lock
> > +static inline bool virt_spin_lock(struct qspinlock *lock)
> > +{
> > + if (!static_branch_likely(&virt_spin_lock_key))
> > + return false;
> > +
> > + do {
> > + while (atomic_read(&lock->val) != 0)
> > + cpu_relax();
> > + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
> > +
> > + return true;
> > +}
> > +
> > #define _Q_PENDING_LOOPS (1 << 9)
> > #endif
> >
> > diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> > index 88eea3a99ee0..cdd45edc8db4 100644
> > --- a/arch/riscv/kernel/sbi.c
> > +++ b/arch/riscv/kernel/sbi.c
> > @@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
> > return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
> > }
> >
> > -static inline long sbi_get_firmware_id(void)
> > +long sbi_get_firmware_id(void)
> > {
> > return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
> > }
> > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > index 0f084f037651..c57d15b05160 100644
> > --- a/arch/riscv/kernel/setup.c
> > +++ b/arch/riscv/kernel/setup.c
> > @@ -26,6 +26,7 @@
> > #include <asm/alternative.h>
> > #include <asm/cacheflush.h>
> > #include <asm/cpu_ops.h>
> > +#include <asm/cpufeature.h>
> > #include <asm/early_ioremap.h>
> > #include <asm/pgtable.h>
> > #include <asm/setup.h>
> > @@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > EXPORT_SYMBOL(combo_qspinlock_key);
> > #endif
> >
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > +static bool no_virt_spin_key = false;
>
> I suggest no _key, also there is no need for "= false".
> To be consistent with enable_qspinlock, I also suggest
> adding __ro_after_init:
>
> static bool no_virt_spin __ro_after_init;
okay.
>
>
>
> > +DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > +
> > +static int __init no_virt_spin_setup(char *p)
> > +{
> > + no_virt_spin_key = true;
> > +
> > + return 0;
> > +}
> > +early_param("no_virt_spin", no_virt_spin_setup);
> > +
> > +static void __init virt_spin_lock_init(void)
> > +{
> > + if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
> > + no_virt_spin_key)
> > + static_branch_disable(&virt_spin_lock_key);
> > + else
> > + pr_info("Enable virt_spin_lock\n");
> > +}
> > +#endif
> > +
>
> A new virt_no_spin kernel parameter was introduced, but without
> CONFIG_QUEUED_SPINLOCKS it will silently fail.
>
> I would suggest an #else clause here with a function to print an error /
> warning message about no_virt_spin being invalid in this scenario.
> It will probably help future debugging.
If CONFIG_QUEUED_SPINLOCKS=n, no_virt_spin should be quiet. The
no_virt_spin is one path of qspinlock.
>
>
> > static void __init riscv_spinlock_init(void)
> > {
> > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > - if (!enable_qspinlock_key) {
> > + if (!enable_qspinlock_key &&
> > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > static_branch_disable(&combo_qspinlock_key);
> > pr_info("Ticket spinlock: enabled\n");
> > } else {
> > pr_info("Queued spinlock: enabled\n");
> > }
> > #endif
> > +
> > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > + virt_spin_lock_init();
> > +#endif
> > }
> >
> > extern void __init init_rt_signal_env(void);
> > --
> > 2.36.1
> >
>
> I am probably missing something out, but it looks to me that this patch is
> causing 2 different changes:
> 1 - Enabling no_virt_spin parameter
> 2 - Disabling queued spinlocks for some firmware_id
>
> Wouldn't be better to split those changes in multiple patches?
> Or am I missing the point on why they need to be together?
>
> Thanks!
> Leo
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 09/17] riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
2023-09-14 8:32 ` Leonardo Bras
@ 2023-09-17 15:15 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 15:15 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 4:32 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:29:03AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The early version of T-Head C9xx cores has a store merge buffer
> > delay problem. The store merge buffer could improve the store queue
> > performance by merging multi-store requests, but when there are not
> > continued store requests, the prior single store request would be
> > waiting in the store queue for a long time. That would cause
> > significant problems for communication between multi-cores. This
> > problem was found on sg2042 & th1520 platforms with the qspinlock
> > lock torture test.
> >
> > So appending a fence w.o could immediately flush the store merge
> > buffer and let other cores see the write result.
> >
> > This will apply the WRITE_ONCE errata to handle the non-standard
> > behavior via appending a fence w.o instruction for WRITE_ONCE().
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig.errata | 19 +++++++++++++++++++
> > arch/riscv/errata/thead/errata.c | 20 ++++++++++++++++++++
> > arch/riscv/include/asm/errata_list.h | 13 -------------
> > arch/riscv/include/asm/rwonce.h | 24 ++++++++++++++++++++++++
> > arch/riscv/include/asm/vendorid_list.h | 14 ++++++++++++++
> > include/asm-generic/rwonce.h | 2 ++
> > 6 files changed, 79 insertions(+), 13 deletions(-)
> > create mode 100644 arch/riscv/include/asm/rwonce.h
> >
> > diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata
> > index 1aa85a427ff3..c919cc3f1a3a 100644
> > --- a/arch/riscv/Kconfig.errata
> > +++ b/arch/riscv/Kconfig.errata
> > @@ -77,4 +77,23 @@ config ERRATA_THEAD_PMU
> >
> > If you don't know what to do here, say "Y".
> >
> > +config ERRATA_THEAD_WRITE_ONCE
> > + bool "Apply T-Head WRITE_ONCE errata"
> > + depends on ERRATA_THEAD
> > + default y
> > + help
> > + The early version of T-Head C9xx cores has a store merge buffer
> > + delay problem. The store merge buffer could improve the store queue
> > + performance by merging multi-store requests, but when there are no
> > + continued store requests, the prior single store request would be
> > + waiting in the store queue for a long time. That would cause
> > + significant problems for communication between multi-cores. Appending
> > + a fence w.o could immediately flush the store merge buffer and let
> > + other cores see the write result.
> > +
> > + This will apply the WRITE_ONCE errata to handle the non-standard
> > + behavior via appending a fence w.o instruction for WRITE_ONCE().
> > +
> > + If you don't know what to do here, say "Y".
> > +
> > endmenu # "CPU errata selection"
> > diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
> > index be84b14f0118..751eb5a7f614 100644
> > --- a/arch/riscv/errata/thead/errata.c
> > +++ b/arch/riscv/errata/thead/errata.c
> > @@ -69,6 +69,23 @@ static bool errata_probe_pmu(unsigned int stage,
> > return true;
> > }
> >
> > +static bool errata_probe_write_once(unsigned int stage,
> > + unsigned long arch_id, unsigned long impid)
> > +{
> > + if (!IS_ENABLED(CONFIG_ERRATA_THEAD_WRITE_ONCE))
> > + return false;
> > +
> > + /* target-c9xx cores report arch_id and impid as 0 */
> > + if (arch_id != 0 || impid != 0)
> > + return false;
> > +
> > + if (stage == RISCV_ALTERNATIVES_BOOT ||
> > + stage == RISCV_ALTERNATIVES_MODULE)
> > + return true;
> > +
> > + return false;
> > +}
> > +
> > static u32 thead_errata_probe(unsigned int stage,
> > unsigned long archid, unsigned long impid)
> > {
> > @@ -83,6 +100,9 @@ static u32 thead_errata_probe(unsigned int stage,
> > if (errata_probe_pmu(stage, archid, impid))
> > cpu_req_errata |= BIT(ERRATA_THEAD_PMU);
> >
> > + if (errata_probe_write_once(stage, archid, impid))
> > + cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
> > +
> > return cpu_req_errata;
> > }
> >
> > diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
> > index 712cab7adffe..fbb2b8d39321 100644
> > --- a/arch/riscv/include/asm/errata_list.h
> > +++ b/arch/riscv/include/asm/errata_list.h
> > @@ -11,19 +11,6 @@
> > #include <asm/hwcap.h>
> > #include <asm/vendorid_list.h>
> >
> > -#ifdef CONFIG_ERRATA_SIFIVE
> > -#define ERRATA_SIFIVE_CIP_453 0
> > -#define ERRATA_SIFIVE_CIP_1200 1
> > -#define ERRATA_SIFIVE_NUMBER 2
> > -#endif
> > -
> > -#ifdef CONFIG_ERRATA_THEAD
> > -#define ERRATA_THEAD_PBMT 0
> > -#define ERRATA_THEAD_CMO 1
> > -#define ERRATA_THEAD_PMU 2
> > -#define ERRATA_THEAD_NUMBER 3
> > -#endif
> > -
>
> Here I understand you are moving stuff from errata_list.h to
> vendorid_list.h. Wouldn't it be better to do this on a separated patch
> before this one?
Okay.
>
> I understand this is used here, but it looks like it's unrelated.
>
> > #ifdef __ASSEMBLY__
> >
> > #define ALT_INSN_FAULT(x) \
> > diff --git a/arch/riscv/include/asm/rwonce.h b/arch/riscv/include/asm/rwonce.h
> > new file mode 100644
> > index 000000000000..be0b8864969d
> > --- /dev/null
> > +++ b/arch/riscv/include/asm/rwonce.h
> > @@ -0,0 +1,24 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +
> > +#ifndef __ASM_RWONCE_H
> > +#define __ASM_RWONCE_H
> > +
> > +#include <linux/compiler_types.h>
> > +#include <asm/alternative-macros.h>
> > +#include <asm/vendorid_list.h>
> > +
> > +#define __WRITE_ONCE(x, val) \
> > +do { \
> > + *(volatile typeof(x) *)&(x) = (val); \
> > + asm volatile(ALTERNATIVE( \
> > + __nops(1), \
> > + "fence w, o\n\t", \
> > + THEAD_VENDOR_ID, \
> > + ERRATA_THEAD_WRITE_ONCE, \
> > + CONFIG_ERRATA_THEAD_WRITE_ONCE) \
> > + : : : "memory"); \
> > +} while (0)
> > +
> > +#include <asm-generic/rwonce.h>
> > +
> > +#endif /* __ASM_RWONCE_H */
>
> IIUC the idea here is to have an alternative __WRITE_ONCE that replaces the
> asm-generic one.
>
> Honestly, this asm alternative here seems too much information, and too
> cryptic. I mean, yeah in the patch it all makes sense, but I imagine myself
> in the future looking at all this and trying to understand what is going
> on.
>
> Wouldn't it look better to have something like:
>
> #####
>
> /* Some explanation like the one on Kconfig */
>
> #define write_once_flush() \
> do { \
> asm volatile(ALTERNATIVE( \
> __nops(1), \
> "fence w, o\n\t", \
> THEAD_VENDOR_ID, \
> ERRATA_THEAD_WRITE_ONCE, \
> CONFIG_ERRATA_THEAD_WRITE_ONCE) \
> : : : "memory"); \
> } while(0)
>
>
> #define __WRITE_ONCE(x, val) \
> do { \
> *(volatile typeof(x) *)&(x) = (val); \
> write_once_flush(); \
> } while(0)
>
> #####
>
>
> This way I could quickly see there is a flush after the writting of
> WRITE_ONCE(), and this flush is the above "complicated" asm.
>
> What do you think?
Okay, good point, and I would take it.
>
> > diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
> > index cb89af3f0704..73078cfe4029 100644
> > --- a/arch/riscv/include/asm/vendorid_list.h
> > +++ b/arch/riscv/include/asm/vendorid_list.h
> > @@ -8,4 +8,18 @@
> > #define SIFIVE_VENDOR_ID 0x489
> > #define THEAD_VENDOR_ID 0x5b7
> >
> > +#ifdef CONFIG_ERRATA_SIFIVE
> > +#define ERRATA_SIFIVE_CIP_453 0
> > +#define ERRATA_SIFIVE_CIP_1200 1
> > +#define ERRATA_SIFIVE_NUMBER 2
> > +#endif
> > +
> > +#ifdef CONFIG_ERRATA_THEAD
> > +#define ERRATA_THEAD_PBMT 0
> > +#define ERRATA_THEAD_CMO 1
> > +#define ERRATA_THEAD_PMU 2
> > +#define ERRATA_THEAD_WRITE_ONCE 3
> > +#define ERRATA_THEAD_NUMBER 4
> > +#endif
> > +
> > #endif
> > diff --git a/include/asm-generic/rwonce.h b/include/asm-generic/rwonce.h
> > index 8d0a6280e982..fb07fe8c6e45 100644
> > --- a/include/asm-generic/rwonce.h
> > +++ b/include/asm-generic/rwonce.h
> > @@ -50,10 +50,12 @@
> > __READ_ONCE(x); \
> > })
> >
> > +#ifndef __WRITE_ONCE
> > #define __WRITE_ONCE(x, val) \
> > do { \
> > *(volatile typeof(x) *)&(x) = (val); \
> > } while (0)
> > +#endif
> >
> > #define WRITE_ONCE(x, val) \
> > do { \
> > --
> > 2.36.1
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 09/17] riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
@ 2023-09-17 15:15 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-17 15:15 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Thu, Sep 14, 2023 at 4:32 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:29:03AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The early version of T-Head C9xx cores has a store merge buffer
> > delay problem. The store merge buffer could improve the store queue
> > performance by merging multi-store requests, but when there are not
> > continued store requests, the prior single store request would be
> > waiting in the store queue for a long time. That would cause
> > significant problems for communication between multi-cores. This
> > problem was found on sg2042 & th1520 platforms with the qspinlock
> > lock torture test.
> >
> > So appending a fence w.o could immediately flush the store merge
> > buffer and let other cores see the write result.
> >
> > This will apply the WRITE_ONCE errata to handle the non-standard
> > behavior via appending a fence w.o instruction for WRITE_ONCE().
> >
> > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > Signed-off-by: Guo Ren <guoren@kernel.org>
> > ---
> > arch/riscv/Kconfig.errata | 19 +++++++++++++++++++
> > arch/riscv/errata/thead/errata.c | 20 ++++++++++++++++++++
> > arch/riscv/include/asm/errata_list.h | 13 -------------
> > arch/riscv/include/asm/rwonce.h | 24 ++++++++++++++++++++++++
> > arch/riscv/include/asm/vendorid_list.h | 14 ++++++++++++++
> > include/asm-generic/rwonce.h | 2 ++
> > 6 files changed, 79 insertions(+), 13 deletions(-)
> > create mode 100644 arch/riscv/include/asm/rwonce.h
> >
> > diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata
> > index 1aa85a427ff3..c919cc3f1a3a 100644
> > --- a/arch/riscv/Kconfig.errata
> > +++ b/arch/riscv/Kconfig.errata
> > @@ -77,4 +77,23 @@ config ERRATA_THEAD_PMU
> >
> > If you don't know what to do here, say "Y".
> >
> > +config ERRATA_THEAD_WRITE_ONCE
> > + bool "Apply T-Head WRITE_ONCE errata"
> > + depends on ERRATA_THEAD
> > + default y
> > + help
> > + The early version of T-Head C9xx cores has a store merge buffer
> > + delay problem. The store merge buffer could improve the store queue
> > + performance by merging multi-store requests, but when there are no
> > + continued store requests, the prior single store request would be
> > + waiting in the store queue for a long time. That would cause
> > + significant problems for communication between multi-cores. Appending
> > + a fence w.o could immediately flush the store merge buffer and let
> > + other cores see the write result.
> > +
> > + This will apply the WRITE_ONCE errata to handle the non-standard
> > + behavior via appending a fence w.o instruction for WRITE_ONCE().
> > +
> > + If you don't know what to do here, say "Y".
> > +
> > endmenu # "CPU errata selection"
> > diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
> > index be84b14f0118..751eb5a7f614 100644
> > --- a/arch/riscv/errata/thead/errata.c
> > +++ b/arch/riscv/errata/thead/errata.c
> > @@ -69,6 +69,23 @@ static bool errata_probe_pmu(unsigned int stage,
> > return true;
> > }
> >
> > +static bool errata_probe_write_once(unsigned int stage,
> > + unsigned long arch_id, unsigned long impid)
> > +{
> > + if (!IS_ENABLED(CONFIG_ERRATA_THEAD_WRITE_ONCE))
> > + return false;
> > +
> > + /* target-c9xx cores report arch_id and impid as 0 */
> > + if (arch_id != 0 || impid != 0)
> > + return false;
> > +
> > + if (stage == RISCV_ALTERNATIVES_BOOT ||
> > + stage == RISCV_ALTERNATIVES_MODULE)
> > + return true;
> > +
> > + return false;
> > +}
> > +
> > static u32 thead_errata_probe(unsigned int stage,
> > unsigned long archid, unsigned long impid)
> > {
> > @@ -83,6 +100,9 @@ static u32 thead_errata_probe(unsigned int stage,
> > if (errata_probe_pmu(stage, archid, impid))
> > cpu_req_errata |= BIT(ERRATA_THEAD_PMU);
> >
> > + if (errata_probe_write_once(stage, archid, impid))
> > + cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
> > +
> > return cpu_req_errata;
> > }
> >
> > diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
> > index 712cab7adffe..fbb2b8d39321 100644
> > --- a/arch/riscv/include/asm/errata_list.h
> > +++ b/arch/riscv/include/asm/errata_list.h
> > @@ -11,19 +11,6 @@
> > #include <asm/hwcap.h>
> > #include <asm/vendorid_list.h>
> >
> > -#ifdef CONFIG_ERRATA_SIFIVE
> > -#define ERRATA_SIFIVE_CIP_453 0
> > -#define ERRATA_SIFIVE_CIP_1200 1
> > -#define ERRATA_SIFIVE_NUMBER 2
> > -#endif
> > -
> > -#ifdef CONFIG_ERRATA_THEAD
> > -#define ERRATA_THEAD_PBMT 0
> > -#define ERRATA_THEAD_CMO 1
> > -#define ERRATA_THEAD_PMU 2
> > -#define ERRATA_THEAD_NUMBER 3
> > -#endif
> > -
>
> Here I understand you are moving stuff from errata_list.h to
> vendorid_list.h. Wouldn't it be better to do this on a separated patch
> before this one?
Okay.
>
> I understand this is used here, but it looks like it's unrelated.
>
> > #ifdef __ASSEMBLY__
> >
> > #define ALT_INSN_FAULT(x) \
> > diff --git a/arch/riscv/include/asm/rwonce.h b/arch/riscv/include/asm/rwonce.h
> > new file mode 100644
> > index 000000000000..be0b8864969d
> > --- /dev/null
> > +++ b/arch/riscv/include/asm/rwonce.h
> > @@ -0,0 +1,24 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +
> > +#ifndef __ASM_RWONCE_H
> > +#define __ASM_RWONCE_H
> > +
> > +#include <linux/compiler_types.h>
> > +#include <asm/alternative-macros.h>
> > +#include <asm/vendorid_list.h>
> > +
> > +#define __WRITE_ONCE(x, val) \
> > +do { \
> > + *(volatile typeof(x) *)&(x) = (val); \
> > + asm volatile(ALTERNATIVE( \
> > + __nops(1), \
> > + "fence w, o\n\t", \
> > + THEAD_VENDOR_ID, \
> > + ERRATA_THEAD_WRITE_ONCE, \
> > + CONFIG_ERRATA_THEAD_WRITE_ONCE) \
> > + : : : "memory"); \
> > +} while (0)
> > +
> > +#include <asm-generic/rwonce.h>
> > +
> > +#endif /* __ASM_RWONCE_H */
>
> IIUC the idea here is to have an alternative __WRITE_ONCE that replaces the
> asm-generic one.
>
> Honestly, this asm alternative here seems too much information, and too
> cryptic. I mean, yeah in the patch it all makes sense, but I imagine myself
> in the future looking at all this and trying to understand what is going
> on.
>
> Wouldn't it look better to have something like:
>
> #####
>
> /* Some explanation like the one on Kconfig */
>
> #define write_once_flush() \
> do { \
> asm volatile(ALTERNATIVE( \
> __nops(1), \
> "fence w, o\n\t", \
> THEAD_VENDOR_ID, \
> ERRATA_THEAD_WRITE_ONCE, \
> CONFIG_ERRATA_THEAD_WRITE_ONCE) \
> : : : "memory"); \
> } while(0)
>
>
> #define __WRITE_ONCE(x, val) \
> do { \
> *(volatile typeof(x) *)&(x) = (val); \
> write_once_flush(); \
> } while(0)
>
> #####
>
>
> This way I could quickly see there is a flush after the writting of
> WRITE_ONCE(), and this flush is the above "complicated" asm.
>
> What do you think?
Okay, good point, and I would take it.
>
> > diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
> > index cb89af3f0704..73078cfe4029 100644
> > --- a/arch/riscv/include/asm/vendorid_list.h
> > +++ b/arch/riscv/include/asm/vendorid_list.h
> > @@ -8,4 +8,18 @@
> > #define SIFIVE_VENDOR_ID 0x489
> > #define THEAD_VENDOR_ID 0x5b7
> >
> > +#ifdef CONFIG_ERRATA_SIFIVE
> > +#define ERRATA_SIFIVE_CIP_453 0
> > +#define ERRATA_SIFIVE_CIP_1200 1
> > +#define ERRATA_SIFIVE_NUMBER 2
> > +#endif
> > +
> > +#ifdef CONFIG_ERRATA_THEAD
> > +#define ERRATA_THEAD_PBMT 0
> > +#define ERRATA_THEAD_CMO 1
> > +#define ERRATA_THEAD_PMU 2
> > +#define ERRATA_THEAD_WRITE_ONCE 3
> > +#define ERRATA_THEAD_NUMBER 4
> > +#endif
> > +
> > #endif
> > diff --git a/include/asm-generic/rwonce.h b/include/asm-generic/rwonce.h
> > index 8d0a6280e982..fb07fe8c6e45 100644
> > --- a/include/asm-generic/rwonce.h
> > +++ b/include/asm-generic/rwonce.h
> > @@ -50,10 +50,12 @@
> > __READ_ONCE(x); \
> > })
> >
> > +#ifndef __WRITE_ONCE
> > #define __WRITE_ONCE(x, val) \
> > do { \
> > *(volatile typeof(x) *)&(x) = (val); \
> > } while (0)
> > +#endif
> >
> > #define WRITE_ONCE(x, val) \
> > do { \
> > --
> > 2.36.1
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-17 14:34 ` Guo Ren
@ 2023-09-19 5:13 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:13 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 10:34:36PM +0800, Guo Ren wrote:
> On Sat, Sep 16, 2023 at 9:25 AM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Fri, Sep 15, 2023 at 08:36:31PM +0800, Guo Ren wrote:
> > > On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > >
> > > > > Cache-block prefetch instructions are HINTs to the hardware to
> > > > > indicate that software intends to perform a particular type of
> > > > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > > > improve the arch_xchg for qspinlock xchg_tail.
> > > > >
> > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > ---
> > > > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > > > arch/riscv/include/asm/hwcap.h | 1 +
> > > > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > > > arch/riscv/kernel/cpufeature.c | 1 +
> > > > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > index e9ae6fa232c3..2c346fe169c1 100644
> > > > > --- a/arch/riscv/Kconfig
> > > > > +++ b/arch/riscv/Kconfig
> > > > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > > > >
> > > > > If you don't know what to do here, say Y.
> > > > >
> > > > > +config RISCV_ISA_ZICBOP
> > > > > + bool "Zicbop extension support for cache block prefetch"
> > > > > + depends on MMU
> > > > > + depends on RISCV_ALTERNATIVE
> > > > > + default y
> > > > > + help
> > > > > + Adds support to dynamically detect the presence of the ZICBOP
> > > > > + extension (Cache Block Prefetch Operations) and enable its
> > > > > + usage.
> > > > > +
> > > > > + The Zicbop extension can be used to prefetch cache block for
> > > > > + read/write/instruction fetch.
> > > > > +
> > > > > + If you don't know what to do here, say Y.
> > > > > +
> > > > > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > > > > bool
> > > > > default y
> > > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > > index 702725727671..56eff7a9d2d2 100644
> > > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > > @@ -11,6 +11,7 @@
> > > > >
> > > > > #include <asm/barrier.h>
> > > > > #include <asm/fence.h>
> > > > > +#include <asm/processor.h>
> > > > >
> > > > > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > > > > ({ \
> > > > > @@ -25,6 +26,7 @@
> > > > > \
> > > > > __asm__ __volatile__ ( \
> > > > > prepend \
> > > > > + PREFETCHW_ASM(%5) \
> > > > > "0: lr.w %0, %2\n" \
> > > > > " and %1, %0, %z4\n" \
> > > > > " or %1, %1, %z3\n" \
> > > > > @@ -32,7 +34,7 @@
> > > > > " bnez %1, 0b\n" \
> > > > > append \
> > > > > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > > > > - : "rJ" (__newx), "rJ" (~__mask) \
> > > > > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > > > > : "memory"); \
> > > > > \
> > > > > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > > > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > > > > index b7b58258f6c7..78b7b8b53778 100644
> > > > > --- a/arch/riscv/include/asm/hwcap.h
> > > > > +++ b/arch/riscv/include/asm/hwcap.h
> > > > > @@ -58,6 +58,7 @@
> > > > > #define RISCV_ISA_EXT_ZICSR 40
> > > > > #define RISCV_ISA_EXT_ZIFENCEI 41
> > > > > #define RISCV_ISA_EXT_ZIHPM 42
> > > > > +#define RISCV_ISA_EXT_ZICBOP 43
> > > > >
> > > > > #define RISCV_ISA_EXT_MAX 64
> > > > >
> > > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > > index 6960beb75f32..dc590d331894 100644
> > > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > > @@ -134,6 +134,7 @@
> > > > >
> > > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > > > >
> > > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > > @@ -196,4 +197,8 @@
> > > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > > RS1(base), SIMM12(4))
> > > > >
> > > > > +#define CBO_prefetchw(base) \
> > > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > > + RD(x0), RS1(base), RS2(x0))
> > > > > +
> > > >
> > > > I understand that here you create the instruction via bitfield, following
> > > > the ISA, and this enables using instructions not available on the
> > > > toolchain.
> > > >
> > > > It took me some time to find the document with this instruction, so please
> > > > add this to the commit msg:
> > > >
> > > > https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> > > > Page 23.
> > > >
> > > > IIUC, the instruction is "prefetch.w".
> > > >
> > > > Maybe I am missing something, but in the document the rs2 field
> > > > (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
> > > >
> > > > rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
> > > >
> > > > Is the above correct, or am I missing something?
> > > Oh, you are right. My fault, thx for pointing out. It should be:
> > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > + RD(x0), RS1(base), RS2(x3))
> >
> > Now I am curious to check if / how will this impact performance. :)
> > (Please let me know)
> Ref:
> commit 0ea366f5e1b6 ("arm64: atomics: prefetch the destination word
> for write prior to stxr")
> commit 86d231459d6d ("bpf: cpumap memory prefetchw optimizations for
> struct page")
Oh, I understand that prefetch.w is very useful for performance :)
What I meant is that previously this patch was issuing a prefetch.i,
and now it's issuing a prefetch.w (as intended).
What got me curious is how much would it impact the performance to change
the prefetch.i to prefetch.w. :)
Thanks!
Leo
>
> >
> >
> > >
> > > >
> > > >
> > > > Thanks!
> > > > Leo
> > > >
> > > > > #endif /* __ASM_INSN_DEF_H */
> > > > > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > > > > index de9da852f78d..7ad3a24212e8 100644
> > > > > --- a/arch/riscv/include/asm/processor.h
> > > > > +++ b/arch/riscv/include/asm/processor.h
> > > > > @@ -12,6 +12,8 @@
> > > > > #include <vdso/processor.h>
> > > > >
> > > > > #include <asm/ptrace.h>
> > > > > +#include <asm/insn-def.h>
> > > > > +#include <asm/hwcap.h>
> > > > >
> > > > > #ifdef CONFIG_64BIT
> > > > > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > > > > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > > > > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > > > > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> > > > >
> > > > > +#define ARCH_HAS_PREFETCHW
> > > > > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > > > > + CBO_prefetchw(base), \
> > > > > + 0, \
> > > > > + RISCV_ISA_EXT_ZICBOP, \
> > > > > + CONFIG_RISCV_ISA_ZICBOP)
> > > > > +static inline void prefetchw(const void *ptr)
> > > > > +{
> > > > > + asm volatile(PREFETCHW_ASM(%0)
> > > > > + : : "r" (ptr) : "memory");
> > > > > +}
> > > > >
> > > > > /* Do necessary setup to start up a newly executed thread. */
> > > > > extern void start_thread(struct pt_regs *regs,
> > > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > > > index ef7b4fd9e876..e0b897db0b97 100644
> > > > > --- a/arch/riscv/kernel/cpufeature.c
> > > > > +++ b/arch/riscv/kernel/cpufeature.c
> > > > > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > > > > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > > > > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > > > > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > > > > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > > > > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > > > > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > > > > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > > > > --
> > > > > 2.36.1
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-19 5:13 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:13 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 10:34:36PM +0800, Guo Ren wrote:
> On Sat, Sep 16, 2023 at 9:25 AM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Fri, Sep 15, 2023 at 08:36:31PM +0800, Guo Ren wrote:
> > > On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > >
> > > > > Cache-block prefetch instructions are HINTs to the hardware to
> > > > > indicate that software intends to perform a particular type of
> > > > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > > > improve the arch_xchg for qspinlock xchg_tail.
> > > > >
> > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > ---
> > > > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > > > arch/riscv/include/asm/hwcap.h | 1 +
> > > > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > > > arch/riscv/kernel/cpufeature.c | 1 +
> > > > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > index e9ae6fa232c3..2c346fe169c1 100644
> > > > > --- a/arch/riscv/Kconfig
> > > > > +++ b/arch/riscv/Kconfig
> > > > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > > > >
> > > > > If you don't know what to do here, say Y.
> > > > >
> > > > > +config RISCV_ISA_ZICBOP
> > > > > + bool "Zicbop extension support for cache block prefetch"
> > > > > + depends on MMU
> > > > > + depends on RISCV_ALTERNATIVE
> > > > > + default y
> > > > > + help
> > > > > + Adds support to dynamically detect the presence of the ZICBOP
> > > > > + extension (Cache Block Prefetch Operations) and enable its
> > > > > + usage.
> > > > > +
> > > > > + The Zicbop extension can be used to prefetch cache block for
> > > > > + read/write/instruction fetch.
> > > > > +
> > > > > + If you don't know what to do here, say Y.
> > > > > +
> > > > > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > > > > bool
> > > > > default y
> > > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > > index 702725727671..56eff7a9d2d2 100644
> > > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > > @@ -11,6 +11,7 @@
> > > > >
> > > > > #include <asm/barrier.h>
> > > > > #include <asm/fence.h>
> > > > > +#include <asm/processor.h>
> > > > >
> > > > > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > > > > ({ \
> > > > > @@ -25,6 +26,7 @@
> > > > > \
> > > > > __asm__ __volatile__ ( \
> > > > > prepend \
> > > > > + PREFETCHW_ASM(%5) \
> > > > > "0: lr.w %0, %2\n" \
> > > > > " and %1, %0, %z4\n" \
> > > > > " or %1, %1, %z3\n" \
> > > > > @@ -32,7 +34,7 @@
> > > > > " bnez %1, 0b\n" \
> > > > > append \
> > > > > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > > > > - : "rJ" (__newx), "rJ" (~__mask) \
> > > > > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > > > > : "memory"); \
> > > > > \
> > > > > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > > > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > > > > index b7b58258f6c7..78b7b8b53778 100644
> > > > > --- a/arch/riscv/include/asm/hwcap.h
> > > > > +++ b/arch/riscv/include/asm/hwcap.h
> > > > > @@ -58,6 +58,7 @@
> > > > > #define RISCV_ISA_EXT_ZICSR 40
> > > > > #define RISCV_ISA_EXT_ZIFENCEI 41
> > > > > #define RISCV_ISA_EXT_ZIHPM 42
> > > > > +#define RISCV_ISA_EXT_ZICBOP 43
> > > > >
> > > > > #define RISCV_ISA_EXT_MAX 64
> > > > >
> > > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > > index 6960beb75f32..dc590d331894 100644
> > > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > > @@ -134,6 +134,7 @@
> > > > >
> > > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > > > >
> > > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > > @@ -196,4 +197,8 @@
> > > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > > RS1(base), SIMM12(4))
> > > > >
> > > > > +#define CBO_prefetchw(base) \
> > > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > > + RD(x0), RS1(base), RS2(x0))
> > > > > +
> > > >
> > > > I understand that here you create the instruction via bitfield, following
> > > > the ISA, and this enables using instructions not available on the
> > > > toolchain.
> > > >
> > > > It took me some time to find the document with this instruction, so please
> > > > add this to the commit msg:
> > > >
> > > > https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> > > > Page 23.
> > > >
> > > > IIUC, the instruction is "prefetch.w".
> > > >
> > > > Maybe I am missing something, but in the document the rs2 field
> > > > (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
> > > >
> > > > rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
> > > >
> > > > Is the above correct, or am I missing something?
> > > Oh, you are right. My fault, thx for pointing out. It should be:
> > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > + RD(x0), RS1(base), RS2(x3))
> >
> > Now I am curious to check if / how will this impact performance. :)
> > (Please let me know)
> Ref:
> commit 0ea366f5e1b6 ("arm64: atomics: prefetch the destination word
> for write prior to stxr")
> commit 86d231459d6d ("bpf: cpumap memory prefetchw optimizations for
> struct page")
Oh, I understand that prefetch.w is very useful for performance :)
What I meant is that previously this patch was issuing a prefetch.i,
and now it's issuing a prefetch.w (as intended).
What got me curious is how much would it impact the performance to change
the prefetch.i to prefetch.w. :)
Thanks!
Leo
>
> >
> >
> > >
> > > >
> > > >
> > > > Thanks!
> > > > Leo
> > > >
> > > > > #endif /* __ASM_INSN_DEF_H */
> > > > > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > > > > index de9da852f78d..7ad3a24212e8 100644
> > > > > --- a/arch/riscv/include/asm/processor.h
> > > > > +++ b/arch/riscv/include/asm/processor.h
> > > > > @@ -12,6 +12,8 @@
> > > > > #include <vdso/processor.h>
> > > > >
> > > > > #include <asm/ptrace.h>
> > > > > +#include <asm/insn-def.h>
> > > > > +#include <asm/hwcap.h>
> > > > >
> > > > > #ifdef CONFIG_64BIT
> > > > > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > > > > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > > > > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > > > > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> > > > >
> > > > > +#define ARCH_HAS_PREFETCHW
> > > > > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > > > > + CBO_prefetchw(base), \
> > > > > + 0, \
> > > > > + RISCV_ISA_EXT_ZICBOP, \
> > > > > + CONFIG_RISCV_ISA_ZICBOP)
> > > > > +static inline void prefetchw(const void *ptr)
> > > > > +{
> > > > > + asm volatile(PREFETCHW_ASM(%0)
> > > > > + : : "r" (ptr) : "memory");
> > > > > +}
> > > > >
> > > > > /* Do necessary setup to start up a newly executed thread. */
> > > > > extern void start_thread(struct pt_regs *regs,
> > > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > > > index ef7b4fd9e876..e0b897db0b97 100644
> > > > > --- a/arch/riscv/kernel/cpufeature.c
> > > > > +++ b/arch/riscv/kernel/cpufeature.c
> > > > > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > > > > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > > > > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > > > > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > > > > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > > > > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > > > > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > > > > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > > > > --
> > > > > 2.36.1
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
2023-09-17 15:02 ` Guo Ren
@ 2023-09-19 5:20 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:20 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 11:02:47PM +0800, Guo Ren wrote:
> On Fri, Sep 15, 2023 at 5:08 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Fri, Sep 15, 2023 at 10:10:25AM +0800, Guo Ren wrote:
> > > On Thu, Sep 14, 2023 at 5:43 PM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote:
> > > > > On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > > > >
> > > > > > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > > > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > > > >
> > > > > > > The requirements of qspinlock have been documented by commit:
> > > > > > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > > > > > > atomics").
> > > > > > >
> > > > > > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > > > > > > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > > > > > > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > > > > > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > > > > > > is the one.
> > > > > > >
> > > > > > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > > > > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > > > > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > > > > > >
> > > > > > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > > > > > queued_spinlock 0.5109/0.00
> > > > > > > ticket_spinlock 0.5814/0.00
> > > > > > >
> > > > > > > perf futex/hash (+6.7%):
> > > > > > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > > > > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > > > > > >
> > > > > > > perf futex/wake-parallel (+8.6%):
> > > > > > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > > > > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > > > > > >
> > > > > > > perf futex/requeue (+4.2%):
> > > > > > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > > > > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > > > > > >
> > > > > > > System Benchmarks (+6.4%)
> > > > > > > queued_spinlock:
> > > > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > > > > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > > > > > Execl Throughput 43.0 13116.6 3050.4
> > > > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > > > > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > > > > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > > > > > Process Creation 126.0 12537.3 995.0
> > > > > > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > > > > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > > > > > System Call Overhead 15000.0 33308301.3 22205.5
> > > > > > > ========
> > > > > > > System Benchmarks Index Score 12426.1
> > > > > > >
> > > > > > > ticket_spinlock:
> > > > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > > > > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > > > > > Execl Throughput 43.0 12625.1 2936.1
> > > > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > > > > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > > > > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > > > > > Process Creation 126.0 10798.4 857.0
> > > > > > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > > > > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > > > > > System Call Overhead 15000.0 30766778.4 20511.2
> > > > > > > ========
> > > > > > > System Benchmarks Index Score 11670.7
> > > > > > >
> > > > > > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > > > > > cores platform than the ticket_lock.
> > > > > > >
> > > > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > > > ---
> > > > > > > arch/riscv/Kconfig | 16 ++++++++++++++++
> > > > > > > arch/riscv/include/asm/Kbuild | 3 ++-
> > > > > > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > > > > > > 3 files changed, 35 insertions(+), 1 deletion(-)
> > > > > > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > > > > > >
> > > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > > > index 2c346fe169c1..7f39bfc75744 100644
> > > > > > > --- a/arch/riscv/Kconfig
> > > > > > > +++ b/arch/riscv/Kconfig
> > > > > > > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > > > > > > Specify the maximum number of NUMA Nodes available on the target
> > > > > > > system. Increases memory reserved to accommodate various tables.
> > > > > > >
> > > > > > > +choice
> > > > > > > + prompt "RISC-V spinlock type"
> > > > > > > + default RISCV_TICKET_SPINLOCKS
> > > > > > > +
> > > > > > > +config RISCV_TICKET_SPINLOCKS
> > > > > > > + bool "Using ticket spinlock"
> > > > > > > +
> > > > > > > +config RISCV_QUEUED_SPINLOCKS
> > > > > > > + bool "Using queued spinlock"
> > > > > > > + depends on SMP && MMU
> > > > > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > > > > + help
> > > > > > > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > > > > + Otherwise, stay at ticket-lock.
> > > > > > > +endchoice
> > > > > > > +
> > > > > > > config RISCV_ALTERNATIVE
> > > > > > > bool
> > > > > > > depends on !XIP_KERNEL
> > > > > > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > > > > > index 504f8b7e72d4..a0dc85e4a754 100644
> > > > > > > --- a/arch/riscv/include/asm/Kbuild
> > > > > > > +++ b/arch/riscv/include/asm/Kbuild
> > > > > > > @@ -2,10 +2,11 @@
> > > > > > > generic-y += early_ioremap.h
> > > > > > > generic-y += flat.h
> > > > > > > generic-y += kvm_para.h
> > > > > > > +generic-y += mcs_spinlock.h
> > > > > > > generic-y += parport.h
> > > > > > > -generic-y += spinlock.h
> > > > > >
> > > > > > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> > > > > > and include the asm-generic headers of mcs_spinlock and qspinlock.
> > > > > >
> > > > > > In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> > > > > > which seems the oposite of the above description (ticket spinlocks being
> > > > > > the standard).
> > > > > >
> > > > > > Shouldn't ticket-spinlock.h also get included here?
> > > > > > (Also, I am probably missing something, as I dont' see the use of
> > > > > > mcs_spinlock here.)
> > > > > No, because asm-generic/spinlock.h:
> > > > > ...
> > > > > #include <asm-generic/ticket_spinlock.h>
> > > > > ...
> > > > >
> > > >
> > > > But aren't you removing asm-generic/spinlock.h below ?
> > > > -generic-y += spinlock.h
> > > Yes, current is:
> > >
> > > arch/riscv/include/asm/spinlock.h -> include/asm-generic/spinlock.h ->
> > > include/asm-generic/ticket_spinlock.h
> >
> > I did a little reading on how generic-y works (which I was unaware):
> >
> > "If an architecture uses a verbatim copy of a header from
> > include/asm-generic then this is listed in the file
> > arch/$(SRCARCH)/include/asm/Kbuild [...] During the prepare phase of the
> > build a wrapper include file is generated in the directory [...]"
> >
> > Oh, so you are removing the asm-generic/spinlock.h because it's link
> > was replaced by a new asm/spinlock.h.
> >
> > You add qspinlock.h to generic-y because it's new in riscv, and add
> > mcs_spinlock.h because it's needed by qspinlock.h.
> >
> > Ok, it makes sense now.
> >
> > Sorry about this noise.
> > I was unaware of how generic-y worked, and (wrongly)
> > assumed it was about including headers automatically in the build.
> >
> >
> > >
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > +#include <asm/qspinlock.h>
> > > +#include <asm/qrwlock.h>
> > > +#else
> > > +#include <asm-generic/spinlock.h>
> > > +#endif
> > >
> > > So, you want me:
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > +#include <asm/qspinlock.h>
> > > +#else
> > > +#include <asm-generic/ticket_spinlock.h>
> > > +#endif
> > >
> > > +#include <asm/qrwlock.h>
> > >
> > > Right?
> >
> > No, I didn't mean that.
> > I was just worried about the arch_spin_*() interfaces, but they should be
> > fine.
> >
> > BTW, according to kernel doc on generic-y, shouldn't be a better idea to
> > add 'ticket_spinlock.h' to generic-y, and include above as
> > asm/ticket_spinlock.h?
> >
> > Or is generic-y reserved only for stuff which is indirectly included by
> > other headers?
> It's okay to add generic-y for ticket_spinlock.h, and I'm okay with
> the following:
>
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#include <asm/qspinlock.h>
> +#else
> +#include <asm/ticket_spinlock.h>
> +#endif
>
> +#include <asm/qrwlock.h>
It does look more intuitive, so I am glad it works for you :)
FWIW:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
>
> >
> > Thanks!
> > Leo
> >
> > >
> > > >
> > > > > >
> > > > > > > generic-y += spinlock_types.h
> > > > > > > generic-y += qrwlock.h
> > > > > > > generic-y += qrwlock_types.h
> > > > > > > +generic-y += qspinlock.h
> > > > > > > generic-y += user.h
> > > > > > > generic-y += vmlinux.lds.h
> > > > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > > > new file mode 100644
> > > > > > > index 000000000000..c644a92d4548
> > > > > > > --- /dev/null
> > > > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > > > @@ -0,0 +1,17 @@
> > > > > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > > > > +
> > > > > > > +#ifndef __ASM_RISCV_SPINLOCK_H
> > > > > > > +#define __ASM_RISCV_SPINLOCK_H
> > > > > > > +
> > > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > > > +#define _Q_PENDING_LOOPS (1 << 9)
> > > > > > > +#endif
> > > > > >
> > > > > > Any reason the above define couldn't be merged on the ifdef below?
> > > > > Easy for the next patch to modify. See Waiman's comment:
> > > > >
> > > > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> > > > >
> > > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > > index c644a92d4548..9eb3ad31e564 100644
> > > > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > > @@ -7,11 +7,94 @@
> > > > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > > > #endif
> > > > > >
> > > > >
> > > > > I see why you separated the _Q_PENDING_LOOPS out.
> > > > >
> > > >
> > > > I see, should be fine then.
> > > >
> > > > Thanks!
> > > > Leo
> > > >
> > > > >
> > > > > >
> > > > > > > +
> > > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > > > +#include <asm/qspinlock.h>
> > > > > > > +#include <asm/qrwlock.h>
> > > > > > > +#else
> > > > > > > +#include <asm-generic/spinlock.h>
> > > > > > > +#endif
> > > > > > > +
> > > > > > > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > > > > > > --
> > > > > > > 2.36.1
> > > > > > >
> > > > > >
> > > > > > Thanks!
> > > > > > Leo
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best Regards
> > > > > Guo Ren
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support
@ 2023-09-19 5:20 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:20 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 11:02:47PM +0800, Guo Ren wrote:
> On Fri, Sep 15, 2023 at 5:08 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Fri, Sep 15, 2023 at 10:10:25AM +0800, Guo Ren wrote:
> > > On Thu, Sep 14, 2023 at 5:43 PM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > On Thu, Sep 14, 2023 at 12:46:56PM +0800, Guo Ren wrote:
> > > > > On Thu, Sep 14, 2023 at 4:29 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > > > >
> > > > > > On Sun, Sep 10, 2023 at 04:28:59AM -0400, guoren@kernel.org wrote:
> > > > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > > > >
> > > > > > > The requirements of qspinlock have been documented by commit:
> > > > > > > a8ad07e5240c ("asm-generic: qspinlock: Indicate the use of mixed-size
> > > > > > > atomics").
> > > > > > >
> > > > > > > Although RISC-V ISA gives out a weaker forward guarantee LR/SC, which
> > > > > > > doesn't satisfy the requirements of qspinlock above, it won't prevent
> > > > > > > some riscv vendors from implementing a strong fwd guarantee LR/SC in
> > > > > > > microarchitecture to match xchg_tail requirement. T-HEAD C9xx processor
> > > > > > > is the one.
> > > > > > >
> > > > > > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > > > > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > > > > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > > > > > >
> > > > > > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > > > > > queued_spinlock 0.5109/0.00
> > > > > > > ticket_spinlock 0.5814/0.00
> > > > > > >
> > > > > > > perf futex/hash (+6.7%):
> > > > > > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > > > > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > > > > > >
> > > > > > > perf futex/wake-parallel (+8.6%):
> > > > > > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > > > > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > > > > > >
> > > > > > > perf futex/requeue (+4.2%):
> > > > > > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > > > > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > > > > > >
> > > > > > > System Benchmarks (+6.4%)
> > > > > > > queued_spinlock:
> > > > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > > > > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > > > > > Execl Throughput 43.0 13116.6 3050.4
> > > > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > > > > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > > > > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > > > > > Process Creation 126.0 12537.3 995.0
> > > > > > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > > > > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > > > > > System Call Overhead 15000.0 33308301.3 22205.5
> > > > > > > ========
> > > > > > > System Benchmarks Index Score 12426.1
> > > > > > >
> > > > > > > ticket_spinlock:
> > > > > > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > > > > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > > > > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > > > > > Execl Throughput 43.0 12625.1 2936.1
> > > > > > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > > > > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > > > > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > > > > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > > > > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > > > > > Process Creation 126.0 10798.4 857.0
> > > > > > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > > > > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > > > > > System Call Overhead 15000.0 30766778.4 20511.2
> > > > > > > ========
> > > > > > > System Benchmarks Index Score 11670.7
> > > > > > >
> > > > > > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > > > > > cores platform than the ticket_lock.
> > > > > > >
> > > > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > > > ---
> > > > > > > arch/riscv/Kconfig | 16 ++++++++++++++++
> > > > > > > arch/riscv/include/asm/Kbuild | 3 ++-
> > > > > > > arch/riscv/include/asm/spinlock.h | 17 +++++++++++++++++
> > > > > > > 3 files changed, 35 insertions(+), 1 deletion(-)
> > > > > > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > > > > > >
> > > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > > > index 2c346fe169c1..7f39bfc75744 100644
> > > > > > > --- a/arch/riscv/Kconfig
> > > > > > > +++ b/arch/riscv/Kconfig
> > > > > > > @@ -471,6 +471,22 @@ config NODES_SHIFT
> > > > > > > Specify the maximum number of NUMA Nodes available on the target
> > > > > > > system. Increases memory reserved to accommodate various tables.
> > > > > > >
> > > > > > > +choice
> > > > > > > + prompt "RISC-V spinlock type"
> > > > > > > + default RISCV_TICKET_SPINLOCKS
> > > > > > > +
> > > > > > > +config RISCV_TICKET_SPINLOCKS
> > > > > > > + bool "Using ticket spinlock"
> > > > > > > +
> > > > > > > +config RISCV_QUEUED_SPINLOCKS
> > > > > > > + bool "Using queued spinlock"
> > > > > > > + depends on SMP && MMU
> > > > > > > + select ARCH_USE_QUEUED_SPINLOCKS
> > > > > > > + help
> > > > > > > + Make sure your micro arch LL/SC has a strong forward progress guarantee.
> > > > > > > + Otherwise, stay at ticket-lock.
> > > > > > > +endchoice
> > > > > > > +
> > > > > > > config RISCV_ALTERNATIVE
> > > > > > > bool
> > > > > > > depends on !XIP_KERNEL
> > > > > > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > > > > > index 504f8b7e72d4..a0dc85e4a754 100644
> > > > > > > --- a/arch/riscv/include/asm/Kbuild
> > > > > > > +++ b/arch/riscv/include/asm/Kbuild
> > > > > > > @@ -2,10 +2,11 @@
> > > > > > > generic-y += early_ioremap.h
> > > > > > > generic-y += flat.h
> > > > > > > generic-y += kvm_para.h
> > > > > > > +generic-y += mcs_spinlock.h
> > > > > > > generic-y += parport.h
> > > > > > > -generic-y += spinlock.h
> > > > > >
> > > > > > IIUC here you take the asm-generic/spinlock.h (which defines arch_spin_*())
> > > > > > and include the asm-generic headers of mcs_spinlock and qspinlock.
> > > > > >
> > > > > > In this case, the qspinlock.h will provide the arch_spin_*() interfaces,
> > > > > > which seems the oposite of the above description (ticket spinlocks being
> > > > > > the standard).
> > > > > >
> > > > > > Shouldn't ticket-spinlock.h also get included here?
> > > > > > (Also, I am probably missing something, as I dont' see the use of
> > > > > > mcs_spinlock here.)
> > > > > No, because asm-generic/spinlock.h:
> > > > > ...
> > > > > #include <asm-generic/ticket_spinlock.h>
> > > > > ...
> > > > >
> > > >
> > > > But aren't you removing asm-generic/spinlock.h below ?
> > > > -generic-y += spinlock.h
> > > Yes, current is:
> > >
> > > arch/riscv/include/asm/spinlock.h -> include/asm-generic/spinlock.h ->
> > > include/asm-generic/ticket_spinlock.h
> >
> > I did a little reading on how generic-y works (which I was unaware):
> >
> > "If an architecture uses a verbatim copy of a header from
> > include/asm-generic then this is listed in the file
> > arch/$(SRCARCH)/include/asm/Kbuild [...] During the prepare phase of the
> > build a wrapper include file is generated in the directory [...]"
> >
> > Oh, so you are removing the asm-generic/spinlock.h because it's link
> > was replaced by a new asm/spinlock.h.
> >
> > You add qspinlock.h to generic-y because it's new in riscv, and add
> > mcs_spinlock.h because it's needed by qspinlock.h.
> >
> > Ok, it makes sense now.
> >
> > Sorry about this noise.
> > I was unaware of how generic-y worked, and (wrongly)
> > assumed it was about including headers automatically in the build.
> >
> >
> > >
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > +#include <asm/qspinlock.h>
> > > +#include <asm/qrwlock.h>
> > > +#else
> > > +#include <asm-generic/spinlock.h>
> > > +#endif
> > >
> > > So, you want me:
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > +#include <asm/qspinlock.h>
> > > +#else
> > > +#include <asm-generic/ticket_spinlock.h>
> > > +#endif
> > >
> > > +#include <asm/qrwlock.h>
> > >
> > > Right?
> >
> > No, I didn't mean that.
> > I was just worried about the arch_spin_*() interfaces, but they should be
> > fine.
> >
> > BTW, according to kernel doc on generic-y, shouldn't be a better idea to
> > add 'ticket_spinlock.h' to generic-y, and include above as
> > asm/ticket_spinlock.h?
> >
> > Or is generic-y reserved only for stuff which is indirectly included by
> > other headers?
> It's okay to add generic-y for ticket_spinlock.h, and I'm okay with
> the following:
>
> +#ifdef CONFIG_QUEUED_SPINLOCKS
> +#include <asm/qspinlock.h>
> +#else
> +#include <asm/ticket_spinlock.h>
> +#endif
>
> +#include <asm/qrwlock.h>
It does look more intuitive, so I am glad it works for you :)
FWIW:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thanks!
Leo
>
> >
> > Thanks!
> > Leo
> >
> > >
> > > >
> > > > > >
> > > > > > > generic-y += spinlock_types.h
> > > > > > > generic-y += qrwlock.h
> > > > > > > generic-y += qrwlock_types.h
> > > > > > > +generic-y += qspinlock.h
> > > > > > > generic-y += user.h
> > > > > > > generic-y += vmlinux.lds.h
> > > > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > > > new file mode 100644
> > > > > > > index 000000000000..c644a92d4548
> > > > > > > --- /dev/null
> > > > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > > > @@ -0,0 +1,17 @@
> > > > > > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > > > > > +
> > > > > > > +#ifndef __ASM_RISCV_SPINLOCK_H
> > > > > > > +#define __ASM_RISCV_SPINLOCK_H
> > > > > > > +
> > > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > > > +#define _Q_PENDING_LOOPS (1 << 9)
> > > > > > > +#endif
> > > > > >
> > > > > > Any reason the above define couldn't be merged on the ifdef below?
> > > > > Easy for the next patch to modify. See Waiman's comment:
> > > > >
> > > > > https://lore.kernel.org/linux-riscv/4cc7113a-0e4e-763a-cba2-7963bcd26c7a@redhat.com/
> > > > >
> > > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > > index c644a92d4548..9eb3ad31e564 100644
> > > > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > > @@ -7,11 +7,94 @@
> > > > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > > > #endif
> > > > > >
> > > > >
> > > > > I see why you separated the _Q_PENDING_LOOPS out.
> > > > >
> > > >
> > > > I see, should be fine then.
> > > >
> > > > Thanks!
> > > > Leo
> > > >
> > > > >
> > > > > >
> > > > > > > +
> > > > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > > > +#include <asm/qspinlock.h>
> > > > > > > +#include <asm/qrwlock.h>
> > > > > > > +#else
> > > > > > > +#include <asm-generic/spinlock.h>
> > > > > > > +#endif
> > > > > > > +
> > > > > > > +#endif /* __ASM_RISCV_SPINLOCK_H */
> > > > > > > --
> > > > > > > 2.36.1
> > > > > > >
> > > > > >
> > > > > > Thanks!
> > > > > > Leo
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best Regards
> > > > > Guo Ren
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
2023-09-17 15:12 ` Guo Ren
@ 2023-09-19 5:30 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:30 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 11:12:31PM +0800, Guo Ren wrote:
> On Thu, Sep 14, 2023 at 4:02 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:29:02AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Add a static key controlling whether virt_spin_lock() should be
> > > called or not. When running on bare metal set the new key to
> > > false.
> > >
> > > The KVM guests fall back to a Test-and-Set spinlock, because fair
> > > locks have horrible lock 'holder' preemption issues. The
> > > virt_spin_lock_key would shortcut for the
> > > queued_spin_lock_slowpath() function that allow virt_spin_lock to
> > > hijack it.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > .../admin-guide/kernel-parameters.txt | 4 +++
> > > arch/riscv/include/asm/sbi.h | 8 +++++
> > > arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
> > > arch/riscv/kernel/sbi.c | 2 +-
> > > arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
> > > 5 files changed, 64 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > index 61cacb8dfd0e..f75bedc50e00 100644
> > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > @@ -3927,6 +3927,10 @@
> > > no_uaccess_flush
> > > [PPC] Don't flush the L1-D cache after accessing user data.
> > >
> > > + no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
> > > + native_queued_spinlock when the nopvspin option is enabled.
> > > + This would help vcpu=pcpu scenarios.
> > > +
> > > novmcoredd [KNL,KDUMP]
> > > Disable device dump. Device dump allows drivers to
> > > append dump data to vmcore so you can collect driver
> > > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > > index 501e06e52078..e0233b3d7a5f 100644
> > > --- a/arch/riscv/include/asm/sbi.h
> > > +++ b/arch/riscv/include/asm/sbi.h
> > > @@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
> > > SBI_EXT_BASE_GET_MIMPID,
> > > };
> > >
> > > +enum sbi_ext_base_impl_id {
> > > + SBI_EXT_BASE_IMPL_ID_BBL = 0,
> > > + SBI_EXT_BASE_IMPL_ID_OPENSBI,
> > > + SBI_EXT_BASE_IMPL_ID_XVISOR,
> > > + SBI_EXT_BASE_IMPL_ID_KVM,
> > > +};
> > > +
> > > enum sbi_ext_time_fid {
> > > SBI_EXT_TIME_SET_TIMER = 0,
> > > };
> > > @@ -269,6 +276,7 @@ int sbi_console_getchar(void);
> > > long sbi_get_mvendorid(void);
> > > long sbi_get_marchid(void);
> > > long sbi_get_mimpid(void);
> > > +long sbi_get_firmware_id(void);
> > > void sbi_set_timer(uint64_t stime_value);
> > > void sbi_shutdown(void);
> > > void sbi_send_ipi(unsigned int cpu);
> > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > index 8ea0fee80652..6b38d6616f14 100644
> > > --- a/arch/riscv/include/asm/spinlock.h
> > > +++ b/arch/riscv/include/asm/spinlock.h
> > > @@ -4,6 +4,28 @@
> > > #define __ASM_RISCV_SPINLOCK_H
> > >
> > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > +/*
> > > + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
> > > + * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
> > > + * would shortcut for the queued_spin_lock_slowpath() function that allow
> > > + * virt_spin_lock to hijack it.
> > > + */
> > > +DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > +
> > > +#define virt_spin_lock virt_spin_lock
> > > +static inline bool virt_spin_lock(struct qspinlock *lock)
> > > +{
> > > + if (!static_branch_likely(&virt_spin_lock_key))
> > > + return false;
> > > +
> > > + do {
> > > + while (atomic_read(&lock->val) != 0)
> > > + cpu_relax();
> > > + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
> > > +
> > > + return true;
> > > +}
> > > +
> > > #define _Q_PENDING_LOOPS (1 << 9)
> > > #endif
> > >
> > > diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> > > index 88eea3a99ee0..cdd45edc8db4 100644
> > > --- a/arch/riscv/kernel/sbi.c
> > > +++ b/arch/riscv/kernel/sbi.c
> > > @@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
> > > return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
> > > }
> > >
> > > -static inline long sbi_get_firmware_id(void)
> > > +long sbi_get_firmware_id(void)
> > > {
> > > return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
> > > }
> > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > index 0f084f037651..c57d15b05160 100644
> > > --- a/arch/riscv/kernel/setup.c
> > > +++ b/arch/riscv/kernel/setup.c
> > > @@ -26,6 +26,7 @@
> > > #include <asm/alternative.h>
> > > #include <asm/cacheflush.h>
> > > #include <asm/cpu_ops.h>
> > > +#include <asm/cpufeature.h>
> > > #include <asm/early_ioremap.h>
> > > #include <asm/pgtable.h>
> > > #include <asm/setup.h>
> > > @@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > EXPORT_SYMBOL(combo_qspinlock_key);
> > > #endif
> > >
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > +static bool no_virt_spin_key = false;
> >
> > I suggest no _key, also there is no need for "= false".
> > To be consistent with enable_qspinlock, I also suggest
> > adding __ro_after_init:
> >
> > static bool no_virt_spin __ro_after_init;
> okay.
>
> >
> >
> >
> > > +DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > +
> > > +static int __init no_virt_spin_setup(char *p)
> > > +{
> > > + no_virt_spin_key = true;
> > > +
> > > + return 0;
> > > +}
> > > +early_param("no_virt_spin", no_virt_spin_setup);
> > > +
> > > +static void __init virt_spin_lock_init(void)
> > > +{
> > > + if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
> > > + no_virt_spin_key)
> > > + static_branch_disable(&virt_spin_lock_key);
> > > + else
> > > + pr_info("Enable virt_spin_lock\n");
> > > +}
> > > +#endif
> > > +
> >
> > A new virt_no_spin kernel parameter was introduced, but without
> > CONFIG_QUEUED_SPINLOCKS it will silently fail.
> >
> > I would suggest an #else clause here with a function to print an error /
> > warning message about no_virt_spin being invalid in this scenario.
> > It will probably help future debugging.
> If CONFIG_QUEUED_SPINLOCKS=n, no_virt_spin should be quiet. The
> no_virt_spin is one path of qspinlock.
IIUC having no_virt_spin being passed as parameter to a kernel with
CONFIG_QUEUED_SPINLOCKS=n is not supposed to have any warning this
parameter is useless.
I was just thinking it would be nice to have this warning during debugging,
but if it's standard practice then I am ok with this.
>
> >
> >
> > > static void __init riscv_spinlock_init(void)
> > > {
> > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > - if (!enable_qspinlock_key) {
> > > + if (!enable_qspinlock_key &&
> > > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > > static_branch_disable(&combo_qspinlock_key);
> > > pr_info("Ticket spinlock: enabled\n");
> > > } else {
> > > pr_info("Queued spinlock: enabled\n");
> > > }
> > > #endif
> > > +
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > + virt_spin_lock_init();
> > > +#endif
> > > }
> > >
> > > extern void __init init_rt_signal_env(void);
> > > --
> > > 2.36.1
> > >
> >
> > I am probably missing something out, but it looks to me that this patch is
> > causing 2 different changes:
> > 1 - Enabling no_virt_spin parameter
> > 2 - Disabling queued spinlocks for some firmware_id
> >
> > Wouldn't be better to split those changes in multiple patches?
> > Or am I missing the point on why they need to be together?
^ Want your input on this
Thanks!
Leo
> >
> > Thanks!
> > Leo
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
@ 2023-09-19 5:30 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:30 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 11:12:31PM +0800, Guo Ren wrote:
> On Thu, Sep 14, 2023 at 4:02 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:29:02AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Add a static key controlling whether virt_spin_lock() should be
> > > called or not. When running on bare metal set the new key to
> > > false.
> > >
> > > The KVM guests fall back to a Test-and-Set spinlock, because fair
> > > locks have horrible lock 'holder' preemption issues. The
> > > virt_spin_lock_key would shortcut for the
> > > queued_spin_lock_slowpath() function that allow virt_spin_lock to
> > > hijack it.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > .../admin-guide/kernel-parameters.txt | 4 +++
> > > arch/riscv/include/asm/sbi.h | 8 +++++
> > > arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
> > > arch/riscv/kernel/sbi.c | 2 +-
> > > arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
> > > 5 files changed, 64 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > index 61cacb8dfd0e..f75bedc50e00 100644
> > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > @@ -3927,6 +3927,10 @@
> > > no_uaccess_flush
> > > [PPC] Don't flush the L1-D cache after accessing user data.
> > >
> > > + no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
> > > + native_queued_spinlock when the nopvspin option is enabled.
> > > + This would help vcpu=pcpu scenarios.
> > > +
> > > novmcoredd [KNL,KDUMP]
> > > Disable device dump. Device dump allows drivers to
> > > append dump data to vmcore so you can collect driver
> > > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > > index 501e06e52078..e0233b3d7a5f 100644
> > > --- a/arch/riscv/include/asm/sbi.h
> > > +++ b/arch/riscv/include/asm/sbi.h
> > > @@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
> > > SBI_EXT_BASE_GET_MIMPID,
> > > };
> > >
> > > +enum sbi_ext_base_impl_id {
> > > + SBI_EXT_BASE_IMPL_ID_BBL = 0,
> > > + SBI_EXT_BASE_IMPL_ID_OPENSBI,
> > > + SBI_EXT_BASE_IMPL_ID_XVISOR,
> > > + SBI_EXT_BASE_IMPL_ID_KVM,
> > > +};
> > > +
> > > enum sbi_ext_time_fid {
> > > SBI_EXT_TIME_SET_TIMER = 0,
> > > };
> > > @@ -269,6 +276,7 @@ int sbi_console_getchar(void);
> > > long sbi_get_mvendorid(void);
> > > long sbi_get_marchid(void);
> > > long sbi_get_mimpid(void);
> > > +long sbi_get_firmware_id(void);
> > > void sbi_set_timer(uint64_t stime_value);
> > > void sbi_shutdown(void);
> > > void sbi_send_ipi(unsigned int cpu);
> > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > index 8ea0fee80652..6b38d6616f14 100644
> > > --- a/arch/riscv/include/asm/spinlock.h
> > > +++ b/arch/riscv/include/asm/spinlock.h
> > > @@ -4,6 +4,28 @@
> > > #define __ASM_RISCV_SPINLOCK_H
> > >
> > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > +/*
> > > + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
> > > + * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
> > > + * would shortcut for the queued_spin_lock_slowpath() function that allow
> > > + * virt_spin_lock to hijack it.
> > > + */
> > > +DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > +
> > > +#define virt_spin_lock virt_spin_lock
> > > +static inline bool virt_spin_lock(struct qspinlock *lock)
> > > +{
> > > + if (!static_branch_likely(&virt_spin_lock_key))
> > > + return false;
> > > +
> > > + do {
> > > + while (atomic_read(&lock->val) != 0)
> > > + cpu_relax();
> > > + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
> > > +
> > > + return true;
> > > +}
> > > +
> > > #define _Q_PENDING_LOOPS (1 << 9)
> > > #endif
> > >
> > > diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> > > index 88eea3a99ee0..cdd45edc8db4 100644
> > > --- a/arch/riscv/kernel/sbi.c
> > > +++ b/arch/riscv/kernel/sbi.c
> > > @@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
> > > return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
> > > }
> > >
> > > -static inline long sbi_get_firmware_id(void)
> > > +long sbi_get_firmware_id(void)
> > > {
> > > return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
> > > }
> > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > index 0f084f037651..c57d15b05160 100644
> > > --- a/arch/riscv/kernel/setup.c
> > > +++ b/arch/riscv/kernel/setup.c
> > > @@ -26,6 +26,7 @@
> > > #include <asm/alternative.h>
> > > #include <asm/cacheflush.h>
> > > #include <asm/cpu_ops.h>
> > > +#include <asm/cpufeature.h>
> > > #include <asm/early_ioremap.h>
> > > #include <asm/pgtable.h>
> > > #include <asm/setup.h>
> > > @@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > EXPORT_SYMBOL(combo_qspinlock_key);
> > > #endif
> > >
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > +static bool no_virt_spin_key = false;
> >
> > I suggest no _key, also there is no need for "= false".
> > To be consistent with enable_qspinlock, I also suggest
> > adding __ro_after_init:
> >
> > static bool no_virt_spin __ro_after_init;
> okay.
>
> >
> >
> >
> > > +DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > +
> > > +static int __init no_virt_spin_setup(char *p)
> > > +{
> > > + no_virt_spin_key = true;
> > > +
> > > + return 0;
> > > +}
> > > +early_param("no_virt_spin", no_virt_spin_setup);
> > > +
> > > +static void __init virt_spin_lock_init(void)
> > > +{
> > > + if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
> > > + no_virt_spin_key)
> > > + static_branch_disable(&virt_spin_lock_key);
> > > + else
> > > + pr_info("Enable virt_spin_lock\n");
> > > +}
> > > +#endif
> > > +
> >
> > A new virt_no_spin kernel parameter was introduced, but without
> > CONFIG_QUEUED_SPINLOCKS it will silently fail.
> >
> > I would suggest an #else clause here with a function to print an error /
> > warning message about no_virt_spin being invalid in this scenario.
> > It will probably help future debugging.
> If CONFIG_QUEUED_SPINLOCKS=n, no_virt_spin should be quiet. The
> no_virt_spin is one path of qspinlock.
IIUC having no_virt_spin being passed as parameter to a kernel with
CONFIG_QUEUED_SPINLOCKS=n is not supposed to have any warning this
parameter is useless.
I was just thinking it would be nice to have this warning during debugging,
but if it's standard practice then I am ok with this.
>
> >
> >
> > > static void __init riscv_spinlock_init(void)
> > > {
> > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > - if (!enable_qspinlock_key) {
> > > + if (!enable_qspinlock_key &&
> > > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > > static_branch_disable(&combo_qspinlock_key);
> > > pr_info("Ticket spinlock: enabled\n");
> > > } else {
> > > pr_info("Queued spinlock: enabled\n");
> > > }
> > > #endif
> > > +
> > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > + virt_spin_lock_init();
> > > +#endif
> > > }
> > >
> > > extern void __init init_rt_signal_env(void);
> > > --
> > > 2.36.1
> > >
> >
> > I am probably missing something out, but it looks to me that this patch is
> > causing 2 different changes:
> > 1 - Enabling no_virt_spin parameter
> > 2 - Disabling queued spinlocks for some firmware_id
> >
> > Wouldn't be better to split those changes in multiple patches?
> > Or am I missing the point on why they need to be together?
^ Want your input on this
Thanks!
Leo
> >
> > Thanks!
> > Leo
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 09/17] riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
2023-09-17 15:15 ` Guo Ren
@ 2023-09-19 5:34 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:34 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 11:15:51PM +0800, Guo Ren wrote:
> On Thu, Sep 14, 2023 at 4:32 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:29:03AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > The early version of T-Head C9xx cores has a store merge buffer
> > > delay problem. The store merge buffer could improve the store queue
> > > performance by merging multi-store requests, but when there are not
> > > continued store requests, the prior single store request would be
> > > waiting in the store queue for a long time. That would cause
> > > significant problems for communication between multi-cores. This
> > > problem was found on sg2042 & th1520 platforms with the qspinlock
> > > lock torture test.
> > >
> > > So appending a fence w.o could immediately flush the store merge
> > > buffer and let other cores see the write result.
> > >
> > > This will apply the WRITE_ONCE errata to handle the non-standard
> > > behavior via appending a fence w.o instruction for WRITE_ONCE().
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > arch/riscv/Kconfig.errata | 19 +++++++++++++++++++
> > > arch/riscv/errata/thead/errata.c | 20 ++++++++++++++++++++
> > > arch/riscv/include/asm/errata_list.h | 13 -------------
> > > arch/riscv/include/asm/rwonce.h | 24 ++++++++++++++++++++++++
> > > arch/riscv/include/asm/vendorid_list.h | 14 ++++++++++++++
> > > include/asm-generic/rwonce.h | 2 ++
> > > 6 files changed, 79 insertions(+), 13 deletions(-)
> > > create mode 100644 arch/riscv/include/asm/rwonce.h
> > >
> > > diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata
> > > index 1aa85a427ff3..c919cc3f1a3a 100644
> > > --- a/arch/riscv/Kconfig.errata
> > > +++ b/arch/riscv/Kconfig.errata
> > > @@ -77,4 +77,23 @@ config ERRATA_THEAD_PMU
> > >
> > > If you don't know what to do here, say "Y".
> > >
> > > +config ERRATA_THEAD_WRITE_ONCE
> > > + bool "Apply T-Head WRITE_ONCE errata"
> > > + depends on ERRATA_THEAD
> > > + default y
> > > + help
> > > + The early version of T-Head C9xx cores has a store merge buffer
> > > + delay problem. The store merge buffer could improve the store queue
> > > + performance by merging multi-store requests, but when there are no
> > > + continued store requests, the prior single store request would be
> > > + waiting in the store queue for a long time. That would cause
> > > + significant problems for communication between multi-cores. Appending
> > > + a fence w.o could immediately flush the store merge buffer and let
> > > + other cores see the write result.
> > > +
> > > + This will apply the WRITE_ONCE errata to handle the non-standard
> > > + behavior via appending a fence w.o instruction for WRITE_ONCE().
> > > +
> > > + If you don't know what to do here, say "Y".
> > > +
> > > endmenu # "CPU errata selection"
> > > diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
> > > index be84b14f0118..751eb5a7f614 100644
> > > --- a/arch/riscv/errata/thead/errata.c
> > > +++ b/arch/riscv/errata/thead/errata.c
> > > @@ -69,6 +69,23 @@ static bool errata_probe_pmu(unsigned int stage,
> > > return true;
> > > }
> > >
> > > +static bool errata_probe_write_once(unsigned int stage,
> > > + unsigned long arch_id, unsigned long impid)
> > > +{
> > > + if (!IS_ENABLED(CONFIG_ERRATA_THEAD_WRITE_ONCE))
> > > + return false;
> > > +
> > > + /* target-c9xx cores report arch_id and impid as 0 */
> > > + if (arch_id != 0 || impid != 0)
> > > + return false;
> > > +
> > > + if (stage == RISCV_ALTERNATIVES_BOOT ||
> > > + stage == RISCV_ALTERNATIVES_MODULE)
> > > + return true;
> > > +
> > > + return false;
> > > +}
> > > +
> > > static u32 thead_errata_probe(unsigned int stage,
> > > unsigned long archid, unsigned long impid)
> > > {
> > > @@ -83,6 +100,9 @@ static u32 thead_errata_probe(unsigned int stage,
> > > if (errata_probe_pmu(stage, archid, impid))
> > > cpu_req_errata |= BIT(ERRATA_THEAD_PMU);
> > >
> > > + if (errata_probe_write_once(stage, archid, impid))
> > > + cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
> > > +
> > > return cpu_req_errata;
> > > }
> > >
> > > diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
> > > index 712cab7adffe..fbb2b8d39321 100644
> > > --- a/arch/riscv/include/asm/errata_list.h
> > > +++ b/arch/riscv/include/asm/errata_list.h
> > > @@ -11,19 +11,6 @@
> > > #include <asm/hwcap.h>
> > > #include <asm/vendorid_list.h>
> > >
> > > -#ifdef CONFIG_ERRATA_SIFIVE
> > > -#define ERRATA_SIFIVE_CIP_453 0
> > > -#define ERRATA_SIFIVE_CIP_1200 1
> > > -#define ERRATA_SIFIVE_NUMBER 2
> > > -#endif
> > > -
> > > -#ifdef CONFIG_ERRATA_THEAD
> > > -#define ERRATA_THEAD_PBMT 0
> > > -#define ERRATA_THEAD_CMO 1
> > > -#define ERRATA_THEAD_PMU 2
> > > -#define ERRATA_THEAD_NUMBER 3
> > > -#endif
> > > -
> >
> > Here I understand you are moving stuff from errata_list.h to
> > vendorid_list.h. Wouldn't it be better to do this on a separated patch
> > before this one?
> Okay.
>
> >
> > I understand this is used here, but it looks like it's unrelated.
> >
> > > #ifdef __ASSEMBLY__
> > >
> > > #define ALT_INSN_FAULT(x) \
> > > diff --git a/arch/riscv/include/asm/rwonce.h b/arch/riscv/include/asm/rwonce.h
> > > new file mode 100644
> > > index 000000000000..be0b8864969d
> > > --- /dev/null
> > > +++ b/arch/riscv/include/asm/rwonce.h
> > > @@ -0,0 +1,24 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +
> > > +#ifndef __ASM_RWONCE_H
> > > +#define __ASM_RWONCE_H
> > > +
> > > +#include <linux/compiler_types.h>
> > > +#include <asm/alternative-macros.h>
> > > +#include <asm/vendorid_list.h>
> > > +
> > > +#define __WRITE_ONCE(x, val) \
> > > +do { \
> > > + *(volatile typeof(x) *)&(x) = (val); \
> > > + asm volatile(ALTERNATIVE( \
> > > + __nops(1), \
> > > + "fence w, o\n\t", \
> > > + THEAD_VENDOR_ID, \
> > > + ERRATA_THEAD_WRITE_ONCE, \
> > > + CONFIG_ERRATA_THEAD_WRITE_ONCE) \
> > > + : : : "memory"); \
> > > +} while (0)
> > > +
> > > +#include <asm-generic/rwonce.h>
> > > +
> > > +#endif /* __ASM_RWONCE_H */
> >
> > IIUC the idea here is to have an alternative __WRITE_ONCE that replaces the
> > asm-generic one.
> >
> > Honestly, this asm alternative here seems too much information, and too
> > cryptic. I mean, yeah in the patch it all makes sense, but I imagine myself
> > in the future looking at all this and trying to understand what is going
> > on.
> >
> > Wouldn't it look better to have something like:
> >
> > #####
> >
> > /* Some explanation like the one on Kconfig */
> >
> > #define write_once_flush() \
> > do { \
> > asm volatile(ALTERNATIVE( \
> > __nops(1), \
> > "fence w, o\n\t", \
> > THEAD_VENDOR_ID, \
> > ERRATA_THEAD_WRITE_ONCE, \
> > CONFIG_ERRATA_THEAD_WRITE_ONCE) \
> > : : : "memory"); \
> > } while(0)
> >
> >
> > #define __WRITE_ONCE(x, val) \
> > do { \
> > *(volatile typeof(x) *)&(x) = (val); \
> > write_once_flush(); \
> > } while(0)
> >
> > #####
> >
> >
> > This way I could quickly see there is a flush after the writting of
> > WRITE_ONCE(), and this flush is the above "complicated" asm.
> >
> > What do you think?
> Okay, good point, and I would take it.
Thanks!
Once you take the above suggestions, please include in your next patch:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
> >
> > > diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
> > > index cb89af3f0704..73078cfe4029 100644
> > > --- a/arch/riscv/include/asm/vendorid_list.h
> > > +++ b/arch/riscv/include/asm/vendorid_list.h
> > > @@ -8,4 +8,18 @@
> > > #define SIFIVE_VENDOR_ID 0x489
> > > #define THEAD_VENDOR_ID 0x5b7
> > >
> > > +#ifdef CONFIG_ERRATA_SIFIVE
> > > +#define ERRATA_SIFIVE_CIP_453 0
> > > +#define ERRATA_SIFIVE_CIP_1200 1
> > > +#define ERRATA_SIFIVE_NUMBER 2
> > > +#endif
> > > +
> > > +#ifdef CONFIG_ERRATA_THEAD
> > > +#define ERRATA_THEAD_PBMT 0
> > > +#define ERRATA_THEAD_CMO 1
> > > +#define ERRATA_THEAD_PMU 2
> > > +#define ERRATA_THEAD_WRITE_ONCE 3
> > > +#define ERRATA_THEAD_NUMBER 4
> > > +#endif
> > > +
> > > #endif
> > > diff --git a/include/asm-generic/rwonce.h b/include/asm-generic/rwonce.h
> > > index 8d0a6280e982..fb07fe8c6e45 100644
> > > --- a/include/asm-generic/rwonce.h
> > > +++ b/include/asm-generic/rwonce.h
> > > @@ -50,10 +50,12 @@
> > > __READ_ONCE(x); \
> > > })
> > >
> > > +#ifndef __WRITE_ONCE
> > > #define __WRITE_ONCE(x, val) \
> > > do { \
> > > *(volatile typeof(x) *)&(x) = (val); \
> > > } while (0)
> > > +#endif
> > >
> > > #define WRITE_ONCE(x, val) \
> > > do { \
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 09/17] riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
@ 2023-09-19 5:34 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:34 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 11:15:51PM +0800, Guo Ren wrote:
> On Thu, Sep 14, 2023 at 4:32 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:29:03AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > The early version of T-Head C9xx cores has a store merge buffer
> > > delay problem. The store merge buffer could improve the store queue
> > > performance by merging multi-store requests, but when there are not
> > > continued store requests, the prior single store request would be
> > > waiting in the store queue for a long time. That would cause
> > > significant problems for communication between multi-cores. This
> > > problem was found on sg2042 & th1520 platforms with the qspinlock
> > > lock torture test.
> > >
> > > So appending a fence w.o could immediately flush the store merge
> > > buffer and let other cores see the write result.
> > >
> > > This will apply the WRITE_ONCE errata to handle the non-standard
> > > behavior via appending a fence w.o instruction for WRITE_ONCE().
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > arch/riscv/Kconfig.errata | 19 +++++++++++++++++++
> > > arch/riscv/errata/thead/errata.c | 20 ++++++++++++++++++++
> > > arch/riscv/include/asm/errata_list.h | 13 -------------
> > > arch/riscv/include/asm/rwonce.h | 24 ++++++++++++++++++++++++
> > > arch/riscv/include/asm/vendorid_list.h | 14 ++++++++++++++
> > > include/asm-generic/rwonce.h | 2 ++
> > > 6 files changed, 79 insertions(+), 13 deletions(-)
> > > create mode 100644 arch/riscv/include/asm/rwonce.h
> > >
> > > diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata
> > > index 1aa85a427ff3..c919cc3f1a3a 100644
> > > --- a/arch/riscv/Kconfig.errata
> > > +++ b/arch/riscv/Kconfig.errata
> > > @@ -77,4 +77,23 @@ config ERRATA_THEAD_PMU
> > >
> > > If you don't know what to do here, say "Y".
> > >
> > > +config ERRATA_THEAD_WRITE_ONCE
> > > + bool "Apply T-Head WRITE_ONCE errata"
> > > + depends on ERRATA_THEAD
> > > + default y
> > > + help
> > > + The early version of T-Head C9xx cores has a store merge buffer
> > > + delay problem. The store merge buffer could improve the store queue
> > > + performance by merging multi-store requests, but when there are no
> > > + continued store requests, the prior single store request would be
> > > + waiting in the store queue for a long time. That would cause
> > > + significant problems for communication between multi-cores. Appending
> > > + a fence w.o could immediately flush the store merge buffer and let
> > > + other cores see the write result.
> > > +
> > > + This will apply the WRITE_ONCE errata to handle the non-standard
> > > + behavior via appending a fence w.o instruction for WRITE_ONCE().
> > > +
> > > + If you don't know what to do here, say "Y".
> > > +
> > > endmenu # "CPU errata selection"
> > > diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
> > > index be84b14f0118..751eb5a7f614 100644
> > > --- a/arch/riscv/errata/thead/errata.c
> > > +++ b/arch/riscv/errata/thead/errata.c
> > > @@ -69,6 +69,23 @@ static bool errata_probe_pmu(unsigned int stage,
> > > return true;
> > > }
> > >
> > > +static bool errata_probe_write_once(unsigned int stage,
> > > + unsigned long arch_id, unsigned long impid)
> > > +{
> > > + if (!IS_ENABLED(CONFIG_ERRATA_THEAD_WRITE_ONCE))
> > > + return false;
> > > +
> > > + /* target-c9xx cores report arch_id and impid as 0 */
> > > + if (arch_id != 0 || impid != 0)
> > > + return false;
> > > +
> > > + if (stage == RISCV_ALTERNATIVES_BOOT ||
> > > + stage == RISCV_ALTERNATIVES_MODULE)
> > > + return true;
> > > +
> > > + return false;
> > > +}
> > > +
> > > static u32 thead_errata_probe(unsigned int stage,
> > > unsigned long archid, unsigned long impid)
> > > {
> > > @@ -83,6 +100,9 @@ static u32 thead_errata_probe(unsigned int stage,
> > > if (errata_probe_pmu(stage, archid, impid))
> > > cpu_req_errata |= BIT(ERRATA_THEAD_PMU);
> > >
> > > + if (errata_probe_write_once(stage, archid, impid))
> > > + cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
> > > +
> > > return cpu_req_errata;
> > > }
> > >
> > > diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
> > > index 712cab7adffe..fbb2b8d39321 100644
> > > --- a/arch/riscv/include/asm/errata_list.h
> > > +++ b/arch/riscv/include/asm/errata_list.h
> > > @@ -11,19 +11,6 @@
> > > #include <asm/hwcap.h>
> > > #include <asm/vendorid_list.h>
> > >
> > > -#ifdef CONFIG_ERRATA_SIFIVE
> > > -#define ERRATA_SIFIVE_CIP_453 0
> > > -#define ERRATA_SIFIVE_CIP_1200 1
> > > -#define ERRATA_SIFIVE_NUMBER 2
> > > -#endif
> > > -
> > > -#ifdef CONFIG_ERRATA_THEAD
> > > -#define ERRATA_THEAD_PBMT 0
> > > -#define ERRATA_THEAD_CMO 1
> > > -#define ERRATA_THEAD_PMU 2
> > > -#define ERRATA_THEAD_NUMBER 3
> > > -#endif
> > > -
> >
> > Here I understand you are moving stuff from errata_list.h to
> > vendorid_list.h. Wouldn't it be better to do this on a separated patch
> > before this one?
> Okay.
>
> >
> > I understand this is used here, but it looks like it's unrelated.
> >
> > > #ifdef __ASSEMBLY__
> > >
> > > #define ALT_INSN_FAULT(x) \
> > > diff --git a/arch/riscv/include/asm/rwonce.h b/arch/riscv/include/asm/rwonce.h
> > > new file mode 100644
> > > index 000000000000..be0b8864969d
> > > --- /dev/null
> > > +++ b/arch/riscv/include/asm/rwonce.h
> > > @@ -0,0 +1,24 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +
> > > +#ifndef __ASM_RWONCE_H
> > > +#define __ASM_RWONCE_H
> > > +
> > > +#include <linux/compiler_types.h>
> > > +#include <asm/alternative-macros.h>
> > > +#include <asm/vendorid_list.h>
> > > +
> > > +#define __WRITE_ONCE(x, val) \
> > > +do { \
> > > + *(volatile typeof(x) *)&(x) = (val); \
> > > + asm volatile(ALTERNATIVE( \
> > > + __nops(1), \
> > > + "fence w, o\n\t", \
> > > + THEAD_VENDOR_ID, \
> > > + ERRATA_THEAD_WRITE_ONCE, \
> > > + CONFIG_ERRATA_THEAD_WRITE_ONCE) \
> > > + : : : "memory"); \
> > > +} while (0)
> > > +
> > > +#include <asm-generic/rwonce.h>
> > > +
> > > +#endif /* __ASM_RWONCE_H */
> >
> > IIUC the idea here is to have an alternative __WRITE_ONCE that replaces the
> > asm-generic one.
> >
> > Honestly, this asm alternative here seems too much information, and too
> > cryptic. I mean, yeah in the patch it all makes sense, but I imagine myself
> > in the future looking at all this and trying to understand what is going
> > on.
> >
> > Wouldn't it look better to have something like:
> >
> > #####
> >
> > /* Some explanation like the one on Kconfig */
> >
> > #define write_once_flush() \
> > do { \
> > asm volatile(ALTERNATIVE( \
> > __nops(1), \
> > "fence w, o\n\t", \
> > THEAD_VENDOR_ID, \
> > ERRATA_THEAD_WRITE_ONCE, \
> > CONFIG_ERRATA_THEAD_WRITE_ONCE) \
> > : : : "memory"); \
> > } while(0)
> >
> >
> > #define __WRITE_ONCE(x, val) \
> > do { \
> > *(volatile typeof(x) *)&(x) = (val); \
> > write_once_flush(); \
> > } while(0)
> >
> > #####
> >
> >
> > This way I could quickly see there is a flush after the writting of
> > WRITE_ONCE(), and this flush is the above "complicated" asm.
> >
> > What do you think?
> Okay, good point, and I would take it.
Thanks!
Once you take the above suggestions, please include in your next patch:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
>
> >
> > > diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
> > > index cb89af3f0704..73078cfe4029 100644
> > > --- a/arch/riscv/include/asm/vendorid_list.h
> > > +++ b/arch/riscv/include/asm/vendorid_list.h
> > > @@ -8,4 +8,18 @@
> > > #define SIFIVE_VENDOR_ID 0x489
> > > #define THEAD_VENDOR_ID 0x5b7
> > >
> > > +#ifdef CONFIG_ERRATA_SIFIVE
> > > +#define ERRATA_SIFIVE_CIP_453 0
> > > +#define ERRATA_SIFIVE_CIP_1200 1
> > > +#define ERRATA_SIFIVE_NUMBER 2
> > > +#endif
> > > +
> > > +#ifdef CONFIG_ERRATA_THEAD
> > > +#define ERRATA_THEAD_PBMT 0
> > > +#define ERRATA_THEAD_CMO 1
> > > +#define ERRATA_THEAD_PMU 2
> > > +#define ERRATA_THEAD_WRITE_ONCE 3
> > > +#define ERRATA_THEAD_NUMBER 4
> > > +#endif
> > > +
> > > #endif
> > > diff --git a/include/asm-generic/rwonce.h b/include/asm-generic/rwonce.h
> > > index 8d0a6280e982..fb07fe8c6e45 100644
> > > --- a/include/asm-generic/rwonce.h
> > > +++ b/include/asm-generic/rwonce.h
> > > @@ -50,10 +50,12 @@
> > > __READ_ONCE(x); \
> > > })
> > >
> > > +#ifndef __WRITE_ONCE
> > > #define __WRITE_ONCE(x, val) \
> > > do { \
> > > *(volatile typeof(x) *)&(x) = (val); \
> > > } while (0)
> > > +#endif
> > >
> > > #define WRITE_ONCE(x, val) \
> > > do { \
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 11/17] RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
2023-09-17 14:58 ` Guo Ren
@ 2023-09-19 5:43 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:43 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 10:58:18PM +0800, Guo Ren wrote:
> On Fri, Sep 15, 2023 at 1:42 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:29:05AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Using static_call to switch between:
> > > native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath()
> > > native_queued_spin_unlock() __pv_queued_spin_unlock()
> > >
> > > Finish the pv_wait implementation, but pv_kick needs the SBI
> > > definition of the next patches.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > arch/riscv/include/asm/Kbuild | 1 -
> > > arch/riscv/include/asm/qspinlock.h | 35 +++++++++++++
> > > arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++++++++
> > > arch/riscv/include/asm/spinlock.h | 2 +-
> > > arch/riscv/kernel/qspinlock_paravirt.c | 57 +++++++++++++++++++++
> > > arch/riscv/kernel/setup.c | 4 ++
> > > 6 files changed, 126 insertions(+), 2 deletions(-)
> > > create mode 100644 arch/riscv/include/asm/qspinlock.h
> > > create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> > > create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> > >
> > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > index a0dc85e4a754..b89cb3b73c13 100644
> > > --- a/arch/riscv/include/asm/Kbuild
> > > +++ b/arch/riscv/include/asm/Kbuild
> > > @@ -7,6 +7,5 @@ generic-y += parport.h
> > > generic-y += spinlock_types.h
> > > generic-y += qrwlock.h
> > > generic-y += qrwlock_types.h
> > > -generic-y += qspinlock.h
> > > generic-y += user.h
> > > generic-y += vmlinux.lds.h
> > > diff --git a/arch/riscv/include/asm/qspinlock.h b/arch/riscv/include/asm/qspinlock.h
> > > new file mode 100644
> > > index 000000000000..7d4f416c908c
> > > --- /dev/null
> > > +++ b/arch/riscv/include/asm/qspinlock.h
> > > @@ -0,0 +1,35 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Copyright (c), 2023 Alibaba Cloud
> > > + * Authors:
> > > + * Guo Ren <guoren@linux.alibaba.com>
> > > + */
> > > +
> > > +#ifndef _ASM_RISCV_QSPINLOCK_H
> > > +#define _ASM_RISCV_QSPINLOCK_H
> > > +
> > > +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> > > +#include <asm/qspinlock_paravirt.h>
> > > +
> > > +/* How long a lock should spin before we consider blocking */
> > > +#define SPIN_THRESHOLD (1 << 15)
> > > +
> > > +void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > > +void __pv_init_lock_hash(void);
> > > +void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > > +
> > > +static inline void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
> > > +{
> > > + static_call(pv_queued_spin_lock_slowpath)(lock, val);
> > > +}
> > > +
> > > +#define queued_spin_unlock queued_spin_unlock
> > > +static inline void queued_spin_unlock(struct qspinlock *lock)
> > > +{
> > > + static_call(pv_queued_spin_unlock)(lock);
> > > +}
> > > +#endif /* CONFIG_PARAVIRT_SPINLOCKS */
> > > +
> > > +#include <asm-generic/qspinlock.h>
> > > +
> > > +#endif /* _ASM_RISCV_QSPINLOCK_H */
> > > diff --git a/arch/riscv/include/asm/qspinlock_paravirt.h b/arch/riscv/include/asm/qspinlock_paravirt.h
> > > new file mode 100644
> > > index 000000000000..9681e851f69d
> > > --- /dev/null
> > > +++ b/arch/riscv/include/asm/qspinlock_paravirt.h
> > > @@ -0,0 +1,29 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Copyright (c), 2023 Alibaba Cloud
> > > + * Authors:
> > > + * Guo Ren <guoren@linux.alibaba.com>
> > > + */
> > > +
> > > +#ifndef _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> > > +#define _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> > > +
> > > +void pv_wait(u8 *ptr, u8 val);
> > > +void pv_kick(int cpu);
> > > +
> > > +void dummy_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > > +void dummy_queued_spin_unlock(struct qspinlock *lock);
> > > +
> > > +DECLARE_STATIC_CALL(pv_queued_spin_lock_slowpath, dummy_queued_spin_lock_slowpath);
> > > +DECLARE_STATIC_CALL(pv_queued_spin_unlock, dummy_queued_spin_unlock);
> > > +
> > > +void __init pv_qspinlock_init(void);
> > > +
> > > +static inline bool pv_is_native_spin_unlock(void)
> > > +{
> > > + return false;
> > > +}
> > > +
> > > +void __pv_queued_spin_unlock(struct qspinlock *lock);
> > > +
> > > +#endif /* _ASM_RISCV_QSPINLOCK_PARAVIRT_H */
> > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > index 6b38d6616f14..ed4253f491fe 100644
> > > --- a/arch/riscv/include/asm/spinlock.h
> > > +++ b/arch/riscv/include/asm/spinlock.h
> > > @@ -39,7 +39,7 @@ static inline bool virt_spin_lock(struct qspinlock *lock)
> > > #undef arch_spin_trylock
> > > #undef arch_spin_unlock
> > >
> > > -#include <asm-generic/qspinlock.h>
> > > +#include <asm/qspinlock.h>
> > > #include <linux/jump_label.h>
> > >
> > > #undef arch_spin_is_locked
> > > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > > new file mode 100644
> > > index 000000000000..85ff5a3ec234
> > > --- /dev/null
> > > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > > @@ -0,0 +1,57 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * Copyright (c), 2023 Alibaba Cloud
> > > + * Authors:
> > > + * Guo Ren <guoren@linux.alibaba.com>
> > > + */
> > > +
> > > +#include <linux/static_call.h>
> > > +#include <asm/qspinlock_paravirt.h>
> > > +#include <asm/sbi.h>
> > > +
> > > +void pv_kick(int cpu)
> > > +{
> > > + return;
> > > +}
> > > +
> > > +void pv_wait(u8 *ptr, u8 val)
> > > +{
> > > + unsigned long flags;
> > > +
> > > + if (in_nmi())
> > > + return;
> > > +
> > > + local_irq_save(flags);
> > > + if (READ_ONCE(*ptr) != val)
> > > + goto out;
> > > +
> > > + /* wait_for_interrupt(); */
> > > +out:
> > > + local_irq_restore(flags);
> > > +}
> > > +
> > > +static void native_queued_spin_unlock(struct qspinlock *lock)
> > > +{
> > > + smp_store_release(&lock->locked, 0);
> > > +}
> > > +
> > > +DEFINE_STATIC_CALL(pv_queued_spin_lock_slowpath, native_queued_spin_lock_slowpath);
> > > +EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> > > +
> > > +DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> > > +EXPORT_STATIC_CALL(pv_queued_spin_unlock);
> > > +
> > > +void __init pv_qspinlock_init(void)
> > > +{
> > > + if (num_possible_cpus() == 1)
> > > + return;
> > > +
> > > + if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
> >
> > Checks like this seem to be very common on this patchset.
> > For someone not much familiar with this, it can be hard to
> > understand.
> >
> > I mean, on patch 8/17 you introduce those IDs, which look to be
> > incremental ( ID == N includes stuff from ID < N ), but I am not sure as I
> > couln't find much documentation on that.
> It's from sbi spec:
> https://github.com/riscv-non-isa/riscv-sbi-doc/releases
>
> 0 Berkeley Boot Loader (BBL)
> 1 OpenSBI
> 2 Xvisor
> 3 KVM
> 4 RustSBI
> 5 Diosix
> 6 Coffer
> 7 Xen Project
> 8 PolarFire Hart Software Service
Oh, I see. Thanks for the reference!
Please also include the github link and/or the doc name into the commit
file for future references :)
>
> >
> > Then above you test for the id being different than
> > SBI_EXT_BASE_IMPL_ID_KVM, but if they are actually incremental and a new
> > version lands, the new version will also return early because it passes the
> > test.
> >
> > I am no sure if above is right, but it's all I could understand without
> > documentation.
> >
> > Well, my point is: this seems hard to understand & review, so it would be
> > nice to have a macro like this to be used instead:
> >
> > #define sbi_fw_implements_kvm() \
> > (sbi_get_firmware_id() >= SBI_EXT_BASE_IMPL_ID_KVM)
> No, it's not correct. It must be:
> (sbi_get_firmware_id() == SBI_EXT_BASE_IMPL_ID_KVM)
Looking at the doc you provided, I think to be able to understand it.
The idea is to provide a code for given implementation of SBI, so in those
tests you check if the SBI implementation being used is KVM, meaning it's a
KVM guest. Ok, that makes sense now. Thanks!
>
> >
> > if(!sbi_fw_implements_kvm())
> I'm okay with sbi_fw_implements_kvm().
Thanks! also, thanks again for sharing the doc!
With above suggestions, please feel free to include in next versions:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thx
Leo
>
> > return;
> >
> > What do you think?
> >
> > Other than that, LGTM.
> >
> > Thanks!
> > Leo
> >
> > > + return;
> > > +
> > > + pr_info("PV qspinlocks enabled\n");
> > > + __pv_init_lock_hash();
> > > +
> > > + static_call_update(pv_queued_spin_lock_slowpath, __pv_queued_spin_lock_slowpath);
> > > + static_call_update(pv_queued_spin_unlock, __pv_queued_spin_unlock);
> > > +}
> > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > index c57d15b05160..88690751f2ee 100644
> > > --- a/arch/riscv/kernel/setup.c
> > > +++ b/arch/riscv/kernel/setup.c
> > > @@ -321,6 +321,10 @@ static void __init riscv_spinlock_init(void)
> > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > virt_spin_lock_init();
> > > #endif
> > > +
> > > +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> > > + pv_qspinlock_init();
> > > +#endif
> > > }
> > >
> > > extern void __init init_rt_signal_env(void);
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 11/17] RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
@ 2023-09-19 5:43 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:43 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 10:58:18PM +0800, Guo Ren wrote:
> On Fri, Sep 15, 2023 at 1:42 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:29:05AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Using static_call to switch between:
> > > native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath()
> > > native_queued_spin_unlock() __pv_queued_spin_unlock()
> > >
> > > Finish the pv_wait implementation, but pv_kick needs the SBI
> > > definition of the next patches.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > arch/riscv/include/asm/Kbuild | 1 -
> > > arch/riscv/include/asm/qspinlock.h | 35 +++++++++++++
> > > arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++++++++
> > > arch/riscv/include/asm/spinlock.h | 2 +-
> > > arch/riscv/kernel/qspinlock_paravirt.c | 57 +++++++++++++++++++++
> > > arch/riscv/kernel/setup.c | 4 ++
> > > 6 files changed, 126 insertions(+), 2 deletions(-)
> > > create mode 100644 arch/riscv/include/asm/qspinlock.h
> > > create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> > > create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> > >
> > > diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
> > > index a0dc85e4a754..b89cb3b73c13 100644
> > > --- a/arch/riscv/include/asm/Kbuild
> > > +++ b/arch/riscv/include/asm/Kbuild
> > > @@ -7,6 +7,5 @@ generic-y += parport.h
> > > generic-y += spinlock_types.h
> > > generic-y += qrwlock.h
> > > generic-y += qrwlock_types.h
> > > -generic-y += qspinlock.h
> > > generic-y += user.h
> > > generic-y += vmlinux.lds.h
> > > diff --git a/arch/riscv/include/asm/qspinlock.h b/arch/riscv/include/asm/qspinlock.h
> > > new file mode 100644
> > > index 000000000000..7d4f416c908c
> > > --- /dev/null
> > > +++ b/arch/riscv/include/asm/qspinlock.h
> > > @@ -0,0 +1,35 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Copyright (c), 2023 Alibaba Cloud
> > > + * Authors:
> > > + * Guo Ren <guoren@linux.alibaba.com>
> > > + */
> > > +
> > > +#ifndef _ASM_RISCV_QSPINLOCK_H
> > > +#define _ASM_RISCV_QSPINLOCK_H
> > > +
> > > +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> > > +#include <asm/qspinlock_paravirt.h>
> > > +
> > > +/* How long a lock should spin before we consider blocking */
> > > +#define SPIN_THRESHOLD (1 << 15)
> > > +
> > > +void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > > +void __pv_init_lock_hash(void);
> > > +void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > > +
> > > +static inline void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
> > > +{
> > > + static_call(pv_queued_spin_lock_slowpath)(lock, val);
> > > +}
> > > +
> > > +#define queued_spin_unlock queued_spin_unlock
> > > +static inline void queued_spin_unlock(struct qspinlock *lock)
> > > +{
> > > + static_call(pv_queued_spin_unlock)(lock);
> > > +}
> > > +#endif /* CONFIG_PARAVIRT_SPINLOCKS */
> > > +
> > > +#include <asm-generic/qspinlock.h>
> > > +
> > > +#endif /* _ASM_RISCV_QSPINLOCK_H */
> > > diff --git a/arch/riscv/include/asm/qspinlock_paravirt.h b/arch/riscv/include/asm/qspinlock_paravirt.h
> > > new file mode 100644
> > > index 000000000000..9681e851f69d
> > > --- /dev/null
> > > +++ b/arch/riscv/include/asm/qspinlock_paravirt.h
> > > @@ -0,0 +1,29 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +/*
> > > + * Copyright (c), 2023 Alibaba Cloud
> > > + * Authors:
> > > + * Guo Ren <guoren@linux.alibaba.com>
> > > + */
> > > +
> > > +#ifndef _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> > > +#define _ASM_RISCV_QSPINLOCK_PARAVIRT_H
> > > +
> > > +void pv_wait(u8 *ptr, u8 val);
> > > +void pv_kick(int cpu);
> > > +
> > > +void dummy_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val);
> > > +void dummy_queued_spin_unlock(struct qspinlock *lock);
> > > +
> > > +DECLARE_STATIC_CALL(pv_queued_spin_lock_slowpath, dummy_queued_spin_lock_slowpath);
> > > +DECLARE_STATIC_CALL(pv_queued_spin_unlock, dummy_queued_spin_unlock);
> > > +
> > > +void __init pv_qspinlock_init(void);
> > > +
> > > +static inline bool pv_is_native_spin_unlock(void)
> > > +{
> > > + return false;
> > > +}
> > > +
> > > +void __pv_queued_spin_unlock(struct qspinlock *lock);
> > > +
> > > +#endif /* _ASM_RISCV_QSPINLOCK_PARAVIRT_H */
> > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > index 6b38d6616f14..ed4253f491fe 100644
> > > --- a/arch/riscv/include/asm/spinlock.h
> > > +++ b/arch/riscv/include/asm/spinlock.h
> > > @@ -39,7 +39,7 @@ static inline bool virt_spin_lock(struct qspinlock *lock)
> > > #undef arch_spin_trylock
> > > #undef arch_spin_unlock
> > >
> > > -#include <asm-generic/qspinlock.h>
> > > +#include <asm/qspinlock.h>
> > > #include <linux/jump_label.h>
> > >
> > > #undef arch_spin_is_locked
> > > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > > new file mode 100644
> > > index 000000000000..85ff5a3ec234
> > > --- /dev/null
> > > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > > @@ -0,0 +1,57 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * Copyright (c), 2023 Alibaba Cloud
> > > + * Authors:
> > > + * Guo Ren <guoren@linux.alibaba.com>
> > > + */
> > > +
> > > +#include <linux/static_call.h>
> > > +#include <asm/qspinlock_paravirt.h>
> > > +#include <asm/sbi.h>
> > > +
> > > +void pv_kick(int cpu)
> > > +{
> > > + return;
> > > +}
> > > +
> > > +void pv_wait(u8 *ptr, u8 val)
> > > +{
> > > + unsigned long flags;
> > > +
> > > + if (in_nmi())
> > > + return;
> > > +
> > > + local_irq_save(flags);
> > > + if (READ_ONCE(*ptr) != val)
> > > + goto out;
> > > +
> > > + /* wait_for_interrupt(); */
> > > +out:
> > > + local_irq_restore(flags);
> > > +}
> > > +
> > > +static void native_queued_spin_unlock(struct qspinlock *lock)
> > > +{
> > > + smp_store_release(&lock->locked, 0);
> > > +}
> > > +
> > > +DEFINE_STATIC_CALL(pv_queued_spin_lock_slowpath, native_queued_spin_lock_slowpath);
> > > +EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> > > +
> > > +DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> > > +EXPORT_STATIC_CALL(pv_queued_spin_unlock);
> > > +
> > > +void __init pv_qspinlock_init(void)
> > > +{
> > > + if (num_possible_cpus() == 1)
> > > + return;
> > > +
> > > + if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
> >
> > Checks like this seem to be very common on this patchset.
> > For someone not much familiar with this, it can be hard to
> > understand.
> >
> > I mean, on patch 8/17 you introduce those IDs, which look to be
> > incremental ( ID == N includes stuff from ID < N ), but I am not sure as I
> > couln't find much documentation on that.
> It's from sbi spec:
> https://github.com/riscv-non-isa/riscv-sbi-doc/releases
>
> 0 Berkeley Boot Loader (BBL)
> 1 OpenSBI
> 2 Xvisor
> 3 KVM
> 4 RustSBI
> 5 Diosix
> 6 Coffer
> 7 Xen Project
> 8 PolarFire Hart Software Service
Oh, I see. Thanks for the reference!
Please also include the github link and/or the doc name into the commit
file for future references :)
>
> >
> > Then above you test for the id being different than
> > SBI_EXT_BASE_IMPL_ID_KVM, but if they are actually incremental and a new
> > version lands, the new version will also return early because it passes the
> > test.
> >
> > I am no sure if above is right, but it's all I could understand without
> > documentation.
> >
> > Well, my point is: this seems hard to understand & review, so it would be
> > nice to have a macro like this to be used instead:
> >
> > #define sbi_fw_implements_kvm() \
> > (sbi_get_firmware_id() >= SBI_EXT_BASE_IMPL_ID_KVM)
> No, it's not correct. It must be:
> (sbi_get_firmware_id() == SBI_EXT_BASE_IMPL_ID_KVM)
Looking at the doc you provided, I think to be able to understand it.
The idea is to provide a code for given implementation of SBI, so in those
tests you check if the SBI implementation being used is KVM, meaning it's a
KVM guest. Ok, that makes sense now. Thanks!
>
> >
> > if(!sbi_fw_implements_kvm())
> I'm okay with sbi_fw_implements_kvm().
Thanks! also, thanks again for sharing the doc!
With above suggestions, please feel free to include in next versions:
Reviewed-by: Leonardo Bras <leobras@redhat.com>
Thx
Leo
>
> > return;
> >
> > What do you think?
> >
> > Other than that, LGTM.
> >
> > Thanks!
> > Leo
> >
> > > + return;
> > > +
> > > + pr_info("PV qspinlocks enabled\n");
> > > + __pv_init_lock_hash();
> > > +
> > > + static_call_update(pv_queued_spin_lock_slowpath, __pv_queued_spin_lock_slowpath);
> > > + static_call_update(pv_queued_spin_unlock, __pv_queued_spin_unlock);
> > > +}
> > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > index c57d15b05160..88690751f2ee 100644
> > > --- a/arch/riscv/kernel/setup.c
> > > +++ b/arch/riscv/kernel/setup.c
> > > @@ -321,6 +321,10 @@ static void __init riscv_spinlock_init(void)
> > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > virt_spin_lock_init();
> > > #endif
> > > +
> > > +#ifdef CONFIG_PARAVIRT_SPINLOCKS
> > > + pv_qspinlock_init();
> > > +#endif
> > > }
> > >
> > > extern void __init init_rt_signal_env(void);
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 12/17] RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
2023-09-17 15:03 ` Guo Ren
@ 2023-09-19 5:44 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:44 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 11:03:30PM +0800, Guo Ren wrote:
> On Fri, Sep 15, 2023 at 2:05 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:29:06AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Disables the qspinlock slow path using PV optimizations which
> > > allow the hypervisor to 'idle' the guest on lock contention.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > Documentation/admin-guide/kernel-parameters.txt | 2 +-
> > > arch/riscv/kernel/qspinlock_paravirt.c | 13 +++++++++++++
> > > 2 files changed, 14 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > index f75bedc50e00..e74aed631573 100644
> > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > @@ -3857,7 +3857,7 @@
> > > as generic guest with no PV drivers. Currently support
> > > XEN HVM, KVM, HYPER_V and VMWARE guest.
> > >
> > > - nopvspin [X86,XEN,KVM]
> > > + nopvspin [X86,XEN,KVM,RISC-V]
> > > Disables the qspinlock slow path using PV optimizations
> > > which allow the hypervisor to 'idle' the guest on lock
> > > contention.
> > > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > > index 85ff5a3ec234..a0ad4657f437 100644
> > > --- a/arch/riscv/kernel/qspinlock_paravirt.c
> > > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > > @@ -41,8 +41,21 @@ EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> > > DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> > > EXPORT_STATIC_CALL(pv_queued_spin_unlock);
> > >
> > > +static bool nopvspin;
> >
> > It is only used in init, so it makes sense to add __initdata.
> >
> > static bool nopvspin __initdata;
> Okay.
Thx!
Leo
>
> >
> > Other than that, LGTM:
> > Reviewed-by: Leonardo Bras <leobras@redhat.com>
> >
> > Thanks!
> > Leo
> >
> > > +static __init int parse_nopvspin(char *arg)
> > > +{
> > > + nopvspin = true;
> > > + return 0;
> > > +}
> > > +early_param("nopvspin", parse_nopvspin);
> > > +
> > > void __init pv_qspinlock_init(void)
> > > {
> > > + if (nopvspin) {
> > > + pr_info("PV qspinlocks disabled\n");
> > > + return;
> > > + }
> > > +
> > > if (num_possible_cpus() == 1)
> > > return;
> > >
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 12/17] RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
@ 2023-09-19 5:44 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:44 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 11:03:30PM +0800, Guo Ren wrote:
> On Fri, Sep 15, 2023 at 2:05 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:29:06AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Disables the qspinlock slow path using PV optimizations which
> > > allow the hypervisor to 'idle' the guest on lock contention.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > Documentation/admin-guide/kernel-parameters.txt | 2 +-
> > > arch/riscv/kernel/qspinlock_paravirt.c | 13 +++++++++++++
> > > 2 files changed, 14 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > index f75bedc50e00..e74aed631573 100644
> > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > @@ -3857,7 +3857,7 @@
> > > as generic guest with no PV drivers. Currently support
> > > XEN HVM, KVM, HYPER_V and VMWARE guest.
> > >
> > > - nopvspin [X86,XEN,KVM]
> > > + nopvspin [X86,XEN,KVM,RISC-V]
> > > Disables the qspinlock slow path using PV optimizations
> > > which allow the hypervisor to 'idle' the guest on lock
> > > contention.
> > > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > > index 85ff5a3ec234..a0ad4657f437 100644
> > > --- a/arch/riscv/kernel/qspinlock_paravirt.c
> > > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > > @@ -41,8 +41,21 @@ EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath);
> > > DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock);
> > > EXPORT_STATIC_CALL(pv_queued_spin_unlock);
> > >
> > > +static bool nopvspin;
> >
> > It is only used in init, so it makes sense to add __initdata.
> >
> > static bool nopvspin __initdata;
> Okay.
Thx!
Leo
>
> >
> > Other than that, LGTM:
> > Reviewed-by: Leonardo Bras <leobras@redhat.com>
> >
> > Thanks!
> > Leo
> >
> > > +static __init int parse_nopvspin(char *arg)
> > > +{
> > > + nopvspin = true;
> > > + return 0;
> > > +}
> > > +early_param("nopvspin", parse_nopvspin);
> > > +
> > > void __init pv_qspinlock_init(void)
> > > {
> > > + if (nopvspin) {
> > > + pr_info("PV qspinlocks disabled\n");
> > > + return;
> > > + }
> > > +
> > > if (num_possible_cpus() == 1)
> > > return;
> > >
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 13/17] RISC-V: paravirt: pvqspinlock: Add SBI implementation
2023-09-17 15:06 ` Guo Ren
@ 2023-09-19 5:45 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:45 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 11:06:48PM +0800, Guo Ren wrote:
> On Fri, Sep 15, 2023 at 2:23 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:29:07AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Implement pv_kick with SBI implementation, and add SBI_EXT_PVLOCK
> > > extension detection.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > arch/riscv/include/asm/sbi.h | 6 ++++++
> > > arch/riscv/kernel/qspinlock_paravirt.c | 7 ++++++-
> > > 2 files changed, 12 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > > index e0233b3d7a5f..3533f8d4f3e2 100644
> > > --- a/arch/riscv/include/asm/sbi.h
> > > +++ b/arch/riscv/include/asm/sbi.h
> > > @@ -30,6 +30,7 @@ enum sbi_ext_id {
> > > SBI_EXT_HSM = 0x48534D,
> > > SBI_EXT_SRST = 0x53525354,
> > > SBI_EXT_PMU = 0x504D55,
> > > + SBI_EXT_PVLOCK = 0xAB0401,
> > >
> > > /* Experimentals extensions must lie within this range */
> > > SBI_EXT_EXPERIMENTAL_START = 0x08000000,
> > > @@ -243,6 +244,11 @@ enum sbi_pmu_ctr_type {
> > > /* Flags defined for counter stop function */
> > > #define SBI_PMU_STOP_FLAG_RESET (1 << 0)
> > >
> > > +/* SBI PVLOCK (kick cpu out of wfi) */
> > > +enum sbi_ext_pvlock_fid {
> > > + SBI_EXT_PVLOCK_KICK_CPU = 0,
> > > +};
> > > +
> > > #define SBI_SPEC_VERSION_DEFAULT 0x1
> > > #define SBI_SPEC_VERSION_MAJOR_SHIFT 24
> > > #define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
> > > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > > index a0ad4657f437..571626f350be 100644
> > > --- a/arch/riscv/kernel/qspinlock_paravirt.c
> > > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > > @@ -11,6 +11,8 @@
> > >
> > > void pv_kick(int cpu)
> > > {
> > > + sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
> > > + cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
> > > return;
> > > }
> > >
> > > @@ -25,7 +27,7 @@ void pv_wait(u8 *ptr, u8 val)
> > > if (READ_ONCE(*ptr) != val)
> > > goto out;
> > >
> > > - /* wait_for_interrupt(); */
> > > + wait_for_interrupt();
> > > out:
> > > local_irq_restore(flags);
> > > }
> > > @@ -62,6 +64,9 @@ void __init pv_qspinlock_init(void)
> > > if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
> > > return;
> > >
> > > + if (!sbi_probe_extension(SBI_EXT_PVLOCK))
> > > + return;
> > > +
> > > pr_info("PV qspinlocks enabled\n");
> > > __pv_init_lock_hash();
> > >
> > > --
> > > 2.36.1
> > >
> >
> > IIUC this PVLOCK extension is now a requirement to use pv_qspinlock(), and
> > it allows a cpu to use an instruction to wait for interrupt in pv_wait(),
> > and kicks it out of this wait using a new sbi_ecall() on pv_kick().
> Yes.
>
> >
> > Overall it LGTM, but would be nice to have the reference doc in the commit
> > msg. I end up inferring some of the inner workings by your implementation,
> > which is not ideal for reviewing.
> I would improve the commit msg in the next version of patch.
Thx!
Leo
>
> >
> > If understanding above is right,
> > Reviewed-by: Leonardo Bras <leobras@redhat.com>
> >
> > Thanks!
> > Leo
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 13/17] RISC-V: paravirt: pvqspinlock: Add SBI implementation
@ 2023-09-19 5:45 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 5:45 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 17, 2023 at 11:06:48PM +0800, Guo Ren wrote:
> On Fri, Sep 15, 2023 at 2:23 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:29:07AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > Implement pv_kick with SBI implementation, and add SBI_EXT_PVLOCK
> > > extension detection.
> > >
> > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > ---
> > > arch/riscv/include/asm/sbi.h | 6 ++++++
> > > arch/riscv/kernel/qspinlock_paravirt.c | 7 ++++++-
> > > 2 files changed, 12 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > > index e0233b3d7a5f..3533f8d4f3e2 100644
> > > --- a/arch/riscv/include/asm/sbi.h
> > > +++ b/arch/riscv/include/asm/sbi.h
> > > @@ -30,6 +30,7 @@ enum sbi_ext_id {
> > > SBI_EXT_HSM = 0x48534D,
> > > SBI_EXT_SRST = 0x53525354,
> > > SBI_EXT_PMU = 0x504D55,
> > > + SBI_EXT_PVLOCK = 0xAB0401,
> > >
> > > /* Experimentals extensions must lie within this range */
> > > SBI_EXT_EXPERIMENTAL_START = 0x08000000,
> > > @@ -243,6 +244,11 @@ enum sbi_pmu_ctr_type {
> > > /* Flags defined for counter stop function */
> > > #define SBI_PMU_STOP_FLAG_RESET (1 << 0)
> > >
> > > +/* SBI PVLOCK (kick cpu out of wfi) */
> > > +enum sbi_ext_pvlock_fid {
> > > + SBI_EXT_PVLOCK_KICK_CPU = 0,
> > > +};
> > > +
> > > #define SBI_SPEC_VERSION_DEFAULT 0x1
> > > #define SBI_SPEC_VERSION_MAJOR_SHIFT 24
> > > #define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
> > > diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qspinlock_paravirt.c
> > > index a0ad4657f437..571626f350be 100644
> > > --- a/arch/riscv/kernel/qspinlock_paravirt.c
> > > +++ b/arch/riscv/kernel/qspinlock_paravirt.c
> > > @@ -11,6 +11,8 @@
> > >
> > > void pv_kick(int cpu)
> > > {
> > > + sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU,
> > > + cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0);
> > > return;
> > > }
> > >
> > > @@ -25,7 +27,7 @@ void pv_wait(u8 *ptr, u8 val)
> > > if (READ_ONCE(*ptr) != val)
> > > goto out;
> > >
> > > - /* wait_for_interrupt(); */
> > > + wait_for_interrupt();
> > > out:
> > > local_irq_restore(flags);
> > > }
> > > @@ -62,6 +64,9 @@ void __init pv_qspinlock_init(void)
> > > if(sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)
> > > return;
> > >
> > > + if (!sbi_probe_extension(SBI_EXT_PVLOCK))
> > > + return;
> > > +
> > > pr_info("PV qspinlocks enabled\n");
> > > __pv_init_lock_hash();
> > >
> > > --
> > > 2.36.1
> > >
> >
> > IIUC this PVLOCK extension is now a requirement to use pv_qspinlock(), and
> > it allows a cpu to use an instruction to wait for interrupt in pv_wait(),
> > and kicks it out of this wait using a new sbi_ecall() on pv_kick().
> Yes.
>
> >
> > Overall it LGTM, but would be nice to have the reference doc in the commit
> > msg. I end up inferring some of the inner workings by your implementation,
> > which is not ideal for reviewing.
> I would improve the commit msg in the next version of patch.
Thx!
Leo
>
> >
> > If understanding above is right,
> > Reviewed-by: Leonardo Bras <leobras@redhat.com>
> >
> > Thanks!
> > Leo
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-19 5:13 ` Leonardo Bras
@ 2023-09-19 7:53 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-19 7:53 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 19, 2023 at 1:13 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 17, 2023 at 10:34:36PM +0800, Guo Ren wrote:
> > On Sat, Sep 16, 2023 at 9:25 AM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > On Fri, Sep 15, 2023 at 08:36:31PM +0800, Guo Ren wrote:
> > > > On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
> > > > >
> > > > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > > >
> > > > > > Cache-block prefetch instructions are HINTs to the hardware to
> > > > > > indicate that software intends to perform a particular type of
> > > > > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > > > > improve the arch_xchg for qspinlock xchg_tail.
> > > > > >
> > > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > > ---
> > > > > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > > > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > > > > arch/riscv/include/asm/hwcap.h | 1 +
> > > > > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > > > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > > > > arch/riscv/kernel/cpufeature.c | 1 +
> > > > > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > > index e9ae6fa232c3..2c346fe169c1 100644
> > > > > > --- a/arch/riscv/Kconfig
> > > > > > +++ b/arch/riscv/Kconfig
> > > > > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > > > > >
> > > > > > If you don't know what to do here, say Y.
> > > > > >
> > > > > > +config RISCV_ISA_ZICBOP
> > > > > > + bool "Zicbop extension support for cache block prefetch"
> > > > > > + depends on MMU
> > > > > > + depends on RISCV_ALTERNATIVE
> > > > > > + default y
> > > > > > + help
> > > > > > + Adds support to dynamically detect the presence of the ZICBOP
> > > > > > + extension (Cache Block Prefetch Operations) and enable its
> > > > > > + usage.
> > > > > > +
> > > > > > + The Zicbop extension can be used to prefetch cache block for
> > > > > > + read/write/instruction fetch.
> > > > > > +
> > > > > > + If you don't know what to do here, say Y.
> > > > > > +
> > > > > > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > > > > > bool
> > > > > > default y
> > > > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > > > index 702725727671..56eff7a9d2d2 100644
> > > > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > > > @@ -11,6 +11,7 @@
> > > > > >
> > > > > > #include <asm/barrier.h>
> > > > > > #include <asm/fence.h>
> > > > > > +#include <asm/processor.h>
> > > > > >
> > > > > > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > > > > > ({ \
> > > > > > @@ -25,6 +26,7 @@
> > > > > > \
> > > > > > __asm__ __volatile__ ( \
> > > > > > prepend \
> > > > > > + PREFETCHW_ASM(%5) \
> > > > > > "0: lr.w %0, %2\n" \
> > > > > > " and %1, %0, %z4\n" \
> > > > > > " or %1, %1, %z3\n" \
> > > > > > @@ -32,7 +34,7 @@
> > > > > > " bnez %1, 0b\n" \
> > > > > > append \
> > > > > > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > > > > > - : "rJ" (__newx), "rJ" (~__mask) \
> > > > > > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > > > > > : "memory"); \
> > > > > > \
> > > > > > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > > > > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > > > > > index b7b58258f6c7..78b7b8b53778 100644
> > > > > > --- a/arch/riscv/include/asm/hwcap.h
> > > > > > +++ b/arch/riscv/include/asm/hwcap.h
> > > > > > @@ -58,6 +58,7 @@
> > > > > > #define RISCV_ISA_EXT_ZICSR 40
> > > > > > #define RISCV_ISA_EXT_ZIFENCEI 41
> > > > > > #define RISCV_ISA_EXT_ZIHPM 42
> > > > > > +#define RISCV_ISA_EXT_ZICBOP 43
> > > > > >
> > > > > > #define RISCV_ISA_EXT_MAX 64
> > > > > >
> > > > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > > > index 6960beb75f32..dc590d331894 100644
> > > > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > > > @@ -134,6 +134,7 @@
> > > > > >
> > > > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > > > > >
> > > > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > > > @@ -196,4 +197,8 @@
> > > > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > > > RS1(base), SIMM12(4))
> > > > > >
> > > > > > +#define CBO_prefetchw(base) \
> > > > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > > > + RD(x0), RS1(base), RS2(x0))
> > > > > > +
> > > > >
> > > > > I understand that here you create the instruction via bitfield, following
> > > > > the ISA, and this enables using instructions not available on the
> > > > > toolchain.
> > > > >
> > > > > It took me some time to find the document with this instruction, so please
> > > > > add this to the commit msg:
> > > > >
> > > > > https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> > > > > Page 23.
> > > > >
> > > > > IIUC, the instruction is "prefetch.w".
> > > > >
> > > > > Maybe I am missing something, but in the document the rs2 field
> > > > > (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
> > > > >
> > > > > rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
> > > > >
> > > > > Is the above correct, or am I missing something?
> > > > Oh, you are right. My fault, thx for pointing out. It should be:
> > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > + RD(x0), RS1(base), RS2(x3))
> > >
> > > Now I am curious to check if / how will this impact performance. :)
> > > (Please let me know)
> > Ref:
> > commit 0ea366f5e1b6 ("arm64: atomics: prefetch the destination word
> > for write prior to stxr")
> > commit 86d231459d6d ("bpf: cpumap memory prefetchw optimizations for
> > struct page")
>
> Oh, I understand that prefetch.w is very useful for performance :)
>
> What I meant is that previously this patch was issuing a prefetch.i,
> and now it's issuing a prefetch.w (as intended).
>
> What got me curious is how much would it impact the performance to change
> the prefetch.i to prefetch.w. :)
The current SOPHO sg2042 hardware platform didn't support prefetch.w
instruction. So there is no performance result I could share with you.
Our next generation of processors would support ZICBOP.
>
> Thanks!
> Leo
>
>
> >
> > >
> > >
> > > >
> > > > >
> > > > >
> > > > > Thanks!
> > > > > Leo
> > > > >
> > > > > > #endif /* __ASM_INSN_DEF_H */
> > > > > > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > > > > > index de9da852f78d..7ad3a24212e8 100644
> > > > > > --- a/arch/riscv/include/asm/processor.h
> > > > > > +++ b/arch/riscv/include/asm/processor.h
> > > > > > @@ -12,6 +12,8 @@
> > > > > > #include <vdso/processor.h>
> > > > > >
> > > > > > #include <asm/ptrace.h>
> > > > > > +#include <asm/insn-def.h>
> > > > > > +#include <asm/hwcap.h>
> > > > > >
> > > > > > #ifdef CONFIG_64BIT
> > > > > > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > > > > > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > > > > > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > > > > > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> > > > > >
> > > > > > +#define ARCH_HAS_PREFETCHW
> > > > > > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > > > > > + CBO_prefetchw(base), \
> > > > > > + 0, \
> > > > > > + RISCV_ISA_EXT_ZICBOP, \
> > > > > > + CONFIG_RISCV_ISA_ZICBOP)
> > > > > > +static inline void prefetchw(const void *ptr)
> > > > > > +{
> > > > > > + asm volatile(PREFETCHW_ASM(%0)
> > > > > > + : : "r" (ptr) : "memory");
> > > > > > +}
> > > > > >
> > > > > > /* Do necessary setup to start up a newly executed thread. */
> > > > > > extern void start_thread(struct pt_regs *regs,
> > > > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > > > > index ef7b4fd9e876..e0b897db0b97 100644
> > > > > > --- a/arch/riscv/kernel/cpufeature.c
> > > > > > +++ b/arch/riscv/kernel/cpufeature.c
> > > > > > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > > > > > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > > > > > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > > > > > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > > > > > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > > > > > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > > > > > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > > > > > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > > > > > --
> > > > > > 2.36.1
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best Regards
> > > > Guo Ren
> > > >
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-19 7:53 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-19 7:53 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 19, 2023 at 1:13 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 17, 2023 at 10:34:36PM +0800, Guo Ren wrote:
> > On Sat, Sep 16, 2023 at 9:25 AM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > On Fri, Sep 15, 2023 at 08:36:31PM +0800, Guo Ren wrote:
> > > > On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
> > > > >
> > > > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > > >
> > > > > > Cache-block prefetch instructions are HINTs to the hardware to
> > > > > > indicate that software intends to perform a particular type of
> > > > > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > > > > improve the arch_xchg for qspinlock xchg_tail.
> > > > > >
> > > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > > ---
> > > > > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > > > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > > > > arch/riscv/include/asm/hwcap.h | 1 +
> > > > > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > > > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > > > > arch/riscv/kernel/cpufeature.c | 1 +
> > > > > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > > index e9ae6fa232c3..2c346fe169c1 100644
> > > > > > --- a/arch/riscv/Kconfig
> > > > > > +++ b/arch/riscv/Kconfig
> > > > > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > > > > >
> > > > > > If you don't know what to do here, say Y.
> > > > > >
> > > > > > +config RISCV_ISA_ZICBOP
> > > > > > + bool "Zicbop extension support for cache block prefetch"
> > > > > > + depends on MMU
> > > > > > + depends on RISCV_ALTERNATIVE
> > > > > > + default y
> > > > > > + help
> > > > > > + Adds support to dynamically detect the presence of the ZICBOP
> > > > > > + extension (Cache Block Prefetch Operations) and enable its
> > > > > > + usage.
> > > > > > +
> > > > > > + The Zicbop extension can be used to prefetch cache block for
> > > > > > + read/write/instruction fetch.
> > > > > > +
> > > > > > + If you don't know what to do here, say Y.
> > > > > > +
> > > > > > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > > > > > bool
> > > > > > default y
> > > > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > > > index 702725727671..56eff7a9d2d2 100644
> > > > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > > > @@ -11,6 +11,7 @@
> > > > > >
> > > > > > #include <asm/barrier.h>
> > > > > > #include <asm/fence.h>
> > > > > > +#include <asm/processor.h>
> > > > > >
> > > > > > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > > > > > ({ \
> > > > > > @@ -25,6 +26,7 @@
> > > > > > \
> > > > > > __asm__ __volatile__ ( \
> > > > > > prepend \
> > > > > > + PREFETCHW_ASM(%5) \
> > > > > > "0: lr.w %0, %2\n" \
> > > > > > " and %1, %0, %z4\n" \
> > > > > > " or %1, %1, %z3\n" \
> > > > > > @@ -32,7 +34,7 @@
> > > > > > " bnez %1, 0b\n" \
> > > > > > append \
> > > > > > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > > > > > - : "rJ" (__newx), "rJ" (~__mask) \
> > > > > > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > > > > > : "memory"); \
> > > > > > \
> > > > > > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > > > > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > > > > > index b7b58258f6c7..78b7b8b53778 100644
> > > > > > --- a/arch/riscv/include/asm/hwcap.h
> > > > > > +++ b/arch/riscv/include/asm/hwcap.h
> > > > > > @@ -58,6 +58,7 @@
> > > > > > #define RISCV_ISA_EXT_ZICSR 40
> > > > > > #define RISCV_ISA_EXT_ZIFENCEI 41
> > > > > > #define RISCV_ISA_EXT_ZIHPM 42
> > > > > > +#define RISCV_ISA_EXT_ZICBOP 43
> > > > > >
> > > > > > #define RISCV_ISA_EXT_MAX 64
> > > > > >
> > > > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > > > index 6960beb75f32..dc590d331894 100644
> > > > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > > > @@ -134,6 +134,7 @@
> > > > > >
> > > > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > > > > >
> > > > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > > > @@ -196,4 +197,8 @@
> > > > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > > > RS1(base), SIMM12(4))
> > > > > >
> > > > > > +#define CBO_prefetchw(base) \
> > > > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > > > + RD(x0), RS1(base), RS2(x0))
> > > > > > +
> > > > >
> > > > > I understand that here you create the instruction via bitfield, following
> > > > > the ISA, and this enables using instructions not available on the
> > > > > toolchain.
> > > > >
> > > > > It took me some time to find the document with this instruction, so please
> > > > > add this to the commit msg:
> > > > >
> > > > > https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> > > > > Page 23.
> > > > >
> > > > > IIUC, the instruction is "prefetch.w".
> > > > >
> > > > > Maybe I am missing something, but in the document the rs2 field
> > > > > (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
> > > > >
> > > > > rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
> > > > >
> > > > > Is the above correct, or am I missing something?
> > > > Oh, you are right. My fault, thx for pointing out. It should be:
> > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > + RD(x0), RS1(base), RS2(x3))
> > >
> > > Now I am curious to check if / how will this impact performance. :)
> > > (Please let me know)
> > Ref:
> > commit 0ea366f5e1b6 ("arm64: atomics: prefetch the destination word
> > for write prior to stxr")
> > commit 86d231459d6d ("bpf: cpumap memory prefetchw optimizations for
> > struct page")
>
> Oh, I understand that prefetch.w is very useful for performance :)
>
> What I meant is that previously this patch was issuing a prefetch.i,
> and now it's issuing a prefetch.w (as intended).
>
> What got me curious is how much would it impact the performance to change
> the prefetch.i to prefetch.w. :)
The current SOPHO sg2042 hardware platform didn't support prefetch.w
instruction. So there is no performance result I could share with you.
Our next generation of processors would support ZICBOP.
>
> Thanks!
> Leo
>
>
> >
> > >
> > >
> > > >
> > > > >
> > > > >
> > > > > Thanks!
> > > > > Leo
> > > > >
> > > > > > #endif /* __ASM_INSN_DEF_H */
> > > > > > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > > > > > index de9da852f78d..7ad3a24212e8 100644
> > > > > > --- a/arch/riscv/include/asm/processor.h
> > > > > > +++ b/arch/riscv/include/asm/processor.h
> > > > > > @@ -12,6 +12,8 @@
> > > > > > #include <vdso/processor.h>
> > > > > >
> > > > > > #include <asm/ptrace.h>
> > > > > > +#include <asm/insn-def.h>
> > > > > > +#include <asm/hwcap.h>
> > > > > >
> > > > > > #ifdef CONFIG_64BIT
> > > > > > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > > > > > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > > > > > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > > > > > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> > > > > >
> > > > > > +#define ARCH_HAS_PREFETCHW
> > > > > > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > > > > > + CBO_prefetchw(base), \
> > > > > > + 0, \
> > > > > > + RISCV_ISA_EXT_ZICBOP, \
> > > > > > + CONFIG_RISCV_ISA_ZICBOP)
> > > > > > +static inline void prefetchw(const void *ptr)
> > > > > > +{
> > > > > > + asm volatile(PREFETCHW_ASM(%0)
> > > > > > + : : "r" (ptr) : "memory");
> > > > > > +}
> > > > > >
> > > > > > /* Do necessary setup to start up a newly executed thread. */
> > > > > > extern void start_thread(struct pt_regs *regs,
> > > > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > > > > index ef7b4fd9e876..e0b897db0b97 100644
> > > > > > --- a/arch/riscv/kernel/cpufeature.c
> > > > > > +++ b/arch/riscv/kernel/cpufeature.c
> > > > > > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > > > > > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > > > > > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > > > > > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > > > > > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > > > > > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > > > > > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > > > > > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > > > > > --
> > > > > > 2.36.1
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best Regards
> > > > Guo Ren
> > > >
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
2023-09-19 5:30 ` Leonardo Bras
@ 2023-09-19 8:04 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-19 8:04 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 19, 2023 at 1:30 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 17, 2023 at 11:12:31PM +0800, Guo Ren wrote:
> > On Thu, Sep 14, 2023 at 4:02 PM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 04:29:02AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > Add a static key controlling whether virt_spin_lock() should be
> > > > called or not. When running on bare metal set the new key to
> > > > false.
> > > >
> > > > The KVM guests fall back to a Test-and-Set spinlock, because fair
> > > > locks have horrible lock 'holder' preemption issues. The
> > > > virt_spin_lock_key would shortcut for the
> > > > queued_spin_lock_slowpath() function that allow virt_spin_lock to
> > > > hijack it.
> > > >
> > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > ---
> > > > .../admin-guide/kernel-parameters.txt | 4 +++
> > > > arch/riscv/include/asm/sbi.h | 8 +++++
> > > > arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
> > > > arch/riscv/kernel/sbi.c | 2 +-
> > > > arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
> > > > 5 files changed, 64 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > > index 61cacb8dfd0e..f75bedc50e00 100644
> > > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > > @@ -3927,6 +3927,10 @@
> > > > no_uaccess_flush
> > > > [PPC] Don't flush the L1-D cache after accessing user data.
> > > >
> > > > + no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
> > > > + native_queued_spinlock when the nopvspin option is enabled.
> > > > + This would help vcpu=pcpu scenarios.
> > > > +
> > > > novmcoredd [KNL,KDUMP]
> > > > Disable device dump. Device dump allows drivers to
> > > > append dump data to vmcore so you can collect driver
> > > > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > > > index 501e06e52078..e0233b3d7a5f 100644
> > > > --- a/arch/riscv/include/asm/sbi.h
> > > > +++ b/arch/riscv/include/asm/sbi.h
> > > > @@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
> > > > SBI_EXT_BASE_GET_MIMPID,
> > > > };
> > > >
> > > > +enum sbi_ext_base_impl_id {
> > > > + SBI_EXT_BASE_IMPL_ID_BBL = 0,
> > > > + SBI_EXT_BASE_IMPL_ID_OPENSBI,
> > > > + SBI_EXT_BASE_IMPL_ID_XVISOR,
> > > > + SBI_EXT_BASE_IMPL_ID_KVM,
> > > > +};
> > > > +
> > > > enum sbi_ext_time_fid {
> > > > SBI_EXT_TIME_SET_TIMER = 0,
> > > > };
> > > > @@ -269,6 +276,7 @@ int sbi_console_getchar(void);
> > > > long sbi_get_mvendorid(void);
> > > > long sbi_get_marchid(void);
> > > > long sbi_get_mimpid(void);
> > > > +long sbi_get_firmware_id(void);
> > > > void sbi_set_timer(uint64_t stime_value);
> > > > void sbi_shutdown(void);
> > > > void sbi_send_ipi(unsigned int cpu);
> > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > index 8ea0fee80652..6b38d6616f14 100644
> > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > @@ -4,6 +4,28 @@
> > > > #define __ASM_RISCV_SPINLOCK_H
> > > >
> > > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > > +/*
> > > > + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
> > > > + * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
> > > > + * would shortcut for the queued_spin_lock_slowpath() function that allow
> > > > + * virt_spin_lock to hijack it.
> > > > + */
> > > > +DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > > +
> > > > +#define virt_spin_lock virt_spin_lock
> > > > +static inline bool virt_spin_lock(struct qspinlock *lock)
> > > > +{
> > > > + if (!static_branch_likely(&virt_spin_lock_key))
> > > > + return false;
> > > > +
> > > > + do {
> > > > + while (atomic_read(&lock->val) != 0)
> > > > + cpu_relax();
> > > > + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
> > > > +
> > > > + return true;
> > > > +}
> > > > +
> > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > #endif
> > > >
> > > > diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> > > > index 88eea3a99ee0..cdd45edc8db4 100644
> > > > --- a/arch/riscv/kernel/sbi.c
> > > > +++ b/arch/riscv/kernel/sbi.c
> > > > @@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
> > > > return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
> > > > }
> > > >
> > > > -static inline long sbi_get_firmware_id(void)
> > > > +long sbi_get_firmware_id(void)
> > > > {
> > > > return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
> > > > }
> > > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > > index 0f084f037651..c57d15b05160 100644
> > > > --- a/arch/riscv/kernel/setup.c
> > > > +++ b/arch/riscv/kernel/setup.c
> > > > @@ -26,6 +26,7 @@
> > > > #include <asm/alternative.h>
> > > > #include <asm/cacheflush.h>
> > > > #include <asm/cpu_ops.h>
> > > > +#include <asm/cpufeature.h>
> > > > #include <asm/early_ioremap.h>
> > > > #include <asm/pgtable.h>
> > > > #include <asm/setup.h>
> > > > @@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > EXPORT_SYMBOL(combo_qspinlock_key);
> > > > #endif
> > > >
> > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > +static bool no_virt_spin_key = false;
> > >
> > > I suggest no _key, also there is no need for "= false".
> > > To be consistent with enable_qspinlock, I also suggest
> > > adding __ro_after_init:
> > >
> > > static bool no_virt_spin __ro_after_init;
> > okay.
> >
> > >
> > >
> > >
> > > > +DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > > +
> > > > +static int __init no_virt_spin_setup(char *p)
> > > > +{
> > > > + no_virt_spin_key = true;
> > > > +
> > > > + return 0;
> > > > +}
> > > > +early_param("no_virt_spin", no_virt_spin_setup);
> > > > +
> > > > +static void __init virt_spin_lock_init(void)
> > > > +{
> > > > + if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
> > > > + no_virt_spin_key)
> > > > + static_branch_disable(&virt_spin_lock_key);
> > > > + else
> > > > + pr_info("Enable virt_spin_lock\n");
> > > > +}
> > > > +#endif
> > > > +
> > >
> > > A new virt_no_spin kernel parameter was introduced, but without
> > > CONFIG_QUEUED_SPINLOCKS it will silently fail.
> > >
> > > I would suggest an #else clause here with a function to print an error /
> > > warning message about no_virt_spin being invalid in this scenario.
> > > It will probably help future debugging.
> > If CONFIG_QUEUED_SPINLOCKS=n, no_virt_spin should be quiet. The
> > no_virt_spin is one path of qspinlock.
>
> IIUC having no_virt_spin being passed as parameter to a kernel with
> CONFIG_QUEUED_SPINLOCKS=n is not supposed to have any warning this
> parameter is useless.
>
> I was just thinking it would be nice to have this warning during debugging,
> but if it's standard practice then I am ok with this.
Yes, I think it's okay, e.g.,
x86: early_param("hv_nopvspin", hv_parse_nopvspin);
depends on CONFIG_PARAVIRT_SPINLOCKS=y
>
> >
> > >
> > >
> > > > static void __init riscv_spinlock_init(void)
> > > > {
> > > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > - if (!enable_qspinlock_key) {
> > > > + if (!enable_qspinlock_key &&
> > > > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > > > static_branch_disable(&combo_qspinlock_key);
> > > > pr_info("Ticket spinlock: enabled\n");
> > > > } else {
> > > > pr_info("Queued spinlock: enabled\n");
> > > > }
> > > > #endif
> > > > +
> > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > + virt_spin_lock_init();
> > > > +#endif
> > > > }
> > > >
> > > > extern void __init init_rt_signal_env(void);
> > > > --
> > > > 2.36.1
> > > >
> > >
> > > I am probably missing something out, but it looks to me that this patch is
> > > causing 2 different changes:
> > > 1 - Enabling no_virt_spin parameter
> > > 2 - Disabling queued spinlocks for some firmware_id
> > >
> > > Wouldn't be better to split those changes in multiple patches?
> > > Or am I missing the point on why they need to be together?
>
> ^ Want your input on this
Sorry, I missed that. Okay, I would split those changes.
>
> Thanks!
> Leo
>
> > >
> > > Thanks!
> > > Leo
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
@ 2023-09-19 8:04 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-09-19 8:04 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 19, 2023 at 1:30 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 17, 2023 at 11:12:31PM +0800, Guo Ren wrote:
> > On Thu, Sep 14, 2023 at 4:02 PM Leonardo Bras <leobras@redhat.com> wrote:
> > >
> > > On Sun, Sep 10, 2023 at 04:29:02AM -0400, guoren@kernel.org wrote:
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > Add a static key controlling whether virt_spin_lock() should be
> > > > called or not. When running on bare metal set the new key to
> > > > false.
> > > >
> > > > The KVM guests fall back to a Test-and-Set spinlock, because fair
> > > > locks have horrible lock 'holder' preemption issues. The
> > > > virt_spin_lock_key would shortcut for the
> > > > queued_spin_lock_slowpath() function that allow virt_spin_lock to
> > > > hijack it.
> > > >
> > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > ---
> > > > .../admin-guide/kernel-parameters.txt | 4 +++
> > > > arch/riscv/include/asm/sbi.h | 8 +++++
> > > > arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
> > > > arch/riscv/kernel/sbi.c | 2 +-
> > > > arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
> > > > 5 files changed, 64 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > > index 61cacb8dfd0e..f75bedc50e00 100644
> > > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > > @@ -3927,6 +3927,10 @@
> > > > no_uaccess_flush
> > > > [PPC] Don't flush the L1-D cache after accessing user data.
> > > >
> > > > + no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
> > > > + native_queued_spinlock when the nopvspin option is enabled.
> > > > + This would help vcpu=pcpu scenarios.
> > > > +
> > > > novmcoredd [KNL,KDUMP]
> > > > Disable device dump. Device dump allows drivers to
> > > > append dump data to vmcore so you can collect driver
> > > > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > > > index 501e06e52078..e0233b3d7a5f 100644
> > > > --- a/arch/riscv/include/asm/sbi.h
> > > > +++ b/arch/riscv/include/asm/sbi.h
> > > > @@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
> > > > SBI_EXT_BASE_GET_MIMPID,
> > > > };
> > > >
> > > > +enum sbi_ext_base_impl_id {
> > > > + SBI_EXT_BASE_IMPL_ID_BBL = 0,
> > > > + SBI_EXT_BASE_IMPL_ID_OPENSBI,
> > > > + SBI_EXT_BASE_IMPL_ID_XVISOR,
> > > > + SBI_EXT_BASE_IMPL_ID_KVM,
> > > > +};
> > > > +
> > > > enum sbi_ext_time_fid {
> > > > SBI_EXT_TIME_SET_TIMER = 0,
> > > > };
> > > > @@ -269,6 +276,7 @@ int sbi_console_getchar(void);
> > > > long sbi_get_mvendorid(void);
> > > > long sbi_get_marchid(void);
> > > > long sbi_get_mimpid(void);
> > > > +long sbi_get_firmware_id(void);
> > > > void sbi_set_timer(uint64_t stime_value);
> > > > void sbi_shutdown(void);
> > > > void sbi_send_ipi(unsigned int cpu);
> > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > index 8ea0fee80652..6b38d6616f14 100644
> > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > @@ -4,6 +4,28 @@
> > > > #define __ASM_RISCV_SPINLOCK_H
> > > >
> > > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > > +/*
> > > > + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
> > > > + * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
> > > > + * would shortcut for the queued_spin_lock_slowpath() function that allow
> > > > + * virt_spin_lock to hijack it.
> > > > + */
> > > > +DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > > +
> > > > +#define virt_spin_lock virt_spin_lock
> > > > +static inline bool virt_spin_lock(struct qspinlock *lock)
> > > > +{
> > > > + if (!static_branch_likely(&virt_spin_lock_key))
> > > > + return false;
> > > > +
> > > > + do {
> > > > + while (atomic_read(&lock->val) != 0)
> > > > + cpu_relax();
> > > > + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
> > > > +
> > > > + return true;
> > > > +}
> > > > +
> > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > #endif
> > > >
> > > > diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> > > > index 88eea3a99ee0..cdd45edc8db4 100644
> > > > --- a/arch/riscv/kernel/sbi.c
> > > > +++ b/arch/riscv/kernel/sbi.c
> > > > @@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
> > > > return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
> > > > }
> > > >
> > > > -static inline long sbi_get_firmware_id(void)
> > > > +long sbi_get_firmware_id(void)
> > > > {
> > > > return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
> > > > }
> > > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > > index 0f084f037651..c57d15b05160 100644
> > > > --- a/arch/riscv/kernel/setup.c
> > > > +++ b/arch/riscv/kernel/setup.c
> > > > @@ -26,6 +26,7 @@
> > > > #include <asm/alternative.h>
> > > > #include <asm/cacheflush.h>
> > > > #include <asm/cpu_ops.h>
> > > > +#include <asm/cpufeature.h>
> > > > #include <asm/early_ioremap.h>
> > > > #include <asm/pgtable.h>
> > > > #include <asm/setup.h>
> > > > @@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > EXPORT_SYMBOL(combo_qspinlock_key);
> > > > #endif
> > > >
> > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > +static bool no_virt_spin_key = false;
> > >
> > > I suggest no _key, also there is no need for "= false".
> > > To be consistent with enable_qspinlock, I also suggest
> > > adding __ro_after_init:
> > >
> > > static bool no_virt_spin __ro_after_init;
> > okay.
> >
> > >
> > >
> > >
> > > > +DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > > +
> > > > +static int __init no_virt_spin_setup(char *p)
> > > > +{
> > > > + no_virt_spin_key = true;
> > > > +
> > > > + return 0;
> > > > +}
> > > > +early_param("no_virt_spin", no_virt_spin_setup);
> > > > +
> > > > +static void __init virt_spin_lock_init(void)
> > > > +{
> > > > + if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
> > > > + no_virt_spin_key)
> > > > + static_branch_disable(&virt_spin_lock_key);
> > > > + else
> > > > + pr_info("Enable virt_spin_lock\n");
> > > > +}
> > > > +#endif
> > > > +
> > >
> > > A new virt_no_spin kernel parameter was introduced, but without
> > > CONFIG_QUEUED_SPINLOCKS it will silently fail.
> > >
> > > I would suggest an #else clause here with a function to print an error /
> > > warning message about no_virt_spin being invalid in this scenario.
> > > It will probably help future debugging.
> > If CONFIG_QUEUED_SPINLOCKS=n, no_virt_spin should be quiet. The
> > no_virt_spin is one path of qspinlock.
>
> IIUC having no_virt_spin being passed as parameter to a kernel with
> CONFIG_QUEUED_SPINLOCKS=n is not supposed to have any warning this
> parameter is useless.
>
> I was just thinking it would be nice to have this warning during debugging,
> but if it's standard practice then I am ok with this.
Yes, I think it's okay, e.g.,
x86: early_param("hv_nopvspin", hv_parse_nopvspin);
depends on CONFIG_PARAVIRT_SPINLOCKS=y
>
> >
> > >
> > >
> > > > static void __init riscv_spinlock_init(void)
> > > > {
> > > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > - if (!enable_qspinlock_key) {
> > > > + if (!enable_qspinlock_key &&
> > > > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > > > static_branch_disable(&combo_qspinlock_key);
> > > > pr_info("Ticket spinlock: enabled\n");
> > > > } else {
> > > > pr_info("Queued spinlock: enabled\n");
> > > > }
> > > > #endif
> > > > +
> > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > + virt_spin_lock_init();
> > > > +#endif
> > > > }
> > > >
> > > > extern void __init init_rt_signal_env(void);
> > > > --
> > > > 2.36.1
> > > >
> > >
> > > I am probably missing something out, but it looks to me that this patch is
> > > causing 2 different changes:
> > > 1 - Enabling no_virt_spin parameter
> > > 2 - Disabling queued spinlocks for some firmware_id
> > >
> > > Wouldn't be better to split those changes in multiple patches?
> > > Or am I missing the point on why they need to be together?
>
> ^ Want your input on this
Sorry, I missed that. Okay, I would split those changes.
>
> Thanks!
> Leo
>
> > >
> > > Thanks!
> > > Leo
> > >
> >
> >
> > --
> > Best Regards
> > Guo Ren
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
2023-09-19 7:53 ` Guo Ren
@ 2023-09-19 14:38 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 14:38 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 19, 2023 at 03:53:22PM +0800, Guo Ren wrote:
> On Tue, Sep 19, 2023 at 1:13 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 17, 2023 at 10:34:36PM +0800, Guo Ren wrote:
> > > On Sat, Sep 16, 2023 at 9:25 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > On Fri, Sep 15, 2023 at 08:36:31PM +0800, Guo Ren wrote:
> > > > > On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
> > > > > >
> > > > > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > > > >
> > > > > > > Cache-block prefetch instructions are HINTs to the hardware to
> > > > > > > indicate that software intends to perform a particular type of
> > > > > > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > > > > > improve the arch_xchg for qspinlock xchg_tail.
> > > > > > >
> > > > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > > > ---
> > > > > > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > > > > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > > > > > arch/riscv/include/asm/hwcap.h | 1 +
> > > > > > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > > > > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > > > > > arch/riscv/kernel/cpufeature.c | 1 +
> > > > > > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > > > > > >
> > > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > > > index e9ae6fa232c3..2c346fe169c1 100644
> > > > > > > --- a/arch/riscv/Kconfig
> > > > > > > +++ b/arch/riscv/Kconfig
> > > > > > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > > > > > >
> > > > > > > If you don't know what to do here, say Y.
> > > > > > >
> > > > > > > +config RISCV_ISA_ZICBOP
> > > > > > > + bool "Zicbop extension support for cache block prefetch"
> > > > > > > + depends on MMU
> > > > > > > + depends on RISCV_ALTERNATIVE
> > > > > > > + default y
> > > > > > > + help
> > > > > > > + Adds support to dynamically detect the presence of the ZICBOP
> > > > > > > + extension (Cache Block Prefetch Operations) and enable its
> > > > > > > + usage.
> > > > > > > +
> > > > > > > + The Zicbop extension can be used to prefetch cache block for
> > > > > > > + read/write/instruction fetch.
> > > > > > > +
> > > > > > > + If you don't know what to do here, say Y.
> > > > > > > +
> > > > > > > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > > > > > > bool
> > > > > > > default y
> > > > > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > > > > index 702725727671..56eff7a9d2d2 100644
> > > > > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > > > > @@ -11,6 +11,7 @@
> > > > > > >
> > > > > > > #include <asm/barrier.h>
> > > > > > > #include <asm/fence.h>
> > > > > > > +#include <asm/processor.h>
> > > > > > >
> > > > > > > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > > > > > > ({ \
> > > > > > > @@ -25,6 +26,7 @@
> > > > > > > \
> > > > > > > __asm__ __volatile__ ( \
> > > > > > > prepend \
> > > > > > > + PREFETCHW_ASM(%5) \
> > > > > > > "0: lr.w %0, %2\n" \
> > > > > > > " and %1, %0, %z4\n" \
> > > > > > > " or %1, %1, %z3\n" \
> > > > > > > @@ -32,7 +34,7 @@
> > > > > > > " bnez %1, 0b\n" \
> > > > > > > append \
> > > > > > > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > > > > > > - : "rJ" (__newx), "rJ" (~__mask) \
> > > > > > > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > > > > > > : "memory"); \
> > > > > > > \
> > > > > > > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > > > > > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > > > > > > index b7b58258f6c7..78b7b8b53778 100644
> > > > > > > --- a/arch/riscv/include/asm/hwcap.h
> > > > > > > +++ b/arch/riscv/include/asm/hwcap.h
> > > > > > > @@ -58,6 +58,7 @@
> > > > > > > #define RISCV_ISA_EXT_ZICSR 40
> > > > > > > #define RISCV_ISA_EXT_ZIFENCEI 41
> > > > > > > #define RISCV_ISA_EXT_ZIHPM 42
> > > > > > > +#define RISCV_ISA_EXT_ZICBOP 43
> > > > > > >
> > > > > > > #define RISCV_ISA_EXT_MAX 64
> > > > > > >
> > > > > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > > > > index 6960beb75f32..dc590d331894 100644
> > > > > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > > > > @@ -134,6 +134,7 @@
> > > > > > >
> > > > > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > > > > > >
> > > > > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > > > > @@ -196,4 +197,8 @@
> > > > > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > > > > RS1(base), SIMM12(4))
> > > > > > >
> > > > > > > +#define CBO_prefetchw(base) \
> > > > > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > > > > + RD(x0), RS1(base), RS2(x0))
> > > > > > > +
> > > > > >
> > > > > > I understand that here you create the instruction via bitfield, following
> > > > > > the ISA, and this enables using instructions not available on the
> > > > > > toolchain.
> > > > > >
> > > > > > It took me some time to find the document with this instruction, so please
> > > > > > add this to the commit msg:
> > > > > >
> > > > > > https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> > > > > > Page 23.
> > > > > >
> > > > > > IIUC, the instruction is "prefetch.w".
> > > > > >
> > > > > > Maybe I am missing something, but in the document the rs2 field
> > > > > > (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
> > > > > >
> > > > > > rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
> > > > > >
> > > > > > Is the above correct, or am I missing something?
> > > > > Oh, you are right. My fault, thx for pointing out. It should be:
> > > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > > + RD(x0), RS1(base), RS2(x3))
> > > >
> > > > Now I am curious to check if / how will this impact performance. :)
> > > > (Please let me know)
> > > Ref:
> > > commit 0ea366f5e1b6 ("arm64: atomics: prefetch the destination word
> > > for write prior to stxr")
> > > commit 86d231459d6d ("bpf: cpumap memory prefetchw optimizations for
> > > struct page")
> >
> > Oh, I understand that prefetch.w is very useful for performance :)
> >
> > What I meant is that previously this patch was issuing a prefetch.i,
> > and now it's issuing a prefetch.w (as intended).
> >
> > What got me curious is how much would it impact the performance to change
> > the prefetch.i to prefetch.w. :)
> The current SOPHO sg2042 hardware platform didn't support prefetch.w
> instruction. So there is no performance result I could share with you.
>
> Our next generation of processors would support ZICBOP.
Oh, okay then.
Thanks for sharing this info!
Leo
>
> >
> > Thanks!
> > Leo
> >
> >
> > >
> > > >
> > > >
> > > > >
> > > > > >
> > > > > >
> > > > > > Thanks!
> > > > > > Leo
> > > > > >
> > > > > > > #endif /* __ASM_INSN_DEF_H */
> > > > > > > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > > > > > > index de9da852f78d..7ad3a24212e8 100644
> > > > > > > --- a/arch/riscv/include/asm/processor.h
> > > > > > > +++ b/arch/riscv/include/asm/processor.h
> > > > > > > @@ -12,6 +12,8 @@
> > > > > > > #include <vdso/processor.h>
> > > > > > >
> > > > > > > #include <asm/ptrace.h>
> > > > > > > +#include <asm/insn-def.h>
> > > > > > > +#include <asm/hwcap.h>
> > > > > > >
> > > > > > > #ifdef CONFIG_64BIT
> > > > > > > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > > > > > > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > > > > > > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > > > > > > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> > > > > > >
> > > > > > > +#define ARCH_HAS_PREFETCHW
> > > > > > > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > > > > > > + CBO_prefetchw(base), \
> > > > > > > + 0, \
> > > > > > > + RISCV_ISA_EXT_ZICBOP, \
> > > > > > > + CONFIG_RISCV_ISA_ZICBOP)
> > > > > > > +static inline void prefetchw(const void *ptr)
> > > > > > > +{
> > > > > > > + asm volatile(PREFETCHW_ASM(%0)
> > > > > > > + : : "r" (ptr) : "memory");
> > > > > > > +}
> > > > > > >
> > > > > > > /* Do necessary setup to start up a newly executed thread. */
> > > > > > > extern void start_thread(struct pt_regs *regs,
> > > > > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > > > > > index ef7b4fd9e876..e0b897db0b97 100644
> > > > > > > --- a/arch/riscv/kernel/cpufeature.c
> > > > > > > +++ b/arch/riscv/kernel/cpufeature.c
> > > > > > > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > > > > > > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > > > > > > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > > > > > > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > > > > > > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > > > > > > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > > > > > > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > > > > > > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > > > > > > --
> > > > > > > 2.36.1
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best Regards
> > > > > Guo Ren
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-19 14:38 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 14:38 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 19, 2023 at 03:53:22PM +0800, Guo Ren wrote:
> On Tue, Sep 19, 2023 at 1:13 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 17, 2023 at 10:34:36PM +0800, Guo Ren wrote:
> > > On Sat, Sep 16, 2023 at 9:25 AM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > On Fri, Sep 15, 2023 at 08:36:31PM +0800, Guo Ren wrote:
> > > > > On Wed, Sep 13, 2023 at 4:50 PM Leonardo Bras <leobras@redhat.com> wrote:
> > > > > >
> > > > > > On Sun, Sep 10, 2023 at 04:28:57AM -0400, guoren@kernel.org wrote:
> > > > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > > > >
> > > > > > > Cache-block prefetch instructions are HINTs to the hardware to
> > > > > > > indicate that software intends to perform a particular type of
> > > > > > > memory access in the near future. Enable ARCH_HAS_PREFETCHW and
> > > > > > > improve the arch_xchg for qspinlock xchg_tail.
> > > > > > >
> > > > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > > > ---
> > > > > > > arch/riscv/Kconfig | 15 +++++++++++++++
> > > > > > > arch/riscv/include/asm/cmpxchg.h | 4 +++-
> > > > > > > arch/riscv/include/asm/hwcap.h | 1 +
> > > > > > > arch/riscv/include/asm/insn-def.h | 5 +++++
> > > > > > > arch/riscv/include/asm/processor.h | 13 +++++++++++++
> > > > > > > arch/riscv/kernel/cpufeature.c | 1 +
> > > > > > > 6 files changed, 38 insertions(+), 1 deletion(-)
> > > > > > >
> > > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > > > index e9ae6fa232c3..2c346fe169c1 100644
> > > > > > > --- a/arch/riscv/Kconfig
> > > > > > > +++ b/arch/riscv/Kconfig
> > > > > > > @@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
> > > > > > >
> > > > > > > If you don't know what to do here, say Y.
> > > > > > >
> > > > > > > +config RISCV_ISA_ZICBOP
> > > > > > > + bool "Zicbop extension support for cache block prefetch"
> > > > > > > + depends on MMU
> > > > > > > + depends on RISCV_ALTERNATIVE
> > > > > > > + default y
> > > > > > > + help
> > > > > > > + Adds support to dynamically detect the presence of the ZICBOP
> > > > > > > + extension (Cache Block Prefetch Operations) and enable its
> > > > > > > + usage.
> > > > > > > +
> > > > > > > + The Zicbop extension can be used to prefetch cache block for
> > > > > > > + read/write/instruction fetch.
> > > > > > > +
> > > > > > > + If you don't know what to do here, say Y.
> > > > > > > +
> > > > > > > config TOOLCHAIN_HAS_ZIHINTPAUSE
> > > > > > > bool
> > > > > > > default y
> > > > > > > diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
> > > > > > > index 702725727671..56eff7a9d2d2 100644
> > > > > > > --- a/arch/riscv/include/asm/cmpxchg.h
> > > > > > > +++ b/arch/riscv/include/asm/cmpxchg.h
> > > > > > > @@ -11,6 +11,7 @@
> > > > > > >
> > > > > > > #include <asm/barrier.h>
> > > > > > > #include <asm/fence.h>
> > > > > > > +#include <asm/processor.h>
> > > > > > >
> > > > > > > #define __arch_xchg_masked(prepend, append, r, p, n) \
> > > > > > > ({ \
> > > > > > > @@ -25,6 +26,7 @@
> > > > > > > \
> > > > > > > __asm__ __volatile__ ( \
> > > > > > > prepend \
> > > > > > > + PREFETCHW_ASM(%5) \
> > > > > > > "0: lr.w %0, %2\n" \
> > > > > > > " and %1, %0, %z4\n" \
> > > > > > > " or %1, %1, %z3\n" \
> > > > > > > @@ -32,7 +34,7 @@
> > > > > > > " bnez %1, 0b\n" \
> > > > > > > append \
> > > > > > > : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
> > > > > > > - : "rJ" (__newx), "rJ" (~__mask) \
> > > > > > > + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
> > > > > > > : "memory"); \
> > > > > > > \
> > > > > > > r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
> > > > > > > diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> > > > > > > index b7b58258f6c7..78b7b8b53778 100644
> > > > > > > --- a/arch/riscv/include/asm/hwcap.h
> > > > > > > +++ b/arch/riscv/include/asm/hwcap.h
> > > > > > > @@ -58,6 +58,7 @@
> > > > > > > #define RISCV_ISA_EXT_ZICSR 40
> > > > > > > #define RISCV_ISA_EXT_ZIFENCEI 41
> > > > > > > #define RISCV_ISA_EXT_ZIHPM 42
> > > > > > > +#define RISCV_ISA_EXT_ZICBOP 43
> > > > > > >
> > > > > > > #define RISCV_ISA_EXT_MAX 64
> > > > > > >
> > > > > > > diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
> > > > > > > index 6960beb75f32..dc590d331894 100644
> > > > > > > --- a/arch/riscv/include/asm/insn-def.h
> > > > > > > +++ b/arch/riscv/include/asm/insn-def.h
> > > > > > > @@ -134,6 +134,7 @@
> > > > > > >
> > > > > > > #define RV_OPCODE_MISC_MEM RV_OPCODE(15)
> > > > > > > #define RV_OPCODE_SYSTEM RV_OPCODE(115)
> > > > > > > +#define RV_OPCODE_PREFETCH RV_OPCODE(19)
> > > > > > >
> > > > > > > #define HFENCE_VVMA(vaddr, asid) \
> > > > > > > INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
> > > > > > > @@ -196,4 +197,8 @@
> > > > > > > INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
> > > > > > > RS1(base), SIMM12(4))
> > > > > > >
> > > > > > > +#define CBO_prefetchw(base) \
> > > > > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > > > > + RD(x0), RS1(base), RS2(x0))
> > > > > > > +
> > > > > >
> > > > > > I understand that here you create the instruction via bitfield, following
> > > > > > the ISA, and this enables using instructions not available on the
> > > > > > toolchain.
> > > > > >
> > > > > > It took me some time to find the document with this instruction, so please
> > > > > > add this to the commit msg:
> > > > > >
> > > > > > https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.pdf
> > > > > > Page 23.
> > > > > >
> > > > > > IIUC, the instruction is "prefetch.w".
> > > > > >
> > > > > > Maybe I am missing something, but in the document the rs2 field
> > > > > > (PREFETCH.W) contains a 0x3, while the above looks to have a 0 instead.
> > > > > >
> > > > > > rs2 field = 0x0 would be a prefetch.i (instruction prefetch) instead.
> > > > > >
> > > > > > Is the above correct, or am I missing something?
> > > > > Oh, you are right. My fault, thx for pointing out. It should be:
> > > > > + INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
> > > > > + RD(x0), RS1(base), RS2(x3))
> > > >
> > > > Now I am curious to check if / how will this impact performance. :)
> > > > (Please let me know)
> > > Ref:
> > > commit 0ea366f5e1b6 ("arm64: atomics: prefetch the destination word
> > > for write prior to stxr")
> > > commit 86d231459d6d ("bpf: cpumap memory prefetchw optimizations for
> > > struct page")
> >
> > Oh, I understand that prefetch.w is very useful for performance :)
> >
> > What I meant is that previously this patch was issuing a prefetch.i,
> > and now it's issuing a prefetch.w (as intended).
> >
> > What got me curious is how much would it impact the performance to change
> > the prefetch.i to prefetch.w. :)
> The current SOPHO sg2042 hardware platform didn't support prefetch.w
> instruction. So there is no performance result I could share with you.
>
> Our next generation of processors would support ZICBOP.
Oh, okay then.
Thanks for sharing this info!
Leo
>
> >
> > Thanks!
> > Leo
> >
> >
> > >
> > > >
> > > >
> > > > >
> > > > > >
> > > > > >
> > > > > > Thanks!
> > > > > > Leo
> > > > > >
> > > > > > > #endif /* __ASM_INSN_DEF_H */
> > > > > > > diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> > > > > > > index de9da852f78d..7ad3a24212e8 100644
> > > > > > > --- a/arch/riscv/include/asm/processor.h
> > > > > > > +++ b/arch/riscv/include/asm/processor.h
> > > > > > > @@ -12,6 +12,8 @@
> > > > > > > #include <vdso/processor.h>
> > > > > > >
> > > > > > > #include <asm/ptrace.h>
> > > > > > > +#include <asm/insn-def.h>
> > > > > > > +#include <asm/hwcap.h>
> > > > > > >
> > > > > > > #ifdef CONFIG_64BIT
> > > > > > > #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
> > > > > > > @@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
> > > > > > > #define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
> > > > > > > #define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
> > > > > > >
> > > > > > > +#define ARCH_HAS_PREFETCHW
> > > > > > > +#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
> > > > > > > + CBO_prefetchw(base), \
> > > > > > > + 0, \
> > > > > > > + RISCV_ISA_EXT_ZICBOP, \
> > > > > > > + CONFIG_RISCV_ISA_ZICBOP)
> > > > > > > +static inline void prefetchw(const void *ptr)
> > > > > > > +{
> > > > > > > + asm volatile(PREFETCHW_ASM(%0)
> > > > > > > + : : "r" (ptr) : "memory");
> > > > > > > +}
> > > > > > >
> > > > > > > /* Do necessary setup to start up a newly executed thread. */
> > > > > > > extern void start_thread(struct pt_regs *regs,
> > > > > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > > > > > index ef7b4fd9e876..e0b897db0b97 100644
> > > > > > > --- a/arch/riscv/kernel/cpufeature.c
> > > > > > > +++ b/arch/riscv/kernel/cpufeature.c
> > > > > > > @@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
> > > > > > > __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
> > > > > > > __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
> > > > > > > __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
> > > > > > > + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
> > > > > > > __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
> > > > > > > __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
> > > > > > > __RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
> > > > > > > --
> > > > > > > 2.36.1
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best Regards
> > > > > Guo Ren
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
2023-09-19 8:04 ` Guo Ren
@ 2023-09-19 14:40 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 14:40 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 19, 2023 at 04:04:48PM +0800, Guo Ren wrote:
> On Tue, Sep 19, 2023 at 1:30 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 17, 2023 at 11:12:31PM +0800, Guo Ren wrote:
> > > On Thu, Sep 14, 2023 at 4:02 PM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > On Sun, Sep 10, 2023 at 04:29:02AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > >
> > > > > Add a static key controlling whether virt_spin_lock() should be
> > > > > called or not. When running on bare metal set the new key to
> > > > > false.
> > > > >
> > > > > The KVM guests fall back to a Test-and-Set spinlock, because fair
> > > > > locks have horrible lock 'holder' preemption issues. The
> > > > > virt_spin_lock_key would shortcut for the
> > > > > queued_spin_lock_slowpath() function that allow virt_spin_lock to
> > > > > hijack it.
> > > > >
> > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > ---
> > > > > .../admin-guide/kernel-parameters.txt | 4 +++
> > > > > arch/riscv/include/asm/sbi.h | 8 +++++
> > > > > arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
> > > > > arch/riscv/kernel/sbi.c | 2 +-
> > > > > arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
> > > > > 5 files changed, 64 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > > > index 61cacb8dfd0e..f75bedc50e00 100644
> > > > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > > > @@ -3927,6 +3927,10 @@
> > > > > no_uaccess_flush
> > > > > [PPC] Don't flush the L1-D cache after accessing user data.
> > > > >
> > > > > + no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
> > > > > + native_queued_spinlock when the nopvspin option is enabled.
> > > > > + This would help vcpu=pcpu scenarios.
> > > > > +
> > > > > novmcoredd [KNL,KDUMP]
> > > > > Disable device dump. Device dump allows drivers to
> > > > > append dump data to vmcore so you can collect driver
> > > > > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > > > > index 501e06e52078..e0233b3d7a5f 100644
> > > > > --- a/arch/riscv/include/asm/sbi.h
> > > > > +++ b/arch/riscv/include/asm/sbi.h
> > > > > @@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
> > > > > SBI_EXT_BASE_GET_MIMPID,
> > > > > };
> > > > >
> > > > > +enum sbi_ext_base_impl_id {
> > > > > + SBI_EXT_BASE_IMPL_ID_BBL = 0,
> > > > > + SBI_EXT_BASE_IMPL_ID_OPENSBI,
> > > > > + SBI_EXT_BASE_IMPL_ID_XVISOR,
> > > > > + SBI_EXT_BASE_IMPL_ID_KVM,
> > > > > +};
> > > > > +
> > > > > enum sbi_ext_time_fid {
> > > > > SBI_EXT_TIME_SET_TIMER = 0,
> > > > > };
> > > > > @@ -269,6 +276,7 @@ int sbi_console_getchar(void);
> > > > > long sbi_get_mvendorid(void);
> > > > > long sbi_get_marchid(void);
> > > > > long sbi_get_mimpid(void);
> > > > > +long sbi_get_firmware_id(void);
> > > > > void sbi_set_timer(uint64_t stime_value);
> > > > > void sbi_shutdown(void);
> > > > > void sbi_send_ipi(unsigned int cpu);
> > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > index 8ea0fee80652..6b38d6616f14 100644
> > > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > @@ -4,6 +4,28 @@
> > > > > #define __ASM_RISCV_SPINLOCK_H
> > > > >
> > > > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > +/*
> > > > > + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
> > > > > + * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
> > > > > + * would shortcut for the queued_spin_lock_slowpath() function that allow
> > > > > + * virt_spin_lock to hijack it.
> > > > > + */
> > > > > +DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > > > +
> > > > > +#define virt_spin_lock virt_spin_lock
> > > > > +static inline bool virt_spin_lock(struct qspinlock *lock)
> > > > > +{
> > > > > + if (!static_branch_likely(&virt_spin_lock_key))
> > > > > + return false;
> > > > > +
> > > > > + do {
> > > > > + while (atomic_read(&lock->val) != 0)
> > > > > + cpu_relax();
> > > > > + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
> > > > > +
> > > > > + return true;
> > > > > +}
> > > > > +
> > > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > > #endif
> > > > >
> > > > > diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> > > > > index 88eea3a99ee0..cdd45edc8db4 100644
> > > > > --- a/arch/riscv/kernel/sbi.c
> > > > > +++ b/arch/riscv/kernel/sbi.c
> > > > > @@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
> > > > > return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
> > > > > }
> > > > >
> > > > > -static inline long sbi_get_firmware_id(void)
> > > > > +long sbi_get_firmware_id(void)
> > > > > {
> > > > > return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
> > > > > }
> > > > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > > > index 0f084f037651..c57d15b05160 100644
> > > > > --- a/arch/riscv/kernel/setup.c
> > > > > +++ b/arch/riscv/kernel/setup.c
> > > > > @@ -26,6 +26,7 @@
> > > > > #include <asm/alternative.h>
> > > > > #include <asm/cacheflush.h>
> > > > > #include <asm/cpu_ops.h>
> > > > > +#include <asm/cpufeature.h>
> > > > > #include <asm/early_ioremap.h>
> > > > > #include <asm/pgtable.h>
> > > > > #include <asm/setup.h>
> > > > > @@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > > EXPORT_SYMBOL(combo_qspinlock_key);
> > > > > #endif
> > > > >
> > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > +static bool no_virt_spin_key = false;
> > > >
> > > > I suggest no _key, also there is no need for "= false".
> > > > To be consistent with enable_qspinlock, I also suggest
> > > > adding __ro_after_init:
> > > >
> > > > static bool no_virt_spin __ro_after_init;
> > > okay.
> > >
> > > >
> > > >
> > > >
> > > > > +DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > > > +
> > > > > +static int __init no_virt_spin_setup(char *p)
> > > > > +{
> > > > > + no_virt_spin_key = true;
> > > > > +
> > > > > + return 0;
> > > > > +}
> > > > > +early_param("no_virt_spin", no_virt_spin_setup);
> > > > > +
> > > > > +static void __init virt_spin_lock_init(void)
> > > > > +{
> > > > > + if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
> > > > > + no_virt_spin_key)
> > > > > + static_branch_disable(&virt_spin_lock_key);
> > > > > + else
> > > > > + pr_info("Enable virt_spin_lock\n");
> > > > > +}
> > > > > +#endif
> > > > > +
> > > >
> > > > A new virt_no_spin kernel parameter was introduced, but without
> > > > CONFIG_QUEUED_SPINLOCKS it will silently fail.
> > > >
> > > > I would suggest an #else clause here with a function to print an error /
> > > > warning message about no_virt_spin being invalid in this scenario.
> > > > It will probably help future debugging.
> > > If CONFIG_QUEUED_SPINLOCKS=n, no_virt_spin should be quiet. The
> > > no_virt_spin is one path of qspinlock.
> >
> > IIUC having no_virt_spin being passed as parameter to a kernel with
> > CONFIG_QUEUED_SPINLOCKS=n is not supposed to have any warning this
> > parameter is useless.
> >
> > I was just thinking it would be nice to have this warning during debugging,
> > but if it's standard practice then I am ok with this.
> Yes, I think it's okay, e.g.,
> x86: early_param("hv_nopvspin", hv_parse_nopvspin);
> depends on CONFIG_PARAVIRT_SPINLOCKS=y
Okay then, thanks for sharing this info!
>
> >
> > >
> > > >
> > > >
> > > > > static void __init riscv_spinlock_init(void)
> > > > > {
> > > > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > > - if (!enable_qspinlock_key) {
> > > > > + if (!enable_qspinlock_key &&
> > > > > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > > > > static_branch_disable(&combo_qspinlock_key);
> > > > > pr_info("Ticket spinlock: enabled\n");
> > > > > } else {
> > > > > pr_info("Queued spinlock: enabled\n");
> > > > > }
> > > > > #endif
> > > > > +
> > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > + virt_spin_lock_init();
> > > > > +#endif
> > > > > }
> > > > >
> > > > > extern void __init init_rt_signal_env(void);
> > > > > --
> > > > > 2.36.1
> > > > >
> > > >
> > > > I am probably missing something out, but it looks to me that this patch is
> > > > causing 2 different changes:
> > > > 1 - Enabling no_virt_spin parameter
> > > > 2 - Disabling queued spinlocks for some firmware_id
> > > >
> > > > Wouldn't be better to split those changes in multiple patches?
> > > > Or am I missing the point on why they need to be together?
> >
> > ^ Want your input on this
> Sorry, I missed that. Okay, I would split those changes.
Thanks!
Leo
>
> >
> > Thanks!
> > Leo
> >
> > > >
> > > > Thanks!
> > > > Leo
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest
@ 2023-09-19 14:40 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-09-19 14:40 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Tue, Sep 19, 2023 at 04:04:48PM +0800, Guo Ren wrote:
> On Tue, Sep 19, 2023 at 1:30 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 17, 2023 at 11:12:31PM +0800, Guo Ren wrote:
> > > On Thu, Sep 14, 2023 at 4:02 PM Leonardo Bras <leobras@redhat.com> wrote:
> > > >
> > > > On Sun, Sep 10, 2023 at 04:29:02AM -0400, guoren@kernel.org wrote:
> > > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > > >
> > > > > Add a static key controlling whether virt_spin_lock() should be
> > > > > called or not. When running on bare metal set the new key to
> > > > > false.
> > > > >
> > > > > The KVM guests fall back to a Test-and-Set spinlock, because fair
> > > > > locks have horrible lock 'holder' preemption issues. The
> > > > > virt_spin_lock_key would shortcut for the
> > > > > queued_spin_lock_slowpath() function that allow virt_spin_lock to
> > > > > hijack it.
> > > > >
> > > > > Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
> > > > > Signed-off-by: Guo Ren <guoren@kernel.org>
> > > > > ---
> > > > > .../admin-guide/kernel-parameters.txt | 4 +++
> > > > > arch/riscv/include/asm/sbi.h | 8 +++++
> > > > > arch/riscv/include/asm/spinlock.h | 22 ++++++++++++++
> > > > > arch/riscv/kernel/sbi.c | 2 +-
> > > > > arch/riscv/kernel/setup.c | 30 ++++++++++++++++++-
> > > > > 5 files changed, 64 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > > > index 61cacb8dfd0e..f75bedc50e00 100644
> > > > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > > > @@ -3927,6 +3927,10 @@
> > > > > no_uaccess_flush
> > > > > [PPC] Don't flush the L1-D cache after accessing user data.
> > > > >
> > > > > + no_virt_spin [RISC-V] Disable virt_spin_lock in KVM guest to use
> > > > > + native_queued_spinlock when the nopvspin option is enabled.
> > > > > + This would help vcpu=pcpu scenarios.
> > > > > +
> > > > > novmcoredd [KNL,KDUMP]
> > > > > Disable device dump. Device dump allows drivers to
> > > > > append dump data to vmcore so you can collect driver
> > > > > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > > > > index 501e06e52078..e0233b3d7a5f 100644
> > > > > --- a/arch/riscv/include/asm/sbi.h
> > > > > +++ b/arch/riscv/include/asm/sbi.h
> > > > > @@ -50,6 +50,13 @@ enum sbi_ext_base_fid {
> > > > > SBI_EXT_BASE_GET_MIMPID,
> > > > > };
> > > > >
> > > > > +enum sbi_ext_base_impl_id {
> > > > > + SBI_EXT_BASE_IMPL_ID_BBL = 0,
> > > > > + SBI_EXT_BASE_IMPL_ID_OPENSBI,
> > > > > + SBI_EXT_BASE_IMPL_ID_XVISOR,
> > > > > + SBI_EXT_BASE_IMPL_ID_KVM,
> > > > > +};
> > > > > +
> > > > > enum sbi_ext_time_fid {
> > > > > SBI_EXT_TIME_SET_TIMER = 0,
> > > > > };
> > > > > @@ -269,6 +276,7 @@ int sbi_console_getchar(void);
> > > > > long sbi_get_mvendorid(void);
> > > > > long sbi_get_marchid(void);
> > > > > long sbi_get_mimpid(void);
> > > > > +long sbi_get_firmware_id(void);
> > > > > void sbi_set_timer(uint64_t stime_value);
> > > > > void sbi_shutdown(void);
> > > > > void sbi_send_ipi(unsigned int cpu);
> > > > > diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h
> > > > > index 8ea0fee80652..6b38d6616f14 100644
> > > > > --- a/arch/riscv/include/asm/spinlock.h
> > > > > +++ b/arch/riscv/include/asm/spinlock.h
> > > > > @@ -4,6 +4,28 @@
> > > > > #define __ASM_RISCV_SPINLOCK_H
> > > > >
> > > > > #ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > +/*
> > > > > + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks
> > > > > + * have horrible lock 'holder' preemption issues. The virt_spin_lock_key
> > > > > + * would shortcut for the queued_spin_lock_slowpath() function that allow
> > > > > + * virt_spin_lock to hijack it.
> > > > > + */
> > > > > +DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > > > +
> > > > > +#define virt_spin_lock virt_spin_lock
> > > > > +static inline bool virt_spin_lock(struct qspinlock *lock)
> > > > > +{
> > > > > + if (!static_branch_likely(&virt_spin_lock_key))
> > > > > + return false;
> > > > > +
> > > > > + do {
> > > > > + while (atomic_read(&lock->val) != 0)
> > > > > + cpu_relax();
> > > > > + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0);
> > > > > +
> > > > > + return true;
> > > > > +}
> > > > > +
> > > > > #define _Q_PENDING_LOOPS (1 << 9)
> > > > > #endif
> > > > >
> > > > > diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
> > > > > index 88eea3a99ee0..cdd45edc8db4 100644
> > > > > --- a/arch/riscv/kernel/sbi.c
> > > > > +++ b/arch/riscv/kernel/sbi.c
> > > > > @@ -555,7 +555,7 @@ static inline long sbi_get_spec_version(void)
> > > > > return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION);
> > > > > }
> > > > >
> > > > > -static inline long sbi_get_firmware_id(void)
> > > > > +long sbi_get_firmware_id(void)
> > > > > {
> > > > > return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID);
> > > > > }
> > > > > diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> > > > > index 0f084f037651..c57d15b05160 100644
> > > > > --- a/arch/riscv/kernel/setup.c
> > > > > +++ b/arch/riscv/kernel/setup.c
> > > > > @@ -26,6 +26,7 @@
> > > > > #include <asm/alternative.h>
> > > > > #include <asm/cacheflush.h>
> > > > > #include <asm/cpu_ops.h>
> > > > > +#include <asm/cpufeature.h>
> > > > > #include <asm/early_ioremap.h>
> > > > > #include <asm/pgtable.h>
> > > > > #include <asm/setup.h>
> > > > > @@ -283,16 +284,43 @@ DEFINE_STATIC_KEY_TRUE(combo_qspinlock_key);
> > > > > EXPORT_SYMBOL(combo_qspinlock_key);
> > > > > #endif
> > > > >
> > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > +static bool no_virt_spin_key = false;
> > > >
> > > > I suggest no _key, also there is no need for "= false".
> > > > To be consistent with enable_qspinlock, I also suggest
> > > > adding __ro_after_init:
> > > >
> > > > static bool no_virt_spin __ro_after_init;
> > > okay.
> > >
> > > >
> > > >
> > > >
> > > > > +DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);
> > > > > +
> > > > > +static int __init no_virt_spin_setup(char *p)
> > > > > +{
> > > > > + no_virt_spin_key = true;
> > > > > +
> > > > > + return 0;
> > > > > +}
> > > > > +early_param("no_virt_spin", no_virt_spin_setup);
> > > > > +
> > > > > +static void __init virt_spin_lock_init(void)
> > > > > +{
> > > > > + if (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM ||
> > > > > + no_virt_spin_key)
> > > > > + static_branch_disable(&virt_spin_lock_key);
> > > > > + else
> > > > > + pr_info("Enable virt_spin_lock\n");
> > > > > +}
> > > > > +#endif
> > > > > +
> > > >
> > > > A new virt_no_spin kernel parameter was introduced, but without
> > > > CONFIG_QUEUED_SPINLOCKS it will silently fail.
> > > >
> > > > I would suggest an #else clause here with a function to print an error /
> > > > warning message about no_virt_spin being invalid in this scenario.
> > > > It will probably help future debugging.
> > > If CONFIG_QUEUED_SPINLOCKS=n, no_virt_spin should be quiet. The
> > > no_virt_spin is one path of qspinlock.
> >
> > IIUC having no_virt_spin being passed as parameter to a kernel with
> > CONFIG_QUEUED_SPINLOCKS=n is not supposed to have any warning this
> > parameter is useless.
> >
> > I was just thinking it would be nice to have this warning during debugging,
> > but if it's standard practice then I am ok with this.
> Yes, I think it's okay, e.g.,
> x86: early_param("hv_nopvspin", hv_parse_nopvspin);
> depends on CONFIG_PARAVIRT_SPINLOCKS=y
Okay then, thanks for sharing this info!
>
> >
> > >
> > > >
> > > >
> > > > > static void __init riscv_spinlock_init(void)
> > > > > {
> > > > > #ifdef CONFIG_RISCV_COMBO_SPINLOCKS
> > > > > - if (!enable_qspinlock_key) {
> > > > > + if (!enable_qspinlock_key &&
> > > > > + (sbi_get_firmware_id() != SBI_EXT_BASE_IMPL_ID_KVM)) {
> > > > > static_branch_disable(&combo_qspinlock_key);
> > > > > pr_info("Ticket spinlock: enabled\n");
> > > > > } else {
> > > > > pr_info("Queued spinlock: enabled\n");
> > > > > }
> > > > > #endif
> > > > > +
> > > > > +#ifdef CONFIG_QUEUED_SPINLOCKS
> > > > > + virt_spin_lock_init();
> > > > > +#endif
> > > > > }
> > > > >
> > > > > extern void __init init_rt_signal_env(void);
> > > > > --
> > > > > 2.36.1
> > > > >
> > > >
> > > > I am probably missing something out, but it looks to me that this patch is
> > > > causing 2 different changes:
> > > > 1 - Enabling no_virt_spin parameter
> > > > 2 - Disabling queued spinlocks for some firmware_id
> > > >
> > > > Wouldn't be better to split those changes in multiple patches?
> > > > Or am I missing the point on why they need to be together?
> >
> > ^ Want your input on this
> Sorry, I missed that. Okay, I would split those changes.
Thanks!
Leo
>
> >
> > Thanks!
> > Leo
> >
> > > >
> > > > Thanks!
> > > > Leo
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > > Guo Ren
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-09-10 8:28 ` guoren
@ 2023-11-06 20:42 ` Leonardo Bras
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-11-06 20:42 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> patch[1 - 10]: Native qspinlock
> patch[11 -17]: Paravirt qspinlock
>
> patch[4]: Add prefetchw in qspinlock's xchg_tail when cpus >= 16k
>
> This series based on:
> - [RFC PATCH v5 0/5] Rework & improve riscv cmpxchg.h and atomic.h
> https://lore.kernel.org/linux-riscv/20230810040349.92279-2-leobras@redhat.com/
> - [PATCH V3] asm-generic: ticket-lock: Optimize arch_spin_value_unlocked
> https://lore.kernel.org/linux-riscv/20230908154339.3250567-1-guoren@kernel.org/
>
> I merge them into sg2042-master branch, then you could directly try it on
> sg2042 hardware platform:
>
> https://github.com/guoren83/linux/tree/sg2042-master-qspinlock-64ilp32_v5
>
> Use sophgo_mango_ubuntu_defconfig for sg2042 64/128 cores hardware
> platform.
>
> Native qspinlock
> ================
>
> This time we've proved the qspinlock on th1520 [1] & sg2042 [2], which
> gives stability and performance improvement. All T-HEAD processors have
> a strong LR/SC forward progress guarantee than the requirements of the
> ISA, which could satisfy the xchg_tail of native_qspinlock. Now,
> qspinlock has been run with us for more than 1 year, and we have enough
> confidence to enable it for all the T-HEAD processors. Of causes, we
> found a livelock problem with the qspinlock lock torture test from the
> CPU store merge buffer delay mechanism, which caused the queued spinlock
> becomes a dead ring and RCU warning to come out. We introduce a custom
> WRITE_ONCE to solve this. Do we need explicit ISA instruction to signal
> it? Or let hardware handle this.
>
> We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> comparison between qspinlock and ticket_lock on sg2042 (64 cores):
>
> sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> queued_spinlock 0.5109/0.00
> ticket_spinlock 0.5814/0.00
>
> perf futex/hash (+6.7%):
> queued_spinlock 1444393 operations/sec (+- 0.09%)
> ticket_spinlock 1353215 operations/sec (+- 0.15%)
>
> perf futex/wake-parallel (+8.6%):
> queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
>
> perf futex/requeue (+4.2%):
> queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
>
> System Benchmarks (+6.4%)
> queued_spinlock:
> System Benchmarks Index Values BASELINE RESULT INDEX
> Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> Double-Precision Whetstone 55.0 182422.8 33167.8
> Execl Throughput 43.0 13116.6 3050.4
> File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> Pipe Throughput 12440.0 23058600.5 18535.9
> Pipe-based Context Switching 4000.0 2835617.7 7089.0
> Process Creation 126.0 12537.3 995.0
> Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> System Call Overhead 15000.0 33308301.3 22205.5
> ========
> System Benchmarks Index Score 12426.1
>
> ticket_spinlock:
> System Benchmarks Index Values BASELINE RESULT INDEX
> Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> Double-Precision Whetstone 55.0 181921.0 33076.5
> Execl Throughput 43.0 12625.1 2936.1
> File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> Pipe Throughput 12440.0 20594018.7 16554.7
> Pipe-based Context Switching 4000.0 2571117.7 6427.8
> Process Creation 126.0 10798.4 857.0
> Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> System Call Overhead 15000.0 30766778.4 20511.2
> ========
> System Benchmarks Index Score 11670.7
>
> The qspinlock has a significant improvement on SOPHGO SG2042 64
> cores platform than the ticket_lock.
>
> Paravirt qspinlock
> ==================
>
> We implemented kvm_kick_cpu/kvm_wait_cpu and add tracepoints to observe the
> behaviors. Also, introduce a new SBI extension SBI_EXT_PVLOCK (0xAB0401). If the
> name and number are approved, I will send a formal proposal to the SBI spec.
>
Hello Guo Ren,
Any update on this series?
Thanks!
Leo
> Changlog:
> V11:
> - Based on Leonardo Bras's cmpxchg_small patches v5.
> - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> - Remove abusing alternative framework and use jump_label instead.
> - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> guarantee.
> - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
>
> V10:
> https://lore.kernel.org/linux-riscv/20230802164701.192791-1-guoren@kernel.org/
> - Using an alternative framework instead of static_key_branch in the
> asm/spinlock.h.
> - Fixup store merge buffer problem, which causes qspinlock lock
> torture test livelock.
> - Add paravirt qspinlock support, include KVM backend
> - Add Compact NUMA-awared qspinlock support
>
> V9:
> https://lore.kernel.org/linux-riscv/20220808071318.3335746-1-guoren@kernel.org/
> - Cleanup generic ticket-lock code, (Using smp_mb__after_spinlock as
> RCsc)
> - Add qspinlock and combo-lock for riscv
> - Add qspinlock to openrisc
> - Use generic header in csky
> - Optimize cmpxchg & atomic code
>
> V8:
> https://lore.kernel.org/linux-riscv/20220724122517.1019187-1-guoren@kernel.org/
> - Coding convention ticket fixup
> - Move combo spinlock into riscv and simply asm-generic/spinlock.h
> - Fixup xchg16 with wrong return value
> - Add csky qspinlock
> - Add combo & qspinlock & ticket-lock comparison
> - Clean up unnecessary riscv acquire and release definitions
> - Enable ARCH_INLINE_READ*/WRITE*/SPIN* for riscv & csky
>
> V7:
> https://lore.kernel.org/linux-riscv/20220628081946.1999419-1-guoren@kernel.org/
> - Add combo spinlock (ticket & queued) support
> - Rename ticket_spinlock.h
> - Remove unnecessary atomic_read in ticket_spin_value_unlocked
>
> V6:
> https://lore.kernel.org/linux-riscv/20220621144920.2945595-1-guoren@kernel.org/
> - Fixup Clang compile problem Reported-by: kernel test robot
> - Cleanup asm-generic/spinlock.h
> - Remove changelog in patch main comment part, suggested by
> Conor.Dooley
> - Remove "default y if NUMA" in Kconfig
>
> V5:
> https://lore.kernel.org/linux-riscv/20220620155404.1968739-1-guoren@kernel.org/
> - Update comment with RISC-V forward guarantee feature.
> - Back to V3 direction and optimize asm code.
>
> V4:
> https://lore.kernel.org/linux-riscv/1616868399-82848-4-git-send-email-guoren@kernel.org/
> - Remove custom sub-word xchg implementation
> - Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 in locking/qspinlock
>
> V3:
> https://lore.kernel.org/linux-riscv/1616658937-82063-1-git-send-email-guoren@kernel.org/
> - Coding convention by Peter Zijlstra's advices
>
> V2:
> https://lore.kernel.org/linux-riscv/1606225437-22948-2-git-send-email-guoren@kernel.org/
> - Coding convention in cmpxchg.h
> - Re-implement short xchg
> - Remove char & cmpxchg implementations
>
> V1:
> https://lore.kernel.org/linux-riscv/20190211043829.30096-1-michaeljclark@mac.com/
> - Using cmpxchg loop to implement sub-word atomic
>
>
> Guo Ren (17):
> asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
> asm-generic: ticket-lock: Move into ticket_spinlock.h
> riscv: Use Zicbop in arch_xchg when available
> locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
> riscv: qspinlock: Add basic queued_spinlock support
> riscv: qspinlock: Introduce combo spinlock
> riscv: qspinlock: Introduce qspinlock param for command line
> riscv: qspinlock: Add virt_spin_lock() support for KVM guest
> riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
> RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
> RISC-V: paravirt: pvqspinlock: Add SBI implementation
> RISC-V: paravirt: pvqspinlock: Add kconfig entry
> RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
> RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
> RISC-V: paravirt: pvqspinlock: KVM: Implement
> kvm_sbi_ext_pvlock_kick_cpu()
>
> .../admin-guide/kernel-parameters.txt | 8 +-
> arch/riscv/Kconfig | 50 ++++++++
> arch/riscv/Kconfig.errata | 19 +++
> arch/riscv/errata/thead/errata.c | 29 +++++
> arch/riscv/include/asm/Kbuild | 2 +-
> arch/riscv/include/asm/cmpxchg.h | 4 +-
> arch/riscv/include/asm/errata_list.h | 13 --
> arch/riscv/include/asm/hwcap.h | 1 +
> arch/riscv/include/asm/insn-def.h | 5 +
> arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
> arch/riscv/include/asm/processor.h | 13 ++
> arch/riscv/include/asm/qspinlock.h | 35 ++++++
> arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++
> arch/riscv/include/asm/rwonce.h | 24 ++++
> arch/riscv/include/asm/sbi.h | 14 +++
> arch/riscv/include/asm/spinlock.h | 113 ++++++++++++++++++
> arch/riscv/include/asm/vendorid_list.h | 14 +++
> arch/riscv/include/uapi/asm/kvm.h | 1 +
> arch/riscv/kernel/Makefile | 1 +
> arch/riscv/kernel/cpufeature.c | 1 +
> arch/riscv/kernel/qspinlock_paravirt.c | 83 +++++++++++++
> arch/riscv/kernel/sbi.c | 2 +-
> arch/riscv/kernel/setup.c | 60 ++++++++++
> .../kernel/trace_events_filter_paravirt.h | 60 ++++++++++
> arch/riscv/kvm/Makefile | 1 +
> arch/riscv/kvm/vcpu_sbi.c | 4 +
> arch/riscv/kvm/vcpu_sbi_pvlock.c | 57 +++++++++
> include/asm-generic/rwonce.h | 2 +
> include/asm-generic/spinlock.h | 87 +-------------
> include/asm-generic/spinlock_types.h | 12 +-
> include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++
> kernel/locking/qspinlock.c | 5 +-
> 32 files changed, 739 insertions(+), 114 deletions(-)
> create mode 100644 arch/riscv/include/asm/qspinlock.h
> create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> create mode 100644 arch/riscv/include/asm/rwonce.h
> create mode 100644 arch/riscv/include/asm/spinlock.h
> create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
> create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
> create mode 100644 include/asm-generic/ticket_spinlock.h
>
> --
> 2.36.1
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-11-06 20:42 ` Leonardo Bras
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras @ 2023-11-06 20:42 UTC (permalink / raw)
To: guoren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> From: Guo Ren <guoren@linux.alibaba.com>
>
> patch[1 - 10]: Native qspinlock
> patch[11 -17]: Paravirt qspinlock
>
> patch[4]: Add prefetchw in qspinlock's xchg_tail when cpus >= 16k
>
> This series based on:
> - [RFC PATCH v5 0/5] Rework & improve riscv cmpxchg.h and atomic.h
> https://lore.kernel.org/linux-riscv/20230810040349.92279-2-leobras@redhat.com/
> - [PATCH V3] asm-generic: ticket-lock: Optimize arch_spin_value_unlocked
> https://lore.kernel.org/linux-riscv/20230908154339.3250567-1-guoren@kernel.org/
>
> I merge them into sg2042-master branch, then you could directly try it on
> sg2042 hardware platform:
>
> https://github.com/guoren83/linux/tree/sg2042-master-qspinlock-64ilp32_v5
>
> Use sophgo_mango_ubuntu_defconfig for sg2042 64/128 cores hardware
> platform.
>
> Native qspinlock
> ================
>
> This time we've proved the qspinlock on th1520 [1] & sg2042 [2], which
> gives stability and performance improvement. All T-HEAD processors have
> a strong LR/SC forward progress guarantee than the requirements of the
> ISA, which could satisfy the xchg_tail of native_qspinlock. Now,
> qspinlock has been run with us for more than 1 year, and we have enough
> confidence to enable it for all the T-HEAD processors. Of causes, we
> found a livelock problem with the qspinlock lock torture test from the
> CPU store merge buffer delay mechanism, which caused the queued spinlock
> becomes a dead ring and RCU warning to come out. We introduce a custom
> WRITE_ONCE to solve this. Do we need explicit ISA instruction to signal
> it? Or let hardware handle this.
>
> We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> comparison between qspinlock and ticket_lock on sg2042 (64 cores):
>
> sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> queued_spinlock 0.5109/0.00
> ticket_spinlock 0.5814/0.00
>
> perf futex/hash (+6.7%):
> queued_spinlock 1444393 operations/sec (+- 0.09%)
> ticket_spinlock 1353215 operations/sec (+- 0.15%)
>
> perf futex/wake-parallel (+8.6%):
> queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
>
> perf futex/requeue (+4.2%):
> queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
>
> System Benchmarks (+6.4%)
> queued_spinlock:
> System Benchmarks Index Values BASELINE RESULT INDEX
> Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> Double-Precision Whetstone 55.0 182422.8 33167.8
> Execl Throughput 43.0 13116.6 3050.4
> File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> Pipe Throughput 12440.0 23058600.5 18535.9
> Pipe-based Context Switching 4000.0 2835617.7 7089.0
> Process Creation 126.0 12537.3 995.0
> Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> System Call Overhead 15000.0 33308301.3 22205.5
> ========
> System Benchmarks Index Score 12426.1
>
> ticket_spinlock:
> System Benchmarks Index Values BASELINE RESULT INDEX
> Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> Double-Precision Whetstone 55.0 181921.0 33076.5
> Execl Throughput 43.0 12625.1 2936.1
> File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> Pipe Throughput 12440.0 20594018.7 16554.7
> Pipe-based Context Switching 4000.0 2571117.7 6427.8
> Process Creation 126.0 10798.4 857.0
> Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> System Call Overhead 15000.0 30766778.4 20511.2
> ========
> System Benchmarks Index Score 11670.7
>
> The qspinlock has a significant improvement on SOPHGO SG2042 64
> cores platform than the ticket_lock.
>
> Paravirt qspinlock
> ==================
>
> We implemented kvm_kick_cpu/kvm_wait_cpu and add tracepoints to observe the
> behaviors. Also, introduce a new SBI extension SBI_EXT_PVLOCK (0xAB0401). If the
> name and number are approved, I will send a formal proposal to the SBI spec.
>
Hello Guo Ren,
Any update on this series?
Thanks!
Leo
> Changlog:
> V11:
> - Based on Leonardo Bras's cmpxchg_small patches v5.
> - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> - Remove abusing alternative framework and use jump_label instead.
> - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> guarantee.
> - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
>
> V10:
> https://lore.kernel.org/linux-riscv/20230802164701.192791-1-guoren@kernel.org/
> - Using an alternative framework instead of static_key_branch in the
> asm/spinlock.h.
> - Fixup store merge buffer problem, which causes qspinlock lock
> torture test livelock.
> - Add paravirt qspinlock support, include KVM backend
> - Add Compact NUMA-awared qspinlock support
>
> V9:
> https://lore.kernel.org/linux-riscv/20220808071318.3335746-1-guoren@kernel.org/
> - Cleanup generic ticket-lock code, (Using smp_mb__after_spinlock as
> RCsc)
> - Add qspinlock and combo-lock for riscv
> - Add qspinlock to openrisc
> - Use generic header in csky
> - Optimize cmpxchg & atomic code
>
> V8:
> https://lore.kernel.org/linux-riscv/20220724122517.1019187-1-guoren@kernel.org/
> - Coding convention ticket fixup
> - Move combo spinlock into riscv and simply asm-generic/spinlock.h
> - Fixup xchg16 with wrong return value
> - Add csky qspinlock
> - Add combo & qspinlock & ticket-lock comparison
> - Clean up unnecessary riscv acquire and release definitions
> - Enable ARCH_INLINE_READ*/WRITE*/SPIN* for riscv & csky
>
> V7:
> https://lore.kernel.org/linux-riscv/20220628081946.1999419-1-guoren@kernel.org/
> - Add combo spinlock (ticket & queued) support
> - Rename ticket_spinlock.h
> - Remove unnecessary atomic_read in ticket_spin_value_unlocked
>
> V6:
> https://lore.kernel.org/linux-riscv/20220621144920.2945595-1-guoren@kernel.org/
> - Fixup Clang compile problem Reported-by: kernel test robot
> - Cleanup asm-generic/spinlock.h
> - Remove changelog in patch main comment part, suggested by
> Conor.Dooley
> - Remove "default y if NUMA" in Kconfig
>
> V5:
> https://lore.kernel.org/linux-riscv/20220620155404.1968739-1-guoren@kernel.org/
> - Update comment with RISC-V forward guarantee feature.
> - Back to V3 direction and optimize asm code.
>
> V4:
> https://lore.kernel.org/linux-riscv/1616868399-82848-4-git-send-email-guoren@kernel.org/
> - Remove custom sub-word xchg implementation
> - Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 in locking/qspinlock
>
> V3:
> https://lore.kernel.org/linux-riscv/1616658937-82063-1-git-send-email-guoren@kernel.org/
> - Coding convention by Peter Zijlstra's advices
>
> V2:
> https://lore.kernel.org/linux-riscv/1606225437-22948-2-git-send-email-guoren@kernel.org/
> - Coding convention in cmpxchg.h
> - Re-implement short xchg
> - Remove char & cmpxchg implementations
>
> V1:
> https://lore.kernel.org/linux-riscv/20190211043829.30096-1-michaeljclark@mac.com/
> - Using cmpxchg loop to implement sub-word atomic
>
>
> Guo Ren (17):
> asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
> asm-generic: ticket-lock: Move into ticket_spinlock.h
> riscv: Use Zicbop in arch_xchg when available
> locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
> riscv: qspinlock: Add basic queued_spinlock support
> riscv: qspinlock: Introduce combo spinlock
> riscv: qspinlock: Introduce qspinlock param for command line
> riscv: qspinlock: Add virt_spin_lock() support for KVM guest
> riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
> RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
> RISC-V: paravirt: pvqspinlock: Add SBI implementation
> RISC-V: paravirt: pvqspinlock: Add kconfig entry
> RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
> RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
> RISC-V: paravirt: pvqspinlock: KVM: Implement
> kvm_sbi_ext_pvlock_kick_cpu()
>
> .../admin-guide/kernel-parameters.txt | 8 +-
> arch/riscv/Kconfig | 50 ++++++++
> arch/riscv/Kconfig.errata | 19 +++
> arch/riscv/errata/thead/errata.c | 29 +++++
> arch/riscv/include/asm/Kbuild | 2 +-
> arch/riscv/include/asm/cmpxchg.h | 4 +-
> arch/riscv/include/asm/errata_list.h | 13 --
> arch/riscv/include/asm/hwcap.h | 1 +
> arch/riscv/include/asm/insn-def.h | 5 +
> arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
> arch/riscv/include/asm/processor.h | 13 ++
> arch/riscv/include/asm/qspinlock.h | 35 ++++++
> arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++
> arch/riscv/include/asm/rwonce.h | 24 ++++
> arch/riscv/include/asm/sbi.h | 14 +++
> arch/riscv/include/asm/spinlock.h | 113 ++++++++++++++++++
> arch/riscv/include/asm/vendorid_list.h | 14 +++
> arch/riscv/include/uapi/asm/kvm.h | 1 +
> arch/riscv/kernel/Makefile | 1 +
> arch/riscv/kernel/cpufeature.c | 1 +
> arch/riscv/kernel/qspinlock_paravirt.c | 83 +++++++++++++
> arch/riscv/kernel/sbi.c | 2 +-
> arch/riscv/kernel/setup.c | 60 ++++++++++
> .../kernel/trace_events_filter_paravirt.h | 60 ++++++++++
> arch/riscv/kvm/Makefile | 1 +
> arch/riscv/kvm/vcpu_sbi.c | 4 +
> arch/riscv/kvm/vcpu_sbi_pvlock.c | 57 +++++++++
> include/asm-generic/rwonce.h | 2 +
> include/asm-generic/spinlock.h | 87 +-------------
> include/asm-generic/spinlock_types.h | 12 +-
> include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++
> kernel/locking/qspinlock.c | 5 +-
> 32 files changed, 739 insertions(+), 114 deletions(-)
> create mode 100644 arch/riscv/include/asm/qspinlock.h
> create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> create mode 100644 arch/riscv/include/asm/rwonce.h
> create mode 100644 arch/riscv/include/asm/spinlock.h
> create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
> create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
> create mode 100644 include/asm-generic/ticket_spinlock.h
>
> --
> 2.36.1
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-11-06 20:42 ` Leonardo Bras
@ 2023-11-12 4:23 ` Guo Ren
-1 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-11-12 4:23 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Mon, Nov 6, 2023 at 3:42 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > patch[1 - 10]: Native qspinlock
> > patch[11 -17]: Paravirt qspinlock
> >
> > patch[4]: Add prefetchw in qspinlock's xchg_tail when cpus >= 16k
> >
> > This series based on:
> > - [RFC PATCH v5 0/5] Rework & improve riscv cmpxchg.h and atomic.h
> > https://lore.kernel.org/linux-riscv/20230810040349.92279-2-leobras@redhat.com/
> > - [PATCH V3] asm-generic: ticket-lock: Optimize arch_spin_value_unlocked
> > https://lore.kernel.org/linux-riscv/20230908154339.3250567-1-guoren@kernel.org/
> >
> > I merge them into sg2042-master branch, then you could directly try it on
> > sg2042 hardware platform:
> >
> > https://github.com/guoren83/linux/tree/sg2042-master-qspinlock-64ilp32_v5
> >
> > Use sophgo_mango_ubuntu_defconfig for sg2042 64/128 cores hardware
> > platform.
> >
> > Native qspinlock
> > ================
> >
> > This time we've proved the qspinlock on th1520 [1] & sg2042 [2], which
> > gives stability and performance improvement. All T-HEAD processors have
> > a strong LR/SC forward progress guarantee than the requirements of the
> > ISA, which could satisfy the xchg_tail of native_qspinlock. Now,
> > qspinlock has been run with us for more than 1 year, and we have enough
> > confidence to enable it for all the T-HEAD processors. Of causes, we
> > found a livelock problem with the qspinlock lock torture test from the
> > CPU store merge buffer delay mechanism, which caused the queued spinlock
> > becomes a dead ring and RCU warning to come out. We introduce a custom
> > WRITE_ONCE to solve this. Do we need explicit ISA instruction to signal
> > it? Or let hardware handle this.
> >
> > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> >
> > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > queued_spinlock 0.5109/0.00
> > ticket_spinlock 0.5814/0.00
> >
> > perf futex/hash (+6.7%):
> > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> >
> > perf futex/wake-parallel (+8.6%):
> > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> >
> > perf futex/requeue (+4.2%):
> > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> >
> > System Benchmarks (+6.4%)
> > queued_spinlock:
> > System Benchmarks Index Values BASELINE RESULT INDEX
> > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > Double-Precision Whetstone 55.0 182422.8 33167.8
> > Execl Throughput 43.0 13116.6 3050.4
> > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > Pipe Throughput 12440.0 23058600.5 18535.9
> > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > Process Creation 126.0 12537.3 995.0
> > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > System Call Overhead 15000.0 33308301.3 22205.5
> > ========
> > System Benchmarks Index Score 12426.1
> >
> > ticket_spinlock:
> > System Benchmarks Index Values BASELINE RESULT INDEX
> > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > Double-Precision Whetstone 55.0 181921.0 33076.5
> > Execl Throughput 43.0 12625.1 2936.1
> > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > Pipe Throughput 12440.0 20594018.7 16554.7
> > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > Process Creation 126.0 10798.4 857.0
> > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > System Call Overhead 15000.0 30766778.4 20511.2
> > ========
> > System Benchmarks Index Score 11670.7
> >
> > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > cores platform than the ticket_lock.
> >
> > Paravirt qspinlock
> > ==================
> >
> > We implemented kvm_kick_cpu/kvm_wait_cpu and add tracepoints to observe the
> > behaviors. Also, introduce a new SBI extension SBI_EXT_PVLOCK (0xAB0401). If the
> > name and number are approved, I will send a formal proposal to the SBI spec.
> >
>
> Hello Guo Ren,
>
> Any update on this series?
Found a nested virtualization problem, and I'm solving that. After
that, I'll update v12.
>
> Thanks!
> Leo
>
>
> > Changlog:
> > V11:
> > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > - Remove abusing alternative framework and use jump_label instead.
> > - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> > guarantee.
> > - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
> >
> > V10:
> > https://lore.kernel.org/linux-riscv/20230802164701.192791-1-guoren@kernel.org/
> > - Using an alternative framework instead of static_key_branch in the
> > asm/spinlock.h.
> > - Fixup store merge buffer problem, which causes qspinlock lock
> > torture test livelock.
> > - Add paravirt qspinlock support, include KVM backend
> > - Add Compact NUMA-awared qspinlock support
> >
> > V9:
> > https://lore.kernel.org/linux-riscv/20220808071318.3335746-1-guoren@kernel.org/
> > - Cleanup generic ticket-lock code, (Using smp_mb__after_spinlock as
> > RCsc)
> > - Add qspinlock and combo-lock for riscv
> > - Add qspinlock to openrisc
> > - Use generic header in csky
> > - Optimize cmpxchg & atomic code
> >
> > V8:
> > https://lore.kernel.org/linux-riscv/20220724122517.1019187-1-guoren@kernel.org/
> > - Coding convention ticket fixup
> > - Move combo spinlock into riscv and simply asm-generic/spinlock.h
> > - Fixup xchg16 with wrong return value
> > - Add csky qspinlock
> > - Add combo & qspinlock & ticket-lock comparison
> > - Clean up unnecessary riscv acquire and release definitions
> > - Enable ARCH_INLINE_READ*/WRITE*/SPIN* for riscv & csky
> >
> > V7:
> > https://lore.kernel.org/linux-riscv/20220628081946.1999419-1-guoren@kernel.org/
> > - Add combo spinlock (ticket & queued) support
> > - Rename ticket_spinlock.h
> > - Remove unnecessary atomic_read in ticket_spin_value_unlocked
> >
> > V6:
> > https://lore.kernel.org/linux-riscv/20220621144920.2945595-1-guoren@kernel.org/
> > - Fixup Clang compile problem Reported-by: kernel test robot
> > - Cleanup asm-generic/spinlock.h
> > - Remove changelog in patch main comment part, suggested by
> > Conor.Dooley
> > - Remove "default y if NUMA" in Kconfig
> >
> > V5:
> > https://lore.kernel.org/linux-riscv/20220620155404.1968739-1-guoren@kernel.org/
> > - Update comment with RISC-V forward guarantee feature.
> > - Back to V3 direction and optimize asm code.
> >
> > V4:
> > https://lore.kernel.org/linux-riscv/1616868399-82848-4-git-send-email-guoren@kernel.org/
> > - Remove custom sub-word xchg implementation
> > - Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 in locking/qspinlock
> >
> > V3:
> > https://lore.kernel.org/linux-riscv/1616658937-82063-1-git-send-email-guoren@kernel.org/
> > - Coding convention by Peter Zijlstra's advices
> >
> > V2:
> > https://lore.kernel.org/linux-riscv/1606225437-22948-2-git-send-email-guoren@kernel.org/
> > - Coding convention in cmpxchg.h
> > - Re-implement short xchg
> > - Remove char & cmpxchg implementations
> >
> > V1:
> > https://lore.kernel.org/linux-riscv/20190211043829.30096-1-michaeljclark@mac.com/
> > - Using cmpxchg loop to implement sub-word atomic
> >
> >
> > Guo Ren (17):
> > asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
> > asm-generic: ticket-lock: Move into ticket_spinlock.h
> > riscv: Use Zicbop in arch_xchg when available
> > locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
> > riscv: qspinlock: Add basic queued_spinlock support
> > riscv: qspinlock: Introduce combo spinlock
> > riscv: qspinlock: Introduce qspinlock param for command line
> > riscv: qspinlock: Add virt_spin_lock() support for KVM guest
> > riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
> > RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
> > RISC-V: paravirt: pvqspinlock: Add SBI implementation
> > RISC-V: paravirt: pvqspinlock: Add kconfig entry
> > RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
> > RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
> > RISC-V: paravirt: pvqspinlock: KVM: Implement
> > kvm_sbi_ext_pvlock_kick_cpu()
> >
> > .../admin-guide/kernel-parameters.txt | 8 +-
> > arch/riscv/Kconfig | 50 ++++++++
> > arch/riscv/Kconfig.errata | 19 +++
> > arch/riscv/errata/thead/errata.c | 29 +++++
> > arch/riscv/include/asm/Kbuild | 2 +-
> > arch/riscv/include/asm/cmpxchg.h | 4 +-
> > arch/riscv/include/asm/errata_list.h | 13 --
> > arch/riscv/include/asm/hwcap.h | 1 +
> > arch/riscv/include/asm/insn-def.h | 5 +
> > arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
> > arch/riscv/include/asm/processor.h | 13 ++
> > arch/riscv/include/asm/qspinlock.h | 35 ++++++
> > arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++
> > arch/riscv/include/asm/rwonce.h | 24 ++++
> > arch/riscv/include/asm/sbi.h | 14 +++
> > arch/riscv/include/asm/spinlock.h | 113 ++++++++++++++++++
> > arch/riscv/include/asm/vendorid_list.h | 14 +++
> > arch/riscv/include/uapi/asm/kvm.h | 1 +
> > arch/riscv/kernel/Makefile | 1 +
> > arch/riscv/kernel/cpufeature.c | 1 +
> > arch/riscv/kernel/qspinlock_paravirt.c | 83 +++++++++++++
> > arch/riscv/kernel/sbi.c | 2 +-
> > arch/riscv/kernel/setup.c | 60 ++++++++++
> > .../kernel/trace_events_filter_paravirt.h | 60 ++++++++++
> > arch/riscv/kvm/Makefile | 1 +
> > arch/riscv/kvm/vcpu_sbi.c | 4 +
> > arch/riscv/kvm/vcpu_sbi_pvlock.c | 57 +++++++++
> > include/asm-generic/rwonce.h | 2 +
> > include/asm-generic/spinlock.h | 87 +-------------
> > include/asm-generic/spinlock_types.h | 12 +-
> > include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++
> > kernel/locking/qspinlock.c | 5 +-
> > 32 files changed, 739 insertions(+), 114 deletions(-)
> > create mode 100644 arch/riscv/include/asm/qspinlock.h
> > create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> > create mode 100644 arch/riscv/include/asm/rwonce.h
> > create mode 100644 arch/riscv/include/asm/spinlock.h
> > create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> > create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
> > create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
> > create mode 100644 include/asm-generic/ticket_spinlock.h
> >
> > --
> > 2.36.1
> >
>
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-11-12 4:23 ` Guo Ren
0 siblings, 0 replies; 215+ messages in thread
From: Guo Ren @ 2023-11-12 4:23 UTC (permalink / raw)
To: Leonardo Bras
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Mon, Nov 6, 2023 at 3:42 PM Leonardo Bras <leobras@redhat.com> wrote:
>
> On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > patch[1 - 10]: Native qspinlock
> > patch[11 -17]: Paravirt qspinlock
> >
> > patch[4]: Add prefetchw in qspinlock's xchg_tail when cpus >= 16k
> >
> > This series based on:
> > - [RFC PATCH v5 0/5] Rework & improve riscv cmpxchg.h and atomic.h
> > https://lore.kernel.org/linux-riscv/20230810040349.92279-2-leobras@redhat.com/
> > - [PATCH V3] asm-generic: ticket-lock: Optimize arch_spin_value_unlocked
> > https://lore.kernel.org/linux-riscv/20230908154339.3250567-1-guoren@kernel.org/
> >
> > I merge them into sg2042-master branch, then you could directly try it on
> > sg2042 hardware platform:
> >
> > https://github.com/guoren83/linux/tree/sg2042-master-qspinlock-64ilp32_v5
> >
> > Use sophgo_mango_ubuntu_defconfig for sg2042 64/128 cores hardware
> > platform.
> >
> > Native qspinlock
> > ================
> >
> > This time we've proved the qspinlock on th1520 [1] & sg2042 [2], which
> > gives stability and performance improvement. All T-HEAD processors have
> > a strong LR/SC forward progress guarantee than the requirements of the
> > ISA, which could satisfy the xchg_tail of native_qspinlock. Now,
> > qspinlock has been run with us for more than 1 year, and we have enough
> > confidence to enable it for all the T-HEAD processors. Of causes, we
> > found a livelock problem with the qspinlock lock torture test from the
> > CPU store merge buffer delay mechanism, which caused the queued spinlock
> > becomes a dead ring and RCU warning to come out. We introduce a custom
> > WRITE_ONCE to solve this. Do we need explicit ISA instruction to signal
> > it? Or let hardware handle this.
> >
> > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> >
> > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > queued_spinlock 0.5109/0.00
> > ticket_spinlock 0.5814/0.00
> >
> > perf futex/hash (+6.7%):
> > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> >
> > perf futex/wake-parallel (+8.6%):
> > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> >
> > perf futex/requeue (+4.2%):
> > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> >
> > System Benchmarks (+6.4%)
> > queued_spinlock:
> > System Benchmarks Index Values BASELINE RESULT INDEX
> > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > Double-Precision Whetstone 55.0 182422.8 33167.8
> > Execl Throughput 43.0 13116.6 3050.4
> > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > Pipe Throughput 12440.0 23058600.5 18535.9
> > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > Process Creation 126.0 12537.3 995.0
> > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > System Call Overhead 15000.0 33308301.3 22205.5
> > ========
> > System Benchmarks Index Score 12426.1
> >
> > ticket_spinlock:
> > System Benchmarks Index Values BASELINE RESULT INDEX
> > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > Double-Precision Whetstone 55.0 181921.0 33076.5
> > Execl Throughput 43.0 12625.1 2936.1
> > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > Pipe Throughput 12440.0 20594018.7 16554.7
> > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > Process Creation 126.0 10798.4 857.0
> > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > System Call Overhead 15000.0 30766778.4 20511.2
> > ========
> > System Benchmarks Index Score 11670.7
> >
> > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > cores platform than the ticket_lock.
> >
> > Paravirt qspinlock
> > ==================
> >
> > We implemented kvm_kick_cpu/kvm_wait_cpu and add tracepoints to observe the
> > behaviors. Also, introduce a new SBI extension SBI_EXT_PVLOCK (0xAB0401). If the
> > name and number are approved, I will send a formal proposal to the SBI spec.
> >
>
> Hello Guo Ren,
>
> Any update on this series?
Found a nested virtualization problem, and I'm solving that. After
that, I'll update v12.
>
> Thanks!
> Leo
>
>
> > Changlog:
> > V11:
> > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > - Remove abusing alternative framework and use jump_label instead.
> > - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> > guarantee.
> > - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
> >
> > V10:
> > https://lore.kernel.org/linux-riscv/20230802164701.192791-1-guoren@kernel.org/
> > - Using an alternative framework instead of static_key_branch in the
> > asm/spinlock.h.
> > - Fixup store merge buffer problem, which causes qspinlock lock
> > torture test livelock.
> > - Add paravirt qspinlock support, include KVM backend
> > - Add Compact NUMA-awared qspinlock support
> >
> > V9:
> > https://lore.kernel.org/linux-riscv/20220808071318.3335746-1-guoren@kernel.org/
> > - Cleanup generic ticket-lock code, (Using smp_mb__after_spinlock as
> > RCsc)
> > - Add qspinlock and combo-lock for riscv
> > - Add qspinlock to openrisc
> > - Use generic header in csky
> > - Optimize cmpxchg & atomic code
> >
> > V8:
> > https://lore.kernel.org/linux-riscv/20220724122517.1019187-1-guoren@kernel.org/
> > - Coding convention ticket fixup
> > - Move combo spinlock into riscv and simply asm-generic/spinlock.h
> > - Fixup xchg16 with wrong return value
> > - Add csky qspinlock
> > - Add combo & qspinlock & ticket-lock comparison
> > - Clean up unnecessary riscv acquire and release definitions
> > - Enable ARCH_INLINE_READ*/WRITE*/SPIN* for riscv & csky
> >
> > V7:
> > https://lore.kernel.org/linux-riscv/20220628081946.1999419-1-guoren@kernel.org/
> > - Add combo spinlock (ticket & queued) support
> > - Rename ticket_spinlock.h
> > - Remove unnecessary atomic_read in ticket_spin_value_unlocked
> >
> > V6:
> > https://lore.kernel.org/linux-riscv/20220621144920.2945595-1-guoren@kernel.org/
> > - Fixup Clang compile problem Reported-by: kernel test robot
> > - Cleanup asm-generic/spinlock.h
> > - Remove changelog in patch main comment part, suggested by
> > Conor.Dooley
> > - Remove "default y if NUMA" in Kconfig
> >
> > V5:
> > https://lore.kernel.org/linux-riscv/20220620155404.1968739-1-guoren@kernel.org/
> > - Update comment with RISC-V forward guarantee feature.
> > - Back to V3 direction and optimize asm code.
> >
> > V4:
> > https://lore.kernel.org/linux-riscv/1616868399-82848-4-git-send-email-guoren@kernel.org/
> > - Remove custom sub-word xchg implementation
> > - Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 in locking/qspinlock
> >
> > V3:
> > https://lore.kernel.org/linux-riscv/1616658937-82063-1-git-send-email-guoren@kernel.org/
> > - Coding convention by Peter Zijlstra's advices
> >
> > V2:
> > https://lore.kernel.org/linux-riscv/1606225437-22948-2-git-send-email-guoren@kernel.org/
> > - Coding convention in cmpxchg.h
> > - Re-implement short xchg
> > - Remove char & cmpxchg implementations
> >
> > V1:
> > https://lore.kernel.org/linux-riscv/20190211043829.30096-1-michaeljclark@mac.com/
> > - Using cmpxchg loop to implement sub-word atomic
> >
> >
> > Guo Ren (17):
> > asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
> > asm-generic: ticket-lock: Move into ticket_spinlock.h
> > riscv: Use Zicbop in arch_xchg when available
> > locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
> > riscv: qspinlock: Add basic queued_spinlock support
> > riscv: qspinlock: Introduce combo spinlock
> > riscv: qspinlock: Introduce qspinlock param for command line
> > riscv: qspinlock: Add virt_spin_lock() support for KVM guest
> > riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
> > RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
> > RISC-V: paravirt: pvqspinlock: Add SBI implementation
> > RISC-V: paravirt: pvqspinlock: Add kconfig entry
> > RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
> > RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
> > RISC-V: paravirt: pvqspinlock: KVM: Implement
> > kvm_sbi_ext_pvlock_kick_cpu()
> >
> > .../admin-guide/kernel-parameters.txt | 8 +-
> > arch/riscv/Kconfig | 50 ++++++++
> > arch/riscv/Kconfig.errata | 19 +++
> > arch/riscv/errata/thead/errata.c | 29 +++++
> > arch/riscv/include/asm/Kbuild | 2 +-
> > arch/riscv/include/asm/cmpxchg.h | 4 +-
> > arch/riscv/include/asm/errata_list.h | 13 --
> > arch/riscv/include/asm/hwcap.h | 1 +
> > arch/riscv/include/asm/insn-def.h | 5 +
> > arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
> > arch/riscv/include/asm/processor.h | 13 ++
> > arch/riscv/include/asm/qspinlock.h | 35 ++++++
> > arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++
> > arch/riscv/include/asm/rwonce.h | 24 ++++
> > arch/riscv/include/asm/sbi.h | 14 +++
> > arch/riscv/include/asm/spinlock.h | 113 ++++++++++++++++++
> > arch/riscv/include/asm/vendorid_list.h | 14 +++
> > arch/riscv/include/uapi/asm/kvm.h | 1 +
> > arch/riscv/kernel/Makefile | 1 +
> > arch/riscv/kernel/cpufeature.c | 1 +
> > arch/riscv/kernel/qspinlock_paravirt.c | 83 +++++++++++++
> > arch/riscv/kernel/sbi.c | 2 +-
> > arch/riscv/kernel/setup.c | 60 ++++++++++
> > .../kernel/trace_events_filter_paravirt.h | 60 ++++++++++
> > arch/riscv/kvm/Makefile | 1 +
> > arch/riscv/kvm/vcpu_sbi.c | 4 +
> > arch/riscv/kvm/vcpu_sbi_pvlock.c | 57 +++++++++
> > include/asm-generic/rwonce.h | 2 +
> > include/asm-generic/spinlock.h | 87 +-------------
> > include/asm-generic/spinlock_types.h | 12 +-
> > include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++
> > kernel/locking/qspinlock.c | 5 +-
> > 32 files changed, 739 insertions(+), 114 deletions(-)
> > create mode 100644 arch/riscv/include/asm/qspinlock.h
> > create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> > create mode 100644 arch/riscv/include/asm/rwonce.h
> > create mode 100644 arch/riscv/include/asm/spinlock.h
> > create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> > create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
> > create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
> > create mode 100644 include/asm-generic/ticket_spinlock.h
> >
> > --
> > 2.36.1
> >
>
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
2023-11-12 4:23 ` Guo Ren
@ 2023-11-13 10:19 ` Leonardo Bras Soares Passos
-1 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras Soares Passos @ 2023-11-13 10:19 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Nov 12, 2023 at 1:24 AM Guo Ren <guoren@kernel.org> wrote:
>
> On Mon, Nov 6, 2023 at 3:42 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > patch[1 - 10]: Native qspinlock
> > > patch[11 -17]: Paravirt qspinlock
> > >
> > > patch[4]: Add prefetchw in qspinlock's xchg_tail when cpus >= 16k
> > >
> > > This series based on:
> > > - [RFC PATCH v5 0/5] Rework & improve riscv cmpxchg.h and atomic.h
> > > https://lore.kernel.org/linux-riscv/20230810040349.92279-2-leobras@redhat.com/
> > > - [PATCH V3] asm-generic: ticket-lock: Optimize arch_spin_value_unlocked
> > > https://lore.kernel.org/linux-riscv/20230908154339.3250567-1-guoren@kernel.org/
> > >
> > > I merge them into sg2042-master branch, then you could directly try it on
> > > sg2042 hardware platform:
> > >
> > > https://github.com/guoren83/linux/tree/sg2042-master-qspinlock-64ilp32_v5
> > >
> > > Use sophgo_mango_ubuntu_defconfig for sg2042 64/128 cores hardware
> > > platform.
> > >
> > > Native qspinlock
> > > ================
> > >
> > > This time we've proved the qspinlock on th1520 [1] & sg2042 [2], which
> > > gives stability and performance improvement. All T-HEAD processors have
> > > a strong LR/SC forward progress guarantee than the requirements of the
> > > ISA, which could satisfy the xchg_tail of native_qspinlock. Now,
> > > qspinlock has been run with us for more than 1 year, and we have enough
> > > confidence to enable it for all the T-HEAD processors. Of causes, we
> > > found a livelock problem with the qspinlock lock torture test from the
> > > CPU store merge buffer delay mechanism, which caused the queued spinlock
> > > becomes a dead ring and RCU warning to come out. We introduce a custom
> > > WRITE_ONCE to solve this. Do we need explicit ISA instruction to signal
> > > it? Or let hardware handle this.
> > >
> > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > >
> > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > queued_spinlock 0.5109/0.00
> > > ticket_spinlock 0.5814/0.00
> > >
> > > perf futex/hash (+6.7%):
> > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > >
> > > perf futex/wake-parallel (+8.6%):
> > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > >
> > > perf futex/requeue (+4.2%):
> > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > >
> > > System Benchmarks (+6.4%)
> > > queued_spinlock:
> > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > Execl Throughput 43.0 13116.6 3050.4
> > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > Process Creation 126.0 12537.3 995.0
> > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > System Call Overhead 15000.0 33308301.3 22205.5
> > > ========
> > > System Benchmarks Index Score 12426.1
> > >
> > > ticket_spinlock:
> > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > Execl Throughput 43.0 12625.1 2936.1
> > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > Process Creation 126.0 10798.4 857.0
> > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > System Call Overhead 15000.0 30766778.4 20511.2
> > > ========
> > > System Benchmarks Index Score 11670.7
> > >
> > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > cores platform than the ticket_lock.
> > >
> > > Paravirt qspinlock
> > > ==================
> > >
> > > We implemented kvm_kick_cpu/kvm_wait_cpu and add tracepoints to observe the
> > > behaviors. Also, introduce a new SBI extension SBI_EXT_PVLOCK (0xAB0401). If the
> > > name and number are approved, I will send a formal proposal to the SBI spec.
> > >
> >
> > Hello Guo Ren,
> >
> > Any update on this series?
> Found a nested virtualization problem, and I'm solving that. After
> that, I'll update v12.
Oh, nice to hear :)
I am very excited about this series, please let me know of any update.
Thanks!
Leo
>
> >
> > Thanks!
> > Leo
> >
> >
> > > Changlog:
> > > V11:
> > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > - Remove abusing alternative framework and use jump_label instead.
> > > - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> > > guarantee.
> > > - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
> > >
> > > V10:
> > > https://lore.kernel.org/linux-riscv/20230802164701.192791-1-guoren@kernel.org/
> > > - Using an alternative framework instead of static_key_branch in the
> > > asm/spinlock.h.
> > > - Fixup store merge buffer problem, which causes qspinlock lock
> > > torture test livelock.
> > > - Add paravirt qspinlock support, include KVM backend
> > > - Add Compact NUMA-awared qspinlock support
> > >
> > > V9:
> > > https://lore.kernel.org/linux-riscv/20220808071318.3335746-1-guoren@kernel.org/
> > > - Cleanup generic ticket-lock code, (Using smp_mb__after_spinlock as
> > > RCsc)
> > > - Add qspinlock and combo-lock for riscv
> > > - Add qspinlock to openrisc
> > > - Use generic header in csky
> > > - Optimize cmpxchg & atomic code
> > >
> > > V8:
> > > https://lore.kernel.org/linux-riscv/20220724122517.1019187-1-guoren@kernel.org/
> > > - Coding convention ticket fixup
> > > - Move combo spinlock into riscv and simply asm-generic/spinlock.h
> > > - Fixup xchg16 with wrong return value
> > > - Add csky qspinlock
> > > - Add combo & qspinlock & ticket-lock comparison
> > > - Clean up unnecessary riscv acquire and release definitions
> > > - Enable ARCH_INLINE_READ*/WRITE*/SPIN* for riscv & csky
> > >
> > > V7:
> > > https://lore.kernel.org/linux-riscv/20220628081946.1999419-1-guoren@kernel.org/
> > > - Add combo spinlock (ticket & queued) support
> > > - Rename ticket_spinlock.h
> > > - Remove unnecessary atomic_read in ticket_spin_value_unlocked
> > >
> > > V6:
> > > https://lore.kernel.org/linux-riscv/20220621144920.2945595-1-guoren@kernel.org/
> > > - Fixup Clang compile problem Reported-by: kernel test robot
> > > - Cleanup asm-generic/spinlock.h
> > > - Remove changelog in patch main comment part, suggested by
> > > Conor.Dooley
> > > - Remove "default y if NUMA" in Kconfig
> > >
> > > V5:
> > > https://lore.kernel.org/linux-riscv/20220620155404.1968739-1-guoren@kernel.org/
> > > - Update comment with RISC-V forward guarantee feature.
> > > - Back to V3 direction and optimize asm code.
> > >
> > > V4:
> > > https://lore.kernel.org/linux-riscv/1616868399-82848-4-git-send-email-guoren@kernel.org/
> > > - Remove custom sub-word xchg implementation
> > > - Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 in locking/qspinlock
> > >
> > > V3:
> > > https://lore.kernel.org/linux-riscv/1616658937-82063-1-git-send-email-guoren@kernel.org/
> > > - Coding convention by Peter Zijlstra's advices
> > >
> > > V2:
> > > https://lore.kernel.org/linux-riscv/1606225437-22948-2-git-send-email-guoren@kernel.org/
> > > - Coding convention in cmpxchg.h
> > > - Re-implement short xchg
> > > - Remove char & cmpxchg implementations
> > >
> > > V1:
> > > https://lore.kernel.org/linux-riscv/20190211043829.30096-1-michaeljclark@mac.com/
> > > - Using cmpxchg loop to implement sub-word atomic
> > >
> > >
> > > Guo Ren (17):
> > > asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
> > > asm-generic: ticket-lock: Move into ticket_spinlock.h
> > > riscv: Use Zicbop in arch_xchg when available
> > > locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
> > > riscv: qspinlock: Add basic queued_spinlock support
> > > riscv: qspinlock: Introduce combo spinlock
> > > riscv: qspinlock: Introduce qspinlock param for command line
> > > riscv: qspinlock: Add virt_spin_lock() support for KVM guest
> > > riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
> > > RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
> > > RISC-V: paravirt: pvqspinlock: Add SBI implementation
> > > RISC-V: paravirt: pvqspinlock: Add kconfig entry
> > > RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
> > > RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
> > > RISC-V: paravirt: pvqspinlock: KVM: Implement
> > > kvm_sbi_ext_pvlock_kick_cpu()
> > >
> > > .../admin-guide/kernel-parameters.txt | 8 +-
> > > arch/riscv/Kconfig | 50 ++++++++
> > > arch/riscv/Kconfig.errata | 19 +++
> > > arch/riscv/errata/thead/errata.c | 29 +++++
> > > arch/riscv/include/asm/Kbuild | 2 +-
> > > arch/riscv/include/asm/cmpxchg.h | 4 +-
> > > arch/riscv/include/asm/errata_list.h | 13 --
> > > arch/riscv/include/asm/hwcap.h | 1 +
> > > arch/riscv/include/asm/insn-def.h | 5 +
> > > arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
> > > arch/riscv/include/asm/processor.h | 13 ++
> > > arch/riscv/include/asm/qspinlock.h | 35 ++++++
> > > arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++
> > > arch/riscv/include/asm/rwonce.h | 24 ++++
> > > arch/riscv/include/asm/sbi.h | 14 +++
> > > arch/riscv/include/asm/spinlock.h | 113 ++++++++++++++++++
> > > arch/riscv/include/asm/vendorid_list.h | 14 +++
> > > arch/riscv/include/uapi/asm/kvm.h | 1 +
> > > arch/riscv/kernel/Makefile | 1 +
> > > arch/riscv/kernel/cpufeature.c | 1 +
> > > arch/riscv/kernel/qspinlock_paravirt.c | 83 +++++++++++++
> > > arch/riscv/kernel/sbi.c | 2 +-
> > > arch/riscv/kernel/setup.c | 60 ++++++++++
> > > .../kernel/trace_events_filter_paravirt.h | 60 ++++++++++
> > > arch/riscv/kvm/Makefile | 1 +
> > > arch/riscv/kvm/vcpu_sbi.c | 4 +
> > > arch/riscv/kvm/vcpu_sbi_pvlock.c | 57 +++++++++
> > > include/asm-generic/rwonce.h | 2 +
> > > include/asm-generic/spinlock.h | 87 +-------------
> > > include/asm-generic/spinlock_types.h | 12 +-
> > > include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++
> > > kernel/locking/qspinlock.c | 5 +-
> > > 32 files changed, 739 insertions(+), 114 deletions(-)
> > > create mode 100644 arch/riscv/include/asm/qspinlock.h
> > > create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> > > create mode 100644 arch/riscv/include/asm/rwonce.h
> > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > > create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> > > create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
> > > create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
> > > create mode 100644 include/asm-generic/ticket_spinlock.h
> > >
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 215+ messages in thread
* Re: [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support
@ 2023-11-13 10:19 ` Leonardo Bras Soares Passos
0 siblings, 0 replies; 215+ messages in thread
From: Leonardo Bras Soares Passos @ 2023-11-13 10:19 UTC (permalink / raw)
To: Guo Ren
Cc: paul.walmsley, anup, peterz, mingo, will, palmer, longman,
boqun.feng, tglx, paulmck, rostedt, rdunlap, catalin.marinas,
conor.dooley, xiaoguang.xing, bjorn, alexghiti, keescook,
greentime.hu, ajones, jszhang, wefu, wuwei2016, linux-arch,
linux-riscv, linux-doc, kvm, virtualization, linux-csky, Guo Ren
On Sun, Nov 12, 2023 at 1:24 AM Guo Ren <guoren@kernel.org> wrote:
>
> On Mon, Nov 6, 2023 at 3:42 PM Leonardo Bras <leobras@redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren@kernel.org wrote:
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > >
> > > patch[1 - 10]: Native qspinlock
> > > patch[11 -17]: Paravirt qspinlock
> > >
> > > patch[4]: Add prefetchw in qspinlock's xchg_tail when cpus >= 16k
> > >
> > > This series based on:
> > > - [RFC PATCH v5 0/5] Rework & improve riscv cmpxchg.h and atomic.h
> > > https://lore.kernel.org/linux-riscv/20230810040349.92279-2-leobras@redhat.com/
> > > - [PATCH V3] asm-generic: ticket-lock: Optimize arch_spin_value_unlocked
> > > https://lore.kernel.org/linux-riscv/20230908154339.3250567-1-guoren@kernel.org/
> > >
> > > I merge them into sg2042-master branch, then you could directly try it on
> > > sg2042 hardware platform:
> > >
> > > https://github.com/guoren83/linux/tree/sg2042-master-qspinlock-64ilp32_v5
> > >
> > > Use sophgo_mango_ubuntu_defconfig for sg2042 64/128 cores hardware
> > > platform.
> > >
> > > Native qspinlock
> > > ================
> > >
> > > This time we've proved the qspinlock on th1520 [1] & sg2042 [2], which
> > > gives stability and performance improvement. All T-HEAD processors have
> > > a strong LR/SC forward progress guarantee than the requirements of the
> > > ISA, which could satisfy the xchg_tail of native_qspinlock. Now,
> > > qspinlock has been run with us for more than 1 year, and we have enough
> > > confidence to enable it for all the T-HEAD processors. Of causes, we
> > > found a livelock problem with the qspinlock lock torture test from the
> > > CPU store merge buffer delay mechanism, which caused the queued spinlock
> > > becomes a dead ring and RCU warning to come out. We introduce a custom
> > > WRITE_ONCE to solve this. Do we need explicit ISA instruction to signal
> > > it? Or let hardware handle this.
> > >
> > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > >
> > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > > queued_spinlock 0.5109/0.00
> > > ticket_spinlock 0.5814/0.00
> > >
> > > perf futex/hash (+6.7%):
> > > queued_spinlock 1444393 operations/sec (+- 0.09%)
> > > ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > >
> > > perf futex/wake-parallel (+8.6%):
> > > queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > > ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > >
> > > perf futex/requeue (+4.2%):
> > > queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > > ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > >
> > > System Benchmarks (+6.4%)
> > > queued_spinlock:
> > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > Dhrystone 2 using register variables 116700.0 628613745.4 53865.8
> > > Double-Precision Whetstone 55.0 182422.8 33167.8
> > > Execl Throughput 43.0 13116.6 3050.4
> > > File Copy 1024 bufsize 2000 maxblocks 3960.0 7762306.2 19601.8
> > > File Copy 256 bufsize 500 maxblocks 1655.0 3417556.8 20649.9
> > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7427995.7 12806.9
> > > Pipe Throughput 12440.0 23058600.5 18535.9
> > > Pipe-based Context Switching 4000.0 2835617.7 7089.0
> > > Process Creation 126.0 12537.3 995.0
> > > Shell Scripts (1 concurrent) 42.4 57057.4 13456.9
> > > Shell Scripts (8 concurrent) 6.0 7367.1 12278.5
> > > System Call Overhead 15000.0 33308301.3 22205.5
> > > ========
> > > System Benchmarks Index Score 12426.1
> > >
> > > ticket_spinlock:
> > > System Benchmarks Index Values BASELINE RESULT INDEX
> > > Dhrystone 2 using register variables 116700.0 626541701.9 53688.2
> > > Double-Precision Whetstone 55.0 181921.0 33076.5
> > > Execl Throughput 43.0 12625.1 2936.1
> > > File Copy 1024 bufsize 2000 maxblocks 3960.0 6553792.9 16550.0
> > > File Copy 256 bufsize 500 maxblocks 1655.0 3189231.6 19270.3
> > > File Copy 4096 bufsize 8000 maxblocks 5800.0 7221277.0 12450.5
> > > Pipe Throughput 12440.0 20594018.7 16554.7
> > > Pipe-based Context Switching 4000.0 2571117.7 6427.8
> > > Process Creation 126.0 10798.4 857.0
> > > Shell Scripts (1 concurrent) 42.4 57227.5 13497.1
> > > Shell Scripts (8 concurrent) 6.0 7329.2 12215.3
> > > System Call Overhead 15000.0 30766778.4 20511.2
> > > ========
> > > System Benchmarks Index Score 11670.7
> > >
> > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > cores platform than the ticket_lock.
> > >
> > > Paravirt qspinlock
> > > ==================
> > >
> > > We implemented kvm_kick_cpu/kvm_wait_cpu and add tracepoints to observe the
> > > behaviors. Also, introduce a new SBI extension SBI_EXT_PVLOCK (0xAB0401). If the
> > > name and number are approved, I will send a formal proposal to the SBI spec.
> > >
> >
> > Hello Guo Ren,
> >
> > Any update on this series?
> Found a nested virtualization problem, and I'm solving that. After
> that, I'll update v12.
Oh, nice to hear :)
I am very excited about this series, please let me know of any update.
Thanks!
Leo
>
> >
> > Thanks!
> > Leo
> >
> >
> > > Changlog:
> > > V11:
> > > - Based on Leonardo Bras's cmpxchg_small patches v5.
> > > - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > > - Remove abusing alternative framework and use jump_label instead.
> > > - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> > > guarantee.
> > > - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
> > >
> > > V10:
> > > https://lore.kernel.org/linux-riscv/20230802164701.192791-1-guoren@kernel.org/
> > > - Using an alternative framework instead of static_key_branch in the
> > > asm/spinlock.h.
> > > - Fixup store merge buffer problem, which causes qspinlock lock
> > > torture test livelock.
> > > - Add paravirt qspinlock support, include KVM backend
> > > - Add Compact NUMA-awared qspinlock support
> > >
> > > V9:
> > > https://lore.kernel.org/linux-riscv/20220808071318.3335746-1-guoren@kernel.org/
> > > - Cleanup generic ticket-lock code, (Using smp_mb__after_spinlock as
> > > RCsc)
> > > - Add qspinlock and combo-lock for riscv
> > > - Add qspinlock to openrisc
> > > - Use generic header in csky
> > > - Optimize cmpxchg & atomic code
> > >
> > > V8:
> > > https://lore.kernel.org/linux-riscv/20220724122517.1019187-1-guoren@kernel.org/
> > > - Coding convention ticket fixup
> > > - Move combo spinlock into riscv and simply asm-generic/spinlock.h
> > > - Fixup xchg16 with wrong return value
> > > - Add csky qspinlock
> > > - Add combo & qspinlock & ticket-lock comparison
> > > - Clean up unnecessary riscv acquire and release definitions
> > > - Enable ARCH_INLINE_READ*/WRITE*/SPIN* for riscv & csky
> > >
> > > V7:
> > > https://lore.kernel.org/linux-riscv/20220628081946.1999419-1-guoren@kernel.org/
> > > - Add combo spinlock (ticket & queued) support
> > > - Rename ticket_spinlock.h
> > > - Remove unnecessary atomic_read in ticket_spin_value_unlocked
> > >
> > > V6:
> > > https://lore.kernel.org/linux-riscv/20220621144920.2945595-1-guoren@kernel.org/
> > > - Fixup Clang compile problem Reported-by: kernel test robot
> > > - Cleanup asm-generic/spinlock.h
> > > - Remove changelog in patch main comment part, suggested by
> > > Conor.Dooley
> > > - Remove "default y if NUMA" in Kconfig
> > >
> > > V5:
> > > https://lore.kernel.org/linux-riscv/20220620155404.1968739-1-guoren@kernel.org/
> > > - Update comment with RISC-V forward guarantee feature.
> > > - Back to V3 direction and optimize asm code.
> > >
> > > V4:
> > > https://lore.kernel.org/linux-riscv/1616868399-82848-4-git-send-email-guoren@kernel.org/
> > > - Remove custom sub-word xchg implementation
> > > - Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 in locking/qspinlock
> > >
> > > V3:
> > > https://lore.kernel.org/linux-riscv/1616658937-82063-1-git-send-email-guoren@kernel.org/
> > > - Coding convention by Peter Zijlstra's advices
> > >
> > > V2:
> > > https://lore.kernel.org/linux-riscv/1606225437-22948-2-git-send-email-guoren@kernel.org/
> > > - Coding convention in cmpxchg.h
> > > - Re-implement short xchg
> > > - Remove char & cmpxchg implementations
> > >
> > > V1:
> > > https://lore.kernel.org/linux-riscv/20190211043829.30096-1-michaeljclark@mac.com/
> > > - Using cmpxchg loop to implement sub-word atomic
> > >
> > >
> > > Guo Ren (17):
> > > asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
> > > asm-generic: ticket-lock: Move into ticket_spinlock.h
> > > riscv: Use Zicbop in arch_xchg when available
> > > locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
> > > riscv: qspinlock: Add basic queued_spinlock support
> > > riscv: qspinlock: Introduce combo spinlock
> > > riscv: qspinlock: Introduce qspinlock param for command line
> > > riscv: qspinlock: Add virt_spin_lock() support for KVM guest
> > > riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > > riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > > RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
> > > RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
> > > RISC-V: paravirt: pvqspinlock: Add SBI implementation
> > > RISC-V: paravirt: pvqspinlock: Add kconfig entry
> > > RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
> > > RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
> > > RISC-V: paravirt: pvqspinlock: KVM: Implement
> > > kvm_sbi_ext_pvlock_kick_cpu()
> > >
> > > .../admin-guide/kernel-parameters.txt | 8 +-
> > > arch/riscv/Kconfig | 50 ++++++++
> > > arch/riscv/Kconfig.errata | 19 +++
> > > arch/riscv/errata/thead/errata.c | 29 +++++
> > > arch/riscv/include/asm/Kbuild | 2 +-
> > > arch/riscv/include/asm/cmpxchg.h | 4 +-
> > > arch/riscv/include/asm/errata_list.h | 13 --
> > > arch/riscv/include/asm/hwcap.h | 1 +
> > > arch/riscv/include/asm/insn-def.h | 5 +
> > > arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 +
> > > arch/riscv/include/asm/processor.h | 13 ++
> > > arch/riscv/include/asm/qspinlock.h | 35 ++++++
> > > arch/riscv/include/asm/qspinlock_paravirt.h | 29 +++++
> > > arch/riscv/include/asm/rwonce.h | 24 ++++
> > > arch/riscv/include/asm/sbi.h | 14 +++
> > > arch/riscv/include/asm/spinlock.h | 113 ++++++++++++++++++
> > > arch/riscv/include/asm/vendorid_list.h | 14 +++
> > > arch/riscv/include/uapi/asm/kvm.h | 1 +
> > > arch/riscv/kernel/Makefile | 1 +
> > > arch/riscv/kernel/cpufeature.c | 1 +
> > > arch/riscv/kernel/qspinlock_paravirt.c | 83 +++++++++++++
> > > arch/riscv/kernel/sbi.c | 2 +-
> > > arch/riscv/kernel/setup.c | 60 ++++++++++
> > > .../kernel/trace_events_filter_paravirt.h | 60 ++++++++++
> > > arch/riscv/kvm/Makefile | 1 +
> > > arch/riscv/kvm/vcpu_sbi.c | 4 +
> > > arch/riscv/kvm/vcpu_sbi_pvlock.c | 57 +++++++++
> > > include/asm-generic/rwonce.h | 2 +
> > > include/asm-generic/spinlock.h | 87 +-------------
> > > include/asm-generic/spinlock_types.h | 12 +-
> > > include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++
> > > kernel/locking/qspinlock.c | 5 +-
> > > 32 files changed, 739 insertions(+), 114 deletions(-)
> > > create mode 100644 arch/riscv/include/asm/qspinlock.h
> > > create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> > > create mode 100644 arch/riscv/include/asm/rwonce.h
> > > create mode 100644 arch/riscv/include/asm/spinlock.h
> > > create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> > > create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
> > > create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
> > > create mode 100644 include/asm-generic/ticket_spinlock.h
> > >
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
> Guo Ren
>
^ permalink raw reply [flat|nested] 215+ messages in thread
* [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available
@ 2023-09-10 8:28 ` guoren
0 siblings, 0 replies; 215+ messages in thread
From: guoren @ 2023-12-31 8:29 UTC (permalink / raw)
To: paul.walmsley, palmer, guoren, panqinglin2020, bjorn,
conor.dooley, leobras, peterz, keescook, wuwei2016,
xiaoguang.xing, chao.wei, unicorn_wang, uwu, jszhang, wefu,
atishp, ajones, anup, mingo, will, palmer, longman, boqun.feng,
tglx, paulmck, rostedt, rdunlap, catalin.marinas, alexghiti,
greentime.hu
Cc: linux-riscv, linux-kernel, linux-arch, linux-doc, kvm,
virtualization, linux-csky, Guo Ren
From: Guo Ren <guoren@linux.alibaba.com>
Cache-block prefetch instructions are HINTs to the hardware to
indicate that software intends to perform a particular type of
memory access in the near future. Enable ARCH_HAS_PREFETCHW and
improve the arch_xchg for qspinlock xchg_tail.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/Kconfig | 15 +++++++++++++++
arch/riscv/include/asm/cmpxchg.h | 4 +++-
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/insn-def.h | 5 +++++
arch/riscv/include/asm/processor.h | 13 +++++++++++++
arch/riscv/kernel/cpufeature.c | 1 +
6 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index e9ae6fa232c3..2c346fe169c1 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -617,6 +617,21 @@ config RISCV_ISA_ZICBOZ
If you don't know what to do here, say Y.
+config RISCV_ISA_ZICBOP
+ bool "Zicbop extension support for cache block prefetch"
+ depends on MMU
+ depends on RISCV_ALTERNATIVE
+ default y
+ help
+ Adds support to dynamically detect the presence of the ZICBOP
+ extension (Cache Block Prefetch Operations) and enable its
+ usage.
+
+ The Zicbop extension can be used to prefetch cache block for
+ read/write/instruction fetch.
+
+ If you don't know what to do here, say Y.
+
config TOOLCHAIN_HAS_ZIHINTPAUSE
bool
default y
diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
index 702725727671..56eff7a9d2d2 100644
--- a/arch/riscv/include/asm/cmpxchg.h
+++ b/arch/riscv/include/asm/cmpxchg.h
@@ -11,6 +11,7 @@
#include <asm/barrier.h>
#include <asm/fence.h>
+#include <asm/processor.h>
#define __arch_xchg_masked(prepend, append, r, p, n) \
({ \
@@ -25,6 +26,7 @@
\
__asm__ __volatile__ ( \
prepend \
+ PREFETCHW_ASM(%5) \
"0: lr.w %0, %2\n" \
" and %1, %0, %z4\n" \
" or %1, %1, %z3\n" \
@@ -32,7 +34,7 @@
" bnez %1, 0b\n" \
append \
: "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
- : "rJ" (__newx), "rJ" (~__mask) \
+ : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
: "memory"); \
\
r = (__typeof__(*(p)))((__retx & __mask) >> __s); \
diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
index b7b58258f6c7..78b7b8b53778 100644
--- a/arch/riscv/include/asm/hwcap.h
+++ b/arch/riscv/include/asm/hwcap.h
@@ -58,6 +58,7 @@
#define RISCV_ISA_EXT_ZICSR 40
#define RISCV_ISA_EXT_ZIFENCEI 41
#define RISCV_ISA_EXT_ZIHPM 42
+#define RISCV_ISA_EXT_ZICBOP 43
#define RISCV_ISA_EXT_MAX 64
diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
index 6960beb75f32..dc590d331894 100644
--- a/arch/riscv/include/asm/insn-def.h
+++ b/arch/riscv/include/asm/insn-def.h
@@ -134,6 +134,7 @@
#define RV_OPCODE_MISC_MEM RV_OPCODE(15)
#define RV_OPCODE_SYSTEM RV_OPCODE(115)
+#define RV_OPCODE_PREFETCH RV_OPCODE(19)
#define HFENCE_VVMA(vaddr, asid) \
INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(17), \
@@ -196,4 +197,8 @@
INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \
RS1(base), SIMM12(4))
+#define CBO_prefetchw(base) \
+ INSN_R(OPCODE_PREFETCH, FUNC3(6), FUNC7(0), \
+ RD(x0), RS1(base), RS2(x0))
+
#endif /* __ASM_INSN_DEF_H */
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index de9da852f78d..7ad3a24212e8 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -12,6 +12,8 @@
#include <vdso/processor.h>
#include <asm/ptrace.h>
+#include <asm/insn-def.h>
+#include <asm/hwcap.h>
#ifdef CONFIG_64BIT
#define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1))
@@ -103,6 +105,17 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,
#define KSTK_EIP(tsk) (ulong)(task_pt_regs(tsk)->epc)
#define KSTK_ESP(tsk) (ulong)(task_pt_regs(tsk)->sp)
+#define ARCH_HAS_PREFETCHW
+#define PREFETCHW_ASM(base) ALTERNATIVE(__nops(1), \
+ CBO_prefetchw(base), \
+ 0, \
+ RISCV_ISA_EXT_ZICBOP, \
+ CONFIG_RISCV_ISA_ZICBOP)
+static inline void prefetchw(const void *ptr)
+{
+ asm volatile(PREFETCHW_ASM(%0)
+ : : "r" (ptr) : "memory");
+}
/* Do necessary setup to start up a newly executed thread. */
extern void start_thread(struct pt_regs *regs,
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index ef7b4fd9e876..e0b897db0b97 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -159,6 +159,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
__RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h),
__RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM),
__RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ),
+ __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP),
__RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR),
__RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR),
__RISCV_ISA_EXT_DATA(zifencei, RISCV_ISA_EXT_ZIFENCEI),
--
2.36.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 215+ messages in thread
end of thread, other threads:[~2023-12-31 8:30 UTC | newest]
Thread overview: 215+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-09-10 8:28 [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support guoren
2023-09-10 8:28 ` guoren
2023-09-10 8:28 ` [PATCH V11 01/17] asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock guoren
2023-09-10 8:28 ` guoren
2023-09-11 19:05 ` Leonardo Brás
2023-09-11 19:05 ` Leonardo Brás
2023-09-13 1:55 ` Guo Ren
2023-09-13 1:55 ` Guo Ren
2023-09-13 7:59 ` Leonardo Bras
2023-09-13 7:59 ` Leonardo Bras
2023-09-10 8:28 ` [PATCH V11 02/17] asm-generic: ticket-lock: Move into ticket_spinlock.h guoren
2023-09-10 8:28 ` guoren
2023-09-13 8:15 ` Leonardo Bras
2023-09-13 8:15 ` Leonardo Bras
2023-09-10 8:28 ` [PATCH V11 03/17] riscv: Use Zicbop in arch_xchg when available guoren
2023-12-31 8:29 ` guoren
2023-09-10 8:28 ` guoren
2023-09-13 8:49 ` Leonardo Bras
2023-09-13 8:49 ` Leonardo Bras
2023-09-15 12:36 ` Guo Ren
2023-09-15 12:36 ` Guo Ren
2023-09-16 1:25 ` Leonardo Bras
2023-09-16 1:25 ` Leonardo Bras
2023-09-17 14:34 ` Guo Ren
2023-09-17 14:34 ` Guo Ren
2023-09-19 5:13 ` Leonardo Bras
2023-09-19 5:13 ` Leonardo Bras
2023-09-19 7:53 ` Guo Ren
2023-09-19 7:53 ` Guo Ren
2023-09-19 14:38 ` Leonardo Bras
2023-09-19 14:38 ` Leonardo Bras
2023-09-14 13:47 ` Andrew Jones
2023-09-14 13:47 ` Andrew Jones
2023-09-15 8:22 ` Leonardo Bras
2023-09-15 8:22 ` Leonardo Bras
2023-09-15 11:07 ` Andrew Jones
2023-09-15 11:07 ` Andrew Jones
2023-09-15 11:26 ` Conor Dooley
2023-09-15 11:26 ` Conor Dooley
2023-09-15 12:22 ` Andrew Jones
2023-09-15 12:22 ` Andrew Jones
2023-09-15 12:42 ` Conor Dooley
2023-09-15 12:42 ` Conor Dooley
2023-09-16 0:05 ` Conor Dooley
2023-09-16 0:05 ` Conor Dooley
2023-09-15 20:32 ` Leonardo Bras
2023-09-15 20:32 ` Leonardo Bras
2023-09-14 14:25 ` Andrew Jones
2023-09-14 14:25 ` Andrew Jones
2023-09-14 14:47 ` Andrew Jones
2023-09-14 14:47 ` Andrew Jones
2023-09-15 11:37 ` Conor Dooley
2023-09-15 11:37 ` Conor Dooley
2023-09-15 12:14 ` Andrew Jones
2023-09-15 12:14 ` Andrew Jones
2023-09-15 12:53 ` Conor Dooley
2023-09-15 12:53 ` Conor Dooley
2023-09-10 8:28 ` [PATCH V11 04/17] locking/qspinlock: Improve xchg_tail for number of cpus >= 16k guoren
2023-09-10 8:28 ` guoren
2023-09-11 2:35 ` Waiman Long
2023-09-11 2:35 ` Waiman Long
2023-09-11 2:35 ` Waiman Long
2023-09-11 3:09 ` Guo Ren
2023-09-11 3:09 ` Guo Ren
2023-09-11 13:03 ` Waiman Long
2023-09-11 13:03 ` Waiman Long
2023-09-11 13:03 ` Waiman Long
2023-09-12 1:10 ` Guo Ren
2023-09-12 1:10 ` Guo Ren
2023-09-13 8:55 ` Leonardo Bras
2023-09-13 8:55 ` Leonardo Bras
2023-09-13 12:52 ` Guo Ren
2023-09-13 12:52 ` Guo Ren
2023-09-13 13:06 ` Waiman Long
2023-09-13 13:06 ` Waiman Long
2023-09-13 13:06 ` Waiman Long
2023-09-14 3:45 ` Guo Ren
2023-09-14 3:45 ` Guo Ren
2023-09-10 8:28 ` [PATCH V11 05/17] riscv: qspinlock: Add basic queued_spinlock support guoren
2023-09-10 8:28 ` guoren
2023-09-13 20:28 ` Leonardo Bras
2023-09-13 20:28 ` Leonardo Bras
2023-09-14 4:46 ` Guo Ren
2023-09-14 4:46 ` Guo Ren
2023-09-14 9:43 ` Leonardo Bras
2023-09-14 9:43 ` Leonardo Bras
2023-09-15 2:10 ` Guo Ren
2023-09-15 2:10 ` Guo Ren
2023-09-15 9:08 ` Leonardo Bras
2023-09-15 9:08 ` Leonardo Bras
2023-09-17 15:02 ` Guo Ren
2023-09-17 15:02 ` Guo Ren
2023-09-19 5:20 ` Leonardo Bras
2023-09-19 5:20 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 06/17] riscv: qspinlock: Introduce combo spinlock guoren
2023-09-10 8:29 ` guoren
2023-09-10 11:06 ` Guo Ren
2023-09-10 11:06 ` Guo Ren
2023-09-13 20:37 ` Leonardo Bras
2023-09-13 20:37 ` Leonardo Bras
2023-09-13 20:49 ` Leonardo Bras
2023-09-13 20:49 ` Leonardo Bras
2023-09-14 4:49 ` Guo Ren
2023-09-14 4:49 ` Guo Ren
2023-09-14 7:17 ` Leonardo Bras
2023-09-14 7:17 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 07/17] riscv: qspinlock: Introduce qspinlock param for command line guoren
2023-09-10 8:29 ` guoren
2023-09-11 15:22 ` Waiman Long
2023-09-11 15:22 ` Waiman Long
2023-09-11 15:22 ` Waiman Long
2023-09-12 1:06 ` Guo Ren
2023-09-12 1:06 ` Guo Ren
2023-09-11 15:34 ` Waiman Long
2023-09-11 15:34 ` Waiman Long
2023-09-11 15:34 ` Waiman Long
2023-09-12 1:08 ` Guo Ren
2023-09-12 1:08 ` Guo Ren
2023-09-14 7:32 ` Leonardo Bras
2023-09-14 7:32 ` Leonardo Bras
2023-09-14 17:23 ` Waiman Long
2023-09-14 17:23 ` Waiman Long
2023-09-14 17:23 ` Waiman Long
2023-09-10 8:29 ` [PATCH V11 08/17] riscv: qspinlock: Add virt_spin_lock() support for KVM guest guoren
2023-09-10 8:29 ` guoren
2023-09-14 8:02 ` Leonardo Bras
2023-09-14 8:02 ` Leonardo Bras
2023-09-17 15:12 ` Guo Ren
2023-09-17 15:12 ` Guo Ren
2023-09-19 5:30 ` Leonardo Bras
2023-09-19 5:30 ` Leonardo Bras
2023-09-19 8:04 ` Guo Ren
2023-09-19 8:04 ` Guo Ren
2023-09-19 14:40 ` Leonardo Bras
2023-09-19 14:40 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 09/17] riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup guoren
2023-09-10 8:29 ` guoren
2023-09-14 8:32 ` Leonardo Bras
2023-09-14 8:32 ` Leonardo Bras
2023-09-17 15:15 ` Guo Ren
2023-09-17 15:15 ` Guo Ren
2023-09-19 5:34 ` Leonardo Bras
2023-09-19 5:34 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 10/17] riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors guoren
2023-09-10 8:29 ` guoren
2023-09-14 9:36 ` Leonardo Bras
2023-09-14 9:36 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 11/17] RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton guoren
2023-09-10 8:29 ` guoren
2023-09-15 5:42 ` Leonardo Bras
2023-09-15 5:42 ` Leonardo Bras
2023-09-17 14:58 ` Guo Ren
2023-09-17 14:58 ` Guo Ren
2023-09-19 5:43 ` Leonardo Bras
2023-09-19 5:43 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 12/17] RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter guoren
2023-09-10 8:29 ` guoren
2023-09-15 6:05 ` Leonardo Bras
2023-09-15 6:05 ` Leonardo Bras
2023-09-17 15:03 ` Guo Ren
2023-09-17 15:03 ` Guo Ren
2023-09-19 5:44 ` Leonardo Bras
2023-09-19 5:44 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 13/17] RISC-V: paravirt: pvqspinlock: Add SBI implementation guoren
2023-09-10 8:29 ` guoren
2023-09-15 6:23 ` Leonardo Bras
2023-09-15 6:23 ` Leonardo Bras
2023-09-17 15:06 ` Guo Ren
2023-09-17 15:06 ` Guo Ren
2023-09-19 5:45 ` Leonardo Bras
2023-09-19 5:45 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 14/17] RISC-V: paravirt: pvqspinlock: Add kconfig entry guoren
2023-09-10 8:29 ` guoren
2023-09-15 6:25 ` Leonardo Bras
2023-09-15 6:25 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 15/17] RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait guoren
2023-09-10 8:29 ` guoren
2023-09-15 6:33 ` Leonardo Bras
2023-09-15 6:33 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 16/17] RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton guoren
2023-09-10 8:29 ` guoren
2023-09-15 6:46 ` Leonardo Bras
2023-09-15 6:46 ` Leonardo Bras
2023-09-10 8:29 ` [PATCH V11 17/17] RISC-V: paravirt: pvqspinlock: KVM: Implement kvm_sbi_ext_pvlock_kick_cpu() guoren
2023-09-10 8:29 ` guoren
2023-09-15 6:52 ` Leonardo Bras
2023-09-15 6:52 ` Leonardo Bras
2023-09-10 8:58 ` [PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support Conor Dooley
2023-09-10 8:58 ` Conor Dooley
2023-09-10 9:16 ` Guo Ren
2023-09-10 9:16 ` Guo Ren
2023-09-10 9:20 ` Guo Ren
2023-09-10 9:20 ` Guo Ren
2023-09-10 9:31 ` Conor Dooley
2023-09-10 9:31 ` Conor Dooley
2023-09-10 9:49 ` Guo Ren
2023-09-10 9:49 ` Guo Ren
2023-09-10 19:45 ` Conor Dooley
2023-09-10 19:45 ` Conor Dooley
2023-09-11 3:36 ` Guo Ren
2023-09-11 3:36 ` Guo Ren
2023-09-11 12:52 ` Conor Dooley
2023-09-11 12:52 ` Conor Dooley
2023-09-12 1:33 ` Guo Ren
2023-09-12 1:33 ` Guo Ren
2023-09-12 8:07 ` Conor Dooley
2023-09-12 8:07 ` Conor Dooley
2023-09-12 10:58 ` Guo Ren
2023-09-12 10:58 ` Guo Ren
2023-11-06 20:42 ` Leonardo Bras
2023-11-06 20:42 ` Leonardo Bras
2023-11-12 4:23 ` Guo Ren
2023-11-12 4:23 ` Guo Ren
2023-11-13 10:19 ` Leonardo Bras Soares Passos
2023-11-13 10:19 ` Leonardo Bras Soares Passos
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.