Re: [PATCH v8 00/11] Add support for SBI v0.2 and CPU hotplug

[Atish Patra <Atish.Patra@wdc.com>]

On Wed, 2020-02-19 at 13:48 -0800, Palmer Dabbelt wrote:
> On Tue, 11 Feb 2020 17:48:11 PST (-0800), Atish Patra wrote:
> > The Supervisor Binary Interface(SBI) specification[1] now defines a
> > base extension that provides extendability to add future extensions
> > while maintaining backward compatibility with previous versions.
> > The new version is defined as 0.2 and older version is marked as
> > 0.1.
> > 
> > This series adds support v0.2 and a unified calling convention
> > implementation between 0.1 and 0.2. It also add other SBI v0.2
> > functionality defined in [2]. The base support for SBI v0.2 is
> > already
> > available in OpenSBI v0.5. It also adds SBI HSM extension and cpu-
> > hotplug
> > support for RISC-V which requires additional patches[3] in OpenSBI.
> 
> Now that 0.2-rc1 has been tagged we should really start to get this
> into shape
> to merge this.  My biggest worry is being able to put together a
> kernel that
> can boot on both 0.1 and 0.2 SBIs, with the hart lottery being my
> major worry
> there.  I just skimmed this, but I was expected to see something
> like 
> 
>     diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
>     index 271860fc2c3f..849ba75959ba 100644
>     --- a/arch/riscv/kernel/head.S
>     +++ b/arch/riscv/kernel/head.S
>     @@ -187,6 +187,7 @@ relocate:
>      	la a3, .Lsecondary_park
>      	csrw CSR_TVEC, a3
>      
>     +#ifdef CONFIG_SBI_V01
>      	slli a3, a0, LGREG
>      	la a1, __cpu_up_stack_pointer
>      	la a2, __cpu_up_task_pointer
>     @@ -212,7 +213,10 @@ relocate:
>      #endif
>      
>      	tail smp_callin
>     -#endif
>     +#else /* !CONFIG_SBI_V01 */
>     +	pr_warn("multiple harts booted an SBI v0.2+ only kernel");
>     +#endif /* CONFIG_SBI_V01 */
>     +#endif /* CONFIG_SMP */
>      
>      END(_start)
>  
> but I don't.  Is there something else doing this?
> 

Yes. Current series ensures that it boots both 0.1 and 0.2 SBIs.  It
is done based on dynamic detection of HSM extension and cpu_ops. You
can take a look at the patch 9.

+#if IS_ENABLED(CONFIG_RISCV_SBI)
+     if (sbi_probe_extension(SBI_EXT_HSM) > 0) {
+     if (!cpuid)
+          pr_info("SBI v0.2 HSM extension detected\n");
+     cpu_ops[cpuid] = &cpu_ops_sbi;
+ } else
+#endif
+    cpu_ops[cpuid] = &cpu_ops_spinwait;

https://patchwork.kernel.org/patch/11377359/

For SBI v0.1, probing will fail for HSM extension and cpu_ops_spinwait
will be set as th cpu_ops.


> > [1] 
> > https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.adoc
> > [2] https://github.com/riscv/riscv-sbi-doc/pull/27
> > [3] 
> > http://lists.infradead.org/pipermail/opensbi/2020-January/001050.html
> > 
> > The patches are also available in following github repositery.
> > 
> > OpenSBI     : https://github.com/atishp04/opensbi/tree/sbi_hsm_v1
> > Linux Kernel: https://github.com/atishp04/linux/tree/sbi_v0.2_v8
> > 
> > Changes from v7->v8:
> > 1. Refactored to code to have modular cpu_ops calls.
> > 2. Refactored HSM extension from sbi.c to cpu_ops_sbi.c.
> > 3. Fix plic driver to handle cpu hotplug.
> > 
> > Changes from v6->v7:
> > 1. Rebased on v5.5
> > 2. Fixed few compilation issues for !CONFIG_SMP and
> > !CONFIG_RISCV_SBI
> > 3. Added SBI HSM extension
> > 4. Add CPU hotplug support
> > 
> > Changes from v5->v6
> > 1. Fixed few compilation issues around config.
> > 2. Fixed hart mask generation issues for RFENCE & IPI extensions.
> > 
> > Changes from v4->v5
> > 1. Fixed few minor comments related to static & inline.
> > 2. Make sure that every patch is boot tested individually.
> > 
> > Changes from v3->v4.
> > 1. Rebased on for-next.
> > 2. Fixed issuses with checkpatch --strict.
> > 3. Unfied all IPI/fence related functions.
> > 4. Added Hfence related SBI calls.
> > 
> > Changes from v2->v3.
> > 1. Moved v0.1 extensions to a new config.
> > 2. Added support for relacement extensions of v0.1 extensions.
> > 
> > Changes from v1->v2
> > 1. Removed the legacy calling convention.
> > 2. Moved all SBI related calls to sbi.c.
> > 3. Moved all SBI related macros to uapi.
> > 
> > Atish Patra (11):
> > RISC-V: Mark existing SBI as 0.1 SBI.
> > RISC-V: Add basic support for SBI v0.2
> > RISC-V: Add SBI v0.2 extension definitions
> > RISC-V: Introduce a new config for SBI v0.1
> > RISC-V: Implement new SBI v0.2 extensions
> > RISC-V: Move relocate and few other functions out of __init
> > RISC-V: Add cpu_ops and modify default booting method
> > RISC-V: Add SBI HSM extension
> > RISC-V: Add supported for ordered booting method using HSM
> > irqchip/sifive-plic: Initialize the plic handler when cpu comes
> > online
> > RISC-V: Support cpu hotplug
> > 
> > arch/riscv/Kconfig                   |  19 +-
> > arch/riscv/include/asm/cpu_ops.h     |  46 +++
> > arch/riscv/include/asm/sbi.h         | 194 ++++++----
> > arch/riscv/include/asm/smp.h         |  24 ++
> > arch/riscv/kernel/Makefile           |   6 +
> > arch/riscv/kernel/cpu-hotplug.c      |  87 +++++
> > arch/riscv/kernel/cpu_ops.c          |  48 +++
> > arch/riscv/kernel/cpu_ops_sbi.c      | 113 ++++++
> > arch/riscv/kernel/cpu_ops_spinwait.c |  42 +++
> > arch/riscv/kernel/head.S             | 179 +++++----
> > arch/riscv/kernel/sbi.c              | 524
> > ++++++++++++++++++++++++++-
> > arch/riscv/kernel/setup.c            |  24 +-
> > arch/riscv/kernel/smpboot.c          |  56 +--
> > arch/riscv/kernel/traps.c            |   2 +-
> > arch/riscv/kernel/vmlinux.lds.S      |   5 +-
> > drivers/irqchip/irq-sifive-plic.c    |  34 +-
> > include/linux/cpuhotplug.h           |   1 +
> > 17 files changed, 1227 insertions(+), 177 deletions(-)
> > create mode 100644 arch/riscv/include/asm/cpu_ops.h
> > create mode 100644 arch/riscv/kernel/cpu-hotplug.c
> > create mode 100644 arch/riscv/kernel/cpu_ops.c
> > create mode 100644 arch/riscv/kernel/cpu_ops_sbi.c
> > create mode 100644 arch/riscv/kernel/cpu_ops_spinwait.c

-- 
Regards,
Atish