* [U-Boot] [PATCH 0/8] RISC-V AX25-AE350 support SPL @ 2019-10-25 6:10 Andes 2019-10-25 6:10 ` [U-Boot] [PATCH 1/8] riscv: ax25: add SPL support Andes ` (7 more replies) 0 siblings, 8 replies; 71+ messages in thread From: Andes @ 2019-10-25 6:10 UTC (permalink / raw) To: u-boot From: Rick Chen <rick@andestech.com> This series add support for SPL to AX25-AE350. U-Boot SPL can boots from RAM or ROM and jump to OPenSbi(FW_DYNAMIC firmware) and U-Boot proper from RAM or MMC devices. Also fix some bugs for andes plic driver and improve cache configurations for SPL. Following are the booting messages on AE350 four cores SMP platform: U-Boot 2019.10-00602-g693b70f (Oct 23 2019 - 16:32:19 +0800) DRAM: 1 GiB U-Boot SPL 2019.10-00601-g04f8f09-dirty (Oct 24 2019 - 10:46:49 +0800) Trying to boot from RAM OpenSBI v0.4-32-g98ee15c (Sep 17 2019 10:41:30) ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_| Platform Name : Andes AE350 Platform HART Features : RV64ACIMSUX Platform Max HARTs : 4 Current Hart : 0 Firmware Base : 0x0 Firmware Size : 80 KB Runtime SBI Version : 0.1 PMP0: 0x0000000000000000-0x000000000001ffff (A) PMP1: 0x0000000000000000-0x00000001ffffffff (A,R,W,X) U-Boot 2019.10-00601-g04f8f09-dirty (Oct 24 2019 - 10:46:49 +0800) DRAM: 1 GiB Flash: 64 MiB MMC: mmc at f0e00000: 0 Loading Environment from SPI Flash... SF: Detected mx25u1635e with page size 256 Bytes, erase size 4 KiB, total 2 MiB OK In: serial at f0300000 Out: serial at f0300000 Err: serial at f0300000 Net: no alias for ethernet0 Warning: mac at e0100000 (eth0) using random MAC address - 06:2b:40:2d:38:d1 eth0: mac at e0100000 Hit any key to stop autoboot: 0 6455 bytes read in 30 ms (210 KiB/s) 20421684 bytes read in 8614 ms (2.3 MiB/s) ## Booting kernel from Legacy Image at 00600000 ... Image Name: Image Type: RISC-V Linux Kernel Image (uncompressed) Data Size: 20421620 Bytes = 19.5 MiB Load Address: 00200000 Entry Point: 00200000 Verifying Checksum ... OK ## Flattened Device Tree blob at 20000000 Booting using the fdt blob at 0x20000000 Loading Kernel Image Loading Device Tree to 000000001effb000, end 000000001efff936 ... OK Starting kernel ... OF: fdt: Ignoring memory range 0x0 - 0x200000 Linux version 4.17.0-00253-g49136e10bcb2 (sqa at atcsqa07) (gcc version 7.3.0 (2019-04-06_nds64le-linux-glibc-v5_experimental)) #1 SMP PREEMPT Sat Apr 6 23:41:49 CST 2019 bootconsole [early0] enabled Initial ramdisk at: 0x (ptrval) (13665712 bytes) Zone ranges: DMA32 [mem 0x0000000000200000-0x000000003fffffff] Normal empty Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000000200000-0x000000003fffffff] Initmem setup node 0 [mem 0x0000000000200000-0x000000003fffffff] software IO TLB [mem 0x3b1f8000-0x3f1f8000] (64MB) mapped at [ (ptrval)- (ptrval)] elf_platform is rv64i2p0m2p0a2p0c2p0xv5-0p0 compatible privileged spec version 1.10 percpu: Embedded 16 pages/cpu @ (ptrval) s28184 r8192 d29160 u65536 Built 1 zonelists, mobility grouping on. Total pages: 258055 Kernel command line: console=ttyS0,38400n8 debug loglevel=7 log_buf_len individual max cpu contribution: 4096 bytes log_buf_len total cpu_extra contributions: 12288 bytes log_buf_len min size: 16384 bytes log_buf_len: 32768 bytes early log buf free: 14608(89%) Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes) Inode-cache hash table entries: 65536 (order: 7, 524288 bytes) Sorting __ex_table... Memory: 944428K/1046528K available (3979K kernel code, 246K rwdata, 1490K rodata, 13523K init, 688K bss, 102100K reserved, 0K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 Preemptible hierarchical RCU implementation. Tasks RCU enabled. NR_IRQS: 72, nr_irqs: 72, preallocated irqs: 0 riscv,cpu_intc,0: 64 local interrupts mapped riscv,cpu_intc,1: 64 local interrupts mapped riscv,cpu_intc,2: 64 local interrupts mapped riscv,cpu_intc,3: 64 local interrupts mapped riscv,plic0,e4000000: mapped 71 interrupts to 8/8 handlers clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x1bacf917bf, max_idle_ns: 881590412290 ns sched_clock: 64 bits@60MHz, resolution 16ns, wraps every 4398046511098ns Console: colour dummy device 40x30 Calibrating delay loop (skipped), value calculated using timer frequency.. 120.00 BogoMIPS (lpj=600000) pid_max: default: 32768 minimum: 301 Mount-cache hash table entries: 2048 (order: 2, 16384 bytes) Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes) Hierarchical SRCU implementation. smp: Bringing up secondary CPUs ... CPU0: online CPU2: online CPU3: online smp: Brought up 1 node, 4 CPUs ... ... ... Rick Chen (8): riscv: ax25: add SPL support riscv: ax25-ae350: add SPL configuration riscv: ax25-ae350: Use generic memory size setup riscv: andes_plic: Fix some wrong configurations riscv: ax25: cache: Add SPL_RISCV_MMODE for SPL spl: cache: Allow cache drivers in SPL riscv: Fix clear bss loop in the start-up code riscv: dts: Support four cores SMP arch/riscv/cpu/ax25/Kconfig | 4 ++- arch/riscv/cpu/ax25/cache.c | 60 +++++++++++++++++++++++++-------- arch/riscv/cpu/start.S | 4 +-- arch/riscv/dts/ae350_32.dts | 51 ++++++++++++++++++++++++++-- arch/riscv/dts/ae350_64.dts | 51 ++++++++++++++++++++++++++-- arch/riscv/lib/andes_plic.c | 11 +++--- board/AndesTech/ax25-ae350/Kconfig | 10 ++++++ board/AndesTech/ax25-ae350/MAINTAINERS | 4 +++ board/AndesTech/ax25-ae350/ax25-ae350.c | 48 +++++++++++++++----------- common/spl/Kconfig | 7 ++++ configs/ae350_rv32_spl_defconfig | 37 ++++++++++++++++++++ configs/ae350_rv32_spl_xip_defconfig | 39 +++++++++++++++++++++ configs/ae350_rv64_spl_defconfig | 38 +++++++++++++++++++++ configs/ae350_rv64_spl_xip_defconfig | 40 ++++++++++++++++++++++ drivers/Makefile | 1 + include/configs/ax25-ae350.h | 17 ++++++++++ 16 files changed, 376 insertions(+), 46 deletions(-) create mode 100644 configs/ae350_rv32_spl_defconfig create mode 100644 configs/ae350_rv32_spl_xip_defconfig create mode 100644 configs/ae350_rv64_spl_defconfig create mode 100644 configs/ae350_rv64_spl_xip_defconfig -- 2.7.4 ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 1/8] riscv: ax25: add SPL support 2019-10-25 6:10 [U-Boot] [PATCH 0/8] RISC-V AX25-AE350 support SPL Andes @ 2019-10-25 6:10 ` Andes 2019-10-29 14:29 ` Bin Meng 2019-10-25 6:10 ` [U-Boot] [PATCH 2/8] riscv: ax25-ae350: add SPL configuration Andes ` (6 subsequent siblings) 7 siblings, 1 reply; 71+ messages in thread From: Andes @ 2019-10-25 6:10 UTC (permalink / raw) To: u-boot From: Rick Chen <rick@andestech.com> The U-Boot SPL will boot in M mode and load the FIT image which include OpenSbi and U-Boot proper images. After loading progress, it will jump to OpenSbi first and then U-Boot proper which will run in S mode. Signed-off-by: Rick Chen <rick@andestech.com> Cc: KC Lin <kclin@andestech.com> Cc: Alan Kao <alankao@andestech.com> --- arch/riscv/cpu/ax25/Kconfig | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/riscv/cpu/ax25/Kconfig b/arch/riscv/cpu/ax25/Kconfig index d411a79..8d8d71d 100644 --- a/arch/riscv/cpu/ax25/Kconfig +++ b/arch/riscv/cpu/ax25/Kconfig @@ -6,7 +6,9 @@ config RISCV_NDS imply RISCV_TIMER imply ANDES_PLIC if (RISCV_MMODE || SPL_RISCV_MMODE) imply ANDES_PLMT if (RISCV_MMODE || SPL_RISCV_MMODE) - imply V5L2_CACHE + imply SPL_CPU_SUPPORT + imply SPL_OPENSBI + imply SPL_LOAD_FIT help Run U-Boot on AndeStar V5 platforms and use some specific features which are provided by Andes Technology AndeStar V5 families. -- 2.7.4 ^ permalink raw reply related [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 1/8] riscv: ax25: add SPL support 2019-10-25 6:10 ` [U-Boot] [PATCH 1/8] riscv: ax25: add SPL support Andes @ 2019-10-29 14:29 ` Bin Meng 2019-10-30 0:42 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-29 14:29 UTC (permalink / raw) To: u-boot Hi Rick, On Fri, Oct 25, 2019 at 2:17 PM Andes <uboot@andestech.com> wrote: > > From: Rick Chen <rick@andestech.com> > > The U-Boot SPL will boot in M mode and load the > FIT image which include OpenSbi and U-Boot proper nits: OpenSBI > images. After loading progress, it will jump to > OpenSbi first and then U-Boot proper which will ditto > run in S mode. > > Signed-off-by: Rick Chen <rick@andestech.com> > Cc: KC Lin <kclin@andestech.com> > Cc: Alan Kao <alankao@andestech.com> > --- > arch/riscv/cpu/ax25/Kconfig | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/arch/riscv/cpu/ax25/Kconfig b/arch/riscv/cpu/ax25/Kconfig > index d411a79..8d8d71d 100644 > --- a/arch/riscv/cpu/ax25/Kconfig > +++ b/arch/riscv/cpu/ax25/Kconfig > @@ -6,7 +6,9 @@ config RISCV_NDS > imply RISCV_TIMER > imply ANDES_PLIC if (RISCV_MMODE || SPL_RISCV_MMODE) > imply ANDES_PLMT if (RISCV_MMODE || SPL_RISCV_MMODE) > - imply V5L2_CACHE Why this is removed? > + imply SPL_CPU_SUPPORT > + imply SPL_OPENSBI > + imply SPL_LOAD_FIT > help > Run U-Boot on AndeStar V5 platforms and use some specific features > which are provided by Andes Technology AndeStar V5 families. > -- Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 1/8] riscv: ax25: add SPL support 2019-10-29 14:29 ` Bin Meng @ 2019-10-30 0:42 ` Rick Chen 2019-10-30 10:06 ` Bin Meng 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-10-30 0:42 UTC (permalink / raw) To: u-boot Hi Bin > > Hi Rick, > > On Fri, Oct 25, 2019 at 2:17 PM Andes <uboot@andestech.com> wrote: > > > > From: Rick Chen <rick@andestech.com> > > > > The U-Boot SPL will boot in M mode and load the > > FIT image which include OpenSbi and U-Boot proper > > nits: OpenSBI OK > > > images. After loading progress, it will jump to > > OpenSbi first and then U-Boot proper which will > > ditto OK > > > run in S mode. > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > Cc: KC Lin <kclin@andestech.com> > > Cc: Alan Kao <alankao@andestech.com> > > --- > > arch/riscv/cpu/ax25/Kconfig | 4 +++- > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > diff --git a/arch/riscv/cpu/ax25/Kconfig b/arch/riscv/cpu/ax25/Kconfig > > index d411a79..8d8d71d 100644 > > --- a/arch/riscv/cpu/ax25/Kconfig > > +++ b/arch/riscv/cpu/ax25/Kconfig > > @@ -6,7 +6,9 @@ config RISCV_NDS > > imply RISCV_TIMER > > imply ANDES_PLIC if (RISCV_MMODE || SPL_RISCV_MMODE) > > imply ANDES_PLMT if (RISCV_MMODE || SPL_RISCV_MMODE) > > - imply V5L2_CACHE > > Why this is removed? Without CACHE_SUPPORT, the compiling will fail while enable V5l2_CACHE. That is why I remove it. I hope it can be enable manually. Because it will enlarge U-Boot SPL size. Thanks Rick > > > + imply SPL_CPU_SUPPORT > > + imply SPL_OPENSBI > > + imply SPL_LOAD_FIT > > help > > Run U-Boot on AndeStar V5 platforms and use some specific features > > which are provided by Andes Technology AndeStar V5 families. > > -- > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 1/8] riscv: ax25: add SPL support 2019-10-30 0:42 ` Rick Chen @ 2019-10-30 10:06 ` Bin Meng 2019-10-31 2:11 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-30 10:06 UTC (permalink / raw) To: u-boot Hi Rick, On Wed, Oct 30, 2019 at 8:42 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Bin > > > > > Hi Rick, > > > > On Fri, Oct 25, 2019 at 2:17 PM Andes <uboot@andestech.com> wrote: > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > The U-Boot SPL will boot in M mode and load the > > > FIT image which include OpenSbi and U-Boot proper > > > > nits: OpenSBI > > OK > > > > > > images. After loading progress, it will jump to > > > OpenSbi first and then U-Boot proper which will > > > > ditto > > OK > > > > > > run in S mode. > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > Cc: KC Lin <kclin@andestech.com> > > > Cc: Alan Kao <alankao@andestech.com> > > > --- > > > arch/riscv/cpu/ax25/Kconfig | 4 +++- > > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > > > diff --git a/arch/riscv/cpu/ax25/Kconfig b/arch/riscv/cpu/ax25/Kconfig > > > index d411a79..8d8d71d 100644 > > > --- a/arch/riscv/cpu/ax25/Kconfig > > > +++ b/arch/riscv/cpu/ax25/Kconfig > > > @@ -6,7 +6,9 @@ config RISCV_NDS > > > imply RISCV_TIMER > > > imply ANDES_PLIC if (RISCV_MMODE || SPL_RISCV_MMODE) > > > imply ANDES_PLMT if (RISCV_MMODE || SPL_RISCV_MMODE) > > > - imply V5L2_CACHE > > > > Why this is removed? > > Without CACHE_SUPPORT, the compiling will fail while enable V5l2_CACHE. > That is why I remove it. I hope it can be enable manually. > Because it will enlarge U-Boot SPL size. > Could you please describe it in the commit message? Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 1/8] riscv: ax25: add SPL support 2019-10-30 10:06 ` Bin Meng @ 2019-10-31 2:11 ` Rick Chen 0 siblings, 0 replies; 71+ messages in thread From: Rick Chen @ 2019-10-31 2:11 UTC (permalink / raw) To: u-boot Hi Bin > > Hi Rick, > > On Wed, Oct 30, 2019 at 8:42 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Bin > > > > > > > > Hi Rick, > > > > > > On Fri, Oct 25, 2019 at 2:17 PM Andes <uboot@andestech.com> wrote: > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > The U-Boot SPL will boot in M mode and load the > > > > FIT image which include OpenSbi and U-Boot proper > > > > > > nits: OpenSBI > > > > OK > > > > > > > > > images. After loading progress, it will jump to > > > > OpenSbi first and then U-Boot proper which will > > > > > > ditto > > > > OK > > > > > > > > > run in S mode. > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > Cc: KC Lin <kclin@andestech.com> > > > > Cc: Alan Kao <alankao@andestech.com> > > > > --- > > > > arch/riscv/cpu/ax25/Kconfig | 4 +++- > > > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/arch/riscv/cpu/ax25/Kconfig b/arch/riscv/cpu/ax25/Kconfig > > > > index d411a79..8d8d71d 100644 > > > > --- a/arch/riscv/cpu/ax25/Kconfig > > > > +++ b/arch/riscv/cpu/ax25/Kconfig > > > > @@ -6,7 +6,9 @@ config RISCV_NDS > > > > imply RISCV_TIMER > > > > imply ANDES_PLIC if (RISCV_MMODE || SPL_RISCV_MMODE) > > > > imply ANDES_PLMT if (RISCV_MMODE || SPL_RISCV_MMODE) > > > > - imply V5L2_CACHE > > > > > > Why this is removed? > > > > Without CACHE_SUPPORT, the compiling will fail while enable V5l2_CACHE. > > That is why I remove it. I hope it can be enable manually. > > Because it will enlarge U-Boot SPL size. > > > > Could you please describe it in the commit message? OK. I will add it. Thanks Rick > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 2/8] riscv: ax25-ae350: add SPL configuration 2019-10-25 6:10 [U-Boot] [PATCH 0/8] RISC-V AX25-AE350 support SPL Andes 2019-10-25 6:10 ` [U-Boot] [PATCH 1/8] riscv: ax25: add SPL support Andes @ 2019-10-25 6:10 ` Andes 2019-10-29 14:39 ` Bin Meng 2019-10-25 6:10 ` [U-Boot] [PATCH 3/8] riscv: ax25-ae350: Use generic memory size setup Andes ` (5 subsequent siblings) 7 siblings, 1 reply; 71+ messages in thread From: Andes @ 2019-10-25 6:10 UTC (permalink / raw) To: u-boot From: Rick Chen <rick@andestech.com> This patch provides four configurations which can support U-Boot SPL to boot from RAM or FLASH and then boot FIT image including OpenSBI FW_DYNAMIC firmware and U-Boot proper images from RAM or MMC boot devices. With ae350_rv[32|64]_spl_defconfigs: U-Boot SPL will be loaded by gdb or FSBL and runs in RAM in machine mode and then load FIT image from RAM device on AE350. With ae350_rv[32|64]_spl_xip_defconfigs: U-Boot SPL can be burned into SPI flash and run in flash in machine mode and then load FIT image from SPI flash or MMC device on AE350. Signed-off-by: Rick Chen <rick@andestech.com> Cc: KC Lin <kclin@andestech.com> Cc: Alan Kao <alankao@andestech.com> --- board/AndesTech/ax25-ae350/Kconfig | 10 +++++++++ board/AndesTech/ax25-ae350/MAINTAINERS | 4 ++++ board/AndesTech/ax25-ae350/ax25-ae350.c | 27 ++++++++++++++++++++++ configs/ae350_rv32_spl_defconfig | 37 ++++++++++++++++++++++++++++++ configs/ae350_rv32_spl_xip_defconfig | 39 ++++++++++++++++++++++++++++++++ configs/ae350_rv64_spl_defconfig | 38 +++++++++++++++++++++++++++++++ configs/ae350_rv64_spl_xip_defconfig | 40 +++++++++++++++++++++++++++++++++ include/configs/ax25-ae350.h | 17 ++++++++++++++ 8 files changed, 212 insertions(+) create mode 100644 configs/ae350_rv32_spl_defconfig create mode 100644 configs/ae350_rv32_spl_xip_defconfig create mode 100644 configs/ae350_rv64_spl_defconfig create mode 100644 configs/ae350_rv64_spl_xip_defconfig diff --git a/board/AndesTech/ax25-ae350/Kconfig b/board/AndesTech/ax25-ae350/Kconfig index 5e682b6..2e1e2bb 100644 --- a/board/AndesTech/ax25-ae350/Kconfig +++ b/board/AndesTech/ax25-ae350/Kconfig @@ -21,9 +21,19 @@ config ENV_SIZE config ENV_OFFSET default 0x140000 if ENV_IS_IN_SPI_FLASH +config SPL_TEXT_BASE + default 0x00000000 + +config SPL_OPENSBI_LOAD_ADDR + default 0x01000000 + config BOARD_SPECIFIC_OPTIONS # dummy def_bool y select RISCV_NDS + select SUPPORT_SPL + imply SYS_NS16550 imply SMP + imply SPL_RAM_SUPPORT + imply SPL_RAM_DEVICE endif diff --git a/board/AndesTech/ax25-ae350/MAINTAINERS b/board/AndesTech/ax25-ae350/MAINTAINERS index feed5d1..eebee16 100644 --- a/board/AndesTech/ax25-ae350/MAINTAINERS +++ b/board/AndesTech/ax25-ae350/MAINTAINERS @@ -7,3 +7,7 @@ F: configs/ae350_rv32_defconfig F: configs/ae350_rv64_defconfig F: configs/ae350_rv32_xip_defconfig F: configs/ae350_rv64_xip_defconfig +F: configs/ae350_rv32_spl_defconfig +F: configs/ae350_rv64_spl_defconfig +F: configs/ae350_rv32_spl_xip_defconfig +F: configs/ae350_rv64_spl_xip_defconfig diff --git a/board/AndesTech/ax25-ae350/ax25-ae350.c b/board/AndesTech/ax25-ae350/ax25-ae350.c index b43eebb..b0164a9 100644 --- a/board/AndesTech/ax25-ae350/ax25-ae350.c +++ b/board/AndesTech/ax25-ae350/ax25-ae350.c @@ -12,6 +12,7 @@ #include <faraday/ftsmc020.h> #include <fdtdec.h> #include <dm.h> +#include <spl.h> DECLARE_GLOBAL_DATA_PTR; @@ -110,3 +111,29 @@ int board_early_init_f(void) return 0; } #endif + +#ifdef CONFIG_SPL +void board_boot_order(u32 *spl_boot_list) +{ + u8 i; + u32 boot_devices[] = { +#ifdef CONFIG_SPL_RAM_SUPPORT + BOOT_DEVICE_RAM, +#endif +#ifdef CONFIG_SPL_MMC_SUPPORT + BOOT_DEVICE_MMC1, +#endif + }; + + for (i = 0; i < ARRAY_SIZE(boot_devices); i++) + spl_boot_list[i] = boot_devices[i]; +} +#endif + +#ifdef CONFIG_SPL_LOAD_FIT +int board_fit_config_name_match(const char *name) +{ + /* boot using first FIT config */ + return 0; +} +#endif diff --git a/configs/ae350_rv32_spl_defconfig b/configs/ae350_rv32_spl_defconfig new file mode 100644 index 0000000..53055b7 --- /dev/null +++ b/configs/ae350_rv32_spl_defconfig @@ -0,0 +1,37 @@ +CONFIG_RISCV=y +CONFIG_SPL=y +CONFIG_RISCV_SMODE=y +CONFIG_SYS_TEXT_BASE=0x01200000 +CONFIG_NR_DRAM_BANKS=2 +CONFIG_TARGET_AX25_AE350=y +CONFIG_DISTRO_DEFAULTS=y +CONFIG_FIT=y +CONFIG_BOOTDELAY=3 +CONFIG_BOARD_EARLY_INIT_F=y +CONFIG_SYS_PROMPT="RISC-V # " +CONFIG_CMD_IMLS=y +CONFIG_CMD_MMC=y +CONFIG_CMD_SF=y +CONFIG_CMD_SF_TEST=y +# CONFIG_CMD_SETEXPR is not set +CONFIG_BOOTP_PREFER_SERVERIP=y +CONFIG_CMD_CACHE=y +CONFIG_OF_PRIOR_STAGE=y +CONFIG_ENV_IS_IN_SPI_FLASH=y +CONFIG_NET_RANDOM_ETHADDR=y +CONFIG_MMC=y +CONFIG_FTSDC010=y +CONFIG_FTSDC010_SDIO=y +CONFIG_MTD_NOR_FLASH=y +CONFIG_FLASH_CFI_DRIVER=y +CONFIG_CFI_FLASH=y +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y +CONFIG_SYS_FLASH_CFI=y +CONFIG_SPI_FLASH=y +CONFIG_SF_DEFAULT_MODE=0x0 +CONFIG_SPI_FLASH_MACRONIX=y +CONFIG_FTMAC100=y +CONFIG_BAUDRATE=38400 +CONFIG_SYS_NS16550=y +CONFIG_SPI=y +CONFIG_ATCSPI200_SPI=y diff --git a/configs/ae350_rv32_spl_xip_defconfig b/configs/ae350_rv32_spl_xip_defconfig new file mode 100644 index 0000000..fdbab43 --- /dev/null +++ b/configs/ae350_rv32_spl_xip_defconfig @@ -0,0 +1,39 @@ +CONFIG_RISCV=y +CONFIG_SPL=y +CONFIG_RISCV_SMODE=y +CONFIG_SPL_TEXT_BASE=0x80000000 +CONFIG_SYS_TEXT_BASE=0x01200000 +CONFIG_NR_DRAM_BANKS=2 +CONFIG_TARGET_AX25_AE350=y +CONFIG_XIP=y +CONFIG_DISTRO_DEFAULTS=y +CONFIG_FIT=y +CONFIG_BOOTDELAY=3 +CONFIG_BOARD_EARLY_INIT_F=y +CONFIG_SYS_PROMPT="RISC-V # " +CONFIG_CMD_IMLS=y +CONFIG_CMD_MMC=y +CONFIG_CMD_SF=y +CONFIG_CMD_SF_TEST=y +# CONFIG_CMD_SETEXPR is not set +CONFIG_BOOTP_PREFER_SERVERIP=y +CONFIG_CMD_CACHE=y +CONFIG_OF_BOARD=y +CONFIG_ENV_IS_IN_SPI_FLASH=y +CONFIG_NET_RANDOM_ETHADDR=y +CONFIG_MMC=y +CONFIG_FTSDC010=y +CONFIG_FTSDC010_SDIO=y +CONFIG_MTD_NOR_FLASH=y +CONFIG_FLASH_CFI_DRIVER=y +CONFIG_CFI_FLASH=y +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y +CONFIG_SYS_FLASH_CFI=y +CONFIG_SPI_FLASH=y +CONFIG_SF_DEFAULT_MODE=0x0 +CONFIG_SPI_FLASH_MACRONIX=y +CONFIG_FTMAC100=y +CONFIG_BAUDRATE=38400 +CONFIG_SYS_NS16550=y +CONFIG_SPI=y +CONFIG_ATCSPI200_SPI=y diff --git a/configs/ae350_rv64_spl_defconfig b/configs/ae350_rv64_spl_defconfig new file mode 100644 index 0000000..866b9b4 --- /dev/null +++ b/configs/ae350_rv64_spl_defconfig @@ -0,0 +1,38 @@ +CONFIG_RISCV=y +CONFIG_SPL=y +CONFIG_RISCV_SMODE=y +CONFIG_SYS_TEXT_BASE=0x01200000 +CONFIG_NR_DRAM_BANKS=2 +CONFIG_TARGET_AX25_AE350=y +CONFIG_ARCH_RV64I=y +CONFIG_DISTRO_DEFAULTS=y +CONFIG_FIT=y +CONFIG_BOOTDELAY=3 +CONFIG_BOARD_EARLY_INIT_F=y +CONFIG_SYS_PROMPT="RISC-V # " +CONFIG_CMD_IMLS=y +CONFIG_CMD_MMC=y +CONFIG_CMD_SF=y +CONFIG_CMD_SF_TEST=y +# CONFIG_CMD_SETEXPR is not set +CONFIG_BOOTP_PREFER_SERVERIP=y +CONFIG_CMD_CACHE=y +CONFIG_OF_PRIOR_STAGE=y +CONFIG_ENV_IS_IN_SPI_FLASH=y +CONFIG_NET_RANDOM_ETHADDR=y +CONFIG_MMC=y +CONFIG_FTSDC010=y +CONFIG_FTSDC010_SDIO=y +CONFIG_MTD_NOR_FLASH=y +CONFIG_FLASH_CFI_DRIVER=y +CONFIG_CFI_FLASH=y +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y +CONFIG_SYS_FLASH_CFI=y +CONFIG_SPI_FLASH=y +CONFIG_SF_DEFAULT_MODE=0x0 +CONFIG_SPI_FLASH_MACRONIX=y +CONFIG_FTMAC100=y +CONFIG_BAUDRATE=38400 +CONFIG_SYS_NS16550=y +CONFIG_SPI=y +CONFIG_ATCSPI200_SPI=y diff --git a/configs/ae350_rv64_spl_xip_defconfig b/configs/ae350_rv64_spl_xip_defconfig new file mode 100644 index 0000000..271ec21 --- /dev/null +++ b/configs/ae350_rv64_spl_xip_defconfig @@ -0,0 +1,40 @@ +CONFIG_RISCV=y +CONFIG_SPL=y +CONFIG_RISCV_SMODE=y +CONFIG_SPL_TEXT_BASE=0x80000000 +CONFIG_SYS_TEXT_BASE=0x01200000 +CONFIG_NR_DRAM_BANKS=2 +CONFIG_TARGET_AX25_AE350=y +CONFIG_ARCH_RV64I=y +CONFIG_XIP=y +CONFIG_DISTRO_DEFAULTS=y +CONFIG_FIT=y +CONFIG_BOOTDELAY=3 +CONFIG_BOARD_EARLY_INIT_F=y +CONFIG_SYS_PROMPT="RISC-V # " +CONFIG_CMD_IMLS=y +CONFIG_CMD_MMC=y +CONFIG_CMD_SF=y +CONFIG_CMD_SF_TEST=y +# CONFIG_CMD_SETEXPR is not set +CONFIG_BOOTP_PREFER_SERVERIP=y +CONFIG_CMD_CACHE=y +CONFIG_OF_BOARD=y +CONFIG_ENV_IS_IN_SPI_FLASH=y +CONFIG_NET_RANDOM_ETHADDR=y +CONFIG_MMC=y +CONFIG_FTSDC010=y +CONFIG_FTSDC010_SDIO=y +CONFIG_MTD_NOR_FLASH=y +CONFIG_FLASH_CFI_DRIVER=y +CONFIG_CFI_FLASH=y +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y +CONFIG_SYS_FLASH_CFI=y +CONFIG_SPI_FLASH=y +CONFIG_SF_DEFAULT_MODE=0x0 +CONFIG_SPI_FLASH_MACRONIX=y +CONFIG_FTMAC100=y +CONFIG_BAUDRATE=38400 +CONFIG_SYS_NS16550=y +CONFIG_SPI=y +CONFIG_ATCSPI200_SPI=y diff --git a/include/configs/ax25-ae350.h b/include/configs/ax25-ae350.h index a4037f3..ef3864a 100644 --- a/include/configs/ax25-ae350.h +++ b/include/configs/ax25-ae350.h @@ -7,6 +7,23 @@ #ifndef __CONFIG_H #define __CONFIG_H +#ifdef CONFIG_SPL +#define CONFIG_SPL_MAX_SIZE 0x00100000 +#define CONFIG_SPL_BSS_START_ADDR 0x04000000 +#define CONFIG_SPL_BSS_MAX_SIZE 0x00100000 + +#ifndef CONFIG_XIP +#define CONFIG_SPL_LOAD_FIT_ADDRESS 0x00200000 +#else +#define CONFIG_SPL_LOAD_FIT_ADDRESS 0x80010000 +#endif + +#ifdef CONFIG_SPL_MMC_SUPPORT +#define CONFIG_SYS_MMCSD_FS_BOOT_PARTITION 1 +#define CONFIG_SPL_FS_LOAD_PAYLOAD_NAME "u-boot.itb" +#endif +#endif + /* * CPU and Board Configuration Options */ -- 2.7.4 ^ permalink raw reply related [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 2/8] riscv: ax25-ae350: add SPL configuration 2019-10-25 6:10 ` [U-Boot] [PATCH 2/8] riscv: ax25-ae350: add SPL configuration Andes @ 2019-10-29 14:39 ` Bin Meng 2019-10-30 2:06 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-29 14:39 UTC (permalink / raw) To: u-boot Hi Rick, On Fri, Oct 25, 2019 at 2:17 PM Andes <uboot@andestech.com> wrote: > > From: Rick Chen <rick@andestech.com> > > This patch provides four configurations > which can support U-Boot SPL to boot from > RAM or FLASH and then boot FIT image > including OpenSBI FW_DYNAMIC firmware > and U-Boot proper images from RAM or > MMC boot devices. > > With ae350_rv[32|64]_spl_defconfigs: > > U-Boot SPL will be loaded by gdb or FSBL > and runs in RAM in machine mode and then > load FIT image from RAM device on AE350. > > With ae350_rv[32|64]_spl_xip_defconfigs: > > U-Boot SPL can be burned into SPI flash > and run in flash in machine mode and then > load FIT image from SPI flash or MMC device > on AE350. > Could you please rewrite the commit message so that every line has about 70 characters? > Signed-off-by: Rick Chen <rick@andestech.com> > Cc: KC Lin <kclin@andestech.com> > Cc: Alan Kao <alankao@andestech.com> > --- > board/AndesTech/ax25-ae350/Kconfig | 10 +++++++++ > board/AndesTech/ax25-ae350/MAINTAINERS | 4 ++++ > board/AndesTech/ax25-ae350/ax25-ae350.c | 27 ++++++++++++++++++++++ > configs/ae350_rv32_spl_defconfig | 37 ++++++++++++++++++++++++++++++ > configs/ae350_rv32_spl_xip_defconfig | 39 ++++++++++++++++++++++++++++++++ > configs/ae350_rv64_spl_defconfig | 38 +++++++++++++++++++++++++++++++ > configs/ae350_rv64_spl_xip_defconfig | 40 +++++++++++++++++++++++++++++++++ > include/configs/ax25-ae350.h | 17 ++++++++++++++ > 8 files changed, 212 insertions(+) > create mode 100644 configs/ae350_rv32_spl_defconfig > create mode 100644 configs/ae350_rv32_spl_xip_defconfig > create mode 100644 configs/ae350_rv64_spl_defconfig > create mode 100644 configs/ae350_rv64_spl_xip_defconfig > > diff --git a/board/AndesTech/ax25-ae350/Kconfig b/board/AndesTech/ax25-ae350/Kconfig > index 5e682b6..2e1e2bb 100644 > --- a/board/AndesTech/ax25-ae350/Kconfig > +++ b/board/AndesTech/ax25-ae350/Kconfig > @@ -21,9 +21,19 @@ config ENV_SIZE > config ENV_OFFSET > default 0x140000 if ENV_IS_IN_SPI_FLASH > > +config SPL_TEXT_BASE > + default 0x00000000 > + > +config SPL_OPENSBI_LOAD_ADDR > + default 0x01000000 > + > config BOARD_SPECIFIC_OPTIONS # dummy > def_bool y > select RISCV_NDS > + select SUPPORT_SPL > + imply SYS_NS16550 Is there any 16550 on the AX25 boards? If not, please remove. If yes, this should be a separate patch with DTS changes. > imply SMP > + imply SPL_RAM_SUPPORT > + imply SPL_RAM_DEVICE > > endif > diff --git a/board/AndesTech/ax25-ae350/MAINTAINERS b/board/AndesTech/ax25-ae350/MAINTAINERS > index feed5d1..eebee16 100644 > --- a/board/AndesTech/ax25-ae350/MAINTAINERS > +++ b/board/AndesTech/ax25-ae350/MAINTAINERS > @@ -7,3 +7,7 @@ F: configs/ae350_rv32_defconfig > F: configs/ae350_rv64_defconfig > F: configs/ae350_rv32_xip_defconfig > F: configs/ae350_rv64_xip_defconfig > +F: configs/ae350_rv32_spl_defconfig > +F: configs/ae350_rv64_spl_defconfig > +F: configs/ae350_rv32_spl_xip_defconfig > +F: configs/ae350_rv64_spl_xip_defconfig Could you please update the doc/board/AndesTech/ax25-ae350.rst to add descriptions about all these different configurations of the same board? > diff --git a/board/AndesTech/ax25-ae350/ax25-ae350.c b/board/AndesTech/ax25-ae350/ax25-ae350.c > index b43eebb..b0164a9 100644 > --- a/board/AndesTech/ax25-ae350/ax25-ae350.c > +++ b/board/AndesTech/ax25-ae350/ax25-ae350.c > @@ -12,6 +12,7 @@ > #include <faraday/ftsmc020.h> > #include <fdtdec.h> > #include <dm.h> > +#include <spl.h> > > DECLARE_GLOBAL_DATA_PTR; > > @@ -110,3 +111,29 @@ int board_early_init_f(void) > return 0; > } > #endif > + > +#ifdef CONFIG_SPL > +void board_boot_order(u32 *spl_boot_list) > +{ > + u8 i; > + u32 boot_devices[] = { > +#ifdef CONFIG_SPL_RAM_SUPPORT > + BOOT_DEVICE_RAM, > +#endif > +#ifdef CONFIG_SPL_MMC_SUPPORT > + BOOT_DEVICE_MMC1, > +#endif > + }; > + > + for (i = 0; i < ARRAY_SIZE(boot_devices); i++) > + spl_boot_list[i] = boot_devices[i]; > +} > +#endif > + > +#ifdef CONFIG_SPL_LOAD_FIT > +int board_fit_config_name_match(const char *name) > +{ > + /* boot using first FIT config */ > + return 0; > +} > +#endif > diff --git a/configs/ae350_rv32_spl_defconfig b/configs/ae350_rv32_spl_defconfig > new file mode 100644 > index 0000000..53055b7 > --- /dev/null > +++ b/configs/ae350_rv32_spl_defconfig > @@ -0,0 +1,37 @@ > +CONFIG_RISCV=y > +CONFIG_SPL=y > +CONFIG_RISCV_SMODE=y > +CONFIG_SYS_TEXT_BASE=0x01200000 > +CONFIG_NR_DRAM_BANKS=2 > +CONFIG_TARGET_AX25_AE350=y > +CONFIG_DISTRO_DEFAULTS=y > +CONFIG_FIT=y > +CONFIG_BOOTDELAY=3 > +CONFIG_BOARD_EARLY_INIT_F=y > +CONFIG_SYS_PROMPT="RISC-V # " > +CONFIG_CMD_IMLS=y > +CONFIG_CMD_MMC=y > +CONFIG_CMD_SF=y > +CONFIG_CMD_SF_TEST=y > +# CONFIG_CMD_SETEXPR is not set > +CONFIG_BOOTP_PREFER_SERVERIP=y > +CONFIG_CMD_CACHE=y > +CONFIG_OF_PRIOR_STAGE=y > +CONFIG_ENV_IS_IN_SPI_FLASH=y > +CONFIG_NET_RANDOM_ETHADDR=y > +CONFIG_MMC=y > +CONFIG_FTSDC010=y > +CONFIG_FTSDC010_SDIO=y > +CONFIG_MTD_NOR_FLASH=y > +CONFIG_FLASH_CFI_DRIVER=y > +CONFIG_CFI_FLASH=y > +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y > +CONFIG_SYS_FLASH_CFI=y > +CONFIG_SPI_FLASH=y > +CONFIG_SF_DEFAULT_MODE=0x0 > +CONFIG_SPI_FLASH_MACRONIX=y > +CONFIG_FTMAC100=y > +CONFIG_BAUDRATE=38400 > +CONFIG_SYS_NS16550=y > +CONFIG_SPI=y > +CONFIG_ATCSPI200_SPI=y > diff --git a/configs/ae350_rv32_spl_xip_defconfig b/configs/ae350_rv32_spl_xip_defconfig > new file mode 100644 > index 0000000..fdbab43 > --- /dev/null > +++ b/configs/ae350_rv32_spl_xip_defconfig > @@ -0,0 +1,39 @@ > +CONFIG_RISCV=y > +CONFIG_SPL=y > +CONFIG_RISCV_SMODE=y > +CONFIG_SPL_TEXT_BASE=0x80000000 > +CONFIG_SYS_TEXT_BASE=0x01200000 > +CONFIG_NR_DRAM_BANKS=2 > +CONFIG_TARGET_AX25_AE350=y > +CONFIG_XIP=y > +CONFIG_DISTRO_DEFAULTS=y > +CONFIG_FIT=y > +CONFIG_BOOTDELAY=3 > +CONFIG_BOARD_EARLY_INIT_F=y > +CONFIG_SYS_PROMPT="RISC-V # " > +CONFIG_CMD_IMLS=y > +CONFIG_CMD_MMC=y > +CONFIG_CMD_SF=y > +CONFIG_CMD_SF_TEST=y > +# CONFIG_CMD_SETEXPR is not set > +CONFIG_BOOTP_PREFER_SERVERIP=y > +CONFIG_CMD_CACHE=y > +CONFIG_OF_BOARD=y > +CONFIG_ENV_IS_IN_SPI_FLASH=y > +CONFIG_NET_RANDOM_ETHADDR=y > +CONFIG_MMC=y > +CONFIG_FTSDC010=y > +CONFIG_FTSDC010_SDIO=y > +CONFIG_MTD_NOR_FLASH=y > +CONFIG_FLASH_CFI_DRIVER=y > +CONFIG_CFI_FLASH=y > +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y > +CONFIG_SYS_FLASH_CFI=y > +CONFIG_SPI_FLASH=y > +CONFIG_SF_DEFAULT_MODE=0x0 > +CONFIG_SPI_FLASH_MACRONIX=y > +CONFIG_FTMAC100=y > +CONFIG_BAUDRATE=38400 > +CONFIG_SYS_NS16550=y > +CONFIG_SPI=y > +CONFIG_ATCSPI200_SPI=y > diff --git a/configs/ae350_rv64_spl_defconfig b/configs/ae350_rv64_spl_defconfig > new file mode 100644 > index 0000000..866b9b4 > --- /dev/null > +++ b/configs/ae350_rv64_spl_defconfig > @@ -0,0 +1,38 @@ > +CONFIG_RISCV=y > +CONFIG_SPL=y > +CONFIG_RISCV_SMODE=y > +CONFIG_SYS_TEXT_BASE=0x01200000 > +CONFIG_NR_DRAM_BANKS=2 > +CONFIG_TARGET_AX25_AE350=y > +CONFIG_ARCH_RV64I=y > +CONFIG_DISTRO_DEFAULTS=y > +CONFIG_FIT=y > +CONFIG_BOOTDELAY=3 > +CONFIG_BOARD_EARLY_INIT_F=y > +CONFIG_SYS_PROMPT="RISC-V # " > +CONFIG_CMD_IMLS=y > +CONFIG_CMD_MMC=y > +CONFIG_CMD_SF=y > +CONFIG_CMD_SF_TEST=y > +# CONFIG_CMD_SETEXPR is not set > +CONFIG_BOOTP_PREFER_SERVERIP=y > +CONFIG_CMD_CACHE=y > +CONFIG_OF_PRIOR_STAGE=y > +CONFIG_ENV_IS_IN_SPI_FLASH=y > +CONFIG_NET_RANDOM_ETHADDR=y > +CONFIG_MMC=y > +CONFIG_FTSDC010=y > +CONFIG_FTSDC010_SDIO=y > +CONFIG_MTD_NOR_FLASH=y > +CONFIG_FLASH_CFI_DRIVER=y > +CONFIG_CFI_FLASH=y > +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y > +CONFIG_SYS_FLASH_CFI=y > +CONFIG_SPI_FLASH=y > +CONFIG_SF_DEFAULT_MODE=0x0 > +CONFIG_SPI_FLASH_MACRONIX=y > +CONFIG_FTMAC100=y > +CONFIG_BAUDRATE=38400 > +CONFIG_SYS_NS16550=y > +CONFIG_SPI=y > +CONFIG_ATCSPI200_SPI=y > diff --git a/configs/ae350_rv64_spl_xip_defconfig b/configs/ae350_rv64_spl_xip_defconfig > new file mode 100644 > index 0000000..271ec21 > --- /dev/null > +++ b/configs/ae350_rv64_spl_xip_defconfig > @@ -0,0 +1,40 @@ > +CONFIG_RISCV=y > +CONFIG_SPL=y > +CONFIG_RISCV_SMODE=y > +CONFIG_SPL_TEXT_BASE=0x80000000 > +CONFIG_SYS_TEXT_BASE=0x01200000 > +CONFIG_NR_DRAM_BANKS=2 > +CONFIG_TARGET_AX25_AE350=y > +CONFIG_ARCH_RV64I=y > +CONFIG_XIP=y > +CONFIG_DISTRO_DEFAULTS=y > +CONFIG_FIT=y > +CONFIG_BOOTDELAY=3 > +CONFIG_BOARD_EARLY_INIT_F=y > +CONFIG_SYS_PROMPT="RISC-V # " > +CONFIG_CMD_IMLS=y > +CONFIG_CMD_MMC=y > +CONFIG_CMD_SF=y > +CONFIG_CMD_SF_TEST=y > +# CONFIG_CMD_SETEXPR is not set > +CONFIG_BOOTP_PREFER_SERVERIP=y > +CONFIG_CMD_CACHE=y > +CONFIG_OF_BOARD=y > +CONFIG_ENV_IS_IN_SPI_FLASH=y > +CONFIG_NET_RANDOM_ETHADDR=y > +CONFIG_MMC=y > +CONFIG_FTSDC010=y > +CONFIG_FTSDC010_SDIO=y > +CONFIG_MTD_NOR_FLASH=y > +CONFIG_FLASH_CFI_DRIVER=y > +CONFIG_CFI_FLASH=y > +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y > +CONFIG_SYS_FLASH_CFI=y > +CONFIG_SPI_FLASH=y > +CONFIG_SF_DEFAULT_MODE=0x0 > +CONFIG_SPI_FLASH_MACRONIX=y > +CONFIG_FTMAC100=y > +CONFIG_BAUDRATE=38400 > +CONFIG_SYS_NS16550=y > +CONFIG_SPI=y > +CONFIG_ATCSPI200_SPI=y > diff --git a/include/configs/ax25-ae350.h b/include/configs/ax25-ae350.h > index a4037f3..ef3864a 100644 > --- a/include/configs/ax25-ae350.h > +++ b/include/configs/ax25-ae350.h > @@ -7,6 +7,23 @@ > #ifndef __CONFIG_H > #define __CONFIG_H > > +#ifdef CONFIG_SPL > +#define CONFIG_SPL_MAX_SIZE 0x00100000 > +#define CONFIG_SPL_BSS_START_ADDR 0x04000000 > +#define CONFIG_SPL_BSS_MAX_SIZE 0x00100000 > + > +#ifndef CONFIG_XIP > +#define CONFIG_SPL_LOAD_FIT_ADDRESS 0x00200000 > +#else > +#define CONFIG_SPL_LOAD_FIT_ADDRESS 0x80010000 > +#endif > + > +#ifdef CONFIG_SPL_MMC_SUPPORT > +#define CONFIG_SYS_MMCSD_FS_BOOT_PARTITION 1 > +#define CONFIG_SPL_FS_LOAD_PAYLOAD_NAME "u-boot.itb" > +#endif > +#endif > + > /* > * CPU and Board Configuration Options > */ > -- Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 2/8] riscv: ax25-ae350: add SPL configuration 2019-10-29 14:39 ` Bin Meng @ 2019-10-30 2:06 ` Rick Chen 0 siblings, 0 replies; 71+ messages in thread From: Rick Chen @ 2019-10-30 2:06 UTC (permalink / raw) To: u-boot Hi Bin > Hi Rick, > > On Fri, Oct 25, 2019 at 2:17 PM Andes <uboot@andestech.com> wrote: > > > > From: Rick Chen <rick@andestech.com> > > > > This patch provides four configurations > > which can support U-Boot SPL to boot from > > RAM or FLASH and then boot FIT image > > including OpenSBI FW_DYNAMIC firmware > > and U-Boot proper images from RAM or > > MMC boot devices. > > > > With ae350_rv[32|64]_spl_defconfigs: > > > > U-Boot SPL will be loaded by gdb or FSBL > > and runs in RAM in machine mode and then > > load FIT image from RAM device on AE350. > > > > With ae350_rv[32|64]_spl_xip_defconfigs: > > > > U-Boot SPL can be burned into SPI flash > > and run in flash in machine mode and then > > load FIT image from SPI flash or MMC device > > on AE350. > > > > Could you please rewrite the commit message so that every line has > about 70 characters? OK. I will rewrite it. > > > Signed-off-by: Rick Chen <rick@andestech.com> > > Cc: KC Lin <kclin@andestech.com> > > Cc: Alan Kao <alankao@andestech.com> > > --- > > board/AndesTech/ax25-ae350/Kconfig | 10 +++++++++ > > board/AndesTech/ax25-ae350/MAINTAINERS | 4 ++++ > > board/AndesTech/ax25-ae350/ax25-ae350.c | 27 ++++++++++++++++++++++ > > configs/ae350_rv32_spl_defconfig | 37 ++++++++++++++++++++++++++++++ > > configs/ae350_rv32_spl_xip_defconfig | 39 ++++++++++++++++++++++++++++++++ > > configs/ae350_rv64_spl_defconfig | 38 +++++++++++++++++++++++++++++++ > > configs/ae350_rv64_spl_xip_defconfig | 40 +++++++++++++++++++++++++++++++++ > > include/configs/ax25-ae350.h | 17 ++++++++++++++ > > 8 files changed, 212 insertions(+) > > create mode 100644 configs/ae350_rv32_spl_defconfig > > create mode 100644 configs/ae350_rv32_spl_xip_defconfig > > create mode 100644 configs/ae350_rv64_spl_defconfig > > create mode 100644 configs/ae350_rv64_spl_xip_defconfig > > > > diff --git a/board/AndesTech/ax25-ae350/Kconfig b/board/AndesTech/ax25-ae350/Kconfig > > index 5e682b6..2e1e2bb 100644 > > --- a/board/AndesTech/ax25-ae350/Kconfig > > +++ b/board/AndesTech/ax25-ae350/Kconfig > > @@ -21,9 +21,19 @@ config ENV_SIZE > > config ENV_OFFSET > > default 0x140000 if ENV_IS_IN_SPI_FLASH > > > > +config SPL_TEXT_BASE > > + default 0x00000000 > > + > > +config SPL_OPENSBI_LOAD_ADDR > > + default 0x01000000 > > + > > config BOARD_SPECIFIC_OPTIONS # dummy > > def_bool y > > select RISCV_NDS > > + select SUPPORT_SPL > > + imply SYS_NS16550 > > Is there any 16550 on the AX25 boards? If not, please remove. If yes, > this should be a separate patch with DTS changes. Yes, it needs SYS_NS16550, but it has been declared in ae350_XXX_defconfig. It is duplicate declaration here. I will remove it. > > > imply SMP > > + imply SPL_RAM_SUPPORT > > + imply SPL_RAM_DEVICE > > > > endif > > diff --git a/board/AndesTech/ax25-ae350/MAINTAINERS b/board/AndesTech/ax25-ae350/MAINTAINERS > > index feed5d1..eebee16 100644 > > --- a/board/AndesTech/ax25-ae350/MAINTAINERS > > +++ b/board/AndesTech/ax25-ae350/MAINTAINERS > > @@ -7,3 +7,7 @@ F: configs/ae350_rv32_defconfig > > F: configs/ae350_rv64_defconfig > > F: configs/ae350_rv32_xip_defconfig > > F: configs/ae350_rv64_xip_defconfig > > +F: configs/ae350_rv32_spl_defconfig > > +F: configs/ae350_rv64_spl_defconfig > > +F: configs/ae350_rv32_spl_xip_defconfig > > +F: configs/ae350_rv64_spl_xip_defconfig > > Could you please update the doc/board/AndesTech/ax25-ae350.rst to add > descriptions about all these different configurations of the same > board? OK. I will update doc/board/AndesTech/ax25-ae350.rst. Thanks Rick > > > diff --git a/board/AndesTech/ax25-ae350/ax25-ae350.c b/board/AndesTech/ax25-ae350/ax25-ae350.c > > index b43eebb..b0164a9 100644 > > --- a/board/AndesTech/ax25-ae350/ax25-ae350.c > > +++ b/board/AndesTech/ax25-ae350/ax25-ae350.c > > @@ -12,6 +12,7 @@ > > #include <faraday/ftsmc020.h> > > #include <fdtdec.h> > > #include <dm.h> > > +#include <spl.h> > > > > DECLARE_GLOBAL_DATA_PTR; > > > > @@ -110,3 +111,29 @@ int board_early_init_f(void) > > return 0; > > } > > #endif > > + > > +#ifdef CONFIG_SPL > > +void board_boot_order(u32 *spl_boot_list) > > +{ > > + u8 i; > > + u32 boot_devices[] = { > > +#ifdef CONFIG_SPL_RAM_SUPPORT > > + BOOT_DEVICE_RAM, > > +#endif > > +#ifdef CONFIG_SPL_MMC_SUPPORT > > + BOOT_DEVICE_MMC1, > > +#endif > > + }; > > + > > + for (i = 0; i < ARRAY_SIZE(boot_devices); i++) > > + spl_boot_list[i] = boot_devices[i]; > > +} > > +#endif > > + > > +#ifdef CONFIG_SPL_LOAD_FIT > > +int board_fit_config_name_match(const char *name) > > +{ > > + /* boot using first FIT config */ > > + return 0; > > +} > > +#endif > > diff --git a/configs/ae350_rv32_spl_defconfig b/configs/ae350_rv32_spl_defconfig > > new file mode 100644 > > index 0000000..53055b7 > > --- /dev/null > > +++ b/configs/ae350_rv32_spl_defconfig > > @@ -0,0 +1,37 @@ > > +CONFIG_RISCV=y > > +CONFIG_SPL=y > > +CONFIG_RISCV_SMODE=y > > +CONFIG_SYS_TEXT_BASE=0x01200000 > > +CONFIG_NR_DRAM_BANKS=2 > > +CONFIG_TARGET_AX25_AE350=y > > +CONFIG_DISTRO_DEFAULTS=y > > +CONFIG_FIT=y > > +CONFIG_BOOTDELAY=3 > > +CONFIG_BOARD_EARLY_INIT_F=y > > +CONFIG_SYS_PROMPT="RISC-V # " > > +CONFIG_CMD_IMLS=y > > +CONFIG_CMD_MMC=y > > +CONFIG_CMD_SF=y > > +CONFIG_CMD_SF_TEST=y > > +# CONFIG_CMD_SETEXPR is not set > > +CONFIG_BOOTP_PREFER_SERVERIP=y > > +CONFIG_CMD_CACHE=y > > +CONFIG_OF_PRIOR_STAGE=y > > +CONFIG_ENV_IS_IN_SPI_FLASH=y > > +CONFIG_NET_RANDOM_ETHADDR=y > > +CONFIG_MMC=y > > +CONFIG_FTSDC010=y > > +CONFIG_FTSDC010_SDIO=y > > +CONFIG_MTD_NOR_FLASH=y > > +CONFIG_FLASH_CFI_DRIVER=y > > +CONFIG_CFI_FLASH=y > > +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y > > +CONFIG_SYS_FLASH_CFI=y > > +CONFIG_SPI_FLASH=y > > +CONFIG_SF_DEFAULT_MODE=0x0 > > +CONFIG_SPI_FLASH_MACRONIX=y > > +CONFIG_FTMAC100=y > > +CONFIG_BAUDRATE=38400 > > +CONFIG_SYS_NS16550=y > > +CONFIG_SPI=y > > +CONFIG_ATCSPI200_SPI=y > > diff --git a/configs/ae350_rv32_spl_xip_defconfig b/configs/ae350_rv32_spl_xip_defconfig > > new file mode 100644 > > index 0000000..fdbab43 > > --- /dev/null > > +++ b/configs/ae350_rv32_spl_xip_defconfig > > @@ -0,0 +1,39 @@ > > +CONFIG_RISCV=y > > +CONFIG_SPL=y > > +CONFIG_RISCV_SMODE=y > > +CONFIG_SPL_TEXT_BASE=0x80000000 > > +CONFIG_SYS_TEXT_BASE=0x01200000 > > +CONFIG_NR_DRAM_BANKS=2 > > +CONFIG_TARGET_AX25_AE350=y > > +CONFIG_XIP=y > > +CONFIG_DISTRO_DEFAULTS=y > > +CONFIG_FIT=y > > +CONFIG_BOOTDELAY=3 > > +CONFIG_BOARD_EARLY_INIT_F=y > > +CONFIG_SYS_PROMPT="RISC-V # " > > +CONFIG_CMD_IMLS=y > > +CONFIG_CMD_MMC=y > > +CONFIG_CMD_SF=y > > +CONFIG_CMD_SF_TEST=y > > +# CONFIG_CMD_SETEXPR is not set > > +CONFIG_BOOTP_PREFER_SERVERIP=y > > +CONFIG_CMD_CACHE=y > > +CONFIG_OF_BOARD=y > > +CONFIG_ENV_IS_IN_SPI_FLASH=y > > +CONFIG_NET_RANDOM_ETHADDR=y > > +CONFIG_MMC=y > > +CONFIG_FTSDC010=y > > +CONFIG_FTSDC010_SDIO=y > > +CONFIG_MTD_NOR_FLASH=y > > +CONFIG_FLASH_CFI_DRIVER=y > > +CONFIG_CFI_FLASH=y > > +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y > > +CONFIG_SYS_FLASH_CFI=y > > +CONFIG_SPI_FLASH=y > > +CONFIG_SF_DEFAULT_MODE=0x0 > > +CONFIG_SPI_FLASH_MACRONIX=y > > +CONFIG_FTMAC100=y > > +CONFIG_BAUDRATE=38400 > > +CONFIG_SYS_NS16550=y > > +CONFIG_SPI=y > > +CONFIG_ATCSPI200_SPI=y > > diff --git a/configs/ae350_rv64_spl_defconfig b/configs/ae350_rv64_spl_defconfig > > new file mode 100644 > > index 0000000..866b9b4 > > --- /dev/null > > +++ b/configs/ae350_rv64_spl_defconfig > > @@ -0,0 +1,38 @@ > > +CONFIG_RISCV=y > > +CONFIG_SPL=y > > +CONFIG_RISCV_SMODE=y > > +CONFIG_SYS_TEXT_BASE=0x01200000 > > +CONFIG_NR_DRAM_BANKS=2 > > +CONFIG_TARGET_AX25_AE350=y > > +CONFIG_ARCH_RV64I=y > > +CONFIG_DISTRO_DEFAULTS=y > > +CONFIG_FIT=y > > +CONFIG_BOOTDELAY=3 > > +CONFIG_BOARD_EARLY_INIT_F=y > > +CONFIG_SYS_PROMPT="RISC-V # " > > +CONFIG_CMD_IMLS=y > > +CONFIG_CMD_MMC=y > > +CONFIG_CMD_SF=y > > +CONFIG_CMD_SF_TEST=y > > +# CONFIG_CMD_SETEXPR is not set > > +CONFIG_BOOTP_PREFER_SERVERIP=y > > +CONFIG_CMD_CACHE=y > > +CONFIG_OF_PRIOR_STAGE=y > > +CONFIG_ENV_IS_IN_SPI_FLASH=y > > +CONFIG_NET_RANDOM_ETHADDR=y > > +CONFIG_MMC=y > > +CONFIG_FTSDC010=y > > +CONFIG_FTSDC010_SDIO=y > > +CONFIG_MTD_NOR_FLASH=y > > +CONFIG_FLASH_CFI_DRIVER=y > > +CONFIG_CFI_FLASH=y > > +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y > > +CONFIG_SYS_FLASH_CFI=y > > +CONFIG_SPI_FLASH=y > > +CONFIG_SF_DEFAULT_MODE=0x0 > > +CONFIG_SPI_FLASH_MACRONIX=y > > +CONFIG_FTMAC100=y > > +CONFIG_BAUDRATE=38400 > > +CONFIG_SYS_NS16550=y > > +CONFIG_SPI=y > > +CONFIG_ATCSPI200_SPI=y > > diff --git a/configs/ae350_rv64_spl_xip_defconfig b/configs/ae350_rv64_spl_xip_defconfig > > new file mode 100644 > > index 0000000..271ec21 > > --- /dev/null > > +++ b/configs/ae350_rv64_spl_xip_defconfig > > @@ -0,0 +1,40 @@ > > +CONFIG_RISCV=y > > +CONFIG_SPL=y > > +CONFIG_RISCV_SMODE=y > > +CONFIG_SPL_TEXT_BASE=0x80000000 > > +CONFIG_SYS_TEXT_BASE=0x01200000 > > +CONFIG_NR_DRAM_BANKS=2 > > +CONFIG_TARGET_AX25_AE350=y > > +CONFIG_ARCH_RV64I=y > > +CONFIG_XIP=y > > +CONFIG_DISTRO_DEFAULTS=y > > +CONFIG_FIT=y > > +CONFIG_BOOTDELAY=3 > > +CONFIG_BOARD_EARLY_INIT_F=y > > +CONFIG_SYS_PROMPT="RISC-V # " > > +CONFIG_CMD_IMLS=y > > +CONFIG_CMD_MMC=y > > +CONFIG_CMD_SF=y > > +CONFIG_CMD_SF_TEST=y > > +# CONFIG_CMD_SETEXPR is not set > > +CONFIG_BOOTP_PREFER_SERVERIP=y > > +CONFIG_CMD_CACHE=y > > +CONFIG_OF_BOARD=y > > +CONFIG_ENV_IS_IN_SPI_FLASH=y > > +CONFIG_NET_RANDOM_ETHADDR=y > > +CONFIG_MMC=y > > +CONFIG_FTSDC010=y > > +CONFIG_FTSDC010_SDIO=y > > +CONFIG_MTD_NOR_FLASH=y > > +CONFIG_FLASH_CFI_DRIVER=y > > +CONFIG_CFI_FLASH=y > > +CONFIG_SYS_FLASH_USE_BUFFER_WRITE=y > > +CONFIG_SYS_FLASH_CFI=y > > +CONFIG_SPI_FLASH=y > > +CONFIG_SF_DEFAULT_MODE=0x0 > > +CONFIG_SPI_FLASH_MACRONIX=y > > +CONFIG_FTMAC100=y > > +CONFIG_BAUDRATE=38400 > > +CONFIG_SYS_NS16550=y > > +CONFIG_SPI=y > > +CONFIG_ATCSPI200_SPI=y > > diff --git a/include/configs/ax25-ae350.h b/include/configs/ax25-ae350.h > > index a4037f3..ef3864a 100644 > > --- a/include/configs/ax25-ae350.h > > +++ b/include/configs/ax25-ae350.h > > @@ -7,6 +7,23 @@ > > #ifndef __CONFIG_H > > #define __CONFIG_H > > > > +#ifdef CONFIG_SPL > > +#define CONFIG_SPL_MAX_SIZE 0x00100000 > > +#define CONFIG_SPL_BSS_START_ADDR 0x04000000 > > +#define CONFIG_SPL_BSS_MAX_SIZE 0x00100000 > > + > > +#ifndef CONFIG_XIP > > +#define CONFIG_SPL_LOAD_FIT_ADDRESS 0x00200000 > > +#else > > +#define CONFIG_SPL_LOAD_FIT_ADDRESS 0x80010000 > > +#endif > > + > > +#ifdef CONFIG_SPL_MMC_SUPPORT > > +#define CONFIG_SYS_MMCSD_FS_BOOT_PARTITION 1 > > +#define CONFIG_SPL_FS_LOAD_PAYLOAD_NAME "u-boot.itb" > > +#endif > > +#endif > > + > > /* > > * CPU and Board Configuration Options > > */ > > -- > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 3/8] riscv: ax25-ae350: Use generic memory size setup 2019-10-25 6:10 [U-Boot] [PATCH 0/8] RISC-V AX25-AE350 support SPL Andes 2019-10-25 6:10 ` [U-Boot] [PATCH 1/8] riscv: ax25: add SPL support Andes 2019-10-25 6:10 ` [U-Boot] [PATCH 2/8] riscv: ax25-ae350: add SPL configuration Andes @ 2019-10-25 6:10 ` Andes 2019-10-29 14:42 ` Bin Meng 2019-10-25 6:10 ` [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations Andes ` (4 subsequent siblings) 7 siblings, 1 reply; 71+ messages in thread From: Andes @ 2019-10-25 6:10 UTC (permalink / raw) To: u-boot From: Rick Chen <rick@andestech.com> To get memory size from device tree instead of get_ram_size(). This can avoid memory access fault in U-Boot proper after PMP configurations in OpenSbi. Signed-off-by: Rick Chen <rick@andestech.com> Cc: KC Lin <kclin@andestech.com> Cc: Alan Kao <alankao@andestech.com> --- board/AndesTech/ax25-ae350/ax25-ae350.c | 21 ++------------------- 1 file changed, 2 insertions(+), 19 deletions(-) diff --git a/board/AndesTech/ax25-ae350/ax25-ae350.c b/board/AndesTech/ax25-ae350/ax25-ae350.c index b0164a9..47e6929 100644 --- a/board/AndesTech/ax25-ae350/ax25-ae350.c +++ b/board/AndesTech/ax25-ae350/ax25-ae350.c @@ -30,29 +30,12 @@ int board_init(void) int dram_init(void) { - unsigned long sdram_base = PHYS_SDRAM_0; - unsigned long expected_size = PHYS_SDRAM_0_SIZE + PHYS_SDRAM_1_SIZE; - unsigned long actual_size; - - actual_size = get_ram_size((void *)sdram_base, expected_size); - gd->ram_size = actual_size; - - if (expected_size != actual_size) { - printf("Warning: Only %lu of %lu MiB SDRAM is working\n", - actual_size >> 20, expected_size >> 20); - } - - return 0; + return fdtdec_setup_mem_size_base(); } int dram_init_banksize(void) { - gd->bd->bi_dram[0].start = PHYS_SDRAM_0; - gd->bd->bi_dram[0].size = PHYS_SDRAM_0_SIZE; - gd->bd->bi_dram[1].start = PHYS_SDRAM_1; - gd->bd->bi_dram[1].size = PHYS_SDRAM_1_SIZE; - - return 0; + return fdtdec_setup_memory_banksize(); } #if defined(CONFIG_FTMAC100) && !defined(CONFIG_DM_ETH) -- 2.7.4 ^ permalink raw reply related [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 3/8] riscv: ax25-ae350: Use generic memory size setup 2019-10-25 6:10 ` [U-Boot] [PATCH 3/8] riscv: ax25-ae350: Use generic memory size setup Andes @ 2019-10-29 14:42 ` Bin Meng 2019-10-30 2:19 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-29 14:42 UTC (permalink / raw) To: u-boot Hi Rick, On Fri, Oct 25, 2019 at 2:17 PM Andes <uboot@andestech.com> wrote: > > From: Rick Chen <rick@andestech.com> > > To get memory size from device tree instead of > get_ram_size(). This can avoid memory access fault Could you please explain a little more about why get_ram_size() causes PMP access fault? > in U-Boot proper after PMP configurations in OpenSbi. > > Signed-off-by: Rick Chen <rick@andestech.com> > Cc: KC Lin <kclin@andestech.com> > Cc: Alan Kao <alankao@andestech.com> > --- > board/AndesTech/ax25-ae350/ax25-ae350.c | 21 ++------------------- > 1 file changed, 2 insertions(+), 19 deletions(-) > Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 3/8] riscv: ax25-ae350: Use generic memory size setup 2019-10-29 14:42 ` Bin Meng @ 2019-10-30 2:19 ` Rick Chen 0 siblings, 0 replies; 71+ messages in thread From: Rick Chen @ 2019-10-30 2:19 UTC (permalink / raw) To: u-boot Hi Bin Bin Meng <bmeng.cn@gmail.com> 於 2019年10月29日 週二 下午10:42寫道: > > Hi Rick, > > On Fri, Oct 25, 2019 at 2:17 PM Andes <uboot@andestech.com> wrote: > > > > From: Rick Chen <rick@andestech.com> > > > > To get memory size from device tree instead of > > get_ram_size(). This can avoid memory access fault > > Could you please explain a little more about why get_ram_size() causes > PMP access fault? OpenSBI will configure pmpcfg and pmpaddr for memory protection in specific range. When get_ram_size() try to store or load memory address, it will appear access fault exception. Thanks Rick > > > in U-Boot proper after PMP configurations in OpenSbi. > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > Cc: KC Lin <kclin@andestech.com> > > Cc: Alan Kao <alankao@andestech.com> > > --- > > board/AndesTech/ax25-ae350/ax25-ae350.c | 21 ++------------------- > > 1 file changed, 2 insertions(+), 19 deletions(-) > > > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-25 6:10 [U-Boot] [PATCH 0/8] RISC-V AX25-AE350 support SPL Andes ` (2 preceding siblings ...) 2019-10-25 6:10 ` [U-Boot] [PATCH 3/8] riscv: ax25-ae350: Use generic memory size setup Andes @ 2019-10-25 6:10 ` Andes 2019-10-29 14:51 ` Bin Meng 2019-10-25 6:10 ` [U-Boot] [PATCH 5/8] riscv: ax25: cache: Add SPL_RISCV_MMODE for SPL Andes ` (3 subsequent siblings) 7 siblings, 1 reply; 71+ messages in thread From: Andes @ 2019-10-25 6:10 UTC (permalink / raw) To: u-boot From: Rick Chen <rick@andestech.com> It will work fine due to hart 0 always will be main hart coincidentally. When develop SPL flow, I try to force other harts to be main hart. And it will go wrong in sending IPI flow. So fix it. Having this fix, any hart can be main hart in U-Boot SPL theoretically, but it still fail somewhere. After dig in and found there is an assumption that hart 0 shall be main hart in OpenSbi. After some work-arounds, it can pass the verifications that any hart can be main hart and boots U-Boot SPL -> OpenSbi -> U-Boot proper -> Linux Kernel successfully. Signed-off-by: Rick Chen <rick@andestech.com> Cc: KC Lin <kclin@andestech.com> Cc: Alan Kao <alankao@andestech.com> --- arch/riscv/lib/andes_plic.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c index 28568e4..42394b9 100644 --- a/arch/riscv/lib/andes_plic.c +++ b/arch/riscv/lib/andes_plic.c @@ -19,7 +19,7 @@ #include <cpu.h> /* pending register */ -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) /* enable register */ #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) /* claim register */ @@ -46,7 +46,7 @@ static int init_plic(void); static int enable_ipi(int hart) { - int en; + unsigned int en; en = ENABLE_HART_IPI >> hart; writel(en, (void __iomem *)ENABLE_REG(gd->arch.plic, hart)); @@ -94,10 +94,13 @@ static int init_plic(void) int riscv_send_ipi(int hart) { + unsigned int ipi; + PLIC_BASE_GET(); - writel(SEND_IPI_TO_HART(hart), - (void __iomem *)PENDING_REG(gd->arch.plic, gd->arch.boot_hart)); + ipi = (SEND_IPI_TO_HART(hart) << (8 * gd->arch.boot_hart)); + writel(ipi, (void __iomem *)PENDING_REG(gd->arch.plic, + gd->arch.boot_hart)); return 0; } -- 2.7.4 ^ permalink raw reply related [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-25 6:10 ` [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations Andes @ 2019-10-29 14:51 ` Bin Meng 2019-10-30 2:50 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-29 14:51 UTC (permalink / raw) To: u-boot Hi Rick, On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > From: Rick Chen <rick@andestech.com> > > It will work fine due to hart 0 always will be main > hart coincidentally. When develop SPL flow, I try to > force other harts to be main hart. And it will go > wrong in sending IPI flow. So fix it. Fix what? Does this commit contain 2 fixes, or just 1 fix? > > Having this fix, any hart can be main hart in U-Boot SPL > theoretically, but it still fail somewhere. After dig in > and found there is an assumption that hart 0 shall be > main hart in OpenSbi. So does this mean there is a bug in OpenSBI too? > > After some work-arounds, it can pass the verifications > that any hart can be main hart and boots U-Boot SPL -> > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > It's a bit hard for me to understand what was described here in the commit message. Maybe you need rewrite something. > Signed-off-by: Rick Chen <rick@andestech.com> > Cc: KC Lin <kclin@andestech.com> > Cc: Alan Kao <alankao@andestech.com> > --- > arch/riscv/lib/andes_plic.c | 11 +++++++---- > 1 file changed, 7 insertions(+), 4 deletions(-) > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > index 28568e4..42394b9 100644 > --- a/arch/riscv/lib/andes_plic.c > +++ b/arch/riscv/lib/andes_plic.c > @@ -19,7 +19,7 @@ > #include <cpu.h> > > /* pending register */ > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > /* enable register */ > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > /* claim register */ > @@ -46,7 +46,7 @@ static int init_plic(void); > > static int enable_ipi(int hart) > { > - int en; > + unsigned int en; Is this for some compiler warning fix? > > en = ENABLE_HART_IPI >> hart; > writel(en, (void __iomem *)ENABLE_REG(gd->arch.plic, hart)); > @@ -94,10 +94,13 @@ static int init_plic(void) > > int riscv_send_ipi(int hart) > { > + unsigned int ipi; > + > PLIC_BASE_GET(); > > - writel(SEND_IPI_TO_HART(hart), > - (void __iomem *)PENDING_REG(gd->arch.plic, gd->arch.boot_hart)); > + ipi = (SEND_IPI_TO_HART(hart) << (8 * gd->arch.boot_hart)); > + writel(ipi, (void __iomem *)PENDING_REG(gd->arch.plic, > + gd->arch.boot_hart)); > > return 0; > } > -- Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-29 14:51 ` Bin Meng @ 2019-10-30 2:50 ` Rick Chen 2019-10-30 10:38 ` Bin Meng 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-10-30 2:50 UTC (permalink / raw) To: u-boot Hi Bin > > Hi Rick, > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > From: Rick Chen <rick@andestech.com> > > > > It will work fine due to hart 0 always will be main > > hart coincidentally. When develop SPL flow, I try to > > force other harts to be main hart. And it will go > > wrong in sending IPI flow. So fix it. > > Fix what? Does this commit contain 2 fixes, or just 1 fix? Yes, it include two fixs. But they will cause one negative result that only hart 0 can send ipi to other harts. > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > theoretically, but it still fail somewhere. After dig in > > and found there is an assumption that hart 0 shall be > > main hart in OpenSbi. > > So does this mean there is a bug in OpenSBI too? I am not sure if it is a bug. Maybe it is a compatible issue. There is a limitation that only hart 0 can be main hart in OpenSBI. But any hart can be main hart in U-Boot. In general case, hart 0 will be main and it is fine when U-Boot jump ot OpenSBI. But if we force hart 1 to be main hart, when hart 0 jump to OPenSBI from U-Boot, It will do relocation flow in OpenSBI which willcorrupt U-Boot SPL, but hart 0 still in U-Boot SPL. > > > > > After some work-arounds, it can pass the verifications > > that any hart can be main hart and boots U-Boot SPL -> > > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > > > > It's a bit hard for me to understand what was described here in the > commit message. Maybe you need rewrite something. OK. I will rewrite this commit message. > > > Signed-off-by: Rick Chen <rick@andestech.com> > > Cc: KC Lin <kclin@andestech.com> > > Cc: Alan Kao <alankao@andestech.com> > > --- > > arch/riscv/lib/andes_plic.c | 11 +++++++---- > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > > index 28568e4..42394b9 100644 > > --- a/arch/riscv/lib/andes_plic.c > > +++ b/arch/riscv/lib/andes_plic.c > > @@ -19,7 +19,7 @@ > > #include <cpu.h> > > > > /* pending register */ > > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > > /* enable register */ > > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > > /* claim register */ > > @@ -46,7 +46,7 @@ static int init_plic(void); > > > > static int enable_ipi(int hart) > > { > > - int en; > > + unsigned int en; > > Is this for some compiler warning fix? No, it is not a warning fix. It is a bug fix. I hope en can be 0x0000000080808080 instead of 0xffffffff80808080, or it will cause IPI sending errors. Thanks Rick > > > > > en = ENABLE_HART_IPI >> hart; > > writel(en, (void __iomem *)ENABLE_REG(gd->arch.plic, hart)); > > @@ -94,10 +94,13 @@ static int init_plic(void) > > > > int riscv_send_ipi(int hart) > > { > > + unsigned int ipi; > > + > > PLIC_BASE_GET(); > > > > - writel(SEND_IPI_TO_HART(hart), > > - (void __iomem *)PENDING_REG(gd->arch.plic, gd->arch.boot_hart)); > > + ipi = (SEND_IPI_TO_HART(hart) << (8 * gd->arch.boot_hart)); > > + writel(ipi, (void __iomem *)PENDING_REG(gd->arch.plic, > > + gd->arch.boot_hart)); > > > > return 0; > > } > > -- > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-30 2:50 ` Rick Chen @ 2019-10-30 10:38 ` Bin Meng 2019-10-31 1:00 ` Alan Kao 2019-10-31 2:23 ` Rick Chen 0 siblings, 2 replies; 71+ messages in thread From: Bin Meng @ 2019-10-30 10:38 UTC (permalink / raw) To: u-boot Hi Rick, On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Bin > > > > > Hi Rick, > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > It will work fine due to hart 0 always will be main > > > hart coincidentally. When develop SPL flow, I try to > > > force other harts to be main hart. And it will go > > > wrong in sending IPI flow. So fix it. > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > Yes, it include two fixs. But they will cause one negative result > that only hart 0 can send ipi to other harts. > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > theoretically, but it still fail somewhere. After dig in > > > and found there is an assumption that hart 0 shall be > > > main hart in OpenSbi. > > > > So does this mean there is a bug in OpenSBI too? > > I am not sure if it is a bug. Maybe it is a compatible issue. > There is a limitation that only hart 0 can be main hart in OpenSBI. I don't think OpenSBI has such limitation. > But any hart can be main hart in U-Boot. > > In general case, hart 0 will be main and it is fine when U-Boot jump ot OpenSBI. > But if we force hart 1 to be main hart, when hart 0 jump to OPenSBI from U-Boot, > It will do relocation flow in OpenSBI which willcorrupt U-Boot SPL, > but hart 0 still in U-Boot SPL. > > > > > > > > > After some work-arounds, it can pass the verifications > > > that any hart can be main hart and boots U-Boot SPL -> > > > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > > > > > > > It's a bit hard for me to understand what was described here in the > > commit message. Maybe you need rewrite something. > > OK. I will rewrite this commit message. > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > Cc: KC Lin <kclin@andestech.com> > > > Cc: Alan Kao <alankao@andestech.com> > > > --- > > > arch/riscv/lib/andes_plic.c | 11 +++++++---- > > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > > > index 28568e4..42394b9 100644 > > > --- a/arch/riscv/lib/andes_plic.c > > > +++ b/arch/riscv/lib/andes_plic.c > > > @@ -19,7 +19,7 @@ > > > #include <cpu.h> > > > > > > /* pending register */ > > > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > > > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > > > /* enable register */ > > > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > > > /* claim register */ > > > @@ -46,7 +46,7 @@ static int init_plic(void); > > > > > > static int enable_ipi(int hart) > > > { > > > - int en; > > > + unsigned int en; > > > > Is this for some compiler warning fix? > > No, it is not a warning fix. It is a bug fix. > I hope en can be 0x0000000080808080 instead of 0xffffffff80808080, But it is int, which is only 32-bit. The example you gave was a 64-bit number. > or it will cause IPI sending errors. > Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-30 10:38 ` Bin Meng @ 2019-10-31 1:00 ` Alan Kao 2019-10-31 3:36 ` Bin Meng 2019-10-31 8:12 ` Anup Patel 2019-10-31 2:23 ` Rick Chen 1 sibling, 2 replies; 71+ messages in thread From: Alan Kao @ 2019-10-31 1:00 UTC (permalink / raw) To: u-boot Hi Bin, Thanks for the critics. Comments below. On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > Hi Rick, > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Bin > > > > > > > > Hi Rick, > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > It will work fine due to hart 0 always will be main > > > > hart coincidentally. When develop SPL flow, I try to > > > > force other harts to be main hart. And it will go > > > > wrong in sending IPI flow. So fix it. > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > Yes, it include two fixs. But they will cause one negative result > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > theoretically, but it still fail somewhere. After dig in > > > > and found there is an assumption that hart 0 shall be > > > > main hart in OpenSbi. > > > > > > So does this mean there is a bug in OpenSBI too? > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > I don't think OpenSBI has such limitation. > Please check the source. https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 Apparently, the FIRST TWO LINEs of the initialization are the 1. get hart ID. 2. determine which route to take based on their ID respectively. So, I do think OpenSBI has this signature, if you are not willing to call it a limitation. > > But any hart can be main hart in U-Boot. > > > > In general case, hart 0 will be main and it is fine when U-Boot jump ot OpenSBI. > > But if we force hart 1 to be main hart, when hart 0 jump to OPenSBI from U-Boot, > > It will do relocation flow in OpenSBI which willcorrupt U-Boot SPL, > > but hart 0 still in U-Boot SPL. > > > > > > > > > > > > > After some work-arounds, it can pass the verifications > > > > that any hart can be main hart and boots U-Boot SPL -> > > > > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > > > > > > > > > > It's a bit hard for me to understand what was described here in the > > > commit message. Maybe you need rewrite something. > > > > OK. I will rewrite this commit message. > > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > Cc: KC Lin <kclin@andestech.com> > > > > Cc: Alan Kao <alankao@andestech.com> > > > > --- > > > > arch/riscv/lib/andes_plic.c | 11 +++++++---- > > > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > > > > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > > > > index 28568e4..42394b9 100644 > > > > --- a/arch/riscv/lib/andes_plic.c > > > > +++ b/arch/riscv/lib/andes_plic.c > > > > @@ -19,7 +19,7 @@ > > > > #include <cpu.h> > > > > > > > > /* pending register */ > > > > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > > > > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > > > > /* enable register */ > > > > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > > > > /* claim register */ > > > > @@ -46,7 +46,7 @@ static int init_plic(void); > > > > > > > > static int enable_ipi(int hart) > > > > { > > > > - int en; > > > > + unsigned int en; > > > > > > Is this for some compiler warning fix? > > > > No, it is not a warning fix. It is a bug fix. > > I hope en can be 0x0000000080808080 instead of 0xffffffff80808080, > > But it is int, which is only 32-bit. The example you gave was a 64-bit number. > Please consider the following simple program: > #define MASK 0x80808080 >int main(){ > int en; > en = MASK; > printf("%x, shifted %x\n", en, en >> 1); > return 0; >} Would you mind sharing what you get after running this on your x86_64 (if you have one) computer? Really appreiciate that. The almost identical episode is in the patch, specifically, > en = ENABLE_HART_IPI >> hart > > or it will cause IPI sending errors. > > > > Regards, > Bin Best, Alan ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-31 1:00 ` Alan Kao @ 2019-10-31 3:36 ` Bin Meng 2019-10-31 7:48 ` Alan Kao 2019-10-31 8:12 ` Anup Patel 1 sibling, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-31 3:36 UTC (permalink / raw) To: u-boot Hi Alan, On Thu, Oct 31, 2019 at 9:00 AM Alan Kao <alankao@andestech.com> wrote: > > Hi Bin, > > Thanks for the critics. Comments below. > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > Hi Rick, > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > Hi Bin > > > > > > > > > > > Hi Rick, > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > force other harts to be main hart. And it will go > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > Yes, it include two fixs. But they will cause one negative result > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > theoretically, but it still fail somewhere. After dig in > > > > > and found there is an assumption that hart 0 shall be > > > > > main hart in OpenSbi. > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > I don't think OpenSBI has such limitation. > > > > Please check the source. > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > Apparently, the FIRST TWO LINEs of the initialization are the > 1. get hart ID. > 2. determine which route to take based on their ID respectively. > This is true only for the very first a few instructions when OpenSBI boots. Later OpenSBI main initialization does not require hart to be zero. > So, I do think OpenSBI has this signature, if you are not willing to call it > a limitation. > > > > But any hart can be main hart in U-Boot. > > > > > > In general case, hart 0 will be main and it is fine when U-Boot jump ot OpenSBI. > > > But if we force hart 1 to be main hart, when hart 0 jump to OPenSBI from U-Boot, > > > It will do relocation flow in OpenSBI which willcorrupt U-Boot SPL, > > > but hart 0 still in U-Boot SPL. > > > > > > > > > > > > > > > > > After some work-arounds, it can pass the verifications > > > > > that any hart can be main hart and boots U-Boot SPL -> > > > > > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > > > > > > > > > > > > > It's a bit hard for me to understand what was described here in the > > > > commit message. Maybe you need rewrite something. > > > > > > OK. I will rewrite this commit message. > > > > > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > > Cc: KC Lin <kclin@andestech.com> > > > > > Cc: Alan Kao <alankao@andestech.com> > > > > > --- > > > > > arch/riscv/lib/andes_plic.c | 11 +++++++---- > > > > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > > > > > > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > > > > > index 28568e4..42394b9 100644 > > > > > --- a/arch/riscv/lib/andes_plic.c > > > > > +++ b/arch/riscv/lib/andes_plic.c > > > > > @@ -19,7 +19,7 @@ > > > > > #include <cpu.h> > > > > > > > > > > /* pending register */ > > > > > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > > > > > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > > > > > /* enable register */ > > > > > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > > > > > /* claim register */ > > > > > @@ -46,7 +46,7 @@ static int init_plic(void); > > > > > > > > > > static int enable_ipi(int hart) > > > > > { > > > > > - int en; > > > > > + unsigned int en; > > > > > > > > Is this for some compiler warning fix? > > > > > > No, it is not a warning fix. It is a bug fix. > > > I hope en can be 0x0000000080808080 instead of 0xffffffff80808080, > > > > But it is int, which is only 32-bit. The example you gave was a 64-bit number. > > > > Please consider the following simple program: > > > #define MASK 0x80808080 > >int main(){ > > int en; > > en = MASK; > > printf("%x, shifted %x\n", en, en >> 1); > > return 0; > >} > > Would you mind sharing what you get after running this on your x86_64 > (if you have one) computer? Really appreiciate that. > > The almost identical episode is in the patch, specifically, > > en = ENABLE_HART_IPI >> hart Yes, this is a bug. But I was confused by Rick's comments as he was using a 64-bit number as int is never to be a 64-bit for both 32-bit and 64-bit. Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-31 3:36 ` Bin Meng @ 2019-10-31 7:48 ` Alan Kao 2019-10-31 8:10 ` Bin Meng 0 siblings, 1 reply; 71+ messages in thread From: Alan Kao @ 2019-10-31 7:48 UTC (permalink / raw) To: u-boot On Thu, Oct 31, 2019 at 11:36:48AM +0800, Bin Meng wrote: > Hi Alan, > > On Thu, Oct 31, 2019 at 9:00 AM Alan Kao <alankao@andestech.com> wrote: > > > > Hi Bin, > > > > Thanks for the critics. Comments below. > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > Hi Rick, > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > Hi Bin > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > force other harts to be main hart. And it will go > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > and found there is an assumption that hart 0 shall be > > > > > > main hart in OpenSbi. > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > I don't think OpenSBI has such limitation. > > > > > > > Please check the source. > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > 1. get hart ID. > > 2. determine which route to take based on their ID respectively. > > > > This is true only for the very first a few instructions when OpenSBI > boots. Later OpenSBI main initialization does not require hart to be > zero. > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > a limitation. > > > > > > But any hart can be main hart in U-Boot. > > > > > > > > In general case, hart 0 will be main and it is fine when U-Boot jump ot OpenSBI. > > > > But if we force hart 1 to be main hart, when hart 0 jump to OPenSBI from U-Boot, > > > > It will do relocation flow in OpenSBI which willcorrupt U-Boot SPL, > > > > but hart 0 still in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > After some work-arounds, it can pass the verifications > > > > > > that any hart can be main hart and boots U-Boot SPL -> > > > > > > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > > > > > > > > > > > > > > > > It's a bit hard for me to understand what was described here in the > > > > > commit message. Maybe you need rewrite something. > > > > > > > > OK. I will rewrite this commit message. > > > > > > > > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > > > Cc: KC Lin <kclin@andestech.com> > > > > > > Cc: Alan Kao <alankao@andestech.com> > > > > > > --- > > > > > > arch/riscv/lib/andes_plic.c | 11 +++++++---- > > > > > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > > > > > > > > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > > > > > > index 28568e4..42394b9 100644 > > > > > > --- a/arch/riscv/lib/andes_plic.c > > > > > > +++ b/arch/riscv/lib/andes_plic.c > > > > > > @@ -19,7 +19,7 @@ > > > > > > #include <cpu.h> > > > > > > > > > > > > /* pending register */ > > > > > > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > > > > > > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > > > > > > /* enable register */ > > > > > > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > > > > > > /* claim register */ > > > > > > @@ -46,7 +46,7 @@ static int init_plic(void); > > > > > > > > > > > > static int enable_ipi(int hart) > > > > > > { > > > > > > - int en; > > > > > > + unsigned int en; > > > > > > > > > > Is this for some compiler warning fix? > > > > > > > > No, it is not a warning fix. It is a bug fix. > > > > I hope en can be 0x0000000080808080 instead of 0xffffffff80808080, > > > > > > But it is int, which is only 32-bit. The example you gave was a 64-bit number. > > > > > > > Please consider the following simple program: > > > > > #define MASK 0x80808080 > > >int main(){ > > > int en; > > > en = MASK; > > > printf("%x, shifted %x\n", en, en >> 1); > > > return 0; > > >} > > > > Would you mind sharing what you get after running this on your x86_64 > > (if you have one) computer? Really appreiciate that. > > > > The almost identical episode is in the patch, specifically, > > > en = ENABLE_HART_IPI >> hart > > Yes, this is a bug. ... Wait, what do you mean but "this"? What is a bug here? If you want to be helpful, please also be specific or anyone else reviewing this patch will be confused. > ... But I was confused by Rick's comments as he was > using a 64-bit number as int is never to be a 64-bit for both 32-bit > and 64-bit. It was just an example. Nothing to do with bit width, but just a sign- extension issue. > > Regards, > Bin Thanks, Alan ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-31 7:48 ` Alan Kao @ 2019-10-31 8:10 ` Bin Meng 0 siblings, 0 replies; 71+ messages in thread From: Bin Meng @ 2019-10-31 8:10 UTC (permalink / raw) To: u-boot Hi Alan, On Thu, Oct 31, 2019 at 3:49 PM Alan Kao <alankao@andestech.com> wrote: > > > On Thu, Oct 31, 2019 at 11:36:48AM +0800, Bin Meng wrote: > > Hi Alan, > > > > On Thu, Oct 31, 2019 at 9:00 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > Hi Bin, > > > > > > Thanks for the critics. Comments below. > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > Hi Rick, > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > force other harts to be main hart. And it will go > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > Please check the source. > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > 1. get hart ID. > > > 2. determine which route to take based on their ID respectively. > > > > > > > This is true only for the very first a few instructions when OpenSBI > > boots. Later OpenSBI main initialization does not require hart to be > > zero. > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > a limitation. > > > > > > > > But any hart can be main hart in U-Boot. > > > > > > > > > > In general case, hart 0 will be main and it is fine when U-Boot jump ot OpenSBI. > > > > > But if we force hart 1 to be main hart, when hart 0 jump to OPenSBI from U-Boot, > > > > > It will do relocation flow in OpenSBI which willcorrupt U-Boot SPL, > > > > > but hart 0 still in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > After some work-arounds, it can pass the verifications > > > > > > > that any hart can be main hart and boots U-Boot SPL -> > > > > > > > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > > > > > > > > > > > > > > > > > > > It's a bit hard for me to understand what was described here in the > > > > > > commit message. Maybe you need rewrite something. > > > > > > > > > > OK. I will rewrite this commit message. > > > > > > > > > > > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > > > > Cc: KC Lin <kclin@andestech.com> > > > > > > > Cc: Alan Kao <alankao@andestech.com> > > > > > > > --- > > > > > > > arch/riscv/lib/andes_plic.c | 11 +++++++---- > > > > > > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > > > > > > > > > > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > > > > > > > index 28568e4..42394b9 100644 > > > > > > > --- a/arch/riscv/lib/andes_plic.c > > > > > > > +++ b/arch/riscv/lib/andes_plic.c > > > > > > > @@ -19,7 +19,7 @@ > > > > > > > #include <cpu.h> > > > > > > > > > > > > > > /* pending register */ > > > > > > > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > > > > > > > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > > > > > > > /* enable register */ > > > > > > > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > > > > > > > /* claim register */ > > > > > > > @@ -46,7 +46,7 @@ static int init_plic(void); > > > > > > > > > > > > > > static int enable_ipi(int hart) > > > > > > > { > > > > > > > - int en; > > > > > > > + unsigned int en; > > > > > > > > > > > > Is this for some compiler warning fix? > > > > > > > > > > No, it is not a warning fix. It is a bug fix. > > > > > I hope en can be 0x0000000080808080 instead of 0xffffffff80808080, > > > > > > > > But it is int, which is only 32-bit. The example you gave was a 64-bit number. > > > > > > > > > > Please consider the following simple program: > > > > > > > #define MASK 0x80808080 > > > >int main(){ > > > > int en; > > > > en = MASK; > > > > printf("%x, shifted %x\n", en, en >> 1); > > > > return 0; > > > >} > > > > > > Would you mind sharing what you get after running this on your x86_64 > > > (if you have one) computer? Really appreiciate that. > > > > > > The almost identical episode is in the patch, specifically, > > > > en = ENABLE_HART_IPI >> hart > > > > Yes, this is a bug. ... > > Wait, what do you mean but "this"? What is a bug here? The bug you mentioned. > If you want to be helpful, please also be specific or anyone else reviewing > this patch will be confused. It's just the explanation that Rick gave confuses people like me. > > > ... But I was confused by Rick's comments as he was > > using a 64-bit number as int is never to be a 64-bit for both 32-bit > > and 64-bit. > > It was just an example. Nothing to do with bit width, but just a sign- > extension issue. Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-31 1:00 ` Alan Kao 2019-10-31 3:36 ` Bin Meng @ 2019-10-31 8:12 ` Anup Patel 2019-10-31 10:43 ` Anup Patel 1 sibling, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-10-31 8:12 UTC (permalink / raw) To: u-boot On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > Hi Bin, > > Thanks for the critics. Comments below. > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > Hi Rick, > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > Hi Bin > > > > > > > > > > > Hi Rick, > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > force other harts to be main hart. And it will go > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > Yes, it include two fixs. But they will cause one negative result > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > theoretically, but it still fail somewhere. After dig in > > > > > and found there is an assumption that hart 0 shall be > > > > > main hart in OpenSbi. > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > I don't think OpenSBI has such limitation. > > > > Please check the source. > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > Apparently, the FIRST TWO LINEs of the initialization are the > 1. get hart ID. > 2. determine which route to take based on their ID respectively. > > So, I do think OpenSBI has this signature, if you are not willing to call it > a limitation. This dependency on hart id #0 was not there until we added self-relocation in OpenSBI for FW_DYNAMIC. I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > But any hart can be main hart in U-Boot. > > > > > > In general case, hart 0 will be main and it is fine when U-Boot jump ot OpenSBI. > > > But if we force hart 1 to be main hart, when hart 0 jump to OPenSBI from U-Boot, > > > It will do relocation flow in OpenSBI which willcorrupt U-Boot SPL, > > > but hart 0 still in U-Boot SPL. > > > > > > > > > > > > > > > > > After some work-arounds, it can pass the verifications > > > > > that any hart can be main hart and boots U-Boot SPL -> > > > > > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > > > > > > > > > > > > > It's a bit hard for me to understand what was described here in the > > > > commit message. Maybe you need rewrite something. > > > > > > OK. I will rewrite this commit message. > > > > > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > > Cc: KC Lin <kclin@andestech.com> > > > > > Cc: Alan Kao <alankao@andestech.com> > > > > > --- > > > > > arch/riscv/lib/andes_plic.c | 11 +++++++---- > > > > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > > > > > > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > > > > > index 28568e4..42394b9 100644 > > > > > --- a/arch/riscv/lib/andes_plic.c > > > > > +++ b/arch/riscv/lib/andes_plic.c > > > > > @@ -19,7 +19,7 @@ > > > > > #include <cpu.h> > > > > > > > > > > /* pending register */ > > > > > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > > > > > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > > > > > /* enable register */ > > > > > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > > > > > /* claim register */ > > > > > @@ -46,7 +46,7 @@ static int init_plic(void); > > > > > > > > > > static int enable_ipi(int hart) > > > > > { > > > > > - int en; > > > > > + unsigned int en; > > > > > > > > Is this for some compiler warning fix? > > > > > > No, it is not a warning fix. It is a bug fix. > > > I hope en can be 0x0000000080808080 instead of 0xffffffff80808080, > > > > But it is int, which is only 32-bit. The example you gave was a 64-bit number. > > > > Please consider the following simple program: > > > #define MASK 0x80808080 > >int main(){ > > int en; > > en = MASK; > > printf("%x, shifted %x\n", en, en >> 1); > > return 0; > >} > > Would you mind sharing what you get after running this on your x86_64 > (if you have one) computer? Really appreiciate that. > > The almost identical episode is in the patch, specifically, > > en = ENABLE_HART_IPI >> hart > > > > or it will cause IPI sending errors. > > > > > > > Regards, > > Bin > > Best, > Alan Regards, Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-31 8:12 ` Anup Patel @ 2019-10-31 10:43 ` Anup Patel 2019-11-01 5:25 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-10-31 10:43 UTC (permalink / raw) To: u-boot On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > Hi Bin, > > > > Thanks for the critics. Comments below. > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > Hi Rick, > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > Hi Bin > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > force other harts to be main hart. And it will go > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > and found there is an assumption that hart 0 shall be > > > > > > main hart in OpenSbi. > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > I don't think OpenSBI has such limitation. > > > > > > > Please check the source. > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > 1. get hart ID. > > 2. determine which route to take based on their ID respectively. > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > a limitation. > > This dependency on hart id #0 was not there until we added self-relocation > in OpenSBI for FW_DYNAMIC. > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. I have send a patch to fix this OpenSBI: "[PATCH] firmware: Introduce relocation lottery" Can you try above patch and see if that helps ? It will be great if you can provide Tested-by to my patch as well. Regards, Anup > > > > > > > But any hart can be main hart in U-Boot. > > > > > > > > In general case, hart 0 will be main and it is fine when U-Boot jump ot OpenSBI. > > > > But if we force hart 1 to be main hart, when hart 0 jump to OPenSBI from U-Boot, > > > > It will do relocation flow in OpenSBI which willcorrupt U-Boot SPL, > > > > but hart 0 still in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > After some work-arounds, it can pass the verifications > > > > > > that any hart can be main hart and boots U-Boot SPL -> > > > > > > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > > > > > > > > > > > > > > > > It's a bit hard for me to understand what was described here in the > > > > > commit message. Maybe you need rewrite something. > > > > > > > > OK. I will rewrite this commit message. > > > > > > > > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > > > Cc: KC Lin <kclin@andestech.com> > > > > > > Cc: Alan Kao <alankao@andestech.com> > > > > > > --- > > > > > > arch/riscv/lib/andes_plic.c | 11 +++++++---- > > > > > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > > > > > > > > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > > > > > > index 28568e4..42394b9 100644 > > > > > > --- a/arch/riscv/lib/andes_plic.c > > > > > > +++ b/arch/riscv/lib/andes_plic.c > > > > > > @@ -19,7 +19,7 @@ > > > > > > #include <cpu.h> > > > > > > > > > > > > /* pending register */ > > > > > > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > > > > > > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > > > > > > /* enable register */ > > > > > > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > > > > > > /* claim register */ > > > > > > @@ -46,7 +46,7 @@ static int init_plic(void); > > > > > > > > > > > > static int enable_ipi(int hart) > > > > > > { > > > > > > - int en; > > > > > > + unsigned int en; > > > > > > > > > > Is this for some compiler warning fix? > > > > > > > > No, it is not a warning fix. It is a bug fix. > > > > I hope en can be 0x0000000080808080 instead of 0xffffffff80808080, > > > > > > But it is int, which is only 32-bit. The example you gave was a 64-bit number. > > > > > > > Please consider the following simple program: > > > > > #define MASK 0x80808080 > > >int main(){ > > > int en; > > > en = MASK; > > > printf("%x, shifted %x\n", en, en >> 1); > > > return 0; > > >} > > > > Would you mind sharing what you get after running this on your x86_64 > > (if you have one) computer? Really appreiciate that. > > > > The almost identical episode is in the patch, specifically, > > > en = ENABLE_HART_IPI >> hart > > > > > > or it will cause IPI sending errors. > > > > > > > > > > Regards, > > > Bin > > > > Best, > > Alan > > Regards, > Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-31 10:43 ` Anup Patel @ 2019-11-01 5:25 ` Rick Chen 2019-11-05 1:50 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-01 5:25 UTC (permalink / raw) To: u-boot Hi Anup > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > Hi Bin, > > > > > > Thanks for the critics. Comments below. > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > Hi Rick, > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > force other harts to be main hart. And it will go > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > Please check the source. > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > 1. get hart ID. > > > 2. determine which route to take based on their ID respectively. > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > a limitation. > > > > This dependency on hart id #0 was not there until we added self-relocation > > in OpenSBI for FW_DYNAMIC. > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > I have send a patch to fix this OpenSBI: > "[PATCH] firmware: Introduce relocation lottery" > > Can you try above patch and see if that helps ? > > It will be great if you can provide Tested-by to my patch as well. > OK Thanks Rick > Regards, > Anup > > > > > > > > > > > But any hart can be main hart in U-Boot. > > > > > > > > > > In general case, hart 0 will be main and it is fine when U-Boot jump ot OpenSBI. > > > > > But if we force hart 1 to be main hart, when hart 0 jump to OPenSBI from U-Boot, > > > > > It will do relocation flow in OpenSBI which willcorrupt U-Boot SPL, > > > > > but hart 0 still in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > After some work-arounds, it can pass the verifications > > > > > > > that any hart can be main hart and boots U-Boot SPL -> > > > > > > > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > > > > > > > > > > > > > > > > > > > It's a bit hard for me to understand what was described here in the > > > > > > commit message. Maybe you need rewrite something. > > > > > > > > > > OK. I will rewrite this commit message. > > > > > > > > > > > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > > > > Cc: KC Lin <kclin@andestech.com> > > > > > > > Cc: Alan Kao <alankao@andestech.com> > > > > > > > --- > > > > > > > arch/riscv/lib/andes_plic.c | 11 +++++++---- > > > > > > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > > > > > > > > > > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > > > > > > > index 28568e4..42394b9 100644 > > > > > > > --- a/arch/riscv/lib/andes_plic.c > > > > > > > +++ b/arch/riscv/lib/andes_plic.c > > > > > > > @@ -19,7 +19,7 @@ > > > > > > > #include <cpu.h> > > > > > > > > > > > > > > /* pending register */ > > > > > > > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > > > > > > > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > > > > > > > /* enable register */ > > > > > > > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > > > > > > > /* claim register */ > > > > > > > @@ -46,7 +46,7 @@ static int init_plic(void); > > > > > > > > > > > > > > static int enable_ipi(int hart) > > > > > > > { > > > > > > > - int en; > > > > > > > + unsigned int en; > > > > > > > > > > > > Is this for some compiler warning fix? > > > > > > > > > > No, it is not a warning fix. It is a bug fix. > > > > > I hope en can be 0x0000000080808080 instead of 0xffffffff80808080, > > > > > > > > But it is int, which is only 32-bit. The example you gave was a 64-bit number. > > > > > > > > > > Please consider the following simple program: > > > > > > > #define MASK 0x80808080 > > > >int main(){ > > > > int en; > > > > en = MASK; > > > > printf("%x, shifted %x\n", en, en >> 1); > > > > return 0; > > > >} > > > > > > Would you mind sharing what you get after running this on your x86_64 > > > (if you have one) computer? Really appreiciate that. > > > > > > The almost identical episode is in the patch, specifically, > > > > en = ENABLE_HART_IPI >> hart > > > > > > > > or it will cause IPI sending errors. > > > > > > > > > > > > > Regards, > > > > Bin > > > > > > Best, > > > Alan > > > > Regards, > > Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-01 5:25 ` Rick Chen @ 2019-11-05 1:50 ` Rick Chen 2019-11-05 6:34 ` Anup Patel 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-05 1:50 UTC (permalink / raw) To: u-boot Hi Anup > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > Hi Bin, > > > > > > > > Thanks for the critics. Comments below. > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > Hi Rick, > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > Please check the source. > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > 1. get hart ID. > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > a limitation. > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > in OpenSBI for FW_DYNAMIC. > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > I have send a patch to fix this OpenSBI: > > "[PATCH] firmware: Introduce relocation lottery" > > > > Can you try above patch and see if that helps ? > > > > It will be great if you can provide Tested-by to my patch as well. > > > I can not find this patch in mailing list. Can you provide a hyperlink ? Thanks Rick ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-05 1:50 ` Rick Chen @ 2019-11-05 6:34 ` Anup Patel 2019-11-06 6:44 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-11-05 6:34 UTC (permalink / raw) To: u-boot On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Anup > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > Hi Bin, > > > > > > > > > > Thanks for the critics. Comments below. > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > Hi Rick, > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > Please check the source. > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > 1. get hart ID. > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > a limitation. > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > I have send a patch to fix this OpenSBI: > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > Can you try above patch and see if that helps ? > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > I can not find this patch in mailing list. > Can you provide a hyperlink ? You can try latest riscv/opensbi master. I have tested the patch on SiFive Unleashed multiple times. Regards, Anup > > Thanks > Rick ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-05 6:34 ` Anup Patel @ 2019-11-06 6:44 ` Rick Chen 2019-11-06 8:48 ` Anup Patel 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-06 6:44 UTC (permalink / raw) To: u-boot Hi Anup > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Anup > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > Hi Rick, > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > 1. get hart ID. > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > a limitation. > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > I have send a patch to fix this OpenSBI: > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > I can not find this patch in mailing list. > > Can you provide a hyperlink ? > > You can try latest riscv/opensbi master. > > I have tested the patch on SiFive Unleashed multiple times. I have tried this patch, but it fail firmware: Introduce relocation lottery( 98f4a208995b027662a7b04a25e4fa5df5f3eefe) The scenario was as below: There are 4 harts run in U-Boot SPL, hart 0 play as main hart. The hart 1 will receive ipi and come into OpenSBI(0x1000000) from U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. Then hart 1 will do _relocate_copy_to_lower which will copy data from 0x1000000 to 0x0. And it will corrupt U-Boot SPL. Thanks Rick > > Regards, > Anup > > > > > Thanks > > Rick ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-06 6:44 ` Rick Chen @ 2019-11-06 8:48 ` Anup Patel 2019-11-06 8:58 ` Anup Patel 0 siblings, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-11-06 8:48 UTC (permalink / raw) To: u-boot On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Anup > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > Hi Anup > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > 1. get hart ID. > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > a limitation. > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > Can you provide a hyperlink ? > > > > You can try latest riscv/opensbi master. > > > > I have tested the patch on SiFive Unleashed multiple times. > > I have tried this patch, but it fail > firmware: Introduce relocation lottery( > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > The scenario was as below: > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > Then hart 1 will do _relocate_copy_to_lower which will copy data from > 0x1000000 to 0x0. > And it will corrupt U-Boot SPL. The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware are moved to the FW_TEXT_START before entering C code. This helps us load OpenSBI firmwares anywhere in RAM. However, OpenSBI firmwares don't know where the U-Boot SPL is running. In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to address same address 0x0. This means secondary HARTs cannot safely wait while primary HART enters OpenSBI. You should hold secondary HARTs in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are loaded in RAM by primary HART. All your HARTs should jump to OpenSBI at the same time after everything is loaded in RAM. Regards, Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-06 8:48 ` Anup Patel @ 2019-11-06 8:58 ` Anup Patel 2019-11-06 9:21 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-11-06 8:58 UTC (permalink / raw) To: u-boot On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Anup > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > Hi Anup > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > 1. get hart ID. > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > a limitation. > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > Can you provide a hyperlink ? > > > > > > You can try latest riscv/opensbi master. > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > I have tried this patch, but it fail > > firmware: Introduce relocation lottery( > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > The scenario was as below: > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > 0x1000000 to 0x0. > > And it will corrupt U-Boot SPL. > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > are moved to the FW_TEXT_START before entering C code. This helps > us load OpenSBI firmwares anywhere in RAM. > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > address same address 0x0. This means secondary HARTs cannot safely > wait while primary HART enters OpenSBI. You should hold secondary HARTs > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > at the same time after everything is loaded in RAM. I see the issue now. The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary HART jumps to OpenSBI at the end. (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. If possible please change TEXT base for U-Boot SPL or OpenSBI. I think changing U-Boot SPL TEXT_START would be convenient since this series is under review. Thoughts ? Regards, Anup > > Regards, > Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-06 8:58 ` Anup Patel @ 2019-11-06 9:21 ` Rick Chen 2019-11-06 11:11 ` Anup Patel 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-06 9:21 UTC (permalink / raw) To: u-boot Hi Anup > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > Hi Anup > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > Hi Anup > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > 1. get hart ID. > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > Can you provide a hyperlink ? > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > I have tried this patch, but it fail > > > firmware: Introduce relocation lottery( > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > The scenario was as below: > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > 0x1000000 to 0x0. > > > And it will corrupt U-Boot SPL. > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > are moved to the FW_TEXT_START before entering C code. This helps > > us load OpenSBI firmwares anywhere in RAM. > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > address same address 0x0. This means secondary HARTs cannot safely > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > at the same time after everything is loaded in RAM. > > I see the issue now. > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > HART jumps to OpenSBI at the end. > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > changing U-Boot SPL TEXT_START would be convenient since this series > is under review. Thoughts ? Yes. I know it can avoid corrupting issue with changing U-Boot SPL TEXT_START not equal to OpenSBI TEXT base. With the following changes, U-Boot SPL text base can equal to OpenSBI text base 1 U-Boot pass main hart information (a2) when jumping to OpenSBI 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to _wait_relocate_copy_done It will be convenient to change U-Boot SPL text base currently. Thanks Rick > > Regards, > Anup > > > > > Regards, > > Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-06 9:21 ` Rick Chen @ 2019-11-06 11:11 ` Anup Patel 2019-11-07 1:34 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-11-06 11:11 UTC (permalink / raw) To: u-boot On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Anup > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > Hi Anup > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > 1. get hart ID. > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > Can you provide a hyperlink ? > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > I have tried this patch, but it fail > > > > firmware: Introduce relocation lottery( > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > The scenario was as below: > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > 0x1000000 to 0x0. > > > > And it will corrupt U-Boot SPL. > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > are moved to the FW_TEXT_START before entering C code. This helps > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > address same address 0x0. This means secondary HARTs cannot safely > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > at the same time after everything is loaded in RAM. > > > > I see the issue now. > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > HART jumps to OpenSBI at the end. > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > changing U-Boot SPL TEXT_START would be convenient since this series > > is under review. Thoughts ? > > Yes. > I know it can avoid corrupting issue with changing U-Boot SPL > TEXT_START not equal to OpenSBI TEXT base. I think this issue will be seen on U-Boot SPL running on QEMU as well. > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > _wait_relocate_copy_done Overall it's a good suggestion but we cannot use a2 register because this will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred boot HART id in struct fw_dynamic_info. I have a patch for this in preferred_boot_hart_v1 branch of https://github.com/avpatel/opensbi.git Can you try OpenSBI from above branch ? You will have to update the "struct fw_dynamic_info" passed to OpenSBI by U-Boot SPL. Meanwhile, I will try above patch on QEMU and SiFive Unleashed. > > It will be convenient to change U-Boot SPL text base currently. Let's fix this correctly now itself. Regards, Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-06 11:11 ` Anup Patel @ 2019-11-07 1:34 ` Rick Chen 2019-11-07 5:15 ` Anup Patel 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-07 1:34 UTC (permalink / raw) To: u-boot Hi Anup > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Anup > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > I have tried this patch, but it fail > > > > > firmware: Introduce relocation lottery( > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > The scenario was as below: > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > 0x1000000 to 0x0. > > > > > And it will corrupt U-Boot SPL. > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > at the same time after everything is loaded in RAM. > > > > > > I see the issue now. > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > HART jumps to OpenSBI at the end. > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > is under review. Thoughts ? > > > > Yes. > > I know it can avoid corrupting issue with changing U-Boot SPL > > TEXT_START not equal to OpenSBI TEXT base. > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > _wait_relocate_copy_done > > Overall it's a good suggestion but we cannot use a2 register because this > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > boot HART id in struct fw_dynamic_info. Sorry, what I want to say shall be a3. > > I have a patch for this in preferred_boot_hart_v1 branch of > https://github.com/avpatel/opensbi.git > > Can you try OpenSBI from above branch ? > > You will have to update the "struct fw_dynamic_info" passed to > OpenSBI by U-Boot SPL. Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. So if U-Boot SPL can pass main hart information via a3, OpenSBI just have the following change blt zero, a6, _wait_relocate_copy_done change to bne a3, a6, _wait_relocate_copy_done before this commit 98f4a208995b027662a7b04a25e4fa5df5f3eefe firmware: Introduce relocation lottery But after this commit 98f4a, main hart become chosen from lottery mechanism. Maybe I will prefer to change U-Boot SPL text base not overlap with OpenSBI text start. :) Thanks Rick > > Meanwhile, I will try above patch on QEMU and SiFive Unleashed. > > > > > It will be convenient to change U-Boot SPL text base currently. > > Let's fix this correctly now itself. > > Regards, > Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 1:34 ` Rick Chen @ 2019-11-07 5:15 ` Anup Patel 2019-11-07 5:45 ` Anup Patel 0 siblings, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-11-07 5:15 UTC (permalink / raw) To: u-boot On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Anup > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > Hi Anup > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > firmware: Introduce relocation lottery( > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > The scenario was as below: > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > 0x1000000 to 0x0. > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > at the same time after everything is loaded in RAM. > > > > > > > > I see the issue now. > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > HART jumps to OpenSBI at the end. > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > is under review. Thoughts ? > > > > > > Yes. > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > TEXT_START not equal to OpenSBI TEXT base. > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > _wait_relocate_copy_done > > > > Overall it's a good suggestion but we cannot use a2 register because this > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > boot HART id in struct fw_dynamic_info. > > Sorry, what I want to say shall be a3. > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > https://github.com/avpatel/opensbi.git > > > > Can you try OpenSBI from above branch ? > > > > You will have to update the "struct fw_dynamic_info" passed to > > OpenSBI by U-Boot SPL. > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. That's wrong in U-Boot SPL. All HARTs have to follow FW_DYNAMIC protocol and pass "struct fw_dynamic_info" pointer in 'a2' register. > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > have the following change > blt zero, a6, _wait_relocate_copy_done > change to > bne a3, a6, _wait_relocate_copy_done > before this commit > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > firmware: Introduce relocation lottery What about FW_JUMP and FW_PAYLOAD? We have no way of passing value in a3 for these firmwares because these are not booted by U-Boot SPL. Also, U-Boot-2019.10 already uses U-Boot SPL support which does not pass anything in 'a3' register. We should definitely use "struct fw_dynamic_info" for this so that we can maintain backward compatibility as well. Please make sure that U-Boot SPL passes "struct fw_dynamic_info" pointer in 'a2' register for all HARTs. > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > Maybe I will prefer to change U-Boot SPL text base not overlap with > OpenSBI text start. :) Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's just that most of us did not notice it for U-Boot SPL on QEMU. Let's fix this in the right way from start itself. Regards, Anup > > Thanks > Rick > > > > > Meanwhile, I will try above patch on QEMU and SiFive Unleashed. > > > > > > > > It will be convenient to change U-Boot SPL text base currently. > > > > Let's fix this correctly now itself. > > > > Regards, > > Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 5:15 ` Anup Patel @ 2019-11-07 5:45 ` Anup Patel 2019-11-07 6:10 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-11-07 5:45 UTC (permalink / raw) To: u-boot On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Anup > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > Hi Anup > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > firmware: Introduce relocation lottery( > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > The scenario was as below: > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > 0x1000000 to 0x0. > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > I see the issue now. > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > HART jumps to OpenSBI at the end. > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > is under review. Thoughts ? > > > > > > > > Yes. > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > _wait_relocate_copy_done > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > boot HART id in struct fw_dynamic_info. > > > > Sorry, what I want to say shall be a3. > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > https://github.com/avpatel/opensbi.git > > > > > > Can you try OpenSBI from above branch ? > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > OpenSBI by U-Boot SPL. > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > That's wrong in U-Boot SPL. > > All HARTs have to follow FW_DYNAMIC protocol and pass > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > have the following change > > blt zero, a6, _wait_relocate_copy_done > > change to > > bne a3, a6, _wait_relocate_copy_done > > before this commit > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > firmware: Introduce relocation lottery > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > value in a3 for these firmwares because these are not booted by U-Boot > SPL. > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > pass anything in 'a3' register. > > We should definitely use "struct fw_dynamic_info" for this so that we can > maintain backward compatibility as well. > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > pointer in 'a2' register for all HARTs. > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > OpenSBI text start. :) > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > just that most of us did not notice it for U-Boot SPL on QEMU. > > Let's fix this in the right way from start itself. I double checked spl_invoke_opensbi() and it is doing the right thing by passing "struct fw_dyanmic_info" pointer in 'a2' register. (Refer, common/spl/spl_opensbi.c) Not sure, why it is not passing 'a2' register correctly for you ?? Regards, Anup > > Regards, > Anup > > > > > Thanks > > Rick > > > > > > > > Meanwhile, I will try above patch on QEMU and SiFive Unleashed. > > > > > > > > > > > It will be convenient to change U-Boot SPL text base currently. > > > > > > Let's fix this correctly now itself. > > > > > > Regards, > > > Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 5:45 ` Anup Patel @ 2019-11-07 6:10 ` Rick Chen 2019-11-07 6:18 ` Anup Patel 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-07 6:10 UTC (permalink / raw) To: u-boot Hi Anup > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > Hi Anup > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > 0x1000000 to 0x0. > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > HART jumps to OpenSBI at the end. > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > is under review. Thoughts ? > > > > > > > > > > Yes. > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > _wait_relocate_copy_done > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > boot HART id in struct fw_dynamic_info. > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > https://github.com/avpatel/opensbi.git > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > OpenSBI by U-Boot SPL. > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > That's wrong in U-Boot SPL. > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > have the following change > > > blt zero, a6, _wait_relocate_copy_done > > > change to > > > bne a3, a6, _wait_relocate_copy_done > > > before this commit > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > firmware: Introduce relocation lottery > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > value in a3 for these firmwares because these are not booted by U-Boot > > SPL. > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > pass anything in 'a3' register. > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > maintain backward compatibility as well. > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > pointer in 'a2' register for all HARTs. > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > OpenSBI text start. :) > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > Let's fix this in the right way from start itself. > > I double checked spl_invoke_opensbi() and it is doing the right thing > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > (Refer, common/spl/spl_opensbi.c) > > Not sure, why it is not passing 'a2' register correctly for you ?? > Yes, you are right. I reply too quickly. Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. Thanks for your corrections Regards, Rick ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 6:10 ` Rick Chen @ 2019-11-07 6:18 ` Anup Patel 2019-11-07 9:41 ` Auer, Lukas 0 siblings, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-11-07 6:18 UTC (permalink / raw) To: u-boot On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Anup > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > Hi Anup > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > Yes. > > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > > _wait_relocate_copy_done > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > > OpenSBI by U-Boot SPL. > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > That's wrong in U-Boot SPL. > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > > have the following change > > > > blt zero, a6, _wait_relocate_copy_done > > > > change to > > > > bne a3, a6, _wait_relocate_copy_done > > > > before this commit > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > firmware: Introduce relocation lottery > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > > value in a3 for these firmwares because these are not booted by U-Boot > > > SPL. > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > > pass anything in 'a3' register. > > > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > > maintain backward compatibility as well. > > > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > > OpenSBI text start. :) > > > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > > > Let's fix this in the right way from start itself. > > > > I double checked spl_invoke_opensbi() and it is doing the right thing > > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > > (Refer, common/spl/spl_opensbi.c) > > > > Not sure, why it is not passing 'a2' register correctly for you ?? > > > > Yes, you are right. I reply too quickly. > Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. > > Thanks for your corrections No problem, I am happy to help. BTW, I tried to play around with U-Boot SPL on QEMU. Maybe below changes can help you... diff --git a/common/spl/spl_opensbi.c b/common/spl/spl_opensbi.c index a6b4480ed2..79ee7edcf9 100644 --- a/common/spl/spl_opensbi.c +++ b/common/spl/spl_opensbi.c @@ -69,6 +69,7 @@ void spl_invoke_opensbi(struct spl_image_info *spl_image) opensbi_info.next_addr = uboot_entry; opensbi_info.next_mode = FW_DYNAMIC_INFO_NEXT_MODE_S; opensbi_info.options = SBI_SCRATCH_NO_BOOT_PRINTS; + opensbi_info.boot_hart = gd->arch.boot_hart; opensbi_entry = (void (*)(ulong, ulong, ulong))spl_image->entry_point; invalidate_icache_all(); diff --git a/include/opensbi.h b/include/opensbi.h index 9f1d62e7dd..29a95fdfb6 100644 --- a/include/opensbi.h +++ b/include/opensbi.h @@ -11,7 +11,7 @@ #define FW_DYNAMIC_INFO_MAGIC_VALUE 0x4942534f /** Maximum supported info version */ -#define FW_DYNAMIC_INFO_VERSION 0x1 +#define FW_DYNAMIC_INFO_VERSION 0x2 /** Possible next mode values */ #define FW_DYNAMIC_INFO_NEXT_MODE_U 0x0 @@ -35,6 +35,8 @@ struct fw_dynamic_info { unsigned long next_mode; /** Options for OpenSBI library */ unsigned long options; + /** Preferred boot HART id */ + unsigned long boot_hart; } __packed; #endif Regards, Anup ^ permalink raw reply related [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 6:18 ` Anup Patel @ 2019-11-07 9:41 ` Auer, Lukas 2019-11-07 10:44 ` Anup Patel 0 siblings, 1 reply; 71+ messages in thread From: Auer, Lukas @ 2019-11-07 9:41 UTC (permalink / raw) To: u-boot On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Anup > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > Hi Anup > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > Hi Anup > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > Yes. > > > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > > > have the following change > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > change to > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > before this commit > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > firmware: Introduce relocation lottery > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > > > value in a3 for these firmwares because these are not booted by U-Boot > > > > SPL. > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > > > pass anything in 'a3' register. > > > > > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > > > maintain backward compatibility as well. > > > > > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > > > OpenSBI text start. :) > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > > > > > Let's fix this in the right way from start itself. > > > > > > I double checked spl_invoke_opensbi() and it is doing the right thing > > > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > > > (Refer, common/spl/spl_opensbi.c) > > > > > > Not sure, why it is not passing 'a2' register correctly for you ?? > > > > > > > Yes, you are right. I reply too quickly. > > Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. > > > > Thanks for your corrections > > No problem, I am happy to help. > > BTW, I tried to play around with U-Boot SPL on QEMU. > > Maybe below changes can help you... Thanks for looking into this issue! I successfully tested it on QEMU, I had to add a short delay between sending the IPIs to trigger the problem. We might still run into problems however. Right now, we are assuming that the main hart is the last one to enter OpenSBI. If this is not the case (some delay when handling the IPI), we will have the same problem again. To fix this we could pass the hart mask, containing all harts that have entered U-Boot, to OpenSBI and wait for all harts to be running in OpenSBI. I am not sure how realistic this scenario is, so this might not be needed. Regards, Lukas ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 9:41 ` Auer, Lukas @ 2019-11-07 10:44 ` Anup Patel 2019-11-07 11:41 ` Rick Chen 2019-11-07 11:44 ` Auer, Lukas 0 siblings, 2 replies; 71+ messages in thread From: Anup Patel @ 2019-11-07 10:44 UTC (permalink / raw) To: u-boot On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas <lukas.auer@aisec.fraunhofer.de> wrote: > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com> wrote: > > > Hi Anup > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > Hi Anup > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > Yes. > > > > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > > > > have the following change > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > change to > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > before this commit > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > > > > value in a3 for these firmwares because these are not booted by U-Boot > > > > > SPL. > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > > > > pass anything in 'a3' register. > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > > > > maintain backward compatibility as well. > > > > > > > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > > > > OpenSBI text start. :) > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > > > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the right thing > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > Not sure, why it is not passing 'a2' register correctly for you ?? > > > > > > > > > > Yes, you are right. I reply too quickly. > > > Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. > > > > > > Thanks for your corrections > > > > No problem, I am happy to help. > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > Maybe below changes can help you... > > Thanks for looking into this issue! I successfully tested it on QEMU, I > had to add a short delay between sending the IPIs to trigger the > problem. > > We might still run into problems however. Right now, we are assuming > that the main hart is the last one to enter OpenSBI. If this is not the > case (some delay when handling the IPI), we will have the same problem > again. To fix this we could pass the hart mask, containing all harts > that have entered U-Boot, to OpenSBI and wait for all harts to be > running in OpenSBI. I am not sure how realistic this scenario is, so > this might not be needed. I agree that we might still run into this issue if primary HART enters OpenSBI before secondary HARTs. I think this situation can only happen on QEMU where each CPU is a thread running on host but very unlikely/impossible on real HW. Maybe a delay on primary HART in U-Boot SPL after SMP calls to secondary HARTs and before jumping to OpenSBI ? Regarding hart_mask in fw_dynamic_info, I think the issue will be the size of the hart_mask. It is possible in-future SOC vendors come-up with SOC having huge number of HARTs OR SOC with discontinuous HART IDs which can cause a 64bit hart_mask to be not sufficient for all HARTs. Also, waiting for all HARTs to enter OpenSBI will be one more wait-loop in fw_base.S which will add to the boot-time as well. I still think the root cause of the issue is that TEXT_START of U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can insist SOC vendors to not use same TEXT_START ? Regards, Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 10:44 ` Anup Patel @ 2019-11-07 11:41 ` Rick Chen 2019-11-07 12:22 ` Anup Patel 2019-11-07 18:44 ` Atish Patra 2019-11-07 11:44 ` Auer, Lukas 1 sibling, 2 replies; 71+ messages in thread From: Rick Chen @ 2019-11-07 11:41 UTC (permalink / raw) To: u-boot Hi Anup & Lukas Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Anup > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > Hi Anup > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > > > > > have the following change > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > change to > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > before this commit > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > > > > > value in a3 for these firmwares because these are not booted by U-Boot > > > > > > SPL. > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > > > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > > > > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the right thing > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for you ?? > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. > > > > > > > > Thanks for your corrections > > > > > > No problem, I am happy to help. > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > Maybe below changes can help you... > > > > Thanks for looking into this issue! I successfully tested it on QEMU, I > > had to add a short delay between sending the IPIs to trigger the > > problem. > > > > We might still run into problems however. Right now, we are assuming > > that the main hart is the last one to enter OpenSBI. If this is not the > > case (some delay when handling the IPI), we will have the same problem > > again. To fix this we could pass the hart mask, containing all harts > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > running in OpenSBI. I am not sure how realistic this scenario is, so > > this might not be needed. > > I agree that we might still run into this issue if primary HART enters > OpenSBI before secondary HARTs. I think this situation can only > happen on QEMU where each CPU is a thread running on host but > very unlikely/impossible on real HW. > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > secondary HARTs and before jumping to OpenSBI ? > > Regarding hart_mask in fw_dynamic_info, I think the issue will be the > size of the hart_mask. It is possible in-future SOC vendors come-up > with SOC having huge number of HARTs OR SOC with discontinuous > HART IDs which can cause a 64bit hart_mask to be not sufficient for > all HARTs. > > Also, waiting for all HARTs to enter OpenSBI will be one more wait-loop > in fw_base.S which will add to the boot-time as well. > > I still think the root cause of the issue is that TEXT_START of > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > insist SOC vendors to not use same TEXT_START ? I have try your changes about boot_hart for U-Boot SPL with OpenSBI, preferred_boot_hart_v2 branch It still encounter some booting problems. I try to find out the root cause but in vain. I am very agree with options of Lukas. After modifying U-Boot SPL text base not equal to zero and the booting progress will be pass. Thanks Rick > > Regards, > Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 11:41 ` Rick Chen @ 2019-11-07 12:22 ` Anup Patel 2019-11-08 1:23 ` Rick Chen 2019-11-07 18:44 ` Atish Patra 1 sibling, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-11-07 12:22 UTC (permalink / raw) To: u-boot On Thu, Nov 7, 2019 at 5:11 PM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Anup & Lukas > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > Hi Anup > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > > > > > > have the following change > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > change to > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > before this commit > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > > > > > > value in a3 for these firmwares because these are not booted by U-Boot > > > > > > > SPL. > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > > > > > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the right thing > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for you ?? > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. > > > > > > > > > > Thanks for your corrections > > > > > > > > No problem, I am happy to help. > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > Maybe below changes can help you... > > > > > > Thanks for looking into this issue! I successfully tested it on QEMU, I > > > had to add a short delay between sending the IPIs to trigger the > > > problem. > > > > > > We might still run into problems however. Right now, we are assuming > > > that the main hart is the last one to enter OpenSBI. If this is not the > > > case (some delay when handling the IPI), we will have the same problem > > > again. To fix this we could pass the hart mask, containing all harts > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > running in OpenSBI. I am not sure how realistic this scenario is, so > > > this might not be needed. > > > > I agree that we might still run into this issue if primary HART enters > > OpenSBI before secondary HARTs. I think this situation can only > > happen on QEMU where each CPU is a thread running on host but > > very unlikely/impossible on real HW. > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > secondary HARTs and before jumping to OpenSBI ? > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be the > > size of the hart_mask. It is possible in-future SOC vendors come-up > > with SOC having huge number of HARTs OR SOC with discontinuous > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > all HARTs. > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait-loop > > in fw_base.S which will add to the boot-time as well. > > > > I still think the root cause of the issue is that TEXT_START of > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > insist SOC vendors to not use same TEXT_START ? > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > preferred_boot_hart_v2 branch > It still encounter some booting problems. I try to find out the root > cause but in vain. > > I am very agree with options of Lukas. > After modifying U-Boot SPL text base not equal to zero and the booting > progress will be pass. No problem, it will be your decision to go with different TEXT_BASE for AndesTech platform. We will keep the "boot_hart" field in OpenSBI for U-Boot SPL on QEMU and I will wait for Lukas to add more checks and small delay in U-Boot SPL (like he mentioned). Thanks, Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 12:22 ` Anup Patel @ 2019-11-08 1:23 ` Rick Chen 2019-11-08 12:14 ` Anup Patel 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-08 1:23 UTC (permalink / raw) To: u-boot Hi Anup > > On Thu, Nov 7, 2019 at 5:11 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Anup & Lukas > > > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > Hi Anup > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > > > > > > > have the following change > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > change to > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > before this commit > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > > > > > > > value in a3 for these firmwares because these are not booted by U-Boot > > > > > > > > SPL. > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > > > > > > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the right thing > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for you ?? > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > Maybe below changes can help you... > > > > > > > > Thanks for looking into this issue! I successfully tested it on QEMU, I > > > > had to add a short delay between sending the IPIs to trigger the > > > > problem. > > > > > > > > We might still run into problems however. Right now, we are assuming > > > > that the main hart is the last one to enter OpenSBI. If this is not the > > > > case (some delay when handling the IPI), we will have the same problem > > > > again. To fix this we could pass the hart mask, containing all harts > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > running in OpenSBI. I am not sure how realistic this scenario is, so > > > > this might not be needed. > > > > > > I agree that we might still run into this issue if primary HART enters > > > OpenSBI before secondary HARTs. I think this situation can only > > > happen on QEMU where each CPU is a thread running on host but > > > very unlikely/impossible on real HW. > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be the > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > all HARTs. > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait-loop > > > in fw_base.S which will add to the boot-time as well. > > > > > > I still think the root cause of the issue is that TEXT_START of > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > insist SOC vendors to not use same TEXT_START ? > > > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > > preferred_boot_hart_v2 branch > > It still encounter some booting problems. I try to find out the root > > cause but in vain. > > > > I am very agree with options of Lukas. > > After modifying U-Boot SPL text base not equal to zero and the booting > > progress will be pass. > > No problem, it will be your decision to go with different TEXT_BASE for > AndesTech platform. You misunderstand my intention It is just a temporary solution for debugging. I prefer U-Boot SPL text base can be sync with you finally. > > We will keep the "boot_hart" field in OpenSBI for U-Boot SPL on QEMU > and I will wait for Lukas to add more checks and small delay in U-Boot SPL > (like he mentioned). I am happy to hear that. :) Thanks Rick > > Thanks, > Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-08 1:23 ` Rick Chen @ 2019-11-08 12:14 ` Anup Patel 0 siblings, 0 replies; 71+ messages in thread From: Anup Patel @ 2019-11-08 12:14 UTC (permalink / raw) To: u-boot On Fri, Nov 8, 2019 at 6:53 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Anup > > > > > On Thu, Nov 7, 2019 at 5:11 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > Hi Anup & Lukas > > > > > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > > > > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > > > > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > Hi Anup > > > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > > > > > > > > have the following change > > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > > change to > > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > > before this commit > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > > > > > > > > value in a3 for these firmwares because these are not booted by U-Boot > > > > > > > > > SPL. > > > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > > > > > > > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the right thing > > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for you ?? > > > > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. > > > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > > > Maybe below changes can help you... > > > > > > > > > > Thanks for looking into this issue! I successfully tested it on QEMU, I > > > > > had to add a short delay between sending the IPIs to trigger the > > > > > problem. > > > > > > > > > > We might still run into problems however. Right now, we are assuming > > > > > that the main hart is the last one to enter OpenSBI. If this is not the > > > > > case (some delay when handling the IPI), we will have the same problem > > > > > again. To fix this we could pass the hart mask, containing all harts > > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > > running in OpenSBI. I am not sure how realistic this scenario is, so > > > > > this might not be needed. > > > > > > > > I agree that we might still run into this issue if primary HART enters > > > > OpenSBI before secondary HARTs. I think this situation can only > > > > happen on QEMU where each CPU is a thread running on host but > > > > very unlikely/impossible on real HW. > > > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be the > > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > > all HARTs. > > > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait-loop > > > > in fw_base.S which will add to the boot-time as well. > > > > > > > > I still think the root cause of the issue is that TEXT_START of > > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > > insist SOC vendors to not use same TEXT_START ? > > > > > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > > > preferred_boot_hart_v2 branch > > > It still encounter some booting problems. I try to find out the root > > > cause but in vain. > > > > > > I am very agree with options of Lukas. > > > After modifying U-Boot SPL text base not equal to zero and the booting > > > progress will be pass. > > > > No problem, it will be your decision to go with different TEXT_BASE for > > AndesTech platform. > > You misunderstand my intention > It is just a temporary solution for debugging. > I prefer U-Boot SPL text base can be sync with you finally. No worries, we are on same page here. If we get U-Boot SPL -> OpenSBI boot-flow stable on AndesTech platform then it will make things easy for SiFive Unleashed as well. If possible please try your patches on-top-of Lukas's changes. I am sure this will help you achieve 100% reliability. > > > > > We will keep the "boot_hart" field in OpenSBI for U-Boot SPL on QEMU > > and I will wait for Lukas to add more checks and small delay in U-Boot SPL > > (like he mentioned). > > I am happy to hear that. :) Cool. Best Regards, Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 11:41 ` Rick Chen 2019-11-07 12:22 ` Anup Patel @ 2019-11-07 18:44 ` Atish Patra 2019-11-08 1:13 ` Rick Chen 1 sibling, 1 reply; 71+ messages in thread From: Atish Patra @ 2019-11-07 18:44 UTC (permalink / raw) To: u-boot On Thu, 2019-11-07 at 19:41 +0800, Rick Chen wrote: > Hi Anup & Lukas > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com > > > > > wrote: > > > > > Hi Anup > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel < > > > > > > anup at brainfault.org> wrote: > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen < > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen < > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel < > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen < > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen < > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup > > > > > > > > > > > > > > > > > Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM > > > > > > > > > > > > > > > > > > Alan Kao <alankao@andestech.com> > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments > > > > > > > > > > > > > > > > > > > below. > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at > > > > > > > > > > > > > > > > > > > 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 > > > > > > > > > > > > > > > > > > > > AM Rick Chen < > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at > > > > > > > > > > > > > > > > > > > > > > 2:18 PM Andes < > > > > > > > > > > > > > > > > > > > > > > uboot at andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > rick at andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to > > > > > > > > > > > > > > > > > > > > > > > hart 0 always will be > > > > > > > > > > > > > > > > > > > > > > > main > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When > > > > > > > > > > > > > > > > > > > > > > > develop SPL flow, I try > > > > > > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > force other harts to be > > > > > > > > > > > > > > > > > > > > > > > main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI > > > > > > > > > > > > > > > > > > > > > > > flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit > > > > > > > > > > > > > > > > > > > > > > contain 2 fixes, or just 1 > > > > > > > > > > > > > > > > > > > > > > fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But > > > > > > > > > > > > > > > > > > > > > they will cause one negative > > > > > > > > > > > > > > > > > > > > > result > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi > > > > > > > > > > > > > > > > > > > > > to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart > > > > > > > > > > > > > > > > > > > > > > > can be main hart in U- > > > > > > > > > > > > > > > > > > > > > > > Boot SPL > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it > > > > > > > > > > > > > > > > > > > > > > > still fail somewhere. > > > > > > > > > > > > > > > > > > > > > > > After dig in > > > > > > > > > > > > > > > > > > > > > > > and found there is an > > > > > > > > > > > > > > > > > > > > > > > assumption that hart 0 > > > > > > > > > > > > > > > > > > > > > > > shall be > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is > > > > > > > > > > > > > > > > > > > > > > a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. > > > > > > > > > > > > > > > > > > > > > Maybe it is a compatible > > > > > > > > > > > > > > > > > > > > > issue. > > > > > > > > > > > > > > > > > > > > > There is a limitation that > > > > > > > > > > > > > > > > > > > > > only hart 0 can be main hart > > > > > > > > > > > > > > > > > > > > > in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such > > > > > > > > > > > > > > > > > > > > limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs > > > > > > > > > > > > > > > > > > > of the initialization are the > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > 2. determine which route to take > > > > > > > > > > > > > > > > > > > based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this > > > > > > > > > > > > > > > > > > > signature, if you are not willing > > > > > > > > > > > > > > > > > > > to call it > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was > > > > > > > > > > > > > > > > > > not there until we added self- > > > > > > > > > > > > > > > > > > relocation > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI > > > > > > > > > > > > > > > > > > but we might end-up having > > > > > > > > > > > > > > > > > > boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this > > > > > > > > > > > > > > > > > OpenSBI: > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce > > > > > > > > > > > > > > > > > relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if > > > > > > > > > > > > > > > > > that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide > > > > > > > > > > > > > > > > > Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing > > > > > > > > > > > > > > > list. > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed > > > > > > > > > > > > > > multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 > > > > > > > > > > > > > play as main hart. > > > > > > > > > > > > > The hart 1 will receive ipi and come into > > > > > > > > > > > > > OpenSBI(0x1000000) from > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still > > > > > > > > > > > > > run in U-Boot SPL. > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower > > > > > > > > > > > > > which will copy data from > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares > > > > > > > > > > > > ensures that OpenSBI firmware > > > > > > > > > > > > are moved to the FW_TEXT_START before entering > > > > > > > > > > > > C code. This helps > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the > > > > > > > > > > > > U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U- > > > > > > > > > > > > Boot SPL are linked to > > > > > > > > > > > > address same address 0x0. This means secondary > > > > > > > > > > > > HARTs cannot safely > > > > > > > > > > > > wait while primary HART enters OpenSBI. You > > > > > > > > > > > > should hold secondary HARTs > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and > > > > > > > > > > > > U-Boot proper are > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs > > > > > > > > > > > > should jump to OpenSBI > > > > > > > > > > > > at the same time after everything is loaded in > > > > > > > > > > > > RAM. > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART > > > > > > > > > > > jump to OpenSBI and primary > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > (Refer, jump_to_image_no_args() in > > > > > > > > > > > arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U- > > > > > > > > > > > Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot > > > > > > > > > > > SPL or OpenSBI. I think > > > > > > > > > > > changing U-Boot SPL TEXT_START would be > > > > > > > > > > > convenient since this series > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > I know it can avoid corrupting issue with > > > > > > > > > > changing U-Boot SPL > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running > > > > > > > > > on QEMU as well. > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base > > > > > > > > > > can equal to OpenSBI text base > > > > > > > > > > 1 U-Boot pass main hart information (a2) when > > > > > > > > > > jumping to OpenSBI > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, > > > > > > > > > > other harts go to > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 > > > > > > > > > register because this > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should > > > > > > > > > pass preferred > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 > > > > > > > > > branch of > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" > > > > > > > > > passed to > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI > > > > > > > > by U-Boot SPL. > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" > > > > > > > > to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, > > > > > > > > OpenSBI just > > > > > > > > have the following change > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > change to > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > before this commit > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of > > > > > > > passing > > > > > > > value in a3 for these firmwares because these are not > > > > > > > booted by U-Boot > > > > > > > SPL. > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support > > > > > > > which does not > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for > > > > > > > this so that we can > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct > > > > > > > fw_dynamic_info" > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen > > > > > > > > from lottery mechanism. > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not > > > > > > > > overlap with > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on > > > > > > > QEMU as well. It's > > > > > > > just that most of us did not notice it for U-Boot SPL on > > > > > > > QEMU. > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the > > > > > > right thing > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' > > > > > > register. > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for > > > > > > you ?? > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > Other harts will pass struct fw_dyanmic_info in a2 to > > > > > OpenSBI. > > > > > > > > > > Thanks for your corrections > > > > > > > > No problem, I am happy to help. > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > Maybe below changes can help you... > > > > > > Thanks for looking into this issue! I successfully tested it on > > > QEMU, I > > > had to add a short delay between sending the IPIs to trigger the > > > problem. > > > > > > We might still run into problems however. Right now, we are > > > assuming > > > that the main hart is the last one to enter OpenSBI. If this is > > > not the > > > case (some delay when handling the IPI), we will have the same > > > problem > > > again. To fix this we could pass the hart mask, containing all > > > harts > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > running in OpenSBI. I am not sure how realistic this scenario is, > > > so > > > this might not be needed. > > > > I agree that we might still run into this issue if primary HART > > enters > > OpenSBI before secondary HARTs. I think this situation can only > > happen on QEMU where each CPU is a thread running on host but > > very unlikely/impossible on real HW. > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > secondary HARTs and before jumping to OpenSBI ? > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be > > the > > size of the hart_mask. It is possible in-future SOC vendors come-up > > with SOC having huge number of HARTs OR SOC with discontinuous > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > all HARTs. > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait- > > loop > > in fw_base.S which will add to the boot-time as well. > > > > I still think the root cause of the issue is that TEXT_START of > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > insist SOC vendors to not use same TEXT_START ? > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > preferred_boot_hart_v2 branch > It still encounter some booting problems. I try to find out the root > cause but in vain. > Just wanted to make sure that you have tried this patch. http://lists.infradead.org/pipermail/opensbi/2019-November/000672.html We should investigate the issue why it did not work for you if this patch did not work for you. > I am very agree with options of Lukas. > After modifying U-Boot SPL text base not equal to zero and the > booting > progress will be pass. > > Thanks > Rick > > > Regards, > > Anup -- Regards, Atish ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 18:44 ` Atish Patra @ 2019-11-08 1:13 ` Rick Chen 2019-11-08 7:27 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-08 1:13 UTC (permalink / raw) To: u-boot Hi Atish > > On Thu, 2019-11-07 at 19:41 +0800, Rick Chen wrote: > > Hi Anup & Lukas > > > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com > > > > > > wrote: > > > > > > Hi Anup > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel < > > > > > > > anup at brainfault.org> wrote: > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen < > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen < > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel < > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen < > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen < > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup > > > > > > > > > > > > > > > > > > Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM > > > > > > > > > > > > > > > > > > > Alan Kao <alankao@andestech.com> > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments > > > > > > > > > > > > > > > > > > > > below. > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at > > > > > > > > > > > > > > > > > > > > 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 > > > > > > > > > > > > > > > > > > > > > AM Rick Chen < > > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at > > > > > > > > > > > > > > > > > > > > > > > 2:18 PM Andes < > > > > > > > > > > > > > > > > > > > > > > > uboot at andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > rick at andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to > > > > > > > > > > > > > > > > > > > > > > > > hart 0 always will be > > > > > > > > > > > > > > > > > > > > > > > > main > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When > > > > > > > > > > > > > > > > > > > > > > > > develop SPL flow, I try > > > > > > > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be > > > > > > > > > > > > > > > > > > > > > > > > main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI > > > > > > > > > > > > > > > > > > > > > > > > flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit > > > > > > > > > > > > > > > > > > > > > > > contain 2 fixes, or just 1 > > > > > > > > > > > > > > > > > > > > > > > fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But > > > > > > > > > > > > > > > > > > > > > > they will cause one negative > > > > > > > > > > > > > > > > > > > > > > result > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi > > > > > > > > > > > > > > > > > > > > > > to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart > > > > > > > > > > > > > > > > > > > > > > > > can be main hart in U- > > > > > > > > > > > > > > > > > > > > > > > > Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it > > > > > > > > > > > > > > > > > > > > > > > > still fail somewhere. > > > > > > > > > > > > > > > > > > > > > > > > After dig in > > > > > > > > > > > > > > > > > > > > > > > > and found there is an > > > > > > > > > > > > > > > > > > > > > > > > assumption that hart 0 > > > > > > > > > > > > > > > > > > > > > > > > shall be > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is > > > > > > > > > > > > > > > > > > > > > > > a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. > > > > > > > > > > > > > > > > > > > > > > Maybe it is a compatible > > > > > > > > > > > > > > > > > > > > > > issue. > > > > > > > > > > > > > > > > > > > > > > There is a limitation that > > > > > > > > > > > > > > > > > > > > > > only hart 0 can be main hart > > > > > > > > > > > > > > > > > > > > > > in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such > > > > > > > > > > > > > > > > > > > > > limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs > > > > > > > > > > > > > > > > > > > > of the initialization are the > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > 2. determine which route to take > > > > > > > > > > > > > > > > > > > > based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this > > > > > > > > > > > > > > > > > > > > signature, if you are not willing > > > > > > > > > > > > > > > > > > > > to call it > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was > > > > > > > > > > > > > > > > > > > not there until we added self- > > > > > > > > > > > > > > > > > > > relocation > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI > > > > > > > > > > > > > > > > > > > but we might end-up having > > > > > > > > > > > > > > > > > > > boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this > > > > > > > > > > > > > > > > > > OpenSBI: > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce > > > > > > > > > > > > > > > > > > relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if > > > > > > > > > > > > > > > > > > that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide > > > > > > > > > > > > > > > > > > Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing > > > > > > > > > > > > > > > > list. > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed > > > > > > > > > > > > > > > multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 > > > > > > > > > > > > > > play as main hart. > > > > > > > > > > > > > > The hart 1 will receive ipi and come into > > > > > > > > > > > > > > OpenSBI(0x1000000) from > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still > > > > > > > > > > > > > > run in U-Boot SPL. > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower > > > > > > > > > > > > > > which will copy data from > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares > > > > > > > > > > > > > ensures that OpenSBI firmware > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering > > > > > > > > > > > > > C code. This helps > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the > > > > > > > > > > > > > U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U- > > > > > > > > > > > > > Boot SPL are linked to > > > > > > > > > > > > > address same address 0x0. This means secondary > > > > > > > > > > > > > HARTs cannot safely > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You > > > > > > > > > > > > > should hold secondary HARTs > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and > > > > > > > > > > > > > U-Boot proper are > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs > > > > > > > > > > > > > should jump to OpenSBI > > > > > > > > > > > > > at the same time after everything is loaded in > > > > > > > > > > > > > RAM. > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART > > > > > > > > > > > > jump to OpenSBI and primary > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > (Refer, jump_to_image_no_args() in > > > > > > > > > > > > arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U- > > > > > > > > > > > > Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot > > > > > > > > > > > > SPL or OpenSBI. I think > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be > > > > > > > > > > > > convenient since this series > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > I know it can avoid corrupting issue with > > > > > > > > > > > changing U-Boot SPL > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running > > > > > > > > > > on QEMU as well. > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base > > > > > > > > > > > can equal to OpenSBI text base > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when > > > > > > > > > > > jumping to OpenSBI > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, > > > > > > > > > > > other harts go to > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 > > > > > > > > > > register because this > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should > > > > > > > > > > pass preferred > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 > > > > > > > > > > branch of > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" > > > > > > > > > > passed to > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI > > > > > > > > > by U-Boot SPL. > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" > > > > > > > > > to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, > > > > > > > > > OpenSBI just > > > > > > > > > have the following change > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > change to > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > before this commit > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of > > > > > > > > passing > > > > > > > > value in a3 for these firmwares because these are not > > > > > > > > booted by U-Boot > > > > > > > > SPL. > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support > > > > > > > > which does not > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for > > > > > > > > this so that we can > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct > > > > > > > > fw_dynamic_info" > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen > > > > > > > > > from lottery mechanism. > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not > > > > > > > > > overlap with > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on > > > > > > > > QEMU as well. It's > > > > > > > > just that most of us did not notice it for U-Boot SPL on > > > > > > > > QEMU. > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the > > > > > > > right thing > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' > > > > > > > register. > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for > > > > > > > you ?? > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to > > > > > > OpenSBI. > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > Maybe below changes can help you... > > > > > > > > Thanks for looking into this issue! I successfully tested it on > > > > QEMU, I > > > > had to add a short delay between sending the IPIs to trigger the > > > > problem. > > > > > > > > We might still run into problems however. Right now, we are > > > > assuming > > > > that the main hart is the last one to enter OpenSBI. If this is > > > > not the > > > > case (some delay when handling the IPI), we will have the same > > > > problem > > > > again. To fix this we could pass the hart mask, containing all > > > > harts > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > running in OpenSBI. I am not sure how realistic this scenario is, > > > > so > > > > this might not be needed. > > > > > > I agree that we might still run into this issue if primary HART > > > enters > > > OpenSBI before secondary HARTs. I think this situation can only > > > happen on QEMU where each CPU is a thread running on host but > > > very unlikely/impossible on real HW. > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be > > > the > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > all HARTs. > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait- > > > loop > > > in fw_base.S which will add to the boot-time as well. > > > > > > I still think the root cause of the issue is that TEXT_START of > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > insist SOC vendors to not use same TEXT_START ? > > > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > > preferred_boot_hart_v2 branch > > It still encounter some booting problems. I try to find out the root > > cause but in vain. > > > > Just wanted to make sure that you have tried this patch. > > http://lists.infradead.org/pipermail/opensbi/2019-November/000672.html > > We should investigate the issue why it did not work for you if this > patch did not work for you. Yes, I try with this commit 831aa3c1ad2546a2b35ddf5b1aa0ce91cdc7fe89 firmware: Add preferred boot HART field in struct fw_dynamic_info It fail randomly yesterday, but this morning I try several times it will pass. I will keep trying. Thanks Rick > > > I am very agree with options of Lukas. > > After modifying U-Boot SPL text base not equal to zero and the > > booting > > progress will be pass. > > > > Thanks > > Rick > > > > > Regards, > > > Anup > > -- > Regards, > Atish ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-08 1:13 ` Rick Chen @ 2019-11-08 7:27 ` Rick Chen 2019-11-08 8:59 ` Auer, Lukas 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-08 7:27 UTC (permalink / raw) To: u-boot Hi Atish > > Hi Atish > > > > > On Thu, 2019-11-07 at 19:41 +0800, Rick Chen wrote: > > > Hi Anup & Lukas > > > > > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com > > > > > > > wrote: > > > > > > > Hi Anup > > > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel < > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen < > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen < > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel < > > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen < > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen < > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup > > > > > > > > > > > > > > > > > > > Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM > > > > > > > > > > > > > > > > > > > > Alan Kao <alankao@andestech.com> > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments > > > > > > > > > > > > > > > > > > > > > below. > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at > > > > > > > > > > > > > > > > > > > > > 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 > > > > > > > > > > > > > > > > > > > > > > AM Rick Chen < > > > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > 2:18 PM Andes < > > > > > > > > > > > > > > > > > > > > > > > > uboot at andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > rick at andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to > > > > > > > > > > > > > > > > > > > > > > > > > hart 0 always will be > > > > > > > > > > > > > > > > > > > > > > > > > main > > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When > > > > > > > > > > > > > > > > > > > > > > > > > develop SPL flow, I try > > > > > > > > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be > > > > > > > > > > > > > > > > > > > > > > > > > main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI > > > > > > > > > > > > > > > > > > > > > > > > > flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit > > > > > > > > > > > > > > > > > > > > > > > > contain 2 fixes, or just 1 > > > > > > > > > > > > > > > > > > > > > > > > fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But > > > > > > > > > > > > > > > > > > > > > > > they will cause one negative > > > > > > > > > > > > > > > > > > > > > > > result > > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi > > > > > > > > > > > > > > > > > > > > > > > to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart > > > > > > > > > > > > > > > > > > > > > > > > > can be main hart in U- > > > > > > > > > > > > > > > > > > > > > > > > > Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it > > > > > > > > > > > > > > > > > > > > > > > > > still fail somewhere. > > > > > > > > > > > > > > > > > > > > > > > > > After dig in > > > > > > > > > > > > > > > > > > > > > > > > > and found there is an > > > > > > > > > > > > > > > > > > > > > > > > > assumption that hart 0 > > > > > > > > > > > > > > > > > > > > > > > > > shall be > > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is > > > > > > > > > > > > > > > > > > > > > > > > a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. > > > > > > > > > > > > > > > > > > > > > > > Maybe it is a compatible > > > > > > > > > > > > > > > > > > > > > > > issue. > > > > > > > > > > > > > > > > > > > > > > > There is a limitation that > > > > > > > > > > > > > > > > > > > > > > > only hart 0 can be main hart > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such > > > > > > > > > > > > > > > > > > > > > > limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs > > > > > > > > > > > > > > > > > > > > > of the initialization are the > > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > > 2. determine which route to take > > > > > > > > > > > > > > > > > > > > > based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this > > > > > > > > > > > > > > > > > > > > > signature, if you are not willing > > > > > > > > > > > > > > > > > > > > > to call it > > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was > > > > > > > > > > > > > > > > > > > > not there until we added self- > > > > > > > > > > > > > > > > > > > > relocation > > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI > > > > > > > > > > > > > > > > > > > > but we might end-up having > > > > > > > > > > > > > > > > > > > > boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this > > > > > > > > > > > > > > > > > > > OpenSBI: > > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce > > > > > > > > > > > > > > > > > > > relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if > > > > > > > > > > > > > > > > > > > that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide > > > > > > > > > > > > > > > > > > > Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing > > > > > > > > > > > > > > > > > list. > > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed > > > > > > > > > > > > > > > > multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 > > > > > > > > > > > > > > > play as main hart. > > > > > > > > > > > > > > > The hart 1 will receive ipi and come into > > > > > > > > > > > > > > > OpenSBI(0x1000000) from > > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still > > > > > > > > > > > > > > > run in U-Boot SPL. > > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower > > > > > > > > > > > > > > > which will copy data from > > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares > > > > > > > > > > > > > > ensures that OpenSBI firmware > > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering > > > > > > > > > > > > > > C code. This helps > > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the > > > > > > > > > > > > > > U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U- > > > > > > > > > > > > > > Boot SPL are linked to > > > > > > > > > > > > > > address same address 0x0. This means secondary > > > > > > > > > > > > > > HARTs cannot safely > > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You > > > > > > > > > > > > > > should hold secondary HARTs > > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and > > > > > > > > > > > > > > U-Boot proper are > > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs > > > > > > > > > > > > > > should jump to OpenSBI > > > > > > > > > > > > > > at the same time after everything is loaded in > > > > > > > > > > > > > > RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART > > > > > > > > > > > > > jump to OpenSBI and primary > > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > > (Refer, jump_to_image_no_args() in > > > > > > > > > > > > > arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U- > > > > > > > > > > > > > Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot > > > > > > > > > > > > > SPL or OpenSBI. I think > > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be > > > > > > > > > > > > > convenient since this series > > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > I know it can avoid corrupting issue with > > > > > > > > > > > > changing U-Boot SPL > > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running > > > > > > > > > > > on QEMU as well. > > > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base > > > > > > > > > > > > can equal to OpenSBI text base > > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when > > > > > > > > > > > > jumping to OpenSBI > > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, > > > > > > > > > > > > other harts go to > > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 > > > > > > > > > > > register because this > > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should > > > > > > > > > > > pass preferred > > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 > > > > > > > > > > > branch of > > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" > > > > > > > > > > > passed to > > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI > > > > > > > > > > by U-Boot SPL. > > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" > > > > > > > > > > to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, > > > > > > > > > > OpenSBI just > > > > > > > > > > have the following change > > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > > change to > > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > > before this commit > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of > > > > > > > > > passing > > > > > > > > > value in a3 for these firmwares because these are not > > > > > > > > > booted by U-Boot > > > > > > > > > SPL. > > > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support > > > > > > > > > which does not > > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for > > > > > > > > > this so that we can > > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct > > > > > > > > > fw_dynamic_info" > > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen > > > > > > > > > > from lottery mechanism. > > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not > > > > > > > > > > overlap with > > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on > > > > > > > > > QEMU as well. It's > > > > > > > > > just that most of us did not notice it for U-Boot SPL on > > > > > > > > > QEMU. > > > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the > > > > > > > > right thing > > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' > > > > > > > > register. > > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for > > > > > > > > you ?? > > > > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to > > > > > > > OpenSBI. > > > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > > > Maybe below changes can help you... > > > > > > > > > > Thanks for looking into this issue! I successfully tested it on > > > > > QEMU, I > > > > > had to add a short delay between sending the IPIs to trigger the > > > > > problem. > > > > > > > > > > We might still run into problems however. Right now, we are > > > > > assuming > > > > > that the main hart is the last one to enter OpenSBI. If this is > > > > > not the > > > > > case (some delay when handling the IPI), we will have the same > > > > > problem > > > > > again. To fix this we could pass the hart mask, containing all > > > > > harts > > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > > running in OpenSBI. I am not sure how realistic this scenario is, > > > > > so > > > > > this might not be needed. > > > > > > > > I agree that we might still run into this issue if primary HART > > > > enters > > > > OpenSBI before secondary HARTs. I think this situation can only > > > > happen on QEMU where each CPU is a thread running on host but > > > > very unlikely/impossible on real HW. > > > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be > > > > the > > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > > all HARTs. > > > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait- > > > > loop > > > > in fw_base.S which will add to the boot-time as well. > > > > > > > > I still think the root cause of the issue is that TEXT_START of > > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > > insist SOC vendors to not use same TEXT_START ? > > > > > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > > > preferred_boot_hart_v2 branch > > > It still encounter some booting problems. I try to find out the root > > > cause but in vain. > > > > > > > Just wanted to make sure that you have tried this patch. > > > > http://lists.infradead.org/pipermail/opensbi/2019-November/000672.html > > > > We should investigate the issue why it did not work for you if this > > patch did not work for you. > > Yes, I try with this > commit 831aa3c1ad2546a2b35ddf5b1aa0ce91cdc7fe89 > firmware: Add preferred boot HART field in struct fw_dynamic_info > > It fail randomly yesterday, but this morning I try several times it will pass. > I will keep trying. > I have figure out one fail case which is belong to main hart of U-Boot SPL is not the last hart while entering OpenSBI case 1 (fail) hb *0x1000000 Thread 2 hit Breakpoint 3, 0x0000000001000000 in ?? () (gdb) info threads Id Target Id Frame 1 Thread 1 (hart 1) 0x0000000001000000 in ?? () * 2 Thread 2 (hart 2) 0x0000000001000000 in ?? () 3 Thread 3 (hart 3) secondary_hart_loop () at arch/riscv/cpu/start.S:416 4 Thread 4 (hart 4) secondary_hart_loop () at arch/riscv/cpu/start.S:416 (gdb) p/x gd->arch.boot_hart $3 = 0x1 main hart is hart 1 in U-Boot SPL, hart 1 have arrived OpenSBI, but hart 3, 4 still run in U-Boot SPL c U-Boot console print message U-Boot SPL 2019.10-00603-g850848e-dirty (Nov 08 2019 - 14:37:54 +0800) Trying to boot from RAM U-Boot 2019.10-00603-g850848e-dirty (Nov 08 2019 - 14:37:54 +0800) DRAM: 1 GiB Flash: <== hang here Thanks Rick ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-08 7:27 ` Rick Chen @ 2019-11-08 8:59 ` Auer, Lukas 2019-11-11 7:19 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Auer, Lukas @ 2019-11-08 8:59 UTC (permalink / raw) To: u-boot Hi Rick, On Fri, 2019-11-08 at 15:27 +0800, Rick Chen wrote: > Hi Atish > > > Hi Atish > > > > > On Thu, 2019-11-07 at 19:41 +0800, Rick Chen wrote: > > > > Hi Anup & Lukas > > > > > > > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com > > > > > > > > wrote: > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel < > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen < > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen < > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel < > > > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen < > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen < > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup > > > > > > > > > > > > > > > > > > > > Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM > > > > > > > > > > > > > > > > > > > > > Alan Kao <alankao@andestech.com> > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments > > > > > > > > > > > > > > > > > > > > > > below. > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at > > > > > > > > > > > > > > > > > > > > > > 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 > > > > > > > > > > > > > > > > > > > > > > > AM Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > > 2:18 PM Andes < > > > > > > > > > > > > > > > > > > > > > > > > > uboot at andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > > rick at andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to > > > > > > > > > > > > > > > > > > > > > > > > > > hart 0 always will be > > > > > > > > > > > > > > > > > > > > > > > > > > main > > > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When > > > > > > > > > > > > > > > > > > > > > > > > > > develop SPL flow, I try > > > > > > > > > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be > > > > > > > > > > > > > > > > > > > > > > > > > > main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI > > > > > > > > > > > > > > > > > > > > > > > > > > flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit > > > > > > > > > > > > > > > > > > > > > > > > > contain 2 fixes, or just 1 > > > > > > > > > > > > > > > > > > > > > > > > > fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But > > > > > > > > > > > > > > > > > > > > > > > > they will cause one negative > > > > > > > > > > > > > > > > > > > > > > > > result > > > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi > > > > > > > > > > > > > > > > > > > > > > > > to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart > > > > > > > > > > > > > > > > > > > > > > > > > > can be main hart in U- > > > > > > > > > > > > > > > > > > > > > > > > > > Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it > > > > > > > > > > > > > > > > > > > > > > > > > > still fail somewhere. > > > > > > > > > > > > > > > > > > > > > > > > > > After dig in > > > > > > > > > > > > > > > > > > > > > > > > > > and found there is an > > > > > > > > > > > > > > > > > > > > > > > > > > assumption that hart 0 > > > > > > > > > > > > > > > > > > > > > > > > > > shall be > > > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is > > > > > > > > > > > > > > > > > > > > > > > > > a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. > > > > > > > > > > > > > > > > > > > > > > > > Maybe it is a compatible > > > > > > > > > > > > > > > > > > > > > > > > issue. > > > > > > > > > > > > > > > > > > > > > > > > There is a limitation that > > > > > > > > > > > > > > > > > > > > > > > > only hart 0 can be main hart > > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such > > > > > > > > > > > > > > > > > > > > > > > limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs > > > > > > > > > > > > > > > > > > > > > > of the initialization are the > > > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > > > 2. determine which route to take > > > > > > > > > > > > > > > > > > > > > > based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this > > > > > > > > > > > > > > > > > > > > > > signature, if you are not willing > > > > > > > > > > > > > > > > > > > > > > to call it > > > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was > > > > > > > > > > > > > > > > > > > > > not there until we added self- > > > > > > > > > > > > > > > > > > > > > relocation > > > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI > > > > > > > > > > > > > > > > > > > > > but we might end-up having > > > > > > > > > > > > > > > > > > > > > boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this > > > > > > > > > > > > > > > > > > > > OpenSBI: > > > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce > > > > > > > > > > > > > > > > > > > > relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if > > > > > > > > > > > > > > > > > > > > that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide > > > > > > > > > > > > > > > > > > > > Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing > > > > > > > > > > > > > > > > > > list. > > > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed > > > > > > > > > > > > > > > > > multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 > > > > > > > > > > > > > > > > play as main hart. > > > > > > > > > > > > > > > > The hart 1 will receive ipi and come into > > > > > > > > > > > > > > > > OpenSBI(0x1000000) from > > > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still > > > > > > > > > > > > > > > > run in U-Boot SPL. > > > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower > > > > > > > > > > > > > > > > which will copy data from > > > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares > > > > > > > > > > > > > > > ensures that OpenSBI firmware > > > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering > > > > > > > > > > > > > > > C code. This helps > > > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the > > > > > > > > > > > > > > > U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U- > > > > > > > > > > > > > > > Boot SPL are linked to > > > > > > > > > > > > > > > address same address 0x0. This means secondary > > > > > > > > > > > > > > > HARTs cannot safely > > > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You > > > > > > > > > > > > > > > should hold secondary HARTs > > > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and > > > > > > > > > > > > > > > U-Boot proper are > > > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs > > > > > > > > > > > > > > > should jump to OpenSBI > > > > > > > > > > > > > > > at the same time after everything is loaded in > > > > > > > > > > > > > > > RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART > > > > > > > > > > > > > > jump to OpenSBI and primary > > > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > > > (Refer, jump_to_image_no_args() in > > > > > > > > > > > > > > arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U- > > > > > > > > > > > > > > Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot > > > > > > > > > > > > > > SPL or OpenSBI. I think > > > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be > > > > > > > > > > > > > > convenient since this series > > > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > > I know it can avoid corrupting issue with > > > > > > > > > > > > > changing U-Boot SPL > > > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running > > > > > > > > > > > > on QEMU as well. > > > > > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base > > > > > > > > > > > > > can equal to OpenSBI text base > > > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when > > > > > > > > > > > > > jumping to OpenSBI > > > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, > > > > > > > > > > > > > other harts go to > > > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 > > > > > > > > > > > > register because this > > > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should > > > > > > > > > > > > pass preferred > > > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 > > > > > > > > > > > > branch of > > > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" > > > > > > > > > > > > passed to > > > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI > > > > > > > > > > > by U-Boot SPL. > > > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" > > > > > > > > > > > to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, > > > > > > > > > > > OpenSBI just > > > > > > > > > > > have the following change > > > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > > > change to > > > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > > > before this commit > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of > > > > > > > > > > passing > > > > > > > > > > value in a3 for these firmwares because these are not > > > > > > > > > > booted by U-Boot > > > > > > > > > > SPL. > > > > > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support > > > > > > > > > > which does not > > > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for > > > > > > > > > > this so that we can > > > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct > > > > > > > > > > fw_dynamic_info" > > > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen > > > > > > > > > > > from lottery mechanism. > > > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not > > > > > > > > > > > overlap with > > > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on > > > > > > > > > > QEMU as well. It's > > > > > > > > > > just that most of us did not notice it for U-Boot SPL on > > > > > > > > > > QEMU. > > > > > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the > > > > > > > > > right thing > > > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' > > > > > > > > > register. > > > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for > > > > > > > > > you ?? > > > > > > > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to > > > > > > > > OpenSBI. > > > > > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > > > > > Maybe below changes can help you... > > > > > > > > > > > > Thanks for looking into this issue! I successfully tested it on > > > > > > QEMU, I > > > > > > had to add a short delay between sending the IPIs to trigger the > > > > > > problem. > > > > > > > > > > > > We might still run into problems however. Right now, we are > > > > > > assuming > > > > > > that the main hart is the last one to enter OpenSBI. If this is > > > > > > not the > > > > > > case (some delay when handling the IPI), we will have the same > > > > > > problem > > > > > > again. To fix this we could pass the hart mask, containing all > > > > > > harts > > > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > > > running in OpenSBI. I am not sure how realistic this scenario is, > > > > > > so > > > > > > this might not be needed. > > > > > > > > > > I agree that we might still run into this issue if primary HART > > > > > enters > > > > > OpenSBI before secondary HARTs. I think this situation can only > > > > > happen on QEMU where each CPU is a thread running on host but > > > > > very unlikely/impossible on real HW. > > > > > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be > > > > > the > > > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > > > all HARTs. > > > > > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait- > > > > > loop > > > > > in fw_base.S which will add to the boot-time as well. > > > > > > > > > > I still think the root cause of the issue is that TEXT_START of > > > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > > > insist SOC vendors to not use same TEXT_START ? > > > > > > > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > > > > preferred_boot_hart_v2 branch > > > > It still encounter some booting problems. I try to find out the root > > > > cause but in vain. > > > > > > > > > > Just wanted to make sure that you have tried this patch. > > > > > > http://lists.infradead.org/pipermail/opensbi/2019-November/000672.html > > > > > > We should investigate the issue why it did not work for you if this > > > patch did not work for you. > > > > Yes, I try with this > > commit 831aa3c1ad2546a2b35ddf5b1aa0ce91cdc7fe89 > > firmware: Add preferred boot HART field in struct fw_dynamic_info > > > > It fail randomly yesterday, but this morning I try several times it will pass. > > I will keep trying. > > > > I have figure out one fail case which is belong to main hart of U-Boot > SPL is not the last hart while entering OpenSBI > Can you try this branch [1]? It includes a quick implementation of the changes a mentioned yesterday, where the main hart waits until all harts have received the IPI. [1]: https://github.com/lukasauer/u-boot/commits/riscv-opensbi-boot-hart Thanks, Lukas ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-08 8:59 ` Auer, Lukas @ 2019-11-11 7:19 ` Rick Chen 2019-11-12 9:47 ` Auer, Lukas 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-11 7:19 UTC (permalink / raw) To: u-boot Hi Lukas > > Hi Rick, > > On Fri, 2019-11-08 at 15:27 +0800, Rick Chen wrote: > > Hi Atish > > > > > Hi Atish > > > > > > > On Thu, 2019-11-07 at 19:41 +0800, Rick Chen wrote: > > > > > Hi Anup & Lukas > > > > > > > > > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com > > > > > > > > > wrote: > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel < > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen < > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen < > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel < > > > > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen < > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen < > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup > > > > > > > > > > > > > > > > > > > > > Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM > > > > > > > > > > > > > > > > > > > > > > Alan Kao <alankao@andestech.com> > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments > > > > > > > > > > > > > > > > > > > > > > > below. > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at > > > > > > > > > > > > > > > > > > > > > > > 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 > > > > > > > > > > > > > > > > > > > > > > > > AM Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > > > 2:18 PM Andes < > > > > > > > > > > > > > > > > > > > > > > > > > > uboot at andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > > > rick at andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to > > > > > > > > > > > > > > > > > > > > > > > > > > > hart 0 always will be > > > > > > > > > > > > > > > > > > > > > > > > > > > main > > > > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When > > > > > > > > > > > > > > > > > > > > > > > > > > > develop SPL flow, I try > > > > > > > > > > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be > > > > > > > > > > > > > > > > > > > > > > > > > > > main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI > > > > > > > > > > > > > > > > > > > > > > > > > > > flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit > > > > > > > > > > > > > > > > > > > > > > > > > > contain 2 fixes, or just 1 > > > > > > > > > > > > > > > > > > > > > > > > > > fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But > > > > > > > > > > > > > > > > > > > > > > > > > they will cause one negative > > > > > > > > > > > > > > > > > > > > > > > > > result > > > > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi > > > > > > > > > > > > > > > > > > > > > > > > > to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart > > > > > > > > > > > > > > > > > > > > > > > > > > > can be main hart in U- > > > > > > > > > > > > > > > > > > > > > > > > > > > Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it > > > > > > > > > > > > > > > > > > > > > > > > > > > still fail somewhere. > > > > > > > > > > > > > > > > > > > > > > > > > > > After dig in > > > > > > > > > > > > > > > > > > > > > > > > > > > and found there is an > > > > > > > > > > > > > > > > > > > > > > > > > > > assumption that hart 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > shall be > > > > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is > > > > > > > > > > > > > > > > > > > > > > > > > > a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. > > > > > > > > > > > > > > > > > > > > > > > > > Maybe it is a compatible > > > > > > > > > > > > > > > > > > > > > > > > > issue. > > > > > > > > > > > > > > > > > > > > > > > > > There is a limitation that > > > > > > > > > > > > > > > > > > > > > > > > > only hart 0 can be main hart > > > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such > > > > > > > > > > > > > > > > > > > > > > > > limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs > > > > > > > > > > > > > > > > > > > > > > > of the initialization are the > > > > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > > > > 2. determine which route to take > > > > > > > > > > > > > > > > > > > > > > > based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this > > > > > > > > > > > > > > > > > > > > > > > signature, if you are not willing > > > > > > > > > > > > > > > > > > > > > > > to call it > > > > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was > > > > > > > > > > > > > > > > > > > > > > not there until we added self- > > > > > > > > > > > > > > > > > > > > > > relocation > > > > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI > > > > > > > > > > > > > > > > > > > > > > but we might end-up having > > > > > > > > > > > > > > > > > > > > > > boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this > > > > > > > > > > > > > > > > > > > > > OpenSBI: > > > > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce > > > > > > > > > > > > > > > > > > > > > relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if > > > > > > > > > > > > > > > > > > > > > that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide > > > > > > > > > > > > > > > > > > > > > Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing > > > > > > > > > > > > > > > > > > > list. > > > > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed > > > > > > > > > > > > > > > > > > multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 > > > > > > > > > > > > > > > > > play as main hart. > > > > > > > > > > > > > > > > > The hart 1 will receive ipi and come into > > > > > > > > > > > > > > > > > OpenSBI(0x1000000) from > > > > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still > > > > > > > > > > > > > > > > > run in U-Boot SPL. > > > > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower > > > > > > > > > > > > > > > > > which will copy data from > > > > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares > > > > > > > > > > > > > > > > ensures that OpenSBI firmware > > > > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering > > > > > > > > > > > > > > > > C code. This helps > > > > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the > > > > > > > > > > > > > > > > U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U- > > > > > > > > > > > > > > > > Boot SPL are linked to > > > > > > > > > > > > > > > > address same address 0x0. This means secondary > > > > > > > > > > > > > > > > HARTs cannot safely > > > > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You > > > > > > > > > > > > > > > > should hold secondary HARTs > > > > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and > > > > > > > > > > > > > > > > U-Boot proper are > > > > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs > > > > > > > > > > > > > > > > should jump to OpenSBI > > > > > > > > > > > > > > > > at the same time after everything is loaded in > > > > > > > > > > > > > > > > RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART > > > > > > > > > > > > > > > jump to OpenSBI and primary > > > > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > > > > (Refer, jump_to_image_no_args() in > > > > > > > > > > > > > > > arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U- > > > > > > > > > > > > > > > Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot > > > > > > > > > > > > > > > SPL or OpenSBI. I think > > > > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be > > > > > > > > > > > > > > > convenient since this series > > > > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > > > I know it can avoid corrupting issue with > > > > > > > > > > > > > > changing U-Boot SPL > > > > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running > > > > > > > > > > > > > on QEMU as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base > > > > > > > > > > > > > > can equal to OpenSBI text base > > > > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when > > > > > > > > > > > > > > jumping to OpenSBI > > > > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, > > > > > > > > > > > > > > other harts go to > > > > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 > > > > > > > > > > > > > register because this > > > > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should > > > > > > > > > > > > > pass preferred > > > > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 > > > > > > > > > > > > > branch of > > > > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" > > > > > > > > > > > > > passed to > > > > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI > > > > > > > > > > > > by U-Boot SPL. > > > > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" > > > > > > > > > > > > to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, > > > > > > > > > > > > OpenSBI just > > > > > > > > > > > > have the following change > > > > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > > > > change to > > > > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > > > > before this commit > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of > > > > > > > > > > > passing > > > > > > > > > > > value in a3 for these firmwares because these are not > > > > > > > > > > > booted by U-Boot > > > > > > > > > > > SPL. > > > > > > > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support > > > > > > > > > > > which does not > > > > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for > > > > > > > > > > > this so that we can > > > > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct > > > > > > > > > > > fw_dynamic_info" > > > > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen > > > > > > > > > > > > from lottery mechanism. > > > > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not > > > > > > > > > > > > overlap with > > > > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on > > > > > > > > > > > QEMU as well. It's > > > > > > > > > > > just that most of us did not notice it for U-Boot SPL on > > > > > > > > > > > QEMU. > > > > > > > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the > > > > > > > > > > right thing > > > > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' > > > > > > > > > > register. > > > > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for > > > > > > > > > > you ?? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to > > > > > > > > > OpenSBI. > > > > > > > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > > > > > > > Maybe below changes can help you... > > > > > > > > > > > > > > Thanks for looking into this issue! I successfully tested it on > > > > > > > QEMU, I > > > > > > > had to add a short delay between sending the IPIs to trigger the > > > > > > > problem. > > > > > > > > > > > > > > We might still run into problems however. Right now, we are > > > > > > > assuming > > > > > > > that the main hart is the last one to enter OpenSBI. If this is > > > > > > > not the > > > > > > > case (some delay when handling the IPI), we will have the same > > > > > > > problem > > > > > > > again. To fix this we could pass the hart mask, containing all > > > > > > > harts > > > > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > > > > running in OpenSBI. I am not sure how realistic this scenario is, > > > > > > > so > > > > > > > this might not be needed. > > > > > > > > > > > > I agree that we might still run into this issue if primary HART > > > > > > enters > > > > > > OpenSBI before secondary HARTs. I think this situation can only > > > > > > happen on QEMU where each CPU is a thread running on host but > > > > > > very unlikely/impossible on real HW. > > > > > > > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > > > > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be > > > > > > the > > > > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > > > > all HARTs. > > > > > > > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait- > > > > > > loop > > > > > > in fw_base.S which will add to the boot-time as well. > > > > > > > > > > > > I still think the root cause of the issue is that TEXT_START of > > > > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > > > > insist SOC vendors to not use same TEXT_START ? > > > > > > > > > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > > > > > preferred_boot_hart_v2 branch > > > > > It still encounter some booting problems. I try to find out the root > > > > > cause but in vain. > > > > > > > > > > > > > Just wanted to make sure that you have tried this patch. > > > > > > > > http://lists.infradead.org/pipermail/opensbi/2019-November/000672.html > > > > > > > > We should investigate the issue why it did not work for you if this > > > > patch did not work for you. > > > > > > Yes, I try with this > > > commit 831aa3c1ad2546a2b35ddf5b1aa0ce91cdc7fe89 > > > firmware: Add preferred boot HART field in struct fw_dynamic_info > > > > > > It fail randomly yesterday, but this morning I try several times it will pass. > > > I will keep trying. > > > > > > > I have figure out one fail case which is belong to main hart of U-Boot > > SPL is not the last hart while entering OpenSBI > > > > Can you try this branch [1]? It includes a quick implementation of the > changes a mentioned yesterday, where the main hart waits until all > harts have received the IPI. > > [1]: > https://github.com/lukasauer/u-boot/commits/riscv-opensbi-boot-hart I have try this patch, but it seems can not guarantee main hart to be the last hart while leaving U-Boot SPL. Even the main hart have checked all harts have received the IPI, but it still have opportunities to arrive OpenSBI before other harts. Thanks Rick > > Thanks, > Lukas ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-11 7:19 ` Rick Chen @ 2019-11-12 9:47 ` Auer, Lukas 2019-11-13 3:42 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Auer, Lukas @ 2019-11-12 9:47 UTC (permalink / raw) To: u-boot Hi Rick, On Mon, 2019-11-11 at 15:19 +0800, Rick Chen wrote: > Hi Lukas > > > Hi Rick, > > > > On Fri, 2019-11-08 at 15:27 +0800, Rick Chen wrote: > > > Hi Atish > > > > > > > Hi Atish > > > > > > > > > On Thu, 2019-11-07 at 19:41 +0800, Rick Chen wrote: > > > > > > Hi Anup & Lukas > > > > > > > > > > > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > > > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > > > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel < > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen < > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen < > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel < > > > > > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen < > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen < > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup > > > > > > > > > > > > > > > > > > > > > > Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM > > > > > > > > > > > > > > > > > > > > > > > Alan Kao <alankao@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments > > > > > > > > > > > > > > > > > > > > > > > > below. > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 > > > > > > > > > > > > > > > > > > > > > > > > > AM Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > > > > 2:18 PM Andes < > > > > > > > > > > > > > > > > > > > > > > > > > > > uboot at andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > > > > rick at andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to > > > > > > > > > > > > > > > > > > > > > > > > > > > > hart 0 always will be > > > > > > > > > > > > > > > > > > > > > > > > > > > > main > > > > > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When > > > > > > > > > > > > > > > > > > > > > > > > > > > > develop SPL flow, I try > > > > > > > > > > > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be > > > > > > > > > > > > > > > > > > > > > > > > > > > > main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI > > > > > > > > > > > > > > > > > > > > > > > > > > > > flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit > > > > > > > > > > > > > > > > > > > > > > > > > > > contain 2 fixes, or just 1 > > > > > > > > > > > > > > > > > > > > > > > > > > > fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But > > > > > > > > > > > > > > > > > > > > > > > > > > they will cause one negative > > > > > > > > > > > > > > > > > > > > > > > > > > result > > > > > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi > > > > > > > > > > > > > > > > > > > > > > > > > > to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart > > > > > > > > > > > > > > > > > > > > > > > > > > > > can be main hart in U- > > > > > > > > > > > > > > > > > > > > > > > > > > > > Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it > > > > > > > > > > > > > > > > > > > > > > > > > > > > still fail somewhere. > > > > > > > > > > > > > > > > > > > > > > > > > > > > After dig in > > > > > > > > > > > > > > > > > > > > > > > > > > > > and found there is an > > > > > > > > > > > > > > > > > > > > > > > > > > > > assumption that hart 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > shall be > > > > > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is > > > > > > > > > > > > > > > > > > > > > > > > > > > a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. > > > > > > > > > > > > > > > > > > > > > > > > > > Maybe it is a compatible > > > > > > > > > > > > > > > > > > > > > > > > > > issue. > > > > > > > > > > > > > > > > > > > > > > > > > > There is a limitation that > > > > > > > > > > > > > > > > > > > > > > > > > > only hart 0 can be main hart > > > > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such > > > > > > > > > > > > > > > > > > > > > > > > > limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs > > > > > > > > > > > > > > > > > > > > > > > > of the initialization are the > > > > > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > > > > > 2. determine which route to take > > > > > > > > > > > > > > > > > > > > > > > > based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this > > > > > > > > > > > > > > > > > > > > > > > > signature, if you are not willing > > > > > > > > > > > > > > > > > > > > > > > > to call it > > > > > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was > > > > > > > > > > > > > > > > > > > > > > > not there until we added self- > > > > > > > > > > > > > > > > > > > > > > > relocation > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI > > > > > > > > > > > > > > > > > > > > > > > but we might end-up having > > > > > > > > > > > > > > > > > > > > > > > boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this > > > > > > > > > > > > > > > > > > > > > > OpenSBI: > > > > > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce > > > > > > > > > > > > > > > > > > > > > > relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if > > > > > > > > > > > > > > > > > > > > > > that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide > > > > > > > > > > > > > > > > > > > > > > Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing > > > > > > > > > > > > > > > > > > > > list. > > > > > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed > > > > > > > > > > > > > > > > > > > multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 > > > > > > > > > > > > > > > > > > play as main hart. > > > > > > > > > > > > > > > > > > The hart 1 will receive ipi and come into > > > > > > > > > > > > > > > > > > OpenSBI(0x1000000) from > > > > > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still > > > > > > > > > > > > > > > > > > run in U-Boot SPL. > > > > > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower > > > > > > > > > > > > > > > > > > which will copy data from > > > > > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares > > > > > > > > > > > > > > > > > ensures that OpenSBI firmware > > > > > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering > > > > > > > > > > > > > > > > > C code. This helps > > > > > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the > > > > > > > > > > > > > > > > > U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U- > > > > > > > > > > > > > > > > > Boot SPL are linked to > > > > > > > > > > > > > > > > > address same address 0x0. This means secondary > > > > > > > > > > > > > > > > > HARTs cannot safely > > > > > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You > > > > > > > > > > > > > > > > > should hold secondary HARTs > > > > > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and > > > > > > > > > > > > > > > > > U-Boot proper are > > > > > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs > > > > > > > > > > > > > > > > > should jump to OpenSBI > > > > > > > > > > > > > > > > > at the same time after everything is loaded in > > > > > > > > > > > > > > > > > RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART > > > > > > > > > > > > > > > > jump to OpenSBI and primary > > > > > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > > > > > (Refer, jump_to_image_no_args() in > > > > > > > > > > > > > > > > arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U- > > > > > > > > > > > > > > > > Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot > > > > > > > > > > > > > > > > SPL or OpenSBI. I think > > > > > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be > > > > > > > > > > > > > > > > convenient since this series > > > > > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > > > > I know it can avoid corrupting issue with > > > > > > > > > > > > > > > changing U-Boot SPL > > > > > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running > > > > > > > > > > > > > > on QEMU as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base > > > > > > > > > > > > > > > can equal to OpenSBI text base > > > > > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when > > > > > > > > > > > > > > > jumping to OpenSBI > > > > > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, > > > > > > > > > > > > > > > other harts go to > > > > > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 > > > > > > > > > > > > > > register because this > > > > > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should > > > > > > > > > > > > > > pass preferred > > > > > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 > > > > > > > > > > > > > > branch of > > > > > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" > > > > > > > > > > > > > > passed to > > > > > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI > > > > > > > > > > > > > by U-Boot SPL. > > > > > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" > > > > > > > > > > > > > to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, > > > > > > > > > > > > > OpenSBI just > > > > > > > > > > > > > have the following change > > > > > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > > > > > change to > > > > > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > > > > > before this commit > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of > > > > > > > > > > > > passing > > > > > > > > > > > > value in a3 for these firmwares because these are not > > > > > > > > > > > > booted by U-Boot > > > > > > > > > > > > SPL. > > > > > > > > > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support > > > > > > > > > > > > which does not > > > > > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for > > > > > > > > > > > > this so that we can > > > > > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct > > > > > > > > > > > > fw_dynamic_info" > > > > > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen > > > > > > > > > > > > > from lottery mechanism. > > > > > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not > > > > > > > > > > > > > overlap with > > > > > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on > > > > > > > > > > > > QEMU as well. It's > > > > > > > > > > > > just that most of us did not notice it for U-Boot SPL on > > > > > > > > > > > > QEMU. > > > > > > > > > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the > > > > > > > > > > > right thing > > > > > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' > > > > > > > > > > > register. > > > > > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for > > > > > > > > > > > you ?? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to > > > > > > > > > > OpenSBI. > > > > > > > > > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > > > > > > > > > Maybe below changes can help you... > > > > > > > > > > > > > > > > Thanks for looking into this issue! I successfully tested it on > > > > > > > > QEMU, I > > > > > > > > had to add a short delay between sending the IPIs to trigger the > > > > > > > > problem. > > > > > > > > > > > > > > > > We might still run into problems however. Right now, we are > > > > > > > > assuming > > > > > > > > that the main hart is the last one to enter OpenSBI. If this is > > > > > > > > not the > > > > > > > > case (some delay when handling the IPI), we will have the same > > > > > > > > problem > > > > > > > > again. To fix this we could pass the hart mask, containing all > > > > > > > > harts > > > > > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > > > > > running in OpenSBI. I am not sure how realistic this scenario is, > > > > > > > > so > > > > > > > > this might not be needed. > > > > > > > > > > > > > > I agree that we might still run into this issue if primary HART > > > > > > > enters > > > > > > > OpenSBI before secondary HARTs. I think this situation can only > > > > > > > happen on QEMU where each CPU is a thread running on host but > > > > > > > very unlikely/impossible on real HW. > > > > > > > > > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > > > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > > > > > > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be > > > > > > > the > > > > > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > > > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > > > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > > > > > all HARTs. > > > > > > > > > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait- > > > > > > > loop > > > > > > > in fw_base.S which will add to the boot-time as well. > > > > > > > > > > > > > > I still think the root cause of the issue is that TEXT_START of > > > > > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > > > > > insist SOC vendors to not use same TEXT_START ? > > > > > > > > > > > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > > > > > > preferred_boot_hart_v2 branch > > > > > > It still encounter some booting problems. I try to find out the root > > > > > > cause but in vain. > > > > > > > > > > > > > > > > Just wanted to make sure that you have tried this patch. > > > > > > > > > > http://lists.infradead.org/pipermail/opensbi/2019-November/000672.html > > > > > > > > > > We should investigate the issue why it did not work for you if this > > > > > patch did not work for you. > > > > > > > > Yes, I try with this > > > > commit 831aa3c1ad2546a2b35ddf5b1aa0ce91cdc7fe89 > > > > firmware: Add preferred boot HART field in struct fw_dynamic_info > > > > > > > > It fail randomly yesterday, but this morning I try several times it will pass. > > > > I will keep trying. > > > > > > > > > > I have figure out one fail case which is belong to main hart of U-Boot > > > SPL is not the last hart while entering OpenSBI > > > > > > > Can you try this branch [1]? It includes a quick implementation of the > > changes a mentioned yesterday, where the main hart waits until all > > harts have received the IPI. > > > > [1]: > > https://github.com/lukasauer/u-boot/commits/riscv-opensbi-boot-hart > > I have try this patch, but it seems can not guarantee main hart to be > the last hart while leaving U-Boot SPL. > Even the main hart have checked all harts have received the IPI, but > it still have opportunities to arrive OpenSBI before other harts. > Thanks for testing! Can you try again with the same branch? I added another patch so that clearing the IPI is the very last thing before jumping to the SMP function. If that does not help, we'll have to add a delay in spl_invoke_opensbi() to delay the main hart. Thanks, Lukas ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-12 9:47 ` Auer, Lukas @ 2019-11-13 3:42 ` Rick Chen 2019-11-14 7:27 ` Anup Patel 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-11-13 3:42 UTC (permalink / raw) To: u-boot Hi Lukas > > Hi Rick, > > On Mon, 2019-11-11 at 15:19 +0800, Rick Chen wrote: > > Hi Lukas > > > > > Hi Rick, > > > > > > On Fri, 2019-11-08 at 15:27 +0800, Rick Chen wrote: > > > > Hi Atish > > > > > > > > > Hi Atish > > > > > > > > > > > On Thu, 2019-11-07 at 19:41 +0800, Rick Chen wrote: > > > > > > > Hi Anup & Lukas > > > > > > > > > > > > > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > > > > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > > > > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > > > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com > > > > > > > > > > > wrote: > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel < > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen < > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen < > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel < > > > > > > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen < > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen < > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup > > > > > > > > > > > > > > > > > > > > > > > Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM > > > > > > > > > > > > > > > > > > > > > > > > Alan Kao <alankao@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments > > > > > > > > > > > > > > > > > > > > > > > > > below. > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > > 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 > > > > > > > > > > > > > > > > > > > > > > > > > > AM Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2:18 PM Andes < > > > > > > > > > > > > > > > > > > > > > > > > > > > > uboot at andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > rick at andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to > > > > > > > > > > > > > > > > > > > > > > > > > > > > > hart 0 always will be > > > > > > > > > > > > > > > > > > > > > > > > > > > > > main > > > > > > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When > > > > > > > > > > > > > > > > > > > > > > > > > > > > > develop SPL flow, I try > > > > > > > > > > > > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be > > > > > > > > > > > > > > > > > > > > > > > > > > > > > main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI > > > > > > > > > > > > > > > > > > > > > > > > > > > > > flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit > > > > > > > > > > > > > > > > > > > > > > > > > > > > contain 2 fixes, or just 1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But > > > > > > > > > > > > > > > > > > > > > > > > > > > they will cause one negative > > > > > > > > > > > > > > > > > > > > > > > > > > > result > > > > > > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi > > > > > > > > > > > > > > > > > > > > > > > > > > > to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart > > > > > > > > > > > > > > > > > > > > > > > > > > > > > can be main hart in U- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it > > > > > > > > > > > > > > > > > > > > > > > > > > > > > still fail somewhere. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > After dig in > > > > > > > > > > > > > > > > > > > > > > > > > > > > > and found there is an > > > > > > > > > > > > > > > > > > > > > > > > > > > > > assumption that hart 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > shall be > > > > > > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is > > > > > > > > > > > > > > > > > > > > > > > > > > > > a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. > > > > > > > > > > > > > > > > > > > > > > > > > > > Maybe it is a compatible > > > > > > > > > > > > > > > > > > > > > > > > > > > issue. > > > > > > > > > > > > > > > > > > > > > > > > > > > There is a limitation that > > > > > > > > > > > > > > > > > > > > > > > > > > > only hart 0 can be main hart > > > > > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such > > > > > > > > > > > > > > > > > > > > > > > > > > limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs > > > > > > > > > > > > > > > > > > > > > > > > > of the initialization are the > > > > > > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > > > > > > 2. determine which route to take > > > > > > > > > > > > > > > > > > > > > > > > > based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this > > > > > > > > > > > > > > > > > > > > > > > > > signature, if you are not willing > > > > > > > > > > > > > > > > > > > > > > > > > to call it > > > > > > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was > > > > > > > > > > > > > > > > > > > > > > > > not there until we added self- > > > > > > > > > > > > > > > > > > > > > > > > relocation > > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI > > > > > > > > > > > > > > > > > > > > > > > > but we might end-up having > > > > > > > > > > > > > > > > > > > > > > > > boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this > > > > > > > > > > > > > > > > > > > > > > > OpenSBI: > > > > > > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce > > > > > > > > > > > > > > > > > > > > > > > relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if > > > > > > > > > > > > > > > > > > > > > > > that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide > > > > > > > > > > > > > > > > > > > > > > > Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing > > > > > > > > > > > > > > > > > > > > > list. > > > > > > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed > > > > > > > > > > > > > > > > > > > > multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 > > > > > > > > > > > > > > > > > > > play as main hart. > > > > > > > > > > > > > > > > > > > The hart 1 will receive ipi and come into > > > > > > > > > > > > > > > > > > > OpenSBI(0x1000000) from > > > > > > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still > > > > > > > > > > > > > > > > > > > run in U-Boot SPL. > > > > > > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower > > > > > > > > > > > > > > > > > > > which will copy data from > > > > > > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares > > > > > > > > > > > > > > > > > > ensures that OpenSBI firmware > > > > > > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering > > > > > > > > > > > > > > > > > > C code. This helps > > > > > > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the > > > > > > > > > > > > > > > > > > U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U- > > > > > > > > > > > > > > > > > > Boot SPL are linked to > > > > > > > > > > > > > > > > > > address same address 0x0. This means secondary > > > > > > > > > > > > > > > > > > HARTs cannot safely > > > > > > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You > > > > > > > > > > > > > > > > > > should hold secondary HARTs > > > > > > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and > > > > > > > > > > > > > > > > > > U-Boot proper are > > > > > > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs > > > > > > > > > > > > > > > > > > should jump to OpenSBI > > > > > > > > > > > > > > > > > > at the same time after everything is loaded in > > > > > > > > > > > > > > > > > > RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART > > > > > > > > > > > > > > > > > jump to OpenSBI and primary > > > > > > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > > > > > > (Refer, jump_to_image_no_args() in > > > > > > > > > > > > > > > > > arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U- > > > > > > > > > > > > > > > > > Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot > > > > > > > > > > > > > > > > > SPL or OpenSBI. I think > > > > > > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be > > > > > > > > > > > > > > > > > convenient since this series > > > > > > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > > > > > I know it can avoid corrupting issue with > > > > > > > > > > > > > > > > changing U-Boot SPL > > > > > > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running > > > > > > > > > > > > > > > on QEMU as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base > > > > > > > > > > > > > > > > can equal to OpenSBI text base > > > > > > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when > > > > > > > > > > > > > > > > jumping to OpenSBI > > > > > > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, > > > > > > > > > > > > > > > > other harts go to > > > > > > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 > > > > > > > > > > > > > > > register because this > > > > > > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should > > > > > > > > > > > > > > > pass preferred > > > > > > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 > > > > > > > > > > > > > > > branch of > > > > > > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" > > > > > > > > > > > > > > > passed to > > > > > > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI > > > > > > > > > > > > > > by U-Boot SPL. > > > > > > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" > > > > > > > > > > > > > > to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, > > > > > > > > > > > > > > OpenSBI just > > > > > > > > > > > > > > have the following change > > > > > > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > > > > > > change to > > > > > > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > > > > > > before this commit > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of > > > > > > > > > > > > > passing > > > > > > > > > > > > > value in a3 for these firmwares because these are not > > > > > > > > > > > > > booted by U-Boot > > > > > > > > > > > > > SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support > > > > > > > > > > > > > which does not > > > > > > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for > > > > > > > > > > > > > this so that we can > > > > > > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct > > > > > > > > > > > > > fw_dynamic_info" > > > > > > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen > > > > > > > > > > > > > > from lottery mechanism. > > > > > > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not > > > > > > > > > > > > > > overlap with > > > > > > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on > > > > > > > > > > > > > QEMU as well. It's > > > > > > > > > > > > > just that most of us did not notice it for U-Boot SPL on > > > > > > > > > > > > > QEMU. > > > > > > > > > > > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the > > > > > > > > > > > > right thing > > > > > > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' > > > > > > > > > > > > register. > > > > > > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for > > > > > > > > > > > > you ?? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to > > > > > > > > > > > OpenSBI. > > > > > > > > > > > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > > > > > > > > > > > Maybe below changes can help you... > > > > > > > > > > > > > > > > > > Thanks for looking into this issue! I successfully tested it on > > > > > > > > > QEMU, I > > > > > > > > > had to add a short delay between sending the IPIs to trigger the > > > > > > > > > problem. > > > > > > > > > > > > > > > > > > We might still run into problems however. Right now, we are > > > > > > > > > assuming > > > > > > > > > that the main hart is the last one to enter OpenSBI. If this is > > > > > > > > > not the > > > > > > > > > case (some delay when handling the IPI), we will have the same > > > > > > > > > problem > > > > > > > > > again. To fix this we could pass the hart mask, containing all > > > > > > > > > harts > > > > > > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > > > > > > running in OpenSBI. I am not sure how realistic this scenario is, > > > > > > > > > so > > > > > > > > > this might not be needed. > > > > > > > > > > > > > > > > I agree that we might still run into this issue if primary HART > > > > > > > > enters > > > > > > > > OpenSBI before secondary HARTs. I think this situation can only > > > > > > > > happen on QEMU where each CPU is a thread running on host but > > > > > > > > very unlikely/impossible on real HW. > > > > > > > > > > > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > > > > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > > > > > > > > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be > > > > > > > > the > > > > > > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > > > > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > > > > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > > > > > > all HARTs. > > > > > > > > > > > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait- > > > > > > > > loop > > > > > > > > in fw_base.S which will add to the boot-time as well. > > > > > > > > > > > > > > > > I still think the root cause of the issue is that TEXT_START of > > > > > > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > > > > > > insist SOC vendors to not use same TEXT_START ? > > > > > > > > > > > > > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > > > > > > > preferred_boot_hart_v2 branch > > > > > > > It still encounter some booting problems. I try to find out the root > > > > > > > cause but in vain. > > > > > > > > > > > > > > > > > > > Just wanted to make sure that you have tried this patch. > > > > > > > > > > > > http://lists.infradead.org/pipermail/opensbi/2019-November/000672.html > > > > > > > > > > > > We should investigate the issue why it did not work for you if this > > > > > > patch did not work for you. > > > > > > > > > > Yes, I try with this > > > > > commit 831aa3c1ad2546a2b35ddf5b1aa0ce91cdc7fe89 > > > > > firmware: Add preferred boot HART field in struct fw_dynamic_info > > > > > > > > > > It fail randomly yesterday, but this morning I try several times it will pass. > > > > > I will keep trying. > > > > > > > > > > > > > I have figure out one fail case which is belong to main hart of U-Boot > > > > SPL is not the last hart while entering OpenSBI > > > > > > > > > > Can you try this branch [1]? It includes a quick implementation of the > > > changes a mentioned yesterday, where the main hart waits until all > > > harts have received the IPI. > > > > > > [1]: > > > https://github.com/lukasauer/u-boot/commits/riscv-opensbi-boot-hart > > > > I have try this patch, but it seems can not guarantee main hart to be > > the last hart while leaving U-Boot SPL. > > Even the main hart have checked all harts have received the IPI, but > > it still have opportunities to arrive OpenSBI before other harts. > > > > Thanks for testing! Can you try again with the same branch? I added > another patch so that clearing the IPI is the very last thing before > jumping to the SMP function. > If that does not help, we'll have to add a delay in > spl_invoke_opensbi() to delay the main hart. Thanks for your patch, it indeed solve the problem almost. It always pass the booting verification under free running, I try many times and can not hit the failure case. But if I do something with GDB, E.g Set a break in 0x6c2 (c.jalr s2) in handle_ipi() and then delete the break and free run after it hit this break then it will hit the failure case as below: ----------------------------- U-Boot SPL fail print message ----------------------------- U-Boot SPL 2020.01-rc1-08573-gc42669d-dirty (Nov 13 2019 - 10:31:12 +0800) Trying to boot from RAM boot_hart 0 U-Boot 2020.01-rc1-08573-gc42669d-dirty (Nov 13 2019 - 10:31:12 +0800) DRAM: 1 GiB Flash: --------------- gdb information --------------- Thread 4 hit Breakpoint 3, 0x00000000000006c2 in handle_ipi (hart=3) at arch/riscv/lib/smp.c:110 110 smp_function(hart, gd->arch.ipi[hart].arg0, gd->arch.ipi[hart].arg1); (gdb) inf othreads Undefined info command: "othreads". Try "help info". (gdb) info threads Id Target Id Frame 1 Thread 1 (hart 1) 0x000000000122a8f8 in ?? () 2 Thread 2 (hart 2) 0x00000000000006c2 in handle_ipi (hart=1) at arch/riscv/lib/smp.c:110 3 Thread 3 (hart 3) 0x00000000000006c2 in handle_ipi (hart=2) at arch/riscv/lib/smp.c:110 * 4 Thread 4 (hart 4) 0x00000000000006c2 in handle_ipi (hart=3) at arch/riscv/lib/smp.c:110 (gdb) d Delete all breakpoints? (y or n) y (gdb) c Continuing. ---------- handle_ipi ---------- 0000000000000678 <handle_ipi>: 678: 479d c.li a5,7 67a: 04a7e663 bltu a5,a0,6c6 <handle_ipi+0x4e> 67e: 6a1042ef jal t0,551e <__riscv_save_2> 682: 842a c.mv s0,a0 684: 0330000f fence rw,rw 688: 44e1 c.li s1,24 68a: 029504b3 mul s1,a0,s1 68e: 003487b3 add a5,s1,gp 692: 1507b903 ld s2,336(a5) 696: d57ff0ef jal ra,3ec <invalidate_icache_all> 69a: 0004051b sext.w a0,s0 69e: e4bff0ef jal ra,4e8 <riscv_clear_ipi> 6a2: c911 c.beqz a0,6b6 <handle_ipi+0x3e> 6a4: 85a2 c.mv a1,s0 6a6: 00005517 auipc a0,0x5 6aa: 36250513 addi a0,a0,866 # 5a08 <_ctype+0x440> 6ae: 3f8030ef jal ra,3aa6 <printf> 6b2: 6a50406f j 5556 <__riscv_restore_2> 6b6: 948e c.add s1,gp 6b8: 1604b603 ld a2,352(s1) 6bc: 1584b583 ld a1,344(s1) 6c0: 8522 c.mv a0,s0 6c2: 9902 c.jalr s2 6c4: b7fd c.j 6b2 <handle_ipi+0x3a> 6c6: 8082 c.jr ra Maybe it is a little picky and fussy, what I try to say is that someone maybe can not understand why the main hart shall be the last hart when leaving U-Boot SPL when they review this part. I think the better way shall be that make sure all harts are there before relocation in OpenSBI (You mentioned it before) Thanks for your sharing this way to avoid this problem again. It really provide me another way to work around in current status. I will keep trying it's robustness and reliability under free running. Thanks Rick > > Thanks, > Lukas ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-13 3:42 ` Rick Chen @ 2019-11-14 7:27 ` Anup Patel 2019-11-14 17:10 ` Auer, Lukas 0 siblings, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-11-14 7:27 UTC (permalink / raw) To: u-boot On Wed, Nov 13, 2019 at 9:13 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Lukas > > > > > Hi Rick, > > > > On Mon, 2019-11-11 at 15:19 +0800, Rick Chen wrote: > > > Hi Lukas > > > > > > > Hi Rick, > > > > > > > > On Fri, 2019-11-08 at 15:27 +0800, Rick Chen wrote: > > > > > Hi Atish > > > > > > > > > > > Hi Atish > > > > > > > > > > > > > On Thu, 2019-11-07 at 19:41 +0800, Rick Chen wrote: > > > > > > > > Hi Anup & Lukas > > > > > > > > > > > > > > > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > > > > > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > > > > > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > > > > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > > > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel < > > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen < > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen < > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel < > > > > > > > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen < > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen < > > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup > > > > > > > > > > > > > > > > > > > > > > > > Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM > > > > > > > > > > > > > > > > > > > > > > > > > Alan Kao <alankao@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments > > > > > > > > > > > > > > > > > > > > > > > > > > below. > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > > > 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 > > > > > > > > > > > > > > > > > > > > > > > > > > > AM Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2:18 PM Andes < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > uboot at andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > rick at andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > hart 0 always will be > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > main > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > develop SPL flow, I try > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit > > > > > > > > > > > > > > > > > > > > > > > > > > > > > contain 2 fixes, or just 1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But > > > > > > > > > > > > > > > > > > > > > > > > > > > > they will cause one negative > > > > > > > > > > > > > > > > > > > > > > > > > > > > result > > > > > > > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi > > > > > > > > > > > > > > > > > > > > > > > > > > > > to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > can be main hart in U- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > still fail somewhere. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > After dig in > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > and found there is an > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > assumption that hart 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > shall be > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is > > > > > > > > > > > > > > > > > > > > > > > > > > > > > a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Maybe it is a compatible > > > > > > > > > > > > > > > > > > > > > > > > > > > > issue. > > > > > > > > > > > > > > > > > > > > > > > > > > > > There is a limitation that > > > > > > > > > > > > > > > > > > > > > > > > > > > > only hart 0 can be main hart > > > > > > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such > > > > > > > > > > > > > > > > > > > > > > > > > > > limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs > > > > > > > > > > > > > > > > > > > > > > > > > > of the initialization are the > > > > > > > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > > > > > > > 2. determine which route to take > > > > > > > > > > > > > > > > > > > > > > > > > > based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this > > > > > > > > > > > > > > > > > > > > > > > > > > signature, if you are not willing > > > > > > > > > > > > > > > > > > > > > > > > > > to call it > > > > > > > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was > > > > > > > > > > > > > > > > > > > > > > > > > not there until we added self- > > > > > > > > > > > > > > > > > > > > > > > > > relocation > > > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI > > > > > > > > > > > > > > > > > > > > > > > > > but we might end-up having > > > > > > > > > > > > > > > > > > > > > > > > > boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this > > > > > > > > > > > > > > > > > > > > > > > > OpenSBI: > > > > > > > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce > > > > > > > > > > > > > > > > > > > > > > > > relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if > > > > > > > > > > > > > > > > > > > > > > > > that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide > > > > > > > > > > > > > > > > > > > > > > > > Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing > > > > > > > > > > > > > > > > > > > > > > list. > > > > > > > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed > > > > > > > > > > > > > > > > > > > > > multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 > > > > > > > > > > > > > > > > > > > > play as main hart. > > > > > > > > > > > > > > > > > > > > The hart 1 will receive ipi and come into > > > > > > > > > > > > > > > > > > > > OpenSBI(0x1000000) from > > > > > > > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still > > > > > > > > > > > > > > > > > > > > run in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower > > > > > > > > > > > > > > > > > > > > which will copy data from > > > > > > > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares > > > > > > > > > > > > > > > > > > > ensures that OpenSBI firmware > > > > > > > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering > > > > > > > > > > > > > > > > > > > C code. This helps > > > > > > > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the > > > > > > > > > > > > > > > > > > > U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U- > > > > > > > > > > > > > > > > > > > Boot SPL are linked to > > > > > > > > > > > > > > > > > > > address same address 0x0. This means secondary > > > > > > > > > > > > > > > > > > > HARTs cannot safely > > > > > > > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You > > > > > > > > > > > > > > > > > > > should hold secondary HARTs > > > > > > > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and > > > > > > > > > > > > > > > > > > > U-Boot proper are > > > > > > > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs > > > > > > > > > > > > > > > > > > > should jump to OpenSBI > > > > > > > > > > > > > > > > > > > at the same time after everything is loaded in > > > > > > > > > > > > > > > > > > > RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART > > > > > > > > > > > > > > > > > > jump to OpenSBI and primary > > > > > > > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > > > > > > > (Refer, jump_to_image_no_args() in > > > > > > > > > > > > > > > > > > arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U- > > > > > > > > > > > > > > > > > > Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot > > > > > > > > > > > > > > > > > > SPL or OpenSBI. I think > > > > > > > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be > > > > > > > > > > > > > > > > > > convenient since this series > > > > > > > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > > > > > > I know it can avoid corrupting issue with > > > > > > > > > > > > > > > > > changing U-Boot SPL > > > > > > > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running > > > > > > > > > > > > > > > > on QEMU as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base > > > > > > > > > > > > > > > > > can equal to OpenSBI text base > > > > > > > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when > > > > > > > > > > > > > > > > > jumping to OpenSBI > > > > > > > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, > > > > > > > > > > > > > > > > > other harts go to > > > > > > > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 > > > > > > > > > > > > > > > > register because this > > > > > > > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should > > > > > > > > > > > > > > > > pass preferred > > > > > > > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 > > > > > > > > > > > > > > > > branch of > > > > > > > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" > > > > > > > > > > > > > > > > passed to > > > > > > > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI > > > > > > > > > > > > > > > by U-Boot SPL. > > > > > > > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" > > > > > > > > > > > > > > > to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, > > > > > > > > > > > > > > > OpenSBI just > > > > > > > > > > > > > > > have the following change > > > > > > > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > > > > > > > change to > > > > > > > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > > > > > > > before this commit > > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of > > > > > > > > > > > > > > passing > > > > > > > > > > > > > > value in a3 for these firmwares because these are not > > > > > > > > > > > > > > booted by U-Boot > > > > > > > > > > > > > > SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support > > > > > > > > > > > > > > which does not > > > > > > > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for > > > > > > > > > > > > > > this so that we can > > > > > > > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct > > > > > > > > > > > > > > fw_dynamic_info" > > > > > > > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen > > > > > > > > > > > > > > > from lottery mechanism. > > > > > > > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not > > > > > > > > > > > > > > > overlap with > > > > > > > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on > > > > > > > > > > > > > > QEMU as well. It's > > > > > > > > > > > > > > just that most of us did not notice it for U-Boot SPL on > > > > > > > > > > > > > > QEMU. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the > > > > > > > > > > > > > right thing > > > > > > > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' > > > > > > > > > > > > > register. > > > > > > > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for > > > > > > > > > > > > > you ?? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to > > > > > > > > > > > > OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > > > > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > > > > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > > > > > > > > > > > > > Maybe below changes can help you... > > > > > > > > > > > > > > > > > > > > Thanks for looking into this issue! I successfully tested it on > > > > > > > > > > QEMU, I > > > > > > > > > > had to add a short delay between sending the IPIs to trigger the > > > > > > > > > > problem. > > > > > > > > > > > > > > > > > > > > We might still run into problems however. Right now, we are > > > > > > > > > > assuming > > > > > > > > > > that the main hart is the last one to enter OpenSBI. If this is > > > > > > > > > > not the > > > > > > > > > > case (some delay when handling the IPI), we will have the same > > > > > > > > > > problem > > > > > > > > > > again. To fix this we could pass the hart mask, containing all > > > > > > > > > > harts > > > > > > > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > > > > > > > running in OpenSBI. I am not sure how realistic this scenario is, > > > > > > > > > > so > > > > > > > > > > this might not be needed. > > > > > > > > > > > > > > > > > > I agree that we might still run into this issue if primary HART > > > > > > > > > enters > > > > > > > > > OpenSBI before secondary HARTs. I think this situation can only > > > > > > > > > happen on QEMU where each CPU is a thread running on host but > > > > > > > > > very unlikely/impossible on real HW. > > > > > > > > > > > > > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > > > > > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > > > > > > > > > > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be > > > > > > > > > the > > > > > > > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > > > > > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > > > > > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > > > > > > > all HARTs. > > > > > > > > > > > > > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait- > > > > > > > > > loop > > > > > > > > > in fw_base.S which will add to the boot-time as well. > > > > > > > > > > > > > > > > > > I still think the root cause of the issue is that TEXT_START of > > > > > > > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > > > > > > > insist SOC vendors to not use same TEXT_START ? > > > > > > > > > > > > > > > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > > > > > > > > preferred_boot_hart_v2 branch > > > > > > > > It still encounter some booting problems. I try to find out the root > > > > > > > > cause but in vain. > > > > > > > > > > > > > > > > > > > > > > Just wanted to make sure that you have tried this patch. > > > > > > > > > > > > > > http://lists.infradead.org/pipermail/opensbi/2019-November/000672.html > > > > > > > > > > > > > > We should investigate the issue why it did not work for you if this > > > > > > > patch did not work for you. > > > > > > > > > > > > Yes, I try with this > > > > > > commit 831aa3c1ad2546a2b35ddf5b1aa0ce91cdc7fe89 > > > > > > firmware: Add preferred boot HART field in struct fw_dynamic_info > > > > > > > > > > > > It fail randomly yesterday, but this morning I try several times it will pass. > > > > > > I will keep trying. > > > > > > > > > > > > > > > > I have figure out one fail case which is belong to main hart of U-Boot > > > > > SPL is not the last hart while entering OpenSBI > > > > > > > > > > > > > Can you try this branch [1]? It includes a quick implementation of the > > > > changes a mentioned yesterday, where the main hart waits until all > > > > harts have received the IPI. > > > > > > > > [1]: > > > > https://github.com/lukasauer/u-boot/commits/riscv-opensbi-boot-hart > > > > > > I have try this patch, but it seems can not guarantee main hart to be > > > the last hart while leaving U-Boot SPL. > > > Even the main hart have checked all harts have received the IPI, but > > > it still have opportunities to arrive OpenSBI before other harts. > > > > > > > Thanks for testing! Can you try again with the same branch? I added > > another patch so that clearing the IPI is the very last thing before > > jumping to the SMP function. > > If that does not help, we'll have to add a delay in > > spl_invoke_opensbi() to delay the main hart. > > > Thanks for your patch, it indeed solve the problem almost. > It always pass the booting verification under free running, I try many > times and can not hit the failure case. > > But if I do something with GDB, E.g > Set a break in 0x6c2 (c.jalr s2) in handle_ipi() and then delete the > break and free run after it hit this break > then it will hit the failure case as below: The current booting mechanism where all HARTs jump to each booting stage (including Linux) at the same time is very fragile. Attaching a debugger at boot-time will certainly break current way of booting all HARTs because we are holding a particular HART in debug state. I am not sure if we can handle the debugger case in software but in real-world deployments debugger won't be present so maybe we can ignore this case. The SBI v0.2 HSM extension will make things better. It would be even better if HW designers provide explicit HW mechanism to power-on/off HARTs at run-time. Regards, Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-14 7:27 ` Anup Patel @ 2019-11-14 17:10 ` Auer, Lukas 0 siblings, 0 replies; 71+ messages in thread From: Auer, Lukas @ 2019-11-14 17:10 UTC (permalink / raw) To: u-boot On Thu, 2019-11-14 at 12:57 +0530, Anup Patel wrote: > On Wed, Nov 13, 2019 at 9:13 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Lukas > > > > > Hi Rick, > > > > > > On Mon, 2019-11-11 at 15:19 +0800, Rick Chen wrote: > > > > Hi Lukas > > > > > > > > > Hi Rick, > > > > > > > > > > On Fri, 2019-11-08 at 15:27 +0800, Rick Chen wrote: > > > > > > Hi Atish > > > > > > > > > > > > > Hi Atish > > > > > > > > > > > > > > > On Thu, 2019-11-07 at 19:41 +0800, Rick Chen wrote: > > > > > > > > > Hi Anup & Lukas > > > > > > > > > > > > > > > > > > Anup Patel <anup@brainfault.org> 於 2019年11月7日 週四 下午6:44寫道: > > > > > > > > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > > > > > > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > > > > > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > > > > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel < > > > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen < > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen < > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel < > > > > > > > > > > > > > > > > > > > anup at brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen < > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen < > > > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup > > > > > > > > > > > > > > > > > > > > > > > > > Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM > > > > > > > > > > > > > > > > > > > > > > > > > > Alan Kao <alankao@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments > > > > > > > > > > > > > > > > > > > > > > > > > > > below. > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > > > > 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 > > > > > > > > > > > > > > > > > > > > > > > > > > > > AM Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > > > > rickchen36 at gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2:18 PM Andes < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > uboot at andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > rick at andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > hart 0 always will be > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > main > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > develop SPL flow, I try > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > contain 2 fixes, or just 1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But > > > > > > > > > > > > > > > > > > > > > > > > > > > > > they will cause one negative > > > > > > > > > > > > > > > > > > > > > > > > > > > > > result > > > > > > > > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi > > > > > > > > > > > > > > > > > > > > > > > > > > > > > to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > can be main hart in U- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > still fail somewhere. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > After dig in > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > and found there is an > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > assumption that hart 0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > shall be > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Maybe it is a compatible > > > > > > > > > > > > > > > > > > > > > > > > > > > > > issue. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > There is a limitation that > > > > > > > > > > > > > > > > > > > > > > > > > > > > > only hart 0 can be main hart > > > > > > > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such > > > > > > > > > > > > > > > > > > > > > > > > > > > > limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs > > > > > > > > > > > > > > > > > > > > > > > > > > > of the initialization are the > > > > > > > > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > > > > > > > > 2. determine which route to take > > > > > > > > > > > > > > > > > > > > > > > > > > > based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this > > > > > > > > > > > > > > > > > > > > > > > > > > > signature, if you are not willing > > > > > > > > > > > > > > > > > > > > > > > > > > > to call it > > > > > > > > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was > > > > > > > > > > > > > > > > > > > > > > > > > > not there until we added self- > > > > > > > > > > > > > > > > > > > > > > > > > > relocation > > > > > > > > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI > > > > > > > > > > > > > > > > > > > > > > > > > > but we might end-up having > > > > > > > > > > > > > > > > > > > > > > > > > > boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this > > > > > > > > > > > > > > > > > > > > > > > > > OpenSBI: > > > > > > > > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce > > > > > > > > > > > > > > > > > > > > > > > > > relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if > > > > > > > > > > > > > > > > > > > > > > > > > that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide > > > > > > > > > > > > > > > > > > > > > > > > > Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing > > > > > > > > > > > > > > > > > > > > > > > list. > > > > > > > > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed > > > > > > > > > > > > > > > > > > > > > > multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 > > > > > > > > > > > > > > > > > > > > > play as main hart. > > > > > > > > > > > > > > > > > > > > > The hart 1 will receive ipi and come into > > > > > > > > > > > > > > > > > > > > > OpenSBI(0x1000000) from > > > > > > > > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still > > > > > > > > > > > > > > > > > > > > > run in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower > > > > > > > > > > > > > > > > > > > > > which will copy data from > > > > > > > > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares > > > > > > > > > > > > > > > > > > > > ensures that OpenSBI firmware > > > > > > > > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering > > > > > > > > > > > > > > > > > > > > C code. This helps > > > > > > > > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the > > > > > > > > > > > > > > > > > > > > U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U- > > > > > > > > > > > > > > > > > > > > Boot SPL are linked to > > > > > > > > > > > > > > > > > > > > address same address 0x0. This means secondary > > > > > > > > > > > > > > > > > > > > HARTs cannot safely > > > > > > > > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You > > > > > > > > > > > > > > > > > > > > should hold secondary HARTs > > > > > > > > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and > > > > > > > > > > > > > > > > > > > > U-Boot proper are > > > > > > > > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs > > > > > > > > > > > > > > > > > > > > should jump to OpenSBI > > > > > > > > > > > > > > > > > > > > at the same time after everything is loaded in > > > > > > > > > > > > > > > > > > > > RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART > > > > > > > > > > > > > > > > > > > jump to OpenSBI and primary > > > > > > > > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > > > > > > > > (Refer, jump_to_image_no_args() in > > > > > > > > > > > > > > > > > > > arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U- > > > > > > > > > > > > > > > > > > > Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot > > > > > > > > > > > > > > > > > > > SPL or OpenSBI. I think > > > > > > > > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be > > > > > > > > > > > > > > > > > > > convenient since this series > > > > > > > > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > > > > > > > I know it can avoid corrupting issue with > > > > > > > > > > > > > > > > > > changing U-Boot SPL > > > > > > > > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running > > > > > > > > > > > > > > > > > on QEMU as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base > > > > > > > > > > > > > > > > > > can equal to OpenSBI text base > > > > > > > > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when > > > > > > > > > > > > > > > > > > jumping to OpenSBI > > > > > > > > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, > > > > > > > > > > > > > > > > > > other harts go to > > > > > > > > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 > > > > > > > > > > > > > > > > > register because this > > > > > > > > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should > > > > > > > > > > > > > > > > > pass preferred > > > > > > > > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 > > > > > > > > > > > > > > > > > branch of > > > > > > > > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" > > > > > > > > > > > > > > > > > passed to > > > > > > > > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI > > > > > > > > > > > > > > > > by U-Boot SPL. > > > > > > > > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" > > > > > > > > > > > > > > > > to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, > > > > > > > > > > > > > > > > OpenSBI just > > > > > > > > > > > > > > > > have the following change > > > > > > > > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > > > > > > > > change to > > > > > > > > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > > > > > > > > before this commit > > > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of > > > > > > > > > > > > > > > passing > > > > > > > > > > > > > > > value in a3 for these firmwares because these are not > > > > > > > > > > > > > > > booted by U-Boot > > > > > > > > > > > > > > > SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support > > > > > > > > > > > > > > > which does not > > > > > > > > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for > > > > > > > > > > > > > > > this so that we can > > > > > > > > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct > > > > > > > > > > > > > > > fw_dynamic_info" > > > > > > > > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen > > > > > > > > > > > > > > > > from lottery mechanism. > > > > > > > > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not > > > > > > > > > > > > > > > > overlap with > > > > > > > > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on > > > > > > > > > > > > > > > QEMU as well. It's > > > > > > > > > > > > > > > just that most of us did not notice it for U-Boot SPL on > > > > > > > > > > > > > > > QEMU. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the > > > > > > > > > > > > > > right thing > > > > > > > > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' > > > > > > > > > > > > > > register. > > > > > > > > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for > > > > > > > > > > > > > > you ?? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > > > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to > > > > > > > > > > > > > OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > > > > > > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > > > > > > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > > > > > > > > > > > > > > > Maybe below changes can help you... > > > > > > > > > > > > > > > > > > > > > > Thanks for looking into this issue! I successfully tested it on > > > > > > > > > > > QEMU, I > > > > > > > > > > > had to add a short delay between sending the IPIs to trigger the > > > > > > > > > > > problem. > > > > > > > > > > > > > > > > > > > > > > We might still run into problems however. Right now, we are > > > > > > > > > > > assuming > > > > > > > > > > > that the main hart is the last one to enter OpenSBI. If this is > > > > > > > > > > > not the > > > > > > > > > > > case (some delay when handling the IPI), we will have the same > > > > > > > > > > > problem > > > > > > > > > > > again. To fix this we could pass the hart mask, containing all > > > > > > > > > > > harts > > > > > > > > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > > > > > > > > running in OpenSBI. I am not sure how realistic this scenario is, > > > > > > > > > > > so > > > > > > > > > > > this might not be needed. > > > > > > > > > > > > > > > > > > > > I agree that we might still run into this issue if primary HART > > > > > > > > > > enters > > > > > > > > > > OpenSBI before secondary HARTs. I think this situation can only > > > > > > > > > > happen on QEMU where each CPU is a thread running on host but > > > > > > > > > > very unlikely/impossible on real HW. > > > > > > > > > > > > > > > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > > > > > > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > > > > > > > > > > > > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be > > > > > > > > > > the > > > > > > > > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > > > > > > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > > > > > > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > > > > > > > > all HARTs. > > > > > > > > > > > > > > > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait- > > > > > > > > > > loop > > > > > > > > > > in fw_base.S which will add to the boot-time as well. > > > > > > > > > > > > > > > > > > > > I still think the root cause of the issue is that TEXT_START of > > > > > > > > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > > > > > > > > insist SOC vendors to not use same TEXT_START ? > > > > > > > > > > > > > > > > > > I have try your changes about boot_hart for U-Boot SPL with OpenSBI, > > > > > > > > > preferred_boot_hart_v2 branch > > > > > > > > > It still encounter some booting problems. I try to find out the root > > > > > > > > > cause but in vain. > > > > > > > > > > > > > > > > > > > > > > > > > Just wanted to make sure that you have tried this patch. > > > > > > > > > > > > > > > > http://lists.infradead.org/pipermail/opensbi/2019-November/000672.html > > > > > > > > > > > > > > > > We should investigate the issue why it did not work for you if this > > > > > > > > patch did not work for you. > > > > > > > > > > > > > > Yes, I try with this > > > > > > > commit 831aa3c1ad2546a2b35ddf5b1aa0ce91cdc7fe89 > > > > > > > firmware: Add preferred boot HART field in struct fw_dynamic_info > > > > > > > > > > > > > > It fail randomly yesterday, but this morning I try several times it will pass. > > > > > > > I will keep trying. > > > > > > > > > > > > > > > > > > > I have figure out one fail case which is belong to main hart of U-Boot > > > > > > SPL is not the last hart while entering OpenSBI > > > > > > > > > > > > > > > > Can you try this branch [1]? It includes a quick implementation of the > > > > > changes a mentioned yesterday, where the main hart waits until all > > > > > harts have received the IPI. > > > > > > > > > > [1]: > > > > > https://github.com/lukasauer/u-boot/commits/riscv-opensbi-boot-hart > > > > > > > > I have try this patch, but it seems can not guarantee main hart to be > > > > the last hart while leaving U-Boot SPL. > > > > Even the main hart have checked all harts have received the IPI, but > > > > it still have opportunities to arrive OpenSBI before other harts. > > > > > > > > > > Thanks for testing! Can you try again with the same branch? I added > > > another patch so that clearing the IPI is the very last thing before > > > jumping to the SMP function. > > > If that does not help, we'll have to add a delay in > > > spl_invoke_opensbi() to delay the main hart. > > > > Thanks for your patch, it indeed solve the problem almost. > > It always pass the booting verification under free running, I try many > > times and can not hit the failure case. > > Thanks for testing the changes again, Rick. I will try to clean them up and submit them soon. > > But if I do something with GDB, E.g > > Set a break in 0x6c2 (c.jalr s2) in handle_ipi() and then delete the > > break and free run after it hit this break > > then it will hit the failure case as below: > > The current booting mechanism where all HARTs jump to each > booting stage (including Linux) at the same time is very fragile. > > Attaching a debugger at boot-time will certainly break current way > of booting all HARTs because we are holding a particular HART in > debug state. I am not sure if we can handle the debugger case in > software but in real-world deployments debugger won't be present > so maybe we can ignore this case. > > The SBI v0.2 HSM extension will make things better. It would be > even better if HW designers provide explicit HW mechanism to > power-on/off HARTs at run-time. > I agree with you, Anup. The debugger should not be attached in real- world scenarios. Even if it is, it is likely for debugging something in U-Boot proper or SPL and not the part where we are jumping to OpenSBI, so we should not typically run into any issues here. In addition, the recommended way of using SPL should be with separate text bases for U-Boot SPL and OpenSBI. In this setup we don't run into this problem. All in all, I think the current solution is robust enough. I will also make sure to add a comment on why it is important that the main hart enters OpenSBI last. This way the problem is documented and people looking at the code will be aware of it. Thanks, Lukas ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 10:44 ` Anup Patel 2019-11-07 11:41 ` Rick Chen @ 2019-11-07 11:44 ` Auer, Lukas 2019-11-07 12:27 ` Anup Patel 1 sibling, 1 reply; 71+ messages in thread From: Auer, Lukas @ 2019-11-07 11:44 UTC (permalink / raw) To: u-boot On Thu, 2019-11-07 at 16:14 +0530, Anup Patel wrote: > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > <lukas.auer@aisec.fraunhofer.de> wrote: > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Anup > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > Hi Anup > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > > > > > have the following change > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > change to > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > before this commit > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > > > > > value in a3 for these firmwares because these are not booted by U-Boot > > > > > > SPL. > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > > > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > > > > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the right thing > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for you ?? > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. > > > > > > > > Thanks for your corrections > > > > > > No problem, I am happy to help. > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > Maybe below changes can help you... > > > > Thanks for looking into this issue! I successfully tested it on QEMU, I > > had to add a short delay between sending the IPIs to trigger the > > problem. > > > > We might still run into problems however. Right now, we are assuming > > that the main hart is the last one to enter OpenSBI. If this is not the > > case (some delay when handling the IPI), we will have the same problem > > again. To fix this we could pass the hart mask, containing all harts > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > running in OpenSBI. I am not sure how realistic this scenario is, so > > this might not be needed. > > I agree that we might still run into this issue if primary HART enters > OpenSBI before secondary HARTs. I think this situation can only > happen on QEMU where each CPU is a thread running on host but > very unlikely/impossible on real HW. > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > secondary HARTs and before jumping to OpenSBI ? > You are right, I don't think we will ever see this on real hardware. I can add a check to ensure all harts have processed the IPI before jumping to OpenSBI on the main hart. A simple delay is probably already sufficient however. > Regarding hart_mask in fw_dynamic_info, I think the issue will be the > size of the hart_mask. It is possible in-future SOC vendors come-up > with SOC having huge number of HARTs OR SOC with discontinuous > HART IDs which can cause a 64bit hart_mask to be not sufficient for > all HARTs. > > Also, waiting for all HARTs to enter OpenSBI will be one more wait-loop > in fw_base.S which will add to the boot-time as well. > I agree, it's best to keep everything simple here. > I still think the root cause of the issue is that TEXT_START of > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > insist SOC vendors to not use same TEXT_START ? > That is definitely the best solution. I am not familiar with the details of the Andes platform, so unsure if that would work on it. On the SiFive Unleashed board this will be possible as soon as we have a working DRAM driver. U-Boot SPL could then run from L2 scratchpad memory instead of DRAM. Regards, Lukas ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 11:44 ` Auer, Lukas @ 2019-11-07 12:27 ` Anup Patel 2019-11-07 13:37 ` Auer, Lukas 0 siblings, 1 reply; 71+ messages in thread From: Anup Patel @ 2019-11-07 12:27 UTC (permalink / raw) To: u-boot On Thu, Nov 7, 2019 at 5:14 PM Auer, Lukas <lukas.auer@aisec.fraunhofer.de> wrote: > > On Thu, 2019-11-07 at 16:14 +0530, Anup Patel wrote: > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > Hi Anup > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > > > > > > have the following change > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > change to > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > before this commit > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > > > > > > value in a3 for these firmwares because these are not booted by U-Boot > > > > > > > SPL. > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > > > > > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the right thing > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for you ?? > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. > > > > > > > > > > Thanks for your corrections > > > > > > > > No problem, I am happy to help. > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > Maybe below changes can help you... > > > > > > Thanks for looking into this issue! I successfully tested it on QEMU, I > > > had to add a short delay between sending the IPIs to trigger the > > > problem. > > > > > > We might still run into problems however. Right now, we are assuming > > > that the main hart is the last one to enter OpenSBI. If this is not the > > > case (some delay when handling the IPI), we will have the same problem > > > again. To fix this we could pass the hart mask, containing all harts > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > running in OpenSBI. I am not sure how realistic this scenario is, so > > > this might not be needed. > > > > I agree that we might still run into this issue if primary HART enters > > OpenSBI before secondary HARTs. I think this situation can only > > happen on QEMU where each CPU is a thread running on host but > > very unlikely/impossible on real HW. > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > secondary HARTs and before jumping to OpenSBI ? > > > > You are right, I don't think we will ever see this on real hardware. I > can add a check to ensure all harts have processed the IPI before > jumping to OpenSBI on the main hart. A simple delay is probably already > sufficient however. Yes, please change spl_opensbi like you described. Maybe also add some comments about the issues we discussed here. > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be the > > size of the hart_mask. It is possible in-future SOC vendors come-up > > with SOC having huge number of HARTs OR SOC with discontinuous > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > all HARTs. > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait-loop > > in fw_base.S which will add to the boot-time as well. > > > > I agree, it's best to keep everything simple here. Cool, I guess we are in-sync. > > > I still think the root cause of the issue is that TEXT_START of > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > insist SOC vendors to not use same TEXT_START ? > > > > That is definitely the best solution. I am not familiar with the > details of the Andes platform, so unsure if that would work on it. On > the SiFive Unleashed board this will be possible as soon as we have a > working DRAM driver. U-Boot SPL could then run from L2 scratchpad > memory instead of DRAM. Yes, this won't be a problem on SiFive Unleashed. Regards, Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-11-07 12:27 ` Anup Patel @ 2019-11-07 13:37 ` Auer, Lukas 0 siblings, 0 replies; 71+ messages in thread From: Auer, Lukas @ 2019-11-07 13:37 UTC (permalink / raw) To: u-boot On Thu, 2019-11-07 at 17:57 +0530, Anup Patel wrote: > On Thu, Nov 7, 2019 at 5:14 PM Auer, Lukas > <lukas.auer@aisec.fraunhofer.de> wrote: > > On Thu, 2019-11-07 at 16:14 +0530, Anup Patel wrote: > > > On Thu, Nov 7, 2019 at 3:11 PM Auer, Lukas > > > <lukas.auer@aisec.fraunhofer.de> wrote: > > > > On Thu, 2019-11-07 at 11:48 +0530, Anup Patel wrote: > > > > > On Thu, Nov 7, 2019 at 11:40 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > Hi Anup > > > > > > > > > > > > > On Thu, Nov 7, 2019 at 10:45 AM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > On Thu, Nov 7, 2019 at 7:04 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:51 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 2:18 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > On Wed, Nov 6, 2019 at 12:14 PM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 5, 2019 at 7:19 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > Hi Anup > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 1:42 PM Anup Patel <anup@brainfault.org> wrote: > > > > > > > > > > > > > > > > > > > On Thu, Oct 31, 2019 at 6:30 AM Alan Kao <alankao@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > Hi Bin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the critics. Comments below. > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 06:38:00PM +0800, Bin Meng wrote: > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > Hi Bin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Rick, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will work fine due to hart 0 always will be main > > > > > > > > > > > > > > > > > > > > > > > > hart coincidentally. When develop SPL flow, I try to > > > > > > > > > > > > > > > > > > > > > > > > force other harts to be main hart. And it will go > > > > > > > > > > > > > > > > > > > > > > > > wrong in sending IPI flow. So fix it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, it include two fixs. But they will cause one negative result > > > > > > > > > > > > > > > > > > > > > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > > > > > > > > > > > > > > > > > > > > > theoretically, but it still fail somewhere. After dig in > > > > > > > > > > > > > > > > > > > > > > > > and found there is an assumption that hart 0 shall be > > > > > > > > > > > > > > > > > > > > > > > > main hart in OpenSbi. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So does this mean there is a bug in OpenSBI too? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > > > > > > > > > > > > > > > > > > > > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think OpenSBI has such limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please check the source. > > > > > > > > > > > > > > > > > > > > https://github.com/riscv/opensbi/blob/master/firmware/fw_base.S#L54 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Apparently, the FIRST TWO LINEs of the initialization are the > > > > > > > > > > > > > > > > > > > > 1. get hart ID. > > > > > > > > > > > > > > > > > > > > 2. determine which route to take based on their ID respectively. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > So, I do think OpenSBI has this signature, if you are not willing to call it > > > > > > > > > > > > > > > > > > > > a limitation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This dependency on hart id #0 was not there until we added self-relocation > > > > > > > > > > > > > > > > > > > in OpenSBI for FW_DYNAMIC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I will try to fix this in OpenSBI but we might end-up having boot_lottery. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have send a patch to fix this OpenSBI: > > > > > > > > > > > > > > > > > > "[PATCH] firmware: Introduce relocation lottery" > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you try above patch and see if that helps ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It will be great if you can provide Tested-by to my patch as well. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I can not find this patch in mailing list. > > > > > > > > > > > > > > > > Can you provide a hyperlink ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can try latest riscv/opensbi master. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tested the patch on SiFive Unleashed multiple times. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried this patch, but it fail > > > > > > > > > > > > > > firmware: Introduce relocation lottery( > > > > > > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe) > > > > > > > > > > > > > > > > > > > > > > > > > > > > The scenario was as below: > > > > > > > > > > > > > > There are 4 harts run in U-Boot SPL, hart 0 play as main hart. > > > > > > > > > > > > > > The hart 1 will receive ipi and come into OpenSBI(0x1000000) from > > > > > > > > > > > > > > U-Boot SPL(0x0), meanwhile hart 0,2,3 still run in U-Boot SPL. > > > > > > > > > > > > > > Then hart 1 will do _relocate_copy_to_lower which will copy data from > > > > > > > > > > > > > > 0x1000000 to 0x0. > > > > > > > > > > > > > > And it will corrupt U-Boot SPL. > > > > > > > > > > > > > > > > > > > > > > > > > > The self-relocation in OpenSBI firmwares ensures that OpenSBI firmware > > > > > > > > > > > > > are moved to the FW_TEXT_START before entering C code. This helps > > > > > > > > > > > > > us load OpenSBI firmwares anywhere in RAM. > > > > > > > > > > > > > > > > > > > > > > > > > > However, OpenSBI firmwares don't know where the U-Boot SPL is running. > > > > > > > > > > > > > > > > > > > > > > > > > > In your case, both OpenSBI FW_DYNAMIC and U-Boot SPL are linked to > > > > > > > > > > > > > address same address 0x0. This means secondary HARTs cannot safely > > > > > > > > > > > > > wait while primary HART enters OpenSBI. You should hold secondary HARTs > > > > > > > > > > > > > in U-Boot SPL only till OpenSBI FW_DYNAMIC and U-Boot proper are > > > > > > > > > > > > > loaded in RAM by primary HART. All your HARTs should jump to OpenSBI > > > > > > > > > > > > > at the same time after everything is loaded in RAM. > > > > > > > > > > > > > > > > > > > > > > > > I see the issue now. > > > > > > > > > > > > > > > > > > > > > > > > The U-Boot SPL is first letting secondary HART jump to OpenSBI and primary > > > > > > > > > > > > HART jumps to OpenSBI at the end. > > > > > > > > > > > > (Refer, jump_to_image_no_args() in arch/riscv/lib/spl.c) > > > > > > > > > > > > > > > > > > > > > > > > The real issue is FW_TEXT_START being same as U-Boot SPL TEXT_START. > > > > > > > > > > > > > > > > > > > > > > > > If possible please change TEXT base for U-Boot SPL or OpenSBI. I think > > > > > > > > > > > > changing U-Boot SPL TEXT_START would be convenient since this series > > > > > > > > > > > > is under review. Thoughts ? > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > I know it can avoid corrupting issue with changing U-Boot SPL > > > > > > > > > > > TEXT_START not equal to OpenSBI TEXT base. > > > > > > > > > > > > > > > > > > > > I think this issue will be seen on U-Boot SPL running on QEMU as well. > > > > > > > > > > > > > > > > > > > > > With the following changes, U-Boot SPL text base can equal to OpenSBI text base > > > > > > > > > > > 1 U-Boot pass main hart information (a2) when jumping to OpenSBI > > > > > > > > > > > 2 OpenSBI pick up $a2 to keep playing as main hart, other harts go to > > > > > > > > > > > _wait_relocate_copy_done > > > > > > > > > > > > > > > > > > > > Overall it's a good suggestion but we cannot use a2 register because this > > > > > > > > > > will break FW_JUMP and FW_PAYLOAD. Instead, we should pass preferred > > > > > > > > > > boot HART id in struct fw_dynamic_info. > > > > > > > > > > > > > > > > > > Sorry, what I want to say shall be a3. > > > > > > > > > > > > > > > > > > > I have a patch for this in preferred_boot_hart_v1 branch of > > > > > > > > > > https://github.com/avpatel/opensbi.git > > > > > > > > > > > > > > > > > > > > Can you try OpenSBI from above branch ? > > > > > > > > > > > > > > > > > > > > You will have to update the "struct fw_dynamic_info" passed to > > > > > > > > > > OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > > > Main hart will pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > But other harts will NOT pass struct "fw_dynamic_info" to OpenSBI by U-Boot SPL. > > > > > > > > > > > > > > > > That's wrong in U-Boot SPL. > > > > > > > > > > > > > > > > All HARTs have to follow FW_DYNAMIC protocol and pass > > > > > > > > "struct fw_dynamic_info" pointer in 'a2' register. > > > > > > > > > > > > > > > > > So if U-Boot SPL can pass main hart information via a3, OpenSBI just > > > > > > > > > have the following change > > > > > > > > > blt zero, a6, _wait_relocate_copy_done > > > > > > > > > change to > > > > > > > > > bne a3, a6, _wait_relocate_copy_done > > > > > > > > > before this commit > > > > > > > > > 98f4a208995b027662a7b04a25e4fa5df5f3eefe > > > > > > > > > firmware: Introduce relocation lottery > > > > > > > > > > > > > > > > What about FW_JUMP and FW_PAYLOAD? We have no way of passing > > > > > > > > value in a3 for these firmwares because these are not booted by U-Boot > > > > > > > > SPL. > > > > > > > > > > > > > > > > Also, U-Boot-2019.10 already uses U-Boot SPL support which does not > > > > > > > > pass anything in 'a3' register. > > > > > > > > > > > > > > > > We should definitely use "struct fw_dynamic_info" for this so that we can > > > > > > > > maintain backward compatibility as well. > > > > > > > > > > > > > > > > Please make sure that U-Boot SPL passes "struct fw_dynamic_info" > > > > > > > > pointer in 'a2' register for all HARTs. > > > > > > > > > > > > > > > > > But after this commit 98f4a, main hart become chosen from lottery mechanism. > > > > > > > > > Maybe I will prefer to change U-Boot SPL text base not overlap with > > > > > > > > > OpenSBI text start. :) > > > > > > > > > > > > > > > > Like I mentioned, we have this issue for U-Boot SPL on QEMU as well. It's > > > > > > > > just that most of us did not notice it for U-Boot SPL on QEMU. > > > > > > > > > > > > > > > > Let's fix this in the right way from start itself. > > > > > > > > > > > > > > I double checked spl_invoke_opensbi() and it is doing the right thing > > > > > > > by passing "struct fw_dyanmic_info" pointer in 'a2' register. > > > > > > > (Refer, common/spl/spl_opensbi.c) > > > > > > > > > > > > > > Not sure, why it is not passing 'a2' register correctly for you ?? > > > > > > > > > > > > > > > > > > > Yes, you are right. I reply too quickly. > > > > > > Other harts will pass struct fw_dyanmic_info in a2 to OpenSBI. > > > > > > > > > > > > Thanks for your corrections > > > > > > > > > > No problem, I am happy to help. > > > > > > > > > > BTW, I tried to play around with U-Boot SPL on QEMU. > > > > > > > > > > Maybe below changes can help you... > > > > > > > > Thanks for looking into this issue! I successfully tested it on QEMU, I > > > > had to add a short delay between sending the IPIs to trigger the > > > > problem. > > > > > > > > We might still run into problems however. Right now, we are assuming > > > > that the main hart is the last one to enter OpenSBI. If this is not the > > > > case (some delay when handling the IPI), we will have the same problem > > > > again. To fix this we could pass the hart mask, containing all harts > > > > that have entered U-Boot, to OpenSBI and wait for all harts to be > > > > running in OpenSBI. I am not sure how realistic this scenario is, so > > > > this might not be needed. > > > > > > I agree that we might still run into this issue if primary HART enters > > > OpenSBI before secondary HARTs. I think this situation can only > > > happen on QEMU where each CPU is a thread running on host but > > > very unlikely/impossible on real HW. > > > > > > Maybe a delay on primary HART in U-Boot SPL after SMP calls to > > > secondary HARTs and before jumping to OpenSBI ? > > > > > > > You are right, I don't think we will ever see this on real hardware. I > > can add a check to ensure all harts have processed the IPI before > > jumping to OpenSBI on the main hart. A simple delay is probably already > > sufficient however. > > Yes, please change spl_opensbi like you described. > > Maybe also add some comments about the issues we discussed here. Good idea, I will add the changes and describe the issue in a comment. Regards, Lukas > > > > Regarding hart_mask in fw_dynamic_info, I think the issue will be the > > > size of the hart_mask. It is possible in-future SOC vendors come-up > > > with SOC having huge number of HARTs OR SOC with discontinuous > > > HART IDs which can cause a 64bit hart_mask to be not sufficient for > > > all HARTs. > > > > > > Also, waiting for all HARTs to enter OpenSBI will be one more wait-loop > > > in fw_base.S which will add to the boot-time as well. > > > > > > > I agree, it's best to keep everything simple here. > > Cool, I guess we are in-sync. > > > > I still think the root cause of the issue is that TEXT_START of > > > U-Boot SPL and OpenSBI FW_DYNAMIC is same. Maybe we can > > > insist SOC vendors to not use same TEXT_START ? > > > > > > > That is definitely the best solution. I am not familiar with the > > details of the Andes platform, so unsure if that would work on it. On > > the SiFive Unleashed board this will be possible as soon as we have a > > working DRAM driver. U-Boot SPL could then run from L2 scratchpad > > memory instead of DRAM. > > Yes, this won't be a problem on SiFive Unleashed. > > Regards, > Anup ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations 2019-10-30 10:38 ` Bin Meng 2019-10-31 1:00 ` Alan Kao @ 2019-10-31 2:23 ` Rick Chen 1 sibling, 0 replies; 71+ messages in thread From: Rick Chen @ 2019-10-31 2:23 UTC (permalink / raw) To: u-boot Hi Bin > > Hi Rick, > > On Wed, Oct 30, 2019 at 10:50 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Bin > > > > > > > > Hi Rick, > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > It will work fine due to hart 0 always will be main > > > > hart coincidentally. When develop SPL flow, I try to > > > > force other harts to be main hart. And it will go > > > > wrong in sending IPI flow. So fix it. > > > > > > Fix what? Does this commit contain 2 fixes, or just 1 fix? > > > > Yes, it include two fixs. But they will cause one negative result > > that only hart 0 can send ipi to other harts. > > > > > > > > > > > > > Having this fix, any hart can be main hart in U-Boot SPL > > > > theoretically, but it still fail somewhere. After dig in > > > > and found there is an assumption that hart 0 shall be > > > > main hart in OpenSbi. > > > > > > So does this mean there is a bug in OpenSBI too? > > > > I am not sure if it is a bug. Maybe it is a compatible issue. > > There is a limitation that only hart 0 can be main hart in OpenSBI. > > I don't think OpenSBI has such limitation. OK. But there is a hint in OpenSBI indeed. Maybe it is just different interpretation each other. /* * Jump to warm-boot if this is not the first core booting, * that is, for mhartid != 0 */ > > > But any hart can be main hart in U-Boot. > > > > In general case, hart 0 will be main and it is fine when U-Boot jump ot OpenSBI. > > But if we force hart 1 to be main hart, when hart 0 jump to OPenSBI from U-Boot, > > It will do relocation flow in OpenSBI which willcorrupt U-Boot SPL, > > but hart 0 still in U-Boot SPL. > > > > > > > > > > > > > After some work-arounds, it can pass the verifications > > > > that any hart can be main hart and boots U-Boot SPL -> > > > > OpenSbi -> U-Boot proper -> Linux Kernel successfully. > > > > > > > > > > It's a bit hard for me to understand what was described here in the > > > commit message. Maybe you need rewrite something. > > > > OK. I will rewrite this commit message. > > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > Cc: KC Lin <kclin@andestech.com> > > > > Cc: Alan Kao <alankao@andestech.com> > > > > --- > > > > arch/riscv/lib/andes_plic.c | 11 +++++++---- > > > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > > > > > diff --git a/arch/riscv/lib/andes_plic.c b/arch/riscv/lib/andes_plic.c > > > > index 28568e4..42394b9 100644 > > > > --- a/arch/riscv/lib/andes_plic.c > > > > +++ b/arch/riscv/lib/andes_plic.c > > > > @@ -19,7 +19,7 @@ > > > > #include <cpu.h> > > > > > > > > /* pending register */ > > > > -#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + (hart) * 8) > > > > +#define PENDING_REG(base, hart) ((ulong)(base) + 0x1000 + ((hart) / 4) * 4) > > > > /* enable register */ > > > > #define ENABLE_REG(base, hart) ((ulong)(base) + 0x2000 + (hart) * 0x80) > > > > /* claim register */ > > > > @@ -46,7 +46,7 @@ static int init_plic(void); > > > > > > > > static int enable_ipi(int hart) > > > > { > > > > - int en; > > > > + unsigned int en; > > > > > > Is this for some compiler warning fix? > > > > No, it is not a warning fix. It is a bug fix. > > I hope en can be 0x0000000080808080 instead of 0xffffffff80808080, > > But it is int, which is only 32-bit. The example you gave was a 64-bit number. The verification runs on RV64, so it is a 64-bit number example. Thanks Rick > > > or it will cause IPI sending errors. > > > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 5/8] riscv: ax25: cache: Add SPL_RISCV_MMODE for SPL 2019-10-25 6:10 [U-Boot] [PATCH 0/8] RISC-V AX25-AE350 support SPL Andes ` (3 preceding siblings ...) 2019-10-25 6:10 ` [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations Andes @ 2019-10-25 6:10 ` Andes 2019-10-29 14:59 ` Bin Meng 2019-10-25 6:10 ` [U-Boot] [PATCH 6/8] spl: cache: Allow cache drivers in SPL Andes ` (2 subsequent siblings) 7 siblings, 1 reply; 71+ messages in thread From: Andes @ 2019-10-25 6:10 UTC (permalink / raw) To: u-boot From: Rick Chen <rick@andestech.com> The mcache_ctl csr only can be manipulated in M mode. Add SPL_RISCV_MMODE for U-Boot SPL to control cache operation. Signed-off-by: Rick Chen <rick@andestech.com> Cc: KC Lin <kclin@andestech.com> Cc: Alan Kao <alankao@andestech.com> --- arch/riscv/cpu/ax25/cache.c | 60 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 46 insertions(+), 14 deletions(-) diff --git a/arch/riscv/cpu/ax25/cache.c b/arch/riscv/cpu/ax25/cache.c index 41de30c..9437e81 100644 --- a/arch/riscv/cpu/ax25/cache.c +++ b/arch/riscv/cpu/ax25/cache.c @@ -11,18 +11,46 @@ #include <asm/csr.h> #ifdef CONFIG_RISCV_NDS_CACHE +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) /* mcctlcommand */ #define CCTL_REG_MCCTLCOMMAND_NUM 0x7cc /* D-cache operation */ #define CCTL_L1D_WBINVAL_ALL 6 #endif +#endif + +#ifdef CONFIG_V5L2_CACHE +static void _cache_enable(void) +{ + struct udevice *dev = NULL; + + uclass_find_first_device(UCLASS_CACHE, &dev); + + if (dev) + cache_enable(dev); +} + +static void _cache_disable(void) +{ + struct udevice *dev = NULL; + + uclass_find_first_device(UCLASS_CACHE, &dev); + + if (dev) + cache_disable(dev); +} +#endif void flush_dcache_all(void) { +#if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) #ifdef CONFIG_RISCV_NDS_CACHE +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) csr_write(CCTL_REG_MCCTLCOMMAND_NUM, CCTL_L1D_WBINVAL_ALL); #endif +#endif +#endif } void flush_dcache_range(unsigned long start, unsigned long end) @@ -39,6 +67,7 @@ void icache_enable(void) { #if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) #ifdef CONFIG_RISCV_NDS_CACHE +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) asm volatile ( "csrr t1, mcache_ctl\n\t" "ori t0, t1, 0x1\n\t" @@ -46,12 +75,14 @@ void icache_enable(void) ); #endif #endif +#endif } void icache_disable(void) { #if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) #ifdef CONFIG_RISCV_NDS_CACHE +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) asm volatile ( "fence.i\n\t" "csrr t1, mcache_ctl\n\t" @@ -60,24 +91,23 @@ void icache_disable(void) ); #endif #endif +#endif } void dcache_enable(void) { #if !CONFIG_IS_ENABLED(SYS_DCACHE_OFF) #ifdef CONFIG_RISCV_NDS_CACHE - struct udevice *dev = NULL; - +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) asm volatile ( "csrr t1, mcache_ctl\n\t" "ori t0, t1, 0x2\n\t" "csrw mcache_ctl, t0\n\t" ); - - uclass_find_first_device(UCLASS_CACHE, &dev); - - if (dev) - cache_enable(dev); +#endif +#ifdef CONFIG_V5L2_CACHE + _cache_enable(); +#endif #endif #endif } @@ -86,19 +116,17 @@ void dcache_disable(void) { #if !CONFIG_IS_ENABLED(SYS_DCACHE_OFF) #ifdef CONFIG_RISCV_NDS_CACHE - struct udevice *dev = NULL; - +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) csr_write(CCTL_REG_MCCTLCOMMAND_NUM, CCTL_L1D_WBINVAL_ALL); asm volatile ( "csrr t1, mcache_ctl\n\t" "andi t0, t1, ~0x2\n\t" "csrw mcache_ctl, t0\n\t" ); - - uclass_find_first_device(UCLASS_CACHE, &dev); - - if (dev) - cache_disable(dev); +#endif +#ifdef CONFIG_V5L2_CACHE + _cache_disable(); +#endif #endif #endif } @@ -108,6 +136,7 @@ int icache_status(void) int ret = 0; #ifdef CONFIG_RISCV_NDS_CACHE +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) asm volatile ( "csrr t1, mcache_ctl\n\t" "andi %0, t1, 0x01\n\t" @@ -116,6 +145,7 @@ int icache_status(void) : "memory" ); #endif +#endif return ret; } @@ -125,6 +155,7 @@ int dcache_status(void) int ret = 0; #ifdef CONFIG_RISCV_NDS_CACHE +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) asm volatile ( "csrr t1, mcache_ctl\n\t" "andi %0, t1, 0x02\n\t" @@ -133,6 +164,7 @@ int dcache_status(void) : "memory" ); #endif +#endif return ret; } -- 2.7.4 ^ permalink raw reply related [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 5/8] riscv: ax25: cache: Add SPL_RISCV_MMODE for SPL 2019-10-25 6:10 ` [U-Boot] [PATCH 5/8] riscv: ax25: cache: Add SPL_RISCV_MMODE for SPL Andes @ 2019-10-29 14:59 ` Bin Meng 2019-10-31 2:31 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-29 14:59 UTC (permalink / raw) To: u-boot Hi Rick, On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > From: Rick Chen <rick@andestech.com> > > The mcache_ctl csr only can be manipulated in M mode. > Add SPL_RISCV_MMODE for U-Boot SPL to control cache > operation. > > Signed-off-by: Rick Chen <rick@andestech.com> > Cc: KC Lin <kclin@andestech.com> > Cc: Alan Kao <alankao@andestech.com> > --- > arch/riscv/cpu/ax25/cache.c | 60 ++++++++++++++++++++++++++++++++++----------- > 1 file changed, 46 insertions(+), 14 deletions(-) > > diff --git a/arch/riscv/cpu/ax25/cache.c b/arch/riscv/cpu/ax25/cache.c > index 41de30c..9437e81 100644 > --- a/arch/riscv/cpu/ax25/cache.c > +++ b/arch/riscv/cpu/ax25/cache.c > @@ -11,18 +11,46 @@ > #include <asm/csr.h> > > #ifdef CONFIG_RISCV_NDS_CACHE > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) Use CONFIG_IS_ENABLED(RISCV_MMODE) > /* mcctlcommand */ > #define CCTL_REG_MCCTLCOMMAND_NUM 0x7cc > > /* D-cache operation */ > #define CCTL_L1D_WBINVAL_ALL 6 > #endif > +#endif > + > +#ifdef CONFIG_V5L2_CACHE > +static void _cache_enable(void) > +{ > + struct udevice *dev = NULL; > + > + uclass_find_first_device(UCLASS_CACHE, &dev); > + > + if (dev) > + cache_enable(dev); > +} > + > +static void _cache_disable(void) > +{ > + struct udevice *dev = NULL; > + > + uclass_find_first_device(UCLASS_CACHE, &dev); > + > + if (dev) > + cache_disable(dev); > +} > +#endif > > void flush_dcache_all(void) > { > +#if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) > #ifdef CONFIG_RISCV_NDS_CACHE > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > csr_write(CCTL_REG_MCCTLCOMMAND_NUM, CCTL_L1D_WBINVAL_ALL); > #endif > +#endif > +#endif > } > > void flush_dcache_range(unsigned long start, unsigned long end) > @@ -39,6 +67,7 @@ void icache_enable(void) > { > #if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) > #ifdef CONFIG_RISCV_NDS_CACHE > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > asm volatile ( > "csrr t1, mcache_ctl\n\t" > "ori t0, t1, 0x1\n\t" > @@ -46,12 +75,14 @@ void icache_enable(void) > ); > #endif > #endif > +#endif > } > > void icache_disable(void) > { > #if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) > #ifdef CONFIG_RISCV_NDS_CACHE > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > asm volatile ( > "fence.i\n\t" > "csrr t1, mcache_ctl\n\t" Could you please add a patch to replace the CSR name with CSR number? This way an upstream GCC to build this code without any issue. > @@ -60,24 +91,23 @@ void icache_disable(void) > ); > #endif > #endif > +#endif > } > > void dcache_enable(void) > { > #if !CONFIG_IS_ENABLED(SYS_DCACHE_OFF) > #ifdef CONFIG_RISCV_NDS_CACHE > - struct udevice *dev = NULL; > - > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > asm volatile ( > "csrr t1, mcache_ctl\n\t" > "ori t0, t1, 0x2\n\t" > "csrw mcache_ctl, t0\n\t" > ); > - > - uclass_find_first_device(UCLASS_CACHE, &dev); > - > - if (dev) > - cache_enable(dev); > +#endif > +#ifdef CONFIG_V5L2_CACHE > + _cache_enable(); > +#endif > #endif > #endif > } > @@ -86,19 +116,17 @@ void dcache_disable(void) > { > #if !CONFIG_IS_ENABLED(SYS_DCACHE_OFF) > #ifdef CONFIG_RISCV_NDS_CACHE > - struct udevice *dev = NULL; > - > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > csr_write(CCTL_REG_MCCTLCOMMAND_NUM, CCTL_L1D_WBINVAL_ALL); > asm volatile ( > "csrr t1, mcache_ctl\n\t" > "andi t0, t1, ~0x2\n\t" > "csrw mcache_ctl, t0\n\t" > ); > - > - uclass_find_first_device(UCLASS_CACHE, &dev); > - > - if (dev) > - cache_disable(dev); > +#endif > +#ifdef CONFIG_V5L2_CACHE > + _cache_disable(); > +#endif > #endif > #endif > } > @@ -108,6 +136,7 @@ int icache_status(void) > int ret = 0; > > #ifdef CONFIG_RISCV_NDS_CACHE > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > asm volatile ( > "csrr t1, mcache_ctl\n\t" > "andi %0, t1, 0x01\n\t" > @@ -116,6 +145,7 @@ int icache_status(void) > : "memory" > ); > #endif > +#endif > > return ret; > } > @@ -125,6 +155,7 @@ int dcache_status(void) > int ret = 0; > > #ifdef CONFIG_RISCV_NDS_CACHE > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > asm volatile ( > "csrr t1, mcache_ctl\n\t" > "andi %0, t1, 0x02\n\t" > @@ -133,6 +164,7 @@ int dcache_status(void) > : "memory" > ); > #endif > +#endif > > return ret; > } > -- Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 5/8] riscv: ax25: cache: Add SPL_RISCV_MMODE for SPL 2019-10-29 14:59 ` Bin Meng @ 2019-10-31 2:31 ` Rick Chen 2019-10-31 3:00 ` Bin Meng 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-10-31 2:31 UTC (permalink / raw) To: u-boot Hi Bin > > Hi Rick, > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > From: Rick Chen <rick@andestech.com> > > > > The mcache_ctl csr only can be manipulated in M mode. > > Add SPL_RISCV_MMODE for U-Boot SPL to control cache > > operation. > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > Cc: KC Lin <kclin@andestech.com> > > Cc: Alan Kao <alankao@andestech.com> > > --- > > arch/riscv/cpu/ax25/cache.c | 60 ++++++++++++++++++++++++++++++++++----------- > > 1 file changed, 46 insertions(+), 14 deletions(-) > > > > diff --git a/arch/riscv/cpu/ax25/cache.c b/arch/riscv/cpu/ax25/cache.c > > index 41de30c..9437e81 100644 > > --- a/arch/riscv/cpu/ax25/cache.c > > +++ b/arch/riscv/cpu/ax25/cache.c > > @@ -11,18 +11,46 @@ > > #include <asm/csr.h> > > > > #ifdef CONFIG_RISCV_NDS_CACHE > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > Use CONFIG_IS_ENABLED(RISCV_MMODE) > > > /* mcctlcommand */ > > #define CCTL_REG_MCCTLCOMMAND_NUM 0x7cc > > > > /* D-cache operation */ > > #define CCTL_L1D_WBINVAL_ALL 6 > > #endif > > +#endif > > + > > +#ifdef CONFIG_V5L2_CACHE > > +static void _cache_enable(void) > > +{ > > + struct udevice *dev = NULL; > > + > > + uclass_find_first_device(UCLASS_CACHE, &dev); > > + > > + if (dev) > > + cache_enable(dev); > > +} > > + > > +static void _cache_disable(void) > > +{ > > + struct udevice *dev = NULL; > > + > > + uclass_find_first_device(UCLASS_CACHE, &dev); > > + > > + if (dev) > > + cache_disable(dev); > > +} > > +#endif > > > > void flush_dcache_all(void) > > { > > +#if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) > > #ifdef CONFIG_RISCV_NDS_CACHE > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > csr_write(CCTL_REG_MCCTLCOMMAND_NUM, CCTL_L1D_WBINVAL_ALL); > > #endif > > +#endif > > +#endif > > } > > > > void flush_dcache_range(unsigned long start, unsigned long end) > > @@ -39,6 +67,7 @@ void icache_enable(void) > > { > > #if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) > > #ifdef CONFIG_RISCV_NDS_CACHE > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > asm volatile ( > > "csrr t1, mcache_ctl\n\t" > > "ori t0, t1, 0x1\n\t" > > @@ -46,12 +75,14 @@ void icache_enable(void) > > ); > > #endif > > #endif > > +#endif > > } > > > > void icache_disable(void) > > { > > #if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) > > #ifdef CONFIG_RISCV_NDS_CACHE > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > asm volatile ( > > "fence.i\n\t" > > "csrr t1, mcache_ctl\n\t" > > Could you please add a patch to replace the CSR name with CSR number? > This way an upstream GCC to build this code without any issue. Thanks for your suggestions. But I prefer not modify it in this patchs for SPL. I will send another patch to modify as your suggestions. Thanks Rick > > > @@ -60,24 +91,23 @@ void icache_disable(void) > > ); > > #endif > > #endif > > +#endif > > } > > > > void dcache_enable(void) > > { > > #if !CONFIG_IS_ENABLED(SYS_DCACHE_OFF) > > #ifdef CONFIG_RISCV_NDS_CACHE > > - struct udevice *dev = NULL; > > - > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > asm volatile ( > > "csrr t1, mcache_ctl\n\t" > > "ori t0, t1, 0x2\n\t" > > "csrw mcache_ctl, t0\n\t" > > ); > > - > > - uclass_find_first_device(UCLASS_CACHE, &dev); > > - > > - if (dev) > > - cache_enable(dev); > > +#endif > > +#ifdef CONFIG_V5L2_CACHE > > + _cache_enable(); > > +#endif > > #endif > > #endif > > } > > @@ -86,19 +116,17 @@ void dcache_disable(void) > > { > > #if !CONFIG_IS_ENABLED(SYS_DCACHE_OFF) > > #ifdef CONFIG_RISCV_NDS_CACHE > > - struct udevice *dev = NULL; > > - > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > csr_write(CCTL_REG_MCCTLCOMMAND_NUM, CCTL_L1D_WBINVAL_ALL); > > asm volatile ( > > "csrr t1, mcache_ctl\n\t" > > "andi t0, t1, ~0x2\n\t" > > "csrw mcache_ctl, t0\n\t" > > ); > > - > > - uclass_find_first_device(UCLASS_CACHE, &dev); > > - > > - if (dev) > > - cache_disable(dev); > > +#endif > > +#ifdef CONFIG_V5L2_CACHE > > + _cache_disable(); > > +#endif > > #endif > > #endif > > } > > @@ -108,6 +136,7 @@ int icache_status(void) > > int ret = 0; > > > > #ifdef CONFIG_RISCV_NDS_CACHE > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > asm volatile ( > > "csrr t1, mcache_ctl\n\t" > > "andi %0, t1, 0x01\n\t" > > @@ -116,6 +145,7 @@ int icache_status(void) > > : "memory" > > ); > > #endif > > +#endif > > > > return ret; > > } > > @@ -125,6 +155,7 @@ int dcache_status(void) > > int ret = 0; > > > > #ifdef CONFIG_RISCV_NDS_CACHE > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > asm volatile ( > > "csrr t1, mcache_ctl\n\t" > > "andi %0, t1, 0x02\n\t" > > @@ -133,6 +164,7 @@ int dcache_status(void) > > : "memory" > > ); > > #endif > > +#endif > > > > return ret; > > } > > -- > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 5/8] riscv: ax25: cache: Add SPL_RISCV_MMODE for SPL 2019-10-31 2:31 ` Rick Chen @ 2019-10-31 3:00 ` Bin Meng 0 siblings, 0 replies; 71+ messages in thread From: Bin Meng @ 2019-10-31 3:00 UTC (permalink / raw) To: u-boot Hi Rick, On Thu, Oct 31, 2019 at 10:31 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Bin > > > > > Hi Rick, > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > The mcache_ctl csr only can be manipulated in M mode. > > > Add SPL_RISCV_MMODE for U-Boot SPL to control cache > > > operation. > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > Cc: KC Lin <kclin@andestech.com> > > > Cc: Alan Kao <alankao@andestech.com> > > > --- > > > arch/riscv/cpu/ax25/cache.c | 60 ++++++++++++++++++++++++++++++++++----------- > > > 1 file changed, 46 insertions(+), 14 deletions(-) > > > > > > diff --git a/arch/riscv/cpu/ax25/cache.c b/arch/riscv/cpu/ax25/cache.c > > > index 41de30c..9437e81 100644 > > > --- a/arch/riscv/cpu/ax25/cache.c > > > +++ b/arch/riscv/cpu/ax25/cache.c > > > @@ -11,18 +11,46 @@ > > > #include <asm/csr.h> > > > > > > #ifdef CONFIG_RISCV_NDS_CACHE > > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > > > Use CONFIG_IS_ENABLED(RISCV_MMODE) > > > > > /* mcctlcommand */ > > > #define CCTL_REG_MCCTLCOMMAND_NUM 0x7cc > > > > > > /* D-cache operation */ > > > #define CCTL_L1D_WBINVAL_ALL 6 > > > #endif > > > +#endif > > > + > > > +#ifdef CONFIG_V5L2_CACHE > > > +static void _cache_enable(void) > > > +{ > > > + struct udevice *dev = NULL; > > > + > > > + uclass_find_first_device(UCLASS_CACHE, &dev); > > > + > > > + if (dev) > > > + cache_enable(dev); > > > +} > > > + > > > +static void _cache_disable(void) > > > +{ > > > + struct udevice *dev = NULL; > > > + > > > + uclass_find_first_device(UCLASS_CACHE, &dev); > > > + > > > + if (dev) > > > + cache_disable(dev); > > > +} > > > +#endif > > > > > > void flush_dcache_all(void) > > > { > > > +#if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) > > > #ifdef CONFIG_RISCV_NDS_CACHE > > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > > csr_write(CCTL_REG_MCCTLCOMMAND_NUM, CCTL_L1D_WBINVAL_ALL); > > > #endif > > > +#endif > > > +#endif > > > } > > > > > > void flush_dcache_range(unsigned long start, unsigned long end) > > > @@ -39,6 +67,7 @@ void icache_enable(void) > > > { > > > #if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) > > > #ifdef CONFIG_RISCV_NDS_CACHE > > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > > asm volatile ( > > > "csrr t1, mcache_ctl\n\t" > > > "ori t0, t1, 0x1\n\t" > > > @@ -46,12 +75,14 @@ void icache_enable(void) > > > ); > > > #endif > > > #endif > > > +#endif > > > } > > > > > > void icache_disable(void) > > > { > > > #if !CONFIG_IS_ENABLED(SYS_ICACHE_OFF) > > > #ifdef CONFIG_RISCV_NDS_CACHE > > > +#if defined(CONFIG_RISCV_MMODE) || defined(CONFIG_SPL_RISCV_MMODE) > > > asm volatile ( > > > "fence.i\n\t" > > > "csrr t1, mcache_ctl\n\t" > > > > Could you please add a patch to replace the CSR name with CSR number? > > This way an upstream GCC to build this code without any issue. > > Thanks for your suggestions. > But I prefer not modify it in this patchs for SPL. > Yes, I was saying an additional patch. Definitely not this patch. > I will send another patch to modify as your suggestions. Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 6/8] spl: cache: Allow cache drivers in SPL 2019-10-25 6:10 [U-Boot] [PATCH 0/8] RISC-V AX25-AE350 support SPL Andes ` (4 preceding siblings ...) 2019-10-25 6:10 ` [U-Boot] [PATCH 5/8] riscv: ax25: cache: Add SPL_RISCV_MMODE for SPL Andes @ 2019-10-25 6:10 ` Andes 2019-10-29 15:14 ` Bin Meng 2019-10-25 6:10 ` [U-Boot] [PATCH 7/8] riscv: Fix clear bss loop in the start-up code Andes 2019-10-25 6:10 ` [U-Boot] [PATCH 8/8] riscv: dts: Support four cores SMP Andes 7 siblings, 1 reply; 71+ messages in thread From: Andes @ 2019-10-25 6:10 UTC (permalink / raw) To: u-boot From: Rick Chen <rick@andestech.com> When ax25-ae350 try to enable v5l2 cache driver in SPL configuration, it need this option for cache support in SPL. Signed-off-by: Rick Chen <rick@andestech.com> Cc: KC Lin <kclin@andestech.com> Cc: Alan Kao <alankao@andestech.com> --- common/spl/Kconfig | 7 +++++++ drivers/Makefile | 1 + 2 files changed, 8 insertions(+) diff --git a/common/spl/Kconfig b/common/spl/Kconfig index 86d7edf..4c4023a 100644 --- a/common/spl/Kconfig +++ b/common/spl/Kconfig @@ -456,6 +456,13 @@ config SPL_CRYPTO_SUPPORT this option to build the drivers in drivers/crypto as part of an SPL build. +config SPL_CACHE_SUPPORT + bool "Support CACHE drivers" + help + Enable CACHE drivers in SPL. These drivers can store data so that + future requests for that data can be served faster. Enable this option + to build the drivers in drivers/cache as part of an SPL build. + config SPL_HASH_SUPPORT bool "Support hashing drivers" select SHA1 diff --git a/drivers/Makefile b/drivers/Makefile index a4bb5e4..5d300df 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -33,6 +33,7 @@ ifdef CONFIG_SPL_BUILD obj-$(CONFIG_SPL_BOOTCOUNT_LIMIT) += bootcount/ obj-$(CONFIG_SPL_CPU_SUPPORT) += cpu/ obj-$(CONFIG_SPL_CRYPTO_SUPPORT) += crypto/ +obj-$(CONFIG_SPL_CACHE_SUPPORT) += cache/ obj-$(CONFIG_SPL_MPC8XXX_INIT_DDR_SUPPORT) += ddr/fsl/ obj-$(CONFIG_ARMADA_38X) += ddr/marvell/a38x/ obj-$(CONFIG_ARMADA_XP) += ddr/marvell/axp/ -- 2.7.4 ^ permalink raw reply related [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 6/8] spl: cache: Allow cache drivers in SPL 2019-10-25 6:10 ` [U-Boot] [PATCH 6/8] spl: cache: Allow cache drivers in SPL Andes @ 2019-10-29 15:14 ` Bin Meng 2019-10-31 2:52 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-29 15:14 UTC (permalink / raw) To: u-boot Hi Rick, On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > From: Rick Chen <rick@andestech.com> > > When ax25-ae350 try to enable v5l2 cache > driver in SPL configuration, it need this > option for cache support in SPL. > > Signed-off-by: Rick Chen <rick@andestech.com> > Cc: KC Lin <kclin@andestech.com> > Cc: Alan Kao <alankao@andestech.com> > --- > common/spl/Kconfig | 7 +++++++ > drivers/Makefile | 1 + > 2 files changed, 8 insertions(+) > > diff --git a/common/spl/Kconfig b/common/spl/Kconfig > index 86d7edf..4c4023a 100644 > --- a/common/spl/Kconfig > +++ b/common/spl/Kconfig > @@ -456,6 +456,13 @@ config SPL_CRYPTO_SUPPORT > this option to build the drivers in drivers/crypto as part of an > SPL build. > > +config SPL_CACHE_SUPPORT nits: please insert this option per alphabetical order, so it should come before SPL_CPU > + bool "Support CACHE drivers" > + help > + Enable CACHE drivers in SPL. These drivers can store data so that > + future requests for that data can be served faster. Enable this option The description here "store data so that ..." does not apply to cache drivers > + to build the drivers in drivers/cache as part of an SPL build. > + > config SPL_HASH_SUPPORT > bool "Support hashing drivers" > select SHA1 > diff --git a/drivers/Makefile b/drivers/Makefile > index a4bb5e4..5d300df 100644 > --- a/drivers/Makefile > +++ b/drivers/Makefile > @@ -33,6 +33,7 @@ ifdef CONFIG_SPL_BUILD > obj-$(CONFIG_SPL_BOOTCOUNT_LIMIT) += bootcount/ > obj-$(CONFIG_SPL_CPU_SUPPORT) += cpu/ > obj-$(CONFIG_SPL_CRYPTO_SUPPORT) += crypto/ > +obj-$(CONFIG_SPL_CACHE_SUPPORT) += cache/ ditto > obj-$(CONFIG_SPL_MPC8XXX_INIT_DDR_SUPPORT) += ddr/fsl/ > obj-$(CONFIG_ARMADA_38X) += ddr/marvell/a38x/ > obj-$(CONFIG_ARMADA_XP) += ddr/marvell/axp/ > -- Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 6/8] spl: cache: Allow cache drivers in SPL 2019-10-29 15:14 ` Bin Meng @ 2019-10-31 2:52 ` Rick Chen 2019-10-31 3:01 ` Bin Meng 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-10-31 2:52 UTC (permalink / raw) To: u-boot Hi Bin > > Hi Rick, > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > From: Rick Chen <rick@andestech.com> > > > > When ax25-ae350 try to enable v5l2 cache > > driver in SPL configuration, it need this > > option for cache support in SPL. > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > Cc: KC Lin <kclin@andestech.com> > > Cc: Alan Kao <alankao@andestech.com> > > --- > > common/spl/Kconfig | 7 +++++++ > > drivers/Makefile | 1 + > > 2 files changed, 8 insertions(+) > > > > diff --git a/common/spl/Kconfig b/common/spl/Kconfig > > index 86d7edf..4c4023a 100644 > > --- a/common/spl/Kconfig > > +++ b/common/spl/Kconfig > > @@ -456,6 +456,13 @@ config SPL_CRYPTO_SUPPORT > > this option to build the drivers in drivers/crypto as part of an > > SPL build. > > > > +config SPL_CACHE_SUPPORT > > nits: please insert this option per alphabetical order, so it should > come before SPL_CPU OK > > > + bool "Support CACHE drivers" > > + help > > + Enable CACHE drivers in SPL. These drivers can store data so that > > + future requests for that data can be served faster. Enable this option > > The description here "store data so that ..." does not apply to cache drivers Do you have any suggestions about how to describe it more precisely ? Thanks Rick > > > + to build the drivers in drivers/cache as part of an SPL build. > > + > > config SPL_HASH_SUPPORT > > bool "Support hashing drivers" > > select SHA1 > > diff --git a/drivers/Makefile b/drivers/Makefile > > index a4bb5e4..5d300df 100644 > > --- a/drivers/Makefile > > +++ b/drivers/Makefile > > @@ -33,6 +33,7 @@ ifdef CONFIG_SPL_BUILD > > obj-$(CONFIG_SPL_BOOTCOUNT_LIMIT) += bootcount/ > > obj-$(CONFIG_SPL_CPU_SUPPORT) += cpu/ > > obj-$(CONFIG_SPL_CRYPTO_SUPPORT) += crypto/ > > +obj-$(CONFIG_SPL_CACHE_SUPPORT) += cache/ > > ditto OK > > > obj-$(CONFIG_SPL_MPC8XXX_INIT_DDR_SUPPORT) += ddr/fsl/ > > obj-$(CONFIG_ARMADA_38X) += ddr/marvell/a38x/ > > obj-$(CONFIG_ARMADA_XP) += ddr/marvell/axp/ > > -- > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 6/8] spl: cache: Allow cache drivers in SPL 2019-10-31 2:52 ` Rick Chen @ 2019-10-31 3:01 ` Bin Meng 2019-10-31 3:22 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-31 3:01 UTC (permalink / raw) To: u-boot Hi Rick, On Thu, Oct 31, 2019 at 10:53 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Bin > > > > > Hi Rick, > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > When ax25-ae350 try to enable v5l2 cache > > > driver in SPL configuration, it need this > > > option for cache support in SPL. > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > Cc: KC Lin <kclin@andestech.com> > > > Cc: Alan Kao <alankao@andestech.com> > > > --- > > > common/spl/Kconfig | 7 +++++++ > > > drivers/Makefile | 1 + > > > 2 files changed, 8 insertions(+) > > > > > > diff --git a/common/spl/Kconfig b/common/spl/Kconfig > > > index 86d7edf..4c4023a 100644 > > > --- a/common/spl/Kconfig > > > +++ b/common/spl/Kconfig > > > @@ -456,6 +456,13 @@ config SPL_CRYPTO_SUPPORT > > > this option to build the drivers in drivers/crypto as part of an > > > SPL build. > > > > > > +config SPL_CACHE_SUPPORT > > > > nits: please insert this option per alphabetical order, so it should > > come before SPL_CPU > > OK > > > > > > + bool "Support CACHE drivers" > > > + help > > > + Enable CACHE drivers in SPL. These drivers can store data so that > > > + future requests for that data can be served faster. Enable this option > > > > The description here "store data so that ..." does not apply to cache drivers > > Do you have any suggestions about how to describe it more precisely ? > The descriptions says drivers can store data, but cache driver does not store data at all. I think you enabled it just for boot time performance? Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 6/8] spl: cache: Allow cache drivers in SPL 2019-10-31 3:01 ` Bin Meng @ 2019-10-31 3:22 ` Rick Chen 0 siblings, 0 replies; 71+ messages in thread From: Rick Chen @ 2019-10-31 3:22 UTC (permalink / raw) To: u-boot Hi Bin Bin Meng <bmeng.cn@gmail.com> 於 2019年10月31日 週四 上午11:02寫道: > > Hi Rick, > > On Thu, Oct 31, 2019 at 10:53 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Bin > > > > > > > > Hi Rick, > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > When ax25-ae350 try to enable v5l2 cache > > > > driver in SPL configuration, it need this > > > > option for cache support in SPL. > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > Cc: KC Lin <kclin@andestech.com> > > > > Cc: Alan Kao <alankao@andestech.com> > > > > --- > > > > common/spl/Kconfig | 7 +++++++ > > > > drivers/Makefile | 1 + > > > > 2 files changed, 8 insertions(+) > > > > > > > > diff --git a/common/spl/Kconfig b/common/spl/Kconfig > > > > index 86d7edf..4c4023a 100644 > > > > --- a/common/spl/Kconfig > > > > +++ b/common/spl/Kconfig > > > > @@ -456,6 +456,13 @@ config SPL_CRYPTO_SUPPORT > > > > this option to build the drivers in drivers/crypto as part of an > > > > SPL build. > > > > > > > > +config SPL_CACHE_SUPPORT > > > > > > nits: please insert this option per alphabetical order, so it should > > > come before SPL_CPU > > > > OK > > > > > > > > > + bool "Support CACHE drivers" > > > > + help > > > > + Enable CACHE drivers in SPL. These drivers can store data so that > > > > + future requests for that data can be served faster. Enable this option > > > > > > The description here "store data so that ..." does not apply to cache drivers > > > > Do you have any suggestions about how to describe it more precisely ? > > > > The descriptions says drivers can store data, but cache driver does > not store data at all. I think you enabled it just for boot time > performance? Yes, so the description mention about "served faster", it can represent improve boot time performance widely. Maybe you think store data is that kind of store data in flash. Cache will store data in it, so CPU can execute more faster. > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 7/8] riscv: Fix clear bss loop in the start-up code 2019-10-25 6:10 [U-Boot] [PATCH 0/8] RISC-V AX25-AE350 support SPL Andes ` (5 preceding siblings ...) 2019-10-25 6:10 ` [U-Boot] [PATCH 6/8] spl: cache: Allow cache drivers in SPL Andes @ 2019-10-25 6:10 ` Andes 2019-10-29 15:16 ` Bin Meng 2019-10-25 6:10 ` [U-Boot] [PATCH 8/8] riscv: dts: Support four cores SMP Andes 7 siblings, 1 reply; 71+ messages in thread From: Andes @ 2019-10-25 6:10 UTC (permalink / raw) To: u-boot From: Rick Chen <rick@andestech.com> For RV64, it will use sd instruction to clear t0 register, and the increament will be 8 bytes. So if the difference between__bss_strat and __bss_end was not 8 bytes aligned, the clear bss loop will overflow and acks like system hang. Signed-off-by: Rick Chen <rick@andestech.com> Cc: KC Lin <kclin@andestech.com> Cc: Alan Kao <alankao@andestech.com> --- arch/riscv/cpu/start.S | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S index 0a2ce6d..ee6d471 100644 --- a/arch/riscv/cpu/start.S +++ b/arch/riscv/cpu/start.S @@ -174,7 +174,7 @@ spl_clear_bss: spl_clear_bss_loop: SREG zero, 0(t0) addi t0, t0, REGBYTES - bne t0, t1, spl_clear_bss_loop + blt t0, t1, spl_clear_bss_loop spl_stack_gd_setup: jal spl_relocate_stack_gd @@ -324,7 +324,7 @@ clear_bss: clbss_l: SREG zero, 0(t0) /* clear loop... */ addi t0, t0, REGBYTES - bne t0, t1, clbss_l + blt t0, t1, clbss_l relocate_secondary_harts: #ifdef CONFIG_SMP -- 2.7.4 ^ permalink raw reply related [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 7/8] riscv: Fix clear bss loop in the start-up code 2019-10-25 6:10 ` [U-Boot] [PATCH 7/8] riscv: Fix clear bss loop in the start-up code Andes @ 2019-10-29 15:16 ` Bin Meng 2019-10-31 3:10 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-29 15:16 UTC (permalink / raw) To: u-boot Hi Rick, On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > From: Rick Chen <rick@andestech.com> > > For RV64, it will use sd instruction to clear t0 > register, and the increament will be 8 bytes. So > if the difference between__bss_strat and __bss_end > was not 8 bytes aligned, the clear bss loop will > overflow and acks like system hang. > > Signed-off-by: Rick Chen <rick@andestech.com> > Cc: KC Lin <kclin@andestech.com> > Cc: Alan Kao <alankao@andestech.com> > --- > arch/riscv/cpu/start.S | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S > index 0a2ce6d..ee6d471 100644 > --- a/arch/riscv/cpu/start.S > +++ b/arch/riscv/cpu/start.S > @@ -174,7 +174,7 @@ spl_clear_bss: > spl_clear_bss_loop: > SREG zero, 0(t0) > addi t0, t0, REGBYTES > - bne t0, t1, spl_clear_bss_loop > + blt t0, t1, spl_clear_bss_loop This leaves bss section not completely zeroed. > > spl_stack_gd_setup: > jal spl_relocate_stack_gd > @@ -324,7 +324,7 @@ clear_bss: > clbss_l: > SREG zero, 0(t0) /* clear loop... */ > addi t0, t0, REGBYTES > - bne t0, t1, clbss_l > + blt t0, t1, clbss_l > > relocate_secondary_harts: > #ifdef CONFIG_SMP > -- Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 7/8] riscv: Fix clear bss loop in the start-up code 2019-10-29 15:16 ` Bin Meng @ 2019-10-31 3:10 ` Rick Chen 2019-10-31 12:55 ` Bin Meng 0 siblings, 1 reply; 71+ messages in thread From: Rick Chen @ 2019-10-31 3:10 UTC (permalink / raw) To: u-boot Hi Bin > > Hi Rick, > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > From: Rick Chen <rick@andestech.com> > > > > For RV64, it will use sd instruction to clear t0 > > register, and the increament will be 8 bytes. So > > if the difference between__bss_strat and __bss_end > > was not 8 bytes aligned, the clear bss loop will > > overflow and acks like system hang. > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > Cc: KC Lin <kclin@andestech.com> > > Cc: Alan Kao <alankao@andestech.com> > > --- > > arch/riscv/cpu/start.S | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S > > index 0a2ce6d..ee6d471 100644 > > --- a/arch/riscv/cpu/start.S > > +++ b/arch/riscv/cpu/start.S > > @@ -174,7 +174,7 @@ spl_clear_bss: > > spl_clear_bss_loop: > > SREG zero, 0(t0) > > addi t0, t0, REGBYTES > > - bne t0, t1, spl_clear_bss_loop > > + blt t0, t1, spl_clear_bss_loop > > This leaves bss section not completely zeroed. I don't think this will clear bss section incompletely. Can you check it again ? Or explain more details why you think so ? Thanks Rick > > > > > spl_stack_gd_setup: > > jal spl_relocate_stack_gd > > @@ -324,7 +324,7 @@ clear_bss: > > clbss_l: > > SREG zero, 0(t0) /* clear loop... */ > > addi t0, t0, REGBYTES > > - bne t0, t1, clbss_l > > + blt t0, t1, clbss_l > > > > relocate_secondary_harts: > > #ifdef CONFIG_SMP > > -- > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 7/8] riscv: Fix clear bss loop in the start-up code 2019-10-31 3:10 ` Rick Chen @ 2019-10-31 12:55 ` Bin Meng 2019-11-01 5:24 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-31 12:55 UTC (permalink / raw) To: u-boot Hi Rick, On Thu, Oct 31, 2019 at 11:10 AM Rick Chen <rickchen36@gmail.com> wrote: > > Hi Bin > > > > > Hi Rick, > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > For RV64, it will use sd instruction to clear t0 > > > register, and the increament will be 8 bytes. So > > > if the difference between__bss_strat and __bss_end > > > was not 8 bytes aligned, the clear bss loop will > > > overflow and acks like system hang. > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > Cc: KC Lin <kclin@andestech.com> > > > Cc: Alan Kao <alankao@andestech.com> > > > --- > > > arch/riscv/cpu/start.S | 4 ++-- > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S > > > index 0a2ce6d..ee6d471 100644 > > > --- a/arch/riscv/cpu/start.S > > > +++ b/arch/riscv/cpu/start.S > > > @@ -174,7 +174,7 @@ spl_clear_bss: > > > spl_clear_bss_loop: > > > SREG zero, 0(t0) > > > addi t0, t0, REGBYTES > > > - bne t0, t1, spl_clear_bss_loop > > > + blt t0, t1, spl_clear_bss_loop > > > > This leaves bss section not completely zeroed. > > I don't think this will clear bss section incompletely. > Can you check it again ? > Or explain more details why you think so ? > Sorry I meant to say the other way around. This incorrectly clear the last a few bytes that are less than REGBYTES to zero. Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 7/8] riscv: Fix clear bss loop in the start-up code 2019-10-31 12:55 ` Bin Meng @ 2019-11-01 5:24 ` Rick Chen 0 siblings, 0 replies; 71+ messages in thread From: Rick Chen @ 2019-11-01 5:24 UTC (permalink / raw) To: u-boot Hi Bin > > Hi Rick, > > On Thu, Oct 31, 2019 at 11:10 AM Rick Chen <rickchen36@gmail.com> wrote: > > > > Hi Bin > > > > > > > > Hi Rick, > > > > > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > > > > > From: Rick Chen <rick@andestech.com> > > > > > > > > For RV64, it will use sd instruction to clear t0 > > > > register, and the increament will be 8 bytes. So > > > > if the difference between__bss_strat and __bss_end > > > > was not 8 bytes aligned, the clear bss loop will > > > > overflow and acks like system hang. > > > > > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > > > Cc: KC Lin <kclin@andestech.com> > > > > Cc: Alan Kao <alankao@andestech.com> > > > > --- > > > > arch/riscv/cpu/start.S | 4 ++-- > > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S > > > > index 0a2ce6d..ee6d471 100644 > > > > --- a/arch/riscv/cpu/start.S > > > > +++ b/arch/riscv/cpu/start.S > > > > @@ -174,7 +174,7 @@ spl_clear_bss: > > > > spl_clear_bss_loop: > > > > SREG zero, 0(t0) > > > > addi t0, t0, REGBYTES > > > > - bne t0, t1, spl_clear_bss_loop > > > > + blt t0, t1, spl_clear_bss_loop > > > > > > This leaves bss section not completely zeroed. > > > > I don't think this will clear bss section incompletely. > > Can you check it again ? > > Or explain more details why you think so ? > > > > Sorry I meant to say the other way around. This incorrectly clear the > last a few bytes that are less than REGBYTES to zero. > OK . I will also modify as ALIGN(8) in ld to avoid this boundary exceeding. Thanks Rick > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 8/8] riscv: dts: Support four cores SMP 2019-10-25 6:10 [U-Boot] [PATCH 0/8] RISC-V AX25-AE350 support SPL Andes ` (6 preceding siblings ...) 2019-10-25 6:10 ` [U-Boot] [PATCH 7/8] riscv: Fix clear bss loop in the start-up code Andes @ 2019-10-25 6:10 ` Andes 2019-10-29 15:17 ` Bin Meng 7 siblings, 1 reply; 71+ messages in thread From: Andes @ 2019-10-25 6:10 UTC (permalink / raw) To: u-boot From: Rick Chen <rick@andestech.com> Add CPU2 and CPU3 informations in cpus node to support four cores SMP booting. Signed-off-by: Rick Chen <rick@andestech.com> Cc: KC Lin <kclin@andestech.com> Cc: Alan Kao <alankao@andestech.com> --- arch/riscv/dts/ae350_32.dts | 51 ++++++++++++++++++++++++++++++++++++++++++--- arch/riscv/dts/ae350_64.dts | 51 ++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 96 insertions(+), 6 deletions(-) diff --git a/arch/riscv/dts/ae350_32.dts b/arch/riscv/dts/ae350_32.dts index 97b7cee..c794a7f 100644 --- a/arch/riscv/dts/ae350_32.dts +++ b/arch/riscv/dts/ae350_32.dts @@ -62,6 +62,48 @@ compatible = "riscv,cpu-intc"; }; }; + CPU2: cpu at 2 { + device_type = "cpu"; + reg = <2>; + status = "okay"; + compatible = "riscv"; + riscv,isa = "rv32imafdc"; + riscv,priv-major = <1>; + riscv,priv-minor = <10>; + mmu-type = "riscv,sv32"; + clock-frequency = <60000000>; + i-cache-size = <0x8000>; + i-cache-line-size = <32>; + d-cache-size = <0x8000>; + d-cache-line-size = <32>; + next-level-cache = <&L2>; + CPU2_intc: interrupt-controller { + #interrupt-cells = <1>; + interrupt-controller; + compatible = "riscv,cpu-intc"; + }; + }; + CPU3: cpu at 3 { + device_type = "cpu"; + reg = <3>; + status = "okay"; + compatible = "riscv"; + riscv,isa = "rv32imafdc"; + riscv,priv-major = <1>; + riscv,priv-minor = <10>; + mmu-type = "riscv,sv32"; + clock-frequency = <60000000>; + i-cache-size = <0x8000>; + i-cache-line-size = <32>; + d-cache-size = <0x8000>; + d-cache-line-size = <32>; + next-level-cache = <&L2>; + CPU3_intc: interrupt-controller { + #interrupt-cells = <1>; + interrupt-controller; + compatible = "riscv,cpu-intc"; + }; + }; }; L2: l2-cache at e0500000 { @@ -94,7 +136,8 @@ interrupt-controller; reg = <0xe4000000 0x2000000>; riscv,ndev=<71>; - interrupts-extended = <&CPU0_intc 11 &CPU0_intc 9 &CPU1_intc 11 &CPU1_intc 9>; + interrupts-extended = <&CPU0_intc 11 &CPU0_intc 9 &CPU1_intc 11 &CPU1_intc 9 + &CPU2_intc 11 &CPU2_intc 9 &CPU3_intc 11 &CPU3_intc 9>; }; plic1: interrupt-controller at e6400000 { @@ -104,12 +147,14 @@ interrupt-controller; reg = <0xe6400000 0x400000>; riscv,ndev=<2>; - interrupts-extended = <&CPU0_intc 3 &CPU1_intc 3>; + interrupts-extended = <&CPU0_intc 3 &CPU1_intc 3 + &CPU2_intc 3 &CPU3_intc 3>; }; plmt0 at e6000000 { compatible = "riscv,plmt0"; - interrupts-extended = <&CPU0_intc 7 &CPU1_intc 7>; + interrupts-extended = <&CPU0_intc 7 &CPU1_intc 7 + &CPU2_intc 7 &CPU3_intc 7>; reg = <0xe6000000 0x100000>; }; }; diff --git a/arch/riscv/dts/ae350_64.dts b/arch/riscv/dts/ae350_64.dts index d8f00f8..b3d6d15 100644 --- a/arch/riscv/dts/ae350_64.dts +++ b/arch/riscv/dts/ae350_64.dts @@ -62,6 +62,48 @@ compatible = "riscv,cpu-intc"; }; }; + CPU2: cpu at 2 { + device_type = "cpu"; + reg = <2>; + status = "okay"; + compatible = "riscv"; + riscv,isa = "rv64imafdc"; + riscv,priv-major = <1>; + riscv,priv-minor = <10>; + mmu-type = "riscv,sv39"; + clock-frequency = <60000000>; + i-cache-size = <0x8000>; + i-cache-line-size = <32>; + d-cache-size = <0x8000>; + d-cache-line-size = <32>; + next-level-cache = <&L2>; + CPU2_intc: interrupt-controller { + #interrupt-cells = <1>; + interrupt-controller; + compatible = "riscv,cpu-intc"; + }; + }; + CPU3: cpu at 3 { + device_type = "cpu"; + reg = <3>; + status = "okay"; + compatible = "riscv"; + riscv,isa = "rv64imafdc"; + riscv,priv-major = <1>; + riscv,priv-minor = <10>; + mmu-type = "riscv,sv39"; + clock-frequency = <60000000>; + i-cache-size = <0x8000>; + i-cache-line-size = <32>; + d-cache-size = <0x8000>; + d-cache-line-size = <32>; + next-level-cache = <&L2>; + CPU3_intc: interrupt-controller { + #interrupt-cells = <1>; + interrupt-controller; + compatible = "riscv,cpu-intc"; + }; + }; }; L2: l2-cache at e0500000 { @@ -94,7 +136,8 @@ interrupt-controller; reg = <0x0 0xe4000000 0x0 0x2000000>; riscv,ndev=<71>; - interrupts-extended = <&CPU0_intc 11 &CPU0_intc 9 &CPU1_intc 11 &CPU1_intc 9>; + interrupts-extended = <&CPU0_intc 11 &CPU0_intc 9 &CPU1_intc 11 &CPU1_intc 9 + &CPU2_intc 11 &CPU2_intc 9 &CPU3_intc 11 &CPU3_intc 9>; }; plic1: interrupt-controller at e6400000 { @@ -104,12 +147,14 @@ interrupt-controller; reg = <0x0 0xe6400000 0x0 0x400000>; riscv,ndev=<2>; - interrupts-extended = <&CPU0_intc 3 &CPU1_intc 3>; + interrupts-extended = <&CPU0_intc 3 &CPU1_intc 3 + &CPU2_intc 3 &CPU3_intc 3>; }; plmt0 at e6000000 { compatible = "riscv,plmt0"; - interrupts-extended = <&CPU0_intc 7 &CPU1_intc 7>; + interrupts-extended = <&CPU0_intc 7 &CPU1_intc 7 + &CPU2_intc 7 &CPU3_intc 7>; reg = <0x0 0xe6000000 0x0 0x100000>; }; }; -- 2.7.4 ^ permalink raw reply related [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 8/8] riscv: dts: Support four cores SMP 2019-10-25 6:10 ` [U-Boot] [PATCH 8/8] riscv: dts: Support four cores SMP Andes @ 2019-10-29 15:17 ` Bin Meng 2019-10-31 5:57 ` Rick Chen 0 siblings, 1 reply; 71+ messages in thread From: Bin Meng @ 2019-10-29 15:17 UTC (permalink / raw) To: u-boot Hi Rick, On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > From: Rick Chen <rick@andestech.com> > > Add CPU2 and CPU3 informations in cpus node nits: information > to support four cores SMP booting. > > Signed-off-by: Rick Chen <rick@andestech.com> > Cc: KC Lin <kclin@andestech.com> > Cc: Alan Kao <alankao@andestech.com> > --- > arch/riscv/dts/ae350_32.dts | 51 ++++++++++++++++++++++++++++++++++++++++++--- > arch/riscv/dts/ae350_64.dts | 51 ++++++++++++++++++++++++++++++++++++++++++--- > 2 files changed, 96 insertions(+), 6 deletions(-) > > diff --git a/arch/riscv/dts/ae350_32.dts b/arch/riscv/dts/ae350_32.dts > index 97b7cee..c794a7f 100644 > --- a/arch/riscv/dts/ae350_32.dts > +++ b/arch/riscv/dts/ae350_32.dts > @@ -62,6 +62,48 @@ > compatible = "riscv,cpu-intc"; > }; > }; > + CPU2: cpu at 2 { > + device_type = "cpu"; > + reg = <2>; > + status = "okay"; > + compatible = "riscv"; > + riscv,isa = "rv32imafdc"; > + riscv,priv-major = <1>; > + riscv,priv-minor = <10>; > + mmu-type = "riscv,sv32"; > + clock-frequency = <60000000>; > + i-cache-size = <0x8000>; > + i-cache-line-size = <32>; > + d-cache-size = <0x8000>; > + d-cache-line-size = <32>; > + next-level-cache = <&L2>; > + CPU2_intc: interrupt-controller { > + #interrupt-cells = <1>; > + interrupt-controller; > + compatible = "riscv,cpu-intc"; > + }; > + }; > + CPU3: cpu at 3 { > + device_type = "cpu"; > + reg = <3>; > + status = "okay"; > + compatible = "riscv"; > + riscv,isa = "rv32imafdc"; > + riscv,priv-major = <1>; > + riscv,priv-minor = <10>; > + mmu-type = "riscv,sv32"; > + clock-frequency = <60000000>; > + i-cache-size = <0x8000>; > + i-cache-line-size = <32>; > + d-cache-size = <0x8000>; > + d-cache-line-size = <32>; > + next-level-cache = <&L2>; > + CPU3_intc: interrupt-controller { > + #interrupt-cells = <1>; > + interrupt-controller; > + compatible = "riscv,cpu-intc"; > + }; > + }; > }; > > L2: l2-cache at e0500000 { > @@ -94,7 +136,8 @@ > interrupt-controller; > reg = <0xe4000000 0x2000000>; > riscv,ndev=<71>; > - interrupts-extended = <&CPU0_intc 11 &CPU0_intc 9 &CPU1_intc 11 &CPU1_intc 9>; > + interrupts-extended = <&CPU0_intc 11 &CPU0_intc 9 &CPU1_intc 11 &CPU1_intc 9 > + &CPU2_intc 11 &CPU2_intc 9 &CPU3_intc 11 &CPU3_intc 9>; The indentation looks wrong. > }; > > plic1: interrupt-controller at e6400000 { > @@ -104,12 +147,14 @@ > interrupt-controller; > reg = <0xe6400000 0x400000>; > riscv,ndev=<2>; > - interrupts-extended = <&CPU0_intc 3 &CPU1_intc 3>; > + interrupts-extended = <&CPU0_intc 3 &CPU1_intc 3 > + &CPU2_intc 3 &CPU3_intc 3>; > }; > > plmt0 at e6000000 { > compatible = "riscv,plmt0"; > - interrupts-extended = <&CPU0_intc 7 &CPU1_intc 7>; > + interrupts-extended = <&CPU0_intc 7 &CPU1_intc 7 > + &CPU2_intc 7 &CPU3_intc 7>; > reg = <0xe6000000 0x100000>; > }; > }; [snip] Regards, Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
* [U-Boot] [PATCH 8/8] riscv: dts: Support four cores SMP 2019-10-29 15:17 ` Bin Meng @ 2019-10-31 5:57 ` Rick Chen 0 siblings, 0 replies; 71+ messages in thread From: Rick Chen @ 2019-10-31 5:57 UTC (permalink / raw) To: u-boot Hi Bin > > Hi Rick, > > On Fri, Oct 25, 2019 at 2:18 PM Andes <uboot@andestech.com> wrote: > > > > From: Rick Chen <rick@andestech.com> > > > > Add CPU2 and CPU3 informations in cpus node > > nits: information OK > > > to support four cores SMP booting. > > > > Signed-off-by: Rick Chen <rick@andestech.com> > > Cc: KC Lin <kclin@andestech.com> > > Cc: Alan Kao <alankao@andestech.com> > > --- > > arch/riscv/dts/ae350_32.dts | 51 ++++++++++++++++++++++++++++++++++++++++++--- > > arch/riscv/dts/ae350_64.dts | 51 ++++++++++++++++++++++++++++++++++++++++++--- > > 2 files changed, 96 insertions(+), 6 deletions(-) > > > > diff --git a/arch/riscv/dts/ae350_32.dts b/arch/riscv/dts/ae350_32.dts > > index 97b7cee..c794a7f 100644 > > --- a/arch/riscv/dts/ae350_32.dts > > +++ b/arch/riscv/dts/ae350_32.dts > > @@ -62,6 +62,48 @@ > > compatible = "riscv,cpu-intc"; > > }; > > }; > > + CPU2: cpu at 2 { > > + device_type = "cpu"; > > + reg = <2>; > > + status = "okay"; > > + compatible = "riscv"; > > + riscv,isa = "rv32imafdc"; > > + riscv,priv-major = <1>; > > + riscv,priv-minor = <10>; > > + mmu-type = "riscv,sv32"; > > + clock-frequency = <60000000>; > > + i-cache-size = <0x8000>; > > + i-cache-line-size = <32>; > > + d-cache-size = <0x8000>; > > + d-cache-line-size = <32>; > > + next-level-cache = <&L2>; > > + CPU2_intc: interrupt-controller { > > + #interrupt-cells = <1>; > > + interrupt-controller; > > + compatible = "riscv,cpu-intc"; > > + }; > > + }; > > + CPU3: cpu at 3 { > > + device_type = "cpu"; > > + reg = <3>; > > + status = "okay"; > > + compatible = "riscv"; > > + riscv,isa = "rv32imafdc"; > > + riscv,priv-major = <1>; > > + riscv,priv-minor = <10>; > > + mmu-type = "riscv,sv32"; > > + clock-frequency = <60000000>; > > + i-cache-size = <0x8000>; > > + i-cache-line-size = <32>; > > + d-cache-size = <0x8000>; > > + d-cache-line-size = <32>; > > + next-level-cache = <&L2>; > > + CPU3_intc: interrupt-controller { > > + #interrupt-cells = <1>; > > + interrupt-controller; > > + compatible = "riscv,cpu-intc"; > > + }; > > + }; > > }; > > > > L2: l2-cache at e0500000 { > > @@ -94,7 +136,8 @@ > > interrupt-controller; > > reg = <0xe4000000 0x2000000>; > > riscv,ndev=<71>; > > - interrupts-extended = <&CPU0_intc 11 &CPU0_intc 9 &CPU1_intc 11 &CPU1_intc 9>; > > + interrupts-extended = <&CPU0_intc 11 &CPU0_intc 9 &CPU1_intc 11 &CPU1_intc 9 > > + &CPU2_intc 11 &CPU2_intc 9 &CPU3_intc 11 &CPU3_intc 9>; > > The indentation looks wrong. OK. I will fix it. Thanks Rick > > > }; > > > > plic1: interrupt-controller at e6400000 { > > @@ -104,12 +147,14 @@ > > interrupt-controller; > > reg = <0xe6400000 0x400000>; > > riscv,ndev=<2>; > > - interrupts-extended = <&CPU0_intc 3 &CPU1_intc 3>; > > + interrupts-extended = <&CPU0_intc 3 &CPU1_intc 3 > > + &CPU2_intc 3 &CPU3_intc 3>; > > }; > > > > plmt0 at e6000000 { > > compatible = "riscv,plmt0"; > > - interrupts-extended = <&CPU0_intc 7 &CPU1_intc 7>; > > + interrupts-extended = <&CPU0_intc 7 &CPU1_intc 7 > > + &CPU2_intc 7 &CPU3_intc 7>; > > reg = <0xe6000000 0x100000>; > > }; > > }; > > [snip] > > Regards, > Bin ^ permalink raw reply [flat|nested] 71+ messages in thread
end of thread, other threads:[~2019-11-14 17:10 UTC | newest] Thread overview: 71+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-10-25 6:10 [U-Boot] [PATCH 0/8] RISC-V AX25-AE350 support SPL Andes 2019-10-25 6:10 ` [U-Boot] [PATCH 1/8] riscv: ax25: add SPL support Andes 2019-10-29 14:29 ` Bin Meng 2019-10-30 0:42 ` Rick Chen 2019-10-30 10:06 ` Bin Meng 2019-10-31 2:11 ` Rick Chen 2019-10-25 6:10 ` [U-Boot] [PATCH 2/8] riscv: ax25-ae350: add SPL configuration Andes 2019-10-29 14:39 ` Bin Meng 2019-10-30 2:06 ` Rick Chen 2019-10-25 6:10 ` [U-Boot] [PATCH 3/8] riscv: ax25-ae350: Use generic memory size setup Andes 2019-10-29 14:42 ` Bin Meng 2019-10-30 2:19 ` Rick Chen 2019-10-25 6:10 ` [U-Boot] [PATCH 4/8] riscv: andes_plic: Fix some wrong configurations Andes 2019-10-29 14:51 ` Bin Meng 2019-10-30 2:50 ` Rick Chen 2019-10-30 10:38 ` Bin Meng 2019-10-31 1:00 ` Alan Kao 2019-10-31 3:36 ` Bin Meng 2019-10-31 7:48 ` Alan Kao 2019-10-31 8:10 ` Bin Meng 2019-10-31 8:12 ` Anup Patel 2019-10-31 10:43 ` Anup Patel 2019-11-01 5:25 ` Rick Chen 2019-11-05 1:50 ` Rick Chen 2019-11-05 6:34 ` Anup Patel 2019-11-06 6:44 ` Rick Chen 2019-11-06 8:48 ` Anup Patel 2019-11-06 8:58 ` Anup Patel 2019-11-06 9:21 ` Rick Chen 2019-11-06 11:11 ` Anup Patel 2019-11-07 1:34 ` Rick Chen 2019-11-07 5:15 ` Anup Patel 2019-11-07 5:45 ` Anup Patel 2019-11-07 6:10 ` Rick Chen 2019-11-07 6:18 ` Anup Patel 2019-11-07 9:41 ` Auer, Lukas 2019-11-07 10:44 ` Anup Patel 2019-11-07 11:41 ` Rick Chen 2019-11-07 12:22 ` Anup Patel 2019-11-08 1:23 ` Rick Chen 2019-11-08 12:14 ` Anup Patel 2019-11-07 18:44 ` Atish Patra 2019-11-08 1:13 ` Rick Chen 2019-11-08 7:27 ` Rick Chen 2019-11-08 8:59 ` Auer, Lukas 2019-11-11 7:19 ` Rick Chen 2019-11-12 9:47 ` Auer, Lukas 2019-11-13 3:42 ` Rick Chen 2019-11-14 7:27 ` Anup Patel 2019-11-14 17:10 ` Auer, Lukas 2019-11-07 11:44 ` Auer, Lukas 2019-11-07 12:27 ` Anup Patel 2019-11-07 13:37 ` Auer, Lukas 2019-10-31 2:23 ` Rick Chen 2019-10-25 6:10 ` [U-Boot] [PATCH 5/8] riscv: ax25: cache: Add SPL_RISCV_MMODE for SPL Andes 2019-10-29 14:59 ` Bin Meng 2019-10-31 2:31 ` Rick Chen 2019-10-31 3:00 ` Bin Meng 2019-10-25 6:10 ` [U-Boot] [PATCH 6/8] spl: cache: Allow cache drivers in SPL Andes 2019-10-29 15:14 ` Bin Meng 2019-10-31 2:52 ` Rick Chen 2019-10-31 3:01 ` Bin Meng 2019-10-31 3:22 ` Rick Chen 2019-10-25 6:10 ` [U-Boot] [PATCH 7/8] riscv: Fix clear bss loop in the start-up code Andes 2019-10-29 15:16 ` Bin Meng 2019-10-31 3:10 ` Rick Chen 2019-10-31 12:55 ` Bin Meng 2019-11-01 5:24 ` Rick Chen 2019-10-25 6:10 ` [U-Boot] [PATCH 8/8] riscv: dts: Support four cores SMP Andes 2019-10-29 15:17 ` Bin Meng 2019-10-31 5:57 ` Rick Chen
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.