Watchdog auto-start and TIMER_EARLY broken in RISC-V U-Boot

Watchdog auto-start and TIMER_EARLY broken in RISC-V U-Boot

[Samuel Holland]

Hello,

I am porting U-Boot to the Allwinner D1 SoC, and I ran into an issue
where the board fails to boot if I enable watchdog auto-start.

The call to get_timer() -> get_ticks() panics because no timer is
available. And since panic_finish() calls udelay(), this causes infinite
recursion of trying and failing to get the timer.

The issue is that the RISC-V architectural timer driver is bound in
arch_early_init_r, which is only called _after_ the first instance of
INIT_FUNC_WATCHDOG_RESET in the initcall list.

Below is a boot log where I commented out the calls to get_timer() and
time_after_eq() in watchdog_reset() in order to illustrate the problem.

I modified the log to add the initcall symbol names for clarity.

> U-Boot 2021.10-rc2-00278-ge2be1a426d6-dirty (Jan 01 1970 - 00:00:00 +0000)
> 
> DRAM:  512 MiB
> initcall: 000000005ff63916
> initcall: 000000005ff6391a
> initcall: 000000004a012a4e (initr_reloc_global_data)
> initcall: 000000004a012a88 (initr_barrier)
> initcall: 000000004a012a1c (initr_malloc)
> Pre-reloc malloc() used 0x720 bytes (1 KB)
> initcall: 000000004a01290e (log_init)
> initcall: 000000004a012a94 (initr_bootstage)
> initcall: 000000004a012a8c (initr_of_live)
> initcall: 000000004a012a06 (initr_dm)
> Binding device root_driver to driver root_driver
> Probing device root_driver with driver root_driver
> Binding device osc24M_clk to driver fixed_clock
> Binding device soc to driver simple_bus
> Binding device pinctrl@2000000 to driver sunxi-pinctrl
> Binding device pinctrl@2000000 to driver gpio_sunxi
> Binding device PA to driver gpio_sunxi
> Binding device PB to driver gpio_sunxi
> Binding device PC to driver gpio_sunxi
> Binding device PD to driver gpio_sunxi
> Binding device PE to driver gpio_sunxi
> Binding device PF to driver gpio_sunxi
> Binding device PG to driver gpio_sunxi
> Binding device i2c0-pb10-pins to driver pinconfig
> Binding device i2c2-pb0-pins to driver pinconfig
> Binding device mmc0-pins to driver pinconfig
> Binding device mmc1-pins to driver pinconfig
> Binding device rgmii-pe-pins to driver pinconfig
> Binding device spi0-pins to driver pinconfig
> Binding device spi1-pd-pins to driver pinconfig
> Binding device uart0-pb8-pins to driver pinconfig
> Binding device uart1-pg6-pins to driver pinconfig
> Binding device uart1-pg8-rts-cts-pins to driver pinconfig
> Binding device i2s2-pb-pins to driver pinconfig
> Binding device i2s2-pb3-din-pin to driver pinconfig
> Binding device i2s2-pb4-dout-pin to driver pinconfig
> Binding device ledc-pc0-pin to driver pinconfig
> Binding device pwm0-pd16-pin to driver pinconfig
> Binding device pwm2-pd18-pin to driver pinconfig
> Binding device pwm7-pd22-pin to driver pinconfig
> Binding device spdif-pd22-pin to driver pinconfig
> Binding device clock-controller@2001000 to driver sun50i_d1_ccu
> Binding device reset to driver sunxi_reset
> Binding device serial@2500000 to driver ns16550_serial
> Binding device serial@2500400 to driver ns16550_serial
> Binding device i2c@2502000 to driver i2c_mvtwsi
> Binding device i2c@2502800 to driver i2c_mvtwsi
> Binding device mmc@4020000 to driver sunxi_mmc
> Binding device mmc@4020000.blk to driver mmc_blk
> Binding device mmc@4021000 to driver sunxi_mmc
> Binding device mmc@4021000.blk to driver mmc_blk
> Binding device usb@4100000 to driver sunxi-musb
> Binding device phy@4100400 to driver sun4i_usb_phy
> Binding device usb@4101000 to driver ehci_generic
> Binding device usb@4101400 to driver ohci_generic
> Binding device usb@4200000 to driver ehci_generic
> Binding device usb@4200400 to driver ohci_generic
> Binding device ethernet@4500000 to driver eth_sun8i_emac
> Binding device watchdog@6011000 to driver sunxi_wdt
> initcall: 000000004a001914 (board_init)
> initcall: 000000004a04d814 (efi_memory_init)
> initcall: 000000004a012a9c (initr_binman)
> initcall: 000000004a012a90 (initr_dm_devices)

Here's where dm_timer_init() would be called if CONFIG_TIMER_EARLY was
enabled, but that still doesn't help because the timer is not bound yet.
In fact, CONFIG_TIMER_EARLY actually makes the situation worse, because
the board hangs if dm_timer_init() fails:

initcall sequence 000000005ffdac78 failed at call 000000004a012a06 (err=-19)
### ERROR ### Please RESET the board ###

> initcall: 000000004a01c4f4 (stdio_init_tables)
> initcall: 000000004a039ff8 (serial_initialize)
> Probing device pinctrl@2000000 with driver sunxi-pinctrl
> Probing device soc with driver simple_bus
> Probing device clock-controller@2001000 with driver sun50i_d1_ccu
> Probing device osc24M_clk with driver fixed_clock
> Probing device uart0-pb8-pins with driver pinconfig
> Probing device serial@2500000 with driver ns16550_serial
> Probing device reset with driver sunxi_reset
> initcall: 000000004a0129ea (initr_announce)
> Now running in RAM - U-Boot at: 5ff51000
> initcall: 000000004a0367ce (initr_watchdog)
> Probing device watchdog@6011000 with driver sunxi_wdt
> WDT:   Started with servicing (16s timeout)
> initcall: 000000004a012610 (init_func_watchdog_reset)

Here's where the timer needs to be available for watchdog auto-start to
work. An unpatched U-Boot would panic here.

> initcall: 000000004a012a84 (arch_initr_trap)
> initcall: 000000004a012610 (init_func_watchdog_reset)
> initcall: 000000004a012610 (init_func_watchdog_reset)
> initcall: 000000004a000ed4 (arch_early_init_r)
> Binding device cpus to driver cpu_bus
> Binding device cpu@0 to driver riscv_cpu
> Binding device riscv_timer to driver riscv_timer

But here is where the timer device is actually bound.

Do you have any suggestions for how best to fix this?

Some things I can think of:
 - Add an option to probe the CPU device early in initr_dm_devices()
 - Require the board to probe the CPU device from board_init()
 - Move arch_early_init_r() earlier in the initcall list
 - Skip restarting the watchdog until a timer is available (but this
   would not fix CONFIG_TIMER_EARLY)

Thanks,
Samuel

> Probing device cpus with driver cpu_bus
> Probing device cpu@0 with driver riscv_cpu
> Binding device sbi-sysreset to driver sbi-sysreset
> initcall: 000000004a012a98 (power_init_board)
> initcall: 000000004a012610 (init_func_watchdog_reset)
> initcall: 000000004a0129ca (initr_mmc)
> MMC:   Probing device mmc0-pins with driver pinconfig
> Probing device mmc@4020000 with driver sunxi_mmc
> Probing device pinctrl@2000000 with driver gpio_sunxi
> Probing device PF with driver gpio_sunxi
> Need to init timer!
> Probing device riscv_timer with driver riscv_timer
> Timer is 000000005df34f80
> Read timer, got 0x17dacfb
> Timer is 000000005df34f80
> Read timer, got 0x17f63b8
> Probing device mmc1-pins with driver pinconfig
> Probing device mmc@4021000 with driver sunxi_mmc
> Timer is 000000005df34f80
> Read timer, got 0x186be7e
> Timer is 000000005df34f80
> Read timer, got 0x188752e
> mmc@4020000: 0, mmc@4021000: 1
> initcall: 000000004a012970 (relocated to 000000005ff63970)
> Loading Environment from nowhere... OK
> initcall: 000000004a012610 (relocated to 000000005ff63610)
> initcall: 000000004a012912 (relocated to 000000005ff63912)
> initcall: 000000004a012610 (relocated to 000000005ff63610)
> initcall: 000000004a01c50c (relocated to 000000005ff6d50c)
> initcall: 000000004a010938 (relocated to 000000005ff61938)
> initcall: 000000004a0188bc (relocated to 000000005ff698bc)
> In:    serial@2500000
> Out:   serial@2500000
> Err:   serial@2500000
> initcall: 000000004a012610 (relocated to 000000005ff63610)
> initcall: 000000004a0012dc (relocated to 000000005ff522dc)
> initcall: 000000004a012950 (relocated to 000000005ff63950)
> initcall: 000000004a012610 (relocated to 000000005ff63610)
> initcall: 000000004a012934 (relocated to 000000005ff63934)
> Net:   phy interface8
> Probing device rgmii-pe-pins with driver pinconfig
> Probing device ethernet@4500000 with driver eth_sun8i_emac
> Timer is 000000005df34f80
> Read timer, got 0x1ae9538
> Timer is 000000005df34f80
> Read timer, got 0x1b04bed
> Timer is 000000005df34f80
> Read timer, got 0x1b202af
> Timer is 000000005df34f80
> Read timer, got 0x1b3b96e
> Timer is 000000005df34f80
> Read timer, got 0x1b5702c
> Timer is 000000005df34f80
> Read timer, got 0x1b726ec
> 
> Warning: ethernet@4500000 (eth0) using random MAC address - 62:69:97:68:19:04
> eth0: ethernet@4500000
> initcall: 000000004a01292a (relocated to 000000005ff6392a)
> Hit any key to stop autoboot:  0

Re: Watchdog auto-start and TIMER_EARLY broken in RISC-V U-Boot

[Sean Anderson]

+CC Heinrich

Did you ever try booting with WDT on k210?

On 9/11/21 12:43 PM, Samuel Holland wrote:
> Hello,
> 
> I am porting U-Boot to the Allwinner D1 SoC, and I ran into an issue
> where the board fails to boot if I enable watchdog auto-start.
> 
> The call to get_timer() -> get_ticks() panics because no timer is
> available. And since panic_finish() calls udelay(), this causes infinite
> recursion of trying and failing to get the timer.
> 
> The issue is that the RISC-V architectural timer driver is bound in
> arch_early_init_r, which is only called _after_ the first instance of
> INIT_FUNC_WATCHDOG_RESET in the initcall list.
> 
> Below is a boot log where I commented out the calls to get_timer() and
> time_after_eq() in watchdog_reset() in order to illustrate the problem.
> 
> I modified the log to add the initcall symbol names for clarity.
> 
>> U-Boot 2021.10-rc2-00278-ge2be1a426d6-dirty (Jan 01 1970 - 00:00:00 +0000)
>>
>> DRAM:  512 MiB
>> initcall: 000000005ff63916
>> initcall: 000000005ff6391a
>> initcall: 000000004a012a4e (initr_reloc_global_data)
>> initcall: 000000004a012a88 (initr_barrier)
>> initcall: 000000004a012a1c (initr_malloc)
>> Pre-reloc malloc() used 0x720 bytes (1 KB)
>> initcall: 000000004a01290e (log_init)
>> initcall: 000000004a012a94 (initr_bootstage)
>> initcall: 000000004a012a8c (initr_of_live)
>> initcall: 000000004a012a06 (initr_dm)
>> Binding device root_driver to driver root_driver
>> Probing device root_driver with driver root_driver
>> Binding device osc24M_clk to driver fixed_clock
>> Binding device soc to driver simple_bus
>> Binding device pinctrl@2000000 to driver sunxi-pinctrl
>> Binding device pinctrl@2000000 to driver gpio_sunxi
>> Binding device PA to driver gpio_sunxi
>> Binding device PB to driver gpio_sunxi
>> Binding device PC to driver gpio_sunxi
>> Binding device PD to driver gpio_sunxi
>> Binding device PE to driver gpio_sunxi
>> Binding device PF to driver gpio_sunxi
>> Binding device PG to driver gpio_sunxi
>> Binding device i2c0-pb10-pins to driver pinconfig
>> Binding device i2c2-pb0-pins to driver pinconfig
>> Binding device mmc0-pins to driver pinconfig
>> Binding device mmc1-pins to driver pinconfig
>> Binding device rgmii-pe-pins to driver pinconfig
>> Binding device spi0-pins to driver pinconfig
>> Binding device spi1-pd-pins to driver pinconfig
>> Binding device uart0-pb8-pins to driver pinconfig
>> Binding device uart1-pg6-pins to driver pinconfig
>> Binding device uart1-pg8-rts-cts-pins to driver pinconfig
>> Binding device i2s2-pb-pins to driver pinconfig
>> Binding device i2s2-pb3-din-pin to driver pinconfig
>> Binding device i2s2-pb4-dout-pin to driver pinconfig
>> Binding device ledc-pc0-pin to driver pinconfig
>> Binding device pwm0-pd16-pin to driver pinconfig
>> Binding device pwm2-pd18-pin to driver pinconfig
>> Binding device pwm7-pd22-pin to driver pinconfig
>> Binding device spdif-pd22-pin to driver pinconfig
>> Binding device clock-controller@2001000 to driver sun50i_d1_ccu
>> Binding device reset to driver sunxi_reset
>> Binding device serial@2500000 to driver ns16550_serial
>> Binding device serial@2500400 to driver ns16550_serial
>> Binding device i2c@2502000 to driver i2c_mvtwsi
>> Binding device i2c@2502800 to driver i2c_mvtwsi
>> Binding device mmc@4020000 to driver sunxi_mmc
>> Binding device mmc@4020000.blk to driver mmc_blk
>> Binding device mmc@4021000 to driver sunxi_mmc
>> Binding device mmc@4021000.blk to driver mmc_blk
>> Binding device usb@4100000 to driver sunxi-musb
>> Binding device phy@4100400 to driver sun4i_usb_phy
>> Binding device usb@4101000 to driver ehci_generic
>> Binding device usb@4101400 to driver ohci_generic
>> Binding device usb@4200000 to driver ehci_generic
>> Binding device usb@4200400 to driver ohci_generic
>> Binding device ethernet@4500000 to driver eth_sun8i_emac
>> Binding device watchdog@6011000 to driver sunxi_wdt
>> initcall: 000000004a001914 (board_init)
>> initcall: 000000004a04d814 (efi_memory_init)
>> initcall: 000000004a012a9c (initr_binman)
>> initcall: 000000004a012a90 (initr_dm_devices)
> 
> Here's where dm_timer_init() would be called if CONFIG_TIMER_EARLY was
> enabled, but that still doesn't help because the timer is not bound yet.
> In fact, CONFIG_TIMER_EARLY actually makes the situation worse, because
> the board hangs if dm_timer_init() fails:
>
> initcall sequence 000000005ffdac78 failed at call 000000004a012a06 (err=-19)
> ### ERROR ### Please RESET the board ###

I'm not sure what's going on here. Why does dm_timer_init get called from initr_dm?
Why does it fail?

--Sean

>> initcall: 000000004a01c4f4 (stdio_init_tables)
>> initcall: 000000004a039ff8 (serial_initialize)
>> Probing device pinctrl@2000000 with driver sunxi-pinctrl
>> Probing device soc with driver simple_bus
>> Probing device clock-controller@2001000 with driver sun50i_d1_ccu
>> Probing device osc24M_clk with driver fixed_clock
>> Probing device uart0-pb8-pins with driver pinconfig
>> Probing device serial@2500000 with driver ns16550_serial
>> Probing device reset with driver sunxi_reset
>> initcall: 000000004a0129ea (initr_announce)
>> Now running in RAM - U-Boot at: 5ff51000
>> initcall: 000000004a0367ce (initr_watchdog)
>> Probing device watchdog@6011000 with driver sunxi_wdt
>> WDT:   Started with servicing (16s timeout)
>> initcall: 000000004a012610 (init_func_watchdog_reset)
> 
> Here's where the timer needs to be available for watchdog auto-start to
> work. An unpatched U-Boot would panic here.
> 
>> initcall: 000000004a012a84 (arch_initr_trap)
>> initcall: 000000004a012610 (init_func_watchdog_reset)
>> initcall: 000000004a012610 (init_func_watchdog_reset)
>> initcall: 000000004a000ed4 (arch_early_init_r)
>> Binding device cpus to driver cpu_bus
>> Binding device cpu@0 to driver riscv_cpu
>> Binding device riscv_timer to driver riscv_timer
> 
> But here is where the timer device is actually bound.
> 
> Do you have any suggestions for how best to fix this?
> 
> Some things I can think of:
>   - Add an option to probe the CPU device early in initr_dm_devices()
>   - Require the board to probe the CPU device from board_init()
>   - Move arch_early_init_r() earlier in the initcall list
>   - Skip restarting the watchdog until a timer is available (but this
>     would not fix CONFIG_TIMER_EARLY)
> 
> Thanks,
> Samuel
> 
>> Probing device cpus with driver cpu_bus
>> Probing device cpu@0 with driver riscv_cpu
>> Binding device sbi-sysreset to driver sbi-sysreset
>> initcall: 000000004a012a98 (power_init_board)
>> initcall: 000000004a012610 (init_func_watchdog_reset)
>> initcall: 000000004a0129ca (initr_mmc)
>> MMC:   Probing device mmc0-pins with driver pinconfig
>> Probing device mmc@4020000 with driver sunxi_mmc
>> Probing device pinctrl@2000000 with driver gpio_sunxi
>> Probing device PF with driver gpio_sunxi
>> Need to init timer!
>> Probing device riscv_timer with driver riscv_timer
>> Timer is 000000005df34f80
>> Read timer, got 0x17dacfb
>> Timer is 000000005df34f80
>> Read timer, got 0x17f63b8
>> Probing device mmc1-pins with driver pinconfig
>> Probing device mmc@4021000 with driver sunxi_mmc
>> Timer is 000000005df34f80
>> Read timer, got 0x186be7e
>> Timer is 000000005df34f80
>> Read timer, got 0x188752e
>> mmc@4020000: 0, mmc@4021000: 1
>> initcall: 000000004a012970 (relocated to 000000005ff63970)
>> Loading Environment from nowhere... OK
>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>> initcall: 000000004a012912 (relocated to 000000005ff63912)
>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>> initcall: 000000004a01c50c (relocated to 000000005ff6d50c)
>> initcall: 000000004a010938 (relocated to 000000005ff61938)
>> initcall: 000000004a0188bc (relocated to 000000005ff698bc)
>> In:    serial@2500000
>> Out:   serial@2500000
>> Err:   serial@2500000
>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>> initcall: 000000004a0012dc (relocated to 000000005ff522dc)
>> initcall: 000000004a012950 (relocated to 000000005ff63950)
>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>> initcall: 000000004a012934 (relocated to 000000005ff63934)
>> Net:   phy interface8
>> Probing device rgmii-pe-pins with driver pinconfig
>> Probing device ethernet@4500000 with driver eth_sun8i_emac
>> Timer is 000000005df34f80
>> Read timer, got 0x1ae9538
>> Timer is 000000005df34f80
>> Read timer, got 0x1b04bed
>> Timer is 000000005df34f80
>> Read timer, got 0x1b202af
>> Timer is 000000005df34f80
>> Read timer, got 0x1b3b96e
>> Timer is 000000005df34f80
>> Read timer, got 0x1b5702c
>> Timer is 000000005df34f80
>> Read timer, got 0x1b726ec
>>
>> Warning: ethernet@4500000 (eth0) using random MAC address - 62:69:97:68:19:04
>> eth0: ethernet@4500000
>> initcall: 000000004a01292a (relocated to 000000005ff6392a)
>> Hit any key to stop autoboot:  0
> 
> 

Re: Watchdog auto-start and TIMER_EARLY broken in RISC-V U-Boot

[Samuel Holland]

On 9/11/21 1:31 PM, Sean Anderson wrote:
> +CC Heinrich
> 
> Did you ever try booting with WDT on k210?
> 
> On 9/11/21 12:43 PM, Samuel Holland wrote:
>> Hello,
>>
>> I am porting U-Boot to the Allwinner D1 SoC, and I ran into an issue
>> where the board fails to boot if I enable watchdog auto-start.
>>
>> The call to get_timer() -> get_ticks() panics because no timer is
>> available. And since panic_finish() calls udelay(), this causes infinite
>> recursion of trying and failing to get the timer.
>>
>> The issue is that the RISC-V architectural timer driver is bound in
>> arch_early_init_r, which is only called _after_ the first instance of
>> INIT_FUNC_WATCHDOG_RESET in the initcall list.
>>
>> Below is a boot log where I commented out the calls to get_timer() and
>> time_after_eq() in watchdog_reset() in order to illustrate the problem.
>>
>> I modified the log to add the initcall symbol names for clarity.
>>
>>> U-Boot 2021.10-rc2-00278-ge2be1a426d6-dirty (Jan 01 1970 - 00:00:00
>>> +0000)
>>>
>>> DRAM:  512 MiB
>>> initcall: 000000005ff63916
>>> initcall: 000000005ff6391a
>>> initcall: 000000004a012a4e (initr_reloc_global_data)
>>> initcall: 000000004a012a88 (initr_barrier)
>>> initcall: 000000004a012a1c (initr_malloc)
>>> Pre-reloc malloc() used 0x720 bytes (1 KB)
>>> initcall: 000000004a01290e (log_init)
>>> initcall: 000000004a012a94 (initr_bootstage)
>>> initcall: 000000004a012a8c (initr_of_live)
>>> initcall: 000000004a012a06 (initr_dm)
>>> Binding device root_driver to driver root_driver
>>> Probing device root_driver with driver root_driver
>>> Binding device osc24M_clk to driver fixed_clock
>>> Binding device soc to driver simple_bus
>>> Binding device pinctrl@2000000 to driver sunxi-pinctrl
>>> Binding device pinctrl@2000000 to driver gpio_sunxi
>>> Binding device PA to driver gpio_sunxi
>>> Binding device PB to driver gpio_sunxi
>>> Binding device PC to driver gpio_sunxi
>>> Binding device PD to driver gpio_sunxi
>>> Binding device PE to driver gpio_sunxi
>>> Binding device PF to driver gpio_sunxi
>>> Binding device PG to driver gpio_sunxi
>>> Binding device i2c0-pb10-pins to driver pinconfig
>>> Binding device i2c2-pb0-pins to driver pinconfig
>>> Binding device mmc0-pins to driver pinconfig
>>> Binding device mmc1-pins to driver pinconfig
>>> Binding device rgmii-pe-pins to driver pinconfig
>>> Binding device spi0-pins to driver pinconfig
>>> Binding device spi1-pd-pins to driver pinconfig
>>> Binding device uart0-pb8-pins to driver pinconfig
>>> Binding device uart1-pg6-pins to driver pinconfig
>>> Binding device uart1-pg8-rts-cts-pins to driver pinconfig
>>> Binding device i2s2-pb-pins to driver pinconfig
>>> Binding device i2s2-pb3-din-pin to driver pinconfig
>>> Binding device i2s2-pb4-dout-pin to driver pinconfig
>>> Binding device ledc-pc0-pin to driver pinconfig
>>> Binding device pwm0-pd16-pin to driver pinconfig
>>> Binding device pwm2-pd18-pin to driver pinconfig
>>> Binding device pwm7-pd22-pin to driver pinconfig
>>> Binding device spdif-pd22-pin to driver pinconfig
>>> Binding device clock-controller@2001000 to driver sun50i_d1_ccu
>>> Binding device reset to driver sunxi_reset
>>> Binding device serial@2500000 to driver ns16550_serial
>>> Binding device serial@2500400 to driver ns16550_serial
>>> Binding device i2c@2502000 to driver i2c_mvtwsi
>>> Binding device i2c@2502800 to driver i2c_mvtwsi
>>> Binding device mmc@4020000 to driver sunxi_mmc
>>> Binding device mmc@4020000.blk to driver mmc_blk
>>> Binding device mmc@4021000 to driver sunxi_mmc
>>> Binding device mmc@4021000.blk to driver mmc_blk
>>> Binding device usb@4100000 to driver sunxi-musb
>>> Binding device phy@4100400 to driver sun4i_usb_phy
>>> Binding device usb@4101000 to driver ehci_generic
>>> Binding device usb@4101400 to driver ohci_generic
>>> Binding device usb@4200000 to driver ehci_generic
>>> Binding device usb@4200400 to driver ohci_generic
>>> Binding device ethernet@4500000 to driver eth_sun8i_emac
>>> Binding device watchdog@6011000 to driver sunxi_wdt
>>> initcall: 000000004a001914 (board_init)
>>> initcall: 000000004a04d814 (efi_memory_init)
>>> initcall: 000000004a012a9c (initr_binman)
>>> initcall: 000000004a012a90 (initr_dm_devices)
>>
>> Here's where dm_timer_init() would be called if CONFIG_TIMER_EARLY was
>> enabled, but that still doesn't help because the timer is not bound yet.
>> In fact, CONFIG_TIMER_EARLY actually makes the situation worse, because
>> the board hangs if dm_timer_init() fails:
>>
>> initcall sequence 000000005ffdac78 failed at call 000000004a012a06
>> (err=-19)
>> ### ERROR ### Please RESET the board ###
> 
> I'm not sure what's going on here. Why does dm_timer_init get called
> from initr_dm?

dm_timer_init gets called from initr_dm_devices (not initr_dm) if
CONFIG_TIMER_EARLY is enabled.

> Why does it fail?

It fails because at this point there are no devices bound for the timer
uclass, so uclass_first_device_err returns -ENODEV.

Regards,
Samuel

> --Sean
> 
>>> initcall: 000000004a01c4f4 (stdio_init_tables)
>>> initcall: 000000004a039ff8 (serial_initialize)
>>> Probing device pinctrl@2000000 with driver sunxi-pinctrl
>>> Probing device soc with driver simple_bus
>>> Probing device clock-controller@2001000 with driver sun50i_d1_ccu
>>> Probing device osc24M_clk with driver fixed_clock
>>> Probing device uart0-pb8-pins with driver pinconfig
>>> Probing device serial@2500000 with driver ns16550_serial
>>> Probing device reset with driver sunxi_reset
>>> initcall: 000000004a0129ea (initr_announce)
>>> Now running in RAM - U-Boot at: 5ff51000
>>> initcall: 000000004a0367ce (initr_watchdog)
>>> Probing device watchdog@6011000 with driver sunxi_wdt
>>> WDT:   Started with servicing (16s timeout)
>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>
>> Here's where the timer needs to be available for watchdog auto-start to
>> work. An unpatched U-Boot would panic here.
>>
>>> initcall: 000000004a012a84 (arch_initr_trap)
>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>> initcall: 000000004a000ed4 (arch_early_init_r)
>>> Binding device cpus to driver cpu_bus
>>> Binding device cpu@0 to driver riscv_cpu
>>> Binding device riscv_timer to driver riscv_timer
>>
>> But here is where the timer device is actually bound.
>>
>> Do you have any suggestions for how best to fix this?
>>
>> Some things I can think of:
>>   - Add an option to probe the CPU device early in initr_dm_devices()
>>   - Require the board to probe the CPU device from board_init()
>>   - Move arch_early_init_r() earlier in the initcall list
>>   - Skip restarting the watchdog until a timer is available (but this
>>     would not fix CONFIG_TIMER_EARLY)
>>
>> Thanks,
>> Samuel
>>
>>> Probing device cpus with driver cpu_bus
>>> Probing device cpu@0 with driver riscv_cpu
>>> Binding device sbi-sysreset to driver sbi-sysreset
>>> initcall: 000000004a012a98 (power_init_board)
>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>> initcall: 000000004a0129ca (initr_mmc)
>>> MMC:   Probing device mmc0-pins with driver pinconfig
>>> Probing device mmc@4020000 with driver sunxi_mmc
>>> Probing device pinctrl@2000000 with driver gpio_sunxi
>>> Probing device PF with driver gpio_sunxi
>>> Need to init timer!
>>> Probing device riscv_timer with driver riscv_timer
>>> Timer is 000000005df34f80
>>> Read timer, got 0x17dacfb
>>> Timer is 000000005df34f80
>>> Read timer, got 0x17f63b8
>>> Probing device mmc1-pins with driver pinconfig
>>> Probing device mmc@4021000 with driver sunxi_mmc
>>> Timer is 000000005df34f80
>>> Read timer, got 0x186be7e
>>> Timer is 000000005df34f80
>>> Read timer, got 0x188752e
>>> mmc@4020000: 0, mmc@4021000: 1
>>> initcall: 000000004a012970 (relocated to 000000005ff63970)
>>> Loading Environment from nowhere... OK
>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>> initcall: 000000004a012912 (relocated to 000000005ff63912)
>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>> initcall: 000000004a01c50c (relocated to 000000005ff6d50c)
>>> initcall: 000000004a010938 (relocated to 000000005ff61938)
>>> initcall: 000000004a0188bc (relocated to 000000005ff698bc)
>>> In:    serial@2500000
>>> Out:   serial@2500000
>>> Err:   serial@2500000
>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>> initcall: 000000004a0012dc (relocated to 000000005ff522dc)
>>> initcall: 000000004a012950 (relocated to 000000005ff63950)
>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>> initcall: 000000004a012934 (relocated to 000000005ff63934)
>>> Net:   phy interface8
>>> Probing device rgmii-pe-pins with driver pinconfig
>>> Probing device ethernet@4500000 with driver eth_sun8i_emac
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1ae9538
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1b04bed
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1b202af
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1b3b96e
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1b5702c
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1b726ec
>>>
>>> Warning: ethernet@4500000 (eth0) using random MAC address -
>>> 62:69:97:68:19:04
>>> eth0: ethernet@4500000
>>> initcall: 000000004a01292a (relocated to 000000005ff6392a)
>>> Hit any key to stop autoboot:  0
>>
>>
> 

Re: Watchdog auto-start and TIMER_EARLY broken in RISC-V U-Boot

[Sean Anderson]

On 9/11/21 2:37 PM, Samuel Holland wrote:
> On 9/11/21 1:31 PM, Sean Anderson wrote:
>> +CC Heinrich
>>
>> Did you ever try booting with WDT on k210?
>>
>> On 9/11/21 12:43 PM, Samuel Holland wrote:
>>> Hello,
>>>
>>> I am porting U-Boot to the Allwinner D1 SoC, and I ran into an issue
>>> where the board fails to boot if I enable watchdog auto-start.
>>>
>>> The call to get_timer() -> get_ticks() panics because no timer is
>>> available. And since panic_finish() calls udelay(), this causes infinite
>>> recursion of trying and failing to get the timer.
>>>
>>> The issue is that the RISC-V architectural timer driver is bound in
>>> arch_early_init_r, which is only called _after_ the first instance of
>>> INIT_FUNC_WATCHDOG_RESET in the initcall list.
>>>
>>> Below is a boot log where I commented out the calls to get_timer() and
>>> time_after_eq() in watchdog_reset() in order to illustrate the problem.
>>>
>>> I modified the log to add the initcall symbol names for clarity.
>>>
>>>> U-Boot 2021.10-rc2-00278-ge2be1a426d6-dirty (Jan 01 1970 - 00:00:00
>>>> +0000)
>>>>
>>>> DRAM:  512 MiB
>>>> initcall: 000000005ff63916
>>>> initcall: 000000005ff6391a
>>>> initcall: 000000004a012a4e (initr_reloc_global_data)
>>>> initcall: 000000004a012a88 (initr_barrier)
>>>> initcall: 000000004a012a1c (initr_malloc)
>>>> Pre-reloc malloc() used 0x720 bytes (1 KB)
>>>> initcall: 000000004a01290e (log_init)
>>>> initcall: 000000004a012a94 (initr_bootstage)
>>>> initcall: 000000004a012a8c (initr_of_live)
>>>> initcall: 000000004a012a06 (initr_dm)
>>>> Binding device root_driver to driver root_driver
>>>> Probing device root_driver with driver root_driver
>>>> Binding device osc24M_clk to driver fixed_clock
>>>> Binding device soc to driver simple_bus
>>>> Binding device pinctrl@2000000 to driver sunxi-pinctrl
>>>> Binding device pinctrl@2000000 to driver gpio_sunxi
>>>> Binding device PA to driver gpio_sunxi
>>>> Binding device PB to driver gpio_sunxi
>>>> Binding device PC to driver gpio_sunxi
>>>> Binding device PD to driver gpio_sunxi
>>>> Binding device PE to driver gpio_sunxi
>>>> Binding device PF to driver gpio_sunxi
>>>> Binding device PG to driver gpio_sunxi
>>>> Binding device i2c0-pb10-pins to driver pinconfig
>>>> Binding device i2c2-pb0-pins to driver pinconfig
>>>> Binding device mmc0-pins to driver pinconfig
>>>> Binding device mmc1-pins to driver pinconfig
>>>> Binding device rgmii-pe-pins to driver pinconfig
>>>> Binding device spi0-pins to driver pinconfig
>>>> Binding device spi1-pd-pins to driver pinconfig
>>>> Binding device uart0-pb8-pins to driver pinconfig
>>>> Binding device uart1-pg6-pins to driver pinconfig
>>>> Binding device uart1-pg8-rts-cts-pins to driver pinconfig
>>>> Binding device i2s2-pb-pins to driver pinconfig
>>>> Binding device i2s2-pb3-din-pin to driver pinconfig
>>>> Binding device i2s2-pb4-dout-pin to driver pinconfig
>>>> Binding device ledc-pc0-pin to driver pinconfig
>>>> Binding device pwm0-pd16-pin to driver pinconfig
>>>> Binding device pwm2-pd18-pin to driver pinconfig
>>>> Binding device pwm7-pd22-pin to driver pinconfig
>>>> Binding device spdif-pd22-pin to driver pinconfig
>>>> Binding device clock-controller@2001000 to driver sun50i_d1_ccu
>>>> Binding device reset to driver sunxi_reset
>>>> Binding device serial@2500000 to driver ns16550_serial
>>>> Binding device serial@2500400 to driver ns16550_serial
>>>> Binding device i2c@2502000 to driver i2c_mvtwsi
>>>> Binding device i2c@2502800 to driver i2c_mvtwsi
>>>> Binding device mmc@4020000 to driver sunxi_mmc
>>>> Binding device mmc@4020000.blk to driver mmc_blk
>>>> Binding device mmc@4021000 to driver sunxi_mmc
>>>> Binding device mmc@4021000.blk to driver mmc_blk
>>>> Binding device usb@4100000 to driver sunxi-musb
>>>> Binding device phy@4100400 to driver sun4i_usb_phy
>>>> Binding device usb@4101000 to driver ehci_generic
>>>> Binding device usb@4101400 to driver ohci_generic
>>>> Binding device usb@4200000 to driver ehci_generic
>>>> Binding device usb@4200400 to driver ohci_generic
>>>> Binding device ethernet@4500000 to driver eth_sun8i_emac
>>>> Binding device watchdog@6011000 to driver sunxi_wdt
>>>> initcall: 000000004a001914 (board_init)
>>>> initcall: 000000004a04d814 (efi_memory_init)
>>>> initcall: 000000004a012a9c (initr_binman)
>>>> initcall: 000000004a012a90 (initr_dm_devices)
>>>
>>> Here's where dm_timer_init() would be called if CONFIG_TIMER_EARLY was
>>> enabled, but that still doesn't help because the timer is not bound yet.
>>> In fact, CONFIG_TIMER_EARLY actually makes the situation worse, because
>>> the board hangs if dm_timer_init() fails:
>>>
>>> initcall sequence 000000005ffdac78 failed at call 000000004a012a06
>>> (err=-19)
>>> ### ERROR ### Please RESET the board ###
>>
>> I'm not sure what's going on here. Why does dm_timer_init get called
>> from initr_dm?
> 
> dm_timer_init gets called from initr_dm_devices (not initr_dm) if

Ah, 000000004a012a06 earlier referred to initr_dm.

> CONFIG_TIMER_EARLY is enabled.

Hmm. It looks like this came up before and was addressed by 84b2416b6a
("board_r: move initr_watchdog to be called after initr_serial"). That
commit suggests just moving watchdog init later.

Perhaps it would be better to introduce a generic "bind_timer"
initcall?

We could also do something like

diff --git i/lib/time.c w/lib/time.c
index 38a9758292..3fccffc010 100644
--- i/lib/time.c
+++ w/lib/time.c
@@ -86,15 +86,15 @@ uint64_t notrace get_ticks(void)
         int ret;
  
         if (!gd->timer) {
-#ifdef CONFIG_TIMER_EARLY
-               return timer_early_get_count();
-#else
                 int ret;
  
                 ret = dm_timer_init();
-               if (ret)
-                       panic("Could not initialize timer (err %d)\n", ret);
-#endif
+               if (ret) {
+                       if (IS_ENABLED(CONFIG_TIMER_EARLY))
+                               return timer_early_get_count();
+                       else
+                               panic("Could not initialize timer (err %d)\n", ret);
+               }
         }
  
         ret = timer_get_count(gd->timer, &count);
--

which would probably keep this from coming up again.

--Sean

> 
>> Why does it fail?
> 
> It fails because at this point there are no devices bound for the timer
> uclass, so uclass_first_device_err returns -ENODEV.
> 
> Regards,
> Samuel
> 
>> --Sean
>>
>>>> initcall: 000000004a01c4f4 (stdio_init_tables)
>>>> initcall: 000000004a039ff8 (serial_initialize)
>>>> Probing device pinctrl@2000000 with driver sunxi-pinctrl
>>>> Probing device soc with driver simple_bus
>>>> Probing device clock-controller@2001000 with driver sun50i_d1_ccu
>>>> Probing device osc24M_clk with driver fixed_clock
>>>> Probing device uart0-pb8-pins with driver pinconfig
>>>> Probing device serial@2500000 with driver ns16550_serial
>>>> Probing device reset with driver sunxi_reset
>>>> initcall: 000000004a0129ea (initr_announce)
>>>> Now running in RAM - U-Boot at: 5ff51000
>>>> initcall: 000000004a0367ce (initr_watchdog)
>>>> Probing device watchdog@6011000 with driver sunxi_wdt
>>>> WDT:   Started with servicing (16s timeout)
>>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>>
>>> Here's where the timer needs to be available for watchdog auto-start to
>>> work. An unpatched U-Boot would panic here.
>>>
>>>> initcall: 000000004a012a84 (arch_initr_trap)
>>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>>> initcall: 000000004a000ed4 (arch_early_init_r)
>>>> Binding device cpus to driver cpu_bus
>>>> Binding device cpu@0 to driver riscv_cpu
>>>> Binding device riscv_timer to driver riscv_timer
>>>
>>> But here is where the timer device is actually bound.
>>>
>>> Do you have any suggestions for how best to fix this?
>>>
>>> Some things I can think of:
>>>    - Add an option to probe the CPU device early in initr_dm_devices()
>>>    - Require the board to probe the CPU device from board_init()
>>>    - Move arch_early_init_r() earlier in the initcall list
>>>    - Skip restarting the watchdog until a timer is available (but this
>>>      would not fix CONFIG_TIMER_EARLY)
>>>
>>> Thanks,
>>> Samuel
>>>
>>>> Probing device cpus with driver cpu_bus
>>>> Probing device cpu@0 with driver riscv_cpu
>>>> Binding device sbi-sysreset to driver sbi-sysreset
>>>> initcall: 000000004a012a98 (power_init_board)
>>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>>> initcall: 000000004a0129ca (initr_mmc)
>>>> MMC:   Probing device mmc0-pins with driver pinconfig
>>>> Probing device mmc@4020000 with driver sunxi_mmc
>>>> Probing device pinctrl@2000000 with driver gpio_sunxi
>>>> Probing device PF with driver gpio_sunxi
>>>> Need to init timer!
>>>> Probing device riscv_timer with driver riscv_timer
>>>> Timer is 000000005df34f80
>>>> Read timer, got 0x17dacfb
>>>> Timer is 000000005df34f80
>>>> Read timer, got 0x17f63b8
>>>> Probing device mmc1-pins with driver pinconfig
>>>> Probing device mmc@4021000 with driver sunxi_mmc
>>>> Timer is 000000005df34f80
>>>> Read timer, got 0x186be7e
>>>> Timer is 000000005df34f80
>>>> Read timer, got 0x188752e
>>>> mmc@4020000: 0, mmc@4021000: 1
>>>> initcall: 000000004a012970 (relocated to 000000005ff63970)
>>>> Loading Environment from nowhere... OK
>>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>>> initcall: 000000004a012912 (relocated to 000000005ff63912)
>>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>>> initcall: 000000004a01c50c (relocated to 000000005ff6d50c)
>>>> initcall: 000000004a010938 (relocated to 000000005ff61938)
>>>> initcall: 000000004a0188bc (relocated to 000000005ff698bc)
>>>> In:    serial@2500000
>>>> Out:   serial@2500000
>>>> Err:   serial@2500000
>>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>>> initcall: 000000004a0012dc (relocated to 000000005ff522dc)
>>>> initcall: 000000004a012950 (relocated to 000000005ff63950)
>>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>>> initcall: 000000004a012934 (relocated to 000000005ff63934)
>>>> Net:   phy interface8
>>>> Probing device rgmii-pe-pins with driver pinconfig
>>>> Probing device ethernet@4500000 with driver eth_sun8i_emac
>>>> Timer is 000000005df34f80
>>>> Read timer, got 0x1ae9538
>>>> Timer is 000000005df34f80
>>>> Read timer, got 0x1b04bed
>>>> Timer is 000000005df34f80
>>>> Read timer, got 0x1b202af
>>>> Timer is 000000005df34f80
>>>> Read timer, got 0x1b3b96e
>>>> Timer is 000000005df34f80
>>>> Read timer, got 0x1b5702c
>>>> Timer is 000000005df34f80
>>>> Read timer, got 0x1b726ec
>>>>
>>>> Warning: ethernet@4500000 (eth0) using random MAC address -
>>>> 62:69:97:68:19:04
>>>> eth0: ethernet@4500000
>>>> initcall: 000000004a01292a (relocated to 000000005ff6392a)
>>>> Hit any key to stop autoboot:  0
>>>
>>>
>>
>