Re: [RFC PATCH 00/16] arm: Add Rockchip RK3588 support

[Jagan Teki <jagan@edgeble.ai>]

On Fri, 27 Jan 2023 at 00:33, Jonas Karlman <jonas@kwiboo.se> wrote:
>
> On 2023-01-26 19:26, Jagan Teki wrote:
> > Hi Simon,
> >
> > On Thu, 26 Jan 2023 at 23:34, Simon Glass <sjg@chromium.org> wrote:
> >>
> >> Hi Jagan,
> >>
> >> On Thu, 26 Jan 2023 at 10:42, Jagan Teki <jagan@edgeble.ai> wrote:
> >>>
> >>> On Thu, 26 Jan 2023 at 22:28, Jonas Karlman <jonas@kwiboo.se> wrote:
> >>>>
> >>>> Hi Jagan,
> >>>> On 2023-01-26 17:51, Jagan Teki wrote:
> >>>>> Hi Jonas,
> >>>>>
> >>>>> On Thu, 26 Jan 2023 at 04:17, Jonas Karlman <jonas@kwiboo.se> wrote:
> >>>>>>
> >>>>>> Hi Jagan,
> >>>>>>
> >>>>>> On 2023-01-25 23:27, Jagan Teki wrote:
> >>>>>>> This series support Rockchip RK3588. All the device tree files are
> >>>>>>> synced from linux-next with the proper SHA1 mentioned in the commit
> >>>>>>> messages.
> >>>>>>>
> >>>>>>> Unfortunately, the BL31 from rkbin is not compatible with U-Boot so
> >>>>>>> it is failing to load ATF entry from SPL and hang.
> >>>>>>>
> >>>>>>> Verified below BL31 versions,
> >>>>>>>   bl31-v1.15
> >>>>>>>   bl31-v1.21
> >>>>>>>   bl31-v1.22
> >>>>>>>   bl31-v1.23
> >>>>>>>   bl31-v1.24
> >>>>>>>   bl31-v1.25
> >>>>>>>   bl31-v1.26
> >>>>>>>
> >>>>>>> Rever-engineered with respect to rockchip u-boot by using the same
> >>>>>>> FIT_GENERATOR being used in Mainline, rockchip u-boot is booting but
> >>>>>>> mainline showing the same issue.
> >>>>>>>
> >>>>>>> Log:
> >>>>>>>
> >>>>>>> LPDDR4X, 2112MHz01-00642-g6bdfd31756-dirty (Jan 26 2023 ���3:44:34 +0530)
> >>>>>>> channel[0] BW=16 Col=10 Bk=8 CS0 Row=17 CS1 Row=17 CS=2 Die BW=8 Size=4096MB
> >>>>>>> channel[1] BW=16 Col=10 Bk=8 CS0 Row=17 CS1 Row=17 CS=2 Die BW=8 Size=4096MB
> >>>>>>> channel[2] BW=16 Col=10 Bk=8 CS0 Row=17 CS1 Row=17 CS=2 Die BW=8 Size=4096MB
> >>>>>>> channel[3] BW=16 Col=10 Bk=8 CS0 Row=17 CS1 Row=17 CS=2 Die BW=8 Size=4096MB
> >>>>>>> change to F1: 528MHz
> >>>>>>> change to F2: 1068MHz
> >>>>>>> change to F3: 1560MHz
> >>>>>>> change to F0: 2112MHz
> >>>>>>> out
> >>>>>>>
> >>>>>>> U-Boot SPL 2023.01-00642-g6bdfd31756-dirty (Jan 26 2023 - 03:44:34 +0530)
> >>>>>>> Trying to boot from MMC1
> >>>>>>> bl31_entry: atf_entry start
> >>>>>>> << hang >>
> >>>>>>>
> >>>>>>> Any information on BL31 for RK3588 please share.
> >>>>>>
> >>>>>> I had a similar strange booing issue with RK3568 and mainline U-Boot,
> >>>>>> turned out to be related to all parts of ATF not being properly loaded
> >>>>>> into PMU SRAM.
> >>>>>>
> >>>>>> Using my series at [1] I managed to get ATF to be fully loaded into
> >>>>>> PMU SRAM. Using CONFIG_SPL_FIT_SIGNATURE=y helped me finding out that
> >>>>>> the segment being loaded ended up corrupted.
> >>>>>>
> >>>>>> The use of 512 bytes alignment of the FIT helped mitigate that issue.
> >>>>>> Vendor U-Boot use a bounce buffer for all parts that is written into
> >>>>>> SRAM (anything loaded outside the gd->ram_base to gd->ram_top range).
> >>>>>>
> >>>>>> You can also find newer bl31 at [2], up to version v1.32.
> >>>>>>
> >>>>>> [1] https://patchwork.ozlabs.org/project/uboot/list/?series=337891 [2] https://gitlab.com/rk3588_linux/rk/rkbin/-/tree/linux-5.10-gen-rkr3.5/bin/rk35>>>>>> Thanks for the details. I did apply this set on the master. No change
> >>>>> in the behavior, used BL31 and ddr from [2] as well as in
> >>>>> rkbin/master.
> >>>>
> >>>> I did some tests on my Radxa ROCK 3 Model B with CONFIG_SPL_FIT_SIGNATURE=y
> >>>> and it looked like it failed to read data into memory, see below.
> >>>>
> >>>> It also looks like the sdhci compatible is not supported by the driver.
> >>>> Something that may need to be added to driver to properly read data?
> >>>>
> >>>>
> >>>> DDR V1.09 a930779e06 typ 22/11/21-17:50:56
> >>>> LPDDR4X, 2112MHz
> >>>> channel[0] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB
> >>>> channel[1] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB
> >>>> channel[2] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB
> >>>> channel[3] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB
> >>>> Manufacturer ID:0x6
> >>>> CH0 RX Vref:31.7%, TX Vref:21.8%,20.8%
> >>>> CH1 RX Vref:31.7%, TX Vref:21.8%,21.8%
> >>>> CH2 RX Vref:32.7%, TX Vref:22.8%,21.8%
> >>>> CH3 RX Vref:32.7%, TX Vref:21.8%,20.8%
> >>>> change to F1: 528MHz
> >>>> change to F2: 1068MHz
> >>>> change to F3: 1560MHz
> >>>> change to F0: 2112MHz
> >>>> out
> >>>>
> >>>> U-Boot SPL 2023.01 (Jan 26 2023 - 00:24:53 +0000)
> >>>> Trying to boot from MMC1
> >>>> ## Checking hash(es) for config config_1 ... OK
> >>>> ## Checking hash(es) for Image atf_1 ... sha256 error!
> >>>> Bad hash value for 'hash' hash node in 'atf_1' image node
> >>>> mmc_load_image_raw_sector: mmc block read error
> >>>> Trying to boot from MMC1
> >>>> ## Checking hash(es) for config config_1 ... OK
> >>>> ## Checking hash(es) for Image atf_1 ... sha256 error!
> >>>> Bad hash value for 'hash' hash node in 'atf_1' image node
> >>>
> >>> Look like this is something wrong with your patch series or master
> >>> changes on binman, not with the driver. I have observed the same if I
> >>> enable CONFIG_SPL_FIT_SIGNATURE.
> >>
> >> There are some more changes in dm/master that I'm about to pull in.
> >> One of them from Jonas Karlman adds hash nodes so may be involved.
> >
> > I found the same issue on the dm/master
> >
> > U-Boot SPL 2023.01-00176-gb21fb7a9c0 (Jan 26 2023 - 23:55:11 +0530)
> > Trying to boot from MMC1
> > ## Checking hash(es) for config config-1 ... OK
> > ## Checking hash(es) for Image atf-1 ... sha256 error!
> > Bad hash value for 'hash' hash node in 'atf-1' image node
> > mmc_load_image_raw_sector: mmc block read error
> > SPL: failed to boot from all boot devices
> > ### ERROR ### Please RESET the board ###
>
> On my RK3568 ROCK 3A board this is working correctly, see below.
>
> This was using u-boot master 17e8e58fe62c019b2cc26af221b6defc3368229f
> with a few patches on top, see [1].
>
> -------------
> U-Boot SPL 2023.01 (Jan 26 2023 - 18:29:56 +0000)
> Trying to boot from MMC1
> ## Checking hash(es) for config config-1 ... OK
> ## Checking hash(es) for Image atf-1 ... sha256+ OK
> ## Checking hash(es) for Image u-boot ... sha256+ OK
> ## Checking hash(es) for Image fdt-1 ... sha256+ OK
> ## Checking hash(es) for Image atf-2 ... sha256+ OK
> ## Checking hash(es) for Image atf-3 ... sha256+ OK
> ## Checking hash(es) for Image atf-4 ... sha256+ OK
> ## Checking hash(es) for Image atf-5 ... sha256+ OK
> ## Checking hash(es) for Image atf-6 ... sha256+ OK
> INFO:    Preloader serial: 2
> NOTICE:  BL31: v2.3():v2.3-460-g2c8be93f9:huan.he
> NOTICE:  BL31: Built : 19:45:17, Nov  8 2022
> INFO:    GICv3 without legacy support detected.
> INFO:    ARM GICv3 driver initialized in EL3
> INFO:    pmu v1 is valid 220114
> INFO:    dfs DDR fsp_param[0].freq_mhz= 1560MHz
> INFO:    dfs DDR fsp_param[1].freq_mhz= 324MHz
> INFO:    dfs DDR fsp_param[2].freq_mhz= 528MHz
> INFO:    dfs DDR fsp_param[3].freq_mhz= 780MHz
> INFO:    Using opteed sec cpu_context!
> INFO:    boot cpu mask: 0
> INFO:    BL31: Initializing runtime services
> WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
> ERROR:   Error initializing runtime service opteed_fast
> INFO:    BL31: Preparing for EL3 exit to normal world
> INFO:    Entry point address = 0xa00000
> INFO:    SPSR = 0x3c9
>
>
> U-Boot 2023.01 (Jan 26 2023 - 18:29:56 +0000)
>
> Model: Rockchip RK3568 EVB1 DDR4 V10 Board
> DRAM:  8 GiB (effective 7.7 GiB)
> Core:  58 devices, 17 uclasses, devicetree: separate
> MMC:   mmc@fe2b0000: 1, mmc@fe310000: 0
> ...
> -------------
>
> However, on my RK3588 ROCK 5B device I get a similar error you get.
> Seems to be the mmc reading that times out all of sudden.
>
> It can read the FIP and config, and then some mmc command/transfer times out.
> See below for details with LOG_DEBUG defined at top of include/log.h
>
> This was trying to boot from a SD-card, trying to use eMMC fails earlier.
> Because there is no driver for emmc compatible, SPL only tries to boot from SD-card.
>
> I believe this is related to your << hang >>, reading atf
> from mmc fails in the background and there is no error message.
>
> That is why I suggested trying with CONFIG_SPL_FIT_SIGNATURE=y,
> to see if there is any hidden issue trying to load ATF.
>
> -------------
> Trying to boot from MMC1
> ## Checking hash(es) for config config-1 ... OK
> mmc_load_image_raw_sector: mmc block read error
> Trying to boot from MMC1
> mmc_load_image_raw_sector: mmc block read error
> SPL: failed to boot from all boot devices
> ### ERROR ### Please RESET the board ###
> -------------
> Trying to boot from MMC1
> 0
>    - 0 'mmc@fe2c0000'
>    - found
> blk_find_device: uclass_id=67, devnum=0: mmc@fe2c0000.blk, 67, 0
> clock is disabled (0Hz)
> Buswidth = 0, clock: 0
> Buswidth = 1, clock: 0
> clock is enabled (400000Hz)
> Buswidth = 1, clock: 400000
> Sending CMD0
> Sending CMD8
> Sending CMD55
> Sending CMD41
> Sending CMD55
> Sending CMD41
> Sending CMD2
> Sending CMD3
> Sending CMD9
> Sending CMD7
> Sending CMD55
> Sending CMD51
> Sending CMD6
> Sending CMD55
> Sending CMD6
> Buswidth = 4, clock: 400000
> Sending CMD6
> clock is enabled (50000000Hz)
> Buswidth = 4, clock: 50000000
> clk_set_rate(clk=500000, rate=50000000)
> rockchip_dwmmc_get_mmc_clk: err=-2
> spl: mmc boot mode: raw
> blk_find_device: uclass_id=67, devnum=0: mmc@fe2c0000.blk, 67, 0
> Sending CMD16
> Sending CMD17
> hdr read sector 4000, count=1
> Found FIT
> size=a00, ptr=ac0, limit=100000: aligned to 5000c0
> blk_find_device: uclass_id=67, devnum=0: mmc@fe2c0000.blk, 67, 0
> Sending CMD16
> Sending CMD18
> Sending CMD12
> fit read sector 4000, sectors=5, dst=5000c0, count=5, size=0xa00
> Selecting default config 'rk3588-evb.dtb'
> ## Checking hash(es) for config config-1 ... fit_config_verify_required_keys: No signature node found: FDT_ERR_NOTFOUND
> OK
> firmware: 'atf-1'
> blk_find_device: uclass_id=67, devnum=0: mmc@fe2c0000.blk, 67, 0
> Sending CMD16
> Sending CMD18
> dwmci_data_transfer: Timeout waiting for data!
> mmc_load_image_raw_sector: mmc block read error
> spl: mmc boot mode: fs
> Trying to boot from MMC1
> 0
>    - 0 'mmc@fe2c0000'
>    - found
> blk_find_device: uclass_id=67, devnum=0: mmc@fe2c0000.blk, 67, 0
> spl: mmc boot mode: raw
> blk_find_device: uclass_id=67, devnum=0: mmc@fe2c0000.blk, 67, 0
> Sending CMD16
> dwmci_send_cmd: Response Timeout.
> hdr read sector 4000, count=0
> mmc_load_image_raw_sector: mmc block read error
> spl: mmc boot mode: fs
> SPL: failed to boot from all boot devices
> ### ERROR ### Please RESET the board ###

I'm thinking this is fit/binman issue rather than mmc.

U-Boot SPL 2023.01-00176-gb21fb7a9c0-dirty (Jan 27 2023 - 00:43:52 +0530)
SPL malloc() before relocation used 0xbe0 bytes (2 KB)
>>SPL: board_init_r()
spl_init
size=a0, ptr=a0, limit=100000: 500000
size=8, ptr=a8, limit=100000: 5000a0
fdtdec_get_addr_size_auto_parent: na=2, ns=2,
fdtdec_get_addr_size_fixed: reg: addr=00000000x
ofnode_read_u32_index: bus-width: x (4)
ofnode_read_bool: non-removable: false
ofnode_read_u32_index: fifo-depth: x (256)
ofnode_read_bool: fifo-mode: false
ofnode_read_bool: u-boot,spl-fifo-mode: false
ofnode_read_u32_array: clock-freq-min-max: fdtdec_get_int_array:
clock-freq-min-max
get_prop_check_min_len: clock-freq-min-max
ofnode_read_u32_index: max-frequency: x (200000000)
clk_set_defaults(mmc@fe2c0000)
clk_set_default_parents: could not read assigned-clock-parents for 3f60700
ofnode_read_prop: assigned-clock-rates: <not found>
fdtdec_get_int: #clock-cells: x (1)
fdtdec_get_int: #clock-cells: x (1)
Looking for clock-controller@fd7c0000
Looking for clock-controller@fd7c0000
      - checking clock-1
      - checking clock-controller@fd7c0000
   - result for clock-controller@fd7c0000: clock-controller@fd7c0000 (ret=0)
   - result for clock-controller@fd7c0000: clock-controller@fd7c0000 (ret=0)
clk_of_xlate_default(clk=500000)
clk_request(dev=3f60320, clk=500000)
clk_set_rate(clk=500000, rate=400000)
rockchip_dwmmc_get_mmc_clk: err=-2
Trying to boot from MMC1
0
   - 0 'mmc@fe2c0000'
   - found
blk_find_device: uclass_id=67, devnum=0: mmc@fe2c0000.blk, 67, 0
clock is disabled (0Hz)
Buswidth = 0, clock: 0
Buswidth = 1, clock: 0
clock is enabled (400000Hz)
Buswidth = 1, clock: 400000
Sending CMD0
Sending CMD8
Sending CMD55
Sending CMD41
Sending CMD55
Sending CMD41
Sending CMD55
Sending CMD41
Sending CMD2
Sending CMD3
Sending CMD9
Sending CMD7
Sending CMD55
Unaligned buffer length u
size=40, ptr=100, limit=100000: aligned to 5000c0
Sending CMD51
Sending CMD6
Sending CMD55
Sending CMD6
Buswidth = 4, clock: 400000
Sending CMD6
clock is enabled (50000000Hz)
Buswidth = 4, clock: 50000000
clk_set_rate(clk=500000, rate=50000000)
rockchip_dwmmc_get_mmc_clk: err=-2
spl: mmc boot mode: raw
blk_find_device: uclass_id=67, devnum=0: mmc@fe2c0000.blk, 67, 0
Sending CMD16
Sending CMD17
hdr read sector 4000, count=1
Found FIT
size=a00, ptr=b00, limit=100000: aligned to 500100
blk_find_device: uclass_id=67, devnum=0: mmc@fe2c0000.blk, 67, 0
Sending CMD16
Sending CMD18
Sending CMD12
fit read sector 4000, sectors=5, dst=500100, count=5, size=0xa00
Selecting default config 'rk3588-edgeble-neu6a-io.dtb'
## Checking hash(es) for config config-1 ...
fit_config_verify_required_keys: No signature node found:
FDT_ERR_NOTFOUND
OK
firmware: 'atf-1'
blk_find_device: uclass_id=67, devnum=0: mmc@fe2c0000.blk, 67, 0
Sending CMD16
Sending CMD18
dwmci_data_transfer: Timeout waiting for data!
Sending CMD12
External data: dst=40000, offset=ad200, size=2e940
## Checking hash(es) for Image atf-1 ...
fit_image_verify_required_sigs: No signature node found:
FDT_ERR_NOTFOUND
sha256 error!
Bad hash value for 'hash' hash node in 'atf-1' image node
mmc_load_image_raw_sector: mmc block read error
spl: mmc boot mode: fs
SPL: failed to boot from all boot devices
### ERROR ### Please RESET the board ###

Please check this repo, it is on top of the dm/master
https://github.com/edgeble/u-boot/tree/neu6-v1

Jagan.