Re: [PATCH RFC u-boot-mvebu 0/2] arm: mvebu: Fix eMMC boot

[Martin Rowe <martin.p.rowe@gmail.com>]

On Wed, 22 Mar 2023 at 12:38, Martin Rowe <martin.p.rowe@gmail.com> wrote:
>
> On Tue, 21 Mar 2023 at 08:08, Pali Rohár <pali@kernel.org> wrote:
>>
>> On Tuesday 21 March 2023 08:01:16 Martin Rowe wrote:
>> > On Mon, 20 Mar 2023 at 17:33, Pali Rohár <pali@kernel.org> wrote:
>> >
>> > > On Monday 20 March 2023 11:48:59 Martin Rowe wrote:
>> > > > On Sun, 19 Mar 2023 at 16:22, Pali Rohár <pali@kernel.org> wrote:
>> > > >
>> > > > > On Sunday 19 March 2023 00:32:01 Martin Rowe wrote:
>> > > > > > On Mon, 6 Mar 2023 at 11:53, Pali Rohár <pali@kernel.org> wrote:
>> > > > > >
>> > > > > > > Could you try to print mmc->part_config (ideally as early as
>> > > possible)?
>> > > > > > >
>> > > > > >
>> > > > > > In SPL mmc->part_config is 255
>> > > > > > In main u-boot at the start of clearfog.c board_init()
>> > > mmc->part_config
>> > > > > is
>> > > > > > 255
>> > > > > > In main u-boot at the start of clearfog.c checkboard()
>> > > mmc->part_config
>> > > > > is
>> > > > > > 8 (ack: 0, partition_enable: 1, access: 0)
>> > > > >
>> > > > > 255 is uninitialized value.
>> > > > >
>> > > > > > If I set partition_enable to 2, I get the same result except the
>> > > value is
>> > > > > > 16  (ack: 0, partition_enable: 2, access: 0) instead of 8 for the
>> > > last
>> > > > > value
>> > > > >
>> > > > > Try to change "access" bits.
>> > > > >
>> > > > > > <partition_enable 1>
>> > > > > > BootROM - 1.73
>> > > > > >
>> > > > > > Booting from MMC
>> > > > > >
>> > > > > > U-Boot SPL 2023.04-rc3-00159-gd1653548d2-dirty (Mar 19 2023 -
>> > > 10:05:32
>> > > > > > +1000)
>> > > > > > High speed PHY - Version: 2.0
>> > > > > > EEPROM TLV detection failed: Using static config for Clearfog Pro.
>> > > > > > Detected Device ID 6828
>> > > > > > board SerDes lanes topology details:
>> > > > > >  | Lane # | Speed |  Type       |
>> > > > > >  --------------------------------
>> > > > > >  |   0    |   3   | SATA0       |
>> > > > > >  |   1    |   0   | SGMII1      |
>> > > > > >  |   2    |   5   | PCIe1       |
>> > > > > >  |   3    |   5   | USB3 HOST1  |
>> > > > > >  |   4    |   5   | PCIe2       |
>> > > > > >  |   5    |   0   | SGMII2      |
>> > > > > >  --------------------------------
>> > > > > > High speed PHY - Ended Successfully
>> > > > > > mv_ddr: 14.0.0
>> > > > > > DDR3 Training Sequence - Switching XBAR Window to FastPath Window
>> > > > > > mv_ddr: completed successfully
>> > > > > > spl.c spl_boot_device part_config = 255
>> > > > > > Trying to boot from MMC1
>> > > > > >
>> > > > > >
>> > > > > > U-Boot 2023.04-rc3-00159-gd1653548d2-dirty (Mar 19 2023 - 10:05:32
>> > > +1000)
>> > > > > >
>> > > > > > SoC:   MV88F6828-A0 at 1600 MHz
>> > > > > > DRAM:  1 GiB (800 MHz, 32-bit, ECC not enabled)
>> > > > > > clearfog.c board_init part_config = 255
>> > > > > > Core:  38 devices, 22 uclasses, devicetree: separate
>> > > > > > MMC:   mv_sdh: 0
>> > > > > > Loading Environment from MMC... *** Warning - bad CRC, using default
>> > > > > > environment
>> > > > > >
>> > > > > > Model: SolidRun Clearfog A1
>> > > > > > clearfog.c checkboard part_config = 8
>> > > > > > Board: SolidRun Clearfog Pro
>> > > > > > Net:
>> > > > > > Warning: ethernet@70000 (eth1) using random MAC address -
>> > > > > 32:16:0e:b4:d1:d8
>> > > > > > eth1: ethernet@70000
>> > > > > > Warning: ethernet@30000 (eth2) using random MAC address -
>> > > > > 72:30:3f:79:07:12
>> > > > > > , eth2: ethernet@30000
>> > > > > > Warning: ethernet@34000 (eth3) using random MAC address -
>> > > > > 82:fb:71:23:46:4f
>> > > > > > , eth3: ethernet@34000
>> > > > > > Hit any key to stop autoboot:  0
>> > > > > > => mmc partconf 0
>> > > > > > EXT_CSD[179], PARTITION_CONFIG:
>> > > > > > BOOT_ACK: 0x0
>> > > > > > BOOT_PARTITION_ENABLE: 0x1
>> > > > > > PARTITION_ACCESS: 0x0
>> > > > > > </partition_enable 1>
>> > > > > >
>> > > > > > <partition_enable 2>
>> > > > > > BootROM - 1.73
>> > > > > >
>> > > > > > Booting from MMC
>> > > > > >
>> > > > > > U-Boot SPL 2023.04-rc3-00159-gd1653548d2-dirty (Mar 19 2023 -
>> > > 10:05:32
>> > > > > > +1000)
>> > > > > > High speed PHY - Version: 2.0
>> > > > > > EEPROM TLV detection failed: Using static config for Clearfog Pro.
>> > > > > > Detected Device ID 6828
>> > > > > > board SerDes lanes topology details:
>> > > > > >  | Lane # | Speed |  Type       |
>> > > > > >  --------------------------------
>> > > > > >  |   0    |   3   | SATA0       |
>> > > > > >  |   1    |   0   | SGMII1      |
>> > > > > >  |   2    |   5   | PCIe1       |
>> > > > > >  |   3    |   5   | USB3 HOST1  |
>> > > > > >  |   4    |   5   | PCIe2       |
>> > > > > >  |   5    |   0   | SGMII2      |
>> > > > > >  --------------------------------
>> > > > > > High speed PHY - Ended Successfully
>> > > > > > mv_ddr: 14.0.0
>> > > > > > DDR3 Training Sequence - Switching XBAR Window to FastPath Window
>> > > > > > mv_ddr: completed successfully
>> > > > > > spl.c spl_boot_device part_config = 255
>> > > > > > Trying to boot from MMC1
>> > > > > >
>> > > > > >
>> > > > > > U-Boot 2023.04-rc3-00159-gd1653548d2-dirty (Mar 19 2023 - 10:05:32
>> > > +1000)
>> > > > > >
>> > > > > > SoC:   MV88F6828-A0 at 1600 MHz
>> > > > > > DRAM:  1 GiB (800 MHz, 32-bit, ECC not enabled)
>> > > > > > clearfog.c board_init part_config = 255
>> > > > > > Core:  38 devices, 22 uclasses, devicetree: separate
>> > > > > > MMC:   mv_sdh: 0
>> > > > > > Loading Environment from MMC... *** Warning - bad CRC, using default
>> > > > > > environment
>> > > > > >
>> > > > > > Model: SolidRun Clearfog A1
>> > > > > > clearfog.c checkboard part_config = 16
>> > > > > > Board: SolidRun Clearfog Pro
>> > > > > > Net:
>> > > > > > Warning: ethernet@70000 (eth1) using random MAC address -
>> > > > > 92:5a:fc:14:e8:f6
>> > > > > > eth1: ethernet@70000
>> > > > > > Warning: ethernet@30000 (eth2) using random MAC address -
>> > > > > 42:9c:d8:3a:cb:b2
>> > > > > > , eth2: ethernet@30000
>> > > > > > Warning: ethernet@34000 (eth3) using random MAC address -
>> > > > > c6:99:20:f4:02:a0
>> > > > > > , eth3: ethernet@34000
>> > > > > > Hit any key to stop autoboot:  0
>> > > > > > => mmc partconf 0
>> > > > > > EXT_CSD[179], PARTITION_CONFIG:
>> > > > > > BOOT_ACK: 0x0
>> > > > > > BOOT_PARTITION_ENABLE: 0x2
>> > > > > > PARTITION_ACCESS: 0x0
>> > > > > > </partition_enable 2>
>> > > > >
>> > > > > Are both logs from the configuration when SPL+u-boot is stored on
>> > > Boot0?
>> > > > > Could you try to erase Boot0 and store SPL+u-boot to Boot1? I'm
>> > > > > interested to see if "access" bits are changed in SPL (before loading
>> > > > > main u-boot).
>> > > > >
>> > > > > > I'm having trouble trying to find the hooks which run between
>> > > board_init
>> > > > > > and checkboard. If you can point me in the right direction I'm happy
>> > > to
>> > > > > > re-run and try to narrow down where the valid values are being set
>> > > from.
>> > > > >
>> > > > > Print it directly in drivers/mmc/mmc.c mmc_startup_v4() where
>> > > > > mmc->part_config = is set from ext_csd[EXT_CSD_PART_CONF] register.
>> > > > > I want to see original value from EXT_CSD_PART_CONF.
>> > > > >
>> > > > > I do not know which hook is the best, so printing it from mmc.c driver
>> > > > > should work better.
>> > > > >
>> > > >
>> > > > u-boot in boot0, partconf set to 0x1:
>> > > > mmc->part_config = 8
>> > > >
>> > > > u-boot in boot0, partconf set to 0x2:
>> > > > mmc->part_config = 16
>> > > >
>> > > > u-boot in boot1 (boot0 zeroed), partconf set to 0x1:
>> > > > mmc->part_config = 8
>> > > >
>> > > > u-boot in boot1 (boot0 zeroed), partconf set to 0x2:
>> > > > mmc->part_config = 16
>> > >
>> > > Ah, that does not look useful :-(
>> > >
>> > > Just to confirm, is this output from SPL or from main U-Boot?
>> > >
>> >
>> > Definitely SPL. I triple checked because I was also disappointed with those
>> > results. With BootROM hardcoded with its boot order it seems like neither
>> > CONFIG_SYS_MMCSD_RAW_MODE_EMMC_BOOT_PARTITION nor relying on
>> > mmc->part_config is going to work well.
>>
>> In emmc spec is written:
>>
>> Each time the host wants to access a partition the following flow shall be executed:
>> 1. Set PARTITION_ACCESS bits in the PARTITION_CONFIG field of the Extended CSD register in order to address one of the partitions
>> 2. Issue commands referred to the selected partition
>> 3. Restore default access to the User Data Area or re-direction the access to another partition
>> All the reset events (CMD0 or hardware reset) will restore the access by default to the User Data Area.
>>
>> I'm feeling that partition_access bits should be preserved between
>> reading data from boot0 and starting SPL. And these bits somehow could
>> be used to determinate from which source bootrom loaded SPL. Maybe the
>> last point ("all the reset events...") applies there and u-boot mmc
>> driver does some reset in its init phase? And need to figure
>> out how to read PARTITION_ACCESS without u-boot's mmc driver?
>
>
> I enabled MMC tracing and added some printfs in mmc.c functions to see if we can get a better idea of where best to read the value from:
>
> <output>
> BootROM - 1.73
>
> Booting from MMC
>
> U-Boot SPL 2023.04-rc4-00342-g7e562609bb-dirty (Mar 22 2023 - 22:14:28 +1000)
> High speed PHY - Version: 2.0
> EEPROM TLV detection failed: Using static config for Clearfog Pro.
> Detected Device ID 6828
> board SerDes lanes topology details:
>  | Lane # | Speed |  Type       |
>  --------------------------------
>  |   0    |   3   | SATA0 |
>  |   1    |   0   | SGMII1 |
>  |   2    |   5   | PCIe1 |
>  |   3    |   5   | USB3 HOST1 |
>  |   4    |   5   | PCIe2 |
>  |   5    |   0   | SGMII2 |
>  --------------------------------
> High speed PHY - Ended Successfully
> mv_ddr: 14.0.0
> DDR3 Training Sequence - Switching XBAR Window to FastPath Window
> mv_ddr: completed successfully
> Trying to boot from MMC1
> ===mmc_start_init start===
> ===Getting ext_csd===
> ===mmc->ext_csd[EXT_CSD_PART_CONF] = 0
> ===mmc_power_on===
> ===mmc->ext_csd[EXT_CSD_PART_CONF] = 0
> ===mmc_select_mode===
> ===mmc->ext_csd[EXT_CSD_PART_CONF] = 0
> ===mmc_mode2freq===
> ===mmc->ext_csd[EXT_CSD_PART_CONF] = 0
> ===mmc_set_initial_state===
> ===mmc->ext_csd[EXT_CSD_PART_CONF] = 0
> CMD_SEND:0
> ARG 0x00000000
> MMC_RSP_NONE
> CMD_SEND:8
> ARG 0x000001aa
> RET -110
> CMD_SEND:55
> ARG 0x00000000
> RET -110
> CMD_SEND:0
> ARG 0x00000000
> MMC_RSP_NONE
> CMD_SEND:1
> ARG 0x00000000
> MMC_RSP_R3,4 0x40ff8080
> CMD_SEND:1
> ARG 0x40300080
> MMC_RSP_R3,4 0x40ff8080
> CMD_SEND:1
> ARG 0x40300080
> MMC_RSP_R3,4 0xc0ff8080
> ===mmc_start_init end===
> ===mmc->ext_csd[EXT_CSD_PART_CONF] = 0
> CMD_SEND:2
> ARG 0x00000000
> MMC_RSP_R2 0x15010038
>           0x474d4534
>           0x52010418
>           0xfc4f7300
>
> DUMPING DATA
> 000 - 15 01 00 38
> 004 - 47 4d 45 34
> 008 - 52 01 04 18
> 012 - fc 4f 73 00
> CMD_SEND:3
> ARG 0x00010000
> MMC_RSP_R1,5,6,7 0x00000500
> CMD_SEND:9
> ARG 0x00010000
> MMC_RSP_R2 0xd0270132
>           0x0f5903ff
>           0xf6dbffef
>           0x8e404000
>
> DUMPING DATA
> 000 - d0 27 01 32
> 004 - 0f 59 03 ff
> 008 - f6 db ff ef
> 012 - 8e 40 40 00
> ===mmc_select_mode===
> ===mmc->ext_csd[EXT_CSD_PART_CONF] = 0
> ===mmc_mode2freq===
> ===mmc->ext_csd[EXT_CSD_PART_CONF] = 0
> CMD_SEND:7
> ARG 0x00010000
> MMC_RSP_R1,5,6,7 0x00000700
> CMD_SEND:8
> ARG 0x00000000
> MMC_RSP_R1,5,6,7 0x00000900
> ===mmc_startup_v4===
> ===mmc->ext_csd[EXT_CSD_PART_CONF] = 8
> <snip>
> </output>
>

You are correct Pali, it is preserved :)

The first time I can get the value is at the end of mmc_set_initial_state using:

static void mmc_set_initial_state(struct mmc *mmc)
{
   printf("+mmc_set_initial_state\n");
   int err;

   /* First try to set 3.3V. If it fails set to 1.8V */
   err = mmc_set_signal_voltage(mmc, MMC_SIGNAL_VOLTAGE_330);
   if (err != 0)
      err = mmc_set_signal_voltage(mmc, MMC_SIGNAL_VOLTAGE_180);
   if (err != 0)
      pr_warn("mmc: failed to set signal voltage\n");

   mmc_select_mode(mmc, MMC_LEGACY);
   mmc_set_bus_width(mmc, 1);
   mmc_set_clock(mmc, 0, MMC_CLK_ENABLE);
   ALLOC_CACHE_ALIGN_BUFFER(u8, ext_csd, MMC_MAX_BLOCK_LEN);
   err = mmc_send_ext_csd(mmc, ext_csd);
   if (!err) {
      printf("++mmc_set_initial_state ext_csd[EXT_CSD_PART_CONF] =
%d\n", ext_csd[EXT_CSD_PART_CONF]);
   }
}

I set mmc partconf 0 0 0 0, zeroed the first boot area and loaded
u-boot in the second. I had a few extra attempts to call
mmc_send_ext_csd in earlier functions that timeout and a lot of extra
printfs:

<output>
BootROM - 1.73

Booting from MMC
BootROM: Bad header at offset 00000000
BootROM: Bad header at offset 00200000
Switching BootPartitions.

U-Boot SPL 2023.04-rc4-00342-g7e562609bb-dirty (Mar 22 2023 - 23:27:20 +1000)
High speed PHY - Version: 2.0
EEPROM TLV detection failed: Using static config for Clearfog Pro.
Detected Device ID 6828
board SerDes lanes topology details:
 | Lane # | Speed |  Type       |
 --------------------------------
 |   0    |   3   | SATA0    |
 |   1    |   0   | SGMII1    |
 |   2    |   5   | PCIe1    |
 |   3    |   5   | USB3 HOST1    |
 |   4    |   5   | PCIe2    |
 |   5    |   0   | SGMII2    |
 --------------------------------
High speed PHY - Ended Successfully
mv_ddr: 14.0.0
DDR3 Training Sequence - Switching XBAR Window to FastPath Window
mv_ddr: completed successfully
Trying to boot from MMC1
+mmc_power_init
+mmc_power_cycle
+mmc_power_off
+mmc_power_on
CMD_SEND:8
        ARG             0x00000000
sdhci_send_command: Timeout for status update!
        RET             -110
+mmc_set_initial_state
+mmc_set_signal_voltage
+mmc_select_mode
+mmc_mode2freq
CMD_SEND:8
        ARG             0x00000000
sdhci_send_command: Timeout for status update!
        RET             -110
+mmc_set_bus_width
CMD_SEND:8
        ARG             0x00000000
sdhci_send_command: Timeout for status update!
        RET             -110
CMD_SEND:8
        ARG             0x00000000
        MMC_RSP_R1,5,6,7      0x00000900
++mmc_set_initial_state ext_csd[EXT_CSD_PART_CONF] = 2
+mmc_go_idle
CMD_SEND:0
        ARG             0x00000000
        MMC_RSP_NONE
+mmc_send_if_cond
CMD_SEND:8
        ARG             0x000001aa
        RET             -110
+sd_send_op_cond
CMD_SEND:55
        ARG             0x00000000
        RET             -110
+mmc_send_op_cond
+mmc_go_idle
CMD_SEND:0
        ARG             0x00000000
        MMC_RSP_NONE
+mmc_send_op_cond_iter
CMD_SEND:1
        ARG             0x00000000
        MMC_RSP_R3,4         0x40ff8080
+mmc_send_op_cond_iter
CMD_SEND:1
        ARG             0x40300080
        MMC_RSP_R3,4         0x40ff8080
+mmc_send_op_cond_iter
CMD_SEND:1
        ARG             0x40300080
        MMC_RSP_R3,4         0xc0ff8080
+mmc_complete_init
CMD_SEND:8
        ARG             0x00000000
        RET             -110
+mmc_complete_op_cond
+mmc_startup
CMD_SEND:8
        ARG             0x00000000
        RET             -110
CMD_SEND:2
        ARG             0x00000000
        MMC_RSP_R2         0x15010038
                           0x474d4534
                           0x52010418
                           0xfc4f7300

                    DUMPING DATA
                    000 - 15 01 00 38
                    004 - 47 4d 45 34
                    008 - 52 01 04 18
                    012 - fc 4f 73 00
CMD_SEND:3
        ARG             0x00010000
        MMC_RSP_R1,5,6,7      0x00400500
CMD_SEND:9
        ARG             0x00010000
        MMC_RSP_R2         0xd0270132
                           0x0f5903ff
                           0xf6dbffef
                           0x8e404000

                    DUMPING DATA
                    000 - d0 27 01 32
                    004 - 0f 59 03 ff
                    008 - f6 db ff ef
                    012 - 8e 40 40 00
+mmc_select_mode
+mmc_mode2freq
CMD_SEND:8
        ARG             0x00000000
        RET             -110
CMD_SEND:7
        ARG             0x00010000
        MMC_RSP_R1,5,6,7      0x00000700
+mmc_startup_v4
CMD_SEND:8
        ARG             0x00000000
        MMC_RSP_R1,5,6,7      0x00000900
+mmc_set_capacity
+mmc_get_capabilities
+mmc_select_mode_and_width
CMD_SEND:6
        ARG             0x03b70100
        MMC_RSP_R1b         0x00000900
CMD_SEND:13
        ARG             0x00010000
        MMC_RSP_R1,5,6,7      0x00000900
CURR STATE:4
+bus_width
+mmc_set_bus_width
CMD_SEND:8
        ARG             0x00000000
        RET             -70
CMD_SEND:6
        ARG             0x03b90100
        MMC_RSP_R1b         0x00000900
CMD_SEND:13
        ARG             0x00010000
        MMC_RSP_R1,5,6,7      0x00000900
CURR STATE:4
CMD_SEND:8
        ARG             0x00000000
        MMC_RSP_R1,5,6,7      0x00000900
+mmc_select_mode
+mmc_mode2freq
CMD_SEND:8
        ARG             0x00000000
        MMC_RSP_R1,5,6,7      0x00000900
++mmc_select_mode ext_csd[EXT_CSD_PART_CONF] = 0
+mmc_read_and_compare_ext_csd
CMD_SEND:8
        ARG             0x00000000
        MMC_RSP_R1,5,6,7      0x00000900
CMD_SEND:8
        ARG             0x00000000
        MMC_RSP_R1,5,6,7      0x00000900
++ext_csd[EXT_CSD_PART_CONF] = 0
CMD_SEND:16
        ARG             0x00000200
        MMC_RSP_R1,5,6,7      0x00000900
CMD_SEND:17
        ARG             0x00000000
        MMC_RSP_R1,5,6,7      0x00000900
CMD_SEND:16
        ARG             0x00000200
        MMC_RSP_R1,5,6,7      0x00000900
CMD_SEND:18
        ARG             0x000000de
        MMC_RSP_R1,5,6,7      0x00000900
CMD_SEND:12
        ARG             0x00000000
        MMC_RSP_R1b         0x00000b00


U-Boot 2023.04-rc3-00159-gd1653548d2-dirty (Mar 19 2023 - 10:05:32 +1000)

SoC:   MV88F6828-A0 at 1600 MHz
DRAM:  1 GiB (800 MHz, 32-bit, ECC not enabled)
clearfog.c board_init part_config = 247
Core:  38 devices, 22 uclasses, devicetree: separate
MMC:   mv_sdh: 0
Loading Environment from MMC... OK
Model: SolidRun Clearfog A1
clearfog.c checkboard part_config = 0
Board: SolidRun Clearfog Pro
Net:   eth1: ethernet@70000, eth2: ethernet@30000, eth3: ethernet@34000
Hit any key to stop autoboot:  0
</output>

When I set mmc partconf 0 0 2 2, I get:
++mmc_set_initial_state ext_csd[EXT_CSD_PART_CONF] = 18

When I load u-boot to the first boot area with mmc partconf 0 0 2 2, I get:
++mmc_set_initial_state ext_csd[EXT_CSD_PART_CONF] = 17

When I load u-boot to the first boot area with mmc partconf 0 0 0 0, I get:
++mmc_set_initial_state ext_csd[EXT_CSD_PART_CONF] = 1

When I zero both boot areas and load u-boot to the data/user area with
mmc partconf 0 0 0 0, I get:
++mmc_set_initial_state ext_csd[EXT_CSD_PART_CONF] = 0

I'm not sure where to take it from here, but I'm assuming we'll need
to stash that value somewhere so we can refer to it later.