From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mauro Condarelli Date: Tue, 14 Jan 2020 12:03:44 +0100 Subject: How to debug HW startup? In-Reply-To: <8b561059-4370-3d6a-731f-f066742217fa@mclink.it> References: <0d0c04d3-ec53-f9ec-93ab-95189f241ddf@mclink.it> <0cdf764b-7178-e09c-e1a3-831a38d0b47a@denx.de> <7b3ca5c2-f84d-a4b1-ce60-591939e3cd01@denx.de> <489f31f3-d31d-c6c7-2fd8-ef7d4408e8b0@mclink.it> <0556487a-00d6-86f5-7538-78373d90ca56@mclink.it> <8b561059-4370-3d6a-731f-f066742217fa@mclink.it> Message-ID: List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit To: u-boot@lists.denx.de Hi Stefan, update, see below. On 1/14/20 12:08 AM, Mauro Condarelli wrote: > Next episode of this telenovela: > > I rebuilt u-boot for ROM at BC030000 (my code, very similar to LinkIt). > I flashed it at 30000 in SPI NOR: > > => usb start; sf probe > starting USB... > Bus ehci at 101c0000: pinctrl_select_state_full('ehci at 101c0000', 'default'): > USB EHCI 1.00 > scanning bus ehci at 101c0000 for devices... 3 USB Device(s) found >        scanning usb for storage devices... 1 Storage Device(s) found > SF: Detected w25q128 with page size 256 Bytes, erase size 4 KiB, total > 16 MiB > => load usb 0:1 85000000 u-boot.secondary > 390089 bytes read in 18 ms (20.7 MiB/s) > => sf update ${fileaddr} 30000 ${filesize} > device 0 offset 0x30000, size 0x5f3c9 > 390089 bytes written, 0 bytes skipped in 9.751s, speed 40952 B/s > => reset > > Restarted old u-boot and jumped to new: > > U-Boot 1.1.3 (Apr 23 2017 - 14:59:01) > VoCore2 > go bc030000 > ## Starting application at 0xBC030000 ... > board_debug_uart_init(): > > System stopped here several minutes, enough for me to start writing > this email, then it continued boot sequence: >              board_debug_uart_init(): > board_early_init_f(): Here is the first strange thing: apparently `board_debug_uart_init()` is called twice. I am not 100% sure as code is full of `#ifdef`s, but it seems first time it's called in `arch/mips/cpu/start.S`, most likely in line 257 as I couldn't find traces of CONFIG_MIPS_INIT_STACK_IN_SRAM (besides `arch/mips/cpu/Kconfig:381` where is defined, defaults to `n` and none seems to touch anymore; second strange thing is I find no trace of it in `.config`) This, again, does not add-up (third "strange thing") because I see nothing between calls to `debug_uart_init` (start.S:257) and `board_init_f` (start.S:264) that could trigger a ~5min delay (but I'm really out of my depth, here!) unless there's something in the `printascii()`itself (e.g.: loop at serial_mtk.c:449 blocks). > pinctrl_select_state_full('palmbus at 10000000', 'default'): > pinctrl_select_state_full('uart2 at e00', 'default'): > pinctrl_select_state_full: uclass_get_device_by_phandle_id: err=-19 > pinctrl_select_state_full('clkctrl at 0x2c', 'default'): > > > U-Boot 2020.01-00370-g97a60844bd-dirty (Jan 13 2020 - 23:21:39 +0100) > > CPU:   MT7628 Rev 1.2 - Boot from XTAL (3-Byte SPI Addr) > Model: VoCore2 > DRAM:  128 MiB > pinctrl_select_state_full('palmbus at 10000000', 'default'): > pinctrl_select_state_full('pinctrl at 60', 'default'): > pinctrl_select_state_full('pin_state', 'default'): > pinctrl_select_state_full('uart2 at e00', 'default'): > pinctrl_select_state_full('uart2_pins', 'default'): > pinctrl_select_state_full('clkctrl at 0x2c', 'default'): > pinctrl_select_state_full('watchdog at 100', 'default'): > WDT:   Started with servicing (60s timeout) > board_early_init_r(): > arch_early_init_r(): > MMC:   pinctrl_select_state_full('mmc at 10130000', 'default'): > pinctrl_select_state_full('sd_iot_mode', 'default'): > pinctrl_select_state_full('clk48m at 0', 'default'): > pinctrl_select_state_full('mmc at 10130000', 'default'): > mmc at 10130000: 0 > Loading Environment from FAT... *** Warning - bad CRC, using default > environment > > In:    uart2 at e00 > Out:   uart2 at e00 > Err:   uart2 at e00 > Model: VoCore2 > arch_misc_init(): > => > > Code, beside being targeted to ROM and with a different SYS_TEXT_BASE, > is identical to what runs fine when started from RAM. > > I also tried copying flash to ram: > > => cp.b bc030000 80010000 5f3c9 > => go 80010000 > ## Starting application at 0x80010000 ... > board_debug_uart_init(): > [04010D08][04010D08] > DDR Calibration DQS reg = 00008888 > relocate_code Pointer at: 87f5c000 > *********************** > Watchdog Reset Occurred > *********************** > > ... but this is almost expected because I relocated at another address > without changing SYS_TEXT_BASE. > > A further measurement shows booting u-boot from flash stops for > almost 5 minutes (4m48s, using a manual stopwatch). > > I sincerely have no idea where to bang my head :( > > TiA > Mauro > > > On 1/13/20 3:14 PM, Mauro Condarelli wrote: >> Hi Stefan, >> unfortunately it does *not* work. >> >> Ram based is ok, but rom  is just as silent as mine: >> >> => usb start; sf probe; >> starting USB... >> Bus ehci at 101c0000: pinctrl_select_state_full('ehci at 101c0000', 'default'): >> USB EHCI 1.00 >> scanning bus ehci at 101c0000 for devices... 3 USB Device(s) found >>        scanning usb for storage devices... 1 Storage Device(s) found >> pinctrl_select_state_full('spi at b00', 'default'): >> SF: Detected gd25q128 with page size 256 Bytes, erase size 4 KiB, total >> 16 MiB >> => load usb 0:1 85000000 u-boot_linkit.bin-rom >> 455708 bytes read in 22 ms (19.8 MiB/s) >> => sf update ${fileaddr} 0 ${filesize} >> device 0 offset 0x0, size 0x6f41c >> 455708 bytes written, 0 bytes skipped in 12.288s, speed 37966 B/s >> => reset >> resetting ... >> pinctrl_select_state_full('syscon-reboot', 'default'): >> pinctrl_select_state_full('system-controller at 0', 'default'): >> >> I might have done something stupid; Now I need do stop, but I'll >> test again this evening. >> >> Many thanks >> Mauro >> >> >> On 1/13/20 1:45 PM, Stefan Roese wrote: >>> On 13.01.20 13:24, Mauro Condarelli wrote: >>>> On 1/13/20 12:39 PM, Stefan Roese wrote: >>>>> Hi Mauro, >>>>> >>>>> On 13.01.20 11:24, Mauro Condarelli wrote: >>>>>> On 1/13/20 7:53 AM, Stefan Roese wrote: >>>>>>> Hi Mauro, >>>>>>> >>>>>>> On 11.01.20 20:00, Mauro Condarelli wrote: >>>>>>>> I managed to find ONE of the reasons why my ROM build didn't run: >>>>>>>> I forgot to enable `CONFIG_BOARD_EARLY_INIT_F=y`. >>>>>>> I see. This explains of course, why your board does not boot without >>>>>>> any "preloader". >>>>>>> >>>>>>>> I wanted, nonetheless, be prepared for further mishaps, but >>>>>>>> I have some other problems (see below). >>>>>>> Are these issues now fixed? I scanned the discussion about the DEBUG >>>>>>> UART on the list. Is this working for you now or do you have any >>>>>>> other >>>>>>> issues still? Did you successfully boot your new U-Boot from SPI NOR? >>>>>> Yes and no :( >>>>>> >>>>>> I fixed my RAM-based problems, but booting from SPI NOR still >>>>>> refuses to >>>>>> utter a single byte. >>>>>> I do attach  my defconfigs and my board.c for your reading pleasure >>>>>> (in >>>>>> case you have troubles getting asleep they should provide a good >>>>>> cure). >>>>> Before looking into this in more depth (I actually do have problems >>>>> sleeping >>>>> though, so I might take a look at it later ;)), why don't you re-start >>>>> with >>>>> your defconfig by using the linkit defconfig file. If you are lucky, >>>>> the >>>>> LinkIt binary might even work for the VoCore2. Or what are the >>>>> differences >>>>> except the SoC type and WLAN? Its the same DDR2 config and the same >>>>> UART. >>>>> So it might get you pretty far - also with ROM based booting. >>>> I will try it. >>>> Just to be on the safe side, could You attach a binary (or two: ROM and >>>> RAM) to >>>> the mail? If it works it would really give me a start base. >>> See below. >>>    >>>>>> I also added a couple of printouts in drivers/pinctrl/pinctrl-uclass.c >>>>>> and it >>>>>> turns out system tries to setup pins for uart2, but fails (maybe >>>>>> because >>>>>> pinctrl at 60 is not yet setup correctly?). >>>>>> This is the output I get from RAM-based u-boot: >>>>>> >>>>>> board_debug_uart_init(): >>>>>> board_early_init_f(): >>>>>> pinctrl_select_state_full('palmbus at 10000000', 'default'): >>>>>> pinctrl_select_state_full('uart2 at e00', 'default'): >>>>>> pinctrl_select_state_full: uclass_get_device_by_phandle_id: err=-19 >>>>>> pinctrl_select_state_full('clkctrl at 0x2c', 'default'): >>>>>> >>>>>> U-Boot 2020.01-00370-g97a60844bd-dirty (Jan 13 2020 - 01:03:59 +0100) >>>>>> >>>>>> CPU:   MT7628 Rev 1.2 - Boot from XTAL (3-Byte SPI Addr) >>>>>> Model: VoCore2 >>>>>> DRAM:  128 MiB >>>>>> pinctrl_select_state_full('palmbus at 10000000', 'default'): >>>>>> pinctrl_select_state_full('pinctrl at 60', 'default'): >>>>>> pinctrl_select_state_full('pin_state', 'default'): >>>>>> pinctrl_select_state_full('uart2 at e00', 'default'): >>>>>> pinctrl_select_state_full('uart2_pins', 'default'): >>>>>> pinctrl_select_state_full('clkctrl at 0x2c', 'default'): >>>>>> pinctrl_select_state_full('watchdog at 100', 'default'): >>>>>> WDT:   Started with servicing (60s timeout) >>>>>> board_early_init_r(): >>>>>> arch_early_init_r(): >>>>>> MMC:   pinctrl_select_state_full('mmc at 10130000', 'default'): >>>>>> pinctrl_select_state_full('sd_iot_mode', 'default'): >>>>>> pinctrl_select_state_full('clk48m at 0', 'default'): >>>>>> pinctrl_select_state_full('mmc at 10130000', 'default'): >>>>>> mmc at 10130000: 0 >>>>>> Loading Environment from FAT... *** Warning - bad CRC, using default >>>>>> environment >>>>>> >>>>>> In:    uart2 at e00 >>>>>> Out:   uart2 at e00 >>>>>> Err:   uart2 at e00 >>>>>> Model: VoCore2 >>>>>> arch_misc_init(): >>>>>> => >>>>> I don't have all these pinctrl issues on LinkIt. I suspect a config >>>>> issue and / or dts issue. Please compare. >>>> I will, but I was trying to say something different: >>>> Apparently the stock software tries to initialize uart2 pinctrl and it >>>> *seems* to succeed later in initialization. >>>> This *seems* to indicate explicit pin setup in board_early_init_f() is >>>> not strictly needed (we would lose only the first few lines even if >>>> we don't fix the error). >>>> I had a look to pinctrl-mt7628.c and it seems to do the right thing >>>> upon call to ('uart2_pins', 'default'), so anything after that *should* >>>> work even without low-level setup. >>> You might be correct here. So this explicit pin-muxing for UART2 is >>> only needed when DEBUG UART is used. This makes the code a bit clearer >>> and smaller. Lets keep it on our list to remove this, once all this >>> is resolved. >>>   >>>>>> ROM-based u-boot, as said, does not print *anything*, not even >>>>>> garbage. >>>>>> I'm beginning to suspect I have some mishap with start address or >>>>>> similar. >>>>>> I have absolutely no idea how to debug this without a JTAG probe >>>>>> (which >>>>>> I don't have and would be non-trivial to get working; soldering >>>>>> required). >>>>> Yes. JTAG requires soldering IIRC. >>>>>   >>>>>> The only idea I currently have is to modify my "old" u-boot to do >>>>>> initialization >>>>>> and then jump to beginning of "new" u-boot. >>>>>> If I can make it work in an automatic way I can try removing >>>>>> initialization steps >>>>>> till I find the "culprit". >>>>>> Any better idea would be welcome, of course! >>>>>> >>>>>> If I have to resort to that clumsy method, would be enough to put  >>>>>> "new" >>>>>> in  SPI NOR (of course at an higher address, e.g.: 30000 as "old" is >>>>>> only >>>>>> 183272 bytes long) and jump to the first location? >>>>>> I assume I will have to relocate CONFIG_SYS_TEXT_BASE=0xbc030000 >>>>> This is the TEXT_BASE for ROM based booting: >>>>> >>>>> CONFIG_SYS_TEXT_BASE=0x9c000000 >>>> AFAIK 0x9c000000 and 0xbc000000 map to the same (unmapped) memory >>>> region, the latter being uncached. >>>> I proposed 0xbc030000 (but it could be 0x9c030000) as base to leave some >>>> space for "old" (possibly crippled) u-boot doing initialization and then >>>> "jumping" >>>> to "new". >>> I see. Please give the LinkIt version a try first. This sounds easier >>> (if it works) and more promising for my taste. >>>   >>>>> Please see configs/linkit-smart-7688_defconfig. >>>>> >>>>>> Did I forget something? >>>>> Not sure. I still think using the linkit port as a base for SPI NOR >>>>> based booting would be a very good test at least. Do you see any >>>>> problems with this approach (board / SoC differences)? >>>> No. I see no troubles, for a basic startup. >>>> MMC would probably need some changes, but that can come later. >>> Yes. This test is just for basic very low level bootup checking. If >>> this works, then you can use it as a basis to fine-tune your VoCore2 >>> version from it by changing MMC etc support. >>> >>>> As said: I would like to try a prebuilt binary as first thing, so I can >>>> be sure I have no other problems (like gcc version or other local >>>> variations); then I would try to reproduce locally the equivalent binary >>>> with no changes and finally I would try to incorporate my specific >>>> needs. >>>> What do You think? >>> Please find attached the current mainline versions for LinkIt, one >>> for RAM booting and one for ROM booting (in SPI NOR). I checked both >>> version and they work fine on my LinkIt 7688 board: >>> >>> U-Boot 2020.01-00473-g88366b96ee (Jan 13 2020 - 13:37:15 +0100) >>> >>> CPU:   MT7628 Rev 1.2 - Boot from XTAL (4-Byte SPI Addr) >>> Model: LinkIt-Smart-7688 >>> DRAM:  128 MiB >>> Loading Environment from SPI Flash... SF: Detected mx25l25635e with >>> page size 256 Bytes, erase size 64 KiB, total 32 MiB >>> OK >>> Net:   eth0: eth at 10110000 >>> Hit any key to stop autoboot:  0 >>> => >>> >>> >>> Thanks, >>> Stefan Regards Mauro