i am pleased to be able to announce the successful booting of microwatt-5.7
linux buildroot... under a veriilator simulation of the microwatt VHDL.
from a hardware development and research perspective this is highly
significant because unlike the FPGA boot which was previously reported,
https://shenki.github.io/boot-linux-on-microwatt/
full memory read/write snooping and full Signal tracing (gtkwave) is possible.

https://ftp.libre-soc.org/microwatt-linux-5.7-verilator-boot-buildroot.txt

the branch of microwatt HDL which is being used is here
https://git.libre-soc.org/?p=microwatt.git;a=shortlog;h=refs/heads/verilator_trace

some minor strategic changes to microwatt HDL were required, including
adding a new SYSCON parameter to specify a BRAM chain-boot address,
and also it was necessary to turn sdram_init into a stand-alone "mini-BIOS"
which performed the role of early-initialising the 16550 uart followed by
chain-loading to the BRAM chain-boot memory location, at which the linux
5.7 dtbImage.microwatt had been loaded (0x600000).

microwatt-verilator.cpp itself needed some changes to add support for
emulation in c++ of 512 mbyte of "Block" RAM.  the interface for BRAM
(aka SRAM) was far simpler than attempting to emulate DRAM, and
also meant that much of the mini-BIOS could be entirely cut.

i also had to  further modify microwatt-verilator.cpp to allow it to load
from files directly into memory, at run-time.  this means it is possible
to execute hello_world.bin, zephyr.bin, micropython.bin, dtbImage-microwatt
all without recompiling the verilator binary.

(not that you want to try compiling a 6 MB binary into VHDL like i did:
it resulted in the creation of a 512 MB verilog file which, at 60 GB resident
RAM by verilator attempting to compile that to c++, i decided that mayyybe
doing that at runtime was a better approach?)

i also had to fix a couple of things in the linux kernel source
https://git.kernel.org/pub/scm/linux/kernel/git/joel/microwatt.git

first attempts to boot a compressed image were quite hilarious: a
quick back-of-the-envelope calculation by examining the rate at which
LD/STs were being generated showed that the GZIP decompression
would complete maybe some time in about 1 hour of real-world time.
this led me to add support for CONFIG_KERNEL_UNCOMPRESSED
and cut that time entirely, hence why you can see this in the console log:

    0x5b0e10 bytes of uncompressed data copied

secondly, the microwatt Makefile assumes that verilator clock rate
runs at 50 mhz, where the microwatt.dts file says 100 mhz for both
the UART clock as well as the system clock.  it would be really nice
to have microwatt-linux read the SYSCON parameter for the
clock rate, and for that to be dynamically inserted into the dtb.
however in the interim, the attached patch suffices by manually
altering the clock in microwatt.dts to match that of the SYSCON
parameter.

the initial boots without sdram_init.bin did not go well.  this is
probably because the udbg0 (early ns16550.c) is not correctly
initialised (critically relying on the use of the microwatt console_init()
library). what was great - and this really is the whole point - i was
able to track down the source of the problem...
by examining the VCD trace wires of the 16550 Wishbone Bus
and internal UART registers... from the HDL! :)
if there had been such a problem on the FPGA side, that would
have been outright impossible and impractical.

for anyone thinking of following this and using it, please be under
no illusion: it took *two hours* to get to that boot prompt on a 4.8ghz
Intel i9.  1000 ns of "simulated" 50 mhz clock rate takes a stunning
15-20 seconds of real time.  you can do the math on the number
of instructions per second, there, but the huge advantage is: direct
snoop access to the memory, and the entire signal tracing of the
HDL - all of it: every single signal, for every single cycle.

the other downside: running for even 30 seconds produces an
astounding *10 gigabytes* of VCD trace log output.  normally
you would switch on command-line options in verilator to
only enable the VCD tracing at certain ranges of clock cycles,
so that you have access to the Signals that you are interested
in.  i have seen people enable that over a debug interface
(from a separate program, communicating with the verilator
executable) but that is outside the scope of this message.

the next task will be to swap out the microwatt core and drop in
the libresoc core.  with the successful passing of 17/19 of the
microwatt mmu.bin unit tests last week this is expected to be
relatively straightforward, especially given that we already have
microwatt-compatible XICS, microwatt-compatible DMI, exactly
the same sized I and D wishbone buses, and a direct port of
microwatt's MMU, L1 and D1 Caches.  missing is a SYSCON
device and the Wishbone Bus Arbiter.

however once that (relatively straightforward) work is done,
we will be able to boot the *exact* same linux buildroot image
(and i can debug it under verilator, which is why i've gone
to all the trouble, above....) and once that passes i will then
try an ECP5 FPGA boot.

hurrah.

l.