[PATCH v1 0/7] enabling RasperryPi 4 emulation - WIP state

[PATCH v1 0/7] enabling RasperryPi 4 emulation - WIP state

[Alex Bennée]

Hi Phillipe,

I needed to test some Xen things on a model I could debug so I took
the liberty of re-basing you patches from a 2 years ago. Most of the
changes involve dropping patches since merged and dealing with the
resultant merging fallout. The fix I posted on Friday for the
calculation of the ram base for kernel loading is included. Other
tweaks involve moving the location of peri_base to what is indicated
by the DTBs (I'm still not sure where the Linux addresses come from,
does it re-map the physical address space somewhere?).

My testing ended at getting Xen to boot up and at least echo some
stuff to the console:

  ./qemu-system-aarch64 -machine raspi4b -serial mon:stdio -kernel ~/lsrc/xen/xen-only.cross.arm64/xen/xen -append "dom0_mem=1G,max:1G loglvl=all guest_loglvl=all console=d
  tuart dtuart=serial0 sync_console" -display none -dtb ./bcm2711-rpi-4-b.dtb            
  - UART enabled -                                                                      
  - Boot CPU booting -                       
  - Current EL 00000008 -                    
  - Initialize CPU -                                                                    
  - Turning on paging -               
  - Zero BSS -                                                                          
  - Ready -                             
  (XEN) Checking for initrd in /chosen
  (XEN) RAM: 0000000000000000 - 000000003bffffff            
  (XEN)                               
  (XEN) MODULE[0]: 0000000000200000 - 000000000034d0c8 Xen         
  (XEN) MODULE[1]: 0000000008000000 - 000000000801cea4 Device Tree 
  (XEN)  RESVD[0]: 0000000000000000 - 0000000000001000
  (XEN)                                
  (XEN)                               
  (XEN) Command line: <NULL>              
  (XEN) Domain heap initialised                                                         
  (XEN) Booting using Device Tree        
  (XEN) Platform: Raspberry Pi 4                                                        
  (XEN) No dtuart path configured
  (XEN) Bad console= option 'dtuart'

My next steps involve debugging Xen so I thought I'd post where I am
for the next person that wants to properly wire up the Pi4.

Alex Bennée (1):
  hw/arm: fix the position of vcram for raspi

Philippe Mathieu-Daudé (6):
  hw/misc/bcm2835_property: Add FIXME comment for uninitialized memory
  hw/misc/bcm2835_property: Handle the 'domain state' property
  hw/arm/bcm2835_peripherals: Map various BCM2711 blocks
  hw/arm/bcm2836: Add the BCM2711 which uses a GICv2
  hw/arm/bcm2838: Map the PCIe memory space
  hw/arm/raspi: Add the Raspberry Pi 4B board

 include/hw/arm/bcm2835_peripherals.h |  9 ++-
 include/hw/arm/bcm2836.h             |  3 +
 include/hw/arm/raspi_platform.h      | 14 +++++
 hw/arm/bcm2835_peripherals.c         | 38 +++++++++---
 hw/arm/bcm2836.c                     | 89 ++++++++++++++++++++++++++++
 hw/arm/raspi.c                       | 34 ++++++++---
 hw/misc/bcm2835_property.c           | 10 ++++
 7 files changed, 179 insertions(+), 18 deletions(-)

-- 
2.30.2

[PATCH v1 1/7] hw/misc/bcm2835_property: Add FIXME comment for uninitialized memory

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

We are returning unintialized memory, this is probably unsafe.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 hw/misc/bcm2835_property.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/hw/misc/bcm2835_property.c b/hw/misc/bcm2835_property.c
index 73941bdae9..791c7554ec 100644
--- a/hw/misc/bcm2835_property.c
+++ b/hw/misc/bcm2835_property.c
@@ -61,6 +61,7 @@ static void bcm2835_property_mbox_push(BCM2835PropertyState *s, uint32_t value)
             qemu_log_mask(LOG_UNIMP,
                           "bcm2835_property: 0x%08x get board model NYI\n",
                           tag);
+            /* FIXME returning uninitialized memory */
             resplen = 4;
             break;
         case 0x00010002: /* Get board revision */
@@ -75,6 +76,7 @@ static void bcm2835_property_mbox_push(BCM2835PropertyState *s, uint32_t value)
             qemu_log_mask(LOG_UNIMP,
                           "bcm2835_property: 0x%08x get board serial NYI\n",
                           tag);
+            /* FIXME returning uninitialized memory */
             resplen = 8;
             break;
         case 0x00010005: /* Get ARM memory */
-- 
2.30.2

[PATCH v1 2/7] hw/misc/bcm2835_property: Handle the 'domain state' property

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

The kernel is happy with this change, so we don't need
to do anything more sophisticated.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 hw/misc/bcm2835_property.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/hw/misc/bcm2835_property.c b/hw/misc/bcm2835_property.c
index 791c7554ec..b089e47584 100644
--- a/hw/misc/bcm2835_property.c
+++ b/hw/misc/bcm2835_property.c
@@ -133,6 +133,14 @@ static void bcm2835_property_mbox_push(BCM2835PropertyState *s, uint32_t value)
             resplen = 8;
             break;
 
+        case 0x00030030: /* Get domain state */
+            qemu_log_mask(LOG_UNIMP,
+                          "bcm2835_property: 0x%08x get domain state NYI\n",
+                          tag);
+            /* FIXME returning uninitialized memory */
+            resplen = 8;
+            break;
+
         case 0x00038002: /* Set clock rate */
         case 0x00038004: /* Set max clock rate */
         case 0x00038007: /* Set min clock rate */
-- 
2.30.2

[PATCH v1 3/7] hw/arm: fix the position of vcram for raspi

[Alex Bennée]

The previous calculation fell over when I tried to create a 8gb Pi 4
because the values were only 32 bit. However the quirk of the Pi
hardware is the vcram can only appear in the first 1gb of address
space. This also limits where the initial kernel and DTB can be
loaded (notice the DTS for the 8gb Pi4 still only uses 32 bit sizes).
Fix this cleaning up setup_boot to directly use vcram_base and
documenting what is going on.

NB: the aliases are confusing.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Michael Bishop <cleverca22@gmail.com>
---
 hw/arm/bcm2835_peripherals.c | 14 +++++++++++---
 hw/arm/bcm2836.c             |  2 ++
 hw/arm/raspi.c               | 19 ++++++++++++-------
 3 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/hw/arm/bcm2835_peripherals.c b/hw/arm/bcm2835_peripherals.c
index 48538c9360..46852bc8a6 100644
--- a/hw/arm/bcm2835_peripherals.c
+++ b/hw/arm/bcm2835_peripherals.c
@@ -12,6 +12,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "qemu/module.h"
+#include "qemu/units.h"
 #include "hw/arm/bcm2835_peripherals.h"
 #include "hw/misc/bcm2835_mbox_defs.h"
 #include "hw/arm/raspi_platform.h"
@@ -74,6 +75,7 @@ static void bcm2835_peripherals_init(Object *obj)
     /* Framebuffer */
     object_initialize_child(obj, "fb", &s->fb, TYPE_BCM2835_FB);
     object_property_add_alias(obj, "vcram-size", OBJECT(&s->fb), "vcram-size");
+    object_property_add_alias(obj, "vcram-base", OBJECT(&s->fb), "vcram-base");
 
     object_property_add_const_link(OBJECT(&s->fb), "dma-mr",
                                    OBJECT(&s->gpu_bus_mr));
@@ -138,7 +140,7 @@ static void bcm2835_peripherals_realize(DeviceState *dev, Error **errp)
     Object *obj;
     MemoryRegion *ram;
     Error *err = NULL;
-    uint64_t ram_size, vcram_size;
+    uint64_t ram_size, vcram_size, vcram_base;
     int n;
 
     obj = object_property_get_link(OBJECT(dev), "ram", &error_abort);
@@ -235,15 +237,21 @@ static void bcm2835_peripherals_realize(DeviceState *dev, Error **errp)
         qdev_get_gpio_in_named(DEVICE(&s->ic), BCM2835_IC_ARM_IRQ,
                                INTERRUPT_ARM_MAILBOX));
 
-    /* Framebuffer */
+    /*
+     * The framebuffer has to live in the first 1gb of addressable
+     * space which is fine for older Pi's with less than 1gb of RAM
+     * but we need to take care not to put it too high otherwise
+     */
     vcram_size = object_property_get_uint(OBJECT(s), "vcram-size", &err);
     if (err) {
         error_propagate(errp, err);
         return;
     }
 
+    vcram_base = MIN(ram_size, 1 * GiB) - vcram_size;
+
     if (!object_property_set_uint(OBJECT(&s->fb), "vcram-base",
-                                  ram_size - vcram_size, errp)) {
+                                  vcram_base, errp)) {
         return;
     }
 
diff --git a/hw/arm/bcm2836.c b/hw/arm/bcm2836.c
index 24354338ca..255ba8265a 100644
--- a/hw/arm/bcm2836.c
+++ b/hw/arm/bcm2836.c
@@ -62,6 +62,8 @@ static void bcm2836_init(Object *obj)
                               "board-rev");
     object_property_add_alias(obj, "vcram-size", OBJECT(&s->peripherals),
                               "vcram-size");
+    object_property_add_alias(obj, "vcram-base", OBJECT(&s->peripherals),
+                              "vcram-base");
 }
 
 static bool bcm283x_common_realize(DeviceState *dev, Error **errp)
diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c
index 146d35382b..3fb1c3138b 100644
--- a/hw/arm/raspi.c
+++ b/hw/arm/raspi.c
@@ -196,14 +196,19 @@ static void reset_secondary(ARMCPU *cpu, const struct arm_boot_info *info)
     cpu_set_pc(cs, info->smp_loader_start);
 }
 
+/*
+ * NB: ram_limit isn't the same as ram_size - it indicates the portion
+ * of RAM that boot components can live in (up to the first 1gb - the
+ * vcram_size, aka vcram_base)
+ */
 static void setup_boot(MachineState *machine, RaspiProcessorId processor_id,
-                       size_t ram_size)
+                       size_t ram_limit)
 {
     RaspiMachineState *s = RASPI_MACHINE(machine);
     int r;
 
     s->binfo.board_id = MACH_TYPE_BCM2708;
-    s->binfo.ram_size = ram_size;
+    s->binfo.ram_size = ram_limit;
     s->binfo.nb_cpus = machine->smp.cpus;
 
     if (processor_id <= PROCESSOR_ID_BCM2836) {
@@ -239,7 +244,7 @@ static void setup_boot(MachineState *machine, RaspiProcessorId processor_id,
                              ? FIRMWARE_ADDR_2 : FIRMWARE_ADDR_3;
         /* load the firmware image (typically kernel.img) */
         r = load_image_targphys(machine->firmware, firmware_addr,
-                                ram_size - firmware_addr);
+                                ram_limit - firmware_addr);
         if (r < 0) {
             error_report("Failed to load firmware from %s", machine->firmware);
             exit(1);
@@ -258,7 +263,7 @@ static void raspi_machine_init(MachineState *machine)
     RaspiMachineState *s = RASPI_MACHINE(machine);
     uint32_t board_rev = mc->board_rev;
     uint64_t ram_size = board_ram_size(board_rev);
-    uint32_t vcram_size;
+    uint32_t vcram_base;
     DriveInfo *di;
     BlockBackend *blk;
     BusState *bus;
@@ -295,10 +300,10 @@ static void raspi_machine_init(MachineState *machine)
     qdev_prop_set_drive_err(carddev, "drive", blk, &error_fatal);
     qdev_realize_and_unref(carddev, bus, &error_fatal);
 
-    vcram_size = object_property_get_uint(OBJECT(&s->soc), "vcram-size",
+    vcram_base = object_property_get_uint(OBJECT(&s->soc), "vcram-base",
                                           &error_abort);
-    setup_boot(machine, board_processor_id(mc->board_rev),
-               machine->ram_size - vcram_size);
+
+    setup_boot(machine, board_processor_id(mc->board_rev), vcram_base);
 }
 
 static void raspi_machine_class_common_init(MachineClass *mc,
-- 
2.30.2

[PATCH v1 4/7] hw/arm/bcm2835_peripherals: Map various BCM2711 blocks

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

The BCM2711 provides more peripherals. Add them as 'unimplemented' so
we can track when/how firmware and kernel access them.

Based on various sources:

* U-boot: https://github.com/raspberrypi/firmware/tree/next/boot

  - arch/arm/dts/bcm283x.dtsi
  - arch/arm/dts/bcm2838.dtsi
  - arch/arm/dts/bcm2838-rpi-4-b.dts

* Arnd Bergmann analysis: https://www.cnx-software.com/2019/06/24/raspberry-pi-4-features-broadcom-bcm2711-processor-up-to-4gb-ram/#comment-563948

* Linux: https://patchwork.kernel.org/patch/11053097/

  - arch/arm/boot/dts/bcm283x.dtsi
  - arch/arm/boot/dts/bcm283x-rpi-usb-peripheral.dtsi
  - arch/arm/boot/dts/bcm2711.dtsi
  - arch/arm/boot/dts/bcm2711-rpi-4-b.dts

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: merge fixes, drop dwc2 unimp]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
TODO: At least use proper pl011[2] and pl011[3]
vAJB:
  - fixed merge conflict on re-base
  - rename to BCM2711
  - drop dwc2 unimp (as it is now imp)
---
 include/hw/arm/bcm2835_peripherals.h |  9 ++++++---
 include/hw/arm/raspi_platform.h      | 14 ++++++++++++++
 hw/arm/bcm2835_peripherals.c         | 24 +++++++++++++++++++-----
 3 files changed, 39 insertions(+), 8 deletions(-)

diff --git a/include/hw/arm/bcm2835_peripherals.h b/include/hw/arm/bcm2835_peripherals.h
index d864879421..8f7f23712a 100644
--- a/include/hw/arm/bcm2835_peripherals.h
+++ b/include/hw/arm/bcm2835_peripherals.h
@@ -52,6 +52,7 @@ struct BCM2835PeripheralState {
     BCM2835PowerMgtState powermgt;
     BCM2835CprmanState cprman;
     PL011State uart0;
+    UnimplementedDeviceState uartu[6];
     BCM2835AuxState aux;
     BCM2835FBState fb;
     BCM2835DMAState dma;
@@ -63,16 +64,18 @@ struct BCM2835PeripheralState {
     BCM2835SDHostState sdhost;
     BCM2835GpioState gpio;
     Bcm2835ThermalState thermal;
+    DWC2State dwc2;
     UnimplementedDeviceState i2s;
-    UnimplementedDeviceState spi[1];
-    UnimplementedDeviceState i2c[3];
+    UnimplementedDeviceState spi[7];
+    UnimplementedDeviceState i2c[7];
     UnimplementedDeviceState otp;
     UnimplementedDeviceState dbus;
     UnimplementedDeviceState ave0;
     UnimplementedDeviceState v3d;
     UnimplementedDeviceState bscsl;
     UnimplementedDeviceState smi;
-    DWC2State dwc2;
+    UnimplementedDeviceState xhci;
+    UnimplementedDeviceState argon;
     UnimplementedDeviceState sdramc;
 };
 
diff --git a/include/hw/arm/raspi_platform.h b/include/hw/arm/raspi_platform.h
index e0e6c8ce94..7b6393542a 100644
--- a/include/hw/arm/raspi_platform.h
+++ b/include/hw/arm/raspi_platform.h
@@ -51,10 +51,22 @@
 #define RNG_OFFSET              0x104000
 #define GPIO_OFFSET             0x200000
 #define UART0_OFFSET            0x201000 /* PL011 */
+#define UART2_OFFSET            0x201400 /* PL011 */
+#define UART3_OFFSET            0x201600 /* PL011 */
+#define UART4_OFFSET            0x201800 /* PL011 */
+#define UART5_OFFSET            0x201a00 /* PL011 */
 #define MMCI0_OFFSET            0x202000 /* Legacy MMC */
 #define I2S_OFFSET              0x203000 /* PCM */
 #define SPI0_OFFSET             0x204000 /* SPI master */
+#define SPI3_OFFSET             0x204600
+#define SPI4_OFFSET             0x204800
+#define SPI5_OFFSET             0x204a00
+#define SPI6_OFFSET             0x204c00
 #define BSC0_OFFSET             0x205000 /* BSC0 I2C/TWI */
+#define BSC3_OFFSET             0x205600
+#define BSC4_OFFSET             0x205800
+#define BSC5_OFFSET             0x205a00
+#define BSC6_OFFSET             0x205c00
 #define PIXV0_OFFSET            0x206000
 #define PIXV1_OFFSET            0x207000
 #define DPI_OFFSET              0x208000
@@ -86,6 +98,8 @@
 #define DBUS_OFFSET             0x900000
 #define AVE0_OFFSET             0x910000
 #define USB_OTG_OFFSET          0x980000 /* DTC_OTG USB controller */
+#define USB_XHCI_OFFSET         0x9c0000 /* generic-xhci controller */
+#define ARGON_OFFSET            0xb00000
 #define V3D_OFFSET              0xc00000
 #define SDRAMC_OFFSET           0xe00000
 #define L2CC_OFFSET             0xe01000 /* Level 2 Cache controller */
diff --git a/hw/arm/bcm2835_peripherals.c b/hw/arm/bcm2835_peripherals.c
index 46852bc8a6..3856c7d267 100644
--- a/hw/arm/bcm2835_peripherals.c
+++ b/hw/arm/bcm2835_peripherals.c
@@ -388,15 +388,29 @@ static void bcm2835_peripherals_realize(DeviceState *dev, Error **errp)
     create_unimp(s, &s->armtmr, "bcm2835-sp804", ARMCTRL_TIMER0_1_OFFSET, 0x40);
     create_unimp(s, &s->i2s, "bcm2835-i2s", I2S_OFFSET, 0x100);
     create_unimp(s, &s->smi, "bcm2835-smi", SMI_OFFSET, 0x100);
-    create_unimp(s, &s->spi[0], "bcm2835-spi0", SPI0_OFFSET, 0x20);
+    create_unimp(s, &s->uartu[2], "!pl011[2]", UART2_OFFSET, 0x100);
+    create_unimp(s, &s->uartu[3], "!pl011[3]", UART3_OFFSET, 0x100);
+    create_unimp(s, &s->uartu[4], "!pl011[4]", UART4_OFFSET, 0x100);
+    create_unimp(s, &s->uartu[5], "!pl011[5]", UART5_OFFSET, 0x100);
+    create_unimp(s, &s->spi[0], "bcm2835-spi[0]", SPI0_OFFSET, 0x20);
+    create_unimp(s, &s->spi[3], "bcm2835-spi[3]", SPI3_OFFSET, 0x20);
+    create_unimp(s, &s->spi[4], "bcm2835-spi[4]", SPI4_OFFSET, 0x20);
+    create_unimp(s, &s->spi[5], "bcm2835-spi[5]", SPI5_OFFSET, 0x20);
+    create_unimp(s, &s->spi[6], "bcm2835-spi[6]", SPI6_OFFSET, 0x20);
     create_unimp(s, &s->bscsl, "bcm2835-spis", BSC_SL_OFFSET, 0x100);
-    create_unimp(s, &s->i2c[0], "bcm2835-i2c0", BSC0_OFFSET, 0x20);
-    create_unimp(s, &s->i2c[1], "bcm2835-i2c1", BSC1_OFFSET, 0x20);
-    create_unimp(s, &s->i2c[2], "bcm2835-i2c2", BSC2_OFFSET, 0x20);
+    create_unimp(s, &s->i2c[0], "bcm2835-i2c[0]", BSC0_OFFSET, 0x20);
+    create_unimp(s, &s->i2c[1], "bcm2835-i2c[1]", BSC1_OFFSET, 0x20);
+    create_unimp(s, &s->i2c[2], "bcm2835-i2c[2]", BSC2_OFFSET, 0x20);
+    create_unimp(s, &s->i2c[3], "bcm2835-i2c[3]", BSC3_OFFSET, 0x20);
+    create_unimp(s, &s->i2c[4], "bcm2835-i2c[4]", BSC4_OFFSET, 0x20);
+    create_unimp(s, &s->i2c[5], "bcm2835-i2c[5]", BSC5_OFFSET, 0x20);
+    create_unimp(s, &s->i2c[6], "bcm2835-i2c[6]", BSC6_OFFSET, 0x20);
     create_unimp(s, &s->otp, "bcm2835-otp", OTP_OFFSET, 0x80);
     create_unimp(s, &s->dbus, "bcm2835-dbus", DBUS_OFFSET, 0x8000);
     create_unimp(s, &s->ave0, "bcm2835-ave0", AVE0_OFFSET, 0x8000);
-    create_unimp(s, &s->v3d, "bcm2835-v3d", V3D_OFFSET, 0x1000);
+    create_unimp(s, &s->xhci, "bcm2838-xhci", USB_XHCI_OFFSET, 0x100000);
+    create_unimp(s, &s->argon, "bcm2838-argon", ARGON_OFFSET, 4 * 0x10000);
+    create_unimp(s, &s->v3d, "bcm2835-v3d", V3D_OFFSET, 0x10000);
     create_unimp(s, &s->sdramc, "bcm2835-sdramc", SDRAMC_OFFSET, 0x100);
 }
 
-- 
2.30.2

[PATCH v1 5/7] hw/arm/bcm2836: Add the BCM2711 which uses a GICv2

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

The BCM2711 is improvement of the BCM2837:
- Cortex-A72 instead of the A53
- peripheral block and local soc controller are mapped differently,
- GICv2
- PCIe block
- exhanced MMU to address over 4GiB of SDRAM

See https://www.raspberrypi.org/forums/viewtopic.php?t=244479&start=25
and https://patchwork.kernel.org/patch/11053097/

This patch starts mapping the GICv2 but interrupt lines are NOT
wired (yet).

This is enough to start running the Ubuntu kernel8.img from [1].

Extract the kernel with:

  $ mkdir bootpart
  $ guestfish \
      --ro \
      -a ubuntu-18.04.3-preinstalled-server-arm64+raspi4.img \
      -m /dev/sda1
  Welcome to guestfish, the guest filesystem shell for
  editing virtual machine filesystems and disk images.

  ><fs> ls /
  COPYING.linux
  LICENCE.broadcom
  System.map
  armstub8-gic.bin
  bcm2710-rpi-3-b-plus.dtb
  bcm2710-rpi-3-b.dtb
  bcm2710-rpi-cm3.dtb
  bcm2711-rpi-4-b.dtb
  bcm2837-rpi-3-b-plus.dtb
  bcm2837-rpi-3-b.dtb
  cmdline.txt
  config.txt
  fixup4.dat
  fixup4cd.dat
  fixup4db.dat
  fixup4x.dat
  kernel8.img
  overlays
  start4.elf
  start4cd.elf
  start4db.elf
  start4x.elf
  ><fs> copy-out / bootpart/
  ><fs> q

Then some progress can be noticed running:

  $ qemu-system-aarch64 -d unimp,guest_errors,int,in_asm \
      -M raspi4 \
      -kernel bootpart/kernel8.img \
      -dtb bootpart/bcm2711-rpi-4-b.dtb \
      -initrd bootpart/boot/initrd.img \
      -append \
         "earlycon=pl011,0xfe201000 console=ttyAMA0 console=tty1 loglevel=8"

Not very interesting, but it runs until configuring the GIC.
(remove 'in_asm' if too verbose).

TODO:

- wire IRQs to the GIC :)

- map the SPI bootrom from [3] (boot sequence: [4])

- per [2] we could try booting without using the GIC, adding "enable_gic=0"
  in config.txt. this variable is parsed by the firmware:

  $ fgrep -r enable_gic bootpart
  Binary file bootpart/start4x.elf matches
  Binary file bootpart/start4.elf matches
  Binary file bootpart/start4db.elf matches
  Binary file bootpart/start4cd.elf matches
  bootpart/config.txt:enable_gic=1

  the stub [5] doesn't seem to check a register for it.
  maybe it falls back to kernel7l?

- decompile start4.elf to check how 'enable_gic' is used
  using vc4 toolchain from [6]

[1] https://github.com/TheRemote/Ubuntu-Server-raspi4-unofficial/releases
[2] https://jamesachambers.com/raspberry-pi-ubuntu-server-18-04-2-installation-guide/
[3] https://www.raspberrypi.org/documentation/hardware/raspberrypi/booteeprom.md
[4] https://raspberrypi.stackexchange.com/questions/10442/what-is-the-boot-sequence
[5] https://github.com/raspberrypi/tools/commit/7f4a937e1bacbc111a22552169bc890b4bb26a94#diff-8c41083e9fa0c98f1c3015e11b897444
[6] https://github.com/christinaa/rpi-open-firmware

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: QOM fixes and 2711 rename]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
vAJB:
  - fix QOM operations following refactoring since last post
  - rename to BCM2711
  - use sysbus_realize(SYS_BUS_DEVICE(&s->gic)
  - move peri_base/ctrl to locations pointed to by DTB (0x7e000000/0x40000000)
---
 include/hw/arm/bcm2836.h |  3 ++
 hw/arm/bcm2836.c         | 77 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/raspi.c           |  2 ++
 3 files changed, 82 insertions(+)

diff --git a/include/hw/arm/bcm2836.h b/include/hw/arm/bcm2836.h
index 6f90cabfa3..c8f64fa06d 100644
--- a/include/hw/arm/bcm2836.h
+++ b/include/hw/arm/bcm2836.h
@@ -14,6 +14,7 @@
 
 #include "hw/arm/bcm2835_peripherals.h"
 #include "hw/intc/bcm2836_control.h"
+#include "hw/intc/arm_gic.h"
 #include "target/arm/cpu.h"
 #include "qom/object.h"
 
@@ -29,6 +30,7 @@ OBJECT_DECLARE_TYPE(BCM283XState, BCM283XClass, BCM283X)
 #define TYPE_BCM2835 "bcm2835"
 #define TYPE_BCM2836 "bcm2836"
 #define TYPE_BCM2837 "bcm2837"
+#define TYPE_BCM2711 "bcm2711"
 
 struct BCM283XState {
     /*< private >*/
@@ -40,6 +42,7 @@ struct BCM283XState {
     struct {
         ARMCPU core;
     } cpu[BCM283X_NCPUS];
+    GICState gic;
     BCM2836ControlState control;
     BCM2835PeripheralState peripherals;
 };
diff --git a/hw/arm/bcm2836.c b/hw/arm/bcm2836.c
index 255ba8265a..99dc15e6e4 100644
--- a/hw/arm/bcm2836.c
+++ b/hw/arm/bcm2836.c
@@ -25,6 +25,7 @@ typedef struct BCM283XClass {
     unsigned core_count;
     hwaddr peri_base; /* Peripheral base address seen by the CPU */
     hwaddr ctrl_base; /* Interrupt controller and mailboxes etc. */
+    hwaddr gic_base;
     int clusterid;
 } BCM283XClass;
 
@@ -36,6 +37,15 @@ typedef struct BCM283XClass {
 static Property bcm2836_enabled_cores_property =
     DEFINE_PROP_UINT32("enabled-cpus", BCM283XState, enabled_cpus, 0);
 
+#define GIC_NUM_IRQS                256
+
+#define GIC_BASE_OFS                0x0000
+#define GIC_DIST_OFS                0x1000
+#define GIC_CPU_OFS                 0x2000
+#define GIC_VIFACE_THIS_OFS         0x4000
+#define GIC_VIFACE_OTHER_OFS(cpu)  (0x5000 + (cpu) * 0x200)
+#define GIC_VCPU_OFS                0x6000
+
 static void bcm2836_init(Object *obj)
 {
     BCM283XState *s = BCM283X(obj);
@@ -56,6 +66,10 @@ static void bcm2836_init(Object *obj)
                                 TYPE_BCM2836_CONTROL);
     }
 
+    if (bc->gic_base) {
+        object_initialize_child(obj, "gic", &s->gic, TYPE_ARM_GIC);
+    }
+
     object_initialize_child(obj, "peripherals", &s->peripherals,
                             TYPE_BCM2835_PERIPHERALS);
     object_property_add_alias(obj, "board-rev", OBJECT(&s->peripherals),
@@ -126,6 +140,50 @@ static void bcm2836_realize(DeviceState *dev, Error **errp)
 
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->control), 0, bc->ctrl_base);
 
+    /* bcm2838 GICv2 */
+    if (bc->gic_base) {
+        if (!object_property_set_uint(OBJECT(&s->gic), "revision", 2, errp)) {
+            return;
+        }
+
+        if (!object_property_set_uint(OBJECT(&s->gic), "num-cpu",
+                                      BCM283X_NCPUS, errp)) {
+            return;
+        }
+
+        if (!object_property_set_uint(OBJECT(&s->gic), "num-irq",
+                                      32 + GIC_NUM_IRQS, errp)) {
+            return;
+        }
+
+        if (!object_property_set_bool(OBJECT(&s->gic),
+                                      "has-virtualization-extensions",
+                                      true, errp)) {
+            return;
+        }
+
+        if (!sysbus_realize(SYS_BUS_DEVICE(&s->gic), errp)) {
+            return;
+        }
+
+        sysbus_mmio_map(SYS_BUS_DEVICE(&s->gic), 0,
+                        bc->ctrl_base + bc->gic_base + GIC_DIST_OFS);
+        sysbus_mmio_map(SYS_BUS_DEVICE(&s->gic), 1,
+                        bc->ctrl_base + bc->gic_base + GIC_CPU_OFS);
+        sysbus_mmio_map(SYS_BUS_DEVICE(&s->gic), 2,
+                        bc->ctrl_base + bc->gic_base + GIC_VIFACE_THIS_OFS);
+        sysbus_mmio_map(SYS_BUS_DEVICE(&s->gic), 3,
+                        bc->ctrl_base + bc->gic_base + GIC_VCPU_OFS);
+
+        for (n = 0; n < BCM283X_NCPUS; n++) {
+            sysbus_mmio_map(SYS_BUS_DEVICE(&s->gic), 4 + n,
+                            bc->ctrl_base + bc->gic_base
+                            + GIC_VIFACE_OTHER_OFS(n));
+        }
+
+        /* TODO wire IRQs!!! */
+    }
+
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->peripherals), 0,
         qdev_get_gpio_in_named(DEVICE(&s->control), "gpu-irq", 0));
     sysbus_connect_irq(SYS_BUS_DEVICE(&s->peripherals), 1,
@@ -216,6 +274,21 @@ static void bcm2837_class_init(ObjectClass *oc, void *data)
     bc->clusterid = 0x0;
     dc->realize = bcm2836_realize;
 };
+
+static void bcm2711_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+    BCM283XClass *bc = BCM283X_CLASS(oc);
+
+    bc->cpu_type = ARM_CPU_TYPE_NAME("cortex-a72");
+    bc->core_count = BCM283X_NCPUS;
+    bc->peri_base = 0x7e000000;
+    bc->ctrl_base = 0x40000000;
+    bc->clusterid = 0x0;
+    bc->gic_base = 0x40000,
+    dc->realize = bcm2836_realize;
+}
+
 #endif
 
 static const TypeInfo bcm283x_types[] = {
@@ -232,6 +305,10 @@ static const TypeInfo bcm283x_types[] = {
         .name           = TYPE_BCM2837,
         .parent         = TYPE_BCM283X,
         .class_init     = bcm2837_class_init,
+    }, {
+        .name           = TYPE_BCM2711,
+        .parent         = TYPE_BCM283X,
+        .class_init     = bcm2711_class_init,
 #endif
     }, {
         .name           = TYPE_BCM283X,
diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c
index 3fb1c3138b..03f54887f4 100644
--- a/hw/arm/raspi.c
+++ b/hw/arm/raspi.c
@@ -71,6 +71,7 @@ typedef enum RaspiProcessorId {
     PROCESSOR_ID_BCM2835 = 0,
     PROCESSOR_ID_BCM2836 = 1,
     PROCESSOR_ID_BCM2837 = 2,
+    PROCESSOR_ID_BCM2711 = 3,
 } RaspiProcessorId;
 
 static const struct {
@@ -80,6 +81,7 @@ static const struct {
     [PROCESSOR_ID_BCM2835] = {TYPE_BCM2835, 1},
     [PROCESSOR_ID_BCM2836] = {TYPE_BCM2836, BCM283X_NCPUS},
     [PROCESSOR_ID_BCM2837] = {TYPE_BCM2837, BCM283X_NCPUS},
+    [PROCESSOR_ID_BCM2711] = {TYPE_BCM2711, BCM283X_NCPUS},
 };
 
 static uint64_t board_ram_size(uint32_t board_rev)
-- 
2.30.2

[PATCH v1 6/7] hw/arm/bcm2838: Map the PCIe memory space

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

The BCM2711 has a BCM54213 Gigabit Ethernet block mapped in the PCIe
range.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
FIXME: create the bcm54213-geth in raspi.c?
vAJB:
  - fix for move of gic_base to bc
---
 hw/arm/bcm2836.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/hw/arm/bcm2836.c b/hw/arm/bcm2836.c
index 99dc15e6e4..ff62e741ce 100644
--- a/hw/arm/bcm2836.c
+++ b/hw/arm/bcm2836.c
@@ -15,6 +15,7 @@
 #include "hw/arm/bcm2836.h"
 #include "hw/arm/raspi_platform.h"
 #include "hw/sysbus.h"
+#include "hw/misc/unimp.h"
 
 typedef struct BCM283XClass {
     /*< private >*/
@@ -46,6 +47,8 @@ static Property bcm2836_enabled_cores_property =
 #define GIC_VIFACE_OTHER_OFS(cpu)  (0x5000 + (cpu) * 0x200)
 #define GIC_VCPU_OFS                0x6000
 
+#define PCIE_BASE                   0x7d500000
+
 static void bcm2836_init(Object *obj)
 {
     BCM283XState *s = BCM283X(obj);
@@ -227,6 +230,13 @@ static void bcm2836_realize(DeviceState *dev, Error **errp)
         qdev_connect_gpio_out(DEVICE(&s->cpu[n].core), GTIMER_SEC,
                 qdev_get_gpio_in_named(DEVICE(&s->control), "cntpsirq", n));
     }
+
+    /* bcm2838 kludge to easily create PCIe */
+    if (bc->gic_base) {
+        create_unimplemented_device("bcm2838-pcie", PCIE_BASE, 0x100000);
+        create_unimplemented_device("bcm54213-geth",
+                                    PCIE_BASE + 0x80000, 0x10000);
+    }
 }
 
 static void bcm283x_class_init(ObjectClass *oc, void *data)
-- 
2.30.2

[PATCH v1 7/7] hw/arm/raspi: Add the Raspberry Pi 4B board

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

The Raspberry Pi 4 uses a BCM2711 SoC (based on a BCM2838).

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
vAJB:
  - use 8gb version
---
 hw/arm/raspi.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c
index 03f54887f4..0342aeab03 100644
--- a/hw/arm/raspi.c
+++ b/hw/arm/raspi.c
@@ -369,6 +369,15 @@ static void raspi3b_machine_class_init(ObjectClass *oc, void *data)
     rmc->board_rev = 0xa02082;
     raspi_machine_class_common_init(mc, rmc->board_rev);
 };
+
+static void raspi4b_machine_class_init(ObjectClass *oc, void *data)
+{
+    MachineClass *mc = MACHINE_CLASS(oc);
+    RaspiMachineClass *rmc = RASPI_MACHINE_CLASS(oc);
+
+    rmc->board_rev = 0xd03114;
+    raspi_machine_class_common_init(mc, rmc->board_rev);
+};
 #endif /* TARGET_AARCH64 */
 
 static const TypeInfo raspi_machine_types[] = {
@@ -393,6 +402,10 @@ static const TypeInfo raspi_machine_types[] = {
         .name           = MACHINE_TYPE_NAME("raspi3b"),
         .parent         = TYPE_RASPI_MACHINE,
         .class_init     = raspi3b_machine_class_init,
+    }, {
+        .name           = MACHINE_TYPE_NAME("raspi4b"),
+        .parent         = TYPE_RASPI_MACHINE,
+        .class_init     = raspi4b_machine_class_init,
 #endif
     }, {
         .name           = TYPE_RASPI_MACHINE,
-- 
2.30.2

Re: [PATCH v1 5/7] hw/arm/bcm2836: Add the BCM2711 which uses a GICv2

[Alex Bennée]

Alex Bennée <alex.bennee@linaro.org> writes:

> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
> The BCM2711 is improvement of the BCM2837:
> - Cortex-A72 instead of the A53
> - peripheral block and local soc controller are mapped differently,
> - GICv2
> - PCIe block
> - exhanced MMU to address over 4GiB of SDRAM
>
<snip>
> ---
> vAJB:
<snip>
>   - move peri_base/ctrl to locations pointed to by DTB (0x7e000000/0x40000000)
<snip>
> +
> +static void bcm2711_class_init(ObjectClass *oc, void *data)
> +{
> +    DeviceClass *dc = DEVICE_CLASS(oc);
> +    BCM283XClass *bc = BCM283X_CLASS(oc);
> +
> +    bc->cpu_type = ARM_CPU_TYPE_NAME("cortex-a72");
> +    bc->core_count = BCM283X_NCPUS;
> +    bc->peri_base = 0x7e000000;
> +    bc->ctrl_base = 0x40000000;
> +    bc->clusterid = 0x0;
> +    bc->gic_base = 0x40000,
> +    dc->realize = bcm2836_realize;
> +}

It turns out I was misreading the way you calculate addresses from DTS
files. Reverted to:

    bc->peri_base = 0xfe000000;
    bc->ctrl_base = 0xff800000;

-- 
Alex Bennée