* [PATCH v2] aspeed: Add boot stub for smp booting
@ 2020-04-09 6:31 Joel Stanley
2020-04-16 16:48 ` Peter Maydell
0 siblings, 1 reply; 3+ messages in thread
From: Joel Stanley @ 2020-04-09 6:31 UTC (permalink / raw)
To: Cédric Le Goater, Peter Maydell
Cc: Andrew Jeffery, qemu-arm, Philippe Mathieu-Daudé, qemu-devel
This is a boot stub that is similar to the code u-boot runs, allowing
the kernel to boot the secondary CPU.
u-boot works as follows:
1. Initialises the SMP mailbox area in the SCU at 0x1e6e2180 with default values
2. Copies a stub named 'mailbox_insn' from flash to the SCU, just above the
mailbox area
3. Sets AST_SMP_MBOX_FIELD_READY to a magic value to indicate the
secondary can begin execution from the stub
4. The stub waits until the AST_SMP_MBOX_FIELD_GOSIGN register is set to
a magic value
5. Jumps to the address in AST_SMP_MBOX_FIELD_ENTRY, starting Linux
Linux indicates it is ready by writing the address of its entrypoint
function to AST_SMP_MBOX_FIELD_ENTRY and the 'go' magic number to
AST_SMP_MBOX_FIELD_GOSIGN. The secondary CPU sees this at step 4 and
breaks out of it's loop.
To be compatible, a fixed qemu stub is loaded into the mailbox area. As
qemu can ensure the stub is loaded before execution starts, we do not
need to emulate the AST_SMP_MBOX_FIELD_READY behaviour of u-boot. The
secondary CPU's program counter points to the beginning of the stub,
allowing qemu to start secondaries at step four.
Reboot behaviour is preserved by resetting AST_SMP_MBOX_FIELD_GOSIGN
when the secondaries are reset.
This is only configured when the system is booted with -kernel and qemu
does not execute u-boot first.
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Tested-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Joel Stanley <joel@jms.id.au>
---
v2: test for number of CPUs
---
hw/arm/aspeed.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 65 insertions(+)
diff --git a/hw/arm/aspeed.c b/hw/arm/aspeed.c
index 88bcd6ff3fbd..e363e495ef48 100644
--- a/hw/arm/aspeed.c
+++ b/hw/arm/aspeed.c
@@ -116,6 +116,58 @@ static const MemoryRegionOps max_ram_ops = {
.endianness = DEVICE_NATIVE_ENDIAN,
};
+#define AST_SMP_MAILBOX_BASE 0x1e6e2180
+#define AST_SMP_MBOX_FIELD_ENTRY (AST_SMP_MAILBOX_BASE + 0x0)
+#define AST_SMP_MBOX_FIELD_GOSIGN (AST_SMP_MAILBOX_BASE + 0x4)
+#define AST_SMP_MBOX_FIELD_READY (AST_SMP_MAILBOX_BASE + 0x8)
+#define AST_SMP_MBOX_FIELD_POLLINSN (AST_SMP_MAILBOX_BASE + 0xc)
+#define AST_SMP_MBOX_CODE (AST_SMP_MAILBOX_BASE + 0x10)
+#define AST_SMP_MBOX_GOSIGN 0xabbaab00
+
+static void aspeed_write_smpboot(ARMCPU *cpu,
+ const struct arm_boot_info *info)
+{
+ static const uint32_t poll_mailbox_ready[] = {
+ /*
+ * r2 = per-cpu go sign value
+ * r1 = AST_SMP_MBOX_FIELD_ENTRY
+ * r0 = AST_SMP_MBOX_FIELD_GOSIGN
+ */
+ 0xee100fb0, /* mrc p15, 0, r0, c0, c0, 5 */
+ 0xe21000ff, /* ands r0, r0, #255 */
+ 0xe59f201c, /* ldr r2, [pc, #28] */
+ 0xe1822000, /* orr r2, r2, r0 */
+
+ 0xe59f1018, /* ldr r1, [pc, #24] */
+ 0xe59f0018, /* ldr r0, [pc, #24] */
+
+ 0xe320f002, /* wfe */
+ 0xe5904000, /* ldr r4, [r0] */
+ 0xe1520004, /* cmp r2, r4 */
+ 0x1afffffb, /* bne <wfe> */
+ 0xe591f000, /* ldr pc, [r1] */
+ AST_SMP_MBOX_GOSIGN,
+ AST_SMP_MBOX_FIELD_ENTRY,
+ AST_SMP_MBOX_FIELD_GOSIGN,
+ };
+
+ rom_add_blob_fixed("aspeed.smpboot", poll_mailbox_ready,
+ sizeof(poll_mailbox_ready),
+ info->smp_loader_start);
+}
+
+static void aspeed_reset_secondary(ARMCPU *cpu,
+ const struct arm_boot_info *info)
+{
+ AddressSpace *as = arm_boot_address_space(cpu, info);
+ CPUState *cs = CPU(cpu);
+
+ /* info->smp_bootreg_addr */
+ address_space_stl_notdirty(as, AST_SMP_MBOX_FIELD_GOSIGN, 0,
+ MEMTXATTRS_UNSPECIFIED, NULL);
+ cpu_set_pc(cs, info->smp_loader_start);
+}
+
#define FIRMWARE_ADDR 0x0
static void write_boot_rom(DriveInfo *dinfo, hwaddr addr, size_t rom_size,
@@ -270,6 +322,19 @@ static void aspeed_machine_init(MachineState *machine)
}
}
+ if (machine->kernel_filename && bmc->soc.num_cpus > 1) {
+ /* With no u-boot we must set up a boot stub for the secondary CPU */
+ MemoryRegion *smpboot = g_new(MemoryRegion, 1);
+ memory_region_init_ram(smpboot, OBJECT(bmc), "aspeed.smpboot",
+ 0x80, &error_abort);
+ memory_region_add_subregion(get_system_memory(),
+ AST_SMP_MAILBOX_BASE, smpboot);
+
+ aspeed_board_binfo.write_secondary_boot = aspeed_write_smpboot;
+ aspeed_board_binfo.secondary_cpu_reset_hook = aspeed_reset_secondary;
+ aspeed_board_binfo.smp_loader_start = AST_SMP_MBOX_CODE;
+ }
+
aspeed_board_binfo.ram_size = ram_size;
aspeed_board_binfo.loader_start = sc->memmap[ASPEED_SDRAM];
aspeed_board_binfo.nb_cpus = bmc->soc.num_cpus;
--
2.25.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v2] aspeed: Add boot stub for smp booting
2020-04-09 6:31 [PATCH v2] aspeed: Add boot stub for smp booting Joel Stanley
@ 2020-04-16 16:48 ` Peter Maydell
2020-05-04 8:18 ` Joel Stanley
0 siblings, 1 reply; 3+ messages in thread
From: Peter Maydell @ 2020-04-16 16:48 UTC (permalink / raw)
To: Joel Stanley
Cc: Andrew Jeffery, qemu-arm, Philippe Mathieu-Daudé,
Cédric Le Goater, QEMU Developers
On Thu, 9 Apr 2020 at 07:31, Joel Stanley <joel@jms.id.au> wrote:
>
> This is a boot stub that is similar to the code u-boot runs, allowing
> the kernel to boot the secondary CPU.
> +static void aspeed_write_smpboot(ARMCPU *cpu,
> + const struct arm_boot_info *info)
> +{
> + static const uint32_t poll_mailbox_ready[] = {
> + /*
> + * r2 = per-cpu go sign value
> + * r1 = AST_SMP_MBOX_FIELD_ENTRY
> + * r0 = AST_SMP_MBOX_FIELD_GOSIGN
> + */
> + 0xee100fb0, /* mrc p15, 0, r0, c0, c0, 5 */
> + 0xe21000ff, /* ands r0, r0, #255 */
> + 0xe59f201c, /* ldr r2, [pc, #28] */
> + 0xe1822000, /* orr r2, r2, r0 */
> +
> + 0xe59f1018, /* ldr r1, [pc, #24] */
> + 0xe59f0018, /* ldr r0, [pc, #24] */
> +
> + 0xe320f002, /* wfe */
> + 0xe5904000, /* ldr r4, [r0] */
> + 0xe1520004, /* cmp r2, r4 */
> + 0x1afffffb, /* bne <wfe> */
Note that unlike "wfi", QEMU's "wfe" implementation is merely
a 'yield', so a secondary-CPU boot loop that has wfe in it
will basically be a busy-loop of those vcpu threads.
(This is why the smpboot code in hw/arm/boot.c uses wfi.)
I don't suppose the secondary boot protocol on these boards
is such that a wfi loop will work ? (Depends on what the
primary code in the kernel does to prod the secondary after
writing the magic value.)
> + 0xe591f000, /* ldr pc, [r1] */
> + AST_SMP_MBOX_GOSIGN,
> + AST_SMP_MBOX_FIELD_ENTRY,
> + AST_SMP_MBOX_FIELD_GOSIGN,
> + };
thanks
-- PMM
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v2] aspeed: Add boot stub for smp booting
2020-04-16 16:48 ` Peter Maydell
@ 2020-05-04 8:18 ` Joel Stanley
0 siblings, 0 replies; 3+ messages in thread
From: Joel Stanley @ 2020-05-04 8:18 UTC (permalink / raw)
To: Peter Maydell
Cc: Andrew Jeffery, qemu-arm, Philippe Mathieu-Daudé,
Cédric Le Goater, QEMU Developers
On Thu, 16 Apr 2020 at 16:48, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Thu, 9 Apr 2020 at 07:31, Joel Stanley <joel@jms.id.au> wrote:
> >
> > This is a boot stub that is similar to the code u-boot runs, allowing
> > the kernel to boot the secondary CPU.
>
> > +static void aspeed_write_smpboot(ARMCPU *cpu,
> > + const struct arm_boot_info *info)
> > +{
> > + static const uint32_t poll_mailbox_ready[] = {
> > + /*
> > + * r2 = per-cpu go sign value
> > + * r1 = AST_SMP_MBOX_FIELD_ENTRY
> > + * r0 = AST_SMP_MBOX_FIELD_GOSIGN
> > + */
> > + 0xee100fb0, /* mrc p15, 0, r0, c0, c0, 5 */
> > + 0xe21000ff, /* ands r0, r0, #255 */
> > + 0xe59f201c, /* ldr r2, [pc, #28] */
> > + 0xe1822000, /* orr r2, r2, r0 */
> > +
> > + 0xe59f1018, /* ldr r1, [pc, #24] */
> > + 0xe59f0018, /* ldr r0, [pc, #24] */
> > +
> > + 0xe320f002, /* wfe */
> > + 0xe5904000, /* ldr r4, [r0] */
> > + 0xe1520004, /* cmp r2, r4 */
> > + 0x1afffffb, /* bne <wfe> */
>
> Note that unlike "wfi", QEMU's "wfe" implementation is merely
> a 'yield', so a secondary-CPU boot loop that has wfe in it
> will basically be a busy-loop of those vcpu threads.
> (This is why the smpboot code in hw/arm/boot.c uses wfi.)
>
> I don't suppose the secondary boot protocol on these boards
> is such that a wfi loop will work ? (Depends on what the
> primary code in the kernel does to prod the secondary after
> writing the magic value.)
Thanks for the review.
I chose wfe as it's what aspeed's u-boot has. I don't have a strong
understand of this area of ARM processors, nor do I have any insight
into why that choice was made.
I did some testing and wfi in the qemu stub works fine with Linux so I
will respin with that instead.
Cheers,
Joel
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-05-04 8:21 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-09 6:31 [PATCH v2] aspeed: Add boot stub for smp booting Joel Stanley
2020-04-16 16:48 ` Peter Maydell
2020-05-04 8:18 ` Joel Stanley
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.