Re: [PATCH/RFC 0/4] ARM: shmobile: Reserve boot area when SMP is enabled

From: Geert Uytterhoeven <geert@linux-m68k.org>
To: Geert Uytterhoeven <geert+renesas@glider.be>
Cc: Magnus Damm <magnus.damm@gmail.com>,
	Marek Vasut <marek.vasut+renesas@gmail.com>,
	Russell King <linux@armlinux.org.uk>,
	Arnd Bergmann <arnd@arndb.de>, Ard Biesheuvel <ardb@kernel.org>,
	Linus Walleij <linus.walleij@linaro.org>,
	linux-renesas-soc@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	linux-mtd@lists.infradead.org
Subject: Re: [PATCH/RFC 0/4] ARM: shmobile: Reserve boot area when SMP is enabled
Date: Mon, 25 Sep 2023 09:15:17 +0200	[thread overview]
Message-ID: <CAMuHMdVvX9F2-Bkwkw0ihTxfuQ3C2hHK3sQTjpqMq07w2f84aw@mail.gmail.com> (raw)
In-Reply-To: <cover.1693409184.git.geert+renesas@glider.be>

On Thu, Aug 31, 2023 at 1:17 PM Geert Uytterhoeven
<geert+renesas@glider.be> wrote:
> On Renesas ARM SoCs predating R-Car Gen3, CPU core bringup does not use
> a hardware register to program the CPU boot address directly.  Instead,
> it uses a set of registers (thus not involving the MMU) to remap any CPU
> access to the page(s) at physical address zero to the specified address
> block containing the CPU core startup code.  the MMU.  Hence when this
> is enabled, any device residing in this low part of physical address
> space cannot be accessed.  On some boards, a CFI FLASH lives there.
>
> Hence this causes conflicts between CPU onlining and FLASH operation:
>   - Reading the first page(s) of FLASH returns the CPU startup code
>     instead of the FLASH contents,
>   - CFI FLASH probing fails, as it operates on the RAM that contains the
>     CPU startup code.  Moreover, as this probing involves writes, it
>     corrupts the CPU startup code, causing any subsequent CPU onlining
>     to fail.
>
> CPU cores can be onlined in 3 places:
>   A. Secondary core bringup, during early boot (FLASH not yet in use),
>   B. Enabling secondary CPUs when resuming after s2ram (FLASH not in
>      use),
>   C. Manual CPU hotplug (at any time, FLASH may or may not be in use).
>
> Possible solutions:
>   1. Disable FLASH in SMP initialization:
>        - Simple to implement,
>        - Accessing the FLASH from Linux needs a reboot with nosmp.
>
>   2. Disable SMP when FLASH@0 is used:
>        - How to check if @0 is actually used/mapped?
>          SMP is initialized before FLASH.
>
>   3. Map core startup code only when needed:
>       - Unmap core startup code after secondary core bringup,
>       - Map/unmap core startup code during s2ram suspend/resume,
>       - Call cpu_hotplug_disable() from smp_cpus_done(),
>           - CPU hotplug is broken.
>           - How to check if @0 is actually used/mapped?
>
>   4. As this FLASH is typically used to boot the system, Marek suggested
>      to fix this in the boot loader (e.g. put a (modified) copy of the
>      CPU bringup code in FLASH).  But I think this is fragile, and cannot
>      be a generic solution, as there are other ways to boot the system,
>      and the FLASH may be used for other purposes.
>
> This patch series implements solution 1, by marking the boot area region
> reserved during SMP initialization (an alternative method would be to
> disable any device node in DT that resides in this area).  Subsequent
> probing of FLASH will fail with -EBUSY:
>
>     physmap-flash 0.flash: can't request region for resource [mem 0x00000000-0x03ffffff]
>     physmap-flash: probe of 0.flash failed with error -16
>
> When CONFIG_SMP is disabled, or when booted with "nosmp", the FLASH
> is available again.
>
> The first patch is a small cleanup in code affected by the third patch.
> The other three patches fix the issue on R-Car Gen2 (and RZ/G1), R-Car
> H1, and SH-Mobile AG5 SoCs.
>
> Other Renesas SoCs:
>   - R-Mobile APE6 is also affected, but has no upstream SMP support yet,
>   - EMMA Mobile EV2 is unaffected, as it uses internal boot ROM code
>     that watches a special general purpose register in the SMU block,
>   - RZ/N1 is not affected, as it uses a "cpu-release-addr" property in
>     DT,
>   - R-Car Gen3 is not affected, as it extended the R-Car Gen2 RST block
>     with Cortex-A53/A57 Boot Address Registers for 64-bit mode
>     (CA5[37]CPUnBAR[HL]), which control the boot address directly,
>   - R-Car Gen4 is not affected, as it has Reset Vector Base Address
>     Registers (RVBAR[HL]Cn), which control the boot address directly.
>
> This series has been tested on R-Car V2H (Blanche) and R-Car H1
> (Marzen).  With this, the CFI FLASHes on Marzen[1] and Blanche (DTS
> patch to be posted) work when booted with nosmp.
>
> Thanks for your comments!
>
> [1] "[PATCH/RFC] ARM: dts: marzen: Add FLASH node"
>     https://lore.kernel.org/r/07cf5e2b466f3ba217403afc66a8246460609e09.1679330105.git.geert+renesas@glider.be/
>
> Geert Uytterhoeven (4):
>   ARM: shmobile: rcar-gen2: Remove unneeded once handling
>   ARM: shmobile: rcar-gen2: Reserve boot area when SMP is enabled
>   ARM: shmobile: r8a7779: Reserve boot area when SMP is enabled
>   ARM: shmobile: sh73a0: Reserve boot area when SMP is enabled

Thanks, queuing in renesas-devel for v6.7.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds