[PATCH/RFC 0/4] ARM: shmobile: Reserve boot area when SMP is enabled

[PATCH/RFC 0/4] ARM: shmobile: Reserve boot area when SMP is enabled

[Geert Uytterhoeven]

	Hi all,

On Renesas ARM SoCs predating R-Car Gen3, CPU core bringup does not use
a hardware register to program the CPU boot address directly.  Instead,
it uses a set of registers (thus not involving the MMU) to remap any CPU
access to the page(s) at physical address zero to the specified address
block containing the CPU core startup code.  the MMU.  Hence when this
is enabled, any device residing in this low part of physical address
space cannot be accessed.  On some boards, a CFI FLASH lives there.

Hence this causes conflicts between CPU onlining and FLASH operation:
  - Reading the first page(s) of FLASH returns the CPU startup code
    instead of the FLASH contents,
  - CFI FLASH probing fails, as it operates on the RAM that contains the
    CPU startup code.  Moreover, as this probing involves writes, it
    corrupts the CPU startup code, causing any subsequent CPU onlining
    to fail.

CPU cores can be onlined in 3 places:
  A. Secondary core bringup, during early boot (FLASH not yet in use),
  B. Enabling secondary CPUs when resuming after s2ram (FLASH not in
     use),
  C. Manual CPU hotplug (at any time, FLASH may or may not be in use).

Possible solutions:
  1. Disable FLASH in SMP initialization:
       - Simple to implement,
       - Accessing the FLASH from Linux needs a reboot with nosmp.

  2. Disable SMP when FLASH@0 is used:
       - How to check if @0 is actually used/mapped?
         SMP is initialized before FLASH.

  3. Map core startup code only when needed:
      - Unmap core startup code after secondary core bringup,
      - Map/unmap core startup code during s2ram suspend/resume,
      - Call cpu_hotplug_disable() from smp_cpus_done(),
	  - CPU hotplug is broken.
	  - How to check if @0 is actually used/mapped?

  4. As this FLASH is typically used to boot the system, Marek suggested
     to fix this in the boot loader (e.g. put a (modified) copy of the
     CPU bringup code in FLASH).  But I think this is fragile, and cannot
     be a generic solution, as there are other ways to boot the system,
     and the FLASH may be used for other purposes.

This patch series implements solution 1, by marking the boot area region
reserved during SMP initialization (an alternative method would be to
disable any device node in DT that resides in this area).  Subsequent
probing of FLASH will fail with -EBUSY:

    physmap-flash 0.flash: can't request region for resource [mem 0x00000000-0x03ffffff]
    physmap-flash: probe of 0.flash failed with error -16

When CONFIG_SMP is disabled, or when booted with "nosmp", the FLASH
is available again.

The first patch is a small cleanup in code affected by the third patch.
The other three patches fix the issue on R-Car Gen2 (and RZ/G1), R-Car
H1, and SH-Mobile AG5 SoCs.

Other Renesas SoCs:
  - R-Mobile APE6 is also affected, but has no upstream SMP support yet,
  - EMMA Mobile EV2 is unaffected, as it uses internal boot ROM code
    that watches a special general purpose register in the SMU block,
  - RZ/N1 is not affected, as it uses a "cpu-release-addr" property in
    DT,
  - R-Car Gen3 is not affected, as it extended the R-Car Gen2 RST block
    with Cortex-A53/A57 Boot Address Registers for 64-bit mode
    (CA5[37]CPUnBAR[HL]), which control the boot address directly,
  - R-Car Gen4 is not affected, as it has Reset Vector Base Address
    Registers (RVBAR[HL]Cn), which control the boot address directly.

This series has been tested on R-Car V2H (Blanche) and R-Car H1
(Marzen).  With this, the CFI FLASHes on Marzen[1] and Blanche (DTS
patch to be posted) work when booted with nosmp.

Thanks for your comments!

[1] "[PATCH/RFC] ARM: dts: marzen: Add FLASH node"
    https://lore.kernel.org/r/07cf5e2b466f3ba217403afc66a8246460609e09.1679330105.git.geert+renesas@glider.be/

Geert Uytterhoeven (4):
  ARM: shmobile: rcar-gen2: Remove unneeded once handling
  ARM: shmobile: rcar-gen2: Reserve boot area when SMP is enabled
  ARM: shmobile: r8a7779: Reserve boot area when SMP is enabled
  ARM: shmobile: sh73a0: Reserve boot area when SMP is enabled

 arch/arm/mach-shmobile/pm-rcar-gen2.c |  5 +++--
 arch/arm/mach-shmobile/smp-r8a7779.c  |  9 ++++++++-
 arch/arm/mach-shmobile/smp-sh73a0.c   | 10 ++++++++--
 3 files changed, 19 insertions(+), 5 deletions(-)

-- 
2.34.1

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 1/4] ARM: shmobile: rcar-gen2: Remove unneeded once handling

[Geert Uytterhoeven]

rcar_gen2_pm_init() is only called from the
smp_operations.smp_prepare_cpus() callback, which is called at most
once.

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
 arch/arm/mach-shmobile/pm-rcar-gen2.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/arch/arm/mach-shmobile/pm-rcar-gen2.c b/arch/arm/mach-shmobile/pm-rcar-gen2.c
index 672081405a7ed7aa..7447e5fd7ed41e99 100644
--- a/arch/arm/mach-shmobile/pm-rcar-gen2.c
+++ b/arch/arm/mach-shmobile/pm-rcar-gen2.c
@@ -46,16 +46,12 @@ void __init rcar_gen2_pm_init(void)
 {
 	void __iomem *p;
 	u32 bar;
-	static int once;
 	struct device_node *np;
 	bool has_a7 = false;
 	bool has_a15 = false;
 	struct resource res;
 	int error;
 
-	if (once++)
-		return;
-
 	for_each_of_cpu_node(np) {
 		if (of_device_is_compatible(np, "arm,cortex-a15"))
 			has_a15 = true;
-- 
2.34.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH/RFC 2/4] ARM: shmobile: rcar-gen2: Reserve boot area when SMP is enabled

[Geert Uytterhoeven]

CPU core bringup on R-Car Gen2 SoCs uses the Cortex-A7/A15 Boot Address
Register to specify the boot area of the System CPU.  With this enabled,
when the System CPU accesses a physical address in the range from 0x0 to
0x3ffff, the top address bits are replaced by those specified in the
SBAR register.  Hence any device residing in the low 256 KiB of physical
address space cannot be accessed.

Prevent conflicts by reserving this memory region using
request_mem_region().

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
 arch/arm/mach-shmobile/pm-rcar-gen2.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/arm/mach-shmobile/pm-rcar-gen2.c b/arch/arm/mach-shmobile/pm-rcar-gen2.c
index 7447e5fd7ed41e99..3453d5733224df44 100644
--- a/arch/arm/mach-shmobile/pm-rcar-gen2.c
+++ b/arch/arm/mach-shmobile/pm-rcar-gen2.c
@@ -52,6 +52,11 @@ void __init rcar_gen2_pm_init(void)
 	struct resource res;
 	int error;
 
+	if (!request_mem_region(0, SZ_256K, "Boot Area")) {
+		pr_err("Failed to request boot area\n");
+		return;
+	};
+
 	for_each_of_cpu_node(np) {
 		if (of_device_is_compatible(np, "arm,cortex-a15"))
 			has_a15 = true;
-- 
2.34.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH/RFC 3/4] ARM: shmobile: r8a7779: Reserve boot area when SMP is enabled

[Geert Uytterhoeven]

CPU core bringup on R-Car H1 uses the ARM Reset Vector Address Register
(AVECR) to specify the base address and size of the boot area of the
System CPU.  With this enabled, when the System CPU accesses a physical
address in the range from zero up to the configured size, the top
address bits are replaced by those specified in the AVECR register.
Hence any device residing in this low part of physical address space
cannot be accessed,

Prevent conflicts by reserving this memory region using
request_mem_region().

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
 arch/arm/mach-shmobile/smp-r8a7779.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/arm/mach-shmobile/smp-r8a7779.c b/arch/arm/mach-shmobile/smp-r8a7779.c
index 1bc609986c160d2b..2b1b8248450c8dd2 100644
--- a/arch/arm/mach-shmobile/smp-r8a7779.c
+++ b/arch/arm/mach-shmobile/smp-r8a7779.c
@@ -38,7 +38,14 @@ static int r8a7779_boot_secondary(unsigned int cpu, struct task_struct *idle)
 
 static void __init r8a7779_smp_prepare_cpus(unsigned int max_cpus)
 {
-	void __iomem *base = ioremap(HPBREG_BASE, 0x1000);
+	void __iomem *base;
+
+	if (!request_mem_region(0, SZ_4K, "Boot Area")) {
+		pr_err("Failed to request boot area\n");
+		return;
+	};
+
+	base = ioremap(HPBREG_BASE, 0x1000);
 
 	/* Map the reset vector (in headsmp-scu.S, headsmp.S) */
 	writel(__pa(shmobile_boot_vector), base + AVECR);
-- 
2.34.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH/RFC 4/4] ARM: shmobile: sh73a0: Reserve boot area when SMP is enabled

[Geert Uytterhoeven]

CPU core bringup on SH-Mobile AG5 uses the SYS Boot Address (SBAR) and
Address Translation Area (APARMBAREA) registers to specify the base
address and size of the boot area of the System CPU.  With this enabled,
when the System CPU accesses a physical address in the range from zero
up to the configured size, the top address bits are replaced by those
specified in the SBAR register.  Hence any device residing in this low
part of physical address space cannot be accessed.

Prevent conflicts by reserving this memory region using
request_mem_region().

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
 arch/arm/mach-shmobile/smp-sh73a0.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/arch/arm/mach-shmobile/smp-sh73a0.c b/arch/arm/mach-shmobile/smp-sh73a0.c
index 453d488650295842..3757aed64c4241a5 100644
--- a/arch/arm/mach-shmobile/smp-sh73a0.c
+++ b/arch/arm/mach-shmobile/smp-sh73a0.c
@@ -44,10 +44,16 @@ static int sh73a0_boot_secondary(unsigned int cpu, struct task_struct *idle)
 
 static void __init sh73a0_smp_prepare_cpus(unsigned int max_cpus)
 {
-	void __iomem *ap = ioremap(AP_BASE, PAGE_SIZE);
-	void __iomem *sysc = ioremap(SYSC_BASE, PAGE_SIZE);
+	void __iomem *ap, *sysc;
+
+	if (!request_mem_region(0, SZ_4K, "Boot Area")) {
+		pr_err("Failed to request boot area\n");
+		return;
+	};
 
 	/* Map the reset vector (in headsmp.S) */
+	ap = ioremap(AP_BASE, PAGE_SIZE);
+	sysc = ioremap(SYSC_BASE, PAGE_SIZE);
 	writel(0, ap + APARMBAREA);      /* 4k */
 	writel(__pa(shmobile_boot_vector), sysc + SBAR);
 	iounmap(sysc);
-- 
2.34.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH/RFC 0/4] ARM: shmobile: Reserve boot area when SMP is enabled

[Geert Uytterhoeven]

On Thu, Aug 31, 2023 at 1:17 PM Geert Uytterhoeven
<geert+renesas@glider.be> wrote:
> On Renesas ARM SoCs predating R-Car Gen3, CPU core bringup does not use
> a hardware register to program the CPU boot address directly.  Instead,
> it uses a set of registers (thus not involving the MMU) to remap any CPU
> access to the page(s) at physical address zero to the specified address
> block containing the CPU core startup code.  the MMU.  Hence when this
> is enabled, any device residing in this low part of physical address
> space cannot be accessed.  On some boards, a CFI FLASH lives there.
>
> Hence this causes conflicts between CPU onlining and FLASH operation:
>   - Reading the first page(s) of FLASH returns the CPU startup code
>     instead of the FLASH contents,
>   - CFI FLASH probing fails, as it operates on the RAM that contains the
>     CPU startup code.  Moreover, as this probing involves writes, it
>     corrupts the CPU startup code, causing any subsequent CPU onlining
>     to fail.
>
> CPU cores can be onlined in 3 places:
>   A. Secondary core bringup, during early boot (FLASH not yet in use),
>   B. Enabling secondary CPUs when resuming after s2ram (FLASH not in
>      use),
>   C. Manual CPU hotplug (at any time, FLASH may or may not be in use).
>
> Possible solutions:
>   1. Disable FLASH in SMP initialization:
>        - Simple to implement,
>        - Accessing the FLASH from Linux needs a reboot with nosmp.
>
>   2. Disable SMP when FLASH@0 is used:
>        - How to check if @0 is actually used/mapped?
>          SMP is initialized before FLASH.
>
>   3. Map core startup code only when needed:
>       - Unmap core startup code after secondary core bringup,
>       - Map/unmap core startup code during s2ram suspend/resume,
>       - Call cpu_hotplug_disable() from smp_cpus_done(),
>           - CPU hotplug is broken.
>           - How to check if @0 is actually used/mapped?
>
>   4. As this FLASH is typically used to boot the system, Marek suggested
>      to fix this in the boot loader (e.g. put a (modified) copy of the
>      CPU bringup code in FLASH).  But I think this is fragile, and cannot
>      be a generic solution, as there are other ways to boot the system,
>      and the FLASH may be used for other purposes.
>
> This patch series implements solution 1, by marking the boot area region
> reserved during SMP initialization (an alternative method would be to
> disable any device node in DT that resides in this area).  Subsequent
> probing of FLASH will fail with -EBUSY:
>
>     physmap-flash 0.flash: can't request region for resource [mem 0x00000000-0x03ffffff]
>     physmap-flash: probe of 0.flash failed with error -16
>
> When CONFIG_SMP is disabled, or when booted with "nosmp", the FLASH
> is available again.
>
> The first patch is a small cleanup in code affected by the third patch.
> The other three patches fix the issue on R-Car Gen2 (and RZ/G1), R-Car
> H1, and SH-Mobile AG5 SoCs.
>
> Other Renesas SoCs:
>   - R-Mobile APE6 is also affected, but has no upstream SMP support yet,
>   - EMMA Mobile EV2 is unaffected, as it uses internal boot ROM code
>     that watches a special general purpose register in the SMU block,
>   - RZ/N1 is not affected, as it uses a "cpu-release-addr" property in
>     DT,
>   - R-Car Gen3 is not affected, as it extended the R-Car Gen2 RST block
>     with Cortex-A53/A57 Boot Address Registers for 64-bit mode
>     (CA5[37]CPUnBAR[HL]), which control the boot address directly,
>   - R-Car Gen4 is not affected, as it has Reset Vector Base Address
>     Registers (RVBAR[HL]Cn), which control the boot address directly.
>
> This series has been tested on R-Car V2H (Blanche) and R-Car H1
> (Marzen).  With this, the CFI FLASHes on Marzen[1] and Blanche (DTS
> patch to be posted) work when booted with nosmp.
>
> Thanks for your comments!
>
> [1] "[PATCH/RFC] ARM: dts: marzen: Add FLASH node"
>     https://lore.kernel.org/r/07cf5e2b466f3ba217403afc66a8246460609e09.1679330105.git.geert+renesas@glider.be/
>
> Geert Uytterhoeven (4):
>   ARM: shmobile: rcar-gen2: Remove unneeded once handling
>   ARM: shmobile: rcar-gen2: Reserve boot area when SMP is enabled
>   ARM: shmobile: r8a7779: Reserve boot area when SMP is enabled
>   ARM: shmobile: sh73a0: Reserve boot area when SMP is enabled

Thanks, queuing in renesas-devel for v6.7.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel