RE: STM32MP1: Adding TF-A causes kernel errors

[Patrick DELAUNAY <patrick.delaunay@st.com>]

Hi Jan,

> From: Jan Kiszka <jan.kiszka@siemens.com>
> Sent: mardi 13 octobre 2020 00:02
> 
> On 05.10.20 08:07, Jan Kiszka wrote:
> > On 01.10.20 11:52, Jan Kiszka wrote:
> >> On 30.09.20 11:51, Jan Kiszka wrote:
> >>> [BCC'ed TF-A only, migrating to u-boot, including folks involved
> >>> there]
> >>>
> >>> On 30.09.20 11:20, Yann GAUTIER wrote:
> >>>> Hi Jan,
> >>>>
> >>>> After discussing with my colleagues, it seems there are 2 issues there.
> >>>> One patch is missing in U-Boot:
> >>>> http://patchwork.ozlabs.org/project/uboot/patch/20200605092244.1.I7
> >>>> 73bf523d9f4d1a6212483d030e34113b832a779@changeid/
> >>>>
> >>>
> >>> I can confirm that this resolves the errors I've seen.
> >>>
> >>
> >> Picking up again, this time for OP-TEE:
> >> Do I need more patches, wherever, to get that one running as well?
> >>
> >> NOTICE:  CPU: STM32MP157AAA Rev.B
> >> NOTICE:  Model: STMicroelectronics STM32MP157C eval daughter on eval
> mother
> >> NOTICE:  Board: MB1263 Var1 Rev.C-01
> >> NOTICE:  BL2: v2.3():
> >> NOTICE:  BL2: Built : 10:11:55, Sep 30 2020
> >> NOTICE:  BL2: Booting BL32
> >> I/TC: Early console on UART#4
> >> I/TC:
> >> I/TC: Pager is enabled. Hashes: 2144 bytes
> >> I/TC: Pager pool size: 100kB
> >> I/TC: No non-secure external DT
> >> I/TC: Embedded DTB found
> >> I/TC: OP-TEE version: Unknown (gcc version 8.3.0 (Debian 8.3.0-2)) #2
> >> Thu Oct  1 06:53:58 UTC 2020 arm
> >> I/TC: Primary CPU initializing
> >> I/TC: Platform stm32mp1: flavor PLATFORM_FLAVOR - DT
> >> stm32mp157c-ev1.dts
> >> I/TC: RCC is non-secure
> >> I/TC: DTB enables console (non-secure)
> >> I/TC: Primary CPU switching to normal world boot
> >>
> >>
> >> U-Boot 2020.07 (Oct 01 2020 - 06:54:18 +0000)
> >>
> >> CPU: STM32MP157AAA Rev.B
> >> Model: STMicroelectronics STM32MP157C eval daughter on eval mother
> >> Board: stm32mp1 in trusted mode (st,stm32mp157c-ev1)
> >> Board: MB1263 Var1.0 Rev.C-01
> >> DRAM:  1 GiB
> >> Clocks:
> >> - MPU : 650 MHz
> >> - MCU : 208.878 MHz
> >> - AXI : 266.500 MHz
> >> - PER : 24 MHz
> >> - DDR : 533 MHz
> >> NAND:  1024 MiB
> >> MMC:   STM32 SD/MMC: 0, STM32 SD/MMC: 1
> >> Loading Environment from EXT4... ** File not found /uboot.env **
> >>
> >> ** Unable to read "/uboot.env" from mmc0:7 **
> >> In:    serial
> >> Out:   serial
> >> Err:   serial
> >> Net:   eth0: ethernet@5800a000
> >> Hit any key to stop autoboot:  0
> >> Boot over mmc0!
> >> Saving Environment to EXT4... Unsupported feature metadata_csum found, not
> writing.
> >>
> >> ** Unable to write "/uboot.env" from mmc0:7 ** Failed (1) switch to
> >> partitions #0, OK
> >> mmc0 is current device
> >> Scanning mmc 0:7...
> >> Found U-Boot script /boot/boot.scr
> >> 562 bytes read in 26 ms (20.5 KiB/s)
> >> ## Executing script at c4100000
> >> 57629 bytes read in 38 ms (1.4 MiB/s)
> >> 9474560 bytes read in 429 ms (21.1 MiB/s)
> >> 4410487 bytes read in 212 ms (19.8 MiB/s) Kernel image @ 0xc2000000 [
> >> 0x000000 - 0x909200 ] ## Flattened Device Tree blob at c4000000
> >>    Booting using the fdt blob at 0xc4000000
> >>    Loading Ramdisk to cfbcb000, end cffffc77 ... OK
> >>    Loading Device Tree to cfbb9000, end cfbca11c ... OK
> >> OP-TEE: revision 3.10
> >>
> >> Starting kernel ...
> >>
> >> I/TC: Secondary CI/TC: Secondary CPU 1 switching to normal world boot
> >> E/TC:1   tzc_it_handler:19 TZC permission failure
> >> E/TC:1   dump_fail_filter:417 Overrun violation on filter 0
> >> E/TC:1   dump_fail_filter:420 Permission violation on filter 0
> >> E/TC:1   dump_fail_filter:430 Violation @0xff000000, non-secure privileged
> read, AXI ID 4a0
> >> E/TC:1   Panic
> >>
> >>
> >> Besides the U-Boot patch I also have the kernel fixup for
> >> gpu_reserved applied.
> >>
> >> Thanks,
> >> Jan
> >>
> >
> > Gentle ping, at least for a pointer where to report this best.
> >
> 
> As I received no reply, I debugged that further along the line of reservations. And
> it quickly turned out that mainline is missing [1].
> With that patch applied and the gpu reservation change [2], the kernel can finally
> boot when OP-TEE is present.
> 
> Any reason why all this is only in a downstream repo?
> 
> Jan
> 
> [1]
> https://github.com/STMicroelectronics/linux/commit/d17e72a1c58a2786d60d68852
> b710a7aae95b676
> [2]
> https://github.com/STMicroelectronics/linux/commit/4707072246129cd68390e59b7
> c0dc3b878a6bf5c

Sorry for the delay.

The management of OP-TEE reserved memory was not stable in downstream
and we are upstreaming only the final solution:

1/ OP-TEE node present only in U-Boot device tree and absent in kernel device tree 

2/ the nodes is copied by U-Boot in kernel device tree
    (in lib/optee/optee.c::optee_copy_fdt_nodes() )

[1] will be never upstreamed and it will be reverted in next downstream release
    This patch avoid U-Boot copy to kernel device tree ( U-Boot don't update
    the existing op-tee  nodes)

[2] upstream is in progress => target v5.10 then U-Boot DT need to be aligned after
     But it shul be blocking for OP-TEE (it is not the root cause of the issue)

I checked the OP-TEE config and node in U-Boot: the configuration are ok for DK1 and EV1

./core/arch/arm/plat-stm32mp1/conf.mk:50:CFG_TZDRAM_START ?= 0xde000000
./core/arch/arm/plat-stm32mp1/conf.mk:59:CFG_TZDRAM_START ?= 0xfe000000

	reserved-memory {
		optee@de000000 {
			reg = <0xde000000 0x02000000>;
			no-map;
		};
	};

	reserved-memory {
		optee@fe000000 {
			reg = <0xfe000000 0x02000000>;
			no-map;
		};
	};

Then  I check copied node in kernel device tree (tested on EV1 board) on  U-Boot master:

/ {
	serial-number = "004700223338511534383330";
	#address-cells = <0x00000001>;
	#size-cells = <0x00000001>;
	model = "STMicroelectronics STM32MP157C eval daughter on eval mother";
	compatible = "st,stm32mp157c-ev1", "st,stm32mp157c-ed1", "st,stm32mp157";
	firmware {
		optee {
			method = "smc";
			compatible = "linaro,optee-tz";
		};
	};
	.....
	reserved-memory {
		#address-cells = <0x00000001>;
		#size-cells = <0x00000001>;
		ranges;
		optee@fe000000 {
			no-map;
			reg = <0xfe000000 0x02000000>;
		};
	....

The nodes for OP-TEE is correctly copied in kernel device tree.

But it is not working on v2020.10 (the no-map property is missing)

reserved-memory {
	#address-cells = <0x00000001>;
	#size-cells = <0x00000001>;
	ranges;
	optee@fe000000 {
		reg = <0xfe000000 0x02000000>;
	};

=> this issue is corrected by the 2 commit in master branch

- de51e96daf6b ("dtdec: optionnaly add property no-map to created reserved memory node")
- 12c3caa6494 ("optee: add property no-map to secure reserved memory")

Sorry again the delay of my answer, at the first reading the issue was linked for other OP-TEE issue
(speculative access to OP-TEE reserved memory corrected by [3])

PS: in future, with FIT support in TF-A, the management of OP-TEE node change again:

The OP-TEE nodes is absent in U-Boot and in kernel device tree.

1/ TF-A BL2 load OP-TEE, U-Boot and its device tree in DDR
2/ OP-TEE update the U-Boot device tree to add its nodes
3/ U-Boot copy the OP-TEE nodes in kernel device tree

So only OP-TEE manage its node and we have no more alignment issue.

Patrick

[3) http://patchwork.ozlabs.org/project/uboot/list/?series=206296


> --
> Siemens AG, T RDA IOT
> Corporate Competence Center Embedded Linux
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel