[U-Boot] [PATCH v6 0/2] Make most DDR non-secure in MMU while keep a small block secure

[U-Boot] [PATCH v6 0/2] Make most DDR non-secure in MMU while keep a small block secure

[York Sun]

This set is to change MMU tables so DDR is in non-secure mode that
non-secure master such as SDHC DMA can access the data. To mix
secure and non-secure MMU entries, the MMU tables themselves have
to be in secure memory. A small portion memory is reserved at the
end of DDR (before debug server and MC) to host secure application
and the MMU tables.

This is different from existing armv7 secure_ram_addr() solution.
U-boot can run in the middle of memory if the memory is large.
Having security memory at the very end simplifies MMU setup.

Tested on LS2085AQDS with a known non-secure master test.


Changes in v6:
  Move flag macros and comments of secure_ram into 1st path in this set
  Move cmd_bdinfo change to 1st patch in this set
  Rearrange #ifdef CONFIG_SYS_MEM_RESERVE_SECURE

Changes in v5:
  Put ifdef where gd->secure_ram is used

Changes in v4:
  Drop RFC from patch prefix
  Drop excessive mmu table for secure ram for early MMU
  Update commit message accordingly
  Mark QBMan cacheable portal memory non-secure

Changes in v3:
  Put ifdef around secure_ram
  Move defining CONFIG_SYS_MEM_RESERVE_SECURE to patch 2/2
  Replace CONFIG_FSL_PPA_RESERVED_DRAM_SIZE with CONFIG_SYS_MEM_RESERVE_SECURE
  Sanity check gd->secure_ram before using
  Define CONFIG_SYS_MEM_RESERVE_SECURE in SoC header file
  Include ls1043ardb
  Modified commit message.

Changes in v2:
  Do not use CONFIG_SYS_MEM_TOP_HIDE mechanism
  Move gd->arch.secure_ram to gd->secure_ram.
  Change the calculation of gd->secure_ram accordingly.
  Chnage commit message slightly accordingly.

Changes in v1:
  Initial patch.
  Depends on http://patchwork.ozlabs.org/patch/540248/

York Sun (2):
  Reserve secure memory
  armv8: fsl-layerscape: Make DDR non secure in MMU tables

 README                                            |    8 ++
 arch/arm/cpu/armv8/fsl-layerscape/cpu.c           |  111 +++++++++++++++++++--
 arch/arm/include/asm/arch-fsl-layerscape/config.h |    6 ++
 arch/arm/include/asm/arch-fsl-layerscape/cpu.h    |   14 ++-
 board/freescale/ls1043ardb/ddr.c                  |    4 +
 board/freescale/ls2085a/ddr.c                     |   15 +++
 board/freescale/ls2085aqds/ddr.c                  |   15 +++
 board/freescale/ls2085ardb/ddr.c                  |   15 +++
 common/board_f.c                                  |    9 ++
 common/cmd_bdinfo.c                               |    6 ++
 include/asm-generic/global_data.h                 |   14 +++
 11 files changed, 202 insertions(+), 15 deletions(-)

-- 
1.7.9.5

[U-Boot] [PATCH v6 1/2] Reserve secure memory

[York Sun]

Secure memory is at the end of memory, separated and reserved
from OS, tracked by gd->secure_ram. Secure memory can host
MMU tables, security monitor, etc. This is different from PRAM
used to reserve private memory. PRAM offers memory at the top
of u-boot memory, not necessarily the real end of memory for
systems with very large DDR. Using the end of memory simplifies
MMU setup and avoid memory fragmentation.

"bdinfo" command shows gd->secure_ram value if this memory is
marked as secured.

Signed-off-by: York Sun <yorksun@freescale.com>

---

Changes in v6:
  Move cmd_bdinfo change into this patch
  Move flag macros and comments of secure_ram into this patch

Changes in v5: None
Changes in v4: None
Changes in v3:
  Put ifdef around secure_ram
  Move defining CONFIG_SYS_MEM_RESERVE_SECURE to patch 2/2

Changes in v2:
  Do not use CONFIG_SYS_MEM_TOP_HIDE mechanism

Changes in v1:
  Initial patch.
  Depends on http://patchwork.ozlabs.org/patch/540248/

 README                            |    8 ++++++++
 common/board_f.c                  |    9 +++++++++
 common/cmd_bdinfo.c               |    6 ++++++
 include/asm-generic/global_data.h |   14 ++++++++++++++
 4 files changed, 37 insertions(+)

diff --git a/README b/README
index ef8d437..61cbc82 100644
--- a/README
+++ b/README
@@ -3881,6 +3881,14 @@ Configuration Settings:
 		Scratch address used by the alternate memory test
 		You only need to set this if address zero isn't writeable
 
+- CONFIG_SYS_MEM_RESERVE_SECURE
+		If defined, the size of CONFIG_SYS_MEM_RESERVE_SECURE memory
+		is substracted from total RAM and won't be reported to OS.
+		This memory can be used as secure memory. A variable
+		gd->secure_ram is used to track the location. In systems
+		the RAM base is not zero, or RAM is divided into banks,
+		this variable needs to be recalcuated to get the address.
+
 - CONFIG_SYS_MEM_TOP_HIDE (PPC only):
 		If CONFIG_SYS_MEM_TOP_HIDE is defined in the board config header,
 		this specified memory area will get subtracted from the top
diff --git a/common/board_f.c b/common/board_f.c
index 725eb18..8061105 100644
--- a/common/board_f.c
+++ b/common/board_f.c
@@ -323,6 +323,15 @@ static int setup_dest_addr(void)
 	 * Ram is setup, size stored in gd !!
 	 */
 	debug("Ram size: %08lX\n", (ulong)gd->ram_size);
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+	/* Reserve memory for secure MMU tables, and/or security monitor */
+	gd->ram_size -= CONFIG_SYS_MEM_RESERVE_SECURE;
+	/*
+	 * Record secure memory location. Need recalcuate if memory splits
+	 * into banks, or the ram base is not zero.
+	 */
+	gd->secure_ram = gd->ram_size;
+#endif
 #if defined(CONFIG_SYS_MEM_TOP_HIDE)
 	/*
 	 * Subtract specified amount of memory to hide so that it won't
diff --git a/common/cmd_bdinfo.c b/common/cmd_bdinfo.c
index adda55a..deed6d8 100644
--- a/common/cmd_bdinfo.c
+++ b/common/cmd_bdinfo.c
@@ -382,6 +382,12 @@ static int do_bdinfo(cmd_tbl_t *cmdtp, int flag, int argc,
 		print_num("-> size",	bd->bi_dram[i].size);
 	}
 
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+	if (gd->secure_ram & MEM_RESERVE_SECURE_SECURED) {
+		print_num("Secure ram",
+			  gd->secure_ram & MEM_RESERVE_SECURE_ADDR_MASK);
+	}
+#endif
 #if defined(CONFIG_CMD_NET) && !defined(CONFIG_DM_ETH)
 	print_eths();
 #endif
diff --git a/include/asm-generic/global_data.h b/include/asm-generic/global_data.h
index d0383f3..fc10528 100644
--- a/include/asm-generic/global_data.h
+++ b/include/asm-generic/global_data.h
@@ -58,6 +58,20 @@ typedef struct global_data {
 
 	unsigned long relocaddr;	/* Start address of U-Boot in RAM */
 	phys_size_t ram_size;	/* RAM size */
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+#define MEM_RESERVE_SECURE_SECURED	0x1
+#define MEM_RESERVE_SECURE_MAINTAINED	0x2
+#define MEM_RESERVE_SECURE_ADDR_MASK	(~0x3)
+	/*
+	 * Secure memory addr
+	 * This variable needs maintenance if the RAM base is not zero,
+	 * or if RAM splits into non-consecutive banks. It also has a
+	 * flag indicating the secure memory is marked as secure by MMU.
+	 * Flags used: 0x1 secured
+	 *             0x2 maintained
+	 */
+	phys_addr_t secure_ram;
+#endif
 	unsigned long mon_len;	/* monitor len */
 	unsigned long irq_sp;		/* irq stack pointer */
 	unsigned long start_addr_sp;	/* start_addr_stackpointer */
-- 
1.7.9.5

[U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[York Sun]

DDR has been set as secure in MMU tables. Non-secure master such
as SDHC DMA cannot access data correctly. Mixing secure and non-
secure MMU entries requirs the MMU tables themselves in secure
memory. This patch moves MMU tables into a secure DDR area.

Early MMU tables are changed to set DDR as non-secure. A new
table is added into final MMU tables so secure memory can have
2MB granuality.

gd->secure_ram tracks the location of this secure memory. For
ARMv8 SoCs, the RAM base is not zero and RAM is divided into several
banks. gd->secure_ram needs to be maintained before using. This
maintenance is board-specific, depending on the SoC and memory
bank of the secure memory falls into.

Signed-off-by: York Sun <yorksun@freescale.com>

fixup
---

Changes in v6:
  Move cmd_bdinfo change to 1st patch in this set
  Rearrange #ifdef CONFIG_SYS_MEM_RESERVE_SECURE

Changes in v5:
  Put ifdef where gd->secure_ram is used

Changes in v4:
  Drop RFC from patch prefix
  Drop excessive mmu table for secure ram for early MMU
  Update commit message accordingly
  Mark QBMan cacheable portal memory non-secure

Changes in v3:
  Replace CONFIG_FSL_PPA_RESERVED_DRAM_SIZE with CONFIG_SYS_MEM_RESERVE_SECURE
  Sanity check gd->secure_ram before using
  Define CONFIG_SYS_MEM_RESERVE_SECURE in SoC header file
  Include ls1043ardb
  Modified commit message.

Changes in v2:
  Move gd->arch.secure_ram to gd->secure_ram.
  Change the calculation of gd->secure_ram accordingly.
  Chnage commit message slightly accordingly.

Changes in v1: None

 arch/arm/cpu/armv8/fsl-layerscape/cpu.c           |  111 +++++++++++++++++++--
 arch/arm/include/asm/arch-fsl-layerscape/config.h |    6 ++
 arch/arm/include/asm/arch-fsl-layerscape/cpu.h    |   14 ++-
 board/freescale/ls1043ardb/ddr.c                  |    4 +
 board/freescale/ls2085a/ddr.c                     |   15 +++
 board/freescale/ls2085aqds/ddr.c                  |   15 +++
 board/freescale/ls2085ardb/ddr.c                  |   15 +++
 7 files changed, 165 insertions(+), 15 deletions(-)

diff --git a/arch/arm/cpu/armv8/fsl-layerscape/cpu.c b/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
index 9d1c70f..501feb3 100644
--- a/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
+++ b/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
@@ -206,11 +206,65 @@ static inline void early_mmu_setup(void)
 	set_sctlr(get_sctlr() | CR_M);
 }
 
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+/*
+ * Called from final mmu setup. The phys_addr is new, non-existing
+ * address. A new sub table is created @level2_table_secure to cover
+ * size of CONFIG_SYS_MEM_RESERVE_SECURE memory.
+ */
+static inline int final_secure_ddr(u64 *level0_table,
+				   u64 *level2_table_secure,
+				   phys_addr_t phys_addr)
+{
+	int ret = -EINVAL;
+	struct table_info table = {};
+	struct sys_mmu_table ddr_entry = {
+		0, 0, BLOCK_SIZE_L1, MT_NORMAL,
+		PMD_SECT_OUTER_SHARE | PMD_SECT_NS
+	};
+	u64 index;
+
+	/* Need to create a new table */
+	ddr_entry.virt_addr = phys_addr & ~(BLOCK_SIZE_L1 - 1);
+	ddr_entry.phys_addr = phys_addr & ~(BLOCK_SIZE_L1 - 1);
+	ret = find_table(&ddr_entry, &table, level0_table);
+	if (ret)
+		return ret;
+	index = (ddr_entry.virt_addr - table.table_base) >> SECTION_SHIFT_L1;
+	set_pgtable_table(table.ptr, index, level2_table_secure);
+	table.ptr = level2_table_secure;
+	table.table_base = ddr_entry.virt_addr;
+	table.entry_size = BLOCK_SIZE_L2;
+	ret = set_block_entry(&ddr_entry, &table);
+	if (ret) {
+		printf("MMU error: could not fill non-secure ddr block entries\n");
+		return ret;
+	}
+	ddr_entry.virt_addr = phys_addr;
+	ddr_entry.phys_addr = phys_addr;
+	ddr_entry.size = CONFIG_SYS_MEM_RESERVE_SECURE;
+	ddr_entry.attribute = PMD_SECT_OUTER_SHARE;
+	ret = find_table(&ddr_entry, &table, level0_table);
+	if (ret) {
+		printf("MMU error: could not find secure ddr table\n");
+		return ret;
+	}
+	ret = set_block_entry(&ddr_entry, &table);
+	if (ret)
+		printf("MMU error: could not set secure ddr block entry\n");
+
+	return ret;
+}
+#endif
+
 /*
  * The final tables look similar to early tables, but different in detail.
  * These tables are in DRAM. Sub tables are added to enable cache for
  * QBMan and OCRAM.
  *
+ * Put the MMU table in secure memory if gd->secure_ram is valid.
+ * OCRAM will be not used for this purpose so gd->secure_ram can't be 0.
+ *
  * Level 1 table 0 contains 512 entries for each 1GB from 0 to 512GB.
  * Level 1 table 1 contains 512 entries for each 1GB from 512GB to 1TB.
  * Level 2 table 0 contains 512 entries for each 2MB from 0 to 1GB.
@@ -224,17 +278,38 @@ static inline void early_mmu_setup(void)
 static inline void final_mmu_setup(void)
 {
 	unsigned int el, i;
-	u64 *level0_table = (u64 *)gd->arch.tlb_addr;
-	u64 *level1_table0 = (u64 *)(gd->arch.tlb_addr + 0x1000);
-	u64 *level1_table1 = (u64 *)(gd->arch.tlb_addr + 0x2000);
-	u64 *level2_table0 = (u64 *)(gd->arch.tlb_addr + 0x3000);
-#ifdef CONFIG_FSL_LSCH3
-	u64 *level2_table1 = (u64 *)(gd->arch.tlb_addr + 0x4000);
-#elif defined(CONFIG_FSL_LSCH2)
-	u64 *level2_table1 = (u64 *)(gd->arch.tlb_addr + 0x4000);
-	u64 *level2_table2 = (u64 *)(gd->arch.tlb_addr + 0x5000);
+	u64 *level0_table;
+	u64 *level1_table0;
+	u64 *level1_table1;
+	u64 *level2_table0;
+	u64 *level2_table1;
+#ifdef CONFIG_FSL_LSCH2
+	u64 *level2_table2;
 #endif
-	struct table_info table = {level0_table, 0, BLOCK_SIZE_L0};
+	struct table_info table = {NULL, 0, BLOCK_SIZE_L0};
+
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+	u64 *level2_table_secure;
+
+	/* Only use gd->secure_ram if the address is recalculated */
+	if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) {
+		/* Align to 4KB */
+		level0_table = (u64 *)(gd->secure_ram & ~0xfff);
+	} else {
+		level0_table = (u64 *)gd->arch.tlb_addr;
+		printf("MMU warning: gd->secure_ram is not maintained, disabled.\n");
+	}
+#else
+	level0_table = (u64 *)gd->arch.tlb_addr;
+#endif
+	level1_table0 = level0_table + 512;
+	level1_table1 = level1_table0 + 512;
+	level2_table0 = level1_table1 + 512;
+	level2_table1 = level2_table0 + 512;
+#ifdef CONFIG_FSL_LSCH2
+	level2_table2 = level2_table1 + 512;
+#endif
+	table.ptr = level0_table;
 
 	/* Invalidate all table entries */
 	memset(level0_table, 0, PGTABLE_SIZE);
@@ -269,6 +344,22 @@ static inline void final_mmu_setup(void)
 			       &final_mmu_table[i]);
 		}
 	}
+	/* Set the secure memory to secure in MMU */
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+	if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) {
+#ifdef CONFIG_FSL_LSCH3
+		level2_table_secure = level2_table1 + 512;
+#elif defined(CONFIG_FSL_LSCH2)
+		level2_table_secure = level2_table2 + 512;
+#endif
+		/* update tlb pointer */
+		gd->arch.tlb_addr = gd->secure_ram & ~0x3;
+		if (!final_secure_ddr(level0_table,
+				      level2_table_secure,
+				      gd->secure_ram & ~0x3))
+			gd->secure_ram |= MEM_RESERVE_SECURE_SECURED;
+	}
+#endif
 
 	/* flush new MMU table */
 	flush_dcache_range(gd->arch.tlb_addr,
diff --git a/arch/arm/include/asm/arch-fsl-layerscape/config.h b/arch/arm/include/asm/arch-fsl-layerscape/config.h
index 87bb937..5cfc9b3 100644
--- a/arch/arm/include/asm/arch-fsl-layerscape/config.h
+++ b/arch/arm/include/asm/arch-fsl-layerscape/config.h
@@ -17,6 +17,12 @@
 #define CONFIG_SYS_FSL_DDR		/* Freescale DDR driver */
 #define CONFIG_SYS_FSL_DDR_VER		FSL_DDR_VER_5_0
 
+/*
+ * Reserve secure memory
+ * To be aligned with MMU block size
+ */
+#define CONFIG_SYS_MEM_RESERVE_SECURE	(2048 * 1024)	/* 2MB */
+
 #if defined(CONFIG_LS2085A)
 #define CONFIG_MAX_CPUS				16
 #define CONFIG_SYS_FSL_IFC_BANK_COUNT		8
diff --git a/arch/arm/include/asm/arch-fsl-layerscape/cpu.h b/arch/arm/include/asm/arch-fsl-layerscape/cpu.h
index b4b85a8..a5fcef0 100644
--- a/arch/arm/include/asm/arch-fsl-layerscape/cpu.h
+++ b/arch/arm/include/asm/arch-fsl-layerscape/cpu.h
@@ -129,12 +129,14 @@ static const struct sys_mmu_table early_mmu_table[] = {
 	{ CONFIG_SYS_FLASH_BASE, CONFIG_SYS_FSL_IFC_BASE1,
 	  CONFIG_SYS_FSL_IFC_SIZE1, MT_DEVICE_NGNRNE, PMD_SECT_NON_SHARE },
 	{ CONFIG_SYS_FSL_DRAM_BASE1, CONFIG_SYS_FSL_DRAM_BASE1,
-	  CONFIG_SYS_FSL_DRAM_SIZE1, MT_NORMAL, PMD_SECT_OUTER_SHARE },
+	  CONFIG_SYS_FSL_DRAM_SIZE1, MT_NORMAL,
+	  PMD_SECT_OUTER_SHARE | PMD_SECT_NS},
 	{ CONFIG_SYS_FSL_DCSR_BASE, CONFIG_SYS_FSL_DCSR_BASE,
 	  CONFIG_SYS_FSL_DCSR_SIZE, MT_DEVICE_NGNRNE,
 	  PMD_SECT_NON_SHARE | PMD_SECT_PXN | PMD_SECT_UXN },
 	{ CONFIG_SYS_FSL_DRAM_BASE2, CONFIG_SYS_FSL_DRAM_BASE2,
-	  CONFIG_SYS_FSL_DRAM_SIZE2, MT_NORMAL, PMD_SECT_OUTER_SHARE },
+	  CONFIG_SYS_FSL_DRAM_SIZE2, MT_NORMAL,
+	  PMD_SECT_OUTER_SHARE | PMD_SECT_NS },
 #elif defined(CONFIG_FSL_LSCH2)
 	{ CONFIG_SYS_FSL_CCSR_BASE, CONFIG_SYS_FSL_CCSR_BASE,
 	  CONFIG_SYS_FSL_CCSR_SIZE, MT_DEVICE_NGNRNE,
@@ -161,7 +163,8 @@ static const struct sys_mmu_table final_mmu_table[] = {
 	{ CONFIG_SYS_FSL_OCRAM_BASE, CONFIG_SYS_FSL_OCRAM_BASE,
 	  CONFIG_SYS_FSL_OCRAM_SIZE, MT_NORMAL, PMD_SECT_NON_SHARE },
 	{ CONFIG_SYS_FSL_DRAM_BASE1, CONFIG_SYS_FSL_DRAM_BASE1,
-	  CONFIG_SYS_FSL_DRAM_SIZE1, MT_NORMAL, PMD_SECT_OUTER_SHARE },
+	  CONFIG_SYS_FSL_DRAM_SIZE1, MT_NORMAL,
+	  PMD_SECT_OUTER_SHARE | PMD_SECT_NS },
 	{ CONFIG_SYS_FSL_QSPI_BASE2, CONFIG_SYS_FSL_QSPI_BASE2,
 	  CONFIG_SYS_FSL_QSPI_SIZE2, MT_DEVICE_NGNRNE,
 	  PMD_SECT_NON_SHARE | PMD_SECT_PXN | PMD_SECT_UXN },
@@ -179,7 +182,7 @@ static const struct sys_mmu_table final_mmu_table[] = {
 	/* For QBMAN portal, only the first 64MB is cache-enabled */
 	{ CONFIG_SYS_FSL_QBMAN_BASE, CONFIG_SYS_FSL_QBMAN_BASE,
 	  CONFIG_SYS_FSL_QBMAN_SIZE_1, MT_NORMAL,
-	  PMD_SECT_NON_SHARE | PMD_SECT_PXN | PMD_SECT_UXN },
+	  PMD_SECT_NON_SHARE | PMD_SECT_PXN | PMD_SECT_UXN | PMD_SECT_NS },
 	{ CONFIG_SYS_FSL_QBMAN_BASE + CONFIG_SYS_FSL_QBMAN_SIZE_1,
 	  CONFIG_SYS_FSL_QBMAN_BASE + CONFIG_SYS_FSL_QBMAN_SIZE_1,
 	  CONFIG_SYS_FSL_QBMAN_SIZE - CONFIG_SYS_FSL_QBMAN_SIZE_1,
@@ -208,7 +211,8 @@ static const struct sys_mmu_table final_mmu_table[] = {
 	  CONFIG_SYS_FSL_PEBUF_SIZE, MT_DEVICE_NGNRNE,
 	  PMD_SECT_NON_SHARE | PMD_SECT_PXN | PMD_SECT_UXN },
 	{ CONFIG_SYS_FSL_DRAM_BASE2, CONFIG_SYS_FSL_DRAM_BASE2,
-	  CONFIG_SYS_FSL_DRAM_SIZE2, MT_NORMAL, PMD_SECT_OUTER_SHARE },
+	  CONFIG_SYS_FSL_DRAM_SIZE2, MT_NORMAL,
+	  PMD_SECT_OUTER_SHARE | PMD_SECT_NS },
 #elif defined(CONFIG_FSL_LSCH2)
 	{ CONFIG_SYS_FSL_BOOTROM_BASE, CONFIG_SYS_FSL_BOOTROM_BASE,
 	  CONFIG_SYS_FSL_BOOTROM_SIZE, MT_DEVICE_NGNRNE,
diff --git a/board/freescale/ls1043ardb/ddr.c b/board/freescale/ls1043ardb/ddr.c
index b181579..82be50b 100644
--- a/board/freescale/ls1043ardb/ddr.c
+++ b/board/freescale/ls1043ardb/ddr.c
@@ -188,4 +188,8 @@ void dram_init_banksize(void)
 {
 	gd->bd->bi_dram[0].start = CONFIG_SYS_SDRAM_BASE;
 	gd->bd->bi_dram[0].size = gd->ram_size;
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+	gd->secure_ram = gd->bd->bi_dram[0].start + gd->secure_ram;
+	gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED;
+#endif
 }
diff --git a/board/freescale/ls2085a/ddr.c b/board/freescale/ls2085a/ddr.c
index 4884fa2..bd8a98a 100644
--- a/board/freescale/ls2085a/ddr.c
+++ b/board/freescale/ls2085a/ddr.c
@@ -174,14 +174,29 @@ void dram_init_banksize(void)
 	phys_size_t dp_ddr_size;
 #endif
 
+	/*
+	 * gd->secure_ram tracks the location of secure memory.
+	 * It was set as if the memory starts from 0.
+	 * The address needs to add the offset of its bank.
+	 */
 	gd->bd->bi_dram[0].start = CONFIG_SYS_SDRAM_BASE;
 	if (gd->ram_size > CONFIG_SYS_LS2_DDR_BLOCK1_SIZE) {
 		gd->bd->bi_dram[0].size = CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
 		gd->bd->bi_dram[1].start = CONFIG_SYS_DDR_BLOCK2_BASE;
 		gd->bd->bi_dram[1].size = gd->ram_size -
 					  CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+		gd->secure_ram = gd->bd->bi_dram[1].start +
+				 gd->secure_ram -
+				 CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
+		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED;
+#endif
 	} else {
 		gd->bd->bi_dram[0].size = gd->ram_size;
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+		gd->secure_ram = gd->bd->bi_dram[0].start + gd->secure_ram;
+		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED;
+#endif
 	}
 
 #ifdef CONFIG_SYS_DP_DDR_BASE_PHY
diff --git a/board/freescale/ls2085aqds/ddr.c b/board/freescale/ls2085aqds/ddr.c
index 8d71ae1..0604230 100644
--- a/board/freescale/ls2085aqds/ddr.c
+++ b/board/freescale/ls2085aqds/ddr.c
@@ -164,14 +164,29 @@ void dram_init_banksize(void)
 	phys_size_t dp_ddr_size;
 #endif
 
+	/*
+	 * gd->secure_ram tracks the location of secure memory.
+	 * It was set as if the memory starts from 0.
+	 * The address needs to add the offset of its bank.
+	 */
 	gd->bd->bi_dram[0].start = CONFIG_SYS_SDRAM_BASE;
 	if (gd->ram_size > CONFIG_SYS_LS2_DDR_BLOCK1_SIZE) {
 		gd->bd->bi_dram[0].size = CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
 		gd->bd->bi_dram[1].start = CONFIG_SYS_DDR_BLOCK2_BASE;
 		gd->bd->bi_dram[1].size = gd->ram_size -
 					  CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+		gd->secure_ram = gd->bd->bi_dram[1].start +
+				 gd->secure_ram -
+				 CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
+		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED;
+#endif
 	} else {
 		gd->bd->bi_dram[0].size = gd->ram_size;
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+		gd->secure_ram = gd->bd->bi_dram[0].start + gd->secure_ram;
+		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED;
+#endif
 	}
 
 #ifdef CONFIG_SYS_DP_DDR_BASE_PHY
diff --git a/board/freescale/ls2085ardb/ddr.c b/board/freescale/ls2085ardb/ddr.c
index 8d71ae1..0604230 100644
--- a/board/freescale/ls2085ardb/ddr.c
+++ b/board/freescale/ls2085ardb/ddr.c
@@ -164,14 +164,29 @@ void dram_init_banksize(void)
 	phys_size_t dp_ddr_size;
 #endif
 
+	/*
+	 * gd->secure_ram tracks the location of secure memory.
+	 * It was set as if the memory starts from 0.
+	 * The address needs to add the offset of its bank.
+	 */
 	gd->bd->bi_dram[0].start = CONFIG_SYS_SDRAM_BASE;
 	if (gd->ram_size > CONFIG_SYS_LS2_DDR_BLOCK1_SIZE) {
 		gd->bd->bi_dram[0].size = CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
 		gd->bd->bi_dram[1].start = CONFIG_SYS_DDR_BLOCK2_BASE;
 		gd->bd->bi_dram[1].size = gd->ram_size -
 					  CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+		gd->secure_ram = gd->bd->bi_dram[1].start +
+				 gd->secure_ram -
+				 CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
+		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED;
+#endif
 	} else {
 		gd->bd->bi_dram[0].size = gd->ram_size;
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+		gd->secure_ram = gd->bd->bi_dram[0].start + gd->secure_ram;
+		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED;
+#endif
 	}
 
 #ifdef CONFIG_SYS_DP_DDR_BASE_PHY
-- 
1.7.9.5

[U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[Hou Zhiqiang]

Hi York,

> -----Original Message-----
> From: York Sun [mailto:yorksun at freescale.com]
> Sent: 2015?11?19? 2:02
> To: U-Boot Mailing List
> Cc: Sun York-R58495; Pan Lijun-B44306; Hu Mingkai-B21284; Sharma Bhupesh-
> B45370; Gong Qianyu-B52263; Tom Rini; Li Yang-Leo-R58472; Albert Aribaud;
> Hou Zhiqiang-B48286; Kushwaha Prabhakar-B32579; Wang Huan-B18965
> Subject: [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU
> tables
> 
> DDR has been set as secure in MMU tables. Non-secure master such as SDHC
> DMA cannot access data correctly. Mixing secure and non- secure MMU
> entries requirs the MMU tables themselves in secure memory. This patch
> moves MMU tables into a secure DDR area.
> 
> Early MMU tables are changed to set DDR as non-secure. A new table is
> added into final MMU tables so secure memory can have 2MB granuality.
> 
> gd->secure_ram tracks the location of this secure memory. For
> ARMv8 SoCs, the RAM base is not zero and RAM is divided into several
> banks. gd->secure_ram needs to be maintained before using. This
> maintenance is board-specific, depending on the SoC and memory bank of
> the secure memory falls into.
> 
> Signed-off-by: York Sun <yorksun@freescale.com>
> 
> fixup
> ---
> 
> Changes in v6:
>   Move cmd_bdinfo change to 1st patch in this set
>   Rearrange #ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> 
> Changes in v5:
>   Put ifdef where gd->secure_ram is used
> 
> Changes in v4:
>   Drop RFC from patch prefix
>   Drop excessive mmu table for secure ram for early MMU
>   Update commit message accordingly
>   Mark QBMan cacheable portal memory non-secure
> 
> Changes in v3:
>   Replace CONFIG_FSL_PPA_RESERVED_DRAM_SIZE with
> CONFIG_SYS_MEM_RESERVE_SECURE
>   Sanity check gd->secure_ram before using
>   Define CONFIG_SYS_MEM_RESERVE_SECURE in SoC header file
>   Include ls1043ardb
>   Modified commit message.
> 
> Changes in v2:
>   Move gd->arch.secure_ram to gd->secure_ram.
>   Change the calculation of gd->secure_ram accordingly.
>   Chnage commit message slightly accordingly.
> 
> Changes in v1: None
> 
>  arch/arm/cpu/armv8/fsl-layerscape/cpu.c           |  111
> +++++++++++++++++++--
>  arch/arm/include/asm/arch-fsl-layerscape/config.h |    6 ++
>  arch/arm/include/asm/arch-fsl-layerscape/cpu.h    |   14 ++-
>  board/freescale/ls1043ardb/ddr.c                  |    4 +
>  board/freescale/ls2085a/ddr.c                     |   15 +++
>  board/freescale/ls2085aqds/ddr.c                  |   15 +++
>  board/freescale/ls2085ardb/ddr.c                  |   15 +++
>  7 files changed, 165 insertions(+), 15 deletions(-)
> 
> diff --git a/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> b/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> index 9d1c70f..501feb3 100644
> --- a/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> +++ b/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> @@ -206,11 +206,65 @@ static inline void early_mmu_setup(void)
>  	set_sctlr(get_sctlr() | CR_M);
>  }
> 
> +#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> +/*
> + * Called from final mmu setup. The phys_addr is new, non-existing
> + * address. A new sub table is created @level2_table_secure to cover
> + * size of CONFIG_SYS_MEM_RESERVE_SECURE memory.
> + */
> +static inline int final_secure_ddr(u64 *level0_table,
> +				   u64 *level2_table_secure,
> +				   phys_addr_t phys_addr)
> +{
> +	int ret = -EINVAL;
> +	struct table_info table = {};
> +	struct sys_mmu_table ddr_entry = {
> +		0, 0, BLOCK_SIZE_L1, MT_NORMAL,
> +		PMD_SECT_OUTER_SHARE | PMD_SECT_NS
> +	};
> +	u64 index;
> +
> +	/* Need to create a new table */
> +	ddr_entry.virt_addr = phys_addr & ~(BLOCK_SIZE_L1 - 1);
> +	ddr_entry.phys_addr = phys_addr & ~(BLOCK_SIZE_L1 - 1);
> +	ret = find_table(&ddr_entry, &table, level0_table);
> +	if (ret)
> +		return ret;
> +	index = (ddr_entry.virt_addr - table.table_base) >>
> SECTION_SHIFT_L1;
> +	set_pgtable_table(table.ptr, index, level2_table_secure);
> +	table.ptr = level2_table_secure;
> +	table.table_base = ddr_entry.virt_addr;
> +	table.entry_size = BLOCK_SIZE_L2;
> +	ret = set_block_entry(&ddr_entry, &table);
> +	if (ret) {
> +		printf("MMU error: could not fill non-secure ddr block
> entries\n");
> +		return ret;
> +	}
> +	ddr_entry.virt_addr = phys_addr;
> +	ddr_entry.phys_addr = phys_addr;
> +	ddr_entry.size = CONFIG_SYS_MEM_RESERVE_SECURE;
> +	ddr_entry.attribute = PMD_SECT_OUTER_SHARE;
> +	ret = find_table(&ddr_entry, &table, level0_table);
> +	if (ret) {
> +		printf("MMU error: could not find secure ddr table\n");
> +		return ret;
> +	}
> +	ret = set_block_entry(&ddr_entry, &table);
> +	if (ret)
> +		printf("MMU error: could not set secure ddr block entry\n");
> +
> +	return ret;
> +}
> +#endif
> +
>  /*
>   * The final tables look similar to early tables, but different in
> detail.
>   * These tables are in DRAM. Sub tables are added to enable cache for
>   * QBMan and OCRAM.
>   *
> + * Put the MMU table in secure memory if gd->secure_ram is valid.
> + * OCRAM will be not used for this purpose so gd->secure_ram can't be 0.
> + *
>   * Level 1 table 0 contains 512 entries for each 1GB from 0 to 512GB.
>   * Level 1 table 1 contains 512 entries for each 1GB from 512GB to 1TB.
>   * Level 2 table 0 contains 512 entries for each 2MB from 0 to 1GB.
> @@ -224,17 +278,38 @@ static inline void early_mmu_setup(void)  static
> inline void final_mmu_setup(void)  {
>  	unsigned int el, i;
> -	u64 *level0_table = (u64 *)gd->arch.tlb_addr;
> -	u64 *level1_table0 = (u64 *)(gd->arch.tlb_addr + 0x1000);
> -	u64 *level1_table1 = (u64 *)(gd->arch.tlb_addr + 0x2000);
> -	u64 *level2_table0 = (u64 *)(gd->arch.tlb_addr + 0x3000);
> -#ifdef CONFIG_FSL_LSCH3
> -	u64 *level2_table1 = (u64 *)(gd->arch.tlb_addr + 0x4000);
> -#elif defined(CONFIG_FSL_LSCH2)
> -	u64 *level2_table1 = (u64 *)(gd->arch.tlb_addr + 0x4000);
> -	u64 *level2_table2 = (u64 *)(gd->arch.tlb_addr + 0x5000);
> +	u64 *level0_table;
> +	u64 *level1_table0;
> +	u64 *level1_table1;
> +	u64 *level2_table0;
> +	u64 *level2_table1;
> +#ifdef CONFIG_FSL_LSCH2
> +	u64 *level2_table2;
>  #endif
> -	struct table_info table = {level0_table, 0, BLOCK_SIZE_L0};
> +	struct table_info table = {NULL, 0, BLOCK_SIZE_L0};
> +
> +#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> +	u64 *level2_table_secure;
> +
> +	/* Only use gd->secure_ram if the address is recalculated */
> +	if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) {
> +		/* Align to 4KB */
> +		level0_table = (u64 *)(gd->secure_ram & ~0xfff);
> +	} else {
> +		level0_table = (u64 *)gd->arch.tlb_addr;
> +		printf("MMU warning: gd->secure_ram is not maintained,
> disabled.\n");
> +	}
> +#else
> +	level0_table = (u64 *)gd->arch.tlb_addr; #endif
> +	level1_table0 = level0_table + 512;
> +	level1_table1 = level1_table0 + 512;
> +	level2_table0 = level1_table1 + 512;
> +	level2_table1 = level2_table0 + 512;
> +#ifdef CONFIG_FSL_LSCH2
> +	level2_table2 = level2_table1 + 512;
> +#endif
> +	table.ptr = level0_table;
> 
>  	/* Invalidate all table entries */
>  	memset(level0_table, 0, PGTABLE_SIZE); @@ -269,6 +344,22 @@ static
> inline void final_mmu_setup(void)
>  			       &final_mmu_table[i]);
>  		}
>  	}
> +	/* Set the secure memory to secure in MMU */ #ifdef
> +CONFIG_SYS_MEM_RESERVE_SECURE
> +	if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) { #ifdef
> +CONFIG_FSL_LSCH3
> +		level2_table_secure = level2_table1 + 512; #elif
> +defined(CONFIG_FSL_LSCH2)
> +		level2_table_secure = level2_table2 + 512; #endif
> +		/* update tlb pointer */
> +		gd->arch.tlb_addr = gd->secure_ram & ~0x3;

The memory reserved for mmu table was lost? If it's better to record it and use for non-sec mmu table?

If this func is called from non-secure state, for example EL2, the secure memory cannot be accessed and the PMD_SECT_NS bit should be cleared.

> +		if (!final_secure_ddr(level0_table,
> +				      level2_table_secure,
> +				      gd->secure_ram & ~0x3))
> +			gd->secure_ram |= MEM_RESERVE_SECURE_SECURED;
> +	}
> +#endif
> 
>  	/* flush new MMU table */
>  	flush_dcache_range(gd->arch.tlb_addr,
> diff --git a/arch/arm/include/asm/arch-fsl-layerscape/config.h
> b/arch/arm/include/asm/arch-fsl-layerscape/config.h
> index 87bb937..5cfc9b3 100644
> --- a/arch/arm/include/asm/arch-fsl-layerscape/config.h
> +++ b/arch/arm/include/asm/arch-fsl-layerscape/config.h
> @@ -17,6 +17,12 @@
>  #define CONFIG_SYS_FSL_DDR		/* Freescale DDR driver */
>  #define CONFIG_SYS_FSL_DDR_VER		FSL_DDR_VER_5_0
> 
> +/*
> + * Reserve secure memory
> + * To be aligned with MMU block size
> + */
> +#define CONFIG_SYS_MEM_RESERVE_SECURE	(2048 * 1024)	/* 2MB */
> +
>  #if defined(CONFIG_LS2085A)
>  #define CONFIG_MAX_CPUS				16
>  #define CONFIG_SYS_FSL_IFC_BANK_COUNT		8
> diff --git a/arch/arm/include/asm/arch-fsl-layerscape/cpu.h
> b/arch/arm/include/asm/arch-fsl-layerscape/cpu.h
> index b4b85a8..a5fcef0 100644
> --- a/arch/arm/include/asm/arch-fsl-layerscape/cpu.h
> +++ b/arch/arm/include/asm/arch-fsl-layerscape/cpu.h
> @@ -129,12 +129,14 @@ static const struct sys_mmu_table early_mmu_table[]
> = {
>  	{ CONFIG_SYS_FLASH_BASE, CONFIG_SYS_FSL_IFC_BASE1,
>  	  CONFIG_SYS_FSL_IFC_SIZE1, MT_DEVICE_NGNRNE, PMD_SECT_NON_SHARE },
>  	{ CONFIG_SYS_FSL_DRAM_BASE1, CONFIG_SYS_FSL_DRAM_BASE1,
> -	  CONFIG_SYS_FSL_DRAM_SIZE1, MT_NORMAL, PMD_SECT_OUTER_SHARE },
> +	  CONFIG_SYS_FSL_DRAM_SIZE1, MT_NORMAL,
> +	  PMD_SECT_OUTER_SHARE | PMD_SECT_NS},
>  	{ CONFIG_SYS_FSL_DCSR_BASE, CONFIG_SYS_FSL_DCSR_BASE,
>  	  CONFIG_SYS_FSL_DCSR_SIZE, MT_DEVICE_NGNRNE,
>  	  PMD_SECT_NON_SHARE | PMD_SECT_PXN | PMD_SECT_UXN },
>  	{ CONFIG_SYS_FSL_DRAM_BASE2, CONFIG_SYS_FSL_DRAM_BASE2,
> -	  CONFIG_SYS_FSL_DRAM_SIZE2, MT_NORMAL, PMD_SECT_OUTER_SHARE },
> +	  CONFIG_SYS_FSL_DRAM_SIZE2, MT_NORMAL,
> +	  PMD_SECT_OUTER_SHARE | PMD_SECT_NS },
>  #elif defined(CONFIG_FSL_LSCH2)
>  	{ CONFIG_SYS_FSL_CCSR_BASE, CONFIG_SYS_FSL_CCSR_BASE,
>  	  CONFIG_SYS_FSL_CCSR_SIZE, MT_DEVICE_NGNRNE, @@ -161,7 +163,8 @@
> static const struct sys_mmu_table final_mmu_table[] = {
>  	{ CONFIG_SYS_FSL_OCRAM_BASE, CONFIG_SYS_FSL_OCRAM_BASE,
>  	  CONFIG_SYS_FSL_OCRAM_SIZE, MT_NORMAL, PMD_SECT_NON_SHARE },
>  	{ CONFIG_SYS_FSL_DRAM_BASE1, CONFIG_SYS_FSL_DRAM_BASE1,
> -	  CONFIG_SYS_FSL_DRAM_SIZE1, MT_NORMAL, PMD_SECT_OUTER_SHARE },
> +	  CONFIG_SYS_FSL_DRAM_SIZE1, MT_NORMAL,
> +	  PMD_SECT_OUTER_SHARE | PMD_SECT_NS },
>  	{ CONFIG_SYS_FSL_QSPI_BASE2, CONFIG_SYS_FSL_QSPI_BASE2,
>  	  CONFIG_SYS_FSL_QSPI_SIZE2, MT_DEVICE_NGNRNE,
>  	  PMD_SECT_NON_SHARE | PMD_SECT_PXN | PMD_SECT_UXN }, @@ -179,7
> +182,7 @@ static const struct sys_mmu_table final_mmu_table[] = {
>  	/* For QBMAN portal, only the first 64MB is cache-enabled */
>  	{ CONFIG_SYS_FSL_QBMAN_BASE, CONFIG_SYS_FSL_QBMAN_BASE,
>  	  CONFIG_SYS_FSL_QBMAN_SIZE_1, MT_NORMAL,
> -	  PMD_SECT_NON_SHARE | PMD_SECT_PXN | PMD_SECT_UXN },
> +	  PMD_SECT_NON_SHARE | PMD_SECT_PXN | PMD_SECT_UXN | PMD_SECT_NS },
>  	{ CONFIG_SYS_FSL_QBMAN_BASE + CONFIG_SYS_FSL_QBMAN_SIZE_1,
>  	  CONFIG_SYS_FSL_QBMAN_BASE + CONFIG_SYS_FSL_QBMAN_SIZE_1,
>  	  CONFIG_SYS_FSL_QBMAN_SIZE - CONFIG_SYS_FSL_QBMAN_SIZE_1, @@ -
> 208,7 +211,8 @@ static const struct sys_mmu_table final_mmu_table[] = {
>  	  CONFIG_SYS_FSL_PEBUF_SIZE, MT_DEVICE_NGNRNE,
>  	  PMD_SECT_NON_SHARE | PMD_SECT_PXN | PMD_SECT_UXN },
>  	{ CONFIG_SYS_FSL_DRAM_BASE2, CONFIG_SYS_FSL_DRAM_BASE2,
> -	  CONFIG_SYS_FSL_DRAM_SIZE2, MT_NORMAL, PMD_SECT_OUTER_SHARE },
> +	  CONFIG_SYS_FSL_DRAM_SIZE2, MT_NORMAL,
> +	  PMD_SECT_OUTER_SHARE | PMD_SECT_NS },
>  #elif defined(CONFIG_FSL_LSCH2)
>  	{ CONFIG_SYS_FSL_BOOTROM_BASE, CONFIG_SYS_FSL_BOOTROM_BASE,
>  	  CONFIG_SYS_FSL_BOOTROM_SIZE, MT_DEVICE_NGNRNE, diff --git
> a/board/freescale/ls1043ardb/ddr.c b/board/freescale/ls1043ardb/ddr.c
> index b181579..82be50b 100644
> --- a/board/freescale/ls1043ardb/ddr.c
> +++ b/board/freescale/ls1043ardb/ddr.c
> @@ -188,4 +188,8 @@ void dram_init_banksize(void)  {
>  	gd->bd->bi_dram[0].start = CONFIG_SYS_SDRAM_BASE;
>  	gd->bd->bi_dram[0].size = gd->ram_size;
> +#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> +	gd->secure_ram = gd->bd->bi_dram[0].start + gd->secure_ram;
> +	gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED; #endif
>  }
> diff --git a/board/freescale/ls2085a/ddr.c
> b/board/freescale/ls2085a/ddr.c index 4884fa2..bd8a98a 100644
> --- a/board/freescale/ls2085a/ddr.c
> +++ b/board/freescale/ls2085a/ddr.c
> @@ -174,14 +174,29 @@ void dram_init_banksize(void)
>  	phys_size_t dp_ddr_size;
>  #endif
> 
> +	/*
> +	 * gd->secure_ram tracks the location of secure memory.
> +	 * It was set as if the memory starts from 0.
> +	 * The address needs to add the offset of its bank.
> +	 */
>  	gd->bd->bi_dram[0].start = CONFIG_SYS_SDRAM_BASE;
>  	if (gd->ram_size > CONFIG_SYS_LS2_DDR_BLOCK1_SIZE) {
>  		gd->bd->bi_dram[0].size = CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
>  		gd->bd->bi_dram[1].start = CONFIG_SYS_DDR_BLOCK2_BASE;
>  		gd->bd->bi_dram[1].size = gd->ram_size -
>  					  CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
> +#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> +		gd->secure_ram = gd->bd->bi_dram[1].start +
> +				 gd->secure_ram -
> +				 CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
> +		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED; #endif
>  	} else {
>  		gd->bd->bi_dram[0].size = gd->ram_size;
> +#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> +		gd->secure_ram = gd->bd->bi_dram[0].start + gd->secure_ram;
> +		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED; #endif
>  	}
> 
>  #ifdef CONFIG_SYS_DP_DDR_BASE_PHY
> diff --git a/board/freescale/ls2085aqds/ddr.c
> b/board/freescale/ls2085aqds/ddr.c
> index 8d71ae1..0604230 100644
> --- a/board/freescale/ls2085aqds/ddr.c
> +++ b/board/freescale/ls2085aqds/ddr.c
> @@ -164,14 +164,29 @@ void dram_init_banksize(void)
>  	phys_size_t dp_ddr_size;
>  #endif
> 
> +	/*
> +	 * gd->secure_ram tracks the location of secure memory.
> +	 * It was set as if the memory starts from 0.
> +	 * The address needs to add the offset of its bank.
> +	 */
>  	gd->bd->bi_dram[0].start = CONFIG_SYS_SDRAM_BASE;
>  	if (gd->ram_size > CONFIG_SYS_LS2_DDR_BLOCK1_SIZE) {
>  		gd->bd->bi_dram[0].size = CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
>  		gd->bd->bi_dram[1].start = CONFIG_SYS_DDR_BLOCK2_BASE;
>  		gd->bd->bi_dram[1].size = gd->ram_size -
>  					  CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
> +#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> +		gd->secure_ram = gd->bd->bi_dram[1].start +
> +				 gd->secure_ram -
> +				 CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
> +		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED; #endif
>  	} else {
>  		gd->bd->bi_dram[0].size = gd->ram_size;
> +#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> +		gd->secure_ram = gd->bd->bi_dram[0].start + gd->secure_ram;
> +		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED; #endif
>  	}
> 
>  #ifdef CONFIG_SYS_DP_DDR_BASE_PHY
> diff --git a/board/freescale/ls2085ardb/ddr.c
> b/board/freescale/ls2085ardb/ddr.c
> index 8d71ae1..0604230 100644
> --- a/board/freescale/ls2085ardb/ddr.c
> +++ b/board/freescale/ls2085ardb/ddr.c
> @@ -164,14 +164,29 @@ void dram_init_banksize(void)
>  	phys_size_t dp_ddr_size;
>  #endif
> 
> +	/*
> +	 * gd->secure_ram tracks the location of secure memory.
> +	 * It was set as if the memory starts from 0.
> +	 * The address needs to add the offset of its bank.
> +	 */
>  	gd->bd->bi_dram[0].start = CONFIG_SYS_SDRAM_BASE;
>  	if (gd->ram_size > CONFIG_SYS_LS2_DDR_BLOCK1_SIZE) {
>  		gd->bd->bi_dram[0].size = CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
>  		gd->bd->bi_dram[1].start = CONFIG_SYS_DDR_BLOCK2_BASE;
>  		gd->bd->bi_dram[1].size = gd->ram_size -
>  					  CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
> +#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> +		gd->secure_ram = gd->bd->bi_dram[1].start +
> +				 gd->secure_ram -
> +				 CONFIG_SYS_LS2_DDR_BLOCK1_SIZE;
> +		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED; #endif
>  	} else {
>  		gd->bd->bi_dram[0].size = gd->ram_size;
> +#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> +		gd->secure_ram = gd->bd->bi_dram[0].start + gd->secure_ram;
> +		gd->secure_ram |= MEM_RESERVE_SECURE_MAINTAINED; #endif
>  	}
> 
>  #ifdef CONFIG_SYS_DP_DDR_BASE_PHY
> --
> 1.7.9.5

Thanks,
Zhiqiang

[U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[York Sun]

On 12/01/2015 10:38 PM, Hou Zhiqiang-B48286 wrote:

<snip>

>>  	/* Invalidate all table entries */
>>  	memset(level0_table, 0, PGTABLE_SIZE); @@ -269,6 +344,22 @@ static
>> inline void final_mmu_setup(void)
>>  			       &final_mmu_table[i]);
>>  		}
>>  	}
>> +	/* Set the secure memory to secure in MMU */ #ifdef
>> +CONFIG_SYS_MEM_RESERVE_SECURE
>> +	if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) { #ifdef
>> +CONFIG_FSL_LSCH3
>> +		level2_table_secure = level2_table1 + 512; #elif
>> +defined(CONFIG_FSL_LSCH2)
>> +		level2_table_secure = level2_table2 + 512; #endif
>> +		/* update tlb pointer */
>> +		gd->arch.tlb_addr = gd->secure_ram & ~0x3;
> 
> The memory reserved for mmu table was lost? If it's better to record it and use for non-sec mmu table?

That can be arranged.

> 
> If this func is called from non-secure state, for example EL2, the secure memory cannot be accessed and the PMD_SECT_NS bit should be cleared.

We need to setup secure memory at EL3 to have secure memory to begin with. But
you have a point here. Will look into it.

York

[U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[Hou Zhiqiang]

> -----Original Message-----
> From: York Sun [mailto:yorksun at freescale.com]
> Sent: 2015?12?3? 0:00
> To: Hou Zhiqiang-B48286; U-Boot Mailing List
> Cc: Pan Lijun-B44306; Hu Mingkai-B21284; Sharma Bhupesh-B45370; Gong
> Qianyu-B52263; Tom Rini; Li Yang-Leo-R58472; Albert Aribaud; Kushwaha
> Prabhakar-B32579; Wang Huan-B18965
> Subject: Re: [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in
> MMU tables
> 
> 
> 
> On 12/01/2015 10:38 PM, Hou Zhiqiang-B48286 wrote:
> 
> <snip>
> 
> >>  	/* Invalidate all table entries */
> >>  	memset(level0_table, 0, PGTABLE_SIZE); @@ -269,6 +344,22 @@ static
> >> inline void final_mmu_setup(void)
> >>  			       &final_mmu_table[i]);
> >>  		}
> >>  	}
> >> +	/* Set the secure memory to secure in MMU */ #ifdef
> >> +CONFIG_SYS_MEM_RESERVE_SECURE
> >> +	if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) { #ifdef
> >> +CONFIG_FSL_LSCH3
> >> +		level2_table_secure = level2_table1 + 512; #elif
> >> +defined(CONFIG_FSL_LSCH2)
> >> +		level2_table_secure = level2_table2 + 512; #endif
> >> +		/* update tlb pointer */
> >> +		gd->arch.tlb_addr = gd->secure_ram & ~0x3;
> >
> > The memory reserved for mmu table was lost? If it's better to record it
> and use for non-sec mmu table?
> 
> That can be arranged.
> 
> >
> > If this func is called from non-secure state, for example EL2, the
> secure memory cannot be accessed and the PMD_SECT_NS bit should be
> cleared.
> 
> We need to setup secure memory at EL3 to have secure memory to begin with.
> But you have a point here. Will look into it.
> 

Yes, this func is called at EL3 for now, it is a assumption for this func.
I mean we'd better make it a API that can be invoked at both EL3 and EL2 
to setup the MMU table. And do you know if the EL1 should be taken into 
account in u-boot?

Thanks,
Zhiqiang

[U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[York Sun]

On 12/02/2015 06:48 PM, Hou Zhiqiang-B48286 wrote:
> 
> 
>> -----Original Message-----
>> From: York Sun [mailto:yorksun at freescale.com]
>> Sent: 2015?12?3? 0:00
>> To: Hou Zhiqiang-B48286; U-Boot Mailing List
>> Cc: Pan Lijun-B44306; Hu Mingkai-B21284; Sharma Bhupesh-B45370; Gong
>> Qianyu-B52263; Tom Rini; Li Yang-Leo-R58472; Albert Aribaud; Kushwaha
>> Prabhakar-B32579; Wang Huan-B18965
>> Subject: Re: [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in
>> MMU tables
>>
>>
>>
>> On 12/01/2015 10:38 PM, Hou Zhiqiang-B48286 wrote:
>>
>> <snip>
>>
>>>>  	/* Invalidate all table entries */
>>>>  	memset(level0_table, 0, PGTABLE_SIZE); @@ -269,6 +344,22 @@ static
>>>> inline void final_mmu_setup(void)
>>>>  			       &final_mmu_table[i]);
>>>>  		}
>>>>  	}
>>>> +	/* Set the secure memory to secure in MMU */ #ifdef
>>>> +CONFIG_SYS_MEM_RESERVE_SECURE
>>>> +	if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) { #ifdef
>>>> +CONFIG_FSL_LSCH3
>>>> +		level2_table_secure = level2_table1 + 512; #elif
>>>> +defined(CONFIG_FSL_LSCH2)
>>>> +		level2_table_secure = level2_table2 + 512; #endif
>>>> +		/* update tlb pointer */
>>>> +		gd->arch.tlb_addr = gd->secure_ram & ~0x3;
>>>
>>> The memory reserved for mmu table was lost? If it's better to record it
>> and use for non-sec mmu table?
>>
>> That can be arranged.
>>
>>>
>>> If this func is called from non-secure state, for example EL2, the
>> secure memory cannot be accessed and the PMD_SECT_NS bit should be
>> cleared.
>>
>> We need to setup secure memory at EL3 to have secure memory to begin with.
>> But you have a point here. Will look into it.
>>
> 
> Yes, this func is called at EL3 for now, it is a assumption for this func.
> I mean we'd better make it a API that can be invoked at both EL3 and EL2 
> to setup the MMU table. And do you know if the EL1 should be taken into 
> account in u-boot?

We don't need to differentiate this function for EL2 or EL1 if we ever need to
run at. The only difference is EL3 can handle secure access. So adding a
detection of exception level before setting secure bit will suffice.

York

[U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[Hou Zhiqiang]

> -----Original Message-----
> From: York Sun [mailto:yorksun at freescale.com]
> Sent: 2015?12?3? 11:05
> To: Hou Zhiqiang-B48286; U-Boot Mailing List
> Cc: Pan Lijun-B44306; Hu Mingkai-B21284; Sharma Bhupesh-B45370; Gong
> Qianyu-B52263; Tom Rini; Li Yang-Leo-R58472; Albert Aribaud; Kushwaha
> Prabhakar-B32579; Wang Huan-B18965
> Subject: Re: [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in
> MMU tables
> 
> 
> 
> On 12/02/2015 06:48 PM, Hou Zhiqiang-B48286 wrote:
> >
> >
> >> -----Original Message-----
> >> From: York Sun [mailto:yorksun at freescale.com]
> >> Sent: 2015?12?3? 0:00
> >> To: Hou Zhiqiang-B48286; U-Boot Mailing List
> >> Cc: Pan Lijun-B44306; Hu Mingkai-B21284; Sharma Bhupesh-B45370; Gong
> >> Qianyu-B52263; Tom Rini; Li Yang-Leo-R58472; Albert Aribaud; Kushwaha
> >> Prabhakar-B32579; Wang Huan-B18965
> >> Subject: Re: [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non
> >> secure in MMU tables
> >>
> >>
> >>
> >> On 12/01/2015 10:38 PM, Hou Zhiqiang-B48286 wrote:
> >>
> >> <snip>
> >>
> >>>>  	/* Invalidate all table entries */
> >>>>  	memset(level0_table, 0, PGTABLE_SIZE); @@ -269,6 +344,22 @@
> >>>> static inline void final_mmu_setup(void)
> >>>>  			       &final_mmu_table[i]);
> >>>>  		}
> >>>>  	}
> >>>> +	/* Set the secure memory to secure in MMU */ #ifdef
> >>>> +CONFIG_SYS_MEM_RESERVE_SECURE
> >>>> +	if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) { #ifdef
> >>>> +CONFIG_FSL_LSCH3
> >>>> +		level2_table_secure = level2_table1 + 512; #elif
> >>>> +defined(CONFIG_FSL_LSCH2)
> >>>> +		level2_table_secure = level2_table2 + 512; #endif
> >>>> +		/* update tlb pointer */
> >>>> +		gd->arch.tlb_addr = gd->secure_ram & ~0x3;
> >>>
> >>> The memory reserved for mmu table was lost? If it's better to record
> >>> it
> >> and use for non-sec mmu table?
> >>
> >> That can be arranged.
> >>
> >>>
> >>> If this func is called from non-secure state, for example EL2, the
> >> secure memory cannot be accessed and the PMD_SECT_NS bit should be
> >> cleared.
> >>
> >> We need to setup secure memory at EL3 to have secure memory to begin
> with.
> >> But you have a point here. Will look into it.
> >>
> >
> > Yes, this func is called at EL3 for now, it is a assumption for this
> func.
> > I mean we'd better make it a API that can be invoked at both EL3 and
> > EL2 to setup the MMU table. And do you know if the EL1 should be taken
> > into account in u-boot?
> 
> We don't need to differentiate this function for EL2 or EL1 if we ever
> need to run at. The only difference is EL3 can handle secure access. So
> adding a detection of exception level before setting secure bit will
> suffice.

ok.

Thanks,
Zhiqiang