[PATCH 00/14] arm64: 16K translation granule support

[PATCH 00/14] arm64: 16K translation granule support

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

This series enables the 16K page size support on Linux for arm64.
This series adds support for 48bit VA(4 level), 47bit VA(3 level) and
36bit VA(2 level) with 16K. 16K was a late addition to the architecture
and is not implemented by all CPUs. Added a check to ensure the
selected granule size is supported by the CPU, failing which the CPU
won't proceed with booting.

KVM bits have been tested on a fast model with GICv3 using Andre's kvmtool
with gicv3 support[1].

Patches 1-7 cleans up the kernel page size handling code.
Patches 8-11 Fixes some issues with the KVM bits, mainly the fake PGD
             handling code.
Patch 12    Adds a check to ensure the CPU supports the selected granule size.
Patch 13-14 Adds the 16k page size support bits.

This series applies on top of for-next/core branch of the aarch64 tree and is
also available here:

	git://linux-arm.org/linux-skp.git  16k/v1

[1] git://linux-arm.org/kvmtool.git gicv3/v4

TODO:
 1) Testing on a silicon
 2) Analyse the performance of HugePages with 16K (32MB) on a
    silicon.
 3) SMMU driver

Suzuki K. Poulose (14):
  arm64: Move swapper pagetable definitions
  arm64: Handle section maps for swapper/idmap
  arm64: Introduce helpers for page table levels
  arm64: Calculate size for idmap_pg_dir at compile time
  arm64: Handle 4 level page table for swapper
  arm64: Clean config usages for page size
  arm64: Kconfig: Fix help text about AArch32 support with 64K pages
  arm64: kvm: Fix {V}TCR_EL2_TG0 mask
  arm64: Cleanup VTCR_EL2 computation
  arm: kvm: Move fake PGD handling to arch specific files
  arm64: kvm: Rewrite fake pgd handling
  arm64: Check for selected granule support
  arm64: Add 16K page size support
  arm64: 36 bit VA

 arch/arm/include/asm/kvm_mmu.h          |    7 ++
 arch/arm/kvm/mmu.c                      |   44 ++--------
 arch/arm64/Kconfig                      |   37 +++++++--
 arch/arm64/Kconfig.debug                |    2 +-
 arch/arm64/include/asm/boot.h           |    1 +
 arch/arm64/include/asm/fixmap.h         |    4 +-
 arch/arm64/include/asm/kernel-pgtable.h |   77 ++++++++++++++++++
 arch/arm64/include/asm/kvm_arm.h        |   29 +++++--
 arch/arm64/include/asm/kvm_mmu.h        |  135 +++++++++++++++++++++----------
 arch/arm64/include/asm/page.h           |   20 +----
 arch/arm64/include/asm/pgtable-hwdef.h  |   15 +++-
 arch/arm64/include/asm/sysreg.h         |    8 ++
 arch/arm64/include/asm/thread_info.h    |    4 +-
 arch/arm64/kernel/head.S                |   71 +++++++++-------
 arch/arm64/kernel/vmlinux.lds.S         |    1 +
 arch/arm64/mm/mmu.c                     |   70 +++++++---------
 arch/arm64/mm/proc.S                    |    4 +-
 17 files changed, 337 insertions(+), 192 deletions(-)
 create mode 100644 arch/arm64/include/asm/kernel-pgtable.h

-- 
1.7.9.5

[PATCH 00/14] arm64: 16K translation granule support

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

This series enables the 16K page size support on Linux for arm64.
This series adds support for 48bit VA(4 level), 47bit VA(3 level) and
36bit VA(2 level) with 16K. 16K was a late addition to the architecture
and is not implemented by all CPUs. Added a check to ensure the
selected granule size is supported by the CPU, failing which the CPU
won't proceed with booting.

KVM bits have been tested on a fast model with GICv3 using Andre's kvmtool
with gicv3 support[1].

Patches 1-7 cleans up the kernel page size handling code.
Patches 8-11 Fixes some issues with the KVM bits, mainly the fake PGD
             handling code.
Patch 12    Adds a check to ensure the CPU supports the selected granule size.
Patch 13-14 Adds the 16k page size support bits.

This series applies on top of for-next/core branch of the aarch64 tree and is
also available here:

	git://linux-arm.org/linux-skp.git  16k/v1

[1] git://linux-arm.org/kvmtool.git gicv3/v4

TODO:
 1) Testing on a silicon
 2) Analyse the performance of HugePages with 16K (32MB) on a
    silicon.
 3) SMMU driver

Suzuki K. Poulose (14):
  arm64: Move swapper pagetable definitions
  arm64: Handle section maps for swapper/idmap
  arm64: Introduce helpers for page table levels
  arm64: Calculate size for idmap_pg_dir at compile time
  arm64: Handle 4 level page table for swapper
  arm64: Clean config usages for page size
  arm64: Kconfig: Fix help text about AArch32 support with 64K pages
  arm64: kvm: Fix {V}TCR_EL2_TG0 mask
  arm64: Cleanup VTCR_EL2 computation
  arm: kvm: Move fake PGD handling to arch specific files
  arm64: kvm: Rewrite fake pgd handling
  arm64: Check for selected granule support
  arm64: Add 16K page size support
  arm64: 36 bit VA

 arch/arm/include/asm/kvm_mmu.h          |    7 ++
 arch/arm/kvm/mmu.c                      |   44 ++--------
 arch/arm64/Kconfig                      |   37 +++++++--
 arch/arm64/Kconfig.debug                |    2 +-
 arch/arm64/include/asm/boot.h           |    1 +
 arch/arm64/include/asm/fixmap.h         |    4 +-
 arch/arm64/include/asm/kernel-pgtable.h |   77 ++++++++++++++++++
 arch/arm64/include/asm/kvm_arm.h        |   29 +++++--
 arch/arm64/include/asm/kvm_mmu.h        |  135 +++++++++++++++++++++----------
 arch/arm64/include/asm/page.h           |   20 +----
 arch/arm64/include/asm/pgtable-hwdef.h  |   15 +++-
 arch/arm64/include/asm/sysreg.h         |    8 ++
 arch/arm64/include/asm/thread_info.h    |    4 +-
 arch/arm64/kernel/head.S                |   71 +++++++++-------
 arch/arm64/kernel/vmlinux.lds.S         |    1 +
 arch/arm64/mm/mmu.c                     |   70 +++++++---------
 arch/arm64/mm/proc.S                    |    4 +-
 17 files changed, 337 insertions(+), 192 deletions(-)
 create mode 100644 arch/arm64/include/asm/kernel-pgtable.h

-- 
1.7.9.5

[PATCH 00/14] arm64: 16K translation granule support

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

This series enables the 16K page size support on Linux for arm64.
This series adds support for 48bit VA(4 level), 47bit VA(3 level) and
36bit VA(2 level) with 16K. 16K was a late addition to the architecture
and is not implemented by all CPUs. Added a check to ensure the
selected granule size is supported by the CPU, failing which the CPU
won't proceed with booting.

KVM bits have been tested on a fast model with GICv3 using Andre's kvmtool
with gicv3 support[1].

Patches 1-7 cleans up the kernel page size handling code.
Patches 8-11 Fixes some issues with the KVM bits, mainly the fake PGD
             handling code.
Patch 12    Adds a check to ensure the CPU supports the selected granule size.
Patch 13-14 Adds the 16k page size support bits.

This series applies on top of for-next/core branch of the aarch64 tree and is
also available here:

	git://linux-arm.org/linux-skp.git  16k/v1

[1] git://linux-arm.org/kvmtool.git gicv3/v4

TODO:
 1) Testing on a silicon
 2) Analyse the performance of HugePages with 16K (32MB) on a
    silicon.
 3) SMMU driver

Suzuki K. Poulose (14):
  arm64: Move swapper pagetable definitions
  arm64: Handle section maps for swapper/idmap
  arm64: Introduce helpers for page table levels
  arm64: Calculate size for idmap_pg_dir at compile time
  arm64: Handle 4 level page table for swapper
  arm64: Clean config usages for page size
  arm64: Kconfig: Fix help text about AArch32 support with 64K pages
  arm64: kvm: Fix {V}TCR_EL2_TG0 mask
  arm64: Cleanup VTCR_EL2 computation
  arm: kvm: Move fake PGD handling to arch specific files
  arm64: kvm: Rewrite fake pgd handling
  arm64: Check for selected granule support
  arm64: Add 16K page size support
  arm64: 36 bit VA

 arch/arm/include/asm/kvm_mmu.h          |    7 ++
 arch/arm/kvm/mmu.c                      |   44 ++--------
 arch/arm64/Kconfig                      |   37 +++++++--
 arch/arm64/Kconfig.debug                |    2 +-
 arch/arm64/include/asm/boot.h           |    1 +
 arch/arm64/include/asm/fixmap.h         |    4 +-
 arch/arm64/include/asm/kernel-pgtable.h |   77 ++++++++++++++++++
 arch/arm64/include/asm/kvm_arm.h        |   29 +++++--
 arch/arm64/include/asm/kvm_mmu.h        |  135 +++++++++++++++++++++----------
 arch/arm64/include/asm/page.h           |   20 +----
 arch/arm64/include/asm/pgtable-hwdef.h  |   15 +++-
 arch/arm64/include/asm/sysreg.h         |    8 ++
 arch/arm64/include/asm/thread_info.h    |    4 +-
 arch/arm64/kernel/head.S                |   71 +++++++++-------
 arch/arm64/kernel/vmlinux.lds.S         |    1 +
 arch/arm64/mm/mmu.c                     |   70 +++++++---------
 arch/arm64/mm/proc.S                    |    4 +-
 17 files changed, 337 insertions(+), 192 deletions(-)
 create mode 100644 arch/arm64/include/asm/kernel-pgtable.h

-- 
1.7.9.5

[PATCH 01/14] arm64: Move swapper pagetable definitions

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Move the kernel pagetable (both swapper and idmap) definitions
from the generic asm/page.h to a new file, asm/kernel-pgtable.h.

This is mostly a cosmetic change, to clean up the asm/page.h to
get rid of the arch specific details which are not needed by the
generic code.

Also renames the symbols to prevent conflicts. e.g,
 	BLOCK_SHIFT => SWAPPER_BLOCK_SHIFT

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kernel-pgtable.h |   65 +++++++++++++++++++++++++++++++
 arch/arm64/include/asm/page.h           |   18 ---------
 arch/arm64/kernel/head.S                |   37 ++++--------------
 arch/arm64/kernel/vmlinux.lds.S         |    1 +
 4 files changed, 74 insertions(+), 47 deletions(-)
 create mode 100644 arch/arm64/include/asm/kernel-pgtable.h

diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
new file mode 100644
index 0000000..622929d
--- /dev/null
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -0,0 +1,65 @@
+/*
+ * asm/kernel-pgtable.h : Kernel page table mapping
+ *
+ * Copyright (C) 2015 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_KERNEL_PGTABLE_H
+#define __ASM_KERNEL_PGTABLE_H
+
+/*
+ * The idmap and swapper page tables need some space reserved in the kernel
+ * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
+ * map the kernel. With the 64K page configuration, swapper and idmap need to
+ * map to pte level. The swapper also maps the FDT (see __create_page_tables
+ * for more information). Note that the number of ID map translation levels
+ * could be increased on the fly if system RAM is out of reach for the default
+ * VA range, so 3 pages are reserved in all cases.
+ */
+#ifdef CONFIG_ARM64_64K_PAGES
+#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
+#else
+#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
+#endif
+
+#define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
+#define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
+
+/* Initial memory map size */
+#ifdef CONFIG_ARM64_64K_PAGES
+#define SWAPPER_BLOCK_SHIFT	PAGE_SHIFT
+#define SWAPPER_BLOCK_SIZE	PAGE_SIZE
+#define SWAPPER_TABLE_SHIFT	PMD_SHIFT
+#else
+#define SWAPPER_BLOCK_SHIFT	SECTION_SHIFT
+#define SWAPPER_BLOCK_SIZE	SECTION_SIZE
+#define SWAPPER_TABLE_SHIFT	PUD_SHIFT
+#endif
+
+
+/*
+ * Initial memory map attributes.
+ */
+#define SWAPPER_PTE_FLAGS	PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
+#define SWAPPER_PMD_FLAGS	PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
+
+#ifdef CONFIG_ARM64_64K_PAGES
+#define SWAPPER_MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
+#else
+#define SWAPPER_MM_MMUFLAGS	PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS
+#endif
+
+
+#endif
diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h
index 7d9c7e4..3c9ce8c 100644
--- a/arch/arm64/include/asm/page.h
+++ b/arch/arm64/include/asm/page.h
@@ -28,24 +28,6 @@
 #define PAGE_SIZE		(_AC(1,UL) << PAGE_SHIFT)
 #define PAGE_MASK		(~(PAGE_SIZE-1))
 
-/*
- * The idmap and swapper page tables need some space reserved in the kernel
- * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
- * map the kernel. With the 64K page configuration, swapper and idmap need to
- * map to pte level. The swapper also maps the FDT (see __create_page_tables
- * for more information). Note that the number of ID map translation levels
- * could be increased on the fly if system RAM is out of reach for the default
- * VA range, so 3 pages are reserved in all cases.
- */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
-#else
-#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
-#endif
-
-#define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
-#define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
-
 #ifndef __ASSEMBLY__
 
 #include <asm/pgtable-types.h>
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index a055be6..46670bf 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -29,6 +29,7 @@
 #include <asm/asm-offsets.h>
 #include <asm/cache.h>
 #include <asm/cputype.h>
+#include <asm/kernel-pgtable.h>
 #include <asm/memory.h>
 #include <asm/thread_info.h>
 #include <asm/pgtable-hwdef.h>
@@ -46,32 +47,10 @@
 #error TEXT_OFFSET must be less than 2MB
 #endif
 
-#ifdef CONFIG_ARM64_64K_PAGES
-#define BLOCK_SHIFT	PAGE_SHIFT
-#define BLOCK_SIZE	PAGE_SIZE
-#define TABLE_SHIFT	PMD_SHIFT
-#else
-#define BLOCK_SHIFT	SECTION_SHIFT
-#define BLOCK_SIZE	SECTION_SIZE
-#define TABLE_SHIFT	PUD_SHIFT
-#endif
-
 #define KERNEL_START	_text
 #define KERNEL_END	_end
 
 /*
- * Initial memory map attributes.
- */
-#define PTE_FLAGS	PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
-#define PMD_FLAGS	PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
-
-#ifdef CONFIG_ARM64_64K_PAGES
-#define MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | PTE_FLAGS
-#else
-#define MM_MMUFLAGS	PMD_ATTRINDX(MT_NORMAL) | PMD_FLAGS
-#endif
-
-/*
  * Kernel startup entry point.
  * ---------------------------
  *
@@ -293,7 +272,7 @@ ENDPROC(preserve_boot_args)
 	.macro	create_pgd_entry, tbl, virt, tmp1, tmp2
 	create_table_entry \tbl, \virt, PGDIR_SHIFT, PTRS_PER_PGD, \tmp1, \tmp2
 #if SWAPPER_PGTABLE_LEVELS == 3
-	create_table_entry \tbl, \virt, TABLE_SHIFT, PTRS_PER_PTE, \tmp1, \tmp2
+	create_table_entry \tbl, \virt, SWAPPER_TABLE_SHIFT, PTRS_PER_PTE, \tmp1, \tmp2
 #endif
 	.endm
 
@@ -305,15 +284,15 @@ ENDPROC(preserve_boot_args)
  * Corrupts:	phys, start, end, pstate
  */
 	.macro	create_block_map, tbl, flags, phys, start, end
-	lsr	\phys, \phys, #BLOCK_SHIFT
-	lsr	\start, \start, #BLOCK_SHIFT
+	lsr	\phys, \phys, #SWAPPER_BLOCK_SHIFT
+	lsr	\start, \start, #SWAPPER_BLOCK_SHIFT
 	and	\start, \start, #PTRS_PER_PTE - 1	// table index
-	orr	\phys, \flags, \phys, lsl #BLOCK_SHIFT	// table entry
-	lsr	\end, \end, #BLOCK_SHIFT
+	orr	\phys, \flags, \phys, lsl #SWAPPER_BLOCK_SHIFT	// table entry
+	lsr	\end, \end, #SWAPPER_BLOCK_SHIFT
 	and	\end, \end, #PTRS_PER_PTE - 1		// table end index
 9999:	str	\phys, [\tbl, \start, lsl #3]		// store the entry
 	add	\start, \start, #1			// next entry
-	add	\phys, \phys, #BLOCK_SIZE		// next block
+	add	\phys, \phys, #SWAPPER_BLOCK_SIZE		// next block
 	cmp	\start, \end
 	b.ls	9999b
 	.endm
@@ -350,7 +329,7 @@ __create_page_tables:
 	cmp	x0, x6
 	b.lo	1b
 
-	ldr	x7, =MM_MMUFLAGS
+	ldr	x7, =SWAPPER_MM_MMUFLAGS
 
 	/*
 	 * Create the identity mapping.
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index 9807333..8a5d97b 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -5,6 +5,7 @@
  */
 
 #include <asm-generic/vmlinux.lds.h>
+#include <asm/kernel-pgtable.h>
 #include <asm/thread_info.h>
 #include <asm/memory.h>
 #include <asm/page.h>
-- 
1.7.9.5

[PATCH 01/14] arm64: Move swapper pagetable definitions

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Move the kernel pagetable (both swapper and idmap) definitions
from the generic asm/page.h to a new file, asm/kernel-pgtable.h.

This is mostly a cosmetic change, to clean up the asm/page.h to
get rid of the arch specific details which are not needed by the
generic code.

Also renames the symbols to prevent conflicts. e.g,
 	BLOCK_SHIFT => SWAPPER_BLOCK_SHIFT

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kernel-pgtable.h |   65 +++++++++++++++++++++++++++++++
 arch/arm64/include/asm/page.h           |   18 ---------
 arch/arm64/kernel/head.S                |   37 ++++--------------
 arch/arm64/kernel/vmlinux.lds.S         |    1 +
 4 files changed, 74 insertions(+), 47 deletions(-)
 create mode 100644 arch/arm64/include/asm/kernel-pgtable.h

diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
new file mode 100644
index 0000000..622929d
--- /dev/null
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -0,0 +1,65 @@
+/*
+ * asm/kernel-pgtable.h : Kernel page table mapping
+ *
+ * Copyright (C) 2015 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_KERNEL_PGTABLE_H
+#define __ASM_KERNEL_PGTABLE_H
+
+/*
+ * The idmap and swapper page tables need some space reserved in the kernel
+ * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
+ * map the kernel. With the 64K page configuration, swapper and idmap need to
+ * map to pte level. The swapper also maps the FDT (see __create_page_tables
+ * for more information). Note that the number of ID map translation levels
+ * could be increased on the fly if system RAM is out of reach for the default
+ * VA range, so 3 pages are reserved in all cases.
+ */
+#ifdef CONFIG_ARM64_64K_PAGES
+#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
+#else
+#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
+#endif
+
+#define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
+#define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
+
+/* Initial memory map size */
+#ifdef CONFIG_ARM64_64K_PAGES
+#define SWAPPER_BLOCK_SHIFT	PAGE_SHIFT
+#define SWAPPER_BLOCK_SIZE	PAGE_SIZE
+#define SWAPPER_TABLE_SHIFT	PMD_SHIFT
+#else
+#define SWAPPER_BLOCK_SHIFT	SECTION_SHIFT
+#define SWAPPER_BLOCK_SIZE	SECTION_SIZE
+#define SWAPPER_TABLE_SHIFT	PUD_SHIFT
+#endif
+
+
+/*
+ * Initial memory map attributes.
+ */
+#define SWAPPER_PTE_FLAGS	PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
+#define SWAPPER_PMD_FLAGS	PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
+
+#ifdef CONFIG_ARM64_64K_PAGES
+#define SWAPPER_MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
+#else
+#define SWAPPER_MM_MMUFLAGS	PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS
+#endif
+
+
+#endif
diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h
index 7d9c7e4..3c9ce8c 100644
--- a/arch/arm64/include/asm/page.h
+++ b/arch/arm64/include/asm/page.h
@@ -28,24 +28,6 @@
 #define PAGE_SIZE		(_AC(1,UL) << PAGE_SHIFT)
 #define PAGE_MASK		(~(PAGE_SIZE-1))
 
-/*
- * The idmap and swapper page tables need some space reserved in the kernel
- * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
- * map the kernel. With the 64K page configuration, swapper and idmap need to
- * map to pte level. The swapper also maps the FDT (see __create_page_tables
- * for more information). Note that the number of ID map translation levels
- * could be increased on the fly if system RAM is out of reach for the default
- * VA range, so 3 pages are reserved in all cases.
- */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
-#else
-#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
-#endif
-
-#define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
-#define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
-
 #ifndef __ASSEMBLY__
 
 #include <asm/pgtable-types.h>
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index a055be6..46670bf 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -29,6 +29,7 @@
 #include <asm/asm-offsets.h>
 #include <asm/cache.h>
 #include <asm/cputype.h>
+#include <asm/kernel-pgtable.h>
 #include <asm/memory.h>
 #include <asm/thread_info.h>
 #include <asm/pgtable-hwdef.h>
@@ -46,32 +47,10 @@
 #error TEXT_OFFSET must be less than 2MB
 #endif
 
-#ifdef CONFIG_ARM64_64K_PAGES
-#define BLOCK_SHIFT	PAGE_SHIFT
-#define BLOCK_SIZE	PAGE_SIZE
-#define TABLE_SHIFT	PMD_SHIFT
-#else
-#define BLOCK_SHIFT	SECTION_SHIFT
-#define BLOCK_SIZE	SECTION_SIZE
-#define TABLE_SHIFT	PUD_SHIFT
-#endif
-
 #define KERNEL_START	_text
 #define KERNEL_END	_end
 
 /*
- * Initial memory map attributes.
- */
-#define PTE_FLAGS	PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
-#define PMD_FLAGS	PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
-
-#ifdef CONFIG_ARM64_64K_PAGES
-#define MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | PTE_FLAGS
-#else
-#define MM_MMUFLAGS	PMD_ATTRINDX(MT_NORMAL) | PMD_FLAGS
-#endif
-
-/*
  * Kernel startup entry point.
  * ---------------------------
  *
@@ -293,7 +272,7 @@ ENDPROC(preserve_boot_args)
 	.macro	create_pgd_entry, tbl, virt, tmp1, tmp2
 	create_table_entry \tbl, \virt, PGDIR_SHIFT, PTRS_PER_PGD, \tmp1, \tmp2
 #if SWAPPER_PGTABLE_LEVELS == 3
-	create_table_entry \tbl, \virt, TABLE_SHIFT, PTRS_PER_PTE, \tmp1, \tmp2
+	create_table_entry \tbl, \virt, SWAPPER_TABLE_SHIFT, PTRS_PER_PTE, \tmp1, \tmp2
 #endif
 	.endm
 
@@ -305,15 +284,15 @@ ENDPROC(preserve_boot_args)
  * Corrupts:	phys, start, end, pstate
  */
 	.macro	create_block_map, tbl, flags, phys, start, end
-	lsr	\phys, \phys, #BLOCK_SHIFT
-	lsr	\start, \start, #BLOCK_SHIFT
+	lsr	\phys, \phys, #SWAPPER_BLOCK_SHIFT
+	lsr	\start, \start, #SWAPPER_BLOCK_SHIFT
 	and	\start, \start, #PTRS_PER_PTE - 1	// table index
-	orr	\phys, \flags, \phys, lsl #BLOCK_SHIFT	// table entry
-	lsr	\end, \end, #BLOCK_SHIFT
+	orr	\phys, \flags, \phys, lsl #SWAPPER_BLOCK_SHIFT	// table entry
+	lsr	\end, \end, #SWAPPER_BLOCK_SHIFT
 	and	\end, \end, #PTRS_PER_PTE - 1		// table end index
 9999:	str	\phys, [\tbl, \start, lsl #3]		// store the entry
 	add	\start, \start, #1			// next entry
-	add	\phys, \phys, #BLOCK_SIZE		// next block
+	add	\phys, \phys, #SWAPPER_BLOCK_SIZE		// next block
 	cmp	\start, \end
 	b.ls	9999b
 	.endm
@@ -350,7 +329,7 @@ __create_page_tables:
 	cmp	x0, x6
 	b.lo	1b
 
-	ldr	x7, =MM_MMUFLAGS
+	ldr	x7, =SWAPPER_MM_MMUFLAGS
 
 	/*
 	 * Create the identity mapping.
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index 9807333..8a5d97b 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -5,6 +5,7 @@
  */
 
 #include <asm-generic/vmlinux.lds.h>
+#include <asm/kernel-pgtable.h>
 #include <asm/thread_info.h>
 #include <asm/memory.h>
 #include <asm/page.h>
-- 
1.7.9.5

[PATCH 01/14] arm64: Move swapper pagetable definitions

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Move the kernel pagetable (both swapper and idmap) definitions
from the generic asm/page.h to a new file, asm/kernel-pgtable.h.

This is mostly a cosmetic change, to clean up the asm/page.h to
get rid of the arch specific details which are not needed by the
generic code.

Also renames the symbols to prevent conflicts. e.g,
 	BLOCK_SHIFT => SWAPPER_BLOCK_SHIFT

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kernel-pgtable.h |   65 +++++++++++++++++++++++++++++++
 arch/arm64/include/asm/page.h           |   18 ---------
 arch/arm64/kernel/head.S                |   37 ++++--------------
 arch/arm64/kernel/vmlinux.lds.S         |    1 +
 4 files changed, 74 insertions(+), 47 deletions(-)
 create mode 100644 arch/arm64/include/asm/kernel-pgtable.h

diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
new file mode 100644
index 0000000..622929d
--- /dev/null
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -0,0 +1,65 @@
+/*
+ * asm/kernel-pgtable.h : Kernel page table mapping
+ *
+ * Copyright (C) 2015 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_KERNEL_PGTABLE_H
+#define __ASM_KERNEL_PGTABLE_H
+
+/*
+ * The idmap and swapper page tables need some space reserved in the kernel
+ * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
+ * map the kernel. With the 64K page configuration, swapper and idmap need to
+ * map to pte level. The swapper also maps the FDT (see __create_page_tables
+ * for more information). Note that the number of ID map translation levels
+ * could be increased on the fly if system RAM is out of reach for the default
+ * VA range, so 3 pages are reserved in all cases.
+ */
+#ifdef CONFIG_ARM64_64K_PAGES
+#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
+#else
+#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
+#endif
+
+#define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
+#define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
+
+/* Initial memory map size */
+#ifdef CONFIG_ARM64_64K_PAGES
+#define SWAPPER_BLOCK_SHIFT	PAGE_SHIFT
+#define SWAPPER_BLOCK_SIZE	PAGE_SIZE
+#define SWAPPER_TABLE_SHIFT	PMD_SHIFT
+#else
+#define SWAPPER_BLOCK_SHIFT	SECTION_SHIFT
+#define SWAPPER_BLOCK_SIZE	SECTION_SIZE
+#define SWAPPER_TABLE_SHIFT	PUD_SHIFT
+#endif
+
+
+/*
+ * Initial memory map attributes.
+ */
+#define SWAPPER_PTE_FLAGS	PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
+#define SWAPPER_PMD_FLAGS	PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
+
+#ifdef CONFIG_ARM64_64K_PAGES
+#define SWAPPER_MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
+#else
+#define SWAPPER_MM_MMUFLAGS	PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS
+#endif
+
+
+#endif
diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h
index 7d9c7e4..3c9ce8c 100644
--- a/arch/arm64/include/asm/page.h
+++ b/arch/arm64/include/asm/page.h
@@ -28,24 +28,6 @@
 #define PAGE_SIZE		(_AC(1,UL) << PAGE_SHIFT)
 #define PAGE_MASK		(~(PAGE_SIZE-1))
 
-/*
- * The idmap and swapper page tables need some space reserved in the kernel
- * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
- * map the kernel. With the 64K page configuration, swapper and idmap need to
- * map to pte level. The swapper also maps the FDT (see __create_page_tables
- * for more information). Note that the number of ID map translation levels
- * could be increased on the fly if system RAM is out of reach for the default
- * VA range, so 3 pages are reserved in all cases.
- */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
-#else
-#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
-#endif
-
-#define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
-#define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
-
 #ifndef __ASSEMBLY__
 
 #include <asm/pgtable-types.h>
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index a055be6..46670bf 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -29,6 +29,7 @@
 #include <asm/asm-offsets.h>
 #include <asm/cache.h>
 #include <asm/cputype.h>
+#include <asm/kernel-pgtable.h>
 #include <asm/memory.h>
 #include <asm/thread_info.h>
 #include <asm/pgtable-hwdef.h>
@@ -46,32 +47,10 @@
 #error TEXT_OFFSET must be less than 2MB
 #endif
 
-#ifdef CONFIG_ARM64_64K_PAGES
-#define BLOCK_SHIFT	PAGE_SHIFT
-#define BLOCK_SIZE	PAGE_SIZE
-#define TABLE_SHIFT	PMD_SHIFT
-#else
-#define BLOCK_SHIFT	SECTION_SHIFT
-#define BLOCK_SIZE	SECTION_SIZE
-#define TABLE_SHIFT	PUD_SHIFT
-#endif
-
 #define KERNEL_START	_text
 #define KERNEL_END	_end
 
 /*
- * Initial memory map attributes.
- */
-#define PTE_FLAGS	PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
-#define PMD_FLAGS	PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
-
-#ifdef CONFIG_ARM64_64K_PAGES
-#define MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | PTE_FLAGS
-#else
-#define MM_MMUFLAGS	PMD_ATTRINDX(MT_NORMAL) | PMD_FLAGS
-#endif
-
-/*
  * Kernel startup entry point.
  * ---------------------------
  *
@@ -293,7 +272,7 @@ ENDPROC(preserve_boot_args)
 	.macro	create_pgd_entry, tbl, virt, tmp1, tmp2
 	create_table_entry \tbl, \virt, PGDIR_SHIFT, PTRS_PER_PGD, \tmp1, \tmp2
 #if SWAPPER_PGTABLE_LEVELS == 3
-	create_table_entry \tbl, \virt, TABLE_SHIFT, PTRS_PER_PTE, \tmp1, \tmp2
+	create_table_entry \tbl, \virt, SWAPPER_TABLE_SHIFT, PTRS_PER_PTE, \tmp1, \tmp2
 #endif
 	.endm
 
@@ -305,15 +284,15 @@ ENDPROC(preserve_boot_args)
  * Corrupts:	phys, start, end, pstate
  */
 	.macro	create_block_map, tbl, flags, phys, start, end
-	lsr	\phys, \phys, #BLOCK_SHIFT
-	lsr	\start, \start, #BLOCK_SHIFT
+	lsr	\phys, \phys, #SWAPPER_BLOCK_SHIFT
+	lsr	\start, \start, #SWAPPER_BLOCK_SHIFT
 	and	\start, \start, #PTRS_PER_PTE - 1	// table index
-	orr	\phys, \flags, \phys, lsl #BLOCK_SHIFT	// table entry
-	lsr	\end, \end, #BLOCK_SHIFT
+	orr	\phys, \flags, \phys, lsl #SWAPPER_BLOCK_SHIFT	// table entry
+	lsr	\end, \end, #SWAPPER_BLOCK_SHIFT
 	and	\end, \end, #PTRS_PER_PTE - 1		// table end index
 9999:	str	\phys, [\tbl, \start, lsl #3]		// store the entry
 	add	\start, \start, #1			// next entry
-	add	\phys, \phys, #BLOCK_SIZE		// next block
+	add	\phys, \phys, #SWAPPER_BLOCK_SIZE		// next block
 	cmp	\start, \end
 	b.ls	9999b
 	.endm
@@ -350,7 +329,7 @@ __create_page_tables:
 	cmp	x0, x6
 	b.lo	1b
 
-	ldr	x7, =MM_MMUFLAGS
+	ldr	x7, =SWAPPER_MM_MMUFLAGS
 
 	/*
 	 * Create the identity mapping.
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index 9807333..8a5d97b 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -5,6 +5,7 @@
  */
 
 #include <asm-generic/vmlinux.lds.h>
+#include <asm/kernel-pgtable.h>
 #include <asm/thread_info.h>
 #include <asm/memory.h>
 #include <asm/page.h>
-- 
1.7.9.5

[PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

We use section maps with 4K page size to create the
swapper/idmaps. So far we have used !64K or 4K checks
to handle the case where we use the section maps. This
patch adds a symbol to make it clear those cases.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kernel-pgtable.h |   31 +++++++++-----
 arch/arm64/mm/mmu.c                     |   70 ++++++++++++++-----------------
 2 files changed, 51 insertions(+), 50 deletions(-)

diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
index 622929d..5876a36 100644
--- a/arch/arm64/include/asm/kernel-pgtable.h
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -19,6 +19,13 @@
 #ifndef __ASM_KERNEL_PGTABLE_H
 #define __ASM_KERNEL_PGTABLE_H
 
+/* With 4K pages, we use section maps. */
+#ifdef CONFIG_ARM64_4K_PAGES
+#define ARM64_SWAPPER_USES_SECTION_MAPS 1
+#else
+#define ARM64_SWAPPER_USES_SECTION_MAPS 0
+#endif
+
 /*
  * The idmap and swapper page tables need some space reserved in the kernel
  * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
@@ -28,26 +35,28 @@
  * could be increased on the fly if system RAM is out of reach for the default
  * VA range, so 3 pages are reserved in all cases.
  */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
-#else
+#if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
+#else
+#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
 #endif
 
 #define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
 #define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
 
 /* Initial memory map size */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_BLOCK_SHIFT	PAGE_SHIFT
-#define SWAPPER_BLOCK_SIZE	PAGE_SIZE
-#define SWAPPER_TABLE_SHIFT	PMD_SHIFT
-#else
+#if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_BLOCK_SHIFT	SECTION_SHIFT
 #define SWAPPER_BLOCK_SIZE	SECTION_SIZE
 #define SWAPPER_TABLE_SHIFT	PUD_SHIFT
+#else
+#define SWAPPER_BLOCK_SHIFT	PAGE_SHIFT
+#define SWAPPER_BLOCK_SIZE	PAGE_SIZE
+#define SWAPPER_TABLE_SHIFT	PMD_SHIFT
 #endif
 
+/* The size of the initial kernel direct mapping */
+#define SWAPPER_INIT_MAP_SIZE	(_AC(1, UL) << SWAPPER_TABLE_SHIFT)
 
 /*
  * Initial memory map attributes.
@@ -55,10 +64,10 @@
 #define SWAPPER_PTE_FLAGS	PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
 #define SWAPPER_PMD_FLAGS	PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
 
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
-#else
+#if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_MM_MMUFLAGS	PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS
+#else
+#define SWAPPER_MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
 #endif
 
 
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 9211b85..71230488 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -32,6 +32,7 @@
 
 #include <asm/cputype.h>
 #include <asm/fixmap.h>
+#include <asm/kernel-pgtable.h>
 #include <asm/sections.h>
 #include <asm/setup.h>
 #include <asm/sizes.h>
@@ -353,14 +354,11 @@ static void __init map_mem(void)
 	 * memory addressable from the initial direct kernel mapping.
 	 *
 	 * The initial direct kernel mapping, located at swapper_pg_dir, gives
-	 * us PUD_SIZE (4K pages) or PMD_SIZE (64K pages) memory starting from
-	 * PHYS_OFFSET (which must be aligned to 2MB as per
-	 * Documentation/arm64/booting.txt).
+	 * us PUD_SIZE (with SECTION maps, i.e, 4K) or PMD_SIZE (without
+	 * SECTION maps, i.e, 64K pages) memory starting from PHYS_OFFSET
+	 * (which must be aligned to 2MB as per Documentation/arm64/booting.txt).
 	 */
-	if (IS_ENABLED(CONFIG_ARM64_64K_PAGES))
-		limit = PHYS_OFFSET + PMD_SIZE;
-	else
-		limit = PHYS_OFFSET + PUD_SIZE;
+	limit = PHYS_OFFSET + SWAPPER_INIT_MAP_SIZE;
 	memblock_set_current_limit(limit);
 
 	/* map all the memory banks */
@@ -371,21 +369,24 @@ static void __init map_mem(void)
 		if (start >= end)
 			break;
 
-#ifndef CONFIG_ARM64_64K_PAGES
-		/*
-		 * For the first memory bank align the start address and
-		 * current memblock limit to prevent create_mapping() from
-		 * allocating pte page tables from unmapped memory.
-		 * When 64K pages are enabled, the pte page table for the
-		 * first PGDIR_SIZE is already present in swapper_pg_dir.
-		 */
-		if (start < limit)
-			start = ALIGN(start, PMD_SIZE);
-		if (end < limit) {
-			limit = end & PMD_MASK;
-			memblock_set_current_limit(limit);
+		if (ARM64_SWAPPER_USES_SECTION_MAPS) {
+			/*
+			 * For the first memory bank align the start address and
+			 * current memblock limit to prevent create_mapping() from
+			 * allocating pte page tables from unmapped memory. With
+			 * the section maps, if the first block doesn't end on section
+			 * size boundary, create_mapping() will try to allocate a pte
+			 * page, which may be returned from an unmapped area.
+			 * When section maps are not used, the pte page table for the
+			 * current limit is already present in swapper_pg_dir.
+			 */
+			if (start < limit)
+				start = ALIGN(start, SECTION_SIZE);
+			if (end < limit) {
+				limit = end & SECTION_MASK;
+				memblock_set_current_limit(limit);
+			}
 		}
-#endif
 		__map_memblock(start, end);
 	}
 
@@ -638,7 +639,7 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
 {
 	const u64 dt_virt_base = __fix_to_virt(FIX_FDT);
 	pgprot_t prot = PAGE_KERNEL | PTE_RDONLY;
-	int granularity, size, offset;
+	int size, offset;
 	void *dt_virt;
 
 	/*
@@ -664,24 +665,15 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
 	 */
 	BUILD_BUG_ON(dt_virt_base % SZ_2M);
 
-	if (IS_ENABLED(CONFIG_ARM64_64K_PAGES)) {
-		BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PMD_SHIFT !=
-			     __fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT);
-
-		granularity = PAGE_SIZE;
-	} else {
-		BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PUD_SHIFT !=
-			     __fix_to_virt(FIX_BTMAP_BEGIN) >> PUD_SHIFT);
-
-		granularity = PMD_SIZE;
-	}
+	BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> SWAPPER_TABLE_SHIFT !=
+		     __fix_to_virt(FIX_BTMAP_BEGIN) >> SWAPPER_TABLE_SHIFT);
 
-	offset = dt_phys % granularity;
+	offset = dt_phys % SWAPPER_BLOCK_SIZE;
 	dt_virt = (void *)dt_virt_base + offset;
 
 	/* map the first chunk so we can read the size from the header */
-	create_mapping(round_down(dt_phys, granularity), dt_virt_base,
-		       granularity, prot);
+	create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
+		       SWAPPER_BLOCK_SIZE, prot);
 
 	if (fdt_check_header(dt_virt) != 0)
 		return NULL;
@@ -690,9 +682,9 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
 	if (size > MAX_FDT_SIZE)
 		return NULL;
 
-	if (offset + size > granularity)
-		create_mapping(round_down(dt_phys, granularity), dt_virt_base,
-			       round_up(offset + size, granularity), prot);
+	if (offset + size > SWAPPER_BLOCK_SIZE)
+		create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
+			       round_up(offset + size, SWAPPER_BLOCK_SIZE), prot);
 
 	memblock_reserve(dt_phys, size);
 
-- 
1.7.9.5

[PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

We use section maps with 4K page size to create the
swapper/idmaps. So far we have used !64K or 4K checks
to handle the case where we use the section maps. This
patch adds a symbol to make it clear those cases.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kernel-pgtable.h |   31 +++++++++-----
 arch/arm64/mm/mmu.c                     |   70 ++++++++++++++-----------------
 2 files changed, 51 insertions(+), 50 deletions(-)

diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
index 622929d..5876a36 100644
--- a/arch/arm64/include/asm/kernel-pgtable.h
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -19,6 +19,13 @@
 #ifndef __ASM_KERNEL_PGTABLE_H
 #define __ASM_KERNEL_PGTABLE_H
 
+/* With 4K pages, we use section maps. */
+#ifdef CONFIG_ARM64_4K_PAGES
+#define ARM64_SWAPPER_USES_SECTION_MAPS 1
+#else
+#define ARM64_SWAPPER_USES_SECTION_MAPS 0
+#endif
+
 /*
  * The idmap and swapper page tables need some space reserved in the kernel
  * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
@@ -28,26 +35,28 @@
  * could be increased on the fly if system RAM is out of reach for the default
  * VA range, so 3 pages are reserved in all cases.
  */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
-#else
+#if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
+#else
+#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
 #endif
 
 #define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
 #define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
 
 /* Initial memory map size */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_BLOCK_SHIFT	PAGE_SHIFT
-#define SWAPPER_BLOCK_SIZE	PAGE_SIZE
-#define SWAPPER_TABLE_SHIFT	PMD_SHIFT
-#else
+#if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_BLOCK_SHIFT	SECTION_SHIFT
 #define SWAPPER_BLOCK_SIZE	SECTION_SIZE
 #define SWAPPER_TABLE_SHIFT	PUD_SHIFT
+#else
+#define SWAPPER_BLOCK_SHIFT	PAGE_SHIFT
+#define SWAPPER_BLOCK_SIZE	PAGE_SIZE
+#define SWAPPER_TABLE_SHIFT	PMD_SHIFT
 #endif
 
+/* The size of the initial kernel direct mapping */
+#define SWAPPER_INIT_MAP_SIZE	(_AC(1, UL) << SWAPPER_TABLE_SHIFT)
 
 /*
  * Initial memory map attributes.
@@ -55,10 +64,10 @@
 #define SWAPPER_PTE_FLAGS	PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
 #define SWAPPER_PMD_FLAGS	PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
 
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
-#else
+#if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_MM_MMUFLAGS	PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS
+#else
+#define SWAPPER_MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
 #endif
 
 
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 9211b85..71230488 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -32,6 +32,7 @@
 
 #include <asm/cputype.h>
 #include <asm/fixmap.h>
+#include <asm/kernel-pgtable.h>
 #include <asm/sections.h>
 #include <asm/setup.h>
 #include <asm/sizes.h>
@@ -353,14 +354,11 @@ static void __init map_mem(void)
 	 * memory addressable from the initial direct kernel mapping.
 	 *
 	 * The initial direct kernel mapping, located at swapper_pg_dir, gives
-	 * us PUD_SIZE (4K pages) or PMD_SIZE (64K pages) memory starting from
-	 * PHYS_OFFSET (which must be aligned to 2MB as per
-	 * Documentation/arm64/booting.txt).
+	 * us PUD_SIZE (with SECTION maps, i.e, 4K) or PMD_SIZE (without
+	 * SECTION maps, i.e, 64K pages) memory starting from PHYS_OFFSET
+	 * (which must be aligned to 2MB as per Documentation/arm64/booting.txt).
 	 */
-	if (IS_ENABLED(CONFIG_ARM64_64K_PAGES))
-		limit = PHYS_OFFSET + PMD_SIZE;
-	else
-		limit = PHYS_OFFSET + PUD_SIZE;
+	limit = PHYS_OFFSET + SWAPPER_INIT_MAP_SIZE;
 	memblock_set_current_limit(limit);
 
 	/* map all the memory banks */
@@ -371,21 +369,24 @@ static void __init map_mem(void)
 		if (start >= end)
 			break;
 
-#ifndef CONFIG_ARM64_64K_PAGES
-		/*
-		 * For the first memory bank align the start address and
-		 * current memblock limit to prevent create_mapping() from
-		 * allocating pte page tables from unmapped memory.
-		 * When 64K pages are enabled, the pte page table for the
-		 * first PGDIR_SIZE is already present in swapper_pg_dir.
-		 */
-		if (start < limit)
-			start = ALIGN(start, PMD_SIZE);
-		if (end < limit) {
-			limit = end & PMD_MASK;
-			memblock_set_current_limit(limit);
+		if (ARM64_SWAPPER_USES_SECTION_MAPS) {
+			/*
+			 * For the first memory bank align the start address and
+			 * current memblock limit to prevent create_mapping() from
+			 * allocating pte page tables from unmapped memory. With
+			 * the section maps, if the first block doesn't end on section
+			 * size boundary, create_mapping() will try to allocate a pte
+			 * page, which may be returned from an unmapped area.
+			 * When section maps are not used, the pte page table for the
+			 * current limit is already present in swapper_pg_dir.
+			 */
+			if (start < limit)
+				start = ALIGN(start, SECTION_SIZE);
+			if (end < limit) {
+				limit = end & SECTION_MASK;
+				memblock_set_current_limit(limit);
+			}
 		}
-#endif
 		__map_memblock(start, end);
 	}
 
@@ -638,7 +639,7 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
 {
 	const u64 dt_virt_base = __fix_to_virt(FIX_FDT);
 	pgprot_t prot = PAGE_KERNEL | PTE_RDONLY;
-	int granularity, size, offset;
+	int size, offset;
 	void *dt_virt;
 
 	/*
@@ -664,24 +665,15 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
 	 */
 	BUILD_BUG_ON(dt_virt_base % SZ_2M);
 
-	if (IS_ENABLED(CONFIG_ARM64_64K_PAGES)) {
-		BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PMD_SHIFT !=
-			     __fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT);
-
-		granularity = PAGE_SIZE;
-	} else {
-		BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PUD_SHIFT !=
-			     __fix_to_virt(FIX_BTMAP_BEGIN) >> PUD_SHIFT);
-
-		granularity = PMD_SIZE;
-	}
+	BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> SWAPPER_TABLE_SHIFT !=
+		     __fix_to_virt(FIX_BTMAP_BEGIN) >> SWAPPER_TABLE_SHIFT);
 
-	offset = dt_phys % granularity;
+	offset = dt_phys % SWAPPER_BLOCK_SIZE;
 	dt_virt = (void *)dt_virt_base + offset;
 
 	/* map the first chunk so we can read the size from the header */
-	create_mapping(round_down(dt_phys, granularity), dt_virt_base,
-		       granularity, prot);
+	create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
+		       SWAPPER_BLOCK_SIZE, prot);
 
 	if (fdt_check_header(dt_virt) != 0)
 		return NULL;
@@ -690,9 +682,9 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
 	if (size > MAX_FDT_SIZE)
 		return NULL;
 
-	if (offset + size > granularity)
-		create_mapping(round_down(dt_phys, granularity), dt_virt_base,
-			       round_up(offset + size, granularity), prot);
+	if (offset + size > SWAPPER_BLOCK_SIZE)
+		create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
+			       round_up(offset + size, SWAPPER_BLOCK_SIZE), prot);
 
 	memblock_reserve(dt_phys, size);
 
-- 
1.7.9.5

[PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

We use section maps with 4K page size to create the
swapper/idmaps. So far we have used !64K or 4K checks
to handle the case where we use the section maps. This
patch adds a symbol to make it clear those cases.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kernel-pgtable.h |   31 +++++++++-----
 arch/arm64/mm/mmu.c                     |   70 ++++++++++++++-----------------
 2 files changed, 51 insertions(+), 50 deletions(-)

diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
index 622929d..5876a36 100644
--- a/arch/arm64/include/asm/kernel-pgtable.h
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -19,6 +19,13 @@
 #ifndef __ASM_KERNEL_PGTABLE_H
 #define __ASM_KERNEL_PGTABLE_H
 
+/* With 4K pages, we use section maps. */
+#ifdef CONFIG_ARM64_4K_PAGES
+#define ARM64_SWAPPER_USES_SECTION_MAPS 1
+#else
+#define ARM64_SWAPPER_USES_SECTION_MAPS 0
+#endif
+
 /*
  * The idmap and swapper page tables need some space reserved in the kernel
  * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
@@ -28,26 +35,28 @@
  * could be increased on the fly if system RAM is out of reach for the default
  * VA range, so 3 pages are reserved in all cases.
  */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
-#else
+#if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
+#else
+#define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
 #endif
 
 #define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
 #define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
 
 /* Initial memory map size */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_BLOCK_SHIFT	PAGE_SHIFT
-#define SWAPPER_BLOCK_SIZE	PAGE_SIZE
-#define SWAPPER_TABLE_SHIFT	PMD_SHIFT
-#else
+#if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_BLOCK_SHIFT	SECTION_SHIFT
 #define SWAPPER_BLOCK_SIZE	SECTION_SIZE
 #define SWAPPER_TABLE_SHIFT	PUD_SHIFT
+#else
+#define SWAPPER_BLOCK_SHIFT	PAGE_SHIFT
+#define SWAPPER_BLOCK_SIZE	PAGE_SIZE
+#define SWAPPER_TABLE_SHIFT	PMD_SHIFT
 #endif
 
+/* The size of the initial kernel direct mapping */
+#define SWAPPER_INIT_MAP_SIZE	(_AC(1, UL) << SWAPPER_TABLE_SHIFT)
 
 /*
  * Initial memory map attributes.
@@ -55,10 +64,10 @@
 #define SWAPPER_PTE_FLAGS	PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
 #define SWAPPER_PMD_FLAGS	PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
 
-#ifdef CONFIG_ARM64_64K_PAGES
-#define SWAPPER_MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
-#else
+#if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_MM_MMUFLAGS	PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS
+#else
+#define SWAPPER_MM_MMUFLAGS	PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
 #endif
 
 
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 9211b85..71230488 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -32,6 +32,7 @@
 
 #include <asm/cputype.h>
 #include <asm/fixmap.h>
+#include <asm/kernel-pgtable.h>
 #include <asm/sections.h>
 #include <asm/setup.h>
 #include <asm/sizes.h>
@@ -353,14 +354,11 @@ static void __init map_mem(void)
 	 * memory addressable from the initial direct kernel mapping.
 	 *
 	 * The initial direct kernel mapping, located at swapper_pg_dir, gives
-	 * us PUD_SIZE (4K pages) or PMD_SIZE (64K pages) memory starting from
-	 * PHYS_OFFSET (which must be aligned to 2MB as per
-	 * Documentation/arm64/booting.txt).
+	 * us PUD_SIZE (with SECTION maps, i.e, 4K) or PMD_SIZE (without
+	 * SECTION maps, i.e, 64K pages) memory starting from PHYS_OFFSET
+	 * (which must be aligned to 2MB as per Documentation/arm64/booting.txt).
 	 */
-	if (IS_ENABLED(CONFIG_ARM64_64K_PAGES))
-		limit = PHYS_OFFSET + PMD_SIZE;
-	else
-		limit = PHYS_OFFSET + PUD_SIZE;
+	limit = PHYS_OFFSET + SWAPPER_INIT_MAP_SIZE;
 	memblock_set_current_limit(limit);
 
 	/* map all the memory banks */
@@ -371,21 +369,24 @@ static void __init map_mem(void)
 		if (start >= end)
 			break;
 
-#ifndef CONFIG_ARM64_64K_PAGES
-		/*
-		 * For the first memory bank align the start address and
-		 * current memblock limit to prevent create_mapping() from
-		 * allocating pte page tables from unmapped memory.
-		 * When 64K pages are enabled, the pte page table for the
-		 * first PGDIR_SIZE is already present in swapper_pg_dir.
-		 */
-		if (start < limit)
-			start = ALIGN(start, PMD_SIZE);
-		if (end < limit) {
-			limit = end & PMD_MASK;
-			memblock_set_current_limit(limit);
+		if (ARM64_SWAPPER_USES_SECTION_MAPS) {
+			/*
+			 * For the first memory bank align the start address and
+			 * current memblock limit to prevent create_mapping() from
+			 * allocating pte page tables from unmapped memory. With
+			 * the section maps, if the first block doesn't end on section
+			 * size boundary, create_mapping() will try to allocate a pte
+			 * page, which may be returned from an unmapped area.
+			 * When section maps are not used, the pte page table for the
+			 * current limit is already present in swapper_pg_dir.
+			 */
+			if (start < limit)
+				start = ALIGN(start, SECTION_SIZE);
+			if (end < limit) {
+				limit = end & SECTION_MASK;
+				memblock_set_current_limit(limit);
+			}
 		}
-#endif
 		__map_memblock(start, end);
 	}
 
@@ -638,7 +639,7 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
 {
 	const u64 dt_virt_base = __fix_to_virt(FIX_FDT);
 	pgprot_t prot = PAGE_KERNEL | PTE_RDONLY;
-	int granularity, size, offset;
+	int size, offset;
 	void *dt_virt;
 
 	/*
@@ -664,24 +665,15 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
 	 */
 	BUILD_BUG_ON(dt_virt_base % SZ_2M);
 
-	if (IS_ENABLED(CONFIG_ARM64_64K_PAGES)) {
-		BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PMD_SHIFT !=
-			     __fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT);
-
-		granularity = PAGE_SIZE;
-	} else {
-		BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PUD_SHIFT !=
-			     __fix_to_virt(FIX_BTMAP_BEGIN) >> PUD_SHIFT);
-
-		granularity = PMD_SIZE;
-	}
+	BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> SWAPPER_TABLE_SHIFT !=
+		     __fix_to_virt(FIX_BTMAP_BEGIN) >> SWAPPER_TABLE_SHIFT);
 
-	offset = dt_phys % granularity;
+	offset = dt_phys % SWAPPER_BLOCK_SIZE;
 	dt_virt = (void *)dt_virt_base + offset;
 
 	/* map the first chunk so we can read the size from the header */
-	create_mapping(round_down(dt_phys, granularity), dt_virt_base,
-		       granularity, prot);
+	create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
+		       SWAPPER_BLOCK_SIZE, prot);
 
 	if (fdt_check_header(dt_virt) != 0)
 		return NULL;
@@ -690,9 +682,9 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
 	if (size > MAX_FDT_SIZE)
 		return NULL;
 
-	if (offset + size > granularity)
-		create_mapping(round_down(dt_phys, granularity), dt_virt_base,
-			       round_up(offset + size, granularity), prot);
+	if (offset + size > SWAPPER_BLOCK_SIZE)
+		create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
+			       round_up(offset + size, SWAPPER_BLOCK_SIZE), prot);
 
 	memblock_reserve(dt_phys, size);
 
-- 
1.7.9.5

[PATCH 03/14] arm64: Introduce helpers for page table levels

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Introduce helpers for finding the number of page table
levels required for a given VA width, shift for a particular
page table level.

Convert the existing users to the new helpers. More users
to follow.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/pgtable-hwdef.h |   15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h
index 24154b0..ce18389 100644
--- a/arch/arm64/include/asm/pgtable-hwdef.h
+++ b/arch/arm64/include/asm/pgtable-hwdef.h
@@ -16,13 +16,21 @@
 #ifndef __ASM_PGTABLE_HWDEF_H
 #define __ASM_PGTABLE_HWDEF_H
 
+/*
+ * Number of page-table levels required to address 'va_bits' wide
+ * address, without section mapping
+ */
+#define ARM64_HW_PGTABLE_LEVELS(va_bits) (((va_bits) - 4) / (PAGE_SHIFT - 3))
+#define ARM64_HW_PGTABLE_LEVEL_SHIFT(level) \
+		((PAGE_SHIFT - 3) * (level) + 3)
+
 #define PTRS_PER_PTE		(1 << (PAGE_SHIFT - 3))
 
 /*
  * PMD_SHIFT determines the size a level 2 page table entry can map.
  */
 #if CONFIG_PGTABLE_LEVELS > 2
-#define PMD_SHIFT		((PAGE_SHIFT - 3) * 2 + 3)
+#define PMD_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(2)
 #define PMD_SIZE		(_AC(1, UL) << PMD_SHIFT)
 #define PMD_MASK		(~(PMD_SIZE-1))
 #define PTRS_PER_PMD		PTRS_PER_PTE
@@ -32,7 +40,7 @@
  * PUD_SHIFT determines the size a level 1 page table entry can map.
  */
 #if CONFIG_PGTABLE_LEVELS > 3
-#define PUD_SHIFT		((PAGE_SHIFT - 3) * 3 + 3)
+#define PUD_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(3)
 #define PUD_SIZE		(_AC(1, UL) << PUD_SHIFT)
 #define PUD_MASK		(~(PUD_SIZE-1))
 #define PTRS_PER_PUD		PTRS_PER_PTE
@@ -42,7 +50,8 @@
  * PGDIR_SHIFT determines the size a top-level page table entry can map
  * (depending on the configuration, this level can be 0, 1 or 2).
  */
-#define PGDIR_SHIFT		((PAGE_SHIFT - 3) * CONFIG_PGTABLE_LEVELS + 3)
+#define PGDIR_SHIFT	\
+		ARM64_HW_PGTABLE_LEVEL_SHIFT(CONFIG_PGTABLE_LEVELS)
 #define PGDIR_SIZE		(_AC(1, UL) << PGDIR_SHIFT)
 #define PGDIR_MASK		(~(PGDIR_SIZE-1))
 #define PTRS_PER_PGD		(1 << (VA_BITS - PGDIR_SHIFT))
-- 
1.7.9.5

[PATCH 03/14] arm64: Introduce helpers for page table levels

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Introduce helpers for finding the number of page table
levels required for a given VA width, shift for a particular
page table level.

Convert the existing users to the new helpers. More users
to follow.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/pgtable-hwdef.h |   15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h
index 24154b0..ce18389 100644
--- a/arch/arm64/include/asm/pgtable-hwdef.h
+++ b/arch/arm64/include/asm/pgtable-hwdef.h
@@ -16,13 +16,21 @@
 #ifndef __ASM_PGTABLE_HWDEF_H
 #define __ASM_PGTABLE_HWDEF_H
 
+/*
+ * Number of page-table levels required to address 'va_bits' wide
+ * address, without section mapping
+ */
+#define ARM64_HW_PGTABLE_LEVELS(va_bits) (((va_bits) - 4) / (PAGE_SHIFT - 3))
+#define ARM64_HW_PGTABLE_LEVEL_SHIFT(level) \
+		((PAGE_SHIFT - 3) * (level) + 3)
+
 #define PTRS_PER_PTE		(1 << (PAGE_SHIFT - 3))
 
 /*
  * PMD_SHIFT determines the size a level 2 page table entry can map.
  */
 #if CONFIG_PGTABLE_LEVELS > 2
-#define PMD_SHIFT		((PAGE_SHIFT - 3) * 2 + 3)
+#define PMD_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(2)
 #define PMD_SIZE		(_AC(1, UL) << PMD_SHIFT)
 #define PMD_MASK		(~(PMD_SIZE-1))
 #define PTRS_PER_PMD		PTRS_PER_PTE
@@ -32,7 +40,7 @@
  * PUD_SHIFT determines the size a level 1 page table entry can map.
  */
 #if CONFIG_PGTABLE_LEVELS > 3
-#define PUD_SHIFT		((PAGE_SHIFT - 3) * 3 + 3)
+#define PUD_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(3)
 #define PUD_SIZE		(_AC(1, UL) << PUD_SHIFT)
 #define PUD_MASK		(~(PUD_SIZE-1))
 #define PTRS_PER_PUD		PTRS_PER_PTE
@@ -42,7 +50,8 @@
  * PGDIR_SHIFT determines the size a top-level page table entry can map
  * (depending on the configuration, this level can be 0, 1 or 2).
  */
-#define PGDIR_SHIFT		((PAGE_SHIFT - 3) * CONFIG_PGTABLE_LEVELS + 3)
+#define PGDIR_SHIFT	\
+		ARM64_HW_PGTABLE_LEVEL_SHIFT(CONFIG_PGTABLE_LEVELS)
 #define PGDIR_SIZE		(_AC(1, UL) << PGDIR_SHIFT)
 #define PGDIR_MASK		(~(PGDIR_SIZE-1))
 #define PTRS_PER_PGD		(1 << (VA_BITS - PGDIR_SHIFT))
-- 
1.7.9.5

[PATCH 03/14] arm64: Introduce helpers for page table levels

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Introduce helpers for finding the number of page table
levels required for a given VA width, shift for a particular
page table level.

Convert the existing users to the new helpers. More users
to follow.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/pgtable-hwdef.h |   15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h
index 24154b0..ce18389 100644
--- a/arch/arm64/include/asm/pgtable-hwdef.h
+++ b/arch/arm64/include/asm/pgtable-hwdef.h
@@ -16,13 +16,21 @@
 #ifndef __ASM_PGTABLE_HWDEF_H
 #define __ASM_PGTABLE_HWDEF_H
 
+/*
+ * Number of page-table levels required to address 'va_bits' wide
+ * address, without section mapping
+ */
+#define ARM64_HW_PGTABLE_LEVELS(va_bits) (((va_bits) - 4) / (PAGE_SHIFT - 3))
+#define ARM64_HW_PGTABLE_LEVEL_SHIFT(level) \
+		((PAGE_SHIFT - 3) * (level) + 3)
+
 #define PTRS_PER_PTE		(1 << (PAGE_SHIFT - 3))
 
 /*
  * PMD_SHIFT determines the size a level 2 page table entry can map.
  */
 #if CONFIG_PGTABLE_LEVELS > 2
-#define PMD_SHIFT		((PAGE_SHIFT - 3) * 2 + 3)
+#define PMD_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(2)
 #define PMD_SIZE		(_AC(1, UL) << PMD_SHIFT)
 #define PMD_MASK		(~(PMD_SIZE-1))
 #define PTRS_PER_PMD		PTRS_PER_PTE
@@ -32,7 +40,7 @@
  * PUD_SHIFT determines the size a level 1 page table entry can map.
  */
 #if CONFIG_PGTABLE_LEVELS > 3
-#define PUD_SHIFT		((PAGE_SHIFT - 3) * 3 + 3)
+#define PUD_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(3)
 #define PUD_SIZE		(_AC(1, UL) << PUD_SHIFT)
 #define PUD_MASK		(~(PUD_SIZE-1))
 #define PTRS_PER_PUD		PTRS_PER_PTE
@@ -42,7 +50,8 @@
  * PGDIR_SHIFT determines the size a top-level page table entry can map
  * (depending on the configuration, this level can be 0, 1 or 2).
  */
-#define PGDIR_SHIFT		((PAGE_SHIFT - 3) * CONFIG_PGTABLE_LEVELS + 3)
+#define PGDIR_SHIFT	\
+		ARM64_HW_PGTABLE_LEVEL_SHIFT(CONFIG_PGTABLE_LEVELS)
 #define PGDIR_SIZE		(_AC(1, UL) << PGDIR_SHIFT)
 #define PGDIR_MASK		(~(PGDIR_SIZE-1))
 #define PTRS_PER_PGD		(1 << (VA_BITS - PGDIR_SHIFT))
-- 
1.7.9.5

[PATCH 04/14] arm64: Calculate size for idmap_pg_dir at compile time

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Now that we can calculate the number of levels required for
mapping a va width, reserve exact number of pages that would
be required to cover the idmap. The idmap should be able to handle
the maximum physical address size supported.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/boot.h           |    1 +
 arch/arm64/include/asm/kernel-pgtable.h |    7 +++++--
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/boot.h b/arch/arm64/include/asm/boot.h
index 81151b6..678b63e 100644
--- a/arch/arm64/include/asm/boot.h
+++ b/arch/arm64/include/asm/boot.h
@@ -2,6 +2,7 @@
 #ifndef __ASM_BOOT_H
 #define __ASM_BOOT_H
 
+#include <asm/page.h>
 #include <asm/sizes.h>
 
 /*
diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
index 5876a36..def7168 100644
--- a/arch/arm64/include/asm/kernel-pgtable.h
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -33,16 +33,19 @@
  * map to pte level. The swapper also maps the FDT (see __create_page_tables
  * for more information). Note that the number of ID map translation levels
  * could be increased on the fly if system RAM is out of reach for the default
- * VA range, so 3 pages are reserved in all cases.
+ * VA range, so pages required to map highest possible PA are reserved in all
+ * cases.
  */
 #if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
+#define IDMAP_PGTABLE_LEVELS	(ARM64_HW_PGTABLE_LEVELS(PHYS_MASK_SHIFT) - 1)
 #else
 #define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
+#define IDMAP_PGTABLE_LEVELS	(ARM64_HW_PGTABLE_LEVELS(PHYS_MASK_SHIFT))
 #endif
 
 #define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
-#define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
+#define IDMAP_DIR_SIZE		(IDMAP_PGTABLE_LEVELS * PAGE_SIZE)
 
 /* Initial memory map size */
 #if ARM64_SWAPPER_USES_SECTION_MAPS
-- 
1.7.9.5

[PATCH 04/14] arm64: Calculate size for idmap_pg_dir at compile time

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Now that we can calculate the number of levels required for
mapping a va width, reserve exact number of pages that would
be required to cover the idmap. The idmap should be able to handle
the maximum physical address size supported.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/boot.h           |    1 +
 arch/arm64/include/asm/kernel-pgtable.h |    7 +++++--
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/boot.h b/arch/arm64/include/asm/boot.h
index 81151b6..678b63e 100644
--- a/arch/arm64/include/asm/boot.h
+++ b/arch/arm64/include/asm/boot.h
@@ -2,6 +2,7 @@
 #ifndef __ASM_BOOT_H
 #define __ASM_BOOT_H
 
+#include <asm/page.h>
 #include <asm/sizes.h>
 
 /*
diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
index 5876a36..def7168 100644
--- a/arch/arm64/include/asm/kernel-pgtable.h
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -33,16 +33,19 @@
  * map to pte level. The swapper also maps the FDT (see __create_page_tables
  * for more information). Note that the number of ID map translation levels
  * could be increased on the fly if system RAM is out of reach for the default
- * VA range, so 3 pages are reserved in all cases.
+ * VA range, so pages required to map highest possible PA are reserved in all
+ * cases.
  */
 #if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
+#define IDMAP_PGTABLE_LEVELS	(ARM64_HW_PGTABLE_LEVELS(PHYS_MASK_SHIFT) - 1)
 #else
 #define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
+#define IDMAP_PGTABLE_LEVELS	(ARM64_HW_PGTABLE_LEVELS(PHYS_MASK_SHIFT))
 #endif
 
 #define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
-#define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
+#define IDMAP_DIR_SIZE		(IDMAP_PGTABLE_LEVELS * PAGE_SIZE)
 
 /* Initial memory map size */
 #if ARM64_SWAPPER_USES_SECTION_MAPS
-- 
1.7.9.5

[PATCH 04/14] arm64: Calculate size for idmap_pg_dir at compile time

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Now that we can calculate the number of levels required for
mapping a va width, reserve exact number of pages that would
be required to cover the idmap. The idmap should be able to handle
the maximum physical address size supported.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/boot.h           |    1 +
 arch/arm64/include/asm/kernel-pgtable.h |    7 +++++--
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/boot.h b/arch/arm64/include/asm/boot.h
index 81151b6..678b63e 100644
--- a/arch/arm64/include/asm/boot.h
+++ b/arch/arm64/include/asm/boot.h
@@ -2,6 +2,7 @@
 #ifndef __ASM_BOOT_H
 #define __ASM_BOOT_H
 
+#include <asm/page.h>
 #include <asm/sizes.h>
 
 /*
diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
index 5876a36..def7168 100644
--- a/arch/arm64/include/asm/kernel-pgtable.h
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -33,16 +33,19 @@
  * map to pte level. The swapper also maps the FDT (see __create_page_tables
  * for more information). Note that the number of ID map translation levels
  * could be increased on the fly if system RAM is out of reach for the default
- * VA range, so 3 pages are reserved in all cases.
+ * VA range, so pages required to map highest possible PA are reserved in all
+ * cases.
  */
 #if ARM64_SWAPPER_USES_SECTION_MAPS
 #define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - 1)
+#define IDMAP_PGTABLE_LEVELS	(ARM64_HW_PGTABLE_LEVELS(PHYS_MASK_SHIFT) - 1)
 #else
 #define SWAPPER_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS)
+#define IDMAP_PGTABLE_LEVELS	(ARM64_HW_PGTABLE_LEVELS(PHYS_MASK_SHIFT))
 #endif
 
 #define SWAPPER_DIR_SIZE	(SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
-#define IDMAP_DIR_SIZE		(3 * PAGE_SIZE)
+#define IDMAP_DIR_SIZE		(IDMAP_PGTABLE_LEVELS * PAGE_SIZE)
 
 /* Initial memory map size */
 #if ARM64_SWAPPER_USES_SECTION_MAPS
-- 
1.7.9.5

[PATCH 05/14] arm64: Handle 4 level page table for swapper

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

At the moment, we only support maximum of 3-level page table for
swapper. With 48bit VA, 64K has only 3 levels and 4K uses section
mapping. Add support for 4-level page table for swapper, needed
by 16K pages.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kernel/head.S |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 46670bf..01b8e58 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -271,7 +271,10 @@ ENDPROC(preserve_boot_args)
  */
 	.macro	create_pgd_entry, tbl, virt, tmp1, tmp2
 	create_table_entry \tbl, \virt, PGDIR_SHIFT, PTRS_PER_PGD, \tmp1, \tmp2
-#if SWAPPER_PGTABLE_LEVELS == 3
+#if SWAPPER_PGTABLE_LEVELS > 3
+	create_table_entry \tbl, \virt, PUD_SHIFT, PTRS_PER_PUD, \tmp1, \tmp2
+#endif
+#if SWAPPER_PGTABLE_LEVELS > 2
 	create_table_entry \tbl, \virt, SWAPPER_TABLE_SHIFT, PTRS_PER_PTE, \tmp1, \tmp2
 #endif
 	.endm
-- 
1.7.9.5

[PATCH 05/14] arm64: Handle 4 level page table for swapper

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

At the moment, we only support maximum of 3-level page table for
swapper. With 48bit VA, 64K has only 3 levels and 4K uses section
mapping. Add support for 4-level page table for swapper, needed
by 16K pages.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kernel/head.S |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 46670bf..01b8e58 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -271,7 +271,10 @@ ENDPROC(preserve_boot_args)
  */
 	.macro	create_pgd_entry, tbl, virt, tmp1, tmp2
 	create_table_entry \tbl, \virt, PGDIR_SHIFT, PTRS_PER_PGD, \tmp1, \tmp2
-#if SWAPPER_PGTABLE_LEVELS == 3
+#if SWAPPER_PGTABLE_LEVELS > 3
+	create_table_entry \tbl, \virt, PUD_SHIFT, PTRS_PER_PUD, \tmp1, \tmp2
+#endif
+#if SWAPPER_PGTABLE_LEVELS > 2
 	create_table_entry \tbl, \virt, SWAPPER_TABLE_SHIFT, PTRS_PER_PTE, \tmp1, \tmp2
 #endif
 	.endm
-- 
1.7.9.5

[PATCH 05/14] arm64: Handle 4 level page table for swapper

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

At the moment, we only support maximum of 3-level page table for
swapper. With 48bit VA, 64K has only 3 levels and 4K uses section
mapping. Add support for 4-level page table for swapper, needed
by 16K pages.

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/kernel/head.S |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 46670bf..01b8e58 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -271,7 +271,10 @@ ENDPROC(preserve_boot_args)
  */
 	.macro	create_pgd_entry, tbl, virt, tmp1, tmp2
 	create_table_entry \tbl, \virt, PGDIR_SHIFT, PTRS_PER_PGD, \tmp1, \tmp2
-#if SWAPPER_PGTABLE_LEVELS == 3
+#if SWAPPER_PGTABLE_LEVELS > 3
+	create_table_entry \tbl, \virt, PUD_SHIFT, PTRS_PER_PUD, \tmp1, \tmp2
+#endif
+#if SWAPPER_PGTABLE_LEVELS > 2
 	create_table_entry \tbl, \virt, SWAPPER_TABLE_SHIFT, PTRS_PER_PTE, \tmp1, \tmp2
 #endif
 	.endm
-- 
1.7.9.5

[PATCH 06/14] arm64: Clean config usages for page size

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

We use !CONFIG_ARM64_64K_PAGES for CONFIG_ARM64_4K_PAGES
(and vice versa) in code. It all worked well, so far since
we only had two options. Now, with the introduction of 16K,
these cases will break. This patch cleans up the code to
use the required CONFIG symbol expression without the assumption
that !64K => 4K (and vice versa)

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Steve Capper <steve.capper@linaro.org>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig                   |    4 ++--
 arch/arm64/Kconfig.debug             |    2 +-
 arch/arm64/include/asm/thread_info.h |    2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 78b89fa..d1fb2a3 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -550,7 +550,7 @@ config ARCH_WANT_GENERAL_HUGETLB
 	def_bool y
 
 config ARCH_WANT_HUGE_PMD_SHARE
-	def_bool y if !ARM64_64K_PAGES
+	def_bool y if ARM64_4K_PAGES
 
 config HAVE_ARCH_TRANSPARENT_HUGEPAGE
 	def_bool y
@@ -762,7 +762,7 @@ source "fs/Kconfig.binfmt"
 
 config COMPAT
 	bool "Kernel support for 32-bit EL0"
-	depends on !ARM64_64K_PAGES || EXPERT
+	depends on ARM64_4K_PAGES || EXPERT
 	select COMPAT_BINFMT_ELF
 	select HAVE_UID16
 	select OLD_SIGSUSPEND3
diff --git a/arch/arm64/Kconfig.debug b/arch/arm64/Kconfig.debug
index d6285ef..c24d6ad 100644
--- a/arch/arm64/Kconfig.debug
+++ b/arch/arm64/Kconfig.debug
@@ -77,7 +77,7 @@ config DEBUG_RODATA
           If in doubt, say Y
 
 config DEBUG_ALIGN_RODATA
-	depends on DEBUG_RODATA && !ARM64_64K_PAGES
+	depends on DEBUG_RODATA && ARM64_4K_PAGES
 	bool "Align linker sections up to SECTION_SIZE"
 	help
 	  If this option is enabled, sections that may potentially be marked as
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index dcd06d1..d9c8c9f 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -23,7 +23,7 @@
 
 #include <linux/compiler.h>
 
-#ifndef CONFIG_ARM64_64K_PAGES
+#ifdef CONFIG_ARM64_4K_PAGES
 #define THREAD_SIZE_ORDER	2
 #endif
 
-- 
1.7.9.5

[PATCH 06/14] arm64: Clean config usages for page size

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

We use !CONFIG_ARM64_64K_PAGES for CONFIG_ARM64_4K_PAGES
(and vice versa) in code. It all worked well, so far since
we only had two options. Now, with the introduction of 16K,
these cases will break. This patch cleans up the code to
use the required CONFIG symbol expression without the assumption
that !64K => 4K (and vice versa)

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Steve Capper <steve.capper@linaro.org>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig                   |    4 ++--
 arch/arm64/Kconfig.debug             |    2 +-
 arch/arm64/include/asm/thread_info.h |    2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 78b89fa..d1fb2a3 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -550,7 +550,7 @@ config ARCH_WANT_GENERAL_HUGETLB
 	def_bool y
 
 config ARCH_WANT_HUGE_PMD_SHARE
-	def_bool y if !ARM64_64K_PAGES
+	def_bool y if ARM64_4K_PAGES
 
 config HAVE_ARCH_TRANSPARENT_HUGEPAGE
 	def_bool y
@@ -762,7 +762,7 @@ source "fs/Kconfig.binfmt"
 
 config COMPAT
 	bool "Kernel support for 32-bit EL0"
-	depends on !ARM64_64K_PAGES || EXPERT
+	depends on ARM64_4K_PAGES || EXPERT
 	select COMPAT_BINFMT_ELF
 	select HAVE_UID16
 	select OLD_SIGSUSPEND3
diff --git a/arch/arm64/Kconfig.debug b/arch/arm64/Kconfig.debug
index d6285ef..c24d6ad 100644
--- a/arch/arm64/Kconfig.debug
+++ b/arch/arm64/Kconfig.debug
@@ -77,7 +77,7 @@ config DEBUG_RODATA
           If in doubt, say Y
 
 config DEBUG_ALIGN_RODATA
-	depends on DEBUG_RODATA && !ARM64_64K_PAGES
+	depends on DEBUG_RODATA && ARM64_4K_PAGES
 	bool "Align linker sections up to SECTION_SIZE"
 	help
 	  If this option is enabled, sections that may potentially be marked as
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index dcd06d1..d9c8c9f 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -23,7 +23,7 @@
 
 #include <linux/compiler.h>
 
-#ifndef CONFIG_ARM64_64K_PAGES
+#ifdef CONFIG_ARM64_4K_PAGES
 #define THREAD_SIZE_ORDER	2
 #endif
 
-- 
1.7.9.5

[PATCH 06/14] arm64: Clean config usages for page size

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

We use !CONFIG_ARM64_64K_PAGES for CONFIG_ARM64_4K_PAGES
(and vice versa) in code. It all worked well, so far since
we only had two options. Now, with the introduction of 16K,
these cases will break. This patch cleans up the code to
use the required CONFIG symbol expression without the assumption
that !64K => 4K (and vice versa)

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Steve Capper <steve.capper@linaro.org>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig                   |    4 ++--
 arch/arm64/Kconfig.debug             |    2 +-
 arch/arm64/include/asm/thread_info.h |    2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 78b89fa..d1fb2a3 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -550,7 +550,7 @@ config ARCH_WANT_GENERAL_HUGETLB
 	def_bool y
 
 config ARCH_WANT_HUGE_PMD_SHARE
-	def_bool y if !ARM64_64K_PAGES
+	def_bool y if ARM64_4K_PAGES
 
 config HAVE_ARCH_TRANSPARENT_HUGEPAGE
 	def_bool y
@@ -762,7 +762,7 @@ source "fs/Kconfig.binfmt"
 
 config COMPAT
 	bool "Kernel support for 32-bit EL0"
-	depends on !ARM64_64K_PAGES || EXPERT
+	depends on ARM64_4K_PAGES || EXPERT
 	select COMPAT_BINFMT_ELF
 	select HAVE_UID16
 	select OLD_SIGSUSPEND3
diff --git a/arch/arm64/Kconfig.debug b/arch/arm64/Kconfig.debug
index d6285ef..c24d6ad 100644
--- a/arch/arm64/Kconfig.debug
+++ b/arch/arm64/Kconfig.debug
@@ -77,7 +77,7 @@ config DEBUG_RODATA
           If in doubt, say Y
 
 config DEBUG_ALIGN_RODATA
-	depends on DEBUG_RODATA && !ARM64_64K_PAGES
+	depends on DEBUG_RODATA && ARM64_4K_PAGES
 	bool "Align linker sections up to SECTION_SIZE"
 	help
 	  If this option is enabled, sections that may potentially be marked as
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index dcd06d1..d9c8c9f 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -23,7 +23,7 @@
 
 #include <linux/compiler.h>
 
-#ifndef CONFIG_ARM64_64K_PAGES
+#ifdef CONFIG_ARM64_4K_PAGES
 #define THREAD_SIZE_ORDER	2
 #endif
 
-- 
1.7.9.5

[PATCH 07/14] arm64: Kconfig: Fix help text about AArch32 support with 64K pages

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Update the help text for ARM64_64K_PAGES to reflect the reality
about AArch32 support.

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index d1fb2a3..b247897 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -449,8 +449,8 @@ config ARM64_64K_PAGES
 	help
 	  This feature enables 64KB pages support (4KB by default)
 	  allowing only two levels of page tables and faster TLB
-	  look-up. AArch32 emulation is not available when this feature
-	  is enabled.
+	  look-up. AArch32 emulation requires applications compiled
+	  with 64K aligned segments.
 
 endchoice
 
-- 
1.7.9.5

[PATCH 07/14] arm64: Kconfig: Fix help text about AArch32 support with 64K pages

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Update the help text for ARM64_64K_PAGES to reflect the reality
about AArch32 support.

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index d1fb2a3..b247897 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -449,8 +449,8 @@ config ARM64_64K_PAGES
 	help
 	  This feature enables 64KB pages support (4KB by default)
 	  allowing only two levels of page tables and faster TLB
-	  look-up. AArch32 emulation is not available when this feature
-	  is enabled.
+	  look-up. AArch32 emulation requires applications compiled
+	  with 64K aligned segments.
 
 endchoice
 
-- 
1.7.9.5

[PATCH 07/14] arm64: Kconfig: Fix help text about AArch32 support with 64K pages

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Update the help text for ARM64_64K_PAGES to reflect the reality
about AArch32 support.

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index d1fb2a3..b247897 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -449,8 +449,8 @@ config ARM64_64K_PAGES
 	help
 	  This feature enables 64KB pages support (4KB by default)
 	  allowing only two levels of page tables and faster TLB
-	  look-up. AArch32 emulation is not available when this feature
-	  is enabled.
+	  look-up. AArch32 emulation requires applications compiled
+	  with 64K aligned segments.
 
 endchoice
 
-- 
1.7.9.5

[PATCH 08/14] arm64: kvm: Fix {V}TCR_EL2_TG0 mask

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

{V}TCR_EL2_TG0 is a 2bit wide field, where:

 00 - 4K
 01 - 64K
 10 - 16K

But we use only 1 bit, which has worked well so far since
we never cared about 16K. Fix it for 16K support.

Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: kvmarm@lists.cs.columbia.edu
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_arm.h |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index ac6fafb..52dc9cc 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -98,7 +98,7 @@
 #define TCR_EL2_TBI	(1 << 20)
 #define TCR_EL2_PS	(7 << 16)
 #define TCR_EL2_PS_40B	(2 << 16)
-#define TCR_EL2_TG0	(1 << 14)
+#define TCR_EL2_TG0	(3 << 14)
 #define TCR_EL2_SH0	(3 << 12)
 #define TCR_EL2_ORGN0	(3 << 10)
 #define TCR_EL2_IRGN0	(3 << 8)
@@ -110,7 +110,7 @@
 
 /* VTCR_EL2 Registers bits */
 #define VTCR_EL2_PS_MASK	(7 << 16)
-#define VTCR_EL2_TG0_MASK	(1 << 14)
+#define VTCR_EL2_TG0_MASK	(3 << 14)
 #define VTCR_EL2_TG0_4K		(0 << 14)
 #define VTCR_EL2_TG0_64K	(1 << 14)
 #define VTCR_EL2_SH0_MASK	(3 << 12)
-- 
1.7.9.5

[PATCH 08/14] arm64: kvm: Fix {V}TCR_EL2_TG0 mask

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

{V}TCR_EL2_TG0 is a 2bit wide field, where:

 00 - 4K
 01 - 64K
 10 - 16K

But we use only 1 bit, which has worked well so far since
we never cared about 16K. Fix it for 16K support.

Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: kvmarm@lists.cs.columbia.edu
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_arm.h |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index ac6fafb..52dc9cc 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -98,7 +98,7 @@
 #define TCR_EL2_TBI	(1 << 20)
 #define TCR_EL2_PS	(7 << 16)
 #define TCR_EL2_PS_40B	(2 << 16)
-#define TCR_EL2_TG0	(1 << 14)
+#define TCR_EL2_TG0	(3 << 14)
 #define TCR_EL2_SH0	(3 << 12)
 #define TCR_EL2_ORGN0	(3 << 10)
 #define TCR_EL2_IRGN0	(3 << 8)
@@ -110,7 +110,7 @@
 
 /* VTCR_EL2 Registers bits */
 #define VTCR_EL2_PS_MASK	(7 << 16)
-#define VTCR_EL2_TG0_MASK	(1 << 14)
+#define VTCR_EL2_TG0_MASK	(3 << 14)
 #define VTCR_EL2_TG0_4K		(0 << 14)
 #define VTCR_EL2_TG0_64K	(1 << 14)
 #define VTCR_EL2_SH0_MASK	(3 << 12)
-- 
1.7.9.5

[PATCH 08/14] arm64: kvm: Fix {V}TCR_EL2_TG0 mask

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

{V}TCR_EL2_TG0 is a 2bit wide field, where:

 00 - 4K
 01 - 64K
 10 - 16K

But we use only 1 bit, which has worked well so far since
we never cared about 16K. Fix it for 16K support.

Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: kvmarm at lists.cs.columbia.edu
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_arm.h |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index ac6fafb..52dc9cc 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -98,7 +98,7 @@
 #define TCR_EL2_TBI	(1 << 20)
 #define TCR_EL2_PS	(7 << 16)
 #define TCR_EL2_PS_40B	(2 << 16)
-#define TCR_EL2_TG0	(1 << 14)
+#define TCR_EL2_TG0	(3 << 14)
 #define TCR_EL2_SH0	(3 << 12)
 #define TCR_EL2_ORGN0	(3 << 10)
 #define TCR_EL2_IRGN0	(3 << 8)
@@ -110,7 +110,7 @@
 
 /* VTCR_EL2 Registers bits */
 #define VTCR_EL2_PS_MASK	(7 << 16)
-#define VTCR_EL2_TG0_MASK	(1 << 14)
+#define VTCR_EL2_TG0_MASK	(3 << 14)
 #define VTCR_EL2_TG0_4K		(0 << 14)
 #define VTCR_EL2_TG0_64K	(1 << 14)
 #define VTCR_EL2_SH0_MASK	(3 << 12)
-- 
1.7.9.5

[PATCH 09/14] arm64: Cleanup VTCR_EL2 computation

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

No functional changes. Group the common bits for VCTR_EL2
initialisation for better readability. The granule size
and the entry level are controlled by the page size.

Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: kvmarm@lists.cs.columbia.edu
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_arm.h |   13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 52dc9cc..dcaf799 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -138,6 +138,9 @@
  * The magic numbers used for VTTBR_X in this patch can be found in Tables
  * D4-23 and D4-25 in ARM DDI 0487A.b.
  */
+#define VTCR_EL2_COMMON_BITS	(VTCR_EL2_SH0_INNER | VTCR_EL2_ORGN0_WBWA | \
+				 VTCR_EL2_IRGN0_WBWA | VTCR_EL2_T0SZ_40B)
+
 #ifdef CONFIG_ARM64_64K_PAGES
 /*
  * Stage2 translation configuration:
@@ -145,9 +148,8 @@
  * 64kB pages (TG0 = 1)
  * 2 level page tables (SL = 1)
  */
-#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_64K | VTCR_EL2_SH0_INNER | \
-				 VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \
-				 VTCR_EL2_SL0_LVL1 | VTCR_EL2_T0SZ_40B)
+#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_64K | VTCR_EL2_SL0_LVL1 | \
+				 VTCR_EL2_COMMON_BITS)
 #define VTTBR_X		(38 - VTCR_EL2_T0SZ_40B)
 #else
 /*
@@ -156,9 +158,8 @@
  * 4kB pages (TG0 = 0)
  * 3 level page tables (SL = 1)
  */
-#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_4K | VTCR_EL2_SH0_INNER | \
-				 VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \
-				 VTCR_EL2_SL0_LVL1 | VTCR_EL2_T0SZ_40B)
+#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_4K | VTCR_EL2_SL0_LVL1 | \
+				 VTCR_EL2_COMMON_BITS)
 #define VTTBR_X		(37 - VTCR_EL2_T0SZ_40B)
 #endif
 
-- 
1.7.9.5

[PATCH 09/14] arm64: Cleanup VTCR_EL2 computation

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

No functional changes. Group the common bits for VCTR_EL2
initialisation for better readability. The granule size
and the entry level are controlled by the page size.

Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: kvmarm@lists.cs.columbia.edu
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_arm.h |   13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 52dc9cc..dcaf799 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -138,6 +138,9 @@
  * The magic numbers used for VTTBR_X in this patch can be found in Tables
  * D4-23 and D4-25 in ARM DDI 0487A.b.
  */
+#define VTCR_EL2_COMMON_BITS	(VTCR_EL2_SH0_INNER | VTCR_EL2_ORGN0_WBWA | \
+				 VTCR_EL2_IRGN0_WBWA | VTCR_EL2_T0SZ_40B)
+
 #ifdef CONFIG_ARM64_64K_PAGES
 /*
  * Stage2 translation configuration:
@@ -145,9 +148,8 @@
  * 64kB pages (TG0 = 1)
  * 2 level page tables (SL = 1)
  */
-#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_64K | VTCR_EL2_SH0_INNER | \
-				 VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \
-				 VTCR_EL2_SL0_LVL1 | VTCR_EL2_T0SZ_40B)
+#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_64K | VTCR_EL2_SL0_LVL1 | \
+				 VTCR_EL2_COMMON_BITS)
 #define VTTBR_X		(38 - VTCR_EL2_T0SZ_40B)
 #else
 /*
@@ -156,9 +158,8 @@
  * 4kB pages (TG0 = 0)
  * 3 level page tables (SL = 1)
  */
-#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_4K | VTCR_EL2_SH0_INNER | \
-				 VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \
-				 VTCR_EL2_SL0_LVL1 | VTCR_EL2_T0SZ_40B)
+#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_4K | VTCR_EL2_SL0_LVL1 | \
+				 VTCR_EL2_COMMON_BITS)
 #define VTTBR_X		(37 - VTCR_EL2_T0SZ_40B)
 #endif
 
-- 
1.7.9.5

[PATCH 09/14] arm64: Cleanup VTCR_EL2 computation

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

No functional changes. Group the common bits for VCTR_EL2
initialisation for better readability. The granule size
and the entry level are controlled by the page size.

Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: kvmarm at lists.cs.columbia.edu
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_arm.h |   13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 52dc9cc..dcaf799 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -138,6 +138,9 @@
  * The magic numbers used for VTTBR_X in this patch can be found in Tables
  * D4-23 and D4-25 in ARM DDI 0487A.b.
  */
+#define VTCR_EL2_COMMON_BITS	(VTCR_EL2_SH0_INNER | VTCR_EL2_ORGN0_WBWA | \
+				 VTCR_EL2_IRGN0_WBWA | VTCR_EL2_T0SZ_40B)
+
 #ifdef CONFIG_ARM64_64K_PAGES
 /*
  * Stage2 translation configuration:
@@ -145,9 +148,8 @@
  * 64kB pages (TG0 = 1)
  * 2 level page tables (SL = 1)
  */
-#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_64K | VTCR_EL2_SH0_INNER | \
-				 VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \
-				 VTCR_EL2_SL0_LVL1 | VTCR_EL2_T0SZ_40B)
+#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_64K | VTCR_EL2_SL0_LVL1 | \
+				 VTCR_EL2_COMMON_BITS)
 #define VTTBR_X		(38 - VTCR_EL2_T0SZ_40B)
 #else
 /*
@@ -156,9 +158,8 @@
  * 4kB pages (TG0 = 0)
  * 3 level page tables (SL = 1)
  */
-#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_4K | VTCR_EL2_SH0_INNER | \
-				 VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \
-				 VTCR_EL2_SL0_LVL1 | VTCR_EL2_T0SZ_40B)
+#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_4K | VTCR_EL2_SL0_LVL1 | \
+				 VTCR_EL2_COMMON_BITS)
 #define VTTBR_X		(37 - VTCR_EL2_T0SZ_40B)
 #endif
 
-- 
1.7.9.5

[PATCH 10/14] arm: kvm: Move fake PGD handling to arch specific files

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Rearrange the code for fake pgd handling, which is applicable
to only ARM64. The intention is to keep the common code cleaner,
unaware of the underlying hacks.

Cc: kvmarm@lists.cs.columbia.edu
Cc: christoffer.dall@linaro.org
Cc: Marc.Zyngier@arm.com
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm/include/asm/kvm_mmu.h   |    7 ++++++
 arch/arm/kvm/mmu.c               |   44 +++++---------------------------------
 arch/arm64/include/asm/kvm_mmu.h |   43 +++++++++++++++++++++++++++++++++++++
 3 files changed, 55 insertions(+), 39 deletions(-)

diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 405aa18..1c9aa8a 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -173,6 +173,13 @@ static inline unsigned int kvm_get_hwpgd_size(void)
 	return PTRS_PER_S2_PGD * sizeof(pgd_t);
 }
 
+static inline pgd_t *kvm_setup_fake_pgd(pgd_t *pgd)
+{
+	return pgd;
+}
+
+static inline void kvm_free_fake_pgd(pgd_t *pgd) {}
+
 struct kvm;
 
 #define kvm_flush_dcache_to_poc(a,l)	__cpuc_flush_dcache_area((a), (l))
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 7b42012..b210622 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -677,43 +677,11 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
 	 * guest, we allocate a fake PGD and pre-populate it to point
 	 * to the next-level page table, which will be the real
 	 * initial page table pointed to by the VTTBR.
-	 *
-	 * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
-	 * the PMD and the kernel will use folded pud.
-	 * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
-	 * pages.
 	 */
-	if (KVM_PREALLOC_LEVEL > 0) {
-		int i;
-
-		/*
-		 * Allocate fake pgd for the page table manipulation macros to
-		 * work.  This is not used by the hardware and we have no
-		 * alignment requirement for this allocation.
-		 */
-		pgd = kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
-				GFP_KERNEL | __GFP_ZERO);
-
-		if (!pgd) {
-			kvm_free_hwpgd(hwpgd);
-			return -ENOMEM;
-		}
-
-		/* Plug the HW PGD into the fake one. */
-		for (i = 0; i < PTRS_PER_S2_PGD; i++) {
-			if (KVM_PREALLOC_LEVEL == 1)
-				pgd_populate(NULL, pgd + i,
-					     (pud_t *)hwpgd + i * PTRS_PER_PUD);
-			else if (KVM_PREALLOC_LEVEL == 2)
-				pud_populate(NULL, pud_offset(pgd, 0) + i,
-					     (pmd_t *)hwpgd + i * PTRS_PER_PMD);
-		}
-	} else {
-		/*
-		 * Allocate actual first-level Stage-2 page table used by the
-		 * hardware for Stage-2 page table walks.
-		 */
-		pgd = (pgd_t *)hwpgd;
+	pgd = kvm_setup_fake_pgd(hwpgd);
+	if (IS_ERR(pgd)) {
+		kvm_free_hwpgd(hwpgd);
+		return PTR_ERR(pgd);
 	}
 
 	kvm_clean_pgd(pgd);
@@ -820,9 +788,7 @@ void kvm_free_stage2_pgd(struct kvm *kvm)
 
 	unmap_stage2_range(kvm, 0, KVM_PHYS_SIZE);
 	kvm_free_hwpgd(kvm_get_hwpgd(kvm));
-	if (KVM_PREALLOC_LEVEL > 0)
-		kfree(kvm->arch.pgd);
-
+	kvm_free_fake_pgd(kvm->arch.pgd);
 	kvm->arch.pgd = NULL;
 }
 
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 6150567..2567fe8 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -198,6 +198,49 @@ static inline unsigned int kvm_get_hwpgd_size(void)
 	return PTRS_PER_S2_PGD * sizeof(pgd_t);
 }
 
+/*
+ * Allocate fake pgd for the page table manipulation macros to
+ * work.  This is not used by the hardware and we have no
+ * alignment requirement for this allocation.
+ */
+static inline pgd_t* kvm_setup_fake_pgd(pgd_t *hwpgd)
+{
+	int i;
+	pgd_t *pgd;
+
+	if (!KVM_PREALLOC_LEVEL)
+		return hwpgd;
+	/*
+	 * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
+	 * the PMD and the kernel will use folded pud.
+	 * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
+	 * pages.
+	 */
+	pgd = kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
+			GFP_KERNEL | __GFP_ZERO);
+
+	if (!pgd)
+		return ERR_PTR(-ENOMEM);
+
+	/* Plug the HW PGD into the fake one. */
+	for (i = 0; i < PTRS_PER_S2_PGD; i++) {
+		if (KVM_PREALLOC_LEVEL == 1)
+			pgd_populate(NULL, pgd + i,
+				     (pud_t *)hwpgd + i * PTRS_PER_PUD);
+		else if (KVM_PREALLOC_LEVEL == 2)
+			pud_populate(NULL, pud_offset(pgd, 0) + i,
+				     (pmd_t *)hwpgd + i * PTRS_PER_PMD);
+	}
+
+	return pgd;
+}
+
+static inline void kvm_free_fake_pgd(pgd_t *pgd)
+{
+	if (KVM_PREALLOC_LEVEL > 0)
+		kfree(pgd);
+}
+
 static inline bool kvm_page_empty(void *ptr)
 {
 	struct page *ptr_page = virt_to_page(ptr);
-- 
1.7.9.5

[PATCH 10/14] arm: kvm: Move fake PGD handling to arch specific files

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Rearrange the code for fake pgd handling, which is applicable
to only ARM64. The intention is to keep the common code cleaner,
unaware of the underlying hacks.

Cc: kvmarm@lists.cs.columbia.edu
Cc: christoffer.dall@linaro.org
Cc: Marc.Zyngier@arm.com
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm/include/asm/kvm_mmu.h   |    7 ++++++
 arch/arm/kvm/mmu.c               |   44 +++++---------------------------------
 arch/arm64/include/asm/kvm_mmu.h |   43 +++++++++++++++++++++++++++++++++++++
 3 files changed, 55 insertions(+), 39 deletions(-)

diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 405aa18..1c9aa8a 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -173,6 +173,13 @@ static inline unsigned int kvm_get_hwpgd_size(void)
 	return PTRS_PER_S2_PGD * sizeof(pgd_t);
 }
 
+static inline pgd_t *kvm_setup_fake_pgd(pgd_t *pgd)
+{
+	return pgd;
+}
+
+static inline void kvm_free_fake_pgd(pgd_t *pgd) {}
+
 struct kvm;
 
 #define kvm_flush_dcache_to_poc(a,l)	__cpuc_flush_dcache_area((a), (l))
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 7b42012..b210622 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -677,43 +677,11 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
 	 * guest, we allocate a fake PGD and pre-populate it to point
 	 * to the next-level page table, which will be the real
 	 * initial page table pointed to by the VTTBR.
-	 *
-	 * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
-	 * the PMD and the kernel will use folded pud.
-	 * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
-	 * pages.
 	 */
-	if (KVM_PREALLOC_LEVEL > 0) {
-		int i;
-
-		/*
-		 * Allocate fake pgd for the page table manipulation macros to
-		 * work.  This is not used by the hardware and we have no
-		 * alignment requirement for this allocation.
-		 */
-		pgd = kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
-				GFP_KERNEL | __GFP_ZERO);
-
-		if (!pgd) {
-			kvm_free_hwpgd(hwpgd);
-			return -ENOMEM;
-		}
-
-		/* Plug the HW PGD into the fake one. */
-		for (i = 0; i < PTRS_PER_S2_PGD; i++) {
-			if (KVM_PREALLOC_LEVEL == 1)
-				pgd_populate(NULL, pgd + i,
-					     (pud_t *)hwpgd + i * PTRS_PER_PUD);
-			else if (KVM_PREALLOC_LEVEL == 2)
-				pud_populate(NULL, pud_offset(pgd, 0) + i,
-					     (pmd_t *)hwpgd + i * PTRS_PER_PMD);
-		}
-	} else {
-		/*
-		 * Allocate actual first-level Stage-2 page table used by the
-		 * hardware for Stage-2 page table walks.
-		 */
-		pgd = (pgd_t *)hwpgd;
+	pgd = kvm_setup_fake_pgd(hwpgd);
+	if (IS_ERR(pgd)) {
+		kvm_free_hwpgd(hwpgd);
+		return PTR_ERR(pgd);
 	}
 
 	kvm_clean_pgd(pgd);
@@ -820,9 +788,7 @@ void kvm_free_stage2_pgd(struct kvm *kvm)
 
 	unmap_stage2_range(kvm, 0, KVM_PHYS_SIZE);
 	kvm_free_hwpgd(kvm_get_hwpgd(kvm));
-	if (KVM_PREALLOC_LEVEL > 0)
-		kfree(kvm->arch.pgd);
-
+	kvm_free_fake_pgd(kvm->arch.pgd);
 	kvm->arch.pgd = NULL;
 }
 
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 6150567..2567fe8 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -198,6 +198,49 @@ static inline unsigned int kvm_get_hwpgd_size(void)
 	return PTRS_PER_S2_PGD * sizeof(pgd_t);
 }
 
+/*
+ * Allocate fake pgd for the page table manipulation macros to
+ * work.  This is not used by the hardware and we have no
+ * alignment requirement for this allocation.
+ */
+static inline pgd_t* kvm_setup_fake_pgd(pgd_t *hwpgd)
+{
+	int i;
+	pgd_t *pgd;
+
+	if (!KVM_PREALLOC_LEVEL)
+		return hwpgd;
+	/*
+	 * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
+	 * the PMD and the kernel will use folded pud.
+	 * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
+	 * pages.
+	 */
+	pgd = kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
+			GFP_KERNEL | __GFP_ZERO);
+
+	if (!pgd)
+		return ERR_PTR(-ENOMEM);
+
+	/* Plug the HW PGD into the fake one. */
+	for (i = 0; i < PTRS_PER_S2_PGD; i++) {
+		if (KVM_PREALLOC_LEVEL == 1)
+			pgd_populate(NULL, pgd + i,
+				     (pud_t *)hwpgd + i * PTRS_PER_PUD);
+		else if (KVM_PREALLOC_LEVEL == 2)
+			pud_populate(NULL, pud_offset(pgd, 0) + i,
+				     (pmd_t *)hwpgd + i * PTRS_PER_PMD);
+	}
+
+	return pgd;
+}
+
+static inline void kvm_free_fake_pgd(pgd_t *pgd)
+{
+	if (KVM_PREALLOC_LEVEL > 0)
+		kfree(pgd);
+}
+
 static inline bool kvm_page_empty(void *ptr)
 {
 	struct page *ptr_page = virt_to_page(ptr);
-- 
1.7.9.5

[PATCH 10/14] arm: kvm: Move fake PGD handling to arch specific files

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Rearrange the code for fake pgd handling, which is applicable
to only ARM64. The intention is to keep the common code cleaner,
unaware of the underlying hacks.

Cc: kvmarm at lists.cs.columbia.edu
Cc: christoffer.dall at linaro.org
Cc: Marc.Zyngier at arm.com
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm/include/asm/kvm_mmu.h   |    7 ++++++
 arch/arm/kvm/mmu.c               |   44 +++++---------------------------------
 arch/arm64/include/asm/kvm_mmu.h |   43 +++++++++++++++++++++++++++++++++++++
 3 files changed, 55 insertions(+), 39 deletions(-)

diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 405aa18..1c9aa8a 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -173,6 +173,13 @@ static inline unsigned int kvm_get_hwpgd_size(void)
 	return PTRS_PER_S2_PGD * sizeof(pgd_t);
 }
 
+static inline pgd_t *kvm_setup_fake_pgd(pgd_t *pgd)
+{
+	return pgd;
+}
+
+static inline void kvm_free_fake_pgd(pgd_t *pgd) {}
+
 struct kvm;
 
 #define kvm_flush_dcache_to_poc(a,l)	__cpuc_flush_dcache_area((a), (l))
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 7b42012..b210622 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -677,43 +677,11 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
 	 * guest, we allocate a fake PGD and pre-populate it to point
 	 * to the next-level page table, which will be the real
 	 * initial page table pointed to by the VTTBR.
-	 *
-	 * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
-	 * the PMD and the kernel will use folded pud.
-	 * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
-	 * pages.
 	 */
-	if (KVM_PREALLOC_LEVEL > 0) {
-		int i;
-
-		/*
-		 * Allocate fake pgd for the page table manipulation macros to
-		 * work.  This is not used by the hardware and we have no
-		 * alignment requirement for this allocation.
-		 */
-		pgd = kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
-				GFP_KERNEL | __GFP_ZERO);
-
-		if (!pgd) {
-			kvm_free_hwpgd(hwpgd);
-			return -ENOMEM;
-		}
-
-		/* Plug the HW PGD into the fake one. */
-		for (i = 0; i < PTRS_PER_S2_PGD; i++) {
-			if (KVM_PREALLOC_LEVEL == 1)
-				pgd_populate(NULL, pgd + i,
-					     (pud_t *)hwpgd + i * PTRS_PER_PUD);
-			else if (KVM_PREALLOC_LEVEL == 2)
-				pud_populate(NULL, pud_offset(pgd, 0) + i,
-					     (pmd_t *)hwpgd + i * PTRS_PER_PMD);
-		}
-	} else {
-		/*
-		 * Allocate actual first-level Stage-2 page table used by the
-		 * hardware for Stage-2 page table walks.
-		 */
-		pgd = (pgd_t *)hwpgd;
+	pgd = kvm_setup_fake_pgd(hwpgd);
+	if (IS_ERR(pgd)) {
+		kvm_free_hwpgd(hwpgd);
+		return PTR_ERR(pgd);
 	}
 
 	kvm_clean_pgd(pgd);
@@ -820,9 +788,7 @@ void kvm_free_stage2_pgd(struct kvm *kvm)
 
 	unmap_stage2_range(kvm, 0, KVM_PHYS_SIZE);
 	kvm_free_hwpgd(kvm_get_hwpgd(kvm));
-	if (KVM_PREALLOC_LEVEL > 0)
-		kfree(kvm->arch.pgd);
-
+	kvm_free_fake_pgd(kvm->arch.pgd);
 	kvm->arch.pgd = NULL;
 }
 
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 6150567..2567fe8 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -198,6 +198,49 @@ static inline unsigned int kvm_get_hwpgd_size(void)
 	return PTRS_PER_S2_PGD * sizeof(pgd_t);
 }
 
+/*
+ * Allocate fake pgd for the page table manipulation macros to
+ * work.  This is not used by the hardware and we have no
+ * alignment requirement for this allocation.
+ */
+static inline pgd_t* kvm_setup_fake_pgd(pgd_t *hwpgd)
+{
+	int i;
+	pgd_t *pgd;
+
+	if (!KVM_PREALLOC_LEVEL)
+		return hwpgd;
+	/*
+	 * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
+	 * the PMD and the kernel will use folded pud.
+	 * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
+	 * pages.
+	 */
+	pgd = kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
+			GFP_KERNEL | __GFP_ZERO);
+
+	if (!pgd)
+		return ERR_PTR(-ENOMEM);
+
+	/* Plug the HW PGD into the fake one. */
+	for (i = 0; i < PTRS_PER_S2_PGD; i++) {
+		if (KVM_PREALLOC_LEVEL == 1)
+			pgd_populate(NULL, pgd + i,
+				     (pud_t *)hwpgd + i * PTRS_PER_PUD);
+		else if (KVM_PREALLOC_LEVEL == 2)
+			pud_populate(NULL, pud_offset(pgd, 0) + i,
+				     (pmd_t *)hwpgd + i * PTRS_PER_PMD);
+	}
+
+	return pgd;
+}
+
+static inline void kvm_free_fake_pgd(pgd_t *pgd)
+{
+	if (KVM_PREALLOC_LEVEL > 0)
+		kfree(pgd);
+}
+
 static inline bool kvm_page_empty(void *ptr)
 {
 	struct page *ptr_page = virt_to_page(ptr);
-- 
1.7.9.5

[PATCH 11/14] arm64: kvm: Rewrite fake pgd handling

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

The existing fake pgd handling code assumes that the stage-2 entry
level can only be one level down that of the host, which may not be
true always(e.g, with the introduction of 16k pagesize).

e.g.
With 16k page size and 48bit VA and 40bit IPA we have the following
split for page table levels:

level:  0       1         2         3
bits : [47] [46 - 36] [35 - 25] [24 - 14] [13 - 0]
         ^       ^     ^
         |       |     |
   host entry    |     x---- stage-2 entry
                 |
        IPA -----x

The stage-2 entry level is 2, due to the concatenation of 16tables
at level 2(mandated by the hardware). So, we need to fake two levels
to actually reach the hyp page table. This case cannot be handled
with the existing code, as, all we know about is KVM_PREALLOC_LEVEL
which kind of stands for two different pieces of information.

1) Whether we have fake page table entry levels.
2) The entry level of stage-2 translation.

We loose the information about the number of fake levels that
we may have to use. Also, KVM_PREALLOC_LEVEL computation itself
is wrong, as we assume the hw entry level is always 1 level down
from the host.

This patch introduces two seperate indicators :
1) Accurate entry level for stage-2 translation - HYP_PGTABLE_ENTRY_LEVEL -
   using the new helpers.
2) Number of levels of fake pagetable entries. (KVM_FAKE_PGTABLE_LEVELS)

The following conditions hold true for all cases(with 40bit IPA)
1) The stage-2 entry level <= 2
2) Number of fake page-table entries is in the inclusive range [0, 2].

Cc: kvmarm@lists.cs.columbia.edu
Cc: christoffer.dall@linaro.org
Cc: Marc.Zyngier@arm.com
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_mmu.h |  114 ++++++++++++++++++++------------------
 1 file changed, 61 insertions(+), 53 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 2567fe8..72cfd9e 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -41,18 +41,6 @@
  */
 #define TRAMPOLINE_VA		(HYP_PAGE_OFFSET_MASK & PAGE_MASK)
 
-/*
- * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation
- * levels in addition to the PGD and potentially the PUD which are
- * pre-allocated (we pre-allocate the fake PGD and the PUD when the Stage-2
- * tables use one level of tables less than the kernel.
- */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define KVM_MMU_CACHE_MIN_PAGES	1
-#else
-#define KVM_MMU_CACHE_MIN_PAGES	2
-#endif
-
 #ifdef __ASSEMBLY__
 
 /*
@@ -80,6 +68,26 @@
 #define KVM_PHYS_SIZE	(1UL << KVM_PHYS_SHIFT)
 #define KVM_PHYS_MASK	(KVM_PHYS_SIZE - 1UL)
 
+/*
+ * At stage-2 entry level, upto 16 tables can be concatenated and
+ * the hardware expects us to use concatenation, whenever possible.
+ * So, number of page table levels for KVM_PHYS_SHIFT is always
+ * the number of normal page table levels for (KVM_PHYS_SHIFT - 4).
+ */
+#define HYP_PGTABLE_LEVELS	ARM64_HW_PGTABLE_LEVELS(KVM_PHYS_SHIFT - 4)
+/* Number of bits normally addressed by HYP_PGTABLE_LEVELS */
+#define HYP_PGTABLE_SHIFT	ARM64_HW_PGTABLE_LEVEL_SHIFT(HYP_PGTABLE_LEVELS + 1)
+#define HYP_PGDIR_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(HYP_PGTABLE_LEVELS)
+#define HYP_PGTABLE_ENTRY_LEVEL	(4 - HYP_PGTABLE_LEVELS)
+
+/*
+ * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation
+ * levels in addition to the PGD and potentially the PUD which are
+ * pre-allocated (we pre-allocate the fake PGD and the PUD when the Stage-2
+ * tables use one level of tables less than the kernel.
+ */
+#define KVM_MMU_CACHE_MIN_PAGES	(HYP_PGTABLE_LEVELS - 1)
+
 int create_hyp_mappings(void *from, void *to);
 int create_hyp_io_mappings(void *from, void *to, phys_addr_t);
 void free_boot_hyp_pgd(void);
@@ -145,56 +153,41 @@ static inline bool kvm_s2pmd_readonly(pmd_t *pmd)
 #define kvm_pud_addr_end(addr, end)	pud_addr_end(addr, end)
 #define kvm_pmd_addr_end(addr, end)	pmd_addr_end(addr, end)
 
-/*
- * In the case where PGDIR_SHIFT is larger than KVM_PHYS_SHIFT, we can address
- * the entire IPA input range with a single pgd entry, and we would only need
- * one pgd entry.  Note that in this case, the pgd is actually not used by
- * the MMU for Stage-2 translations, but is merely a fake pgd used as a data
- * structure for the kernel pgtable macros to work.
- */
-#if PGDIR_SHIFT > KVM_PHYS_SHIFT
-#define PTRS_PER_S2_PGD_SHIFT	0
+/* Number of concatenated tables in stage-2 entry level */
+#if KVM_PHYS_SHIFT > HYP_PGTABLE_SHIFT
+#define S2_ENTRY_TABLES_SHIFT	(KVM_PHYS_SHIFT - HYP_PGTABLE_SHIFT)
 #else
-#define PTRS_PER_S2_PGD_SHIFT	(KVM_PHYS_SHIFT - PGDIR_SHIFT)
+#define S2_ENTRY_TABLES_SHIFT	0
 #endif
+#define S2_ENTRY_TABLES		(1 << (S2_ENTRY_TABLES_SHIFT))
+
+/* Number of page table levels we fake to reach the hw pgtable for hyp */
+#define KVM_FAKE_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - HYP_PGTABLE_LEVELS)
+
+#define PTRS_PER_S2_PGD_SHIFT	(KVM_PHYS_SHIFT - HYP_PGDIR_SHIFT)
 #define PTRS_PER_S2_PGD		(1 << PTRS_PER_S2_PGD_SHIFT)
 #define S2_PGD_ORDER		get_order(PTRS_PER_S2_PGD * sizeof(pgd_t))
 
 #define kvm_pgd_index(addr)	(((addr) >> PGDIR_SHIFT) & (PTRS_PER_S2_PGD - 1))
 
-/*
- * If we are concatenating first level stage-2 page tables, we would have less
- * than or equal to 16 pointers in the fake PGD, because that's what the
- * architecture allows.  In this case, (4 - CONFIG_PGTABLE_LEVELS)
- * represents the first level for the host, and we add 1 to go to the next
- * level (which uses contatenation) for the stage-2 tables.
- */
-#if PTRS_PER_S2_PGD <= 16
-#define KVM_PREALLOC_LEVEL	(4 - CONFIG_PGTABLE_LEVELS + 1)
-#else
-#define KVM_PREALLOC_LEVEL	(0)
-#endif
-
 static inline void *kvm_get_hwpgd(struct kvm *kvm)
 {
 	pgd_t *pgd = kvm->arch.pgd;
 	pud_t *pud;
 
-	if (KVM_PREALLOC_LEVEL == 0)
+	if (KVM_FAKE_PGTABLE_LEVELS == 0)
 		return pgd;
 
 	pud = pud_offset(pgd, 0);
-	if (KVM_PREALLOC_LEVEL == 1)
+	if (HYP_PGTABLE_ENTRY_LEVEL == 1)
 		return pud;
 
-	BUG_ON(KVM_PREALLOC_LEVEL != 2);
+	BUG_ON(HYP_PGTABLE_ENTRY_LEVEL != 2);
 	return pmd_offset(pud, 0);
 }
 
 static inline unsigned int kvm_get_hwpgd_size(void)
 {
-	if (KVM_PREALLOC_LEVEL > 0)
-		return PTRS_PER_S2_PGD * PAGE_SIZE;
 	return PTRS_PER_S2_PGD * sizeof(pgd_t);
 }
 
@@ -207,27 +200,38 @@ static inline pgd_t* kvm_setup_fake_pgd(pgd_t *hwpgd)
 {
 	int i;
 	pgd_t *pgd;
+	pud_t *pud;
 
-	if (!KVM_PREALLOC_LEVEL)
+	if (KVM_FAKE_PGTABLE_LEVELS == 0)
 		return hwpgd;
-	/*
-	 * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
-	 * the PMD and the kernel will use folded pud.
-	 * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
-	 * pages.
-	 */
+
 	pgd = kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
 			GFP_KERNEL | __GFP_ZERO);
 
 	if (!pgd)
 		return ERR_PTR(-ENOMEM);
+	/*
+	 * If the stage-2 entry is two level down from that of the host,
+	 * we are using a 4-level table on host (since HYP_PGTABLE_ENTRY_LEVEL
+	 * cannot be < 2. So, this implies we need to allocat a PUD table
+	 * to map the concatenated PMD tables.
+	 */
+	if (KVM_FAKE_PGTABLE_LEVELS == 2) {
+		pud = (pud_t *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, 0);
+		if (!pud) {
+			kfree(pgd);
+			return ERR_PTR(-ENOMEM);
+		}
+		/* plug the pud into the PGD */
+		pgd_populate(NULL, pgd, pud);
+	}
 
 	/* Plug the HW PGD into the fake one. */
-	for (i = 0; i < PTRS_PER_S2_PGD; i++) {
-		if (KVM_PREALLOC_LEVEL == 1)
+	for (i = 0; i < S2_ENTRY_TABLES; i++) {
+		if (HYP_PGTABLE_ENTRY_LEVEL == 1)
 			pgd_populate(NULL, pgd + i,
 				     (pud_t *)hwpgd + i * PTRS_PER_PUD);
-		else if (KVM_PREALLOC_LEVEL == 2)
+		else if (HYP_PGTABLE_ENTRY_LEVEL == 2)
 			pud_populate(NULL, pud_offset(pgd, 0) + i,
 				     (pmd_t *)hwpgd + i * PTRS_PER_PMD);
 	}
@@ -237,8 +241,12 @@ static inline pgd_t* kvm_setup_fake_pgd(pgd_t *hwpgd)
 
 static inline void kvm_free_fake_pgd(pgd_t *pgd)
 {
-	if (KVM_PREALLOC_LEVEL > 0)
+	if (KVM_FAKE_PGTABLE_LEVELS > 0) {
+		/* free the PUD table */
+		if (KVM_FAKE_PGTABLE_LEVELS == 2)
+			free_page((unsigned long)pud_offset(pgd, 0));
 		kfree(pgd);
+	}
 }
 
 static inline bool kvm_page_empty(void *ptr)
@@ -253,14 +261,14 @@ static inline bool kvm_page_empty(void *ptr)
 #define kvm_pmd_table_empty(kvm, pmdp) (0)
 #else
 #define kvm_pmd_table_empty(kvm, pmdp) \
-	(kvm_page_empty(pmdp) && (!(kvm) || KVM_PREALLOC_LEVEL < 2))
+	(kvm_page_empty(pmdp) && (!(kvm) || HYP_PGTABLE_ENTRY_LEVEL < 2))
 #endif
 
 #ifdef __PAGETABLE_PUD_FOLDED
 #define kvm_pud_table_empty(kvm, pudp) (0)
 #else
 #define kvm_pud_table_empty(kvm, pudp) \
-	(kvm_page_empty(pudp) && (!(kvm) || KVM_PREALLOC_LEVEL < 1))
+	(kvm_page_empty(pudp) && (!(kvm) || HYP_PGTABLE_ENTRY_LEVEL < 1))
 #endif
 
 
-- 
1.7.9.5

[PATCH 11/14] arm64: kvm: Rewrite fake pgd handling

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

The existing fake pgd handling code assumes that the stage-2 entry
level can only be one level down that of the host, which may not be
true always(e.g, with the introduction of 16k pagesize).

e.g.
With 16k page size and 48bit VA and 40bit IPA we have the following
split for page table levels:

level:  0       1         2         3
bits : [47] [46 - 36] [35 - 25] [24 - 14] [13 - 0]
         ^       ^     ^
         |       |     |
   host entry    |     x---- stage-2 entry
                 |
        IPA -----x

The stage-2 entry level is 2, due to the concatenation of 16tables
at level 2(mandated by the hardware). So, we need to fake two levels
to actually reach the hyp page table. This case cannot be handled
with the existing code, as, all we know about is KVM_PREALLOC_LEVEL
which kind of stands for two different pieces of information.

1) Whether we have fake page table entry levels.
2) The entry level of stage-2 translation.

We loose the information about the number of fake levels that
we may have to use. Also, KVM_PREALLOC_LEVEL computation itself
is wrong, as we assume the hw entry level is always 1 level down
from the host.

This patch introduces two seperate indicators :
1) Accurate entry level for stage-2 translation - HYP_PGTABLE_ENTRY_LEVEL -
   using the new helpers.
2) Number of levels of fake pagetable entries. (KVM_FAKE_PGTABLE_LEVELS)

The following conditions hold true for all cases(with 40bit IPA)
1) The stage-2 entry level <= 2
2) Number of fake page-table entries is in the inclusive range [0, 2].

Cc: kvmarm@lists.cs.columbia.edu
Cc: christoffer.dall@linaro.org
Cc: Marc.Zyngier@arm.com
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_mmu.h |  114 ++++++++++++++++++++------------------
 1 file changed, 61 insertions(+), 53 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 2567fe8..72cfd9e 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -41,18 +41,6 @@
  */
 #define TRAMPOLINE_VA		(HYP_PAGE_OFFSET_MASK & PAGE_MASK)
 
-/*
- * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation
- * levels in addition to the PGD and potentially the PUD which are
- * pre-allocated (we pre-allocate the fake PGD and the PUD when the Stage-2
- * tables use one level of tables less than the kernel.
- */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define KVM_MMU_CACHE_MIN_PAGES	1
-#else
-#define KVM_MMU_CACHE_MIN_PAGES	2
-#endif
-
 #ifdef __ASSEMBLY__
 
 /*
@@ -80,6 +68,26 @@
 #define KVM_PHYS_SIZE	(1UL << KVM_PHYS_SHIFT)
 #define KVM_PHYS_MASK	(KVM_PHYS_SIZE - 1UL)
 
+/*
+ * At stage-2 entry level, upto 16 tables can be concatenated and
+ * the hardware expects us to use concatenation, whenever possible.
+ * So, number of page table levels for KVM_PHYS_SHIFT is always
+ * the number of normal page table levels for (KVM_PHYS_SHIFT - 4).
+ */
+#define HYP_PGTABLE_LEVELS	ARM64_HW_PGTABLE_LEVELS(KVM_PHYS_SHIFT - 4)
+/* Number of bits normally addressed by HYP_PGTABLE_LEVELS */
+#define HYP_PGTABLE_SHIFT	ARM64_HW_PGTABLE_LEVEL_SHIFT(HYP_PGTABLE_LEVELS + 1)
+#define HYP_PGDIR_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(HYP_PGTABLE_LEVELS)
+#define HYP_PGTABLE_ENTRY_LEVEL	(4 - HYP_PGTABLE_LEVELS)
+
+/*
+ * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation
+ * levels in addition to the PGD and potentially the PUD which are
+ * pre-allocated (we pre-allocate the fake PGD and the PUD when the Stage-2
+ * tables use one level of tables less than the kernel.
+ */
+#define KVM_MMU_CACHE_MIN_PAGES	(HYP_PGTABLE_LEVELS - 1)
+
 int create_hyp_mappings(void *from, void *to);
 int create_hyp_io_mappings(void *from, void *to, phys_addr_t);
 void free_boot_hyp_pgd(void);
@@ -145,56 +153,41 @@ static inline bool kvm_s2pmd_readonly(pmd_t *pmd)
 #define kvm_pud_addr_end(addr, end)	pud_addr_end(addr, end)
 #define kvm_pmd_addr_end(addr, end)	pmd_addr_end(addr, end)
 
-/*
- * In the case where PGDIR_SHIFT is larger than KVM_PHYS_SHIFT, we can address
- * the entire IPA input range with a single pgd entry, and we would only need
- * one pgd entry.  Note that in this case, the pgd is actually not used by
- * the MMU for Stage-2 translations, but is merely a fake pgd used as a data
- * structure for the kernel pgtable macros to work.
- */
-#if PGDIR_SHIFT > KVM_PHYS_SHIFT
-#define PTRS_PER_S2_PGD_SHIFT	0
+/* Number of concatenated tables in stage-2 entry level */
+#if KVM_PHYS_SHIFT > HYP_PGTABLE_SHIFT
+#define S2_ENTRY_TABLES_SHIFT	(KVM_PHYS_SHIFT - HYP_PGTABLE_SHIFT)
 #else
-#define PTRS_PER_S2_PGD_SHIFT	(KVM_PHYS_SHIFT - PGDIR_SHIFT)
+#define S2_ENTRY_TABLES_SHIFT	0
 #endif
+#define S2_ENTRY_TABLES		(1 << (S2_ENTRY_TABLES_SHIFT))
+
+/* Number of page table levels we fake to reach the hw pgtable for hyp */
+#define KVM_FAKE_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - HYP_PGTABLE_LEVELS)
+
+#define PTRS_PER_S2_PGD_SHIFT	(KVM_PHYS_SHIFT - HYP_PGDIR_SHIFT)
 #define PTRS_PER_S2_PGD		(1 << PTRS_PER_S2_PGD_SHIFT)
 #define S2_PGD_ORDER		get_order(PTRS_PER_S2_PGD * sizeof(pgd_t))
 
 #define kvm_pgd_index(addr)	(((addr) >> PGDIR_SHIFT) & (PTRS_PER_S2_PGD - 1))
 
-/*
- * If we are concatenating first level stage-2 page tables, we would have less
- * than or equal to 16 pointers in the fake PGD, because that's what the
- * architecture allows.  In this case, (4 - CONFIG_PGTABLE_LEVELS)
- * represents the first level for the host, and we add 1 to go to the next
- * level (which uses contatenation) for the stage-2 tables.
- */
-#if PTRS_PER_S2_PGD <= 16
-#define KVM_PREALLOC_LEVEL	(4 - CONFIG_PGTABLE_LEVELS + 1)
-#else
-#define KVM_PREALLOC_LEVEL	(0)
-#endif
-
 static inline void *kvm_get_hwpgd(struct kvm *kvm)
 {
 	pgd_t *pgd = kvm->arch.pgd;
 	pud_t *pud;
 
-	if (KVM_PREALLOC_LEVEL == 0)
+	if (KVM_FAKE_PGTABLE_LEVELS == 0)
 		return pgd;
 
 	pud = pud_offset(pgd, 0);
-	if (KVM_PREALLOC_LEVEL == 1)
+	if (HYP_PGTABLE_ENTRY_LEVEL == 1)
 		return pud;
 
-	BUG_ON(KVM_PREALLOC_LEVEL != 2);
+	BUG_ON(HYP_PGTABLE_ENTRY_LEVEL != 2);
 	return pmd_offset(pud, 0);
 }
 
 static inline unsigned int kvm_get_hwpgd_size(void)
 {
-	if (KVM_PREALLOC_LEVEL > 0)
-		return PTRS_PER_S2_PGD * PAGE_SIZE;
 	return PTRS_PER_S2_PGD * sizeof(pgd_t);
 }
 
@@ -207,27 +200,38 @@ static inline pgd_t* kvm_setup_fake_pgd(pgd_t *hwpgd)
 {
 	int i;
 	pgd_t *pgd;
+	pud_t *pud;
 
-	if (!KVM_PREALLOC_LEVEL)
+	if (KVM_FAKE_PGTABLE_LEVELS == 0)
 		return hwpgd;
-	/*
-	 * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
-	 * the PMD and the kernel will use folded pud.
-	 * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
-	 * pages.
-	 */
+
 	pgd = kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
 			GFP_KERNEL | __GFP_ZERO);
 
 	if (!pgd)
 		return ERR_PTR(-ENOMEM);
+	/*
+	 * If the stage-2 entry is two level down from that of the host,
+	 * we are using a 4-level table on host (since HYP_PGTABLE_ENTRY_LEVEL
+	 * cannot be < 2. So, this implies we need to allocat a PUD table
+	 * to map the concatenated PMD tables.
+	 */
+	if (KVM_FAKE_PGTABLE_LEVELS == 2) {
+		pud = (pud_t *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, 0);
+		if (!pud) {
+			kfree(pgd);
+			return ERR_PTR(-ENOMEM);
+		}
+		/* plug the pud into the PGD */
+		pgd_populate(NULL, pgd, pud);
+	}
 
 	/* Plug the HW PGD into the fake one. */
-	for (i = 0; i < PTRS_PER_S2_PGD; i++) {
-		if (KVM_PREALLOC_LEVEL == 1)
+	for (i = 0; i < S2_ENTRY_TABLES; i++) {
+		if (HYP_PGTABLE_ENTRY_LEVEL == 1)
 			pgd_populate(NULL, pgd + i,
 				     (pud_t *)hwpgd + i * PTRS_PER_PUD);
-		else if (KVM_PREALLOC_LEVEL == 2)
+		else if (HYP_PGTABLE_ENTRY_LEVEL == 2)
 			pud_populate(NULL, pud_offset(pgd, 0) + i,
 				     (pmd_t *)hwpgd + i * PTRS_PER_PMD);
 	}
@@ -237,8 +241,12 @@ static inline pgd_t* kvm_setup_fake_pgd(pgd_t *hwpgd)
 
 static inline void kvm_free_fake_pgd(pgd_t *pgd)
 {
-	if (KVM_PREALLOC_LEVEL > 0)
+	if (KVM_FAKE_PGTABLE_LEVELS > 0) {
+		/* free the PUD table */
+		if (KVM_FAKE_PGTABLE_LEVELS == 2)
+			free_page((unsigned long)pud_offset(pgd, 0));
 		kfree(pgd);
+	}
 }
 
 static inline bool kvm_page_empty(void *ptr)
@@ -253,14 +261,14 @@ static inline bool kvm_page_empty(void *ptr)
 #define kvm_pmd_table_empty(kvm, pmdp) (0)
 #else
 #define kvm_pmd_table_empty(kvm, pmdp) \
-	(kvm_page_empty(pmdp) && (!(kvm) || KVM_PREALLOC_LEVEL < 2))
+	(kvm_page_empty(pmdp) && (!(kvm) || HYP_PGTABLE_ENTRY_LEVEL < 2))
 #endif
 
 #ifdef __PAGETABLE_PUD_FOLDED
 #define kvm_pud_table_empty(kvm, pudp) (0)
 #else
 #define kvm_pud_table_empty(kvm, pudp) \
-	(kvm_page_empty(pudp) && (!(kvm) || KVM_PREALLOC_LEVEL < 1))
+	(kvm_page_empty(pudp) && (!(kvm) || HYP_PGTABLE_ENTRY_LEVEL < 1))
 #endif
 
 
-- 
1.7.9.5

[PATCH 11/14] arm64: kvm: Rewrite fake pgd handling

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

The existing fake pgd handling code assumes that the stage-2 entry
level can only be one level down that of the host, which may not be
true always(e.g, with the introduction of 16k pagesize).

e.g.
With 16k page size and 48bit VA and 40bit IPA we have the following
split for page table levels:

level:  0       1         2         3
bits : [47] [46 - 36] [35 - 25] [24 - 14] [13 - 0]
         ^       ^     ^
         |       |     |
   host entry    |     x---- stage-2 entry
                 |
        IPA -----x

The stage-2 entry level is 2, due to the concatenation of 16tables
at level 2(mandated by the hardware). So, we need to fake two levels
to actually reach the hyp page table. This case cannot be handled
with the existing code, as, all we know about is KVM_PREALLOC_LEVEL
which kind of stands for two different pieces of information.

1) Whether we have fake page table entry levels.
2) The entry level of stage-2 translation.

We loose the information about the number of fake levels that
we may have to use. Also, KVM_PREALLOC_LEVEL computation itself
is wrong, as we assume the hw entry level is always 1 level down
from the host.

This patch introduces two seperate indicators :
1) Accurate entry level for stage-2 translation - HYP_PGTABLE_ENTRY_LEVEL -
   using the new helpers.
2) Number of levels of fake pagetable entries. (KVM_FAKE_PGTABLE_LEVELS)

The following conditions hold true for all cases(with 40bit IPA)
1) The stage-2 entry level <= 2
2) Number of fake page-table entries is in the inclusive range [0, 2].

Cc: kvmarm at lists.cs.columbia.edu
Cc: christoffer.dall at linaro.org
Cc: Marc.Zyngier at arm.com
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/kvm_mmu.h |  114 ++++++++++++++++++++------------------
 1 file changed, 61 insertions(+), 53 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 2567fe8..72cfd9e 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -41,18 +41,6 @@
  */
 #define TRAMPOLINE_VA		(HYP_PAGE_OFFSET_MASK & PAGE_MASK)
 
-/*
- * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation
- * levels in addition to the PGD and potentially the PUD which are
- * pre-allocated (we pre-allocate the fake PGD and the PUD when the Stage-2
- * tables use one level of tables less than the kernel.
- */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define KVM_MMU_CACHE_MIN_PAGES	1
-#else
-#define KVM_MMU_CACHE_MIN_PAGES	2
-#endif
-
 #ifdef __ASSEMBLY__
 
 /*
@@ -80,6 +68,26 @@
 #define KVM_PHYS_SIZE	(1UL << KVM_PHYS_SHIFT)
 #define KVM_PHYS_MASK	(KVM_PHYS_SIZE - 1UL)
 
+/*
+ * At stage-2 entry level, upto 16 tables can be concatenated and
+ * the hardware expects us to use concatenation, whenever possible.
+ * So, number of page table levels for KVM_PHYS_SHIFT is always
+ * the number of normal page table levels for (KVM_PHYS_SHIFT - 4).
+ */
+#define HYP_PGTABLE_LEVELS	ARM64_HW_PGTABLE_LEVELS(KVM_PHYS_SHIFT - 4)
+/* Number of bits normally addressed by HYP_PGTABLE_LEVELS */
+#define HYP_PGTABLE_SHIFT	ARM64_HW_PGTABLE_LEVEL_SHIFT(HYP_PGTABLE_LEVELS + 1)
+#define HYP_PGDIR_SHIFT		ARM64_HW_PGTABLE_LEVEL_SHIFT(HYP_PGTABLE_LEVELS)
+#define HYP_PGTABLE_ENTRY_LEVEL	(4 - HYP_PGTABLE_LEVELS)
+
+/*
+ * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation
+ * levels in addition to the PGD and potentially the PUD which are
+ * pre-allocated (we pre-allocate the fake PGD and the PUD when the Stage-2
+ * tables use one level of tables less than the kernel.
+ */
+#define KVM_MMU_CACHE_MIN_PAGES	(HYP_PGTABLE_LEVELS - 1)
+
 int create_hyp_mappings(void *from, void *to);
 int create_hyp_io_mappings(void *from, void *to, phys_addr_t);
 void free_boot_hyp_pgd(void);
@@ -145,56 +153,41 @@ static inline bool kvm_s2pmd_readonly(pmd_t *pmd)
 #define kvm_pud_addr_end(addr, end)	pud_addr_end(addr, end)
 #define kvm_pmd_addr_end(addr, end)	pmd_addr_end(addr, end)
 
-/*
- * In the case where PGDIR_SHIFT is larger than KVM_PHYS_SHIFT, we can address
- * the entire IPA input range with a single pgd entry, and we would only need
- * one pgd entry.  Note that in this case, the pgd is actually not used by
- * the MMU for Stage-2 translations, but is merely a fake pgd used as a data
- * structure for the kernel pgtable macros to work.
- */
-#if PGDIR_SHIFT > KVM_PHYS_SHIFT
-#define PTRS_PER_S2_PGD_SHIFT	0
+/* Number of concatenated tables in stage-2 entry level */
+#if KVM_PHYS_SHIFT > HYP_PGTABLE_SHIFT
+#define S2_ENTRY_TABLES_SHIFT	(KVM_PHYS_SHIFT - HYP_PGTABLE_SHIFT)
 #else
-#define PTRS_PER_S2_PGD_SHIFT	(KVM_PHYS_SHIFT - PGDIR_SHIFT)
+#define S2_ENTRY_TABLES_SHIFT	0
 #endif
+#define S2_ENTRY_TABLES		(1 << (S2_ENTRY_TABLES_SHIFT))
+
+/* Number of page table levels we fake to reach the hw pgtable for hyp */
+#define KVM_FAKE_PGTABLE_LEVELS	(CONFIG_PGTABLE_LEVELS - HYP_PGTABLE_LEVELS)
+
+#define PTRS_PER_S2_PGD_SHIFT	(KVM_PHYS_SHIFT - HYP_PGDIR_SHIFT)
 #define PTRS_PER_S2_PGD		(1 << PTRS_PER_S2_PGD_SHIFT)
 #define S2_PGD_ORDER		get_order(PTRS_PER_S2_PGD * sizeof(pgd_t))
 
 #define kvm_pgd_index(addr)	(((addr) >> PGDIR_SHIFT) & (PTRS_PER_S2_PGD - 1))
 
-/*
- * If we are concatenating first level stage-2 page tables, we would have less
- * than or equal to 16 pointers in the fake PGD, because that's what the
- * architecture allows.  In this case, (4 - CONFIG_PGTABLE_LEVELS)
- * represents the first level for the host, and we add 1 to go to the next
- * level (which uses contatenation) for the stage-2 tables.
- */
-#if PTRS_PER_S2_PGD <= 16
-#define KVM_PREALLOC_LEVEL	(4 - CONFIG_PGTABLE_LEVELS + 1)
-#else
-#define KVM_PREALLOC_LEVEL	(0)
-#endif
-
 static inline void *kvm_get_hwpgd(struct kvm *kvm)
 {
 	pgd_t *pgd = kvm->arch.pgd;
 	pud_t *pud;
 
-	if (KVM_PREALLOC_LEVEL == 0)
+	if (KVM_FAKE_PGTABLE_LEVELS == 0)
 		return pgd;
 
 	pud = pud_offset(pgd, 0);
-	if (KVM_PREALLOC_LEVEL == 1)
+	if (HYP_PGTABLE_ENTRY_LEVEL == 1)
 		return pud;
 
-	BUG_ON(KVM_PREALLOC_LEVEL != 2);
+	BUG_ON(HYP_PGTABLE_ENTRY_LEVEL != 2);
 	return pmd_offset(pud, 0);
 }
 
 static inline unsigned int kvm_get_hwpgd_size(void)
 {
-	if (KVM_PREALLOC_LEVEL > 0)
-		return PTRS_PER_S2_PGD * PAGE_SIZE;
 	return PTRS_PER_S2_PGD * sizeof(pgd_t);
 }
 
@@ -207,27 +200,38 @@ static inline pgd_t* kvm_setup_fake_pgd(pgd_t *hwpgd)
 {
 	int i;
 	pgd_t *pgd;
+	pud_t *pud;
 
-	if (!KVM_PREALLOC_LEVEL)
+	if (KVM_FAKE_PGTABLE_LEVELS == 0)
 		return hwpgd;
-	/*
-	 * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
-	 * the PMD and the kernel will use folded pud.
-	 * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
-	 * pages.
-	 */
+
 	pgd = kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
 			GFP_KERNEL | __GFP_ZERO);
 
 	if (!pgd)
 		return ERR_PTR(-ENOMEM);
+	/*
+	 * If the stage-2 entry is two level down from that of the host,
+	 * we are using a 4-level table on host (since HYP_PGTABLE_ENTRY_LEVEL
+	 * cannot be < 2. So, this implies we need to allocat a PUD table
+	 * to map the concatenated PMD tables.
+	 */
+	if (KVM_FAKE_PGTABLE_LEVELS == 2) {
+		pud = (pud_t *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, 0);
+		if (!pud) {
+			kfree(pgd);
+			return ERR_PTR(-ENOMEM);
+		}
+		/* plug the pud into the PGD */
+		pgd_populate(NULL, pgd, pud);
+	}
 
 	/* Plug the HW PGD into the fake one. */
-	for (i = 0; i < PTRS_PER_S2_PGD; i++) {
-		if (KVM_PREALLOC_LEVEL == 1)
+	for (i = 0; i < S2_ENTRY_TABLES; i++) {
+		if (HYP_PGTABLE_ENTRY_LEVEL == 1)
 			pgd_populate(NULL, pgd + i,
 				     (pud_t *)hwpgd + i * PTRS_PER_PUD);
-		else if (KVM_PREALLOC_LEVEL == 2)
+		else if (HYP_PGTABLE_ENTRY_LEVEL == 2)
 			pud_populate(NULL, pud_offset(pgd, 0) + i,
 				     (pmd_t *)hwpgd + i * PTRS_PER_PMD);
 	}
@@ -237,8 +241,12 @@ static inline pgd_t* kvm_setup_fake_pgd(pgd_t *hwpgd)
 
 static inline void kvm_free_fake_pgd(pgd_t *pgd)
 {
-	if (KVM_PREALLOC_LEVEL > 0)
+	if (KVM_FAKE_PGTABLE_LEVELS > 0) {
+		/* free the PUD table */
+		if (KVM_FAKE_PGTABLE_LEVELS == 2)
+			free_page((unsigned long)pud_offset(pgd, 0));
 		kfree(pgd);
+	}
 }
 
 static inline bool kvm_page_empty(void *ptr)
@@ -253,14 +261,14 @@ static inline bool kvm_page_empty(void *ptr)
 #define kvm_pmd_table_empty(kvm, pmdp) (0)
 #else
 #define kvm_pmd_table_empty(kvm, pmdp) \
-	(kvm_page_empty(pmdp) && (!(kvm) || KVM_PREALLOC_LEVEL < 2))
+	(kvm_page_empty(pmdp) && (!(kvm) || HYP_PGTABLE_ENTRY_LEVEL < 2))
 #endif
 
 #ifdef __PAGETABLE_PUD_FOLDED
 #define kvm_pud_table_empty(kvm, pudp) (0)
 #else
 #define kvm_pud_table_empty(kvm, pudp) \
-	(kvm_page_empty(pudp) && (!(kvm) || KVM_PREALLOC_LEVEL < 1))
+	(kvm_page_empty(pudp) && (!(kvm) || HYP_PGTABLE_ENTRY_LEVEL < 1))
 #endif
 
 
-- 
1.7.9.5

[PATCH 12/14] arm64: Check for selected granule support

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Ensure that the selected page size is supported by the
CPU(s).

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/sysreg.h |    6 ++++++
 arch/arm64/kernel/head.S        |   24 +++++++++++++++++++++++-
 2 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index a7f3d4b..e01d323 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -87,4 +87,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
 }
 #endif
 
+#define ID_AA64MMFR0_TGran4_SHIFT	28
+#define ID_AA64MMFR0_TGran64_SHIFT	24
+
+#define ID_AA64MMFR0_TGran4_ENABLED	0x0
+#define ID_AA64MMFR0_TGran64_ENABLED	0x0
+
 #endif	/* __ASM_SYSREG_H */
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 01b8e58..0cb04db 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -31,10 +31,11 @@
 #include <asm/cputype.h>
 #include <asm/kernel-pgtable.h>
 #include <asm/memory.h>
-#include <asm/thread_info.h>
 #include <asm/pgtable-hwdef.h>
 #include <asm/pgtable.h>
 #include <asm/page.h>
+#include <asm/sysreg.h>
+#include <asm/thread_info.h>
 #include <asm/virt.h>
 
 #define __PHYS_OFFSET	(KERNEL_START - TEXT_OFFSET)
@@ -606,9 +607,25 @@ ENDPROC(__secondary_switched)
  *  x27 = *virtual* address to jump to upon completion
  *
  * other registers depend on the function called upon completion
+ * Checks if the selected granule size is supported by the CPU.
  */
+#if	defined(CONFIG_ARM64_64K_PAGES)
+
+#define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran64_SHIFT
+#define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran64_ENABLED
+
+#else
+
+#define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran4_SHIFT
+#define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran4_ENABLED
+
+#endif
 	.section	".idmap.text", "ax"
 __enable_mmu:
+	mrs	x1, ID_AA64MMFR0_EL1
+	ubfx	x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
+	cmp	x2, #ID_AA64MMFR0_TGran_ENABLED
+	b.ne	__no_granule_support
 	ldr	x5, =vectors
 	msr	vbar_el1, x5
 	msr	ttbr0_el1, x25			// load TTBR0
@@ -626,3 +643,8 @@ __enable_mmu:
 	isb
 	br	x27
 ENDPROC(__enable_mmu)
+
+__no_granule_support:
+	wfe
+	b __no_granule_support
+ENDPROC(__no_granule_support)
-- 
1.7.9.5

[PATCH 12/14] arm64: Check for selected granule support

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Ensure that the selected page size is supported by the
CPU(s).

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/sysreg.h |    6 ++++++
 arch/arm64/kernel/head.S        |   24 +++++++++++++++++++++++-
 2 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index a7f3d4b..e01d323 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -87,4 +87,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
 }
 #endif
 
+#define ID_AA64MMFR0_TGran4_SHIFT	28
+#define ID_AA64MMFR0_TGran64_SHIFT	24
+
+#define ID_AA64MMFR0_TGran4_ENABLED	0x0
+#define ID_AA64MMFR0_TGran64_ENABLED	0x0
+
 #endif	/* __ASM_SYSREG_H */
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 01b8e58..0cb04db 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -31,10 +31,11 @@
 #include <asm/cputype.h>
 #include <asm/kernel-pgtable.h>
 #include <asm/memory.h>
-#include <asm/thread_info.h>
 #include <asm/pgtable-hwdef.h>
 #include <asm/pgtable.h>
 #include <asm/page.h>
+#include <asm/sysreg.h>
+#include <asm/thread_info.h>
 #include <asm/virt.h>
 
 #define __PHYS_OFFSET	(KERNEL_START - TEXT_OFFSET)
@@ -606,9 +607,25 @@ ENDPROC(__secondary_switched)
  *  x27 = *virtual* address to jump to upon completion
  *
  * other registers depend on the function called upon completion
+ * Checks if the selected granule size is supported by the CPU.
  */
+#if	defined(CONFIG_ARM64_64K_PAGES)
+
+#define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran64_SHIFT
+#define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran64_ENABLED
+
+#else
+
+#define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran4_SHIFT
+#define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran4_ENABLED
+
+#endif
 	.section	".idmap.text", "ax"
 __enable_mmu:
+	mrs	x1, ID_AA64MMFR0_EL1
+	ubfx	x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
+	cmp	x2, #ID_AA64MMFR0_TGran_ENABLED
+	b.ne	__no_granule_support
 	ldr	x5, =vectors
 	msr	vbar_el1, x5
 	msr	ttbr0_el1, x25			// load TTBR0
@@ -626,3 +643,8 @@ __enable_mmu:
 	isb
 	br	x27
 ENDPROC(__enable_mmu)
+
+__no_granule_support:
+	wfe
+	b __no_granule_support
+ENDPROC(__no_granule_support)
-- 
1.7.9.5

[PATCH 12/14] arm64: Check for selected granule support

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

Ensure that the selected page size is supported by the
CPU(s).

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/include/asm/sysreg.h |    6 ++++++
 arch/arm64/kernel/head.S        |   24 +++++++++++++++++++++++-
 2 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index a7f3d4b..e01d323 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -87,4 +87,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
 }
 #endif
 
+#define ID_AA64MMFR0_TGran4_SHIFT	28
+#define ID_AA64MMFR0_TGran64_SHIFT	24
+
+#define ID_AA64MMFR0_TGran4_ENABLED	0x0
+#define ID_AA64MMFR0_TGran64_ENABLED	0x0
+
 #endif	/* __ASM_SYSREG_H */
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 01b8e58..0cb04db 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -31,10 +31,11 @@
 #include <asm/cputype.h>
 #include <asm/kernel-pgtable.h>
 #include <asm/memory.h>
-#include <asm/thread_info.h>
 #include <asm/pgtable-hwdef.h>
 #include <asm/pgtable.h>
 #include <asm/page.h>
+#include <asm/sysreg.h>
+#include <asm/thread_info.h>
 #include <asm/virt.h>
 
 #define __PHYS_OFFSET	(KERNEL_START - TEXT_OFFSET)
@@ -606,9 +607,25 @@ ENDPROC(__secondary_switched)
  *  x27 = *virtual* address to jump to upon completion
  *
  * other registers depend on the function called upon completion
+ * Checks if the selected granule size is supported by the CPU.
  */
+#if	defined(CONFIG_ARM64_64K_PAGES)
+
+#define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran64_SHIFT
+#define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran64_ENABLED
+
+#else
+
+#define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran4_SHIFT
+#define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran4_ENABLED
+
+#endif
 	.section	".idmap.text", "ax"
 __enable_mmu:
+	mrs	x1, ID_AA64MMFR0_EL1
+	ubfx	x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
+	cmp	x2, #ID_AA64MMFR0_TGran_ENABLED
+	b.ne	__no_granule_support
 	ldr	x5, =vectors
 	msr	vbar_el1, x5
 	msr	ttbr0_el1, x25			// load TTBR0
@@ -626,3 +643,8 @@ __enable_mmu:
 	isb
 	br	x27
 ENDPROC(__enable_mmu)
+
+__no_granule_support:
+	wfe
+	b __no_granule_support
+ENDPROC(__no_granule_support)
-- 
1.7.9.5

[PATCH 13/14] arm64: Add 16K page size support

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

This patch turns on the 16K page support in the kernel. We
support 48bit VA (4 level page tables) and 47bit VA (3 level
page tables).

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Steve Capper <steve.capper@linaro.org>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig                   |   25 ++++++++++++++++++++-----
 arch/arm64/include/asm/fixmap.h      |    4 +++-
 arch/arm64/include/asm/kvm_arm.h     |   12 ++++++++++++
 arch/arm64/include/asm/page.h        |    2 ++
 arch/arm64/include/asm/sysreg.h      |    2 ++
 arch/arm64/include/asm/thread_info.h |    2 ++
 arch/arm64/kernel/head.S             |    7 ++++++-
 arch/arm64/mm/proc.S                 |    4 +++-
 8 files changed, 50 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b247897..8327edf 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -167,7 +167,8 @@ config PGTABLE_LEVELS
 	default 2 if ARM64_64K_PAGES && ARM64_VA_BITS_42
 	default 3 if ARM64_64K_PAGES && ARM64_VA_BITS_48
 	default 3 if ARM64_4K_PAGES && ARM64_VA_BITS_39
-	default 4 if ARM64_4K_PAGES && ARM64_VA_BITS_48
+	default 3 if ARM64_16K_PAGES && ARM64_VA_BITS_47
+	default 4 if !ARM64_64K_PAGES && ARM64_VA_BITS_48
 
 source "init/Kconfig"
 
@@ -444,6 +445,13 @@ config ARM64_4K_PAGES
 	help
 	  This feature enables 4KB pages support.
 
+config ARM64_16K_PAGES
+	bool "16KB"
+	help
+	  The system will use 16KB pages support. AArch32 emulation
+	  requires applications compiled with 16K(or multiple of 16K)
+	  aligned segments.
+
 config ARM64_64K_PAGES
 	bool "64KB"
 	help
@@ -457,6 +465,7 @@ endchoice
 choice
 	prompt "Virtual address space size"
 	default ARM64_VA_BITS_39 if ARM64_4K_PAGES
+	default ARM64_VA_BITS_47 if ARM64_16K_PAGES
 	default ARM64_VA_BITS_42 if ARM64_64K_PAGES
 	help
 	  Allows choosing one of multiple possible virtual address
@@ -471,6 +480,10 @@ config ARM64_VA_BITS_42
 	bool "42-bit"
 	depends on ARM64_64K_PAGES
 
+config ARM64_VA_BITS_47
+	bool "47-bit"
+	depends on ARM64_16K_PAGES
+
 config ARM64_VA_BITS_48
 	bool "48-bit"
 
@@ -480,6 +493,7 @@ config ARM64_VA_BITS
 	int
 	default 39 if ARM64_VA_BITS_39
 	default 42 if ARM64_VA_BITS_42
+	default 47 if ARM64_VA_BITS_47
 	default 48 if ARM64_VA_BITS_48
 
 config CPU_BIG_ENDIAN
@@ -550,7 +564,7 @@ config ARCH_WANT_GENERAL_HUGETLB
 	def_bool y
 
 config ARCH_WANT_HUGE_PMD_SHARE
-	def_bool y if ARM64_4K_PAGES
+	def_bool y if ARM64_4K_PAGES || ARM64_16K_PAGES
 
 config HAVE_ARCH_TRANSPARENT_HUGEPAGE
 	def_bool y
@@ -587,6 +601,7 @@ config XEN
 config FORCE_MAX_ZONEORDER
 	int
 	default "14" if (ARM64_64K_PAGES && TRANSPARENT_HUGEPAGE)
+	default "12" if (ARM64_16K_PAGES && TRANSPARENT_HUGEPAGE)
 	default "11"
 
 menuconfig ARMV8_DEPRECATED
@@ -773,9 +788,9 @@ config COMPAT
 	  the user helper functions, VFP support and the ptrace interface are
 	  handled appropriately by the kernel.
 
-	  If you also enabled CONFIG_ARM64_64K_PAGES, please be aware that you
-	  will only be able to execute AArch32 binaries that were compiled with
-	  64k aligned segments.
+	  If you use a page size other than 4KB(i.e, 16KB or 64KB), please be aware
+	  that you will only be able to execute AArch32 binaries that were compiled
+	  with page size aligned segments.
 
 	  If you want to execute 32-bit userspace applications, say Y.
 
diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h
index c0739187..f44a390 100644
--- a/arch/arm64/include/asm/fixmap.h
+++ b/arch/arm64/include/asm/fixmap.h
@@ -55,8 +55,10 @@ enum fixed_addresses {
 	 * Temporary boot-time mappings, used by early_ioremap(),
 	 * before ioremap() is functional.
 	 */
-#ifdef CONFIG_ARM64_64K_PAGES
+#if	defined(CONFIG_ARM64_64K_PAGES)
 #define NR_FIX_BTMAPS		4
+#elif	defined (CONFIG_ARM64_16K_PAGES)
+#define NR_FIX_BTMAPS		16
 #else
 #define NR_FIX_BTMAPS		64
 #endif
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index dcaf799..4d6a022 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -113,6 +113,7 @@
 #define VTCR_EL2_TG0_MASK	(3 << 14)
 #define VTCR_EL2_TG0_4K		(0 << 14)
 #define VTCR_EL2_TG0_64K	(1 << 14)
+#define VTCR_EL2_TG0_16K	(2 << 14)
 #define VTCR_EL2_SH0_MASK	(3 << 12)
 #define VTCR_EL2_SH0_INNER	(3 << 12)
 #define VTCR_EL2_ORGN0_MASK	(3 << 10)
@@ -134,6 +135,8 @@
  *
  * Note that when using 4K pages, we concatenate two first level page tables
  * together.
+ * With 16K pages, we concatenate 16 first level page tables and enter at
+ * level 2.
  *
  * The magic numbers used for VTTBR_X in this patch can be found in Tables
  * D4-23 and D4-25 in ARM DDI 0487A.b.
@@ -151,6 +154,15 @@
 #define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_64K | VTCR_EL2_SL0_LVL1 | \
 				 VTCR_EL2_COMMON_BITS)
 #define VTTBR_X		(38 - VTCR_EL2_T0SZ_40B)
+#elif defined(CONFIG_ARM64_16K_PAGES)
+/*
+ * Stage2 translation configuration:
+ * 16kB pages (TG0 = 2)
+ * 2 level page tables (SL0 = 1)
+ */
+#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_16K | VTCR_EL2_SL0_LVL1 | \
+				 VTCR_EL2_COMMON_BITS)
+#define VTTBR_X			(42 - VTCR_EL2_T0SZ_40B)
 #else
 /*
  * Stage2 translation configuration:
diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h
index 3c9ce8c..6aee732 100644
--- a/arch/arm64/include/asm/page.h
+++ b/arch/arm64/include/asm/page.h
@@ -22,6 +22,8 @@
 /* PAGE_SHIFT determines the page size */
 #ifdef CONFIG_ARM64_64K_PAGES
 #define PAGE_SHIFT		16
+#elif defined(CONFIG_ARM64_16K_PAGES)
+#define PAGE_SHIFT		14
 #else
 #define PAGE_SHIFT		12
 #endif
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index e01d323..310592c 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -89,8 +89,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
 
 #define ID_AA64MMFR0_TGran4_SHIFT	28
 #define ID_AA64MMFR0_TGran64_SHIFT	24
+#define ID_AA64MMFR0_TGran16_SHIFT	20
 
 #define ID_AA64MMFR0_TGran4_ENABLED	0x0
 #define ID_AA64MMFR0_TGran64_ENABLED	0x0
+#define ID_AA64MMFR0_TGran16_ENABLED	0x1
 
 #endif	/* __ASM_SYSREG_H */
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index d9c8c9f..338a4ac 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -25,6 +25,8 @@
 
 #ifdef CONFIG_ARM64_4K_PAGES
 #define THREAD_SIZE_ORDER	2
+#elif defined(CONFIG_ARM64_16K_PAGES)
+#define THREAD_SIZE_ORDER	0
 #endif
 
 #define THREAD_SIZE		16384
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 0cb04db..959100b 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -614,7 +614,12 @@ ENDPROC(__secondary_switched)
 #define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran64_SHIFT
 #define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran64_ENABLED
 
-#else
+#elif	defined(CONFIG_ARM64_16K_PAGES)
+
+#define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran16_SHIFT
+#define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran16_ENABLED
+
+#elif	defined(CONFIG_ARM64_4K_PAGES)
 
 #define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran4_SHIFT
 #define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran4_ENABLED
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 6e8765a..182bced 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -30,7 +30,9 @@
 
 #ifdef CONFIG_ARM64_64K_PAGES
 #define TCR_TG_FLAGS	TCR_TG0_64K | TCR_TG1_64K
-#else
+#elif defined(CONFIG_ARM64_16K_PAGES)
+#define TCR_TG_FLAGS	TCR_TG0_16K | TCR_TG1_16K
+#else /* CONFIG_ARM64_4K_PAGES */
 #define TCR_TG_FLAGS	TCR_TG0_4K | TCR_TG1_4K
 #endif
 
-- 
1.7.9.5

[PATCH 13/14] arm64: Add 16K page size support

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

This patch turns on the 16K page support in the kernel. We
support 48bit VA (4 level page tables) and 47bit VA (3 level
page tables).

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Steve Capper <steve.capper@linaro.org>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig                   |   25 ++++++++++++++++++++-----
 arch/arm64/include/asm/fixmap.h      |    4 +++-
 arch/arm64/include/asm/kvm_arm.h     |   12 ++++++++++++
 arch/arm64/include/asm/page.h        |    2 ++
 arch/arm64/include/asm/sysreg.h      |    2 ++
 arch/arm64/include/asm/thread_info.h |    2 ++
 arch/arm64/kernel/head.S             |    7 ++++++-
 arch/arm64/mm/proc.S                 |    4 +++-
 8 files changed, 50 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b247897..8327edf 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -167,7 +167,8 @@ config PGTABLE_LEVELS
 	default 2 if ARM64_64K_PAGES && ARM64_VA_BITS_42
 	default 3 if ARM64_64K_PAGES && ARM64_VA_BITS_48
 	default 3 if ARM64_4K_PAGES && ARM64_VA_BITS_39
-	default 4 if ARM64_4K_PAGES && ARM64_VA_BITS_48
+	default 3 if ARM64_16K_PAGES && ARM64_VA_BITS_47
+	default 4 if !ARM64_64K_PAGES && ARM64_VA_BITS_48
 
 source "init/Kconfig"
 
@@ -444,6 +445,13 @@ config ARM64_4K_PAGES
 	help
 	  This feature enables 4KB pages support.
 
+config ARM64_16K_PAGES
+	bool "16KB"
+	help
+	  The system will use 16KB pages support. AArch32 emulation
+	  requires applications compiled with 16K(or multiple of 16K)
+	  aligned segments.
+
 config ARM64_64K_PAGES
 	bool "64KB"
 	help
@@ -457,6 +465,7 @@ endchoice
 choice
 	prompt "Virtual address space size"
 	default ARM64_VA_BITS_39 if ARM64_4K_PAGES
+	default ARM64_VA_BITS_47 if ARM64_16K_PAGES
 	default ARM64_VA_BITS_42 if ARM64_64K_PAGES
 	help
 	  Allows choosing one of multiple possible virtual address
@@ -471,6 +480,10 @@ config ARM64_VA_BITS_42
 	bool "42-bit"
 	depends on ARM64_64K_PAGES
 
+config ARM64_VA_BITS_47
+	bool "47-bit"
+	depends on ARM64_16K_PAGES
+
 config ARM64_VA_BITS_48
 	bool "48-bit"
 
@@ -480,6 +493,7 @@ config ARM64_VA_BITS
 	int
 	default 39 if ARM64_VA_BITS_39
 	default 42 if ARM64_VA_BITS_42
+	default 47 if ARM64_VA_BITS_47
 	default 48 if ARM64_VA_BITS_48
 
 config CPU_BIG_ENDIAN
@@ -550,7 +564,7 @@ config ARCH_WANT_GENERAL_HUGETLB
 	def_bool y
 
 config ARCH_WANT_HUGE_PMD_SHARE
-	def_bool y if ARM64_4K_PAGES
+	def_bool y if ARM64_4K_PAGES || ARM64_16K_PAGES
 
 config HAVE_ARCH_TRANSPARENT_HUGEPAGE
 	def_bool y
@@ -587,6 +601,7 @@ config XEN
 config FORCE_MAX_ZONEORDER
 	int
 	default "14" if (ARM64_64K_PAGES && TRANSPARENT_HUGEPAGE)
+	default "12" if (ARM64_16K_PAGES && TRANSPARENT_HUGEPAGE)
 	default "11"
 
 menuconfig ARMV8_DEPRECATED
@@ -773,9 +788,9 @@ config COMPAT
 	  the user helper functions, VFP support and the ptrace interface are
 	  handled appropriately by the kernel.
 
-	  If you also enabled CONFIG_ARM64_64K_PAGES, please be aware that you
-	  will only be able to execute AArch32 binaries that were compiled with
-	  64k aligned segments.
+	  If you use a page size other than 4KB(i.e, 16KB or 64KB), please be aware
+	  that you will only be able to execute AArch32 binaries that were compiled
+	  with page size aligned segments.
 
 	  If you want to execute 32-bit userspace applications, say Y.
 
diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h
index c0739187..f44a390 100644
--- a/arch/arm64/include/asm/fixmap.h
+++ b/arch/arm64/include/asm/fixmap.h
@@ -55,8 +55,10 @@ enum fixed_addresses {
 	 * Temporary boot-time mappings, used by early_ioremap(),
 	 * before ioremap() is functional.
 	 */
-#ifdef CONFIG_ARM64_64K_PAGES
+#if	defined(CONFIG_ARM64_64K_PAGES)
 #define NR_FIX_BTMAPS		4
+#elif	defined (CONFIG_ARM64_16K_PAGES)
+#define NR_FIX_BTMAPS		16
 #else
 #define NR_FIX_BTMAPS		64
 #endif
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index dcaf799..4d6a022 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -113,6 +113,7 @@
 #define VTCR_EL2_TG0_MASK	(3 << 14)
 #define VTCR_EL2_TG0_4K		(0 << 14)
 #define VTCR_EL2_TG0_64K	(1 << 14)
+#define VTCR_EL2_TG0_16K	(2 << 14)
 #define VTCR_EL2_SH0_MASK	(3 << 12)
 #define VTCR_EL2_SH0_INNER	(3 << 12)
 #define VTCR_EL2_ORGN0_MASK	(3 << 10)
@@ -134,6 +135,8 @@
  *
  * Note that when using 4K pages, we concatenate two first level page tables
  * together.
+ * With 16K pages, we concatenate 16 first level page tables and enter at
+ * level 2.
  *
  * The magic numbers used for VTTBR_X in this patch can be found in Tables
  * D4-23 and D4-25 in ARM DDI 0487A.b.
@@ -151,6 +154,15 @@
 #define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_64K | VTCR_EL2_SL0_LVL1 | \
 				 VTCR_EL2_COMMON_BITS)
 #define VTTBR_X		(38 - VTCR_EL2_T0SZ_40B)
+#elif defined(CONFIG_ARM64_16K_PAGES)
+/*
+ * Stage2 translation configuration:
+ * 16kB pages (TG0 = 2)
+ * 2 level page tables (SL0 = 1)
+ */
+#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_16K | VTCR_EL2_SL0_LVL1 | \
+				 VTCR_EL2_COMMON_BITS)
+#define VTTBR_X			(42 - VTCR_EL2_T0SZ_40B)
 #else
 /*
  * Stage2 translation configuration:
diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h
index 3c9ce8c..6aee732 100644
--- a/arch/arm64/include/asm/page.h
+++ b/arch/arm64/include/asm/page.h
@@ -22,6 +22,8 @@
 /* PAGE_SHIFT determines the page size */
 #ifdef CONFIG_ARM64_64K_PAGES
 #define PAGE_SHIFT		16
+#elif defined(CONFIG_ARM64_16K_PAGES)
+#define PAGE_SHIFT		14
 #else
 #define PAGE_SHIFT		12
 #endif
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index e01d323..310592c 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -89,8 +89,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
 
 #define ID_AA64MMFR0_TGran4_SHIFT	28
 #define ID_AA64MMFR0_TGran64_SHIFT	24
+#define ID_AA64MMFR0_TGran16_SHIFT	20
 
 #define ID_AA64MMFR0_TGran4_ENABLED	0x0
 #define ID_AA64MMFR0_TGran64_ENABLED	0x0
+#define ID_AA64MMFR0_TGran16_ENABLED	0x1
 
 #endif	/* __ASM_SYSREG_H */
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index d9c8c9f..338a4ac 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -25,6 +25,8 @@
 
 #ifdef CONFIG_ARM64_4K_PAGES
 #define THREAD_SIZE_ORDER	2
+#elif defined(CONFIG_ARM64_16K_PAGES)
+#define THREAD_SIZE_ORDER	0
 #endif
 
 #define THREAD_SIZE		16384
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 0cb04db..959100b 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -614,7 +614,12 @@ ENDPROC(__secondary_switched)
 #define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran64_SHIFT
 #define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran64_ENABLED
 
-#else
+#elif	defined(CONFIG_ARM64_16K_PAGES)
+
+#define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran16_SHIFT
+#define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran16_ENABLED
+
+#elif	defined(CONFIG_ARM64_4K_PAGES)
 
 #define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran4_SHIFT
 #define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran4_ENABLED
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 6e8765a..182bced 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -30,7 +30,9 @@
 
 #ifdef CONFIG_ARM64_64K_PAGES
 #define TCR_TG_FLAGS	TCR_TG0_64K | TCR_TG1_64K
-#else
+#elif defined(CONFIG_ARM64_16K_PAGES)
+#define TCR_TG_FLAGS	TCR_TG0_16K | TCR_TG1_16K
+#else /* CONFIG_ARM64_4K_PAGES */
 #define TCR_TG_FLAGS	TCR_TG0_4K | TCR_TG1_4K
 #endif
 
-- 
1.7.9.5

[PATCH 13/14] arm64: Add 16K page size support

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

This patch turns on the 16K page support in the kernel. We
support 48bit VA (4 level page tables) and 47bit VA (3 level
page tables).

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Steve Capper <steve.capper@linaro.org>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig                   |   25 ++++++++++++++++++++-----
 arch/arm64/include/asm/fixmap.h      |    4 +++-
 arch/arm64/include/asm/kvm_arm.h     |   12 ++++++++++++
 arch/arm64/include/asm/page.h        |    2 ++
 arch/arm64/include/asm/sysreg.h      |    2 ++
 arch/arm64/include/asm/thread_info.h |    2 ++
 arch/arm64/kernel/head.S             |    7 ++++++-
 arch/arm64/mm/proc.S                 |    4 +++-
 8 files changed, 50 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b247897..8327edf 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -167,7 +167,8 @@ config PGTABLE_LEVELS
 	default 2 if ARM64_64K_PAGES && ARM64_VA_BITS_42
 	default 3 if ARM64_64K_PAGES && ARM64_VA_BITS_48
 	default 3 if ARM64_4K_PAGES && ARM64_VA_BITS_39
-	default 4 if ARM64_4K_PAGES && ARM64_VA_BITS_48
+	default 3 if ARM64_16K_PAGES && ARM64_VA_BITS_47
+	default 4 if !ARM64_64K_PAGES && ARM64_VA_BITS_48
 
 source "init/Kconfig"
 
@@ -444,6 +445,13 @@ config ARM64_4K_PAGES
 	help
 	  This feature enables 4KB pages support.
 
+config ARM64_16K_PAGES
+	bool "16KB"
+	help
+	  The system will use 16KB pages support. AArch32 emulation
+	  requires applications compiled with 16K(or multiple of 16K)
+	  aligned segments.
+
 config ARM64_64K_PAGES
 	bool "64KB"
 	help
@@ -457,6 +465,7 @@ endchoice
 choice
 	prompt "Virtual address space size"
 	default ARM64_VA_BITS_39 if ARM64_4K_PAGES
+	default ARM64_VA_BITS_47 if ARM64_16K_PAGES
 	default ARM64_VA_BITS_42 if ARM64_64K_PAGES
 	help
 	  Allows choosing one of multiple possible virtual address
@@ -471,6 +480,10 @@ config ARM64_VA_BITS_42
 	bool "42-bit"
 	depends on ARM64_64K_PAGES
 
+config ARM64_VA_BITS_47
+	bool "47-bit"
+	depends on ARM64_16K_PAGES
+
 config ARM64_VA_BITS_48
 	bool "48-bit"
 
@@ -480,6 +493,7 @@ config ARM64_VA_BITS
 	int
 	default 39 if ARM64_VA_BITS_39
 	default 42 if ARM64_VA_BITS_42
+	default 47 if ARM64_VA_BITS_47
 	default 48 if ARM64_VA_BITS_48
 
 config CPU_BIG_ENDIAN
@@ -550,7 +564,7 @@ config ARCH_WANT_GENERAL_HUGETLB
 	def_bool y
 
 config ARCH_WANT_HUGE_PMD_SHARE
-	def_bool y if ARM64_4K_PAGES
+	def_bool y if ARM64_4K_PAGES || ARM64_16K_PAGES
 
 config HAVE_ARCH_TRANSPARENT_HUGEPAGE
 	def_bool y
@@ -587,6 +601,7 @@ config XEN
 config FORCE_MAX_ZONEORDER
 	int
 	default "14" if (ARM64_64K_PAGES && TRANSPARENT_HUGEPAGE)
+	default "12" if (ARM64_16K_PAGES && TRANSPARENT_HUGEPAGE)
 	default "11"
 
 menuconfig ARMV8_DEPRECATED
@@ -773,9 +788,9 @@ config COMPAT
 	  the user helper functions, VFP support and the ptrace interface are
 	  handled appropriately by the kernel.
 
-	  If you also enabled CONFIG_ARM64_64K_PAGES, please be aware that you
-	  will only be able to execute AArch32 binaries that were compiled with
-	  64k aligned segments.
+	  If you use a page size other than 4KB(i.e, 16KB or 64KB), please be aware
+	  that you will only be able to execute AArch32 binaries that were compiled
+	  with page size aligned segments.
 
 	  If you want to execute 32-bit userspace applications, say Y.
 
diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h
index c0739187..f44a390 100644
--- a/arch/arm64/include/asm/fixmap.h
+++ b/arch/arm64/include/asm/fixmap.h
@@ -55,8 +55,10 @@ enum fixed_addresses {
 	 * Temporary boot-time mappings, used by early_ioremap(),
 	 * before ioremap() is functional.
 	 */
-#ifdef CONFIG_ARM64_64K_PAGES
+#if	defined(CONFIG_ARM64_64K_PAGES)
 #define NR_FIX_BTMAPS		4
+#elif	defined (CONFIG_ARM64_16K_PAGES)
+#define NR_FIX_BTMAPS		16
 #else
 #define NR_FIX_BTMAPS		64
 #endif
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index dcaf799..4d6a022 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -113,6 +113,7 @@
 #define VTCR_EL2_TG0_MASK	(3 << 14)
 #define VTCR_EL2_TG0_4K		(0 << 14)
 #define VTCR_EL2_TG0_64K	(1 << 14)
+#define VTCR_EL2_TG0_16K	(2 << 14)
 #define VTCR_EL2_SH0_MASK	(3 << 12)
 #define VTCR_EL2_SH0_INNER	(3 << 12)
 #define VTCR_EL2_ORGN0_MASK	(3 << 10)
@@ -134,6 +135,8 @@
  *
  * Note that when using 4K pages, we concatenate two first level page tables
  * together.
+ * With 16K pages, we concatenate 16 first level page tables and enter at
+ * level 2.
  *
  * The magic numbers used for VTTBR_X in this patch can be found in Tables
  * D4-23 and D4-25 in ARM DDI 0487A.b.
@@ -151,6 +154,15 @@
 #define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_64K | VTCR_EL2_SL0_LVL1 | \
 				 VTCR_EL2_COMMON_BITS)
 #define VTTBR_X		(38 - VTCR_EL2_T0SZ_40B)
+#elif defined(CONFIG_ARM64_16K_PAGES)
+/*
+ * Stage2 translation configuration:
+ * 16kB pages (TG0 = 2)
+ * 2 level page tables (SL0 = 1)
+ */
+#define VTCR_EL2_FLAGS		(VTCR_EL2_TG0_16K | VTCR_EL2_SL0_LVL1 | \
+				 VTCR_EL2_COMMON_BITS)
+#define VTTBR_X			(42 - VTCR_EL2_T0SZ_40B)
 #else
 /*
  * Stage2 translation configuration:
diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h
index 3c9ce8c..6aee732 100644
--- a/arch/arm64/include/asm/page.h
+++ b/arch/arm64/include/asm/page.h
@@ -22,6 +22,8 @@
 /* PAGE_SHIFT determines the page size */
 #ifdef CONFIG_ARM64_64K_PAGES
 #define PAGE_SHIFT		16
+#elif defined(CONFIG_ARM64_16K_PAGES)
+#define PAGE_SHIFT		14
 #else
 #define PAGE_SHIFT		12
 #endif
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index e01d323..310592c 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -89,8 +89,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
 
 #define ID_AA64MMFR0_TGran4_SHIFT	28
 #define ID_AA64MMFR0_TGran64_SHIFT	24
+#define ID_AA64MMFR0_TGran16_SHIFT	20
 
 #define ID_AA64MMFR0_TGran4_ENABLED	0x0
 #define ID_AA64MMFR0_TGran64_ENABLED	0x0
+#define ID_AA64MMFR0_TGran16_ENABLED	0x1
 
 #endif	/* __ASM_SYSREG_H */
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index d9c8c9f..338a4ac 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -25,6 +25,8 @@
 
 #ifdef CONFIG_ARM64_4K_PAGES
 #define THREAD_SIZE_ORDER	2
+#elif defined(CONFIG_ARM64_16K_PAGES)
+#define THREAD_SIZE_ORDER	0
 #endif
 
 #define THREAD_SIZE		16384
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 0cb04db..959100b 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -614,7 +614,12 @@ ENDPROC(__secondary_switched)
 #define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran64_SHIFT
 #define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran64_ENABLED
 
-#else
+#elif	defined(CONFIG_ARM64_16K_PAGES)
+
+#define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran16_SHIFT
+#define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran16_ENABLED
+
+#elif	defined(CONFIG_ARM64_4K_PAGES)
 
 #define ID_AA64MMFR0_TGran_SHIFT	ID_AA64MMFR0_TGran4_SHIFT
 #define ID_AA64MMFR0_TGran_ENABLED	ID_AA64MMFR0_TGran4_ENABLED
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 6e8765a..182bced 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -30,7 +30,9 @@
 
 #ifdef CONFIG_ARM64_64K_PAGES
 #define TCR_TG_FLAGS	TCR_TG0_64K | TCR_TG1_64K
-#else
+#elif defined(CONFIG_ARM64_16K_PAGES)
+#define TCR_TG_FLAGS	TCR_TG0_16K | TCR_TG1_16K
+#else /* CONFIG_ARM64_4K_PAGES */
 #define TCR_TG_FLAGS	TCR_TG0_4K | TCR_TG1_4K
 #endif
 
-- 
1.7.9.5

[PATCH 14/14] arm64: 36 bit VA

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

36bit VA lets us use 2 level page tables while limiting the
available address space to 64GB.

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 8327edf..0407fd3 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -164,6 +164,7 @@ config FIX_EARLYCON_MEM
 
 config PGTABLE_LEVELS
 	int
+	default 2 if ARM64_16K_PAGES && ARM64_VA_BITS_36
 	default 2 if ARM64_64K_PAGES && ARM64_VA_BITS_42
 	default 3 if ARM64_64K_PAGES && ARM64_VA_BITS_48
 	default 3 if ARM64_4K_PAGES && ARM64_VA_BITS_39
@@ -472,6 +473,10 @@ choice
 	  space sizes. The level of translation table is determined by
 	  a combination of page size and virtual address space size.
 
+config ARM64_VA_BITS_36
+	bool "36-bit"
+	depends on ARM64_16K_PAGES
+
 config ARM64_VA_BITS_39
 	bool "39-bit"
 	depends on ARM64_4K_PAGES
@@ -491,6 +496,7 @@ endchoice
 
 config ARM64_VA_BITS
 	int
+	default 36 if ARM64_VA_BITS_36
 	default 39 if ARM64_VA_BITS_39
 	default 42 if ARM64_VA_BITS_42
 	default 47 if ARM64_VA_BITS_47
@@ -564,7 +570,7 @@ config ARCH_WANT_GENERAL_HUGETLB
 	def_bool y
 
 config ARCH_WANT_HUGE_PMD_SHARE
-	def_bool y if ARM64_4K_PAGES || ARM64_16K_PAGES
+	def_bool y if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
 
 config HAVE_ARCH_TRANSPARENT_HUGEPAGE
 	def_bool y
-- 
1.7.9.5

[PATCH 14/14] arm64: 36 bit VA

[Suzuki K. Poulose]

From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>

36bit VA lets us use 2 level page tables while limiting the
available address space to 64GB.

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/Kconfig |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 8327edf..0407fd3 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -164,6 +164,7 @@ config FIX_EARLYCON_MEM
 
 config PGTABLE_LEVELS
 	int
+	default 2 if ARM64_16K_PAGES && ARM64_VA_BITS_36
 	default 2 if ARM64_64K_PAGES && ARM64_VA_BITS_42
 	default 3 if ARM64_64K_PAGES && ARM64_VA_BITS_48
 	default 3 if ARM64_4K_PAGES && ARM64_VA_BITS_39
@@ -472,6 +473,10 @@ choice
 	  space sizes. The level of translation table is determined by
 	  a combination of page size and virtual address space size.
 
+config ARM64_VA_BITS_36
+	bool "36-bit"
+	depends on ARM64_16K_PAGES
+
 config ARM64_VA_BITS_39
 	bool "39-bit"
 	depends on ARM64_4K_PAGES
@@ -491,6 +496,7 @@ endchoice
 
 config ARM64_VA_BITS
 	int
+	default 36 if ARM64_VA_BITS_36
 	default 39 if ARM64_VA_BITS_39
 	default 42 if ARM64_VA_BITS_42
 	default 47 if ARM64_VA_BITS_47
@@ -564,7 +570,7 @@ config ARCH_WANT_GENERAL_HUGETLB
 	def_bool y
 
 config ARCH_WANT_HUGE_PMD_SHARE
-	def_bool y if ARM64_4K_PAGES || ARM64_16K_PAGES
+	def_bool y if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
 
 config HAVE_ARCH_TRANSPARENT_HUGEPAGE
 	def_bool y
-- 
1.7.9.5

Re: [PATCH 12/14] arm64: Check for selected granule support

[Steve Capper]

On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>
> Ensure that the selected page size is supported by the
> CPU(s).
>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> ---
>  arch/arm64/include/asm/sysreg.h |    6 ++++++
>  arch/arm64/kernel/head.S        |   24 +++++++++++++++++++++++-
>  2 files changed, 29 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index a7f3d4b..e01d323 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -87,4 +87,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
>  }
>  #endif
>
> +#define ID_AA64MMFR0_TGran4_SHIFT      28
> +#define ID_AA64MMFR0_TGran64_SHIFT     24
> +
> +#define ID_AA64MMFR0_TGran4_ENABLED    0x0
> +#define ID_AA64MMFR0_TGran64_ENABLED   0x0
> +
>  #endif /* __ASM_SYSREG_H */
> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index 01b8e58..0cb04db 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -31,10 +31,11 @@
>  #include <asm/cputype.h>
>  #include <asm/kernel-pgtable.h>
>  #include <asm/memory.h>
> -#include <asm/thread_info.h>
>  #include <asm/pgtable-hwdef.h>
>  #include <asm/pgtable.h>
>  #include <asm/page.h>
> +#include <asm/sysreg.h>
> +#include <asm/thread_info.h>
>  #include <asm/virt.h>
>
>  #define __PHYS_OFFSET  (KERNEL_START - TEXT_OFFSET)
> @@ -606,9 +607,25 @@ ENDPROC(__secondary_switched)
>   *  x27 = *virtual* address to jump to upon completion
>   *
>   * other registers depend on the function called upon completion
> + * Checks if the selected granule size is supported by the CPU.
>   */
> +#if    defined(CONFIG_ARM64_64K_PAGES)
> +
> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran64_SHIFT
> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran64_ENABLED
> +
> +#else
> +
> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran4_SHIFT
> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran4_ENABLED
> +
> +#endif
>         .section        ".idmap.text", "ax"
>  __enable_mmu:
> +       mrs     x1, ID_AA64MMFR0_EL1
> +       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
> +       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
> +       b.ne    __no_granule_support
>         ldr     x5, =vectors
>         msr     vbar_el1, x5
>         msr     ttbr0_el1, x25                  // load TTBR0
> @@ -626,3 +643,8 @@ __enable_mmu:
>         isb
>         br      x27
>  ENDPROC(__enable_mmu)
> +
> +__no_granule_support:
> +       wfe
> +       b __no_granule_support
> +ENDPROC(__no_granule_support)
> --
> 1.7.9.5
>

Hi Suzuki,
Is is possible to tell the user that the kernel has failed to boot due
to the kernel granule being unsupported?

Cheers,
--
Steve

Re: [PATCH 12/14] arm64: Check for selected granule support

[Steve Capper]

On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>
> Ensure that the selected page size is supported by the
> CPU(s).
>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> ---
>  arch/arm64/include/asm/sysreg.h |    6 ++++++
>  arch/arm64/kernel/head.S        |   24 +++++++++++++++++++++++-
>  2 files changed, 29 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index a7f3d4b..e01d323 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -87,4 +87,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
>  }
>  #endif
>
> +#define ID_AA64MMFR0_TGran4_SHIFT      28
> +#define ID_AA64MMFR0_TGran64_SHIFT     24
> +
> +#define ID_AA64MMFR0_TGran4_ENABLED    0x0
> +#define ID_AA64MMFR0_TGran64_ENABLED   0x0
> +
>  #endif /* __ASM_SYSREG_H */
> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index 01b8e58..0cb04db 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -31,10 +31,11 @@
>  #include <asm/cputype.h>
>  #include <asm/kernel-pgtable.h>
>  #include <asm/memory.h>
> -#include <asm/thread_info.h>
>  #include <asm/pgtable-hwdef.h>
>  #include <asm/pgtable.h>
>  #include <asm/page.h>
> +#include <asm/sysreg.h>
> +#include <asm/thread_info.h>
>  #include <asm/virt.h>
>
>  #define __PHYS_OFFSET  (KERNEL_START - TEXT_OFFSET)
> @@ -606,9 +607,25 @@ ENDPROC(__secondary_switched)
>   *  x27 = *virtual* address to jump to upon completion
>   *
>   * other registers depend on the function called upon completion
> + * Checks if the selected granule size is supported by the CPU.
>   */
> +#if    defined(CONFIG_ARM64_64K_PAGES)
> +
> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran64_SHIFT
> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran64_ENABLED
> +
> +#else
> +
> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran4_SHIFT
> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran4_ENABLED
> +
> +#endif
>         .section        ".idmap.text", "ax"
>  __enable_mmu:
> +       mrs     x1, ID_AA64MMFR0_EL1
> +       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
> +       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
> +       b.ne    __no_granule_support
>         ldr     x5, =vectors
>         msr     vbar_el1, x5
>         msr     ttbr0_el1, x25                  // load TTBR0
> @@ -626,3 +643,8 @@ __enable_mmu:
>         isb
>         br      x27
>  ENDPROC(__enable_mmu)
> +
> +__no_granule_support:
> +       wfe
> +       b __no_granule_support
> +ENDPROC(__no_granule_support)
> --
> 1.7.9.5
>

Hi Suzuki,
Is is possible to tell the user that the kernel has failed to boot due
to the kernel granule being unsupported?

Cheers,
--
Steve

[PATCH 12/14] arm64: Check for selected granule support

[Steve Capper]

On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>
> Ensure that the selected page size is supported by the
> CPU(s).
>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> ---
>  arch/arm64/include/asm/sysreg.h |    6 ++++++
>  arch/arm64/kernel/head.S        |   24 +++++++++++++++++++++++-
>  2 files changed, 29 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index a7f3d4b..e01d323 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -87,4 +87,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
>  }
>  #endif
>
> +#define ID_AA64MMFR0_TGran4_SHIFT      28
> +#define ID_AA64MMFR0_TGran64_SHIFT     24
> +
> +#define ID_AA64MMFR0_TGran4_ENABLED    0x0
> +#define ID_AA64MMFR0_TGran64_ENABLED   0x0
> +
>  #endif /* __ASM_SYSREG_H */
> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index 01b8e58..0cb04db 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -31,10 +31,11 @@
>  #include <asm/cputype.h>
>  #include <asm/kernel-pgtable.h>
>  #include <asm/memory.h>
> -#include <asm/thread_info.h>
>  #include <asm/pgtable-hwdef.h>
>  #include <asm/pgtable.h>
>  #include <asm/page.h>
> +#include <asm/sysreg.h>
> +#include <asm/thread_info.h>
>  #include <asm/virt.h>
>
>  #define __PHYS_OFFSET  (KERNEL_START - TEXT_OFFSET)
> @@ -606,9 +607,25 @@ ENDPROC(__secondary_switched)
>   *  x27 = *virtual* address to jump to upon completion
>   *
>   * other registers depend on the function called upon completion
> + * Checks if the selected granule size is supported by the CPU.
>   */
> +#if    defined(CONFIG_ARM64_64K_PAGES)
> +
> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran64_SHIFT
> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran64_ENABLED
> +
> +#else
> +
> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran4_SHIFT
> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran4_ENABLED
> +
> +#endif
>         .section        ".idmap.text", "ax"
>  __enable_mmu:
> +       mrs     x1, ID_AA64MMFR0_EL1
> +       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
> +       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
> +       b.ne    __no_granule_support
>         ldr     x5, =vectors
>         msr     vbar_el1, x5
>         msr     ttbr0_el1, x25                  // load TTBR0
> @@ -626,3 +643,8 @@ __enable_mmu:
>         isb
>         br      x27
>  ENDPROC(__enable_mmu)
> +
> +__no_granule_support:
> +       wfe
> +       b __no_granule_support
> +ENDPROC(__no_granule_support)
> --
> 1.7.9.5
>

Hi Suzuki,
Is is possible to tell the user that the kernel has failed to boot due
to the kernel granule being unsupported?

Cheers,
--
Steve

Re: [PATCH 12/14] arm64: Check for selected granule support

[Suzuki K. Poulose]

On 13/08/15 13:28, Steve Capper wrote:
> On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>
>> Ensure that the selected page size is supported by the
>> CPU(s).
>>
>> Cc: Mark Rutland <mark.rutland@arm.com>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
>> ---
>>   arch/arm64/include/asm/sysreg.h |    6 ++++++
>>   arch/arm64/kernel/head.S        |   24 +++++++++++++++++++++++-
>>   2 files changed, 29 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
>> index a7f3d4b..e01d323 100644
>> --- a/arch/arm64/include/asm/sysreg.h
>> +++ b/arch/arm64/include/asm/sysreg.h
>> @@ -87,4 +87,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
>>   }
>>   #endif
>>
>> +#define ID_AA64MMFR0_TGran4_SHIFT      28
>> +#define ID_AA64MMFR0_TGran64_SHIFT     24
>> +
>> +#define ID_AA64MMFR0_TGran4_ENABLED    0x0
>> +#define ID_AA64MMFR0_TGran64_ENABLED   0x0
>> +
>>   #endif /* __ASM_SYSREG_H */
>> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
>> index 01b8e58..0cb04db 100644
>> --- a/arch/arm64/kernel/head.S
>> +++ b/arch/arm64/kernel/head.S
>> @@ -31,10 +31,11 @@
>>   #include <asm/cputype.h>
>>   #include <asm/kernel-pgtable.h>
>>   #include <asm/memory.h>
>> -#include <asm/thread_info.h>
>>   #include <asm/pgtable-hwdef.h>
>>   #include <asm/pgtable.h>
>>   #include <asm/page.h>
>> +#include <asm/sysreg.h>
>> +#include <asm/thread_info.h>
>>   #include <asm/virt.h>
>>
>>   #define __PHYS_OFFSET  (KERNEL_START - TEXT_OFFSET)
>> @@ -606,9 +607,25 @@ ENDPROC(__secondary_switched)
>>    *  x27 = *virtual* address to jump to upon completion
>>    *
>>    * other registers depend on the function called upon completion
>> + * Checks if the selected granule size is supported by the CPU.
>>    */
>> +#if    defined(CONFIG_ARM64_64K_PAGES)
>> +
>> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran64_SHIFT
>> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran64_ENABLED
>> +
>> +#else
>> +
>> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran4_SHIFT
>> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran4_ENABLED
>> +
>> +#endif
>>          .section        ".idmap.text", "ax"
>>   __enable_mmu:
>> +       mrs     x1, ID_AA64MMFR0_EL1
>> +       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
>> +       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
>> +       b.ne    __no_granule_support
>>          ldr     x5, =vectors
>>          msr     vbar_el1, x5
>>          msr     ttbr0_el1, x25                  // load TTBR0
>> @@ -626,3 +643,8 @@ __enable_mmu:
>>          isb
>>          br      x27
>>   ENDPROC(__enable_mmu)
>> +
>> +__no_granule_support:
>> +       wfe
>> +       b __no_granule_support
>> +ENDPROC(__no_granule_support)
>> --
>> 1.7.9.5
>>
>
> Hi Suzuki,
> Is is possible to tell the user that the kernel has failed to boot due
> to the kernel granule being unsupported?

We don't have anything up at this time. The "looping address" is actually a clue
to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)
Or we should let it continue and end in a panic(?). The current situation can boot a
multi-cluster system with boot cluster having the Tgran support(which doesn't make a
strong use case though). I will try out some options and get back to you.


Thanks
Suzuki

>
> Cheers,
> --
> Steve
>

Re: [PATCH 12/14] arm64: Check for selected granule support

[Suzuki K. Poulose]

On 13/08/15 13:28, Steve Capper wrote:
> On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>
>> Ensure that the selected page size is supported by the
>> CPU(s).
>>
>> Cc: Mark Rutland <mark.rutland@arm.com>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
>> ---
>>   arch/arm64/include/asm/sysreg.h |    6 ++++++
>>   arch/arm64/kernel/head.S        |   24 +++++++++++++++++++++++-
>>   2 files changed, 29 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
>> index a7f3d4b..e01d323 100644
>> --- a/arch/arm64/include/asm/sysreg.h
>> +++ b/arch/arm64/include/asm/sysreg.h
>> @@ -87,4 +87,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
>>   }
>>   #endif
>>
>> +#define ID_AA64MMFR0_TGran4_SHIFT      28
>> +#define ID_AA64MMFR0_TGran64_SHIFT     24
>> +
>> +#define ID_AA64MMFR0_TGran4_ENABLED    0x0
>> +#define ID_AA64MMFR0_TGran64_ENABLED   0x0
>> +
>>   #endif /* __ASM_SYSREG_H */
>> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
>> index 01b8e58..0cb04db 100644
>> --- a/arch/arm64/kernel/head.S
>> +++ b/arch/arm64/kernel/head.S
>> @@ -31,10 +31,11 @@
>>   #include <asm/cputype.h>
>>   #include <asm/kernel-pgtable.h>
>>   #include <asm/memory.h>
>> -#include <asm/thread_info.h>
>>   #include <asm/pgtable-hwdef.h>
>>   #include <asm/pgtable.h>
>>   #include <asm/page.h>
>> +#include <asm/sysreg.h>
>> +#include <asm/thread_info.h>
>>   #include <asm/virt.h>
>>
>>   #define __PHYS_OFFSET  (KERNEL_START - TEXT_OFFSET)
>> @@ -606,9 +607,25 @@ ENDPROC(__secondary_switched)
>>    *  x27 = *virtual* address to jump to upon completion
>>    *
>>    * other registers depend on the function called upon completion
>> + * Checks if the selected granule size is supported by the CPU.
>>    */
>> +#if    defined(CONFIG_ARM64_64K_PAGES)
>> +
>> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran64_SHIFT
>> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran64_ENABLED
>> +
>> +#else
>> +
>> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran4_SHIFT
>> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran4_ENABLED
>> +
>> +#endif
>>          .section        ".idmap.text", "ax"
>>   __enable_mmu:
>> +       mrs     x1, ID_AA64MMFR0_EL1
>> +       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
>> +       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
>> +       b.ne    __no_granule_support
>>          ldr     x5, =vectors
>>          msr     vbar_el1, x5
>>          msr     ttbr0_el1, x25                  // load TTBR0
>> @@ -626,3 +643,8 @@ __enable_mmu:
>>          isb
>>          br      x27
>>   ENDPROC(__enable_mmu)
>> +
>> +__no_granule_support:
>> +       wfe
>> +       b __no_granule_support
>> +ENDPROC(__no_granule_support)
>> --
>> 1.7.9.5
>>
>
> Hi Suzuki,
> Is is possible to tell the user that the kernel has failed to boot due
> to the kernel granule being unsupported?

We don't have anything up at this time. The "looping address" is actually a clue
to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)
Or we should let it continue and end in a panic(?). The current situation can boot a
multi-cluster system with boot cluster having the Tgran support(which doesn't make a
strong use case though). I will try out some options and get back to you.


Thanks
Suzuki

>
> Cheers,
> --
> Steve
>

[PATCH 12/14] arm64: Check for selected granule support

[Suzuki K. Poulose]

On 13/08/15 13:28, Steve Capper wrote:
> On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>
>> Ensure that the selected page size is supported by the
>> CPU(s).
>>
>> Cc: Mark Rutland <mark.rutland@arm.com>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
>> ---
>>   arch/arm64/include/asm/sysreg.h |    6 ++++++
>>   arch/arm64/kernel/head.S        |   24 +++++++++++++++++++++++-
>>   2 files changed, 29 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
>> index a7f3d4b..e01d323 100644
>> --- a/arch/arm64/include/asm/sysreg.h
>> +++ b/arch/arm64/include/asm/sysreg.h
>> @@ -87,4 +87,10 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
>>   }
>>   #endif
>>
>> +#define ID_AA64MMFR0_TGran4_SHIFT      28
>> +#define ID_AA64MMFR0_TGran64_SHIFT     24
>> +
>> +#define ID_AA64MMFR0_TGran4_ENABLED    0x0
>> +#define ID_AA64MMFR0_TGran64_ENABLED   0x0
>> +
>>   #endif /* __ASM_SYSREG_H */
>> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
>> index 01b8e58..0cb04db 100644
>> --- a/arch/arm64/kernel/head.S
>> +++ b/arch/arm64/kernel/head.S
>> @@ -31,10 +31,11 @@
>>   #include <asm/cputype.h>
>>   #include <asm/kernel-pgtable.h>
>>   #include <asm/memory.h>
>> -#include <asm/thread_info.h>
>>   #include <asm/pgtable-hwdef.h>
>>   #include <asm/pgtable.h>
>>   #include <asm/page.h>
>> +#include <asm/sysreg.h>
>> +#include <asm/thread_info.h>
>>   #include <asm/virt.h>
>>
>>   #define __PHYS_OFFSET  (KERNEL_START - TEXT_OFFSET)
>> @@ -606,9 +607,25 @@ ENDPROC(__secondary_switched)
>>    *  x27 = *virtual* address to jump to upon completion
>>    *
>>    * other registers depend on the function called upon completion
>> + * Checks if the selected granule size is supported by the CPU.
>>    */
>> +#if    defined(CONFIG_ARM64_64K_PAGES)
>> +
>> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran64_SHIFT
>> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran64_ENABLED
>> +
>> +#else
>> +
>> +#define ID_AA64MMFR0_TGran_SHIFT       ID_AA64MMFR0_TGran4_SHIFT
>> +#define ID_AA64MMFR0_TGran_ENABLED     ID_AA64MMFR0_TGran4_ENABLED
>> +
>> +#endif
>>          .section        ".idmap.text", "ax"
>>   __enable_mmu:
>> +       mrs     x1, ID_AA64MMFR0_EL1
>> +       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
>> +       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
>> +       b.ne    __no_granule_support
>>          ldr     x5, =vectors
>>          msr     vbar_el1, x5
>>          msr     ttbr0_el1, x25                  // load TTBR0
>> @@ -626,3 +643,8 @@ __enable_mmu:
>>          isb
>>          br      x27
>>   ENDPROC(__enable_mmu)
>> +
>> +__no_granule_support:
>> +       wfe
>> +       b __no_granule_support
>> +ENDPROC(__no_granule_support)
>> --
>> 1.7.9.5
>>
>
> Hi Suzuki,
> Is is possible to tell the user that the kernel has failed to boot due
> to the kernel granule being unsupported?

We don't have anything up at this time. The "looping address" is actually a clue
to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)
Or we should let it continue and end in a panic(?). The current situation can boot a
multi-cluster system with boot cluster having the Tgran support(which doesn't make a
strong use case though). I will try out some options and get back to you.


Thanks
Suzuki

>
> Cheers,
> --
> Steve
>

Re: [PATCH 12/14] arm64: Check for selected granule support

[Catalin Marinas]

On Thu, Aug 13, 2015 at 03:45:07PM +0100, Suzuki K. Poulose wrote:
> On 13/08/15 13:28, Steve Capper wrote:
> >On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> >>  __enable_mmu:
> >>+       mrs     x1, ID_AA64MMFR0_EL1
> >>+       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
> >>+       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
> >>+       b.ne    __no_granule_support
> >>         ldr     x5, =vectors
> >>         msr     vbar_el1, x5
> >>         msr     ttbr0_el1, x25                  // load TTBR0
> >>@@ -626,3 +643,8 @@ __enable_mmu:
> >>         isb
> >>         br      x27
> >>  ENDPROC(__enable_mmu)
> >>+
> >>+__no_granule_support:
> >>+       wfe
> >>+       b __no_granule_support
> >>+ENDPROC(__no_granule_support)
> >>--
> >>1.7.9.5
> >>
> >
> >Is is possible to tell the user that the kernel has failed to boot due
> >to the kernel granule being unsupported?
> 
> We don't have anything up at this time. The "looping address" is actually a clue
> to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)

No.

> Or we should let it continue and end in a panic(?). The current situation can boot a
> multi-cluster system with boot cluster having the Tgran support(which doesn't make a
> strong use case though). I will try out some options and get back to you.

If the boot CPU does not support 16KB pages, in general there isn't much
we can do since the console printing is done after we enabled the MMU.
Even mapping the UART address requires fixmap support and the PAGE_SIZE
is hard-coded in the kernel image. The DT is also mapped at run-time.

While in theory it's possible to fall back to a 4KB page size just
enough to load the DT and figure out the early console, I suggest we
just live with the "looping address" clue.

-- 
Catalin

Re: [PATCH 12/14] arm64: Check for selected granule support

[Catalin Marinas]

On Thu, Aug 13, 2015 at 03:45:07PM +0100, Suzuki K. Poulose wrote:
> On 13/08/15 13:28, Steve Capper wrote:
> >On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> >>  __enable_mmu:
> >>+       mrs     x1, ID_AA64MMFR0_EL1
> >>+       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
> >>+       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
> >>+       b.ne    __no_granule_support
> >>         ldr     x5, =vectors
> >>         msr     vbar_el1, x5
> >>         msr     ttbr0_el1, x25                  // load TTBR0
> >>@@ -626,3 +643,8 @@ __enable_mmu:
> >>         isb
> >>         br      x27
> >>  ENDPROC(__enable_mmu)
> >>+
> >>+__no_granule_support:
> >>+       wfe
> >>+       b __no_granule_support
> >>+ENDPROC(__no_granule_support)
> >>--
> >>1.7.9.5
> >>
> >
> >Is is possible to tell the user that the kernel has failed to boot due
> >to the kernel granule being unsupported?
> 
> We don't have anything up at this time. The "looping address" is actually a clue
> to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)

No.

> Or we should let it continue and end in a panic(?). The current situation can boot a
> multi-cluster system with boot cluster having the Tgran support(which doesn't make a
> strong use case though). I will try out some options and get back to you.

If the boot CPU does not support 16KB pages, in general there isn't much
we can do since the console printing is done after we enabled the MMU.
Even mapping the UART address requires fixmap support and the PAGE_SIZE
is hard-coded in the kernel image. The DT is also mapped at run-time.

While in theory it's possible to fall back to a 4KB page size just
enough to load the DT and figure out the early console, I suggest we
just live with the "looping address" clue.

-- 
Catalin

[PATCH 12/14] arm64: Check for selected granule support

[Catalin Marinas]

On Thu, Aug 13, 2015 at 03:45:07PM +0100, Suzuki K. Poulose wrote:
> On 13/08/15 13:28, Steve Capper wrote:
> >On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> >>  __enable_mmu:
> >>+       mrs     x1, ID_AA64MMFR0_EL1
> >>+       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
> >>+       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
> >>+       b.ne    __no_granule_support
> >>         ldr     x5, =vectors
> >>         msr     vbar_el1, x5
> >>         msr     ttbr0_el1, x25                  // load TTBR0
> >>@@ -626,3 +643,8 @@ __enable_mmu:
> >>         isb
> >>         br      x27
> >>  ENDPROC(__enable_mmu)
> >>+
> >>+__no_granule_support:
> >>+       wfe
> >>+       b __no_granule_support
> >>+ENDPROC(__no_granule_support)
> >>--
> >>1.7.9.5
> >>
> >
> >Is is possible to tell the user that the kernel has failed to boot due
> >to the kernel granule being unsupported?
> 
> We don't have anything up at this time. The "looping address" is actually a clue
> to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)

No.

> Or we should let it continue and end in a panic(?). The current situation can boot a
> multi-cluster system with boot cluster having the Tgran support(which doesn't make a
> strong use case though). I will try out some options and get back to you.

If the boot CPU does not support 16KB pages, in general there isn't much
we can do since the console printing is done after we enabled the MMU.
Even mapping the UART address requires fixmap support and the PAGE_SIZE
is hard-coded in the kernel image. The DT is also mapped at run-time.

While in theory it's possible to fall back to a 4KB page size just
enough to load the DT and figure out the early console, I suggest we
just live with the "looping address" clue.

-- 
Catalin

Re: [PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Ard Biesheuvel]

On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>
> We use section maps with 4K page size to create the
> swapper/idmaps. So far we have used !64K or 4K checks
> to handle the case where we use the section maps. This
> patch adds a symbol to make it clear those cases.
>

That sentence does not make sense.

> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> ---
>  arch/arm64/include/asm/kernel-pgtable.h |   31 +++++++++-----
>  arch/arm64/mm/mmu.c                     |   70 ++++++++++++++-----------------
>  2 files changed, 51 insertions(+), 50 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
> index 622929d..5876a36 100644
> --- a/arch/arm64/include/asm/kernel-pgtable.h
> +++ b/arch/arm64/include/asm/kernel-pgtable.h
> @@ -19,6 +19,13 @@
>  #ifndef __ASM_KERNEL_PGTABLE_H
>  #define __ASM_KERNEL_PGTABLE_H
>
> +/* With 4K pages, we use section maps. */
> +#ifdef CONFIG_ARM64_4K_PAGES
> +#define ARM64_SWAPPER_USES_SECTION_MAPS 1
> +#else
> +#define ARM64_SWAPPER_USES_SECTION_MAPS 0
> +#endif
> +
>  /*
>   * The idmap and swapper page tables need some space reserved in the kernel
>   * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
> @@ -28,26 +35,28 @@
>   * could be increased on the fly if system RAM is out of reach for the default
>   * VA range, so 3 pages are reserved in all cases.
>   */
> -#ifdef CONFIG_ARM64_64K_PAGES
> -#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS)
> -#else
> +#if ARM64_SWAPPER_USES_SECTION_MAPS
>  #define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS - 1)
> +#else
> +#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS)
>  #endif
>
>  #define SWAPPER_DIR_SIZE       (SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
>  #define IDMAP_DIR_SIZE         (3 * PAGE_SIZE)
>
>  /* Initial memory map size */
> -#ifdef CONFIG_ARM64_64K_PAGES
> -#define SWAPPER_BLOCK_SHIFT    PAGE_SHIFT
> -#define SWAPPER_BLOCK_SIZE     PAGE_SIZE
> -#define SWAPPER_TABLE_SHIFT    PMD_SHIFT
> -#else
> +#if ARM64_SWAPPER_USES_SECTION_MAPS
>  #define SWAPPER_BLOCK_SHIFT    SECTION_SHIFT
>  #define SWAPPER_BLOCK_SIZE     SECTION_SIZE
>  #define SWAPPER_TABLE_SHIFT    PUD_SHIFT
> +#else
> +#define SWAPPER_BLOCK_SHIFT    PAGE_SHIFT
> +#define SWAPPER_BLOCK_SIZE     PAGE_SIZE
> +#define SWAPPER_TABLE_SHIFT    PMD_SHIFT
>  #endif
>
> +/* The size of the initial kernel direct mapping */
> +#define SWAPPER_INIT_MAP_SIZE  (_AC(1, UL) << SWAPPER_TABLE_SHIFT)
>
>  /*
>   * Initial memory map attributes.
> @@ -55,10 +64,10 @@
>  #define SWAPPER_PTE_FLAGS      PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
>  #define SWAPPER_PMD_FLAGS      PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
>
> -#ifdef CONFIG_ARM64_64K_PAGES
> -#define SWAPPER_MM_MMUFLAGS    PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
> -#else
> +#if ARM64_SWAPPER_USES_SECTION_MAPS
>  #define SWAPPER_MM_MMUFLAGS    PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS
> +#else
> +#define SWAPPER_MM_MMUFLAGS    PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
>  #endif
>
>
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index 9211b85..71230488 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -32,6 +32,7 @@
>
>  #include <asm/cputype.h>
>  #include <asm/fixmap.h>
> +#include <asm/kernel-pgtable.h>
>  #include <asm/sections.h>
>  #include <asm/setup.h>
>  #include <asm/sizes.h>
> @@ -353,14 +354,11 @@ static void __init map_mem(void)
>          * memory addressable from the initial direct kernel mapping.
>          *
>          * The initial direct kernel mapping, located at swapper_pg_dir, gives
> -        * us PUD_SIZE (4K pages) or PMD_SIZE (64K pages) memory starting from
> -        * PHYS_OFFSET (which must be aligned to 2MB as per
> -        * Documentation/arm64/booting.txt).
> +        * us PUD_SIZE (with SECTION maps, i.e, 4K) or PMD_SIZE (without
> +        * SECTION maps, i.e, 64K pages) memory starting from PHYS_OFFSET
> +        * (which must be aligned to 2MB as per Documentation/arm64/booting.txt).
>          */
> -       if (IS_ENABLED(CONFIG_ARM64_64K_PAGES))
> -               limit = PHYS_OFFSET + PMD_SIZE;
> -       else
> -               limit = PHYS_OFFSET + PUD_SIZE;
> +       limit = PHYS_OFFSET + SWAPPER_INIT_MAP_SIZE;
>         memblock_set_current_limit(limit);
>
>         /* map all the memory banks */
> @@ -371,21 +369,24 @@ static void __init map_mem(void)
>                 if (start >= end)
>                         break;
>
> -#ifndef CONFIG_ARM64_64K_PAGES
> -               /*
> -                * For the first memory bank align the start address and
> -                * current memblock limit to prevent create_mapping() from
> -                * allocating pte page tables from unmapped memory.
> -                * When 64K pages are enabled, the pte page table for the
> -                * first PGDIR_SIZE is already present in swapper_pg_dir.
> -                */
> -               if (start < limit)
> -                       start = ALIGN(start, PMD_SIZE);
> -               if (end < limit) {
> -                       limit = end & PMD_MASK;
> -                       memblock_set_current_limit(limit);
> +               if (ARM64_SWAPPER_USES_SECTION_MAPS) {
> +                       /*
> +                        * For the first memory bank align the start address and
> +                        * current memblock limit to prevent create_mapping() from
> +                        * allocating pte page tables from unmapped memory. With
> +                        * the section maps, if the first block doesn't end on section
> +                        * size boundary, create_mapping() will try to allocate a pte
> +                        * page, which may be returned from an unmapped area.
> +                        * When section maps are not used, the pte page table for the
> +                        * current limit is already present in swapper_pg_dir.
> +                        */
> +                       if (start < limit)
> +                               start = ALIGN(start, SECTION_SIZE);
> +                       if (end < limit) {
> +                               limit = end & SECTION_MASK;
> +                               memblock_set_current_limit(limit);
> +                       }
>                 }
> -#endif
>                 __map_memblock(start, end);
>         }
>
> @@ -638,7 +639,7 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
>  {
>         const u64 dt_virt_base = __fix_to_virt(FIX_FDT);
>         pgprot_t prot = PAGE_KERNEL | PTE_RDONLY;
> -       int granularity, size, offset;
> +       int size, offset;
>         void *dt_virt;
>
>         /*
> @@ -664,24 +665,15 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
>          */
>         BUILD_BUG_ON(dt_virt_base % SZ_2M);
>
> -       if (IS_ENABLED(CONFIG_ARM64_64K_PAGES)) {
> -               BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PMD_SHIFT !=
> -                            __fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT);
> -
> -               granularity = PAGE_SIZE;
> -       } else {
> -               BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PUD_SHIFT !=
> -                            __fix_to_virt(FIX_BTMAP_BEGIN) >> PUD_SHIFT);
> -
> -               granularity = PMD_SIZE;
> -       }
> +       BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> SWAPPER_TABLE_SHIFT !=
> +                    __fix_to_virt(FIX_BTMAP_BEGIN) >> SWAPPER_TABLE_SHIFT);
>
> -       offset = dt_phys % granularity;
> +       offset = dt_phys % SWAPPER_BLOCK_SIZE;
>         dt_virt = (void *)dt_virt_base + offset;
>
>         /* map the first chunk so we can read the size from the header */
> -       create_mapping(round_down(dt_phys, granularity), dt_virt_base,
> -                      granularity, prot);
> +       create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
> +                      SWAPPER_BLOCK_SIZE, prot);
>
>         if (fdt_check_header(dt_virt) != 0)
>                 return NULL;
> @@ -690,9 +682,9 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
>         if (size > MAX_FDT_SIZE)
>                 return NULL;
>
> -       if (offset + size > granularity)
> -               create_mapping(round_down(dt_phys, granularity), dt_virt_base,
> -                              round_up(offset + size, granularity), prot);
> +       if (offset + size > SWAPPER_BLOCK_SIZE)
> +               create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
> +                              round_up(offset + size, SWAPPER_BLOCK_SIZE), prot);
>
>         memblock_reserve(dt_phys, size);
>
> --
> 1.7.9.5
>

Re: [PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Ard Biesheuvel]

On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>
> We use section maps with 4K page size to create the
> swapper/idmaps. So far we have used !64K or 4K checks
> to handle the case where we use the section maps. This
> patch adds a symbol to make it clear those cases.
>

That sentence does not make sense.

> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> ---
>  arch/arm64/include/asm/kernel-pgtable.h |   31 +++++++++-----
>  arch/arm64/mm/mmu.c                     |   70 ++++++++++++++-----------------
>  2 files changed, 51 insertions(+), 50 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
> index 622929d..5876a36 100644
> --- a/arch/arm64/include/asm/kernel-pgtable.h
> +++ b/arch/arm64/include/asm/kernel-pgtable.h
> @@ -19,6 +19,13 @@
>  #ifndef __ASM_KERNEL_PGTABLE_H
>  #define __ASM_KERNEL_PGTABLE_H
>
> +/* With 4K pages, we use section maps. */
> +#ifdef CONFIG_ARM64_4K_PAGES
> +#define ARM64_SWAPPER_USES_SECTION_MAPS 1
> +#else
> +#define ARM64_SWAPPER_USES_SECTION_MAPS 0
> +#endif
> +
>  /*
>   * The idmap and swapper page tables need some space reserved in the kernel
>   * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
> @@ -28,26 +35,28 @@
>   * could be increased on the fly if system RAM is out of reach for the default
>   * VA range, so 3 pages are reserved in all cases.
>   */
> -#ifdef CONFIG_ARM64_64K_PAGES
> -#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS)
> -#else
> +#if ARM64_SWAPPER_USES_SECTION_MAPS
>  #define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS - 1)
> +#else
> +#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS)
>  #endif
>
>  #define SWAPPER_DIR_SIZE       (SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
>  #define IDMAP_DIR_SIZE         (3 * PAGE_SIZE)
>
>  /* Initial memory map size */
> -#ifdef CONFIG_ARM64_64K_PAGES
> -#define SWAPPER_BLOCK_SHIFT    PAGE_SHIFT
> -#define SWAPPER_BLOCK_SIZE     PAGE_SIZE
> -#define SWAPPER_TABLE_SHIFT    PMD_SHIFT
> -#else
> +#if ARM64_SWAPPER_USES_SECTION_MAPS
>  #define SWAPPER_BLOCK_SHIFT    SECTION_SHIFT
>  #define SWAPPER_BLOCK_SIZE     SECTION_SIZE
>  #define SWAPPER_TABLE_SHIFT    PUD_SHIFT
> +#else
> +#define SWAPPER_BLOCK_SHIFT    PAGE_SHIFT
> +#define SWAPPER_BLOCK_SIZE     PAGE_SIZE
> +#define SWAPPER_TABLE_SHIFT    PMD_SHIFT
>  #endif
>
> +/* The size of the initial kernel direct mapping */
> +#define SWAPPER_INIT_MAP_SIZE  (_AC(1, UL) << SWAPPER_TABLE_SHIFT)
>
>  /*
>   * Initial memory map attributes.
> @@ -55,10 +64,10 @@
>  #define SWAPPER_PTE_FLAGS      PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
>  #define SWAPPER_PMD_FLAGS      PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
>
> -#ifdef CONFIG_ARM64_64K_PAGES
> -#define SWAPPER_MM_MMUFLAGS    PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
> -#else
> +#if ARM64_SWAPPER_USES_SECTION_MAPS
>  #define SWAPPER_MM_MMUFLAGS    PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS
> +#else
> +#define SWAPPER_MM_MMUFLAGS    PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
>  #endif
>
>
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index 9211b85..71230488 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -32,6 +32,7 @@
>
>  #include <asm/cputype.h>
>  #include <asm/fixmap.h>
> +#include <asm/kernel-pgtable.h>
>  #include <asm/sections.h>
>  #include <asm/setup.h>
>  #include <asm/sizes.h>
> @@ -353,14 +354,11 @@ static void __init map_mem(void)
>          * memory addressable from the initial direct kernel mapping.
>          *
>          * The initial direct kernel mapping, located at swapper_pg_dir, gives
> -        * us PUD_SIZE (4K pages) or PMD_SIZE (64K pages) memory starting from
> -        * PHYS_OFFSET (which must be aligned to 2MB as per
> -        * Documentation/arm64/booting.txt).
> +        * us PUD_SIZE (with SECTION maps, i.e, 4K) or PMD_SIZE (without
> +        * SECTION maps, i.e, 64K pages) memory starting from PHYS_OFFSET
> +        * (which must be aligned to 2MB as per Documentation/arm64/booting.txt).
>          */
> -       if (IS_ENABLED(CONFIG_ARM64_64K_PAGES))
> -               limit = PHYS_OFFSET + PMD_SIZE;
> -       else
> -               limit = PHYS_OFFSET + PUD_SIZE;
> +       limit = PHYS_OFFSET + SWAPPER_INIT_MAP_SIZE;
>         memblock_set_current_limit(limit);
>
>         /* map all the memory banks */
> @@ -371,21 +369,24 @@ static void __init map_mem(void)
>                 if (start >= end)
>                         break;
>
> -#ifndef CONFIG_ARM64_64K_PAGES
> -               /*
> -                * For the first memory bank align the start address and
> -                * current memblock limit to prevent create_mapping() from
> -                * allocating pte page tables from unmapped memory.
> -                * When 64K pages are enabled, the pte page table for the
> -                * first PGDIR_SIZE is already present in swapper_pg_dir.
> -                */
> -               if (start < limit)
> -                       start = ALIGN(start, PMD_SIZE);
> -               if (end < limit) {
> -                       limit = end & PMD_MASK;
> -                       memblock_set_current_limit(limit);
> +               if (ARM64_SWAPPER_USES_SECTION_MAPS) {
> +                       /*
> +                        * For the first memory bank align the start address and
> +                        * current memblock limit to prevent create_mapping() from
> +                        * allocating pte page tables from unmapped memory. With
> +                        * the section maps, if the first block doesn't end on section
> +                        * size boundary, create_mapping() will try to allocate a pte
> +                        * page, which may be returned from an unmapped area.
> +                        * When section maps are not used, the pte page table for the
> +                        * current limit is already present in swapper_pg_dir.
> +                        */
> +                       if (start < limit)
> +                               start = ALIGN(start, SECTION_SIZE);
> +                       if (end < limit) {
> +                               limit = end & SECTION_MASK;
> +                               memblock_set_current_limit(limit);
> +                       }
>                 }
> -#endif
>                 __map_memblock(start, end);
>         }
>
> @@ -638,7 +639,7 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
>  {
>         const u64 dt_virt_base = __fix_to_virt(FIX_FDT);
>         pgprot_t prot = PAGE_KERNEL | PTE_RDONLY;
> -       int granularity, size, offset;
> +       int size, offset;
>         void *dt_virt;
>
>         /*
> @@ -664,24 +665,15 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
>          */
>         BUILD_BUG_ON(dt_virt_base % SZ_2M);
>
> -       if (IS_ENABLED(CONFIG_ARM64_64K_PAGES)) {
> -               BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PMD_SHIFT !=
> -                            __fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT);
> -
> -               granularity = PAGE_SIZE;
> -       } else {
> -               BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PUD_SHIFT !=
> -                            __fix_to_virt(FIX_BTMAP_BEGIN) >> PUD_SHIFT);
> -
> -               granularity = PMD_SIZE;
> -       }
> +       BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> SWAPPER_TABLE_SHIFT !=
> +                    __fix_to_virt(FIX_BTMAP_BEGIN) >> SWAPPER_TABLE_SHIFT);
>
> -       offset = dt_phys % granularity;
> +       offset = dt_phys % SWAPPER_BLOCK_SIZE;
>         dt_virt = (void *)dt_virt_base + offset;
>
>         /* map the first chunk so we can read the size from the header */
> -       create_mapping(round_down(dt_phys, granularity), dt_virt_base,
> -                      granularity, prot);
> +       create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
> +                      SWAPPER_BLOCK_SIZE, prot);
>
>         if (fdt_check_header(dt_virt) != 0)
>                 return NULL;
> @@ -690,9 +682,9 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
>         if (size > MAX_FDT_SIZE)
>                 return NULL;
>
> -       if (offset + size > granularity)
> -               create_mapping(round_down(dt_phys, granularity), dt_virt_base,
> -                              round_up(offset + size, granularity), prot);
> +       if (offset + size > SWAPPER_BLOCK_SIZE)
> +               create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
> +                              round_up(offset + size, SWAPPER_BLOCK_SIZE), prot);
>
>         memblock_reserve(dt_phys, size);
>
> --
> 1.7.9.5
>

[PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Ard Biesheuvel]

On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>
> We use section maps with 4K page size to create the
> swapper/idmaps. So far we have used !64K or 4K checks
> to handle the case where we use the section maps. This
> patch adds a symbol to make it clear those cases.
>

That sentence does not make sense.

> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Suzuki K. Poulose <suzuki.poulose@arm.com>
> ---
>  arch/arm64/include/asm/kernel-pgtable.h |   31 +++++++++-----
>  arch/arm64/mm/mmu.c                     |   70 ++++++++++++++-----------------
>  2 files changed, 51 insertions(+), 50 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
> index 622929d..5876a36 100644
> --- a/arch/arm64/include/asm/kernel-pgtable.h
> +++ b/arch/arm64/include/asm/kernel-pgtable.h
> @@ -19,6 +19,13 @@
>  #ifndef __ASM_KERNEL_PGTABLE_H
>  #define __ASM_KERNEL_PGTABLE_H
>
> +/* With 4K pages, we use section maps. */
> +#ifdef CONFIG_ARM64_4K_PAGES
> +#define ARM64_SWAPPER_USES_SECTION_MAPS 1
> +#else
> +#define ARM64_SWAPPER_USES_SECTION_MAPS 0
> +#endif
> +
>  /*
>   * The idmap and swapper page tables need some space reserved in the kernel
>   * image. Both require pgd, pud (4 levels only) and pmd tables to (section)
> @@ -28,26 +35,28 @@
>   * could be increased on the fly if system RAM is out of reach for the default
>   * VA range, so 3 pages are reserved in all cases.
>   */
> -#ifdef CONFIG_ARM64_64K_PAGES
> -#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS)
> -#else
> +#if ARM64_SWAPPER_USES_SECTION_MAPS
>  #define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS - 1)
> +#else
> +#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS)
>  #endif
>
>  #define SWAPPER_DIR_SIZE       (SWAPPER_PGTABLE_LEVELS * PAGE_SIZE)
>  #define IDMAP_DIR_SIZE         (3 * PAGE_SIZE)
>
>  /* Initial memory map size */
> -#ifdef CONFIG_ARM64_64K_PAGES
> -#define SWAPPER_BLOCK_SHIFT    PAGE_SHIFT
> -#define SWAPPER_BLOCK_SIZE     PAGE_SIZE
> -#define SWAPPER_TABLE_SHIFT    PMD_SHIFT
> -#else
> +#if ARM64_SWAPPER_USES_SECTION_MAPS
>  #define SWAPPER_BLOCK_SHIFT    SECTION_SHIFT
>  #define SWAPPER_BLOCK_SIZE     SECTION_SIZE
>  #define SWAPPER_TABLE_SHIFT    PUD_SHIFT
> +#else
> +#define SWAPPER_BLOCK_SHIFT    PAGE_SHIFT
> +#define SWAPPER_BLOCK_SIZE     PAGE_SIZE
> +#define SWAPPER_TABLE_SHIFT    PMD_SHIFT
>  #endif
>
> +/* The size of the initial kernel direct mapping */
> +#define SWAPPER_INIT_MAP_SIZE  (_AC(1, UL) << SWAPPER_TABLE_SHIFT)
>
>  /*
>   * Initial memory map attributes.
> @@ -55,10 +64,10 @@
>  #define SWAPPER_PTE_FLAGS      PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
>  #define SWAPPER_PMD_FLAGS      PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
>
> -#ifdef CONFIG_ARM64_64K_PAGES
> -#define SWAPPER_MM_MMUFLAGS    PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
> -#else
> +#if ARM64_SWAPPER_USES_SECTION_MAPS
>  #define SWAPPER_MM_MMUFLAGS    PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS
> +#else
> +#define SWAPPER_MM_MMUFLAGS    PTE_ATTRINDX(MT_NORMAL) | SWAPPER_PTE_FLAGS
>  #endif
>
>
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index 9211b85..71230488 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -32,6 +32,7 @@
>
>  #include <asm/cputype.h>
>  #include <asm/fixmap.h>
> +#include <asm/kernel-pgtable.h>
>  #include <asm/sections.h>
>  #include <asm/setup.h>
>  #include <asm/sizes.h>
> @@ -353,14 +354,11 @@ static void __init map_mem(void)
>          * memory addressable from the initial direct kernel mapping.
>          *
>          * The initial direct kernel mapping, located at swapper_pg_dir, gives
> -        * us PUD_SIZE (4K pages) or PMD_SIZE (64K pages) memory starting from
> -        * PHYS_OFFSET (which must be aligned to 2MB as per
> -        * Documentation/arm64/booting.txt).
> +        * us PUD_SIZE (with SECTION maps, i.e, 4K) or PMD_SIZE (without
> +        * SECTION maps, i.e, 64K pages) memory starting from PHYS_OFFSET
> +        * (which must be aligned to 2MB as per Documentation/arm64/booting.txt).
>          */
> -       if (IS_ENABLED(CONFIG_ARM64_64K_PAGES))
> -               limit = PHYS_OFFSET + PMD_SIZE;
> -       else
> -               limit = PHYS_OFFSET + PUD_SIZE;
> +       limit = PHYS_OFFSET + SWAPPER_INIT_MAP_SIZE;
>         memblock_set_current_limit(limit);
>
>         /* map all the memory banks */
> @@ -371,21 +369,24 @@ static void __init map_mem(void)
>                 if (start >= end)
>                         break;
>
> -#ifndef CONFIG_ARM64_64K_PAGES
> -               /*
> -                * For the first memory bank align the start address and
> -                * current memblock limit to prevent create_mapping() from
> -                * allocating pte page tables from unmapped memory.
> -                * When 64K pages are enabled, the pte page table for the
> -                * first PGDIR_SIZE is already present in swapper_pg_dir.
> -                */
> -               if (start < limit)
> -                       start = ALIGN(start, PMD_SIZE);
> -               if (end < limit) {
> -                       limit = end & PMD_MASK;
> -                       memblock_set_current_limit(limit);
> +               if (ARM64_SWAPPER_USES_SECTION_MAPS) {
> +                       /*
> +                        * For the first memory bank align the start address and
> +                        * current memblock limit to prevent create_mapping() from
> +                        * allocating pte page tables from unmapped memory. With
> +                        * the section maps, if the first block doesn't end on section
> +                        * size boundary, create_mapping() will try to allocate a pte
> +                        * page, which may be returned from an unmapped area.
> +                        * When section maps are not used, the pte page table for the
> +                        * current limit is already present in swapper_pg_dir.
> +                        */
> +                       if (start < limit)
> +                               start = ALIGN(start, SECTION_SIZE);
> +                       if (end < limit) {
> +                               limit = end & SECTION_MASK;
> +                               memblock_set_current_limit(limit);
> +                       }
>                 }
> -#endif
>                 __map_memblock(start, end);
>         }
>
> @@ -638,7 +639,7 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
>  {
>         const u64 dt_virt_base = __fix_to_virt(FIX_FDT);
>         pgprot_t prot = PAGE_KERNEL | PTE_RDONLY;
> -       int granularity, size, offset;
> +       int size, offset;
>         void *dt_virt;
>
>         /*
> @@ -664,24 +665,15 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
>          */
>         BUILD_BUG_ON(dt_virt_base % SZ_2M);
>
> -       if (IS_ENABLED(CONFIG_ARM64_64K_PAGES)) {
> -               BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PMD_SHIFT !=
> -                            __fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT);
> -
> -               granularity = PAGE_SIZE;
> -       } else {
> -               BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> PUD_SHIFT !=
> -                            __fix_to_virt(FIX_BTMAP_BEGIN) >> PUD_SHIFT);
> -
> -               granularity = PMD_SIZE;
> -       }
> +       BUILD_BUG_ON(__fix_to_virt(FIX_FDT_END) >> SWAPPER_TABLE_SHIFT !=
> +                    __fix_to_virt(FIX_BTMAP_BEGIN) >> SWAPPER_TABLE_SHIFT);
>
> -       offset = dt_phys % granularity;
> +       offset = dt_phys % SWAPPER_BLOCK_SIZE;
>         dt_virt = (void *)dt_virt_base + offset;
>
>         /* map the first chunk so we can read the size from the header */
> -       create_mapping(round_down(dt_phys, granularity), dt_virt_base,
> -                      granularity, prot);
> +       create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
> +                      SWAPPER_BLOCK_SIZE, prot);
>
>         if (fdt_check_header(dt_virt) != 0)
>                 return NULL;
> @@ -690,9 +682,9 @@ void *__init fixmap_remap_fdt(phys_addr_t dt_phys)
>         if (size > MAX_FDT_SIZE)
>                 return NULL;
>
> -       if (offset + size > granularity)
> -               create_mapping(round_down(dt_phys, granularity), dt_virt_base,
> -                              round_up(offset + size, granularity), prot);
> +       if (offset + size > SWAPPER_BLOCK_SIZE)
> +               create_mapping(round_down(dt_phys, SWAPPER_BLOCK_SIZE), dt_virt_base,
> +                              round_up(offset + size, SWAPPER_BLOCK_SIZE), prot);
>
>         memblock_reserve(dt_phys, size);
>
> --
> 1.7.9.5
>

Re: [PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Suzuki K. Poulose]

On 02/09/15 10:38, Ard Biesheuvel wrote:
> On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>
>> We use section maps with 4K page size to create the
>> swapper/idmaps. So far we have used !64K or 4K checks
>> to handle the case where we use the section maps. This
>> patch adds a symbol to make it clear those cases.
>>
>
> That sentence does not make sense.

I agree. How about :

"This patch adds a new symbol, 'ARM64_SWAPPER_USES_SECTION_MAPS', to
handle cases where we use section maps, instead of using the page size
symbols."

Suzuki

Re: [PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Suzuki K. Poulose]

On 02/09/15 10:38, Ard Biesheuvel wrote:
> On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>
>> We use section maps with 4K page size to create the
>> swapper/idmaps. So far we have used !64K or 4K checks
>> to handle the case where we use the section maps. This
>> patch adds a symbol to make it clear those cases.
>>
>
> That sentence does not make sense.

I agree. How about :

"This patch adds a new symbol, 'ARM64_SWAPPER_USES_SECTION_MAPS', to
handle cases where we use section maps, instead of using the page size
symbols."

Suzuki

[PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Suzuki K. Poulose]

On 02/09/15 10:38, Ard Biesheuvel wrote:
> On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>
>> We use section maps with 4K page size to create the
>> swapper/idmaps. So far we have used !64K or 4K checks
>> to handle the case where we use the section maps. This
>> patch adds a symbol to make it clear those cases.
>>
>
> That sentence does not make sense.

I agree. How about :

"This patch adds a new symbol, 'ARM64_SWAPPER_USES_SECTION_MAPS', to
handle cases where we use section maps, instead of using the page size
symbols."

Suzuki

Re: [PATCH 12/14] arm64: Check for selected granule support

[Ard Biesheuvel]

On 13 August 2015 at 19:29, Catalin Marinas <catalin.marinas@arm.com> wrote:
> On Thu, Aug 13, 2015 at 03:45:07PM +0100, Suzuki K. Poulose wrote:
>> On 13/08/15 13:28, Steve Capper wrote:
>> >On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> >>  __enable_mmu:
>> >>+       mrs     x1, ID_AA64MMFR0_EL1
>> >>+       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
>> >>+       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
>> >>+       b.ne    __no_granule_support
>> >>         ldr     x5, =vectors
>> >>         msr     vbar_el1, x5
>> >>         msr     ttbr0_el1, x25                  // load TTBR0
>> >>@@ -626,3 +643,8 @@ __enable_mmu:
>> >>         isb
>> >>         br      x27
>> >>  ENDPROC(__enable_mmu)
>> >>+
>> >>+__no_granule_support:
>> >>+       wfe
>> >>+       b __no_granule_support
>> >>+ENDPROC(__no_granule_support)
>> >>--
>> >>1.7.9.5
>> >>
>> >
>> >Is is possible to tell the user that the kernel has failed to boot due
>> >to the kernel granule being unsupported?
>>
>> We don't have anything up at this time. The "looping address" is actually a clue
>> to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)
>
> No.
>
>> Or we should let it continue and end in a panic(?). The current situation can boot a
>> multi-cluster system with boot cluster having the Tgran support(which doesn't make a
>> strong use case though). I will try out some options and get back to you.
>
> If the boot CPU does not support 16KB pages, in general there isn't much
> we can do since the console printing is done after we enabled the MMU.
> Even mapping the UART address requires fixmap support and the PAGE_SIZE
> is hard-coded in the kernel image. The DT is also mapped at run-time.
>
> While in theory it's possible to fall back to a 4KB page size just
> enough to load the DT and figure out the early console, I suggest we
> just live with the "looping address" clue.
>

Couldn't we allocate some flag bits in the Image header to communicate
the page size to the bootloader?

Re: [PATCH 12/14] arm64: Check for selected granule support

[Ard Biesheuvel]

On 13 August 2015 at 19:29, Catalin Marinas <catalin.marinas@arm.com> wrote:
> On Thu, Aug 13, 2015 at 03:45:07PM +0100, Suzuki K. Poulose wrote:
>> On 13/08/15 13:28, Steve Capper wrote:
>> >On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> >>  __enable_mmu:
>> >>+       mrs     x1, ID_AA64MMFR0_EL1
>> >>+       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
>> >>+       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
>> >>+       b.ne    __no_granule_support
>> >>         ldr     x5, =vectors
>> >>         msr     vbar_el1, x5
>> >>         msr     ttbr0_el1, x25                  // load TTBR0
>> >>@@ -626,3 +643,8 @@ __enable_mmu:
>> >>         isb
>> >>         br      x27
>> >>  ENDPROC(__enable_mmu)
>> >>+
>> >>+__no_granule_support:
>> >>+       wfe
>> >>+       b __no_granule_support
>> >>+ENDPROC(__no_granule_support)
>> >>--
>> >>1.7.9.5
>> >>
>> >
>> >Is is possible to tell the user that the kernel has failed to boot due
>> >to the kernel granule being unsupported?
>>
>> We don't have anything up at this time. The "looping address" is actually a clue
>> to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)
>
> No.
>
>> Or we should let it continue and end in a panic(?). The current situation can boot a
>> multi-cluster system with boot cluster having the Tgran support(which doesn't make a
>> strong use case though). I will try out some options and get back to you.
>
> If the boot CPU does not support 16KB pages, in general there isn't much
> we can do since the console printing is done after we enabled the MMU.
> Even mapping the UART address requires fixmap support and the PAGE_SIZE
> is hard-coded in the kernel image. The DT is also mapped at run-time.
>
> While in theory it's possible to fall back to a 4KB page size just
> enough to load the DT and figure out the early console, I suggest we
> just live with the "looping address" clue.
>

Couldn't we allocate some flag bits in the Image header to communicate
the page size to the bootloader?

[PATCH 12/14] arm64: Check for selected granule support

[Ard Biesheuvel]

On 13 August 2015 at 19:29, Catalin Marinas <catalin.marinas@arm.com> wrote:
> On Thu, Aug 13, 2015 at 03:45:07PM +0100, Suzuki K. Poulose wrote:
>> On 13/08/15 13:28, Steve Capper wrote:
>> >On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> >>  __enable_mmu:
>> >>+       mrs     x1, ID_AA64MMFR0_EL1
>> >>+       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
>> >>+       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
>> >>+       b.ne    __no_granule_support
>> >>         ldr     x5, =vectors
>> >>         msr     vbar_el1, x5
>> >>         msr     ttbr0_el1, x25                  // load TTBR0
>> >>@@ -626,3 +643,8 @@ __enable_mmu:
>> >>         isb
>> >>         br      x27
>> >>  ENDPROC(__enable_mmu)
>> >>+
>> >>+__no_granule_support:
>> >>+       wfe
>> >>+       b __no_granule_support
>> >>+ENDPROC(__no_granule_support)
>> >>--
>> >>1.7.9.5
>> >>
>> >
>> >Is is possible to tell the user that the kernel has failed to boot due
>> >to the kernel granule being unsupported?
>>
>> We don't have anything up at this time. The "looping address" is actually a clue
>> to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)
>
> No.
>
>> Or we should let it continue and end in a panic(?). The current situation can boot a
>> multi-cluster system with boot cluster having the Tgran support(which doesn't make a
>> strong use case though). I will try out some options and get back to you.
>
> If the boot CPU does not support 16KB pages, in general there isn't much
> we can do since the console printing is done after we enabled the MMU.
> Even mapping the UART address requires fixmap support and the PAGE_SIZE
> is hard-coded in the kernel image. The DT is also mapped at run-time.
>
> While in theory it's possible to fall back to a 4KB page size just
> enough to load the DT and figure out the early console, I suggest we
> just live with the "looping address" clue.
>

Couldn't we allocate some flag bits in the Image header to communicate
the page size to the bootloader?

Re: [PATCH 00/14] arm64: 16K translation granule support

[Ard Biesheuvel]

On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>
> This series enables the 16K page size support on Linux for arm64.
> This series adds support for 48bit VA(4 level), 47bit VA(3 level) and
> 36bit VA(2 level) with 16K. 16K was a late addition to the architecture
> and is not implemented by all CPUs. Added a check to ensure the
> selected granule size is supported by the CPU, failing which the CPU
> won't proceed with booting.
>
> KVM bits have been tested on a fast model with GICv3 using Andre's kvmtool
> with gicv3 support[1].
>
> Patches 1-7 cleans up the kernel page size handling code.
> Patches 8-11 Fixes some issues with the KVM bits, mainly the fake PGD
>              handling code.
> Patch 12    Adds a check to ensure the CPU supports the selected granule size.
> Patch 13-14 Adds the 16k page size support bits.
>
> This series applies on top of for-next/core branch of the aarch64 tree and is
> also available here:
>
>         git://linux-arm.org/linux-skp.git  16k/v1
>


Hi Suzuki,

I have given this a spin on the FVP Base model to check UEFI booting,
and everything seems to work fine. (I tested 2-level and 3-level)
I didn't test the KVM changes, so for all patches except those:

Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Regards,
Ard.

Re: [PATCH 00/14] arm64: 16K translation granule support

[Ard Biesheuvel]

On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>
> This series enables the 16K page size support on Linux for arm64.
> This series adds support for 48bit VA(4 level), 47bit VA(3 level) and
> 36bit VA(2 level) with 16K. 16K was a late addition to the architecture
> and is not implemented by all CPUs. Added a check to ensure the
> selected granule size is supported by the CPU, failing which the CPU
> won't proceed with booting.
>
> KVM bits have been tested on a fast model with GICv3 using Andre's kvmtool
> with gicv3 support[1].
>
> Patches 1-7 cleans up the kernel page size handling code.
> Patches 8-11 Fixes some issues with the KVM bits, mainly the fake PGD
>              handling code.
> Patch 12    Adds a check to ensure the CPU supports the selected granule size.
> Patch 13-14 Adds the 16k page size support bits.
>
> This series applies on top of for-next/core branch of the aarch64 tree and is
> also available here:
>
>         git://linux-arm.org/linux-skp.git  16k/v1
>


Hi Suzuki,

I have given this a spin on the FVP Base model to check UEFI booting,
and everything seems to work fine. (I tested 2-level and 3-level)
I didn't test the KVM changes, so for all patches except those:

Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Regards,
Ard.

[PATCH 00/14] arm64: 16K translation granule support

[Ard Biesheuvel]

On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>
> This series enables the 16K page size support on Linux for arm64.
> This series adds support for 48bit VA(4 level), 47bit VA(3 level) and
> 36bit VA(2 level) with 16K. 16K was a late addition to the architecture
> and is not implemented by all CPUs. Added a check to ensure the
> selected granule size is supported by the CPU, failing which the CPU
> won't proceed with booting.
>
> KVM bits have been tested on a fast model with GICv3 using Andre's kvmtool
> with gicv3 support[1].
>
> Patches 1-7 cleans up the kernel page size handling code.
> Patches 8-11 Fixes some issues with the KVM bits, mainly the fake PGD
>              handling code.
> Patch 12    Adds a check to ensure the CPU supports the selected granule size.
> Patch 13-14 Adds the 16k page size support bits.
>
> This series applies on top of for-next/core branch of the aarch64 tree and is
> also available here:
>
>         git://linux-arm.org/linux-skp.git  16k/v1
>


Hi Suzuki,

I have given this a spin on the FVP Base model to check UEFI booting,
and everything seems to work fine. (I tested 2-level and 3-level)
I didn't test the KVM changes, so for all patches except those:

Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Regards,
Ard.

Re: [PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Ard Biesheuvel]

On 2 September 2015 at 11:42, Suzuki K. Poulose <Suzuki.Poulose@arm.com> wrote:
> On 02/09/15 10:38, Ard Biesheuvel wrote:
>>
>> On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com>
>> wrote:
>>>
>>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>>
>>> We use section maps with 4K page size to create the
>>> swapper/idmaps. So far we have used !64K or 4K checks
>>> to handle the case where we use the section maps. This
>>> patch adds a symbol to make it clear those cases.
>>>
>>
>> That sentence does not make sense.
>
>
> I agree. How about :
>
> "This patch adds a new symbol, 'ARM64_SWAPPER_USES_SECTION_MAPS', to
> handle cases where we use section maps, instead of using the page size
> symbols."
>

Yep, much better

Re: [PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Ard Biesheuvel]

On 2 September 2015 at 11:42, Suzuki K. Poulose <Suzuki.Poulose@arm.com> wrote:
> On 02/09/15 10:38, Ard Biesheuvel wrote:
>>
>> On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com>
>> wrote:
>>>
>>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>>
>>> We use section maps with 4K page size to create the
>>> swapper/idmaps. So far we have used !64K or 4K checks
>>> to handle the case where we use the section maps. This
>>> patch adds a symbol to make it clear those cases.
>>>
>>
>> That sentence does not make sense.
>
>
> I agree. How about :
>
> "This patch adds a new symbol, 'ARM64_SWAPPER_USES_SECTION_MAPS', to
> handle cases where we use section maps, instead of using the page size
> symbols."
>

Yep, much better

[PATCH 02/14] arm64: Handle section maps for swapper/idmap

[Ard Biesheuvel]

On 2 September 2015 at 11:42, Suzuki K. Poulose <Suzuki.Poulose@arm.com> wrote:
> On 02/09/15 10:38, Ard Biesheuvel wrote:
>>
>> On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com>
>> wrote:
>>>
>>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>>
>>> We use section maps with 4K page size to create the
>>> swapper/idmaps. So far we have used !64K or 4K checks
>>> to handle the case where we use the section maps. This
>>> patch adds a symbol to make it clear those cases.
>>>
>>
>> That sentence does not make sense.
>
>
> I agree. How about :
>
> "This patch adds a new symbol, 'ARM64_SWAPPER_USES_SECTION_MAPS', to
> handle cases where we use section maps, instead of using the page size
> symbols."
>

Yep, much better

Re: [PATCH 00/14] arm64: 16K translation granule support

[Suzuki K. Poulose]

On 02/09/15 10:55, Ard Biesheuvel wrote:
> On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>

>> Patches 1-7 cleans up the kernel page size handling code.
>> Patches 8-11 Fixes some issues with the KVM bits, mainly the fake PGD
>>               handling code.
>> Patch 12    Adds a check to ensure the CPU supports the selected granule size.
>> Patch 13-14 Adds the 16k page size support bits.
>>
>> This series applies on top of for-next/core branch of the aarch64 tree and is
>> also available here:
>>
>>          git://linux-arm.org/linux-skp.git  16k/v1
>>
>
>
> Hi Suzuki,
>
> I have given this a spin on the FVP Base model to check UEFI booting,
> and everything seems to work fine. (I tested 2-level and 3-level)
> I didn't test the KVM changes, so for all patches except those:
>
> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Thanks for the review and testing !!

Suzuki

Re: [PATCH 00/14] arm64: 16K translation granule support

[Suzuki K. Poulose]

On 02/09/15 10:55, Ard Biesheuvel wrote:
> On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>

>> Patches 1-7 cleans up the kernel page size handling code.
>> Patches 8-11 Fixes some issues with the KVM bits, mainly the fake PGD
>>               handling code.
>> Patch 12    Adds a check to ensure the CPU supports the selected granule size.
>> Patch 13-14 Adds the 16k page size support bits.
>>
>> This series applies on top of for-next/core branch of the aarch64 tree and is
>> also available here:
>>
>>          git://linux-arm.org/linux-skp.git  16k/v1
>>
>
>
> Hi Suzuki,
>
> I have given this a spin on the FVP Base model to check UEFI booting,
> and everything seems to work fine. (I tested 2-level and 3-level)
> I didn't test the KVM changes, so for all patches except those:
>
> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Thanks for the review and testing !!

Suzuki

[PATCH 00/14] arm64: 16K translation granule support

[Suzuki K. Poulose]

On 02/09/15 10:55, Ard Biesheuvel wrote:
> On 13 August 2015 at 13:33, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>> From: "Suzuki K. Poulose" <suzuki.poulose@arm.com>
>>

>> Patches 1-7 cleans up the kernel page size handling code.
>> Patches 8-11 Fixes some issues with the KVM bits, mainly the fake PGD
>>               handling code.
>> Patch 12    Adds a check to ensure the CPU supports the selected granule size.
>> Patch 13-14 Adds the 16k page size support bits.
>>
>> This series applies on top of for-next/core branch of the aarch64 tree and is
>> also available here:
>>
>>          git://linux-arm.org/linux-skp.git  16k/v1
>>
>
>
> Hi Suzuki,
>
> I have given this a spin on the FVP Base model to check UEFI booting,
> and everything seems to work fine. (I tested 2-level and 3-level)
> I didn't test the KVM changes, so for all patches except those:
>
> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Thanks for the review and testing !!

Suzuki

Re: [PATCH 12/14] arm64: Check for selected granule support

[Ard Biesheuvel]

On 2 September 2015 at 11:48, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 13 August 2015 at 19:29, Catalin Marinas <catalin.marinas@arm.com> wrote:
>> On Thu, Aug 13, 2015 at 03:45:07PM +0100, Suzuki K. Poulose wrote:
>>> On 13/08/15 13:28, Steve Capper wrote:
>>> >On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>>> >>  __enable_mmu:
>>> >>+       mrs     x1, ID_AA64MMFR0_EL1
>>> >>+       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
>>> >>+       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
>>> >>+       b.ne    __no_granule_support
>>> >>         ldr     x5, =vectors
>>> >>         msr     vbar_el1, x5
>>> >>         msr     ttbr0_el1, x25                  // load TTBR0
>>> >>@@ -626,3 +643,8 @@ __enable_mmu:
>>> >>         isb
>>> >>         br      x27
>>> >>  ENDPROC(__enable_mmu)
>>> >>+
>>> >>+__no_granule_support:
>>> >>+       wfe
>>> >>+       b __no_granule_support
>>> >>+ENDPROC(__no_granule_support)
>>> >>--
>>> >>1.7.9.5
>>> >>
>>> >
>>> >Is is possible to tell the user that the kernel has failed to boot due
>>> >to the kernel granule being unsupported?
>>>
>>> We don't have anything up at this time. The "looping address" is actually a clue
>>> to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)
>>
>> No.
>>
>>> Or we should let it continue and end in a panic(?). The current situation can boot a
>>> multi-cluster system with boot cluster having the Tgran support(which doesn't make a
>>> strong use case though). I will try out some options and get back to you.
>>
>> If the boot CPU does not support 16KB pages, in general there isn't much
>> we can do since the console printing is done after we enabled the MMU.
>> Even mapping the UART address requires fixmap support and the PAGE_SIZE
>> is hard-coded in the kernel image. The DT is also mapped at run-time.
>>
>> While in theory it's possible to fall back to a 4KB page size just
>> enough to load the DT and figure out the early console, I suggest we
>> just live with the "looping address" clue.
>>
>
> Couldn't we allocate some flag bits in the Image header to communicate
> the page size to the bootloader?

Something like this perhaps?

------------8<---------------
diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index 7d9d3c2286b2..13a8aaa9a6e9 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -104,7 +104,8 @@ Header notes:
 - The flags field (introduced in v3.17) is a little-endian 64-bit field
   composed as follows:
   Bit 0:       Kernel endianness.  1 if BE, 0 if LE.
-  Bits 1-63:   Reserved.
+  Bits 1-2:    Kernel page size.   0=unspecified, 1=4K, 2=16K, 3=64K
+  Bits 3-63:   Reserved.

 - When image_size is zero, a bootloader should attempt to keep as much
   memory as possible free for use by the kernel immediately after the
diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h
index 8fae0756e175..5def289bda84 100644
--- a/arch/arm64/kernel/image.h
+++ b/arch/arm64/kernel/image.h
@@ -47,7 +47,9 @@
 #define __HEAD_FLAG_BE 0
 #endif

-#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0)
+#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2)
+
+#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0) | (__HEAD_FLAG_PAGE_SIZE << 1)

 /*
  * These will output as part of the Image header, which should be little-endian

Re: [PATCH 12/14] arm64: Check for selected granule support

[Ard Biesheuvel]

On 2 September 2015 at 11:48, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 13 August 2015 at 19:29, Catalin Marinas <catalin.marinas@arm.com> wrote:
>> On Thu, Aug 13, 2015 at 03:45:07PM +0100, Suzuki K. Poulose wrote:
>>> On 13/08/15 13:28, Steve Capper wrote:
>>> >On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>>> >>  __enable_mmu:
>>> >>+       mrs     x1, ID_AA64MMFR0_EL1
>>> >>+       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
>>> >>+       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
>>> >>+       b.ne    __no_granule_support
>>> >>         ldr     x5, =vectors
>>> >>         msr     vbar_el1, x5
>>> >>         msr     ttbr0_el1, x25                  // load TTBR0
>>> >>@@ -626,3 +643,8 @@ __enable_mmu:
>>> >>         isb
>>> >>         br      x27
>>> >>  ENDPROC(__enable_mmu)
>>> >>+
>>> >>+__no_granule_support:
>>> >>+       wfe
>>> >>+       b __no_granule_support
>>> >>+ENDPROC(__no_granule_support)
>>> >>--
>>> >>1.7.9.5
>>> >>
>>> >
>>> >Is is possible to tell the user that the kernel has failed to boot due
>>> >to the kernel granule being unsupported?
>>>
>>> We don't have anything up at this time. The "looping address" is actually a clue
>>> to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)
>>
>> No.
>>
>>> Or we should let it continue and end in a panic(?). The current situation can boot a
>>> multi-cluster system with boot cluster having the Tgran support(which doesn't make a
>>> strong use case though). I will try out some options and get back to you.
>>
>> If the boot CPU does not support 16KB pages, in general there isn't much
>> we can do since the console printing is done after we enabled the MMU.
>> Even mapping the UART address requires fixmap support and the PAGE_SIZE
>> is hard-coded in the kernel image. The DT is also mapped at run-time.
>>
>> While in theory it's possible to fall back to a 4KB page size just
>> enough to load the DT and figure out the early console, I suggest we
>> just live with the "looping address" clue.
>>
>
> Couldn't we allocate some flag bits in the Image header to communicate
> the page size to the bootloader?

Something like this perhaps?

------------8<---------------
diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index 7d9d3c2286b2..13a8aaa9a6e9 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -104,7 +104,8 @@ Header notes:
 - The flags field (introduced in v3.17) is a little-endian 64-bit field
   composed as follows:
   Bit 0:       Kernel endianness.  1 if BE, 0 if LE.
-  Bits 1-63:   Reserved.
+  Bits 1-2:    Kernel page size.   0=unspecified, 1=4K, 2=16K, 3=64K
+  Bits 3-63:   Reserved.

 - When image_size is zero, a bootloader should attempt to keep as much
   memory as possible free for use by the kernel immediately after the
diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h
index 8fae0756e175..5def289bda84 100644
--- a/arch/arm64/kernel/image.h
+++ b/arch/arm64/kernel/image.h
@@ -47,7 +47,9 @@
 #define __HEAD_FLAG_BE 0
 #endif

-#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0)
+#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2)
+
+#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0) | (__HEAD_FLAG_PAGE_SIZE << 1)

 /*
  * These will output as part of the Image header, which should be little-endian

[PATCH 12/14] arm64: Check for selected granule support

[Ard Biesheuvel]

On 2 September 2015 at 11:48, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 13 August 2015 at 19:29, Catalin Marinas <catalin.marinas@arm.com> wrote:
>> On Thu, Aug 13, 2015 at 03:45:07PM +0100, Suzuki K. Poulose wrote:
>>> On 13/08/15 13:28, Steve Capper wrote:
>>> >On 13 August 2015 at 12:34, Suzuki K. Poulose <suzuki.poulose@arm.com> wrote:
>>> >>  __enable_mmu:
>>> >>+       mrs     x1, ID_AA64MMFR0_EL1
>>> >>+       ubfx    x2, x1, #ID_AA64MMFR0_TGran_SHIFT, 4
>>> >>+       cmp     x2, #ID_AA64MMFR0_TGran_ENABLED
>>> >>+       b.ne    __no_granule_support
>>> >>         ldr     x5, =vectors
>>> >>         msr     vbar_el1, x5
>>> >>         msr     ttbr0_el1, x25                  // load TTBR0
>>> >>@@ -626,3 +643,8 @@ __enable_mmu:
>>> >>         isb
>>> >>         br      x27
>>> >>  ENDPROC(__enable_mmu)
>>> >>+
>>> >>+__no_granule_support:
>>> >>+       wfe
>>> >>+       b __no_granule_support
>>> >>+ENDPROC(__no_granule_support)
>>> >>--
>>> >>1.7.9.5
>>> >>
>>> >
>>> >Is is possible to tell the user that the kernel has failed to boot due
>>> >to the kernel granule being unsupported?
>>>
>>> We don't have anything up at this time. The "looping address" is actually a clue
>>> to the (expert) user. Not sure we can do something, until we get something like DEBUG_LL(?)
>>
>> No.
>>
>>> Or we should let it continue and end in a panic(?). The current situation can boot a
>>> multi-cluster system with boot cluster having the Tgran support(which doesn't make a
>>> strong use case though). I will try out some options and get back to you.
>>
>> If the boot CPU does not support 16KB pages, in general there isn't much
>> we can do since the console printing is done after we enabled the MMU.
>> Even mapping the UART address requires fixmap support and the PAGE_SIZE
>> is hard-coded in the kernel image. The DT is also mapped at run-time.
>>
>> While in theory it's possible to fall back to a 4KB page size just
>> enough to load the DT and figure out the early console, I suggest we
>> just live with the "looping address" clue.
>>
>
> Couldn't we allocate some flag bits in the Image header to communicate
> the page size to the bootloader?

Something like this perhaps?

------------8<---------------
diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index 7d9d3c2286b2..13a8aaa9a6e9 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -104,7 +104,8 @@ Header notes:
 - The flags field (introduced in v3.17) is a little-endian 64-bit field
   composed as follows:
   Bit 0:       Kernel endianness.  1 if BE, 0 if LE.
-  Bits 1-63:   Reserved.
+  Bits 1-2:    Kernel page size.   0=unspecified, 1=4K, 2=16K, 3=64K
+  Bits 3-63:   Reserved.

 - When image_size is zero, a bootloader should attempt to keep as much
   memory as possible free for use by the kernel immediately after the
diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h
index 8fae0756e175..5def289bda84 100644
--- a/arch/arm64/kernel/image.h
+++ b/arch/arm64/kernel/image.h
@@ -47,7 +47,9 @@
 #define __HEAD_FLAG_BE 0
 #endif

-#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0)
+#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2)
+
+#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0) | (__HEAD_FLAG_PAGE_SIZE << 1)

 /*
  * These will output as part of the Image header, which should be little-endian

Re: [PATCH 12/14] arm64: Check for selected granule support

[Catalin Marinas]

On Wed, Sep 02, 2015 at 12:19:07PM +0200, Ard Biesheuvel wrote:
> On 2 September 2015 at 11:48, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > Couldn't we allocate some flag bits in the Image header to communicate
> > the page size to the bootloader?
> 
> Something like this perhaps?
> 
> ------------8<---------------
> diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
> index 7d9d3c2286b2..13a8aaa9a6e9 100644
> --- a/Documentation/arm64/booting.txt
> +++ b/Documentation/arm64/booting.txt
> @@ -104,7 +104,8 @@ Header notes:
>  - The flags field (introduced in v3.17) is a little-endian 64-bit field
>    composed as follows:
>    Bit 0:       Kernel endianness.  1 if BE, 0 if LE.
> -  Bits 1-63:   Reserved.
> +  Bits 1-2:    Kernel page size.   0=unspecified, 1=4K, 2=16K, 3=64K
> +  Bits 3-63:   Reserved.
> 
>  - When image_size is zero, a bootloader should attempt to keep as much
>    memory as possible free for use by the kernel immediately after the
> diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h
> index 8fae0756e175..5def289bda84 100644
> --- a/arch/arm64/kernel/image.h
> +++ b/arch/arm64/kernel/image.h
> @@ -47,7 +47,9 @@
>  #define __HEAD_FLAG_BE 0
>  #endif
> 
> -#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0)
> +#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2)
> +
> +#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0) | (__HEAD_FLAG_PAGE_SIZE << 1)

I'm fine with this.

-- 
Catalin

Re: [PATCH 12/14] arm64: Check for selected granule support

[Catalin Marinas]

On Wed, Sep 02, 2015 at 12:19:07PM +0200, Ard Biesheuvel wrote:
> On 2 September 2015 at 11:48, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > Couldn't we allocate some flag bits in the Image header to communicate
> > the page size to the bootloader?
> 
> Something like this perhaps?
> 
> ------------8<---------------
> diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
> index 7d9d3c2286b2..13a8aaa9a6e9 100644
> --- a/Documentation/arm64/booting.txt
> +++ b/Documentation/arm64/booting.txt
> @@ -104,7 +104,8 @@ Header notes:
>  - The flags field (introduced in v3.17) is a little-endian 64-bit field
>    composed as follows:
>    Bit 0:       Kernel endianness.  1 if BE, 0 if LE.
> -  Bits 1-63:   Reserved.
> +  Bits 1-2:    Kernel page size.   0=unspecified, 1=4K, 2=16K, 3=64K
> +  Bits 3-63:   Reserved.
> 
>  - When image_size is zero, a bootloader should attempt to keep as much
>    memory as possible free for use by the kernel immediately after the
> diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h
> index 8fae0756e175..5def289bda84 100644
> --- a/arch/arm64/kernel/image.h
> +++ b/arch/arm64/kernel/image.h
> @@ -47,7 +47,9 @@
>  #define __HEAD_FLAG_BE 0
>  #endif
> 
> -#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0)
> +#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2)
> +
> +#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0) | (__HEAD_FLAG_PAGE_SIZE << 1)

I'm fine with this.

-- 
Catalin

[PATCH 12/14] arm64: Check for selected granule support

[Catalin Marinas]

On Wed, Sep 02, 2015 at 12:19:07PM +0200, Ard Biesheuvel wrote:
> On 2 September 2015 at 11:48, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > Couldn't we allocate some flag bits in the Image header to communicate
> > the page size to the bootloader?
> 
> Something like this perhaps?
> 
> ------------8<---------------
> diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
> index 7d9d3c2286b2..13a8aaa9a6e9 100644
> --- a/Documentation/arm64/booting.txt
> +++ b/Documentation/arm64/booting.txt
> @@ -104,7 +104,8 @@ Header notes:
>  - The flags field (introduced in v3.17) is a little-endian 64-bit field
>    composed as follows:
>    Bit 0:       Kernel endianness.  1 if BE, 0 if LE.
> -  Bits 1-63:   Reserved.
> +  Bits 1-2:    Kernel page size.   0=unspecified, 1=4K, 2=16K, 3=64K
> +  Bits 3-63:   Reserved.
> 
>  - When image_size is zero, a bootloader should attempt to keep as much
>    memory as possible free for use by the kernel immediately after the
> diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h
> index 8fae0756e175..5def289bda84 100644
> --- a/arch/arm64/kernel/image.h
> +++ b/arch/arm64/kernel/image.h
> @@ -47,7 +47,9 @@
>  #define __HEAD_FLAG_BE 0
>  #endif
> 
> -#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0)
> +#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2)
> +
> +#define __HEAD_FLAGS   (__HEAD_FLAG_BE << 0) | (__HEAD_FLAG_PAGE_SIZE << 1)

I'm fine with this.

-- 
Catalin