* [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit
@ 2020-10-08 15:35 Ard Biesheuvel
2020-10-08 15:35 ` [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation Ard Biesheuvel
` (6 more replies)
0 siblings, 7 replies; 26+ messages in thread
From: Ard Biesheuvel @ 2020-10-08 15:35 UTC (permalink / raw)
To: linux-arm-kernel
Cc: catalin.marinas, Anshuman Khandual, will, Ard Biesheuvel, Steve Capper
This series reorganizes the kernel VA space slightly so that 52-bit VA
configurations can use more virtual address space, i.e., the usable
linear address space almost doubles, from 2^51 to 2^52-2^47.
Patch #1 merges the physvirt_offset and memstart_addr variables, both of
which represent translations between the physical address space and the
linear region, and there is no need for having both. This fixes a bug too,
as the two values were not properly kept in sync when booting with KASLR
enabled.
Patch #2 updates the definitions for the boundaries of the linear space,
so that 52-bit VA builds use all available space for the linear region.
Patches #3 and #4 simplify the way the vmemmap region is configured, and
unify the 48-bit/4k and 52-bit/64k layouts.
Changes since v1:
- drop incorrect use of untagged_addr() from #2
- add patches #3 and #4
Not boot tested yet on a 52-bit VA capable system.
Cc: Steve Capper <steve.capper@arm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Ard Biesheuvel (4):
arm64: mm: use single quantity to represent the PA to VA translation
arm64: mm: extend linear region for 52-bit VA configurations
arm64: mm: make vmemmap region a projection of the linear region
arm64: mm: tidy up top of kernel VA space
Documentation/arm64/kasan-offsets.sh | 3 +-
Documentation/arm64/memory.rst | 45 +++++++++-----------
arch/arm64/Kconfig | 20 ++++-----
arch/arm64/include/asm/memory.h | 35 +++++++--------
arch/arm64/include/asm/pgtable.h | 6 +--
arch/arm64/mm/init.c | 34 ++++++---------
6 files changed, 63 insertions(+), 80 deletions(-)
--
2.17.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation
2020-10-08 15:35 [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
@ 2020-10-08 15:35 ` Ard Biesheuvel
2020-10-13 16:14 ` Steve Capper
` (2 more replies)
2020-10-08 15:36 ` [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations Ard Biesheuvel
` (5 subsequent siblings)
6 siblings, 3 replies; 26+ messages in thread
From: Ard Biesheuvel @ 2020-10-08 15:35 UTC (permalink / raw)
To: linux-arm-kernel
Cc: catalin.marinas, Anshuman Khandual, will, Ard Biesheuvel, Steve Capper
On arm64, the global variable memstart_addr represents the physical
address of PAGE_OFFSET, and so physical to virtual translations or
vice versa used to come down to simple additions or subtractions
involving the values of PAGE_OFFSET and memstart_addr.
When support for 52-bit virtual addressing was introduced, we had to
deal with PAGE_OFFSET potentially being outside of the region that
can be covered by the virtual range (as the 52-bit VA capable build
needs to be able to run on systems that are only 48-bit VA capable),
and for this reason, another translation was introduced, and recorded
in the global variable physvirt_offset.
However, if we go back to the original definition of memstart_addr,
i.e., the physical address of PAGE_OFFSET, it turns out that there is
no need for two separate translations: instead, we can simply subtract
the size of the unaddressable VA space from memstart_addr to make the
available physical memory appear in the 48-bit addressable VA region.
This simplifies things, but also fixes a bug on KASLR builds, which
may update memstart_addr later on in arm64_memblock_init(), but fails
to update vmemmap and physvirt_offset accordingly.
Fixes: 5383cc6efed13 ("arm64: mm: Introduce vabits_actual")
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/arm64/include/asm/memory.h | 5 ++--
arch/arm64/include/asm/pgtable.h | 4 +--
arch/arm64/mm/init.c | 30 +++++++-------------
3 files changed, 14 insertions(+), 25 deletions(-)
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index afa722504bfd..1ded73189874 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -164,7 +164,6 @@
extern u64 vabits_actual;
#define PAGE_END (_PAGE_END(vabits_actual))
-extern s64 physvirt_offset;
extern s64 memstart_addr;
/* PHYS_OFFSET - the physical address of the start of memory. */
#define PHYS_OFFSET ({ VM_BUG_ON(memstart_addr & 1); memstart_addr; })
@@ -240,7 +239,7 @@ static inline const void *__tag_set(const void *addr, u8 tag)
*/
#define __is_lm_address(addr) (!(((u64)addr) & BIT(vabits_actual - 1)))
-#define __lm_to_phys(addr) (((addr) + physvirt_offset))
+#define __lm_to_phys(addr) (((addr) & ~PAGE_OFFSET) + PHYS_OFFSET)
#define __kimg_to_phys(addr) ((addr) - kimage_voffset)
#define __virt_to_phys_nodebug(x) ({ \
@@ -258,7 +257,7 @@ extern phys_addr_t __phys_addr_symbol(unsigned long x);
#define __phys_addr_symbol(x) __pa_symbol_nodebug(x)
#endif /* CONFIG_DEBUG_VIRTUAL */
-#define __phys_to_virt(x) ((unsigned long)((x) - physvirt_offset))
+#define __phys_to_virt(x) ((unsigned long)((x) - PHYS_OFFSET) | PAGE_OFFSET)
#define __phys_to_kimg(x) ((unsigned long)((x) + kimage_voffset))
/*
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index d5d3fbe73953..88233d42d9c2 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -23,6 +23,8 @@
#define VMALLOC_START (MODULES_END)
#define VMALLOC_END (- PUD_SIZE - VMEMMAP_SIZE - SZ_64K)
+#define vmemmap ((struct page *)VMEMMAP_START - (memstart_addr >> PAGE_SHIFT))
+
#define FIRST_USER_ADDRESS 0UL
#ifndef __ASSEMBLY__
@@ -33,8 +35,6 @@
#include <linux/mm_types.h>
#include <linux/sched.h>
-extern struct page *vmemmap;
-
extern void __pte_error(const char *file, int line, unsigned long val);
extern void __pmd_error(const char *file, int line, unsigned long val);
extern void __pud_error(const char *file, int line, unsigned long val);
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 481d22c32a2e..324f0e0894f6 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -54,12 +54,6 @@
s64 memstart_addr __ro_after_init = -1;
EXPORT_SYMBOL(memstart_addr);
-s64 physvirt_offset __ro_after_init;
-EXPORT_SYMBOL(physvirt_offset);
-
-struct page *vmemmap __ro_after_init;
-EXPORT_SYMBOL(vmemmap);
-
/*
* We create both ZONE_DMA and ZONE_DMA32. ZONE_DMA covers the first 1G of
* memory as some devices, namely the Raspberry Pi 4, have peripherals with
@@ -290,20 +284,6 @@ void __init arm64_memblock_init(void)
memstart_addr = round_down(memblock_start_of_DRAM(),
ARM64_MEMSTART_ALIGN);
- physvirt_offset = PHYS_OFFSET - PAGE_OFFSET;
-
- vmemmap = ((struct page *)VMEMMAP_START - (memstart_addr >> PAGE_SHIFT));
-
- /*
- * If we are running with a 52-bit kernel VA config on a system that
- * does not support it, we have to offset our vmemmap and physvirt_offset
- * s.t. we avoid the 52-bit portion of the direct linear map
- */
- if (IS_ENABLED(CONFIG_ARM64_VA_BITS_52) && (vabits_actual != 52)) {
- vmemmap += (_PAGE_OFFSET(48) - _PAGE_OFFSET(52)) >> PAGE_SHIFT;
- physvirt_offset = PHYS_OFFSET - _PAGE_OFFSET(48);
- }
-
/*
* Remove the memory that we will not be able to cover with the
* linear mapping. Take care not to clip the kernel which may be
@@ -318,6 +298,16 @@ void __init arm64_memblock_init(void)
memblock_remove(0, memstart_addr);
}
+ /*
+ * If we are running with a 52-bit kernel VA config on a system that
+ * does not support it, we have to place the available physical
+ * memory in the 48-bit addressable part of the linear region, i.e.,
+ * we have to move it upward. Since memstart_addr represents the
+ * physical address of PAGE_OFFSET, we have to *subtract* from it.
+ */
+ if (IS_ENABLED(CONFIG_ARM64_VA_BITS_52) && (vabits_actual != 52))
+ memstart_addr -= _PAGE_OFFSET(48) - _PAGE_OFFSET(52);
+
/*
* Apply the memory limit if it was set. Since the kernel may be loaded
* high up in memory, add back the kernel region that must be accessible
--
2.17.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations
2020-10-08 15:35 [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
2020-10-08 15:35 ` [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation Ard Biesheuvel
@ 2020-10-08 15:36 ` Ard Biesheuvel
2020-10-13 16:51 ` Steve Capper
2020-10-14 3:44 ` Anshuman Khandual
2020-10-08 15:36 ` [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region Ard Biesheuvel
` (4 subsequent siblings)
6 siblings, 2 replies; 26+ messages in thread
From: Ard Biesheuvel @ 2020-10-08 15:36 UTC (permalink / raw)
To: linux-arm-kernel
Cc: catalin.marinas, Anshuman Khandual, will, Ard Biesheuvel, Steve Capper
For historical reasons, the arm64 kernel VA space is configured as two
equally sized halves, i.e., on a 48-bit VA build, the VA space is split
into a 47-bit vmalloc region and a 47-bit linear region.
When support for 52-bit virtual addressing was added, this equal split
was kept, resulting in a substantial waste of virtual address space in
the linear region:
48-bit VA 52-bit VA
0xffff_ffff_ffff_ffff +-------------+ +-------------+
| vmalloc | | vmalloc |
0xffff_8000_0000_0000 +-------------+ _PAGE_END(48) +-------------+
| linear | : :
0xffff_0000_0000_0000 +-------------+ : :
: : : :
: : : :
: : : :
: : : currently :
: unusable : : :
: : : unused :
: by : : :
: : : :
: hardware : : :
: : : :
0xfff8_0000_0000_0000 : : _PAGE_END(52) +-------------+
: : | |
: : | |
: : | |
: : | |
: : | |
: unusable : | |
: : | linear |
: by : | |
: : | region |
: hardware : | |
: : | |
: : | |
: : | |
: : | |
: : | |
: : | |
0xfff0_0000_0000_0000 +-------------+ PAGE_OFFSET +-------------+
As illustrated above, the 52-bit VA kernel uses 47 bits for the vmalloc
space (as before), to ensure that a single 64k granule kernel image can
support any 64k granule capable system, regardless of whether it supports
the 52-bit virtual addressing extension. However, due to the fact that
the VA space is still split in equal halves, the linear region is only
2^51 bytes in size, wasting almost half of the 52-bit VA space.
Let's fix this, by abandoning the equal split, and simply assigning all
VA space outside of the vmalloc region to the linear region.
The KASAN shadow region is reconfigured so that it ends at the start of
the vmalloc region, and grows downwards. That way, the arrangement of
the vmalloc space (which contains kernel mappings, modules, BPF region,
the vmemmap array etc) is identical between non-KASAN and KASAN builds,
which aids debugging.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
Documentation/arm64/kasan-offsets.sh | 3 +--
Documentation/arm64/memory.rst | 19 +++++++++----------
arch/arm64/Kconfig | 20 ++++++++++----------
arch/arm64/include/asm/memory.h | 12 +++++-------
arch/arm64/mm/init.c | 2 +-
5 files changed, 26 insertions(+), 30 deletions(-)
diff --git a/Documentation/arm64/kasan-offsets.sh b/Documentation/arm64/kasan-offsets.sh
index 2b7a021db363..2dc5f9e18039 100644
--- a/Documentation/arm64/kasan-offsets.sh
+++ b/Documentation/arm64/kasan-offsets.sh
@@ -1,12 +1,11 @@
#!/bin/sh
# Print out the KASAN_SHADOW_OFFSETS required to place the KASAN SHADOW
-# start address at the mid-point of the kernel VA space
+# start address at the top of the linear region
print_kasan_offset () {
printf "%02d\t" $1
printf "0x%08x00000000\n" $(( (0xffffffff & (-1 << ($1 - 1 - 32))) \
- + (1 << ($1 - 32 - $2)) \
- (1 << (64 - 32 - $2)) ))
}
diff --git a/Documentation/arm64/memory.rst b/Documentation/arm64/memory.rst
index cf03b3290800..ee51eb66a578 100644
--- a/Documentation/arm64/memory.rst
+++ b/Documentation/arm64/memory.rst
@@ -32,10 +32,10 @@ AArch64 Linux memory layout with 4KB pages + 4 levels (48-bit)::
-----------------------------------------------------------------------
0000000000000000 0000ffffffffffff 256TB user
ffff000000000000 ffff7fffffffffff 128TB kernel logical memory map
- ffff800000000000 ffff9fffffffffff 32TB kasan shadow region
- ffffa00000000000 ffffa00007ffffff 128MB bpf jit region
- ffffa00008000000 ffffa0000fffffff 128MB modules
- ffffa00010000000 fffffdffbffeffff ~93TB vmalloc
+[ ffff600000000000 ffff7fffffffffff ] 32TB [ kasan shadow region ]
+ ffff800000000000 ffff800007ffffff 128MB bpf jit region
+ ffff800008000000 ffff80000fffffff 128MB modules
+ ffff800010000000 fffffdffbffeffff 125TB vmalloc
fffffdffbfff0000 fffffdfffe5f8fff ~998MB [guard region]
fffffdfffe5f9000 fffffdfffe9fffff 4124KB fixed mappings
fffffdfffea00000 fffffdfffebfffff 2MB [guard region]
@@ -50,12 +50,11 @@ AArch64 Linux memory layout with 64KB pages + 3 levels (52-bit with HW support):
Start End Size Use
-----------------------------------------------------------------------
0000000000000000 000fffffffffffff 4PB user
- fff0000000000000 fff7ffffffffffff 2PB kernel logical memory map
- fff8000000000000 fffd9fffffffffff 1440TB [gap]
- fffda00000000000 ffff9fffffffffff 512TB kasan shadow region
- ffffa00000000000 ffffa00007ffffff 128MB bpf jit region
- ffffa00008000000 ffffa0000fffffff 128MB modules
- ffffa00010000000 fffff81ffffeffff ~88TB vmalloc
+ fff0000000000000 ffff7fffffffffff ~4PB kernel logical memory map
+[ fffd800000000000 ffff7fffffffffff ] 512TB [ kasan shadow region ]
+ ffff800000000000 ffff800007ffffff 128MB bpf jit region
+ ffff800008000000 ffff80000fffffff 128MB modules
+ ffff800010000000 fffff81ffffeffff 120TB vmalloc
fffff81fffff0000 fffffc1ffe58ffff ~3TB [guard region]
fffffc1ffe590000 fffffc1ffe9fffff 4544KB fixed mappings
fffffc1ffea00000 fffffc1ffebfffff 2MB [guard region]
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 6d232837cbee..896a46a71d23 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -321,16 +321,16 @@ config BROKEN_GAS_INST
config KASAN_SHADOW_OFFSET
hex
depends on KASAN
- default 0xdfffa00000000000 if (ARM64_VA_BITS_48 || ARM64_VA_BITS_52) && !KASAN_SW_TAGS
- default 0xdfffd00000000000 if ARM64_VA_BITS_47 && !KASAN_SW_TAGS
- default 0xdffffe8000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS
- default 0xdfffffd000000000 if ARM64_VA_BITS_39 && !KASAN_SW_TAGS
- default 0xdffffffa00000000 if ARM64_VA_BITS_36 && !KASAN_SW_TAGS
- default 0xefff900000000000 if (ARM64_VA_BITS_48 || ARM64_VA_BITS_52) && KASAN_SW_TAGS
- default 0xefffc80000000000 if ARM64_VA_BITS_47 && KASAN_SW_TAGS
- default 0xeffffe4000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS
- default 0xefffffc800000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS
- default 0xeffffff900000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS
+ default 0xdfff800000000000 if (ARM64_VA_BITS_48 || ARM64_VA_BITS_52) && !KASAN_SW_TAGS
+ default 0xdfffc00000000000 if ARM64_VA_BITS_47 && !KASAN_SW_TAGS
+ default 0xdffffe0000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS
+ default 0xdfffffc000000000 if ARM64_VA_BITS_39 && !KASAN_SW_TAGS
+ default 0xdffffff800000000 if ARM64_VA_BITS_36 && !KASAN_SW_TAGS
+ default 0xefff800000000000 if (ARM64_VA_BITS_48 || ARM64_VA_BITS_52) && KASAN_SW_TAGS
+ default 0xefffc00000000000 if ARM64_VA_BITS_47 && KASAN_SW_TAGS
+ default 0xeffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS
+ default 0xefffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS
+ default 0xeffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS
default 0xffffffffffffffff
source "arch/arm64/Kconfig.platforms"
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 1ded73189874..58932f434433 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -44,7 +44,7 @@
#define _PAGE_OFFSET(va) (-(UL(1) << (va)))
#define PAGE_OFFSET (_PAGE_OFFSET(VA_BITS))
#define KIMAGE_VADDR (MODULES_END)
-#define BPF_JIT_REGION_START (KASAN_SHADOW_END)
+#define BPF_JIT_REGION_START (_PAGE_END(VA_BITS_MIN))
#define BPF_JIT_REGION_SIZE (SZ_128M)
#define BPF_JIT_REGION_END (BPF_JIT_REGION_START + BPF_JIT_REGION_SIZE)
#define MODULES_END (MODULES_VADDR + MODULES_VSIZE)
@@ -76,10 +76,11 @@
#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)
#define KASAN_SHADOW_END ((UL(1) << (64 - KASAN_SHADOW_SCALE_SHIFT)) \
+ KASAN_SHADOW_OFFSET)
+#define PAGE_END (KASAN_SHADOW_END - (1UL << (vabits_actual - KASAN_SHADOW_SCALE_SHIFT)))
#define KASAN_THREAD_SHIFT 1
#else
#define KASAN_THREAD_SHIFT 0
-#define KASAN_SHADOW_END (_PAGE_END(VA_BITS_MIN))
+#define PAGE_END (_PAGE_END(VA_BITS_MIN))
#endif /* CONFIG_KASAN */
#define MIN_THREAD_SHIFT (14 + KASAN_THREAD_SHIFT)
@@ -162,7 +163,6 @@
#include <asm/bug.h>
extern u64 vabits_actual;
-#define PAGE_END (_PAGE_END(vabits_actual))
extern s64 memstart_addr;
/* PHYS_OFFSET - the physical address of the start of memory. */
@@ -233,11 +233,9 @@ static inline const void *__tag_set(const void *addr, u8 tag)
/*
- * The linear kernel range starts at the bottom of the virtual address
- * space. Testing the top bit for the start of the region is a
- * sufficient check and avoids having to worry about the tag.
+ * The linear kernel range starts at the bottom of the virtual address space.
*/
-#define __is_lm_address(addr) (!(((u64)addr) & BIT(vabits_actual - 1)))
+#define __is_lm_address(addr) (((u64)(addr) & ~PAGE_OFFSET) < (PAGE_END - PAGE_OFFSET))
#define __lm_to_phys(addr) (((addr) & ~PAGE_OFFSET) + PHYS_OFFSET)
#define __kimg_to_phys(addr) ((addr) - kimage_voffset)
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 324f0e0894f6..9090779dd3cd 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -270,7 +270,7 @@ static void __init fdt_enforce_memory_region(void)
void __init arm64_memblock_init(void)
{
- const s64 linear_region_size = BIT(vabits_actual - 1);
+ const s64 linear_region_size = PAGE_END - _PAGE_OFFSET(vabits_actual);
/* Handle linux,usable-memory-range property */
fdt_enforce_memory_region();
--
2.17.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region
2020-10-08 15:35 [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
2020-10-08 15:35 ` [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation Ard Biesheuvel
2020-10-08 15:36 ` [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations Ard Biesheuvel
@ 2020-10-08 15:36 ` Ard Biesheuvel
2020-10-13 16:52 ` Steve Capper
2020-11-10 12:55 ` Geert Uytterhoeven
2020-10-08 15:36 ` [PATCH v2 4/4] arm64: mm: tidy up top of kernel VA space Ard Biesheuvel
` (3 subsequent siblings)
6 siblings, 2 replies; 26+ messages in thread
From: Ard Biesheuvel @ 2020-10-08 15:36 UTC (permalink / raw)
To: linux-arm-kernel
Cc: catalin.marinas, Anshuman Khandual, will, Ard Biesheuvel, Steve Capper
Now that we have reverted the introduction of the vmemmap struct page
pointer and the separate physvirt_offset, we can simplify things further,
and place the vmemmap region in the VA space in such a way that virtual
to page translations and vice versa can be implemented using a single
arithmetic shift.
One happy coincidence resulting from this is that the 48-bit/4k and
52-bit/64k configurations (which are assumed to be the two most
prevalent) end up with the same placement of the vmemmap region. In
a subsequent patch, we will take advantage of this, and unify the
memory maps even more.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
Documentation/arm64/memory.rst | 30 ++++++++++----------
arch/arm64/include/asm/memory.h | 14 ++++-----
arch/arm64/mm/init.c | 2 ++
3 files changed, 23 insertions(+), 23 deletions(-)
diff --git a/Documentation/arm64/memory.rst b/Documentation/arm64/memory.rst
index ee51eb66a578..476edb6015b2 100644
--- a/Documentation/arm64/memory.rst
+++ b/Documentation/arm64/memory.rst
@@ -35,14 +35,14 @@ AArch64 Linux memory layout with 4KB pages + 4 levels (48-bit)::
[ ffff600000000000 ffff7fffffffffff ] 32TB [ kasan shadow region ]
ffff800000000000 ffff800007ffffff 128MB bpf jit region
ffff800008000000 ffff80000fffffff 128MB modules
- ffff800010000000 fffffdffbffeffff 125TB vmalloc
- fffffdffbfff0000 fffffdfffe5f8fff ~998MB [guard region]
- fffffdfffe5f9000 fffffdfffe9fffff 4124KB fixed mappings
- fffffdfffea00000 fffffdfffebfffff 2MB [guard region]
- fffffdfffec00000 fffffdffffbfffff 16MB PCI I/O space
- fffffdffffc00000 fffffdffffdfffff 2MB [guard region]
- fffffdffffe00000 ffffffffffdfffff 2TB vmemmap
- ffffffffffe00000 ffffffffffffffff 2MB [guard region]
+ ffff800010000000 fffffbffbffeffff 123TB vmalloc
+ fffffbffbfff0000 fffffbfffe7f8fff ~998MB [guard region]
+ fffffbfffe7f9000 fffffbfffebfffff 4124KB fixed mappings
+ fffffbfffec00000 fffffbfffedfffff 2MB [guard region]
+ fffffbfffee00000 fffffbffffdfffff 16MB PCI I/O space
+ fffffbffffe00000 fffffbffffffffff 2MB [guard region]
+ fffffc0000000000 fffffdffffffffff 2TB vmemmap
+ fffffe0000000000 ffffffffffffffff 2TB [guard region]
AArch64 Linux memory layout with 64KB pages + 3 levels (52-bit with HW support)::
@@ -55,13 +55,13 @@ AArch64 Linux memory layout with 64KB pages + 3 levels (52-bit with HW support):
ffff800000000000 ffff800007ffffff 128MB bpf jit region
ffff800008000000 ffff80000fffffff 128MB modules
ffff800010000000 fffff81ffffeffff 120TB vmalloc
- fffff81fffff0000 fffffc1ffe58ffff ~3TB [guard region]
- fffffc1ffe590000 fffffc1ffe9fffff 4544KB fixed mappings
- fffffc1ffea00000 fffffc1ffebfffff 2MB [guard region]
- fffffc1ffec00000 fffffc1fffbfffff 16MB PCI I/O space
- fffffc1fffc00000 fffffc1fffdfffff 2MB [guard region]
- fffffc1fffe00000 ffffffffffdfffff 3968GB vmemmap
- ffffffffffe00000 ffffffffffffffff 2MB [guard region]
+ fffff81fffff0000 fffffbfffe38ffff ~3TB [guard region]
+ fffffbfffe390000 fffffbfffebfffff 4544KB fixed mappings
+ fffffbfffec00000 fffffbfffedfffff 2MB [guard region]
+ fffffbfffee00000 fffffbffffdfffff 16MB PCI I/O space
+ fffffbffffe00000 fffffbffffffffff 2MB [guard region]
+ fffffc0000000000 ffffffdfffffffff ~4TB vmemmap
+ ffffffe000000000 ffffffffffffffff 128GB [guard region]
Translation table lookup with 4KB pages::
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 58932f434433..39ea35f7e34e 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -30,8 +30,8 @@
* keep a constant PAGE_OFFSET and "fallback" to using the higher end
* of the VMEMMAP where 52-bit support is not available in hardware.
*/
-#define VMEMMAP_SIZE ((_PAGE_END(VA_BITS_MIN) - PAGE_OFFSET) \
- >> (PAGE_SHIFT - STRUCT_PAGE_MAX_SHIFT))
+#define VMEMMAP_SHIFT (PAGE_SHIFT - STRUCT_PAGE_MAX_SHIFT)
+#define VMEMMAP_SIZE ((_PAGE_END(VA_BITS_MIN) - PAGE_OFFSET) >> VMEMMAP_SHIFT)
/*
* PAGE_OFFSET - the virtual address of the start of the linear map, at the
@@ -50,7 +50,7 @@
#define MODULES_END (MODULES_VADDR + MODULES_VSIZE)
#define MODULES_VADDR (BPF_JIT_REGION_END)
#define MODULES_VSIZE (SZ_128M)
-#define VMEMMAP_START (-VMEMMAP_SIZE - SZ_2M)
+#define VMEMMAP_START (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT)))
#define VMEMMAP_END (VMEMMAP_START + VMEMMAP_SIZE)
#define PCI_IO_END (VMEMMAP_START - SZ_2M)
#define PCI_IO_START (PCI_IO_END - PCI_IO_SIZE)
@@ -303,15 +303,13 @@ static inline void *phys_to_virt(phys_addr_t x)
#else
#define page_to_virt(x) ({ \
__typeof__(x) __page = x; \
- u64 __idx = ((u64)__page - VMEMMAP_START) / sizeof(struct page);\
- u64 __addr = PAGE_OFFSET + (__idx * PAGE_SIZE); \
+ u64 __addr = (u64)__page << VMEMMAP_SHIFT; \
(void *)__tag_set((const void *)__addr, page_kasan_tag(__page));\
})
#define virt_to_page(x) ({ \
- u64 __idx = (__tag_reset((u64)x) - PAGE_OFFSET) / PAGE_SIZE; \
- u64 __addr = VMEMMAP_START + (__idx * sizeof(struct page)); \
- (struct page *)__addr; \
+ u64 __addr = __tag_reset((u64)(x)) & PAGE_MASK; \
+ (struct page *)((s64)__addr >> VMEMMAP_SHIFT); \
})
#endif /* !CONFIG_SPARSEMEM_VMEMMAP || CONFIG_DEBUG_VIRTUAL */
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 9090779dd3cd..f0599ae73b8d 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -504,6 +504,8 @@ static void __init free_unused_memmap(void)
*/
void __init mem_init(void)
{
+ BUILD_BUG_ON(!is_power_of_2(sizeof(struct page)));
+
if (swiotlb_force == SWIOTLB_FORCE ||
max_pfn > PFN_DOWN(arm64_dma_phys_limit ? : arm64_dma32_phys_limit))
swiotlb_init(1);
--
2.17.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH v2 4/4] arm64: mm: tidy up top of kernel VA space
2020-10-08 15:35 [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
` (2 preceding siblings ...)
2020-10-08 15:36 ` [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region Ard Biesheuvel
@ 2020-10-08 15:36 ` Ard Biesheuvel
2020-10-13 16:52 ` Steve Capper
2020-10-09 14:16 ` [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
` (2 subsequent siblings)
6 siblings, 1 reply; 26+ messages in thread
From: Ard Biesheuvel @ 2020-10-08 15:36 UTC (permalink / raw)
To: linux-arm-kernel
Cc: catalin.marinas, Anshuman Khandual, will, Ard Biesheuvel, Steve Capper
Tidy up the way the top of the kernel VA space is organized, by mirroring
the 256 MB region we have below the vmalloc space, and populating it top
down with the PCI I/O space, some guard regions, and the fixmap region.
The latter region is itself populated top down, and today only covers
about 4 MB, and so 224 MB is ample, and no guard region is therefore
required.
The resulting layout is identical between 48-bit/4k and 52-bit/64k
configurations.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
Documentation/arm64/memory.rst | 22 +++++++++-----------
arch/arm64/include/asm/memory.h | 4 ++--
arch/arm64/include/asm/pgtable.h | 2 +-
3 files changed, 13 insertions(+), 15 deletions(-)
diff --git a/Documentation/arm64/memory.rst b/Documentation/arm64/memory.rst
index 476edb6015b2..f4d24753e0e7 100644
--- a/Documentation/arm64/memory.rst
+++ b/Documentation/arm64/memory.rst
@@ -35,12 +35,11 @@ AArch64 Linux memory layout with 4KB pages + 4 levels (48-bit)::
[ ffff600000000000 ffff7fffffffffff ] 32TB [ kasan shadow region ]
ffff800000000000 ffff800007ffffff 128MB bpf jit region
ffff800008000000 ffff80000fffffff 128MB modules
- ffff800010000000 fffffbffbffeffff 123TB vmalloc
- fffffbffbfff0000 fffffbfffe7f8fff ~998MB [guard region]
- fffffbfffe7f9000 fffffbfffebfffff 4124KB fixed mappings
- fffffbfffec00000 fffffbfffedfffff 2MB [guard region]
- fffffbfffee00000 fffffbffffdfffff 16MB PCI I/O space
- fffffbffffe00000 fffffbffffffffff 2MB [guard region]
+ ffff800010000000 fffffbffefffffff 124TB vmalloc
+ fffffbfff0000000 fffffbfffdffffff 224MB fixed mappings (top down)
+ fffffbfffe000000 fffffbfffe7fffff 8MB [guard region]
+ fffffbfffe800000 fffffbffff7fffff 16MB PCI I/O space
+ fffffbffff800000 fffffbffffffffff 8MB [guard region]
fffffc0000000000 fffffdffffffffff 2TB vmemmap
fffffe0000000000 ffffffffffffffff 2TB [guard region]
@@ -54,12 +53,11 @@ AArch64 Linux memory layout with 64KB pages + 3 levels (52-bit with HW support):
[ fffd800000000000 ffff7fffffffffff ] 512TB [ kasan shadow region ]
ffff800000000000 ffff800007ffffff 128MB bpf jit region
ffff800008000000 ffff80000fffffff 128MB modules
- ffff800010000000 fffff81ffffeffff 120TB vmalloc
- fffff81fffff0000 fffffbfffe38ffff ~3TB [guard region]
- fffffbfffe390000 fffffbfffebfffff 4544KB fixed mappings
- fffffbfffec00000 fffffbfffedfffff 2MB [guard region]
- fffffbfffee00000 fffffbffffdfffff 16MB PCI I/O space
- fffffbffffe00000 fffffbffffffffff 2MB [guard region]
+ ffff800010000000 fffffbffefffffff 124TB vmalloc
+ fffffbfff0000000 fffffbfffdffffff 224MB fixed mappings (top down)
+ fffffbfffe000000 fffffbfffe7fffff 8MB [guard region]
+ fffffbfffe800000 fffffbffff7fffff 16MB PCI I/O space
+ fffffbffff800000 fffffbffffffffff 8MB [guard region]
fffffc0000000000 ffffffdfffffffff ~4TB vmemmap
ffffffe000000000 ffffffffffffffff 128GB [guard region]
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 39ea35f7e34e..3a32bc9136eb 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -52,9 +52,9 @@
#define MODULES_VSIZE (SZ_128M)
#define VMEMMAP_START (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT)))
#define VMEMMAP_END (VMEMMAP_START + VMEMMAP_SIZE)
-#define PCI_IO_END (VMEMMAP_START - SZ_2M)
+#define PCI_IO_END (VMEMMAP_START - SZ_8M)
#define PCI_IO_START (PCI_IO_END - PCI_IO_SIZE)
-#define FIXADDR_TOP (PCI_IO_START - SZ_2M)
+#define FIXADDR_TOP (VMEMMAP_START - SZ_32M)
#if VA_BITS > 48
#define VA_BITS_MIN (48)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 88233d42d9c2..5a3bf92c85e7 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -21,7 +21,7 @@
* and fixed mappings
*/
#define VMALLOC_START (MODULES_END)
-#define VMALLOC_END (- PUD_SIZE - VMEMMAP_SIZE - SZ_64K)
+#define VMALLOC_END (VMEMMAP_START - SZ_256M)
#define vmemmap ((struct page *)VMEMMAP_START - (memstart_addr >> PAGE_SHIFT))
--
2.17.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit
2020-10-08 15:35 [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
` (3 preceding siblings ...)
2020-10-08 15:36 ` [PATCH v2 4/4] arm64: mm: tidy up top of kernel VA space Ard Biesheuvel
@ 2020-10-09 14:16 ` Ard Biesheuvel
2020-10-15 20:40 ` Will Deacon
2020-11-09 18:51 ` Catalin Marinas
6 siblings, 0 replies; 26+ messages in thread
From: Ard Biesheuvel @ 2020-10-09 14:16 UTC (permalink / raw)
To: Linux ARM; +Cc: Catalin Marinas, Anshuman Khandual, Will Deacon, Steve Capper
On Thu, 8 Oct 2020 at 17:36, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> This series reorganizes the kernel VA space slightly so that 52-bit VA
> configurations can use more virtual address space, i.e., the usable
> linear address space almost doubles, from 2^51 to 2^52-2^47.
>
> Patch #1 merges the physvirt_offset and memstart_addr variables, both of
> which represent translations between the physical address space and the
> linear region, and there is no need for having both. This fixes a bug too,
> as the two values were not properly kept in sync when booting with KASLR
> enabled.
>
> Patch #2 updates the definitions for the boundaries of the linear space,
> so that 52-bit VA builds use all available space for the linear region.
>
> Patches #3 and #4 simplify the way the vmemmap region is configured, and
> unify the 48-bit/4k and 52-bit/64k layouts.
>
> Changes since v1:
> - drop incorrect use of untagged_addr() from #2
> - add patches #3 and #4
>
> Not boot tested yet on a 52-bit VA capable system.
>
Boot tested now on FVP Base with and without LVA support enabled.
> Cc: Steve Capper <steve.capper@arm.com>
> Cc: Anshuman Khandual <anshuman.khandual@arm.com>
>
> Ard Biesheuvel (4):
> arm64: mm: use single quantity to represent the PA to VA translation
> arm64: mm: extend linear region for 52-bit VA configurations
> arm64: mm: make vmemmap region a projection of the linear region
> arm64: mm: tidy up top of kernel VA space
>
> Documentation/arm64/kasan-offsets.sh | 3 +-
> Documentation/arm64/memory.rst | 45 +++++++++-----------
> arch/arm64/Kconfig | 20 ++++-----
> arch/arm64/include/asm/memory.h | 35 +++++++--------
> arch/arm64/include/asm/pgtable.h | 6 +--
> arch/arm64/mm/init.c | 34 ++++++---------
> 6 files changed, 63 insertions(+), 80 deletions(-)
>
> --
> 2.17.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation
2020-10-08 15:35 ` [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation Ard Biesheuvel
@ 2020-10-13 16:14 ` Steve Capper
2020-10-13 16:47 ` Steve Capper
2020-10-15 10:47 ` Will Deacon
2 siblings, 0 replies; 26+ messages in thread
From: Steve Capper @ 2020-10-13 16:14 UTC (permalink / raw)
To: Ard Biesheuvel, linux-arm-kernel
Cc: catalin.marinas, nd, will, Anshuman Khandual
Hi Ard,
On 08/10/2020 16:35, Ard Biesheuvel wrote:
> On arm64, the global variable memstart_addr represents the physical
> address of PAGE_OFFSET, and so physical to virtual translations or
> vice versa used to come down to simple additions or subtractions
> involving the values of PAGE_OFFSET and memstart_addr.
>
> When support for 52-bit virtual addressing was introduced, we had to
> deal with PAGE_OFFSET potentially being outside of the region that
> can be covered by the virtual range (as the 52-bit VA capable build
> needs to be able to run on systems that are only 48-bit VA capable),
> and for this reason, another translation was introduced, and recorded
> in the global variable physvirt_offset.
>
> However, if we go back to the original definition of memstart_addr,
> i.e., the physical address of PAGE_OFFSET, it turns out that there is
> no need for two separate translations: instead, we can simply subtract
> the size of the unaddressable VA space from memstart_addr to make the
> available physical memory appear in the 48-bit addressable VA region.
>
> This simplifies things, but also fixes a bug on KASLR builds, which
> may update memstart_addr later on in arm64_memblock_init(), but fails
> to update vmemmap and physvirt_offset accordingly.
>
Apologies, I didn't spot that before.
> Fixes: 5383cc6efed13 ("arm64: mm: Introduce vabits_actual")
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Thanks for this. It is much better to modify memstart_addr rather than
introducing needless complexity.
Reviewed-by: Steve Capper <steve.capper@arm.com>
Cheers,
--
Steve
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* RE: [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation
2020-10-08 15:35 ` [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation Ard Biesheuvel
2020-10-13 16:14 ` Steve Capper
@ 2020-10-13 16:47 ` Steve Capper
2020-10-15 10:47 ` Will Deacon
2 siblings, 0 replies; 26+ messages in thread
From: Steve Capper @ 2020-10-13 16:47 UTC (permalink / raw)
To: Ard Biesheuvel, linux-arm-kernel
Cc: Catalin Marinas, nd, will, Anshuman Khandual
Hi Ard,
On 08/10/2020 16:35, Ard Biesheuvel wrote:
> On arm64, the global variable memstart_addr represents the physical
> address of PAGE_OFFSET, and so physical to virtual translations or
> vice versa used to come down to simple additions or subtractions
> involving the values of PAGE_OFFSET and memstart_addr.
>
> When support for 52-bit virtual addressing was introduced, we had to
> deal with PAGE_OFFSET potentially being outside of the region that can
> be covered by the virtual range (as the 52-bit VA capable build needs
> to be able to run on systems that are only 48-bit VA capable), and for
> this reason, another translation was introduced, and recorded in the
> global variable physvirt_offset.
>
> However, if we go back to the original definition of memstart_addr,
> i.e., the physical address of PAGE_OFFSET, it turns out that there is
> no need for two separate translations: instead, we can simply subtract
> the size of the unaddressable VA space from memstart_addr to make the
> available physical memory appear in the 48-bit addressable VA region.
>
> This simplifies things, but also fixes a bug on KASLR builds, which
> may update memstart_addr later on in arm64_memblock_init(), but fails
> to update vmemmap and physvirt_offset accordingly.
>
Apologies, I didn't spot that before.
> Fixes: 5383cc6efed13 ("arm64: mm: Introduce vabits_actual")
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Thanks for this. It is much better to modify memstart_addr rather than introducing needless complexity.
Reviewed-by: Steve Capper <steve.capper@arm.com>
Cheers,
--
Steve
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations
2020-10-08 15:36 ` [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations Ard Biesheuvel
@ 2020-10-13 16:51 ` Steve Capper
2020-10-13 16:57 ` Ard Biesheuvel
2020-10-14 3:44 ` Anshuman Khandual
1 sibling, 1 reply; 26+ messages in thread
From: Steve Capper @ 2020-10-13 16:51 UTC (permalink / raw)
To: Ard Biesheuvel, linux-arm-kernel
Cc: Catalin Marinas, nd, will, Anshuman Khandual
Hi Ard,
One comment below...
On 08/10/2020 16:36, Ard Biesheuvel wrote:
> For historical reasons, the arm64 kernel VA space is configured as two
> equally sized halves, i.e., on a 48-bit VA build, the VA space is split
> into a 47-bit vmalloc region and a 47-bit linear region.
>
> When support for 52-bit virtual addressing was added, this equal split
> was kept, resulting in a substantial waste of virtual address space in
> the linear region:
>
> 48-bit VA 52-bit VA
> 0xffff_ffff_ffff_ffff +-------------+ +-------------+
> | vmalloc | | vmalloc |
> 0xffff_8000_0000_0000 +-------------+ _PAGE_END(48) +-------------+
> | linear | : :
> 0xffff_0000_0000_0000 +-------------+ : :
> : : : :
> : : : :
> : : : :
> : : : currently :
> : unusable : : :
> : : : unused :
> : by : : :
> : : : :
> : hardware : : :
> : : : :
> 0xfff8_0000_0000_0000 : : _PAGE_END(52) +-------------+
> : : | |
> : : | |
> : : | |
> : : | |
> : : | |
> : unusable : | |
> : : | linear |
> : by : | |
> : : | region |
> : hardware : | |
> : : | |
> : : | |
> : : | |
> : : | |
> : : | |
> : : | |
> 0xfff0_0000_0000_0000 +-------------+ PAGE_OFFSET +-------------+
>
> As illustrated above, the 52-bit VA kernel uses 47 bits for the vmalloc
> space (as before), to ensure that a single 64k granule kernel image can
> support any 64k granule capable system, regardless of whether it supports
> the 52-bit virtual addressing extension. However, due to the fact that
> the VA space is still split in equal halves, the linear region is only
> 2^51 bytes in size, wasting almost half of the 52-bit VA space.
>
> Let's fix this, by abandoning the equal split, and simply assigning all
> VA space outside of the vmalloc region to the linear region.
>
> The KASAN shadow region is reconfigured so that it ends at the start of
> the vmalloc region, and grows downwards. That way, the arrangement of
> the vmalloc space (which contains kernel mappings, modules, BPF region,
> the vmemmap array etc) is identical between non-KASAN and KASAN builds,
> which aids debugging.
>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> ---
> Documentation/arm64/kasan-offsets.sh | 3 +--
> Documentation/arm64/memory.rst | 19 +++++++++----------
> arch/arm64/Kconfig | 20 ++++++++++----------
> arch/arm64/include/asm/memory.h | 12 +++++-------
> arch/arm64/mm/init.c | 2 +-
> 5 files changed, 26 insertions(+), 30 deletions(-)
>
[...]
> diff --git a/Documentation/arm64/memory.rst b/Documentation/arm64/memory.rst
> index cf03b3290800..ee51eb66a578 100644
> --- a/Documentation/arm64/memory.rst
> +++ b/Documentation/arm64/memory.rst
> @@ -32,10 +32,10 @@ AArch64 Linux memory layout with 4KB pages + 4 levels (48-bit)::
> -----------------------------------------------------------------------
> 0000000000000000 0000ffffffffffff 256TB user
> ffff000000000000 ffff7fffffffffff 128TB kernel logical memory map
> - ffff800000000000 ffff9fffffffffff 32TB kasan shadow region
> - ffffa00000000000 ffffa00007ffffff 128MB bpf jit region
> - ffffa00008000000 ffffa0000fffffff 128MB modules
> - ffffa00010000000 fffffdffbffeffff ~93TB vmalloc
> +[ ffff600000000000 ffff7fffffffffff ] 32TB [ kasan shadow region ]
We have the KASAN shadow region intersecting with the kernel logical
memory map now. Could this present a problem if both the KASAN and
phys_to_virt paths land on the same VA? Or is that not possible? (Also
what about smaller VA sizes?)
If it is a problem, could carving out the appropriate memblocks (and
warning the user) be a way forward?
Cheers,
--
Steve
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region
2020-10-08 15:36 ` [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region Ard Biesheuvel
@ 2020-10-13 16:52 ` Steve Capper
2020-11-10 12:55 ` Geert Uytterhoeven
1 sibling, 0 replies; 26+ messages in thread
From: Steve Capper @ 2020-10-13 16:52 UTC (permalink / raw)
To: Ard Biesheuvel, linux-arm-kernel
Cc: Catalin Marinas, nd, will, Anshuman Khandual
On 08/10/2020 16:36, Ard Biesheuvel wrote:
> Now that we have reverted the introduction of the vmemmap struct page
> pointer and the separate physvirt_offset, we can simplify things further,
> and place the vmemmap region in the VA space in such a way that virtual
> to page translations and vice versa can be implemented using a single
> arithmetic shift.
>
> One happy coincidence resulting from this is that the 48-bit/4k and
> 52-bit/64k configurations (which are assumed to be the two most
> prevalent) end up with the same placement of the vmemmap region. In
> a subsequent patch, we will take advantage of this, and unify the
> memory maps even more.
>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Steve Capper <steve.capper@arm.com>
Cheers,
--
Steve
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 4/4] arm64: mm: tidy up top of kernel VA space
2020-10-08 15:36 ` [PATCH v2 4/4] arm64: mm: tidy up top of kernel VA space Ard Biesheuvel
@ 2020-10-13 16:52 ` Steve Capper
0 siblings, 0 replies; 26+ messages in thread
From: Steve Capper @ 2020-10-13 16:52 UTC (permalink / raw)
To: Ard Biesheuvel, linux-arm-kernel
Cc: Catalin Marinas, nd, will, Anshuman Khandual
On 08/10/2020 16:36, Ard Biesheuvel wrote:
> Tidy up the way the top of the kernel VA space is organized, by mirroring
> the 256 MB region we have below the vmalloc space, and populating it top
> down with the PCI I/O space, some guard regions, and the fixmap region.
> The latter region is itself populated top down, and today only covers
> about 4 MB, and so 224 MB is ample, and no guard region is therefore
> required.
>
> The resulting layout is identical between 48-bit/4k and 52-bit/64k
> configurations.
Nice! :-)
>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Steve Capper <steve.capper@arm.com>
Cheers,
--
Steve
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations
2020-10-13 16:51 ` Steve Capper
@ 2020-10-13 16:57 ` Ard Biesheuvel
2020-10-13 17:38 ` Steve Capper
0 siblings, 1 reply; 26+ messages in thread
From: Ard Biesheuvel @ 2020-10-13 16:57 UTC (permalink / raw)
To: Steve Capper
Cc: Catalin Marinas, nd, will, linux-arm-kernel, Anshuman Khandual
On Tue, 13 Oct 2020 at 18:51, Steve Capper <Steve.Capper@arm.com> wrote:
>
> Hi Ard,
>
> One comment below...
>
> On 08/10/2020 16:36, Ard Biesheuvel wrote:
> > For historical reasons, the arm64 kernel VA space is configured as two
> > equally sized halves, i.e., on a 48-bit VA build, the VA space is split
> > into a 47-bit vmalloc region and a 47-bit linear region.
> >
> > When support for 52-bit virtual addressing was added, this equal split
> > was kept, resulting in a substantial waste of virtual address space in
> > the linear region:
> >
> > 48-bit VA 52-bit VA
> > 0xffff_ffff_ffff_ffff +-------------+ +-------------+
> > | vmalloc | | vmalloc |
> > 0xffff_8000_0000_0000 +-------------+ _PAGE_END(48) +-------------+
> > | linear | : :
> > 0xffff_0000_0000_0000 +-------------+ : :
> > : : : :
> > : : : :
> > : : : :
> > : : : currently :
> > : unusable : : :
> > : : : unused :
> > : by : : :
> > : : : :
> > : hardware : : :
> > : : : :
> > 0xfff8_0000_0000_0000 : : _PAGE_END(52) +-------------+
> > : : | |
> > : : | |
> > : : | |
> > : : | |
> > : : | |
> > : unusable : | |
> > : : | linear |
> > : by : | |
> > : : | region |
> > : hardware : | |
> > : : | |
> > : : | |
> > : : | |
> > : : | |
> > : : | |
> > : : | |
> > 0xfff0_0000_0000_0000 +-------------+ PAGE_OFFSET +-------------+
> >
> > As illustrated above, the 52-bit VA kernel uses 47 bits for the vmalloc
> > space (as before), to ensure that a single 64k granule kernel image can
> > support any 64k granule capable system, regardless of whether it supports
> > the 52-bit virtual addressing extension. However, due to the fact that
> > the VA space is still split in equal halves, the linear region is only
> > 2^51 bytes in size, wasting almost half of the 52-bit VA space.
> >
> > Let's fix this, by abandoning the equal split, and simply assigning all
> > VA space outside of the vmalloc region to the linear region.
> >
> > The KASAN shadow region is reconfigured so that it ends at the start of
> > the vmalloc region, and grows downwards. That way, the arrangement of
> > the vmalloc space (which contains kernel mappings, modules, BPF region,
> > the vmemmap array etc) is identical between non-KASAN and KASAN builds,
> > which aids debugging.
> >
> > Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> > ---
> > Documentation/arm64/kasan-offsets.sh | 3 +--
> > Documentation/arm64/memory.rst | 19 +++++++++----------
> > arch/arm64/Kconfig | 20 ++++++++++----------
> > arch/arm64/include/asm/memory.h | 12 +++++-------
> > arch/arm64/mm/init.c | 2 +-
> > 5 files changed, 26 insertions(+), 30 deletions(-)
> >
>
> [...]
>
> > diff --git a/Documentation/arm64/memory.rst b/Documentation/arm64/memory.rst
> > index cf03b3290800..ee51eb66a578 100644
> > --- a/Documentation/arm64/memory.rst
> > +++ b/Documentation/arm64/memory.rst
> > @@ -32,10 +32,10 @@ AArch64 Linux memory layout with 4KB pages + 4 levels (48-bit)::
> > -----------------------------------------------------------------------
> > 0000000000000000 0000ffffffffffff 256TB user
> > ffff000000000000 ffff7fffffffffff 128TB kernel logical memory map
> > - ffff800000000000 ffff9fffffffffff 32TB kasan shadow region
> > - ffffa00000000000 ffffa00007ffffff 128MB bpf jit region
> > - ffffa00008000000 ffffa0000fffffff 128MB modules
> > - ffffa00010000000 fffffdffbffeffff ~93TB vmalloc
> > +[ ffff600000000000 ffff7fffffffffff ] 32TB [ kasan shadow region ]
>
> We have the KASAN shadow region intersecting with the kernel logical
> memory map now. Could this present a problem if both the KASAN and
> phys_to_virt paths land on the same VA? Or is that not possible? (Also
> what about smaller VA sizes?)
>
> If it is a problem, could carving out the appropriate memblocks (and
> warning the user) be a way forward?
>
Hi Steve,
This is currently taken into account in arm64_memblock_init():
const s64 linear_region_size = PAGE_END - _PAGE_OFFSET(vabits_actual);
When CONFIG_KASAN=y, PAGE_END is #defined as
(KASAN_SHADOW_END - (1UL << (vabits_actual - KASAN_SHADOW_SCALE_SHIFT)))
and so the linear region ends where the KASAN shadow region starts. So
even if the static definitions overlap, the shared region will not be
used for mapping memory if it is needed for allocating shadow pages.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations
2020-10-13 16:57 ` Ard Biesheuvel
@ 2020-10-13 17:38 ` Steve Capper
0 siblings, 0 replies; 26+ messages in thread
From: Steve Capper @ 2020-10-13 17:38 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Catalin Marinas, nd, will, linux-arm-kernel, Anshuman Khandual
On 13/10/2020 17:57, Ard Biesheuvel wrote:
> On Tue, 13 Oct 2020 at 18:51, Steve Capper <Steve.Capper@arm.com> wrote:
>>
>> Hi Ard,
>>
>> One comment below...
>>
>> On 08/10/2020 16:36, Ard Biesheuvel wrote:
>>> For historical reasons, the arm64 kernel VA space is configured as two
>>> equally sized halves, i.e., on a 48-bit VA build, the VA space is split
>>> into a 47-bit vmalloc region and a 47-bit linear region.
>>>
>>> When support for 52-bit virtual addressing was added, this equal split
>>> was kept, resulting in a substantial waste of virtual address space in
>>> the linear region:
>>>
>>> 48-bit VA 52-bit VA
>>> 0xffff_ffff_ffff_ffff +-------------+ +-------------+
>>> | vmalloc | | vmalloc |
>>> 0xffff_8000_0000_0000 +-------------+ _PAGE_END(48) +-------------+
>>> | linear | : :
>>> 0xffff_0000_0000_0000 +-------------+ : :
>>> : : : :
>>> : : : :
>>> : : : :
>>> : : : currently :
>>> : unusable : : :
>>> : : : unused :
>>> : by : : :
>>> : : : :
>>> : hardware : : :
>>> : : : :
>>> 0xfff8_0000_0000_0000 : : _PAGE_END(52) +-------------+
>>> : : | |
>>> : : | |
>>> : : | |
>>> : : | |
>>> : : | |
>>> : unusable : | |
>>> : : | linear |
>>> : by : | |
>>> : : | region |
>>> : hardware : | |
>>> : : | |
>>> : : | |
>>> : : | |
>>> : : | |
>>> : : | |
>>> : : | |
>>> 0xfff0_0000_0000_0000 +-------------+ PAGE_OFFSET +-------------+
>>>
>>> As illustrated above, the 52-bit VA kernel uses 47 bits for the vmalloc
>>> space (as before), to ensure that a single 64k granule kernel image can
>>> support any 64k granule capable system, regardless of whether it supports
>>> the 52-bit virtual addressing extension. However, due to the fact that
>>> the VA space is still split in equal halves, the linear region is only
>>> 2^51 bytes in size, wasting almost half of the 52-bit VA space.
>>>
>>> Let's fix this, by abandoning the equal split, and simply assigning all
>>> VA space outside of the vmalloc region to the linear region.
>>>
>>> The KASAN shadow region is reconfigured so that it ends at the start of
>>> the vmalloc region, and grows downwards. That way, the arrangement of
>>> the vmalloc space (which contains kernel mappings, modules, BPF region,
>>> the vmemmap array etc) is identical between non-KASAN and KASAN builds,
>>> which aids debugging.
>>>
>>> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
>>> ---
>>> Documentation/arm64/kasan-offsets.sh | 3 +--
>>> Documentation/arm64/memory.rst | 19 +++++++++----------
>>> arch/arm64/Kconfig | 20 ++++++++++----------
>>> arch/arm64/include/asm/memory.h | 12 +++++-------
>>> arch/arm64/mm/init.c | 2 +-
>>> 5 files changed, 26 insertions(+), 30 deletions(-)
>>>
>>
>> [...]
>>
>>> diff --git a/Documentation/arm64/memory.rst b/Documentation/arm64/memory.rst
>>> index cf03b3290800..ee51eb66a578 100644
>>> --- a/Documentation/arm64/memory.rst
>>> +++ b/Documentation/arm64/memory.rst
>>> @@ -32,10 +32,10 @@ AArch64 Linux memory layout with 4KB pages + 4 levels (48-bit)::
>>> -----------------------------------------------------------------------
>>> 0000000000000000 0000ffffffffffff 256TB user
>>> ffff000000000000 ffff7fffffffffff 128TB kernel logical memory map
>>> - ffff800000000000 ffff9fffffffffff 32TB kasan shadow region
>>> - ffffa00000000000 ffffa00007ffffff 128MB bpf jit region
>>> - ffffa00008000000 ffffa0000fffffff 128MB modules
>>> - ffffa00010000000 fffffdffbffeffff ~93TB vmalloc
>>> +[ ffff600000000000 ffff7fffffffffff ] 32TB [ kasan shadow region ]
>>
>> We have the KASAN shadow region intersecting with the kernel logical
>> memory map now. Could this present a problem if both the KASAN and
>> phys_to_virt paths land on the same VA? Or is that not possible? (Also
>> what about smaller VA sizes?)
>>
>> If it is a problem, could carving out the appropriate memblocks (and
>> warning the user) be a way forward?
>>
>
> Hi Steve,
>
> This is currently taken into account in arm64_memblock_init():
>
> const s64 linear_region_size = PAGE_END - _PAGE_OFFSET(vabits_actual);
>
> When CONFIG_KASAN=y, PAGE_END is #defined as
>
> (KASAN_SHADOW_END - (1UL << (vabits_actual - KASAN_SHADOW_SCALE_SHIFT)))
>
> and so the linear region ends where the KASAN shadow region starts. So
> even if the static definitions overlap, the shared region will not be
> used for mapping memory if it is needed for allocating shadow pages.
>
Ahhh, yes, thanks that will prevent the intersection from occurring.
This looks good to me, feel free to add:
Reviewed-by: Steve Capper <steve.capper@arm.com>
Cheers,
--
Steve
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations
2020-10-08 15:36 ` [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations Ard Biesheuvel
2020-10-13 16:51 ` Steve Capper
@ 2020-10-14 3:44 ` Anshuman Khandual
2020-10-14 7:18 ` Ard Biesheuvel
1 sibling, 1 reply; 26+ messages in thread
From: Anshuman Khandual @ 2020-10-14 3:44 UTC (permalink / raw)
To: Ard Biesheuvel, linux-arm-kernel; +Cc: catalin.marinas, will, Steve Capper
On 10/08/2020 09:06 PM, Ard Biesheuvel wrote:
> For historical reasons, the arm64 kernel VA space is configured as two
> equally sized halves, i.e., on a 48-bit VA build, the VA space is split
> into a 47-bit vmalloc region and a 47-bit linear region.
>
> When support for 52-bit virtual addressing was added, this equal split
> was kept, resulting in a substantial waste of virtual address space in
> the linear region:
>
> 48-bit VA 52-bit VA
> 0xffff_ffff_ffff_ffff +-------------+ +-------------+
> | vmalloc | | vmalloc |
> 0xffff_8000_0000_0000 +-------------+ _PAGE_END(48) +-------------+
> | linear | : :
> 0xffff_0000_0000_0000 +-------------+ : :
> : : : :
> : : : :
> : : : :
> : : : currently :
> : unusable : : :
> : : : unused :
> : by : : :
> : : : :
> : hardware : : :
> : : : :
> 0xfff8_0000_0000_0000 : : _PAGE_END(52) +-------------+
> : : | |
> : : | |
> : : | |
> : : | |
> : : | |
> : unusable : | |
> : : | linear |
> : by : | |
> : : | region |
> : hardware : | |
> : : | |
> : : | |
> : : | |
> : : | |
> : : | |
> : : | |
> 0xfff0_0000_0000_0000 +-------------+ PAGE_OFFSET +-------------+
>
> As illustrated above, the 52-bit VA kernel uses 47 bits for the vmalloc
> space (as before), to ensure that a single 64k granule kernel image can
> support any 64k granule capable system, regardless of whether it supports
> the 52-bit virtual addressing extension. However, due to the fact that
> the VA space is still split in equal halves, the linear region is only
> 2^51 bytes in size, wasting almost half of the 52-bit VA space.
Right, that is a huge wastage. Increasing the linear mapping will definitely
help in getting more memory used on the system. But now do we have enough
vmemmap range to support this enlarged linear mapping ?
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations
2020-10-14 3:44 ` Anshuman Khandual
@ 2020-10-14 7:18 ` Ard Biesheuvel
0 siblings, 0 replies; 26+ messages in thread
From: Ard Biesheuvel @ 2020-10-14 7:18 UTC (permalink / raw)
To: Anshuman Khandual; +Cc: Catalin Marinas, Will Deacon, Linux ARM, Steve Capper
On Wed, 14 Oct 2020 at 05:45, Anshuman Khandual
<anshuman.khandual@arm.com> wrote:
>
>
>
> On 10/08/2020 09:06 PM, Ard Biesheuvel wrote:
> > For historical reasons, the arm64 kernel VA space is configured as two
> > equally sized halves, i.e., on a 48-bit VA build, the VA space is split
> > into a 47-bit vmalloc region and a 47-bit linear region.
> >
> > When support for 52-bit virtual addressing was added, this equal split
> > was kept, resulting in a substantial waste of virtual address space in
> > the linear region:
> >
> > 48-bit VA 52-bit VA
> > 0xffff_ffff_ffff_ffff +-------------+ +-------------+
> > | vmalloc | | vmalloc |
> > 0xffff_8000_0000_0000 +-------------+ _PAGE_END(48) +-------------+
> > | linear | : :
> > 0xffff_0000_0000_0000 +-------------+ : :
> > : : : :
> > : : : :
> > : : : :
> > : : : currently :
> > : unusable : : :
> > : : : unused :
> > : by : : :
> > : : : :
> > : hardware : : :
> > : : : :
> > 0xfff8_0000_0000_0000 : : _PAGE_END(52) +-------------+
> > : : | |
> > : : | |
> > : : | |
> > : : | |
> > : : | |
> > : unusable : | |
> > : : | linear |
> > : by : | |
> > : : | region |
> > : hardware : | |
> > : : | |
> > : : | |
> > : : | |
> > : : | |
> > : : | |
> > : : | |
> > 0xfff0_0000_0000_0000 +-------------+ PAGE_OFFSET +-------------+
> >
> > As illustrated above, the 52-bit VA kernel uses 47 bits for the vmalloc
> > space (as before), to ensure that a single 64k granule kernel image can
> > support any 64k granule capable system, regardless of whether it supports
> > the 52-bit virtual addressing extension. However, due to the fact that
> > the VA space is still split in equal halves, the linear region is only
> > 2^51 bytes in size, wasting almost half of the 52-bit VA space.
>
> Right, that is a huge wastage. Increasing the linear mapping will definitely
> help in getting more memory used on the system. But now do we have enough
> vmemmap range to support this enlarged linear mapping ?
Yes.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation
2020-10-08 15:35 ` [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation Ard Biesheuvel
2020-10-13 16:14 ` Steve Capper
2020-10-13 16:47 ` Steve Capper
@ 2020-10-15 10:47 ` Will Deacon
2 siblings, 0 replies; 26+ messages in thread
From: Will Deacon @ 2020-10-15 10:47 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: catalin.marinas, Anshuman Khandual, linux-arm-kernel, Steve Capper
On Thu, Oct 08, 2020 at 05:35:59PM +0200, Ard Biesheuvel wrote:
> On arm64, the global variable memstart_addr represents the physical
> address of PAGE_OFFSET, and so physical to virtual translations or
> vice versa used to come down to simple additions or subtractions
> involving the values of PAGE_OFFSET and memstart_addr.
>
> When support for 52-bit virtual addressing was introduced, we had to
> deal with PAGE_OFFSET potentially being outside of the region that
> can be covered by the virtual range (as the 52-bit VA capable build
> needs to be able to run on systems that are only 48-bit VA capable),
> and for this reason, another translation was introduced, and recorded
> in the global variable physvirt_offset.
>
> However, if we go back to the original definition of memstart_addr,
> i.e., the physical address of PAGE_OFFSET, it turns out that there is
> no need for two separate translations: instead, we can simply subtract
> the size of the unaddressable VA space from memstart_addr to make the
> available physical memory appear in the 48-bit addressable VA region.
>
> This simplifies things, but also fixes a bug on KASLR builds, which
> may update memstart_addr later on in arm64_memblock_init(), but fails
> to update vmemmap and physvirt_offset accordingly.
>
> Fixes: 5383cc6efed13 ("arm64: mm: Introduce vabits_actual")
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> ---
> arch/arm64/include/asm/memory.h | 5 ++--
> arch/arm64/include/asm/pgtable.h | 4 +--
> arch/arm64/mm/init.c | 30 +++++++-------------
> 3 files changed, 14 insertions(+), 25 deletions(-)
I'll pick this one patch up for 5.10, thanks.
Will
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit
2020-10-08 15:35 [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
` (4 preceding siblings ...)
2020-10-09 14:16 ` [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
@ 2020-10-15 20:40 ` Will Deacon
2020-11-09 18:51 ` Catalin Marinas
6 siblings, 0 replies; 26+ messages in thread
From: Will Deacon @ 2020-10-15 20:40 UTC (permalink / raw)
To: linux-arm-kernel, Ard Biesheuvel
Cc: catalin.marinas, Anshuman Khandual, kernel-team, Will Deacon,
Steve Capper
On Thu, 8 Oct 2020 17:35:58 +0200, Ard Biesheuvel wrote:
> This series reorganizes the kernel VA space slightly so that 52-bit VA
> configurations can use more virtual address space, i.e., the usable
> linear address space almost doubles, from 2^51 to 2^52-2^47.
>
> Patch #1 merges the physvirt_offset and memstart_addr variables, both of
> which represent translations between the physical address space and the
> linear region, and there is no need for having both. This fixes a bug too,
> as the two values were not properly kept in sync when booting with KASLR
> enabled.
>
> [...]
Applied the first patch to arm64 (for-next/core), thanks!
[1/1] arm64: mm: use single quantity to represent the PA to VA translation
https://git.kernel.org/arm64/c/7bc1a0f9e176
Cheers,
--
Will
https://fixes.arm64.dev
https://next.arm64.dev
https://will.arm64.dev
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit
2020-10-08 15:35 [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
` (5 preceding siblings ...)
2020-10-15 20:40 ` Will Deacon
@ 2020-11-09 18:51 ` Catalin Marinas
6 siblings, 0 replies; 26+ messages in thread
From: Catalin Marinas @ 2020-11-09 18:51 UTC (permalink / raw)
To: linux-arm-kernel, Ard Biesheuvel; +Cc: Steve Capper, will, Anshuman Khandual
On Thu, 8 Oct 2020 17:35:58 +0200, Ard Biesheuvel wrote:
> This series reorganizes the kernel VA space slightly so that 52-bit VA
> configurations can use more virtual address space, i.e., the usable
> linear address space almost doubles, from 2^51 to 2^52-2^47.
>
> Patch #1 merges the physvirt_offset and memstart_addr variables, both of
> which represent translations between the physical address space and the
> linear region, and there is no need for having both. This fixes a bug too,
> as the two values were not properly kept in sync when booting with KASLR
> enabled.
>
> [...]
Applied to arm64 (for-next/va-refactor), thanks!
[2/4] arm64: mm: extend linear region for 52-bit VA configurations
https://git.kernel.org/arm64/c/f4693c2716b3
[3/4] arm64: mm: make vmemmap region a projection of the linear region
https://git.kernel.org/arm64/c/8c96400d6a39
[4/4] arm64: mm: tidy up top of kernel VA space
https://git.kernel.org/arm64/c/9ad7c6d5e75b
--
Catalin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region
2020-10-08 15:36 ` [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region Ard Biesheuvel
2020-10-13 16:52 ` Steve Capper
@ 2020-11-10 12:55 ` Geert Uytterhoeven
2020-11-10 13:10 ` Ard Biesheuvel
1 sibling, 1 reply; 26+ messages in thread
From: Geert Uytterhoeven @ 2020-11-10 12:55 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Catalin Marinas, Steve Capper, Will Deacon, Linux ARM, Anshuman Khandual
Hi Ard,
On Thu, Oct 8, 2020 at 5:43 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> Now that we have reverted the introduction of the vmemmap struct page
> pointer and the separate physvirt_offset, we can simplify things further,
> and place the vmemmap region in the VA space in such a way that virtual
> to page translations and vice versa can be implemented using a single
> arithmetic shift.
>
> One happy coincidence resulting from this is that the 48-bit/4k and
> 52-bit/64k configurations (which are assumed to be the two most
> prevalent) end up with the same placement of the vmemmap region. In
> a subsequent patch, we will take advantage of this, and unify the
> memory maps even more.
>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
This is now commit 8c96400d6a39be76 ("arm64: mm: make vmemmap region a
projection of the linear region") in arm64/for-next/core.
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -504,6 +504,8 @@ static void __init free_unused_memmap(void)
> */
> void __init mem_init(void)
> {
> + BUILD_BUG_ON(!is_power_of_2(sizeof(struct page)));
This check is triggering for me.
If CONFIG_MEMCG=n, sizeof(struct page) = 56.
If CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y, this is mitigated by
the explicit alignment:
#ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
#define _struct_page_alignment __aligned(2 * sizeof(unsigned long))
#else
#define _struct_page_alignment
#endif
struct page { ... } _struct_page_alignment;
However, HAVE_ALIGNED_STRUCT_PAGE is selected only if SLUB,
while my .config is using SLAB.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region
2020-11-10 12:55 ` Geert Uytterhoeven
@ 2020-11-10 13:10 ` Ard Biesheuvel
2020-11-10 14:08 ` Ard Biesheuvel
0 siblings, 1 reply; 26+ messages in thread
From: Ard Biesheuvel @ 2020-11-10 13:10 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Catalin Marinas, Steve Capper, Will Deacon, Linux ARM, Anshuman Khandual
On Tue, 10 Nov 2020 at 13:55, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> Hi Ard,
>
> On Thu, Oct 8, 2020 at 5:43 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > Now that we have reverted the introduction of the vmemmap struct page
> > pointer and the separate physvirt_offset, we can simplify things further,
> > and place the vmemmap region in the VA space in such a way that virtual
> > to page translations and vice versa can be implemented using a single
> > arithmetic shift.
> >
> > One happy coincidence resulting from this is that the 48-bit/4k and
> > 52-bit/64k configurations (which are assumed to be the two most
> > prevalent) end up with the same placement of the vmemmap region. In
> > a subsequent patch, we will take advantage of this, and unify the
> > memory maps even more.
> >
> > Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
>
> This is now commit 8c96400d6a39be76 ("arm64: mm: make vmemmap region a
> projection of the linear region") in arm64/for-next/core.
>
> > --- a/arch/arm64/mm/init.c
> > +++ b/arch/arm64/mm/init.c
> > @@ -504,6 +504,8 @@ static void __init free_unused_memmap(void)
> > */
> > void __init mem_init(void)
> > {
> > + BUILD_BUG_ON(!is_power_of_2(sizeof(struct page)));
>
> This check is triggering for me.
>
> If CONFIG_MEMCG=n, sizeof(struct page) = 56.
>
> If CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y, this is mitigated by
> the explicit alignment:
>
> #ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
> #define _struct_page_alignment __aligned(2 * sizeof(unsigned long))
> #else
> #define _struct_page_alignment
> #endif
>
> struct page { ... } _struct_page_alignment;
>
> However, HAVE_ALIGNED_STRUCT_PAGE is selected only if SLUB,
> while my .config is using SLAB.
>
Thanks for the report. I will look into this.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region
2020-11-10 13:10 ` Ard Biesheuvel
@ 2020-11-10 14:08 ` Ard Biesheuvel
2020-11-10 14:56 ` Geert Uytterhoeven
2020-11-10 15:39 ` Catalin Marinas
0 siblings, 2 replies; 26+ messages in thread
From: Ard Biesheuvel @ 2020-11-10 14:08 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Catalin Marinas, Steve Capper, Will Deacon, Linux ARM, Anshuman Khandual
On Tue, 10 Nov 2020 at 14:10, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Tue, 10 Nov 2020 at 13:55, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> >
> > Hi Ard,
> >
> > On Thu, Oct 8, 2020 at 5:43 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > Now that we have reverted the introduction of the vmemmap struct page
> > > pointer and the separate physvirt_offset, we can simplify things further,
> > > and place the vmemmap region in the VA space in such a way that virtual
> > > to page translations and vice versa can be implemented using a single
> > > arithmetic shift.
> > >
> > > One happy coincidence resulting from this is that the 48-bit/4k and
> > > 52-bit/64k configurations (which are assumed to be the two most
> > > prevalent) end up with the same placement of the vmemmap region. In
> > > a subsequent patch, we will take advantage of this, and unify the
> > > memory maps even more.
> > >
> > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> >
> > This is now commit 8c96400d6a39be76 ("arm64: mm: make vmemmap region a
> > projection of the linear region") in arm64/for-next/core.
> >
> > > --- a/arch/arm64/mm/init.c
> > > +++ b/arch/arm64/mm/init.c
> > > @@ -504,6 +504,8 @@ static void __init free_unused_memmap(void)
> > > */
> > > void __init mem_init(void)
> > > {
> > > + BUILD_BUG_ON(!is_power_of_2(sizeof(struct page)));
> >
> > This check is triggering for me.
> >
> > If CONFIG_MEMCG=n, sizeof(struct page) = 56.
> >
> > If CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y, this is mitigated by
> > the explicit alignment:
> >
> > #ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
> > #define _struct_page_alignment __aligned(2 * sizeof(unsigned long))
> > #else
> > #define _struct_page_alignment
> > #endif
> >
> > struct page { ... } _struct_page_alignment;
> >
> > However, HAVE_ALIGNED_STRUCT_PAGE is selected only if SLUB,
> > while my .config is using SLAB.
> >
>
> Thanks for the report. I will look into this.
OK, so we can obviously fix this easily by setting
CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y unconditionally instead of only 'if
SLUB'. The question is whether that is likely to lead to any
performance regressions.
So first of all, having a smaller struct page means we can fit more of
them into memory. On a 4k pages config with SPARSEMEM_VMEMMAP enabled
(which allocates struct pages in 2M blocks), every 2M block can cover
146 MB of DRAM instead of 128 MB. I'm not sure what kind of DRAM
arrangement would be needed to take advantage of this in practice,
though.
Another aspect is D-cache utilization: cache lines are typically 64
bytes on arm64, and while we can improve Dcache utilization in theory
(by virtue of the smaller size), the random access nature of struct
pages may well result in the opposite, given that 3 out of 4 struct
pages now straddle two cachelines.
Given the above, and given the purpose of this patch series, which was
to tidy up and unify different configurations, in order to reduce the
size of the validation matrix, I think it would be reasonable to
simply set CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y for arm64 in all cases.
--
Ard.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region
2020-11-10 14:08 ` Ard Biesheuvel
@ 2020-11-10 14:56 ` Geert Uytterhoeven
2020-11-10 15:39 ` Catalin Marinas
1 sibling, 0 replies; 26+ messages in thread
From: Geert Uytterhoeven @ 2020-11-10 14:56 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Catalin Marinas, Steve Capper, Will Deacon, Linux ARM, Anshuman Khandual
Hi Ard,
On Tue, Nov 10, 2020 at 3:09 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> On Tue, 10 Nov 2020 at 14:10, Ard Biesheuvel <ardb@kernel.org> wrote:
> > On Tue, 10 Nov 2020 at 13:55, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > On Thu, Oct 8, 2020 at 5:43 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > Now that we have reverted the introduction of the vmemmap struct page
> > > > pointer and the separate physvirt_offset, we can simplify things further,
> > > > and place the vmemmap region in the VA space in such a way that virtual
> > > > to page translations and vice versa can be implemented using a single
> > > > arithmetic shift.
> > > >
> > > > One happy coincidence resulting from this is that the 48-bit/4k and
> > > > 52-bit/64k configurations (which are assumed to be the two most
> > > > prevalent) end up with the same placement of the vmemmap region. In
> > > > a subsequent patch, we will take advantage of this, and unify the
> > > > memory maps even more.
> > > >
> > > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> > >
> > > This is now commit 8c96400d6a39be76 ("arm64: mm: make vmemmap region a
> > > projection of the linear region") in arm64/for-next/core.
> > >
> > > > --- a/arch/arm64/mm/init.c
> > > > +++ b/arch/arm64/mm/init.c
> > > > @@ -504,6 +504,8 @@ static void __init free_unused_memmap(void)
> > > > */
> > > > void __init mem_init(void)
> > > > {
> > > > + BUILD_BUG_ON(!is_power_of_2(sizeof(struct page)));
> > >
> > > This check is triggering for me.
> > >
> > > If CONFIG_MEMCG=n, sizeof(struct page) = 56.
> > >
> > > If CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y, this is mitigated by
> > > the explicit alignment:
> > >
> > > #ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
> > > #define _struct_page_alignment __aligned(2 * sizeof(unsigned long))
> > > #else
> > > #define _struct_page_alignment
> > > #endif
> > >
> > > struct page { ... } _struct_page_alignment;
> > >
> > > However, HAVE_ALIGNED_STRUCT_PAGE is selected only if SLUB,
> > > while my .config is using SLAB.
> > >
> >
> > Thanks for the report. I will look into this.
>
> OK, so we can obviously fix this easily by setting
> CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y unconditionally instead of only 'if
> SLUB'. The question is whether that is likely to lead to any
> performance regressions.
>
> So first of all, having a smaller struct page means we can fit more of
> them into memory. On a 4k pages config with SPARSEMEM_VMEMMAP enabled
> (which allocates struct pages in 2M blocks), every 2M block can cover
> 146 MB of DRAM instead of 128 MB. I'm not sure what kind of DRAM
> arrangement would be needed to take advantage of this in practice,
> though.
So this starts making a difference only for systems with more than 1 GiB
RAM, where we can probably afford losing 2 MiB.
> Another aspect is D-cache utilization: cache lines are typically 64
> bytes on arm64, and while we can improve Dcache utilization in theory
> (by virtue of the smaller size), the random access nature of struct
> pages may well result in the opposite, given that 3 out of 4 struct
> pages now straddle two cachelines.
>
> Given the above, and given the purpose of this patch series, which was
> to tidy up and unify different configurations, in order to reduce the
> size of the validation matrix, I think it would be reasonable to
> simply set CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y for arm64 in all cases.
Thanks, sounds reasonable to me.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region
2020-11-10 14:08 ` Ard Biesheuvel
2020-11-10 14:56 ` Geert Uytterhoeven
@ 2020-11-10 15:39 ` Catalin Marinas
2020-11-10 15:42 ` Ard Biesheuvel
1 sibling, 1 reply; 26+ messages in thread
From: Catalin Marinas @ 2020-11-10 15:39 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Will Deacon, Steve Capper, Geert Uytterhoeven, Linux ARM,
Anshuman Khandual
On Tue, Nov 10, 2020 at 03:08:45PM +0100, Ard Biesheuvel wrote:
> On Tue, 10 Nov 2020 at 14:10, Ard Biesheuvel <ardb@kernel.org> wrote:
> > On Tue, 10 Nov 2020 at 13:55, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > On Thu, Oct 8, 2020 at 5:43 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > Now that we have reverted the introduction of the vmemmap struct page
> > > > pointer and the separate physvirt_offset, we can simplify things further,
> > > > and place the vmemmap region in the VA space in such a way that virtual
> > > > to page translations and vice versa can be implemented using a single
> > > > arithmetic shift.
> > > >
> > > > One happy coincidence resulting from this is that the 48-bit/4k and
> > > > 52-bit/64k configurations (which are assumed to be the two most
> > > > prevalent) end up with the same placement of the vmemmap region. In
> > > > a subsequent patch, we will take advantage of this, and unify the
> > > > memory maps even more.
> > > >
> > > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> > >
> > > This is now commit 8c96400d6a39be76 ("arm64: mm: make vmemmap region a
> > > projection of the linear region") in arm64/for-next/core.
> > >
> > > > --- a/arch/arm64/mm/init.c
> > > > +++ b/arch/arm64/mm/init.c
> > > > @@ -504,6 +504,8 @@ static void __init free_unused_memmap(void)
> > > > */
> > > > void __init mem_init(void)
> > > > {
> > > > + BUILD_BUG_ON(!is_power_of_2(sizeof(struct page)));
> > >
> > > This check is triggering for me.
> > >
> > > If CONFIG_MEMCG=n, sizeof(struct page) = 56.
> > >
> > > If CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y, this is mitigated by
> > > the explicit alignment:
> > >
> > > #ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
> > > #define _struct_page_alignment __aligned(2 * sizeof(unsigned long))
> > > #else
> > > #define _struct_page_alignment
> > > #endif
> > >
> > > struct page { ... } _struct_page_alignment;
> > >
> > > However, HAVE_ALIGNED_STRUCT_PAGE is selected only if SLUB,
> > > while my .config is using SLAB.
> > >
> >
> > Thanks for the report. I will look into this.
>
> OK, so we can obviously fix this easily by setting
> CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y unconditionally instead of only 'if
> SLUB'. The question is whether that is likely to lead to any
> performance regressions.
I'm not sure I understand. The mem_init() bug triggers if sizeof(struct
page) is not a power of 2. HAVE_ALIGNED_STRUCT_PAGE forces the page
alignment to 16 bytes but, for example, a 48-byte structure is not a
power of 2.
--
Catalin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region
2020-11-10 15:39 ` Catalin Marinas
@ 2020-11-10 15:42 ` Ard Biesheuvel
2020-11-10 16:14 ` Catalin Marinas
0 siblings, 1 reply; 26+ messages in thread
From: Ard Biesheuvel @ 2020-11-10 15:42 UTC (permalink / raw)
To: Catalin Marinas
Cc: Will Deacon, Steve Capper, Geert Uytterhoeven, Linux ARM,
Anshuman Khandual
On Tue, 10 Nov 2020 at 16:39, Catalin Marinas <catalin.marinas@arm.com> wrote:
>
> On Tue, Nov 10, 2020 at 03:08:45PM +0100, Ard Biesheuvel wrote:
> > On Tue, 10 Nov 2020 at 14:10, Ard Biesheuvel <ardb@kernel.org> wrote:
> > > On Tue, 10 Nov 2020 at 13:55, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > On Thu, Oct 8, 2020 at 5:43 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > > Now that we have reverted the introduction of the vmemmap struct page
> > > > > pointer and the separate physvirt_offset, we can simplify things further,
> > > > > and place the vmemmap region in the VA space in such a way that virtual
> > > > > to page translations and vice versa can be implemented using a single
> > > > > arithmetic shift.
> > > > >
> > > > > One happy coincidence resulting from this is that the 48-bit/4k and
> > > > > 52-bit/64k configurations (which are assumed to be the two most
> > > > > prevalent) end up with the same placement of the vmemmap region. In
> > > > > a subsequent patch, we will take advantage of this, and unify the
> > > > > memory maps even more.
> > > > >
> > > > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> > > >
> > > > This is now commit 8c96400d6a39be76 ("arm64: mm: make vmemmap region a
> > > > projection of the linear region") in arm64/for-next/core.
> > > >
> > > > > --- a/arch/arm64/mm/init.c
> > > > > +++ b/arch/arm64/mm/init.c
> > > > > @@ -504,6 +504,8 @@ static void __init free_unused_memmap(void)
> > > > > */
> > > > > void __init mem_init(void)
> > > > > {
> > > > > + BUILD_BUG_ON(!is_power_of_2(sizeof(struct page)));
> > > >
> > > > This check is triggering for me.
> > > >
> > > > If CONFIG_MEMCG=n, sizeof(struct page) = 56.
> > > >
> > > > If CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y, this is mitigated by
> > > > the explicit alignment:
> > > >
> > > > #ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
> > > > #define _struct_page_alignment __aligned(2 * sizeof(unsigned long))
> > > > #else
> > > > #define _struct_page_alignment
> > > > #endif
> > > >
> > > > struct page { ... } _struct_page_alignment;
> > > >
> > > > However, HAVE_ALIGNED_STRUCT_PAGE is selected only if SLUB,
> > > > while my .config is using SLAB.
> > > >
> > >
> > > Thanks for the report. I will look into this.
> >
> > OK, so we can obviously fix this easily by setting
> > CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y unconditionally instead of only 'if
> > SLUB'. The question is whether that is likely to lead to any
> > performance regressions.
>
> I'm not sure I understand. The mem_init() bug triggers if sizeof(struct
> page) is not a power of 2. HAVE_ALIGNED_STRUCT_PAGE forces the page
> alignment to 16 bytes but, for example, a 48-byte structure is not a
> power of 2.
>
True, but looking at include/linux/mm_types.h, I don't see how that
would happen.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region
2020-11-10 15:42 ` Ard Biesheuvel
@ 2020-11-10 16:14 ` Catalin Marinas
2020-11-10 16:18 ` Ard Biesheuvel
0 siblings, 1 reply; 26+ messages in thread
From: Catalin Marinas @ 2020-11-10 16:14 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Will Deacon, Steve Capper, Geert Uytterhoeven, Linux ARM,
Anshuman Khandual
On Tue, Nov 10, 2020 at 04:42:53PM +0100, Ard Biesheuvel wrote:
> On Tue, 10 Nov 2020 at 16:39, Catalin Marinas <catalin.marinas@arm.com> wrote:
> > On Tue, Nov 10, 2020 at 03:08:45PM +0100, Ard Biesheuvel wrote:
> > > On Tue, 10 Nov 2020 at 14:10, Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > On Tue, 10 Nov 2020 at 13:55, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > On Thu, Oct 8, 2020 at 5:43 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > > > --- a/arch/arm64/mm/init.c
> > > > > > +++ b/arch/arm64/mm/init.c
> > > > > > @@ -504,6 +504,8 @@ static void __init free_unused_memmap(void)
> > > > > > */
> > > > > > void __init mem_init(void)
> > > > > > {
> > > > > > + BUILD_BUG_ON(!is_power_of_2(sizeof(struct page)));
> > > > >
> > > > > This check is triggering for me.
> > > > >
> > > > > If CONFIG_MEMCG=n, sizeof(struct page) = 56.
> > > > >
> > > > > If CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y, this is mitigated by
> > > > > the explicit alignment:
> > > > >
> > > > > #ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
> > > > > #define _struct_page_alignment __aligned(2 * sizeof(unsigned long))
> > > > > #else
> > > > > #define _struct_page_alignment
> > > > > #endif
> > > > >
> > > > > struct page { ... } _struct_page_alignment;
> > > > >
> > > > > However, HAVE_ALIGNED_STRUCT_PAGE is selected only if SLUB,
> > > > > while my .config is using SLAB.
> > > > >
> > > >
> > > > Thanks for the report. I will look into this.
> > >
> > > OK, so we can obviously fix this easily by setting
> > > CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y unconditionally instead of only 'if
> > > SLUB'. The question is whether that is likely to lead to any
> > > performance regressions.
> >
> > I'm not sure I understand. The mem_init() bug triggers if sizeof(struct
> > page) is not a power of 2. HAVE_ALIGNED_STRUCT_PAGE forces the page
> > alignment to 16 bytes but, for example, a 48-byte structure is not a
> > power of 2.
>
> True, but looking at include/linux/mm_types.h, I don't see how that
> would happen.
Not with 48 and probably won't ever go beyond 64 for "production"
builds. But say someone wants to experiment with some debug data in
struct page and adds a long. The structure (I think 64 now with MEMCG=y)
becomes 72, force-aligned to 80. That triggers the build-bug.
Anyway, I don't mind the forced alignment, only that the build-bug you
added has a different requirement than what HAVE_ALIGNED_STRUCT_PAGE
provides (power of 2 vs 16-byte aligned).
AFAICT, VMEMMAP_START is a compile-time value, PAGE_OFFSET is also
compile-time, so can we not revert to the original virt_to_page and
page_to_virt macros? They'd not be as efficient but it may not matter
much (and if the struct size is a power of 2, the compiler should change
the division/multiplication by shifts).
--
Catalin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region
2020-11-10 16:14 ` Catalin Marinas
@ 2020-11-10 16:18 ` Ard Biesheuvel
0 siblings, 0 replies; 26+ messages in thread
From: Ard Biesheuvel @ 2020-11-10 16:18 UTC (permalink / raw)
To: Catalin Marinas
Cc: Will Deacon, Steve Capper, Geert Uytterhoeven, Linux ARM,
Anshuman Khandual
On Tue, 10 Nov 2020 at 17:14, Catalin Marinas <catalin.marinas@arm.com> wrote:
>
> On Tue, Nov 10, 2020 at 04:42:53PM +0100, Ard Biesheuvel wrote:
> > On Tue, 10 Nov 2020 at 16:39, Catalin Marinas <catalin.marinas@arm.com> wrote:
> > > On Tue, Nov 10, 2020 at 03:08:45PM +0100, Ard Biesheuvel wrote:
> > > > On Tue, 10 Nov 2020 at 14:10, Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > > On Tue, 10 Nov 2020 at 13:55, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > On Thu, Oct 8, 2020 at 5:43 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > > > > --- a/arch/arm64/mm/init.c
> > > > > > > +++ b/arch/arm64/mm/init.c
> > > > > > > @@ -504,6 +504,8 @@ static void __init free_unused_memmap(void)
> > > > > > > */
> > > > > > > void __init mem_init(void)
> > > > > > > {
> > > > > > > + BUILD_BUG_ON(!is_power_of_2(sizeof(struct page)));
> > > > > >
> > > > > > This check is triggering for me.
> > > > > >
> > > > > > If CONFIG_MEMCG=n, sizeof(struct page) = 56.
> > > > > >
> > > > > > If CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y, this is mitigated by
> > > > > > the explicit alignment:
> > > > > >
> > > > > > #ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE
> > > > > > #define _struct_page_alignment __aligned(2 * sizeof(unsigned long))
> > > > > > #else
> > > > > > #define _struct_page_alignment
> > > > > > #endif
> > > > > >
> > > > > > struct page { ... } _struct_page_alignment;
> > > > > >
> > > > > > However, HAVE_ALIGNED_STRUCT_PAGE is selected only if SLUB,
> > > > > > while my .config is using SLAB.
> > > > > >
> > > > >
> > > > > Thanks for the report. I will look into this.
> > > >
> > > > OK, so we can obviously fix this easily by setting
> > > > CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y unconditionally instead of only 'if
> > > > SLUB'. The question is whether that is likely to lead to any
> > > > performance regressions.
> > >
> > > I'm not sure I understand. The mem_init() bug triggers if sizeof(struct
> > > page) is not a power of 2. HAVE_ALIGNED_STRUCT_PAGE forces the page
> > > alignment to 16 bytes but, for example, a 48-byte structure is not a
> > > power of 2.
> >
> > True, but looking at include/linux/mm_types.h, I don't see how that
> > would happen.
>
> Not with 48 and probably won't ever go beyond 64 for "production"
> builds. But say someone wants to experiment with some debug data in
> struct page and adds a long. The structure (I think 64 now with MEMCG=y)
> becomes 72, force-aligned to 80. That triggers the build-bug.
>
> Anyway, I don't mind the forced alignment, only that the build-bug you
> added has a different requirement than what HAVE_ALIGNED_STRUCT_PAGE
> provides (power of 2 vs 16-byte aligned).
>
> AFAICT, VMEMMAP_START is a compile-time value, PAGE_OFFSET is also
> compile-time, so can we not revert to the original virt_to_page and
> page_to_virt macros? They'd not be as efficient but it may not matter
> much (and if the struct size is a power of 2, the compiler should change
> the division/multiplication by shifts).
>
Works for me. I'll go and code that up.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2020-11-10 16:19 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-08 15:35 [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
2020-10-08 15:35 ` [PATCH v2 1/4] arm64: mm: use single quantity to represent the PA to VA translation Ard Biesheuvel
2020-10-13 16:14 ` Steve Capper
2020-10-13 16:47 ` Steve Capper
2020-10-15 10:47 ` Will Deacon
2020-10-08 15:36 ` [PATCH v2 2/4] arm64: mm: extend linear region for 52-bit VA configurations Ard Biesheuvel
2020-10-13 16:51 ` Steve Capper
2020-10-13 16:57 ` Ard Biesheuvel
2020-10-13 17:38 ` Steve Capper
2020-10-14 3:44 ` Anshuman Khandual
2020-10-14 7:18 ` Ard Biesheuvel
2020-10-08 15:36 ` [PATCH v2 3/4] arm64: mm: make vmemmap region a projection of the linear region Ard Biesheuvel
2020-10-13 16:52 ` Steve Capper
2020-11-10 12:55 ` Geert Uytterhoeven
2020-11-10 13:10 ` Ard Biesheuvel
2020-11-10 14:08 ` Ard Biesheuvel
2020-11-10 14:56 ` Geert Uytterhoeven
2020-11-10 15:39 ` Catalin Marinas
2020-11-10 15:42 ` Ard Biesheuvel
2020-11-10 16:14 ` Catalin Marinas
2020-11-10 16:18 ` Ard Biesheuvel
2020-10-08 15:36 ` [PATCH v2 4/4] arm64: mm: tidy up top of kernel VA space Ard Biesheuvel
2020-10-13 16:52 ` Steve Capper
2020-10-09 14:16 ` [PATCH v2 0/4] arm64: mm: optimize VA space organization for 52-bit Ard Biesheuvel
2020-10-15 20:40 ` Will Deacon
2020-11-09 18:51 ` Catalin Marinas
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.