* [PATCH 1/5 v8] ARM: Disable KASan instrumentation for some code
2020-05-07 12:45 [PATCH 0/5 v8] KASan for Arm Linus Walleij
@ 2020-05-07 12:45 ` Linus Walleij
2020-05-07 12:45 ` [PATCH 2/5 v8] ARM: Replace string mem* functions for KASan Linus Walleij
` (4 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Linus Walleij @ 2020-05-07 12:45 UTC (permalink / raw)
To: Florian Fainelli, Abbott Liu, Russell King, Ard Biesheuvel,
Andrey Ryabinin
Cc: Arnd Bergmann, Marc Zyngier, Linus Walleij, kasan-dev,
Alexander Potapenko, linux-arm-kernel, Dmitry Vyukov
From: Andrey Ryabinin <aryabinin@virtuozzo.com>
Disable instrumentation for arch/arm/boot/compressed/*
since that code is executed before the kernel has even
set up its mappings and definately out of scope for
KASan.
Disable instrumentation of arch/arm/vdso/* because that code
is not linked with the kernel image, so the KASan management
code would fail to link.
Disable instrumentation of arch/arm/mm/physaddr.c. See commit
ec6d06efb0ba ("arm64: Add support for CONFIG_DEBUG_VIRTUAL")
for more details.
Disable kasan check in the function unwind_pop_register because
it does not matter that kasan checks failed when unwind_pop_register()
reads the stack memory of a task.
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: kasan-dev@googlegroups.com
Reported-by: Florian Fainelli <f.fainelli@gmail.com>
Reported-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Abbott Liu <liuwenliang@huawei.com>
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
ChangeLog v7->v8:
- Do not sanitize arch/arm/mm/mmu.c.
Apart from being intuitively correct, it turns out that KASan
will insert a __asan_load4() into the set_pte_at() function
in mmu.c and this is something that KASan calls in the early
initialization, to set up the shadow memory. Naturally,
__asan_load4() cannot be called before the shadow memory is
set up so we need to exclude mmu.c from sanitization.
ChangeLog v6->v7:
- Removed the KVM instrumentaton disablement since KVM
on ARM32 is gone.
---
arch/arm/boot/compressed/Makefile | 1 +
arch/arm/kernel/unwind.c | 6 +++++-
arch/arm/mm/Makefile | 2 ++
arch/arm/vdso/Makefile | 2 ++
4 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
index 9c11e7490292..abd6f3d5c2ba 100644
--- a/arch/arm/boot/compressed/Makefile
+++ b/arch/arm/boot/compressed/Makefile
@@ -24,6 +24,7 @@ OBJS += hyp-stub.o
endif
GCOV_PROFILE := n
+KASAN_SANITIZE := n
# Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
KCOV_INSTRUMENT := n
diff --git a/arch/arm/kernel/unwind.c b/arch/arm/kernel/unwind.c
index 11a964fd66f4..739a77f39a8f 100644
--- a/arch/arm/kernel/unwind.c
+++ b/arch/arm/kernel/unwind.c
@@ -236,7 +236,11 @@ static int unwind_pop_register(struct unwind_ctrl_block *ctrl,
if (*vsp >= (unsigned long *)ctrl->sp_high)
return -URC_FAILURE;
- ctrl->vrs[reg] = *(*vsp)++;
+ /* Use READ_ONCE_NOCHECK here to avoid this memory access
+ * from being tracked by KASAN.
+ */
+ ctrl->vrs[reg] = READ_ONCE_NOCHECK(*(*vsp));
+ (*vsp)++;
return URC_OK;
}
diff --git a/arch/arm/mm/Makefile b/arch/arm/mm/Makefile
index 7cb1699fbfc4..99699c32d8a5 100644
--- a/arch/arm/mm/Makefile
+++ b/arch/arm/mm/Makefile
@@ -7,6 +7,7 @@ obj-y := extable.o fault.o init.o iomap.o
obj-y += dma-mapping$(MMUEXT).o
obj-$(CONFIG_MMU) += fault-armv.o flush.o idmap.o ioremap.o \
mmap.o pgd.o mmu.o pageattr.o
+KASAN_SANITIZE_mmu.o := n
ifneq ($(CONFIG_MMU),y)
obj-y += nommu.o
@@ -16,6 +17,7 @@ endif
obj-$(CONFIG_ARM_PTDUMP_CORE) += dump.o
obj-$(CONFIG_ARM_PTDUMP_DEBUGFS) += ptdump_debugfs.o
obj-$(CONFIG_MODULES) += proc-syms.o
+KASAN_SANITIZE_physaddr.o := n
obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o
obj-$(CONFIG_ALIGNMENT_TRAP) += alignment.o
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index d3c9f03e7e79..71d18d59bd35 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -42,6 +42,8 @@ GCOV_PROFILE := n
# Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
KCOV_INSTRUMENT := n
+KASAN_SANITIZE := n
+
# Force dependency
$(obj)/vdso.o : $(obj)/vdso.so
--
2.25.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 2/5 v8] ARM: Replace string mem* functions for KASan
2020-05-07 12:45 [PATCH 0/5 v8] KASan for Arm Linus Walleij
2020-05-07 12:45 ` [PATCH 1/5 v8] ARM: Disable KASan instrumentation for some code Linus Walleij
@ 2020-05-07 12:45 ` Linus Walleij
2020-05-07 12:45 ` [PATCH 3/5 v8] ARM: Define the virtual space of KASan's shadow region Linus Walleij
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Linus Walleij @ 2020-05-07 12:45 UTC (permalink / raw)
To: Florian Fainelli, Abbott Liu, Russell King, Ard Biesheuvel,
Andrey Ryabinin
Cc: Arnd Bergmann, Linus Walleij, kasan-dev, Alexander Potapenko,
linux-arm-kernel, Dmitry Vyukov
From: Andrey Ryabinin <aryabinin@virtuozzo.com>
Functions like memset()/memmove()/memcpy() do a lot of memory
accesses.
If a bad pointer is passed to one of these functions it is important
to catch this. Compiler instrumentation cannot do this since these
functions are written in assembly.
KASan replaces these memory functions with instrumented variants.
The original functions are declared as weak symbols so that
the strong definitions in mm/kasan/kasan.c can replace them.
The original functions have aliases with a '__' prefix in their
name, so we can call the non-instrumented variant if needed.
We must use __memcpy()/__memset() in place of memcpy()/memset()
when we copy .data to RAM and when we clear .bss, because
kasan_early_init cannot be called before the initialization of
.data and .bss.
For the kernel compression and EFI libstub's custom string
libraries we need a special quirk: even if these are built
without KASan enabled, they rely on the global headers for their
custom string libraries, which means that e.g. memcpy()
will be defined to __memcpy() and we get link failures.
Since these implementations are written i C rather than
assembly we use e.g. __alias(memcpy) to redirected any
users back to the local implementation.
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: kasan-dev@googlegroups.com
Reported-by: Russell King - ARM Linux <linux@armlinux.org.uk>
Signed-off-by: Abbott Liu <liuwenliang@huawei.com>
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
ChangeLog v7->v8:
- Use the less invasive version of handling the global redefines
of the string functions in the decompressor: __alias() the
functions locally in the library.
- Put in some more comments so readers of the code knows what
is going on.
ChangeLog v6->v7:
- Move the hacks around __SANITIZE_ADDRESS__ into this file
- Edit the commit message
- Rebase on the other v2 patches
---
arch/arm/boot/compressed/string.c | 19 +++++++++++++++++++
arch/arm/include/asm/string.h | 21 +++++++++++++++++++++
arch/arm/kernel/head-common.S | 4 ++--
arch/arm/lib/memcpy.S | 3 +++
arch/arm/lib/memmove.S | 5 ++++-
arch/arm/lib/memset.S | 3 +++
6 files changed, 52 insertions(+), 3 deletions(-)
diff --git a/arch/arm/boot/compressed/string.c b/arch/arm/boot/compressed/string.c
index ade5079bebbf..8c0fa276d994 100644
--- a/arch/arm/boot/compressed/string.c
+++ b/arch/arm/boot/compressed/string.c
@@ -7,6 +7,25 @@
#include <linux/string.h>
+/*
+ * The decompressor is built without KASan but uses the same redirects as the
+ * rest of the kernel when CONFIG_KASAN is enabled, defining e.g. memcpy()
+ * to __memcpy() but since we are not linking with the main kernel string
+ * library in the decompressor, that will lead to link failures.
+ *
+ * Undefine KASan's versions, define the wrapped functions and alias them to
+ * the right names so that when e.g. __memcpy() appear in the code, it will
+ * still be linked to this local version of memcpy().
+ */
+#ifdef CONFIG_KASAN
+#undef memcpy
+#undef memmove
+#undef memset
+void *__memcpy(void *__dest, __const void *__src, size_t __n) __alias(memcpy);
+void *__memmove(void *__dest, __const void *__src, size_t count) __alias(memmove);
+void *__memset(void *s, int c, size_t count) __alias(memset);
+#endif
+
void *memcpy(void *__dest, __const void *__src, size_t __n)
{
int i = 0;
diff --git a/arch/arm/include/asm/string.h b/arch/arm/include/asm/string.h
index 111a1d8a41dd..947f93037d87 100644
--- a/arch/arm/include/asm/string.h
+++ b/arch/arm/include/asm/string.h
@@ -5,6 +5,9 @@
/*
* We don't do inline string functions, since the
* optimised inline asm versions are not small.
+ *
+ * The __underscore versions of some functions are for KASan to be able
+ * to replace them with instrumented versions.
*/
#define __HAVE_ARCH_STRRCHR
@@ -15,15 +18,18 @@ extern char * strchr(const char * s, int c);
#define __HAVE_ARCH_MEMCPY
extern void * memcpy(void *, const void *, __kernel_size_t);
+extern void *__memcpy(void *dest, const void *src, __kernel_size_t n);
#define __HAVE_ARCH_MEMMOVE
extern void * memmove(void *, const void *, __kernel_size_t);
+extern void *__memmove(void *dest, const void *src, __kernel_size_t n);
#define __HAVE_ARCH_MEMCHR
extern void * memchr(const void *, int, __kernel_size_t);
#define __HAVE_ARCH_MEMSET
extern void * memset(void *, int, __kernel_size_t);
+extern void *__memset(void *s, int c, __kernel_size_t n);
#define __HAVE_ARCH_MEMSET32
extern void *__memset32(uint32_t *, uint32_t v, __kernel_size_t);
@@ -39,4 +45,19 @@ static inline void *memset64(uint64_t *p, uint64_t v, __kernel_size_t n)
return __memset64(p, v, n * 8, v >> 32);
}
+/*
+ * For files that are not instrumented (e.g. mm/slub.c) we
+ * must use non-instrumented versions of the mem*
+ * functions named __memcpy() etc. All such kernel code has
+ * been tagged with KASAN_SANITIZE_file.o = n, which means
+ * that the address sanitization argument isn't passed to the
+ * compiler, and __SANITIZE_ADDRESS__ is not set. As a result
+ * these defines kick in.
+ */
+#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)
+#define memcpy(dst, src, len) __memcpy(dst, src, len)
+#define memmove(dst, src, len) __memmove(dst, src, len)
+#define memset(s, c, n) __memset(s, c, n)
+#endif
+
#endif
diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S
index 4a3982812a40..6840c7c60a85 100644
--- a/arch/arm/kernel/head-common.S
+++ b/arch/arm/kernel/head-common.S
@@ -95,7 +95,7 @@ __mmap_switched:
THUMB( ldmia r4!, {r0, r1, r2, r3} )
THUMB( mov sp, r3 )
sub r2, r2, r1
- bl memcpy @ copy .data to RAM
+ bl __memcpy @ copy .data to RAM
#endif
ARM( ldmia r4!, {r0, r1, sp} )
@@ -103,7 +103,7 @@ __mmap_switched:
THUMB( mov sp, r3 )
sub r2, r1, r0
mov r1, #0
- bl memset @ clear .bss
+ bl __memset @ clear .bss
ldmia r4, {r0, r1, r2, r3}
str r9, [r0] @ Save processor ID
diff --git a/arch/arm/lib/memcpy.S b/arch/arm/lib/memcpy.S
index 09a333153dc6..ad4625d16e11 100644
--- a/arch/arm/lib/memcpy.S
+++ b/arch/arm/lib/memcpy.S
@@ -58,6 +58,8 @@
/* Prototype: void *memcpy(void *dest, const void *src, size_t n); */
+.weak memcpy
+ENTRY(__memcpy)
ENTRY(mmiocpy)
ENTRY(memcpy)
@@ -65,3 +67,4 @@ ENTRY(memcpy)
ENDPROC(memcpy)
ENDPROC(mmiocpy)
+ENDPROC(__memcpy)
diff --git a/arch/arm/lib/memmove.S b/arch/arm/lib/memmove.S
index b50e5770fb44..fd123ea5a5a4 100644
--- a/arch/arm/lib/memmove.S
+++ b/arch/arm/lib/memmove.S
@@ -24,12 +24,14 @@
* occurring in the opposite direction.
*/
+.weak memmove
+ENTRY(__memmove)
ENTRY(memmove)
UNWIND( .fnstart )
subs ip, r0, r1
cmphi r2, ip
- bls memcpy
+ bls __memcpy
stmfd sp!, {r0, r4, lr}
UNWIND( .fnend )
@@ -222,3 +224,4 @@ ENTRY(memmove)
18: backward_copy_shift push=24 pull=8
ENDPROC(memmove)
+ENDPROC(__memmove)
diff --git a/arch/arm/lib/memset.S b/arch/arm/lib/memset.S
index 6ca4535c47fb..0e7ff0423f50 100644
--- a/arch/arm/lib/memset.S
+++ b/arch/arm/lib/memset.S
@@ -13,6 +13,8 @@
.text
.align 5
+.weak memset
+ENTRY(__memset)
ENTRY(mmioset)
ENTRY(memset)
UNWIND( .fnstart )
@@ -132,6 +134,7 @@ UNWIND( .fnstart )
UNWIND( .fnend )
ENDPROC(memset)
ENDPROC(mmioset)
+ENDPROC(__memset)
ENTRY(__memset32)
UNWIND( .fnstart )
--
2.25.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 3/5 v8] ARM: Define the virtual space of KASan's shadow region
2020-05-07 12:45 [PATCH 0/5 v8] KASan for Arm Linus Walleij
2020-05-07 12:45 ` [PATCH 1/5 v8] ARM: Disable KASan instrumentation for some code Linus Walleij
2020-05-07 12:45 ` [PATCH 2/5 v8] ARM: Replace string mem* functions for KASan Linus Walleij
@ 2020-05-07 12:45 ` Linus Walleij
2020-05-07 17:28 ` Ard Biesheuvel
2020-05-07 12:45 ` [PATCH 4/5 v8] ARM: Initialize the mapping of KASan shadow memory Linus Walleij
` (2 subsequent siblings)
5 siblings, 1 reply; 8+ messages in thread
From: Linus Walleij @ 2020-05-07 12:45 UTC (permalink / raw)
To: Florian Fainelli, Abbott Liu, Russell King, Ard Biesheuvel,
Andrey Ryabinin
Cc: Arnd Bergmann, Ard Biesheuvel, Linus Walleij, kasan-dev,
Alexander Potapenko, linux-arm-kernel, Dmitry Vyukov
From: Abbott Liu <liuwenliang@huawei.com>
Define KASAN_SHADOW_OFFSET,KASAN_SHADOW_START and KASAN_SHADOW_END for
the Arm kernel address sanitizer. We are "stealing" lowmem (the 4GB
addressable by a 32bit architecture) out of the virtual address
space to use as shadow memory for KASan as follows:
+----+ 0xffffffff
| |\
| | |-> Static kernel image (vmlinux) BSS and page table
| |/
+----+ PAGE_OFFSET
| |\
| | |-> Loadable kernel modules virtual address space area
| |/
+----+ MODULES_VADDR = KASAN_SHADOW_END
| |\
| | |-> The shadow area of kernel virtual address.
| |/
+----+-> TASK_SIZE (start of kernel space) = KASAN_SHADOW_START the
| |\ shadow address of MODULES_VADDR
| | |
| | |
| | |-> The user space area in lowmem. The kernel address
| | | sanitizer do not use this space, nor does it map it.
| | |
| | |
| | |
| | |
| |/
------ 0
0 .. TASK_SIZE is the memory that can be used by shared
userspace/kernelspace. It us used for userspace processes and for
passing parameters and memory buffers in system calls etc. We do not
need to shadow this area.
KASAN_SHADOW_START:
This value begins with the MODULE_VADDR's shadow address. It is the
start of kernel virtual space. Since we have modules to load, we need
to cover also that area with shadow memory so we can find memory
bugs in modules.
KASAN_SHADOW_END
This value is the 0x100000000's shadow address: the mapping that would
be after the end of the kernel memory at 0xffffffff. It is the end of
kernel address sanitizer shadow area. It is also the start of the
module area.
KASAN_SHADOW_OFFSET:
This value is used to map an address to the corresponding shadow
address by the following formula:
shadow_addr = (address >> 3) + KASAN_SHADOW_OFFSET;
As you would expect, >> 3 is equal to dividing by 8, meaning each
byte in the shadow memory covers 8 bytes of kernel memory, so one
bit shadow memory per byte of kernel memory is used.
The KASAN_SHADOW_OFFSET is provided in a Kconfig option depending
on the VMSPLIT layout of the system: the kernel and userspace can
split up lowmem in different ways according to needs, so we calculate
the shadow offset depending on this.
When kasan is enabled, the definition of TASK_SIZE is not an 8-bit
rotated constant, so we need to modify the TASK_SIZE access code in the
*.s file.
The kernel and modules may use different amounts of memory,
according to the VMSPLIT configuration, which in turn
determines the PAGE_OFFSET.
We use the following KASAN_SHADOW_OFFSETs depending on how the
virtual memory is split up:
- 0x1f000000 if we have 1G userspace / 3G kernelspace split:
- The kernel address space is 3G (0xc0000000)
- PAGE_OFFSET is then set to 0x40000000 so the kernel static
image (vmlinux) uses addresses 0x40000000 .. 0xffffffff
- On top of that we have the MODULES_VADDR which under
the worst case (using ARM instructions) is
PAGE_OFFSET - 16M (0x01000000) = 0x3f000000
so the modules use addresses 0x3f000000 .. 0x3fffffff
- So the addresses 0x3f000000 .. 0xffffffff need to be
covered with shadow memory. That is 0xc1000000 bytes
of memory.
- 1/8 of that is needed for its shadow memory, so
0x18200000 bytes of shadow memory is needed. We
"steal" that from the remaining lowmem.
- The KASAN_SHADOW_START becomes 0x26e00000, to
KASAN_SHADOW_END at 0x3effffff.
- Now we can calculate the KASAN_SHADOW_OFFSET for any
kernel address as 0x3f000000 needs to map to the first
byte of shadow memory and 0xffffffff needs to map to
the last byte of shadow memory. Since:
SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET
0x26e00000 = (0x3f000000 >> 3) + KASAN_SHADOW_OFFSET
KASAN_SHADOW_OFFSET = 0x26e00000 - (0x3f000000 >> 3)
KASAN_SHADOW_OFFSET = 0x26e00000 - 0x07e00000
KASAN_SHADOW_OFFSET = 0x1f000000
- 0x5f000000 if we have 2G userspace / 2G kernelspace split:
- The kernel space is 2G (0x80000000)
- PAGE_OFFSET is set to 0x80000000 so the kernel static
image uses 0x80000000 .. 0xffffffff.
- On top of that we have the MODULES_VADDR which under
the worst case (using ARM instructions) is
PAGE_OFFSET - 16M (0x01000000) = 0x7f000000
so the modules use addresses 0x7f000000 .. 0x7fffffff
- So the addresses 0x7f000000 .. 0xffffffff need to be
covered with shadow memory. That is 0x81000000 bytes
of memory.
- 1/8 of that is needed for its shadow memory, so
0x10200000 bytes of shadow memory is needed. We
"steal" that from the remaining lowmem.
- The KASAN_SHADOW_START becomes 0x6ee00000, to
KASAN_SHADOW_END at 0x7effffff.
- Now we can calculate the KASAN_SHADOW_OFFSET for any
kernel address as 0x7f000000 needs to map to the first
byte of shadow memory and 0xffffffff needs to map to
the last byte of shadow memory. Since:
SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET
0x6ee00000 = (0x7f000000 >> 3) + KASAN_SHADOW_OFFSET
KASAN_SHADOW_OFFSET = 0x6ee00000 - (0x7f000000 >> 3)
KASAN_SHADOW_OFFSET = 0x6ee00000 - 0x0fe00000
KASAN_SHADOW_OFFSET = 0x5f000000
- 0x9f000000 if we have 3G userspace / 1G kernelspace split,
and this is the default split for ARM:
- The kernel address space is 1GB (0x40000000)
- PAGE_OFFSET is set to 0xc0000000 so the kernel static
image uses 0xc0000000 .. 0xffffffff.
- On top of that we have the MODULES_VADDR which under
the worst case (using ARM instructions) is
PAGE_OFFSET - 16M (0x01000000) = 0xbf000000
so the modules use addresses 0xbf000000 .. 0xbfffffff
- So the addresses 0xbf000000 .. 0xffffffff need to be
covered with shadow memory. That is 0x41000000 bytes
of memory.
- 1/8 of that is needed for its shadow memory, so
0x08200000 bytes of shadow memory is needed. We
"steal" that from the remaining lowmem.
- The KASAN_SHADOW_START becomes 0xb6e00000, to
KASAN_SHADOW_END at 0xbfffffff.
- Now we can calculate the KASAN_SHADOW_OFFSET for any
kernel address as 0xbf000000 needs to map to the first
byte of shadow memory and 0xffffffff needs to map to
the last byte of shadow memory. Since:
SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET
0xb6e00000 = (0xbf000000 >> 3) + KASAN_SHADOW_OFFSET
KASAN_SHADOW_OFFSET = 0xb6e00000 - (0xbf000000 >> 3)
KASAN_SHADOW_OFFSET = 0xb6e00000 - 0x17e00000
KASAN_SHADOW_OFFSET = 0x9f000000
- 0x8f000000 if we have 3G userspace / 1G kernelspace with
full 1 GB low memory (VMSPLIT_3G_OPT):
- The kernel address space is 1GB (0x40000000)
- PAGE_OFFSET is set to 0xb0000000 so the kernel static
image uses 0xb0000000 .. 0xffffffff.
- On top of that we have the MODULES_VADDR which under
the worst case (using ARM instructions) is
PAGE_OFFSET - 16M (0x01000000) = 0xaf000000
so the modules use addresses 0xaf000000 .. 0xaffffff
- So the addresses 0xaf000000 .. 0xffffffff need to be
covered with shadow memory. That is 0x51000000 bytes
of memory.
- 1/8 of that is needed for its shadow memory, so
0x0a200000 bytes of shadow memory is needed. We
"steal" that from the remaining lowmem.
- The KASAN_SHADOW_START becomes 0xa4e00000, to
KASAN_SHADOW_END at 0xaeffffff.
- Now we can calculate the KASAN_SHADOW_OFFSET for any
kernel address as 0xaf000000 needs to map to the first
byte of shadow memory and 0xffffffff needs to map to
the last byte of shadow memory. Since:
SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET
0xa4e00000 = (0xaf000000 >> 3) + KASAN_SHADOW_OFFSET
KASAN_SHADOW_OFFSET = 0xa4e00000 - (0xaf000000 >> 3)
KASAN_SHADOW_OFFSET = 0xa4e00000 - 0x15e00000
KASAN_SHADOW_OFFSET = 0x8f000000
- The default value of 0xffffffff for KASAN_SHADOW_OFFSET
is an error value. We should always match one of the
above shadow offsets.
When we do this, TASK_SIZE will sometimes get a bit odd values
that will not fit into immediate mov assembly instructions.
To account for this, we need to rewrite some assembly using
TASK_SIZE like this:
- mov r1, #TASK_SIZE
+ ldr r1, =TASK_SIZE
or
- cmp r4, #TASK_SIZE
+ ldr r0, =TASK_SIZE
+ cmp r4, r0
this is done to avoid the immediate #TASK_SIZE that need to
fit into a limited number of bits.
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: kasan-dev@googlegroups.com
Reported-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Abbott Liu <liuwenliang@huawei.com>
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
ChangeLog v7->v8:
- Rewrote the PMD clearing code to take into account that
KASan may not always be adjacent to MODULES_VADDR: if we
compile for thumb, then there will be an 8 MB hole between
the shadow memory and MODULES_VADDR. Make this explicit and
use the KASAN defines with an explicit ifdef so it is clear
what is going on in the prepare_page_table().
- Patch memory.rst to reflect the location of KASan shadow
memory.
ChangeLog v6->v7:
- Use the SPDX license identifier.
- Rewrote the commit message and updates the illustration.
- Move KASAN_OFFSET Kconfig set-up into this patch and put it
right after PAGE_OFFSET so it is clear how this works, and
we have all defines in one patch.
- Added KASAN_SHADOW_OFFSET of 0x8f000000 for 3G_OPT.
See the calculation in the commit message.
- Updated the commit message with detailed information on
how KASAN_SHADOW_OFFSET is obtained for the different
VMSPLIT/PAGE_OFFSET options.
---
Documentation/arm/memory.rst | 5 ++
arch/arm/Kconfig | 9 ++++
arch/arm/include/asm/kasan_def.h | 81 ++++++++++++++++++++++++++++++++
arch/arm/include/asm/memory.h | 5 ++
arch/arm/kernel/entry-armv.S | 5 +-
arch/arm/kernel/entry-common.S | 9 ++--
arch/arm/mm/mmu.c | 18 +++++++
7 files changed, 127 insertions(+), 5 deletions(-)
create mode 100644 arch/arm/include/asm/kasan_def.h
diff --git a/Documentation/arm/memory.rst b/Documentation/arm/memory.rst
index 0521b4ce5c96..36bae90cfb1e 100644
--- a/Documentation/arm/memory.rst
+++ b/Documentation/arm/memory.rst
@@ -72,6 +72,11 @@ MODULES_VADDR MODULES_END-1 Kernel module space
Kernel modules inserted via insmod are
placed here using dynamic mappings.
+TASK_SIZE MODULES_VADDR-1 KASAn shadow memory when KASan is in use.
+ The range from MODULES_VADDR to the top
+ of the memory is shadowed here with 1 bit
+ per byte of memory.
+
00001000 TASK_SIZE-1 User space mappings
Per-thread mappings are placed here via
the mmap() system call.
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 66a04f6f4775..f6f2d3b202f5 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -1325,6 +1325,15 @@ config PAGE_OFFSET
default 0xB0000000 if VMSPLIT_3G_OPT
default 0xC0000000
+config KASAN_SHADOW_OFFSET
+ hex
+ depends on KASAN
+ default 0x1f000000 if PAGE_OFFSET=0x40000000
+ default 0x5f000000 if PAGE_OFFSET=0x80000000
+ default 0x9f000000 if PAGE_OFFSET=0xC0000000
+ default 0x8f000000 if PAGE_OFFSET=0xB0000000
+ default 0xffffffff
+
config NR_CPUS
int "Maximum number of CPUs (2-32)"
range 2 32
diff --git a/arch/arm/include/asm/kasan_def.h b/arch/arm/include/asm/kasan_def.h
new file mode 100644
index 000000000000..5739605aa7cf
--- /dev/null
+++ b/arch/arm/include/asm/kasan_def.h
@@ -0,0 +1,81 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * arch/arm/include/asm/kasan_def.h
+ *
+ * Copyright (c) 2018 Huawei Technologies Co., Ltd.
+ *
+ * Author: Abbott Liu <liuwenliang@huawei.com>
+ */
+
+#ifndef __ASM_KASAN_DEF_H
+#define __ASM_KASAN_DEF_H
+
+#ifdef CONFIG_KASAN
+
+/*
+ * Define KASAN_SHADOW_OFFSET,KASAN_SHADOW_START and KASAN_SHADOW_END for
+ * the Arm kernel address sanitizer. We are "stealing" lowmem (the 4GB
+ * addressable by a 32bit architecture) out of the virtual address
+ * space to use as shadow memory for KASan as follows:
+ *
+ * +----+ 0xffffffff
+ * | | \
+ * | | |-> Static kernel image (vmlinux) BSS and page table
+ * | |/
+ * +----+ PAGE_OFFSET
+ * | | \
+ * | | |-> Loadable kernel modules virtual address space area
+ * | |/
+ * +----+ MODULES_VADDR = KASAN_SHADOW_END
+ * | | \
+ * | | |-> The shadow area of kernel virtual address.
+ * | |/
+ * +----+-> TASK_SIZE (start of kernel space) = KASAN_SHADOW_START the
+ * | |\ shadow address of MODULES_VADDR
+ * | | |
+ * | | |
+ * | | |-> The user space area in lowmem. The kernel address
+ * | | | sanitizer do not use this space, nor does it map it.
+ * | | |
+ * | | |
+ * | | |
+ * | | |
+ * | |/
+ * ------ 0
+ *
+ * 1) KASAN_SHADOW_START
+ * This value begins with the MODULE_VADDR's shadow address. It is the
+ * start of kernel virtual space. Since we have modules to load, we need
+ * to cover also that area with shadow memory so we can find memory
+ * bugs in modules.
+ *
+ * 2) KASAN_SHADOW_END
+ * This value is the 0x100000000's shadow address: the mapping that would
+ * be after the end of the kernel memory at 0xffffffff. It is the end of
+ * kernel address sanitizer shadow area. It is also the start of the
+ * module area.
+ *
+ * 3) KASAN_SHADOW_OFFSET:
+ * This value is used to map an address to the corresponding shadow
+ * address by the following formula:
+ *
+ * shadow_addr = (address >> 3) + KASAN_SHADOW_OFFSET;
+ *
+ * As you would expect, >> 3 is equal to dividing by 8, meaning each
+ * byte in the shadow memory covers 8 bytes of kernel memory, so one
+ * bit shadow memory per byte of kernel memory is used.
+ *
+ * The KASAN_SHADOW_OFFSET is provided in a Kconfig option depending
+ * on the VMSPLIT layout of the system: the kernel and userspace can
+ * split up lowmem in different ways according to needs, so we calculate
+ * the shadow offset depending on this.
+ */
+
+#define KASAN_SHADOW_SCALE_SHIFT 3
+#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)
+#define KASAN_SHADOW_END ((UL(1) << (32 - KASAN_SHADOW_SCALE_SHIFT)) \
+ + KASAN_SHADOW_OFFSET)
+#define KASAN_SHADOW_START ((KASAN_SHADOW_END >> 3) + KASAN_SHADOW_OFFSET)
+
+#endif
+#endif
diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
index 99035b5891ef..5cfa9e5dc733 100644
--- a/arch/arm/include/asm/memory.h
+++ b/arch/arm/include/asm/memory.h
@@ -18,6 +18,7 @@
#ifdef CONFIG_NEED_MACH_MEMORY_H
#include <mach/memory.h>
#endif
+#include <asm/kasan_def.h>
/* PAGE_OFFSET - the virtual address of the start of the kernel image */
#define PAGE_OFFSET UL(CONFIG_PAGE_OFFSET)
@@ -28,7 +29,11 @@
* TASK_SIZE - the maximum size of a user space task.
* TASK_UNMAPPED_BASE - the lower boundary of the mmap VM area
*/
+#ifndef CONFIG_KASAN
#define TASK_SIZE (UL(CONFIG_PAGE_OFFSET) - UL(SZ_16M))
+#else
+#define TASK_SIZE (KASAN_SHADOW_START)
+#endif
#define TASK_UNMAPPED_BASE ALIGN(TASK_SIZE / 3, SZ_16M)
/*
diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S
index 77f54830554c..7548d38599ae 100644
--- a/arch/arm/kernel/entry-armv.S
+++ b/arch/arm/kernel/entry-armv.S
@@ -180,7 +180,7 @@ ENDPROC(__und_invalid)
get_thread_info tsk
ldr r0, [tsk, #TI_ADDR_LIMIT]
- mov r1, #TASK_SIZE
+ ldr r1, =TASK_SIZE
str r1, [tsk, #TI_ADDR_LIMIT]
str r0, [sp, #SVC_ADDR_LIMIT]
@@ -434,7 +434,8 @@ ENDPROC(__fiq_abt)
@ if it was interrupted in a critical region. Here we
@ perform a quick test inline since it should be false
@ 99.9999% of the time. The rest is done out of line.
- cmp r4, #TASK_SIZE
+ ldr r0, =TASK_SIZE
+ cmp r4, r0
blhs kuser_cmpxchg64_fixup
#endif
#endif
diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S
index 271cb8a1eba1..fee279e28a72 100644
--- a/arch/arm/kernel/entry-common.S
+++ b/arch/arm/kernel/entry-common.S
@@ -50,7 +50,8 @@ __ret_fast_syscall:
UNWIND(.cantunwind )
disable_irq_notrace @ disable interrupts
ldr r2, [tsk, #TI_ADDR_LIMIT]
- cmp r2, #TASK_SIZE
+ ldr r1, =TASK_SIZE
+ cmp r2, r1
blne addr_limit_check_failed
ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing
tst r1, #_TIF_SYSCALL_WORK | _TIF_WORK_MASK
@@ -87,7 +88,8 @@ __ret_fast_syscall:
#endif
disable_irq_notrace @ disable interrupts
ldr r2, [tsk, #TI_ADDR_LIMIT]
- cmp r2, #TASK_SIZE
+ ldr r1, =TASK_SIZE
+ cmp r2, r1
blne addr_limit_check_failed
ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing
tst r1, #_TIF_SYSCALL_WORK | _TIF_WORK_MASK
@@ -128,7 +130,8 @@ ret_slow_syscall:
disable_irq_notrace @ disable interrupts
ENTRY(ret_to_user_from_irq)
ldr r2, [tsk, #TI_ADDR_LIMIT]
- cmp r2, #TASK_SIZE
+ ldr r1, =TASK_SIZE
+ cmp r2, r1
blne addr_limit_check_failed
ldr r1, [tsk, #TI_FLAGS]
tst r1, #_TIF_WORK_MASK
diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
index ec8d0008bfa1..092bebd9ffc2 100644
--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -29,6 +29,7 @@
#include <asm/traps.h>
#include <asm/procinfo.h>
#include <asm/memory.h>
+#include <asm/kasan_def.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
@@ -1246,8 +1247,25 @@ static inline void prepare_page_table(void)
/*
* Clear out all the mappings below the kernel image.
*/
+#ifdef CONFIG_KASAN
+ /*
+ * KASan's shadow memory inserts itself between the TASK_SIZE
+ * and MODULES_VADDR. Do not clear the KASan shadow memory mappings.
+ */
+ for (addr = 0; addr < KASAN_SHADOW_START; addr += PMD_SIZE)
+ pmd_clear(pmd_off_k(addr));
+ /*
+ * Skip over the KASan shadow area. KASAN_SHADOW_END is sometimes
+ * equal to MODULES_VADDR and then we exit the pmd clearing. If we
+ * are using a thumb-compiled kernel, there there will be 8MB more
+ * to clear as KASan always offset to 16 MB below MODULES_VADDR.
+ */
+ for (addr = KASAN_SHADOW_END; addr < MODULES_VADDR; addr += PMD_SIZE)
+ pmd_clear(pmd_off_k(addr));
+#else
for (addr = 0; addr < MODULES_VADDR; addr += PMD_SIZE)
pmd_clear(pmd_off_k(addr));
+#endif
#ifdef CONFIG_XIP_KERNEL
/* The XIP kernel is mapped in the module area -- skip over it */
--
2.25.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH 3/5 v8] ARM: Define the virtual space of KASan's shadow region
2020-05-07 12:45 ` [PATCH 3/5 v8] ARM: Define the virtual space of KASan's shadow region Linus Walleij
@ 2020-05-07 17:28 ` Ard Biesheuvel
0 siblings, 0 replies; 8+ messages in thread
From: Ard Biesheuvel @ 2020-05-07 17:28 UTC (permalink / raw)
To: Linus Walleij
Cc: Florian Fainelli, Arnd Bergmann, Ard Biesheuvel, Abbott Liu,
Russell King, kasan-dev, Alexander Potapenko, Dmitry Vyukov,
Andrey Ryabinin, Linux ARM
On Thu, 7 May 2020 at 14:47, Linus Walleij <linus.walleij@linaro.org> wrote:
>
> From: Abbott Liu <liuwenliang@huawei.com>
>
> Define KASAN_SHADOW_OFFSET,KASAN_SHADOW_START and KASAN_SHADOW_END for
> the Arm kernel address sanitizer. We are "stealing" lowmem (the 4GB
> addressable by a 32bit architecture) out of the virtual address
> space to use as shadow memory for KASan as follows:
>
> +----+ 0xffffffff
> | |\
> | | |-> Static kernel image (vmlinux) BSS and page table
> | |/
> +----+ PAGE_OFFSET
> | |\
> | | |-> Loadable kernel modules virtual address space area
> | |/
> +----+ MODULES_VADDR = KASAN_SHADOW_END
> | |\
> | | |-> The shadow area of kernel virtual address.
> | |/
> +----+-> TASK_SIZE (start of kernel space) = KASAN_SHADOW_START the
> | |\ shadow address of MODULES_VADDR
> | | |
> | | |
> | | |-> The user space area in lowmem. The kernel address
> | | | sanitizer do not use this space, nor does it map it.
> | | |
> | | |
> | | |
> | | |
> | |/
> ------ 0
>
> 0 .. TASK_SIZE is the memory that can be used by shared
> userspace/kernelspace. It us used for userspace processes and for
> passing parameters and memory buffers in system calls etc. We do not
> need to shadow this area.
>
> KASAN_SHADOW_START:
> This value begins with the MODULE_VADDR's shadow address. It is the
> start of kernel virtual space. Since we have modules to load, we need
> to cover also that area with shadow memory so we can find memory
> bugs in modules.
>
> KASAN_SHADOW_END
> This value is the 0x100000000's shadow address: the mapping that would
> be after the end of the kernel memory at 0xffffffff. It is the end of
> kernel address sanitizer shadow area. It is also the start of the
> module area.
>
> KASAN_SHADOW_OFFSET:
> This value is used to map an address to the corresponding shadow
> address by the following formula:
>
> shadow_addr = (address >> 3) + KASAN_SHADOW_OFFSET;
>
> As you would expect, >> 3 is equal to dividing by 8, meaning each
> byte in the shadow memory covers 8 bytes of kernel memory, so one
> bit shadow memory per byte of kernel memory is used.
>
> The KASAN_SHADOW_OFFSET is provided in a Kconfig option depending
> on the VMSPLIT layout of the system: the kernel and userspace can
> split up lowmem in different ways according to needs, so we calculate
> the shadow offset depending on this.
>
> When kasan is enabled, the definition of TASK_SIZE is not an 8-bit
> rotated constant, so we need to modify the TASK_SIZE access code in the
> *.s file.
>
> The kernel and modules may use different amounts of memory,
> according to the VMSPLIT configuration, which in turn
> determines the PAGE_OFFSET.
>
> We use the following KASAN_SHADOW_OFFSETs depending on how the
> virtual memory is split up:
>
> - 0x1f000000 if we have 1G userspace / 3G kernelspace split:
> - The kernel address space is 3G (0xc0000000)
> - PAGE_OFFSET is then set to 0x40000000 so the kernel static
> image (vmlinux) uses addresses 0x40000000 .. 0xffffffff
> - On top of that we have the MODULES_VADDR which under
> the worst case (using ARM instructions) is
> PAGE_OFFSET - 16M (0x01000000) = 0x3f000000
> so the modules use addresses 0x3f000000 .. 0x3fffffff
> - So the addresses 0x3f000000 .. 0xffffffff need to be
> covered with shadow memory. That is 0xc1000000 bytes
> of memory.
> - 1/8 of that is needed for its shadow memory, so
> 0x18200000 bytes of shadow memory is needed. We
> "steal" that from the remaining lowmem.
> - The KASAN_SHADOW_START becomes 0x26e00000, to
> KASAN_SHADOW_END at 0x3effffff.
> - Now we can calculate the KASAN_SHADOW_OFFSET for any
> kernel address as 0x3f000000 needs to map to the first
> byte of shadow memory and 0xffffffff needs to map to
> the last byte of shadow memory. Since:
> SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET
> 0x26e00000 = (0x3f000000 >> 3) + KASAN_SHADOW_OFFSET
> KASAN_SHADOW_OFFSET = 0x26e00000 - (0x3f000000 >> 3)
> KASAN_SHADOW_OFFSET = 0x26e00000 - 0x07e00000
> KASAN_SHADOW_OFFSET = 0x1f000000
>
> - 0x5f000000 if we have 2G userspace / 2G kernelspace split:
> - The kernel space is 2G (0x80000000)
> - PAGE_OFFSET is set to 0x80000000 so the kernel static
> image uses 0x80000000 .. 0xffffffff.
> - On top of that we have the MODULES_VADDR which under
> the worst case (using ARM instructions) is
> PAGE_OFFSET - 16M (0x01000000) = 0x7f000000
> so the modules use addresses 0x7f000000 .. 0x7fffffff
> - So the addresses 0x7f000000 .. 0xffffffff need to be
> covered with shadow memory. That is 0x81000000 bytes
> of memory.
> - 1/8 of that is needed for its shadow memory, so
> 0x10200000 bytes of shadow memory is needed. We
> "steal" that from the remaining lowmem.
> - The KASAN_SHADOW_START becomes 0x6ee00000, to
> KASAN_SHADOW_END at 0x7effffff.
> - Now we can calculate the KASAN_SHADOW_OFFSET for any
> kernel address as 0x7f000000 needs to map to the first
> byte of shadow memory and 0xffffffff needs to map to
> the last byte of shadow memory. Since:
> SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET
> 0x6ee00000 = (0x7f000000 >> 3) + KASAN_SHADOW_OFFSET
> KASAN_SHADOW_OFFSET = 0x6ee00000 - (0x7f000000 >> 3)
> KASAN_SHADOW_OFFSET = 0x6ee00000 - 0x0fe00000
> KASAN_SHADOW_OFFSET = 0x5f000000
>
> - 0x9f000000 if we have 3G userspace / 1G kernelspace split,
> and this is the default split for ARM:
> - The kernel address space is 1GB (0x40000000)
> - PAGE_OFFSET is set to 0xc0000000 so the kernel static
> image uses 0xc0000000 .. 0xffffffff.
> - On top of that we have the MODULES_VADDR which under
> the worst case (using ARM instructions) is
> PAGE_OFFSET - 16M (0x01000000) = 0xbf000000
> so the modules use addresses 0xbf000000 .. 0xbfffffff
> - So the addresses 0xbf000000 .. 0xffffffff need to be
> covered with shadow memory. That is 0x41000000 bytes
> of memory.
> - 1/8 of that is needed for its shadow memory, so
> 0x08200000 bytes of shadow memory is needed. We
> "steal" that from the remaining lowmem.
> - The KASAN_SHADOW_START becomes 0xb6e00000, to
> KASAN_SHADOW_END at 0xbfffffff.
> - Now we can calculate the KASAN_SHADOW_OFFSET for any
> kernel address as 0xbf000000 needs to map to the first
> byte of shadow memory and 0xffffffff needs to map to
> the last byte of shadow memory. Since:
> SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET
> 0xb6e00000 = (0xbf000000 >> 3) + KASAN_SHADOW_OFFSET
> KASAN_SHADOW_OFFSET = 0xb6e00000 - (0xbf000000 >> 3)
> KASAN_SHADOW_OFFSET = 0xb6e00000 - 0x17e00000
> KASAN_SHADOW_OFFSET = 0x9f000000
>
> - 0x8f000000 if we have 3G userspace / 1G kernelspace with
> full 1 GB low memory (VMSPLIT_3G_OPT):
> - The kernel address space is 1GB (0x40000000)
> - PAGE_OFFSET is set to 0xb0000000 so the kernel static
> image uses 0xb0000000 .. 0xffffffff.
> - On top of that we have the MODULES_VADDR which under
> the worst case (using ARM instructions) is
> PAGE_OFFSET - 16M (0x01000000) = 0xaf000000
> so the modules use addresses 0xaf000000 .. 0xaffffff
> - So the addresses 0xaf000000 .. 0xffffffff need to be
> covered with shadow memory. That is 0x51000000 bytes
> of memory.
> - 1/8 of that is needed for its shadow memory, so
> 0x0a200000 bytes of shadow memory is needed. We
> "steal" that from the remaining lowmem.
> - The KASAN_SHADOW_START becomes 0xa4e00000, to
> KASAN_SHADOW_END at 0xaeffffff.
> - Now we can calculate the KASAN_SHADOW_OFFSET for any
> kernel address as 0xaf000000 needs to map to the first
> byte of shadow memory and 0xffffffff needs to map to
> the last byte of shadow memory. Since:
> SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET
> 0xa4e00000 = (0xaf000000 >> 3) + KASAN_SHADOW_OFFSET
> KASAN_SHADOW_OFFSET = 0xa4e00000 - (0xaf000000 >> 3)
> KASAN_SHADOW_OFFSET = 0xa4e00000 - 0x15e00000
> KASAN_SHADOW_OFFSET = 0x8f000000
>
> - The default value of 0xffffffff for KASAN_SHADOW_OFFSET
> is an error value. We should always match one of the
> above shadow offsets.
>
> When we do this, TASK_SIZE will sometimes get a bit odd values
> that will not fit into immediate mov assembly instructions.
> To account for this, we need to rewrite some assembly using
> TASK_SIZE like this:
>
> - mov r1, #TASK_SIZE
> + ldr r1, =TASK_SIZE
>
> or
>
> - cmp r4, #TASK_SIZE
> + ldr r0, =TASK_SIZE
> + cmp r4, r0
>
> this is done to avoid the immediate #TASK_SIZE that need to
> fit into a limited number of bits.
>
It would be good to note that the assembler will actually turn this
into a mov instruction if the immediate fits, so these LDRs will only
be true loads in the CONFIG_KASAN=y case.
> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> Cc: Alexander Potapenko <glider@google.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: kasan-dev@googlegroups.com
> Reported-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Signed-off-by: Abbott Liu <liuwenliang@huawei.com>
> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
> Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
> ---
> ChangeLog v7->v8:
> - Rewrote the PMD clearing code to take into account that
> KASan may not always be adjacent to MODULES_VADDR: if we
> compile for thumb, then there will be an 8 MB hole between
> the shadow memory and MODULES_VADDR. Make this explicit and
> use the KASAN defines with an explicit ifdef so it is clear
> what is going on in the prepare_page_table().
> - Patch memory.rst to reflect the location of KASan shadow
> memory.
> ChangeLog v6->v7:
> - Use the SPDX license identifier.
> - Rewrote the commit message and updates the illustration.
> - Move KASAN_OFFSET Kconfig set-up into this patch and put it
> right after PAGE_OFFSET so it is clear how this works, and
> we have all defines in one patch.
> - Added KASAN_SHADOW_OFFSET of 0x8f000000 for 3G_OPT.
> See the calculation in the commit message.
> - Updated the commit message with detailed information on
> how KASAN_SHADOW_OFFSET is obtained for the different
> VMSPLIT/PAGE_OFFSET options.
> ---
> Documentation/arm/memory.rst | 5 ++
> arch/arm/Kconfig | 9 ++++
> arch/arm/include/asm/kasan_def.h | 81 ++++++++++++++++++++++++++++++++
> arch/arm/include/asm/memory.h | 5 ++
> arch/arm/kernel/entry-armv.S | 5 +-
> arch/arm/kernel/entry-common.S | 9 ++--
> arch/arm/mm/mmu.c | 18 +++++++
> 7 files changed, 127 insertions(+), 5 deletions(-)
> create mode 100644 arch/arm/include/asm/kasan_def.h
>
> diff --git a/Documentation/arm/memory.rst b/Documentation/arm/memory.rst
> index 0521b4ce5c96..36bae90cfb1e 100644
> --- a/Documentation/arm/memory.rst
> +++ b/Documentation/arm/memory.rst
> @@ -72,6 +72,11 @@ MODULES_VADDR MODULES_END-1 Kernel module space
> Kernel modules inserted via insmod are
> placed here using dynamic mappings.
>
> +TASK_SIZE MODULES_VADDR-1 KASAn shadow memory when KASan is in use.
> + The range from MODULES_VADDR to the top
> + of the memory is shadowed here with 1 bit
> + per byte of memory.
> +
> 00001000 TASK_SIZE-1 User space mappings
> Per-thread mappings are placed here via
> the mmap() system call.
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 66a04f6f4775..f6f2d3b202f5 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -1325,6 +1325,15 @@ config PAGE_OFFSET
> default 0xB0000000 if VMSPLIT_3G_OPT
> default 0xC0000000
>
> +config KASAN_SHADOW_OFFSET
> + hex
> + depends on KASAN
> + default 0x1f000000 if PAGE_OFFSET=0x40000000
> + default 0x5f000000 if PAGE_OFFSET=0x80000000
> + default 0x9f000000 if PAGE_OFFSET=0xC0000000
> + default 0x8f000000 if PAGE_OFFSET=0xB0000000
> + default 0xffffffff
> +
> config NR_CPUS
> int "Maximum number of CPUs (2-32)"
> range 2 32
> diff --git a/arch/arm/include/asm/kasan_def.h b/arch/arm/include/asm/kasan_def.h
> new file mode 100644
> index 000000000000..5739605aa7cf
> --- /dev/null
> +++ b/arch/arm/include/asm/kasan_def.h
> @@ -0,0 +1,81 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * arch/arm/include/asm/kasan_def.h
> + *
> + * Copyright (c) 2018 Huawei Technologies Co., Ltd.
> + *
> + * Author: Abbott Liu <liuwenliang@huawei.com>
> + */
> +
> +#ifndef __ASM_KASAN_DEF_H
> +#define __ASM_KASAN_DEF_H
> +
> +#ifdef CONFIG_KASAN
> +
> +/*
> + * Define KASAN_SHADOW_OFFSET,KASAN_SHADOW_START and KASAN_SHADOW_END for
> + * the Arm kernel address sanitizer. We are "stealing" lowmem (the 4GB
> + * addressable by a 32bit architecture) out of the virtual address
> + * space to use as shadow memory for KASan as follows:
> + *
> + * +----+ 0xffffffff
> + * | | \
> + * | | |-> Static kernel image (vmlinux) BSS and page table
> + * | |/
> + * +----+ PAGE_OFFSET
> + * | | \
> + * | | |-> Loadable kernel modules virtual address space area
> + * | |/
> + * +----+ MODULES_VADDR = KASAN_SHADOW_END
> + * | | \
> + * | | |-> The shadow area of kernel virtual address.
> + * | |/
> + * +----+-> TASK_SIZE (start of kernel space) = KASAN_SHADOW_START the
> + * | |\ shadow address of MODULES_VADDR
> + * | | |
> + * | | |
> + * | | |-> The user space area in lowmem. The kernel address
> + * | | | sanitizer do not use this space, nor does it map it.
> + * | | |
> + * | | |
> + * | | |
> + * | | |
> + * | |/
> + * ------ 0
> + *
> + * 1) KASAN_SHADOW_START
> + * This value begins with the MODULE_VADDR's shadow address. It is the
> + * start of kernel virtual space. Since we have modules to load, we need
> + * to cover also that area with shadow memory so we can find memory
> + * bugs in modules.
> + *
> + * 2) KASAN_SHADOW_END
> + * This value is the 0x100000000's shadow address: the mapping that would
> + * be after the end of the kernel memory at 0xffffffff. It is the end of
> + * kernel address sanitizer shadow area. It is also the start of the
> + * module area.
> + *
> + * 3) KASAN_SHADOW_OFFSET:
> + * This value is used to map an address to the corresponding shadow
> + * address by the following formula:
> + *
> + * shadow_addr = (address >> 3) + KASAN_SHADOW_OFFSET;
> + *
> + * As you would expect, >> 3 is equal to dividing by 8, meaning each
> + * byte in the shadow memory covers 8 bytes of kernel memory, so one
> + * bit shadow memory per byte of kernel memory is used.
> + *
> + * The KASAN_SHADOW_OFFSET is provided in a Kconfig option depending
> + * on the VMSPLIT layout of the system: the kernel and userspace can
> + * split up lowmem in different ways according to needs, so we calculate
> + * the shadow offset depending on this.
> + */
> +
> +#define KASAN_SHADOW_SCALE_SHIFT 3
> +#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)
> +#define KASAN_SHADOW_END ((UL(1) << (32 - KASAN_SHADOW_SCALE_SHIFT)) \
> + + KASAN_SHADOW_OFFSET)
> +#define KASAN_SHADOW_START ((KASAN_SHADOW_END >> 3) + KASAN_SHADOW_OFFSET)
> +
> +#endif
> +#endif
> diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
> index 99035b5891ef..5cfa9e5dc733 100644
> --- a/arch/arm/include/asm/memory.h
> +++ b/arch/arm/include/asm/memory.h
> @@ -18,6 +18,7 @@
> #ifdef CONFIG_NEED_MACH_MEMORY_H
> #include <mach/memory.h>
> #endif
> +#include <asm/kasan_def.h>
>
> /* PAGE_OFFSET - the virtual address of the start of the kernel image */
> #define PAGE_OFFSET UL(CONFIG_PAGE_OFFSET)
> @@ -28,7 +29,11 @@
> * TASK_SIZE - the maximum size of a user space task.
> * TASK_UNMAPPED_BASE - the lower boundary of the mmap VM area
> */
> +#ifndef CONFIG_KASAN
> #define TASK_SIZE (UL(CONFIG_PAGE_OFFSET) - UL(SZ_16M))
> +#else
> +#define TASK_SIZE (KASAN_SHADOW_START)
> +#endif
> #define TASK_UNMAPPED_BASE ALIGN(TASK_SIZE / 3, SZ_16M)
>
> /*
> diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S
> index 77f54830554c..7548d38599ae 100644
> --- a/arch/arm/kernel/entry-armv.S
> +++ b/arch/arm/kernel/entry-armv.S
> @@ -180,7 +180,7 @@ ENDPROC(__und_invalid)
>
> get_thread_info tsk
> ldr r0, [tsk, #TI_ADDR_LIMIT]
> - mov r1, #TASK_SIZE
> + ldr r1, =TASK_SIZE
> str r1, [tsk, #TI_ADDR_LIMIT]
> str r0, [sp, #SVC_ADDR_LIMIT]
>
> @@ -434,7 +434,8 @@ ENDPROC(__fiq_abt)
> @ if it was interrupted in a critical region. Here we
> @ perform a quick test inline since it should be false
> @ 99.9999% of the time. The rest is done out of line.
> - cmp r4, #TASK_SIZE
> + ldr r0, =TASK_SIZE
> + cmp r4, r0
> blhs kuser_cmpxchg64_fixup
> #endif
> #endif
> diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S
> index 271cb8a1eba1..fee279e28a72 100644
> --- a/arch/arm/kernel/entry-common.S
> +++ b/arch/arm/kernel/entry-common.S
> @@ -50,7 +50,8 @@ __ret_fast_syscall:
> UNWIND(.cantunwind )
> disable_irq_notrace @ disable interrupts
> ldr r2, [tsk, #TI_ADDR_LIMIT]
> - cmp r2, #TASK_SIZE
> + ldr r1, =TASK_SIZE
> + cmp r2, r1
> blne addr_limit_check_failed
> ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing
> tst r1, #_TIF_SYSCALL_WORK | _TIF_WORK_MASK
> @@ -87,7 +88,8 @@ __ret_fast_syscall:
> #endif
> disable_irq_notrace @ disable interrupts
> ldr r2, [tsk, #TI_ADDR_LIMIT]
> - cmp r2, #TASK_SIZE
> + ldr r1, =TASK_SIZE
> + cmp r2, r1
> blne addr_limit_check_failed
> ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing
> tst r1, #_TIF_SYSCALL_WORK | _TIF_WORK_MASK
> @@ -128,7 +130,8 @@ ret_slow_syscall:
> disable_irq_notrace @ disable interrupts
> ENTRY(ret_to_user_from_irq)
> ldr r2, [tsk, #TI_ADDR_LIMIT]
> - cmp r2, #TASK_SIZE
> + ldr r1, =TASK_SIZE
> + cmp r2, r1
> blne addr_limit_check_failed
> ldr r1, [tsk, #TI_FLAGS]
> tst r1, #_TIF_WORK_MASK
> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> index ec8d0008bfa1..092bebd9ffc2 100644
> --- a/arch/arm/mm/mmu.c
> +++ b/arch/arm/mm/mmu.c
> @@ -29,6 +29,7 @@
> #include <asm/traps.h>
> #include <asm/procinfo.h>
> #include <asm/memory.h>
> +#include <asm/kasan_def.h>
>
> #include <asm/mach/arch.h>
> #include <asm/mach/map.h>
> @@ -1246,8 +1247,25 @@ static inline void prepare_page_table(void)
> /*
> * Clear out all the mappings below the kernel image.
> */
> +#ifdef CONFIG_KASAN
> + /*
> + * KASan's shadow memory inserts itself between the TASK_SIZE
> + * and MODULES_VADDR. Do not clear the KASan shadow memory mappings.
> + */
> + for (addr = 0; addr < KASAN_SHADOW_START; addr += PMD_SIZE)
> + pmd_clear(pmd_off_k(addr));
> + /*
> + * Skip over the KASan shadow area. KASAN_SHADOW_END is sometimes
> + * equal to MODULES_VADDR and then we exit the pmd clearing. If we
> + * are using a thumb-compiled kernel, there there will be 8MB more
> + * to clear as KASan always offset to 16 MB below MODULES_VADDR.
> + */
> + for (addr = KASAN_SHADOW_END; addr < MODULES_VADDR; addr += PMD_SIZE)
> + pmd_clear(pmd_off_k(addr));
> +#else
> for (addr = 0; addr < MODULES_VADDR; addr += PMD_SIZE)
> pmd_clear(pmd_off_k(addr));
> +#endif
>
> #ifdef CONFIG_XIP_KERNEL
> /* The XIP kernel is mapped in the module area -- skip over it */
> --
> 2.25.4
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 4/5 v8] ARM: Initialize the mapping of KASan shadow memory
2020-05-07 12:45 [PATCH 0/5 v8] KASan for Arm Linus Walleij
` (2 preceding siblings ...)
2020-05-07 12:45 ` [PATCH 3/5 v8] ARM: Define the virtual space of KASan's shadow region Linus Walleij
@ 2020-05-07 12:45 ` Linus Walleij
2020-05-07 12:45 ` [PATCH 5/5 v8] ARM: Enable KASan for ARM Linus Walleij
2020-05-07 17:29 ` [PATCH 0/5 v8] KASan for Arm Ard Biesheuvel
5 siblings, 0 replies; 8+ messages in thread
From: Linus Walleij @ 2020-05-07 12:45 UTC (permalink / raw)
To: Florian Fainelli, Abbott Liu, Russell King, Ard Biesheuvel,
Andrey Ryabinin
Cc: Arnd Bergmann, Linus Walleij, kasan-dev, Alexander Potapenko,
linux-arm-kernel, Dmitry Vyukov
From: Andrey Ryabinin <aryabinin@virtuozzo.com>
This patch initializes KASan shadow region's page table and memory.
There are two stage for KASan initializing:
1. At early boot stage the whole shadow region is mapped to just
one physical page (kasan_zero_page). It is finished by the function
kasan_early_init which is called by __mmap_switched(arch/arm/kernel/
head-common.S)
---Andrey Ryabinin <aryabinin@virtuozzo.com>
2. After the calling of paging_init, we use kasan_zero_page as zero
shadow for some memory that KASan does not need to track, and we
allocate a new shadow space for the other memory that KASan need to
track. These issues are finished by the function kasan_init which is
call by setup_arch.
---Andrey Ryabinin <aryabinin@virtuozzo.com>
3. Add support ARM LPAE
If LPAE is enabled, KASan shadow region's mapping table need be copied
in the pgd_alloc() function.
---Abbott Liu <liuwenliang@huawei.com>
4. Change kasan_pte_populate,kasan_pmd_populate,kasan_pud_populate,
kasan_pgd_populate from .meminit.text section to .init.text section.
---Reported by: Florian Fainelli <f.fainelli@gmail.com>
---Signed off by: Abbott Liu <liuwenliang@huawei.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: kasan-dev@googlegroups.com
Co-Developed-by: Abbott Liu <liuwenliang@huawei.com>
Reported-by: Russell King - ARM Linux <linux@armlinux.org.uk>
Reported-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Abbott Liu <liuwenliang@huawei.com>
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
ChangeLog v7->v8:
- Rebased.
ChangeLog v6->v7:
- Use SPDX identifer for the license.
- Move the TTBR0 accessor calls into this patch.
---
arch/arm/include/asm/kasan.h | 32 +++
arch/arm/include/asm/pgalloc.h | 9 +-
arch/arm/include/asm/thread_info.h | 4 +
arch/arm/kernel/head-common.S | 3 +
arch/arm/kernel/setup.c | 2 +
arch/arm/mm/Makefile | 3 +
arch/arm/mm/kasan_init.c | 324 +++++++++++++++++++++++++++++
arch/arm/mm/pgd.c | 15 +-
8 files changed, 389 insertions(+), 3 deletions(-)
create mode 100644 arch/arm/include/asm/kasan.h
create mode 100644 arch/arm/mm/kasan_init.c
diff --git a/arch/arm/include/asm/kasan.h b/arch/arm/include/asm/kasan.h
new file mode 100644
index 000000000000..56b954db160e
--- /dev/null
+++ b/arch/arm/include/asm/kasan.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * arch/arm/include/asm/kasan.h
+ *
+ * Copyright (c) 2015 Samsung Electronics Co., Ltd.
+ * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
+ *
+ */
+
+#ifndef __ASM_KASAN_H
+#define __ASM_KASAN_H
+
+#ifdef CONFIG_KASAN
+
+#include <asm/kasan_def.h>
+
+#define KASAN_SHADOW_SCALE_SHIFT 3
+
+/*
+ * The compiler uses a shadow offset assuming that addresses start
+ * from 0. Kernel addresses don't start from 0, so shadow
+ * for kernel really starts from 'compiler's shadow offset' +
+ * ('kernel address space start' >> KASAN_SHADOW_SCALE_SHIFT)
+ */
+
+extern void kasan_init(void);
+
+#else
+static inline void kasan_init(void) { }
+#endif
+
+#endif
diff --git a/arch/arm/include/asm/pgalloc.h b/arch/arm/include/asm/pgalloc.h
index 069da393110c..d969f8058b26 100644
--- a/arch/arm/include/asm/pgalloc.h
+++ b/arch/arm/include/asm/pgalloc.h
@@ -21,6 +21,7 @@
#define _PAGE_KERNEL_TABLE (PMD_TYPE_TABLE | PMD_BIT4 | PMD_DOMAIN(DOMAIN_KERNEL))
#ifdef CONFIG_ARM_LPAE
+#define PGD_SIZE (PTRS_PER_PGD * sizeof(pgd_t))
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
{
@@ -39,14 +40,18 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
}
#else /* !CONFIG_ARM_LPAE */
+#define PGD_SIZE (PAGE_SIZE << 2)
/*
* Since we have only two-level page tables, these are trivial
*/
#define pmd_alloc_one(mm,addr) ({ BUG(); ((pmd_t *)2); })
#define pmd_free(mm, pmd) do { } while (0)
-#define pud_populate(mm,pmd,pte) BUG()
-
+#ifndef CONFIG_KASAN
+#define pud_populate(mm, pmd, pte) BUG()
+#else
+#define pud_populate(mm, pmd, pte) do { } while (0)
+#endif
#endif /* CONFIG_ARM_LPAE */
extern pgd_t *pgd_alloc(struct mm_struct *mm);
diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h
index 3609a6980c34..cf47cf9c4742 100644
--- a/arch/arm/include/asm/thread_info.h
+++ b/arch/arm/include/asm/thread_info.h
@@ -13,7 +13,11 @@
#include <asm/fpstate.h>
#include <asm/page.h>
+#ifdef CONFIG_KASAN
+#define THREAD_SIZE_ORDER 2
+#else
#define THREAD_SIZE_ORDER 1
+#endif
#define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER)
#define THREAD_START_SP (THREAD_SIZE - 8)
diff --git a/arch/arm/kernel/head-common.S b/arch/arm/kernel/head-common.S
index 6840c7c60a85..89c80154b9ef 100644
--- a/arch/arm/kernel/head-common.S
+++ b/arch/arm/kernel/head-common.S
@@ -111,6 +111,9 @@ __mmap_switched:
str r8, [r2] @ Save atags pointer
cmp r3, #0
strne r10, [r3] @ Save control register values
+#ifdef CONFIG_KASAN
+ bl kasan_early_init
+#endif
mov lr, #0
b start_kernel
ENDPROC(__mmap_switched)
diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index d8e18cdd96d3..b0820847bb92 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -58,6 +58,7 @@
#include <asm/unwind.h>
#include <asm/memblock.h>
#include <asm/virt.h>
+#include <asm/kasan.h>
#include "atags.h"
@@ -1130,6 +1131,7 @@ void __init setup_arch(char **cmdline_p)
early_ioremap_reset();
paging_init(mdesc);
+ kasan_init();
request_standard_resources(mdesc);
if (mdesc->restart)
diff --git a/arch/arm/mm/Makefile b/arch/arm/mm/Makefile
index 99699c32d8a5..4536159bc8fa 100644
--- a/arch/arm/mm/Makefile
+++ b/arch/arm/mm/Makefile
@@ -113,3 +113,6 @@ obj-$(CONFIG_CACHE_L2X0_PMU) += cache-l2x0-pmu.o
obj-$(CONFIG_CACHE_XSC3L2) += cache-xsc3l2.o
obj-$(CONFIG_CACHE_TAUROS2) += cache-tauros2.o
obj-$(CONFIG_CACHE_UNIPHIER) += cache-uniphier.o
+
+KASAN_SANITIZE_kasan_init.o := n
+obj-$(CONFIG_KASAN) += kasan_init.o
diff --git a/arch/arm/mm/kasan_init.c b/arch/arm/mm/kasan_init.c
new file mode 100644
index 000000000000..043b4f33380e
--- /dev/null
+++ b/arch/arm/mm/kasan_init.c
@@ -0,0 +1,324 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * This file contains kasan initialization code for ARM.
+ *
+ * Copyright (c) 2018 Samsung Electronics Co., Ltd.
+ * Author: Andrey Ryabinin <ryabinin.a.a@gmail.com>
+ */
+
+#define pr_fmt(fmt) "kasan: " fmt
+#include <linux/kasan.h>
+#include <linux/kernel.h>
+#include <linux/memblock.h>
+#include <linux/sched/task.h>
+#include <linux/start_kernel.h>
+#include <asm/cputype.h>
+#include <asm/highmem.h>
+#include <asm/mach/map.h>
+#include <asm/memory.h>
+#include <asm/page.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/procinfo.h>
+#include <asm/proc-fns.h>
+#include <asm/tlbflush.h>
+#include <asm/cp15.h>
+
+#include "mm.h"
+
+static pgd_t tmp_pgd_table[PTRS_PER_PGD] __initdata __aligned(PGD_SIZE);
+
+pmd_t tmp_pmd_table[PTRS_PER_PMD] __page_aligned_bss;
+
+static __init void *kasan_alloc_block(size_t size, int node)
+{
+ return memblock_alloc_try_nid(size, size, __pa(MAX_DMA_ADDRESS),
+ MEMBLOCK_ALLOC_KASAN, node);
+}
+
+static void __init kasan_early_pmd_populate(unsigned long start,
+ unsigned long end, pud_t *pud)
+{
+ unsigned long addr;
+ unsigned long next;
+ pmd_t *pmd;
+
+ pmd = pmd_offset(pud, start);
+ for (addr = start; addr < end;) {
+ pmd_populate_kernel(&init_mm, pmd, kasan_early_shadow_pte);
+ next = pmd_addr_end(addr, end);
+ addr = next;
+ flush_pmd_entry(pmd);
+ pmd++;
+ }
+}
+
+static void __init kasan_early_pud_populate(unsigned long start,
+ unsigned long end, pgd_t *pgd)
+{
+ unsigned long addr;
+ unsigned long next;
+ pud_t *pud;
+
+ pud = pud_offset(pgd, start);
+ for (addr = start; addr < end;) {
+ next = pud_addr_end(addr, end);
+ kasan_early_pmd_populate(addr, next, pud);
+ addr = next;
+ pud++;
+ }
+}
+
+void __init kasan_map_early_shadow(pgd_t *pgdp)
+{
+ int i;
+ unsigned long start = KASAN_SHADOW_START;
+ unsigned long end = KASAN_SHADOW_END;
+ unsigned long addr;
+ unsigned long next;
+ pgd_t *pgd;
+
+ for (i = 0; i < PTRS_PER_PTE; i++)
+ set_pte_at(&init_mm, KASAN_SHADOW_START + i*PAGE_SIZE,
+ &kasan_early_shadow_pte[i], pfn_pte(
+ virt_to_pfn(kasan_early_shadow_page),
+ __pgprot(_L_PTE_DEFAULT | L_PTE_DIRTY
+ | L_PTE_XN)));
+
+ pgd = pgd_offset_k(start);
+ for (addr = start; addr < end;) {
+ next = pgd_addr_end(addr, end);
+ kasan_early_pud_populate(addr, next, pgd);
+ addr = next;
+ pgd++;
+ }
+}
+
+extern struct proc_info_list *lookup_processor_type(unsigned int);
+
+void __init kasan_early_init(void)
+{
+ struct proc_info_list *list;
+
+ /*
+ * locate processor in the list of supported processor
+ * types. The linker builds this table for us from the
+ * entries in arch/arm/mm/proc-*.S
+ */
+ list = lookup_processor_type(read_cpuid_id());
+ if (list) {
+#ifdef MULTI_CPU
+ processor = *list->proc;
+#endif
+ }
+
+ BUILD_BUG_ON((KASAN_SHADOW_END - (1UL << 29)) != KASAN_SHADOW_OFFSET);
+ kasan_map_early_shadow(swapper_pg_dir);
+}
+
+static void __init clear_pgds(unsigned long start,
+ unsigned long end)
+{
+ for (; start && start < end; start += PMD_SIZE)
+ pmd_clear(pmd_off_k(start));
+}
+
+pte_t * __init kasan_pte_populate(pmd_t *pmd, unsigned long addr, int node)
+{
+ pte_t *pte = pte_offset_kernel(pmd, addr);
+
+ if (pte_none(*pte)) {
+ pte_t entry;
+ void *p = kasan_alloc_block(PAGE_SIZE, node);
+
+ if (!p)
+ return NULL;
+ entry = pfn_pte(virt_to_pfn(p),
+ __pgprot(pgprot_val(PAGE_KERNEL)));
+ set_pte_at(&init_mm, addr, pte, entry);
+ }
+ return pte;
+}
+
+pmd_t * __init kasan_pmd_populate(pud_t *pud, unsigned long addr, int node)
+{
+ pmd_t *pmd = pmd_offset(pud, addr);
+
+ if (pmd_none(*pmd)) {
+ void *p = kasan_alloc_block(PAGE_SIZE, node);
+
+ if (!p)
+ return NULL;
+ pmd_populate_kernel(&init_mm, pmd, p);
+ }
+ return pmd;
+}
+
+pud_t * __init kasan_pud_populate(pgd_t *pgd, unsigned long addr, int node)
+{
+ pud_t *pud = pud_offset(pgd, addr);
+
+ /*
+ * On non-LPAE systems using just 2-level page tables pud_none()
+ * will alwats be zero and we bail out of here.
+ */
+ if (pud_none(*pud)) {
+ void *p = kasan_alloc_block(PAGE_SIZE, node);
+
+ if (!p)
+ return NULL;
+ pr_err("populating pud addr %lx\n", addr);
+ pud_populate(&init_mm, pud, p);
+ }
+ return pud;
+}
+
+pgd_t * __init kasan_pgd_populate(unsigned long addr, int node)
+{
+ pgd_t *pgd = pgd_offset_k(addr);
+
+ if (pgd_none(*pgd)) {
+ void *p = kasan_alloc_block(PAGE_SIZE, node);
+
+ if (!p)
+ return NULL;
+ pgd_populate(&init_mm, pgd, p);
+ }
+ return pgd;
+}
+
+static int __init create_mapping(unsigned long start, unsigned long end,
+ int node)
+{
+ unsigned long addr = start;
+ pgd_t *pgd;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ pr_info("populating shadow for %lx, %lx\n", start, end);
+
+ for (; addr < end; addr += PAGE_SIZE) {
+ pgd = kasan_pgd_populate(addr, node);
+ if (!pgd)
+ return -ENOMEM;
+
+ pud = kasan_pud_populate(pgd, addr, node);
+ if (!pud)
+ return -ENOMEM;
+
+ pmd = kasan_pmd_populate(pud, addr, node);
+ if (!pmd)
+ return -ENOMEM;
+
+ pte = kasan_pte_populate(pmd, addr, node);
+ if (!pte)
+ return -ENOMEM;
+ }
+ return 0;
+}
+
+/*
+ * TTBR0 accessors
+ */
+#define TTBR0_32 __ACCESS_CP15(c2, 0, c0, 0)
+#define TTBR0_64 __ACCESS_CP15_64(0, c2)
+
+static inline void set_ttbr0(u64 val)
+{
+ if (IS_ENABLED(CONFIG_ARM_LPAE))
+ write_sysreg(val, TTBR0_64);
+ else
+ write_sysreg(val, TTBR0_32);
+}
+
+static inline u64 get_ttbr0(void)
+{
+ if (IS_ENABLED(CONFIG_ARM_LPAE))
+ return read_sysreg(TTBR0_64);
+ else
+ return read_sysreg(TTBR0_32);
+}
+
+void __init kasan_init(void)
+{
+ struct memblock_region *reg;
+ u64 orig_ttbr0;
+ int i;
+
+ /*
+ * We are going to perform proper setup of shadow memory.
+ *
+ * At first we should unmap early shadow (clear_pgds() call bellow).
+ * However, instrumented code couldn't execute without shadow memory.
+ * tmp_pgd_table and tmp_pmd_table used to keep the early shadow memory
+ * mapped until the full shadow setup is finished.
+ */
+ orig_ttbr0 = get_ttbr0();
+
+#ifdef CONFIG_ARM_LPAE
+ memcpy(tmp_pmd_table,
+ pgd_page_vaddr(*pgd_offset_k(KASAN_SHADOW_START)),
+ sizeof(tmp_pmd_table));
+ memcpy(tmp_pgd_table, swapper_pg_dir, sizeof(tmp_pgd_table));
+ set_pgd(&tmp_pgd_table[pgd_index(KASAN_SHADOW_START)],
+ __pgd(__pa(tmp_pmd_table) | PMD_TYPE_TABLE | L_PGD_SWAPPER));
+ set_ttbr0(__pa(tmp_pgd_table));
+#else
+ memcpy(tmp_pgd_table, swapper_pg_dir, sizeof(tmp_pgd_table));
+ set_ttbr0((u64)__pa(tmp_pgd_table));
+#endif
+ flush_cache_all();
+ local_flush_bp_all();
+ local_flush_tlb_all();
+
+ clear_pgds(KASAN_SHADOW_START, KASAN_SHADOW_END);
+
+ kasan_populate_early_shadow(kasan_mem_to_shadow((void *)VMALLOC_START),
+ kasan_mem_to_shadow((void *)-1UL) + 1);
+
+ for_each_memblock(memory, reg) {
+ void *start = __va(reg->base);
+ void *end = __va(reg->base + reg->size);
+
+ if (reg->base + reg->size > arm_lowmem_limit)
+ end = __va(arm_lowmem_limit);
+ if (start >= end)
+ break;
+
+ create_mapping((unsigned long)kasan_mem_to_shadow(start),
+ (unsigned long)kasan_mem_to_shadow(end),
+ NUMA_NO_NODE);
+ }
+
+ /*
+ * 1. The module global variables are in MODULES_VADDR ~ MODULES_END,
+ * so we need to map this area.
+ * 2. PKMAP_BASE ~ PKMAP_BASE+PMD_SIZE's shadow and MODULES_VADDR
+ * ~ MODULES_END's shadow is in the same PMD_SIZE, so we can't
+ * use kasan_populate_zero_shadow.
+ */
+ create_mapping(
+ (unsigned long)kasan_mem_to_shadow((void *)MODULES_VADDR),
+ (unsigned long)kasan_mem_to_shadow((void *)(PKMAP_BASE +
+ PMD_SIZE)),
+ NUMA_NO_NODE);
+
+ /*
+ * KAsan may reuse the contents of kasan_early_shadow_pte directly, so
+ * we should make sure that it maps the zero page read-only.
+ */
+ for (i = 0; i < PTRS_PER_PTE; i++)
+ set_pte_at(&init_mm, KASAN_SHADOW_START + i*PAGE_SIZE,
+ &kasan_early_shadow_pte[i],
+ pfn_pte(virt_to_pfn(kasan_early_shadow_page),
+ __pgprot(pgprot_val(PAGE_KERNEL)
+ | L_PTE_RDONLY)));
+ memset(kasan_early_shadow_page, 0, PAGE_SIZE);
+ set_ttbr0(orig_ttbr0);
+ flush_cache_all();
+ local_flush_bp_all();
+ local_flush_tlb_all();
+ pr_info("Kernel address sanitizer initialized\n");
+ init_task.kasan_depth = 0;
+}
diff --git a/arch/arm/mm/pgd.c b/arch/arm/mm/pgd.c
index 478bd2c6aa50..3a25c3fa6a92 100644
--- a/arch/arm/mm/pgd.c
+++ b/arch/arm/mm/pgd.c
@@ -61,7 +61,20 @@ pgd_t *pgd_alloc(struct mm_struct *mm)
new_pmd = pmd_alloc(mm, new_pud, 0);
if (!new_pmd)
goto no_pmd;
-#endif
+#ifdef CONFIG_KASAN
+ /*
+ * Copy PMD table for KASAN shadow mappings.
+ */
+ init_pgd = pgd_offset_k(TASK_SIZE);
+ init_pud = pud_offset(init_pgd, TASK_SIZE);
+ init_pmd = pmd_offset(init_pud, TASK_SIZE);
+ new_pmd = pmd_offset(new_pud, TASK_SIZE);
+ memcpy(new_pmd, init_pmd,
+ (pmd_index(MODULES_VADDR) - pmd_index(TASK_SIZE))
+ * sizeof(pmd_t));
+ clean_dcache_area(new_pmd, PTRS_PER_PMD * sizeof(pmd_t));
+#endif /* CONFIG_KASAN */
+#endif /* CONFIG_LPAE */
if (!vectors_high()) {
/*
--
2.25.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH 5/5 v8] ARM: Enable KASan for ARM
2020-05-07 12:45 [PATCH 0/5 v8] KASan for Arm Linus Walleij
` (3 preceding siblings ...)
2020-05-07 12:45 ` [PATCH 4/5 v8] ARM: Initialize the mapping of KASan shadow memory Linus Walleij
@ 2020-05-07 12:45 ` Linus Walleij
2020-05-07 17:29 ` [PATCH 0/5 v8] KASan for Arm Ard Biesheuvel
5 siblings, 0 replies; 8+ messages in thread
From: Linus Walleij @ 2020-05-07 12:45 UTC (permalink / raw)
To: Florian Fainelli, Abbott Liu, Russell King, Ard Biesheuvel,
Andrey Ryabinin
Cc: Arnd Bergmann, Linus Walleij, kasan-dev, Alexander Potapenko,
linux-arm-kernel, Andrey Ryabinin, Dmitry Vyukov
From: Andrey Ryabinin <ryabinin@virtuozzo.com>
This patch enables the kernel address sanitizer for ARM. XIP_KERNEL
has not been tested and is therefore not allowed for now.
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: kasan-dev@googlegroups.com
Acked-by: Dmitry Vyukov <dvyukov@google.com>
Signed-off-by: Abbott Liu <liuwenliang@huawei.com>
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
ChangeLog v7->v8:
- Moved the hacks to __ADDRESS_SANITIZE__ to the patch
replacing the memory access functions.
- Moved the definition of KASAN_OFFSET out of this patch
and to the patch that defines the virtual memory used by
KASan.
---
Documentation/dev-tools/kasan.rst | 4 ++--
arch/arm/Kconfig | 1 +
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst
index c652d740735d..0962365e1405 100644
--- a/Documentation/dev-tools/kasan.rst
+++ b/Documentation/dev-tools/kasan.rst
@@ -21,8 +21,8 @@ global variables yet.
Tag-based KASAN is only supported in Clang and requires version 7.0.0 or later.
-Currently generic KASAN is supported for the x86_64, arm64, xtensa, s390 and
-riscv architectures, and tag-based KASAN is supported only for arm64.
+Currently generic KASAN is supported for the x86_64, arm, arm64, xtensa, s390
+and riscv architectures, and tag-based KASAN is supported only for arm64.
Usage
-----
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index f6f2d3b202f5..f5d26cbe2f42 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -64,6 +64,7 @@ config ARM
select HAVE_ARCH_BITREVERSE if (CPU_32v7M || CPU_32v7) && !CPU_32v6
select HAVE_ARCH_JUMP_LABEL if !XIP_KERNEL && !CPU_ENDIAN_BE32 && MMU
select HAVE_ARCH_KGDB if !CPU_ENDIAN_BE32 && MMU
+ select HAVE_ARCH_KASAN if MMU && !XIP_KERNEL
select HAVE_ARCH_MMAP_RND_BITS if MMU
select HAVE_ARCH_SECCOMP_FILTER if AEABI && !OABI_COMPAT
select HAVE_ARCH_THREAD_STRUCT_WHITELIST
--
2.25.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH 0/5 v8] KASan for Arm
2020-05-07 12:45 [PATCH 0/5 v8] KASan for Arm Linus Walleij
` (4 preceding siblings ...)
2020-05-07 12:45 ` [PATCH 5/5 v8] ARM: Enable KASan for ARM Linus Walleij
@ 2020-05-07 17:29 ` Ard Biesheuvel
5 siblings, 0 replies; 8+ messages in thread
From: Ard Biesheuvel @ 2020-05-07 17:29 UTC (permalink / raw)
To: Linus Walleij
Cc: Florian Fainelli, Arnd Bergmann, Abbott Liu, Russell King,
Andrey Ryabinin, Linux ARM
On Thu, 7 May 2020 at 14:47, Linus Walleij <linus.walleij@linaro.org> wrote:
>
> It's time to repost this, now that I got it into better shape.
>
> Main changes:
> - Restore the series number to v8 following my previous
> "v1" which was actually v7, following Florians versions.
> What was I thinking. :(
> - Doesn't crash randomly in QEMU etc. This was because
> mmu.c was instrumented, and when we were lucky this did
> not poke around in memory that crashed us.
> - Handle the undefined symbols better (like suggested by
> Ard)
> - Fix up documentation.
> - Better patch split in logical chunks.
>
> Remaining issues:
> - Does it work for you?
> - Crash on ARMv5/v5 in v4wbi_flush_user_tlb_range+0x10/0x4c.
> I wonder why. If you know, please tell me, I naturally
> have to fix this.
> - The code in kasan_init.c to first back up TTBR0 and then
> restore it looks *fragile* even if it is working. I think
> we need to do something better in a per-proc file such
> as is done in cpu_v7_switch_mm() and other well organized
> MMU manipulation, I certainly feel this should be done
> like that.
>
> Abbott Liu (1):
> ARM: Define the virtual space of KASan's shadow region
>
> Andrey Ryabinin (4):
> ARM: Disable KASan instrumentation for some code
> ARM: Replace string mem* functions for KASan
> ARM: Initialize the mapping of KASan shadow memory
> ARM: Enable KASan for ARM
>
For the series,
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Tested-by: Ard Biesheuvel <ardb@kernel.org> # QEMU/KVM/mach-virt/LPAE/8G
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread