[PATCH v8 1/7] powerpc/mm: Implement set_memory() routines

[PATCH v8 1/7] powerpc/mm: Implement set_memory() routines

[Russell Currey]

The set_memory_{ro/rw/nx/x}() functions are required for STRICT_MODULE_RWX,
and are generally useful primitives to have.  This implementation is
designed to be completely generic across powerpc's many MMUs.

It's possible that this could be optimised to be faster for specific
MMUs, but the focus is on having a generic and safe implementation for
now.

This implementation does not handle cases where the caller is attempting
to change the mapping of the page it is executing from, or if another
CPU is concurrently using the page being altered.  These cases likely
shouldn't happen, but a more complex implementation with MMU-specific code
could safely handle them, so that is left as a TODO for now.

These functions do nothing if STRICT_KERNEL_RWX is not enabled.

Reviewed-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Russell Currey <ruscur@russell.cc>
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
---

 arch/powerpc/Kconfig                  |  1 +
 arch/powerpc/include/asm/set_memory.h | 32 +++++++++++
 arch/powerpc/mm/Makefile              |  2 +-
 arch/powerpc/mm/pageattr.c            | 81 +++++++++++++++++++++++++++
 4 files changed, 115 insertions(+), 1 deletion(-)
 create mode 100644 arch/powerpc/include/asm/set_memory.h
 create mode 100644 arch/powerpc/mm/pageattr.c

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 6f40af294685..399a4de28ff0 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -129,6 +129,7 @@ config PPC
 	select ARCH_HAS_PTE_SPECIAL
 	select ARCH_HAS_MEMBARRIER_CALLBACKS
 	select ARCH_HAS_SCALED_CPUTIME		if VIRT_CPU_ACCOUNTING_NATIVE && PPC_BOOK3S_64
+	select ARCH_HAS_SET_MEMORY
 	select ARCH_HAS_STRICT_KERNEL_RWX	if ((PPC_BOOK3S_64 || PPC32) && !HIBERNATION)
 	select ARCH_HAS_TICK_BROADCAST		if GENERIC_CLOCKEVENTS_BROADCAST
 	select ARCH_HAS_UACCESS_FLUSHCACHE
diff --git a/arch/powerpc/include/asm/set_memory.h b/arch/powerpc/include/asm/set_memory.h
new file mode 100644
index 000000000000..64011ea444b4
--- /dev/null
+++ b/arch/powerpc/include/asm/set_memory.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_POWERPC_SET_MEMORY_H
+#define _ASM_POWERPC_SET_MEMORY_H
+
+#define SET_MEMORY_RO	0
+#define SET_MEMORY_RW	1
+#define SET_MEMORY_NX	2
+#define SET_MEMORY_X	3
+
+int change_memory_attr(unsigned long addr, int numpages, long action);
+
+static inline int set_memory_ro(unsigned long addr, int numpages)
+{
+	return change_memory_attr(addr, numpages, SET_MEMORY_RO);
+}
+
+static inline int set_memory_rw(unsigned long addr, int numpages)
+{
+	return change_memory_attr(addr, numpages, SET_MEMORY_RW);
+}
+
+static inline int set_memory_nx(unsigned long addr, int numpages)
+{
+	return change_memory_attr(addr, numpages, SET_MEMORY_NX);
+}
+
+static inline int set_memory_x(unsigned long addr, int numpages)
+{
+	return change_memory_attr(addr, numpages, SET_MEMORY_X);
+}
+
+#endif
diff --git a/arch/powerpc/mm/Makefile b/arch/powerpc/mm/Makefile
index 5e147986400d..a998fdac52f9 100644
--- a/arch/powerpc/mm/Makefile
+++ b/arch/powerpc/mm/Makefile
@@ -5,7 +5,7 @@
 
 ccflags-$(CONFIG_PPC64)	:= $(NO_MINIMAL_TOC)
 
-obj-y				:= fault.o mem.o pgtable.o mmap.o \
+obj-y				:= fault.o mem.o pgtable.o mmap.o pageattr.o \
 				   init_$(BITS).o pgtable_$(BITS).o \
 				   pgtable-frag.o ioremap.o ioremap_$(BITS).o \
 				   init-common.o mmu_context.o drmem.o
diff --git a/arch/powerpc/mm/pageattr.c b/arch/powerpc/mm/pageattr.c
new file mode 100644
index 000000000000..2da3fbab6ff7
--- /dev/null
+++ b/arch/powerpc/mm/pageattr.c
@@ -0,0 +1,81 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * MMU-generic set_memory implementation for powerpc
+ *
+ * Copyright 2019, IBM Corporation.
+ */
+
+#include <linux/mm.h>
+#include <linux/set_memory.h>
+
+#include <asm/mmu.h>
+#include <asm/page.h>
+#include <asm/pgtable.h>
+
+
+/*
+ * Updates the attributes of a page in three steps:
+ *
+ * 1. invalidate the page table entry
+ * 2. flush the TLB
+ * 3. install the new entry with the updated attributes
+ *
+ * This is unsafe if the caller is attempting to change the mapping of the
+ * page it is executing from, or if another CPU is concurrently using the
+ * page being altered.
+ *
+ * TODO make the implementation resistant to this.
+ *
+ * NOTE: can be dangerous to call without STRICT_KERNEL_RWX
+ */
+static int change_page_attr(pte_t *ptep, unsigned long addr, void *data)
+{
+	long action = (long)data;
+	pte_t pte;
+
+	spin_lock(&init_mm.page_table_lock);
+
+	/* invalidate the PTE so it's safe to modify */
+	pte = ptep_get_and_clear(&init_mm, addr, ptep);
+	flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
+
+	/* modify the PTE bits as desired, then apply */
+	switch (action) {
+	case SET_MEMORY_RO:
+		pte = pte_wrprotect(pte);
+		break;
+	case SET_MEMORY_RW:
+		pte = pte_mkwrite(pte);
+		break;
+	case SET_MEMORY_NX:
+		pte = pte_exprotect(pte);
+		break;
+	case SET_MEMORY_X:
+		pte = pte_mkexec(pte);
+		break;
+	default:
+		WARN_ON_ONCE(1);
+		break;
+	}
+
+	set_pte_at(&init_mm, addr, ptep, pte);
+	spin_unlock(&init_mm.page_table_lock);
+
+	return 0;
+}
+
+int change_memory_attr(unsigned long addr, int numpages, long action)
+{
+	unsigned long start = ALIGN_DOWN(addr, PAGE_SIZE);
+	unsigned long sz = numpages * PAGE_SIZE;
+
+	if (!IS_ENABLED(CONFIG_STRICT_KERNEL_RWX))
+		return 0;
+
+	if (numpages <= 0)
+		return 0;
+
+	return apply_to_existing_page_range(&init_mm, start, sz,
+					    change_page_attr, (void *)action);
+}
-- 
2.26.0

[PATCH v8 2/7] powerpc/kprobes: Mark newly allocated probes as RO

[Russell Currey]

With CONFIG_STRICT_KERNEL_RWX=y and CONFIG_KPROBES=y, there will be one
W+X page at boot by default.  This can be tested with
CONFIG_PPC_PTDUMP=y and CONFIG_PPC_DEBUG_WX=y set, and checking the
kernel log during boot.

powerpc doesn't implement its own alloc() for kprobes like other
architectures do, but we couldn't immediately mark RO anyway since we do
a memcpy to the page we allocate later.  After that, nothing should be
allowed to modify the page, and write permissions are removed well
before the kprobe is armed.

The memcpy() would fail if >1 probes were allocated, so use
patch_instruction() instead which is safe for RO.

Reviewed-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Russell Currey <ruscur@russell.cc>
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
---
 arch/powerpc/kernel/kprobes.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/arch/powerpc/kernel/kprobes.c b/arch/powerpc/kernel/kprobes.c
index 81efb605113e..fa4502b4de35 100644
--- a/arch/powerpc/kernel/kprobes.c
+++ b/arch/powerpc/kernel/kprobes.c
@@ -24,6 +24,8 @@
 #include <asm/sstep.h>
 #include <asm/sections.h>
 #include <linux/uaccess.h>
+#include <linux/set_memory.h>
+#include <linux/vmalloc.h>
 
 DEFINE_PER_CPU(struct kprobe *, current_kprobe) = NULL;
 DEFINE_PER_CPU(struct kprobe_ctlblk, kprobe_ctlblk);
@@ -102,6 +104,16 @@ kprobe_opcode_t *kprobe_lookup_name(const char *name, unsigned int offset)
 	return addr;
 }
 
+void *alloc_insn_page(void)
+{
+	void *page = vmalloc_exec(PAGE_SIZE);
+
+	if (page)
+		set_memory_ro((unsigned long)page, 1);
+
+	return page;
+}
+
 int arch_prepare_kprobe(struct kprobe *p)
 {
 	int ret = 0;
@@ -124,11 +136,8 @@ int arch_prepare_kprobe(struct kprobe *p)
 	}
 
 	if (!ret) {
-		memcpy(p->ainsn.insn, p->addr,
-				MAX_INSN_SIZE * sizeof(kprobe_opcode_t));
+		patch_instruction(p->ainsn.insn, *p->addr);
 		p->opcode = *p->addr;
-		flush_icache_range((unsigned long)p->ainsn.insn,
-			(unsigned long)p->ainsn.insn + sizeof(kprobe_opcode_t));
 	}
 
 	p->ainsn.boostable = 0;
-- 
2.26.0

[PATCH v8 3/7] powerpc/mm/ptdump: debugfs handler for W+X checks at runtime

[Russell Currey]

Very rudimentary, just

	echo 1 > [debugfs]/check_wx_pages

and check the kernel log.  Useful for testing strict module RWX.

Updated the Kconfig entry to reflect this.

Also fixed a typo.

Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Russell Currey <ruscur@russell.cc>
---
 arch/powerpc/Kconfig.debug      |  6 ++++--
 arch/powerpc/mm/ptdump/ptdump.c | 21 ++++++++++++++++++++-
 2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/Kconfig.debug b/arch/powerpc/Kconfig.debug
index 0b063830eea8..e37960ef68c6 100644
--- a/arch/powerpc/Kconfig.debug
+++ b/arch/powerpc/Kconfig.debug
@@ -370,7 +370,7 @@ config PPC_PTDUMP
 	  If you are unsure, say N.
 
 config PPC_DEBUG_WX
-	bool "Warn on W+X mappings at boot"
+	bool "Warn on W+X mappings at boot & enable manual checks at runtime"
 	depends on PPC_PTDUMP && STRICT_KERNEL_RWX
 	help
 	  Generate a warning if any W+X mappings are found at boot.
@@ -384,7 +384,9 @@ config PPC_DEBUG_WX
 	  of other unfixed kernel bugs easier.
 
 	  There is no runtime or memory usage effect of this option
-	  once the kernel has booted up - it's a one time check.
+	  once the kernel has booted up, it only automatically checks once.
+
+	  Enables the "check_wx_pages" debugfs entry for checking at runtime.
 
 	  If in doubt, say "Y".
 
diff --git a/arch/powerpc/mm/ptdump/ptdump.c b/arch/powerpc/mm/ptdump/ptdump.c
index d92bb8ea229c..525ca5aeaa01 100644
--- a/arch/powerpc/mm/ptdump/ptdump.c
+++ b/arch/powerpc/mm/ptdump/ptdump.c
@@ -4,7 +4,7 @@
  *
  * This traverses the kernel pagetables and dumps the
  * information about the used sections of memory to
- * /sys/kernel/debug/kernel_pagetables.
+ * /sys/kernel/debug/kernel_page_tables.
  *
  * Derived from the arm64 implementation:
  * Copyright (c) 2014, The Linux Foundation, Laura Abbott.
@@ -413,6 +413,25 @@ void ptdump_check_wx(void)
 	else
 		pr_info("Checked W+X mappings: passed, no W+X pages found\n");
 }
+
+static int check_wx_debugfs_set(void *data, u64 val)
+{
+	if (val != 1ULL)
+		return -EINVAL;
+
+	ptdump_check_wx();
+
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(check_wx_fops, NULL, check_wx_debugfs_set, "%llu\n");
+
+static int ptdump_check_wx_init(void)
+{
+	return debugfs_create_file("check_wx_pages", 0200, NULL,
+				   NULL, &check_wx_fops) ? 0 : -ENOMEM;
+}
+device_initcall(ptdump_check_wx_init);
 #endif
 
 static int ptdump_init(void)
-- 
2.26.0

[PATCH v8 4/7] powerpc: Set ARCH_HAS_STRICT_MODULE_RWX

[Russell Currey]

To enable strict module RWX on powerpc, set:

    CONFIG_STRICT_MODULE_RWX=y

You should also have CONFIG_STRICT_KERNEL_RWX=y set to have any real
security benefit.

ARCH_HAS_STRICT_MODULE_RWX is set to require ARCH_HAS_STRICT_KERNEL_RWX.
This is due to a quirk in arch/Kconfig and arch/powerpc/Kconfig that
makes STRICT_MODULE_RWX *on by default* in configurations where
STRICT_KERNEL_RWX is *unavailable*.

Since this doesn't make much sense, and module RWX without kernel RWX
doesn't make much sense, having the same dependencies as kernel RWX
works around this problem.

Signed-off-by: Russell Currey <ruscur@russell.cc>
---
 arch/powerpc/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 399a4de28ff0..1488bb5f4179 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -131,6 +131,7 @@ config PPC
 	select ARCH_HAS_SCALED_CPUTIME		if VIRT_CPU_ACCOUNTING_NATIVE && PPC_BOOK3S_64
 	select ARCH_HAS_SET_MEMORY
 	select ARCH_HAS_STRICT_KERNEL_RWX	if ((PPC_BOOK3S_64 || PPC32) && !HIBERNATION)
+	select ARCH_HAS_STRICT_MODULE_RWX	if ARCH_HAS_STRICT_KERNEL_RWX
 	select ARCH_HAS_TICK_BROADCAST		if GENERIC_CLOCKEVENTS_BROADCAST
 	select ARCH_HAS_UACCESS_FLUSHCACHE
 	select ARCH_HAS_UACCESS_MCSAFE		if PPC64
-- 
2.26.0

[PATCH v8 5/7] powerpc/configs: Enable STRICT_MODULE_RWX in skiroot_defconfig

[Russell Currey]

skiroot_defconfig is the only powerpc defconfig with STRICT_KERNEL_RWX
enabled, and if you want memory protection for kernel text you'd want it
for modules too, so enable STRICT_MODULE_RWX there.

Acked-by: Joel Stanley <joel@joel.id.au>
Signed-off-by: Russell Currey <ruscur@russell.cc>
---
 arch/powerpc/configs/skiroot_defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/powerpc/configs/skiroot_defconfig b/arch/powerpc/configs/skiroot_defconfig
index 1b6bdad36b13..66d20dbe67b7 100644
--- a/arch/powerpc/configs/skiroot_defconfig
+++ b/arch/powerpc/configs/skiroot_defconfig
@@ -51,6 +51,7 @@ CONFIG_CMDLINE="console=tty0 console=hvc0 ipr.fast_reboot=1 quiet"
 # CONFIG_PPC_MEM_KEYS is not set
 CONFIG_JUMP_LABEL=y
 CONFIG_STRICT_KERNEL_RWX=y
+CONFIG_STRICT_MODULE_RWX=y
 CONFIG_MODULES=y
 CONFIG_MODULE_UNLOAD=y
 CONFIG_MODULE_SIG_FORCE=y
-- 
2.26.0

[PATCH v8 6/7] powerpc/mm: implement set_memory_attr()

[Russell Currey]

From: Christophe Leroy <christophe.leroy@c-s.fr>

In addition to the set_memory_xx() functions which allows to change
the memory attributes of not (yet) used memory regions, implement a
set_memory_attr() function to:
- set the final memory protection after init on currently used
kernel regions.
- enable/disable kernel memory regions in the scope of DEBUG_PAGEALLOC.

Unlike the set_memory_xx() which can act in three step as the regions
are unused, this function must modify 'on the fly' as the kernel is
executing from them. At the moment only PPC32 will use it and changing
page attributes on the fly is not an issue.

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Reported-by: kbuild test robot <lkp@intel.com>
[ruscur: cast "data" to unsigned long instead of int]
Signed-off-by: Russell Currey <ruscur@russell.cc>
---
 arch/powerpc/include/asm/set_memory.h |  2 ++
 arch/powerpc/mm/pageattr.c            | 33 +++++++++++++++++++++++++++
 2 files changed, 35 insertions(+)

diff --git a/arch/powerpc/include/asm/set_memory.h b/arch/powerpc/include/asm/set_memory.h
index 64011ea444b4..b040094f7920 100644
--- a/arch/powerpc/include/asm/set_memory.h
+++ b/arch/powerpc/include/asm/set_memory.h
@@ -29,4 +29,6 @@ static inline int set_memory_x(unsigned long addr, int numpages)
 	return change_memory_attr(addr, numpages, SET_MEMORY_X);
 }
 
+int set_memory_attr(unsigned long addr, int numpages, pgprot_t prot);
+
 #endif
diff --git a/arch/powerpc/mm/pageattr.c b/arch/powerpc/mm/pageattr.c
index 2da3fbab6ff7..2fde1b195c85 100644
--- a/arch/powerpc/mm/pageattr.c
+++ b/arch/powerpc/mm/pageattr.c
@@ -79,3 +79,36 @@ int change_memory_attr(unsigned long addr, int numpages, long action)
 	return apply_to_existing_page_range(&init_mm, start, sz,
 					    change_page_attr, (void *)action);
 }
+
+/*
+ * Set the attributes of a page:
+ *
+ * This function is used by PPC32 at the end of init to set final kernel memory
+ * protection. It includes changing the maping of the page it is executing from
+ * and data pages it is using.
+ */
+static int set_page_attr(pte_t *ptep, unsigned long addr, void *data)
+{
+	pgprot_t prot = __pgprot((unsigned long)data);
+
+	spin_lock(&init_mm.page_table_lock);
+
+	set_pte_at(&init_mm, addr, ptep, pte_modify(*ptep, prot));
+	flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
+
+	spin_unlock(&init_mm.page_table_lock);
+
+	return 0;
+}
+
+int set_memory_attr(unsigned long addr, int numpages, pgprot_t prot)
+{
+	unsigned long start = ALIGN_DOWN(addr, PAGE_SIZE);
+	unsigned long sz = numpages * PAGE_SIZE;
+
+	if (numpages <= 0)
+		return 0;
+
+	return apply_to_existing_page_range(&init_mm, start, sz, set_page_attr,
+					    (void *)pgprot_val(prot));
+}
-- 
2.26.0

[PATCH v8 7/7] powerpc/32: use set_memory_attr()

[Russell Currey]

From: Christophe Leroy <christophe.leroy@c-s.fr>

Use set_memory_attr() instead of the PPC32 specific change_page_attr()

change_page_attr() was checking that the address was not mapped by
blocks and was handling highmem, but that's unneeded because the
affected pages can't be in highmem and block mapping verification
is already done by the callers.

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
[ruscur: rebase on powerpc/merge with Christophe's new patches]
Signed-off-by: Russell Currey <ruscur@russell.cc>
---
v8: Rebase on powerpc/merge

 arch/powerpc/mm/pgtable_32.c | 60 ++++++------------------------------
 1 file changed, 10 insertions(+), 50 deletions(-)

diff --git a/arch/powerpc/mm/pgtable_32.c b/arch/powerpc/mm/pgtable_32.c
index f62de06e3d07..0d9d164fad26 100644
--- a/arch/powerpc/mm/pgtable_32.c
+++ b/arch/powerpc/mm/pgtable_32.c
@@ -23,6 +23,7 @@
 #include <linux/highmem.h>
 #include <linux/memblock.h>
 #include <linux/slab.h>
+#include <linux/set_memory.h>
 
 #include <asm/pgtable.h>
 #include <asm/pgalloc.h>
@@ -121,64 +122,20 @@ void __init mapin_ram(void)
 	}
 }
 
-static int __change_page_attr_noflush(struct page *page, pgprot_t prot)
-{
-	pte_t *kpte;
-	unsigned long address;
-
-	BUG_ON(PageHighMem(page));
-	address = (unsigned long)page_address(page);
-
-	if (v_block_mapped(address))
-		return 0;
-	kpte = virt_to_kpte(address);
-	if (!kpte)
-		return -EINVAL;
-	__set_pte_at(&init_mm, address, kpte, mk_pte(page, prot), 0);
-
-	return 0;
-}
-
-/*
- * Change the page attributes of an page in the linear mapping.
- *
- * THIS DOES NOTHING WITH BAT MAPPINGS, DEBUG USE ONLY
- */
-static int change_page_attr(struct page *page, int numpages, pgprot_t prot)
-{
-	int i, err = 0;
-	unsigned long flags;
-	struct page *start = page;
-
-	local_irq_save(flags);
-	for (i = 0; i < numpages; i++, page++) {
-		err = __change_page_attr_noflush(page, prot);
-		if (err)
-			break;
-	}
-	wmb();
-	local_irq_restore(flags);
-	flush_tlb_kernel_range((unsigned long)page_address(start),
-			       (unsigned long)page_address(page));
-	return err;
-}
-
 void mark_initmem_nx(void)
 {
-	struct page *page = virt_to_page(_sinittext);
 	unsigned long numpages = PFN_UP((unsigned long)_einittext) -
 				 PFN_DOWN((unsigned long)_sinittext);
 
 	if (v_block_mapped((unsigned long)_stext + 1))
 		mmu_mark_initmem_nx();
 	else
-		change_page_attr(page, numpages, PAGE_KERNEL);
+		set_memory_attr((unsigned long)_sinittext, numpages, PAGE_KERNEL);
 }
 
 #ifdef CONFIG_STRICT_KERNEL_RWX
 void mark_rodata_ro(void)
 {
-	struct page *page;
 	unsigned long numpages;
 
 	if (v_block_mapped((unsigned long)_sinittext)) {
@@ -187,20 +144,18 @@ void mark_rodata_ro(void)
 		return;
 	}
 
-	page = virt_to_page(_stext);
 	numpages = PFN_UP((unsigned long)_etext) -
 		   PFN_DOWN((unsigned long)_stext);
 
-	change_page_attr(page, numpages, PAGE_KERNEL_ROX);
+	set_memory_attr((unsigned long)_stext, numpages, PAGE_KERNEL_ROX);
 	/*
 	 * mark .rodata as read only. Use __init_begin rather than __end_rodata
 	 * to cover NOTES and EXCEPTION_TABLE.
 	 */
-	page = virt_to_page(__start_rodata);
 	numpages = PFN_UP((unsigned long)__init_begin) -
 		   PFN_DOWN((unsigned long)__start_rodata);
 
-	change_page_attr(page, numpages, PAGE_KERNEL_RO);
+	set_memory_attr((unsigned long)__start_rodata, numpages, PAGE_KERNEL_RO);
 
 	// mark_initmem_nx() should have already run by now
 	ptdump_check_wx();
@@ -210,9 +165,14 @@ void mark_rodata_ro(void)
 #ifdef CONFIG_DEBUG_PAGEALLOC
 void __kernel_map_pages(struct page *page, int numpages, int enable)
 {
+	unsigned long addr = (unsigned long)page_address(page);
+
 	if (PageHighMem(page))
 		return;
 
-	change_page_attr(page, numpages, enable ? PAGE_KERNEL : __pgprot(0));
+	if (enable)
+		set_memory_attr(addr, numpages, PAGE_KERNEL);
+	else
+		set_memory_attr(addr, numpages, __pgprot(0));
 }
 #endif /* CONFIG_DEBUG_PAGEALLOC */
-- 
2.26.0

Re: [PATCH v8 2/7] powerpc/kprobes: Mark newly allocated probes as RO

[Naveen N. Rao]

Russell Currey wrote:
> With CONFIG_STRICT_KERNEL_RWX=y and CONFIG_KPROBES=y, there will be one
> W+X page at boot by default.  This can be tested with
> CONFIG_PPC_PTDUMP=y and CONFIG_PPC_DEBUG_WX=y set, and checking the
> kernel log during boot.
> 
> powerpc doesn't implement its own alloc() for kprobes like other
> architectures do, but we couldn't immediately mark RO anyway since we do
> a memcpy to the page we allocate later.  After that, nothing should be
> allowed to modify the page, and write permissions are removed well
> before the kprobe is armed.
> 
> The memcpy() would fail if >1 probes were allocated, so use
> patch_instruction() instead which is safe for RO.
> 
> Reviewed-by: Daniel Axtens <dja@axtens.net>
> Signed-off-by: Russell Currey <ruscur@russell.cc>
> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
> ---
>  arch/powerpc/kernel/kprobes.c | 17 +++++++++++++----
>  1 file changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/kprobes.c b/arch/powerpc/kernel/kprobes.c
> index 81efb605113e..fa4502b4de35 100644
> --- a/arch/powerpc/kernel/kprobes.c
> +++ b/arch/powerpc/kernel/kprobes.c
> @@ -24,6 +24,8 @@
>  #include <asm/sstep.h>
>  #include <asm/sections.h>
>  #include <linux/uaccess.h>
> +#include <linux/set_memory.h>
> +#include <linux/vmalloc.h>
>  
>  DEFINE_PER_CPU(struct kprobe *, current_kprobe) = NULL;
>  DEFINE_PER_CPU(struct kprobe_ctlblk, kprobe_ctlblk);
> @@ -102,6 +104,16 @@ kprobe_opcode_t *kprobe_lookup_name(const char *name, unsigned int offset)
>  	return addr;
>  }
>  
> +void *alloc_insn_page(void)
> +{
> +	void *page = vmalloc_exec(PAGE_SIZE);
> +
> +	if (page)
> +		set_memory_ro((unsigned long)page, 1);
> +
> +	return page;
> +}
> +

This crashes for me with KPROBES_SANITY_TEST during the kretprobe test.  
It seems to be handling the kretprobe itself properly, but seems to 
crash on the return path. I haven't yet been able to work out what's 
going wrong.

[    0.517880] Kprobe smoke test: started
[    0.626680] Oops: Exception in kernel mode, sig: 4 [#1]
[    0.626708] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA PowerNV
[    0.626735] Modules linked in:
[    0.626760] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.0-06592-g2e64694b9137 #51
[    0.626795] NIP:  c008000000020000 LR: c00000000021ce34 CTR: c00000000021c5f8
[    0.626829] REGS: c0000000fd3e3860 TRAP: 0e40   Not tainted  (5.6.0-06592-g2e64694b9137)
[    0.626862] MSR:  9000000002089433 <SF,HV,VEC,EE,ME,SE,IR,DR,RI,LE>  CR: 28000284  XER: 00040000
[    0.626922] CFAR: c00000000000ef20 IRQMASK: 0 
[    0.626922] GPR00: c000000000052250 c0000000fd3e3af0 c000000002330200 000000002db8ad86 
[    0.626922] GPR04: 0000000000000000 c00000000006ba3c 0000000000000800 c0000000fd3a0000 
[    0.626922] GPR08: 0000000000000000 ffffffffaaaaaaab c0000000fd3a0000 0000000040000000 
[    0.626922] GPR12: c00000000021c5f0 c000000002520000 c000000000011790 0000000000000000 
[    0.626922] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[    0.626922] GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[    0.626922] GPR24: c0000000020034bc c0000000012068b8 c000000002062e50 c0000000fd2319a0 
[    0.626922] GPR28: c000000000f5ebb0 0000000000000000 c0000000021bc278 c000000002458540 
[    0.627234] NIP [c008000000020000] 0xc008000000020000
[    0.627264] LR [c00000000021ce34] init_test_probes+0x424/0x560
[    0.627291] Call Trace:
[    0.627313] [c0000000fd3e3af0] [c00000000021ce34] init_test_probes+0x424/0x560 (unreliable)
[    0.627356] [c0000000fd3e3b90] [c00000000202de2c] init_kprobes+0x1a8/0x1c8
[    0.627392] [c0000000fd3e3c00] [c000000000011140] do_one_initcall+0x60/0x2b0
[    0.627432] [c0000000fd3e3cd0] [c000000002004674] kernel_init_freeable+0x2e0/0x3a0
[    0.627471] [c0000000fd3e3db0] [c0000000000117ac] kernel_init+0x24/0x178
[    0.627510] [c0000000fd3e3e20] [c00000000000c7a8] ret_from_kernel_thread+0x5c/0x74
[    0.627543] Instruction dump:
[    0.627562] XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX 
[    0.627607] XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX <00000000> 00000000 00000000 00000000 
[    0.627660] ---[ end trace 964ab92781f5d99d ]---
[    0.629607] 


- Naveen

Re: [PATCH v8 2/7] powerpc/kprobes: Mark newly allocated probes as RO

[Naveen N. Rao]

Naveen N. Rao wrote:
> Russell Currey wrote:
>> With CONFIG_STRICT_KERNEL_RWX=y and CONFIG_KPROBES=y, there will be one
>> W+X page at boot by default.  This can be tested with
>> CONFIG_PPC_PTDUMP=y and CONFIG_PPC_DEBUG_WX=y set, and checking the
>> kernel log during boot.
>> 
>> powerpc doesn't implement its own alloc() for kprobes like other
>> architectures do, but we couldn't immediately mark RO anyway since we do
>> a memcpy to the page we allocate later.  After that, nothing should be
>> allowed to modify the page, and write permissions are removed well
>> before the kprobe is armed.
>> 
>> The memcpy() would fail if >1 probes were allocated, so use
>> patch_instruction() instead which is safe for RO.
>> 
>> Reviewed-by: Daniel Axtens <dja@axtens.net>
>> Signed-off-by: Russell Currey <ruscur@russell.cc>
>> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
>> ---
>>  arch/powerpc/kernel/kprobes.c | 17 +++++++++++++----
>>  1 file changed, 13 insertions(+), 4 deletions(-)
>> 
>> diff --git a/arch/powerpc/kernel/kprobes.c b/arch/powerpc/kernel/kprobes.c
>> index 81efb605113e..fa4502b4de35 100644
>> --- a/arch/powerpc/kernel/kprobes.c
>> +++ b/arch/powerpc/kernel/kprobes.c
>> @@ -24,6 +24,8 @@
>>  #include <asm/sstep.h>
>>  #include <asm/sections.h>
>>  #include <linux/uaccess.h>
>> +#include <linux/set_memory.h>
>> +#include <linux/vmalloc.h>
>>  
>>  DEFINE_PER_CPU(struct kprobe *, current_kprobe) = NULL;
>>  DEFINE_PER_CPU(struct kprobe_ctlblk, kprobe_ctlblk);
>> @@ -102,6 +104,16 @@ kprobe_opcode_t *kprobe_lookup_name(const char *name, unsigned int offset)
>>  	return addr;
>>  }
>>  
>> +void *alloc_insn_page(void)
>> +{
>> +	void *page = vmalloc_exec(PAGE_SIZE);
>> +
>> +	if (page)
>> +		set_memory_ro((unsigned long)page, 1);
>> +
>> +	return page;
>> +}
>> +
> 
> This crashes for me with KPROBES_SANITY_TEST during the kretprobe test.  

That isn't needed to reproduce this. After bootup, disabling optprobes 
also shows the crash with kretprobes:
	sysctl debug.kprobes-optimization=0

The problem happens to be with patch_instruction() in 
arch_prepare_kprobe(). During boot, on kprobe init, we register a probe 
on kretprobe_trampoline for use with kretprobes (see 
arch_init_kprobes()). This results in an instruction slot being 
allocated, and arch_prepare_kprobe() to be called for copying the 
instruction (nop) at kretprobe_trampoline. patch_instruction() is 
failing resulting in corrupt instruction which we try to emulate/single 
step causing the crash.


- Naveen

Re: [PATCH v8 2/7] powerpc/kprobes: Mark newly allocated probes as RO

[Russell Currey]

On Fri, 2020-04-03 at 00:18 +0530, Naveen N. Rao wrote:
> Naveen N. Rao wrote:
> > Russell Currey wrote:
> > > With CONFIG_STRICT_KERNEL_RWX=y and CONFIG_KPROBES=y, there will
> > > be one
> > > W+X page at boot by default.  This can be tested with
> > > CONFIG_PPC_PTDUMP=y and CONFIG_PPC_DEBUG_WX=y set, and checking
> > > the
> > > kernel log during boot.
> > > 
> > > powerpc doesn't implement its own alloc() for kprobes like other
> > > architectures do, but we couldn't immediately mark RO anyway
> > > since we do
> > > a memcpy to the page we allocate later.  After that, nothing
> > > should be
> > > allowed to modify the page, and write permissions are removed
> > > well
> > > before the kprobe is armed.
> > > 
> > > The memcpy() would fail if >1 probes were allocated, so use
> > > patch_instruction() instead which is safe for RO.
> > > 
> > > Reviewed-by: Daniel Axtens <dja@axtens.net>
> > > Signed-off-by: Russell Currey <ruscur@russell.cc>
> > > Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
> > > ---
> > >  arch/powerpc/kernel/kprobes.c | 17 +++++++++++++----
> > >  1 file changed, 13 insertions(+), 4 deletions(-)
> > > 
> > > diff --git a/arch/powerpc/kernel/kprobes.c
> > > b/arch/powerpc/kernel/kprobes.c
> > > index 81efb605113e..fa4502b4de35 100644
> > > --- a/arch/powerpc/kernel/kprobes.c
> > > +++ b/arch/powerpc/kernel/kprobes.c
> > > @@ -24,6 +24,8 @@
> > >  #include <asm/sstep.h>
> > >  #include <asm/sections.h>
> > >  #include <linux/uaccess.h>
> > > +#include <linux/set_memory.h>
> > > +#include <linux/vmalloc.h>
> > >  
> > >  DEFINE_PER_CPU(struct kprobe *, current_kprobe) = NULL;
> > >  DEFINE_PER_CPU(struct kprobe_ctlblk, kprobe_ctlblk);
> > > @@ -102,6 +104,16 @@ kprobe_opcode_t *kprobe_lookup_name(const
> > > char *name, unsigned int offset)
> > >  	return addr;
> > >  }
> > >  
> > > +void *alloc_insn_page(void)
> > > +{
> > > +	void *page = vmalloc_exec(PAGE_SIZE);
> > > +
> > > +	if (page)
> > > +		set_memory_ro((unsigned long)page, 1);
> > > +
> > > +	return page;
> > > +}
> > > +
> > 
> > This crashes for me with KPROBES_SANITY_TEST during the kretprobe
> > test.  
> 
> That isn't needed to reproduce this. After bootup, disabling
> optprobes 
> also shows the crash with kretprobes:
> 	sysctl debug.kprobes-optimization=0
> 
> The problem happens to be with patch_instruction() in 
> arch_prepare_kprobe(). During boot, on kprobe init, we register a
> probe 
> on kretprobe_trampoline for use with kretprobes (see 
> arch_init_kprobes()). This results in an instruction slot being 
> allocated, and arch_prepare_kprobe() to be called for copying the 
> instruction (nop) at kretprobe_trampoline. patch_instruction() is 
> failing resulting in corrupt instruction which we try to
> emulate/single 
> step causing the crash.

Thanks a lot for the info Naveen, I'll investigate.

- Russell

> 
> 
> - Naveen
> 

Re: [PATCH v8 2/7] powerpc/kprobes: Mark newly allocated probes as RO

[Russell Currey]

On Fri, 2020-04-03 at 00:18 +0530, Naveen N. Rao wrote:
> Naveen N. Rao wrote:
> > Russell Currey wrote:
> > > With CONFIG_STRICT_KERNEL_RWX=y and CONFIG_KPROBES=y, there will
> > > be one
> > > W+X page at boot by default.  This can be tested with
> > > CONFIG_PPC_PTDUMP=y and CONFIG_PPC_DEBUG_WX=y set, and checking
> > > the
> > > kernel log during boot.
> > > 
> > > powerpc doesn't implement its own alloc() for kprobes like other
> > > architectures do, but we couldn't immediately mark RO anyway
> > > since we do
> > > a memcpy to the page we allocate later.  After that, nothing
> > > should be
> > > allowed to modify the page, and write permissions are removed
> > > well
> > > before the kprobe is armed.
> > > 
> > > The memcpy() would fail if >1 probes were allocated, so use
> > > patch_instruction() instead which is safe for RO.
> > > 
> > > Reviewed-by: Daniel Axtens <dja@axtens.net>
> > > Signed-off-by: Russell Currey <ruscur@russell.cc>
> > > Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
> > > ---
> > >  arch/powerpc/kernel/kprobes.c | 17 +++++++++++++----
> > >  1 file changed, 13 insertions(+), 4 deletions(-)
> > > 
> > > diff --git a/arch/powerpc/kernel/kprobes.c
> > > b/arch/powerpc/kernel/kprobes.c
> > > index 81efb605113e..fa4502b4de35 100644
> > > --- a/arch/powerpc/kernel/kprobes.c
> > > +++ b/arch/powerpc/kernel/kprobes.c
> > > @@ -24,6 +24,8 @@
> > >  #include <asm/sstep.h>
> > >  #include <asm/sections.h>
> > >  #include <linux/uaccess.h>
> > > +#include <linux/set_memory.h>
> > > +#include <linux/vmalloc.h>
> > >  
> > >  DEFINE_PER_CPU(struct kprobe *, current_kprobe) = NULL;
> > >  DEFINE_PER_CPU(struct kprobe_ctlblk, kprobe_ctlblk);
> > > @@ -102,6 +104,16 @@ kprobe_opcode_t *kprobe_lookup_name(const
> > > char *name, unsigned int offset)
> > >  	return addr;
> > >  }
> > >  
> > > +void *alloc_insn_page(void)
> > > +{
> > > +	void *page = vmalloc_exec(PAGE_SIZE);
> > > +
> > > +	if (page)
> > > +		set_memory_ro((unsigned long)page, 1);
> > > +
> > > +	return page;
> > > +}
> > > +
> > 
> > This crashes for me with KPROBES_SANITY_TEST during the kretprobe
> > test.  
> 
> That isn't needed to reproduce this. After bootup, disabling
> optprobes 
> also shows the crash with kretprobes:
> 	sysctl debug.kprobes-optimization=0
> 
> The problem happens to be with patch_instruction() in 
> arch_prepare_kprobe(). During boot, on kprobe init, we register a
> probe 
> on kretprobe_trampoline for use with kretprobes (see 
> arch_init_kprobes()). This results in an instruction slot being 
> allocated, and arch_prepare_kprobe() to be called for copying the 
> instruction (nop) at kretprobe_trampoline. patch_instruction() is 
> failing resulting in corrupt instruction which we try to
> emulate/single 
> step causing the crash.

OK I think I've fixed it, KPROBES_SANITY_TEST passes too.  I'd
appreciate it if you could test v9, and thanks again for finding this -
very embarrassing bug on my side.

- Russell

> 
> 
> - Naveen
> 

Re: [PATCH v8 2/7] powerpc/kprobes: Mark newly allocated probes as RO

[Naveen N. Rao]

Russell Currey wrote:
> On Fri, 2020-04-03 at 00:18 +0530, Naveen N. Rao wrote:
>> Naveen N. Rao wrote:
>> > Russell Currey wrote:
>> > > With CONFIG_STRICT_KERNEL_RWX=y and CONFIG_KPROBES=y, there will
>> > > be one
>> > > W+X page at boot by default.  This can be tested with
>> > > CONFIG_PPC_PTDUMP=y and CONFIG_PPC_DEBUG_WX=y set, and checking
>> > > the
>> > > kernel log during boot.
>> > > 
>> > > powerpc doesn't implement its own alloc() for kprobes like other
>> > > architectures do, but we couldn't immediately mark RO anyway
>> > > since we do
>> > > a memcpy to the page we allocate later.  After that, nothing
>> > > should be
>> > > allowed to modify the page, and write permissions are removed
>> > > well
>> > > before the kprobe is armed.
>> > > 
>> > > The memcpy() would fail if >1 probes were allocated, so use
>> > > patch_instruction() instead which is safe for RO.
>> > > 
>> > > Reviewed-by: Daniel Axtens <dja@axtens.net>
>> > > Signed-off-by: Russell Currey <ruscur@russell.cc>
>> > > Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
>> > > ---
>> > >  arch/powerpc/kernel/kprobes.c | 17 +++++++++++++----
>> > >  1 file changed, 13 insertions(+), 4 deletions(-)
>> > > 
>> > > diff --git a/arch/powerpc/kernel/kprobes.c
>> > > b/arch/powerpc/kernel/kprobes.c
>> > > index 81efb605113e..fa4502b4de35 100644
>> > > --- a/arch/powerpc/kernel/kprobes.c
>> > > +++ b/arch/powerpc/kernel/kprobes.c
>> > > @@ -24,6 +24,8 @@
>> > >  #include <asm/sstep.h>
>> > >  #include <asm/sections.h>
>> > >  #include <linux/uaccess.h>
>> > > +#include <linux/set_memory.h>
>> > > +#include <linux/vmalloc.h>
>> > >  
>> > >  DEFINE_PER_CPU(struct kprobe *, current_kprobe) = NULL;
>> > >  DEFINE_PER_CPU(struct kprobe_ctlblk, kprobe_ctlblk);
>> > > @@ -102,6 +104,16 @@ kprobe_opcode_t *kprobe_lookup_name(const
>> > > char *name, unsigned int offset)
>> > >  	return addr;
>> > >  }
>> > >  
>> > > +void *alloc_insn_page(void)
>> > > +{
>> > > +	void *page = vmalloc_exec(PAGE_SIZE);
>> > > +
>> > > +	if (page)
>> > > +		set_memory_ro((unsigned long)page, 1);
>> > > +
>> > > +	return page;
>> > > +}
>> > > +
>> > 
>> > This crashes for me with KPROBES_SANITY_TEST during the kretprobe
>> > test.  
>> 
>> That isn't needed to reproduce this. After bootup, disabling
>> optprobes 
>> also shows the crash with kretprobes:
>> 	sysctl debug.kprobes-optimization=0
>> 
>> The problem happens to be with patch_instruction() in 
>> arch_prepare_kprobe(). During boot, on kprobe init, we register a
>> probe 
>> on kretprobe_trampoline for use with kretprobes (see 
>> arch_init_kprobes()). This results in an instruction slot being 
>> allocated, and arch_prepare_kprobe() to be called for copying the 
>> instruction (nop) at kretprobe_trampoline. patch_instruction() is 
>> failing resulting in corrupt instruction which we try to
>> emulate/single 
>> step causing the crash.
> 
> OK I think I've fixed it, KPROBES_SANITY_TEST passes too.  I'd
> appreciate it if you could test v9, and thanks again for finding this -
> very embarrassing bug on my side.

Great! Thanks.

I think I should also add appropriate error checking to kprobes' use of 
patch_instruction() which would have caught this much more easily.

On a related note, I notice that x86 seems to prefer not having any RWX 
pages, and so they continue to do 'module_alloc()' followed by 
'set_memory_ro()' and then 'set_memory_x()'. Is that something worth 
following for powerpc?


- Naveen

Re: [PATCH v8 2/7] powerpc/kprobes: Mark newly allocated probes as RO

[Russell Currey]

On Fri, 2020-04-03 at 15:06 +0530, Naveen N. Rao wrote:
> Russell Currey wrote:
> > On Fri, 2020-04-03 at 00:18 +0530, Naveen N. Rao wrote:
> > > Naveen N. Rao wrote:
> > > > Russell Currey wrote:
> > > > > With CONFIG_STRICT_KERNEL_RWX=y and CONFIG_KPROBES=y, there
> > > > > will
> > > > > be one
> > > > > W+X page at boot by default.  This can be tested with
> > > > > CONFIG_PPC_PTDUMP=y and CONFIG_PPC_DEBUG_WX=y set, and
> > > > > checking
> > > > > the
> > > > > kernel log during boot.
> > > > > 
> > > > > powerpc doesn't implement its own alloc() for kprobes like
> > > > > other
> > > > > architectures do, but we couldn't immediately mark RO anyway
> > > > > since we do
> > > > > a memcpy to the page we allocate later.  After that, nothing
> > > > > should be
> > > > > allowed to modify the page, and write permissions are removed
> > > > > well
> > > > > before the kprobe is armed.
> > > > > 
> > > > > The memcpy() would fail if >1 probes were allocated, so use
> > > > > patch_instruction() instead which is safe for RO.
> > > > > 
> > > > > Reviewed-by: Daniel Axtens <dja@axtens.net>
> > > > > Signed-off-by: Russell Currey <ruscur@russell.cc>
> > > > > Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
> > > > > ---
> > > > >  arch/powerpc/kernel/kprobes.c | 17 +++++++++++++----
> > > > >  1 file changed, 13 insertions(+), 4 deletions(-)
> > > > > 
> > > > > diff --git a/arch/powerpc/kernel/kprobes.c
> > > > > b/arch/powerpc/kernel/kprobes.c
> > > > > index 81efb605113e..fa4502b4de35 100644
> > > > > --- a/arch/powerpc/kernel/kprobes.c
> > > > > +++ b/arch/powerpc/kernel/kprobes.c
> > > > > @@ -24,6 +24,8 @@
> > > > >  #include <asm/sstep.h>
> > > > >  #include <asm/sections.h>
> > > > >  #include <linux/uaccess.h>
> > > > > +#include <linux/set_memory.h>
> > > > > +#include <linux/vmalloc.h>
> > > > >  
> > > > >  DEFINE_PER_CPU(struct kprobe *, current_kprobe) = NULL;
> > > > >  DEFINE_PER_CPU(struct kprobe_ctlblk, kprobe_ctlblk);
> > > > > @@ -102,6 +104,16 @@ kprobe_opcode_t
> > > > > *kprobe_lookup_name(const
> > > > > char *name, unsigned int offset)
> > > > >  	return addr;
> > > > >  }
> > > > >  
> > > > > +void *alloc_insn_page(void)
> > > > > +{
> > > > > +	void *page = vmalloc_exec(PAGE_SIZE);
> > > > > +
> > > > > +	if (page)
> > > > > +		set_memory_ro((unsigned long)page, 1);
> > > > > +
> > > > > +	return page;
> > > > > +}
> > > > > +
> > > > 
> > > > This crashes for me with KPROBES_SANITY_TEST during the
> > > > kretprobe
> > > > test.  
> > > 
> > > That isn't needed to reproduce this. After bootup, disabling
> > > optprobes 
> > > also shows the crash with kretprobes:
> > > 	sysctl debug.kprobes-optimization=0
> > > 
> > > The problem happens to be with patch_instruction() in 
> > > arch_prepare_kprobe(). During boot, on kprobe init, we register a
> > > probe 
> > > on kretprobe_trampoline for use with kretprobes (see 
> > > arch_init_kprobes()). This results in an instruction slot being 
> > > allocated, and arch_prepare_kprobe() to be called for copying
> > > the 
> > > instruction (nop) at kretprobe_trampoline. patch_instruction()
> > > is 
> > > failing resulting in corrupt instruction which we try to
> > > emulate/single 
> > > step causing the crash.
> > 
> > OK I think I've fixed it, KPROBES_SANITY_TEST passes too.  I'd
> > appreciate it if you could test v9, and thanks again for finding
> > this -
> > very embarrassing bug on my side.
> 
> Great! Thanks.
> 
> I think I should also add appropriate error checking to kprobes' use
> of 
> patch_instruction() which would have caught this much more easily.

Only kind of!  It turns out that if the initial setup fails for
KPROBES_SANITY_TEST, it silently doesn't run - so you miss the "Kprobe
smoke test" text, but you don't get any kind of error either.  I'll
send a patch so that it fails more loudly later.

> 
> On a related note, I notice that x86 seems to prefer not having any
> RWX 
> pages, and so they continue to do 'module_alloc()' followed by 
> 'set_memory_ro()' and then 'set_memory_x()'. Is that something worth 
> following for powerpc?

I just noticed that too.  arm64 doesn't set theirs executable, as far
as I can tell powerpc doesn't need to.

> 
> - Naveen
> 

Re: [PATCH v8 2/7] powerpc/kprobes: Mark newly allocated probes as RO

[Naveen N. Rao]

Russell Currey wrote:
> On Fri, 2020-04-03 at 15:06 +0530, Naveen N. Rao wrote:
>> Russell Currey wrote:
>> > On Fri, 2020-04-03 at 00:18 +0530, Naveen N. Rao wrote:
>> > > Naveen N. Rao wrote:
>> > > > Russell Currey wrote:
>> > > > >  
>> > > > > +void *alloc_insn_page(void)
>> > > > > +{
>> > > > > +	void *page = vmalloc_exec(PAGE_SIZE);
>> > > > > +
>> > > > > +	if (page)
>> > > > > +		set_memory_ro((unsigned long)page, 1);
>> > > > > +
>> > > > > +	return page;
>> > > > > +}
>> > > > > +
>> > > > 
>> > > > This crashes for me with KPROBES_SANITY_TEST during the
>> > > > kretprobe
>> > > > test.  
>> > > 
>> > > That isn't needed to reproduce this. After bootup, disabling
>> > > optprobes 
>> > > also shows the crash with kretprobes:
>> > > 	sysctl debug.kprobes-optimization=0
>> > > 
>> > > The problem happens to be with patch_instruction() in 
>> > > arch_prepare_kprobe(). During boot, on kprobe init, we register a
>> > > probe 
>> > > on kretprobe_trampoline for use with kretprobes (see 
>> > > arch_init_kprobes()). This results in an instruction slot being 
>> > > allocated, and arch_prepare_kprobe() to be called for copying
>> > > the 
>> > > instruction (nop) at kretprobe_trampoline. patch_instruction()
>> > > is 
>> > > failing resulting in corrupt instruction which we try to
>> > > emulate/single 
>> > > step causing the crash.
>> > 
>> > OK I think I've fixed it, KPROBES_SANITY_TEST passes too.  I'd
>> > appreciate it if you could test v9, and thanks again for finding
>> > this -
>> > very embarrassing bug on my side.
>> 
>> Great! Thanks.
>> 
>> I think I should also add appropriate error checking to kprobes' use
>> of 
>> patch_instruction() which would have caught this much more easily.
> 
> Only kind of!  It turns out that if the initial setup fails for
> KPROBES_SANITY_TEST, it silently doesn't run - so you miss the "Kprobe
> smoke test" text, but you don't get any kind of error either.  I'll
> send a patch so that it fails more loudly later.

Ha, I see what you mean. Good catch, we should pass the kprobe init 
status to the test and have it error out.

> 
>> 
>> On a related note, I notice that x86 seems to prefer not having any
>> RWX 
>> pages, and so they continue to do 'module_alloc()' followed by 
>> 'set_memory_ro()' and then 'set_memory_x()'. Is that something worth 
>> following for powerpc?
> 
> I just noticed that too.  arm64 doesn't set theirs executable, as far
> as I can tell powerpc doesn't need to.

I didn't follow that. We do need it to be executable so that we can 
single step the original instruction.

arm64 does vmalloc_exec(), which looks like it sets the page to RWX, 
then marks it RO. There is a small window where the page would be WX.  
x86 instead seems to first allocate the page as RW, mark as RO, and only 
then enable X - removing that small window where the page is both W and 
X.

- Naveen