* [PATCH 1/3] powerpc/modules: Load modules closer to kernel text
@ 2021-04-01 13:30 ` Christophe Leroy
0 siblings, 0 replies; 7+ messages in thread
From: Christophe Leroy @ 2021-04-01 13:30 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, jniethe5
Cc: linux-kernel, linuxppc-dev
On book3s/32, when STRICT_KERNEL_RWX is selected, modules are
allocated on the segment just before kernel text, ie on the
0xb0000000-0xbfffffff when PAGE_OFFSET is 0xc0000000.
On the 8xx, TASK_SIZE is 0x80000000. The space between TASK_SIZE and
PAGE_OFFSET is not used and could be used for modules.
The idea comes from ARM architecture.
Having modules just below PAGE_OFFSET offers an opportunity to
minimise the distance between kernel text and modules and avoid
trampolines in modules to access kernel functions or other module
functions.
When MODULES_VADDR is defined, powerpc has it's own module_alloc()
function. In that function, first try to allocate the module
above the limit defined by '_etext - 32M'. Then if the allocation
fails, fallback to the entire MODULES area.
DEBUG logs in module_32.c without the patch:
[ 1572.588822] module_32: Applying ADD relocate section 13 to 12
[ 1572.588891] module_32: Doing plt for call to 0xc00671a4 at 0xcae04024
[ 1572.588964] module_32: Initialized plt for 0xc00671a4 at cae04000
[ 1572.589037] module_32: REL24 value = CAE04000. location = CAE04024
[ 1572.589110] module_32: Location before: 48000001.
[ 1572.589171] module_32: Location after: 4BFFFFDD.
[ 1572.589231] module_32: ie. jump to 03FFFFDC+CAE04024 = CEE04000
[ 1572.589317] module_32: Applying ADD relocate section 15 to 14
[ 1572.589386] module_32: Doing plt for call to 0xc00671a4 at 0xcadfc018
[ 1572.589457] module_32: Initialized plt for 0xc00671a4 at cadfc000
[ 1572.589529] module_32: REL24 value = CADFC000. location = CADFC018
[ 1572.589601] module_32: Location before: 48000000.
[ 1572.589661] module_32: Location after: 4BFFFFE8.
[ 1572.589723] module_32: ie. jump to 03FFFFE8+CADFC018 = CEDFC000
With the patch:
[ 279.404671] module_32: Applying ADD relocate section 13 to 12
[ 279.404741] module_32: REL24 value = C00671B4. location = BF808024
[ 279.404814] module_32: Location before: 48000001.
[ 279.404874] module_32: Location after: 4885F191.
[ 279.404933] module_32: ie. jump to 0085F190+BF808024 = C00671B4
[ 279.405016] module_32: Applying ADD relocate section 15 to 14
[ 279.405085] module_32: REL24 value = C00671B4. location = BF800018
[ 279.405156] module_32: Location before: 48000000.
[ 279.405215] module_32: Location after: 4886719C.
[ 279.405275] module_32: ie. jump to 0086719C+BF800018 = C00671B4
We see that with the patch, no plt entries are set.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
arch/powerpc/kernel/module.c | 23 ++++++++++++++++++++---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/kernel/module.c b/arch/powerpc/kernel/module.c
index a211b0253cdb..fab84024650c 100644
--- a/arch/powerpc/kernel/module.c
+++ b/arch/powerpc/kernel/module.c
@@ -14,6 +14,7 @@
#include <asm/firmware.h>
#include <linux/sort.h>
#include <asm/setup.h>
+#include <asm/sections.h>
static LIST_HEAD(module_bug_list);
@@ -88,12 +89,28 @@ int module_finalize(const Elf_Ehdr *hdr,
}
#ifdef MODULES_VADDR
+static __always_inline void *
+__module_alloc(unsigned long size, unsigned long start, unsigned long end)
+{
+ return __vmalloc_node_range(size, 1, start, end, GFP_KERNEL,
+ PAGE_KERNEL_EXEC, VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
+ __builtin_return_address(0));
+}
+
void *module_alloc(unsigned long size)
{
+ unsigned long limit = (unsigned long)_etext - SZ_32M;
+ void *ptr = NULL;
+
BUILD_BUG_ON(TASK_SIZE > MODULES_VADDR);
- return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END, GFP_KERNEL,
- PAGE_KERNEL_EXEC, VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
- __builtin_return_address(0));
+ /* First try within 32M limit from _etext to avoid branch trampolines */
+ if (MODULES_VADDR < PAGE_OFFSET && MODULES_END > limit)
+ ptr = __module_alloc(size, limit, MODULES_END);
+
+ if (!ptr)
+ ptr = __module_alloc(size, MODULES_VADDR, MODULES_END);
+
+ return ptr;
}
#endif
--
2.25.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 1/3] powerpc/modules: Load modules closer to kernel text
@ 2021-04-01 13:30 ` Christophe Leroy
0 siblings, 0 replies; 7+ messages in thread
From: Christophe Leroy @ 2021-04-01 13:30 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, jniethe5
Cc: linuxppc-dev, linux-kernel
On book3s/32, when STRICT_KERNEL_RWX is selected, modules are
allocated on the segment just before kernel text, ie on the
0xb0000000-0xbfffffff when PAGE_OFFSET is 0xc0000000.
On the 8xx, TASK_SIZE is 0x80000000. The space between TASK_SIZE and
PAGE_OFFSET is not used and could be used for modules.
The idea comes from ARM architecture.
Having modules just below PAGE_OFFSET offers an opportunity to
minimise the distance between kernel text and modules and avoid
trampolines in modules to access kernel functions or other module
functions.
When MODULES_VADDR is defined, powerpc has it's own module_alloc()
function. In that function, first try to allocate the module
above the limit defined by '_etext - 32M'. Then if the allocation
fails, fallback to the entire MODULES area.
DEBUG logs in module_32.c without the patch:
[ 1572.588822] module_32: Applying ADD relocate section 13 to 12
[ 1572.588891] module_32: Doing plt for call to 0xc00671a4 at 0xcae04024
[ 1572.588964] module_32: Initialized plt for 0xc00671a4 at cae04000
[ 1572.589037] module_32: REL24 value = CAE04000. location = CAE04024
[ 1572.589110] module_32: Location before: 48000001.
[ 1572.589171] module_32: Location after: 4BFFFFDD.
[ 1572.589231] module_32: ie. jump to 03FFFFDC+CAE04024 = CEE04000
[ 1572.589317] module_32: Applying ADD relocate section 15 to 14
[ 1572.589386] module_32: Doing plt for call to 0xc00671a4 at 0xcadfc018
[ 1572.589457] module_32: Initialized plt for 0xc00671a4 at cadfc000
[ 1572.589529] module_32: REL24 value = CADFC000. location = CADFC018
[ 1572.589601] module_32: Location before: 48000000.
[ 1572.589661] module_32: Location after: 4BFFFFE8.
[ 1572.589723] module_32: ie. jump to 03FFFFE8+CADFC018 = CEDFC000
With the patch:
[ 279.404671] module_32: Applying ADD relocate section 13 to 12
[ 279.404741] module_32: REL24 value = C00671B4. location = BF808024
[ 279.404814] module_32: Location before: 48000001.
[ 279.404874] module_32: Location after: 4885F191.
[ 279.404933] module_32: ie. jump to 0085F190+BF808024 = C00671B4
[ 279.405016] module_32: Applying ADD relocate section 15 to 14
[ 279.405085] module_32: REL24 value = C00671B4. location = BF800018
[ 279.405156] module_32: Location before: 48000000.
[ 279.405215] module_32: Location after: 4886719C.
[ 279.405275] module_32: ie. jump to 0086719C+BF800018 = C00671B4
We see that with the patch, no plt entries are set.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
arch/powerpc/kernel/module.c | 23 ++++++++++++++++++++---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/kernel/module.c b/arch/powerpc/kernel/module.c
index a211b0253cdb..fab84024650c 100644
--- a/arch/powerpc/kernel/module.c
+++ b/arch/powerpc/kernel/module.c
@@ -14,6 +14,7 @@
#include <asm/firmware.h>
#include <linux/sort.h>
#include <asm/setup.h>
+#include <asm/sections.h>
static LIST_HEAD(module_bug_list);
@@ -88,12 +89,28 @@ int module_finalize(const Elf_Ehdr *hdr,
}
#ifdef MODULES_VADDR
+static __always_inline void *
+__module_alloc(unsigned long size, unsigned long start, unsigned long end)
+{
+ return __vmalloc_node_range(size, 1, start, end, GFP_KERNEL,
+ PAGE_KERNEL_EXEC, VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
+ __builtin_return_address(0));
+}
+
void *module_alloc(unsigned long size)
{
+ unsigned long limit = (unsigned long)_etext - SZ_32M;
+ void *ptr = NULL;
+
BUILD_BUG_ON(TASK_SIZE > MODULES_VADDR);
- return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END, GFP_KERNEL,
- PAGE_KERNEL_EXEC, VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
- __builtin_return_address(0));
+ /* First try within 32M limit from _etext to avoid branch trampolines */
+ if (MODULES_VADDR < PAGE_OFFSET && MODULES_END > limit)
+ ptr = __module_alloc(size, limit, MODULES_END);
+
+ if (!ptr)
+ ptr = __module_alloc(size, MODULES_VADDR, MODULES_END);
+
+ return ptr;
}
#endif
--
2.25.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/3] powerpc/8xx: Define a MODULE area below kernel text
2021-04-01 13:30 ` Christophe Leroy
@ 2021-04-01 13:30 ` Christophe Leroy
-1 siblings, 0 replies; 7+ messages in thread
From: Christophe Leroy @ 2021-04-01 13:30 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, jniethe5
Cc: linux-kernel, linuxppc-dev
On the 8xx, TASK_SIZE is 0x80000000. The space between TASK_SIZE
and PAGE_OFFSET is not used.
In order to benefit from the powerpc specific module_alloc()
function which allocate modules with 32 Mbytes from
end of kernel text, define MODULES_VADDR and MODULES_END.
Set a 256Mb area just below PAGE_OFFSET, like book3s/32.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
arch/powerpc/include/asm/nohash/32/mmu-8xx.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/powerpc/include/asm/nohash/32/mmu-8xx.h b/arch/powerpc/include/asm/nohash/32/mmu-8xx.h
index 478249959baa..6e4faa0a9b35 100644
--- a/arch/powerpc/include/asm/nohash/32/mmu-8xx.h
+++ b/arch/powerpc/include/asm/nohash/32/mmu-8xx.h
@@ -172,6 +172,9 @@
#define mmu_linear_psize MMU_PAGE_8M
+#define MODULES_VADDR (PAGE_OFFSET - SZ_256M)
+#define MODULES_END PAGE_OFFSET
+
#ifndef __ASSEMBLY__
#include <linux/mmdebug.h>
--
2.25.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/3] powerpc/8xx: Define a MODULE area below kernel text
@ 2021-04-01 13:30 ` Christophe Leroy
0 siblings, 0 replies; 7+ messages in thread
From: Christophe Leroy @ 2021-04-01 13:30 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, jniethe5
Cc: linuxppc-dev, linux-kernel
On the 8xx, TASK_SIZE is 0x80000000. The space between TASK_SIZE
and PAGE_OFFSET is not used.
In order to benefit from the powerpc specific module_alloc()
function which allocate modules with 32 Mbytes from
end of kernel text, define MODULES_VADDR and MODULES_END.
Set a 256Mb area just below PAGE_OFFSET, like book3s/32.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
arch/powerpc/include/asm/nohash/32/mmu-8xx.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/powerpc/include/asm/nohash/32/mmu-8xx.h b/arch/powerpc/include/asm/nohash/32/mmu-8xx.h
index 478249959baa..6e4faa0a9b35 100644
--- a/arch/powerpc/include/asm/nohash/32/mmu-8xx.h
+++ b/arch/powerpc/include/asm/nohash/32/mmu-8xx.h
@@ -172,6 +172,9 @@
#define mmu_linear_psize MMU_PAGE_8M
+#define MODULES_VADDR (PAGE_OFFSET - SZ_256M)
+#define MODULES_END PAGE_OFFSET
+
#ifndef __ASSEMBLY__
#include <linux/mmdebug.h>
--
2.25.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/3] powerpc/32s: Define a MODULE area below kernel text all the time
2021-04-01 13:30 ` Christophe Leroy
@ 2021-04-01 13:30 ` Christophe Leroy
-1 siblings, 0 replies; 7+ messages in thread
From: Christophe Leroy @ 2021-04-01 13:30 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, jniethe5
Cc: linux-kernel, linuxppc-dev
On book3s/32, the segment below kernel text is used for module
allocation when CONFIG_STRICT_KERNEL_RWX is defined.
In order to benefit from the powerpc specific module_alloc()
function which allocate modules with 32 Mbytes from
end of kernel text, use that segment below PAGE_OFFSET at all time.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
arch/powerpc/Kconfig | 2 +-
arch/powerpc/include/asm/book3s/32/pgtable.h | 2 --
arch/powerpc/mm/book3s32/mmu.c | 7 -------
3 files changed, 1 insertion(+), 10 deletions(-)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index c1344c05226c..15a91202d5c3 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -1219,7 +1219,7 @@ config TASK_SIZE_BOOL
config TASK_SIZE
hex "Size of user task space" if TASK_SIZE_BOOL
default "0x80000000" if PPC_8xx
- default "0xb0000000" if PPC_BOOK3S_32 && STRICT_KERNEL_RWX
+ default "0xb0000000" if PPC_BOOK3S_32
default "0xc0000000"
endmenu
diff --git a/arch/powerpc/include/asm/book3s/32/pgtable.h b/arch/powerpc/include/asm/book3s/32/pgtable.h
index 415ae29fa73a..83c65845a1a9 100644
--- a/arch/powerpc/include/asm/book3s/32/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/32/pgtable.h
@@ -194,10 +194,8 @@ int map_kernel_page(unsigned long va, phys_addr_t pa, pgprot_t prot);
#define VMALLOC_END ioremap_bot
#endif
-#ifdef CONFIG_STRICT_KERNEL_RWX
#define MODULES_END ALIGN_DOWN(PAGE_OFFSET, SZ_256M)
#define MODULES_VADDR (MODULES_END - SZ_256M)
-#endif
#ifndef __ASSEMBLY__
#include <linux/sched.h>
diff --git a/arch/powerpc/mm/book3s32/mmu.c b/arch/powerpc/mm/book3s32/mmu.c
index a0db398b5c26..159930351d9f 100644
--- a/arch/powerpc/mm/book3s32/mmu.c
+++ b/arch/powerpc/mm/book3s32/mmu.c
@@ -184,17 +184,10 @@ static bool is_module_segment(unsigned long addr)
{
if (!IS_ENABLED(CONFIG_MODULES))
return false;
-#ifdef MODULES_VADDR
if (addr < ALIGN_DOWN(MODULES_VADDR, SZ_256M))
return false;
if (addr > ALIGN(MODULES_END, SZ_256M) - 1)
return false;
-#else
- if (addr < ALIGN_DOWN(VMALLOC_START, SZ_256M))
- return false;
- if (addr > ALIGN(VMALLOC_END, SZ_256M) - 1)
- return false;
-#endif
return true;
}
--
2.25.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/3] powerpc/32s: Define a MODULE area below kernel text all the time
@ 2021-04-01 13:30 ` Christophe Leroy
0 siblings, 0 replies; 7+ messages in thread
From: Christophe Leroy @ 2021-04-01 13:30 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, jniethe5
Cc: linuxppc-dev, linux-kernel
On book3s/32, the segment below kernel text is used for module
allocation when CONFIG_STRICT_KERNEL_RWX is defined.
In order to benefit from the powerpc specific module_alloc()
function which allocate modules with 32 Mbytes from
end of kernel text, use that segment below PAGE_OFFSET at all time.
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
arch/powerpc/Kconfig | 2 +-
arch/powerpc/include/asm/book3s/32/pgtable.h | 2 --
arch/powerpc/mm/book3s32/mmu.c | 7 -------
3 files changed, 1 insertion(+), 10 deletions(-)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index c1344c05226c..15a91202d5c3 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -1219,7 +1219,7 @@ config TASK_SIZE_BOOL
config TASK_SIZE
hex "Size of user task space" if TASK_SIZE_BOOL
default "0x80000000" if PPC_8xx
- default "0xb0000000" if PPC_BOOK3S_32 && STRICT_KERNEL_RWX
+ default "0xb0000000" if PPC_BOOK3S_32
default "0xc0000000"
endmenu
diff --git a/arch/powerpc/include/asm/book3s/32/pgtable.h b/arch/powerpc/include/asm/book3s/32/pgtable.h
index 415ae29fa73a..83c65845a1a9 100644
--- a/arch/powerpc/include/asm/book3s/32/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/32/pgtable.h
@@ -194,10 +194,8 @@ int map_kernel_page(unsigned long va, phys_addr_t pa, pgprot_t prot);
#define VMALLOC_END ioremap_bot
#endif
-#ifdef CONFIG_STRICT_KERNEL_RWX
#define MODULES_END ALIGN_DOWN(PAGE_OFFSET, SZ_256M)
#define MODULES_VADDR (MODULES_END - SZ_256M)
-#endif
#ifndef __ASSEMBLY__
#include <linux/sched.h>
diff --git a/arch/powerpc/mm/book3s32/mmu.c b/arch/powerpc/mm/book3s32/mmu.c
index a0db398b5c26..159930351d9f 100644
--- a/arch/powerpc/mm/book3s32/mmu.c
+++ b/arch/powerpc/mm/book3s32/mmu.c
@@ -184,17 +184,10 @@ static bool is_module_segment(unsigned long addr)
{
if (!IS_ENABLED(CONFIG_MODULES))
return false;
-#ifdef MODULES_VADDR
if (addr < ALIGN_DOWN(MODULES_VADDR, SZ_256M))
return false;
if (addr > ALIGN(MODULES_END, SZ_256M) - 1)
return false;
-#else
- if (addr < ALIGN_DOWN(VMALLOC_START, SZ_256M))
- return false;
- if (addr > ALIGN(VMALLOC_END, SZ_256M) - 1)
- return false;
-#endif
return true;
}
--
2.25.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/3] powerpc/modules: Load modules closer to kernel text
2021-04-01 13:30 ` Christophe Leroy
` (2 preceding siblings ...)
(?)
@ 2021-04-19 3:59 ` Michael Ellerman
-1 siblings, 0 replies; 7+ messages in thread
From: Michael Ellerman @ 2021-04-19 3:59 UTC (permalink / raw)
To: Paul Mackerras, Benjamin Herrenschmidt, jniethe5,
Christophe Leroy, Michael Ellerman
Cc: linuxppc-dev, linux-kernel
On Thu, 1 Apr 2021 13:30:41 +0000 (UTC), Christophe Leroy wrote:
> On book3s/32, when STRICT_KERNEL_RWX is selected, modules are
> allocated on the segment just before kernel text, ie on the
> 0xb0000000-0xbfffffff when PAGE_OFFSET is 0xc0000000.
>
> On the 8xx, TASK_SIZE is 0x80000000. The space between TASK_SIZE and
> PAGE_OFFSET is not used and could be used for modules.
>
> [...]
Applied to powerpc/next.
[1/3] powerpc/modules: Load modules closer to kernel text
https://git.kernel.org/powerpc/c/2ec13df167040cd153c25c4d96d0ffc573ac4c40
[2/3] powerpc/8xx: Define a MODULE area below kernel text
https://git.kernel.org/powerpc/c/9132a2e82adc6e5a1c7c7385df3bfb25576bdd80
[3/3] powerpc/32s: Define a MODULE area below kernel text all the time
https://git.kernel.org/powerpc/c/80edc68e0479bafdc4869ec3351e42316b824596
cheers
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-04-19 4:04 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-01 13:30 [PATCH 1/3] powerpc/modules: Load modules closer to kernel text Christophe Leroy
2021-04-01 13:30 ` Christophe Leroy
2021-04-01 13:30 ` [PATCH 2/3] powerpc/8xx: Define a MODULE area below " Christophe Leroy
2021-04-01 13:30 ` Christophe Leroy
2021-04-01 13:30 ` [PATCH 3/3] powerpc/32s: Define a MODULE area below kernel text all the time Christophe Leroy
2021-04-01 13:30 ` Christophe Leroy
2021-04-19 3:59 ` [PATCH 1/3] powerpc/modules: Load modules closer to kernel text Michael Ellerman
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.