[PATCH 0/3] ARM/ARM64: silence large module first time allocation

[PATCH 0/3] ARM/ARM64: silence large module first time allocation

[Florian Fainelli]

With kernels built with CONFIG_ARM{,64}_MODULES_PLTS=y, the first allocation
done from module space will fail, produce a general OOM allocation and also a
vmap warning. The second allocation from vmalloc space may or may not be
successful, but is actually the one we are interested about in these cases.

This patch series passed __GFP_NOWARN to silence such allocations from the
ARM/ARM64 module loader's first time allocation when the MODULES_PLT option is
enabled, and also makes alloc_vmap_area() react to the caller setting
__GFP_NOWARN to silence "vmap allocation for size..." messages.

Here is an example of what we would get without these two patches, pretty
scary huh?

# insmod /mnt/nfs/huge.ko 
[   22.114143] random: nonblocking pool is initialized
[   22.183575] vmap allocation for size 15736832 failed: use vmalloc=<size> to increase size.
[   22.191873] vmalloc: allocation failure: 15729534 bytes
[   22.197112] insmod: page allocation failure: order:0, mode:0xd0
[   22.203048] CPU: 2 PID: 1506 Comm: insmod Tainted: G           O    4.1.20-1.9pre-01082-gbbbff07bc3ce #9
[   22.212536] Hardware name: Broadcom STB (Flattened Device Tree)
[   22.218480] [<c0017eec>] (unwind_backtrace) from [<c00135c8>] (show_stack+0x10/0x14)
[   22.226238] [<c00135c8>] (show_stack) from [<c0638684>] (dump_stack+0x90/0xa4)
[   22.233473] [<c0638684>] (dump_stack) from [<c00aae1c>] (warn_alloc_failed+0x104/0x144)
[   22.241490] [<c00aae1c>] (warn_alloc_failed) from [<c00d72e0>] (__vmalloc_node_range+0x170/0x218)
[   22.250375] [<c00d72e0>] (__vmalloc_node_range) from [<c00147d0>] (module_alloc+0x50/0xac)
[   22.258651] [<c00147d0>] (module_alloc) from [<c008ae2c>] (module_alloc_update_bounds+0xc/0x6c)
[   22.267360] [<c008ae2c>] (module_alloc_update_bounds) from [<c008b778>] (load_module+0x8ec/0x2058)
[   22.276329] [<c008b778>] (load_module) from [<c008cfd4>] (SyS_init_module+0xf0/0x174)
[   22.284170] [<c008cfd4>] (SyS_init_module) from [<c0010140>] (ret_fast_syscall+0x0/0x3c)
[   22.292277] Mem-Info:
[   22.294567] active_anon:5236 inactive_anon:1773 isolated_anon:0
[   22.294567]  active_file:1 inactive_file:3822 isolated_file:0
[   22.294567]  unevictable:0 dirty:0 writeback:0 unstable:0
[   22.294567]  slab_reclaimable:238 slab_unreclaimable:1594
[   22.294567]  mapped:855 shmem:2950 pagetables:36 bounce:0
[   22.294567]  free:39031 free_pcp:198 free_cma:3928
[   22.327196] DMA free:156124kB min:1880kB low:2348kB high:2820kB active_anon:20944kB inactive_anon:7092kB active_file:4kB inactive_file:15288kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:262144kB managed:227676kB mlocked:0kB dirty:0kB writeback:0kB mapped:3420kB shmem:11800kB slab_reclaimable:952kB slab_unreclaimable:6376kB kernel_stack:560kB pagetables:144kB unstable:0kB bounce:0kB free_pcp:792kB local_pcp:68kB free_cma:15712kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[   22.371631] lowmem_reserve[]: 0 0 0 0
[   22.375372] HighMem free:0kB min:128kB low:128kB high:128kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:2883584kB managed:0kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[   22.416249] lowmem_reserve[]: 0 0 0 0
[   22.419986] DMA: 3*4kB (UEM) 4*8kB (UE) 1*16kB (M) 4*32kB (UEMC) 3*64kB (EMC) 1*128kB (E) 4*256kB (UEMC) 2*512kB (UE) 2*1024kB (MC) 4*2048kB (UEMC) 35*4096kB (MRC) = 156156kB
[   22.435922] HighMem: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB
[   22.446130] 6789 total pagecache pages
[   22.449889] 0 pages in swap cache
[   22.453212] Swap cache stats: add 0, delete 0, find 0/0
[   22.458447] Free swap  = 0kB
[   22.461334] Total swap = 0kB
[   22.464222] 786432 pages RAM
[   22.467110] 720896 pages HighMem/MovableOnly
[   22.471388] 725417 pages reserved
[   22.474711] 4096 pages cma reserved
[   22.511310] big_init: I am a big module using 3932160 bytes of data!

Florian Fainelli (3):
  mm: Silence vmap() allocation failures based on caller gfp_flags
  ARM: Silence first allocation with CONFIG_ARM_MODULE_PLTS=y
  arm64: Silence first allocation with CONFIG_ARM64_MODULE_PLTS=y

 arch/arm/kernel/module.c   | 11 +++++++++--
 arch/arm64/kernel/module.c |  7 ++++++-
 mm/vmalloc.c               |  2 +-
 3 files changed, 16 insertions(+), 4 deletions(-)

-- 
2.9.3

[PATCH 1/2] ARM: Silence first allocation with CONFIG_ARM_MODULE_PLTS=y

[Florian Fainelli]

When CONFIG_ARM_MODULE_PLTS is enabled, the first allocation using the
module space fails, because the module is too big, and then the module
allocation is attempted from vmalloc space. Silence the first allocation
failure in that case by setting __GFP_NOWARN.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
 arch/arm/kernel/module.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c
index 80254b47dc34..503d1a39464a 100644
--- a/arch/arm/kernel/module.c
+++ b/arch/arm/kernel/module.c
@@ -40,8 +40,15 @@
 #ifdef CONFIG_MMU
 void *module_alloc(unsigned long size)
 {
-	void *p = __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END,
-				GFP_KERNEL, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE,
+	gfp_t gfp_mask = GFP_KERNEL;
+	void *p;
+
+#if IS_ENABLED(CONFIG_ARM_MODULE_PLTS)
+	/* Silence the initial allocation */
+	gfp_mask |= __GFP_NOWARN;
+#endif
+	p = __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END,
+				gfp_mask, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE,
 				__builtin_return_address(0));
 	if (!IS_ENABLED(CONFIG_ARM_MODULE_PLTS) || p)
 		return p;
-- 
2.9.3

[PATCH 1/2] ARM: Silence non PLT allocation with CONFIG_ARM_MODULE_PLTS=y

[Florian Fainelli]

When CONFIG_ARM_MODULE_PLTS is enabled, the first allocation using the
module space fails, because the module is too big, and then the module
allocation is attempted from vmalloc space. Silence the first allocation
failure in that case, since that scares people out.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
 arch/arm/kernel/module.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c
index 80254b47dc34..503d1a39464a 100644
--- a/arch/arm/kernel/module.c
+++ b/arch/arm/kernel/module.c
@@ -40,8 +40,15 @@
 #ifdef CONFIG_MMU
 void *module_alloc(unsigned long size)
 {
-	void *p = __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END,
-				GFP_KERNEL, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE,
+	gfp_t gfp_mask = GFP_KERNEL;
+	void *p;
+
+#if IS_ENABLED(CONFIG_ARM_MODULE_PLTS)
+	/* Silence the initial allocation */
+	gfp_mask |= __GFP_NOWARN;
+#endif
+	p = __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END,
+				gfp_mask, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE,
 				__builtin_return_address(0));
 	if (!IS_ENABLED(CONFIG_ARM_MODULE_PLTS) || p)
 		return p;
-- 
2.9.3

[PATCH 1/3] mm: Silence vmap() allocation failures based on caller gfp_flags

[Florian Fainelli]

If the caller has set __GFP_NOWARN don't print the following message:
vmap allocation for size 15736832 failed: use vmalloc=<size> to increase
size.

This can happen with the ARM/Linux module loader built with
CONFIG_ARM_MODULE_PLTS=y which does a first attempt at loading a large
module from module space, then falls back to vmalloc space.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
 mm/vmalloc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 0b057628a7ba..5a788eb58741 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -521,7 +521,7 @@ static struct vmap_area *alloc_vmap_area(unsigned long size,
 		}
 	}
 
-	if (printk_ratelimit())
+	if (printk_ratelimit() && !(gfp_mask & __GFP_NOWARN))
 		pr_warn("vmap allocation for size %lu failed: use vmalloc=<size> to increase size\n",
 			size);
 	kfree(va);
-- 
2.9.3

[PATCH 2/3] ARM: Silence first allocation with CONFIG_ARM_MODULE_PLTS=y

[Florian Fainelli]

When CONFIG_ARM_MODULE_PLTS is enabled, the first allocation using the
module space fails, because the module is too big, and then the module
allocation is attempted from vmalloc space. Silence the first allocation
failure in that case by setting __GFP_NOWARN.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
 arch/arm/kernel/module.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c
index 80254b47dc34..503d1a39464a 100644
--- a/arch/arm/kernel/module.c
+++ b/arch/arm/kernel/module.c
@@ -40,8 +40,15 @@
 #ifdef CONFIG_MMU
 void *module_alloc(unsigned long size)
 {
-	void *p = __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END,
-				GFP_KERNEL, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE,
+	gfp_t gfp_mask = GFP_KERNEL;
+	void *p;
+
+#if IS_ENABLED(CONFIG_ARM_MODULE_PLTS)
+	/* Silence the initial allocation */
+	gfp_mask |= __GFP_NOWARN;
+#endif
+	p = __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END,
+				gfp_mask, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE,
 				__builtin_return_address(0));
 	if (!IS_ENABLED(CONFIG_ARM_MODULE_PLTS) || p)
 		return p;
-- 
2.9.3

[PATCH 2/2] mm: Silence vmap() allocation failures based on caller gfp_flags

[Florian Fainelli]

If the caller has set __GFP_NOWARN don't print the following message:
vmap allocation for size 15736832 failed: use vmalloc=<size> to increase
size.

This can happen with the ARM/Linux module loader built with
CONFIG_ARM_MODULE_PLTS=y which does a first attempt at loading a large
module from module space, then falls back to vmalloc space.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
 mm/vmalloc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 0b057628a7ba..5a788eb58741 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -521,7 +521,7 @@ static struct vmap_area *alloc_vmap_area(unsigned long size,
 		}
 	}
 
-	if (printk_ratelimit())
+	if (printk_ratelimit() && !(gfp_mask & __GFP_NOWARN))
 		pr_warn("vmap allocation for size %lu failed: use vmalloc=<size> to increase size\n",
 			size);
 	kfree(va);
-- 
2.9.3

[PATCH 3/3] arm64: Silence first allocation with CONFIG_ARM64_MODULE_PLTS=y

[Florian Fainelli]

When CONFIG_ARM64_MODULE_PLTS is enabled, the first allocation using the
module space fails, because the module is too big, and then the module
allocation is attempted from vmalloc space. Silence the first allocation
failure in that case by setting __GFP_NOWARN.

Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
 arch/arm64/kernel/module.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
index 7f316982ce00..58bd5cfdd544 100644
--- a/arch/arm64/kernel/module.c
+++ b/arch/arm64/kernel/module.c
@@ -32,11 +32,16 @@
 
 void *module_alloc(unsigned long size)
 {
+	gfp_t gfp_mask = GFP_KERNEL;
 	void *p;
 
+#if IS_ENABLED(CONFIG_ARM64_MODULE_PLTS)
+	/* Silence the initial allocation */
+	gfp_mask |= __GFP_NOWARN;
+#endif
 	p = __vmalloc_node_range(size, MODULE_ALIGN, module_alloc_base,
 				module_alloc_base + MODULES_VSIZE,
-				GFP_KERNEL, PAGE_KERNEL_EXEC, 0,
+				gfp_mask, PAGE_KERNEL_EXEC, 0,
 				NUMA_NO_NODE, __builtin_return_address(0));
 
 	if (!p && IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) &&
-- 
2.9.3

Re: [PATCH 0/3] ARM/ARM64: silence large module first time allocation

[Florian Fainelli]

On 04/25/2017 03:33 PM, Florian Fainelli wrote:
> With kernels built with CONFIG_ARM{,64}_MODULES_PLTS=y, the first allocation
> done from module space will fail, produce a general OOM allocation and also a
> vmap warning. The second allocation from vmalloc space may or may not be
> successful, but is actually the one we are interested about in these cases.
> 
> This patch series passed __GFP_NOWARN to silence such allocations from the
> ARM/ARM64 module loader's first time allocation when the MODULES_PLT option is
> enabled, and also makes alloc_vmap_area() react to the caller setting
> __GFP_NOWARN to silence "vmap allocation for size..." messages.
> 
> Here is an example of what we would get without these two patches, pretty
> scary huh?

Apologies for sending twice the same patches 1/2 and 2/2 should be
discarded...

> 
> # insmod /mnt/nfs/huge.ko 
> [   22.114143] random: nonblocking pool is initialized
> [   22.183575] vmap allocation for size 15736832 failed: use vmalloc=<size> to increase size.
> [   22.191873] vmalloc: allocation failure: 15729534 bytes
> [   22.197112] insmod: page allocation failure: order:0, mode:0xd0
> [   22.203048] CPU: 2 PID: 1506 Comm: insmod Tainted: G           O    4.1.20-1.9pre-01082-gbbbff07bc3ce #9
> [   22.212536] Hardware name: Broadcom STB (Flattened Device Tree)
> [   22.218480] [<c0017eec>] (unwind_backtrace) from [<c00135c8>] (show_stack+0x10/0x14)
> [   22.226238] [<c00135c8>] (show_stack) from [<c0638684>] (dump_stack+0x90/0xa4)
> [   22.233473] [<c0638684>] (dump_stack) from [<c00aae1c>] (warn_alloc_failed+0x104/0x144)
> [   22.241490] [<c00aae1c>] (warn_alloc_failed) from [<c00d72e0>] (__vmalloc_node_range+0x170/0x218)
> [   22.250375] [<c00d72e0>] (__vmalloc_node_range) from [<c00147d0>] (module_alloc+0x50/0xac)
> [   22.258651] [<c00147d0>] (module_alloc) from [<c008ae2c>] (module_alloc_update_bounds+0xc/0x6c)
> [   22.267360] [<c008ae2c>] (module_alloc_update_bounds) from [<c008b778>] (load_module+0x8ec/0x2058)
> [   22.276329] [<c008b778>] (load_module) from [<c008cfd4>] (SyS_init_module+0xf0/0x174)
> [   22.284170] [<c008cfd4>] (SyS_init_module) from [<c0010140>] (ret_fast_syscall+0x0/0x3c)
> [   22.292277] Mem-Info:
> [   22.294567] active_anon:5236 inactive_anon:1773 isolated_anon:0
> [   22.294567]  active_file:1 inactive_file:3822 isolated_file:0
> [   22.294567]  unevictable:0 dirty:0 writeback:0 unstable:0
> [   22.294567]  slab_reclaimable:238 slab_unreclaimable:1594
> [   22.294567]  mapped:855 shmem:2950 pagetables:36 bounce:0
> [   22.294567]  free:39031 free_pcp:198 free_cma:3928
> [   22.327196] DMA free:156124kB min:1880kB low:2348kB high:2820kB active_anon:20944kB inactive_anon:7092kB active_file:4kB inactive_file:15288kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:262144kB managed:227676kB mlocked:0kB dirty:0kB writeback:0kB mapped:3420kB shmem:11800kB slab_reclaimable:952kB slab_unreclaimable:6376kB kernel_stack:560kB pagetables:144kB unstable:0kB bounce:0kB free_pcp:792kB local_pcp:68kB free_cma:15712kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
> [   22.371631] lowmem_reserve[]: 0 0 0 0
> [   22.375372] HighMem free:0kB min:128kB low:128kB high:128kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:2883584kB managed:0kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
> [   22.416249] lowmem_reserve[]: 0 0 0 0
> [   22.419986] DMA: 3*4kB (UEM) 4*8kB (UE) 1*16kB (M) 4*32kB (UEMC) 3*64kB (EMC) 1*128kB (E) 4*256kB (UEMC) 2*512kB (UE) 2*1024kB (MC) 4*2048kB (UEMC) 35*4096kB (MRC) = 156156kB
> [   22.435922] HighMem: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB
> [   22.446130] 6789 total pagecache pages
> [   22.449889] 0 pages in swap cache
> [   22.453212] Swap cache stats: add 0, delete 0, find 0/0
> [   22.458447] Free swap  = 0kB
> [   22.461334] Total swap = 0kB
> [   22.464222] 786432 pages RAM
> [   22.467110] 720896 pages HighMem/MovableOnly
> [   22.471388] 725417 pages reserved
> [   22.474711] 4096 pages cma reserved
> [   22.511310] big_init: I am a big module using 3932160 bytes of data!
> 
> Florian Fainelli (3):
>   mm: Silence vmap() allocation failures based on caller gfp_flags
>   ARM: Silence first allocation with CONFIG_ARM_MODULE_PLTS=y
>   arm64: Silence first allocation with CONFIG_ARM64_MODULE_PLTS=y
> 
>  arch/arm/kernel/module.c   | 11 +++++++++--
>  arch/arm64/kernel/module.c |  7 ++++++-
>  mm/vmalloc.c               |  2 +-
>  3 files changed, 16 insertions(+), 4 deletions(-)
> 


-- 
Florian

Re: [PATCH 1/3] mm: Silence vmap() allocation failures based on caller gfp_flags

[Michal Hocko]

On Tue 25-04-17 15:33:29, Florian Fainelli wrote:
> If the caller has set __GFP_NOWARN don't print the following message:
> vmap allocation for size 15736832 failed: use vmalloc=<size> to increase
> size.
> 
> This can happen with the ARM/Linux module loader built with
> CONFIG_ARM_MODULE_PLTS=y which does a first attempt at loading a large
> module from module space, then falls back to vmalloc space.
> 
> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  mm/vmalloc.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 0b057628a7ba..5a788eb58741 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -521,7 +521,7 @@ static struct vmap_area *alloc_vmap_area(unsigned long size,
>  		}
>  	}
>  
> -	if (printk_ratelimit())
> +	if (printk_ratelimit() && !(gfp_mask & __GFP_NOWARN))

Are you sure about this ordering? Should NOWARN requests alter the
ratelimit state?

>  		pr_warn("vmap allocation for size %lu failed: use vmalloc=<size> to increase size\n",
>  			size);
>  	kfree(va);
> -- 
> 2.9.3
> 

-- 
Michal Hocko
SUSE Labs