All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] mm, sparse: do not swamp log with huge vmemmap allocation failures
@ 2017-11-06  9:22 ` Michal Hocko
  0 siblings, 0 replies; 51+ messages in thread
From: Michal Hocko @ 2017-11-06  9:22 UTC (permalink / raw)
  To: Andrew Morton, Johannes Weiner
  Cc: Vlastimil Babka, linux-mm, LKML, Michal Hocko

From: Michal Hocko <mhocko@suse.com>

While doing a memory hotplug tests under a heavy memory pressure we have
noticed too many page allocation failures when allocating vmemmap memmap
backed by huge page
[146792.281354] kworker/u3072:1: page allocation failure: order:9, mode:0x24084c0(GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO)
[...]
[146792.281394] Call Trace:
[146792.281430]  [<ffffffff81019a99>] dump_trace+0x59/0x310
[146792.281436]  [<ffffffff81019e3a>] show_stack_log_lvl+0xea/0x170
[146792.281440]  [<ffffffff8101abc1>] show_stack+0x21/0x40
[146792.281448]  [<ffffffff8130f040>] dump_stack+0x5c/0x7c
[146792.281464]  [<ffffffff8118c982>] warn_alloc_failed+0xe2/0x150
[146792.281471]  [<ffffffff8118cddd>] __alloc_pages_nodemask+0x3ed/0xb20
[146792.281489]  [<ffffffff811d3aaf>] alloc_pages_current+0x7f/0x100
[146792.281503]  [<ffffffff815dfa2c>] vmemmap_alloc_block+0x79/0xb6
[146792.281510]  [<ffffffff815dfbd3>] __vmemmap_alloc_block_buf+0x136/0x145
[146792.281524]  [<ffffffff815dd0c5>] vmemmap_populate+0xd2/0x2b9
[146792.281529]  [<ffffffff815dffd9>] sparse_mem_map_populate+0x23/0x30
[146792.281532]  [<ffffffff815df88d>] sparse_add_one_section+0x68/0x18e
[146792.281537]  [<ffffffff815d9f5a>] __add_pages+0x10a/0x1d0
[146792.281553]  [<ffffffff8106249a>] arch_add_memory+0x4a/0xc0
[146792.281559]  [<ffffffff815da1f9>] add_memory_resource+0x89/0x160
[146792.281564]  [<ffffffff815da33d>] add_memory+0x6d/0xd0
[146792.281585]  [<ffffffff813d36c4>] acpi_memory_device_add+0x181/0x251
[146792.281597]  [<ffffffff813946e5>] acpi_bus_attach+0xfd/0x19b
[146792.281602]  [<ffffffff81394866>] acpi_bus_scan+0x59/0x69
[146792.281604]  [<ffffffff813949de>] acpi_device_hotplug+0xd2/0x41f
[146792.281608]  [<ffffffff8138db67>] acpi_hotplug_work_fn+0x1a/0x23
[146792.281623]  [<ffffffff81093cee>] process_one_work+0x14e/0x410
[146792.281630]  [<ffffffff81094546>] worker_thread+0x116/0x490
[146792.281637]  [<ffffffff810999ed>] kthread+0xbd/0xe0
[146792.281651]  [<ffffffff815e4e7f>] ret_from_fork+0x3f/0x70

and we do see many of those because essentially every the allocation
failes for each memory section. This is overly excessive way to tell
user that there is nothing to really worry about because we do have
a fallback mechanism to use base pages. The only downside might be a
performance degradation due to TLB pressure.

This patch changes vmemmap_alloc_block to use __GFP_NOWARN and warn
explicitly once on the first allocation failure. This will reduce the
noise in the kernel log considerably, while we still have an indication
that a performance might be impacted.

Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
Hi,
this has somehow fell of my radar completely. The patch is essentially
what Johannes suggested [1] so I have added his s-o-b and added the
changelog into it.

Can we have this merged?

[1] http://lkml.kernel.org/r/20170711214541.GA11141@cmpxchg.org

 arch/x86/mm/init_64.c |  1 -
 mm/sparse-vmemmap.c   | 11 +++++++++--
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index 126e09625979..5eb954f930be 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -1405,7 +1405,6 @@ static int __meminit vmemmap_populate_hugepages(unsigned long start,
 			vmemmap_verify((pte_t *)pmd, node, addr, next);
 			continue;
 		}
-		pr_warn_once("vmemmap: falling back to regular page backing\n");
 		if (vmemmap_populate_basepages(addr, next, node))
 			return -ENOMEM;
 	}
diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
index d0860aab1c89..3f85084cb8bb 100644
--- a/mm/sparse-vmemmap.c
+++ b/mm/sparse-vmemmap.c
@@ -52,12 +52,19 @@ void * __meminit vmemmap_alloc_block(unsigned long size, int node)
 {
 	/* If the main allocator is up use that, fallback to bootmem. */
 	if (slab_is_available()) {
+		gfp_t gfp_mask = GFP_KERNEL | __GFP_RETRY_MAYFAIL | __GFP_NOWARN;
+		int order = get_order(size);
+		static bool warned;
 		struct page *page;
 
-		page = alloc_pages_node(node, GFP_KERNEL | __GFP_RETRY_MAYFAIL,
-					get_order(size));
+		page = alloc_pages_node(node, gfp_mask, order);
 		if (page)
 			return page_address(page);
+
+		if (!warned) {
+			warn_alloc(gfp_mask, NULL, "vmemmap alloc failure: order:%u", order);
+			warned = true;
+		}
 		return NULL;
 	} else
 		return __earlyonly_bootmem_alloc(node, size, size,
-- 
2.14.2

^ permalink raw reply related	[flat|nested] 51+ messages in thread
* [RFC PATCH for 4.15 00/14] Restartable sequences and CPU op vector v10
@ 2017-11-06 20:56 ` Mathieu Desnoyers
  0 siblings, 0 replies; 51+ messages in thread
From: Mathieu Desnoyers @ 2017-11-06 20:56 UTC (permalink / raw)
  To: Peter Zijlstra, Paul E . McKenney, Boqun Feng, Andy Lutomirski,
	Dave Watson
  Cc: linux-kernel, linux-api, Paul Turner, Andrew Morton,
	Russell King, Thomas Gleixner, Ingo Molnar, H . Peter Anvin,
	Andrew Hunter, Andi Kleen, Chris Lameter, Ben Maurer,
	Steven Rostedt, Josh Triplett, Linus Torvalds, Catalin Marinas,
	Will Deacon, Michael Kerrisk, Mathieu Desnoyers

Here is an updated rseq patchset taking into account the feedback
received at kernel summit and afterwards.

Use-cases explanation and benchmarks can be found in patch 01
"Restartable sequences system call".

This is still submitted as RFC. I'm keeping a linux-rseq
tree with this patchset at:

https://git.kernel.org/pub/scm/linux/kernel/git/rseq/linux-rseq.git/tag/?h=v4.14-rc8-rseq-20171106.1

If everyone is happy with it, I plan to submit this officially
when the merge window opens.

Thanks,

Mathieu

Boqun Feng (2):
  Restartable sequences: powerpc architecture support
  Restartable sequences: Wire up powerpc system call

Mathieu Desnoyers (12):
  Restartable sequences system call (v10)
  Restartable sequences: ARM 32 architecture support
  Restartable sequences: wire up ARM 32 system call
  Restartable sequences: x86 32/64 architecture support
  Restartable sequences: wire up x86 32/64 system call
  Provide cpu_opv system call (v2)
  cpu_opv: Wire up x86 32/64 system call
  cpu_opv: Wire up powerpc system call
  cpu_opv: Wire up ARM32 system call
  cpu_opv: Implement selftests (v2)
  Restartable sequences: Provide self-tests (v2)
  Restartable sequences selftests: arm: workaround gcc asm size guess

 MAINTAINERS                                        |   20 +
 arch/Kconfig                                       |    7 +
 arch/arm/Kconfig                                   |    1 +
 arch/arm/kernel/signal.c                           |    7 +
 arch/arm/tools/syscall.tbl                         |    2 +
 arch/powerpc/Kconfig                               |    1 +
 arch/powerpc/include/asm/systbl.h                  |    2 +
 arch/powerpc/include/asm/unistd.h                  |    2 +-
 arch/powerpc/include/uapi/asm/unistd.h             |    2 +
 arch/powerpc/kernel/signal.c                       |    3 +
 arch/x86/Kconfig                                   |    1 +
 arch/x86/entry/common.c                            |    1 +
 arch/x86/entry/syscalls/syscall_32.tbl             |    2 +
 arch/x86/entry/syscalls/syscall_64.tbl             |    2 +
 arch/x86/kernel/signal.c                           |    6 +
 fs/exec.c                                          |    1 +
 include/linux/sched.h                              |   89 ++
 include/trace/events/rseq.h                        |   60 +
 include/uapi/linux/cpu_opv.h                       |  117 ++
 include/uapi/linux/rseq.h                          |  138 +++
 init/Kconfig                                       |   28 +
 kernel/Makefile                                    |    2 +
 kernel/cpu_opv.c                                   |  952 +++++++++++++++
 kernel/fork.c                                      |    2 +
 kernel/rseq.c                                      |  329 +++++
 kernel/sched/core.c                                |   41 +
 kernel/sched/sched.h                               |    2 +
 kernel/sys_ni.c                                    |    4 +
 tools/testing/selftests/Makefile                   |    2 +
 tools/testing/selftests/cpu-opv/.gitignore         |    1 +
 tools/testing/selftests/cpu-opv/Makefile           |   15 +
 .../testing/selftests/cpu-opv/basic_cpu_opv_test.c | 1157 ++++++++++++++++++
 tools/testing/selftests/cpu-opv/cpu-op.c           |  348 ++++++
 tools/testing/selftests/cpu-opv/cpu-op.h           |   68 ++
 tools/testing/selftests/rseq/.gitignore            |    4 +
 tools/testing/selftests/rseq/Makefile              |   22 +
 .../testing/selftests/rseq/basic_percpu_ops_test.c |  333 +++++
 tools/testing/selftests/rseq/basic_test.c          |   55 +
 tools/testing/selftests/rseq/param_test.c          | 1285 ++++++++++++++++++++
 tools/testing/selftests/rseq/rseq-arm.h            |  568 +++++++++
 tools/testing/selftests/rseq/rseq-ppc.h            |  567 +++++++++
 tools/testing/selftests/rseq/rseq-x86.h            |  898 ++++++++++++++
 tools/testing/selftests/rseq/rseq.c                |  116 ++
 tools/testing/selftests/rseq/rseq.h                |  154 +++
 tools/testing/selftests/rseq/run_param_test.sh     |  124 ++
 45 files changed, 7540 insertions(+), 1 deletion(-)
 create mode 100644 include/trace/events/rseq.h
 create mode 100644 include/uapi/linux/cpu_opv.h
 create mode 100644 include/uapi/linux/rseq.h
 create mode 100644 kernel/cpu_opv.c
 create mode 100644 kernel/rseq.c
 create mode 100644 tools/testing/selftests/cpu-opv/.gitignore
 create mode 100644 tools/testing/selftests/cpu-opv/Makefile
 create mode 100644 tools/testing/selftests/cpu-opv/basic_cpu_opv_test.c
 create mode 100644 tools/testing/selftests/cpu-opv/cpu-op.c
 create mode 100644 tools/testing/selftests/cpu-opv/cpu-op.h
 create mode 100644 tools/testing/selftests/rseq/.gitignore
 create mode 100644 tools/testing/selftests/rseq/Makefile
 create mode 100644 tools/testing/selftests/rseq/basic_percpu_ops_test.c
 create mode 100644 tools/testing/selftests/rseq/basic_test.c
 create mode 100644 tools/testing/selftests/rseq/param_test.c
 create mode 100644 tools/testing/selftests/rseq/rseq-arm.h
 create mode 100644 tools/testing/selftests/rseq/rseq-ppc.h
 create mode 100644 tools/testing/selftests/rseq/rseq-x86.h
 create mode 100644 tools/testing/selftests/rseq/rseq.c
 create mode 100644 tools/testing/selftests/rseq/rseq.h
 create mode 100755 tools/testing/selftests/rseq/run_param_test.sh

-- 
2.11.0

^ permalink raw reply	[flat|nested] 51+ messages in thread

end of thread, other threads:[~2017-11-07  9:06 UTC | newest]

Thread overview: 51+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-06  9:22 [PATCH] mm, sparse: do not swamp log with huge vmemmap allocation failures Michal Hocko
2017-11-06  9:22 ` Michal Hocko
2017-11-06 17:35 ` Johannes Weiner
2017-11-06 17:35   ` Johannes Weiner
2017-11-06 17:57   ` Joe Perches
2017-11-06 18:14 ` Khalid Aziz
2017-11-06 18:14   ` Khalid Aziz
2017-11-06 18:18   ` Michal Hocko
2017-11-06 18:18     ` Michal Hocko
2017-11-06 20:17     ` Khalid Aziz
2017-11-06 20:17       ` Khalid Aziz
2017-11-07  9:06 ` Michal Hocko
2017-11-07  9:06   ` Michal Hocko
2017-11-06 20:56 [RFC PATCH for 4.15 00/14] Restartable sequences and CPU op vector v10 Mathieu Desnoyers
2017-11-06 20:56 ` Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH v10 for 4.15 01/14] Restartable sequences system call Mathieu Desnoyers
2017-11-07  1:24   ` Boqun Feng
2017-11-07  1:24     ` Boqun Feng
2017-11-07  2:20     ` Mathieu Desnoyers
2017-11-07  2:20       ` Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH for 4.15 02/14] Restartable sequences: ARM 32 architecture support Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH for 4.15 03/14] Restartable sequences: wire up ARM 32 system call Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH for 4.15 04/14] Restartable sequences: x86 32/64 architecture support Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH for 4.15 05/14] Restartable sequences: wire up x86 32/64 system call Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH for 4.15 06/14] Restartable sequences: powerpc architecture support Mathieu Desnoyers
2017-11-06 20:56   ` Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH for 4.15 07/14] Restartable sequences: Wire up powerpc system call Mathieu Desnoyers
2017-11-06 20:56   ` Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH v2 for 4.15 08/14] Provide cpu_opv " Mathieu Desnoyers
2017-11-07  2:07   ` Boqun Feng
2017-11-07  2:07     ` Boqun Feng
2017-11-07  2:40     ` Mathieu Desnoyers
2017-11-07  2:40       ` Mathieu Desnoyers
2017-11-07  3:03       ` Boqun Feng
2017-11-07  3:03         ` Boqun Feng
2017-11-06 20:56 ` [RFC PATCH for 4.15 09/14] cpu_opv: Wire up x86 32/64 " Mathieu Desnoyers
2017-11-06 20:56   ` Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH for 4.15 10/14] cpu_opv: Wire up powerpc " Mathieu Desnoyers
2017-11-06 20:56   ` Mathieu Desnoyers
2017-11-07  0:37   ` Nicholas Piggin
2017-11-07  0:37     ` Nicholas Piggin
2017-11-07  0:47     ` Mathieu Desnoyers
2017-11-07  0:47       ` Mathieu Desnoyers
2017-11-07  1:21       ` Nicholas Piggin
2017-11-07  1:21         ` Nicholas Piggin
2017-11-06 20:56 ` [RFC PATCH for 4.15 11/14] cpu_opv: Wire up ARM32 " Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH v2 for 4.15 12/14] cpu_opv: Implement selftests Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH v2 for 4.15 13/14] Restartable sequences: Provide self-tests Mathieu Desnoyers
2017-11-06 20:56   ` Mathieu Desnoyers
2017-11-06 20:56 ` [RFC PATCH for 4.15 14/14] Restartable sequences selftests: arm: workaround gcc asm size guess Mathieu Desnoyers
2017-11-06 20:56   ` Mathieu Desnoyers

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.