linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/3 v3] Fix kdump failures with crashkernel=high
@ 2015-06-05 10:29 Joerg Roedel
  2015-06-05 10:30 ` [PATCH 1/3] swiotlb: Warn on allocation failure in swiotlb_alloc_coherent Joerg Roedel
                   ` (3 more replies)
  0 siblings, 4 replies; 10+ messages in thread
From: Joerg Roedel @ 2015-06-05 10:29 UTC (permalink / raw)
  To: Ingo Molnar, Thomas Gleixner, H. Peter Anvin, Borislav Petkov
  Cc: Konrad Rzeszutek Wilk, Vivek Goyal, Dave Young, Baoquan He, x86,
	kexec, joro, jroedel, linux-kernel

v2->v3

* Rebased to v4.1-rc6
* Updated changelogs as requested and added outcome of
  previous discussions
* Added more Acked-bys

v1->v2:

* Updated comments based on feedback from Konrad
* Added Acked-bys
* Rebased to v3.19-rc3

Hi,

here is a patch-set to fix failed kdump kernel boots when
the systems was booted with crashkernel=X,high. On those
systems the kernel allocates only 72MiB of low-memory for
DMA buffers, which showed to be too low on some systems.

The problem is that 64MiB of the low-memory is allocated by
swiotlb, leaving 8MB for the page-allocator. But swiotlb
tries to allocate DMA memory from the page-allocator first,
which fails pretty fast in the boot sequence, causing
warnings. This patch-set removes these warnings.

But even the 64MiB for swiotlb are eaten up on some systems,
so that the default of low-memory allocated for the
crash-kernel is increase from 72MB to 256MB (only changing
the defaults, can still be overwritten by crashkernel=X,low).

This number comes from experiments on the affected systems,
128MiB low-memory was still not enough there, thus I set the
value to 256MiB to fix the issues.

Any feedback appreciated.

Thanks,

	Joerg

Joerg Roedel (3):
  swiotlb: Warn on allocation failure in swiotlb_alloc_coherent
  x86, swiotlb: Try coherent allocations with __GFP_NOWARN
  x86, crash: Allocate enough low-mem when crashkernel=high

 arch/x86/kernel/pci-swiotlb.c |  7 +++++++
 arch/x86/kernel/setup.c       |  5 ++++-
 lib/swiotlb.c                 | 11 +++++++++--
 3 files changed, 20 insertions(+), 3 deletions(-)

-- 
1.9.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* [PATCH 1/3] swiotlb: Warn on allocation failure in swiotlb_alloc_coherent
  2015-06-05 10:29 [PATCH 0/3 v3] Fix kdump failures with crashkernel=high Joerg Roedel
@ 2015-06-05 10:30 ` Joerg Roedel
  2015-06-12  8:49   ` [tip:x86/kdump] swiotlb: Warn on allocation failure in swiotlb_alloc_coherent() tip-bot for Joerg Roedel
  2015-06-05 10:30 ` [PATCH 2/3] x86, swiotlb: Try coherent allocations with __GFP_NOWARN Joerg Roedel
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 10+ messages in thread
From: Joerg Roedel @ 2015-06-05 10:30 UTC (permalink / raw)
  To: Ingo Molnar, Thomas Gleixner, H. Peter Anvin, Borislav Petkov
  Cc: Konrad Rzeszutek Wilk, Vivek Goyal, Dave Young, Baoquan He, x86,
	kexec, joro, jroedel, linux-kernel

From: Joerg Roedel <jroedel@suse.de>

Print a warning when all allocation tries have been failed
and the function is about to return NULL. This prepares for
calling the function with __GFP_NOWARN to suppress
allocation failure warnings before all fall-backs have
failed.

Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 lib/swiotlb.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 4abda07..e0e9212 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -655,7 +655,7 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 		 */
 		phys_addr_t paddr = map_single(hwdev, 0, size, DMA_FROM_DEVICE);
 		if (paddr == SWIOTLB_MAP_ERROR)
-			return NULL;
+			goto err_warn;
 
 		ret = phys_to_virt(paddr);
 		dev_addr = phys_to_dma(hwdev, paddr);
@@ -669,7 +669,7 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 			/* DMA_TO_DEVICE to avoid memcpy in unmap_single */
 			swiotlb_tbl_unmap_single(hwdev, paddr,
 						 size, DMA_TO_DEVICE);
-			return NULL;
+			goto err_warn;
 		}
 	}
 
@@ -677,6 +677,13 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 	memset(ret, 0, size);
 
 	return ret;
+
+err_warn:
+	pr_warn("swiotlb: coherent allocation failed for device %s size=%zu\n",
+		dev_name(hwdev), size);
+	dump_stack();
+
+	return NULL;
 }
 EXPORT_SYMBOL(swiotlb_alloc_coherent);
 
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH 2/3] x86, swiotlb: Try coherent allocations with __GFP_NOWARN
  2015-06-05 10:29 [PATCH 0/3 v3] Fix kdump failures with crashkernel=high Joerg Roedel
  2015-06-05 10:30 ` [PATCH 1/3] swiotlb: Warn on allocation failure in swiotlb_alloc_coherent Joerg Roedel
@ 2015-06-05 10:30 ` Joerg Roedel
  2015-06-12  8:50   ` [tip:x86/kdump] x86/swiotlb: " tip-bot for Joerg Roedel
  2015-06-05 10:30 ` [PATCH 3/3] x86, crash: Allocate enough low-mem when crashkernel=high Joerg Roedel
  2015-06-05 13:27 ` [PATCH 0/3 v3] Fix kdump failures with crashkernel=high Borislav Petkov
  3 siblings, 1 reply; 10+ messages in thread
From: Joerg Roedel @ 2015-06-05 10:30 UTC (permalink / raw)
  To: Ingo Molnar, Thomas Gleixner, H. Peter Anvin, Borislav Petkov
  Cc: Konrad Rzeszutek Wilk, Vivek Goyal, Dave Young, Baoquan He, x86,
	kexec, joro, jroedel, linux-kernel

From: Joerg Roedel <jroedel@suse.de>

When we boot a kdump kernel in high memory, there is by
default only 72MB of low memory available. The swiotlb code
takes 64MB of it (by default) so that there are only 8MB
left to allocate from. On systems with many devices this
causes page allocator warnings from dma_generic_alloc_coherent():

systemd-udevd: page allocation failure: order:0, mode:0x280d4
CPU: 0 PID: 197 Comm: systemd-udevd Tainted: G        W       3.12.28-4-default #1
Hardware name: HP ProLiant DL980 G7, BIOS P66 07/30/2012
 ffff8800781335e0 ffffffff8150b1db 00000000000280d4 ffffffff8113af90
 0000000000000000 0000000000000000 ffff88007efdbb00 0000000100000000
 0000000000000000 0000000000000000 0000000000000000 0000000000000001
Call Trace:
  dump_trace+0x7d/0x2d0
  show_stack_log_lvl+0x94/0x170
  show_stack+0x21/0x50
  dump_stack+0x41/0x51
  warn_alloc_failed+0xf0/0x160
  __alloc_pages_slowpath+0x72f/0x796
  __alloc_pages_nodemask+0x1ea/0x210
  dma_generic_alloc_coherent+0x96/0x140
  x86_swiotlb_alloc_coherent+0x1c/0x50
  ttm_dma_pool_alloc_new_pages+0xab/0x320 [ttm]
  ttm_dma_populate+0x3ce/0x640 [ttm]
  ttm_tt_bind+0x36/0x60 [ttm]
  ttm_bo_handle_move_mem+0x55f/0x5c0 [ttm]
  ttm_bo_move_buffer+0x105/0x130 [ttm]
  ttm_bo_validate+0xc1/0x130 [ttm]
  ttm_bo_init+0x24b/0x400 [ttm]
  radeon_bo_create+0x16c/0x200 [radeon]
  radeon_ring_init+0x11e/0x2b0 [radeon]
  r100_cp_init+0x123/0x5b0 [radeon]
  r100_startup+0x194/0x230 [radeon]
  r100_init+0x223/0x410 [radeon]
  radeon_device_init+0x6af/0x830 [radeon]
  radeon_driver_load_kms+0x89/0x180 [radeon]
  drm_get_pci_dev+0x121/0x2f0 [drm]
  local_pci_probe+0x39/0x60
  pci_device_probe+0xa9/0x120
  driver_probe_device+0x9d/0x3d0
  __driver_attach+0x8b/0x90
  bus_for_each_dev+0x5b/0x90
  bus_add_driver+0x1f8/0x2c0
  driver_register+0x5b/0xe0
  do_one_initcall+0xf2/0x1a0
  load_module+0x1207/0x1c70
  SYSC_finit_module+0x75/0xa0
  system_call_fastpath+0x16/0x1b
  0x7fac533d2788

After these warnings the code enters a fall-back path and
allocated directly from the swiotlb aperture in the end.
So remove these warnings as this is not a fatal error.

Acked-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 arch/x86/kernel/pci-swiotlb.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 77dd0ad..6d6894c 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -20,6 +20,13 @@ void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 {
 	void *vaddr;
 
+	/*
+         * Don't print a warning when the first allocation attempt
+         * fails. The swiotlb_alloc_coherent() function will print a
+         * warning when the allocation of DMA memory ultimatly failed.
+	 */
+	flags |= __GFP_NOWARN;
+
 	vaddr = dma_generic_alloc_coherent(hwdev, size, dma_handle, flags,
 					   attrs);
 	if (vaddr)
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH 3/3] x86, crash: Allocate enough low-mem when crashkernel=high
  2015-06-05 10:29 [PATCH 0/3 v3] Fix kdump failures with crashkernel=high Joerg Roedel
  2015-06-05 10:30 ` [PATCH 1/3] swiotlb: Warn on allocation failure in swiotlb_alloc_coherent Joerg Roedel
  2015-06-05 10:30 ` [PATCH 2/3] x86, swiotlb: Try coherent allocations with __GFP_NOWARN Joerg Roedel
@ 2015-06-05 10:30 ` Joerg Roedel
  2015-06-12  8:50   ` [tip:x86/kdump] x86/crash: Allocate enough low memory " tip-bot for Joerg Roedel
  2015-06-05 13:27 ` [PATCH 0/3 v3] Fix kdump failures with crashkernel=high Borislav Petkov
  3 siblings, 1 reply; 10+ messages in thread
From: Joerg Roedel @ 2015-06-05 10:30 UTC (permalink / raw)
  To: Ingo Molnar, Thomas Gleixner, H. Peter Anvin, Borislav Petkov
  Cc: Konrad Rzeszutek Wilk, Vivek Goyal, Dave Young, Baoquan He, x86,
	kexec, joro, jroedel, linux-kernel

From: Joerg Roedel <jroedel@suse.de>

When the crashkernel is loaded above 4GiB in memory the
first kernel only allocates 72MiB of low-memory for the DMA
requirements of the second kernel. On systems with many
devices this is not enough and causes device driver
initialization errors and failed crash dumps. Testing by
SUSE and Redhat have shown that 256MiB is a good default
value for now and the discussion has agreed on this value as
well. So set this default value to 256MiB to make sure there
is enough memory available for DMA.

Acked-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 arch/x86/kernel/setup.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index d74ac33..a09f368 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -535,8 +535,11 @@ static void __init reserve_crashkernel_low(void)
 		 *	swiotlb overflow buffer: now is hardcoded to 32k.
 		 *		We round it to 8M for other buffers that
 		 *		may need to stay low too.
+		 *		Also make sure we allocate enough extra memory
+		 *		low memory so that we don't run out of DMA
+		 *		buffers for 32bit devices.
 		 */
-		low_size = swiotlb_size_or_default() + (8UL<<20);
+		low_size = max(swiotlb_size_or_default() + (8UL<<20), 256UL<<20);
 		auto_set = true;
 	} else {
 		/* passed with crashkernel=0,low ? */
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH 0/3 v3] Fix kdump failures with crashkernel=high
  2015-06-05 10:29 [PATCH 0/3 v3] Fix kdump failures with crashkernel=high Joerg Roedel
                   ` (2 preceding siblings ...)
  2015-06-05 10:30 ` [PATCH 3/3] x86, crash: Allocate enough low-mem when crashkernel=high Joerg Roedel
@ 2015-06-05 13:27 ` Borislav Petkov
  2015-06-07 13:41   ` Ingo Molnar
  3 siblings, 1 reply; 10+ messages in thread
From: Borislav Petkov @ 2015-06-05 13:27 UTC (permalink / raw)
  To: Joerg Roedel, H. Peter Anvin, Ingo Molnar
  Cc: Thomas Gleixner, Konrad Rzeszutek Wilk, Vivek Goyal, Dave Young,
	Baoquan He, x86, kexec, jroedel, linux-kernel

On Fri, Jun 05, 2015 at 12:29:59PM +0200, Joerg Roedel wrote:
> v2->v3
> 
> * Rebased to v4.1-rc6
> * Updated changelogs as requested and added outcome of
>   previous discussions
> * Added more Acked-bys
> 
> v1->v2:
> 
> * Updated comments based on feedback from Konrad
> * Added Acked-bys
> * Rebased to v3.19-rc3
> 
> Hi,
> 
> here is a patch-set to fix failed kdump kernel boots when
> the systems was booted with crashkernel=X,high. On those
> systems the kernel allocates only 72MiB of low-memory for
> DMA buffers, which showed to be too low on some systems.
> 
> The problem is that 64MiB of the low-memory is allocated by
> swiotlb, leaving 8MB for the page-allocator. But swiotlb
> tries to allocate DMA memory from the page-allocator first,
> which fails pretty fast in the boot sequence, causing
> warnings. This patch-set removes these warnings.
> 
> But even the 64MiB for swiotlb are eaten up on some systems,
> so that the default of low-memory allocated for the
> crash-kernel is increase from 72MB to 256MB (only changing
> the defaults, can still be overwritten by crashkernel=X,low).
> 
> This number comes from experiments on the affected systems,
> 128MiB low-memory was still not enough there, thus I set the
> value to 256MiB to fix the issues.
> 
> Any feedback appreciated.
> 
> Thanks,
> 
> 	Joerg
> 
> Joerg Roedel (3):
>   swiotlb: Warn on allocation failure in swiotlb_alloc_coherent
>   x86, swiotlb: Try coherent allocations with __GFP_NOWARN
>   x86, crash: Allocate enough low-mem when crashkernel=high
> 
>  arch/x86/kernel/pci-swiotlb.c |  7 +++++++
>  arch/x86/kernel/setup.c       |  5 ++++-
>  lib/swiotlb.c                 | 11 +++++++++--
>  3 files changed, 20 insertions(+), 3 deletions(-)

Looks simple enough, all applied, thanks.

hpa, Ingo, any concerns?

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 0/3 v3] Fix kdump failures with crashkernel=high
  2015-06-05 13:27 ` [PATCH 0/3 v3] Fix kdump failures with crashkernel=high Borislav Petkov
@ 2015-06-07 13:41   ` Ingo Molnar
  2015-06-07 14:38     ` Borislav Petkov
  0 siblings, 1 reply; 10+ messages in thread
From: Ingo Molnar @ 2015-06-07 13:41 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Joerg Roedel, H. Peter Anvin, Ingo Molnar, Thomas Gleixner,
	Konrad Rzeszutek Wilk, Vivek Goyal, Dave Young, Baoquan He, x86,
	kexec, jroedel, linux-kernel


* Borislav Petkov <bp@alien8.de> wrote:

> > Joerg Roedel (3):
> >   swiotlb: Warn on allocation failure in swiotlb_alloc_coherent
> >   x86, swiotlb: Try coherent allocations with __GFP_NOWARN
> >   x86, crash: Allocate enough low-mem when crashkernel=high
> > 
> >  arch/x86/kernel/pci-swiotlb.c |  7 +++++++
> >  arch/x86/kernel/setup.c       |  5 ++++-
> >  lib/swiotlb.c                 | 11 +++++++++--
> >  3 files changed, 20 insertions(+), 3 deletions(-)
> 
> Looks simple enough, all applied, thanks.
> 
> hpa, Ingo, any concerns?

Modulo a couple of typos it all looks sane.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 0/3 v3] Fix kdump failures with crashkernel=high
  2015-06-07 13:41   ` Ingo Molnar
@ 2015-06-07 14:38     ` Borislav Petkov
  0 siblings, 0 replies; 10+ messages in thread
From: Borislav Petkov @ 2015-06-07 14:38 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Joerg Roedel, H. Peter Anvin, Ingo Molnar, Thomas Gleixner,
	Konrad Rzeszutek Wilk, Vivek Goyal, Dave Young, Baoquan He, x86,
	kexec, jroedel, linux-kernel

On Sun, Jun 07, 2015 at 03:41:36PM +0200, Ingo Molnar wrote:
> Modulo a couple of typos it all looks sane.

Ah yes, I caught a few while applying. I'll send you the final result
tomorrow.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [tip:x86/kdump] swiotlb: Warn on allocation failure in swiotlb_alloc_coherent()
  2015-06-05 10:30 ` [PATCH 1/3] swiotlb: Warn on allocation failure in swiotlb_alloc_coherent Joerg Roedel
@ 2015-06-12  8:49   ` tip-bot for Joerg Roedel
  0 siblings, 0 replies; 10+ messages in thread
From: tip-bot for Joerg Roedel @ 2015-06-12  8:49 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: torvalds, jroedel, bp, linux-kernel, akpm, hpa, dyoung, joro,
	peterz, tglx, konrad.wilk, brgerst, luto, dvlasenk, mingo, bp,
	vgoyal, bhe

Commit-ID:  94cc81f9a8f995923e35e2db936741dd62d18350
Gitweb:     http://git.kernel.org/tip/94cc81f9a8f995923e35e2db936741dd62d18350
Author:     Joerg Roedel <jroedel@suse.de>
AuthorDate: Wed, 10 Jun 2015 17:49:40 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 11 Jun 2015 08:28:38 +0200

swiotlb: Warn on allocation failure in swiotlb_alloc_coherent()

Print a warning when all allocation tries have been failed
and the function is about to return NULL.

This prepares for calling the function with __GFP_NOWARN to
suppress allocation failure warnings before all fall-backs
have failed - which we'll do to improve kdump behavior.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Baoquan He <bhe@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Jörg Rödel <joro@8bytes.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: kexec@lists.infradead.org
Link: http://lkml.kernel.org/r/1433500202-25531-2-git-send-email-joro@8bytes.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 lib/swiotlb.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 3c365ab..42e192d 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -656,7 +656,7 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 		 */
 		phys_addr_t paddr = map_single(hwdev, 0, size, DMA_FROM_DEVICE);
 		if (paddr == SWIOTLB_MAP_ERROR)
-			return NULL;
+			goto err_warn;
 
 		ret = phys_to_virt(paddr);
 		dev_addr = phys_to_dma(hwdev, paddr);
@@ -670,7 +670,7 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 			/* DMA_TO_DEVICE to avoid memcpy in unmap_single */
 			swiotlb_tbl_unmap_single(hwdev, paddr,
 						 size, DMA_TO_DEVICE);
-			return NULL;
+			goto err_warn;
 		}
 	}
 
@@ -678,6 +678,13 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 	memset(ret, 0, size);
 
 	return ret;
+
+err_warn:
+	pr_warn("swiotlb: coherent allocation failed for device %s size=%zu\n",
+		dev_name(hwdev), size);
+	dump_stack();
+
+	return NULL;
 }
 EXPORT_SYMBOL(swiotlb_alloc_coherent);
 

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [tip:x86/kdump] x86/swiotlb: Try coherent allocations with __GFP_NOWARN
  2015-06-05 10:30 ` [PATCH 2/3] x86, swiotlb: Try coherent allocations with __GFP_NOWARN Joerg Roedel
@ 2015-06-12  8:50   ` tip-bot for Joerg Roedel
  0 siblings, 0 replies; 10+ messages in thread
From: tip-bot for Joerg Roedel @ 2015-06-12  8:50 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: tglx, peterz, akpm, joro, dvlasenk, bp, luto, bp, linux-kernel,
	torvalds, hpa, mingo, bhe, jroedel, dyoung, vgoyal, brgerst

Commit-ID:  186dfc9d69b96a38ec6ec654127dba4432184494
Gitweb:     http://git.kernel.org/tip/186dfc9d69b96a38ec6ec654127dba4432184494
Author:     Joerg Roedel <jroedel@suse.de>
AuthorDate: Wed, 10 Jun 2015 17:49:41 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 11 Jun 2015 08:28:38 +0200

x86/swiotlb: Try coherent allocations with __GFP_NOWARN

When we boot a kdump kernel in high memory, there is by
default only 72MB of low memory available. The swiotlb code
takes 64MB of it (by default) so that there are only 8MB
left to allocate from. On systems with many devices this
causes page allocator warnings from
dma_generic_alloc_coherent():

  systemd-udevd: page allocation failure: order:0, mode:0x280d4
  CPU: 0 PID: 197 Comm: systemd-udevd Tainted: G        W
  3.12.28-4-default #1 Hardware name: HP ProLiant DL980 G7, BIOS
  P66 07/30/2012  ffff8800781335e0 ffffffff8150b1db 00000000000280d4 ffffffff8113af90
   0000000000000000 0000000000000000 ffff88007efdbb00 0000000100000000
   0000000000000000 0000000000000000 0000000000000000 0000000000000001
  Call Trace:
    dump_trace+0x7d/0x2d0
    show_stack_log_lvl+0x94/0x170
    show_stack+0x21/0x50
    dump_stack+0x41/0x51
    warn_alloc_failed+0xf0/0x160
    __alloc_pages_slowpath+0x72f/0x796
    __alloc_pages_nodemask+0x1ea/0x210
    dma_generic_alloc_coherent+0x96/0x140
    x86_swiotlb_alloc_coherent+0x1c/0x50
    ttm_dma_pool_alloc_new_pages+0xab/0x320 [ttm]
    ttm_dma_populate+0x3ce/0x640 [ttm]
    ttm_tt_bind+0x36/0x60 [ttm]
    ttm_bo_handle_move_mem+0x55f/0x5c0 [ttm]
    ttm_bo_move_buffer+0x105/0x130 [ttm]
    ttm_bo_validate+0xc1/0x130 [ttm]
    ttm_bo_init+0x24b/0x400 [ttm]
    radeon_bo_create+0x16c/0x200 [radeon]
    radeon_ring_init+0x11e/0x2b0 [radeon]
    r100_cp_init+0x123/0x5b0 [radeon]
    r100_startup+0x194/0x230 [radeon]
    r100_init+0x223/0x410 [radeon]
    radeon_device_init+0x6af/0x830 [radeon]
    radeon_driver_load_kms+0x89/0x180 [radeon]
    drm_get_pci_dev+0x121/0x2f0 [drm]
    local_pci_probe+0x39/0x60
    pci_device_probe+0xa9/0x120
    driver_probe_device+0x9d/0x3d0
    __driver_attach+0x8b/0x90
    bus_for_each_dev+0x5b/0x90
    bus_add_driver+0x1f8/0x2c0
    driver_register+0x5b/0xe0
    do_one_initcall+0xf2/0x1a0
    load_module+0x1207/0x1c70
    SYSC_finit_module+0x75/0xa0
    system_call_fastpath+0x16/0x1b
    0x7fac533d2788

After these warnings the code enters a fall-back path and
allocated directly from the swiotlb aperture in the end.
So remove these warnings as this is not a fatal error.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
[ Simplify, reflow comment. ]
Signed-off-by: Borislav Petkov <bp@suse.de>
Acked-by: Baoquan He <bhe@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Jörg Rödel <joro@8bytes.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: kexec@lists.infradead.org
Link: http://lkml.kernel.org/r/1433500202-25531-3-git-send-email-joro@8bytes.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/kernel/pci-swiotlb.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 77dd0ad..adf0392 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -20,6 +20,13 @@ void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 {
 	void *vaddr;
 
+	/*
+	 * Don't print a warning when the first allocation attempt fails.
+	 * swiotlb_alloc_coherent() will print a warning when the DMA
+	 * memory allocation ultimately failed.
+	 */
+	flags |= __GFP_NOWARN;
+
 	vaddr = dma_generic_alloc_coherent(hwdev, size, dma_handle, flags,
 					   attrs);
 	if (vaddr)

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [tip:x86/kdump] x86/crash: Allocate enough low memory when crashkernel=high
  2015-06-05 10:30 ` [PATCH 3/3] x86, crash: Allocate enough low-mem when crashkernel=high Joerg Roedel
@ 2015-06-12  8:50   ` tip-bot for Joerg Roedel
  0 siblings, 0 replies; 10+ messages in thread
From: tip-bot for Joerg Roedel @ 2015-06-12  8:50 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: peterz, akpm, dvlasenk, hpa, bhe, linux-kernel, mingo, luto,
	vgoyal, joro, dyoung, tglx, jroedel, torvalds, bp, brgerst, bp

Commit-ID:  94fb9334182284e8e7e4bcb9125c25dc33af19d4
Gitweb:     http://git.kernel.org/tip/94fb9334182284e8e7e4bcb9125c25dc33af19d4
Author:     Joerg Roedel <jroedel@suse.de>
AuthorDate: Wed, 10 Jun 2015 17:49:42 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 11 Jun 2015 08:28:39 +0200

x86/crash: Allocate enough low memory when crashkernel=high

When the crash kernel is loaded above 4GiB in memory, the
first kernel allocates only 72MiB of low-memory for the DMA
requirements of the second kernel. On systems with many
devices this is not enough and causes device driver
initialization errors and failed crash dumps. Testing by
SUSE and Redhat has shown that 256MiB is a good default
value for now and the discussion has lead to this value as
well. So set this default value to 256MiB to make sure there
is enough memory available for DMA.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
[ Reflow comment. ]
Signed-off-by: Borislav Petkov <bp@suse.de>
Acked-by: Baoquan He <bhe@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Jörg Rödel <joro@8bytes.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: kexec@lists.infradead.org
Link: http://lkml.kernel.org/r/1433500202-25531-4-git-send-email-joro@8bytes.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/kernel/setup.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index d74ac33..cba8288 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -531,12 +531,14 @@ static void __init reserve_crashkernel_low(void)
 	if (ret != 0) {
 		/*
 		 * two parts from lib/swiotlb.c:
-		 *	swiotlb size: user specified with swiotlb= or default.
-		 *	swiotlb overflow buffer: now is hardcoded to 32k.
-		 *		We round it to 8M for other buffers that
-		 *		may need to stay low too.
+		 * -swiotlb size: user-specified with swiotlb= or default.
+		 *
+		 * -swiotlb overflow buffer: now hardcoded to 32k. We round it
+		 * to 8M for other buffers that may need to stay low too. Also
+		 * make sure we allocate enough extra low memory so that we
+		 * don't run out of DMA buffers for 32-bit devices.
 		 */
-		low_size = swiotlb_size_or_default() + (8UL<<20);
+		low_size = max(swiotlb_size_or_default() + (8UL<<20), 256UL<<20);
 		auto_set = true;
 	} else {
 		/* passed with crashkernel=0,low ? */

^ permalink raw reply related	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2015-06-12  8:52 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-06-05 10:29 [PATCH 0/3 v3] Fix kdump failures with crashkernel=high Joerg Roedel
2015-06-05 10:30 ` [PATCH 1/3] swiotlb: Warn on allocation failure in swiotlb_alloc_coherent Joerg Roedel
2015-06-12  8:49   ` [tip:x86/kdump] swiotlb: Warn on allocation failure in swiotlb_alloc_coherent() tip-bot for Joerg Roedel
2015-06-05 10:30 ` [PATCH 2/3] x86, swiotlb: Try coherent allocations with __GFP_NOWARN Joerg Roedel
2015-06-12  8:50   ` [tip:x86/kdump] x86/swiotlb: " tip-bot for Joerg Roedel
2015-06-05 10:30 ` [PATCH 3/3] x86, crash: Allocate enough low-mem when crashkernel=high Joerg Roedel
2015-06-12  8:50   ` [tip:x86/kdump] x86/crash: Allocate enough low memory " tip-bot for Joerg Roedel
2015-06-05 13:27 ` [PATCH 0/3 v3] Fix kdump failures with crashkernel=high Borislav Petkov
2015-06-07 13:41   ` Ingo Molnar
2015-06-07 14:38     ` Borislav Petkov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).