* [PATCH 0/2] Clean up document and small improvement of atomic pool
@ 2021-12-17 9:08 Baoquan He
2021-12-17 9:08 ` [PATCH 1/2] docs: kernel-parameters: Update to reflect the current default size " Baoquan He
2021-12-17 9:08 ` [PATCH 2/2] dma-pool: allow user to disable " Baoquan He
0 siblings, 2 replies; 7+ messages in thread
From: Baoquan He @ 2021-12-17 9:08 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-mm, akpm, corbet, hch, cl, bhe
This patchset contains two small patches. Identified this when I tried
to fix a kdump bug that page allocation failure is always warned out
during kdump kernel on x86_64. Since dma atomic pool is initialized very
early during bootup, that allocation failure is always triggered in
dma_atomic_pool_init() firstly. I originally planned to add an kernel
option to disable each of the three dma atomic pool. Finally I realized
it's not only dma atomic pool getting the issue, any later requsting
page from DMA zone will fail. So I changed solution to fix the kdump bug.
While the document clean up and the small fix which allows user to
disable atomic pool makes sense to improve. The small fix which makes
'coherent_pool=0' disable dma atomic pool like 'cma=0', looks more
reasonable than the old behavioud which will take the default atomic
size.
[PATCH RESEND v2 0/5] Avoid requesting page from DMA zone when no managed pages
https://lore.kernel.org/all/20211207030750.30824-1-bhe@redhat.com/T/#u
Baoquan He (2):
docs: kernel-parameters: Update to reflect the current default size of
atomic pool
dma-pool: allow user to disable atomic pool
Documentation/admin-guide/kernel-parameters.txt | 5 ++++-
kernel/dma/pool.c | 7 +++++--
2 files changed, 9 insertions(+), 3 deletions(-)
--
2.26.0
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/2] docs: kernel-parameters: Update to reflect the current default size of atomic pool
2021-12-17 9:08 [PATCH 0/2] Clean up document and small improvement of atomic pool Baoquan He
@ 2021-12-17 9:08 ` Baoquan He
2021-12-21 8:50 ` Christoph Hellwig
2021-12-17 9:08 ` [PATCH 2/2] dma-pool: allow user to disable " Baoquan He
1 sibling, 1 reply; 7+ messages in thread
From: Baoquan He @ 2021-12-17 9:08 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-mm, akpm, corbet, hch, cl, bhe
Since commit 1d659236fb43("dma-pool: scale the default DMA coherent pool
size with memory capacity"), the default size of atomic pool has been
changed to take by scaling with system memory capacity. So update the
document in kerenl-parameter.txt accordingly.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
Documentation/admin-guide/kernel-parameters.txt | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 9725c546a0d4..ec4d25e854a8 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -664,7 +664,9 @@
coherent_pool=nn[KMG] [ARM,KNL]
Sets the size of memory pool for coherent, atomic dma
- allocations, by default set to 256K.
+ allocations. Otherwise the default size will be scaled
+ with memory capacity, while clamped between 128K and
+ 1 << (PAGE_SHIFT + MAX_ORDER-1).
com20020= [HW,NET] ARCnet - COM20020 chipset
Format:
--
2.26.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] docs: kernel-parameters: Update to reflect the current default size of atomic pool
2021-12-17 9:08 ` [PATCH 1/2] docs: kernel-parameters: Update to reflect the current default size " Baoquan He
@ 2021-12-21 8:50 ` Christoph Hellwig
0 siblings, 0 replies; 7+ messages in thread
From: Christoph Hellwig @ 2021-12-21 8:50 UTC (permalink / raw)
To: Baoquan He; +Cc: linux-kernel, linux-mm, akpm, corbet, hch, cl
On Fri, Dec 17, 2021 at 05:08:26PM +0800, Baoquan He wrote:
> Since commit 1d659236fb43("dma-pool: scale the default DMA coherent pool
> size with memory capacity"), the default size of atomic pool has been
> changed to take by scaling with system memory capacity. So update the
> document in kerenl-parameter.txt accordingly.
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/2] dma-pool: allow user to disable atomic pool
2021-12-17 9:08 [PATCH 0/2] Clean up document and small improvement of atomic pool Baoquan He
2021-12-17 9:08 ` [PATCH 1/2] docs: kernel-parameters: Update to reflect the current default size " Baoquan He
@ 2021-12-17 9:08 ` Baoquan He
2021-12-21 8:52 ` Christoph Hellwig
1 sibling, 1 reply; 7+ messages in thread
From: Baoquan He @ 2021-12-17 9:08 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-mm, akpm, corbet, hch, cl, bhe
In the current code, three atomic memory pools are always created,
atomic_pool_kernel|dma|dma32, even though 'coherent_pool=0' is
specified in kernel command line. In fact, atomic pool is only
necessary when CONFIG_DMA_DIRECT_REMAP=y or mem_encrypt_active=y
which are needed on few ARCHes.
So change code to allow user to disable atomic pool by specifying
'coherent_pool=0'.
Meanwhile, update the relevant document in kernel-parameter.txt.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
Documentation/admin-guide/kernel-parameters.txt | 3 ++-
kernel/dma/pool.c | 7 +++++--
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index ec4d25e854a8..d7015309614b 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -664,7 +664,8 @@
coherent_pool=nn[KMG] [ARM,KNL]
Sets the size of memory pool for coherent, atomic dma
- allocations. Otherwise the default size will be scaled
+ allocations. A value of 0 disables the three atomic
+ memory pool. Otherwise the default size will be scaled
with memory capacity, while clamped between 128K and
1 << (PAGE_SHIFT + MAX_ORDER-1).
diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
index 5f84e6cdb78e..5a85804b5beb 100644
--- a/kernel/dma/pool.c
+++ b/kernel/dma/pool.c
@@ -21,7 +21,7 @@ static struct gen_pool *atomic_pool_kernel __ro_after_init;
static unsigned long pool_size_kernel;
/* Size can be defined by the coherent_pool command line */
-static size_t atomic_pool_size;
+static unsigned long atomic_pool_size = -1;
/* Dynamic background expansion when the atomic pool is near capacity */
static struct work_struct atomic_pool_work;
@@ -188,11 +188,14 @@ static int __init dma_atomic_pool_init(void)
{
int ret = 0;
+ if (!atomic_pool_size)
+ return 0;
+
/*
* If coherent_pool was not used on the command line, default the pool
* sizes to 128KB per 1GB of memory, min 128KB, max MAX_ORDER-1.
*/
- if (!atomic_pool_size) {
+ if (atomic_pool_size == -1) {
unsigned long pages = totalram_pages() / (SZ_1G / SZ_128K);
pages = min_t(unsigned long, pages, MAX_ORDER_NR_PAGES);
atomic_pool_size = max_t(size_t, pages << PAGE_SHIFT, SZ_128K);
--
2.26.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] dma-pool: allow user to disable atomic pool
2021-12-17 9:08 ` [PATCH 2/2] dma-pool: allow user to disable " Baoquan He
@ 2021-12-21 8:52 ` Christoph Hellwig
0 siblings, 0 replies; 7+ messages in thread
From: Christoph Hellwig @ 2021-12-21 8:52 UTC (permalink / raw)
To: Baoquan He; +Cc: linux-kernel, linux-mm, akpm, corbet, hch, cl
On Fri, Dec 17, 2021 at 05:08:27PM +0800, Baoquan He wrote:
> In the current code, three atomic memory pools are always created,
> atomic_pool_kernel|dma|dma32, even though 'coherent_pool=0' is
> specified in kernel command line. In fact, atomic pool is only
> necessary when CONFIG_DMA_DIRECT_REMAP=y or mem_encrypt_active=y
> which are needed on few ARCHes.
>
> So change code to allow user to disable atomic pool by specifying
> 'coherent_pool=0'.
>
> Meanwhile, update the relevant document in kernel-parameter.txt.
>
> Signed-off-by: Baoquan He <bhe@redhat.com>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 3 ++-
> kernel/dma/pool.c | 7 +++++--
> 2 files changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index ec4d25e854a8..d7015309614b 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -664,7 +664,8 @@
>
> coherent_pool=nn[KMG] [ARM,KNL]
> Sets the size of memory pool for coherent, atomic dma
> - allocations. Otherwise the default size will be scaled
> + allocations. A value of 0 disables the three atomic
> + memory pool. Otherwise the default size will be scaled
> with memory capacity, while clamped between 128K and
> 1 << (PAGE_SHIFT + MAX_ORDER-1).
>
> diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
> index 5f84e6cdb78e..5a85804b5beb 100644
> --- a/kernel/dma/pool.c
> +++ b/kernel/dma/pool.c
> @@ -21,7 +21,7 @@ static struct gen_pool *atomic_pool_kernel __ro_after_init;
> static unsigned long pool_size_kernel;
>
> /* Size can be defined by the coherent_pool command line */
> -static size_t atomic_pool_size;
> +static unsigned long atomic_pool_size = -1;
Using UINT_MAX instead of -1 here and below would probably be more
descriptive.
Otherwise looks good:
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH RFC 0/2] dma-pool: allow user to disable atomic pool
@ 2021-06-24 5:20 Baoquan He
2021-06-24 5:20 ` Baoquan He
0 siblings, 1 reply; 7+ messages in thread
From: Baoquan He @ 2021-06-24 5:20 UTC (permalink / raw)
To: linux-kernel
Cc: linux-mm, x86, rientjes, rppt, thomas.lendacky, brijesh.singh,
kexec, bhe
On x86_64, when crash triggered, an OOM can always be observed in kdump
kernel as below:
~~~~~~~~~
DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations
swapper/0: page allocation failure: order:5, mode:0xcc1(GFP_KERNEL|GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.13.0-0.rc5.20210611git929d931f2b40.42.fc35.x86_64 #1
Hardware name: Dell Inc. PowerEdge R910/0P658H, BIOS 2.12.0 06/04/2018
Call Trace:
dump_stack+0x7f/0xa1
warn_alloc.cold+0x72/0xd6
? _raw_spin_unlock_irq+0x24/0x40
? __alloc_pages_direct_compact+0x90/0x1b0
__alloc_pages_slowpath.constprop.0+0xf29/0xf50
? __cond_resched+0x16/0x50
? prepare_alloc_pages.constprop.0+0x19d/0x1b0
__alloc_pages+0x24d/0x2c0
? __dma_atomic_pool_init+0x93/0x93
alloc_page_interleave+0x13/0xb0
atomic_pool_expand+0x118/0x210
? __dma_atomic_pool_init+0x93/0x93
__dma_atomic_pool_init+0x45/0x93
dma_atomic_pool_init+0xdb/0x176
do_one_initcall+0x67/0x320
? rcu_read_lock_sched_held+0x3f/0x80
kernel_init_freeable+0x290/0x2dc
? rest_init+0x24f/0x24f
kernel_init+0xa/0x111
ret_from_fork+0x22/0x30
Mem-Info:
......
DMA: failed to allocate 128 KiB GFP_KERNEL|GFP_DMA pool for atomic allocation
DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
~~~~~~~~~~~
This OOM can be noticed since commit f1d4d47c5851 ("x86/setup: Always
reserve the first 1M of RAM") is merged. The root cause is there's no
available memory in DMA zone in kdump kernel after commit f1d4d47c5851.
In the current atomic pool implementation, there are three atomic mem
pools: atomic_pool_kernel, atomic_pool_dma, atomic_pool_dma32, initialized
with flag GFP_KERNEL, GFP_KERNEL|GFP_DMA, GFP_KERNEL|GFP_DMA32. On x86_64,
normal kernel can allocate all three of them. While, kdump kernel can't
satisfy atomic_pool_dma initialization because there's only low-1M present
for DMA zone, and locked in commit f1d4d47c5851 so that the low-1M won't be
put in buddy allocator.
The atomic pool is generic, and enabled always no matter if
coherent_pool is specify in kernel cmdline or not. Seems the always enabling
of atomic pool is required by AMD MEM ENCRYPTION if CONFIG_DMA_DIRECT_REMAP
is not set, even though the system is non-AMD cpu, or non-x86 ARCHes.
AFAIK, SME requires swiotlb by default. Not sure if atomic has to be
provided, can we disable it in some cases, e.g in kdump kernel?
In this RFC patch, I change the current coherent_pool kernel parameter
to make it allow user to disable atomic pool if not needed with
coherent_pool=0.
If enabling atomic pool is mandatory for SME, maybe we can adjust and
add kernel parameter like, coherent_pool= to specify which pool is
needed, coherent_pool_size= to specify the initialization size:
coherent_pool= (bit0:kernel, bit1: dma, bit2:dma32,
coherent_pool_size= size (range from 128K to 4M).
Any comment or suggestion is appreciated.
Baoquan He (2):
docs: kernel-parameters: Update to reflect the current default size of
atomic pool
dma-pool: allow user to disable atomic pool
Documentation/admin-guide/kernel-parameters.txt | 4 +++-
kernel/dma/pool.c | 7 +++++--
2 files changed, 8 insertions(+), 3 deletions(-)
--
2.17.2
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/2] docs: kernel-parameters: Update to reflect the current default size of atomic pool
2021-06-24 5:20 [PATCH RFC 0/2] " Baoquan He
@ 2021-06-24 5:20 ` Baoquan He
0 siblings, 0 replies; 7+ messages in thread
From: Baoquan He @ 2021-06-24 5:20 UTC (permalink / raw)
To: linux-kernel
Cc: linux-mm, x86, rientjes, rppt, thomas.lendacky, brijesh.singh,
kexec, bhe
Since commit 1d659236fb43("dma-pool: scale the default DMA coherent pool
size with memory capacity"), the default size of atomic pool has been
changed to take by scaling with system memory capacity. So update the
document in kerenl-parameter.txt accordingly.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
Documentation/admin-guide/kernel-parameters.txt | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index cb89dbdedc46..2c5017db2621 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -623,7 +623,9 @@
coherent_pool=nn[KMG] [ARM,KNL]
Sets the size of memory pool for coherent, atomic dma
- allocations, by default set to 256K.
+ allocations. Otherwise the default size will be scaled
+ with memory capacity, while clamped between 128K and
+ 1 << (PAGE_SHIFT + MAX_ORDER-1).
com20020= [HW,NET] ARCnet - COM20020 chipset
Format:
--
2.17.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 1/2] docs: kernel-parameters: Update to reflect the current default size of atomic pool
@ 2021-06-24 5:20 ` Baoquan He
0 siblings, 0 replies; 7+ messages in thread
From: Baoquan He @ 2021-06-24 5:20 UTC (permalink / raw)
To: linux-kernel
Cc: linux-mm, x86, rientjes, rppt, thomas.lendacky, brijesh.singh,
kexec, bhe
Since commit 1d659236fb43("dma-pool: scale the default DMA coherent pool
size with memory capacity"), the default size of atomic pool has been
changed to take by scaling with system memory capacity. So update the
document in kerenl-parameter.txt accordingly.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
Documentation/admin-guide/kernel-parameters.txt | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index cb89dbdedc46..2c5017db2621 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -623,7 +623,9 @@
coherent_pool=nn[KMG] [ARM,KNL]
Sets the size of memory pool for coherent, atomic dma
- allocations, by default set to 256K.
+ allocations. Otherwise the default size will be scaled
+ with memory capacity, while clamped between 128K and
+ 1 << (PAGE_SHIFT + MAX_ORDER-1).
com20020= [HW,NET] ARCnet - COM20020 chipset
Format:
--
2.17.2
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-12-21 8:53 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-17 9:08 [PATCH 0/2] Clean up document and small improvement of atomic pool Baoquan He
2021-12-17 9:08 ` [PATCH 1/2] docs: kernel-parameters: Update to reflect the current default size " Baoquan He
2021-12-21 8:50 ` Christoph Hellwig
2021-12-17 9:08 ` [PATCH 2/2] dma-pool: allow user to disable " Baoquan He
2021-12-21 8:52 ` Christoph Hellwig
-- strict thread matches above, loose matches on Subject: below --
2021-06-24 5:20 [PATCH RFC 0/2] " Baoquan He
2021-06-24 5:20 ` [PATCH 1/2] docs: kernel-parameters: Update to reflect the current default size of " Baoquan He
2021-06-24 5:20 ` Baoquan He
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.