[PATCH] mm: clarify COMPACTION Kconfig text

[PATCH] mm: clarify COMPACTION Kconfig text

[Michal Hocko]

From: Michal Hocko <mhocko@suse.com>

The current wording of the COMPACTION Kconfig help text doesn't
emphasise that disabling COMPACTION might cripple the page allocator
which relies on the compaction quite heavily for high order requests and
an unexpected OOM can happen with the lack of compaction. Make sure
we are vocal about that.

Signed-off-by: Michal Hocko <mhocko@suse.com>
---
 mm/Kconfig | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/mm/Kconfig b/mm/Kconfig
index 78a23c5c302d..0dff2f05b6d1 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -262,7 +262,14 @@ config COMPACTION
 	select MIGRATION
 	depends on MMU
 	help
-	  Allows the compaction of memory for the allocation of huge pages.
+          Compaction is the only memory management component to form
+          high order (larger physically contiguous) memory blocks
+          reliably. Page allocator relies on the compaction heavily and
+          the lack of the feature can lead to unexpected OOM killer
+          invocation for high order memory requests. You shouldnm't
+          disable this option unless there is really a strong reason for
+          it and then we are really interested to hear about that at
+          linux-mm@kvack.org.
 
 #
 # support for page migration
-- 
2.8.1

Re: [PATCH] mm: clarify COMPACTION Kconfig text

[Markus Trippelsdorf]

On 2016.08.23 at 10:09 +0200, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
> 
> The current wording of the COMPACTION Kconfig help text doesn't
> emphasise that disabling COMPACTION might cripple the page allocator
> which relies on the compaction quite heavily for high order requests and
> an unexpected OOM can happen with the lack of compaction. Make sure
> we are vocal about that.

Just a few nitpicks inline below:

>  mm/Kconfig | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 78a23c5c302d..0dff2f05b6d1 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -262,7 +262,14 @@ config COMPACTION
>  	select MIGRATION
>  	depends on MMU
>  	help
> -	  Allows the compaction of memory for the allocation of huge pages.
> +          Compaction is the only memory management component to form
> +          high order (larger physically contiguous) memory blocks
> +          reliably. Page allocator relies on the compaction heavily and
                       The page allo...      on compaction    
> +          the lack of the feature can lead to unexpected OOM killer
> +          invocation for high order memory requests. You shouldnm't
             invocations                                    shouldn't  
> +          disable this option unless there is really a strong reason for
                                              really is      
> +          it and then we are really interested to hear about that at
                            would be    

-- 
Markus

Re: [PATCH] mm: clarify COMPACTION Kconfig text

[Michal Hocko]

On Tue 23-08-16 10:38:30, Markus Trippelsdorf wrote:
> On 2016.08.23 at 10:09 +0200, Michal Hocko wrote:
> > From: Michal Hocko <mhocko@suse.com>
> > 
> > The current wording of the COMPACTION Kconfig help text doesn't
> > emphasise that disabling COMPACTION might cripple the page allocator
> > which relies on the compaction quite heavily for high order requests and
> > an unexpected OOM can happen with the lack of compaction. Make sure
> > we are vocal about that.
> 
> Just a few nitpicks inline below:
> 
> >  mm/Kconfig | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> > 
> > diff --git a/mm/Kconfig b/mm/Kconfig
> > index 78a23c5c302d..0dff2f05b6d1 100644
> > --- a/mm/Kconfig
> > +++ b/mm/Kconfig
> > @@ -262,7 +262,14 @@ config COMPACTION
> >  	select MIGRATION
> >  	depends on MMU
> >  	help
> > -	  Allows the compaction of memory for the allocation of huge pages.
> > +          Compaction is the only memory management component to form
> > +          high order (larger physically contiguous) memory blocks
> > +          reliably. Page allocator relies on the compaction heavily and
>                        The page allo...      on compaction    
> > +          the lack of the feature can lead to unexpected OOM killer
> > +          invocation for high order memory requests. You shouldnm't
>              invocations                                    shouldn't  
> > +          disable this option unless there is really a strong reason for
>                                               really is      
> > +          it and then we are really interested to hear about that at
>                             would be    

Thanks for the review. Updated patch follows:
---
>From 177212367432e9fa5e090fafdfe3774d5793e46b Mon Sep 17 00:00:00 2001
From: Michal Hocko <mhocko@suse.com>
Date: Tue, 23 Aug 2016 10:03:37 +0200
Subject: [PATCH] mm: clarify COMPACTION Kconfig text

The current wording of the COMPACTION Kconfig help text doesn't
emphasise that disabling COMPACTION might cripple the page allocator
which relies on the compaction quite heavily for high order requests and
an unexpected OOM can happen with the lack of compaction. Make sure
we are vocal about that.

Signed-off-by: Michal Hocko <mhocko@suse.com>
---
 mm/Kconfig | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/mm/Kconfig b/mm/Kconfig
index 78a23c5c302d..be0ee11fa0d9 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -262,7 +262,14 @@ config COMPACTION
 	select MIGRATION
 	depends on MMU
 	help
-	  Allows the compaction of memory for the allocation of huge pages.
+          Compaction is the only memory management component to form
+          high order (larger physically contiguous) memory blocks
+          reliably. The page allocator relies on compaction heavily and
+          the lack of the feature can lead to unexpected OOM killer
+          invocations for high order memory requests. You shouldn't
+          disable this option unless there really is a strong reason for
+          it and then we would be really interested to hear about that at
+          linux-mm@kvack.org.
 
 #
 # support for page migration
-- 
2.8.1

-- 
Michal Hocko
SUSE Labs

Re: [PATCH] mm: clarify COMPACTION Kconfig text

[David Rientjes]

On Tue, 23 Aug 2016, Michal Hocko wrote:

> From: Michal Hocko <mhocko@suse.com>
> 
> The current wording of the COMPACTION Kconfig help text doesn't
> emphasise that disabling COMPACTION might cripple the page allocator
> which relies on the compaction quite heavily for high order requests and
> an unexpected OOM can happen with the lack of compaction. Make sure
> we are vocal about that.
> 

Since when has this been an issue?  I don't believe it has been an issue 
in the past for any archs that don't use thp.

> Signed-off-by: Michal Hocko <mhocko@suse.com>
> ---
>  mm/Kconfig | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 78a23c5c302d..0dff2f05b6d1 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -262,7 +262,14 @@ config COMPACTION
>  	select MIGRATION
>  	depends on MMU
>  	help
> -	  Allows the compaction of memory for the allocation of huge pages.
> +          Compaction is the only memory management component to form
> +          high order (larger physically contiguous) memory blocks
> +          reliably. Page allocator relies on the compaction heavily and
> +          the lack of the feature can lead to unexpected OOM killer
> +          invocation for high order memory requests. You shouldnm't
> +          disable this option unless there is really a strong reason for
> +          it and then we are really interested to hear about that at
> +          linux-mm@kvack.org.
>  
>  #
>  # support for page migration

This seems to strongly suggest that all kernels should be built with 
CONFIG_COMPACTION and its requirement, CONFIG_MIGRATION.  Migration has a 
dependency of NUMA or memory hot-remove (not all popular).  Compaction can 
defragment memory within single zone without reliance on NUMA.

This seems like a very bizarre requirement and I'm wondering where we 
regressed from this thp-only behavior.

Re: [PATCH] mm: clarify COMPACTION Kconfig text

[Michal Hocko]

On Wed 24-08-16 17:54:52, David Rientjes wrote:
> On Tue, 23 Aug 2016, Michal Hocko wrote:
> 
> > From: Michal Hocko <mhocko@suse.com>
> > 
> > The current wording of the COMPACTION Kconfig help text doesn't
> > emphasise that disabling COMPACTION might cripple the page allocator
> > which relies on the compaction quite heavily for high order requests and
> > an unexpected OOM can happen with the lack of compaction. Make sure
> > we are vocal about that.
> > 
> 
> Since when has this been an issue? 

Well, pretty much since we have dropped the lumpy reclaim. 

> I don't believe it has been an issue in the past for any archs that
> don't use thp.

Well, fragmentation is a real problem and order-0 reclaim will be never
anywhere close to reliably provide higher order pages. Well, reclaiming
a lot of memory can increase the probability of a success but that
can quite often lead to over reclaim and long stalls. There are other
sources of high order requests than THP so this is not about THP at all
IMHO.

> > Signed-off-by: Michal Hocko <mhocko@suse.com>
> > ---
> >  mm/Kconfig | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> > 
> > diff --git a/mm/Kconfig b/mm/Kconfig
> > index 78a23c5c302d..0dff2f05b6d1 100644
> > --- a/mm/Kconfig
> > +++ b/mm/Kconfig
> > @@ -262,7 +262,14 @@ config COMPACTION
> >  	select MIGRATION
> >  	depends on MMU
> >  	help
> > -	  Allows the compaction of memory for the allocation of huge pages.
> > +          Compaction is the only memory management component to form
> > +          high order (larger physically contiguous) memory blocks
> > +          reliably. Page allocator relies on the compaction heavily and
> > +          the lack of the feature can lead to unexpected OOM killer
> > +          invocation for high order memory requests. You shouldnm't
> > +          disable this option unless there is really a strong reason for
> > +          it and then we are really interested to hear about that at
> > +          linux-mm@kvack.org.
> >  
> >  #
> >  # support for page migration
> 
> This seems to strongly suggest that all kernels should be built with 
> CONFIG_COMPACTION and its requirement

Yes. Do you see any reason why the compaction should be disabled and we
should rely solely on order-0 reclaim?

> , CONFIG_MIGRATION.  Migration has a 
> dependency of NUMA or memory hot-remove (not all popular).  Compaction can 
> defragment memory within single zone without reliance on NUMA.

I am not sure I am following you here.
MIGRATION depends on (NUMA || ARCH_ENABLE_MEMORY_HOTREMOVE || COMPACTION || CMA) && MMU
 
> This seems like a very bizarre requirement and I'm wondering where we 
> regressed from this thp-only behavior.

-- 
Michal Hocko
SUSE Labs

Re: [PATCH] mm: clarify COMPACTION Kconfig text

[David Rientjes]

On Thu, 25 Aug 2016, Michal Hocko wrote:

> > I don't believe it has been an issue in the past for any archs that
> > don't use thp.
> 
> Well, fragmentation is a real problem and order-0 reclaim will be never
> anywhere close to reliably provide higher order pages. Well, reclaiming
> a lot of memory can increase the probability of a success but that
> can quite often lead to over reclaim and long stalls. There are other
> sources of high order requests than THP so this is not about THP at all
> IMHO.
> 

Would it be possible to list the high-order allocations you are concerned 
about other than thp that doesn't have fallback behavior like skbuff and 
slub allocations?  struct task_struct is an order-1 allocation and there 
may be order-1 slab bucket usage, but what is higher order or requires 
aggressive compaction to allocate?  Surely you're not suggesting that 
order-0 reclaim cannot form order-1 memory.  I am concerned about kernels 
that require a small memory footprint and cannot enable all of 
CONFIG_COMPACTION and CONFIG_MIGRATION.  Embedded devices are not a 
negligible minority of kernels.

> > , CONFIG_MIGRATION.  Migration has a 
> > dependency of NUMA or memory hot-remove (not all popular).  Compaction can 
> > defragment memory within single zone without reliance on NUMA.
> 
> I am not sure I am following you here.
> MIGRATION depends on (NUMA || ARCH_ENABLE_MEMORY_HOTREMOVE || COMPACTION || CMA) && MMU
>  

Embedded device may be UMA and not care for memory hotplug or failure 
handling and rely solely on order-0 and order-1 memory.

Re: [PATCH] mm: clarify COMPACTION Kconfig text

[Michal Hocko]

On Thu 25-08-16 15:34:54, David Rientjes wrote:
> On Thu, 25 Aug 2016, Michal Hocko wrote:
> 
> > > I don't believe it has been an issue in the past for any archs that
> > > don't use thp.
> > 
> > Well, fragmentation is a real problem and order-0 reclaim will be never
> > anywhere close to reliably provide higher order pages. Well, reclaiming
> > a lot of memory can increase the probability of a success but that
> > can quite often lead to over reclaim and long stalls. There are other
> > sources of high order requests than THP so this is not about THP at all
> > IMHO.
> > 
> 
> Would it be possible to list the high-order allocations you are concerned 
> about other than thp that doesn't have fallback behavior like skbuff and 
> slub allocations?  struct task_struct is an order-1 allocation and there 
> may be order-1 slab bucket usage, but what is higher order or requires 
> aggressive compaction to allocate?

kernel stacks (order-2 on many arches), some arches need higher order
pages for page table allocations (at least the upper level AFAIR).

> Surely you're not suggesting that order-0 reclaim cannot form order-1
> memory.

I haven't seen fragmentation that bad that order-1 would be completely
depleted so I wouldn't be all that worried about this. But order-2 can
get depleted as our last oom reports show.

> I am concerned about kernels that require a small memory footprint and
> cannot enable all of CONFIG_COMPACTION and CONFIG_MIGRATION.  Embedded
> devices are not a negligible minority of kernels.

Fair enough. And nobody discourages them from disabling the
compaction. I would expect that kernels for those machines are
configured by people who know what they are doing. They have to be
careful about disabling many other things already and carefully weight
the missing functionality vs. code size savings. I also expect that
workloads on those machines are also careful to not require large
physically contiguous memory blocks very much. Otherwise they would have
problems described by the help text.

So I am not really sure what you are objecting to. I am not making
COMPACTION on unconditionally. I just want to make sure that regular
users do not think this is just a THP thing which is not true since the
lumpy reclaim is gone. On my laptop I have more than 40 slab caches
which have pagesperslab > 2.
-- 
Michal Hocko
SUSE Labs

Re: [PATCH] mm: clarify COMPACTION Kconfig text

[Johannes Weiner]

On Tue, Aug 23, 2016 at 10:09:17AM +0200, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
> 
> The current wording of the COMPACTION Kconfig help text doesn't
> emphasise that disabling COMPACTION might cripple the page allocator
> which relies on the compaction quite heavily for high order requests and
> an unexpected OOM can happen with the lack of compaction. Make sure
> we are vocal about that.

I find it weird to even have this as a config option after we removed
lumpy reclaim. Why offer a configuration that may easily OOM on allocs
that we don't even consider "costly" to generate? There might be some
specialized setups that know they can live without the higher-order
allocations and rather have the savings in kernel size, but I'd argue
that for the vast majority of Linux setups compaction is an essential
part of our VM at this point. Seems like a candidate for EXPERT to me.

Re: [PATCH] mm: clarify COMPACTION Kconfig text

[Michal Hocko]

On Mon 29-08-16 10:10:45, Johannes Weiner wrote:
> On Tue, Aug 23, 2016 at 10:09:17AM +0200, Michal Hocko wrote:
> > From: Michal Hocko <mhocko@suse.com>
> > 
> > The current wording of the COMPACTION Kconfig help text doesn't
> > emphasise that disabling COMPACTION might cripple the page allocator
> > which relies on the compaction quite heavily for high order requests and
> > an unexpected OOM can happen with the lack of compaction. Make sure
> > we are vocal about that.
> 
> I find it weird to even have this as a config option after we removed
> lumpy reclaim. Why offer a configuration that may easily OOM on allocs
> that we don't even consider "costly" to generate? There might be some
> specialized setups that know they can live without the higher-order
> allocations and rather have the savings in kernel size, but I'd argue
> that for the vast majority of Linux setups compaction is an essential
> part of our VM at this point. Seems like a candidate for EXPERT to me.

I was thinking about making it depend on EXPERT as well but then I just
felt like making the text more verbose should be sufficient. If somebody
runs a kernel without COMPACTION and doesn't see any issues then why
should we make life harder for him. But I was thinking about a different
thing. We should warn that the compaction is disabled when the oom
killer hits for higher order. What do you think?
--- 
diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 10f686969fc4..b3c47072a206 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -406,6 +406,8 @@ static void dump_header(struct oom_control *oc, struct task_struct *p)
 	pr_warn("%s invoked oom-killer: gfp_mask=%#x(%pGg), order=%d, oom_score_adj=%hd\n",
 		current->comm, oc->gfp_mask, &oc->gfp_mask, oc->order,
 		current->signal->oom_score_adj);
+	if (!IS_ENABLED(COMPACTION) && oc->order)
+		pr_warn("COMPACTION is disabled!!!\n");
 
 	cpuset_print_current_mems_allowed();
 	dump_stack();

-- 
Michal Hocko
SUSE Labs