[PATCH v2] mm: Warn about costly page allocation

[PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

Since lumpy reclaim was introduced at 2.6.23, it helped higher
order allocation.
Recently, we removed it at 3.4 and we didn't enable compaction
forcingly[1]. The reason makes sense that compaction.o + migration.o
isn't trivial for system doesn't use higher order allocation.
But the problem is that we have to enable compaction explicitly
while lumpy reclaim enabled unconditionally.

Normally, admin doesn't know his system have used higher order
allocation and even lumpy reclaim have helped it.
Admin in embdded system have a tendency to minimise code size so that
they can disable compaction. In this case, we can see page allocation
failure we can never see in the past. It's critical on embedded side
because...

Let's think this scenario.

There is QA team in embedded company and they have tested their product.
In test scenario, they can allocate 100 high order allocation.
(they don't matter how many high order allocations in kernel are needed
during test. their concern is just only working well or fail of their
middleware/application) High order allocation will be serviced well
by natural buddy allocation without lumpy's help. So they released
the product and sold out all over the world.
Unfortunately, in real practice, sometime, 105 high order allocation was
needed rarely and fortunately, lumpy reclaim could help it so the product
doesn't have a problem until now.

If they use latest kernel, they will see the new config CONFIG_COMPACTION
which is very poor documentation, and they can't know it's replacement of
lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
that option for size optimization. Of course, QA team still test it but they
can't find the problem if they don't do test stronger than old.
It ends up release the product and sold out all over the world, again.
But in this time, we don't have both lumpy and compaction so the problem
would happen in real practice. A poor enginner from Korea have to flight
to the USA for the fix a ton of products. Otherwise, should recall products
from all over the world. Maybe he can lose a job. :(

This patch adds warning for notice. If the system try to allocate
PAGE_ALLOC_COSTLY_ORDER above page and system enters reclaim path,
it emits the warning. At least, it gives a chance to look into their
system before the relase.

Please keep in mind. It's not a good idea to depend lumpy/compaction
for regular high-order allocations. Both depends on being able to move
MIGRATE_MOVABLE allocations to satisfy the high-order allocation. If used
reregularly for high-order kernel allocations and tehy are long-lived,
the system will eventually be unable to grant these allocations, with or
without compaction or lumpy reclaim. Hatchet jobs that work around this problem
include forcing MIGRATE_RESERVE to be only used for high-order allocations
and tuning its size. It's a major hack though and is unlikely to be merged
to mainline but might suit an embedded product.

This patch avoids false positive by alloc_large_system_hash which
allocates with GFP_ATOMIC and a fallback mechanism so it can make
this warning useless.

[1] c53919ad(mm: vmscan: remove lumpy reclaim)

Signed-off-by: Minchan Kim <minchan@kernel.org>
---
Changelog

* from v1
 - add more description about warning failure of high-order allocation
 - use printk_ratelimited/pr_warn and dump stack - [Mel, Andrew]

 mm/page_alloc.c |   25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a4d3a19..710d0e90 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2276,6 +2276,29 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
 	return alloc_flags;
 }
 
+#if defined(CONFIG_DEBUG_VM) && !defined(CONFIG_COMPACTION)
+static inline void check_page_alloc_costly_order(unsigned int order, gfp_t flags)
+{
+	if (likely(order <= PAGE_ALLOC_COSTLY_ORDER))
+		return;
+
+	if (!printk_ratelimited())
+		return;
+
+	pr_warn("%s: page allocation high-order stupidity: "
+		"order:%d, mode:0x%x\n", current->comm, order, flags);
+	pr_warn("Enable compaction if high-order allocations are "
+		"very few and rare.\n");
+	pr_warn("If you need regular high-order allocation, "
+		"compaction wouldn't help it.\n");
+	dump_stack();
+}
+#else
+static inline void check_page_alloc_costly_order(unsigned int order)
+{
+}
+#endif
+
 static inline struct page *
 __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	struct zonelist *zonelist, enum zone_type high_zoneidx,
@@ -2353,6 +2376,8 @@ rebalance:
 	if (!wait)
 		goto nopage;
 
+	check_page_alloc_costly_order(order);
+
 	/* Avoid recursion of direct reclaim */
 	if (current->flags & PF_MEMALLOC)
 		goto nopage;
-- 
1.7.9.5

[PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

Since lumpy reclaim was introduced at 2.6.23, it helped higher
order allocation.
Recently, we removed it at 3.4 and we didn't enable compaction
forcingly[1]. The reason makes sense that compaction.o + migration.o
isn't trivial for system doesn't use higher order allocation.
But the problem is that we have to enable compaction explicitly
while lumpy reclaim enabled unconditionally.

Normally, admin doesn't know his system have used higher order
allocation and even lumpy reclaim have helped it.
Admin in embdded system have a tendency to minimise code size so that
they can disable compaction. In this case, we can see page allocation
failure we can never see in the past. It's critical on embedded side
because...

Let's think this scenario.

There is QA team in embedded company and they have tested their product.
In test scenario, they can allocate 100 high order allocation.
(they don't matter how many high order allocations in kernel are needed
during test. their concern is just only working well or fail of their
middleware/application) High order allocation will be serviced well
by natural buddy allocation without lumpy's help. So they released
the product and sold out all over the world.
Unfortunately, in real practice, sometime, 105 high order allocation was
needed rarely and fortunately, lumpy reclaim could help it so the product
doesn't have a problem until now.

If they use latest kernel, they will see the new config CONFIG_COMPACTION
which is very poor documentation, and they can't know it's replacement of
lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
that option for size optimization. Of course, QA team still test it but they
can't find the problem if they don't do test stronger than old.
It ends up release the product and sold out all over the world, again.
But in this time, we don't have both lumpy and compaction so the problem
would happen in real practice. A poor enginner from Korea have to flight
to the USA for the fix a ton of products. Otherwise, should recall products
from all over the world. Maybe he can lose a job. :(

This patch adds warning for notice. If the system try to allocate
PAGE_ALLOC_COSTLY_ORDER above page and system enters reclaim path,
it emits the warning. At least, it gives a chance to look into their
system before the relase.

Please keep in mind. It's not a good idea to depend lumpy/compaction
for regular high-order allocations. Both depends on being able to move
MIGRATE_MOVABLE allocations to satisfy the high-order allocation. If used
reregularly for high-order kernel allocations and tehy are long-lived,
the system will eventually be unable to grant these allocations, with or
without compaction or lumpy reclaim. Hatchet jobs that work around this problem
include forcing MIGRATE_RESERVE to be only used for high-order allocations
and tuning its size. It's a major hack though and is unlikely to be merged
to mainline but might suit an embedded product.

This patch avoids false positive by alloc_large_system_hash which
allocates with GFP_ATOMIC and a fallback mechanism so it can make
this warning useless.

[1] c53919ad(mm: vmscan: remove lumpy reclaim)

Signed-off-by: Minchan Kim <minchan@kernel.org>
---
Changelog

* from v1
 - add more description about warning failure of high-order allocation
 - use printk_ratelimited/pr_warn and dump stack - [Mel, Andrew]

 mm/page_alloc.c |   25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a4d3a19..710d0e90 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2276,6 +2276,29 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
 	return alloc_flags;
 }
 
+#if defined(CONFIG_DEBUG_VM) && !defined(CONFIG_COMPACTION)
+static inline void check_page_alloc_costly_order(unsigned int order, gfp_t flags)
+{
+	if (likely(order <= PAGE_ALLOC_COSTLY_ORDER))
+		return;
+
+	if (!printk_ratelimited())
+		return;
+
+	pr_warn("%s: page allocation high-order stupidity: "
+		"order:%d, mode:0x%x\n", current->comm, order, flags);
+	pr_warn("Enable compaction if high-order allocations are "
+		"very few and rare.\n");
+	pr_warn("If you need regular high-order allocation, "
+		"compaction wouldn't help it.\n");
+	dump_stack();
+}
+#else
+static inline void check_page_alloc_costly_order(unsigned int order)
+{
+}
+#endif
+
 static inline struct page *
 __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	struct zonelist *zonelist, enum zone_type high_zoneidx,
@@ -2353,6 +2376,8 @@ rebalance:
 	if (!wait)
 		goto nopage;
 
+	check_page_alloc_costly_order(order);
+
 	/* Avoid recursion of direct reclaim */
 	if (current->flags & PF_MEMALLOC)
 		goto nopage;
-- 
1.7.9.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

Please ignore,
It is sent by mistake. :(
Sorry for the noise.

On Tue, Jul 10, 2012 at 08:55:53AM +0900, Minchan Kim wrote:
> Since lumpy reclaim was introduced at 2.6.23, it helped higher
> order allocation.
> Recently, we removed it at 3.4 and we didn't enable compaction
> forcingly[1]. The reason makes sense that compaction.o + migration.o
> isn't trivial for system doesn't use higher order allocation.
> But the problem is that we have to enable compaction explicitly
> while lumpy reclaim enabled unconditionally.
> 
> Normally, admin doesn't know his system have used higher order
> allocation and even lumpy reclaim have helped it.
> Admin in embdded system have a tendency to minimise code size so that
> they can disable compaction. In this case, we can see page allocation
> failure we can never see in the past. It's critical on embedded side
> because...
> 
> Let's think this scenario.
> 
> There is QA team in embedded company and they have tested their product.
> In test scenario, they can allocate 100 high order allocation.
> (they don't matter how many high order allocations in kernel are needed
> during test. their concern is just only working well or fail of their
> middleware/application) High order allocation will be serviced well
> by natural buddy allocation without lumpy's help. So they released
> the product and sold out all over the world.
> Unfortunately, in real practice, sometime, 105 high order allocation was
> needed rarely and fortunately, lumpy reclaim could help it so the product
> doesn't have a problem until now.
> 
> If they use latest kernel, they will see the new config CONFIG_COMPACTION
> which is very poor documentation, and they can't know it's replacement of
> lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
> that option for size optimization. Of course, QA team still test it but they
> can't find the problem if they don't do test stronger than old.
> It ends up release the product and sold out all over the world, again.
> But in this time, we don't have both lumpy and compaction so the problem
> would happen in real practice. A poor enginner from Korea have to flight
> to the USA for the fix a ton of products. Otherwise, should recall products
> from all over the world. Maybe he can lose a job. :(
> 
> This patch adds warning for notice. If the system try to allocate
> PAGE_ALLOC_COSTLY_ORDER above page and system enters reclaim path,
> it emits the warning. At least, it gives a chance to look into their
> system before the relase.
> 
> Please keep in mind. It's not a good idea to depend lumpy/compaction
> for regular high-order allocations. Both depends on being able to move
> MIGRATE_MOVABLE allocations to satisfy the high-order allocation. If used
> reregularly for high-order kernel allocations and tehy are long-lived,
> the system will eventually be unable to grant these allocations, with or
> without compaction or lumpy reclaim. Hatchet jobs that work around this problem
> include forcing MIGRATE_RESERVE to be only used for high-order allocations
> and tuning its size. It's a major hack though and is unlikely to be merged
> to mainline but might suit an embedded product.
> 
> This patch avoids false positive by alloc_large_system_hash which
> allocates with GFP_ATOMIC and a fallback mechanism so it can make
> this warning useless.
> 
> [1] c53919ad(mm: vmscan: remove lumpy reclaim)
> 
> Signed-off-by: Minchan Kim <minchan@kernel.org>
> ---
> Changelog
> 
> * from v1
>  - add more description about warning failure of high-order allocation
>  - use printk_ratelimited/pr_warn and dump stack - [Mel, Andrew]
> 
>  mm/page_alloc.c |   25 +++++++++++++++++++++++++
>  1 file changed, 25 insertions(+)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index a4d3a19..710d0e90 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -2276,6 +2276,29 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
>  	return alloc_flags;
>  }
>  
> +#if defined(CONFIG_DEBUG_VM) && !defined(CONFIG_COMPACTION)
> +static inline void check_page_alloc_costly_order(unsigned int order, gfp_t flags)
> +{
> +	if (likely(order <= PAGE_ALLOC_COSTLY_ORDER))
> +		return;
> +
> +	if (!printk_ratelimited())
> +		return;
> +
> +	pr_warn("%s: page allocation high-order stupidity: "
> +		"order:%d, mode:0x%x\n", current->comm, order, flags);
> +	pr_warn("Enable compaction if high-order allocations are "
> +		"very few and rare.\n");
> +	pr_warn("If you need regular high-order allocation, "
> +		"compaction wouldn't help it.\n");
> +	dump_stack();
> +}
> +#else
> +static inline void check_page_alloc_costly_order(unsigned int order)
> +{
> +}
> +#endif
> +
>  static inline struct page *
>  __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	struct zonelist *zonelist, enum zone_type high_zoneidx,
> @@ -2353,6 +2376,8 @@ rebalance:
>  	if (!wait)
>  		goto nopage;
>  
> +	check_page_alloc_costly_order(order);
> +
>  	/* Avoid recursion of direct reclaim */
>  	if (current->flags & PF_MEMALLOC)
>  		goto nopage;
> -- 
> 1.7.9.5
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

Please ignore,
It is sent by mistake. :(
Sorry for the noise.

On Tue, Jul 10, 2012 at 08:55:53AM +0900, Minchan Kim wrote:
> Since lumpy reclaim was introduced at 2.6.23, it helped higher
> order allocation.
> Recently, we removed it at 3.4 and we didn't enable compaction
> forcingly[1]. The reason makes sense that compaction.o + migration.o
> isn't trivial for system doesn't use higher order allocation.
> But the problem is that we have to enable compaction explicitly
> while lumpy reclaim enabled unconditionally.
> 
> Normally, admin doesn't know his system have used higher order
> allocation and even lumpy reclaim have helped it.
> Admin in embdded system have a tendency to minimise code size so that
> they can disable compaction. In this case, we can see page allocation
> failure we can never see in the past. It's critical on embedded side
> because...
> 
> Let's think this scenario.
> 
> There is QA team in embedded company and they have tested their product.
> In test scenario, they can allocate 100 high order allocation.
> (they don't matter how many high order allocations in kernel are needed
> during test. their concern is just only working well or fail of their
> middleware/application) High order allocation will be serviced well
> by natural buddy allocation without lumpy's help. So they released
> the product and sold out all over the world.
> Unfortunately, in real practice, sometime, 105 high order allocation was
> needed rarely and fortunately, lumpy reclaim could help it so the product
> doesn't have a problem until now.
> 
> If they use latest kernel, they will see the new config CONFIG_COMPACTION
> which is very poor documentation, and they can't know it's replacement of
> lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
> that option for size optimization. Of course, QA team still test it but they
> can't find the problem if they don't do test stronger than old.
> It ends up release the product and sold out all over the world, again.
> But in this time, we don't have both lumpy and compaction so the problem
> would happen in real practice. A poor enginner from Korea have to flight
> to the USA for the fix a ton of products. Otherwise, should recall products
> from all over the world. Maybe he can lose a job. :(
> 
> This patch adds warning for notice. If the system try to allocate
> PAGE_ALLOC_COSTLY_ORDER above page and system enters reclaim path,
> it emits the warning. At least, it gives a chance to look into their
> system before the relase.
> 
> Please keep in mind. It's not a good idea to depend lumpy/compaction
> for regular high-order allocations. Both depends on being able to move
> MIGRATE_MOVABLE allocations to satisfy the high-order allocation. If used
> reregularly for high-order kernel allocations and tehy are long-lived,
> the system will eventually be unable to grant these allocations, with or
> without compaction or lumpy reclaim. Hatchet jobs that work around this problem
> include forcing MIGRATE_RESERVE to be only used for high-order allocations
> and tuning its size. It's a major hack though and is unlikely to be merged
> to mainline but might suit an embedded product.
> 
> This patch avoids false positive by alloc_large_system_hash which
> allocates with GFP_ATOMIC and a fallback mechanism so it can make
> this warning useless.
> 
> [1] c53919ad(mm: vmscan: remove lumpy reclaim)
> 
> Signed-off-by: Minchan Kim <minchan@kernel.org>
> ---
> Changelog
> 
> * from v1
>  - add more description about warning failure of high-order allocation
>  - use printk_ratelimited/pr_warn and dump stack - [Mel, Andrew]
> 
>  mm/page_alloc.c |   25 +++++++++++++++++++++++++
>  1 file changed, 25 insertions(+)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index a4d3a19..710d0e90 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -2276,6 +2276,29 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
>  	return alloc_flags;
>  }
>  
> +#if defined(CONFIG_DEBUG_VM) && !defined(CONFIG_COMPACTION)
> +static inline void check_page_alloc_costly_order(unsigned int order, gfp_t flags)
> +{
> +	if (likely(order <= PAGE_ALLOC_COSTLY_ORDER))
> +		return;
> +
> +	if (!printk_ratelimited())
> +		return;
> +
> +	pr_warn("%s: page allocation high-order stupidity: "
> +		"order:%d, mode:0x%x\n", current->comm, order, flags);
> +	pr_warn("Enable compaction if high-order allocations are "
> +		"very few and rare.\n");
> +	pr_warn("If you need regular high-order allocation, "
> +		"compaction wouldn't help it.\n");
> +	dump_stack();
> +}
> +#else
> +static inline void check_page_alloc_costly_order(unsigned int order)
> +{
> +}
> +#endif
> +
>  static inline struct page *
>  __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	struct zonelist *zonelist, enum zone_type high_zoneidx,
> @@ -2353,6 +2376,8 @@ rebalance:
>  	if (!wait)
>  		goto nopage;
>  
> +	check_page_alloc_costly_order(order);
> +
>  	/* Avoid recursion of direct reclaim */
>  	if (current->flags & PF_MEMALLOC)
>  		goto nopage;
> -- 
> 1.7.9.5
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Andrew Morton]

On Tue, 10 Jul 2012 08:55:53 +0900
Minchan Kim <minchan@kernel.org> wrote:

> ...
>
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -2276,6 +2276,29 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
>  	return alloc_flags;
>  }
>  
> +#if defined(CONFIG_DEBUG_VM) && !defined(CONFIG_COMPACTION)
> +static inline void check_page_alloc_costly_order(unsigned int order, gfp_t flags)
> +{
> +	if (likely(order <= PAGE_ALLOC_COSTLY_ORDER))
> +		return;
> +
> +	if (!printk_ratelimited())
> +		return;
> +
> +	pr_warn("%s: page allocation high-order stupidity: "
> +		"order:%d, mode:0x%x\n", current->comm, order, flags);
> +	pr_warn("Enable compaction if high-order allocations are "
> +		"very few and rare.\n");
> +	pr_warn("If you need regular high-order allocation, "
> +		"compaction wouldn't help it.\n");
> +	dump_stack();
> +}
> +#else
> +static inline void check_page_alloc_costly_order(unsigned int order)
> +{
> +}
> +#endif

Let's remember that plain old "inline" is ignored by the compiler.  If
we really really want to inline something then we should use
__always_inline.

And inlining this function would be a bad thing to do - it causes the
outer function to have an increased cache footprint.  A good way to
optimise this function is probably to move the unlikely stuff
out-of-line:

	if (unlikely(order > PAGE_ALLOC_COSTLY_ORDER))
		check_page_alloc_costly_order(...);

or

static noinline void __check_page_alloc_costly_order(...)
{
}

static __always_inline void check_page_alloc_costly_order(...)
{
	if (unlikely(order > PAGE_ALLOC_COSTLY_ORDER))
		__check_page_alloc_costly_order(...);
}
	

Also, the displayed messages don't seem very, umm, professional.  Who
was stupid - us or the kernel-configurer?  And "Enable
CONFIG_COMPACTION" would be more specific (and hence helpful) than
"Enable compaction").

And how on earth is the user, or the person who is configuring kernels
for customers to determine whether the kernel will be frequently
performing higher-order allocations?


So I dunno, this all looks like we have a kernel problem and we're
throwing our problem onto hopelessly ill-equipped users of that kernel?

Re: [PATCH v2] mm: Warn about costly page allocation

[Andrew Morton]

On Tue, 10 Jul 2012 08:55:53 +0900
Minchan Kim <minchan@kernel.org> wrote:

> ...
>
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -2276,6 +2276,29 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
>  	return alloc_flags;
>  }
>  
> +#if defined(CONFIG_DEBUG_VM) && !defined(CONFIG_COMPACTION)
> +static inline void check_page_alloc_costly_order(unsigned int order, gfp_t flags)
> +{
> +	if (likely(order <= PAGE_ALLOC_COSTLY_ORDER))
> +		return;
> +
> +	if (!printk_ratelimited())
> +		return;
> +
> +	pr_warn("%s: page allocation high-order stupidity: "
> +		"order:%d, mode:0x%x\n", current->comm, order, flags);
> +	pr_warn("Enable compaction if high-order allocations are "
> +		"very few and rare.\n");
> +	pr_warn("If you need regular high-order allocation, "
> +		"compaction wouldn't help it.\n");
> +	dump_stack();
> +}
> +#else
> +static inline void check_page_alloc_costly_order(unsigned int order)
> +{
> +}
> +#endif

Let's remember that plain old "inline" is ignored by the compiler.  If
we really really want to inline something then we should use
__always_inline.

And inlining this function would be a bad thing to do - it causes the
outer function to have an increased cache footprint.  A good way to
optimise this function is probably to move the unlikely stuff
out-of-line:

	if (unlikely(order > PAGE_ALLOC_COSTLY_ORDER))
		check_page_alloc_costly_order(...);

or

static noinline void __check_page_alloc_costly_order(...)
{
}

static __always_inline void check_page_alloc_costly_order(...)
{
	if (unlikely(order > PAGE_ALLOC_COSTLY_ORDER))
		__check_page_alloc_costly_order(...);
}
	

Also, the displayed messages don't seem very, umm, professional.  Who
was stupid - us or the kernel-configurer?  And "Enable
CONFIG_COMPACTION" would be more specific (and hence helpful) than
"Enable compaction").

And how on earth is the user, or the person who is configuring kernels
for customers to determine whether the kernel will be frequently
performing higher-order allocations?


So I dunno, this all looks like we have a kernel problem and we're
throwing our problem onto hopelessly ill-equipped users of that kernel?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

Hi Andrew,

On Mon, Jul 09, 2012 at 05:08:56PM -0700, Andrew Morton wrote:
> On Tue, 10 Jul 2012 08:55:53 +0900
> Minchan Kim <minchan@kernel.org> wrote:
> 
> > ...
> >
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -2276,6 +2276,29 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
> >  	return alloc_flags;
> >  }
> >  
> > +#if defined(CONFIG_DEBUG_VM) && !defined(CONFIG_COMPACTION)
> > +static inline void check_page_alloc_costly_order(unsigned int order, gfp_t flags)
> > +{
> > +	if (likely(order <= PAGE_ALLOC_COSTLY_ORDER))
> > +		return;
> > +
> > +	if (!printk_ratelimited())
> > +		return;
> > +
> > +	pr_warn("%s: page allocation high-order stupidity: "
> > +		"order:%d, mode:0x%x\n", current->comm, order, flags);
> > +	pr_warn("Enable compaction if high-order allocations are "
> > +		"very few and rare.\n");
> > +	pr_warn("If you need regular high-order allocation, "
> > +		"compaction wouldn't help it.\n");
> > +	dump_stack();
> > +}
> > +#else
> > +static inline void check_page_alloc_costly_order(unsigned int order)
> > +{
> > +}
> > +#endif
> 
> Let's remember that plain old "inline" is ignored by the compiler.  If
> we really really want to inline something then we should use
> __always_inline.

I didn't know about that. Thanks for the pointing out.

> 
> And inlining this function would be a bad thing to do - it causes the
> outer function to have an increased cache footprint.  A good way to
> optimise this function is probably to move the unlikely stuff
> out-of-line:

Okay. will do.

> 
> 	if (unlikely(order > PAGE_ALLOC_COSTLY_ORDER))
> 		check_page_alloc_costly_order(...);
> 
> or
> 
> static noinline void __check_page_alloc_costly_order(...)
> {
> }
> 
> static __always_inline void check_page_alloc_costly_order(...)
> {
> 	if (unlikely(order > PAGE_ALLOC_COSTLY_ORDER))
> 		__check_page_alloc_costly_order(...);
> }
> 	
> 
> Also, the displayed messages don't seem very, umm, professional.  Who
> was stupid - us or the kernel-configurer?  And "Enable
> CONFIG_COMPACTION" would be more specific (and hence helpful) than
> "Enable compaction").

Okay.

> 
> And how on earth is the user, or the person who is configuring kernels
> for customers to determine whether the kernel will be frequently
> performing higher-order allocations?
> 
> 
> So I dunno, this all looks like we have a kernel problem and we're
> throwing our problem onto hopelessly ill-equipped users of that kernel?

As you know, this patch isn't for solving regular high-order allocations.
As I wrote down, The problem is that we removed lumpy reclaim without any
notification for user who might have used it implicitly.
If such user disable compaction which is a replacement of lumpy reclaim,
their system might be broken in real practice while test is passing.
So, the goal is that let them know it in advance so that I expect they can
test it stronger than old.

Although they see the page allocation failure with compaction, it would
be very helpful reports. It means we need to make compaction more
aggressive about reclaiming pages.

> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

Hi Andrew,

On Mon, Jul 09, 2012 at 05:08:56PM -0700, Andrew Morton wrote:
> On Tue, 10 Jul 2012 08:55:53 +0900
> Minchan Kim <minchan@kernel.org> wrote:
> 
> > ...
> >
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -2276,6 +2276,29 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
> >  	return alloc_flags;
> >  }
> >  
> > +#if defined(CONFIG_DEBUG_VM) && !defined(CONFIG_COMPACTION)
> > +static inline void check_page_alloc_costly_order(unsigned int order, gfp_t flags)
> > +{
> > +	if (likely(order <= PAGE_ALLOC_COSTLY_ORDER))
> > +		return;
> > +
> > +	if (!printk_ratelimited())
> > +		return;
> > +
> > +	pr_warn("%s: page allocation high-order stupidity: "
> > +		"order:%d, mode:0x%x\n", current->comm, order, flags);
> > +	pr_warn("Enable compaction if high-order allocations are "
> > +		"very few and rare.\n");
> > +	pr_warn("If you need regular high-order allocation, "
> > +		"compaction wouldn't help it.\n");
> > +	dump_stack();
> > +}
> > +#else
> > +static inline void check_page_alloc_costly_order(unsigned int order)
> > +{
> > +}
> > +#endif
> 
> Let's remember that plain old "inline" is ignored by the compiler.  If
> we really really want to inline something then we should use
> __always_inline.

I didn't know about that. Thanks for the pointing out.

> 
> And inlining this function would be a bad thing to do - it causes the
> outer function to have an increased cache footprint.  A good way to
> optimise this function is probably to move the unlikely stuff
> out-of-line:

Okay. will do.

> 
> 	if (unlikely(order > PAGE_ALLOC_COSTLY_ORDER))
> 		check_page_alloc_costly_order(...);
> 
> or
> 
> static noinline void __check_page_alloc_costly_order(...)
> {
> }
> 
> static __always_inline void check_page_alloc_costly_order(...)
> {
> 	if (unlikely(order > PAGE_ALLOC_COSTLY_ORDER))
> 		__check_page_alloc_costly_order(...);
> }
> 	
> 
> Also, the displayed messages don't seem very, umm, professional.  Who
> was stupid - us or the kernel-configurer?  And "Enable
> CONFIG_COMPACTION" would be more specific (and hence helpful) than
> "Enable compaction").

Okay.

> 
> And how on earth is the user, or the person who is configuring kernels
> for customers to determine whether the kernel will be frequently
> performing higher-order allocations?
> 
> 
> So I dunno, this all looks like we have a kernel problem and we're
> throwing our problem onto hopelessly ill-equipped users of that kernel?

As you know, this patch isn't for solving regular high-order allocations.
As I wrote down, The problem is that we removed lumpy reclaim without any
notification for user who might have used it implicitly.
If such user disable compaction which is a replacement of lumpy reclaim,
their system might be broken in real practice while test is passing.
So, the goal is that let them know it in advance so that I expect they can
test it stronger than old.

Although they see the page allocation failure with compaction, it would
be very helpful reports. It means we need to make compaction more
aggressive about reclaiming pages.

> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[David Rientjes]

On Tue, 10 Jul 2012, Minchan Kim wrote:

> > So I dunno, this all looks like we have a kernel problem and we're
> > throwing our problem onto hopelessly ill-equipped users of that kernel?
> 
> As you know, this patch isn't for solving regular high-order allocations.
> As I wrote down, The problem is that we removed lumpy reclaim without any
> notification for user who might have used it implicitly.

And so now they're running with CONFIG_DEBUG_VM to try to figure out why 
they have seen a regression, which is required for your patch to have an 
effect?

> If such user disable compaction which is a replacement of lumpy reclaim,
> their system might be broken in real practice while test is passing.
> So, the goal is that let them know it in advance so that I expect they can
> test it stronger than old.
> 

So what are they supposed to do?  Enable CONFIG_COMPACTION as soon as they 
see the warning?  When they have seen the warning a specific number of 
times?  How much is "very few" high-order allocations over what time 
period?  This is what anybody seeing these messages for the first time is 
going to ask.

> Although they see the page allocation failure with compaction, it would
> be very helpful reports. It means we need to make compaction more
> aggressive about reclaiming pages.
> 

If CONFIG_COMPACTION is disabled, then how will making compaction more 
aggressive about reclaiming pages help?

Should we consider enabling CONFIG_COMPACTION in defconfig?  If not, would 
it be possible with a different extfrag_threshold (and more aggressive 
when things like THP are enabled)?

Re: [PATCH v2] mm: Warn about costly page allocation

[David Rientjes]

On Tue, 10 Jul 2012, Minchan Kim wrote:

> > So I dunno, this all looks like we have a kernel problem and we're
> > throwing our problem onto hopelessly ill-equipped users of that kernel?
> 
> As you know, this patch isn't for solving regular high-order allocations.
> As I wrote down, The problem is that we removed lumpy reclaim without any
> notification for user who might have used it implicitly.

And so now they're running with CONFIG_DEBUG_VM to try to figure out why 
they have seen a regression, which is required for your patch to have an 
effect?

> If such user disable compaction which is a replacement of lumpy reclaim,
> their system might be broken in real practice while test is passing.
> So, the goal is that let them know it in advance so that I expect they can
> test it stronger than old.
> 

So what are they supposed to do?  Enable CONFIG_COMPACTION as soon as they 
see the warning?  When they have seen the warning a specific number of 
times?  How much is "very few" high-order allocations over what time 
period?  This is what anybody seeing these messages for the first time is 
going to ask.

> Although they see the page allocation failure with compaction, it would
> be very helpful reports. It means we need to make compaction more
> aggressive about reclaiming pages.
> 

If CONFIG_COMPACTION is disabled, then how will making compaction more 
aggressive about reclaiming pages help?

Should we consider enabling CONFIG_COMPACTION in defconfig?  If not, would 
it be possible with a different extfrag_threshold (and more aggressive 
when things like THP are enabled)?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

Hi David,

On Tue, Jul 10, 2012 at 06:02:06PM -0700, David Rientjes wrote:
> On Tue, 10 Jul 2012, Minchan Kim wrote:
> 
> > > So I dunno, this all looks like we have a kernel problem and we're
> > > throwing our problem onto hopelessly ill-equipped users of that kernel?
> > 
> > As you know, this patch isn't for solving regular high-order allocations.
> > As I wrote down, The problem is that we removed lumpy reclaim without any
> > notification for user who might have used it implicitly.
> 
> And so now they're running with CONFIG_DEBUG_VM to try to figure out why 
> they have seen a regression, which is required for your patch to have an 
> effect?

Enabling that warning if some debug option is enabled was Mel's comment by
private discussion and I thought new debug option is overkill for it
so I added in CONFIG_DEBUG_VM.

> 
> > If such user disable compaction which is a replacement of lumpy reclaim,
> > their system might be broken in real practice while test is passing.
> > So, the goal is that let them know it in advance so that I expect they can
> > test it stronger than old.
> > 
> 
> So what are they supposed to do?  Enable CONFIG_COMPACTION as soon as they 
> see the warning?  When they have seen the warning a specific number of 
> times?  How much is "very few" high-order allocations over what time 
> period?  This is what anybody seeing these messages for the first time is 
> going to ask.

I admit that is a bit vague but we don't have a way to specify it because
it depends on workload(ex, how many we have movable pages and swap system at
the moment) so it's the best I can do for achieve the goal
which is alert for careful investigating/tuning the system
since lumpy reclaim is gone.

If you have a better idea, please suggest me.

> 
> > Although they see the page allocation failure with compaction, it would
> > be very helpful reports. It means we need to make compaction more
> > aggressive about reclaiming pages.
> > 
> 
> If CONFIG_COMPACTION is disabled, then how will making compaction more 
> aggressive about reclaiming pages help?

I mentioned "Although they see the page allocation failture with *compaction*"

> 
> Should we consider enabling CONFIG_COMPACTION in defconfig?  If not, would 

I hope so but Mel didn't like it because some users want to have a smallest
kernel if they don't care of high-order allocation.


> it be possible with a different extfrag_threshold (and more aggressive 
> when things like THP are enabled)?

Anyway, we should enable compaction for it although the system doesn't 
care about high-order allocation and it ends up make bloting kernel unnecessary.

I tend to agree Andrew and your concern but I don't have a good idea but
alert vague warning message. Anyway, we need *alert* this fact which removed
lumpy reclaim for being able to disabling CONFIG_COMPACTION.
Then, what do you think about this?

At least, it would give a chance to think over about disabing
this options more carefully so then it depends on their choices which are
more hard testing, ask it to the community or just turn off.

diff --git a/mm/Kconfig b/mm/Kconfig
index 16f6b42..099e681 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -195,6 +195,10 @@ config COMPACTION
        depends on MMU
        help
          Allows the compaction of memory for the allocation of huge pages.
+         Notice. We replaced lumpy reclaim which was scheme of helping
+         high-order allocations with compaction at 3.4 so if you don't
+         care about high-order allocations but want the smallest kernel,
+         you could select "N", otherwise, "y" is preferred.
 
 #
 # support for page migration


> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

Hi David,

On Tue, Jul 10, 2012 at 06:02:06PM -0700, David Rientjes wrote:
> On Tue, 10 Jul 2012, Minchan Kim wrote:
> 
> > > So I dunno, this all looks like we have a kernel problem and we're
> > > throwing our problem onto hopelessly ill-equipped users of that kernel?
> > 
> > As you know, this patch isn't for solving regular high-order allocations.
> > As I wrote down, The problem is that we removed lumpy reclaim without any
> > notification for user who might have used it implicitly.
> 
> And so now they're running with CONFIG_DEBUG_VM to try to figure out why 
> they have seen a regression, which is required for your patch to have an 
> effect?

Enabling that warning if some debug option is enabled was Mel's comment by
private discussion and I thought new debug option is overkill for it
so I added in CONFIG_DEBUG_VM.

> 
> > If such user disable compaction which is a replacement of lumpy reclaim,
> > their system might be broken in real practice while test is passing.
> > So, the goal is that let them know it in advance so that I expect they can
> > test it stronger than old.
> > 
> 
> So what are they supposed to do?  Enable CONFIG_COMPACTION as soon as they 
> see the warning?  When they have seen the warning a specific number of 
> times?  How much is "very few" high-order allocations over what time 
> period?  This is what anybody seeing these messages for the first time is 
> going to ask.

I admit that is a bit vague but we don't have a way to specify it because
it depends on workload(ex, how many we have movable pages and swap system at
the moment) so it's the best I can do for achieve the goal
which is alert for careful investigating/tuning the system
since lumpy reclaim is gone.

If you have a better idea, please suggest me.

> 
> > Although they see the page allocation failure with compaction, it would
> > be very helpful reports. It means we need to make compaction more
> > aggressive about reclaiming pages.
> > 
> 
> If CONFIG_COMPACTION is disabled, then how will making compaction more 
> aggressive about reclaiming pages help?

I mentioned "Although they see the page allocation failture with *compaction*"

> 
> Should we consider enabling CONFIG_COMPACTION in defconfig?  If not, would 

I hope so but Mel didn't like it because some users want to have a smallest
kernel if they don't care of high-order allocation.


> it be possible with a different extfrag_threshold (and more aggressive 
> when things like THP are enabled)?

Anyway, we should enable compaction for it although the system doesn't 
care about high-order allocation and it ends up make bloting kernel unnecessary.

I tend to agree Andrew and your concern but I don't have a good idea but
alert vague warning message. Anyway, we need *alert* this fact which removed
lumpy reclaim for being able to disabling CONFIG_COMPACTION.
Then, what do you think about this?

At least, it would give a chance to think over about disabing
this options more carefully so then it depends on their choices which are
more hard testing, ask it to the community or just turn off.

diff --git a/mm/Kconfig b/mm/Kconfig
index 16f6b42..099e681 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -195,6 +195,10 @@ config COMPACTION
        depends on MMU
        help
          Allows the compaction of memory for the allocation of huge pages.
+         Notice. We replaced lumpy reclaim which was scheme of helping
+         high-order allocations with compaction at 3.4 so if you don't
+         care about high-order allocations but want the smallest kernel,
+         you could select "N", otherwise, "y" is preferred.
 
 #
 # support for page migration


> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Cong Wang]

On Wed, 11 Jul 2012 at 02:23 GMT, Minchan Kim <minchan@kernel.org> wrote:
> On Tue, Jul 10, 2012 at 06:02:06PM -0700, David Rientjes wrote:
>> Should we consider enabling CONFIG_COMPACTION in defconfig?  If not, would 
>
> I hope so but Mel didn't like it because some users want to have a smallest
> kernel if they don't care of high-order allocation.
>

If they want a smallest kernel, they probably don't use defconfig,
they should custom their own config.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[David Rientjes]

On Wed, 11 Jul 2012, Minchan Kim wrote:

> > Should we consider enabling CONFIG_COMPACTION in defconfig?  If not, would 
> 
> I hope so but Mel didn't like it because some users want to have a smallest
> kernel if they don't care of high-order allocation.
> 

CONFIG_COMPACTION adds 0.1% to my kernel image using x86_64 defconfig, 
that's the only reason we don't enable it by default?

> > it be possible with a different extfrag_threshold (and more aggressive 
> > when things like THP are enabled)?
> 
> Anyway, we should enable compaction for it although the system doesn't 
> care about high-order allocation and it ends up make bloting kernel unnecessary.
> 

The problem with this approach (and the appended patch) is that we can't 
define a system that "doesn't care about high-order allocations."  Even if 
you discount thp, an admin has no way of knowing how many high-order 
allocations his or her kernel will be doing and it will change between 
kernel versions.  Almost 50% of slab caches on my desktop machine running 
with slub have a default order greater than 0.

So I don't believe that adding this warning will be helpful and will 
simply lead to confusion.

> I tend to agree Andrew and your concern but I don't have a good idea but
> alert vague warning message. Anyway, we need *alert* this fact which removed
> lumpy reclaim for being able to disabling CONFIG_COMPACTION.

Can we ignore the fact that lumpy reclaim was removed and look at 
individual issues as they arise and address them by fixing the VM or by 
making a case for enabling CONFIG_COMPACTION by default?

Re: [PATCH v2] mm: Warn about costly page allocation

[David Rientjes]

On Wed, 11 Jul 2012, Minchan Kim wrote:

> > Should we consider enabling CONFIG_COMPACTION in defconfig?  If not, would 
> 
> I hope so but Mel didn't like it because some users want to have a smallest
> kernel if they don't care of high-order allocation.
> 

CONFIG_COMPACTION adds 0.1% to my kernel image using x86_64 defconfig, 
that's the only reason we don't enable it by default?

> > it be possible with a different extfrag_threshold (and more aggressive 
> > when things like THP are enabled)?
> 
> Anyway, we should enable compaction for it although the system doesn't 
> care about high-order allocation and it ends up make bloting kernel unnecessary.
> 

The problem with this approach (and the appended patch) is that we can't 
define a system that "doesn't care about high-order allocations."  Even if 
you discount thp, an admin has no way of knowing how many high-order 
allocations his or her kernel will be doing and it will change between 
kernel versions.  Almost 50% of slab caches on my desktop machine running 
with slub have a default order greater than 0.

So I don't believe that adding this warning will be helpful and will 
simply lead to confusion.

> I tend to agree Andrew and your concern but I don't have a good idea but
> alert vague warning message. Anyway, we need *alert* this fact which removed
> lumpy reclaim for being able to disabling CONFIG_COMPACTION.

Can we ignore the fact that lumpy reclaim was removed and look at 
individual issues as they arise and address them by fixing the VM or by 
making a case for enabling CONFIG_COMPACTION by default?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

On 07/11/2012 02:33 PM, David Rientjes wrote:
> On Wed, 11 Jul 2012, Minchan Kim wrote:
> 
>>> Should we consider enabling CONFIG_COMPACTION in defconfig?  If not, would 
>>
>> I hope so but Mel didn't like it because some users want to have a smallest
>> kernel if they don't care of high-order allocation.
>>
> 
> CONFIG_COMPACTION adds 0.1% to my kernel image using x86_64 defconfig, 

barrios@bbox:~/linux-next$ size mm/compaction.o mm/migrate.o
   text	   data	    bss	    dec	    hex	filename
   8550	   1114	      4	   9668	   25c4	mm/compaction.o
  10891	    520	      0	  11411	   2c93	mm/migrate.o

It couldn't be a trivial on small system.

> that's the only reason we don't enable it by default?

AFAIK, that's all. Mel. Do you think others?

> 
>>> it be possible with a different extfrag_threshold (and more aggressive 
>>> when things like THP are enabled)?
>>
>> Anyway, we should enable compaction for it although the system doesn't 
>> care about high-order allocation and it ends up make bloting kernel unnecessary.
>>
> 
> The problem with this approach (and the appended patch) is that we can't 
> define a system that "doesn't care about high-order allocations."  Even if 
> you discount thp, an admin has no way of knowing how many high-order 
> allocations his or her kernel will be doing and it will change between 

Of course.

> kernel versions.  Almost 50% of slab caches on my desktop machine running 
> with slub have a default order greater than 0.
> 
> So I don't believe that adding this warning will be helpful and will 
> simply lead to confusion.
> 
>> I tend to agree Andrew and your concern but I don't have a good idea but
>> alert vague warning message. Anyway, we need *alert* this fact which removed
>> lumpy reclaim for being able to disabling CONFIG_COMPACTION.
> 
> Can we ignore the fact that lumpy reclaim was removed and look at 
> individual issues as they arise and address them by fixing the VM or by 
> making a case for enabling CONFIG_COMPACTION by default?

I agree it's an ideal but the problem is that it's too late.
Once product is released, we have to recall all products in the worst case.
The fact is that lumpy have helped high order allocation implicitly but we removed it
without any notification or information. It's a sort of regression and we can't say
them "Please report us if it happens". It's irresponsible, too.
IMHO, at least, what we can do is to warn about it before it's too late.


> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 


-- 
Kind regards,
Minchan Kim

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

On 07/11/2012 02:33 PM, David Rientjes wrote:
> On Wed, 11 Jul 2012, Minchan Kim wrote:
> 
>>> Should we consider enabling CONFIG_COMPACTION in defconfig?  If not, would 
>>
>> I hope so but Mel didn't like it because some users want to have a smallest
>> kernel if they don't care of high-order allocation.
>>
> 
> CONFIG_COMPACTION adds 0.1% to my kernel image using x86_64 defconfig, 

barrios@bbox:~/linux-next$ size mm/compaction.o mm/migrate.o
   text	   data	    bss	    dec	    hex	filename
   8550	   1114	      4	   9668	   25c4	mm/compaction.o
  10891	    520	      0	  11411	   2c93	mm/migrate.o

It couldn't be a trivial on small system.

> that's the only reason we don't enable it by default?

AFAIK, that's all. Mel. Do you think others?

> 
>>> it be possible with a different extfrag_threshold (and more aggressive 
>>> when things like THP are enabled)?
>>
>> Anyway, we should enable compaction for it although the system doesn't 
>> care about high-order allocation and it ends up make bloting kernel unnecessary.
>>
> 
> The problem with this approach (and the appended patch) is that we can't 
> define a system that "doesn't care about high-order allocations."  Even if 
> you discount thp, an admin has no way of knowing how many high-order 
> allocations his or her kernel will be doing and it will change between 

Of course.

> kernel versions.  Almost 50% of slab caches on my desktop machine running 
> with slub have a default order greater than 0.
> 
> So I don't believe that adding this warning will be helpful and will 
> simply lead to confusion.
> 
>> I tend to agree Andrew and your concern but I don't have a good idea but
>> alert vague warning message. Anyway, we need *alert* this fact which removed
>> lumpy reclaim for being able to disabling CONFIG_COMPACTION.
> 
> Can we ignore the fact that lumpy reclaim was removed and look at 
> individual issues as they arise and address them by fixing the VM or by 
> making a case for enabling CONFIG_COMPACTION by default?

I agree it's an ideal but the problem is that it's too late.
Once product is released, we have to recall all products in the worst case.
The fact is that lumpy have helped high order allocation implicitly but we removed it
without any notification or information. It's a sort of regression and we can't say
them "Please report us if it happens". It's irresponsible, too.
IMHO, at least, what we can do is to warn about it before it's too late.


> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 


-- 
Kind regards,
Minchan Kim


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[David Rientjes]

On Wed, 11 Jul 2012, Minchan Kim wrote:

> I agree it's an ideal but the problem is that it's too late.
> Once product is released, we have to recall all products in the worst case.
> The fact is that lumpy have helped high order allocation implicitly but we removed it
> without any notification or information. It's a sort of regression and we can't say
> them "Please report us if it happens". It's irresponsible, too.
> IMHO, at least, what we can do is to warn about it before it's too late.
> 

High order allocations that fail should still display a warning message 
when __GFP_NOWARN is not set, so I don't see what this additional warning 
adds.  I don't think it's responsible to ask admins to know what lumpy 
reclaim is, what memory compaction is, or when a system tends to have more 
high order allocations when memory compaction would be helpful.

What we can do, though, is address bug reports as they are reported when 
high order allocations fail and previous kernels are successful.  I 
haven't seen any lately.

Re: [PATCH v2] mm: Warn about costly page allocation

[David Rientjes]

On Wed, 11 Jul 2012, Minchan Kim wrote:

> I agree it's an ideal but the problem is that it's too late.
> Once product is released, we have to recall all products in the worst case.
> The fact is that lumpy have helped high order allocation implicitly but we removed it
> without any notification or information. It's a sort of regression and we can't say
> them "Please report us if it happens". It's irresponsible, too.
> IMHO, at least, what we can do is to warn about it before it's too late.
> 

High order allocations that fail should still display a warning message 
when __GFP_NOWARN is not set, so I don't see what this additional warning 
adds.  I don't think it's responsible to ask admins to know what lumpy 
reclaim is, what memory compaction is, or when a system tends to have more 
high order allocations when memory compaction would be helpful.

What we can do, though, is address bug reports as they are reported when 
high order allocations fail and previous kernels are successful.  I 
haven't seen any lately.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

On Thu, Jul 12, 2012 at 5:40 AM, David Rientjes <rientjes@google.com> wrote:
> On Wed, 11 Jul 2012, Minchan Kim wrote:
>
>> I agree it's an ideal but the problem is that it's too late.
>> Once product is released, we have to recall all products in the worst case.
>> The fact is that lumpy have helped high order allocation implicitly but we removed it
>> without any notification or information. It's a sort of regression and we can't say
>> them "Please report us if it happens". It's irresponsible, too.
>> IMHO, at least, what we can do is to warn about it before it's too late.
>>
>
> High order allocations that fail should still display a warning message
> when __GFP_NOWARN is not set, so I don't see what this additional warning
> adds.  I don't think it's responsible to ask admins to know what lumpy
> reclaim is, what memory compaction is, or when a system tends to have more
> high order allocations when memory compaction would be helpful.
>
> What we can do, though, is address bug reports as they are reported when
> high order allocations fail and previous kernels are successful.  I
> haven't seen any lately.

Did you read my description?

"
Let's think this scenario.

There is QA team in embedded company and they have tested their product.
In test scenario, they can allocate 100 high order allocation.
(they don't matter how many high order allocations in kernel are needed
during test. their concern is just only working well or fail of their
middleware/application) High order allocation will be serviced well
by natural buddy allocation without lumpy's help. So they released
the product and sold out all over the world.
Unfortunately, in real practice, sometime, 105 high order allocation was
needed rarely and fortunately, lumpy reclaim could help it so the product
doesn't have a problem until now.

If they use latest kernel, they will see the new config CONFIG_COMPACTION
which is very poor documentation, and they can't know it's replacement of
lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
that option for size optimization. Of course, QA team still test it but they
can't find the problem if they don't do test stronger than old.
It ends up release the product and sold out all over the world, again.
But in this time, we don't have both lumpy and compaction so the problem
would happen in real practice. A poor enginner from Korea have to flight
to the USA for the fix a ton of products. Otherwise, should recall products
from all over the world. Maybe he can lose a job. :(
"
It's not much exaggerated. who should we blame?

-- 
Kind regards,
Minchan Kim

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

On Thu, Jul 12, 2012 at 5:40 AM, David Rientjes <rientjes@google.com> wrote:
> On Wed, 11 Jul 2012, Minchan Kim wrote:
>
>> I agree it's an ideal but the problem is that it's too late.
>> Once product is released, we have to recall all products in the worst case.
>> The fact is that lumpy have helped high order allocation implicitly but we removed it
>> without any notification or information. It's a sort of regression and we can't say
>> them "Please report us if it happens". It's irresponsible, too.
>> IMHO, at least, what we can do is to warn about it before it's too late.
>>
>
> High order allocations that fail should still display a warning message
> when __GFP_NOWARN is not set, so I don't see what this additional warning
> adds.  I don't think it's responsible to ask admins to know what lumpy
> reclaim is, what memory compaction is, or when a system tends to have more
> high order allocations when memory compaction would be helpful.
>
> What we can do, though, is address bug reports as they are reported when
> high order allocations fail and previous kernels are successful.  I
> haven't seen any lately.

Did you read my description?

"
Let's think this scenario.

There is QA team in embedded company and they have tested their product.
In test scenario, they can allocate 100 high order allocation.
(they don't matter how many high order allocations in kernel are needed
during test. their concern is just only working well or fail of their
middleware/application) High order allocation will be serviced well
by natural buddy allocation without lumpy's help. So they released
the product and sold out all over the world.
Unfortunately, in real practice, sometime, 105 high order allocation was
needed rarely and fortunately, lumpy reclaim could help it so the product
doesn't have a problem until now.

If they use latest kernel, they will see the new config CONFIG_COMPACTION
which is very poor documentation, and they can't know it's replacement of
lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
that option for size optimization. Of course, QA team still test it but they
can't find the problem if they don't do test stronger than old.
It ends up release the product and sold out all over the world, again.
But in this time, we don't have both lumpy and compaction so the problem
would happen in real practice. A poor enginner from Korea have to flight
to the USA for the fix a ton of products. Otherwise, should recall products
from all over the world. Maybe he can lose a job. :(
"
It's not much exaggerated. who should we blame?

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[David Rientjes]

On Thu, 12 Jul 2012, Minchan Kim wrote:

> There is QA team in embedded company and they have tested their product.
> In test scenario, they can allocate 100 high order allocation.
> (they don't matter how many high order allocations in kernel are needed
> during test. their concern is just only working well or fail of their
> middleware/application) High order allocation will be serviced well
> by natural buddy allocation without lumpy's help. So they released
> the product and sold out all over the world.
> Unfortunately, in real practice, sometime, 105 high order allocation was
> needed rarely and fortunately, lumpy reclaim could help it so the product
> doesn't have a problem until now.
> 

If the QA team is going to consider upgrading to a kernel since lumpy 
reclaim has been removed, before they qualify such a kernel they would 
(hopefully) do some due diligence in running this workload and noticing 
the page allocation failure that is emitted to the kernel log for the high 
order page allocations.

> If they use latest kernel, they will see the new config CONFIG_COMPACTION
> which is very poor documentation, and they can't know it's replacement of
> lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
> that option for size optimization.

Improving the description for CONFIG_COMPACTION or adding additional 
documentation in Documentation/vm would be very appreciated by both me and 
this hypothetical engineer :)

Re: [PATCH v2] mm: Warn about costly page allocation

[David Rientjes]

On Thu, 12 Jul 2012, Minchan Kim wrote:

> There is QA team in embedded company and they have tested their product.
> In test scenario, they can allocate 100 high order allocation.
> (they don't matter how many high order allocations in kernel are needed
> during test. their concern is just only working well or fail of their
> middleware/application) High order allocation will be serviced well
> by natural buddy allocation without lumpy's help. So they released
> the product and sold out all over the world.
> Unfortunately, in real practice, sometime, 105 high order allocation was
> needed rarely and fortunately, lumpy reclaim could help it so the product
> doesn't have a problem until now.
> 

If the QA team is going to consider upgrading to a kernel since lumpy 
reclaim has been removed, before they qualify such a kernel they would 
(hopefully) do some due diligence in running this workload and noticing 
the page allocation failure that is emitted to the kernel log for the high 
order page allocations.

> If they use latest kernel, they will see the new config CONFIG_COMPACTION
> which is very poor documentation, and they can't know it's replacement of
> lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
> that option for size optimization.

Improving the description for CONFIG_COMPACTION or adding additional 
documentation in Documentation/vm would be very appreciated by both me and 
this hypothetical engineer :)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

On Wed, Jul 11, 2012 at 04:02:00PM -0700, David Rientjes wrote:
> On Thu, 12 Jul 2012, Minchan Kim wrote:
> 
> > There is QA team in embedded company and they have tested their product.
> > In test scenario, they can allocate 100 high order allocation.
> > (they don't matter how many high order allocations in kernel are needed
> > during test. their concern is just only working well or fail of their
> > middleware/application) High order allocation will be serviced well
> > by natural buddy allocation without lumpy's help. So they released
> > the product and sold out all over the world.
> > Unfortunately, in real practice, sometime, 105 high order allocation was
> > needed rarely and fortunately, lumpy reclaim could help it so the product
> > doesn't have a problem until now.
> > 
> 
> If the QA team is going to consider upgrading to a kernel since lumpy 
> reclaim has been removed, before they qualify such a kernel they would 
> (hopefully) do some due diligence in running this workload and noticing 
> the page allocation failure that is emitted to the kernel log for the high 
> order page allocations.

hopefully, but I guess they will run same test which worked well because
they didn't noticed any problems until now.

> 
> > If they use latest kernel, they will see the new config CONFIG_COMPACTION
> > which is very poor documentation, and they can't know it's replacement of
> > lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
> > that option for size optimization.
> 
> Improving the description for CONFIG_COMPACTION or adding additional 
> documentation in Documentation/vm would be very appreciated by both me and 
> this hypothetical engineer :)

Agreed and that's why I suggested following patch.
It's not elegant but at least, it could attract interest of configuration
people and they could find a regression during test phase.
This description could be improved later by writing new documenation which
includes more detailed story and method for capturing high order allocation
by ftrace once we see regression report.

At the moment, I would like to post this patch, simply.
(Of course, I hope fluent native people will correct a sentence. :) )

Any objections, Andrew, David?

diff --git a/mm/Kconfig b/mm/Kconfig
index 16f6b42..099e681 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -195,6 +195,10 @@ config COMPACTION
        depends on MMU
        help
          Allows the compaction of memory for the allocation of huge pages.
+         Notice. We replaced lumpy reclaim which was scheme of helping
+         high-order allocations with compaction at 3.4 so if you don't
+         care about high-order allocations but want the smallest kernel,
+         you could select "N", otherwise, "y" is preferred.
 
 #
 # support for page migration

> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

On Wed, Jul 11, 2012 at 04:02:00PM -0700, David Rientjes wrote:
> On Thu, 12 Jul 2012, Minchan Kim wrote:
> 
> > There is QA team in embedded company and they have tested their product.
> > In test scenario, they can allocate 100 high order allocation.
> > (they don't matter how many high order allocations in kernel are needed
> > during test. their concern is just only working well or fail of their
> > middleware/application) High order allocation will be serviced well
> > by natural buddy allocation without lumpy's help. So they released
> > the product and sold out all over the world.
> > Unfortunately, in real practice, sometime, 105 high order allocation was
> > needed rarely and fortunately, lumpy reclaim could help it so the product
> > doesn't have a problem until now.
> > 
> 
> If the QA team is going to consider upgrading to a kernel since lumpy 
> reclaim has been removed, before they qualify such a kernel they would 
> (hopefully) do some due diligence in running this workload and noticing 
> the page allocation failure that is emitted to the kernel log for the high 
> order page allocations.

hopefully, but I guess they will run same test which worked well because
they didn't noticed any problems until now.

> 
> > If they use latest kernel, they will see the new config CONFIG_COMPACTION
> > which is very poor documentation, and they can't know it's replacement of
> > lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
> > that option for size optimization.
> 
> Improving the description for CONFIG_COMPACTION or adding additional 
> documentation in Documentation/vm would be very appreciated by both me and 
> this hypothetical engineer :)

Agreed and that's why I suggested following patch.
It's not elegant but at least, it could attract interest of configuration
people and they could find a regression during test phase.
This description could be improved later by writing new documenation which
includes more detailed story and method for capturing high order allocation
by ftrace once we see regression report.

At the moment, I would like to post this patch, simply.
(Of course, I hope fluent native people will correct a sentence. :) )

Any objections, Andrew, David?

diff --git a/mm/Kconfig b/mm/Kconfig
index 16f6b42..099e681 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -195,6 +195,10 @@ config COMPACTION
        depends on MMU
        help
          Allows the compaction of memory for the allocation of huge pages.
+         Notice. We replaced lumpy reclaim which was scheme of helping
+         high-order allocations with compaction at 3.4 so if you don't
+         care about high-order allocations but want the smallest kernel,
+         you could select "N", otherwise, "y" is preferred.
 
 #
 # support for page migration

> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[David Rientjes]

On Thu, 12 Jul 2012, Minchan Kim wrote:

> Agreed and that's why I suggested following patch.
> It's not elegant but at least, it could attract interest of configuration
> people and they could find a regression during test phase.
> This description could be improved later by writing new documenation which
> includes more detailed story and method for capturing high order allocation
> by ftrace once we see regression report.
> 
> At the moment, I would like to post this patch, simply.
> (Of course, I hope fluent native people will correct a sentence. :) )
> 
> Any objections, Andrew, David?
> 

There are other config options like CONFIG_SLOB that are used for a very 
small memory footprint on systems like this.  We used to have 
CONFIG_EMBEDDED to suggest options like this but that has since been 
renamed as CONFIG_EXPERT and has become obscured.

If size is really the only difference, I would think that people who want 
the smallest kernel possible would be doing allnoconfig and then 
selectively enabling what they need, so defconfig isn't really relevant 
here.  And it's very difficult for an admin to know whether or not they 
"care about high-order allocations."

I'd reconsider disabling compaction by default unless there are other 
considerations that haven't been mentioned.

Re: [PATCH v2] mm: Warn about costly page allocation

[David Rientjes]

On Thu, 12 Jul 2012, Minchan Kim wrote:

> Agreed and that's why I suggested following patch.
> It's not elegant but at least, it could attract interest of configuration
> people and they could find a regression during test phase.
> This description could be improved later by writing new documenation which
> includes more detailed story and method for capturing high order allocation
> by ftrace once we see regression report.
> 
> At the moment, I would like to post this patch, simply.
> (Of course, I hope fluent native people will correct a sentence. :) )
> 
> Any objections, Andrew, David?
> 

There are other config options like CONFIG_SLOB that are used for a very 
small memory footprint on systems like this.  We used to have 
CONFIG_EMBEDDED to suggest options like this but that has since been 
renamed as CONFIG_EXPERT and has become obscured.

If size is really the only difference, I would think that people who want 
the smallest kernel possible would be doing allnoconfig and then 
selectively enabling what they need, so defconfig isn't really relevant 
here.  And it's very difficult for an admin to know whether or not they 
"care about high-order allocations."

I'd reconsider disabling compaction by default unless there are other 
considerations that haven't been mentioned.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

On Wed, Jul 11, 2012 at 07:33:41PM -0700, David Rientjes wrote:
> On Thu, 12 Jul 2012, Minchan Kim wrote:
> 
> > Agreed and that's why I suggested following patch.
> > It's not elegant but at least, it could attract interest of configuration
> > people and they could find a regression during test phase.
> > This description could be improved later by writing new documenation which
> > includes more detailed story and method for capturing high order allocation
> > by ftrace once we see regression report.
> > 
> > At the moment, I would like to post this patch, simply.
> > (Of course, I hope fluent native people will correct a sentence. :) )
> > 
> > Any objections, Andrew, David?
> > 
> 
> There are other config options like CONFIG_SLOB that are used for a very 
> small memory footprint on systems like this.  We used to have 
> CONFIG_EMBEDDED to suggest options like this but that has since been 
> renamed as CONFIG_EXPERT and has become obscured.
> 
> If size is really the only difference, I would think that people who want 
> the smallest kernel possible would be doing allnoconfig and then 
> selectively enabling what they need, so defconfig isn't really relevant 
> here.  And it's very difficult for an admin to know whether or not they 
> "care about high-order allocations."
> 
> I'd reconsider disabling compaction by default unless there are other 
> considerations that haven't been mentioned.

I agree but it doesn't matter with current problem.
The point of current problem is to let admin know dangerous of regression
about high order allocation before releasing the product.

Although we enable it by defaut, he can change it with "N" unless he
knows removing of lumpy reclaim.

> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Kind regards,
Minchan Kim

Re: [PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

On Wed, Jul 11, 2012 at 07:33:41PM -0700, David Rientjes wrote:
> On Thu, 12 Jul 2012, Minchan Kim wrote:
> 
> > Agreed and that's why I suggested following patch.
> > It's not elegant but at least, it could attract interest of configuration
> > people and they could find a regression during test phase.
> > This description could be improved later by writing new documenation which
> > includes more detailed story and method for capturing high order allocation
> > by ftrace once we see regression report.
> > 
> > At the moment, I would like to post this patch, simply.
> > (Of course, I hope fluent native people will correct a sentence. :) )
> > 
> > Any objections, Andrew, David?
> > 
> 
> There are other config options like CONFIG_SLOB that are used for a very 
> small memory footprint on systems like this.  We used to have 
> CONFIG_EMBEDDED to suggest options like this but that has since been 
> renamed as CONFIG_EXPERT and has become obscured.
> 
> If size is really the only difference, I would think that people who want 
> the smallest kernel possible would be doing allnoconfig and then 
> selectively enabling what they need, so defconfig isn't really relevant 
> here.  And it's very difficult for an admin to know whether or not they 
> "care about high-order allocations."
> 
> I'd reconsider disabling compaction by default unless there are other 
> considerations that haven't been mentioned.

I agree but it doesn't matter with current problem.
The point of current problem is to let admin know dangerous of regression
about high order allocation before releasing the product.

Although we enable it by defaut, he can change it with "N" unless he
knows removing of lumpy reclaim.

> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

Since lumpy reclaim was introduced at 2.6.23, it helped higher
order allocation.
Recently, we removed it at 3.4 and we didn't enable compaction
forcingly[1]. The reason makes sense that compaction.o + migration.o
isn't trivial for system doesn't use higher order allocation.
But the problem is that we have to enable compaction explicitly
while lumpy reclaim enabled unconditionally.

Normally, admin doesn't know his system have used higher order
allocation and even lumpy reclaim have helped it.
Admin in embdded system have a tendency to minimise code size so that
they can disable compaction. In this case, we can see page allocation
failure we can never see in the past. It's critical on embedded side
because...

Let's think this scenario.

There is QA team in embedded company and they have tested their product.
In test scenario, they can allocate 100 high order allocation.
(they don't matter how many high order allocations in kernel are needed
during test. their concern is just only working well or fail of their
middleware/application) High order allocation will be serviced well
by natural buddy allocation without lumpy's help. So they released
the product and sold out all over the world.
Unfortunately, in real practice, sometime, 105 high order allocation was
needed rarely and fortunately, lumpy reclaim could help it so the product
doesn't have a problem until now.

If they use latest kernel, they will see the new config CONFIG_COMPACTION
which is very poor documentation, and they can't know it's replacement of
lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
that option for size optimization. Of course, QA team still test it but they
can't find the problem if they don't do test stronger than old.
It ends up release the product and sold out all over the world, again.
But in this time, we don't have both lumpy and compaction so the problem
would happen in real practice. A poor enginner from Korea have to flight
to the USA for the fix a ton of products. Otherwise, should recall products
from all over the world. Maybe he can lose a job. :(

This patch adds warning for notice. If the system try to allocate
PAGE_ALLOC_COSTLY_ORDER above page and system enters reclaim path,
it emits the warning. At least, it gives a chance to look into their
system before the relase.

Please keep in mind. It's not a good idea to depend lumpy/compaction
for regular high-order allocations. Both depends on being able to move
MIGRATE_MOVABLE allocations to satisfy the high-order allocation. If used
reregularly for high-order kernel allocations and tehy are long-lived,
the system will eventually be unable to grant these allocations, with or
without compaction or lumpy reclaim. Hatchet jobs that work around this problem
include forcing MIGRATE_RESERVE to be only used for high-order allocations
and tuning its size. It's a major hack though and is unlikely to be merged
to mainline but might suit an embedded product.

This patch avoids false positive by alloc_large_system_hash which
allocates with GFP_ATOMIC and a fallback mechanism so it can make
this warning useless.

[1] c53919ad(mm: vmscan: remove lumpy reclaim)

Signed-off-by: Minchan Kim <minchan@kernel.org>
---
Changelog

* from v1
 - add more description about warning failure of high-order allocation
 - use printk_ratelimited/pr_warn and dump stack - [Mel, Andrew]
 - noinline/__always_inline optimization - Andrew
 - modify warning message - Andrew

 mm/page_alloc.c |   37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a4d3a19..a8f60d0 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2276,6 +2276,41 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
 	return alloc_flags;
 }
 
+#if defined(CONFIG_DEBUG_VM) && !defined(CONFIG_COMPACTION)
+
+static DEFINE_RATELIMIT_STATE(highorderalloc_rs,
+		DEFAULT_RATELIMIT_INTERVAL,
+		DEFAULT_RATELIMIT_BURST);
+
+static noinline void __check_page_alloc_costly_order(unsigned int order,
+							gfp_t flags)
+{
+	if ((flags & __GFP_NOWARN) || !__ratelimit(&highorderalloc_rs))
+		return;
+
+	pr_warn("%s: try allocating high-order allocation: "
+		"order:%d, mode:0x%x\n", current->comm, order, flags);
+	pr_warn("Enable CONFIG_COMPACTION if high-order allocations are "
+		"very few and rare.\n");
+	pr_warn("If you see this message frequently and regularly, "
+		"CONFIG_COMPACTION wouldn't help it. Then, please send "
+		"an email to linux-mm@kvack.org\n");
+	dump_stack();
+}
+
+static __always_inline void check_page_alloc_costly_order(unsigned int order,
+							gfp_t flags)
+{
+	if (unlikely(order > PAGE_ALLOC_COSTLY_ORDER))
+		__check_page_alloc_costly_order(order, flags);
+}
+#else
+static inline void check_page_alloc_costly_order(unsigned int order,
+							gfp_t flags)
+{
+}
+#endif
+
 static inline struct page *
 __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	struct zonelist *zonelist, enum zone_type high_zoneidx,
@@ -2353,6 +2388,8 @@ rebalance:
 	if (!wait)
 		goto nopage;
 
+	check_page_alloc_costly_order(order, gfp_mask);
+
 	/* Avoid recursion of direct reclaim */
 	if (current->flags & PF_MEMALLOC)
 		goto nopage;
-- 
1.7.9.5

[PATCH v2] mm: Warn about costly page allocation

[Minchan Kim]

Since lumpy reclaim was introduced at 2.6.23, it helped higher
order allocation.
Recently, we removed it at 3.4 and we didn't enable compaction
forcingly[1]. The reason makes sense that compaction.o + migration.o
isn't trivial for system doesn't use higher order allocation.
But the problem is that we have to enable compaction explicitly
while lumpy reclaim enabled unconditionally.

Normally, admin doesn't know his system have used higher order
allocation and even lumpy reclaim have helped it.
Admin in embdded system have a tendency to minimise code size so that
they can disable compaction. In this case, we can see page allocation
failure we can never see in the past. It's critical on embedded side
because...

Let's think this scenario.

There is QA team in embedded company and they have tested their product.
In test scenario, they can allocate 100 high order allocation.
(they don't matter how many high order allocations in kernel are needed
during test. their concern is just only working well or fail of their
middleware/application) High order allocation will be serviced well
by natural buddy allocation without lumpy's help. So they released
the product and sold out all over the world.
Unfortunately, in real practice, sometime, 105 high order allocation was
needed rarely and fortunately, lumpy reclaim could help it so the product
doesn't have a problem until now.

If they use latest kernel, they will see the new config CONFIG_COMPACTION
which is very poor documentation, and they can't know it's replacement of
lumpy reclaim(even, they don't know lumpy reclaim) so they simply disable
that option for size optimization. Of course, QA team still test it but they
can't find the problem if they don't do test stronger than old.
It ends up release the product and sold out all over the world, again.
But in this time, we don't have both lumpy and compaction so the problem
would happen in real practice. A poor enginner from Korea have to flight
to the USA for the fix a ton of products. Otherwise, should recall products
from all over the world. Maybe he can lose a job. :(

This patch adds warning for notice. If the system try to allocate
PAGE_ALLOC_COSTLY_ORDER above page and system enters reclaim path,
it emits the warning. At least, it gives a chance to look into their
system before the relase.

Please keep in mind. It's not a good idea to depend lumpy/compaction
for regular high-order allocations. Both depends on being able to move
MIGRATE_MOVABLE allocations to satisfy the high-order allocation. If used
reregularly for high-order kernel allocations and tehy are long-lived,
the system will eventually be unable to grant these allocations, with or
without compaction or lumpy reclaim. Hatchet jobs that work around this problem
include forcing MIGRATE_RESERVE to be only used for high-order allocations
and tuning its size. It's a major hack though and is unlikely to be merged
to mainline but might suit an embedded product.

This patch avoids false positive by alloc_large_system_hash which
allocates with GFP_ATOMIC and a fallback mechanism so it can make
this warning useless.

[1] c53919ad(mm: vmscan: remove lumpy reclaim)

Signed-off-by: Minchan Kim <minchan@kernel.org>
---
Changelog

* from v1
 - add more description about warning failure of high-order allocation
 - use printk_ratelimited/pr_warn and dump stack - [Mel, Andrew]
 - noinline/__always_inline optimization - Andrew
 - modify warning message - Andrew

 mm/page_alloc.c |   37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a4d3a19..a8f60d0 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2276,6 +2276,41 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
 	return alloc_flags;
 }
 
+#if defined(CONFIG_DEBUG_VM) && !defined(CONFIG_COMPACTION)
+
+static DEFINE_RATELIMIT_STATE(highorderalloc_rs,
+		DEFAULT_RATELIMIT_INTERVAL,
+		DEFAULT_RATELIMIT_BURST);
+
+static noinline void __check_page_alloc_costly_order(unsigned int order,
+							gfp_t flags)
+{
+	if ((flags & __GFP_NOWARN) || !__ratelimit(&highorderalloc_rs))
+		return;
+
+	pr_warn("%s: try allocating high-order allocation: "
+		"order:%d, mode:0x%x\n", current->comm, order, flags);
+	pr_warn("Enable CONFIG_COMPACTION if high-order allocations are "
+		"very few and rare.\n");
+	pr_warn("If you see this message frequently and regularly, "
+		"CONFIG_COMPACTION wouldn't help it. Then, please send "
+		"an email to linux-mm@kvack.org\n");
+	dump_stack();
+}
+
+static __always_inline void check_page_alloc_costly_order(unsigned int order,
+							gfp_t flags)
+{
+	if (unlikely(order > PAGE_ALLOC_COSTLY_ORDER))
+		__check_page_alloc_costly_order(order, flags);
+}
+#else
+static inline void check_page_alloc_costly_order(unsigned int order,
+							gfp_t flags)
+{
+}
+#endif
+
 static inline struct page *
 __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	struct zonelist *zonelist, enum zone_type high_zoneidx,
@@ -2353,6 +2388,8 @@ rebalance:
 	if (!wait)
 		goto nopage;
 
+	check_page_alloc_costly_order(order, gfp_mask);
+
 	/* Avoid recursion of direct reclaim */
 	if (current->flags & PF_MEMALLOC)
 		goto nopage;
-- 
1.7.9.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>