[PATCH V2] mm:memory hot-add: memory can not been added to movable zone

[PATCH V2] mm:memory hot-add: memory can not been added to movable zone

[Changsheng Liu]

From: Changsheng Liu <liuchangcheng@inspur.com>

When memory is hot added, should_add_memory_movable() always returns 0
because the movable zone is empty, so the memory that was hot added will
add to the normal zone even if we want to remove the memory.

So we change should_add_memory_movable(): if the user config
CONFIG_MOVABLE_NODE it will return 1 when the movable zone is empty.

Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Changsheng Liu <liuchangcheng@inspur.com>
Tested-by: Dongdong Fan <fandd@inspur.com>
---
 mm/memory_hotplug.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 26fbba7..ff658f2 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1199,8 +1199,7 @@ static int should_add_memory_movable(int nid, u64 start, u64 size)
 	struct zone *movable_zone = pgdat->node_zones + ZONE_MOVABLE;
 
 	if (zone_is_empty(movable_zone))
-		return 0;
-
+		return IS_ENABLED(CONFIG_MOVABLE_NODE);
 	if (movable_zone->zone_start_pfn <= start_pfn)
 		return 1;
 
-- 
1.7.1

Re: [PATCH V2] mm:memory hot-add: memory can not been added to movable zone

[Vlastimil Babka]

On 08/20/2015 09:28 AM, Changsheng Liu wrote:
> From: Changsheng Liu <liuchangcheng@inspur.com>
>
> When memory is hot added, should_add_memory_movable() always returns 0
> because the movable zone is empty, so the memory that was hot added will
> add to the normal zone even if we want to remove the memory.

I'm not expert on memory hot-plug, but since you CC'd me, I wonder... 
the function has this comment: " * If movable zone has already been 
setup, newly added memory should be check."

So I read it like "if you want movable memory *at all*, you should do 
some setup first" (but don't ask me what setup). After your patch, every 
hot-added memory would be automatically movable? Isn't that silently 
changing behavior against user expectations? What about those that don't 
want to hot-remove and don't want movable zones (which limit what kind 
of allocations are possible), is there a way to prevent memory being 
movable after your patch?

> So we change should_add_memory_movable(): if the user config
> CONFIG_MOVABLE_NODE it will return 1 when the movable zone is empty.
>
> Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
> Signed-off-by: Changsheng Liu <liuchangcheng@inspur.com>
> Tested-by: Dongdong Fan <fandd@inspur.com>
> ---
>   mm/memory_hotplug.c |    3 +--
>   1 files changed, 1 insertions(+), 2 deletions(-)
>
> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index 26fbba7..ff658f2 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -1199,8 +1199,7 @@ static int should_add_memory_movable(int nid, u64 start, u64 size)
>   	struct zone *movable_zone = pgdat->node_zones + ZONE_MOVABLE;
>
>   	if (zone_is_empty(movable_zone))
> -		return 0;
> -
> +		return IS_ENABLED(CONFIG_MOVABLE_NODE);
>   	if (movable_zone->zone_start_pfn <= start_pfn)
>   		return 1;
>
>

Re: [PATCH V2] mm:memory hot-add: memory can not been added to movable zone

[Vlastimil Babka]

On 08/20/2015 09:41 AM, Vlastimil Babka wrote:
> want to hot-remove and don't want movable zones (which limit what kind
> of allocations are possible), is there a way to prevent memory being
> movable after your patch?

Oh, and not enabling CONFIG_MOVABLE_NODE is not sufficient here IMHO, as 
users might want to use a distro kernel that has the option enabled, but 
still not want to use it.

Re: [PATCH V2] mm:memory hot-add: memory can not been added to movable zone

[Changsheng Liu]

On 08/20/201515:41, Vlastimil Babka wrote:
> On 08/20/2015 09:28 AM, Changsheng Liu wrote:
>> From: Changsheng Liu <liuchangcheng@inspur.com>
>>
>> When memory is hot added, should_add_memory_movable() always returns 0
>> because the movable zone is empty, so the memory that was hot added will
>> add to the normal zone even if we want to remove the memory.
>
> I'm not expert on memory hot-plug, but since you CC'd me, I wonder... 
> the function has this comment: " * If movable zone has already been 
> setup, newly added memory should be check."
>
> So I read it like "if you want movable memory *at all*, you should do 
> some setup first" (but don't ask me what setup). After your patch, 
> every hot-added memory would be automatically movable? Isn't that 
> silently changing behavior against user expectations? What about those 
> that don't want to hot-remove and don't want movable zones (which 
> limit what kind of allocations are possible), is there a way to 
> prevent memory being movable after your patch?
     After the system startup, we hot added one cpu with memory, The 
function arch_add_memory() will add the memory to
     normal zone defaultly but now all zones including normal zone and 
movable zone are empty.So If we want to add the memory
     to movable zone we need change should_add_memory_movable().
>
>> So we change should_add_memory_movable(): if the user config
>> CONFIG_MOVABLE_NODE it will return 1 when the movable zone is empty.
>>
>> Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
>> Signed-off-by: Changsheng Liu <liuchangcheng@inspur.com>
>> Tested-by: Dongdong Fan <fandd@inspur.com>
>> ---
>>   mm/memory_hotplug.c |    3 +--
>>   1 files changed, 1 insertions(+), 2 deletions(-)
>>
>> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
>> index 26fbba7..ff658f2 100644
>> --- a/mm/memory_hotplug.c
>> +++ b/mm/memory_hotplug.c
>> @@ -1199,8 +1199,7 @@ static int should_add_memory_movable(int nid, 
>> u64 start, u64 size)
>>       struct zone *movable_zone = pgdat->node_zones + ZONE_MOVABLE;
>>
>>       if (zone_is_empty(movable_zone))
>> -        return 0;
>> -
>> +        return IS_ENABLED(CONFIG_MOVABLE_NODE);
>>       if (movable_zone->zone_start_pfn <= start_pfn)
>>           return 1;
>>
>>
>
> .
>

Re: [PATCH V2] mm:memory hot-add: memory can not been added to movable zone

[Vlastimil Babka]

On 08/21/2015 04:00 AM, Changsheng Liu wrote:
>
> On 08/20/201515:41, Vlastimil Babka wrote:
>> On 08/20/2015 09:28 AM, Changsheng Liu wrote:
>>> From: Changsheng Liu <liuchangcheng@inspur.com>
>>>
>>> When memory is hot added, should_add_memory_movable() always returns 0
>>> because the movable zone is empty, so the memory that was hot added will
>>> add to the normal zone even if we want to remove the memory.
>>
>> I'm not expert on memory hot-plug, but since you CC'd me, I wonder...
>> the function has this comment: " * If movable zone has already been
>> setup, newly added memory should be check."
>>
>> So I read it like "if you want movable memory *at all*, you should do
>> some setup first" (but don't ask me what setup). After your patch,
>> every hot-added memory would be automatically movable? Isn't that
>> silently changing behavior against user expectations? What about those
>> that don't want to hot-remove and don't want movable zones (which
>> limit what kind of allocations are possible), is there a way to
>> prevent memory being movable after your patch?
>       After the system startup, we hot added one cpu with memory, The
> function arch_add_memory() will add the memory to
>       normal zone defaultly but now all zones including normal zone and
> movable zone are empty.So If we want to add the memory
>       to movable zone we need change should_add_memory_movable().

I have poked a bit at the code and documentation, and I may still not 
have the complete picture.

Are you using movable_node kernel option to expect all hotpluggable 
memory to be movable? Then it's probably a bug. But then your patch 
should probably use movable_node_is_enabled() instead of checking just 
the config. Otherwise it would be making zone movable also for those who 
enabled the config, but don't pass the kernel option, and that would be 
wrong?

Or are you onlining memory by "echo online_movable > 
/sys/devices/system/memory/memoryXXX/state" without node_movable kernel 
option?

>>
>>> So we change should_add_memory_movable(): if the user config
>>> CONFIG_MOVABLE_NODE it will return 1 when the movable zone is empty.
>>>
>>> Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
>>> Signed-off-by: Changsheng Liu <liuchangcheng@inspur.com>
>>> Tested-by: Dongdong Fan <fandd@inspur.com>
>>> ---
>>>    mm/memory_hotplug.c |    3 +--
>>>    1 files changed, 1 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
>>> index 26fbba7..ff658f2 100644
>>> --- a/mm/memory_hotplug.c
>>> +++ b/mm/memory_hotplug.c
>>> @@ -1199,8 +1199,7 @@ static int should_add_memory_movable(int nid,
>>> u64 start, u64 size)
>>>        struct zone *movable_zone = pgdat->node_zones + ZONE_MOVABLE;
>>>
>>>        if (zone_is_empty(movable_zone))
>>> -        return 0;
>>> -
>>> +        return IS_ENABLED(CONFIG_MOVABLE_NODE);
>>>        if (movable_zone->zone_start_pfn <= start_pfn)
>>>            return 1;
>>>
>>>
>>
>> .
>>
>

Re: [PATCH V2] mm:memory hot-add: memory can not been added to movable zone

[Yasuaki Ishimatsu]

On Thu, 20 Aug 2015 03:28:05 -0400
Changsheng Liu <liuchangsheng@inspur.com> wrote:

> From: Changsheng Liu <liuchangcheng@inspur.com>
> 
> When memory is hot added, should_add_memory_movable() always returns 0
> because the movable zone is empty, so the memory that was hot added will
> add to the normal zone even if we want to remove the memory.
> 
> So we change should_add_memory_movable(): if the user config
> CONFIG_MOVABLE_NODE it will return 1 when the movable zone is empty.
> 
> Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
> Signed-off-by: Changsheng Liu <liuchangcheng@inspur.com>
> Tested-by: Dongdong Fan <fandd@inspur.com>
> ---
>  mm/memory_hotplug.c |    3 +--
>  1 files changed, 1 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index 26fbba7..ff658f2 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -1199,8 +1199,7 @@ static int should_add_memory_movable(int nid, u64 start, u64 size)
>  	struct zone *movable_zone = pgdat->node_zones + ZONE_MOVABLE;
>  

>  	if (zone_is_empty(movable_zone))
> -		return 0;
> -
> +		return IS_ENABLED(CONFIG_MOVABLE_NODE);
>  	if (movable_zone->zone_start_pfn <= start_pfn)
>  		return 1;

Currently, kernel allows to create ZONE_MOVABLE after ZONE_NORMAL as follows:
 PFN low                                 high 
       ---|-------------|-------------|---
            ZONE_NORMAL   ZONE_MOVABLE

But kernel does not allow to create ZONE_MOVABLE before ZONE_NORMAL as follows:
 PFN low                                 high 
       ---|-------------|-------------|---
            ZONE_MOVABLE  ZONE_NORMAL

Also, kernel does not allow to create ZONE_MOVABLE in ZOME_NORMAL as follows:
 PFN low                                              high 
       ---|-------------|-------------|-------------|---
            ZONE_NORMAL   ZONE_MOVABLE  ZONE_NORMAL

So should_add_memory_movable() checks them.

Accoring to your patch, when movable_zone is empty, the hot added memory is
always managed to ZONE_MOVABLE. It means that ZONE_MOVALBE will puts before/in
ZONE_NORMAL.

You must prevent from creating ZONE_MOVABLE before/in ZONE_NORMAL.

Thanks,
Yasuaki Ishimatsu

>  
> -- 
> 1.7.1
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH V2] mm:memory hot-add: memory can not been added to movable zone

[Changsheng Liu]

First, thanks very much for your review, I will update codes according 
to your suggestion

On 2015/8/25 21:12, Yasuaki Ishimatsu wrote:
> On Thu, 20 Aug 2015 03:28:05 -0400
> Changsheng Liu <liuchangsheng@inspur.com> wrote:
>
>> From: Changsheng Liu <liuchangcheng@inspur.com>
>>
>> When memory is hot added, should_add_memory_movable() always returns 0
>> because the movable zone is empty, so the memory that was hot added will
>> add to the normal zone even if we want to remove the memory.
>>
>> So we change should_add_memory_movable(): if the user config
>> CONFIG_MOVABLE_NODE it will return 1 when the movable zone is empty.
>>
>> Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
>> Signed-off-by: Changsheng Liu <liuchangcheng@inspur.com>
>> Tested-by: Dongdong Fan <fandd@inspur.com>
>> ---
>>   mm/memory_hotplug.c |    3 +--
>>   1 files changed, 1 insertions(+), 2 deletions(-)
>>
>> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
>> index 26fbba7..ff658f2 100644
>> --- a/mm/memory_hotplug.c
>> +++ b/mm/memory_hotplug.c
>> @@ -1199,8 +1199,7 @@ static int should_add_memory_movable(int nid, u64 start, u64 size)
>>   	struct zone *movable_zone = pgdat->node_zones + ZONE_MOVABLE;
>>   
>>   	if (zone_is_empty(movable_zone))
>> -		return 0;
>> -
>> +		return IS_ENABLED(CONFIG_MOVABLE_NODE);
>>   	if (movable_zone->zone_start_pfn <= start_pfn)
>>   		return 1;
> Currently, kernel allows to create ZONE_MOVABLE after ZONE_NORMAL as follows:
>   PFN low                                 high
>         ---|-------------|-------------|---
>              ZONE_NORMAL   ZONE_MOVABLE
>
> But kernel does not allow to create ZONE_MOVABLE before ZONE_NORMAL as follows:
>   PFN low                                 high
>         ---|-------------|-------------|---
>              ZONE_MOVABLE  ZONE_NORMAL
>
> Also, kernel does not allow to create ZONE_MOVABLE in ZOME_NORMAL as follows:
>   PFN low                                              high
>         ---|-------------|-------------|-------------|---
>              ZONE_NORMAL   ZONE_MOVABLE  ZONE_NORMAL
>
> So should_add_memory_movable() checks them.
>
> Accoring to your patch, when movable_zone is empty, the hot added memory is
> always managed to ZONE_MOVABLE. It means that ZONE_MOVALBE will puts before/in
> ZONE_NORMAL.
>
> You must prevent from creating ZONE_MOVABLE before/in ZONE_NORMAL.
>
> Thanks,
> Yasuaki Ishimatsu
>
>>   
>> -- 
>> 1.7.1
>>
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org.  For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> .
>