[PATCH V7] mm: memory hot-add: memory can not be added to movable zone defaultly

[PATCH V7] mm: memory hot-add: memory can not be added to movable zone defaultly

[Changsheng Liu]

From: Changsheng Liu <liuchangcheng@inspur.com>

After the user config CONFIG_MOVABLE_NODE,
When the memory is hot added, should_add_memory_movable() return 0
because all zones including ZONE_MOVABLE are empty,
so the memory that was hot added will be assigned to ZONE_NORMAL
and ZONE_NORMAL will be created firstly.
But we want the whole node to be added to ZONE_MOVABLE by default.

So we change should_add_memory_movable(): if the user config
CONFIG_MOVABLE_NODE and sysctl parameter hotadd_memory_as_movable is 1
and the ZONE_NORMAL is empty or the pfn of the hot-added memory
is after the end of the ZONE_NORMAL it will always return 1
and then the whole node will be added to ZONE_MOVABLE by default.
If we want the node to be assigned to ZONE_NORMAL,
we can do it as follows:
"echo online_kernel > /sys/devices/system/memory/memoryXXX/state"

By the patch, the behavious of kernel is changed by sysctl,
user can automatically create movable memory
by only the following udev rule:
SUBSYSTEM=="memory", ACTION=="add",
ATTR{state}=="offline", ATTR{state}="online"

Signed-off-by: Changsheng Liu <liuchangsheng@inspur.com>
Signed-off-by: Xiaofeng Yan <yanxiaofeng@inspur.com>
Tested-by: Dongdong Fan <fandd@inspur.com>
Cc: Wang Nan <wangnan0@huawei.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Tang Chen <tangchen@cn.fujitsu.com>
Cc: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: Xishi Qiu <qiuxishi@huawei.com>
---
 Documentation/memory-hotplug.txt |    5 ++++-
 kernel/sysctl.c                  |   15 +++++++++++++++
 mm/memory_hotplug.c              |   24 ++++++++++++++++++++++++
 3 files changed, 43 insertions(+), 1 deletions(-)

diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt
index ce2cfcf..7ac7485 100644
--- a/Documentation/memory-hotplug.txt
+++ b/Documentation/memory-hotplug.txt
@@ -277,7 +277,7 @@ And if the memory block is in ZONE_MOVABLE, you can change it to ZONE_NORMAL:
 After this, memory block XXX's state will be 'online' and the amount of
 available memory will be increased.
 
-Currently, newly added memory is added as ZONE_NORMAL (for powerpc, ZONE_DMA).
+Currently, newly added memory is added as ZONE_NORMAL or ZONE_MOVABLE (for powerpc, ZONE_DMA).
 This may be changed in future.
 
 
@@ -319,6 +319,9 @@ creates ZONE_MOVABLE as following.
   Size of memory not for movable pages (not for offline) is TOTAL - ZZZZ.
   Size of memory for movable pages (for offline) is ZZZZ.
 
+And a sysctl parameter for assigning the hot added memory to ZONE_MOVABLE is
+supported. If the value of "kernel/hotadd_memory_as_movable" is 1,the hot added
+memory will be assigned to ZONE_MOVABLE by default.
 
 Note: Unfortunately, there is no information to show which memory block belongs
 to ZONE_MOVABLE. This is TBD.
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 19b62b5..16b1501 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -166,6 +166,10 @@ extern int unaligned_dump_stack;
 extern int no_unaligned_warning;
 #endif
 
+#ifdef CONFIG_MOVABLE_NODE
+extern int hotadd_memory_as_movable;
+#endif
+
 #ifdef CONFIG_PROC_SYSCTL
 
 #define SYSCTL_WRITES_LEGACY	-1
@@ -1139,6 +1143,17 @@ static struct ctl_table kern_table[] = {
 		.proc_handler	= timer_migration_handler,
 	},
 #endif
+/*If the value of "kernel/hotadd_memory_as_movable" is 1,the hot added
+ * memory will be assigned to ZONE_MOVABLE by default.*/
+#ifdef CONFIG_MOVABLE_NODE
+	{
+		.procname	= "hotadd_memory_as_movable",
+		.data		= &hotadd_memory_as_movable,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec,
+	},
+#endif
 	{ }
 };
 
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 26fbba7..eca5512 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -37,6 +37,11 @@
 
 #include "internal.h"
 
+/*If the global variable value is 1,
+ * the hot added memory will be assigned to ZONE_MOVABLE by default
+ */
+int hotadd_memory_as_movable;
+
 /*
  * online_page_callback contains pointer to current page onlining function.
  * Initially it is generic_online_page(). If it is required it could be
@@ -1190,6 +1195,9 @@ static int check_hotplug_memory_range(u64 start, u64 size)
 /*
  * If movable zone has already been setup, newly added memory should be check.
  * If its address is higher than movable zone, it should be added as movable.
+ * And if system config CONFIG_MOVABLE_NODE and set the sysctl parameter
+ * "hotadd_memory_as_movable" and added memory does not overlap the zone
+ * before MOVABLE_ZONE,the memory will be added as movable.
  * Without this check, movable zone may overlap with other zone.
  */
 static int should_add_memory_movable(int nid, u64 start, u64 size)
@@ -1197,6 +1205,22 @@ static int should_add_memory_movable(int nid, u64 start, u64 size)
 	unsigned long start_pfn = start >> PAGE_SHIFT;
 	pg_data_t *pgdat = NODE_DATA(nid);
 	struct zone *movable_zone = pgdat->node_zones + ZONE_MOVABLE;
+	struct zone *pre_zone = pgdat->node_zones + (ZONE_MOVABLE - 1);
+	/*
+	 * The system configs CONFIG_MOVABLE_NODE to assign a node
+	 * which has only movable memory,so the hot-added memory should
+	 * be assigned to ZONE_MOVABLE by default,
+	 * but the function zone_for_memory() assign the hot-added memory
+	 * to ZONE_NORMAL(x86_64) by default.Kernel does not allow to
+	 * create ZONE_MOVABLE before ZONE_NORMAL,So if the value of
+	 * sysctl parameter "hotadd_memory_as_movable" is 1
+	 * and the ZONE_NORMAL is empty or the pfn of the hot-added memory
+	 * is after the end of ZONE_NORMAL
+	 * the hot-added memory will be assigned to ZONE_MOVABLE.
+	 */
+	if (hotadd_memory_as_movable
+	&& (zone_is_empty(pre_zone) || zone_end_pfn(pre_zone) <= start_pfn))
+		return 1;
 
 	if (zone_is_empty(movable_zone))
 		return 0;
-- 
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH V7] mm: memory hot-add: memory can not be added to movable zone defaultly

[Vlastimil Babka]

On 10/12/2015 08:58 AM, Changsheng Liu wrote:
> From: Changsheng Liu <liuchangcheng@inspur.com>
> 
> After the user config CONFIG_MOVABLE_NODE,
> When the memory is hot added, should_add_memory_movable() return 0
> because all zones including ZONE_MOVABLE are empty,
> so the memory that was hot added will be assigned to ZONE_NORMAL
> and ZONE_NORMAL will be created firstly.
> But we want the whole node to be added to ZONE_MOVABLE by default.
> 
> So we change should_add_memory_movable(): if the user config
> CONFIG_MOVABLE_NODE and sysctl parameter hotadd_memory_as_movable is 1
> and the ZONE_NORMAL is empty or the pfn of the hot-added memory
> is after the end of the ZONE_NORMAL it will always return 1
> and then the whole node will be added to ZONE_MOVABLE by default.
> If we want the node to be assigned to ZONE_NORMAL,
> we can do it as follows:
> "echo online_kernel > /sys/devices/system/memory/memoryXXX/state"
> 
> By the patch, the behavious of kernel is changed by sysctl,
> user can automatically create movable memory
> by only the following udev rule:
> SUBSYSTEM=="memory", ACTION=="add",
> ATTR{state}=="offline", ATTR{state}="online"

So just to be clear, we are adding a new sysctl, because the existing
movable_node kernel option, which is checked by movable_node_is_enabled(), and
does the same thing for non-hot-added-memory (?) cannot be reused for hot-added
memory, as that would be a potentially surprising behavior change? Correct? Then
this should be mentioned in the changelog too, and wherever "movable_node" is
documented should also mention the new sysctl. Personally, I would expect
movable_node to affect hot-added memory as well, and would be surprised that it
doesn't...

> Signed-off-by: Changsheng Liu <liuchangsheng@inspur.com>
> Signed-off-by: Xiaofeng Yan <yanxiaofeng@inspur.com>
> Tested-by: Dongdong Fan <fandd@inspur.com>
> Cc: Wang Nan <wangnan0@huawei.com>
> Cc: Dave Hansen <dave.hansen@intel.com>
> Cc: Yinghai Lu <yinghai@kernel.org>
> Cc: Tang Chen <tangchen@cn.fujitsu.com>
> Cc: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
> Cc: Toshi Kani <toshi.kani@hp.com>
> Cc: Xishi Qiu <qiuxishi@huawei.com>
> ---
>  Documentation/memory-hotplug.txt |    5 ++++-
>  kernel/sysctl.c                  |   15 +++++++++++++++
>  mm/memory_hotplug.c              |   24 ++++++++++++++++++++++++
>  3 files changed, 43 insertions(+), 1 deletions(-)
> 
> diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt
> index ce2cfcf..7ac7485 100644
> --- a/Documentation/memory-hotplug.txt
> +++ b/Documentation/memory-hotplug.txt
> @@ -277,7 +277,7 @@ And if the memory block is in ZONE_MOVABLE, you can change it to ZONE_NORMAL:
>  After this, memory block XXX's state will be 'online' and the amount of
>  available memory will be increased.
>  
> -Currently, newly added memory is added as ZONE_NORMAL (for powerpc, ZONE_DMA).
> +Currently, newly added memory is added as ZONE_NORMAL or ZONE_MOVABLE (for powerpc, ZONE_DMA).
>  This may be changed in future.
>  
>  
> @@ -319,6 +319,9 @@ creates ZONE_MOVABLE as following.
>    Size of memory not for movable pages (not for offline) is TOTAL - ZZZZ.
>    Size of memory for movable pages (for offline) is ZZZZ.
>  
> +And a sysctl parameter for assigning the hot added memory to ZONE_MOVABLE is
> +supported. If the value of "kernel/hotadd_memory_as_movable" is 1,the hot added
> +memory will be assigned to ZONE_MOVABLE by default.
>  
>  Note: Unfortunately, there is no information to show which memory block belongs
>  to ZONE_MOVABLE. This is TBD.
> diff --git a/kernel/sysctl.c b/kernel/sysctl.c
> index 19b62b5..16b1501 100644
> --- a/kernel/sysctl.c
> +++ b/kernel/sysctl.c
> @@ -166,6 +166,10 @@ extern int unaligned_dump_stack;
>  extern int no_unaligned_warning;
>  #endif
>  
> +#ifdef CONFIG_MOVABLE_NODE
> +extern int hotadd_memory_as_movable;
> +#endif
> +
>  #ifdef CONFIG_PROC_SYSCTL
>  
>  #define SYSCTL_WRITES_LEGACY	-1
> @@ -1139,6 +1143,17 @@ static struct ctl_table kern_table[] = {
>  		.proc_handler	= timer_migration_handler,
>  	},
>  #endif
> +/*If the value of "kernel/hotadd_memory_as_movable" is 1,the hot added
> + * memory will be assigned to ZONE_MOVABLE by default.*/
> +#ifdef CONFIG_MOVABLE_NODE
> +	{
> +		.procname	= "hotadd_memory_as_movable",
> +		.data		= &hotadd_memory_as_movable,
> +		.maxlen		= sizeof(int),
> +		.mode		= 0644,
> +		.proc_handler	= proc_dointvec,
> +	},
> +#endif
>  	{ }
>  };
>  
> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index 26fbba7..eca5512 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -37,6 +37,11 @@
>  
>  #include "internal.h"
>  
> +/*If the global variable value is 1,
> + * the hot added memory will be assigned to ZONE_MOVABLE by default
> + */
> +int hotadd_memory_as_movable;
> +
>  /*
>   * online_page_callback contains pointer to current page onlining function.
>   * Initially it is generic_online_page(). If it is required it could be
> @@ -1190,6 +1195,9 @@ static int check_hotplug_memory_range(u64 start, u64 size)
>  /*
>   * If movable zone has already been setup, newly added memory should be check.
>   * If its address is higher than movable zone, it should be added as movable.
> + * And if system config CONFIG_MOVABLE_NODE and set the sysctl parameter
> + * "hotadd_memory_as_movable" and added memory does not overlap the zone
> + * before MOVABLE_ZONE,the memory will be added as movable.
>   * Without this check, movable zone may overlap with other zone.
>   */
>  static int should_add_memory_movable(int nid, u64 start, u64 size)
> @@ -1197,6 +1205,22 @@ static int should_add_memory_movable(int nid, u64 start, u64 size)
>  	unsigned long start_pfn = start >> PAGE_SHIFT;
>  	pg_data_t *pgdat = NODE_DATA(nid);
>  	struct zone *movable_zone = pgdat->node_zones + ZONE_MOVABLE;
> +	struct zone *pre_zone = pgdat->node_zones + (ZONE_MOVABLE - 1);
> +	/*
> +	 * The system configs CONFIG_MOVABLE_NODE to assign a node
> +	 * which has only movable memory,so the hot-added memory should
> +	 * be assigned to ZONE_MOVABLE by default,
> +	 * but the function zone_for_memory() assign the hot-added memory
> +	 * to ZONE_NORMAL(x86_64) by default.Kernel does not allow to
> +	 * create ZONE_MOVABLE before ZONE_NORMAL,So if the value of
> +	 * sysctl parameter "hotadd_memory_as_movable" is 1
> +	 * and the ZONE_NORMAL is empty or the pfn of the hot-added memory
> +	 * is after the end of ZONE_NORMAL
> +	 * the hot-added memory will be assigned to ZONE_MOVABLE.
> +	 */
> +	if (hotadd_memory_as_movable
> +	&& (zone_is_empty(pre_zone) || zone_end_pfn(pre_zone) <= start_pfn))
> +		return 1;
>  
>  	if (zone_is_empty(movable_zone))
>  		return 0;
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH V7] mm: memory hot-add: memory can not be added to movable zone defaultly

[Changsheng Liu]

a?? 2015/10/15 0:18, Vlastimil Babka a??e??:
> On 10/12/2015 08:58 AM, Changsheng Liu wrote:
>> From: Changsheng Liu <liuchangcheng@inspur.com>
>>
>> After the user config CONFIG_MOVABLE_NODE,
>> When the memory is hot added, should_add_memory_movable() return 0
>> because all zones including ZONE_MOVABLE are empty,
>> so the memory that was hot added will be assigned to ZONE_NORMAL
>> and ZONE_NORMAL will be created firstly.
>> But we want the whole node to be added to ZONE_MOVABLE by default.
>>
>> So we change should_add_memory_movable(): if the user config
>> CONFIG_MOVABLE_NODE and sysctl parameter hotadd_memory_as_movable is 1
>> and the ZONE_NORMAL is empty or the pfn of the hot-added memory
>> is after the end of the ZONE_NORMAL it will always return 1
>> and then the whole node will be added to ZONE_MOVABLE by default.
>> If we want the node to be assigned to ZONE_NORMAL,
>> we can do it as follows:
>> "echo online_kernel > /sys/devices/system/memory/memoryXXX/state"
>>
>> By the patch, the behavious of kernel is changed by sysctl,
>> user can automatically create movable memory
>> by only the following udev rule:
>> SUBSYSTEM=="memory", ACTION=="add",
>> ATTR{state}=="offline", ATTR{state}="online"
     I'm sorry for replying you so late due to the busy business trip.
> So just to be clear, we are adding a new sysctl, because the existing
> movable_node kernel option, which is checked by movable_node_is_enabled(), and
> does the same thing for non-hot-added-memory (?) cannot be reused for hot-added
> memory, as that would be a potentially surprising behavior change? Correct? Then
> this should be mentioned in the changelog too, and wherever "movable_node" is
> documented should also mention the new sysctl. Personally, I would expect
> movable_node to affect hot-added memory as well, and would be surprised that it
> doesn't...
     I think it can let the user decides when to use this feature.
     The user can enable the feature when making the hot_added memory  
of a node movable and
     make the feature disable to assign the hot_added memory of the next 
node to ZONE_NORMAL .
>
>> Signed-off-by: Changsheng Liu <liuchangsheng@inspur.com>
>> Signed-off-by: Xiaofeng Yan <yanxiaofeng@inspur.com>
>> Tested-by: Dongdong Fan <fandd@inspur.com>
>> Cc: Wang Nan <wangnan0@huawei.com>
>> Cc: Dave Hansen <dave.hansen@intel.com>
>> Cc: Yinghai Lu <yinghai@kernel.org>
>> Cc: Tang Chen <tangchen@cn.fujitsu.com>
>> Cc: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
>> Cc: Toshi Kani <toshi.kani@hp.com>
>> Cc: Xishi Qiu <qiuxishi@huawei.com>
>> ---
>>   Documentation/memory-hotplug.txt |    5 ++++-
>>   kernel/sysctl.c                  |   15 +++++++++++++++
>>   mm/memory_hotplug.c              |   24 ++++++++++++++++++++++++
>>   3 files changed, 43 insertions(+), 1 deletions(-)
>>
>> diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt
>> index ce2cfcf..7ac7485 100644
>> --- a/Documentation/memory-hotplug.txt
>> +++ b/Documentation/memory-hotplug.txt
>> @@ -277,7 +277,7 @@ And if the memory block is in ZONE_MOVABLE, you can change it to ZONE_NORMAL:
>>   After this, memory block XXX's state will be 'online' and the amount of
>>   available memory will be increased.
>>   
>> -Currently, newly added memory is added as ZONE_NORMAL (for powerpc, ZONE_DMA).
>> +Currently, newly added memory is added as ZONE_NORMAL or ZONE_MOVABLE (for powerpc, ZONE_DMA).
>>   This may be changed in future.
>>   
>>   
>> @@ -319,6 +319,9 @@ creates ZONE_MOVABLE as following.
>>     Size of memory not for movable pages (not for offline) is TOTAL - ZZZZ.
>>     Size of memory for movable pages (for offline) is ZZZZ.
>>   
>> +And a sysctl parameter for assigning the hot added memory to ZONE_MOVABLE is
>> +supported. If the value of "kernel/hotadd_memory_as_movable" is 1,the hot added
>> +memory will be assigned to ZONE_MOVABLE by default.
>>   
>>   Note: Unfortunately, there is no information to show which memory block belongs
>>   to ZONE_MOVABLE. This is TBD.
>> diff --git a/kernel/sysctl.c b/kernel/sysctl.c
>> index 19b62b5..16b1501 100644
>> --- a/kernel/sysctl.c
>> +++ b/kernel/sysctl.c
>> @@ -166,6 +166,10 @@ extern int unaligned_dump_stack;
>>   extern int no_unaligned_warning;
>>   #endif
>>   
>> +#ifdef CONFIG_MOVABLE_NODE
>> +extern int hotadd_memory_as_movable;
>> +#endif
>> +
>>   #ifdef CONFIG_PROC_SYSCTL
>>   
>>   #define SYSCTL_WRITES_LEGACY	-1
>> @@ -1139,6 +1143,17 @@ static struct ctl_table kern_table[] = {
>>   		.proc_handler	= timer_migration_handler,
>>   	},
>>   #endif
>> +/*If the value of "kernel/hotadd_memory_as_movable" is 1,the hot added
>> + * memory will be assigned to ZONE_MOVABLE by default.*/
>> +#ifdef CONFIG_MOVABLE_NODE
>> +	{
>> +		.procname	= "hotadd_memory_as_movable",
>> +		.data		= &hotadd_memory_as_movable,
>> +		.maxlen		= sizeof(int),
>> +		.mode		= 0644,
>> +		.proc_handler	= proc_dointvec,
>> +	},
>> +#endif
>>   	{ }
>>   };
>>   
>> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
>> index 26fbba7..eca5512 100644
>> --- a/mm/memory_hotplug.c
>> +++ b/mm/memory_hotplug.c
>> @@ -37,6 +37,11 @@
>>   
>>   #include "internal.h"
>>   
>> +/*If the global variable value is 1,
>> + * the hot added memory will be assigned to ZONE_MOVABLE by default
>> + */
>> +int hotadd_memory_as_movable;
>> +
>>   /*
>>    * online_page_callback contains pointer to current page onlining function.
>>    * Initially it is generic_online_page(). If it is required it could be
>> @@ -1190,6 +1195,9 @@ static int check_hotplug_memory_range(u64 start, u64 size)
>>   /*
>>    * If movable zone has already been setup, newly added memory should be check.
>>    * If its address is higher than movable zone, it should be added as movable.
>> + * And if system config CONFIG_MOVABLE_NODE and set the sysctl parameter
>> + * "hotadd_memory_as_movable" and added memory does not overlap the zone
>> + * before MOVABLE_ZONE,the memory will be added as movable.
>>    * Without this check, movable zone may overlap with other zone.
>>    */
>>   static int should_add_memory_movable(int nid, u64 start, u64 size)
>> @@ -1197,6 +1205,22 @@ static int should_add_memory_movable(int nid, u64 start, u64 size)
>>   	unsigned long start_pfn = start >> PAGE_SHIFT;
>>   	pg_data_t *pgdat = NODE_DATA(nid);
>>   	struct zone *movable_zone = pgdat->node_zones + ZONE_MOVABLE;
>> +	struct zone *pre_zone = pgdat->node_zones + (ZONE_MOVABLE - 1);
>> +	/*
>> +	 * The system configs CONFIG_MOVABLE_NODE to assign a node
>> +	 * which has only movable memory,so the hot-added memory should
>> +	 * be assigned to ZONE_MOVABLE by default,
>> +	 * but the function zone_for_memory() assign the hot-added memory
>> +	 * to ZONE_NORMAL(x86_64) by default.Kernel does not allow to
>> +	 * create ZONE_MOVABLE before ZONE_NORMAL,So if the value of
>> +	 * sysctl parameter "hotadd_memory_as_movable" is 1
>> +	 * and the ZONE_NORMAL is empty or the pfn of the hot-added memory
>> +	 * is after the end of ZONE_NORMAL
>> +	 * the hot-added memory will be assigned to ZONE_MOVABLE.
>> +	 */
>> +	if (hotadd_memory_as_movable
>> +	&& (zone_is_empty(pre_zone) || zone_end_pfn(pre_zone) <= start_pfn))
>> +		return 1;
>>   
>>   	if (zone_is_empty(movable_zone))
>>   		return 0;
>>
> .
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH V7] mm: memory hot-add: memory can not be added to movable zone defaultly

[Vlastimil Babka]

On 10/21/2015 04:20 AM, Changsheng Liu wrote:
>
>
> a?? 2015/10/15 0:18, Vlastimil Babka a??e??:
>> On 10/12/2015 08:58 AM, Changsheng Liu wrote:
>>> From: Changsheng Liu <liuchangcheng@inspur.com>
>>>
>>> After the user config CONFIG_MOVABLE_NODE,
>>> When the memory is hot added, should_add_memory_movable() return 0
>>> because all zones including ZONE_MOVABLE are empty,
>>> so the memory that was hot added will be assigned to ZONE_NORMAL
>>> and ZONE_NORMAL will be created firstly.
>>> But we want the whole node to be added to ZONE_MOVABLE by default.
>>>
>>> So we change should_add_memory_movable(): if the user config
>>> CONFIG_MOVABLE_NODE and sysctl parameter hotadd_memory_as_movable is 1
>>> and the ZONE_NORMAL is empty or the pfn of the hot-added memory
>>> is after the end of the ZONE_NORMAL it will always return 1
>>> and then the whole node will be added to ZONE_MOVABLE by default.
>>> If we want the node to be assigned to ZONE_NORMAL,
>>> we can do it as follows:
>>> "echo online_kernel > /sys/devices/system/memory/memoryXXX/state"
>>>
>>> By the patch, the behavious of kernel is changed by sysctl,
>>> user can automatically create movable memory
>>> by only the following udev rule:
>>> SUBSYSTEM=="memory", ACTION=="add",
>>> ATTR{state}=="offline", ATTR{state}="online"
>       I'm sorry for replying you so late due to the busy business trip.
>> So just to be clear, we are adding a new sysctl, because the existing
>> movable_node kernel option, which is checked by movable_node_is_enabled(), and
>> does the same thing for non-hot-added-memory (?) cannot be reused for hot-added
>> memory, as that would be a potentially surprising behavior change? Correct? Then
>> this should be mentioned in the changelog too, and wherever "movable_node" is
>> documented should also mention the new sysctl. Personally, I would expect
>> movable_node to affect hot-added memory as well, and would be surprised that it
>> doesn't...
>       I think it can let the user decides when to use this feature.
>       The user can enable the feature when making the hot_added memory
> of a node movable and
>       make the feature disable to assign the hot_added memory of the next
> node to ZONE_NORMAL .

So you mean sysctl is more flexible than boot option. OK, but wasn't 
such flexibility already provided by "echo online_kernel" vs "echo 
online_movable"? It doesn't sound like a strong reason for a new sysctl? 
Not doing surprising behavior change maybe does...

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH V7] mm: memory hot-add: memory can not be added to movable zone defaultly

[Tang Chen]

On 10/21/2015 05:00 PM, Vlastimil Babka wrote:
> On 10/21/2015 04:20 AM, Changsheng Liu wrote:
>>
>>
>> 在 2015/10/15 0:18, Vlastimil Babka 写道:
>>> On 10/12/2015 08:58 AM, Changsheng Liu wrote:
>>>> From: Changsheng Liu <liuchangcheng@inspur.com>
>>>>
>>>> After the user config CONFIG_MOVABLE_NODE,
>>>> When the memory is hot added, should_add_memory_movable() return 0
>>>> because all zones including ZONE_MOVABLE are empty,
>>>> so the memory that was hot added will be assigned to ZONE_NORMAL
>>>> and ZONE_NORMAL will be created firstly.
>>>> But we want the whole node to be added to ZONE_MOVABLE by default.
>>>>
>>>> So we change should_add_memory_movable(): if the user config
>>>> CONFIG_MOVABLE_NODE and sysctl parameter hotadd_memory_as_movable is 1
>>>> and the ZONE_NORMAL is empty or the pfn of the hot-added memory
>>>> is after the end of the ZONE_NORMAL it will always return 1
>>>> and then the whole node will be added to ZONE_MOVABLE by default.
>>>> If we want the node to be assigned to ZONE_NORMAL,
>>>> we can do it as follows:
>>>> "echo online_kernel > /sys/devices/system/memory/memoryXXX/state"
>>>>
>>>> By the patch, the behavious of kernel is changed by sysctl,
>>>> user can automatically create movable memory
>>>> by only the following udev rule:
>>>> SUBSYSTEM=="memory", ACTION=="add",
>>>> ATTR{state}=="offline", ATTR{state}="online"
>>       I'm sorry for replying you so late due to the busy business trip.
>>> So just to be clear, we are adding a new sysctl, because the existing
>>> movable_node kernel option, which is checked by 
>>> movable_node_is_enabled(), and
>>> does the same thing for non-hot-added-memory (?) cannot be reused 
>>> for hot-added
>>> memory, as that would be a potentially surprising behavior change? 
>>> Correct? Then
>>> this should be mentioned in the changelog too, and wherever 
>>> "movable_node" is
>>> documented should also mention the new sysctl. Personally, I would 
>>> expect
>>> movable_node to affect hot-added memory as well, and would be 
>>> surprised that it
>>> doesn't...
>>       I think it can let the user decides when to use this feature.
>>       The user can enable the feature when making the hot_added memory
>> of a node movable and
>>       make the feature disable to assign the hot_added memory of the 
>> next
>> node to ZONE_NORMAL .
>
> So you mean sysctl is more flexible than boot option. OK, but wasn't 
> such flexibility already provided by "echo online_kernel" vs "echo 
> online_movable"? It doesn't sound like a strong reason for a new 
> sysctl? Not doing surprising behavior change maybe does...
> .
>
Hi Vlastimil,

The current kernel will add hot-added memory to ZONE_NORMAL by default. 
If users use a udev rule as below:

SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"

it will online the memory as normal memory, which will not be hotpluggable.

Please refer to: https://lkml.org/lkml/2015/10/9/58

I think this is the root motivation of the patch.

But BTW, I'm quite familiar with udev rules, but can something like this 
work ?

SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online_movable"

I'm not sure. I added Ishimatu in.

For now, I think, if the above rule works, we don't need this patch. If 
not, maybe we should just change the kernel behavior to make the 
hot-added memory be added to ZONE_MOVABLE by default.

I don't have objection. But a sysctl doesn't sound necessary.

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH V7] mm: memory hot-add: memory can not be added to movable zone defaultly

[Tang Chen]

On 10/21/2015 05:18 PM, Tang Chen wrote:
>
> On 10/21/2015 05:00 PM, Vlastimil Babka wrote:
>> On 10/21/2015 04:20 AM, Changsheng Liu wrote:
>>>
>>>
>>> 在 2015/10/15 0:18, Vlastimil Babka 写道:
>>>> On 10/12/2015 08:58 AM, Changsheng Liu wrote:
>>>>> From: Changsheng Liu <liuchangcheng@inspur.com>
>>>>>
>>>>> After the user config CONFIG_MOVABLE_NODE,
>>>>> When the memory is hot added, should_add_memory_movable() return 0
>>>>> because all zones including ZONE_MOVABLE are empty,
>>>>> so the memory that was hot added will be assigned to ZONE_NORMAL
>>>>> and ZONE_NORMAL will be created firstly.
>>>>> But we want the whole node to be added to ZONE_MOVABLE by default.
>>>>>
>>>>> So we change should_add_memory_movable(): if the user config
>>>>> CONFIG_MOVABLE_NODE and sysctl parameter hotadd_memory_as_movable 
>>>>> is 1
>>>>> and the ZONE_NORMAL is empty or the pfn of the hot-added memory
>>>>> is after the end of the ZONE_NORMAL it will always return 1
>>>>> and then the whole node will be added to ZONE_MOVABLE by default.
>>>>> If we want the node to be assigned to ZONE_NORMAL,
>>>>> we can do it as follows:
>>>>> "echo online_kernel > /sys/devices/system/memory/memoryXXX/state"
>>>>>
>>>>> By the patch, the behavious of kernel is changed by sysctl,
>>>>> user can automatically create movable memory
>>>>> by only the following udev rule:
>>>>> SUBSYSTEM=="memory", ACTION=="add",
>>>>> ATTR{state}=="offline", ATTR{state}="online"
>>>       I'm sorry for replying you so late due to the busy business trip.
>>>> So just to be clear, we are adding a new sysctl, because the existing
>>>> movable_node kernel option, which is checked by 
>>>> movable_node_is_enabled(), and
>>>> does the same thing for non-hot-added-memory (?) cannot be reused 
>>>> for hot-added
>>>> memory, as that would be a potentially surprising behavior change? 
>>>> Correct? Then
>>>> this should be mentioned in the changelog too, and wherever 
>>>> "movable_node" is
>>>> documented should also mention the new sysctl. Personally, I would 
>>>> expect
>>>> movable_node to affect hot-added memory as well, and would be 
>>>> surprised that it
>>>> doesn't...
>>>       I think it can let the user decides when to use this feature.
>>>       The user can enable the feature when making the hot_added memory
>>> of a node movable and
>>>       make the feature disable to assign the hot_added memory of the 
>>> next
>>> node to ZONE_NORMAL .
>>
>> So you mean sysctl is more flexible than boot option. OK, but wasn't 
>> such flexibility already provided by "echo online_kernel" vs "echo 
>> online_movable"? It doesn't sound like a strong reason for a new 
>> sysctl? Not doing surprising behavior change maybe does...
>> .
>>
> Hi Vlastimil,
>
> The current kernel will add hot-added memory to ZONE_NORMAL by 
> default. If users use a udev rule as below:
>
> SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", 
> ATTR{state}="online"
>
> it will online the memory as normal memory, which will not be 
> hotpluggable.
>
> Please refer to: https://lkml.org/lkml/2015/10/9/58
>
> I think this is the root motivation of the patch.
>
> But BTW, I'm quite familiar with udev rules, but can something like 
> this work ?

sorry, I'm NOT quite familiar with udev rules, ......

>
> SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", 
> ATTR{state}="online_movable"
>
> I'm not sure. I added Ishimatu in.
>
> For now, I think, if the above rule works, we don't need this patch. 
> If not, maybe we should just change the kernel behavior to make the 
> hot-added memory be added to ZONE_MOVABLE by default.
>
> I don't have objection. But a sysctl doesn't sound necessary.
>
> Thanks.
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe 
> linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> .
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH V7] mm: memory hot-add: memory can not be added to movable zone defaultly

[Yasuaki Ishimatsu]

On Wed, 21 Oct 2015 17:18:36 +0800
Tang Chen <tangchen@cn.fujitsu.com> wrote:

> 
> On 10/21/2015 05:00 PM, Vlastimil Babka wrote:
> > On 10/21/2015 04:20 AM, Changsheng Liu wrote:
> >>
> >>
> >> a?? 2015/10/15 0:18, Vlastimil Babka a??e??:
> >>> On 10/12/2015 08:58 AM, Changsheng Liu wrote:
> >>>> From: Changsheng Liu <liuchangcheng@inspur.com>
> >>>>
> >>>> After the user config CONFIG_MOVABLE_NODE,
> >>>> When the memory is hot added, should_add_memory_movable() return 0
> >>>> because all zones including ZONE_MOVABLE are empty,
> >>>> so the memory that was hot added will be assigned to ZONE_NORMAL
> >>>> and ZONE_NORMAL will be created firstly.
> >>>> But we want the whole node to be added to ZONE_MOVABLE by default.
> >>>>
> >>>> So we change should_add_memory_movable(): if the user config
> >>>> CONFIG_MOVABLE_NODE and sysctl parameter hotadd_memory_as_movable is 1
> >>>> and the ZONE_NORMAL is empty or the pfn of the hot-added memory
> >>>> is after the end of the ZONE_NORMAL it will always return 1
> >>>> and then the whole node will be added to ZONE_MOVABLE by default.
> >>>> If we want the node to be assigned to ZONE_NORMAL,
> >>>> we can do it as follows:
> >>>> "echo online_kernel > /sys/devices/system/memory/memoryXXX/state"
> >>>>
> >>>> By the patch, the behavious of kernel is changed by sysctl,
> >>>> user can automatically create movable memory
> >>>> by only the following udev rule:
> >>>> SUBSYSTEM=="memory", ACTION=="add",
> >>>> ATTR{state}=="offline", ATTR{state}="online"
> >>       I'm sorry for replying you so late due to the busy business trip.
> >>> So just to be clear, we are adding a new sysctl, because the existing
> >>> movable_node kernel option, which is checked by 
> >>> movable_node_is_enabled(), and
> >>> does the same thing for non-hot-added-memory (?) cannot be reused 
> >>> for hot-added
> >>> memory, as that would be a potentially surprising behavior change? 
> >>> Correct? Then
> >>> this should be mentioned in the changelog too, and wherever 
> >>> "movable_node" is
> >>> documented should also mention the new sysctl. Personally, I would 
> >>> expect
> >>> movable_node to affect hot-added memory as well, and would be 
> >>> surprised that it
> >>> doesn't...
> >>       I think it can let the user decides when to use this feature.
> >>       The user can enable the feature when making the hot_added memory
> >> of a node movable and
> >>       make the feature disable to assign the hot_added memory of the 
> >> next
> >> node to ZONE_NORMAL .
> >
> > So you mean sysctl is more flexible than boot option. OK, but wasn't 
> > such flexibility already provided by "echo online_kernel" vs "echo 
> > online_movable"? It doesn't sound like a strong reason for a new 
> > sysctl? Not doing surprising behavior change maybe does...
> > .
> >
> Hi Vlastimil,
> 
> The current kernel will add hot-added memory to ZONE_NORMAL by default. 
> If users use a udev rule as below:
> 
> SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"
> 
> it will online the memory as normal memory, which will not be hotpluggable.
> 
> Please refer to: https://lkml.org/lkml/2015/10/9/58
> 
> I think this is the root motivation of the patch.
> 

> But BTW, I'm quite familiar with udev rules, but can something like this 
> work ?
> 
> SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online_movable"
> 
> I'm not sure. I added Ishimatu in.

I think the udev rules fails to online memory as movable.

When hot adding memory, the memory is managed as ZONE_NORMAL.
And add events of memory section are notified to udev in ascending
order, like 0->1->2->3. Thus udev starts to online memory from section 0.
But to change zone from ZONE_NORMAL to ZONE_MOVALBE, udev onlines memory
in descending order, like 3->2->1->0. So the udev rules cannot online
memory as movable.

Thanks,
Yasuaki Ishimatsu

> 
> For now, I think, if the above rule works, we don't need this patch. If 
> not, maybe we should just change the kernel behavior to make the 
> hot-added memory be added to ZONE_MOVABLE by default.
> 
> I don't have objection. But a sysctl doesn't sound necessary.



> 
> Thanks.
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>