From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wm0-f71.google.com (mail-wm0-f71.google.com [74.125.82.71])
	by kanga.kvack.org (Postfix) with ESMTP id DC97A6B0292
	for <linux-mm@kvack.org>; Thu, 29 Jun 2017 03:35:20 -0400 (EDT)
Received: by mail-wm0-f71.google.com with SMTP id 21so756498wmt.15
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 00:35:20 -0700 (PDT)
Received: from mail-wr0-f196.google.com (mail-wr0-f196.google.com. [209.85.128.196])
        by mx.google.com with ESMTPS id 62si509476wmc.60.2017.06.29.00.35.19
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 29 Jun 2017 00:35:19 -0700 (PDT)
Received: by mail-wr0-f196.google.com with SMTP id x23so36072663wrb.0
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 00:35:19 -0700 (PDT)
From: Michal Hocko <mhocko@kernel.org>
Subject: [RFC PATCH 0/2] mm, memory_hotplug: remove zone onlining restriction
Date: Thu, 29 Jun 2017 09:35:07 +0200
Message-Id: <20170629073509.623-1-mhocko@kernel.org>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>

Hi,
I am sending this as an RFC because this hasn't seen a lot of testing
yet but I would like to see whether the semantic I came up with (see
patch 2) is sensible. This work should help Joonsoo with his CMA zone
based approach when reusing MOVABLE zone. I think it will also help to
remove more code from the memory hotplug (e.g. zone shrinking).

Patch 1 restores original memoryXY/valid_zones semantic wrt zone
ordering. This can be merged without patch 2 which removes the zone
overlap restriction and defines a semantic for the default onlining. See
more in the patch.

Questions, concerns, objections?

Shortlog
Michal Hocko (2):
      mm, memory_hotplug: display allowed zones in the preferred ordering
      mm, memory_hotplug: remove zone restrictions

Diffstat
 drivers/base/memory.c          | 30 ++++++++++-----
 include/linux/memory_hotplug.h |  2 +-
 mm/memory_hotplug.c            | 87 +++++++++++++++++-------------------------
 3 files changed, 55 insertions(+), 64 deletions(-)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wm0-f70.google.com (mail-wm0-f70.google.com [74.125.82.70])
	by kanga.kvack.org (Postfix) with ESMTP id 3E2DF6B0292
	for <linux-mm@kvack.org>; Thu, 29 Jun 2017 03:35:22 -0400 (EDT)
Received: by mail-wm0-f70.google.com with SMTP id 12so785851wmn.1
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 00:35:22 -0700 (PDT)
Received: from mail-wr0-f196.google.com (mail-wr0-f196.google.com. [209.85.128.196])
        by mx.google.com with ESMTPS id 201si7311531wms.7.2017.06.29.00.35.20
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 29 Jun 2017 00:35:20 -0700 (PDT)
Received: by mail-wr0-f196.google.com with SMTP id z45so36136679wrb.2
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 00:35:20 -0700 (PDT)
From: Michal Hocko <mhocko@kernel.org>
Subject: [PATCH 1/2] mm, memory_hotplug: display allowed zones in the preferred ordering
Date: Thu, 29 Jun 2017 09:35:08 +0200
Message-Id: <20170629073509.623-2-mhocko@kernel.org>
In-Reply-To: <20170629073509.623-1-mhocko@kernel.org>
References: <20170629073509.623-1-mhocko@kernel.org>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Michal Hocko <mhocko@suse.com>

From: Michal Hocko <mhocko@suse.com>

Prior to "mm, memory_hotplug: do not associate hotadded memory to zones
until online" we used to allow to change the valid zone types of a
memory block if it is adjacent to a different zone type. This fact was
reflected in memoryNN/valid_zones by the ordering of printed zones.
The first one was default (echo online > memoryNN/state) and the other
one could be onlined explicitly by online_{movable,kernel}. This
behavior was removed by the said patch and as such the ordering was
not all that important. In most cases a kernel zone would be default
anyway. The only exception is movable_node handled by "mm,
memory_hotplug: support movable_node for hotpluggable nodes".

Let's reintroduce this behavior again because later patch will remove
the zone overlap restriction and so user will be allowed to online
kernel resp. movable block regardless of its placement. Original
behavior will then become significant again because it would be
non-trivial for users to see what is the default zone to online into.

Implementation is really simple. Pull out zone selection out of
move_pfn_range into zone_for_pfn_range helper and use it in
show_valid_zones to display the zone for default onlining and then
both kernel and movable if they are allowed. Default online zone is not
duplicated.

Signed-off-by: Michal Hocko <mhocko@suse.com>

fold me "mm, memory_hotplug: display allowed zones in the preferred ordering"
---
 drivers/base/memory.c          | 33 +++++++++++++------
 include/linux/memory_hotplug.h |  2 +-
 mm/memory_hotplug.c            | 73 ++++++++++++++++++++++++------------------
 3 files changed, 65 insertions(+), 43 deletions(-)

diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index c7c4e0325cdb..26383af9900c 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -388,6 +388,22 @@ static ssize_t show_phys_device(struct device *dev,
 }
 
 #ifdef CONFIG_MEMORY_HOTREMOVE
+static void print_allowed_zone(char *buf, int nid, unsigned long start_pfn,
+		unsigned long nr_pages, int online_type,
+		struct zone *default_zone)
+{
+	struct zone *zone;
+
+	if (!allow_online_pfn_range(nid, start_pfn, nr_pages, online_type))
+		return;
+
+	zone = zone_for_pfn_range(online_type, nid, start_pfn, nr_pages);
+	if (zone != default_zone) {
+		strcat(buf, " ");
+		strcat(buf, zone->name);
+	}
+}
+
 static ssize_t show_valid_zones(struct device *dev,
 				struct device_attribute *attr, char *buf)
 {
@@ -395,7 +411,7 @@ static ssize_t show_valid_zones(struct device *dev,
 	unsigned long start_pfn = section_nr_to_pfn(mem->start_section_nr);
 	unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
 	unsigned long valid_start_pfn, valid_end_pfn;
-	bool append = false;
+	struct zone *default_zone;
 	int nid;
 
 	/*
@@ -418,16 +434,13 @@ static ssize_t show_valid_zones(struct device *dev,
 	}
 
 	nid = pfn_to_nid(start_pfn);
-	if (allow_online_pfn_range(nid, start_pfn, nr_pages, MMOP_ONLINE_KERNEL)) {
-		strcat(buf, default_zone_for_pfn(nid, start_pfn, nr_pages)->name);
-		append = true;
-	}
+	default_zone = zone_for_pfn_range(MMOP_ONLINE_KEEP, nid, start_pfn, nr_pages);
+	strcat(buf, default_zone->name);
 
-	if (allow_online_pfn_range(nid, start_pfn, nr_pages, MMOP_ONLINE_MOVABLE)) {
-		if (append)
-			strcat(buf, " ");
-		strcat(buf, NODE_DATA(nid)->node_zones[ZONE_MOVABLE].name);
-	}
+	print_allowed_zone(buf, nid, start_pfn, nr_pages, MMOP_ONLINE_KERNEL,
+			default_zone);
+	print_allowed_zone(buf, nid, start_pfn, nr_pages, MMOP_ONLINE_MOVABLE,
+			default_zone);
 out:
 	strcat(buf, "\n");
 
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index c8a5056a5ae0..5e6e4cc36ff4 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -319,6 +319,6 @@ extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map,
 					  unsigned long pnum);
 extern bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages,
 		int online_type);
-extern struct zone *default_zone_for_pfn(int nid, unsigned long pfn,
+extern struct zone *zone_for_pfn_range(int online_type, int nid, unsigned start_pfn,
 		unsigned long nr_pages);
 #endif /* __LINUX_MEMORY_HOTPLUG_H */
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index b4015a39d108..6b9a60115e37 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -836,31 +836,6 @@ static void node_states_set_node(int node, struct memory_notify *arg)
 	node_set_state(node, N_MEMORY);
 }
 
-bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages, int online_type)
-{
-	struct pglist_data *pgdat = NODE_DATA(nid);
-	struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
-	struct zone *default_zone = default_zone_for_pfn(nid, pfn, nr_pages);
-
-	/*
-	 * TODO there shouldn't be any inherent reason to have ZONE_NORMAL
-	 * physically before ZONE_MOVABLE. All we need is they do not
-	 * overlap. Historically we didn't allow ZONE_NORMAL after ZONE_MOVABLE
-	 * though so let's stick with it for simplicity for now.
-	 * TODO make sure we do not overlap with ZONE_DEVICE
-	 */
-	if (online_type == MMOP_ONLINE_KERNEL) {
-		if (zone_is_empty(movable_zone))
-			return true;
-		return movable_zone->zone_start_pfn >= pfn + nr_pages;
-	} else if (online_type == MMOP_ONLINE_MOVABLE) {
-		return zone_end_pfn(default_zone) <= pfn;
-	}
-
-	/* MMOP_ONLINE_KEEP will always succeed and inherits the current zone */
-	return online_type == MMOP_ONLINE_KEEP;
-}
-
 static void __meminit resize_zone_range(struct zone *zone, unsigned long start_pfn,
 		unsigned long nr_pages)
 {
@@ -919,7 +894,7 @@ void __ref move_pfn_range_to_zone(struct zone *zone,
  * If no kernel zone covers this pfn range it will automatically go
  * to the ZONE_NORMAL.
  */
-struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
+static struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
 		unsigned long nr_pages)
 {
 	struct pglist_data *pgdat = NODE_DATA(nid);
@@ -935,6 +910,31 @@ struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
 	return &pgdat->node_zones[ZONE_NORMAL];
 }
 
+bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages, int online_type)
+{
+	struct pglist_data *pgdat = NODE_DATA(nid);
+	struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
+	struct zone *default_zone = default_zone_for_pfn(nid, pfn, nr_pages);
+
+	/*
+	 * TODO there shouldn't be any inherent reason to have ZONE_NORMAL
+	 * physically before ZONE_MOVABLE. All we need is they do not
+	 * overlap. Historically we didn't allow ZONE_NORMAL after ZONE_MOVABLE
+	 * though so let's stick with it for simplicity for now.
+	 * TODO make sure we do not overlap with ZONE_DEVICE
+	 */
+	if (online_type == MMOP_ONLINE_KERNEL) {
+		if (zone_is_empty(movable_zone))
+			return true;
+		return movable_zone->zone_start_pfn >= pfn + nr_pages;
+	} else if (online_type == MMOP_ONLINE_MOVABLE) {
+		return zone_end_pfn(default_zone) <= pfn;
+	}
+
+	/* MMOP_ONLINE_KEEP will always succeed and inherits the current zone */
+	return online_type == MMOP_ONLINE_KEEP;
+}
+
 static inline bool movable_pfn_range(int nid, struct zone *default_zone,
 		unsigned long start_pfn, unsigned long nr_pages)
 {
@@ -948,12 +948,8 @@ static inline bool movable_pfn_range(int nid, struct zone *default_zone,
 	return !zone_intersects(default_zone, start_pfn, nr_pages);
 }
 
-/*
- * Associates the given pfn range with the given node and the zone appropriate
- * for the given online type.
- */
-static struct zone * __meminit move_pfn_range(int online_type, int nid,
-		unsigned long start_pfn, unsigned long nr_pages)
+struct zone * zone_for_pfn_range(int online_type, int nid, unsigned start_pfn,
+		unsigned long nr_pages)
 {
 	struct pglist_data *pgdat = NODE_DATA(nid);
 	struct zone *zone = default_zone_for_pfn(nid, start_pfn, nr_pages);
@@ -972,6 +968,19 @@ static struct zone * __meminit move_pfn_range(int online_type, int nid,
 		zone = &pgdat->node_zones[ZONE_MOVABLE];
 	}
 
+	return zone;
+}
+
+/*
+ * Associates the given pfn range with the given node and the zone appropriate
+ * for the given online type.
+ */
+static struct zone * __meminit move_pfn_range(int online_type, int nid,
+		unsigned long start_pfn, unsigned long nr_pages)
+{
+	struct zone *zone;
+
+	zone = zone_for_pfn_range(online_type, nid, start_pfn, nr_pages);
 	move_pfn_range_to_zone(zone, start_pfn, nr_pages);
 	return zone;
 }
-- 
2.11.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f200.google.com (mail-wr0-f200.google.com [209.85.128.200])
	by kanga.kvack.org (Postfix) with ESMTP id 651E2280300
	for <linux-mm@kvack.org>; Thu, 29 Jun 2017 03:35:23 -0400 (EDT)
Received: by mail-wr0-f200.google.com with SMTP id v88so35273766wrb.1
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 00:35:23 -0700 (PDT)
Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com. [74.125.82.67])
        by mx.google.com with ESMTPS id r15si3214352wrr.99.2017.06.29.00.35.21
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 29 Jun 2017 00:35:21 -0700 (PDT)
Received: by mail-wm0-f67.google.com with SMTP id u23so843158wma.2
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 00:35:21 -0700 (PDT)
From: Michal Hocko <mhocko@kernel.org>
Subject: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Date: Thu, 29 Jun 2017 09:35:09 +0200
Message-Id: <20170629073509.623-3-mhocko@kernel.org>
In-Reply-To: <20170629073509.623-1-mhocko@kernel.org>
References: <20170629073509.623-1-mhocko@kernel.org>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Michal Hocko <mhocko@suse.com>

From: Michal Hocko <mhocko@suse.com>

Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
to precede the Movable zone in the physical memory range. The purpose of
the movable zone is, however, not bound to any physical memory restriction.
It merely defines a class of migrateable and reclaimable memory.

There are users (e.g. CMA) who might want to reserve specific physical
memory ranges for their own purpose. Moreover our pfn walkers have to be
prepared for zones overlapping in the physical range already because we
do support interleaving NUMA nodes and therefore zones can interleave as
well. This means we can allow each memory block to be associated with a
different zone.

Loosen the current onlining semantic and allow explicit onlining type on
any memblock. That means that online_{kernel,movable} will be allowed
regardless of the physical address of the memblock as long as it is
offline of course. This might result in moveble zone overlapping with
other kernel zones. Default onlining then becomes a bit tricky but still
sensible. echo online > memoryXY/state will online the given block to
	1) the default zone if the given range is outside of any zone
	2) the enclosing zone if such a zone doesn't interleave with
	   any other zone
        3) the default zone if more zones interleave for this range
where default zone is movable zone only if movable_node is enabled
otherwise it is a kernel zone.

Here is an example of the semantic with (movable_node is not present but
it work in an analogous way). We start with following memblocks, all of
them offline
memory34/valid_zones:Normal Movable
memory35/valid_zones:Normal Movable
memory36/valid_zones:Normal Movable
memory37/valid_zones:Normal Movable
memory38/valid_zones:Normal Movable
memory39/valid_zones:Normal Movable
memory40/valid_zones:Normal Movable
memory41/valid_zones:Normal Movable

Now, we online block 34 in default mode and block 37 as movable
root@test1:/sys/devices/system/node/node1# echo online > memory34/state
root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
memory34/valid_zones:Normal
memory35/valid_zones:Normal Movable
memory36/valid_zones:Normal Movable
memory37/valid_zones:Movable
memory38/valid_zones:Normal Movable
memory39/valid_zones:Normal Movable
memory40/valid_zones:Normal Movable
memory41/valid_zones:Normal Movable

As we can see all other blocks can still be onlined both into Normal and
Movable zones and the Normal is default because the Movable zone spans
only block37 now.
root@test1:/sys/devices/system/node/node1# echo online_movable > memory41/state
memory34/valid_zones:Normal
memory35/valid_zones:Normal Movable
memory36/valid_zones:Normal Movable
memory37/valid_zones:Movable
memory38/valid_zones:Movable Normal
memory39/valid_zones:Movable Normal
memory40/valid_zones:Movable Normal
memory41/valid_zones:Movable

Now the default zone for blocks 37-41 has changed because movable zone
spans that range.
root@test1:/sys/devices/system/node/node1# echo online_kernel > memory39/state
memory34/valid_zones:Normal
memory35/valid_zones:Normal Movable
memory36/valid_zones:Normal Movable
memory37/valid_zones:Movable
memory38/valid_zones:Normal Movable
memory39/valid_zones:Normal
memory40/valid_zones:Movable Normal
memory41/valid_zones:Movable

Note that the block 39 now belongs to the zone Normal and so block38
falls into Normal by default as well.

For completness
root@test1:/sys/devices/system/node/node1# for i in memory[34]?
do
	echo online > $i/state 2>/dev/null
done

memory34/valid_zones:Normal
memory35/valid_zones:Normal
memory36/valid_zones:Normal
memory37/valid_zones:Movable
memory38/valid_zones:Normal
memory39/valid_zones:Normal
memory40/valid_zones:Movable
memory41/valid_zones:Movable

Implementation wise the change is quite straightforward. We can get rid
of allow_online_pfn_range altogether. online_pages allows only offline
nodes already. The original default_zone_for_pfn will become
default_kernel_zone_for_pfn. New default_zone_for_pfn implements the
above semantic. zone_for_pfn_range is slightly reorganized to implement
kernel and movable online type explicitly and MMOP_ONLINE_KEEP becomes
a catch all default behavior.

Signed-off-by: Michal Hocko <mhocko@suse.com>
---
 drivers/base/memory.c |  3 ---
 mm/memory_hotplug.c   | 74 ++++++++++++++++-----------------------------------
 2 files changed, 23 insertions(+), 54 deletions(-)

diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 26383af9900c..4e3b61cda520 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -394,9 +394,6 @@ static void print_allowed_zone(char *buf, int nid, unsigned long start_pfn,
 {
 	struct zone *zone;
 
-	if (!allow_online_pfn_range(nid, start_pfn, nr_pages, online_type))
-		return;
-
 	zone = zone_for_pfn_range(online_type, nid, start_pfn, nr_pages);
 	if (zone != default_zone) {
 		strcat(buf, " ");
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 6b9a60115e37..670f7acbecf4 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -894,7 +894,7 @@ void __ref move_pfn_range_to_zone(struct zone *zone,
  * If no kernel zone covers this pfn range it will automatically go
  * to the ZONE_NORMAL.
  */
-static struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
+static struct zone *default_kernel_zone_for_pfn(int nid, unsigned long start_pfn,
 		unsigned long nr_pages)
 {
 	struct pglist_data *pgdat = NODE_DATA(nid);
@@ -910,65 +910,40 @@ static struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
 	return &pgdat->node_zones[ZONE_NORMAL];
 }
 
-bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages, int online_type)
+static inline struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
+		unsigned long nr_pages)
 {
-	struct pglist_data *pgdat = NODE_DATA(nid);
-	struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
-	struct zone *default_zone = default_zone_for_pfn(nid, pfn, nr_pages);
+	struct zone *kernel_zone = default_kernel_zone_for_pfn(nid, start_pfn,
+			nr_pages);
+	struct zone *movable_zone = &NODE_DATA(nid)->node_zones[ZONE_MOVABLE];
+	bool in_kernel = zone_intersects(kernel_zone, start_pfn, nr_pages);
+	bool in_movable = zone_intersects(movable_zone, start_pfn, nr_pages);
 
 	/*
-	 * TODO there shouldn't be any inherent reason to have ZONE_NORMAL
-	 * physically before ZONE_MOVABLE. All we need is they do not
-	 * overlap. Historically we didn't allow ZONE_NORMAL after ZONE_MOVABLE
-	 * though so let's stick with it for simplicity for now.
-	 * TODO make sure we do not overlap with ZONE_DEVICE
+	 * We inherit the existing zone in a simple case where zones do not
+	 * overlap in the given range
 	 */
-	if (online_type == MMOP_ONLINE_KERNEL) {
-		if (zone_is_empty(movable_zone))
-			return true;
-		return movable_zone->zone_start_pfn >= pfn + nr_pages;
-	} else if (online_type == MMOP_ONLINE_MOVABLE) {
-		return zone_end_pfn(default_zone) <= pfn;
-	}
-
-	/* MMOP_ONLINE_KEEP will always succeed and inherits the current zone */
-	return online_type == MMOP_ONLINE_KEEP;
-}
-
-static inline bool movable_pfn_range(int nid, struct zone *default_zone,
-		unsigned long start_pfn, unsigned long nr_pages)
-{
-	if (!allow_online_pfn_range(nid, start_pfn, nr_pages,
-				MMOP_ONLINE_KERNEL))
-		return true;
-
-	if (!movable_node_is_enabled())
-		return false;
+	if (in_kernel ^ in_movable)
+		return (in_kernel) ? kernel_zone : movable_zone;
 
-	return !zone_intersects(default_zone, start_pfn, nr_pages);
+	/*
+	 * If the range doesn't belong to any zone or two zones overlap in the
+	 * given range then we use movable zone only if movable_node is
+	 * enabled because we always online to a kernel zone by default.
+	 */
+	return movable_node_enabled ? movable_zone : kernel_zone;
 }
 
 struct zone * zone_for_pfn_range(int online_type, int nid, unsigned start_pfn,
 		unsigned long nr_pages)
 {
-	struct pglist_data *pgdat = NODE_DATA(nid);
-	struct zone *zone = default_zone_for_pfn(nid, start_pfn, nr_pages);
+	if (online_type == MMOP_ONLINE_KERNEL)
+		return default_kernel_zone_for_pfn(nid, start_pfn, nr_pages);
 
-	if (online_type == MMOP_ONLINE_KEEP) {
-		struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
-		/*
-		 * MMOP_ONLINE_KEEP defaults to MMOP_ONLINE_KERNEL but use
-		 * movable zone if that is not possible (e.g. we are within
-		 * or past the existing movable zone). movable_node overrides
-		 * this default and defaults to movable zone
-		 */
-		if (movable_pfn_range(nid, zone, start_pfn, nr_pages))
-			zone = movable_zone;
-	} else if (online_type == MMOP_ONLINE_MOVABLE) {
-		zone = &pgdat->node_zones[ZONE_MOVABLE];
-	}
+	if (online_type == MMOP_ONLINE_MOVABLE)
+		return &NODE_DATA(nid)->node_zones[ZONE_MOVABLE];
 
-	return zone;
+	return default_zone_for_pfn(nid, start_pfn, nr_pages);
 }
 
 /*
@@ -997,9 +972,6 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_typ
 	struct memory_notify arg;
 
 	nid = pfn_to_nid(pfn);
-	if (!allow_online_pfn_range(nid, pfn, nr_pages, online_type))
-		return -EINVAL;
-
 	/* associate pfn range with the zone */
 	zone = move_pfn_range(online_type, nid, pfn, nr_pages);
 
-- 
2.11.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199])
	by kanga.kvack.org (Postfix) with ESMTP id 4629A6B02C3
	for <linux-mm@kvack.org>; Thu, 29 Jun 2017 20:45:36 -0400 (EDT)
Received: by mail-pf0-f199.google.com with SMTP id g27so101691533pfj.6
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 17:45:36 -0700 (PDT)
Received: from mail-pf0-x243.google.com (mail-pf0-x243.google.com. [2607:f8b0:400e:c00::243])
        by mx.google.com with ESMTPS id g4si5106352pln.186.2017.06.29.17.45.35
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 29 Jun 2017 17:45:35 -0700 (PDT)
Received: by mail-pf0-x243.google.com with SMTP id e199so14978893pfh.0
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 17:45:35 -0700 (PDT)
Date: Fri, 30 Jun 2017 09:45:24 +0900
From: Joonsoo Kim <js1304@gmail.com>
Subject: Re: [PATCH 1/2] mm, memory_hotplug: display allowed zones in the
 preferred ordering
Message-ID: <20170630004522.GA13062@js1304-desktop>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-2-mhocko@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170629073509.623-2-mhocko@kernel.org>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Michal Hocko <mhocko@suse.com>

On Thu, Jun 29, 2017 at 09:35:08AM +0200, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
> 
> Prior to "mm, memory_hotplug: do not associate hotadded memory to zones
> until online" we used to allow to change the valid zone types of a
> memory block if it is adjacent to a different zone type. This fact was
> reflected in memoryNN/valid_zones by the ordering of printed zones.
> The first one was default (echo online > memoryNN/state) and the other
> one could be onlined explicitly by online_{movable,kernel}. This
> behavior was removed by the said patch and as such the ordering was
> not all that important. In most cases a kernel zone would be default
> anyway. The only exception is movable_node handled by "mm,
> memory_hotplug: support movable_node for hotpluggable nodes".
> 
> Let's reintroduce this behavior again because later patch will remove
> the zone overlap restriction and so user will be allowed to online
> kernel resp. movable block regardless of its placement. Original
> behavior will then become significant again because it would be
> non-trivial for users to see what is the default zone to online into.
> 
> Implementation is really simple. Pull out zone selection out of
> move_pfn_range into zone_for_pfn_range helper and use it in
> show_valid_zones to display the zone for default onlining and then
> both kernel and movable if they are allowed. Default online zone is not
> duplicated.
> 
> Signed-off-by: Michal Hocko <mhocko@suse.com>
> 

Acked-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-pg0-f71.google.com (mail-pg0-f71.google.com [74.125.83.71])
	by kanga.kvack.org (Postfix) with ESMTP id 3A0ED6B0279
	for <linux-mm@kvack.org>; Thu, 29 Jun 2017 21:16:29 -0400 (EDT)
Received: by mail-pg0-f71.google.com with SMTP id 76so106901897pgh.11
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 18:16:29 -0700 (PDT)
Received: from mail-pg0-x243.google.com (mail-pg0-x243.google.com. [2607:f8b0:400e:c05::243])
        by mx.google.com with ESMTPS id h11si5176865plk.114.2017.06.29.18.16.28
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 29 Jun 2017 18:16:28 -0700 (PDT)
Received: by mail-pg0-x243.google.com with SMTP id f127so13609822pgc.2
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 18:16:28 -0700 (PDT)
Date: Fri, 30 Jun 2017 10:16:20 +0900
From: Joonsoo Kim <js1304@gmail.com>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170630011618.GB13062@js1304-desktop>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170629073509.623-3-mhocko@kernel.org>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Michal Hocko <mhocko@suse.com>

On Thu, Jun 29, 2017 at 09:35:09AM +0200, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
> 
> Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
> to precede the Movable zone in the physical memory range. The purpose of
> the movable zone is, however, not bound to any physical memory restriction.
> It merely defines a class of migrateable and reclaimable memory.
> 
> There are users (e.g. CMA) who might want to reserve specific physical
> memory ranges for their own purpose. Moreover our pfn walkers have to be
> prepared for zones overlapping in the physical range already because we
> do support interleaving NUMA nodes and therefore zones can interleave as
> well. This means we can allow each memory block to be associated with a
> different zone.
> 
> Loosen the current onlining semantic and allow explicit onlining type on
> any memblock. That means that online_{kernel,movable} will be allowed
> regardless of the physical address of the memblock as long as it is
> offline of course. This might result in moveble zone overlapping with
> other kernel zones. Default onlining then becomes a bit tricky but still
> sensible. echo online > memoryXY/state will online the given block to
> 	1) the default zone if the given range is outside of any zone
> 	2) the enclosing zone if such a zone doesn't interleave with
> 	   any other zone
>         3) the default zone if more zones interleave for this range
> where default zone is movable zone only if movable_node is enabled
> otherwise it is a kernel zone.
> 
> Here is an example of the semantic with (movable_node is not present but
> it work in an analogous way). We start with following memblocks, all of
> them offline
> memory34/valid_zones:Normal Movable
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Normal Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal Movable
> memory40/valid_zones:Normal Movable
> memory41/valid_zones:Normal Movable
> 
> Now, we online block 34 in default mode and block 37 as movable
> root@test1:/sys/devices/system/node/node1# echo online > memory34/state
> root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal Movable
> memory40/valid_zones:Normal Movable
> memory41/valid_zones:Normal Movable
> 
> As we can see all other blocks can still be onlined both into Normal and
> Movable zones and the Normal is default because the Movable zone spans
> only block37 now.
> root@test1:/sys/devices/system/node/node1# echo online_movable > memory41/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Movable Normal
> memory39/valid_zones:Movable Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable
> 
> Now the default zone for blocks 37-41 has changed because movable zone
> spans that range.
> root@test1:/sys/devices/system/node/node1# echo online_kernel > memory39/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable
> 
> Note that the block 39 now belongs to the zone Normal and so block38
> falls into Normal by default as well.
> 
> For completness
> root@test1:/sys/devices/system/node/node1# for i in memory[34]?
> do
> 	echo online > $i/state 2>/dev/null
> done
> 
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal
> memory36/valid_zones:Normal
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable
> memory41/valid_zones:Movable
> 
> Implementation wise the change is quite straightforward. We can get rid
> of allow_online_pfn_range altogether. online_pages allows only offline
> nodes already. The original default_zone_for_pfn will become
> default_kernel_zone_for_pfn. New default_zone_for_pfn implements the
> above semantic. zone_for_pfn_range is slightly reorganized to implement
> kernel and movable online type explicitly and MMOP_ONLINE_KEEP becomes
> a catch all default behavior.
> 
> Signed-off-by: Michal Hocko <mhocko@suse.com>

I appreaciate your help!

Acked-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-vk0-f69.google.com (mail-vk0-f69.google.com [209.85.213.69])
	by kanga.kvack.org (Postfix) with ESMTP id C2D636B0279
	for <linux-mm@kvack.org>; Thu, 29 Jun 2017 23:10:13 -0400 (EDT)
Received: by mail-vk0-f69.google.com with SMTP id 195so36952447vkj.8
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 20:10:13 -0700 (PDT)
Received: from mail-vk0-x22d.google.com (mail-vk0-x22d.google.com. [2607:f8b0:400c:c05::22d])
        by mx.google.com with ESMTPS id d25si3409316uai.219.2017.06.29.20.10.12
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 29 Jun 2017 20:10:12 -0700 (PDT)
Received: by mail-vk0-x22d.google.com with SMTP id r125so60145045vkf.1
        for <linux-mm@kvack.org>; Thu, 29 Jun 2017 20:10:12 -0700 (PDT)
MIME-Version: 1.0
In-Reply-To: <20170629073509.623-3-mhocko@kernel.org>
References: <20170629073509.623-1-mhocko@kernel.org> <20170629073509.623-3-mhocko@kernel.org>
From: Wei Yang <richard.weiyang@gmail.com>
Date: Fri, 30 Jun 2017 11:09:51 +0800
Message-ID: <CADZGycaXs-TsVN2xy_rpFE_ML5_rs=iYN6ZQZsAfjTVHFyLyEQ@mail.gmail.com>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Content-Type: text/plain; charset="UTF-8"
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>
Cc: Linux-MM <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, Xishi Qiu <qiuxishi@huawei.com>, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, LKML <linux-kernel@vger.kernel.org>, Michal Hocko <mhocko@suse.com>

On Thu, Jun 29, 2017 at 3:35 PM, Michal Hocko <mhocko@kernel.org> wrote:
> From: Michal Hocko <mhocko@suse.com>
>

Michal,

I love the idea very much.

> Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
> to precede the Movable zone in the physical memory range. The purpose of
> the movable zone is, however, not bound to any physical memory restriction.
> It merely defines a class of migrateable and reclaimable memory.
>
> There are users (e.g. CMA) who might want to reserve specific physical
> memory ranges for their own purpose. Moreover our pfn walkers have to be
> prepared for zones overlapping in the physical range already because we
> do support interleaving NUMA nodes and therefore zones can interleave as
> well. This means we can allow each memory block to be associated with a
> different zone.
>
> Loosen the current onlining semantic and allow explicit onlining type on
> any memblock. That means that online_{kernel,movable} will be allowed
> regardless of the physical address of the memblock as long as it is
> offline of course. This might result in moveble zone overlapping with
> other kernel zones. Default onlining then becomes a bit tricky but still

As here mentioned, we just remove the restriction for zone_movable.
For other zones, we still keep the restriction and the order as before.

Maybe the title is a little misleading. Audience may thinks no restriction
for all zones.

> sensible. echo online > memoryXY/state will online the given block to
>         1) the default zone if the given range is outside of any zone
>         2) the enclosing zone if such a zone doesn't interleave with
>            any other zone
>         3) the default zone if more zones interleave for this range
> where default zone is movable zone only if movable_node is enabled
> otherwise it is a kernel zone.
>
> Here is an example of the semantic with (movable_node is not present but
> it work in an analogous way). We start with following memblocks, all of
> them offline
> memory34/valid_zones:Normal Movable
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Normal Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal Movable
> memory40/valid_zones:Normal Movable
> memory41/valid_zones:Normal Movable
>
> Now, we online block 34 in default mode and block 37 as movable
> root@test1:/sys/devices/system/node/node1# echo online > memory34/state
> root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal Movable
> memory40/valid_zones:Normal Movable
> memory41/valid_zones:Normal Movable
>
> As we can see all other blocks can still be onlined both into Normal and
> Movable zones and the Normal is default because the Movable zone spans
> only block37 now.
> root@test1:/sys/devices/system/node/node1# echo online_movable > memory41/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Movable Normal
> memory39/valid_zones:Movable Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable
>

As I spotted on the previous patch, after several round of online/offline,
The output of valid_zones will differ.

For example in this case, after I offline memory37 and 41, I expect this:

 memory34/valid_zones:Normal
 memory35/valid_zones:Normal Movable
 memory36/valid_zones:Normal Movable
 memory37/valid_zones:Normal Movable
 memory38/valid_zones:Normal Movable
 memory39/valid_zones:Normal Movable
 memory40/valid_zones:Normal Movable
 memory41/valid_zones:Normal Movable

While the current result would be

 memory34/valid_zones:Normal
 memory35/valid_zones:Normal Movable
 memory36/valid_zones:Normal Movable
 memory37/valid_zones:Movable Normal
 memory38/valid_zones:Movable Normal
 memory39/valid_zones:Movable Normal
 memory40/valid_zones:Movable Normal
 memory41/valid_zones:Movable Normal

The reason is the same, we don't adjust the zone's range when offline
memory.

This is also a known issue?

> Now the default zone for blocks 37-41 has changed because movable zone
> spans that range.
> root@test1:/sys/devices/system/node/node1# echo online_kernel > memory39/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable
>
> Note that the block 39 now belongs to the zone Normal and so block38
> falls into Normal by default as well.
>
> For completness
> root@test1:/sys/devices/system/node/node1# for i in memory[34]?
> do
>         echo online > $i/state 2>/dev/null
> done
>
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal
> memory36/valid_zones:Normal
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable
> memory41/valid_zones:Movable
>
> Implementation wise the change is quite straightforward. We can get rid
> of allow_online_pfn_range altogether. online_pages allows only offline
> nodes already. The original default_zone_for_pfn will become
> default_kernel_zone_for_pfn. New default_zone_for_pfn implements the
> above semantic. zone_for_pfn_range is slightly reorganized to implement
> kernel and movable online type explicitly and MMOP_ONLINE_KEEP becomes
> a catch all default behavior.
>
> Signed-off-by: Michal Hocko <mhocko@suse.com>
> ---

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wm0-f70.google.com (mail-wm0-f70.google.com [74.125.82.70])
	by kanga.kvack.org (Postfix) with ESMTP id C420B2802FE
	for <linux-mm@kvack.org>; Fri, 30 Jun 2017 04:39:30 -0400 (EDT)
Received: by mail-wm0-f70.google.com with SMTP id l81so6115551wmg.8
        for <linux-mm@kvack.org>; Fri, 30 Jun 2017 01:39:30 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id k16si5530737wrc.385.2017.06.30.01.39.29
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Fri, 30 Jun 2017 01:39:29 -0700 (PDT)
Date: Fri, 30 Jun 2017 10:39:26 +0200
From: Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170630083926.GA22923@dhcp22.suse.cz>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <CADZGycaXs-TsVN2xy_rpFE_ML5_rs=iYN6ZQZsAfjTVHFyLyEQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CADZGycaXs-TsVN2xy_rpFE_ML5_rs=iYN6ZQZsAfjTVHFyLyEQ@mail.gmail.com>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Wei Yang <richard.weiyang@gmail.com>
Cc: Linux-MM <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, Xishi Qiu <qiuxishi@huawei.com>, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, LKML <linux-kernel@vger.kernel.org>

On Fri 30-06-17 11:09:51, Wei Yang wrote:
> On Thu, Jun 29, 2017 at 3:35 PM, Michal Hocko <mhocko@kernel.org> wrote:
> > From: Michal Hocko <mhocko@suse.com>
> >
> 
> Michal,
> 
> I love the idea very much.
> 
> > Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
> > to precede the Movable zone in the physical memory range. The purpose of
> > the movable zone is, however, not bound to any physical memory restriction.
> > It merely defines a class of migrateable and reclaimable memory.
> >
> > There are users (e.g. CMA) who might want to reserve specific physical
> > memory ranges for their own purpose. Moreover our pfn walkers have to be
> > prepared for zones overlapping in the physical range already because we
> > do support interleaving NUMA nodes and therefore zones can interleave as
> > well. This means we can allow each memory block to be associated with a
> > different zone.
> >
> > Loosen the current onlining semantic and allow explicit onlining type on
> > any memblock. That means that online_{kernel,movable} will be allowed
> > regardless of the physical address of the memblock as long as it is
> > offline of course. This might result in moveble zone overlapping with
> > other kernel zones. Default onlining then becomes a bit tricky but still
> 
> As here mentioned, we just remove the restriction for zone_movable.
> For other zones, we still keep the restriction and the order as before.

All other zones except for ZONE_NORMAL are subject of the physical
memory restrictions.
 
> Maybe the title is a little misleading. Audience may thinks no restriction
> for all zones.

I thought the context was clear from the fact that this is a hotplug
related patch. As such we do not allow online_{dma,dma32,normal} we only
allow to online into a kernel zone. I can update the wording but do not
have a good idea how.

[...]
> As I spotted on the previous patch, after several round of online/offline,
> The output of valid_zones will differ.
> 
> For example in this case, after I offline memory37 and 41, I expect this:
> 
>  memory34/valid_zones:Normal
>  memory35/valid_zones:Normal Movable
>  memory36/valid_zones:Normal Movable
>  memory37/valid_zones:Normal Movable
>  memory38/valid_zones:Normal Movable
>  memory39/valid_zones:Normal Movable
>  memory40/valid_zones:Normal Movable
>  memory41/valid_zones:Normal Movable
> 
> While the current result would be
> 
>  memory34/valid_zones:Normal
>  memory35/valid_zones:Normal Movable
>  memory36/valid_zones:Normal Movable
>  memory37/valid_zones:Movable Normal
>  memory38/valid_zones:Movable Normal
>  memory39/valid_zones:Movable Normal
>  memory40/valid_zones:Movable Normal
>  memory41/valid_zones:Movable Normal

You haven't written your sequence of onlining but if you used the same
one as mentioned in the patch then you should get
memory34/valid_zones:Normal
memory35/valid_zones:Normal Movable
memory36/valid_zones:Normal Movable
memory37/valid_zones:Normal Movable
memory38/valid_zones:Normal Movable
memory39/valid_zones:Normal
memory40/valid_zones:Movable Normal
memory41/valid_zones:Movable Normal

Even if you kept 37 as movable and offline 38 you wouldn't get 38-41
movable by default because...

> The reason is the same, we don't adjust the zone's range when offline
> memory.

.. of this.

> This is also a known issue?

yes and to be honest I do not plan to fix it unless somebody has a real
life usecase for it. Now that we allow explicit onlininig type anywhere
it seems like a reasonable behavior and this will allow us to remove
quite some code which is always a good deal wrt longterm maintenance.

Thanks!
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-ua0-f200.google.com (mail-ua0-f200.google.com [209.85.217.200])
	by kanga.kvack.org (Postfix) with ESMTP id BF9132802FE
	for <linux-mm@kvack.org>; Fri, 30 Jun 2017 05:40:17 -0400 (EDT)
Received: by mail-ua0-f200.google.com with SMTP id w19so40480952uac.0
        for <linux-mm@kvack.org>; Fri, 30 Jun 2017 02:40:17 -0700 (PDT)
Received: from mail-ua0-x243.google.com (mail-ua0-x243.google.com. [2607:f8b0:400c:c08::243])
        by mx.google.com with ESMTPS id h15si3431563vkd.231.2017.06.30.02.40.16
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Fri, 30 Jun 2017 02:40:16 -0700 (PDT)
Received: by mail-ua0-x243.google.com with SMTP id j53so8314226uaa.2
        for <linux-mm@kvack.org>; Fri, 30 Jun 2017 02:40:16 -0700 (PDT)
MIME-Version: 1.0
In-Reply-To: <20170630083926.GA22923@dhcp22.suse.cz>
References: <20170629073509.623-1-mhocko@kernel.org> <20170629073509.623-3-mhocko@kernel.org>
 <CADZGycaXs-TsVN2xy_rpFE_ML5_rs=iYN6ZQZsAfjTVHFyLyEQ@mail.gmail.com> <20170630083926.GA22923@dhcp22.suse.cz>
From: Wei Yang <richard.weiyang@gmail.com>
Date: Fri, 30 Jun 2017 17:39:56 +0800
Message-ID: <CADZGyca1-CzaHoR-==DN4kK_YrwmMVnKvowUv-5M4GQP7ZYubg@mail.gmail.com>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Content-Type: text/plain; charset="UTF-8"
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>
Cc: Linux-MM <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, Xishi Qiu <qiuxishi@huawei.com>, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, LKML <linux-kernel@vger.kernel.org>

On Fri, Jun 30, 2017 at 4:39 PM, Michal Hocko <mhocko@kernel.org> wrote:
> On Fri 30-06-17 11:09:51, Wei Yang wrote:
>> On Thu, Jun 29, 2017 at 3:35 PM, Michal Hocko <mhocko@kernel.org> wrote:
>> > From: Michal Hocko <mhocko@suse.com>
>> >
>>
>> Michal,
>>
>> I love the idea very much.
>>

>
> You haven't written your sequence of onlining but if you used the same
> one as mentioned in the patch then you should get
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Normal Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable Normal
>
> Even if you kept 37 as movable and offline 38 you wouldn't get 38-41
> movable by default because...
>

Yes, it depends on the zone range.

>> The reason is the same, we don't adjust the zone's range when offline
>> memory.
>
> .. of this.
>
>> This is also a known issue?
>
> yes and to be honest I do not plan to fix it unless somebody has a real
> life usecase for it. Now that we allow explicit onlininig type anywhere
> it seems like a reasonable behavior and this will allow us to remove
> quite some code which is always a good deal wrt longterm maintenance.
>

hmm... the statistics displayed in /proc/zoneinfo would be meaningless
for zone_normal and zone_movable.

I am not sure, maybe no one care about these fields.

> Thanks!
> --
> Michal Hocko
> SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wm0-f70.google.com (mail-wm0-f70.google.com [74.125.82.70])
	by kanga.kvack.org (Postfix) with ESMTP id 6803E2802FE
	for <linux-mm@kvack.org>; Fri, 30 Jun 2017 05:55:50 -0400 (EDT)
Received: by mail-wm0-f70.google.com with SMTP id 62so6358239wmw.13
        for <linux-mm@kvack.org>; Fri, 30 Jun 2017 02:55:50 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id 10si9950390wme.0.2017.06.30.02.55.48
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Fri, 30 Jun 2017 02:55:49 -0700 (PDT)
Date: Fri, 30 Jun 2017 11:55:45 +0200
From: Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170630095545.GF22917@dhcp22.suse.cz>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <CADZGycaXs-TsVN2xy_rpFE_ML5_rs=iYN6ZQZsAfjTVHFyLyEQ@mail.gmail.com>
 <20170630083926.GA22923@dhcp22.suse.cz>
 <CADZGyca1-CzaHoR-==DN4kK_YrwmMVnKvowUv-5M4GQP7ZYubg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CADZGyca1-CzaHoR-==DN4kK_YrwmMVnKvowUv-5M4GQP7ZYubg@mail.gmail.com>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Wei Yang <richard.weiyang@gmail.com>
Cc: Linux-MM <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, Xishi Qiu <qiuxishi@huawei.com>, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, LKML <linux-kernel@vger.kernel.org>

On Fri 30-06-17 17:39:56, Wei Yang wrote:
> On Fri, Jun 30, 2017 at 4:39 PM, Michal Hocko <mhocko@kernel.org> wrote:
[...]
> > yes and to be honest I do not plan to fix it unless somebody has a real
> > life usecase for it. Now that we allow explicit onlininig type anywhere
> > it seems like a reasonable behavior and this will allow us to remove
> > quite some code which is always a good deal wrt longterm maintenance.
> >
> 
> hmm... the statistics displayed in /proc/zoneinfo would be meaningless
> for zone_normal and zone_movable.

Why would they be meaningless? Counters will always reflect the actual
use - if not then it is a bug. And wrt to zone description what is
meaningless about
memory34/valid_zones:Normal
memory35/valid_zones:Normal Movable
memory36/valid_zones:Movable
memory37/valid_zones:Movable Normal
memory38/valid_zones:Movable Normal
memory39/valid_zones:Movable Normal
memory40/valid_zones:Normal
memory41/valid_zones:Movable

And
Node 1, zone   Normal
  pages free     65465
        min      156
        low      221
        high     286
        spanned  229376
        present  65536
        managed  65536
[...]
  start_pfn:           1114112
Node 1, zone  Movable
  pages free     65443
        min      156
        low      221
        high     286
        spanned  196608
        present  65536
        managed  65536
[...]
  start_pfn:           1179648

ranges are clearly defined as [start_pfn, start_pfn+managed] and managed
matches the number of onlined pages (256MB).
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wm0-f72.google.com (mail-wm0-f72.google.com [74.125.82.72])
	by kanga.kvack.org (Postfix) with ESMTP id B8EC92802FE
	for <linux-mm@kvack.org>; Fri, 30 Jun 2017 07:01:23 -0400 (EDT)
Received: by mail-wm0-f72.google.com with SMTP id s187so3307432wmd.5
        for <linux-mm@kvack.org>; Fri, 30 Jun 2017 04:01:23 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id d72si3484033wmh.46.2017.06.30.04.01.22
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Fri, 30 Jun 2017 04:01:22 -0700 (PDT)
Date: Fri, 30 Jun 2017 13:01:18 +0200
From: Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170630110118.GG22917@dhcp22.suse.cz>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <CADZGycaXs-TsVN2xy_rpFE_ML5_rs=iYN6ZQZsAfjTVHFyLyEQ@mail.gmail.com>
 <20170630083926.GA22923@dhcp22.suse.cz>
 <CADZGyca1-CzaHoR-==DN4kK_YrwmMVnKvowUv-5M4GQP7ZYubg@mail.gmail.com>
 <20170630095545.GF22917@dhcp22.suse.cz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170630095545.GF22917@dhcp22.suse.cz>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Wei Yang <richard.weiyang@gmail.com>
Cc: Linux-MM <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, Xishi Qiu <qiuxishi@huawei.com>, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, LKML <linux-kernel@vger.kernel.org>

On Fri 30-06-17 11:55:45, Michal Hocko wrote:
> On Fri 30-06-17 17:39:56, Wei Yang wrote:
> > On Fri, Jun 30, 2017 at 4:39 PM, Michal Hocko <mhocko@kernel.org> wrote:
> [...]
> > > yes and to be honest I do not plan to fix it unless somebody has a real
> > > life usecase for it. Now that we allow explicit onlininig type anywhere
> > > it seems like a reasonable behavior and this will allow us to remove
> > > quite some code which is always a good deal wrt longterm maintenance.
> > >
> > 
> > hmm... the statistics displayed in /proc/zoneinfo would be meaningless
> > for zone_normal and zone_movable.
> 
> Why would they be meaningless? Counters will always reflect the actual
> use - if not then it is a bug. And wrt to zone description what is
> meaningless about
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Movable
> memory37/valid_zones:Movable Normal
> memory38/valid_zones:Movable Normal
> memory39/valid_zones:Movable Normal
> memory40/valid_zones:Normal
> memory41/valid_zones:Movable
> 
> And
> Node 1, zone   Normal
>   pages free     65465
>         min      156
>         low      221
>         high     286
>         spanned  229376
>         present  65536
>         managed  65536
> [...]
>   start_pfn:           1114112
> Node 1, zone  Movable
>   pages free     65443
>         min      156
>         low      221
>         high     286
>         spanned  196608
>         present  65536
>         managed  65536
> [...]
>   start_pfn:           1179648
> 
> ranges are clearly defined as [start_pfn, start_pfn+managed] and managed

errr, this should be [start_pfn, start_pfn + spanned] of course.

> matches the number of onlined pages (256MB).

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199])
	by kanga.kvack.org (Postfix) with ESMTP id 043E26B03C8
	for <linux-mm@kvack.org>; Wed,  5 Jul 2017 19:16:50 -0400 (EDT)
Received: by mail-pf0-f199.google.com with SMTP id 1so3771768pfi.14
        for <linux-mm@kvack.org>; Wed, 05 Jul 2017 16:16:49 -0700 (PDT)
Received: from mail-pg0-x241.google.com (mail-pg0-x241.google.com. [2607:f8b0:400e:c05::241])
        by mx.google.com with ESMTPS id o13si222139pgs.13.2017.07.05.16.16.49
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Wed, 05 Jul 2017 16:16:49 -0700 (PDT)
Received: by mail-pg0-x241.google.com with SMTP id j186so409345pge.1
        for <linux-mm@kvack.org>; Wed, 05 Jul 2017 16:16:49 -0700 (PDT)
Date: Thu, 6 Jul 2017 07:16:49 +0800
From: Wei Yang <richard.weiyang@gmail.com>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170705231649.GA10155@WeideMacBook-Pro.local>
Reply-To: Wei Yang <richard.weiyang@gmail.com>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <CADZGycaXs-TsVN2xy_rpFE_ML5_rs=iYN6ZQZsAfjTVHFyLyEQ@mail.gmail.com>
 <20170630083926.GA22923@dhcp22.suse.cz>
 <CADZGyca1-CzaHoR-==DN4kK_YrwmMVnKvowUv-5M4GQP7ZYubg@mail.gmail.com>
 <20170630095545.GF22917@dhcp22.suse.cz>
 <20170630110118.GG22917@dhcp22.suse.cz>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="C7zPtVaVf+AK4Oqc"
Content-Disposition: inline
In-Reply-To: <20170630110118.GG22917@dhcp22.suse.cz>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>
Cc: Wei Yang <richard.weiyang@gmail.com>, Linux-MM <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, Xishi Qiu <qiuxishi@huawei.com>, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, LKML <linux-kernel@vger.kernel.org>


--C7zPtVaVf+AK4Oqc
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Jun 30, 2017 at 01:01:18PM +0200, Michal Hocko wrote:
>On Fri 30-06-17 11:55:45, Michal Hocko wrote:
>> On Fri 30-06-17 17:39:56, Wei Yang wrote:
>> > On Fri, Jun 30, 2017 at 4:39 PM, Michal Hocko <mhocko@kernel.org> wrot=
e:
>> [...]
>> > > yes and to be honest I do not plan to fix it unless somebody has a r=
eal
>> > > life usecase for it. Now that we allow explicit onlininig type anywh=
ere
>> > > it seems like a reasonable behavior and this will allow us to remove
>> > > quite some code which is always a good deal wrt longterm maintenance.
>> > >
>> >=20
>> > hmm... the statistics displayed in /proc/zoneinfo would be meaningless
>> > for zone_normal and zone_movable.
>>=20
>> Why would they be meaningless? Counters will always reflect the actual
>> use - if not then it is a bug. And wrt to zone description what is
>> meaningless about
>> memory34/valid_zones:Normal
>> memory35/valid_zones:Normal Movable
>> memory36/valid_zones:Movable
>> memory37/valid_zones:Movable Normal
>> memory38/valid_zones:Movable Normal
>> memory39/valid_zones:Movable Normal
>> memory40/valid_zones:Normal
>> memory41/valid_zones:Movable
>>=20
>> And
>> Node 1, zone   Normal
>>   pages free     65465
>>         min      156
>>         low      221
>>         high     286
>>         spanned  229376
>>         present  65536
>>         managed  65536
>> [...]
>>   start_pfn:           1114112
>> Node 1, zone  Movable
>>   pages free     65443
>>         min      156
>>         low      221
>>         high     286
>>         spanned  196608
>>         present  65536
>>         managed  65536
>> [...]
>>   start_pfn:           1179648
>>=20
>> ranges are clearly defined as [start_pfn, start_pfn+managed] and managed
>
>errr, this should be [start_pfn, start_pfn + spanned] of course.
>

The spanned is not adjusted after offline, neither does start_pfn. For exam=
ple,
even offline all the movable_zone range, we can still see the spanned.

Below is a result with a little changed kernel to show the start_pfn always.
The sequence is:
1. bootup

Node 0, zone  Movable
        spanned  65536
	present  0
	managed  0
  start_pfn:           0

2. online movable 2 continuous memory_blocks

Node 0, zone  Movable
        spanned  65536
	present  65536
	managed  65536
  start_pfn:           1310720

3. offline 2nd memory_blocks

Node 0, zone  Movable
        spanned  65536
	present  32768
	managed  32768
  start_pfn:           1310720

4. offline 1st memory_blocks

Node 0, zone  Movable
        spanned  65536
	present  0
	managed  0
  start_pfn:           1310720

So I am not sure this is still clearly defined?

>> matches the number of onlined pages (256MB).
>
>--=20
>Michal Hocko
>SUSE Labs

--=20
Wei Yang
Help you, Help me

--C7zPtVaVf+AK4Oqc
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJZXXNhAAoJEKcLNpZP5cTdkx8QAK6MKFdOR81MTDChHy5erI8x
8Z9cq7ZXivryS4ovYZIMAe+2FbjyztcZOnu74pWCQtVKVUEMBFSieetusRyKZ3Gx
Nk0K3+kI/PiZFyM51Cj2v4I0OQW2ZmqNXZjdzYXm0lSlGyp7bnjoJkNp+VJ1SGmi
5TcumSPh4gWDQw/LPZ65TwBkJ9yuWAnoTO5gm4n4IP4KDS1ysbuwN1nRQgTvP3oF
tCfbsP7ev9ELBa+T3ZVhfaLuIIh7rIAnX1DtpCQ+UTepqe6A2awEbfLx3tTwVZyl
hzgvuOoUcBTEtE+HeQwn6ucNqKIcR0POrfpPu2qcKgSoX0yxn0sQOdpkOIuv9s7k
AKlylHVa9hupFlHJ2UFj8ftPJ2lGMB35YgkV3Qm0d5t0Lm7eeGpVl0om0V6WRkFI
CgJ47uh1wUkR3UCci9btTyDv+q9wXpMsgL1MZz0NYFT5DtcRevOllYCiFa+vvA2Z
HHe9R+MK5G/pNNsrn1Mjyx34dwKzYDIJfoJDZPil948vrznTGssI/Y8G5GkJUqhU
Hm13YgUGmSdBSauMGOB/WysYnwJFuz91TknLCLwnxm0ydCJk9r6tvGSu/19RJqTJ
LoH8P0mO0pkSwG61N/TrK52XIK6IyjEehdw/lShuzz9phrhExHLsl8cbGDaVDrBf
zxaWb3mYtj5ytDjb3Fgy
=Xpx9
-----END PGP SIGNATURE-----

--C7zPtVaVf+AK4Oqc--

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f200.google.com (mail-wr0-f200.google.com [209.85.128.200])
	by kanga.kvack.org (Postfix) with ESMTP id AF85F6B03E7
	for <linux-mm@kvack.org>; Thu,  6 Jul 2017 02:56:55 -0400 (EDT)
Received: by mail-wr0-f200.google.com with SMTP id z1so2514274wrz.10
        for <linux-mm@kvack.org>; Wed, 05 Jul 2017 23:56:55 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id u76si883243wrc.211.2017.07.05.23.56.54
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Wed, 05 Jul 2017 23:56:54 -0700 (PDT)
Date: Thu, 6 Jul 2017 08:56:50 +0200
From: Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170706065649.GC29724@dhcp22.suse.cz>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <CADZGycaXs-TsVN2xy_rpFE_ML5_rs=iYN6ZQZsAfjTVHFyLyEQ@mail.gmail.com>
 <20170630083926.GA22923@dhcp22.suse.cz>
 <CADZGyca1-CzaHoR-==DN4kK_YrwmMVnKvowUv-5M4GQP7ZYubg@mail.gmail.com>
 <20170630095545.GF22917@dhcp22.suse.cz>
 <20170630110118.GG22917@dhcp22.suse.cz>
 <20170705231649.GA10155@WeideMacBook-Pro.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170705231649.GA10155@WeideMacBook-Pro.local>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Wei Yang <richard.weiyang@gmail.com>
Cc: Linux-MM <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, Xishi Qiu <qiuxishi@huawei.com>, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, LKML <linux-kernel@vger.kernel.org>

On Thu 06-07-17 07:16:49, Wei Yang wrote:
> On Fri, Jun 30, 2017 at 01:01:18PM +0200, Michal Hocko wrote:
> >On Fri 30-06-17 11:55:45, Michal Hocko wrote:
> >> On Fri 30-06-17 17:39:56, Wei Yang wrote:
> >> > On Fri, Jun 30, 2017 at 4:39 PM, Michal Hocko <mhocko@kernel.org> wrote:
> >> [...]
> >> > > yes and to be honest I do not plan to fix it unless somebody has a real
> >> > > life usecase for it. Now that we allow explicit onlininig type anywhere
> >> > > it seems like a reasonable behavior and this will allow us to remove
> >> > > quite some code which is always a good deal wrt longterm maintenance.
> >> > >
> >> > 
> >> > hmm... the statistics displayed in /proc/zoneinfo would be meaningless
> >> > for zone_normal and zone_movable.
> >> 
> >> Why would they be meaningless? Counters will always reflect the actual
> >> use - if not then it is a bug. And wrt to zone description what is
> >> meaningless about
> >> memory34/valid_zones:Normal
> >> memory35/valid_zones:Normal Movable
> >> memory36/valid_zones:Movable
> >> memory37/valid_zones:Movable Normal
> >> memory38/valid_zones:Movable Normal
> >> memory39/valid_zones:Movable Normal
> >> memory40/valid_zones:Normal
> >> memory41/valid_zones:Movable
> >> 
> >> And
> >> Node 1, zone   Normal
> >>   pages free     65465
> >>         min      156
> >>         low      221
> >>         high     286
> >>         spanned  229376
> >>         present  65536
> >>         managed  65536
> >> [...]
> >>   start_pfn:           1114112
> >> Node 1, zone  Movable
> >>   pages free     65443
> >>         min      156
> >>         low      221
> >>         high     286
> >>         spanned  196608
> >>         present  65536
> >>         managed  65536
> >> [...]
> >>   start_pfn:           1179648
> >> 
> >> ranges are clearly defined as [start_pfn, start_pfn+managed] and managed
> >
> >errr, this should be [start_pfn, start_pfn + spanned] of course.
> >
> 
> The spanned is not adjusted after offline, neither does start_pfn. For example,
> even offline all the movable_zone range, we can still see the spanned.

Which is completely valid. Offline only changes present/managed.

> Below is a result with a little changed kernel to show the start_pfn always.
> The sequence is:
> 1. bootup
> 
> Node 0, zone  Movable
>         spanned  65536
> 	present  0
> 	managed  0
>   start_pfn:           0
> 
> 2. online movable 2 continuous memory_blocks
> 
> Node 0, zone  Movable
>         spanned  65536
> 	present  65536
> 	managed  65536
>   start_pfn:           1310720
> 
> 3. offline 2nd memory_blocks
> 
> Node 0, zone  Movable
>         spanned  65536
> 	present  32768
> 	managed  32768
>   start_pfn:           1310720
> 
> 4. offline 1st memory_blocks
> 
> Node 0, zone  Movable
>         spanned  65536
> 	present  0
> 	managed  0
>   start_pfn:           1310720
> 
> So I am not sure this is still clearly defined?

Could you be more specific what is not clearly defined? You have
offlined all online memory blocks so present/managed is 0 while the
spanned is unchanged because the zone is still defined in range
[1310720, 1376256].

I also do not see how this is related with the discussed patch as there
is no zone interleaving involved.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-pg0-f71.google.com (mail-pg0-f71.google.com [74.125.83.71])
	by kanga.kvack.org (Postfix) with ESMTP id 8B12C6B0279
	for <linux-mm@kvack.org>; Fri,  7 Jul 2017 04:37:28 -0400 (EDT)
Received: by mail-pg0-f71.google.com with SMTP id g7so27677766pgp.1
        for <linux-mm@kvack.org>; Fri, 07 Jul 2017 01:37:28 -0700 (PDT)
Received: from mail-pg0-x242.google.com (mail-pg0-x242.google.com. [2607:f8b0:400e:c05::242])
        by mx.google.com with ESMTPS id c83si1767122pfd.95.2017.07.07.01.37.27
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Fri, 07 Jul 2017 01:37:27 -0700 (PDT)
Received: by mail-pg0-x242.google.com with SMTP id d193so3284483pgc.2
        for <linux-mm@kvack.org>; Fri, 07 Jul 2017 01:37:27 -0700 (PDT)
Date: Fri, 7 Jul 2017 16:37:23 +0800
From: Wei Yang <richard.weiyang@gmail.com>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170707083723.GA19821@WeideMacBook-Pro.local>
Reply-To: Wei Yang <richard.weiyang@gmail.com>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <CADZGycaXs-TsVN2xy_rpFE_ML5_rs=iYN6ZQZsAfjTVHFyLyEQ@mail.gmail.com>
 <20170630083926.GA22923@dhcp22.suse.cz>
 <CADZGyca1-CzaHoR-==DN4kK_YrwmMVnKvowUv-5M4GQP7ZYubg@mail.gmail.com>
 <20170630095545.GF22917@dhcp22.suse.cz>
 <20170630110118.GG22917@dhcp22.suse.cz>
 <20170705231649.GA10155@WeideMacBook-Pro.local>
 <20170706065649.GC29724@dhcp22.suse.cz>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="AqsLC8rIMeq19msA"
Content-Disposition: inline
In-Reply-To: <20170706065649.GC29724@dhcp22.suse.cz>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>
Cc: Wei Yang <richard.weiyang@gmail.com>, Linux-MM <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, Xishi Qiu <qiuxishi@huawei.com>, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, LKML <linux-kernel@vger.kernel.org>


--AqsLC8rIMeq19msA
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Jul 06, 2017 at 08:56:50AM +0200, Michal Hocko wrote:
>> Below is a result with a little changed kernel to show the start_pfn alw=
ays.
>> The sequence is:
>> 1. bootup
>>=20
>> Node 0, zone  Movable
>>         spanned  65536
>> 	present  0
>> 	managed  0
>>   start_pfn:           0
>>=20
>> 2. online movable 2 continuous memory_blocks
>>=20
>> Node 0, zone  Movable
>>         spanned  65536
>> 	present  65536
>> 	managed  65536
>>   start_pfn:           1310720
>>=20
>> 3. offline 2nd memory_blocks
>>=20
>> Node 0, zone  Movable
>>         spanned  65536
>> 	present  32768
>> 	managed  32768
>>   start_pfn:           1310720
>>=20
>> 4. offline 1st memory_blocks
>>=20
>> Node 0, zone  Movable
>>         spanned  65536
>> 	present  0
>> 	managed  0
>>   start_pfn:           1310720
>>=20
>> So I am not sure this is still clearly defined?
>
>Could you be more specific what is not clearly defined? You have
>offlined all online memory blocks so present/managed is 0 while the
>spanned is unchanged because the zone is still defined in range
>[1310720, 1376256].
>

The zone is empty after remove these two memory blocks, while we still think
it is defined in range [1310720, 1376256]. This is what I want to point.

>I also do not see how this is related with the discussed patch as there
>is no zone interleaving involved.

I had a patch which fix the behavior, which means we can make sure the zone=
 is
empty after remove these two memory blocks. As you mentioned in the reply,
http://www.spinics.net/lists/linux-mm/msg130230.html, I thought you would h=
ave
this fixed in this cycle. While it looks we will still have this behavior in
this cycle and looks no intend to fix this?

>--=20
>Michal Hocko
>SUSE Labs

--=20
Wei Yang
Help you, Help me

--AqsLC8rIMeq19msA
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJZX0hDAAoJEKcLNpZP5cTdizQP+gM9VAADXkT08cquUGeUdNDL
YF7rIkILCBmPOOg9Y9xrHMhdQLgzxIvQcs6Sv5vXx7Kj24LzwVlIrRoJUH+2P/yR
TGKFccZ9N0RZ8V6YvPMoT5HBVgaLENSQYCxYkJvcz83uu/9MrnvcYCh/6qn2vr+Y
djgy564P1bI+lchNAeilxwkqedEvzuJ67A/wXR7eLyj3G3Gpq/H7KhWXyiaZ/qCk
NTdoACDnn9LfjqEad4IKy1+1yVyRyX3ObVIRaO+YyKA7pHKDTNJFeaa8+rHmnzXy
lK5W9w3Xes9kUk8Rxj+KxZv/VP835zd+jVQeF9Iyk5dC0P3L8XiI9pF/7E7sOTxW
neLkKu8RRUMzpUyjeDD4P/wWoKulKVrhs3JlnWsdWhxi5bB0FsaLdOyoqOeWdXEc
qKeAvx9zvEMtTQRz9yVlHiBhHSci/TBybgdOJjweRbvbjINPHMS9tC9OVMP3pr7x
ODivoQVmSXLuCcVN5HteKBZl2Z+aIexAAdbNgk/X30FATEGFokColNo7xuVZr2ez
44WmpAOYu7D+v0xp4nGhMUltvrIc0Dd6xC9iMTeJR6CTlXiHqO+HQroUwWO7/u/F
FTd+UUyJPkPXQqGtEouUlFQYg7XO3DRbAt+2+CRvZBtOR+wBsM7YMIUyCjfXXqiV
z856sFdHjIQd/KwbA98Y
=dasa
-----END PGP SIGNATURE-----

--AqsLC8rIMeq19msA--

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f200.google.com (mail-wr0-f200.google.com [209.85.128.200])
	by kanga.kvack.org (Postfix) with ESMTP id C66ED6B0279
	for <linux-mm@kvack.org>; Fri,  7 Jul 2017 08:41:17 -0400 (EDT)
Received: by mail-wr0-f200.google.com with SMTP id 23so7643376wry.4
        for <linux-mm@kvack.org>; Fri, 07 Jul 2017 05:41:17 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id o23si2009601wro.277.2017.07.07.05.41.16
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Fri, 07 Jul 2017 05:41:16 -0700 (PDT)
Date: Fri, 7 Jul 2017 14:41:12 +0200
From: Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170707124112.GB16187@dhcp22.suse.cz>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <CADZGycaXs-TsVN2xy_rpFE_ML5_rs=iYN6ZQZsAfjTVHFyLyEQ@mail.gmail.com>
 <20170630083926.GA22923@dhcp22.suse.cz>
 <CADZGyca1-CzaHoR-==DN4kK_YrwmMVnKvowUv-5M4GQP7ZYubg@mail.gmail.com>
 <20170630095545.GF22917@dhcp22.suse.cz>
 <20170630110118.GG22917@dhcp22.suse.cz>
 <20170705231649.GA10155@WeideMacBook-Pro.local>
 <20170706065649.GC29724@dhcp22.suse.cz>
 <20170707083723.GA19821@WeideMacBook-Pro.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170707083723.GA19821@WeideMacBook-Pro.local>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Wei Yang <richard.weiyang@gmail.com>
Cc: Linux-MM <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, Xishi Qiu <qiuxishi@huawei.com>, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, LKML <linux-kernel@vger.kernel.org>

On Fri 07-07-17 16:37:23, Wei Yang wrote:
> On Thu, Jul 06, 2017 at 08:56:50AM +0200, Michal Hocko wrote:
> >> Below is a result with a little changed kernel to show the start_pfn always.
> >> The sequence is:
> >> 1. bootup
> >> 
> >> Node 0, zone  Movable
> >>         spanned  65536
> >> 	present  0
> >> 	managed  0
> >>   start_pfn:           0
> >> 
> >> 2. online movable 2 continuous memory_blocks
> >> 
> >> Node 0, zone  Movable
> >>         spanned  65536
> >> 	present  65536
> >> 	managed  65536
> >>   start_pfn:           1310720
> >> 
> >> 3. offline 2nd memory_blocks
> >> 
> >> Node 0, zone  Movable
> >>         spanned  65536
> >> 	present  32768
> >> 	managed  32768
> >>   start_pfn:           1310720
> >> 
> >> 4. offline 1st memory_blocks
> >> 
> >> Node 0, zone  Movable
> >>         spanned  65536
> >> 	present  0
> >> 	managed  0
> >>   start_pfn:           1310720
> >> 
> >> So I am not sure this is still clearly defined?
> >
> >Could you be more specific what is not clearly defined? You have
> >offlined all online memory blocks so present/managed is 0 while the
> >spanned is unchanged because the zone is still defined in range
> >[1310720, 1376256].
> >
> 
> The zone is empty after remove these two memory blocks, while we still think
> it is defined in range [1310720, 1376256].

Yes and present/managed shows that the zone is empty. It's range spans
some range but there are no online pages.

> This is what I want to point.

As I've said several times already. This is somemething that _could_ be
fixed but I would rather not to do so until there is a _readl_ usecase
which would depend on it. Especially when we can online any memory block
to the zone you like. We should really strive to reduce the amount of
code rather than keep it just in case without anybody actually using it.

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f200.google.com (mail-wr0-f200.google.com [209.85.128.200])
	by kanga.kvack.org (Postfix) with ESMTP id BE2AC6B0279
	for <linux-mm@kvack.org>; Fri,  7 Jul 2017 10:34:58 -0400 (EDT)
Received: by mail-wr0-f200.google.com with SMTP id z45so8321277wrb.13
        for <linux-mm@kvack.org>; Fri, 07 Jul 2017 07:34:58 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id q8si3255246wmb.13.2017.07.07.07.34.56
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Fri, 07 Jul 2017 07:34:57 -0700 (PDT)
Subject: Re: [PATCH 1/2] mm, memory_hotplug: display allowed zones in the
 preferred ordering
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-2-mhocko@kernel.org>
From: Vlastimil Babka <vbabka@suse.cz>
Message-ID: <f58b1497-752f-5dd3-2612-6f0c59305fbf@suse.cz>
Date: Fri, 7 Jul 2017 16:34:53 +0200
MIME-Version: 1.0
In-Reply-To: <20170629073509.623-2-mhocko@kernel.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>, linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Michal Hocko <mhocko@suse.com>

On 06/29/2017 09:35 AM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
> 
> Prior to "mm, memory_hotplug: do not associate hotadded memory to zones
> until online" we used to allow to change the valid zone types of a
> memory block if it is adjacent to a different zone type. This fact was
> reflected in memoryNN/valid_zones by the ordering of printed zones.
> The first one was default (echo online > memoryNN/state) and the other
> one could be onlined explicitly by online_{movable,kernel}. This
> behavior was removed by the said patch and as such the ordering was
> not all that important. In most cases a kernel zone would be default
> anyway. The only exception is movable_node handled by "mm,
> memory_hotplug: support movable_node for hotpluggable nodes".
> 
> Let's reintroduce this behavior again because later patch will remove
> the zone overlap restriction and so user will be allowed to online
> kernel resp. movable block regardless of its placement. Original
> behavior will then become significant again because it would be
> non-trivial for users to see what is the default zone to online into.
> 
> Implementation is really simple. Pull out zone selection out of
> move_pfn_range into zone_for_pfn_range helper and use it in
> show_valid_zones to display the zone for default onlining and then
> both kernel and movable if they are allowed. Default online zone is not
> duplicated.

Hm I wouldn't call this maze of functions simple, but seems to be correct.
Maybe Patch 2/2 will simplify the code...

> Signed-off-by: Michal Hocko <mhocko@suse.com>

Acked-by: Vlastimil Babka <vbabka@suse.cz>

> 
> fold me "mm, memory_hotplug: display allowed zones in the preferred ordering"
> ---
>  drivers/base/memory.c          | 33 +++++++++++++------
>  include/linux/memory_hotplug.h |  2 +-
>  mm/memory_hotplug.c            | 73 ++++++++++++++++++++++++------------------
>  3 files changed, 65 insertions(+), 43 deletions(-)
> 
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index c7c4e0325cdb..26383af9900c 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -388,6 +388,22 @@ static ssize_t show_phys_device(struct device *dev,
>  }
>  
>  #ifdef CONFIG_MEMORY_HOTREMOVE
> +static void print_allowed_zone(char *buf, int nid, unsigned long start_pfn,
> +		unsigned long nr_pages, int online_type,
> +		struct zone *default_zone)
> +{
> +	struct zone *zone;
> +
> +	if (!allow_online_pfn_range(nid, start_pfn, nr_pages, online_type))
> +		return;
> +
> +	zone = zone_for_pfn_range(online_type, nid, start_pfn, nr_pages);
> +	if (zone != default_zone) {
> +		strcat(buf, " ");
> +		strcat(buf, zone->name);
> +	}
> +}
> +
>  static ssize_t show_valid_zones(struct device *dev,
>  				struct device_attribute *attr, char *buf)
>  {
> @@ -395,7 +411,7 @@ static ssize_t show_valid_zones(struct device *dev,
>  	unsigned long start_pfn = section_nr_to_pfn(mem->start_section_nr);
>  	unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
>  	unsigned long valid_start_pfn, valid_end_pfn;
> -	bool append = false;
> +	struct zone *default_zone;
>  	int nid;
>  
>  	/*
> @@ -418,16 +434,13 @@ static ssize_t show_valid_zones(struct device *dev,
>  	}
>  
>  	nid = pfn_to_nid(start_pfn);
> -	if (allow_online_pfn_range(nid, start_pfn, nr_pages, MMOP_ONLINE_KERNEL)) {
> -		strcat(buf, default_zone_for_pfn(nid, start_pfn, nr_pages)->name);
> -		append = true;
> -	}
> +	default_zone = zone_for_pfn_range(MMOP_ONLINE_KEEP, nid, start_pfn, nr_pages);
> +	strcat(buf, default_zone->name);
>  
> -	if (allow_online_pfn_range(nid, start_pfn, nr_pages, MMOP_ONLINE_MOVABLE)) {
> -		if (append)
> -			strcat(buf, " ");
> -		strcat(buf, NODE_DATA(nid)->node_zones[ZONE_MOVABLE].name);
> -	}
> +	print_allowed_zone(buf, nid, start_pfn, nr_pages, MMOP_ONLINE_KERNEL,
> +			default_zone);
> +	print_allowed_zone(buf, nid, start_pfn, nr_pages, MMOP_ONLINE_MOVABLE,
> +			default_zone);
>  out:
>  	strcat(buf, "\n");
>  
> diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
> index c8a5056a5ae0..5e6e4cc36ff4 100644
> --- a/include/linux/memory_hotplug.h
> +++ b/include/linux/memory_hotplug.h
> @@ -319,6 +319,6 @@ extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map,
>  					  unsigned long pnum);
>  extern bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages,
>  		int online_type);
> -extern struct zone *default_zone_for_pfn(int nid, unsigned long pfn,
> +extern struct zone *zone_for_pfn_range(int online_type, int nid, unsigned start_pfn,
>  		unsigned long nr_pages);
>  #endif /* __LINUX_MEMORY_HOTPLUG_H */
> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index b4015a39d108..6b9a60115e37 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -836,31 +836,6 @@ static void node_states_set_node(int node, struct memory_notify *arg)
>  	node_set_state(node, N_MEMORY);
>  }
>  
> -bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages, int online_type)
> -{
> -	struct pglist_data *pgdat = NODE_DATA(nid);
> -	struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
> -	struct zone *default_zone = default_zone_for_pfn(nid, pfn, nr_pages);
> -
> -	/*
> -	 * TODO there shouldn't be any inherent reason to have ZONE_NORMAL
> -	 * physically before ZONE_MOVABLE. All we need is they do not
> -	 * overlap. Historically we didn't allow ZONE_NORMAL after ZONE_MOVABLE
> -	 * though so let's stick with it for simplicity for now.
> -	 * TODO make sure we do not overlap with ZONE_DEVICE
> -	 */
> -	if (online_type == MMOP_ONLINE_KERNEL) {
> -		if (zone_is_empty(movable_zone))
> -			return true;
> -		return movable_zone->zone_start_pfn >= pfn + nr_pages;
> -	} else if (online_type == MMOP_ONLINE_MOVABLE) {
> -		return zone_end_pfn(default_zone) <= pfn;
> -	}
> -
> -	/* MMOP_ONLINE_KEEP will always succeed and inherits the current zone */
> -	return online_type == MMOP_ONLINE_KEEP;
> -}
> -
>  static void __meminit resize_zone_range(struct zone *zone, unsigned long start_pfn,
>  		unsigned long nr_pages)
>  {
> @@ -919,7 +894,7 @@ void __ref move_pfn_range_to_zone(struct zone *zone,
>   * If no kernel zone covers this pfn range it will automatically go
>   * to the ZONE_NORMAL.
>   */
> -struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
> +static struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
>  		unsigned long nr_pages)
>  {
>  	struct pglist_data *pgdat = NODE_DATA(nid);
> @@ -935,6 +910,31 @@ struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
>  	return &pgdat->node_zones[ZONE_NORMAL];
>  }
>  
> +bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages, int online_type)
> +{
> +	struct pglist_data *pgdat = NODE_DATA(nid);
> +	struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
> +	struct zone *default_zone = default_zone_for_pfn(nid, pfn, nr_pages);
> +
> +	/*
> +	 * TODO there shouldn't be any inherent reason to have ZONE_NORMAL
> +	 * physically before ZONE_MOVABLE. All we need is they do not
> +	 * overlap. Historically we didn't allow ZONE_NORMAL after ZONE_MOVABLE
> +	 * though so let's stick with it for simplicity for now.
> +	 * TODO make sure we do not overlap with ZONE_DEVICE
> +	 */
> +	if (online_type == MMOP_ONLINE_KERNEL) {
> +		if (zone_is_empty(movable_zone))
> +			return true;
> +		return movable_zone->zone_start_pfn >= pfn + nr_pages;
> +	} else if (online_type == MMOP_ONLINE_MOVABLE) {
> +		return zone_end_pfn(default_zone) <= pfn;
> +	}
> +
> +	/* MMOP_ONLINE_KEEP will always succeed and inherits the current zone */
> +	return online_type == MMOP_ONLINE_KEEP;
> +}
> +
>  static inline bool movable_pfn_range(int nid, struct zone *default_zone,
>  		unsigned long start_pfn, unsigned long nr_pages)
>  {
> @@ -948,12 +948,8 @@ static inline bool movable_pfn_range(int nid, struct zone *default_zone,
>  	return !zone_intersects(default_zone, start_pfn, nr_pages);
>  }
>  
> -/*
> - * Associates the given pfn range with the given node and the zone appropriate
> - * for the given online type.
> - */
> -static struct zone * __meminit move_pfn_range(int online_type, int nid,
> -		unsigned long start_pfn, unsigned long nr_pages)
> +struct zone * zone_for_pfn_range(int online_type, int nid, unsigned start_pfn,
> +		unsigned long nr_pages)
>  {
>  	struct pglist_data *pgdat = NODE_DATA(nid);
>  	struct zone *zone = default_zone_for_pfn(nid, start_pfn, nr_pages);
> @@ -972,6 +968,19 @@ static struct zone * __meminit move_pfn_range(int online_type, int nid,
>  		zone = &pgdat->node_zones[ZONE_MOVABLE];
>  	}
>  
> +	return zone;
> +}
> +
> +/*
> + * Associates the given pfn range with the given node and the zone appropriate
> + * for the given online type.
> + */
> +static struct zone * __meminit move_pfn_range(int online_type, int nid,
> +		unsigned long start_pfn, unsigned long nr_pages)
> +{
> +	struct zone *zone;
> +
> +	zone = zone_for_pfn_range(online_type, nid, start_pfn, nr_pages);
>  	move_pfn_range_to_zone(zone, start_pfn, nr_pages);
>  	return zone;
>  }
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f200.google.com (mail-wr0-f200.google.com [209.85.128.200])
	by kanga.kvack.org (Postfix) with ESMTP id 2B3E26B0315
	for <linux-mm@kvack.org>; Fri,  7 Jul 2017 11:03:04 -0400 (EDT)
Received: by mail-wr0-f200.google.com with SMTP id 4so8547268wrc.15
        for <linux-mm@kvack.org>; Fri, 07 Jul 2017 08:03:04 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id v185si3235425wmb.36.2017.07.07.08.03.02
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Fri, 07 Jul 2017 08:03:02 -0700 (PDT)
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
From: Vlastimil Babka <vbabka@suse.cz>
Message-ID: <64e889ae-24ab-b845-5751-978a76dd0dd9@suse.cz>
Date: Fri, 7 Jul 2017 17:02:59 +0200
MIME-Version: 1.0
In-Reply-To: <20170629073509.623-3-mhocko@kernel.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>, linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Michal Hocko <mhocko@suse.com>, Linux API <linux-api@vger.kernel.org>

[+CC linux-api]

On 06/29/2017 09:35 AM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
> 
> Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
> to precede the Movable zone in the physical memory range. The purpose of
> the movable zone is, however, not bound to any physical memory restriction.
> It merely defines a class of migrateable and reclaimable memory.
> 
> There are users (e.g. CMA) who might want to reserve specific physical
> memory ranges for their own purpose. Moreover our pfn walkers have to be
> prepared for zones overlapping in the physical range already because we
> do support interleaving NUMA nodes and therefore zones can interleave as
> well. This means we can allow each memory block to be associated with a
> different zone.
> 
> Loosen the current onlining semantic and allow explicit onlining type on
> any memblock. That means that online_{kernel,movable} will be allowed
> regardless of the physical address of the memblock as long as it is
> offline of course. This might result in moveble zone overlapping with
> other kernel zones. Default onlining then becomes a bit tricky but still
> sensible. echo online > memoryXY/state will online the given block to
> 	1) the default zone if the given range is outside of any zone
> 	2) the enclosing zone if such a zone doesn't interleave with
> 	   any other zone
>         3) the default zone if more zones interleave for this range
> where default zone is movable zone only if movable_node is enabled
> otherwise it is a kernel zone.
> 
> Here is an example of the semantic with (movable_node is not present but
> it work in an analogous way). We start with following memblocks, all of
> them offline
> memory34/valid_zones:Normal Movable
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Normal Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal Movable
> memory40/valid_zones:Normal Movable
> memory41/valid_zones:Normal Movable
> 
> Now, we online block 34 in default mode and block 37 as movable
> root@test1:/sys/devices/system/node/node1# echo online > memory34/state
> root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal Movable
> memory40/valid_zones:Normal Movable
> memory41/valid_zones:Normal Movable

Hm so previously, blocks 37-41 would only allow Movable at this point, right?
Shouldn't we still default to Movable for them? We might be breaking some
existing userspace here.
IMHO onlining new memory past existing blocks is more common use case than
onlining memory between two blocks that are already online?

I also agree with Wei Yang that it's rather fuzzy that a zone that has been
completely offlined will affect the defaults for the next onlining just because
it has some spanned range, which is however empty of actual populated memory.

Maybe it would simplest for everyone to just default to Normal, except
movable_node? That's if we decide that the potential breakage I described above
is a non-issue.

> As we can see all other blocks can still be onlined both into Normal and
> Movable zones and the Normal is default because the Movable zone spans
> only block37 now.
> root@test1:/sys/devices/system/node/node1# echo online_movable > memory41/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Movable Normal
> memory39/valid_zones:Movable Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable
> 
> Now the default zone for blocks 37-41 has changed because movable zone
> spans that range.
> root@test1:/sys/devices/system/node/node1# echo online_kernel > memory39/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable
> 
> Note that the block 39 now belongs to the zone Normal and so block38
> falls into Normal by default as well.
> 
> For completness
> root@test1:/sys/devices/system/node/node1# for i in memory[34]?
> do
> 	echo online > $i/state 2>/dev/null
> done
> 
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal
> memory36/valid_zones:Normal
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable
> memory41/valid_zones:Movable
> 
> Implementation wise the change is quite straightforward. We can get rid
> of allow_online_pfn_range altogether. online_pages allows only offline
> nodes already. The original default_zone_for_pfn will become
> default_kernel_zone_for_pfn. New default_zone_for_pfn implements the
> above semantic. zone_for_pfn_range is slightly reorganized to implement
> kernel and movable online type explicitly and MMOP_ONLINE_KEEP becomes
> a catch all default behavior.
> 
> Signed-off-by: Michal Hocko <mhocko@suse.com>
> ---
>  drivers/base/memory.c |  3 ---
>  mm/memory_hotplug.c   | 74 ++++++++++++++++-----------------------------------
>  2 files changed, 23 insertions(+), 54 deletions(-)
> 
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index 26383af9900c..4e3b61cda520 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -394,9 +394,6 @@ static void print_allowed_zone(char *buf, int nid, unsigned long start_pfn,
>  {
>  	struct zone *zone;
>  
> -	if (!allow_online_pfn_range(nid, start_pfn, nr_pages, online_type))
> -		return;
> -
>  	zone = zone_for_pfn_range(online_type, nid, start_pfn, nr_pages);
>  	if (zone != default_zone) {
>  		strcat(buf, " ");
> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index 6b9a60115e37..670f7acbecf4 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -894,7 +894,7 @@ void __ref move_pfn_range_to_zone(struct zone *zone,
>   * If no kernel zone covers this pfn range it will automatically go
>   * to the ZONE_NORMAL.
>   */
> -static struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
> +static struct zone *default_kernel_zone_for_pfn(int nid, unsigned long start_pfn,
>  		unsigned long nr_pages)
>  {
>  	struct pglist_data *pgdat = NODE_DATA(nid);
> @@ -910,65 +910,40 @@ static struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
>  	return &pgdat->node_zones[ZONE_NORMAL];
>  }
>  
> -bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages, int online_type)
> +static inline struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
> +		unsigned long nr_pages)
>  {
> -	struct pglist_data *pgdat = NODE_DATA(nid);
> -	struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
> -	struct zone *default_zone = default_zone_for_pfn(nid, pfn, nr_pages);
> +	struct zone *kernel_zone = default_kernel_zone_for_pfn(nid, start_pfn,
> +			nr_pages);
> +	struct zone *movable_zone = &NODE_DATA(nid)->node_zones[ZONE_MOVABLE];
> +	bool in_kernel = zone_intersects(kernel_zone, start_pfn, nr_pages);
> +	bool in_movable = zone_intersects(movable_zone, start_pfn, nr_pages);
>  
>  	/*
> -	 * TODO there shouldn't be any inherent reason to have ZONE_NORMAL
> -	 * physically before ZONE_MOVABLE. All we need is they do not
> -	 * overlap. Historically we didn't allow ZONE_NORMAL after ZONE_MOVABLE
> -	 * though so let's stick with it for simplicity for now.
> -	 * TODO make sure we do not overlap with ZONE_DEVICE
> +	 * We inherit the existing zone in a simple case where zones do not
> +	 * overlap in the given range
>  	 */
> -	if (online_type == MMOP_ONLINE_KERNEL) {
> -		if (zone_is_empty(movable_zone))
> -			return true;
> -		return movable_zone->zone_start_pfn >= pfn + nr_pages;
> -	} else if (online_type == MMOP_ONLINE_MOVABLE) {
> -		return zone_end_pfn(default_zone) <= pfn;
> -	}
> -
> -	/* MMOP_ONLINE_KEEP will always succeed and inherits the current zone */
> -	return online_type == MMOP_ONLINE_KEEP;
> -}
> -
> -static inline bool movable_pfn_range(int nid, struct zone *default_zone,
> -		unsigned long start_pfn, unsigned long nr_pages)
> -{
> -	if (!allow_online_pfn_range(nid, start_pfn, nr_pages,
> -				MMOP_ONLINE_KERNEL))
> -		return true;
> -
> -	if (!movable_node_is_enabled())
> -		return false;
> +	if (in_kernel ^ in_movable)
> +		return (in_kernel) ? kernel_zone : movable_zone;
>  
> -	return !zone_intersects(default_zone, start_pfn, nr_pages);
> +	/*
> +	 * If the range doesn't belong to any zone or two zones overlap in the
> +	 * given range then we use movable zone only if movable_node is
> +	 * enabled because we always online to a kernel zone by default.
> +	 */
> +	return movable_node_enabled ? movable_zone : kernel_zone;
>  }
>  
>  struct zone * zone_for_pfn_range(int online_type, int nid, unsigned start_pfn,
>  		unsigned long nr_pages)
>  {
> -	struct pglist_data *pgdat = NODE_DATA(nid);
> -	struct zone *zone = default_zone_for_pfn(nid, start_pfn, nr_pages);
> +	if (online_type == MMOP_ONLINE_KERNEL)
> +		return default_kernel_zone_for_pfn(nid, start_pfn, nr_pages);
>  
> -	if (online_type == MMOP_ONLINE_KEEP) {
> -		struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
> -		/*
> -		 * MMOP_ONLINE_KEEP defaults to MMOP_ONLINE_KERNEL but use
> -		 * movable zone if that is not possible (e.g. we are within
> -		 * or past the existing movable zone). movable_node overrides
> -		 * this default and defaults to movable zone
> -		 */
> -		if (movable_pfn_range(nid, zone, start_pfn, nr_pages))
> -			zone = movable_zone;
> -	} else if (online_type == MMOP_ONLINE_MOVABLE) {
> -		zone = &pgdat->node_zones[ZONE_MOVABLE];
> -	}
> +	if (online_type == MMOP_ONLINE_MOVABLE)
> +		return &NODE_DATA(nid)->node_zones[ZONE_MOVABLE];
>  
> -	return zone;
> +	return default_zone_for_pfn(nid, start_pfn, nr_pages);
>  }
>  
>  /*
> @@ -997,9 +972,6 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_typ
>  	struct memory_notify arg;
>  
>  	nid = pfn_to_nid(pfn);
> -	if (!allow_online_pfn_range(nid, pfn, nr_pages, online_type))
> -		return -EINVAL;
> -
>  	/* associate pfn range with the zone */
>  	zone = move_pfn_range(online_type, nid, pfn, nr_pages);
>  
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f199.google.com (mail-wr0-f199.google.com [209.85.128.199])
	by kanga.kvack.org (Postfix) with ESMTP id 278366B03B4
	for <linux-mm@kvack.org>; Mon, 10 Jul 2017 02:45:45 -0400 (EDT)
Received: by mail-wr0-f199.google.com with SMTP id g46so21925421wrd.3
        for <linux-mm@kvack.org>; Sun, 09 Jul 2017 23:45:45 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id g145si5512901wmg.198.2017.07.09.23.45.43
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Sun, 09 Jul 2017 23:45:43 -0700 (PDT)
Date: Mon, 10 Jul 2017 08:45:40 +0200
From: Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170710064540.GA19185@dhcp22.suse.cz>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <64e889ae-24ab-b845-5751-978a76dd0dd9@suse.cz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <64e889ae-24ab-b845-5751-978a76dd0dd9@suse.cz>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Linux API <linux-api@vger.kernel.org>

On Fri 07-07-17 17:02:59, Vlastimil Babka wrote:
> [+CC linux-api]
> 
> On 06/29/2017 09:35 AM, Michal Hocko wrote:
> > From: Michal Hocko <mhocko@suse.com>
> > 
> > Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
> > to precede the Movable zone in the physical memory range. The purpose of
> > the movable zone is, however, not bound to any physical memory restriction.
> > It merely defines a class of migrateable and reclaimable memory.
> > 
> > There are users (e.g. CMA) who might want to reserve specific physical
> > memory ranges for their own purpose. Moreover our pfn walkers have to be
> > prepared for zones overlapping in the physical range already because we
> > do support interleaving NUMA nodes and therefore zones can interleave as
> > well. This means we can allow each memory block to be associated with a
> > different zone.
> > 
> > Loosen the current onlining semantic and allow explicit onlining type on
> > any memblock. That means that online_{kernel,movable} will be allowed
> > regardless of the physical address of the memblock as long as it is
> > offline of course. This might result in moveble zone overlapping with
> > other kernel zones. Default onlining then becomes a bit tricky but still
> > sensible. echo online > memoryXY/state will online the given block to
> > 	1) the default zone if the given range is outside of any zone
> > 	2) the enclosing zone if such a zone doesn't interleave with
> > 	   any other zone
> >         3) the default zone if more zones interleave for this range
> > where default zone is movable zone only if movable_node is enabled
> > otherwise it is a kernel zone.
> > 
> > Here is an example of the semantic with (movable_node is not present but
> > it work in an analogous way). We start with following memblocks, all of
> > them offline
> > memory34/valid_zones:Normal Movable
> > memory35/valid_zones:Normal Movable
> > memory36/valid_zones:Normal Movable
> > memory37/valid_zones:Normal Movable
> > memory38/valid_zones:Normal Movable
> > memory39/valid_zones:Normal Movable
> > memory40/valid_zones:Normal Movable
> > memory41/valid_zones:Normal Movable
> > 
> > Now, we online block 34 in default mode and block 37 as movable
> > root@test1:/sys/devices/system/node/node1# echo online > memory34/state
> > root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
> > memory34/valid_zones:Normal
> > memory35/valid_zones:Normal Movable
> > memory36/valid_zones:Normal Movable
> > memory37/valid_zones:Movable
> > memory38/valid_zones:Normal Movable
> > memory39/valid_zones:Normal Movable
> > memory40/valid_zones:Normal Movable
> > memory41/valid_zones:Normal Movable
> 
> Hm so previously, blocks 37-41 would only allow Movable at this point, right?

yes

> Shouldn't we still default to Movable for them? We might be breaking some
> existing userspace here.

I do not think so. Prior to this merge window f1dd2cd13c4b ("mm,
memory_hotplug: do not associate hotadded memory to zones until online")
we allowed only the last offline or the adjacent to existing movable
memory block to be onlined movable. So the above wasn't possible. I
doubt we have grown a new user since the rework has been merged but if
you think we should make sure nothing like that happens then we should
probably merge this patch in this release cycle.

> IMHO onlining new memory past existing blocks is more common use case than
> onlining memory between two blocks that are already online?

I am not really sure. It is quite common to online and offline within an
existing zones for the memory ballooning. I do not know what kind of
online operation they use but using the default online operation has
historically preserved the zone so I would be really reluctant to change
that.

> I also agree with Wei Yang that it's rather fuzzy that a zone that has been
> completely offlined will affect the defaults for the next onlining just because
> it has some spanned range, which is however empty of actual populated memory.

I am sorry but I still do not see why. The zone is not empty. It has a
range spanned. It just doesn't have any pages online. I really fail to
see how that is different from zones with large offline holes.

> Maybe it would simplest for everyone to just default to Normal, except
> movable_node? That's if we decide that the potential breakage I
> described above is a non-issue.

This would break the usecase where the memory is onlined a certain type
initially and the offline/online it later on demand for ballooning.

I wish this could be more clear but the default onlining has been fuzzy
since the movable online has been introduced and it is hard to buil
something really clear since then. The proposed semantic is the most
clean I could come up with but I am open to any suggestions that
wouldn't break existing usage.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f200.google.com (mail-wr0-f200.google.com [209.85.128.200])
	by kanga.kvack.org (Postfix) with ESMTP id 0D4E7440844
	for <linux-mm@kvack.org>; Mon, 10 Jul 2017 07:12:20 -0400 (EDT)
Received: by mail-wr0-f200.google.com with SMTP id g46so23431606wrd.3
        for <linux-mm@kvack.org>; Mon, 10 Jul 2017 04:12:19 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id e84si1953117wme.194.2017.07.10.04.12.18
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Mon, 10 Jul 2017 04:12:18 -0700 (PDT)
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <64e889ae-24ab-b845-5751-978a76dd0dd9@suse.cz>
 <20170710064540.GA19185@dhcp22.suse.cz>
From: Vlastimil Babka <vbabka@suse.cz>
Message-ID: <24c3606d-837a-266d-a294-7e100d1430f0@suse.cz>
Date: Mon, 10 Jul 2017 13:11:29 +0200
MIME-Version: 1.0
In-Reply-To: <20170710064540.GA19185@dhcp22.suse.cz>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Linux API <linux-api@vger.kernel.org>

On 07/10/2017 08:45 AM, Michal Hocko wrote:
> On Fri 07-07-17 17:02:59, Vlastimil Babka wrote:
>> [+CC linux-api]
>>
>> On 06/29/2017 09:35 AM, Michal Hocko wrote:
>>> From: Michal Hocko <mhocko@suse.com>
>>>
>>> Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
>>> to precede the Movable zone in the physical memory range. The purpose of
>>> the movable zone is, however, not bound to any physical memory restriction.
>>> It merely defines a class of migrateable and reclaimable memory.
>>>
>>> There are users (e.g. CMA) who might want to reserve specific physical
>>> memory ranges for their own purpose. Moreover our pfn walkers have to be
>>> prepared for zones overlapping in the physical range already because we
>>> do support interleaving NUMA nodes and therefore zones can interleave as
>>> well. This means we can allow each memory block to be associated with a
>>> different zone.
>>>
>>> Loosen the current onlining semantic and allow explicit onlining type on
>>> any memblock. That means that online_{kernel,movable} will be allowed
>>> regardless of the physical address of the memblock as long as it is
>>> offline of course. This might result in moveble zone overlapping with
>>> other kernel zones. Default onlining then becomes a bit tricky but still
>>> sensible. echo online > memoryXY/state will online the given block to
>>> 	1) the default zone if the given range is outside of any zone
>>> 	2) the enclosing zone if such a zone doesn't interleave with
>>> 	   any other zone
>>>         3) the default zone if more zones interleave for this range
>>> where default zone is movable zone only if movable_node is enabled
>>> otherwise it is a kernel zone.
>>>
>>> Here is an example of the semantic with (movable_node is not present but
>>> it work in an analogous way). We start with following memblocks, all of
>>> them offline
>>> memory34/valid_zones:Normal Movable
>>> memory35/valid_zones:Normal Movable
>>> memory36/valid_zones:Normal Movable
>>> memory37/valid_zones:Normal Movable
>>> memory38/valid_zones:Normal Movable
>>> memory39/valid_zones:Normal Movable
>>> memory40/valid_zones:Normal Movable
>>> memory41/valid_zones:Normal Movable
>>>
>>> Now, we online block 34 in default mode and block 37 as movable
>>> root@test1:/sys/devices/system/node/node1# echo online > memory34/state
>>> root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
>>> memory34/valid_zones:Normal
>>> memory35/valid_zones:Normal Movable
>>> memory36/valid_zones:Normal Movable
>>> memory37/valid_zones:Movable
>>> memory38/valid_zones:Normal Movable
>>> memory39/valid_zones:Normal Movable
>>> memory40/valid_zones:Normal Movable
>>> memory41/valid_zones:Normal Movable
>>
>> Hm so previously, blocks 37-41 would only allow Movable at this point, right?
> 
> yes
> 
>> Shouldn't we still default to Movable for them? We might be breaking some
>> existing userspace here.
> 
> I do not think so. Prior to this merge window f1dd2cd13c4b ("mm,
> memory_hotplug: do not associate hotadded memory to zones until online")
> we allowed only the last offline or the adjacent to existing movable
> memory block to be onlined movable. So the above wasn't possible.

Not exactly the above, but let's say 1-34 is onlined as Normal, 35-37 is
Movable. Then the only possible action before would be online 38 as
Movable? Now it defaults to Normal?

> I
> doubt we have grown a new user since the rework has been merged but if
> you think we should make sure nothing like that happens then we should
> probably merge this patch in this release cycle.

If I'm right and this is a change compared to pre-rework, then it
doesn't matter.

>> IMHO onlining new memory past existing blocks is more common use case than
>> onlining memory between two blocks that are already online?
> 
> I am not really sure. It is quite common to online and offline within an
> existing zones for the memory ballooning. I do not know what kind of
> online operation they use but using the default online operation has
> historically preserved the zone so I would be really reluctant to change
> that.

Hmm all right, ballooning...

>> I also agree with Wei Yang that it's rather fuzzy that a zone that has been
>> completely offlined will affect the defaults for the next onlining just because
>> it has some spanned range, which is however empty of actual populated memory.
> 
> I am sorry but I still do not see why. The zone is not empty. It has a
> range spanned. It just doesn't have any pages online. I really fail to
> see how that is different from zones with large offline holes.
> 
>> Maybe it would simplest for everyone to just default to Normal, except
>> movable_node? That's if we decide that the potential breakage I
>> described above is a non-issue.
> 
> This would break the usecase where the memory is onlined a certain type
> initially and the offline/online it later on demand for ballooning.
> 
> I wish this could be more clear but the default onlining has been fuzzy
> since the movable online has been introduced and it is hard to buil
> something really clear since then. The proposed semantic is the most
> clean I could come up with but I am open to any suggestions that
> wouldn't break existing usage.

OK I can live with the semantics, if we clear question of breaking
existing users.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f198.google.com (mail-wr0-f198.google.com [209.85.128.198])
	by kanga.kvack.org (Postfix) with ESMTP id 0B224440844
	for <linux-mm@kvack.org>; Mon, 10 Jul 2017 07:17:54 -0400 (EDT)
Received: by mail-wr0-f198.google.com with SMTP id x23so23447477wrb.6
        for <linux-mm@kvack.org>; Mon, 10 Jul 2017 04:17:53 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id x4si6179041wmg.27.2017.07.10.04.17.52
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Mon, 10 Jul 2017 04:17:52 -0700 (PDT)
Date: Mon, 10 Jul 2017 13:17:50 +0200
From: Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170710111750.GG19185@dhcp22.suse.cz>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <64e889ae-24ab-b845-5751-978a76dd0dd9@suse.cz>
 <20170710064540.GA19185@dhcp22.suse.cz>
 <24c3606d-837a-266d-a294-7e100d1430f0@suse.cz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <24c3606d-837a-266d-a294-7e100d1430f0@suse.cz>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Linux API <linux-api@vger.kernel.org>

On Mon 10-07-17 13:11:29, Vlastimil Babka wrote:
> On 07/10/2017 08:45 AM, Michal Hocko wrote:
> > On Fri 07-07-17 17:02:59, Vlastimil Babka wrote:
> >> [+CC linux-api]
> >>
> >> On 06/29/2017 09:35 AM, Michal Hocko wrote:
> >>> From: Michal Hocko <mhocko@suse.com>
> >>>
> >>> Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
> >>> to precede the Movable zone in the physical memory range. The purpose of
> >>> the movable zone is, however, not bound to any physical memory restriction.
> >>> It merely defines a class of migrateable and reclaimable memory.
> >>>
> >>> There are users (e.g. CMA) who might want to reserve specific physical
> >>> memory ranges for their own purpose. Moreover our pfn walkers have to be
> >>> prepared for zones overlapping in the physical range already because we
> >>> do support interleaving NUMA nodes and therefore zones can interleave as
> >>> well. This means we can allow each memory block to be associated with a
> >>> different zone.
> >>>
> >>> Loosen the current onlining semantic and allow explicit onlining type on
> >>> any memblock. That means that online_{kernel,movable} will be allowed
> >>> regardless of the physical address of the memblock as long as it is
> >>> offline of course. This might result in moveble zone overlapping with
> >>> other kernel zones. Default onlining then becomes a bit tricky but still
> >>> sensible. echo online > memoryXY/state will online the given block to
> >>> 	1) the default zone if the given range is outside of any zone
> >>> 	2) the enclosing zone if such a zone doesn't interleave with
> >>> 	   any other zone
> >>>         3) the default zone if more zones interleave for this range
> >>> where default zone is movable zone only if movable_node is enabled
> >>> otherwise it is a kernel zone.
> >>>
> >>> Here is an example of the semantic with (movable_node is not present but
> >>> it work in an analogous way). We start with following memblocks, all of
> >>> them offline
> >>> memory34/valid_zones:Normal Movable
> >>> memory35/valid_zones:Normal Movable
> >>> memory36/valid_zones:Normal Movable
> >>> memory37/valid_zones:Normal Movable
> >>> memory38/valid_zones:Normal Movable
> >>> memory39/valid_zones:Normal Movable
> >>> memory40/valid_zones:Normal Movable
> >>> memory41/valid_zones:Normal Movable
> >>>
> >>> Now, we online block 34 in default mode and block 37 as movable
> >>> root@test1:/sys/devices/system/node/node1# echo online > memory34/state
> >>> root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
> >>> memory34/valid_zones:Normal
> >>> memory35/valid_zones:Normal Movable
> >>> memory36/valid_zones:Normal Movable
> >>> memory37/valid_zones:Movable
> >>> memory38/valid_zones:Normal Movable
> >>> memory39/valid_zones:Normal Movable
> >>> memory40/valid_zones:Normal Movable
> >>> memory41/valid_zones:Normal Movable
> >>
> >> Hm so previously, blocks 37-41 would only allow Movable at this point, right?
> > 
> > yes
> > 
> >> Shouldn't we still default to Movable for them? We might be breaking some
> >> existing userspace here.
> > 
> > I do not think so. Prior to this merge window f1dd2cd13c4b ("mm,
> > memory_hotplug: do not associate hotadded memory to zones until online")
> > we allowed only the last offline or the adjacent to existing movable
> > memory block to be onlined movable. So the above wasn't possible.
> 
> Not exactly the above, but let's say 1-34 is onlined as Normal, 35-37 is
> Movable. Then the only possible action before would be online 38 as
> Movable? Now it defaults to Normal?

Yes. And let me repeat you couldn't onlne 35-37 as movable before. So no
userspace could depend on that before the rework. Or do I still miss
your point?
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f199.google.com (mail-wr0-f199.google.com [209.85.128.199])
	by kanga.kvack.org (Postfix) with ESMTP id 4F4C36B0493
	for <linux-mm@kvack.org>; Mon, 10 Jul 2017 08:12:59 -0400 (EDT)
Received: by mail-wr0-f199.google.com with SMTP id z81so23770078wrc.2
        for <linux-mm@kvack.org>; Mon, 10 Jul 2017 05:12:59 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id u27si7885379wru.142.2017.07.10.05.12.57
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Mon, 10 Jul 2017 05:12:58 -0700 (PDT)
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <64e889ae-24ab-b845-5751-978a76dd0dd9@suse.cz>
 <20170710064540.GA19185@dhcp22.suse.cz>
 <24c3606d-837a-266d-a294-7e100d1430f0@suse.cz>
 <20170710111750.GG19185@dhcp22.suse.cz>
From: Vlastimil Babka <vbabka@suse.cz>
Message-ID: <cd2352fb-9831-d0f1-5cb8-b081b292eb38@suse.cz>
Date: Mon, 10 Jul 2017 14:12:09 +0200
MIME-Version: 1.0
In-Reply-To: <20170710111750.GG19185@dhcp22.suse.cz>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Linux API <linux-api@vger.kernel.org>

On 07/10/2017 01:17 PM, Michal Hocko wrote:
> On Mon 10-07-17 13:11:29, Vlastimil Babka wrote:
>> On 07/10/2017 08:45 AM, Michal Hocko wrote:
>>> On Fri 07-07-17 17:02:59, Vlastimil Babka wrote:
>>>> [+CC linux-api]
>>>>
>>>>
>>>> Hm so previously, blocks 37-41 would only allow Movable at this point, right?
>>>
>>> yes
>>>
>>>> Shouldn't we still default to Movable for them? We might be breaking some
>>>> existing userspace here.
>>>
>>> I do not think so. Prior to this merge window f1dd2cd13c4b ("mm,
>>> memory_hotplug: do not associate hotadded memory to zones until online")
>>> we allowed only the last offline or the adjacent to existing movable
>>> memory block to be onlined movable. So the above wasn't possible.
>>
>> Not exactly the above, but let's say 1-34 is onlined as Normal, 35-37 is
>> Movable. Then the only possible action before would be online 38 as
>> Movable? Now it defaults to Normal?
> 
> Yes. And let me repeat you couldn't onlne 35-37 as movable before. So no
> userspace could depend on that before the rework. Or do I still miss
> your point?

Ah, I see. "the last offline or the adjacent to existing movable". OK then.

It would be indeed better to not change behavour twice then and merge
this to 4.13, but it's the middle of merge window, so it's not simple...

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f199.google.com (mail-wr0-f199.google.com [209.85.128.199])
	by kanga.kvack.org (Postfix) with ESMTP id 159716B049A
	for <linux-mm@kvack.org>; Mon, 10 Jul 2017 08:30:32 -0400 (EDT)
Received: by mail-wr0-f199.google.com with SMTP id u30so23820118wrc.9
        for <linux-mm@kvack.org>; Mon, 10 Jul 2017 05:30:32 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id s74si6430735wmb.51.2017.07.10.05.30.30
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Mon, 10 Jul 2017 05:30:31 -0700 (PDT)
Date: Mon, 10 Jul 2017 14:30:28 +0200
From: Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170710123028.GH19185@dhcp22.suse.cz>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
 <64e889ae-24ab-b845-5751-978a76dd0dd9@suse.cz>
 <20170710064540.GA19185@dhcp22.suse.cz>
 <24c3606d-837a-266d-a294-7e100d1430f0@suse.cz>
 <20170710111750.GG19185@dhcp22.suse.cz>
 <cd2352fb-9831-d0f1-5cb8-b081b292eb38@suse.cz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <cd2352fb-9831-d0f1-5cb8-b081b292eb38@suse.cz>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>, Linux API <linux-api@vger.kernel.org>

On Mon 10-07-17 14:12:09, Vlastimil Babka wrote:
> On 07/10/2017 01:17 PM, Michal Hocko wrote:
> > On Mon 10-07-17 13:11:29, Vlastimil Babka wrote:
> >> On 07/10/2017 08:45 AM, Michal Hocko wrote:
> >>> On Fri 07-07-17 17:02:59, Vlastimil Babka wrote:
> >>>> [+CC linux-api]
> >>>>
> >>>>
> >>>> Hm so previously, blocks 37-41 would only allow Movable at this point, right?
> >>>
> >>> yes
> >>>
> >>>> Shouldn't we still default to Movable for them? We might be breaking some
> >>>> existing userspace here.
> >>>
> >>> I do not think so. Prior to this merge window f1dd2cd13c4b ("mm,
> >>> memory_hotplug: do not associate hotadded memory to zones until online")
> >>> we allowed only the last offline or the adjacent to existing movable
> >>> memory block to be onlined movable. So the above wasn't possible.
> >>
> >> Not exactly the above, but let's say 1-34 is onlined as Normal, 35-37 is
> >> Movable. Then the only possible action before would be online 38 as
> >> Movable? Now it defaults to Normal?
> > 
> > Yes. And let me repeat you couldn't onlne 35-37 as movable before. So no
> > userspace could depend on that before the rework. Or do I still miss
> > your point?
> 
> Ah, I see. "the last offline or the adjacent to existing movable". OK then.
> 
> It would be indeed better to not change behavour twice then and merge
> this to 4.13, but it's the middle of merge window, so it's not simple...

yeah. I was thinking about about how to make the change reasonably
incremental but failed to find a way. I also didn't want to bring too
many changes at once (the code base is just too fragile already).

If there is a general consensus about the semantic we might want to push
the patch this week. I just do not want to rush it too much as this is a
users visible change and it might kick us back in future.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f200.google.com (mail-wr0-f200.google.com [209.85.128.200])
	by kanga.kvack.org (Postfix) with ESMTP id 03C676B0539
	for <linux-mm@kvack.org>; Wed, 12 Jul 2017 08:49:38 -0400 (EDT)
Received: by mail-wr0-f200.google.com with SMTP id 23so5358606wry.4
        for <linux-mm@kvack.org>; Wed, 12 Jul 2017 05:49:37 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id k31si1800083wrk.167.2017.07.12.05.49.36
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Wed, 12 Jul 2017 05:49:36 -0700 (PDT)
Date: Wed, 12 Jul 2017 14:49:33 +0200
From: Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Message-ID: <20170712124933.GK28912@dhcp22.suse.cz>
References: <20170629073509.623-1-mhocko@kernel.org>
 <20170629073509.623-3-mhocko@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170629073509.623-3-mhocko@kernel.org>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, LKML <linux-kernel@vger.kernel.org>

Are there any other concerns regarding this patch? Can I repost it for
inclusion?

On Thu 29-06-17 09:35:09, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
> 
> Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
> to precede the Movable zone in the physical memory range. The purpose of
> the movable zone is, however, not bound to any physical memory restriction.
> It merely defines a class of migrateable and reclaimable memory.
> 
> There are users (e.g. CMA) who might want to reserve specific physical
> memory ranges for their own purpose. Moreover our pfn walkers have to be
> prepared for zones overlapping in the physical range already because we
> do support interleaving NUMA nodes and therefore zones can interleave as
> well. This means we can allow each memory block to be associated with a
> different zone.
> 
> Loosen the current onlining semantic and allow explicit onlining type on
> any memblock. That means that online_{kernel,movable} will be allowed
> regardless of the physical address of the memblock as long as it is
> offline of course. This might result in moveble zone overlapping with
> other kernel zones. Default onlining then becomes a bit tricky but still
> sensible. echo online > memoryXY/state will online the given block to
> 	1) the default zone if the given range is outside of any zone
> 	2) the enclosing zone if such a zone doesn't interleave with
> 	   any other zone
>         3) the default zone if more zones interleave for this range
> where default zone is movable zone only if movable_node is enabled
> otherwise it is a kernel zone.
> 
> Here is an example of the semantic with (movable_node is not present but
> it work in an analogous way). We start with following memblocks, all of
> them offline
> memory34/valid_zones:Normal Movable
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Normal Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal Movable
> memory40/valid_zones:Normal Movable
> memory41/valid_zones:Normal Movable
> 
> Now, we online block 34 in default mode and block 37 as movable
> root@test1:/sys/devices/system/node/node1# echo online > memory34/state
> root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal Movable
> memory40/valid_zones:Normal Movable
> memory41/valid_zones:Normal Movable
> 
> As we can see all other blocks can still be onlined both into Normal and
> Movable zones and the Normal is default because the Movable zone spans
> only block37 now.
> root@test1:/sys/devices/system/node/node1# echo online_movable > memory41/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Movable Normal
> memory39/valid_zones:Movable Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable
> 
> Now the default zone for blocks 37-41 has changed because movable zone
> spans that range.
> root@test1:/sys/devices/system/node/node1# echo online_kernel > memory39/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable
> 
> Note that the block 39 now belongs to the zone Normal and so block38
> falls into Normal by default as well.
> 
> For completness
> root@test1:/sys/devices/system/node/node1# for i in memory[34]?
> do
> 	echo online > $i/state 2>/dev/null
> done
> 
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal
> memory36/valid_zones:Normal
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable
> memory41/valid_zones:Movable
> 
> Implementation wise the change is quite straightforward. We can get rid
> of allow_online_pfn_range altogether. online_pages allows only offline
> nodes already. The original default_zone_for_pfn will become
> default_kernel_zone_for_pfn. New default_zone_for_pfn implements the
> above semantic. zone_for_pfn_range is slightly reorganized to implement
> kernel and movable online type explicitly and MMOP_ONLINE_KEEP becomes
> a catch all default behavior.
> 
> Signed-off-by: Michal Hocko <mhocko@suse.com>
> ---
>  drivers/base/memory.c |  3 ---
>  mm/memory_hotplug.c   | 74 ++++++++++++++++-----------------------------------
>  2 files changed, 23 insertions(+), 54 deletions(-)
> 
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index 26383af9900c..4e3b61cda520 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -394,9 +394,6 @@ static void print_allowed_zone(char *buf, int nid, unsigned long start_pfn,
>  {
>  	struct zone *zone;
>  
> -	if (!allow_online_pfn_range(nid, start_pfn, nr_pages, online_type))
> -		return;
> -
>  	zone = zone_for_pfn_range(online_type, nid, start_pfn, nr_pages);
>  	if (zone != default_zone) {
>  		strcat(buf, " ");
> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index 6b9a60115e37..670f7acbecf4 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -894,7 +894,7 @@ void __ref move_pfn_range_to_zone(struct zone *zone,
>   * If no kernel zone covers this pfn range it will automatically go
>   * to the ZONE_NORMAL.
>   */
> -static struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
> +static struct zone *default_kernel_zone_for_pfn(int nid, unsigned long start_pfn,
>  		unsigned long nr_pages)
>  {
>  	struct pglist_data *pgdat = NODE_DATA(nid);
> @@ -910,65 +910,40 @@ static struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
>  	return &pgdat->node_zones[ZONE_NORMAL];
>  }
>  
> -bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages, int online_type)
> +static inline struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
> +		unsigned long nr_pages)
>  {
> -	struct pglist_data *pgdat = NODE_DATA(nid);
> -	struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
> -	struct zone *default_zone = default_zone_for_pfn(nid, pfn, nr_pages);
> +	struct zone *kernel_zone = default_kernel_zone_for_pfn(nid, start_pfn,
> +			nr_pages);
> +	struct zone *movable_zone = &NODE_DATA(nid)->node_zones[ZONE_MOVABLE];
> +	bool in_kernel = zone_intersects(kernel_zone, start_pfn, nr_pages);
> +	bool in_movable = zone_intersects(movable_zone, start_pfn, nr_pages);
>  
>  	/*
> -	 * TODO there shouldn't be any inherent reason to have ZONE_NORMAL
> -	 * physically before ZONE_MOVABLE. All we need is they do not
> -	 * overlap. Historically we didn't allow ZONE_NORMAL after ZONE_MOVABLE
> -	 * though so let's stick with it for simplicity for now.
> -	 * TODO make sure we do not overlap with ZONE_DEVICE
> +	 * We inherit the existing zone in a simple case where zones do not
> +	 * overlap in the given range
>  	 */
> -	if (online_type == MMOP_ONLINE_KERNEL) {
> -		if (zone_is_empty(movable_zone))
> -			return true;
> -		return movable_zone->zone_start_pfn >= pfn + nr_pages;
> -	} else if (online_type == MMOP_ONLINE_MOVABLE) {
> -		return zone_end_pfn(default_zone) <= pfn;
> -	}
> -
> -	/* MMOP_ONLINE_KEEP will always succeed and inherits the current zone */
> -	return online_type == MMOP_ONLINE_KEEP;
> -}
> -
> -static inline bool movable_pfn_range(int nid, struct zone *default_zone,
> -		unsigned long start_pfn, unsigned long nr_pages)
> -{
> -	if (!allow_online_pfn_range(nid, start_pfn, nr_pages,
> -				MMOP_ONLINE_KERNEL))
> -		return true;
> -
> -	if (!movable_node_is_enabled())
> -		return false;
> +	if (in_kernel ^ in_movable)
> +		return (in_kernel) ? kernel_zone : movable_zone;
>  
> -	return !zone_intersects(default_zone, start_pfn, nr_pages);
> +	/*
> +	 * If the range doesn't belong to any zone or two zones overlap in the
> +	 * given range then we use movable zone only if movable_node is
> +	 * enabled because we always online to a kernel zone by default.
> +	 */
> +	return movable_node_enabled ? movable_zone : kernel_zone;
>  }
>  
>  struct zone * zone_for_pfn_range(int online_type, int nid, unsigned start_pfn,
>  		unsigned long nr_pages)
>  {
> -	struct pglist_data *pgdat = NODE_DATA(nid);
> -	struct zone *zone = default_zone_for_pfn(nid, start_pfn, nr_pages);
> +	if (online_type == MMOP_ONLINE_KERNEL)
> +		return default_kernel_zone_for_pfn(nid, start_pfn, nr_pages);
>  
> -	if (online_type == MMOP_ONLINE_KEEP) {
> -		struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
> -		/*
> -		 * MMOP_ONLINE_KEEP defaults to MMOP_ONLINE_KERNEL but use
> -		 * movable zone if that is not possible (e.g. we are within
> -		 * or past the existing movable zone). movable_node overrides
> -		 * this default and defaults to movable zone
> -		 */
> -		if (movable_pfn_range(nid, zone, start_pfn, nr_pages))
> -			zone = movable_zone;
> -	} else if (online_type == MMOP_ONLINE_MOVABLE) {
> -		zone = &pgdat->node_zones[ZONE_MOVABLE];
> -	}
> +	if (online_type == MMOP_ONLINE_MOVABLE)
> +		return &NODE_DATA(nid)->node_zones[ZONE_MOVABLE];
>  
> -	return zone;
> +	return default_zone_for_pfn(nid, start_pfn, nr_pages);
>  }
>  
>  /*
> @@ -997,9 +972,6 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_typ
>  	struct memory_notify arg;
>  
>  	nid = pfn_to_nid(pfn);
> -	if (!allow_online_pfn_range(nid, pfn, nr_pages, online_type))
> -		return -EINVAL;
> -
>  	/* associate pfn range with the zone */
>  	zone = move_pfn_range(online_type, nid, pfn, nr_pages);
>  
> -- 
> 2.11.0
> 

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wm0-f70.google.com (mail-wm0-f70.google.com [74.125.82.70])
	by kanga.kvack.org (Postfix) with ESMTP id A228D440905
	for <linux-mm@kvack.org>; Fri, 14 Jul 2017 08:14:58 -0400 (EDT)
Received: by mail-wm0-f70.google.com with SMTP id l81so8950646wmg.8
        for <linux-mm@kvack.org>; Fri, 14 Jul 2017 05:14:58 -0700 (PDT)
Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com. [74.125.82.66])
        by mx.google.com with ESMTPS id v69si2184718wmv.1.2017.07.14.05.14.57
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Fri, 14 Jul 2017 05:14:57 -0700 (PDT)
Received: by mail-wm0-f66.google.com with SMTP id u23so10529388wma.2
        for <linux-mm@kvack.org>; Fri, 14 Jul 2017 05:14:57 -0700 (PDT)
From: Michal Hocko <mhocko@kernel.org>
Subject: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
Date: Fri, 14 Jul 2017 14:12:33 +0200
Message-Id: <20170714121233.16861-3-mhocko@kernel.org>
In-Reply-To: <20170714121233.16861-1-mhocko@kernel.org>
References: <20170714121233.16861-1-mhocko@kernel.org>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>, Michal Hocko <mhocko@suse.com>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, linux-api@vger.kernel.org

From: Michal Hocko <mhocko@suse.com>

Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
to precede the Movable zone in the physical memory range. The purpose of
the movable zone is, however, not bound to any physical memory restriction.
It merely defines a class of migrateable and reclaimable memory.

There are users (e.g. CMA) who might want to reserve specific physical
memory ranges for their own purpose. Moreover our pfn walkers have to be
prepared for zones overlapping in the physical range already because we
do support interleaving NUMA nodes and therefore zones can interleave as
well. This means we can allow each memory block to be associated with a
different zone.

Loosen the current onlining semantic and allow explicit onlining type on
any memblock. That means that online_{kernel,movable} will be allowed
regardless of the physical address of the memblock as long as it is
offline of course. This might result in moveble zone overlapping with
other kernel zones. Default onlining then becomes a bit tricky but still
sensible. echo online > memoryXY/state will online the given block to
	1) the default zone if the given range is outside of any zone
	2) the enclosing zone if such a zone doesn't interleave with
	   any other zone
        3) the default zone if more zones interleave for this range
where default zone is movable zone only if movable_node is enabled
otherwise it is a kernel zone.

Here is an example of the semantic with (movable_node is not present but
it work in an analogous way). We start with following memblocks, all of
them offline
memory34/valid_zones:Normal Movable
memory35/valid_zones:Normal Movable
memory36/valid_zones:Normal Movable
memory37/valid_zones:Normal Movable
memory38/valid_zones:Normal Movable
memory39/valid_zones:Normal Movable
memory40/valid_zones:Normal Movable
memory41/valid_zones:Normal Movable

Now, we online block 34 in default mode and block 37 as movable
root@test1:/sys/devices/system/node/node1# echo online > memory34/state
root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
memory34/valid_zones:Normal
memory35/valid_zones:Normal Movable
memory36/valid_zones:Normal Movable
memory37/valid_zones:Movable
memory38/valid_zones:Normal Movable
memory39/valid_zones:Normal Movable
memory40/valid_zones:Normal Movable
memory41/valid_zones:Normal Movable

As we can see all other blocks can still be onlined both into Normal and
Movable zones and the Normal is default because the Movable zone spans
only block37 now.
root@test1:/sys/devices/system/node/node1# echo online_movable > memory41/state
memory34/valid_zones:Normal
memory35/valid_zones:Normal Movable
memory36/valid_zones:Normal Movable
memory37/valid_zones:Movable
memory38/valid_zones:Movable Normal
memory39/valid_zones:Movable Normal
memory40/valid_zones:Movable Normal
memory41/valid_zones:Movable

Now the default zone for blocks 37-41 has changed because movable zone
spans that range.
root@test1:/sys/devices/system/node/node1# echo online_kernel > memory39/state
memory34/valid_zones:Normal
memory35/valid_zones:Normal Movable
memory36/valid_zones:Normal Movable
memory37/valid_zones:Movable
memory38/valid_zones:Normal Movable
memory39/valid_zones:Normal
memory40/valid_zones:Movable Normal
memory41/valid_zones:Movable

Note that the block 39 now belongs to the zone Normal and so block38
falls into Normal by default as well.

For completness
root@test1:/sys/devices/system/node/node1# for i in memory[34]?
do
	echo online > $i/state 2>/dev/null
done

memory34/valid_zones:Normal
memory35/valid_zones:Normal
memory36/valid_zones:Normal
memory37/valid_zones:Movable
memory38/valid_zones:Normal
memory39/valid_zones:Normal
memory40/valid_zones:Movable
memory41/valid_zones:Movable

Implementation wise the change is quite straightforward. We can get rid
of allow_online_pfn_range altogether. online_pages allows only offline
nodes already. The original default_zone_for_pfn will become
default_kernel_zone_for_pfn. New default_zone_for_pfn implements the
above semantic. zone_for_pfn_range is slightly reorganized to implement
kernel and movable online type explicitly and MMOP_ONLINE_KEEP becomes
a catch all default behavior.

Acked-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: <linux-api@vger.kernel.org>
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
 drivers/base/memory.c |  3 ---
 mm/memory_hotplug.c   | 74 ++++++++++++++++-----------------------------------
 2 files changed, 23 insertions(+), 54 deletions(-)

diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 26383af9900c..4e3b61cda520 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -394,9 +394,6 @@ static void print_allowed_zone(char *buf, int nid, unsigned long start_pfn,
 {
 	struct zone *zone;
 
-	if (!allow_online_pfn_range(nid, start_pfn, nr_pages, online_type))
-		return;
-
 	zone = zone_for_pfn_range(online_type, nid, start_pfn, nr_pages);
 	if (zone != default_zone) {
 		strcat(buf, " ");
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index b4f2583677b1..d8b771b1ae29 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -835,7 +835,7 @@ void __ref move_pfn_range_to_zone(struct zone *zone,
  * If no kernel zone covers this pfn range it will automatically go
  * to the ZONE_NORMAL.
  */
-static struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
+static struct zone *default_kernel_zone_for_pfn(int nid, unsigned long start_pfn,
 		unsigned long nr_pages)
 {
 	struct pglist_data *pgdat = NODE_DATA(nid);
@@ -851,65 +851,40 @@ static struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
 	return &pgdat->node_zones[ZONE_NORMAL];
 }
 
-bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages, int online_type)
+static inline struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn,
+		unsigned long nr_pages)
 {
-	struct pglist_data *pgdat = NODE_DATA(nid);
-	struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
-	struct zone *default_zone = default_zone_for_pfn(nid, pfn, nr_pages);
+	struct zone *kernel_zone = default_kernel_zone_for_pfn(nid, start_pfn,
+			nr_pages);
+	struct zone *movable_zone = &NODE_DATA(nid)->node_zones[ZONE_MOVABLE];
+	bool in_kernel = zone_intersects(kernel_zone, start_pfn, nr_pages);
+	bool in_movable = zone_intersects(movable_zone, start_pfn, nr_pages);
 
 	/*
-	 * TODO there shouldn't be any inherent reason to have ZONE_NORMAL
-	 * physically before ZONE_MOVABLE. All we need is they do not
-	 * overlap. Historically we didn't allow ZONE_NORMAL after ZONE_MOVABLE
-	 * though so let's stick with it for simplicity for now.
-	 * TODO make sure we do not overlap with ZONE_DEVICE
+	 * We inherit the existing zone in a simple case where zones do not
+	 * overlap in the given range
 	 */
-	if (online_type == MMOP_ONLINE_KERNEL) {
-		if (zone_is_empty(movable_zone))
-			return true;
-		return movable_zone->zone_start_pfn >= pfn + nr_pages;
-	} else if (online_type == MMOP_ONLINE_MOVABLE) {
-		return zone_end_pfn(default_zone) <= pfn;
-	}
-
-	/* MMOP_ONLINE_KEEP will always succeed and inherits the current zone */
-	return online_type == MMOP_ONLINE_KEEP;
-}
-
-static inline bool movable_pfn_range(int nid, struct zone *default_zone,
-		unsigned long start_pfn, unsigned long nr_pages)
-{
-	if (!allow_online_pfn_range(nid, start_pfn, nr_pages,
-				MMOP_ONLINE_KERNEL))
-		return true;
-
-	if (!movable_node_is_enabled())
-		return false;
+	if (in_kernel ^ in_movable)
+		return (in_kernel) ? kernel_zone : movable_zone;
 
-	return !zone_intersects(default_zone, start_pfn, nr_pages);
+	/*
+	 * If the range doesn't belong to any zone or two zones overlap in the
+	 * given range then we use movable zone only if movable_node is
+	 * enabled because we always online to a kernel zone by default.
+	 */
+	return movable_node_enabled ? movable_zone : kernel_zone;
 }
 
 struct zone * zone_for_pfn_range(int online_type, int nid, unsigned start_pfn,
 		unsigned long nr_pages)
 {
-	struct pglist_data *pgdat = NODE_DATA(nid);
-	struct zone *zone = default_zone_for_pfn(nid, start_pfn, nr_pages);
+	if (online_type == MMOP_ONLINE_KERNEL)
+		return default_kernel_zone_for_pfn(nid, start_pfn, nr_pages);
 
-	if (online_type == MMOP_ONLINE_KEEP) {
-		struct zone *movable_zone = &pgdat->node_zones[ZONE_MOVABLE];
-		/*
-		 * MMOP_ONLINE_KEEP defaults to MMOP_ONLINE_KERNEL but use
-		 * movable zone if that is not possible (e.g. we are within
-		 * or past the existing movable zone). movable_node overrides
-		 * this default and defaults to movable zone
-		 */
-		if (movable_pfn_range(nid, zone, start_pfn, nr_pages))
-			zone = movable_zone;
-	} else if (online_type == MMOP_ONLINE_MOVABLE) {
-		zone = &pgdat->node_zones[ZONE_MOVABLE];
-	}
+	if (online_type == MMOP_ONLINE_MOVABLE)
+		return &NODE_DATA(nid)->node_zones[ZONE_MOVABLE];
 
-	return zone;
+	return default_zone_for_pfn(nid, start_pfn, nr_pages);
 }
 
 /*
@@ -938,9 +913,6 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_typ
 	struct memory_notify arg;
 
 	nid = pfn_to_nid(pfn);
-	if (!allow_online_pfn_range(nid, pfn, nr_pages, online_type))
-		return -EINVAL;
-
 	/* associate pfn range with the zone */
 	zone = move_pfn_range(online_type, nid, pfn, nr_pages);
 
-- 
2.11.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-wr0-f198.google.com (mail-wr0-f198.google.com [209.85.128.198])
	by kanga.kvack.org (Postfix) with ESMTP id A9372440905
	for <linux-mm@kvack.org>; Fri, 14 Jul 2017 08:18:00 -0400 (EDT)
Received: by mail-wr0-f198.google.com with SMTP id z81so11150966wrc.2
        for <linux-mm@kvack.org>; Fri, 14 Jul 2017 05:18:00 -0700 (PDT)
Received: from mx1.suse.de (mx2.suse.de. [195.135.220.15])
        by mx.google.com with ESMTPS id t72si2152379wmt.135.2017.07.14.05.17.59
        for <linux-mm@kvack.org>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Fri, 14 Jul 2017 05:17:59 -0700 (PDT)
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
References: <20170714121233.16861-1-mhocko@kernel.org>
 <20170714121233.16861-3-mhocko@kernel.org>
From: Vlastimil Babka <vbabka@suse.cz>
Message-ID: <c65abd07-f024-6c66-fa55-3c23926a6ebe@suse.cz>
Date: Fri, 14 Jul 2017 14:17:55 +0200
MIME-Version: 1.0
In-Reply-To: <20170714121233.16861-3-mhocko@kernel.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>, Andrew Morton <akpm@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>, Andrea Arcangeli <aarcange@redhat.com>, Reza Arbab <arbab@linux.vnet.ibm.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>, Michal Hocko <mhocko@suse.com>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, linux-api@vger.kernel.org

On 07/14/2017 02:12 PM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
> 
> Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
> to precede the Movable zone in the physical memory range. The purpose of
> the movable zone is, however, not bound to any physical memory restriction.
> It merely defines a class of migrateable and reclaimable memory.
> 
> There are users (e.g. CMA) who might want to reserve specific physical
> memory ranges for their own purpose. Moreover our pfn walkers have to be
> prepared for zones overlapping in the physical range already because we
> do support interleaving NUMA nodes and therefore zones can interleave as
> well. This means we can allow each memory block to be associated with a
> different zone.
> 
> Loosen the current onlining semantic and allow explicit onlining type on
> any memblock. That means that online_{kernel,movable} will be allowed
> regardless of the physical address of the memblock as long as it is
> offline of course. This might result in moveble zone overlapping with
> other kernel zones. Default onlining then becomes a bit tricky but still
> sensible. echo online > memoryXY/state will online the given block to
> 	1) the default zone if the given range is outside of any zone
> 	2) the enclosing zone if such a zone doesn't interleave with
> 	   any other zone
>         3) the default zone if more zones interleave for this range
> where default zone is movable zone only if movable_node is enabled
> otherwise it is a kernel zone.
> 
> Here is an example of the semantic with (movable_node is not present but
> it work in an analogous way). We start with following memblocks, all of
> them offline
> memory34/valid_zones:Normal Movable
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Normal Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal Movable
> memory40/valid_zones:Normal Movable
> memory41/valid_zones:Normal Movable
> 
> Now, we online block 34 in default mode and block 37 as movable
> root@test1:/sys/devices/system/node/node1# echo online > memory34/state
> root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal Movable
> memory40/valid_zones:Normal Movable
> memory41/valid_zones:Normal Movable
> 
> As we can see all other blocks can still be onlined both into Normal and
> Movable zones and the Normal is default because the Movable zone spans
> only block37 now.
> root@test1:/sys/devices/system/node/node1# echo online_movable > memory41/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Movable Normal
> memory39/valid_zones:Movable Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable
> 
> Now the default zone for blocks 37-41 has changed because movable zone
> spans that range.
> root@test1:/sys/devices/system/node/node1# echo online_kernel > memory39/state
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal Movable
> memory36/valid_zones:Normal Movable
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal Movable
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable Normal
> memory41/valid_zones:Movable
> 
> Note that the block 39 now belongs to the zone Normal and so block38
> falls into Normal by default as well.
> 
> For completness
> root@test1:/sys/devices/system/node/node1# for i in memory[34]?
> do
> 	echo online > $i/state 2>/dev/null
> done
> 
> memory34/valid_zones:Normal
> memory35/valid_zones:Normal
> memory36/valid_zones:Normal
> memory37/valid_zones:Movable
> memory38/valid_zones:Normal
> memory39/valid_zones:Normal
> memory40/valid_zones:Movable
> memory41/valid_zones:Movable
> 
> Implementation wise the change is quite straightforward. We can get rid
> of allow_online_pfn_range altogether. online_pages allows only offline
> nodes already. The original default_zone_for_pfn will become
> default_kernel_zone_for_pfn. New default_zone_for_pfn implements the
> above semantic. zone_for_pfn_range is slightly reorganized to implement
> kernel and movable online type explicitly and MMOP_ONLINE_KEEP becomes
> a catch all default behavior.
> 
> Acked-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>

Acked-by: Vlastimil Babka <vbabka@suse.cz>

> Cc: <linux-api@vger.kernel.org>
> Signed-off-by: Michal Hocko <mhocko@suse.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <owner-linux-mm@kvack.org>
Received: from mail-pg0-f71.google.com (mail-pg0-f71.google.com [74.125.83.71])
	by kanga.kvack.org (Postfix) with ESMTP id A34CE440905
	for <linux-mm@kvack.org>; Fri, 14 Jul 2017 10:26:57 -0400 (EDT)
Received: by mail-pg0-f71.google.com with SMTP id 76so93526009pgh.11
        for <linux-mm@kvack.org>; Fri, 14 Jul 2017 07:26:57 -0700 (PDT)
Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com. [148.163.156.1])
        by mx.google.com with ESMTPS id 1si6729584pgk.415.2017.07.14.07.26.56
        for <linux-mm@kvack.org>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Fri, 14 Jul 2017 07:26:56 -0700 (PDT)
Received: from pps.filterd (m0098410.ppops.net [127.0.0.1])
	by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v6EEOARU058772
	for <linux-mm@kvack.org>; Fri, 14 Jul 2017 10:26:56 -0400
Received: from e11.ny.us.ibm.com (e11.ny.us.ibm.com [129.33.205.201])
	by mx0a-001b2d01.pphosted.com with ESMTP id 2bpmft9gcc-1
	(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)
	for <linux-mm@kvack.org>; Fri, 14 Jul 2017 10:26:55 -0400
Received: from localhost
	by e11.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted
	for <linux-mm@kvack.org> from <arbab@linux.vnet.ibm.com>;
	Fri, 14 Jul 2017 10:26:54 -0400
Date: Fri, 14 Jul 2017 09:26:45 -0500
From: Reza Arbab <arbab@linux.vnet.ibm.com>
Subject: Re: [PATCH 2/2] mm, memory_hotplug: remove zone restrictions
References: <20170714121233.16861-1-mhocko@kernel.org>
 <20170714121233.16861-3-mhocko@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
In-Reply-To: <20170714121233.16861-3-mhocko@kernel.org>
Message-Id: <20170714142645.dmetqyfucnc7jeur@arbab-laptop.localdomain>
Sender: owner-linux-mm@kvack.org
List-ID: <linux-mm.kvack.org>
To: Michal Hocko <mhocko@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>, Mel Gorman <mgorman@suse.de>, Vlastimil Babka <vbabka@suse.cz>, Andrea Arcangeli <aarcange@redhat.com>, Yasuaki Ishimatsu <yasu.isimatu@gmail.com>, qiuxishi@huawei.com, Kani Toshimitsu <toshi.kani@hpe.com>, slaoub@gmail.com, Joonsoo Kim <js1304@gmail.com>, Daniel Kiper <daniel.kiper@oracle.com>, Igor Mammedov <imammedo@redhat.com>, Vitaly Kuznetsov <vkuznets@redhat.com>, Wei Yang <richard.weiyang@gmail.com>, linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>, Michal Hocko <mhocko@suse.com>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, linux-api@vger.kernel.org

On Fri, Jul 14, 2017 at 02:12:33PM +0200, Michal Hocko wrote: 
>Historically we have enforced that any kernel zone (e.g ZONE_NORMAL) has
>to precede the Movable zone in the physical memory range. The purpose of
>the movable zone is, however, not bound to any physical memory restriction.
>It merely defines a class of migrateable and reclaimable memory.
>
>There are users (e.g. CMA) who might want to reserve specific physical
>memory ranges for their own purpose. Moreover our pfn walkers have to be
>prepared for zones overlapping in the physical range already because we
>do support interleaving NUMA nodes and therefore zones can interleave as
>well. This means we can allow each memory block to be associated with a
>different zone.
>
>Loosen the current onlining semantic and allow explicit onlining type on
>any memblock. That means that online_{kernel,movable} will be allowed
>regardless of the physical address of the memblock as long as it is
>offline of course. This might result in moveble zone overlapping with
>other kernel zones. Default onlining then becomes a bit tricky but still
>sensible. echo online > memoryXY/state will online the given block to
>	1) the default zone if the given range is outside of any zone
>	2) the enclosing zone if such a zone doesn't interleave with
>	   any other zone
>        3) the default zone if more zones interleave for this range
>where default zone is movable zone only if movable_node is enabled
>otherwise it is a kernel zone.
>
>Here is an example of the semantic with (movable_node is not present but
>it work in an analogous way). We start with following memblocks, all of
>them offline
>memory34/valid_zones:Normal Movable
>memory35/valid_zones:Normal Movable
>memory36/valid_zones:Normal Movable
>memory37/valid_zones:Normal Movable
>memory38/valid_zones:Normal Movable
>memory39/valid_zones:Normal Movable
>memory40/valid_zones:Normal Movable
>memory41/valid_zones:Normal Movable
>
>Now, we online block 34 in default mode and block 37 as movable
>root@test1:/sys/devices/system/node/node1# echo online > memory34/state
>root@test1:/sys/devices/system/node/node1# echo online_movable > memory37/state
>memory34/valid_zones:Normal
>memory35/valid_zones:Normal Movable
>memory36/valid_zones:Normal Movable
>memory37/valid_zones:Movable
>memory38/valid_zones:Normal Movable
>memory39/valid_zones:Normal Movable
>memory40/valid_zones:Normal Movable
>memory41/valid_zones:Normal Movable
>
>As we can see all other blocks can still be onlined both into Normal and
>Movable zones and the Normal is default because the Movable zone spans
>only block37 now.
>root@test1:/sys/devices/system/node/node1# echo online_movable > memory41/state
>memory34/valid_zones:Normal
>memory35/valid_zones:Normal Movable
>memory36/valid_zones:Normal Movable
>memory37/valid_zones:Movable
>memory38/valid_zones:Movable Normal
>memory39/valid_zones:Movable Normal
>memory40/valid_zones:Movable Normal
>memory41/valid_zones:Movable
>
>Now the default zone for blocks 37-41 has changed because movable zone
>spans that range.
>root@test1:/sys/devices/system/node/node1# echo online_kernel > memory39/state
>memory34/valid_zones:Normal
>memory35/valid_zones:Normal Movable
>memory36/valid_zones:Normal Movable
>memory37/valid_zones:Movable
>memory38/valid_zones:Normal Movable
>memory39/valid_zones:Normal
>memory40/valid_zones:Movable Normal
>memory41/valid_zones:Movable
>
>Note that the block 39 now belongs to the zone Normal and so block38
>falls into Normal by default as well.
>
>For completness
>root@test1:/sys/devices/system/node/node1# for i in memory[34]?
>do
>	echo online > $i/state 2>/dev/null
>done
>
>memory34/valid_zones:Normal
>memory35/valid_zones:Normal
>memory36/valid_zones:Normal
>memory37/valid_zones:Movable
>memory38/valid_zones:Normal
>memory39/valid_zones:Normal
>memory40/valid_zones:Movable
>memory41/valid_zones:Movable
>
>Implementation wise the change is quite straightforward. We can get rid
>of allow_online_pfn_range altogether. online_pages allows only offline
>nodes already. The original default_zone_for_pfn will become
>default_kernel_zone_for_pfn. New default_zone_for_pfn implements the
>above semantic. zone_for_pfn_range is slightly reorganized to implement
>kernel and movable online type explicitly and MMOP_ONLINE_KEEP becomes
>a catch all default behavior.
>
>Acked-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>

Acked-by: Reza Arbab <arbab@linux.vnet.ibm.com>

>Cc: <linux-api@vger.kernel.org>
>Signed-off-by: Michal Hocko <mhocko@suse.com>

-- 
Reza Arbab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>