linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v6 0/4] enable movable nodes on non-x86 configs
@ 2016-11-07 23:44 Reza Arbab
  2016-11-07 23:44 ` [PATCH v6 1/4] powerpc/mm: allow memory hotplug into a memoryless node Reza Arbab
                   ` (3 more replies)
  0 siblings, 4 replies; 17+ messages in thread
From: Reza Arbab @ 2016-11-07 23:44 UTC (permalink / raw)
  To: Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras,
	Andrew Morton, Rob Herring, Frank Rowand, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin
  Cc: linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple, Balbir Singh,
	Aneesh Kumar K.V, linux-kernel

This patchset allows more configs to make use of movable nodes. When
CONFIG_MOVABLE_NODE is selected, there are two ways to introduce such
nodes into the system:

1. Discover movable nodes at boot. Currently this is only possible on
   x86, but we will enable configs supporting fdt to do the same.

2. Hotplug and online all of a node's memory using online_movable. This
   is already possible on any config supporting memory hotplug, not 
   just x86, but the Kconfig doesn't say so. We will fix that.

We'll also remove some cruft on power which would prevent (2).

/* changelog */

v6:
* Add a patch enabling the fdt to describe hotpluggable memory.

v5:
* http://lkml.kernel.org/r/1477339089-5455-1-git-send-email-arbab@linux.vnet.ibm.com

* Drop the patches which recognize the "status" property of dt memory
  nodes. Firmware can set the size of "linux,usable-memory" to zero instead.

v4:
* http://lkml.kernel.org/r/1475778995-1420-1-git-send-email-arbab@linux.vnet.ibm.com

* Rename of_fdt_is_available() to of_fdt_device_is_available().
  Rename of_flat_dt_is_available() to of_flat_dt_device_is_available().

* Instead of restoring top-down allocation, ensure it never goes
  bottom-up in the first place, by making movable_node arch-specific.

* Use MEMORY_HOTPLUG instead of PPC64 in the mm/Kconfig patch.

v3:
* http://lkml.kernel.org/r/1474828616-16608-1-git-send-email-arbab@linux.vnet.ibm.com

* Use Rob Herring's suggestions to improve the node availability check.

* More verbose commit log in the patch enabling CONFIG_MOVABLE_NODE.

* Add a patch to restore top-down allocation the way x86 does.

v2:
* http://lkml.kernel.org/r/1473883618-14998-1-git-send-email-arbab@linux.vnet.ibm.com

* Use the "status" property of standard dt memory nodes instead of
  introducing a new "ibm,hotplug-aperture" compatible id.

* Remove the patch which explicitly creates a memoryless node. This set
  no longer has any bearing on whether the pgdat is created at boot or
  at the time of memory addition.

v1:
* http://lkml.kernel.org/r/1470680843-28702-1-git-send-email-arbab@linux.vnet.ibm.com

Reza Arbab (4):
  powerpc/mm: allow memory hotplug into a memoryless node
  mm: remove x86-only restriction of movable_node
  mm: enable CONFIG_MOVABLE_NODE on non-x86 arches
  of/fdt: mark hotpluggable memory

 Documentation/kernel-parameters.txt |  2 +-
 arch/powerpc/mm/numa.c              | 13 +------------
 arch/x86/kernel/setup.c             | 24 ++++++++++++++++++++++++
 drivers/of/fdt.c                    |  6 ++++++
 mm/Kconfig                          |  2 +-
 mm/memory_hotplug.c                 | 20 --------------------
 6 files changed, 33 insertions(+), 34 deletions(-)

-- 
1.8.3.1

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH v6 1/4] powerpc/mm: allow memory hotplug into a memoryless node
  2016-11-07 23:44 [PATCH v6 0/4] enable movable nodes on non-x86 configs Reza Arbab
@ 2016-11-07 23:44 ` Reza Arbab
  2016-11-10  1:37   ` Michael Ellerman
  2016-11-07 23:44 ` [PATCH v6 2/4] mm: remove x86-only restriction of movable_node Reza Arbab
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 17+ messages in thread
From: Reza Arbab @ 2016-11-07 23:44 UTC (permalink / raw)
  To: Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras,
	Andrew Morton, Rob Herring, Frank Rowand, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin
  Cc: linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple, Balbir Singh,
	Aneesh Kumar K.V, linux-kernel

Remove the check which prevents us from hotplugging into an empty node.

The original commit b226e4621245 ("[PATCH] powerpc: don't add memory to
empty node/zone"), states that this was intended to be a temporary measure.
It is a workaround for an oops which no longer occurs.

Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com>
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Acked-by: Balbir Singh <bsingharora@gmail.com>
Cc: Nathan Fontenot <nfont@linux.vnet.ibm.com>
Cc: Bharata B Rao <bharata@linux.vnet.ibm.com>
---
 arch/powerpc/mm/numa.c | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)

diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c
index a51c188..0cb6bd8 100644
--- a/arch/powerpc/mm/numa.c
+++ b/arch/powerpc/mm/numa.c
@@ -1085,7 +1085,7 @@ static int hot_add_node_scn_to_nid(unsigned long scn_addr)
 int hot_add_scn_to_nid(unsigned long scn_addr)
 {
 	struct device_node *memory = NULL;
-	int nid, found = 0;
+	int nid;
 
 	if (!numa_enabled || (min_common_depth < 0))
 		return first_online_node;
@@ -1101,17 +1101,6 @@ int hot_add_scn_to_nid(unsigned long scn_addr)
 	if (nid < 0 || !node_online(nid))
 		nid = first_online_node;
 
-	if (NODE_DATA(nid)->node_spanned_pages)
-		return nid;
-
-	for_each_online_node(nid) {
-		if (NODE_DATA(nid)->node_spanned_pages) {
-			found = 1;
-			break;
-		}
-	}
-
-	BUG_ON(!found);
 	return nid;
 }
 
-- 
1.8.3.1

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH v6 2/4] mm: remove x86-only restriction of movable_node
  2016-11-07 23:44 [PATCH v6 0/4] enable movable nodes on non-x86 configs Reza Arbab
  2016-11-07 23:44 ` [PATCH v6 1/4] powerpc/mm: allow memory hotplug into a memoryless node Reza Arbab
@ 2016-11-07 23:44 ` Reza Arbab
  2016-11-07 23:44 ` [PATCH v6 3/4] mm: enable CONFIG_MOVABLE_NODE on non-x86 arches Reza Arbab
  2016-11-07 23:44 ` [PATCH v6 4/4] of/fdt: mark hotpluggable memory Reza Arbab
  3 siblings, 0 replies; 17+ messages in thread
From: Reza Arbab @ 2016-11-07 23:44 UTC (permalink / raw)
  To: Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras,
	Andrew Morton, Rob Herring, Frank Rowand, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin
  Cc: linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple, Balbir Singh,
	Aneesh Kumar K.V, linux-kernel

In commit c5320926e370 ("mem-hotplug: introduce movable_node boot
option"), the memblock allocation direction is changed to bottom-up and
then back to top-down like this:

1. memblock_set_bottom_up(true), called by cmdline_parse_movable_node().
2. memblock_set_bottom_up(false), called by x86's numa_init().

Even though (1) occurs in generic mm code, it is wrapped by #ifdef
CONFIG_MOVABLE_NODE, which depends on X86_64.

This means that when we extend CONFIG_MOVABLE_NODE to non-x86 arches,
things will be unbalanced. (1) will happen for them, but (2) will not.

This toggle was added in the first place because x86 has a delay between
adding memblocks and marking them as hotpluggable. Since other arches do
this marking either immediately or not at all, they do not require the
bottom-up toggle.

So, resolve things by moving (1) from cmdline_parse_movable_node() to
x86's setup_arch(), immediately after the movable_node parameter has
been parsed.

Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com>
---
 Documentation/kernel-parameters.txt |  2 +-
 arch/x86/kernel/setup.c             | 24 ++++++++++++++++++++++++
 mm/memory_hotplug.c                 | 20 --------------------
 3 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 37babf9..adcccd5 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -2401,7 +2401,7 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 			that the amount of memory usable for all allocations
 			is not too small.
 
-	movable_node	[KNL,X86] Boot-time switch to enable the effects
+	movable_node	[KNL] Boot-time switch to enable the effects
 			of CONFIG_MOVABLE_NODE=y. See mm/Kconfig for details.
 
 	MTD_Partition=	[MTD]
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 9c337b0..4cfba94 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -985,6 +985,30 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
+#ifdef CONFIG_MEMORY_HOTPLUG
+	/*
+	 * Memory used by the kernel cannot be hot-removed because Linux
+	 * cannot migrate the kernel pages. When memory hotplug is
+	 * enabled, we should prevent memblock from allocating memory
+	 * for the kernel.
+	 *
+	 * ACPI SRAT records all hotpluggable memory ranges. But before
+	 * SRAT is parsed, we don't know about it.
+	 *
+	 * The kernel image is loaded into memory at very early time. We
+	 * cannot prevent this anyway. So on NUMA system, we set any
+	 * node the kernel resides in as un-hotpluggable.
+	 *
+	 * Since on modern servers, one node could have double-digit
+	 * gigabytes memory, we can assume the memory around the kernel
+	 * image is also un-hotpluggable. So before SRAT is parsed, just
+	 * allocate memory near the kernel image to try the best to keep
+	 * the kernel away from hotpluggable memory.
+	 */
+	if (movable_node_is_enabled())
+		memblock_set_bottom_up(true);
+#endif
+
 	x86_report_nx();
 
 	/* after early param, so could get panic from serial */
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index cad4b91..e43142c1 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1727,26 +1727,6 @@ static bool can_offline_normal(struct zone *zone, unsigned long nr_pages)
 static int __init cmdline_parse_movable_node(char *p)
 {
 #ifdef CONFIG_MOVABLE_NODE
-	/*
-	 * Memory used by the kernel cannot be hot-removed because Linux
-	 * cannot migrate the kernel pages. When memory hotplug is
-	 * enabled, we should prevent memblock from allocating memory
-	 * for the kernel.
-	 *
-	 * ACPI SRAT records all hotpluggable memory ranges. But before
-	 * SRAT is parsed, we don't know about it.
-	 *
-	 * The kernel image is loaded into memory at very early time. We
-	 * cannot prevent this anyway. So on NUMA system, we set any
-	 * node the kernel resides in as un-hotpluggable.
-	 *
-	 * Since on modern servers, one node could have double-digit
-	 * gigabytes memory, we can assume the memory around the kernel
-	 * image is also un-hotpluggable. So before SRAT is parsed, just
-	 * allocate memory near the kernel image to try the best to keep
-	 * the kernel away from hotpluggable memory.
-	 */
-	memblock_set_bottom_up(true);
 	movable_node_enabled = true;
 #else
 	pr_warn("movable_node option not supported\n");
-- 
1.8.3.1

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH v6 3/4] mm: enable CONFIG_MOVABLE_NODE on non-x86 arches
  2016-11-07 23:44 [PATCH v6 0/4] enable movable nodes on non-x86 configs Reza Arbab
  2016-11-07 23:44 ` [PATCH v6 1/4] powerpc/mm: allow memory hotplug into a memoryless node Reza Arbab
  2016-11-07 23:44 ` [PATCH v6 2/4] mm: remove x86-only restriction of movable_node Reza Arbab
@ 2016-11-07 23:44 ` Reza Arbab
  2016-11-07 23:44 ` [PATCH v6 4/4] of/fdt: mark hotpluggable memory Reza Arbab
  3 siblings, 0 replies; 17+ messages in thread
From: Reza Arbab @ 2016-11-07 23:44 UTC (permalink / raw)
  To: Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras,
	Andrew Morton, Rob Herring, Frank Rowand, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin
  Cc: linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple, Balbir Singh,
	Aneesh Kumar K.V, linux-kernel

To support movable memory nodes (CONFIG_MOVABLE_NODE), at least one of
the following must be true:

1. This config has the capability to identify movable nodes at boot.
   Right now, only x86 can do this.

2. Our config supports memory hotplug, which means that a movable node
   can be created by hotplugging all of its memory into ZONE_MOVABLE.

Fix the Kconfig definition of CONFIG_MOVABLE_NODE, which currently
recognizes (1), but not (2).

Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com>
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Acked-by: Balbir Singh <bsingharora@gmail.com>
---
 mm/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/Kconfig b/mm/Kconfig
index 86e3e0e..061b46b 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -153,7 +153,7 @@ config MOVABLE_NODE
 	bool "Enable to assign a node which has only movable memory"
 	depends on HAVE_MEMBLOCK
 	depends on NO_BOOTMEM
-	depends on X86_64
+	depends on X86_64 || MEMORY_HOTPLUG
 	depends on NUMA
 	default n
 	help
-- 
1.8.3.1

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-07 23:44 [PATCH v6 0/4] enable movable nodes on non-x86 configs Reza Arbab
                   ` (2 preceding siblings ...)
  2016-11-07 23:44 ` [PATCH v6 3/4] mm: enable CONFIG_MOVABLE_NODE on non-x86 arches Reza Arbab
@ 2016-11-07 23:44 ` Reza Arbab
  2016-11-08  1:59   ` kbuild test robot
                     ` (5 more replies)
  3 siblings, 6 replies; 17+ messages in thread
From: Reza Arbab @ 2016-11-07 23:44 UTC (permalink / raw)
  To: Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras,
	Andrew Morton, Rob Herring, Frank Rowand, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin
  Cc: linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple, Balbir Singh,
	Aneesh Kumar K.V, linux-kernel

When movable nodes are enabled, any node containing only hotpluggable
memory is made movable at boot time.

On x86, hotpluggable memory is discovered by parsing the ACPI SRAT,
making corresponding calls to memblock_mark_hotplug().

If we introduce a dt property to describe memory as hotpluggable,
configs supporting early fdt may then also do this marking and use
movable nodes.

Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com>
---
 drivers/of/fdt.c | 6 ++++++
 mm/Kconfig       | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
index c89d5d2..2cf1d66 100644
--- a/drivers/of/fdt.c
+++ b/drivers/of/fdt.c
@@ -1015,6 +1015,7 @@ int __init early_init_dt_scan_memory(unsigned long node, const char *uname,
 	const char *type = of_get_flat_dt_prop(node, "device_type", NULL);
 	const __be32 *reg, *endp;
 	int l;
+	bool hotpluggable;
 
 	/* We are scanning "memory" nodes only */
 	if (type == NULL) {
@@ -1034,6 +1035,7 @@ int __init early_init_dt_scan_memory(unsigned long node, const char *uname,
 		return 0;
 
 	endp = reg + (l / sizeof(__be32));
+	hotpluggable = of_get_flat_dt_prop(node, "linux,hotpluggable", NULL);
 
 	pr_debug("memory scan node %s, reg size %d,\n", uname, l);
 
@@ -1049,6 +1051,10 @@ int __init early_init_dt_scan_memory(unsigned long node, const char *uname,
 		    (unsigned long long)size);
 
 		early_init_dt_add_memory_arch(base, size);
+
+		if (hotpluggable && memblock_mark_hotplug(base, size))
+			pr_warn("failed to mark hotplug range 0x%llx - 0x%llx\n",
+				base, base + size);
 	}
 
 	return 0;
diff --git a/mm/Kconfig b/mm/Kconfig
index 061b46b..33a9b06 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -153,7 +153,7 @@ config MOVABLE_NODE
 	bool "Enable to assign a node which has only movable memory"
 	depends on HAVE_MEMBLOCK
 	depends on NO_BOOTMEM
-	depends on X86_64 || MEMORY_HOTPLUG
+	depends on X86_64 || OF_EARLY_FLATTREE || MEMORY_HOTPLUG
 	depends on NUMA
 	default n
 	help
-- 
1.8.3.1

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-07 23:44 ` [PATCH v6 4/4] of/fdt: mark hotpluggable memory Reza Arbab
@ 2016-11-08  1:59   ` kbuild test robot
  2016-11-08 19:59     ` Reza Arbab
  2016-11-09 18:12   ` Rob Herring
                     ` (4 subsequent siblings)
  5 siblings, 1 reply; 17+ messages in thread
From: kbuild test robot @ 2016-11-08  1:59 UTC (permalink / raw)
  To: Reza Arbab
  Cc: kbuild-all, Michael Ellerman, Benjamin Herrenschmidt,
	Paul Mackerras, Andrew Morton, Rob Herring, Frank Rowand,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin, Stewart Smith,
	devicetree, linux-kernel, linux-mm, Alistair Popple,
	Aneesh Kumar K.V, Bharata B Rao, Nathan Fontenot, linuxppc-dev

[-- Attachment #1: Type: text/plain, Size: 1582 bytes --]

Hi Reza,

[auto build test ERROR on powerpc/next]
[also build test ERROR on v4.9-rc4 next-20161028]
[cannot apply to mmotm/master]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Reza-Arbab/enable-movable-nodes-on-non-x86-configs/20161108-081711
base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
config: tile-allmodconfig (attached as .config)
compiler: tilegx-linux-gcc (GCC) 4.6.2
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=tile 

All errors (new ones prefixed by >>):

   drivers/of/fdt.c: In function 'early_init_dt_scan_memory':
>> drivers/of/fdt.c:1064:3: error: implicit declaration of function 'memblock_mark_hotplug'
   cc1: some warnings being treated as errors

vim +/memblock_mark_hotplug +1064 drivers/of/fdt.c

  1058				continue;
  1059			pr_debug(" - %llx ,  %llx\n", (unsigned long long)base,
  1060			    (unsigned long long)size);
  1061	
  1062			early_init_dt_add_memory_arch(base, size);
  1063	
> 1064			if (hotpluggable && memblock_mark_hotplug(base, size))
  1065				pr_warn("failed to mark hotplug range 0x%llx - 0x%llx\n",
  1066					base, base + size);
  1067		}

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 45509 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-08  1:59   ` kbuild test robot
@ 2016-11-08 19:59     ` Reza Arbab
  0 siblings, 0 replies; 17+ messages in thread
From: Reza Arbab @ 2016-11-08 19:59 UTC (permalink / raw)
  To: kbuild test robot
  Cc: kbuild-all, Michael Ellerman, Benjamin Herrenschmidt,
	Paul Mackerras, Andrew Morton, Rob Herring, Frank Rowand,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin, Stewart Smith,
	devicetree, linux-kernel, linux-mm, Alistair Popple,
	Aneesh Kumar K.V, Bharata B Rao, Nathan Fontenot, linuxppc-dev

On Tue, Nov 08, 2016 at 09:59:26AM +0800, kbuild test robot wrote:
>All errors (new ones prefixed by >>):
>
>   drivers/of/fdt.c: In function 'early_init_dt_scan_memory':
>>> drivers/of/fdt.c:1064:3: error: implicit declaration of function 'memblock_mark_hotplug'
>   cc1: some warnings being treated as errors
>
>vim +/memblock_mark_hotplug +1064 drivers/of/fdt.c
>
>  1058				continue;
>  1059			pr_debug(" - %llx ,  %llx\n", (unsigned long long)base,
>  1060			    (unsigned long long)size);
>  1061	
>  1062			early_init_dt_add_memory_arch(base, size);
>  1063	
>> 1064			if (hotpluggable && memblock_mark_hotplug(base, size))
>  1065				pr_warn("failed to mark hotplug range 0x%llx - 0x%llx\n",
>  1066					base, base + size);
>  1067		}

Ah, I need to adjust for !CONFIG_HAVE_MEMBLOCK. Will correct in v7.

-- 
Reza Arbab

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-07 23:44 ` [PATCH v6 4/4] of/fdt: mark hotpluggable memory Reza Arbab
  2016-11-08  1:59   ` kbuild test robot
@ 2016-11-09 18:12   ` Rob Herring
  2016-11-09 20:15     ` Reza Arbab
  2016-11-10  0:56   ` Balbir Singh
                     ` (3 subsequent siblings)
  5 siblings, 1 reply; 17+ messages in thread
From: Rob Herring @ 2016-11-09 18:12 UTC (permalink / raw)
  To: Reza Arbab
  Cc: Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras,
	Andrew Morton, Frank Rowand, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, linuxppc-dev, linux-mm, devicetree,
	Bharata B Rao, Nathan Fontenot, Stewart Smith, Alistair Popple,
	Balbir Singh, Aneesh Kumar K.V, linux-kernel

On Mon, Nov 7, 2016 at 5:44 PM, Reza Arbab <arbab@linux.vnet.ibm.com> wrote:
> When movable nodes are enabled, any node containing only hotpluggable
> memory is made movable at boot time.
>
> On x86, hotpluggable memory is discovered by parsing the ACPI SRAT,
> making corresponding calls to memblock_mark_hotplug().
>
> If we introduce a dt property to describe memory as hotpluggable,
> configs supporting early fdt may then also do this marking and use
> movable nodes.
>
> Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com>
> ---
>  drivers/of/fdt.c | 6 ++++++
>  mm/Kconfig       | 2 +-
>  2 files changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
> index c89d5d2..2cf1d66 100644
> --- a/drivers/of/fdt.c
> +++ b/drivers/of/fdt.c
> @@ -1015,6 +1015,7 @@ int __init early_init_dt_scan_memory(unsigned long node, const char *uname,
>         const char *type = of_get_flat_dt_prop(node, "device_type", NULL);
>         const __be32 *reg, *endp;
>         int l;
> +       bool hotpluggable;
>
>         /* We are scanning "memory" nodes only */
>         if (type == NULL) {
> @@ -1034,6 +1035,7 @@ int __init early_init_dt_scan_memory(unsigned long node, const char *uname,
>                 return 0;
>
>         endp = reg + (l / sizeof(__be32));
> +       hotpluggable = of_get_flat_dt_prop(node, "linux,hotpluggable", NULL);

Memory being hotpluggable doesn't seem like a linux property to me.
I'd drop the linux prefix. Also, this needs to be documented.

Rob

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-09 18:12   ` Rob Herring
@ 2016-11-09 20:15     ` Reza Arbab
  0 siblings, 0 replies; 17+ messages in thread
From: Reza Arbab @ 2016-11-09 20:15 UTC (permalink / raw)
  To: Rob Herring
  Cc: Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras,
	Andrew Morton, Frank Rowand, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, linuxppc-dev, linux-mm, devicetree,
	Bharata B Rao, Nathan Fontenot, Stewart Smith, Alistair Popple,
	Balbir Singh, Aneesh Kumar K.V, linux-kernel

On Wed, Nov 09, 2016 at 12:12:55PM -0600, Rob Herring wrote:
>On Mon, Nov 7, 2016 at 5:44 PM, Reza Arbab <arbab@linux.vnet.ibm.com> wrote:
>> +       hotpluggable = of_get_flat_dt_prop(node, "linux,hotpluggable", NULL);
>
>Memory being hotpluggable doesn't seem like a linux property to me.
>I'd drop the linux prefix. Also, this needs to be documented.

Sure, that makes sense. I'll do both in v7.

-- 
Reza Arbab

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-07 23:44 ` [PATCH v6 4/4] of/fdt: mark hotpluggable memory Reza Arbab
  2016-11-08  1:59   ` kbuild test robot
  2016-11-09 18:12   ` Rob Herring
@ 2016-11-10  0:56   ` Balbir Singh
  2016-11-10 20:52     ` Reza Arbab
  2016-11-11  1:17   ` Balbir Singh
                     ` (2 subsequent siblings)
  5 siblings, 1 reply; 17+ messages in thread
From: Balbir Singh @ 2016-11-10  0:56 UTC (permalink / raw)
  To: Reza Arbab, Michael Ellerman, Benjamin Herrenschmidt,
	Paul Mackerras, Andrew Morton, Rob Herring, Frank Rowand,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin
  Cc: linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple,
	Aneesh Kumar K.V, linux-kernel



On 08/11/16 10:44, Reza Arbab wrote:
> When movable nodes are enabled, any node containing only hotpluggable
> memory is made movable at boot time.
> 
> On x86, hotpluggable memory is discovered by parsing the ACPI SRAT,
> making corresponding calls to memblock_mark_hotplug().
> 
> If we introduce a dt property to describe memory as hotpluggable,
> configs supporting early fdt may then also do this marking and use
> movable nodes.

This looks much better, like the other comments pointed out

We need documentation around the changes. One quick question

Have you tested this across all combinations of skiboot/kexec/SLOF boots?

Balbir Singh.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 1/4] powerpc/mm: allow memory hotplug into a memoryless node
  2016-11-07 23:44 ` [PATCH v6 1/4] powerpc/mm: allow memory hotplug into a memoryless node Reza Arbab
@ 2016-11-10  1:37   ` Michael Ellerman
  0 siblings, 0 replies; 17+ messages in thread
From: Michael Ellerman @ 2016-11-10  1:37 UTC (permalink / raw)
  To: Reza Arbab, Benjamin Herrenschmidt, Paul Mackerras,
	Andrew Morton, Rob Herring, Frank Rowand, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin
  Cc: linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple, Balbir Singh,
	Aneesh Kumar K.V, linux-kernel

Reza Arbab <arbab@linux.vnet.ibm.com> writes:

> Remove the check which prevents us from hotplugging into an empty node.
>
> The original commit b226e4621245 ("[PATCH] powerpc: don't add memory to
> empty node/zone"), states that this was intended to be a temporary measure.
> It is a workaround for an oops which no longer occurs.
>
> Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com>
> Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> Acked-by: Balbir Singh <bsingharora@gmail.com>
> Cc: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> Cc: Bharata B Rao <bharata@linux.vnet.ibm.com>
> ---
>  arch/powerpc/mm/numa.c | 13 +------------
>  1 file changed, 1 insertion(+), 12 deletions(-)

This seems OK from a powerpc perspective.

Acked-by: Michael Ellerman <mpe@ellerman.id.au>

cheers

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-10  0:56   ` Balbir Singh
@ 2016-11-10 20:52     ` Reza Arbab
  0 siblings, 0 replies; 17+ messages in thread
From: Reza Arbab @ 2016-11-10 20:52 UTC (permalink / raw)
  To: Balbir Singh
  Cc: Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras,
	Andrew Morton, Rob Herring, Frank Rowand, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin, linuxppc-dev, linux-mm, devicetree,
	Bharata B Rao, Nathan Fontenot, Stewart Smith, Alistair Popple,
	Aneesh Kumar K.V, linux-kernel

On Thu, Nov 10, 2016 at 11:56:02AM +1100, Balbir Singh wrote:
>Have you tested this across all combinations of skiboot/kexec/SLOF 
>boots?

I've tested it under qemu/grub, simics/skiboot, and via kexec.

-- 
Reza Arbab

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-07 23:44 ` [PATCH v6 4/4] of/fdt: mark hotpluggable memory Reza Arbab
                     ` (2 preceding siblings ...)
  2016-11-10  0:56   ` Balbir Singh
@ 2016-11-11  1:17   ` Balbir Singh
  2016-11-14 11:59   ` Michael Ellerman
  2016-12-25  9:02   ` Heinrich Schuchardt
  5 siblings, 0 replies; 17+ messages in thread
From: Balbir Singh @ 2016-11-11  1:17 UTC (permalink / raw)
  To: Reza Arbab, Michael Ellerman, Benjamin Herrenschmidt,
	Paul Mackerras, Andrew Morton, Rob Herring, Frank Rowand,
	Thomas Gleixner, Ingo Molnar, H. Peter Anvin
  Cc: linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple,
	Aneesh Kumar K.V, linux-kernel



On 08/11/16 10:44, Reza Arbab wrote:
> When movable nodes are enabled, any node containing only hotpluggable
> memory is made movable at boot time.
> 
> On x86, hotpluggable memory is discovered by parsing the ACPI SRAT,
> making corresponding calls to memblock_mark_hotplug().
> 
> If we introduce a dt property to describe memory as hotpluggable,
> configs supporting early fdt may then also do this marking and use
> movable nodes.
> 
> Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com>
> ---

Tested-by: Balbir Singh <bsingharora@gmail.com>

I tested this with a custom device tree and it worked quite well for me.
It also means that the guest and bare-metal have two different mechanisms
of marking something as hotpluggable. But given that your patch enables
all architectures using OF, it might be worth it.

Balbir Singh.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-07 23:44 ` [PATCH v6 4/4] of/fdt: mark hotpluggable memory Reza Arbab
                     ` (3 preceding siblings ...)
  2016-11-11  1:17   ` Balbir Singh
@ 2016-11-14 11:59   ` Michael Ellerman
  2016-11-14 19:34     ` Reza Arbab
  2016-12-25  9:02   ` Heinrich Schuchardt
  5 siblings, 1 reply; 17+ messages in thread
From: Michael Ellerman @ 2016-11-14 11:59 UTC (permalink / raw)
  To: Reza Arbab, Benjamin Herrenschmidt, Paul Mackerras,
	Andrew Morton, Rob Herring, Frank Rowand, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin
  Cc: linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple, Balbir Singh,
	Aneesh Kumar K.V, linux-kernel

Reza Arbab <arbab@linux.vnet.ibm.com> writes:

> When movable nodes are enabled, any node containing only hotpluggable
> memory is made movable at boot time.
>
> On x86, hotpluggable memory is discovered by parsing the ACPI SRAT,
> making corresponding calls to memblock_mark_hotplug().
>
> If we introduce a dt property to describe memory as hotpluggable,
> configs supporting early fdt may then also do this marking and use
> movable nodes.

So I'm not opposed to this, but it is a little vague.

What does the "hotpluggable" property really mean?

Is it just a hint to the operating system? (which may or may not be
Linux).

Or is it a direction, "this memory must be able to be hotunplugged"?

I think you're intending the former, ie. a hint, which is probably OK.
But it needs to be documented clearly.

cheers

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-14 11:59   ` Michael Ellerman
@ 2016-11-14 19:34     ` Reza Arbab
  0 siblings, 0 replies; 17+ messages in thread
From: Reza Arbab @ 2016-11-14 19:34 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: Benjamin Herrenschmidt, Paul Mackerras, Andrew Morton,
	Rob Herring, Frank Rowand, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, linuxppc-dev, linux-mm, devicetree,
	Bharata B Rao, Nathan Fontenot, Stewart Smith, Alistair Popple,
	Balbir Singh, Aneesh Kumar K.V, linux-kernel

On Mon, Nov 14, 2016 at 10:59:43PM +1100, Michael Ellerman wrote:
>So I'm not opposed to this, but it is a little vague.
>
>What does the "hotpluggable" property really mean?
>
>Is it just a hint to the operating system? (which may or may not be
>Linux).
>
>Or is it a direction, "this memory must be able to be hotunplugged"?
>
>I think you're intending the former, ie. a hint, which is probably OK.
>But it needs to be documented clearly.

Yes, you've got it right. It's just a hint, not a mandate.

I'm about to send v7 which adds a description of "hotpluggable" in the 
documentation. Hopefully I've explained it well enough there.

-- 
Reza Arbab

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-11-07 23:44 ` [PATCH v6 4/4] of/fdt: mark hotpluggable memory Reza Arbab
                     ` (4 preceding siblings ...)
  2016-11-14 11:59   ` Michael Ellerman
@ 2016-12-25  9:02   ` Heinrich Schuchardt
  2016-12-27  0:09     ` Frank Rowand
  5 siblings, 1 reply; 17+ messages in thread
From: Heinrich Schuchardt @ 2016-12-25  9:02 UTC (permalink / raw)
  To: Reza Arbab
  Cc: Balbir Singh, Aneesh Kumar K . V, H . Peter Anvin,
	Alistair Popple, Benjamin Herrenschmidt, Bharata B Rao,
	Frank Rowand, Ingo Molnar, Michael Ellerman, Nathan Fontenot,
	Paul Mackerras, Rob Herring, Stewart Smith, Thomas Gleixner,
	Andrew Morton, devicetree, linux-kernel, linux-mm,
	Heinrich Schuchardt

The patch adds a new property "linux,hotpluggable" to memory nodes of the
device tree.

memory@0 {
	reg = <0x0 0x01000000 0x0 0x7f000000>;
	linux,hotpluggable;
}

Memory areas marked by this property can later be disabled using the hotplugging
API. Especially for virtual machines this is a very useful capability.

Unfortunately the notation chosen does not fit well with the concept of
devicetree overlays which allow to change the devicetree during runtime.

I suggest to use the following notation

memory@0 {
	compatible = "linux,hotpluggable-memory";
	reg = <0x0 0x01000000 0x0 0x7f000000>;
	status = "disabled";
}

This will allow us to write a device driver that can react to changes of the
devicetree made via devicetree overlays.

This driver could react to the change of the status between "okay" and
"disabled" and update the memory status accordingly.

Further we could use devicetree overlays to provide additional hotpluggable
memory.

The referenced patch has already been pulled for 4.10. But I hope it is not
too late for this design change.

Best regards

Heinrich Schuchardt

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
  2016-12-25  9:02   ` Heinrich Schuchardt
@ 2016-12-27  0:09     ` Frank Rowand
  0 siblings, 0 replies; 17+ messages in thread
From: Frank Rowand @ 2016-12-27  0:09 UTC (permalink / raw)
  To: Heinrich Schuchardt, Reza Arbab
  Cc: Balbir Singh, Aneesh Kumar K . V, H . Peter Anvin,
	Alistair Popple, Benjamin Herrenschmidt, Bharata B Rao,
	Ingo Molnar, Michael Ellerman, Nathan Fontenot, Paul Mackerras,
	Rob Herring, Stewart Smith, Thomas Gleixner, Andrew Morton,
	devicetree, linux-kernel, linux-mm

On 12/25/16 01:02, Heinrich Schuchardt wrote:
> The patch adds a new property "linux,hotpluggable" to memory nodes of the
> device tree.
> 
> memory@0 {
> 	reg = <0x0 0x01000000 0x0 0x7f000000>;
> 	linux,hotpluggable;
> }
> 
> Memory areas marked by this property can later be disabled using the hotplugging
> API. Especially for virtual machines this is a very useful capability.
> 
> Unfortunately the notation chosen does not fit well with the concept of
> devicetree overlays which allow to change the devicetree during runtime.

Why would one want to change the hot pluggable memory node via an overlay?
In other words, what is missing from the hot pluggable memory paradigm
that instead requires overlays?

If something is missing from the hot pluggable memory code, then it seems
to me that it should be added to that code instead of hacking around it
by using device tree overlays.

-Frank

> 
> I suggest to use the following notation
> 
> memory@0 {
> 	compatible = "linux,hotpluggable-memory";
> 	reg = <0x0 0x01000000 0x0 0x7f000000>;
> 	status = "disabled";
> }
> 
> This will allow us to write a device driver that can react to changes of the
> devicetree made via devicetree overlays.
> 
> This driver could react to the change of the status between "okay" and
> "disabled" and update the memory status accordingly.
> 
> Further we could use devicetree overlays to provide additional hotpluggable
> memory.
> 
> The referenced patch has already been pulled for 4.10. But I hope it is not
> too late for this design change.
> 
> Best regards
> 
> Heinrich Schuchardt
> 

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2016-12-27  0:10 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-07 23:44 [PATCH v6 0/4] enable movable nodes on non-x86 configs Reza Arbab
2016-11-07 23:44 ` [PATCH v6 1/4] powerpc/mm: allow memory hotplug into a memoryless node Reza Arbab
2016-11-10  1:37   ` Michael Ellerman
2016-11-07 23:44 ` [PATCH v6 2/4] mm: remove x86-only restriction of movable_node Reza Arbab
2016-11-07 23:44 ` [PATCH v6 3/4] mm: enable CONFIG_MOVABLE_NODE on non-x86 arches Reza Arbab
2016-11-07 23:44 ` [PATCH v6 4/4] of/fdt: mark hotpluggable memory Reza Arbab
2016-11-08  1:59   ` kbuild test robot
2016-11-08 19:59     ` Reza Arbab
2016-11-09 18:12   ` Rob Herring
2016-11-09 20:15     ` Reza Arbab
2016-11-10  0:56   ` Balbir Singh
2016-11-10 20:52     ` Reza Arbab
2016-11-11  1:17   ` Balbir Singh
2016-11-14 11:59   ` Michael Ellerman
2016-11-14 19:34     ` Reza Arbab
2016-12-25  9:02   ` Heinrich Schuchardt
2016-12-27  0:09     ` Frank Rowand

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).