[PATCH v6 00/14] ACPI NUMA support for ARM64

[PATCH v6 00/14] ACPI NUMA support for ARM64

[David Daney]

From: David Daney <david.daney@cavium.com>

Based on git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
for-next/core branch at commit 643d703d2d2d ("arm64: compat: Check for
AArch32 state")

ACPI 5.1 already introduced NUMA support for ARM64, which can get the
NUMA domain information from SRAT and SLIT table, so parse those two
tables to get mappings from cpu/mem to numa node configuration and
system locality.

v6 updates:

 - Changed message printed in 13/14 to omit meaningless "cpu"
   number. (noted by Dennis Chen and Hanjun Guo)

 - New patch 7/14 to print different messages for NUMA disabled case.
   (noted by Dennis Chen)

 - Squashed two patches into 14/14.

 - Added Reviewed-by to 4/14

v5 updates:

 - Fixed ia64 build failure by gating some unused functions with #if
   CONFIG_{X86,ARM64}.

 - Fixed section mismatch errors for X86 case.

 - Removed unneeded #include from some files.

 - Tested to build cleanly on ARM64, X86_64, IA64

v4 updates:

 - Updated from Hanjun Guo's v3 patches.

 - Rebased on top of v16 of device-tree NUMA patches.

 - Reordered some of the changes so that we don't introduce code and
   then change it several times in the patch set.  New code is
   introduced in its final form.  Code reused from x86 is first moved
   with no change, and then a separate patch to make any needed
   changes.

 - code that is used only by ia64, moved to architecture specific
   files.

v3 updates:
 - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
   that for acpi/numa.c (patch 2/12)

 - Remove the duplicate NULL check for table print (patch 3/12)

 - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
   dummy function for acpi_numa_arch_fixup()

 - Solve the problem that the mapping from logical cpu to numa node
   is wrong which spotted out by Lorenzo

 - cleanups for x86 and move acpi_numa_slit_init() and some other
   functions to common place, then reduce the duplicate of x86
   and arm64 (patch 7-12/12).

 - rebased on top of 4.4 and Ganapat's v9 patch set.


David Daney (2):
  arm64, numa: Cleanup NUMA disabled messages.
  acpi, numa, srat: Improve SRAT error detection and add messages.

Hanjun Guo (11):
  acpi, numa: Use pr_fmt() instead of printk
  acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
  acpi, numa: remove duplicate NULL check
  acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
  arm64, numa: rework numa_add_memblk()
  x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
  acpi, numa: move bad_srat() and srat_disabled() to
    drivers/acpi/numa.c
  acpi, numa: remove unneeded acpi_numa=1
  acpi, numa: Move acpi_numa_memory_affinity_init() to
    drivers/acpi/numa.c
  arm64, acpi, numa: NUMA support based on SRAT and SLIT
  acpi, numa: Enable ACPI based NUMA on ARM64

Robert Richter (1):
  acpi, numa: Move acpi_numa_arch_fixup() to ia64 only

 arch/arm64/include/asm/acpi.h |   8 ++
 arch/arm64/include/asm/numa.h |   2 +
 arch/arm64/kernel/Makefile    |   1 +
 arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
 arch/arm64/kernel/smp.c       |   2 +
 arch/arm64/mm/numa.c          |  28 +++---
 arch/ia64/include/asm/acpi.h  |   3 +
 arch/ia64/kernel/acpi.c       |   2 +-
 arch/ia64/kernel/setup.c      |   1 +
 arch/x86/include/asm/acpi.h   |   1 -
 arch/x86/mm/numa.c            |   2 +-
 arch/x86/mm/srat.c            | 116 +---------------------
 drivers/acpi/Kconfig          |   4 +-
 drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
 drivers/of/of_numa.c          |   4 +-
 include/acpi/acpi_numa.h      |   4 +
 include/linux/acpi.h          |  18 +++-
 17 files changed, 382 insertions(+), 187 deletions(-)
 create mode 100644 arch/arm64/kernel/acpi_numa.c

-- 
1.7.11.7

[PATCH v6 01/14] acpi, numa: Use pr_fmt() instead of printk

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Just do some cleanups to replace printk with pr_fmt().

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 72b6e9e..4e427fc 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -18,6 +18,9 @@
  * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  *
  */
+
+#define pr_fmt(fmt) "ACPI: " fmt
+
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/kernel.h>
@@ -28,8 +31,6 @@
 #include <linux/nodemask.h>
 #include <linux/topology.h>
 
-#define PREFIX "ACPI: "
-
 #define ACPI_NUMA	0x80000000
 #define _COMPONENT	ACPI_NUMA
 ACPI_MODULE_NAME("numa");
@@ -187,9 +188,8 @@ acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 #endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 	default:
-		printk(KERN_WARNING PREFIX
-		       "Found unsupported SRAT entry (type = 0x%x)\n",
-		       header->type);
+		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
+			header->type);
 		break;
 	}
 }
@@ -222,7 +222,7 @@ static int __init acpi_parse_slit(struct acpi_table_header *table)
 	struct acpi_table_slit *slit = (struct acpi_table_slit *)table;
 
 	if (!slit_valid(slit)) {
-		printk(KERN_INFO "ACPI: SLIT table looks invalid. Not used.\n");
+		pr_info("SLIT table looks invalid. Not used.\n");
 		return -EINVAL;
 	}
 	acpi_numa_slit_init(slit);
@@ -233,12 +233,9 @@ static int __init acpi_parse_slit(struct acpi_table_header *table)
 void __init __weak
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 {
-	printk(KERN_WARNING PREFIX
-	       "Found unsupported x2apic [0x%08x] SRAT entry\n", pa->apic_id);
-	return;
+	pr_warn("Found unsupported x2apic [0x%08x] SRAT entry\n", pa->apic_id);
 }
 
-
 static int __init
 acpi_parse_x2apic_affinity(struct acpi_subtable_header *header,
 			   const unsigned long end)
-- 
1.7.11.7

[PATCH v6 02/14] acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

ACPI_DEBUG_PRINT is a bit fragile in acpi/numa.c, the first thing
is that component ACPI_NUMA(0x80000000) is not described in the
Documentation/acpi/debug.txt, and even not defined in the struct
acpi_dlayer acpi_debug_layers which we can not dynamically enable/disable
it with /sys/modules/acpi/parameters/debug_layer. another thing
is that ACPI_DEBUG_OUTPUT is controlled by ACPICA which not coordinate
well with ACPI drivers.

Replace ACPI_DEBUG_PRINT() with pr_debug() in this patch as pr_debug
will do the same thing for debug purpose and it can make the code much
cleaner, also remove the related code which not needed anymore if
ACPI_DEBUG_PRINT() is gone.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 59 +++++++++++++++++++----------------------------------
 1 file changed, 21 insertions(+), 38 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 4e427fc..e34b5d0 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -31,10 +31,6 @@
 #include <linux/nodemask.h>
 #include <linux/topology.h>
 
-#define ACPI_NUMA	0x80000000
-#define _COMPONENT	ACPI_NUMA
-ACPI_MODULE_NAME("numa");
-
 static nodemask_t nodes_found_map = NODE_MASK_NONE;
 
 /* maps to convert between proximity domain and logical node ID */
@@ -129,64 +125,51 @@ EXPORT_SYMBOL(acpi_map_pxm_to_online_node);
 static void __init
 acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 {
-
-	ACPI_FUNCTION_NAME("acpi_table_print_srat_entry");
-
 	if (!header)
 		return;
 
 	switch (header->type) {
-
 	case ACPI_SRAT_TYPE_CPU_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_cpu_affinity *p =
 			    (struct acpi_srat_cpu_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Processor (id[0x%02x] eid[0x%02x]) in proximity domain %d %s\n",
-					  p->apic_id, p->local_sapic_eid,
-					  p->proximity_domain_lo,
-					  (p->flags & ACPI_SRAT_CPU_ENABLED)?
-					  "enabled" : "disabled"));
+			pr_debug("SRAT Processor (id[0x%02x] eid[0x%02x]) in proximity domain %d %s\n",
+				 p->apic_id, p->local_sapic_eid,
+				 p->proximity_domain_lo,
+				 (p->flags & ACPI_SRAT_CPU_ENABLED) ?
+				 "enabled" : "disabled");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 
 	case ACPI_SRAT_TYPE_MEMORY_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_mem_affinity *p =
 			    (struct acpi_srat_mem_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Memory (0x%lx length 0x%lx) in proximity domain %d %s%s%s\n",
-					  (unsigned long)p->base_address,
-					  (unsigned long)p->length,
-					  p->proximity_domain,
-					  (p->flags & ACPI_SRAT_MEM_ENABLED)?
-					  "enabled" : "disabled",
-					  (p->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE)?
-					  " hot-pluggable" : "",
-					  (p->flags & ACPI_SRAT_MEM_NON_VOLATILE)?
-					  " non-volatile" : ""));
+			pr_debug("SRAT Memory (0x%lx length 0x%lx) in proximity domain %d %s%s%s\n",
+				 (unsigned long)p->base_address,
+				 (unsigned long)p->length,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_MEM_ENABLED) ?
+				 "enabled" : "disabled",
+				 (p->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) ?
+				 " hot-pluggable" : "",
+				 (p->flags & ACPI_SRAT_MEM_NON_VOLATILE) ?
+				 " non-volatile" : "");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 
 	case ACPI_SRAT_TYPE_X2APIC_CPU_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_x2apic_cpu_affinity *p =
 			    (struct acpi_srat_x2apic_cpu_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Processor (x2apicid[0x%08x]) in"
-					  " proximity domain %d %s\n",
-					  p->apic_id,
-					  p->proximity_domain,
-					  (p->flags & ACPI_SRAT_CPU_ENABLED) ?
-					  "enabled" : "disabled"));
+			pr_debug("SRAT Processor (x2apicid[0x%08x]) in proximity domain %d %s\n",
+				 p->apic_id,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_CPU_ENABLED) ?
+				 "enabled" : "disabled");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
+
 	default:
 		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
 			header->type);
-- 
1.7.11.7

[PATCH v6 03/14] acpi, numa: remove duplicate NULL check

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

The argument "header" for acpi_table_print_srat_entry()
is always checked before the function is called, it's
duplicate to check it again, remove it.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index e34b5d0..2de6068 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -125,9 +125,6 @@ EXPORT_SYMBOL(acpi_map_pxm_to_online_node);
 static void __init
 acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 {
-	if (!header)
-		return;
-
 	switch (header->type) {
 	case ACPI_SRAT_TYPE_CPU_AFFINITY:
 		{
-- 
1.7.11.7

[PATCH v6 04/14] acpi, numa: Move acpi_numa_arch_fixup() to ia64 only

[David Daney]

From: Robert Richter <rrichter@cavium.com>

Since acpi_numa_arch_fixup() is only used in arch ia64, move it there
to make a generic interface easier. This avoids empty function stubs
or some complex kconfig options for x86 and arm64.

Signed-off-by: Robert Richter <rrichter@cavium.com>
Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/ia64/include/asm/acpi.h | 3 +++
 arch/ia64/kernel/acpi.c      | 2 +-
 arch/ia64/kernel/setup.c     | 1 +
 arch/x86/mm/srat.c           | 2 --
 drivers/acpi/numa.c          | 2 --
 include/linux/acpi.h         | 1 -
 6 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/arch/ia64/include/asm/acpi.h b/arch/ia64/include/asm/acpi.h
index aa0fdf1..a3d0211 100644
--- a/arch/ia64/include/asm/acpi.h
+++ b/arch/ia64/include/asm/acpi.h
@@ -140,6 +140,9 @@ static inline void per_cpu_scan_finalize(int min_cpus, int reserve_cpus)
 		}
 	}
 }
+
+extern void acpi_numa_fixup(void);
+
 #endif /* CONFIG_ACPI_NUMA */
 
 #endif /*__KERNEL__*/
diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
index b1698bc..92b7bc9 100644
--- a/arch/ia64/kernel/acpi.c
+++ b/arch/ia64/kernel/acpi.c
@@ -524,7 +524,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	return 0;
 }
 
-void __init acpi_numa_arch_fixup(void)
+void __init acpi_numa_fixup(void)
 {
 	int i, j, node_from, node_to;
 
diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
index 2029a38..afddb3e 100644
--- a/arch/ia64/kernel/setup.c
+++ b/arch/ia64/kernel/setup.c
@@ -552,6 +552,7 @@ setup_arch (char **cmdline_p)
 	early_acpi_boot_init();
 # ifdef CONFIG_ACPI_NUMA
 	acpi_numa_init();
+	acpi_numa_fixup();
 #  ifdef CONFIG_ACPI_HOTPLUG_CPU
 	prefill_possible_map();
 #  endif
diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index b5f8218..90b6ed9 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -212,8 +212,6 @@ out_err:
 	return -1;
 }
 
-void __init acpi_numa_arch_fixup(void) {}
-
 int __init x86_acpi_numa_init(void)
 {
 	int ret;
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 2de6068..f41f06b 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -316,8 +316,6 @@ int __init acpi_numa_init(void)
 	/* SLIT: System Locality Information Table */
 	acpi_table_parse(ACPI_SIG_SLIT, acpi_parse_slit);
 
-	acpi_numa_arch_fixup();
-
 	if (cnt < 0)
 		return cnt;
 	else if (!parsed_numa_memblks)
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index 06ed7e5..ab6fd96 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -244,7 +244,6 @@ void acpi_numa_slit_init (struct acpi_table_slit *slit);
 void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
 void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
 int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
-void acpi_numa_arch_fixup(void);
 
 #ifndef PHYS_CPUID_INVALID
 typedef u32 phys_cpuid_t;
-- 
1.7.11.7

[PATCH v6 05/14] acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Identical implementations of acpi_numa_slit_init() are used by both
x86 and follow-on arm64 support.  Move it to drivers/acpi/numa.c, and
guard with CONFIG_X86 || CONFIG_ARM64 because ia64 has its own
architecture specific implementation.

No code change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c  | 27 ---------------------------
 drivers/acpi/numa.c | 29 +++++++++++++++++++++++++++++
 2 files changed, 29 insertions(+), 27 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 90b6ed9..f242a11 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -42,33 +42,6 @@ static __init inline int srat_disabled(void)
 	return acpi_numa < 0;
 }
 
-/*
- * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
- * I/O localities since SRAT does not list them.  I/O localities are
- * not supported at this point.
- */
-void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
-{
-	int i, j;
-
-	for (i = 0; i < slit->locality_count; i++) {
-		const int from_node = pxm_to_node(i);
-
-		if (from_node == NUMA_NO_NODE)
-			continue;
-
-		for (j = 0; j < slit->locality_count; j++) {
-			const int to_node = pxm_to_node(j);
-
-			if (to_node == NUMA_NO_NODE)
-				continue;
-
-			numa_set_distance(from_node, to_node,
-				slit->entry[slit->locality_count * i + j]);
-		}
-	}
-}
-
 /* Callback for Proximity Domain -> x2APIC mapping */
 void __init
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index f41f06b..6fa528a 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -197,6 +197,35 @@ static int __init slit_valid(struct acpi_table_slit *slit)
 	return 1;
 }
 
+#if defined(CONFIG_X86) || defined (CONFIG_ARM64)
+/*
+ * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
+ * I/O localities since SRAT does not list them.  I/O localities are
+ * not supported at this point.
+ */
+void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
+{
+	int i, j;
+
+	for (i = 0; i < slit->locality_count; i++) {
+		const int from_node = pxm_to_node(i);
+
+		if (from_node == NUMA_NO_NODE)
+			continue;
+
+		for (j = 0; j < slit->locality_count; j++) {
+			const int to_node = pxm_to_node(j);
+
+			if (to_node == NUMA_NO_NODE)
+				continue;
+
+			numa_set_distance(from_node, to_node,
+				slit->entry[slit->locality_count * i + j]);
+		}
+	}
+}
+#endif /* defined(CONFIG_X86) || defined (CONFIG_ARM64) */
+
 static int __init acpi_parse_slit(struct acpi_table_header *table)
 {
 	struct acpi_table_slit *slit = (struct acpi_table_slit *)table;
-- 
1.7.11.7

[PATCH v6 06/14] arm64, numa: rework numa_add_memblk()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Rework numa_add_memblk() to update the parameter "u64 size" to "u64
end", this will make it consistent with x86 and simplifies the arm64
ACPI NUMA code to be added later.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/mm/numa.c | 12 ++++++------
 drivers/of/of_numa.c |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index 98dc104..6cb03f9 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -131,25 +131,25 @@ void __init early_map_cpu_to_node(unsigned int cpu, int nid)
  * numa_add_memblk - Set node id to memblk
  * @nid: NUMA node ID of the new memblk
  * @start: Start address of the new memblk
- * @size:  Size of the new memblk
+ * @end:  End address of the new memblk
  *
  * RETURNS:
  * 0 on success, -errno on failure.
  */
-int __init numa_add_memblk(int nid, u64 start, u64 size)
+int __init numa_add_memblk(int nid, u64 start, u64 end)
 {
 	int ret;
 
-	ret = memblock_set_node(start, size, &memblock.memory, nid);
+	ret = memblock_set_node(start, (end - start), &memblock.memory, nid);
 	if (ret < 0) {
 		pr_err("NUMA: memblock [0x%llx - 0x%llx] failed to add on node %d\n",
-			start, (start + size - 1), nid);
+			start, (end - 1), nid);
 		return ret;
 	}
 
 	node_set(nid, numa_nodes_parsed);
 	pr_info("NUMA: Adding memblock [0x%llx - 0x%llx] on node %d\n",
-			start, (start + size - 1), nid);
+			start, (end - 1), nid);
 	return ret;
 }
 
@@ -367,7 +367,7 @@ static int __init dummy_numa_init(void)
 	       0LLU, PFN_PHYS(max_pfn) - 1);
 
 	for_each_memblock(memory, mblk) {
-		ret = numa_add_memblk(0, mblk->base, mblk->size);
+		ret = numa_add_memblk(0, mblk->base, mblk->base + mblk->size);
 		if (!ret)
 			continue;
 
diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index 0f2784b..ed5a097 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -91,8 +91,8 @@ static int __init of_numa_parse_memory_nodes(void)
 		pr_debug("NUMA:  base = %llx len = %llx, node = %u\n",
 			 rsrc.start, rsrc.end - rsrc.start + 1, nid);
 
-		r = numa_add_memblk(nid, rsrc.start,
-				    rsrc.end - rsrc.start + 1);
+
+		r = numa_add_memblk(nid, rsrc.start, rsrc.end + 1);
 		if (r)
 			break;
 	}
-- 
1.7.11.7

[PATCH v6 07/14] arm64, numa: Cleanup NUMA disabled messages.

[David Daney]

From: David Daney <david.daney@cavium.com>

As noted by Dennis Chen, we don't want to print "No NUMA configuration
found" if NUMA was forced off from the command line.

Change the type of numa_off to bool, and clean up printing code.
Print "NUMA disabled" if forced off on command line and "No NUMA
configuration found" if there was no firmware NUMA information.

Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/mm/numa.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index 6cb03f9..1def1de 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -29,7 +29,7 @@ static int cpu_to_node_map[NR_CPUS] = { [0 ... NR_CPUS-1] = NUMA_NO_NODE };
 
 static int numa_distance_cnt;
 static u8 *numa_distance;
-static int numa_off;
+static bool numa_off;
 
 static __init int numa_parse_early_param(char *opt)
 {
@@ -37,7 +37,7 @@ static __init int numa_parse_early_param(char *opt)
 		return -EINVAL;
 	if (!strncmp(opt, "off", 3)) {
 		pr_info("%s\n", "NUMA turned off");
-		numa_off = 1;
+		numa_off = true;
 	}
 	return 0;
 }
@@ -362,7 +362,10 @@ static int __init dummy_numa_init(void)
 	int ret;
 	struct memblock_region *mblk;
 
-	pr_info("%s\n", "No NUMA configuration found");
+	if (numa_off)
+		pr_info("NUMA disabled\n"); /* Forced off on command line. */
+	else
+		pr_info("No NUMA configuration found\n");
 	pr_info("NUMA: Faking a node at [mem %#018Lx-%#018Lx]\n",
 	       0LLU, PFN_PHYS(max_pfn) - 1);
 
@@ -375,7 +378,7 @@ static int __init dummy_numa_init(void)
 		return ret;
 	}
 
-	numa_off = 1;
+	numa_off = true;
 	return 0;
 }
 
-- 
1.7.11.7

[PATCH v6 08/14] x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Cleanup acpi_numa_processor_affinity_init() in preparation for its
move to drivers/acpi/numa.c.  It will be reused by arm64, this has no
functional change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c | 21 +++++----------------
 1 file changed, 5 insertions(+), 16 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index f242a11..9e2a833 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -26,11 +26,6 @@
 
 int acpi_numa __initdata;
 
-static __init int setup_node(int pxm)
-{
-	return acpi_map_pxm_to_node(pxm);
-}
-
 static __init void bad_srat(void)
 {
 	printk(KERN_ERR "SRAT: SRAT not used.\n");
@@ -64,7 +59,7 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 			 pxm, apic_id);
 		return;
 	}
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
 		bad_srat();
@@ -100,7 +95,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	pxm = pa->proximity_domain_lo;
 	if (acpi_srat_revision >= 2)
 		pxm |= *((unsigned int*)pa->proximity_domain_hi) << 8;
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
 		bad_srat();
@@ -124,12 +119,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	       pxm, apic_id, node);
 }
 
-#ifdef CONFIG_MEMORY_HOTPLUG
-static inline int save_add_info(void) {return 1;}
-#else
-static inline int save_add_info(void) {return 0;}
-#endif
-
 /* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
 int __init
 acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
@@ -145,7 +134,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
 		goto out_err;
 	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
-	if (hotpluggable && !save_add_info())
+	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
 		goto out_err;
 
 	start = ma->base_address;
@@ -154,7 +143,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if (acpi_srat_revision <= 1)
 		pxm &= 0xff;
 
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
 		goto out_err_bad_srat;
@@ -182,7 +171,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 out_err_bad_srat:
 	bad_srat();
 out_err:
-	return -1;
+	return -EINVAL;
 }
 
 int __init x86_acpi_numa_init(void)
-- 
1.7.11.7

[PATCH v6 09/14] acpi, numa: move bad_srat() and srat_disabled() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

bad_srat() and srat_disabled() are shared by x86 and follow-on arm64
patches.  Move them to drivers/acpi/numa.c in preparation for arm64
support.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
[david.daney@cavium.com moved definitions to drivers/acpi/numa.c]
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/include/asm/acpi.h |  1 -
 arch/x86/mm/numa.c          |  2 +-
 arch/x86/mm/srat.c          | 13 -------------
 drivers/acpi/numa.c         | 12 ++++++++++++
 include/acpi/acpi_numa.h    |  4 ++++
 5 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/arch/x86/include/asm/acpi.h b/arch/x86/include/asm/acpi.h
index 94c18eb..65f1e95 100644
--- a/arch/x86/include/asm/acpi.h
+++ b/arch/x86/include/asm/acpi.h
@@ -145,7 +145,6 @@ static inline void disable_acpi(void) { }
 #define ARCH_HAS_POWER_INIT	1
 
 #ifdef CONFIG_ACPI_NUMA
-extern int acpi_numa;
 extern int x86_acpi_numa_init(void);
 #endif /* CONFIG_ACPI_NUMA */
 
diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
index f70c1ff..57673f5 100644
--- a/arch/x86/mm/numa.c
+++ b/arch/x86/mm/numa.c
@@ -1,4 +1,5 @@
 /* Common code for 32 and 64-bit NUMA */
+#include <linux/acpi.h>
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/string.h>
@@ -15,7 +16,6 @@
 #include <asm/e820.h>
 #include <asm/proto.h>
 #include <asm/dma.h>
-#include <asm/acpi.h>
 #include <asm/amd_nb.h>
 
 #include "numa_internal.h"
diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 9e2a833..4217071f 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -24,19 +24,6 @@
 #include <asm/apic.h>
 #include <asm/uv/uv.h>
 
-int acpi_numa __initdata;
-
-static __init void bad_srat(void)
-{
-	printk(KERN_ERR "SRAT: SRAT not used.\n");
-	acpi_numa = -1;
-}
-
-static __init inline int srat_disabled(void)
-{
-	return acpi_numa < 0;
-}
-
 /* Callback for Proximity Domain -> x2APIC mapping */
 void __init
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 6fa528a..f6fd87c 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -40,6 +40,7 @@ static int node_to_pxm_map[MAX_NUMNODES]
 			= { [0 ... MAX_NUMNODES - 1] = PXM_INVAL };
 
 unsigned char acpi_srat_revision __initdata;
+int acpi_numa __initdata;
 
 int pxm_to_node(int pxm)
 {
@@ -197,6 +198,17 @@ static int __init slit_valid(struct acpi_table_slit *slit)
 	return 1;
 }
 
+void __init bad_srat(void)
+{
+	pr_err("SRAT: SRAT not used.\n");
+	acpi_numa = -1;
+}
+
+int __init srat_disabled(void)
+{
+	return acpi_numa < 0;
+}
+
 #if defined(CONFIG_X86) || defined (CONFIG_ARM64)
 /*
  * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
diff --git a/include/acpi/acpi_numa.h b/include/acpi/acpi_numa.h
index 94a37cd..d4b7294 100644
--- a/include/acpi/acpi_numa.h
+++ b/include/acpi/acpi_numa.h
@@ -15,6 +15,10 @@ extern int pxm_to_node(int);
 extern int node_to_pxm(int);
 extern int acpi_map_pxm_to_node(int);
 extern unsigned char acpi_srat_revision;
+extern int acpi_numa __initdata;
+
+extern void bad_srat(void);
+extern int srat_disabled(void);
 
 #endif				/* CONFIG_ACPI_NUMA */
 #endif				/* __ACP_NUMA_H */
-- 
1.7.11.7

[PATCH v6 10/14] acpi, numa: remove unneeded acpi_numa=1

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

acpi_numa is default to 0, it's set to -1 when disable acpi numa or
when a bad SRAT is parsed, and it's only consumed in srat_disabled()
(compare it with 0) to continue parse the SRAT or not, so we don't
need to set acpi_numa to 1 when we get a valid SRAT entry.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 4217071f..30460f5 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -59,7 +59,6 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 	}
 	set_apicid_to_node(apic_id, node);
 	node_set(node, numa_nodes_parsed);
-	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%04x -> Node %u\n",
 	       pxm, apic_id, node);
 }
@@ -101,7 +100,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 
 	set_apicid_to_node(apic_id, node);
 	node_set(node, numa_nodes_parsed);
-	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%02x -> Node %u\n",
 	       pxm, apic_id, node);
 }
-- 
1.7.11.7

[PATCH v6 11/14] acpi, numa: Move acpi_numa_memory_affinity_init() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

acpi_numa_memory_affinity_init() will be reused by arm64.  Move it to
drivers/acpi/numa.c to facilitate reuse.

No code change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c  | 57 --------------------------------------------------
 drivers/acpi/numa.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+), 57 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 30460f5..b1ecff4 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -15,8 +15,6 @@
 #include <linux/bitmap.h>
 #include <linux/module.h>
 #include <linux/topology.h>
-#include <linux/bootmem.h>
-#include <linux/memblock.h>
 #include <linux/mm.h>
 #include <asm/proto.h>
 #include <asm/numa.h>
@@ -104,61 +102,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	       pxm, apic_id, node);
 }
 
-/* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
-int __init
-acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
-{
-	u64 start, end;
-	u32 hotpluggable;
-	int node, pxm;
-
-	if (srat_disabled())
-		goto out_err;
-	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
-		goto out_err_bad_srat;
-	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
-		goto out_err;
-	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
-	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
-		goto out_err;
-
-	start = ma->base_address;
-	end = start + ma->length;
-	pxm = ma->proximity_domain;
-	if (acpi_srat_revision <= 1)
-		pxm &= 0xff;
-
-	node = acpi_map_pxm_to_node(pxm);
-	if (node < 0) {
-		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
-		goto out_err_bad_srat;
-	}
-
-	if (numa_add_memblk(node, start, end) < 0)
-		goto out_err_bad_srat;
-
-	node_set(node, numa_nodes_parsed);
-
-	pr_info("SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]%s%s\n",
-		node, pxm,
-		(unsigned long long) start, (unsigned long long) end - 1,
-		hotpluggable ? " hotplug" : "",
-		ma->flags & ACPI_SRAT_MEM_NON_VOLATILE ? " non-volatile" : "");
-
-	/* Mark hotplug range in memblock. */
-	if (hotpluggable && memblock_mark_hotplug(start, ma->length))
-		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
-			(unsigned long long)start, (unsigned long long)end - 1);
-
-	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
-
-	return 0;
-out_err_bad_srat:
-	bad_srat();
-out_err:
-	return -EINVAL;
-}
-
 int __init x86_acpi_numa_init(void)
 {
 	int ret;
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index f6fd87c..d7d0499 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -27,6 +27,8 @@
 #include <linux/types.h>
 #include <linux/errno.h>
 #include <linux/acpi.h>
+#include <linux/bootmem.h>
+#include <linux/memblock.h>
 #include <linux/numa.h>
 #include <linux/nodemask.h>
 #include <linux/topology.h>
@@ -236,6 +238,64 @@ void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
 		}
 	}
 }
+
+/*
+ * Default callback for parsing of the Proximity Domain <-> Memory
+ * Area mappings
+ */
+int __init
+acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
+{
+	u64 start, end;
+	u32 hotpluggable;
+	int node, pxm;
+
+	if (srat_disabled())
+		goto out_err;
+	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
+		goto out_err_bad_srat;
+	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
+		goto out_err;
+	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
+	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		goto out_err;
+
+	start = ma->base_address;
+	end = start + ma->length;
+	pxm = ma->proximity_domain;
+	if (acpi_srat_revision <= 1)
+		pxm &= 0xff;
+
+	node = acpi_map_pxm_to_node(pxm);
+	if (node < 0) {
+		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
+		goto out_err_bad_srat;
+	}
+
+	if (numa_add_memblk(node, start, end) < 0)
+		goto out_err_bad_srat;
+
+	node_set(node, numa_nodes_parsed);
+
+	pr_info("SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]%s%s\n",
+		node, pxm,
+		(unsigned long long) start, (unsigned long long) end - 1,
+		hotpluggable ? " hotplug" : "",
+		ma->flags & ACPI_SRAT_MEM_NON_VOLATILE ? " non-volatile" : "");
+
+	/* Mark hotplug range in memblock. */
+	if (hotpluggable && memblock_mark_hotplug(start, ma->length))
+		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
+			(unsigned long long)start, (unsigned long long)end - 1);
+
+	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
+
+	return 0;
+out_err_bad_srat:
+	bad_srat();
+out_err:
+	return -EINVAL;
+}
 #endif /* defined(CONFIG_X86) || defined (CONFIG_ARM64) */
 
 static int __init acpi_parse_slit(struct acpi_table_header *table)
-- 
1.7.11.7

[PATCH v6 12/14] acpi, numa, srat: Improve SRAT error detection and add messages.

[David Daney]

From: David Daney <david.daney@cavium.com>

Loosely based on code from Robert Richter and Hanjun Guo.

Improve out of range node detection as well as allow for Larger SRAT
entities.

Add printing of nice messages.

Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index d7d0499..50c8f42 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -252,8 +252,11 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 
 	if (srat_disabled())
 		goto out_err;
-	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
+	if (ma->header.length < sizeof(struct acpi_srat_mem_affinity)) {
+		pr_err("SRAT: Unexpected header length: %d\n",
+		       ma->header.length);
 		goto out_err_bad_srat;
+	}
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
 		goto out_err;
 	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
@@ -267,13 +270,17 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 		pxm &= 0xff;
 
 	node = acpi_map_pxm_to_node(pxm);
-	if (node < 0) {
-		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
+	if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
+		pr_err("SRAT: Too many proximity domains.\n");
 		goto out_err_bad_srat;
 	}
 
-	if (numa_add_memblk(node, start, end) < 0)
+	if (numa_add_memblk(node, start, end) < 0) {
+		pr_err("SRAT: Failed to add memblk to node %u [mem %#010Lx-%#010Lx]\n",
+		       node, (unsigned long long) start,
+		       (unsigned long long) end - 1);
 		goto out_err_bad_srat;
+	}
 
 	node_set(node, numa_nodes_parsed);
 
-- 
1.7.11.7

[PATCH v6 13/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Introduce a new file to hold ACPI based NUMA information parsing from
SRAT and SLIT.

SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
ranges to Proximity Domain mapping.  SLIT has the information of inter
node distances(relative number for access latency).

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
[rrichter@cavium.com Reworked for numa v10 series ]
Signed-off-by: Robert Richter <rrichter@cavium.com>
[david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/include/asm/acpi.h |   8 +++
 arch/arm64/include/asm/numa.h |   2 +
 arch/arm64/kernel/Makefile    |   1 +
 arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
 arch/arm64/kernel/smp.c       |   2 +
 arch/arm64/mm/numa.c          |   5 +-
 6 files changed, 166 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/kernel/acpi_numa.c

diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
index aee323b..4b13ecd 100644
--- a/arch/arm64/include/asm/acpi.h
+++ b/arch/arm64/include/asm/acpi.h
@@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
 pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
 #endif
 
+#ifdef CONFIG_ACPI_NUMA
+int arm64_acpi_numa_init(void);
+int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
+#else
+static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
+static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
+#endif /* CONFIG_ACPI_NUMA */
+
 #endif /*_ASM_ACPI_H*/
diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
index e9b4f29..600887e 100644
--- a/arch/arm64/include/asm/numa.h
+++ b/arch/arm64/include/asm/numa.h
@@ -5,6 +5,8 @@
 
 #ifdef CONFIG_NUMA
 
+#define NR_NODE_MEMBLKS		(MAX_NUMNODES * 2)
+
 /* currently, arm64 implements flat NUMA topology */
 #define parent_node(node)	(node)
 
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 3793003..69569c6 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)			+= efi.o efi-entry.stub.o
 arm64-obj-$(CONFIG_PCI)			+= pci.o
 arm64-obj-$(CONFIG_ARMV8_DEPRECATED)	+= armv8_deprecated.o
 arm64-obj-$(CONFIG_ACPI)		+= acpi.o
+arm64-obj-$(CONFIG_ACPI_NUMA)		+= acpi_numa.o
 arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)	+= acpi_parking_protocol.o
 arm64-obj-$(CONFIG_PARAVIRT)		+= paravirt.o
 arm64-obj-$(CONFIG_RANDOMIZE_BASE)	+= kaslr.o
diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
new file mode 100644
index 0000000..94a6b35
--- /dev/null
+++ b/arch/arm64/kernel/acpi_numa.c
@@ -0,0 +1,149 @@
+/*
+ * ACPI 5.1 based NUMA setup for ARM64
+ * Lots of code was borrowed from arch/x86/mm/srat.c
+ *
+ * Copyright 2004 Andi Kleen, SuSE Labs.
+ * Copyright (C) 2013-2016, Linaro Ltd.
+ *		Author: Hanjun Guo <hanjun.guo@linaro.org>
+ *
+ * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
+ *
+ * Called from acpi_numa_init while reading the SRAT and SLIT tables.
+ * Assumes all memory regions belonging to a single proximity domain
+ * are in one chunk. Holes between them will be included in the node.
+ */
+
+#define pr_fmt(fmt) "ACPI: NUMA: " fmt
+
+#include <linux/acpi.h>
+#include <linux/bitmap.h>
+#include <linux/bootmem.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/memblock.h>
+#include <linux/mmzone.h>
+#include <linux/module.h>
+#include <linux/topology.h>
+
+#include <acpi/processor.h>
+#include <asm/numa.h>
+
+static int cpus_in_srat;
+
+struct __node_cpu_hwid {
+	u32 node_id;    /* logical node containing this CPU */
+	u64 cpu_hwid;   /* MPIDR for this CPU */
+};
+
+static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
+[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
+
+int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
+{
+	int i;
+
+	for (i = 0; i < cpus_in_srat; i++) {
+		if (hwid == early_node_cpu_hwid[i].cpu_hwid)
+			return early_node_cpu_hwid[i].node_id;
+	}
+
+	return NUMA_NO_NODE;
+}
+
+static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
+{
+	unsigned long madt_end, entry;
+	struct acpi_table_madt *madt;
+	acpi_size tbl_size;
+
+	if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
+			(struct acpi_table_header **)&madt, &tbl_size)))
+		return -ENODEV;
+
+	entry = (unsigned long)madt;
+	madt_end = entry + madt->header.length;
+
+	/* Parse all entries looking for a match. */
+	entry += sizeof(struct acpi_table_madt);
+	while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
+		struct acpi_subtable_header *header =
+			(struct acpi_subtable_header *)entry;
+
+		if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
+			struct acpi_madt_generic_interrupt *gicc =
+				container_of(header,
+				struct acpi_madt_generic_interrupt, header);
+
+			if ((gicc->flags & ACPI_MADT_ENABLED) &&
+			    (gicc->uid == acpi_id)) {
+				*mpidr = gicc->arm_mpidr;
+				early_acpi_os_unmap_memory(madt, tbl_size);
+				return 0;
+			}
+		}
+		entry += header->length;
+	}
+
+	early_acpi_os_unmap_memory(madt, tbl_size);
+	return -ENODEV;
+}
+
+/* Callback for Proximity Domain -> ACPI processor UID mapping */
+void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
+{
+	int pxm, node;
+	u64 mpidr;
+
+	if (srat_disabled())
+		return;
+
+	if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
+		pr_err("SRAT: Invalid SRAT header length: %d\n",
+			pa->header.length);
+		bad_srat();
+		return;
+	}
+
+	if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
+		return;
+
+	if (cpus_in_srat >= NR_CPUS) {
+		pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
+			     NR_CPUS);
+		return;
+	}
+
+	pxm = pa->proximity_domain;
+	node = acpi_map_pxm_to_node(pxm);
+
+	if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
+		pr_err("SRAT: Too many proximity domains %d\n", pxm);
+		bad_srat();
+		return;
+	}
+
+	if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
+		pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
+			pxm, pa->acpi_processor_uid);
+		bad_srat();
+		return;
+	}
+
+	early_node_cpu_hwid[cpus_in_srat].node_id = node;
+	early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
+	node_set(node, numa_nodes_parsed);
+	cpus_in_srat++;
+	pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d\n",
+		pxm, mpidr, node);
+}
+
+int __init arm64_acpi_numa_init(void)
+{
+	int ret;
+
+	ret = acpi_numa_init();
+	if (ret)
+		return ret;
+
+	return srat_disabled() ? -EINVAL : 0;
+}
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index fef2e73..fe85940 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -562,6 +562,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
 	 */
 	acpi_set_mailbox_entry(cpu_count, processor);
 
+	early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
+
 	cpu_count++;
 }
 
diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index 1def1de..c7fe3ec 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -17,6 +17,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/acpi.h>
 #include <linux/bootmem.h>
 #include <linux/memblock.h>
 #include <linux/module.h>
@@ -391,7 +392,9 @@ static int __init dummy_numa_init(void)
 void __init arm64_numa_init(void)
 {
 	if (!numa_off) {
-		if (!numa_init(of_numa_init))
+		if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
+			return;
+		if (acpi_disabled && !numa_init(of_numa_init))
 			return;
 	}
 
-- 
1.7.11.7

[PATCH v6 14/14] acpi, numa: Enable ACPI based NUMA on ARM64

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Add function needed for cpu to node mapping, and enable ACPI based
NUMA for ARM64 in Kconfig

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
[david.daney@cavium.com added ACPI_NUMA default to y for ARM64]
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/Kconfig |  4 ++--
 drivers/acpi/numa.c  | 35 +++++++++++++++++++++++++++++++++++
 include/linux/acpi.h | 17 ++++++++++++++++-
 3 files changed, 53 insertions(+), 3 deletions(-)

diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index fa91ea2..5a3b161 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -291,8 +291,8 @@ config ACPI_THERMAL
 config ACPI_NUMA
 	bool "NUMA support"
 	depends on NUMA
-	depends on (X86 || IA64)
-	default y if IA64_GENERIC || IA64_SGI_SN2
+	depends on (X86 || IA64 || ARM64)
+	default y if IA64_GENERIC || IA64_SGI_SN2 || ARM64
 
 config ACPI_CUSTOM_DSDT_FILE
 	string "Custom DSDT Table file to include"
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 50c8f42..f6696ac 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -170,6 +170,18 @@ acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 		}
 		break;
 
+	case ACPI_SRAT_TYPE_GICC_AFFINITY:
+		{
+			struct acpi_srat_gicc_affinity *p =
+			    (struct acpi_srat_gicc_affinity *)header;
+			pr_debug("SRAT Processor (acpi id[0x%04x]) in proximity domain %d %s\n",
+				 p->acpi_processor_uid,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_GICC_ENABLED) ?
+				 "enabled" : "disabled");
+		}
+		break;
+
 	default:
 		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
 			header->type);
@@ -360,6 +372,24 @@ acpi_parse_processor_affinity(struct acpi_subtable_header *header,
 	return 0;
 }
 
+static int __init
+acpi_parse_gicc_affinity(struct acpi_subtable_header *header,
+			 const unsigned long end)
+{
+	struct acpi_srat_gicc_affinity *processor_affinity;
+
+	processor_affinity = (struct acpi_srat_gicc_affinity *)header;
+	if (!processor_affinity)
+		return -EINVAL;
+
+	acpi_table_print_srat_entry(header);
+
+	/* let architecture-dependent part to do it */
+	acpi_numa_gicc_affinity_init(processor_affinity);
+
+	return 0;
+}
+
 static int __initdata parsed_numa_memblks;
 
 static int __init
@@ -404,6 +434,9 @@ int __init acpi_numa_init(void)
 {
 	int cnt = 0;
 
+	if (acpi_disabled)
+		return -EINVAL;
+
 	/*
 	 * Should not limit number with cpu num that is from NR_CPUS or nr_cpus=
 	 * SRAT cpu entries could have different order with that in MADT.
@@ -416,6 +449,8 @@ int __init acpi_numa_init(void)
 				     acpi_parse_x2apic_affinity, 0);
 		acpi_table_parse_srat(ACPI_SRAT_TYPE_CPU_AFFINITY,
 				     acpi_parse_processor_affinity, 0);
+		acpi_table_parse_srat(ACPI_SRAT_TYPE_GICC_AFFINITY,
+				      acpi_parse_gicc_affinity, 0);
 		cnt = acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
 					    acpi_parse_memory_affinity,
 					    NR_NODE_MEMBLKS);
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index ab6fd96..af34064 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -239,10 +239,25 @@ int acpi_table_parse_madt(enum acpi_madt_type id,
 int acpi_parse_mcfg (struct acpi_table_header *header);
 void acpi_table_print_madt_entry (struct acpi_subtable_header *madt);
 
-/* the following four functions are architecture-dependent */
+/* the following numa functions are architecture-dependent */
 void acpi_numa_slit_init (struct acpi_table_slit *slit);
+
+#if defined(CONFIG_X86) || defined(CONFIG_IA64)
 void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
+#else
+static inline void
+acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa) { }
+#endif
+
 void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
+
+#ifdef CONFIG_ARM64
+void acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa);
+#else
+static inline void
+acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa) { }
+#endif
+
 int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
 
 #ifndef PHYS_CPUID_INVALID
-- 
1.7.11.7

Re: [PATCH v6 00/14] ACPI NUMA support for ARM64

[Rafael J. Wysocki]

On Wed, Apr 27, 2016 at 8:07 PM, David Daney <ddaney.cavm@gmail.com> wrote:
> From: David Daney <david.daney@cavium.com>
>
> Based on git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
> for-next/core branch at commit 643d703d2d2d ("arm64: compat: Check for
> AArch32 state")
>
> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
> NUMA domain information from SRAT and SLIT table, so parse those two
> tables to get mappings from cpu/mem to numa node configuration and
> system locality.
>
> v6 updates:
>
>  - Changed message printed in 13/14 to omit meaningless "cpu"
>    number. (noted by Dennis Chen and Hanjun Guo)
>
>  - New patch 7/14 to print different messages for NUMA disabled case.
>    (noted by Dennis Chen)
>
>  - Squashed two patches into 14/14.
>
>  - Added Reviewed-by to 4/14
>
> v5 updates:
>
>  - Fixed ia64 build failure by gating some unused functions with #if
>    CONFIG_{X86,ARM64}.
>
>  - Fixed section mismatch errors for X86 case.
>
>  - Removed unneeded #include from some files.
>
>  - Tested to build cleanly on ARM64, X86_64, IA64
>
> v4 updates:
>
>  - Updated from Hanjun Guo's v3 patches.
>
>  - Rebased on top of v16 of device-tree NUMA patches.
>
>  - Reordered some of the changes so that we don't introduce code and
>    then change it several times in the patch set.  New code is
>    introduced in its final form.  Code reused from x86 is first moved
>    with no change, and then a separate patch to make any needed
>    changes.
>
>  - code that is used only by ia64, moved to architecture specific
>    files.
>
> v3 updates:
>  - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
>    that for acpi/numa.c (patch 2/12)
>
>  - Remove the duplicate NULL check for table print (patch 3/12)
>
>  - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
>    dummy function for acpi_numa_arch_fixup()
>
>  - Solve the problem that the mapping from logical cpu to numa node
>    is wrong which spotted out by Lorenzo
>
>  - cleanups for x86 and move acpi_numa_slit_init() and some other
>    functions to common place, then reduce the duplicate of x86
>    and arm64 (patch 7-12/12).
>
>  - rebased on top of 4.4 and Ganapat's v9 patch set.
>
>
> David Daney (2):
>   arm64, numa: Cleanup NUMA disabled messages.
>   acpi, numa, srat: Improve SRAT error detection and add messages.
>
> Hanjun Guo (11):
>   acpi, numa: Use pr_fmt() instead of printk
>   acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>   acpi, numa: remove duplicate NULL check
>   acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>   arm64, numa: rework numa_add_memblk()
>   x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>   acpi, numa: move bad_srat() and srat_disabled() to
>     drivers/acpi/numa.c
>   acpi, numa: remove unneeded acpi_numa=1
>   acpi, numa: Move acpi_numa_memory_affinity_init() to
>     drivers/acpi/numa.c
>   arm64, acpi, numa: NUMA support based on SRAT and SLIT
>   acpi, numa: Enable ACPI based NUMA on ARM64
>
> Robert Richter (1):
>   acpi, numa: Move acpi_numa_arch_fixup() to ia64 only

I need ACKs from the ARM64 maintainers on patches [6-7/13] and [13-14/14].

The rest looks good to me, so I can apply the series once the above
have been ACKed.

Thanks,
Rafael

Re: [PATCH v6 06/14] arm64, numa: rework numa_add_memblk()

[Catalin Marinas]

On Wed, Apr 27, 2016 at 11:07:08AM -0700, David Daney wrote:
> From: Hanjun Guo <hanjun.guo@linaro.org>
> 
> Rework numa_add_memblk() to update the parameter "u64 size" to "u64
> end", this will make it consistent with x86 and simplifies the arm64
> ACPI NUMA code to be added later.
> 
> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> Signed-off-by: David Daney <david.daney@cavium.com>

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

Re: [PATCH v6 07/14] arm64, numa: Cleanup NUMA disabled messages.

[Catalin Marinas]

On Wed, Apr 27, 2016 at 11:07:09AM -0700, David Daney wrote:
> From: David Daney <david.daney@cavium.com>
> 
> As noted by Dennis Chen, we don't want to print "No NUMA configuration
> found" if NUMA was forced off from the command line.
> 
> Change the type of numa_off to bool, and clean up printing code.
> Print "NUMA disabled" if forced off on command line and "No NUMA
> configuration found" if there was no firmware NUMA information.
> 
> Signed-off-by: David Daney <david.daney@cavium.com>

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

Re: [PATCH v6 13/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Catalin Marinas]

On Wed, Apr 27, 2016 at 11:07:15AM -0700, David Daney wrote:
> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
> +{
> +	unsigned long madt_end, entry;
> +	struct acpi_table_madt *madt;
> +	acpi_size tbl_size;
> +
> +	if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
> +			(struct acpi_table_header **)&madt, &tbl_size)))
> +		return -ENODEV;
> +
> +	entry = (unsigned long)madt;
> +	madt_end = entry + madt->header.length;
> +
> +	/* Parse all entries looking for a match. */
> +	entry += sizeof(struct acpi_table_madt);
> +	while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
> +		struct acpi_subtable_header *header =
> +			(struct acpi_subtable_header *)entry;
> +
> +		if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
> +			struct acpi_madt_generic_interrupt *gicc =
> +				container_of(header,
> +				struct acpi_madt_generic_interrupt, header);
> +
> +			if ((gicc->flags & ACPI_MADT_ENABLED) &&
> +			    (gicc->uid == acpi_id)) {
> +				*mpidr = gicc->arm_mpidr;
> +				early_acpi_os_unmap_memory(madt, tbl_size);
> +				return 0;
> +			}
> +		}
> +		entry += header->length;
> +	}
> +
> +	early_acpi_os_unmap_memory(madt, tbl_size);
> +	return -ENODEV;
> +}
> +
> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
> +{
> +	int pxm, node;
> +	u64 mpidr;
> +
> +	if (srat_disabled())
> +		return;
> +
> +	if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
> +		pr_err("SRAT: Invalid SRAT header length: %d\n",
> +			pa->header.length);
> +		bad_srat();
> +		return;
> +	}
> +
> +	if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
> +		return;
> +
> +	if (cpus_in_srat >= NR_CPUS) {
> +		pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
> +			     NR_CPUS);
> +		return;
> +	}
> +
> +	pxm = pa->proximity_domain;
> +	node = acpi_map_pxm_to_node(pxm);
> +
> +	if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
> +		pr_err("SRAT: Too many proximity domains %d\n", pxm);
> +		bad_srat();
> +		return;
> +	}
> +
> +	if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
> +		pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
> +			pxm, pa->acpi_processor_uid);
> +		bad_srat();
> +		return;
> +	}

I wonder whether you could replace the get_mpidr_in_madt() function with
something like acpi_get_phys_id(). It looks like get_mpidr_in_madt()
duplicates functionality already available elsewhere.

-- 
Catalin

Re: [PATCH v6 14/14] acpi, numa: Enable ACPI based NUMA on ARM64

[Catalin Marinas]

On Wed, Apr 27, 2016 at 11:07:16AM -0700, David Daney wrote:
> From: Hanjun Guo <hanjun.guo@linaro.org>
> 
> Add function needed for cpu to node mapping, and enable ACPI based
> NUMA for ARM64 in Kconfig
> 
> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> [david.daney@cavium.com added ACPI_NUMA default to y for ARM64]
> Signed-off-by: David Daney <david.daney@cavium.com>

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

Re: [PATCH v6 00/14] ACPI NUMA support for ARM64

[Will Deacon]

On Wed, May 11, 2016 at 02:43:11AM +0200, Rafael J. Wysocki wrote:
> On Wed, Apr 27, 2016 at 8:07 PM, David Daney <ddaney.cavm@gmail.com> wrote:
> > From: David Daney <david.daney@cavium.com>
> >
> > Based on git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
> > for-next/core branch at commit 643d703d2d2d ("arm64: compat: Check for
> > AArch32 state")

[...]

> > David Daney (2):
> >   arm64, numa: Cleanup NUMA disabled messages.
> >   acpi, numa, srat: Improve SRAT error detection and add messages.
> >
> > Hanjun Guo (11):
> >   acpi, numa: Use pr_fmt() instead of printk
> >   acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
> >   acpi, numa: remove duplicate NULL check
> >   acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
> >   arm64, numa: rework numa_add_memblk()
> >   x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
> >   acpi, numa: move bad_srat() and srat_disabled() to
> >     drivers/acpi/numa.c
> >   acpi, numa: remove unneeded acpi_numa=1
> >   acpi, numa: Move acpi_numa_memory_affinity_init() to
> >     drivers/acpi/numa.c
> >   arm64, acpi, numa: NUMA support based on SRAT and SLIT
> >   acpi, numa: Enable ACPI based NUMA on ARM64
> >
> > Robert Richter (1):
> >   acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
> 
> I need ACKs from the ARM64 maintainers on patches [6-7/13] and [13-14/14].

There's also a dependency on the arm64 for-next/core branch, so I've been
largely ignoring this as far as 4.6 is concerned and was planning to take
a proper look for 4.7 once the upcoming merge window is out of the way.

Will

Re: [PATCH v6 00/14] ACPI NUMA support for ARM64

[Rafael J. Wysocki]

On Wed, May 11, 2016 at 12:40 PM, Will Deacon <will.deacon@arm.com> wrote:
> On Wed, May 11, 2016 at 02:43:11AM +0200, Rafael J. Wysocki wrote:
>> On Wed, Apr 27, 2016 at 8:07 PM, David Daney <ddaney.cavm@gmail.com> wrote:
>> > From: David Daney <david.daney@cavium.com>
>> >
>> > Based on git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
>> > for-next/core branch at commit 643d703d2d2d ("arm64: compat: Check for
>> > AArch32 state")
>
> [...]
>
>> > David Daney (2):
>> >   arm64, numa: Cleanup NUMA disabled messages.
>> >   acpi, numa, srat: Improve SRAT error detection and add messages.
>> >
>> > Hanjun Guo (11):
>> >   acpi, numa: Use pr_fmt() instead of printk
>> >   acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>> >   acpi, numa: remove duplicate NULL check
>> >   acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>> >   arm64, numa: rework numa_add_memblk()
>> >   x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>> >   acpi, numa: move bad_srat() and srat_disabled() to
>> >     drivers/acpi/numa.c
>> >   acpi, numa: remove unneeded acpi_numa=1
>> >   acpi, numa: Move acpi_numa_memory_affinity_init() to
>> >     drivers/acpi/numa.c
>> >   arm64, acpi, numa: NUMA support based on SRAT and SLIT
>> >   acpi, numa: Enable ACPI based NUMA on ARM64
>> >
>> > Robert Richter (1):
>> >   acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>>
>> I need ACKs from the ARM64 maintainers on patches [6-7/13] and [13-14/14].
>
> There's also a dependency on the arm64 for-next/core branch, so I've been
> largely ignoring this as far as 4.6 is concerned and was planning to take
> a proper look for 4.7 once the upcoming merge window is out of the way.

That would be 4.7 and 4.8 respectively I suppose?

Anyway, Catalin has ACKed all of them except for the [13/14], so
technically I can apply [1-12/14] now and then [13-14/14] can be
applied when they are ready.

Do you think there will be any problems with merging [6-7/14] into 4.7
via the ACPI tree?

Re: [PATCH v6 00/14] ACPI NUMA support for ARM64

[David Daney]

On 05/11/2016 01:35 PM, Rafael J. Wysocki wrote:
> On Wed, May 11, 2016 at 12:40 PM, Will Deacon <will.deacon@arm.com> wrote:
>> On Wed, May 11, 2016 at 02:43:11AM +0200, Rafael J. Wysocki wrote:
>>> On Wed, Apr 27, 2016 at 8:07 PM, David Daney <ddaney.cavm@gmail.com> wrote:
>>>> From: David Daney <david.daney@cavium.com>
>>>>
>>>> Based on git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
>>>> for-next/core branch at commit 643d703d2d2d ("arm64: compat: Check for
>>>> AArch32 state")
>>
>> [...]
>>
>>>> David Daney (2):
>>>>    arm64, numa: Cleanup NUMA disabled messages.
>>>>    acpi, numa, srat: Improve SRAT error detection and add messages.
>>>>
>>>> Hanjun Guo (11):
>>>>    acpi, numa: Use pr_fmt() instead of printk
>>>>    acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>>>>    acpi, numa: remove duplicate NULL check
>>>>    acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>>>>    arm64, numa: rework numa_add_memblk()
>>>>    x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>>>>    acpi, numa: move bad_srat() and srat_disabled() to
>>>>      drivers/acpi/numa.c
>>>>    acpi, numa: remove unneeded acpi_numa=1
>>>>    acpi, numa: Move acpi_numa_memory_affinity_init() to
>>>>      drivers/acpi/numa.c
>>>>    arm64, acpi, numa: NUMA support based on SRAT and SLIT
>>>>    acpi, numa: Enable ACPI based NUMA on ARM64
>>>>
>>>> Robert Richter (1):
>>>>    acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>>>
>>> I need ACKs from the ARM64 maintainers on patches [6-7/13] and [13-14/14].
>>
>> There's also a dependency on the arm64 for-next/core branch, so I've been
>> largely ignoring this as far as 4.6 is concerned and was planning to take
>> a proper look for 4.7 once the upcoming merge window is out of the way.
>
> That would be 4.7 and 4.8 respectively I suppose?
>
> Anyway, Catalin has ACKed all of them except for the [13/14], so
> technically I can apply [1-12/14] now and then [13-14/14] can be
> applied when they are ready.
>
> Do you think there will be any problems with merging [6-7/14] into 4.7
> via the ACPI tree?
>

I would defer to the arm64 maintainers for decisions about the arm64 
specific parts of the patch set.  That said, many of the arm64 specific 
patches depend on the arm64 for-next/core branch, so you would have to 
be careful about merge ordering if you pull these in before the 
for-next/core branch is merged.

Also FWIW, I plan on addressing Catalin's comments about 13/14 and 
posting a new version of the patch set in the next day or two.

Thanks for looking at these,
David Daney

Re: [PATCH v6 00/14] ACPI NUMA support for ARM64

[Rafael J. Wysocki]

On Wed, May 11, 2016 at 11:08 PM, David Daney <ddaney@caviumnetworks.com> wrote:
> On 05/11/2016 01:35 PM, Rafael J. Wysocki wrote:
>>
>> On Wed, May 11, 2016 at 12:40 PM, Will Deacon <will.deacon@arm.com> wrote:
>>>
>>> On Wed, May 11, 2016 at 02:43:11AM +0200, Rafael J. Wysocki wrote:
>>>>
>>>> On Wed, Apr 27, 2016 at 8:07 PM, David Daney <ddaney.cavm@gmail.com>
>>>> wrote:
>>>>>
>>>>> From: David Daney <david.daney@cavium.com>
>>>>>
>>>>> Based on git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
>>>>> for-next/core branch at commit 643d703d2d2d ("arm64: compat: Check for
>>>>> AArch32 state")
>>>
>>>
>>> [...]
>>>
>>>>> David Daney (2):
>>>>>    arm64, numa: Cleanup NUMA disabled messages.
>>>>>    acpi, numa, srat: Improve SRAT error detection and add messages.
>>>>>
>>>>> Hanjun Guo (11):
>>>>>    acpi, numa: Use pr_fmt() instead of printk
>>>>>    acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>>>>>    acpi, numa: remove duplicate NULL check
>>>>>    acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>>>>>    arm64, numa: rework numa_add_memblk()
>>>>>    x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>>>>>    acpi, numa: move bad_srat() and srat_disabled() to
>>>>>      drivers/acpi/numa.c
>>>>>    acpi, numa: remove unneeded acpi_numa=1
>>>>>    acpi, numa: Move acpi_numa_memory_affinity_init() to
>>>>>      drivers/acpi/numa.c
>>>>>    arm64, acpi, numa: NUMA support based on SRAT and SLIT
>>>>>    acpi, numa: Enable ACPI based NUMA on ARM64
>>>>>
>>>>> Robert Richter (1):
>>>>>    acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>>>>
>>>>
>>>> I need ACKs from the ARM64 maintainers on patches [6-7/13] and
>>>> [13-14/14].
>>>
>>>
>>> There's also a dependency on the arm64 for-next/core branch, so I've been
>>> largely ignoring this as far as 4.6 is concerned and was planning to take
>>> a proper look for 4.7 once the upcoming merge window is out of the way.
>>
>>
>> That would be 4.7 and 4.8 respectively I suppose?
>>
>> Anyway, Catalin has ACKed all of them except for the [13/14], so
>> technically I can apply [1-12/14] now and then [13-14/14] can be
>> applied when they are ready.
>>
>> Do you think there will be any problems with merging [6-7/14] into 4.7
>> via the ACPI tree?
>>
>
> I would defer to the arm64 maintainers for decisions about the arm64
> specific parts of the patch set.  That said, many of the arm64 specific
> patches depend on the arm64 for-next/core branch, so you would have to be
> careful about merge ordering if you pull these in before the for-next/core
> branch is merged.

Fair enough.  I will wait for an update then.

> Also FWIW, I plan on addressing Catalin's comments about 13/14 and posting a
> new version of the patch set in the next day or two.

OK, but in that case it won't be considered for 4.7 (at least not by
me), so I'd suggest sending it in the second half of the 4.7 merge
window (or about that time).

Thanks,
Rafael

Re: [PATCH v6 00/14] ACPI NUMA support for ARM64

[David Daney]

On 05/11/2016 02:22 PM, Rafael J. Wysocki wrote:
> On Wed, May 11, 2016 at 11:08 PM, David Daney <ddaney@caviumnetworks.com> wrote:
>> On 05/11/2016 01:35 PM, Rafael J. Wysocki wrote:
>>>
>>> On Wed, May 11, 2016 at 12:40 PM, Will Deacon <will.deacon@arm.com> wrote:
>>>>
>>>> On Wed, May 11, 2016 at 02:43:11AM +0200, Rafael J. Wysocki wrote:
>>>>>
>>>>> On Wed, Apr 27, 2016 at 8:07 PM, David Daney <ddaney.cavm@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> From: David Daney <david.daney@cavium.com>
>>>>>>
>>>>>> Based on git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
>>>>>> for-next/core branch at commit 643d703d2d2d ("arm64: compat: Check for
>>>>>> AArch32 state")
>>>>
>>>>
>>>> [...]
>>>>
>>>>>> David Daney (2):
>>>>>>     arm64, numa: Cleanup NUMA disabled messages.
>>>>>>     acpi, numa, srat: Improve SRAT error detection and add messages.
>>>>>>
>>>>>> Hanjun Guo (11):
>>>>>>     acpi, numa: Use pr_fmt() instead of printk
>>>>>>     acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>>>>>>     acpi, numa: remove duplicate NULL check
>>>>>>     acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>>>>>>     arm64, numa: rework numa_add_memblk()
>>>>>>     x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>>>>>>     acpi, numa: move bad_srat() and srat_disabled() to
>>>>>>       drivers/acpi/numa.c
>>>>>>     acpi, numa: remove unneeded acpi_numa=1
>>>>>>     acpi, numa: Move acpi_numa_memory_affinity_init() to
>>>>>>       drivers/acpi/numa.c
>>>>>>     arm64, acpi, numa: NUMA support based on SRAT and SLIT
>>>>>>     acpi, numa: Enable ACPI based NUMA on ARM64
>>>>>>
>>>>>> Robert Richter (1):
>>>>>>     acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>>>>>
>>>>>
>>>>> I need ACKs from the ARM64 maintainers on patches [6-7/13] and
>>>>> [13-14/14].
>>>>
>>>>
>>>> There's also a dependency on the arm64 for-next/core branch, so I've been
>>>> largely ignoring this as far as 4.6 is concerned and was planning to take
>>>> a proper look for 4.7 once the upcoming merge window is out of the way.
>>>
>>>
>>> That would be 4.7 and 4.8 respectively I suppose?
>>>
>>> Anyway, Catalin has ACKed all of them except for the [13/14], so
>>> technically I can apply [1-12/14] now and then [13-14/14] can be
>>> applied when they are ready.
>>>
>>> Do you think there will be any problems with merging [6-7/14] into 4.7
>>> via the ACPI tree?
>>>
>>
>> I would defer to the arm64 maintainers for decisions about the arm64
>> specific parts of the patch set.  That said, many of the arm64 specific
>> patches depend on the arm64 for-next/core branch, so you would have to be
>> careful about merge ordering if you pull these in before the for-next/core
>> branch is merged.
>
> Fair enough.  I will wait for an update then.
>
>> Also FWIW, I plan on addressing Catalin's comments about 13/14 and posting a
>> new version of the patch set in the next day or two.
>
> OK, but in that case it won't be considered for 4.7 (at least not by
> me), so I'd suggest sending it in the second half of the 4.7 merge
> window (or about that time).

To be candid, I would very much like for you to pull in as many of the 
patches as you are comfortable with as soon as possible.

I don't know where Will and Catalin stand on this, and their opinion is 
obviously important, but getting 1-12/14 merged to v4.7 and deferring 
the last two for v4.8 would simplify the whole process for me.  The 
drawback is carrying dead code around until the final parts are merged.

David Daney

Re: [PATCH v6 00/14] ACPI NUMA support for ARM64

[Rafael J. Wysocki]

On Wed, May 11, 2016 at 11:30 PM, David Daney <ddaney@caviumnetworks.com> wrote:
> On 05/11/2016 02:22 PM, Rafael J. Wysocki wrote:
>>
>> On Wed, May 11, 2016 at 11:08 PM, David Daney <ddaney@caviumnetworks.com>
>> wrote:
>>>
>>> On 05/11/2016 01:35 PM, Rafael J. Wysocki wrote:
>>>>
>>>>
>>>> On Wed, May 11, 2016 at 12:40 PM, Will Deacon <will.deacon@arm.com>
>>>> wrote:
>>>>>
>>>>>
>>>>> On Wed, May 11, 2016 at 02:43:11AM +0200, Rafael J. Wysocki wrote:
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 27, 2016 at 8:07 PM, David Daney <ddaney.cavm@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> From: David Daney <david.daney@cavium.com>
>>>>>>>
>>>>>>> Based on
>>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git
>>>>>>> for-next/core branch at commit 643d703d2d2d ("arm64: compat: Check
>>>>>>> for
>>>>>>> AArch32 state")
>>>>>
>>>>>
>>>>>
>>>>> [...]
>>>>>
>>>>>>> David Daney (2):
>>>>>>>     arm64, numa: Cleanup NUMA disabled messages.
>>>>>>>     acpi, numa, srat: Improve SRAT error detection and add messages.
>>>>>>>
>>>>>>> Hanjun Guo (11):
>>>>>>>     acpi, numa: Use pr_fmt() instead of printk
>>>>>>>     acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>>>>>>>     acpi, numa: remove duplicate NULL check
>>>>>>>     acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>>>>>>>     arm64, numa: rework numa_add_memblk()
>>>>>>>     x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>>>>>>>     acpi, numa: move bad_srat() and srat_disabled() to
>>>>>>>       drivers/acpi/numa.c
>>>>>>>     acpi, numa: remove unneeded acpi_numa=1
>>>>>>>     acpi, numa: Move acpi_numa_memory_affinity_init() to
>>>>>>>       drivers/acpi/numa.c
>>>>>>>     arm64, acpi, numa: NUMA support based on SRAT and SLIT
>>>>>>>     acpi, numa: Enable ACPI based NUMA on ARM64
>>>>>>>
>>>>>>> Robert Richter (1):
>>>>>>>     acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>>>>>>
>>>>>>
>>>>>>
>>>>>> I need ACKs from the ARM64 maintainers on patches [6-7/13] and
>>>>>> [13-14/14].
>>>>>
>>>>>
>>>>>
>>>>> There's also a dependency on the arm64 for-next/core branch, so I've
>>>>> been
>>>>> largely ignoring this as far as 4.6 is concerned and was planning to
>>>>> take
>>>>> a proper look for 4.7 once the upcoming merge window is out of the way.
>>>>
>>>>
>>>>
>>>> That would be 4.7 and 4.8 respectively I suppose?
>>>>
>>>> Anyway, Catalin has ACKed all of them except for the [13/14], so
>>>> technically I can apply [1-12/14] now and then [13-14/14] can be
>>>> applied when they are ready.
>>>>
>>>> Do you think there will be any problems with merging [6-7/14] into 4.7
>>>> via the ACPI tree?
>>>>
>>>
>>> I would defer to the arm64 maintainers for decisions about the arm64
>>> specific parts of the patch set.  That said, many of the arm64 specific
>>> patches depend on the arm64 for-next/core branch, so you would have to be
>>> careful about merge ordering if you pull these in before the
>>> for-next/core
>>> branch is merged.
>>
>>
>> Fair enough.  I will wait for an update then.
>>
>>> Also FWIW, I plan on addressing Catalin's comments about 13/14 and
>>> posting a
>>> new version of the patch set in the next day or two.
>>
>>
>> OK, but in that case it won't be considered for 4.7 (at least not by
>> me), so I'd suggest sending it in the second half of the 4.7 merge
>> window (or about that time).
>
>
> To be candid, I would very much like for you to pull in as many of the
> patches as you are comfortable with as soon as possible.
>
> I don't know where Will and Catalin stand on this, and their opinion is
> obviously important, but getting 1-12/14 merged to v4.7 and deferring the
> last two for v4.8 would simplify the whole process for me.  The drawback is
> carrying dead code around until the final parts are merged.

That is not unheard of, however.

OK, I'll try to put the [1-12/14] into my linux-next branch early next
week and we'll see if that triggers any conflicts.

Re: [PATCH v6 13/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 05/11/2016 03:39 AM, Catalin Marinas wrote:
> On Wed, Apr 27, 2016 at 11:07:15AM -0700, David Daney wrote:
>> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
>> +{
>> +	unsigned long madt_end, entry;
>> +	struct acpi_table_madt *madt;
>> +	acpi_size tbl_size;
>> +
>> +	if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
>> +			(struct acpi_table_header **)&madt, &tbl_size)))
>> +		return -ENODEV;
>> +
>> +	entry = (unsigned long)madt;
>> +	madt_end = entry + madt->header.length;
>> +
>> +	/* Parse all entries looking for a match. */
>> +	entry += sizeof(struct acpi_table_madt);
>> +	while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
>> +		struct acpi_subtable_header *header =
>> +			(struct acpi_subtable_header *)entry;
>> +
>> +		if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
>> +			struct acpi_madt_generic_interrupt *gicc =
>> +				container_of(header,
>> +				struct acpi_madt_generic_interrupt, header);
>> +
>> +			if ((gicc->flags & ACPI_MADT_ENABLED) &&
>> +			    (gicc->uid == acpi_id)) {
>> +				*mpidr = gicc->arm_mpidr;
>> +				early_acpi_os_unmap_memory(madt, tbl_size);
>> +				return 0;
>> +			}
>> +		}
>> +		entry += header->length;
>> +	}
>> +
>> +	early_acpi_os_unmap_memory(madt, tbl_size);
>> +	return -ENODEV;
>> +}
>> +
>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
>> +{
>> +	int pxm, node;
>> +	u64 mpidr;
>> +
>> +	if (srat_disabled())
>> +		return;
>> +
>> +	if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
>> +		pr_err("SRAT: Invalid SRAT header length: %d\n",
>> +			pa->header.length);
>> +		bad_srat();
>> +		return;
>> +	}
>> +
>> +	if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>> +		return;
>> +
>> +	if (cpus_in_srat >= NR_CPUS) {
>> +		pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
>> +			     NR_CPUS);
>> +		return;
>> +	}
>> +
>> +	pxm = pa->proximity_domain;
>> +	node = acpi_map_pxm_to_node(pxm);
>> +
>> +	if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>> +		pr_err("SRAT: Too many proximity domains %d\n", pxm);
>> +		bad_srat();
>> +		return;
>> +	}
>> +
>> +	if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>> +		pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
>> +			pxm, pa->acpi_processor_uid);
>> +		bad_srat();
>> +		return;
>> +	}
>
> I wonder whether you could replace the get_mpidr_in_madt() function with
> something like acpi_get_phys_id(). It looks like get_mpidr_in_madt()
> duplicates functionality already available elsewhere.
>

I just tried that, and it doesn't work.

The problem is that this code is being run very early in the boot, and 
kmalloc cannot be used.  acpi_get_phys_id() and its ilk can only be used 
once we have working kmalloc.  We need to extract the NUMA information 
early like this precisely because it is needed to initializing the slab 
system

Notice that we are using early_acpi_os_unmap_memory() et al. in 
get_mpidr_in_madt() explicitly for this reason.

In summary: I don't think we need another revision of this patch, it is 
like this for a good reason.

David Daney

Re: [PATCH v6 13/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Hanjun Guo]

On 2016/5/12 8:06, David Daney wrote:
> On 05/11/2016 03:39 AM, Catalin Marinas wrote:
>> On Wed, Apr 27, 2016 at 11:07:15AM -0700, David Daney wrote:
>>> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
>>> +{
>>> +    unsigned long madt_end, entry;
>>> +    struct acpi_table_madt *madt;
>>> +    acpi_size tbl_size;
>>> +
>>> +    if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
>>> +            (struct acpi_table_header **)&madt, &tbl_size)))
>>> +        return -ENODEV;
>>> +
>>> +    entry = (unsigned long)madt;
>>> +    madt_end = entry + madt->header.length;
>>> +
>>> +    /* Parse all entries looking for a match. */
>>> +    entry += sizeof(struct acpi_table_madt);
>>> +    while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
>>> +        struct acpi_subtable_header *header =
>>> +            (struct acpi_subtable_header *)entry;
>>> +
>>> +        if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
>>> +            struct acpi_madt_generic_interrupt *gicc =
>>> +                container_of(header,
>>> +                struct acpi_madt_generic_interrupt, header);
>>> +
>>> +            if ((gicc->flags & ACPI_MADT_ENABLED) &&
>>> +                (gicc->uid == acpi_id)) {
>>> +                *mpidr = gicc->arm_mpidr;
>>> +                early_acpi_os_unmap_memory(madt, tbl_size);
>>> +                return 0;
>>> +            }
>>> +        }
>>> +        entry += header->length;
>>> +    }
>>> +
>>> +    early_acpi_os_unmap_memory(madt, tbl_size);
>>> +    return -ENODEV;
>>> +}
>>> +
>>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>>> +void __init acpi_numa_gicc_affinity_init(struct
>>> acpi_srat_gicc_affinity *pa)
>>> +{
>>> +    int pxm, node;
>>> +    u64 mpidr;
>>> +
>>> +    if (srat_disabled())
>>> +        return;
>>> +
>>> +    if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
>>> +        pr_err("SRAT: Invalid SRAT header length: %d\n",
>>> +            pa->header.length);
>>> +        bad_srat();
>>> +        return;
>>> +    }
>>> +
>>> +    if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>>> +        return;
>>> +
>>> +    if (cpus_in_srat >= NR_CPUS) {
>>> +        pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may
>>> not be able to use all cpus\n",
>>> +                 NR_CPUS);
>>> +        return;
>>> +    }
>>> +
>>> +    pxm = pa->proximity_domain;
>>> +    node = acpi_map_pxm_to_node(pxm);
>>> +
>>> +    if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>>> +        pr_err("SRAT: Too many proximity domains %d\n", pxm);
>>> +        bad_srat();
>>> +        return;
>>> +    }
>>> +
>>> +    if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>>> +        pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in
>>> MADT\n",
>>> +            pxm, pa->acpi_processor_uid);
>>> +        bad_srat();
>>> +        return;
>>> +    }
>>
>> I wonder whether you could replace the get_mpidr_in_madt() function with
>> something like acpi_get_phys_id(). It looks like get_mpidr_in_madt()
>> duplicates functionality already available elsewhere.
>>
>
> I just tried that, and it doesn't work.
>
> The problem is that this code is being run very early in the boot, and
> kmalloc cannot be used.  acpi_get_phys_id() and its ilk can only be used
> once we have working kmalloc.  We need to extract the NUMA information
> early like this precisely because it is needed to initializing the slab
> system
>
> Notice that we are using early_acpi_os_unmap_memory() et al. in
> get_mpidr_in_madt() explicitly for this reason.

I got the same conclusion when I was preparing this patch set.

Thanks
Hanjun

Re: [PATCH v6 00/14] ACPI NUMA support for ARM64

[Will Deacon]

On Thu, May 12, 2016 at 12:29:02AM +0200, Rafael J. Wysocki wrote:
> On Wed, May 11, 2016 at 11:30 PM, David Daney <ddaney@caviumnetworks.com> wrote:
> > On 05/11/2016 02:22 PM, Rafael J. Wysocki wrote:
> >> On Wed, May 11, 2016 at 11:08 PM, David Daney <ddaney@caviumnetworks.com>
> >> wrote:
> >>> On 05/11/2016 01:35 PM, Rafael J. Wysocki wrote:
> >>>> On Wed, May 11, 2016 at 12:40 PM, Will Deacon <will.deacon@arm.com>
> >>>> wrote:
> >>>>> On Wed, May 11, 2016 at 02:43:11AM +0200, Rafael J. Wysocki wrote:
> >>>>> There's also a dependency on the arm64 for-next/core branch, so I've
> >>>>> been
> >>>>> largely ignoring this as far as 4.6 is concerned and was planning to
> >>>>> take
> >>>>> a proper look for 4.7 once the upcoming merge window is out of the way.
> >>>>
> >>>>
> >>>>
> >>>> That would be 4.7 and 4.8 respectively I suppose?

Argh, yes, of course! :)

> >>>>
> >>>> Anyway, Catalin has ACKed all of them except for the [13/14], so
> >>>> technically I can apply [1-12/14] now and then [13-14/14] can be
> >>>> applied when they are ready.
> >>>>
> >>>> Do you think there will be any problems with merging [6-7/14] into 4.7
> >>>> via the ACPI tree?
> >>>>
> >>>
> >>> I would defer to the arm64 maintainers for decisions about the arm64
> >>> specific parts of the patch set.  That said, many of the arm64 specific
> >>> patches depend on the arm64 for-next/core branch, so you would have to be
> >>> careful about merge ordering if you pull these in before the
> >>> for-next/core
> >>> branch is merged.
> >>
> >>
> >> Fair enough.  I will wait for an update then.
> >>
> >>> Also FWIW, I plan on addressing Catalin's comments about 13/14 and
> >>> posting a
> >>> new version of the patch set in the next day or two.
> >>
> >>
> >> OK, but in that case it won't be considered for 4.7 (at least not by
> >> me), so I'd suggest sending it in the second half of the 4.7 merge
> >> window (or about that time).
> >
> >
> > To be candid, I would very much like for you to pull in as many of the
> > patches as you are comfortable with as soon as possible.
> >
> > I don't know where Will and Catalin stand on this, and their opinion is
> > obviously important, but getting 1-12/14 merged to v4.7 and deferring the
> > last two for v4.8 would simplify the whole process for me.  The drawback is
> > carrying dead code around until the final parts are merged.
> 
> That is not unheard of, however.
> 
> OK, I'll try to put the [1-12/14] into my linux-next branch early next
> week and we'll see if that triggers any conflicts.

I'd really much rather this waited until after the merge window. My
understanding is that it's bad practice to put stuff into -next during the
merge window, and you'd end up having to send a pull based on a random
commit (the arm64 pull request?) in the second half. On top of that, this
series would get very little exposure in -next during that time.

On the other hand, putting this into linux-next after the merge window
gives us time for testing, allows David to rework patch 13 (which is aiming
for 4.8 anyway iiuc) and avoids merge window churn.

Will

Re: [PATCH v6 13/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Catalin Marinas]

On Wed, May 11, 2016 at 05:06:13PM -0700, David Daney wrote:
> On 05/11/2016 03:39 AM, Catalin Marinas wrote:
> >On Wed, Apr 27, 2016 at 11:07:15AM -0700, David Daney wrote:
> >>+static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
> >>+{
> >>+	unsigned long madt_end, entry;
> >>+	struct acpi_table_madt *madt;
> >>+	acpi_size tbl_size;
> >>+
> >>+	if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
> >>+			(struct acpi_table_header **)&madt, &tbl_size)))
> >>+		return -ENODEV;
> >>+
> >>+	entry = (unsigned long)madt;
> >>+	madt_end = entry + madt->header.length;
> >>+
> >>+	/* Parse all entries looking for a match. */
> >>+	entry += sizeof(struct acpi_table_madt);
> >>+	while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
> >>+		struct acpi_subtable_header *header =
> >>+			(struct acpi_subtable_header *)entry;
> >>+
> >>+		if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
> >>+			struct acpi_madt_generic_interrupt *gicc =
> >>+				container_of(header,
> >>+				struct acpi_madt_generic_interrupt, header);
> >>+
> >>+			if ((gicc->flags & ACPI_MADT_ENABLED) &&
> >>+			    (gicc->uid == acpi_id)) {
> >>+				*mpidr = gicc->arm_mpidr;
> >>+				early_acpi_os_unmap_memory(madt, tbl_size);
> >>+				return 0;
> >>+			}
> >>+		}
> >>+		entry += header->length;
> >>+	}
> >>+
> >>+	early_acpi_os_unmap_memory(madt, tbl_size);
> >>+	return -ENODEV;
> >>+}
> >>+
> >>+/* Callback for Proximity Domain -> ACPI processor UID mapping */
> >>+void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
> >>+{
> >>+	int pxm, node;
> >>+	u64 mpidr;
> >>+
> >>+	if (srat_disabled())
> >>+		return;
> >>+
> >>+	if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
> >>+		pr_err("SRAT: Invalid SRAT header length: %d\n",
> >>+			pa->header.length);
> >>+		bad_srat();
> >>+		return;
> >>+	}
> >>+
> >>+	if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
> >>+		return;
> >>+
> >>+	if (cpus_in_srat >= NR_CPUS) {
> >>+		pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
> >>+			     NR_CPUS);
> >>+		return;
> >>+	}
> >>+
> >>+	pxm = pa->proximity_domain;
> >>+	node = acpi_map_pxm_to_node(pxm);
> >>+
> >>+	if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
> >>+		pr_err("SRAT: Too many proximity domains %d\n", pxm);
> >>+		bad_srat();
> >>+		return;
> >>+	}
> >>+
> >>+	if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
> >>+		pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
> >>+			pxm, pa->acpi_processor_uid);
> >>+		bad_srat();
> >>+		return;
> >>+	}
> >
> >I wonder whether you could replace the get_mpidr_in_madt() function with
> >something like acpi_get_phys_id(). It looks like get_mpidr_in_madt()
> >duplicates functionality already available elsewhere.
> 
> I just tried that, and it doesn't work.
> 
> The problem is that this code is being run very early in the boot, and
> kmalloc cannot be used.  acpi_get_phys_id() and its ilk can only be used
> once we have working kmalloc.  We need to extract the NUMA information early
> like this precisely because it is needed to initializing the slab system
> 
> Notice that we are using early_acpi_os_unmap_memory() et al. in
> get_mpidr_in_madt() explicitly for this reason.
> 
> In summary: I don't think we need another revision of this patch, it is like
> this for a good reason.

Slightly confusing, in another reply you said you are going to address
my comment. So, is it doable?

-- 
Catalin

Re: [PATCH v6 00/14] ACPI NUMA support for ARM64

[Rafael J. Wysocki]

On Thu, May 12, 2016 at 10:56 AM, Will Deacon <will.deacon@arm.com> wrote:
> On Thu, May 12, 2016 at 12:29:02AM +0200, Rafael J. Wysocki wrote:
>> On Wed, May 11, 2016 at 11:30 PM, David Daney <ddaney@caviumnetworks.com> wrote:
>> > On 05/11/2016 02:22 PM, Rafael J. Wysocki wrote:
>> >> On Wed, May 11, 2016 at 11:08 PM, David Daney <ddaney@caviumnetworks.com>
>> >> wrote:
>> >>> On 05/11/2016 01:35 PM, Rafael J. Wysocki wrote:
>> >>>> On Wed, May 11, 2016 at 12:40 PM, Will Deacon <will.deacon@arm.com>
>> >>>> wrote:
>> >>>>> On Wed, May 11, 2016 at 02:43:11AM +0200, Rafael J. Wysocki wrote:
>> >>>>> There's also a dependency on the arm64 for-next/core branch, so I've
>> >>>>> been
>> >>>>> largely ignoring this as far as 4.6 is concerned and was planning to
>> >>>>> take
>> >>>>> a proper look for 4.7 once the upcoming merge window is out of the way.
>> >>>>
>> >>>>
>> >>>>
>> >>>> That would be 4.7 and 4.8 respectively I suppose?
>
> Argh, yes, of course! :)
>
>> >>>>
>> >>>> Anyway, Catalin has ACKed all of them except for the [13/14], so
>> >>>> technically I can apply [1-12/14] now and then [13-14/14] can be
>> >>>> applied when they are ready.
>> >>>>
>> >>>> Do you think there will be any problems with merging [6-7/14] into 4.7
>> >>>> via the ACPI tree?
>> >>>>
>> >>>
>> >>> I would defer to the arm64 maintainers for decisions about the arm64
>> >>> specific parts of the patch set.  That said, many of the arm64 specific
>> >>> patches depend on the arm64 for-next/core branch, so you would have to be
>> >>> careful about merge ordering if you pull these in before the
>> >>> for-next/core
>> >>> branch is merged.
>> >>
>> >>
>> >> Fair enough.  I will wait for an update then.
>> >>
>> >>> Also FWIW, I plan on addressing Catalin's comments about 13/14 and
>> >>> posting a
>> >>> new version of the patch set in the next day or two.
>> >>
>> >>
>> >> OK, but in that case it won't be considered for 4.7 (at least not by
>> >> me), so I'd suggest sending it in the second half of the 4.7 merge
>> >> window (or about that time).
>> >
>> >
>> > To be candid, I would very much like for you to pull in as many of the
>> > patches as you are comfortable with as soon as possible.
>> >
>> > I don't know where Will and Catalin stand on this, and their opinion is
>> > obviously important, but getting 1-12/14 merged to v4.7 and deferring the
>> > last two for v4.8 would simplify the whole process for me.  The drawback is
>> > carrying dead code around until the final parts are merged.
>>
>> That is not unheard of, however.
>>
>> OK, I'll try to put the [1-12/14] into my linux-next branch early next
>> week and we'll see if that triggers any conflicts.
>
> I'd really much rather this waited until after the merge window.

That's fine by me, so I guess my previous request for the next version
to be sent by the end of the merge window applies. :-)

> My understanding is that it's bad practice to put stuff into -next during the
> merge window,

Unless you want to push that stuff to Linus during the merge window in
progress, that is.

> and you'd end up having to send a pull based on a random
> commit (the arm64 pull request?) in the second half. On top of that, this
> series would get very little exposure in -next during that time.
>
> On the other hand, putting this into linux-next after the merge window
> gives us time for testing, allows David to rework patch 13 (which is aiming
> for 4.8 anyway iiuc) and avoids merge window churn.

OK

Re: [PATCH v6 13/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 05/12/2016 02:49 AM, Catalin Marinas wrote:
> On Wed, May 11, 2016 at 05:06:13PM -0700, David Daney wrote:
>> On 05/11/2016 03:39 AM, Catalin Marinas wrote:
[...]
>>>
>>> I wonder whether you could replace the get_mpidr_in_madt() function with
>>> something like acpi_get_phys_id(). It looks like get_mpidr_in_madt()
>>> duplicates functionality already available elsewhere.
>>
>> I just tried that, and it doesn't work.
>>
>> The problem is that this code is being run very early in the boot, and
>> kmalloc cannot be used.  acpi_get_phys_id() and its ilk can only be used
>> once we have working kmalloc.  We need to extract the NUMA information early
>> like this precisely because it is needed to initializing the slab system
>>
>> Notice that we are using early_acpi_os_unmap_memory() et al. in
>> get_mpidr_in_madt() explicitly for this reason.
>>
>> In summary: I don't think we need another revision of this patch, it is like
>> this for a good reason.
>
> Slightly confusing, in another reply you said you are going to address
> my comment. So, is it doable?

I don't think so.

My previous reply, to the thread in 0/14, was prematurely made with the 
incorrect assumption that it was a simple change.  Now, after really 
digging in to the code, and attempting to do as you suggested, I have 
changed my mind.

As I indicated above, there are no remaining suggestions or requests for 
changes to this patch set pending action.  Unless people find additional 
problems, I think it is ready for merging.

I know that Will and Rafael were discussing the proper timing for 
merging this in the other thread, so I leave it in their hands at this 
point.

Thanks,
David Daney


>

Re: [PATCH v6 13/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Catalin Marinas]

On Thu, May 12, 2016 at 08:27:08AM -0700, David Daney wrote:
> On 05/12/2016 02:49 AM, Catalin Marinas wrote:
> >On Wed, May 11, 2016 at 05:06:13PM -0700, David Daney wrote:
> >>On 05/11/2016 03:39 AM, Catalin Marinas wrote:
> [...]
> >>>
> >>>I wonder whether you could replace the get_mpidr_in_madt() function with
> >>>something like acpi_get_phys_id(). It looks like get_mpidr_in_madt()
> >>>duplicates functionality already available elsewhere.
> >>
> >>I just tried that, and it doesn't work.
> >>
> >>The problem is that this code is being run very early in the boot, and
> >>kmalloc cannot be used.  acpi_get_phys_id() and its ilk can only be used
> >>once we have working kmalloc.  We need to extract the NUMA information early
> >>like this precisely because it is needed to initializing the slab system
> >>
> >>Notice that we are using early_acpi_os_unmap_memory() et al. in
> >>get_mpidr_in_madt() explicitly for this reason.
> >>
> >>In summary: I don't think we need another revision of this patch, it is like
> >>this for a good reason.
> >
> >Slightly confusing, in another reply you said you are going to address
> >my comment. So, is it doable?
> 
> I don't think so.
> 
> My previous reply, to the thread in 0/14, was prematurely made with the
> incorrect assumption that it was a simple change.  Now, after really digging
> in to the code, and attempting to do as you suggested, I have changed my
> mind.

Would the snippet below help with avoiding any kmalloc calls? At a quick
look, it seems that it's only map_mat_entry() that ends up using
kmalloc() calls. Alternatively, exporting map_madt_entry() may work as
well.

diff --git a/drivers/acpi/processor_core.c b/drivers/acpi/processor_core.c
index 33a38d604630..77af0a7df914 100644
--- a/drivers/acpi/processor_core.c
+++ b/drivers/acpi/processor_core.c
@@ -152,6 +152,9 @@ static phys_cpuid_t map_mat_entry(acpi_handle handle, int type, u32 acpi_id)
 	struct acpi_subtable_header *header;
 	phys_cpuid_t phys_id = PHYS_CPUID_INVALID;
 
+	if (!acpi_gbl_permanent_mmap)
+		return phys_id;
+
 	if (ACPI_FAILURE(acpi_evaluate_object(handle, "_MAT", NULL, &buffer)))
 		goto exit;
 

-- 
Catalin

Re: [PATCH v6 13/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 05/12/2016 09:24 AM, Catalin Marinas wrote:
> On Thu, May 12, 2016 at 08:27:08AM -0700, David Daney wrote:
>> On 05/12/2016 02:49 AM, Catalin Marinas wrote:
>>> On Wed, May 11, 2016 at 05:06:13PM -0700, David Daney wrote:
>>>> On 05/11/2016 03:39 AM, Catalin Marinas wrote:
>> [...]
>>>>>
>>>>> I wonder whether you could replace the get_mpidr_in_madt() function with
>>>>> something like acpi_get_phys_id(). It looks like get_mpidr_in_madt()
>>>>> duplicates functionality already available elsewhere.
>>>>
>>>> I just tried that, and it doesn't work.
>>>>
>>>> The problem is that this code is being run very early in the boot, and
>>>> kmalloc cannot be used.  acpi_get_phys_id() and its ilk can only be used
>>>> once we have working kmalloc.  We need to extract the NUMA information early
>>>> like this precisely because it is needed to initializing the slab system
>>>>
>>>> Notice that we are using early_acpi_os_unmap_memory() et al. in
>>>> get_mpidr_in_madt() explicitly for this reason.
>>>>
>>>> In summary: I don't think we need another revision of this patch, it is like
>>>> this for a good reason.
>>>
>>> Slightly confusing, in another reply you said you are going to address
>>> my comment. So, is it doable?
>>
>> I don't think so.
>>
>> My previous reply, to the thread in 0/14, was prematurely made with the
>> incorrect assumption that it was a simple change.  Now, after really digging
>> in to the code, and attempting to do as you suggested, I have changed my
>> mind.
>
> Would the snippet below help with avoiding any kmalloc calls? At a quick
> look, it seems that it's only map_mat_entry() that ends up using
> kmalloc() calls. Alternatively, exporting map_madt_entry() may work as
> well.

I think making a non-static version of map_madt_entry() is the cleanest 
solution.  That works, so I will send a new version of the patch set 
that does that instead.

David.


>
> diff --git a/drivers/acpi/processor_core.c b/drivers/acpi/processor_core.c
> index 33a38d604630..77af0a7df914 100644
> --- a/drivers/acpi/processor_core.c
> +++ b/drivers/acpi/processor_core.c
> @@ -152,6 +152,9 @@ static phys_cpuid_t map_mat_entry(acpi_handle handle, int type, u32 acpi_id)
>   	struct acpi_subtable_header *header;
>   	phys_cpuid_t phys_id = PHYS_CPUID_INVALID;
>
> +	if (!acpi_gbl_permanent_mmap)
> +		return phys_id;
> +
>   	if (ACPI_FAILURE(acpi_evaluate_object(handle, "_MAT", NULL, &buffer)))
>   		goto exit;
>
>