[PATCH v5 00/14] ACPI NUMA support for ARM64

[PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

From: David Daney <david.daney@cavium.com>

Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
set introduce the ACPI based configuration to provide NUMA
information.

ACPI 5.1 already introduced NUMA support for ARM64, which can get the
NUMA domain information from SRAT and SLIT table, so parse those two
tables to get mappings from cpu/mem to numa node configuration and
system locality.

v5 updates:

 - Fixed ia64 build failure by gating some unused functions with #if
   CONFIG_{X86,ARM64}.

 - Fixed section mismatch errors for X86 case.

 - Removed unneeded #include from some files.

 - Tested to build cleanly on ARM64, X86_64, IA64

v4 updates:

 - Updated from Hanjun Guo's v3 patches.

 - Rebased on top of v16 of device-tree NUMA patches.

 - Reordered some of the changes so that we don't introduce code and
   then change it several times in the patch set.  New code is
   introduced in its final form.  Code reused from x86 is first moved
   with no change, and then a separate patch to make any needed
   changes.

 - code that is used only by ia64, moved to architecture specific
   files.

v3 updates:
 - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
   that for acpi/numa.c (patch 2/12)

 - Remove the duplicate NULL check for table print (patch 3/12)

 - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
   dummy function for acpi_numa_arch_fixup()

 - Solve the problem that the mapping from logical cpu to numa node
   is wrong which spotted out by Lorenzo

 - cleanups for x86 and move acpi_numa_slit_init() and some other
   functions to common place, then reduce the duplicate of x86
   and arm64 (patch 7-12/12).

 - rebased on top of 4.4 and Ganapat's v9 patch set.

[1]: https://lkml.org/lkml/2016/4/8/571

David Daney (1):
  acpi, numa, srat: Improve SRAT error detection and add messages.

Hanjun Guo (11):
  acpi, numa: Use pr_fmt() instead of printk
  acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
  acpi, numa: remove duplicate NULL check
  acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
  arm64, numa: rework numa_add_memblk()
  x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
  acpi, numa: move bad_srat() and srat_disabled() to
    drivers/acpi/numa.c
  acpi, numa: remove unneeded acpi_numa=1
  acpi, numa: Move acpi_numa_memory_affinity_init() to
    drivers/acpi/numa.c
  arm64, acpi, numa: NUMA support based on SRAT and SLIT
  acpi, numa: Enable ACPI based NUMA on ARM64

Robert Richter (2):
  acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
  arm64, acpi, numa: Default enable ACPI_NUMA with NUMA

 arch/arm64/include/asm/acpi.h |   8 ++
 arch/arm64/include/asm/numa.h |   2 +
 arch/arm64/kernel/Makefile    |   1 +
 arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
 arch/arm64/kernel/smp.c       |   2 +
 arch/arm64/mm/numa.c          |  17 ++--
 arch/ia64/include/asm/acpi.h  |   3 +
 arch/ia64/kernel/acpi.c       |   2 +-
 arch/ia64/kernel/setup.c      |   1 +
 arch/x86/include/asm/acpi.h   |   1 -
 arch/x86/mm/numa.c            |   2 +-
 arch/x86/mm/srat.c            | 116 +---------------------
 drivers/acpi/Kconfig          |   4 +-
 drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
 drivers/of/of_numa.c          |   4 +-
 include/acpi/acpi_numa.h      |   4 +
 include/linux/acpi.h          |  18 +++-
 17 files changed, 375 insertions(+), 183 deletions(-)
 create mode 100644 arch/arm64/kernel/acpi_numa.c

-- 
1.7.11.7

[PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

From: David Daney <david.daney@cavium.com>

Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
set introduce the ACPI based configuration to provide NUMA
information.

ACPI 5.1 already introduced NUMA support for ARM64, which can get the
NUMA domain information from SRAT and SLIT table, so parse those two
tables to get mappings from cpu/mem to numa node configuration and
system locality.

v5 updates:

 - Fixed ia64 build failure by gating some unused functions with #if
   CONFIG_{X86,ARM64}.

 - Fixed section mismatch errors for X86 case.

 - Removed unneeded #include from some files.

 - Tested to build cleanly on ARM64, X86_64, IA64

v4 updates:

 - Updated from Hanjun Guo's v3 patches.

 - Rebased on top of v16 of device-tree NUMA patches.

 - Reordered some of the changes so that we don't introduce code and
   then change it several times in the patch set.  New code is
   introduced in its final form.  Code reused from x86 is first moved
   with no change, and then a separate patch to make any needed
   changes.

 - code that is used only by ia64, moved to architecture specific
   files.

v3 updates:
 - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
   that for acpi/numa.c (patch 2/12)

 - Remove the duplicate NULL check for table print (patch 3/12)

 - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
   dummy function for acpi_numa_arch_fixup()

 - Solve the problem that the mapping from logical cpu to numa node
   is wrong which spotted out by Lorenzo

 - cleanups for x86 and move acpi_numa_slit_init() and some other
   functions to common place, then reduce the duplicate of x86
   and arm64 (patch 7-12/12).

 - rebased on top of 4.4 and Ganapat's v9 patch set.

[1]: https://lkml.org/lkml/2016/4/8/571

David Daney (1):
  acpi, numa, srat: Improve SRAT error detection and add messages.

Hanjun Guo (11):
  acpi, numa: Use pr_fmt() instead of printk
  acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
  acpi, numa: remove duplicate NULL check
  acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
  arm64, numa: rework numa_add_memblk()
  x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
  acpi, numa: move bad_srat() and srat_disabled() to
    drivers/acpi/numa.c
  acpi, numa: remove unneeded acpi_numa=1
  acpi, numa: Move acpi_numa_memory_affinity_init() to
    drivers/acpi/numa.c
  arm64, acpi, numa: NUMA support based on SRAT and SLIT
  acpi, numa: Enable ACPI based NUMA on ARM64

Robert Richter (2):
  acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
  arm64, acpi, numa: Default enable ACPI_NUMA with NUMA

 arch/arm64/include/asm/acpi.h |   8 ++
 arch/arm64/include/asm/numa.h |   2 +
 arch/arm64/kernel/Makefile    |   1 +
 arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
 arch/arm64/kernel/smp.c       |   2 +
 arch/arm64/mm/numa.c          |  17 ++--
 arch/ia64/include/asm/acpi.h  |   3 +
 arch/ia64/kernel/acpi.c       |   2 +-
 arch/ia64/kernel/setup.c      |   1 +
 arch/x86/include/asm/acpi.h   |   1 -
 arch/x86/mm/numa.c            |   2 +-
 arch/x86/mm/srat.c            | 116 +---------------------
 drivers/acpi/Kconfig          |   4 +-
 drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
 drivers/of/of_numa.c          |   4 +-
 include/acpi/acpi_numa.h      |   4 +
 include/linux/acpi.h          |  18 +++-
 17 files changed, 375 insertions(+), 183 deletions(-)
 create mode 100644 arch/arm64/kernel/acpi_numa.c

-- 
1.7.11.7

[PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

From: David Daney <david.daney@cavium.com>

Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
set introduce the ACPI based configuration to provide NUMA
information.

ACPI 5.1 already introduced NUMA support for ARM64, which can get the
NUMA domain information from SRAT and SLIT table, so parse those two
tables to get mappings from cpu/mem to numa node configuration and
system locality.

v5 updates:

 - Fixed ia64 build failure by gating some unused functions with #if
   CONFIG_{X86,ARM64}.

 - Fixed section mismatch errors for X86 case.

 - Removed unneeded #include from some files.

 - Tested to build cleanly on ARM64, X86_64, IA64

v4 updates:

 - Updated from Hanjun Guo's v3 patches.

 - Rebased on top of v16 of device-tree NUMA patches.

 - Reordered some of the changes so that we don't introduce code and
   then change it several times in the patch set.  New code is
   introduced in its final form.  Code reused from x86 is first moved
   with no change, and then a separate patch to make any needed
   changes.

 - code that is used only by ia64, moved to architecture specific
   files.

v3 updates:
 - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
   that for acpi/numa.c (patch 2/12)

 - Remove the duplicate NULL check for table print (patch 3/12)

 - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
   dummy function for acpi_numa_arch_fixup()

 - Solve the problem that the mapping from logical cpu to numa node
   is wrong which spotted out by Lorenzo

 - cleanups for x86 and move acpi_numa_slit_init() and some other
   functions to common place, then reduce the duplicate of x86
   and arm64 (patch 7-12/12).

 - rebased on top of 4.4 and Ganapat's v9 patch set.

[1]: https://lkml.org/lkml/2016/4/8/571

David Daney (1):
  acpi, numa, srat: Improve SRAT error detection and add messages.

Hanjun Guo (11):
  acpi, numa: Use pr_fmt() instead of printk
  acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
  acpi, numa: remove duplicate NULL check
  acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
  arm64, numa: rework numa_add_memblk()
  x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
  acpi, numa: move bad_srat() and srat_disabled() to
    drivers/acpi/numa.c
  acpi, numa: remove unneeded acpi_numa=1
  acpi, numa: Move acpi_numa_memory_affinity_init() to
    drivers/acpi/numa.c
  arm64, acpi, numa: NUMA support based on SRAT and SLIT
  acpi, numa: Enable ACPI based NUMA on ARM64

Robert Richter (2):
  acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
  arm64, acpi, numa: Default enable ACPI_NUMA with NUMA

 arch/arm64/include/asm/acpi.h |   8 ++
 arch/arm64/include/asm/numa.h |   2 +
 arch/arm64/kernel/Makefile    |   1 +
 arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
 arch/arm64/kernel/smp.c       |   2 +
 arch/arm64/mm/numa.c          |  17 ++--
 arch/ia64/include/asm/acpi.h  |   3 +
 arch/ia64/kernel/acpi.c       |   2 +-
 arch/ia64/kernel/setup.c      |   1 +
 arch/x86/include/asm/acpi.h   |   1 -
 arch/x86/mm/numa.c            |   2 +-
 arch/x86/mm/srat.c            | 116 +---------------------
 drivers/acpi/Kconfig          |   4 +-
 drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
 drivers/of/of_numa.c          |   4 +-
 include/acpi/acpi_numa.h      |   4 +
 include/linux/acpi.h          |  18 +++-
 17 files changed, 375 insertions(+), 183 deletions(-)
 create mode 100644 arch/arm64/kernel/acpi_numa.c

-- 
1.7.11.7

[PATCH v5 01/14] acpi, numa: Use pr_fmt() instead of printk

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Just do some cleanups to replace printk with pr_fmt().

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 72b6e9e..4e427fc 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -18,6 +18,9 @@
  * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  *
  */
+
+#define pr_fmt(fmt) "ACPI: " fmt
+
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/kernel.h>
@@ -28,8 +31,6 @@
 #include <linux/nodemask.h>
 #include <linux/topology.h>
 
-#define PREFIX "ACPI: "
-
 #define ACPI_NUMA	0x80000000
 #define _COMPONENT	ACPI_NUMA
 ACPI_MODULE_NAME("numa");
@@ -187,9 +188,8 @@ acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 #endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 	default:
-		printk(KERN_WARNING PREFIX
-		       "Found unsupported SRAT entry (type = 0x%x)\n",
-		       header->type);
+		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
+			header->type);
 		break;
 	}
 }
@@ -222,7 +222,7 @@ static int __init acpi_parse_slit(struct acpi_table_header *table)
 	struct acpi_table_slit *slit = (struct acpi_table_slit *)table;
 
 	if (!slit_valid(slit)) {
-		printk(KERN_INFO "ACPI: SLIT table looks invalid. Not used.\n");
+		pr_info("SLIT table looks invalid. Not used.\n");
 		return -EINVAL;
 	}
 	acpi_numa_slit_init(slit);
@@ -233,12 +233,9 @@ static int __init acpi_parse_slit(struct acpi_table_header *table)
 void __init __weak
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 {
-	printk(KERN_WARNING PREFIX
-	       "Found unsupported x2apic [0x%08x] SRAT entry\n", pa->apic_id);
-	return;
+	pr_warn("Found unsupported x2apic [0x%08x] SRAT entry\n", pa->apic_id);
 }
 
-
 static int __init
 acpi_parse_x2apic_affinity(struct acpi_subtable_header *header,
 			   const unsigned long end)
-- 
1.7.11.7

[PATCH v5 01/14] acpi, numa: Use pr_fmt() instead of printk

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Just do some cleanups to replace printk with pr_fmt().

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 72b6e9e..4e427fc 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -18,6 +18,9 @@
  * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  *
  */
+
+#define pr_fmt(fmt) "ACPI: " fmt
+
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/kernel.h>
@@ -28,8 +31,6 @@
 #include <linux/nodemask.h>
 #include <linux/topology.h>
 
-#define PREFIX "ACPI: "
-
 #define ACPI_NUMA	0x80000000
 #define _COMPONENT	ACPI_NUMA
 ACPI_MODULE_NAME("numa");
@@ -187,9 +188,8 @@ acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 #endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 	default:
-		printk(KERN_WARNING PREFIX
-		       "Found unsupported SRAT entry (type = 0x%x)\n",
-		       header->type);
+		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
+			header->type);
 		break;
 	}
 }
@@ -222,7 +222,7 @@ static int __init acpi_parse_slit(struct acpi_table_header *table)
 	struct acpi_table_slit *slit = (struct acpi_table_slit *)table;
 
 	if (!slit_valid(slit)) {
-		printk(KERN_INFO "ACPI: SLIT table looks invalid. Not used.\n");
+		pr_info("SLIT table looks invalid. Not used.\n");
 		return -EINVAL;
 	}
 	acpi_numa_slit_init(slit);
@@ -233,12 +233,9 @@ static int __init acpi_parse_slit(struct acpi_table_header *table)
 void __init __weak
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 {
-	printk(KERN_WARNING PREFIX
-	       "Found unsupported x2apic [0x%08x] SRAT entry\n", pa->apic_id);
-	return;
+	pr_warn("Found unsupported x2apic [0x%08x] SRAT entry\n", pa->apic_id);
 }
 
-
 static int __init
 acpi_parse_x2apic_affinity(struct acpi_subtable_header *header,
 			   const unsigned long end)
-- 
1.7.11.7

[PATCH v5 01/14] acpi, numa: Use pr_fmt() instead of printk

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Just do some cleanups to replace printk with pr_fmt().

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 72b6e9e..4e427fc 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -18,6 +18,9 @@
  * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  *
  */
+
+#define pr_fmt(fmt) "ACPI: " fmt
+
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/kernel.h>
@@ -28,8 +31,6 @@
 #include <linux/nodemask.h>
 #include <linux/topology.h>
 
-#define PREFIX "ACPI: "
-
 #define ACPI_NUMA	0x80000000
 #define _COMPONENT	ACPI_NUMA
 ACPI_MODULE_NAME("numa");
@@ -187,9 +188,8 @@ acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 #endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 	default:
-		printk(KERN_WARNING PREFIX
-		       "Found unsupported SRAT entry (type = 0x%x)\n",
-		       header->type);
+		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
+			header->type);
 		break;
 	}
 }
@@ -222,7 +222,7 @@ static int __init acpi_parse_slit(struct acpi_table_header *table)
 	struct acpi_table_slit *slit = (struct acpi_table_slit *)table;
 
 	if (!slit_valid(slit)) {
-		printk(KERN_INFO "ACPI: SLIT table looks invalid. Not used.\n");
+		pr_info("SLIT table looks invalid. Not used.\n");
 		return -EINVAL;
 	}
 	acpi_numa_slit_init(slit);
@@ -233,12 +233,9 @@ static int __init acpi_parse_slit(struct acpi_table_header *table)
 void __init __weak
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 {
-	printk(KERN_WARNING PREFIX
-	       "Found unsupported x2apic [0x%08x] SRAT entry\n", pa->apic_id);
-	return;
+	pr_warn("Found unsupported x2apic [0x%08x] SRAT entry\n", pa->apic_id);
 }
 
-
 static int __init
 acpi_parse_x2apic_affinity(struct acpi_subtable_header *header,
 			   const unsigned long end)
-- 
1.7.11.7

[PATCH v5 02/14] acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

ACPI_DEBUG_PRINT is a bit fragile in acpi/numa.c, the first thing
is that component ACPI_NUMA(0x80000000) is not described in the
Documentation/acpi/debug.txt, and even not defined in the struct
acpi_dlayer acpi_debug_layers which we can not dynamically enable/disable
it with /sys/modules/acpi/parameters/debug_layer. another thing
is that ACPI_DEBUG_OUTPUT is controlled by ACPICA which not coordinate
well with ACPI drivers.

Replace ACPI_DEBUG_PRINT() with pr_debug() in this patch as pr_debug
will do the same thing for debug purpose and it can make the code much
cleaner, also remove the related code which not needed anymore if
ACPI_DEBUG_PRINT() is gone.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 59 +++++++++++++++++++----------------------------------
 1 file changed, 21 insertions(+), 38 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 4e427fc..e34b5d0 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -31,10 +31,6 @@
 #include <linux/nodemask.h>
 #include <linux/topology.h>
 
-#define ACPI_NUMA	0x80000000
-#define _COMPONENT	ACPI_NUMA
-ACPI_MODULE_NAME("numa");
-
 static nodemask_t nodes_found_map = NODE_MASK_NONE;
 
 /* maps to convert between proximity domain and logical node ID */
@@ -129,64 +125,51 @@ EXPORT_SYMBOL(acpi_map_pxm_to_online_node);
 static void __init
 acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 {
-
-	ACPI_FUNCTION_NAME("acpi_table_print_srat_entry");
-
 	if (!header)
 		return;
 
 	switch (header->type) {
-
 	case ACPI_SRAT_TYPE_CPU_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_cpu_affinity *p =
 			    (struct acpi_srat_cpu_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Processor (id[0x%02x] eid[0x%02x]) in proximity domain %d %s\n",
-					  p->apic_id, p->local_sapic_eid,
-					  p->proximity_domain_lo,
-					  (p->flags & ACPI_SRAT_CPU_ENABLED)?
-					  "enabled" : "disabled"));
+			pr_debug("SRAT Processor (id[0x%02x] eid[0x%02x]) in proximity domain %d %s\n",
+				 p->apic_id, p->local_sapic_eid,
+				 p->proximity_domain_lo,
+				 (p->flags & ACPI_SRAT_CPU_ENABLED) ?
+				 "enabled" : "disabled");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 
 	case ACPI_SRAT_TYPE_MEMORY_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_mem_affinity *p =
 			    (struct acpi_srat_mem_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Memory (0x%lx length 0x%lx) in proximity domain %d %s%s%s\n",
-					  (unsigned long)p->base_address,
-					  (unsigned long)p->length,
-					  p->proximity_domain,
-					  (p->flags & ACPI_SRAT_MEM_ENABLED)?
-					  "enabled" : "disabled",
-					  (p->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE)?
-					  " hot-pluggable" : "",
-					  (p->flags & ACPI_SRAT_MEM_NON_VOLATILE)?
-					  " non-volatile" : ""));
+			pr_debug("SRAT Memory (0x%lx length 0x%lx) in proximity domain %d %s%s%s\n",
+				 (unsigned long)p->base_address,
+				 (unsigned long)p->length,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_MEM_ENABLED) ?
+				 "enabled" : "disabled",
+				 (p->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) ?
+				 " hot-pluggable" : "",
+				 (p->flags & ACPI_SRAT_MEM_NON_VOLATILE) ?
+				 " non-volatile" : "");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 
 	case ACPI_SRAT_TYPE_X2APIC_CPU_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_x2apic_cpu_affinity *p =
 			    (struct acpi_srat_x2apic_cpu_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Processor (x2apicid[0x%08x]) in"
-					  " proximity domain %d %s\n",
-					  p->apic_id,
-					  p->proximity_domain,
-					  (p->flags & ACPI_SRAT_CPU_ENABLED) ?
-					  "enabled" : "disabled"));
+			pr_debug("SRAT Processor (x2apicid[0x%08x]) in proximity domain %d %s\n",
+				 p->apic_id,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_CPU_ENABLED) ?
+				 "enabled" : "disabled");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
+
 	default:
 		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
 			header->type);
-- 
1.7.11.7

[PATCH v5 02/14] acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

ACPI_DEBUG_PRINT is a bit fragile in acpi/numa.c, the first thing
is that component ACPI_NUMA(0x80000000) is not described in the
Documentation/acpi/debug.txt, and even not defined in the struct
acpi_dlayer acpi_debug_layers which we can not dynamically enable/disable
it with /sys/modules/acpi/parameters/debug_layer. another thing
is that ACPI_DEBUG_OUTPUT is controlled by ACPICA which not coordinate
well with ACPI drivers.

Replace ACPI_DEBUG_PRINT() with pr_debug() in this patch as pr_debug
will do the same thing for debug purpose and it can make the code much
cleaner, also remove the related code which not needed anymore if
ACPI_DEBUG_PRINT() is gone.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 59 +++++++++++++++++++----------------------------------
 1 file changed, 21 insertions(+), 38 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 4e427fc..e34b5d0 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -31,10 +31,6 @@
 #include <linux/nodemask.h>
 #include <linux/topology.h>
 
-#define ACPI_NUMA	0x80000000
-#define _COMPONENT	ACPI_NUMA
-ACPI_MODULE_NAME("numa");
-
 static nodemask_t nodes_found_map = NODE_MASK_NONE;
 
 /* maps to convert between proximity domain and logical node ID */
@@ -129,64 +125,51 @@ EXPORT_SYMBOL(acpi_map_pxm_to_online_node);
 static void __init
 acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 {
-
-	ACPI_FUNCTION_NAME("acpi_table_print_srat_entry");
-
 	if (!header)
 		return;
 
 	switch (header->type) {
-
 	case ACPI_SRAT_TYPE_CPU_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_cpu_affinity *p =
 			    (struct acpi_srat_cpu_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Processor (id[0x%02x] eid[0x%02x]) in proximity domain %d %s\n",
-					  p->apic_id, p->local_sapic_eid,
-					  p->proximity_domain_lo,
-					  (p->flags & ACPI_SRAT_CPU_ENABLED)?
-					  "enabled" : "disabled"));
+			pr_debug("SRAT Processor (id[0x%02x] eid[0x%02x]) in proximity domain %d %s\n",
+				 p->apic_id, p->local_sapic_eid,
+				 p->proximity_domain_lo,
+				 (p->flags & ACPI_SRAT_CPU_ENABLED) ?
+				 "enabled" : "disabled");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 
 	case ACPI_SRAT_TYPE_MEMORY_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_mem_affinity *p =
 			    (struct acpi_srat_mem_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Memory (0x%lx length 0x%lx) in proximity domain %d %s%s%s\n",
-					  (unsigned long)p->base_address,
-					  (unsigned long)p->length,
-					  p->proximity_domain,
-					  (p->flags & ACPI_SRAT_MEM_ENABLED)?
-					  "enabled" : "disabled",
-					  (p->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE)?
-					  " hot-pluggable" : "",
-					  (p->flags & ACPI_SRAT_MEM_NON_VOLATILE)?
-					  " non-volatile" : ""));
+			pr_debug("SRAT Memory (0x%lx length 0x%lx) in proximity domain %d %s%s%s\n",
+				 (unsigned long)p->base_address,
+				 (unsigned long)p->length,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_MEM_ENABLED) ?
+				 "enabled" : "disabled",
+				 (p->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) ?
+				 " hot-pluggable" : "",
+				 (p->flags & ACPI_SRAT_MEM_NON_VOLATILE) ?
+				 " non-volatile" : "");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 
 	case ACPI_SRAT_TYPE_X2APIC_CPU_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_x2apic_cpu_affinity *p =
 			    (struct acpi_srat_x2apic_cpu_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Processor (x2apicid[0x%08x]) in"
-					  " proximity domain %d %s\n",
-					  p->apic_id,
-					  p->proximity_domain,
-					  (p->flags & ACPI_SRAT_CPU_ENABLED) ?
-					  "enabled" : "disabled"));
+			pr_debug("SRAT Processor (x2apicid[0x%08x]) in proximity domain %d %s\n",
+				 p->apic_id,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_CPU_ENABLED) ?
+				 "enabled" : "disabled");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
+
 	default:
 		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
 			header->type);
-- 
1.7.11.7

[PATCH v5 02/14] acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

ACPI_DEBUG_PRINT is a bit fragile in acpi/numa.c, the first thing
is that component ACPI_NUMA(0x80000000) is not described in the
Documentation/acpi/debug.txt, and even not defined in the struct
acpi_dlayer acpi_debug_layers which we can not dynamically enable/disable
it with /sys/modules/acpi/parameters/debug_layer. another thing
is that ACPI_DEBUG_OUTPUT is controlled by ACPICA which not coordinate
well with ACPI drivers.

Replace ACPI_DEBUG_PRINT() with pr_debug() in this patch as pr_debug
will do the same thing for debug purpose and it can make the code much
cleaner, also remove the related code which not needed anymore if
ACPI_DEBUG_PRINT() is gone.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 59 +++++++++++++++++++----------------------------------
 1 file changed, 21 insertions(+), 38 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 4e427fc..e34b5d0 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -31,10 +31,6 @@
 #include <linux/nodemask.h>
 #include <linux/topology.h>
 
-#define ACPI_NUMA	0x80000000
-#define _COMPONENT	ACPI_NUMA
-ACPI_MODULE_NAME("numa");
-
 static nodemask_t nodes_found_map = NODE_MASK_NONE;
 
 /* maps to convert between proximity domain and logical node ID */
@@ -129,64 +125,51 @@ EXPORT_SYMBOL(acpi_map_pxm_to_online_node);
 static void __init
 acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 {
-
-	ACPI_FUNCTION_NAME("acpi_table_print_srat_entry");
-
 	if (!header)
 		return;
 
 	switch (header->type) {
-
 	case ACPI_SRAT_TYPE_CPU_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_cpu_affinity *p  			    (struct acpi_srat_cpu_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Processor (id[0x%02x] eid[0x%02x]) in proximity domain %d %s\n",
-					  p->apic_id, p->local_sapic_eid,
-					  p->proximity_domain_lo,
-					  (p->flags & ACPI_SRAT_CPU_ENABLED)?
-					  "enabled" : "disabled"));
+			pr_debug("SRAT Processor (id[0x%02x] eid[0x%02x]) in proximity domain %d %s\n",
+				 p->apic_id, p->local_sapic_eid,
+				 p->proximity_domain_lo,
+				 (p->flags & ACPI_SRAT_CPU_ENABLED) ?
+				 "enabled" : "disabled");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 
 	case ACPI_SRAT_TYPE_MEMORY_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_mem_affinity *p  			    (struct acpi_srat_mem_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Memory (0x%lx length 0x%lx) in proximity domain %d %s%s%s\n",
-					  (unsigned long)p->base_address,
-					  (unsigned long)p->length,
-					  p->proximity_domain,
-					  (p->flags & ACPI_SRAT_MEM_ENABLED)?
-					  "enabled" : "disabled",
-					  (p->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE)?
-					  " hot-pluggable" : "",
-					  (p->flags & ACPI_SRAT_MEM_NON_VOLATILE)?
-					  " non-volatile" : ""));
+			pr_debug("SRAT Memory (0x%lx length 0x%lx) in proximity domain %d %s%s%s\n",
+				 (unsigned long)p->base_address,
+				 (unsigned long)p->length,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_MEM_ENABLED) ?
+				 "enabled" : "disabled",
+				 (p->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) ?
+				 " hot-pluggable" : "",
+				 (p->flags & ACPI_SRAT_MEM_NON_VOLATILE) ?
+				 " non-volatile" : "");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
 
 	case ACPI_SRAT_TYPE_X2APIC_CPU_AFFINITY:
-#ifdef ACPI_DEBUG_OUTPUT
 		{
 			struct acpi_srat_x2apic_cpu_affinity *p  			    (struct acpi_srat_x2apic_cpu_affinity *)header;
-			ACPI_DEBUG_PRINT((ACPI_DB_INFO,
-					  "SRAT Processor (x2apicid[0x%08x]) in"
-					  " proximity domain %d %s\n",
-					  p->apic_id,
-					  p->proximity_domain,
-					  (p->flags & ACPI_SRAT_CPU_ENABLED) ?
-					  "enabled" : "disabled"));
+			pr_debug("SRAT Processor (x2apicid[0x%08x]) in proximity domain %d %s\n",
+				 p->apic_id,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_CPU_ENABLED) ?
+				 "enabled" : "disabled");
 		}
-#endif				/* ACPI_DEBUG_OUTPUT */
 		break;
+
 	default:
 		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
 			header->type);
-- 
1.7.11.7

[PATCH v5 03/14] acpi, numa: remove duplicate NULL check

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

The argument "header" for acpi_table_print_srat_entry()
is always checked before the function is called, it's
duplicate to check it again, remove it.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index e34b5d0..2de6068 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -125,9 +125,6 @@ EXPORT_SYMBOL(acpi_map_pxm_to_online_node);
 static void __init
 acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 {
-	if (!header)
-		return;
-
 	switch (header->type) {
 	case ACPI_SRAT_TYPE_CPU_AFFINITY:
 		{
-- 
1.7.11.7

[PATCH v5 03/14] acpi, numa: remove duplicate NULL check

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

The argument "header" for acpi_table_print_srat_entry()
is always checked before the function is called, it's
duplicate to check it again, remove it.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index e34b5d0..2de6068 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -125,9 +125,6 @@ EXPORT_SYMBOL(acpi_map_pxm_to_online_node);
 static void __init
 acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 {
-	if (!header)
-		return;
-
 	switch (header->type) {
 	case ACPI_SRAT_TYPE_CPU_AFFINITY:
 		{
-- 
1.7.11.7

[PATCH v5 03/14] acpi, numa: remove duplicate NULL check

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

The argument "header" for acpi_table_print_srat_entry()
is always checked before the function is called, it's
duplicate to check it again, remove it.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index e34b5d0..2de6068 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -125,9 +125,6 @@ EXPORT_SYMBOL(acpi_map_pxm_to_online_node);
 static void __init
 acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 {
-	if (!header)
-		return;
-
 	switch (header->type) {
 	case ACPI_SRAT_TYPE_CPU_AFFINITY:
 		{
-- 
1.7.11.7

[PATCH v5 03/14] acpi, numa: remove duplicate NULL check

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

The argument "header" for acpi_table_print_srat_entry()
is always checked before the function is called, it's
duplicate to check it again, remove it.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index e34b5d0..2de6068 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -125,9 +125,6 @@ EXPORT_SYMBOL(acpi_map_pxm_to_online_node);
 static void __init
 acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 {
-	if (!header)
-		return;
-
 	switch (header->type) {
 	case ACPI_SRAT_TYPE_CPU_AFFINITY:
 		{
-- 
1.7.11.7

[PATCH v5 04/14] acpi, numa: Move acpi_numa_arch_fixup() to ia64 only

[David Daney]

From: Robert Richter <rrichter@cavium.com>

Since acpi_numa_arch_fixup() is only used in arch ia64, move it there
to make a generic interface easier. This avoids empty function stubs
or some complex kconfig options for x86 and arm64.

Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/ia64/include/asm/acpi.h | 3 +++
 arch/ia64/kernel/acpi.c      | 2 +-
 arch/ia64/kernel/setup.c     | 1 +
 arch/x86/mm/srat.c           | 2 --
 drivers/acpi/numa.c          | 2 --
 include/linux/acpi.h         | 1 -
 6 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/arch/ia64/include/asm/acpi.h b/arch/ia64/include/asm/acpi.h
index aa0fdf1..a3d0211 100644
--- a/arch/ia64/include/asm/acpi.h
+++ b/arch/ia64/include/asm/acpi.h
@@ -140,6 +140,9 @@ static inline void per_cpu_scan_finalize(int min_cpus, int reserve_cpus)
 		}
 	}
 }
+
+extern void acpi_numa_fixup(void);
+
 #endif /* CONFIG_ACPI_NUMA */
 
 #endif /*__KERNEL__*/
diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
index b1698bc..92b7bc9 100644
--- a/arch/ia64/kernel/acpi.c
+++ b/arch/ia64/kernel/acpi.c
@@ -524,7 +524,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	return 0;
 }
 
-void __init acpi_numa_arch_fixup(void)
+void __init acpi_numa_fixup(void)
 {
 	int i, j, node_from, node_to;
 
diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
index 2029a38..afddb3e 100644
--- a/arch/ia64/kernel/setup.c
+++ b/arch/ia64/kernel/setup.c
@@ -552,6 +552,7 @@ setup_arch (char **cmdline_p)
 	early_acpi_boot_init();
 # ifdef CONFIG_ACPI_NUMA
 	acpi_numa_init();
+	acpi_numa_fixup();
 #  ifdef CONFIG_ACPI_HOTPLUG_CPU
 	prefill_possible_map();
 #  endif
diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index b5f8218..90b6ed9 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -212,8 +212,6 @@ out_err:
 	return -1;
 }
 
-void __init acpi_numa_arch_fixup(void) {}
-
 int __init x86_acpi_numa_init(void)
 {
 	int ret;
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 2de6068..f41f06b 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -316,8 +316,6 @@ int __init acpi_numa_init(void)
 	/* SLIT: System Locality Information Table */
 	acpi_table_parse(ACPI_SIG_SLIT, acpi_parse_slit);
 
-	acpi_numa_arch_fixup();
-
 	if (cnt < 0)
 		return cnt;
 	else if (!parsed_numa_memblks)
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index 06ed7e5..ab6fd96 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -244,7 +244,6 @@ void acpi_numa_slit_init (struct acpi_table_slit *slit);
 void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
 void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
 int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
-void acpi_numa_arch_fixup(void);
 
 #ifndef PHYS_CPUID_INVALID
 typedef u32 phys_cpuid_t;
-- 
1.7.11.7

[PATCH v5 04/14] acpi, numa: Move acpi_numa_arch_fixup() to ia64 only

[David Daney]

From: Robert Richter <rrichter@cavium.com>

Since acpi_numa_arch_fixup() is only used in arch ia64, move it there
to make a generic interface easier. This avoids empty function stubs
or some complex kconfig options for x86 and arm64.

Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/ia64/include/asm/acpi.h | 3 +++
 arch/ia64/kernel/acpi.c      | 2 +-
 arch/ia64/kernel/setup.c     | 1 +
 arch/x86/mm/srat.c           | 2 --
 drivers/acpi/numa.c          | 2 --
 include/linux/acpi.h         | 1 -
 6 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/arch/ia64/include/asm/acpi.h b/arch/ia64/include/asm/acpi.h
index aa0fdf1..a3d0211 100644
--- a/arch/ia64/include/asm/acpi.h
+++ b/arch/ia64/include/asm/acpi.h
@@ -140,6 +140,9 @@ static inline void per_cpu_scan_finalize(int min_cpus, int reserve_cpus)
 		}
 	}
 }
+
+extern void acpi_numa_fixup(void);
+
 #endif /* CONFIG_ACPI_NUMA */
 
 #endif /*__KERNEL__*/
diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
index b1698bc..92b7bc9 100644
--- a/arch/ia64/kernel/acpi.c
+++ b/arch/ia64/kernel/acpi.c
@@ -524,7 +524,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	return 0;
 }
 
-void __init acpi_numa_arch_fixup(void)
+void __init acpi_numa_fixup(void)
 {
 	int i, j, node_from, node_to;
 
diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
index 2029a38..afddb3e 100644
--- a/arch/ia64/kernel/setup.c
+++ b/arch/ia64/kernel/setup.c
@@ -552,6 +552,7 @@ setup_arch (char **cmdline_p)
 	early_acpi_boot_init();
 # ifdef CONFIG_ACPI_NUMA
 	acpi_numa_init();
+	acpi_numa_fixup();
 #  ifdef CONFIG_ACPI_HOTPLUG_CPU
 	prefill_possible_map();
 #  endif
diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index b5f8218..90b6ed9 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -212,8 +212,6 @@ out_err:
 	return -1;
 }
 
-void __init acpi_numa_arch_fixup(void) {}
-
 int __init x86_acpi_numa_init(void)
 {
 	int ret;
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 2de6068..f41f06b 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -316,8 +316,6 @@ int __init acpi_numa_init(void)
 	/* SLIT: System Locality Information Table */
 	acpi_table_parse(ACPI_SIG_SLIT, acpi_parse_slit);
 
-	acpi_numa_arch_fixup();
-
 	if (cnt < 0)
 		return cnt;
 	else if (!parsed_numa_memblks)
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index 06ed7e5..ab6fd96 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -244,7 +244,6 @@ void acpi_numa_slit_init (struct acpi_table_slit *slit);
 void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
 void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
 int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
-void acpi_numa_arch_fixup(void);
 
 #ifndef PHYS_CPUID_INVALID
 typedef u32 phys_cpuid_t;
-- 
1.7.11.7

[PATCH v5 04/14] acpi, numa: Move acpi_numa_arch_fixup() to ia64 only

[David Daney]

From: Robert Richter <rrichter@cavium.com>

Since acpi_numa_arch_fixup() is only used in arch ia64, move it there
to make a generic interface easier. This avoids empty function stubs
or some complex kconfig options for x86 and arm64.

Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/ia64/include/asm/acpi.h | 3 +++
 arch/ia64/kernel/acpi.c      | 2 +-
 arch/ia64/kernel/setup.c     | 1 +
 arch/x86/mm/srat.c           | 2 --
 drivers/acpi/numa.c          | 2 --
 include/linux/acpi.h         | 1 -
 6 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/arch/ia64/include/asm/acpi.h b/arch/ia64/include/asm/acpi.h
index aa0fdf1..a3d0211 100644
--- a/arch/ia64/include/asm/acpi.h
+++ b/arch/ia64/include/asm/acpi.h
@@ -140,6 +140,9 @@ static inline void per_cpu_scan_finalize(int min_cpus, int reserve_cpus)
 		}
 	}
 }
+
+extern void acpi_numa_fixup(void);
+
 #endif /* CONFIG_ACPI_NUMA */
 
 #endif /*__KERNEL__*/
diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
index b1698bc..92b7bc9 100644
--- a/arch/ia64/kernel/acpi.c
+++ b/arch/ia64/kernel/acpi.c
@@ -524,7 +524,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	return 0;
 }
 
-void __init acpi_numa_arch_fixup(void)
+void __init acpi_numa_fixup(void)
 {
 	int i, j, node_from, node_to;
 
diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
index 2029a38..afddb3e 100644
--- a/arch/ia64/kernel/setup.c
+++ b/arch/ia64/kernel/setup.c
@@ -552,6 +552,7 @@ setup_arch (char **cmdline_p)
 	early_acpi_boot_init();
 # ifdef CONFIG_ACPI_NUMA
 	acpi_numa_init();
+	acpi_numa_fixup();
 #  ifdef CONFIG_ACPI_HOTPLUG_CPU
 	prefill_possible_map();
 #  endif
diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index b5f8218..90b6ed9 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -212,8 +212,6 @@ out_err:
 	return -1;
 }
 
-void __init acpi_numa_arch_fixup(void) {}
-
 int __init x86_acpi_numa_init(void)
 {
 	int ret;
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 2de6068..f41f06b 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -316,8 +316,6 @@ int __init acpi_numa_init(void)
 	/* SLIT: System Locality Information Table */
 	acpi_table_parse(ACPI_SIG_SLIT, acpi_parse_slit);
 
-	acpi_numa_arch_fixup();
-
 	if (cnt < 0)
 		return cnt;
 	else if (!parsed_numa_memblks)
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index 06ed7e5..ab6fd96 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -244,7 +244,6 @@ void acpi_numa_slit_init (struct acpi_table_slit *slit);
 void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
 void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
 int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
-void acpi_numa_arch_fixup(void);
 
 #ifndef PHYS_CPUID_INVALID
 typedef u32 phys_cpuid_t;
-- 
1.7.11.7

[PATCH v5 05/14] acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Identical implementations of acpi_numa_slit_init() are used by both
x86 and follow-on arm64 support.  Move it to drivers/acpi/numa.c, and
guard with CONFIG_X86 || CONFIG_ARM64 because ia64 has its own
architecture specific implementation.

No code change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c  | 27 ---------------------------
 drivers/acpi/numa.c | 29 +++++++++++++++++++++++++++++
 2 files changed, 29 insertions(+), 27 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 90b6ed9..f242a11 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -42,33 +42,6 @@ static __init inline int srat_disabled(void)
 	return acpi_numa < 0;
 }
 
-/*
- * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
- * I/O localities since SRAT does not list them.  I/O localities are
- * not supported at this point.
- */
-void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
-{
-	int i, j;
-
-	for (i = 0; i < slit->locality_count; i++) {
-		const int from_node = pxm_to_node(i);
-
-		if (from_node == NUMA_NO_NODE)
-			continue;
-
-		for (j = 0; j < slit->locality_count; j++) {
-			const int to_node = pxm_to_node(j);
-
-			if (to_node == NUMA_NO_NODE)
-				continue;
-
-			numa_set_distance(from_node, to_node,
-				slit->entry[slit->locality_count * i + j]);
-		}
-	}
-}
-
 /* Callback for Proximity Domain -> x2APIC mapping */
 void __init
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index f41f06b..acc361d 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -197,6 +197,35 @@ static int __init slit_valid(struct acpi_table_slit *slit)
 	return 1;
 }
 
+#if defined(CONFIG_X86) || defined(CONFIG_ARM64)
+/*
+ * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
+ * I/O localities since SRAT does not list them.  I/O localities are
+ * not supported at this point.
+ */
+void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
+{
+	int i, j;
+
+	for (i = 0; i < slit->locality_count; i++) {
+		const int from_node = pxm_to_node(i);
+
+		if (from_node == NUMA_NO_NODE)
+			continue;
+
+		for (j = 0; j < slit->locality_count; j++) {
+			const int to_node = pxm_to_node(j);
+
+			if (to_node == NUMA_NO_NODE)
+				continue;
+
+			numa_set_distance(from_node, to_node,
+				slit->entry[slit->locality_count * i + j]);
+		}
+	}
+}
+#endif /* defined(CONFIG_X86) || defined (CONFIG_ARM64) */
+
 static int __init acpi_parse_slit(struct acpi_table_header *table)
 {
 	struct acpi_table_slit *slit = (struct acpi_table_slit *)table;
-- 
1.7.11.7

[PATCH v5 05/14] acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Identical implementations of acpi_numa_slit_init() are used by both
x86 and follow-on arm64 support.  Move it to drivers/acpi/numa.c, and
guard with CONFIG_X86 || CONFIG_ARM64 because ia64 has its own
architecture specific implementation.

No code change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c  | 27 ---------------------------
 drivers/acpi/numa.c | 29 +++++++++++++++++++++++++++++
 2 files changed, 29 insertions(+), 27 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 90b6ed9..f242a11 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -42,33 +42,6 @@ static __init inline int srat_disabled(void)
 	return acpi_numa < 0;
 }
 
-/*
- * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
- * I/O localities since SRAT does not list them.  I/O localities are
- * not supported@this point.
- */
-void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
-{
-	int i, j;
-
-	for (i = 0; i < slit->locality_count; i++) {
-		const int from_node = pxm_to_node(i);
-
-		if (from_node == NUMA_NO_NODE)
-			continue;
-
-		for (j = 0; j < slit->locality_count; j++) {
-			const int to_node = pxm_to_node(j);
-
-			if (to_node == NUMA_NO_NODE)
-				continue;
-
-			numa_set_distance(from_node, to_node,
-				slit->entry[slit->locality_count * i + j]);
-		}
-	}
-}
-
 /* Callback for Proximity Domain -> x2APIC mapping */
 void __init
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index f41f06b..acc361d 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -197,6 +197,35 @@ static int __init slit_valid(struct acpi_table_slit *slit)
 	return 1;
 }
 
+#if defined(CONFIG_X86) || defined(CONFIG_ARM64)
+/*
+ * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
+ * I/O localities since SRAT does not list them.  I/O localities are
+ * not supported at this point.
+ */
+void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
+{
+	int i, j;
+
+	for (i = 0; i < slit->locality_count; i++) {
+		const int from_node = pxm_to_node(i);
+
+		if (from_node == NUMA_NO_NODE)
+			continue;
+
+		for (j = 0; j < slit->locality_count; j++) {
+			const int to_node = pxm_to_node(j);
+
+			if (to_node == NUMA_NO_NODE)
+				continue;
+
+			numa_set_distance(from_node, to_node,
+				slit->entry[slit->locality_count * i + j]);
+		}
+	}
+}
+#endif /* defined(CONFIG_X86) || defined (CONFIG_ARM64) */
+
 static int __init acpi_parse_slit(struct acpi_table_header *table)
 {
 	struct acpi_table_slit *slit = (struct acpi_table_slit *)table;
-- 
1.7.11.7

[PATCH v5 05/14] acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Identical implementations of acpi_numa_slit_init() are used by both
x86 and follow-on arm64 support.  Move it to drivers/acpi/numa.c, and
guard with CONFIG_X86 || CONFIG_ARM64 because ia64 has its own
architecture specific implementation.

No code change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c  | 27 ---------------------------
 drivers/acpi/numa.c | 29 +++++++++++++++++++++++++++++
 2 files changed, 29 insertions(+), 27 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 90b6ed9..f242a11 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -42,33 +42,6 @@ static __init inline int srat_disabled(void)
 	return acpi_numa < 0;
 }
 
-/*
- * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
- * I/O localities since SRAT does not list them.  I/O localities are
- * not supported at this point.
- */
-void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
-{
-	int i, j;
-
-	for (i = 0; i < slit->locality_count; i++) {
-		const int from_node = pxm_to_node(i);
-
-		if (from_node = NUMA_NO_NODE)
-			continue;
-
-		for (j = 0; j < slit->locality_count; j++) {
-			const int to_node = pxm_to_node(j);
-
-			if (to_node = NUMA_NO_NODE)
-				continue;
-
-			numa_set_distance(from_node, to_node,
-				slit->entry[slit->locality_count * i + j]);
-		}
-	}
-}
-
 /* Callback for Proximity Domain -> x2APIC mapping */
 void __init
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index f41f06b..acc361d 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -197,6 +197,35 @@ static int __init slit_valid(struct acpi_table_slit *slit)
 	return 1;
 }
 
+#if defined(CONFIG_X86) || defined(CONFIG_ARM64)
+/*
+ * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
+ * I/O localities since SRAT does not list them.  I/O localities are
+ * not supported at this point.
+ */
+void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
+{
+	int i, j;
+
+	for (i = 0; i < slit->locality_count; i++) {
+		const int from_node = pxm_to_node(i);
+
+		if (from_node = NUMA_NO_NODE)
+			continue;
+
+		for (j = 0; j < slit->locality_count; j++) {
+			const int to_node = pxm_to_node(j);
+
+			if (to_node = NUMA_NO_NODE)
+				continue;
+
+			numa_set_distance(from_node, to_node,
+				slit->entry[slit->locality_count * i + j]);
+		}
+	}
+}
+#endif /* defined(CONFIG_X86) || defined (CONFIG_ARM64) */
+
 static int __init acpi_parse_slit(struct acpi_table_header *table)
 {
 	struct acpi_table_slit *slit = (struct acpi_table_slit *)table;
-- 
1.7.11.7

[PATCH v5 06/14] arm64, numa: rework numa_add_memblk()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Rework numa_add_memblk() to update the parameter "u64 size" to "u64
end", this will make it consistent with x86 and simplifies the arm64
ACPI NUMA code to be added later.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/mm/numa.c | 12 ++++++------
 drivers/of/of_numa.c |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index 98dc104..6cb03f9 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -131,25 +131,25 @@ void __init early_map_cpu_to_node(unsigned int cpu, int nid)
  * numa_add_memblk - Set node id to memblk
  * @nid: NUMA node ID of the new memblk
  * @start: Start address of the new memblk
- * @size:  Size of the new memblk
+ * @end:  End address of the new memblk
  *
  * RETURNS:
  * 0 on success, -errno on failure.
  */
-int __init numa_add_memblk(int nid, u64 start, u64 size)
+int __init numa_add_memblk(int nid, u64 start, u64 end)
 {
 	int ret;
 
-	ret = memblock_set_node(start, size, &memblock.memory, nid);
+	ret = memblock_set_node(start, (end - start), &memblock.memory, nid);
 	if (ret < 0) {
 		pr_err("NUMA: memblock [0x%llx - 0x%llx] failed to add on node %d\n",
-			start, (start + size - 1), nid);
+			start, (end - 1), nid);
 		return ret;
 	}
 
 	node_set(nid, numa_nodes_parsed);
 	pr_info("NUMA: Adding memblock [0x%llx - 0x%llx] on node %d\n",
-			start, (start + size - 1), nid);
+			start, (end - 1), nid);
 	return ret;
 }
 
@@ -367,7 +367,7 @@ static int __init dummy_numa_init(void)
 	       0LLU, PFN_PHYS(max_pfn) - 1);
 
 	for_each_memblock(memory, mblk) {
-		ret = numa_add_memblk(0, mblk->base, mblk->size);
+		ret = numa_add_memblk(0, mblk->base, mblk->base + mblk->size);
 		if (!ret)
 			continue;
 
diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index 0f2784b..ed5a097 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -91,8 +91,8 @@ static int __init of_numa_parse_memory_nodes(void)
 		pr_debug("NUMA:  base = %llx len = %llx, node = %u\n",
 			 rsrc.start, rsrc.end - rsrc.start + 1, nid);
 
-		r = numa_add_memblk(nid, rsrc.start,
-				    rsrc.end - rsrc.start + 1);
+
+		r = numa_add_memblk(nid, rsrc.start, rsrc.end + 1);
 		if (r)
 			break;
 	}
-- 
1.7.11.7

[PATCH v5 06/14] arm64, numa: rework numa_add_memblk()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Rework numa_add_memblk() to update the parameter "u64 size" to "u64
end", this will make it consistent with x86 and simplifies the arm64
ACPI NUMA code to be added later.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/mm/numa.c | 12 ++++++------
 drivers/of/of_numa.c |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index 98dc104..6cb03f9 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -131,25 +131,25 @@ void __init early_map_cpu_to_node(unsigned int cpu, int nid)
  * numa_add_memblk - Set node id to memblk
  * @nid: NUMA node ID of the new memblk
  * @start: Start address of the new memblk
- * @size:  Size of the new memblk
+ * @end:  End address of the new memblk
  *
  * RETURNS:
  * 0 on success, -errno on failure.
  */
-int __init numa_add_memblk(int nid, u64 start, u64 size)
+int __init numa_add_memblk(int nid, u64 start, u64 end)
 {
 	int ret;
 
-	ret = memblock_set_node(start, size, &memblock.memory, nid);
+	ret = memblock_set_node(start, (end - start), &memblock.memory, nid);
 	if (ret < 0) {
 		pr_err("NUMA: memblock [0x%llx - 0x%llx] failed to add on node %d\n",
-			start, (start + size - 1), nid);
+			start, (end - 1), nid);
 		return ret;
 	}
 
 	node_set(nid, numa_nodes_parsed);
 	pr_info("NUMA: Adding memblock [0x%llx - 0x%llx] on node %d\n",
-			start, (start + size - 1), nid);
+			start, (end - 1), nid);
 	return ret;
 }
 
@@ -367,7 +367,7 @@ static int __init dummy_numa_init(void)
 	       0LLU, PFN_PHYS(max_pfn) - 1);
 
 	for_each_memblock(memory, mblk) {
-		ret = numa_add_memblk(0, mblk->base, mblk->size);
+		ret = numa_add_memblk(0, mblk->base, mblk->base + mblk->size);
 		if (!ret)
 			continue;
 
diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index 0f2784b..ed5a097 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -91,8 +91,8 @@ static int __init of_numa_parse_memory_nodes(void)
 		pr_debug("NUMA:  base = %llx len = %llx, node = %u\n",
 			 rsrc.start, rsrc.end - rsrc.start + 1, nid);
 
-		r = numa_add_memblk(nid, rsrc.start,
-				    rsrc.end - rsrc.start + 1);
+
+		r = numa_add_memblk(nid, rsrc.start, rsrc.end + 1);
 		if (r)
 			break;
 	}
-- 
1.7.11.7

[PATCH v5 06/14] arm64, numa: rework numa_add_memblk()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Rework numa_add_memblk() to update the parameter "u64 size" to "u64
end", this will make it consistent with x86 and simplifies the arm64
ACPI NUMA code to be added later.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/mm/numa.c | 12 ++++++------
 drivers/of/of_numa.c |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index 98dc104..6cb03f9 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -131,25 +131,25 @@ void __init early_map_cpu_to_node(unsigned int cpu, int nid)
  * numa_add_memblk - Set node id to memblk
  * @nid: NUMA node ID of the new memblk
  * @start: Start address of the new memblk
- * @size:  Size of the new memblk
+ * @end:  End address of the new memblk
  *
  * RETURNS:
  * 0 on success, -errno on failure.
  */
-int __init numa_add_memblk(int nid, u64 start, u64 size)
+int __init numa_add_memblk(int nid, u64 start, u64 end)
 {
 	int ret;
 
-	ret = memblock_set_node(start, size, &memblock.memory, nid);
+	ret = memblock_set_node(start, (end - start), &memblock.memory, nid);
 	if (ret < 0) {
 		pr_err("NUMA: memblock [0x%llx - 0x%llx] failed to add on node %d\n",
-			start, (start + size - 1), nid);
+			start, (end - 1), nid);
 		return ret;
 	}
 
 	node_set(nid, numa_nodes_parsed);
 	pr_info("NUMA: Adding memblock [0x%llx - 0x%llx] on node %d\n",
-			start, (start + size - 1), nid);
+			start, (end - 1), nid);
 	return ret;
 }
 
@@ -367,7 +367,7 @@ static int __init dummy_numa_init(void)
 	       0LLU, PFN_PHYS(max_pfn) - 1);
 
 	for_each_memblock(memory, mblk) {
-		ret = numa_add_memblk(0, mblk->base, mblk->size);
+		ret = numa_add_memblk(0, mblk->base, mblk->base + mblk->size);
 		if (!ret)
 			continue;
 
diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index 0f2784b..ed5a097 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -91,8 +91,8 @@ static int __init of_numa_parse_memory_nodes(void)
 		pr_debug("NUMA:  base = %llx len = %llx, node = %u\n",
 			 rsrc.start, rsrc.end - rsrc.start + 1, nid);
 
-		r = numa_add_memblk(nid, rsrc.start,
-				    rsrc.end - rsrc.start + 1);
+
+		r = numa_add_memblk(nid, rsrc.start, rsrc.end + 1);
 		if (r)
 			break;
 	}
-- 
1.7.11.7

[PATCH v5 06/14] arm64, numa: rework numa_add_memblk()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Rework numa_add_memblk() to update the parameter "u64 size" to "u64
end", this will make it consistent with x86 and simplifies the arm64
ACPI NUMA code to be added later.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/mm/numa.c | 12 ++++++------
 drivers/of/of_numa.c |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index 98dc104..6cb03f9 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -131,25 +131,25 @@ void __init early_map_cpu_to_node(unsigned int cpu, int nid)
  * numa_add_memblk - Set node id to memblk
  * @nid: NUMA node ID of the new memblk
  * @start: Start address of the new memblk
- * @size:  Size of the new memblk
+ * @end:  End address of the new memblk
  *
  * RETURNS:
  * 0 on success, -errno on failure.
  */
-int __init numa_add_memblk(int nid, u64 start, u64 size)
+int __init numa_add_memblk(int nid, u64 start, u64 end)
 {
 	int ret;
 
-	ret = memblock_set_node(start, size, &memblock.memory, nid);
+	ret = memblock_set_node(start, (end - start), &memblock.memory, nid);
 	if (ret < 0) {
 		pr_err("NUMA: memblock [0x%llx - 0x%llx] failed to add on node %d\n",
-			start, (start + size - 1), nid);
+			start, (end - 1), nid);
 		return ret;
 	}
 
 	node_set(nid, numa_nodes_parsed);
 	pr_info("NUMA: Adding memblock [0x%llx - 0x%llx] on node %d\n",
-			start, (start + size - 1), nid);
+			start, (end - 1), nid);
 	return ret;
 }
 
@@ -367,7 +367,7 @@ static int __init dummy_numa_init(void)
 	       0LLU, PFN_PHYS(max_pfn) - 1);
 
 	for_each_memblock(memory, mblk) {
-		ret = numa_add_memblk(0, mblk->base, mblk->size);
+		ret = numa_add_memblk(0, mblk->base, mblk->base + mblk->size);
 		if (!ret)
 			continue;
 
diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index 0f2784b..ed5a097 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -91,8 +91,8 @@ static int __init of_numa_parse_memory_nodes(void)
 		pr_debug("NUMA:  base = %llx len = %llx, node = %u\n",
 			 rsrc.start, rsrc.end - rsrc.start + 1, nid);
 
-		r = numa_add_memblk(nid, rsrc.start,
-				    rsrc.end - rsrc.start + 1);
+
+		r = numa_add_memblk(nid, rsrc.start, rsrc.end + 1);
 		if (r)
 			break;
 	}
-- 
1.7.11.7

[PATCH v5 07/14] x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Cleanup acpi_numa_processor_affinity_init() in preparation for its
move to drivers/acpi/numa.c.  It will be reused by arm64, this has no
functional change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c | 21 +++++----------------
 1 file changed, 5 insertions(+), 16 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index f242a11..9e2a833 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -26,11 +26,6 @@
 
 int acpi_numa __initdata;
 
-static __init int setup_node(int pxm)
-{
-	return acpi_map_pxm_to_node(pxm);
-}
-
 static __init void bad_srat(void)
 {
 	printk(KERN_ERR "SRAT: SRAT not used.\n");
@@ -64,7 +59,7 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 			 pxm, apic_id);
 		return;
 	}
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
 		bad_srat();
@@ -100,7 +95,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	pxm = pa->proximity_domain_lo;
 	if (acpi_srat_revision >= 2)
 		pxm |= *((unsigned int*)pa->proximity_domain_hi) << 8;
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
 		bad_srat();
@@ -124,12 +119,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	       pxm, apic_id, node);
 }
 
-#ifdef CONFIG_MEMORY_HOTPLUG
-static inline int save_add_info(void) {return 1;}
-#else
-static inline int save_add_info(void) {return 0;}
-#endif
-
 /* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
 int __init
 acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
@@ -145,7 +134,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
 		goto out_err;
 	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
-	if (hotpluggable && !save_add_info())
+	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
 		goto out_err;
 
 	start = ma->base_address;
@@ -154,7 +143,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if (acpi_srat_revision <= 1)
 		pxm &= 0xff;
 
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
 		goto out_err_bad_srat;
@@ -182,7 +171,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 out_err_bad_srat:
 	bad_srat();
 out_err:
-	return -1;
+	return -EINVAL;
 }
 
 int __init x86_acpi_numa_init(void)
-- 
1.7.11.7

[PATCH v5 07/14] x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Cleanup acpi_numa_processor_affinity_init() in preparation for its
move to drivers/acpi/numa.c.  It will be reused by arm64, this has no
functional change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c | 21 +++++----------------
 1 file changed, 5 insertions(+), 16 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index f242a11..9e2a833 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -26,11 +26,6 @@
 
 int acpi_numa __initdata;
 
-static __init int setup_node(int pxm)
-{
-	return acpi_map_pxm_to_node(pxm);
-}
-
 static __init void bad_srat(void)
 {
 	printk(KERN_ERR "SRAT: SRAT not used.\n");
@@ -64,7 +59,7 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 			 pxm, apic_id);
 		return;
 	}
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
 		bad_srat();
@@ -100,7 +95,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	pxm = pa->proximity_domain_lo;
 	if (acpi_srat_revision >= 2)
 		pxm |= *((unsigned int*)pa->proximity_domain_hi) << 8;
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
 		bad_srat();
@@ -124,12 +119,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	       pxm, apic_id, node);
 }
 
-#ifdef CONFIG_MEMORY_HOTPLUG
-static inline int save_add_info(void) {return 1;}
-#else
-static inline int save_add_info(void) {return 0;}
-#endif
-
 /* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
 int __init
 acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
@@ -145,7 +134,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
 		goto out_err;
 	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
-	if (hotpluggable && !save_add_info())
+	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
 		goto out_err;
 
 	start = ma->base_address;
@@ -154,7 +143,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if (acpi_srat_revision <= 1)
 		pxm &= 0xff;
 
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
 		goto out_err_bad_srat;
@@ -182,7 +171,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 out_err_bad_srat:
 	bad_srat();
 out_err:
-	return -1;
+	return -EINVAL;
 }
 
 int __init x86_acpi_numa_init(void)
-- 
1.7.11.7

[PATCH v5 07/14] x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Cleanup acpi_numa_processor_affinity_init() in preparation for its
move to drivers/acpi/numa.c.  It will be reused by arm64, this has no
functional change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c | 21 +++++----------------
 1 file changed, 5 insertions(+), 16 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index f242a11..9e2a833 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -26,11 +26,6 @@
 
 int acpi_numa __initdata;
 
-static __init int setup_node(int pxm)
-{
-	return acpi_map_pxm_to_node(pxm);
-}
-
 static __init void bad_srat(void)
 {
 	printk(KERN_ERR "SRAT: SRAT not used.\n");
@@ -64,7 +59,7 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 			 pxm, apic_id);
 		return;
 	}
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
 		bad_srat();
@@ -100,7 +95,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	pxm = pa->proximity_domain_lo;
 	if (acpi_srat_revision >= 2)
 		pxm |= *((unsigned int*)pa->proximity_domain_hi) << 8;
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
 		bad_srat();
@@ -124,12 +119,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	       pxm, apic_id, node);
 }
 
-#ifdef CONFIG_MEMORY_HOTPLUG
-static inline int save_add_info(void) {return 1;}
-#else
-static inline int save_add_info(void) {return 0;}
-#endif
-
 /* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
 int __init
 acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
@@ -145,7 +134,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
 		goto out_err;
 	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
-	if (hotpluggable && !save_add_info())
+	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
 		goto out_err;
 
 	start = ma->base_address;
@@ -154,7 +143,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if (acpi_srat_revision <= 1)
 		pxm &= 0xff;
 
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
 		goto out_err_bad_srat;
@@ -182,7 +171,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 out_err_bad_srat:
 	bad_srat();
 out_err:
-	return -1;
+	return -EINVAL;
 }
 
 int __init x86_acpi_numa_init(void)
-- 
1.7.11.7

[PATCH v5 07/14] x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Cleanup acpi_numa_processor_affinity_init() in preparation for its
move to drivers/acpi/numa.c.  It will be reused by arm64, this has no
functional change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c | 21 +++++----------------
 1 file changed, 5 insertions(+), 16 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index f242a11..9e2a833 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -26,11 +26,6 @@
 
 int acpi_numa __initdata;
 
-static __init int setup_node(int pxm)
-{
-	return acpi_map_pxm_to_node(pxm);
-}
-
 static __init void bad_srat(void)
 {
 	printk(KERN_ERR "SRAT: SRAT not used.\n");
@@ -64,7 +59,7 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 			 pxm, apic_id);
 		return;
 	}
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
 		bad_srat();
@@ -100,7 +95,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	pxm = pa->proximity_domain_lo;
 	if (acpi_srat_revision >= 2)
 		pxm |= *((unsigned int*)pa->proximity_domain_hi) << 8;
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
 		bad_srat();
@@ -124,12 +119,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	       pxm, apic_id, node);
 }
 
-#ifdef CONFIG_MEMORY_HOTPLUG
-static inline int save_add_info(void) {return 1;}
-#else
-static inline int save_add_info(void) {return 0;}
-#endif
-
 /* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
 int __init
 acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
@@ -145,7 +134,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) = 0)
 		goto out_err;
 	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
-	if (hotpluggable && !save_add_info())
+	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
 		goto out_err;
 
 	start = ma->base_address;
@@ -154,7 +143,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if (acpi_srat_revision <= 1)
 		pxm &= 0xff;
 
-	node = setup_node(pxm);
+	node = acpi_map_pxm_to_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
 		goto out_err_bad_srat;
@@ -182,7 +171,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 out_err_bad_srat:
 	bad_srat();
 out_err:
-	return -1;
+	return -EINVAL;
 }
 
 int __init x86_acpi_numa_init(void)
-- 
1.7.11.7

[PATCH v5 08/14] acpi, numa: move bad_srat() and srat_disabled() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

bad_srat() and srat_disabled() are shared by x86 and follow-on arm64
patches.  Move them to drivers/acpi/numa.c in preparation for arm64
support.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
[david.daney@cavium.com moved definitions to drivers/acpi/numa.c]
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/include/asm/acpi.h |  1 -
 arch/x86/mm/numa.c          |  2 +-
 arch/x86/mm/srat.c          | 13 -------------
 drivers/acpi/numa.c         | 12 ++++++++++++
 include/acpi/acpi_numa.h    |  4 ++++
 5 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/arch/x86/include/asm/acpi.h b/arch/x86/include/asm/acpi.h
index 94c18eb..65f1e95 100644
--- a/arch/x86/include/asm/acpi.h
+++ b/arch/x86/include/asm/acpi.h
@@ -145,7 +145,6 @@ static inline void disable_acpi(void) { }
 #define ARCH_HAS_POWER_INIT	1
 
 #ifdef CONFIG_ACPI_NUMA
-extern int acpi_numa;
 extern int x86_acpi_numa_init(void);
 #endif /* CONFIG_ACPI_NUMA */
 
diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
index f70c1ff..57673f5 100644
--- a/arch/x86/mm/numa.c
+++ b/arch/x86/mm/numa.c
@@ -1,4 +1,5 @@
 /* Common code for 32 and 64-bit NUMA */
+#include <linux/acpi.h>
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/string.h>
@@ -15,7 +16,6 @@
 #include <asm/e820.h>
 #include <asm/proto.h>
 #include <asm/dma.h>
-#include <asm/acpi.h>
 #include <asm/amd_nb.h>
 
 #include "numa_internal.h"
diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 9e2a833..4217071f 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -24,19 +24,6 @@
 #include <asm/apic.h>
 #include <asm/uv/uv.h>
 
-int acpi_numa __initdata;
-
-static __init void bad_srat(void)
-{
-	printk(KERN_ERR "SRAT: SRAT not used.\n");
-	acpi_numa = -1;
-}
-
-static __init inline int srat_disabled(void)
-{
-	return acpi_numa < 0;
-}
-
 /* Callback for Proximity Domain -> x2APIC mapping */
 void __init
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index acc361d..7174a38 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -40,6 +40,7 @@ static int node_to_pxm_map[MAX_NUMNODES]
 			= { [0 ... MAX_NUMNODES - 1] = PXM_INVAL };
 
 unsigned char acpi_srat_revision __initdata;
+int acpi_numa __initdata;
 
 int pxm_to_node(int pxm)
 {
@@ -197,6 +198,17 @@ static int __init slit_valid(struct acpi_table_slit *slit)
 	return 1;
 }
 
+void __init bad_srat(void)
+{
+	pr_err("SRAT: SRAT not used.\n");
+	acpi_numa = -1;
+}
+
+int __init srat_disabled(void)
+{
+	return acpi_numa < 0;
+}
+
 #if defined(CONFIG_X86) || defined(CONFIG_ARM64)
 /*
  * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
diff --git a/include/acpi/acpi_numa.h b/include/acpi/acpi_numa.h
index 94a37cd..d4b7294 100644
--- a/include/acpi/acpi_numa.h
+++ b/include/acpi/acpi_numa.h
@@ -15,6 +15,10 @@ extern int pxm_to_node(int);
 extern int node_to_pxm(int);
 extern int acpi_map_pxm_to_node(int);
 extern unsigned char acpi_srat_revision;
+extern int acpi_numa __initdata;
+
+extern void bad_srat(void);
+extern int srat_disabled(void);
 
 #endif				/* CONFIG_ACPI_NUMA */
 #endif				/* __ACP_NUMA_H */
-- 
1.7.11.7

[PATCH v5 08/14] acpi, numa: move bad_srat() and srat_disabled() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

bad_srat() and srat_disabled() are shared by x86 and follow-on arm64
patches.  Move them to drivers/acpi/numa.c in preparation for arm64
support.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
[david.daney at cavium.com moved definitions to drivers/acpi/numa.c]
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/include/asm/acpi.h |  1 -
 arch/x86/mm/numa.c          |  2 +-
 arch/x86/mm/srat.c          | 13 -------------
 drivers/acpi/numa.c         | 12 ++++++++++++
 include/acpi/acpi_numa.h    |  4 ++++
 5 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/arch/x86/include/asm/acpi.h b/arch/x86/include/asm/acpi.h
index 94c18eb..65f1e95 100644
--- a/arch/x86/include/asm/acpi.h
+++ b/arch/x86/include/asm/acpi.h
@@ -145,7 +145,6 @@ static inline void disable_acpi(void) { }
 #define ARCH_HAS_POWER_INIT	1
 
 #ifdef CONFIG_ACPI_NUMA
-extern int acpi_numa;
 extern int x86_acpi_numa_init(void);
 #endif /* CONFIG_ACPI_NUMA */
 
diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
index f70c1ff..57673f5 100644
--- a/arch/x86/mm/numa.c
+++ b/arch/x86/mm/numa.c
@@ -1,4 +1,5 @@
 /* Common code for 32 and 64-bit NUMA */
+#include <linux/acpi.h>
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/string.h>
@@ -15,7 +16,6 @@
 #include <asm/e820.h>
 #include <asm/proto.h>
 #include <asm/dma.h>
-#include <asm/acpi.h>
 #include <asm/amd_nb.h>
 
 #include "numa_internal.h"
diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 9e2a833..4217071f 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -24,19 +24,6 @@
 #include <asm/apic.h>
 #include <asm/uv/uv.h>
 
-int acpi_numa __initdata;
-
-static __init void bad_srat(void)
-{
-	printk(KERN_ERR "SRAT: SRAT not used.\n");
-	acpi_numa = -1;
-}
-
-static __init inline int srat_disabled(void)
-{
-	return acpi_numa < 0;
-}
-
 /* Callback for Proximity Domain -> x2APIC mapping */
 void __init
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index acc361d..7174a38 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -40,6 +40,7 @@ static int node_to_pxm_map[MAX_NUMNODES]
 			= { [0 ... MAX_NUMNODES - 1] = PXM_INVAL };
 
 unsigned char acpi_srat_revision __initdata;
+int acpi_numa __initdata;
 
 int pxm_to_node(int pxm)
 {
@@ -197,6 +198,17 @@ static int __init slit_valid(struct acpi_table_slit *slit)
 	return 1;
 }
 
+void __init bad_srat(void)
+{
+	pr_err("SRAT: SRAT not used.\n");
+	acpi_numa = -1;
+}
+
+int __init srat_disabled(void)
+{
+	return acpi_numa < 0;
+}
+
 #if defined(CONFIG_X86) || defined(CONFIG_ARM64)
 /*
  * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
diff --git a/include/acpi/acpi_numa.h b/include/acpi/acpi_numa.h
index 94a37cd..d4b7294 100644
--- a/include/acpi/acpi_numa.h
+++ b/include/acpi/acpi_numa.h
@@ -15,6 +15,10 @@ extern int pxm_to_node(int);
 extern int node_to_pxm(int);
 extern int acpi_map_pxm_to_node(int);
 extern unsigned char acpi_srat_revision;
+extern int acpi_numa __initdata;
+
+extern void bad_srat(void);
+extern int srat_disabled(void);
 
 #endif				/* CONFIG_ACPI_NUMA */
 #endif				/* __ACP_NUMA_H */
-- 
1.7.11.7

[PATCH v5 08/14] acpi, numa: move bad_srat() and srat_disabled() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

bad_srat() and srat_disabled() are shared by x86 and follow-on arm64
patches.  Move them to drivers/acpi/numa.c in preparation for arm64
support.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
[david.daney@cavium.com moved definitions to drivers/acpi/numa.c]
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/include/asm/acpi.h |  1 -
 arch/x86/mm/numa.c          |  2 +-
 arch/x86/mm/srat.c          | 13 -------------
 drivers/acpi/numa.c         | 12 ++++++++++++
 include/acpi/acpi_numa.h    |  4 ++++
 5 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/arch/x86/include/asm/acpi.h b/arch/x86/include/asm/acpi.h
index 94c18eb..65f1e95 100644
--- a/arch/x86/include/asm/acpi.h
+++ b/arch/x86/include/asm/acpi.h
@@ -145,7 +145,6 @@ static inline void disable_acpi(void) { }
 #define ARCH_HAS_POWER_INIT	1
 
 #ifdef CONFIG_ACPI_NUMA
-extern int acpi_numa;
 extern int x86_acpi_numa_init(void);
 #endif /* CONFIG_ACPI_NUMA */
 
diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
index f70c1ff..57673f5 100644
--- a/arch/x86/mm/numa.c
+++ b/arch/x86/mm/numa.c
@@ -1,4 +1,5 @@
 /* Common code for 32 and 64-bit NUMA */
+#include <linux/acpi.h>
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/string.h>
@@ -15,7 +16,6 @@
 #include <asm/e820.h>
 #include <asm/proto.h>
 #include <asm/dma.h>
-#include <asm/acpi.h>
 #include <asm/amd_nb.h>
 
 #include "numa_internal.h"
diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 9e2a833..4217071f 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -24,19 +24,6 @@
 #include <asm/apic.h>
 #include <asm/uv/uv.h>
 
-int acpi_numa __initdata;
-
-static __init void bad_srat(void)
-{
-	printk(KERN_ERR "SRAT: SRAT not used.\n");
-	acpi_numa = -1;
-}
-
-static __init inline int srat_disabled(void)
-{
-	return acpi_numa < 0;
-}
-
 /* Callback for Proximity Domain -> x2APIC mapping */
 void __init
 acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index acc361d..7174a38 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -40,6 +40,7 @@ static int node_to_pxm_map[MAX_NUMNODES]
 			= { [0 ... MAX_NUMNODES - 1] = PXM_INVAL };
 
 unsigned char acpi_srat_revision __initdata;
+int acpi_numa __initdata;
 
 int pxm_to_node(int pxm)
 {
@@ -197,6 +198,17 @@ static int __init slit_valid(struct acpi_table_slit *slit)
 	return 1;
 }
 
+void __init bad_srat(void)
+{
+	pr_err("SRAT: SRAT not used.\n");
+	acpi_numa = -1;
+}
+
+int __init srat_disabled(void)
+{
+	return acpi_numa < 0;
+}
+
 #if defined(CONFIG_X86) || defined(CONFIG_ARM64)
 /*
  * Callback for SLIT parsing.  pxm_to_node() returns NUMA_NO_NODE for
diff --git a/include/acpi/acpi_numa.h b/include/acpi/acpi_numa.h
index 94a37cd..d4b7294 100644
--- a/include/acpi/acpi_numa.h
+++ b/include/acpi/acpi_numa.h
@@ -15,6 +15,10 @@ extern int pxm_to_node(int);
 extern int node_to_pxm(int);
 extern int acpi_map_pxm_to_node(int);
 extern unsigned char acpi_srat_revision;
+extern int acpi_numa __initdata;
+
+extern void bad_srat(void);
+extern int srat_disabled(void);
 
 #endif				/* CONFIG_ACPI_NUMA */
 #endif				/* __ACP_NUMA_H */
-- 
1.7.11.7

[PATCH v5 09/14] acpi, numa: remove unneeded acpi_numa=1

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

acpi_numa is default to 0, it's set to -1 when disable acpi numa or
when a bad SRAT is parsed, and it's only consumed in srat_disabled()
(compare it with 0) to continue parse the SRAT or not, so we don't
need to set acpi_numa to 1 when we get a valid SRAT entry.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 4217071f..30460f5 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -59,7 +59,6 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 	}
 	set_apicid_to_node(apic_id, node);
 	node_set(node, numa_nodes_parsed);
-	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%04x -> Node %u\n",
 	       pxm, apic_id, node);
 }
@@ -101,7 +100,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 
 	set_apicid_to_node(apic_id, node);
 	node_set(node, numa_nodes_parsed);
-	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%02x -> Node %u\n",
 	       pxm, apic_id, node);
 }
-- 
1.7.11.7

[PATCH v5 09/14] acpi, numa: remove unneeded acpi_numa=1

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

acpi_numa is default to 0, it's set to -1 when disable acpi numa or
when a bad SRAT is parsed, and it's only consumed in srat_disabled()
(compare it with 0) to continue parse the SRAT or not, so we don't
need to set acpi_numa to 1 when we get a valid SRAT entry.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 4217071f..30460f5 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -59,7 +59,6 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 	}
 	set_apicid_to_node(apic_id, node);
 	node_set(node, numa_nodes_parsed);
-	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%04x -> Node %u\n",
 	       pxm, apic_id, node);
 }
@@ -101,7 +100,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 
 	set_apicid_to_node(apic_id, node);
 	node_set(node, numa_nodes_parsed);
-	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%02x -> Node %u\n",
 	       pxm, apic_id, node);
 }
-- 
1.7.11.7

[PATCH v5 09/14] acpi, numa: remove unneeded acpi_numa=1

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

acpi_numa is default to 0, it's set to -1 when disable acpi numa or
when a bad SRAT is parsed, and it's only consumed in srat_disabled()
(compare it with 0) to continue parse the SRAT or not, so we don't
need to set acpi_numa to 1 when we get a valid SRAT entry.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 4217071f..30460f5 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -59,7 +59,6 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 	}
 	set_apicid_to_node(apic_id, node);
 	node_set(node, numa_nodes_parsed);
-	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%04x -> Node %u\n",
 	       pxm, apic_id, node);
 }
@@ -101,7 +100,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 
 	set_apicid_to_node(apic_id, node);
 	node_set(node, numa_nodes_parsed);
-	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%02x -> Node %u\n",
 	       pxm, apic_id, node);
 }
-- 
1.7.11.7

[PATCH v5 09/14] acpi, numa: remove unneeded acpi_numa=1

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

acpi_numa is default to 0, it's set to -1 when disable acpi numa or
when a bad SRAT is parsed, and it's only consumed in srat_disabled()
(compare it with 0) to continue parse the SRAT or not, so we don't
need to set acpi_numa to 1 when we get a valid SRAT entry.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 4217071f..30460f5 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -59,7 +59,6 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 	}
 	set_apicid_to_node(apic_id, node);
 	node_set(node, numa_nodes_parsed);
-	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%04x -> Node %u\n",
 	       pxm, apic_id, node);
 }
@@ -101,7 +100,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 
 	set_apicid_to_node(apic_id, node);
 	node_set(node, numa_nodes_parsed);
-	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%02x -> Node %u\n",
 	       pxm, apic_id, node);
 }
-- 
1.7.11.7

[PATCH v5 10/14] acpi, numa: Move acpi_numa_memory_affinity_init() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

acpi_numa_memory_affinity_init() will be reused by arm64.  Move it to
drivers/acpi/numa.c to facilitate reuse.

No code change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c  | 57 --------------------------------------------------
 drivers/acpi/numa.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+), 57 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 30460f5..b1ecff4 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -15,8 +15,6 @@
 #include <linux/bitmap.h>
 #include <linux/module.h>
 #include <linux/topology.h>
-#include <linux/bootmem.h>
-#include <linux/memblock.h>
 #include <linux/mm.h>
 #include <asm/proto.h>
 #include <asm/numa.h>
@@ -104,61 +102,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	       pxm, apic_id, node);
 }
 
-/* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
-int __init
-acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
-{
-	u64 start, end;
-	u32 hotpluggable;
-	int node, pxm;
-
-	if (srat_disabled())
-		goto out_err;
-	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
-		goto out_err_bad_srat;
-	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
-		goto out_err;
-	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
-	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
-		goto out_err;
-
-	start = ma->base_address;
-	end = start + ma->length;
-	pxm = ma->proximity_domain;
-	if (acpi_srat_revision <= 1)
-		pxm &= 0xff;
-
-	node = acpi_map_pxm_to_node(pxm);
-	if (node < 0) {
-		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
-		goto out_err_bad_srat;
-	}
-
-	if (numa_add_memblk(node, start, end) < 0)
-		goto out_err_bad_srat;
-
-	node_set(node, numa_nodes_parsed);
-
-	pr_info("SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]%s%s\n",
-		node, pxm,
-		(unsigned long long) start, (unsigned long long) end - 1,
-		hotpluggable ? " hotplug" : "",
-		ma->flags & ACPI_SRAT_MEM_NON_VOLATILE ? " non-volatile" : "");
-
-	/* Mark hotplug range in memblock. */
-	if (hotpluggable && memblock_mark_hotplug(start, ma->length))
-		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
-			(unsigned long long)start, (unsigned long long)end - 1);
-
-	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
-
-	return 0;
-out_err_bad_srat:
-	bad_srat();
-out_err:
-	return -EINVAL;
-}
-
 int __init x86_acpi_numa_init(void)
 {
 	int ret;
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 7174a38..8ba90ad 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -27,6 +27,8 @@
 #include <linux/types.h>
 #include <linux/errno.h>
 #include <linux/acpi.h>
+#include <linux/bootmem.h>
+#include <linux/memblock.h>
 #include <linux/numa.h>
 #include <linux/nodemask.h>
 #include <linux/topology.h>
@@ -236,6 +238,64 @@ void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
 		}
 	}
 }
+
+/*
+ * Default callback for parsing of the Proximity Domain <-> Memory
+ * Area mappings
+ */
+int __init
+acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
+{
+	u64 start, end;
+	u32 hotpluggable;
+	int node, pxm;
+
+	if (srat_disabled())
+		goto out_err;
+	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
+		goto out_err_bad_srat;
+	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
+		goto out_err;
+	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
+	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		goto out_err;
+
+	start = ma->base_address;
+	end = start + ma->length;
+	pxm = ma->proximity_domain;
+	if (acpi_srat_revision <= 1)
+		pxm &= 0xff;
+
+	node = acpi_map_pxm_to_node(pxm);
+	if (node < 0) {
+		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
+		goto out_err_bad_srat;
+	}
+
+	if (numa_add_memblk(node, start, end) < 0)
+		goto out_err_bad_srat;
+
+	node_set(node, numa_nodes_parsed);
+
+	pr_info("SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]%s%s\n",
+		node, pxm,
+		(unsigned long long) start, (unsigned long long) end - 1,
+		hotpluggable ? " hotplug" : "",
+		ma->flags & ACPI_SRAT_MEM_NON_VOLATILE ? " non-volatile" : "");
+
+	/* Mark hotplug range in memblock. */
+	if (hotpluggable && memblock_mark_hotplug(start, ma->length))
+		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
+			(unsigned long long)start, (unsigned long long)end - 1);
+
+	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
+
+	return 0;
+out_err_bad_srat:
+	bad_srat();
+out_err:
+	return -EINVAL;
+}
 #endif /* defined(CONFIG_X86) || defined (CONFIG_ARM64) */
 
 static int __init acpi_parse_slit(struct acpi_table_header *table)
-- 
1.7.11.7

[PATCH v5 10/14] acpi, numa: Move acpi_numa_memory_affinity_init() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

acpi_numa_memory_affinity_init() will be reused by arm64.  Move it to
drivers/acpi/numa.c to facilitate reuse.

No code change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c  | 57 --------------------------------------------------
 drivers/acpi/numa.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+), 57 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 30460f5..b1ecff4 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -15,8 +15,6 @@
 #include <linux/bitmap.h>
 #include <linux/module.h>
 #include <linux/topology.h>
-#include <linux/bootmem.h>
-#include <linux/memblock.h>
 #include <linux/mm.h>
 #include <asm/proto.h>
 #include <asm/numa.h>
@@ -104,61 +102,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	       pxm, apic_id, node);
 }
 
-/* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
-int __init
-acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
-{
-	u64 start, end;
-	u32 hotpluggable;
-	int node, pxm;
-
-	if (srat_disabled())
-		goto out_err;
-	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
-		goto out_err_bad_srat;
-	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
-		goto out_err;
-	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
-	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
-		goto out_err;
-
-	start = ma->base_address;
-	end = start + ma->length;
-	pxm = ma->proximity_domain;
-	if (acpi_srat_revision <= 1)
-		pxm &= 0xff;
-
-	node = acpi_map_pxm_to_node(pxm);
-	if (node < 0) {
-		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
-		goto out_err_bad_srat;
-	}
-
-	if (numa_add_memblk(node, start, end) < 0)
-		goto out_err_bad_srat;
-
-	node_set(node, numa_nodes_parsed);
-
-	pr_info("SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]%s%s\n",
-		node, pxm,
-		(unsigned long long) start, (unsigned long long) end - 1,
-		hotpluggable ? " hotplug" : "",
-		ma->flags & ACPI_SRAT_MEM_NON_VOLATILE ? " non-volatile" : "");
-
-	/* Mark hotplug range in memblock. */
-	if (hotpluggable && memblock_mark_hotplug(start, ma->length))
-		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
-			(unsigned long long)start, (unsigned long long)end - 1);
-
-	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
-
-	return 0;
-out_err_bad_srat:
-	bad_srat();
-out_err:
-	return -EINVAL;
-}
-
 int __init x86_acpi_numa_init(void)
 {
 	int ret;
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 7174a38..8ba90ad 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -27,6 +27,8 @@
 #include <linux/types.h>
 #include <linux/errno.h>
 #include <linux/acpi.h>
+#include <linux/bootmem.h>
+#include <linux/memblock.h>
 #include <linux/numa.h>
 #include <linux/nodemask.h>
 #include <linux/topology.h>
@@ -236,6 +238,64 @@ void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
 		}
 	}
 }
+
+/*
+ * Default callback for parsing of the Proximity Domain <-> Memory
+ * Area mappings
+ */
+int __init
+acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
+{
+	u64 start, end;
+	u32 hotpluggable;
+	int node, pxm;
+
+	if (srat_disabled())
+		goto out_err;
+	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
+		goto out_err_bad_srat;
+	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
+		goto out_err;
+	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
+	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		goto out_err;
+
+	start = ma->base_address;
+	end = start + ma->length;
+	pxm = ma->proximity_domain;
+	if (acpi_srat_revision <= 1)
+		pxm &= 0xff;
+
+	node = acpi_map_pxm_to_node(pxm);
+	if (node < 0) {
+		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
+		goto out_err_bad_srat;
+	}
+
+	if (numa_add_memblk(node, start, end) < 0)
+		goto out_err_bad_srat;
+
+	node_set(node, numa_nodes_parsed);
+
+	pr_info("SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]%s%s\n",
+		node, pxm,
+		(unsigned long long) start, (unsigned long long) end - 1,
+		hotpluggable ? " hotplug" : "",
+		ma->flags & ACPI_SRAT_MEM_NON_VOLATILE ? " non-volatile" : "");
+
+	/* Mark hotplug range in memblock. */
+	if (hotpluggable && memblock_mark_hotplug(start, ma->length))
+		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
+			(unsigned long long)start, (unsigned long long)end - 1);
+
+	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
+
+	return 0;
+out_err_bad_srat:
+	bad_srat();
+out_err:
+	return -EINVAL;
+}
 #endif /* defined(CONFIG_X86) || defined (CONFIG_ARM64) */
 
 static int __init acpi_parse_slit(struct acpi_table_header *table)
-- 
1.7.11.7

[PATCH v5 10/14] acpi, numa: Move acpi_numa_memory_affinity_init() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

acpi_numa_memory_affinity_init() will be reused by arm64.  Move it to
drivers/acpi/numa.c to facilitate reuse.

No code change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c  | 57 --------------------------------------------------
 drivers/acpi/numa.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+), 57 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 30460f5..b1ecff4 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -15,8 +15,6 @@
 #include <linux/bitmap.h>
 #include <linux/module.h>
 #include <linux/topology.h>
-#include <linux/bootmem.h>
-#include <linux/memblock.h>
 #include <linux/mm.h>
 #include <asm/proto.h>
 #include <asm/numa.h>
@@ -104,61 +102,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	       pxm, apic_id, node);
 }
 
-/* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
-int __init
-acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
-{
-	u64 start, end;
-	u32 hotpluggable;
-	int node, pxm;
-
-	if (srat_disabled())
-		goto out_err;
-	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
-		goto out_err_bad_srat;
-	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
-		goto out_err;
-	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
-	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
-		goto out_err;
-
-	start = ma->base_address;
-	end = start + ma->length;
-	pxm = ma->proximity_domain;
-	if (acpi_srat_revision <= 1)
-		pxm &= 0xff;
-
-	node = acpi_map_pxm_to_node(pxm);
-	if (node < 0) {
-		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
-		goto out_err_bad_srat;
-	}
-
-	if (numa_add_memblk(node, start, end) < 0)
-		goto out_err_bad_srat;
-
-	node_set(node, numa_nodes_parsed);
-
-	pr_info("SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]%s%s\n",
-		node, pxm,
-		(unsigned long long) start, (unsigned long long) end - 1,
-		hotpluggable ? " hotplug" : "",
-		ma->flags & ACPI_SRAT_MEM_NON_VOLATILE ? " non-volatile" : "");
-
-	/* Mark hotplug range in memblock. */
-	if (hotpluggable && memblock_mark_hotplug(start, ma->length))
-		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
-			(unsigned long long)start, (unsigned long long)end - 1);
-
-	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
-
-	return 0;
-out_err_bad_srat:
-	bad_srat();
-out_err:
-	return -EINVAL;
-}
-
 int __init x86_acpi_numa_init(void)
 {
 	int ret;
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 7174a38..8ba90ad 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -27,6 +27,8 @@
 #include <linux/types.h>
 #include <linux/errno.h>
 #include <linux/acpi.h>
+#include <linux/bootmem.h>
+#include <linux/memblock.h>
 #include <linux/numa.h>
 #include <linux/nodemask.h>
 #include <linux/topology.h>
@@ -236,6 +238,64 @@ void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
 		}
 	}
 }
+
+/*
+ * Default callback for parsing of the Proximity Domain <-> Memory
+ * Area mappings
+ */
+int __init
+acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
+{
+	u64 start, end;
+	u32 hotpluggable;
+	int node, pxm;
+
+	if (srat_disabled())
+		goto out_err;
+	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
+		goto out_err_bad_srat;
+	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
+		goto out_err;
+	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
+	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		goto out_err;
+
+	start = ma->base_address;
+	end = start + ma->length;
+	pxm = ma->proximity_domain;
+	if (acpi_srat_revision <= 1)
+		pxm &= 0xff;
+
+	node = acpi_map_pxm_to_node(pxm);
+	if (node < 0) {
+		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
+		goto out_err_bad_srat;
+	}
+
+	if (numa_add_memblk(node, start, end) < 0)
+		goto out_err_bad_srat;
+
+	node_set(node, numa_nodes_parsed);
+
+	pr_info("SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]%s%s\n",
+		node, pxm,
+		(unsigned long long) start, (unsigned long long) end - 1,
+		hotpluggable ? " hotplug" : "",
+		ma->flags & ACPI_SRAT_MEM_NON_VOLATILE ? " non-volatile" : "");
+
+	/* Mark hotplug range in memblock. */
+	if (hotpluggable && memblock_mark_hotplug(start, ma->length))
+		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
+			(unsigned long long)start, (unsigned long long)end - 1);
+
+	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
+
+	return 0;
+out_err_bad_srat:
+	bad_srat();
+out_err:
+	return -EINVAL;
+}
 #endif /* defined(CONFIG_X86) || defined (CONFIG_ARM64) */
 
 static int __init acpi_parse_slit(struct acpi_table_header *table)
-- 
1.7.11.7

[PATCH v5 10/14] acpi, numa: Move acpi_numa_memory_affinity_init() to drivers/acpi/numa.c

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

acpi_numa_memory_affinity_init() will be reused by arm64.  Move it to
drivers/acpi/numa.c to facilitate reuse.

No code change.

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/x86/mm/srat.c  | 57 --------------------------------------------------
 drivers/acpi/numa.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+), 57 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 30460f5..b1ecff4 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -15,8 +15,6 @@
 #include <linux/bitmap.h>
 #include <linux/module.h>
 #include <linux/topology.h>
-#include <linux/bootmem.h>
-#include <linux/memblock.h>
 #include <linux/mm.h>
 #include <asm/proto.h>
 #include <asm/numa.h>
@@ -104,61 +102,6 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	       pxm, apic_id, node);
 }
 
-/* Callback for parsing of the Proximity Domain <-> Memory Area mappings */
-int __init
-acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
-{
-	u64 start, end;
-	u32 hotpluggable;
-	int node, pxm;
-
-	if (srat_disabled())
-		goto out_err;
-	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
-		goto out_err_bad_srat;
-	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) = 0)
-		goto out_err;
-	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
-	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
-		goto out_err;
-
-	start = ma->base_address;
-	end = start + ma->length;
-	pxm = ma->proximity_domain;
-	if (acpi_srat_revision <= 1)
-		pxm &= 0xff;
-
-	node = acpi_map_pxm_to_node(pxm);
-	if (node < 0) {
-		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
-		goto out_err_bad_srat;
-	}
-
-	if (numa_add_memblk(node, start, end) < 0)
-		goto out_err_bad_srat;
-
-	node_set(node, numa_nodes_parsed);
-
-	pr_info("SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]%s%s\n",
-		node, pxm,
-		(unsigned long long) start, (unsigned long long) end - 1,
-		hotpluggable ? " hotplug" : "",
-		ma->flags & ACPI_SRAT_MEM_NON_VOLATILE ? " non-volatile" : "");
-
-	/* Mark hotplug range in memblock. */
-	if (hotpluggable && memblock_mark_hotplug(start, ma->length))
-		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
-			(unsigned long long)start, (unsigned long long)end - 1);
-
-	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
-
-	return 0;
-out_err_bad_srat:
-	bad_srat();
-out_err:
-	return -EINVAL;
-}
-
 int __init x86_acpi_numa_init(void)
 {
 	int ret;
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 7174a38..8ba90ad 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -27,6 +27,8 @@
 #include <linux/types.h>
 #include <linux/errno.h>
 #include <linux/acpi.h>
+#include <linux/bootmem.h>
+#include <linux/memblock.h>
 #include <linux/numa.h>
 #include <linux/nodemask.h>
 #include <linux/topology.h>
@@ -236,6 +238,64 @@ void __init acpi_numa_slit_init(struct acpi_table_slit *slit)
 		}
 	}
 }
+
+/*
+ * Default callback for parsing of the Proximity Domain <-> Memory
+ * Area mappings
+ */
+int __init
+acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
+{
+	u64 start, end;
+	u32 hotpluggable;
+	int node, pxm;
+
+	if (srat_disabled())
+		goto out_err;
+	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
+		goto out_err_bad_srat;
+	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) = 0)
+		goto out_err;
+	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
+	if (hotpluggable && !IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		goto out_err;
+
+	start = ma->base_address;
+	end = start + ma->length;
+	pxm = ma->proximity_domain;
+	if (acpi_srat_revision <= 1)
+		pxm &= 0xff;
+
+	node = acpi_map_pxm_to_node(pxm);
+	if (node < 0) {
+		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
+		goto out_err_bad_srat;
+	}
+
+	if (numa_add_memblk(node, start, end) < 0)
+		goto out_err_bad_srat;
+
+	node_set(node, numa_nodes_parsed);
+
+	pr_info("SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]%s%s\n",
+		node, pxm,
+		(unsigned long long) start, (unsigned long long) end - 1,
+		hotpluggable ? " hotplug" : "",
+		ma->flags & ACPI_SRAT_MEM_NON_VOLATILE ? " non-volatile" : "");
+
+	/* Mark hotplug range in memblock. */
+	if (hotpluggable && memblock_mark_hotplug(start, ma->length))
+		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
+			(unsigned long long)start, (unsigned long long)end - 1);
+
+	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
+
+	return 0;
+out_err_bad_srat:
+	bad_srat();
+out_err:
+	return -EINVAL;
+}
 #endif /* defined(CONFIG_X86) || defined (CONFIG_ARM64) */
 
 static int __init acpi_parse_slit(struct acpi_table_header *table)
-- 
1.7.11.7

[PATCH v5 11/14] acpi, numa, srat: Improve SRAT error detection and add messages.

[David Daney]

From: David Daney <david.daney@cavium.com>

Loosely based on code from Robert Richter and Hanjun Guo.

Improve out of range node detection as well as allow for Larger SRAT
entities.

Add printing of nice messages.

Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 8ba90ad..d244018 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -252,8 +252,11 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 
 	if (srat_disabled())
 		goto out_err;
-	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
+	if (ma->header.length < sizeof(struct acpi_srat_mem_affinity)) {
+		pr_err("SRAT: Unexpected header length: %d\n",
+		       ma->header.length);
 		goto out_err_bad_srat;
+	}
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
 		goto out_err;
 	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
@@ -267,13 +270,17 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 		pxm &= 0xff;
 
 	node = acpi_map_pxm_to_node(pxm);
-	if (node < 0) {
-		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
+	if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
+		pr_err("SRAT: Too many proximity domains.\n");
 		goto out_err_bad_srat;
 	}
 
-	if (numa_add_memblk(node, start, end) < 0)
+	if (numa_add_memblk(node, start, end) < 0) {
+		pr_err("SRAT: Failed to add memblk to node %u [mem %#010Lx-%#010Lx]\n",
+		       node, (unsigned long long) start,
+		       (unsigned long long) end - 1);
 		goto out_err_bad_srat;
+	}
 
 	node_set(node, numa_nodes_parsed);
 
-- 
1.7.11.7

[PATCH v5 11/14] acpi, numa, srat: Improve SRAT error detection and add messages.

[David Daney]

From: David Daney <david.daney@cavium.com>

Loosely based on code from Robert Richter and Hanjun Guo.

Improve out of range node detection as well as allow for Larger SRAT
entities.

Add printing of nice messages.

Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 8ba90ad..d244018 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -252,8 +252,11 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 
 	if (srat_disabled())
 		goto out_err;
-	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
+	if (ma->header.length < sizeof(struct acpi_srat_mem_affinity)) {
+		pr_err("SRAT: Unexpected header length: %d\n",
+		       ma->header.length);
 		goto out_err_bad_srat;
+	}
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
 		goto out_err;
 	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
@@ -267,13 +270,17 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 		pxm &= 0xff;
 
 	node = acpi_map_pxm_to_node(pxm);
-	if (node < 0) {
-		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
+	if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
+		pr_err("SRAT: Too many proximity domains.\n");
 		goto out_err_bad_srat;
 	}
 
-	if (numa_add_memblk(node, start, end) < 0)
+	if (numa_add_memblk(node, start, end) < 0) {
+		pr_err("SRAT: Failed to add memblk to node %u [mem %#010Lx-%#010Lx]\n",
+		       node, (unsigned long long) start,
+		       (unsigned long long) end - 1);
 		goto out_err_bad_srat;
+	}
 
 	node_set(node, numa_nodes_parsed);
 
-- 
1.7.11.7

[PATCH v5 11/14] acpi, numa, srat: Improve SRAT error detection and add messages.

[David Daney]

From: David Daney <david.daney@cavium.com>

Loosely based on code from Robert Richter and Hanjun Guo.

Improve out of range node detection as well as allow for Larger SRAT
entities.

Add printing of nice messages.

Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/numa.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index 8ba90ad..d244018 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -252,8 +252,11 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 
 	if (srat_disabled())
 		goto out_err;
-	if (ma->header.length != sizeof(struct acpi_srat_mem_affinity))
+	if (ma->header.length < sizeof(struct acpi_srat_mem_affinity)) {
+		pr_err("SRAT: Unexpected header length: %d\n",
+		       ma->header.length);
 		goto out_err_bad_srat;
+	}
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) = 0)
 		goto out_err;
 	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
@@ -267,13 +270,17 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 		pxm &= 0xff;
 
 	node = acpi_map_pxm_to_node(pxm);
-	if (node < 0) {
-		printk(KERN_ERR "SRAT: Too many proximity domains.\n");
+	if (node = NUMA_NO_NODE || node >= MAX_NUMNODES) {
+		pr_err("SRAT: Too many proximity domains.\n");
 		goto out_err_bad_srat;
 	}
 
-	if (numa_add_memblk(node, start, end) < 0)
+	if (numa_add_memblk(node, start, end) < 0) {
+		pr_err("SRAT: Failed to add memblk to node %u [mem %#010Lx-%#010Lx]\n",
+		       node, (unsigned long long) start,
+		       (unsigned long long) end - 1);
 		goto out_err_bad_srat;
+	}
 
 	node_set(node, numa_nodes_parsed);
 
-- 
1.7.11.7

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Introduce a new file to hold ACPI based NUMA information parsing from
SRAT and SLIT.

SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
ranges to Proximity Domain mapping.  SLIT has the information of inter
node distances(relative number for access latency).

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
[rrichter@cavium.com Reworked for numa v10 series ]
Signed-off-by: Robert Richter <rrichter@cavium.com>
[david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/include/asm/acpi.h |   8 +++
 arch/arm64/include/asm/numa.h |   2 +
 arch/arm64/kernel/Makefile    |   1 +
 arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
 arch/arm64/kernel/smp.c       |   2 +
 arch/arm64/mm/numa.c          |   5 +-
 6 files changed, 166 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/kernel/acpi_numa.c

diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
index aee323b..4b13ecd 100644
--- a/arch/arm64/include/asm/acpi.h
+++ b/arch/arm64/include/asm/acpi.h
@@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
 pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
 #endif
 
+#ifdef CONFIG_ACPI_NUMA
+int arm64_acpi_numa_init(void);
+int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
+#else
+static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
+static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
+#endif /* CONFIG_ACPI_NUMA */
+
 #endif /*_ASM_ACPI_H*/
diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
index e9b4f29..600887e 100644
--- a/arch/arm64/include/asm/numa.h
+++ b/arch/arm64/include/asm/numa.h
@@ -5,6 +5,8 @@
 
 #ifdef CONFIG_NUMA
 
+#define NR_NODE_MEMBLKS		(MAX_NUMNODES * 2)
+
 /* currently, arm64 implements flat NUMA topology */
 #define parent_node(node)	(node)
 
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 3793003..69569c6 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)			+= efi.o efi-entry.stub.o
 arm64-obj-$(CONFIG_PCI)			+= pci.o
 arm64-obj-$(CONFIG_ARMV8_DEPRECATED)	+= armv8_deprecated.o
 arm64-obj-$(CONFIG_ACPI)		+= acpi.o
+arm64-obj-$(CONFIG_ACPI_NUMA)		+= acpi_numa.o
 arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)	+= acpi_parking_protocol.o
 arm64-obj-$(CONFIG_PARAVIRT)		+= paravirt.o
 arm64-obj-$(CONFIG_RANDOMIZE_BASE)	+= kaslr.o
diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
new file mode 100644
index 0000000..fd72070
--- /dev/null
+++ b/arch/arm64/kernel/acpi_numa.c
@@ -0,0 +1,149 @@
+/*
+ * ACPI 5.1 based NUMA setup for ARM64
+ * Lots of code was borrowed from arch/x86/mm/srat.c
+ *
+ * Copyright 2004 Andi Kleen, SuSE Labs.
+ * Copyright (C) 2013-2016, Linaro Ltd.
+ *		Author: Hanjun Guo <hanjun.guo@linaro.org>
+ *
+ * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
+ *
+ * Called from acpi_numa_init while reading the SRAT and SLIT tables.
+ * Assumes all memory regions belonging to a single proximity domain
+ * are in one chunk. Holes between them will be included in the node.
+ */
+
+#define pr_fmt(fmt) "ACPI: NUMA: " fmt
+
+#include <linux/acpi.h>
+#include <linux/bitmap.h>
+#include <linux/bootmem.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/memblock.h>
+#include <linux/mmzone.h>
+#include <linux/module.h>
+#include <linux/topology.h>
+
+#include <acpi/processor.h>
+#include <asm/numa.h>
+
+static int cpus_in_srat;
+
+struct __node_cpu_hwid {
+	u32 node_id;    /* logical node containing this CPU */
+	u64 cpu_hwid;   /* MPIDR for this CPU */
+};
+
+static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
+[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
+
+int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
+{
+	int i;
+
+	for (i = 0; i < cpus_in_srat; i++) {
+		if (hwid == early_node_cpu_hwid[i].cpu_hwid)
+			return early_node_cpu_hwid[i].node_id;
+	}
+
+	return NUMA_NO_NODE;
+}
+
+static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
+{
+	unsigned long madt_end, entry;
+	struct acpi_table_madt *madt;
+	acpi_size tbl_size;
+
+	if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
+			(struct acpi_table_header **)&madt, &tbl_size)))
+		return -ENODEV;
+
+	entry = (unsigned long)madt;
+	madt_end = entry + madt->header.length;
+
+	/* Parse all entries looking for a match. */
+	entry += sizeof(struct acpi_table_madt);
+	while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
+		struct acpi_subtable_header *header =
+			(struct acpi_subtable_header *)entry;
+
+		if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
+			struct acpi_madt_generic_interrupt *gicc =
+				container_of(header,
+				struct acpi_madt_generic_interrupt, header);
+
+			if ((gicc->flags & ACPI_MADT_ENABLED) &&
+			    (gicc->uid == acpi_id)) {
+				*mpidr = gicc->arm_mpidr;
+				early_acpi_os_unmap_memory(madt, tbl_size);
+				return 0;
+			}
+		}
+		entry += header->length;
+	}
+
+	early_acpi_os_unmap_memory(madt, tbl_size);
+	return -ENODEV;
+}
+
+/* Callback for Proximity Domain -> ACPI processor UID mapping */
+void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
+{
+	int pxm, node;
+	u64 mpidr;
+
+	if (srat_disabled())
+		return;
+
+	if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
+		pr_err("SRAT: Invalid SRAT header length: %d\n",
+			pa->header.length);
+		bad_srat();
+		return;
+	}
+
+	if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
+		return;
+
+	if (cpus_in_srat >= NR_CPUS) {
+		pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
+			     NR_CPUS);
+		return;
+	}
+
+	pxm = pa->proximity_domain;
+	node = acpi_map_pxm_to_node(pxm);
+
+	if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
+		pr_err("SRAT: Too many proximity domains %d\n", pxm);
+		bad_srat();
+		return;
+	}
+
+	if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
+		pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
+			pxm, pa->acpi_processor_uid);
+		bad_srat();
+		return;
+	}
+
+	early_node_cpu_hwid[cpus_in_srat].node_id = node;
+	early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
+	node_set(node, numa_nodes_parsed);
+	cpus_in_srat++;
+	pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
+		pxm, mpidr, node, cpus_in_srat);
+}
+
+int __init arm64_acpi_numa_init(void)
+{
+	int ret;
+
+	ret = acpi_numa_init();
+	if (ret)
+		return ret;
+
+	return srat_disabled() ? -EINVAL : 0;
+}
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index bebc4c6..6c7ef8f 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
 	 */
 	acpi_set_mailbox_entry(cpu_count, processor);
 
+	early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
+
 	cpu_count++;
 }
 
diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index 6cb03f9..fc15186 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -17,6 +17,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/acpi.h>
 #include <linux/bootmem.h>
 #include <linux/memblock.h>
 #include <linux/module.h>
@@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
 void __init arm64_numa_init(void)
 {
 	if (!numa_off) {
-		if (!numa_init(of_numa_init))
+		if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
+			return;
+		if (acpi_disabled && !numa_init(of_numa_init))
 			return;
 	}
 
-- 
1.7.11.7

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Introduce a new file to hold ACPI based NUMA information parsing from
SRAT and SLIT.

SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
ranges to Proximity Domain mapping.  SLIT has the information of inter
node distances(relative number for access latency).

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
[rrichter at cavium.com Reworked for numa v10 series ]
Signed-off-by: Robert Richter <rrichter@cavium.com>
[david.daney at cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/include/asm/acpi.h |   8 +++
 arch/arm64/include/asm/numa.h |   2 +
 arch/arm64/kernel/Makefile    |   1 +
 arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
 arch/arm64/kernel/smp.c       |   2 +
 arch/arm64/mm/numa.c          |   5 +-
 6 files changed, 166 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/kernel/acpi_numa.c

diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
index aee323b..4b13ecd 100644
--- a/arch/arm64/include/asm/acpi.h
+++ b/arch/arm64/include/asm/acpi.h
@@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
 pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
 #endif
 
+#ifdef CONFIG_ACPI_NUMA
+int arm64_acpi_numa_init(void);
+int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
+#else
+static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
+static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
+#endif /* CONFIG_ACPI_NUMA */
+
 #endif /*_ASM_ACPI_H*/
diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
index e9b4f29..600887e 100644
--- a/arch/arm64/include/asm/numa.h
+++ b/arch/arm64/include/asm/numa.h
@@ -5,6 +5,8 @@
 
 #ifdef CONFIG_NUMA
 
+#define NR_NODE_MEMBLKS		(MAX_NUMNODES * 2)
+
 /* currently, arm64 implements flat NUMA topology */
 #define parent_node(node)	(node)
 
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 3793003..69569c6 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)			+= efi.o efi-entry.stub.o
 arm64-obj-$(CONFIG_PCI)			+= pci.o
 arm64-obj-$(CONFIG_ARMV8_DEPRECATED)	+= armv8_deprecated.o
 arm64-obj-$(CONFIG_ACPI)		+= acpi.o
+arm64-obj-$(CONFIG_ACPI_NUMA)		+= acpi_numa.o
 arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)	+= acpi_parking_protocol.o
 arm64-obj-$(CONFIG_PARAVIRT)		+= paravirt.o
 arm64-obj-$(CONFIG_RANDOMIZE_BASE)	+= kaslr.o
diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
new file mode 100644
index 0000000..fd72070
--- /dev/null
+++ b/arch/arm64/kernel/acpi_numa.c
@@ -0,0 +1,149 @@
+/*
+ * ACPI 5.1 based NUMA setup for ARM64
+ * Lots of code was borrowed from arch/x86/mm/srat.c
+ *
+ * Copyright 2004 Andi Kleen, SuSE Labs.
+ * Copyright (C) 2013-2016, Linaro Ltd.
+ *		Author: Hanjun Guo <hanjun.guo@linaro.org>
+ *
+ * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
+ *
+ * Called from acpi_numa_init while reading the SRAT and SLIT tables.
+ * Assumes all memory regions belonging to a single proximity domain
+ * are in one chunk. Holes between them will be included in the node.
+ */
+
+#define pr_fmt(fmt) "ACPI: NUMA: " fmt
+
+#include <linux/acpi.h>
+#include <linux/bitmap.h>
+#include <linux/bootmem.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/memblock.h>
+#include <linux/mmzone.h>
+#include <linux/module.h>
+#include <linux/topology.h>
+
+#include <acpi/processor.h>
+#include <asm/numa.h>
+
+static int cpus_in_srat;
+
+struct __node_cpu_hwid {
+	u32 node_id;    /* logical node containing this CPU */
+	u64 cpu_hwid;   /* MPIDR for this CPU */
+};
+
+static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
+[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
+
+int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
+{
+	int i;
+
+	for (i = 0; i < cpus_in_srat; i++) {
+		if (hwid == early_node_cpu_hwid[i].cpu_hwid)
+			return early_node_cpu_hwid[i].node_id;
+	}
+
+	return NUMA_NO_NODE;
+}
+
+static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
+{
+	unsigned long madt_end, entry;
+	struct acpi_table_madt *madt;
+	acpi_size tbl_size;
+
+	if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
+			(struct acpi_table_header **)&madt, &tbl_size)))
+		return -ENODEV;
+
+	entry = (unsigned long)madt;
+	madt_end = entry + madt->header.length;
+
+	/* Parse all entries looking for a match. */
+	entry += sizeof(struct acpi_table_madt);
+	while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
+		struct acpi_subtable_header *header =
+			(struct acpi_subtable_header *)entry;
+
+		if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
+			struct acpi_madt_generic_interrupt *gicc =
+				container_of(header,
+				struct acpi_madt_generic_interrupt, header);
+
+			if ((gicc->flags & ACPI_MADT_ENABLED) &&
+			    (gicc->uid == acpi_id)) {
+				*mpidr = gicc->arm_mpidr;
+				early_acpi_os_unmap_memory(madt, tbl_size);
+				return 0;
+			}
+		}
+		entry += header->length;
+	}
+
+	early_acpi_os_unmap_memory(madt, tbl_size);
+	return -ENODEV;
+}
+
+/* Callback for Proximity Domain -> ACPI processor UID mapping */
+void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
+{
+	int pxm, node;
+	u64 mpidr;
+
+	if (srat_disabled())
+		return;
+
+	if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
+		pr_err("SRAT: Invalid SRAT header length: %d\n",
+			pa->header.length);
+		bad_srat();
+		return;
+	}
+
+	if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
+		return;
+
+	if (cpus_in_srat >= NR_CPUS) {
+		pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
+			     NR_CPUS);
+		return;
+	}
+
+	pxm = pa->proximity_domain;
+	node = acpi_map_pxm_to_node(pxm);
+
+	if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
+		pr_err("SRAT: Too many proximity domains %d\n", pxm);
+		bad_srat();
+		return;
+	}
+
+	if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
+		pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
+			pxm, pa->acpi_processor_uid);
+		bad_srat();
+		return;
+	}
+
+	early_node_cpu_hwid[cpus_in_srat].node_id = node;
+	early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
+	node_set(node, numa_nodes_parsed);
+	cpus_in_srat++;
+	pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
+		pxm, mpidr, node, cpus_in_srat);
+}
+
+int __init arm64_acpi_numa_init(void)
+{
+	int ret;
+
+	ret = acpi_numa_init();
+	if (ret)
+		return ret;
+
+	return srat_disabled() ? -EINVAL : 0;
+}
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index bebc4c6..6c7ef8f 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
 	 */
 	acpi_set_mailbox_entry(cpu_count, processor);
 
+	early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
+
 	cpu_count++;
 }
 
diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index 6cb03f9..fc15186 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -17,6 +17,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/acpi.h>
 #include <linux/bootmem.h>
 #include <linux/memblock.h>
 #include <linux/module.h>
@@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
 void __init arm64_numa_init(void)
 {
 	if (!numa_off) {
-		if (!numa_init(of_numa_init))
+		if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
+			return;
+		if (acpi_disabled && !numa_init(of_numa_init))
 			return;
 	}
 
-- 
1.7.11.7

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Introduce a new file to hold ACPI based NUMA information parsing from
SRAT and SLIT.

SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
ranges to Proximity Domain mapping.  SLIT has the information of inter
node distances(relative number for access latency).

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
[rrichter@cavium.com Reworked for numa v10 series ]
Signed-off-by: Robert Richter <rrichter@cavium.com>
[david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/include/asm/acpi.h |   8 +++
 arch/arm64/include/asm/numa.h |   2 +
 arch/arm64/kernel/Makefile    |   1 +
 arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
 arch/arm64/kernel/smp.c       |   2 +
 arch/arm64/mm/numa.c          |   5 +-
 6 files changed, 166 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/kernel/acpi_numa.c

diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
index aee323b..4b13ecd 100644
--- a/arch/arm64/include/asm/acpi.h
+++ b/arch/arm64/include/asm/acpi.h
@@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
 pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
 #endif
 
+#ifdef CONFIG_ACPI_NUMA
+int arm64_acpi_numa_init(void);
+int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
+#else
+static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
+static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
+#endif /* CONFIG_ACPI_NUMA */
+
 #endif /*_ASM_ACPI_H*/
diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
index e9b4f29..600887e 100644
--- a/arch/arm64/include/asm/numa.h
+++ b/arch/arm64/include/asm/numa.h
@@ -5,6 +5,8 @@
 
 #ifdef CONFIG_NUMA
 
+#define NR_NODE_MEMBLKS		(MAX_NUMNODES * 2)
+
 /* currently, arm64 implements flat NUMA topology */
 #define parent_node(node)	(node)
 
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 3793003..69569c6 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)			+= efi.o efi-entry.stub.o
 arm64-obj-$(CONFIG_PCI)			+= pci.o
 arm64-obj-$(CONFIG_ARMV8_DEPRECATED)	+= armv8_deprecated.o
 arm64-obj-$(CONFIG_ACPI)		+= acpi.o
+arm64-obj-$(CONFIG_ACPI_NUMA)		+= acpi_numa.o
 arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)	+= acpi_parking_protocol.o
 arm64-obj-$(CONFIG_PARAVIRT)		+= paravirt.o
 arm64-obj-$(CONFIG_RANDOMIZE_BASE)	+= kaslr.o
diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
new file mode 100644
index 0000000..fd72070
--- /dev/null
+++ b/arch/arm64/kernel/acpi_numa.c
@@ -0,0 +1,149 @@
+/*
+ * ACPI 5.1 based NUMA setup for ARM64
+ * Lots of code was borrowed from arch/x86/mm/srat.c
+ *
+ * Copyright 2004 Andi Kleen, SuSE Labs.
+ * Copyright (C) 2013-2016, Linaro Ltd.
+ *		Author: Hanjun Guo <hanjun.guo@linaro.org>
+ *
+ * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
+ *
+ * Called from acpi_numa_init while reading the SRAT and SLIT tables.
+ * Assumes all memory regions belonging to a single proximity domain
+ * are in one chunk. Holes between them will be included in the node.
+ */
+
+#define pr_fmt(fmt) "ACPI: NUMA: " fmt
+
+#include <linux/acpi.h>
+#include <linux/bitmap.h>
+#include <linux/bootmem.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/memblock.h>
+#include <linux/mmzone.h>
+#include <linux/module.h>
+#include <linux/topology.h>
+
+#include <acpi/processor.h>
+#include <asm/numa.h>
+
+static int cpus_in_srat;
+
+struct __node_cpu_hwid {
+	u32 node_id;    /* logical node containing this CPU */
+	u64 cpu_hwid;   /* MPIDR for this CPU */
+};
+
+static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
+[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
+
+int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
+{
+	int i;
+
+	for (i = 0; i < cpus_in_srat; i++) {
+		if (hwid = early_node_cpu_hwid[i].cpu_hwid)
+			return early_node_cpu_hwid[i].node_id;
+	}
+
+	return NUMA_NO_NODE;
+}
+
+static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
+{
+	unsigned long madt_end, entry;
+	struct acpi_table_madt *madt;
+	acpi_size tbl_size;
+
+	if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
+			(struct acpi_table_header **)&madt, &tbl_size)))
+		return -ENODEV;
+
+	entry = (unsigned long)madt;
+	madt_end = entry + madt->header.length;
+
+	/* Parse all entries looking for a match. */
+	entry += sizeof(struct acpi_table_madt);
+	while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
+		struct acpi_subtable_header *header +			(struct acpi_subtable_header *)entry;
+
+		if (header->type = ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
+			struct acpi_madt_generic_interrupt *gicc +				container_of(header,
+				struct acpi_madt_generic_interrupt, header);
+
+			if ((gicc->flags & ACPI_MADT_ENABLED) &&
+			    (gicc->uid = acpi_id)) {
+				*mpidr = gicc->arm_mpidr;
+				early_acpi_os_unmap_memory(madt, tbl_size);
+				return 0;
+			}
+		}
+		entry += header->length;
+	}
+
+	early_acpi_os_unmap_memory(madt, tbl_size);
+	return -ENODEV;
+}
+
+/* Callback for Proximity Domain -> ACPI processor UID mapping */
+void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
+{
+	int pxm, node;
+	u64 mpidr;
+
+	if (srat_disabled())
+		return;
+
+	if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
+		pr_err("SRAT: Invalid SRAT header length: %d\n",
+			pa->header.length);
+		bad_srat();
+		return;
+	}
+
+	if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
+		return;
+
+	if (cpus_in_srat >= NR_CPUS) {
+		pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
+			     NR_CPUS);
+		return;
+	}
+
+	pxm = pa->proximity_domain;
+	node = acpi_map_pxm_to_node(pxm);
+
+	if (node = NUMA_NO_NODE || node >= MAX_NUMNODES) {
+		pr_err("SRAT: Too many proximity domains %d\n", pxm);
+		bad_srat();
+		return;
+	}
+
+	if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
+		pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
+			pxm, pa->acpi_processor_uid);
+		bad_srat();
+		return;
+	}
+
+	early_node_cpu_hwid[cpus_in_srat].node_id = node;
+	early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
+	node_set(node, numa_nodes_parsed);
+	cpus_in_srat++;
+	pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
+		pxm, mpidr, node, cpus_in_srat);
+}
+
+int __init arm64_acpi_numa_init(void)
+{
+	int ret;
+
+	ret = acpi_numa_init();
+	if (ret)
+		return ret;
+
+	return srat_disabled() ? -EINVAL : 0;
+}
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index bebc4c6..6c7ef8f 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
 	 */
 	acpi_set_mailbox_entry(cpu_count, processor);
 
+	early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
+
 	cpu_count++;
 }
 
diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index 6cb03f9..fc15186 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -17,6 +17,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/acpi.h>
 #include <linux/bootmem.h>
 #include <linux/memblock.h>
 #include <linux/module.h>
@@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
 void __init arm64_numa_init(void)
 {
 	if (!numa_off) {
-		if (!numa_init(of_numa_init))
+		if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
+			return;
+		if (acpi_disabled && !numa_init(of_numa_init))
 			return;
 	}
 
-- 
1.7.11.7

[PATCH v5 13/14] acpi, numa: Enable ACPI based NUMA on ARM64

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Add function needed for cpu to node mapping, and enable ACPI based
NUMA for ARM64 in Kconfig

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/Kconfig |  2 +-
 drivers/acpi/numa.c  | 35 +++++++++++++++++++++++++++++++++++
 include/linux/acpi.h | 17 ++++++++++++++++-
 3 files changed, 52 insertions(+), 2 deletions(-)

diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 82b96ee..5ce9f66 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -291,7 +291,7 @@ config ACPI_THERMAL
 config ACPI_NUMA
 	bool "NUMA support"
 	depends on NUMA
-	depends on (X86 || IA64)
+	depends on (X86 || IA64 || ARM64)
 	default y if IA64_GENERIC || IA64_SGI_SN2
 
 config ACPI_CUSTOM_DSDT_FILE
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index d244018..02c6dd8 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -170,6 +170,18 @@ acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 		}
 		break;
 
+	case ACPI_SRAT_TYPE_GICC_AFFINITY:
+		{
+			struct acpi_srat_gicc_affinity *p =
+			    (struct acpi_srat_gicc_affinity *)header;
+			pr_debug("SRAT Processor (acpi id[0x%04x]) in proximity domain %d %s\n",
+				 p->acpi_processor_uid,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_GICC_ENABLED) ?
+				 "enabled" : "disabled");
+		}
+		break;
+
 	default:
 		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
 			header->type);
@@ -360,6 +372,24 @@ acpi_parse_processor_affinity(struct acpi_subtable_header *header,
 	return 0;
 }
 
+static int __init
+acpi_parse_gicc_affinity(struct acpi_subtable_header *header,
+			 const unsigned long end)
+{
+	struct acpi_srat_gicc_affinity *processor_affinity;
+
+	processor_affinity = (struct acpi_srat_gicc_affinity *)header;
+	if (!processor_affinity)
+		return -EINVAL;
+
+	acpi_table_print_srat_entry(header);
+
+	/* let architecture-dependent part to do it */
+	acpi_numa_gicc_affinity_init(processor_affinity);
+
+	return 0;
+}
+
 static int __initdata parsed_numa_memblks;
 
 static int __init
@@ -404,6 +434,9 @@ int __init acpi_numa_init(void)
 {
 	int cnt = 0;
 
+	if (acpi_disabled)
+		return -EINVAL;
+
 	/*
 	 * Should not limit number with cpu num that is from NR_CPUS or nr_cpus=
 	 * SRAT cpu entries could have different order with that in MADT.
@@ -416,6 +449,8 @@ int __init acpi_numa_init(void)
 				     acpi_parse_x2apic_affinity, 0);
 		acpi_table_parse_srat(ACPI_SRAT_TYPE_CPU_AFFINITY,
 				     acpi_parse_processor_affinity, 0);
+		acpi_table_parse_srat(ACPI_SRAT_TYPE_GICC_AFFINITY,
+				      acpi_parse_gicc_affinity, 0);
 		cnt = acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
 					    acpi_parse_memory_affinity,
 					    NR_NODE_MEMBLKS);
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index ab6fd96..af34064 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -239,10 +239,25 @@ int acpi_table_parse_madt(enum acpi_madt_type id,
 int acpi_parse_mcfg (struct acpi_table_header *header);
 void acpi_table_print_madt_entry (struct acpi_subtable_header *madt);
 
-/* the following four functions are architecture-dependent */
+/* the following numa functions are architecture-dependent */
 void acpi_numa_slit_init (struct acpi_table_slit *slit);
+
+#if defined(CONFIG_X86) || defined(CONFIG_IA64)
 void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
+#else
+static inline void
+acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa) { }
+#endif
+
 void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
+
+#ifdef CONFIG_ARM64
+void acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa);
+#else
+static inline void
+acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa) { }
+#endif
+
 int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
 
 #ifndef PHYS_CPUID_INVALID
-- 
1.7.11.7

[PATCH v5 13/14] acpi, numa: Enable ACPI based NUMA on ARM64

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Add function needed for cpu to node mapping, and enable ACPI based
NUMA for ARM64 in Kconfig

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/Kconfig |  2 +-
 drivers/acpi/numa.c  | 35 +++++++++++++++++++++++++++++++++++
 include/linux/acpi.h | 17 ++++++++++++++++-
 3 files changed, 52 insertions(+), 2 deletions(-)

diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 82b96ee..5ce9f66 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -291,7 +291,7 @@ config ACPI_THERMAL
 config ACPI_NUMA
 	bool "NUMA support"
 	depends on NUMA
-	depends on (X86 || IA64)
+	depends on (X86 || IA64 || ARM64)
 	default y if IA64_GENERIC || IA64_SGI_SN2
 
 config ACPI_CUSTOM_DSDT_FILE
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index d244018..02c6dd8 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -170,6 +170,18 @@ acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 		}
 		break;
 
+	case ACPI_SRAT_TYPE_GICC_AFFINITY:
+		{
+			struct acpi_srat_gicc_affinity *p =
+			    (struct acpi_srat_gicc_affinity *)header;
+			pr_debug("SRAT Processor (acpi id[0x%04x]) in proximity domain %d %s\n",
+				 p->acpi_processor_uid,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_GICC_ENABLED) ?
+				 "enabled" : "disabled");
+		}
+		break;
+
 	default:
 		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
 			header->type);
@@ -360,6 +372,24 @@ acpi_parse_processor_affinity(struct acpi_subtable_header *header,
 	return 0;
 }
 
+static int __init
+acpi_parse_gicc_affinity(struct acpi_subtable_header *header,
+			 const unsigned long end)
+{
+	struct acpi_srat_gicc_affinity *processor_affinity;
+
+	processor_affinity = (struct acpi_srat_gicc_affinity *)header;
+	if (!processor_affinity)
+		return -EINVAL;
+
+	acpi_table_print_srat_entry(header);
+
+	/* let architecture-dependent part to do it */
+	acpi_numa_gicc_affinity_init(processor_affinity);
+
+	return 0;
+}
+
 static int __initdata parsed_numa_memblks;
 
 static int __init
@@ -404,6 +434,9 @@ int __init acpi_numa_init(void)
 {
 	int cnt = 0;
 
+	if (acpi_disabled)
+		return -EINVAL;
+
 	/*
 	 * Should not limit number with cpu num that is from NR_CPUS or nr_cpus=
 	 * SRAT cpu entries could have different order with that in MADT.
@@ -416,6 +449,8 @@ int __init acpi_numa_init(void)
 				     acpi_parse_x2apic_affinity, 0);
 		acpi_table_parse_srat(ACPI_SRAT_TYPE_CPU_AFFINITY,
 				     acpi_parse_processor_affinity, 0);
+		acpi_table_parse_srat(ACPI_SRAT_TYPE_GICC_AFFINITY,
+				      acpi_parse_gicc_affinity, 0);
 		cnt = acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
 					    acpi_parse_memory_affinity,
 					    NR_NODE_MEMBLKS);
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index ab6fd96..af34064 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -239,10 +239,25 @@ int acpi_table_parse_madt(enum acpi_madt_type id,
 int acpi_parse_mcfg (struct acpi_table_header *header);
 void acpi_table_print_madt_entry (struct acpi_subtable_header *madt);
 
-/* the following four functions are architecture-dependent */
+/* the following numa functions are architecture-dependent */
 void acpi_numa_slit_init (struct acpi_table_slit *slit);
+
+#if defined(CONFIG_X86) || defined(CONFIG_IA64)
 void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
+#else
+static inline void
+acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa) { }
+#endif
+
 void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
+
+#ifdef CONFIG_ARM64
+void acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa);
+#else
+static inline void
+acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa) { }
+#endif
+
 int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
 
 #ifndef PHYS_CPUID_INVALID
-- 
1.7.11.7

[PATCH v5 13/14] acpi, numa: Enable ACPI based NUMA on ARM64

[David Daney]

From: Hanjun Guo <hanjun.guo@linaro.org>

Add function needed for cpu to node mapping, and enable ACPI based
NUMA for ARM64 in Kconfig

Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/Kconfig |  2 +-
 drivers/acpi/numa.c  | 35 +++++++++++++++++++++++++++++++++++
 include/linux/acpi.h | 17 ++++++++++++++++-
 3 files changed, 52 insertions(+), 2 deletions(-)

diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 82b96ee..5ce9f66 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -291,7 +291,7 @@ config ACPI_THERMAL
 config ACPI_NUMA
 	bool "NUMA support"
 	depends on NUMA
-	depends on (X86 || IA64)
+	depends on (X86 || IA64 || ARM64)
 	default y if IA64_GENERIC || IA64_SGI_SN2
 
 config ACPI_CUSTOM_DSDT_FILE
diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
index d244018..02c6dd8 100644
--- a/drivers/acpi/numa.c
+++ b/drivers/acpi/numa.c
@@ -170,6 +170,18 @@ acpi_table_print_srat_entry(struct acpi_subtable_header *header)
 		}
 		break;
 
+	case ACPI_SRAT_TYPE_GICC_AFFINITY:
+		{
+			struct acpi_srat_gicc_affinity *p +			    (struct acpi_srat_gicc_affinity *)header;
+			pr_debug("SRAT Processor (acpi id[0x%04x]) in proximity domain %d %s\n",
+				 p->acpi_processor_uid,
+				 p->proximity_domain,
+				 (p->flags & ACPI_SRAT_GICC_ENABLED) ?
+				 "enabled" : "disabled");
+		}
+		break;
+
 	default:
 		pr_warn("Found unsupported SRAT entry (type = 0x%x)\n",
 			header->type);
@@ -360,6 +372,24 @@ acpi_parse_processor_affinity(struct acpi_subtable_header *header,
 	return 0;
 }
 
+static int __init
+acpi_parse_gicc_affinity(struct acpi_subtable_header *header,
+			 const unsigned long end)
+{
+	struct acpi_srat_gicc_affinity *processor_affinity;
+
+	processor_affinity = (struct acpi_srat_gicc_affinity *)header;
+	if (!processor_affinity)
+		return -EINVAL;
+
+	acpi_table_print_srat_entry(header);
+
+	/* let architecture-dependent part to do it */
+	acpi_numa_gicc_affinity_init(processor_affinity);
+
+	return 0;
+}
+
 static int __initdata parsed_numa_memblks;
 
 static int __init
@@ -404,6 +434,9 @@ int __init acpi_numa_init(void)
 {
 	int cnt = 0;
 
+	if (acpi_disabled)
+		return -EINVAL;
+
 	/*
 	 * Should not limit number with cpu num that is from NR_CPUS or nr_cpus 	 * SRAT cpu entries could have different order with that in MADT.
@@ -416,6 +449,8 @@ int __init acpi_numa_init(void)
 				     acpi_parse_x2apic_affinity, 0);
 		acpi_table_parse_srat(ACPI_SRAT_TYPE_CPU_AFFINITY,
 				     acpi_parse_processor_affinity, 0);
+		acpi_table_parse_srat(ACPI_SRAT_TYPE_GICC_AFFINITY,
+				      acpi_parse_gicc_affinity, 0);
 		cnt = acpi_table_parse_srat(ACPI_SRAT_TYPE_MEMORY_AFFINITY,
 					    acpi_parse_memory_affinity,
 					    NR_NODE_MEMBLKS);
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index ab6fd96..af34064 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -239,10 +239,25 @@ int acpi_table_parse_madt(enum acpi_madt_type id,
 int acpi_parse_mcfg (struct acpi_table_header *header);
 void acpi_table_print_madt_entry (struct acpi_subtable_header *madt);
 
-/* the following four functions are architecture-dependent */
+/* the following numa functions are architecture-dependent */
 void acpi_numa_slit_init (struct acpi_table_slit *slit);
+
+#if defined(CONFIG_X86) || defined(CONFIG_IA64)
 void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
+#else
+static inline void
+acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa) { }
+#endif
+
 void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
+
+#ifdef CONFIG_ARM64
+void acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa);
+#else
+static inline void
+acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa) { }
+#endif
+
 int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
 
 #ifndef PHYS_CPUID_INVALID
-- 
1.7.11.7

[PATCH v5 14/14] arm64, acpi, numa: Default enable ACPI_NUMA with NUMA

[David Daney]

From: Robert Richter <rrichter@cavium.com>

Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 5ce9f66..bacf245 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -292,7 +292,7 @@ config ACPI_NUMA
 	bool "NUMA support"
 	depends on NUMA
 	depends on (X86 || IA64 || ARM64)
-	default y if IA64_GENERIC || IA64_SGI_SN2
+	default y if IA64_GENERIC || IA64_SGI_SN2 || ARM64
 
 config ACPI_CUSTOM_DSDT_FILE
 	string "Custom DSDT Table file to include"
-- 
1.7.11.7

[PATCH v5 14/14] arm64, acpi, numa: Default enable ACPI_NUMA with NUMA

[David Daney]

From: Robert Richter <rrichter@cavium.com>

Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 5ce9f66..bacf245 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -292,7 +292,7 @@ config ACPI_NUMA
 	bool "NUMA support"
 	depends on NUMA
 	depends on (X86 || IA64 || ARM64)
-	default y if IA64_GENERIC || IA64_SGI_SN2
+	default y if IA64_GENERIC || IA64_SGI_SN2 || ARM64
 
 config ACPI_CUSTOM_DSDT_FILE
 	string "Custom DSDT Table file to include"
-- 
1.7.11.7

[PATCH v5 14/14] arm64, acpi, numa: Default enable ACPI_NUMA with NUMA

[David Daney]

From: Robert Richter <rrichter@cavium.com>

Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/acpi/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 5ce9f66..bacf245 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -292,7 +292,7 @@ config ACPI_NUMA
 	bool "NUMA support"
 	depends on NUMA
 	depends on (X86 || IA64 || ARM64)
-	default y if IA64_GENERIC || IA64_SGI_SN2
+	default y if IA64_GENERIC || IA64_SGI_SN2 || ARM64
 
 config ACPI_CUSTOM_DSDT_FILE
 	string "Custom DSDT Table file to include"
-- 
1.7.11.7

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> From: Hanjun Guo <hanjun.guo@linaro.org>
>
> Introduce a new file to hold ACPI based NUMA information parsing from
> SRAT and SLIT.
>
> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
> ranges to Proximity Domain mapping.  SLIT has the information of inter
> node distances(relative number for access latency).
>
> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> [rrichter@cavium.com Reworked for numa v10 series ]
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> [david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>  arch/arm64/include/asm/acpi.h |   8 +++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |   5 +-
>  6 files changed, 166 insertions(+), 1 deletion(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>
> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
> index aee323b..4b13ecd 100644
> --- a/arch/arm64/include/asm/acpi.h
> +++ b/arch/arm64/include/asm/acpi.h
> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>  #endif
>
> +#ifdef CONFIG_ACPI_NUMA
> +int arm64_acpi_numa_init(void);
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
> +#else
> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
> +#endif /* CONFIG_ACPI_NUMA */
> +
>  #endif /*_ASM_ACPI_H*/
> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
> index e9b4f29..600887e 100644
> --- a/arch/arm64/include/asm/numa.h
> +++ b/arch/arm64/include/asm/numa.h
> @@ -5,6 +5,8 @@
>
>  #ifdef CONFIG_NUMA
>
> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
> +
>  /* currently, arm64 implements flat NUMA topology */
>  #define parent_node(node)      (node)
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index 3793003..69569c6 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
> new file mode 100644
> index 0000000..fd72070
> --- /dev/null
> +++ b/arch/arm64/kernel/acpi_numa.c
> @@ -0,0 +1,149 @@
> +/*
> + * ACPI 5.1 based NUMA setup for ARM64
> + * Lots of code was borrowed from arch/x86/mm/srat.c
> + *
> + * Copyright 2004 Andi Kleen, SuSE Labs.
> + * Copyright (C) 2013-2016, Linaro Ltd.
> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
> + *
> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
> + *
> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
> + * Assumes all memory regions belonging to a single proximity domain
> + * are in one chunk. Holes between them will be included in the node.
> + */
> +
> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
> +
> +#include <linux/acpi.h>
> +#include <linux/bitmap.h>
> +#include <linux/bootmem.h>
> +#include <linux/kernel.h>
> +#include <linux/mm.h>
> +#include <linux/memblock.h>
> +#include <linux/mmzone.h>
> +#include <linux/module.h>
> +#include <linux/topology.h>
> +
> +#include <acpi/processor.h>
> +#include <asm/numa.h>
> +
> +static int cpus_in_srat;
> +
> +struct __node_cpu_hwid {
> +       u32 node_id;    /* logical node containing this CPU */
> +       u64 cpu_hwid;   /* MPIDR for this CPU */
> +};
> +
> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
> +
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
> +{
> +       int i;
> +
> +       for (i = 0; i < cpus_in_srat; i++) {
> +               if (hwid == early_node_cpu_hwid[i].cpu_hwid)
> +                       return early_node_cpu_hwid[i].node_id;
> +       }
> +
> +       return NUMA_NO_NODE;
> +}
> +
> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
> +{
> +       unsigned long madt_end, entry;
> +       struct acpi_table_madt *madt;
> +       acpi_size tbl_size;
> +
> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
> +                       (struct acpi_table_header **)&madt, &tbl_size)))
> +               return -ENODEV;
> +
> +       entry = (unsigned long)madt;
> +       madt_end = entry + madt->header.length;
> +
> +       /* Parse all entries looking for a match. */
> +       entry += sizeof(struct acpi_table_madt);
> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
> +               struct acpi_subtable_header *header =
> +                       (struct acpi_subtable_header *)entry;
> +
> +               if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
> +                       struct acpi_madt_generic_interrupt *gicc =
> +                               container_of(header,
> +                               struct acpi_madt_generic_interrupt, header);
> +
> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
> +                           (gicc->uid == acpi_id)) {
> +                               *mpidr = gicc->arm_mpidr;
> +                               early_acpi_os_unmap_memory(madt, tbl_size);
> +                               return 0;
> +                       }
> +               }
> +               entry += header->length;
> +       }
> +
> +       early_acpi_os_unmap_memory(madt, tbl_size);
> +       return -ENODEV;
> +}
> +
> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
> +{
> +       int pxm, node;
> +       u64 mpidr;
> +
> +       if (srat_disabled())
> +               return;
> +
> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
> +                       pa->header.length);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
> +               return;
> +
> +       if (cpus_in_srat >= NR_CPUS) {
> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
> +                            NR_CPUS);
> +               return;
> +       }
> +
> +       pxm = pa->proximity_domain;
> +       node = acpi_map_pxm_to_node(pxm);
> +
> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
> +                       pxm, pa->acpi_processor_uid);
> +               bad_srat();
> +               return;
> +       }
> +
> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
> +       node_set(node, numa_nodes_parsed);
> +       cpus_in_srat++;
> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
> +               pxm, mpidr, node, cpus_in_srat);
> +}
> +
> +int __init arm64_acpi_numa_init(void)
> +{
> +       int ret;
> +
> +       ret = acpi_numa_init();
> +       if (ret)
> +               return ret;
> +
> +       return srat_disabled() ? -EINVAL : 0;
> +}
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index bebc4c6..6c7ef8f 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>          */
>         acpi_set_mailbox_entry(cpu_count, processor);
>
> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
> +
>         cpu_count++;
>  }
>
> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> index 6cb03f9..fc15186 100644
> --- a/arch/arm64/mm/numa.c
> +++ b/arch/arm64/mm/numa.c
> @@ -17,6 +17,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>
> +#include <linux/acpi.h>
>  #include <linux/bootmem.h>
>  #include <linux/memblock.h>
>  #include <linux/module.h>
> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>  void __init arm64_numa_init(void)
>  {
>         if (!numa_off) {
> -               if (!numa_init(of_numa_init))
> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
> +                       return;
> +               if (acpi_disabled && !numa_init(of_numa_init))
>                         return;
>         }
>
On top of the latest version of the dt-based numa patch, if 'numa=off'
specified in the command line,
this function will fallback to invoke dummy_numa_init(), which give
rise the question here is, do we need to
touch any ACPI related stuff in the case? If not, then the output
message "No NUMA configuration found" followed
seems is not necessary since it's a little bit confusing in case of
numa has already been turned off explicitly.

Thanks,
Dennis

> --
> 1.7.11.7
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> From: Hanjun Guo <hanjun.guo@linaro.org>
>
> Introduce a new file to hold ACPI based NUMA information parsing from
> SRAT and SLIT.
>
> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
> ranges to Proximity Domain mapping.  SLIT has the information of inter
> node distances(relative number for access latency).
>
> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> [rrichter@cavium.com Reworked for numa v10 series ]
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> [david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>  arch/arm64/include/asm/acpi.h |   8 +++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |   5 +-
>  6 files changed, 166 insertions(+), 1 deletion(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>
> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
> index aee323b..4b13ecd 100644
> --- a/arch/arm64/include/asm/acpi.h
> +++ b/arch/arm64/include/asm/acpi.h
> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>  #endif
>
> +#ifdef CONFIG_ACPI_NUMA
> +int arm64_acpi_numa_init(void);
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
> +#else
> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
> +#endif /* CONFIG_ACPI_NUMA */
> +
>  #endif /*_ASM_ACPI_H*/
> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
> index e9b4f29..600887e 100644
> --- a/arch/arm64/include/asm/numa.h
> +++ b/arch/arm64/include/asm/numa.h
> @@ -5,6 +5,8 @@
>
>  #ifdef CONFIG_NUMA
>
> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
> +
>  /* currently, arm64 implements flat NUMA topology */
>  #define parent_node(node)      (node)
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index 3793003..69569c6 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
> new file mode 100644
> index 0000000..fd72070
> --- /dev/null
> +++ b/arch/arm64/kernel/acpi_numa.c
> @@ -0,0 +1,149 @@
> +/*
> + * ACPI 5.1 based NUMA setup for ARM64
> + * Lots of code was borrowed from arch/x86/mm/srat.c
> + *
> + * Copyright 2004 Andi Kleen, SuSE Labs.
> + * Copyright (C) 2013-2016, Linaro Ltd.
> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
> + *
> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
> + *
> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
> + * Assumes all memory regions belonging to a single proximity domain
> + * are in one chunk. Holes between them will be included in the node.
> + */
> +
> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
> +
> +#include <linux/acpi.h>
> +#include <linux/bitmap.h>
> +#include <linux/bootmem.h>
> +#include <linux/kernel.h>
> +#include <linux/mm.h>
> +#include <linux/memblock.h>
> +#include <linux/mmzone.h>
> +#include <linux/module.h>
> +#include <linux/topology.h>
> +
> +#include <acpi/processor.h>
> +#include <asm/numa.h>
> +
> +static int cpus_in_srat;
> +
> +struct __node_cpu_hwid {
> +       u32 node_id;    /* logical node containing this CPU */
> +       u64 cpu_hwid;   /* MPIDR for this CPU */
> +};
> +
> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
> +
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
> +{
> +       int i;
> +
> +       for (i = 0; i < cpus_in_srat; i++) {
> +               if (hwid == early_node_cpu_hwid[i].cpu_hwid)
> +                       return early_node_cpu_hwid[i].node_id;
> +       }
> +
> +       return NUMA_NO_NODE;
> +}
> +
> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
> +{
> +       unsigned long madt_end, entry;
> +       struct acpi_table_madt *madt;
> +       acpi_size tbl_size;
> +
> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
> +                       (struct acpi_table_header **)&madt, &tbl_size)))
> +               return -ENODEV;
> +
> +       entry = (unsigned long)madt;
> +       madt_end = entry + madt->header.length;
> +
> +       /* Parse all entries looking for a match. */
> +       entry += sizeof(struct acpi_table_madt);
> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
> +               struct acpi_subtable_header *header =
> +                       (struct acpi_subtable_header *)entry;
> +
> +               if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
> +                       struct acpi_madt_generic_interrupt *gicc =
> +                               container_of(header,
> +                               struct acpi_madt_generic_interrupt, header);
> +
> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
> +                           (gicc->uid == acpi_id)) {
> +                               *mpidr = gicc->arm_mpidr;
> +                               early_acpi_os_unmap_memory(madt, tbl_size);
> +                               return 0;
> +                       }
> +               }
> +               entry += header->length;
> +       }
> +
> +       early_acpi_os_unmap_memory(madt, tbl_size);
> +       return -ENODEV;
> +}
> +
> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
> +{
> +       int pxm, node;
> +       u64 mpidr;
> +
> +       if (srat_disabled())
> +               return;
> +
> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
> +                       pa->header.length);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
> +               return;
> +
> +       if (cpus_in_srat >= NR_CPUS) {
> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
> +                            NR_CPUS);
> +               return;
> +       }
> +
> +       pxm = pa->proximity_domain;
> +       node = acpi_map_pxm_to_node(pxm);
> +
> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
> +                       pxm, pa->acpi_processor_uid);
> +               bad_srat();
> +               return;
> +       }
> +
> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
> +       node_set(node, numa_nodes_parsed);
> +       cpus_in_srat++;
> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
> +               pxm, mpidr, node, cpus_in_srat);
> +}
> +
> +int __init arm64_acpi_numa_init(void)
> +{
> +       int ret;
> +
> +       ret = acpi_numa_init();
> +       if (ret)
> +               return ret;
> +
> +       return srat_disabled() ? -EINVAL : 0;
> +}
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index bebc4c6..6c7ef8f 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>          */
>         acpi_set_mailbox_entry(cpu_count, processor);
>
> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
> +
>         cpu_count++;
>  }
>
> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> index 6cb03f9..fc15186 100644
> --- a/arch/arm64/mm/numa.c
> +++ b/arch/arm64/mm/numa.c
> @@ -17,6 +17,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>
> +#include <linux/acpi.h>
>  #include <linux/bootmem.h>
>  #include <linux/memblock.h>
>  #include <linux/module.h>
> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>  void __init arm64_numa_init(void)
>  {
>         if (!numa_off) {
> -               if (!numa_init(of_numa_init))
> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
> +                       return;
> +               if (acpi_disabled && !numa_init(of_numa_init))
>                         return;
>         }
>
On top of the latest version of the dt-based numa patch, if 'numa=off'
specified in the command line,
this function will fallback to invoke dummy_numa_init(), which give
rise the question here is, do we need to
touch any ACPI related stuff in the case? If not, then the output
message "No NUMA configuration found" followed
seems is not necessary since it's a little bit confusing in case of
numa has already been turned off explicitly.

Thanks,
Dennis

> --
> 1.7.11.7
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> From: Hanjun Guo <hanjun.guo@linaro.org>
>
> Introduce a new file to hold ACPI based NUMA information parsing from
> SRAT and SLIT.
>
> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
> ranges to Proximity Domain mapping.  SLIT has the information of inter
> node distances(relative number for access latency).
>
> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> [rrichter at cavium.com Reworked for numa v10 series ]
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> [david.daney at cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>  arch/arm64/include/asm/acpi.h |   8 +++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |   5 +-
>  6 files changed, 166 insertions(+), 1 deletion(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>
> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
> index aee323b..4b13ecd 100644
> --- a/arch/arm64/include/asm/acpi.h
> +++ b/arch/arm64/include/asm/acpi.h
> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>  #endif
>
> +#ifdef CONFIG_ACPI_NUMA
> +int arm64_acpi_numa_init(void);
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
> +#else
> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
> +#endif /* CONFIG_ACPI_NUMA */
> +
>  #endif /*_ASM_ACPI_H*/
> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
> index e9b4f29..600887e 100644
> --- a/arch/arm64/include/asm/numa.h
> +++ b/arch/arm64/include/asm/numa.h
> @@ -5,6 +5,8 @@
>
>  #ifdef CONFIG_NUMA
>
> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
> +
>  /* currently, arm64 implements flat NUMA topology */
>  #define parent_node(node)      (node)
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index 3793003..69569c6 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
> new file mode 100644
> index 0000000..fd72070
> --- /dev/null
> +++ b/arch/arm64/kernel/acpi_numa.c
> @@ -0,0 +1,149 @@
> +/*
> + * ACPI 5.1 based NUMA setup for ARM64
> + * Lots of code was borrowed from arch/x86/mm/srat.c
> + *
> + * Copyright 2004 Andi Kleen, SuSE Labs.
> + * Copyright (C) 2013-2016, Linaro Ltd.
> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
> + *
> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
> + *
> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
> + * Assumes all memory regions belonging to a single proximity domain
> + * are in one chunk. Holes between them will be included in the node.
> + */
> +
> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
> +
> +#include <linux/acpi.h>
> +#include <linux/bitmap.h>
> +#include <linux/bootmem.h>
> +#include <linux/kernel.h>
> +#include <linux/mm.h>
> +#include <linux/memblock.h>
> +#include <linux/mmzone.h>
> +#include <linux/module.h>
> +#include <linux/topology.h>
> +
> +#include <acpi/processor.h>
> +#include <asm/numa.h>
> +
> +static int cpus_in_srat;
> +
> +struct __node_cpu_hwid {
> +       u32 node_id;    /* logical node containing this CPU */
> +       u64 cpu_hwid;   /* MPIDR for this CPU */
> +};
> +
> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
> +
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
> +{
> +       int i;
> +
> +       for (i = 0; i < cpus_in_srat; i++) {
> +               if (hwid == early_node_cpu_hwid[i].cpu_hwid)
> +                       return early_node_cpu_hwid[i].node_id;
> +       }
> +
> +       return NUMA_NO_NODE;
> +}
> +
> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
> +{
> +       unsigned long madt_end, entry;
> +       struct acpi_table_madt *madt;
> +       acpi_size tbl_size;
> +
> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
> +                       (struct acpi_table_header **)&madt, &tbl_size)))
> +               return -ENODEV;
> +
> +       entry = (unsigned long)madt;
> +       madt_end = entry + madt->header.length;
> +
> +       /* Parse all entries looking for a match. */
> +       entry += sizeof(struct acpi_table_madt);
> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
> +               struct acpi_subtable_header *header =
> +                       (struct acpi_subtable_header *)entry;
> +
> +               if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
> +                       struct acpi_madt_generic_interrupt *gicc =
> +                               container_of(header,
> +                               struct acpi_madt_generic_interrupt, header);
> +
> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
> +                           (gicc->uid == acpi_id)) {
> +                               *mpidr = gicc->arm_mpidr;
> +                               early_acpi_os_unmap_memory(madt, tbl_size);
> +                               return 0;
> +                       }
> +               }
> +               entry += header->length;
> +       }
> +
> +       early_acpi_os_unmap_memory(madt, tbl_size);
> +       return -ENODEV;
> +}
> +
> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
> +{
> +       int pxm, node;
> +       u64 mpidr;
> +
> +       if (srat_disabled())
> +               return;
> +
> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
> +                       pa->header.length);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
> +               return;
> +
> +       if (cpus_in_srat >= NR_CPUS) {
> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
> +                            NR_CPUS);
> +               return;
> +       }
> +
> +       pxm = pa->proximity_domain;
> +       node = acpi_map_pxm_to_node(pxm);
> +
> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
> +                       pxm, pa->acpi_processor_uid);
> +               bad_srat();
> +               return;
> +       }
> +
> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
> +       node_set(node, numa_nodes_parsed);
> +       cpus_in_srat++;
> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
> +               pxm, mpidr, node, cpus_in_srat);
> +}
> +
> +int __init arm64_acpi_numa_init(void)
> +{
> +       int ret;
> +
> +       ret = acpi_numa_init();
> +       if (ret)
> +               return ret;
> +
> +       return srat_disabled() ? -EINVAL : 0;
> +}
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index bebc4c6..6c7ef8f 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>          */
>         acpi_set_mailbox_entry(cpu_count, processor);
>
> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
> +
>         cpu_count++;
>  }
>
> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> index 6cb03f9..fc15186 100644
> --- a/arch/arm64/mm/numa.c
> +++ b/arch/arm64/mm/numa.c
> @@ -17,6 +17,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>
> +#include <linux/acpi.h>
>  #include <linux/bootmem.h>
>  #include <linux/memblock.h>
>  #include <linux/module.h>
> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>  void __init arm64_numa_init(void)
>  {
>         if (!numa_off) {
> -               if (!numa_init(of_numa_init))
> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
> +                       return;
> +               if (acpi_disabled && !numa_init(of_numa_init))
>                         return;
>         }
>
On top of the latest version of the dt-based numa patch, if 'numa=off'
specified in the command line,
this function will fallback to invoke dummy_numa_init(), which give
rise the question here is, do we need to
touch any ACPI related stuff in the case? If not, then the output
message "No NUMA configuration found" followed
seems is not necessary since it's a little bit confusing in case of
numa has already been turned off explicitly.

Thanks,
Dennis

> --
> 1.7.11.7
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> From: Hanjun Guo <hanjun.guo@linaro.org>
>
> Introduce a new file to hold ACPI based NUMA information parsing from
> SRAT and SLIT.
>
> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
> ranges to Proximity Domain mapping.  SLIT has the information of inter
> node distances(relative number for access latency).
>
> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> [rrichter@cavium.com Reworked for numa v10 series ]
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> [david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>  arch/arm64/include/asm/acpi.h |   8 +++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |   5 +-
>  6 files changed, 166 insertions(+), 1 deletion(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>
> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
> index aee323b..4b13ecd 100644
> --- a/arch/arm64/include/asm/acpi.h
> +++ b/arch/arm64/include/asm/acpi.h
> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>  #endif
>
> +#ifdef CONFIG_ACPI_NUMA
> +int arm64_acpi_numa_init(void);
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
> +#else
> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
> +#endif /* CONFIG_ACPI_NUMA */
> +
>  #endif /*_ASM_ACPI_H*/
> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
> index e9b4f29..600887e 100644
> --- a/arch/arm64/include/asm/numa.h
> +++ b/arch/arm64/include/asm/numa.h
> @@ -5,6 +5,8 @@
>
>  #ifdef CONFIG_NUMA
>
> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
> +
>  /* currently, arm64 implements flat NUMA topology */
>  #define parent_node(node)      (node)
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index 3793003..69569c6 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
> new file mode 100644
> index 0000000..fd72070
> --- /dev/null
> +++ b/arch/arm64/kernel/acpi_numa.c
> @@ -0,0 +1,149 @@
> +/*
> + * ACPI 5.1 based NUMA setup for ARM64
> + * Lots of code was borrowed from arch/x86/mm/srat.c
> + *
> + * Copyright 2004 Andi Kleen, SuSE Labs.
> + * Copyright (C) 2013-2016, Linaro Ltd.
> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
> + *
> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
> + *
> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
> + * Assumes all memory regions belonging to a single proximity domain
> + * are in one chunk. Holes between them will be included in the node.
> + */
> +
> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
> +
> +#include <linux/acpi.h>
> +#include <linux/bitmap.h>
> +#include <linux/bootmem.h>
> +#include <linux/kernel.h>
> +#include <linux/mm.h>
> +#include <linux/memblock.h>
> +#include <linux/mmzone.h>
> +#include <linux/module.h>
> +#include <linux/topology.h>
> +
> +#include <acpi/processor.h>
> +#include <asm/numa.h>
> +
> +static int cpus_in_srat;
> +
> +struct __node_cpu_hwid {
> +       u32 node_id;    /* logical node containing this CPU */
> +       u64 cpu_hwid;   /* MPIDR for this CPU */
> +};
> +
> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
> +
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
> +{
> +       int i;
> +
> +       for (i = 0; i < cpus_in_srat; i++) {
> +               if (hwid = early_node_cpu_hwid[i].cpu_hwid)
> +                       return early_node_cpu_hwid[i].node_id;
> +       }
> +
> +       return NUMA_NO_NODE;
> +}
> +
> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
> +{
> +       unsigned long madt_end, entry;
> +       struct acpi_table_madt *madt;
> +       acpi_size tbl_size;
> +
> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
> +                       (struct acpi_table_header **)&madt, &tbl_size)))
> +               return -ENODEV;
> +
> +       entry = (unsigned long)madt;
> +       madt_end = entry + madt->header.length;
> +
> +       /* Parse all entries looking for a match. */
> +       entry += sizeof(struct acpi_table_madt);
> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
> +               struct acpi_subtable_header *header > +                       (struct acpi_subtable_header *)entry;
> +
> +               if (header->type = ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
> +                       struct acpi_madt_generic_interrupt *gicc > +                               container_of(header,
> +                               struct acpi_madt_generic_interrupt, header);
> +
> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
> +                           (gicc->uid = acpi_id)) {
> +                               *mpidr = gicc->arm_mpidr;
> +                               early_acpi_os_unmap_memory(madt, tbl_size);
> +                               return 0;
> +                       }
> +               }
> +               entry += header->length;
> +       }
> +
> +       early_acpi_os_unmap_memory(madt, tbl_size);
> +       return -ENODEV;
> +}
> +
> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
> +{
> +       int pxm, node;
> +       u64 mpidr;
> +
> +       if (srat_disabled())
> +               return;
> +
> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
> +                       pa->header.length);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
> +               return;
> +
> +       if (cpus_in_srat >= NR_CPUS) {
> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
> +                            NR_CPUS);
> +               return;
> +       }
> +
> +       pxm = pa->proximity_domain;
> +       node = acpi_map_pxm_to_node(pxm);
> +
> +       if (node = NUMA_NO_NODE || node >= MAX_NUMNODES) {
> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
> +                       pxm, pa->acpi_processor_uid);
> +               bad_srat();
> +               return;
> +       }
> +
> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
> +       node_set(node, numa_nodes_parsed);
> +       cpus_in_srat++;
> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
> +               pxm, mpidr, node, cpus_in_srat);
> +}
> +
> +int __init arm64_acpi_numa_init(void)
> +{
> +       int ret;
> +
> +       ret = acpi_numa_init();
> +       if (ret)
> +               return ret;
> +
> +       return srat_disabled() ? -EINVAL : 0;
> +}
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index bebc4c6..6c7ef8f 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>          */
>         acpi_set_mailbox_entry(cpu_count, processor);
>
> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
> +
>         cpu_count++;
>  }
>
> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> index 6cb03f9..fc15186 100644
> --- a/arch/arm64/mm/numa.c
> +++ b/arch/arm64/mm/numa.c
> @@ -17,6 +17,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>
> +#include <linux/acpi.h>
>  #include <linux/bootmem.h>
>  #include <linux/memblock.h>
>  #include <linux/module.h>
> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>  void __init arm64_numa_init(void)
>  {
>         if (!numa_off) {
> -               if (!numa_init(of_numa_init))
> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
> +                       return;
> +               if (acpi_disabled && !numa_init(of_numa_init))
>                         return;
>         }
>
On top of the latest version of the dt-based numa patch, if 'numa=off'
specified in the command line,
this function will fallback to invoke dummy_numa_init(), which give
rise the question here is, do we need to
touch any ACPI related stuff in the case? If not, then the output
message "No NUMA configuration found" followed
seems is not necessary since it's a little bit confusing in case of
numa has already been turned off explicitly.

Thanks,
Dennis

> --
> 1.7.11.7
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Ganapatrao Kulkarni]

On Wed, Apr 20, 2016 at 1:11 PM, Dennis Chen <dennis.chen@linaro.org> wrote:
> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
>> From: Hanjun Guo <hanjun.guo@linaro.org>
>>
>> Introduce a new file to hold ACPI based NUMA information parsing from
>> SRAT and SLIT.
>>
>> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
>> ranges to Proximity Domain mapping.  SLIT has the information of inter
>> node distances(relative number for access latency).
>>
>> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
>> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
>> [rrichter@cavium.com Reworked for numa v10 series ]
>> Signed-off-by: Robert Richter <rrichter@cavium.com>
>> [david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
>> Signed-off-by: David Daney <david.daney@cavium.com>
>> ---
>>  arch/arm64/include/asm/acpi.h |   8 +++
>>  arch/arm64/include/asm/numa.h |   2 +
>>  arch/arm64/kernel/Makefile    |   1 +
>>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>>  arch/arm64/kernel/smp.c       |   2 +
>>  arch/arm64/mm/numa.c          |   5 +-
>>  6 files changed, 166 insertions(+), 1 deletion(-)
>>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>>
>> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
>> index aee323b..4b13ecd 100644
>> --- a/arch/arm64/include/asm/acpi.h
>> +++ b/arch/arm64/include/asm/acpi.h
>> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>>  #endif
>>
>> +#ifdef CONFIG_ACPI_NUMA
>> +int arm64_acpi_numa_init(void);
>> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
>> +#else
>> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
>> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
>> +#endif /* CONFIG_ACPI_NUMA */
>> +
>>  #endif /*_ASM_ACPI_H*/
>> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
>> index e9b4f29..600887e 100644
>> --- a/arch/arm64/include/asm/numa.h
>> +++ b/arch/arm64/include/asm/numa.h
>> @@ -5,6 +5,8 @@
>>
>>  #ifdef CONFIG_NUMA
>>
>> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
>> +
>>  /* currently, arm64 implements flat NUMA topology */
>>  #define parent_node(node)      (node)
>>
>> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
>> index 3793003..69569c6 100644
>> --- a/arch/arm64/kernel/Makefile
>> +++ b/arch/arm64/kernel/Makefile
>> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
>> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
>> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
>> new file mode 100644
>> index 0000000..fd72070
>> --- /dev/null
>> +++ b/arch/arm64/kernel/acpi_numa.c
>> @@ -0,0 +1,149 @@
>> +/*
>> + * ACPI 5.1 based NUMA setup for ARM64
>> + * Lots of code was borrowed from arch/x86/mm/srat.c
>> + *
>> + * Copyright 2004 Andi Kleen, SuSE Labs.
>> + * Copyright (C) 2013-2016, Linaro Ltd.
>> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
>> + *
>> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
>> + *
>> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
>> + * Assumes all memory regions belonging to a single proximity domain
>> + * are in one chunk. Holes between them will be included in the node.
>> + */
>> +
>> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
>> +
>> +#include <linux/acpi.h>
>> +#include <linux/bitmap.h>
>> +#include <linux/bootmem.h>
>> +#include <linux/kernel.h>
>> +#include <linux/mm.h>
>> +#include <linux/memblock.h>
>> +#include <linux/mmzone.h>
>> +#include <linux/module.h>
>> +#include <linux/topology.h>
>> +
>> +#include <acpi/processor.h>
>> +#include <asm/numa.h>
>> +
>> +static int cpus_in_srat;
>> +
>> +struct __node_cpu_hwid {
>> +       u32 node_id;    /* logical node containing this CPU */
>> +       u64 cpu_hwid;   /* MPIDR for this CPU */
>> +};
>> +
>> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
>> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
>> +
>> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
>> +{
>> +       int i;
>> +
>> +       for (i = 0; i < cpus_in_srat; i++) {
>> +               if (hwid == early_node_cpu_hwid[i].cpu_hwid)
>> +                       return early_node_cpu_hwid[i].node_id;
>> +       }
>> +
>> +       return NUMA_NO_NODE;
>> +}
>> +
>> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
>> +{
>> +       unsigned long madt_end, entry;
>> +       struct acpi_table_madt *madt;
>> +       acpi_size tbl_size;
>> +
>> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
>> +                       (struct acpi_table_header **)&madt, &tbl_size)))
>> +               return -ENODEV;
>> +
>> +       entry = (unsigned long)madt;
>> +       madt_end = entry + madt->header.length;
>> +
>> +       /* Parse all entries looking for a match. */
>> +       entry += sizeof(struct acpi_table_madt);
>> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
>> +               struct acpi_subtable_header *header =
>> +                       (struct acpi_subtable_header *)entry;
>> +
>> +               if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
>> +                       struct acpi_madt_generic_interrupt *gicc =
>> +                               container_of(header,
>> +                               struct acpi_madt_generic_interrupt, header);
>> +
>> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
>> +                           (gicc->uid == acpi_id)) {
>> +                               *mpidr = gicc->arm_mpidr;
>> +                               early_acpi_os_unmap_memory(madt, tbl_size);
>> +                               return 0;
>> +                       }
>> +               }
>> +               entry += header->length;
>> +       }
>> +
>> +       early_acpi_os_unmap_memory(madt, tbl_size);
>> +       return -ENODEV;
>> +}
>> +
>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
>> +{
>> +       int pxm, node;
>> +       u64 mpidr;
>> +
>> +       if (srat_disabled())
>> +               return;
>> +
>> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>> +                       pa->header.length);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>> +               return;
>> +
>> +       if (cpus_in_srat >= NR_CPUS) {
>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
>> +                            NR_CPUS);
>> +               return;
>> +       }
>> +
>> +       pxm = pa->proximity_domain;
>> +       node = acpi_map_pxm_to_node(pxm);
>> +
>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
>> +                       pxm, pa->acpi_processor_uid);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>> +       node_set(node, numa_nodes_parsed);
>> +       cpus_in_srat++;
>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>> +               pxm, mpidr, node, cpus_in_srat);
>> +}
>> +
>> +int __init arm64_acpi_numa_init(void)
>> +{
>> +       int ret;
>> +
>> +       ret = acpi_numa_init();
>> +       if (ret)
>> +               return ret;
>> +
>> +       return srat_disabled() ? -EINVAL : 0;
>> +}
>> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
>> index bebc4c6..6c7ef8f 100644
>> --- a/arch/arm64/kernel/smp.c
>> +++ b/arch/arm64/kernel/smp.c
>> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>>          */
>>         acpi_set_mailbox_entry(cpu_count, processor);
>>
>> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
>> +
>>         cpu_count++;
>>  }
>>
>> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
>> index 6cb03f9..fc15186 100644
>> --- a/arch/arm64/mm/numa.c
>> +++ b/arch/arm64/mm/numa.c
>> @@ -17,6 +17,7 @@
>>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>   */
>>
>> +#include <linux/acpi.h>
>>  #include <linux/bootmem.h>
>>  #include <linux/memblock.h>
>>  #include <linux/module.h>
>> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>>  void __init arm64_numa_init(void)
>>  {
>>         if (!numa_off) {
>> -               if (!numa_init(of_numa_init))
>> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
>> +                       return;
>> +               if (acpi_disabled && !numa_init(of_numa_init))
>>                         return;
>>         }
>>
> On top of the latest version of the dt-based numa patch, if 'numa=off'
> specified in the command line,
> this function will fallback to invoke dummy_numa_init(), which give
> rise the question here is, do we need to
> touch any ACPI related stuff in the case? If not, then the output

no, ACPI is not fallback configuration for DT and vice versa.

> message "No NUMA configuration found" followed
> seems is not necessary since it's a little bit confusing in case of
> numa has already been turned off explicitly.

thanks, this print can be moved out.
from function dummy_numa_init and it can be added in function arm64_numa_init
as a last line of if (!numa_off) to indicate, ACPI/DT based NUMA
configuration failed.

more appropriate would be,
pr_info("%s\n", "NUMA configuration failed or not found");

thanks
Ganapat

>
> Thanks,
> Dennis
>
>> --
>> 1.7.11.7
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Ganapatrao Kulkarni]

On Wed, Apr 20, 2016 at 1:11 PM, Dennis Chen <dennis.chen@linaro.org> wrote:
> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
>> From: Hanjun Guo <hanjun.guo@linaro.org>
>>
>> Introduce a new file to hold ACPI based NUMA information parsing from
>> SRAT and SLIT.
>>
>> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
>> ranges to Proximity Domain mapping.  SLIT has the information of inter
>> node distances(relative number for access latency).
>>
>> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
>> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
>> [rrichter@cavium.com Reworked for numa v10 series ]
>> Signed-off-by: Robert Richter <rrichter@cavium.com>
>> [david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
>> Signed-off-by: David Daney <david.daney@cavium.com>
>> ---
>>  arch/arm64/include/asm/acpi.h |   8 +++
>>  arch/arm64/include/asm/numa.h |   2 +
>>  arch/arm64/kernel/Makefile    |   1 +
>>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>>  arch/arm64/kernel/smp.c       |   2 +
>>  arch/arm64/mm/numa.c          |   5 +-
>>  6 files changed, 166 insertions(+), 1 deletion(-)
>>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>>
>> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
>> index aee323b..4b13ecd 100644
>> --- a/arch/arm64/include/asm/acpi.h
>> +++ b/arch/arm64/include/asm/acpi.h
>> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>>  #endif
>>
>> +#ifdef CONFIG_ACPI_NUMA
>> +int arm64_acpi_numa_init(void);
>> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
>> +#else
>> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
>> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
>> +#endif /* CONFIG_ACPI_NUMA */
>> +
>>  #endif /*_ASM_ACPI_H*/
>> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
>> index e9b4f29..600887e 100644
>> --- a/arch/arm64/include/asm/numa.h
>> +++ b/arch/arm64/include/asm/numa.h
>> @@ -5,6 +5,8 @@
>>
>>  #ifdef CONFIG_NUMA
>>
>> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
>> +
>>  /* currently, arm64 implements flat NUMA topology */
>>  #define parent_node(node)      (node)
>>
>> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
>> index 3793003..69569c6 100644
>> --- a/arch/arm64/kernel/Makefile
>> +++ b/arch/arm64/kernel/Makefile
>> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
>> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
>> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
>> new file mode 100644
>> index 0000000..fd72070
>> --- /dev/null
>> +++ b/arch/arm64/kernel/acpi_numa.c
>> @@ -0,0 +1,149 @@
>> +/*
>> + * ACPI 5.1 based NUMA setup for ARM64
>> + * Lots of code was borrowed from arch/x86/mm/srat.c
>> + *
>> + * Copyright 2004 Andi Kleen, SuSE Labs.
>> + * Copyright (C) 2013-2016, Linaro Ltd.
>> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
>> + *
>> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
>> + *
>> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
>> + * Assumes all memory regions belonging to a single proximity domain
>> + * are in one chunk. Holes between them will be included in the node.
>> + */
>> +
>> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
>> +
>> +#include <linux/acpi.h>
>> +#include <linux/bitmap.h>
>> +#include <linux/bootmem.h>
>> +#include <linux/kernel.h>
>> +#include <linux/mm.h>
>> +#include <linux/memblock.h>
>> +#include <linux/mmzone.h>
>> +#include <linux/module.h>
>> +#include <linux/topology.h>
>> +
>> +#include <acpi/processor.h>
>> +#include <asm/numa.h>
>> +
>> +static int cpus_in_srat;
>> +
>> +struct __node_cpu_hwid {
>> +       u32 node_id;    /* logical node containing this CPU */
>> +       u64 cpu_hwid;   /* MPIDR for this CPU */
>> +};
>> +
>> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
>> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
>> +
>> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
>> +{
>> +       int i;
>> +
>> +       for (i = 0; i < cpus_in_srat; i++) {
>> +               if (hwid == early_node_cpu_hwid[i].cpu_hwid)
>> +                       return early_node_cpu_hwid[i].node_id;
>> +       }
>> +
>> +       return NUMA_NO_NODE;
>> +}
>> +
>> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
>> +{
>> +       unsigned long madt_end, entry;
>> +       struct acpi_table_madt *madt;
>> +       acpi_size tbl_size;
>> +
>> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
>> +                       (struct acpi_table_header **)&madt, &tbl_size)))
>> +               return -ENODEV;
>> +
>> +       entry = (unsigned long)madt;
>> +       madt_end = entry + madt->header.length;
>> +
>> +       /* Parse all entries looking for a match. */
>> +       entry += sizeof(struct acpi_table_madt);
>> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
>> +               struct acpi_subtable_header *header =
>> +                       (struct acpi_subtable_header *)entry;
>> +
>> +               if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
>> +                       struct acpi_madt_generic_interrupt *gicc =
>> +                               container_of(header,
>> +                               struct acpi_madt_generic_interrupt, header);
>> +
>> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
>> +                           (gicc->uid == acpi_id)) {
>> +                               *mpidr = gicc->arm_mpidr;
>> +                               early_acpi_os_unmap_memory(madt, tbl_size);
>> +                               return 0;
>> +                       }
>> +               }
>> +               entry += header->length;
>> +       }
>> +
>> +       early_acpi_os_unmap_memory(madt, tbl_size);
>> +       return -ENODEV;
>> +}
>> +
>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
>> +{
>> +       int pxm, node;
>> +       u64 mpidr;
>> +
>> +       if (srat_disabled())
>> +               return;
>> +
>> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>> +                       pa->header.length);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>> +               return;
>> +
>> +       if (cpus_in_srat >= NR_CPUS) {
>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
>> +                            NR_CPUS);
>> +               return;
>> +       }
>> +
>> +       pxm = pa->proximity_domain;
>> +       node = acpi_map_pxm_to_node(pxm);
>> +
>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
>> +                       pxm, pa->acpi_processor_uid);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>> +       node_set(node, numa_nodes_parsed);
>> +       cpus_in_srat++;
>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>> +               pxm, mpidr, node, cpus_in_srat);
>> +}
>> +
>> +int __init arm64_acpi_numa_init(void)
>> +{
>> +       int ret;
>> +
>> +       ret = acpi_numa_init();
>> +       if (ret)
>> +               return ret;
>> +
>> +       return srat_disabled() ? -EINVAL : 0;
>> +}
>> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
>> index bebc4c6..6c7ef8f 100644
>> --- a/arch/arm64/kernel/smp.c
>> +++ b/arch/arm64/kernel/smp.c
>> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>>          */
>>         acpi_set_mailbox_entry(cpu_count, processor);
>>
>> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
>> +
>>         cpu_count++;
>>  }
>>
>> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
>> index 6cb03f9..fc15186 100644
>> --- a/arch/arm64/mm/numa.c
>> +++ b/arch/arm64/mm/numa.c
>> @@ -17,6 +17,7 @@
>>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>   */
>>
>> +#include <linux/acpi.h>
>>  #include <linux/bootmem.h>
>>  #include <linux/memblock.h>
>>  #include <linux/module.h>
>> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>>  void __init arm64_numa_init(void)
>>  {
>>         if (!numa_off) {
>> -               if (!numa_init(of_numa_init))
>> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
>> +                       return;
>> +               if (acpi_disabled && !numa_init(of_numa_init))
>>                         return;
>>         }
>>
> On top of the latest version of the dt-based numa patch, if 'numa=off'
> specified in the command line,
> this function will fallback to invoke dummy_numa_init(), which give
> rise the question here is, do we need to
> touch any ACPI related stuff in the case? If not, then the output

no, ACPI is not fallback configuration for DT and vice versa.

> message "No NUMA configuration found" followed
> seems is not necessary since it's a little bit confusing in case of
> numa has already been turned off explicitly.

thanks, this print can be moved out.
from function dummy_numa_init and it can be added in function arm64_numa_init
as a last line of if (!numa_off) to indicate, ACPI/DT based NUMA
configuration failed.

more appropriate would be,
pr_info("%s\n", "NUMA configuration failed or not found");

thanks
Ganapat

>
> Thanks,
> Dennis
>
>> --
>> 1.7.11.7
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Ganapatrao Kulkarni]

On Wed, Apr 20, 2016 at 1:11 PM, Dennis Chen <dennis.chen@linaro.org> wrote:
> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
>> From: Hanjun Guo <hanjun.guo@linaro.org>
>>
>> Introduce a new file to hold ACPI based NUMA information parsing from
>> SRAT and SLIT.
>>
>> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
>> ranges to Proximity Domain mapping.  SLIT has the information of inter
>> node distances(relative number for access latency).
>>
>> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
>> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
>> [rrichter at cavium.com Reworked for numa v10 series ]
>> Signed-off-by: Robert Richter <rrichter@cavium.com>
>> [david.daney at cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
>> Signed-off-by: David Daney <david.daney@cavium.com>
>> ---
>>  arch/arm64/include/asm/acpi.h |   8 +++
>>  arch/arm64/include/asm/numa.h |   2 +
>>  arch/arm64/kernel/Makefile    |   1 +
>>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>>  arch/arm64/kernel/smp.c       |   2 +
>>  arch/arm64/mm/numa.c          |   5 +-
>>  6 files changed, 166 insertions(+), 1 deletion(-)
>>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>>
>> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
>> index aee323b..4b13ecd 100644
>> --- a/arch/arm64/include/asm/acpi.h
>> +++ b/arch/arm64/include/asm/acpi.h
>> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>>  #endif
>>
>> +#ifdef CONFIG_ACPI_NUMA
>> +int arm64_acpi_numa_init(void);
>> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
>> +#else
>> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
>> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
>> +#endif /* CONFIG_ACPI_NUMA */
>> +
>>  #endif /*_ASM_ACPI_H*/
>> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
>> index e9b4f29..600887e 100644
>> --- a/arch/arm64/include/asm/numa.h
>> +++ b/arch/arm64/include/asm/numa.h
>> @@ -5,6 +5,8 @@
>>
>>  #ifdef CONFIG_NUMA
>>
>> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
>> +
>>  /* currently, arm64 implements flat NUMA topology */
>>  #define parent_node(node)      (node)
>>
>> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
>> index 3793003..69569c6 100644
>> --- a/arch/arm64/kernel/Makefile
>> +++ b/arch/arm64/kernel/Makefile
>> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
>> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
>> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
>> new file mode 100644
>> index 0000000..fd72070
>> --- /dev/null
>> +++ b/arch/arm64/kernel/acpi_numa.c
>> @@ -0,0 +1,149 @@
>> +/*
>> + * ACPI 5.1 based NUMA setup for ARM64
>> + * Lots of code was borrowed from arch/x86/mm/srat.c
>> + *
>> + * Copyright 2004 Andi Kleen, SuSE Labs.
>> + * Copyright (C) 2013-2016, Linaro Ltd.
>> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
>> + *
>> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
>> + *
>> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
>> + * Assumes all memory regions belonging to a single proximity domain
>> + * are in one chunk. Holes between them will be included in the node.
>> + */
>> +
>> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
>> +
>> +#include <linux/acpi.h>
>> +#include <linux/bitmap.h>
>> +#include <linux/bootmem.h>
>> +#include <linux/kernel.h>
>> +#include <linux/mm.h>
>> +#include <linux/memblock.h>
>> +#include <linux/mmzone.h>
>> +#include <linux/module.h>
>> +#include <linux/topology.h>
>> +
>> +#include <acpi/processor.h>
>> +#include <asm/numa.h>
>> +
>> +static int cpus_in_srat;
>> +
>> +struct __node_cpu_hwid {
>> +       u32 node_id;    /* logical node containing this CPU */
>> +       u64 cpu_hwid;   /* MPIDR for this CPU */
>> +};
>> +
>> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
>> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
>> +
>> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
>> +{
>> +       int i;
>> +
>> +       for (i = 0; i < cpus_in_srat; i++) {
>> +               if (hwid == early_node_cpu_hwid[i].cpu_hwid)
>> +                       return early_node_cpu_hwid[i].node_id;
>> +       }
>> +
>> +       return NUMA_NO_NODE;
>> +}
>> +
>> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
>> +{
>> +       unsigned long madt_end, entry;
>> +       struct acpi_table_madt *madt;
>> +       acpi_size tbl_size;
>> +
>> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
>> +                       (struct acpi_table_header **)&madt, &tbl_size)))
>> +               return -ENODEV;
>> +
>> +       entry = (unsigned long)madt;
>> +       madt_end = entry + madt->header.length;
>> +
>> +       /* Parse all entries looking for a match. */
>> +       entry += sizeof(struct acpi_table_madt);
>> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
>> +               struct acpi_subtable_header *header =
>> +                       (struct acpi_subtable_header *)entry;
>> +
>> +               if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
>> +                       struct acpi_madt_generic_interrupt *gicc =
>> +                               container_of(header,
>> +                               struct acpi_madt_generic_interrupt, header);
>> +
>> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
>> +                           (gicc->uid == acpi_id)) {
>> +                               *mpidr = gicc->arm_mpidr;
>> +                               early_acpi_os_unmap_memory(madt, tbl_size);
>> +                               return 0;
>> +                       }
>> +               }
>> +               entry += header->length;
>> +       }
>> +
>> +       early_acpi_os_unmap_memory(madt, tbl_size);
>> +       return -ENODEV;
>> +}
>> +
>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
>> +{
>> +       int pxm, node;
>> +       u64 mpidr;
>> +
>> +       if (srat_disabled())
>> +               return;
>> +
>> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>> +                       pa->header.length);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>> +               return;
>> +
>> +       if (cpus_in_srat >= NR_CPUS) {
>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
>> +                            NR_CPUS);
>> +               return;
>> +       }
>> +
>> +       pxm = pa->proximity_domain;
>> +       node = acpi_map_pxm_to_node(pxm);
>> +
>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
>> +                       pxm, pa->acpi_processor_uid);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>> +       node_set(node, numa_nodes_parsed);
>> +       cpus_in_srat++;
>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>> +               pxm, mpidr, node, cpus_in_srat);
>> +}
>> +
>> +int __init arm64_acpi_numa_init(void)
>> +{
>> +       int ret;
>> +
>> +       ret = acpi_numa_init();
>> +       if (ret)
>> +               return ret;
>> +
>> +       return srat_disabled() ? -EINVAL : 0;
>> +}
>> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
>> index bebc4c6..6c7ef8f 100644
>> --- a/arch/arm64/kernel/smp.c
>> +++ b/arch/arm64/kernel/smp.c
>> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>>          */
>>         acpi_set_mailbox_entry(cpu_count, processor);
>>
>> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
>> +
>>         cpu_count++;
>>  }
>>
>> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
>> index 6cb03f9..fc15186 100644
>> --- a/arch/arm64/mm/numa.c
>> +++ b/arch/arm64/mm/numa.c
>> @@ -17,6 +17,7 @@
>>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>   */
>>
>> +#include <linux/acpi.h>
>>  #include <linux/bootmem.h>
>>  #include <linux/memblock.h>
>>  #include <linux/module.h>
>> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>>  void __init arm64_numa_init(void)
>>  {
>>         if (!numa_off) {
>> -               if (!numa_init(of_numa_init))
>> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
>> +                       return;
>> +               if (acpi_disabled && !numa_init(of_numa_init))
>>                         return;
>>         }
>>
> On top of the latest version of the dt-based numa patch, if 'numa=off'
> specified in the command line,
> this function will fallback to invoke dummy_numa_init(), which give
> rise the question here is, do we need to
> touch any ACPI related stuff in the case? If not, then the output

no, ACPI is not fallback configuration for DT and vice versa.

> message "No NUMA configuration found" followed
> seems is not necessary since it's a little bit confusing in case of
> numa has already been turned off explicitly.

thanks, this print can be moved out.
from function dummy_numa_init and it can be added in function arm64_numa_init
as a last line of if (!numa_off) to indicate, ACPI/DT based NUMA
configuration failed.

more appropriate would be,
pr_info("%s\n", "NUMA configuration failed or not found");

thanks
Ganapat

>
> Thanks,
> Dennis
>
>> --
>> 1.7.11.7
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>> the body of a message to majordomo at vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Ganapatrao Kulkarni]

On Wed, Apr 20, 2016 at 1:11 PM, Dennis Chen <dennis.chen@linaro.org> wrote:
> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
>> From: Hanjun Guo <hanjun.guo@linaro.org>
>>
>> Introduce a new file to hold ACPI based NUMA information parsing from
>> SRAT and SLIT.
>>
>> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
>> ranges to Proximity Domain mapping.  SLIT has the information of inter
>> node distances(relative number for access latency).
>>
>> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
>> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
>> [rrichter@cavium.com Reworked for numa v10 series ]
>> Signed-off-by: Robert Richter <rrichter@cavium.com>
>> [david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
>> Signed-off-by: David Daney <david.daney@cavium.com>
>> ---
>>  arch/arm64/include/asm/acpi.h |   8 +++
>>  arch/arm64/include/asm/numa.h |   2 +
>>  arch/arm64/kernel/Makefile    |   1 +
>>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>>  arch/arm64/kernel/smp.c       |   2 +
>>  arch/arm64/mm/numa.c          |   5 +-
>>  6 files changed, 166 insertions(+), 1 deletion(-)
>>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>>
>> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
>> index aee323b..4b13ecd 100644
>> --- a/arch/arm64/include/asm/acpi.h
>> +++ b/arch/arm64/include/asm/acpi.h
>> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>>  #endif
>>
>> +#ifdef CONFIG_ACPI_NUMA
>> +int arm64_acpi_numa_init(void);
>> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
>> +#else
>> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
>> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
>> +#endif /* CONFIG_ACPI_NUMA */
>> +
>>  #endif /*_ASM_ACPI_H*/
>> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
>> index e9b4f29..600887e 100644
>> --- a/arch/arm64/include/asm/numa.h
>> +++ b/arch/arm64/include/asm/numa.h
>> @@ -5,6 +5,8 @@
>>
>>  #ifdef CONFIG_NUMA
>>
>> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
>> +
>>  /* currently, arm64 implements flat NUMA topology */
>>  #define parent_node(node)      (node)
>>
>> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
>> index 3793003..69569c6 100644
>> --- a/arch/arm64/kernel/Makefile
>> +++ b/arch/arm64/kernel/Makefile
>> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
>> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
>> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
>> new file mode 100644
>> index 0000000..fd72070
>> --- /dev/null
>> +++ b/arch/arm64/kernel/acpi_numa.c
>> @@ -0,0 +1,149 @@
>> +/*
>> + * ACPI 5.1 based NUMA setup for ARM64
>> + * Lots of code was borrowed from arch/x86/mm/srat.c
>> + *
>> + * Copyright 2004 Andi Kleen, SuSE Labs.
>> + * Copyright (C) 2013-2016, Linaro Ltd.
>> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
>> + *
>> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
>> + *
>> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
>> + * Assumes all memory regions belonging to a single proximity domain
>> + * are in one chunk. Holes between them will be included in the node.
>> + */
>> +
>> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
>> +
>> +#include <linux/acpi.h>
>> +#include <linux/bitmap.h>
>> +#include <linux/bootmem.h>
>> +#include <linux/kernel.h>
>> +#include <linux/mm.h>
>> +#include <linux/memblock.h>
>> +#include <linux/mmzone.h>
>> +#include <linux/module.h>
>> +#include <linux/topology.h>
>> +
>> +#include <acpi/processor.h>
>> +#include <asm/numa.h>
>> +
>> +static int cpus_in_srat;
>> +
>> +struct __node_cpu_hwid {
>> +       u32 node_id;    /* logical node containing this CPU */
>> +       u64 cpu_hwid;   /* MPIDR for this CPU */
>> +};
>> +
>> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
>> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
>> +
>> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
>> +{
>> +       int i;
>> +
>> +       for (i = 0; i < cpus_in_srat; i++) {
>> +               if (hwid = early_node_cpu_hwid[i].cpu_hwid)
>> +                       return early_node_cpu_hwid[i].node_id;
>> +       }
>> +
>> +       return NUMA_NO_NODE;
>> +}
>> +
>> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
>> +{
>> +       unsigned long madt_end, entry;
>> +       struct acpi_table_madt *madt;
>> +       acpi_size tbl_size;
>> +
>> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
>> +                       (struct acpi_table_header **)&madt, &tbl_size)))
>> +               return -ENODEV;
>> +
>> +       entry = (unsigned long)madt;
>> +       madt_end = entry + madt->header.length;
>> +
>> +       /* Parse all entries looking for a match. */
>> +       entry += sizeof(struct acpi_table_madt);
>> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
>> +               struct acpi_subtable_header *header >> +                       (struct acpi_subtable_header *)entry;
>> +
>> +               if (header->type = ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
>> +                       struct acpi_madt_generic_interrupt *gicc >> +                               container_of(header,
>> +                               struct acpi_madt_generic_interrupt, header);
>> +
>> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
>> +                           (gicc->uid = acpi_id)) {
>> +                               *mpidr = gicc->arm_mpidr;
>> +                               early_acpi_os_unmap_memory(madt, tbl_size);
>> +                               return 0;
>> +                       }
>> +               }
>> +               entry += header->length;
>> +       }
>> +
>> +       early_acpi_os_unmap_memory(madt, tbl_size);
>> +       return -ENODEV;
>> +}
>> +
>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
>> +{
>> +       int pxm, node;
>> +       u64 mpidr;
>> +
>> +       if (srat_disabled())
>> +               return;
>> +
>> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>> +                       pa->header.length);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>> +               return;
>> +
>> +       if (cpus_in_srat >= NR_CPUS) {
>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
>> +                            NR_CPUS);
>> +               return;
>> +       }
>> +
>> +       pxm = pa->proximity_domain;
>> +       node = acpi_map_pxm_to_node(pxm);
>> +
>> +       if (node = NUMA_NO_NODE || node >= MAX_NUMNODES) {
>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
>> +                       pxm, pa->acpi_processor_uid);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>> +       node_set(node, numa_nodes_parsed);
>> +       cpus_in_srat++;
>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>> +               pxm, mpidr, node, cpus_in_srat);
>> +}
>> +
>> +int __init arm64_acpi_numa_init(void)
>> +{
>> +       int ret;
>> +
>> +       ret = acpi_numa_init();
>> +       if (ret)
>> +               return ret;
>> +
>> +       return srat_disabled() ? -EINVAL : 0;
>> +}
>> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
>> index bebc4c6..6c7ef8f 100644
>> --- a/arch/arm64/kernel/smp.c
>> +++ b/arch/arm64/kernel/smp.c
>> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>>          */
>>         acpi_set_mailbox_entry(cpu_count, processor);
>>
>> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
>> +
>>         cpu_count++;
>>  }
>>
>> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
>> index 6cb03f9..fc15186 100644
>> --- a/arch/arm64/mm/numa.c
>> +++ b/arch/arm64/mm/numa.c
>> @@ -17,6 +17,7 @@
>>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>   */
>>
>> +#include <linux/acpi.h>
>>  #include <linux/bootmem.h>
>>  #include <linux/memblock.h>
>>  #include <linux/module.h>
>> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>>  void __init arm64_numa_init(void)
>>  {
>>         if (!numa_off) {
>> -               if (!numa_init(of_numa_init))
>> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
>> +                       return;
>> +               if (acpi_disabled && !numa_init(of_numa_init))
>>                         return;
>>         }
>>
> On top of the latest version of the dt-based numa patch, if 'numa=off'
> specified in the command line,
> this function will fallback to invoke dummy_numa_init(), which give
> rise the question here is, do we need to
> touch any ACPI related stuff in the case? If not, then the output

no, ACPI is not fallback configuration for DT and vice versa.

> message "No NUMA configuration found" followed
> seems is not necessary since it's a little bit confusing in case of
> numa has already been turned off explicitly.

thanks, this print can be moved out.
from function dummy_numa_init and it can be added in function arm64_numa_init
as a last line of if (!numa_off) to indicate, ACPI/DT based NUMA
configuration failed.

more appropriate would be,
pr_info("%s\n", "NUMA configuration failed or not found");

thanks
Ganapat

>
> Thanks,
> Dennis
>
>> --
>> 1.7.11.7
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/20/2016 01:31 AM, Ganapatrao Kulkarni wrote:
> On Wed, Apr 20, 2016 at 1:11 PM, Dennis Chen <dennis.chen@linaro.org> wrote:
>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
[...]
>>> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>>>   void __init arm64_numa_init(void)
>>>   {
>>>          if (!numa_off) {
>>> -               if (!numa_init(of_numa_init))
>>> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
>>> +                       return;
>>> +               if (acpi_disabled && !numa_init(of_numa_init))
>>>                          return;
>>>          }
>>>
>> On top of the latest version of the dt-based numa patch, if 'numa=off'
>> specified in the command line,
>> this function will fallback to invoke dummy_numa_init(), which give
>> rise the question here is, do we need to
>> touch any ACPI related stuff in the case? If not, then the output
>
> no, ACPI is not fallback configuration for DT and vice versa.
>
>> message "No NUMA configuration found" followed
>> seems is not necessary since it's a little bit confusing in case of
>> numa has already been turned off explicitly.
>
> thanks, this print can be moved out.
> from function dummy_numa_init and it can be added in function arm64_numa_init
> as a last line of if (!numa_off) to indicate, ACPI/DT based NUMA
> configuration failed.
>
> more appropriate would be,
> pr_info("%s\n", "NUMA configuration failed or not found");
>

Although purely cosmetic, I agree that we need to improve the messages 
as to not confuse people.

I will rework the messages with your suggestions in mind to see if we 
can get something that is both concise and unambiguously reflects what 
the user asked for.

David.

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/20/2016 01:31 AM, Ganapatrao Kulkarni wrote:
> On Wed, Apr 20, 2016 at 1:11 PM, Dennis Chen <dennis.chen@linaro.org> wrote:
>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
[...]
>>> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>>>   void __init arm64_numa_init(void)
>>>   {
>>>          if (!numa_off) {
>>> -               if (!numa_init(of_numa_init))
>>> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
>>> +                       return;
>>> +               if (acpi_disabled && !numa_init(of_numa_init))
>>>                          return;
>>>          }
>>>
>> On top of the latest version of the dt-based numa patch, if 'numa=off'
>> specified in the command line,
>> this function will fallback to invoke dummy_numa_init(), which give
>> rise the question here is, do we need to
>> touch any ACPI related stuff in the case? If not, then the output
>
> no, ACPI is not fallback configuration for DT and vice versa.
>
>> message "No NUMA configuration found" followed
>> seems is not necessary since it's a little bit confusing in case of
>> numa has already been turned off explicitly.
>
> thanks, this print can be moved out.
> from function dummy_numa_init and it can be added in function arm64_numa_init
> as a last line of if (!numa_off) to indicate, ACPI/DT based NUMA
> configuration failed.
>
> more appropriate would be,
> pr_info("%s\n", "NUMA configuration failed or not found");
>

Although purely cosmetic, I agree that we need to improve the messages 
as to not confuse people.

I will rework the messages with your suggestions in mind to see if we 
can get something that is both concise and unambiguously reflects what 
the user asked for.

David.

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/20/2016 01:31 AM, Ganapatrao Kulkarni wrote:
> On Wed, Apr 20, 2016 at 1:11 PM, Dennis Chen <dennis.chen@linaro.org> wrote:
>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
[...]
>>> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>>>   void __init arm64_numa_init(void)
>>>   {
>>>          if (!numa_off) {
>>> -               if (!numa_init(of_numa_init))
>>> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
>>> +                       return;
>>> +               if (acpi_disabled && !numa_init(of_numa_init))
>>>                          return;
>>>          }
>>>
>> On top of the latest version of the dt-based numa patch, if 'numa=off'
>> specified in the command line,
>> this function will fallback to invoke dummy_numa_init(), which give
>> rise the question here is, do we need to
>> touch any ACPI related stuff in the case? If not, then the output
>
> no, ACPI is not fallback configuration for DT and vice versa.
>
>> message "No NUMA configuration found" followed
>> seems is not necessary since it's a little bit confusing in case of
>> numa has already been turned off explicitly.
>
> thanks, this print can be moved out.
> from function dummy_numa_init and it can be added in function arm64_numa_init
> as a last line of if (!numa_off) to indicate, ACPI/DT based NUMA
> configuration failed.
>
> more appropriate would be,
> pr_info("%s\n", "NUMA configuration failed or not found");
>

Although purely cosmetic, I agree that we need to improve the messages 
as to not confuse people.

I will rework the messages with your suggestions in mind to see if we 
can get something that is both concise and unambiguously reflects what 
the user asked for.

David.

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/20/2016 01:31 AM, Ganapatrao Kulkarni wrote:
> On Wed, Apr 20, 2016 at 1:11 PM, Dennis Chen <dennis.chen@linaro.org> wrote:
>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
[...]
>>> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>>>   void __init arm64_numa_init(void)
>>>   {
>>>          if (!numa_off) {
>>> -               if (!numa_init(of_numa_init))
>>> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
>>> +                       return;
>>> +               if (acpi_disabled && !numa_init(of_numa_init))
>>>                          return;
>>>          }
>>>
>> On top of the latest version of the dt-based numa patch, if 'numa=off'
>> specified in the command line,
>> this function will fallback to invoke dummy_numa_init(), which give
>> rise the question here is, do we need to
>> touch any ACPI related stuff in the case? If not, then the output
>
> no, ACPI is not fallback configuration for DT and vice versa.
>
>> message "No NUMA configuration found" followed
>> seems is not necessary since it's a little bit confusing in case of
>> numa has already been turned off explicitly.
>
> thanks, this print can be moved out.
> from function dummy_numa_init and it can be added in function arm64_numa_init
> as a last line of if (!numa_off) to indicate, ACPI/DT based NUMA
> configuration failed.
>
> more appropriate would be,
> pr_info("%s\n", "NUMA configuration failed or not found");
>

Although purely cosmetic, I agree that we need to improve the messages 
as to not confuse people.

I will rework the messages with your suggestions in mind to see if we 
can get something that is both concise and unambiguously reflects what 
the user asked for.

David.

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> From: Hanjun Guo <hanjun.guo@linaro.org>
>
> Introduce a new file to hold ACPI based NUMA information parsing from
> SRAT and SLIT.
>
> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
> ranges to Proximity Domain mapping.  SLIT has the information of inter
> node distances(relative number for access latency).
>
> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> [rrichter@cavium.com Reworked for numa v10 series ]
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> [david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>  arch/arm64/include/asm/acpi.h |   8 +++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |   5 +-
>  6 files changed, 166 insertions(+), 1 deletion(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>
> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
> index aee323b..4b13ecd 100644
> --- a/arch/arm64/include/asm/acpi.h
> +++ b/arch/arm64/include/asm/acpi.h
> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>  #endif
>
> +#ifdef CONFIG_ACPI_NUMA
> +int arm64_acpi_numa_init(void);
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
> +#else
> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
> +#endif /* CONFIG_ACPI_NUMA */
> +
>  #endif /*_ASM_ACPI_H*/
> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
> index e9b4f29..600887e 100644
> --- a/arch/arm64/include/asm/numa.h
> +++ b/arch/arm64/include/asm/numa.h
> @@ -5,6 +5,8 @@
>
>  #ifdef CONFIG_NUMA
>
> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
> +
>  /* currently, arm64 implements flat NUMA topology */
>  #define parent_node(node)      (node)
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index 3793003..69569c6 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
> new file mode 100644
> index 0000000..fd72070
> --- /dev/null
> +++ b/arch/arm64/kernel/acpi_numa.c
> @@ -0,0 +1,149 @@
> +/*
> + * ACPI 5.1 based NUMA setup for ARM64
> + * Lots of code was borrowed from arch/x86/mm/srat.c
> + *
> + * Copyright 2004 Andi Kleen, SuSE Labs.
> + * Copyright (C) 2013-2016, Linaro Ltd.
> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
> + *
> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
> + *
> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
> + * Assumes all memory regions belonging to a single proximity domain
> + * are in one chunk. Holes between them will be included in the node.
> + */
> +
> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
> +
> +#include <linux/acpi.h>
> +#include <linux/bitmap.h>
> +#include <linux/bootmem.h>
> +#include <linux/kernel.h>
> +#include <linux/mm.h>
> +#include <linux/memblock.h>
> +#include <linux/mmzone.h>
> +#include <linux/module.h>
> +#include <linux/topology.h>
> +
> +#include <acpi/processor.h>
> +#include <asm/numa.h>
> +
> +static int cpus_in_srat;
> +
> +struct __node_cpu_hwid {
> +       u32 node_id;    /* logical node containing this CPU */
> +       u64 cpu_hwid;   /* MPIDR for this CPU */
> +};
> +
> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
> +
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
> +{
> +       int i;
> +
> +       for (i = 0; i < cpus_in_srat; i++) {
> +               if (hwid == early_node_cpu_hwid[i].cpu_hwid)
> +                       return early_node_cpu_hwid[i].node_id;
> +       }
> +
> +       return NUMA_NO_NODE;
> +}
> +
> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
> +{
> +       unsigned long madt_end, entry;
> +       struct acpi_table_madt *madt;
> +       acpi_size tbl_size;
> +
> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
> +                       (struct acpi_table_header **)&madt, &tbl_size)))
> +               return -ENODEV;
> +
> +       entry = (unsigned long)madt;
> +       madt_end = entry + madt->header.length;
> +
> +       /* Parse all entries looking for a match. */
> +       entry += sizeof(struct acpi_table_madt);
> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
> +               struct acpi_subtable_header *header =
> +                       (struct acpi_subtable_header *)entry;
> +
> +               if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
> +                       struct acpi_madt_generic_interrupt *gicc =
> +                               container_of(header,
> +                               struct acpi_madt_generic_interrupt, header);
> +
> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
> +                           (gicc->uid == acpi_id)) {
> +                               *mpidr = gicc->arm_mpidr;
> +                               early_acpi_os_unmap_memory(madt, tbl_size);
> +                               return 0;
> +                       }
> +               }
> +               entry += header->length;
> +       }
> +
> +       early_acpi_os_unmap_memory(madt, tbl_size);
> +       return -ENODEV;
> +}
> +
> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
> +{
> +       int pxm, node;
> +       u64 mpidr;
> +
> +       if (srat_disabled())
> +               return;
> +
> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
> +                       pa->header.length);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
> +               return;
> +
> +       if (cpus_in_srat >= NR_CPUS) {
> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
> +                            NR_CPUS);
> +               return;
> +       }
> +
> +       pxm = pa->proximity_domain;
> +       node = acpi_map_pxm_to_node(pxm);
> +
> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
> +                       pxm, pa->acpi_processor_uid);
> +               bad_srat();
> +               return;
> +       }
> +
> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
> +       node_set(node, numa_nodes_parsed);
> +       cpus_in_srat++;
> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
> +               pxm, mpidr, node, cpus_in_srat);
> +}

What does the *cpu* means in above pr_info function? If it's the
logical processor ID or ACPI processor UID, then I suggest to use
pa->acpi_processor_uid instead of cpus_in_srat, I understand the
cpus_in_srat is just a count number of the entries of GICC Affinity
Struct instance in SRAT, correct me if I am wrong. So at least it sees
to me, the above pr_info will output message looks like:
SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3

While the /sys/devices/system/cpu will use the ACPI processor UID to
generate the index of the cpu, like:
cpu0  cpu1  cpu2 ...

As the GICC Affinity Struct indicated, the ps->proximity_domain is the
domain to which the logical processor belongs...

Thanks,
Dennis

> +
> +int __init arm64_acpi_numa_init(void)
> +{
> +       int ret;
> +
> +       ret = acpi_numa_init();
> +       if (ret)
> +               return ret;
> +
> +       return srat_disabled() ? -EINVAL : 0;
> +}
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index bebc4c6..6c7ef8f 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>          */
>         acpi_set_mailbox_entry(cpu_count, processor);
>
> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
> +
>         cpu_count++;
>  }
>
> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> index 6cb03f9..fc15186 100644
> --- a/arch/arm64/mm/numa.c
> +++ b/arch/arm64/mm/numa.c
> @@ -17,6 +17,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>
> +#include <linux/acpi.h>
>  #include <linux/bootmem.h>
>  #include <linux/memblock.h>
>  #include <linux/module.h>
> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>  void __init arm64_numa_init(void)
>  {
>         if (!numa_off) {
> -               if (!numa_init(of_numa_init))
> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
> +                       return;
> +               if (acpi_disabled && !numa_init(of_numa_init))
>                         return;
>         }
>
> --
> 1.7.11.7
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Regards,
Dennis

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> From: Hanjun Guo <hanjun.guo@linaro.org>
>
> Introduce a new file to hold ACPI based NUMA information parsing from
> SRAT and SLIT.
>
> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
> ranges to Proximity Domain mapping.  SLIT has the information of inter
> node distances(relative number for access latency).
>
> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> [rrichter@cavium.com Reworked for numa v10 series ]
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> [david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>  arch/arm64/include/asm/acpi.h |   8 +++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |   5 +-
>  6 files changed, 166 insertions(+), 1 deletion(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>
> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
> index aee323b..4b13ecd 100644
> --- a/arch/arm64/include/asm/acpi.h
> +++ b/arch/arm64/include/asm/acpi.h
> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>  #endif
>
> +#ifdef CONFIG_ACPI_NUMA
> +int arm64_acpi_numa_init(void);
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
> +#else
> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
> +#endif /* CONFIG_ACPI_NUMA */
> +
>  #endif /*_ASM_ACPI_H*/
> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
> index e9b4f29..600887e 100644
> --- a/arch/arm64/include/asm/numa.h
> +++ b/arch/arm64/include/asm/numa.h
> @@ -5,6 +5,8 @@
>
>  #ifdef CONFIG_NUMA
>
> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
> +
>  /* currently, arm64 implements flat NUMA topology */
>  #define parent_node(node)      (node)
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index 3793003..69569c6 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
> new file mode 100644
> index 0000000..fd72070
> --- /dev/null
> +++ b/arch/arm64/kernel/acpi_numa.c
> @@ -0,0 +1,149 @@
> +/*
> + * ACPI 5.1 based NUMA setup for ARM64
> + * Lots of code was borrowed from arch/x86/mm/srat.c
> + *
> + * Copyright 2004 Andi Kleen, SuSE Labs.
> + * Copyright (C) 2013-2016, Linaro Ltd.
> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
> + *
> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
> + *
> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
> + * Assumes all memory regions belonging to a single proximity domain
> + * are in one chunk. Holes between them will be included in the node.
> + */
> +
> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
> +
> +#include <linux/acpi.h>
> +#include <linux/bitmap.h>
> +#include <linux/bootmem.h>
> +#include <linux/kernel.h>
> +#include <linux/mm.h>
> +#include <linux/memblock.h>
> +#include <linux/mmzone.h>
> +#include <linux/module.h>
> +#include <linux/topology.h>
> +
> +#include <acpi/processor.h>
> +#include <asm/numa.h>
> +
> +static int cpus_in_srat;
> +
> +struct __node_cpu_hwid {
> +       u32 node_id;    /* logical node containing this CPU */
> +       u64 cpu_hwid;   /* MPIDR for this CPU */
> +};
> +
> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
> +
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
> +{
> +       int i;
> +
> +       for (i = 0; i < cpus_in_srat; i++) {
> +               if (hwid == early_node_cpu_hwid[i].cpu_hwid)
> +                       return early_node_cpu_hwid[i].node_id;
> +       }
> +
> +       return NUMA_NO_NODE;
> +}
> +
> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
> +{
> +       unsigned long madt_end, entry;
> +       struct acpi_table_madt *madt;
> +       acpi_size tbl_size;
> +
> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
> +                       (struct acpi_table_header **)&madt, &tbl_size)))
> +               return -ENODEV;
> +
> +       entry = (unsigned long)madt;
> +       madt_end = entry + madt->header.length;
> +
> +       /* Parse all entries looking for a match. */
> +       entry += sizeof(struct acpi_table_madt);
> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
> +               struct acpi_subtable_header *header =
> +                       (struct acpi_subtable_header *)entry;
> +
> +               if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
> +                       struct acpi_madt_generic_interrupt *gicc =
> +                               container_of(header,
> +                               struct acpi_madt_generic_interrupt, header);
> +
> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
> +                           (gicc->uid == acpi_id)) {
> +                               *mpidr = gicc->arm_mpidr;
> +                               early_acpi_os_unmap_memory(madt, tbl_size);
> +                               return 0;
> +                       }
> +               }
> +               entry += header->length;
> +       }
> +
> +       early_acpi_os_unmap_memory(madt, tbl_size);
> +       return -ENODEV;
> +}
> +
> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
> +{
> +       int pxm, node;
> +       u64 mpidr;
> +
> +       if (srat_disabled())
> +               return;
> +
> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
> +                       pa->header.length);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
> +               return;
> +
> +       if (cpus_in_srat >= NR_CPUS) {
> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
> +                            NR_CPUS);
> +               return;
> +       }
> +
> +       pxm = pa->proximity_domain;
> +       node = acpi_map_pxm_to_node(pxm);
> +
> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
> +                       pxm, pa->acpi_processor_uid);
> +               bad_srat();
> +               return;
> +       }
> +
> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
> +       node_set(node, numa_nodes_parsed);
> +       cpus_in_srat++;
> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
> +               pxm, mpidr, node, cpus_in_srat);
> +}

What does the *cpu* means in above pr_info function? If it's the
logical processor ID or ACPI processor UID, then I suggest to use
pa->acpi_processor_uid instead of cpus_in_srat, I understand the
cpus_in_srat is just a count number of the entries of GICC Affinity
Struct instance in SRAT, correct me if I am wrong. So at least it sees
to me, the above pr_info will output message looks like:
SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3

While the /sys/devices/system/cpu will use the ACPI processor UID to
generate the index of the cpu, like:
cpu0  cpu1  cpu2 ...

As the GICC Affinity Struct indicated, the ps->proximity_domain is the
domain to which the logical processor belongs...

Thanks,
Dennis

> +
> +int __init arm64_acpi_numa_init(void)
> +{
> +       int ret;
> +
> +       ret = acpi_numa_init();
> +       if (ret)
> +               return ret;
> +
> +       return srat_disabled() ? -EINVAL : 0;
> +}
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index bebc4c6..6c7ef8f 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>          */
>         acpi_set_mailbox_entry(cpu_count, processor);
>
> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
> +
>         cpu_count++;
>  }
>
> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> index 6cb03f9..fc15186 100644
> --- a/arch/arm64/mm/numa.c
> +++ b/arch/arm64/mm/numa.c
> @@ -17,6 +17,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>
> +#include <linux/acpi.h>
>  #include <linux/bootmem.h>
>  #include <linux/memblock.h>
>  #include <linux/module.h>
> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>  void __init arm64_numa_init(void)
>  {
>         if (!numa_off) {
> -               if (!numa_init(of_numa_init))
> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
> +                       return;
> +               if (acpi_disabled && !numa_init(of_numa_init))
>                         return;
>         }
>
> --
> 1.7.11.7
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Regards,
Dennis

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> From: Hanjun Guo <hanjun.guo@linaro.org>
>
> Introduce a new file to hold ACPI based NUMA information parsing from
> SRAT and SLIT.
>
> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
> ranges to Proximity Domain mapping.  SLIT has the information of inter
> node distances(relative number for access latency).
>
> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> [rrichter at cavium.com Reworked for numa v10 series ]
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> [david.daney at cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>  arch/arm64/include/asm/acpi.h |   8 +++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |   5 +-
>  6 files changed, 166 insertions(+), 1 deletion(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>
> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
> index aee323b..4b13ecd 100644
> --- a/arch/arm64/include/asm/acpi.h
> +++ b/arch/arm64/include/asm/acpi.h
> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>  #endif
>
> +#ifdef CONFIG_ACPI_NUMA
> +int arm64_acpi_numa_init(void);
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
> +#else
> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
> +#endif /* CONFIG_ACPI_NUMA */
> +
>  #endif /*_ASM_ACPI_H*/
> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
> index e9b4f29..600887e 100644
> --- a/arch/arm64/include/asm/numa.h
> +++ b/arch/arm64/include/asm/numa.h
> @@ -5,6 +5,8 @@
>
>  #ifdef CONFIG_NUMA
>
> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
> +
>  /* currently, arm64 implements flat NUMA topology */
>  #define parent_node(node)      (node)
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index 3793003..69569c6 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
> new file mode 100644
> index 0000000..fd72070
> --- /dev/null
> +++ b/arch/arm64/kernel/acpi_numa.c
> @@ -0,0 +1,149 @@
> +/*
> + * ACPI 5.1 based NUMA setup for ARM64
> + * Lots of code was borrowed from arch/x86/mm/srat.c
> + *
> + * Copyright 2004 Andi Kleen, SuSE Labs.
> + * Copyright (C) 2013-2016, Linaro Ltd.
> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
> + *
> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
> + *
> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
> + * Assumes all memory regions belonging to a single proximity domain
> + * are in one chunk. Holes between them will be included in the node.
> + */
> +
> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
> +
> +#include <linux/acpi.h>
> +#include <linux/bitmap.h>
> +#include <linux/bootmem.h>
> +#include <linux/kernel.h>
> +#include <linux/mm.h>
> +#include <linux/memblock.h>
> +#include <linux/mmzone.h>
> +#include <linux/module.h>
> +#include <linux/topology.h>
> +
> +#include <acpi/processor.h>
> +#include <asm/numa.h>
> +
> +static int cpus_in_srat;
> +
> +struct __node_cpu_hwid {
> +       u32 node_id;    /* logical node containing this CPU */
> +       u64 cpu_hwid;   /* MPIDR for this CPU */
> +};
> +
> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
> +
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
> +{
> +       int i;
> +
> +       for (i = 0; i < cpus_in_srat; i++) {
> +               if (hwid == early_node_cpu_hwid[i].cpu_hwid)
> +                       return early_node_cpu_hwid[i].node_id;
> +       }
> +
> +       return NUMA_NO_NODE;
> +}
> +
> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
> +{
> +       unsigned long madt_end, entry;
> +       struct acpi_table_madt *madt;
> +       acpi_size tbl_size;
> +
> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
> +                       (struct acpi_table_header **)&madt, &tbl_size)))
> +               return -ENODEV;
> +
> +       entry = (unsigned long)madt;
> +       madt_end = entry + madt->header.length;
> +
> +       /* Parse all entries looking for a match. */
> +       entry += sizeof(struct acpi_table_madt);
> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
> +               struct acpi_subtable_header *header =
> +                       (struct acpi_subtable_header *)entry;
> +
> +               if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
> +                       struct acpi_madt_generic_interrupt *gicc =
> +                               container_of(header,
> +                               struct acpi_madt_generic_interrupt, header);
> +
> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
> +                           (gicc->uid == acpi_id)) {
> +                               *mpidr = gicc->arm_mpidr;
> +                               early_acpi_os_unmap_memory(madt, tbl_size);
> +                               return 0;
> +                       }
> +               }
> +               entry += header->length;
> +       }
> +
> +       early_acpi_os_unmap_memory(madt, tbl_size);
> +       return -ENODEV;
> +}
> +
> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
> +{
> +       int pxm, node;
> +       u64 mpidr;
> +
> +       if (srat_disabled())
> +               return;
> +
> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
> +                       pa->header.length);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
> +               return;
> +
> +       if (cpus_in_srat >= NR_CPUS) {
> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
> +                            NR_CPUS);
> +               return;
> +       }
> +
> +       pxm = pa->proximity_domain;
> +       node = acpi_map_pxm_to_node(pxm);
> +
> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
> +                       pxm, pa->acpi_processor_uid);
> +               bad_srat();
> +               return;
> +       }
> +
> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
> +       node_set(node, numa_nodes_parsed);
> +       cpus_in_srat++;
> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
> +               pxm, mpidr, node, cpus_in_srat);
> +}

What does the *cpu* means in above pr_info function? If it's the
logical processor ID or ACPI processor UID, then I suggest to use
pa->acpi_processor_uid instead of cpus_in_srat, I understand the
cpus_in_srat is just a count number of the entries of GICC Affinity
Struct instance in SRAT, correct me if I am wrong. So at least it sees
to me, the above pr_info will output message looks like:
SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3

While the /sys/devices/system/cpu will use the ACPI processor UID to
generate the index of the cpu, like:
cpu0  cpu1  cpu2 ...

As the GICC Affinity Struct indicated, the ps->proximity_domain is the
domain to which the logical processor belongs...

Thanks,
Dennis

> +
> +int __init arm64_acpi_numa_init(void)
> +{
> +       int ret;
> +
> +       ret = acpi_numa_init();
> +       if (ret)
> +               return ret;
> +
> +       return srat_disabled() ? -EINVAL : 0;
> +}
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index bebc4c6..6c7ef8f 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>          */
>         acpi_set_mailbox_entry(cpu_count, processor);
>
> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
> +
>         cpu_count++;
>  }
>
> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> index 6cb03f9..fc15186 100644
> --- a/arch/arm64/mm/numa.c
> +++ b/arch/arm64/mm/numa.c
> @@ -17,6 +17,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>
> +#include <linux/acpi.h>
>  #include <linux/bootmem.h>
>  #include <linux/memblock.h>
>  #include <linux/module.h>
> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>  void __init arm64_numa_init(void)
>  {
>         if (!numa_off) {
> -               if (!numa_init(of_numa_init))
> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
> +                       return;
> +               if (acpi_disabled && !numa_init(of_numa_init))
>                         return;
>         }
>
> --
> 1.7.11.7
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Regards,
Dennis

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> From: Hanjun Guo <hanjun.guo@linaro.org>
>
> Introduce a new file to hold ACPI based NUMA information parsing from
> SRAT and SLIT.
>
> SRAT includes the CPU ACPI ID to Proximity Domain mappings and memory
> ranges to Proximity Domain mapping.  SLIT has the information of inter
> node distances(relative number for access latency).
>
> Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> [rrichter@cavium.com Reworked for numa v10 series ]
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> [david.daney@cavium.com reorderd and combinded with other patches in Hanjun Guo's original set]
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>  arch/arm64/include/asm/acpi.h |   8 +++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |   5 +-
>  6 files changed, 166 insertions(+), 1 deletion(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
>
> diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h
> index aee323b..4b13ecd 100644
> --- a/arch/arm64/include/asm/acpi.h
> +++ b/arch/arm64/include/asm/acpi.h
> @@ -113,4 +113,12 @@ static inline const char *acpi_get_enable_method(int cpu)
>  pgprot_t arch_apei_get_mem_attribute(phys_addr_t addr);
>  #endif
>
> +#ifdef CONFIG_ACPI_NUMA
> +int arm64_acpi_numa_init(void);
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid);
> +#else
> +static inline int arm64_acpi_numa_init(void) { return -ENOSYS; }
> +static inline int acpi_numa_get_nid(unsigned int cpu, u64 hwid) { return NUMA_NO_NODE; }
> +#endif /* CONFIG_ACPI_NUMA */
> +
>  #endif /*_ASM_ACPI_H*/
> diff --git a/arch/arm64/include/asm/numa.h b/arch/arm64/include/asm/numa.h
> index e9b4f29..600887e 100644
> --- a/arch/arm64/include/asm/numa.h
> +++ b/arch/arm64/include/asm/numa.h
> @@ -5,6 +5,8 @@
>
>  #ifdef CONFIG_NUMA
>
> +#define NR_NODE_MEMBLKS                (MAX_NUMNODES * 2)
> +
>  /* currently, arm64 implements flat NUMA topology */
>  #define parent_node(node)      (node)
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index 3793003..69569c6 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -42,6 +42,7 @@ arm64-obj-$(CONFIG_EFI)                       += efi.o efi-entry.stub.o
>  arm64-obj-$(CONFIG_PCI)                        += pci.o
>  arm64-obj-$(CONFIG_ARMV8_DEPRECATED)   += armv8_deprecated.o
>  arm64-obj-$(CONFIG_ACPI)               += acpi.o
> +arm64-obj-$(CONFIG_ACPI_NUMA)          += acpi_numa.o
>  arm64-obj-$(CONFIG_ARM64_ACPI_PARKING_PROTOCOL)        += acpi_parking_protocol.o
>  arm64-obj-$(CONFIG_PARAVIRT)           += paravirt.o
>  arm64-obj-$(CONFIG_RANDOMIZE_BASE)     += kaslr.o
> diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c
> new file mode 100644
> index 0000000..fd72070
> --- /dev/null
> +++ b/arch/arm64/kernel/acpi_numa.c
> @@ -0,0 +1,149 @@
> +/*
> + * ACPI 5.1 based NUMA setup for ARM64
> + * Lots of code was borrowed from arch/x86/mm/srat.c
> + *
> + * Copyright 2004 Andi Kleen, SuSE Labs.
> + * Copyright (C) 2013-2016, Linaro Ltd.
> + *             Author: Hanjun Guo <hanjun.guo@linaro.org>
> + *
> + * Reads the ACPI SRAT table to figure out what memory belongs to which CPUs.
> + *
> + * Called from acpi_numa_init while reading the SRAT and SLIT tables.
> + * Assumes all memory regions belonging to a single proximity domain
> + * are in one chunk. Holes between them will be included in the node.
> + */
> +
> +#define pr_fmt(fmt) "ACPI: NUMA: " fmt
> +
> +#include <linux/acpi.h>
> +#include <linux/bitmap.h>
> +#include <linux/bootmem.h>
> +#include <linux/kernel.h>
> +#include <linux/mm.h>
> +#include <linux/memblock.h>
> +#include <linux/mmzone.h>
> +#include <linux/module.h>
> +#include <linux/topology.h>
> +
> +#include <acpi/processor.h>
> +#include <asm/numa.h>
> +
> +static int cpus_in_srat;
> +
> +struct __node_cpu_hwid {
> +       u32 node_id;    /* logical node containing this CPU */
> +       u64 cpu_hwid;   /* MPIDR for this CPU */
> +};
> +
> +static struct __node_cpu_hwid early_node_cpu_hwid[NR_CPUS] = {
> +[0 ... NR_CPUS - 1] = {NUMA_NO_NODE, PHYS_CPUID_INVALID} };
> +
> +int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
> +{
> +       int i;
> +
> +       for (i = 0; i < cpus_in_srat; i++) {
> +               if (hwid = early_node_cpu_hwid[i].cpu_hwid)
> +                       return early_node_cpu_hwid[i].node_id;
> +       }
> +
> +       return NUMA_NO_NODE;
> +}
> +
> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr)
> +{
> +       unsigned long madt_end, entry;
> +       struct acpi_table_madt *madt;
> +       acpi_size tbl_size;
> +
> +       if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0,
> +                       (struct acpi_table_header **)&madt, &tbl_size)))
> +               return -ENODEV;
> +
> +       entry = (unsigned long)madt;
> +       madt_end = entry + madt->header.length;
> +
> +       /* Parse all entries looking for a match. */
> +       entry += sizeof(struct acpi_table_madt);
> +       while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
> +               struct acpi_subtable_header *header > +                       (struct acpi_subtable_header *)entry;
> +
> +               if (header->type = ACPI_MADT_TYPE_GENERIC_INTERRUPT) {
> +                       struct acpi_madt_generic_interrupt *gicc > +                               container_of(header,
> +                               struct acpi_madt_generic_interrupt, header);
> +
> +                       if ((gicc->flags & ACPI_MADT_ENABLED) &&
> +                           (gicc->uid = acpi_id)) {
> +                               *mpidr = gicc->arm_mpidr;
> +                               early_acpi_os_unmap_memory(madt, tbl_size);
> +                               return 0;
> +                       }
> +               }
> +               entry += header->length;
> +       }
> +
> +       early_acpi_os_unmap_memory(madt, tbl_size);
> +       return -ENODEV;
> +}
> +
> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
> +{
> +       int pxm, node;
> +       u64 mpidr;
> +
> +       if (srat_disabled())
> +               return;
> +
> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
> +                       pa->header.length);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
> +               return;
> +
> +       if (cpus_in_srat >= NR_CPUS) {
> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
> +                            NR_CPUS);
> +               return;
> +       }
> +
> +       pxm = pa->proximity_domain;
> +       node = acpi_map_pxm_to_node(pxm);
> +
> +       if (node = NUMA_NO_NODE || node >= MAX_NUMNODES) {
> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
> +               bad_srat();
> +               return;
> +       }
> +
> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
> +                       pxm, pa->acpi_processor_uid);
> +               bad_srat();
> +               return;
> +       }
> +
> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
> +       node_set(node, numa_nodes_parsed);
> +       cpus_in_srat++;
> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
> +               pxm, mpidr, node, cpus_in_srat);
> +}

What does the *cpu* means in above pr_info function? If it's the
logical processor ID or ACPI processor UID, then I suggest to use
pa->acpi_processor_uid instead of cpus_in_srat, I understand the
cpus_in_srat is just a count number of the entries of GICC Affinity
Struct instance in SRAT, correct me if I am wrong. So at least it sees
to me, the above pr_info will output message looks like:
SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3

While the /sys/devices/system/cpu will use the ACPI processor UID to
generate the index of the cpu, like:
cpu0  cpu1  cpu2 ...

As the GICC Affinity Struct indicated, the ps->proximity_domain is the
domain to which the logical processor belongs...

Thanks,
Dennis

> +
> +int __init arm64_acpi_numa_init(void)
> +{
> +       int ret;
> +
> +       ret = acpi_numa_init();
> +       if (ret)
> +               return ret;
> +
> +       return srat_disabled() ? -EINVAL : 0;
> +}
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index bebc4c6..6c7ef8f 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -524,6 +524,8 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>          */
>         acpi_set_mailbox_entry(cpu_count, processor);
>
> +       early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
> +
>         cpu_count++;
>  }
>
> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> index 6cb03f9..fc15186 100644
> --- a/arch/arm64/mm/numa.c
> +++ b/arch/arm64/mm/numa.c
> @@ -17,6 +17,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
>
> +#include <linux/acpi.h>
>  #include <linux/bootmem.h>
>  #include <linux/memblock.h>
>  #include <linux/module.h>
> @@ -388,7 +389,9 @@ static int __init dummy_numa_init(void)
>  void __init arm64_numa_init(void)
>  {
>         if (!numa_off) {
> -               if (!numa_init(of_numa_init))
> +               if (!acpi_disabled && !numa_init(arm64_acpi_numa_init))
> +                       return;
> +               if (acpi_disabled && !numa_init(of_numa_init))
>                         return;
>         }
>
> --
> 1.7.11.7
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Regards,
Dennis

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Will Deacon]

Hi David,

On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
> From: David Daney <david.daney@cavium.com>
> 
> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
> set introduce the ACPI based configuration to provide NUMA
> information.
> 
> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
> NUMA domain information from SRAT and SLIT table, so parse those two
> tables to get mappings from cpu/mem to numa node configuration and
> system locality.

Whilst I've queued the main NUMA series for arm64, I'd really like to
see more movement on the generic header file cleanups that you posted
separately:

http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com

Given that this ACPI series already requires some significant cross-arch
interaction (which is actually good!), perhaps extending the clean-up
patches to encompass some of the ACPI bits might make sense, and we can
get that queued as a pre-requisite.

Will

> v5 updates:
> 
>  - Fixed ia64 build failure by gating some unused functions with #if
>    CONFIG_{X86,ARM64}.
> 
>  - Fixed section mismatch errors for X86 case.
> 
>  - Removed unneeded #include from some files.
> 
>  - Tested to build cleanly on ARM64, X86_64, IA64
> 
> v4 updates:
> 
>  - Updated from Hanjun Guo's v3 patches.
> 
>  - Rebased on top of v16 of device-tree NUMA patches.
> 
>  - Reordered some of the changes so that we don't introduce code and
>    then change it several times in the patch set.  New code is
>    introduced in its final form.  Code reused from x86 is first moved
>    with no change, and then a separate patch to make any needed
>    changes.
> 
>  - code that is used only by ia64, moved to architecture specific
>    files.
> 
> v3 updates:
>  - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
>    that for acpi/numa.c (patch 2/12)
> 
>  - Remove the duplicate NULL check for table print (patch 3/12)
> 
>  - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
>    dummy function for acpi_numa_arch_fixup()
> 
>  - Solve the problem that the mapping from logical cpu to numa node
>    is wrong which spotted out by Lorenzo
> 
>  - cleanups for x86 and move acpi_numa_slit_init() and some other
>    functions to common place, then reduce the duplicate of x86
>    and arm64 (patch 7-12/12).
> 
>  - rebased on top of 4.4 and Ganapat's v9 patch set.
> 
> [1]: https://lkml.org/lkml/2016/4/8/571
> 
> David Daney (1):
>   acpi, numa, srat: Improve SRAT error detection and add messages.
> 
> Hanjun Guo (11):
>   acpi, numa: Use pr_fmt() instead of printk
>   acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>   acpi, numa: remove duplicate NULL check
>   acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>   arm64, numa: rework numa_add_memblk()
>   x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>   acpi, numa: move bad_srat() and srat_disabled() to
>     drivers/acpi/numa.c
>   acpi, numa: remove unneeded acpi_numa=1
>   acpi, numa: Move acpi_numa_memory_affinity_init() to
>     drivers/acpi/numa.c
>   arm64, acpi, numa: NUMA support based on SRAT and SLIT
>   acpi, numa: Enable ACPI based NUMA on ARM64
> 
> Robert Richter (2):
>   acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>   arm64, acpi, numa: Default enable ACPI_NUMA with NUMA
> 
>  arch/arm64/include/asm/acpi.h |   8 ++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |  17 ++--
>  arch/ia64/include/asm/acpi.h  |   3 +
>  arch/ia64/kernel/acpi.c       |   2 +-
>  arch/ia64/kernel/setup.c      |   1 +
>  arch/x86/include/asm/acpi.h   |   1 -
>  arch/x86/mm/numa.c            |   2 +-
>  arch/x86/mm/srat.c            | 116 +---------------------
>  drivers/acpi/Kconfig          |   4 +-
>  drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
>  drivers/of/of_numa.c          |   4 +-
>  include/acpi/acpi_numa.h      |   4 +
>  include/linux/acpi.h          |  18 +++-
>  17 files changed, 375 insertions(+), 183 deletions(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
> 
> -- 
> 1.7.11.7
> 

[PATCH v5 00/14] ACPI NUMA support for ARM64

[Will Deacon]

Hi David,

On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
> From: David Daney <david.daney@cavium.com>
> 
> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
> set introduce the ACPI based configuration to provide NUMA
> information.
> 
> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
> NUMA domain information from SRAT and SLIT table, so parse those two
> tables to get mappings from cpu/mem to numa node configuration and
> system locality.

Whilst I've queued the main NUMA series for arm64, I'd really like to
see more movement on the generic header file cleanups that you posted
separately:

http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm at gmail.com

Given that this ACPI series already requires some significant cross-arch
interaction (which is actually good!), perhaps extending the clean-up
patches to encompass some of the ACPI bits might make sense, and we can
get that queued as a pre-requisite.

Will

> v5 updates:
> 
>  - Fixed ia64 build failure by gating some unused functions with #if
>    CONFIG_{X86,ARM64}.
> 
>  - Fixed section mismatch errors for X86 case.
> 
>  - Removed unneeded #include from some files.
> 
>  - Tested to build cleanly on ARM64, X86_64, IA64
> 
> v4 updates:
> 
>  - Updated from Hanjun Guo's v3 patches.
> 
>  - Rebased on top of v16 of device-tree NUMA patches.
> 
>  - Reordered some of the changes so that we don't introduce code and
>    then change it several times in the patch set.  New code is
>    introduced in its final form.  Code reused from x86 is first moved
>    with no change, and then a separate patch to make any needed
>    changes.
> 
>  - code that is used only by ia64, moved to architecture specific
>    files.
> 
> v3 updates:
>  - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
>    that for acpi/numa.c (patch 2/12)
> 
>  - Remove the duplicate NULL check for table print (patch 3/12)
> 
>  - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
>    dummy function for acpi_numa_arch_fixup()
> 
>  - Solve the problem that the mapping from logical cpu to numa node
>    is wrong which spotted out by Lorenzo
> 
>  - cleanups for x86 and move acpi_numa_slit_init() and some other
>    functions to common place, then reduce the duplicate of x86
>    and arm64 (patch 7-12/12).
> 
>  - rebased on top of 4.4 and Ganapat's v9 patch set.
> 
> [1]: https://lkml.org/lkml/2016/4/8/571
> 
> David Daney (1):
>   acpi, numa, srat: Improve SRAT error detection and add messages.
> 
> Hanjun Guo (11):
>   acpi, numa: Use pr_fmt() instead of printk
>   acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>   acpi, numa: remove duplicate NULL check
>   acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>   arm64, numa: rework numa_add_memblk()
>   x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>   acpi, numa: move bad_srat() and srat_disabled() to
>     drivers/acpi/numa.c
>   acpi, numa: remove unneeded acpi_numa=1
>   acpi, numa: Move acpi_numa_memory_affinity_init() to
>     drivers/acpi/numa.c
>   arm64, acpi, numa: NUMA support based on SRAT and SLIT
>   acpi, numa: Enable ACPI based NUMA on ARM64
> 
> Robert Richter (2):
>   acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>   arm64, acpi, numa: Default enable ACPI_NUMA with NUMA
> 
>  arch/arm64/include/asm/acpi.h |   8 ++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |  17 ++--
>  arch/ia64/include/asm/acpi.h  |   3 +
>  arch/ia64/kernel/acpi.c       |   2 +-
>  arch/ia64/kernel/setup.c      |   1 +
>  arch/x86/include/asm/acpi.h   |   1 -
>  arch/x86/mm/numa.c            |   2 +-
>  arch/x86/mm/srat.c            | 116 +---------------------
>  drivers/acpi/Kconfig          |   4 +-
>  drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
>  drivers/of/of_numa.c          |   4 +-
>  include/acpi/acpi_numa.h      |   4 +
>  include/linux/acpi.h          |  18 +++-
>  17 files changed, 375 insertions(+), 183 deletions(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
> 
> -- 
> 1.7.11.7
> 

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Will Deacon]

Hi David,

On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
> From: David Daney <david.daney@cavium.com>
> 
> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
> set introduce the ACPI based configuration to provide NUMA
> information.
> 
> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
> NUMA domain information from SRAT and SLIT table, so parse those two
> tables to get mappings from cpu/mem to numa node configuration and
> system locality.

Whilst I've queued the main NUMA series for arm64, I'd really like to
see more movement on the generic header file cleanups that you posted
separately:

http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com

Given that this ACPI series already requires some significant cross-arch
interaction (which is actually good!), perhaps extending the clean-up
patches to encompass some of the ACPI bits might make sense, and we can
get that queued as a pre-requisite.

Will

> v5 updates:
> 
>  - Fixed ia64 build failure by gating some unused functions with #if
>    CONFIG_{X86,ARM64}.
> 
>  - Fixed section mismatch errors for X86 case.
> 
>  - Removed unneeded #include from some files.
> 
>  - Tested to build cleanly on ARM64, X86_64, IA64
> 
> v4 updates:
> 
>  - Updated from Hanjun Guo's v3 patches.
> 
>  - Rebased on top of v16 of device-tree NUMA patches.
> 
>  - Reordered some of the changes so that we don't introduce code and
>    then change it several times in the patch set.  New code is
>    introduced in its final form.  Code reused from x86 is first moved
>    with no change, and then a separate patch to make any needed
>    changes.
> 
>  - code that is used only by ia64, moved to architecture specific
>    files.
> 
> v3 updates:
>  - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
>    that for acpi/numa.c (patch 2/12)
> 
>  - Remove the duplicate NULL check for table print (patch 3/12)
> 
>  - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
>    dummy function for acpi_numa_arch_fixup()
> 
>  - Solve the problem that the mapping from logical cpu to numa node
>    is wrong which spotted out by Lorenzo
> 
>  - cleanups for x86 and move acpi_numa_slit_init() and some other
>    functions to common place, then reduce the duplicate of x86
>    and arm64 (patch 7-12/12).
> 
>  - rebased on top of 4.4 and Ganapat's v9 patch set.
> 
> [1]: https://lkml.org/lkml/2016/4/8/571
> 
> David Daney (1):
>   acpi, numa, srat: Improve SRAT error detection and add messages.
> 
> Hanjun Guo (11):
>   acpi, numa: Use pr_fmt() instead of printk
>   acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>   acpi, numa: remove duplicate NULL check
>   acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>   arm64, numa: rework numa_add_memblk()
>   x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>   acpi, numa: move bad_srat() and srat_disabled() to
>     drivers/acpi/numa.c
>   acpi, numa: remove unneeded acpi_numa=1
>   acpi, numa: Move acpi_numa_memory_affinity_init() to
>     drivers/acpi/numa.c
>   arm64, acpi, numa: NUMA support based on SRAT and SLIT
>   acpi, numa: Enable ACPI based NUMA on ARM64
> 
> Robert Richter (2):
>   acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>   arm64, acpi, numa: Default enable ACPI_NUMA with NUMA
> 
>  arch/arm64/include/asm/acpi.h |   8 ++
>  arch/arm64/include/asm/numa.h |   2 +
>  arch/arm64/kernel/Makefile    |   1 +
>  arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
>  arch/arm64/kernel/smp.c       |   2 +
>  arch/arm64/mm/numa.c          |  17 ++--
>  arch/ia64/include/asm/acpi.h  |   3 +
>  arch/ia64/kernel/acpi.c       |   2 +-
>  arch/ia64/kernel/setup.c      |   1 +
>  arch/x86/include/asm/acpi.h   |   1 -
>  arch/x86/mm/numa.c            |   2 +-
>  arch/x86/mm/srat.c            | 116 +---------------------
>  drivers/acpi/Kconfig          |   4 +-
>  drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
>  drivers/of/of_numa.c          |   4 +-
>  include/acpi/acpi_numa.h      |   4 +
>  include/linux/acpi.h          |  18 +++-
>  17 files changed, 375 insertions(+), 183 deletions(-)
>  create mode 100644 arch/arm64/kernel/acpi_numa.c
> 
> -- 
> 1.7.11.7
> 

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

On 04/25/2016 04:13 AM, Will Deacon wrote:
> Hi David,
>
> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>> From: David Daney <david.daney@cavium.com>
>>
>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>> set introduce the ACPI based configuration to provide NUMA
>> information.
>>
>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>> NUMA domain information from SRAT and SLIT table, so parse those two
>> tables to get mappings from cpu/mem to numa node configuration and
>> system locality.
>
> Whilst I've queued the main NUMA series for arm64, I'd really like to
> see more movement on the generic header file cleanups that you posted
> separately:
>
> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com

FWIW:  Those patches should still apply.  I am carrying them in my 
development trees, and have not changed them in any way.

>
> Given that this ACPI series already requires some significant cross-arch
> interaction (which is actually good!), perhaps extending the clean-up
> patches to encompass some of the ACPI bits might make sense, and we can
> get that queued as a pre-requisite.

The cleanup patches you mention above are really independent of the ACPI 
things.  I have applied them both before and after the ACPI patches, and 
both seem to work.  With a quick perusal of the ACPI patches nothing 
jumps out at me as being a candidate for inclusion in the header file 
cleanup series.

If you think it would be a good idea, I can repost the cleanup series.

David Daney


>
> Will
>
>> v5 updates:
>>
>>   - Fixed ia64 build failure by gating some unused functions with #if
>>     CONFIG_{X86,ARM64}.
>>
>>   - Fixed section mismatch errors for X86 case.
>>
>>   - Removed unneeded #include from some files.
>>
>>   - Tested to build cleanly on ARM64, X86_64, IA64
>>
>> v4 updates:
>>
>>   - Updated from Hanjun Guo's v3 patches.
>>
>>   - Rebased on top of v16 of device-tree NUMA patches.
>>
>>   - Reordered some of the changes so that we don't introduce code and
>>     then change it several times in the patch set.  New code is
>>     introduced in its final form.  Code reused from x86 is first moved
>>     with no change, and then a separate patch to make any needed
>>     changes.
>>
>>   - code that is used only by ia64, moved to architecture specific
>>     files.
>>
>> v3 updates:
>>   - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
>>     that for acpi/numa.c (patch 2/12)
>>
>>   - Remove the duplicate NULL check for table print (patch 3/12)
>>
>>   - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
>>     dummy function for acpi_numa_arch_fixup()
>>
>>   - Solve the problem that the mapping from logical cpu to numa node
>>     is wrong which spotted out by Lorenzo
>>
>>   - cleanups for x86 and move acpi_numa_slit_init() and some other
>>     functions to common place, then reduce the duplicate of x86
>>     and arm64 (patch 7-12/12).
>>
>>   - rebased on top of 4.4 and Ganapat's v9 patch set.
>>
>> [1]: https://lkml.org/lkml/2016/4/8/571
>>
>> David Daney (1):
>>    acpi, numa, srat: Improve SRAT error detection and add messages.
>>
>> Hanjun Guo (11):
>>    acpi, numa: Use pr_fmt() instead of printk
>>    acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>>    acpi, numa: remove duplicate NULL check
>>    acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>>    arm64, numa: rework numa_add_memblk()
>>    x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>>    acpi, numa: move bad_srat() and srat_disabled() to
>>      drivers/acpi/numa.c
>>    acpi, numa: remove unneeded acpi_numa=1
>>    acpi, numa: Move acpi_numa_memory_affinity_init() to
>>      drivers/acpi/numa.c
>>    arm64, acpi, numa: NUMA support based on SRAT and SLIT
>>    acpi, numa: Enable ACPI based NUMA on ARM64
>>
>> Robert Richter (2):
>>    acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>>    arm64, acpi, numa: Default enable ACPI_NUMA with NUMA
>>
>>   arch/arm64/include/asm/acpi.h |   8 ++
>>   arch/arm64/include/asm/numa.h |   2 +
>>   arch/arm64/kernel/Makefile    |   1 +
>>   arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
>>   arch/arm64/kernel/smp.c       |   2 +
>>   arch/arm64/mm/numa.c          |  17 ++--
>>   arch/ia64/include/asm/acpi.h  |   3 +
>>   arch/ia64/kernel/acpi.c       |   2 +-
>>   arch/ia64/kernel/setup.c      |   1 +
>>   arch/x86/include/asm/acpi.h   |   1 -
>>   arch/x86/mm/numa.c            |   2 +-
>>   arch/x86/mm/srat.c            | 116 +---------------------
>>   drivers/acpi/Kconfig          |   4 +-
>>   drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
>>   drivers/of/of_numa.c          |   4 +-
>>   include/acpi/acpi_numa.h      |   4 +
>>   include/linux/acpi.h          |  18 +++-
>>   17 files changed, 375 insertions(+), 183 deletions(-)
>>   create mode 100644 arch/arm64/kernel/acpi_numa.c
>>
>> --
>> 1.7.11.7
>>

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

On 04/25/2016 04:13 AM, Will Deacon wrote:
> Hi David,
>
> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>> From: David Daney <david.daney@cavium.com>
>>
>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>> set introduce the ACPI based configuration to provide NUMA
>> information.
>>
>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>> NUMA domain information from SRAT and SLIT table, so parse those two
>> tables to get mappings from cpu/mem to numa node configuration and
>> system locality.
>
> Whilst I've queued the main NUMA series for arm64, I'd really like to
> see more movement on the generic header file cleanups that you posted
> separately:
>
> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com

FWIW:  Those patches should still apply.  I am carrying them in my 
development trees, and have not changed them in any way.

>
> Given that this ACPI series already requires some significant cross-arch
> interaction (which is actually good!), perhaps extending the clean-up
> patches to encompass some of the ACPI bits might make sense, and we can
> get that queued as a pre-requisite.

The cleanup patches you mention above are really independent of the ACPI 
things.  I have applied them both before and after the ACPI patches, and 
both seem to work.  With a quick perusal of the ACPI patches nothing 
jumps out at me as being a candidate for inclusion in the header file 
cleanup series.

If you think it would be a good idea, I can repost the cleanup series.

David Daney


>
> Will
>
>> v5 updates:
>>
>>   - Fixed ia64 build failure by gating some unused functions with #if
>>     CONFIG_{X86,ARM64}.
>>
>>   - Fixed section mismatch errors for X86 case.
>>
>>   - Removed unneeded #include from some files.
>>
>>   - Tested to build cleanly on ARM64, X86_64, IA64
>>
>> v4 updates:
>>
>>   - Updated from Hanjun Guo's v3 patches.
>>
>>   - Rebased on top of v16 of device-tree NUMA patches.
>>
>>   - Reordered some of the changes so that we don't introduce code and
>>     then change it several times in the patch set.  New code is
>>     introduced in its final form.  Code reused from x86 is first moved
>>     with no change, and then a separate patch to make any needed
>>     changes.
>>
>>   - code that is used only by ia64, moved to architecture specific
>>     files.
>>
>> v3 updates:
>>   - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
>>     that for acpi/numa.c (patch 2/12)
>>
>>   - Remove the duplicate NULL check for table print (patch 3/12)
>>
>>   - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
>>     dummy function for acpi_numa_arch_fixup()
>>
>>   - Solve the problem that the mapping from logical cpu to numa node
>>     is wrong which spotted out by Lorenzo
>>
>>   - cleanups for x86 and move acpi_numa_slit_init() and some other
>>     functions to common place, then reduce the duplicate of x86
>>     and arm64 (patch 7-12/12).
>>
>>   - rebased on top of 4.4 and Ganapat's v9 patch set.
>>
>> [1]: https://lkml.org/lkml/2016/4/8/571
>>
>> David Daney (1):
>>    acpi, numa, srat: Improve SRAT error detection and add messages.
>>
>> Hanjun Guo (11):
>>    acpi, numa: Use pr_fmt() instead of printk
>>    acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>>    acpi, numa: remove duplicate NULL check
>>    acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>>    arm64, numa: rework numa_add_memblk()
>>    x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>>    acpi, numa: move bad_srat() and srat_disabled() to
>>      drivers/acpi/numa.c
>>    acpi, numa: remove unneeded acpi_numa=1
>>    acpi, numa: Move acpi_numa_memory_affinity_init() to
>>      drivers/acpi/numa.c
>>    arm64, acpi, numa: NUMA support based on SRAT and SLIT
>>    acpi, numa: Enable ACPI based NUMA on ARM64
>>
>> Robert Richter (2):
>>    acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>>    arm64, acpi, numa: Default enable ACPI_NUMA with NUMA
>>
>>   arch/arm64/include/asm/acpi.h |   8 ++
>>   arch/arm64/include/asm/numa.h |   2 +
>>   arch/arm64/kernel/Makefile    |   1 +
>>   arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
>>   arch/arm64/kernel/smp.c       |   2 +
>>   arch/arm64/mm/numa.c          |  17 ++--
>>   arch/ia64/include/asm/acpi.h  |   3 +
>>   arch/ia64/kernel/acpi.c       |   2 +-
>>   arch/ia64/kernel/setup.c      |   1 +
>>   arch/x86/include/asm/acpi.h   |   1 -
>>   arch/x86/mm/numa.c            |   2 +-
>>   arch/x86/mm/srat.c            | 116 +---------------------
>>   drivers/acpi/Kconfig          |   4 +-
>>   drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
>>   drivers/of/of_numa.c          |   4 +-
>>   include/acpi/acpi_numa.h      |   4 +
>>   include/linux/acpi.h          |  18 +++-
>>   17 files changed, 375 insertions(+), 183 deletions(-)
>>   create mode 100644 arch/arm64/kernel/acpi_numa.c
>>
>> --
>> 1.7.11.7
>>

[PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

On 04/25/2016 04:13 AM, Will Deacon wrote:
> Hi David,
>
> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>> From: David Daney <david.daney@cavium.com>
>>
>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>> set introduce the ACPI based configuration to provide NUMA
>> information.
>>
>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>> NUMA domain information from SRAT and SLIT table, so parse those two
>> tables to get mappings from cpu/mem to numa node configuration and
>> system locality.
>
> Whilst I've queued the main NUMA series for arm64, I'd really like to
> see more movement on the generic header file cleanups that you posted
> separately:
>
> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm at gmail.com

FWIW:  Those patches should still apply.  I am carrying them in my 
development trees, and have not changed them in any way.

>
> Given that this ACPI series already requires some significant cross-arch
> interaction (which is actually good!), perhaps extending the clean-up
> patches to encompass some of the ACPI bits might make sense, and we can
> get that queued as a pre-requisite.

The cleanup patches you mention above are really independent of the ACPI 
things.  I have applied them both before and after the ACPI patches, and 
both seem to work.  With a quick perusal of the ACPI patches nothing 
jumps out at me as being a candidate for inclusion in the header file 
cleanup series.

If you think it would be a good idea, I can repost the cleanup series.

David Daney


>
> Will
>
>> v5 updates:
>>
>>   - Fixed ia64 build failure by gating some unused functions with #if
>>     CONFIG_{X86,ARM64}.
>>
>>   - Fixed section mismatch errors for X86 case.
>>
>>   - Removed unneeded #include from some files.
>>
>>   - Tested to build cleanly on ARM64, X86_64, IA64
>>
>> v4 updates:
>>
>>   - Updated from Hanjun Guo's v3 patches.
>>
>>   - Rebased on top of v16 of device-tree NUMA patches.
>>
>>   - Reordered some of the changes so that we don't introduce code and
>>     then change it several times in the patch set.  New code is
>>     introduced in its final form.  Code reused from x86 is first moved
>>     with no change, and then a separate patch to make any needed
>>     changes.
>>
>>   - code that is used only by ia64, moved to architecture specific
>>     files.
>>
>> v3 updates:
>>   - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
>>     that for acpi/numa.c (patch 2/12)
>>
>>   - Remove the duplicate NULL check for table print (patch 3/12)
>>
>>   - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
>>     dummy function for acpi_numa_arch_fixup()
>>
>>   - Solve the problem that the mapping from logical cpu to numa node
>>     is wrong which spotted out by Lorenzo
>>
>>   - cleanups for x86 and move acpi_numa_slit_init() and some other
>>     functions to common place, then reduce the duplicate of x86
>>     and arm64 (patch 7-12/12).
>>
>>   - rebased on top of 4.4 and Ganapat's v9 patch set.
>>
>> [1]: https://lkml.org/lkml/2016/4/8/571
>>
>> David Daney (1):
>>    acpi, numa, srat: Improve SRAT error detection and add messages.
>>
>> Hanjun Guo (11):
>>    acpi, numa: Use pr_fmt() instead of printk
>>    acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>>    acpi, numa: remove duplicate NULL check
>>    acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>>    arm64, numa: rework numa_add_memblk()
>>    x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>>    acpi, numa: move bad_srat() and srat_disabled() to
>>      drivers/acpi/numa.c
>>    acpi, numa: remove unneeded acpi_numa=1
>>    acpi, numa: Move acpi_numa_memory_affinity_init() to
>>      drivers/acpi/numa.c
>>    arm64, acpi, numa: NUMA support based on SRAT and SLIT
>>    acpi, numa: Enable ACPI based NUMA on ARM64
>>
>> Robert Richter (2):
>>    acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>>    arm64, acpi, numa: Default enable ACPI_NUMA with NUMA
>>
>>   arch/arm64/include/asm/acpi.h |   8 ++
>>   arch/arm64/include/asm/numa.h |   2 +
>>   arch/arm64/kernel/Makefile    |   1 +
>>   arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
>>   arch/arm64/kernel/smp.c       |   2 +
>>   arch/arm64/mm/numa.c          |  17 ++--
>>   arch/ia64/include/asm/acpi.h  |   3 +
>>   arch/ia64/kernel/acpi.c       |   2 +-
>>   arch/ia64/kernel/setup.c      |   1 +
>>   arch/x86/include/asm/acpi.h   |   1 -
>>   arch/x86/mm/numa.c            |   2 +-
>>   arch/x86/mm/srat.c            | 116 +---------------------
>>   drivers/acpi/Kconfig          |   4 +-
>>   drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
>>   drivers/of/of_numa.c          |   4 +-
>>   include/acpi/acpi_numa.h      |   4 +
>>   include/linux/acpi.h          |  18 +++-
>>   17 files changed, 375 insertions(+), 183 deletions(-)
>>   create mode 100644 arch/arm64/kernel/acpi_numa.c
>>
>> --
>> 1.7.11.7
>>

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

On 04/25/2016 04:13 AM, Will Deacon wrote:
> Hi David,
>
> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>> From: David Daney <david.daney@cavium.com>
>>
>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>> set introduce the ACPI based configuration to provide NUMA
>> information.
>>
>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>> NUMA domain information from SRAT and SLIT table, so parse those two
>> tables to get mappings from cpu/mem to numa node configuration and
>> system locality.
>
> Whilst I've queued the main NUMA series for arm64, I'd really like to
> see more movement on the generic header file cleanups that you posted
> separately:
>
> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com

FWIW:  Those patches should still apply.  I am carrying them in my 
development trees, and have not changed them in any way.

>
> Given that this ACPI series already requires some significant cross-arch
> interaction (which is actually good!), perhaps extending the clean-up
> patches to encompass some of the ACPI bits might make sense, and we can
> get that queued as a pre-requisite.

The cleanup patches you mention above are really independent of the ACPI 
things.  I have applied them both before and after the ACPI patches, and 
both seem to work.  With a quick perusal of the ACPI patches nothing 
jumps out at me as being a candidate for inclusion in the header file 
cleanup series.

If you think it would be a good idea, I can repost the cleanup series.

David Daney


>
> Will
>
>> v5 updates:
>>
>>   - Fixed ia64 build failure by gating some unused functions with #if
>>     CONFIG_{X86,ARM64}.
>>
>>   - Fixed section mismatch errors for X86 case.
>>
>>   - Removed unneeded #include from some files.
>>
>>   - Tested to build cleanly on ARM64, X86_64, IA64
>>
>> v4 updates:
>>
>>   - Updated from Hanjun Guo's v3 patches.
>>
>>   - Rebased on top of v16 of device-tree NUMA patches.
>>
>>   - Reordered some of the changes so that we don't introduce code and
>>     then change it several times in the patch set.  New code is
>>     introduced in its final form.  Code reused from x86 is first moved
>>     with no change, and then a separate patch to make any needed
>>     changes.
>>
>>   - code that is used only by ia64, moved to architecture specific
>>     files.
>>
>> v3 updates:
>>   - Deep investigation about the ACPI_DEBUG_PRINT() and remvoe
>>     that for acpi/numa.c (patch 2/12)
>>
>>   - Remove the duplicate NULL check for table print (patch 3/12)
>>
>>   - Introduce CONFIG_ACPI_HAS_NUMA_ARCH_FIXUP to remove duplicate
>>     dummy function for acpi_numa_arch_fixup()
>>
>>   - Solve the problem that the mapping from logical cpu to numa node
>>     is wrong which spotted out by Lorenzo
>>
>>   - cleanups for x86 and move acpi_numa_slit_init() and some other
>>     functions to common place, then reduce the duplicate of x86
>>     and arm64 (patch 7-12/12).
>>
>>   - rebased on top of 4.4 and Ganapat's v9 patch set.
>>
>> [1]: https://lkml.org/lkml/2016/4/8/571
>>
>> David Daney (1):
>>    acpi, numa, srat: Improve SRAT error detection and add messages.
>>
>> Hanjun Guo (11):
>>    acpi, numa: Use pr_fmt() instead of printk
>>    acpi, numa: Replace ACPI_DEBUG_PRINT() with pr_debug()
>>    acpi, numa: remove duplicate NULL check
>>    acpi, numa: move acpi_numa_slit_init() to drivers/acpi/numa.c
>>    arm64, numa: rework numa_add_memblk()
>>    x86, acpi, numa: cleanup acpi_numa_processor_affinity_init()
>>    acpi, numa: move bad_srat() and srat_disabled() to
>>      drivers/acpi/numa.c
>>    acpi, numa: remove unneeded acpi_numa=1
>>    acpi, numa: Move acpi_numa_memory_affinity_init() to
>>      drivers/acpi/numa.c
>>    arm64, acpi, numa: NUMA support based on SRAT and SLIT
>>    acpi, numa: Enable ACPI based NUMA on ARM64
>>
>> Robert Richter (2):
>>    acpi, numa: Move acpi_numa_arch_fixup() to ia64 only
>>    arm64, acpi, numa: Default enable ACPI_NUMA with NUMA
>>
>>   arch/arm64/include/asm/acpi.h |   8 ++
>>   arch/arm64/include/asm/numa.h |   2 +
>>   arch/arm64/kernel/Makefile    |   1 +
>>   arch/arm64/kernel/acpi_numa.c | 149 ++++++++++++++++++++++++++++
>>   arch/arm64/kernel/smp.c       |   2 +
>>   arch/arm64/mm/numa.c          |  17 ++--
>>   arch/ia64/include/asm/acpi.h  |   3 +
>>   arch/ia64/kernel/acpi.c       |   2 +-
>>   arch/ia64/kernel/setup.c      |   1 +
>>   arch/x86/include/asm/acpi.h   |   1 -
>>   arch/x86/mm/numa.c            |   2 +-
>>   arch/x86/mm/srat.c            | 116 +---------------------
>>   drivers/acpi/Kconfig          |   4 +-
>>   drivers/acpi/numa.c           | 224 ++++++++++++++++++++++++++++++++----------
>>   drivers/of/of_numa.c          |   4 +-
>>   include/acpi/acpi_numa.h      |   4 +
>>   include/linux/acpi.h          |  18 +++-
>>   17 files changed, 375 insertions(+), 183 deletions(-)
>>   create mode 100644 arch/arm64/kernel/acpi_numa.c
>>
>> --
>> 1.7.11.7
>>

Re: [PATCH v5 04/14] acpi, numa: Move acpi_numa_arch_fixup() to ia64 only

[Hanjun Guo]

On 2016/4/20 9:40, David Daney wrote:
> From: Robert Richter <rrichter@cavium.com>
>
> Since acpi_numa_arch_fixup() is only used in arch ia64, move it there
> to make a generic interface easier. This avoids empty function stubs
> or some complex kconfig options for x86 and arm64.
>
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>   arch/ia64/include/asm/acpi.h | 3 +++
>   arch/ia64/kernel/acpi.c      | 2 +-
>   arch/ia64/kernel/setup.c     | 1 +
>   arch/x86/mm/srat.c           | 2 --
>   drivers/acpi/numa.c          | 2 --
>   include/linux/acpi.h         | 1 -
>   6 files changed, 5 insertions(+), 6 deletions(-)
>
> diff --git a/arch/ia64/include/asm/acpi.h b/arch/ia64/include/asm/acpi.h
> index aa0fdf1..a3d0211 100644
> --- a/arch/ia64/include/asm/acpi.h
> +++ b/arch/ia64/include/asm/acpi.h
> @@ -140,6 +140,9 @@ static inline void per_cpu_scan_finalize(int min_cpus, int reserve_cpus)
>   		}
>   	}
>   }
> +
> +extern void acpi_numa_fixup(void);
> +
>   #endif /* CONFIG_ACPI_NUMA */
>
>   #endif /*__KERNEL__*/
> diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
> index b1698bc..92b7bc9 100644
> --- a/arch/ia64/kernel/acpi.c
> +++ b/arch/ia64/kernel/acpi.c
> @@ -524,7 +524,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>   	return 0;
>   }
>
> -void __init acpi_numa_arch_fixup(void)
> +void __init acpi_numa_fixup(void)
>   {
>   	int i, j, node_from, node_to;
>
> diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
> index 2029a38..afddb3e 100644
> --- a/arch/ia64/kernel/setup.c
> +++ b/arch/ia64/kernel/setup.c
> @@ -552,6 +552,7 @@ setup_arch (char **cmdline_p)
>   	early_acpi_boot_init();
>   # ifdef CONFIG_ACPI_NUMA
>   	acpi_numa_init();
> +	acpi_numa_fixup();
>   #  ifdef CONFIG_ACPI_HOTPLUG_CPU
>   	prefill_possible_map();
>   #  endif
> diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
> index b5f8218..90b6ed9 100644
> --- a/arch/x86/mm/srat.c
> +++ b/arch/x86/mm/srat.c
> @@ -212,8 +212,6 @@ out_err:
>   	return -1;
>   }
>
> -void __init acpi_numa_arch_fixup(void) {}
> -
>   int __init x86_acpi_numa_init(void)
>   {
>   	int ret;
> diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
> index 2de6068..f41f06b 100644
> --- a/drivers/acpi/numa.c
> +++ b/drivers/acpi/numa.c
> @@ -316,8 +316,6 @@ int __init acpi_numa_init(void)
>   	/* SLIT: System Locality Information Table */
>   	acpi_table_parse(ACPI_SIG_SLIT, acpi_parse_slit);
>
> -	acpi_numa_arch_fixup();
> -
>   	if (cnt < 0)
>   		return cnt;
>   	else if (!parsed_numa_memblks)
> diff --git a/include/linux/acpi.h b/include/linux/acpi.h
> index 06ed7e5..ab6fd96 100644
> --- a/include/linux/acpi.h
> +++ b/include/linux/acpi.h
> @@ -244,7 +244,6 @@ void acpi_numa_slit_init (struct acpi_table_slit *slit);
>   void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
>   void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
>   int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
> -void acpi_numa_arch_fixup(void);
>
>   #ifndef PHYS_CPUID_INVALID
>   typedef u32 phys_cpuid_t;

Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

[PATCH v5 04/14] acpi, numa: Move acpi_numa_arch_fixup() to ia64 only

[Hanjun Guo]

On 2016/4/20 9:40, David Daney wrote:
> From: Robert Richter <rrichter@cavium.com>
>
> Since acpi_numa_arch_fixup() is only used in arch ia64, move it there
> to make a generic interface easier. This avoids empty function stubs
> or some complex kconfig options for x86 and arm64.
>
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>   arch/ia64/include/asm/acpi.h | 3 +++
>   arch/ia64/kernel/acpi.c      | 2 +-
>   arch/ia64/kernel/setup.c     | 1 +
>   arch/x86/mm/srat.c           | 2 --
>   drivers/acpi/numa.c          | 2 --
>   include/linux/acpi.h         | 1 -
>   6 files changed, 5 insertions(+), 6 deletions(-)
>
> diff --git a/arch/ia64/include/asm/acpi.h b/arch/ia64/include/asm/acpi.h
> index aa0fdf1..a3d0211 100644
> --- a/arch/ia64/include/asm/acpi.h
> +++ b/arch/ia64/include/asm/acpi.h
> @@ -140,6 +140,9 @@ static inline void per_cpu_scan_finalize(int min_cpus, int reserve_cpus)
>   		}
>   	}
>   }
> +
> +extern void acpi_numa_fixup(void);
> +
>   #endif /* CONFIG_ACPI_NUMA */
>
>   #endif /*__KERNEL__*/
> diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
> index b1698bc..92b7bc9 100644
> --- a/arch/ia64/kernel/acpi.c
> +++ b/arch/ia64/kernel/acpi.c
> @@ -524,7 +524,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>   	return 0;
>   }
>
> -void __init acpi_numa_arch_fixup(void)
> +void __init acpi_numa_fixup(void)
>   {
>   	int i, j, node_from, node_to;
>
> diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
> index 2029a38..afddb3e 100644
> --- a/arch/ia64/kernel/setup.c
> +++ b/arch/ia64/kernel/setup.c
> @@ -552,6 +552,7 @@ setup_arch (char **cmdline_p)
>   	early_acpi_boot_init();
>   # ifdef CONFIG_ACPI_NUMA
>   	acpi_numa_init();
> +	acpi_numa_fixup();
>   #  ifdef CONFIG_ACPI_HOTPLUG_CPU
>   	prefill_possible_map();
>   #  endif
> diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
> index b5f8218..90b6ed9 100644
> --- a/arch/x86/mm/srat.c
> +++ b/arch/x86/mm/srat.c
> @@ -212,8 +212,6 @@ out_err:
>   	return -1;
>   }
>
> -void __init acpi_numa_arch_fixup(void) {}
> -
>   int __init x86_acpi_numa_init(void)
>   {
>   	int ret;
> diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
> index 2de6068..f41f06b 100644
> --- a/drivers/acpi/numa.c
> +++ b/drivers/acpi/numa.c
> @@ -316,8 +316,6 @@ int __init acpi_numa_init(void)
>   	/* SLIT: System Locality Information Table */
>   	acpi_table_parse(ACPI_SIG_SLIT, acpi_parse_slit);
>
> -	acpi_numa_arch_fixup();
> -
>   	if (cnt < 0)
>   		return cnt;
>   	else if (!parsed_numa_memblks)
> diff --git a/include/linux/acpi.h b/include/linux/acpi.h
> index 06ed7e5..ab6fd96 100644
> --- a/include/linux/acpi.h
> +++ b/include/linux/acpi.h
> @@ -244,7 +244,6 @@ void acpi_numa_slit_init (struct acpi_table_slit *slit);
>   void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
>   void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
>   int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
> -void acpi_numa_arch_fixup(void);
>
>   #ifndef PHYS_CPUID_INVALID
>   typedef u32 phys_cpuid_t;

Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

Re: [PATCH v5 04/14] acpi, numa: Move acpi_numa_arch_fixup() to ia64 only

[Hanjun Guo]

On 2016/4/20 9:40, David Daney wrote:
> From: Robert Richter <rrichter@cavium.com>
>
> Since acpi_numa_arch_fixup() is only used in arch ia64, move it there
> to make a generic interface easier. This avoids empty function stubs
> or some complex kconfig options for x86 and arm64.
>
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>   arch/ia64/include/asm/acpi.h | 3 +++
>   arch/ia64/kernel/acpi.c      | 2 +-
>   arch/ia64/kernel/setup.c     | 1 +
>   arch/x86/mm/srat.c           | 2 --
>   drivers/acpi/numa.c          | 2 --
>   include/linux/acpi.h         | 1 -
>   6 files changed, 5 insertions(+), 6 deletions(-)
>
> diff --git a/arch/ia64/include/asm/acpi.h b/arch/ia64/include/asm/acpi.h
> index aa0fdf1..a3d0211 100644
> --- a/arch/ia64/include/asm/acpi.h
> +++ b/arch/ia64/include/asm/acpi.h
> @@ -140,6 +140,9 @@ static inline void per_cpu_scan_finalize(int min_cpus, int reserve_cpus)
>   		}
>   	}
>   }
> +
> +extern void acpi_numa_fixup(void);
> +
>   #endif /* CONFIG_ACPI_NUMA */
>
>   #endif /*__KERNEL__*/
> diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
> index b1698bc..92b7bc9 100644
> --- a/arch/ia64/kernel/acpi.c
> +++ b/arch/ia64/kernel/acpi.c
> @@ -524,7 +524,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>   	return 0;
>   }
>
> -void __init acpi_numa_arch_fixup(void)
> +void __init acpi_numa_fixup(void)
>   {
>   	int i, j, node_from, node_to;
>
> diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
> index 2029a38..afddb3e 100644
> --- a/arch/ia64/kernel/setup.c
> +++ b/arch/ia64/kernel/setup.c
> @@ -552,6 +552,7 @@ setup_arch (char **cmdline_p)
>   	early_acpi_boot_init();
>   # ifdef CONFIG_ACPI_NUMA
>   	acpi_numa_init();
> +	acpi_numa_fixup();
>   #  ifdef CONFIG_ACPI_HOTPLUG_CPU
>   	prefill_possible_map();
>   #  endif
> diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
> index b5f8218..90b6ed9 100644
> --- a/arch/x86/mm/srat.c
> +++ b/arch/x86/mm/srat.c
> @@ -212,8 +212,6 @@ out_err:
>   	return -1;
>   }
>
> -void __init acpi_numa_arch_fixup(void) {}
> -
>   int __init x86_acpi_numa_init(void)
>   {
>   	int ret;
> diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
> index 2de6068..f41f06b 100644
> --- a/drivers/acpi/numa.c
> +++ b/drivers/acpi/numa.c
> @@ -316,8 +316,6 @@ int __init acpi_numa_init(void)
>   	/* SLIT: System Locality Information Table */
>   	acpi_table_parse(ACPI_SIG_SLIT, acpi_parse_slit);
>
> -	acpi_numa_arch_fixup();
> -
>   	if (cnt < 0)
>   		return cnt;
>   	else if (!parsed_numa_memblks)
> diff --git a/include/linux/acpi.h b/include/linux/acpi.h
> index 06ed7e5..ab6fd96 100644
> --- a/include/linux/acpi.h
> +++ b/include/linux/acpi.h
> @@ -244,7 +244,6 @@ void acpi_numa_slit_init (struct acpi_table_slit *slit);
>   void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
>   void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
>   int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
> -void acpi_numa_arch_fixup(void);
>
>   #ifndef PHYS_CPUID_INVALID
>   typedef u32 phys_cpuid_t;

Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

Re: [Devel] [PATCH v5 04/14] acpi, numa: Move acpi_numa_arch_fixup() to ia64 only

[Hanjun Guo]

[-- Attachment #1: Type: text/plain, Size: 3344 bytes --]

On 2016/4/20 9:40, David Daney wrote:
> From: Robert Richter <rrichter(a)cavium.com>
>
> Since acpi_numa_arch_fixup() is only used in arch ia64, move it there
> to make a generic interface easier. This avoids empty function stubs
> or some complex kconfig options for x86 and arm64.
>
> Signed-off-by: Robert Richter <rrichter(a)cavium.com>
> Signed-off-by: David Daney <david.daney(a)cavium.com>
> ---
>   arch/ia64/include/asm/acpi.h | 3 +++
>   arch/ia64/kernel/acpi.c      | 2 +-
>   arch/ia64/kernel/setup.c     | 1 +
>   arch/x86/mm/srat.c           | 2 --
>   drivers/acpi/numa.c          | 2 --
>   include/linux/acpi.h         | 1 -
>   6 files changed, 5 insertions(+), 6 deletions(-)
>
> diff --git a/arch/ia64/include/asm/acpi.h b/arch/ia64/include/asm/acpi.h
> index aa0fdf1..a3d0211 100644
> --- a/arch/ia64/include/asm/acpi.h
> +++ b/arch/ia64/include/asm/acpi.h
> @@ -140,6 +140,9 @@ static inline void per_cpu_scan_finalize(int min_cpus, int reserve_cpus)
>   		}
>   	}
>   }
> +
> +extern void acpi_numa_fixup(void);
> +
>   #endif /* CONFIG_ACPI_NUMA */
>
>   #endif /*__KERNEL__*/
> diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
> index b1698bc..92b7bc9 100644
> --- a/arch/ia64/kernel/acpi.c
> +++ b/arch/ia64/kernel/acpi.c
> @@ -524,7 +524,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>   	return 0;
>   }
>
> -void __init acpi_numa_arch_fixup(void)
> +void __init acpi_numa_fixup(void)
>   {
>   	int i, j, node_from, node_to;
>
> diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
> index 2029a38..afddb3e 100644
> --- a/arch/ia64/kernel/setup.c
> +++ b/arch/ia64/kernel/setup.c
> @@ -552,6 +552,7 @@ setup_arch (char **cmdline_p)
>   	early_acpi_boot_init();
>   # ifdef CONFIG_ACPI_NUMA
>   	acpi_numa_init();
> +	acpi_numa_fixup();
>   #  ifdef CONFIG_ACPI_HOTPLUG_CPU
>   	prefill_possible_map();
>   #  endif
> diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
> index b5f8218..90b6ed9 100644
> --- a/arch/x86/mm/srat.c
> +++ b/arch/x86/mm/srat.c
> @@ -212,8 +212,6 @@ out_err:
>   	return -1;
>   }
>
> -void __init acpi_numa_arch_fixup(void) {}
> -
>   int __init x86_acpi_numa_init(void)
>   {
>   	int ret;
> diff --git a/drivers/acpi/numa.c b/drivers/acpi/numa.c
> index 2de6068..f41f06b 100644
> --- a/drivers/acpi/numa.c
> +++ b/drivers/acpi/numa.c
> @@ -316,8 +316,6 @@ int __init acpi_numa_init(void)
>   	/* SLIT: System Locality Information Table */
>   	acpi_table_parse(ACPI_SIG_SLIT, acpi_parse_slit);
>
> -	acpi_numa_arch_fixup();
> -
>   	if (cnt < 0)
>   		return cnt;
>   	else if (!parsed_numa_memblks)
> diff --git a/include/linux/acpi.h b/include/linux/acpi.h
> index 06ed7e5..ab6fd96 100644
> --- a/include/linux/acpi.h
> +++ b/include/linux/acpi.h
> @@ -244,7 +244,6 @@ void acpi_numa_slit_init (struct acpi_table_slit *slit);
>   void acpi_numa_processor_affinity_init (struct acpi_srat_cpu_affinity *pa);
>   void acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa);
>   int acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma);
> -void acpi_numa_arch_fixup(void);
>
>   #ifndef PHYS_CPUID_INVALID
>   typedef u32 phys_cpuid_t;

Reviewed-by: Hanjun Guo <hanjun.guo(a)linaro.org>

Thanks
Hanjun

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

Hi Will, David,

On 2016/4/26 0:47, David Daney wrote:
> On 04/25/2016 04:13 AM, Will Deacon wrote:
>> Hi David,
>>
>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>> From: David Daney <david.daney@cavium.com>
>>>
>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>> set introduce the ACPI based configuration to provide NUMA
>>> information.
>>>
>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>> tables to get mappings from cpu/mem to numa node configuration and
>>> system locality.
>>
>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>> see more movement on the generic header file cleanups that you posted
>> separately:
>>
>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com
>>
>
> FWIW:  Those patches should still apply.  I am carrying them in my
> development trees, and have not changed them in any way.
>
>>
>> Given that this ACPI series already requires some significant cross-arch
>> interaction (which is actually good!), perhaps extending the clean-up
>> patches to encompass some of the ACPI bits might make sense, and we can
>> get that queued as a pre-requisite.
>
> The cleanup patches you mention above are really independent of the ACPI
> things.  I have applied them both before and after the ACPI patches, and
> both seem to work.  With a quick perusal of the ACPI patches nothing
> jumps out at me as being a candidate for inclusion in the header file
> cleanup series.

I agree. My patch set is ACPI related enablement, cleanups and
consolidations, it would be good to merge as a single patch set
as it's self-contained.

Thanks
Hanjun

[PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

Hi Will, David,

On 2016/4/26 0:47, David Daney wrote:
> On 04/25/2016 04:13 AM, Will Deacon wrote:
>> Hi David,
>>
>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>> From: David Daney <david.daney@cavium.com>
>>>
>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>> set introduce the ACPI based configuration to provide NUMA
>>> information.
>>>
>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>> tables to get mappings from cpu/mem to numa node configuration and
>>> system locality.
>>
>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>> see more movement on the generic header file cleanups that you posted
>> separately:
>>
>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm at gmail.com
>>
>
> FWIW:  Those patches should still apply.  I am carrying them in my
> development trees, and have not changed them in any way.
>
>>
>> Given that this ACPI series already requires some significant cross-arch
>> interaction (which is actually good!), perhaps extending the clean-up
>> patches to encompass some of the ACPI bits might make sense, and we can
>> get that queued as a pre-requisite.
>
> The cleanup patches you mention above are really independent of the ACPI
> things.  I have applied them both before and after the ACPI patches, and
> both seem to work.  With a quick perusal of the ACPI patches nothing
> jumps out at me as being a candidate for inclusion in the header file
> cleanup series.

I agree. My patch set is ACPI related enablement, cleanups and
consolidations, it would be good to merge as a single patch set
as it's self-contained.

Thanks
Hanjun

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

Hi Will, David,

On 2016/4/26 0:47, David Daney wrote:
> On 04/25/2016 04:13 AM, Will Deacon wrote:
>> Hi David,
>>
>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>> From: David Daney <david.daney@cavium.com>
>>>
>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>> set introduce the ACPI based configuration to provide NUMA
>>> information.
>>>
>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>> tables to get mappings from cpu/mem to numa node configuration and
>>> system locality.
>>
>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>> see more movement on the generic header file cleanups that you posted
>> separately:
>>
>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com
>>
>
> FWIW:  Those patches should still apply.  I am carrying them in my
> development trees, and have not changed them in any way.
>
>>
>> Given that this ACPI series already requires some significant cross-arch
>> interaction (which is actually good!), perhaps extending the clean-up
>> patches to encompass some of the ACPI bits might make sense, and we can
>> get that queued as a pre-requisite.
>
> The cleanup patches you mention above are really independent of the ACPI
> things.  I have applied them both before and after the ACPI patches, and
> both seem to work.  With a quick perusal of the ACPI patches nothing
> jumps out at me as being a candidate for inclusion in the header file
> cleanup series.

I agree. My patch set is ACPI related enablement, cleanups and
consolidations, it would be good to merge as a single patch set
as it's self-contained.

Thanks
Hanjun

Re: [Devel] [PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

[-- Attachment #1: Type: text/plain, Size: 1808 bytes --]

Hi Will, David,

On 2016/4/26 0:47, David Daney wrote:
> On 04/25/2016 04:13 AM, Will Deacon wrote:
>> Hi David,
>>
>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>> From: David Daney <david.daney(a)cavium.com>
>>>
>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>> set introduce the ACPI based configuration to provide NUMA
>>> information.
>>>
>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>> tables to get mappings from cpu/mem to numa node configuration and
>>> system locality.
>>
>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>> see more movement on the generic header file cleanups that you posted
>> separately:
>>
>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm(a)gmail.com
>>
>
> FWIW:  Those patches should still apply.  I am carrying them in my
> development trees, and have not changed them in any way.
>
>>
>> Given that this ACPI series already requires some significant cross-arch
>> interaction (which is actually good!), perhaps extending the clean-up
>> patches to encompass some of the ACPI bits might make sense, and we can
>> get that queued as a pre-requisite.
>
> The cleanup patches you mention above are really independent of the ACPI
> things.  I have applied them both before and after the ACPI patches, and
> both seem to work.  With a quick perusal of the ACPI patches nothing
> jumps out at me as being a candidate for inclusion in the header file
> cleanup series.

I agree. My patch set is ACPI related enablement, cleanups and
consolidations, it would be good to merge as a single patch set
as it's self-contained.

Thanks
Hanjun

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Will Deacon]

On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
> On 2016/4/26 0:47, David Daney wrote:
> >On 04/25/2016 04:13 AM, Will Deacon wrote:
> >>On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
> >>>From: David Daney <david.daney@cavium.com>
> >>>
> >>>Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
> >>>set introduce the ACPI based configuration to provide NUMA
> >>>information.
> >>>
> >>>ACPI 5.1 already introduced NUMA support for ARM64, which can get the
> >>>NUMA domain information from SRAT and SLIT table, so parse those two
> >>>tables to get mappings from cpu/mem to numa node configuration and
> >>>system locality.
> >>
> >>Whilst I've queued the main NUMA series for arm64, I'd really like to
> >>see more movement on the generic header file cleanups that you posted
> >>separately:
> >>
> >>http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com
> >>
> >
> >FWIW:  Those patches should still apply.  I am carrying them in my
> >development trees, and have not changed them in any way.

What's your plan for getting them merged?

> >>Given that this ACPI series already requires some significant cross-arch
> >>interaction (which is actually good!), perhaps extending the clean-up
> >>patches to encompass some of the ACPI bits might make sense, and we can
> >>get that queued as a pre-requisite.
> >
> >The cleanup patches you mention above are really independent of the ACPI
> >things.  I have applied them both before and after the ACPI patches, and
> >both seem to work.  With a quick perusal of the ACPI patches nothing
> >jumps out at me as being a candidate for inclusion in the header file
> >cleanup series.
> 
> I agree. My patch set is ACPI related enablement, cleanups and
> consolidations, it would be good to merge as a single patch set
> as it's self-contained.

Up to you. I just thought you might want to avoid having two sets of
cross-arch changes and the associated merging headaches that go with
that.

Will

[PATCH v5 00/14] ACPI NUMA support for ARM64

[Will Deacon]

On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
> On 2016/4/26 0:47, David Daney wrote:
> >On 04/25/2016 04:13 AM, Will Deacon wrote:
> >>On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
> >>>From: David Daney <david.daney@cavium.com>
> >>>
> >>>Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
> >>>set introduce the ACPI based configuration to provide NUMA
> >>>information.
> >>>
> >>>ACPI 5.1 already introduced NUMA support for ARM64, which can get the
> >>>NUMA domain information from SRAT and SLIT table, so parse those two
> >>>tables to get mappings from cpu/mem to numa node configuration and
> >>>system locality.
> >>
> >>Whilst I've queued the main NUMA series for arm64, I'd really like to
> >>see more movement on the generic header file cleanups that you posted
> >>separately:
> >>
> >>http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm at gmail.com
> >>
> >
> >FWIW:  Those patches should still apply.  I am carrying them in my
> >development trees, and have not changed them in any way.

What's your plan for getting them merged?

> >>Given that this ACPI series already requires some significant cross-arch
> >>interaction (which is actually good!), perhaps extending the clean-up
> >>patches to encompass some of the ACPI bits might make sense, and we can
> >>get that queued as a pre-requisite.
> >
> >The cleanup patches you mention above are really independent of the ACPI
> >things.  I have applied them both before and after the ACPI patches, and
> >both seem to work.  With a quick perusal of the ACPI patches nothing
> >jumps out at me as being a candidate for inclusion in the header file
> >cleanup series.
> 
> I agree. My patch set is ACPI related enablement, cleanups and
> consolidations, it would be good to merge as a single patch set
> as it's self-contained.

Up to you. I just thought you might want to avoid having two sets of
cross-arch changes and the associated merging headaches that go with
that.

Will

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Will Deacon]

On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
> On 2016/4/26 0:47, David Daney wrote:
> >On 04/25/2016 04:13 AM, Will Deacon wrote:
> >>On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
> >>>From: David Daney <david.daney@cavium.com>
> >>>
> >>>Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
> >>>set introduce the ACPI based configuration to provide NUMA
> >>>information.
> >>>
> >>>ACPI 5.1 already introduced NUMA support for ARM64, which can get the
> >>>NUMA domain information from SRAT and SLIT table, so parse those two
> >>>tables to get mappings from cpu/mem to numa node configuration and
> >>>system locality.
> >>
> >>Whilst I've queued the main NUMA series for arm64, I'd really like to
> >>see more movement on the generic header file cleanups that you posted
> >>separately:
> >>
> >>http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com
> >>
> >
> >FWIW:  Those patches should still apply.  I am carrying them in my
> >development trees, and have not changed them in any way.

What's your plan for getting them merged?

> >>Given that this ACPI series already requires some significant cross-arch
> >>interaction (which is actually good!), perhaps extending the clean-up
> >>patches to encompass some of the ACPI bits might make sense, and we can
> >>get that queued as a pre-requisite.
> >
> >The cleanup patches you mention above are really independent of the ACPI
> >things.  I have applied them both before and after the ACPI patches, and
> >both seem to work.  With a quick perusal of the ACPI patches nothing
> >jumps out at me as being a candidate for inclusion in the header file
> >cleanup series.
> 
> I agree. My patch set is ACPI related enablement, cleanups and
> consolidations, it would be good to merge as a single patch set
> as it's self-contained.

Up to you. I just thought you might want to avoid having two sets of
cross-arch changes and the associated merging headaches that go with
that.

Will

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

On 2016/4/26 20:15, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 0:47, David Daney wrote:
>>> On 04/25/2016 04:13 AM, Will Deacon wrote:
>>>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>>>> From: David Daney <david.daney@cavium.com>
>>>>>
>>>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>>>> set introduce the ACPI based configuration to provide NUMA
>>>>> information.
>>>>>
>>>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>>>> tables to get mappings from cpu/mem to numa node configuration and
>>>>> system locality.
>>>>
>>>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>>>> see more movement on the generic header file cleanups that you posted
>>>> separately:
>>>>
>>>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com
>>>>
>>>
>>> FWIW:  Those patches should still apply.  I am carrying them in my
>>> development trees, and have not changed them in any way.
>
> What's your plan for getting them merged?

This patch set touches lots of ACPI related file in arch/x86,
arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
merged via ACPI tree by Rafael with your ack to ARM64 code, does
it make sense?

>
>>>> Given that this ACPI series already requires some significant cross-arch
>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>> get that queued as a pre-requisite.
>>>
>>> The cleanup patches you mention above are really independent of the ACPI
>>> things.  I have applied them both before and after the ACPI patches, and
>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>> jumps out at me as being a candidate for inclusion in the header file
>>> cleanup series.
>>
>> I agree. My patch set is ACPI related enablement, cleanups and
>> consolidations, it would be good to merge as a single patch set
>> as it's self-contained.
>
> Up to you. I just thought you might want to avoid having two sets of
> cross-arch changes and the associated merging headaches that go with
> that.

Good point, as I suggested above, it can go with ACPI tree if it's ok
to you and Rafael. The problem we have now is that dt based core NUMA
support for ARM64 is queued in your tree, that would be the headache.

Thanks
Hanjun

[PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

On 2016/4/26 20:15, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 0:47, David Daney wrote:
>>> On 04/25/2016 04:13 AM, Will Deacon wrote:
>>>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>>>> From: David Daney <david.daney@cavium.com>
>>>>>
>>>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>>>> set introduce the ACPI based configuration to provide NUMA
>>>>> information.
>>>>>
>>>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>>>> tables to get mappings from cpu/mem to numa node configuration and
>>>>> system locality.
>>>>
>>>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>>>> see more movement on the generic header file cleanups that you posted
>>>> separately:
>>>>
>>>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm at gmail.com
>>>>
>>>
>>> FWIW:  Those patches should still apply.  I am carrying them in my
>>> development trees, and have not changed them in any way.
>
> What's your plan for getting them merged?

This patch set touches lots of ACPI related file in arch/x86,
arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
merged via ACPI tree by Rafael with your ack to ARM64 code, does
it make sense?

>
>>>> Given that this ACPI series already requires some significant cross-arch
>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>> get that queued as a pre-requisite.
>>>
>>> The cleanup patches you mention above are really independent of the ACPI
>>> things.  I have applied them both before and after the ACPI patches, and
>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>> jumps out at me as being a candidate for inclusion in the header file
>>> cleanup series.
>>
>> I agree. My patch set is ACPI related enablement, cleanups and
>> consolidations, it would be good to merge as a single patch set
>> as it's self-contained.
>
> Up to you. I just thought you might want to avoid having two sets of
> cross-arch changes and the associated merging headaches that go with
> that.

Good point, as I suggested above, it can go with ACPI tree if it's ok
to you and Rafael. The problem we have now is that dt based core NUMA
support for ARM64 is queued in your tree, that would be the headache.

Thanks
Hanjun

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

On 2016/4/26 20:15, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 0:47, David Daney wrote:
>>> On 04/25/2016 04:13 AM, Will Deacon wrote:
>>>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>>>> From: David Daney <david.daney@cavium.com>
>>>>>
>>>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>>>> set introduce the ACPI based configuration to provide NUMA
>>>>> information.
>>>>>
>>>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>>>> tables to get mappings from cpu/mem to numa node configuration and
>>>>> system locality.
>>>>
>>>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>>>> see more movement on the generic header file cleanups that you posted
>>>> separately:
>>>>
>>>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com
>>>>
>>>
>>> FWIW:  Those patches should still apply.  I am carrying them in my
>>> development trees, and have not changed them in any way.
>
> What's your plan for getting them merged?

This patch set touches lots of ACPI related file in arch/x86,
arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
merged via ACPI tree by Rafael with your ack to ARM64 code, does
it make sense?

>
>>>> Given that this ACPI series already requires some significant cross-arch
>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>> get that queued as a pre-requisite.
>>>
>>> The cleanup patches you mention above are really independent of the ACPI
>>> things.  I have applied them both before and after the ACPI patches, and
>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>> jumps out at me as being a candidate for inclusion in the header file
>>> cleanup series.
>>
>> I agree. My patch set is ACPI related enablement, cleanups and
>> consolidations, it would be good to merge as a single patch set
>> as it's self-contained.
>
> Up to you. I just thought you might want to avoid having two sets of
> cross-arch changes and the associated merging headaches that go with
> that.

Good point, as I suggested above, it can go with ACPI tree if it's ok
to you and Rafael. The problem we have now is that dt based core NUMA
support for ARM64 is queued in your tree, that would be the headache.

Thanks
Hanjun

Re: [Devel] [PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

[-- Attachment #1: Type: text/plain, Size: 2583 bytes --]

On 2016/4/26 20:15, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 0:47, David Daney wrote:
>>> On 04/25/2016 04:13 AM, Will Deacon wrote:
>>>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>>>> From: David Daney <david.daney(a)cavium.com>
>>>>>
>>>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>>>> set introduce the ACPI based configuration to provide NUMA
>>>>> information.
>>>>>
>>>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>>>> tables to get mappings from cpu/mem to numa node configuration and
>>>>> system locality.
>>>>
>>>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>>>> see more movement on the generic header file cleanups that you posted
>>>> separately:
>>>>
>>>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm(a)gmail.com
>>>>
>>>
>>> FWIW:  Those patches should still apply.  I am carrying them in my
>>> development trees, and have not changed them in any way.
>
> What's your plan for getting them merged?

This patch set touches lots of ACPI related file in arch/x86,
arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
merged via ACPI tree by Rafael with your ack to ARM64 code, does
it make sense?

>
>>>> Given that this ACPI series already requires some significant cross-arch
>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>> get that queued as a pre-requisite.
>>>
>>> The cleanup patches you mention above are really independent of the ACPI
>>> things.  I have applied them both before and after the ACPI patches, and
>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>> jumps out at me as being a candidate for inclusion in the header file
>>> cleanup series.
>>
>> I agree. My patch set is ACPI related enablement, cleanups and
>> consolidations, it would be good to merge as a single patch set
>> as it's self-contained.
>
> Up to you. I just thought you might want to avoid having two sets of
> cross-arch changes and the associated merging headaches that go with
> that.

Good point, as I suggested above, it can go with ACPI tree if it's ok
to you and Rafael. The problem we have now is that dt based core NUMA
support for ARM64 is queued in your tree, that would be the headache.

Thanks
Hanjun

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Will Deacon]

On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
> On 2016/4/26 20:15, Will Deacon wrote:
> >On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
> >>On 2016/4/26 0:47, David Daney wrote:
> >>>On 04/25/2016 04:13 AM, Will Deacon wrote:
> >>>>On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
> >>>>>From: David Daney <david.daney@cavium.com>
> >>>>>
> >>>>>Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
> >>>>>set introduce the ACPI based configuration to provide NUMA
> >>>>>information.
> >>>>>
> >>>>>ACPI 5.1 already introduced NUMA support for ARM64, which can get the
> >>>>>NUMA domain information from SRAT and SLIT table, so parse those two
> >>>>>tables to get mappings from cpu/mem to numa node configuration and
> >>>>>system locality.
> >>>>
> >>>>Whilst I've queued the main NUMA series for arm64, I'd really like to
> >>>>see more movement on the generic header file cleanups that you posted
> >>>>separately:
> >>>>
> >>>>http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com
> >>>>
> >>>
> >>>FWIW:  Those patches should still apply.  I am carrying them in my
> >>>development trees, and have not changed them in any way.
> >
> >What's your plan for getting them merged?
> 
> This patch set touches lots of ACPI related file in arch/x86,
> arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
> merged via ACPI tree by Rafael with your ack to ARM64 code, does
> it make sense?

It doesn't touch anything in drivers/acpi/... are you following the link
above?

> >>>>Given that this ACPI series already requires some significant cross-arch
> >>>>interaction (which is actually good!), perhaps extending the clean-up
> >>>>patches to encompass some of the ACPI bits might make sense, and we can
> >>>>get that queued as a pre-requisite.
> >>>
> >>>The cleanup patches you mention above are really independent of the ACPI
> >>>things.  I have applied them both before and after the ACPI patches, and
> >>>both seem to work.  With a quick perusal of the ACPI patches nothing
> >>>jumps out at me as being a candidate for inclusion in the header file
> >>>cleanup series.
> >>
> >>I agree. My patch set is ACPI related enablement, cleanups and
> >>consolidations, it would be good to merge as a single patch set
> >>as it's self-contained.
> >
> >Up to you. I just thought you might want to avoid having two sets of
> >cross-arch changes and the associated merging headaches that go with
> >that.
> 
> Good point, as I suggested above, it can go with ACPI tree if it's ok
> to you and Rafael. The problem we have now is that dt based core NUMA
> support for ARM64 is queued in your tree, that would be the headache.

Sorry, but if you wanted me *not* to queue the patches, then you should
have said so (similarly, if you wanted a stable branch). I'm not rebasing
our for-next/core branch now.

Will

[PATCH v5 00/14] ACPI NUMA support for ARM64

[Will Deacon]

On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
> On 2016/4/26 20:15, Will Deacon wrote:
> >On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
> >>On 2016/4/26 0:47, David Daney wrote:
> >>>On 04/25/2016 04:13 AM, Will Deacon wrote:
> >>>>On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
> >>>>>From: David Daney <david.daney@cavium.com>
> >>>>>
> >>>>>Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
> >>>>>set introduce the ACPI based configuration to provide NUMA
> >>>>>information.
> >>>>>
> >>>>>ACPI 5.1 already introduced NUMA support for ARM64, which can get the
> >>>>>NUMA domain information from SRAT and SLIT table, so parse those two
> >>>>>tables to get mappings from cpu/mem to numa node configuration and
> >>>>>system locality.
> >>>>
> >>>>Whilst I've queued the main NUMA series for arm64, I'd really like to
> >>>>see more movement on the generic header file cleanups that you posted
> >>>>separately:
> >>>>
> >>>>http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm at gmail.com
> >>>>
> >>>
> >>>FWIW:  Those patches should still apply.  I am carrying them in my
> >>>development trees, and have not changed them in any way.
> >
> >What's your plan for getting them merged?
> 
> This patch set touches lots of ACPI related file in arch/x86,
> arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
> merged via ACPI tree by Rafael with your ack to ARM64 code, does
> it make sense?

It doesn't touch anything in drivers/acpi/... are you following the link
above?

> >>>>Given that this ACPI series already requires some significant cross-arch
> >>>>interaction (which is actually good!), perhaps extending the clean-up
> >>>>patches to encompass some of the ACPI bits might make sense, and we can
> >>>>get that queued as a pre-requisite.
> >>>
> >>>The cleanup patches you mention above are really independent of the ACPI
> >>>things.  I have applied them both before and after the ACPI patches, and
> >>>both seem to work.  With a quick perusal of the ACPI patches nothing
> >>>jumps out at me as being a candidate for inclusion in the header file
> >>>cleanup series.
> >>
> >>I agree. My patch set is ACPI related enablement, cleanups and
> >>consolidations, it would be good to merge as a single patch set
> >>as it's self-contained.
> >
> >Up to you. I just thought you might want to avoid having two sets of
> >cross-arch changes and the associated merging headaches that go with
> >that.
> 
> Good point, as I suggested above, it can go with ACPI tree if it's ok
> to you and Rafael. The problem we have now is that dt based core NUMA
> support for ARM64 is queued in your tree, that would be the headache.

Sorry, but if you wanted me *not* to queue the patches, then you should
have said so (similarly, if you wanted a stable branch). I'm not rebasing
our for-next/core branch now.

Will

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Will Deacon]

On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
> On 2016/4/26 20:15, Will Deacon wrote:
> >On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
> >>On 2016/4/26 0:47, David Daney wrote:
> >>>On 04/25/2016 04:13 AM, Will Deacon wrote:
> >>>>On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
> >>>>>From: David Daney <david.daney@cavium.com>
> >>>>>
> >>>>>Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
> >>>>>set introduce the ACPI based configuration to provide NUMA
> >>>>>information.
> >>>>>
> >>>>>ACPI 5.1 already introduced NUMA support for ARM64, which can get the
> >>>>>NUMA domain information from SRAT and SLIT table, so parse those two
> >>>>>tables to get mappings from cpu/mem to numa node configuration and
> >>>>>system locality.
> >>>>
> >>>>Whilst I've queued the main NUMA series for arm64, I'd really like to
> >>>>see more movement on the generic header file cleanups that you posted
> >>>>separately:
> >>>>
> >>>>http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com
> >>>>
> >>>
> >>>FWIW:  Those patches should still apply.  I am carrying them in my
> >>>development trees, and have not changed them in any way.
> >
> >What's your plan for getting them merged?
> 
> This patch set touches lots of ACPI related file in arch/x86,
> arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
> merged via ACPI tree by Rafael with your ack to ARM64 code, does
> it make sense?

It doesn't touch anything in drivers/acpi/... are you following the link
above?

> >>>>Given that this ACPI series already requires some significant cross-arch
> >>>>interaction (which is actually good!), perhaps extending the clean-up
> >>>>patches to encompass some of the ACPI bits might make sense, and we can
> >>>>get that queued as a pre-requisite.
> >>>
> >>>The cleanup patches you mention above are really independent of the ACPI
> >>>things.  I have applied them both before and after the ACPI patches, and
> >>>both seem to work.  With a quick perusal of the ACPI patches nothing
> >>>jumps out at me as being a candidate for inclusion in the header file
> >>>cleanup series.
> >>
> >>I agree. My patch set is ACPI related enablement, cleanups and
> >>consolidations, it would be good to merge as a single patch set
> >>as it's self-contained.
> >
> >Up to you. I just thought you might want to avoid having two sets of
> >cross-arch changes and the associated merging headaches that go with
> >that.
> 
> Good point, as I suggested above, it can go with ACPI tree if it's ok
> to you and Rafael. The problem we have now is that dt based core NUMA
> support for ARM64 is queued in your tree, that would be the headache.

Sorry, but if you wanted me *not* to queue the patches, then you should
have said so (similarly, if you wanted a stable branch). I'm not rebasing
our for-next/core branch now.

Will

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

On 04/26/2016 06:35 AM, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 20:15, Will Deacon wrote:
>>> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>>>> On 2016/4/26 0:47, David Daney wrote:
[...]
>>>>>> Given that this ACPI series already requires some significant cross-arch
>>>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>>>> get that queued as a pre-requisite.
>>>>>
>>>>> The cleanup patches you mention above are really independent of the ACPI
>>>>> things.  I have applied them both before and after the ACPI patches, and
>>>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>>>> jumps out at me as being a candidate for inclusion in the header file
>>>>> cleanup series.
>>>>
>>>> I agree. My patch set is ACPI related enablement, cleanups and
>>>> consolidations, it would be good to merge as a single patch set
>>>> as it's self-contained.
>>>
>>> Up to you. I just thought you might want to avoid having two sets of
>>> cross-arch changes and the associated merging headaches that go with
>>> that.
>>
>> Good point, as I suggested above, it can go with ACPI tree if it's ok
>> to you and Rafael. The problem we have now is that dt based core NUMA
>> support for ARM64 is queued in your tree, that would be the headache.
>
> Sorry, but if you wanted me *not* to queue the patches, then you should
> have said so (similarly, if you wanted a stable branch). I'm not rebasing
> our for-next/core branch now.

I am quite happy with the fact that you put the base device-tree based 
NUMA patches on for-next/core.

There is only a very small adjustment to those in the ACPI-NUMA patches 
([PATCH v5 06/14] arm64, numa: rework numa_add_memblk()), so I think we 
are fine as far as that goes.

My plan is to post a v6 later today that adjusts some of the messages 
printed out and adds some Reviewed-by and Acked-by that were accumulated.

David.

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

On 04/26/2016 06:35 AM, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 20:15, Will Deacon wrote:
>>> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>>>> On 2016/4/26 0:47, David Daney wrote:
[...]
>>>>>> Given that this ACPI series already requires some significant cross-arch
>>>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>>>> get that queued as a pre-requisite.
>>>>>
>>>>> The cleanup patches you mention above are really independent of the ACPI
>>>>> things.  I have applied them both before and after the ACPI patches, and
>>>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>>>> jumps out at me as being a candidate for inclusion in the header file
>>>>> cleanup series.
>>>>
>>>> I agree. My patch set is ACPI related enablement, cleanups and
>>>> consolidations, it would be good to merge as a single patch set
>>>> as it's self-contained.
>>>
>>> Up to you. I just thought you might want to avoid having two sets of
>>> cross-arch changes and the associated merging headaches that go with
>>> that.
>>
>> Good point, as I suggested above, it can go with ACPI tree if it's ok
>> to you and Rafael. The problem we have now is that dt based core NUMA
>> support for ARM64 is queued in your tree, that would be the headache.
>
> Sorry, but if you wanted me *not* to queue the patches, then you should
> have said so (similarly, if you wanted a stable branch). I'm not rebasing
> our for-next/core branch now.

I am quite happy with the fact that you put the base device-tree based 
NUMA patches on for-next/core.

There is only a very small adjustment to those in the ACPI-NUMA patches 
([PATCH v5 06/14] arm64, numa: rework numa_add_memblk()), so I think we 
are fine as far as that goes.

My plan is to post a v6 later today that adjusts some of the messages 
printed out and adds some Reviewed-by and Acked-by that were accumulated.

David.

[PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

On 04/26/2016 06:35 AM, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 20:15, Will Deacon wrote:
>>> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>>>> On 2016/4/26 0:47, David Daney wrote:
[...]
>>>>>> Given that this ACPI series already requires some significant cross-arch
>>>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>>>> get that queued as a pre-requisite.
>>>>>
>>>>> The cleanup patches you mention above are really independent of the ACPI
>>>>> things.  I have applied them both before and after the ACPI patches, and
>>>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>>>> jumps out at me as being a candidate for inclusion in the header file
>>>>> cleanup series.
>>>>
>>>> I agree. My patch set is ACPI related enablement, cleanups and
>>>> consolidations, it would be good to merge as a single patch set
>>>> as it's self-contained.
>>>
>>> Up to you. I just thought you might want to avoid having two sets of
>>> cross-arch changes and the associated merging headaches that go with
>>> that.
>>
>> Good point, as I suggested above, it can go with ACPI tree if it's ok
>> to you and Rafael. The problem we have now is that dt based core NUMA
>> support for ARM64 is queued in your tree, that would be the headache.
>
> Sorry, but if you wanted me *not* to queue the patches, then you should
> have said so (similarly, if you wanted a stable branch). I'm not rebasing
> our for-next/core branch now.

I am quite happy with the fact that you put the base device-tree based 
NUMA patches on for-next/core.

There is only a very small adjustment to those in the ACPI-NUMA patches 
([PATCH v5 06/14] arm64, numa: rework numa_add_memblk()), so I think we 
are fine as far as that goes.

My plan is to post a v6 later today that adjusts some of the messages 
printed out and adds some Reviewed-by and Acked-by that were accumulated.

David.

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[David Daney]

On 04/26/2016 06:35 AM, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 20:15, Will Deacon wrote:
>>> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>>>> On 2016/4/26 0:47, David Daney wrote:
[...]
>>>>>> Given that this ACPI series already requires some significant cross-arch
>>>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>>>> get that queued as a pre-requisite.
>>>>>
>>>>> The cleanup patches you mention above are really independent of the ACPI
>>>>> things.  I have applied them both before and after the ACPI patches, and
>>>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>>>> jumps out at me as being a candidate for inclusion in the header file
>>>>> cleanup series.
>>>>
>>>> I agree. My patch set is ACPI related enablement, cleanups and
>>>> consolidations, it would be good to merge as a single patch set
>>>> as it's self-contained.
>>>
>>> Up to you. I just thought you might want to avoid having two sets of
>>> cross-arch changes and the associated merging headaches that go with
>>> that.
>>
>> Good point, as I suggested above, it can go with ACPI tree if it's ok
>> to you and Rafael. The problem we have now is that dt based core NUMA
>> support for ARM64 is queued in your tree, that would be the headache.
>
> Sorry, but if you wanted me *not* to queue the patches, then you should
> have said so (similarly, if you wanted a stable branch). I'm not rebasing
> our for-next/core branch now.

I am quite happy with the fact that you put the base device-tree based 
NUMA patches on for-next/core.

There is only a very small adjustment to those in the ACPI-NUMA patches 
([PATCH v5 06/14] arm64, numa: rework numa_add_memblk()), so I think we 
are fine as far as that goes.

My plan is to post a v6 later today that adjusts some of the messages 
printed out and adds some Reviewed-by and Acked-by that were accumulated.

David.

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/21/2016 03:06 AM, Dennis Chen wrote:
> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
[...]
>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
>> +{
>> +       int pxm, node;
>> +       u64 mpidr;
>> +
>> +       if (srat_disabled())
>> +               return;
>> +
>> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>> +                       pa->header.length);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>> +               return;
>> +
>> +       if (cpus_in_srat >= NR_CPUS) {
>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
>> +                            NR_CPUS);
>> +               return;
>> +       }
>> +
>> +       pxm = pa->proximity_domain;
>> +       node = acpi_map_pxm_to_node(pxm);
>> +
>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
>> +                       pxm, pa->acpi_processor_uid);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>> +       node_set(node, numa_nodes_parsed);
>> +       cpus_in_srat++;
>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>> +               pxm, mpidr, node, cpus_in_srat);
>> +}
>
> What does the *cpu* means in above pr_info function? If it's the
> logical processor ID or ACPI processor UID, then I suggest to use
> pa->acpi_processor_uid instead of cpus_in_srat, I understand the
> cpus_in_srat is just a count number of the entries of GICC Affinity
> Struct instance in SRAT, correct me if I am wrong. So at least it sees
> to me, the above pr_info will output message looks like:
> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>

Yes, that is correct, and for my system seems to be what we want as the 
names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the 
sequential numbering (0..95) with 48 CPUs on each node.

If I make the change you suggest, I get :
.
.
.
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu 65536
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu 65537
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu 65538
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu 65539
.
.
.

Not really what I would want.


> While the /sys/devices/system/cpu will use the ACPI processor UID to
> generate the index of the cpu, like:
> cpu0  cpu1  cpu2 ...
>
> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
> domain to which the logical processor belongs...
>
> Thanks,
> Dennis
>

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/21/2016 03:06 AM, Dennis Chen wrote:
> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
[...]
>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
>> +{
>> +       int pxm, node;
>> +       u64 mpidr;
>> +
>> +       if (srat_disabled())
>> +               return;
>> +
>> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>> +                       pa->header.length);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>> +               return;
>> +
>> +       if (cpus_in_srat >= NR_CPUS) {
>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
>> +                            NR_CPUS);
>> +               return;
>> +       }
>> +
>> +       pxm = pa->proximity_domain;
>> +       node = acpi_map_pxm_to_node(pxm);
>> +
>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
>> +                       pxm, pa->acpi_processor_uid);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>> +       node_set(node, numa_nodes_parsed);
>> +       cpus_in_srat++;
>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>> +               pxm, mpidr, node, cpus_in_srat);
>> +}
>
> What does the *cpu* means in above pr_info function? If it's the
> logical processor ID or ACPI processor UID, then I suggest to use
> pa->acpi_processor_uid instead of cpus_in_srat, I understand the
> cpus_in_srat is just a count number of the entries of GICC Affinity
> Struct instance in SRAT, correct me if I am wrong. So at least it sees
> to me, the above pr_info will output message looks like:
> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>

Yes, that is correct, and for my system seems to be what we want as the 
names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the 
sequential numbering (0..95) with 48 CPUs on each node.

If I make the change you suggest, I get :
.
.
.
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu 65536
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu 65537
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu 65538
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu 65539
.
.
.

Not really what I would want.


> While the /sys/devices/system/cpu will use the ACPI processor UID to
> generate the index of the cpu, like:
> cpu0  cpu1  cpu2 ...
>
> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
> domain to which the logical processor belongs...
>
> Thanks,
> Dennis
>

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/21/2016 03:06 AM, Dennis Chen wrote:
> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
[...]
>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
>> +{
>> +       int pxm, node;
>> +       u64 mpidr;
>> +
>> +       if (srat_disabled())
>> +               return;
>> +
>> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>> +                       pa->header.length);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>> +               return;
>> +
>> +       if (cpus_in_srat >= NR_CPUS) {
>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
>> +                            NR_CPUS);
>> +               return;
>> +       }
>> +
>> +       pxm = pa->proximity_domain;
>> +       node = acpi_map_pxm_to_node(pxm);
>> +
>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
>> +                       pxm, pa->acpi_processor_uid);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>> +       node_set(node, numa_nodes_parsed);
>> +       cpus_in_srat++;
>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>> +               pxm, mpidr, node, cpus_in_srat);
>> +}
>
> What does the *cpu* means in above pr_info function? If it's the
> logical processor ID or ACPI processor UID, then I suggest to use
> pa->acpi_processor_uid instead of cpus_in_srat, I understand the
> cpus_in_srat is just a count number of the entries of GICC Affinity
> Struct instance in SRAT, correct me if I am wrong. So at least it sees
> to me, the above pr_info will output message looks like:
> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>

Yes, that is correct, and for my system seems to be what we want as the 
names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the 
sequential numbering (0..95) with 48 CPUs on each node.

If I make the change you suggest, I get :
.
.
.
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu 65536
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu 65537
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu 65538
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu 65539
.
.
.

Not really what I would want.


> While the /sys/devices/system/cpu will use the ACPI processor UID to
> generate the index of the cpu, like:
> cpu0  cpu1  cpu2 ...
>
> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
> domain to which the logical processor belongs...
>
> Thanks,
> Dennis
>

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/21/2016 03:06 AM, Dennis Chen wrote:
> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
[...]
>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa)
>> +{
>> +       int pxm, node;
>> +       u64 mpidr;
>> +
>> +       if (srat_disabled())
>> +               return;
>> +
>> +       if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) {
>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>> +                       pa->header.length);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>> +               return;
>> +
>> +       if (cpus_in_srat >= NR_CPUS) {
>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n",
>> +                            NR_CPUS);
>> +               return;
>> +       }
>> +
>> +       pxm = pa->proximity_domain;
>> +       node = acpi_map_pxm_to_node(pxm);
>> +
>> +       if (node = NUMA_NO_NODE || node >= MAX_NUMNODES) {
>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n",
>> +                       pxm, pa->acpi_processor_uid);
>> +               bad_srat();
>> +               return;
>> +       }
>> +
>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>> +       node_set(node, numa_nodes_parsed);
>> +       cpus_in_srat++;
>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>> +               pxm, mpidr, node, cpus_in_srat);
>> +}
>
> What does the *cpu* means in above pr_info function? If it's the
> logical processor ID or ACPI processor UID, then I suggest to use
> pa->acpi_processor_uid instead of cpus_in_srat, I understand the
> cpus_in_srat is just a count number of the entries of GICC Affinity
> Struct instance in SRAT, correct me if I am wrong. So at least it sees
> to me, the above pr_info will output message looks like:
> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>

Yes, that is correct, and for my system seems to be what we want as the 
names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the 
sequential numbering (0..95) with 48 CPUs on each node.

If I make the change you suggest, I get :
.
.
.
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu 65536
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu 65537
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu 65538
[    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu 65539
.
.
.

Not really what I would want.


> While the /sys/devices/system/cpu will use the ACPI processor UID to
> generate the index of the cpu, like:
> cpu0  cpu1  cpu2 ...
>
> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
> domain to which the logical processor belongs...
>
> Thanks,
> Dennis
>

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

Hi Will,

On 2016/4/26 21:35, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 20:15, Will Deacon wrote:
>>> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>>>> On 2016/4/26 0:47, David Daney wrote:
>>>>> On 04/25/2016 04:13 AM, Will Deacon wrote:
>>>>>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>>>>>> From: David Daney <david.daney@cavium.com>
>>>>>>>
>>>>>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>>>>>> set introduce the ACPI based configuration to provide NUMA
>>>>>>> information.
>>>>>>>
>>>>>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>>>>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>>>>>> tables to get mappings from cpu/mem to numa node configuration and
>>>>>>> system locality.
>>>>>>
>>>>>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>>>>>> see more movement on the generic header file cleanups that you posted
>>>>>> separately:
>>>>>>
>>>>>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com
>>>>>>
>>>>>
>>>>> FWIW:  Those patches should still apply.  I am carrying them in my
>>>>> development trees, and have not changed them in any way.
>>>
>>> What's your plan for getting them merged?
>>
>> This patch set touches lots of ACPI related file in arch/x86,
>> arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
>> merged via ACPI tree by Rafael with your ack to ARM64 code, does
>> it make sense?
>
> It doesn't touch anything in drivers/acpi/... are you following the link
> above?

Sorry, my bad, I though you were talking about this ACPI NUMA support
for ARM64 patch set.

>
>>>>>> Given that this ACPI series already requires some significant cross-arch
>>>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>>>> get that queued as a pre-requisite.
>>>>>
>>>>> The cleanup patches you mention above are really independent of the ACPI
>>>>> things.  I have applied them both before and after the ACPI patches, and
>>>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>>>> jumps out at me as being a candidate for inclusion in the header file
>>>>> cleanup series.
>>>>
>>>> I agree. My patch set is ACPI related enablement, cleanups and
>>>> consolidations, it would be good to merge as a single patch set
>>>> as it's self-contained.
>>>
>>> Up to you. I just thought you might want to avoid having two sets of
>>> cross-arch changes and the associated merging headaches that go with
>>> that.
>>
>> Good point, as I suggested above, it can go with ACPI tree if it's ok
>> to you and Rafael. The problem we have now is that dt based core NUMA
>> support for ARM64 is queued in your tree, that would be the headache.
>
> Sorry, but if you wanted me *not* to queue the patches, then you should
> have said so (similarly, if you wanted a stable branch). I'm not rebasing
> our for-next/core branch now.

I misread the message above, I'm really sorry if I did something
offending you, I didn't mean that.

How about this patch set? We only get few comments on it, your
comments on it are appreciated.

Thanks
Hanjun

[PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

Hi Will,

On 2016/4/26 21:35, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 20:15, Will Deacon wrote:
>>> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>>>> On 2016/4/26 0:47, David Daney wrote:
>>>>> On 04/25/2016 04:13 AM, Will Deacon wrote:
>>>>>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>>>>>> From: David Daney <david.daney@cavium.com>
>>>>>>>
>>>>>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>>>>>> set introduce the ACPI based configuration to provide NUMA
>>>>>>> information.
>>>>>>>
>>>>>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>>>>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>>>>>> tables to get mappings from cpu/mem to numa node configuration and
>>>>>>> system locality.
>>>>>>
>>>>>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>>>>>> see more movement on the generic header file cleanups that you posted
>>>>>> separately:
>>>>>>
>>>>>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm at gmail.com
>>>>>>
>>>>>
>>>>> FWIW:  Those patches should still apply.  I am carrying them in my
>>>>> development trees, and have not changed them in any way.
>>>
>>> What's your plan for getting them merged?
>>
>> This patch set touches lots of ACPI related file in arch/x86,
>> arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
>> merged via ACPI tree by Rafael with your ack to ARM64 code, does
>> it make sense?
>
> It doesn't touch anything in drivers/acpi/... are you following the link
> above?

Sorry, my bad, I though you were talking about this ACPI NUMA support
for ARM64 patch set.

>
>>>>>> Given that this ACPI series already requires some significant cross-arch
>>>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>>>> get that queued as a pre-requisite.
>>>>>
>>>>> The cleanup patches you mention above are really independent of the ACPI
>>>>> things.  I have applied them both before and after the ACPI patches, and
>>>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>>>> jumps out at me as being a candidate for inclusion in the header file
>>>>> cleanup series.
>>>>
>>>> I agree. My patch set is ACPI related enablement, cleanups and
>>>> consolidations, it would be good to merge as a single patch set
>>>> as it's self-contained.
>>>
>>> Up to you. I just thought you might want to avoid having two sets of
>>> cross-arch changes and the associated merging headaches that go with
>>> that.
>>
>> Good point, as I suggested above, it can go with ACPI tree if it's ok
>> to you and Rafael. The problem we have now is that dt based core NUMA
>> support for ARM64 is queued in your tree, that would be the headache.
>
> Sorry, but if you wanted me *not* to queue the patches, then you should
> have said so (similarly, if you wanted a stable branch). I'm not rebasing
> our for-next/core branch now.

I misread the message above, I'm really sorry if I did something
offending you, I didn't mean that.

How about this patch set? We only get few comments on it, your
comments on it are appreciated.

Thanks
Hanjun

Re: [PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

Hi Will,

On 2016/4/26 21:35, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 20:15, Will Deacon wrote:
>>> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>>>> On 2016/4/26 0:47, David Daney wrote:
>>>>> On 04/25/2016 04:13 AM, Will Deacon wrote:
>>>>>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>>>>>> From: David Daney <david.daney@cavium.com>
>>>>>>>
>>>>>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>>>>>> set introduce the ACPI based configuration to provide NUMA
>>>>>>> information.
>>>>>>>
>>>>>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>>>>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>>>>>> tables to get mappings from cpu/mem to numa node configuration and
>>>>>>> system locality.
>>>>>>
>>>>>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>>>>>> see more movement on the generic header file cleanups that you posted
>>>>>> separately:
>>>>>>
>>>>>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm@gmail.com
>>>>>>
>>>>>
>>>>> FWIW:  Those patches should still apply.  I am carrying them in my
>>>>> development trees, and have not changed them in any way.
>>>
>>> What's your plan for getting them merged?
>>
>> This patch set touches lots of ACPI related file in arch/x86,
>> arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
>> merged via ACPI tree by Rafael with your ack to ARM64 code, does
>> it make sense?
>
> It doesn't touch anything in drivers/acpi/... are you following the link
> above?

Sorry, my bad, I though you were talking about this ACPI NUMA support
for ARM64 patch set.

>
>>>>>> Given that this ACPI series already requires some significant cross-arch
>>>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>>>> get that queued as a pre-requisite.
>>>>>
>>>>> The cleanup patches you mention above are really independent of the ACPI
>>>>> things.  I have applied them both before and after the ACPI patches, and
>>>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>>>> jumps out at me as being a candidate for inclusion in the header file
>>>>> cleanup series.
>>>>
>>>> I agree. My patch set is ACPI related enablement, cleanups and
>>>> consolidations, it would be good to merge as a single patch set
>>>> as it's self-contained.
>>>
>>> Up to you. I just thought you might want to avoid having two sets of
>>> cross-arch changes and the associated merging headaches that go with
>>> that.
>>
>> Good point, as I suggested above, it can go with ACPI tree if it's ok
>> to you and Rafael. The problem we have now is that dt based core NUMA
>> support for ARM64 is queued in your tree, that would be the headache.
>
> Sorry, but if you wanted me *not* to queue the patches, then you should
> have said so (similarly, if you wanted a stable branch). I'm not rebasing
> our for-next/core branch now.

I misread the message above, I'm really sorry if I did something
offending you, I didn't mean that.

How about this patch set? We only get few comments on it, your
comments on it are appreciated.

Thanks
Hanjun

Re: [Devel] [PATCH v5 00/14] ACPI NUMA support for ARM64

[Hanjun Guo]

[-- Attachment #1: Type: text/plain, Size: 3389 bytes --]

Hi Will,

On 2016/4/26 21:35, Will Deacon wrote:
> On Tue, Apr 26, 2016 at 09:03:25PM +0800, Hanjun Guo wrote:
>> On 2016/4/26 20:15, Will Deacon wrote:
>>> On Tue, Apr 26, 2016 at 01:31:07PM +0800, Hanjun Guo wrote:
>>>> On 2016/4/26 0:47, David Daney wrote:
>>>>> On 04/25/2016 04:13 AM, Will Deacon wrote:
>>>>>> On Tue, Apr 19, 2016 at 06:40:25PM -0700, David Daney wrote:
>>>>>>> From: David Daney <david.daney(a)cavium.com>
>>>>>>>
>>>>>>> Based on v16 of device-tree NUMA patch set for arm64 [1],this patch
>>>>>>> set introduce the ACPI based configuration to provide NUMA
>>>>>>> information.
>>>>>>>
>>>>>>> ACPI 5.1 already introduced NUMA support for ARM64, which can get the
>>>>>>> NUMA domain information from SRAT and SLIT table, so parse those two
>>>>>>> tables to get mappings from cpu/mem to numa node configuration and
>>>>>>> system locality.
>>>>>>
>>>>>> Whilst I've queued the main NUMA series for arm64, I'd really like to
>>>>>> see more movement on the generic header file cleanups that you posted
>>>>>> separately:
>>>>>>
>>>>>> http://lkml.kernel.org/r/1456358528-24213-1-git-send-email-ddaney.cavm(a)gmail.com
>>>>>>
>>>>>
>>>>> FWIW:  Those patches should still apply.  I am carrying them in my
>>>>> development trees, and have not changed them in any way.
>>>
>>> What's your plan for getting them merged?
>>
>> This patch set touches lots of ACPI related file in arch/x86,
>> arch/ia64, and drivers/acpi/ (also arch/arm64), I think it can be
>> merged via ACPI tree by Rafael with your ack to ARM64 code, does
>> it make sense?
>
> It doesn't touch anything in drivers/acpi/... are you following the link
> above?

Sorry, my bad, I though you were talking about this ACPI NUMA support
for ARM64 patch set.

>
>>>>>> Given that this ACPI series already requires some significant cross-arch
>>>>>> interaction (which is actually good!), perhaps extending the clean-up
>>>>>> patches to encompass some of the ACPI bits might make sense, and we can
>>>>>> get that queued as a pre-requisite.
>>>>>
>>>>> The cleanup patches you mention above are really independent of the ACPI
>>>>> things.  I have applied them both before and after the ACPI patches, and
>>>>> both seem to work.  With a quick perusal of the ACPI patches nothing
>>>>> jumps out at me as being a candidate for inclusion in the header file
>>>>> cleanup series.
>>>>
>>>> I agree. My patch set is ACPI related enablement, cleanups and
>>>> consolidations, it would be good to merge as a single patch set
>>>> as it's self-contained.
>>>
>>> Up to you. I just thought you might want to avoid having two sets of
>>> cross-arch changes and the associated merging headaches that go with
>>> that.
>>
>> Good point, as I suggested above, it can go with ACPI tree if it's ok
>> to you and Rafael. The problem we have now is that dt based core NUMA
>> support for ARM64 is queued in your tree, that would be the headache.
>
> Sorry, but if you wanted me *not* to queue the patches, then you should
> have said so (similarly, if you wanted a stable branch). I'm not rebasing
> our for-next/core branch now.

I misread the message above, I'm really sorry if I did something
offending you, I didn't mean that.

How about this patch set? We only get few comments on it, your
comments on it are appreciated.

Thanks
Hanjun

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Hanjun Guo]

Hi Dennis, David,

Sorry for the late reply, please see my comments below.

On 2016/4/27 9:14, David Daney wrote:
> On 04/21/2016 03:06 AM, Dennis Chen wrote:
>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> [...]
>>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>>> +void __init acpi_numa_gicc_affinity_init(struct
>>> acpi_srat_gicc_affinity *pa)
>>> +{
>>> +       int pxm, node;
>>> +       u64 mpidr;
>>> +
>>> +       if (srat_disabled())
>>> +               return;
>>> +
>>> +       if (pa->header.length < sizeof(struct
>>> acpi_srat_gicc_affinity)) {
>>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>>> +                       pa->header.length);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>>> +               return;
>>> +
>>> +       if (cpus_in_srat >= NR_CPUS) {
>>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small,
>>> may not be able to use all cpus\n",
>>> +                            NR_CPUS);
>>> +               return;
>>> +       }
>>> +
>>> +       pxm = pa->proximity_domain;
>>> +       node = acpi_map_pxm_to_node(pxm);
>>> +
>>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid
>>> MPIDR in MADT\n",
>>> +                       pxm, pa->acpi_processor_uid);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>>> +       node_set(node, numa_nodes_parsed);
>>> +       cpus_in_srat++;
>>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>>> +               pxm, mpidr, node, cpus_in_srat);
>>> +}
>>
>> What does the *cpu* means in above pr_info function? If it's the
>> logical processor ID or ACPI processor UID, then I suggest to use
>> pa->acpi_processor_uid instead of cpus_in_srat, I understand the

I think print cpus_in_srat is pointless here, as the logic cpu number
is allocated by OS when initializing SMP by scanning MADT table. As
Dennis said, it's just a count number, not a number mapping to MPIDR.

ACPI processor UID is the key value to connect MADT, SRAT, DSDT.

For MADT, it will have MPIDR and ACPI processor UID, and OS will
create mappings to MPIDR and cpu logical number,
ACPI processor UID <------> MPIDR <------> CPU logical number

In SRAT, there is ACPI processor UID represented, mappings will be
ACPI processor UID <------> PXM <------> NUMA node logical number

So we can use ACPI processor UID to get the MPIDR by scanning the
MADT, then we can map NUMA node logical number to cpu logical
number later.

>> cpus_in_srat is just a count number of the entries of GICC Affinity
>> Struct instance in SRAT, correct me if I am wrong. So at least it sees
>> to me, the above pr_info will output message looks like:
>> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
>> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
>> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>>
>
> Yes, that is correct, and for my system seems to be what we want as the
> names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the
> sequential numbering (0..95) with 48 CPUs on each node.

That's because you place CPUs in the same order both in MADT and SRAT :)
if not, that will be not match.

>
> If I make the change you suggest, I get :
> .
> .
> .
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu 65536
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu 65537
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu 65538
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu 65539
> .
> .
> .
>
> Not really what I would want.

How about remove the print for "cpu"? it's not the right value we want,
and we can get such mapping information under in sysfs.

>
>
>> While the /sys/devices/system/cpu will use the ACPI processor UID to
>> generate the index of the cpu, like:
>> cpu0  cpu1  cpu2 ...
>>
>> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
>> domain to which the logical processor belongs...

Yes, we can get such information in /sys/devices/system/node, I think
we can only print:

ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z

Thanks
Hanjun

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Hanjun Guo]

Hi Dennis, David,

Sorry for the late reply, please see my comments below.

On 2016/4/27 9:14, David Daney wrote:
> On 04/21/2016 03:06 AM, Dennis Chen wrote:
>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> [...]
>>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>>> +void __init acpi_numa_gicc_affinity_init(struct
>>> acpi_srat_gicc_affinity *pa)
>>> +{
>>> +       int pxm, node;
>>> +       u64 mpidr;
>>> +
>>> +       if (srat_disabled())
>>> +               return;
>>> +
>>> +       if (pa->header.length < sizeof(struct
>>> acpi_srat_gicc_affinity)) {
>>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>>> +                       pa->header.length);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>>> +               return;
>>> +
>>> +       if (cpus_in_srat >= NR_CPUS) {
>>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small,
>>> may not be able to use all cpus\n",
>>> +                            NR_CPUS);
>>> +               return;
>>> +       }
>>> +
>>> +       pxm = pa->proximity_domain;
>>> +       node = acpi_map_pxm_to_node(pxm);
>>> +
>>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid
>>> MPIDR in MADT\n",
>>> +                       pxm, pa->acpi_processor_uid);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>>> +       node_set(node, numa_nodes_parsed);
>>> +       cpus_in_srat++;
>>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>>> +               pxm, mpidr, node, cpus_in_srat);
>>> +}
>>
>> What does the *cpu* means in above pr_info function? If it's the
>> logical processor ID or ACPI processor UID, then I suggest to use
>> pa->acpi_processor_uid instead of cpus_in_srat, I understand the

I think print cpus_in_srat is pointless here, as the logic cpu number
is allocated by OS when initializing SMP by scanning MADT table. As
Dennis said, it's just a count number, not a number mapping to MPIDR.

ACPI processor UID is the key value to connect MADT, SRAT, DSDT.

For MADT, it will have MPIDR and ACPI processor UID, and OS will
create mappings to MPIDR and cpu logical number,
ACPI processor UID <------> MPIDR <------> CPU logical number

In SRAT, there is ACPI processor UID represented, mappings will be
ACPI processor UID <------> PXM <------> NUMA node logical number

So we can use ACPI processor UID to get the MPIDR by scanning the
MADT, then we can map NUMA node logical number to cpu logical
number later.

>> cpus_in_srat is just a count number of the entries of GICC Affinity
>> Struct instance in SRAT, correct me if I am wrong. So at least it sees
>> to me, the above pr_info will output message looks like:
>> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
>> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
>> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>>
>
> Yes, that is correct, and for my system seems to be what we want as the
> names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the
> sequential numbering (0..95) with 48 CPUs on each node.

That's because you place CPUs in the same order both in MADT and SRAT :)
if not, that will be not match.

>
> If I make the change you suggest, I get :
> .
> .
> .
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu 65536
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu 65537
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu 65538
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu 65539
> .
> .
> .
>
> Not really what I would want.

How about remove the print for "cpu"? it's not the right value we want,
and we can get such mapping information under in sysfs.

>
>
>> While the /sys/devices/system/cpu will use the ACPI processor UID to
>> generate the index of the cpu, like:
>> cpu0  cpu1  cpu2 ...
>>
>> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
>> domain to which the logical processor belongs...

Yes, we can get such information in /sys/devices/system/node, I think
we can only print:

ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z

Thanks
Hanjun

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Hanjun Guo]

Hi Dennis, David,

Sorry for the late reply, please see my comments below.

On 2016/4/27 9:14, David Daney wrote:
> On 04/21/2016 03:06 AM, Dennis Chen wrote:
>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> [...]
>>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>>> +void __init acpi_numa_gicc_affinity_init(struct
>>> acpi_srat_gicc_affinity *pa)
>>> +{
>>> +       int pxm, node;
>>> +       u64 mpidr;
>>> +
>>> +       if (srat_disabled())
>>> +               return;
>>> +
>>> +       if (pa->header.length < sizeof(struct
>>> acpi_srat_gicc_affinity)) {
>>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>>> +                       pa->header.length);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>>> +               return;
>>> +
>>> +       if (cpus_in_srat >= NR_CPUS) {
>>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small,
>>> may not be able to use all cpus\n",
>>> +                            NR_CPUS);
>>> +               return;
>>> +       }
>>> +
>>> +       pxm = pa->proximity_domain;
>>> +       node = acpi_map_pxm_to_node(pxm);
>>> +
>>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid
>>> MPIDR in MADT\n",
>>> +                       pxm, pa->acpi_processor_uid);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>>> +       node_set(node, numa_nodes_parsed);
>>> +       cpus_in_srat++;
>>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>>> +               pxm, mpidr, node, cpus_in_srat);
>>> +}
>>
>> What does the *cpu* means in above pr_info function? If it's the
>> logical processor ID or ACPI processor UID, then I suggest to use
>> pa->acpi_processor_uid instead of cpus_in_srat, I understand the

I think print cpus_in_srat is pointless here, as the logic cpu number
is allocated by OS when initializing SMP by scanning MADT table. As
Dennis said, it's just a count number, not a number mapping to MPIDR.

ACPI processor UID is the key value to connect MADT, SRAT, DSDT.

For MADT, it will have MPIDR and ACPI processor UID, and OS will
create mappings to MPIDR and cpu logical number,
ACPI processor UID <------> MPIDR <------> CPU logical number

In SRAT, there is ACPI processor UID represented, mappings will be
ACPI processor UID <------> PXM <------> NUMA node logical number

So we can use ACPI processor UID to get the MPIDR by scanning the
MADT, then we can map NUMA node logical number to cpu logical
number later.

>> cpus_in_srat is just a count number of the entries of GICC Affinity
>> Struct instance in SRAT, correct me if I am wrong. So at least it sees
>> to me, the above pr_info will output message looks like:
>> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
>> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
>> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>>
>
> Yes, that is correct, and for my system seems to be what we want as the
> names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the
> sequential numbering (0..95) with 48 CPUs on each node.

That's because you place CPUs in the same order both in MADT and SRAT :)
if not, that will be not match.

>
> If I make the change you suggest, I get :
> .
> .
> .
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu 65536
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu 65537
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu 65538
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu 65539
> .
> .
> .
>
> Not really what I would want.

How about remove the print for "cpu"? it's not the right value we want,
and we can get such mapping information under in sysfs.

>
>
>> While the /sys/devices/system/cpu will use the ACPI processor UID to
>> generate the index of the cpu, like:
>> cpu0  cpu1  cpu2 ...
>>
>> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
>> domain to which the logical processor belongs...

Yes, we can get such information in /sys/devices/system/node, I think
we can only print:

ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z

Thanks
Hanjun

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Hanjun Guo]

Hi Dennis, David,

Sorry for the late reply, please see my comments below.

On 2016/4/27 9:14, David Daney wrote:
> On 04/21/2016 03:06 AM, Dennis Chen wrote:
>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
> [...]
>>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>>> +void __init acpi_numa_gicc_affinity_init(struct
>>> acpi_srat_gicc_affinity *pa)
>>> +{
>>> +       int pxm, node;
>>> +       u64 mpidr;
>>> +
>>> +       if (srat_disabled())
>>> +               return;
>>> +
>>> +       if (pa->header.length < sizeof(struct
>>> acpi_srat_gicc_affinity)) {
>>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>>> +                       pa->header.length);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>>> +               return;
>>> +
>>> +       if (cpus_in_srat >= NR_CPUS) {
>>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small,
>>> may not be able to use all cpus\n",
>>> +                            NR_CPUS);
>>> +               return;
>>> +       }
>>> +
>>> +       pxm = pa->proximity_domain;
>>> +       node = acpi_map_pxm_to_node(pxm);
>>> +
>>> +       if (node = NUMA_NO_NODE || node >= MAX_NUMNODES) {
>>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid
>>> MPIDR in MADT\n",
>>> +                       pxm, pa->acpi_processor_uid);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>>> +       node_set(node, numa_nodes_parsed);
>>> +       cpus_in_srat++;
>>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>>> +               pxm, mpidr, node, cpus_in_srat);
>>> +}
>>
>> What does the *cpu* means in above pr_info function? If it's the
>> logical processor ID or ACPI processor UID, then I suggest to use
>> pa->acpi_processor_uid instead of cpus_in_srat, I understand the

I think print cpus_in_srat is pointless here, as the logic cpu number
is allocated by OS when initializing SMP by scanning MADT table. As
Dennis said, it's just a count number, not a number mapping to MPIDR.

ACPI processor UID is the key value to connect MADT, SRAT, DSDT.

For MADT, it will have MPIDR and ACPI processor UID, and OS will
create mappings to MPIDR and cpu logical number,
ACPI processor UID <------> MPIDR <------> CPU logical number

In SRAT, there is ACPI processor UID represented, mappings will be
ACPI processor UID <------> PXM <------> NUMA node logical number

So we can use ACPI processor UID to get the MPIDR by scanning the
MADT, then we can map NUMA node logical number to cpu logical
number later.

>> cpus_in_srat is just a count number of the entries of GICC Affinity
>> Struct instance in SRAT, correct me if I am wrong. So at least it sees
>> to me, the above pr_info will output message looks like:
>> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
>> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
>> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>>
>
> Yes, that is correct, and for my system seems to be what we want as the
> names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the
> sequential numbering (0..95) with 48 CPUs on each node.

That's because you place CPUs in the same order both in MADT and SRAT :)
if not, that will be not match.

>
> If I make the change you suggest, I get :
> .
> .
> .
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu 65536
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu 65537
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu 65538
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu 65539
> .
> .
> .
>
> Not really what I would want.

How about remove the print for "cpu"? it's not the right value we want,
and we can get such mapping information under in sysfs.

>
>
>> While the /sys/devices/system/cpu will use the ACPI processor UID to
>> generate the index of the cpu, like:
>> cpu0  cpu1  cpu2 ...
>>
>> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
>> domain to which the logical processor belongs...

Yes, we can get such information in /sys/devices/system/node, I think
we can only print:

ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z

Thanks
Hanjun

Re: [Devel] [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Hanjun Guo]

[-- Attachment #1: Type: text/plain, Size: 8323 bytes --]

Hi Dennis, David,

Sorry for the late reply, please see my comments below.

On 2016/4/27 9:14, David Daney wrote:
> On 04/21/2016 03:06 AM, Dennis Chen wrote:
>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm(a)gmail.com> wrote:
> [...]
>>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>>> +void __init acpi_numa_gicc_affinity_init(struct
>>> acpi_srat_gicc_affinity *pa)
>>> +{
>>> +       int pxm, node;
>>> +       u64 mpidr;
>>> +
>>> +       if (srat_disabled())
>>> +               return;
>>> +
>>> +       if (pa->header.length < sizeof(struct
>>> acpi_srat_gicc_affinity)) {
>>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>>> +                       pa->header.length);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>>> +               return;
>>> +
>>> +       if (cpus_in_srat >= NR_CPUS) {
>>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small,
>>> may not be able to use all cpus\n",
>>> +                            NR_CPUS);
>>> +               return;
>>> +       }
>>> +
>>> +       pxm = pa->proximity_domain;
>>> +       node = acpi_map_pxm_to_node(pxm);
>>> +
>>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid
>>> MPIDR in MADT\n",
>>> +                       pxm, pa->acpi_processor_uid);
>>> +               bad_srat();
>>> +               return;
>>> +       }
>>> +
>>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>>> +       node_set(node, numa_nodes_parsed);
>>> +       cpus_in_srat++;
>>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>>> +               pxm, mpidr, node, cpus_in_srat);
>>> +}
>>
>> What does the *cpu* means in above pr_info function? If it's the
>> logical processor ID or ACPI processor UID, then I suggest to use
>> pa->acpi_processor_uid instead of cpus_in_srat, I understand the

I think print cpus_in_srat is pointless here, as the logic cpu number
is allocated by OS when initializing SMP by scanning MADT table. As
Dennis said, it's just a count number, not a number mapping to MPIDR.

ACPI processor UID is the key value to connect MADT, SRAT, DSDT.

For MADT, it will have MPIDR and ACPI processor UID, and OS will
create mappings to MPIDR and cpu logical number,
ACPI processor UID <------> MPIDR <------> CPU logical number

In SRAT, there is ACPI processor UID represented, mappings will be
ACPI processor UID <------> PXM <------> NUMA node logical number

So we can use ACPI processor UID to get the MPIDR by scanning the
MADT, then we can map NUMA node logical number to cpu logical
number later.

>> cpus_in_srat is just a count number of the entries of GICC Affinity
>> Struct instance in SRAT, correct me if I am wrong. So at least it sees
>> to me, the above pr_info will output message looks like:
>> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
>> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
>> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>>
>
> Yes, that is correct, and for my system seems to be what we want as the
> names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the
> sequential numbering (0..95) with 48 CPUs on each node.

That's because you place CPUs in the same order both in MADT and SRAT :)
if not, that will be not match.

>
> If I make the change you suggest, I get :
> .
> .
> .
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu 65536
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu 65537
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu 65538
> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu 65539
> .
> .
> .
>
> Not really what I would want.

How about remove the print for "cpu"? it's not the right value we want,
and we can get such mapping information under in sysfs.

>
>
>> While the /sys/devices/system/cpu will use the ACPI processor UID to
>> generate the index of the cpu, like:
>> cpu0  cpu1  cpu2 ...
>>
>> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
>> domain to which the logical processor belongs...

Yes, we can get such information in /sys/devices/system/node, I think
we can only print:

ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z

Thanks
Hanjun

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

Hi Hanjun,

Thanks for the clarification and some little comments ;-)

On 27 April 2016 at 12:04, Hanjun Guo <hanjun.guo@linaro.org> wrote:
> Hi Dennis, David,
>
> Sorry for the late reply, please see my comments below.
>
>
> On 2016/4/27 9:14, David Daney wrote:
>>
>> On 04/21/2016 03:06 AM, Dennis Chen wrote:
>>>
>>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
>>
>> [...]
>>>>
>>>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>>>> +void __init acpi_numa_gicc_affinity_init(struct
>>>> acpi_srat_gicc_affinity *pa)
>>>> +{
>>>> +       int pxm, node;
>>>> +       u64 mpidr;
>>>> +
>>>> +       if (srat_disabled())
>>>> +               return;
>>>> +
>>>> +       if (pa->header.length < sizeof(struct
>>>> acpi_srat_gicc_affinity)) {
>>>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>>>> +                       pa->header.length);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>>>> +               return;
>>>> +
>>>> +       if (cpus_in_srat >= NR_CPUS) {
>>>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small,
>>>> may not be able to use all cpus\n",
>>>> +                            NR_CPUS);
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       pxm = pa->proximity_domain;
>>>> +       node = acpi_map_pxm_to_node(pxm);
>>>> +
>>>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>>>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>>>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid
>>>> MPIDR in MADT\n",
>>>> +                       pxm, pa->acpi_processor_uid);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>>>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>>>> +       node_set(node, numa_nodes_parsed);
>>>> +       cpus_in_srat++;
>>>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>>>> +               pxm, mpidr, node, cpus_in_srat);
>>>> +}
>>>
>>>
>>> What does the *cpu* means in above pr_info function? If it's the
>>> logical processor ID or ACPI processor UID, then I suggest to use
>>> pa->acpi_processor_uid instead of cpus_in_srat, I understand the
>
>
> I think print cpus_in_srat is pointless here, as the logic cpu number
> is allocated by OS when initializing SMP by scanning MADT table. As
> Dennis said, it's just a count number, not a number mapping to MPIDR.
>
> ACPI processor UID is the key value to connect MADT, SRAT, DSDT.
>
> For MADT, it will have MPIDR and ACPI processor UID, and OS will
> create mappings to MPIDR and cpu logical number,
> ACPI processor UID <------> MPIDR <------> CPU logical number
>
> In SRAT, there is ACPI processor UID represented, mappings will be
> ACPI processor UID <------> PXM <------> NUMA node logical number
>
> So we can use ACPI processor UID to get the MPIDR by scanning the
> MADT, then we can map NUMA node logical number to cpu logical
> number later.
>
Right, kernel will record the logical cpu index info (begins from 0,
the boot cpu) into the cpu_possible bit map by parsing the MADT GICC
sub-table. So I am thinking here if we can reduce the parsing walk by
only one, because I see the acpi_numa_gicc_affinity_init() calls
get_mpidr_in_madt() to traverse the entire MADT, actually the kernel
will also traverse the MADT in smp_init_cpus(), merge them into one?
>
>>> cpus_in_srat is just a count number of the entries of GICC Affinity
>>> Struct instance in SRAT, correct me if I am wrong. So at least it sees
>>> to me, the above pr_info will output message looks like:
>>> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
>>> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
>>> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>>>
>>
>> Yes, that is correct, and for my system seems to be what we want as the
>> names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the
>> sequential numbering (0..95) with 48 CPUs on each node.
>
>
> That's because you place CPUs in the same order both in MADT and SRAT :)
> if not, that will be not match.
>
Hmm, I think I made a mistake presumption here that the ACPI processor
UID is the logical processor number, just take a look at the ACPI
processor driver module code, the logic process number should comes
from the bit map. And from the below output message pasted by David,
we can see that his firmware is using MPIDR as the ACPI processor UID,
it's not incorrect implementation according to the ACPI spec...
>
>>
>> If I make the change you suggest, I get :
>> .
>> .
>> .
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu
>> 65536
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu
>> 65537
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu
>> 65538
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu
>> 65539
>> .
>> .
>> .
>>
>> Not really what I would want.
>
>
> How about remove the print for "cpu"? it's not the right value we want,
> and we can get such mapping information under in sysfs.
>
I think you may remove the *cpu* in the print, indeed it's pointless.
You can, IMO, use cpu_possible bitmask if you want to get the logical
processor number just as the ACPI processor does. About the
early_node_cpu_hwid, the only reason I can see now is to map logic cpu
into the numa node number, can we refer to the implementation under
x86, anyway I am not sure since not very familiar with that codes
dependency...
>
>>
>>
>>> While the /sys/devices/system/cpu will use the ACPI processor UID to
>>> generate the index of the cpu, like:
>>> cpu0  cpu1  cpu2 ...
>>>
>>> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
>>> domain to which the logical processor belongs...
>
>
> Yes, we can get such information in /sys/devices/system/node, I think
> we can only print:
>
> ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z
Agree.
>
>
> Thanks
> Hanjun
>



-- 
Regards,
Dennis

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

Hi Hanjun,

Thanks for the clarification and some little comments ;-)

On 27 April 2016 at 12:04, Hanjun Guo <hanjun.guo@linaro.org> wrote:
> Hi Dennis, David,
>
> Sorry for the late reply, please see my comments below.
>
>
> On 2016/4/27 9:14, David Daney wrote:
>>
>> On 04/21/2016 03:06 AM, Dennis Chen wrote:
>>>
>>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
>>
>> [...]
>>>>
>>>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>>>> +void __init acpi_numa_gicc_affinity_init(struct
>>>> acpi_srat_gicc_affinity *pa)
>>>> +{
>>>> +       int pxm, node;
>>>> +       u64 mpidr;
>>>> +
>>>> +       if (srat_disabled())
>>>> +               return;
>>>> +
>>>> +       if (pa->header.length < sizeof(struct
>>>> acpi_srat_gicc_affinity)) {
>>>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>>>> +                       pa->header.length);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>>>> +               return;
>>>> +
>>>> +       if (cpus_in_srat >= NR_CPUS) {
>>>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small,
>>>> may not be able to use all cpus\n",
>>>> +                            NR_CPUS);
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       pxm = pa->proximity_domain;
>>>> +       node = acpi_map_pxm_to_node(pxm);
>>>> +
>>>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>>>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>>>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid
>>>> MPIDR in MADT\n",
>>>> +                       pxm, pa->acpi_processor_uid);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>>>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>>>> +       node_set(node, numa_nodes_parsed);
>>>> +       cpus_in_srat++;
>>>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>>>> +               pxm, mpidr, node, cpus_in_srat);
>>>> +}
>>>
>>>
>>> What does the *cpu* means in above pr_info function? If it's the
>>> logical processor ID or ACPI processor UID, then I suggest to use
>>> pa->acpi_processor_uid instead of cpus_in_srat, I understand the
>
>
> I think print cpus_in_srat is pointless here, as the logic cpu number
> is allocated by OS when initializing SMP by scanning MADT table. As
> Dennis said, it's just a count number, not a number mapping to MPIDR.
>
> ACPI processor UID is the key value to connect MADT, SRAT, DSDT.
>
> For MADT, it will have MPIDR and ACPI processor UID, and OS will
> create mappings to MPIDR and cpu logical number,
> ACPI processor UID <------> MPIDR <------> CPU logical number
>
> In SRAT, there is ACPI processor UID represented, mappings will be
> ACPI processor UID <------> PXM <------> NUMA node logical number
>
> So we can use ACPI processor UID to get the MPIDR by scanning the
> MADT, then we can map NUMA node logical number to cpu logical
> number later.
>
Right, kernel will record the logical cpu index info (begins from 0,
the boot cpu) into the cpu_possible bit map by parsing the MADT GICC
sub-table. So I am thinking here if we can reduce the parsing walk by
only one, because I see the acpi_numa_gicc_affinity_init() calls
get_mpidr_in_madt() to traverse the entire MADT, actually the kernel
will also traverse the MADT in smp_init_cpus(), merge them into one?
>
>>> cpus_in_srat is just a count number of the entries of GICC Affinity
>>> Struct instance in SRAT, correct me if I am wrong. So at least it sees
>>> to me, the above pr_info will output message looks like:
>>> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
>>> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
>>> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>>>
>>
>> Yes, that is correct, and for my system seems to be what we want as the
>> names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the
>> sequential numbering (0..95) with 48 CPUs on each node.
>
>
> That's because you place CPUs in the same order both in MADT and SRAT :)
> if not, that will be not match.
>
Hmm, I think I made a mistake presumption here that the ACPI processor
UID is the logical processor number, just take a look at the ACPI
processor driver module code, the logic process number should comes
from the bit map. And from the below output message pasted by David,
we can see that his firmware is using MPIDR as the ACPI processor UID,
it's not incorrect implementation according to the ACPI spec...
>
>>
>> If I make the change you suggest, I get :
>> .
>> .
>> .
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu
>> 65536
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu
>> 65537
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu
>> 65538
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu
>> 65539
>> .
>> .
>> .
>>
>> Not really what I would want.
>
>
> How about remove the print for "cpu"? it's not the right value we want,
> and we can get such mapping information under in sysfs.
>
I think you may remove the *cpu* in the print, indeed it's pointless.
You can, IMO, use cpu_possible bitmask if you want to get the logical
processor number just as the ACPI processor does. About the
early_node_cpu_hwid, the only reason I can see now is to map logic cpu
into the numa node number, can we refer to the implementation under
x86, anyway I am not sure since not very familiar with that codes
dependency...
>
>>
>>
>>> While the /sys/devices/system/cpu will use the ACPI processor UID to
>>> generate the index of the cpu, like:
>>> cpu0  cpu1  cpu2 ...
>>>
>>> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
>>> domain to which the logical processor belongs...
>
>
> Yes, we can get such information in /sys/devices/system/node, I think
> we can only print:
>
> ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z
Agree.
>
>
> Thanks
> Hanjun
>



-- 
Regards,
Dennis

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

Hi Hanjun,

Thanks for the clarification and some little comments ;-)

On 27 April 2016 at 12:04, Hanjun Guo <hanjun.guo@linaro.org> wrote:
> Hi Dennis, David,
>
> Sorry for the late reply, please see my comments below.
>
>
> On 2016/4/27 9:14, David Daney wrote:
>>
>> On 04/21/2016 03:06 AM, Dennis Chen wrote:
>>>
>>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
>>
>> [...]
>>>>
>>>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>>>> +void __init acpi_numa_gicc_affinity_init(struct
>>>> acpi_srat_gicc_affinity *pa)
>>>> +{
>>>> +       int pxm, node;
>>>> +       u64 mpidr;
>>>> +
>>>> +       if (srat_disabled())
>>>> +               return;
>>>> +
>>>> +       if (pa->header.length < sizeof(struct
>>>> acpi_srat_gicc_affinity)) {
>>>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>>>> +                       pa->header.length);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>>>> +               return;
>>>> +
>>>> +       if (cpus_in_srat >= NR_CPUS) {
>>>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small,
>>>> may not be able to use all cpus\n",
>>>> +                            NR_CPUS);
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       pxm = pa->proximity_domain;
>>>> +       node = acpi_map_pxm_to_node(pxm);
>>>> +
>>>> +       if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) {
>>>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>>>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid
>>>> MPIDR in MADT\n",
>>>> +                       pxm, pa->acpi_processor_uid);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>>>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>>>> +       node_set(node, numa_nodes_parsed);
>>>> +       cpus_in_srat++;
>>>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>>>> +               pxm, mpidr, node, cpus_in_srat);
>>>> +}
>>>
>>>
>>> What does the *cpu* means in above pr_info function? If it's the
>>> logical processor ID or ACPI processor UID, then I suggest to use
>>> pa->acpi_processor_uid instead of cpus_in_srat, I understand the
>
>
> I think print cpus_in_srat is pointless here, as the logic cpu number
> is allocated by OS when initializing SMP by scanning MADT table. As
> Dennis said, it's just a count number, not a number mapping to MPIDR.
>
> ACPI processor UID is the key value to connect MADT, SRAT, DSDT.
>
> For MADT, it will have MPIDR and ACPI processor UID, and OS will
> create mappings to MPIDR and cpu logical number,
> ACPI processor UID <------> MPIDR <------> CPU logical number
>
> In SRAT, there is ACPI processor UID represented, mappings will be
> ACPI processor UID <------> PXM <------> NUMA node logical number
>
> So we can use ACPI processor UID to get the MPIDR by scanning the
> MADT, then we can map NUMA node logical number to cpu logical
> number later.
>
Right, kernel will record the logical cpu index info (begins from 0,
the boot cpu) into the cpu_possible bit map by parsing the MADT GICC
sub-table. So I am thinking here if we can reduce the parsing walk by
only one, because I see the acpi_numa_gicc_affinity_init() calls
get_mpidr_in_madt() to traverse the entire MADT, actually the kernel
will also traverse the MADT in smp_init_cpus(), merge them into one?
>
>>> cpus_in_srat is just a count number of the entries of GICC Affinity
>>> Struct instance in SRAT, correct me if I am wrong. So at least it sees
>>> to me, the above pr_info will output message looks like:
>>> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
>>> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
>>> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>>>
>>
>> Yes, that is correct, and for my system seems to be what we want as the
>> names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the
>> sequential numbering (0..95) with 48 CPUs on each node.
>
>
> That's because you place CPUs in the same order both in MADT and SRAT :)
> if not, that will be not match.
>
Hmm, I think I made a mistake presumption here that the ACPI processor
UID is the logical processor number, just take a look at the ACPI
processor driver module code, the logic process number should comes
from the bit map. And from the below output message pasted by David,
we can see that his firmware is using MPIDR as the ACPI processor UID,
it's not incorrect implementation according to the ACPI spec...
>
>>
>> If I make the change you suggest, I get :
>> .
>> .
>> .
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu
>> 65536
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu
>> 65537
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu
>> 65538
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu
>> 65539
>> .
>> .
>> .
>>
>> Not really what I would want.
>
>
> How about remove the print for "cpu"? it's not the right value we want,
> and we can get such mapping information under in sysfs.
>
I think you may remove the *cpu* in the print, indeed it's pointless.
You can, IMO, use cpu_possible bitmask if you want to get the logical
processor number just as the ACPI processor does. About the
early_node_cpu_hwid, the only reason I can see now is to map logic cpu
into the numa node number, can we refer to the implementation under
x86, anyway I am not sure since not very familiar with that codes
dependency...
>
>>
>>
>>> While the /sys/devices/system/cpu will use the ACPI processor UID to
>>> generate the index of the cpu, like:
>>> cpu0  cpu1  cpu2 ...
>>>
>>> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
>>> domain to which the logical processor belongs...
>
>
> Yes, we can get such information in /sys/devices/system/node, I think
> we can only print:
>
> ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z
Agree.
>
>
> Thanks
> Hanjun
>



-- 
Regards,
Dennis

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[Dennis Chen]

Hi Hanjun,

Thanks for the clarification and some little comments ;-)

On 27 April 2016 at 12:04, Hanjun Guo <hanjun.guo@linaro.org> wrote:
> Hi Dennis, David,
>
> Sorry for the late reply, please see my comments below.
>
>
> On 2016/4/27 9:14, David Daney wrote:
>>
>> On 04/21/2016 03:06 AM, Dennis Chen wrote:
>>>
>>> On 20 April 2016 at 09:40, David Daney <ddaney.cavm@gmail.com> wrote:
>>
>> [...]
>>>>
>>>> +/* Callback for Proximity Domain -> ACPI processor UID mapping */
>>>> +void __init acpi_numa_gicc_affinity_init(struct
>>>> acpi_srat_gicc_affinity *pa)
>>>> +{
>>>> +       int pxm, node;
>>>> +       u64 mpidr;
>>>> +
>>>> +       if (srat_disabled())
>>>> +               return;
>>>> +
>>>> +       if (pa->header.length < sizeof(struct
>>>> acpi_srat_gicc_affinity)) {
>>>> +               pr_err("SRAT: Invalid SRAT header length: %d\n",
>>>> +                       pa->header.length);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       if (!(pa->flags & ACPI_SRAT_GICC_ENABLED))
>>>> +               return;
>>>> +
>>>> +       if (cpus_in_srat >= NR_CPUS) {
>>>> +               pr_warn_once("SRAT: cpu_to_node_map[%d] is too small,
>>>> may not be able to use all cpus\n",
>>>> +                            NR_CPUS);
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       pxm = pa->proximity_domain;
>>>> +       node = acpi_map_pxm_to_node(pxm);
>>>> +
>>>> +       if (node = NUMA_NO_NODE || node >= MAX_NUMNODES) {
>>>> +               pr_err("SRAT: Too many proximity domains %d\n", pxm);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) {
>>>> +               pr_err("SRAT: PXM %d with ACPI ID %d has no valid
>>>> MPIDR in MADT\n",
>>>> +                       pxm, pa->acpi_processor_uid);
>>>> +               bad_srat();
>>>> +               return;
>>>> +       }
>>>> +
>>>> +       early_node_cpu_hwid[cpus_in_srat].node_id = node;
>>>> +       early_node_cpu_hwid[cpus_in_srat].cpu_hwid =  mpidr;
>>>> +       node_set(node, numa_nodes_parsed);
>>>> +       cpus_in_srat++;
>>>> +       pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d cpu %d\n",
>>>> +               pxm, mpidr, node, cpus_in_srat);
>>>> +}
>>>
>>>
>>> What does the *cpu* means in above pr_info function? If it's the
>>> logical processor ID or ACPI processor UID, then I suggest to use
>>> pa->acpi_processor_uid instead of cpus_in_srat, I understand the
>
>
> I think print cpus_in_srat is pointless here, as the logic cpu number
> is allocated by OS when initializing SMP by scanning MADT table. As
> Dennis said, it's just a count number, not a number mapping to MPIDR.
>
> ACPI processor UID is the key value to connect MADT, SRAT, DSDT.
>
> For MADT, it will have MPIDR and ACPI processor UID, and OS will
> create mappings to MPIDR and cpu logical number,
> ACPI processor UID <------> MPIDR <------> CPU logical number
>
> In SRAT, there is ACPI processor UID represented, mappings will be
> ACPI processor UID <------> PXM <------> NUMA node logical number
>
> So we can use ACPI processor UID to get the MPIDR by scanning the
> MADT, then we can map NUMA node logical number to cpu logical
> number later.
>
Right, kernel will record the logical cpu index info (begins from 0,
the boot cpu) into the cpu_possible bit map by parsing the MADT GICC
sub-table. So I am thinking here if we can reduce the parsing walk by
only one, because I see the acpi_numa_gicc_affinity_init() calls
get_mpidr_in_madt() to traverse the entire MADT, actually the kernel
will also traverse the MADT in smp_init_cpus(), merge them into one?
>
>>> cpus_in_srat is just a count number of the entries of GICC Affinity
>>> Struct instance in SRAT, correct me if I am wrong. So at least it sees
>>> to me, the above pr_info will output message looks like:
>>> SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 1
>>> SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 2
>>> SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 3
>>>
>>
>> Yes, that is correct, and for my system seems to be what we want as the
>> names in /sys/devices/system/cpu/ and /proc/cpu_info agree with the
>> sequential numbering (0..95) with 48 CPUs on each node.
>
>
> That's because you place CPUs in the same order both in MADT and SRAT :)
> if not, that will be not match.
>
Hmm, I think I made a mistake presumption here that the ACPI processor
UID is the logical processor number, just take a look at the ACPI
processor driver module code, the logic process number should comes
from the bit map. And from the below output message pasted by David,
we can see that his firmware is using MPIDR as the ACPI processor UID,
it's not incorrect implementation according to the ACPI spec...
>
>>
>> If I make the change you suggest, I get :
>> .
>> .
>> .
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0 cpu 0
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0 cpu 1
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0 cpu 2
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0 cpu 3
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x4 -> Node 0 cpu 4
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x5 -> Node 0 cpu 5
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x6 -> Node 0 cpu 6
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x7 -> Node 0 cpu 7
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x8 -> Node 0 cpu 8
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x9 -> Node 0 cpu 9
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa -> Node 0 cpu 10
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb -> Node 0 cpu 11
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc -> Node 0 cpu 12
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd -> Node 0 cpu 13
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe -> Node 0 cpu 14
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf -> Node 0 cpu 15
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100 -> Node 0 cpu 256
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x101 -> Node 0 cpu 257
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x102 -> Node 0 cpu 258
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x103 -> Node 0 cpu 259
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x104 -> Node 0 cpu 260
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x105 -> Node 0 cpu 261
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x106 -> Node 0 cpu 262
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x107 -> Node 0 cpu 263
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x108 -> Node 0 cpu 264
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x109 -> Node 0 cpu 265
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10a -> Node 0 cpu 266
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10b -> Node 0 cpu 267
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10c -> Node 0 cpu 268
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10d -> Node 0 cpu 269
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10e -> Node 0 cpu 270
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x10f -> Node 0 cpu 271
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200 -> Node 0 cpu 512
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x201 -> Node 0 cpu 513
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x202 -> Node 0 cpu 514
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x203 -> Node 0 cpu 515
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x204 -> Node 0 cpu 516
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x205 -> Node 0 cpu 517
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x206 -> Node 0 cpu 518
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x207 -> Node 0 cpu 519
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x208 -> Node 0 cpu 520
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x209 -> Node 0 cpu 521
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20a -> Node 0 cpu 522
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20b -> Node 0 cpu 523
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20c -> Node 0 cpu 524
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20d -> Node 0 cpu 525
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20e -> Node 0 cpu 526
>> [    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x20f -> Node 0 cpu 527
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10000 -> Node 1 cpu
>> 65536
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10001 -> Node 1 cpu
>> 65537
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10002 -> Node 1 cpu
>> 65538
>> [    0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x10003 -> Node 1 cpu
>> 65539
>> .
>> .
>> .
>>
>> Not really what I would want.
>
>
> How about remove the print for "cpu"? it's not the right value we want,
> and we can get such mapping information under in sysfs.
>
I think you may remove the *cpu* in the print, indeed it's pointless.
You can, IMO, use cpu_possible bitmask if you want to get the logical
processor number just as the ACPI processor does. About the
early_node_cpu_hwid, the only reason I can see now is to map logic cpu
into the numa node number, can we refer to the implementation under
x86, anyway I am not sure since not very familiar with that codes
dependency...
>
>>
>>
>>> While the /sys/devices/system/cpu will use the ACPI processor UID to
>>> generate the index of the cpu, like:
>>> cpu0  cpu1  cpu2 ...
>>>
>>> As the GICC Affinity Struct indicated, the ps->proximity_domain is the
>>> domain to which the logical processor belongs...
>
>
> Yes, we can get such information in /sys/devices/system/node, I think
> we can only print:
>
> ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z
Agree.
>
>
> Thanks
> Hanjun
>



-- 
Regards,
Dennis

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/27/2016 04:37 AM, Dennis Chen wrote:
> Hi Hanjun,
>
> Thanks for the clarification and some little comments ;-)
>
> On 27 April 2016 at 12:04, Hanjun Guo <hanjun.guo@linaro.org> wrote:
>> Hi Dennis, David,
>>
[...]
>>
>>
>> Yes, we can get such information in /sys/devices/system/node, I think
>> we can only print:
>>
>> ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z
> Agree.

I am also in agreement.  I will send a revised patch that implements this.

Thanks,
David.

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/27/2016 04:37 AM, Dennis Chen wrote:
> Hi Hanjun,
>
> Thanks for the clarification and some little comments ;-)
>
> On 27 April 2016 at 12:04, Hanjun Guo <hanjun.guo@linaro.org> wrote:
>> Hi Dennis, David,
>>
[...]
>>
>>
>> Yes, we can get such information in /sys/devices/system/node, I think
>> we can only print:
>>
>> ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z
> Agree.

I am also in agreement.  I will send a revised patch that implements this.

Thanks,
David.

[PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/27/2016 04:37 AM, Dennis Chen wrote:
> Hi Hanjun,
>
> Thanks for the clarification and some little comments ;-)
>
> On 27 April 2016 at 12:04, Hanjun Guo <hanjun.guo@linaro.org> wrote:
>> Hi Dennis, David,
>>
[...]
>>
>>
>> Yes, we can get such information in /sys/devices/system/node, I think
>> we can only print:
>>
>> ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z
> Agree.

I am also in agreement.  I will send a revised patch that implements this.

Thanks,
David.

Re: [PATCH v5 12/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT

[David Daney]

On 04/27/2016 04:37 AM, Dennis Chen wrote:
> Hi Hanjun,
>
> Thanks for the clarification and some little comments ;-)
>
> On 27 April 2016 at 12:04, Hanjun Guo <hanjun.guo@linaro.org> wrote:
>> Hi Dennis, David,
>>
[...]
>>
>>
>> Yes, we can get such information in /sys/devices/system/node, I think
>> we can only print:
>>
>> ACPI: NUMA: SRAT: PXM x -> MPIDR y -> Node z
> Agree.

I am also in agreement.  I will send a revised patch that implements this.

Thanks,
David.