From: Ira Weiny <ira.weiny@intel.com>
To: alison.schofield@intel.com
Cc: "Rafael J. Wysocki" <rafael@kernel.org>,
Len Brown <lenb@kernel.org>,
Vishal Verma <vishal.l.verma@intel.com>,
Ben Widawsky <ben.widawsky@intel.com>,
Dan Williams <dan.j.williams@intel.com>,
linux-cxl@vger.kernel.org, linux-acpi@vger.kernel.org
Subject: Re: [PATCH] ACPI: NUMA: Add a node and memblk for each CFMWS not in SRAT
Date: Mon, 11 Oct 2021 10:13:16 -0700 [thread overview]
Message-ID: <20211011171315.GL3114988@iweiny-DESK2.sc.intel.com> (raw)
In-Reply-To: <20211009015339.400383-1-alison.schofield@intel.com>
On Fri, Oct 08, 2021 at 06:53:39PM -0700, Schofield, Alison wrote:
> From: Alison Schofield <alison.schofield@intel.com>
>
> During NUMA init, CXL memory defined in the SRAT Memory Affinity
> subtable may be assigned to a NUMA node. Since there is no
> requirement that the SRAT be comprehensive for CXL memory another
> mechanism is needed to assign NUMA nodes to CXL memory not identified
> in the SRAT.
>
> Use the CXL Fixed Memory Window Structure's (CFMWS) of the ACPI CXL
> Early Discovery Table (CEDT) to find all CXL memory ranges. Create a
> NUMA node for each range that is not already assigned to a NUMA node.
> Add a memblk attaching its host physical address range to the node.
>
> Note that these ranges may not actually map any memory at boot time.
> They may describe persistent capacity or may be present to enable
> hot-plug.
>
> Consumers can use phys_to_target_node() to discover the NUMA node.
>
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> ---
> drivers/acpi/numa/srat.c | 58 ++++++++++++++++++++++++++++++++++++++++
> drivers/cxl/acpi.c | 8 +++---
> 2 files changed, 63 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c
> index b8795fc49097..568e033e6c3f 100644
> --- a/drivers/acpi/numa/srat.c
> +++ b/drivers/acpi/numa/srat.c
> @@ -300,6 +300,61 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
> }
> #endif /* defined(CONFIG_X86) || defined (CONFIG_ARM64) */
>
> +/* Add a NUMA node and memblk for each node-less CFMWS */
IMO this comment does not make sense for a function called 'parse cfmws'. I
would just drop it.
> +static int __init acpi_parse_cfmws(struct acpi_table_header *acpi_cedt)
> +{
> + struct acpi_cedt_cfmws *cfmws;
> + acpi_size len, cur = 0;
> + void *cedt_subtable;
> + int i, pxm, node;
> + u64 start, end;
> +
> + /* Use fake PXM values starting after the max PXM found in the SRAT */
> + for (i = 0; i < MAX_PXM_DOMAINS - 1; i++)
> + if (node_to_pxm_map[i] > pxm)
> + pxm = node_to_pxm_map[i];
> + pxm++;
> +
> + len = acpi_cedt->length - sizeof(*acpi_cedt);
> + cedt_subtable = acpi_cedt + 1;
> +
> + while (cur < len) {
> + struct acpi_cedt_header *c = cedt_subtable + cur;
> +
> + if (c->type != ACPI_CEDT_TYPE_CFMWS)
> + goto next;
> +
> + cfmws = cedt_subtable + cur;
> + if (cfmws->header.length < sizeof(*cfmws)) {
> + pr_warn_once("CFMWS entry skipped:invalid length:%u\n",
> + cfmws->header.length);
> + goto next;
> + }
> +
> + start = cfmws->base_hpa;
> + end = cfmws->base_hpa + cfmws->window_size;
> +
> + /* Skip if the HPA is already assigned to a NUMA node */
> + node = phys_to_target_node(start);
> + if (node != NUMA_NO_NODE)
> + goto next;
> +
> + node = acpi_map_pxm_to_node(pxm);
> + if (node == NUMA_NO_NODE) {
> + pr_err("ACPI NUMA: Too many proximity domains.\n");
> + return -EINVAL;
> + }
> + if (numa_add_memblk(node, start, end) < 0) {
> + pr_warn("ACPI NUMA: Failed to add memblk for CFMWS node %d [mem %#llx-%#llx]\n",
> + node, start, end);
Is there anything which could go wrong if the block is not added to the numa
node?
Ira
> + }
> + pxm++;
> +next:
> + cur += c->length;
> + }
> + return 0;
> +}
> +
> static int __init acpi_parse_slit(struct acpi_table_header *table)
> {
> struct acpi_table_slit *slit = (struct acpi_table_slit *)table;
> @@ -478,6 +533,9 @@ int __init acpi_numa_init(void)
> /* SLIT: System Locality Information Table */
> acpi_table_parse(ACPI_SIG_SLIT, acpi_parse_slit);
>
> + /* CEDT: CXL Early Discovery Table */
> + acpi_table_parse(ACPI_SIG_CEDT, acpi_parse_cfmws);
> +
> if (cnt < 0)
> return cnt;
> else if (!parsed_numa_memblks)
> diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
> index dadc7f64b9ff..3798841c3418 100644
> --- a/drivers/cxl/acpi.c
> +++ b/drivers/cxl/acpi.c
> @@ -144,9 +144,11 @@ static void cxl_add_cfmws_decoders(struct device *dev,
> cfmws->window_size - 1);
> goto next;
> }
> - dev_dbg(dev, "add: %s range %#llx-%#llx\n",
> - dev_name(&cxld->dev), cfmws->base_hpa,
> - cfmws->base_hpa + cfmws->window_size - 1);
> + dev_dbg(dev, "add:%s node:%d range:%#llx-%#llx restrict:%#x\n",
> + dev_name(&cxld->dev),
> + phys_to_target_node(cxld->range.start),
> + cxld->range.start, cxld->range.end,
> + cfmws->restrictions);
> next:
> cur += c->length;
> }
> --
> 2.31.1
>
next prev parent reply other threads:[~2021-10-11 17:13 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-09 1:53 [PATCH] ACPI: NUMA: Add a node and memblk for each CFMWS not in SRAT alison.schofield
2021-10-09 2:00 ` Alison Schofield
2021-10-09 3:56 ` kernel test robot
2021-10-09 13:23 ` kernel test robot
2021-10-11 17:13 ` Ira Weiny [this message]
2021-10-11 22:00 ` Alison Schofield
2021-10-14 0:42 ` Dan Williams
2021-10-13 23:18 ` Dan Williams
2021-10-15 16:59 ` Jonathan Cameron
2021-10-15 18:58 ` Dan Williams
2021-10-18 9:25 ` Jonathan Cameron
2021-10-18 18:15 ` Dan Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211011171315.GL3114988@iweiny-DESK2.sc.intel.com \
--to=ira.weiny@intel.com \
--cc=alison.schofield@intel.com \
--cc=ben.widawsky@intel.com \
--cc=dan.j.williams@intel.com \
--cc=lenb@kernel.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-cxl@vger.kernel.org \
--cc=rafael@kernel.org \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).