From: "Rafael J. Wysocki" <rafael@kernel.org>
To: Keith Busch <keith.busch@intel.com>
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
ACPI Devel Maling List <linux-acpi@vger.kernel.org>,
Linux Memory Management List <linux-mm@kvack.org>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Rafael Wysocki <rafael@kernel.org>,
Dave Hansen <dave.hansen@intel.com>,
Dan Williams <dan.j.williams@intel.com>
Subject: Re: [PATCHv4 07/13] node: Add heterogenous memory access attributes
Date: Thu, 17 Jan 2019 16:03:42 +0100 [thread overview]
Message-ID: <CAJZ5v0jCEdhKndgZgJ=SdHgFBM1Bcxusm_crYzAOTZDx3s=PdQ@mail.gmail.com> (raw)
In-Reply-To: <20190116175804.30196-8-keith.busch@intel.com>
On Wed, Jan 16, 2019 at 6:59 PM Keith Busch <keith.busch@intel.com> wrote:
>
> Heterogeneous memory systems provide memory nodes with different latency
> and bandwidth performance attributes. Provide a new kernel interface for
> subsystems to register the attributes under the memory target node's
> initiator access class. If the system provides this information, applications
> may query these attributes when deciding which node to request memory.
>
> The following example shows the new sysfs hierarchy for a node exporting
> performance attributes:
>
> # tree -P "read*|write*" /sys/devices/system/node/nodeY/classZ/
> /sys/devices/system/node/nodeY/classZ/
> |-- read_bandwidth
> |-- read_latency
> |-- write_bandwidth
> `-- write_latency
>
> The bandwidth is exported as MB/s and latency is reported in nanoseconds.
> Memory accesses from an initiator node that is not one of the memory's
> class "Z" initiator nodes may encounter different performance than
> reported here. When a subsystem makes use of this interface, initiators
> of a lower class number, "Z", have better performance relative to higher
> class numbers. When provided, class 0 is the highest performing access
> class.
>
> Signed-off-by: Keith Busch <keith.busch@intel.com>
> ---
> drivers/base/Kconfig | 8 ++++++++
> drivers/base/node.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
> include/linux/node.h | 25 +++++++++++++++++++++++++
> 3 files changed, 81 insertions(+)
>
> diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
> index 3e63a900b330..6014980238e8 100644
> --- a/drivers/base/Kconfig
> +++ b/drivers/base/Kconfig
> @@ -149,6 +149,14 @@ config DEBUG_TEST_DRIVER_REMOVE
> unusable. You should say N here unless you are explicitly looking to
> test this functionality.
>
> +config HMEM_REPORTING
> + bool
> + default y
> + depends on NUMA
> + help
> + Enable reporting for heterogenous memory access attributes under
> + their non-uniform memory nodes.
Why would anyone ever want to say "no" to this?
Distros will set it anyway.
> +
> source "drivers/base/test/Kconfig"
>
> config SYS_HYPERVISOR
> diff --git a/drivers/base/node.c b/drivers/base/node.c
> index 1da5072116ab..1e909f61e8b1 100644
> --- a/drivers/base/node.c
> +++ b/drivers/base/node.c
> @@ -66,6 +66,9 @@ struct node_class_nodes {
> unsigned class;
> nodemask_t initiator_nodes;
> nodemask_t target_nodes;
> +#ifdef CONFIG_HMEM_REPORTING
> + struct node_hmem_attrs hmem_attrs;
> +#endif
> };
> #define to_class_nodes(dev) container_of(dev, struct node_class_nodes, dev)
>
> @@ -145,6 +148,51 @@ static struct node_class_nodes *node_init_node_class(struct device *parent,
> return NULL;
> }
>
> +#ifdef CONFIG_HMEM_REPORTING
> +#define ACCESS_ATTR(name) \
> +static ssize_t name##_show(struct device *dev, \
> + struct device_attribute *attr, \
> + char *buf) \
> +{ \
> + return sprintf(buf, "%u\n", to_class_nodes(dev)->hmem_attrs.name); \
> +} \
> +static DEVICE_ATTR_RO(name);
> +
> +ACCESS_ATTR(read_bandwidth)
> +ACCESS_ATTR(read_latency)
> +ACCESS_ATTR(write_bandwidth)
> +ACCESS_ATTR(write_latency)
> +
> +static struct attribute *access_attrs[] = {
> + &dev_attr_read_bandwidth.attr,
> + &dev_attr_read_latency.attr,
> + &dev_attr_write_bandwidth.attr,
> + &dev_attr_write_latency.attr,
> + NULL,
> +};
> +ATTRIBUTE_GROUPS(access);
> +
Kerneldoc?
And who is going to call this?
> +void node_set_perf_attrs(unsigned int nid, struct node_hmem_attrs *hmem_attrs,
> + unsigned class)
> +{
> + struct node_class_nodes *c;
> + struct node *node;
> +
> + if (WARN_ON_ONCE(!node_online(nid)))
> + return;
> +
> + node = node_devices[nid];
> + c = node_init_node_class(&node->dev, &node->class_list, class);
> + if (!c)
> + return;
> +
> + c->hmem_attrs = *hmem_attrs;
> + if (sysfs_create_groups(&c->dev.kobj, access_groups))
> + pr_info("failed to add performance attribute group to node %d\n",
> + nid);
> +}
> +#endif
> +
> #define K(x) ((x) << (PAGE_SHIFT - 10))
> static ssize_t node_read_meminfo(struct device *dev,
> struct device_attribute *attr, char *buf)
> diff --git a/include/linux/node.h b/include/linux/node.h
> index 8e3666c12ef2..e22940a593c2 100644
> --- a/include/linux/node.h
> +++ b/include/linux/node.h
> @@ -20,6 +20,31 @@
> #include <linux/list.h>
> #include <linux/workqueue.h>
>
> +#ifdef CONFIG_HMEM_REPORTING
> +/**
> + * struct node_hmem_attrs - heterogeneous memory performance attributes
> + *
> + * @read_bandwidth: Read bandwidth in MB/s
> + * @write_bandwidth: Write bandwidth in MB/s
> + * @read_latency: Read latency in nanoseconds
> + * @write_latency: Write latency in nanoseconds
> + */
> +struct node_hmem_attrs {
> + unsigned int read_bandwidth;
> + unsigned int write_bandwidth;
> + unsigned int read_latency;
> + unsigned int write_latency;
> +};
> +void node_set_perf_attrs(unsigned int nid, struct node_hmem_attrs *hmem_attrs,
> + unsigned class);
> +#else
> +static inline void node_set_perf_attrs(unsigned int nid,
> + struct node_hmem_attrs *hmem_attrs,
> + unsigned class)
> +{
> +}
Have you tried to compile this with CONFIG_HMEM_REPORTING unset?
> +#endif
> +
> struct node {
> struct device dev;
> struct list_head class_list;
> --
next prev parent reply other threads:[~2019-01-17 15:03 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-16 17:57 [PATCHv4 00/13] Heterogeneuos memory node attributes Keith Busch
2019-01-16 17:57 ` [PATCHv4 01/13] acpi: Create subtable parsing infrastructure Keith Busch
2019-01-16 17:57 ` [PATCHv4 02/13] acpi: Add HMAT to generic parsing tables Keith Busch
2019-01-16 17:57 ` [PATCHv4 03/13] acpi/hmat: Parse and report heterogeneous memory Keith Busch
2019-01-17 11:00 ` Rafael J. Wysocki
2019-01-16 17:57 ` [PATCHv4 04/13] node: Link memory nodes to their compute nodes Keith Busch
2019-01-17 11:26 ` Rafael J. Wysocki
2019-01-16 17:57 ` [PATCHv4 05/13] Documentation/ABI: Add new node sysfs attributes Keith Busch
2019-01-17 11:41 ` Rafael J. Wysocki
2019-01-18 20:42 ` Keith Busch
2019-01-18 21:08 ` Dan Williams
2019-01-19 9:01 ` Greg Kroah-Hartman
2019-01-19 16:56 ` Dan Williams
2019-01-20 16:19 ` Rafael J. Wysocki
2019-01-20 17:34 ` Dan Williams
2019-01-21 9:54 ` Rafael J. Wysocki
2019-01-20 16:16 ` Rafael J. Wysocki
2019-01-22 16:36 ` Keith Busch
2019-01-22 16:51 ` Rafael J. Wysocki
2019-01-22 16:54 ` Rafael J. Wysocki
2019-01-18 11:21 ` Jonathan Cameron
2019-01-18 16:35 ` Dan Williams
2019-01-16 17:57 ` [PATCHv4 06/13] acpi/hmat: Register processor domain to its memory Keith Busch
2019-01-17 12:11 ` Rafael J. Wysocki
2019-01-17 17:01 ` Dan Williams
2019-01-16 17:57 ` [PATCHv4 07/13] node: Add heterogenous memory access attributes Keith Busch
2019-01-17 15:03 ` Rafael J. Wysocki [this message]
2019-01-17 15:41 ` Greg Kroah-Hartman
2019-01-16 17:57 ` [PATCHv4 08/13] Documentation/ABI: Add node performance attributes Keith Busch
2019-01-17 15:09 ` Rafael J. Wysocki
2019-01-16 17:58 ` [PATCHv4 09/13] acpi/hmat: Register " Keith Busch
2019-01-17 15:21 ` Rafael J. Wysocki
2019-01-16 17:58 ` [PATCHv4 10/13] node: Add memory caching attributes Keith Busch
2019-01-17 16:00 ` Rafael J. Wysocki
2019-02-09 8:20 ` Brice Goglin
2019-02-10 17:19 ` Jonathan Cameron
2019-02-11 15:23 ` Keith Busch
2019-02-12 8:11 ` Brice Goglin
2019-02-12 8:49 ` Jonathan Cameron
2019-02-12 17:31 ` Keith Busch
2019-01-16 17:58 ` [PATCHv4 11/13] Documentation/ABI: Add node cache attributes Keith Busch
2019-01-17 16:25 ` Rafael J. Wysocki
2019-01-16 17:58 ` [PATCHv4 12/13] acpi/hmat: Register memory side " Keith Busch
2019-01-17 17:42 ` Rafael J. Wysocki
2019-01-16 17:58 ` [PATCHv4 13/13] doc/mm: New documentation for memory performance Keith Busch
2019-01-17 12:58 ` [PATCHv4 00/13] Heterogeneuos memory node attributes Balbir Singh
2019-01-17 15:44 ` Keith Busch
2019-01-18 13:16 ` Balbir Singh
2019-01-17 18:18 ` Jonathan Cameron
2019-01-17 19:47 ` Keith Busch
2019-01-18 11:12 ` Jonathan Cameron
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAJZ5v0jCEdhKndgZgJ=SdHgFBM1Bcxusm_crYzAOTZDx3s=PdQ@mail.gmail.com' \
--to=rafael@kernel.org \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=gregkh@linuxfoundation.org \
--cc=keith.busch@intel.com \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).