From: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
To: linux-mm@kvack.org, akpm@linux-foundation.org
Cc: Wei Xu <weixugc@google.com>, Huang Ying <ying.huang@intel.com>,
Greg Thelen <gthelen@google.com>, Yang Shi <shy828301@gmail.com>,
Davidlohr Bueso <dave@stgolabs.net>,
Tim C Chen <tim.c.chen@intel.com>,
Brice Goglin <brice.goglin@gmail.com>,
Michal Hocko <mhocko@kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Hesham Almatary <hesham.almatary@huawei.com>,
Dave Hansen <dave.hansen@intel.com>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>,
Alistair Popple <apopple@nvidia.com>,
Dan Williams <dan.j.williams@intel.com>,
Feng Tang <feng.tang@intel.com>,
Jagdish Gediya <jvgediya@linux.ibm.com>,
Baolin Wang <baolin.wang@linux.alibaba.com>,
David Rientjes <rientjes@google.com>,
"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
Subject: [PATCH v6 00/13] mm/demotion: Memory tiers and demotion
Date: Fri, 10 Jun 2022 19:22:16 +0530 [thread overview]
Message-ID: <20220610135229.182859-1-aneesh.kumar@linux.ibm.com> (raw)
The current kernel has the basic memory tiering support: Inactive
pages on a higher tier NUMA node can be migrated (demoted) to a lower
tier NUMA node to make room for new allocations on the higher tier
NUMA node. Frequently accessed pages on a lower tier NUMA node can be
migrated (promoted) to a higher tier NUMA node to improve the
performance.
In the current kernel, memory tiers are defined implicitly via a
demotion path relationship between NUMA nodes, which is created during
the kernel initialization and updated when a NUMA node is hot-added or
hot-removed. The current implementation puts all nodes with CPU into
the top tier, and builds the tier hierarchy tier-by-tier by establishing
the per-node demotion targets based on the distances between nodes.
This current memory tier kernel interface needs to be improved for
several important use cases:
* The current tier initialization code always initializes
each memory-only NUMA node into a lower tier. But a memory-only
NUMA node may have a high performance memory device (e.g. a DRAM
device attached via CXL.mem or a DRAM-backed memory-only node on
a virtual machine) and should be put into a higher tier.
* The current tier hierarchy always puts CPU nodes into the top
tier. But on a system with HBM (e.g. GPU memory) devices, these
memory-only HBM NUMA nodes should be in the top tier, and DRAM nodes
with CPUs are better to be placed into the next lower tier.
* Also because the current tier hierarchy always puts CPU nodes
into the top tier, when a CPU is hot-added (or hot-removed) and
triggers a memory node from CPU-less into a CPU node (or vice
versa), the memory tier hierarchy gets changed, even though no
memory node is added or removed. This can make the tier
hierarchy unstable and make it difficult to support tier-based
memory accounting.
* A higher tier node can only be demoted to selected nodes on the
next lower tier as defined by the demotion path, not any other
node from any lower tier. This strict, hard-coded demotion order
does not work in all use cases (e.g. some use cases may want to
allow cross-socket demotion to another node in the same demotion
tier as a fallback when the preferred demotion node is out of
space), and has resulted in the feature request for an interface to
override the system-wide, per-node demotion order from the
userspace. This demotion order is also inconsistent with the page
allocation fallback order when all the nodes in a higher tier are
out of space: The page allocation can fall back to any node from
any lower tier, whereas the demotion order doesn't allow that.
* There are no interfaces for the userspace to learn about the memory
tier hierarchy in order to optimize its memory allocations.
This patch series make the creation of memory tiers explicit under
the control of userspace or device driver.
Memory Tier Initialization
==========================
By default, all memory nodes are assigned to the default tier (1).
The default tier device has a rank value (200).
A device driver can move up or down its memory nodes from the default
tier. For example, PMEM can move down its memory nodes below the
default tier, whereas GPU can move up its memory nodes above the
default tier.
The kernel initialization code makes the decision on which exact tier
a memory node should be assigned to based on the requests from the
device drivers as well as the memory device hardware information
provided by the firmware.
Hot-adding/removing CPUs doesn't affect memory tier hierarchy.
Memory Allocation for Demotion
==============================
This patch series keep the demotion target page allocation logic same.
The demotion page allocation pick the closest NUMA node in the
next lower tier to the current NUMA node allocating pages from.
This will be later improved to use the same page allocation strategy
using fallback list.
Sysfs Interface:
=======================
Listing current list of memory tiers and rank details:
:/sys/devices/system/memtier$ ls
default_tier max_tier memtier1 power uevent
:/sys/devices/system/memtier$ cat default_tier
memtier1
:/sys/devices/system/memtier$ cat max_tier
3
:/sys/devices/system/memtier$
Per node memory tier details:
For a cpu only NUMA node:
:/sys/devices/system/node# cat node0/memtier
:/sys/devices/system/node# echo 1 > node0/memtier
:/sys/devices/system/node# cat node0/memtier
:/sys/devices/system/node#
For a NUMA node with memory:
:/sys/devices/system/node# cat node1/memtier
1
:/sys/devices/system/node# ls ../memtier/
default_tier max_tier memtier1 power uevent
:/sys/devices/system/node# echo 2 > node1/memtier
:/sys/devices/system/node#
:/sys/devices/system/node# ls ../memtier/
default_tier max_tier memtier1 memtier2 power uevent
:/sys/devices/system/node# cat node1/memtier
2
:/sys/devices/system/node#
:/sys/devices/system/node# cat ../memtier/memtier2/rank
100
:/sys/devices/system/node#
:/sys/devices/system/node# cat ../memtier/memtier1/rank
200
:/sys/devices/system/node#
Removing a NUMA node from demotion:
:/sys/devices/system/node# cat node1/memtier
2
:/sys/devices/system/node# echo none > node1/memtier
:/sys/devices/system/node#
:/sys/devices/system/node# cat node1/memtier
:/sys/devices/system/node#
:/sys/devices/system/node# ls ../memtier/
default_tier max_tier memtier1 power uevent
:/sys/devices/system/node#
The above also resulted in removal of memtier2 which was created in the earlier step.
Changes from v5:
* Remove patch supporting N_MEMORY node removal from memory tiers. memory tiers
are going to be used for features other than demotion. Hence keep all N_MEMORY
nodes in memory tiers irrespective of whether they want to participate in promotion or demotion.
* Add NODE_DATA->memtier
* Rearrage patches to add sysfs files later.
* Add support to create memory tiers from userspace.
* Address other review feedback.
Changes from v4:
* Address review feedback.
* Reverse the meaning of "rank": higher rank value means higher tier.
* Add "/sys/devices/system/memtier/default_tier".
* Add node_is_toptier
v4:
Add support for explicit memory tiers and ranks.
v3:
- Modify patch 1 subject to make it more specific
- Remove /sys/kernel/mm/numa/demotion_targets interface, use
/sys/devices/system/node/demotion_targets instead and make
it writable to override node_states[N_DEMOTION_TARGETS].
- Add support to view per node demotion targets via sysfs
v2:
In v1, only 1st patch of this patch series was sent, which was
implemented to avoid some of the limitations on the demotion
target sharing, however for certain numa topology, the demotion
targets found by that patch was not most optimal, so 1st patch
in this series is modified according to suggestions from Huang
and Baolin. Different examples of demotion list comparasion
between existing implementation and changed implementation can
be found in the commit message of 1st patch.
Aneesh Kumar K.V (11):
mm/demotion: Add support for explicit memory tiers
mm/demotion: Move memory demotion related code
mm/demotion: Return error on write to numa_demotion sysfs
mm/demotion/dax/kmem: Set node's memory tier to MEMORY_TIER_PMEM
mm/demotion: Build demotion targets based on explicit memory tiers
mm/demotion: Expose memory tier details via sysfs
mm/demotion: Add per node memory tier attribute to sysfs
mm/demotion: Add support for memory tier creation from userspace
mm/demotion: Add pg_data_t member to track node memory tier details
mm/demotion: Update node_is_toptier to work with memory tiers
mm/demotion: Add sysfs ABI documentation
Jagdish Gediya (2):
mm/demotion: Demote pages according to allocation fallback order
mm/demotion: Add documentation for memory tiering
.../ABI/testing/sysfs-kernel-mm-memory-tiers | 87 ++
Documentation/admin-guide/mm/index.rst | 1 +
.../admin-guide/mm/memory-tiering.rst | 181 ++++
drivers/base/node.c | 39 +
drivers/dax/kmem.c | 4 +
include/linux/memory-tiers.h | 63 ++
include/linux/migrate.h | 15 -
include/linux/mmzone.h | 3 +
include/linux/node.h | 5 -
mm/Kconfig | 3 +
mm/Makefile | 1 +
mm/huge_memory.c | 1 +
mm/memory-tiers.c | 888 ++++++++++++++++++
mm/migrate.c | 453 +--------
mm/mprotect.c | 1 +
mm/vmscan.c | 57 +-
mm/vmstat.c | 4 -
17 files changed, 1316 insertions(+), 490 deletions(-)
create mode 100644 Documentation/ABI/testing/sysfs-kernel-mm-memory-tiers
create mode 100644 Documentation/admin-guide/mm/memory-tiering.rst
create mode 100644 include/linux/memory-tiers.h
create mode 100644 mm/memory-tiers.c
--
2.36.1
next reply other threads:[~2022-06-10 13:52 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-10 13:52 Aneesh Kumar K.V [this message]
2022-06-10 13:52 ` [PATCH v6 01/13] mm/demotion: Add support for explicit memory tiers Aneesh Kumar K.V
2022-06-13 3:22 ` Ying Huang
2022-06-13 3:31 ` Aneesh Kumar K V
2022-06-13 5:30 ` Ying Huang
2022-06-13 13:16 ` Johannes Weiner
2022-06-13 13:28 ` Aneesh Kumar K V
2022-06-14 8:20 ` Aneesh Kumar K.V
2022-06-14 15:13 ` Davidlohr Bueso
2022-06-10 13:52 ` [PATCH v6 02/13] mm/demotion: Move memory demotion related code Aneesh Kumar K.V
2022-06-10 13:52 ` [PATCH v6 03/13] mm/demotion: Return error on write to numa_demotion sysfs Aneesh Kumar K.V
2022-06-13 3:26 ` Ying Huang
2022-06-13 3:35 ` Aneesh Kumar K V
2022-06-13 5:33 ` Ying Huang
2022-06-13 5:48 ` Aneesh Kumar K V
2022-06-14 8:40 ` Ying Huang
2022-06-10 13:52 ` [PATCH v6 04/13] mm/demotion/dax/kmem: Set node's memory tier to MEMORY_TIER_PMEM Aneesh Kumar K.V
2022-06-13 6:59 ` Ying Huang
2022-06-13 7:05 ` Aneesh Kumar K V
2022-06-10 13:52 ` [PATCH v6 05/13] mm/demotion: Build demotion targets based on explicit memory tiers Aneesh Kumar K.V
2022-06-10 13:52 ` [PATCH v6 06/13] mm/demotion: Expose memory tier details via sysfs Aneesh Kumar K.V
2022-06-10 13:52 ` [PATCH v6 07/13] mm/demotion: Add per node memory tier attribute to sysfs Aneesh Kumar K.V
2022-06-10 13:52 ` [PATCH v6 08/13] mm/demotion: Add support for memory tier creation from userspace Aneesh Kumar K.V
2022-06-10 13:52 ` [PATCH v6 09/13] mm/demotion: Add pg_data_t member to track node memory tier details Aneesh Kumar K.V
2022-06-13 7:07 ` Ying Huang
2022-06-10 13:52 ` [PATCH v6 10/13] mm/demotion: Demote pages according to allocation fallback order Aneesh Kumar K.V
2022-06-10 13:52 ` [PATCH v6 11/13] mm/demotion: Update node_is_toptier to work with memory tiers Aneesh Kumar K.V
2022-06-10 13:52 ` [PATCH v6 12/13] mm/demotion: Add documentation for memory tiering Aneesh Kumar K.V
2022-06-10 13:52 ` [PATCH v6 13/13] mm/demotion: Add sysfs ABI documentation Aneesh Kumar K.V
-- strict thread matches above, loose matches on Subject: below --
2022-06-10 13:49 [PATCH v6 00/13] mm/demotion: Memory tiers and demotion Aneesh Kumar K.V
2022-06-20 9:18 ` Hesham Almatary
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220610135229.182859-1-aneesh.kumar@linux.ibm.com \
--to=aneesh.kumar@linux.ibm.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=apopple@nvidia.com \
--cc=baolin.wang@linux.alibaba.com \
--cc=brice.goglin@gmail.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=dave@stgolabs.net \
--cc=feng.tang@intel.com \
--cc=gthelen@google.com \
--cc=hesham.almatary@huawei.com \
--cc=jvgediya@linux.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=rientjes@google.com \
--cc=shy828301@gmail.com \
--cc=tim.c.chen@intel.com \
--cc=weixugc@google.com \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).