From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org, David Hildenbrand <david@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
Jason Wang <jasowang@redhat.com>,
Marek Kedzierski <mkedzier@redhat.com>,
Hui Zhu <teawater@gmail.com>,
Pankaj Gupta <pankaj.gupta.linux@gmail.com>,
Wei Yang <richard.weiyang@linux.alibaba.com>,
Oscar Salvador <osalvador@suse.de>,
Michal Hocko <mhocko@kernel.org>,
Dan Williams <dan.j.williams@intel.com>,
Anshuman Khandual <anshuman.khandual@arm.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
Vlastimil Babka <vbabka@suse.cz>, Mike Rapoport <rppt@kernel.org>,
"Rafael J. Wysocki" <rjw@rjwysocki.net>,
Len Brown <lenb@kernel.org>,
Pavel Tatashin <pasha.tatashin@soleen.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
virtualization@lists.linux-foundation.org,
linux-acpi@vger.kernel.org
Subject: [PATCH v2 4/9] mm/memory_hotplug: track present pages in memory groups
Date: Fri, 23 Jul 2021 14:52:05 +0200 [thread overview]
Message-ID: <20210723125210.29987-5-david@redhat.com> (raw)
In-Reply-To: <20210723125210.29987-1-david@redhat.com>
Let's track all present pages in each memory group. Especially, track
memory present in ZONE_MOVABLE and memory present in one of the kernel
zones (which really only is ZONE_NORMAL right now as memory groups only
apply to hotplugged memory) separately within a memory group, to prepare
for making smart auto-online decision for individual memory blocks within a
memory group based on group statistics.
Signed-off-by: David Hildenbrand <david@redhat.com>
---
drivers/base/memory.c | 10 +++++-----
include/linux/memory.h | 4 ++++
include/linux/memory_hotplug.h | 13 +++++++++----
mm/memory_hotplug.c | 19 ++++++++++++++-----
4 files changed, 32 insertions(+), 14 deletions(-)
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 42109e7fb0b5..e96c4f436ac3 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -198,7 +198,7 @@ static int memory_block_online(struct memory_block *mem)
}
ret = online_pages(start_pfn + nr_vmemmap_pages,
- nr_pages - nr_vmemmap_pages, zone);
+ nr_pages - nr_vmemmap_pages, zone, mem->group);
if (ret) {
if (nr_vmemmap_pages)
mhp_deinit_memmap_on_memory(start_pfn, nr_vmemmap_pages);
@@ -210,7 +210,7 @@ static int memory_block_online(struct memory_block *mem)
* now already properly populated.
*/
if (nr_vmemmap_pages)
- adjust_present_page_count(pfn_to_page(start_pfn),
+ adjust_present_page_count(pfn_to_page(start_pfn), mem->group,
nr_vmemmap_pages);
return ret;
@@ -228,16 +228,16 @@ static int memory_block_offline(struct memory_block *mem)
* can properly be torn down in offline_pages().
*/
if (nr_vmemmap_pages)
- adjust_present_page_count(pfn_to_page(start_pfn),
+ adjust_present_page_count(pfn_to_page(start_pfn), mem->group,
-nr_vmemmap_pages);
ret = offline_pages(start_pfn + nr_vmemmap_pages,
- nr_pages - nr_vmemmap_pages);
+ nr_pages - nr_vmemmap_pages, mem->group);
if (ret) {
/* offline_pages() failed. Account back. */
if (nr_vmemmap_pages)
adjust_present_page_count(pfn_to_page(start_pfn),
- nr_vmemmap_pages);
+ mem->group, nr_vmemmap_pages);
return ret;
}
diff --git a/include/linux/memory.h b/include/linux/memory.h
index 6e20a6174fe5..0eceb8467d9a 100644
--- a/include/linux/memory.h
+++ b/include/linux/memory.h
@@ -28,6 +28,10 @@ struct memory_group {
int nid;
/* References from memory blocks + 1. */
refcount_t refcount;
+ /* Present (online) memory outside ZONE_MOVABLE of this memory group. */
+ unsigned long present_kernel_pages;
+ /* Present (online) memory in ZONE_MOVABLE of this memory group. */
+ unsigned long present_movable_pages;
/*
* Memory group type: static vs. dynamic.
*
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index 5beb65497ae3..23c4d369ad30 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -12,6 +12,7 @@ struct zone;
struct pglist_data;
struct mem_section;
struct memory_block;
+struct memory_group;
struct resource;
struct vmem_altmap;
@@ -99,13 +100,15 @@ static inline void zone_seqlock_init(struct zone *zone)
extern int zone_grow_free_lists(struct zone *zone, unsigned long new_nr_pages);
extern int zone_grow_waitqueues(struct zone *zone, unsigned long nr_pages);
extern int add_one_highpage(struct page *page, int pfn, int bad_ppro);
-extern void adjust_present_page_count(struct page *page, long nr_pages);
+extern void adjust_present_page_count(struct page *page,
+ struct memory_group *group,
+ long nr_pages);
/* VM interface that may be used by firmware interface */
extern int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
struct zone *zone);
extern void mhp_deinit_memmap_on_memory(unsigned long pfn, unsigned long nr_pages);
extern int online_pages(unsigned long pfn, unsigned long nr_pages,
- struct zone *zone);
+ struct zone *zone, struct memory_group *group);
extern struct zone *test_pages_in_a_zone(unsigned long start_pfn,
unsigned long end_pfn);
extern void __offline_isolated_pages(unsigned long start_pfn,
@@ -295,7 +298,8 @@ static inline void pgdat_resize_init(struct pglist_data *pgdat) {}
#ifdef CONFIG_MEMORY_HOTREMOVE
extern void try_offline_node(int nid);
-extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages);
+extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
+ struct memory_group *group);
extern int remove_memory(u64 start, u64 size);
extern void __remove_memory(u64 start, u64 size);
extern int offline_and_remove_memory(u64 start, u64 size);
@@ -303,7 +307,8 @@ extern int offline_and_remove_memory(u64 start, u64 size);
#else
static inline void try_offline_node(int nid) {}
-static inline int offline_pages(unsigned long start_pfn, unsigned long nr_pages)
+static inline int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
+ struct memory_group *group)
{
return -EINVAL;
}
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index b091d21d8796..8d556396b5d4 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -913,9 +913,11 @@ struct zone *zone_for_pfn_range(int online_type, int nid,
* This function should only be called by memory_block_{online,offline},
* and {online,offline}_pages.
*/
-void adjust_present_page_count(struct page *page, long nr_pages)
+void adjust_present_page_count(struct page *page, struct memory_group *group,
+ long nr_pages)
{
struct zone *zone = page_zone(page);
+ const bool movable = zone_idx(zone) == ZONE_MOVABLE;
/*
* We only support onlining/offlining/adding/removing of complete
@@ -925,6 +927,11 @@ void adjust_present_page_count(struct page *page, long nr_pages)
zone->present_early_pages += nr_pages;
zone->present_pages += nr_pages;
zone->zone_pgdat->node_present_pages += nr_pages;
+
+ if (group && movable)
+ group->present_movable_pages += nr_pages;
+ else if (group && !movable)
+ group->present_kernel_pages += nr_pages;
}
int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
@@ -970,7 +977,8 @@ void mhp_deinit_memmap_on_memory(unsigned long pfn, unsigned long nr_pages)
kasan_remove_zero_shadow(__va(PFN_PHYS(pfn)), PFN_PHYS(nr_pages));
}
-int __ref online_pages(unsigned long pfn, unsigned long nr_pages, struct zone *zone)
+int __ref online_pages(unsigned long pfn, unsigned long nr_pages,
+ struct zone *zone, struct memory_group *group)
{
unsigned long flags;
int need_zonelists_rebuild = 0;
@@ -1023,7 +1031,7 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages, struct zone *z
}
online_pages_range(pfn, nr_pages);
- adjust_present_page_count(pfn_to_page(pfn), nr_pages);
+ adjust_present_page_count(pfn_to_page(pfn), group, nr_pages);
node_states_set_node(nid, &arg);
if (need_zonelists_rebuild)
@@ -1774,7 +1782,8 @@ static int count_system_ram_pages_cb(unsigned long start_pfn,
return 0;
}
-int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages)
+int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages,
+ struct memory_group *group)
{
const unsigned long end_pfn = start_pfn + nr_pages;
unsigned long pfn, system_ram_pages = 0;
@@ -1910,7 +1919,7 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages)
/* removal success */
adjust_managed_page_count(pfn_to_page(start_pfn), -nr_pages);
- adjust_present_page_count(pfn_to_page(start_pfn), -nr_pages);
+ adjust_present_page_count(pfn_to_page(start_pfn), group, -nr_pages);
/* reinitialise watermarks and update pcp limits */
init_per_zone_wmark_min();
--
2.31.1
next prev parent reply other threads:[~2021-07-23 12:53 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-23 12:52 [PATCH v2 0/9] mm/memory_hotplug: "auto-movable" online policy and memory groups David Hildenbrand
2021-07-23 12:52 ` [PATCH v2 1/9] mm: track present early pages per zone David Hildenbrand
2021-07-23 12:52 ` [PATCH v2 2/9] mm/memory_hotplug: introduce "auto-movable" online policy David Hildenbrand
2021-07-26 7:15 ` David Hildenbrand
2021-07-23 12:52 ` [PATCH v2 3/9] drivers/base/memory: introduce "memory groups" to logically group memory blocks David Hildenbrand
2021-07-28 13:39 ` Greg Kroah-Hartman
2021-07-28 14:16 ` David Hildenbrand
2021-07-23 12:52 ` David Hildenbrand [this message]
2021-07-23 12:52 ` [PATCH v2 5/9] ACPI: memhotplug: use a single static memory group for a single memory device David Hildenbrand
2021-07-23 12:52 ` [PATCH v2 6/9] dax/kmem: use a single static memory group for a single probed unit David Hildenbrand
2021-07-23 12:52 ` [PATCH v2 7/9] virtio-mem: use a single dynamic memory group for a single virtio-mem device David Hildenbrand
2021-07-23 12:52 ` [PATCH v2 8/9] mm/memory_hotplug: memory group aware "auto-movable" online policy David Hildenbrand
2021-07-23 12:52 ` [PATCH v2 9/9] mm/memory_hotplug: improved dynamic " David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210723125210.29987-5-david@redhat.com \
--to=david@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=anshuman.khandual@arm.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=gregkh@linuxfoundation.org \
--cc=jasowang@redhat.com \
--cc=lenb@kernel.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=mkedzier@redhat.com \
--cc=mst@redhat.com \
--cc=osalvador@suse.de \
--cc=pankaj.gupta.linux@gmail.com \
--cc=pasha.tatashin@soleen.com \
--cc=richard.weiyang@linux.alibaba.com \
--cc=rjw@rjwysocki.net \
--cc=rppt@kernel.org \
--cc=teawater@gmail.com \
--cc=vbabka@suse.cz \
--cc=virtualization@lists.linux-foundation.org \
--cc=vkuznets@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).