* [PATCH] powerpc/memtrace: Remove memory in chunks
@ 2018-08-17 4:25 Rashmica Gupta
2018-09-20 4:20 ` Michael Ellerman
0 siblings, 1 reply; 2+ messages in thread
From: Rashmica Gupta @ 2018-08-17 4:25 UTC (permalink / raw)
To: mpe, bsingharora, mikey, linuxppc-dev, anton; +Cc: Rashmica Gupta
When hot-removing memory release_mem_region_adjustable() splits iomem
resources if they are not the exact size of the memory being
hot-deleted. Adding this memory back to the kernel adds a new resource.
Eg a node has memory 0x0 - 0xfffffffff. Hot-removing 1GB from
0xf40000000 results in the single resource 0x0-0xfffffffff being split
into two resources: 0x0-0xf3fffffff and 0xf80000000-0xfffffffff.
When we hot-add the memory back we now have three resources:
0x0-0xf3fffffff, 0xf40000000-0xf7fffffff, and 0xf80000000-0xfffffffff.
This is an issue if we try to remove some memory that overlaps
resources. Eg when trying to remove 2GB at address 0xf40000000,
release_mem_region_adjustable() fails as it expects the chunk of memory
to be within the boundaries of a single resource. We then get the
warning: "Unable to release resource" and attempting to use memtrace
again gives us this error: "bash: echo: write error: Resource
temporarily unavailable"
This patch makes memtrace remove memory in chunks that are always the
same size from an address that is always equal to end_of_memory -
n*size, for some n. So hotremoving and hotadding memory of different
sizes will now not attempt to remove memory that spans multiple
resources.
Signed-off-by: Rashmica Gupta <rashmica.g@gmail.com>
---
To replicate the above issue hot-remove and hot-add memory of different
sizes a bunch of times. This does it for me on POWER8 and POWER9:
for i in `seq 1 10`; do
echo $(( $i * 268435456)) > /sys/kernel/debug/powerpc/memtrace/enable
echo '.'
done
arch/powerpc/platforms/powernv/memtrace.c | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/memtrace.c b/arch/powerpc/platforms/powernv/memtrace.c
index 51fe0862dcab..c5749f898652 100644
--- a/arch/powerpc/platforms/powernv/memtrace.c
+++ b/arch/powerpc/platforms/powernv/memtrace.c
@@ -119,17 +119,15 @@ static bool memtrace_offline_pages(u32 nid, u64 start_pfn, u64 nr_pages)
walk_memory_range(start_pfn, end_pfn, (void *)MEM_OFFLINE,
change_memblock_state);
- lock_device_hotplug();
- remove_memory(nid, start_pfn << PAGE_SHIFT, nr_pages << PAGE_SHIFT);
- unlock_device_hotplug();
return true;
}
static u64 memtrace_alloc_node(u32 nid, u64 size)
{
- u64 start_pfn, end_pfn, nr_pages;
+ u64 start_pfn, end_pfn, nr_pages, pfn;
u64 base_pfn;
+ u64 bytes = memory_block_size_bytes();
if (!node_spanned_pages(nid))
return 0;
@@ -142,8 +140,20 @@ static u64 memtrace_alloc_node(u32 nid, u64 size)
end_pfn = round_down(end_pfn - nr_pages, nr_pages);
for (base_pfn = end_pfn; base_pfn > start_pfn; base_pfn -= nr_pages) {
- if (memtrace_offline_pages(nid, base_pfn, nr_pages) == true)
+ if (memtrace_offline_pages(nid, base_pfn, nr_pages) == true) {
+ /* Remove memory in memory block size chunks so that
+ * iomem resources are always split to the same size
+ * and we never try to remove memory that spans two
+ * iomem resources.
+ */
+ lock_device_hotplug();
+ end_pfn = base_pfn + nr_pages;
+ for (pfn = base_pfn; pfn < end_pfn; pfn += bytes>> PAGE_SHIFT) {
+ remove_memory(nid, pfn << PAGE_SHIFT, bytes);
+ }
+ unlock_device_hotplug();
return base_pfn << PAGE_SHIFT;
+ }
}
return 0;
--
2.14.4
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: powerpc/memtrace: Remove memory in chunks
2018-08-17 4:25 [PATCH] powerpc/memtrace: Remove memory in chunks Rashmica Gupta
@ 2018-09-20 4:20 ` Michael Ellerman
0 siblings, 0 replies; 2+ messages in thread
From: Michael Ellerman @ 2018-09-20 4:20 UTC (permalink / raw)
To: Rashmica Gupta, bsingharora, mikey, linuxppc-dev, anton; +Cc: Rashmica Gupta
On Fri, 2018-08-17 at 04:25:01 UTC, Rashmica Gupta wrote:
> When hot-removing memory release_mem_region_adjustable() splits iomem
> resources if they are not the exact size of the memory being
> hot-deleted. Adding this memory back to the kernel adds a new resource.
>
> Eg a node has memory 0x0 - 0xfffffffff. Hot-removing 1GB from
> 0xf40000000 results in the single resource 0x0-0xfffffffff being split
> into two resources: 0x0-0xf3fffffff and 0xf80000000-0xfffffffff.
>
> When we hot-add the memory back we now have three resources:
> 0x0-0xf3fffffff, 0xf40000000-0xf7fffffff, and 0xf80000000-0xfffffffff.
>
> This is an issue if we try to remove some memory that overlaps
> resources. Eg when trying to remove 2GB at address 0xf40000000,
> release_mem_region_adjustable() fails as it expects the chunk of memory
> to be within the boundaries of a single resource. We then get the
> warning: "Unable to release resource" and attempting to use memtrace
> again gives us this error: "bash: echo: write error: Resource
> temporarily unavailable"
>
> This patch makes memtrace remove memory in chunks that are always the
> same size from an address that is always equal to end_of_memory -
> n*size, for some n. So hotremoving and hotadding memory of different
> sizes will now not attempt to remove memory that spans multiple
> resources.
>
> Signed-off-by: Rashmica Gupta <rashmica.g@gmail.com>
Applied to powerpc next, thanks.
https://git.kernel.org/powerpc/c/3f7daf3d7582dc6628ac40a9045dd1
cheers
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2018-09-20 4:20 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-17 4:25 [PATCH] powerpc/memtrace: Remove memory in chunks Rashmica Gupta
2018-09-20 4:20 ` Michael Ellerman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).