From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: virtualization@lists.linux-foundation.org, linux-mm@kvack.org,
linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org,
linux-acpi@vger.kernel.org, linux-nvdimm@lists.01.org,
linux-s390@vger.kernel.org,
"Andrew Morton" <akpm@linux-foundation.org>,
"David Hildenbrand" <david@redhat.com>,
"Anton Blanchard" <anton@ozlabs.org>,
"Ard Biesheuvel" <ardb@kernel.org>, "Baoquan He" <bhe@redhat.com>,
"Benjamin Herrenschmidt" <benh@kernel.crashing.org>,
"Boris Ostrovsky" <boris.ostrovsky@oracle.com>,
"Christian Borntraeger" <borntraeger@de.ibm.com>,
"Eric Biederman" <ebiederm@xmission.com>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
"Haiyang Zhang" <haiyangz@microsoft.com>,
"Heiko Carstens" <hca@linux.ibm.com>,
"Jason Gunthorpe" <jgg@ziepe.ca>,
"Jason Wang" <jasowang@redhat.com>,
"Juergen Gross" <jgross@suse.com>,
"Julien Grall" <julien@xen.org>,
"Kees Cook" <keescook@chromium.org>,
"K. Y. Srinivasan" <kys@microsoft.com>,
"Len Brown" <lenb@kernel.org>,
"Leonardo Bras" <leobras.c@gmail.com>,
"Libor Pechacek" <lpechacek@suse.cz>,
"Michael Ellerman" <mpe@ellerman.id.au>,
"Michael S. Tsirkin" <mst@redhat.com>,
"Michal Hocko" <mhocko@suse.com>,
"Nathan Lynch" <nathanl@linux.ibm.com>,
"Pankaj Gupta" <pankaj.gupta.linux@gmail.com>,
"Paul Mackerras" <paulus@samba.org>,
"Pingfan Liu" <kernelfans@gmail.com>,
"Rafael J. Wysocki" <rjw@rjwysocki.net>,
"Roger Pau Monné" <roger.pau@citrix.com>,
"Stefano Stabellini" <sstabellini@kernel.org>,
"Stephen Hemminger" <sthemmin@microsoft.com>,
"Thomas Gleixner" <tglx@linutronix.de>,
"Vasily Gorbik" <gor@linux.ibm.com>,
"Wei Liu" <wei.liu@kernel.org>
Subject: [PATCH v2 0/7] mm/memory_hotplug: selective merging of system ram resources
Date: Tue, 8 Sep 2020 22:10:05 +0200 [thread overview]
Message-ID: <20200908201012.44168-1-david@redhat.com> (raw)
Some add_memory*() users add memory in small, contiguous memory blocks.
Examples include virtio-mem, hyper-v balloon, and the XEN balloon.
This can quickly result in a lot of memory resources, whereby the actual
resource boundaries are not of interest (e.g., it might be relevant for
DIMMs, exposed via /proc/iomem to user space). We really want to merge
added resources in this scenario where possible.
Resources are effectively stored in a list-based tree. Having a lot of
resources not only wastes memory, it also makes traversing that tree more
expensive, and makes /proc/iomem explode in size (e.g., requiring
kexec-tools to manually merge resources when creating a kdump header. The
current kexec-tools resource count limit does not allow for more than
~100GB of memory with a memory block size of 128MB on x86-64).
Let's allow to selectively merge system ram resources by specifying a
new flag for add_memory*(). Patch #5 contains a /proc/iomem example. Only
tested with virtio-mem.
v1 -> v2:
- I had another look at v1 after vacation and didn't like it - it felt like
a hack. So I want forward and added a proper flag to add_memory*(), and
introduce a clean (non-racy) way to mark System RAM resources mergeable.
- "kernel/resource: move and rename IORESOURCE_MEM_DRIVER_MANAGED"
-- Clean that flag up, felt wrong in the PnP section
- "mm/memory_hotplug: prepare passing flags to add_memory() and friends"
-- Previously sent in other context - decided to keep Wei's ack
- "mm/memory_hotplug: MEMHP_MERGE_RESOURCE to specify merging of System
RAM resources"
-- Cleaner approach to get the job done by using proper flags and only
merging the single, specified resource
- "virtio-mem: try to merge system ram resources"
"xen/balloon: try to merge system ram resources"
"hv_balloon: try to merge system ram resources"
-- Use the new flag MEMHP_MERGE_RESOURCE, much cleaner
RFC -> v1:
- Switch from rather generic "merge_child_mem_resources()" where a resource
name has to be specified to "merge_system_ram_resources().
- Smaller comment/documentation/patch description changes/fixes
David Hildenbrand (7):
kernel/resource: make release_mem_region_adjustable() never fail
kernel/resource: move and rename IORESOURCE_MEM_DRIVER_MANAGED
mm/memory_hotplug: prepare passing flags to add_memory() and friends
mm/memory_hotplug: MEMHP_MERGE_RESOURCE to specify merging of System
RAM resources
virtio-mem: try to merge system ram resources
xen/balloon: try to merge system ram resources
hv_balloon: try to merge system ram resources
arch/powerpc/platforms/powernv/memtrace.c | 2 +-
.../platforms/pseries/hotplug-memory.c | 2 +-
drivers/acpi/acpi_memhotplug.c | 2 +-
drivers/base/memory.c | 2 +-
drivers/dax/kmem.c | 2 +-
drivers/hv/hv_balloon.c | 2 +-
drivers/s390/char/sclp_cmd.c | 2 +-
drivers/virtio/virtio_mem.c | 3 +-
drivers/xen/balloon.c | 2 +-
include/linux/ioport.h | 12 +-
include/linux/memory_hotplug.h | 19 ++-
kernel/kexec_file.c | 2 +-
kernel/resource.c | 109 ++++++++++++++----
mm/memory_hotplug.c | 48 +++-----
14 files changed, 141 insertions(+), 68 deletions(-)
--
2.26.2
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
next reply other threads:[~2020-09-08 20:10 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-08 20:10 David Hildenbrand [this message]
2020-09-08 20:10 ` [PATCH v2 1/7] kernel/resource: make release_mem_region_adjustable() never fail David Hildenbrand
2020-09-15 2:07 ` Wei Yang
2020-09-15 2:10 ` Wei Yang
2020-09-15 7:35 ` David Hildenbrand
2020-09-15 9:06 ` Wei Yang
2020-09-15 9:15 ` David Hildenbrand
2020-09-15 9:33 ` Wei Yang
2020-09-08 20:10 ` [PATCH v2 2/7] kernel/resource: move and rename IORESOURCE_MEM_DRIVER_MANAGED David Hildenbrand
2020-09-09 7:16 ` Greg Kroah-Hartman
2020-09-09 7:27 ` David Hildenbrand
2020-09-15 2:20 ` Wei Yang
2020-09-15 7:37 ` David Hildenbrand
2020-09-08 20:10 ` [PATCH v2 3/7] mm/memory_hotplug: prepare passing flags to add_memory() and friends David Hildenbrand
2020-09-09 5:18 ` Jürgen Groß
2020-09-09 7:17 ` Greg Kroah-Hartman
2020-09-09 7:28 ` David Hildenbrand
2020-09-09 11:24 ` Michael Ellerman
2020-09-09 11:37 ` David Hildenbrand
2020-09-09 11:51 ` David Hildenbrand
2020-09-08 20:10 ` [PATCH v2 4/7] mm/memory_hotplug: MEMHP_MERGE_RESOURCE to specify merging of System RAM resources David Hildenbrand
2020-09-08 20:10 ` [PATCH v2 5/7] virtio-mem: try to merge system ram resources David Hildenbrand
2020-09-08 20:10 ` [PATCH v2 6/7] xen/balloon: " David Hildenbrand
2020-09-09 5:18 ` Jürgen Groß
2020-09-08 20:10 ` [PATCH v2 7/7] hv_balloon: " David Hildenbrand
2020-09-09 9:43 ` Wei Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200908201012.44168-1-david@redhat.com \
--to=david@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=anton@ozlabs.org \
--cc=ardb@kernel.org \
--cc=benh@kernel.crashing.org \
--cc=bhe@redhat.com \
--cc=boris.ostrovsky@oracle.com \
--cc=borntraeger@de.ibm.com \
--cc=ebiederm@xmission.com \
--cc=gor@linux.ibm.com \
--cc=gregkh@linuxfoundation.org \
--cc=haiyangz@microsoft.com \
--cc=hca@linux.ibm.com \
--cc=jasowang@redhat.com \
--cc=jgg@ziepe.ca \
--cc=jgross@suse.com \
--cc=julien@xen.org \
--cc=keescook@chromium.org \
--cc=kernelfans@gmail.com \
--cc=kys@microsoft.com \
--cc=lenb@kernel.org \
--cc=leobras.c@gmail.com \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linux-s390@vger.kernel.org \
--cc=lpechacek@suse.cz \
--cc=mhocko@suse.com \
--cc=mpe@ellerman.id.au \
--cc=mst@redhat.com \
--cc=nathanl@linux.ibm.com \
--cc=pankaj.gupta.linux@gmail.com \
--cc=paulus@samba.org \
--cc=rjw@rjwysocki.net \
--cc=roger.pau@citrix.com \
--cc=sstabellini@kernel.org \
--cc=sthemmin@microsoft.com \
--cc=tglx@linutronix.de \
--cc=virtualization@lists.linux-foundation.org \
--cc=wei.liu@kernel.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).