linux-nvdimm.lists.01.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 0/7] mm/memory_hotplug: selective merging of system ram resources
@ 2020-09-08 20:10 David Hildenbrand
  2020-09-08 20:10 ` [PATCH v2 1/7] kernel/resource: make release_mem_region_adjustable() never fail David Hildenbrand
                   ` (6 more replies)
  0 siblings, 7 replies; 26+ messages in thread
From: David Hildenbrand @ 2020-09-08 20:10 UTC (permalink / raw)
  To: linux-kernel
  Cc: virtualization, linux-mm, linux-hyperv, xen-devel, linux-acpi,
	linux-nvdimm, linux-s390, Andrew Morton, David Hildenbrand,
	Anton Blanchard, Ard Biesheuvel, Baoquan He,
	Benjamin Herrenschmidt, Boris Ostrovsky, Christian Borntraeger,
	Eric Biederman, Greg Kroah-Hartman, Haiyang Zhang,
	Heiko Carstens, Jason Gunthorpe, Jason Wang, Juergen Gross,
	Julien Grall, Kees Cook, K. Y. Srinivasan, Len Brown,
	Leonardo Bras, Libor Pechacek, Michael Ellerman,
	Michael S. Tsirkin, Michal Hocko, Nathan Lynch, Pankaj Gupta,
	Paul Mackerras, Pingfan Liu, Rafael J. Wysocki,
	Roger Pau Monné,
	Stefano Stabellini, Stephen Hemminger, Thomas Gleixner,
	Vasily Gorbik, Wei Liu

Some add_memory*() users add memory in small, contiguous memory blocks.
Examples include virtio-mem, hyper-v balloon, and the XEN balloon.

This can quickly result in a lot of memory resources, whereby the actual
resource boundaries are not of interest (e.g., it might be relevant for
DIMMs, exposed via /proc/iomem to user space). We really want to merge
added resources in this scenario where possible.

Resources are effectively stored in a list-based tree. Having a lot of
resources not only wastes memory, it also makes traversing that tree more
expensive, and makes /proc/iomem explode in size (e.g., requiring
kexec-tools to manually merge resources when creating a kdump header. The
current kexec-tools resource count limit does not allow for more than
~100GB of memory with a memory block size of 128MB on x86-64).

Let's allow to selectively merge system ram resources by specifying a
new flag for add_memory*(). Patch #5 contains a /proc/iomem example. Only
tested with virtio-mem.

v1 -> v2:
- I had another look at v1 after vacation and didn't like it - it felt like
  a hack. So I want forward and added a proper flag to add_memory*(), and
  introduce a clean (non-racy) way to mark System RAM resources mergeable.
- "kernel/resource: move and rename IORESOURCE_MEM_DRIVER_MANAGED"
-- Clean that flag up, felt wrong in the PnP section
- "mm/memory_hotplug: prepare passing flags to add_memory() and friends"
-- Previously sent in other context - decided to keep Wei's ack
- "mm/memory_hotplug: MEMHP_MERGE_RESOURCE to specify merging of System
   RAM resources"
-- Cleaner approach to get the job done by using proper flags and only
   merging the single, specified resource
- "virtio-mem: try to merge system ram resources"
  "xen/balloon: try to merge system ram resources"
  "hv_balloon: try to merge system ram resources"
-- Use the new flag MEMHP_MERGE_RESOURCE, much cleaner

RFC -> v1:
- Switch from rather generic "merge_child_mem_resources()" where a resource
  name has to be specified to "merge_system_ram_resources().
- Smaller comment/documentation/patch description changes/fixes

David Hildenbrand (7):
  kernel/resource: make release_mem_region_adjustable() never fail
  kernel/resource: move and rename IORESOURCE_MEM_DRIVER_MANAGED
  mm/memory_hotplug: prepare passing flags to add_memory() and friends
  mm/memory_hotplug: MEMHP_MERGE_RESOURCE to specify merging of System
    RAM resources
  virtio-mem: try to merge system ram resources
  xen/balloon: try to merge system ram resources
  hv_balloon: try to merge system ram resources

 arch/powerpc/platforms/powernv/memtrace.c     |   2 +-
 .../platforms/pseries/hotplug-memory.c        |   2 +-
 drivers/acpi/acpi_memhotplug.c                |   2 +-
 drivers/base/memory.c                         |   2 +-
 drivers/dax/kmem.c                            |   2 +-
 drivers/hv/hv_balloon.c                       |   2 +-
 drivers/s390/char/sclp_cmd.c                  |   2 +-
 drivers/virtio/virtio_mem.c                   |   3 +-
 drivers/xen/balloon.c                         |   2 +-
 include/linux/ioport.h                        |  12 +-
 include/linux/memory_hotplug.h                |  19 ++-
 kernel/kexec_file.c                           |   2 +-
 kernel/resource.c                             | 109 ++++++++++++++----
 mm/memory_hotplug.c                           |  48 +++-----
 14 files changed, 141 insertions(+), 68 deletions(-)

-- 
2.26.2
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org

^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2020-09-15  9:33 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-08 20:10 [PATCH v2 0/7] mm/memory_hotplug: selective merging of system ram resources David Hildenbrand
2020-09-08 20:10 ` [PATCH v2 1/7] kernel/resource: make release_mem_region_adjustable() never fail David Hildenbrand
2020-09-15  2:07   ` Wei Yang
2020-09-15  2:10   ` Wei Yang
2020-09-15  7:35     ` David Hildenbrand
2020-09-15  9:06       ` Wei Yang
2020-09-15  9:15         ` David Hildenbrand
2020-09-15  9:33           ` Wei Yang
2020-09-08 20:10 ` [PATCH v2 2/7] kernel/resource: move and rename IORESOURCE_MEM_DRIVER_MANAGED David Hildenbrand
2020-09-09  7:16   ` Greg Kroah-Hartman
2020-09-09  7:27     ` David Hildenbrand
2020-09-15  2:20   ` Wei Yang
2020-09-15  7:37     ` David Hildenbrand
2020-09-08 20:10 ` [PATCH v2 3/7] mm/memory_hotplug: prepare passing flags to add_memory() and friends David Hildenbrand
2020-09-09  5:18   ` Jürgen Groß
2020-09-09  7:17   ` Greg Kroah-Hartman
2020-09-09  7:28     ` David Hildenbrand
2020-09-09 11:24       ` Michael Ellerman
2020-09-09 11:37         ` David Hildenbrand
2020-09-09 11:51           ` David Hildenbrand
2020-09-08 20:10 ` [PATCH v2 4/7] mm/memory_hotplug: MEMHP_MERGE_RESOURCE to specify merging of System RAM resources David Hildenbrand
2020-09-08 20:10 ` [PATCH v2 5/7] virtio-mem: try to merge system ram resources David Hildenbrand
2020-09-08 20:10 ` [PATCH v2 6/7] xen/balloon: " David Hildenbrand
2020-09-09  5:18   ` Jürgen Groß
2020-09-08 20:10 ` [PATCH v2 7/7] hv_balloon: " David Hildenbrand
2020-09-09  9:43   ` Wei Liu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).