xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: virtualization@lists.linux-foundation.org, linux-mm@kvack.org,
	linux-hyperv@vger.kernel.org, xen-devel@lists.xenproject.org,
	"David Hildenbrand" <david@redhat.com>,
	"Andrew Morton" <akpm@linux-foundation.org>,
	"Ard Biesheuvel" <ardb@kernel.org>, "Baoquan He" <bhe@redhat.com>,
	"Boris Ostrovsky" <boris.ostrovsky@oracle.com>,
	"Dan Williams" <dan.j.williams@intel.com>,
	"Haiyang Zhang" <haiyangz@microsoft.com>,
	"Jason Gunthorpe" <jgg@ziepe.ca>,
	"Jason Wang" <jasowang@redhat.com>,
	"Juergen Gross" <jgross@suse.com>,
	"Julien Grall" <julien@xen.org>,
	"Kees Cook" <keescook@chromium.org>,
	"K. Y. Srinivasan" <kys@microsoft.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	"Michal Hocko" <mhocko@suse.com>,
	"Pankaj Gupta" <pankaj.gupta.linux@gmail.com>,
	"Roger Pau Monné" <roger.pau@citrix.com>,
	"Stefano Stabellini" <sstabellini@kernel.org>,
	"Stephen Hemminger" <sthemmin@microsoft.com>,
	"Thomas Gleixner" <tglx@linutronix.de>,
	"Wei Liu" <wei.liu@kernel.org>,
	"Wei Yang" <richardw.yang@linux.intel.com>
Subject: [PATCH v1 0/5] mm/memory_hotplug: selective merging of system ram resources
Date: Fri, 21 Aug 2020 12:34:26 +0200	[thread overview]
Message-ID: <20200821103431.13481-1-david@redhat.com> (raw)

This is the follow-up of "[PATCH RFCv1 0/5] mm/memory_hotplug: selective
merging of memory resources" [1]

Some add_memory*() users add memory in small, contiguous memory blocks.
Examples include virtio-mem, hyper-v balloon, and the XEN balloon.

This can quickly result in a lot of memory resources, whereby the actual
resource boundaries are not of interest (e.g., it might be relevant for
DIMMs, exposed via /proc/iomem to user space). We really want to merge
added resources in this scenario where possible.

Resources are effectively stored in a list-based tree. Having a lot of
resources not only wastes memory, it also makes traversing that tree more
expensive, and makes /proc/iomem explode in size (e.g., requiring
kexec-tools to manually merge resources when creating a kdump header. The
current kexec-tools resource count limit does not allow for more than
~100GB of memory with a memory block size of 128MB on x86-64).

Let's allow to selectively merge system ram resources directly below a
specific parent resource. Patch #3 contains a /proc/iomem example. Only
tested with virtio-mem.

Note: This gets the job done and is comparably simple. More complicated
approaches would require introducing IORESOURCE_MERGEABLE and extending our
add_memory*() interfaces with a flag, specifying that merging after adding
succeeded is acceptable. I'd like to avoid that complexity and code churn
for now.

[1] https://lkml.kernel.org/r/20200731091838.7490-1-david@redhat.com

RFC -> v1:
- Switch from rather generic "merge_child_mem_resources()" where a resource
  name has to be specified to "merge_system_ram_resources().
- Smaller comment/documentation/patch description changes/fixes

David Hildenbrand (5):
  kernel/resource: make release_mem_region_adjustable() never fail
  kernel/resource: merge_system_ram_resources() to merge resources after
    hotplug
  virtio-mem: try to merge system ram resources
  xen/balloon: try to merge system ram resources
  hv_balloon: try to merge system ram resources

 drivers/hv/hv_balloon.c     |   3 ++
 drivers/virtio/virtio_mem.c |  14 ++++-
 drivers/xen/balloon.c       |   4 ++
 include/linux/ioport.h      |   7 ++-
 kernel/resource.c           | 101 ++++++++++++++++++++++++++++--------
 mm/memory_hotplug.c         |  22 +-------
 6 files changed, 105 insertions(+), 46 deletions(-)

-- 
2.26.2



             reply	other threads:[~2020-08-21 10:35 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-21 10:34 David Hildenbrand [this message]
2020-08-21 10:34 ` [PATCH v1 1/5] kernel/resource: make release_mem_region_adjustable() never fail David Hildenbrand
2020-08-21 10:34 ` [PATCH v1 2/5] kernel/resource: merge_system_ram_resources() to merge resources after hotplug David Hildenbrand
2020-08-31  9:35   ` Pankaj Gupta
2020-09-08 10:26     ` David Hildenbrand
2020-08-21 10:34 ` [PATCH v1 3/5] virtio-mem: try to merge system ram resources David Hildenbrand
2020-08-21 10:34 ` [PATCH v1 4/5] xen/balloon: " David Hildenbrand
2020-09-02 10:15   ` Jürgen Groß
2020-09-02 10:34     ` David Hildenbrand
2020-08-21 10:34 ` [PATCH v1 5/5] hv_balloon: " David Hildenbrand
2020-09-04 17:30   ` Wei Liu
2020-09-08 10:26     ` David Hildenbrand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200821103431.13481-1-david@redhat.com \
    --to=david@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=ardb@kernel.org \
    --cc=bhe@redhat.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=dan.j.williams@intel.com \
    --cc=haiyangz@microsoft.com \
    --cc=jasowang@redhat.com \
    --cc=jgg@ziepe.ca \
    --cc=jgross@suse.com \
    --cc=julien@xen.org \
    --cc=keescook@chromium.org \
    --cc=kys@microsoft.com \
    --cc=linux-hyperv@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.com \
    --cc=mst@redhat.com \
    --cc=pankaj.gupta.linux@gmail.com \
    --cc=richardw.yang@linux.intel.com \
    --cc=roger.pau@citrix.com \
    --cc=sstabellini@kernel.org \
    --cc=sthemmin@microsoft.com \
    --cc=tglx@linutronix.de \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=wei.liu@kernel.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).