All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vlastimil Babka <vbabka@suse.cz>
To: Michal Hocko <mhocko@kernel.org>, linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>,
	Xishi Qiu <qiuxishi@huawei.com>,
	zhong jiang <zhongjiang@huawei.com>,
	Joonsoo Kim <js1304@gmail.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Michal Hocko <mhocko@suse.com>
Subject: Re: [PATCH 2/4] hugetlb, memory_hotplug: prefer to use reserved pages for migration
Date: Thu, 8 Jun 2017 10:22:32 +0200	[thread overview]
Message-ID: <faef20f5-80b4-fcb0-6460-ddae9856f35e@suse.cz> (raw)
In-Reply-To: <20170608074553.22152-3-mhocko@kernel.org>

On 06/08/2017 09:45 AM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
> 
> new_node_page will try to use the origin's next NUMA node as the
> migration destination for hugetlb pages. If such a node doesn't have any
> preallocated pool it falls back to __alloc_buddy_huge_page_no_mpol to
> allocate a surplus page instead. This is quite subotpimal for any
> configuration when hugetlb pages are no distributed to all NUMA nodes
> evenly. Say we have a hotplugable node 4 and spare hugetlb pages are
> node 0
> /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages:10000
> /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages:0
> /sys/devices/system/node/node2/hugepages/hugepages-2048kB/nr_hugepages:0
> /sys/devices/system/node/node3/hugepages/hugepages-2048kB/nr_hugepages:0
> /sys/devices/system/node/node4/hugepages/hugepages-2048kB/nr_hugepages:10000
> /sys/devices/system/node/node5/hugepages/hugepages-2048kB/nr_hugepages:0
> /sys/devices/system/node/node6/hugepages/hugepages-2048kB/nr_hugepages:0
> /sys/devices/system/node/node7/hugepages/hugepages-2048kB/nr_hugepages:0
> 
> Now we consume the whole pool on node 4 and try to offline this
> node. All the allocated pages should be moved to node0 which has enough
> preallocated pages to hold them. With the current implementation
> offlining very likely fails because hugetlb allocations during runtime
> are much less reliable.
> 
> Fix this by reusing the nodemask which excludes migration source and try
> to find a first node which has a page in the preallocated pool first and
> fall back to __alloc_buddy_huge_page_no_mpol only when the whole pool is
> consumed.
> 
> Signed-off-by: Michal Hocko <mhocko@suse.com>

Acked-by: Vlastimil Babka <vbabka@suse.cz>

WARNING: multiple messages have this Message-ID (diff)
From: Vlastimil Babka <vbabka@suse.cz>
To: Michal Hocko <mhocko@kernel.org>, linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>,
	Xishi Qiu <qiuxishi@huawei.com>,
	zhong jiang <zhongjiang@huawei.com>,
	Joonsoo Kim <js1304@gmail.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Michal Hocko <mhocko@suse.com>
Subject: Re: [PATCH 2/4] hugetlb, memory_hotplug: prefer to use reserved pages for migration
Date: Thu, 8 Jun 2017 10:22:32 +0200	[thread overview]
Message-ID: <faef20f5-80b4-fcb0-6460-ddae9856f35e@suse.cz> (raw)
In-Reply-To: <20170608074553.22152-3-mhocko@kernel.org>

On 06/08/2017 09:45 AM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
> 
> new_node_page will try to use the origin's next NUMA node as the
> migration destination for hugetlb pages. If such a node doesn't have any
> preallocated pool it falls back to __alloc_buddy_huge_page_no_mpol to
> allocate a surplus page instead. This is quite subotpimal for any
> configuration when hugetlb pages are no distributed to all NUMA nodes
> evenly. Say we have a hotplugable node 4 and spare hugetlb pages are
> node 0
> /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages:10000
> /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages:0
> /sys/devices/system/node/node2/hugepages/hugepages-2048kB/nr_hugepages:0
> /sys/devices/system/node/node3/hugepages/hugepages-2048kB/nr_hugepages:0
> /sys/devices/system/node/node4/hugepages/hugepages-2048kB/nr_hugepages:10000
> /sys/devices/system/node/node5/hugepages/hugepages-2048kB/nr_hugepages:0
> /sys/devices/system/node/node6/hugepages/hugepages-2048kB/nr_hugepages:0
> /sys/devices/system/node/node7/hugepages/hugepages-2048kB/nr_hugepages:0
> 
> Now we consume the whole pool on node 4 and try to offline this
> node. All the allocated pages should be moved to node0 which has enough
> preallocated pages to hold them. With the current implementation
> offlining very likely fails because hugetlb allocations during runtime
> are much less reliable.
> 
> Fix this by reusing the nodemask which excludes migration source and try
> to find a first node which has a page in the preallocated pool first and
> fall back to __alloc_buddy_huge_page_no_mpol only when the whole pool is
> consumed.
> 
> Signed-off-by: Michal Hocko <mhocko@suse.com>

Acked-by: Vlastimil Babka <vbabka@suse.cz>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2017-06-08  8:22 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-06-08  7:45 [PATCH 0/4] more sensible hugetlb migration for hotplug/CMA Michal Hocko
2017-06-08  7:45 ` Michal Hocko
2017-06-08  7:45 ` [PATCH 1/4] mm, memory_hotplug: simplify empty node mask handling in new_node_page Michal Hocko
2017-06-08  7:45   ` Michal Hocko
2017-06-08  8:15   ` Vlastimil Babka
2017-06-08  8:15     ` Vlastimil Babka
2017-06-08  7:45 ` [PATCH 2/4] hugetlb, memory_hotplug: prefer to use reserved pages for migration Michal Hocko
2017-06-08  7:45   ` Michal Hocko
2017-06-08  8:22   ` Vlastimil Babka [this message]
2017-06-08  8:22     ` Vlastimil Babka
2017-06-08  7:45 ` [PATCH 3/4] mm: unify new_node_page and alloc_migrate_target Michal Hocko
2017-06-08  7:45   ` Michal Hocko
2017-06-08  8:36   ` Vlastimil Babka
2017-06-08  8:36     ` Vlastimil Babka
2017-06-08  8:40     ` Michal Hocko
2017-06-08  8:40       ` Michal Hocko
2017-06-08  7:45 ` [RFC PATCH 4/4] hugetlb: add support for preferred node to alloc_huge_page_nodemask Michal Hocko
2017-06-08  7:45   ` Michal Hocko
2017-06-08  8:38   ` Vlastimil Babka
2017-06-08  8:38     ` Vlastimil Babka
2017-06-12  9:06     ` Michal Hocko
2017-06-12  9:06       ` Michal Hocko
2017-06-12 11:48       ` Michal Hocko
2017-06-12 11:48         ` Michal Hocko
2017-06-12 11:53       ` Vlastimil Babka
2017-06-12 11:53         ` Vlastimil Babka
2017-06-12 12:20         ` Michal Hocko
2017-06-12 12:20           ` Michal Hocko
2017-06-12 15:21   ` Michal Hocko
2017-06-12 15:21     ` Michal Hocko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=faef20f5-80b4-fcb0-6460-ddae9856f35e@suse.cz \
    --to=vbabka@suse.cz \
    --cc=akpm@linux-foundation.org \
    --cc=js1304@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@kernel.org \
    --cc=mhocko@suse.com \
    --cc=n-horiguchi@ah.jp.nec.com \
    --cc=qiuxishi@huawei.com \
    --cc=zhongjiang@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.