From: akpm@linux-foundation.org
To: mm-commits@vger.kernel.org, ziy@nvidia.com,
songmuchun@bytedance.com, rientjes@google.com, osalvador@suse.de,
naoya.horiguchi@linux.dev, mhocko@suse.com, david@redhat.com,
mike.kravetz@oracle.com
Subject: + hugetlb-add-hpagecma-flag-and-code-to-free-non-gigantic-pages-in-cma.patch added to -mm tree
Date: Mon, 16 Aug 2021 16:29:21 -0700 [thread overview]
Message-ID: <20210816232921.bH9kQ%akpm@linux-foundation.org> (raw)
The patch titled
Subject: hugetlb: add HPageCma flag and code to free non-gigantic pages in CMA
has been added to the -mm tree. Its filename is
hugetlb-add-hpagecma-flag-and-code-to-free-non-gigantic-pages-in-cma.patch
This patch should soon appear at
https://ozlabs.org/~akpm/mmots/broken-out/hugetlb-add-hpagecma-flag-and-code-to-free-non-gigantic-pages-in-cma.patch
and later at
https://ozlabs.org/~akpm/mmotm/broken-out/hugetlb-add-hpagecma-flag-and-code-to-free-non-gigantic-pages-in-cma.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Mike Kravetz <mike.kravetz@oracle.com>
Subject: hugetlb: add HPageCma flag and code to free non-gigantic pages in CMA
When huge page demotion is fully implemented, gigantic pages can be
demoted to a smaller huge page size. For example, on x86 a 1G page can be
demoted to 512 2M pages. However, gigantic pages can potentially be
allocated from CMA. If a gigantic page which was allocated from CMA is
demoted, the corresponding demoted pages needs to be returned to CMA.
In order to track hugetlb pages that need to be returned to CMA, add the
hugetlb specific flag HPageCma. Flag is set when a huge page is allocated
from CMA and transferred to any demoted pages. Non-gigantic huge page
freeing code checks for the flag and takes appropriate action.
This also requires a change to CMA reservations for gigantic pages.
Currently, the 'order_per_bit' is set to the gigantic page size. However,
if gigantic pages can be demoted this needs to be set to the order of the
smallest huge page. At CMA reservation time we do not know the size of
the smallest huge page size, so use HUGETLB_PAGE_ORDER. Also, prohibit
demotion to huge page sizes smaller than HUGETLB_PAGE_ORDER.
Link: https://lkml.kernel.org/r/20210816224953.157796-3-mike.kravetz@oracle.com
Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Muchun Song <songmuchun@bytedance.com>
Cc: Naoya Horiguchi <naoya.horiguchi@linux.dev>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
include/linux/hugetlb.h | 7 +++++
mm/hugetlb.c | 46 ++++++++++++++++++++++++++++----------
2 files changed, 41 insertions(+), 12 deletions(-)
--- a/include/linux/hugetlb.h~hugetlb-add-hpagecma-flag-and-code-to-free-non-gigantic-pages-in-cma
+++ a/include/linux/hugetlb.h
@@ -533,6 +533,11 @@ unsigned long hugetlb_get_unmapped_area(
* HPG_freed - Set when page is on the free lists.
* Synchronization: hugetlb_lock held for examination and modification.
* HPG_vmemmap_optimized - Set when the vmemmap pages of the page are freed.
+ * HPG_cma - Set if huge page was directly allocated from CMA area via
+ * cma_alloc. Initially set for gigantic page cma allocations, but can
+ * be set in non-gigantic pages if gigantic pages are demoted.
+ * Synchronization: Only accessed or modified when there is only one
+ * reference to the page at allocation, free or demote time.
*/
enum hugetlb_page_flags {
HPG_restore_reserve = 0,
@@ -540,6 +545,7 @@ enum hugetlb_page_flags {
HPG_temporary,
HPG_freed,
HPG_vmemmap_optimized,
+ HPG_cma,
__NR_HPAGEFLAGS,
};
@@ -586,6 +592,7 @@ HPAGEFLAG(Migratable, migratable)
HPAGEFLAG(Temporary, temporary)
HPAGEFLAG(Freed, freed)
HPAGEFLAG(VmemmapOptimized, vmemmap_optimized)
+HPAGEFLAG(Cma, cma)
#ifdef CONFIG_HUGETLB_PAGE
--- a/mm/hugetlb.c~hugetlb-add-hpagecma-flag-and-code-to-free-non-gigantic-pages-in-cma
+++ a/mm/hugetlb.c
@@ -1262,6 +1262,7 @@ static void destroy_compound_gigantic_pa
atomic_set(compound_pincount_ptr(page), 0);
for (i = 1; i < nr_pages; i++, p = mem_map_next(p, page, i)) {
+ p->mapping = NULL;
clear_compound_head(p);
set_page_refcounted(p);
}
@@ -1273,16 +1274,12 @@ static void destroy_compound_gigantic_pa
static void free_gigantic_page(struct page *page, unsigned int order)
{
- /*
- * If the page isn't allocated using the cma allocator,
- * cma_release() returns false.
- */
#ifdef CONFIG_CMA
- if (cma_release(hugetlb_cma[page_to_nid(page)], page, 1 << order))
- return;
+ if (HPageCma(page))
+ cma_release(hugetlb_cma[page_to_nid(page)], page, 1 << order);
+ else
#endif
-
- free_contig_range(page_to_pfn(page), 1 << order);
+ free_contig_range(page_to_pfn(page), 1 << order);
}
#ifdef CONFIG_CONTIG_ALLOC
@@ -1301,8 +1298,10 @@ static struct page *alloc_gigantic_page(
if (hugetlb_cma[nid]) {
page = cma_alloc(hugetlb_cma[nid], nr_pages,
huge_page_order(h), true);
- if (page)
+ if (page) {
+ SetHPageCma(page);
return page;
+ }
}
if (!(gfp_mask & __GFP_THISNODE)) {
@@ -1312,8 +1311,10 @@ static struct page *alloc_gigantic_page(
page = cma_alloc(hugetlb_cma[node], nr_pages,
huge_page_order(h), true);
- if (page)
+ if (page) {
+ SetHPageCma(page);
return page;
+ }
}
}
}
@@ -1470,6 +1471,20 @@ static void __update_and_free_page(struc
destroy_compound_gigantic_page(page, huge_page_order(h));
free_gigantic_page(page, huge_page_order(h));
} else {
+#ifdef CONFIG_CMA
+ /*
+ * Could be a page that was demoted from a gigantic page
+ * which was allocated in a CMA area.
+ */
+ if (HPageCma(page)) {
+ destroy_compound_gigantic_page(page,
+ huge_page_order(h));
+ if (!cma_release(hugetlb_cma[page_to_nid(page)], page,
+ 1 << huge_page_order(h)))
+ VM_BUG_ON_PAGE(1, page);
+ return;
+ }
+#endif
__free_pages(page, huge_page_order(h));
}
}
@@ -2983,7 +2998,8 @@ static void __init hugetlb_init_hstates(
for_each_hstate(h2) {
if (h2 == h)
continue;
- if (h2->order < h->order && h2->order > h->demote_order)
+ if (h2->order >= HUGETLB_PAGE_ORDER &&
+ h2->order < h->order && h2->order > h->demote_order)
h->demote_order = h2->order;
}
}
@@ -6498,7 +6514,13 @@ void __init hugetlb_cma_reserve(int orde
size = round_up(size, PAGE_SIZE << order);
snprintf(name, sizeof(name), "hugetlb%d", nid);
- res = cma_declare_contiguous_nid(0, size, 0, PAGE_SIZE << order,
+ /*
+ * Note that 'order per bit' is based on smallest size that
+ * may be returned to CMA allocator in the case of
+ * huge page demotion.
+ */
+ res = cma_declare_contiguous_nid(0, size, 0,
+ PAGE_SIZE << HUGETLB_PAGE_ORDER,
0, false, name,
&hugetlb_cma[nid], nid);
if (res) {
_
Patches currently in -mm which might be from mike.kravetz@oracle.com are
hugetlb-simplify-prep_compound_gigantic_page-ref-count-racing-code.patch
hugetlb-drop-ref-count-earlier-after-page-allocation.patch
hugetlb-before-freeing-hugetlb-page-set-dtor-to-appropriate-value.patch
hugetlb-add-demote-hugetlb-page-sysfs-interfaces.patch
hugetlb-add-hpagecma-flag-and-code-to-free-non-gigantic-pages-in-cma.patch
hugetlb-add-demote-bool-to-gigantic-page-routines.patch
hugetlb-add-hugetlb-demote-page-support.patch
hugetlb-document-the-demote-sysfs-interfaces.patch
hugetlb-vmemmap-optimizations-when-demoting-hugetlb-pages.patch
hugetlb-prepare-destroy-and-prep-routines-for-vmemmap-optimized-pages.patch
hugetlb-optimized-demote-vmemmap-optimizatized-pages.patch
next reply other threads:[~2021-08-16 23:29 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-16 23:29 akpm [this message]
2021-09-23 21:25 + hugetlb-add-hpagecma-flag-and-code-to-free-non-gigantic-pages-in-cma.patch added to -mm tree akpm
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210816232921.bH9kQ%akpm@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=david@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mhocko@suse.com \
--cc=mike.kravetz@oracle.com \
--cc=mm-commits@vger.kernel.org \
--cc=naoya.horiguchi@linux.dev \
--cc=osalvador@suse.de \
--cc=rientjes@google.com \
--cc=songmuchun@bytedance.com \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).