From: Mike Kravetz <mike.kravetz@oracle.com>
To: linux-mm@kvack.org, linux-kernel@vger.kernel.org
Cc: Michal Hocko <mhocko@suse.com>,
Shakeel Butt <shakeelb@google.com>,
Oscar Salvador <osalvador@suse.de>,
David Hildenbrand <david@redhat.com>,
Muchun Song <songmuchun@bytedance.com>,
David Rientjes <rientjes@google.com>,
Miaohe Lin <linmiaohe@huawei.com>,
Peter Zijlstra <peterz@infradead.org>,
Matthew Wilcox <willy@infradead.org>,
HORIGUCHI NAOYA <naoya.horiguchi@nec.com>,
"Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>,
Waiman Long <longman@redhat.com>, Peter Xu <peterx@redhat.com>,
Mina Almasry <almasrymina@google.com>,
Andrew Morton <akpm@linux-foundation.org>,
Mike Kravetz <mike.kravetz@oracle.com>
Subject: [RFC PATCH 8/8] hugetlb: track hugetlb pages allocated via cma_alloc
Date: Fri, 19 Mar 2021 15:42:09 -0700 [thread overview]
Message-ID: <20210319224209.150047-9-mike.kravetz@oracle.com> (raw)
In-Reply-To: <20210319224209.150047-1-mike.kravetz@oracle.com>
When a hugetlb page allocated via cma_alloc is freed via cma_release
the cma_release call may sleep. For now, only gigantic pages can be
allocated via cma_alloc. The routine free_huge_page can not sleep,
so it defers freeing of all gigantic pages to a workqueue.
Use a new hugetlb page specific flag HPageCma to indicate the page was
allocated via cma_alloc. This flag can be used so that only gigantic
pages allocated via cma_alloc will have deferred freeing.
Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com>
---
include/linux/hugetlb.h | 7 +++++++
mm/hugetlb.c | 18 ++++++++++--------
2 files changed, 17 insertions(+), 8 deletions(-)
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index a81ca39c06be..0aba6957a73a 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -498,12 +498,18 @@ unsigned long hugetlb_get_unmapped_area(struct file *file, unsigned long addr,
* modifications require hugetlb_lock.
* HPG_freed - Set when page is on the free lists.
* Synchronization: hugetlb_lock held for examination and modification.
+ * HPG_cma - Set if huge page was directly allocated from CMA area via
+ * cma_alloc. Initially set for gigantic page cma allocations, but can
+ * be set in non-gigantic pages if gigantic pages are demoted.
+ * Synchronization: Only accessed or modified when there is only one
+ * reference to the page at allocation or free time.
*/
enum hugetlb_page_flags {
HPG_restore_reserve = 0,
HPG_migratable,
HPG_temporary,
HPG_freed,
+ HPG_cma,
__NR_HPAGEFLAGS,
};
@@ -549,6 +555,7 @@ HPAGEFLAG(RestoreReserve, restore_reserve)
HPAGEFLAG(Migratable, migratable)
HPAGEFLAG(Temporary, temporary)
HPAGEFLAG(Freed, freed)
+HPAGEFLAG(Cma, cma)
#ifdef CONFIG_HUGETLB_PAGE
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index b8304b290a73..5efff5ce337f 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1241,13 +1241,11 @@ static void destroy_compound_gigantic_page(struct page *page,
static void free_gigantic_page(struct page *page, unsigned int order)
{
- /*
- * If the page isn't allocated using the cma allocator,
- * cma_release() returns false.
- */
#ifdef CONFIG_CMA
- if (cma_release(hugetlb_cma[page_to_nid(page)], page, 1 << order))
+ if (HPageCma(page)) {
+ cma_release(hugetlb_cma[page_to_nid(page)], page, 1 << order);
return;
+ }
#endif
free_contig_range(page_to_pfn(page), 1 << order);
@@ -1269,8 +1267,10 @@ static struct page *alloc_gigantic_page(struct hstate *h, gfp_t gfp_mask,
if (hugetlb_cma[nid]) {
page = cma_alloc(hugetlb_cma[nid], nr_pages,
huge_page_order(h), true);
- if (page)
+ if (page) {
+ SetHPageCma(page);
return page;
+ }
}
if (!(gfp_mask & __GFP_THISNODE)) {
@@ -1280,8 +1280,10 @@ static struct page *alloc_gigantic_page(struct hstate *h, gfp_t gfp_mask,
page = cma_alloc(hugetlb_cma[node], nr_pages,
huge_page_order(h), true);
- if (page)
+ if (page) {
+ SetHPageCma(page);
return page;
+ }
}
}
}
@@ -1397,7 +1399,7 @@ static DECLARE_WORK(free_hpage_work, free_hpage_workfn);
static bool free_page_may_sleep(struct hstate *h, struct page *page)
{
/* freeing gigantic pages in CMA may sleep */
- if (hstate_is_gigantic(h))
+ if (HPageCma(page))
return true;
return false;
--
2.30.2
prev parent reply other threads:[~2021-03-19 22:43 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-19 22:42 [RFC PATCH 0/8] make hugetlb put_page safe for all calling contexts Mike Kravetz
2021-03-19 22:42 ` [RFC PATCH 1/8] hugetlb: add per-hstate mutex to synchronize user adjustments Mike Kravetz
2021-03-22 13:59 ` Michal Hocko
2021-03-22 16:57 ` Mike Kravetz
2021-03-23 7:48 ` Michal Hocko
2021-03-19 22:42 ` [RFC PATCH 2/8] hugetlb: recompute min_count when dropping hugetlb_lock Mike Kravetz
2021-03-22 14:07 ` Michal Hocko
2021-03-22 23:07 ` Mike Kravetz
2021-03-23 7:50 ` Michal Hocko
2021-03-23 8:01 ` Peter Zijlstra
2021-03-23 8:14 ` Michal Hocko
2021-03-23 23:18 ` Mike Kravetz
2021-03-24 8:36 ` Michal Hocko
2021-03-24 16:43 ` Mike Kravetz
2021-03-19 22:42 ` [RFC PATCH 3/8] hugetlb: create remove_hugetlb_page() to separate functionality Mike Kravetz
2021-03-22 14:15 ` Michal Hocko
2021-03-22 17:01 ` Mike Kravetz
2021-03-19 22:42 ` [RFC PATCH 4/8] hugetlb: call update_and_free_page without hugetlb_lock Mike Kravetz
2021-03-22 14:19 ` Michal Hocko
2021-03-19 22:42 ` [RFC PATCH 5/8] hugetlb: change free_pool_huge_page to remove_pool_huge_page Mike Kravetz
2021-03-22 14:31 ` Michal Hocko
2021-03-22 23:28 ` Mike Kravetz
2021-03-23 7:57 ` Michal Hocko
2021-03-24 1:03 ` Mike Kravetz
2021-03-24 8:40 ` Michal Hocko
2021-03-24 16:38 ` Mike Kravetz
2021-03-24 16:50 ` Michal Hocko
2021-03-19 22:42 ` [RFC PATCH 6/8] hugetlb: make free_huge_page irq safe Mike Kravetz
2021-03-21 19:55 ` Mike Kravetz
2021-03-22 13:36 ` [hugetlb] cd190f60f9: BUG:sleeping_function_called_from_invalid_context_at_mm/hugetlb.c kernel test robot
2021-03-22 14:35 ` [RFC PATCH 6/8] hugetlb: make free_huge_page irq safe Michal Hocko
2021-03-19 22:42 ` [RFC PATCH 7/8] hugetlb: add update_and_free_page_no_sleep for irq context Mike Kravetz
2021-03-20 1:18 ` Hillf Danton
2021-03-25 0:26 ` Mike Kravetz
2021-03-22 8:41 ` Peter Zijlstra
2021-03-22 17:42 ` Mike Kravetz
2021-03-22 18:10 ` Roman Gushchin
2021-03-23 18:51 ` Mike Kravetz
2021-03-23 19:07 ` Roman Gushchin
2021-03-24 8:43 ` Michal Hocko
2021-03-24 16:53 ` Mike Kravetz
2021-03-22 20:43 ` Peter Zijlstra
2021-03-22 14:42 ` Michal Hocko
2021-03-22 14:46 ` Michal Hocko
2021-03-19 22:42 ` Mike Kravetz [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210319224209.150047-9-mike.kravetz@oracle.com \
--to=mike.kravetz@oracle.com \
--cc=akpm@linux-foundation.org \
--cc=almasrymina@google.com \
--cc=aneesh.kumar@linux.ibm.com \
--cc=david@redhat.com \
--cc=linmiaohe@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=longman@redhat.com \
--cc=mhocko@suse.com \
--cc=naoya.horiguchi@nec.com \
--cc=osalvador@suse.de \
--cc=peterx@redhat.com \
--cc=peterz@infradead.org \
--cc=rientjes@google.com \
--cc=shakeelb@google.com \
--cc=songmuchun@bytedance.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).