From: Muchun Song <songmuchun@bytedance.com>
To: mike.kravetz@oracle.com, akpm@linux-foundation.org
Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Muchun Song <songmuchun@bytedance.com>
Subject: [PATCH v2 4/6] mm: hugetlb: add return -EAGAIN for dissolve_free_huge_page
Date: Wed, 6 Jan 2021 16:47:37 +0800 [thread overview]
Message-ID: <20210106084739.63318-5-songmuchun@bytedance.com> (raw)
In-Reply-To: <20210106084739.63318-1-songmuchun@bytedance.com>
When dissolve_free_huge_page() races with __free_huge_page(), we can
do a retry. Because the race window is small.
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
---
mm/hugetlb.c | 26 +++++++++++++++++++++-----
1 file changed, 21 insertions(+), 5 deletions(-)
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 8ff138c17129..bf02e81e3953 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1775,10 +1775,11 @@ static int free_pool_huge_page(struct hstate *h, nodemask_t *nodes_allowed,
* nothing for in-use hugepages and non-hugepages.
* This function returns values like below:
*
- * -EBUSY: failed to dissolved free hugepages or the hugepage is in-use
- * (allocated or reserved.)
- * 0: successfully dissolved free hugepages or the page is not a
- * hugepage (considered as already dissolved)
+ * -EAGAIN: race with __free_huge_page() and can do a retry
+ * -EBUSY: failed to dissolved free hugepages or the hugepage is in-use
+ * (allocated or reserved.)
+ * 0: successfully dissolved free hugepages or the page is not a
+ * hugepage (considered as already dissolved)
*/
int dissolve_free_huge_page(struct page *page)
{
@@ -1805,8 +1806,10 @@ int dissolve_free_huge_page(struct page *page)
* We should make sure that the page is already on the free list
* when it is dissolved.
*/
- if (unlikely(!PageHugeFreed(head)))
+ if (unlikely(!PageHugeFreed(head))) {
+ rc = -EAGAIN;
goto out;
+ }
/*
* Move PageHWPoison flag from head page to the raw error page,
@@ -1825,6 +1828,14 @@ int dissolve_free_huge_page(struct page *page)
}
out:
spin_unlock(&hugetlb_lock);
+
+ /*
+ * If the freeing of the HugeTLB page is put on a work queue, we should
+ * flush the work before retrying.
+ */
+ if (unlikely(rc == -EAGAIN))
+ flush_work(&free_hpage_work);
+
return rc;
}
@@ -1847,7 +1858,12 @@ int dissolve_free_huge_pages(unsigned long start_pfn, unsigned long end_pfn)
for (pfn = start_pfn; pfn < end_pfn; pfn += 1 << minimum_order) {
page = pfn_to_page(pfn);
+retry:
rc = dissolve_free_huge_page(page);
+ if (rc == -EAGAIN) {
+ cpu_relax();
+ goto retry;
+ }
if (rc)
break;
}
--
2.11.0
next prev parent reply other threads:[~2021-01-06 8:50 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-06 8:47 [PATCH v2 0/6] Fix some bugs about HugeTLB code Muchun Song
2021-01-06 8:47 ` [PATCH v2 1/6] mm: migrate: do not migrate HugeTLB page whose refcount is one Muchun Song
2021-01-06 16:13 ` Michal Hocko
2021-01-07 2:52 ` [External] " Muchun Song
2021-01-07 11:16 ` Michal Hocko
2021-01-07 11:24 ` Muchun Song
2021-01-07 11:48 ` Michal Hocko
2021-01-06 8:47 ` [PATCH v2 2/6] mm: hugetlbfs: fix cannot migrate the fallocated HugeTLB page Muchun Song
2021-01-06 16:35 ` Michal Hocko
2021-01-06 19:30 ` Mike Kravetz
2021-01-06 20:02 ` Michal Hocko
2021-01-06 21:07 ` Mike Kravetz
2021-01-07 8:36 ` Michal Hocko
2021-01-07 2:58 ` [External] " Muchun Song
2021-01-06 8:47 ` [PATCH v2 3/6] mm: hugetlb: fix a race between freeing and dissolving the page Muchun Song
2021-01-06 16:56 ` Michal Hocko
2021-01-06 20:58 ` Mike Kravetz
2021-01-07 3:08 ` [External] " Muchun Song
2021-01-07 8:40 ` Michal Hocko
2021-01-08 0:52 ` Mike Kravetz
2021-01-08 8:28 ` Michal Hocko
2021-01-07 5:39 ` [External] " Muchun Song
2021-01-07 8:41 ` Michal Hocko
2021-01-07 8:53 ` Muchun Song
2021-01-07 11:18 ` Michal Hocko
2021-01-07 11:38 ` Muchun Song
2021-01-07 12:38 ` Michal Hocko
2021-01-07 12:59 ` Muchun Song
2021-01-07 14:11 ` Michal Hocko
2021-01-07 15:11 ` Muchun Song
2021-01-08 1:06 ` Mike Kravetz
2021-01-08 2:38 ` Muchun Song
2021-01-08 8:43 ` Michal Hocko
2021-01-08 9:01 ` Muchun Song
2021-01-08 9:31 ` Michal Hocko
2021-01-08 10:08 ` Muchun Song
2021-01-08 11:44 ` Michal Hocko
2021-01-08 11:52 ` Muchun Song
2021-01-08 12:04 ` Michal Hocko
2021-01-08 12:24 ` Muchun Song
2021-01-06 8:47 ` Muchun Song [this message]
2021-01-06 17:07 ` [PATCH v2 4/6] mm: hugetlb: add return -EAGAIN for dissolve_free_huge_page Michal Hocko
2021-01-07 3:11 ` [External] " Muchun Song
2021-01-07 8:39 ` Michal Hocko
2021-01-07 9:01 ` Muchun Song
2021-01-07 11:22 ` Michal Hocko
2021-01-06 8:47 ` [PATCH v2 5/6] mm: hugetlb: fix a race between isolating and freeing page Muchun Song
2021-01-06 17:10 ` Michal Hocko
2021-01-06 8:47 ` [PATCH v2 6/6] mm: hugetlb: remove VM_BUG_ON_PAGE from page_huge_active Muchun Song
2021-01-06 17:16 ` Michal Hocko
2021-01-08 22:24 ` Mike Kravetz
2021-01-09 4:07 ` [External] " Muchun Song
2021-01-07 9:30 ` [PATCH v2 0/6] Fix some bugs about HugeTLB code David Hildenbrand
2021-01-07 9:40 ` [External] " Muchun Song
2021-01-07 10:10 ` David Hildenbrand
2021-01-07 10:16 ` Muchun Song
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210106084739.63318-5-songmuchun@bytedance.com \
--to=songmuchun@bytedance.com \
--cc=ak@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.cz \
--cc=mike.kravetz@oracle.com \
--cc=n-horiguchi@ah.jp.nec.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).