From: Oscar Salvador <osalvador@suse.de>
To: n-horiguchi@ah.jp.nec.com
Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org,
linux-kernel@vger.kernel.org, Oscar Salvador <osalvador@suse.de>
Subject: [RFC PATCH v2 10/16] mm,hwpoison: Rework soft offline for free pages
Date: Thu, 17 Oct 2019 16:21:17 +0200 [thread overview]
Message-ID: <20191017142123.24245-11-osalvador@suse.de> (raw)
In-Reply-To: <20191017142123.24245-1-osalvador@suse.de>
When trying to soft-offline a free page, we need to first take it off
the buddy allocator.
Once we know is out of reach, we can safely flag it as poisoned.
take_page_off_buddy will be used to take a page meant to be poisoned
off the buddy allocator.
take_page_off_buddy calls break_down_buddy_pages, which splits a
higher-order page in case our page belongs to one.
Once the page is under our control, we call page_set_poison to set it
as poisoned and grab a refcount on it.
Signed-off-by: Oscar Salvador <osalvador@suse.de>
---
mm/memory-failure.c | 20 +++++++++++-----
mm/page_alloc.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 82 insertions(+), 6 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 37b230b8cfe7..1d986580522d 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -78,6 +78,15 @@ EXPORT_SYMBOL_GPL(hwpoison_filter_dev_minor);
EXPORT_SYMBOL_GPL(hwpoison_filter_flags_mask);
EXPORT_SYMBOL_GPL(hwpoison_filter_flags_value);
+extern bool take_page_off_buddy(struct page *page);
+
+static void page_handle_poison(struct page *page)
+{
+ SetPageHWPoison(page);
+ page_ref_inc(page);
+ num_poisoned_pages_inc();
+}
+
static int hwpoison_filter_dev(struct page *p)
{
struct address_space *mapping;
@@ -1830,14 +1839,13 @@ static int soft_offline_in_use_page(struct page *page)
static int soft_offline_free_page(struct page *page)
{
- int rc = dissolve_free_huge_page(page);
+ int rc = -EBUSY;
- if (!rc) {
- if (set_hwpoison_free_buddy_page(page))
- num_poisoned_pages_inc();
- else
- rc = -EBUSY;
+ if (!dissolve_free_huge_page(page) && take_page_off_buddy(page)) {
+ page_handle_poison(page);
+ rc = 0;
}
+
return rc;
}
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index cd1dd0712624..255df0c76a40 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -8632,6 +8632,74 @@ bool is_free_buddy_page(struct page *page)
#ifdef CONFIG_MEMORY_FAILURE
/*
+ * Break down a higher-order page in sub-pages, and keep our target out of
+ * buddy allocator.
+ */
+static void break_down_buddy_pages(struct zone *zone, struct page *page,
+ struct page *target, int low, int high,
+ struct free_area *area, int migratetype)
+{
+ unsigned long size = 1 << high;
+ struct page *current_buddy, *next_page;
+
+ while (high > low) {
+ area--;
+ high--;
+ size >>= 1;
+
+ if (target >= &page[size]) {
+ next_page = page + size;
+ current_buddy = page;
+ } else {
+ next_page = page;
+ current_buddy = page + size;
+ }
+
+ if (set_page_guard(zone, current_buddy, high, migratetype))
+ continue;
+
+ if (current_buddy != target) {
+ add_to_free_area(current_buddy, area, migratetype);
+ set_page_order(current_buddy, high);
+ page = next_page;
+ }
+ }
+}
+
+/*
+ * Take a page that will be marked as poisoned off the buddy allocator.
+ */
+bool take_page_off_buddy(struct page *page)
+ {
+ struct zone *zone = page_zone(page);
+ unsigned long pfn = page_to_pfn(page);
+ unsigned long flags;
+ unsigned int order;
+ bool ret = false;
+
+ spin_lock_irqsave(&zone->lock, flags);
+ for (order = 0; order < MAX_ORDER; order++) {
+ struct page *page_head = page - (pfn & ((1 << order) - 1));
+ int buddy_order = page_order(page_head);
+ struct free_area *area = &(zone->free_area[buddy_order]);
+
+ if (PageBuddy(page_head) && buddy_order >= order) {
+ unsigned long pfn_head = page_to_pfn(page_head);
+ int migratetype = get_pfnblock_migratetype(page_head,
+ pfn_head);
+
+ del_page_from_free_area(page_head, area);
+ break_down_buddy_pages(zone, page_head, page, 0,
+ buddy_order, area, migratetype);
+ ret = true;
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&zone->lock, flags);
+ return ret;
+ }
+
+/*
* Set PG_hwpoison flag if a given page is confirmed to be a free page. This
* test is performed under the zone lock to prevent a race against page
* allocation.
--
2.12.3
next prev parent reply other threads:[~2019-10-17 14:22 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-17 14:21 [RFC PATCH v2 00/16] Hwpoison rework {hard,soft}-offline Oscar Salvador
2019-10-17 14:21 ` [RFC PATCH v2 01/16] mm,hwpoison: cleanup unused PageHuge() check Oscar Salvador
2019-10-18 11:48 ` Michal Hocko
2019-10-21 7:00 ` Naoya Horiguchi
2019-10-21 12:16 ` Michal Hocko
2019-11-12 12:22 ` Aneesh Kumar K.V
2019-11-13 6:02 ` Naoya Horiguchi
2019-10-17 14:21 ` [RFC PATCH v2 02/16] mm,madvise: call soft_offline_page() without MF_COUNT_INCREASED Oscar Salvador
2019-10-18 11:52 ` Michal Hocko
2019-10-21 7:02 ` Naoya Horiguchi
2019-10-21 12:20 ` Michal Hocko
2019-10-17 14:21 ` [RFC PATCH v2 03/16] mm,madvise: Refactor madvise_inject_error Oscar Salvador
2019-10-21 7:03 ` Naoya Horiguchi
2019-10-17 14:21 ` [RFC PATCH v2 04/16] mm,hwpoison-inject: don't pin for hwpoison_filter Oscar Salvador
2019-10-17 14:21 ` [RFC PATCH v2 05/16] mm,hwpoison: Un-export get_hwpoison_page and make it static Oscar Salvador
2019-10-21 7:03 ` Naoya Horiguchi
2019-10-17 14:21 ` [RFC PATCH v2 06/16] mm,hwpoison: Kill put_hwpoison_page Oscar Salvador
2019-10-21 7:04 ` Naoya Horiguchi
2019-10-17 14:21 ` [RFC PATCH v2 07/16] mm,hwpoison: remove MF_COUNT_INCREASED Oscar Salvador
2019-10-17 14:21 ` [RFC PATCH v2 08/16] mm,hwpoison: remove flag argument from soft offline functions Oscar Salvador
2019-10-17 14:21 ` [RFC PATCH v2 09/16] mm,hwpoison: Unify THP handling for hard and soft offline Oscar Salvador
2019-10-21 7:04 ` Naoya Horiguchi
2019-10-21 9:51 ` [PATCH 17/16] mm,hwpoison: introduce MF_MSG_UNSPLIT_THP Naoya Horiguchi
2019-10-22 8:00 ` Oscar Salvador
2019-10-17 14:21 ` Oscar Salvador [this message]
2019-10-18 12:06 ` [RFC PATCH v2 10/16] mm,hwpoison: Rework soft offline for free pages Michal Hocko
2019-10-21 12:58 ` Oscar Salvador
2019-10-21 15:41 ` Michal Hocko
2019-10-22 7:46 ` Oscar Salvador
2019-10-22 8:26 ` Michal Hocko
2019-10-22 8:35 ` Oscar Salvador
2019-10-22 9:22 ` Michal Hocko
2019-10-22 9:58 ` Oscar Salvador
2019-10-22 10:24 ` Michal Hocko
2019-10-22 10:33 ` Oscar Salvador
2019-10-23 2:15 ` Naoya Horiguchi
2019-10-23 2:01 ` Naoya Horiguchi
2019-10-21 7:45 ` Naoya Horiguchi
2019-10-22 8:00 ` Oscar Salvador
2019-10-17 14:21 ` [RFC PATCH v2 11/16] mm,hwpoison: Rework soft offline for in-use pages Oscar Salvador
2019-10-18 12:39 ` Michal Hocko
2019-10-21 13:48 ` Oscar Salvador
2019-10-21 14:06 ` Michal Hocko
2019-10-22 7:56 ` Oscar Salvador
2019-10-22 8:30 ` Michal Hocko
2019-10-22 9:40 ` Oscar Salvador
2019-10-17 14:21 ` [RFC PATCH v2 12/16] mm,hwpoison: Refactor soft_offline_huge_page and __soft_offline_page Oscar Salvador
2019-10-17 14:21 ` [RFC PATCH v2 13/16] mm,hwpoison: Take pages off the buddy when hard-offlining Oscar Salvador
2019-10-17 14:21 ` [RFC PATCH v2 14/16] mm,hwpoison: Return 0 if the page is already poisoned in soft-offline Oscar Salvador
2019-10-21 9:20 ` Naoya Horiguchi
2019-10-17 14:21 ` [RFC PATCH v2 15/16] mm/hwpoison-inject: Rip off duplicated checks Oscar Salvador
2019-10-21 9:40 ` David Hildenbrand
2019-10-22 7:57 ` Oscar Salvador
2019-10-17 14:21 ` [RFC PATCH v2 16/16] mm, soft-offline: convert parameter to pfn Oscar Salvador
2019-10-18 8:15 ` David Hildenbrand
2020-06-11 16:43 ` [RFC PATCH v2 00/16] Hwpoison rework {hard,soft}-offline Dmitry Yakunin
2020-06-15 6:19 ` HORIGUCHI NAOYA(堀口 直也)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191017142123.24245-11-osalvador@suse.de \
--to=osalvador@suse.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=mike.kravetz@oracle.com \
--cc=n-horiguchi@ah.jp.nec.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).