From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C744BC433B4 for ; Wed, 19 May 2021 00:23:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9F514610CC for ; Wed, 19 May 2021 00:23:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237083AbhESAYa (ORCPT ); Tue, 18 May 2021 20:24:30 -0400 Received: from mail.kernel.org ([198.145.29.99]:58796 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233756AbhESAYa (ORCPT ); Tue, 18 May 2021 20:24:30 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 9D38561209; Wed, 19 May 2021 00:23:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1621383790; bh=4MPW1u6SUnlSK20BkIZi9YQG/jv0YeW84AEGRcHx1OA=; h=Date:From:To:Subject:From; b=VqPcs23V1pocUkDvCf5p2hdyW6DunSXA303aEf+eZMg+p/y4o/SbUoIyBS6yeJY8m IaTmGXc1/lVhnlRQfKk/lhCosXuMWDG4lfYY8/7Qkm15mpLy41DXApY8nv7rZCJBBi cDggmWb94Dy3/vlxeOXsE4kAMqI0GIyxs40qTsdU= Date: Tue, 18 May 2021 17:23:10 -0700 From: akpm@linux-foundation.org To: mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, songmuchun@bytedance.com, tony.luck@intel.com Subject: + mmhwpoison-make-get_hwpoison_page-call-get_any_page.patch added to -mm tree Message-ID: <20210519002310.E-yFKVkr7%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 Precedence: bulk Reply-To: linux-kernel@vger.kernel.org List-ID: X-Mailing-List: mm-commits@vger.kernel.org The patch titled Subject: mm,hwpoison: make get_hwpoison_page call get_any_page() has been added to the -mm tree. Its filename is mmhwpoison-make-get_hwpoison_page-call-get_any_page.patch This patch should soon appear at https://ozlabs.org/~akpm/mmots/broken-out/mmhwpoison-make-get_hwpoison_page-call-get_any_page.patch and later at https://ozlabs.org/~akpm/mmotm/broken-out/mmhwpoison-make-get_hwpoison_page-call-get_any_page.patch Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next and is updated there every 3-4 working days ------------------------------------------------------ From: Naoya Horiguchi Subject: mm,hwpoison: make get_hwpoison_page call get_any_page() __get_hwpoison_page() could fail to grab refcount by some race condition, so it's helpful if we can handle it by retrying. We already have retry logic, so make get_hwpoison_page() call get_any_page() when called from memory_failure(). Link: https://lkml.kernel.org/r/20210518231259.2553203-3-nao.horiguchi@gmail.com Signed-off-by: Naoya Horiguchi Cc: Muchun Song Cc: Mike Kravetz Cc: Michal Hocko Cc: Tony Luck Signed-off-by: Andrew Morton --- mm/hugetlb.c | 2 mm/memory-failure.c | 110 ++++++++++++++++++++++-------------------- 2 files changed, 62 insertions(+), 50 deletions(-) --- a/mm/hugetlb.c~mmhwpoison-make-get_hwpoison_page-call-get_any_page +++ a/mm/hugetlb.c @@ -5857,6 +5857,8 @@ int get_hwpoison_huge_page(struct page * *hugetlb = true; if (HPageFreed(page) || HPageMigratable(page)) ret = get_page_unless_zero(page); + else + ret = -EBUSY; } spin_unlock_irq(&hugetlb_lock); return ret; --- a/mm/memory-failure.c~mmhwpoison-make-get_hwpoison_page-call-get_any_page +++ a/mm/memory-failure.c @@ -949,13 +949,6 @@ static int page_action(struct page_state return (result == MF_RECOVERED || result == MF_DELAYED) ? 0 : -EBUSY; } -/** - * __get_hwpoison_page() - Get refcount for memory error handling: - * @page: raw error page (hit by memory error) - * - * Return: return 0 if failed to grab the refcount, otherwise true (some - * non-zero value.) - */ static int __get_hwpoison_page(struct page *page) { struct page *head = compound_head(page); @@ -992,15 +985,6 @@ static int __get_hwpoison_page(struct pa return 0; } -/* - * Safely get reference count of an arbitrary page. - * - * Returns 0 for a free page, 1 for an in-use page, - * -EIO for a page-type we cannot handle and -EBUSY if we raced with an - * allocation. - * We only incremented refcount in case the page was already in-use and it - * is a known type we can handle. - */ static int get_any_page(struct page *p, unsigned long flags) { int ret = 0, pass = 0; @@ -1010,50 +994,76 @@ static int get_any_page(struct page *p, count_increased = true; try_again: - if (!count_increased && !__get_hwpoison_page(p)) { - if (page_count(p)) { - /* We raced with an allocation, retry. */ - if (pass++ < 3) - goto try_again; - ret = -EBUSY; - } else if (!PageHuge(p) && !is_free_buddy_page(p)) { - /* We raced with put_page, retry. */ + if (!count_increased) { + ret = __get_hwpoison_page(p); + if (!ret) { + if (page_count(p)) { + /* We raced with an allocation, retry. */ + if (pass++ < 3) + goto try_again; + ret = -EBUSY; + } else if (!PageHuge(p) && !is_free_buddy_page(p)) { + /* We raced with put_page, retry. */ + if (pass++ < 3) + goto try_again; + ret = -EIO; + } + goto out; + } else if (ret == -EBUSY) { + /* We raced with freeing huge page to buddy, retry. */ if (pass++ < 3) goto try_again; - ret = -EIO; } + } + + if (PageHuge(p) || PageLRU(p) || __PageMovable(p)) { + ret = 1; } else { - if (PageHuge(p) || PageLRU(p) || __PageMovable(p)) { - ret = 1; - } else { - /* - * A page we cannot handle. Check whether we can turn - * it into something we can handle. - */ - if (pass++ < 3) { - put_page(p); - shake_page(p, 1); - count_increased = false; - goto try_again; - } + /* + * A page we cannot handle. Check whether we can turn + * it into something we can handle. + */ + if (pass++ < 3) { put_page(p); - ret = -EIO; + shake_page(p, 1); + count_increased = false; + goto try_again; } + put_page(p); + ret = -EIO; } - +out: return ret; } -static int get_hwpoison_page(struct page *p, unsigned long flags, - enum mf_flags ctxt) +/** + * get_hwpoison_page() - Get refcount for memory error handling + * @p: Raw error page (hit by memory error) + * @flags: Flags controlling behavior of error handling + * + * get_hwpoison_page() takes a page refcount of an error page to handle memory + * error on it, after checking that the error page is in a well-defined state + * (defined as a page-type we can successfully handle the memor error on it, + * such as LRU page and hugetlb page). + * + * Memory error handling could be triggered at any time on any type of page, + * so it's prone to race with typical memory management lifecycle (like + * allocation and free). So to avoid such races, get_hwpoison_page() takes + * extra care for the error page's state (as done in __get_hwpoison_page()), + * and has some retry logic in get_any_page(). + * + * Return: 0 on failure, + * 1 on success for in-use pages in a well-defined state, + * -EIO for pages on which we can not handle memory errors, + * -EBUSY when get_hwpoison_page() has raced with page lifecycle + * operations like allocation and free. + */ +static int get_hwpoison_page(struct page *p, unsigned long flags) { int ret; zone_pcp_disable(page_zone(p)); - if (ctxt == MF_SOFT_OFFLINE) - ret = get_any_page(p, flags); - else - ret = __get_hwpoison_page(p); + ret = get_any_page(p, flags); zone_pcp_enable(page_zone(p)); return ret; @@ -1239,7 +1249,7 @@ static int memory_failure_hugetlb(unsign num_poisoned_pages_inc(); - if (!(flags & MF_COUNT_INCREASED) && !get_hwpoison_page(p, flags, 0)) { + if (!(flags & MF_COUNT_INCREASED) && !get_hwpoison_page(p, flags)) { /* * Check "filter hit" and "race with other subpage." */ @@ -1453,7 +1463,7 @@ try_again: * In fact it's dangerous to directly bump up page count from 0, * that may make page_ref_freeze()/page_ref_unfreeze() mismatch. */ - if (!(flags & MF_COUNT_INCREASED) && !get_hwpoison_page(p, flags, 0)) { + if (!(flags & MF_COUNT_INCREASED) && !get_hwpoison_page(p, flags)) { if (is_free_buddy_page(p)) { if (take_page_off_buddy(p)) { page_ref_inc(p); @@ -1741,7 +1751,7 @@ int unpoison_memory(unsigned long pfn) return 0; } - if (!get_hwpoison_page(p, flags, 0)) { + if (!get_hwpoison_page(p, flags)) { if (TestClearPageHWPoison(p)) num_poisoned_pages_dec(); unpoison_pr_info("Unpoison: Software-unpoisoned free page %#lx\n", @@ -1957,7 +1967,7 @@ int soft_offline_page(unsigned long pfn, retry: get_online_mems(); - ret = get_hwpoison_page(page, flags, MF_SOFT_OFFLINE); + ret = get_hwpoison_page(page, flags); put_online_mems(); if (ret > 0) { _ Patches currently in -mm which might be from naoya.horiguchi@nec.com are mmhwpoison-fix-race-with-hugetlb-page-allocation.patch mmhwpoison-make-get_hwpoison_page-call-get_any_page.patch