From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A493C47082 for ; Thu, 3 Jun 2021 23:36:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1AE0061008 for ; Thu, 3 Jun 2021 23:36:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1AE0061008 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B26E26B0073; Thu, 3 Jun 2021 19:36:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AFE266B0074; Thu, 3 Jun 2021 19:36:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99EDB6B0075; Thu, 3 Jun 2021 19:36:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0048.hostedemail.com [216.40.44.48]) by kanga.kvack.org (Postfix) with ESMTP id 69D8F6B0073 for ; Thu, 3 Jun 2021 19:36:55 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 0008B8249980 for ; Thu, 3 Jun 2021 23:36:54 +0000 (UTC) X-FDA: 78214025190.27.B359FB2 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf26.hostedemail.com (Postfix) with ESMTP id D55CF4202A29 for ; Thu, 3 Jun 2021 23:36:44 +0000 (UTC) Received: by mail-pg1-f175.google.com with SMTP id t8so6346893pgb.9 for ; Thu, 03 Jun 2021 16:36:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0QZP/2zPNWkHUV5d1g3n6GdeLfOnqJ4ULZ1taVr6o9I=; b=KrycwZLVn+kAjG83eM/L4ROh/Va5YK4B46NIa7USCRnrh4BUcLgYnRB8zirUQSan1x 6csfxI4qne5CbZFVb4MJchVk1GsgOZhiVENCy1NFEHnFDwLjWFW1cJWH+GE/RXInDTPZ QI0mSaXixJCWz+MkdWYulVIvPigSduxNCcqUPAlDGLAwPNi8zsTbXr2C+uZ++ZwmKIeE FsdL1f28PbD35wUSGzqr2/C4JG5SS+iR4DGns/dbRpAmKVyk3LTCKCqvvlvk4/38BVEq JadJff5qJE3RrPZH/aD9KDyU7F7jAqrMEe3PAR9VkM/QgI7LsT7sKQJOCwgnQjGVFwx/ YM5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0QZP/2zPNWkHUV5d1g3n6GdeLfOnqJ4ULZ1taVr6o9I=; b=V/iPDszopes/8MlxBteV11SYlSyyqLnXCpQYK69KPrZ0kDl6OTXNkobZvRHn9qGHbq T9rp5vVHjtpGtwNdxcWZ/p47bqy63Brbofm8w5vyl0Q76z+22rcLOENINTBc4mGxlJpd lSHoYZ+7Wp1Hl+8ixH7PhsVfCILqnvb9yEBd0y0H9HWdWecjTREz5K1nUK1pVG9t8bDm dq8yU4dFpl/fUlscwIQefVLELnnUS2OasV9mBPzDbSukNobcTBmd60sAUjIIyhJQGauL 4m67sj6G1tp7WKm+lffZnj8Yqjxh1vqR99aNJIc3m06LM65LzVTNXpFAJHsUxvk+SOdn MYLA== X-Gm-Message-State: AOAM533Pd4lcgmjPK57QmMruPUBPsnsE+YmnFPqUOtTcXVrj2banZJ6m 4GtCR+o7udOb65Tw8Pkaag== X-Google-Smtp-Source: ABdhPJx09r/EWdCNBtiolI2VCU/oNAT+q3xo0UVQ1Kbrvu0hzfmhwbMsQImXmQCQoa90xHQIKWewXA== X-Received: by 2002:aa7:9ab0:0:b029:2ea:1d0:f326 with SMTP id x16-20020aa79ab00000b02902ea01d0f326mr1748313pfi.1.1622763406327; Thu, 03 Jun 2021 16:36:46 -0700 (PDT) Received: from localhost.localdomain (h175-177-040-153.catv02.itscom.jp. [175.177.40.153]) by smtp.gmail.com with ESMTPSA id gg22sm3009818pjb.17.2021.06.03.16.36.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Jun 2021 16:36:46 -0700 (PDT) From: Naoya Horiguchi To: Oscar Salvador , Muchun Song , Mike Kravetz , linux-mm@kvack.org Cc: Andrew Morton , Michal Hocko , Tony Luck , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v6 2/2] mm,hwpoison: make get_hwpoison_page() call get_any_page() Date: Fri, 4 Jun 2021 08:36:32 +0900 Message-Id: <20210603233632.2964832-3-nao.horiguchi@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210603233632.2964832-1-nao.horiguchi@gmail.com> References: <20210603233632.2964832-1-nao.horiguchi@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: D55CF4202A29 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=KrycwZLV; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of naohoriguchi@gmail.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=naohoriguchi@gmail.com X-Rspamd-Server: rspam03 X-Stat-Signature: z8mnpktnjofaeck739re191awygsoki9 X-HE-Tag: 1622763404-257981 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi __get_hwpoison_page() could fail to grab refcount by some race condition, so it's helpful if we can handle it by retrying. We already have retry logic, so make get_hwpoison_page() call get_any_page() when called from memory_failure(). As a result, get_hwpoison_page() can return negative values (i.e. error code), so some callers are also changed to handle error cases. soft_offline_page() does nothing for -EBUSY because that's enough and users in userspace can easily handle it. unpoison_memory() is also unchanged because it's broken and need thorough fixes (will be done later). Signed-off-by: Naoya Horiguchi --- ChangeLog v6: - some callers of get_hwpoison_page() have code for negative return value= s. --- mm/hugetlb.c | 2 + mm/memory-failure.c | 194 +++++++++++++++++++++++++------------------- 2 files changed, 111 insertions(+), 85 deletions(-) diff --git v5.13-rc2/mm/hugetlb.c v5.13-rc2_patched/mm/hugetlb.c index 0c1d87b1e303..5480cdcf62cd 100644 --- v5.13-rc2/mm/hugetlb.c +++ v5.13-rc2_patched/mm/hugetlb.c @@ -5857,6 +5857,8 @@ int get_hwpoison_huge_page(struct page *page, bool = *hugetlb) *hugetlb =3D true; if (HPageFreed(page) || HPageMigratable(page)) ret =3D get_page_unless_zero(page); + else + ret =3D -EBUSY; } spin_unlock_irq(&hugetlb_lock); return ret; diff --git v5.13-rc2/mm/memory-failure.c v5.13-rc2_patched/mm/memory-fail= ure.c index 0caedbeeed77..359ca13b3cad 100644 --- v5.13-rc2/mm/memory-failure.c +++ v5.13-rc2_patched/mm/memory-failure.c @@ -1103,13 +1103,6 @@ static inline bool HWPoisonHandlable(struct page *= page) return PageLRU(page) || __PageMovable(page); } =20 -/** - * __get_hwpoison_page() - Get refcount for memory error handling: - * @page: raw error page (hit by memory error) - * - * Return: return 0 if failed to grab the refcount, otherwise true (some - * non-zero value.) - */ static int __get_hwpoison_page(struct page *page) { struct page *head =3D compound_head(page); @@ -1154,15 +1147,6 @@ static int __get_hwpoison_page(struct page *page) return 0; } =20 -/* - * Safely get reference count of an arbitrary page. - * - * Returns 0 for a free page, 1 for an in-use page, - * -EIO for a page-type we cannot handle and -EBUSY if we raced with an - * allocation. - * We only incremented refcount in case the page was already in-use and = it - * is a known type we can handle. - */ static int get_any_page(struct page *p, unsigned long flags) { int ret =3D 0, pass =3D 0; @@ -1172,50 +1156,77 @@ static int get_any_page(struct page *p, unsigned = long flags) count_increased =3D true; =20 try_again: - if (!count_increased && !__get_hwpoison_page(p)) { - if (page_count(p)) { - /* We raced with an allocation, retry. */ - if (pass++ < 3) - goto try_again; - ret =3D -EBUSY; - } else if (!PageHuge(p) && !is_free_buddy_page(p)) { - /* We raced with put_page, retry. */ + if (!count_increased) { + ret =3D __get_hwpoison_page(p); + if (!ret) { + if (page_count(p)) { + /* We raced with an allocation, retry. */ + if (pass++ < 3) + goto try_again; + ret =3D -EBUSY; + } else if (!PageHuge(p) && !is_free_buddy_page(p)) { + /* We raced with put_page, retry. */ + if (pass++ < 3) + goto try_again; + ret =3D -EIO; + } + goto out; + } else if (ret =3D=3D -EBUSY) { + /* We raced with freeing huge page to buddy, retry. */ if (pass++ < 3) goto try_again; - ret =3D -EIO; + goto out; } + } + + if (PageHuge(p) || HWPoisonHandlable(p)) { + ret =3D 1; } else { - if (PageHuge(p) || HWPoisonHandlable(p)) { - ret =3D 1; - } else { - /* - * A page we cannot handle. Check whether we can turn - * it into something we can handle. - */ - if (pass++ < 3) { - put_page(p); - shake_page(p, 1); - count_increased =3D false; - goto try_again; - } + /* + * A page we cannot handle. Check whether we can turn + * it into something we can handle. + */ + if (pass++ < 3) { put_page(p); - ret =3D -EIO; + shake_page(p, 1); + count_increased =3D false; + goto try_again; } + put_page(p); + ret =3D -EIO; } - +out: return ret; } =20 -static int get_hwpoison_page(struct page *p, unsigned long flags, - enum mf_flags ctxt) +/** + * get_hwpoison_page() - Get refcount for memory error handling + * @p: Raw error page (hit by memory error) + * @flags: Flags controlling behavior of error handling + * + * get_hwpoison_page() takes a page refcount of an error page to handle = memory + * error on it, after checking that the error page is in a well-defined = state + * (defined as a page-type we can successfully handle the memor error on= it, + * such as LRU page and hugetlb page). + * + * Memory error handling could be triggered at any time on any type of p= age, + * so it's prone to race with typical memory management lifecycle (like + * allocation and free). So to avoid such races, get_hwpoison_page() ta= kes + * extra care for the error page's state (as done in __get_hwpoison_page= ()), + * and has some retry logic in get_any_page(). + * + * Return: 0 on failure, + * 1 on success for in-use pages in a well-defined state, + * -EIO for pages on which we can not handle memory errors, + * -EBUSY when get_hwpoison_page() has raced with page lifecycle + * operations like allocation and free. + */ +static int get_hwpoison_page(struct page *p, unsigned long flags) { int ret; =20 zone_pcp_disable(page_zone(p)); - if (ctxt =3D=3D MF_SOFT_OFFLINE) - ret =3D get_any_page(p, flags); - else - ret =3D __get_hwpoison_page(p); + ret =3D get_any_page(p, flags); zone_pcp_enable(page_zone(p)); =20 return ret; @@ -1404,27 +1415,33 @@ static int memory_failure_hugetlb(unsigned long p= fn, int flags) =20 num_poisoned_pages_inc(); =20 - if (!(flags & MF_COUNT_INCREASED) && !get_hwpoison_page(p, flags, 0)) { - /* - * Check "filter hit" and "race with other subpage." - */ - lock_page(head); - if (PageHWPoison(head)) { - if ((hwpoison_filter(p) && TestClearPageHWPoison(p)) - || (p !=3D head && TestSetPageHWPoison(head))) { - num_poisoned_pages_dec(); - unlock_page(head); - return 0; + if (!(flags & MF_COUNT_INCREASED)) { + res =3D get_hwpoison_page(p, flags); + if (!res) { + /* + * Check "filter hit" and "race with other subpage." + */ + lock_page(head); + if (PageHWPoison(head)) { + if ((hwpoison_filter(p) && TestClearPageHWPoison(p)) + || (p !=3D head && TestSetPageHWPoison(head))) { + num_poisoned_pages_dec(); + unlock_page(head); + return 0; + } } + unlock_page(head); + res =3D MF_FAILED; + if (!dissolve_free_huge_page(p) && take_page_off_buddy(p)) { + page_ref_inc(p); + res =3D MF_RECOVERED; + } + action_result(pfn, MF_MSG_FREE_HUGE, res); + return res =3D=3D MF_RECOVERED ? 0 : -EBUSY; + } else if (res < 0) { + action_result(pfn, MF_MSG_UNKNOWN, MF_IGNORED); + return -EBUSY; } - unlock_page(head); - res =3D MF_FAILED; - if (!dissolve_free_huge_page(p) && take_page_off_buddy(p)) { - page_ref_inc(p); - res =3D MF_RECOVERED; - } - action_result(pfn, MF_MSG_FREE_HUGE, res); - return res =3D=3D MF_RECOVERED ? 0 : -EBUSY; } =20 lock_page(head); @@ -1627,28 +1644,35 @@ int memory_failure(unsigned long pfn, int flags) * In fact it's dangerous to directly bump up page count from 0, * that may make page_ref_freeze()/page_ref_unfreeze() mismatch. */ - if (!(flags & MF_COUNT_INCREASED) && !get_hwpoison_page(p, flags, 0)) { - if (is_free_buddy_page(p)) { - if (take_page_off_buddy(p)) { - page_ref_inc(p); - res =3D MF_RECOVERED; - } else { - /* We lost the race, try again */ - if (retry) { - ClearPageHWPoison(p); - num_poisoned_pages_dec(); - retry =3D false; - goto try_again; + if (!(flags & MF_COUNT_INCREASED)) { + res =3D get_hwpoison_page(p, flags); + if (!res) { + if (is_free_buddy_page(p)) { + if (take_page_off_buddy(p)) { + page_ref_inc(p); + res =3D MF_RECOVERED; + } else { + /* We lost the race, try again */ + if (retry) { + ClearPageHWPoison(p); + num_poisoned_pages_dec(); + retry =3D false; + goto try_again; + } + res =3D MF_FAILED; } - res =3D MF_FAILED; + action_result(pfn, MF_MSG_BUDDY, res); + res =3D res =3D=3D MF_RECOVERED ? 0 : -EBUSY; + } else { + action_result(pfn, MF_MSG_KERNEL_HIGH_ORDER, MF_IGNORED); + res =3D -EBUSY; } - action_result(pfn, MF_MSG_BUDDY, res); - res =3D res =3D=3D MF_RECOVERED ? 0 : -EBUSY; - } else { - action_result(pfn, MF_MSG_KERNEL_HIGH_ORDER, MF_IGNORED); + goto unlock_mutex; + } else if (res < 0) { + action_result(pfn, MF_MSG_UNKNOWN, MF_IGNORED); res =3D -EBUSY; + goto unlock_mutex; } - goto unlock_mutex; } =20 if (PageTransHuge(hpage)) { @@ -1919,7 +1943,7 @@ int unpoison_memory(unsigned long pfn) return 0; } =20 - if (!get_hwpoison_page(p, flags, 0)) { + if (!get_hwpoison_page(p, flags)) { if (TestClearPageHWPoison(p)) num_poisoned_pages_dec(); unpoison_pr_info("Unpoison: Software-unpoisoned free page %#lx\n", @@ -2135,7 +2159,7 @@ int soft_offline_page(unsigned long pfn, int flags) =20 retry: get_online_mems(); - ret =3D get_hwpoison_page(page, flags, MF_SOFT_OFFLINE); + ret =3D get_hwpoison_page(page, flags); put_online_mems(); =20 if (ret > 0) { --=20 2.25.1