From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7DA1C38145 for ; Tue, 6 Sep 2022 08:14:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DDFAC80269; Tue, 6 Sep 2022 04:14:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D8F4780224; Tue, 6 Sep 2022 04:14:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7DDA80269; Tue, 6 Sep 2022 04:14:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B981480224 for ; Tue, 6 Sep 2022 04:14:48 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8DD91A03EE for ; Tue, 6 Sep 2022 08:14:48 +0000 (UTC) X-FDA: 79880949456.06.598BBC3 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf01.hostedemail.com (Postfix) with ESMTP id 6DEDB4006B for ; Tue, 6 Sep 2022 08:14:46 +0000 (UTC) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.57]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4MMJ3y5rZdz1P6tM; Tue, 6 Sep 2022 16:10:54 +0800 (CST) Received: from [10.174.177.76] (10.174.177.76) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Tue, 6 Sep 2022 16:14:41 +0800 Subject: Re: [PATCH v2 1/4] mm,hwpoison,hugetlb,memory_hotplug: hotremove memory section with hwpoisoned hugepage To: =?UTF-8?B?SE9SSUdVQ0hJIE5BT1lBKOWggOWPoyDnm7TkuZ8p?= CC: Naoya Horiguchi , Andrew Morton , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , "linux-kernel@vger.kernel.org" , Linux-MM References: <20220905062137.1455537-1-naoya.horiguchi@linux.dev> <20220905062137.1455537-2-naoya.horiguchi@linux.dev> <3e302aa5-d63d-097e-2cb7-831b7c99e736@huawei.com> <20220906061417.GA1406504@hori.linux.bs1.fc.nec.co.jp> From: Miaohe Lin Message-ID: <0edc3912-475e-0366-666c-9be516af1df1@huawei.com> Date: Tue, 6 Sep 2022 16:14:40 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: <20220906061417.GA1406504@hori.linux.bs1.fc.nec.co.jp> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [10.174.177.76] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662452087; a=rsa-sha256; cv=none; b=W+J2RFGNbiHPDurUzk9sDOnpFJ0wfYUSkUnzVCAFEVQ5/EavRjPjrzcT9YFQL9OZifb4o7 wNjT+Nnm7lftetXFhxU4yBIecsRkaate5x4jIbcGT8Jb0xFt3acMqrp/Thj72LRpMYFRju EI3NotYLAG1quWc5f4VJOI8PDLF/s3o= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of linmiaohe@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=linmiaohe@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662452087; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PNcUFbsFkd4M7DraaFLig42bvZC/zG0Jpo32fTwBZDU=; b=UoygSiJH23en97ouvMHKxiWDJp55UerzWs37Dmk8FDp4I7t6XOe8KUBbBlEfDLmnob+KJd bXrQ1DcZCCbhMYcNEmklTGLxJHPg/Ywdp1DtacolqDlwONu++GSIqMqKUWaQAzGPI2iEsz O6sSpnbfBAb4P2ZCeQYQgaKcFo+P/1o= X-Stat-Signature: nycnszj6jmpsctgc9b35key4rqg514wu X-Rspamd-Queue-Id: 6DEDB4006B Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of linmiaohe@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=linmiaohe@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1662452086-798534 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2022/9/6 14:14, HORIGUCHI NAOYA(堀口 直也) wrote: > On Tue, Sep 06, 2022 at 10:59:58AM +0800, Miaohe Lin wrote: >> On 2022/9/5 14:21, Naoya Horiguchi wrote: >>> From: Naoya Horiguchi >>> >>> HWPoisoned page is not supposed to be accessed once marked, but currently >>> such accesses can happen during memory hotremove because do_migrate_range() >>> can be called before dissolve_free_huge_pages() is called. >>> >>> Move dissolve_free_huge_pages() before scan_movable_pages(). Recently >>> delayed dissolve has been implemented, so the dissolving can turn >>> a hwpoisoned hugepage into 4kB hwpoison page, which memory hotplug can >>> handle safely. >> >> Yes, thanks for your work, Naoya. ;) >> >>> >>> Reported-by: Miaohe Lin >>> Signed-off-by: Naoya Horiguchi >>> --- >>> mm/memory_hotplug.c | 22 +++++++++++----------- >>> 1 file changed, 11 insertions(+), 11 deletions(-) >>> >>> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c >>> index fad6d1f2262a..c24735d63b25 100644 >>> --- a/mm/memory_hotplug.c >>> +++ b/mm/memory_hotplug.c >>> @@ -1880,6 +1880,17 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages, >>> >>> cond_resched(); >>> >>> + /* >>> + * Dissolve free hugepages in the memory block before doing >>> + * offlining actually in order to make hugetlbfs's object >>> + * counting consistent. >>> + */ >>> + ret = dissolve_free_huge_pages(start_pfn, end_pfn); >>> + if (ret) { >>> + reason = "failure to dissolve huge pages"; >>> + goto failed_removal_isolated; >>> + } >> >> This change has a side-effect. If hugetlb pages are in-use, dissolve_free_huge_pages() will always return -EBUSY >> even if those pages can be migrated. So we fail to hotremove the memory even if they could be offlined. >> Or am I miss something? > > Thank you for the comment, you're right. (Taking a look over my test result > carefully, it showed failures for the related cases, I somehow overlooked > them, really sorry.) So my second thought is that we keep offline_pages() > as is, and insert a few line in do_migrate_range() to handle the case of > hwpoisoned hugepage like below: > > @@ -1642,6 +1642,8 @@ do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) > > if (PageHuge(page)) { > pfn = page_to_pfn(head) + compound_nr(head) - 1; > + if (PageHWPoison(head)) > + continue; Thanks for your update. But it seems this is not enough. With the above code change, HWPoisoned hugetlb pages will always be ignored in do_migrate_range(). And if these pages are HPageMigratable, they will be returned in scan_movable_pages() then passed into the do_migrate_range() again. Thus a possible deadloop will occur until these pages become un-movable? > isolate_hugetlb(head, &source); > continue; > } else if (PageTransHuge(page)) > > This is slightly different from your original suggestion > https://lore.kernel.org/linux-mm/20220421135129.19767-1-linmiaohe@huawei.com/T > , as discussed in the thread existing "if (PageHWPoison(page))" branch in > this function can't be used for hugetlb. We could adjust them to handle > hugetlb, but maybe separating code for hugetlb first from the others looks > less compicated to me. It might be better to do something, e.g. unmap the hugetlb pages to prevent accessing from process if mapped, even try truncating the error page from pagecache. But I have no strong opinion as handling memory failure would always be a best try. ;) Thanks, Miaohe Lin > > If you have any suggestion on this, please let me know. > > Thanks, > Naoya Horiguchi >