From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC96EC761A6 for ; Mon, 27 Mar 2023 20:46:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 141E0900003; Mon, 27 Mar 2023 16:46:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F2D5900002; Mon, 27 Mar 2023 16:46:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED547900003; Mon, 27 Mar 2023 16:46:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D8120900002 for ; Mon, 27 Mar 2023 16:46:47 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A7C3A807FF for ; Mon, 27 Mar 2023 20:46:47 +0000 (UTC) X-FDA: 80615862054.15.66D589D Received: from mail-yw1-f180.google.com (mail-yw1-f180.google.com [209.85.128.180]) by imf25.hostedemail.com (Postfix) with ESMTP id D49C0A0009 for ; Mon, 27 Mar 2023 20:46:45 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=UR8NJiXJ; spf=pass (imf25.hostedemail.com: domain of jiaqiyan@google.com designates 209.85.128.180 as permitted sender) smtp.mailfrom=jiaqiyan@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679950005; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2DND7mD0TD+G8uMVwVEkeDqshCUfVeRJdHizEHaV+8E=; b=vZib/IWZVOz9ygOPnPYoMs+ELXJXrzO39mSNaSyk54ILCclkO2CCONteL287WKLVB+ajh8 WNEtI+Y+1pm3WTWlojLplfMtjvQqIVqbn0hWusj0juAUm7PEJDvS/3I0fklf0nhfZ6rR8i ddmQdzYz0O7KHAhAGnj2tr82Hg3cf2c= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=UR8NJiXJ; spf=pass (imf25.hostedemail.com: domain of jiaqiyan@google.com designates 209.85.128.180 as permitted sender) smtp.mailfrom=jiaqiyan@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679950005; a=rsa-sha256; cv=none; b=YyX+n4fN2sos3EFxyXXLE8s8z1THOzoZRGSWBVbpDiwycgUbUyvoaSJ7q/WlU241zZaTV1 VqDSM4JESG8ZAazZ5j3jQLB5OoJ8d2yFpEBD+KrzyGj8UtEkx4wiBWacTInvtI6bYAkomG 5H/V9MfDiF/lXfNC0EEZvenlXIisCCg= Received: by mail-yw1-f180.google.com with SMTP id 00721157ae682-5445009c26bso193171347b3.8 for ; Mon, 27 Mar 2023 13:46:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679950005; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=2DND7mD0TD+G8uMVwVEkeDqshCUfVeRJdHizEHaV+8E=; b=UR8NJiXJ7YYb4m1C3P0VIsZGtjEhdJWSg6Pmp6DBLDzSiB83+5fUF/rozcQIB3zJew orsVlfPflPKGMzj2n+65F6bqHm+PWtA1rZ5o+1b/Ge9tN7b82TLvo28zHiWo3N6/H8N9 mFKTgEd9S8MCW3aVBFRnpGvHBZU7y5seFa7eQ/NU9sg14ngJpozA8uwMecoUU2+hcKh8 JW5n/ApnWlxdeu52s8AGLIz7X1yIi5xhWpj94qguXLkilwOoQ50//oTd0OF0jcKIWV/f TMrkJhQreTkBz3y3w4PLssbeDBifZGoJ7iEkz6iO91Ejq48zhK9brch3LVJ/T3Tm45I0 73sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679950005; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2DND7mD0TD+G8uMVwVEkeDqshCUfVeRJdHizEHaV+8E=; b=mqeeAT3HH/IAHV1OXzhXqaZ58fpv9/diamKTmFFtlh+ok5ugmlsyYI0fWdEfVEMkop /2NDN6LpeOuMMYixLtAG9J8dlv4r80W0HeoxGEGZFXmcuNBZvCSDf9u3SSxF6xkZEUiv 4D4Bx1RB628+vYmU/m60/FuLXunM0YbtQDsX5dJcFhPVVN7hKbu9ReJ1vHkFNn3W3Os8 TZmF4mSOcB6qFD8YDDDE2yA9tLkwaeXPAUefJ/BsxBNBlpsdNawLqsK37Ple/H0lb0rd Fgo2SvcgN4cjfwLVhuiz/lmsZChtS2b3vwMxXVstxnA5uxetXqa992PFO9N3yD2YBlfI 8INQ== X-Gm-Message-State: AAQBX9c13egdL2jA+bmy0s3/HYWKT0tckYFozvx14dRCTtMsc90GhsZG HGCvHUI0d/co+fKDC29kvc6A/AawrO2PBcGGWgI/0w== X-Google-Smtp-Source: AKy350Z5Q7NzOmwhqFLQ70ldeFzii2wZwJv6hESkwnoiAdArUYpmYxOKJdQgrk98ymP82JOuVuyNvKZWtxxzRUWOshY= X-Received: by 2002:a0d:ec52:0:b0:541:822f:af40 with SMTP id r18-20020a0dec52000000b00541822faf40mr5932018ywn.10.1679950004561; Mon, 27 Mar 2023 13:46:44 -0700 (PDT) MIME-Version: 1.0 References: <20230305065112.1932255-1-jiaqiyan@google.com> <20230305065112.1932255-2-jiaqiyan@google.com> In-Reply-To: From: Jiaqi Yan Date: Mon, 27 Mar 2023 13:46:33 -0700 Message-ID: Subject: Re: [PATCH v10 1/3] mm/khugepaged: recover from poisoned anonymous memory To: Yang Shi Cc: kirill.shutemov@linux.intel.com, kirill@shutemov.name, tongtiangen@huawei.com, tony.luck@intel.com, akpm@linux-foundation.org, naoya.horiguchi@nec.com, linmiaohe@huawei.com, linux-mm@kvack.org, osalvador@suse.de, wangkefeng.wang@huawei.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: w9p9aqtzjd7jsyhd7jexedrqecnwdj54 X-Rspamd-Queue-Id: D49C0A0009 X-HE-Tag: 1679950005-718078 X-HE-Meta: U2FsdGVkX19W+QfmRENucGNJlWxrqO5vYY1YZg9+nmbFHEF8K2hP4e8U/O5nGtHArYj3QihNqUJMcIt5nmBgz4xF5DqQrFyqbR3dS6/ISOrWhjDPJdOAW6tiaBil7AcqTXMhCCgAJYEM/LXElckvII0o5tVfZ19IgD8nl2P1wSWRk8GFTIbQYw4EwcRyNGHFuAoY8eE17B1W2Iw96drImXoDJ/t/fG/sD0+iRx1deskNHOM0rGSLZfh2HFhNXtAeJbEeT/xm5eZUlsNaKeAGLhmNR9Ow5CioA1UAB2w5egnY180gPr5ECqUnY8aB3j06Xol0Itv0QR46womdTjFXKC+GDLM+JHVTrRXam//f0CcLB+Le0fjTLWTqYJGt2zaqWvqEmfx92jhLOxZP7reDYKa5ERx8hWek0dEvTc2gvpiGHCNspjlcw+Z+r6D7uXMq5Cl6mmthOw+omy5xZ503ePWjUi93yrNvyZeakmrxlvmFS0OVLMSSTTNd2+bjRZ5O0yLFvc8/SZ21oJym3wmjE0GmDqT/q0JpImjcmuQHZ79EM0Zpj9iTMUPuuJM5GKMiWjHAbk/cWBIHyMeJeRqiZMYOlc+5GSmrEunaCDwbTiLPIkgHLPzAj1WcuIpKDUUKLPX9v1zD/zqfBpkow49Fe8TrV7n+pN/GlUkNolqPh16uersAdKT/ot7rgm66iDpP0LSTJ2n3LBeRc8SqIcktTRg9BLzfgZqA9dTjpCMWx8biRvl2blhf7A3Fp9+sCXnJK0DCduEWyZgeA17gQVRzTdfb1VaGRM8dC01uyUWgElHxZVSRmfMiTaQrNKmMq+7rxhqr4fVWHh22C5uedqFnaPPQ3hHBsJlob/t4MO4aW4BIAkiSxJmvSSD173axu/5l0mBYOQR1MT+4PAY1NCNT2G55zwlAyieUt29QxV9jq3e/aoLzc0xoU6EmdKoMT5eVWN0pR/vPsBQKRW37K9R xjN1mWIt /6TQgpSzXhLHQq3DBDIBeB+C19cJresyywz56Rk/n2CBzIvVTW49uu/j6D8Cf8P7fFpSFq3alMRdZi6lze2+ObiBp44iXDWm+sf+3XZB1yThqspHSdy+35fo2gkTGG2zmtL25czsq7mFFWN7GEIsHkdzAj8Bfu8FcyU5hEjwVKejIctaxdo1UhBIlRtqQPwvav5Gp3yeYraO6b+oHec8EA4vkoWM9QCyHHH0zHolcajDkf9M2j5bCpK+5xnJZ2z6upDgv3C6tNXBs2Ol8WByTvBOPs1pLyeZUxOOVVe9A5sSETSI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Mar 24, 2023 at 3:31=E2=80=AFPM Jiaqi Yan wro= te: > > On Fri, Mar 24, 2023 at 1:11=E2=80=AFPM Yang Shi wr= ote: > > > > On Fri, Mar 24, 2023 at 8:34=E2=80=AFAM Jiaqi Yan = wrote: > > > > > > On Thu, Mar 23, 2023 at 2:38=E2=80=AFPM Yang Shi wrote: > > > > > > > > On Sat, Mar 4, 2023 at 10:51=E2=80=AFPM Jiaqi Yan wrote: > > > > > > > > > > Make __collapse_huge_page_copy return whether copying anonymous p= ages > > > > > succeeded, and make collapse_huge_page handle the return status. > > > > > > > > > > Break existing PTE scan loop into two for-loops. The first loop c= opies > > > > > source pages into target huge page, and can fail gracefully when = running > > > > > into memory errors in source pages. If copying all pages succeeds= , the > > > > > second loop releases and clears up these normal pages. Otherwise,= the > > > > > second loop rolls back the page table and page states by: > > > > > - re-establishing the original PTEs-to-PMD connection. > > > > > - releasing source pages back to their LRU list. > > > > > > > > > > Tested manually: > > > > > 0. Enable khugepaged on system under test. > > > > > 1. Start a two-thread application. Each thread allocates a chunk = of > > > > > non-huge anonymous memory buffer. > > > > > 2. Pick 4 random buffer locations (2 in each thread) and inject > > > > > uncorrectable memory errors at corresponding physical addresse= s. > > > > > 3. Signal both threads to make their memory buffer collapsible, i= .e. > > > > > calling madvise(MADV_HUGEPAGE). > > > > > 4. Wait and check kernel log: khugepaged is able to recover from = poisoned > > > > > pages and skips collapsing them. > > > > > 5. Signal both threads to inspect their buffer contents and make = sure no > > > > > data corruption. > > > > > > > > > > Signed-off-by: Jiaqi Yan > > > > > --- > > > > > include/trace/events/huge_memory.h | 3 +- > > > > > mm/khugepaged.c | 148 +++++++++++++++++++++++= +----- > > > > > 2 files changed, 128 insertions(+), 23 deletions(-) > > > > > > > > > > diff --git a/include/trace/events/huge_memory.h b/include/trace/e= vents/huge_memory.h > > > > > index 3e6fb05852f9a..46cce509957ba 100644 > > > > > --- a/include/trace/events/huge_memory.h > > > > > +++ b/include/trace/events/huge_memory.h > > > > > @@ -36,7 +36,8 @@ > > > > > EM( SCAN_ALLOC_HUGE_PAGE_FAIL, "alloc_huge_page_failed")= \ > > > > > EM( SCAN_CGROUP_CHARGE_FAIL, "ccgroup_charge_failed") = \ > > > > > EM( SCAN_TRUNCATED, "truncated") = \ > > > > > - EMe(SCAN_PAGE_HAS_PRIVATE, "page_has_private") = \ > > > > > + EM( SCAN_PAGE_HAS_PRIVATE, "page_has_private") = \ > > > > > + EMe(SCAN_COPY_MC, "copy_poisoned_page") = \ > > > > > > > > > > #undef EM > > > > > #undef EMe > > > > > diff --git a/mm/khugepaged.c b/mm/khugepaged.c > > > > > index 27956d4404134..c3c217f6ebc6e 100644 > > > > > --- a/mm/khugepaged.c > > > > > +++ b/mm/khugepaged.c > > > > > @@ -19,6 +19,7 @@ > > > > > #include > > > > > #include > > > > > #include > > > > > +#include > > > > > > > > > > #include > > > > > #include > > > > > @@ -55,6 +56,7 @@ enum scan_result { > > > > > SCAN_CGROUP_CHARGE_FAIL, > > > > > SCAN_TRUNCATED, > > > > > SCAN_PAGE_HAS_PRIVATE, > > > > > + SCAN_COPY_MC, > > > > > }; > > > > > > > > > > #define CREATE_TRACE_POINTS > > > > > @@ -681,47 +683,47 @@ static int __collapse_huge_page_isolate(str= uct vm_area_struct *vma, > > > > > return result; > > > > > } > > > > > > > > > > -static void __collapse_huge_page_copy(pte_t *pte, struct page *p= age, > > > > > - struct vm_area_struct *vma, > > > > > - unsigned long address, > > > > > - spinlock_t *ptl, > > > > > - struct list_head *compound_= pagelist) > > > > > +static void __collapse_huge_page_copy_succeeded(pte_t *pte, > > > > > + pmd_t *pmd, > > > > > + struct vm_area_st= ruct *vma, > > > > > + unsigned long add= ress, > > > > > + spinlock_t *pte_p= tl, > > > > > + struct list_head = *compound_pagelist) > > > > > { > > > > > struct page *src_page, *tmp; > > > > > pte_t *_pte; > > > > > - for (_pte =3D pte; _pte < pte + HPAGE_PMD_NR; > > > > > - _pte++, page++, address +=3D PAGE= _SIZE) { > > > > > - pte_t pteval =3D *_pte; > > > > > + pte_t pteval; > > > > > + unsigned long _address; > > > > > > > > > > + for (_pte =3D pte, _address =3D address; _pte < pte + HPA= GE_PMD_NR; > > > > > + _pte++, _address +=3D PAGE_SIZE) { > > > > > + pteval =3D *_pte; > > > > > if (pte_none(pteval) || is_zero_pfn(pte_pfn(pteva= l))) { > > > > > - clear_user_highpage(page, address); > > > > > add_mm_counter(vma->vm_mm, MM_ANONPAGES, = 1); > > > > > if (is_zero_pfn(pte_pfn(pteval))) { > > > > > /* > > > > > - * ptl mostly unnecessary. > > > > > + * pte_ptl mostly unnecessary. > > > > > */ > > > > > - spin_lock(ptl); > > > > > - ptep_clear(vma->vm_mm, address, _= pte); > > > > > - spin_unlock(ptl); > > > > > + spin_lock(pte_ptl); > > > > > > > > Why did you have to rename ptl to pte_ptl? It seems unnecessary. > > > > > > Thanks, I will use `ptl` in the next version. > > > > > > > > > > > > + pte_clear(vma->vm_mm, _address, _= pte); A bug here: calling pte_clear is wrong; we should use ptep_clear. V11 will make sure it is fixed. BTW, __collapse_huge_page_copy_succeeded can drop _address and use address directly. > > > > > + spin_unlock(pte_ptl); > > > > > } > > > > > } else { > > > > > src_page =3D pte_page(pteval); > > > > > - copy_user_highpage(page, src_page, addres= s, vma); > > > > > if (!PageCompound(src_page)) > > > > > release_pte_page(src_page); > > > > > /* > > > > > - * ptl mostly unnecessary, but preempt ha= s to > > > > > - * be disabled to update the per-cpu stat= s > > > > > + * pte_ptl mostly unnecessary, but preemp= t has > > > > > + * to be disabled to update the per-cpu s= tats > > > > > * inside page_remove_rmap(). > > > > > */ > > > > > - spin_lock(ptl); > > > > > - ptep_clear(vma->vm_mm, address, _pte); > > > > > + spin_lock(pte_ptl); > > > > > + ptep_clear(vma->vm_mm, _address, _pte); > > > > > page_remove_rmap(src_page, vma, false); > > > > > - spin_unlock(ptl); > > > > > + spin_unlock(pte_ptl); > > > > > free_page_and_swap_cache(src_page); > > > > > } > > > > > } > > > > > - > > > > > list_for_each_entry_safe(src_page, tmp, compound_pagelist= , lru) { > > > > > list_del(&src_page->lru); > > > > > mod_node_page_state(page_pgdat(src_page), > > > > > @@ -733,6 +735,104 @@ static void __collapse_huge_page_copy(pte_t= *pte, struct page *page, > > > > > } > > > > > } > > > > > > > > > > +static void __collapse_huge_page_copy_failed(pte_t *pte, > > > > > + pmd_t *pmd, > > > > > + pmd_t orig_pmd, > > > > > + struct vm_area_struc= t *vma, > > > > > + unsigned long addres= s, > > > > > + struct list_head *co= mpound_pagelist) > > > > > +{ > > > > > + struct page *src_page, *tmp; > > > > > + pte_t *_pte; > > > > > + pte_t pteval; > > > > > + unsigned long _address; > > > > > + spinlock_t *pmd_ptl; > > > > > + > > > > > + /* > > > > > + * Re-establish the PMD to point to the original page tab= le > > > > > + * entry. Restoring PMD needs to be done prior to releasi= ng > > > > > + * pages. Since pages are still isolated and locked here, > > > > > + * acquiring anon_vma_lock_write is unnecessary. > > > > > + */ > > > > > + pmd_ptl =3D pmd_lock(vma->vm_mm, pmd); > > > > > + pmd_populate(vma->vm_mm, pmd, pmd_pgtable(orig_pmd)); > > > > > + spin_unlock(pmd_ptl); > > > > > + /* > > > > > + * Release both raw and compound pages isolated > > > > > + * in __collapse_huge_page_isolate. > > > > > + */ > > > > > > > > It looks like the below code could be replaced by release_pte_pages= () > > > > with advancing _pte to (pte + HPAGE_PMD_NR - 1). > > > > > > > > > > Yeah, but I think _pte should be (pte + HPAGE_PMR_NR) because _pte is > > > decremented before comparison in release_pte_pages(pte, _pte, > > > compound_pagelist): > > > > > > while (--_pte >=3D pte) {...} > > > > > > Advancing _pte to (pte + HPAGE_PMD_NR - 1) may leave the last page no= t released. > > > > Yeah, good catch. I think it is because the only user of > > release_pte_pages() is __collapse_huge_page_isolate(). Once the loop > > in it is done _pte is pte + HPAGE_PMD_NR. > > > > Thanks for confirming ;) > Since you have reviewed other 2 commits, I will soon send out V11 for > your comments on this commit. > > > > > > > > > > > > > > + for (_pte =3D pte, _address =3D address; _pte < pte + HPA= GE_PMD_NR; > > > > > + _pte++, _address +=3D PAGE_SIZE) { > > > > > + pteval =3D *_pte; > > > > > + if (pte_none(pteval) || is_zero_pfn(pte_pfn(pteva= l))) > > > > > + continue; > > > > > + src_page =3D pte_page(pteval); > > > > > + if (!PageCompound(src_page)) > > > > > + release_pte_page(src_page); > > > > > + } > > > > > + list_for_each_entry_safe(src_page, tmp, compound_pagelist= , lru) { > > > > > + list_del(&src_page->lru); > > > > > + release_pte_page(src_page); > > > > > + } > > > > > +} > > > > > + > > > > > +/* > > > > > + * __collapse_huge_page_copy - attempts to copy memory contents = from raw > > > > > + * pages to a hugepage. Cleans up the raw pages if copying succe= eds; > > > > > + * otherwise restores the original page table and releases isola= ted raw pages. > > > > > + * Returns SCAN_SUCCEED if copying succeeds, otherwise returns S= CAN_COPY_MC. > > > > > + * > > > > > + * @pte: starting of the PTEs to copy from > > > > > + * @page: the new hugepage to copy contents to > > > > > + * @pmd: pointer to the new hugepage's PMD > > > > > + * @orig_pmd: the original raw pages' PMD > > > > > + * @vma: the original raw pages' virtual memory area > > > > > + * @address: starting address to copy > > > > > + * @pte_ptl: lock on raw pages' PTEs > > > > > + * @compound_pagelist: list that stores compound pages > > > > > + */ > > > > > +static int __collapse_huge_page_copy(pte_t *pte, > > > > > + struct page *page, > > > > > + pmd_t *pmd, > > > > > + pmd_t orig_pmd, > > > > > + struct vm_area_struct *vma, > > > > > + unsigned long address, > > > > > + spinlock_t *pte_ptl, > > > > > + struct list_head *compound_p= agelist) > > > > > +{ > > > > > + struct page *src_page; > > > > > + pte_t *_pte; > > > > > + pte_t pteval; > > > > > + unsigned long _address; > > > > > + int result =3D SCAN_SUCCEED; > > > > > + > > > > > + /* > > > > > + * Copying pages' contents is subject to memory poison at= any iteration. > > > > > + */ > > > > > + for (_pte =3D pte, _address =3D address; _pte < pte + HPA= GE_PMD_NR; > > > > > + _pte++, page++, _address +=3D PAGE_SIZE) { > > > > > + pteval =3D *_pte; > > > > > + if (pte_none(pteval) || is_zero_pfn(pte_pfn(pteva= l))) { > > > > > + clear_user_highpage(page, _address); > > > > > + continue; > > > > > + } > > > > > + src_page =3D pte_page(pteval); > > > > > + if (copy_mc_user_highpage(page, src_page, _addres= s, vma) > 0) { > > > > > + result =3D SCAN_COPY_MC; > > > > > + break; > > > > > + } > > > > > + } > > > > > + > > > > > + if (likely(result =3D=3D SCAN_SUCCEED)) > > > > > + __collapse_huge_page_copy_succeeded(pte, pmd, vma= , address, > > > > > + pte_ptl, comp= ound_pagelist); > > > > > + else > > > > > + __collapse_huge_page_copy_failed(pte, pmd, orig_p= md, vma, > > > > > + address, compoun= d_pagelist); > > > > > + > > > > > + return result; > > > > > +} > > > > > + > > > > > static void khugepaged_alloc_sleep(void) > > > > > { > > > > > DEFINE_WAIT(wait); > > > > > @@ -1106,9 +1206,13 @@ static int collapse_huge_page(struct mm_st= ruct *mm, unsigned long address, > > > > > */ > > > > > anon_vma_unlock_write(vma->anon_vma); > > > > > > > > > > - __collapse_huge_page_copy(pte, hpage, vma, address, pte_p= tl, > > > > > - &compound_pagelist); > > > > > + result =3D __collapse_huge_page_copy(pte, hpage, pmd, _pm= d, > > > > > + vma, address, pte_ptl, > > > > > + &compound_pagelist); > > > > > pte_unmap(pte); > > > > > + if (unlikely(result !=3D SCAN_SUCCEED)) > > > > > + goto out_up_write; > > > > > + > > > > > /* > > > > > * spin_lock() below is not the equivalent of smp_wmb(), = but > > > > > * the smp_wmb() inside __SetPageUptodate() can be reused= to > > > > > -- > > > > > 2.40.0.rc0.216.gc4246ad0f0-goog > > > > >