From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 195E5C43465 for ; Sun, 20 Sep 2020 09:59:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D22CB2158C for ; Sun, 20 Sep 2020 09:59:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="uf2OGboF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726454AbgITJ7k (ORCPT ); Sun, 20 Sep 2020 05:59:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726280AbgITJ7j (ORCPT ); Sun, 20 Sep 2020 05:59:39 -0400 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0B4DC0613CE for ; Sun, 20 Sep 2020 02:59:38 -0700 (PDT) Received: by mail-pl1-x644.google.com with SMTP id y6so5359069plt.9 for ; Sun, 20 Sep 2020 02:59:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=bsQK8LpuxrhLjsMMeuAwU+LPx3UZkeZ5l4b7tYvmwRw=; b=uf2OGboFLy/L86Po2Es5Zv7ilwDEHjiD68HdWMnC+2TTVeQq6+3TpQ8S4hx54ztRz6 byRNGnVskkzKHg8WColeN7JdIRLxsd260424uAjNg9dOsUdX7kTpbuuiWjtlFCdyXxxB d6Ubn2mPsUlkDVPO/GCNxHunwKCXPScicvQszzSzCGq2eusF2u40NN0hrA9iqGzwzBvv oAM9YZQQn1IcETFhLtCe6kJ90ZtQoanr/NjfI2jG1VGZLhviiFKwbdZuJZbY3VsavHer xv1SnI9UsEv6Mwwu8/vVj4sP5FTPPSnySzsOPSPXrzB8ga5sgjTi1RXltGX/o1IXvm50 RBzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bsQK8LpuxrhLjsMMeuAwU+LPx3UZkeZ5l4b7tYvmwRw=; b=aQq9h0cB+VTzZ1f+/JP9wZOehU6pmPGKI+ZLe8CYoDS6OaJIxdvDF5xFOckbUU/4p0 TfpmtJOC2gwrIY6PMj4IVRIY7USsa113ezZMrl2Wk2cHC6ZtLdq5fB2QKDBm1eAFjVFI cW0giVe0kt3ErWaWc1XDd67I9v+xecPiB/Hr759MGamUtUY9UWWFmVdqdPXd1AseVV4D d/2uOY2jYHhOCWcGt5S9UAU/SKxsbdNoqcatswNYM48aIwepkRmUO3NOQDmvu9SLSXvV YS5KGhcSIazQ3/f+juLbKJbWJQQiji0iCwF9AhylowwyddK3kmK4PtcvuhQDLN3myORX iuwA== X-Gm-Message-State: AOAM532mBGKcHVXhXBT134GKSvyX5Lr782rGniJlTulAjxFDcX/HrHiE Lh3bP9FtvuHPGUjKvtASocx+2RMER4/OkLLwF8H6VQ== X-Google-Smtp-Source: ABdhPJyLB6JeulC3J3qt+PdRVgQHD1bgo33KeQ7Z9TGvBKZsB/ZmYNsfj5G5KTW1QcP1SkVeA5KsxqzEv001KyERvQA= X-Received: by 2002:a17:90a:bc8d:: with SMTP id x13mr20950424pjr.229.1600595978266; Sun, 20 Sep 2020 02:59:38 -0700 (PDT) MIME-Version: 1.0 References: <20200915125947.26204-1-songmuchun@bytedance.com> <20200915125947.26204-22-songmuchun@bytedance.com> In-Reply-To: <20200915125947.26204-22-songmuchun@bytedance.com> From: Muchun Song Date: Sun, 20 Sep 2020 17:59:01 +0800 Message-ID: Subject: Re: [RFC PATCH 21/24] mm/hugetlb: Merge pte to huge pmd only for gigantic page To: Jonathan Corbet , Mike Kravetz , Thomas Gleixner , mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, Peter Zijlstra , viro@zeniv.linux.org.uk, Andrew Morton , paulmck@kernel.org, mchehab+huawei@kernel.org, pawan.kumar.gupta@linux.intel.com, Randy Dunlap , oneukum@suse.com, anshuman.khandual@arm.com, jroedel@suse.de, Mina Almasry , David Rientjes Cc: linux-doc@vger.kernel.org, LKML , Linux Memory Management List , linux-fsdevel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Sep 15, 2020 at 9:03 PM Muchun Song wrote: > > Merge pte to huge pmd if it has ever been split. Now only support > gigantic page which's vmemmap pages size is an integer multiple of > PMD_SIZE. This is the simplest case to handle. > > Signed-off-by: Muchun Song > --- > include/linux/hugetlb.h | 7 +++ > mm/hugetlb.c | 104 +++++++++++++++++++++++++++++++++++++++- > 2 files changed, 109 insertions(+), 2 deletions(-) > > diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h > index e3aa192f1c39..c56df0da7ae5 100644 > --- a/include/linux/hugetlb.h > +++ b/include/linux/hugetlb.h > @@ -611,6 +611,13 @@ static inline bool vmemmap_pmd_huge(pmd_t *pmd) > } > #endif > > +#ifndef vmemmap_pmd_mkhuge > +static inline pmd_t vmemmap_pmd_mkhuge(struct page *page) > +{ > + return pmd_mkhuge(mk_pmd(page, PAGE_KERNEL)); > +} > +#endif > + > #ifndef VMEMMAP_HPAGE_SHIFT > #define VMEMMAP_HPAGE_SHIFT PMD_SHIFT > #endif > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index 28c154679838..3ca36e259b4e 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -1759,6 +1759,62 @@ static void __remap_huge_page_pte_vmemmap(struct page *reuse, pte_t *ptep, > } > } > > +static void __replace_huge_page_pte_vmemmap(pte_t *ptep, unsigned long start, > + unsigned int nr, struct page *huge, > + struct list_head *free_pages) > +{ > + unsigned long addr; > + unsigned long end = start + (nr << PAGE_SHIFT); > + > + for (addr = start; addr < end; addr += PAGE_SIZE, ptep++) { > + struct page *page; > + pte_t old = *ptep; > + pte_t entry; > + > + prepare_vmemmap_page(huge); > + > + entry = mk_pte(huge++, PAGE_KERNEL); > + VM_WARN_ON(!pte_present(old)); > + page = pte_page(old); > + list_add(&page->lru, free_pages); > + > + set_pte_at(&init_mm, addr, ptep, entry); > + } > +} > + > +static void replace_huge_page_pmd_vmemmap(pmd_t *pmd, unsigned long start, > + struct page *huge, > + struct list_head *free_pages) > +{ > + unsigned long end = start + VMEMMAP_HPAGE_SIZE; > + > + flush_cache_vunmap(start, end); > + __replace_huge_page_pte_vmemmap(pte_offset_kernel(pmd, start), start, > + VMEMMAP_HPAGE_NR, huge, free_pages); > + flush_tlb_kernel_range(start, end); > +} > + > +static pte_t *merge_vmemmap_pte(pmd_t *pmdp, unsigned long addr) > +{ > + pte_t *pte; > + struct page *page; > + > + pte = pte_offset_kernel(pmdp, addr); > + page = pte_page(*pte); > + set_pmd(pmdp, vmemmap_pmd_mkhuge(page)); > + > + return pte; > +} > + > +static void merge_huge_page_pmd_vmemmap(pmd_t *pmd, unsigned long start, > + struct page *huge, > + struct list_head *free_pages) > +{ > + replace_huge_page_pmd_vmemmap(pmd, start, huge, free_pages); > + pte_free_kernel(&init_mm, merge_vmemmap_pte(pmd, start)); > + flush_tlb_kernel_range(start, start + VMEMMAP_HPAGE_SIZE); > +} > + > static inline void alloc_vmemmap_pages(struct hstate *h, struct list_head *list) > { > int i; > @@ -1772,6 +1828,15 @@ static inline void alloc_vmemmap_pages(struct hstate *h, struct list_head *list) > } > } > > +static inline void dissolve_compound_page(struct page *page, unsigned int order) > +{ > + int i; > + unsigned int nr_pages = 1 << order; > + > + for (i = 1; i < nr_pages; i++) > + set_page_refcounted(page + i); > +} > + > static void alloc_huge_page_vmemmap(struct hstate *h, struct page *head) > { > pmd_t *pmd; > @@ -1791,10 +1856,45 @@ static void alloc_huge_page_vmemmap(struct hstate *h, struct page *head) > __remap_huge_page_pte_vmemmap); > if (!freed_vmemmap_hpage_dec(pmd_page(*pmd)) && pmd_split(pmd)) { > /* > - * Todo: > - * Merge pte to huge pmd if it has ever been split. > + * Merge pte to huge pmd if it has ever been split. Now only > + * support gigantic page which's vmemmap pages size is an > + * integer multiple of PMD_SIZE. This is the simplest case > + * to handle. > */ > clear_pmd_split(pmd); > + > + if (IS_ALIGNED(nr_vmemmap(h), VMEMMAP_HPAGE_NR)) { > + unsigned long addr = (unsigned long)head; > + unsigned long end = addr + nr_vmemmap_size(h); > + > + spin_unlock(ptl); > + > + for (; addr < end; addr += VMEMMAP_HPAGE_SIZE) { > + void *to; > + struct page *page; > + > + page = alloc_pages(GFP_VMEMMAP_PAGE & ~__GFP_NOFAIL, > + VMEMMAP_HPAGE_ORDER); > + if (!page) > + goto out; Here forget to call dissolve_compound_page(). + dissolve_compound_page(page, + VMEMMAP_HPAGE_ORDER); > + > + to = page_to_virt(page); > + memcpy(to, (void *)addr, VMEMMAP_HPAGE_SIZE); > + > + /* > + * Make sure that any data that writes to the > + * @to is made visible to the physical page. > + */ > + flush_kernel_vmap_range(to, VMEMMAP_HPAGE_SIZE); > + > + merge_huge_page_pmd_vmemmap(pmd++, addr, page, > + &remap_pages); > + } > + > +out: > + free_vmemmap_page_list(&remap_pages); > + return; > + } > } > spin_unlock(ptl); > } > -- > 2.20.1 > -- Yours, Muchun From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 628BEC43468 for ; Sun, 20 Sep 2020 09:59:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A662A23718 for ; Sun, 20 Sep 2020 09:59:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="uf2OGboF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A662A23718 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B82FA6B006E; Sun, 20 Sep 2020 05:59:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B34D86B0070; Sun, 20 Sep 2020 05:59:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A24276B0071; Sun, 20 Sep 2020 05:59:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0068.hostedemail.com [216.40.44.68]) by kanga.kvack.org (Postfix) with ESMTP id 811926B006E for ; Sun, 20 Sep 2020 05:59:40 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 2CED6181AEF10 for ; Sun, 20 Sep 2020 09:59:40 +0000 (UTC) X-FDA: 77282992920.16.music25_0f090d22713b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 10402100E5495 for ; Sun, 20 Sep 2020 09:59:40 +0000 (UTC) X-HE-Tag: music25_0f090d22713b X-Filterd-Recvd-Size: 9914 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Sun, 20 Sep 2020 09:59:39 +0000 (UTC) Received: by mail-pl1-f195.google.com with SMTP id f1so5348861plo.13 for ; Sun, 20 Sep 2020 02:59:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=bsQK8LpuxrhLjsMMeuAwU+LPx3UZkeZ5l4b7tYvmwRw=; b=uf2OGboFLy/L86Po2Es5Zv7ilwDEHjiD68HdWMnC+2TTVeQq6+3TpQ8S4hx54ztRz6 byRNGnVskkzKHg8WColeN7JdIRLxsd260424uAjNg9dOsUdX7kTpbuuiWjtlFCdyXxxB d6Ubn2mPsUlkDVPO/GCNxHunwKCXPScicvQszzSzCGq2eusF2u40NN0hrA9iqGzwzBvv oAM9YZQQn1IcETFhLtCe6kJ90ZtQoanr/NjfI2jG1VGZLhviiFKwbdZuJZbY3VsavHer xv1SnI9UsEv6Mwwu8/vVj4sP5FTPPSnySzsOPSPXrzB8ga5sgjTi1RXltGX/o1IXvm50 RBzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bsQK8LpuxrhLjsMMeuAwU+LPx3UZkeZ5l4b7tYvmwRw=; b=iV+itElJknuSPuinAFaT0RowVL1I0YOBsh2SBsGLnN5PcIOXz2Ol+/DXywzgr6NbH/ j/pBz+VuepAHNxUIv2XmfCHgFeZS/MN79h7I+3Lrj7WEsJYZxMR9R3LdMdStiEpedzqN 3P+3WXA3KUUAwzktRj+/dXyHmxO1VS1ioa/9p5N09mvgmzaCOA5t3xkikP+AvW0AZ9+z BlZcG+DecFbZjESjJekqqbxLIFxw8KCHL7s+lnz8m/1MRNv+/a5u+Weyc0zscs0o6uOS cEsegieCxfLdammMEg7whJKXxAxqZKQnEzhzb5Bh8aEpjgs9AO0/hkoKHOysF7g8BzmV g93w== X-Gm-Message-State: AOAM532mswpWTbAJhgup9rE/BbXwSVFgPOqT6lWclaDxZvvcP6LAS+bF XnMbybYy4DZHOuXl6E/71x6Rco4a9sjwH3nLQT4NrA== X-Google-Smtp-Source: ABdhPJyLB6JeulC3J3qt+PdRVgQHD1bgo33KeQ7Z9TGvBKZsB/ZmYNsfj5G5KTW1QcP1SkVeA5KsxqzEv001KyERvQA= X-Received: by 2002:a17:90a:bc8d:: with SMTP id x13mr20950424pjr.229.1600595978266; Sun, 20 Sep 2020 02:59:38 -0700 (PDT) MIME-Version: 1.0 References: <20200915125947.26204-1-songmuchun@bytedance.com> <20200915125947.26204-22-songmuchun@bytedance.com> In-Reply-To: <20200915125947.26204-22-songmuchun@bytedance.com> From: Muchun Song Date: Sun, 20 Sep 2020 17:59:01 +0800 Message-ID: Subject: Re: [RFC PATCH 21/24] mm/hugetlb: Merge pte to huge pmd only for gigantic page To: Jonathan Corbet , Mike Kravetz , Thomas Gleixner , mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, Peter Zijlstra , viro@zeniv.linux.org.uk, Andrew Morton , paulmck@kernel.org, mchehab+huawei@kernel.org, pawan.kumar.gupta@linux.intel.com, Randy Dunlap , oneukum@suse.com, anshuman.khandual@arm.com, jroedel@suse.de, Mina Almasry , David Rientjes Cc: linux-doc@vger.kernel.org, LKML , Linux Memory Management List , linux-fsdevel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Sep 15, 2020 at 9:03 PM Muchun Song wrote: > > Merge pte to huge pmd if it has ever been split. Now only support > gigantic page which's vmemmap pages size is an integer multiple of > PMD_SIZE. This is the simplest case to handle. > > Signed-off-by: Muchun Song > --- > include/linux/hugetlb.h | 7 +++ > mm/hugetlb.c | 104 +++++++++++++++++++++++++++++++++++++++- > 2 files changed, 109 insertions(+), 2 deletions(-) > > diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h > index e3aa192f1c39..c56df0da7ae5 100644 > --- a/include/linux/hugetlb.h > +++ b/include/linux/hugetlb.h > @@ -611,6 +611,13 @@ static inline bool vmemmap_pmd_huge(pmd_t *pmd) > } > #endif > > +#ifndef vmemmap_pmd_mkhuge > +static inline pmd_t vmemmap_pmd_mkhuge(struct page *page) > +{ > + return pmd_mkhuge(mk_pmd(page, PAGE_KERNEL)); > +} > +#endif > + > #ifndef VMEMMAP_HPAGE_SHIFT > #define VMEMMAP_HPAGE_SHIFT PMD_SHIFT > #endif > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index 28c154679838..3ca36e259b4e 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -1759,6 +1759,62 @@ static void __remap_huge_page_pte_vmemmap(struct page *reuse, pte_t *ptep, > } > } > > +static void __replace_huge_page_pte_vmemmap(pte_t *ptep, unsigned long start, > + unsigned int nr, struct page *huge, > + struct list_head *free_pages) > +{ > + unsigned long addr; > + unsigned long end = start + (nr << PAGE_SHIFT); > + > + for (addr = start; addr < end; addr += PAGE_SIZE, ptep++) { > + struct page *page; > + pte_t old = *ptep; > + pte_t entry; > + > + prepare_vmemmap_page(huge); > + > + entry = mk_pte(huge++, PAGE_KERNEL); > + VM_WARN_ON(!pte_present(old)); > + page = pte_page(old); > + list_add(&page->lru, free_pages); > + > + set_pte_at(&init_mm, addr, ptep, entry); > + } > +} > + > +static void replace_huge_page_pmd_vmemmap(pmd_t *pmd, unsigned long start, > + struct page *huge, > + struct list_head *free_pages) > +{ > + unsigned long end = start + VMEMMAP_HPAGE_SIZE; > + > + flush_cache_vunmap(start, end); > + __replace_huge_page_pte_vmemmap(pte_offset_kernel(pmd, start), start, > + VMEMMAP_HPAGE_NR, huge, free_pages); > + flush_tlb_kernel_range(start, end); > +} > + > +static pte_t *merge_vmemmap_pte(pmd_t *pmdp, unsigned long addr) > +{ > + pte_t *pte; > + struct page *page; > + > + pte = pte_offset_kernel(pmdp, addr); > + page = pte_page(*pte); > + set_pmd(pmdp, vmemmap_pmd_mkhuge(page)); > + > + return pte; > +} > + > +static void merge_huge_page_pmd_vmemmap(pmd_t *pmd, unsigned long start, > + struct page *huge, > + struct list_head *free_pages) > +{ > + replace_huge_page_pmd_vmemmap(pmd, start, huge, free_pages); > + pte_free_kernel(&init_mm, merge_vmemmap_pte(pmd, start)); > + flush_tlb_kernel_range(start, start + VMEMMAP_HPAGE_SIZE); > +} > + > static inline void alloc_vmemmap_pages(struct hstate *h, struct list_head *list) > { > int i; > @@ -1772,6 +1828,15 @@ static inline void alloc_vmemmap_pages(struct hstate *h, struct list_head *list) > } > } > > +static inline void dissolve_compound_page(struct page *page, unsigned int order) > +{ > + int i; > + unsigned int nr_pages = 1 << order; > + > + for (i = 1; i < nr_pages; i++) > + set_page_refcounted(page + i); > +} > + > static void alloc_huge_page_vmemmap(struct hstate *h, struct page *head) > { > pmd_t *pmd; > @@ -1791,10 +1856,45 @@ static void alloc_huge_page_vmemmap(struct hstate *h, struct page *head) > __remap_huge_page_pte_vmemmap); > if (!freed_vmemmap_hpage_dec(pmd_page(*pmd)) && pmd_split(pmd)) { > /* > - * Todo: > - * Merge pte to huge pmd if it has ever been split. > + * Merge pte to huge pmd if it has ever been split. Now only > + * support gigantic page which's vmemmap pages size is an > + * integer multiple of PMD_SIZE. This is the simplest case > + * to handle. > */ > clear_pmd_split(pmd); > + > + if (IS_ALIGNED(nr_vmemmap(h), VMEMMAP_HPAGE_NR)) { > + unsigned long addr = (unsigned long)head; > + unsigned long end = addr + nr_vmemmap_size(h); > + > + spin_unlock(ptl); > + > + for (; addr < end; addr += VMEMMAP_HPAGE_SIZE) { > + void *to; > + struct page *page; > + > + page = alloc_pages(GFP_VMEMMAP_PAGE & ~__GFP_NOFAIL, > + VMEMMAP_HPAGE_ORDER); > + if (!page) > + goto out; Here forget to call dissolve_compound_page(). + dissolve_compound_page(page, + VMEMMAP_HPAGE_ORDER); > + > + to = page_to_virt(page); > + memcpy(to, (void *)addr, VMEMMAP_HPAGE_SIZE); > + > + /* > + * Make sure that any data that writes to the > + * @to is made visible to the physical page. > + */ > + flush_kernel_vmap_range(to, VMEMMAP_HPAGE_SIZE); > + > + merge_huge_page_pmd_vmemmap(pmd++, addr, page, > + &remap_pages); > + } > + > +out: > + free_vmemmap_page_list(&remap_pages); > + return; > + } > } > spin_unlock(ptl); > } > -- > 2.20.1 > -- Yours, Muchun