From: Muchun Song <songmuchun@bytedance.com>
To: David Hildenbrand <david@redhat.com>
Cc: "David Rientjes" <rientjes@google.com>,
"Jonathan Corbet" <corbet@lwn.net>,
"Mike Kravetz" <mike.kravetz@oracle.com>,
"Thomas Gleixner" <tglx@linutronix.de>,
mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com,
dave.hansen@linux.intel.com, luto@kernel.org,
"Peter Zijlstra" <peterz@infradead.org>,
viro@zeniv.linux.org.uk,
"Andrew Morton" <akpm@linux-foundation.org>,
paulmck@kernel.org, mchehab+huawei@kernel.org,
pawan.kumar.gupta@linux.intel.com,
"Randy Dunlap" <rdunlap@infradead.org>,
oneukum@suse.com, anshuman.khandual@arm.com, jroedel@suse.de,
"Mina Almasry" <almasrymina@google.com>,
"Matthew Wilcox" <willy@infradead.org>,
"Oscar Salvador" <osalvador@suse.de>,
"Michal Hocko" <mhocko@suse.com>,
"Song Bao Hua (Barry Song)" <song.bao.hua@hisilicon.com>,
"HORIGUCHI NAOYA(堀口 直也)" <naoya.horiguchi@nec.com>,
"Xiongchun duan" <duanxiongchun@bytedance.com>,
linux-doc@vger.kernel.org, LKML <linux-kernel@vger.kernel.org>,
"Linux Memory Management List" <linux-mm@kvack.org>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: Re: [External] Re: [PATCH v13 05/12] mm: hugetlb: allocate the vmemmap pages associated with each HugeTLB page
Date: Mon, 25 Jan 2021 17:34:09 +0800 [thread overview]
Message-ID: <CAMZfGtWK=zBri_zAx=uP_dLv2Kh-2_vfjAyN7XtESwqukg5Eug@mail.gmail.com> (raw)
In-Reply-To: <552e8214-bc6f-8d90-0ed8-b3aff75d0e47@redhat.com>
On Mon, Jan 25, 2021 at 5:15 PM David Hildenbrand <david@redhat.com> wrote:
>
> On 25.01.21 08:41, Muchun Song wrote:
> > On Mon, Jan 25, 2021 at 2:40 PM Muchun Song <songmuchun@bytedance.com> wrote:
> >>
> >> On Mon, Jan 25, 2021 at 8:05 AM David Rientjes <rientjes@google.com> wrote:
> >>>
> >>>
> >>> On Sun, 17 Jan 2021, Muchun Song wrote:
> >>>
> >>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
> >>>> index ce4be1fa93c2..3b146d5949f3 100644
> >>>> --- a/mm/sparse-vmemmap.c
> >>>> +++ b/mm/sparse-vmemmap.c
> >>>> @@ -29,6 +29,7 @@
> >>>> #include <linux/sched.h>
> >>>> #include <linux/pgtable.h>
> >>>> #include <linux/bootmem_info.h>
> >>>> +#include <linux/delay.h>
> >>>>
> >>>> #include <asm/dma.h>
> >>>> #include <asm/pgalloc.h>
> >>>> @@ -40,7 +41,8 @@
> >>>> * @remap_pte: called for each non-empty PTE (lowest-level) entry.
> >>>> * @reuse_page: the page which is reused for the tail vmemmap pages.
> >>>> * @reuse_addr: the virtual address of the @reuse_page page.
> >>>> - * @vmemmap_pages: the list head of the vmemmap pages that can be freed.
> >>>> + * @vmemmap_pages: the list head of the vmemmap pages that can be freed
> >>>> + * or is mapped from.
> >>>> */
> >>>> struct vmemmap_remap_walk {
> >>>> void (*remap_pte)(pte_t *pte, unsigned long addr,
> >>>> @@ -50,6 +52,10 @@ struct vmemmap_remap_walk {
> >>>> struct list_head *vmemmap_pages;
> >>>> };
> >>>>
> >>>> +/* The gfp mask of allocating vmemmap page */
> >>>> +#define GFP_VMEMMAP_PAGE \
> >>>> + (GFP_KERNEL | __GFP_RETRY_MAYFAIL | __GFP_NOWARN | __GFP_THISNODE)
> >>>> +
> >>>
> >>> This is unnecessary, just use the gfp mask directly in allocator.
> >>
> >> Will do. Thanks.
> >>
> >>>
> >>>> static void vmemmap_pte_range(pmd_t *pmd, unsigned long addr,
> >>>> unsigned long end,
> >>>> struct vmemmap_remap_walk *walk)
> >>>> @@ -228,6 +234,75 @@ void vmemmap_remap_free(unsigned long start, unsigned long end,
> >>>> free_vmemmap_page_list(&vmemmap_pages);
> >>>> }
> >>>>
> >>>> +static void vmemmap_restore_pte(pte_t *pte, unsigned long addr,
> >>>> + struct vmemmap_remap_walk *walk)
> >>>> +{
> >>>> + pgprot_t pgprot = PAGE_KERNEL;
> >>>> + struct page *page;
> >>>> + void *to;
> >>>> +
> >>>> + BUG_ON(pte_page(*pte) != walk->reuse_page);
> >>>> +
> >>>> + page = list_first_entry(walk->vmemmap_pages, struct page, lru);
> >>>> + list_del(&page->lru);
> >>>> + to = page_to_virt(page);
> >>>> + copy_page(to, (void *)walk->reuse_addr);
> >>>> +
> >>>> + set_pte_at(&init_mm, addr, pte, mk_pte(page, pgprot));
> >>>> +}
> >>>> +
> >>>> +static void alloc_vmemmap_page_list(struct list_head *list,
> >>>> + unsigned long start, unsigned long end)
> >>>> +{
> >>>> + unsigned long addr;
> >>>> +
> >>>> + for (addr = start; addr < end; addr += PAGE_SIZE) {
> >>>> + struct page *page;
> >>>> + int nid = page_to_nid((const void *)addr);
> >>>> +
> >>>> +retry:
> >>>> + page = alloc_pages_node(nid, GFP_VMEMMAP_PAGE, 0);
> >>>> + if (unlikely(!page)) {
> >>>> + msleep(100);
> >>>> + /*
> >>>> + * We should retry infinitely, because we cannot
> >>>> + * handle allocation failures. Once we allocate
> >>>> + * vmemmap pages successfully, then we can free
> >>>> + * a HugeTLB page.
> >>>> + */
> >>>> + goto retry;
> >>>
> >>> Ugh, I don't think this will work, there's no guarantee that we'll ever
> >>> succeed and now we can't free a 2MB hugepage because we cannot allocate a
> >>> 4KB page. We absolutely have to ensure we make forward progress here.
> >>
> >> This can trigger a OOM when there is no memory and kill someone to release
> >> some memory. Right?
> >>
> >>>
> >>> We're going to be freeing the hugetlb page after this succeeeds, can we
> >>> not use part of the hugetlb page that we're freeing for this memory
> >>> instead?
> >>
> >> It seems a good idea. We can try to allocate memory firstly, if successful,
> >> just use the new page to remap (it can reduce memory fragmentation).
> >> If not, we can use part of the hugetlb page to remap. What's your opinion
> >> about this?
> >
> > If the HugeTLB page is a gigantic page which is allocated from
> > CMA. In this case, we cannot use part of the hugetlb page to remap.
> > Right?
>
> Right; and I don't think the "reuse part of a huge page as vmemmap while
> freeing, while that part itself might not have a proper vmemmap yet (or
> might cover itself now)" is particularly straight forward. Maybe I'm
> wrong :)
>
> Also, watch out for huge pages on ZONE_MOVABLE, in that case you also
> shouldn't allocate the vmemmap from there ...
Yeah, you are right. So I tend to trigger OOM to kill other processes to
reclaim some memory when we allocate memory fails.
>
> --
> Thanks,
>
> David / dhildenb
>
next prev parent reply other threads:[~2021-01-25 9:34 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-17 15:10 [PATCH v13 00/12] Free some vmemmap pages of HugeTLB page Muchun Song
2021-01-17 15:10 ` [PATCH v13 01/12] mm: memory_hotplug: factor out bootmem core functions to bootmem_info.c Muchun Song
2021-01-25 2:48 ` Miaohe Lin
2021-01-17 15:10 ` [PATCH v13 02/12] mm: hugetlb: introduce a new config HUGETLB_PAGE_FREE_VMEMMAP Muchun Song
2021-01-24 23:58 ` David Rientjes
2021-01-25 3:16 ` Randy Dunlap
2021-01-25 4:06 ` [External] " Muchun Song
2021-01-25 4:08 ` Randy Dunlap
2021-01-25 5:06 ` Muchun Song
2021-01-25 18:47 ` David Rientjes
2021-01-26 2:45 ` Muchun Song
2021-01-26 20:13 ` David Rientjes
2021-01-26 2:07 ` Miaohe Lin
2021-01-17 15:10 ` [PATCH v13 03/12] mm: hugetlb: free the vmemmap pages associated with each HugeTLB page Muchun Song
2021-01-17 15:29 ` Muchun Song
2021-01-23 0:59 ` Mike Kravetz
2021-01-23 3:22 ` [External] " Muchun Song
2021-01-23 17:52 ` Oscar Salvador
2021-01-24 6:48 ` [External] " Muchun Song
2021-01-17 15:10 ` [PATCH v13 04/12] mm: hugetlb: defer freeing of HugeTLB pages Muchun Song
2021-01-24 23:55 ` David Rientjes
2021-01-25 3:58 ` [External] " Muchun Song
2021-01-17 15:10 ` [PATCH v13 05/12] mm: hugetlb: allocate the vmemmap pages associated with each HugeTLB page Muchun Song
2021-01-25 0:05 ` David Rientjes
2021-01-25 6:40 ` [External] " Muchun Song
2021-01-25 7:41 ` Muchun Song
2021-01-25 9:15 ` David Hildenbrand
2021-01-25 9:34 ` Muchun Song [this message]
2021-01-25 23:25 ` Mike Kravetz
2021-01-26 7:48 ` Oscar Salvador
2021-01-26 9:29 ` Oscar Salvador
2021-01-26 9:36 ` David Hildenbrand
2021-01-26 14:58 ` Oscar Salvador
2021-01-26 15:10 ` David Hildenbrand
2021-01-26 15:34 ` Oscar Salvador
2021-01-26 15:56 ` David Hildenbrand
2021-01-27 10:36 ` David Hildenbrand
2021-01-28 12:37 ` [External] " Muchun Song
2021-01-28 13:08 ` Muchun Song
2021-01-29 1:04 ` Mike Kravetz
2021-01-29 6:56 ` Muchun Song
2021-02-01 16:10 ` David Hildenbrand
2021-02-02 0:05 ` Mike Kravetz
2021-01-28 22:29 ` Oscar Salvador
2021-01-29 6:16 ` [External] " Muchun Song
2021-02-01 15:50 ` David Hildenbrand
2021-01-17 15:10 ` [PATCH v13 06/12] mm: hugetlb: set the PageHWPoison to the raw error page Muchun Song
2021-01-25 0:06 ` David Rientjes
2021-01-25 5:06 ` [External] " Muchun Song
2021-01-17 15:10 ` [PATCH v13 07/12] mm: hugetlb: flush work when dissolving a HugeTLB page Muchun Song
2021-01-17 15:10 ` [PATCH v13 08/12] mm: hugetlb: introduce PageHugeInflight Muchun Song
2021-01-17 15:10 ` [PATCH v13 09/12] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap Muchun Song
2021-01-25 11:43 ` David Hildenbrand
2021-01-25 12:08 ` Oscar Salvador
2021-01-25 12:31 ` [External] " Muchun Song
2021-01-25 12:30 ` Muchun Song
2021-01-17 15:10 ` [PATCH v13 10/12] mm: hugetlb: introduce nr_free_vmemmap_pages in the struct hstate Muchun Song
2021-01-17 15:10 ` [PATCH v13 11/12] mm: hugetlb: gather discrete indexes of tail page Muchun Song
2021-01-17 15:10 ` [PATCH v13 12/12] mm: hugetlb: optimize the code with the help of the compiler Muchun Song
2021-01-20 12:52 ` [PATCH v13 00/12] Free some vmemmap pages of HugeTLB page Muchun Song
2021-01-20 13:10 ` Oscar Salvador
2021-01-20 14:22 ` [External] " Muchun Song
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAMZfGtWK=zBri_zAx=uP_dLv2Kh-2_vfjAyN7XtESwqukg5Eug@mail.gmail.com' \
--to=songmuchun@bytedance.com \
--cc=akpm@linux-foundation.org \
--cc=almasrymina@google.com \
--cc=anshuman.khandual@arm.com \
--cc=bp@alien8.de \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=david@redhat.com \
--cc=duanxiongchun@bytedance.com \
--cc=hpa@zytor.com \
--cc=jroedel@suse.de \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=luto@kernel.org \
--cc=mchehab+huawei@kernel.org \
--cc=mhocko@suse.com \
--cc=mike.kravetz@oracle.com \
--cc=mingo@redhat.com \
--cc=naoya.horiguchi@nec.com \
--cc=oneukum@suse.com \
--cc=osalvador@suse.de \
--cc=paulmck@kernel.org \
--cc=pawan.kumar.gupta@linux.intel.com \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=rientjes@google.com \
--cc=song.bao.hua@hisilicon.com \
--cc=tglx@linutronix.de \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).