* Re: [PATCH v19 7/8] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap [not found] ` <20210315092015.35396-8-songmuchun@bytedance.com> @ 2021-03-19 8:59 ` Oscar Salvador 2021-03-19 12:15 ` [External] " Muchun Song 0 siblings, 1 reply; 4+ messages in thread From: Oscar Salvador @ 2021-03-19 8:59 UTC (permalink / raw) To: Muchun Song Cc: corbet, mike.kravetz, tglx, mingo, bp, x86, hpa, dave.hansen, luto, peterz, viro, akpm, paulmck, mchehab+huawei, pawan.kumar.gupta, rdunlap, oneukum, anshuman.khandual, jroedel, almasrymina, rientjes, willy, mhocko, song.bao.hua, david, naoya.horiguchi, joao.m.martins, duanxiongchun, linux-doc, linux-kernel, linux-mm, linux-fsdevel, Miaohe Lin, Chen Huang, Bodeddula Balasubramaniam On Mon, Mar 15, 2021 at 05:20:14PM +0800, Muchun Song wrote: > --- a/arch/x86/mm/init_64.c > +++ b/arch/x86/mm/init_64.c > @@ -34,6 +34,7 @@ > #include <linux/gfp.h> > #include <linux/kcore.h> > #include <linux/bootmem_info.h> > +#include <linux/hugetlb.h> > > #include <asm/processor.h> > #include <asm/bios_ebda.h> > @@ -1557,7 +1558,8 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, > { > int err; > > - if (end - start < PAGES_PER_SECTION * sizeof(struct page)) > + if ((is_hugetlb_free_vmemmap_enabled() && !altmap) || > + end - start < PAGES_PER_SECTION * sizeof(struct page)) > err = vmemmap_populate_basepages(start, end, node, NULL); > else if (boot_cpu_has(X86_FEATURE_PSE)) > err = vmemmap_populate_hugepages(start, end, node, altmap); I've been thinking about this some more. Assume you opt-in the hugetlb-vmemmap feature, and assume you pass a valid altmap to vmemmap_populate. This will lead to use populating the vmemmap array with hugepages. What if then, a HugeTLB gets allocated and falls within that memory range (backed by hugetpages)? AFAIK, this will get us in trouble as currently the code can only operate on memory backed by PAGE_SIZE pages, right? I cannot remember, but I do not think nothing prevents that from happening? Am I missing anything? -- Oscar Salvador SUSE L3 ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [External] Re: [PATCH v19 7/8] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap 2021-03-19 8:59 ` [PATCH v19 7/8] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap Oscar Salvador @ 2021-03-19 12:15 ` Muchun Song 2021-03-19 12:36 ` David Hildenbrand 0 siblings, 1 reply; 4+ messages in thread From: Muchun Song @ 2021-03-19 12:15 UTC (permalink / raw) To: Oscar Salvador, David Hildenbrand Cc: Jonathan Corbet, Mike Kravetz, Thomas Gleixner, Ingo Molnar, bp, X86 ML, hpa, dave.hansen, luto, Peter Zijlstra, Alexander Viro, Andrew Morton, paulmck, mchehab+huawei, pawan.kumar.gupta, Randy Dunlap, oneukum, anshuman.khandual, jroedel, Mina Almasry, David Rientjes, Matthew Wilcox, Michal Hocko, Song Bao Hua (Barry Song), HORIGUCHI NAOYA(堀口 直也), Joao Martins, Xiongchun duan, linux-doc, LKML, Linux Memory Management List, linux-fsdevel, Miaohe Lin, Chen Huang, Bodeddula Balasubramaniam On Fri, Mar 19, 2021 at 4:59 PM Oscar Salvador <osalvador@suse.de> wrote: > > On Mon, Mar 15, 2021 at 05:20:14PM +0800, Muchun Song wrote: > > --- a/arch/x86/mm/init_64.c > > +++ b/arch/x86/mm/init_64.c > > @@ -34,6 +34,7 @@ > > #include <linux/gfp.h> > > #include <linux/kcore.h> > > #include <linux/bootmem_info.h> > > +#include <linux/hugetlb.h> > > > > #include <asm/processor.h> > > #include <asm/bios_ebda.h> > > @@ -1557,7 +1558,8 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, > > { > > int err; > > > > - if (end - start < PAGES_PER_SECTION * sizeof(struct page)) > > + if ((is_hugetlb_free_vmemmap_enabled() && !altmap) || > > + end - start < PAGES_PER_SECTION * sizeof(struct page)) > > err = vmemmap_populate_basepages(start, end, node, NULL); > > else if (boot_cpu_has(X86_FEATURE_PSE)) > > err = vmemmap_populate_hugepages(start, end, node, altmap); > > I've been thinking about this some more. > > Assume you opt-in the hugetlb-vmemmap feature, and assume you pass a valid altmap > to vmemmap_populate. > This will lead to use populating the vmemmap array with hugepages. Right. > > What if then, a HugeTLB gets allocated and falls within that memory range (backed > by hugetpages)? I am not sure whether we can allocate the HugeTLB pages from there. Will only device memory pass a valid altmap parameter to vmemmap_populate()? If yes, can we allocate HugeTLB pages from device memory? Sorry, I am not an expert on this. > AFAIK, this will get us in trouble as currently the code can only operate on memory > backed by PAGE_SIZE pages, right? > > I cannot remember, but I do not think nothing prevents that from happening? > Am I missing anything? Maybe David H is more familiar with this. Hi David, Do you have some suggestions on this? Thanks. > > -- > Oscar Salvador > SUSE L3 ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [External] Re: [PATCH v19 7/8] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap 2021-03-19 12:15 ` [External] " Muchun Song @ 2021-03-19 12:36 ` David Hildenbrand 2021-03-19 12:42 ` David Hildenbrand 0 siblings, 1 reply; 4+ messages in thread From: David Hildenbrand @ 2021-03-19 12:36 UTC (permalink / raw) To: Muchun Song, Oscar Salvador Cc: Jonathan Corbet, Mike Kravetz, Thomas Gleixner, Ingo Molnar, bp, X86 ML, hpa, dave.hansen, luto, Peter Zijlstra, Alexander Viro, Andrew Morton, paulmck, mchehab+huawei, pawan.kumar.gupta, Randy Dunlap, oneukum, anshuman.khandual, jroedel, Mina Almasry, David Rientjes, Matthew Wilcox, Michal Hocko, Song Bao Hua (Barry Song), HORIGUCHI NAOYA(堀口 直也), Joao Martins, Xiongchun duan, linux-doc, LKML, Linux Memory Management List, linux-fsdevel, Miaohe Lin, Chen Huang, Bodeddula Balasubramaniam On 19.03.21 13:15, Muchun Song wrote: > On Fri, Mar 19, 2021 at 4:59 PM Oscar Salvador <osalvador@suse.de> wrote: >> >> On Mon, Mar 15, 2021 at 05:20:14PM +0800, Muchun Song wrote: >>> --- a/arch/x86/mm/init_64.c >>> +++ b/arch/x86/mm/init_64.c >>> @@ -34,6 +34,7 @@ >>> #include <linux/gfp.h> >>> #include <linux/kcore.h> >>> #include <linux/bootmem_info.h> >>> +#include <linux/hugetlb.h> >>> >>> #include <asm/processor.h> >>> #include <asm/bios_ebda.h> >>> @@ -1557,7 +1558,8 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, >>> { >>> int err; >>> >>> - if (end - start < PAGES_PER_SECTION * sizeof(struct page)) >>> + if ((is_hugetlb_free_vmemmap_enabled() && !altmap) || >>> + end - start < PAGES_PER_SECTION * sizeof(struct page)) >>> err = vmemmap_populate_basepages(start, end, node, NULL); >>> else if (boot_cpu_has(X86_FEATURE_PSE)) >>> err = vmemmap_populate_hugepages(start, end, node, altmap); >> >> I've been thinking about this some more. >> >> Assume you opt-in the hugetlb-vmemmap feature, and assume you pass a valid altmap >> to vmemmap_populate. >> This will lead to use populating the vmemmap array with hugepages. > > Right. > >> >> What if then, a HugeTLB gets allocated and falls within that memory range (backed >> by hugetpages)? > > I am not sure whether we can allocate the HugeTLB pages from there. > Will only device memory pass a valid altmap parameter to > vmemmap_populate()? If yes, can we allocate HugeTLB pages from > device memory? Sorry, I am not an expert on this. I think, right now, yes. System RAM that's applicable for HugePages never uses an altmap. But Oscar's patch will change that, maybe before your series might get included from what I've been reading. [1] [1] https://lkml.kernel.org/r/20210319092635.6214-1-osalvador@suse.de > > >> AFAIK, this will get us in trouble as currently the code can only operate on memory >> backed by PAGE_SIZE pages, right? >> >> I cannot remember, but I do not think nothing prevents that from happening? >> Am I missing anything? > > Maybe David H is more familiar with this. > > Hi David, > > Do you have some suggestions on this? There has to be some way to identify whether we can optimize specific vmemmap pages or should just leave them alone. altmap vs. !altmap. Unfortunately, there is no easy way to detect that - e.g., PageReserved() applies also to boot memory. We could go back to setting a special PageType for these vmemmap pages, indicating "this is a page allocated from an altmap, don't touch it". -- Thanks, David / dhildenb ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [External] Re: [PATCH v19 7/8] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap 2021-03-19 12:36 ` David Hildenbrand @ 2021-03-19 12:42 ` David Hildenbrand 0 siblings, 0 replies; 4+ messages in thread From: David Hildenbrand @ 2021-03-19 12:42 UTC (permalink / raw) To: Muchun Song, Oscar Salvador Cc: Jonathan Corbet, Mike Kravetz, Thomas Gleixner, Ingo Molnar, bp, X86 ML, hpa, dave.hansen, luto, Peter Zijlstra, Alexander Viro, Andrew Morton, paulmck, mchehab+huawei, pawan.kumar.gupta, Randy Dunlap, oneukum, anshuman.khandual, jroedel, Mina Almasry, David Rientjes, Matthew Wilcox, Michal Hocko, Song Bao Hua (Barry Song), HORIGUCHI NAOYA(堀口 直也), Joao Martins, Xiongchun duan, linux-doc, LKML, Linux Memory Management List, linux-fsdevel, Miaohe Lin, Chen Huang, Bodeddula Balasubramaniam On 19.03.21 13:36, David Hildenbrand wrote: > On 19.03.21 13:15, Muchun Song wrote: >> On Fri, Mar 19, 2021 at 4:59 PM Oscar Salvador <osalvador@suse.de> wrote: >>> >>> On Mon, Mar 15, 2021 at 05:20:14PM +0800, Muchun Song wrote: >>>> --- a/arch/x86/mm/init_64.c >>>> +++ b/arch/x86/mm/init_64.c >>>> @@ -34,6 +34,7 @@ >>>> #include <linux/gfp.h> >>>> #include <linux/kcore.h> >>>> #include <linux/bootmem_info.h> >>>> +#include <linux/hugetlb.h> >>>> >>>> #include <asm/processor.h> >>>> #include <asm/bios_ebda.h> >>>> @@ -1557,7 +1558,8 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, >>>> { >>>> int err; >>>> >>>> - if (end - start < PAGES_PER_SECTION * sizeof(struct page)) >>>> + if ((is_hugetlb_free_vmemmap_enabled() && !altmap) || >>>> + end - start < PAGES_PER_SECTION * sizeof(struct page)) >>>> err = vmemmap_populate_basepages(start, end, node, NULL); >>>> else if (boot_cpu_has(X86_FEATURE_PSE)) >>>> err = vmemmap_populate_hugepages(start, end, node, altmap); >>> >>> I've been thinking about this some more. >>> >>> Assume you opt-in the hugetlb-vmemmap feature, and assume you pass a valid altmap >>> to vmemmap_populate. >>> This will lead to use populating the vmemmap array with hugepages. >> >> Right. >> >>> >>> What if then, a HugeTLB gets allocated and falls within that memory range (backed >>> by hugetpages)? >> >> I am not sure whether we can allocate the HugeTLB pages from there. >> Will only device memory pass a valid altmap parameter to >> vmemmap_populate()? If yes, can we allocate HugeTLB pages from >> device memory? Sorry, I am not an expert on this. > > I think, right now, yes. System RAM that's applicable for HugePages > never uses an altmap. But Oscar's patch will change that, maybe before > your series might get included from what I've been reading. [1] > > [1] https://lkml.kernel.org/r/20210319092635.6214-1-osalvador@suse.de > >> >> >>> AFAIK, this will get us in trouble as currently the code can only operate on memory >>> backed by PAGE_SIZE pages, right? >>> >>> I cannot remember, but I do not think nothing prevents that from happening? >>> Am I missing anything? >> >> Maybe David H is more familiar with this. >> >> Hi David, >> >> Do you have some suggestions on this? > > There has to be some way to identify whether we can optimize specific > vmemmap pages or should just leave them alone. altmap vs. !altmap. > > Unfortunately, there is no easy way to detect that - e.g., > PageReserved() applies also to boot memory. > > We could go back to setting a special PageType for these vmemmap pages, > indicating "this is a page allocated from an altmap, don't touch it". > With SPARSEMEM we can use PageReserved(page) && early_section(): vmemmap from bootmem PageReserved(page) && !early_section(): vmemmap from altmap !PageReserved(page): vmemmap from buddy But it's a bit shaky :) -- Thanks, David / dhildenb ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-03-19 12:42 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <20210315092015.35396-1-songmuchun@bytedance.com> [not found] ` <20210315092015.35396-8-songmuchun@bytedance.com> 2021-03-19 8:59 ` [PATCH v19 7/8] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap Oscar Salvador 2021-03-19 12:15 ` [External] " Muchun Song 2021-03-19 12:36 ` David Hildenbrand 2021-03-19 12:42 ` David Hildenbrand
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).