[PATCH 0/3] Add support for free vmemmap pages of HugeTLB for arm64

[PATCH 0/3] Add support for free vmemmap pages of HugeTLB for arm64

[Muchun Song]

This patchset is based on the series of "Free some vmemmap pages of HugeTLB
page". More details can refer to the below link.

  https://lkml.kernel.org/r/20210308102807.59745-1-songmuchun@bytedance.com

I often received some feedback (We want to test this feature on arm64) before.
Because the previous code has been reviewed for 18 versions and is merged
into mm tree, I think that it is time to release this patchset. If you want
to test then you can start now :-). And I also hope someone can review this.

Thanks.

Muchun Song (3):
  mm: bootmem_info: mark register_page_bootmem_info_section __init
  mm: hugetlb: introduce arch_free_vmemmap_page
  arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB

 arch/arm64/mm/mmu.c   | 5 +++++
 arch/x86/mm/init_64.c | 5 +++++
 fs/Kconfig            | 4 ++--
 mm/bootmem_info.c     | 4 ++--
 mm/sparse-vmemmap.c   | 9 +++++++--
 5 files changed, 21 insertions(+), 6 deletions(-)

-- 
2.11.0

[PATCH 1/3] mm: bootmem_info: mark register_page_bootmem_info_section __init

[Muchun Song]

The register_page_bootmem_info_section() is only called from __init
functions, so mark it __init as well.

Signed-off-by: Muchun Song <songmuchun@bytedance.com>
---
 mm/bootmem_info.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/bootmem_info.c b/mm/bootmem_info.c
index 5b152dba7344..f03f42f426f6 100644
--- a/mm/bootmem_info.c
+++ b/mm/bootmem_info.c
@@ -39,7 +39,7 @@ void put_page_bootmem(struct page *page)
 }
 
 #ifndef CONFIG_SPARSEMEM_VMEMMAP
-static void register_page_bootmem_info_section(unsigned long start_pfn)
+static void __init register_page_bootmem_info_section(unsigned long start_pfn)
 {
 	unsigned long mapsize, section_nr, i;
 	struct mem_section *ms;
@@ -74,7 +74,7 @@ static void register_page_bootmem_info_section(unsigned long start_pfn)
 
 }
 #else /* CONFIG_SPARSEMEM_VMEMMAP */
-static void register_page_bootmem_info_section(unsigned long start_pfn)
+static void __init register_page_bootmem_info_section(unsigned long start_pfn)
 {
 	unsigned long mapsize, section_nr, i;
 	struct mem_section *ms;
-- 
2.11.0

[PATCH 2/3] mm: hugetlb: introduce arch_free_vmemmap_page

[Muchun Song]

We register bootmem info for vmemmap pages when boot on x86-64, so
the vmemmap pages must be freed by using free_bootmem_page(). But
on some other architectures, we do not need bootmem info. In this
case, free_reserved_page() is enough to free vmemmap pages.
Currently, only x86-64 need free_bootmem_page(), so introduce a
default arch_free_vmemmap_page() which use free_reserved_page()
to free vmemmap pages directly. On x86-64, we can implement
arch_free_vmemmap_page() to override the default behavior.

Signed-off-by: Muchun Song <songmuchun@bytedance.com>
---
 arch/x86/mm/init_64.c | 5 +++++
 mm/sparse-vmemmap.c   | 9 +++++++--
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index 39f88c5faadc..732609dad0ec 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -1575,6 +1575,11 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
 }
 
 #ifdef CONFIG_HAVE_BOOTMEM_INFO_NODE
+void arch_free_vmemmap_page(struct page *page)
+{
+	free_bootmem_page(page);
+}
+
 void register_page_bootmem_memmap(unsigned long section_nr,
 				  struct page *start_page, unsigned long nr_pages)
 {
diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
index 60fc6cd6cd23..76f7b52820e3 100644
--- a/mm/sparse-vmemmap.c
+++ b/mm/sparse-vmemmap.c
@@ -156,16 +156,21 @@ static void vmemmap_remap_range(unsigned long start, unsigned long end,
 	flush_tlb_kernel_range(start + PAGE_SIZE, end);
 }
 
+void __weak arch_free_vmemmap_page(struct page *page)
+{
+	free_reserved_page(page);
+}
+
 /*
  * Free a vmemmap page. A vmemmap page can be allocated from the memblock
  * allocator or buddy allocator. If the PG_reserved flag is set, it means
  * that it allocated from the memblock allocator, just free it via the
- * free_bootmem_page(). Otherwise, use __free_page().
+ * arch_free_vmemmap_page(). Otherwise, use __free_page().
  */
 static inline void free_vmemmap_page(struct page *page)
 {
 	if (PageReserved(page))
-		free_bootmem_page(page);
+		arch_free_vmemmap_page(page);
 	else
 		__free_page(page);
 }
-- 
2.11.0

[PATCH 3/3] arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB

[Muchun Song]

The preparation of supporting freeing vmemmap associated with each
infrastructure is ready, so now we can support it for arm64.

Signed-off-by: Muchun Song <songmuchun@bytedance.com>
---
 arch/arm64/mm/mmu.c | 5 +++++
 fs/Kconfig          | 4 ++--
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 3802cfbdd20d..6051a120c906 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -22,6 +22,7 @@
 #include <linux/io.h>
 #include <linux/mm.h>
 #include <linux/vmalloc.h>
+#include <linux/hugetlb.h>
 
 #include <asm/barrier.h>
 #include <asm/cputype.h>
@@ -1109,6 +1110,10 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
 	pmd_t *pmdp;
 
 	WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END));
+
+	if (is_hugetlb_free_vmemmap_enabled() && !altmap)
+		return vmemmap_populate_basepages(start, end, node, altmap);
+
 	do {
 		next = pmd_addr_end(addr, end);
 
diff --git a/fs/Kconfig b/fs/Kconfig
index b5dcc68aab25..d90860135844 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -239,9 +239,9 @@ config HUGETLB_PAGE
 
 config HUGETLB_PAGE_FREE_VMEMMAP
 	def_bool HUGETLB_PAGE
-	depends on X86_64
+	select HAVE_BOOTMEM_INFO_NODE if X86_64
 	depends on SPARSEMEM_VMEMMAP
-	depends on HAVE_BOOTMEM_INFO_NODE
+	depends on X86_64 || ARM64
 
 config MEMFD_CREATE
 	def_bool TMPFS || HUGETLBFS
-- 
2.11.0

Re: [PATCH 2/3] mm: hugetlb: introduce arch_free_vmemmap_page

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 6922 bytes --]

Hi Muchun,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on next-20210310]
[cannot apply to hnaz-linux-mm/master arm64/for-next/core tip/x86/mm linus/master v5.12-rc2 v5.12-rc1 v5.11 v5.12-rc2]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Muchun-Song/mm-bootmem_info-mark-register_page_bootmem_info_section-__init/20210310-161619
base:    b01d57bfdc41c8f635b08b8a5af8a31217d46936
config: s390-randconfig-r021-20210308 (attached as .config)
compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project cd9a69289c7825d54450cb6829fef2c8e0f1963a)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install s390 cross compiling tool for clang build
        # apt-get install binutils-s390x-linux-gnu
        # https://github.com/0day-ci/linux/commit/92924b23a9f6de5bb478cea033cd2d39c227a6c5
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Muchun-Song/mm-bootmem_info-mark-register_page_bootmem_info_section-__init/20210310-161619
        git checkout 92924b23a9f6de5bb478cea033cd2d39c227a6c5
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=s390 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

   In file included from mm/sparse-vmemmap.c:23:
   In file included from include/linux/memblock.h:14:
   In file included from arch/s390/include/asm/dma.h:5:
   In file included from arch/s390/include/asm/io.h:80:
   include/asm-generic/io.h:464:31: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           val = __raw_readb(PCI_IOBASE + addr);
                             ~~~~~~~~~~ ^
   include/asm-generic/io.h:477:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           val = __le16_to_cpu((__le16 __force)__raw_readw(PCI_IOBASE + addr));
                                                           ~~~~~~~~~~ ^
   include/uapi/linux/byteorder/big_endian.h:36:59: note: expanded from macro '__le16_to_cpu'
   #define __le16_to_cpu(x) __swab16((__force __u16)(__le16)(x))
                                                             ^
   include/uapi/linux/swab.h:102:54: note: expanded from macro '__swab16'
   #define __swab16(x) (__u16)__builtin_bswap16((__u16)(x))
                                                        ^
   In file included from mm/sparse-vmemmap.c:23:
   In file included from include/linux/memblock.h:14:
   In file included from arch/s390/include/asm/dma.h:5:
   In file included from arch/s390/include/asm/io.h:80:
   include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
                                                           ~~~~~~~~~~ ^
   include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu'
   #define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
                                                             ^
   include/uapi/linux/swab.h:115:54: note: expanded from macro '__swab32'
   #define __swab32(x) (__u32)__builtin_bswap32((__u32)(x))
                                                        ^
   In file included from mm/sparse-vmemmap.c:23:
   In file included from include/linux/memblock.h:14:
   In file included from arch/s390/include/asm/dma.h:5:
   In file included from arch/s390/include/asm/io.h:80:
   include/asm-generic/io.h:501:33: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           __raw_writeb(value, PCI_IOBASE + addr);
                               ~~~~~~~~~~ ^
   include/asm-generic/io.h:511:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           __raw_writew((u16 __force)cpu_to_le16(value), PCI_IOBASE + addr);
                                                         ~~~~~~~~~~ ^
   include/asm-generic/io.h:521:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           __raw_writel((u32 __force)cpu_to_le32(value), PCI_IOBASE + addr);
                                                         ~~~~~~~~~~ ^
   include/asm-generic/io.h:609:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           readsb(PCI_IOBASE + addr, buffer, count);
                  ~~~~~~~~~~ ^
   include/asm-generic/io.h:617:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           readsw(PCI_IOBASE + addr, buffer, count);
                  ~~~~~~~~~~ ^
   include/asm-generic/io.h:625:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           readsl(PCI_IOBASE + addr, buffer, count);
                  ~~~~~~~~~~ ^
   include/asm-generic/io.h:634:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           writesb(PCI_IOBASE + addr, buffer, count);
                   ~~~~~~~~~~ ^
   include/asm-generic/io.h:643:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           writesw(PCI_IOBASE + addr, buffer, count);
                   ~~~~~~~~~~ ^
   include/asm-generic/io.h:652:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           writesl(PCI_IOBASE + addr, buffer, count);
                   ~~~~~~~~~~ ^
>> mm/sparse-vmemmap.c:159:13: warning: no previous prototype for function 'arch_free_vmemmap_page' [-Wmissing-prototypes]
   void __weak arch_free_vmemmap_page(struct page *page)
               ^
   mm/sparse-vmemmap.c:159:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
   void __weak arch_free_vmemmap_page(struct page *page)
   ^
   static 
   13 warnings generated.


vim +/arch_free_vmemmap_page +159 mm/sparse-vmemmap.c

   158	
 > 159	void __weak arch_free_vmemmap_page(struct page *page)
   160	{
   161		free_reserved_page(page);
   162	}
   163	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 17453 bytes --]

Re: [PATCH 2/3] mm: hugetlb: introduce arch_free_vmemmap_page

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 1859 bytes --]

Hi Muchun,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on next-20210310]
[cannot apply to hnaz-linux-mm/master arm64/for-next/core tip/x86/mm linus/master v5.12-rc2 v5.12-rc1 v5.11 v5.12-rc2]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Muchun-Song/mm-bootmem_info-mark-register_page_bootmem_info_section-__init/20210310-161619
base:    b01d57bfdc41c8f635b08b8a5af8a31217d46936
config: x86_64-randconfig-c002-20210310 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/92924b23a9f6de5bb478cea033cd2d39c227a6c5
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Muchun-Song/mm-bootmem_info-mark-register_page_bootmem_info_section-__init/20210310-161619
        git checkout 92924b23a9f6de5bb478cea033cd2d39c227a6c5
        # save the attached .config to linux build tree
        make W=1 ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

>> mm/sparse-vmemmap.c:159:13: warning: no previous prototype for 'arch_free_vmemmap_page' [-Wmissing-prototypes]
     159 | void __weak arch_free_vmemmap_page(struct page *page)
         |             ^~~~~~~~~~~~~~~~~~~~~~


vim +/arch_free_vmemmap_page +159 mm/sparse-vmemmap.c

   158	
 > 159	void __weak arch_free_vmemmap_page(struct page *page)
   160	{
   161		free_reserved_page(page);
   162	}
   163	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 35443 bytes --]

Re: [PATCH 3/3] arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 3786 bytes --]

Hi Muchun,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on next-20210310]
[cannot apply to hnaz-linux-mm/master arm64/for-next/core tip/x86/mm linus/master v5.12-rc2 v5.12-rc1 v5.11 v5.12-rc2]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Muchun-Song/mm-bootmem_info-mark-register_page_bootmem_info_section-__init/20210310-161619
base:    b01d57bfdc41c8f635b08b8a5af8a31217d46936
config: x86_64-randconfig-r002-20210309 (attached as .config)
compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project cd9a69289c7825d54450cb6829fef2c8e0f1963a)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install x86_64 cross compiling tool for clang build
        # apt-get install binutils-x86-64-linux-gnu
        # https://github.com/0day-ci/linux/commit/d0b8d83bb423fd25a85be4a62a8bed9f8d8a0d96
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Muchun-Song/mm-bootmem_info-mark-register_page_bootmem_info_section-__init/20210310-161619
        git checkout d0b8d83bb423fd25a85be4a62a8bed9f8d8a0d96
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

   arch/x86/mm/init_64.c:1585:6: error: implicit declaration of function 'vmemmap_use_new_sub_pmd' [-Werror,-Wimplicit-function-declaration]
                                           vmemmap_use_new_sub_pmd(addr, next);
                                           ^
   arch/x86/mm/init_64.c:1591:4: error: implicit declaration of function 'vmemmap_use_sub_pmd' [-Werror,-Wimplicit-function-declaration]
                           vmemmap_use_sub_pmd(addr, next);
                           ^
>> arch/x86/mm/init_64.c:1625:6: warning: no previous prototype for function 'arch_free_vmemmap_page' [-Wmissing-prototypes]
   void arch_free_vmemmap_page(struct page *page)
        ^
   arch/x86/mm/init_64.c:1625:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
   void arch_free_vmemmap_page(struct page *page)
   ^
   static 
   1 warning and 2 errors generated.
--
   mm/bootmem_info.c:87:11: error: implicit declaration of function 'sparse_decode_mem_map' [-Werror,-Wimplicit-function-declaration]
           memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr);
                    ^
>> mm/bootmem_info.c:87:9: warning: incompatible integer to pointer conversion assigning to 'struct page *' from 'int' [-Wint-conversion]
           memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr);
                  ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   1 warning and 1 error generated.


vim +/arch_free_vmemmap_page +1625 arch/x86/mm/init_64.c

c2b91e2eec9678 Yinghai Lu  2008-04-12  1623  
cd28b1a6791da0 Muchun Song 2021-03-09  1624  #ifdef CONFIG_HAVE_BOOTMEM_INFO_NODE
92924b23a9f6de Muchun Song 2021-03-10 @1625  void arch_free_vmemmap_page(struct page *page)
92924b23a9f6de Muchun Song 2021-03-10  1626  {
92924b23a9f6de Muchun Song 2021-03-10  1627  	free_bootmem_page(page);
92924b23a9f6de Muchun Song 2021-03-10  1628  }
92924b23a9f6de Muchun Song 2021-03-10  1629  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 32607 bytes --]

Re: [PATCH 0/3] Add support for free vmemmap pages of HugeTLB for arm64

[Chen Huang]

在 2021/3/10 15:15, Muchun Song 写道:
> This patchset is based on the series of "Free some vmemmap pages of HugeTLB
> page". More details can refer to the below link.
> 
>   https://lkml.kernel.org/r/20210308102807.59745-1-songmuchun@bytedance.com
> 
> I often received some feedback (We want to test this feature on arm64) before.
> Because the previous code has been reviewed for 18 versions and is merged
> into mm tree, I think that it is time to release this patchset. If you want
> to test then you can start now :-). And I also hope someone can review this.
> 
> Thanks.
> 
> Muchun Song (3):
>   mm: bootmem_info: mark register_page_bootmem_info_section __init
>   mm: hugetlb: introduce arch_free_vmemmap_page
>   arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
> 
>  arch/arm64/mm/mmu.c   | 5 +++++
>  arch/x86/mm/init_64.c | 5 +++++
>  fs/Kconfig            | 4 ++--
>  mm/bootmem_info.c     | 4 ++--
>  mm/sparse-vmemmap.c   | 9 +++++++--
>  5 files changed, 21 insertions(+), 6 deletions(-)
> 

Tested-by: Chen Huang <chenhuang5@huawei.com>

I have tested the patch and the result is same as the last time.

Re: [PATCH 0/3] Add support for free vmemmap pages of HugeTLB for arm64

[Bodeddula, Balasubramaniam]

Chen, is your testing steps documented somewhere, can you please point us to the same. I followed some steps for testing the x86 patches, just wanted to make sure I am covering your tests as well. We are actively working on building and testing these patches for ARM.

On 11/03/21, 9:44 AM, "Chen Huang" <chenhuang5@huawei.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



    在 2021/3/10 15:15, Muchun Song 写道:
    > This patchset is based on the series of "Free some vmemmap pages of HugeTLB
    > page". More details can refer to the below link.
    >
    >   https://lkml.kernel.org/r/20210308102807.59745-1-songmuchun@bytedance.com
    >
    > I often received some feedback (We want to test this feature on arm64) before.
    > Because the previous code has been reviewed for 18 versions and is merged
    > into mm tree, I think that it is time to release this patchset. If you want
    > to test then you can start now :-). And I also hope someone can review this.
    >
    > Thanks.
    >
    > Muchun Song (3):
    >   mm: bootmem_info: mark register_page_bootmem_info_section __init
    >   mm: hugetlb: introduce arch_free_vmemmap_page
    >   arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
    >
    >  arch/arm64/mm/mmu.c   | 5 +++++
    >  arch/x86/mm/init_64.c | 5 +++++
    >  fs/Kconfig            | 4 ++--
    >  mm/bootmem_info.c     | 4 ++--
    >  mm/sparse-vmemmap.c   | 9 +++++++--
    >  5 files changed, 21 insertions(+), 6 deletions(-)
    >

    Tested-by: Chen Huang <chenhuang5@huawei.com>

    I have tested the patch and the result is same as the last time.

Re: [PATCH 0/3] Add support for free vmemmap pages of HugeTLB for arm64

[Chen Huang]

在 2021/3/11 13:00, Bodeddula, Balasubramaniam 写道:
> Chen, is your testing steps documented somewhere, can you please point us to the same. I followed some steps for testing the x86 patches, just wanted to make sure I am covering your tests as well. We are actively working on building and testing these patches for ARM.
> 
> On 11/03/21, 9:44 AM, "Chen Huang" <chenhuang5@huawei.com> wrote:
> 
>     CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> 
> 
> 
>     在 2021/3/10 15:15, Muchun Song 写道:
>     > This patchset is based on the series of "Free some vmemmap pages of HugeTLB
>     > page". More details can refer to the below link.
>     >
>     >   https://lkml.kernel.org/r/20210308102807.59745-1-songmuchun@bytedance.com
>     >
>     > I often received some feedback (We want to test this feature on arm64) before.
>     > Because the previous code has been reviewed for 18 versions and is merged
>     > into mm tree, I think that it is time to release this patchset. If you want
>     > to test then you can start now :-). And I also hope someone can review this.
>     >
>     > Thanks.
>     >
>     > Muchun Song (3):
>     >   mm: bootmem_info: mark register_page_bootmem_info_section __init
>     >   mm: hugetlb: introduce arch_free_vmemmap_page
>     >   arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
>     >
>     >  arch/arm64/mm/mmu.c   | 5 +++++
>     >  arch/x86/mm/init_64.c | 5 +++++
>     >  fs/Kconfig            | 4 ++--
>     >  mm/bootmem_info.c     | 4 ++--
>     >  mm/sparse-vmemmap.c   | 9 +++++++--
>     >  5 files changed, 21 insertions(+), 6 deletions(-)
>     >
> 
>     Tested-by: Chen Huang <chenhuang5@huawei.com>
> 
>     I have tested the patch and the result is same as the last time.
> 

The test work is that: I set the total memory of 40G, and use 10G for hugepages.
First I reserve 10G hugepages from the command line and the result is that:
------------------------------------------------------------------------------------------------
                    2M page                    |                    1G page                    |
----------------------|------------------------|----------------------|------------------------|
       enable         |        disable         |      enable          |        disable         |
----------------------|------------------------|----------------------|------------------------|
total  |  used | free | total  |  used | free  |total  |  used | free | total  |  used | free  |
39,697 | 10279 |29,415| 39580  | 10279 | 29,297‬|39,739 | 10279 |29,455| 39580  | 10279 | 29,296|
------------------------------------------------------------------------------------------------
For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.

Then I alloc 10G hugepages using "echo XX > /sys/kernel/mm/hugepages/hugepages-XXkB/nr_hugepages",
and get the result:
------------------------------------------------------------------------------------------------
                    2M page                    |                    1G page                    |
----------------------|------------------------|----------------------|------------------------|
       enable         |        disable         |      enable          |        disable         |
----------------------|------------------------|----------------------|------------------------|
total  |  used | free | total  |  used | free  |total  |  used | free | total  |  used | free  |
39,699 | 10279 |29,415| 39580  | 10279 | 29,297‬‬|39,739 | 10279 |29,455| 39580  | 10279 | 29,296|
------------------------------------------------------------------------------------------------
For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.

Re: [PATCH 0/3] Add support for free vmemmap pages of HugeTLB for arm64

[Bodeddula, Balasubramaniam]

Hey, thanks for the testing steps.

I tried applying these patches on 5.11 source tree. These patches were applied on top of the x86 patch, which worked fine. But in this case we don't see the same improvements in our testing. We made sure we set CONFIG_HUGETLB_PAGE_FREE_VMEMMAP=y in our .config file.

Are we missing any more configuration settings to get this patch to work on ARM? Can you please help with general troubleshooting steps to debug what could be going wrong.

On 11/03/21, 11:32 AM, "Chen Huang" <chenhuang5@huawei.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



    在 2021/3/11 13:00, Bodeddula, Balasubramaniam 写道:
    > Chen, is your testing steps documented somewhere, can you please point us to the same. I followed some steps for testing the x86 patches, just wanted to make sure I am covering your tests as well. We are actively working on building and testing these patches for ARM.
    >
    > On 11/03/21, 9:44 AM, "Chen Huang" <chenhuang5@huawei.com> wrote:
    >
    >     CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
    >
    >
    >
    >     在 2021/3/10 15:15, Muchun Song 写道:
    >     > This patchset is based on the series of "Free some vmemmap pages of HugeTLB
    >     > page". More details can refer to the below link.
    >     >
    >     >   https://lkml.kernel.org/r/20210308102807.59745-1-songmuchun@bytedance.com
    >     >
    >     > I often received some feedback (We want to test this feature on arm64) before.
    >     > Because the previous code has been reviewed for 18 versions and is merged
    >     > into mm tree, I think that it is time to release this patchset. If you want
    >     > to test then you can start now :-). And I also hope someone can review this.
    >     >
    >     > Thanks.
    >     >
    >     > Muchun Song (3):
    >     >   mm: bootmem_info: mark register_page_bootmem_info_section __init
    >     >   mm: hugetlb: introduce arch_free_vmemmap_page
    >     >   arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
    >     >
    >     >  arch/arm64/mm/mmu.c   | 5 +++++
    >     >  arch/x86/mm/init_64.c | 5 +++++
    >     >  fs/Kconfig            | 4 ++--
    >     >  mm/bootmem_info.c     | 4 ++--
    >     >  mm/sparse-vmemmap.c   | 9 +++++++--
    >     >  5 files changed, 21 insertions(+), 6 deletions(-)
    >     >
    >
    >     Tested-by: Chen Huang <chenhuang5@huawei.com>
    >
    >     I have tested the patch and the result is same as the last time.
    >

    The test work is that: I set the total memory of 40G, and use 10G for hugepages.
    First I reserve 10G hugepages from the command line and the result is that:
    ------------------------------------------------------------------------------------------------
                        2M page                    |                    1G page                    |
    ----------------------|------------------------|----------------------|------------------------|
           enable         |        disable         |      enable          |        disable         |
    ----------------------|------------------------|----------------------|------------------------|
    total  |  used | free | total  |  used | free  |total  |  used | free | total  |  used | free  |
    39,697 | 10279 |29,415| 39580  | 10279 | 29,297‬|39,739 | 10279 |29,455| 39580  | 10279 | 29,296|
    ------------------------------------------------------------------------------------------------
    For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
    For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.

    Then I alloc 10G hugepages using "echo XX > /sys/kernel/mm/hugepages/hugepages-XXkB/nr_hugepages",
    and get the result:
    ------------------------------------------------------------------------------------------------
                        2M page                    |                    1G page                    |
    ----------------------|------------------------|----------------------|------------------------|
           enable         |        disable         |      enable          |        disable         |
    ----------------------|------------------------|----------------------|------------------------|
    total  |  used | free | total  |  used | free  |total  |  used | free | total  |  used | free  |
    39,699 | 10279 |29,415| 39580  | 10279 | 29,297‬‬|39,739 | 10279 |29,455| 39580  | 10279 | 29,296|
    ------------------------------------------------------------------------------------------------
    For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
    For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.

Re: [PATCH 0/3] Add support for free vmemmap pages of HugeTLB for arm64

[Chen Huang]

Are you saying that the patch works well in x86, but doesn't in arm64? If so, my suggestion is that first you can check the dumpstack to check whether the function free_vmemmap_page has been called.
Then you can dump the page list of vmemmap_pages and the refcount of pages to verify the pages has been freed into the buddy system.

Hope this will help you!

在 2021/3/12 2:00, Bodeddula, Balasubramaniam 写道:
> Hey, thanks for the testing steps.
> 
> I tried applying these patches on 5.11 source tree. These patches were applied on top of the x86 patch, which worked fine. But in this case we don't see the same improvements in our testing. We made sure we set CONFIG_HUGETLB_PAGE_FREE_VMEMMAP=y in our .config file.
> 
> Are we missing any more configuration settings to get this patch to work on ARM? Can you please help with general troubleshooting steps to debug what could be going wrong.
> 
> On 11/03/21, 11:32 AM, "Chen Huang" <chenhuang5@huawei.com> wrote:
> 
>     CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> 
> 
> 
>     在 2021/3/11 13:00, Bodeddula, Balasubramaniam 写道:
>     > Chen, is your testing steps documented somewhere, can you please point us to the same. I followed some steps for testing the x86 patches, just wanted to make sure I am covering your tests as well. We are actively working on building and testing these patches for ARM.
>     >
>     > On 11/03/21, 9:44 AM, "Chen Huang" <chenhuang5@huawei.com> wrote:
>     >
>     >     CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>     >
>     >
>     >
>     >     在 2021/3/10 15:15, Muchun Song 写道:
>     >     > This patchset is based on the series of "Free some vmemmap pages of HugeTLB
>     >     > page". More details can refer to the below link.
>     >     >
>     >     >   https://lkml.kernel.org/r/20210308102807.59745-1-songmuchun@bytedance.com
>     >     >
>     >     > I often received some feedback (We want to test this feature on arm64) before.
>     >     > Because the previous code has been reviewed for 18 versions and is merged
>     >     > into mm tree, I think that it is time to release this patchset. If you want
>     >     > to test then you can start now :-). And I also hope someone can review this.
>     >     >
>     >     > Thanks.
>     >     >
>     >     > Muchun Song (3):
>     >     >   mm: bootmem_info: mark register_page_bootmem_info_section __init
>     >     >   mm: hugetlb: introduce arch_free_vmemmap_page
>     >     >   arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB
>     >     >
>     >     >  arch/arm64/mm/mmu.c   | 5 +++++
>     >     >  arch/x86/mm/init_64.c | 5 +++++
>     >     >  fs/Kconfig            | 4 ++--
>     >     >  mm/bootmem_info.c     | 4 ++--
>     >     >  mm/sparse-vmemmap.c   | 9 +++++++--
>     >     >  5 files changed, 21 insertions(+), 6 deletions(-)
>     >     >
>     >
>     >     Tested-by: Chen Huang <chenhuang5@huawei.com>
>     >
>     >     I have tested the patch and the result is same as the last time.
>     >
> 
>     The test work is that: I set the total memory of 40G, and use 10G for hugepages.
>     First I reserve 10G hugepages from the command line and the result is that:
>     ------------------------------------------------------------------------------------------------
>                         2M page                    |                    1G page                    |
>     ----------------------|------------------------|----------------------|------------------------|
>            enable         |        disable         |      enable          |        disable         |
>     ----------------------|------------------------|----------------------|------------------------|
>     total  |  used | free | total  |  used | free  |total  |  used | free | total  |  used | free  |
>     39,697 | 10279 |29,415| 39580  | 10279 | 29,297‬|39,739 | 10279 |29,455| 39580  | 10279 | 29,296|
>     ------------------------------------------------------------------------------------------------
>     For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
>     For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.
> 
>     Then I alloc 10G hugepages using "echo XX > /sys/kernel/mm/hugepages/hugepages-XXkB/nr_hugepages",
>     and get the result:
>     ------------------------------------------------------------------------------------------------
>                         2M page                    |                    1G page                    |
>     ----------------------|------------------------|----------------------|------------------------|
>            enable         |        disable         |      enable          |        disable         |
>     ----------------------|------------------------|----------------------|------------------------|
>     total  |  used | free | total  |  used | free  |total  |  used | free | total  |  used | free  |
>     39,699 | 10279 |29,415| 39580  | 10279 | 29,297‬‬|39,739 | 10279 |29,455| 39580  | 10279 | 29,296|
>     ------------------------------------------------------------------------------------------------
>     For 2M hugepage, we can save 118M memory which is correspoinding to the expected 120M memory.
>     For 1G hugepage, we can save 159M memory which is correspoinding to the expected 160M memory.
> 

Re: [PATCH 0/3] Add support for free vmemmap pages of HugeTLB for arm64

[Mike Kravetz]

On 3/11/21 10:00 AM, Bodeddula, Balasubramaniam wrote:
> Hey, thanks for the testing steps.
> 
> I tried applying these patches on 5.11 source tree. These patches were applied on top of the x86 patch, which worked fine. But in this case we don't see the same improvements in our testing. We made sure we set CONFIG_HUGETLB_PAGE_FREE_VMEMMAP=y in our .config file.
> 
> Are we missing any more configuration settings to get this patch to work on ARM? Can you please help with general troubleshooting steps to debug what could be going wrong.
> 

Are you specifying 'hugetlb_free_vmemmap=on' on the kernel command line?
This feature is only enabled if you 'opt in' via the command line option.

-- 
Mike Kravetz