[PATCH v3 0/2] Hugetlbfs support for riscv

[PATCH v3 0/2] Hugetlbfs support for riscv

[Alexandre Ghiti]

This series introduces hugetlbfs support for both riscv 32/64. Riscv32           
is architecturally limited to huge pages of size 4MB whereas riscv64 has         
2MB/1G huge pages support. Transparent huge page support is not                  
implemented here, I will submit another series later.                            
                                                                                 
As stated in "The RISC-V Instruction Set Manual, Volume II: Privileged           
Architecture", riscv page table entries are marked as non-leaf entries           
as soon as at least one of the R/W/X bit set:                                    
                                                                                 
- pmd_huge/pud_huge check if one of those bits are set,                          
- pte_mkhuge simply returns the same pte value and does not set any of           
  the R/W/X bits                                                                 
                                                                                 
This series was validated using libhugetlbfs testsuite ported to riscv64         
without linker script support.                                                   
(https://github.com/AlexGhiti/libhugetlbfs.git, branch dev/alex/riscv).          
                                                                                 
- libhugetlbfs testsuite on riscv64/2M:                                          
  - brk_near_huge triggers an assert in malloc.c, does not on x86.               
                                                                                 
- libhugetlbfs testsuite on riscv64/1G:                                          
  - brk_near_huge triggers an assert in malloc.c, does not on x86.               
  - mmap-gettest, mmap-cow: testsuite passes the number of default free          
    pages as parameters and then fails for 1G which is not the default.          
    Otherwise succeeds when given the right number of pages.                     
  - map_high_truncate_2 fails on x86 too: 0x60000000 is not 1G aligned           
    and fails at line 694 of fs/hugetlbfs/inode.c.                               
  - heapshrink on 1G fails on x86 too, not investigated.                         
  - counters.sh on 1G fails on x86 too: alloc_surplus_huge_page returns          
    NULL in case of gigantic pages.                                              
  - icache-hygiene succeeds after patch #3 of this series which lowers           
    the base address of mmap.                                                    
  - fallocate_stress.sh on 1G never ends, on x86 too, not investigated.          
                                                                                 
- libhugetlbfs testsuite on riscv32/4M: kernel build passes, lacks               
  libhugetlbfs support for 32bits.                                               
                                                                                 
* Output for riscv64 2M and 1G libhugetbfs testsuite:                            
                                                                                 
zero_filesize_segment (2M: 64):                                                  
zero_filesize_segment (1024M: 64):                                               
test_root (2M: 64):     PASS                                                     
test_root (1024M: 64):  PASS                                                     
meminfo_nohuge (2M: 64):        PASS                                             
meminfo_nohuge (1024M: 64):     PASS                                             
gethugepagesize (2M: 64):       PASS                                             
gethugepagesize (1024M: 64):    PASS                                             
gethugepagesizes (2M: 64):      PASS                                             
gethugepagesizes (1024M: 64):   PASS                                             
HUGETLB_VERBOSE=1 empty_mounts (2M: 64):        PASS                             
HUGETLB_VERBOSE=1 empty_mounts (1024M: 64):     PASS                             
HUGETLB_VERBOSE=1 large_mounts (2M: 64):        PASS                             
HUGETLB_VERBOSE=1 large_mounts (1024M: 64):     PASS                             
find_path (2M: 64):     PASS                                                     
find_path (1024M: 64):  PASS                                                     
unlinked_fd (2M: 64):   PASS                                                     
unlinked_fd (1024M: 64):        PASS                                             
readback (2M: 64):      PASS                                                     
readback (1024M: 64):   PASS                                                     
truncate (2M: 64):      PASS                                                     
truncate (1024M: 64):   PASS                                                     
shared (2M: 64):        PASS                                                     
shared (1024M: 64):     PASS                                                     
mprotect (2M: 64):      PASS                                                     
mprotect (1024M: 64):   PASS                                                     
mlock (2M: 64): PASS                                                             
mlock (1024M: 64):      PASS                                                     
misalign (2M: 64):      PASS                                                     
misalign (1024M: 64):   PASS                                                     
fallocate_basic.sh (2M: 64):    PASS                                             
fallocate_basic.sh (1024M: 64): PASS                                             
fallocate_align.sh (2M: 64):    PASS                                             
fallocate_align.sh (1024M: 64): PASS                                             
ptrace-write-hugepage (2M: 64): PASS                                             
ptrace-write-hugepage (1024M: 64):      PASS                                     
icache-hygiene (2M: 64):        PASS                                             
icache-hygiene (1024M: 64):     PASS                                             
slbpacaflush (2M: 64):  PASS (inconclusive)                                      
slbpacaflush (1024M: 64):       PASS (inconclusive)                              
straddle_4GB_static (2M: 64):   PASS                                             
straddle_4GB_static (1024M: 64):        PASS                                     
huge_at_4GB_normal_below_static (2M: 64):       PASS                             
huge_at_4GB_normal_below_static (1024M: 64):    PASS                             
huge_below_4GB_normal_above_static (2M: 64):    PASS                             
huge_below_4GB_normal_above_static (1024M: 64): PASS                             
map_high_truncate_2 (2M: 64):   PASS                                             
map_high_truncate_2 (1024M: 64):        FAIL    ftruncate(): Invalid             
argument                                                                         
misaligned_offset (2M: 64):     PASS (inconclusive)                              
misaligned_offset (1024M: 64):  PASS (inconclusive)                              
truncate_above_4GB (2M: 64):    PASS                                             
truncate_above_4GB (1024M: 64): PASS                                             
brk_near_huge (2M: 64): brk_near_huge: malloc.c:2385: sysmalloc:                 
Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned          
long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long)          
old_end & (pagesize - 1)) == 0)' failed.                                         
brk_near_huge (1024M: 64):      brk_near_huge: malloc.c:2385: sysmalloc:         
Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned          
long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long)          
old_end & (pagesize - 1)) == 0)' failed.                                         
task-size-overrun (2M: 64):     PASS                                             
task-size-overrun (1024M: 64):  PASS                                             
stack_grow_into_huge (2M: 64):   PASS                                            
stack_grow_into_huge (1024M: 64): PASS                                           
corrupt-by-cow-opt (2M: 64):    PASS                                             
corrupt-by-cow-opt (1024M: 64): PASS                                             
noresv-preserve-resv-page (2M: 64):     PASS                                     
noresv-preserve-resv-page (1024M: 64):  PASS                                     
noresv-regarded-as-resv (2M: 64):       PASS                                     
noresv-regarded-as-resv (1024M: 64):    PASS                                     
readahead_reserve.sh (2M: 64):  PASS                                             
readahead_reserve.sh (1024M: 64):       PASS                                     
madvise_reserve.sh (2M: 64):    PASS                                             
madvise_reserve.sh (1024M: 64): PASS                                             
fadvise_reserve.sh (2M: 64):    PASS                                             
fadvise_reserve.sh (1024M: 64): PASS                                             
mremap-expand-slice-collision.sh (2M: 64):      PASS                             
mremap-expand-slice-collision.sh (1024M: 64):   PASS                             
mremap-fixed-normal-near-huge.sh (2M: 64):      PASS                             
mremap-fixed-normal-near-huge.sh (1024M: 64):   PASS                             
mremap-fixed-huge-near-normal.sh (2M: 64):      PASS                             
mremap-fixed-huge-near-normal.sh (1024M: 64):   PASS                             
set shmmax limit to 67108864                                                     
shm-perms (2M: 64):     PASS                                                     
private (2M: 64):       PASS                                                     
private (1024M: 64):    PASS                                                     
fork-cow (2M: 64):      PASS                                                     
fork-cow (1024M: 64):   PASS                                                     
direct (2M: 64):        Bad configuration: Failed to open direct-IO              
file: Invalid argument                                                           
direct (1024M: 64):     Bad configuration: Failed to open direct-IO              
file: File exists                                                                
malloc (2M: 64):        PASS                                                     
malloc (1024M: 64):     PASS                                                     
LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes malloc (2M: 64):                 
PASS                                                                             
LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes malloc (1024M: 64):              
PASS                                                                             
LD_PRELOAD=libhugetlbfs.so HUGETLB_RESTRICT_EXE=unknown:none                     
HUGETLB_MORECORE=yes malloc (2M: 64):      PASS                                  
LD_PRELOAD=libhugetlbfs.so HUGETLB_RESTRICT_EXE=unknown:none                     
HUGETLB_MORECORE=yes malloc (1024M: 64):PASS                                     
LD_PRELOAD=libhugetlbfs.so HUGETLB_RESTRICT_EXE=unknown:malloc                   
HUGETLB_MORECORE=yes malloc (2M: 64):    PASS                                    
LD_PRELOAD=libhugetlbfs.so HUGETLB_RESTRICT_EXE=unknown:malloc                   
HUGETLB_MORECORE=yes malloc (1024M: 64): PASS                                    
malloc_manysmall (2M: 64):      PASS                                             
malloc_manysmall (1024M: 64):   PASS                                             
LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes malloc_manysmall (2M:            
64):      PASS                                                                   
LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes malloc_manysmall (1024M:         
64):   PASS                                                                      
heapshrink (2M: 64):    PASS                                                     
heapshrink (1024M: 64): PASS                                                     
LD_PRELOAD=libheapshrink.so heapshrink (2M: 64):        PASS                     
LD_PRELOAD=libheapshrink.so heapshrink (1024M: 64):     PASS                     
LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes heapshrink (2M: 64):             
PASS                                                                             
LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes heapshrink (1024M: 64):          
PASS                                                                             
LD_PRELOAD=libhugetlbfs.so libheapshrink.so HUGETLB_MORECORE=yes                 
heapshrink (2M: 64):   PASS                                                      
LD_PRELOAD=libhugetlbfs.so libheapshrink.so HUGETLB_MORECORE=yes                 
heapshrink (1024M: 64):        PASS                                              
LD_PRELOAD=libheapshrink.so HUGETLB_MORECORE_SHRINK=yes                          
HUGETLB_MORECORE=yes heapshrink (2M: 64):       PASS (inconclusive)              
LD_PRELOAD=libheapshrink.so HUGETLB_MORECORE_SHRINK=yes                          
HUGETLB_MORECORE=yes heapshrink (1024M: 64):    PASS (inconclusive)              
LD_PRELOAD=libhugetlbfs.so libheapshrink.so HUGETLB_MORECORE_SHRINK=yes          
HUGETLB_MORECORE=yes heapshrink (2M: 64):       PASS                             
LD_PRELOAD=libhugetlbfs.so libheapshrink.so HUGETLB_MORECORE_SHRINK=yes          
HUGETLB_MORECORE=yes heapshrink (1024M: 64):    FAIL    Heap did not             
shrink                                                                           
HUGETLB_VERBOSE=1 HUGETLB_MORECORE=yes heap-overflow (2M: 64):  PASS             
HUGETLB_VERBOSE=1 HUGETLB_MORECORE=yes heap-overflow (1024M: 64):                
PASS                                                                             
HUGETLB_VERBOSE=0 linkhuge_nofd (2M: 64):                                        
HUGETLB_VERBOSE=0 linkhuge_nofd (1024M: 64):                                     
LD_PRELOAD=libhugetlbfs.so HUGETLB_VERBOSE=0 linkhuge_nofd (2M: 64):             
LD_PRELOAD=libhugetlbfs.so HUGETLB_VERBOSE=0 linkhuge_nofd (1024M: 64):          
linkhuge (2M: 64):                                                               
linkhuge (1024M: 64):                                                            
LD_PRELOAD=libhugetlbfs.so linkhuge (2M: 64):                                    
LD_PRELOAD=libhugetlbfs.so linkhuge (1024M: 64):                                 
linkhuge_rw (2M: 64):                                                            
linkhuge_rw (1024M: 64):                                                         
HUGETLB_ELFMAP=R linkhuge_rw (2M: 64):                                           
HUGETLB_ELFMAP=R linkhuge_rw (1024M: 64):                                        
HUGETLB_ELFMAP=W linkhuge_rw (2M: 64):                                           
HUGETLB_ELFMAP=W linkhuge_rw (1024M: 64):                                        
HUGETLB_ELFMAP=RW linkhuge_rw (2M: 64):                                          
HUGETLB_ELFMAP=RW linkhuge_rw (1024M: 64):                                       
HUGETLB_ELFMAP=no linkhuge_rw (2M: 64):                                          
HUGETLB_ELFMAP=no linkhuge_rw (1024M: 64):                                       
HUGETLB_ELFMAP=R HUGETLB_MINIMAL_COPY=no linkhuge_rw (2M: 64):                   
HUGETLB_ELFMAP=R HUGETLB_MINIMAL_COPY=no linkhuge_rw (1024M: 64):                
HUGETLB_ELFMAP=W HUGETLB_MINIMAL_COPY=no linkhuge_rw (2M: 64):                   
HUGETLB_ELFMAP=W HUGETLB_MINIMAL_COPY=no linkhuge_rw (1024M: 64):                
HUGETLB_ELFMAP=RW HUGETLB_MINIMAL_COPY=no linkhuge_rw (2M: 64):                  
HUGETLB_ELFMAP=RW HUGETLB_MINIMAL_COPY=no linkhuge_rw (1024M: 64):               
HUGETLB_SHARE=0 HUGETLB_ELFMAP=R linkhuge_rw (2M: 64):                           
HUGETLB_SHARE=0 HUGETLB_ELFMAP=R linkhuge_rw (1024M: 64):                        
HUGETLB_SHARE=1 HUGETLB_ELFMAP=R linkhuge_rw (2M: 64):                           
HUGETLB_SHARE=1 HUGETLB_ELFMAP=R linkhuge_rw (1024M: 64):                        
HUGETLB_SHARE=0 HUGETLB_ELFMAP=W linkhuge_rw (2M: 64):                           
HUGETLB_SHARE=0 HUGETLB_ELFMAP=W linkhuge_rw (1024M: 64):                        
HUGETLB_SHARE=1 HUGETLB_ELFMAP=W linkhuge_rw (2M: 64):                           
HUGETLB_SHARE=1 HUGETLB_ELFMAP=W linkhuge_rw (1024M: 64):                        
HUGETLB_SHARE=0 HUGETLB_ELFMAP=RW linkhuge_rw (2M: 64):                          
HUGETLB_SHARE=0 HUGETLB_ELFMAP=RW linkhuge_rw (1024M: 64):                       
HUGETLB_SHARE=1 HUGETLB_ELFMAP=RW linkhuge_rw (2M: 64):                          
HUGETLB_SHARE=1 HUGETLB_ELFMAP=RW linkhuge_rw (1024M: 64):                       
chunk-overcommit (2M: 64):      PASS                                             
chunk-overcommit (1024M: 64):   PASS                                             
alloc-instantiate-race shared (2M: 64): PASS                                     
alloc-instantiate-race shared (1024M: 64):      PASS                             
alloc-instantiate-race private (2M: 64):        PASS                             
alloc-instantiate-race private (1024M: 64):     PASS                             
truncate_reserve_wraparound (2M: 64):   PASS                                     
truncate_reserve_wraparound (1024M: 64):        PASS                             
truncate_sigbus_versus_oom (2M: 64):    PASS                                     
truncate_sigbus_versus_oom (1024M: 64): PASS                                     
get_huge_pages (2M: 64):        PASS                                             
get_huge_pages (1024M: 64):     PASS                                             
shmoverride_linked (2M: 64):    PASS                                             
HUGETLB_SHM=yes shmoverride_linked (2M: 64):    PASS                             
shmoverride_linked_static (2M: 64):                                              
HUGETLB_SHM=yes shmoverride_linked_static (2M: 64):                              
LD_PRELOAD=libhugetlbfs.so shmoverride_unlinked (2M: 64):       PASS             
LD_PRELOAD=libhugetlbfs.so HUGETLB_SHM=yes shmoverride_unlinked (2M:             
64):       PASS                                                                  
quota.sh (2M: 64):      PASS                                                     
quota.sh (1024M: 64):   PASS                                                     
counters.sh (2M: 64):   PASS                                                     
counters.sh (1024M: 64):        FAIL mmap failed: Invalid argument               
mmap-gettest 10 35 (2M: 64):    PASS                                             
mmap-gettest 10 35 (1024M: 64): FAIL    Failed to mmap the hugetlb file:         
Cannot allocate memory                                                           
mmap-cow 34 35 (2M: 64):        PASS                                             
mmap-cow 34 35 (1024M: 64):     FAIL    Thread 15 (pid=514) failed               
set shmmax limit to 73400320                                                     
shm-fork 10 17 (2M: 64):        PASS                                             
set shmmax limit to 73400320                                                     
shm-fork 10 35 (2M: 64):        PASS                                             
set shmmax limit to 73400320                                                     
shm-getraw 35 /dev/full (2M: 64):       PASS                                     
fallocate_stress.sh (2M: 64):   libgcc_s.so.1 must be installed for              
pthread_cancel to work                                                           
fallocate_stress.sh (1024M: 64):                                                 
********** TEST SUMMARY                                                          
*                      2M             1024M                                      
*                      32-bit 64-bit  32-bit 64-bit                              
*     Total testcases:     0     93       0     83                               
*             Skipped:     0      0       0      0                               
*                PASS:     0     69       0     56                               
*                FAIL:     0      0       0      5                               
*    Killed by signal:     0      1       0      2                               
*   Bad configuration:     0      1       0      1                               
*       Expected FAIL:     0      0       0      0                               
*     Unexpected PASS:     0      0       0      0                               
*    Test not present:     0     21       0     19                               
* Strange test result:     0      1       0      0                               
**********                                                                       
                                                                          
Changes in v3:
  - Remove the config ARCH_WANT_HUGE_PMD_SHARE in arm64, thanks to
    Hanjun.
  - Rebased on top v5.2-rc7.
  - Dropped the reviewed-by Palmer, and did not add Catalin and Ingo
    acked-by.
       
Changes in v2:                                                                   
  - Merge comment fix about task size.                                           
  - Patch about mmap base address was merged separately.                         
  - Rebased on top of linux-next where series about the capability               
    to free gigantic pages regardless of the configuration was merged.           
  - Add huge pmd sharing as suggested by Mike Kravetz.                           
  - Gigantic page hstate is automatically created if CONTIG_ALLOC is             
    set, even if not explicitly asked for in command line, as suggested          
    by Mike.                                                                     
  - Replace #ifdef CONFIG_64BIT into IS_ENABLED(CONFIG_64BIT), as suggested      
    by Christoph Hellwig.                                       

Alexandre Ghiti (2):
  x86, arm64: Move ARCH_WANT_HUGE_PMD_SHARE config in arch/Kconfig
  riscv: Introduce huge page support for 32/64bit kernel

 arch/Kconfig                     |  3 +++
 arch/arm64/Kconfig               |  4 +--
 arch/riscv/Kconfig               |  8 ++++++
 arch/riscv/include/asm/hugetlb.h | 18 +++++++++++++
 arch/riscv/include/asm/page.h    | 10 ++++++++
 arch/riscv/include/asm/pgtable.h |  8 ++++--
 arch/riscv/mm/Makefile           |  2 ++
 arch/riscv/mm/hugetlbpage.c      | 44 ++++++++++++++++++++++++++++++++
 arch/x86/Kconfig                 |  4 +--
 9 files changed, 93 insertions(+), 8 deletions(-)
 create mode 100644 arch/riscv/include/asm/hugetlb.h
 create mode 100644 arch/riscv/mm/hugetlbpage.c

-- 
2.20.1

[PATCH v3 1/2] x86, arm64: Move ARCH_WANT_HUGE_PMD_SHARE config in arch/Kconfig

[Alexandre Ghiti]

ARCH_WANT_HUGE_PMD_SHARE config was declared in both architectures:
move this declaration in arch/Kconfig and make those architectures
select it.

Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
---
 arch/Kconfig       | 3 +++
 arch/arm64/Kconfig | 4 +---
 arch/x86/Kconfig   | 4 +---
 3 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/arch/Kconfig b/arch/Kconfig
index c47b328eada0..d2f212dc8e72 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -577,6 +577,9 @@ config HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
 config HAVE_ARCH_HUGE_VMAP
 	bool
 
+config ARCH_WANT_HUGE_PMD_SHARE
+	bool
+
 config HAVE_ARCH_SOFT_DIRTY
 	bool
 
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 697ea0510729..c862575decd3 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -71,6 +71,7 @@ config ARM64
 	select ARCH_SUPPORTS_NUMA_BALANCING
 	select ARCH_WANT_COMPAT_IPC_PARSE_VERSION if COMPAT
 	select ARCH_WANT_FRAME_POINTERS
+	select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
 	select ARCH_HAS_UBSAN_SANITIZE_ALL
 	select ARM_AMBA
 	select ARM_ARCH_TIMER
@@ -901,9 +902,6 @@ config HW_PERF_EVENTS
 config SYS_SUPPORTS_HUGETLBFS
 	def_bool y
 
-config ARCH_WANT_HUGE_PMD_SHARE
-	def_bool y if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
-
 config ARCH_HAS_CACHE_LINE_SIZE
 	def_bool y
 
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 2bbbd4d1ba31..fa021ec38803 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -93,6 +93,7 @@ config X86
 	select ARCH_USE_QUEUED_SPINLOCKS
 	select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH
 	select ARCH_WANTS_DYNAMIC_TASK_STRUCT
+	select ARCH_WANT_HUGE_PMD_SHARE
 	select ARCH_WANTS_THP_SWAP		if X86_64
 	select BUILDTIME_EXTABLE_SORT
 	select CLKEVT_I8253
@@ -301,9 +302,6 @@ config ARCH_HIBERNATION_POSSIBLE
 config ARCH_SUSPEND_POSSIBLE
 	def_bool y
 
-config ARCH_WANT_HUGE_PMD_SHARE
-	def_bool y
-
 config ARCH_WANT_GENERAL_HUGETLB
 	def_bool y
 
-- 
2.20.1

[PATCH v3 2/2] riscv: Introduce huge page support for 32/64bit kernel

[Alexandre Ghiti]

This patch implements both 4MB huge page support for 32bit kernel
and 2MB/1GB huge pages support for 64bit kernel.

Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
---
 arch/riscv/Kconfig               |  8 ++++++
 arch/riscv/include/asm/hugetlb.h | 18 +++++++++++++
 arch/riscv/include/asm/page.h    | 10 ++++++++
 arch/riscv/include/asm/pgtable.h |  8 ++++--
 arch/riscv/mm/Makefile           |  2 ++
 arch/riscv/mm/hugetlbpage.c      | 44 ++++++++++++++++++++++++++++++++
 6 files changed, 88 insertions(+), 2 deletions(-)
 create mode 100644 arch/riscv/include/asm/hugetlb.h
 create mode 100644 arch/riscv/mm/hugetlbpage.c

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 0c4b12205632..43395f640955 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -50,6 +50,8 @@ config RISCV
 	select ARCH_HAS_PTE_SPECIAL
 	select ARCH_HAS_MMIOWB
 	select HAVE_EBPF_JIT if 64BIT
+	select ARCH_HAS_GIGANTIC_PAGE
+	select ARCH_WANT_HUGE_PMD_SHARE if 64BIT
 
 config MMU
 	def_bool y
@@ -64,6 +66,12 @@ config PAGE_OFFSET
 	default 0xffffffff80000000 if 64BIT && MAXPHYSMEM_2GB
 	default 0xffffffe000000000 if 64BIT && MAXPHYSMEM_128GB
 
+config ARCH_WANT_GENERAL_HUGETLB
+	def_bool y
+
+config SYS_SUPPORTS_HUGETLBFS
+	def_bool y
+
 config STACKTRACE_SUPPORT
 	def_bool y
 
diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h
new file mode 100644
index 000000000000..728a5db66597
--- /dev/null
+++ b/arch/riscv/include/asm/hugetlb.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_RISCV_HUGETLB_H
+#define _ASM_RISCV_HUGETLB_H
+
+#include <asm-generic/hugetlb.h>
+#include <asm/page.h>
+
+static inline int is_hugepage_only_range(struct mm_struct *mm,
+					 unsigned long addr,
+					 unsigned long len) {
+	return 0;
+}
+
+static inline void arch_clear_hugepage_flags(struct page *page)
+{
+}
+
+#endif /* _ASM_RISCV_HUGETLB_H */
diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h
index 8ddb6c7fedac..8551f69049ee 100644
--- a/arch/riscv/include/asm/page.h
+++ b/arch/riscv/include/asm/page.h
@@ -16,6 +16,16 @@
 #define PAGE_SIZE	(_AC(1, UL) << PAGE_SHIFT)
 #define PAGE_MASK	(~(PAGE_SIZE - 1))
 
+#ifdef CONFIG_64BIT
+#define HUGE_MAX_HSTATE		2
+#else
+#define HUGE_MAX_HSTATE		1
+#endif
+#define HPAGE_SHIFT		PMD_SHIFT
+#define HPAGE_SIZE		(_AC(1, UL) << HPAGE_SHIFT)
+#define HPAGE_MASK              (~(HPAGE_SIZE - 1))
+#define HUGETLB_PAGE_ORDER      (HPAGE_SHIFT - PAGE_SHIFT)
+
 /*
  * PAGE_OFFSET -- the first address of the first page of memory.
  * When not using MMU this corresponds to the first free page in
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index f7c3f7de15f2..18ea56fabba8 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -113,7 +113,6 @@ static inline void pmd_clear(pmd_t *pmdp)
 	set_pmd(pmdp, __pmd(0));
 }
 
-
 static inline pgd_t pfn_pgd(unsigned long pfn, pgprot_t prot)
 {
 	return __pgd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
@@ -250,6 +249,11 @@ static inline pte_t pte_mkspecial(pte_t pte)
 	return __pte(pte_val(pte) | _PAGE_SPECIAL);
 }
 
+static inline pte_t pte_mkhuge(pte_t pte)
+{
+	return pte;
+}
+
 /* Modify page protection bits */
 static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
 {
@@ -409,7 +413,7 @@ static inline void pgtable_cache_init(void)
 #define VMALLOC_START    (PAGE_OFFSET - VMALLOC_SIZE)
 
 /*
- * Task size is 0x40000000000 for RV64 or 0xb800000 for RV32.
+ * Task size is 0x4000000000 for RV64 or 0xb800000 for RV32.
  * Note that PGDIR_SIZE must evenly divide TASK_SIZE.
  */
 #ifdef CONFIG_64BIT
diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile
index fc51d3b7876e..74055e1d6f21 100644
--- a/arch/riscv/mm/Makefile
+++ b/arch/riscv/mm/Makefile
@@ -12,3 +12,5 @@ obj-y += ioremap.o
 obj-y += cacheflush.o
 obj-y += context.o
 obj-y += sifive_l2_cache.o
+
+obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c
new file mode 100644
index 000000000000..0d4747e9d5b5
--- /dev/null
+++ b/arch/riscv/mm/hugetlbpage.c
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/hugetlb.h>
+#include <linux/err.h>
+
+int pud_huge(pud_t pud)
+{
+	return pud_present(pud) &&
+		(pud_val(pud) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC));
+}
+
+int pmd_huge(pmd_t pmd)
+{
+	return pmd_present(pmd) &&
+		(pmd_val(pmd) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC));
+}
+
+static __init int setup_hugepagesz(char *opt)
+{
+	unsigned long ps = memparse(opt, &opt);
+
+	if (ps == HPAGE_SIZE) {
+		hugetlb_add_hstate(HPAGE_SHIFT - PAGE_SHIFT);
+	} else if (IS_ENABLED(CONFIG_64BIT) && ps == PUD_SIZE) {
+		hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+	} else {
+		hugetlb_bad_size();
+		pr_err("hugepagesz: Unsupported page size %lu M\n", ps >> 20);
+		return 0;
+	}
+
+	return 1;
+}
+__setup("hugepagesz=", setup_hugepagesz);
+
+#ifdef CONFIG_CONTIG_ALLOC
+static __init int gigantic_pages_init(void)
+{
+	/* With CONTIG_ALLOC, we can allocate gigantic pages at runtime */
+	if (IS_ENABLED(CONFIG_64BIT) && !size_to_hstate(1UL << PUD_SHIFT))
+		hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+	return 0;
+}
+arch_initcall(gigantic_pages_init);
+#endif
-- 
2.20.1

Re: [PATCH v3 1/2] x86, arm64: Move ARCH_WANT_HUGE_PMD_SHARE config in arch/Kconfig

[Paul Walmsley]

Catalin, Palmer,

On Mon, 1 Jul 2019, Alexandre Ghiti wrote:

> ARCH_WANT_HUGE_PMD_SHARE config was declared in both architectures:
> move this declaration in arch/Kconfig and make those architectures
> select it.
> 
> Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>

Since the change from v2 to v3 was minor (the removal of the "config 
ARCH_WANT_HUGE_PMD_SHARE" line from the arm64 port), I'm planning to 
apply your Reviewed-by:s and acks from 

https://lore.kernel.org/linux-riscv/20190603172723.GH63283@arrakis.emea.arm.com/

https://lore.kernel.org/linux-riscv/mhng-4d1d4acb-f65f-4ed4-bc86-85a14b7c3e16@palmer-si-x1e/

If there's any objection, please let me know as soon as possible.


- Paul

Re: [PATCH v3 1/2] x86, arm64: Move ARCH_WANT_HUGE_PMD_SHARE config in arch/Kconfig

[Hanjun Guo]

On 2019/7/2 1:58, Alexandre Ghiti wrote:
> ARCH_WANT_HUGE_PMD_SHARE config was declared in both architectures:
> move this declaration in arch/Kconfig and make those architectures
> select it.
> 
> Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
> ---
>  arch/Kconfig       | 3 +++
>  arch/arm64/Kconfig | 4 +---
>  arch/x86/Kconfig   | 4 +---
>  3 files changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/Kconfig b/arch/Kconfig
> index c47b328eada0..d2f212dc8e72 100644
> --- a/arch/Kconfig
> +++ b/arch/Kconfig
> @@ -577,6 +577,9 @@ config HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
>  config HAVE_ARCH_HUGE_VMAP
>  	bool
>  
> +config ARCH_WANT_HUGE_PMD_SHARE
> +	bool
> +
>  config HAVE_ARCH_SOFT_DIRTY
>  	bool
>  
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 697ea0510729..c862575decd3 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -71,6 +71,7 @@ config ARM64
>  	select ARCH_SUPPORTS_NUMA_BALANCING
>  	select ARCH_WANT_COMPAT_IPC_PARSE_VERSION if COMPAT
>  	select ARCH_WANT_FRAME_POINTERS
> +	select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
>  	select ARCH_HAS_UBSAN_SANITIZE_ALL
>  	select ARM_AMBA
>  	select ARM_ARCH_TIMER
> @@ -901,9 +902,6 @@ config HW_PERF_EVENTS
>  config SYS_SUPPORTS_HUGETLBFS
>  	def_bool y
>  
> -config ARCH_WANT_HUGE_PMD_SHARE
> -	def_bool y if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
> -

Reviewed-by: Hanjun Guo <guohanjun@huawei.com>

Thanks
Hanjun

Re: [PATCH v3 1/2] x86, arm64: Move ARCH_WANT_HUGE_PMD_SHARE config in arch/Kconfig

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v3 2/2] riscv: Introduce huge page support for 32/64bit kernel

[Christoph Hellwig]

> +config ARCH_WANT_GENERAL_HUGETLB
> +	def_bool y
> +
> +config SYS_SUPPORTS_HUGETLBFS
> +	def_bool y

In a perfect world these would be in mm/Kconfig and only selected
by the architectures.  But I don't want to force you to clean up all
that mess first, so:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v3 1/2] x86, arm64: Move ARCH_WANT_HUGE_PMD_SHARE config in arch/Kconfig

[Alexandre Ghiti]

On 7/1/19 8:51 PM, Paul Walmsley wrote:
> Catalin, Palmer,
>
> On Mon, 1 Jul 2019, Alexandre Ghiti wrote:
>
>> ARCH_WANT_HUGE_PMD_SHARE config was declared in both architectures:
>> move this declaration in arch/Kconfig and make those architectures
>> select it.
>>
>> Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
> Since the change from v2 to v3 was minor (the removal of the "config
> ARCH_WANT_HUGE_PMD_SHARE" line from the arm64 port), I'm planning to
> apply your Reviewed-by:s and acks from
>
> https://lore.kernel.org/linux-riscv/20190603172723.GH63283@arrakis.emea.arm.com/
>
> https://lore.kernel.org/linux-riscv/mhng-4d1d4acb-f65f-4ed4-bc86-85a14b7c3e16@palmer-si-x1e/


Ingo acked this patch too in that case. Sorry for that, I was unsure if 
I could add them.

Thanks for your answer,

Alex


>
> If there's any objection, please let me know as soon as possible.
>
>
> - Paul

Re: [PATCH v3 1/2] x86, arm64: Move ARCH_WANT_HUGE_PMD_SHARE config in arch/Kconfig

[Alexandre Ghiti]

On 7/2/19 3:27 AM, Hanjun Guo wrote:
> On 2019/7/2 1:58, Alexandre Ghiti wrote:
>> ARCH_WANT_HUGE_PMD_SHARE config was declared in both architectures:
>> move this declaration in arch/Kconfig and make those architectures
>> select it.
>>
>> Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
>> ---
>>   arch/Kconfig       | 3 +++
>>   arch/arm64/Kconfig | 4 +---
>>   arch/x86/Kconfig   | 4 +---
>>   3 files changed, 5 insertions(+), 6 deletions(-)
>>
>> diff --git a/arch/Kconfig b/arch/Kconfig
>> index c47b328eada0..d2f212dc8e72 100644
>> --- a/arch/Kconfig
>> +++ b/arch/Kconfig
>> @@ -577,6 +577,9 @@ config HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
>>   config HAVE_ARCH_HUGE_VMAP
>>   	bool
>>   
>> +config ARCH_WANT_HUGE_PMD_SHARE
>> +	bool
>> +
>>   config HAVE_ARCH_SOFT_DIRTY
>>   	bool
>>   
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index 697ea0510729..c862575decd3 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -71,6 +71,7 @@ config ARM64
>>   	select ARCH_SUPPORTS_NUMA_BALANCING
>>   	select ARCH_WANT_COMPAT_IPC_PARSE_VERSION if COMPAT
>>   	select ARCH_WANT_FRAME_POINTERS
>> +	select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
>>   	select ARCH_HAS_UBSAN_SANITIZE_ALL
>>   	select ARM_AMBA
>>   	select ARM_ARCH_TIMER
>> @@ -901,9 +902,6 @@ config HW_PERF_EVENTS
>>   config SYS_SUPPORTS_HUGETLBFS
>>   	def_bool y
>>   
>> -config ARCH_WANT_HUGE_PMD_SHARE
>> -	def_bool y if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
>> -
> Reviewed-by: Hanjun Guo <guohanjun@huawei.com>


Thanks for catching the previous mistake again and thanks for that,

Alex


>
> Thanks
> Hanjun
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 2/2] riscv: Introduce huge page support for 32/64bit kernel

[Alexandre Ghiti]

On 7/2/19 3:24 PM, Christoph Hellwig wrote:
>> +config ARCH_WANT_GENERAL_HUGETLB
>> +	def_bool y
>> +
>> +config SYS_SUPPORTS_HUGETLBFS
>> +	def_bool y
> In a perfect world these would be in mm/Kconfig and only selected
> by the architectures.  But I don't want to force you to clean up all
> that mess first, so:
>
> Reviewed-by: Christoph Hellwig <hch@lst.de>


Thanks, I'll clean that up ;)

Alex


>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 0/2] Hugetlbfs support for riscv

[Paul Walmsley]

Hi Alex,

Thanks for writing and testing these patches, and thanks for your patience 
waiting for reviews and testing.

On Mon, 1 Jul 2019, Alexandre Ghiti wrote:

> This series introduces hugetlbfs support for both riscv 32/64. Riscv32           
> is architecturally limited to huge pages of size 4MB whereas riscv64 has         
> 2MB/1G huge pages support. Transparent huge page support is not                  
> implemented here, I will submit another series later.                            
>                                                                                  

[ ... ]

> This series was validated using libhugetlbfs testsuite ported to riscv64         
> without linker script support.                                                   
> (https://github.com/AlexGhiti/libhugetlbfs.git, branch dev/alex/riscv).          
>                                                                                  
> - libhugetlbfs testsuite on riscv64/2M:                                          
>   - brk_near_huge triggers an assert in malloc.c, does not on x86.               

I was able to reproduce the 2MB megapages test results on rv64 QEMU.  On a 
HiFive Unleashed, though, a few more tests fail:

- icache_hygiene fails ("icache unclean")

  # LD_LIBRARY_PATH=obj64 ./tests/obj64/icache-hygiene
  Starting testcase "./tests/obj64/icache-hygiene", pid 732
  SIGILL at 0x15559fff80 (sig_expected=0x15559fff80)
  SIGILL at 0x1555dfff80 (sig_expected=0x1555dfff80)
  SIGILL at 0x15561fff80 (sig_expected=0x15561fff80)
  SIGILL at 0x15565fff80 (sig_expected=0x15565fff80)
  SIGILL at 0x15569fff80 (sig_expected=0x15569fff80)
  SIGILL at 0x1556dfff80 (sig_expected=(nil))
  FAIL   SIGILL somewhere unexpected
  #

- One of the heapshrink tests fails ("Heap did not shrink")

  # LD_PRELOAD="obj64/libhugetlbfs_privutils.so obj64/libhugetlbfs.so tests/obj64/libheapshrink.so" HUGETLB_MORECORE_SHRINK=yes HUGETLB_MORECORE=yes tests/obj64/heapshrink
  Starting testcase "tests/obj64/heapshrink", pid 753
  FAIL    Heap did not shrink
  #

Some of these may be related to the top-down mmap work, but there might be 
more work to do on actual hardware.

> - libhugetlbfs testsuite on riscv64/1G:                                          
>   - brk_near_huge triggers an assert in malloc.c, does not on x86.               
>   - mmap-gettest, mmap-cow: testsuite passes the number of default free          
>     pages as parameters and then fails for 1G which is not the default.          
>     Otherwise succeeds when given the right number of pages.                     
>   - map_high_truncate_2 fails on x86 too: 0x60000000 is not 1G aligned           
>     and fails at line 694 of fs/hugetlbfs/inode.c.                               
>   - heapshrink on 1G fails on x86 too, not investigated.                         
>   - counters.sh on 1G fails on x86 too: alloc_surplus_huge_page returns          
>     NULL in case of gigantic pages.                                              
>   - icache-hygiene succeeds after patch #3 of this series which lowers           
>     the base address of mmap.                                                    
>   - fallocate_stress.sh on 1G never ends, on x86 too, not investigated.          

I can reproduce some of these here on QEMU.  But for reasons that are 
unclear to me, 1G gigapages aren't working on the HiFive Unleashed here.

In any case, these patches are clearly a good start, so I've queued 
them for v5.3.


- Paul

Re: [PATCH v3 0/2] Hugetlbfs support for riscv

[Christoph Hellwig]

On Mon, Jul 01, 2019 at 01:58:58PM -0400, Alexandre Ghiti wrote:
>   - icache-hygiene succeeds after patch #3 of this series which lowers           
>     the base address of mmap.                                                    

I think іcache-hygiene will also need a call to riscv_flush_icache in
cacheflush().

Re: [PATCH v3 0/2] Hugetlbfs support for riscv

[Alexandre Ghiti]

On 7/4/19 12:57 AM, Paul Walmsley wrote:
> Hi Alex,
>
> Thanks for writing and testing these patches, and thanks for your patience
> waiting for reviews and testing.


No problem :)


> On Mon, 1 Jul 2019, Alexandre Ghiti wrote:
>
>> This series introduces hugetlbfs support for both riscv 32/64. Riscv32
>> is architecturally limited to huge pages of size 4MB whereas riscv64 has
>> 2MB/1G huge pages support. Transparent huge page support is not
>> implemented here, I will submit another series later.
>>                                                                                   
> [ ... ]
>
>> This series was validated using libhugetlbfs testsuite ported to riscv64
>> without linker script support.
>> (https://github.com/AlexGhiti/libhugetlbfs.git, branch dev/alex/riscv).
>>                                                                                   
>> - libhugetlbfs testsuite on riscv64/2M:
>>    - brk_near_huge triggers an assert in malloc.c, does not on x86.
> I was able to reproduce the 2MB megapages test results on rv64 QEMU.  On a
> HiFive Unleashed, though, a few more tests fail:
>
> - icache_hygiene fails ("icache unclean")
>
>    # LD_LIBRARY_PATH=obj64 ./tests/obj64/icache-hygiene
>    Starting testcase "./tests/obj64/icache-hygiene", pid 732
>    SIGILL at 0x15559fff80 (sig_expected=0x15559fff80)
>    SIGILL at 0x1555dfff80 (sig_expected=0x1555dfff80)
>    SIGILL at 0x15561fff80 (sig_expected=0x15561fff80)
>    SIGILL at 0x15565fff80 (sig_expected=0x15565fff80)
>    SIGILL at 0x15569fff80 (sig_expected=0x15569fff80)
>    SIGILL at 0x1556dfff80 (sig_expected=(nil))
>    FAIL   SIGILL somewhere unexpected
>    #
>
> - One of the heapshrink tests fails ("Heap did not shrink")
>
>    # LD_PRELOAD="obj64/libhugetlbfs_privutils.so obj64/libhugetlbfs.so tests/obj64/libheapshrink.so" HUGETLB_MORECORE_SHRINK=yes HUGETLB_MORECORE=yes tests/obj64/heapshrink
>    Starting testcase "tests/obj64/heapshrink", pid 753
>    FAIL    Heap did not shrink
>    #
>
> Some of these may be related to the top-down mmap work, but there might be
> more work to do on actual hardware.


I don't think this is related to top-down mmap layout, this test only 
mmaps a huge page.
It might be interesting to see more verbose messages adding 
HUGETLB_VERBOSE=99
when launching the test.


>
>> - libhugetlbfs testsuite on riscv64/1G:
>>    - brk_near_huge triggers an assert in malloc.c, does not on x86.
>>    - mmap-gettest, mmap-cow: testsuite passes the number of default free
>>      pages as parameters and then fails for 1G which is not the default.
>>      Otherwise succeeds when given the right number of pages.
>>    - map_high_truncate_2 fails on x86 too: 0x60000000 is not 1G aligned
>>      and fails at line 694 of fs/hugetlbfs/inode.c.
>>    - heapshrink on 1G fails on x86 too, not investigated.
>>    - counters.sh on 1G fails on x86 too: alloc_surplus_huge_page returns
>>      NULL in case of gigantic pages.
>>    - icache-hygiene succeeds after patch #3 of this series which lowers
>>      the base address of mmap.
>>    - fallocate_stress.sh on 1G never ends, on x86 too, not investigated.
> I can reproduce some of these here on QEMU.  But for reasons that are
> unclear to me, 1G gigapages aren't working on the HiFive Unleashed here.
>
> In any case, these patches are clearly a good start, so I've queued
> them for v5.3.
>
>
> - Paul
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 0/2] Hugetlbfs support for riscv

[Paul Walmsley]

On Thu, 4 Jul 2019, Alexandre Ghiti wrote:

> On 7/4/19 12:57 AM, Paul Walmsley wrote:
> > On Mon, 1 Jul 2019, Alexandre Ghiti wrote:
> > 
> > > - libhugetlbfs testsuite on riscv64/2M:
> > >    - brk_near_huge triggers an assert in malloc.c, does not on x86.
> > I was able to reproduce the 2MB megapages test results on rv64 QEMU.  On a
> > HiFive Unleashed, though, a few more tests fail:

[ ... ]

> > - One of the heapshrink tests fails ("Heap did not shrink")
> > 
> >    # LD_PRELOAD="obj64/libhugetlbfs_privutils.so obj64/libhugetlbfs.so
> > tests/obj64/libheapshrink.so" HUGETLB_MORECORE_SHRINK=yes
> > HUGETLB_MORECORE=yes tests/obj64/heapshrink
> >    Starting testcase "tests/obj64/heapshrink", pid 753
> >    FAIL    Heap did not shrink
> >    #
> > 
> > Some of these may be related to the top-down mmap work, but there might be
> > more work to do on actual hardware.
> 
> 
> I don't think this is related to top-down mmap layout, this test only 
> mmaps a huge page. It might be interesting to see more verbose messages 
> adding HUGETLB_VERBOSE=99 when launching the test.

Here is the HUGETLB_VERBOSE=99 output from the above heapshrink test on an 
FU540:

libhugetlbfs [(none):86]: INFO: Found pagesize 2048 kB
libhugetlbfs [(none):86]: INFO: Parsed kernel version: [5] . [2] . [0]  [pre-release: 6]
libhugetlbfs [(none):86]: INFO: Feature private_reservations is present in this kernel
libhugetlbfs [(none):86]: INFO: Feature noreserve_safe is present in this kernel
libhugetlbfs [(none):86]: INFO: Feature map_hugetlb is present in this kernel
libhugetlbfs [(none):86]: INFO: Kernel has MAP_PRIVATE reservations.  Disabling heap prefaulting.
libhugetlbfs [(none):86]: INFO: Kernel supports MAP_HUGETLB
libhugetlbfs [(none):86]: INFO: HUGETLB_SHARE=0, sharing disabled
libhugetlbfs [(none):86]: INFO: HUGETLB_NO_RESERVE=no, reservations enabled
libhugetlbfs [(none):86]: INFO: No segments were appropriate for remapping
libhugetlbfs [(none):86]: INFO: setup_morecore(): heapaddr = 0x2aaac00000
libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(1052672) = ...
libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaac00000, mapsize = 0, delta=1052672
libhugetlbfs [(none):86]: INFO: Attempting to map 2097152 bytes
libhugetlbfs [(none):86]: INFO: ... = 0x2aaac00000
libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(0) = ...
libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaad01000, mapsize = 200000, delta=-1044480
libhugetlbfs [(none):86]: INFO: ... = 0x2aaad01000
Starting testcase "tests/obj64/heapshrink", pid 86
libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(33558528) = ...
libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaad01000, mapsize = 200000, delta=32514048
libhugetlbfs [(none):86]: INFO: Attempting to map 33554432 bytes
libhugetlbfs [(none):86]: INFO: ... = 0x2aaad01000
FAIL    Heap did not shrink


This is with this hugepage configuration:

# /usr/local/bin/hugeadm --pool-list
      Size  Minimum  Current  Maximum  Default
   2097152       64       64       64        *
#


- Paul

Re: [PATCH v3 0/2] Hugetlbfs support for riscv

[Alex Ghiti]

On 7/4/19 7:35 AM, Paul Walmsley wrote:
> On Thu, 4 Jul 2019, Alexandre Ghiti wrote:
>
>> On 7/4/19 12:57 AM, Paul Walmsley wrote:
>>> On Mon, 1 Jul 2019, Alexandre Ghiti wrote:
>>>
>>>> - libhugetlbfs testsuite on riscv64/2M:
>>>>     - brk_near_huge triggers an assert in malloc.c, does not on x86.
>>> I was able to reproduce the 2MB megapages test results on rv64 QEMU.  On a
>>> HiFive Unleashed, though, a few more tests fail:
> [ ... ]
>
>>> - One of the heapshrink tests fails ("Heap did not shrink")
>>>
>>>     # LD_PRELOAD="obj64/libhugetlbfs_privutils.so obj64/libhugetlbfs.so
>>> tests/obj64/libheapshrink.so" HUGETLB_MORECORE_SHRINK=yes
>>> HUGETLB_MORECORE=yes tests/obj64/heapshrink
>>>     Starting testcase "tests/obj64/heapshrink", pid 753
>>>     FAIL    Heap did not shrink
>>>     #
>>>
>>> Some of these may be related to the top-down mmap work, but there might be
>>> more work to do on actual hardware.
>>
>> I don't think this is related to top-down mmap layout, this test only
>> mmaps a huge page. It might be interesting to see more verbose messages
>> adding HUGETLB_VERBOSE=99 when launching the test.
> Here is the HUGETLB_VERBOSE=99 output from the above heapshrink test on an
> FU540:
>
> libhugetlbfs [(none):86]: INFO: Found pagesize 2048 kB
> libhugetlbfs [(none):86]: INFO: Parsed kernel version: [5] . [2] . [0]  [pre-release: 6]
> libhugetlbfs [(none):86]: INFO: Feature private_reservations is present in this kernel
> libhugetlbfs [(none):86]: INFO: Feature noreserve_safe is present in this kernel
> libhugetlbfs [(none):86]: INFO: Feature map_hugetlb is present in this kernel
> libhugetlbfs [(none):86]: INFO: Kernel has MAP_PRIVATE reservations.  Disabling heap prefaulting.
> libhugetlbfs [(none):86]: INFO: Kernel supports MAP_HUGETLB
> libhugetlbfs [(none):86]: INFO: HUGETLB_SHARE=0, sharing disabled
> libhugetlbfs [(none):86]: INFO: HUGETLB_NO_RESERVE=no, reservations enabled
> libhugetlbfs [(none):86]: INFO: No segments were appropriate for remapping
> libhugetlbfs [(none):86]: INFO: setup_morecore(): heapaddr = 0x2aaac00000
> libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(1052672) = ...
> libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaac00000, mapsize = 0, delta=1052672
> libhugetlbfs [(none):86]: INFO: Attempting to map 2097152 bytes
> libhugetlbfs [(none):86]: INFO: ... = 0x2aaac00000
> libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(0) = ...
> libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaad01000, mapsize = 200000, delta=-1044480
> libhugetlbfs [(none):86]: INFO: ... = 0x2aaad01000
> Starting testcase "tests/obj64/heapshrink", pid 86
> libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(33558528) = ...
> libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaad01000, mapsize = 200000, delta=32514048
> libhugetlbfs [(none):86]: INFO: Attempting to map 33554432 bytes
> libhugetlbfs [(none):86]: INFO: ... = 0x2aaad01000
> FAIL    Heap did not shrink
>
>
> This is with this hugepage configuration:
>
> # /usr/local/bin/hugeadm --pool-list
>        Size  Minimum  Current  Maximum  Default
>     2097152       64       64       64        *
> #
>

Ok thanks for that, but it does not say much :)

While trying to understand why it may fail on HW, I actually failed to 
reproduce the results on qemu (I did not
check the results for v3 and I recently switched from yocto to buildroot 
so I lost my configuration...).

What configuration do you use to reproduce the results on qemu ?

FYI, while playing around, I noticed that with qemu v4.0.0, 
icache_hygiene stalls whereas with
v3.1.0, it does not but I did not investigate though.

Thanks,

Alex


> - Paul