Re: [PATCH v3 0/2] Hugetlbfs support for riscv

From: Alex Ghiti <alex@ghiti.fr>
To: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Hanjun Guo <guohanjun@huawei.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Palmer Dabbelt <palmer@sifive.com>,
	Will Deacon <will.deacon@arm.com>,
	x86@kernel.org, linux-kernel@vger.kernel.org,
	Christoph Hellwig <hch@infradead.org>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	linux-riscv@lists.infradead.org,
	linux-arm-kernel@lists.infradead.org,
	Mike Kravetz <mike.kravetz@oracle.com>
Subject: Re: [PATCH v3 0/2] Hugetlbfs support for riscv
Date: Sun, 7 Jul 2019 11:14:21 -0400	[thread overview]
Message-ID: <040f378d-e483-fa3a-28f4-fdb1bb62591d@ghiti.fr> (raw)
In-Reply-To: <alpine.DEB.2.21.9999.1907040429170.24872@viisi.sifive.com>

On 7/4/19 7:35 AM, Paul Walmsley wrote:
> On Thu, 4 Jul 2019, Alexandre Ghiti wrote:
>
>> On 7/4/19 12:57 AM, Paul Walmsley wrote:
>>> On Mon, 1 Jul 2019, Alexandre Ghiti wrote:
>>>
>>>> - libhugetlbfs testsuite on riscv64/2M:
>>>>     - brk_near_huge triggers an assert in malloc.c, does not on x86.
>>> I was able to reproduce the 2MB megapages test results on rv64 QEMU.  On a
>>> HiFive Unleashed, though, a few more tests fail:
> [ ... ]
>
>>> - One of the heapshrink tests fails ("Heap did not shrink")
>>>
>>>     # LD_PRELOAD="obj64/libhugetlbfs_privutils.so obj64/libhugetlbfs.so
>>> tests/obj64/libheapshrink.so" HUGETLB_MORECORE_SHRINK=yes
>>> HUGETLB_MORECORE=yes tests/obj64/heapshrink
>>>     Starting testcase "tests/obj64/heapshrink", pid 753
>>>     FAIL    Heap did not shrink
>>>     #
>>>
>>> Some of these may be related to the top-down mmap work, but there might be
>>> more work to do on actual hardware.
>>
>> I don't think this is related to top-down mmap layout, this test only
>> mmaps a huge page. It might be interesting to see more verbose messages
>> adding HUGETLB_VERBOSE=99 when launching the test.
> Here is the HUGETLB_VERBOSE=99 output from the above heapshrink test on an
> FU540:
>
> libhugetlbfs [(none):86]: INFO: Found pagesize 2048 kB
> libhugetlbfs [(none):86]: INFO: Parsed kernel version: [5] . [2] . [0]  [pre-release: 6]
> libhugetlbfs [(none):86]: INFO: Feature private_reservations is present in this kernel
> libhugetlbfs [(none):86]: INFO: Feature noreserve_safe is present in this kernel
> libhugetlbfs [(none):86]: INFO: Feature map_hugetlb is present in this kernel
> libhugetlbfs [(none):86]: INFO: Kernel has MAP_PRIVATE reservations.  Disabling heap prefaulting.
> libhugetlbfs [(none):86]: INFO: Kernel supports MAP_HUGETLB
> libhugetlbfs [(none):86]: INFO: HUGETLB_SHARE=0, sharing disabled
> libhugetlbfs [(none):86]: INFO: HUGETLB_NO_RESERVE=no, reservations enabled
> libhugetlbfs [(none):86]: INFO: No segments were appropriate for remapping
> libhugetlbfs [(none):86]: INFO: setup_morecore(): heapaddr = 0x2aaac00000
> libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(1052672) = ...
> libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaac00000, mapsize = 0, delta=1052672
> libhugetlbfs [(none):86]: INFO: Attempting to map 2097152 bytes
> libhugetlbfs [(none):86]: INFO: ... = 0x2aaac00000
> libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(0) = ...
> libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaad01000, mapsize = 200000, delta=-1044480
> libhugetlbfs [(none):86]: INFO: ... = 0x2aaad01000
> Starting testcase "tests/obj64/heapshrink", pid 86
> libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(33558528) = ...
> libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaad01000, mapsize = 200000, delta=32514048
> libhugetlbfs [(none):86]: INFO: Attempting to map 33554432 bytes
> libhugetlbfs [(none):86]: INFO: ... = 0x2aaad01000
> FAIL    Heap did not shrink
>
>
> This is with this hugepage configuration:
>
> # /usr/local/bin/hugeadm --pool-list
>        Size  Minimum  Current  Maximum  Default
>     2097152       64       64       64        *
> #
>

Ok thanks for that, but it does not say much :)

While trying to understand why it may fail on HW, I actually failed to 
reproduce the results on qemu (I did not
check the results for v3 and I recently switched from yocto to buildroot 
so I lost my configuration...).

What configuration do you use to reproduce the results on qemu ?

FYI, while playing around, I noticed that with qemu v4.0.0, 
icache_hygiene stalls whereas with
v3.1.0, it does not but I did not investigate though.

Thanks,

Alex

> - Paul

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv