Re: [RFC PATCH] mm: show mthp_fault_alloc and mthp_fault_fallback of multi-size THPs

From: Barry Song <21cnbao@gmail.com>
To: Matthew Wilcox <willy@infradead.org>,
	david@redhat.com, ryan.roberts@arm.com,  yuzhao@google.com
Cc: akpm@linux-foundation.org, linux-mm@kvack.org,
	kasong@tencent.com,  yosryahmed@google.com,
	cerasuolodomenico@gmail.com, surenb@google.com,
	 Barry Song <v-songbaohua@oppo.com>
Subject: Re: [RFC PATCH] mm: show mthp_fault_alloc and mthp_fault_fallback of multi-size THPs
Date: Wed, 27 Mar 2024 11:19:11 +1300	[thread overview]
Message-ID: <CAGsJ_4xxsVhfexnjby+CoVLu4ujTEsBh7k_xx+QiwH85NskS9Q@mail.gmail.com> (raw)
In-Reply-To: <CAGsJ_4xotrPX0Hz3A9Lm6hGVx=4vOZ75YwSsshsAN4pSiL2nxg@mail.gmail.com>

On Tue, Mar 26, 2024 at 4:40 PM Barry Song <21cnbao@gmail.com> wrote:
>
> On Tue, Mar 26, 2024 at 4:25 PM Matthew Wilcox <willy@infradead.org> wrote:
> >
> > On Tue, Mar 26, 2024 at 04:01:03PM +1300, Barry Song wrote:
> > > Profiling a system blindly with mTHP has become challenging due
> > > to the lack of visibility into its operations. While displaying
> > > additional statistics such as partial map/unmap actions may
> > > spark debate, presenting the success rate of mTHP allocations
> > > appears to be a straightforward and pressing need.
> >
> > Ummm ... no?  Not like this anyway.  It has the bad assumption that
> > "mTHP" only comes in one size.
>
>
> I had initially considered per-size allocation and fallback before sending
> the RFC. However, in order to prompt discussion and exploration
> into profiling possibilities, I opted to send the simplest code instead.
>
> We could consider two options for displaying per-size statistics.
>
> 1. A single file could be used to display data for all sizes.
> 1024KiB fault allocation:
> 1024KiB fault fallback:
> 512KiB fault allocation:
> 512KiB fault fallback:
> ....
> 64KiB fault allocation:
> 64KiB fault fallback:
>
> 2. A separate file for each size
> For example,
>
> /sys/kernel/debug/transparent_hugepage/hugepages-1024kB/vmstat
> /sys/kernel/debug/transparent_hugepage/hugepages-512kB/vmstat
> ...
> /sys/kernel/debug/transparent_hugepage/hugepages-64kB/vmstat
>

Hi Ryan, David, Willy, Yu,

I'm collecting feedback on whether you'd prefer access to something similar
to /sys/kernel/debug/transparent_hugepage/hugepages-<size>/stat to help
determine the direction to take for this patch.

This is important to us because we're keen on understanding how often
folios allocations fail on a system with limited memory, such as a phone.

Presently, I've observed a success rate of under 8% for 64KiB allocations.
Yet, integrating Yu's TAO optimization [1] and establishing an 800MiB
nomerge zone on a phone with 8GiB memory, there's a substantial
enhancement in the success rate, reaching approximately 40%. I'm still
fine-tuning the optimal size for the zone.

[1] https://lore.kernel.org/linux-mm/20240229183436.4110845-1-yuzhao@google.com/

> While the latter option may seem more appealing, it presents a challenge
> in situations where a 512kB allocation may fallback to 256kB, yet a separate
> 256kB allocation succeeds. Demonstrating the connection that the successful
> 256kB allocation is actually a fallback from the 512kB allocation can be complex
> especially if we begin to support per-VMA hints for mTHP sizes.
>

Thanks
Barry