Re: What size anonymous folios should we allocate?

[Hillf Danton <hdanton@sina.com>]

On 28 Mar 2023 11:12:54 +0100 Ryan Roberts <ryan.roberts@arm.com>
> On 27/03/2023 16:48, Vlastimil Babka wrote:
> > 
> > Hm if it's 4 entries on arm64 and presumably 8 on AMD, maybe we can only
> > care about how actively accessed are the individual "subpages" above that
> > size, to avoid dealing with this uncertainty whether HW tracks them. At such
> > smallish sizes we shouldn't induce massive overhead?
> 
> I'm not sure I've fully understood this point. For arm64's HPA, there is no
> "uncertainty [about] whether HW tracks them"; HW will always track access/dirty
> individually for each base page. The problem is the inverse; if SW (or HW) sets
> those bits differently in each page, then TLB coalescing performance may
> decrease. Or are you actually suggesting that SW should always set the bits the
> same for a 4 or 8 page run, and forgo the extra granularity?

That inverse side looks like a bleak cloud above anon order-5/6 pages
for instance.

> >> I'm hearing that there are workloads where being able to use the contiguous bit
> >> really does make a difference, so I would like to explore solutions that can
> >> work when we only have access/dirty at the folio level.
> > 
> > And on the higher orders where we have explicit control via bits, we could
> > split the explicitly contiguous mappings once in a while to determine if the
> > sub-folios are still accessed? Although maybe with 16x4kB pages limit it may
> > still be not worth the trouble?
> 
> I have a bigger-picture question; why is it useful to split these large folios?
> I think there are 2 potential reasons (but would like to be educated):
> 
> 1. If a set of sub-pages that were pre-faulted as part of a large folio have
> _never_ been accessed and we are under memory pressure, I guess we would like to
> split the folio and free those pages?
> 
> 2. If a set of subpages within a folio are cold (but were written in the past)
> and a separate set of subpages within the same folio are hot and we are under
> memory pressure, we would like to swap out the cold pages?
> 
> If the first reason is important, I guess we would want to initially map
> non-contig, then only remap as contig once every subpage has been touched at
> least once.
> 
> For the second reason, my intuition says that a conceptual single access and
> dirty bit per folio should be sufficient, and folios could be split from
> time-to-time to see if one half is cold?

It makes no sense to detect hot pages at the cost of order-5 compound page
particularly given the bleakness above.