Re: [RFC][Patch v9 0/6] KVM: Guest Free Page Hinting

[Alexander Duyck <alexander.duyck@gmail.com>]

On Thu, Mar 7, 2019 at 10:53 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Mar 07, 2019 at 10:45:58AM -0800, Alexander Duyck wrote:
> > To that end what I think w may want to do is instead just walk the LRU
> > list for a given zone/order in reverse order so that we can try to
> > identify the pages that are most likely to be cold and unused and
> > those are the first ones we want to be hinting on rather than the ones
> > that were just freed. If we can look at doing something like adding a
> > jiffies value to the page indicating when it was last freed we could
> > even have a good point for determining when we should stop processing
> > pages in a given zone/order list.
> >
> > In reality the approach wouldn't be too different from what you are
> > doing now, the only real difference would be that we would just want
> > to walk the LRU list for the given zone/order rather then pulling
> > hints on what to free from the calls to free_one_page. In addition we
> > would need to add a couple bits to indicate if the page has been
> > hinted on, is in the middle of getting hinted on, and something such
> > as the jiffies value I mentioned which we could use to determine how
> > old the page is.
>
> Do we really need bits in the page?
> Would it be bad to just have a separate hint list?

The issue is lists are expensive to search. If we have a single bit in
the page we can check it as soon as we have the page.

> If you run out of free memory you can check the hint
> list, if you find stuff there you can spin
> or kick the hypervisor to hurry up.

This implies you are keeping a separate list of pages for what has
been hinted on. If we are pulling pages out of the LRU list for that
it will require the zone lock to move the pages back and forth and for
higher core counts that isn't going to scale very well, and if you are
trying to pull out a page that is currently being hinted on you will
run into the same issue of having to wait for the hint to be completed
before proceeding.

> Core mm/ changes, so nothing's easy, I know.

We might be able to reuse some existing page flags. For example, there
is the PG_young and PG_idle flags that would actually be a pretty good
fit in terms of what we are looking for in behavior. We could set
PG_young when the page is initially freed, then clear it when we start
to perform the hint, and set PG_idle once the hint has been completed.

The check for if we could use a page would be pretty fast as a result
as well since if PG_young or PG_idle are set it means the page is free
to use so the check in arch_alloc_page would be pretty cheap since we
could probably test for both bits in one read.