From: Nitesh Narayan Lal <nitesh@redhat.com>
To: Alexander Duyck <alexander.duyck@gmail.com>
Cc: kvm list <kvm@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
linux-mm <linux-mm@kvack.org>,
Paolo Bonzini <pbonzini@redhat.com>,
lcapitulino@redhat.com, pagupta@redhat.com, wei.w.wang@intel.com,
Yang Zhang <yang.zhang.wz@gmail.com>,
Rik van Riel <riel@surriel.com>,
David Hildenbrand <david@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
dodgen@google.com, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
dhildenb@redhat.com, Andrea Arcangeli <aarcange@redhat.com>
Subject: Re: [RFC][Patch v10 1/2] mm: page_hinting: core infrastructure
Date: Tue, 4 Jun 2019 12:42:21 -0400 [thread overview]
Message-ID: <13b96507-6347-1702-7822-6efb0f1bbf20@redhat.com> (raw)
In-Reply-To: <CAKgT0UeMpcckGpT6OnC2kqgtyT2p9bvNgE2C0eqW1GOJTU-DHA@mail.gmail.com>
[-- Attachment #1.1: Type: text/plain, Size: 4212 bytes --]
On 6/4/19 12:25 PM, Alexander Duyck wrote:
> On Tue, Jun 4, 2019 at 9:08 AM Nitesh Narayan Lal <nitesh@redhat.com> wrote:
>>
>> On 6/4/19 11:14 AM, Alexander Duyck wrote:
>>> On Tue, Jun 4, 2019 at 5:55 AM Nitesh Narayan Lal <nitesh@redhat.com> wrote:
>>>> On 6/3/19 3:04 PM, Alexander Duyck wrote:
>>>>> On Mon, Jun 3, 2019 at 10:04 AM Nitesh Narayan Lal <nitesh@redhat.com> wrote:
>>>>>> This patch introduces the core infrastructure for free page hinting in
>>>>>> virtual environments. It enables the kernel to track the free pages which
>>>>>> can be reported to its hypervisor so that the hypervisor could
>>>>>> free and reuse that memory as per its requirement.
>>>>>>
>>>>>> While the pages are getting processed in the hypervisor (e.g.,
>>>>>> via MADV_FREE), the guest must not use them, otherwise, data loss
>>>>>> would be possible. To avoid such a situation, these pages are
>>>>>> temporarily removed from the buddy. The amount of pages removed
>>>>>> temporarily from the buddy is governed by the backend(virtio-balloon
>>>>>> in our case).
>>>>>>
>>>>>> To efficiently identify free pages that can to be hinted to the
>>>>>> hypervisor, bitmaps in a coarse granularity are used. Only fairly big
>>>>>> chunks are reported to the hypervisor - especially, to not break up THP
>>>>>> in the hypervisor - "MAX_ORDER - 2" on x86, and to save space. The bits
>>>>>> in the bitmap are an indication whether a page *might* be free, not a
>>>>>> guarantee. A new hook after buddy merging sets the bits.
>>>>>>
>>>>>> Bitmaps are stored per zone, protected by the zone lock. A workqueue
>>>>>> asynchronously processes the bitmaps, trying to isolate and report pages
>>>>>> that are still free. The backend (virtio-balloon) is responsible for
>>>>>> reporting these batched pages to the host synchronously. Once reporting/
>>>>>> freeing is complete, isolated pages are returned back to the buddy.
>>>>>>
>>>>>> There are still various things to look into (e.g., memory hotplug, more
>>>>>> efficient locking, possible races when disabling).
>>>>>>
>>>>>> Signed-off-by: Nitesh Narayan Lal <nitesh@redhat.com>
>>>>> So one thing I had thought about, that I don't believe that has been
>>>>> addressed in your solution, is to determine a means to guarantee
>>>>> forward progress. If you have a noisy thread that is allocating and
>>>>> freeing some block of memory repeatedly you will be stuck processing
>>>>> that and cannot get to the other work. Specifically if you have a zone
>>>>> where somebody is just cycling the number of pages needed to fill your
>>>>> hinting queue how do you get around it and get to the data that is
>>>>> actually code instead of getting stuck processing the noise?
>>>> It should not matter. As every time the memory threshold is met, entire
>>>> bitmap
>>>> is scanned and not just a chunk of memory for possible isolation. This
>>>> will guarantee
>>>> forward progress.
>>> So I think there may still be some issues. I see how you go from the
>>> start to the end, but how to you loop back to the start again as pages
>>> are added? The init_hinting_wq doesn't seem to have a way to get back
>>> to the start again if there is still work to do after you have
>>> completed your pass without queue_work_on firing off another thread.
>>>
>> That will be taken care as the part of a new job, which will be
>> en-queued as soon
>> as the free memory count for the respective zone will reach the threshold.
> So does that mean that you have multiple threads all calling
> queue_work_on until you get below the threshold?
Every time a page of order MAX_ORDER - 2 is added to the buddy, free
memory count will be incremented if the bit is not already set and its
value will be checked against the threshold.
> If so it seems like
> that would get expensive since that is an atomic test and set
> operation that would be hammered until you get below that threshold.
Not sure if I understood "until you get below that threshold".
Can you please explain?
test_and_set_bit() will be called every time a page with MAX_ORDER -2
order is added to the buddy. (Not already hinted)
--
Regards
Nitesh
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2019-06-04 16:42 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-03 17:03 [RFC][Patch v10 0/2] mm: Support for page hinting Nitesh Narayan Lal
2019-06-03 17:03 ` [RFC][Patch v10 1/2] mm: page_hinting: core infrastructure Nitesh Narayan Lal
2019-06-03 19:04 ` Alexander Duyck
2019-06-04 12:55 ` Nitesh Narayan Lal
2019-06-04 15:14 ` Alexander Duyck
2019-06-04 16:07 ` Nitesh Narayan Lal
2019-06-04 16:25 ` Alexander Duyck
2019-06-04 16:42 ` Nitesh Narayan Lal [this message]
2019-06-04 17:12 ` Alexander Duyck
2019-06-03 19:57 ` David Hildenbrand
2019-06-04 13:16 ` Nitesh Narayan Lal
2019-06-14 7:24 ` David Hildenbrand
2019-06-03 17:03 ` [RFC][Patch v10 2/2] virtio-balloon: page_hinting: reporting to the host Nitesh Narayan Lal
2019-06-03 22:38 ` Alexander Duyck
2019-06-04 7:12 ` David Hildenbrand
2019-06-04 11:50 ` Nitesh Narayan Lal
2019-06-04 11:31 ` Nitesh Narayan Lal
2019-06-04 16:33 ` Alexander Duyck
2019-06-04 16:44 ` Nitesh Narayan Lal
2019-06-03 17:04 ` [QEMU PATCH] KVM: Support for page hinting Nitesh Narayan Lal
2019-06-03 18:34 ` Alexander Duyck
2019-06-03 18:37 ` Nitesh Narayan Lal
2019-06-03 18:45 ` Nitesh Narayan Lal
2019-06-04 16:41 ` Alexander Duyck
2019-06-04 16:48 ` Nitesh Narayan Lal
2019-06-03 18:04 ` [RFC][Patch v10 0/2] mm: " Michael S. Tsirkin
2019-06-03 18:38 ` Nitesh Narayan Lal
2019-06-11 12:19 ` Nitesh Narayan Lal
2019-06-11 15:00 ` Alexander Duyck
2019-06-25 14:48 ` Nitesh Narayan Lal
2019-06-25 17:10 ` Alexander Duyck
[not found] ` <cc20a6d2-9e95-3de4-301a-f2a6a5b025e4@redhat.com>
2019-06-28 18:25 ` Alexander Duyck
2019-06-28 19:13 ` Nitesh Narayan Lal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=13b96507-6347-1702-7822-6efb0f1bbf20@redhat.com \
--to=nitesh@redhat.com \
--cc=aarcange@redhat.com \
--cc=alexander.duyck@gmail.com \
--cc=david@redhat.com \
--cc=dhildenb@redhat.com \
--cc=dodgen@google.com \
--cc=konrad.wilk@oracle.com \
--cc=kvm@vger.kernel.org \
--cc=lcapitulino@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mst@redhat.com \
--cc=pagupta@redhat.com \
--cc=pbonzini@redhat.com \
--cc=riel@surriel.com \
--cc=wei.w.wang@intel.com \
--cc=yang.zhang.wz@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).