Linux-mm Archive on lore.kernel.org
 help / color / Atom feed
From: David Hildenbrand <david@redhat.com>
To: Alexander Duyck <alexander.h.duyck@linux.intel.com>,
	Alexander Duyck <alexander.duyck@gmail.com>,
	kvm@vger.kernel.org, mst@redhat.com,
	linux-kernel@vger.kernel.org, willy@infradead.org,
	mhocko@kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org,
	mgorman@techsingularity.net, vbabka@suse.cz
Cc: yang.zhang.wz@gmail.com, nitesh@redhat.com,
	konrad.wilk@oracle.com, pagupta@redhat.com, riel@surriel.com,
	lcapitulino@redhat.com, dave.hansen@intel.com,
	wei.w.wang@intel.com, aarcange@redhat.com, pbonzini@redhat.com,
	dan.j.williams@intel.com, osalvador@suse.de
Subject: Re: [PATCH v14 0/6] mm / virtio: Provide support for unused page reporting
Date: Wed, 27 Nov 2019 18:37:21 +0100
Message-ID: <fd866a71-1d1a-1481-ffee-aefe0313ef38@redhat.com> (raw)
In-Reply-To: <57f4c78f298a5e3d929c0026f7b323a3bb911848.camel@linux.intel.com>

On 27.11.19 18:36, Alexander Duyck wrote:
> On Wed, 2019-11-27 at 11:01 +0100, David Hildenbrand wrote:
>> On 26.11.19 17:45, Alexander Duyck wrote:
>>> On Tue, 2019-11-26 at 13:20 +0100, David Hildenbrand wrote:
>>>> On 19.11.19 22:46, Alexander Duyck wrote:
> 
> <snip>
> 
>>>>> Below are the results from various benchmarks. I primarily focused on two
>>>>> tests. The first is the will-it-scale/page_fault2 test, and the other is
>>>>> a modified version of will-it-scale/page_fault1 that was enabled to use
>>>>> THP. I did this as it allows for better visibility into different parts
>>>>> of the memory subsystem. The guest is running with 32G for RAM on one
>>>>> node of a E5-2630 v3. The host has had some power saving features disabled
>>>>> by setting the /dev/cpu_dma_latency value to 10ms.
>>>>>
>>>>> Test                page_fault1 (THP)     page_fault2
>>>>> Name         tasks  Process Iter  STDEV  Process Iter  STDEV
>>>>> Baseline         1    1203934.75  0.04%     379940.75  0.11%
>>>>>                 16    8828217.00  0.85%    3178653.00  1.28%
>>>>>
>>>>> Patches applied  1    1207961.25  0.10%     380852.25  0.25%
>>>>>                 16    8862373.00  0.98%    3246397.25  0.68%
>>>>>
>>>>> Patches enabled  1    1207758.75  0.17%     373079.25  0.60%
>>>>>  MADV disabled  16    8870373.75  0.29%    3204989.75  1.08%
>>>>>
>>>>> Patches enabled  1    1261183.75  0.39%     373201.50  0.50%
>>>>>                 16    8371359.75  0.65%    3233665.50  0.84%
>>>>>
>>>>> Patches enabled  1    1090201.50  0.25%     376967.25  0.29%
>>>>>  page shuffle   16    8108719.75  0.58%    3218450.25  1.07%
>>>>>
>>>>> The results above are for a baseline with a linux-next-20191115 kernel,
>>>>> that kernel with this patch set applied but page reporting disabled in
>>>>> virtio-balloon, patches applied but the madvise disabled by direct
>>>>> assigning a device, the patches applied and page reporting fully
>>>>> enabled, and the patches enabled with page shuffling enabled.  These
>>>>> results include the deviation seen between the average value reported here
>>>>> versus the high and/or low value. I observed that during the test memory
>>>>> usage for the first three tests never dropped whereas with the patches
>>>>> fully enabled the VM would drop to using only a few GB of the host's
>>>>> memory when switching from memhog to page fault tests.
>>>>>
>>>>> Most of the overhead seen with this patch set enabled seems due to page
>>>>> faults caused by accessing the reported pages and the host zeroing the page
>>>>> before giving it back to the guest. This overhead is much more visible when
>>>>> using THP than with standard 4K pages. In addition page shuffling seemed to
>>>>> increase the amount of faults generated due to an increase in memory churn.
>>>>
>>>> MADV_FREE would be interesting.
>>>
>>> I can probably code something up. However that is going to push a bunch of
>>> complexity into the QEMU code and doesn't really mean much to the kernel
>>> code. I can probably add it as another QEMU patch to the set since it is
>>> just a matter of having a function similar to ram_block_discard_range that
>>> uses MADV_FREE instead of MADV_DONTNEED.
>>
>> Yes, addon patch makes perfect sense. The nice thing about MADV_FREE is
>> that you only take back pages from a process when really under memory
>> pressure (before going to SWAP). You will still get a pagefault on the
>> next access (to identify that the page is still in use after all), but
>> don't have to fault in a fresh page.
> 
> So I got things running with a proof of concept using MADV_FREE.
> Apparently another roadblock I hadn't realized is that you have to have
> the right version of glibc for MADV_FREE to be present.
> 
> Anyway with MADV_FREE the numbers actually look pretty close to the
> numbers with the madvise disabled. Apparently the page fault overhead
> isn't all that significant. When I push the next patch set I will include
> the actual numbers, but even with shuffling enabled the results were in
> the 8.7 to 8.8 million iteration range.
> 

Cool, thanks for evaluating!

-- 
Thanks,

David / dhildenb



      reply index

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-19 21:46 Alexander Duyck
2019-11-19 21:46 ` [PATCH v14 1/6] mm: Adjust shuffle code to allow for future coalescing Alexander Duyck
2019-11-27 13:52   ` Mel Gorman
2019-11-19 21:46 ` [PATCH v14 2/6] mm: Use zone and order instead of free area in free_list manipulators Alexander Duyck
2019-11-27 13:54   ` Mel Gorman
2019-11-19 21:46 ` [PATCH v14 3/6] mm: Introduce Reported pages Alexander Duyck
2019-11-27 15:24   ` Mel Gorman
2019-11-27 17:22     ` Alexander Duyck
2019-11-27 18:35       ` Mel Gorman
2019-11-27 21:55         ` Alexander Duyck
2019-11-28  9:22           ` Mel Gorman
2019-11-29 19:25             ` Alexander Duyck
2019-11-19 21:46 ` [PATCH v14 4/6] mm: Add unused page reporting documentation Alexander Duyck
2019-11-19 21:46 ` [PATCH v14 5/6] virtio-balloon: Pull page poisoning config out of free page hinting Alexander Duyck
2019-11-19 21:46 ` [PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host Alexander Duyck
2019-11-28 15:25   ` David Hildenbrand
2019-11-28 17:00     ` Michael S. Tsirkin
2019-12-04 17:48       ` Alexander Duyck
2019-12-04 17:53       ` Alexander Duyck
2019-11-29 21:13     ` Alexander Duyck
2019-12-01 11:46       ` Michael S. Tsirkin
2019-12-01 18:25         ` Alexander Duyck
2019-12-02 10:43       ` David Hildenbrand
2019-11-19 21:54 ` [PATCH v14 QEMU 1/3] virtio-ballon: Implement support for page poison tracking feature Alexander Duyck
2019-11-19 21:54 ` [PATCH v14 QEMU 2/3] virtio-balloon: Add bit to notify guest of unused page reporting Alexander Duyck
2019-11-19 21:54 ` [PATCH v14 QEMU 3/3] virtio-balloon: Provide a interface for " Alexander Duyck
2019-11-26 12:20 ` [PATCH v14 0/6] mm / virtio: Provide support " David Hildenbrand
2019-11-26 16:45   ` Alexander Duyck
2019-11-27 10:01     ` David Hildenbrand
2019-11-27 17:36       ` Alexander Duyck
2019-11-27 17:37         ` David Hildenbrand [this message]

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=fd866a71-1d1a-1481-ffee-aefe0313ef38@redhat.com \
    --to=david@redhat.com \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=alexander.duyck@gmail.com \
    --cc=alexander.h.duyck@linux.intel.com \
    --cc=dan.j.williams@intel.com \
    --cc=dave.hansen@intel.com \
    --cc=konrad.wilk@oracle.com \
    --cc=kvm@vger.kernel.org \
    --cc=lcapitulino@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@techsingularity.net \
    --cc=mhocko@kernel.org \
    --cc=mst@redhat.com \
    --cc=nitesh@redhat.com \
    --cc=osalvador@suse.de \
    --cc=pagupta@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=riel@surriel.com \
    --cc=vbabka@suse.cz \
    --cc=wei.w.wang@intel.com \
    --cc=willy@infradead.org \
    --cc=yang.zhang.wz@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-mm Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \
		linux-mm@kvack.org
	public-inbox-index linux-mm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kvack.linux-mm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git