From: Alexander Duyck <firstname.lastname@example.org> To: David Hildenbrand <email@example.com>, Alexander Duyck <firstname.lastname@example.org>, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com Cc: firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com Subject: Re: [PATCH v14 0/6] mm / virtio: Provide support for unused page reporting Date: Wed, 27 Nov 2019 09:36:06 -0800 Message-ID: <firstname.lastname@example.org> (raw) In-Reply-To: <email@example.com> On Wed, 2019-11-27 at 11:01 +0100, David Hildenbrand wrote: > On 26.11.19 17:45, Alexander Duyck wrote: > > On Tue, 2019-11-26 at 13:20 +0100, David Hildenbrand wrote: > > > On 19.11.19 22:46, Alexander Duyck wrote: <snip> > > > > Below are the results from various benchmarks. I primarily focused on two > > > > tests. The first is the will-it-scale/page_fault2 test, and the other is > > > > a modified version of will-it-scale/page_fault1 that was enabled to use > > > > THP. I did this as it allows for better visibility into different parts > > > > of the memory subsystem. The guest is running with 32G for RAM on one > > > > node of a E5-2630 v3. The host has had some power saving features disabled > > > > by setting the /dev/cpu_dma_latency value to 10ms. > > > > > > > > Test page_fault1 (THP) page_fault2 > > > > Name tasks Process Iter STDEV Process Iter STDEV > > > > Baseline 1 1203934.75 0.04% 379940.75 0.11% > > > > 16 8828217.00 0.85% 3178653.00 1.28% > > > > > > > > Patches applied 1 1207961.25 0.10% 380852.25 0.25% > > > > 16 8862373.00 0.98% 3246397.25 0.68% > > > > > > > > Patches enabled 1 1207758.75 0.17% 373079.25 0.60% > > > > MADV disabled 16 8870373.75 0.29% 3204989.75 1.08% > > > > > > > > Patches enabled 1 1261183.75 0.39% 373201.50 0.50% > > > > 16 8371359.75 0.65% 3233665.50 0.84% > > > > > > > > Patches enabled 1 1090201.50 0.25% 376967.25 0.29% > > > > page shuffle 16 8108719.75 0.58% 3218450.25 1.07% > > > > > > > > The results above are for a baseline with a linux-next-20191115 kernel, > > > > that kernel with this patch set applied but page reporting disabled in > > > > virtio-balloon, patches applied but the madvise disabled by direct > > > > assigning a device, the patches applied and page reporting fully > > > > enabled, and the patches enabled with page shuffling enabled. These > > > > results include the deviation seen between the average value reported here > > > > versus the high and/or low value. I observed that during the test memory > > > > usage for the first three tests never dropped whereas with the patches > > > > fully enabled the VM would drop to using only a few GB of the host's > > > > memory when switching from memhog to page fault tests. > > > > > > > > Most of the overhead seen with this patch set enabled seems due to page > > > > faults caused by accessing the reported pages and the host zeroing the page > > > > before giving it back to the guest. This overhead is much more visible when > > > > using THP than with standard 4K pages. In addition page shuffling seemed to > > > > increase the amount of faults generated due to an increase in memory churn. > > > > > > MADV_FREE would be interesting. > > > > I can probably code something up. However that is going to push a bunch of > > complexity into the QEMU code and doesn't really mean much to the kernel > > code. I can probably add it as another QEMU patch to the set since it is > > just a matter of having a function similar to ram_block_discard_range that > > uses MADV_FREE instead of MADV_DONTNEED. > > Yes, addon patch makes perfect sense. The nice thing about MADV_FREE is > that you only take back pages from a process when really under memory > pressure (before going to SWAP). You will still get a pagefault on the > next access (to identify that the page is still in use after all), but > don't have to fault in a fresh page. So I got things running with a proof of concept using MADV_FREE. Apparently another roadblock I hadn't realized is that you have to have the right version of glibc for MADV_FREE to be present. Anyway with MADV_FREE the numbers actually look pretty close to the numbers with the madvise disabled. Apparently the page fault overhead isn't all that significant. When I push the next patch set I will include the actual numbers, but even with shuffling enabled the results were in the 8.7 to 8.8 million iteration range.
next prev parent reply index Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-11-19 21:46 Alexander Duyck 2019-11-19 21:46 ` [PATCH v14 1/6] mm: Adjust shuffle code to allow for future coalescing Alexander Duyck 2019-11-27 13:52 ` Mel Gorman 2019-11-19 21:46 ` [PATCH v14 2/6] mm: Use zone and order instead of free area in free_list manipulators Alexander Duyck 2019-11-27 13:54 ` Mel Gorman 2019-11-19 21:46 ` [PATCH v14 3/6] mm: Introduce Reported pages Alexander Duyck 2019-11-27 15:24 ` Mel Gorman 2019-11-27 17:22 ` Alexander Duyck 2019-11-27 18:35 ` Mel Gorman 2019-11-27 21:55 ` Alexander Duyck 2019-11-28 9:22 ` Mel Gorman 2019-11-29 19:25 ` Alexander Duyck 2019-11-19 21:46 ` [PATCH v14 4/6] mm: Add unused page reporting documentation Alexander Duyck 2019-11-19 21:46 ` [PATCH v14 5/6] virtio-balloon: Pull page poisoning config out of free page hinting Alexander Duyck 2019-11-19 21:46 ` [PATCH v14 6/6] virtio-balloon: Add support for providing unused page reports to host Alexander Duyck 2019-11-28 15:25 ` David Hildenbrand 2019-11-28 17:00 ` Michael S. Tsirkin 2019-12-04 17:48 ` Alexander Duyck 2019-12-04 17:53 ` Alexander Duyck 2019-11-29 21:13 ` Alexander Duyck 2019-12-01 11:46 ` Michael S. Tsirkin 2019-12-01 18:25 ` Alexander Duyck 2019-12-02 10:43 ` David Hildenbrand 2019-11-19 21:54 ` [PATCH v14 QEMU 1/3] virtio-ballon: Implement support for page poison tracking feature Alexander Duyck 2019-11-19 21:54 ` [PATCH v14 QEMU 2/3] virtio-balloon: Add bit to notify guest of unused page reporting Alexander Duyck 2019-11-19 21:54 ` [PATCH v14 QEMU 3/3] virtio-balloon: Provide a interface for " Alexander Duyck 2019-11-26 12:20 ` [PATCH v14 0/6] mm / virtio: Provide support " David Hildenbrand 2019-11-26 16:45 ` Alexander Duyck 2019-11-27 10:01 ` David Hildenbrand 2019-11-27 17:36 ` Alexander Duyck [this message] 2019-11-27 17:37 ` David Hildenbrand
Reply instructions: You may reply publically to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-mm Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \ email@example.com public-inbox-index linux-mm Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kvack.linux-mm AGPL code for this site: git clone https://public-inbox.org/public-inbox.git