From: Yu Zhao <yuzhao@google.com> To: Alexey Avramov <hakavlad0@seznam.cz> Cc: 21cnbao@gmail.com, Michael@michaellarabel.com, ak@linux.intel.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, axboe@kernel.dk, catalin.marinas@arm.com, corbet@lwn.net, dave.hansen@linux.intel.com, hannes@cmpxchg.org, hdanton@sina.com, jsbarnes@google.com, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, mgorman@suse.de, mhocko@kernel.org, page-reclaim@google.com, riel@surriel.com, rppt@kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, will@kernel.org, willy@infradead.org, x86@kernel.org, ying.huang@intel.com Subject: Re: [PATCH v7 00/12] Multigenerational LRU Framework Date: Sat, 12 Feb 2022 14:01:23 -0700 [thread overview] Message-ID: <YgggI+vvtNvh3jBY@google.com> (raw) In-Reply-To: <20220212051219.183d1baf@PC> On Sat, Feb 12, 2022 at 05:12:19AM +0900, Alexey Avramov wrote: > Aggressive swapping even with vm.swappiness=1 with MGLRU > ======================================================== > > Reading a large mmapped file leads to a super agressive swapping. > Reducing vm.swappiness even to 1 does not have effect. Mind explaining why you think it's "super agressive"? I assume you expected a different behavior that would perform better. If so, please spell it out. > Demo: https://www.youtube.com/watch?v=J81kwJeuW58 > > Linux 5.17-rc3, Multigenerational LRU v7, > vm.swappiness=1, MemTotal: 11.5 GiB. > > $ cache-bench -r 35000 -m1 -b1 -p1 -f test20000 > Reading mmapped file (file size: 20000 MiB) > cache-bench v0.2.0: https://github.com/hakavlad/cache-bench Writing your own benchmark is a good exercise but fio is the standard benchmark in this case. Please use it with --ioengine=mmap. > Swapping started with MemAvailable=71%. > At the end 33 GiB was swapped out when MemAvailable=60%. > > Is it OK? MemAvailable is an estimate (free + page cache), and it doesn't imply any reclaim preferences. In the worst case scenario, e.g., out of swap space, MemAvailable *may* be reclaimed. Here is my benchmark result with file mmap + *high* swap usage. Ram disk was used to reduce the variance in the result (and SSD wear out if you care). More details on additional configurations here: https://lore.kernel.org/linux-mm/20220208081902.3550911-6-yuzhao@google.com/ Mixed workloads: fio (buffered I/O): +13% IOPS BW 5.17-rc3: 275k 1075MiB/s v7: 313k 1222MiB/s memcached (anon): +12% Ops/sec KB/sec 5.17-rc3: 511282.72 19861.04 v7: 572408.80 22235.49 cat mmap.sh systemctl restart memcached swapoff -a umount /mnt rmmod brd modprobe brd rd_nr=2 rd_size=56623104 mkswap /dev/ram0 swapon /dev/ram0 mkfs.ext4 /dev/ram1 mount -t ext4 /dev/ram1 /mnt memtier_benchmark -S /var/run/memcached/memcached.sock \ -P memcache_binary -n allkeys --key-minimum=1 \ --key-maximum=50000000 --key-pattern=P:P -c 1 \ -t 36 --ratio 1:0 --pipeline 8 -d 2000 sysctl vm.overcommit_memory=1 fio -name=mglru --numjobs=36 --directory=/mnt --size=1408m \ --buffered=1 --ioengine=mmap --iodepth=128 --iodepth_batch_submit=32 \ --iodepth_batch_complete=32 --rw=randread --random_distribution=random \ --norandommap --time_based --ramp_time=10m --runtime=990m \ --group_reporting & pid=$! sleep 200 memcached.sock -P memcache_binary -n allkeys --key-minimum=1 \ --key-maximum=50000000 --key-pattern=R:R -c 1 -t 36 --ratio 0:1 \ --pipeline 8 --randomize --distinct-client-seed kill -INT $pid wait
WARNING: multiple messages have this Message-ID (diff)
From: Yu Zhao <yuzhao@google.com> To: Alexey Avramov <hakavlad0@seznam.cz> Cc: 21cnbao@gmail.com, Michael@michaellarabel.com, ak@linux.intel.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, axboe@kernel.dk, catalin.marinas@arm.com, corbet@lwn.net, dave.hansen@linux.intel.com, hannes@cmpxchg.org, hdanton@sina.com, jsbarnes@google.com, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, mgorman@suse.de, mhocko@kernel.org, page-reclaim@google.com, riel@surriel.com, rppt@kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, will@kernel.org, willy@infradead.org, x86@kernel.org, ying.huang@intel.com Subject: Re: [PATCH v7 00/12] Multigenerational LRU Framework Date: Sat, 12 Feb 2022 14:01:23 -0700 [thread overview] Message-ID: <YgggI+vvtNvh3jBY@google.com> (raw) In-Reply-To: <20220212051219.183d1baf@PC> On Sat, Feb 12, 2022 at 05:12:19AM +0900, Alexey Avramov wrote: > Aggressive swapping even with vm.swappiness=1 with MGLRU > ======================================================== > > Reading a large mmapped file leads to a super agressive swapping. > Reducing vm.swappiness even to 1 does not have effect. Mind explaining why you think it's "super agressive"? I assume you expected a different behavior that would perform better. If so, please spell it out. > Demo: https://www.youtube.com/watch?v=J81kwJeuW58 > > Linux 5.17-rc3, Multigenerational LRU v7, > vm.swappiness=1, MemTotal: 11.5 GiB. > > $ cache-bench -r 35000 -m1 -b1 -p1 -f test20000 > Reading mmapped file (file size: 20000 MiB) > cache-bench v0.2.0: https://github.com/hakavlad/cache-bench Writing your own benchmark is a good exercise but fio is the standard benchmark in this case. Please use it with --ioengine=mmap. > Swapping started with MemAvailable=71%. > At the end 33 GiB was swapped out when MemAvailable=60%. > > Is it OK? MemAvailable is an estimate (free + page cache), and it doesn't imply any reclaim preferences. In the worst case scenario, e.g., out of swap space, MemAvailable *may* be reclaimed. Here is my benchmark result with file mmap + *high* swap usage. Ram disk was used to reduce the variance in the result (and SSD wear out if you care). More details on additional configurations here: https://lore.kernel.org/linux-mm/20220208081902.3550911-6-yuzhao@google.com/ Mixed workloads: fio (buffered I/O): +13% IOPS BW 5.17-rc3: 275k 1075MiB/s v7: 313k 1222MiB/s memcached (anon): +12% Ops/sec KB/sec 5.17-rc3: 511282.72 19861.04 v7: 572408.80 22235.49 cat mmap.sh systemctl restart memcached swapoff -a umount /mnt rmmod brd modprobe brd rd_nr=2 rd_size=56623104 mkswap /dev/ram0 swapon /dev/ram0 mkfs.ext4 /dev/ram1 mount -t ext4 /dev/ram1 /mnt memtier_benchmark -S /var/run/memcached/memcached.sock \ -P memcache_binary -n allkeys --key-minimum=1 \ --key-maximum=50000000 --key-pattern=P:P -c 1 \ -t 36 --ratio 1:0 --pipeline 8 -d 2000 sysctl vm.overcommit_memory=1 fio -name=mglru --numjobs=36 --directory=/mnt --size=1408m \ --buffered=1 --ioengine=mmap --iodepth=128 --iodepth_batch_submit=32 \ --iodepth_batch_complete=32 --rw=randread --random_distribution=random \ --norandommap --time_based --ramp_time=10m --runtime=990m \ --group_reporting & pid=$! sleep 200 memcached.sock -P memcache_binary -n allkeys --key-minimum=1 \ --key-maximum=50000000 --key-pattern=R:R -c 1 -t 36 --ratio 0:1 \ --pipeline 8 --randomize --distinct-client-seed kill -INT $pid wait _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2022-02-12 21:01 UTC|newest] Thread overview: 150+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-02-08 8:18 [PATCH v7 00/12] Multigenerational LRU Framework Yu Zhao 2022-02-08 8:18 ` Yu Zhao 2022-02-08 8:18 ` [PATCH v7 01/12] mm: x86, arm64: add arch_has_hw_pte_young() Yu Zhao 2022-02-08 8:18 ` Yu Zhao 2022-02-08 8:24 ` Yu Zhao 2022-02-08 8:24 ` Yu Zhao 2022-02-08 10:33 ` Will Deacon 2022-02-08 10:33 ` Will Deacon 2022-02-08 8:18 ` [PATCH v7 02/12] mm: x86: add CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG Yu Zhao 2022-02-08 8:18 ` Yu Zhao 2022-02-08 8:27 ` Yu Zhao 2022-02-08 8:27 ` Yu Zhao 2022-02-08 8:18 ` [PATCH v7 03/12] mm/vmscan.c: refactor shrink_node() Yu Zhao 2022-02-08 8:18 ` Yu Zhao 2022-02-08 8:18 ` [PATCH v7 04/12] mm: multigenerational LRU: groundwork Yu Zhao 2022-02-08 8:18 ` Yu Zhao 2022-02-08 8:28 ` Yu Zhao 2022-02-08 8:28 ` Yu Zhao 2022-02-10 20:41 ` Johannes Weiner 2022-02-10 20:41 ` Johannes Weiner 2022-02-15 9:43 ` Yu Zhao 2022-02-15 9:43 ` Yu Zhao 2022-02-15 21:53 ` Johannes Weiner 2022-02-15 21:53 ` Johannes Weiner 2022-02-21 8:14 ` Yu Zhao 2022-02-21 8:14 ` Yu Zhao 2022-02-23 21:18 ` Yu Zhao 2022-02-23 21:18 ` Yu Zhao 2022-02-25 16:34 ` Minchan Kim 2022-02-25 16:34 ` Minchan Kim 2022-03-03 15:29 ` Johannes Weiner 2022-03-03 15:29 ` Johannes Weiner 2022-03-03 19:26 ` Yu Zhao 2022-03-03 19:26 ` Yu Zhao 2022-03-03 21:43 ` Johannes Weiner 2022-03-03 21:43 ` Johannes Weiner 2022-03-11 10:16 ` Barry Song 2022-03-11 10:16 ` Barry Song 2022-03-11 23:45 ` Yu Zhao 2022-03-11 23:45 ` Yu Zhao 2022-03-12 10:37 ` Barry Song 2022-03-12 10:37 ` Barry Song 2022-03-12 21:11 ` Yu Zhao 2022-03-12 21:11 ` Yu Zhao 2022-03-13 4:57 ` Barry Song 2022-03-13 4:57 ` Barry Song 2022-03-14 11:11 ` Barry Song 2022-03-14 11:11 ` Barry Song 2022-03-14 16:45 ` Yu Zhao 2022-03-14 16:45 ` Yu Zhao 2022-03-14 23:38 ` Barry Song 2022-03-14 23:38 ` Barry Song 2022-03-15 5:18 ` Yu Zhao 2022-03-15 5:18 ` Yu Zhao 2022-03-15 9:27 ` Barry Song 2022-03-15 9:27 ` Barry Song 2022-03-15 10:29 ` Barry Song 2022-03-15 10:29 ` Barry Song 2022-03-16 2:46 ` Yu Zhao 2022-03-16 2:46 ` Yu Zhao 2022-03-16 4:37 ` Barry Song 2022-03-16 4:37 ` Barry Song 2022-03-16 5:44 ` Yu Zhao 2022-03-16 5:44 ` Yu Zhao 2022-03-16 6:06 ` Barry Song 2022-03-16 6:06 ` Barry Song 2022-03-16 21:37 ` Yu Zhao 2022-03-16 21:37 ` Yu Zhao 2022-02-10 21:37 ` Matthew Wilcox 2022-02-10 21:37 ` Matthew Wilcox 2022-02-13 21:16 ` Yu Zhao 2022-02-13 21:16 ` Yu Zhao 2022-02-08 8:18 ` [PATCH v7 05/12] mm: multigenerational LRU: minimal implementation Yu Zhao 2022-02-08 8:18 ` Yu Zhao 2022-02-08 8:33 ` Yu Zhao 2022-02-08 8:33 ` Yu Zhao 2022-02-08 16:50 ` Johannes Weiner 2022-02-08 16:50 ` Johannes Weiner 2022-02-10 2:53 ` Yu Zhao 2022-02-10 2:53 ` Yu Zhao 2022-02-13 10:04 ` Hillf Danton 2022-02-17 0:13 ` Yu Zhao 2022-02-23 8:27 ` Huang, Ying 2022-02-23 8:27 ` Huang, Ying 2022-02-23 9:36 ` Yu Zhao 2022-02-23 9:36 ` Yu Zhao 2022-02-24 0:59 ` Huang, Ying 2022-02-24 0:59 ` Huang, Ying 2022-02-24 1:34 ` Yu Zhao 2022-02-24 1:34 ` Yu Zhao 2022-02-24 3:31 ` Huang, Ying 2022-02-24 3:31 ` Huang, Ying 2022-02-24 4:09 ` Yu Zhao 2022-02-24 4:09 ` Yu Zhao 2022-02-24 5:27 ` Huang, Ying 2022-02-24 5:27 ` Huang, Ying 2022-02-24 5:35 ` Yu Zhao 2022-02-24 5:35 ` Yu Zhao 2022-02-08 8:18 ` [PATCH v7 06/12] mm: multigenerational LRU: exploit locality in rmap Yu Zhao 2022-02-08 8:18 ` Yu Zhao 2022-02-08 8:40 ` Yu Zhao 2022-02-08 8:40 ` Yu Zhao 2022-02-08 8:18 ` [PATCH v7 07/12] mm: multigenerational LRU: support page table walks Yu Zhao 2022-02-08 8:18 ` Yu Zhao 2022-02-08 8:39 ` Yu Zhao 2022-02-08 8:39 ` Yu Zhao 2022-02-08 8:18 ` [PATCH v7 08/12] mm: multigenerational LRU: optimize multiple memcgs Yu Zhao 2022-02-08 8:18 ` Yu Zhao 2022-02-08 8:18 ` [PATCH v7 09/12] mm: multigenerational LRU: runtime switch Yu Zhao 2022-02-08 8:18 ` Yu Zhao 2022-02-08 8:42 ` Yu Zhao 2022-02-08 8:42 ` Yu Zhao 2022-02-08 8:19 ` [PATCH v7 10/12] mm: multigenerational LRU: thrashing prevention Yu Zhao 2022-02-08 8:19 ` Yu Zhao 2022-02-08 8:43 ` Yu Zhao 2022-02-08 8:43 ` Yu Zhao 2022-02-08 8:19 ` [PATCH v7 11/12] mm: multigenerational LRU: debugfs interface Yu Zhao 2022-02-08 8:19 ` Yu Zhao 2022-02-18 18:56 ` [page-reclaim] " David Rientjes 2022-02-18 18:56 ` David Rientjes 2022-02-08 8:19 ` [PATCH v7 12/12] mm: multigenerational LRU: documentation Yu Zhao 2022-02-08 8:19 ` Yu Zhao 2022-02-08 8:44 ` Yu Zhao 2022-02-08 8:44 ` Yu Zhao 2022-02-14 10:28 ` Mike Rapoport 2022-02-14 10:28 ` Mike Rapoport 2022-02-16 3:22 ` Yu Zhao 2022-02-16 3:22 ` Yu Zhao 2022-02-21 9:01 ` Mike Rapoport 2022-02-21 9:01 ` Mike Rapoport 2022-02-22 1:47 ` Yu Zhao 2022-02-22 1:47 ` Yu Zhao 2022-02-23 10:58 ` Mike Rapoport 2022-02-23 10:58 ` Mike Rapoport 2022-02-23 21:20 ` Yu Zhao 2022-02-23 21:20 ` Yu Zhao 2022-02-08 10:11 ` [PATCH v7 00/12] Multigenerational LRU Framework Oleksandr Natalenko 2022-02-08 10:11 ` Oleksandr Natalenko 2022-02-08 11:14 ` Michal Hocko 2022-02-08 11:14 ` Michal Hocko 2022-02-08 11:23 ` Oleksandr Natalenko 2022-02-08 11:23 ` Oleksandr Natalenko 2022-02-11 20:12 ` Alexey Avramov 2022-02-11 20:12 ` Alexey Avramov 2022-02-12 21:01 ` Yu Zhao [this message] 2022-02-12 21:01 ` Yu Zhao 2022-03-03 6:06 ` Vaibhav Jain 2022-03-03 6:06 ` Vaibhav Jain 2022-03-03 6:47 ` Yu Zhao 2022-03-03 6:47 ` Yu Zhao
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=YgggI+vvtNvh3jBY@google.com \ --to=yuzhao@google.com \ --cc=21cnbao@gmail.com \ --cc=Michael@michaellarabel.com \ --cc=ak@linux.intel.com \ --cc=akpm@linux-foundation.org \ --cc=aneesh.kumar@linux.ibm.com \ --cc=axboe@kernel.dk \ --cc=catalin.marinas@arm.com \ --cc=corbet@lwn.net \ --cc=dave.hansen@linux.intel.com \ --cc=hakavlad0@seznam.cz \ --cc=hannes@cmpxchg.org \ --cc=hdanton@sina.com \ --cc=jsbarnes@google.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-doc@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mgorman@suse.de \ --cc=mhocko@kernel.org \ --cc=page-reclaim@google.com \ --cc=riel@surriel.com \ --cc=rppt@kernel.org \ --cc=torvalds@linux-foundation.org \ --cc=vbabka@suse.cz \ --cc=will@kernel.org \ --cc=willy@infradead.org \ --cc=x86@kernel.org \ --cc=ying.huang@intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.