From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751722AbcF0Msr (ORCPT ); Mon, 27 Jun 2016 08:48:47 -0400 Received: from mail-pa0-f48.google.com ([209.85.220.48]:33923 "EHLO mail-pa0-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751609AbcF0Msn (ORCPT ); Mon, 27 Jun 2016 08:48:43 -0400 Subject: Re: [PATCH 00/27] Move LRU page reclaim from zones to nodes v7 To: Mel Gorman References: <1466518566-30034-1-git-send-email-mgorman@techsingularity.net> <3c062233-1ef7-bc85-5079-255f61f57c7d@gmail.com> <20160624075059.GC1868@techsingularity.net> Cc: Andrew Morton , Linux-MM , Rik van Riel , Vlastimil Babka , Johannes Weiner , LKML From: Balbir Singh Message-ID: <57712083.8060500@gmail.com> Date: Mon, 27 Jun 2016 22:48:03 +1000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 In-Reply-To: <20160624075059.GC1868@techsingularity.net> Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 24/06/16 17:50, Mel Gorman wrote: > On Fri, Jun 24, 2016 at 04:35:45PM +1000, Balbir Singh wrote: >>> 1. The residency of a page partially depends on what zone the page was >>> allocated from. This is partially combatted by the fair zone allocation >>> policy but that is a partial solution that introduces overhead in the >>> page allocator paths. >>> >>> 2. Currently, reclaim on node 0 behaves slightly different to node 1. For >>> example, direct reclaim scans in zonelist order and reclaims even if >>> the zone is over the high watermark regardless of the age of pages >>> in that LRU. Kswapd on the other hand starts reclaim on the highest >>> unbalanced zone. A difference in distribution of file/anon pages due >>> to when they were allocated results can result in a difference in >>> again. While the fair zone allocation policy mitigates some of the >>> problems here, the page reclaim results on a multi-zone node will >>> always be different to a single-zone node. >>> it was scheduled on as a result. >>> >>> 3. kswapd and the page allocator scan zones in the opposite order to >>> avoid interfering with each other but it's sensitive to timing. This >>> mitigates the page allocator using pages that were allocated very recently >>> in the ideal case but it's sensitive to timing. When kswapd is allocating >>> from lower zones then it's great but during the rebalancing of the highest >>> zone, the page allocator and kswapd interfere with each other. It's worse >>> if the highest zone is small and difficult to balance. >>> >>> 4. slab shrinkers are node-based which makes it harder to identify the exact >>> relationship between slab reclaim and LRU reclaim. >>> >> >> Sorry, I am late in reading the thread and the patches, but I am trying to understand >> the key benefits? > > The key benefits were outlined at the beginning of the changelog. The > one that is missing is the large overhead from the fair zone allocation > policy which can be removed safely by the feature. The benefit to page > allocator micro-benchmarks is outlined in the series introduction. I did look at them, but between 1 to 4, it seemed like the largest benefit was mm cleanup and better behaviour of reclaim on node 0. > >> I know that >> zones have grown to be overloaded to mean many things now. What is the contention impact >> of moving the LRU from zone to nodes? > > Expected to be minimal. On NUMA machines, most nodes have only one zone. > On machines with multiple zones, the lock per zone is not that fine-grained > given the size of the zones on large memory configurations. > Makes sense Thanks, Balbir Singh.