From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E2DAC433B4 for ; Thu, 15 Apr 2021 08:20:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 71C03611F1 for ; Thu, 15 Apr 2021 08:20:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231686AbhDOIU2 (ORCPT ); Thu, 15 Apr 2021 04:20:28 -0400 Received: from mga17.intel.com ([192.55.52.151]:41923 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231483AbhDOIU0 (ORCPT ); Thu, 15 Apr 2021 04:20:26 -0400 IronPort-SDR: mfvJ8iApM7hmrz0CvwS/TtodCyto2EEKegfwdjmmjd1WLIwYUviQfFyFma4Q4jNRUGE0EQbhkH ZyY2FK2fLN4Q== X-IronPort-AV: E=McAfee;i="6200,9189,9954"; a="174915921" X-IronPort-AV: E=Sophos;i="5.82,223,1613462400"; d="scan'208";a="174915921" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Apr 2021 01:20:03 -0700 IronPort-SDR: TfNJDq3moFAFzCHkvOEDK/CPLHTJL3Z1oA/NTpxQDcsf8QA3NprtYMKpTvKtuUyqYZ8Nl1NZ2x 1tYBNB7Zgdmg== X-IronPort-AV: E=Sophos;i="5.82,223,1613462400"; d="scan'208";a="522284833" Received: from yhuang6-desk1.sh.intel.com (HELO yhuang6-desk1.ccr.corp.intel.com) ([10.239.13.1]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Apr 2021 01:19:54 -0700 From: "Huang, Ying" To: Yu Zhao Cc: Rik van Riel , Dave Chinner , Jens Axboe , SeongJae Park , Linux-MM , Andrew Morton , Benjamin Manes , Dave Hansen , Hillf Danton , Johannes Weiner , Jonathan Corbet , Joonsoo Kim , Matthew Wilcox , Mel Gorman , Miaohe Lin , Michael Larabel , Michal Hocko , Michel Lespinasse , Roman Gushchin , Rong Chen , SeongJae Park , Tim Chen , Vlastimil Babka , Yang Shi , Zi Yan , linux-kernel , lkp@lists.01.org, Kernel Page Reclaim v2 , Andi Kleen Subject: Re: [PATCH v2 00/16] Multigenerational LRU Framework References: <20210413075155.32652-1-sjpark@amazon.de> <3ddd4f8a-8e51-662b-df11-a63a0e75b2bc@kernel.dk> <20210413231436.GF63242@dread.disaster.area> <20210414155130.GU3762101@tassilo.jf.intel.com> <20210415030002.GX3762101@tassilo.jf.intel.com> Date: Thu, 15 Apr 2021 16:19:52 +0800 In-Reply-To: (Yu Zhao's message of "Thu, 15 Apr 2021 01:13:13 -0600") Message-ID: <87a6q0ot7b.fsf@yhuang6-desk1.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Yu Zhao writes: > On Wed, Apr 14, 2021 at 9:00 PM Andi Kleen wrote: >> >> > We fall back to the rmap when it's obviously not smart to do so. There >> > is still a lot of room for improvement in this function though, i.e., >> > it should be per VMA and NUMA aware. >> >> Okay so it's more a question to tune the cross over heuristic. That >> sounds much easier than replacing everything. >> >> Of course long term it might be a problem to maintain too many >> different ways to do things, but I suppose short term it's a reasonable >> strategy. > > Hi Rik, Ying, > > Sorry for being persistent. I want to make sure we are on the same page: > > Page table scanning doesn't replace the existing rmap walk. It is > complementary and only happens when it is likely that most of the > pages on a system under pressure have been referenced, i.e., out of > *inactive* pages, by definition of the existing implementation. Under > such a condition, scanning *active* pages one by one with the rmap is > likely to cost more than scanning them all at once via page tables. > When we evict *inactive* pages, we still use the rmap and share a > common path with the existing code. > > Page table scanning falls back to the rmap walk if the page tables of > a process are apparently sparse, i.e., rss < size of the page tables. > > I should have clarified this at the very beginning of the discussion. > But it has become so natural to me and I assumed we'd all see it this > way. > > Your concern regarding the NUMA optimization is still valid, and it's > a high priority. Hi, Yu, In general, I think it's a good idea to combine the page table scanning and rmap scanning in the page reclaiming. For example, if the working-set is transitioned, we can take advantage of the fast page table scanning to identify the new working-set quickly. While we can fallback to the rmap scanning if the page table scanning doesn't help. Best Regards, Huang, Ying From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E01E7C43460 for ; Thu, 15 Apr 2021 08:20:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6B0F761166 for ; Thu, 15 Apr 2021 08:20:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6B0F761166 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ED8816B0070; Thu, 15 Apr 2021 04:20:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EAEF96B0071; Thu, 15 Apr 2021 04:20:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D50436B0072; Thu, 15 Apr 2021 04:20:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id B95316B0070 for ; Thu, 15 Apr 2021 04:20:06 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 79B043623 for ; Thu, 15 Apr 2021 08:20:06 +0000 (UTC) X-FDA: 78033903612.20.555E93C Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf09.hostedemail.com (Postfix) with ESMTP id 215D96000117 for ; Thu, 15 Apr 2021 08:20:01 +0000 (UTC) IronPort-SDR: 9Bvdofa0rZqOCtdausXOxkspg3tYocEhyRHcVAdx+9y0vupH6F6W4WQHT0Y9nYPQRyQp38cTnS dvjX5e7hto5A== X-IronPort-AV: E=McAfee;i="6200,9189,9954"; a="191622993" X-IronPort-AV: E=Sophos;i="5.82,223,1613462400"; d="scan'208";a="191622993" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Apr 2021 01:20:03 -0700 IronPort-SDR: TfNJDq3moFAFzCHkvOEDK/CPLHTJL3Z1oA/NTpxQDcsf8QA3NprtYMKpTvKtuUyqYZ8Nl1NZ2x 1tYBNB7Zgdmg== X-IronPort-AV: E=Sophos;i="5.82,223,1613462400"; d="scan'208";a="522284833" Received: from yhuang6-desk1.sh.intel.com (HELO yhuang6-desk1.ccr.corp.intel.com) ([10.239.13.1]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Apr 2021 01:19:54 -0700 From: "Huang, Ying" To: Yu Zhao Cc: Rik van Riel , Dave Chinner , Jens Axboe , SeongJae Park , Linux-MM , Andrew Morton , Benjamin Manes , Dave Hansen , Hillf Danton , Johannes Weiner , Jonathan Corbet , Joonsoo Kim , Matthew Wilcox , Mel Gorman , Miaohe Lin , Michael Larabel , Michal Hocko , Michel Lespinasse , Roman Gushchin , Rong Chen , SeongJae Park , Tim Chen , Vlastimil Babka , Yang Shi , Zi Yan , linux-kernel , lkp@lists.01.org, Kernel Page Reclaim v2 , Andi Kleen Subject: Re: [PATCH v2 00/16] Multigenerational LRU Framework References: <20210413075155.32652-1-sjpark@amazon.de> <3ddd4f8a-8e51-662b-df11-a63a0e75b2bc@kernel.dk> <20210413231436.GF63242@dread.disaster.area> <20210414155130.GU3762101@tassilo.jf.intel.com> <20210415030002.GX3762101@tassilo.jf.intel.com> Date: Thu, 15 Apr 2021 16:19:52 +0800 In-Reply-To: (Yu Zhao's message of "Thu, 15 Apr 2021 01:13:13 -0600") Message-ID: <87a6q0ot7b.fsf@yhuang6-desk1.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 215D96000117 X-Stat-Signature: eyqe9n1eupecjw4bh8atjxiqzkpe9kc7 Received-SPF: none (intel.com>: No applicable sender policy available) receiver=imf09; identity=mailfrom; envelope-from=""; helo=mga11.intel.com; client-ip=192.55.52.93 X-HE-DKIM-Result: none/none X-HE-Tag: 1618474801-755734 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Yu Zhao writes: > On Wed, Apr 14, 2021 at 9:00 PM Andi Kleen wrote: >> >> > We fall back to the rmap when it's obviously not smart to do so. There >> > is still a lot of room for improvement in this function though, i.e., >> > it should be per VMA and NUMA aware. >> >> Okay so it's more a question to tune the cross over heuristic. That >> sounds much easier than replacing everything. >> >> Of course long term it might be a problem to maintain too many >> different ways to do things, but I suppose short term it's a reasonable >> strategy. > > Hi Rik, Ying, > > Sorry for being persistent. I want to make sure we are on the same page: > > Page table scanning doesn't replace the existing rmap walk. It is > complementary and only happens when it is likely that most of the > pages on a system under pressure have been referenced, i.e., out of > *inactive* pages, by definition of the existing implementation. Under > such a condition, scanning *active* pages one by one with the rmap is > likely to cost more than scanning them all at once via page tables. > When we evict *inactive* pages, we still use the rmap and share a > common path with the existing code. > > Page table scanning falls back to the rmap walk if the page tables of > a process are apparently sparse, i.e., rss < size of the page tables. > > I should have clarified this at the very beginning of the discussion. > But it has become so natural to me and I assumed we'd all see it this > way. > > Your concern regarding the NUMA optimization is still valid, and it's > a high priority. Hi, Yu, In general, I think it's a good idea to combine the page table scanning and rmap scanning in the page reclaiming. For example, if the working-set is transitioned, we can take advantage of the fast page table scanning to identify the new working-set quickly. While we can fallback to the rmap scanning if the page table scanning doesn't help. Best Regards, Huang, Ying From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============6195875878771564090==" MIME-Version: 1.0 From: Huang, Ying To: lkp@lists.01.org Subject: Re: [PATCH v2 00/16] Multigenerational LRU Framework Date: Thu, 15 Apr 2021 16:19:52 +0800 Message-ID: <87a6q0ot7b.fsf@yhuang6-desk1.ccr.corp.intel.com> In-Reply-To: List-Id: --===============6195875878771564090== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Yu Zhao writes: > On Wed, Apr 14, 2021 at 9:00 PM Andi Kleen wrote: >> >> > We fall back to the rmap when it's obviously not smart to do so. There >> > is still a lot of room for improvement in this function though, i.e., >> > it should be per VMA and NUMA aware. >> >> Okay so it's more a question to tune the cross over heuristic. That >> sounds much easier than replacing everything. >> >> Of course long term it might be a problem to maintain too many >> different ways to do things, but I suppose short term it's a reasonable >> strategy. > > Hi Rik, Ying, > > Sorry for being persistent. I want to make sure we are on the same page: > > Page table scanning doesn't replace the existing rmap walk. It is > complementary and only happens when it is likely that most of the > pages on a system under pressure have been referenced, i.e., out of > *inactive* pages, by definition of the existing implementation. Under > such a condition, scanning *active* pages one by one with the rmap is > likely to cost more than scanning them all at once via page tables. > When we evict *inactive* pages, we still use the rmap and share a > common path with the existing code. > > Page table scanning falls back to the rmap walk if the page tables of > a process are apparently sparse, i.e., rss < size of the page tables. > > I should have clarified this at the very beginning of the discussion. > But it has become so natural to me and I assumed we'd all see it this > way. > > Your concern regarding the NUMA optimization is still valid, and it's > a high priority. Hi, Yu, In general, I think it's a good idea to combine the page table scanning and rmap scanning in the page reclaiming. For example, if the working-set is transitioned, we can take advantage of the fast page table scanning to identify the new working-set quickly. While we can fallback to the rmap scanning if the page table scanning doesn't help. Best Regards, Huang, Ying --===============6195875878771564090==--