From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFB76C433EF for ; Wed, 6 Apr 2022 16:19:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237330AbiDFQVk (ORCPT ); Wed, 6 Apr 2022 12:21:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237542AbiDFQVX (ORCPT ); Wed, 6 Apr 2022 12:21:23 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 236B02DCCAA; Tue, 5 Apr 2022 19:49:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649213399; x=1680749399; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=z3FJMwq8joZOXA61vyLoUR6apgy9wvWVJ1HAf2QRlg8=; b=Sq8QeBpXEQhWMifulmiCoQiTwmk46+APSOl/ZDTE40CRdZAaoLVlUJCG KT3rlX3S94SJm5x1NyO6BG56UqjHQJcp/eq0GNZFQ8b6Mup+s6mzT97aa f3fTUB6lAySUKqTyjLPJbQsMcITXIcgLfbkTaT2KvAfLHjw8j5zqlJbf7 ThdBvpMhKTazhbbaO5E6mTIpGdVfBzmp5oLAlP9Te5jibUooOO2p4JnlD rRyWAfDLAxVsOCDjIRxsqJR8TIHdx2LfiyOJXam85qmghH97RyW7+GQCz Dky65VskAxeUFm5CXtbJA5/Ige4xX2nDlduOUcJ8eybQVOxLEW/Pac3bm w==; X-IronPort-AV: E=McAfee;i="6200,9189,10308"; a="240867137" X-IronPort-AV: E=Sophos;i="5.90,238,1643702400"; d="scan'208";a="240867137" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Apr 2022 19:49:58 -0700 X-IronPort-AV: E=Sophos;i="5.90,238,1643702400"; d="scan'208";a="570305329" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.239.13.94]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Apr 2022 19:49:53 -0700 From: "Huang, Ying" To: Wei Xu Cc: Michal Hocko , Yosry Ahmed , Johannes Weiner , Shakeel Butt , Andrew Morton , David Rientjes , Tejun Heo , Zefan Li , Roman Gushchin , Cgroups , "open list:DOCUMENTATION" , Linux Kernel Mailing List , Linux MM , Jonathan Corbet , Yu Zhao , Dave Hansen , Greg Thelen Subject: Re: [PATCH resend] memcg: introduce per-memcg reclaim interface References: <20220331084151.2600229-1-yosryahmed@google.com> <87y20nzyw4.fsf@yhuang6-desk2.ccr.corp.intel.com> <87o81fujdc.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Wed, 06 Apr 2022 10:49:51 +0800 In-Reply-To: (Wei Xu's message of "Tue, 5 Apr 2022 18:07:49 -0700") Message-ID: <87bkxfudrk.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Wei Xu writes: > On Tue, Apr 5, 2022 at 5:49 PM Huang, Ying wrote: >> >> Wei Xu writes: >> >> > On Sat, Apr 2, 2022 at 1:13 AM Huang, Ying wrote: >> >> >> >> Wei Xu writes: >> >> >> >> > On Fri, Apr 1, 2022 at 6:54 AM Michal Hocko wrote: >> >> >> >> >> >> On Thu 31-03-22 08:41:51, Yosry Ahmed wrote: >> >> >> > From: Shakeel Butt >> >> >> > >> >> >> >> [snip] >> >> >> >> >> > Possible Extensions: >> >> >> > -------------------- >> >> >> > >> >> >> > - This interface can be extended with an additional parameter or flags >> >> >> > to allow specifying one or more types of memory to reclaim from (e.g. >> >> >> > file, anon, ..). >> >> >> > >> >> >> > - The interface can also be extended with a node mask to reclaim from >> >> >> > specific nodes. This has use cases for reclaim-based demotion in memory >> >> >> > tiering systens. >> >> >> > >> >> >> > - A similar per-node interface can also be added to support proactive >> >> >> > reclaim and reclaim-based demotion in systems without memcg. >> >> >> > >> >> >> > For now, let's keep things simple by adding the basic functionality. >> >> >> >> >> >> Yes, I am for the simplicity and this really looks like a bare minumum >> >> >> interface. But it is not really clear who do you want to add flags on >> >> >> top of it? >> >> >> >> >> >> I am not really sure we really need a node aware interface for memcg. >> >> >> The global reclaim interface will likely need a different node because >> >> >> we do not want to make this CONFIG_MEMCG constrained. >> >> > >> >> > A nodemask argument for memory.reclaim can be useful for memory >> >> > tiering between NUMA nodes with different performance. Similar to >> >> > proactive reclaim, it can allow a userspace daemon to drive >> >> > memcg-based proactive demotion via the reclaim-based demotion >> >> > mechanism in the kernel. >> >> >> >> I am not sure whether nodemask is a good way for demoting pages between >> >> different types of memory. For example, for a system with DRAM and >> >> PMEM, if specifying DRAM node in nodemask means demoting to PMEM, what >> >> is the meaning of specifying PMEM node? reclaiming to disk? >> >> >> >> In general, I have no objection to the idea in general. But we should >> >> have a clear and consistent interface. Per my understanding the default >> >> memcg interface is for memory, regardless of memory types. The memory >> >> reclaiming means reduce the memory usage, regardless of memory types. >> >> We need to either extending the semantics of memory reclaiming (to >> >> include memory demoting too), or add another interface for memory >> >> demoting. >> > >> > Good point. With the "demote pages during reclaim" patch series, >> > reclaim is already extended to demote pages as well. For example, >> > can_reclaim_anon_pages() returns true if demotion is allowed and >> > shrink_page_list() can demote pages instead of reclaiming pages. >> >> These are in-kernel implementation, not the ABI. So we still have >> the opportunity to define the ABI now. >> >> > Currently, demotion is disabled for memcg reclaim, which I think can >> > be relaxed and also necessary for memcg-based proactive demotion. I'd >> > like to suggest that we extend the semantics of memory.reclaim to >> > cover memory demotion as well. A flag can be used to enable/disable >> > the demotion behavior. >> >> If so, >> >> # echo A > memory.reclaim >> >> means >> >> a) "A" bytes memory are freed from the memcg, regardless demoting is >> used or not. >> >> or >> >> b) "A" bytes memory are reclaimed from the memcg, some of them may be >> freed, some of them may be just demoted from DRAM to PMEM. The total >> number is "A". >> >> For me, a) looks more reasonable. >> > > We can use a DEMOTE flag to control the demotion behavior for > memory.reclaim. If the flag is not set (the default), then > no_demotion of scan_control can be set to 1, similar to > reclaim_pages(). If we have to use a flag to control the behavior, I think it's better to have a separate interface (e.g. memory.demote). But do we really need b)? > The question is then whether we want to rename memory.reclaim to > something more general. I think this name is fine if reclaim-based > demotion is an accepted concept. Best Regards, Huang, Ying From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Huang, Ying" Subject: Re: [PATCH resend] memcg: introduce per-memcg reclaim interface Date: Wed, 06 Apr 2022 10:49:51 +0800 Message-ID: <87bkxfudrk.fsf@yhuang6-desk2.ccr.corp.intel.com> References: <20220331084151.2600229-1-yosryahmed@google.com> <87y20nzyw4.fsf@yhuang6-desk2.ccr.corp.intel.com> <87o81fujdc.fsf@yhuang6-desk2.ccr.corp.intel.com> Mime-Version: 1.0 Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649213399; x=1680749399; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=z3FJMwq8joZOXA61vyLoUR6apgy9wvWVJ1HAf2QRlg8=; b=Sq8QeBpXEQhWMifulmiCoQiTwmk46+APSOl/ZDTE40CRdZAaoLVlUJCG KT3rlX3S94SJm5x1NyO6BG56UqjHQJcp/eq0GNZFQ8b6Mup+s6mzT97aa f3fTUB6lAySUKqTyjLPJbQsMcITXIcgLfbkTaT2KvAfLHjw8j5zqlJbf7 ThdBvpMhKTazhbbaO5E6mTIpGdVfBzmp5oLAlP9Te5jibUooOO2p4JnlD rRyWAfDLAxVsOCDjIRxsqJR8TIHdx2LfiyOJXam85qmghH97RyW7+GQCz Dky65VskAxeUFm5CXtbJA5/Ige4xX2nDlduOUcJ8eybQVOxLEW/Pac3bm w==; In-Reply-To: (Wei Xu's message of "Tue, 5 Apr 2022 18:07:49 -0700") List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Wei Xu Cc: Michal Hocko , Yosry Ahmed , Johannes Weiner , Shakeel Butt , Andrew Morton , David Rientjes , Tejun Heo , Zefan Li , Roman Gushchin , Cgroups , "open list:DOCUMENTATION" , Linux Kernel Mailing List , Linux MM , Jonathan Corbet , Yu Zhao , Dave Hansen , Greg Thelen Wei Xu writes: > On Tue, Apr 5, 2022 at 5:49 PM Huang, Ying wrote: >> >> Wei Xu writes: >> >> > On Sat, Apr 2, 2022 at 1:13 AM Huang, Ying wrote: >> >> >> >> Wei Xu writes: >> >> >> >> > On Fri, Apr 1, 2022 at 6:54 AM Michal Hocko wrote: >> >> >> >> >> >> On Thu 31-03-22 08:41:51, Yosry Ahmed wrote: >> >> >> > From: Shakeel Butt >> >> >> > >> >> >> >> [snip] >> >> >> >> >> > Possible Extensions: >> >> >> > -------------------- >> >> >> > >> >> >> > - This interface can be extended with an additional parameter or flags >> >> >> > to allow specifying one or more types of memory to reclaim from (e.g. >> >> >> > file, anon, ..). >> >> >> > >> >> >> > - The interface can also be extended with a node mask to reclaim from >> >> >> > specific nodes. This has use cases for reclaim-based demotion in memory >> >> >> > tiering systens. >> >> >> > >> >> >> > - A similar per-node interface can also be added to support proactive >> >> >> > reclaim and reclaim-based demotion in systems without memcg. >> >> >> > >> >> >> > For now, let's keep things simple by adding the basic functionality. >> >> >> >> >> >> Yes, I am for the simplicity and this really looks like a bare minumum >> >> >> interface. But it is not really clear who do you want to add flags on >> >> >> top of it? >> >> >> >> >> >> I am not really sure we really need a node aware interface for memcg. >> >> >> The global reclaim interface will likely need a different node because >> >> >> we do not want to make this CONFIG_MEMCG constrained. >> >> > >> >> > A nodemask argument for memory.reclaim can be useful for memory >> >> > tiering between NUMA nodes with different performance. Similar to >> >> > proactive reclaim, it can allow a userspace daemon to drive >> >> > memcg-based proactive demotion via the reclaim-based demotion >> >> > mechanism in the kernel. >> >> >> >> I am not sure whether nodemask is a good way for demoting pages between >> >> different types of memory. For example, for a system with DRAM and >> >> PMEM, if specifying DRAM node in nodemask means demoting to PMEM, what >> >> is the meaning of specifying PMEM node? reclaiming to disk? >> >> >> >> In general, I have no objection to the idea in general. But we should >> >> have a clear and consistent interface. Per my understanding the default >> >> memcg interface is for memory, regardless of memory types. The memory >> >> reclaiming means reduce the memory usage, regardless of memory types. >> >> We need to either extending the semantics of memory reclaiming (to >> >> include memory demoting too), or add another interface for memory >> >> demoting. >> > >> > Good point. With the "demote pages during reclaim" patch series, >> > reclaim is already extended to demote pages as well. For example, >> > can_reclaim_anon_pages() returns true if demotion is allowed and >> > shrink_page_list() can demote pages instead of reclaiming pages. >> >> These are in-kernel implementation, not the ABI. So we still have >> the opportunity to define the ABI now. >> >> > Currently, demotion is disabled for memcg reclaim, which I think can >> > be relaxed and also necessary for memcg-based proactive demotion. I'd >> > like to suggest that we extend the semantics of memory.reclaim to >> > cover memory demotion as well. A flag can be used to enable/disable >> > the demotion behavior. >> >> If so, >> >> # echo A > memory.reclaim >> >> means >> >> a) "A" bytes memory are freed from the memcg, regardless demoting is >> used or not. >> >> or >> >> b) "A" bytes memory are reclaimed from the memcg, some of them may be >> freed, some of them may be just demoted from DRAM to PMEM. The total >> number is "A". >> >> For me, a) looks more reasonable. >> > > We can use a DEMOTE flag to control the demotion behavior for > memory.reclaim. If the flag is not set (the default), then > no_demotion of scan_control can be set to 1, similar to > reclaim_pages(). If we have to use a flag to control the behavior, I think it's better to have a separate interface (e.g. memory.demote). But do we really need b)? > The question is then whether we want to rename memory.reclaim to > something more general. I think this name is fine if reclaim-based > demotion is an accepted concept. Best Regards, Huang, Ying