From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ED60C433F5 for ; Thu, 10 Mar 2022 16:58:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB7DC8D0002; Thu, 10 Mar 2022 11:58:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D67EA8D0001; Thu, 10 Mar 2022 11:58:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2F078D0002; Thu, 10 Mar 2022 11:58:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0082.hostedemail.com [216.40.44.82]) by kanga.kvack.org (Postfix) with ESMTP id B4D868D0001 for ; Thu, 10 Mar 2022 11:58:12 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 759F1181C49B7 for ; Thu, 10 Mar 2022 16:58:12 +0000 (UTC) X-FDA: 79229084424.18.58EE51C Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by imf16.hostedemail.com (Postfix) with ESMTP id 703FA18000C for ; Thu, 10 Mar 2022 16:58:11 +0000 (UTC) Received: by mail-qt1-f175.google.com with SMTP id u9so1548052qta.5 for ; Thu, 10 Mar 2022 08:58:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=C9+mR3Lo49B7f4gRjZKQMij6hiCRkwa4M+IGhC44RbY=; b=BFAyztib/ufGhLKZLmTrZfNVcr3Z73I1hn7zB0GHCPxLcRmKmQnQE8yUumQ8MmaTXg 9Z5QwoE73vq8tis3rnQwOw7aiu8GSiTnzaL4mVNBN7nFOM//W+m3gmCqmeFHyCshGv/N xDH0sdeIm50kfYRL8ypvYNExDXKRSFO6mhscmRf8OgCWMze98x8LFMEzBj/waUslwxa+ w8+BHT3xDNssqqL0F6D1KJmUFnfNsnNXdMvip3mBoAt8F5y/7xtFtJATYiWMyaloeIdS WUJkWS4Pn0UxcdZ1gzDMumlnV2VxGxE/6MA+UZUJxT4stm8nnlE9x0ZpfGWmWiNOr9iW 5CVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=C9+mR3Lo49B7f4gRjZKQMij6hiCRkwa4M+IGhC44RbY=; b=Qpcu3uSpXHzIvCyPdmT21OW6vMDjA4QR8lMGV/Yv7N6ND+Gv6mnn+uZP64e2GQPKKN MhZKvrgKO6Gf3rEZScy6xhOysBrGFKRxmETVxQwtAFKTf5MWjGeMNncjIBT8UITh/IBx vVPQuGwaKqzRyI7VUGnkXCORCfn+epn6KaqvGXUuDTn8//5LgMB+g9Gfrtv/z2tayl4Q u0LHfl+F7yTv0ea4KzRTI1fimFe/ld0Wu7xuZKVXHM2UmVS6DDOtI0xIUvfeXFbPsnMf etLDZZ0yNk5W0WEuBx+FOTqCtme5VzIG+u28PgmU4c/sMqlgpGCCPwU7ZUUvBPBeAXNq Wjbg== X-Gm-Message-State: AOAM532k8OKqDe+97OnnCeovAg4pvUw1OGJlgWHL8HF/kbh2oahe6YDp AfOr08EA3wtT/g3SlldPg40WNQ== X-Google-Smtp-Source: ABdhPJx8iBgAl3RU9ryDWlD+miBIGdrVXduFhMUGdwwF8USpzbCXuA98Fww8tChRFjz0U4EL15cw1g== X-Received: by 2002:ac8:5c4d:0:b0:2e0:71b7:2829 with SMTP id j13-20020ac85c4d000000b002e071b72829mr4832825qtj.323.1646931490662; Thu, 10 Mar 2022 08:58:10 -0800 (PST) Received: from localhost (cpe-98-15-154-102.hvc.res.rr.com. [98.15.154.102]) by smtp.gmail.com with ESMTPSA id 22-20020ac85916000000b002d6844c51b9sm3526255qty.86.2022.03.10.08.58.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Mar 2022 08:58:10 -0800 (PST) Date: Thu, 10 Mar 2022 11:58:09 -0500 From: Johannes Weiner To: David Rientjes Cc: Michal Hocko , Shakeel Butt , Andrew Morton , Yu Zhao , Dave Hansen , linux-mm@kvack.org, Yosry Ahmed , Wei Xu , Greg Thelen Subject: Re: [RFC] Mechanism to induce memory reclaim Message-ID: References: <5df21376-7dd1-bf81-8414-32a73cea45dd@google.com> <20220307183141.npa4627fpbsbgwvv@google.com> <63fcd044-7c87-63f3-391e-3b32f8feaae@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <63fcd044-7c87-63f3-391e-3b32f8feaae@google.com> X-Rspamd-Server: rspam10 X-Rspam-User: X-Stat-Signature: xni7ajx7wj98kj1cmowmpc4agjie55iq Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=BFAyztib; spf=pass (imf16.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.175 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org X-Rspamd-Queue-Id: 703FA18000C X-HE-Tag: 1646931491-282540 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Mar 09, 2022 at 02:03:21PM -0800, David Rientjes wrote: > On Tue, 8 Mar 2022, Michal Hocko wrote: > > > > Let me take a stab at this. The specific reasons why high limit is not a > > > good interface to implement proactive reclaim: > > > > > > 1) It can cause allocations from the target application to get > > > throttled. > > > > > > 2) It leaves a state (high limit) in the kernel which needs to be reset > > > by the userspace part of proactive reclaimer. > > > > > > If I remember correctly, Facebook actually tried to use high limit to > > > implement the proactive reclaim but due to exactly these limitations [1] > > > they went the route [2] aligned with this proposal. > > > > I do remember we have discussed this in the past. There were proposals > > for an additional limit to trigger a background reclaim [3] or to add a > > pressure based memcg knob [4]. For the nr_to_reclaim based interface > > there were some challenges outlined in that email thread. I do > > understand that practical experience could have confirmed or diminished > > those concerns. > > > > I am definitely happy to restart those discussion but it would be really > > great to summarize existing options and why they do not work in > > practice. It would be also great to mention why concerns about nr_to_reclaim > > based interface expressed in the past are not standing out anymore wrt. > > other proposals. > > > > Johannes, since you had pointed out that the current approach used at Meta > and described in the TMO paper works well in practice and is based on > prior discussions of memory.reclaim[1], do you have any lingering concerns > from that 2020 thread? I'd be okay with merging the interface proposed in that thread as-is. > My first email in this thread proposes something that can still do memcg > based reclaim but is also possible even without CONFIG_MEMCG enabled. > That's particularly helpful for configs used by customers that don't use > memcg, namely Chrome OS. I assume we're not losing any functionality that > your use case depends on if we are to introduce a per-node sysfs mechanism > for this as an alternative since you can still specify a memcg id? We'd lose the delegation functionality with this proposal. But per the other thread, I wouldn't be opposed to adding a global per-node interface in addition to the cgroupfs one.