From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABF91C433F5 for ; Tue, 8 Mar 2022 16:05:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 203BE8D0003; Tue, 8 Mar 2022 11:05:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B23E8D0001; Tue, 8 Mar 2022 11:05:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 07B248D0003; Tue, 8 Mar 2022 11:05:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0136.hostedemail.com [216.40.44.136]) by kanga.kvack.org (Postfix) with ESMTP id ED8F18D0001 for ; Tue, 8 Mar 2022 11:05:16 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A9329181ECAEA for ; Tue, 8 Mar 2022 16:05:16 +0000 (UTC) X-FDA: 79221693432.17.489ED67 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf28.hostedemail.com (Postfix) with ESMTP id 196D6C0019 for ; Tue, 8 Mar 2022 16:05:15 +0000 (UTC) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id C376B210EF; Tue, 8 Mar 2022 16:05:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1646755514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=5T9SxFJcgmVdFaaz/Qb3yMaris05TZdcYBP8glEbreM=; b=H8+sduPPVZ21vkAzE+dlGVeEh1WpBq+UEhGzSEqw+vhDPw9zJEDdURbND7cIScpHDWyeuD z2fjQd5qa0g2H3xcAA8kwxC9SPt/i18euUPyAyk5ilVAjvlgQ7ItXTiKsV7bxL2xpfge8P IWW5lZtFCb/iepxA5kugV019Rn9ZjQE= Received: from suse.cz (unknown [10.100.201.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 925D8A3B83; Tue, 8 Mar 2022 16:05:14 +0000 (UTC) Date: Tue, 8 Mar 2022 17:05:11 +0100 From: Michal Hocko To: Dan Schatzberg Cc: Johannes Weiner , Shakeel Butt , David Rientjes , Andrew Morton , Yu Zhao , Dave Hansen , linux-mm@kvack.org, Yosry Ahmed , Wei Xu , Greg Thelen Subject: Re: [RFC] Mechanism to induce memory reclaim Message-ID: References: <5df21376-7dd1-bf81-8414-32a73cea45dd@google.com> <20220307183141.npa4627fpbsbgwvv@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 196D6C0019 X-Stat-Signature: 99stnordzdkardc5teofw5tjk9j16r3y Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=H8+sduPP; spf=pass (imf28.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com X-HE-Tag: 1646755515-115943 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue 08-03-22 09:44:35, Dan Schatzberg wrote: > On Tue, Mar 08, 2022 at 01:53:19PM +0100, Michal Hocko wrote: > > On Mon 07-03-22 15:26:18, Johannes Weiner wrote: [...] > > > A mechanism to request a fixed number of pages to reclaim turned out > > > to work much, much better in practice. We've been using a simple > > > per-cgroup knob (like here: https://lkml.org/lkml/2020/9/9/1094). > > > > Could you share more details here please? How have you managed to find > > the reclaim target and how have you overcome challenges to react in time > > to have some head room for the actual reclaim? > > We have a userspace agent that just repeatedly triggers proactive > reclaim and monitors PSI metrics to maintain some constant but low > pressure. In the complete absense of pressure we will reclaim some > configurable percentage of the workload's memory. This reclaim amount > tapers down to zero as PSI approaches the target threshold. > > I don't follow your question regarding head-room. Could you elaborate? One of the concern that was expressed in the past is how effectively can pro-active userspace reclaimer act on memory demand transitions. It takes some time to get refaults/PSI changes and then you should be acting rather swiftly. At least if you aim at somehow smooth transition. Tuning this up to work reliably seems to be far from trivial. Not to mention that changes in the memory reclaim implementation could make the whole tuning rather fragile. -- Michal Hocko SUSE Labs