From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FB96C4332F for ; Fri, 9 Dec 2022 21:40:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B00A98E0003; Fri, 9 Dec 2022 16:40:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AB0EA8E0001; Fri, 9 Dec 2022 16:40:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 978418E0003; Fri, 9 Dec 2022 16:40:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 89FA78E0001 for ; Fri, 9 Dec 2022 16:40:00 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 61D441C5D18 for ; Fri, 9 Dec 2022 21:40:00 +0000 (UTC) X-FDA: 80224085760.17.1CA3CFC Received: from mail-vs1-f43.google.com (mail-vs1-f43.google.com [209.85.217.43]) by imf24.hostedemail.com (Postfix) with ESMTP id C4915180019 for ; Fri, 9 Dec 2022 21:39:58 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="BnhqE1j/"; spf=pass (imf24.hostedemail.com: domain of almasrymina@google.com designates 209.85.217.43 as permitted sender) smtp.mailfrom=almasrymina@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670621998; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=20fWTWqG3b1NNXuV0UqK1yxQ/Ktl7cxXS71QLmWNzcs=; b=uIyBrvHhBfUn1urI+2Zv5XQCni3DWGQk0Mt/t9Ez9AoA+y+NN6tHGBwfZ0f90DLraj0We/ dSqhSUpvTzIYAQcQxEQQVj+hJYJVTCqiYWMR9UYC02P0j6HbTsWqpokofkULym29q/OpZV i26zovLRdB0CfhxhgpwfaXQxXJ+khGw= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="BnhqE1j/"; spf=pass (imf24.hostedemail.com: domain of almasrymina@google.com designates 209.85.217.43 as permitted sender) smtp.mailfrom=almasrymina@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670621998; a=rsa-sha256; cv=none; b=P1Xv0QuVwXL7Vf7qjacrleTHvFHO9GLppdX+MpTvkjnMxYoN5J4m27AK0F0+k2+77EcndH VQEVslLgi4P3YwHDGB/dP42tnfT6wdL6JKq5SBXvGDpTuo9CaMx+y3iErYlw+bNYpGDsuJ cFOwBdj9sDcmnN1Y8kWU8aDS4BLtDWw= Received: by mail-vs1-f43.google.com with SMTP id c184so5764409vsc.3 for ; Fri, 09 Dec 2022 13:39:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=20fWTWqG3b1NNXuV0UqK1yxQ/Ktl7cxXS71QLmWNzcs=; b=BnhqE1j/P331hqbRIW+1zWjX2X6wDnR895yP+axeKfFphfSMfyTNbZYwuTh1KqoWrX g8+5allE98PfMOuEtX1gTvtpIKGLqQNbtIjLIzih3NOOeXIHLfFuTzD1J+84oLDIeUc/ ytiPO2eNVUlKMT3YRiVYPaL3TC/YqPGZIoVFLL5D/ZhXpa0nNVNtagtGFd/92/BsMhoC kHW6h5lEI7nT3JHSq3pks8s8ZY6qZavgBP+0ESTNOJVh6EM5Yi+ZMB82eADgJ55dgG7c eL1e/A1iWoeg+HKE400v2Eht3GiPWflB+K0ov7T6fDO7W304XB4TPbg5whESSa/VHuvn Lx1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=20fWTWqG3b1NNXuV0UqK1yxQ/Ktl7cxXS71QLmWNzcs=; b=cFCOR36w65QUQyOEjyLBxFM84w2XdzMJC7njGSB5EQYiGKkiOdwTIXlI08RS1dL1x7 BWd0WDlYkoaloivfdYJden6J4iouK7KGrhN6vOanQiDKfAJrCn1+bZ35jWUNfBxvBZxP 7MnRvn0M4EB/fOx5qC0BfWaiSuGbTnod9TGg4fhqZve3Yww2MSR/8BLaefgIFgFg05jq hBbsOebmDxRLw2M7rHf8Efh2GDCrR2cQ8/Bnug325c2ryycAhSgC2ypsCZeQMQSDOHak SLXypgggY5ktZGHyo1SqRrfpwyVj4vRAUk8wu9kvD/EHGr22mdkIE7GI6Sm/0woNSMeA Abtw== X-Gm-Message-State: ANoB5pnVAOnVRdVmWRLofGf5SDB7tYRMfgg2qezik39GA280IDYhBpdH R6UwQghgW0UhMM+Le1XyFVNlnEoSbkzvmsXimSbfnw== X-Google-Smtp-Source: AA0mqf79LwG4XbYGxkIBYEJjxz+6ThdW7CDhmx6hjMBhsMOa+WjsBHRBigeL1N5n84sx4YuEJ/CENPIg7E/Qenvq558= X-Received: by 2002:a67:fe53:0:b0:3b1:3d9a:6932 with SMTP id m19-20020a67fe53000000b003b13d9a6932mr9775580vsr.59.1670621997768; Fri, 09 Dec 2022 13:39:57 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Mina Almasry Date: Fri, 9 Dec 2022 13:39:44 -0800 Message-ID: Subject: Re: [PATCH v3] [mm-unstable] mm: Fix memcg reclaim on memory tiered systems To: Michal Hocko Cc: Wei Xu , Andrew Morton , Johannes Weiner , Roman Gushchin , Shakeel Butt , Muchun Song , Huang Ying , Yang Shi , Yosry Ahmed , fvdl@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: C4915180019 X-Stat-Signature: bscncp3uo7q9h3ytgqmgkcxfgpn6mrd9 X-Rspam-User: X-HE-Tag: 1670621998-213136 X-HE-Meta: U2FsdGVkX1/RXMPbJGlLPDs2jeaNZ9QLoj6i0K3GG4c8iFEwqY0ipJgjpPzqFr3SLBhVu0M76M3Xl14mFkReYE4+cxuj11d60PL5MyJ5YnEJJvfi/dm5fWD+IYzzToQElwyAb7lHb0bQiQurFCmA0Y1/4Uh+rBTsk8KvOdFCf+3gzr15I529D1LwKqdx2s5eyYWirl5f1LB3e89binNvXV3HQDGnf0vgtfmaR+G4/kWk+e5wLsIpCyMK4FmsaJrlHAkiTbWW0CoLi0/pLMktReJagmKewkodS0lB8KmUCcDeZCXclswdlJJPeY08T/NVRyA7kzpdv/1p0+Cx8UvVZIEMLO5a+A96pLptI16MDJJFbruESHq/+dfaosYhYShi2CCUYjw0u3QpN3aF4ggmamJdHrrh0foL+NLPArzDYUrsUWGJw7ErcuvFJcE5ItB8Wca3Szhh2lbcShJ8EvIuokIR9Qx3bzwavYgo3V1M82iT6lHSJHzatlr8na4qKr22r3tJmpeSN/u5ewhBhOMi9QvfmduCSmeFM5dJ/rjP01D+gCXW09hyE0YTOx7ZosNhg0cp/FAMAyiCeXMUYyJm03ZXTlWn5wePWj3TljVhRjveGIpGiMBiok4WtYYu2Auy5p+AOuLChpJkzxpLOnxQHyOvk+xxC04AwbKUiUA1n+8PqAgh+xMUA406MF5arJhjTftnER/Oz9pAEtgBWFhcO8T5dCUGKdPnLsb3VhbUNc2mVrFLx2nQ3dCM74xA39TEq0SFfF46hNllK9IRYQ/Qz9FWxntk22b7HIILQ6RF1Pq9qD2HZ69YkGKnpFCVO+hju5xltXpQeF+iilcehBFP1b65CkFQxT0+aFZpMKwfp+pduhSVR/9HSGIn0qBLGEeAgKiatVZ2w6zsk4fk4UzNSkm8kx6SbbUHEsM2ATdIKQ811j4HT0LsTV9K0W+9bTXyl2RF0CKt8sfHmEwU6Mo spU/Swz4 XU99ahs+lhyjD4gQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Dec 9, 2022 at 1:16 PM Michal Hocko wrote: > > On Fri 09-12-22 08:41:47, Wei Xu wrote: > > On Fri, Dec 9, 2022 at 12:08 AM Michal Hocko wrote: > > > > > > On Thu 08-12-22 16:59:36, Wei Xu wrote: > > > [...] > > > > > What I really mean is to add demotion nodes to the nodemask along with > > > > > the set of nodes you want to reclaim from. To me that sounds like a > > > > > more natural interface allowing for all sorts of usecases: > > > > > - free up demotion targets (only specify demotion nodes in the mask) > > > > > - control where to demote (e.g. select specific demotion target(s)) > > > > > - do not demote at all (skip demotion nodes from the node mask) > > > > > > > > For clarification, do you mean to add another argument (e.g. > > > > demotion_nodes) in addition to the "nodes" argument? > > > > > > No, nodes=mask argument should control the domain where the memory > > > reclaim should happen. That includes both aging and the reclaim. If the > > > mask doesn't contain any lower tier node then no demotion will happen. > > > If only a subset of lower tiers are specified then only those could be > > > used for the demotion process. Or put it otherwise, the nodemask is not > > > only used to filter out zonelists during reclaim it also restricts > > > migration targets. > > > > > > Is this more clear now? > > I think putting the demotion sources and demotion targets in the same nodemask is a bit confusing, and prone to error. IIUC the user puts both the demotion source and the demotion target in the nodemaks, and the kernel infers which is which depending on whether the node is a top-tier node, or a bottom tier node. I think in the future this will become ambiguous. What happens in the future when the user when the machine has N memory tiers and the user specifies a node in a middle tier in the nodemask? Does that mean the user wants demotion from or to this node? Middle memory tiers can act as both... I think if your goal is to constrain demotion targets then a much more clear and future proof way is to simply add a second arg to memory.reclaim "allowed_demotion_targets=".\ > > In that case, how can we request demotion only from toptier nodes > > (without counting any reclaimed bytes from other nodes), which is our > > memory tiering use case? > > I am not sure I follow. Could you be more specific please? > > > Besides, when both toptier and demotion nodes are specified, the > > demoted pages should only be counted as aging and not be counted > > towards the requested bytes of try_to_free_mem_cgroup_pages(), which > > is what this patch tries to address. > > This should be addressed by > http://lkml.kernel.org/r/Y5B1K5zAE0PkjFZx@dhcp22.suse.cz, no? I think I provided a test case in [1] showing very clearly that this breaks one of our use cases, i.e. the use case where the user is asking to demote X bytes from the top tier nodes to the lower tier nodes. I would not like to proceed with a fix that breaks one of our use cases. I believe I provided in this patch a fix that caters to all existing users, and we should take the fix in this patch over a fix that breaks use cases. [1] https://lore.kernel.org/all/CAHS8izMKK107wVFSJvg36nQ=WzXd8_cjYBtR0p47L+XLYUSsqA@mail.gmail.com/ > -- > Michal Hocko > SUSE Labs