From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1EBCC34021 for ; Mon, 17 Feb 2020 13:52:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B5D572072C for ; Mon, 17 Feb 2020 13:52:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UIl3TVdL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728676AbgBQNwA (ORCPT ); Mon, 17 Feb 2020 08:52:00 -0500 Received: from mail-io1-f65.google.com ([209.85.166.65]:44446 "EHLO mail-io1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726710AbgBQNwA (ORCPT ); Mon, 17 Feb 2020 08:52:00 -0500 Received: by mail-io1-f65.google.com with SMTP id z16so18472071iod.11 for ; Mon, 17 Feb 2020 05:51:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sCJ+p0Vb+K6tWMUVo+6NRjb+ZnIa1bhM/VCrHSW9Fpw=; b=UIl3TVdL/RKWxNBloWs1txJrtWy+iMo502sWyMwSwjwss3T5MzKecu70L8rSr/GHoY bIYkJG09In0dDU5/WbAgdOIMAG6FNxXXhnBef1k/s3DA8Ju8/Vuqo4JkUqZtS1jOPI3X C+9iRiNhPr9kx/BlmZ4RvhADMg1kOAlObxTb8Pfo5NXK9W/77FB0kY7L9UB/h9LJ5YEs 7Wwk1Taz4nHjejo2rKAB4jtrL4hXpW2AEqWVpCzJhTGYcq9i29g7EbtKOI1lmkgMJM1P f43MPDXPKXJmBrGpSXJJNfQx2LPuHfirKNqxaXzM5gmhGb7IHmgcOuogyL0Z+2J+8KuU pZeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sCJ+p0Vb+K6tWMUVo+6NRjb+ZnIa1bhM/VCrHSW9Fpw=; b=E55S4cTBO4oEQgZMjqp6Hdyqh5IixQa8YTJvPf7olYcWpbP3XugFruJq+sRQadFVPk dWTwwwl385fto/+f7pl/d1+cAOjcVYkvmsx5pKCgsj+HlRhw2qgcZZPrEqfr5xiA8/Ed fnnbnc1Ov+SyBxYGiNYDsVPTX97Wfn4YA7ztSeX5X7GfPgxTqOBSUUv6GsyzzzrpepJV 6UcY/1ERZy6uQGSP8s9s66lh+jqdtUQj4XH1VPSfXts6cAKUtbxahf+vKXDfZsJzKoAa nutzJST8Q+7k0BPLYUadRyIK0O5Oy2d3VHOj6/822UjQqQ8z5RKUIXR721XLpRgfmwsb XYrw== X-Gm-Message-State: APjAAAUUW5pgCkAy4GIQwHU605OHWaNEej4uqmkLudNmgGXhOlNlf0oB cn7ZfOmmD+xialApMfc28AESOcKMjlR7cwhRjO8= X-Google-Smtp-Source: APXvYqwn0NEa/g+vhhWyeD6jEsb7IyCXqAt/zRbG60ECxntPpclt31tC0QlBX0tdn6+ho8SY4UifdLx7FcNbJ7oXqzE= X-Received: by 2002:a02:b615:: with SMTP id h21mr12223540jam.109.1581947519359; Mon, 17 Feb 2020 05:51:59 -0800 (PST) MIME-Version: 1.0 References: <20200216145249.6900-1-laoar.shao@gmail.com> <20200217092459.GG31531@dhcp22.suse.cz> <20200217132443.GM31531@dhcp22.suse.cz> In-Reply-To: <20200217132443.GM31531@dhcp22.suse.cz> From: Yafang Shao Date: Mon, 17 Feb 2020 21:51:23 +0800 Message-ID: Subject: Re: [PATCH resend] mm, memcg: reset memcg's memory.{min, low} for reclaiming itself To: Michal Hocko Cc: Roman Gushchin , Johannes Weiner , Vladimir Davydov , Andrew Morton , Chris Down , Linux MM , stable@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org On Mon, Feb 17, 2020 at 9:24 PM Michal Hocko wrote: > > On Mon 17-02-20 21:08:12, Yafang Shao wrote: > > On Mon, Feb 17, 2020 at 5:25 PM Michal Hocko wrote: > > > > > > On Sun 16-02-20 09:52:49, Yafang Shao wrote: > > > > memory.{emin, elow} are set in mem_cgroup_protected(), and the values of > > > > them won't be changed until next recalculation in this function. After > > > > either or both of them are set, the next reclaimer to relcaim this memcg > > > > may be a different reclaimer, e.g. this memcg is also the root memcg of > > > > the new reclaimer, and then in mem_cgroup_protection() in get_scan_count() > > > > the old values of them will be used to calculate scan count, that is not > > > > proper. We should reset them to zero in this case. > > > > > > > > Here's an example of this issue. > > > > > > > > root_mem_cgroup > > > > / > > > > A memory.max=1024M memory.min=512M memory.current=800M > > > > > > > > Once kswapd is waked up, it will try to scan all MEMCGs, including > > > > this A, and it will assign memory.emin of A with 512M. > > > > After that, A may reach its hard limit(memory.max), and then it will > > > > do memcg reclaim. Because A is the root of this reclaimer, so it will > > > > not calculate its memory.emin. So the memory.emin is the old value > > > > 512M, and then this old value will be used in > > > > mem_cgroup_protection() in get_scan_count() to get the scan count. > > > > That is not proper. > > > > > > Please document user visible effects of this patch. What does it mean > > > that this is not proper behavior? > > > > In the memcg reclaim, if the target memcg is the root of the reclaimer, > > the reclaimer should scan this memcg's all page cache pages in the LRU, > > but now as the old memcg.{emin, elow} value are still there, it will get > > a wrong protection value, > > and the reclaimer can't reclaim the page cache pages protected by this > > wrong protection. > > Could you be more specific please. Your example above says that emin is > not going to be recalculated and stays at 512M even for a potential max > limit reclaim. The min limit is still 512M so why is this value wrong? > Because the relcaimers are changed or the root the relcaimer is changed. Kswapd begins to relcaim memcg-A. kswapd | calculate the {emin, elow} for memcg-A | stores {emin, elow} in memory.{emin, elow} of memcg-A | This memory.{emin, elow} will protect the page cache pages in memcg-A (See get_scan_count->mem_cgroup_protection) | exit (And it won't relcaim memcg-A for a long time) Then memcg relcaimer is woke up (reached the hard limit of memcg-A), and the root of this new reclaimer is memcg-A. This memcg relcaimer begins to reclaim memcg-A. memcg relcaimer | As the root of the relcaimer is memcg-A, it won't calculate emin, elow for memcg-A. (See if (memcg == root) in mem_cgroup_protected()) | The old memory.{emin, elow} will protect the page cache pages in memcg-A (SO WE SHOULD CLEAR THE OLD VALUE) | exit I try my best to illustrate it. Hope it could clarify. > > > What happens if we have concurrent > > > reclaimers at different levels of the hierarchy how that would affect > > > the resulting protection? > > > > > > > Well, I thought the synchronization mechanisms have already existed ? > > Otherwise there must be concurrent issue in the original code of > > setting the memcg.{emin, elow} as well. > > (Because memcg->memory.{emin, elow} are also set at the end of the > > function mem_cgroup_protected()) > > This function is documented to be racy and I believe this is OK because > it doesn't really have to be precise and concurrent updates are not > going to change values much. But does the same apply to reseting the > effective values? Maybe yes. Make sure to document this in the changelog > please. Sure. I will document it. -- Yafang Shao DiDi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B06CC34023 for ; Mon, 17 Feb 2020 13:52:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CC514206E2 for ; Mon, 17 Feb 2020 13:52:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UIl3TVdL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC514206E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 28ED46B0005; Mon, 17 Feb 2020 08:52:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 23F1A6B0006; Mon, 17 Feb 2020 08:52:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1550C6B0007; Mon, 17 Feb 2020 08:52:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0152.hostedemail.com [216.40.44.152]) by kanga.kvack.org (Postfix) with ESMTP id F19766B0005 for ; Mon, 17 Feb 2020 08:52:00 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7FFBC1F06 for ; Mon, 17 Feb 2020 13:52:00 +0000 (UTC) X-FDA: 76499757600.21.party87_4bada9cd2cc03 X-HE-Tag: party87_4bada9cd2cc03 X-Filterd-Recvd-Size: 6927 Received: from mail-io1-f66.google.com (mail-io1-f66.google.com [209.85.166.66]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Mon, 17 Feb 2020 13:51:59 +0000 (UTC) Received: by mail-io1-f66.google.com with SMTP id z193so18514083iof.1 for ; Mon, 17 Feb 2020 05:51:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sCJ+p0Vb+K6tWMUVo+6NRjb+ZnIa1bhM/VCrHSW9Fpw=; b=UIl3TVdL/RKWxNBloWs1txJrtWy+iMo502sWyMwSwjwss3T5MzKecu70L8rSr/GHoY bIYkJG09In0dDU5/WbAgdOIMAG6FNxXXhnBef1k/s3DA8Ju8/Vuqo4JkUqZtS1jOPI3X C+9iRiNhPr9kx/BlmZ4RvhADMg1kOAlObxTb8Pfo5NXK9W/77FB0kY7L9UB/h9LJ5YEs 7Wwk1Taz4nHjejo2rKAB4jtrL4hXpW2AEqWVpCzJhTGYcq9i29g7EbtKOI1lmkgMJM1P f43MPDXPKXJmBrGpSXJJNfQx2LPuHfirKNqxaXzM5gmhGb7IHmgcOuogyL0Z+2J+8KuU pZeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sCJ+p0Vb+K6tWMUVo+6NRjb+ZnIa1bhM/VCrHSW9Fpw=; b=SBPXzTZ9jYq5upHd5s2eNnJ4L3aFSa1b+V2NhfIbVzQdW28hMdB+rzgpP4v3njvfv5 ZNnhgZVF6CzWRkgUkB6ZcirVbjsa4NcfpiGxAcDfrnKPvgotwMhgLZjHe1JWgASpL0TV 1ycrArb5kYuiqAIxSyrUmVOjcucMGfSDhfgClO/mUJ7oFp8RLCpGr20CmIB3XKN+cblF dGZFWgVniNoBS+ifsNiStHKTqeh63WqQkHPzHKMt5jWSNgdKCYu+KHxAVpcq8SwlK5No LFqVf9Uu6Z3B2y6EoTMiVp3HIrAoH4wmGI/mVVQ3lQhAxYpQLc8xai+CW4ezIQBBptvh qvOg== X-Gm-Message-State: APjAAAVG4IT2qVaU1vJPqawKwLRdhaWRQbblSuVKmGcL079ZezLcctrk VtGriRwYGkdNfAsfd3mSeLrvPJ2IfFMWPtNjw5M= X-Google-Smtp-Source: APXvYqwn0NEa/g+vhhWyeD6jEsb7IyCXqAt/zRbG60ECxntPpclt31tC0QlBX0tdn6+ho8SY4UifdLx7FcNbJ7oXqzE= X-Received: by 2002:a02:b615:: with SMTP id h21mr12223540jam.109.1581947519359; Mon, 17 Feb 2020 05:51:59 -0800 (PST) MIME-Version: 1.0 References: <20200216145249.6900-1-laoar.shao@gmail.com> <20200217092459.GG31531@dhcp22.suse.cz> <20200217132443.GM31531@dhcp22.suse.cz> In-Reply-To: <20200217132443.GM31531@dhcp22.suse.cz> From: Yafang Shao Date: Mon, 17 Feb 2020 21:51:23 +0800 Message-ID: Subject: Re: [PATCH resend] mm, memcg: reset memcg's memory.{min, low} for reclaiming itself To: Michal Hocko Cc: Roman Gushchin , Johannes Weiner , Vladimir Davydov , Andrew Morton , Chris Down , Linux MM , stable@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Feb 17, 2020 at 9:24 PM Michal Hocko wrote: > > On Mon 17-02-20 21:08:12, Yafang Shao wrote: > > On Mon, Feb 17, 2020 at 5:25 PM Michal Hocko wrote: > > > > > > On Sun 16-02-20 09:52:49, Yafang Shao wrote: > > > > memory.{emin, elow} are set in mem_cgroup_protected(), and the values of > > > > them won't be changed until next recalculation in this function. After > > > > either or both of them are set, the next reclaimer to relcaim this memcg > > > > may be a different reclaimer, e.g. this memcg is also the root memcg of > > > > the new reclaimer, and then in mem_cgroup_protection() in get_scan_count() > > > > the old values of them will be used to calculate scan count, that is not > > > > proper. We should reset them to zero in this case. > > > > > > > > Here's an example of this issue. > > > > > > > > root_mem_cgroup > > > > / > > > > A memory.max=1024M memory.min=512M memory.current=800M > > > > > > > > Once kswapd is waked up, it will try to scan all MEMCGs, including > > > > this A, and it will assign memory.emin of A with 512M. > > > > After that, A may reach its hard limit(memory.max), and then it will > > > > do memcg reclaim. Because A is the root of this reclaimer, so it will > > > > not calculate its memory.emin. So the memory.emin is the old value > > > > 512M, and then this old value will be used in > > > > mem_cgroup_protection() in get_scan_count() to get the scan count. > > > > That is not proper. > > > > > > Please document user visible effects of this patch. What does it mean > > > that this is not proper behavior? > > > > In the memcg reclaim, if the target memcg is the root of the reclaimer, > > the reclaimer should scan this memcg's all page cache pages in the LRU, > > but now as the old memcg.{emin, elow} value are still there, it will get > > a wrong protection value, > > and the reclaimer can't reclaim the page cache pages protected by this > > wrong protection. > > Could you be more specific please. Your example above says that emin is > not going to be recalculated and stays at 512M even for a potential max > limit reclaim. The min limit is still 512M so why is this value wrong? > Because the relcaimers are changed or the root the relcaimer is changed. Kswapd begins to relcaim memcg-A. kswapd | calculate the {emin, elow} for memcg-A | stores {emin, elow} in memory.{emin, elow} of memcg-A | This memory.{emin, elow} will protect the page cache pages in memcg-A (See get_scan_count->mem_cgroup_protection) | exit (And it won't relcaim memcg-A for a long time) Then memcg relcaimer is woke up (reached the hard limit of memcg-A), and the root of this new reclaimer is memcg-A. This memcg relcaimer begins to reclaim memcg-A. memcg relcaimer | As the root of the relcaimer is memcg-A, it won't calculate emin, elow for memcg-A. (See if (memcg == root) in mem_cgroup_protected()) | The old memory.{emin, elow} will protect the page cache pages in memcg-A (SO WE SHOULD CLEAR THE OLD VALUE) | exit I try my best to illustrate it. Hope it could clarify. > > > What happens if we have concurrent > > > reclaimers at different levels of the hierarchy how that would affect > > > the resulting protection? > > > > > > > Well, I thought the synchronization mechanisms have already existed ? > > Otherwise there must be concurrent issue in the original code of > > setting the memcg.{emin, elow} as well. > > (Because memcg->memory.{emin, elow} are also set at the end of the > > function mem_cgroup_protected()) > > This function is documented to be racy and I believe this is OK because > it doesn't really have to be precise and concurrent updates are not > going to change values much. But does the same apply to reseting the > effective values? Maybe yes. Make sure to document this in the changelog > please. Sure. I will document it. -- Yafang Shao DiDi