From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B977C54FD0 for ; Mon, 27 Apr 2020 11:07:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B47602063A for ; Mon, 27 Apr 2020 11:07:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PlCKK2y9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B47602063A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4D16C8E0005; Mon, 27 Apr 2020 07:07:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 482A58E0001; Mon, 27 Apr 2020 07:07:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BEC38E0005; Mon, 27 Apr 2020 07:07:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0179.hostedemail.com [216.40.44.179]) by kanga.kvack.org (Postfix) with ESMTP id 21E6E8E0001 for ; Mon, 27 Apr 2020 07:07:30 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id D4F18181AC9CC for ; Mon, 27 Apr 2020 11:07:29 +0000 (UTC) X-FDA: 76753359018.10.stage20_5e83d0e7ae95b X-HE-Tag: stage20_5e83d0e7ae95b X-Filterd-Recvd-Size: 6425 Received: from mail-il1-f193.google.com (mail-il1-f193.google.com [209.85.166.193]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Mon, 27 Apr 2020 11:07:29 +0000 (UTC) Received: by mail-il1-f193.google.com with SMTP id r2so16250043ilo.6 for ; Mon, 27 Apr 2020 04:07:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=/kDN5QfiykJ3t6eJ58mHa+kRzjqghoVEdrXQn819CWg=; b=PlCKK2y9D8q6U4bXlRKqeF9w3VH8a3mDZs4/p9sF6UNfPDwrcTTFkB7dnS/xi3HkMW TVy0hRSobBYRIJC5A7ZvGV2iJgUWVoWRxjntAa1huzzZmR2W09pSvtJ3jLfpOnj154TG Ua0EXxTAHbENt5yovs5PEbtjSkwnI/UiBS1BUCMz/J7VMwEC5ZQ1GTpuy4g19AvWeRO9 nhlj49RKBGDNcznpuoA0HP+6hh8mvyQDlg/FGeB+WwiIS5UTqD/AZjfTJi2E2mjTWupk lSYeFya5IrQIqbIZWuJHSpkuS3SdFMFWgwYoyTZB175agImwWLZufVxG4/VAQ/C0aJ/Q 9Cpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/kDN5QfiykJ3t6eJ58mHa+kRzjqghoVEdrXQn819CWg=; b=KmbRxkKVme3lRDTXpWqnk+wRyaeemY0zl1QaWuUgNth8xc1pSNXr15iukKO5tecUz/ xXOLfTReoutYP20W9LXBv4LeiGWyhXXG9wZv0MgirPjEO15f/sH6Q3HGU+49FdwXjG6S O2kIGYbatOXquAshC515fPTb7Nlse8g3fSw1UwhJ5et0Tws5LvrZNmYGdfu6YqL/oSmj K2xaH5sptvm7QzZK4dYAJmvW26XHOUVgGS6mF2DMYcTAeXvwEJhw7Ym3pm3YuoroId8g xtlkRvpgruxX0ElyQ/NrTXZ7ySHMl4EmL9/DEwm3JD35T386Gk8jYRyyDqW24tVu167C kH9w== X-Gm-Message-State: AGi0PuYJ+uyiyATlq76IvS/xYxWV3qdLSB7dTj4BLCaR8xRjH2zXnFV5 5AC/a/9lU44+oaRvZj9C/UqLb1gRFoyc+uIfnfY= X-Google-Smtp-Source: APiQypJBvIw4kkwzZIZEFatKarLGe7gQGcraJZCXVXCYV2frQ+z2Ws4Y9EZfh6aty8lIos93HcTkZ072/p1N0Kycqio= X-Received: by 2002:a92:c004:: with SMTP id q4mr19452690ild.93.1587985648733; Mon, 27 Apr 2020 04:07:28 -0700 (PDT) MIME-Version: 1.0 References: <20200425152418.28388-1-laoar.shao@gmail.com> <20200425152418.28388-4-laoar.shao@gmail.com> <20200427094006.GD28637@dhcp22.suse.cz> <20200427105022.GE28637@dhcp22.suse.cz> In-Reply-To: <20200427105022.GE28637@dhcp22.suse.cz> From: Yafang Shao Date: Mon, 27 Apr 2020 19:06:52 +0800 Message-ID: Subject: Re: [PATCH 3/3] mm: improvements on memcg protection functions To: Michal Hocko Cc: Andrew Morton , Johannes Weiner , Roman Gushchin , Chris Down , Linux MM Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Apr 27, 2020 at 6:50 PM Michal Hocko wrote: > > On Mon 27-04-20 18:09:27, Yafang Shao wrote: > > On Mon, Apr 27, 2020 at 5:40 PM Michal Hocko wrote: > > > > > > On Sat 25-04-20 11:24:18, Yafang Shao wrote: > > > > Since proportional memory.{min, low} reclaim is introduced in > > > > commit 9783aa9917f8 ("mm, memcg: proportional memory.{low,min} reclaim"), > > > > it have been proved that the proportional reclaim is hard to understand and > > > > the issues caused by it is harder to understand.[1]. That dilemma faced by > > > > us is caused by that the proportional reclaim mixed up memcg and the > > > > reclaim context. > > > > > > > > In proportional reclaim, the whole reclaim context - includes the memcg > > > > to be reclaimed and the reclaimer, should be considered, rather than > > > > memcg only. > > > > > > > > To make it clear, a new member 'protection' is introduced in the reclaim > > > > context (struct shrink_control) to replace mem_cgroup_protection(). This > > > > > > s@shrink_control@scan_control@ > > > > > > > Thanks for pointing this out. > > > > > > one is set when we check whether the memcg is protected or not. > > > > > > > > After this change, the issue pointed by me[1] - a really old left-over > > > > value can slow donw target reclaim - can be fixed, and I think it could > > > > also avoid some potential race. > > > > > > The patch would have been much esier to review if you only focused on > > > the effective protection value caching. I really fail to see why you had > > > to make mem_cgroup_protected even more convoluted with more side effects > > > (e.g. sc->memcg_low_skipped). This goes directly opposite to what > > > Johannes was proposing in other email AFAICS. > > > > > > > Sorry, I failed to see what the advantage of Johannes's proposal > > except the better naming. > > The immediate advantage is that predicate should better not have any > side effect. No, it still has side effect. That is not an immediate advantage neither. See bellow, > So splitting into the calculation part which has clearly > defined side effects and having a simple predicate that consults > pre-calculated values makes a lot of sense to me. > When you calculate the effective values, the source values to calculate it may be changed with these values when you do the predicate. IOW, this proposal greatly increase the race window. > > > Your changelog doesn't explain why double caching the effective value is > > > an improvement. > > > > The improvement is, to avoid getting an wrong value calculated by > > other reclaimers and avoid issues in mem_cgroup_protection() that we > > haven't noticed. > > This is not true in general. There is still parallel calculation done > and so parallel reclaimers might affect each other. Your patch only > makes a real difference for leaf memcgs which are the reclaim target as > well. The race between parallel reclaimers is fundamentally exist, and we can do now is reducing the race window as far as possible. > All intermediate nodes really do not care about the cached values > because they do not have any pages on the LRU lists. > But read these cached values can save lot of time against the existing code, as the existing code still calculates them even if they are useless. If you think that is a issue, I think why not skip scanning these intermediate nodes because they don't have any pages on the LRU lists ? -- Thanks Yafang