From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B262C432C0 for ; Mon, 25 Nov 2019 14:45:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5156D2068F for ; Mon, 25 Nov 2019 14:45:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="D7a7b/hB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5156D2068F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D73E16B05D4; Mon, 25 Nov 2019 09:45:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CFE606B05D5; Mon, 25 Nov 2019 09:45:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA08E6B05D6; Mon, 25 Nov 2019 09:45:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id 9FA236B05D4 for ; Mon, 25 Nov 2019 09:45:20 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 6F545180AD81C for ; Mon, 25 Nov 2019 14:45:20 +0000 (UTC) X-FDA: 76195072800.27.veil08_117328c214840 X-HE-Tag: veil08_117328c214840 X-Filterd-Recvd-Size: 6465 Received: from mail-il1-f194.google.com (mail-il1-f194.google.com [209.85.166.194]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Mon, 25 Nov 2019 14:45:19 +0000 (UTC) Received: by mail-il1-f194.google.com with SMTP id o18so14368390ils.12 for ; Mon, 25 Nov 2019 06:45:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=bV8YakA0yZcBIQeANMl3hRhgvl7MxTEnEd6JiKuxC8I=; b=D7a7b/hB2bxj7z6CvOsJAS8bBed8SwpdQl6UZlBN9fB83YsMMAqdaYt8s9kp/qoGTj ADlBsqZNq8Qgnz87JlxjFQ73CmmAcH7osCPCjw/szYGC8lYqfMQO1Z47gQK325t2PgNl H6X3vJ+LkaKN4h69X/dXUXecyS4dP/tCCpOKbll24V5wtG0UG1iv6xpAfWgG6x+uFnd2 vp3nPOCGMGPmSfWlcl2MIInWRG9EYjaiVnKbKwi7l05IvyWaumduHo96J6hIpFe2fruV ImkqYmzTKiQvJP0qaUc7Ce2kCKRbNm+3lBtncqCzjhUhWRrJOxc2LnIZ7/ThDtZnmZ8Z 06Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bV8YakA0yZcBIQeANMl3hRhgvl7MxTEnEd6JiKuxC8I=; b=HXInbsIuSAikNsJS0VsxhT2mex/xc0R3acVUZIz1l+W+uUVOoBHst4e3YbbU8Wh64M f/Hr48TTEA73bdtV4D1Ubnt20rBEK8lTCMw6JFG6paiR/8KaSvGRPYOustbjjOvWIRol fMt/Ohdr7U1a1pG+p5sWFCKk9BqdOTqn7h/5snK1awsmKED1AkxT67t46+TWUai/K9FE wXzOpplIF9J6UbrhTrVUpHgyk8/70/WmYQ3n6cD6ai/z64SfPfOCZ9YGA2tSc3+W+mj4 WVZ1C7KamNhha3rbEZZU5XgqIU409Lk3CQgafOUjrxyMgPhjXCIGwEbOAuz3hPTPc4cu RQEQ== X-Gm-Message-State: APjAAAWO9LEjXhqYTHqIvhtQD82nOUvospa3R6t+2QazRZDdCr6XuFH4 TQX9x63H9y3vr8x4+g+dKZmPzeJP8wVcUKB6pxzJPiQ7 X-Google-Smtp-Source: APXvYqx7CadM4FcvCVSHddqiOW9ebOlW3SzieR45csE4Ka8aEOGFhNDtJXYqbYJ92lBjzaS4Nd9EBB7uB2xSZhZ5eHs= X-Received: by 2002:a92:109c:: with SMTP id 28mr32741486ilq.142.1574693118159; Mon, 25 Nov 2019 06:45:18 -0800 (PST) MIME-Version: 1.0 References: <1574676893-1571-1-git-send-email-laoar.shao@gmail.com> <20191125110848.GH31714@dhcp22.suse.cz> <20191125115409.GJ31714@dhcp22.suse.cz> <20191125123123.GL31714@dhcp22.suse.cz> <20191125124553.GM31714@dhcp22.suse.cz> <20191125142150.GP31714@dhcp22.suse.cz> In-Reply-To: <20191125142150.GP31714@dhcp22.suse.cz> From: Yafang Shao Date: Mon, 25 Nov 2019 22:44:41 +0800 Message-ID: Subject: Re: [PATCH] mm, memcg: clear page protection when memcg oom group happens To: Michal Hocko Cc: Johannes Weiner , Vladimir Davydov , Andrew Morton , Linux MM Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Nov 25, 2019 at 10:21 PM Michal Hocko wrote: > > On Mon 25-11-19 22:11:15, Yafang Shao wrote: > > On Mon, Nov 25, 2019 at 8:45 PM Michal Hocko wrote: > > > > > > On Mon 25-11-19 20:37:52, Yafang Shao wrote: > > > > On Mon, Nov 25, 2019 at 8:31 PM Michal Hocko wrote: > [...] > > > > > Again, what is a problem that you are trying to fix? > > > > > > > > When there's no processes running in a memcg, for example if they are > > > > killed by OOM killer, we can't reclaim the file page cache protected > > > > by memory.min of this memcg. These file page caches are useless in > > > > this case. > > > > That's what I'm trying to fix. > > > > > > Could you be more specific please? I would assume that the group oom > > > configured memcg would either restart its workload when killed (that is > > > why you want to kill the whole workload to restart it cleanly in many > > > case) or simply tear down the memcg altogether. > > > > > > > Yes, we always restart it automatically if these processes are exit > > (no matter because of OOM or some other reason). > > It is safe to do that if OOM happens, because OOM is always because of > > anon pages leaked and the restart can free these anon pages. > > No this is an incorrect assumption. The OOM might happen for many > different reasons. > > > But there may be some cases that we can't success to restart it, while > > if that happens the protected pages will be never be reclaimed until > > the admin reset it or make this memcg offline. > > If the workload cannot be restarted for whatever reason then you need an > admin intervention and a proper cleanup. That would include resetting > reclaim protection when in use. > > > When there're no processes, we don't need to protect the pages. You > > can consider it as 'fault tolerance' . > > I have already tried to explain why this is a bold statement that > doesn't really hold universally and that the kernel doesn't really have > enough information to make an educated guess. > I didn't mean we must relcaim the protected pages in all cases, while I mean sometimes we should relcaim the protected pages. If the kernel can't make an educated guess, we can tell the kernel to do it, for example, to introduce a new controller file to tell the kernel whehter or not relcaim the protected pages if there're no proceses running. > > > In other words why do you care about the oom killer case so much? It is > > > not different that handling a lingering memcg with the workload already > > > finished. You simply have no way to know whether the reclaim protection > > > is still required. Admin is supposed to either offline the memcg that is > > > no longer used or drop the reclaim protection once it is not needed > > > because that has some visible consequences on the overall system > > > operation. > > > > Actually what I concern is the case that there's no process running > > but memory protection coninues protecting the file pages. > > OOM is just one case of them. > > This sounds like a misconfiguration which should be handled by an admin. That may be a misconfiguration, but the kernel can do something before the admin notice it. Thanks Yafang