From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80621C433E0 for ; Thu, 21 May 2020 15:02:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 36D91207D8 for ; Thu, 21 May 2020 15:02:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="SPCXhfg+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 36D91207D8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BF20380008; Thu, 21 May 2020 11:02:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BA25080007; Thu, 21 May 2020 11:02:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ADF8480008; Thu, 21 May 2020 11:02:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0197.hostedemail.com [216.40.44.197]) by kanga.kvack.org (Postfix) with ESMTP id 96FD680007 for ; Thu, 21 May 2020 11:02:16 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5F31C180AD81F for ; Thu, 21 May 2020 15:02:16 +0000 (UTC) X-FDA: 76841041872.30.anger41_30e29d9de2f05 X-HE-Tag: anger41_30e29d9de2f05 X-Filterd-Recvd-Size: 4955 Received: from mail-ej1-f67.google.com (mail-ej1-f67.google.com [209.85.218.67]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 May 2020 15:02:15 +0000 (UTC) Received: by mail-ej1-f67.google.com with SMTP id e2so9133339eje.13 for ; Thu, 21 May 2020 08:02:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=L4UDchNoiuDX4tkUCL0dxdfAR6sFJ+v5DRcDTgeiOEA=; b=SPCXhfg+nCTAzglbOSxpGGWlT4LbcyT5ov7iVCRWlOjDg9vw3QGDFNbTOWFcM0RhsX 2AiHhCFhMdO1ch6CMvgSCrtXVOLUYItHW4m86skN7YWl+QEHjVJ3hnwozUyrpbPbXJg9 3AqRr6E5qlhw697AgVjHizpUyQ1BeM3NRaHuE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=L4UDchNoiuDX4tkUCL0dxdfAR6sFJ+v5DRcDTgeiOEA=; b=RcoQbokrZL8Nio2tP8+UHDAljDR6t5XV64aaWtrFzneiTpjcQtsD5UHP375y8s7AiF 1Z7If2akINWS/lwHqMqynjy3BVCYdlOfkEKiCgHn8N6HbGZQmWs/SXKoOUPypx3cY9mQ oUF/efMGonOSa+kulQaUX0oU9GdtJ2yzfB+BDs0H0uyGA/DX9HiP5fi22k3qGLvFxVX5 s0602STpgw1gzzidZv0YLXMyUl09ZMPEvea5Vh0t0Qcic+Vh6/DuGSKFdwnEMf8XHkwq mvBRJgqu942H+3tvfaY530n2P2qp6X4EDGNSq5LEnBbUDyCC60WXrEBM7YENSvpNIfSV vHzw== X-Gm-Message-State: AOAM530xEukCALHGrFBnw1bBrg4mQtwHmJbK6FXLzHPDuh6vMiUem42T +58ZO2QGQ3mJdH9pacG+H8lgIg== X-Google-Smtp-Source: ABdhPJz9snmtL4cGAURMbIiXEL18i41r6OCNFzdUZ/CRozM3v9OQUHLgtHyT9Dr6Nh1pnbBanYWl0w== X-Received: by 2002:a17:906:2e0e:: with SMTP id n14mr3803821eji.545.1590073334624; Thu, 21 May 2020 08:02:14 -0700 (PDT) Received: from localhost ([2620:10d:c093:400::5:4262]) by smtp.gmail.com with ESMTPSA id b3sm5198656ejq.52.2020.05.21.08.02.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 May 2020 08:02:13 -0700 (PDT) Date: Thu, 21 May 2020 16:02:13 +0100 From: Chris Down To: Michal Hocko Cc: Johannes Weiner , Andrew Morton , Tejun Heo , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: Re: [PATCH] mm, memcg: reclaim more aggressively before high allocator throttling Message-ID: <20200521150213.GH990580@chrisdown.name> References: <20200520143712.GA749486@chrisdown.name> <20200520160756.GE6462@dhcp22.suse.cz> <20200520165131.GB630613@cmpxchg.org> <20200520170430.GG6462@dhcp22.suse.cz> <20200520175135.GA793901@cmpxchg.org> <20200521073245.GI6462@dhcp22.suse.cz> <20200521135152.GA810429@cmpxchg.org> <20200521143515.GU6462@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <20200521143515.GU6462@dhcp22.suse.cz> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Michal Hocko writes: >> I have a good reason why we shouldn't: because it's special casing >> memory.high from other forms of reclaim, and that is a maintainability >> problem. We've recently been discussing ways to make the memory.high >> implementation stand out less, not make it stand out even more. There >> is no solid reason it should be different from memory.max reclaim, >> except that it should sleep instead of invoke OOM at the end. It's >> already a mess we're trying to get on top of and straighten out, and >> you're proposing to add more kinks that will make this work harder. > >I do see your point of course. But I do not give the code consistency >a higher priority than the potential unfairness aspect of the user >visible behavior for something that can do better. Really the direct >reclaim unfairness is really painfull and hard to explain to users. You >can essentially only hand wave that system is struggling so fairness is >not really a priority anymore. It's not handwaving. When using cgroup features, including memory.high, the unit for consideration is a cgroup, not a task. That we happen to act on individual tasks in this case is just an implementation detail. That one task in that cgroup is may be penalised "unfairly" is well within the specification: we set limits as part of a cgroup, we account as part of a cgroup, and we throttle and reclaim as part of a cgroup. We may make some very rudimentary attempts to "be fair" on a per-task basis where that's trivial, but that's just one-off niceties, not a statement of precedent. When exceeding memory.high, the contract is "this cgroup must immediately attempt to shrink". Breaking it down per-task in terms of fairness at that point doesn't make sense: all the tasks in one cgroup are in it together.