From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C762BC4708F for ; Wed, 2 Jun 2021 15:39:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 747F561263 for ; Wed, 2 Jun 2021 15:39:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 747F561263 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E61A76B006C; Wed, 2 Jun 2021 11:39:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E38946B006E; Wed, 2 Jun 2021 11:39:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB2916B0070; Wed, 2 Jun 2021 11:39:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0058.hostedemail.com [216.40.44.58]) by kanga.kvack.org (Postfix) with ESMTP id 98E8A6B006C for ; Wed, 2 Jun 2021 11:39:15 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 374B1180AD804 for ; Wed, 2 Jun 2021 15:39:15 +0000 (UTC) X-FDA: 78209192670.20.F5978DA Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by imf06.hostedemail.com (Postfix) with ESMTP id F2019C00CBE4 for ; Wed, 2 Jun 2021 15:39:01 +0000 (UTC) Received: by mail-lf1-f42.google.com with SMTP id v22so2720964lfa.3 for ; Wed, 02 Jun 2021 08:39:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=674c29vRdYAk6FicWWelzpFqgoTb+93XHQAYD/1H3Rk=; b=bMiDXVuqX3MZ1ebHXMXAnz4i1FuII1AA/TtaUU/88MAika1cxyLKtGEw4MIsREXO+X CmZ71Pfe+P+ouAxW5F5HeR1iDagoaN3GJttsjpbjfSz1LSg+IuI4e5UNdn4UjORrl6K+ XaPqXvgdw8TLFvyMzqZtVFtQS5SFrFKGudfhr+uc9hUomLbUqgFFrFiu3w55ZZk1jP39 hr35swpMuqa9ALpJR7bmDsxEZlhrUqP02UN7iDrYy5z5MKkj0qrIrk3lCYyl/p4apSad EUvuVDgHsnEQqx5zQ1pjEdTpkGHpon7Q8wHvorS7CxXoxpEp+I2oZctN6bdA2OPFk2si DWRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=674c29vRdYAk6FicWWelzpFqgoTb+93XHQAYD/1H3Rk=; b=LfC8XZwaTSgwE5pmHYhcw22eFbSywoc5oHmeBt9leBmxt4lN34XTS5vJL6DH1sYJWJ NlzVQqhfaUx48YQoySJ0wmUHbhDeG2QTM1t5Cs9gMutzk8dn0V/GCVqhHNLfLeQ1vLfE fF8oocHsDqoPydmKLcOvseiIDwrb6cgSe1fFM+kTU5fOVx9EjbH8eDO37t+VaJ+i3s97 d6u7NinR0bl1y8u22TiQPndh58q1GKRrcpvsjvbnDuhCX27cPnllE+JMGnF8UCD8i/JS wpUTQq6AlhUdWzU56L7/R/d2ZL2KYHwRDL95GNXd2n+qCG0PdJxI3w/br1dg/aMmOMfJ 1cBw== X-Gm-Message-State: AOAM533Nif6q/tU6T6TapFiv0fIeJrBmHtlGVhlMzlW69ddyUcbMdCmt sYZEBn3x0TzRzhpZ8Z58FVO3hlmG/2nwPVMjlTU6NA== X-Google-Smtp-Source: ABdhPJzUw8dMj8EA8JBuJZCLP1LL+HR9H3i2lXbH4Cv+UfaMzKjX96HEn5EaYGeRejpEEFOwMIB0RqQcUBfhNXWuomM= X-Received: by 2002:a05:6512:531:: with SMTP id o17mr10519925lfc.358.1622648353129; Wed, 02 Jun 2021 08:39:13 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Shakeel Butt Date: Wed, 2 Jun 2021 08:39:02 -0700 Message-ID: Subject: Re: [RFC 0/7] Introduce memory allocation speed throttle in memcg To: yulei zhang Cc: Chris Down , Tejun Heo , Zefan Li , Johannes Weiner , Christian Brauner , Cgroups , benbjiang@tencent.com, Wanpeng Li , Yulei Zhang , Linux MM , Michal Hocko , Roman Gushchin Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: F2019C00CBE4 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20161025 header.b=bMiDXVuq; spf=pass (imf06.hostedemail.com: domain of shakeelb@google.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam04 X-Stat-Signature: 7as3y1cawzxpqexjrirmf9j6quwtp9w9 X-HE-Tag: 1622648341-51065 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jun 2, 2021 at 2:11 AM yulei zhang wrote: > > On Tue, Jun 1, 2021 at 10:45 PM Chris Down wrote: > > > > yulei zhang writes: > > >Yep, dynamically adjust the memory.high limits can ease the memory pressure > > >and postpone the global reclaim, but it can easily trigger the oom in > > >the cgroups, > > > > To go further on Shakeel's point, which I agree with, memory.high should > > _never_ result in memcg OOM. Even if the limit is breached dramatically, we > > don't OOM the cgroup. If you have a demonstration of memory.high resulting in > > cgroup-level OOM kills in recent kernels, then that needs to be provided. :-) > > You are right, I mistook it for max. Shakeel means the throttling > during context switch > which uses memory.high as threshold to calculate the sleep time. > Currently it only applies > to cgroupv2. In this patchset we explore another idea to throttle the > memory usage, which > rely on setting an average allocation speed in memcg. We hope to > suppress the memory > usage in low priority cgroups when it reaches the system watermark and > still keep the activities > alive. I think you need to make the case: why should we add one more form of throttling? Basically why memory.high is not good for your use-case and the proposed solution works better. Though IMO it would be a hard sell. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Shakeel Butt Subject: Re: [RFC 0/7] Introduce memory allocation speed throttle in memcg Date: Wed, 2 Jun 2021 08:39:02 -0700 Message-ID: References: Mime-Version: 1.0 Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=674c29vRdYAk6FicWWelzpFqgoTb+93XHQAYD/1H3Rk=; b=bMiDXVuqX3MZ1ebHXMXAnz4i1FuII1AA/TtaUU/88MAika1cxyLKtGEw4MIsREXO+X CmZ71Pfe+P+ouAxW5F5HeR1iDagoaN3GJttsjpbjfSz1LSg+IuI4e5UNdn4UjORrl6K+ XaPqXvgdw8TLFvyMzqZtVFtQS5SFrFKGudfhr+uc9hUomLbUqgFFrFiu3w55ZZk1jP39 hr35swpMuqa9ALpJR7bmDsxEZlhrUqP02UN7iDrYy5z5MKkj0qrIrk3lCYyl/p4apSad EUvuVDgHsnEQqx5zQ1pjEdTpkGHpon7Q8wHvorS7CxXoxpEp+I2oZctN6bdA2OPFk2si DWRQ== In-Reply-To: List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: yulei zhang Cc: Chris Down , Tejun Heo , Zefan Li , Johannes Weiner , Christian Brauner , Cgroups , benbjiang-1Nz4purKYjRBDgjK7y7TUQ@public.gmane.org, Wanpeng Li , Yulei Zhang , Linux MM , Michal Hocko , Roman Gushchin On Wed, Jun 2, 2021 at 2:11 AM yulei zhang wrote: > > On Tue, Jun 1, 2021 at 10:45 PM Chris Down wrote: > > > > yulei zhang writes: > > >Yep, dynamically adjust the memory.high limits can ease the memory pressure > > >and postpone the global reclaim, but it can easily trigger the oom in > > >the cgroups, > > > > To go further on Shakeel's point, which I agree with, memory.high should > > _never_ result in memcg OOM. Even if the limit is breached dramatically, we > > don't OOM the cgroup. If you have a demonstration of memory.high resulting in > > cgroup-level OOM kills in recent kernels, then that needs to be provided. :-) > > You are right, I mistook it for max. Shakeel means the throttling > during context switch > which uses memory.high as threshold to calculate the sleep time. > Currently it only applies > to cgroupv2. In this patchset we explore another idea to throttle the > memory usage, which > rely on setting an average allocation speed in memcg. We hope to > suppress the memory > usage in low priority cgroups when it reaches the system watermark and > still keep the activities > alive. I think you need to make the case: why should we add one more form of throttling? Basically why memory.high is not good for your use-case and the proposed solution works better. Though IMO it would be a hard sell.