From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA163C77B7A for ; Fri, 2 Jun 2023 00:10:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232476AbjFBAKg (ORCPT ); Thu, 1 Jun 2023 20:10:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231942AbjFBAKe (ORCPT ); Thu, 1 Jun 2023 20:10:34 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C714218C for ; Thu, 1 Jun 2023 17:09:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685664589; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IQBTL4KWFtPVFbs25Ot9x0LJbTVOUU336wX1+2B8Rjk=; b=KRpQ4p0lsShjgPrAistGE/q1NYddVokZjzPFvH4uYqcHDmRVzEnRJo780JG87vyGphuLt7 i258NdJqtQKHW3iIoWOt6OJj53De3Y/lciw+ZK4Yr2tXHpj7gvN18a4LKkupNqo7vBa7Em Od1Zak1Irl8wtzAl6tsLUdpGWoO8u70= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-569-bkhrC-83MXW-CYPiVx5WNw-1; Thu, 01 Jun 2023 20:09:43 -0400 X-MC-Unique: bkhrC-83MXW-CYPiVx5WNw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 59A152932493; Fri, 2 Jun 2023 00:09:43 +0000 (UTC) Received: from [10.22.8.52] (unknown [10.22.8.52]) by smtp.corp.redhat.com (Postfix) with ESMTP id A1B0A40C6EC4; Fri, 2 Jun 2023 00:09:42 +0000 (UTC) Message-ID: Date: Thu, 1 Jun 2023 20:09:42 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: [PATCH] Documentation: Clarify usage of memory limits Content-Language: en-US To: Johannes Weiner Cc: Dan Schatzberg , Tejun Heo , Chris Down , Zefan Li , Jonathan Corbet , "open list:CONTROL GROUP (CGROUP)" , "open list:DOCUMENTATION" , open list References: <20230601183820.3839891-1-schatzberg.dan@gmail.com> <20230601195345.GB157732@cmpxchg.org> From: Waiman Long In-Reply-To: <20230601195345.GB157732@cmpxchg.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org On 6/1/23 15:53, Johannes Weiner wrote: > On Thu, Jun 01, 2023 at 03:15:28PM -0400, Waiman Long wrote: >> On 6/1/23 14:38, Dan Schatzberg wrote: >>> The existing documentation refers to memory.high as the "main mechanism >>> to control memory usage." This seems incorrect to me - memory.high can >>> result in reclaim pressure which simply leads to stalls unless some >>> external component observes and actions on it (e.g. systemd-oomd can be >>> used for this purpose). While this is feasible, users are unaware of >>> this interaction and are led to believe that memory.high alone is an >>> effective mechanism for limiting memory. >>> >>> The documentation should recommend the use of memory.max as the >>> effective way to enforce memory limits - it triggers reclaim and results >>> in OOM kills by itself. >> That is not how my understanding of memory.high works. When memory usage >> goes past memory.high, memory reclaim will be initiated to reclaim the >> memory back. Stall happens when memory.usage keep increasing like by >> consuming memory faster than what memory reclaim can recover. When >> memory.max is reached, OOM killer will then kill off the tasks. > This was the initial plan indeed: Slow down the workload and thus slow > the growth; hope that the workload recovers with voluntary frees; set > memory.max as a safety if it keeps going beyond. > > This never panned out. Once workloads are stuck, they might not back > down on their own. By increasingly slowing growth, it becomes harder > and harder for them to reach the memory.max intervention point. > > It's a very brittle configuration strategy. Unless you very carefully > calibrate memory.high and memory.max together with awareness of the > throttling algorithm, workloads that hit memory.high will just go to > sleep indefinitely. They require outside intervention that either > adjusts limits or implements kill policies based on observed sleeps > (they're reported as pressure via psi). > > So the common usecases today end up being that memory.max is for > enforcing kernel OOM kills, and memory.high is a tool to implement > userspace OOM killing policies. > > Dan is right to point out the additional expectations for userspace > management when memory.high is in used. And memory.max is still the > primary, works-out-of-the-box method of memory containment. Thanks for clarification. I have to reset my false assumption. Cheers, Longman