From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC24EC35641 for ; Fri, 21 Feb 2020 08:42:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9919820722 for ; Fri, 21 Feb 2020 08:42:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9919820722 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5D2266B0003; Fri, 21 Feb 2020 03:42:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5827F6B0006; Fri, 21 Feb 2020 03:42:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 471CA6B0007; Fri, 21 Feb 2020 03:42:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0140.hostedemail.com [216.40.44.140]) by kanga.kvack.org (Postfix) with ESMTP id 2FDB16B0003 for ; Fri, 21 Feb 2020 03:42:45 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B6FE982499A8 for ; Fri, 21 Feb 2020 08:42:44 +0000 (UTC) X-FDA: 76513493448.16.kick09_521e4b7a84f51 X-HE-Tag: kick09_521e4b7a84f51 X-Filterd-Recvd-Size: 4886 Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Fri, 21 Feb 2020 08:42:44 +0000 (UTC) Received: by mail-wr1-f50.google.com with SMTP id k11so986640wrd.9 for ; Fri, 21 Feb 2020 00:42:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=jEG8RcYEmNm9awljme1S6bDG/vOsz59earfni9IR4J0=; b=KAB8A5+/pWB8QmzDa8gMtEHSX4E2gu8jPvOl4OLp4Ft/ZaognAZPZYb19CCxLukHdK P5xUTIhFR9Pv5mJHp1dekr2VM4L7i8QxHSIeUUMlETj8dO6nIEhc2I1rdlq+j6CEC4gv ZXyy0xHjzzEh/Dox+aqhMXgDXa1c1QB1Bau79omqy4C1JVZnixiVgoe5P9tOYCUxX2iZ 4cfvLbuAEm+pOSVs1FSBqS7DJVLas1GlHjQkZvQu953KkUpmzKrbGL9s8qtkxQTI5EnG WQdqjuMoF9TLLU9XymL09affSgBbT3dm2Rx8vtJTGZNRhXrlHS5CdC+EBxrOfddYEppP iCcg== X-Gm-Message-State: APjAAAX/0NTwYyeZvqfxc7dYjeuezUoPxnrRS9Sj8u+8/qU2BUjE109q fdsJQfvo0GVqg2f77wMobnM= X-Google-Smtp-Source: APXvYqygDGZ24SxwPFrKGP/l7QvGS9ht5y1uEggEi/ufVpQdgvw2ZL+Lvn82/kKR6SkhNhUHlKPBWg== X-Received: by 2002:a5d:6404:: with SMTP id z4mr20393368wru.262.1582274563234; Fri, 21 Feb 2020 00:42:43 -0800 (PST) Received: from localhost (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id d9sm3041397wrx.94.2020.02.21.00.42.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Feb 2020 00:42:42 -0800 (PST) Date: Fri, 21 Feb 2020 09:42:41 +0100 From: Michal Hocko To: Tim Chen Cc: "Kirill A. Shutemov" , lsf-pc@lists.linux-foundation.org, linux-mm@kvack.org, Dave Hansen , Dan Williams , Huang Ying Subject: Re: [Lsf-pc] [LSF/MM TOPIC] Memory cgroups, whether you like it or not Message-ID: <20200221084241.GL20509@dhcp22.suse.cz> References: <20200214104541.GT31689@dhcp22.suse.cz> <20200220160604.ak33n3hrqouyiuyv@box> <20200220161915.GI20509@dhcp22.suse.cz> <36e127fe-814e-9b22-cb7d-73a2e3daf2aa@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <36e127fe-814e-9b22-cb7d-73a2e3daf2aa@linux.intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu 20-02-20 14:16:02, Tim Chen wrote: > On 2/20/20 8:19 AM, Michal Hocko wrote: > > >> > >> Michal, could you remind what the deal with soft limit? Why is it dead? > > > > because of the very disruptive semantic. Essentially the way how it was > > grafted into the normal reclaim. It is essentially a priority 0 reclaim > > round to shrink a hierarchy which is the most in excess before we do a > > normal reclaim. This can lead to an over reclaim, long stalls etc. > > Thanks for the explanation. I wonder if a few factors could mitigate the > stalls in the tiered memory context: > > 1. The speed of demotion between top tier memory and second tier memory > is much faster than reclaiming the pages and swapping them out. You could have accumulated a lot of soft limit excess before it is reclaimed. So I do not think the speed of the demotion is the primary factor. > 2. Demotion targets pages that are colder and less active. > > 3. If we engage the page demotion mostly in the background, say via kswapd, > and not in the direct reclaim path, we can avoid long stalls > during page allocation. If the memory pressure is severe > on the top tier memory, perhaps the memory could be allocated from the second > tier memory node to avoid stalling. > > The stalls could still prove to be problematic. We're implementing > prototypes and we'll have a better ideas on workload latencies once we can collect data. I would really encourage you to not hook into the soft limit reclaim even if you somehow manage to reduce the problem with stalls for at least three reasons 1) soft limit is not going to be added to cgroup v2 because there is a different API to achieve a pro-active reclaim 2) soft limit is not aware of the memory you are reclaiming so using it for tiered memory sounds like a bad fit to me. 3) changing the semantic of the existing interface is always troublesome. Please have to look into mailing list archives when we have attempted that the last time. Have a look at e.g. http://lkml.kernel.org/r/1371557387-22434-1-git-send-email-mhocko@suse.cz Anyway it is hard to comment on without knowing details on how you actually want to use soft limit for different memory types and their balancing. -- Michal Hocko SUSE Labs