From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FB03C433DF for ; Mon, 17 Aug 2020 17:22:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6E22920716 for ; Mon, 17 Aug 2020 17:22:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="c+qkjdRR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389845AbgHQRWq (ORCPT ); Mon, 17 Aug 2020 13:22:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731336AbgHQQpQ (ORCPT ); Mon, 17 Aug 2020 12:45:16 -0400 Received: from mail-lj1-x243.google.com (mail-lj1-x243.google.com [IPv6:2a00:1450:4864:20::243]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26039C061389 for ; Mon, 17 Aug 2020 09:44:55 -0700 (PDT) Received: by mail-lj1-x243.google.com with SMTP id i10so18230929ljn.2 for ; Mon, 17 Aug 2020 09:44:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Ustc/Co8E0AQhbcerO/qjyS5RAvKGH1AciZ7eOvqsmA=; b=c+qkjdRRVAF0xvWA9+DkkVLwK7sGF1z8hQM3CtGa1YD+bJI6w1nj5+9S70Q8FvCQqi 2Ei0s13pec+cTVm+0OSOFIdw7vP7XLcSIvUcZ6EGrALSHKiZ7PPGzM4XEb029aajiYyC aEhBUOpbzPBTQSG4Cdn7X1BjMfLQqn745I4HqH3Abk31kJgYxjgeUpKk7qwB2tzxmqpe AtkUyANm5zAbjaLbrp8RlEB5I2UccCxi0HQae47ThK7uHUPNGSHd8fX+Y3ltxcsiL9Kl 3ENiFhFOnYGf8LXV3bRXJi1SUgcycpC5v7MIYDrXtB34VmBj+Nk2YpIHRR25BtbB6i8Q lRBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Ustc/Co8E0AQhbcerO/qjyS5RAvKGH1AciZ7eOvqsmA=; b=Tjy2wJVBWRVJ7CoGotjd+vZdHX/7tTgbZunxdX//o2eqxrjRU6desMyaMIhHl7v/Mm 0uV0MSw26g/mBjz9vI0XTUfdcCLWq5hJYZBDzz2buaNWeKmKmHwPjSvycp0nUJ7Ud4/A aB61jt6NIKEKhB1hVwWxpxNdP4hO5ZpevZMBZGxwiRG7K7KyAKS9WcBy0QfwMTxrez/X RZZQDZa3O11rcfGd+GKqWJdDO/Ljr+AbRwmX0WFO4WJy6JZ5wEzsIoHoHM2YT643AugI 6FYy29y3Hr4/DWyMv3zjnLIrZsucy2hBZclxM33j6AwiSFN2MqtRDFrHh6NmII3CG1w6 1O9A== X-Gm-Message-State: AOAM530gNf88kXBV4zdJXlsTjk6KUIQnMwm/bO4rzfhRSxmS8dE1hpjD V+x2sWDSFbzvFy0Ls1d4Ag2eJ/bDuEmAV5Fzx2cHTg== X-Google-Smtp-Source: ABdhPJxOE/KUm/g/wNOBiL7kHkyu0ap1d61VlZFxn7CVZ1FQIRsBFPdNAO+qmD/8YZJURfdVWZ4NywWAdRo3uSnMkE4= X-Received: by 2002:a2e:96c3:: with SMTP id d3mr7974441ljj.270.1597682693369; Mon, 17 Aug 2020 09:44:53 -0700 (PDT) MIME-Version: 1.0 References: <20200817140831.30260-1-longman@redhat.com> <20200817140831.30260-2-longman@redhat.com> In-Reply-To: <20200817140831.30260-2-longman@redhat.com> From: Shakeel Butt Date: Mon, 17 Aug 2020 09:44:42 -0700 Message-ID: Subject: Re: [RFC PATCH 1/8] memcg: Enable fine-grained control of over memory.high action To: Waiman Long Cc: Andrew Morton , Johannes Weiner , Michal Hocko , Vladimir Davydov , Jonathan Corbet , Alexey Dobriyan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , LKML , linux-doc@vger.kernel.org, linux-fsdevel , Cgroups , Linux MM Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Aug 17, 2020 at 7:11 AM Waiman Long wrote: > > Memory controller can be used to control and limit the amount of > physical memory used by a task. When a limit is set in "memory.high" > in a non-root memory cgroup, the memory controller will try to reclaim > memory if the limit has been exceeded. Normally, that will be enough > to keep the physical memory consumption of tasks in the memory cgroup > to be around or below the "memory.high" limit. > > Sometimes, memory reclaim may not be able to recover memory in a rate > that can catch up to the physical memory allocation rate especially > when rotating disks are used for swapping or writing dirty pages. In > this case, the physical memory consumption will keep on increasing. Isn't this the real underlying issue? Why not make the guarantees of memory.high more strict instead of adding more interfaces and complexity? By the way, have you observed this issue on real workloads or some test cases? It would be good to get a repro with simple test cases. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Shakeel Butt Subject: Re: [RFC PATCH 1/8] memcg: Enable fine-grained control of over memory.high action Date: Mon, 17 Aug 2020 09:44:42 -0700 Message-ID: References: <20200817140831.30260-1-longman@redhat.com> <20200817140831.30260-2-longman@redhat.com> Mime-Version: 1.0 Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Ustc/Co8E0AQhbcerO/qjyS5RAvKGH1AciZ7eOvqsmA=; b=c+qkjdRRVAF0xvWA9+DkkVLwK7sGF1z8hQM3CtGa1YD+bJI6w1nj5+9S70Q8FvCQqi 2Ei0s13pec+cTVm+0OSOFIdw7vP7XLcSIvUcZ6EGrALSHKiZ7PPGzM4XEb029aajiYyC aEhBUOpbzPBTQSG4Cdn7X1BjMfLQqn745I4HqH3Abk31kJgYxjgeUpKk7qwB2tzxmqpe AtkUyANm5zAbjaLbrp8RlEB5I2UccCxi0HQae47ThK7uHUPNGSHd8fX+Y3ltxcsiL9Kl 3ENiFhFOnYGf8LXV3bRXJi1SUgcycpC5v7MIYDrXtB34VmBj+Nk2YpIHRR25BtbB6i8Q lRBw== In-Reply-To: <20200817140831.30260-2-longman@redhat.com> Sender: linux-doc-owner@vger.kernel.org List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Waiman Long Cc: Andrew Morton , Johannes Weiner , Michal Hocko , Vladimir Davydov , Jonathan Corbet , Alexey Dobriyan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , LKML , linux-doc@vger.kernel.org, linux-fsdevel , Cgroups , Linux MM On Mon, Aug 17, 2020 at 7:11 AM Waiman Long wrote: > > Memory controller can be used to control and limit the amount of > physical memory used by a task. When a limit is set in "memory.high" > in a non-root memory cgroup, the memory controller will try to reclaim > memory if the limit has been exceeded. Normally, that will be enough > to keep the physical memory consumption of tasks in the memory cgroup > to be around or below the "memory.high" limit. > > Sometimes, memory reclaim may not be able to recover memory in a rate > that can catch up to the physical memory allocation rate especially > when rotating disks are used for swapping or writing dirty pages. In > this case, the physical memory consumption will keep on increasing. Isn't this the real underlying issue? Why not make the guarantees of memory.high more strict instead of adding more interfaces and complexity? By the way, have you observed this issue on real workloads or some test cases? It would be good to get a repro with simple test cases.