From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D5B6C4BA24 for ; Wed, 26 Feb 2020 20:25:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 229EA24670 for ; Wed, 26 Feb 2020 20:25:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YuXdKjW0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 229EA24670 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 713406B0003; Wed, 26 Feb 2020 15:25:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C3756B0005; Wed, 26 Feb 2020 15:25:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B1926B0006; Wed, 26 Feb 2020 15:25:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0171.hostedemail.com [216.40.44.171]) by kanga.kvack.org (Postfix) with ESMTP id 3F4C36B0003 for ; Wed, 26 Feb 2020 15:25:47 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id F3D628245571 for ; Wed, 26 Feb 2020 20:25:46 +0000 (UTC) X-FDA: 76533409092.11.beef28_6ca2a6bfcb154 X-HE-Tag: beef28_6ca2a6bfcb154 X-Filterd-Recvd-Size: 5359 Received: from mail-ot1-f65.google.com (mail-ot1-f65.google.com [209.85.210.65]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Wed, 26 Feb 2020 20:25:46 +0000 (UTC) Received: by mail-ot1-f65.google.com with SMTP id g96so666039otb.13 for ; Wed, 26 Feb 2020 12:25:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KwWaD9UNYEDZjJ4EPfYXtQOHPHFo1HH/Frc1ArcblIg=; b=YuXdKjW0oghU8BUd9VZISKdnbYPb7Iij9fXMHDfJe6Jc2da/+3gxW0hpZVtNmG9HsD EgJ+RO+EF51ANqkErW6DkmniTVm1OuwWvXCS/DeUCFFoC7Vg6l0+rsecJFI2MCsws2YZ j4Ug5p4cr2T4EyGr7MQJ58Uc1nWUvJ0ZHX6WNhJ/sYLo7ClXiBwKQFJKsYPpYPRZi/LX pcEkyCMRg5juMyH3rQD2+4+jjLvkP2F0mKSQ2RVAxPF4/BaYnCqS+LfpnBHSfSbsCrW2 ZSHA29PXfqFjNpsuNhDYO25EiDZCs+AABYp7G36xqJMvHoBYN0RH5TR4yiHd6KoaXuk8 dJKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KwWaD9UNYEDZjJ4EPfYXtQOHPHFo1HH/Frc1ArcblIg=; b=UDqEeReaxYOd3Bu+ZgQ5cT15roP2Y5/HFEq/Qq5IdAYUpt8HpjQkR7XinYJ6+xBkO+ z0+dHJfkTy/n9nCq0TkVG+GHIWnfCW3oLcLXwsLjagSvkxkD3P2pkYvBjTPav4DrFWWU vzPS0Yd/FwZeiJoihkI6lSoldHJRxSHUW8bUIraQMEXCCkajQc/cZQT6Ngjl/bjg5wuJ V+mhXq9wFXpNdXAF2pCJCaVin3x2Qiutnf4ulNpc6kRm34AlJdFV2175MA7/joGnckuB K/OY4Omo8Oi3t/5Y9eQKKiZI/I+uDRVdDI5cEQTXW6qZb9Gp+O3lxW8xWpmHN6EJOZkt CMjA== X-Gm-Message-State: APjAAAUkwEMWDIqL6CG/5POH5huHwhesiBaJYviujVgDRqNVLvZJG2Lk WFA+AvmJeQIXGoDj9SNMRvG1kz2RuZsS6h4XBj+7kQ7H X-Google-Smtp-Source: APXvYqzHU2SxaDMnNTVfBJNIjx/bUh8/7WP9i2EqHRNqU74ZzhmkAOB+zj/HsL3Z5tGRsMjcKCesJovREGO3/obZoLM= X-Received: by 2002:a05:6830:1e2b:: with SMTP id t11mr449644otr.81.1582748745385; Wed, 26 Feb 2020 12:25:45 -0800 (PST) MIME-Version: 1.0 References: <20200219181219.54356-1-hannes@cmpxchg.org> In-Reply-To: <20200219181219.54356-1-hannes@cmpxchg.org> From: Shakeel Butt Date: Wed, 26 Feb 2020 12:25:33 -0800 Message-ID: Subject: Re: [PATCH] mm: memcontrol: asynchronous reclaim for memory.high To: Johannes Weiner , Yang Shi Cc: Andrew Morton , Michal Hocko , Tejun Heo , Roman Gushchin , Linux MM , Cgroups , LKML , Kernel Team Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Feb 19, 2020 at 10:12 AM Johannes Weiner wrote: > > We have received regression reports from users whose workloads moved > into containers and subsequently encountered new latencies. For some > users these were a nuisance, but for some it meant missing their SLA > response times. We tracked those delays down to cgroup limits, which > inject direct reclaim stalls into the workload where previously all > reclaim was handled my kswapd. > > This patch adds asynchronous reclaim to the memory.high cgroup limit > while keeping direct reclaim as a fallback. In our testing, this > eliminated all direct reclaim from the affected workload. > > memory.high has a grace buffer of about 4% between when it becomes > exceeded and when allocating threads get throttled. We can use the > same buffer for the async reclaimer to operate in. If the worker > cannot keep up and the grace buffer is exceeded, allocating threads > will fall back to direct reclaim before getting throttled. > > For irq-context, there's already async memory.high enforcement. Re-use > that work item for all allocating contexts, but switch it to the > unbound workqueue so reclaim work doesn't compete with the workload. > The work item is per cgroup, which means the workqueue infrastructure > will create at maximum one worker thread per reclaiming cgroup. > > Signed-off-by: Johannes Weiner > --- > mm/memcontrol.c | 60 +++++++++++++++++++++++++++++++++++++------------ > mm/vmscan.c | 10 +++++++-- This reminds me of the per-memcg kswapd proposal from LSFMM 2018 (https://lwn.net/Articles/753162/). If I understand this correctly, the use-case is that the job instead of direct reclaiming (potentially in latency sensitive tasks), prefers a background non-latency sensitive task to do the reclaim. I am wondering if we can use the memory.high notification along with a new memcg interface (like memory.try_to_free_pages) to implement a user space background reclaimer. That would resolve the cpu accounting concerns as the user space background reclaimer can share the cpu cost with the task. One concern with this approach will be that the memory.high notification is too late and the latency sensitive task has faced the stall. We can either introduce a threshold notification or another notification only limit like memory.near_high which can be set based on the job's rate of allocations and when the usage hits this limit just notify the user space. Shakeel