From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1129C433EF for ; Tue, 15 Feb 2022 18:50:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E66C6B0078; Tue, 15 Feb 2022 13:50:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 36F216B007B; Tue, 15 Feb 2022 13:50:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1EA026B007D; Tue, 15 Feb 2022 13:50:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0143.hostedemail.com [216.40.44.143]) by kanga.kvack.org (Postfix) with ESMTP id 0C4B36B0078 for ; Tue, 15 Feb 2022 13:50:17 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C5DCD181AC9C6 for ; Tue, 15 Feb 2022 18:50:16 +0000 (UTC) X-FDA: 79145904432.14.4AB1425 Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) by imf30.hostedemail.com (Postfix) with ESMTP id 6359C8000F for ; Tue, 15 Feb 2022 18:50:16 +0000 (UTC) Received: by mail-lf1-f41.google.com with SMTP id o2so38729467lfd.1 for ; Tue, 15 Feb 2022 10:50:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+Xnr8BvsNvJcUeAT+XTUM3XKtkq81lQw2/bkBVhK17c=; b=Sfe/kuKEpWEWZjVJDL2psXjG9oUfvqPlmdtCHVjzjva5Wf4hgOym+ssmVRUnZneaA2 rEkXJ6PqgGUbzerluit/C1Y3RqEu1S5bYN/tNFSYc4A5kPeu9G3E82vvyMHH6f16rRsY y+OdypvtdIdQcjShE2Ydr+tNu8kR1mF0qx5XOsBDCXMb09E2+SV+WVbERkryp01EqPei 0+5L0KO/aEa5PsYc4A+s/3lgkQDYrL7LcIUbxKDZxMJD+qr4LQbpfxisHlC99/g6Rd7E qvdfK8AZjD7qDatwPsLGEWXQtAWkFS+YwYoq/b26HhsMtTgjy6v3qudX31TKl19Ek6Fi fetw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+Xnr8BvsNvJcUeAT+XTUM3XKtkq81lQw2/bkBVhK17c=; b=JjMXSvJrcmVnGLpItvW9pr7qgfPAjEQc2xMn9DHBKYjwrcOnjfEQrfuNoG8R6Dw/gx BfrhatuxMGt75cRUFJqB/kJZ/24bfJ6RVsuHPczdTPi2ZcgbjG1JldmXNLNlJFsvkBUM j5eG/C4Mp6EMgilpmdsnG5B5C4Us6WxA6xnSNbuwgt48HHS83IA1VSlk8RvpJmcfVk35 ik589y2ddS8l6UaRLWj8FQMM5ulslwHCwrK1nSM3ksaGI5OXEl2nFZP6+KiuYcPRVFJF tHNeQrGAtGT4KhmQtQFbyIBDDg5j+m8P8Lq4vfYD/eha4oYL2CJAMz5hzesBCeC33o7i 7foA== X-Gm-Message-State: AOAM5301ao0/JJPS36ap4IfExc47VD5fXiKhK07WYUIRrPziTZfv5vfn ncfXhR/IHygnuBjLrLeUkj0bqDdJERtvduxiftQaag== X-Google-Smtp-Source: ABdhPJyIu9dxDVnqJmrU42bbi/YznVPRyTi+a5MPMSB6Gw1A7chbBCL0ab5yyXR171n2U9x2sWStDOrNza+8Sdgt2IU= X-Received: by 2002:a05:6512:10ce:: with SMTP id k14mr328361lfg.210.1644951014338; Tue, 15 Feb 2022 10:50:14 -0800 (PST) MIME-Version: 1.0 References: <20220211064917.2028469-1-shakeelb@google.com> <20220211064917.2028469-5-shakeelb@google.com> In-Reply-To: <20220211064917.2028469-5-shakeelb@google.com> From: Shakeel Butt Date: Tue, 15 Feb 2022 10:50:03 -0800 Message-ID: Subject: Re: [PATCH v2 4/4] memcg: synchronously enforce memory.high for large overcharges To: Johannes Weiner , Michal Hocko , Roman Gushchin Cc: Chris Down , Andrew Morton , Cgroups , Linux MM , LKML Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 6359C8000F X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="Sfe/kuKE"; spf=pass (imf30.hostedemail.com: domain of shakeelb@google.com designates 209.85.167.41 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: 5o4t3rxciurjkzquo3e78thw3xmw9g9r X-Rspamd-Server: rspam03 X-HE-Tag: 1644951016-510077 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Feb 10, 2022 at 10:49 PM Shakeel Butt wrote: > > The high limit is used to throttle the workload without invoking the > oom-killer. Recently we tried to use the high limit to right size our > internal workloads. More specifically dynamically adjusting the limits > of the workload without letting the workload get oom-killed. However due > to the limitation of the implementation of high limit enforcement, we > observed the mechanism fails for some real workloads. > > The high limit is enforced on return-to-userspace i.e. the kernel let > the usage goes over the limit and when the execution returns to > userspace, the high reclaim is triggered and the process can get > throttled as well. However this mechanism fails for workloads which do > large allocations in a single kernel entry e.g. applications that > mlock() a large chunk of memory in a single syscall. Such applications > bypass the high limit and can trigger the oom-killer. > > To make high limit enforcement more robust, this patch makes the limit > enforcement synchronous only if the accumulated overcharge becomes > larger than MEMCG_CHARGE_BATCH. So, most of the allocations would still > be throttled on the return-to-userspace path but only the extreme > allocations which accumulates large amount of overcharge without > returning to the userspace will be throttled synchronously. The value > MEMCG_CHARGE_BATCH is a bit arbitrary but most of other places in the > memcg codebase uses this constant therefore for now uses the same one. > > Signed-off-by: Shakeel Butt Any comments or concerns on this patch? Otherwise I would ask Andrew to add this series into the mm tree. > --- > Changes since v1: > - Based on Roman's comment simply the sync enforcement and only target > the extreme cases. > > mm/memcontrol.c | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 292b0b99a2c7..0da4be4798e7 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -2703,6 +2703,11 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, > } > } while ((memcg = parent_mem_cgroup(memcg))); > > + if (current->memcg_nr_pages_over_high > MEMCG_CHARGE_BATCH && > + !(current->flags & PF_MEMALLOC) && > + gfpflags_allow_blocking(gfp_mask)) { > + mem_cgroup_handle_over_high(); > + } > return 0; > } > > -- > 2.35.1.265.g69c8d7142f-goog >