From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C9E0C4707F for ; Thu, 27 May 2021 09:34:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ADB36613EB for ; Thu, 27 May 2021 09:34:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ADB36613EB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 516FB6B0085; Thu, 27 May 2021 05:34:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4EDC16B0087; Thu, 27 May 2021 05:34:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 319986B0088; Thu, 27 May 2021 05:34:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id F21806B0085 for ; Thu, 27 May 2021 05:34:38 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 90C8E81D0 for ; Thu, 27 May 2021 09:34:38 +0000 (UTC) X-FDA: 78186501036.38.FDDEAA8 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by imf16.hostedemail.com (Postfix) with ESMTP id 217758019124 for ; Thu, 27 May 2021 09:34:31 +0000 (UTC) Received: by mail-pj1-f42.google.com with SMTP id v13-20020a17090abb8db029015f9f7d7290so5166721pjr.0 for ; Thu, 27 May 2021 02:34:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ib7RAPbfbKwKwT5M4f1jjNI8X1DqgVyFDhXu+pU2YDc=; b=mNp6Cb1YLHUxy6cw26u4CUhDgTDs0cRAiiyaSoga5m7N5h7Ak8Tvs4drwUtB2eKDsq BR9FRA2N93+3w3QtwDqRvBOhlA/Lqg+LKwC5f0G0cOuXPq0Yt3YbHFKirW8iCm17JCtN //J1JYMA+O73JMTbQnHwKJj8owPWVgoaxAcqFA1ApVDbozQypBp4QbQqBOjSQfgyUmvM PWXUQ6T9X4W7wNMcG4kcqG8CJa+iUad6qqbDeHTUQa3P+22iX6ujANNij0ErKLEwQTv8 W7ma61dnHJF6yj1QASuNvLz/9i/JF3VekHkTjnF8EdgEVdwoozlRRzvVnqnpYogdsH18 SkLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ib7RAPbfbKwKwT5M4f1jjNI8X1DqgVyFDhXu+pU2YDc=; b=aWg9uk9HnTK4egjB0Ou48/Ubc0M0ArpuQgzRhZnJP1Pns7mK/fJUeyKCFvloYU8RfS 6rV9vb0T39TtgRP7oWW95fX0bpztQZZCDKYT+ySbmCkx7B6cMzxrdqN/DKZRUc71Ff0h DO6LUF6O5UpEOyoEYwtyWrJN3gakaiqey3kkTy0uFDf4wlRNYFDbQVjz5AQKGlGel4Kj yfNeS+34jAMa3zUQ7cV9sARtvN+ZBDz5vTTgK4Esq8gi30A1ejVhXQd3JP1l5VWggU8J cQeeUQplBJtJF8SMR3TtG0lLiv54wO36RnVLbnSaVoOPJgcSvLDDda4WtubNOz3IHt5m Lypw== X-Gm-Message-State: AOAM531ZkXT8NBfcKTWz17fBVWKid6zky/SH5Gm2vuXhmu/H4t3sNEDm lXAvn2YTck1vyBkAwpHsyIycFA== X-Google-Smtp-Source: ABdhPJyVdHLzLicsxAPsLrq7ExyFFcuw1c5u2QVn6IBOaYMRrKOmc4rGueAku/4U/9ofUGoaghdX4A== X-Received: by 2002:a17:902:d30c:b029:fe:c349:14c1 with SMTP id b12-20020a170902d30cb02900fec34914c1mr1936785plc.81.1622108077583; Thu, 27 May 2021 02:34:37 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.254]) by smtp.gmail.com with ESMTPSA id a9sm1418917pfl.57.2021.05.27.02.34.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 27 May 2021 02:34:37 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, bsingharora@gmail.com, shy828301@gmail.com, alexs@kernel.org, smuchun@gmail.com, zhengqi.arch@bytedance.com, Muchun Song Subject: [RFC PATCH v4 08/12] mm: memcontrol: introduce memcg_reparent_ops Date: Thu, 27 May 2021 17:33:32 +0800 Message-Id: <20210527093336.14895-9-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210527093336.14895-1-songmuchun@bytedance.com> References: <20210527093336.14895-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 217758019124 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=bytedance-com.20150623.gappssmtp.com header.s=20150623 header.b=mNp6Cb1Y; spf=pass (imf16.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspamd-Server: rspam04 X-Stat-Signature: u9xmsshafiffsnna1oomghixccp9q6jc X-HE-Tag: 1622108071-951307 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In the previous patch, we know how to make the lruvec lock safe when the LRU pages reparented. We should do something like following. memcg_reparent_objcgs(memcg) 1) lock // lruvec belongs to memcg and lruvec_parent belongs to parent me= mcg. spin_lock(&lruvec->lru_lock); spin_lock(&lruvec_parent->lru_lock); 2) do reparent // Move all the pages from the lruvec list to the parent lruvec l= ist. 3) unlock spin_unlock(&lruvec_parent->lru_lock); spin_unlock(&lruvec->lru_lock); Apart from the page lruvec lock, the deferred split queue lock (THP only) also needs to do something similar. So we extract the necessary three ste= ps in the memcg_reparent_objcgs(). memcg_reparent_objcgs(memcg) 1) lock memcg_reparent_ops->lock(memcg, parent); 2) reparent memcg_reparent_ops->reparent(memcg, reparent); 3) unlock memcg_reparent_ops->unlock(memcg, reparent); Now there are two different locks (e.g. lruvec lock and deferred split queue lock) need to use this infrastructure. In the next patch, we will use those APIs to make those locks safe when the LRU pages reparented. Signed-off-by: Muchun Song --- include/linux/memcontrol.h | 7 +++++++ mm/memcontrol.c | 43 ++++++++++++++++++++++++++++++++++++++++= +-- 2 files changed, 48 insertions(+), 2 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 99f745e23607..336d80605763 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -354,6 +354,13 @@ struct mem_cgroup { struct mem_cgroup_per_node *nodeinfo[]; }; =20 +struct memcg_reparent_ops { + /* Irq is disabled before calling those callbacks. */ + void (*lock)(struct mem_cgroup *memcg, struct mem_cgroup *parent); + void (*unlock)(struct mem_cgroup *memcg, struct mem_cgroup *parent); + void (*reparent)(struct mem_cgroup *memcg, struct mem_cgroup *parent); +}; + /* * size of first charge trial. "32" comes from vmscan.c's magic value. * TODO: maybe necessary to use big numbers in big irons. diff --git a/mm/memcontrol.c b/mm/memcontrol.c index d294504aea0c..470cdf0fbff1 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -330,6 +330,35 @@ static struct obj_cgroup *obj_cgroup_alloc(void) return objcg; } =20 +static const struct memcg_reparent_ops *memcg_reparent_ops[] =3D {}; + +static void memcg_reparent_lock(struct mem_cgroup *memcg, + struct mem_cgroup *parent) +{ + int i; + + for (i =3D 0; i < ARRAY_SIZE(memcg_reparent_ops); i++) + memcg_reparent_ops[i]->lock(memcg, parent); +} + +static void memcg_reparent_unlock(struct mem_cgroup *memcg, + struct mem_cgroup *parent) +{ + int i; + + for (i =3D 0; i < ARRAY_SIZE(memcg_reparent_ops); i++) + memcg_reparent_ops[i]->unlock(memcg, parent); +} + +static void memcg_do_reparent(struct mem_cgroup *memcg, + struct mem_cgroup *parent) +{ + int i; + + for (i =3D 0; i < ARRAY_SIZE(memcg_reparent_ops); i++) + memcg_reparent_ops[i]->reparent(memcg, parent); +} + static void memcg_reparent_objcgs(struct mem_cgroup *memcg) { struct obj_cgroup *objcg, *iter; @@ -339,9 +368,13 @@ static void memcg_reparent_objcgs(struct mem_cgroup = *memcg) if (!parent) parent =3D root_mem_cgroup; =20 + local_irq_disable(); + + memcg_reparent_lock(memcg, parent); + objcg =3D rcu_replace_pointer(memcg->objcg, NULL, true); =20 - spin_lock_irq(&css_set_lock); + spin_lock(&css_set_lock); =20 /* 1) Ready to reparent active objcg. */ list_add(&objcg->list, &memcg->objcg_list); @@ -351,7 +384,13 @@ static void memcg_reparent_objcgs(struct mem_cgroup = *memcg) /* 3) Move already reparented objcgs to the parent's list */ list_splice(&memcg->objcg_list, &parent->objcg_list); =20 - spin_unlock_irq(&css_set_lock); + spin_unlock(&css_set_lock); + + memcg_do_reparent(memcg, parent); + + memcg_reparent_unlock(memcg, parent); + + local_irq_enable(); =20 percpu_ref_kill(&objcg->refcnt); } --=20 2.11.0