From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97B32C433DB for ; Fri, 19 Mar 2021 16:38:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 34C9E61934 for ; Fri, 19 Mar 2021 16:38:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34C9E61934 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C49D78D0010; Fri, 19 Mar 2021 12:38:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C1FF56B008C; Fri, 19 Mar 2021 12:38:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AAE468D0010; Fri, 19 Mar 2021 12:38:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0031.hostedemail.com [216.40.44.31]) by kanga.kvack.org (Postfix) with ESMTP id 8D7C76B008A for ; Fri, 19 Mar 2021 12:38:56 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 3DAB518014C9C for ; Fri, 19 Mar 2021 16:38:56 +0000 (UTC) X-FDA: 77937183072.15.76C1FEC Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf14.hostedemail.com (Postfix) with ESMTP id 70C59C001C7C for ; Fri, 19 Mar 2021 16:38:41 +0000 (UTC) Received: by mail-pf1-f170.google.com with SMTP id j25so6277333pfe.2 for ; Fri, 19 Mar 2021 09:38:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=iJa75g1dgQtxiTA2vP7trUyHDt/yfHb4pEZP0XDxdyk=; b=TD3mNIlSrsMDaKcOHYRI65gHJMne1XlHoPhiRB3+wRhzdE6taAm3jxf9rtSNhT9b66 mg9Qlynm0WqYQuzMrO62bvLQJYCSZnp6x9um5iX+OCiuvzt0BbDcin1Vp7lrjsGek26e XNflfPFLZ50IZwwdsxqrX4fwJqCOa2mVMr86tPHKD2Tg5a5PsFTtPr2EN5Cmxu9g21/u C4qvForVWGxuIhvwwMZeBfmF9JKXkCBKDlO5v6EnABh+7xpnPoaEqw7xFy8jpPgJk3Na EXGImFUfVB5Smo/oRSc/Q3emfiosPsu4Tyn/HQ4MBddkUeQsvZuLfwX+lr94DN0noy0I LKTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=iJa75g1dgQtxiTA2vP7trUyHDt/yfHb4pEZP0XDxdyk=; b=VAkOTFPtUtLX/bymFZEroTDq37gQaHDytdPVxsYLvZGycieAZ2lxr5xTPt62KBtW/T o2q8iN5gvU3Yd7N7d8O/plPx5oWFAhWMet4rjBAWf7T3lOARs6bei8m00AB4nLIzCkJM zed6Ot57u0GLsEFsc7JLc6wkvM+xlkzkS5wFlXYQIhTI4E3zUS7jm+ZcuRAS2gNESEcU 78lkzIqubY8slgT4JPOTud4eI/z+nQR74HzzWN/IHIRLqQTHiL19TLXx9JKT6i7aBGX8 2SzzkaUVMgM55w26+UOjfRh2A0OAPCN1oANFTXxwHVsbtvD5IC/eNJCbHE+BAhRlwvQy wEjQ== X-Gm-Message-State: AOAM532j6VtDJg0VEQjo9Tt1xU8txEckdfOg2JsB9q+VE3LvAsZqmj3V McTxtCn4slAaLiqpNNav+lJw/w== X-Google-Smtp-Source: ABdhPJw7RYbYhSMGGfMO4TDIe8w0j7nS6xxCIu6L88if3semVnkeQaxK/TzMdTh1se/cr6b6QWfuag== X-Received: by 2002:a63:f14b:: with SMTP id o11mr12268388pgk.440.1616171918486; Fri, 19 Mar 2021 09:38:38 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.231]) by smtp.gmail.com with ESMTPSA id z25sm5860239pfn.37.2021.03.19.09.38.34 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Mar 2021 09:38:38 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song Subject: [PATCH v5 0/7] Use obj_cgroup APIs to charge kmem pages Date: Sat, 20 Mar 2021 00:38:13 +0800 Message-Id: <20210319163821.20704-1-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) MIME-Version: 1.0 X-Stat-Signature: qjy79qsct8jnahye44dfpm1irtr87tsn X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 70C59C001C7C Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail-pf1-f170.google.com; client-ip=209.85.210.170 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616171921-652059 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since Roman series "The new cgroup slab memory controller" applied. All slab objects are charged with the new APIs of obj_cgroup. The new APIs introduce a struct obj_cgroup to charge slab objects. It prevents long-living objects from pinning the original memory cgroup in the memory= . But there are still some corner objects (e.g. allocations larger than order-1 page on SLUB) which are not charged with the new APIs. Those objects (include the pages which are allocated from buddy allocator directly) are charged as kmem pages which still hold a reference to the memory cgroup. E.g. We know that the kernel stack is charged as kmem pages because the size of the kernel stack can be greater than 2 pages (e.g. 16KB on x86_64 or arm64). If we create a thread (suppose the thread stack is charged to memory cgroup A) and then move it from memory cgroup A to memory cgroup B. Because the kernel stack of the thread hold a reference to the memory cgroup A. The thread can pin the memory cgroup A in the memory even if we remove the cgroup A. If we want to see this scenario by using the following script. We can see that the system has added 500 dying cgroups (This is not a real world issue, just a script to show that the large kmallocs are charged as kmem pages which can pin the memory cgroup in the memory). #!/bin/bash cat /proc/cgroups | grep memory cd /sys/fs/cgroup/memory echo 1 > memory.move_charge_at_immigrate for i in range{1..500} do mkdir kmem_test echo $$ > kmem_test/cgroup.procs sleep 3600 & echo $$ > cgroup.procs echo `cat kmem_test/cgroup.procs` > cgroup.procs rmdir kmem_test done cat /proc/cgroups | grep memory This patchset aims to make those kmem pages to drop the reference to memo= ry cgroup by using the APIs of obj_cgroup. Finally, we can see that the numb= er of the dying cgroups will not increase if we run the above test script. Changlogs in v5: 1. Add a new patch (1st) to fix a potential issue. 2. Rename get_obj_cgroup_memcg() to get_mem_cgroup_from_objcg(). 3. Remove get_mem_cgroup_from_current() and get_active_memcg(). 4. Add a comment to uncharge_page(). 5. Add a separate patch to inline __memcg_kmem_{un}charge() into obj_cgroup_{un}charge_pages(). 6. Collect Acked-by and Reviewed-by tags. Thanks to Johannes and Shakeel's review and suggestions. Changlogs in v4: 1. Do not change behavior of page_memcg() and page_memcg_rcu(). 2. Rework uncharge_page() and uncharge_batch(). 3. Add two patches (patch #2 and patch #3). Thanks to Johannes and Shakeel and Roman's review and suggestions. Changlogs in v3: 1. Drop "remote objcg charging APIs" patch. 2. Rename obj_cgroup_{un}charge_page to obj_cgroup_{un}charge_pages. 3. Make page_memcg/page_memcg_rcu safe for adding new memcg_data flags. 4. Reuse the ug infrastructure to uncharge the kmem pages. 5. Add a new patch to move PageMemcgKmem to the scope of CONFIG_MEMCG_K= MEM. Thanks to Roman's review and suggestions. Changlogs in v2: 1. Fix some types in the commit log (Thanks Roman). 2. Do not introduce page_memcg_kmem helper (Thanks to Johannes and Shak= eel). 3. Reduce the CC list to mm/memcg folks (Thanks to Johannes). 4. Introduce remote objcg charging APIs instead of convert "remote memc= g charging APIs" to "remote objcg charging APIs". Muchun Song (7): mm: memcontrol: slab: fix obtain a reference to a freeing memcg mm: memcontrol: introduce obj_cgroup_{un}charge_pages mm: memcontrol: directly access page->memcg_data in mm/page_alloc.c mm: memcontrol: change ug->dummy_page only if memcg changed mm: memcontrol: use obj_cgroup APIs to charge kmem pages mm: memcontrol: inline __memcg_kmem_{un}charge() into obj_cgroup_{un}charge_pages() mm: memcontrol: move PageMemcgKmem to the scope of CONFIG_MEMCG_KMEM include/linux/memcontrol.h | 123 +++++++++++++++++++++------ mm/memcontrol.c | 206 +++++++++++++++++++++++----------------= ------ mm/page_alloc.c | 4 +- 3 files changed, 207 insertions(+), 126 deletions(-) --=20 2.11.0