From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED8E2C4707F for ; Wed, 26 May 2021 02:47:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CD1D161432 for ; Wed, 26 May 2021 02:47:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232508AbhEZCsn (ORCPT ); Tue, 25 May 2021 22:48:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230312AbhEZCsm (ORCPT ); Tue, 25 May 2021 22:48:42 -0400 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDC6AC061574 for ; Tue, 25 May 2021 19:47:10 -0700 (PDT) Received: by mail-pj1-x1029.google.com with SMTP id lx17-20020a17090b4b11b029015f3b32b8dbso12621169pjb.0 for ; Tue, 25 May 2021 19:47:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rv5/ezA/8ChULZqCUpXeS3F+dyYW6ATzNriZIRF37oU=; b=x2XFtSpk+a0G0Xh8JqZ6IYdsFxwgPM1uoaWudn2o45O4Kgp5ohOFq8Bxrj2EjXiYwa z1lwL26xH7JpX1pJf8DgME42AIE7GR8S6gGIN0+YfYeQOfFqg7O9XEqTuBimnGq80ANS m7sfN2mG2XZTVJIDJ6WIjcBn4J5oIyJsT5IF/8O8ooRjDwdXGZIFAqMpLK1/y4F6mxaj HPw2lEQzLhN4iyZtfIxyP5DZsS/J3Kj3nbjSYsh0sZmcWk1jMwVGmDBm7zO+nKUxBsy+ QbW7b/KVL+KMu8DDZs5c1mpPg39nXPRwG3fGzoCZ9nVM4qCnUh282TeOmQYGxayP2wsL KUTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rv5/ezA/8ChULZqCUpXeS3F+dyYW6ATzNriZIRF37oU=; b=gnK3l+KT6+HNLz8UfdUJxjxUmjkJaCkd18hx8JSyV8ydgHwwVcPD0KohNA/nKxAbJ/ nLlVpBuyw/eFiaadrBPRHGtsTlZoa0UbhNbD/GP5FpMrMozTSf92GKHKN1yQIdKlxnwn xiKTf5D+PmBQyYCJH+9da7ADGOwawZR5KsHP7DKvpzEKtYCa7GTBdk6a2Dj3+nJzeOcP g8N32y9EZ2njUT7xbWbb2tB+23IoDEtLFSxcEuWSnqHUpCCk1tHRPfcUmMe5gXAq0w8+ KbWP090ajFX7qvaOdinOdVuT0O/Z8PBvjfL55cxBRVeVdZ4aUR5yrIP00E5TgF4DJD5I Talw== X-Gm-Message-State: AOAM531sF+j/PWdPydxreIqoON/QBentGgpmL6eA5OCFG3EA3rUTnxY7 OjttpddURsSyI7uI7NSFzFEL8/prvt0We3Ne6Ml0ew== X-Google-Smtp-Source: ABdhPJxP8OLfQ9amwyzkTyXy09v+A82DJD0GfHZJFLhR+rGr+5DPNA10gLlPgTF9H3tCb4DJAXhSRjGZQTLjLabVWuI= X-Received: by 2002:a17:902:d711:b029:f0:b127:8105 with SMTP id w17-20020a170902d711b02900f0b1278105mr33927023ply.20.1621997230423; Tue, 25 May 2021 19:47:10 -0700 (PDT) MIME-Version: 1.0 References: <20210421070059.69361-1-songmuchun@bytedance.com> <20210421070059.69361-2-songmuchun@bytedance.com> In-Reply-To: From: Muchun Song Date: Wed, 26 May 2021 10:46:34 +0800 Message-ID: Subject: Re: [External] Re: [RFC PATCH v3 01/12] mm: memcontrol: move the objcg infrastructure out of CONFIG_MEMCG_KMEM To: Roman Gushchin Cc: Johannes Weiner , Michal Hocko , Andrew Morton , Shakeel Butt , Vladimir Davydov , LKML , Linux Memory Management List , Xiongchun duan , fam.zheng@bytedance.com, "Singh, Balbir" , Yang Shi , Alex Shi Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, May 26, 2021 at 12:27 AM Roman Gushchin wrote: > > On Wed, Apr 21, 2021 at 03:00:48PM +0800, Muchun Song wrote: > > Because memory allocations pinning memcgs for a long time - it exists > > at a larger scale and is causing recurring problems in the real world: > > page cache doesn't get reclaimed for a long time, or is used by the > > second, third, fourth, ... instance of the same job that was restarted > > into a new cgroup every time. Unreclaimable dying cgroups pile up, > > waste memory, and make page reclaim very inefficient. > > > > We can convert LRU pages and most other raw memcg pins to the objcg > > direction to fix this problem, and then the page->memcg will always > > point to an object cgroup pointer. > > > > Therefore, the infrastructure of objcg no longer only serves > > CONFIG_MEMCG_KMEM. In this patch, we move the infrastructure of the > > objcg out of the scope of the CONFIG_MEMCG_KMEM so that the LRU pages > > can reuse it to charge pages. > > > > We know that the LRU pages are not accounted at the root level. But > > the page->memcg_data points to the root_mem_cgroup. So the > > page->memcg_data of the LRU pages always points to a valid pointer. > > But the root_mem_cgroup dose not have an object cgroup. If we use > > obj_cgroup APIs to charge the LRU pages, we should set the > > page->memcg_data to a root object cgroup. So we also allocate an > > object cgroup for the root_mem_cgroup. > > Overall the patch looks very good to me. There are few small things to enhance: > > 1) I'd rename it. Looking at the title I expect a trivial code move, > however the patch is doing more than this: e.g. allocating an objcg > for the root memcg. Something like "prepare objcg API for non-kmem usage". OK. I will rename it. > 2) How about obj_cgroup_release_kmem() instead of obj_cgroup_release_uncharge()? LGTM. Will use this name. > 3) The first paragraph of the commit log looks a bit vague: which allocations > pinning memcgs? How about something like this? > > Pagecache pages are charged at the allocation time and holding a reference > to the original memory cgroup until being reclaimed. Depending on the memory > pressure, specific patterns of the page sharing between different cgroups and > the cgroup creation and destruction rates, a large number of dying memory > cgroups can be pinned by pagecache pages. It makes the page reclaim less > efficient and wastes memory. More clear. I would love to use this. Thanks Roman. > > > Thanks! From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4551C47087 for ; Wed, 26 May 2021 02:47:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4A91C6142D for ; Wed, 26 May 2021 02:47:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4A91C6142D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C22A96B0036; Tue, 25 May 2021 22:47:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD2586B006E; Tue, 25 May 2021 22:47:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A726E6B0070; Tue, 25 May 2021 22:47:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0114.hostedemail.com [216.40.44.114]) by kanga.kvack.org (Postfix) with ESMTP id 7510F6B0036 for ; Tue, 25 May 2021 22:47:12 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0BA4D8798 for ; Wed, 26 May 2021 02:47:12 +0000 (UTC) X-FDA: 78181845504.12.448679C Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by imf28.hostedemail.com (Postfix) with ESMTP id 2224C20007FA for ; Wed, 26 May 2021 02:47:05 +0000 (UTC) Received: by mail-pj1-f44.google.com with SMTP id ml1-20020a17090b3601b029015f9b1ebce0so5077642pjb.5 for ; Tue, 25 May 2021 19:47:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rv5/ezA/8ChULZqCUpXeS3F+dyYW6ATzNriZIRF37oU=; b=x2XFtSpk+a0G0Xh8JqZ6IYdsFxwgPM1uoaWudn2o45O4Kgp5ohOFq8Bxrj2EjXiYwa z1lwL26xH7JpX1pJf8DgME42AIE7GR8S6gGIN0+YfYeQOfFqg7O9XEqTuBimnGq80ANS m7sfN2mG2XZTVJIDJ6WIjcBn4J5oIyJsT5IF/8O8ooRjDwdXGZIFAqMpLK1/y4F6mxaj HPw2lEQzLhN4iyZtfIxyP5DZsS/J3Kj3nbjSYsh0sZmcWk1jMwVGmDBm7zO+nKUxBsy+ QbW7b/KVL+KMu8DDZs5c1mpPg39nXPRwG3fGzoCZ9nVM4qCnUh282TeOmQYGxayP2wsL KUTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rv5/ezA/8ChULZqCUpXeS3F+dyYW6ATzNriZIRF37oU=; b=EIwSWPVGENbOZU16T2hO26q/4PJ9tAfn0Sr27bNxu+QNQTAVHM4sK9EVBZjEdC5Hsy xVMwt8Nfh13xzmsZaEK+8CjoAG0SfcyxUrLn8lVa6BSWtOlSdLuYLAIvuPJC+K6ZsltR Yh6H9GrqZKl4GovNMAqeZe+42XMtGFyOSBZ1SW+Z880wNVtitrlm7EoVKtGNDGEojFJu gTXBXkv/2GdS260mp8v6DWT0VDrz5PoAPBVSh0iaMfSijR7tralVjNZdtV0gHGuHALbQ ccgCllIRxfcUTfZRCz/AOHQ0tJtMXLOgFk4hhV3Ddjf3bCPSvy8x+CoEeIwzN3/yUATs 5bsg== X-Gm-Message-State: AOAM531wLsJdrIVuLoeNg1KyTARmpSv+Xq9rTBp3KGkS4nDVHT7M5BDO 3zD+vfhYAiCOZBvBrlVEVY/+wFvK2iUqbxA2yHMVtQ== X-Google-Smtp-Source: ABdhPJxP8OLfQ9amwyzkTyXy09v+A82DJD0GfHZJFLhR+rGr+5DPNA10gLlPgTF9H3tCb4DJAXhSRjGZQTLjLabVWuI= X-Received: by 2002:a17:902:d711:b029:f0:b127:8105 with SMTP id w17-20020a170902d711b02900f0b1278105mr33927023ply.20.1621997230423; Tue, 25 May 2021 19:47:10 -0700 (PDT) MIME-Version: 1.0 References: <20210421070059.69361-1-songmuchun@bytedance.com> <20210421070059.69361-2-songmuchun@bytedance.com> In-Reply-To: From: Muchun Song Date: Wed, 26 May 2021 10:46:34 +0800 Message-ID: Subject: Re: [External] Re: [RFC PATCH v3 01/12] mm: memcontrol: move the objcg infrastructure out of CONFIG_MEMCG_KMEM To: Roman Gushchin Cc: Johannes Weiner , Michal Hocko , Andrew Morton , Shakeel Butt , Vladimir Davydov , LKML , Linux Memory Management List , Xiongchun duan , fam.zheng@bytedance.com, "Singh, Balbir" , Yang Shi , Alex Shi Content-Type: text/plain; charset="UTF-8" Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=bytedance-com.20150623.gappssmtp.com header.s=20150623 header.b=x2XFtSpk; spf=pass (imf28.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 2224C20007FA X-Stat-Signature: 5p1eb61gyrzcxqxo7bu4rsxo1ytradft X-HE-Tag: 1621997225-269465 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 26, 2021 at 12:27 AM Roman Gushchin wrote: > > On Wed, Apr 21, 2021 at 03:00:48PM +0800, Muchun Song wrote: > > Because memory allocations pinning memcgs for a long time - it exists > > at a larger scale and is causing recurring problems in the real world: > > page cache doesn't get reclaimed for a long time, or is used by the > > second, third, fourth, ... instance of the same job that was restarted > > into a new cgroup every time. Unreclaimable dying cgroups pile up, > > waste memory, and make page reclaim very inefficient. > > > > We can convert LRU pages and most other raw memcg pins to the objcg > > direction to fix this problem, and then the page->memcg will always > > point to an object cgroup pointer. > > > > Therefore, the infrastructure of objcg no longer only serves > > CONFIG_MEMCG_KMEM. In this patch, we move the infrastructure of the > > objcg out of the scope of the CONFIG_MEMCG_KMEM so that the LRU pages > > can reuse it to charge pages. > > > > We know that the LRU pages are not accounted at the root level. But > > the page->memcg_data points to the root_mem_cgroup. So the > > page->memcg_data of the LRU pages always points to a valid pointer. > > But the root_mem_cgroup dose not have an object cgroup. If we use > > obj_cgroup APIs to charge the LRU pages, we should set the > > page->memcg_data to a root object cgroup. So we also allocate an > > object cgroup for the root_mem_cgroup. > > Overall the patch looks very good to me. There are few small things to enhance: > > 1) I'd rename it. Looking at the title I expect a trivial code move, > however the patch is doing more than this: e.g. allocating an objcg > for the root memcg. Something like "prepare objcg API for non-kmem usage". OK. I will rename it. > 2) How about obj_cgroup_release_kmem() instead of obj_cgroup_release_uncharge()? LGTM. Will use this name. > 3) The first paragraph of the commit log looks a bit vague: which allocations > pinning memcgs? How about something like this? > > Pagecache pages are charged at the allocation time and holding a reference > to the original memory cgroup until being reclaimed. Depending on the memory > pressure, specific patterns of the page sharing between different cgroups and > the cgroup creation and destruction rates, a large number of dying memory > cgroups can be pinned by pagecache pages. It makes the page reclaim less > efficient and wastes memory. More clear. I would love to use this. Thanks Roman. > > > Thanks!