From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CD1BCA90AF for ; Wed, 13 May 2020 12:36:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1098220659 for ; Wed, 13 May 2020 12:36:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20150623.gappssmtp.com header.i=@cmpxchg-org.20150623.gappssmtp.com header.b="zFqMJydT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1098220659 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=cmpxchg.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9FFC990012B; Wed, 13 May 2020 08:36:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 988D89000F3; Wed, 13 May 2020 08:36:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 828D790012B; Wed, 13 May 2020 08:36:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id 67B8C9000F3 for ; Wed, 13 May 2020 08:36:07 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1A1A8181AEF00 for ; Wed, 13 May 2020 12:36:07 +0000 (UTC) X-FDA: 76811643174.18.need96_355b3047f4e48 X-HE-Tag: need96_355b3047f4e48 X-Filterd-Recvd-Size: 5847 Received: from mail-qk1-f195.google.com (mail-qk1-f195.google.com [209.85.222.195]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Wed, 13 May 2020 12:36:06 +0000 (UTC) Received: by mail-qk1-f195.google.com with SMTP id n14so17070343qke.8 for ; Wed, 13 May 2020 05:36:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=XPrK4VVoXQ0vmjjlKF3mWEEfj4+VgK3r8tZECPQos5E=; b=zFqMJydTAPVLxNLXFDGmaVOy8rh6YFK1hKv2Qx0WWlU5242a2O1SoY314bSQ3vHLYG fnm+ll+8xNx0LDpF6ao7AIhPzgBKjsrAzDSVrxXrFWjn0wXikhgApqIaBZ6RRowBmUQr TUHGa7lNx1EMlc4ISPqhND0a1EIPh/7bboJjlqyL+qejWBjIlcT2We1UYz8lcIF8MLfW qfVBuPWRVXCRGYQqNlL/jtMkttkRSBjwdJyyusuu69L1QxXCCbZoUKdMgs67WIohRC9o wilWL6mdq85EgIOi/9ZWphJ3CgLHhOjTSLJM16IGS3GGtyZSM0LJFF+1GAyQb0+JP1J3 cqkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=XPrK4VVoXQ0vmjjlKF3mWEEfj4+VgK3r8tZECPQos5E=; b=Wvn79pJPieCtPAbB7xqcW5RHq/eI7FhiuEDoduzp7l3NPwfUvffLMuinvIq55lMT5H cWpaHyYaBgE2OTioHbhQziv3Of47C6E78c78We2n6UXmvk6ZgzeiYwEgFUCaDH+rgWiC uu7y75wbOvz/bo7q2ci8sGRwzyvXIFijqOM+/rjN0Fuknq+qvWGDPg+cus3AJEa1Z6+q 5rAw8xw+zitVMTtidNXjlvtI06EOy81jhLak9X2GgNglM+CRVlQt8osKs2HQcApWvkQk BVRYBTzmsPEgKusBWfMXEj5W4hiY6ezHNezqNgcq2GwkizCq4K3PnKNWa/zikHv8jbN0 veBQ== X-Gm-Message-State: AGi0PuYosoQpzDvjrrrn9gwt5dcHJBJGT667Duve/8C6jMSAx70nSz4l xfjTyZV52A9/rXegjPu8gSOGNg== X-Google-Smtp-Source: APiQypKyjeVvaBEXNHX6gs8IWZmhdLxlbCAn4XHSVARLevo9HOXc4orsg8JkQKHilUhQQLczQ8MybQ== X-Received: by 2002:a05:620a:1509:: with SMTP id i9mr23815994qkk.445.1589373365539; Wed, 13 May 2020 05:36:05 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::1:2627]) by smtp.gmail.com with ESMTPSA id v44sm12093056qtk.79.2020.05.13.05.36.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 May 2020 05:36:04 -0700 (PDT) Date: Wed, 13 May 2020 08:35:45 -0400 From: Johannes Weiner To: Balbir Singh Cc: Andrew Morton , Alex Shi , Joonsoo Kim , Shakeel Butt , Hugh Dickins , Michal Hocko , "Kirill A. Shutemov" , Roman Gushchin , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: Re: [PATCH 00/19 V2] mm: memcontrol: charge swapin pages on instantiation Message-ID: <20200513123545.GB488426@cmpxchg.org> References: <20200508183105.225460-1-hannes@cmpxchg.org> <20200513113032.GA93568@dev-dsk-sblbir-1c-a524888b.ap-northeast-1.amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200513113032.GA93568@dev-dsk-sblbir-1c-a524888b.ap-northeast-1.amazon.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hello Balbir! On Wed, May 13, 2020 at 11:30:32AM +0000, Balbir Singh wrote: > On Fri, May 08, 2020 at 02:30:47PM -0400, Johannes Weiner wrote: > > To eliminate the page->mapping dependency, memcg needs to ditch its > > private page type counters (MEMCG_CACHE, MEMCG_RSS, NR_SHMEM) in favor > > of the generic vmstat counters and accounting sites, such as > > NR_FILE_PAGES, NR_ANON_MAPPED etc. > > Could you elaborate on what this means and the implications of this on > user space programs? This has no bearing on userspace. It's just simplifying how memory.stat is implemented. The output is the same. For the full story: In the past, memcg has done its own accounting to produce a breakdown of consumers in memory.stat. When a page was charged, we relied on knowing whether it's a file, anon or shmem page, and had our own MEMCG_RSS, MEMCG_CACHE, MEMCG_SHMEM counters. As the general VM code already does this type of classification to produce /proc/vmstat, this meant unnecessary duplication: more places to bump counters, more places that have to make sure the page state is stable in all the right ways, more dependencies on when it's safe to call the charge and the uncharge callbacks. A while ago we added per-cgroup arrays of the vmstat counters and a cgroup-aware accounting callback (mod_lruvec_state) that can be a drop-in replacement for the generic VM code (mod_node_state and friends). We already had some counters converted over to that. These patches just do more of that conversion from private memcg accounting to having callbacks into generic VM accounting sites. Instead of testing PageAnon() and accounting MEMCG_CACHE/MEMCG_RSS in the charge code, we switch __add_to_page_cache_locked() and page_add_new_anon_rmap() to the cgroup-aware mod_lruvec_page_state() to bump our per-cgroup NR_FILE_PAGES and NR_ANON_MAPPED counters along with the node and global counters. As a result, the memcg gets a breakdown for memory.stat without having to have private knowledge of what a page cache page is - how to test it, when it's safe to test it, whether there can be huge pages in the page cache, etc. pp. Memcg can focus on counting bytes, and the VM code that is specialized in dealing with the page cache (or anon pages, or shmem pages) can fill in those kinds of details for us. Less dependencies, less duplication, simpler API rules. The memory.stat output is the same, it's just much simpler code.