From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 695C7C38A2A for ; Fri, 8 May 2020 13:38:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 29E4D2070B for ; Fri, 8 May 2020 13:38:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20150623.gappssmtp.com header.i=@cmpxchg-org.20150623.gappssmtp.com header.b="bdGbD2A/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 29E4D2070B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=cmpxchg.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B317F900002; Fri, 8 May 2020 09:38:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ABB168E0003; Fri, 8 May 2020 09:38:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A99A900002; Fri, 8 May 2020 09:38:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0092.hostedemail.com [216.40.44.92]) by kanga.kvack.org (Postfix) with ESMTP id 8318A8E0003 for ; Fri, 8 May 2020 09:38:51 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 29E438248068 for ; Fri, 8 May 2020 13:38:51 +0000 (UTC) X-FDA: 76793657262.29.dolls88_13060c9159f51 X-HE-Tag: dolls88_13060c9159f51 X-Filterd-Recvd-Size: 5344 Received: from mail-qv1-f67.google.com (mail-qv1-f67.google.com [209.85.219.67]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Fri, 8 May 2020 13:38:50 +0000 (UTC) Received: by mail-qv1-f67.google.com with SMTP id ep1so725592qvb.0 for ; Fri, 08 May 2020 06:38:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=s3c/kHKjv4oi4xRI+SDdDjgcOJfA2m3GEd8rwNL8FJM=; b=bdGbD2A/K34eENIxrCE2mZH62IdbA4NbgUCDWNqpn8XBZAxW2jontvXRjCqJdK6uVp tLCQGN6n1BbZtWtCAdliAh80NpRzCgOR0R3DxFXitpehOva/RVp/LVCtLcQd2slTZXM6 7YN4ubLyjwbRL9qsv13akCcwa6UtQH8gdRTF5dA2KRAc7uI9B6eVzMF5PvMHlXmybtV2 XI0enlpf/OIKfjrQTh0C6FENh2YXYkAEK8nYAATeZeZTHJILEFpQVseDBCA4vz1D0YRs 0P3ehoSHiSOukGW9R/MiEdevBux0MRyrsZJOLqlp6RdRMmH5fhFJcnGsvLYJesv+KOtv ohWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=s3c/kHKjv4oi4xRI+SDdDjgcOJfA2m3GEd8rwNL8FJM=; b=tzIM7crvlDCo2Ph+DjNOhgQTnMS/jrp5DdTkD7dwWDmwh1PsQFUSElFn5qQ+qwlrBh xYBRQkb/a8VMn3Q5LNBk9VovzvEx9sOkMQex6OPFcK332EIu07JdTE0layd0uQwskdTq omOblkzNsGQv2BgDb7G6SvYcR/JrgcQ68ybZ4tthy+SyxVlSILwlfkNcz24GlFL6hVd4 e4j4LF/ZtUgUKxftXYxG5CTVoSExP5EWNb5ynIKM3TnN/hmQ3bYugjEz1Uv7gMiQS762 bLCFdNj+9MV4HRgSU9ix4AWdk/DFNf3VSFjoZltUQiwsyIuMyxfsNxXiaIEUDH+Oik1K paXg== X-Gm-Message-State: AGi0PuZHT0U+fyJxLucSNDD7Wnzpg8qF7XdJtazv+RwTAp84OgGhhEE0 R9Wc447k/BhNrd9elanBGvkR9g== X-Google-Smtp-Source: APiQypLjm7g5uyaNxmKnnhh0sSGk189fzCge53i7i5N/MovX8Gq7Ym3S3ZqrB5FalA/lrRUaYyZKQg== X-Received: by 2002:ad4:4b6b:: with SMTP id m11mr2814178qvx.130.1588945129920; Fri, 08 May 2020 06:38:49 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::1:2627]) by smtp.gmail.com with ESMTPSA id q17sm1162686qkq.111.2020.05.08.06.38.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 May 2020 06:38:49 -0700 (PDT) Date: Fri, 8 May 2020 09:38:33 -0400 From: Johannes Weiner To: Shakeel Butt Cc: Yafang Shao , Mel Gorman , Roman Gushchin , Michal Hocko , Andrew Morton , Linux MM , LKML Subject: Re: [PATCH] mm: vmscan: consistent update to pgsteal and pgscan Message-ID: <20200508133833.GA181181@cmpxchg.org> References: <20200507204913.18661-1-shakeelb@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, May 08, 2020 at 06:25:14AM -0700, Shakeel Butt wrote: > On Fri, May 8, 2020 at 3:34 AM Yafang Shao wrote: > > > > On Fri, May 8, 2020 at 4:49 AM Shakeel Butt wrote: > > > > > > One way to measure the efficiency of memory reclaim is to look at the > > > ratio (pgscan+pfrefill)/pgsteal. However at the moment these stats are > > > not updated consistently at the system level and the ratio of these are > > > not very meaningful. The pgsteal and pgscan are updated for only global > > > reclaim while pgrefill gets updated for global as well as cgroup > > > reclaim. > > > > > > > Hi Shakeel, > > > > We always use pgscan and pgsteal for monitoring the system level > > memory pressure, for example, by using sysstat(sar) or some other > > monitor tools. I'm in the same boat. It's useful to have activity that happens purely due to machine capacity rather than localized activity that happens due to the limits throughout the cgroup tree. > Don't you need pgrefill in addition to pgscan and pgsteal to get the > full picture of the reclaim activity? I actually almost never look at pgrefill. > > But with this change, these two counters include the memcg pressure as > > well. It is not easy to know whether the pgscan and pgsteal are caused > > by system level pressure or only some specific memcgs reaching their > > memory limit. > > > > How about adding cgroup_reclaim() to pgrefill as well ? > > > > I am looking for all the reclaim activity on the system. Adding > !cgroup_reclaim to pgrefill will skip the cgroup reclaim activity. > Maybe adding pgsteal_cgroup and pgscan_cgroup would be better. How would you feel about adding memory.stat at the root cgroup level? There are subtle differences between /proc/vmstat and memory.stat, and cgroup-aware code that wants to watch the full hierarchy currently has to know about these intricacies and translate semantics back and forth. Generally having the fully recursive memory.stat at the root level could help a broader range of usecases.