From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11D9EC433DB for ; Thu, 24 Dec 2020 02:47:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B87B7229CA for ; Thu, 24 Dec 2020 02:47:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728793AbgLXCrQ (ORCPT ); Wed, 23 Dec 2020 21:47:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34918 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728334AbgLXCrP (ORCPT ); Wed, 23 Dec 2020 21:47:15 -0500 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48502C061794 for ; Wed, 23 Dec 2020 18:46:35 -0800 (PST) Received: by mail-pf1-x431.google.com with SMTP id t22so522338pfl.3 for ; Wed, 23 Dec 2020 18:46:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=fDRfu0M1YLV1uhT7YoKaiH9p9Ye/ITZc1yvaQ4SUQRQ=; b=ZJPDaWbo/DIXrpPoj3kbzFZq3ifzLIDVBRcOUeeDau6bNYnMbfXwq+Q3xjq+SqbDy4 Dim1zSamlWYGDHO7vC5p5k5feoa3IXuVpZaqrm86asSB+klu5vOjmug219Kw/52k8nAo eHOfWtYH2gjhQhErEg4GMr3D5/GaUBAn4XQCkV/btHnMMR6M1AEaIiPQpHv9/jr0V9p5 dPyMkWT6VkJRbkcY3AvB9Eta4zTVQCQmf9DRFylxLwf8Zm98OVmndOXtNERC8HmdkP0F kuhZWGA2K7XU/Gz+RkI2T6VYvzo+paOxH/7ud2cUmqITTJde3TZ5DowlQoAvpRH7yspk 1t5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=fDRfu0M1YLV1uhT7YoKaiH9p9Ye/ITZc1yvaQ4SUQRQ=; b=gb2+pvJLW8oCqXd7LyE8bXB/qvLluENirrnYUA4ZKgKNnIg9//nYdzMdLI//3d+EkS ZAU62gBHbJ0axwbYIKVrUDSbn9Ck3VU1vdRkZT7EpVi9dXmg0py4XtYSmvPjVmjP/EDZ W9wHvoaLMANKz4++MYa6AFUMfweyAh5tFefw9eoQdeXW86xhWmzuqY6RGWCSXzT5/ouJ 5LmD+rvJwQvDFZKrFzqnTeMIZlXn2rrUj4IcPyEIwX3GzdWhUF8JYmo0LyxEtlvOcjjV ELKBNDSb10brZ+PUO11DC8P+FRFQHlTd2QV4IFRRPkhhIj396sZtjctuvA0UbhM9uE4J O7Nw== X-Gm-Message-State: AOAM533Td7eFx7olJzdeg8W7LzrNwjpM+60qXaAQp8JpKD8usIgyg5yB GDHzU8qVdyBxuAnwEyKYptxoshbpETD42O2qNzZ9pQ== X-Google-Smtp-Source: ABdhPJwYrW/b7eupKEutfmD9j8Xwe7uZNE+Erred065j6h0pMc4R4dMuAUJfUM4SPXlq+LkRkRNFBEAp1N6bYEuBtHo= X-Received: by 2002:aa7:979d:0:b029:1a4:3b76:a559 with SMTP id o29-20020aa7979d0000b02901a43b76a559mr26008689pfp.49.1608777994836; Wed, 23 Dec 2020 18:46:34 -0800 (PST) MIME-Version: 1.0 References: <20201217034356.4708-1-songmuchun@bytedance.com> <20201217034356.4708-3-songmuchun@bytedance.com> In-Reply-To: From: Muchun Song Date: Thu, 24 Dec 2020 10:45:58 +0800 Message-ID: Subject: Re: [External] Re: [PATCH v5 2/7] mm: memcontrol: convert NR_ANON_THPS account to pages To: Shakeel Butt Cc: Greg Kroah-Hartman , "Rafael J. Wysocki" , Alexey Dobriyan , Andrew Morton , Johannes Weiner , Michal Hocko , Vladimir Davydov , Hugh Dickins , Roman Gushchin , Sami Tolvanen , Feng Tang , Neil Brown , Joonsoo Kim , Randy Dunlap , LKML , linux-fsdevel , Linux MM , Cgroups Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Dec 24, 2020 at 6:08 AM Shakeel Butt wrote: > > On Wed, Dec 16, 2020 at 7:45 PM Muchun Song wrote: > > > > Currently we use struct per_cpu_nodestat to cache the vmstat > > counters, which leads to inaccurate statistics expecially THP > > *especially Thanks. > > > vmstat counters. In the systems with hundreads of processors > > *hundreds Thanks. > > > it can be GBs of memory. For example, for a 96 CPUs system, > > the threshold is the maximum number of 125. And the per cpu > > counters can cache 23.4375 GB in total. > > > > The THP page is already a form of batched addition (it will > > add 512 worth of memory in one go) so skipping the batching > > seems like sensible. Although every THP stats update overflows > > the per-cpu counter, resorting to atomic global updates. But > > it can make the statistics more accuracy for the THP vmstat > > counters. > > > > So we convert the NR_ANON_THPS account to pages. This patch > > is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs > > on compound page arrival"). Doing this also can make the unit > > of vmstat counters more unified. Finally, the unit of the vmstat > > counters are pages, kB and bytes. The B/KB suffix can tell us > > that the unit is bytes or kB. The rest which is without suffix > > are pages. > > > > Signed-off-by: Muchun Song > > I agree with the motivation behind this patch but I would like to see > some performance numbers in the commit message. We might agree to pay > the price but at least we will know what exactly that cost is. Do you have any recommendations about benchmarks? I can do a test. Thanks very much. -- Yours, Muchun From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57795C433E0 for ; Thu, 24 Dec 2020 02:46:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 060CB22955 for ; Thu, 24 Dec 2020 02:46:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 060CB22955 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 16F758D0069; Wed, 23 Dec 2020 21:46:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 11F468D0063; Wed, 23 Dec 2020 21:46:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 065C08D0069; Wed, 23 Dec 2020 21:46:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0207.hostedemail.com [216.40.44.207]) by kanga.kvack.org (Postfix) with ESMTP id E3DF58D0063 for ; Wed, 23 Dec 2020 21:46:36 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A9CCA8249980 for ; Thu, 24 Dec 2020 02:46:36 +0000 (UTC) X-FDA: 77626637592.09.side59_30045f02746d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id 91894180AD82F for ; Thu, 24 Dec 2020 02:46:36 +0000 (UTC) X-HE-Tag: side59_30045f02746d X-Filterd-Recvd-Size: 5226 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Thu, 24 Dec 2020 02:46:35 +0000 (UTC) Received: by mail-pf1-f181.google.com with SMTP id x126so512384pfc.7 for ; Wed, 23 Dec 2020 18:46:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=fDRfu0M1YLV1uhT7YoKaiH9p9Ye/ITZc1yvaQ4SUQRQ=; b=ZJPDaWbo/DIXrpPoj3kbzFZq3ifzLIDVBRcOUeeDau6bNYnMbfXwq+Q3xjq+SqbDy4 Dim1zSamlWYGDHO7vC5p5k5feoa3IXuVpZaqrm86asSB+klu5vOjmug219Kw/52k8nAo eHOfWtYH2gjhQhErEg4GMr3D5/GaUBAn4XQCkV/btHnMMR6M1AEaIiPQpHv9/jr0V9p5 dPyMkWT6VkJRbkcY3AvB9Eta4zTVQCQmf9DRFylxLwf8Zm98OVmndOXtNERC8HmdkP0F kuhZWGA2K7XU/Gz+RkI2T6VYvzo+paOxH/7ud2cUmqITTJde3TZ5DowlQoAvpRH7yspk 1t5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=fDRfu0M1YLV1uhT7YoKaiH9p9Ye/ITZc1yvaQ4SUQRQ=; b=gksijEpC1UYR2q/pXwPpDYCrQWzOU2lVLqgq0QZpSPzGmKKbTApknLLveKdB9H9/aq AlKI2+bgNPbvSFCfeTBcRnGTh7tpPqQUJZpXCO14wkGJj8JZ1+kV7g4eAhfKrdKoyWCk Wz2G+/8RYrTMAs85wmjfpZXbKjq80pA8y/VfYuKRES/itAlNL52/VAk7mNIDGpedVA+q 9aVh141urnsqJZ6fPzFpuFnLBFRG7ChmvLfDwk1fEU3wqwdFXeCbTB7kA53q4Jl3ZJan YOKUslD5DnizpNZPjYf7xKliAVavsgiAJu9NWJiT8lFUJekYZLlcVY5eu6hq/gLA3HCo uODA== X-Gm-Message-State: AOAM5331kqmmr3ceKMa+Mxun3YC5VfThxjOfFWK9F/C4UCbIlF4nRrGy fl42GT3J0ZzQmSWhyey4EoQpBpPJ2EZCxCS2o+6wNQ== X-Google-Smtp-Source: ABdhPJwYrW/b7eupKEutfmD9j8Xwe7uZNE+Erred065j6h0pMc4R4dMuAUJfUM4SPXlq+LkRkRNFBEAp1N6bYEuBtHo= X-Received: by 2002:aa7:979d:0:b029:1a4:3b76:a559 with SMTP id o29-20020aa7979d0000b02901a43b76a559mr26008689pfp.49.1608777994836; Wed, 23 Dec 2020 18:46:34 -0800 (PST) MIME-Version: 1.0 References: <20201217034356.4708-1-songmuchun@bytedance.com> <20201217034356.4708-3-songmuchun@bytedance.com> In-Reply-To: From: Muchun Song Date: Thu, 24 Dec 2020 10:45:58 +0800 Message-ID: Subject: Re: [External] Re: [PATCH v5 2/7] mm: memcontrol: convert NR_ANON_THPS account to pages To: Shakeel Butt Cc: Greg Kroah-Hartman , "Rafael J. Wysocki" , Alexey Dobriyan , Andrew Morton , Johannes Weiner , Michal Hocko , Vladimir Davydov , Hugh Dickins , Roman Gushchin , Sami Tolvanen , Feng Tang , Neil Brown , Joonsoo Kim , Randy Dunlap , LKML , linux-fsdevel , Linux MM , Cgroups Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Dec 24, 2020 at 6:08 AM Shakeel Butt wrote: > > On Wed, Dec 16, 2020 at 7:45 PM Muchun Song wrote: > > > > Currently we use struct per_cpu_nodestat to cache the vmstat > > counters, which leads to inaccurate statistics expecially THP > > *especially Thanks. > > > vmstat counters. In the systems with hundreads of processors > > *hundreds Thanks. > > > it can be GBs of memory. For example, for a 96 CPUs system, > > the threshold is the maximum number of 125. And the per cpu > > counters can cache 23.4375 GB in total. > > > > The THP page is already a form of batched addition (it will > > add 512 worth of memory in one go) so skipping the batching > > seems like sensible. Although every THP stats update overflows > > the per-cpu counter, resorting to atomic global updates. But > > it can make the statistics more accuracy for the THP vmstat > > counters. > > > > So we convert the NR_ANON_THPS account to pages. This patch > > is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs > > on compound page arrival"). Doing this also can make the unit > > of vmstat counters more unified. Finally, the unit of the vmstat > > counters are pages, kB and bytes. The B/KB suffix can tell us > > that the unit is bytes or kB. The rest which is without suffix > > are pages. > > > > Signed-off-by: Muchun Song > > I agree with the motivation behind this patch but I would like to see > some performance numbers in the commit message. We might agree to pay > the price but at least we will know what exactly that cost is. Do you have any recommendations about benchmarks? I can do a test. Thanks very much. -- Yours, Muchun From mboxrd@z Thu Jan 1 00:00:00 1970 From: Muchun Song Subject: Re: [External] Re: [PATCH v5 2/7] mm: memcontrol: convert NR_ANON_THPS account to pages Date: Thu, 24 Dec 2020 10:45:58 +0800 Message-ID: References: <20201217034356.4708-1-songmuchun@bytedance.com> <20201217034356.4708-3-songmuchun@bytedance.com> Mime-Version: 1.0 Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=fDRfu0M1YLV1uhT7YoKaiH9p9Ye/ITZc1yvaQ4SUQRQ=; b=ZJPDaWbo/DIXrpPoj3kbzFZq3ifzLIDVBRcOUeeDau6bNYnMbfXwq+Q3xjq+SqbDy4 Dim1zSamlWYGDHO7vC5p5k5feoa3IXuVpZaqrm86asSB+klu5vOjmug219Kw/52k8nAo eHOfWtYH2gjhQhErEg4GMr3D5/GaUBAn4XQCkV/btHnMMR6M1AEaIiPQpHv9/jr0V9p5 dPyMkWT6VkJRbkcY3AvB9Eta4zTVQCQmf9DRFylxLwf8Zm98OVmndOXtNERC8HmdkP0F kuhZWGA2K7XU/Gz+RkI2T6VYvzo+paOxH/7ud2cUmqITTJde3TZ5DowlQoAvpRH7yspk 1t5Q== In-Reply-To: List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Shakeel Butt Cc: Greg Kroah-Hartman , "Rafael J. Wysocki" , Alexey Dobriyan , Andrew Morton , Johannes Weiner , Michal Hocko , Vladimir Davydov , Hugh Dickins , Roman Gushchin , Sami Tolvanen , Feng Tang , Neil Brown , Joonsoo Kim , Randy Dunlap , LKML , linux-fsdevel , Linux MM , Cgroups On Thu, Dec 24, 2020 at 6:08 AM Shakeel Butt wrote: > > On Wed, Dec 16, 2020 at 7:45 PM Muchun Song wrote: > > > > Currently we use struct per_cpu_nodestat to cache the vmstat > > counters, which leads to inaccurate statistics expecially THP > > *especially Thanks. > > > vmstat counters. In the systems with hundreads of processors > > *hundreds Thanks. > > > it can be GBs of memory. For example, for a 96 CPUs system, > > the threshold is the maximum number of 125. And the per cpu > > counters can cache 23.4375 GB in total. > > > > The THP page is already a form of batched addition (it will > > add 512 worth of memory in one go) so skipping the batching > > seems like sensible. Although every THP stats update overflows > > the per-cpu counter, resorting to atomic global updates. But > > it can make the statistics more accuracy for the THP vmstat > > counters. > > > > So we convert the NR_ANON_THPS account to pages. This patch > > is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs > > on compound page arrival"). Doing this also can make the unit > > of vmstat counters more unified. Finally, the unit of the vmstat > > counters are pages, kB and bytes. The B/KB suffix can tell us > > that the unit is bytes or kB. The rest which is without suffix > > are pages. > > > > Signed-off-by: Muchun Song > > I agree with the motivation behind this patch but I would like to see > some performance numbers in the commit message. We might agree to pay > the price but at least we will know what exactly that cost is. Do you have any recommendations about benchmarks? I can do a test. Thanks very much. -- Yours, Muchun