linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Mel Gorman <mgorman@techsingularity.net>
To: Aaron Lu <aaron.lu@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Vlastimil Babka <vbabka@suse.cz>,
	Michal Hocko <mhocko@kernel.org>,
	Jesper Dangaard Brouer <brouer@redhat.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Linux-MM <linux-mm@kvack.org>
Subject: Re: [PATCH 5/6] mm/page_alloc: Free pages in a single pass during bulk free
Date: Fri, 18 Feb 2022 09:20:06 +0000	[thread overview]
Message-ID: <20220218092006.GX3366@techsingularity.net> (raw)
In-Reply-To: <Yg8ec9MLblOkHTY9@ziqianlu-nuc9qn>

On Fri, Feb 18, 2022 at 12:20:03PM +0800, Aaron Lu wrote:
> > The baseline looks fine. It's different to what I used but the page_alloc
> > shouldn't have much impact.
> > 
> > When looking at will-it-scale, please pay attention to lower CPU counts
> > as well and take account changes in standard deviation. Looking at the
> 
> I'll also test nr_task=4/16/64 on the 4sockets CooperLake(nr_cpu=144) then.
> 

Thanks.

> > I expect there will be different good/bad points based on looking at
> > Zen1 results (8 nodes, varying distances, 64 cores with 128 CPUs HT
> > enabled)
> > 
> >                                                     5.17.0-rc3                 5.17.0-rc3                 5.17.0-rc3
> >                                                        vanilla        mm-reverthighpcp-v1           mm-highpcpopt-v2
> > Hmean     page_fault1-threads-2          2985366.46 (   0.00%)      2984649.41 (  -0.02%)      3028407.35 (   1.44%)
> > Hmean     page_fault1-threads-5          3491833.63 (   0.00%)      3500237.35 (   0.24%)      3489971.99 (  -0.05%)
> > Hmean     page_fault1-threads-8          3254335.58 (   0.00%)      3277515.51 *   0.71%*      3234275.28 *  -0.62%*
> > Hmean     page_fault1-threads-12         5101504.72 (   0.00%)      5390649.46 *   5.67%*      5162047.68 (   1.19%)
> > Hmean     page_fault1-threads-21         7714265.64 (   0.00%)      7714763.10 (   0.01%)      7854367.65 *   1.82%*
> > Hmean     page_fault1-threads-30        10034561.94 (   0.00%)      9865446.68 (  -1.69%)      9746368.76 *  -2.87%*
> > Hmean     page_fault1-threads-48        12571351.99 (   0.00%)     13257508.23 *   5.46%*     12160897.07 *  -3.27%*
> > Hmean     page_fault1-threads-79        11124387.46 (   0.00%)     10641145.82 *  -4.34%*     10677656.39 *  -4.02%*
> > Hmean     page_fault1-threads-110       11980424.12 (   0.00%)     10778220.84 * -10.03%*     10354249.62 * -13.57%* <-- close to nr_cpus
> > Hmean     page_fault1-threads-141        9727528.73 (   0.00%)      9966965.70 (   2.46%)      9656148.13 (  -0.73%) <-- close to nr_cpus
> 
> I have never tested thread mode, because I think the heavy loaded
> thread mode is more about testing the mmap_sem contention than page
> allocator's performance?

You're right, I meant to paste in the processes figures and used
processes for the stddev

Hmean     page_fault1-processes-2        3087765.27 (   0.00%)      3040255.24 *  -1.54%*      3026943.42 *  -1.97%*
Hmean     page_fault1-processes-5        3630079.14 (   0.00%)      3644005.83 *   0.38%*      3641029.26 *   0.30%*
Hmean     page_fault1-processes-8        3435519.22 (   0.00%)      3440525.39 *   0.15%*      3430091.10 *  -0.16%*
Hmean     page_fault1-processes-12       7060647.54 (   0.00%)      7078730.32 *   0.26%*      7066516.90 (   0.08%)
Hmean     page_fault1-processes-21      10529603.15 (   0.00%)     10543342.71 *   0.13%*     10529619.72 (   0.00%)
Hmean     page_fault1-processes-30      13919518.76 (   0.00%)     13916089.66 (  -0.02%)     13911735.60 *  -0.06%*
Hmean     page_fault1-processes-48      20655910.65 (   0.00%)     20680704.25 *   0.12%*     20634196.53 *  -0.11%*
Hmean     page_fault1-processes-79      27154979.79 (   0.00%)     27200579.85 *   0.17%*     27111810.79 *  -0.16%*
Hmean     page_fault1-processes-110     26456190.23 (   0.00%)     26498119.30 *   0.16%*     26414120.14 *  -0.16%*
Hmean     page_fault1-processes-141     25741741.47 (   0.00%)     25377519.19 (  -1.41%)     26020885.64 (   1.08%)
Hmean     page_fault1-processes-172     26029813.28 (   0.00%)     26107861.43 *   0.30%*     26011987.83 *  -0.07%*
Hmean     page_fault1-processes-203     26005230.37 (   0.00%)     26114882.22 *   0.42%*     25999181.70 (  -0.02%)
Hmean     page_fault1-processes-234     26021903.34 (   0.00%)     26123727.47 *   0.39%*     26000412.62 *  -0.08%*
Hmean     page_fault1-processes-265     26019386.67 (   0.00%)     26139301.80 *   0.46%*     26014073.54 (  -0.02%)
Hmean     page_fault1-processes-296     26014579.15 (   0.00%)     26101018.62 *   0.33%*     26009459.16 (  -0.02%)
Hmean     page_fault1-processes-327     26059483.56 (   0.00%)     26279026.62 (   0.84%)     25990821.88 (  -0.26%)
Hmean     page_fault1-processes-358     19604338.34 (   0.00%)     26115341.28 *  33.21%*     25995281.86 *  32.60%*
Hmean     page_fault1-processes-389     26084730.88 (   0.00%)     26058850.78 (  -0.10%)     26007661.51 *  -0.30%*
Hmean     page_fault1-processes-420     25358929.58 (   0.00%)     25097140.75 (  -1.03%)     26005923.68 (   2.55%)
Hmean     page_fault1-processes-451     26172808.51 (   0.00%)     26439611.24 *   1.02%*     26078355.47 (  -0.36%)
Hmean     page_fault1-processes-482     26848297.49 (   0.00%)     26722385.24 (  -0.47%)     26171033.04 *  -2.52%*

> It's surprising this patch caused a
> performance change.
> 

The figures say it meakes little difference. I wasn't really
concentrating on will-it-scale-pf as such when writing the patch. I
included pf because it was the original justification for deferring
the zone lock acquisition until after pages had been taken off the PCP.

> > Hmean     page_fault1-threads-234       11322381.78 (   0.00%)      9163162.66 ( -19.07%)      9141561.16 ( -19.26%)
> > Hmean     page_fault1-threads-265        7956982.52 (   0.00%)      7774650.20 (  -2.29%)      8292405.57 *   4.22%*
> > Hmean     page_fault1-threads-296        7892153.88 (   0.00%)      8272671.84 *   4.82%*      7907026.20 (   0.19%)
> > Hmean     page_fault1-threads-327        7957124.50 (   0.00%)      8078297.34 (   1.52%)      8129776.79 (   2.17%)
> > Hmean     page_fault1-threads-358        7847563.90 (   0.00%)      8202303.36 (   4.52%)      8139027.38 (   3.71%)
> > Hmean     page_fault1-threads-389        7928386.47 (   0.00%)      8104732.41 (   2.22%)      8022002.73 (   1.18%)
> > Hmean     page_fault1-threads-420        7690107.89 (   0.00%)      7587821.54 (  -1.33%)      7783777.95 (   1.22%)
> > Hmean     page_fault1-threads-451        7683132.29 (   0.00%)      7979578.21 (   3.86%)      7693067.13 (   0.13%)
> > Hmean     page_fault1-threads-482        7720646.31 (   0.00%)      7597453.65 (  -1.60%)      7870063.90 (   1.94%)
> > Hmean     page_fault1-threads-512        7353458.45 (   0.00%)      7584407.14 (   3.14%)      8119539.24 (  10.42%)
> > Stddev    page_fault1-processes-2           4086.39 (   0.00%)         1698.11 (  58.44%)         1488.13 (  63.58%)
> > Stddev    page_fault1-processes-5           1448.69 (   0.00%)         1616.59 ( -11.59%)         1567.37 (  -8.19%)
> > Stddev    page_fault1-processes-8           1828.29 (   0.00%)         2628.59 ( -43.77%)         2701.96 ( -47.79%)
> > Stddev    page_fault1-processes-12         14073.12 (   0.00%)         1575.18 (  88.81%)         4880.93 (  65.32%)
> > Stddev    page_fault1-processes-21          4368.35 (   0.00%)         7865.27 ( -80.05%)         3778.03 (  13.51%)
> > Stddev    page_fault1-processes-30          5348.13 (   0.00%)        11751.43 (-119.73%)         3240.22 (  39.41%)
> > Stddev    page_fault1-processes-48         23687.16 (   0.00%)         7803.01 (  67.06%)         2635.85 (  88.87%)
> > Stddev    page_fault1-processes-79         12779.16 (   0.00%)         4311.60 (  66.26%)        22539.03 ( -76.37%)
> > Stddev    page_fault1-processes-110        21031.04 (   0.00%)        15115.36 (  28.13%)        12136.54 (  42.29%)
> > Stddev    page_fault1-processes-141       589804.99 (   0.00%)      1335519.71 (-126.43%)        19560.01 (  96.68%)
> > Stddev    page_fault1-processes-172         7033.94 (   0.00%)         7147.71 (  -1.62%)        11366.64 ( -61.60%)
> > Stddev    page_fault1-processes-203         6322.20 (   0.00%)         5035.55 (  20.35%)         4043.45 (  36.04%)
> > Stddev    page_fault1-processes-234        12046.53 (   0.00%)        24208.37 (-100.96%)         9159.91 (  23.96%)
> > Stddev    page_fault1-processes-265        11869.43 (   0.00%)        13528.26 ( -13.98%)         8943.99 (  24.65%)
> > Stddev    page_fault1-processes-296         8918.50 (   0.00%)        16130.54 ( -80.87%)         5211.80 (  41.56%)
> > Stddev    page_fault1-processes-327       101102.64 (   0.00%)       845864.70 (-736.64%)        16238.99 (  83.94%)
> > Stddev    page_fault1-processes-358      2102190.38 (   0.00%)        11316.00 (  99.46%)         7508.57 (  99.64%)
> > Stddev    page_fault1-processes-389        61012.79 (   0.00%)       121446.55 ( -99.05%)        18279.64 (  70.04%)
> > Stddev    page_fault1-processes-420      2305208.40 (   0.00%)      2347564.71 (  -1.84%)         3202.77 (  99.86%)
> > Stddev    page_fault1-processes-451        20214.37 (   0.00%)       173800.17 (-759.79%)       492258.35 (-2335.19%)
> > Stddev    page_fault1-processes-482       236881.21 (   0.00%)       330501.32 ( -39.52%)        15307.31 (  93.54%)
> > Stddev    page_fault1-processes-512       201354.82 (   0.00%)       207019.93 (  -2.81%)      4900536.90 (-2333.78%)
> > 
> > This is showing there was a impact around the nr_cpus (110 and 141
> > processes measured) but the standard deviation around 141 was particularly
>   ~~~~~~~~~
> 
>   Did you mean threads?
> 

I meant processes both times and based the reasoning on processes and
pasted the wrong thing. I'm going to split this config into threads
versions and processes versions because they measure different things
and considering them together in the context of the same test is hazardous.

> > If possible, it would be nice if you could add something like
> > configs/config-io-trunc from mmtests to lkp if it doesn't exist already
> > to consider the simple case. As its most basic, all it's doing is
> > 
> > ---8<---
> > #!/bin/bash
> > 
> > for i in {1..10}; do
> >         dd if=/dev/zero of=sparse_file-$i bs=1 count=0 seek=1G &>/dev/null
> >         cat sparse_file-$i > /dev/null
> > done
> > sync
> > 
> > # Primary metric
> > time rm sparse_file*
> > ---8<---
> > 
> > The main difference is that the mmtests will report the time to fault the
> > sparse files (bulk simple allocate inserting into page cache) as well as
> > the bulk truncate (bulk simple release of page cache).
> 
> Thanks for the suggestion.
> 
> vm-scalability has a similar test called case-truncate which LKP already uses:
> https://git.kernel.org/pub/scm/linux/kernel/git/wfg/vm-scalability.git/tree/case-truncate
> except in case-truncate, the rm is done concurrently and only the
> truncate time is reported.

This is still a valid test except you may also be measuring LRU lock
contention so it'll be less clear for evaluating this series unless the
scale factor is 1.

> I'll modify the case to make it do the rm in
> sequential mode and also report the fault time.
> 

Thanks.

-- 
Mel Gorman
SUSE Labs

  reply	other threads:[~2022-02-18  9:20 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-02-17  0:22 [PATCH v2 0/6] Follow-up on high-order PCP caching Mel Gorman
2022-02-17  0:22 ` [PATCH 1/6] mm/page_alloc: Fetch the correct pcp buddy during bulk free Mel Gorman
2022-02-17  1:43   ` Aaron Lu
2022-02-17  0:22 ` [PATCH 2/6] mm/page_alloc: Track range of active PCP lists " Mel Gorman
2022-02-17  9:41   ` Vlastimil Babka
2022-02-17  0:22 ` [PATCH 3/6] mm/page_alloc: Simplify how many pages are selected per pcp list " Mel Gorman
2022-02-17  0:22 ` [PATCH 4/6] mm/page_alloc: Drain the requested list first " Mel Gorman
2022-02-17  9:42   ` Vlastimil Babka
2022-02-17  0:22 ` [PATCH 5/6] mm/page_alloc: Free pages in a single pass " Mel Gorman
2022-02-17  1:53   ` Aaron Lu
2022-02-17  8:49     ` Aaron Lu
2022-02-17  9:31     ` Mel Gorman
2022-02-18  4:20       ` Aaron Lu
2022-02-18  9:20         ` Mel Gorman [this message]
2022-02-21 13:38         ` Aaron Lu
2022-02-23 11:30           ` Aaron Lu
2022-02-23 13:05             ` Mel Gorman
2022-02-24  1:34               ` Lu, Aaron
2022-02-18  6:07   ` Aaron Lu
2022-02-18  9:47     ` Mel Gorman
2022-02-18 12:13       ` Aaron Lu
2022-02-17  0:22 ` [PATCH 6/6] mm/page_alloc: Limit number of high-order pages on PCP " Mel Gorman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220218092006.GX3366@techsingularity.net \
    --to=mgorman@techsingularity.net \
    --cc=aaron.lu@intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=brouer@redhat.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@kernel.org \
    --cc=vbabka@suse.cz \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).