From: Ivan Babrou <ivan@cloudflare.com>
To: Shakeel Butt <shakeelb@google.com>
Cc: "Michal Koutný" <mkoutny@suse.com>,
"Daniel Dao" <dqminh@cloudflare.com>,
kernel-team <kernel-team@cloudflare.com>,
"Linux MM" <linux-mm@kvack.org>,
"Johannes Weiner" <hannes@cmpxchg.org>,
"Roman Gushchin" <guro@fb.com>, "Feng Tang" <feng.tang@intel.com>,
"Michal Hocko" <mhocko@kernel.org>,
"Hillf Danton" <hdanton@sina.com>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Linus Torvalds" <torvalds@linux-foundation.org>
Subject: Re: Regression in workingset_refault latency on 5.15
Date: Wed, 2 Mar 2022 14:33:41 -0800 [thread overview]
Message-ID: <CABWYdi09io3jt4-ZoQffS+JbuKQoD32epx4682_2obY5Ap=mzw@mail.gmail.com> (raw)
In-Reply-To: <CABWYdi1sqSj7tb397FxS1KVURUF-RyduZj2Ubia7Gitocw8hsw@mail.gmail.com>
On Tue, Mar 1, 2022 at 7:40 PM Ivan Babrou <ivan@cloudflare.com> wrote:
>
> On Tue, Mar 1, 2022 at 6:50 PM Shakeel Butt <shakeelb@google.com> wrote:
> >
> > On Tue, Mar 01, 2022 at 04:48:00PM -0800, Ivan Babrou wrote:
> >
> > [...]
> >
> > > Looks like you were right that targeted flush is not going to be as good.
> >
> >
> > Thanks a lot. Can you please try the following patch (independent of other
> > patches) as well?
>
> Sure thing, I'll come back with the numbers tomorrow.
Measurements from different configurations:
* Baseline 5.15.25:
$ sudo /usr/share/bcc/tools/funclatency -d 60 workingset_refault
Tracing 1 functions for "workingset_refault"... Hit Ctrl-C to end.
nsecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 15931 |****************************************|
512 -> 1023 : 7040 |***************** |
1024 -> 2047 : 1566 |*** |
2048 -> 4095 : 3844 |********* |
4096 -> 8191 : 1689 |**** |
8192 -> 16383 : 115 | |
16384 -> 32767 : 54 | |
32768 -> 65535 : 42 | |
65536 -> 131071 : 14 | |
131072 -> 262143 : 4 | |
262144 -> 524287 : 4 | |
524288 -> 1048575 : 92 | |
1048576 -> 2097151 : 2322 |***** |
2097152 -> 4194303 : 8244 |******************** |
4194304 -> 8388607 : 775 |* |
8388608 -> 16777215 : 73 | |
avg = 776487 nsecs, total: 32464181849 nsecs, count: 41809
* Delayed flush:
$ sudo /usr/share/bcc/tools/funclatency -d 60 workingset_refault
Tracing 1 functions for "workingset_refault"... Hit Ctrl-C to end.
nsecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 16271 |****************************************|
512 -> 1023 : 8147 |******************** |
1024 -> 2047 : 2743 |****** |
2048 -> 4095 : 3964 |********* |
4096 -> 8191 : 7137 |***************** |
8192 -> 16383 : 4039 |********* |
16384 -> 32767 : 352 | |
32768 -> 65535 : 214 | |
65536 -> 131071 : 80 | |
131072 -> 262143 : 15 | |
262144 -> 524287 : 0 | |
524288 -> 1048575 : 1 | |
avg = 3248 nsecs, total: 139586555 nsecs, count: 42963
* Targeted flush:
$ sudo /usr/share/bcc/tools/funclatency -d 60 workingset_refault
Tracing 1 functions for "workingset_refault"... Hit Ctrl-C to end.
nsecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 25298 |****************************************|
512 -> 1023 : 13334 |********************* |
1024 -> 2047 : 1519 |** |
2048 -> 4095 : 4265 |****** |
4096 -> 8191 : 2339 |*** |
8192 -> 16383 : 162 | |
16384 -> 32767 : 73 | |
32768 -> 65535 : 70 | |
65536 -> 131071 : 23 | |
131072 -> 262143 : 1 | |
262144 -> 524287 : 3 | |
524288 -> 1048575 : 55 | |
1048576 -> 2097151 : 1409 |** |
2097152 -> 4194303 : 8306 |************* |
4194304 -> 8388607 : 1139 |* |
8388608 -> 16777215 : 89 | |
avg = 589361 nsecs, total: 34233041412 nsecs, count: 5808
* Your most recent patch:
$ sudo /usr/share/bcc/tools/funclatency -d 60 workingset_refault
Tracing 1 functions for "workingset_refault"... Hit Ctrl-C to end.
nsecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 37051 |****************************************|
512 -> 1023 : 17144 |****************** |
1024 -> 2047 : 1051 |* |
2048 -> 4095 : 2705 |** |
4096 -> 8191 : 1911 |** |
8192 -> 16383 : 184 | |
16384 -> 32767 : 93 | |
32768 -> 65535 : 70 | |
65536 -> 131071 : 30 | |
131072 -> 262143 : 21 | |
262144 -> 524287 : 216 | |
524288 -> 1048575 : 1100 |* |
1048576 -> 2097151 : 6522 |******* |
2097152 -> 4194303 : 5903 |****** |
4194304 -> 8388607 : 467 | |
8388608 -> 16777215 : 19 | |
avg = 403261 nsecs, total: 30037729036 nsecs, count: 74487
Looks like there are plenty of longer pauses still.
I can also get the numbers from 5.10 for you if you want.
next prev parent reply other threads:[~2022-03-02 22:33 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-23 13:51 Regression in workingset_refault latency on 5.15 Daniel Dao
2022-02-23 15:57 ` Shakeel Butt
2022-02-23 16:00 ` Shakeel Butt
2022-02-23 17:07 ` Daniel Dao
2022-02-23 17:36 ` Shakeel Butt
2022-02-23 19:28 ` Ivan Babrou
2022-02-23 20:28 ` Shakeel Butt
2022-02-23 21:16 ` Ivan Babrou
2022-02-24 14:46 ` Daniel Dao
2022-02-24 16:58 ` Shakeel Butt
2022-02-24 17:34 ` Daniel Dao
2022-02-24 18:00 ` Shakeel Butt
2022-02-24 18:52 ` Shakeel Butt
2022-02-25 10:23 ` Daniel Dao
2022-02-25 17:08 ` Ivan Babrou
2022-02-25 17:22 ` Shakeel Butt
2022-02-25 18:03 ` Michal Koutný
2022-02-25 18:08 ` Ivan Babrou
2022-02-28 23:09 ` Shakeel Butt
2022-02-28 23:34 ` Ivan Babrou
2022-02-28 23:43 ` Shakeel Butt
2022-03-02 0:48 ` Ivan Babrou
2022-03-02 2:50 ` Shakeel Butt
2022-03-02 3:40 ` Ivan Babrou
2022-03-02 22:33 ` Ivan Babrou [this message]
2022-03-03 2:32 ` Shakeel Butt
2022-03-03 2:35 ` Shakeel Butt
2022-03-04 0:21 ` Ivan Babrou
2022-03-04 1:05 ` Shakeel Butt
2022-03-04 1:12 ` Ivan Babrou
2022-03-02 11:49 ` Frank Hofmann
2022-03-02 15:52 ` Shakeel Butt
2022-03-02 10:08 ` Michal Koutný
2022-03-02 15:53 ` Shakeel Butt
2022-03-02 17:28 ` Ivan Babrou
2022-02-24 9:22 ` Thorsten Leemhuis
2022-04-11 10:17 ` Regression in workingset_refault latency on 5.15 #forregzbot Thorsten Leemhuis
2022-05-16 12:51 ` Thorsten Leemhuis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CABWYdi09io3jt4-ZoQffS+JbuKQoD32epx4682_2obY5Ap=mzw@mail.gmail.com' \
--to=ivan@cloudflare.com \
--cc=akpm@linux-foundation.org \
--cc=dqminh@cloudflare.com \
--cc=feng.tang@intel.com \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=hdanton@sina.com \
--cc=kernel-team@cloudflare.com \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=mkoutny@suse.com \
--cc=shakeelb@google.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.