Linux-Block Archive on lore.kernel.org
 help / color / Atom feed
From: Yafang Shao <laoar.shao@gmail.com>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Michal Hocko <mhocko@kernel.org>, Jens Axboe <axboe@kernel.dk>,
	mgorman@suse.de, Steven Rostedt <rostedt@goodmis.org>,
	mingo@redhat.com, Linux MM <linux-mm@kvack.org>,
	linux-block@vger.kernel.org, LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 0/2] psi: enhance psi with the help of ebpf
Date: Fri, 27 Mar 2020 09:17:59 +0800
Message-ID: <CALOAHbCe9msQ+7uON=7iXnud-hzDcrnz_2er4PMQRXtNLM2BSQ@mail.gmail.com> (raw)
In-Reply-To: <20200326143102.GB342070@cmpxchg.org>

On Thu, Mar 26, 2020 at 10:31 PM Johannes Weiner <hannes@cmpxchg.org> wrote:
>
> On Thu, Mar 26, 2020 at 07:12:05AM -0400, Yafang Shao wrote:
> > PSI gives us a powerful way to anaylze memory pressure issue, but we can
> > make it more powerful with the help of tracepoint, kprobe, ebpf and etc.
> > Especially with ebpf we can flexiblely get more details of the memory
> > pressure.
> >
> > In orderc to achieve this goal, a new parameter is added into
> > psi_memstall_{enter, leave}, which indicates the specific type of a
> > memstall. There're totally ten memstalls by now,
> >         MEMSTALL_KSWAPD
> >         MEMSTALL_RECLAIM_DIRECT
> >         MEMSTALL_RECLAIM_MEMCG
> >         MEMSTALL_RECLAIM_HIGH
> >         MEMSTALL_KCOMPACTD
> >         MEMSTALL_COMPACT
> >         MEMSTALL_WORKINGSET_REFAULT
> >         MEMSTALL_WORKINGSET_THRASHING
> >         MEMSTALL_MEMDELAY
> >         MEMSTALL_SWAPIO
>
> What does this provide over the events tracked in /proc/vmstats?
>

/proc/vmstat only tells us which events occured, but it can't tell us
how long these events take.
Sometimes we really want to know how long the event takes and PSI can
provide us the data
For example, in the past days when I did performance tuning for a
database service, I monitored that the latency spike is related with
the workingset_refault counter in /proc/vmstat, and at that time I
really want to know the spread of latencies caused by
workingset_refault, but there's no easy way to get it. Now with newly
added MEMSTALL_WORKINGSET_REFAULT, I can get the latencies caused by
workingset refault.

> Can you elaborate a bit how you are using this information? It's not
> quite clear to me from the example in patch #2.
>

From the traced data in patch #2, we can find that the high latencies
of user tasks are always type 7 of memstall , which is
MEMSTALL_WORKINGSET_THRASHING,  and then we should look into the
details of wokingset of the user tasks and think about how to improve
it - for example, by reducing the workingset.

BTW, there's some error in the definition of show_psi_memstall_type()
in patch #2 ( that's an old version), I will correct it.

To summarize, with the pressure data in /proc/pressure/memroy we know
that the system is under memory pressure, and then with the newly
added tracing facility in this patchset we can get the reason of this
memory pressure, and then thinks about how to make the change.
The workflow can be illustrated as bellow.

                                      REASON        ACTION
                                |    compaction  |  look into the
details of compaction |
Memory pressure -  |    vmscan        |  look into the details of vmscan       |
                                |    workingset   |  look into the
details of workingset   |
                                |     etc              |   ...
                                           |


Thanks

Yafang

  reply index

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-26 11:12 Yafang Shao
2020-03-26 11:12 ` [PATCH 1/2] psi: introduce various types of memstall Yafang Shao
2020-03-26 11:12 ` [PATCH 2/2] psi, tracepoint: introduce tracepoints for psi_memstall_{enter, leave} Yafang Shao
2020-03-26 14:31 ` [PATCH 0/2] psi: enhance psi with the help of ebpf Johannes Weiner
2020-03-27  1:17   ` Yafang Shao [this message]
2020-03-31 15:11     ` Johannes Weiner
2020-04-01  1:22       ` Yafang Shao
2020-04-03 15:48         ` Johannes Weiner
2020-04-04  2:54           ` Yafang Shao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CALOAHbCe9msQ+7uON=7iXnud-hzDcrnz_2er4PMQRXtNLM2BSQ@mail.gmail.com' \
    --to=laoar.shao@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=axboe@kernel.dk \
    --cc=hannes@cmpxchg.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=mhocko@kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Block Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-block/0 linux-block/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-block linux-block/ https://lore.kernel.org/linux-block \
		linux-block@vger.kernel.org
	public-inbox-index linux-block

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-block


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git