From: sspatil@google.com
To: joel@joelfernandes.org, linux-kernel@vger.kernel.org,
adobriyan@gmail.com, akpm@linux-foundation.org,
bgregg@netflix.com, chansen3@cisco.com, dancol@google.com,
fmayer@google.com, joaodias@google.com, joelaf@google.com,
corbet@lwn.net, keescook@chromium.org, kernel-team@android.com,
linux-api@vger.kernel.org, linux-doc@vger.kernel.org,
linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
mhocko@suse.com, rppt@linux.ibm.com, minchan@kernel.org,
namhyung@google.com, guro@fb.com, sfr@canb.auug.org.au,
surenb@google.com, tkjos@google.com, vdavydov.dev@gmail.com,
vbabka@suse.cz, wvw@google.com, sspatil+mutt@google.com
Cc: linux-kernel@vger.kernel.org,
Alexey Dobriyan <adobriyan@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
Brendan Gregg <bgregg@netflix.com>,
Christian Hansen <chansen3@cisco.com>,
dancol@google.com, fmayer@google.com, joaodias@google.com,
joelaf@google.com, Jonathan Corbet <corbet@lwn.net>,
Kees Cook <keescook@chromium.org>,
kernel-team@android.com, linux-api@vger.kernel.org,
linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-mm@kvack.org, Michal Hocko <mhocko@suse.com>,
Mike Rapoport <rppt@linux.ibm.com>,
minchan@kernel.org, namhyung@google.com,
Roman Gushchin <guro@fb.com>,
Stephen Rothwell <sfr@canb.auug.org.au>,
surenb@google.com, tkjos@google.com,
Vladimir Davydov <vdavydov.dev@gmail.com>,
Vlastimil Babka <vbabka@suse.cz>,
wvw@google.com
Subject: Re: [PATCH v3 2/2] doc: Update documentation for page_idle virtual address indexing
Date: Fri, 26 Jul 2019 13:17:10 -0700 [thread overview]
Message-ID: <20190726201710.GA144547@google.com> (raw)
In-Reply-To: <20190726152319.134152-2-joel@joelfernandes.org>
Thanks Joel, just a couple of nits for the doc inline below. Other than that,
Reviewed-by: Sandeep Patil <sspatil@google.com>
I'll plan on making changes to Android to use this instead of the pagemap +
page_idle. I think it will also be considerably faster.
On Fri, Jul 26, 2019 at 11:23:19AM -0400, Joel Fernandes (Google) wrote:
> This patch updates the documentation with the new page_idle tracking
> feature which uses virtual address indexing.
>
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> ---
> .../admin-guide/mm/idle_page_tracking.rst | 43 ++++++++++++++++---
> 1 file changed, 36 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/admin-guide/mm/idle_page_tracking.rst b/Documentation/admin-guide/mm/idle_page_tracking.rst
> index df9394fb39c2..1eeac78c94a7 100644
> --- a/Documentation/admin-guide/mm/idle_page_tracking.rst
> +++ b/Documentation/admin-guide/mm/idle_page_tracking.rst
> @@ -19,10 +19,14 @@ It is enabled by CONFIG_IDLE_PAGE_TRACKING=y.
>
> User API
> ========
> +There are 2 ways to access the idle page tracking API. One uses physical
> +address indexing, another uses a simpler virtual address indexing scheme.
>
> -The idle page tracking API is located at ``/sys/kernel/mm/page_idle``.
> -Currently, it consists of the only read-write file,
> -``/sys/kernel/mm/page_idle/bitmap``.
> +Physical address indexing
> +-------------------------
> +The idle page tracking API for physical address indexing using page frame
> +numbers (PFN) is located at ``/sys/kernel/mm/page_idle``. Currently, it
> +consists of the only read-write file, ``/sys/kernel/mm/page_idle/bitmap``.
>
> The file implements a bitmap where each bit corresponds to a memory page. The
> bitmap is represented by an array of 8-byte integers, and the page at PFN #i is
> @@ -74,6 +78,31 @@ See :ref:`Documentation/admin-guide/mm/pagemap.rst <pagemap>` for more
> information about ``/proc/pid/pagemap``, ``/proc/kpageflags``, and
> ``/proc/kpagecgroup``.
>
> +Virtual address indexing
> +------------------------
> +The idle page tracking API for virtual address indexing using virtual page
> +frame numbers (VFN) is located at ``/proc/<pid>/page_idle``. It is a bitmap
> +that follows the same semantics as ``/sys/kernel/mm/page_idle/bitmap``
> +except that it uses virtual instead of physical frame numbers.
> +
> +This idle page tracking API does not need deal with PFN so it does not require
s/need//
> +prior lookups of ``pagemap`` in order to find if page is idle or not. This is
s/in order to find if page is idle or not//
> +an advantage on some systems where looking up PFN is considered a security
> +issue. Also in some cases, this interface could be slightly more reliable to
> +use than physical address indexing, since in physical address indexing, address
> +space changes can occur between reading the ``pagemap`` and reading the
> +``bitmap``, while in virtual address indexing, the process's ``mmap_sem`` is
> +held for the duration of the access.
> +
> +To estimate the amount of pages that are not used by a workload one should:
> +
> + 1. Mark all the workload's pages as idle by setting corresponding bits in
> + ``/proc/<pid>/page_idle``.
> +
> + 2. Wait until the workload accesses its working set.
> +
> + 3. Read ``/proc/<pid>/page_idle`` and count the number of bits set.
> +
> .. _impl_details:
>
> Implementation Details
> @@ -99,10 +128,10 @@ When a dirty page is written to swap or disk as a result of memory reclaim or
> exceeding the dirty memory limit, it is not marked referenced.
>
> The idle memory tracking feature adds a new page flag, the Idle flag. This flag
> -is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` (see the
> -:ref:`User API <user_api>`
> -section), and cleared automatically whenever a page is referenced as defined
> -above.
> +is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` for physical
> +addressing or by writing to ``/proc/<pid>/page_idle`` for virtual
> +addressing (see the :ref:`User API <user_api>` section), and cleared
> +automatically whenever a page is referenced as defined above.
>
> When a page is marked idle, the Accessed bit must be cleared in all PTEs it is
> mapped to, otherwise we will not be able to detect accesses to the page coming
> --
> 2.22.0.709.g102302147b-goog
>
> --
> To unsubscribe from this group and stop receiving emails from it, send an email to kernel-team+unsubscribe@android.com.
>
next prev parent reply other threads:[~2019-07-26 20:17 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-26 15:23 [PATCH v3 1/2] mm/page_idle: Add per-pid idle page tracking using virtual indexing Joel Fernandes (Google)
2019-07-26 15:23 ` [PATCH v3 2/2] doc: Update documentation for page_idle virtual address indexing Joel Fernandes (Google)
2019-07-26 20:17 ` sspatil [this message]
2019-07-26 20:26 ` Joel Fernandes
2019-07-31 6:44 ` Mike Rapoport
2019-07-30 13:06 ` [PATCH v3 1/2] mm/page_idle: Add per-pid idle page tracking using virtual indexing Joel Fernandes
2019-07-31 8:53 ` Minchan Kim
2019-07-31 17:19 ` Joel Fernandes
2019-08-05 7:55 ` Minchan Kim
2019-08-05 13:10 ` Joel Fernandes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190726201710.GA144547@google.com \
--to=sspatil@google.com \
--cc=adobriyan@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=bgregg@netflix.com \
--cc=chansen3@cisco.com \
--cc=corbet@lwn.net \
--cc=dancol@google.com \
--cc=fmayer@google.com \
--cc=guro@fb.com \
--cc=joaodias@google.com \
--cc=joel@joelfernandes.org \
--cc=joelaf@google.com \
--cc=keescook@chromium.org \
--cc=kernel-team@android.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=minchan@kernel.org \
--cc=namhyung@google.com \
--cc=rppt@linux.ibm.com \
--cc=sfr@canb.auug.org.au \
--cc=sspatil+mutt@google.com \
--cc=surenb@google.com \
--cc=tkjos@google.com \
--cc=vbabka@suse.cz \
--cc=vdavydov.dev@gmail.com \
--cc=wvw@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).