From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00,DKIM_ADSP_ALL, DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7651FC4363C for ; Wed, 7 Oct 2020 07:24:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B98F7207EA for ; Wed, 7 Oct 2020 07:24:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="YCHV0Igc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B98F7207EA Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4A540900003; Wed, 7 Oct 2020 03:24:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4576B6B0062; Wed, 7 Oct 2020 03:24:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 31EC3900003; Wed, 7 Oct 2020 03:24:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0044.hostedemail.com [216.40.44.44]) by kanga.kvack.org (Postfix) with ESMTP id 05EB26B005C for ; Wed, 7 Oct 2020 03:24:07 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A5ACF3630 for ; Wed, 7 Oct 2020 07:24:07 +0000 (UTC) X-FDA: 77344290534.29.wood09_0a17d4d271cd Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id C006A180385D3 for ; Wed, 7 Oct 2020 07:18:15 +0000 (UTC) X-HE-Tag: wood09_0a17d4d271cd X-Filterd-Recvd-Size: 12224 Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com [207.171.184.29]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Wed, 7 Oct 2020 07:18:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1602055095; x=1633591095; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qGKT1F1qVkIMFajZCvJm8BTClvxWeUPf3j13t8snc08=; b=YCHV0IgcRt8wzCU077pvNXes1HKkdZ7f0Y3vZPMCE+m8vxHaoW1k6ooe 2CDOjs7xeBVvs+N5wpUZbrAMDqHFHVxPRwLggqmiotISjNJBOuT3mtghJ aE6mDGPzdKFFeBpE1RmpmvGOZnAaOXk2D5zIg48ungG/MxSbbN5iEJGi2 0=; X-IronPort-AV: E=Sophos;i="5.77,345,1596499200"; d="scan'208";a="82208862" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1d-98acfc19.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 07 Oct 2020 07:18:11 +0000 Received: from EX13D31EUA004.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan2.iad.amazon.com [10.40.163.34]) by email-inbound-relay-1d-98acfc19.us-east-1.amazon.com (Postfix) with ESMTPS id 6DA43A1E6F; Wed, 7 Oct 2020 07:17:59 +0000 (UTC) Received: from u3f2cd687b01c55.ant.amazon.com (10.43.162.73) by EX13D31EUA004.ant.amazon.com (10.43.165.161) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 7 Oct 2020 07:17:43 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC v9 10/10] Docs/DAMON: Document physical memory monitoring support Date: Wed, 7 Oct 2020 09:14:09 +0200 Message-ID: <20201007071409.12174-11-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201007071409.12174-1-sjpark@amazon.com> References: <20201007071409.12174-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Originating-IP: [10.43.162.73] X-ClientProxiedBy: EX13D31UWA003.ant.amazon.com (10.43.160.130) To EX13D31EUA004.ant.amazon.com (10.43.165.161) Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates the DAMON documents for the physical memory monitoring support. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 42 ++++++++++++++++---- Documentation/vm/damon/design.rst | 29 +++++++++----- Documentation/vm/damon/faq.rst | 5 +-- 3 files changed, 54 insertions(+), 22 deletions(-) diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation= /admin-guide/mm/damon/usage.rst index cf0d44ce0ac9..3e2f1519c96a 100644 --- a/Documentation/admin-guide/mm/damon/usage.rst +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -10,15 +10,16 @@ DAMON provides below three interfaces for different u= sers. This is for privileged people such as system administrators who want a just-working human-friendly interface. Using this, users can use the = DAMON=E2=80=99s major features in a human-friendly way. It may not be highly tuned fo= r - special cases, though. It supports only virtual address spaces monito= ring. + special cases, though. It supports both virtual and physical address = spaces + monitoring. - *debugfs interface.* This is for privileged user space programmers who want more optimized = use of DAMON. Using this, users can use DAMON=E2=80=99s major features by re= ading from and writing to special debugfs files. Therefore, you can write a= nd use your personalized DAMON debugfs wrapper programs that reads/writes the debugfs files instead of you. The DAMON user space tool is also a ref= erence - implementation of such programs. It supports only virtual address spa= ces - monitoring. + implementation of such programs. It supports both virtual and physica= l + address spaces monitoring. - *Kernel Space Programming Interface.* This is for kernel space programmers. Using this, users can utilize e= very feature of DAMON most flexibly and efficiently by writing kernel space @@ -49,8 +50,10 @@ Recording Data Access Pattern =20 The ``record`` subcommand records the data access pattern of target work= loads in a file (``./damon.data`` by default). You can specify the target wit= h 1) -the command for execution of the monitoring target process, or 2) pid of -running target process. Below example shows a command target usage:: +the command for execution of the monitoring target process, 2) pid of ru= nning +target process, or 3) the special keyword, 'paddr', if you want to monit= or the +system's physical memory address space. Below example shows a command t= arget +usage:: =20 # cd /tools/damon/ # damo record "sleep 5" @@ -61,6 +64,15 @@ of the process. Below example shows a pid target usag= e:: # sleep 5 & # damo record `pidof sleep` =20 +Finally, below example shows the use of the special keyword, 'paddr':: + + # damo record paddr + +In this case, the monitoring target regions defaults to the largetst 'Sy= stem +RAM' region specified in '/proc/iomem' file. Note that the initial moni= toring +target region is maintained rather than dynamically updated like the vir= tual +memory address spaces monitoring case. + The location of the recorded file can be explicitly set using ``-o`` opt= ion. You can further tune this by setting the monitoring attributes. To know= about the monitoring attributes in detail, please refer to the @@ -319,20 +331,34 @@ check it again:: # cat target_ids 42 4242 =20 +Users can also monitor the physical memory address space of the system b= y +writing a special keyword, "``paddr\n``" to the file. Because physical = address +space monitoring doesn't support multiple targets, reading the file will= show a +fake value, ``42``, as below:: + + # cd /damon + # echo paddr > target_ids + # cat target_ids + 42 + Note that setting the target ids doesn't start the monitoring. =20 =20 Initial Monitoring Target Regions --------------------------------- =20 -In case of the debugfs based monitoring, DAMON automatically sets and up= dates -the monitoring target regions so that entire memory mappings of target -processes can be covered. However, users might want to limit the monitor= ing +In case of the virtual address space monitoring, DAMON automatically set= s and +updates the monitoring target regions so that entire memory mappings of = target +processes can be covered. However, users might want to limit the monito= ring region to specific address ranges, such as the heap, the stack, or speci= fic file-mapped area. Or, some users might know the initial access pattern = of their workloads and therefore want to set optimal initial regions for th= e 'adaptive regions adjustment'. =20 +In contrast, DAMON do not automatically sets and updates the monitoring = target +regions in case of physical memory monitoring. Therefore, users should = set the +monitoring target regions by themselves. + In such cases, users can explicitly set the initial monitoring target re= gions as they want, by writing proper values to the ``init_regions`` file. Ea= ch line of the input should represent one region in below form.:: diff --git a/Documentation/vm/damon/design.rst b/Documentation/vm/damon/d= esign.rst index 727d72093f8f..0666e19018fd 100644 --- a/Documentation/vm/damon/design.rst +++ b/Documentation/vm/damon/design.rst @@ -35,27 +35,34 @@ two parts: 1. Identification of the monitoring target address range for the address= space. 2. Access check of specific address range in the target space. =20 -DAMON currently provides the implementation of the primitives for only t= he -virtual address spaces. Below two subsections describe how it works. +DAMON currently provides the implementations of the primitives for the p= hysical +and virtual address spaces. Below two subsections describe how those wor= k. =20 =20 PTE Accessed-bit Based Access Check ----------------------------------- =20 -The implementation for the virtual address space uses PTE Accessed-bit f= or -basic access checks. It finds the relevant PTE Accessed bit from the ad= dress -by walking the page table for the target task of the address. In this w= ay, the -implementation finds and clears the bit for next sampling target address= and -checks whether the bit set again after one sampling period. This could = disturb -other kernel subsystems using the Accessed bits, namely Idle page tracki= ng and -the reclaim logic. To avoid such disturbances, DAMON makes it mutually -exclusive with Idle page tracking and uses ``PG_idle`` and ``PG_young`` = page -flags to solve the conflict with the reclaim logic, as Idle page trackin= g does. +Both of the implementations for physical and virtual address spaces use = PTE +Accessed-bit for basic access checks. Only one difference is the way of +finding the relevant PTE Accessed bit(s) from the address. While the +implementation for the virtual address walks the page table for the targ= et task +of the address, the implementation for the physical address walks every = page +table having a mapping to the address. In this way, the implementations= find +and clear the bit(s) for next sampling target address and checks whether= the +bit(s) set again after one sampling period. This could disturb other ke= rnel +subsystems using the Accessed bits, namely Idle page tracking and the re= claim +logic. To avoid such disturbances, DAMON makes it mutually exclusive wi= th Idle +page tracking and uses ``PG_idle`` and ``PG_young`` page flags to solve = the +conflict with the reclaim logic, as Idle page tracking does. =20 =20 VMA-based Target Address Range Construction ------------------------------------------- =20 +This is only for the virtual address space primitives implementation. T= hat for +the physical address space simply asks users to manually set the monitor= ing +target address ranges. + Only small parts in the super-huge virtual address space of the processe= s are mapped to the physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, because DAMON can deal with = some diff --git a/Documentation/vm/damon/faq.rst b/Documentation/vm/damon/faq.= rst index 088128bbf22b..6469d54c480f 100644 --- a/Documentation/vm/damon/faq.rst +++ b/Documentation/vm/damon/faq.rst @@ -43,10 +43,9 @@ constructions and actual access checks can be implemen= ted and configured on the DAMON core by the users. In this way, DAMON users can monitor any addre= ss space with any access check technique. =20 -Nonetheless, DAMON provides vma tracking and PTE Accessed bit check base= d +Nonetheless, DAMON provides vma/rmap tracking and PTE Accessed bit check= based implementations of the address space dependent functions for the virtual= memory -by default, for a reference and convenient use. In near future, we will -provide those for physical memory address space. +and the physical memory by default, for a reference and convenient use. =20 =20 Can I simply monitor page granularity? --=20 2.17.1