From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIM_ADSP_ALL,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF2BAC54E8B for ; Tue, 12 May 2020 11:58:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8FFAC206CC for ; Tue, 12 May 2020 11:58:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="JRxw6J3w" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8FFAC206CC Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2BCCC9000B0; Tue, 12 May 2020 07:58:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 29368900036; Tue, 12 May 2020 07:58:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A9439000B0; Tue, 12 May 2020 07:58:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0126.hostedemail.com [216.40.44.126]) by kanga.kvack.org (Postfix) with ESMTP id 02B2A900036 for ; Tue, 12 May 2020 07:58:07 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6C17F181AC9B6 for ; Tue, 12 May 2020 11:58:07 +0000 (UTC) X-FDA: 76807918614.21.trade81_63f6d39a05b52 X-HE-Tag: trade81_63f6d39a05b52 X-Filterd-Recvd-Size: 9212 Received: from smtp-fw-9101.amazon.com (smtp-fw-9101.amazon.com [207.171.184.25]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Tue, 12 May 2020 11:58:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1589284688; x=1620820688; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fVTDlJRsj7WFzWh2TofOuuLmKGmvfqIkitYBNw6RneU=; b=JRxw6J3wr7BhVSitstY2GKVzH6yaPddpC2C0GTKPGJflSMHX9+E0hyZT LXSstc+RZDHbcsT/uyNAvrxdM9zHmYJHn3ncLX7bWbH2RgB6lrIhyBDc9 ckyzUlQbi3B8Cj7KV597T9qZdEvu4TKCZaMmURl0yljDILX5P33temVAj 4=; IronPort-SDR: +rFfzpSQ9silIKThJIsIokiEyFGt29J10p61d5JbNIi06DGUbgqZXFZ1kzDEtQlrcBNxSv6nzx 2A3YzyMaD50A== X-IronPort-AV: E=Sophos;i="5.73,383,1583193600"; d="scan'208";a="34490141" Received: from sea32-co-svc-lb4-vlan2.sea.corp.amazon.com (HELO email-inbound-relay-2b-8cc5d68b.us-west-2.amazon.com) ([10.47.23.34]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP; 12 May 2020 11:58:07 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan3.pdx.amazon.com [10.170.41.166]) by email-inbound-relay-2b-8cc5d68b.us-west-2.amazon.com (Postfix) with ESMTPS id E87EBA1C12; Tue, 12 May 2020 11:58:03 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 12 May 2020 11:58:03 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.160.100) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 12 May 2020 11:57:48 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC v8 8/8] Documentation/admin-guide/mm: Document DAMON-based operation schemes Date: Tue, 12 May 2020 13:53:43 +0200 Message-ID: <20200512115343.27699-9-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200512115343.27699-1-sjpark@amazon.com> References: <20200512115343.27699-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Originating-IP: [10.43.160.100] X-ClientProxiedBy: EX13D17UWB003.ant.amazon.com (10.43.161.42) To EX13D31EUA001.ant.amazon.com (10.43.165.15) Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit documents DAMON-based operation schemes in the DAMON document. Signed-off-by: SeongJae Park --- .../admin-guide/mm/data_access_monitor.rst | 100 +++++++++++++++++- 1 file changed, 98 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/mm/data_access_monitor.rst b/Docum= entation/admin-guide/mm/data_access_monitor.rst index 915956aa1065..d4a48bc63400 100644 --- a/Documentation/admin-guide/mm/data_access_monitor.rst +++ b/Documentation/admin-guide/mm/data_access_monitor.rst @@ -182,8 +182,8 @@ only for each of a user-specified time interval (``re= gions update interval``). ``debugfs`` Interface =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor= _on`` -under its debugfs directory, ``/damon/``. +DAMON exports five files, ``attrs``, ``pids``, ``record``, ``schemes`` a= nd +``monitor_on`` under its debugfs directory, ``/damon/``. =20 Attributes ---------- @@ -227,6 +227,46 @@ be 4 KiB and the result to be saved in ``/damon.data= ``:: # cat record 4096 /damon.data =20 +Schemes +------- + +For usual DAMON-based data access awared memory management optimizations= , users +would simply want the system to apply a memory management action to a me= mory +region of a specific size having a specific access frequency for a speci= fic +time. DAMON receives such formalized operation schemes from user and ap= plies +those to the target processes. + +Users can get and set the schemes by reading from and writing to ``schem= es`` +debugfs file. To the file, each of the schemes should represented in ea= ch line +in below form: + + min-size max-size min-acc max-acc min-age max-age action + +Bytes for size of regions (``min-size`` and ``max-size``), number of mon= itored +accesses per aggregate interval for access frequency (``min-acc`` and +``max-acc``), number of aggregate intervals for age of regions (``min-ag= e`` and +``max-age``), and predefined integer for memory management actions shoul= d be +used. ``madvise()`` system call with specific hint are currently availa= ble. +The numbers and their representing memory hint are as below:: + + 0 MADV_WILLNEED + 1 MADV_COLD + 2 MADV_PAGEOUT + 3 MADV_HUGEPAGE + 4 MADV_NOHUGEPAGE + +You can disable schemes by simply writing empty string to the file. For +example, below commands applies a scheme saying =E2=80=9CIf a memory reg= ion larger than +4 KiB (4096 0) is showing less than 5 accesses per aggregate interval (0= 5) for +more than 5 aggregate interval (5 0), page out the region (2)=E2=80=9D, = check the +entered scheme again, and finally remove the scheme.:: + + # cd /damon + # echo "4096 0 0 5 5 0 2" > schemes + # cat schemes + 4096 0 0 5 5 0 2 + # echo > schemes + Turning On/Off -------------- =20 @@ -426,3 +466,59 @@ made. =20 Users can specify the resolution of the distribution (``--range``). It = also supports 'gnuplot' based simple visualization (``--plot``) of the distri= bution. + + +DAMON-based Operation Schemes +----------------------------- + +The ``schemes`` subcommand applies given data access pattern based opera= tion +schemes to the given target processes. The target processes are describ= ed +using the command to spawn the processes or pid of running processes, as +similar to that of ``record`` subcommand. Meanwhile, the operation sche= mes +should be saved in a text file using below format and passed to ``scheme= s`` +subcommand via ``--schemes`` option. + + min-size max-size min-acc max-acc min-age max-age action + +The format also supports comments, several units for size and age of reg= ions, +and human readable action names. Currently supported operation actions = are +WILLNEED, COLD, PAGEOUT, HUGEPAGE, and NOHUGEPAGE. Each of the actions = works +as same to that of madvise() system call. Below is an example schemes f= ile. +Please also note that 0 for max values means infinite.:: + + # format is: + # + # + # B/K/M/G/T for Bytes/KiB/MiB/GiB/TiB + # us/ms/s/m/h/d for micro-seconds/milli-seconds/seconds/minutes/hour= s/days + # 'null' means zero for size and age. + + # if a region keeps a high access frequency for more than 100ms, put= the + # region on the head of the LRU list (call madvise() with MADV_WILLN= EED). + null null 80 null 100ms 0s willneed + + # if a region keeps a low access frequency for more than 200ms and l= ess + # than one hour, put the # region on the tail of the LRU list (call + # madvise() with MADV_COLD). + 0B 0B 10 20 200ms 1h cold + + # if a region keeps a very low access frequency for more than 1 minu= te, + # swap out the region immediately (call madvise() with MADV_PAGEOUT)= . + 0B null 0 10 60s 0s pageout + + # if a region of a size bigger than 2MiB keeps a very high access fr= equency + # for more than 100ms, let the region to use huge pages (call madvis= e() + # with MADV_HUGEPAGE). + 2M null 90 99 100ms 0s hugepage + + # If a regions of a size bigger than 2MiB keeps small access frequen= cy for + # more than 100ms, avoid the region using huge pages (call madvise()= with + # MADV_NOHUGEPAGE). + 2M null 0 25 100ms 0s nohugepage + +For example, you can make a running process named 'foo' to use THP for m= emory +regions keeping 2MB or larger size and having very high access frequency= for +more than 100 milliseconds using below commands:: + + $ echo "2M null 90 99 100ms 0s hugepage" > my_thp_scheme + $ ./damo schemes --schemes my_thp_scheme `pidof foo` --=20 2.17.1