From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ACA4C5517A for ; Wed, 11 Nov 2020 16:42:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A11C0207BB for ; Wed, 11 Nov 2020 16:42:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="RTe8IGKR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A11C0207BB Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8BDD16B0036; Wed, 11 Nov 2020 11:42:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 847D66B006C; Wed, 11 Nov 2020 11:42:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70D916B0072; Wed, 11 Nov 2020 11:42:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3C3806B0036 for ; Wed, 11 Nov 2020 11:42:04 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id BCA278249980 for ; Wed, 11 Nov 2020 16:42:03 +0000 (UTC) X-FDA: 77472704526.20.legs10_260390e272ff Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id 9C7BE180C07AB for ; Wed, 11 Nov 2020 16:42:03 +0000 (UTC) X-HE-Tag: legs10_260390e272ff X-Filterd-Recvd-Size: 6197 Received: from smtp-fw-6001.amazon.com (smtp-fw-6001.amazon.com [52.95.48.154]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Nov 2020 16:42:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1605112923; x=1636648923; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=twEFaM6H/18ZRD5aG4A3ebeJ7U/rCfvoC5YljalWKk4=; b=RTe8IGKRCjk9IAa1o1wi5w9IjuLCrFeCVfbhxzffg2vlTzE3msGq3GiD rL2O/kcgBcXP/xEnwgfah9sIaKlcv/V4CdngoMOvgcAYZqLjbwr9htozu D+vVV7VzysSGHBOHIXOhEIYy27qvhhoin5Lgd5Ci9NFRUo4tT8E/FX+df A=; X-IronPort-AV: E=Sophos;i="5.77,470,1596499200"; d="scan'208";a="65959363" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1d-38ae4ad2.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-6001.iad6.amazon.com with ESMTP; 11 Nov 2020 16:41:56 +0000 Received: from EX13D31EUA001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-1d-38ae4ad2.us-east-1.amazon.com (Postfix) with ESMTPS id 63583A24BE; Wed, 11 Nov 2020 16:41:44 +0000 (UTC) Received: from u3f2cd687b01c55.ant.amazon.com (10.43.161.237) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 11 Nov 2020 16:41:27 +0000 From: SeongJae Park To: SeongJae Park CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: Re: [PATCH v22 00/18] Introduce Data Access MONitor (DAMON) Date: Wed, 11 Nov 2020 17:41:13 +0100 Message-ID: <20201111164113.23245-1-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201020085940.13875-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.43.161.237] X-ClientProxiedBy: EX13D38UWB004.ant.amazon.com (10.43.161.30) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, I'd like to remind you that I'm still waiting more reviews. Any comments are welcome. Thanks, SeongJae Park On Tue, 20 Oct 2020 10:59:22 +0200 SeongJae Park wrote: > From: SeongJae Park > > Changes from Previous Version (v21) > =================================== > > This version contains below minor changes. > > - Fix build warnings and errors (kernel test robot) > - Fix a memory leak (kmemleak) > - Respect KUNIT_ALL_TESTS > - Rebase on v5.9 > - Update the evaluation results > > Introduction > ============ > > DAMON is a data access monitoring framework for the Linux kernel. The core > mechanisms of DAMON called 'region based sampling' and 'adaptive regions > adjustment' (refer to 'mechanisms.rst' in the 11th patch of this patchset for > the detail) make it > > - accurate (The monitored information is useful for DRAM level memory > management. It might not appropriate for Cache-level accuracy, though.), > - light-weight (The monitoring overhead is low enough to be applied online > while making no impact on the performance of the target workloads.), and > - scalable (the upper-bound of the instrumentation overhead is controllable > regardless of the size of target workloads.). > > Using this framework, therefore, several memory management mechanisms such as > reclamation and THP can be optimized to aware real data access patterns. > Experimental access pattern aware memory management optimization works that > incurring high instrumentation overhead will be able to have another try. > > Though DAMON is for kernel subsystems, it can be easily exposed to the user > space by writing a DAMON-wrapper kernel subsystem. Then, user space users who > have some special workloads will be able to write personalized tools or > applications for deeper understanding and specialized optimizations of their > systems. > > Evaluations > =========== > > We evaluated DAMON's overhead, monitoring quality and usefulness using 24 > realistic workloads on my QEMU/KVM based virtual machine running a kernel that > v22 DAMON patchset is applied. > > DAMON is lightweight. It increases system memory usage by 0.25% and slows > target workloads down by 0.89%. > > DAMON is accurate and useful for memory management optimizations. An > experimental DAMON-based operation scheme for THP, 'ethp', removes 81.73% of > THP memory overheads while preserving 95.29% of THP speedup. Another > experimental DAMON-based 'proactive reclamation' implementation, 'prcl', > reduces 91.30% of residential sets and 23.45% of system memory footprint while > incurring only 2.08% runtime overhead in the best case (parsec3/freqmine). > > NOTE that the experimentail THP optimization and proactive reclamation are not > for production but only for proof of concepts. > > Please refer to the official document[1] or "Documentation/admin-guide/mm: Add > a document for DAMON" patch in this patchset for detailed evaluation setup and > results. > > [1] https://damonitor.github.io/doc/html/latest-damon/admin-guide/mm/damon/eval.html >