From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC488C4741F for ; Fri, 25 Sep 2020 15:00:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1E4E522B2D for ; Fri, 25 Sep 2020 15:00:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="V9vpaNbX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E4E522B2D Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 95E2F900009; Fri, 25 Sep 2020 11:00:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 90E98900006; Fri, 25 Sep 2020 11:00:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D521900009; Fri, 25 Sep 2020 11:00:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0047.hostedemail.com [216.40.44.47]) by kanga.kvack.org (Postfix) with ESMTP id 610D5900006 for ; Fri, 25 Sep 2020 11:00:15 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1790D180AD807 for ; Fri, 25 Sep 2020 15:00:15 +0000 (UTC) X-FDA: 77301894390.11.pest13_4f0842427168 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id E8301180F8B80 for ; Fri, 25 Sep 2020 15:00:14 +0000 (UTC) X-HE-Tag: pest13_4f0842427168 X-Filterd-Recvd-Size: 6751 Received: from smtp-fw-9101.amazon.com (smtp-fw-9101.amazon.com [207.171.184.25]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Fri, 25 Sep 2020 15:00:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1601046015; x=1632582015; h=from:to:cc:subject:date:message-id:in-reply-to: mime-version; bh=1mQoI7SbEAT3GqLVx99dh/bFDFrq0tO1gEtKJuIRcU0=; b=V9vpaNbXzOqZ96ABzvxK3yOPT8zelfiQeqmbNFVBLklHWrw0YHXm4mRb onUso7sjw3w4Wid7uxvrKy4nNI+Q835Gjt4WANxUsTDtB8yY6y/cRSFmD Q5yKEd03cH28s8/vtMnBQLY5Sl9+Jc1D+qSstvGeczfzWvq/VbHzYxTln w=; X-IronPort-AV: E=Sophos;i="5.77,302,1596499200"; d="scan'208";a="71129605" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2b-c300ac87.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP; 25 Sep 2020 15:00:04 +0000 Received: from EX13D31EUA001.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2b-c300ac87.us-west-2.amazon.com (Postfix) with ESMTPS id 04B19A0621; Fri, 25 Sep 2020 14:59:52 +0000 (UTC) Received: from u3f2cd687b01c55.ant.amazon.com (10.43.162.221) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 25 Sep 2020 14:59:35 +0000 From: SeongJae Park To: SeongJae Park CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: Re: [PATCH v20 00/15] Introduce Data Access MONitor (DAMON) Date: Fri, 25 Sep 2020 16:59:19 +0200 Message-ID: <20200925145919.18515-1-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200831112235.2675-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.43.162.221] X-ClientProxiedBy: EX13D30UWC004.ant.amazon.com (10.43.162.4) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, 31 Aug 2020 13:22:35 +0200 SeongJae Park wrote: > On Thu, 20 Aug 2020 09:27:38 +0200 SeongJae Park wrote: > > > On Mon, 17 Aug 2020 12:51:22 +0200 SeongJae Park wrote: > > > > > From: SeongJae Park > > > [...] > > > Introduction > > > ============ > > > > > > DAMON is a data access monitoring framework subsystem for the Linux kernel. > > > The core mechanisms of DAMON called 'region based sampling' and 'adaptive > > > regions adjustment' (refer to 'mechanisms.rst' in the 11th patch of this > > > patchset for the detail) make it > > > > > > - accurate (The monitored information is useful for DRAM level memory > > > management. It might not appropriate for Cache-level accuracy, though.), > > > - light-weight (The monitoring overhead is low enough to be applied online > > > while making no impact on the performance of the target workloads.), and > > > - scalable (the upper-bound of the instrumentation overhead is controllable > > > regardless of the size of target workloads.). > > > > > > Using this framework, therefore, the kernel's core memory management mechanisms > > > such as reclamation and THP can be optimized for better memory management. The > > > experimental memory management optimization works that incurring high > > > instrumentation overhead will be able to have another try. In user space, > > > meanwhile, users who have some special workloads will be able to write > > > personalized tools or applications for deeper understanding and specialized > > > optimizations of their systems. > > > > DAMON will be presented in the next week LPC[1]. To be prepared for a screen > > sharing error (if I get no such error, I will do a live-demo), I recorded a > > simple demo video. I would like to share it here to help your easier > > understanding of DAMON. > > > > https://youtu.be/l63eqbVBZRY > > > > [1] https://linuxplumbersconf.org/event/7/contributions/659/ > > During the session, I introduced the list of future works and asked the > audiences to vote for the priority of the tasks: > https://youtu.be/jOBkKMA0uF0?t=13253 I also promised to make my automated tests for DAMON available as open source. I'm happy to announce that it is not available at Github[1] under GPL v2 license. Using that, you can easily test how well DAMON works on your machine. Hopefully, it could be used as a getting started guide for both users and developers of DAMON. [1] https://github.com/awslabs/damon-tests Thanks, SeongJae Park > > To summarize here, the tasks are (highest priority first): > > 1. Make current DAMON patchset series merged in the mainline (6 votes) > 2. User space interface improvement (4 votes) > - Multiple monitoring contexts > - Charging of the monitoring threads' CPU usage > 3. Support more address spaces (2 votes) > - Cgroups, cached pages, specific file-backed pages, swap slots, ... > 3. DAMON-based MM optimizations (2 votes) > - Page reclaim, THP, compaction, NUMA balancing, ... > 4. Optimize for special use-cases (1 vote) > - Page granularity monitoring, accessed-or-not monitoring, ... > > So, I'd like to focus on polishing current patchset so that it could be merged > in. For that, I'd like to ask your more reviews. > > While waiting for the reviews, I will start implementing other future features > that received many votes. The support of multiple monitoring contexts for the > user space would be the first one. Once the implementation is finished, I will > post it as separated RFC patchset (the user space interface will be compatible > with current one). > > Any comment is welcome. > > > Thanks, > SeongJae Park