From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25C8EC5ACD7 for ; Wed, 18 Mar 2020 19:53:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D695920777 for ; Wed, 18 Mar 2020 19:53:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bzGHtNA2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D695920777 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6EF6D6B0082; Wed, 18 Mar 2020 15:53:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 69F186B0083; Wed, 18 Mar 2020 15:53:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B43F6B0085; Wed, 18 Mar 2020 15:53:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0030.hostedemail.com [216.40.44.30]) by kanga.kvack.org (Postfix) with ESMTP id 42E3A6B0082 for ; Wed, 18 Mar 2020 15:53:02 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 076964427 for ; Wed, 18 Mar 2020 19:53:02 +0000 (UTC) X-FDA: 76609531404.20.field56_1a6f22e23e00e X-HE-Tag: field56_1a6f22e23e00e X-Filterd-Recvd-Size: 6715 Received: from mail-oi1-f194.google.com (mail-oi1-f194.google.com [209.85.167.194]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Wed, 18 Mar 2020 19:53:01 +0000 (UTC) Received: by mail-oi1-f194.google.com with SMTP id k18so130968oib.3 for ; Wed, 18 Mar 2020 12:53:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=/O7ydZcGkEnjjR0DKfEZl8dzF7U8chbFAqM/QMwcPsg=; b=bzGHtNA2tGO9KKY0VBVcKXGej5Hl6Fvi3iAb97Ei+QJuQs5DgXJIGoSSh33cEvrT89 jDfI2j9hBYEP2DFVT2bu8Ma/xTvnr1TvkkmDWuNjWz0QQcQcYcDGPxKgATGCybrDKkcO yeLO/j/P4tnAVMAVXZ8z35Ifth6PtGFOQtt8eqLen2JxNtuVI0Q1V4YAATUGlsKML0xC beVvRLkOWokeQiA0NWCyzDue30siSFbMShz0g1EgEG/+rBUh/GenyejbbQzUxIh+H6T0 SLOdsi/iKplRCqLTp48HO8JJ4hedHC91oIKM3XpsH5FHgGoj/+62KkhKLTZeCOM1HlOq pKhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/O7ydZcGkEnjjR0DKfEZl8dzF7U8chbFAqM/QMwcPsg=; b=h1sOx/RXlf1y5VRvJYO1OANS+vQmAFDq1FCEAylW9Z+Is+k+aoAdE7a55fv+9rgKn2 H9XufSbrUPLeBXVslIhjU5BxqeMeKS/BRTHx0AHEJVQSqCvncIrb3XViVVoKFvw+H3XV phSU2LnPNvUBPizqG74ZjFlJdwP3rFK0uobK8C3IaYZDZH40E3W5DRIf3JJ6frUMT+2W JCckdHrzjvgN3WmDF6Me6m96stJmtD+xEDHFdvKBpNpeD+qT1Lzfg+yXwO16ZDOG7KRc 5whirbsz6fMXN2lw6REJ4rSDGpLvQVtA/UHeExPfyfeB9SxEn7BVUo0myyoF/ZiEeJjf 0B5w== X-Gm-Message-State: ANhLgQ0y4IzuO/EBZzmffwjyKl1ockvCXeqV4U8dVIJ3bIB+M2M9pzWM FVL7F4WAvGbUrh44tCr671CYl9/Qx7ip2XU4FS6R5w== X-Google-Smtp-Source: ADFU+vuOPpGMHPfZ/DDnCyvqaCHk4JwFerE7ZzmLuNMC4CG6jDBlIsZV59kQghivA5FHpMPqabGb6KMfq0nTMT39gMc= X-Received: by 2002:aca:af93:: with SMTP id y141mr4368099oie.144.1584561180301; Wed, 18 Mar 2020 12:53:00 -0700 (PDT) MIME-Version: 1.0 References: <20200312100759.20502-1-sjpark@amazon.com> <20200312104345.10032-1-sjpark@amazon.com> In-Reply-To: <20200312104345.10032-1-sjpark@amazon.com> From: Shakeel Butt Date: Wed, 18 Mar 2020 12:52:48 -0700 Message-ID: Subject: Re: Re: Re: [PATCH v6 00/14] Introduce Data Access MONitor (DAMON) To: SeongJae Park Cc: Andrew Morton , SeongJae Park , Andrea Arcangeli , Yang Shi , acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, brendan.d.gregg@gmail.com, Brendan Higgins , Qian Cai , Colin Ian King , Jonathan Corbet , dwmw@amazon.com, jolsa@redhat.com, "Kirill A. Shutemov" , mark.rutland@arm.com, Mel Gorman , Minchan Kim , Ingo Molnar , namhyung@kernel.org, peterz@infradead.org, Randy Dunlap , David Rientjes , Steven Rostedt , shuah@kernel.org, sj38.park@gmail.com, Vlastimil Babka , Vladimir Davydov , Linux MM , linux-doc@vger.kernel.org, LKML Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Mar 12, 2020 at 3:44 AM SeongJae Park wrote: > > On Thu, 12 Mar 2020 11:07:59 +0100 SeongJae Park wrote: > > > On Tue, 10 Mar 2020 10:21:34 -0700 Shakeel Butt wrote: > > > > > On Mon, Feb 24, 2020 at 4:31 AM SeongJae Park wrote: > > > > > > > > From: SeongJae Park > > > > > > > > Introduction > > > > ============ > > > > > [...] > > > > > > I do want to question the actual motivation of the design followed by this work. > > > > > > With the already present Page Idle Tracking feature in the kernel, I > > > can envision that the region sampling and adaptive region adjustments > > > can be done in the user space. Due to sampling, the additional > > > overhead will be very small and configurable. > > > > > > Additionally the proposed mechanism has inherent assumption of the > > > presence of spatial locality (for virtual memory) in the monitored > > > processes which is very workload dependent. > > > > > > Given that the the same mechanism can be implemented in the user space > > > within tolerable overhead and is workload dependent, why it should be > > > done in the kernel? What exactly is the advantage of implementing this > > > in kernel? > > > > First of all, DAMON is not for only user space processes, but also for kernel > > space core mechanisms. Many of the core mechanisms will be able to use DAMON > > for access pattern based optimizations, with light overhead and reasonable > > accuracy. Which kernel space core mechanisms? I can see memory reclaim, do you envision some other component as well. Let's discuss how this can interact with memory reclaim and we can see if there is any benefit to do this in kernel. > > > > Implementing DAMON in user space is of course possible, but it will be > > inefficient. Using it from kernel space would make no sense, and it would > > incur unnecessarily frequent kernel-user context switches, which is very > > expensive nowadays. > > Forgot mentioning about the spatial locality. Yes, it is workload dependant, > but still pervasive in many case. Also, many core mechanisms in kernel such as > read-ahead or LRU are already using some similar assumptions. > Not sure about the LRU but yes read-ahead in several places does assume spatial locality. However most of those are configurable and the userspace can enable/disable the read-ahead based on the workload. > > If it is so problematic, you could set the maximum number of regions to the > number of pages in the system so that each region monitors each page. > How will this work in the process context? Number of regions equal to the number of mapped pages? Basically I am trying to envision the comparison of physical memory based monitoring (using idle page tracking) vs pid+VA based monitoring. Anyways I am not against your proposal. I am trying to see how to make it more general to be applicable to more use-cases and one such use-case which I am interested in is monitoring all the user pages on the system for proactive reclaim purpose. Shakeel