From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E305C3F2D1 for ; Wed, 4 Mar 2020 09:58:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C238A2166E for ; Wed, 4 Mar 2020 09:58:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C238A2166E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 135676B0003; Wed, 4 Mar 2020 04:58:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E8816B0005; Wed, 4 Mar 2020 04:58:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F176C6B0006; Wed, 4 Mar 2020 04:58:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0079.hostedemail.com [216.40.44.79]) by kanga.kvack.org (Postfix) with ESMTP id D7A0D6B0003 for ; Wed, 4 Mar 2020 04:58:05 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 6EE1FAF79 for ; Wed, 4 Mar 2020 09:58:05 +0000 (UTC) X-FDA: 76557228930.11.cloud33_80cca7c1b4554 X-HE-Tag: cloud33_80cca7c1b4554 X-Filterd-Recvd-Size: 7564 Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Wed, 4 Mar 2020 09:58:04 +0000 (UTC) Received: by mail-wm1-f65.google.com with SMTP id p9so1270801wmc.2 for ; Wed, 04 Mar 2020 01:58:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=tGkuJq9p7Ujsv2LZVWZ4XnLUpkAhIf+I1//fbYjfoC4=; b=BCZz4ohtRRY7pw571IqTOo9nbiHdQwfJG9RLLKNOAlb/9EKULHhynUEKTue2CBKSOK vuCC+VR5V6c+i/0q5NLl05oaF/4l2nCI8lsypi2X1w3SjZlPT1K5lfVYtbND62fzc5Gj p167EEPRnv9KGDYcLIx322rr8JUiaKDF1kJY2Ovinm4rHVVzGdETMRH/ydLnwEVIWGTl UWaA/T41+r5O2hCr3nurUZnp6CtUKIOi1sxK4YA8W/nZA4vLMxN3s07nnV19sS7BPecp tduCdeA2WFRhfN//Nh3UNw91kN/aecDdV/V050fA+eTmwHaj6dNJK7P16KQzZ/CWnI+2 lHAw== X-Gm-Message-State: ANhLgQ0oFY+x6C8o9nxpEXIsX7YflPH9NaArojGA7J2n/OrshEmm7JnN GVW7XE0BPl+4qq/phiQwBqE= X-Google-Smtp-Source: ADFU+vuFxdLBbgqv3WPYMXE5mn7YP+bwIMqE1mGl64ybhaFXG9feJd6vOS3+/Qrgylc2g8UEVXyS2g== X-Received: by 2002:a1c:2701:: with SMTP id n1mr531963wmn.180.1583315884003; Wed, 04 Mar 2020 01:58:04 -0800 (PST) Received: from localhost (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id u17sm13594305wrq.74.2020.03.04.01.58.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Mar 2020 01:58:03 -0800 (PST) Date: Wed, 4 Mar 2020 10:58:02 +0100 From: Michal Hocko To: "Huang, Ying" Cc: Mel Gorman , David Hildenbrand , Johannes Weiner , Matthew Wilcox , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vlastimil Babka , Zi Yan , Peter Zijlstra , Dave Hansen , Minchan Kim , Hugh Dickins , Alexander Duyck Subject: Re: [RFC 0/3] mm: Discard lazily freed pages when migrating Message-ID: <20200304095802.GE16139@dhcp22.suse.cz> References: <871rqf850z.fsf@yhuang-dev.intel.com> <20200228094954.GB3772@suse.de> <87h7z76lwf.fsf@yhuang-dev.intel.com> <20200302151607.GC3772@suse.de> <87zhcy5hoj.fsf@yhuang-dev.intel.com> <20200303080945.GX4380@dhcp22.suse.cz> <87o8td4yf9.fsf@yhuang-dev.intel.com> <20200303085805.GB4380@dhcp22.suse.cz> <87ftep4pzy.fsf@yhuang-dev.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87ftep4pzy.fsf@yhuang-dev.intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue 03-03-20 19:49:53, Huang, Ying wrote: > Michal Hocko writes: > > > On Tue 03-03-20 16:47:54, Huang, Ying wrote: > >> Michal Hocko writes: > >> > >> > On Tue 03-03-20 09:51:56, Huang, Ying wrote: > >> >> Mel Gorman writes: > >> >> > On Mon, Mar 02, 2020 at 07:23:12PM +0800, Huang, Ying wrote: > >> >> >> If some applications cannot tolerate the latency incurred by the memory > >> >> >> allocation and zeroing. Then we cannot discard instead of migrate > >> >> >> always. While in some situations, less memory pressure can help. So > >> >> >> it's better to let the administrator and the application choose the > >> >> >> right behavior in the specific situation? > >> >> >> > >> >> > > >> >> > Is there an application you have in mind that benefits from discarding > >> >> > MADV_FREE pages instead of migrating them? > >> >> > > >> >> > Allowing the administrator or application to tune this would be very > >> >> > problematic. An application would require an update to the system call > >> >> > to take advantage of it and then detect if the running kernel supports > >> >> > it. An administrator would have to detect that MADV_FREE pages are being > >> >> > prematurely discarded leading to a slowdown and that is hard to detect. > >> >> > It could be inferred from monitoring compaction stats and checking > >> >> > if compaction activity is correlated with higher minor faults in the > >> >> > target application. Proving the correlation would require using the perf > >> >> > software event PERF_COUNT_SW_PAGE_FAULTS_MIN and matching the addresses > >> >> > to MADV_FREE regions that were freed prematurely. That is not an obvious > >> >> > debugging step to take when an application detects latency spikes. > >> >> > > >> >> > Now, you could add a counter specifically for MADV_FREE pages freed for > >> >> > reasons other than memory pressure and hope the administrator knows about > >> >> > the counter and what it means. That type of knowledge could take a long > >> >> > time to spread so it's really very important that there is evidence of > >> >> > an application that suffers due to the current MADV_FREE and migration > >> >> > behaviour. > >> >> > >> >> OK. I understand that this patchset isn't a universal win, so we need > >> >> some way to justify it. I will try to find some application for that. > >> >> > >> >> Another thought, as proposed by David Hildenbrand, it's may be a > >> >> universal win to discard clean MADV_FREE pages when migrating if there are > >> >> already memory pressure on the target node. For example, if the free > >> >> memory on the target node is lower than high watermark? > >> > > >> > This is already happening because if the target node is short on memory > >> > it will start to reclaim and if MADV_FREE pages are at the tail of > >> > inactive file LRU list then they will be dropped. Please note how that > >> > follows proper aging and doesn't introduce any special casing. Really > >> > MADV_FREE is an inactive cache for anonymous memory and we treat it like > >> > inactive page cache. This is not carved in stone of course but it really > >> > requires very good justification to change. > >> > >> If my understanding were correct, the newly migrated clean MADV_FREE > >> pages will be put at the head of inactive file LRU list instead of the > >> tail. So it's possible that some useful file cache pages will be > >> reclaimed. > > > > This is the case also when you migrate other pages, right? We simply > > cannot preserve the aging. > > So you consider the priority of the clean MADV_FREE pages is same as > that of page cache pages? This is how MADV_FREE has been implemented, yes. See f7ad2a6cb9f7 ("mm: move MADV_FREE pages into LRU_INACTIVE_FILE list") for the justification. > Because the penalty difference is so large, I > think it may be a good idea to always put clean MADV_FREE pages at the > tail of the inactive file LRU list? You are again making assumptions without giving any actual real examples. Reconstructing MADV_FREE pages cost can differ a lot. This really depends on the specific usecase. Moving pages to the tail of LRU would make them the primary candidate for the reclaim with a strange LIFO semantic. Adding them to the head might be not the universal win but it will at least provide a reasonable FIFO semantic. I also find it much more easier to reason about MADV_FREE as an inactive cache. -- Michal Hocko SUSE Labs