From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E68FBC433DF for ; Mon, 12 Oct 2020 21:30:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3531C21655 for ; Mon, 12 Oct 2020 21:30:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rY2+TsxL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3531C21655 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 60C7F940008; Mon, 12 Oct 2020 17:30:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C040900002; Mon, 12 Oct 2020 17:30:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4ACB9940008; Mon, 12 Oct 2020 17:30:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0209.hostedemail.com [216.40.44.209]) by kanga.kvack.org (Postfix) with ESMTP id 200FD900002 for ; Mon, 12 Oct 2020 17:30:30 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A9D7F181AE866 for ; Mon, 12 Oct 2020 21:30:29 +0000 (UTC) X-FDA: 77364567378.30.fog16_5414fd7271fd Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 8327D180B3AA7 for ; Mon, 12 Oct 2020 21:30:29 +0000 (UTC) X-HE-Tag: fog16_5414fd7271fd X-Filterd-Recvd-Size: 6204 Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Mon, 12 Oct 2020 21:30:29 +0000 (UTC) Received: by mail-ed1-f65.google.com with SMTP id dg9so16260132edb.12 for ; Mon, 12 Oct 2020 14:30:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WhdARHn7ekpE/B6g7hvohtBN+vrCDHD817Ns5WlYqDg=; b=rY2+TsxLiz6pvvr3ngQb0t80ciUoaEE7i0zO5vITri28qzFvp84GEw9YRpxStpMl2d 2cQg5nTQGQZuC1pGc7H5QlQypme3ga5nM3do23Wme/98JRQc70wopxGq97eSVh4n+w2v MlfznvOSBWoLy9KaSipAnSA2oARr5EK/YuAfDkGXmL7pqyA3iAk05Qc1bA+ZY0Bi49Qh FrMUn7KYEI2mrN6zRgZc2qQgaJn8GUwAKF4t6Q2KRUEwAxyDabCRYCoVhqKY/uV8mrT4 +Lrd9Q3j84rDBpzy03XiKNjdHheE12NV/QAz88eMRtAtBVGrUWfjhtEywNbROhLKbAer b9MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WhdARHn7ekpE/B6g7hvohtBN+vrCDHD817Ns5WlYqDg=; b=e7BT07K03OMSTyAv80OnQUCEZTOMg5pJCdzGLPDiT+GCVClCwEwwlLho2mrNNzkpmT D8sO12RXydzwOjgGMyh3UpsmNTcQsfDVUWfcnO8sQcD5Fm0QPbMPcyw9Y7UYldVokAJi fsD2ctR4/qFYINBjz+Vbiw/fkvGf+rtci800wpMbwz7D8kVlVIPQhhVRS6J6fOFbpWRf k675u9JrNcXoTBII7ESwTmkTls31QXq/i3nZExAE3+drTb1CpnKndRo92f4nNvtjBsXS 06YTdvvF5FefIkfL9e6VG7mIobLfRSb4L0ZNBslK82v+kTKMnO1cQ0kOrKKNIrGLEdy7 7smw== X-Gm-Message-State: AOAM533gA8Dp0Gh9NldP69k2egnvL3lLiTiv726whwzLsThKNSKcj38g 1TnsEjn8wGKiSpRHclrI1ly+Gayy9/qBfCL/4pI= X-Google-Smtp-Source: ABdhPJyjrxrzuiVrLGd/YJ9A+ajKo3jWKJciHFwu0QPQQGx0yC4bCQqAaTr9TSIfPZCggVwtVjiLQ76wJTtu9dVjy40= X-Received: by 2002:a50:c05b:: with SMTP id u27mr16114957edd.290.1602538227750; Mon, 12 Oct 2020 14:30:27 -0700 (PDT) MIME-Version: 1.0 References: <20201007161736.ACC6E387@viggo.jf.intel.com> In-Reply-To: <20201007161736.ACC6E387@viggo.jf.intel.com> From: Yang Shi Date: Mon, 12 Oct 2020 14:30:16 -0700 Message-ID: Subject: Re: [RFC][PATCH 0/9] [v4][RESEND] Migrate Pages in lieu of discard To: Dave Hansen Cc: Linux Kernel Mailing List , Linux MM , Yang Shi , David Rientjes , Huang Ying , Dan Williams , David Hildenbrand Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Oct 7, 2020 at 9:17 AM Dave Hansen wrote: > > > Changes since (automigrate-20200818): > * Fall back to normal reclaim when demotion fails > > The full series is also available here: > > https://github.com/hansendc/linux/tree/automigrate-20201007 > > I really just want folks to look at: > > [RFC][PATCH 5/9] mm/migrate: demote pages during reclaim > > I've reworked that so that it can both use the high-level migration > API, and fall back to normal reclaim if migration fails. I think > that gives us the best of both worlds. Thanks for doing this. Although I was inclined to think the kswapds on PMEM nodes could make enough space for retrying migration later instead of doing swap right away, this approach might be over-engineering and over-killing. The simple immediate "retry regular reclaim" approach also looks fine to me for the time being. We always could optimize it later with more test results backed by real life workloads. > > I'm posting the series in case folks want to run the whole thing. > > -- > > We're starting to see systems with more and more kinds of memory such > as Intel's implementation of persistent memory. > > Let's say you have a system with some DRAM and some persistent memory. > Today, once DRAM fills up, reclaim will start and some of the DRAM > contents will be thrown out. Allocations will, at some point, start > falling over to the slower persistent memory. > > That has two nasty properties. First, the newer allocations can end > up in the slower persistent memory. Second, reclaimed data in DRAM > are just discarded even if there are gobs of space in persistent > memory that could be used. > > This set implements a solution to these problems. At the end of the > reclaim process in shrink_page_list() just before the last page > refcount is dropped, the page is migrated to persistent memory instead > of being dropped. > > While I've talked about a DRAM/PMEM pairing, this approach would > function in any environment where memory tiers exist. > > This is not perfect. It "strands" pages in slower memory and never > brings them back to fast DRAM. Other things need to be built to > promote hot pages back to DRAM. > > This is also all based on an upstream mechanism that allows > persistent memory to be onlined and used as if it were volatile: > > http://lkml.kernel.org/r/20190124231441.37A4A305@viggo.jf.intel.com > > == Open Issues == > > * For cpusets and memory policies that restrict allocations > to PMEM, is it OK to demote to PMEM? Do we need a cgroup- > level API to opt-in or opt-out of these migrations? > > -- > > Changes since (https://lwn.net/Articles/824830/): > * Use higher-level migrate_pages() API approach from Yang Shi's > earlier patches. > * made sure to actually check node_reclaim_mode's new bit > * disabled migration entirely before introducing RECLAIM_MIGRATE > * Replace GFP_NOWAIT with explicit __GFP_KSWAPD_RECLAIM and > comment why we want that. > * Comment on effects of that keep multiple source nodes from > sharing target nodes > > Cc: Yang Shi > Cc: David Rientjes > Cc: Huang Ying > Cc: Dan Williams > Cc: David Hildenbrand >