From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 544C9C433F5 for ; Thu, 7 Apr 2022 03:47:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C4D946B0071; Wed, 6 Apr 2022 23:46:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD6736B0073; Wed, 6 Apr 2022 23:46:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A774A6B0074; Wed, 6 Apr 2022 23:46:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0008.hostedemail.com [216.40.44.8]) by kanga.kvack.org (Postfix) with ESMTP id 919806B0071 for ; Wed, 6 Apr 2022 23:46:52 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 3537C183E951F for ; Thu, 7 Apr 2022 03:46:42 +0000 (UTC) X-FDA: 79328696244.27.5C0E1EE Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) by imf11.hostedemail.com (Postfix) with ESMTP id B45AB40005 for ; Thu, 7 Apr 2022 03:46:41 +0000 (UTC) Received: by mail-ej1-f42.google.com with SMTP id i27so8157897ejd.9 for ; Wed, 06 Apr 2022 20:46:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=2osYizLMrar7QqOz8LD8setUutp7c0I8CuIlWyU0pBg=; b=cKZ6do/t2L9vKakesKtxffn9PtC9S2qbMr1610jgtP28xQz6M/jTZXicOp3+64uVHX qijM4rThv3VpwivwHnyedSzyRpf3vh4n1k0TQAAJPm/h8Bz9d85Jpr49IYQeUIVlM+cD ZmTFDpFRzm4xJyYGwYoOJ5+X2mqr09Y7sRAuxURsQ/icMKA7Wu8S7D+O6hq1eDEY8BDA VC8YgH7lGf+Yey31F5KelHmIX5na8552lGQkasonqsXOqDgLwPKyDx4wi/yPs6zzQhcB MZZWYzVqqtyDwlF3O6Ax60y95AqQ2heZZg3yOdHV9oD/k0Powk68JmG7cbwav6DWN2zp M2VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2osYizLMrar7QqOz8LD8setUutp7c0I8CuIlWyU0pBg=; b=7Bt70h55EI+jYKZCtCewEb/ZsFRy2HGDey8+hEpq7eq0eYKwEN1IQ99uT3xk1xKAIv F3d/yii0t7q7hWyJQCy09r2lAbQAvJX092pMdInwakgWuVqlUEwNhdslBQ3W63hxZ61Q H3zv9F1JPdfjh8ODDYAeSVAmPeWe7o3HV2UhXbv2Ie1R6Hd9BffrOXShRrUrhcWdz36L S4PCYGu1uvYepba7PJz2LWiIi8cRdwN/HU2VQkC9GkOFTSrNZJ88skTEoWVVYbQD3zWc Zvp/Hxwc0+3TsxJL/fDEqBGr0PcSKziyjKsmqZo0pjhxXucnOWnb2EGTVIHPxZKUjQ0d Pb3Q== X-Gm-Message-State: AOAM532x2qGJ0Tvw7waPRqJI4D6E3ZFLF4bzpAkTNhW4T7YAnrZNyvf+ tyGB2RuNpMr++ciMDDQ4F6nRtDWGkaTwLxPjgjY= X-Google-Smtp-Source: ABdhPJx0JJX3YOy4xz1wSArtme702dUMio1P+UzX3qxTFY7xH4BfET9qWERzTi+yH9291LSd38MyTL22akuCDuMuecM= X-Received: by 2002:a17:906:39da:b0:6cf:7f09:a7bc with SMTP id i26-20020a17090639da00b006cf7f09a7bcmr11870481eje.457.1649303200136; Wed, 06 Apr 2022 20:46:40 -0700 (PDT) MIME-Version: 1.0 References: <20220309021230.721028-1-yuzhao@google.com> <20220309021230.721028-8-yuzhao@google.com> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Thu, 7 Apr 2022 15:46:29 +1200 Message-ID: Subject: Re: [PATCH v9 07/14] mm: multi-gen LRU: exploit locality in rmap To: Yu Zhao Cc: Andrew Morton , Linus Torvalds , Andi Kleen , Aneesh Kumar , Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Jesse Barnes , Johannes Weiner , Jonathan Corbet , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Mike Rapoport , Rik van Riel , Vlastimil Babka , Will Deacon , Ying Huang , LAK , Linux Doc Mailing List , LKML , Linux-MM , Kernel Page Reclaim v2 , x86 , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , =?UTF-8?Q?Holger_Hoffst=C3=A4tte?= , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh , Vaibhav Jain Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B45AB40005 X-Stat-Signature: frro5o8mods9pmu14nbc57sdy3dwryif Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="cKZ6do/t"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.218.42 as permitted sender) smtp.mailfrom=21cnbao@gmail.com X-Rspam-User: X-HE-Tag: 1649303201-896510 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Apr 7, 2022 at 3:04 PM Yu Zhao wrote: > > On Wed, Apr 6, 2022 at 8:29 PM Barry Song <21cnbao@gmail.com> wrote: > > > > On Wed, Mar 9, 2022 at 3:48 PM Yu Zhao wrote: > > > > > > Searching the rmap for PTEs mapping each page on an LRU list (to test > > > and clear the accessed bit) can be expensive because pages from > > > different VMAs (PA space) are not cache friendly to the rmap (VA > > > space). For workloads mostly using mapped pages, the rmap has a high > > > CPU cost in the reclaim path. > > > > > > This patch exploits spatial locality to reduce the trips into the > > > rmap. When shrink_page_list() walks the rmap and finds a young PTE, a > > > new function lru_gen_look_around() scans at most BITS_PER_LONG-1 > > > adjacent PTEs. On finding another young PTE, it clears the accessed > > > bit and updates the gen counter of the page mapped by this PTE to > > > (max_seq%MAX_NR_GENS)+1. > > > > Hi Yu, > > It seems an interesting feature to save the cost of rmap. but will it lead to > > possible judging of cold pages as hot pages? > > In case a page is mapped by 20 processes, and it has been accessed > > by 5 of them, when we look around one of the 5 processes, the page > > will be young and this pte is cleared. but we still have 4 ptes which are not > > cleared. then we don't access the page for a long time, but the 4 uncleared > > PTEs will still make the page "hot" since they are not cleared, we will find > > the page is hot either due to look-arounding the 4 processes or rmapping > > the page later? > > Why are the remaining 4 accessed PTEs skipped? The rmap should check > all the 20 PTEs. for example page A is the neighbour of page B in process 1, when we do rmap for B, we look-around and clear A's pte in process 1. but A's ptes are still set in process 2,3,4,5. > > Even if they were skipped, it doesn't matter. The same argument could > be made for the rest of 1 millions minus 1 pages that have been timely > scanned, on a 4GB laptop. The fundamental principle (assumption) of > MGLRU is never about making the best choices. Nothing can because it's > impossible to predict the future that well, given the complexity of > today's workloads, not on a phone, definitely not on a server that > runs mixed types of workloads. The primary goal is to avoid the worst > choices at a minimum (scanning) cost. The second goal is to pick good > ones at an acceptable cost, which probably are a half of all possible > choices. thanks barry