From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 206C7C49361 for ; Thu, 17 Jun 2021 15:58:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A5E8E613AA for ; Thu, 17 Jun 2021 15:58:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5E8E613AA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F34496B0071; Thu, 17 Jun 2021 11:58:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EE3606B0072; Thu, 17 Jun 2021 11:58:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D849D6B0073; Thu, 17 Jun 2021 11:58:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id A6FE06B0071 for ; Thu, 17 Jun 2021 11:58:05 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 422EDDE05 for ; Thu, 17 Jun 2021 15:58:05 +0000 (UTC) X-FDA: 78263672130.36.E9F1E0E Received: from mail-yb1-f176.google.com (mail-yb1-f176.google.com [209.85.219.176]) by imf03.hostedemail.com (Postfix) with ESMTP id 63F64C00CBC0 for ; Thu, 17 Jun 2021 15:57:53 +0000 (UTC) Received: by mail-yb1-f176.google.com with SMTP id q21so8912399ybg.8 for ; Thu, 17 Jun 2021 08:58:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KeSL84a4XPJUBzK0wBupmv1FxrnN39pP74DFLtKm87Y=; b=VzAgAZc8FSzleBZIaVO+qCznHbTeuZzuVaQ8SAid7S+nVAv4DNP0Bs4qm0LhP51QoI ldVr4Bvx29BO6vQ8FZL/n+1F71AoKXoW44ehQ9CTNGBT+Sd1PlxV4NelMaSWC6zg726t hXvK8VehCR0bQGESFXs2yGyTpwDMMh5OJMHOZIjgTggSjiMfx7dDdlSpDADVI73hjgxN LJbs2/JHWLlU/zBVIuVBNTLa7ZAcyAVX0aJiL578ftwnsf/2TP/VKC/+CRzBNo2bzLGg 9BlpoHzJk40zCeWlkl+CCZGp7icowtPa0memmQA2BlVHqpyBsc5ZjOagwKaq2wdP3RIS HSmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KeSL84a4XPJUBzK0wBupmv1FxrnN39pP74DFLtKm87Y=; b=ShIhHd3rgArlBZ+Q87k9oO9A+fm+D8iRATgCsQQkjDcnO3M1d2PAAF+G9iekQUdRHI 43eYGgl6onPRIWSwVT+5yuGRazwIUH8OzXrX7zM2YrvYoVS3YuJc2wf7b9M5QfLaf7FR ufxs5POyoskOVBechoM0260IqngGHXB/2hpuFXucOZY+n0CtJ+jxQ1lP9pdIB3KbaROV OG+9J+w4WzzaDRUiom+eGEHnJk9eu/7FtjiMTEDwAXQv4TOVpuRXtjvljJvnoyMuTljD dxgSi99UWq2N8VoST65hkzh7zlI5tkY8K7csVNRoZy9vwsUfhSvMizpcmUa3NwoZxjSy xHGg== X-Gm-Message-State: AOAM531R6xTT0pvf7i6xbxBc5gLRbybe6VW0UEvgVF9W0Pz1M2lCilMo V+V6lXcK+51hmUClYt8fsFWrLAlPLsc1GRLZJTVzgw== X-Google-Smtp-Source: ABdhPJwbC6lAy5X1mJkHkMWHTN+lQkKMj2Zn+52cmu1ajgdOONr2knnv9OchWnhhS+Ua+eTBO0F8cA0v26diPlc3JFI= X-Received: by 2002:a25:9981:: with SMTP id p1mr7386123ybo.246.1623945483987; Thu, 17 Jun 2021 08:58:03 -0700 (PDT) MIME-Version: 1.0 References: <59f61523-cb38-bf8c-51ba-1017ea7212d2@google.com> In-Reply-To: From: Shakeel Butt Date: Thu, 17 Jun 2021 08:57:52 -0700 Message-ID: Subject: Re: Idle THPs To: Yang Shi , Yu Zhao Cc: Hugh Dickins , Matthew Wilcox , Vladimir Davydov , "Kirill A. Shutemov" , Linux MM Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 63F64C00CBC0 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20161025 header.b=VzAgAZc8; spf=pass (imf03.hostedemail.com: domain of shakeelb@google.com designates 209.85.219.176 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: arh95wyd6h33cytybpu4ejq3y186n5po X-HE-Tag: 1623945473-570054 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: +Yu Zhao On Thu, Jun 10, 2021 at 3:54 PM Yang Shi wrote: > > On Thu, Jun 10, 2021 at 2:47 PM Hugh Dickins wrote: > > > > On Thu, 10 Jun 2021, Matthew Wilcox wrote: > > > > > As part of the folio work, I'm looking at PageIdle and PageYoung and > > > they're defined to operate on PF_ANY. So, for example, in > > > pagecache_get_page(), we will call clear_page_idle() on the head page > > > (actually, I changed this in a8cf7f272b5a -- before, it would call > > > clear_page_idle() on the tail page). > > > > > > However, we never actually call set_page_idle() on tail pages. This is > > > because we only call it here: > > > > > > page = page_idle_get_page(pfn); > > > if (page) { > > > page_idle_clear_pte_refs(page); > > > set_page_idle(page); > > > put_page(page); > > > } > > > > > > where page_idle_get_page() does: > > > > > > struct page *page = pfn_to_online_page(pfn); > > > > > > if (!page || !PageLRU(page) || > > > !get_page_unless_zero(page)) > > > return NULL; > > > > > > get_page_unless_zero() will always fail for tail pages (as it uses > > > page_ref_add_unless(), which does not redirect to the head page's > > > refcount). So all tail pages read back as !idle in > > > page_idle_bitmap_read(). Is this intended? Should they rather > > > mirror the state of their head page? >From what I understand the idle bitmap is supposed to be used along with /proc/kpageflags. So, the users should skip tail pages for setting/getting the idle bits. > > > > Good point. > > > > I bet when I made that no-lru_lock cleanup in page_idle_get_page(), > > I was expecting the PageLRU to fail on tail, so get_page_unless_zero() > > irrelevant; but apparently PageLRU is PF_HEAD redirecting to head. > > Either way, yes, it will return NULL on tail, which may not be right. > > > > But maybe the physical scan works out okay with all the action > > happening on the head (Kirill got that to scan the tails in pvmw), > > then skipping the tails in the local scan. > > > > I'm not a page_idle user and don't want to get into the mechanics of it. > > Seems to be largely in maintenance mode these days, maybe nobody cares. > > > > Yang Shi was the last to make a real mod there, f0849ac0b8e0 ("mm: thp: fix > > potential clearing to referenced flag in page_idle_clear_pte_refs_one()"): > > likely he will know best. > > It was more than 3 years ago :-) > > Since the whole THP is considered referenced if any one of sub page is > referenced, so IMHO the tail page should mirror the state of their > head page. > > And AFAIK Google uses the idle flag to reclaim memory proactively, but > it is an out-of-tree feature. Loop Shakeel in this thread. We are not directly using upstream idle page infrastructure but surgically using the parts we need. Yu can provide more details. > > > > > Hugh