From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9366C433F5 for ; Fri, 4 Feb 2022 19:59:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B0D1A6B0078; Fri, 4 Feb 2022 14:59:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AAD786B0072; Fri, 4 Feb 2022 14:59:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 862736B0075; Fri, 4 Feb 2022 14:59:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0004.hostedemail.com [216.40.44.4]) by kanga.kvack.org (Postfix) with ESMTP id 7210C6B0072 for ; Fri, 4 Feb 2022 14:59:04 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 2CC9A181A349E for ; Fri, 4 Feb 2022 19:59:04 +0000 (UTC) X-FDA: 79106161008.21.627FFA0 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf11.hostedemail.com (Postfix) with ESMTP id 9FA8540002 for ; Fri, 4 Feb 2022 19:59:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=bef0cZGDuU+YbKujs3r8MKTZ9MxxmLgLhjNfcds/vHs=; b=ZuQO5Ny4EwFYv7QZX8DPglu2sW S/1xbJCd/issskPZ7rvxxvlkBk8xrhJycNf9d0yPNpa4VOf4g9KsLncba7ius02/1k92rSsITvw0y 1DSqevdHkOiOSx+78PYLjbyamkNnEgEvqH19qvBuv542czbKW1TbJCtLdvoY0w08HHHMir+Aum+gw Klha0sJWu6Z8GxdZY7KH7OgZhaBtfLjX5l9YHG+dYwKCmeNIipOkxGjZz9sThsZfGgchWYSF6SVhu 37yFUEkP/Gox8NpAao6qfm80wwlS8QoRhe6nd6miljHg7MI3Fr0kAO5x2yaEKuk+C052XTCk2XliC sxYpxlwg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nG4jW-007Lm9-82; Fri, 04 Feb 2022 19:59:02 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org Subject: [PATCH 29/75] mm/workingset: Convert workingset_eviction() to take a folio Date: Fri, 4 Feb 2022 19:58:06 +0000 Message-Id: <20220204195852.1751729-30-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220204195852.1751729-1-willy@infradead.org> References: <20220204195852.1751729-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9FA8540002 X-Stat-Signature: 3dj14s6reh5ifceqgg8fx6mqzukd9tpm Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ZuQO5Ny4; dmarc=none; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspam-User: nil X-HE-Tag: 1644004743-328770 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This removes an assumption that THPs are the only kind of compound pages and removes a few hidden calls to compound_head(). Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/swap.h | 2 +- mm/vmscan.c | 7 ++++--- mm/workingset.c | 25 +++++++++++++------------ 3 files changed, 18 insertions(+), 16 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 1d38d9475c4d..de36f140227e 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -328,7 +328,7 @@ static inline swp_entry_t folio_swap_entry(struct fol= io *folio) =20 /* linux/mm/workingset.c */ void workingset_age_nonresident(struct lruvec *lruvec, unsigned long nr_= pages); -void *workingset_eviction(struct page *page, struct mem_cgroup *target_m= emcg); +void *workingset_eviction(struct folio *folio, struct mem_cgroup *target= _memcg); void workingset_refault(struct folio *folio, void *shadow); void workingset_activation(struct folio *folio); =20 diff --git a/mm/vmscan.c b/mm/vmscan.c index e0cc5f0cb999..75223b7d98ec 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1240,6 +1240,7 @@ static pageout_t pageout(struct page *page, struct = address_space *mapping) static int __remove_mapping(struct address_space *mapping, struct page *= page, bool reclaimed, struct mem_cgroup *target_memcg) { + struct folio *folio =3D page_folio(page); int refcount; void *shadow =3D NULL; =20 @@ -1287,7 +1288,7 @@ static int __remove_mapping(struct address_space *m= apping, struct page *page, swp_entry_t swap =3D { .val =3D page_private(page) }; mem_cgroup_swapout(page, swap); if (reclaimed && !mapping_exiting(mapping)) - shadow =3D workingset_eviction(page, target_memcg); + shadow =3D workingset_eviction(folio, target_memcg); __delete_from_swap_cache(page, swap, shadow); xa_unlock_irq(&mapping->i_pages); put_swap_page(page, swap); @@ -1313,8 +1314,8 @@ static int __remove_mapping(struct address_space *m= apping, struct page *page, */ if (reclaimed && page_is_file_lru(page) && !mapping_exiting(mapping) && !dax_mapping(mapping)) - shadow =3D workingset_eviction(page, target_memcg); - __delete_from_page_cache(page, shadow); + shadow =3D workingset_eviction(folio, target_memcg); + __filemap_remove_folio(folio, shadow); xa_unlock_irq(&mapping->i_pages); if (mapping_shrinkable(mapping)) inode_add_lru(mapping->host); diff --git a/mm/workingset.c b/mm/workingset.c index 8c03afe1d67c..b717eae4e0dd 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -245,31 +245,32 @@ void workingset_age_nonresident(struct lruvec *lruv= ec, unsigned long nr_pages) } =20 /** - * workingset_eviction - note the eviction of a page from memory + * workingset_eviction - note the eviction of a folio from memory * @target_memcg: the cgroup that is causing the reclaim - * @page: the page being evicted + * @folio: the folio being evicted * - * Return: a shadow entry to be stored in @page->mapping->i_pages in pla= ce - * of the evicted @page so that a later refault can be detected. + * Return: a shadow entry to be stored in @folio->mapping->i_pages in pl= ace + * of the evicted @folio so that a later refault can be detected. */ -void *workingset_eviction(struct page *page, struct mem_cgroup *target_m= emcg) +void *workingset_eviction(struct folio *folio, struct mem_cgroup *target= _memcg) { - struct pglist_data *pgdat =3D page_pgdat(page); + struct pglist_data *pgdat =3D folio_pgdat(folio); unsigned long eviction; struct lruvec *lruvec; int memcgid; =20 - /* Page is fully exclusive and pins page's memory cgroup pointer */ - VM_BUG_ON_PAGE(PageLRU(page), page); - VM_BUG_ON_PAGE(page_count(page), page); - VM_BUG_ON_PAGE(!PageLocked(page), page); + /* Folio is fully exclusive and pins folio's memory cgroup pointer */ + VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); + VM_BUG_ON_FOLIO(folio_ref_count(folio), folio); + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); =20 lruvec =3D mem_cgroup_lruvec(target_memcg, pgdat); /* XXX: target_memcg can be NULL, go through lruvec */ memcgid =3D mem_cgroup_id(lruvec_memcg(lruvec)); eviction =3D atomic_long_read(&lruvec->nonresident_age); - workingset_age_nonresident(lruvec, thp_nr_pages(page)); - return pack_shadow(memcgid, pgdat, eviction, PageWorkingset(page)); + workingset_age_nonresident(lruvec, folio_nr_pages(folio)); + return pack_shadow(memcgid, pgdat, eviction, + folio_test_workingset(folio)); } =20 /** --=20 2.34.1