From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDAC0C4361B for ; Wed, 16 Dec 2020 18:23:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5E19125E3C for ; Wed, 16 Dec 2020 18:23:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E19125E3C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 95E166B0071; Wed, 16 Dec 2020 13:23:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 868D96B006C; Wed, 16 Dec 2020 13:23:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B8FC6B0070; Wed, 16 Dec 2020 13:23:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0140.hostedemail.com [216.40.44.140]) by kanga.kvack.org (Postfix) with ESMTP id 3DE5E6B006C for ; Wed, 16 Dec 2020 13:23:43 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id ECF0C181AEF10 for ; Wed, 16 Dec 2020 18:23:42 +0000 (UTC) X-FDA: 77599968684.24.rose37_4f00e472742e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id B4B131A4A7 for ; Wed, 16 Dec 2020 18:23:42 +0000 (UTC) X-HE-Tag: rose37_4f00e472742e X-Filterd-Recvd-Size: 9355 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Wed, 16 Dec 2020 18:23:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=CYoAM4DvvSHoLtnhY+OYMLaKtsVx5aWcdLxiE6/isOA=; b=BYk+HbzB0eICsa0vMZLP8cNp7T 9tQ+3//YpOjekHFUb5vmPlJzTkxyhVnJkVj3kBh0jouHGb9GG04G5iAGMC0jtgs+tah8UfxjHuzXt 85IJxR+AsSgQWe252Z7nNzeX5VFM4T9ICKwh/J0hNzwxE7bjy4LouDfKFWdp6lBbgzzQN/+OqM27A IMZvimFxl4sWtxcatTFfCBUCj722BvEmAT5GOZQYcWAco/S3/RCMVSod4cEGtV3FdNbKb+Cx+umAU qrY7Ki7GdOFKt8kBaY5treucJt7Vf00oTHPC4iLVAYADWYROwP9B8iPDhwCaj8ye2rNxFBc3OEKwk qSEgCgJQ==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kpbSb-00076E-J1; Wed, 16 Dec 2020 18:23:37 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org Subject: [PATCH 04/25] mm: Create FolioFlags Date: Wed, 16 Dec 2020 18:23:14 +0000 Message-Id: <20201216182335.27227-5-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201216182335.27227-1-willy@infradead.org> References: <20201216182335.27227-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: These new functions are the folio analogues of the PageFlags functions. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/page-flags.h | 80 ++++++++++++++++++++++++++++++-------- 1 file changed, 63 insertions(+), 17 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index ec5d0290e0ee..446217ce13e9 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -212,6 +212,12 @@ static inline void page_init_poison(struct page *pag= e, size_t size) } #endif =20 +static unsigned long *folio_flags(struct folio *folio) +{ + VM_BUG_ON_PGFLAGS(PagePoisoned(&folio->page), &folio->page); + return &folio->page.flags; +} + /* * Page flags policies wrt compound pages * @@ -260,30 +266,44 @@ static inline void page_init_poison(struct page *pa= ge, size_t size) * Macros to create function definitions for page flags */ #define TESTPAGEFLAG(uname, lname, policy) \ +static __always_inline int Folio##uname(struct folio *folio) \ + { return test_bit(PG_##lname, folio_flags(folio)); } \ static __always_inline int Page##uname(struct page *page) \ { return test_bit(PG_##lname, &policy(page, 0)->flags); } =20 #define SETPAGEFLAG(uname, lname, policy) \ +static __always_inline void SetFolio##uname(struct folio *folio) \ + { set_bit(PG_##lname, folio_flags(folio)); } \ static __always_inline void SetPage##uname(struct page *page) \ { set_bit(PG_##lname, &policy(page, 1)->flags); } =20 #define CLEARPAGEFLAG(uname, lname, policy) \ +static __always_inline void ClearFolio##uname(struct folio *folio) \ + { clear_bit(PG_##lname, folio_flags(folio)); } \ static __always_inline void ClearPage##uname(struct page *page) \ { clear_bit(PG_##lname, &policy(page, 1)->flags); } =20 #define __SETPAGEFLAG(uname, lname, policy) \ +static __always_inline void __SetFolio##uname(struct folio *folio) \ + { __set_bit(PG_##lname, folio_flags(folio)); } \ static __always_inline void __SetPage##uname(struct page *page) \ { __set_bit(PG_##lname, &policy(page, 1)->flags); } =20 #define __CLEARPAGEFLAG(uname, lname, policy) \ +static __always_inline void __ClearFolio##uname(struct folio *folio) \ + { __clear_bit(PG_##lname, folio_flags(folio)); } \ static __always_inline void __ClearPage##uname(struct page *page) \ { __clear_bit(PG_##lname, &policy(page, 1)->flags); } =20 #define TESTSETFLAG(uname, lname, policy) \ +static __always_inline int TestSetFolio##uname(struct folio *folio) \ + { return test_and_set_bit(PG_##lname, folio_flags(folio)); } \ static __always_inline int TestSetPage##uname(struct page *page) \ { return test_and_set_bit(PG_##lname, &policy(page, 1)->flags); } =20 #define TESTCLEARFLAG(uname, lname, policy) \ +static __always_inline int TestClearFolio##uname(struct folio *folio) \ + { return test_and_clear_bit(PG_##lname, folio_flags(folio)); } \ static __always_inline int TestClearPage##uname(struct page *page) \ { return test_and_clear_bit(PG_##lname, &policy(page, 1)->flags); } =20 @@ -302,21 +322,27 @@ static __always_inline int TestClearPage##uname(str= uct page *page) \ TESTCLEARFLAG(uname, lname, policy) =20 #define TESTPAGEFLAG_FALSE(uname) \ +static inline int Folio##uname(const struct folio *folio) { return 0; } = \ static inline int Page##uname(const struct page *page) { return 0; } =20 #define SETPAGEFLAG_NOOP(uname) \ +static inline void SetFolio##uname(struct folio *folio) { } \ static inline void SetPage##uname(struct page *page) { } =20 #define CLEARPAGEFLAG_NOOP(uname) \ +static inline void ClearFolio##uname(struct folio *folio) { } \ static inline void ClearPage##uname(struct page *page) { } =20 #define __CLEARPAGEFLAG_NOOP(uname) \ +static inline void __ClearFolio##uname(struct folio *folio) { } \ static inline void __ClearPage##uname(struct page *page) { } =20 #define TESTSETFLAG_FALSE(uname) \ +static inline int TestSetFolio##uname(struct folio *folio) { return 0; }= \ static inline int TestSetPage##uname(struct page *page) { return 0; } =20 #define TESTCLEARFLAG_FALSE(uname) \ +static inline int TestClearFolio##uname(struct folio *folio) { return 0;= } \ static inline int TestClearPage##uname(struct page *page) { return 0; } =20 #define PAGEFLAG_FALSE(uname) TESTPAGEFLAG_FALSE(uname) \ @@ -393,14 +419,18 @@ PAGEFLAG_FALSE(HighMem) #endif =20 #ifdef CONFIG_SWAP -static __always_inline int PageSwapCache(struct page *page) +static __always_inline bool FolioSwapCache(struct folio *folio) { -#ifdef CONFIG_THP_SWAP - page =3D compound_head(page); -#endif - return PageSwapBacked(page) && test_bit(PG_swapcache, &page->flags); + return FolioSwapBacked(folio) && + test_bit(PG_swapcache, folio_flags(folio)); + +} =20 +static __always_inline bool PageSwapCache(struct page *page) +{ + return FolioSwapCache(page_folio(page)); } + SETPAGEFLAG(SwapCache, swapcache, PF_NO_TAIL) CLEARPAGEFLAG(SwapCache, swapcache, PF_NO_TAIL) #else @@ -509,18 +539,16 @@ TESTPAGEFLAG_FALSE(Ksm) =20 u64 stable_page_flags(struct page *page); =20 -static inline int PageUptodate(struct page *page) +static inline int FolioUptodate(struct folio *folio) { - int ret; - page =3D compound_head(page); - ret =3D test_bit(PG_uptodate, &(page)->flags); + int ret =3D test_bit(PG_uptodate, folio_flags(folio)); /* * Must ensure that the data we read out of the page is loaded * _after_ we've loaded page->flags to check for PageUptodate. * We can skip the barrier if the page is not uptodate, because * we wouldn't be reading anything from it. * - * See SetPageUptodate() for the other side of the story. + * See SetFolioUptodate() for the other side of the story. */ if (ret) smp_rmb(); @@ -528,23 +556,38 @@ static inline int PageUptodate(struct page *page) return ret; } =20 -static __always_inline void __SetPageUptodate(struct page *page) +static inline int PageUptodate(struct page *page) +{ + return FolioUptodate(page_folio(page)); +} + +static __always_inline void __SetFolioUptodate(struct folio *folio) { - VM_BUG_ON_PAGE(PageTail(page), page); smp_wmb(); - __set_bit(PG_uptodate, &page->flags); + __set_bit(PG_uptodate, folio_flags(folio)); } =20 -static __always_inline void SetPageUptodate(struct page *page) +static __always_inline void SetFolioUptodate(struct folio *folio) { - VM_BUG_ON_PAGE(PageTail(page), page); /* * Memory barrier must be issued before setting the PG_uptodate bit, * so that all previous stores issued in order to bring the page * uptodate are actually visible before PageUptodate becomes true. */ smp_wmb(); - set_bit(PG_uptodate, &page->flags); + set_bit(PG_uptodate, folio_flags(folio)); +} + +static __always_inline void __SetPageUptodate(struct page *page) +{ + VM_BUG_ON_PAGE(PageTail(page), page); + __SetFolioUptodate((struct folio *)page); +} + +static __always_inline void SetPageUptodate(struct page *page) +{ + VM_BUG_ON_PAGE(PageTail(page), page); + SetFolioUptodate((struct folio *)page); } =20 CLEARPAGEFLAG(Uptodate, uptodate, PF_NO_TAIL) @@ -593,6 +636,10 @@ static inline void ClearPageCompound(struct page *pa= ge) int PageHuge(struct page *page); int PageHeadHuge(struct page *page); bool page_huge_active(struct page *page); +static inline bool FolioHuge(struct folio *folio) +{ + return PageHeadHuge(&folio->page); +} #else TESTPAGEFLAG_FALSE(Huge) TESTPAGEFLAG_FALSE(HeadHuge) @@ -603,7 +650,6 @@ static inline bool page_huge_active(struct page *page= ) } #endif =20 - #ifdef CONFIG_TRANSPARENT_HUGEPAGE /* * PageHuge() only returns true for hugetlbfs pages, but not for --=20 2.29.2