From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35C03C433E0 for ; Mon, 15 Mar 2021 02:24:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ABC7364E5C for ; Mon, 15 Mar 2021 02:24:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ABC7364E5C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D27576B0006; Sun, 14 Mar 2021 22:24:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CC2946B006C; Sun, 14 Mar 2021 22:24:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B29586B0070; Sun, 14 Mar 2021 22:24:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0099.hostedemail.com [216.40.44.99]) by kanga.kvack.org (Postfix) with ESMTP id 93F576B0006 for ; Sun, 14 Mar 2021 22:24:56 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 422045851 for ; Mon, 15 Mar 2021 02:24:56 +0000 (UTC) X-FDA: 77920515792.22.79BBF7A Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf03.hostedemail.com (Postfix) with ESMTP id 68FEDC0007CF for ; Mon, 15 Mar 2021 02:24:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=2ZbagKXY+gGRN4QfZeaJfgicJ/AkLpAyRYqbVKVHiPU=; b=JKWzad7sSLKdeqo6iLiDUB1ccX ilSNHANc6eWmSTvIo1X5LXjS1ab5ij5MfAQnSxkEG6i7LohbiaTghRSYL+orthkOuwLcBLfLzumFA KjHQ29lHL5DvQc2s/2GJwtJe1qWTiMfCprK0D4XU5gIBVNjJ7TdjiwN5EpomMQa3TkKYSw01hRPY+ beqpr3uWSo6rX3IqyCck7stLeExvSC9OO3wquWVImXf+5F60ALQRbN/uexJ3nIOthH4lDqbAkjG9N CFzG3P+ATKAUMaD3hT8ZprUQu8q0UbAvhBIULuVDJHM3UK1vINZx97D4LQyVss0yeoGuDkxhfxqoj RAPkYhMA==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1lLcuG-00GzqN-G0; Mon, 15 Mar 2021 02:24:39 +0000 Date: Mon, 15 Mar 2021 02:24:32 +0000 From: Matthew Wilcox To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: Re: [PATCH v4 07/25] mm: Create FolioFlags Message-ID: <20210315022432.GS2577561@casper.infradead.org> References: <20210305041901.2396498-1-willy@infradead.org> <20210305041901.2396498-8-willy@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210305041901.2396498-8-willy@infradead.org> X-Stat-Signature: jkphjz7epger5ugrg87ddg5knnbqghk9 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 68FEDC0007CF Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf03; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1615775095-515028 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Mar 05, 2021 at 04:18:43AM +0000, Matthew Wilcox (Oracle) wrote: > These new functions are the folio analogues of the PageFlags functions. > If CONFIG_DEBUG_VM_PGFLAGS is enabled, we check the folio is not a tail > page at every invocation. Note that this will also catch the PagePoisoned > case as a poisoned page has every bit set, which would include PageTail. > > This saves 1740 bytes of text with the distro-derived config that > I'm testing due to removing a double call to compound_head() in > PageSwapCache(). This patch is buggy due to using the wrong page->flags for FolioDoubleMapped. I'm not totally in love with this fix, but it does work without changing every PAGEFLAG definition. (also, I needed FolioTransHuge()) commit fe8ca904171345d113f06f381c255a3c4b20074e Author: Matthew Wilcox (Oracle) Date: Sun Mar 14 17:34:48 2021 -0400 fix FolioFlags diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 01aa4a71bf14..b7fd4c3733ca 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -212,10 +212,13 @@ static inline void page_init_poison(struct page *page, size_t size) } #endif -static unsigned long *folio_flags(struct folio *folio) +static unsigned long *folio_flags(struct folio *folio, unsigned n) { - VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page); - return &folio->page.flags; + struct page *page = &folio->page; + + VM_BUG_ON_PGFLAGS(PageTail(page), page); + VM_BUG_ON_PGFLAGS(n > 0 && !test_bit(PG_head, &page->flags), page); + return &page[n].flags; } /* @@ -262,48 +265,56 @@ static unsigned long *folio_flags(struct folio *folio) VM_BUG_ON_PGFLAGS(!PageHead(page), page); \ PF_POISONED_CHECK(&page[1]); }) +/* Which page is the flag stored in */ +#define FOLIO_PF_ANY 0 +#define FOLIO_PF_HEAD 0 +#define FOLIO_PF_ONLY_HEAD 0 +#define FOLIO_PF_NO_TAIL 0 +#define FOLIO_PF_NO_COMPOUND 0 +#define FOLIO_PF_SECOND 1 + /* * Macros to create function definitions for page flags */ #define TESTPAGEFLAG(uname, lname, policy) \ static __always_inline int Folio##uname(struct folio *folio) \ - { return test_bit(PG_##lname, folio_flags(folio)); } \ + { return test_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ static __always_inline int Page##uname(struct page *page) \ { return test_bit(PG_##lname, &policy(page, 0)->flags); } #define SETPAGEFLAG(uname, lname, policy) \ static __always_inline void SetFolio##uname(struct folio *folio) \ - { set_bit(PG_##lname, folio_flags(folio)); } \ + { set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ static __always_inline void SetPage##uname(struct page *page) \ { set_bit(PG_##lname, &policy(page, 1)->flags); } #define CLEARPAGEFLAG(uname, lname, policy) \ static __always_inline void ClearFolio##uname(struct folio *folio) \ - { clear_bit(PG_##lname, folio_flags(folio)); } \ + { clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ static __always_inline void ClearPage##uname(struct page *page) \ { clear_bit(PG_##lname, &policy(page, 1)->flags); } #define __SETPAGEFLAG(uname, lname, policy) \ static __always_inline void __SetFolio##uname(struct folio *folio) \ - { __set_bit(PG_##lname, folio_flags(folio)); } \ + { __set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ static __always_inline void __SetPage##uname(struct page *page) \ { __set_bit(PG_##lname, &policy(page, 1)->flags); } #define __CLEARPAGEFLAG(uname, lname, policy) \ static __always_inline void __ClearFolio##uname(struct folio *folio) \ - { __clear_bit(PG_##lname, folio_flags(folio)); } \ + { __clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ static __always_inline void __ClearPage##uname(struct page *page) \ { __clear_bit(PG_##lname, &policy(page, 1)->flags); } #define TESTSETFLAG(uname, lname, policy) \ static __always_inline int TestSetFolio##uname(struct folio *folio) \ - { return test_and_set_bit(PG_##lname, folio_flags(folio)); } \ + { return test_and_set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ static __always_inline int TestSetPage##uname(struct page *page) \ { return test_and_set_bit(PG_##lname, &policy(page, 1)->flags); } #define TESTCLEARFLAG(uname, lname, policy) \ static __always_inline int TestClearFolio##uname(struct folio *folio) \ - { return test_and_clear_bit(PG_##lname, folio_flags(folio)); } \ + { return test_and_clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ static __always_inline int TestClearPage##uname(struct page *page) \ { return test_and_clear_bit(PG_##lname, &policy(page, 1)->flags); } @@ -422,7 +433,7 @@ PAGEFLAG_FALSE(HighMem) static __always_inline bool FolioSwapCache(struct folio *folio) { return FolioSwapBacked(folio) && - test_bit(PG_swapcache, folio_flags(folio)); + test_bit(PG_swapcache, folio_flags(folio, 0)); } @@ -545,7 +556,7 @@ u64 stable_page_flags(struct page *page); static inline int FolioUptodate(struct folio *folio) { - int ret = test_bit(PG_uptodate, folio_flags(folio)); + int ret = test_bit(PG_uptodate, folio_flags(folio, 0)); /* * Must ensure that the data we read out of the page is loaded * _after_ we've loaded page->flags to check for PageUptodate. @@ -568,7 +579,7 @@ static inline int PageUptodate(struct page *page) static __always_inline void __SetFolioUptodate(struct folio *folio) { smp_wmb(); - __set_bit(PG_uptodate, folio_flags(folio)); + __set_bit(PG_uptodate, folio_flags(folio, 0)); } static __always_inline void SetFolioUptodate(struct folio *folio) @@ -579,7 +590,7 @@ static __always_inline void SetFolioUptodate(struct folio *folio) * uptodate are actually visible before PageUptodate becomes true. */ smp_wmb(); - set_bit(PG_uptodate, folio_flags(folio)); + set_bit(PG_uptodate, folio_flags(folio, 0)); } static __always_inline void __SetPageUptodate(struct page *page) @@ -672,6 +683,11 @@ static inline int PageTransHuge(struct page *page) return PageHead(page); } +static inline bool FolioTransHuge(struct folio *folio) +{ + return FolioHead(folio); +} + /* * PageTransCompound returns true for both transparent huge pages * and hugetlbfs pages, so it should only be called when it's known