From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AE41C433EF for ; Fri, 4 Feb 2022 19:59:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EA4DB6B0098; Fri, 4 Feb 2022 14:59:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8423A6B0095; Fri, 4 Feb 2022 14:59:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA95C8D0001; Fri, 4 Feb 2022 14:59:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id DB7096B009A for ; Fri, 4 Feb 2022 14:59:06 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B2E386077C for ; Fri, 4 Feb 2022 19:59:06 +0000 (UTC) X-FDA: 79106161092.02.81EB93A Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf01.hostedemail.com (Postfix) with ESMTP id 567AC40003 for ; Fri, 4 Feb 2022 19:59:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=liAWeA8dPj0XyvqTIzZr5yVAs96w1tSEsr17cM/B6ps=; b=mLrvQwdsuODASoJwHF7X3NbFRr 5UAcM3rp05C4rFn/BPxrTMH22MjOM5WxibXx8N6aAqOptEa6fPpZQXcYLnQn522CYRloRJcskqvyP L5/mim8fJ7VTcrYwEdoAxn9/BaKv5gX7yi70ro9qzr88wDkA43wsm9vVtlg53+1IHdRKC8wjeXrTJ isO3fjBcGDjGRVwLJv49gCp1vAEq9QV2QyA2cKi1q8sHJnoD8CG7iLdOQdIZ6fxkqBit4uV9DQl18 PTHMgevbrwzmXPrhGe/sYj/hyzCLgzJd1byXDBDsgCFHrSz6srIt3t9v3uxct66YtMKrr+ZGQWak+ 8VE2w6IQ==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nG4jV-007Lm1-US; Fri, 04 Feb 2022 19:59:01 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, Christoph Hellwig , John Hubbard , Jason Gunthorpe , William Kucharski Subject: [PATCH 25/75] mm/gup: Turn compound_next() into gup_folio_next() Date: Fri, 4 Feb 2022 19:58:02 +0000 Message-Id: <20220204195852.1751729-26-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220204195852.1751729-1-willy@infradead.org> References: <20220204195852.1751729-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 567AC40003 X-Stat-Signature: 5pcc1bjx8h5p37yi4xu1tfqptmqw67zh X-Rspam-User: nil Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=mLrvQwds; spf=none (imf01.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-HE-Tag: 1644004746-377515 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert both callers to work on folios instead of pages. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard Reviewed-by: Jason Gunthorpe Reviewed-by: William Kucharski --- mm/gup.c | 40 +++++++++++++++++++++------------------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index bf196219c189..d90f8e5790c0 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -230,20 +230,19 @@ static inline struct page *compound_range_next(struct page *start, return page; } -static inline struct page *compound_next(struct page **list, +static inline struct folio *gup_folio_next(struct page **list, unsigned long npages, unsigned long i, unsigned int *ntails) { - struct page *page; + struct folio *folio = page_folio(list[i]); unsigned int nr; - page = compound_head(list[i]); for (nr = i + 1; nr < npages; nr++) { - if (compound_head(list[nr]) != page) + if (page_folio(list[nr]) != folio) break; } *ntails = nr - i; - return page; + return folio; } /** @@ -271,17 +270,17 @@ static inline struct page *compound_next(struct page **list, void unpin_user_pages_dirty_lock(struct page **pages, unsigned long npages, bool make_dirty) { - unsigned long index; - struct page *head; - unsigned int ntails; + unsigned long i; + struct folio *folio; + unsigned int nr; if (!make_dirty) { unpin_user_pages(pages, npages); return; } - for (index = 0; index < npages; index += ntails) { - head = compound_next(pages, npages, index, &ntails); + for (i = 0; i < npages; i += nr) { + folio = gup_folio_next(pages, npages, i, &nr); /* * Checking PageDirty at this point may race with * clear_page_dirty_for_io(), but that's OK. Two key @@ -302,9 +301,12 @@ void unpin_user_pages_dirty_lock(struct page **pages, unsigned long npages, * written back, so it gets written back again in the * next writeback cycle. This is harmless. */ - if (!PageDirty(head)) - set_page_dirty_lock(head); - put_compound_head(head, ntails, FOLL_PIN); + if (!folio_test_dirty(folio)) { + folio_lock(folio); + folio_mark_dirty(folio); + folio_unlock(folio); + } + gup_put_folio(folio, nr, FOLL_PIN); } } EXPORT_SYMBOL(unpin_user_pages_dirty_lock); @@ -357,9 +359,9 @@ EXPORT_SYMBOL(unpin_user_page_range_dirty_lock); */ void unpin_user_pages(struct page **pages, unsigned long npages) { - unsigned long index; - struct page *head; - unsigned int ntails; + unsigned long i; + struct folio *folio; + unsigned int nr; /* * If this WARN_ON() fires, then the system *might* be leaking pages (by @@ -369,9 +371,9 @@ void unpin_user_pages(struct page **pages, unsigned long npages) if (WARN_ON(IS_ERR_VALUE(npages))) return; - for (index = 0; index < npages; index += ntails) { - head = compound_next(pages, npages, index, &ntails); - put_compound_head(head, ntails, FOLL_PIN); + for (i = 0; i < npages; i += nr) { + folio = gup_folio_next(pages, npages, i, &nr); + gup_put_folio(folio, nr, FOLL_PIN); } } EXPORT_SYMBOL(unpin_user_pages); -- 2.34.1