From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38403C433B4 for ; Wed, 5 May 2021 17:26:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BC46C61073 for ; Wed, 5 May 2021 17:26:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BC46C61073 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 446626B006E; Wed, 5 May 2021 13:26:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 41C9B6B0070; Wed, 5 May 2021 13:26:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 296076B0071; Wed, 5 May 2021 13:26:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0222.hostedemail.com [216.40.44.222]) by kanga.kvack.org (Postfix) with ESMTP id 0E45E6B006E for ; Wed, 5 May 2021 13:26:09 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C28E0181AC553 for ; Wed, 5 May 2021 17:26:08 +0000 (UTC) X-FDA: 78107855616.06.E832682 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf13.hostedemail.com (Postfix) with ESMTP id 38392E000111 for ; Wed, 5 May 2021 17:25:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=0l28kIgMlAZaNOk08mb6vdiJ5NziLnqKkRr/zi3PWbg=; b=STXgvkisM4vRAhZhySClHXH8o7 CQE2sdRO3fDCUJ+Ejw924a0blC8eG4cADYaDonwZwrZMYFk4nlnReojSYpB6mu7fnQ5mU8W56ONIb 0RUQX9VREUbYTNr9+ul3X/OSvihegzjvaVxxQRrDbbdnK03O57OuyfPphv6WM+hca2yLm/eERc2n2 /TAlwGlA50XuxNbO+cdbzKabdtXpSCtZ8rHIKVpJSb4pcvLqKmMXoj/OyY3mf6DmHUj8g308Uj/ZD ks/Gn0D4EZ/CmHwknscUDeF7VeQwfrIJmPMTFnNCgzOJtZplBn4z+H0gjgoDv5sS6OuNSOW4zVVmN dpQGwGkA==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1leLCx-000dmc-0Z; Wed, 05 May 2021 17:21:44 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org Subject: [PATCH v9 88/96] iomap: Use folio offsets instead of page offsets Date: Wed, 5 May 2021 16:06:20 +0100 Message-Id: <20210505150628.111735-89-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210505150628.111735-1-willy@infradead.org> References: <20210505150628.111735-1-willy@infradead.org> MIME-Version: 1.0 Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=STXgvkis; dmarc=none; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Stat-Signature: 3a3zbnoykdbafwmb734ox5cmucm35wku X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 38392E000111 Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1620235556-809658 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Pass a folio around instead of the page, and make sure the offset is relative to the start of the folio instead of the start of a page. Also use size_t for offset & length to make it clear that these are byte counts, and to support >2GB folios in the future. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 85 ++++++++++++++++++++++-------------------- 1 file changed, 44 insertions(+), 41 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index f40a22a696c6..b61fc7da3d65 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -75,18 +75,18 @@ static void iomap_page_release(struct folio *folio) } =20 /* - * Calculate the range inside the page that we actually need to read. + * Calculate the range inside the folio that we actually need to read. */ -static void -iomap_adjust_read_range(struct inode *inode, struct iomap_page *iop, - loff_t *pos, loff_t length, unsigned *offp, unsigned *lenp) +static void iomap_adjust_read_range(struct inode *inode, struct folio *f= olio, + loff_t *pos, loff_t length, size_t *offp, size_t *lenp) { + struct iomap_page *iop =3D to_iomap_page(folio); loff_t orig_pos =3D *pos; loff_t isize =3D i_size_read(inode); unsigned block_bits =3D inode->i_blkbits; unsigned block_size =3D (1 << block_bits); - unsigned poff =3D offset_in_page(*pos); - unsigned plen =3D min_t(loff_t, PAGE_SIZE - poff, length); + size_t poff =3D offset_in_folio(folio, *pos); + size_t plen =3D min_t(loff_t, folio_size(folio) - poff, length); unsigned first =3D poff >> block_bits; unsigned last =3D (poff + plen - 1) >> block_bits; =20 @@ -124,7 +124,7 @@ iomap_adjust_read_range(struct inode *inode, struct i= omap_page *iop, * page cache for blocks that are entirely outside of i_size. */ if (orig_pos <=3D isize && orig_pos + length > isize) { - unsigned end =3D offset_in_page(isize - 1) >> block_bits; + unsigned end =3D offset_in_folio(folio, isize - 1) >> block_bits; =20 if (first <=3D end && last > end) plen -=3D (last - end) * block_size; @@ -134,31 +134,31 @@ iomap_adjust_read_range(struct inode *inode, struct= iomap_page *iop, *lenp =3D plen; } =20 -static void iomap_iop_set_range_uptodate(struct page *page, - struct iomap_page *iop, unsigned off, unsigned len) +static void iomap_iop_set_range_uptodate(struct folio *folio, + struct iomap_page *iop, size_t off, size_t len) { - struct inode *inode =3D page->mapping->host; + struct inode *inode =3D folio->mapping->host; unsigned first =3D off >> inode->i_blkbits; unsigned last =3D (off + len - 1) >> inode->i_blkbits; unsigned long flags; =20 spin_lock_irqsave(&iop->uptodate_lock, flags); bitmap_set(iop->uptodate, first, last - first + 1); - if (bitmap_full(iop->uptodate, i_blocks_per_page(inode, page))) - SetPageUptodate(page); + if (bitmap_full(iop->uptodate, i_blocks_per_folio(inode, folio))) + folio_mark_uptodate(folio); spin_unlock_irqrestore(&iop->uptodate_lock, flags); } =20 -static void iomap_set_range_uptodate(struct page *page, - struct iomap_page *iop, unsigned off, unsigned len) +static void iomap_set_range_uptodate(struct folio *folio, + struct iomap_page *iop, size_t off, size_t len) { - if (PageError(page)) + if (folio_error(folio)) return; =20 if (iop) - iomap_iop_set_range_uptodate(page, iop, off, len); + iomap_iop_set_range_uptodate(folio, iop, off, len); else - SetPageUptodate(page); + folio_mark_uptodate(folio); } =20 static void @@ -169,15 +169,17 @@ iomap_read_page_end_io(struct bio_vec *bvec, int er= ror) struct iomap_page *iop =3D to_iomap_page(folio); =20 if (unlikely(error)) { - ClearPageUptodate(page); - SetPageError(page); + folio_clear_uptodate_flag(folio); + folio_set_error_flag(folio); } else { - iomap_set_range_uptodate(page, iop, bvec->bv_offset, - bvec->bv_len); + size_t off =3D (page - &folio->page) * PAGE_SIZE + + bvec->bv_offset; + + iomap_set_range_uptodate(folio, iop, off, bvec->bv_len); } =20 if (!iop || atomic_sub_and_test(bvec->bv_len, &iop->read_bytes_pending)= ) - unlock_page(page); + folio_unlock(folio); } =20 static void @@ -237,7 +239,7 @@ iomap_readpage_actor(struct inode *inode, loff_t pos,= loff_t length, void *data, struct iomap_page *iop =3D iomap_page_create(inode, folio); bool same_page =3D false, is_contig =3D false; loff_t orig_pos =3D pos; - unsigned poff, plen; + size_t poff, plen; sector_t sector; =20 if (iomap->type =3D=3D IOMAP_INLINE) { @@ -246,14 +248,14 @@ iomap_readpage_actor(struct inode *inode, loff_t po= s, loff_t length, void *data, return PAGE_SIZE; } =20 - /* zero post-eof blocks as the page may be mapped */ - iomap_adjust_read_range(inode, iop, &pos, length, &poff, &plen); + /* zero post-eof blocks as the folio may be mapped */ + iomap_adjust_read_range(inode, folio, &pos, length, &poff, &plen); if (plen =3D=3D 0) goto done; =20 if (iomap_block_needs_zeroing(inode, iomap, pos)) { - zero_user(page, poff, plen); - iomap_set_range_uptodate(page, iop, poff, plen); + zero_user(&folio->page, poff, plen); + iomap_set_range_uptodate(folio, iop, poff, plen); goto done; } =20 @@ -264,7 +266,7 @@ iomap_readpage_actor(struct inode *inode, loff_t pos,= loff_t length, void *data, /* Try to merge into a previous segment if we can */ sector =3D iomap_sector(iomap, pos); if (ctx->bio && bio_end_sector(ctx->bio) =3D=3D sector) { - if (__bio_try_merge_page(ctx->bio, page, plen, poff, + if (__bio_try_merge_page(ctx->bio, &folio->page, plen, poff, &same_page)) goto done; is_contig =3D true; @@ -296,7 +298,7 @@ iomap_readpage_actor(struct inode *inode, loff_t pos,= loff_t length, void *data, ctx->bio->bi_end_io =3D iomap_read_end_io; } =20 - bio_add_page(ctx->bio, page, plen, poff); + bio_add_folio(ctx->bio, folio, plen, poff); done: /* * Move the caller beyond our range so that it keeps making progress. @@ -531,9 +533,8 @@ iomap_write_failed(struct inode *inode, loff_t pos, u= nsigned len) truncate_pagecache_range(inode, max(pos, i_size), pos + len); } =20 -static int -iomap_read_page_sync(loff_t block_start, struct page *page, unsigned pof= f, - unsigned plen, struct iomap *iomap) +static int iomap_read_folio_sync(loff_t block_start, struct folio *folio= , + size_t poff, size_t plen, struct iomap *iomap) { struct bio_vec bvec; struct bio bio; @@ -542,7 +543,7 @@ iomap_read_page_sync(loff_t block_start, struct page = *page, unsigned poff, bio.bi_opf =3D REQ_OP_READ; bio.bi_iter.bi_sector =3D iomap_sector(iomap, block_start); bio_set_dev(&bio, iomap->bdev); - __bio_add_page(&bio, page, plen, poff); + bio_add_folio(&bio, folio, plen, poff); return submit_bio_wait(&bio); } =20 @@ -555,14 +556,15 @@ __iomap_write_begin(struct inode *inode, loff_t pos= , unsigned len, int flags, loff_t block_size =3D i_blocksize(inode); loff_t block_start =3D round_down(pos, block_size); loff_t block_end =3D round_up(pos + len, block_size); - unsigned from =3D offset_in_page(pos), to =3D from + len, poff, plen; + size_t from =3D offset_in_folio(folio, pos), to =3D from + len; + size_t poff, plen; =20 - if (PageUptodate(page)) + if (folio_uptodate(folio)) return 0; - ClearPageError(page); + folio_clear_error_flag(folio); =20 do { - iomap_adjust_read_range(inode, iop, &block_start, + iomap_adjust_read_range(inode, folio, &block_start, block_end - block_start, &poff, &plen); if (plen =3D=3D 0) break; @@ -575,14 +577,15 @@ __iomap_write_begin(struct inode *inode, loff_t pos= , unsigned len, int flags, if (iomap_block_needs_zeroing(inode, srcmap, block_start)) { if (WARN_ON_ONCE(flags & IOMAP_WRITE_F_UNSHARE)) return -EIO; - zero_user_segments(page, poff, from, to, poff + plen); + zero_user_segments(&folio->page, poff, from, to, + poff + plen); } else { - int status =3D iomap_read_page_sync(block_start, page, + int status =3D iomap_read_folio_sync(block_start, folio, poff, plen, srcmap); if (status) return status; } - iomap_set_range_uptodate(page, iop, poff, plen); + iomap_set_range_uptodate(folio, iop, poff, plen); } while ((block_start +=3D plen) < block_end); =20 return 0; @@ -686,7 +689,7 @@ static size_t __iomap_write_end(struct inode *inode, = loff_t pos, size_t len, */ if (unlikely(copied < len && !PageUptodate(page))) return 0; - iomap_set_range_uptodate(page, iop, offset_in_page(pos), len); + iomap_set_range_uptodate(folio, iop, offset_in_folio(folio, pos), len); iomap_set_page_dirty(page); return copied; } --=20 2.30.2