From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7385C2D0E4 for ; Thu, 12 Nov 2020 21:26:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A4E12216C4 for ; Thu, 12 Nov 2020 21:26:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="f/+BuA+k" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A4E12216C4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3DF26B0070; Thu, 12 Nov 2020 16:26:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9770B6B0068; Thu, 12 Nov 2020 16:26:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 823676B006E; Thu, 12 Nov 2020 16:26:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0221.hostedemail.com [216.40.44.221]) by kanga.kvack.org (Postfix) with ESMTP id 4CD836B005C for ; Thu, 12 Nov 2020 16:26:53 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id DF12E33CD for ; Thu, 12 Nov 2020 21:26:52 +0000 (UTC) X-FDA: 77477051064.04.lock11_1c0274b27309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id C7A8780039D0 for ; Thu, 12 Nov 2020 21:26:52 +0000 (UTC) X-HE-Tag: lock11_1c0274b27309 X-Filterd-Recvd-Size: 8108 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=bTrDsmol5aUu0hNNGO8gHhFZiHT6t38oxGw0to6H7hI=; b=f/+BuA+klSCvPBZ5nrqnr6aDig tnqHLooQ2xnBfUL/bh+29e9RMw2V47ZfmV3cvxL7Ldhv16pkZp/7LVF67l3JVezO/pCdL59fSwqWW pBoJwjhNCAedzkvGZc/kh6aQvud1JKehsfneZ+Ygdp3S/mJqk4uWREZOTNs6L+M0nf/W2Zl0XxZMy BVFpA+vPKQrgk9CRFrxOcHLKm4g5Y4Jy5n8fEiy2s1e0l6SoR0DRMdXfJvGOeAraOyGb4Vh3wXGRk 6yx7wubk7J6+jplA8nhom201WsiHPTfJnw5OPB6PuO/sl3IKgrL0/jZ3FUYdOXjSLM5CSfz9O4XXd IDi9MOAg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7C-0007Ge-Tl; Thu, 12 Nov 2020 21:26:46 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 08/16] iomap: Use mapping_seek_hole_data Date: Thu, 12 Nov 2020 21:26:33 +0000 Message-Id: <20201112212641.27837-9-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Enhance mapping_seek_hole_data() to handle partially uptodate pages and convert the iomap seek code to call it. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/seek.c | 125 +++++------------------------------------------- mm/filemap.c | 37 ++++++++++++-- 2 files changed, 43 insertions(+), 119 deletions(-) diff --git a/fs/iomap/seek.c b/fs/iomap/seek.c index 107ee80c3568..dab1b02eba5b 100644 --- a/fs/iomap/seek.c +++ b/fs/iomap/seek.c @@ -10,122 +10,17 @@ #include #include =20 -/* - * Seek for SEEK_DATA / SEEK_HOLE within @page, starting at @lastoff. - * Returns true if found and updates @lastoff to the offset in file. - */ -static bool -page_seek_hole_data(struct inode *inode, struct page *page, loff_t *last= off, - int whence) -{ - const struct address_space_operations *ops =3D inode->i_mapping->a_ops; - unsigned int bsize =3D i_blocksize(inode), off; - bool seek_data =3D whence =3D=3D SEEK_DATA; - loff_t poff =3D page_offset(page); - - if (WARN_ON_ONCE(*lastoff >=3D poff + PAGE_SIZE)) - return false; - - if (*lastoff < poff) { - /* - * Last offset smaller than the start of the page means we found - * a hole: - */ - if (whence =3D=3D SEEK_HOLE) - return true; - *lastoff =3D poff; - } - - /* - * Just check the page unless we can and should check block ranges: - */ - if (bsize =3D=3D PAGE_SIZE || !ops->is_partially_uptodate) - return PageUptodate(page) =3D=3D seek_data; - - lock_page(page); - if (unlikely(page->mapping !=3D inode->i_mapping)) - goto out_unlock_not_found; - - for (off =3D 0; off < PAGE_SIZE; off +=3D bsize) { - if (offset_in_page(*lastoff) >=3D off + bsize) - continue; - if (ops->is_partially_uptodate(page, off, bsize) =3D=3D seek_data) { - unlock_page(page); - return true; - } - *lastoff =3D poff + off + bsize; - } - -out_unlock_not_found: - unlock_page(page); - return false; -} - -/* - * Seek for SEEK_DATA / SEEK_HOLE in the page cache. - * - * Within unwritten extents, the page cache determines which parts are h= oles - * and which are data: uptodate buffer heads count as data; everything e= lse - * counts as a hole. - * - * Returns the resulting offset on successs, and -ENOENT otherwise. - */ static loff_t -page_cache_seek_hole_data(struct inode *inode, loff_t offset, loff_t len= gth, - int whence) -{ - pgoff_t index =3D offset >> PAGE_SHIFT; - pgoff_t end =3D DIV_ROUND_UP(offset + length, PAGE_SIZE); - loff_t lastoff =3D offset; - struct pagevec pvec; - - if (length <=3D 0) - return -ENOENT; - - pagevec_init(&pvec); - - do { - unsigned nr_pages, i; - - nr_pages =3D pagevec_lookup_range(&pvec, inode->i_mapping, &index, - end - 1); - if (nr_pages =3D=3D 0) - break; - - for (i =3D 0; i < nr_pages; i++) { - struct page *page =3D pvec.pages[i]; - - if (page_seek_hole_data(inode, page, &lastoff, whence)) - goto check_range; - lastoff =3D page_offset(page) + PAGE_SIZE; - } - pagevec_release(&pvec); - } while (index < end); - - /* When no page at lastoff and we are not done, we found a hole. */ - if (whence !=3D SEEK_HOLE) - goto not_found; - -check_range: - if (lastoff < offset + length) - goto out; -not_found: - lastoff =3D -ENOENT; -out: - pagevec_release(&pvec); - return lastoff; -} - - -static loff_t -iomap_seek_hole_actor(struct inode *inode, loff_t offset, loff_t length, +iomap_seek_hole_actor(struct inode *inode, loff_t start, loff_t length, void *data, struct iomap *iomap, struct iomap *srcmap) { + loff_t offset =3D start; + switch (iomap->type) { case IOMAP_UNWRITTEN: - offset =3D page_cache_seek_hole_data(inode, offset, length, - SEEK_HOLE); - if (offset < 0) + offset =3D mapping_seek_hole_data(inode->i_mapping, start, + start + length, SEEK_HOLE); + if (offset =3D=3D start + length) return length; fallthrough; case IOMAP_HOLE: @@ -164,15 +59,17 @@ iomap_seek_hole(struct inode *inode, loff_t offset, = const struct iomap_ops *ops) EXPORT_SYMBOL_GPL(iomap_seek_hole); =20 static loff_t -iomap_seek_data_actor(struct inode *inode, loff_t offset, loff_t length, +iomap_seek_data_actor(struct inode *inode, loff_t start, loff_t length, void *data, struct iomap *iomap, struct iomap *srcmap) { + loff_t offset =3D start; + switch (iomap->type) { case IOMAP_HOLE: return length; case IOMAP_UNWRITTEN: - offset =3D page_cache_seek_hole_data(inode, offset, length, - SEEK_DATA); + offset =3D mapping_seek_hole_data(inode->i_mapping, start, + start + length, SEEK_DATA); if (offset < 0) return length; fallthrough; diff --git a/mm/filemap.c b/mm/filemap.c index ab7103eb7e11..ef7411ea3f91 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2586,11 +2586,36 @@ generic_file_read_iter(struct kiocb *iocb, struct= iov_iter *iter) } EXPORT_SYMBOL(generic_file_read_iter); =20 -static inline bool page_seek_match(struct page *page, bool seek_data) +static inline loff_t page_seek_hole_data(struct xa_state *xas, + struct address_space *mapping, struct page *page, + loff_t start, loff_t end, bool seek_data) { + const struct address_space_operations *ops =3D mapping->a_ops; + size_t offset, bsz =3D i_blocksize(mapping->host); + if (xa_is_value(page) || PageUptodate(page)) - return seek_data; - return !seek_data; + return seek_data ? start : end; + if (!ops->is_partially_uptodate) + return seek_data ? end : start; + + xas_pause(xas); + rcu_read_unlock(); + lock_page(page); + if (unlikely(page->mapping !=3D mapping)) + goto unlock; + + offset =3D offset_in_thp(page, start) & ~(bsz - 1); + + do { + if (ops->is_partially_uptodate(page, offset, bsz) =3D=3D seek_data) + break; + start =3D (start + bsz) & ~(bsz - 1); + offset +=3D bsz; + } while (offset < thp_size(page)); +unlock: + unlock_page(page); + rcu_read_lock(); + return start; } =20 static inline @@ -2640,9 +2665,11 @@ loff_t mapping_seek_hole_data(struct address_space= *mapping, loff_t start, start =3D pos; } =20 - if (page_seek_match(page, seek_data)) + pos +=3D seek_page_size(&xas, page); + start =3D page_seek_hole_data(&xas, mapping, page, start, pos, + seek_data); + if (start < pos) goto unlock; - start =3D pos + seek_page_size(&xas, page); put_page(page); } rcu_read_unlock(); --=20 2.28.0