v9fs.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
From: David Howells <dhowells@redhat.com>
To: Jeff Layton <jlayton@kernel.org>, Steve French <smfrench@gmail.com>
Cc: David Howells <dhowells@redhat.com>,
	Matthew Wilcox <willy@infradead.org>,
	Marc Dionne <marc.dionne@auristor.com>,
	Paulo Alcantara <pc@manguebit.com>,
	Shyam Prasad N <sprasad@microsoft.com>,
	Tom Talpey <tom@talpey.com>,
	Dominique Martinet <asmadeus@codewreck.org>,
	Eric Van Hensbergen <ericvh@kernel.org>,
	Ilya Dryomov <idryomov@gmail.com>,
	Christian Brauner <christian@brauner.io>,
	linux-cachefs@redhat.com, linux-afs@lists.infradead.org,
	linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org,
	ceph-devel@vger.kernel.org, v9fs@lists.linux.dev,
	linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH v5 17/40] netfs: Add func to calculate pagecount/size-limited span of an iterator
Date: Thu, 21 Dec 2023 13:23:12 +0000	[thread overview]
Message-ID: <20231221132400.1601991-18-dhowells@redhat.com> (raw)
In-Reply-To: <20231221132400.1601991-1-dhowells@redhat.com>

Add a function to work out how much of an ITER_BVEC or ITER_XARRAY iterator
we can use in a pagecount-limited and size-limited span.  This will be
used, for example, to limit the number of segments in a subrequest to the
maximum number of elements that an RDMA transfer can handle.

Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
---
 fs/netfs/iterator.c   | 97 +++++++++++++++++++++++++++++++++++++++++++
 include/linux/netfs.h |  2 +
 2 files changed, 99 insertions(+)

diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index 2ff07ba655a0..b781bbbf1d8d 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -101,3 +101,100 @@ ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
 	return npages;
 }
 EXPORT_SYMBOL_GPL(netfs_extract_user_iter);
+
+/*
+ * Select the span of a bvec iterator we're going to use.  Limit it by both maximum
+ * size and maximum number of segments.  Returns the size of the span in bytes.
+ */
+static size_t netfs_limit_bvec(const struct iov_iter *iter, size_t start_offset,
+			       size_t max_size, size_t max_segs)
+{
+	const struct bio_vec *bvecs = iter->bvec;
+	unsigned int nbv = iter->nr_segs, ix = 0, nsegs = 0;
+	size_t len, span = 0, n = iter->count;
+	size_t skip = iter->iov_offset + start_offset;
+
+	if (WARN_ON(!iov_iter_is_bvec(iter)) ||
+	    WARN_ON(start_offset > n) ||
+	    n == 0)
+		return 0;
+
+	while (n && ix < nbv && skip) {
+		len = bvecs[ix].bv_len;
+		if (skip < len)
+			break;
+		skip -= len;
+		n -= len;
+		ix++;
+	}
+
+	while (n && ix < nbv) {
+		len = min3(n, bvecs[ix].bv_len - skip, max_size);
+		span += len;
+		nsegs++;
+		ix++;
+		if (span >= max_size || nsegs >= max_segs)
+			break;
+		skip = 0;
+		n -= len;
+	}
+
+	return min(span, max_size);
+}
+
+/*
+ * Select the span of an xarray iterator we're going to use.  Limit it by both
+ * maximum size and maximum number of segments.  It is assumed that segments
+ * can be larger than a page in size, provided they're physically contiguous.
+ * Returns the size of the span in bytes.
+ */
+static size_t netfs_limit_xarray(const struct iov_iter *iter, size_t start_offset,
+				 size_t max_size, size_t max_segs)
+{
+	struct folio *folio;
+	unsigned int nsegs = 0;
+	loff_t pos = iter->xarray_start + iter->iov_offset;
+	pgoff_t index = pos / PAGE_SIZE;
+	size_t span = 0, n = iter->count;
+
+	XA_STATE(xas, iter->xarray, index);
+
+	if (WARN_ON(!iov_iter_is_xarray(iter)) ||
+	    WARN_ON(start_offset > n) ||
+	    n == 0)
+		return 0;
+	max_size = min(max_size, n - start_offset);
+
+	rcu_read_lock();
+	xas_for_each(&xas, folio, ULONG_MAX) {
+		size_t offset, flen, len;
+		if (xas_retry(&xas, folio))
+			continue;
+		if (WARN_ON(xa_is_value(folio)))
+			break;
+		if (WARN_ON(folio_test_hugetlb(folio)))
+			break;
+
+		flen = folio_size(folio);
+		offset = offset_in_folio(folio, pos);
+		len = min(max_size, flen - offset);
+		span += len;
+		nsegs++;
+		if (span >= max_size || nsegs >= max_segs)
+			break;
+	}
+
+	rcu_read_unlock();
+	return min(span, max_size);
+}
+
+size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
+			size_t max_size, size_t max_segs)
+{
+	if (iov_iter_is_bvec(iter))
+		return netfs_limit_bvec(iter, start_offset, max_size, max_segs);
+	if (iov_iter_is_xarray(iter))
+		return netfs_limit_xarray(iter, start_offset, max_size, max_segs);
+	BUG();
+}
+EXPORT_SYMBOL(netfs_limit_iter);
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index c05150f51beb..41d438e5d6db 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -322,6 +322,8 @@ void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
 ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
 				struct iov_iter *new,
 				iov_iter_extraction_t extraction_flags);
+size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
+			size_t max_size, size_t max_segs);
 
 int netfs_start_io_read(struct inode *inode);
 void netfs_end_io_read(struct inode *inode);


  parent reply	other threads:[~2023-12-21 13:25 UTC|newest]

Thread overview: 75+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-12-21 13:22 [PATCH v5 00/40] netfs, afs, 9p: Delegate high-level I/O to netfslib David Howells
2023-12-21 13:22 ` [PATCH v5 01/40] afs: Remove whitespace before most ')' from the trace header David Howells
2023-12-21 13:22 ` [PATCH v5 02/40] afs: Automatically generate trace tag enums David Howells
2023-12-21 13:22 ` [PATCH v5 03/40] netfs, fscache: Move fs/fscache/* into fs/netfs/ David Howells
2023-12-21 13:22 ` [PATCH v5 04/40] netfs, fscache: Combine fscache with netfs David Howells
2023-12-21 13:23 ` [PATCH v5 05/40] netfs, fscache: Remove ->begin_cache_operation David Howells
2023-12-21 13:23 ` [PATCH v5 06/40] netfs, fscache: Move /proc/fs/fscache to /proc/fs/netfs and put in a symlink David Howells
2024-01-03 16:49   ` Marc Dionne
2023-12-21 13:23 ` [PATCH v5 07/40] netfs: Move pinning-for-writeback from fscache to netfs David Howells
2023-12-21 13:23 ` [PATCH v5 08/40] netfs: Add a procfile to list in-progress requests David Howells
2023-12-21 13:23 ` [PATCH v5 09/40] netfs: Allow the netfs to make the io (sub)request alloc larger David Howells
2023-12-21 13:23 ` [PATCH v5 10/40] netfs: Add a ->free_subrequest() op David Howells
2023-12-21 13:23 ` [PATCH v5 11/40] afs: Don't use folio->private to record partial modification David Howells
2023-12-21 13:23 ` [PATCH v5 12/40] netfs: Provide invalidate_folio and release_folio calls David Howells
2023-12-21 13:23 ` [PATCH v5 13/40] netfs: Implement unbuffered/DIO vs buffered I/O locking David Howells
2023-12-21 13:23 ` [PATCH v5 14/40] netfs: Add iov_iters to (sub)requests to describe various buffers David Howells
2023-12-21 13:23 ` [PATCH v5 15/40] netfs: Add support for DIO buffering David Howells
2023-12-26 16:54   ` Nathan Chancellor
2023-12-28 10:47     ` Christian Brauner
2023-12-28 16:58       ` Nathan Chancellor
2023-12-21 13:23 ` [PATCH v5 16/40] netfs: Provide tools to create a buffer in an xarray David Howells
2023-12-21 13:23 ` David Howells [this message]
2023-12-21 13:23 ` [PATCH v5 18/40] netfs: Limit subrequest by size or number of segments David Howells
2023-12-21 13:23 ` [PATCH v5 19/40] netfs: Extend the netfs_io_*request structs to handle writes David Howells
2023-12-21 13:23 ` [PATCH v5 20/40] netfs: Add a hook to allow tell the netfs to update its i_size David Howells
2023-12-21 13:23 ` [PATCH v5 21/40] netfs: Make netfs_put_request() handle a NULL pointer David Howells
2023-12-21 13:23 ` [PATCH v5 22/40] netfs: Make the refcounting of netfs_begin_read() easier to use David Howells
2023-12-21 13:23 ` [PATCH v5 23/40] netfs: Prep to use folio->private for write grouping and streaming write David Howells
2023-12-21 13:23 ` [PATCH v5 24/40] netfs: Dispatch write requests to process a writeback slice David Howells
2023-12-21 13:23 ` [PATCH v5 25/40] netfs: Provide func to copy data to pagecache for buffered write David Howells
2023-12-21 13:23 ` [PATCH v5 26/40] netfs: Make netfs_read_folio() handle streaming-write pages David Howells
2023-12-21 13:23 ` [PATCH v5 27/40] netfs: Allocate multipage folios in the writepath David Howells
2023-12-21 13:23 ` [PATCH v5 28/40] netfs: Implement unbuffered/DIO read support David Howells
2023-12-21 13:23 ` [PATCH v5 29/40] netfs: Implement unbuffered/DIO write support David Howells
2023-12-21 13:23 ` [PATCH v5 30/40] netfs: Implement buffered write API David Howells
2023-12-21 13:23 ` [PATCH v5 31/40] netfs: Allow buffered shared-writeable mmap through netfs_page_mkwrite() David Howells
2023-12-21 13:23 ` [PATCH v5 32/40] netfs: Provide netfs_file_read_iter() David Howells
2023-12-21 13:23 ` [PATCH v5 33/40] netfs, cachefiles: Pass upper bound length to allow expansion David Howells
2024-01-02 14:04   ` Gao Xiang
2024-01-02 17:11   ` David Howells
2024-01-02 20:37     ` Gao Xiang
2024-01-03 10:15     ` Jia Zhu
2023-12-21 13:23 ` [PATCH v5 34/40] netfs: Provide a writepages implementation David Howells
2023-12-21 13:23 ` [PATCH v5 35/40] netfs: Provide a launder_folio implementation David Howells
2023-12-21 13:23 ` [PATCH v5 36/40] netfs: Implement a write-through caching option David Howells
2023-12-21 13:23 ` [PATCH v5 37/40] netfs: Optimise away reads above the point at which there can be no data David Howells
2023-12-21 23:01   ` Nathan Chancellor
2023-12-22 11:49   ` David Howells
2023-12-22 12:00   ` [PATCH] Fix oops in NFS David Howells
2024-01-05  4:52     ` Matthew Wilcox
2024-01-05 10:12     ` David Howells
2024-01-05 13:17       ` Matthew Wilcox
2024-01-05 17:20         ` Dominique Martinet
2024-01-05 11:48     ` David Howells
2024-01-05 14:33     ` David Howells
2023-12-21 13:23 ` [PATCH v5 38/40] netfs: Export the netfs_sreq tracepoint David Howells
2023-12-21 13:23 ` [PATCH v5 39/40] afs: Use the netfs write helpers David Howells
2023-12-21 13:23 ` [PATCH v5 40/40] 9p: Use netfslib read/write_iter David Howells
2024-01-03  7:22   ` Dominique Martinet
2024-01-03 19:52     ` Eric Van Hensbergen
2024-01-03 12:08   ` David Howells
2024-01-03 12:39   ` David Howells
2024-01-03 13:00     ` Dominique Martinet
2024-05-09 17:15   ` Andrea Righi
2024-05-09 21:33   ` David Howells
2024-05-10  5:53     ` Andrea Righi
2024-05-10  7:57     ` David Howells
2023-12-22 13:02 ` [PATCH] Fix EROFS Kconfig David Howells
2023-12-23  3:55   ` Jingbo Xu
2023-12-23 13:32     ` Gao Xiang
2024-01-02 15:39 ` [PATCH v5 40/40] 9p: Use netfslib read/write_iter David Howells
2024-01-02 21:49 ` [PATCH] 9p: Fix initialisation of netfs_inode for 9p David Howells
2024-01-03 13:10   ` Dominique Martinet
2024-01-03 13:59   ` David Howells
2024-01-03 14:04   ` David Howells

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231221132400.1601991-18-dhowells@redhat.com \
    --to=dhowells@redhat.com \
    --cc=asmadeus@codewreck.org \
    --cc=ceph-devel@vger.kernel.org \
    --cc=christian@brauner.io \
    --cc=ericvh@kernel.org \
    --cc=idryomov@gmail.com \
    --cc=jlayton@kernel.org \
    --cc=linux-afs@lists.infradead.org \
    --cc=linux-cachefs@redhat.com \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=marc.dionne@auristor.com \
    --cc=netdev@vger.kernel.org \
    --cc=pc@manguebit.com \
    --cc=smfrench@gmail.com \
    --cc=sprasad@microsoft.com \
    --cc=tom@talpey.com \
    --cc=v9fs@lists.linux.dev \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).