From: Andreas Gruenbacher <agruenba@redhat.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: cluster-devel@redhat.com, Jan Kara <jack@suse.cz>,
Andreas Gruenbacher <agruenba@redhat.com>,
linux-kernel@vger.kernel.org,
Christoph Hellwig <hch@infradead.org>,
Alexander Viro <viro@zeniv.linux.org.uk>,
linux-fsdevel@vger.kernel.org, ocfs2-devel@oss.oracle.com
Subject: [Ocfs2-devel] [PATCH v2 5/6] iov_iter: Introduce ITER_FLAG_FAST_ONLY flag
Date: Mon, 19 Jul 2021 00:39:31 +0200 [thread overview]
Message-ID: <20210718223932.2703330-6-agruenba@redhat.com> (raw)
In-Reply-To: <20210718223932.2703330-1-agruenba@redhat.com>
Introduce a new ITER_FLAG_FAST_ONLY flag to indicate to get_user_pages to use
the FOLL_FAST_ONLY flag. This will cause get_user_pages to fail when it would
otherwise fault in a page.
Currently, the ITER_FLAG_FAST_ONLY flag is only checked in iov_iter_get_pages
and iov_iter_get_pages_alloc. This is enough for iomaop_dio_rw(), but it may
make sense to check for this flag in other contexts as well.
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
---
include/linux/uio.h | 15 ++++++++++++---
lib/iov_iter.c | 20 +++++++++++++++-----
2 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/include/linux/uio.h b/include/linux/uio.h
index 74f819c41735..d3d629c2153a 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -18,6 +18,9 @@ struct kvec {
};
enum iter_type {
+ /* set if get_user_pages should use FOLL_FAST_ONLY */
+ ITER_FLAG_FAST_ONLY = 2,
+
/* iter types */
ITER_IOVEC = 4,
ITER_KVEC = 8,
@@ -30,8 +33,9 @@ enum iter_type {
struct iov_iter {
/*
* Bit 0 is the read/write bit, set if we're writing.
- * Bit 1 is the BVEC_FLAG_NO_REF bit, set if type is a bvec and
- * the caller isn't expecting to drop a page reference when done.
+ * Bit 1 is the ITER_FLAG_FAST_ONLY bit, set if get_user_pages
+ * should use the FOLL_FAST_ONLY flag when trying to fault in pages
+ * (only useful for type ITER_IOVEC).
*/
unsigned int type;
size_t iov_offset;
@@ -55,7 +59,7 @@ struct iov_iter {
static inline enum iter_type iov_iter_type(const struct iov_iter *i)
{
- return i->type & ~(READ | WRITE);
+ return i->type & ~(READ | WRITE | ITER_FLAG_FAST_ONLY);
}
static inline bool iter_is_iovec(const struct iov_iter *i)
@@ -93,6 +97,11 @@ static inline unsigned char iov_iter_rw(const struct iov_iter *i)
return i->type & (READ | WRITE);
}
+static inline bool iov_iter_is_fast_only(const struct iov_iter *i)
+{
+ return i->type & ITER_FLAG_FAST_ONLY;
+}
+
/*
* Total number of bytes covered by an iovec.
*
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 3beecf8f77de..182ff2afed19 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -1538,6 +1538,8 @@ ssize_t iov_iter_get_pages(struct iov_iter *i,
struct page **pages, size_t maxsize, unsigned maxpages,
size_t *start)
{
+ unsigned int gup_flags = 0;
+
if (maxsize > i->count)
maxsize = i->count;
@@ -1548,6 +1550,11 @@ ssize_t iov_iter_get_pages(struct iov_iter *i,
if (unlikely(iov_iter_is_discard(i)))
return -EFAULT;
+ if (iov_iter_rw(i) != WRITE)
+ gup_flags |= FOLL_WRITE;
+ if (iov_iter_is_fast_only(i))
+ gup_flags |= FOLL_FAST_ONLY;
+
iterate_all_kinds(i, maxsize, v, ({
unsigned long addr = (unsigned long)v.iov_base;
size_t len = v.iov_len + (*start = addr & (PAGE_SIZE - 1));
@@ -1558,9 +1565,7 @@ ssize_t iov_iter_get_pages(struct iov_iter *i,
len = maxpages * PAGE_SIZE;
addr &= ~(PAGE_SIZE - 1);
n = DIV_ROUND_UP(len, PAGE_SIZE);
- res = get_user_pages_fast(addr, n,
- iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0,
- pages);
+ res = get_user_pages_fast(addr, n, gup_flags, pages);
if (unlikely(res < 0))
return res;
return (res == n ? len : res * PAGE_SIZE) - *start;
@@ -1665,6 +1670,7 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i,
struct page ***pages, size_t maxsize,
size_t *start)
{
+ unsigned int gup_flags = 0;
struct page **p;
if (maxsize > i->count)
@@ -1677,6 +1683,11 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i,
if (unlikely(iov_iter_is_discard(i)))
return -EFAULT;
+ if (iov_iter_rw(i) != WRITE)
+ gup_flags |= FOLL_WRITE;
+ if (iov_iter_is_fast_only(i))
+ gup_flags |= FOLL_FAST_ONLY;
+
iterate_all_kinds(i, maxsize, v, ({
unsigned long addr = (unsigned long)v.iov_base;
size_t len = v.iov_len + (*start = addr & (PAGE_SIZE - 1));
@@ -1688,8 +1699,7 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i,
p = get_pages_array(n);
if (!p)
return -ENOMEM;
- res = get_user_pages_fast(addr, n,
- iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, p);
+ res = get_user_pages_fast(addr, n, gup_flags, p);
if (unlikely(res < 0)) {
kvfree(p);
return res;
--
2.26.3
_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel@oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
next prev parent reply other threads:[~2021-07-18 22:43 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-18 22:39 [Ocfs2-devel] [PATCH v2 0/6] gfs2: Fix mmap + page fault deadlocks Andreas Gruenbacher
2021-07-18 22:39 ` [Ocfs2-devel] [PATCH v2 1/6] iov_iter: Introduce fault_in_iov_iter helper Andreas Gruenbacher
2021-07-19 19:26 ` Linus Torvalds
2021-07-18 22:39 ` [Ocfs2-devel] [PATCH v2 2/6] iomap: Fix iomap_dio_rw return value for page faults Andreas Gruenbacher
2021-07-18 22:39 ` [Ocfs2-devel] [PATCH v2 3/6] gfs2: Add wrapper for iomap_file_buffered_write Andreas Gruenbacher
2021-07-18 22:39 ` [Ocfs2-devel] [PATCH v2 4/6] gfs2: Fix mmap + page fault deadlocks for buffered I/O Andreas Gruenbacher
2021-07-18 22:39 ` Andreas Gruenbacher [this message]
2021-07-19 19:29 ` [Ocfs2-devel] [PATCH v2 5/6] iov_iter: Introduce ITER_FLAG_FAST_ONLY flag Linus Torvalds
2021-07-19 20:17 ` Al Viro
2021-07-18 22:39 ` [Ocfs2-devel] [PATCH v2 6/6] gfs2: Fix mmap + page fault deadlocks for direct I/O Andreas Gruenbacher
2021-07-19 19:31 ` [Ocfs2-devel] [PATCH v2 0/6] gfs2: Fix mmap + page fault deadlocks Linus Torvalds
2021-07-19 20:14 ` Al Viro
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210718223932.2703330-6-agruenba@redhat.com \
--to=agruenba@redhat.com \
--cc=cluster-devel@redhat.com \
--cc=hch@infradead.org \
--cc=jack@suse.cz \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=ocfs2-devel@oss.oracle.com \
--cc=torvalds@linux-foundation.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).