From: Jens Axboe <axboe@kernel.dk>
To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
linux-block@vger.kernel.org
Cc: willy@infradead.org, clm@fb.com, torvalds@linux-foundation.org,
david@fromorbit.com, Jens Axboe <axboe@kernel.dk>
Subject: [PATCH 3/6] mm: make buffered writes work with RWF_UNCACHED
Date: Tue, 17 Dec 2019 07:39:45 -0700 [thread overview]
Message-ID: <20191217143948.26380-4-axboe@kernel.dk> (raw)
In-Reply-To: <20191217143948.26380-1-axboe@kernel.dk>
If RWF_UNCACHED is set for io_uring (or pwritev2(2)), we'll drop the
cache instantiated for buffered writes. If new pages aren't
instantiated, we leave them alone. This provides similar semantics to
reads with RWF_UNCACHED set.
Signed-off-by: Jens Axboe <axboe@kernel.dk>
---
include/linux/fs.h | 1 +
mm/filemap.c | 41 +++++++++++++++++++++++++++++++++++++++--
2 files changed, 40 insertions(+), 2 deletions(-)
diff --git a/include/linux/fs.h b/include/linux/fs.h
index bf58db1bc032..5ea5fc167524 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -285,6 +285,7 @@ enum positive_aop_returns {
#define AOP_FLAG_NOFS 0x0002 /* used by filesystem to direct
* helper code (eg buffer layer)
* to clear GFP_FS from alloc */
+#define AOP_FLAG_UNCACHED 0x0004
/*
* oh the beauties of C type declarations.
diff --git a/mm/filemap.c b/mm/filemap.c
index 522152ed86d8..0b5f29b30c34 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -3277,10 +3277,12 @@ struct page *grab_cache_page_write_begin(struct address_space *mapping,
pgoff_t index, unsigned flags)
{
struct page *page;
- int fgp_flags = FGP_LOCK|FGP_WRITE|FGP_CREAT;
+ int fgp_flags = FGP_LOCK|FGP_WRITE;
if (flags & AOP_FLAG_NOFS)
fgp_flags |= FGP_NOFS;
+ if (!(flags & AOP_FLAG_UNCACHED))
+ fgp_flags |= FGP_CREAT;
page = pagecache_get_page(mapping, index, fgp_flags,
mapping_gfp_mask(mapping));
@@ -3301,6 +3303,9 @@ ssize_t generic_perform_write(struct file *file,
ssize_t written = 0;
unsigned int flags = 0;
+ if (iocb->ki_flags & IOCB_UNCACHED)
+ flags |= AOP_FLAG_UNCACHED;
+
do {
struct page *page;
unsigned long offset; /* Offset into pagecache page */
@@ -3333,10 +3338,16 @@ ssize_t generic_perform_write(struct file *file,
break;
}
+retry:
status = a_ops->write_begin(file, mapping, pos, bytes, flags,
&page, &fsdata);
- if (unlikely(status < 0))
+ if (unlikely(status < 0)) {
+ if (status == -ENOMEM && (flags & AOP_FLAG_UNCACHED)) {
+ flags &= ~AOP_FLAG_UNCACHED;
+ goto retry;
+ }
break;
+ }
if (mapping_writably_mapped(mapping))
flush_dcache_page(page);
@@ -3372,6 +3383,32 @@ ssize_t generic_perform_write(struct file *file,
balance_dirty_pages_ratelimited(mapping);
} while (iov_iter_count(i));
+ if (written && (iocb->ki_flags & IOCB_UNCACHED)) {
+ loff_t end;
+
+ pos = iocb->ki_pos;
+ end = pos + written;
+
+ status = filemap_write_and_wait_range(mapping, pos, end);
+ if (status)
+ goto out;
+
+ /*
+ * No pages were created for this range, we're done
+ */
+ if (flags & AOP_FLAG_UNCACHED)
+ goto out;
+
+ /*
+ * Try to invalidate cache pages for the range we just wrote.
+ * We don't care if invalidation fails as the write has still
+ * worked and leaving clean uptodate pages in the page cache
+ * isn't a corruption vector for uncached IO.
+ */
+ invalidate_inode_pages2_range(mapping,
+ pos >> PAGE_SHIFT, end >> PAGE_SHIFT);
+ }
+out:
return written ? written : status;
}
EXPORT_SYMBOL(generic_perform_write);
--
2.24.1
next prev parent reply other threads:[~2019-12-17 14:39 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-17 14:39 [PATCHSET v5 0/6] Support for RWF_UNCACHED Jens Axboe
2019-12-17 14:39 ` [PATCH 1/6] fs: add read support " Jens Axboe
2019-12-17 15:16 ` Guoqing Jiang
2019-12-17 16:42 ` Jens Axboe
2019-12-17 15:57 ` Matthew Wilcox
2019-12-17 16:41 ` Jens Axboe
2019-12-18 3:17 ` Dave Chinner
2019-12-17 14:39 ` [PATCH 2/6] mm: make generic_perform_write() take a struct kiocb Jens Axboe
2019-12-17 14:39 ` Jens Axboe [this message]
2019-12-17 14:39 ` [PATCH 4/6] iomap: add struct iomap_ctx Jens Axboe
2019-12-17 19:39 ` Linus Torvalds
2019-12-17 20:26 ` Linus Torvalds
2019-12-18 0:15 ` Jens Axboe
2019-12-18 1:25 ` Darrick J. Wong
2019-12-17 14:39 ` [PATCH 5/6] iomap: support RWF_UNCACHED for buffered writes Jens Axboe
2019-12-18 1:52 ` Darrick J. Wong
2019-12-18 3:18 ` Jens Axboe
2019-12-18 4:15 ` Darrick J. Wong
2019-12-17 14:39 ` [PATCH 6/6] xfs: don't do delayed allocations for uncached " Jens Axboe
2019-12-18 1:57 ` Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191217143948.26380-4-axboe@kernel.dk \
--to=axboe@kernel.dk \
--cc=clm@fb.com \
--cc=david@fromorbit.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=torvalds@linux-foundation.org \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).