From: jglisse@redhat.com
To: linux-kernel@vger.kernel.org
Cc: "Jérôme Glisse" <jglisse@redhat.com>,
linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org,
linux-mm@kvack.org, "John Hubbard" <jhubbard@nvidia.com>,
"Jan Kara" <jack@suse.cz>,
"Dan Williams" <dan.j.williams@intel.com>,
"Alexander Viro" <viro@zeniv.linux.org.uk>,
"Johannes Thumshirn" <jthumshirn@suse.de>,
"Christoph Hellwig" <hch@lst.de>, "Jens Axboe" <axboe@kernel.dk>,
"Ming Lei" <ming.lei@redhat.com>,
"Dave Chinner" <david@fromorbit.com>,
"Jason Gunthorpe" <jgg@ziepe.ca>,
"Matthew Wilcox" <willy@infradead.org>
Subject: [PATCH v1 11/15] block: make sure bio_add_page*() knows page that are coming from GUP
Date: Thu, 11 Apr 2019 17:08:30 -0400 [thread overview]
Message-ID: <20190411210834.4105-12-jglisse@redhat.com> (raw)
In-Reply-To: <20190411210834.4105-1-jglisse@redhat.com>
From: Jérôme Glisse <jglisse@redhat.com>
When we get a page reference through get_user_page*() we want to keep
track of that so pass down that information to bio_add_page*().
Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-block@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Johannes Thumshirn <jthumshirn@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Ming Lei <ming.lei@redhat.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Matthew Wilcox <willy@infradead.org>
---
block/bio.c | 34 +++++++++++++++++++++++++++-------
1 file changed, 27 insertions(+), 7 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 73227ede9a0a..197b70426aa6 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -708,7 +708,10 @@ int bio_add_pc_page(struct request_queue *q, struct bio *bio, struct page
* cannot add the page
*/
bvec = &bio->bi_io_vec[bio->bi_vcnt];
- bvec_set_page(bvec, page);
+ if (is_gup)
+ bvec_set_gup_page(bvec, page);
+ else
+ bvec_set_page(bvec, page);
bvec->bv_len = len;
bvec->bv_offset = offset;
bio->bi_vcnt++;
@@ -793,6 +796,7 @@ EXPORT_SYMBOL_GPL(__bio_try_merge_page);
* @page: page to add
* @len: length of the data to add
* @off: offset of the data in @page
+ * @is_gup: was the page referenced through GUP (get_user_page*)
*
* Add the data at @page + @off to @bio as a new bvec. The caller must ensure
* that @bio has space for another bvec.
@@ -805,7 +809,10 @@ void __bio_add_page(struct bio *bio, struct page *page,
WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED));
WARN_ON_ONCE(bio_full(bio));
- bvec_set_page(bv, page);
+ if (is_gup)
+ bvec_set_gup_page(bv, page);
+ else
+ bvec_set_page(bv, page);
bv->bv_offset = off;
bv->bv_len = len;
@@ -820,6 +827,7 @@ EXPORT_SYMBOL_GPL(__bio_add_page);
* @page: page to add
* @len: vec entry length
* @offset: vec entry offset
+ * @is_gup: was the page referenced through GUP (get_user_page*)
*
* Attempt to add a page to the bio_vec maplist. This will only fail
* if either bio->bi_vcnt == bio->bi_max_vecs or it's a cloned bio.
@@ -830,7 +838,7 @@ int bio_add_page(struct bio *bio, struct page *page,
if (!__bio_try_merge_page(bio, page, len, offset, false)) {
if (bio_full(bio))
return 0;
- __bio_add_page(bio, page, len, offset, false);
+ __bio_add_page(bio, page, len, offset, is_gup);
}
return len;
}
@@ -885,6 +893,7 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
ssize_t size, left;
unsigned len, i;
size_t offset;
+ bool gup;
/*
* Move page array up in the allocated memory for the bio vecs as far as
@@ -894,6 +903,8 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
BUILD_BUG_ON(PAGE_PTRS_PER_BVEC < 2);
pages += entries_left * (PAGE_PTRS_PER_BVEC - 1);
+ /* Is iov_iter_get_pages() using GUP ? */
+ gup = iov_iter_get_pages_use_gup(iter);
size = iov_iter_get_pages(iter, pages, LONG_MAX, nr_pages, &offset);
if (unlikely(size <= 0))
return size ? size : -EFAULT;
@@ -902,7 +913,8 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
struct page *page = pages[i];
len = min_t(size_t, PAGE_SIZE - offset, left);
- if (WARN_ON_ONCE(bio_add_page(bio, page, len, offset, false) != len))
+ if (WARN_ON_ONCE(bio_add_page(bio, page, len,
+ offset, gup) != len))
return -EINVAL;
offset = 0;
}
@@ -1372,6 +1384,10 @@ struct bio *bio_map_user_iov(struct request_queue *q,
ssize_t bytes;
size_t offs, added = 0;
int npages;
+ bool gup;
+
+ /* Is iov_iter_get_pages() using GUP ? */
+ gup = iov_iter_get_pages_use_gup(iter);
bytes = iov_iter_get_pages_alloc(iter, &pages, LONG_MAX, &offs);
if (unlikely(bytes <= 0)) {
@@ -1393,7 +1409,7 @@ struct bio *bio_map_user_iov(struct request_queue *q,
if (n > bytes)
n = bytes;
- if (!bio_add_pc_page(q, bio, page, n, offs, false))
+ if (!bio_add_pc_page(q, bio, page, n, offs, gup))
break;
/*
@@ -1412,8 +1428,12 @@ struct bio *bio_map_user_iov(struct request_queue *q,
/*
* release the pages we didn't map into the bio, if any
*/
- while (j < npages)
- put_page(pages[j++]);
+ while (j < npages) {
+ if (gup)
+ put_user_page(pages[j++]);
+ else
+ put_page(pages[j++]);
+ }
kvfree(pages);
/* couldn't stuff something into bio? */
if (bytes)
--
2.20.1
next prev parent reply other threads:[~2019-04-11 21:09 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-11 21:08 [PATCH v1 00/15] Keep track of GUPed pages in fs and block jglisse
2019-04-11 21:08 ` [PATCH v1 01/15] fs/direct-io: fix trailing whitespace issues jglisse
2019-04-11 21:08 ` [PATCH v1 02/15] iov_iter: add helper to test if an iter would use GUP jglisse
2019-04-11 21:08 ` [PATCH v1 03/15] block: introduce bvec_page()/bvec_set_page() to get/set bio_vec.bv_page jglisse
2019-04-11 21:08 ` [PATCH v1 04/15] block: introduce BIO_VEC_INIT() macro to initialize bio_vec structure jglisse
2019-04-11 21:08 ` [PATCH v1 05/15] block: replace all bio_vec->bv_page by bvec_page()/bvec_set_page() jglisse
2019-04-11 21:08 ` [PATCH v1 06/15] block: convert bio_vec.bv_page to bv_pfn to store pfn and not page jglisse
2019-04-11 21:08 ` [PATCH v1 07/15] block: add bvec_put_page_dirty*() to replace put_page(bvec_page()) jglisse
2019-04-11 21:08 ` [PATCH v1 08/15] block: use bvec_put_page() instead of put_page(bvec_page()) jglisse
2019-04-11 21:08 ` [PATCH v1 09/15] block: bvec_put_page_dirty* instead of set_page_dirty* and bvec_put_page jglisse
2019-04-11 21:08 ` [PATCH v1 10/15] block: add gup flag to bio_add_page()/bio_add_pc_page()/__bio_add_page() jglisse
2019-04-15 14:59 ` Jan Kara
2019-04-15 15:24 ` Jerome Glisse
2019-04-16 16:46 ` Jan Kara
2019-04-16 16:54 ` Dan Williams
2019-04-16 17:07 ` Jerome Glisse
2019-04-16 0:22 ` Jerome Glisse
2019-04-16 16:52 ` Jan Kara
2019-04-16 18:32 ` Jerome Glisse
2019-04-11 21:08 ` jglisse [this message]
2019-04-11 21:08 ` [PATCH v1 12/15] fs/direct-io: keep track of wether a page is coming from GUP or not jglisse
2019-04-11 23:14 ` Dave Chinner
2019-04-12 0:08 ` Jerome Glisse
2019-04-11 21:08 ` [PATCH v1 13/15] fs/splice: use put_user_page() when appropriate jglisse
2019-04-11 21:08 ` [PATCH v1 14/15] fs: use bvec_set_gup_page() where appropriate jglisse
2019-04-11 21:08 ` [PATCH v1 15/15] ceph: use put_user_pages() instead of ceph_put_page_vector() jglisse
2019-04-15 7:46 ` Yan, Zheng
2019-04-15 15:11 ` Jerome Glisse
2019-04-16 0:00 ` [PATCH v1 00/15] Keep track of GUPed pages in fs and block Dave Chinner
[not found] ` <2c124cc4-b97e-ee28-2926-305bc6bc74bd@plexistor.com>
2019-04-16 18:47 ` Jerome Glisse
2019-04-16 18:59 ` Kent Overstreet
2019-04-16 19:12 ` Dan Williams
2019-04-16 19:49 ` Jerome Glisse
2019-04-17 21:53 ` Dan Williams
2019-04-17 22:28 ` Jerome Glisse
2019-04-17 23:32 ` Dan Williams
2019-04-18 10:42 ` Jan Kara
2019-04-18 14:27 ` Jerome Glisse
2019-04-18 15:30 ` Jan Kara
2019-04-18 15:36 ` Jerome Glisse
2019-04-18 18:03 ` Dan Williams
[not found] ` <ccac6c5a-7120-0455-88de-ca321b01e825@plexistor.com>
2019-04-16 19:57 ` Jerome Glisse
[not found] ` <41e2d7e1-104b-a006-2824-015ca8c76cc8@gmail.com>
2019-04-16 23:16 ` Jerome Glisse
[not found] ` <fa00a2ff-3664-3165-7af8-9d9c53238245@plexistor.com>
2019-04-17 2:03 ` Jerome Glisse
2019-04-17 21:19 ` Jerome Glisse
2019-04-16 23:34 ` Jerome Glisse
2019-04-17 21:54 ` Dan Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190411210834.4105-12-jglisse@redhat.com \
--to=jglisse@redhat.com \
--cc=axboe@kernel.dk \
--cc=dan.j.williams@intel.com \
--cc=david@fromorbit.com \
--cc=hch@lst.de \
--cc=jack@suse.cz \
--cc=jgg@ziepe.ca \
--cc=jhubbard@nvidia.com \
--cc=jthumshirn@suse.de \
--cc=linux-block@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ming.lei@redhat.com \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).