From: john.hubbard@gmail.com To: Andrew Morton <akpm@linux-foundation.org> Cc: "Alexander Viro" <viro@zeniv.linux.org.uk>, "Anna Schumaker" <anna.schumaker@netapp.com>, "David S . Miller" <davem@davemloft.net>, "Dominique Martinet" <asmadeus@codewreck.org>, "Eric Van Hensbergen" <ericvh@gmail.com>, "Jason Gunthorpe" <jgg@ziepe.ca>, "Jason Wang" <jasowang@redhat.com>, "Jens Axboe" <axboe@kernel.dk>, "Latchesar Ionkov" <lucho@ionkov.net>, "Michael S . Tsirkin" <mst@redhat.com>, "Miklos Szeredi" <miklos@szeredi.hu>, "Trond Myklebust" <trond.myklebust@hammerspace.com>, "Christoph Hellwig" <hch@lst.de>, "Matthew Wilcox" <willy@infradead.org>, linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>, ceph-devel@vger.kernel.org, kvm@vger.kernel.org, linux-block@vger.kernel.org, linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, samba-technical@lists.samba.org, v9fs-developer@lists.sourceforge.net, virtualization@lists.linux-foundation.org, "Jérôme Glisse" <jglisse@redhat.com>, "John Hubbard" <jhubbard@nvidia.com>, "Christoph Hellwig" <hch@infradead.org>, "Minwoo Im" <minwoo.im.dev@gmail.com> Subject: [PATCH 04/12] block: bio_release_pages: convert put_page() to put_user_page*() Date: Tue, 23 Jul 2019 21:25:10 -0700 [thread overview] Message-ID: <20190724042518.14363-5-jhubbard@nvidia.com> (raw) In-Reply-To: <20190724042518.14363-1-jhubbard@nvidia.com> From: Jérôme Glisse <jglisse@redhat.com> For pages that were retained via get_user_pages*(), release those pages via the new put_user_page*() routines, instead of via put_page() or release_pages(). This is part a tree-wide conversion, as described in commit fc1d8e7cca2d ("mm: introduce put_user_page*(), placeholder versions"). Changes from Jérôme's original patch: * reworked to be compatible with recent bio_release_pages() changes, * refactored slightly to remove some code duplication, * use an approach that changes fewer bio_check_pages_dirty() callers. Signed-off-by: Jérôme Glisse <jglisse@redhat.com> Signed-off-by: John Hubbard <jhubbard@nvidia.com> Cc: Christoph Hellwig <hch@infradead.org> Cc: Minwoo Im <minwoo.im.dev@gmail.com> Cc: Jens Axboe <axboe@kernel.dk> --- block/bio.c | 60 ++++++++++++++++++++++++++++++++++++--------- include/linux/bio.h | 1 + 2 files changed, 49 insertions(+), 12 deletions(-) diff --git a/block/bio.c b/block/bio.c index 7675e2de509d..74f9eba2583b 100644 --- a/block/bio.c +++ b/block/bio.c @@ -844,7 +844,11 @@ void bio_release_pages(struct bio *bio, enum bio_rp_flags_t flags) bio_for_each_segment_all(bvec, bio, iter_all) { if ((flags & BIO_RP_MARK_DIRTY) && !PageCompound(bvec->bv_page)) set_page_dirty_lock(bvec->bv_page); - put_page(bvec->bv_page); + + if (flags & BIO_RP_FROM_GUP) + put_user_page(bvec->bv_page); + else + put_page(bvec->bv_page); } } @@ -1667,28 +1671,50 @@ static void bio_dirty_fn(struct work_struct *work); static DECLARE_WORK(bio_dirty_work, bio_dirty_fn); static DEFINE_SPINLOCK(bio_dirty_lock); static struct bio *bio_dirty_list; +static struct bio *bio_gup_dirty_list; -/* - * This runs in process context - */ -static void bio_dirty_fn(struct work_struct *work) +static void __bio_dirty_fn(struct work_struct *work, + struct bio **dirty_list, + enum bio_rp_flags_t flags) { struct bio *bio, *next; spin_lock_irq(&bio_dirty_lock); - next = bio_dirty_list; - bio_dirty_list = NULL; + next = *dirty_list; + *dirty_list = NULL; spin_unlock_irq(&bio_dirty_lock); while ((bio = next) != NULL) { next = bio->bi_private; - bio_release_pages(bio, BIO_RP_MARK_DIRTY); + bio_release_pages(bio, BIO_RP_MARK_DIRTY | flags); bio_put(bio); } } -void bio_check_pages_dirty(struct bio *bio) +/* + * This runs in process context + */ +static void bio_dirty_fn(struct work_struct *work) +{ + __bio_dirty_fn(work, &bio_dirty_list, BIO_RP_NORMAL); + __bio_dirty_fn(work, &bio_gup_dirty_list, BIO_RP_FROM_GUP); +} + +/** + * __bio_check_pages_dirty() - queue up pages on a workqueue to dirty them + * @bio: the bio struct containing the pages we should dirty + * @from_gup: did the pages in the bio came from GUP (get_user_pages*()) + * + * This will go over all pages in the bio, and for each non dirty page, the + * bio is added to a list of bio's that need to get their pages dirtied. + * + * We also need to know if the pages in the bio are coming from GUP or not, + * as GUPed pages need to be released via put_user_page(), instead of + * put_page(). Please see Documentation/vm/get_user_pages.rst for details + * on that. + */ +void __bio_check_pages_dirty(struct bio *bio, bool from_gup) { struct bio_vec *bvec; unsigned long flags; @@ -1699,17 +1725,27 @@ void bio_check_pages_dirty(struct bio *bio) goto defer; } - bio_release_pages(bio, BIO_RP_NORMAL); + bio_release_pages(bio, from_gup ? BIO_RP_FROM_GUP : BIO_RP_NORMAL); bio_put(bio); return; defer: spin_lock_irqsave(&bio_dirty_lock, flags); - bio->bi_private = bio_dirty_list; - bio_dirty_list = bio; + if (from_gup) { + bio->bi_private = bio_gup_dirty_list; + bio_gup_dirty_list = bio; + } else { + bio->bi_private = bio_dirty_list; + bio_dirty_list = bio; + } spin_unlock_irqrestore(&bio_dirty_lock, flags); schedule_work(&bio_dirty_work); } +void bio_check_pages_dirty(struct bio *bio) +{ + __bio_check_pages_dirty(bio, false); +} + void update_io_ticks(struct hd_struct *part, unsigned long now) { unsigned long stamp; diff --git a/include/linux/bio.h b/include/linux/bio.h index 2715e55679c1..d68a40c2c9d4 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -444,6 +444,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter); enum bio_rp_flags_t { BIO_RP_NORMAL = 0, BIO_RP_MARK_DIRTY = 1, + BIO_RP_FROM_GUP = 2, }; static inline enum bio_rp_flags_t bio_rp_dirty_flag(bool mark_dirty) -- 2.22.0
WARNING: multiple messages have this Message-ID (diff)
From: john.hubbard@gmail.com To: Andrew Morton <akpm@linux-foundation.org> Cc: Alexander Viro <viro@zeniv.linux.org.uk>, Anna Schumaker <anna.schumaker@netapp.com>, "David S . Miller" <davem@davemloft.net>, Dominique Martinet <asmadeus@codewreck.org>, Eric Van Hensbergen <ericvh@gmail.com>, Jason Gunthorpe <jgg@ziepe.ca>, Jason Wang <jasowang@redhat.com>, Jens Axboe <axboe@kernel.dk>, Latchesar Ionkov <lucho@ionkov.net>, "Michael S . Tsirkin" <mst@redhat.com>, Miklos Szeredi <miklos@szeredi.hu>, Trond Myklebust <trond.myklebust@hammerspace.com>, Christoph Hellwig <hch@lst.de>, Matthew Wilcox <willy@infradead.org>, linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>, ceph-devel@vger.kernel.org, kvm@vger.kernel.org, linux-block@vger.kernel.org, linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-nfs@vger.kernel.orglinu Subject: [PATCH 04/12] block: bio_release_pages: convert put_page() to put_user_page*() Date: Tue, 23 Jul 2019 21:25:10 -0700 [thread overview] Message-ID: <20190724042518.14363-5-jhubbard@nvidia.com> (raw) In-Reply-To: <20190724042518.14363-1-jhubbard@nvidia.com> From: Jérôme Glisse <jglisse@redhat.com> For pages that were retained via get_user_pages*(), release those pages via the new put_user_page*() routines, instead of via put_page() or release_pages(). This is part a tree-wide conversion, as described in commit fc1d8e7cca2d ("mm: introduce put_user_page*(), placeholder versions"). Changes from Jérôme's original patch: * reworked to be compatible with recent bio_release_pages() changes, * refactored slightly to remove some code duplication, * use an approach that changes fewer bio_check_pages_dirty() callers. Signed-off-by: Jérôme Glisse <jglisse@redhat.com> Signed-off-by: John Hubbard <jhubbard@nvidia.com> Cc: Christoph Hellwig <hch@infradead.org> Cc: Minwoo Im <minwoo.im.dev@gmail.com> Cc: Jens Axboe <axboe@kernel.dk> --- block/bio.c | 60 ++++++++++++++++++++++++++++++++++++--------- include/linux/bio.h | 1 + 2 files changed, 49 insertions(+), 12 deletions(-) diff --git a/block/bio.c b/block/bio.c index 7675e2de509d..74f9eba2583b 100644 --- a/block/bio.c +++ b/block/bio.c @@ -844,7 +844,11 @@ void bio_release_pages(struct bio *bio, enum bio_rp_flags_t flags) bio_for_each_segment_all(bvec, bio, iter_all) { if ((flags & BIO_RP_MARK_DIRTY) && !PageCompound(bvec->bv_page)) set_page_dirty_lock(bvec->bv_page); - put_page(bvec->bv_page); + + if (flags & BIO_RP_FROM_GUP) + put_user_page(bvec->bv_page); + else + put_page(bvec->bv_page); } } @@ -1667,28 +1671,50 @@ static void bio_dirty_fn(struct work_struct *work); static DECLARE_WORK(bio_dirty_work, bio_dirty_fn); static DEFINE_SPINLOCK(bio_dirty_lock); static struct bio *bio_dirty_list; +static struct bio *bio_gup_dirty_list; -/* - * This runs in process context - */ -static void bio_dirty_fn(struct work_struct *work) +static void __bio_dirty_fn(struct work_struct *work, + struct bio **dirty_list, + enum bio_rp_flags_t flags) { struct bio *bio, *next; spin_lock_irq(&bio_dirty_lock); - next = bio_dirty_list; - bio_dirty_list = NULL; + next = *dirty_list; + *dirty_list = NULL; spin_unlock_irq(&bio_dirty_lock); while ((bio = next) != NULL) { next = bio->bi_private; - bio_release_pages(bio, BIO_RP_MARK_DIRTY); + bio_release_pages(bio, BIO_RP_MARK_DIRTY | flags); bio_put(bio); } } -void bio_check_pages_dirty(struct bio *bio) +/* + * This runs in process context + */ +static void bio_dirty_fn(struct work_struct *work) +{ + __bio_dirty_fn(work, &bio_dirty_list, BIO_RP_NORMAL); + __bio_dirty_fn(work, &bio_gup_dirty_list, BIO_RP_FROM_GUP); +} + +/** + * __bio_check_pages_dirty() - queue up pages on a workqueue to dirty them + * @bio: the bio struct containing the pages we should dirty + * @from_gup: did the pages in the bio came from GUP (get_user_pages*()) + * + * This will go over all pages in the bio, and for each non dirty page, the + * bio is added to a list of bio's that need to get their pages dirtied. + * + * We also need to know if the pages in the bio are coming from GUP or not, + * as GUPed pages need to be released via put_user_page(), instead of + * put_page(). Please see Documentation/vm/get_user_pages.rst for details + * on that. + */ +void __bio_check_pages_dirty(struct bio *bio, bool from_gup) { struct bio_vec *bvec; unsigned long flags; @@ -1699,17 +1725,27 @@ void bio_check_pages_dirty(struct bio *bio) goto defer; } - bio_release_pages(bio, BIO_RP_NORMAL); + bio_release_pages(bio, from_gup ? BIO_RP_FROM_GUP : BIO_RP_NORMAL); bio_put(bio); return; defer: spin_lock_irqsave(&bio_dirty_lock, flags); - bio->bi_private = bio_dirty_list; - bio_dirty_list = bio; + if (from_gup) { + bio->bi_private = bio_gup_dirty_list; + bio_gup_dirty_list = bio; + } else { + bio->bi_private = bio_dirty_list; + bio_dirty_list = bio; + } spin_unlock_irqrestore(&bio_dirty_lock, flags); schedule_work(&bio_dirty_work); } +void bio_check_pages_dirty(struct bio *bio) +{ + __bio_check_pages_dirty(bio, false); +} + void update_io_ticks(struct hd_struct *part, unsigned long now) { unsigned long stamp; diff --git a/include/linux/bio.h b/include/linux/bio.h index 2715e55679c1..d68a40c2c9d4 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -444,6 +444,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter); enum bio_rp_flags_t { BIO_RP_NORMAL = 0, BIO_RP_MARK_DIRTY = 1, + BIO_RP_FROM_GUP = 2, }; static inline enum bio_rp_flags_t bio_rp_dirty_flag(bool mark_dirty) -- 2.22.0
next prev parent reply other threads:[~2019-07-24 4:27 UTC|newest] Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-07-24 4:25 [PATCH 00/12] block/bio, fs: convert put_page() to put_user_page*() john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 01/12] mm/gup: add make_dirty arg to put_user_pages_dirty_lock() john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 02/12] iov_iter: add helper to test if an iter would use GUP v2 john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 03/12] block: bio_release_pages: use flags arg instead of bool john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 5:30 ` Christoph Hellwig 2019-07-24 5:30 ` Christoph Hellwig 2019-07-24 5:30 ` Christoph Hellwig 2019-07-29 20:57 ` Jerome Glisse 2019-07-29 20:57 ` Jerome Glisse 2019-07-30 10:25 ` Christoph Hellwig 2019-07-30 10:25 ` Christoph Hellwig 2019-07-30 15:57 ` Jerome Glisse 2019-07-30 15:57 ` Jerome Glisse 2019-07-30 15:57 ` Jerome Glisse 2019-08-01 8:20 ` Christoph Hellwig 2019-08-01 8:20 ` Christoph Hellwig 2019-08-01 8:20 ` Christoph Hellwig 2019-07-30 10:25 ` Christoph Hellwig 2019-07-29 20:57 ` Jerome Glisse 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` john.hubbard [this message] 2019-07-24 4:25 ` [PATCH 04/12] block: bio_release_pages: convert put_page() to put_user_page*() john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 05/12] block_dev: " john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 06/12] fs/nfs: " john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 07/12] vhost-scsi: " john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:34 ` John Hubbard 2019-07-24 4:34 ` John Hubbard 2019-07-24 4:34 ` John Hubbard 2019-07-24 8:07 ` Michael S. Tsirkin 2019-07-24 8:07 ` Michael S. Tsirkin 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 08/12] fs/cifs: " john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 09/12] fs/fuse: " john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 10/12] fs/ceph: " john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 11/12] 9p/net: " john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` [PATCH 12/12] fs/ceph: fix a build warning: returning a value from void function john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 4:25 ` john.hubbard 2019-07-24 6:17 ` [PATCH 00/12] block/bio, fs: convert put_page() to put_user_page*() Christoph Hellwig 2019-07-24 6:17 ` Christoph Hellwig 2019-07-24 6:17 ` Christoph Hellwig 2019-07-24 23:23 ` John Hubbard 2019-07-24 23:23 ` John Hubbard 2019-08-05 22:54 ` John Hubbard 2019-08-05 22:54 ` John Hubbard 2019-08-07 6:34 ` Christoph Hellwig 2019-08-07 6:34 ` Christoph Hellwig 2019-08-07 6:34 ` Christoph Hellwig 2019-08-07 6:38 ` John Hubbard 2019-08-07 6:38 ` John Hubbard 2019-07-25 0:41 ` Bob Liu 2019-07-25 0:41 ` Bob Liu 2019-07-26 1:24 ` John Hubbard 2019-07-26 1:24 ` John Hubbard
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190724042518.14363-5-jhubbard@nvidia.com \ --to=john.hubbard@gmail.com \ --cc=akpm@linux-foundation.org \ --cc=anna.schumaker@netapp.com \ --cc=asmadeus@codewreck.org \ --cc=axboe@kernel.dk \ --cc=ceph-devel@vger.kernel.org \ --cc=davem@davemloft.net \ --cc=ericvh@gmail.com \ --cc=hch@infradead.org \ --cc=hch@lst.de \ --cc=jasowang@redhat.com \ --cc=jgg@ziepe.ca \ --cc=jglisse@redhat.com \ --cc=jhubbard@nvidia.com \ --cc=kvm@vger.kernel.org \ --cc=linux-block@vger.kernel.org \ --cc=linux-cifs@vger.kernel.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-nfs@vger.kernel.org \ --cc=linux-rdma@vger.kernel.org \ --cc=lucho@ionkov.net \ --cc=miklos@szeredi.hu \ --cc=minwoo.im.dev@gmail.com \ --cc=mst@redhat.com \ --cc=netdev@vger.kernel.org \ --cc=samba-technical@lists.samba.org \ --cc=trond.myklebust@hammerspace.com \ --cc=v9fs-developer@lists.sourceforge.net \ --cc=viro@zeniv.linux.org.uk \ --cc=virtualization@lists.linux-foundation.org \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.