From: Matthew Wilcox <willy@infradead.org>
To: linux-fsdevel@vger.kernel.org
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v6 21/51] block: Support THPs in page_is_mergeable
Date: Fri, 12 Jun 2020 09:17:05 -0700 [thread overview]
Message-ID: <20200612161705.GE8681@bombadil.infradead.org> (raw)
In-Reply-To: <20200610201345.13273-22-willy@infradead.org>
On Wed, Jun 10, 2020 at 01:13:15PM -0700, Matthew Wilcox wrote:
> page_is_mergeable() would incorrectly claim that two IOs were on different
> pages because they were on different base pages rather than on the
> same THP. This led to a reference counting bug in iomap. Simplify the
> 'same_page' test by just comparing whether we have the same struct page
> instead of doing arithmetic on the physical addresses.
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
> block/bio.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/block/bio.c b/block/bio.c
> index 5235da6434aa..cd677cde853d 100644
> --- a/block/bio.c
> +++ b/block/bio.c
> @@ -747,7 +747,7 @@ static inline bool page_is_mergeable(const struct bio_vec *bv,
> if (xen_domain() && !xen_biovec_phys_mergeable(bv, page))
> return false;
>
> - *same_page = ((vec_end_addr & PAGE_MASK) == page_addr);
> + *same_page = bv->bv_page == page;
> if (!*same_page && pfn_to_page(PFN_DOWN(vec_end_addr)) + 1 != page)
> return false;
> return true;
No, this is also wrong. If you put two order-2 pages into the same
bvec, and then add the contents of each subpage one at a time, page 0
will be !same, pages 1-3 will be same, but then pages 4-7 will all be
!same instead of page 4 being !same and pages 5-7 being !same. And the
reference count will be wrong on the second THP.
But now I'm thinking about the whole interaction with the block
layer, and it's all a bit complicated. Changing the definition
of page_is_mergeable() to treat compound pages differently without
also changing bio_next_segment() seems like it'll cause problems with
refcounting if any current users can submit (parts of) a compound page.
And changing how bio_next_segment() works requires auditing all the users,
and I don't want to do that.
We could use a bio flag to indicate whether this is a THP-bearing BIO
and how it should decide whether two pages are actually part of the same
page, but that seems like a really bad bit of added complexity.
We could also pass a flag to __bio_try_merge_page() from the iomap code,
but again added complexity. We could also add a __bio_try_merge_thp()
that would only be called from iomap for now. That would call a new
thp_is_mergable() which would use the THP definition of what a "same
page" is. I think I hate this idea least of all the ones named so far.
My preferred solution is to change the definition of iop->write_count
(and iop->read_count). Instead of being a count of the number of segments
submitted, make it a count of the number of bytes submitted. Like this:
diff -u b/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
--- b/fs/iomap/buffered-io.c
+++ b/fs/iomap/buffered-io.c
@@ -174,9 +174,9 @@
}
static void
-iomap_read_finish(struct iomap_page *iop, struct page *page)
+iomap_read_finish(struct iomap_page *iop, struct page *page, unsigned int len)
{
- if (!iop || atomic_dec_and_test(&iop->read_count))
+ if (!iop || atomic_sub_and_test(len, &iop->read_count))
unlock_page(page);
}
@@ -193,7 +193,7 @@
iomap_set_range_uptodate(page, bvec->bv_offset, bvec->bv_len);
}
- iomap_read_finish(iop, page);
+ iomap_read_finish(iop, page, bvec->bv_len);
}
static void
@@ -294,8 +294,8 @@
if (is_contig &&
__bio_try_merge_page(ctx->bio, page, plen, poff, &same_page)) {
- if (!same_page && iop)
- atomic_inc(&iop->read_count);
+ if (iop)
+ atomic_add(plen, &iop->read_count);
goto done;
}
@@ -305,7 +305,7 @@
* that we don't prematurely unlock the page.
*/
if (iop)
- atomic_inc(&iop->read_count);
+ atomic_add(plen, &iop->read_count);
if (!ctx->bio || !is_contig || bio_full(ctx->bio, plen)) {
gfp_t gfp = mapping_gfp_constraint(page->mapping, GFP_KERNEL);
@@ -1090,7 +1090,7 @@
static void
iomap_finish_page_writeback(struct inode *inode, struct page *page,
- int error)
+ int error, unsigned int len)
{
struct iomap_page *iop = iomap_page_create(inode, page);
@@ -1101,7 +1101,7 @@
WARN_ON_ONCE(iop && atomic_read(&iop->write_count) <= 0);
- if (!iop || atomic_dec_and_test(&iop->write_count))
+ if (!iop || atomic_sub_and_test(len, &iop->write_count))
end_page_writeback(page);
}
@@ -1135,7 +1135,8 @@
/* walk each page on bio, ending page IO on them */
bio_for_each_thp_segment_all(bv, bio, iter_all)
- iomap_finish_page_writeback(inode, bv->bv_page, error);
+ iomap_finish_page_writeback(inode, bv->bv_page, error,
+ bv->bv_len);
bio_put(bio);
}
/* The ioend has been freed by bio_put() */
@@ -1351,8 +1352,8 @@
merged = __bio_try_merge_page(wpc->ioend->io_bio, page, len, poff,
&same_page);
- if (iop && !same_page)
- atomic_inc(&iop->write_count);
+ if (iop)
+ atomic_add(len, &iop->write_count);
if (!merged) {
if (bio_full(wpc->ioend->io_bio, len)) {
next prev parent reply other threads:[~2020-06-12 16:17 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-10 20:12 [RFC v6 00/51] Large pages in the page cache Matthew Wilcox
2020-06-10 20:12 ` [PATCH v6 01/51] mm: Print head flags in dump_page Matthew Wilcox
2020-06-10 20:12 ` [PATCH v6 02/51] mm: Print the inode number " Matthew Wilcox
2020-06-10 20:12 ` [PATCH v6 03/51] mm: Print hashed address of struct page Matthew Wilcox
2020-06-10 20:12 ` [PATCH v6 04/51] mm: Move PageDoubleMap bit Matthew Wilcox
2020-06-11 15:03 ` Zi Yan
2020-06-10 20:12 ` [PATCH v6 05/51] mm: Simplify PageDoubleMap with PF_SECOND policy Matthew Wilcox
2020-06-11 15:14 ` Zi Yan
2020-06-10 20:13 ` [PATCH v6 06/51] mm: Store compound_nr as well as compound_order Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 07/51] mm: Move page-flags include to top of file Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 08/51] mm: Add thp_order Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 09/51] mm: Add thp_size Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 10/51] mm: Replace hpage_nr_pages with thp_nr_pages Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 11/51] mm: Add thp_head Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 12/51] mm: Introduce offset_in_thp Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 13/51] mm: Support arbitrary THP sizes Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 14/51] fs: Add a filesystem flag for THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 15/51] fs: Do not update nr_thps for mappings which support THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 16/51] fs: Introduce i_blocks_per_page Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 17/51] fs: Make page_mkwrite_check_truncate thp-aware Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 18/51] mm: Support THPs in zero_user_segments Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 19/51] mm: Zero the head page, not the tail page Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 20/51] block: Add bio_for_each_thp_segment_all Matthew Wilcox
2020-06-11 18:20 ` Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 21/51] block: Support THPs in page_is_mergeable Matthew Wilcox
2020-06-12 16:17 ` Matthew Wilcox [this message]
2020-06-10 20:13 ` [PATCH v6 22/51] iomap: Support arbitrarily many blocks per page Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 23/51] iomap: Support THPs in iomap_adjust_read_range Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 24/51] iomap: Support THPs in invalidatepage Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 25/51] iomap: Support THPs in read paths Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 26/51] iomap: Convert iomap_write_end types Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 27/51] iomap: Change calling convention for zeroing Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 28/51] iomap: Change iomap_write_begin calling convention Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 29/51] iomap: Support THPs in write paths Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 30/51] iomap: Inline data shouldn't see THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 31/51] iomap: Handle tail pages in iomap_page_mkwrite Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 32/51] xfs: Support THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 33/51] mm: Make prep_transhuge_page return its argument Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 34/51] mm: Add __page_cache_alloc_order Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 35/51] mm: Allow THPs to be added to the page cache Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 36/51] mm: Allow THPs to be removed from " Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 37/51] mm: Remove page fault assumption of compound page size Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 38/51] mm: Fix total_mapcount assumption of " Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 39/51] mm: Remove assumptions of THP size Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 40/51] mm: Avoid splitting THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 41/51] mm: Fix truncation for pages of arbitrary size Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 42/51] mm: Handle truncates that split THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 43/51] mm: Support storing shadow entries for THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 44/51] mm: Support retrieving tail pages from the page cache Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 45/51] mm: Support tail pages in wait_for_stable_page Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 46/51] mm: Add DEFINE_READAHEAD Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 47/51] mm: Make page_cache_readahead_unbounded take a readahead_control Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 48/51] mm: Make __do_page_cache_readahead " Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 49/51] mm: Allow PageReadahead to be set on head pages Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 50/51] mm: Add THP readahead Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 51/51] mm: Align THP mappings for non-DAX Matthew Wilcox
2020-06-11 6:59 ` [RFC v6 00/51] Large pages in the page cache Christoph Hellwig
2020-06-11 11:24 ` Matthew Wilcox
2020-06-15 13:32 ` Christoph Hellwig
2020-06-14 16:26 ` Matthew Wilcox
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200612161705.GE8681@bombadil.infradead.org \
--to=willy@infradead.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).