linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Matthew Wilcox <willy@infradead.org>
To: linux-fsdevel@vger.kernel.org
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: [PATCH v5 11/39] fs: Support THPs in zero_user_segments
Date: Thu, 28 May 2020 19:57:56 -0700	[thread overview]
Message-ID: <20200529025824.32296-12-willy@infradead.org> (raw)
In-Reply-To: <20200529025824.32296-1-willy@infradead.org>

From: "Matthew Wilcox (Oracle)" <willy@infradead.org>

We can only kmap() one subpage of a THP at a time, so loop over all
relevant subpages, skipping ones which don't need to be zeroed.  This is
too large to inline when THPs are enabled and we actually need highmem,
so put it in highmem.c.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 include/linux/highmem.h | 15 +++++++---
 mm/highmem.c            | 62 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 71 insertions(+), 6 deletions(-)

diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index ea5cdbd8c2c3..74614903619d 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -215,13 +215,18 @@ static inline void clear_highpage(struct page *page)
 	kunmap_atomic(kaddr);
 }
 
+#if defined(CONFIG_HIGHMEM) && defined(CONFIG_TRANSPARENT_HUGEPAGE)
+void zero_user_segments(struct page *page, unsigned start1, unsigned end1,
+		unsigned start2, unsigned end2);
+#else /* !HIGHMEM || !TRANSPARENT_HUGEPAGE */
 static inline void zero_user_segments(struct page *page,
-	unsigned start1, unsigned end1,
-	unsigned start2, unsigned end2)
+		unsigned start1, unsigned end1,
+		unsigned start2, unsigned end2)
 {
+	unsigned long i;
 	void *kaddr = kmap_atomic(page);
 
-	BUG_ON(end1 > PAGE_SIZE || end2 > PAGE_SIZE);
+	BUG_ON(end1 > thp_size(page) || end2 > thp_size(page));
 
 	if (end1 > start1)
 		memset(kaddr + start1, 0, end1 - start1);
@@ -230,8 +235,10 @@ static inline void zero_user_segments(struct page *page,
 		memset(kaddr + start2, 0, end2 - start2);
 
 	kunmap_atomic(kaddr);
-	flush_dcache_page(page);
+	for (i = 0; i < hpage_nr_pages(page); i++)
+		flush_dcache_page(page + i);
 }
+#endif /* !HIGHMEM || !TRANSPARENT_HUGEPAGE */
 
 static inline void zero_user_segment(struct page *page,
 	unsigned start, unsigned end)
diff --git a/mm/highmem.c b/mm/highmem.c
index 64d8dea47dd1..3a85c66ef532 100644
--- a/mm/highmem.c
+++ b/mm/highmem.c
@@ -367,9 +367,67 @@ void kunmap_high(struct page *page)
 	if (need_wakeup)
 		wake_up(pkmap_map_wait);
 }
-
 EXPORT_SYMBOL(kunmap_high);
-#endif
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+void zero_user_segments(struct page *page, unsigned start1, unsigned end1,
+		unsigned start2, unsigned end2)
+{
+	unsigned int i;
+
+	BUG_ON(end1 > thp_size(page) || end2 > thp_size(page));
+
+	for (i = 0; i < hpage_nr_pages(page); i++) {
+		void *kaddr;
+		unsigned this_end;
+
+		if (end1 == 0 && start2 >= PAGE_SIZE) {
+			start2 -= PAGE_SIZE;
+			end2 -= PAGE_SIZE;
+			continue;
+		}
+
+		if (start1 >= PAGE_SIZE) {
+			start1 -= PAGE_SIZE;
+			end1 -= PAGE_SIZE;
+			if (start2) {
+				start2 -= PAGE_SIZE;
+				end2 -= PAGE_SIZE;
+			}
+			continue;
+		}
+
+		kaddr = kmap_atomic(page + i);
+
+		this_end = min_t(unsigned, end1, PAGE_SIZE);
+		if (end1 > start1)
+			memset(kaddr + start1, 0, this_end - start1);
+		end1 -= this_end;
+		start1 = 0;
+
+		if (start2 >= PAGE_SIZE) {
+			start2 -= PAGE_SIZE;
+			end2 -= PAGE_SIZE;
+		} else {
+			this_end = min_t(unsigned, end2, PAGE_SIZE);
+			if (end2 > start2)
+				memset(kaddr + start2, 0, this_end - start2);
+			end2 -= this_end;
+			start2 = 0;
+		}
+
+		kunmap_atomic(kaddr);
+		flush_dcache_page(page + i);
+
+		if (!end1 && !end2)
+			break;
+	}
+
+	BUG_ON((start1 | start2 | end1 | end2) != 0);
+}
+EXPORT_SYMBOL(zero_user_segments);
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+#endif /* CONFIG_HIGHMEM */
 
 #if defined(HASHED_PAGE_VIRTUAL)
 
-- 
2.26.2



  parent reply	other threads:[~2020-05-29  2:58 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-29  2:57 [PATCH v5 00/39] Large pages in the page cache Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 01/39] mm: Move PageDoubleMap bit Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 02/39] mm: Simplify PageDoubleMap with PF_SECOND policy Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 03/39] mm: Allow hpages to be arbitrary order Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 04/39] mm: Introduce thp_size Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 05/39] mm: Introduce thp_order Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 06/39] mm: Introduce offset_in_thp Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 07/39] fs: Add a filesystem flag for large pages Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 08/39] fs: Do not update nr_thps for large page mappings Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 09/39] fs: Introduce i_blocks_per_page Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 10/39] fs: Make page_mkwrite_check_truncate thp-aware Matthew Wilcox
2020-05-29  2:57 ` Matthew Wilcox [this message]
2020-05-29  2:57 ` [PATCH v5 12/39] bio: Add bio_for_each_thp_segment_all Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 13/39] iomap: Support arbitrarily many blocks per page Matthew Wilcox
2020-05-29  2:57 ` [PATCH v5 14/39] iomap: Support large pages in iomap_adjust_read_range Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 15/39] iomap: Support large pages in invalidatepage Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 16/39] iomap: Support large pages in read paths Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 17/39] iomap: Support large pages in write paths Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 18/39] iomap: Inline data shouldn't see large pages Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 19/39] iomap: Handle tail pages in iomap_page_mkwrite Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 20/39] xfs: Support large pages Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 21/39] mm: Make prep_transhuge_page return its argument Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 22/39] mm: Add __page_cache_alloc_order Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 23/39] mm: Allow large pages to be added to the page cache Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 24/39] mm: Allow large pages to be removed from " Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 25/39] mm: Remove page fault assumption of compound page size Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 26/39] mm: Fix total_mapcount assumption of " Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 27/39] mm: Remove assumptions of THP size Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 28/39] mm: Avoid splitting large pages Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 29/39] mm: Fix truncation for pages of arbitrary size Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 30/39] mm: Handle truncates that split large pages Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 31/39] mm: Support storing shadow entries for " Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 32/39] mm: Support retrieving tail pages from the page cache Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 33/39] mm: Support tail pages in wait_for_stable_page Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 34/39] mm: Add DEFINE_READAHEAD Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 35/39] mm: Make page_cache_readahead_unbounded take a readahead_control Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 36/39] mm: Make __do_page_cache_readahead " Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 37/39] mm: Allow PageReadahead to be set on head pages Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 38/39] mm: Add large page readahead Matthew Wilcox
2020-05-29  2:58 ` [PATCH v5 39/39] mm: Align THP mappings for non-DAX Matthew Wilcox

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200529025824.32296-12-willy@infradead.org \
    --to=willy@infradead.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).