From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF966C63798 for ; Tue, 24 Nov 2020 04:15:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1C3E82085B for ; Tue, 24 Nov 2020 04:15:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ML/7j5/z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C3E82085B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 499BB6B006E; Mon, 23 Nov 2020 23:15:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D7EC6B0070; Mon, 23 Nov 2020 23:15:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24EFA6B0071; Mon, 23 Nov 2020 23:15:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0028.hostedemail.com [216.40.44.28]) by kanga.kvack.org (Postfix) with ESMTP id DFAC06B006E for ; Mon, 23 Nov 2020 23:15:12 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 8087E1EE6 for ; Tue, 24 Nov 2020 04:15:12 +0000 (UTC) X-FDA: 77517996864.19.month42_560e15f2736b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 5CB811ACEA4 for ; Tue, 24 Nov 2020 04:15:12 +0000 (UTC) X-HE-Tag: month42_560e15f2736b X-Filterd-Recvd-Size: 5083 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Nov 2020 04:15:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:In-Reply-To:References; bh=yovAbVNekdUqnaLLLsFGB6pe3G97F8Xlt5MNxptk1kI=; b=ML/7j5/zMDPYuELKihEQtbVrM0 ormvOmT3gTMGsfhRdSFz2WV3Ce8eqJYrCPJlepf41poFtsjt8B6iQKV8DaCdb3MeRFRP4XV6mMMEM /2L3coQxbbOBzQj8AMG2c1LVvT4waGhn4UL+c+c0cUjhM9wnfa6mXqiC4gDGzN6qfJ5Bm+RRgiJRS VW6j41rSG+6vFbsBD6sKvPXQ2iuOTGcdWdKx4W27Zba7my5HKux3V0ffa5U873DVcKlh75NSq4yiB gqsI1/KkxwNEYulrVFfKIx0+Z3WJTsheF1HH9faAK9y1QiZc9QqS/PwapCr31p0EOqxpfhd6gAXAx uSTw423w==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1khPjR-0007d3-RY; Tue, 24 Nov 2020 04:15:09 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 1/4] mm: Support THPs in zero_user_segments Date: Tue, 24 Nov 2020 04:15:04 +0000 Message-Id: <20201124041507.28996-1-willy@infradead.org> X-Mailer: git-send-email 2.21.3 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We can only kmap() one subpage of a THP at a time, so loop over all relevant subpages, skipping ones which don't need to be zeroed. This is too large to inline when THPs are enabled and we actually need highmem, so put it in highmem.c. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/highmem.h | 19 ++++++++++--- mm/highmem.c | 59 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 74 insertions(+), 4 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index 7d098bd621f6..f20916707ff9 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -193,13 +193,22 @@ static inline void clear_highpage(struct page *page= ) kunmap_atomic(kaddr); } =20 +/* + * If we pass in a base or tail page, we can zero up to PAGE_SIZE. + * If we pass in a head page, we can zero up to the size of the compound= page. + */ +#if defined(CONFIG_HIGHMEM) && defined(CONFIG_TRANSPARENT_HUGEPAGE) +void zero_user_segments(struct page *page, unsigned start1, unsigned end= 1, + unsigned start2, unsigned end2); +#else /* !HIGHMEM || !TRANSPARENT_HUGEPAGE */ static inline void zero_user_segments(struct page *page, - unsigned start1, unsigned end1, - unsigned start2, unsigned end2) + unsigned start1, unsigned end1, + unsigned start2, unsigned end2) { void *kaddr =3D kmap_atomic(page); + unsigned int i; =20 - BUG_ON(end1 > PAGE_SIZE || end2 > PAGE_SIZE); + BUG_ON(end1 > page_size(page) || end2 > page_size(page)); =20 if (end1 > start1) memset(kaddr + start1, 0, end1 - start1); @@ -208,8 +217,10 @@ static inline void zero_user_segments(struct page *p= age, memset(kaddr + start2, 0, end2 - start2); =20 kunmap_atomic(kaddr); - flush_dcache_page(page); + for (i =3D 0; i < compound_nr(page); i++) + flush_dcache_page(page + i); } +#endif /* !HIGHMEM || !TRANSPARENT_HUGEPAGE */ =20 static inline void zero_user_segment(struct page *page, unsigned start, unsigned end) diff --git a/mm/highmem.c b/mm/highmem.c index 54bd233846c9..3e1087f2b735 100644 --- a/mm/highmem.c +++ b/mm/highmem.c @@ -359,6 +359,65 @@ void kunmap_high(struct page *page) wake_up(pkmap_map_wait); } EXPORT_SYMBOL(kunmap_high); + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +void zero_user_segments(struct page *page, unsigned start1, unsigned end= 1, + unsigned start2, unsigned end2) +{ + unsigned int i; + + BUG_ON(end1 > page_size(page) || end2 > page_size(page)); + + for (i =3D 0; i < compound_nr(page); i++) { + void *kaddr; + unsigned this_end; + + if (end1 =3D=3D 0 && start2 >=3D PAGE_SIZE) { + start2 -=3D PAGE_SIZE; + end2 -=3D PAGE_SIZE; + continue; + } + + if (start1 >=3D PAGE_SIZE) { + start1 -=3D PAGE_SIZE; + end1 -=3D PAGE_SIZE; + if (start2) { + start2 -=3D PAGE_SIZE; + end2 -=3D PAGE_SIZE; + } + continue; + } + + kaddr =3D kmap_atomic(page + i); + + this_end =3D min_t(unsigned, end1, PAGE_SIZE); + if (end1 > start1) + memset(kaddr + start1, 0, this_end - start1); + end1 -=3D this_end; + start1 =3D 0; + + if (start2 >=3D PAGE_SIZE) { + start2 -=3D PAGE_SIZE; + end2 -=3D PAGE_SIZE; + } else { + this_end =3D min_t(unsigned, end2, PAGE_SIZE); + if (end2 > start2) + memset(kaddr + start2, 0, this_end - start2); + end2 -=3D this_end; + start2 =3D 0; + } + + kunmap_atomic(kaddr); + flush_dcache_page(page + i); + + if (!end1 && !end2) + break; + } + + BUG_ON((start1 | start2 | end1 | end2) !=3D 0); +} +EXPORT_SYMBOL(zero_user_segments); +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ #endif /* CONFIG_HIGHMEM */ =20 #ifdef CONFIG_KMAP_LOCAL --=20 2.29.2