From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30EE9C56202 for ; Mon, 26 Oct 2020 18:31:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AB8E92222C for ; Mon, 26 Oct 2020 18:31:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="r+7NWmZ6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB8E92222C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 92B546B006C; Mon, 26 Oct 2020 14:31:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D9606B0070; Mon, 26 Oct 2020 14:31:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77C146B006E; Mon, 26 Oct 2020 14:31:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0032.hostedemail.com [216.40.44.32]) by kanga.kvack.org (Postfix) with ESMTP id 4C5A56B0062 for ; Mon, 26 Oct 2020 14:31:45 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id E71DB8249980 for ; Mon, 26 Oct 2020 18:31:44 +0000 (UTC) X-FDA: 77414920128.10.party14_2911dfc27275 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id AD1A116A0DE for ; Mon, 26 Oct 2020 18:31:44 +0000 (UTC) X-HE-Tag: party14_2911dfc27275 X-Filterd-Recvd-Size: 5268 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 18:31:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=+JsIPUMhHfnx/SncjzGyFlWSq6+WuP+wYEhUHsuc6pU=; b=r+7NWmZ6RWrdYBMYBO2LPkXPp3 R8QYndbr8XFgRWFpIK8Aau+mNdkdvIPJFr3z2Q6WCLqo5TR24fcgQGzlYEGcI9txWjw1ju9EFkv7c iCH6gUDXsERthfOhpPI+ky5/c2ifRG/3RsKcb7dmm/6V1MJaq9hSf29sNJwR+pwAHyqRNXsypCBWF PeUcgNsq3Q6IklNFVpqbMhews/yLPeMbOg3Nub3IMM8WUtV3dROFMzF8kmt6hFDPnUlfL2tTmBgt4 ylg3VVskobRonF3xW0nPVSn1vnLA7uOFP3UvfiaDyBBJSJg6yzAx8YafWMSrkjSlwZ3dGdgRv6xQP 6EFsKdJw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kX7HR-0002jF-3N; Mon, 26 Oct 2020 18:31:41 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org Subject: [PATCH 1/9] mm: Support THPs in zero_user_segments Date: Mon, 26 Oct 2020 18:31:28 +0000 Message-Id: <20201026183136.10404-2-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026183136.10404-1-willy@infradead.org> References: <20201026183136.10404-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We can only kmap() one subpage of a THP at a time, so loop over all relevant subpages, skipping ones which don't need to be zeroed. This is too large to inline when THPs are enabled and we actually need highmem, so put it in highmem.c. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/highmem.h | 19 ++++++++++--- mm/highmem.c | 62 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 75 insertions(+), 6 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index 14e6202ce47f..8e21fe82b3a3 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -284,13 +284,22 @@ static inline void clear_highpage(struct page *page= ) kunmap_atomic(kaddr); } =20 +/* + * If we pass in a base or tail page, we can zero up to PAGE_SIZE. + * If we pass in a head page, we can zero up to the size of the compound= page. + */ +#if defined(CONFIG_HIGHMEM) && defined(CONFIG_TRANSPARENT_HUGEPAGE) +void zero_user_segments(struct page *page, unsigned start1, unsigned end= 1, + unsigned start2, unsigned end2); +#else /* !HIGHMEM || !TRANSPARENT_HUGEPAGE */ static inline void zero_user_segments(struct page *page, - unsigned start1, unsigned end1, - unsigned start2, unsigned end2) + unsigned start1, unsigned end1, + unsigned start2, unsigned end2) { void *kaddr =3D kmap_atomic(page); + unsigned int i; =20 - BUG_ON(end1 > PAGE_SIZE || end2 > PAGE_SIZE); + BUG_ON(end1 > page_size(page) || end2 > page_size(page)); =20 if (end1 > start1) memset(kaddr + start1, 0, end1 - start1); @@ -299,8 +308,10 @@ static inline void zero_user_segments(struct page *p= age, memset(kaddr + start2, 0, end2 - start2); =20 kunmap_atomic(kaddr); - flush_dcache_page(page); + for (i =3D 0; i < compound_nr(page); i++) + flush_dcache_page(page + i); } +#endif /* !HIGHMEM || !TRANSPARENT_HUGEPAGE */ =20 static inline void zero_user_segment(struct page *page, unsigned start, unsigned end) diff --git a/mm/highmem.c b/mm/highmem.c index 1352a27951e3..9901a806617a 100644 --- a/mm/highmem.c +++ b/mm/highmem.c @@ -367,9 +367,67 @@ void kunmap_high(struct page *page) if (need_wakeup) wake_up(pkmap_map_wait); } - EXPORT_SYMBOL(kunmap_high); -#endif /* CONFIG_HIGHMEM */ + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +void zero_user_segments(struct page *page, unsigned start1, unsigned end= 1, + unsigned start2, unsigned end2) +{ + unsigned int i; + + BUG_ON(end1 > page_size(page) || end2 > page_size(page)); + + for (i =3D 0; i < compound_nr(page); i++) { + void *kaddr; + unsigned this_end; + + if (end1 =3D=3D 0 && start2 >=3D PAGE_SIZE) { + start2 -=3D PAGE_SIZE; + end2 -=3D PAGE_SIZE; + continue; + } + + if (start1 >=3D PAGE_SIZE) { + start1 -=3D PAGE_SIZE; + end1 -=3D PAGE_SIZE; + if (start2) { + start2 -=3D PAGE_SIZE; + end2 -=3D PAGE_SIZE; + } + continue; + } + + kaddr =3D kmap_atomic(page + i); + + this_end =3D min_t(unsigned, end1, PAGE_SIZE); + if (end1 > start1) + memset(kaddr + start1, 0, this_end - start1); + end1 -=3D this_end; + start1 =3D 0; + + if (start2 >=3D PAGE_SIZE) { + start2 -=3D PAGE_SIZE; + end2 -=3D PAGE_SIZE; + } else { + this_end =3D min_t(unsigned, end2, PAGE_SIZE); + if (end2 > start2) + memset(kaddr + start2, 0, this_end - start2); + end2 -=3D this_end; + start2 =3D 0; + } + + kunmap_atomic(kaddr); + flush_dcache_page(page + i); + + if (!end1 && !end2) + break; + } + + BUG_ON((start1 | start2 | end1 | end2) !=3D 0); +} +EXPORT_SYMBOL(zero_user_segments); +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_HIGHMEM */ =20 #if defined(HASHED_PAGE_VIRTUAL) =20 --=20 2.28.0