From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C3F9C433E0 for ; Fri, 29 May 2020 03:37:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DE78320721 for ; Fri, 29 May 2020 03:37:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="kcPI2phi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DE78320721 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 876D6800B7; Thu, 28 May 2020 23:37:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 70E0580010; Thu, 28 May 2020 23:37:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64D8F800B7; Thu, 28 May 2020 23:37:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0238.hostedemail.com [216.40.44.238]) by kanga.kvack.org (Postfix) with ESMTP id 4A0FD80010 for ; Thu, 28 May 2020 23:37:36 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id F27D540E1 for ; Fri, 29 May 2020 03:37:35 +0000 (UTC) X-FDA: 76868346870.11.song78_42508ac279135 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id D7887180F8B80 for ; Fri, 29 May 2020 03:37:35 +0000 (UTC) X-HE-Tag: song78_42508ac279135 X-Filterd-Recvd-Size: 4458 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Fri, 29 May 2020 03:37:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=/2fZQu8w66RARx8vOLpWsju1X69BaYw4U1H3jkF+yV0=; b=kcPI2philKd18Jgn0HN8dfP7jN HB8Fc9cJnCn/S8r2aN2OlISHcO/ex/bsIJ5HPJUCjnyGUVbE2ILQpxwztQ4ymcPFj6ssJ0NkKNhTZ k2PhAPNzm+ioYHhFgjXTGe3xA81E3YzSsayCF92SXU1Y2w+uclRKim0cx2yw7bKOlvMvhx6GkQ+tl cT8BgRD1uVzMBy/AzRYyjXCke1aTZ+Zbby1a/+TSjLVQmC+OCFX2JfSOCVLG31irEBr5dazT0yvxs pK/jFhZ7PKn7LtrfYD7C3Har+Hu+E6RRtZhG6PAMo+18B4rFn8YNScNSY4J4mSQsSPl2x83Hb7aja 09yF9/TA==; Received: from willy by bombadil.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1jeVE3-0008SD-Ea; Fri, 29 May 2020 02:58:27 +0000 From: Matthew Wilcox To: linux-fsdevel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 23/39] mm: Allow large pages to be added to the page cache Date: Thu, 28 May 2020 19:58:08 -0700 Message-Id: <20200529025824.32296-24-willy@infradead.org> X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200529025824.32296-1-willy@infradead.org> References: <20200529025824.32296-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: D7887180F8B80 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" We return -EEXIST if there are any non-shadow entries in the page cache in the range covered by the large page. If there are multiple shadow entries in the range, we set *shadowp to one of them (currently the one at the highest index). If that turns out to be the wrong answer, we can implement something more complex. This is mostly modelled after the equivalent function in the shmem code. Signed-off-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 44 +++++++++++++++++++++++++++++++------------- 1 file changed, 31 insertions(+), 13 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 9abba062973a..437484d42b78 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -834,6 +834,7 @@ static int __add_to_page_cache_locked(struct page *pa= ge, int huge =3D PageHuge(page); struct mem_cgroup *memcg; int error; + unsigned int nr =3D 1; void *old; =20 VM_BUG_ON_PAGE(!PageLocked(page), page); @@ -845,31 +846,48 @@ static int __add_to_page_cache_locked(struct page *= page, gfp_mask, &memcg, false); if (error) return error; + xas_set_order(&xas, offset, thp_order(page)); + nr =3D hpage_nr_pages(page); } =20 - get_page(page); + page_ref_add(page, nr); page->mapping =3D mapping; page->index =3D offset; =20 do { + unsigned long exceptional =3D 0; + unsigned int i =3D 0; + xas_lock_irq(&xas); - old =3D xas_load(&xas); - if (old && !xa_is_value(old)) - xas_set_err(&xas, -EEXIST); - xas_store(&xas, page); + xas_for_each_conflict(&xas, old) { + if (!xa_is_value(old)) { + xas_set_err(&xas, -EEXIST); + break; + } + exceptional++; + if (shadowp) + *shadowp =3D old; + } + xas_create_range(&xas); if (xas_error(&xas)) goto unlock; =20 - if (xa_is_value(old)) { - mapping->nrexceptional--; - if (shadowp) - *shadowp =3D old; +next: + xas_store(&xas, page); + if (++i < nr) { + xas_next(&xas); + goto next; } - mapping->nrpages++; + mapping->nrexceptional -=3D exceptional; + mapping->nrpages +=3D nr; =20 /* hugetlb pages do not participate in page cache accounting */ - if (!huge) - __inc_node_page_state(page, NR_FILE_PAGES); + if (!huge) { + __mod_node_page_state(page_pgdat(page), NR_FILE_PAGES, + nr); + if (nr > 1) + __inc_node_page_state(page, NR_FILE_THPS); + } unlock: xas_unlock_irq(&xas); } while (xas_nomem(&xas, gfp_mask & GFP_RECLAIM_MASK)); @@ -886,7 +904,7 @@ static int __add_to_page_cache_locked(struct page *pa= ge, /* Leave page->index set: truncation relies upon it */ if (!huge) mem_cgroup_cancel_charge(page, memcg, false); - put_page(page); + page_ref_sub(page, nr); return xas_error(&xas); } ALLOW_ERROR_INJECTION(__add_to_page_cache_locked, ERRNO); --=20 2.26.2