From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 092A4CA9EB9 for ; Tue, 22 Oct 2019 12:30:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C27EF2184C for ; Tue, 22 Oct 2019 12:30:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=shipmail.org header.i=@shipmail.org header.b="j/cz1avO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C27EF2184C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shipmail.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 615156B0003; Tue, 22 Oct 2019 08:30:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C49F6B0006; Tue, 22 Oct 2019 08:30:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DADD6B0007; Tue, 22 Oct 2019 08:30:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0097.hostedemail.com [216.40.44.97]) by kanga.kvack.org (Postfix) with ESMTP id 2C7816B0003 for ; Tue, 22 Oct 2019 08:30:25 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id E0BE88249980 for ; Tue, 22 Oct 2019 12:30:24 +0000 (UTC) X-FDA: 76071353568.08.nerve37_3d89661c55534 X-HE-Tag: nerve37_3d89661c55534 X-Filterd-Recvd-Size: 5184 Received: from ste-pvt-msa1.bahnhof.se (ste-pvt-msa1.bahnhof.se [213.80.101.70]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Oct 2019 12:30:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ste-pvt-msa1.bahnhof.se (Postfix) with ESMTP id CFD9B3F41D; Tue, 22 Oct 2019 14:30:20 +0200 (CEST) Authentication-Results: ste-pvt-msa1.bahnhof.se; dkim=pass (1024-bit key; unprotected) header.d=shipmail.org header.i=@shipmail.org header.b=j/cz1avO; dkim-atps=neutral X-Virus-Scanned: Debian amavisd-new at bahnhof.se Received: from ste-pvt-msa1.bahnhof.se ([127.0.0.1]) by localhost (ste-pvt-msa1.bahnhof.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yHU_t1oiuxg7; Tue, 22 Oct 2019 14:30:19 +0200 (CEST) Received: from mail1.shipmail.org (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) (Authenticated sender: mb878879) by ste-pvt-msa1.bahnhof.se (Postfix) with ESMTPA id AADFF3F6B6; Tue, 22 Oct 2019 14:30:18 +0200 (CEST) Received: from localhost.localdomain.localdomain (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) by mail1.shipmail.org (Postfix) with ESMTPSA id E0E2C360150; Tue, 22 Oct 2019 14:30:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=shipmail.org; s=mail; t=1571747417; bh=LkBSK5Hwwhp8Ui1I2/duEhGAmp4GNTdWNqAL3TYlptk=; h=From:To:Cc:Subject:Date:From; b=j/cz1avOPL+hoh4VjqyGqaXnQdvQFmDoD121lv94PmcK8RcihgBCZG3W6lvPE5JyC 9rJF7WnIrTeDNOqNOC9mgYQDCs/2k5cBs26hxQ+vuxv7HW8BztxsRJzVDpjWByQCdb hf8JE+l+aNASDxeiuFPjyQOr8XmE6ZpF8lZzC/Qc= From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m=20=28VMware=29?= To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Thomas Hellstrom , Matthew Wilcox Subject: [PATCH] mm: Fix a huge pud insertion race during faulting Date: Tue, 22 Oct 2019 14:30:03 +0200 Message-Id: <20191022123003.37089-1-thomas_os@shipmail.org> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000036, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Thomas Hellstrom A huge pud page can theoretically be faulted in racing with pmd_alloc() in __handle_mm_fault(). That will lead to pmd_alloc() returning an invalid pmd pointer. Fix this by adding a pud_trans_unstable() function similar to pmd_trans_unstable() and check whether the pud is really stabl= e before using the pmd pointer. Race: Thread 1: Thread 2: Comment create_huge_pud() Fallback - not taken. create_huge_pud() Taken. pmd_alloc() Returns an invalid pointe= r. Cc: Matthew Wilcox Fixes: a00cc7d9dd93 ("mm, x86: add support for PUD-sized transparent huge= pages") Signed-off-by: Thomas Hellstrom --- include/asm-generic/pgtable.h | 25 +++++++++++++++++++++++++ mm/memory.c | 6 ++++++ 2 files changed, 31 insertions(+) diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.= h index 818691846c90..70c2058230ba 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -912,6 +912,31 @@ static inline int pud_trans_huge(pud_t pud) } #endif =20 +/* See pmd_none_or_trans_huge_or_clear_bad for discussion. */ +static inline int pud_none_or_trans_huge_or_dev_or_clear_bad(pud_t *pud) +{ + pud_t pudval =3D READ_ONCE(*pud); + + if (pud_none(pudval) || pud_trans_huge(pudval) || pud_devmap(pudval)) + return 1; + if (unlikely(pud_bad(pudval))) { + pud_clear_bad(pud); + return 1; + } + return 0; +} + +/* See pmd_trans_unstable for discussion. */ +static inline int pud_trans_unstable(pud_t *pud) +{ +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \ + defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) + return pud_none_or_trans_huge_or_dev_or_clear_bad(pud); +#else + return 0; +#endif +} + #ifndef pmd_read_atomic static inline pmd_t pmd_read_atomic(pmd_t *pmdp) { diff --git a/mm/memory.c b/mm/memory.c index b1ca51a079f2..43ff372f4f07 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3914,6 +3914,7 @@ static vm_fault_t __handle_mm_fault(struct vm_area_= struct *vma, vmf.pud =3D pud_alloc(mm, p4d, address); if (!vmf.pud) return VM_FAULT_OOM; +retry_pud: if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { ret =3D create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) @@ -3940,6 +3941,11 @@ static vm_fault_t __handle_mm_fault(struct vm_area= _struct *vma, vmf.pmd =3D pmd_alloc(mm, vmf.pud, address); if (!vmf.pmd) return VM_FAULT_OOM; + + /* Huge pud page fault raced with pmd_alloc? */ + if (pud_trans_unstable(vmf.pud)) + goto retry_pud; + if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) { ret =3D create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) --=20 2.21.0