From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5F6BFC6194 for ; Fri, 8 Nov 2019 09:38:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9FCD8214DA for ; Fri, 8 Nov 2019 09:38:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9FCD8214DA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 81CD66B0007; Fri, 8 Nov 2019 04:38:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CE236B0008; Fri, 8 Nov 2019 04:38:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6BDB76B000C; Fri, 8 Nov 2019 04:38:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0136.hostedemail.com [216.40.44.136]) by kanga.kvack.org (Postfix) with ESMTP id 535C36B0008 for ; Fri, 8 Nov 2019 04:38:27 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 0F6F5BBC8 for ; Fri, 8 Nov 2019 09:38:27 +0000 (UTC) X-FDA: 76132609854.14.shape30_83ee7115f2526 X-HE-Tag: shape30_83ee7115f2526 X-Filterd-Recvd-Size: 5031 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Fri, 8 Nov 2019 09:38:26 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 99ECAAE61; Fri, 8 Nov 2019 09:38:24 +0000 (UTC) From: Vlastimil Babka To: stable@vger.kernel.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Ajay Kaher , Vlastimil Babka , Oscar Salvador , Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Juergen Gross , "Kirill A . Shutemov" , Vitaly Kuznetsov , Linus Torvalds , Borislav Petkov , Dave Hansen , Andy Lutomirski Subject: [PATCH STABLE 4.4 8/8] x86, mm, gup: prevent get_page() race with munmap in paravirt guest Date: Fri, 8 Nov 2019 10:38:14 +0100 Message-Id: <20191108093814.16032-9-vbabka@suse.cz> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191108093814.16032-1-vbabka@suse.cz> References: <20191108093814.16032-1-vbabka@suse.cz> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The x86 version of get_user_pages_fast() relies on disabled interrupts to synchronize gup_pte_range() between gup_get_pte(ptep); and get_page() aga= inst a parallel munmap. The munmap side nulls the pte, then flushes TLBs, then releases the page. As TLB flush is done synchronously via IPI disabling interrupts blocks the page release, and get_page(), which assumes existin= g reference on page, is thus safe. However when TLB flush is done by a hypercall, e.g. in a Xen PV guest, th= ere is no blocking thanks to disabled interrupts, and get_page() can succeed on = a page that was already freed or even reused. We have recently seen this happen with our 4.4 and 4.12 based kernels, wi= th userspace (java) that exits a thread, where mm_release() performs a futex= _wake() on tsk->clear_child_tid, and another thread in parallel unmaps the page w= here tsk->clear_child_tid points to. The spurious get_page() succeeds, but fut= ex code immediately releases the page again, while it's already on a freelist. Sy= mptoms include a bad page state warning, general protection faults acessing a po= isoned list prev/next pointer in the freelist, or free page pcplists of two cpus= joined together in a single list. Oscar has also reproduced this scenario, with = a patch inserting delays before the get_page() to make the race window larg= er. Fix this by removing the dependency on TLB flush interrupts the same way = as the generic get_user_pages_fast() code by using page_cache_add_speculative() = and revalidating the PTE contents after pinning the page. Mainline is safe si= nce 4.13 where the x86 gup code was removed in favor of the common code. Acce= ssing the page table itself safely also relies on disabled interrupts and TLB f= lush IPIs that don't happen with hypercalls, which was acknowledged in commit 9e52fc2b50de ("x86/mm: Enable RCU based page table freeing (CONFIG_HAVE_RCU_TABLE_FREE=3Dy)"). That commit with follups should also = be backported for full safety, although our reproducer didn't hit a problem without that backport. Reproduced-by: Oscar Salvador Signed-off-by: Vlastimil Babka Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juergen Gross Cc: Kirill A. Shutemov Cc: Vitaly Kuznetsov Cc: Linus Torvalds Cc: Borislav Petkov Cc: Dave Hansen Cc: Andy Lutomirski Signed-off-by: Vlastimil Babka --- arch/x86/mm/gup.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/x86/mm/gup.c b/arch/x86/mm/gup.c index 6612d532e42e..6379a4883c0a 100644 --- a/arch/x86/mm/gup.c +++ b/arch/x86/mm/gup.c @@ -9,6 +9,7 @@ #include #include #include +#include =20 #include =20 @@ -95,10 +96,23 @@ static noinline int gup_pte_range(pmd_t pmd, unsigned= long addr, } VM_BUG_ON(!pfn_valid(pte_pfn(pte))); page =3D pte_page(pte); - if (unlikely(!try_get_page(page))) { + + if (WARN_ON_ONCE(page_ref_count(page) < 0)) { + pte_unmap(ptep); + return 0; + } + + if (!page_cache_get_speculative(page)) { pte_unmap(ptep); return 0; } + + if (unlikely(pte_val(pte) !=3D pte_val(*ptep))) { + put_page(page); + pte_unmap(ptep); + return 0; + } + SetPageReferenced(page); pages[*nr] =3D page; (*nr)++; --=20 2.23.0