From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AED19C433DB for ; Mon, 18 Jan 2021 02:50:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2EC2E20738 for ; Mon, 18 Jan 2021 02:50:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2EC2E20738 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 69BEA6B0298; Sun, 17 Jan 2021 21:49:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 64C666B0299; Sun, 17 Jan 2021 21:49:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4EB726B029E; Sun, 17 Jan 2021 21:49:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0086.hostedemail.com [216.40.44.86]) by kanga.kvack.org (Postfix) with ESMTP id 368B26B0298 for ; Sun, 17 Jan 2021 21:49:59 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 09C6B364F for ; Mon, 18 Jan 2021 02:49:59 +0000 (UTC) X-FDA: 77717366118.01.car43_3c11dc527545 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id D552D1004DD6F for ; Mon, 18 Jan 2021 02:49:58 +0000 (UTC) X-HE-Tag: car43_3c11dc527545 X-Filterd-Recvd-Size: 8331 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Mon, 18 Jan 2021 02:49:58 +0000 (UTC) Received: by mail-pl1-f172.google.com with SMTP id g3so7856054plp.2 for ; Sun, 17 Jan 2021 18:49:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=oyXzGEzTVn5EWmDkOEirveY6pyyWOy/cjy9jZHLDBhA=; b=KifJQoJMSkkF7WXcAEylNnC9/t4DxId/faenhimlV5cXFeazVaBXk0i3oZJ6UU1egz pTAqV7AkQ4OpM9Ona1XSsnZwrjKtnt/IxSCoIk50vehJ0L5Eo6EqVv9d2Vl6zSa2NZF8 FQwbmzYRMlj73hBYYa5TF/gC/XDFuGKyhJUvc+bZIvi/noVOZTc6vQYBZleeLkqdNDq1 Wq6h91hq5f5zcsc6MquHz3Hnf4mZATQrbm+uHfiAYKg2G83UL06VBaJnPeEOJiMeVXGn 04ch8EV1O5Ym8AtvuCnEPH76Gn9VQD9YQYtmuuyNVhTUsMBLu2FuQrYZMlg+lz19fcSm d43w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=oyXzGEzTVn5EWmDkOEirveY6pyyWOy/cjy9jZHLDBhA=; b=d9HwagwSGpE3B/kc7dK+yi+JATY0gTR4T2LQXPPFfQSEQSuoDLfUwGjZRrKGviwlQs ERlVTHNtDynVEcaczGnVSs7Aw/2chYUnktsSeLCXyZ650YvaSw5u2xuyNGZeZ6i5oa62 LOUf4rJWqbxIqv0NXyqSB7ukqb3nDAVsKyD9j7+PTWolFkjtJlqM5exdxNxlavcUo+qa ZyTIYD0j+OqCk+2BOAsX7RpQnIt3UBoSVzIfZw0kTSltMGvgR+fK1ZA3bfSOJ66pKNgp qoHg+JSGqjJDkliGLL+gXOuHmEMLxGcMhqMWL1AUOvRz86C/Q63touu3tW4gIw/D1Qff kgAg== X-Gm-Message-State: AOAM530Zd/YtvOkN1ANpgx+jQYDLQ9B2p1y/s2T2RSgRLRltKXL+vZXT SIBEhly0Z38dqcQQKfCWLLk= X-Google-Smtp-Source: ABdhPJy/0yfCh/m/qz03tbvSbW0EkJJWVQLFeNt/sZWm409X5BfXwzgIv6MVeKj2qEn7AHbmX2Tjrg== X-Received: by 2002:a17:902:ecca:b029:de:b5bc:c852 with SMTP id a10-20020a170902eccab02900deb5bcc852mr1610645plh.59.1610938197040; Sun, 17 Jan 2021 18:49:57 -0800 (PST) Received: from [192.168.88.245] (c-24-6-216-183.hsd1.ca.comcast.net. [24.6.216.183]) by smtp.gmail.com with ESMTPSA id 85sm6223715pfc.39.2021.01.17.18.49.54 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 17 Jan 2021 18:49:56 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: [PATCH] mm/userfaultfd: fix memory corruption due to writeprotect From: Nadav Amit In-Reply-To: Date: Sun, 17 Jan 2021 18:49:53 -0800 Cc: Will Deacon , Laurent Dufour , Peter Zijlstra , Vinayak Menon , Linus Torvalds , Andy Lutomirski , Peter Xu , Andrea Arcangeli , linux-mm , lkml , Pavel Emelyanov , Mike Kravetz , Mike Rapoport , stable , Minchan Kim , surenb@google.com, Mel Gorman , cai@lca.pw Content-Transfer-Encoding: quoted-printable Message-Id: <0BB9D243-F527-409D-8A9E-612DFD4EE993@gmail.com> References: <2C7AE23B-ACA3-4D55-A907-AF781C5608F0@gmail.com> <20210112214337.GA10434@willie-the-truck> <85DAADF4-2537-40BD-8580-A57C201FF5F3@gmail.com> <1A664155-462A-451D-A21E-D749A0ADBD09@gmail.com> To: Yu Zhao X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > On Jan 17, 2021, at 11:25 AM, Yu Zhao wrote: >=20 > On Sun, Jan 17, 2021 at 02:13:43AM -0800, Nadav Amit wrote: >>> On Jan 17, 2021, at 1:16 AM, Yu Zhao wrote: >>>=20 >>> On Sat, Jan 16, 2021 at 11:32:22PM -0800, Nadav Amit wrote: >>>>> On Jan 16, 2021, at 8:41 PM, Yu Zhao wrote: >>>>>=20 >>>>> On Tue, Jan 12, 2021 at 09:43:38PM +0000, Will Deacon wrote: >>>>>> On Tue, Jan 12, 2021 at 12:38:34PM -0800, Nadav Amit wrote: >>>>>>>> On Jan 12, 2021, at 11:56 AM, Yu Zhao = wrote: >>>>>>>> On Tue, Jan 12, 2021 at 11:15:43AM -0800, Nadav Amit wrote: >>>>>>>>> I will send an RFC soon for per-table deferred TLB flushes = tracking. >>>>>>>>> The basic idea is to save a generation in the page-struct that = tracks >>>>>>>>> when deferred PTE change took place, and track whenever a TLB = flush >>>>>>>>> completed. In addition, other users - such as mprotect - would = use >>>>>>>>> the tlb_gather interface. >>>>>>>>>=20 >>>>>>>>> Unfortunately, due to limited space in page-struct this would = only >>>>>>>>> be possible for 64-bit (and my implementation is only for = x86-64). >>>>>>>>=20 >>>>>>>> I don't want to discourage you but I don't think this would end = up >>>>>>>> well. PPC doesn't necessarily follow one-page-struct-per-table = rule, >>>>>>>> and I've run into problems with this before while trying to do >>>>>>>> something similar. >>>>>>>=20 >>>>>>> Discourage, discourage. Better now than later. >>>>>>>=20 >>>>>>> It will be relatively easy to extend the scheme to be per-VMA = instead of >>>>>>> per-table for architectures that prefer it this way. It does = require >>>>>>> TLB-generation tracking though, which Andy only implemented for = x86, so I >>>>>>> will focus on x86-64 right now. >>>>>>=20 >>>>>> Can you remind me of what we're missing on arm64 in this area, = please? I'm >>>>>> happy to help get this up and running once you have something I = can build >>>>>> on. >>>>>=20 >>>>> I noticed arm/arm64 don't support = ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH. >>>>> Would it be something worth pursuing? Arm has been using = mm_cpumask, >>>>> so it might not be too difficult I guess? >>>>=20 >>>> [ +Mel Gorman who implemented ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH ] >>>>=20 >>>> IIUC, there are at least two bugs in x86 implementation. >>>>=20 >>>> First, there is a missing memory barrier in tlbbatch_add_mm() = between >>>> inc_mm_tlb_gen() and the read of mm_cpumask(). >>>=20 >>> In arch_tlbbatch_add_mm()? inc_mm_tlb_gen() has builtin barrier as = its >>> comment says -- atomic update ops that return values are also full >>> memory barriers. >>=20 >> Yes, you are correct. >>=20 >>>> Second, try_to_unmap_flush() clears flush_required after flushing. = Another >>>> thread can call set_tlb_ubc_flush_pending() after the flush and = before >>>> flush_required is cleared, and the indication that a TLB flush is = pending >>>> can be lost. >>>=20 >>> This isn't a problem either because flush_required is per thread. >>=20 >> Sorry, I meant mm->tlb_flush_batched . It is not per-thread. >> flush_tlb_batched_pending() clears it after flush and indications = that >> set_tlb_ubc_flush_pending() sets in between can be lost. >=20 > Hmm, the PTL argument above flush_tlb_batched_pending() doesn't seem > to hold when USE_SPLIT_PTE_PTLOCKS is set. Do you have a reproducer? > KCSAN might be able to help in this case. I do not have a reproducer. It is just based on my understanding of this code. I will give a short try for building a reproducer, although for some = reason =E2=80=9Cyou guys=E2=80=9D complain that my reproducers do not work for = you (is it PTI that I disable? idle=3Dpoll? running in a VM?). It is also not likely to be = too easy to build a reproducer that actually triggers a memory corruption. Anyhow, apparently KCSAN has already shouted about this code, causing = Qian Cai to add "data_race()" to avoid KCSAN from shouting (9c1177b62a8c "mm/rmap: annotate a data race at tlb_flush_batched=E2=80=9D). Note that Andrea asked me not to hijack this thread and have a different = one on this issue.