From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <SRS0=rHNb=XY=kvack.org=owner-linux-mm@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no
	version=3.4.0
Received: from mail.kernel.org (mail.kernel.org [198.145.29.99])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 4DE90C4360C
	for <linux-mm@archiver.kernel.org>; Sun, 29 Sep 2019 22:50:00 +0000 (UTC)
Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17])
	by mail.kernel.org (Postfix) with ESMTP id E845D20815
	for <linux-mm@archiver.kernel.org>; Sun, 29 Sep 2019 22:49:59 +0000 (UTC)
Authentication-Results: mail.kernel.org;
	dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="acfudgvB"
DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E845D20815
Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com
Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org
Received: by kanga.kvack.org (Postfix)
	id 946F36B0003; Sun, 29 Sep 2019 18:49:59 -0400 (EDT)
Received: by kanga.kvack.org (Postfix, from userid 40)
	id 8F92C6B0005; Sun, 29 Sep 2019 18:49:59 -0400 (EDT)
X-Delivered-To: int-list-linux-mm@kvack.org
Received: by kanga.kvack.org (Postfix, from userid 63042)
	id 80E8B6B0006; Sun, 29 Sep 2019 18:49:59 -0400 (EDT)
X-Delivered-To: linux-mm@kvack.org
Received: from forelay.hostedemail.com (smtprelay0039.hostedemail.com [216.40.44.39])
	by kanga.kvack.org (Postfix) with ESMTP id 6057D6B0003
	for <linux-mm@kvack.org>; Sun, 29 Sep 2019 18:49:59 -0400 (EDT)
Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251])
	by forelay01.hostedemail.com (Postfix) with SMTP id F2979180AD803
	for <linux-mm@kvack.org>; Sun, 29 Sep 2019 22:49:58 +0000 (UTC)
X-FDA: 75989452476.10.flag61_39ef4597b6834
X-HE-Tag: flag61_39ef4597b6834
X-Filterd-Recvd-Size: 12025
Received: from hqemgate15.nvidia.com (hqemgate15.nvidia.com [216.228.121.64])
	by imf03.hostedemail.com (Postfix) with ESMTP
	for <linux-mm@kvack.org>; Sun, 29 Sep 2019 22:49:57 +0000 (UTC)
Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqemgate15.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA)
	id <B5d91351a0000>; Sun, 29 Sep 2019 15:50:03 -0700
Received: from hqmail.nvidia.com ([172.20.161.6])
  by hqpgpgate101.nvidia.com (PGP Universal service);
  Sun, 29 Sep 2019 15:49:55 -0700
X-PGP-Universal: processed;
	by hqpgpgate101.nvidia.com on Sun, 29 Sep 2019 15:49:55 -0700
Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL101.nvidia.com
 (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Sun, 29 Sep
 2019 22:49:54 +0000
Received: from [10.2.173.141] (10.124.1.5) by DRHQMAIL107.nvidia.com
 (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Sun, 29 Sep
 2019 22:49:54 +0000
Subject: Re: [PATCH v3 3/4] mm: don't expose non-hugetlb page to fast gup
 prematurely
From: John Hubbard <jhubbard@nvidia.com>
To: Michal Hocko <mhocko@kernel.org>
CC: "Kirill A. Shutemov" <kirill@shutemov.name>, Yu Zhao <yuzhao@google.com>,
	Peter Zijlstra <peterz@infradead.org>, Andrew Morton
	<akpm@linux-foundation.org>, "Kirill A . Shutemov"
	<kirill.shutemov@linux.intel.com>, Ingo Molnar <mingo@redhat.com>, "Arnaldo
 Carvalho de Melo" <acme@kernel.org>, Alexander Shishkin
	<alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, "Namhyung
 Kim" <namhyung@kernel.org>, Vlastimil Babka <vbabka@suse.cz>, Hugh Dickins
	<hughd@google.com>, =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= <jglisse@redhat.com>,
	Andrea Arcangeli <aarcange@redhat.com>, "Aneesh Kumar K . V"
	<aneesh.kumar@linux.ibm.com>, David Rientjes <rientjes@google.com>, "Matthew
 Wilcox" <willy@infradead.org>, Lance Roy <ldr709@gmail.com>, Ralph Campbell
	<rcampbell@nvidia.com>, Jason Gunthorpe <jgg@ziepe.ca>, Dave Airlie
	<airlied@redhat.com>, Thomas Hellstrom <thellstrom@vmware.com>, "Souptick
 Joarder" <jrdr.linux@gmail.com>, Mel Gorman <mgorman@suse.de>, Jan Kara
	<jack@suse.cz>, Mike Kravetz <mike.kravetz@oracle.com>, Huang Ying
	<ying.huang@intel.com>, Aaron Lu <ziqian.lzq@antfin.com>, Omar Sandoval
	<osandov@fb.com>, Thomas Gleixner <tglx@linutronix.de>, "Vineeth Remanan
 Pillai" <vpillai@digitalocean.com>, Daniel Jordan
	<daniel.m.jordan@oracle.com>, Mike Rapoport <rppt@linux.ibm.com>, "Joel
 Fernandes" <joel@joelfernandes.org>, Mark Rutland <mark.rutland@arm.com>,
	Alexander Duyck <alexander.h.duyck@linux.intel.com>, Pavel Tatashin
	<pavel.tatashin@microsoft.com>, David Hildenbrand <david@redhat.com>,
	"Juergen Gross" <jgross@suse.com>, Anthony Yznaga
	<anthony.yznaga@oracle.com>, "Johannes Weiner" <hannes@cmpxchg.org>, "Darrick
 J . Wong" <darrick.wong@oracle.com>, <linux-kernel@vger.kernel.org>,
	<linux-mm@kvack.org>, Leonardo Bras <leonardo@linux.ibm.com>
References: <20190914070518.112954-1-yuzhao@google.com>
 <20190924232459.214097-1-yuzhao@google.com>
 <20190924232459.214097-3-yuzhao@google.com>
 <20190925082530.GD4536@hirez.programming.kicks-ass.net>
 <20190925222654.GA180125@google.com> <20190926102036.od2wamdx2s7uznvq@box>
 <9465df76-0229-1b44-5646-5cced1bc1718@nvidia.com>
 <20190927123056.GE26848@dhcp22.suse.cz>
 <e0c74055-a112-b338-c885-9c33f0348a5a@nvidia.com>
X-Nvconfidentiality: public
Message-ID: <6fd4f528-709e-ebe1-801c-bd3841f4d27b@nvidia.com>
Date: Sun, 29 Sep 2019 15:47:39 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.8.0
MIME-Version: 1.0
In-Reply-To: <e0c74055-a112-b338-c885-9c33f0348a5a@nvidia.com>
X-Originating-IP: [10.124.1.5]
X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To
 DRHQMAIL107.nvidia.com (10.27.9.16)
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
	t=1569797403; bh=+MeODZwjKvB/oEIv6eOJLhaWoP01uTMvCz8nXft0T3s=;
	h=X-PGP-Universal:Subject:From:To:CC:References:X-Nvconfidentiality:
	 Message-ID:Date:User-Agent:MIME-Version:In-Reply-To:
	 X-Originating-IP:X-ClientProxiedBy:Content-Type:Content-Language:
	 Content-Transfer-Encoding;
	b=acfudgvBoSgKQa8TmQGKwohwx5Jwvg5oHBJTNZqXhOVLWgicUlQpOt2rhuXa1ipq1
	 f0jp9KP4sZIP+IJh1EGmY4g2WcqfyDaXMdC8/2u2ZwyUwqGXpdnJitIDfM4afROpM0
	 ZUHTAu7KrV5NUbUlCgl932a26v/7as4HVtrK0Qg58JHIuCj8WqHC08Iw45TuMtUVVq
	 Nh7pwoOtN6CueoTEPkcZnBLqh1wHeHMGOYv6d/pun/5bw/LB33I0ruNcU6wLlwyi70
	 RZBinni3KE3xgsehTvf6sY9jdxmxzsWXj/a32+hnkIFFNL6J0wBVPR6MZC6AeOmGa/
	 pUtb7OUpAiHng==
X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4
Sender: owner-linux-mm@kvack.org
Precedence: bulk
X-Loop: owner-majordomo@kvack.org
List-ID: <linux-mm.kvack.org>

On 9/27/19 12:31 PM, John Hubbard wrote:
> On 9/27/19 5:33 AM, Michal Hocko wrote:
>> On Thu 26-09-19 20:26:46, John Hubbard wrote:
>>> On 9/26/19 3:20 AM, Kirill A. Shutemov wrote:
>>>> BTW, have you looked at other levels of page table hierarchy. Do we ha=
ve
>>>> the same issue for PMD/PUD/... pages?
>>>>
>>>
>>> Along the lines of "what other memory barriers might be missing for
>>> get_user_pages_fast(), I'm also concerned that the synchronization betw=
een
>>> get_user_pages_fast() and freeing the page tables might be technically =
broken,
>>> due to missing memory barriers on the get_user_pages_fast() side. Detai=
ls:
>>>
>>> gup_fast() disables interrupts, but I think it also needs some sort of
>>> memory barrier(s), in order to prevent reads of the page table (gup_pgd=
_range,
>>> etc) from speculatively happening before the interrupts are disabled.
>>
>> Could you be more specific about the race scenario please? I thought
>> that the unmap path will be serialized by the pte lock.
>>
>=20
> I don't see a pte lock anywhere here.
>=20
> This case is really pretty far out there, but without run-time memory bar=
riers
> I don't think we can completely rule it out:
>=20
> CPU 0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 CPU 1
> --------=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 ---------------
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 get_user_pages_fa=
st()
>=20
> do_unmap()
>  =C2=A0unmap_region()
>  =C2=A0 free_pgtables()
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /*
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * speculati=
ve reads, re-ordered
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * by the CP=
U at run time, not
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * compile t=
ime. The function calls
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * are not r=
eally in this order, but
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * the corre=
sponding reads could be.
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 gup_pgd_range()
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 gup_p4d_ran=
ge()
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 gup_p=
ud_range()
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 gup_pmd_range()
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 pmd =3D READ_ONCE(*pmdp)
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 /* This is stale */
>=20
>  =C2=A0 tlb_finish_mmu()
>  =C2=A0=C2=A0=C2=A0 tlb_flush_mmu()
>  =C2=A0=C2=A0=C2=A0=C2=A0 tlb_flush_mmu_tlbonly()
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 tlb_flush()
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 flush_tlb_mm
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 flush_tlb_mm_range
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 flush_tlb_others
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 native_flus=
h_tlb_others
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 smp_c=
all_function_many: IPIs
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 ...blocks until CPU1 reenables
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 interrupts
>=20
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 local_irq_disable()
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 ...continue the page walk based
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 on stale/freed data...
>=20


...and note that I just asked Leonardo Bras (+CC) to include "the" fix[1] f=
or that (I proposed
a snippet that attempts a fix anyway), in his upcoming feature that speeds =
up PPC THP
operations.

So I hope that doesn't muddy this thread too much, but the other one is awf=
ully quiet
and I was worried that the overlapping changes might go unnoticed.

[1] https://lore.kernel.org/r/4ff1e8e8-929b-9cfc-9bf8-ee88e34de888@nvidia.c=
om

thanks,
--=20
John Hubbard
NVIDIA