From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <SRS0=0n9M=XZ=kvack.org=owner-linux-mm@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.6 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no
	version=3.4.0
Received: from mail.kernel.org (mail.kernel.org [198.145.29.99])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 48E6BC32792
	for <linux-mm@archiver.kernel.org>; Mon, 30 Sep 2019 17:59:29 +0000 (UTC)
Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17])
	by mail.kernel.org (Postfix) with ESMTP id 05DD8224EC
	for <linux-mm@archiver.kernel.org>; Mon, 30 Sep 2019 17:59:28 +0000 (UTC)
Authentication-Results: mail.kernel.org;
	dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="Yt47av+Y"
DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 05DD8224EC
Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com
Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org
Received: by kanga.kvack.org (Postfix)
	id 797FC6B0003; Mon, 30 Sep 2019 13:59:28 -0400 (EDT)
Received: by kanga.kvack.org (Postfix, from userid 40)
	id 721E16B0006; Mon, 30 Sep 2019 13:59:28 -0400 (EDT)
X-Delivered-To: int-list-linux-mm@kvack.org
Received: by kanga.kvack.org (Postfix, from userid 63042)
	id 5C9C06B0007; Mon, 30 Sep 2019 13:59:28 -0400 (EDT)
X-Delivered-To: linux-mm@kvack.org
Received: from forelay.hostedemail.com (smtprelay0220.hostedemail.com [216.40.44.220])
	by kanga.kvack.org (Postfix) with ESMTP id 319736B0003
	for <linux-mm@kvack.org>; Mon, 30 Sep 2019 13:59:28 -0400 (EDT)
Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251])
	by forelay04.hostedemail.com (Postfix) with SMTP id BDC493499
	for <linux-mm@kvack.org>; Mon, 30 Sep 2019 17:59:27 +0000 (UTC)
X-FDA: 75992349174.23.fuel98_35432aa31cb4c
X-HE-Tag: fuel98_35432aa31cb4c
X-Filterd-Recvd-Size: 7544
Received: from hqemgate15.nvidia.com (hqemgate15.nvidia.com [216.228.121.64])
	by imf44.hostedemail.com (Postfix) with ESMTP
	for <linux-mm@kvack.org>; Mon, 30 Sep 2019 17:59:25 +0000 (UTC)
Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqemgate15.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA)
	id <B5d9242830000>; Mon, 30 Sep 2019 10:59:31 -0700
Received: from hqmail.nvidia.com ([172.20.161.6])
  by hqpgpgate101.nvidia.com (PGP Universal service);
  Mon, 30 Sep 2019 10:59:23 -0700
X-PGP-Universal: processed;
	by hqpgpgate101.nvidia.com on Mon, 30 Sep 2019 10:59:23 -0700
Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL107.nvidia.com
 (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 30 Sep
 2019 17:59:22 +0000
Received: from [10.2.173.141] (10.124.1.5) by DRHQMAIL107.nvidia.com
 (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 30 Sep
 2019 17:59:22 +0000
Subject: Re: [PATCH v3 3/4] mm: don't expose non-hugetlb page to fast gup
 prematurely
To: Jan Kara <jack@suse.cz>
CC: Michal Hocko <mhocko@kernel.org>, "Kirill A. Shutemov"
	<kirill@shutemov.name>, Yu Zhao <yuzhao@google.com>, Peter Zijlstra
	<peterz@infradead.org>, Andrew Morton <akpm@linux-foundation.org>, "Kirill A
 . Shutemov" <kirill.shutemov@linux.intel.com>, Ingo Molnar
	<mingo@redhat.com>, Arnaldo Carvalho de Melo <acme@kernel.org>, "Alexander
 Shishkin" <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>,
	Namhyung Kim <namhyung@kernel.org>, Vlastimil Babka <vbabka@suse.cz>, "Hugh
 Dickins" <hughd@google.com>, =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?=
	<jglisse@redhat.com>, Andrea Arcangeli <aarcange@redhat.com>, "Aneesh Kumar K
 . V" <aneesh.kumar@linux.ibm.com>, David Rientjes <rientjes@google.com>,
	Matthew Wilcox <willy@infradead.org>, Lance Roy <ldr709@gmail.com>, "Ralph
 Campbell" <rcampbell@nvidia.com>, Jason Gunthorpe <jgg@ziepe.ca>, Dave Airlie
	<airlied@redhat.com>, Thomas Hellstrom <thellstrom@vmware.com>, "Souptick
 Joarder" <jrdr.linux@gmail.com>, Mel Gorman <mgorman@suse.de>, Mike Kravetz
	<mike.kravetz@oracle.com>, Huang Ying <ying.huang@intel.com>, Aaron Lu
	<ziqian.lzq@antfin.com>, Omar Sandoval <osandov@fb.com>, Thomas Gleixner
	<tglx@linutronix.de>, Vineeth Remanan Pillai <vpillai@digitalocean.com>,
	Daniel Jordan <daniel.m.jordan@oracle.com>, Mike Rapoport
	<rppt@linux.ibm.com>, Joel Fernandes <joel@joelfernandes.org>, Mark Rutland
	<mark.rutland@arm.com>, Alexander Duyck <alexander.h.duyck@linux.intel.com>,
	Pavel Tatashin <pavel.tatashin@microsoft.com>, David Hildenbrand
	<david@redhat.com>, Juergen Gross <jgross@suse.com>, Anthony Yznaga
	<anthony.yznaga@oracle.com>, Johannes Weiner <hannes@cmpxchg.org>, "Darrick J
 . Wong" <darrick.wong@oracle.com>, <linux-kernel@vger.kernel.org>,
	<linux-mm@kvack.org>
References: <20190914070518.112954-1-yuzhao@google.com>
 <20190924232459.214097-1-yuzhao@google.com>
 <20190924232459.214097-3-yuzhao@google.com>
 <20190925082530.GD4536@hirez.programming.kicks-ass.net>
 <20190925222654.GA180125@google.com> <20190926102036.od2wamdx2s7uznvq@box>
 <9465df76-0229-1b44-5646-5cced1bc1718@nvidia.com>
 <20190927123056.GE26848@dhcp22.suse.cz>
 <e0c74055-a112-b338-c885-9c33f0348a5a@nvidia.com>
 <20190930092003.GA22118@quack2.suse.cz>
X-Nvconfidentiality: public
From: John Hubbard <jhubbard@nvidia.com>
Message-ID: <6bba357a-1706-7cdb-8a11-359157a21ae8@nvidia.com>
Date: Mon, 30 Sep 2019 10:57:08 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.8.0
MIME-Version: 1.0
In-Reply-To: <20190930092003.GA22118@quack2.suse.cz>
X-Originating-IP: [10.124.1.5]
X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To
 DRHQMAIL107.nvidia.com (10.27.9.16)
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
	t=1569866371; bh=5VGWEZSheMEV73elpoXHxZPJaH+UXxXob5Und3S7BSc=;
	h=X-PGP-Universal:Subject:To:CC:References:X-Nvconfidentiality:From:
	 Message-ID:Date:User-Agent:MIME-Version:In-Reply-To:
	 X-Originating-IP:X-ClientProxiedBy:Content-Type:Content-Language:
	 Content-Transfer-Encoding;
	b=Yt47av+Y0Bi3OCY5HHJHOHD8UfUIJichTqprLrYAWW8fOWKTBZ+H3+2F8kcn9fPr2
	 ChizxMxNpGKWn00xKu/+KzTp73OHMBgP+RG7MIaUwMfgzhjM2AnC/i+7Sj0GLhVDHs
	 Be2QzWCtlY5OZqo18OKNy2nLOEc92Q76tk/AwVLP6WsmvbVSsk1IuBsl409MytgEhG
	 nx4gbnXwpzyy3+1fTz2dpp1LBRxuTF0MCYDAvRiswHxajmCL2v0ccNdXpxL/OGPfQk
	 KAz6Fz09LOFsKkRKvClMtGJ00UhtBk9KRHCjxbEJbwLwyc0NJHDCZtuBud1JTCdw/S
	 sGZtiuv8SFxDQ==
X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4
Sender: owner-linux-mm@kvack.org
Precedence: bulk
X-Loop: owner-majordomo@kvack.org
List-ID: <linux-mm.kvack.org>

On 9/30/19 2:20 AM, Jan Kara wrote:
> On Fri 27-09-19 12:31:41, John Hubbard wrote:
>> On 9/27/19 5:33 AM, Michal Hocko wrote:
>>> On Thu 26-09-19 20:26:46, John Hubbard wrote:
>>>> On 9/26/19 3:20 AM, Kirill A. Shutemov wrote:
...
>> CPU 0                              CPU 1
>> --------                           ---------------
>>                                     get_user_pages_fast()
>>
>> do_unmap()
>>   unmap_region()
>>    free_pgtables()
>>                                     /*
>>                                      * speculative reads, re-ordered
>>                                      * by the CPU at run time, not
>>                                      * compile time. The function calls
>>                                      * are not really in this order, but
>>                                      * the corresponding reads could be.
>>                                      */
>>                                     gup_pgd_range()
>>                                      gup_p4d_range()
>>                                       gup_pud_range()
>>                                        gup_pmd_range()
>>                                         pmd = READ_ONCE(*pmdp)
>>                                          /* This is stale */
>>
>>    tlb_finish_mmu()
>>      tlb_flush_mmu()
>>       tlb_flush_mmu_tlbonly()
>>         tlb_flush()
>>          flush_tlb_mm
>>           flush_tlb_mm_range
>>            flush_tlb_others
>>             native_flush_tlb_others
>>              smp_call_function_many: IPIs
>>               ...blocks until CPU1 reenables
>>                  interrupts
>>
>>                                          local_irq_disable()
>>                                           ...continue the page walk based
>>                                              on stale/freed data...
> 
> Yes, but then we have:
> 
>                  head = try_get_compound_head(page, 1);
> 
> which has an atomic operation providing barrier in it and then we have:
> 
>                  if (unlikely(pte_val(pte) != pte_val(*ptep))) {
>                          put_page(head);
>                          goto pte_unmap;
>                  }
> 
> So we reload PTE again and check the value didn't change. Which should
> prevent the race you explain above. Or do I miss anything?


Well, a couple of questions:

1. Is there *really* a memory barrier in try_get_compound_head()? Because
I only see a READ_ONCE compile barrier, which would mean that run time
reordering is still possible.

2. Your code above is after the "pmd = READ_ONCE(*pmdp)" line, so by then,
it's already found the pte based on reading a stale pmd. So checking the
pte seems like it's checking the wrong thing--it's too late, for this case,
right?


thanks,
-- 
John Hubbard
NVIDIA