From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC42AC433FE for ; Wed, 9 Dec 2020 15:15:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 86E5923AAC for ; Wed, 9 Dec 2020 15:15:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 86E5923AAC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DEE778D0023; Wed, 9 Dec 2020 10:15:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D9F4C8D001E; Wed, 9 Dec 2020 10:15:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C40C48D0023; Wed, 9 Dec 2020 10:15:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0137.hostedemail.com [216.40.44.137]) by kanga.kvack.org (Postfix) with ESMTP id ABD988D001E for ; Wed, 9 Dec 2020 10:15:08 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 63BCB1EE6 for ; Wed, 9 Dec 2020 15:15:08 +0000 (UTC) X-FDA: 77574091896.09.feast18_540e9e6273f0 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id 41355180AD80F for ; Wed, 9 Dec 2020 15:15:08 +0000 (UTC) X-HE-Tag: feast18_540e9e6273f0 X-Filterd-Recvd-Size: 5685 Received: from mail-qk1-f196.google.com (mail-qk1-f196.google.com [209.85.222.196]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Wed, 9 Dec 2020 15:15:07 +0000 (UTC) Received: by mail-qk1-f196.google.com with SMTP id q22so1494949qkq.6 for ; Wed, 09 Dec 2020 07:15:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=MokHiqZv6KYiy318vGHlmWXx0eO3qIVrW7oaOuNpGB4=; b=IzBWi50rmJpoiboQi4kX5gzP2g8Yh9F9sjfv7POK8VBP8Xbz40narUo0IYcXSParIU sGviJ6qFU+4CXLvphkd1mFZsbwYlEQo2lcLyIHaOitgL9ph6zpNTFqivd0y3UIdNs5oe wmDdfem4LgQEcvyS8FmW+G2iENq1BlWuXP3iB7cqSz+NwNbOfn3JaNmTEpDETjfe1yYV +lQgjx6vYAThI1yxepjGuRWmgQ39Q1GR2x3wKWFFCPz4MahSN0Y9mAhXMm2B3gALzzLL 0wEC1LAaHTvjCe5aYW71EGJs+LkqHFIJ7/j1qi2buC9TC0ujV3T9i26lCACXiMWVh0dg NFMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=MokHiqZv6KYiy318vGHlmWXx0eO3qIVrW7oaOuNpGB4=; b=mHd1Sbjs+HowMqnf+3WJ97znjiokNKGu25nzNaqE2ATsl5JKP+Qu1csN9dMZKbs2a/ i1pURtx+evnpW0RzkHdC0AORjBp5k04iiJG1QB/ZeewIyVWwp0DQYuhdIci7rCIkJdGd cocucZrgHKXgewNYFeqvCrUONNWTzedN/YnhXmyn8kHqMUE4OI8AJkaGQmO+hL50zIhu dDs02II43QktmWaRkBNW+zUUKxLkBbPvGNCzACc3ZVrZ69Ts8oGFtqdzghr9qIWH94Yk 5S789M/fAun+3VxsWQL1/awQhLxUOLGX4ve/j20cG85rau/26gDUwAPb0Tw/WAk+SNhi hrXQ== X-Gm-Message-State: AOAM532Jl5dQWbIsnyKEYkdfQ88xRof0ONZUDB+7Td9dCUpK/uJdUpIi +XgcSWzDWrpcm7d92vliOyjBTA== X-Google-Smtp-Source: ABdhPJxTY0OoiBFV+43jCrW0bRZ2HiIUKdI2A24uRorkOLeCIW10T1XVa15iCTuzQMGxM2mDpx/Frg== X-Received: by 2002:a05:620a:483:: with SMTP id 3mr3553956qkr.328.1607526906981; Wed, 09 Dec 2020 07:15:06 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-162-115-133.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.115.133]) by smtp.gmail.com with ESMTPSA id 190sm672740qkf.61.2020.12.09.07.15.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Dec 2020 07:15:06 -0800 (PST) Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1kn1BJ-008Gdd-Nu; Wed, 09 Dec 2020 11:15:05 -0400 Date: Wed, 9 Dec 2020 11:15:05 -0400 From: Jason Gunthorpe To: Joao Martins Cc: linux-mm@kvack.org, Dan Williams , Ira Weiny , linux-nvdimm@lists.01.org, Matthew Wilcox , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton Subject: Re: [PATCH RFC 6/9] mm/gup: Grab head page refcount once for group of subpages Message-ID: <20201209151505.GV5487@ziepe.ca> References: <20201208172901.17384-1-joao.m.martins@oracle.com> <20201208172901.17384-8-joao.m.martins@oracle.com> <20201208194905.GQ5487@ziepe.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Dec 09, 2020 at 11:05:39AM +0000, Joao Martins wrote: > > Why is all of this special? Any time we see a PMD/PGD/etc pointing to > > PFN we can apply this optimization. How come device has its own > > special path to do this?? > > I think the reason is that zone_device struct pages have no > relationship to one other. So you anyways need to change individual > pages, as opposed to just the head page. Huh? That can't be, unpin doesn't know the memory type when it unpins it, and as your series shows unpin always operates on the compound head. Thus pinning must also operate on compound heads > I made it special to avoid breaking other ZONE_DEVICE users (and > gating that with PGMAP_COMPOUND). But if there's no concerns with > that, I can unilaterally enable it. I didn't understand what PGMAP_COMPOUND was supposed to be for.. > > Why do we need to check PGMAP_COMPOUND? Why do we need to get pgmap? > > (We already removed that from the hmm version of this, was that wrong? > > Is this different?) Dan? And this is the key question - why do we need to get a pgmap here? I'm going to assert that a pgmap cannot be destroyed concurrently with fast gup running. This is surely true on x86 as the TLB flush that must have preceeded a pgmap destroy excludes fast gup. Other arches must emulate this in their pgmap implementations. So, why do we need pgmap here? Hoping Dan might know If we delete the pgmap then the devmap stop being special. CH and I looked at this and deleted it from the hmm side: commit 068354ade5dd9e2b07d9b0c57055a681db6f4e37 Author: Jason Gunthorpe Date: Fri Mar 27 17:00:13 2020 -0300 mm/hmm: remove pgmap checking for devmap pages The checking boils down to some racy check if the pagemap is still available or not. Instead of checking this, rely entirely on the notifiers, if a pagemap is destroyed then all pages that belong to it must be removed from the tables and the notifiers triggered. Link: https://lore.kernel.org/r/20200327200021.29372-2-jgg@ziepe.ca Though I am wondering if this whole hmm thing is racy with memory unplug. Hmm. Jason