From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4EECC433FE for ; Mon, 18 Oct 2021 18:26:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F40F86108E for ; Mon, 18 Oct 2021 18:26:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org F40F86108E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 72C2F900003; Mon, 18 Oct 2021 14:26:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DCD0900002; Mon, 18 Oct 2021 14:26:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57CC0900003; Mon, 18 Oct 2021 14:26:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0179.hostedemail.com [216.40.44.179]) by kanga.kvack.org (Postfix) with ESMTP id 49819900002 for ; Mon, 18 Oct 2021 14:26:02 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id EA9FF1802DDF1 for ; Mon, 18 Oct 2021 18:26:01 +0000 (UTC) X-FDA: 78710387322.18.3A2DB22 Received: from mail-qk1-f177.google.com (mail-qk1-f177.google.com [209.85.222.177]) by imf06.hostedemail.com (Postfix) with ESMTP id C058F801A8A0 for ; Mon, 18 Oct 2021 18:26:00 +0000 (UTC) Received: by mail-qk1-f177.google.com with SMTP id bl14so16193453qkb.4 for ; Mon, 18 Oct 2021 11:26:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=VWav6NFcp7Imar+k++zv5L6bzbFiMGeoi4IbLNr1qXk=; b=iulG1Hi/BWXnGHXIUJtE3Sch4DxTtHHGjYAbR7SgfDwl3Tc6og/olLkcCwEBbB2k6G IfgEoidJStEKIbj2It5DifRzrECL7P9NI1viyq+lBosbBqcDaY1Cx5CKvpaCMMtwcLaK LJk+g8GcJQ9MYUf7YDlnJqNpxqQtZJQQNHXot+kUflb9VL1jiqaRs9roMA774Pq07G8P z6MQINCUTuDZeMIbiJyaMaqTsUS5Xe2aLaKTMnNU2nVECI+IUQvATxqyqgKryVcE6Odi UPrSa/zR8XpvPyVh6RkKVjlgDcQlRQzheAMj9zSJ3nZtedo1kmeWigT4fsV71OfsmWN0 6Uxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=VWav6NFcp7Imar+k++zv5L6bzbFiMGeoi4IbLNr1qXk=; b=mnWfggufCxqx2gkOjU3+Hy/k5ddcy3MIz5yWx/wISXqKdTfmcCmVVZXN+P3Vs/02rP soiL3vw00ksa2dHUJvOumOjV9eR1MfTYGzkAdSWeDpWHRljdk7oxGKtzfiIzJ/ZvDw/y 9/bixWLFcv6CSepcCBJoxnWlOPHrb3k0rzkfnoEeIAHXL/KqCGZuLbFVAH/GMwLoGj8n NiQWuERWwONlyEn4J0KkDnQkzosynHah68nF9/947SB6sp9rkU2/zMdYyMoCR/P3LbTX u2BShiWgkvWcEmyz3KR3oiYL7us5kYuKDgV6ptaj/5wFhw2u9aMSCx0k631RGTpWGf/f VPXQ== X-Gm-Message-State: AOAM530fgNl6iUoCbLN/uc99A1eqcqpGRCvGwQZPedbAkfY91/WTx5MN 52wuHBLQ/EwsW++VazeKftM8zA== X-Google-Smtp-Source: ABdhPJx0w5FpmLsv0LfpzJkRgNCEFvSTyhRWvN2OZSonMTzVl3H3jB0V5GZKEdNIQJ2jOwLftPxV7g== X-Received: by 2002:a05:620a:2947:: with SMTP id n7mr24141155qkp.60.1634581560548; Mon, 18 Oct 2021 11:26:00 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-162-113-129.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.113.129]) by smtp.gmail.com with ESMTPSA id m195sm6853505qke.73.2021.10.18.11.25.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Oct 2021 11:25:59 -0700 (PDT) Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1mcXKh-00GLjg-2o; Mon, 18 Oct 2021 15:25:59 -0300 Date: Mon, 18 Oct 2021 15:25:59 -0300 From: Jason Gunthorpe To: Dan Williams Cc: Matthew Wilcox , Alex Sierra , Andrew Morton , "Kuehling, Felix" , Linux MM , Ralph Campbell , linux-ext4 , linux-xfs , amd-gfx list , Maling list - DRI developers , Christoph Hellwig , =?utf-8?B?SsOpcsO0bWU=?= Glisse , Alistair Popple , Vishal Verma , Dave Jiang , Linux NVDIMM , David Hildenbrand , Joao Martins Subject: Re: [PATCH v1 2/2] mm: remove extra ZONE_DEVICE struct page refcount Message-ID: <20211018182559.GC3686969@ziepe.ca> References: <20211014153928.16805-1-alex.sierra@amd.com> <20211014153928.16805-3-alex.sierra@amd.com> <20211014170634.GV2744544@nvidia.com> <20211014230606.GZ2744544@nvidia.com> <20211016154450.GJ2744544@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b="iulG1Hi/"; dmarc=none; spf=pass (imf06.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.222.177 as permitted sender) smtp.mailfrom=jgg@ziepe.ca X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: C058F801A8A0 X-Stat-Signature: euwdh5omm3wao4y8rt467njyfk5cqh8w X-HE-Tag: 1634581560-974770 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Oct 17, 2021 at 11:35:35AM -0700, Dan Williams wrote: > > DAX is stuffing arrays of 4k pages into the PUD/PMDs. Aligning with > > THP would make using normal refconting much simpler. I looked at > > teaching the mm core to deal with page arrays - it is certainly > > doable, but it is quite inefficient and ugly mm code. > > THP does not support PUD, and neither does FSDAX, so it's only PMDs we > need to worry about. device-dax uses PUD, along with TTM, they are the only places. I'm not sure TTM is a real place though. > > So, can we fix DAX and TTM - the only uses of PUD/PMDs I could find? > > > > Joao has a series that does this to device-dax: > > > > https://lore.kernel.org/all/20210827145819.16471-1-joao.m.martins@oracle.com/ > > That assumes there's never any need to fracture a huge page which > FSDAX could not support unless the filesystem was built with 2MB block > size. As I understand things, something like FSDAX post-folio should generate maximal compound pages for extents in the page cache that are physically contiguous. A high order folio can be placed in any lower order in the page tables, so we never have to fracture it, unless the underlying page are moved around - which requires an unmap_mapping_range() cycle.. > > Assuming changing FSDAX is hard.. How would DAX people feel about just > > deleting the PUD/PMD support until it can be done with compound pages? > > There are end users that would notice the PMD regression, and I think > FSDAX PMDs with proper compound page metadata is on the same order of > work as fixing the refcount. Hmm, I don't know.. I sketched out the refcount stuff and the code is OK but ugly and will add a conditional to some THP cases On the other hand, making THP unmap cases a bit slower is probably a net win compared to making put_page a bit slower.. Considering unmap is already quite heavy. > > 4) Ask what the pgmap owner wants to do: > > > > if (head->pgmap->deny_foll_longterm) > > return FAIL > > The pgmap itself does not know, but the "holder" could specify this > policy. Here I imagine the thing that creates the pgmap would specify the policy it wants. In most cases the policy is tightly coupled to what the free function in the the provided dev_pagemap_ops does.. > Which is in line with the 'dax_holder_ops' concept being introduced > for reverse mapping support. I.e. when the FS claims the dax-device > it can specify at that point that it wants to forbid longterm. Which is a reasonable refinment if we think there are cases where two nvdim users would want different things. Anyhow, I'm wondering on a way forward. There are many balls in the air, all linked: - Joao's compound page support for device_dax and more - Alex's DEVICE_COHERENT - The refcount normalization - Removing the pgmap test from GUP - Removing the need for the PUD/PMD/PTE special bit - Removing the need for the PUD/PMD/PTE devmap bit - Remove PUD/PMD vma_is_special - folios for fsdax - shootdown for fsdax Frankly I'm leery to see more ZONE_DEVICE users crop up that depend on the current semantics as that will only make it even harder to fix.. I think it would be good to see Joao's compound page support move ahead.. So.. Does anyone want to work on finishing this patch series?? I can give some guidance on how I think it should work at least Jason