From: Dan Williams <dan.j.williams@intel.com> To: akpm@linux-foundation.org Cc: "David Hildenbrand" <david@redhat.com>, "Bjorn Helgaas" <bhelgaas@google.com>, "Dave Hansen" <dave.hansen@linux.intel.com>, "David Airlie" <airlied@linux.ie>, "Joao Martins" <joao.m.martins@oracle.com>, "Jonathan Cameron" <Jonathan.Cameron@huawei.com>, "Greg Kroah-Hartman" <gregkh@linuxfoundation.org>, "Pavel Tatashin" <pasha.tatashin@soleen.com>, "Hulk Robot" <hulkci@huawei.com>, "Ben Skeggs" <bskeggs@redhat.com>, "Benjamin Herrenschmidt" <benh@kernel.crashing.org>, "Jia He" <justin.he@arm.com>, "Jérôme Glisse" <jglisse@redhat.com>, "Jason Yan" <yanaijie@huawei.com>, "Paul Mackerras" <paulus@ozlabs.org>, "Boris Ostrovsky" <boris.ostrovsky@oracle.com>, "Brice Goglin" <Brice.Goglin@inria.fr>, "Stefano Stabellini" <sstabellini@kernel.org>, "Michael Ellerman" <mpe@ellerman.id.au>, "Juergen Gross" <jgross@suse.com>, "Daniel Vetter" <daniel@ffwll.ch>, linux-mm@kvack.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 00/17] device-dax: support sub-dividing soft-reserved ranges Date: Fri, 25 Sep 2020 12:11:39 -0700 [thread overview] Message-ID: <160106109960.30709.7379926726669669398.stgit@dwillia2-desk3.amr.corp.intel.com> (raw) Changes since v4 [1]: - Rebased on device-dax-move-instance-creation-parameters-to-struct-dev_dax_data.patch in -mm [2]. I.e. patches that did not need fixups from v4 are not included. - Folded all fixes - Replaced "device-dax: kill dax_kmem_res" with: device-dax/kmem: introduce dax_kmem_range() device-dax/kmem: move resource name tracking to drvdata device-dax/kmem: replace release_resource() with release_mem_region() ...to address David's request to make those cleanups easier to review. Note that I dropped changes to how IORESOURCE_BUSY is manipulated since David and I are still debating the best way forward there. - Broke out some of dax-bus reworks in "device-dax: introduce 'seed' devices" to a new "device-dax: introduce 'struct dev_dax' typed-driver operations" - Added a conversion of xen_alloc_unallocated_pages() from pgmap.res to pgmap.range. I found it odd that there is no corresponding memunmap_pages() triggered by xen_free_unallocated_pages()? - Not included, a conversion of virtio_fs to use pgmap.range for its new usage of devm_memremap_pages(). It appears the virtio_fs changes are merged after -mm? My mental model of -mm was that it applies on top of linux-next? In any event, Vivek, you will need to coordinate a conversion to pgmap.range for the virtio_fs dax-support merge. Maybe that should go through Andrew as well? - Lowercase all the subject lines per akpm's preference - Received a 0day robot build-success notification over 122 configs - Thanks to Joao for looking after this set while I was out. [1]: http://lore.kernel.org/r/159625229779.3040297.11363509688097221416.stgit@dwillia2-desk3.amr.corp.intel.com [2]: https://ozlabs.org/~akpm/mmots/broken-out/device-dax-move-instance-creation-parameters-to-struct-dev_dax_data.patch --- Andrew, this series replaces device-dax-make-pgmap-optional-for-instance-creation.patch ...through... dax-hmem-introduce-dax_hmemregion_idle-parameter.patch ...in your stack. Let me know if there is a different / preferred way to refresh a bulk of patches in your queue when only a subset need updates. --- The device-dax facility allows an address range to be directly mapped through a chardev, or optionally hotplugged to the core kernel page allocator as System-RAM. It is the mechanism for converting persistent memory (pmem) to be used as another volatile memory pool i.e. the current Memory Tiering hot topic on linux-mm. In the case of pmem the nvdimm-namespace-label mechanism can sub-divide it, but that labeling mechanism is not available / applicable to soft-reserved ("EFI specific purpose") memory [3]. This series provides a sysfs-mechanism for the daxctl utility to enable provisioning of volatile-soft-reserved memory ranges. The motivations for this facility are: 1/ Allow performance differentiated memory ranges to be split between kernel-managed and directly-accessed use cases. 2/ Allow physical memory to be provisioned along performance relevant address boundaries. For example, divide a memory-side cache [4] along cache-color boundaries. 3/ Parcel out soft-reserved memory to VMs using device-dax as a security / permissions boundary [5]. Specifically I have seen people (ab)using memmap=nn!ss (mark System-RAM as Persistent Memory) just to get the device-dax interface on custom address ranges. A follow-on for the VM use case is to teach device-dax to dynamically allocate 'struct page' at runtime to reduce the duplication of 'struct page' space in both the guest and the host kernel for the same physical pages. [3]: http://lore.kernel.org/r/157309097008.1579826.12818463304589384434.stgit@dwillia2-desk3.amr.corp.intel.com [4]: http://lore.kernel.org/r/154899811738.3165233.12325692939590944259.stgit@dwillia2-desk3.amr.corp.intel.com [5]: http://lore.kernel.org/r/20200110190313.17144-1-joao.m.martins@oracle.com --- Dan Williams (14): device-dax: make pgmap optional for instance creation device-dax/kmem: introduce dax_kmem_range() device-dax/kmem: move resource name tracking to drvdata device-dax/kmem: replace release_resource() with release_mem_region() device-dax: add an allocation interface for device-dax instances device-dax: introduce 'struct dev_dax' typed-driver operations device-dax: introduce 'seed' devices drivers/base: make device_find_child_by_name() compatible with sysfs inputs device-dax: add resize support mm/memremap_pages: convert to 'struct range' mm/memremap_pages: support multiple ranges per invocation device-dax: add dis-contiguous resource support device-dax: introduce 'mapping' devices device-dax: add an 'align' attribute Joao Martins (3): device-dax: make align a per-device property dax/hmem: introduce dax_hmem.region_idle parameter device-dax: add a range mapping allocation attribute arch/powerpc/kvm/book3s_hv_uvmem.c | 14 drivers/base/core.c | 2 drivers/dax/bus.c | 1039 ++++++++++++++++++++++++++++++-- drivers/dax/bus.h | 11 drivers/dax/dax-private.h | 58 ++ drivers/dax/device.c | 112 ++- drivers/dax/hmem/hmem.c | 17 - drivers/dax/kmem.c | 178 +++-- drivers/dax/pmem/compat.c | 2 drivers/dax/pmem/core.c | 14 drivers/gpu/drm/nouveau/nouveau_dmem.c | 15 drivers/nvdimm/badrange.c | 26 - drivers/nvdimm/claim.c | 13 drivers/nvdimm/nd.h | 3 drivers/nvdimm/pfn_devs.c | 13 drivers/nvdimm/pmem.c | 27 - drivers/nvdimm/region.c | 21 - drivers/pci/p2pdma.c | 12 drivers/xen/unpopulated-alloc.c | 45 + include/linux/memremap.h | 11 include/linux/range.h | 6 lib/test_hmm.c | 15 mm/memremap.c | 299 +++++---- tools/testing/nvdimm/dax-dev.c | 22 - tools/testing/nvdimm/test/iomap.c | 2 25 files changed, 1557 insertions(+), 420 deletions(-) base-commit: 6764736525f27a411ba2c0c430aaa2df7375f3ac _______________________________________________ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
WARNING: multiple messages have this Message-ID (diff)
From: Dan Williams <dan.j.williams@intel.com> To: akpm@linux-foundation.org Cc: "David Hildenbrand" <david@redhat.com>, "Ira Weiny" <ira.weiny@intel.com>, "Bjorn Helgaas" <bhelgaas@google.com>, "Vishal Verma" <vishal.l.verma@intel.com>, "Dave Hansen" <dave.hansen@linux.intel.com>, "David Airlie" <airlied@linux.ie>, "Vivek Goyal" <vgoyal@redhat.com>, "Joao Martins" <joao.m.martins@oracle.com>, "Dave Jiang" <dave.jiang@intel.com>, "Jonathan Cameron" <Jonathan.Cameron@huawei.com>, "Greg Kroah-Hartman" <gregkh@linuxfoundation.org>, "Pavel Tatashin" <pasha.tatashin@soleen.com>, "Hulk Robot" <hulkci@huawei.com>, "Ben Skeggs" <bskeggs@redhat.com>, "Benjamin Herrenschmidt" <benh@kernel.crashing.org>, "Jia He" <justin.he@arm.com>, "Jérôme Glisse" <jglisse@redhat.com>, "Jason Yan" <yanaijie@huawei.com>, "Paul Mackerras" <paulus@ozlabs.org>, "Boris Ostrovsky" <boris.ostrovsky@oracle.com>, "Brice Goglin" <Brice.Goglin@inria.fr>, "Stefano Stabellini" <sstabellini@kernel.org>, "Michael Ellerman" <mpe@ellerman.id.au>, "Juergen Gross" <jgross@suse.com>, "Daniel Vetter" <daniel@ffwll.ch>, linux-mm@kvack.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 00/17] device-dax: support sub-dividing soft-reserved ranges Date: Fri, 25 Sep 2020 12:11:39 -0700 [thread overview] Message-ID: <160106109960.30709.7379926726669669398.stgit@dwillia2-desk3.amr.corp.intel.com> (raw) Changes since v4 [1]: - Rebased on device-dax-move-instance-creation-parameters-to-struct-dev_dax_data.patch in -mm [2]. I.e. patches that did not need fixups from v4 are not included. - Folded all fixes - Replaced "device-dax: kill dax_kmem_res" with: device-dax/kmem: introduce dax_kmem_range() device-dax/kmem: move resource name tracking to drvdata device-dax/kmem: replace release_resource() with release_mem_region() ...to address David's request to make those cleanups easier to review. Note that I dropped changes to how IORESOURCE_BUSY is manipulated since David and I are still debating the best way forward there. - Broke out some of dax-bus reworks in "device-dax: introduce 'seed' devices" to a new "device-dax: introduce 'struct dev_dax' typed-driver operations" - Added a conversion of xen_alloc_unallocated_pages() from pgmap.res to pgmap.range. I found it odd that there is no corresponding memunmap_pages() triggered by xen_free_unallocated_pages()? - Not included, a conversion of virtio_fs to use pgmap.range for its new usage of devm_memremap_pages(). It appears the virtio_fs changes are merged after -mm? My mental model of -mm was that it applies on top of linux-next? In any event, Vivek, you will need to coordinate a conversion to pgmap.range for the virtio_fs dax-support merge. Maybe that should go through Andrew as well? - Lowercase all the subject lines per akpm's preference - Received a 0day robot build-success notification over 122 configs - Thanks to Joao for looking after this set while I was out. [1]: http://lore.kernel.org/r/159625229779.3040297.11363509688097221416.stgit@dwillia2-desk3.amr.corp.intel.com [2]: https://ozlabs.org/~akpm/mmots/broken-out/device-dax-move-instance-creation-parameters-to-struct-dev_dax_data.patch --- Andrew, this series replaces device-dax-make-pgmap-optional-for-instance-creation.patch ...through... dax-hmem-introduce-dax_hmemregion_idle-parameter.patch ...in your stack. Let me know if there is a different / preferred way to refresh a bulk of patches in your queue when only a subset need updates. --- The device-dax facility allows an address range to be directly mapped through a chardev, or optionally hotplugged to the core kernel page allocator as System-RAM. It is the mechanism for converting persistent memory (pmem) to be used as another volatile memory pool i.e. the current Memory Tiering hot topic on linux-mm. In the case of pmem the nvdimm-namespace-label mechanism can sub-divide it, but that labeling mechanism is not available / applicable to soft-reserved ("EFI specific purpose") memory [3]. This series provides a sysfs-mechanism for the daxctl utility to enable provisioning of volatile-soft-reserved memory ranges. The motivations for this facility are: 1/ Allow performance differentiated memory ranges to be split between kernel-managed and directly-accessed use cases. 2/ Allow physical memory to be provisioned along performance relevant address boundaries. For example, divide a memory-side cache [4] along cache-color boundaries. 3/ Parcel out soft-reserved memory to VMs using device-dax as a security / permissions boundary [5]. Specifically I have seen people (ab)using memmap=nn!ss (mark System-RAM as Persistent Memory) just to get the device-dax interface on custom address ranges. A follow-on for the VM use case is to teach device-dax to dynamically allocate 'struct page' at runtime to reduce the duplication of 'struct page' space in both the guest and the host kernel for the same physical pages. [3]: http://lore.kernel.org/r/157309097008.1579826.12818463304589384434.stgit@dwillia2-desk3.amr.corp.intel.com [4]: http://lore.kernel.org/r/154899811738.3165233.12325692939590944259.stgit@dwillia2-desk3.amr.corp.intel.com [5]: http://lore.kernel.org/r/20200110190313.17144-1-joao.m.martins@oracle.com --- Dan Williams (14): device-dax: make pgmap optional for instance creation device-dax/kmem: introduce dax_kmem_range() device-dax/kmem: move resource name tracking to drvdata device-dax/kmem: replace release_resource() with release_mem_region() device-dax: add an allocation interface for device-dax instances device-dax: introduce 'struct dev_dax' typed-driver operations device-dax: introduce 'seed' devices drivers/base: make device_find_child_by_name() compatible with sysfs inputs device-dax: add resize support mm/memremap_pages: convert to 'struct range' mm/memremap_pages: support multiple ranges per invocation device-dax: add dis-contiguous resource support device-dax: introduce 'mapping' devices device-dax: add an 'align' attribute Joao Martins (3): device-dax: make align a per-device property dax/hmem: introduce dax_hmem.region_idle parameter device-dax: add a range mapping allocation attribute arch/powerpc/kvm/book3s_hv_uvmem.c | 14 drivers/base/core.c | 2 drivers/dax/bus.c | 1039 ++++++++++++++++++++++++++++++-- drivers/dax/bus.h | 11 drivers/dax/dax-private.h | 58 ++ drivers/dax/device.c | 112 ++- drivers/dax/hmem/hmem.c | 17 - drivers/dax/kmem.c | 178 +++-- drivers/dax/pmem/compat.c | 2 drivers/dax/pmem/core.c | 14 drivers/gpu/drm/nouveau/nouveau_dmem.c | 15 drivers/nvdimm/badrange.c | 26 - drivers/nvdimm/claim.c | 13 drivers/nvdimm/nd.h | 3 drivers/nvdimm/pfn_devs.c | 13 drivers/nvdimm/pmem.c | 27 - drivers/nvdimm/region.c | 21 - drivers/pci/p2pdma.c | 12 drivers/xen/unpopulated-alloc.c | 45 + include/linux/memremap.h | 11 include/linux/range.h | 6 lib/test_hmm.c | 15 mm/memremap.c | 299 +++++---- tools/testing/nvdimm/dax-dev.c | 22 - tools/testing/nvdimm/test/iomap.c | 2 25 files changed, 1557 insertions(+), 420 deletions(-) base-commit: 6764736525f27a411ba2c0c430aaa2df7375f3ac
next reply other threads:[~2020-09-25 19:30 UTC|newest] Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-09-25 19:11 Dan Williams [this message] 2020-09-25 19:11 ` [PATCH v5 00/17] device-dax: support sub-dividing soft-reserved ranges Dan Williams 2020-09-25 19:11 ` [PATCH v5 01/17] device-dax: make pgmap optional for instance creation Dan Williams 2020-09-25 19:11 ` Dan Williams 2020-10-01 8:41 ` David Hildenbrand 2020-10-01 8:41 ` David Hildenbrand 2020-10-01 16:54 ` Dan Williams 2020-10-01 16:54 ` Dan Williams 2020-10-01 16:54 ` Dan Williams 2020-10-01 17:39 ` David Hildenbrand 2020-10-01 17:39 ` David Hildenbrand 2020-10-01 19:12 ` Dan Williams 2020-10-01 19:12 ` Dan Williams 2020-10-01 19:12 ` Dan Williams 2020-09-25 19:11 ` [PATCH v5 02/17] device-dax/kmem: introduce dax_kmem_range() Dan Williams 2020-09-25 19:11 ` Dan Williams 2020-09-30 16:14 ` David Hildenbrand 2020-09-30 16:14 ` David Hildenbrand 2020-09-25 19:11 ` [PATCH v5 03/17] device-dax/kmem: move resource name tracking to drvdata Dan Williams 2020-09-25 19:11 ` Dan Williams 2020-09-30 16:19 ` David Hildenbrand 2020-09-30 16:19 ` David Hildenbrand 2020-09-25 19:12 ` [PATCH v5 04/17] device-dax/kmem: replace release_resource() with release_mem_region() Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-30 16:23 ` David Hildenbrand 2020-09-30 16:23 ` David Hildenbrand 2020-09-30 17:28 ` Dan Williams 2020-09-30 17:28 ` Dan Williams 2020-09-30 17:28 ` Dan Williams 2020-09-25 19:12 ` [PATCH v5 05/17] device-dax: add an allocation interface for device-dax instances Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-25 19:12 ` [PATCH v5 06/17] device-dax: introduce 'struct dev_dax' typed-driver operations Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-25 19:12 ` [PATCH v5 07/17] device-dax: introduce 'seed' devices Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-25 19:12 ` [PATCH v5 08/17] drivers/base: make device_find_child_by_name() compatible with sysfs inputs Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-25 19:12 ` [PATCH v5 09/17] device-dax: add resize support Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-25 19:12 ` [PATCH v5 10/17] mm/memremap_pages: convert to 'struct range' Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-28 19:12 ` boris.ostrovsky 2020-09-28 19:12 ` boris.ostrovsky 2020-09-25 19:12 ` [PATCH v5 11/17] mm/memremap_pages: support multiple ranges per invocation Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-25 19:12 ` [PATCH v5 12/17] device-dax: add dis-contiguous resource support Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-25 19:12 ` [PATCH v5 13/17] device-dax: introduce 'mapping' devices Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-25 19:12 ` [PATCH v5 14/17] device-dax: make align a per-device property Dan Williams 2020-09-25 19:12 ` Dan Williams 2020-09-25 19:13 ` [PATCH v5 15/17] device-dax: add an 'align' attribute Dan Williams 2020-09-25 19:13 ` Dan Williams 2020-09-26 2:22 ` Andrew Morton 2020-09-26 2:22 ` Andrew Morton 2020-09-26 3:31 ` Dan Williams 2020-09-26 3:31 ` Dan Williams 2020-09-26 3:31 ` Dan Williams 2020-09-25 19:13 ` [PATCH v5 16/17] dax/hmem: introduce dax_hmem.region_idle parameter Dan Williams 2020-09-25 19:13 ` Dan Williams 2020-09-25 19:13 ` [PATCH v5 17/17] device-dax: add a range mapping allocation attribute Dan Williams 2020-09-25 19:13 ` Dan Williams 2020-09-25 20:51 ` [PATCH v5 00/17] device-dax: support sub-dividing soft-reserved ranges Joao Martins 2020-09-25 20:51 ` Joao Martins 2020-09-25 21:01 ` Dan Williams 2020-09-25 21:01 ` Dan Williams 2020-09-25 21:05 ` Joao Martins 2020-09-25 21:05 ` Joao Martins
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=160106109960.30709.7379926726669669398.stgit@dwillia2-desk3.amr.corp.intel.com \ --to=dan.j.williams@intel.com \ --cc=Brice.Goglin@inria.fr \ --cc=Jonathan.Cameron@huawei.com \ --cc=airlied@linux.ie \ --cc=akpm@linux-foundation.org \ --cc=benh@kernel.crashing.org \ --cc=bhelgaas@google.com \ --cc=boris.ostrovsky@oracle.com \ --cc=bskeggs@redhat.com \ --cc=daniel@ffwll.ch \ --cc=dave.hansen@linux.intel.com \ --cc=david@redhat.com \ --cc=gregkh@linuxfoundation.org \ --cc=hulkci@huawei.com \ --cc=jglisse@redhat.com \ --cc=jgross@suse.com \ --cc=joao.m.martins@oracle.com \ --cc=justin.he@arm.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-nvdimm@lists.01.org \ --cc=mpe@ellerman.id.au \ --cc=pasha.tatashin@soleen.com \ --cc=paulus@ozlabs.org \ --cc=sstabellini@kernel.org \ --cc=yanaijie@huawei.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.