* [PATCH 0/5] libnvdimm: Cross-arch compatible namespace alignment
@ 2020-01-30 20:05 Dan Williams
2020-01-30 20:06 ` [PATCH 1/5] mm/memremap_pages: Kill unused __devm_memremap_pages() Dan Williams
` (4 more replies)
0 siblings, 5 replies; 15+ messages in thread
From: Dan Williams @ 2020-01-30 20:05 UTC (permalink / raw)
To: linux-nvdimm
Cc: Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras,
Christoph Hellwig, Aneesh Kumar K.V, linux-kernel, linuxppc-dev
Aneesh reports that PowerPC requires 16MiB alignment for the address
range passed to devm_memremap_pages(), and Jeff reports that it is
possible to create a misaligned namespace which blocks future namespace
creation in that region. Both of these issues require namespace
alignment to be managed at the region level rather than padding at the
namespace level which has been a broken approach to date.
Introduce memremap_compat_align() to indicate the hard requirements of
an arch's memremap_pages() implementation. Use the maximum known
memremap_compat_align() to set the default namespace alignment for
libnvdimm. Consult that alignment when allocating free space. Finally,
allow the default region alignment to be overridden to maintain the same
namespace creation capability as previous kernels.
The ndctl unit tests, which have some misaligned namespace assumptions,
are updated to use the alignment override where necessary.
Thanks to Aneesh for early feedback and testing on this improved
alignment handling.
---
Dan Williams (5):
mm/memremap_pages: Kill unused __devm_memremap_pages()
mm/memremap_pages: Introduce memremap_compat_align()
libnvdimm/namespace: Enforce memremap_compat_align()
libnvdimm/region: Introduce NDD_LABELING
libnvdimm/region: Introduce an 'align' attribute
arch/powerpc/include/asm/io.h | 10 ++
arch/powerpc/platforms/pseries/papr_scm.c | 2
drivers/acpi/nfit/core.c | 4 +
drivers/nvdimm/dimm.c | 2
drivers/nvdimm/dimm_devs.c | 95 +++++++++++++++++----
drivers/nvdimm/namespace_devs.c | 21 ++++-
drivers/nvdimm/nd.h | 3 -
drivers/nvdimm/pfn_devs.c | 2
drivers/nvdimm/region_devs.c | 132 ++++++++++++++++++++++++++---
include/linux/io.h | 23 +++++
include/linux/libnvdimm.h | 2
include/linux/mmzone.h | 1
12 files changed, 255 insertions(+), 42 deletions(-)
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 1/5] mm/memremap_pages: Kill unused __devm_memremap_pages()
2020-01-30 20:05 [PATCH 0/5] libnvdimm: Cross-arch compatible namespace alignment Dan Williams
@ 2020-01-30 20:06 ` Dan Williams
2020-01-31 5:32 ` Christoph Hellwig
2020-02-03 17:08 ` Aneesh Kumar K.V
2020-01-30 20:06 ` [PATCH 2/5] mm/memremap_pages: Introduce memremap_compat_align() Dan Williams
` (3 subsequent siblings)
4 siblings, 2 replies; 15+ messages in thread
From: Dan Williams @ 2020-01-30 20:06 UTC (permalink / raw)
To: linux-nvdimm; +Cc: Christoph Hellwig, linux-kernel, linuxppc-dev
Kill this definition that was introduced in commit 41e94a851304 ("add
devm_memremap_pages") add never used.
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
include/linux/io.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/include/linux/io.h b/include/linux/io.h
index a59834bc0a11..35e8d84935e0 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -79,8 +79,6 @@ void *devm_memremap(struct device *dev, resource_size_t offset,
size_t size, unsigned long flags);
void devm_memunmap(struct device *dev, void *addr);
-void *__devm_memremap_pages(struct device *dev, struct resource *res);
-
#ifdef CONFIG_PCI
/*
* The PCI specifications (Rev 3.0, 3.2.5 "Transaction Ordering and
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 2/5] mm/memremap_pages: Introduce memremap_compat_align()
2020-01-30 20:05 [PATCH 0/5] libnvdimm: Cross-arch compatible namespace alignment Dan Williams
2020-01-30 20:06 ` [PATCH 1/5] mm/memremap_pages: Kill unused __devm_memremap_pages() Dan Williams
@ 2020-01-30 20:06 ` Dan Williams
2020-02-03 17:09 ` Aneesh Kumar K.V
2020-02-05 3:05 ` Michael Ellerman
2020-01-30 20:06 ` [PATCH 3/5] libnvdimm/namespace: Enforce memremap_compat_align() Dan Williams
` (2 subsequent siblings)
4 siblings, 2 replies; 15+ messages in thread
From: Dan Williams @ 2020-01-30 20:06 UTC (permalink / raw)
To: linux-nvdimm
Cc: Aneesh Kumar K.V, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, hch, linux-kernel, linuxppc-dev
The "sub-section memory hotplug" facility allows memremap_pages() users
like libnvdimm to compensate for hardware platforms like x86 that have a
section size larger than their hardware memory mapping granularity. The
compensation that sub-section support affords is being tolerant of
physical memory resources shifting by units smaller (64MiB on x86) than
the memory-hotplug section size (128 MiB). Where the platform
physical-memory mapping granularity is limited by the number and
capability of address-decode-registers in the memory controller.
While the sub-section support allows memremap_pages() to operate on
sub-section (2MiB) granularity, the Power architecture may still
require 16MiB alignment on "!radix_enabled()" platforms.
In order for libnvdimm to be able to detect and manage this per-arch
limitation, introduce memremap_compat_align() as a common minimum
alignment across all driver-facing memory-mapping interfaces, and let
Power override it to 16MiB in the "!radix_enabled()" case.
The assumption / requirement for 16MiB to be a viable
memremap_compat_align() value is that Power does not have platforms
where its equivalent of address-decode-registers never hardware remaps a
persistent memory resource on smaller than 16MiB boundaries.
Based on an initial patch by Aneesh.
Link: http://lore.kernel.org/r/CAPcyv4gBGNP95APYaBcsocEa50tQj9b5h__83vgngjq3ouGX_Q@mail.gmail.com
Reported-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Reported-by: Jeff Moyer <jmoyer@redhat.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/powerpc/include/asm/io.h | 10 ++++++++++
drivers/nvdimm/pfn_devs.c | 2 +-
include/linux/io.h | 23 +++++++++++++++++++++++
include/linux/mmzone.h | 1 +
4 files changed, 35 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/io.h b/arch/powerpc/include/asm/io.h
index a63ec938636d..0fa2dc483008 100644
--- a/arch/powerpc/include/asm/io.h
+++ b/arch/powerpc/include/asm/io.h
@@ -734,6 +734,16 @@ extern void __iomem * __ioremap_at(phys_addr_t pa, void *ea,
unsigned long size, pgprot_t prot);
extern void __iounmap_at(void *ea, unsigned long size);
+#ifdef CONFIG_SPARSEMEM
+static inline unsigned long memremap_compat_align(void)
+{
+ if (radix_enabled())
+ return SUBSECTION_SIZE;
+ return (1UL << mmu_psize_defs[mmu_linear_psize].shift);
+}
+#define memremap_compat_align memremap_compat_align
+#endif
+
/*
* When CONFIG_PPC_INDIRECT_PIO is set, we use the generic iomap implementation
* which needs some additional definitions here. They basically allow PIO
diff --git a/drivers/nvdimm/pfn_devs.c b/drivers/nvdimm/pfn_devs.c
index b94f7a7e94b8..a5c25cb87116 100644
--- a/drivers/nvdimm/pfn_devs.c
+++ b/drivers/nvdimm/pfn_devs.c
@@ -750,7 +750,7 @@ static int nd_pfn_init(struct nd_pfn *nd_pfn)
start = nsio->res.start;
size = resource_size(&nsio->res);
npfns = PHYS_PFN(size - SZ_8K);
- align = max(nd_pfn->align, (1UL << SUBSECTION_SHIFT));
+ align = max(nd_pfn->align, SUBSECTION_SIZE);
end_trunc = start + size - ALIGN_DOWN(start + size, align);
if (nd_pfn->mode == PFN_MODE_PMEM) {
/*
diff --git a/include/linux/io.h b/include/linux/io.h
index 35e8d84935e0..ccd34519fad3 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -6,6 +6,7 @@
#ifndef _LINUX_IO_H
#define _LINUX_IO_H
+#include <linux/mmzone.h>
#include <linux/types.h>
#include <linux/init.h>
#include <linux/bug.h>
@@ -79,6 +80,28 @@ void *devm_memremap(struct device *dev, resource_size_t offset,
size_t size, unsigned long flags);
void devm_memunmap(struct device *dev, void *addr);
+#ifndef memremap_compat_align
+#ifdef CONFIG_SPARSEMEM
+/*
+ * Minimum compatible alignment of the resource (start, end) across
+ * memremap interfaces (i.e. memremap + memremap_pages)
+ */
+static inline unsigned long memremap_compat_align(void)
+{
+ return SUBSECTION_SIZE;
+}
+#else /* CONFIG_SPARSEMEM */
+/*
+ * No ZONE_DEVICE / memremap_pages() support so the minimum mapping
+ * granularity is a single page.
+ */
+static inline unsigned long memremap_compat_align(void)
+{
+ return PAGE_SIZE;
+}
+#endif /* CONFIG_SPARSEMEM */
+#endif /* memremap_compat_align */
+
#ifdef CONFIG_PCI
/*
* The PCI specifications (Rev 3.0, 3.2.5 "Transaction Ordering and
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 89d8ff06c9ce..b0de83620cd7 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -1171,6 +1171,7 @@ static inline unsigned long section_nr_to_pfn(unsigned long sec)
#define SECTION_ALIGN_DOWN(pfn) ((pfn) & PAGE_SECTION_MASK)
#define SUBSECTION_SHIFT 21
+#define SUBSECTION_SIZE (1UL << SUBSECTION_SHIFT)
#define PFN_SUBSECTION_SHIFT (SUBSECTION_SHIFT - PAGE_SHIFT)
#define PAGES_PER_SUBSECTION (1UL << PFN_SUBSECTION_SHIFT)
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 3/5] libnvdimm/namespace: Enforce memremap_compat_align()
2020-01-30 20:05 [PATCH 0/5] libnvdimm: Cross-arch compatible namespace alignment Dan Williams
2020-01-30 20:06 ` [PATCH 1/5] mm/memremap_pages: Kill unused __devm_memremap_pages() Dan Williams
2020-01-30 20:06 ` [PATCH 2/5] mm/memremap_pages: Introduce memremap_compat_align() Dan Williams
@ 2020-01-30 20:06 ` Dan Williams
2020-02-03 17:09 ` Aneesh Kumar K.V
2020-01-30 20:06 ` [PATCH 4/5] libnvdimm/region: Introduce NDD_LABELING Dan Williams
2020-01-30 20:06 ` [PATCH 5/5] libnvdimm/region: Introduce an 'align' attribute Dan Williams
4 siblings, 1 reply; 15+ messages in thread
From: Dan Williams @ 2020-01-30 20:06 UTC (permalink / raw)
To: linux-nvdimm; +Cc: Aneesh Kumar K.V, hch, linux-kernel, linuxppc-dev
The pmem driver on PowerPC crashes with the following signature when
instantiating misaligned namespaces that map their capacity via
memremap_pages().
BUG: Unable to handle kernel data access at 0xc001000406000000
Faulting instruction address: 0xc000000000090790
NIP [c000000000090790] arch_add_memory+0xc0/0x130
LR [c000000000090744] arch_add_memory+0x74/0x130
Call Trace:
arch_add_memory+0x74/0x130 (unreliable)
memremap_pages+0x74c/0xa30
devm_memremap_pages+0x3c/0xa0
pmem_attach_disk+0x188/0x770
nvdimm_bus_probe+0xd8/0x470
With the assumption that only memremap_pages() has alignment
constraints, enforce memremap_compat_align() for
pmem_should_map_pages(), nd_pfn, or nd_dax cases.
Reported-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Cc: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/nvdimm/namespace_devs.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index 032dc61725ff..aff1f32fdb4f 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -1739,6 +1739,16 @@ struct nd_namespace_common *nvdimm_namespace_common_probe(struct device *dev)
return ERR_PTR(-ENODEV);
}
+ if (pmem_should_map_pages(dev) || nd_pfn || nd_dax) {
+ struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
+ resource_size_t start = nsio->res.start;
+
+ if (!IS_ALIGNED(start | size, memremap_compat_align())) {
+ dev_dbg(&ndns->dev, "misaligned, unable to map\n");
+ return ERR_PTR(-EOPNOTSUPP);
+ }
+ }
+
if (is_namespace_pmem(&ndns->dev)) {
struct nd_namespace_pmem *nspm;
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 4/5] libnvdimm/region: Introduce NDD_LABELING
2020-01-30 20:05 [PATCH 0/5] libnvdimm: Cross-arch compatible namespace alignment Dan Williams
` (2 preceding siblings ...)
2020-01-30 20:06 ` [PATCH 3/5] libnvdimm/namespace: Enforce memremap_compat_align() Dan Williams
@ 2020-01-30 20:06 ` Dan Williams
2020-02-03 17:09 ` Aneesh Kumar K.V
2020-01-30 20:06 ` [PATCH 5/5] libnvdimm/region: Introduce an 'align' attribute Dan Williams
4 siblings, 1 reply; 15+ messages in thread
From: Dan Williams @ 2020-01-30 20:06 UTC (permalink / raw)
To: linux-nvdimm; +Cc: Aneesh Kumar K.V, hch, linux-kernel, linuxppc-dev
The NDD_ALIASING flag is used to indicate where pmem capacity might
alias with blk capacity and require labeling. It is also used to
indicate whether the DIMM supports labeling. Separate this latter
capability into its own flag so that the NDD_ALIASING flag is scoped to
true aliased configurations.
To my knowledge aliased configurations only exist in the ACPI spec,
there are no known platforms that ship this support in production.
This clarity allows namespace-capacity alignment constraints around
interleave-ways to be relaxed.
Cc: Vishal Verma <vishal.l.verma@intel.com>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Cc: Oliver O'Halloran <oohall@gmail.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/powerpc/platforms/pseries/papr_scm.c | 2 +-
drivers/acpi/nfit/core.c | 4 +++-
drivers/nvdimm/dimm.c | 2 +-
drivers/nvdimm/dimm_devs.c | 9 +++++----
drivers/nvdimm/namespace_devs.c | 2 +-
drivers/nvdimm/nd.h | 2 +-
drivers/nvdimm/region_devs.c | 10 +++++-----
include/linux/libnvdimm.h | 2 ++
8 files changed, 19 insertions(+), 14 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c
index c2ef320ba1bf..aae60cfd4e38 100644
--- a/arch/powerpc/platforms/pseries/papr_scm.c
+++ b/arch/powerpc/platforms/pseries/papr_scm.c
@@ -326,7 +326,7 @@ static int papr_scm_nvdimm_init(struct papr_scm_priv *p)
}
dimm_flags = 0;
- set_bit(NDD_ALIASING, &dimm_flags);
+ set_bit(NDD_LABELING, &dimm_flags);
p->nvdimm = nvdimm_create(p->bus, p, NULL, dimm_flags,
PAPR_SCM_DIMM_CMD_MASK, 0, NULL);
diff --git a/drivers/acpi/nfit/core.c b/drivers/acpi/nfit/core.c
index a3320f93616d..71d7f2aa1b12 100644
--- a/drivers/acpi/nfit/core.c
+++ b/drivers/acpi/nfit/core.c
@@ -2026,8 +2026,10 @@ static int acpi_nfit_register_dimms(struct acpi_nfit_desc *acpi_desc)
continue;
}
- if (nfit_mem->bdw && nfit_mem->memdev_pmem)
+ if (nfit_mem->bdw && nfit_mem->memdev_pmem) {
set_bit(NDD_ALIASING, &flags);
+ set_bit(NDD_LABELING, &flags);
+ }
/* collate flags across all memdevs for this dimm */
list_for_each_entry(nfit_memdev, &acpi_desc->memdevs, list) {
diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c
index 64776ed15bb3..7d4ddc4d9322 100644
--- a/drivers/nvdimm/dimm.c
+++ b/drivers/nvdimm/dimm.c
@@ -99,7 +99,7 @@ static int nvdimm_probe(struct device *dev)
if (ndd->ns_current >= 0) {
rc = nd_label_reserve_dpa(ndd);
if (rc == 0)
- nvdimm_set_aliasing(dev);
+ nvdimm_set_labeling(dev);
}
nvdimm_bus_unlock(dev);
diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
index 94ea6dba6b4f..64159d4d4b8f 100644
--- a/drivers/nvdimm/dimm_devs.c
+++ b/drivers/nvdimm/dimm_devs.c
@@ -32,7 +32,7 @@ int nvdimm_check_config_data(struct device *dev)
if (!nvdimm->cmd_mask ||
!test_bit(ND_CMD_GET_CONFIG_DATA, &nvdimm->cmd_mask)) {
- if (test_bit(NDD_ALIASING, &nvdimm->flags))
+ if (test_bit(NDD_LABELING, &nvdimm->flags))
return -ENXIO;
else
return -ENOTTY;
@@ -173,11 +173,11 @@ int nvdimm_set_config_data(struct nvdimm_drvdata *ndd, size_t offset,
return rc;
}
-void nvdimm_set_aliasing(struct device *dev)
+void nvdimm_set_labeling(struct device *dev)
{
struct nvdimm *nvdimm = to_nvdimm(dev);
- set_bit(NDD_ALIASING, &nvdimm->flags);
+ set_bit(NDD_LABELING, &nvdimm->flags);
}
void nvdimm_set_locked(struct device *dev)
@@ -312,8 +312,9 @@ static ssize_t flags_show(struct device *dev,
{
struct nvdimm *nvdimm = to_nvdimm(dev);
- return sprintf(buf, "%s%s\n",
+ return sprintf(buf, "%s%s%s\n",
test_bit(NDD_ALIASING, &nvdimm->flags) ? "alias " : "",
+ test_bit(NDD_LABELING, &nvdimm->flags) ? "label" : "",
test_bit(NDD_LOCKED, &nvdimm->flags) ? "lock " : "");
}
static DEVICE_ATTR_RO(flags);
diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index aff1f32fdb4f..30cda9f235de 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -2531,7 +2531,7 @@ static int init_active_labels(struct nd_region *nd_region)
if (!ndd) {
if (test_bit(NDD_LOCKED, &nvdimm->flags))
/* fail, label data may be unreadable */;
- else if (test_bit(NDD_ALIASING, &nvdimm->flags))
+ else if (test_bit(NDD_LABELING, &nvdimm->flags))
/* fail, labels needed to disambiguate dpa */;
else
return 0;
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index c9f6a5b5253a..ca39abe29c7c 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -252,7 +252,7 @@ int nvdimm_set_config_data(struct nvdimm_drvdata *ndd, size_t offset,
void *buf, size_t len);
long nvdimm_clear_poison(struct device *dev, phys_addr_t phys,
unsigned int len);
-void nvdimm_set_aliasing(struct device *dev);
+void nvdimm_set_labeling(struct device *dev);
void nvdimm_set_locked(struct device *dev);
void nvdimm_clear_locked(struct device *dev);
int nvdimm_security_setup_events(struct device *dev);
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index a19e535830d9..a5fc6e4c56ff 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -195,16 +195,16 @@ EXPORT_SYMBOL_GPL(nd_blk_region_set_provider_data);
int nd_region_to_nstype(struct nd_region *nd_region)
{
if (is_memory(&nd_region->dev)) {
- u16 i, alias;
+ u16 i, label;
- for (i = 0, alias = 0; i < nd_region->ndr_mappings; i++) {
+ for (i = 0, label = 0; i < nd_region->ndr_mappings; i++) {
struct nd_mapping *nd_mapping = &nd_region->mapping[i];
struct nvdimm *nvdimm = nd_mapping->nvdimm;
- if (test_bit(NDD_ALIASING, &nvdimm->flags))
- alias++;
+ if (test_bit(NDD_LABELING, &nvdimm->flags))
+ label++;
}
- if (alias)
+ if (label)
return ND_DEVICE_NAMESPACE_PMEM;
else
return ND_DEVICE_NAMESPACE_IO;
diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
index 9df091bd30ba..18da4059be09 100644
--- a/include/linux/libnvdimm.h
+++ b/include/linux/libnvdimm.h
@@ -37,6 +37,8 @@ enum {
NDD_WORK_PENDING = 4,
/* ignore / filter NSLABEL_FLAG_LOCAL for this DIMM, i.e. no aliasing */
NDD_NOBLK = 5,
+ /* dimm supports namespace labels */
+ NDD_LABELING = 6,
/* need to set a limit somewhere, but yes, this is likely overkill */
ND_IOCTL_MAX_BUFLEN = SZ_4M,
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 5/5] libnvdimm/region: Introduce an 'align' attribute
2020-01-30 20:05 [PATCH 0/5] libnvdimm: Cross-arch compatible namespace alignment Dan Williams
` (3 preceding siblings ...)
2020-01-30 20:06 ` [PATCH 4/5] libnvdimm/region: Introduce NDD_LABELING Dan Williams
@ 2020-01-30 20:06 ` Dan Williams
2020-02-03 17:10 ` Aneesh Kumar K.V
4 siblings, 1 reply; 15+ messages in thread
From: Dan Williams @ 2020-01-30 20:06 UTC (permalink / raw)
To: linux-nvdimm; +Cc: Aneesh Kumar K.V, hch, linux-kernel, linuxppc-dev
The align attribute applies an alignment constraint for namespace
creation in a region. Whereas the 'align' attribute of a namespace
applied alignment padding via an info block, the 'align' attribute
applies alignment constraints to the free space allocation.
The default for 'align' is the maximum known memremap_compat_align()
across all archs (16MiB from PowerPC at time of writing) multiplied by
the number of interleave ways if there is blk-aliasing. The minimum is
PAGE_SIZE and allows for the creation of cross-arch incompatible
namespaces, just as previous kernels allowed, but the expectation is
cross-arch and mode-independent compatibility by default.
The regression risk with this change is limited to cases that were
dependent on the ability to create unaligned namespaces, *and* for some
reason are unable to opt-out of aligned namespaces by writing to
'regionX/align'. If such a scenario arises the default can be flipped
from opt-out to opt-in of compat-aligned namespace creation, but that is
a last resort. The kernel will otherwise continue to support existing
defined misaligned namespaces.
Unfortunately this change needs to touch several parts of the
implementation at once:
- region/available_size: expand busy extents to current align
- region/max_available_extent: expand busy extents to current align
- namespace/size: trim free space to current align
...to keep the free space accounting conforming to the dynamic align
setting.
Reported-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Reported-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/nvdimm/dimm_devs.c | 86 +++++++++++++++++++++++----
drivers/nvdimm/namespace_devs.c | 9 ++-
drivers/nvdimm/nd.h | 1
drivers/nvdimm/region_devs.c | 122 ++++++++++++++++++++++++++++++++++++---
4 files changed, 192 insertions(+), 26 deletions(-)
diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
index 64159d4d4b8f..b4994abb655f 100644
--- a/drivers/nvdimm/dimm_devs.c
+++ b/drivers/nvdimm/dimm_devs.c
@@ -563,6 +563,21 @@ int nvdimm_security_freeze(struct nvdimm *nvdimm)
return rc;
}
+static unsigned long dpa_align(struct nd_region *nd_region)
+{
+ struct device *dev = &nd_region->dev;
+
+ if (dev_WARN_ONCE(dev, !is_nvdimm_bus_locked(dev),
+ "bus lock required for capacity provision\n"))
+ return 0;
+ if (dev_WARN_ONCE(dev, !nd_region->ndr_mappings || nd_region->align
+ % nd_region->ndr_mappings,
+ "invalid region align %#lx mappings: %d\n",
+ nd_region->align, nd_region->ndr_mappings))
+ return 0;
+ return nd_region->align / nd_region->ndr_mappings;
+}
+
int alias_dpa_busy(struct device *dev, void *data)
{
resource_size_t map_end, blk_start, new;
@@ -571,6 +586,7 @@ int alias_dpa_busy(struct device *dev, void *data)
struct nd_region *nd_region;
struct nvdimm_drvdata *ndd;
struct resource *res;
+ unsigned long align;
int i;
if (!is_memory(dev))
@@ -608,13 +624,21 @@ int alias_dpa_busy(struct device *dev, void *data)
* Find the free dpa from the end of the last pmem allocation to
* the end of the interleave-set mapping.
*/
+ align = dpa_align(nd_region);
+ if (!align)
+ return 0;
+
for_each_dpa_resource(ndd, res) {
+ resource_size_t start, end;
+
if (strncmp(res->name, "pmem", 4) != 0)
continue;
- if ((res->start >= blk_start && res->start < map_end)
- || (res->end >= blk_start
- && res->end <= map_end)) {
- new = max(blk_start, min(map_end + 1, res->end + 1));
+
+ start = ALIGN_DOWN(res->start, align);
+ end = ALIGN(res->end + 1, align) - 1;
+ if ((start >= blk_start && start < map_end)
+ || (end >= blk_start && end <= map_end)) {
+ new = max(blk_start, min(map_end, end) + 1);
if (new != blk_start) {
blk_start = new;
goto retry;
@@ -654,6 +678,7 @@ resource_size_t nd_blk_available_dpa(struct nd_region *nd_region)
.res = NULL,
};
struct resource *res;
+ unsigned long align;
if (!ndd)
return 0;
@@ -661,10 +686,20 @@ resource_size_t nd_blk_available_dpa(struct nd_region *nd_region)
device_for_each_child(&nvdimm_bus->dev, &info, alias_dpa_busy);
/* now account for busy blk allocations in unaliased dpa */
+ align = dpa_align(nd_region);
+ if (!align)
+ return 0;
for_each_dpa_resource(ndd, res) {
+ resource_size_t start, end, size;
+
if (strncmp(res->name, "blk", 3) != 0)
continue;
- info.available -= resource_size(res);
+ start = ALIGN_DOWN(res->start, align);
+ end = ALIGN(res->end + 1, align) - 1;
+ size = end - start + 1;
+ if (size >= info.available)
+ return 0;
+ info.available -= size;
}
return info.available;
@@ -683,19 +718,31 @@ resource_size_t nd_pmem_max_contiguous_dpa(struct nd_region *nd_region,
struct nvdimm_bus *nvdimm_bus;
resource_size_t max = 0;
struct resource *res;
+ unsigned long align;
/* if a dimm is disabled the available capacity is zero */
if (!ndd)
return 0;
+ align = dpa_align(nd_region);
+ if (!align)
+ return 0;
+
nvdimm_bus = walk_to_nvdimm_bus(ndd->dev);
if (__reserve_free_pmem(&nd_region->dev, nd_mapping->nvdimm))
return 0;
for_each_dpa_resource(ndd, res) {
+ resource_size_t start, end;
+
if (strcmp(res->name, "pmem-reserve") != 0)
continue;
- if (resource_size(res) > max)
- max = resource_size(res);
+ /* trim free space relative to current alignment setting */
+ start = ALIGN(res->start, align);
+ end = ALIGN_DOWN(res->end + 1, align) - 1;
+ if (end < start)
+ continue;
+ if (end - start + 1 > max)
+ max = end - start + 1;
}
release_free_pmem(nvdimm_bus, nd_mapping);
return max;
@@ -723,24 +770,33 @@ resource_size_t nd_pmem_available_dpa(struct nd_region *nd_region,
struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
struct resource *res;
const char *reason;
+ unsigned long align;
if (!ndd)
return 0;
+ align = dpa_align(nd_region);
+ if (!align)
+ return 0;
+
map_start = nd_mapping->start;
map_end = map_start + nd_mapping->size - 1;
blk_start = max(map_start, map_end + 1 - *overlap);
for_each_dpa_resource(ndd, res) {
- if (res->start >= map_start && res->start < map_end) {
+ resource_size_t start, end;
+
+ start = ALIGN_DOWN(res->start, align);
+ end = ALIGN(res->end + 1, align) - 1;
+ if (start >= map_start && start < map_end) {
if (strncmp(res->name, "blk", 3) == 0)
blk_start = min(blk_start,
- max(map_start, res->start));
- else if (res->end > map_end) {
+ max(map_start, start));
+ else if (end > map_end) {
reason = "misaligned to iset";
goto err;
} else
- busy += resource_size(res);
- } else if (res->end >= map_start && res->end <= map_end) {
+ busy += end - start + 1;
+ } else if (end >= map_start && end <= map_end) {
if (strncmp(res->name, "blk", 3) == 0) {
/*
* If a BLK allocation overlaps the start of
@@ -749,8 +805,8 @@ resource_size_t nd_pmem_available_dpa(struct nd_region *nd_region,
*/
blk_start = map_start;
} else
- busy += resource_size(res);
- } else if (map_start > res->start && map_start < res->end) {
+ busy += end - start + 1;
+ } else if (map_start > start && map_start < end) {
/* total eclipse of the mapping */
busy += nd_mapping->size;
blk_start = map_start;
@@ -760,7 +816,7 @@ resource_size_t nd_pmem_available_dpa(struct nd_region *nd_region,
*overlap = map_end + 1 - blk_start;
available = blk_start - map_start;
if (busy < available)
- return available - busy;
+ return ALIGN_DOWN(available - busy, align);
return 0;
err:
diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index 30cda9f235de..4720ad69e1c5 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -541,6 +541,11 @@ static void space_valid(struct nd_region *nd_region, struct nvdimm_drvdata *ndd,
{
bool is_reserve = strcmp(label_id->id, "pmem-reserve") == 0;
bool is_pmem = strncmp(label_id->id, "pmem", 4) == 0;
+ unsigned long align;
+
+ align = nd_region->align / nd_region->ndr_mappings;
+ valid->start = ALIGN(valid->start, align);
+ valid->end = ALIGN_DOWN(valid->end + 1, align) - 1;
if (valid->start >= valid->end)
goto invalid;
@@ -980,10 +985,10 @@ static ssize_t __size_store(struct device *dev, unsigned long long val)
return -ENXIO;
}
- div_u64_rem(val, PAGE_SIZE * nd_region->ndr_mappings, &remainder);
+ div_u64_rem(val, nd_region->align, &remainder);
if (remainder) {
dev_dbg(dev, "%llu is not %ldK aligned\n", val,
- (PAGE_SIZE * nd_region->ndr_mappings) / SZ_1K);
+ nd_region->align / SZ_1K);
return -EINVAL;
}
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index ca39abe29c7c..c4d69c1cce55 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -146,6 +146,7 @@ struct nd_region {
struct device *btt_seed;
struct device *pfn_seed;
struct device *dax_seed;
+ unsigned long align;
u16 ndr_mappings;
u64 ndr_size;
u64 ndr_start;
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index a5fc6e4c56ff..bf239e783940 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -216,21 +216,25 @@ int nd_region_to_nstype(struct nd_region *nd_region)
}
EXPORT_SYMBOL(nd_region_to_nstype);
-static ssize_t size_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static unsigned long long region_size(struct nd_region *nd_region)
{
- struct nd_region *nd_region = to_nd_region(dev);
- unsigned long long size = 0;
-
- if (is_memory(dev)) {
- size = nd_region->ndr_size;
+ if (is_memory(&nd_region->dev)) {
+ return nd_region->ndr_size;
} else if (nd_region->ndr_mappings == 1) {
struct nd_mapping *nd_mapping = &nd_region->mapping[0];
- size = nd_mapping->size;
+ return nd_mapping->size;
}
- return sprintf(buf, "%llu\n", size);
+ return 0;
+}
+
+static ssize_t size_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct nd_region *nd_region = to_nd_region(dev);
+
+ return sprintf(buf, "%llu\n", region_size(nd_region));
}
static DEVICE_ATTR_RO(size);
@@ -529,6 +533,55 @@ static ssize_t read_only_store(struct device *dev,
}
static DEVICE_ATTR_RW(read_only);
+static ssize_t align_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct nd_region *nd_region = to_nd_region(dev);
+
+ return sprintf(buf, "%#lx\n", nd_region->align);
+}
+
+static ssize_t align_store(struct device *dev,
+ struct device_attribute *attr, const char *buf, size_t len)
+{
+ struct nd_region *nd_region = to_nd_region(dev);
+ unsigned long val, dpa;
+ u32 remainder;
+ int rc;
+
+ rc = kstrtoul(buf, 0, &val);
+ if (rc)
+ return rc;
+
+ if (!nd_region->ndr_mappings)
+ return -ENXIO;
+
+ /*
+ * Ensure space-align is evenly divisible by the region
+ * interleave-width because the kernel typically has no facility
+ * to determine which DIMM(s), dimm-physical-addresses, would
+ * contribute to the tail capacity in system-physical-address
+ * space for the namespace.
+ */
+ dpa = val;
+ remainder = do_div(dpa, nd_region->ndr_mappings);
+ if (!is_power_of_2(dpa) || dpa < PAGE_SIZE
+ || val > region_size(nd_region) || remainder)
+ return -EINVAL;
+
+ /*
+ * Given that space allocation consults this value multiple
+ * times ensure it does not change for the duration of the
+ * allocation.
+ */
+ nvdimm_bus_lock(dev);
+ nd_region->align = val;
+ nvdimm_bus_unlock(dev);
+
+ return len;
+}
+static DEVICE_ATTR_RW(align);
+
static ssize_t region_badblocks_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
@@ -571,6 +624,7 @@ static DEVICE_ATTR_RO(persistence_domain);
static struct attribute *nd_region_attributes[] = {
&dev_attr_size.attr,
+ &dev_attr_align.attr,
&dev_attr_nstype.attr,
&dev_attr_mappings.attr,
&dev_attr_btt_seed.attr,
@@ -626,6 +680,19 @@ static umode_t region_visible(struct kobject *kobj, struct attribute *a, int n)
return a->mode;
}
+ if (a == &dev_attr_align.attr) {
+ int i;
+
+ for (i = 0; i < nd_region->ndr_mappings; i++) {
+ struct nd_mapping *nd_mapping = &nd_region->mapping[i];
+ struct nvdimm *nvdimm = nd_mapping->nvdimm;
+
+ if (test_bit(NDD_LABELING, &nvdimm->flags))
+ return a->mode;
+ }
+ return 0;
+ }
+
if (a != &dev_attr_set_cookie.attr
&& a != &dev_attr_available_size.attr)
return a->mode;
@@ -935,6 +1002,42 @@ void nd_region_release_lane(struct nd_region *nd_region, unsigned int lane)
}
EXPORT_SYMBOL(nd_region_release_lane);
+/*
+ * PowerPC requires this alignment for memremap_pages(). All other archs
+ * should be ok with SUBSECTION_SIZE (see memremap_compat_align()).
+ */
+#define MEMREMAP_COMPAT_ALIGN_MAX SZ_16M
+
+static unsigned long default_align(struct nd_region *nd_region)
+{
+ unsigned long align, per_mapping;
+ int i, mappings;
+ u32 remainder;
+
+ if (is_nd_blk(&nd_region->dev))
+ align = PAGE_SIZE;
+ else
+ align = MEMREMAP_COMPAT_ALIGN_MAX;
+
+ for (i = 0; i < nd_region->ndr_mappings; i++) {
+ struct nd_mapping *nd_mapping = &nd_region->mapping[i];
+ struct nvdimm *nvdimm = nd_mapping->nvdimm;
+
+ if (test_bit(NDD_ALIASING, &nvdimm->flags)) {
+ align = MEMREMAP_COMPAT_ALIGN_MAX;
+ break;
+ }
+ }
+
+ mappings = max_t(u16, 1, nd_region->ndr_mappings);
+ per_mapping = align;
+ remainder = do_div(per_mapping, mappings);
+ if (remainder)
+ align *= mappings;
+
+ return align;
+}
+
static struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus,
struct nd_region_desc *ndr_desc,
const struct device_type *dev_type, const char *caller)
@@ -1039,6 +1142,7 @@ static struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus,
dev->of_node = ndr_desc->of_node;
nd_region->ndr_size = resource_size(ndr_desc->res);
nd_region->ndr_start = ndr_desc->res->start;
+ nd_region->align = default_align(nd_region);
if (ndr_desc->flush)
nd_region->flush = ndr_desc->flush;
else
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 1/5] mm/memremap_pages: Kill unused __devm_memremap_pages()
2020-01-30 20:06 ` [PATCH 1/5] mm/memremap_pages: Kill unused __devm_memremap_pages() Dan Williams
@ 2020-01-31 5:32 ` Christoph Hellwig
2020-02-03 17:08 ` Aneesh Kumar K.V
1 sibling, 0 replies; 15+ messages in thread
From: Christoph Hellwig @ 2020-01-31 5:32 UTC (permalink / raw)
To: Dan Williams; +Cc: linux-nvdimm, Christoph Hellwig, linux-kernel, linuxppc-dev
On Thu, Jan 30, 2020 at 12:06:01PM -0800, Dan Williams wrote:
> Kill this definition that was introduced in commit 41e94a851304 ("add
> devm_memremap_pages") add never used.
>
> Cc: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 1/5] mm/memremap_pages: Kill unused __devm_memremap_pages()
2020-01-30 20:06 ` [PATCH 1/5] mm/memremap_pages: Kill unused __devm_memremap_pages() Dan Williams
2020-01-31 5:32 ` Christoph Hellwig
@ 2020-02-03 17:08 ` Aneesh Kumar K.V
1 sibling, 0 replies; 15+ messages in thread
From: Aneesh Kumar K.V @ 2020-02-03 17:08 UTC (permalink / raw)
To: Dan Williams, linux-nvdimm; +Cc: Christoph Hellwig, linux-kernel, linuxppc-dev
Dan Williams <dan.j.williams@intel.com> writes:
> Kill this definition that was introduced in commit 41e94a851304 ("add
> devm_memremap_pages") add never used.
>
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> include/linux/io.h | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/include/linux/io.h b/include/linux/io.h
> index a59834bc0a11..35e8d84935e0 100644
> --- a/include/linux/io.h
> +++ b/include/linux/io.h
> @@ -79,8 +79,6 @@ void *devm_memremap(struct device *dev, resource_size_t offset,
> size_t size, unsigned long flags);
> void devm_memunmap(struct device *dev, void *addr);
>
> -void *__devm_memremap_pages(struct device *dev, struct resource *res);
> -
> #ifdef CONFIG_PCI
> /*
> * The PCI specifications (Rev 3.0, 3.2.5 "Transaction Ordering and
> _______________________________________________
> Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
> To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 2/5] mm/memremap_pages: Introduce memremap_compat_align()
2020-01-30 20:06 ` [PATCH 2/5] mm/memremap_pages: Introduce memremap_compat_align() Dan Williams
@ 2020-02-03 17:09 ` Aneesh Kumar K.V
2020-02-05 3:05 ` Michael Ellerman
1 sibling, 0 replies; 15+ messages in thread
From: Aneesh Kumar K.V @ 2020-02-03 17:09 UTC (permalink / raw)
To: Dan Williams, linux-nvdimm
Cc: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman, hch,
linux-kernel, linuxppc-dev
Dan Williams <dan.j.williams@intel.com> writes:
> The "sub-section memory hotplug" facility allows memremap_pages() users
> like libnvdimm to compensate for hardware platforms like x86 that have a
> section size larger than their hardware memory mapping granularity. The
> compensation that sub-section support affords is being tolerant of
> physical memory resources shifting by units smaller (64MiB on x86) than
> the memory-hotplug section size (128 MiB). Where the platform
> physical-memory mapping granularity is limited by the number and
> capability of address-decode-registers in the memory controller.
>
> While the sub-section support allows memremap_pages() to operate on
> sub-section (2MiB) granularity, the Power architecture may still
> require 16MiB alignment on "!radix_enabled()" platforms.
>
> In order for libnvdimm to be able to detect and manage this per-arch
> limitation, introduce memremap_compat_align() as a common minimum
> alignment across all driver-facing memory-mapping interfaces, and let
> Power override it to 16MiB in the "!radix_enabled()" case.
>
> The assumption / requirement for 16MiB to be a viable
> memremap_compat_align() value is that Power does not have platforms
> where its equivalent of address-decode-registers never hardware remaps a
> persistent memory resource on smaller than 16MiB boundaries.
>
> Based on an initial patch by Aneesh.
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
>
> Link: http://lore.kernel.org/r/CAPcyv4gBGNP95APYaBcsocEa50tQj9b5h__83vgngjq3ouGX_Q@mail.gmail.com
> Reported-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Reported-by: Jeff Moyer <jmoyer@redhat.com>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> arch/powerpc/include/asm/io.h | 10 ++++++++++
> drivers/nvdimm/pfn_devs.c | 2 +-
> include/linux/io.h | 23 +++++++++++++++++++++++
> include/linux/mmzone.h | 1 +
> 4 files changed, 35 insertions(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/include/asm/io.h b/arch/powerpc/include/asm/io.h
> index a63ec938636d..0fa2dc483008 100644
> --- a/arch/powerpc/include/asm/io.h
> +++ b/arch/powerpc/include/asm/io.h
> @@ -734,6 +734,16 @@ extern void __iomem * __ioremap_at(phys_addr_t pa, void *ea,
> unsigned long size, pgprot_t prot);
> extern void __iounmap_at(void *ea, unsigned long size);
>
> +#ifdef CONFIG_SPARSEMEM
> +static inline unsigned long memremap_compat_align(void)
> +{
> + if (radix_enabled())
> + return SUBSECTION_SIZE;
> + return (1UL << mmu_psize_defs[mmu_linear_psize].shift);
> +}
> +#define memremap_compat_align memremap_compat_align
> +#endif
> +
> /*
> * When CONFIG_PPC_INDIRECT_PIO is set, we use the generic iomap implementation
> * which needs some additional definitions here. They basically allow PIO
> diff --git a/drivers/nvdimm/pfn_devs.c b/drivers/nvdimm/pfn_devs.c
> index b94f7a7e94b8..a5c25cb87116 100644
> --- a/drivers/nvdimm/pfn_devs.c
> +++ b/drivers/nvdimm/pfn_devs.c
> @@ -750,7 +750,7 @@ static int nd_pfn_init(struct nd_pfn *nd_pfn)
> start = nsio->res.start;
> size = resource_size(&nsio->res);
> npfns = PHYS_PFN(size - SZ_8K);
> - align = max(nd_pfn->align, (1UL << SUBSECTION_SHIFT));
> + align = max(nd_pfn->align, SUBSECTION_SIZE);
> end_trunc = start + size - ALIGN_DOWN(start + size, align);
> if (nd_pfn->mode == PFN_MODE_PMEM) {
> /*
> diff --git a/include/linux/io.h b/include/linux/io.h
> index 35e8d84935e0..ccd34519fad3 100644
> --- a/include/linux/io.h
> +++ b/include/linux/io.h
> @@ -6,6 +6,7 @@
> #ifndef _LINUX_IO_H
> #define _LINUX_IO_H
>
> +#include <linux/mmzone.h>
> #include <linux/types.h>
> #include <linux/init.h>
> #include <linux/bug.h>
> @@ -79,6 +80,28 @@ void *devm_memremap(struct device *dev, resource_size_t offset,
> size_t size, unsigned long flags);
> void devm_memunmap(struct device *dev, void *addr);
>
> +#ifndef memremap_compat_align
> +#ifdef CONFIG_SPARSEMEM
> +/*
> + * Minimum compatible alignment of the resource (start, end) across
> + * memremap interfaces (i.e. memremap + memremap_pages)
> + */
> +static inline unsigned long memremap_compat_align(void)
> +{
> + return SUBSECTION_SIZE;
> +}
> +#else /* CONFIG_SPARSEMEM */
> +/*
> + * No ZONE_DEVICE / memremap_pages() support so the minimum mapping
> + * granularity is a single page.
> + */
> +static inline unsigned long memremap_compat_align(void)
> +{
> + return PAGE_SIZE;
> +}
> +#endif /* CONFIG_SPARSEMEM */
> +#endif /* memremap_compat_align */
> +
> #ifdef CONFIG_PCI
> /*
> * The PCI specifications (Rev 3.0, 3.2.5 "Transaction Ordering and
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 89d8ff06c9ce..b0de83620cd7 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -1171,6 +1171,7 @@ static inline unsigned long section_nr_to_pfn(unsigned long sec)
> #define SECTION_ALIGN_DOWN(pfn) ((pfn) & PAGE_SECTION_MASK)
>
> #define SUBSECTION_SHIFT 21
> +#define SUBSECTION_SIZE (1UL << SUBSECTION_SHIFT)
>
> #define PFN_SUBSECTION_SHIFT (SUBSECTION_SHIFT - PAGE_SHIFT)
> #define PAGES_PER_SUBSECTION (1UL << PFN_SUBSECTION_SHIFT)
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 3/5] libnvdimm/namespace: Enforce memremap_compat_align()
2020-01-30 20:06 ` [PATCH 3/5] libnvdimm/namespace: Enforce memremap_compat_align() Dan Williams
@ 2020-02-03 17:09 ` Aneesh Kumar K.V
0 siblings, 0 replies; 15+ messages in thread
From: Aneesh Kumar K.V @ 2020-02-03 17:09 UTC (permalink / raw)
To: Dan Williams, linux-nvdimm; +Cc: hch, linux-kernel, linuxppc-dev
Dan Williams <dan.j.williams@intel.com> writes:
> The pmem driver on PowerPC crashes with the following signature when
> instantiating misaligned namespaces that map their capacity via
> memremap_pages().
>
> BUG: Unable to handle kernel data access at 0xc001000406000000
> Faulting instruction address: 0xc000000000090790
> NIP [c000000000090790] arch_add_memory+0xc0/0x130
> LR [c000000000090744] arch_add_memory+0x74/0x130
> Call Trace:
> arch_add_memory+0x74/0x130 (unreliable)
> memremap_pages+0x74c/0xa30
> devm_memremap_pages+0x3c/0xa0
> pmem_attach_disk+0x188/0x770
> nvdimm_bus_probe+0xd8/0x470
>
> With the assumption that only memremap_pages() has alignment
> constraints, enforce memremap_compat_align() for
> pmem_should_map_pages(), nd_pfn, or nd_dax cases.
>
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Reported-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Cc: Jeff Moyer <jmoyer@redhat.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> drivers/nvdimm/namespace_devs.c | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
> index 032dc61725ff..aff1f32fdb4f 100644
> --- a/drivers/nvdimm/namespace_devs.c
> +++ b/drivers/nvdimm/namespace_devs.c
> @@ -1739,6 +1739,16 @@ struct nd_namespace_common *nvdimm_namespace_common_probe(struct device *dev)
> return ERR_PTR(-ENODEV);
> }
>
> + if (pmem_should_map_pages(dev) || nd_pfn || nd_dax) {
> + struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
> + resource_size_t start = nsio->res.start;
> +
> + if (!IS_ALIGNED(start | size, memremap_compat_align())) {
> + dev_dbg(&ndns->dev, "misaligned, unable to map\n");
> + return ERR_PTR(-EOPNOTSUPP);
> + }
> + }
> +
> if (is_namespace_pmem(&ndns->dev)) {
> struct nd_namespace_pmem *nspm;
>
> _______________________________________________
> Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
> To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 4/5] libnvdimm/region: Introduce NDD_LABELING
2020-01-30 20:06 ` [PATCH 4/5] libnvdimm/region: Introduce NDD_LABELING Dan Williams
@ 2020-02-03 17:09 ` Aneesh Kumar K.V
0 siblings, 0 replies; 15+ messages in thread
From: Aneesh Kumar K.V @ 2020-02-03 17:09 UTC (permalink / raw)
To: Dan Williams, linux-nvdimm; +Cc: hch, linux-kernel, linuxppc-dev
Dan Williams <dan.j.williams@intel.com> writes:
> The NDD_ALIASING flag is used to indicate where pmem capacity might
> alias with blk capacity and require labeling. It is also used to
> indicate whether the DIMM supports labeling. Separate this latter
> capability into its own flag so that the NDD_ALIASING flag is scoped to
> true aliased configurations.
>
> To my knowledge aliased configurations only exist in the ACPI spec,
> there are no known platforms that ship this support in production.
>
> This clarity allows namespace-capacity alignment constraints around
> interleave-ways to be relaxed.
>
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Cc: Vishal Verma <vishal.l.verma@intel.com>
> Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Cc: Oliver O'Halloran <oohall@gmail.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> arch/powerpc/platforms/pseries/papr_scm.c | 2 +-
> drivers/acpi/nfit/core.c | 4 +++-
> drivers/nvdimm/dimm.c | 2 +-
> drivers/nvdimm/dimm_devs.c | 9 +++++----
> drivers/nvdimm/namespace_devs.c | 2 +-
> drivers/nvdimm/nd.h | 2 +-
> drivers/nvdimm/region_devs.c | 10 +++++-----
> include/linux/libnvdimm.h | 2 ++
> 8 files changed, 19 insertions(+), 14 deletions(-)
>
> diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c
> index c2ef320ba1bf..aae60cfd4e38 100644
> --- a/arch/powerpc/platforms/pseries/papr_scm.c
> +++ b/arch/powerpc/platforms/pseries/papr_scm.c
> @@ -326,7 +326,7 @@ static int papr_scm_nvdimm_init(struct papr_scm_priv *p)
> }
>
> dimm_flags = 0;
> - set_bit(NDD_ALIASING, &dimm_flags);
> + set_bit(NDD_LABELING, &dimm_flags);
>
> p->nvdimm = nvdimm_create(p->bus, p, NULL, dimm_flags,
> PAPR_SCM_DIMM_CMD_MASK, 0, NULL);
> diff --git a/drivers/acpi/nfit/core.c b/drivers/acpi/nfit/core.c
> index a3320f93616d..71d7f2aa1b12 100644
> --- a/drivers/acpi/nfit/core.c
> +++ b/drivers/acpi/nfit/core.c
> @@ -2026,8 +2026,10 @@ static int acpi_nfit_register_dimms(struct acpi_nfit_desc *acpi_desc)
> continue;
> }
>
> - if (nfit_mem->bdw && nfit_mem->memdev_pmem)
> + if (nfit_mem->bdw && nfit_mem->memdev_pmem) {
> set_bit(NDD_ALIASING, &flags);
> + set_bit(NDD_LABELING, &flags);
> + }
>
> /* collate flags across all memdevs for this dimm */
> list_for_each_entry(nfit_memdev, &acpi_desc->memdevs, list) {
> diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c
> index 64776ed15bb3..7d4ddc4d9322 100644
> --- a/drivers/nvdimm/dimm.c
> +++ b/drivers/nvdimm/dimm.c
> @@ -99,7 +99,7 @@ static int nvdimm_probe(struct device *dev)
> if (ndd->ns_current >= 0) {
> rc = nd_label_reserve_dpa(ndd);
> if (rc == 0)
> - nvdimm_set_aliasing(dev);
> + nvdimm_set_labeling(dev);
> }
> nvdimm_bus_unlock(dev);
>
> diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
> index 94ea6dba6b4f..64159d4d4b8f 100644
> --- a/drivers/nvdimm/dimm_devs.c
> +++ b/drivers/nvdimm/dimm_devs.c
> @@ -32,7 +32,7 @@ int nvdimm_check_config_data(struct device *dev)
>
> if (!nvdimm->cmd_mask ||
> !test_bit(ND_CMD_GET_CONFIG_DATA, &nvdimm->cmd_mask)) {
> - if (test_bit(NDD_ALIASING, &nvdimm->flags))
> + if (test_bit(NDD_LABELING, &nvdimm->flags))
> return -ENXIO;
> else
> return -ENOTTY;
> @@ -173,11 +173,11 @@ int nvdimm_set_config_data(struct nvdimm_drvdata *ndd, size_t offset,
> return rc;
> }
>
> -void nvdimm_set_aliasing(struct device *dev)
> +void nvdimm_set_labeling(struct device *dev)
> {
> struct nvdimm *nvdimm = to_nvdimm(dev);
>
> - set_bit(NDD_ALIASING, &nvdimm->flags);
> + set_bit(NDD_LABELING, &nvdimm->flags);
> }
>
> void nvdimm_set_locked(struct device *dev)
> @@ -312,8 +312,9 @@ static ssize_t flags_show(struct device *dev,
> {
> struct nvdimm *nvdimm = to_nvdimm(dev);
>
> - return sprintf(buf, "%s%s\n",
> + return sprintf(buf, "%s%s%s\n",
> test_bit(NDD_ALIASING, &nvdimm->flags) ? "alias " : "",
> + test_bit(NDD_LABELING, &nvdimm->flags) ? "label" : "",
> test_bit(NDD_LOCKED, &nvdimm->flags) ? "lock " : "");
> }
> static DEVICE_ATTR_RO(flags);
> diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
> index aff1f32fdb4f..30cda9f235de 100644
> --- a/drivers/nvdimm/namespace_devs.c
> +++ b/drivers/nvdimm/namespace_devs.c
> @@ -2531,7 +2531,7 @@ static int init_active_labels(struct nd_region *nd_region)
> if (!ndd) {
> if (test_bit(NDD_LOCKED, &nvdimm->flags))
> /* fail, label data may be unreadable */;
> - else if (test_bit(NDD_ALIASING, &nvdimm->flags))
> + else if (test_bit(NDD_LABELING, &nvdimm->flags))
> /* fail, labels needed to disambiguate dpa */;
> else
> return 0;
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index c9f6a5b5253a..ca39abe29c7c 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h
> @@ -252,7 +252,7 @@ int nvdimm_set_config_data(struct nvdimm_drvdata *ndd, size_t offset,
> void *buf, size_t len);
> long nvdimm_clear_poison(struct device *dev, phys_addr_t phys,
> unsigned int len);
> -void nvdimm_set_aliasing(struct device *dev);
> +void nvdimm_set_labeling(struct device *dev);
> void nvdimm_set_locked(struct device *dev);
> void nvdimm_clear_locked(struct device *dev);
> int nvdimm_security_setup_events(struct device *dev);
> diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
> index a19e535830d9..a5fc6e4c56ff 100644
> --- a/drivers/nvdimm/region_devs.c
> +++ b/drivers/nvdimm/region_devs.c
> @@ -195,16 +195,16 @@ EXPORT_SYMBOL_GPL(nd_blk_region_set_provider_data);
> int nd_region_to_nstype(struct nd_region *nd_region)
> {
> if (is_memory(&nd_region->dev)) {
> - u16 i, alias;
> + u16 i, label;
>
> - for (i = 0, alias = 0; i < nd_region->ndr_mappings; i++) {
> + for (i = 0, label = 0; i < nd_region->ndr_mappings; i++) {
> struct nd_mapping *nd_mapping = &nd_region->mapping[i];
> struct nvdimm *nvdimm = nd_mapping->nvdimm;
>
> - if (test_bit(NDD_ALIASING, &nvdimm->flags))
> - alias++;
> + if (test_bit(NDD_LABELING, &nvdimm->flags))
> + label++;
> }
> - if (alias)
> + if (label)
> return ND_DEVICE_NAMESPACE_PMEM;
> else
> return ND_DEVICE_NAMESPACE_IO;
> diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
> index 9df091bd30ba..18da4059be09 100644
> --- a/include/linux/libnvdimm.h
> +++ b/include/linux/libnvdimm.h
> @@ -37,6 +37,8 @@ enum {
> NDD_WORK_PENDING = 4,
> /* ignore / filter NSLABEL_FLAG_LOCAL for this DIMM, i.e. no aliasing */
> NDD_NOBLK = 5,
> + /* dimm supports namespace labels */
> + NDD_LABELING = 6,
>
> /* need to set a limit somewhere, but yes, this is likely overkill */
> ND_IOCTL_MAX_BUFLEN = SZ_4M,
> _______________________________________________
> Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
> To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 5/5] libnvdimm/region: Introduce an 'align' attribute
2020-01-30 20:06 ` [PATCH 5/5] libnvdimm/region: Introduce an 'align' attribute Dan Williams
@ 2020-02-03 17:10 ` Aneesh Kumar K.V
0 siblings, 0 replies; 15+ messages in thread
From: Aneesh Kumar K.V @ 2020-02-03 17:10 UTC (permalink / raw)
To: Dan Williams, linux-nvdimm; +Cc: hch, linux-kernel, linuxppc-dev
Dan Williams <dan.j.williams@intel.com> writes:
> The align attribute applies an alignment constraint for namespace
> creation in a region. Whereas the 'align' attribute of a namespace
> applied alignment padding via an info block, the 'align' attribute
> applies alignment constraints to the free space allocation.
>
> The default for 'align' is the maximum known memremap_compat_align()
> across all archs (16MiB from PowerPC at time of writing) multiplied by
> the number of interleave ways if there is blk-aliasing. The minimum is
> PAGE_SIZE and allows for the creation of cross-arch incompatible
> namespaces, just as previous kernels allowed, but the expectation is
> cross-arch and mode-independent compatibility by default.
>
> The regression risk with this change is limited to cases that were
> dependent on the ability to create unaligned namespaces, *and* for some
> reason are unable to opt-out of aligned namespaces by writing to
> 'regionX/align'. If such a scenario arises the default can be flipped
> from opt-out to opt-in of compat-aligned namespace creation, but that is
> a last resort. The kernel will otherwise continue to support existing
> defined misaligned namespaces.
>
> Unfortunately this change needs to touch several parts of the
> implementation at once:
>
> - region/available_size: expand busy extents to current align
> - region/max_available_extent: expand busy extents to current align
> - namespace/size: trim free space to current align
>
> ...to keep the free space accounting conforming to the dynamic align
> setting.
>
Reviewed-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Reported-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Reported-by: Jeff Moyer <jmoyer@redhat.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> drivers/nvdimm/dimm_devs.c | 86 +++++++++++++++++++++++----
> drivers/nvdimm/namespace_devs.c | 9 ++-
> drivers/nvdimm/nd.h | 1
> drivers/nvdimm/region_devs.c | 122 ++++++++++++++++++++++++++++++++++++---
> 4 files changed, 192 insertions(+), 26 deletions(-)
>
> diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
> index 64159d4d4b8f..b4994abb655f 100644
> --- a/drivers/nvdimm/dimm_devs.c
> +++ b/drivers/nvdimm/dimm_devs.c
> @@ -563,6 +563,21 @@ int nvdimm_security_freeze(struct nvdimm *nvdimm)
> return rc;
> }
>
> +static unsigned long dpa_align(struct nd_region *nd_region)
> +{
> + struct device *dev = &nd_region->dev;
> +
> + if (dev_WARN_ONCE(dev, !is_nvdimm_bus_locked(dev),
> + "bus lock required for capacity provision\n"))
> + return 0;
> + if (dev_WARN_ONCE(dev, !nd_region->ndr_mappings || nd_region->align
> + % nd_region->ndr_mappings,
> + "invalid region align %#lx mappings: %d\n",
> + nd_region->align, nd_region->ndr_mappings))
> + return 0;
> + return nd_region->align / nd_region->ndr_mappings;
> +}
> +
> int alias_dpa_busy(struct device *dev, void *data)
> {
> resource_size_t map_end, blk_start, new;
> @@ -571,6 +586,7 @@ int alias_dpa_busy(struct device *dev, void *data)
> struct nd_region *nd_region;
> struct nvdimm_drvdata *ndd;
> struct resource *res;
> + unsigned long align;
> int i;
>
> if (!is_memory(dev))
> @@ -608,13 +624,21 @@ int alias_dpa_busy(struct device *dev, void *data)
> * Find the free dpa from the end of the last pmem allocation to
> * the end of the interleave-set mapping.
> */
> + align = dpa_align(nd_region);
> + if (!align)
> + return 0;
> +
> for_each_dpa_resource(ndd, res) {
> + resource_size_t start, end;
> +
> if (strncmp(res->name, "pmem", 4) != 0)
> continue;
> - if ((res->start >= blk_start && res->start < map_end)
> - || (res->end >= blk_start
> - && res->end <= map_end)) {
> - new = max(blk_start, min(map_end + 1, res->end + 1));
> +
> + start = ALIGN_DOWN(res->start, align);
> + end = ALIGN(res->end + 1, align) - 1;
> + if ((start >= blk_start && start < map_end)
> + || (end >= blk_start && end <= map_end)) {
> + new = max(blk_start, min(map_end, end) + 1);
> if (new != blk_start) {
> blk_start = new;
> goto retry;
> @@ -654,6 +678,7 @@ resource_size_t nd_blk_available_dpa(struct nd_region *nd_region)
> .res = NULL,
> };
> struct resource *res;
> + unsigned long align;
>
> if (!ndd)
> return 0;
> @@ -661,10 +686,20 @@ resource_size_t nd_blk_available_dpa(struct nd_region *nd_region)
> device_for_each_child(&nvdimm_bus->dev, &info, alias_dpa_busy);
>
> /* now account for busy blk allocations in unaliased dpa */
> + align = dpa_align(nd_region);
> + if (!align)
> + return 0;
> for_each_dpa_resource(ndd, res) {
> + resource_size_t start, end, size;
> +
> if (strncmp(res->name, "blk", 3) != 0)
> continue;
> - info.available -= resource_size(res);
> + start = ALIGN_DOWN(res->start, align);
> + end = ALIGN(res->end + 1, align) - 1;
> + size = end - start + 1;
> + if (size >= info.available)
> + return 0;
> + info.available -= size;
> }
>
> return info.available;
> @@ -683,19 +718,31 @@ resource_size_t nd_pmem_max_contiguous_dpa(struct nd_region *nd_region,
> struct nvdimm_bus *nvdimm_bus;
> resource_size_t max = 0;
> struct resource *res;
> + unsigned long align;
>
> /* if a dimm is disabled the available capacity is zero */
> if (!ndd)
> return 0;
>
> + align = dpa_align(nd_region);
> + if (!align)
> + return 0;
> +
> nvdimm_bus = walk_to_nvdimm_bus(ndd->dev);
> if (__reserve_free_pmem(&nd_region->dev, nd_mapping->nvdimm))
> return 0;
> for_each_dpa_resource(ndd, res) {
> + resource_size_t start, end;
> +
> if (strcmp(res->name, "pmem-reserve") != 0)
> continue;
> - if (resource_size(res) > max)
> - max = resource_size(res);
> + /* trim free space relative to current alignment setting */
> + start = ALIGN(res->start, align);
> + end = ALIGN_DOWN(res->end + 1, align) - 1;
> + if (end < start)
> + continue;
> + if (end - start + 1 > max)
> + max = end - start + 1;
> }
> release_free_pmem(nvdimm_bus, nd_mapping);
> return max;
> @@ -723,24 +770,33 @@ resource_size_t nd_pmem_available_dpa(struct nd_region *nd_region,
> struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
> struct resource *res;
> const char *reason;
> + unsigned long align;
>
> if (!ndd)
> return 0;
>
> + align = dpa_align(nd_region);
> + if (!align)
> + return 0;
> +
> map_start = nd_mapping->start;
> map_end = map_start + nd_mapping->size - 1;
> blk_start = max(map_start, map_end + 1 - *overlap);
> for_each_dpa_resource(ndd, res) {
> - if (res->start >= map_start && res->start < map_end) {
> + resource_size_t start, end;
> +
> + start = ALIGN_DOWN(res->start, align);
> + end = ALIGN(res->end + 1, align) - 1;
> + if (start >= map_start && start < map_end) {
> if (strncmp(res->name, "blk", 3) == 0)
> blk_start = min(blk_start,
> - max(map_start, res->start));
> - else if (res->end > map_end) {
> + max(map_start, start));
> + else if (end > map_end) {
> reason = "misaligned to iset";
> goto err;
> } else
> - busy += resource_size(res);
> - } else if (res->end >= map_start && res->end <= map_end) {
> + busy += end - start + 1;
> + } else if (end >= map_start && end <= map_end) {
> if (strncmp(res->name, "blk", 3) == 0) {
> /*
> * If a BLK allocation overlaps the start of
> @@ -749,8 +805,8 @@ resource_size_t nd_pmem_available_dpa(struct nd_region *nd_region,
> */
> blk_start = map_start;
> } else
> - busy += resource_size(res);
> - } else if (map_start > res->start && map_start < res->end) {
> + busy += end - start + 1;
> + } else if (map_start > start && map_start < end) {
> /* total eclipse of the mapping */
> busy += nd_mapping->size;
> blk_start = map_start;
> @@ -760,7 +816,7 @@ resource_size_t nd_pmem_available_dpa(struct nd_region *nd_region,
> *overlap = map_end + 1 - blk_start;
> available = blk_start - map_start;
> if (busy < available)
> - return available - busy;
> + return ALIGN_DOWN(available - busy, align);
> return 0;
>
> err:
> diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
> index 30cda9f235de..4720ad69e1c5 100644
> --- a/drivers/nvdimm/namespace_devs.c
> +++ b/drivers/nvdimm/namespace_devs.c
> @@ -541,6 +541,11 @@ static void space_valid(struct nd_region *nd_region, struct nvdimm_drvdata *ndd,
> {
> bool is_reserve = strcmp(label_id->id, "pmem-reserve") == 0;
> bool is_pmem = strncmp(label_id->id, "pmem", 4) == 0;
> + unsigned long align;
> +
> + align = nd_region->align / nd_region->ndr_mappings;
> + valid->start = ALIGN(valid->start, align);
> + valid->end = ALIGN_DOWN(valid->end + 1, align) - 1;
>
> if (valid->start >= valid->end)
> goto invalid;
> @@ -980,10 +985,10 @@ static ssize_t __size_store(struct device *dev, unsigned long long val)
> return -ENXIO;
> }
>
> - div_u64_rem(val, PAGE_SIZE * nd_region->ndr_mappings, &remainder);
> + div_u64_rem(val, nd_region->align, &remainder);
> if (remainder) {
> dev_dbg(dev, "%llu is not %ldK aligned\n", val,
> - (PAGE_SIZE * nd_region->ndr_mappings) / SZ_1K);
> + nd_region->align / SZ_1K);
> return -EINVAL;
> }
>
> diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
> index ca39abe29c7c..c4d69c1cce55 100644
> --- a/drivers/nvdimm/nd.h
> +++ b/drivers/nvdimm/nd.h
> @@ -146,6 +146,7 @@ struct nd_region {
> struct device *btt_seed;
> struct device *pfn_seed;
> struct device *dax_seed;
> + unsigned long align;
> u16 ndr_mappings;
> u64 ndr_size;
> u64 ndr_start;
> diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
> index a5fc6e4c56ff..bf239e783940 100644
> --- a/drivers/nvdimm/region_devs.c
> +++ b/drivers/nvdimm/region_devs.c
> @@ -216,21 +216,25 @@ int nd_region_to_nstype(struct nd_region *nd_region)
> }
> EXPORT_SYMBOL(nd_region_to_nstype);
>
> -static ssize_t size_show(struct device *dev,
> - struct device_attribute *attr, char *buf)
> +static unsigned long long region_size(struct nd_region *nd_region)
> {
> - struct nd_region *nd_region = to_nd_region(dev);
> - unsigned long long size = 0;
> -
> - if (is_memory(dev)) {
> - size = nd_region->ndr_size;
> + if (is_memory(&nd_region->dev)) {
> + return nd_region->ndr_size;
> } else if (nd_region->ndr_mappings == 1) {
> struct nd_mapping *nd_mapping = &nd_region->mapping[0];
>
> - size = nd_mapping->size;
> + return nd_mapping->size;
> }
>
> - return sprintf(buf, "%llu\n", size);
> + return 0;
> +}
> +
> +static ssize_t size_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + struct nd_region *nd_region = to_nd_region(dev);
> +
> + return sprintf(buf, "%llu\n", region_size(nd_region));
> }
> static DEVICE_ATTR_RO(size);
>
> @@ -529,6 +533,55 @@ static ssize_t read_only_store(struct device *dev,
> }
> static DEVICE_ATTR_RW(read_only);
>
> +static ssize_t align_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + struct nd_region *nd_region = to_nd_region(dev);
> +
> + return sprintf(buf, "%#lx\n", nd_region->align);
> +}
> +
> +static ssize_t align_store(struct device *dev,
> + struct device_attribute *attr, const char *buf, size_t len)
> +{
> + struct nd_region *nd_region = to_nd_region(dev);
> + unsigned long val, dpa;
> + u32 remainder;
> + int rc;
> +
> + rc = kstrtoul(buf, 0, &val);
> + if (rc)
> + return rc;
> +
> + if (!nd_region->ndr_mappings)
> + return -ENXIO;
> +
> + /*
> + * Ensure space-align is evenly divisible by the region
> + * interleave-width because the kernel typically has no facility
> + * to determine which DIMM(s), dimm-physical-addresses, would
> + * contribute to the tail capacity in system-physical-address
> + * space for the namespace.
> + */
> + dpa = val;
> + remainder = do_div(dpa, nd_region->ndr_mappings);
> + if (!is_power_of_2(dpa) || dpa < PAGE_SIZE
> + || val > region_size(nd_region) || remainder)
> + return -EINVAL;
> +
> + /*
> + * Given that space allocation consults this value multiple
> + * times ensure it does not change for the duration of the
> + * allocation.
> + */
> + nvdimm_bus_lock(dev);
> + nd_region->align = val;
> + nvdimm_bus_unlock(dev);
> +
> + return len;
> +}
> +static DEVICE_ATTR_RW(align);
> +
> static ssize_t region_badblocks_show(struct device *dev,
> struct device_attribute *attr, char *buf)
> {
> @@ -571,6 +624,7 @@ static DEVICE_ATTR_RO(persistence_domain);
>
> static struct attribute *nd_region_attributes[] = {
> &dev_attr_size.attr,
> + &dev_attr_align.attr,
> &dev_attr_nstype.attr,
> &dev_attr_mappings.attr,
> &dev_attr_btt_seed.attr,
> @@ -626,6 +680,19 @@ static umode_t region_visible(struct kobject *kobj, struct attribute *a, int n)
> return a->mode;
> }
>
> + if (a == &dev_attr_align.attr) {
> + int i;
> +
> + for (i = 0; i < nd_region->ndr_mappings; i++) {
> + struct nd_mapping *nd_mapping = &nd_region->mapping[i];
> + struct nvdimm *nvdimm = nd_mapping->nvdimm;
> +
> + if (test_bit(NDD_LABELING, &nvdimm->flags))
> + return a->mode;
> + }
> + return 0;
> + }
> +
> if (a != &dev_attr_set_cookie.attr
> && a != &dev_attr_available_size.attr)
> return a->mode;
> @@ -935,6 +1002,42 @@ void nd_region_release_lane(struct nd_region *nd_region, unsigned int lane)
> }
> EXPORT_SYMBOL(nd_region_release_lane);
>
> +/*
> + * PowerPC requires this alignment for memremap_pages(). All other archs
> + * should be ok with SUBSECTION_SIZE (see memremap_compat_align()).
> + */
> +#define MEMREMAP_COMPAT_ALIGN_MAX SZ_16M
> +
> +static unsigned long default_align(struct nd_region *nd_region)
> +{
> + unsigned long align, per_mapping;
> + int i, mappings;
> + u32 remainder;
> +
> + if (is_nd_blk(&nd_region->dev))
> + align = PAGE_SIZE;
> + else
> + align = MEMREMAP_COMPAT_ALIGN_MAX;
> +
> + for (i = 0; i < nd_region->ndr_mappings; i++) {
> + struct nd_mapping *nd_mapping = &nd_region->mapping[i];
> + struct nvdimm *nvdimm = nd_mapping->nvdimm;
> +
> + if (test_bit(NDD_ALIASING, &nvdimm->flags)) {
> + align = MEMREMAP_COMPAT_ALIGN_MAX;
> + break;
> + }
> + }
> +
> + mappings = max_t(u16, 1, nd_region->ndr_mappings);
> + per_mapping = align;
> + remainder = do_div(per_mapping, mappings);
> + if (remainder)
> + align *= mappings;
> +
> + return align;
> +}
> +
> static struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus,
> struct nd_region_desc *ndr_desc,
> const struct device_type *dev_type, const char *caller)
> @@ -1039,6 +1142,7 @@ static struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus,
> dev->of_node = ndr_desc->of_node;
> nd_region->ndr_size = resource_size(ndr_desc->res);
> nd_region->ndr_start = ndr_desc->res->start;
> + nd_region->align = default_align(nd_region);
> if (ndr_desc->flush)
> nd_region->flush = ndr_desc->flush;
> else
> _______________________________________________
> Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
> To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 2/5] mm/memremap_pages: Introduce memremap_compat_align()
2020-01-30 20:06 ` [PATCH 2/5] mm/memremap_pages: Introduce memremap_compat_align() Dan Williams
2020-02-03 17:09 ` Aneesh Kumar K.V
@ 2020-02-05 3:05 ` Michael Ellerman
2020-02-06 5:51 ` Dan Williams
1 sibling, 1 reply; 15+ messages in thread
From: Michael Ellerman @ 2020-02-05 3:05 UTC (permalink / raw)
To: Dan Williams, linux-nvdimm
Cc: Aneesh Kumar K.V, Benjamin Herrenschmidt, Paul Mackerras, hch,
linux-kernel, linuxppc-dev
Dan Williams <dan.j.williams@intel.com> writes:
> The "sub-section memory hotplug" facility allows memremap_pages() users
> like libnvdimm to compensate for hardware platforms like x86 that have a
> section size larger than their hardware memory mapping granularity. The
> compensation that sub-section support affords is being tolerant of
> physical memory resources shifting by units smaller (64MiB on x86) than
> the memory-hotplug section size (128 MiB). Where the platform
> physical-memory mapping granularity is limited by the number and
> capability of address-decode-registers in the memory controller.
>
> While the sub-section support allows memremap_pages() to operate on
> sub-section (2MiB) granularity, the Power architecture may still
> require 16MiB alignment on "!radix_enabled()" platforms.
>
> In order for libnvdimm to be able to detect and manage this per-arch
> limitation, introduce memremap_compat_align() as a common minimum
> alignment across all driver-facing memory-mapping interfaces, and let
> Power override it to 16MiB in the "!radix_enabled()" case.
>
> The assumption / requirement for 16MiB to be a viable
> memremap_compat_align() value is that Power does not have platforms
> where its equivalent of address-decode-registers never hardware remaps a
> persistent memory resource on smaller than 16MiB boundaries.
>
> Based on an initial patch by Aneesh.
>
> Link: http://lore.kernel.org/r/CAPcyv4gBGNP95APYaBcsocEa50tQj9b5h__83vgngjq3ouGX_Q@mail.gmail.com
> Reported-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Reported-by: Jeff Moyer <jmoyer@redhat.com>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> arch/powerpc/include/asm/io.h | 10 ++++++++++
> drivers/nvdimm/pfn_devs.c | 2 +-
> include/linux/io.h | 23 +++++++++++++++++++++++
> include/linux/mmzone.h | 1 +
> 4 files changed, 35 insertions(+), 1 deletion(-)
The powerpc change here looks fine to me.
Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc)
cheers
> diff --git a/arch/powerpc/include/asm/io.h b/arch/powerpc/include/asm/io.h
> index a63ec938636d..0fa2dc483008 100644
> --- a/arch/powerpc/include/asm/io.h
> +++ b/arch/powerpc/include/asm/io.h
> @@ -734,6 +734,16 @@ extern void __iomem * __ioremap_at(phys_addr_t pa, void *ea,
> unsigned long size, pgprot_t prot);
> extern void __iounmap_at(void *ea, unsigned long size);
>
> +#ifdef CONFIG_SPARSEMEM
> +static inline unsigned long memremap_compat_align(void)
> +{
> + if (radix_enabled())
> + return SUBSECTION_SIZE;
> + return (1UL << mmu_psize_defs[mmu_linear_psize].shift);
> +}
> +#define memremap_compat_align memremap_compat_align
> +#endif
> +
> /*
> * When CONFIG_PPC_INDIRECT_PIO is set, we use the generic iomap implementation
> * which needs some additional definitions here. They basically allow PIO
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 2/5] mm/memremap_pages: Introduce memremap_compat_align()
2020-02-05 3:05 ` Michael Ellerman
@ 2020-02-06 5:51 ` Dan Williams
2020-02-06 6:21 ` Aneesh Kumar K.V
0 siblings, 1 reply; 15+ messages in thread
From: Dan Williams @ 2020-02-06 5:51 UTC (permalink / raw)
To: Michael Ellerman
Cc: linux-nvdimm, Aneesh Kumar K.V, Benjamin Herrenschmidt,
Paul Mackerras, Christoph Hellwig, Linux Kernel Mailing List,
linuxppc-dev
On Tue, Feb 4, 2020 at 7:05 PM Michael Ellerman <mpe@ellerman.id.au> wrote:
>
> Dan Williams <dan.j.williams@intel.com> writes:
> > The "sub-section memory hotplug" facility allows memremap_pages() users
> > like libnvdimm to compensate for hardware platforms like x86 that have a
> > section size larger than their hardware memory mapping granularity. The
> > compensation that sub-section support affords is being tolerant of
> > physical memory resources shifting by units smaller (64MiB on x86) than
> > the memory-hotplug section size (128 MiB). Where the platform
> > physical-memory mapping granularity is limited by the number and
> > capability of address-decode-registers in the memory controller.
> >
> > While the sub-section support allows memremap_pages() to operate on
> > sub-section (2MiB) granularity, the Power architecture may still
> > require 16MiB alignment on "!radix_enabled()" platforms.
> >
> > In order for libnvdimm to be able to detect and manage this per-arch
> > limitation, introduce memremap_compat_align() as a common minimum
> > alignment across all driver-facing memory-mapping interfaces, and let
> > Power override it to 16MiB in the "!radix_enabled()" case.
> >
> > The assumption / requirement for 16MiB to be a viable
> > memremap_compat_align() value is that Power does not have platforms
> > where its equivalent of address-decode-registers never hardware remaps a
> > persistent memory resource on smaller than 16MiB boundaries.
> >
> > Based on an initial patch by Aneesh.
> >
> > Link: http://lore.kernel.org/r/CAPcyv4gBGNP95APYaBcsocEa50tQj9b5h__83vgngjq3ouGX_Q@mail.gmail.com
> > Reported-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> > Reported-by: Jeff Moyer <jmoyer@redhat.com>
> > Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> > Cc: Paul Mackerras <paulus@samba.org>
> > Cc: Michael Ellerman <mpe@ellerman.id.au>
> > Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> > ---
> > arch/powerpc/include/asm/io.h | 10 ++++++++++
> > drivers/nvdimm/pfn_devs.c | 2 +-
> > include/linux/io.h | 23 +++++++++++++++++++++++
> > include/linux/mmzone.h | 1 +
> > 4 files changed, 35 insertions(+), 1 deletion(-)
>
> The powerpc change here looks fine to me.
>
> Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc)
Thanks Michael, unfortunately the kbuild robot just woke up and said
that mips does not like including mmzone.h from io.h. The
entanglements look intractable.
Is there a file I can stash a strong definition of
memremap_compat_align(), maybe arch/powerpc/mm/mem.c? Then I can put a
generic __weak definition in mm/memremap.c rather than play header
file include games.
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 2/5] mm/memremap_pages: Introduce memremap_compat_align()
2020-02-06 5:51 ` Dan Williams
@ 2020-02-06 6:21 ` Aneesh Kumar K.V
0 siblings, 0 replies; 15+ messages in thread
From: Aneesh Kumar K.V @ 2020-02-06 6:21 UTC (permalink / raw)
To: Dan Williams, Michael Ellerman
Cc: linux-nvdimm, Benjamin Herrenschmidt, Paul Mackerras,
Christoph Hellwig, Linux Kernel Mailing List, linuxppc-dev
On 2/6/20 11:21 AM, Dan Williams wrote:
....
>>>
>>> Link: http://lore.kernel.org/r/CAPcyv4gBGNP95APYaBcsocEa50tQj9b5h__83vgngjq3ouGX_Q@mail.gmail.com
>>> Reported-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
>>> Reported-by: Jeff Moyer <jmoyer@redhat.com>
>>> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>>> Cc: Paul Mackerras <paulus@samba.org>
>>> Cc: Michael Ellerman <mpe@ellerman.id.au>
>>> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>>> ---
>>> arch/powerpc/include/asm/io.h | 10 ++++++++++
>>> drivers/nvdimm/pfn_devs.c | 2 +-
>>> include/linux/io.h | 23 +++++++++++++++++++++++
>>> include/linux/mmzone.h | 1 +
>>> 4 files changed, 35 insertions(+), 1 deletion(-)
>>
>> The powerpc change here looks fine to me.
>>
>> Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc)
>
> Thanks Michael, unfortunately the kbuild robot just woke up and said
> that mips does not like including mmzone.h from io.h. The
> entanglements look intractable.
>
> Is there a file I can stash a strong definition of
> memremap_compat_align(), maybe arch/powerpc/mm/mem.c? Then I can put a
> generic __weak definition in mm/memremap.c rather than play header
> file include games.
>
arch/powerpc/mm/ioremap.c ?
-aneesh
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2020-02-06 6:21 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-30 20:05 [PATCH 0/5] libnvdimm: Cross-arch compatible namespace alignment Dan Williams
2020-01-30 20:06 ` [PATCH 1/5] mm/memremap_pages: Kill unused __devm_memremap_pages() Dan Williams
2020-01-31 5:32 ` Christoph Hellwig
2020-02-03 17:08 ` Aneesh Kumar K.V
2020-01-30 20:06 ` [PATCH 2/5] mm/memremap_pages: Introduce memremap_compat_align() Dan Williams
2020-02-03 17:09 ` Aneesh Kumar K.V
2020-02-05 3:05 ` Michael Ellerman
2020-02-06 5:51 ` Dan Williams
2020-02-06 6:21 ` Aneesh Kumar K.V
2020-01-30 20:06 ` [PATCH 3/5] libnvdimm/namespace: Enforce memremap_compat_align() Dan Williams
2020-02-03 17:09 ` Aneesh Kumar K.V
2020-01-30 20:06 ` [PATCH 4/5] libnvdimm/region: Introduce NDD_LABELING Dan Williams
2020-02-03 17:09 ` Aneesh Kumar K.V
2020-01-30 20:06 ` [PATCH 5/5] libnvdimm/region: Introduce an 'align' attribute Dan Williams
2020-02-03 17:10 ` Aneesh Kumar K.V
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).