[PATCH v3 0/3] mtd: write support for minor-aligned partitions

[PATCH v3 0/3] mtd: write support for minor-aligned partitions

[Thibaut VARÈNE]

This patch series enable write support for minor-aligned mtd partitions
on devices that support multiple erasesizes.

On supporting hardware, they enable RW operations on mtd partitions that
break "major" (largest) erasesize boundary (e.g. too small or not
correctly aligned) without sacrificing performance (i.e. forcing
everything to use 4K sectors). Finally, they address an historical FIXME
in mtdpart.c.

The following performance numbers were measured with v5.10 kernel on OpenWrt:
(CONFIG_MTD_SPI_NOR_USE_VARIABLE_ERASE=y forces the use of the new code path)

OpenWrt 5.10 ramips MT7621 w25q128jv 0xfc0000 firmware partition length
time mtd erase firmware

Without this patch:
MTD_SPI_NOR_USE_4K_SECTORS=n	|y
real    0m 50.86s		|2m 11.66s
user    0m 0.00s		|0m 0.00s
sys     0m 50.80s		|1m 56.20s
can write minor-aligned parts? N|Y

With this patch:
MTD_SPI_NOR_USE_VARIABLE_ERASE=n|y		|4K_SECTORS=y
real    0m 51.68s		|0m 50.85s	|2m 12.89s
user    0m 0.00s		|0m 0.00s	|0m 0.01s
sys     0m 46.94s		|0m 50.38s	|2m 12.46s
can write minor-aligned parts? N|Y		|Y


NB: This patchset was previously submitted as a monolithic patch (RFC[0],
v1[1]). While not submitted upstream v2 was however reviewed, tested and
accepted in OpenWrt, and backported into the upcoming 22.03 stable branch:
https://github.com/openwrt/openwrt/pull/3271

[0] 20210104122853.18428-1-git@johnthomson.fastmail.com.au
[1] 20210608040719.14431-1-git@johnthomson.fastmail.com.au

Changes v2 -> v3:
  - Split patch into incremental, functionally-separate changes.
  - Adjusted erasesize_minor assignment check in spi-nor/core.c
  - Addressed checkpatch warning on "printk(KERN_WARNING..."
  - Documented new sysfs attribute

Changes v1 -> v2:
  - Added mtdcore sysfs attribute for erasesize_minor
  - Removed finding minor erasesize for variable erase regions device,
    (untested and no response to RFC).
  - Moved IF_ENABLED() to guard setting erasesize_minor in spi-nor/core.c
  - Removed setting major erasesize to minor where partition boundaries
    require minor erase to be writable
  - Simplified minor boundary check by relying on minor being a factor
    of major

Changes RFC -> v1:
  - Fixed uninitialized variable smatch warning

Thibaut VARÈNE (3):
  mtd: mtdpart: write support for minor-aligned partitions
  mtd: spi-nor: set erasesize_minor in spi_nor_select_erase()
  mtd: spi-nor: allow overriding uniform erase

 Documentation/ABI/testing/sysfs-class-mtd |  8 ++++++
 drivers/mtd/mtdcore.c                     | 10 +++++++
 drivers/mtd/mtdpart.c                     | 35 ++++++++++++++++-------
 drivers/mtd/spi-nor/Kconfig               | 10 +++++++
 drivers/mtd/spi-nor/core.c                | 13 +++++++--
 include/linux/mtd/mtd.h                   |  2 ++
 6 files changed, 65 insertions(+), 13 deletions(-)


base-commit: e0dccc3b76fb35bb257b4118367a883073d7390e
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 1/3] mtd: mtdpart: write support for minor-aligned partitions

[Thibaut VARÈNE]

This patch enables writing to mtd partitions where a partition boundary
sits on a "minor" erasesize boundary.

This patch adds a uint32_t `erasesize_minor` to struct mtd_info: the
smallest erasesize supported by the device, also exposed in sysfs.

This patch is a no-op if erasesize_minor is unset (0) and has no
userspace-visible side effect (in particular the reported erasesize in
/proc/mtd is unchanged). This new feature is only enabled for single
eraseregion devices, where it makes most sense.

This patch addresses an outstanding mtdpart.c FIXME that has been
present since the start of the linux git history.

Signed-off-by: John Thomson <git@johnthomson.fastmail.com.au>
Signed-off-by: Thibaut VARÈNE <hacks+kernel@slashdirt.org>
---
 Documentation/ABI/testing/sysfs-class-mtd |  8 ++++++
 drivers/mtd/mtdcore.c                     | 10 +++++++
 drivers/mtd/mtdpart.c                     | 35 ++++++++++++++++-------
 include/linux/mtd/mtd.h                   |  2 ++
 4 files changed, 45 insertions(+), 10 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-class-mtd b/Documentation/ABI/testing/sysfs-class-mtd
index 3bc7c0a95..786068f63 100644
--- a/Documentation/ABI/testing/sysfs-class-mtd
+++ b/Documentation/ABI/testing/sysfs-class-mtd
@@ -240,3 +240,11 @@ Contact:	linux-mtd@lists.infradead.org
 Description:
 		Number of bytes available for a client to place data into
 		the out of band area.
+
+What:		/sys/class/mtd/mtdX/erasesize_minor
+Date:		July 2022
+KernelVersion:	5.20
+Contact:	linux-mtd@lists.infradead.org
+Description:
+		"Minor" erase size for the device. If supported, this is
+		the smallest eraseblock size for the device.
diff --git a/drivers/mtd/mtdcore.c b/drivers/mtd/mtdcore.c
index 9eb0680db..ccb80197d 100644
--- a/drivers/mtd/mtdcore.c
+++ b/drivers/mtd/mtdcore.c
@@ -168,6 +168,15 @@ static ssize_t mtd_erasesize_show(struct device *dev,
 }
 MTD_DEVICE_ATTR_RO(erasesize);
 
+static ssize_t mtd_erasesize_minor_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct mtd_info *mtd = dev_get_drvdata(dev);
+
+	return sysfs_emit(buf, "%lu\n", (unsigned long)mtd->erasesize_minor);
+}
+MTD_DEVICE_ATTR_RO(erasesize_minor);
+
 static ssize_t mtd_writesize_show(struct device *dev,
 		struct device_attribute *attr, char *buf)
 {
@@ -313,6 +322,7 @@ static struct attribute *mtd_attrs[] = {
 	&dev_attr_flags.attr,
 	&dev_attr_size.attr,
 	&dev_attr_erasesize.attr,
+	&dev_attr_erasesize_minor.attr,
 	&dev_attr_writesize.attr,
 	&dev_attr_subpagesize.attr,
 	&dev_attr_oobsize.attr,
diff --git a/drivers/mtd/mtdpart.c b/drivers/mtd/mtdpart.c
index d442fa94c..6bf21567f 100644
--- a/drivers/mtd/mtdpart.c
+++ b/drivers/mtd/mtdpart.c
@@ -39,6 +39,7 @@ static struct mtd_info *allocate_partition(struct mtd_info *parent,
 	struct mtd_info *master = mtd_get_master(parent);
 	int wr_alignment = (parent->flags & MTD_NO_ERASE) ?
 			   master->writesize : master->erasesize;
+	int wr_alignment_minor = 0;
 	u64 parent_size = mtd_is_partition(parent) ?
 			  parent->part.size : parent->size;
 	struct mtd_info *child;
@@ -163,6 +164,7 @@ static struct mtd_info *allocate_partition(struct mtd_info *parent,
 	} else {
 		/* Single erase size */
 		child->erasesize = master->erasesize;
+		child->erasesize_minor = master->erasesize_minor;
 	}
 
 	/*
@@ -170,26 +172,39 @@ static struct mtd_info *allocate_partition(struct mtd_info *parent,
 	 * exposes several regions with different erasesize. Adjust
 	 * wr_alignment accordingly.
 	 */
-	if (!(child->flags & MTD_NO_ERASE))
+	if (!(child->flags & MTD_NO_ERASE)) {
 		wr_alignment = child->erasesize;
+		wr_alignment_minor = child->erasesize_minor;
+	}
 
 	tmp = mtd_get_master_ofs(child, 0);
 	remainder = do_div(tmp, wr_alignment);
 	if ((child->flags & MTD_WRITEABLE) && remainder) {
-		/* Doesn't start on a boundary of major erase size */
-		/* FIXME: Let it be writable if it is on a boundary of
-		 * _minor_ erase size though */
-		child->flags &= ~MTD_WRITEABLE;
-		printk(KERN_WARNING"mtd: partition \"%s\" doesn't start on an erase/write block boundary -- force read-only\n",
-			part->name);
+		if (wr_alignment_minor) {
+			/* rely on minor being a factor of major erasesize */
+			tmp = remainder;
+			remainder = do_div(tmp, wr_alignment_minor);
+		}
+		if (remainder) {
+			child->flags &= ~MTD_WRITEABLE;
+			pr_warn("mtd: partition \"%s\" doesn't start on an erase/write block boundary -- force read-only\n",
+				part->name);
+		}
 	}
 
 	tmp = mtd_get_master_ofs(child, 0) + child->part.size;
 	remainder = do_div(tmp, wr_alignment);
 	if ((child->flags & MTD_WRITEABLE) && remainder) {
-		child->flags &= ~MTD_WRITEABLE;
-		printk(KERN_WARNING"mtd: partition \"%s\" doesn't end on an erase/write block -- force read-only\n",
-			part->name);
+		if (wr_alignment_minor) {
+			tmp = remainder;
+			remainder = do_div(tmp, wr_alignment_minor);
+		}
+
+		if (remainder) {
+			child->flags &= ~MTD_WRITEABLE;
+			pr_warn("mtd: partition \"%s\" doesn't end on an erase/write block boundary -- force read-only\n",
+				part->name);
+		}
 	}
 
 	child->size = child->part.size;
diff --git a/include/linux/mtd/mtd.h b/include/linux/mtd/mtd.h
index 955aee14b..8efbc929e 100644
--- a/include/linux/mtd/mtd.h
+++ b/include/linux/mtd/mtd.h
@@ -238,6 +238,8 @@ struct mtd_info {
 	 * information below if they desire
 	 */
 	uint32_t erasesize;
+	/* "Minor" (smallest) erase size supported by the whole device */
+	uint32_t erasesize_minor;
 	/* Minimal writable flash unit size. In case of NOR flash it is 1 (even
 	 * though individual bits can be cleared), in case of NAND flash it is
 	 * one NAND page (or half, or one-fourths of it), in case of ECC-ed NOR
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 2/3] mtd: spi-nor: set erasesize_minor for non-uniform erase

[Thibaut VARÈNE]

This patch sets erasesize_minor on SPI-NOR devices supporting
non-uniform erase, allowing write access to partitions that sit on
"minor" boundary. It preserves backward compatibility with existing
uniform behavior.

Signed-off-by: John Thomson <git@johnthomson.fastmail.com.au>
Signed-off-by: Thibaut VARÈNE <hacks+kernel@slashdirt.org>
---
 drivers/mtd/spi-nor/core.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
index 502967c76..bafd3396e 100644
--- a/drivers/mtd/spi-nor/core.c
+++ b/drivers/mtd/spi-nor/core.c
@@ -2144,6 +2144,7 @@ static int spi_nor_select_erase(struct spi_nor *nor)
 {
 	struct spi_nor_erase_map *map = &nor->params->erase_map;
 	const struct spi_nor_erase_type *erase = NULL;
+	const struct spi_nor_erase_type *erase_minor = NULL;
 	struct mtd_info *mtd = &nor->mtd;
 	u32 wanted_size = nor->info->sector_size;
 	int i;
@@ -2172,12 +2173,14 @@ static int spi_nor_select_erase(struct spi_nor *nor)
 
 	/*
 	 * For non-uniform SPI flash memory, set mtd->erasesize to the
-	 * maximum erase sector size. No need to set nor->erase_opcode.
+	 * maximum erase sector size and mtd->erasesize_minor to the minimum.
+	 * No need to set nor->erase_opcode.
 	 */
 	for (i = SNOR_ERASE_TYPE_MAX - 1; i >= 0; i--) {
 		if (map->erase_type[i].size) {
-			erase = &map->erase_type[i];
-			break;
+			if (!erase)
+				erase = &map->erase_type[i];
+			erase_minor = &map->erase_type[i];
 		}
 	}
 
@@ -2185,6 +2188,8 @@ static int spi_nor_select_erase(struct spi_nor *nor)
 		return -EINVAL;
 
 	mtd->erasesize = erase->size;
+	if (erase_minor->size < erase->size)
+		mtd->erasesize_minor = erase_minor->size;
 	return 0;
 }
 
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH v3 3/3] mtd: spi-nor: allow overriding uniform erase

[Thibaut VARÈNE]

This patch introduces a new CONFIG_MTD_SPI_NOR_USE_VARIABLE_ERASE option
which forces an SPI NOR mtd device to use non-uniform erase, enabling
(on supporting hardware) to complete erase operations using all
available erasesizes through spi_nor_erase_multi_sectors().

The majority of current SPI NOR devices support multiple erasesizes:
instead of forcing all operations to use 4K erase size for everything
(using CONFIG_MTD_SPI_NOR_USE_4K_SECTORS) when write access to a
minor-aligned partition is needed, this patch allows the kernel to use
the largest suitable erase size for a given operation, thus improving
performance.

Signed-off-by: John Thomson <git@johnthomson.fastmail.com.au>
Signed-off-by: Thibaut VARÈNE <hacks+kernel@slashdirt.org>
---
 drivers/mtd/spi-nor/Kconfig | 10 ++++++++++
 drivers/mtd/spi-nor/core.c  |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/drivers/mtd/spi-nor/Kconfig b/drivers/mtd/spi-nor/Kconfig
index 24cd25de2..09df9f1a8 100644
--- a/drivers/mtd/spi-nor/Kconfig
+++ b/drivers/mtd/spi-nor/Kconfig
@@ -10,6 +10,16 @@ menuconfig MTD_SPI_NOR
 
 if MTD_SPI_NOR
 
+config MTD_SPI_NOR_USE_VARIABLE_ERASE
+	bool "Disable uniform_erase to allow use of all hardware supported erasesizes"
+	depends on !MTD_SPI_NOR_USE_4K_SECTORS
+	default n
+	help
+	  Allow mixed use of all hardware supported erasesizes,
+	  by forcing spi_nor to use the multiple eraseregions code path.
+	  For example: A 68K erase will use one 64K erase, and one 4K erase
+	  on supporting hardware.
+
 config MTD_SPI_NOR_USE_4K_SECTORS
 	bool "Use small 4096 B erase sectors"
 	default y
diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
index bafd3396e..b4b539a9f 100644
--- a/drivers/mtd/spi-nor/core.c
+++ b/drivers/mtd/spi-nor/core.c
@@ -1048,6 +1048,8 @@ static u8 spi_nor_convert_3to4_erase(u8 opcode)
 
 static bool spi_nor_has_uniform_erase(const struct spi_nor *nor)
 {
+	if (IS_ENABLED(CONFIG_MTD_SPI_NOR_USE_VARIABLE_ERASE))
+		return false;
 	return !!nor->params->erase_map.uniform_erase_type;
 }
 
-- 
2.30.2


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 0/3] mtd: write support for minor-aligned partitions

[Thibaut VARÈNE]

Hi,

Since there were no objections in the past week, could you please consider merging this patch series?

Thanks,
Thibaut

> Le 29 juil. 2022 à 11:16, Thibaut VARÈNE <hacks+kernel@slashdirt.org> a écrit :
> 
> This patch series enable write support for minor-aligned mtd partitions
> on devices that support multiple erasesizes.
> 
> On supporting hardware, they enable RW operations on mtd partitions that
> break "major" (largest) erasesize boundary (e.g. too small or not
> correctly aligned) without sacrificing performance (i.e. forcing
> everything to use 4K sectors). Finally, they address an historical FIXME
> in mtdpart.c.
> 
> The following performance numbers were measured with v5.10 kernel on OpenWrt:
> (CONFIG_MTD_SPI_NOR_USE_VARIABLE_ERASE=y forces the use of the new code path)
> 
> OpenWrt 5.10 ramips MT7621 w25q128jv 0xfc0000 firmware partition length
> time mtd erase firmware
> 
> Without this patch:
> MTD_SPI_NOR_USE_4K_SECTORS=n	|y
> real    0m 50.86s		|2m 11.66s
> user    0m 0.00s		|0m 0.00s
> sys     0m 50.80s		|1m 56.20s
> can write minor-aligned parts? N|Y
> 
> With this patch:
> MTD_SPI_NOR_USE_VARIABLE_ERASE=n|y		|4K_SECTORS=y
> real    0m 51.68s		|0m 50.85s	|2m 12.89s
> user    0m 0.00s		|0m 0.00s	|0m 0.01s
> sys     0m 46.94s		|0m 50.38s	|2m 12.46s
> can write minor-aligned parts? N|Y		|Y
> 
> 
> NB: This patchset was previously submitted as a monolithic patch (RFC[0],
> v1[1]). While not submitted upstream v2 was however reviewed, tested and
> accepted in OpenWrt, and backported into the upcoming 22.03 stable branch:
> https://github.com/openwrt/openwrt/pull/3271
> 
> [0] 20210104122853.18428-1-git@johnthomson.fastmail.com.au
> [1] 20210608040719.14431-1-git@johnthomson.fastmail.com.au
> 
> Changes v2 -> v3:
>  - Split patch into incremental, functionally-separate changes.
>  - Adjusted erasesize_minor assignment check in spi-nor/core.c
>  - Addressed checkpatch warning on "printk(KERN_WARNING..."
>  - Documented new sysfs attribute
> 
> Changes v1 -> v2:
>  - Added mtdcore sysfs attribute for erasesize_minor
>  - Removed finding minor erasesize for variable erase regions device,
>    (untested and no response to RFC).
>  - Moved IF_ENABLED() to guard setting erasesize_minor in spi-nor/core.c
>  - Removed setting major erasesize to minor where partition boundaries
>    require minor erase to be writable
>  - Simplified minor boundary check by relying on minor being a factor
>    of major
> 
> Changes RFC -> v1:
>  - Fixed uninitialized variable smatch warning
> 
> Thibaut VARÈNE (3):
>  mtd: mtdpart: write support for minor-aligned partitions
>  mtd: spi-nor: set erasesize_minor in spi_nor_select_erase()
>  mtd: spi-nor: allow overriding uniform erase
> 
> Documentation/ABI/testing/sysfs-class-mtd |  8 ++++++
> drivers/mtd/mtdcore.c                     | 10 +++++++
> drivers/mtd/mtdpart.c                     | 35 ++++++++++++++++-------
> drivers/mtd/spi-nor/Kconfig               | 10 +++++++
> drivers/mtd/spi-nor/core.c                | 13 +++++++--
> include/linux/mtd/mtd.h                   |  2 ++
> 6 files changed, 65 insertions(+), 13 deletions(-)
> 
> 
> base-commit: e0dccc3b76fb35bb257b4118367a883073d7390e
> -- 
> 2.30.2
> 


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v3 0/3] mtd: write support for minor-aligned partitions

[Tudor.Ambarus]

On 8/9/22 12:23, Thibaut VARÈNE wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> Hi,
> 
> Since there were no objections in the past week, could you please consider merging this patch series?
> 

Every patch submitted is considered for merging. Please be patient
and allow us time to review it. You can verify the status of any
SPI NOR patch at:
https://patchwork.ozlabs.org/project/linux-mtd/list/?series=&submitter=&state=&q=spi-nor&archive=&delegate=
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/