linux-mtd.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 0/9] mtd: spi-nor: read while write support
@ 2022-11-10 15:55 Miquel Raynal
  2022-11-10 15:55 ` [PATCH v2 1/9] mtd: spi-nor: Create macros to define chip IDs and geometries Miquel Raynal
                   ` (9 more replies)
  0 siblings, 10 replies; 17+ messages in thread
From: Miquel Raynal @ 2022-11-10 15:55 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni, Miquel Raynal

Hello folks,

Here is the follow-up of the RFC trying to bring a little bit of
parallelism to support SPI-NOR Read While Write feature on parts
supporting it and featuring several banks.

I have received some hardware to make it work, so since the RFC, the
series has been updated to fix my mistakes, but the overall idea is the
same.

There is nothing Macronix specific in the implementation, the operations
and opcodes are exactly the same as before. The only difference being:
we may consider the chip usable when it is in the busy state during a
write or an erase. Any chip with an internal split allowing to perform
parallel operations might possibly leverage the benefits of this
implementation.

The first patches are just refactoring and preparation work, there is
almost no functional change, it's just a way to prepare the introduction
of the new locking mechanism and hopefully provide the cleanest and
simplest diff possible for this new feature. The actual change is all
contained in "mtd: spi-nor: Enhance locking to support reads while
writes". The logic is described in the commit log and copy/pasted here
for clarity:

"
The following constraints have to be taken into account:
1#: A single operation can be performed in a given bank.
2#: Only a single program or erase operation can happen on the entire chip.
3#: The I/O bus is unique and thus is the most constrained resource, all
    spi-nor operations requiring access to the spi bus (through the spi
    controller) must be serialized until the bus exchanges are over. So
    we must ensure a single operation can be "sent" at a time.
4#: Any other operation that would not be either a read or a write or an
    erase is considered requiring access to the full chip and cannot be
    parallelized, we then need to ensure the full chip is in the idle
    state when this occurs.

All these constraints can easily be managed with a proper locking model:
1#: Is protected by a per-bank mutex. Only a single operation can happen
    in a specific bank at any times. If the bank mutex is not available,
    the operation cannot start.
2#: Is handled by the pe_mode mutex which is acquired before any write
    or erase, and is released only at the very end of the
    operation. This way, no other destructive operation on the chip can
    start during this time frame.
3#: A device-wide mutex is introduced in order to capture and serialize
    bus accessed. This is the one being released "sooner" than before,
    because we only need to protect the chip against other SPI accesses
    during the I/O phase, which for the destructive operations is the
    beginning of the operation (when we send the command cycles and
    eventually the data), while the second part of the operation (the
    erase delay or the programmation delay) is when we can do something
    else with another bank.
4#: Is handled by the "generic" helpers which existed before, where
    basically all the locks are taken before the operation can start,
    and all locks are released once done.

As many devices still do not support this feature, the original lock is
also kept in a union: either the feature is available and we initialize
and use the new locks, or it is not and we keep using the previous
logic.
"

Here is now a benchmark with a Macronix MX25UW51245G with bank and RWW
support:

     // Testing the two accesses in the same bank
     $ flash_speed -b0 -k0 -c10 -d /dev/mtd0
     [...]
     testing read while write latency
     read while write took 51ms, read ended after 51ms

     // Testing the two accesses within different banks
     $ flash_speed -b0 -k4096 -c10 -d /dev/mtd0
     [...]
     testing read while write latency
     read while write took 51ms, read ended after 20ms

Here is a branch with the mtd-utils patch bringing support for this
additional "-k" parameter (for the second block to use during RWW
testing), used to get the above result:
https://github.com/miquelraynal/mtd-utils/compare/master...rww

Cheers,
Miquèl

Miquel Raynal (9):
  mtd: spi-nor: Create macros to define chip IDs and geometries
  mtd: spi-nor: Introduce the concept of bank
  mtd: spi-nor: Add a macro to define more banks
  mtd: spi-nor: Reorder the preparation vs locking steps
  mtd: spi-nor: Separate preparation and locking
  mtd: spi-nor: Prepare the introduction of a new locking mechanism
  mtd: spi-nor: Add a RWW flag
  mtd: spi-nor: Enhance locking to support reads while writes
  mtd: spi-nor: macronix: Add support for mx25uw51245g with RWW

 drivers/mtd/spi-nor/core.c     | 224 +++++++++++++++++++++++++++++----
 drivers/mtd/spi-nor/core.h     |  61 +++++----
 drivers/mtd/spi-nor/macronix.c |   3 +
 include/linux/mtd/spi-nor.h    |  12 +-
 4 files changed, 250 insertions(+), 50 deletions(-)

-- 
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH v2 1/9] mtd: spi-nor: Create macros to define chip IDs and geometries
  2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
@ 2022-11-10 15:55 ` Miquel Raynal
  2022-11-20 15:57   ` Pratyush Yadav
  2022-11-10 15:55 ` [PATCH v2 2/9] mtd: spi-nor: Introduce the concept of bank Miquel Raynal
                   ` (8 subsequent siblings)
  9 siblings, 1 reply; 17+ messages in thread
From: Miquel Raynal @ 2022-11-10 15:55 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni, Miquel Raynal

The INFO() macro defines an ID array and a couple of geometry
properties. Right now all its lines are duplicated twice because of the
INFO6() macro (for extended IDs) and soon as well we will need to add a
geometry parameter to include the number of banks.

In order to limit the code duplication, let's create a number of
intermediate macros which will facilitate defining high-level INFOX()
macros.

There is not functional change.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
---
 drivers/mtd/spi-nor/core.h | 43 ++++++++++++++++++--------------------
 1 file changed, 20 insertions(+), 23 deletions(-)

diff --git a/drivers/mtd/spi-nor/core.h b/drivers/mtd/spi-nor/core.h
index 85b0cf254e97..dc74c7be3e28 100644
--- a/drivers/mtd/spi-nor/core.h
+++ b/drivers/mtd/spi-nor/core.h
@@ -527,33 +527,30 @@ struct flash_info {
 	const struct spi_nor_fixups *fixups;
 };
 
+#define SPI_NOR_ID_2ITEMS(_id) ((_id) >> 8) & 0xff, (_id) & 0xff
+#define SPI_NOR_ID_3ITEMS(_id) ((_id) >> 16) & 0xff, SPI_NOR_ID_2ITEMS(_id)
+
+#define SPI_NOR_ID(_jedec_id, _ext_id)					\
+	.id = { SPI_NOR_ID_3ITEMS(_jedec_id), SPI_NOR_ID_2ITEMS(_ext_id) }, \
+	.id_len = !(_jedec_id) ? 0 : (3 + ((_ext_id) ? 2 : 0))
+
+#define SPI_NOR_ID6(_jedec_id, _ext_id)					\
+	.id = { SPI_NOR_ID_3ITEMS(_jedec_id), SPI_NOR_ID_3ITEMS(_ext_id) }, \
+	.id_len = 6
+
+#define SPI_NOR_GEOMETRY(_sector_size, _n_sectors)			\
+	.sector_size = (_sector_size),					\
+	.n_sectors = (_n_sectors),					\
+	.page_size = 256
+
 /* Used when the "_ext_id" is two bytes at most */
 #define INFO(_jedec_id, _ext_id, _sector_size, _n_sectors)		\
-		.id = {							\
-			((_jedec_id) >> 16) & 0xff,			\
-			((_jedec_id) >> 8) & 0xff,			\
-			(_jedec_id) & 0xff,				\
-			((_ext_id) >> 8) & 0xff,			\
-			(_ext_id) & 0xff,				\
-			},						\
-		.id_len = (!(_jedec_id) ? 0 : (3 + ((_ext_id) ? 2 : 0))),	\
-		.sector_size = (_sector_size),				\
-		.n_sectors = (_n_sectors),				\
-		.page_size = 256,					\
+	SPI_NOR_ID((_jedec_id), (_ext_id)),				\
+	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors)),
 
 #define INFO6(_jedec_id, _ext_id, _sector_size, _n_sectors)		\
-		.id = {							\
-			((_jedec_id) >> 16) & 0xff,			\
-			((_jedec_id) >> 8) & 0xff,			\
-			(_jedec_id) & 0xff,				\
-			((_ext_id) >> 16) & 0xff,			\
-			((_ext_id) >> 8) & 0xff,			\
-			(_ext_id) & 0xff,				\
-			},						\
-		.id_len = 6,						\
-		.sector_size = (_sector_size),				\
-		.n_sectors = (_n_sectors),				\
-		.page_size = 256,					\
+	SPI_NOR_ID6((_jedec_id), (_ext_id)),				\
+	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors)),
 
 #define CAT25_INFO(_sector_size, _n_sectors, _page_size, _addr_nbytes)	\
 		.sector_size = (_sector_size),				\
-- 
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v2 2/9] mtd: spi-nor: Introduce the concept of bank
  2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
  2022-11-10 15:55 ` [PATCH v2 1/9] mtd: spi-nor: Create macros to define chip IDs and geometries Miquel Raynal
@ 2022-11-10 15:55 ` Miquel Raynal
  2022-11-20 16:11   ` Pratyush Yadav
  2022-11-10 15:55 ` [PATCH v2 3/9] mtd: spi-nor: Add a macro to define more banks Miquel Raynal
                   ` (7 subsequent siblings)
  9 siblings, 1 reply; 17+ messages in thread
From: Miquel Raynal @ 2022-11-10 15:55 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni, Miquel Raynal

SPI-NOR chips are made of pages, which gathered in small groups make
(erase) sectors. Sectors, gathered together, make banks inside the
chip. So far there was only one bank per device supported, but we are
about to introduce support for new chips featuring several banks (up to
4 so far) where different operations may happen in parallel.

Let's allow describing these additional bank parameters.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
---
 drivers/mtd/spi-nor/core.c |  3 ++-
 drivers/mtd/spi-nor/core.h | 16 +++++++++++-----
 2 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
index f2c64006f8d7..38a57aac6754 100644
--- a/drivers/mtd/spi-nor/core.c
+++ b/drivers/mtd/spi-nor/core.c
@@ -2539,7 +2539,8 @@ static void spi_nor_init_default_params(struct spi_nor *nor)
 
 	/* Set SPI NOR sizes. */
 	params->writesize = 1;
-	params->size = (u64)info->sector_size * info->n_sectors;
+	params->bank_size = (u64)info->sector_size * info->n_sectors;
+	params->size = params->bank_size * info->n_banks;
 	params->page_size = info->page_size;
 
 	if (!(info->flags & SPI_NOR_NO_FR)) {
diff --git a/drivers/mtd/spi-nor/core.h b/drivers/mtd/spi-nor/core.h
index dc74c7be3e28..8a067d56c995 100644
--- a/drivers/mtd/spi-nor/core.h
+++ b/drivers/mtd/spi-nor/core.h
@@ -336,7 +336,8 @@ struct spi_nor_otp {
  * by the spi_nor_fixups hooks, or dynamically when parsing the JESD216
  * Serial Flash Discoverable Parameters (SFDP) tables.
  *
- * @size:		the flash memory density in bytes.
+ * @bank_size:		the flash memory bank density in bytes.
+ * @size:		the total flash memory density in bytes.
  * @writesize		Minimal writable flash unit size. Defaults to 1. Set to
  *			ECC unit size for ECC-ed flashes.
  * @page_size:		the page size of the SPI NOR flash memory.
@@ -374,6 +375,7 @@ struct spi_nor_otp {
  * @locking_ops:	SPI NOR locking methods.
  */
 struct spi_nor_flash_parameter {
+	u64				bank_size;
 	u64				size;
 	u32				writesize;
 	u32				page_size;
@@ -434,7 +436,8 @@ struct spi_nor_fixups {
  * @id_len:         the number of bytes of ID.
  * @sector_size:    the size listed here is what works with SPINOR_OP_SE, which
  *                  isn't necessarily called a "sector" by the vendor.
- * @n_sectors:      the number of sectors.
+ * @n_sectors:      the number of sectors per bank.
+ * @n_banks:        the number of banks.
  * @page_size:      the flash's page size.
  * @addr_nbytes:    number of address bytes to send.
  *
@@ -493,6 +496,7 @@ struct flash_info {
 	u8 id_len;
 	unsigned sector_size;
 	u16 n_sectors;
+	u16 n_banks;
 	u16 page_size;
 	u8 addr_nbytes;
 
@@ -538,23 +542,25 @@ struct flash_info {
 	.id = { SPI_NOR_ID_3ITEMS(_jedec_id), SPI_NOR_ID_3ITEMS(_ext_id) }, \
 	.id_len = 6
 
-#define SPI_NOR_GEOMETRY(_sector_size, _n_sectors)			\
+#define SPI_NOR_GEOMETRY(_sector_size, _n_sectors, _n_banks)		\
 	.sector_size = (_sector_size),					\
 	.n_sectors = (_n_sectors),					\
+	.n_banks = (_n_banks),						\
 	.page_size = 256
 
 /* Used when the "_ext_id" is two bytes at most */
 #define INFO(_jedec_id, _ext_id, _sector_size, _n_sectors)		\
 	SPI_NOR_ID((_jedec_id), (_ext_id)),				\
-	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors)),
+	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors), 1),
 
 #define INFO6(_jedec_id, _ext_id, _sector_size, _n_sectors)		\
 	SPI_NOR_ID6((_jedec_id), (_ext_id)),				\
-	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors)),
+	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors), 1),
 
 #define CAT25_INFO(_sector_size, _n_sectors, _page_size, _addr_nbytes)	\
 		.sector_size = (_sector_size),				\
 		.n_sectors = (_n_sectors),				\
+		.n_banks = 1,						\
 		.page_size = (_page_size),				\
 		.addr_nbytes = (_addr_nbytes),				\
 		.flags = SPI_NOR_NO_ERASE | SPI_NOR_NO_FR,		\
-- 
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v2 3/9] mtd: spi-nor: Add a macro to define more banks
  2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
  2022-11-10 15:55 ` [PATCH v2 1/9] mtd: spi-nor: Create macros to define chip IDs and geometries Miquel Raynal
  2022-11-10 15:55 ` [PATCH v2 2/9] mtd: spi-nor: Introduce the concept of bank Miquel Raynal
@ 2022-11-10 15:55 ` Miquel Raynal
  2022-11-20 16:13   ` Pratyush Yadav
  2022-11-10 15:55 ` [PATCH v2 4/9] mtd: spi-nor: Reorder the preparation vs locking steps Miquel Raynal
                   ` (6 subsequent siblings)
  9 siblings, 1 reply; 17+ messages in thread
From: Miquel Raynal @ 2022-11-10 15:55 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni, Miquel Raynal

Most of the chips on the market only feature a single bank. However, new
chips may support more than a single bank, with the possibility to
parallelize some operations. Let's introduce an INFOB() macro which also
takes a n_bank parameter.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
---
 drivers/mtd/spi-nor/core.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/mtd/spi-nor/core.h b/drivers/mtd/spi-nor/core.h
index 8a067d56c995..044d49d749e0 100644
--- a/drivers/mtd/spi-nor/core.h
+++ b/drivers/mtd/spi-nor/core.h
@@ -553,6 +553,10 @@ struct flash_info {
 	SPI_NOR_ID((_jedec_id), (_ext_id)),				\
 	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors), 1),
 
+#define INFOB(_jedec_id, _ext_id, _sector_size, _n_sectors, _n_banks)	\
+	SPI_NOR_ID((_jedec_id), (_ext_id)),				\
+	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors), (_n_banks)),
+
 #define INFO6(_jedec_id, _ext_id, _sector_size, _n_sectors)		\
 	SPI_NOR_ID6((_jedec_id), (_ext_id)),				\
 	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors), 1),
-- 
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v2 4/9] mtd: spi-nor: Reorder the preparation vs locking steps
  2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
                   ` (2 preceding siblings ...)
  2022-11-10 15:55 ` [PATCH v2 3/9] mtd: spi-nor: Add a macro to define more banks Miquel Raynal
@ 2022-11-10 15:55 ` Miquel Raynal
  2022-11-10 15:55 ` [PATCH v2 5/9] mtd: spi-nor: Separate preparation and locking Miquel Raynal
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 17+ messages in thread
From: Miquel Raynal @ 2022-11-10 15:55 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni, Miquel Raynal

The ->prepare()/->unprepare() hooks are now legacy, and there are only
two controllers left supporting them. In both cases, the implementation
acquires a mutex, which is somehow redundant with the spi-nor main lock
that we acquire as well in the spi_nor_[un]lock_and_[un]prep() helpers.

While the mutex taken in the core is necessary, the helper can be
reorganized to first do the preparation, then acquire the core
lock. This is necessary in order to be able to improve the locking
mechanism in the core and should have no side effect.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
---
 drivers/mtd/spi-nor/core.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
index 38a57aac6754..de77ca55f74d 100644
--- a/drivers/mtd/spi-nor/core.c
+++ b/drivers/mtd/spi-nor/core.c
@@ -1074,23 +1074,20 @@ int spi_nor_lock_and_prep(struct spi_nor *nor)
 {
 	int ret = 0;
 
-	mutex_lock(&nor->lock);
-
-	if (nor->controller_ops &&  nor->controller_ops->prepare) {
+	if (nor->controller_ops && nor->controller_ops->prepare)
 		ret = nor->controller_ops->prepare(nor);
-		if (ret) {
-			mutex_unlock(&nor->lock);
-			return ret;
-		}
-	}
+
+	mutex_lock(&nor->lock);
+
 	return ret;
 }
 
 void spi_nor_unlock_and_unprep(struct spi_nor *nor)
 {
+	mutex_unlock(&nor->lock);
+
 	if (nor->controller_ops && nor->controller_ops->unprepare)
 		nor->controller_ops->unprepare(nor);
-	mutex_unlock(&nor->lock);
 }
 
 static u32 spi_nor_convert_addr(struct spi_nor *nor, loff_t addr)
-- 
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v2 5/9] mtd: spi-nor: Separate preparation and locking
  2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
                   ` (3 preceding siblings ...)
  2022-11-10 15:55 ` [PATCH v2 4/9] mtd: spi-nor: Reorder the preparation vs locking steps Miquel Raynal
@ 2022-11-10 15:55 ` Miquel Raynal
  2022-11-10 15:55 ` [PATCH v2 6/9] mtd: spi-nor: Prepare the introduction of a new locking mechanism Miquel Raynal
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 17+ messages in thread
From: Miquel Raynal @ 2022-11-10 15:55 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni, Miquel Raynal

While this operation will remain a single function call in the end,
let's extract the logic of the [un]prepare calls within their own static
helper. We will soon add new flavors of the *_[un]lock_and_[un]prepare()
helper, having the preparation logic outside will save us from duplicating
code over and over again.

There is no functional change.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
---
 drivers/mtd/spi-nor/core.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
index de77ca55f74d..f87e57d97692 100644
--- a/drivers/mtd/spi-nor/core.c
+++ b/drivers/mtd/spi-nor/core.c
@@ -1070,24 +1070,40 @@ static void spi_nor_set_4byte_opcodes(struct spi_nor *nor)
 	}
 }
 
-int spi_nor_lock_and_prep(struct spi_nor *nor)
+static int spi_nor_prep(struct spi_nor *nor)
 {
 	int ret = 0;
 
 	if (nor->controller_ops && nor->controller_ops->prepare)
 		ret = nor->controller_ops->prepare(nor);
 
+	return ret;
+}
+
+static void spi_nor_unprep(struct spi_nor *nor)
+{
+	if (nor->controller_ops && nor->controller_ops->unprepare)
+		nor->controller_ops->unprepare(nor);
+}
+
+int spi_nor_lock_and_prep(struct spi_nor *nor)
+{
+	int ret;
+
+	ret = spi_nor_prep(nor);
+	if (ret)
+		return ret;
+
 	mutex_lock(&nor->lock);
 
-	return ret;
+	return 0;
 }
 
 void spi_nor_unlock_and_unprep(struct spi_nor *nor)
 {
 	mutex_unlock(&nor->lock);
 
-	if (nor->controller_ops && nor->controller_ops->unprepare)
-		nor->controller_ops->unprepare(nor);
+	spi_nor_unprep(nor);
 }
 
 static u32 spi_nor_convert_addr(struct spi_nor *nor, loff_t addr)
-- 
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v2 6/9] mtd: spi-nor: Prepare the introduction of a new locking mechanism
  2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
                   ` (4 preceding siblings ...)
  2022-11-10 15:55 ` [PATCH v2 5/9] mtd: spi-nor: Separate preparation and locking Miquel Raynal
@ 2022-11-10 15:55 ` Miquel Raynal
  2022-11-10 15:55 ` [PATCH v2 7/9] mtd: spi-nor: Add a RWW flag Miquel Raynal
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 17+ messages in thread
From: Miquel Raynal @ 2022-11-10 15:55 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni, Miquel Raynal

This commit alone just introduces two new "lock and prepare" pairs of
helpers which do the exact same thing as before. They will soon be
improved in a followup commit which actually brings the logic, but I
figured out it was more readable to do it this way.

One new pair is suffixed _pe which stands for "program and erase" and
hence is being called by spi_nor_write() and spi_nor_erase().

The other pair is suffixed _rd which stands for "read" and hence is
being called by spi_nor_read().

One note however, these extra helpers will need to know the operation
range, so they come with two new parameters to define it. Otherwise
there is no functional change.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
---
 drivers/mtd/spi-nor/core.c | 57 ++++++++++++++++++++++++++++++++++----
 1 file changed, 51 insertions(+), 6 deletions(-)

diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
index f87e57d97692..dfda350d5056 100644
--- a/drivers/mtd/spi-nor/core.c
+++ b/drivers/mtd/spi-nor/core.c
@@ -1086,6 +1086,7 @@ static void spi_nor_unprep(struct spi_nor *nor)
 		nor->controller_ops->unprepare(nor);
 }
 
+/* Generic helpers for internal locking and serialization */
 int spi_nor_lock_and_prep(struct spi_nor *nor)
 {
 	int ret;
@@ -1106,6 +1107,48 @@ void spi_nor_unlock_and_unprep(struct spi_nor *nor)
 	spi_nor_unprep(nor);
 }
 
+/* Internal locking helpers for program and erase operations */
+static int spi_nor_lock_and_prep_pe(struct spi_nor *nor, loff_t start, size_t len)
+{
+	int ret;
+
+	ret = spi_nor_prep(nor);
+	if (ret)
+		return ret;
+
+	mutex_lock(&nor->lock);
+
+	return 0;
+}
+
+static void spi_nor_unlock_and_unprep_pe(struct spi_nor *nor, loff_t start, size_t len)
+{
+	mutex_unlock(&nor->lock);
+
+	spi_nor_unprep(nor);
+}
+
+/* Internal locking helpers for read operations */
+static int spi_nor_lock_and_prep_rd(struct spi_nor *nor, loff_t start, size_t len)
+{
+	int ret;
+
+	ret = spi_nor_prep(nor);
+	if (ret)
+		return ret;
+
+	mutex_lock(&nor->lock);
+
+	return 0;
+}
+
+static void spi_nor_unlock_and_unprep_rd(struct spi_nor *nor, loff_t start, size_t len)
+{
+	mutex_unlock(&nor->lock);
+
+	spi_nor_unprep(nor);
+}
+
 static u32 spi_nor_convert_addr(struct spi_nor *nor, loff_t addr)
 {
 	if (!nor->params->convert_addr)
@@ -1457,7 +1500,7 @@ static int spi_nor_erase(struct mtd_info *mtd, struct erase_info *instr)
 	addr = instr->addr;
 	len = instr->len;
 
-	ret = spi_nor_lock_and_prep(nor);
+	ret = spi_nor_lock_and_prep_pe(nor, addr, len);
 	if (ret)
 		return ret;
 
@@ -1520,7 +1563,7 @@ static int spi_nor_erase(struct mtd_info *mtd, struct erase_info *instr)
 	ret = spi_nor_write_disable(nor);
 
 erase_err:
-	spi_nor_unlock_and_unprep(nor);
+	spi_nor_unlock_and_unprep_pe(nor, addr, len);
 
 	return ret;
 }
@@ -1694,7 +1737,7 @@ static int spi_nor_read(struct mtd_info *mtd, loff_t from, size_t len,
 
 	dev_dbg(nor->dev, "from 0x%08x, len %zd\n", (u32)from, len);
 
-	ret = spi_nor_lock_and_prep(nor);
+	ret = spi_nor_lock_and_prep_rd(nor, from, len);
 	if (ret)
 		return ret;
 
@@ -1721,7 +1764,8 @@ static int spi_nor_read(struct mtd_info *mtd, loff_t from, size_t len,
 	ret = 0;
 
 read_err:
-	spi_nor_unlock_and_unprep(nor);
+	spi_nor_unlock_and_unprep_rd(nor, from, len);
+
 	return ret;
 }
 
@@ -1740,7 +1784,7 @@ static int spi_nor_write(struct mtd_info *mtd, loff_t to, size_t len,
 
 	dev_dbg(nor->dev, "to 0x%08x, len %zd\n", (u32)to, len);
 
-	ret = spi_nor_lock_and_prep(nor);
+	ret = spi_nor_lock_and_prep_pe(nor, to, len);
 	if (ret)
 		return ret;
 
@@ -1782,7 +1826,8 @@ static int spi_nor_write(struct mtd_info *mtd, loff_t to, size_t len,
 	}
 
 write_err:
-	spi_nor_unlock_and_unprep(nor);
+	spi_nor_unlock_and_unprep_pe(nor, to, len);
+
 	return ret;
 }
 
-- 
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v2 7/9] mtd: spi-nor: Add a RWW flag
  2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
                   ` (5 preceding siblings ...)
  2022-11-10 15:55 ` [PATCH v2 6/9] mtd: spi-nor: Prepare the introduction of a new locking mechanism Miquel Raynal
@ 2022-11-10 15:55 ` Miquel Raynal
  2022-11-10 15:55 ` [PATCH v2 8/9] mtd: spi-nor: Enhance locking to support reads while writes Miquel Raynal
                   ` (2 subsequent siblings)
  9 siblings, 0 replies; 17+ messages in thread
From: Miquel Raynal @ 2022-11-10 15:55 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni, Miquel Raynal

Introduce a new (no SFDP) flag for the feature that we are about to
support: Read While Write. This means, if the chip has several banks and
supports RWW, once a page of data to write has been transferred into the
chip's internal SRAM, another read operation happening on a different
bank can be performed during the tPROG delay.

Adding this new flag involves enlarging the no_sfdp_flags variable to 16
bits.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
---
 drivers/mtd/spi-nor/core.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/mtd/spi-nor/core.h b/drivers/mtd/spi-nor/core.h
index 044d49d749e0..299b60788597 100644
--- a/drivers/mtd/spi-nor/core.h
+++ b/drivers/mtd/spi-nor/core.h
@@ -472,6 +472,7 @@ struct spi_nor_fixups {
  *   SPI_NOR_OCTAL_READ:      flash supports Octal Read.
  *   SPI_NOR_OCTAL_DTR_READ:  flash supports octal DTR Read.
  *   SPI_NOR_OCTAL_DTR_PP:    flash supports Octal DTR Page Program.
+ *   SPI_NOR_RWW:             flash supports reads while write.
  *
  * @fixup_flags:    flags that indicate support that can be discovered via SFDP
  *                  ideally, but can not be discovered for this particular flash
@@ -512,7 +513,7 @@ struct flash_info {
 #define NO_CHIP_ERASE			BIT(7)
 #define SPI_NOR_NO_FR			BIT(8)
 
-	u8 no_sfdp_flags;
+	u16 no_sfdp_flags;
 #define SPI_NOR_SKIP_SFDP		BIT(0)
 #define SECT_4K				BIT(1)
 #define SPI_NOR_DUAL_READ		BIT(3)
@@ -520,6 +521,7 @@ struct flash_info {
 #define SPI_NOR_OCTAL_READ		BIT(5)
 #define SPI_NOR_OCTAL_DTR_READ		BIT(6)
 #define SPI_NOR_OCTAL_DTR_PP		BIT(7)
+#define SPI_NOR_RWW			BIT(8)
 
 	u8 fixup_flags;
 #define SPI_NOR_4B_OPCODES		BIT(0)
-- 
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v2 8/9] mtd: spi-nor: Enhance locking to support reads while writes
  2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
                   ` (6 preceding siblings ...)
  2022-11-10 15:55 ` [PATCH v2 7/9] mtd: spi-nor: Add a RWW flag Miquel Raynal
@ 2022-11-10 15:55 ` Miquel Raynal
  2022-12-01 10:54   ` Miquel Raynal
  2022-11-10 15:55 ` [PATCH v2 9/9] mtd: spi-nor: macronix: Add support for mx25uw51245g with RWW Miquel Raynal
  2022-11-20 15:41 ` [PATCH v2 0/9] mtd: spi-nor: read while write support Pratyush Yadav
  9 siblings, 1 reply; 17+ messages in thread
From: Miquel Raynal @ 2022-11-10 15:55 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni, Miquel Raynal

On devices featuring several banks, the Read While Write (RWW) feature
is here to improve the overall performance when performing parallel
reads and writes at different locations (different banks). The following
constraints have to be taken into account:
1#: A single operation can be performed in a given bank.
2#: Only a single program or erase operation can happen on the entire chip.
3#: The I/O bus is unique and thus is the most constrained resource, all
    spi-nor operations requiring access to the spi bus (through the spi
    controller) must be serialized until the bus exchanges are over. So
    we must ensure a single operation can be "sent" at a time.
4#: Any other operation that would not be either a read or a write or an
    erase is considered requiring access to the full chip and cannot be
    parallelized, we then need to ensure the full chip is in the idle
    state when this occurs.

All these constraints can easily be managed with a proper locking model:
1#: Is protected by a per-bank mutex. Only a single operation can happen
    in a specific bank at any times. If the bank mutex is not available,
    the operation cannot start.
2#: Is handled by the pe_mode mutex which is acquired before any write
    or erase, and is released only at the very end of the
    operation. This way, no other destructive operation on the chip can
    start during this time frame.
3#: A device-wide mutex is introduced in order to capture and serialize
    bus accessed. This is the one being released "sooner" than before,
    because we only need to protect the chip against other SPI accesses
    during the I/O phase, which for the destructive operations is the
    beginning of the operation (when we send the command cycles and
    eventually the data), while the second part of the operation (the
    erase delay or the programmation delay) is when we can do something
    else with another bank.
4#: Is handled by the "generic" helpers which existed before, where
    basically all the locks are taken before the operation can start,
    and all locks are released once done.

As many devices still do not support this feature, the original lock is
also kept in a union: either the feature is available and we initialize
and use the new locks, or it is not and we keep using the previous
logic.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
---
 drivers/mtd/spi-nor/core.c  | 145 ++++++++++++++++++++++++++++++++----
 include/linux/mtd/spi-nor.h |  12 ++-
 2 files changed, 143 insertions(+), 14 deletions(-)

diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
index dfda350d5056..b467d5bf0f2a 100644
--- a/drivers/mtd/spi-nor/core.c
+++ b/drivers/mtd/spi-nor/core.c
@@ -1086,6 +1086,61 @@ static void spi_nor_unprep(struct spi_nor *nor)
 		nor->controller_ops->unprepare(nor);
 }
 
+static bool spi_nor_parallel_locking(struct spi_nor *nor)
+{
+	return nor->info->n_banks > 1 && nor->info->no_sfdp_flags & SPI_NOR_RWW;
+}
+
+static void spi_nor_lock_all_banks(struct spi_nor *nor)
+{
+	int i;
+
+	for (i = 0; i < nor->info->n_banks; i++)
+		mutex_lock(&nor->locks.bank[i]);
+}
+
+static void spi_nor_unlock_all_banks(struct spi_nor *nor)
+{
+	int i;
+
+	for (i = nor->info->n_banks; i >= 0; i--)
+		mutex_unlock(&nor->locks.bank[i]);
+}
+
+static void spi_nor_lock_banks(struct spi_nor *nor, loff_t start, size_t len)
+{
+	int first, last, i;
+
+	first = DIV_ROUND_DOWN_ULL(start, nor->params->bank_size);
+	last = DIV_ROUND_DOWN_ULL(start + len - 1, nor->params->bank_size);
+
+	for (i = first; i <= last; i++)
+		mutex_lock(&nor->locks.bank[i]);
+}
+
+static void spi_nor_unlock_banks(struct spi_nor *nor, loff_t start, size_t len)
+{
+	int first, last, i;
+
+	first = DIV_ROUND_DOWN_ULL(start, nor->params->bank_size);
+	last = DIV_ROUND_DOWN_ULL(start + len - 1, nor->params->bank_size);
+
+	for (i = last; i >= first; i--)
+		mutex_unlock(&nor->locks.bank[i]);
+}
+
+static void spi_nor_lock_device(struct spi_nor *nor)
+{
+	if (spi_nor_parallel_locking(nor))
+		mutex_lock(&nor->locks.device);
+}
+
+static void spi_nor_unlock_device(struct spi_nor *nor)
+{
+	if (spi_nor_parallel_locking(nor))
+		mutex_unlock(&nor->locks.device);
+}
+
 /* Generic helpers for internal locking and serialization */
 int spi_nor_lock_and_prep(struct spi_nor *nor)
 {
@@ -1095,14 +1150,26 @@ int spi_nor_lock_and_prep(struct spi_nor *nor)
 	if (ret)
 		return ret;
 
-	mutex_lock(&nor->lock);
+	if (!spi_nor_parallel_locking(nor)) {
+		mutex_lock(&nor->lock);
+	} else {
+		spi_nor_lock_all_banks(nor);
+		mutex_lock(&nor->locks.pe_mode);
+		mutex_lock(&nor->locks.device);
+	}
 
 	return 0;
 }
 
 void spi_nor_unlock_and_unprep(struct spi_nor *nor)
 {
-	mutex_unlock(&nor->lock);
+	if (!spi_nor_parallel_locking(nor)) {
+		mutex_unlock(&nor->lock);
+	} else {
+		mutex_unlock(&nor->locks.device);
+		mutex_unlock(&nor->locks.pe_mode);
+		spi_nor_unlock_all_banks(nor);
+	}
 
 	spi_nor_unprep(nor);
 }
@@ -1116,14 +1183,24 @@ static int spi_nor_lock_and_prep_pe(struct spi_nor *nor, loff_t start, size_t le
 	if (ret)
 		return ret;
 
-	mutex_lock(&nor->lock);
+	if (!spi_nor_parallel_locking(nor)) {
+		mutex_lock(&nor->lock);
+	} else {
+		spi_nor_lock_banks(nor, start, len);
+		mutex_lock(&nor->locks.pe_mode);
+	}
 
 	return 0;
 }
 
 static void spi_nor_unlock_and_unprep_pe(struct spi_nor *nor, loff_t start, size_t len)
 {
-	mutex_unlock(&nor->lock);
+	if (!spi_nor_parallel_locking(nor)) {
+		mutex_unlock(&nor->lock);
+	} else {
+		mutex_unlock(&nor->locks.pe_mode);
+		spi_nor_unlock_banks(nor, start, len);
+	}
 
 	spi_nor_unprep(nor);
 }
@@ -1137,14 +1214,20 @@ static int spi_nor_lock_and_prep_rd(struct spi_nor *nor, loff_t start, size_t le
 	if (ret)
 		return ret;
 
-	mutex_lock(&nor->lock);
+	if (!spi_nor_parallel_locking(nor))
+		mutex_lock(&nor->lock);
+	else
+		spi_nor_lock_banks(nor, start, len);
 
 	return 0;
 }
 
 static void spi_nor_unlock_and_unprep_rd(struct spi_nor *nor, loff_t start, size_t len)
 {
-	mutex_unlock(&nor->lock);
+	if (!spi_nor_parallel_locking(nor))
+		mutex_unlock(&nor->lock);
+	else
+		spi_nor_unlock_banks(nor, start, len);
 
 	spi_nor_unprep(nor);
 }
@@ -1451,11 +1534,16 @@ static int spi_nor_erase_multi_sectors(struct spi_nor *nor, u64 addr, u32 len)
 			dev_vdbg(nor->dev, "erase_cmd->size = 0x%08x, erase_cmd->opcode = 0x%02x, erase_cmd->count = %u\n",
 				 cmd->size, cmd->opcode, cmd->count);
 
+			spi_nor_lock_device(nor);
+
 			ret = spi_nor_write_enable(nor);
-			if (ret)
+			if (ret) {
+				spi_nor_unlock_device(nor);
 				goto destroy_erase_cmd_list;
+			}
 
 			ret = spi_nor_erase_sector(nor, addr);
+			spi_nor_unlock_device(nor);
 			if (ret)
 				goto destroy_erase_cmd_list;
 
@@ -1508,11 +1596,16 @@ static int spi_nor_erase(struct mtd_info *mtd, struct erase_info *instr)
 	if (len == mtd->size && !(nor->flags & SNOR_F_NO_OP_CHIP_ERASE)) {
 		unsigned long timeout;
 
+		spi_nor_lock_device(nor);
+
 		ret = spi_nor_write_enable(nor);
-		if (ret)
+		if (ret) {
+			spi_nor_unlock_device(nor);
 			goto erase_err;
+		}
 
 		ret = spi_nor_erase_chip(nor);
+		spi_nor_unlock_device(nor);
 		if (ret)
 			goto erase_err;
 
@@ -1537,11 +1630,16 @@ static int spi_nor_erase(struct mtd_info *mtd, struct erase_info *instr)
 	/* "sector"-at-a-time erase */
 	} else if (spi_nor_has_uniform_erase(nor)) {
 		while (len) {
+			spi_nor_lock_device(nor);
+
 			ret = spi_nor_write_enable(nor);
-			if (ret)
+			if (ret) {
+				spi_nor_unlock_device(nor);
 				goto erase_err;
+			}
 
 			ret = spi_nor_erase_sector(nor, addr);
+			spi_nor_unlock_device(nor);
 			if (ret)
 				goto erase_err;
 
@@ -1733,11 +1831,13 @@ static int spi_nor_read(struct mtd_info *mtd, loff_t from, size_t len,
 			size_t *retlen, u_char *buf)
 {
 	struct spi_nor *nor = mtd_to_spi_nor(mtd);
+	loff_t from_lock = from;
+	size_t len_lock = len;
 	ssize_t ret;
 
 	dev_dbg(nor->dev, "from 0x%08x, len %zd\n", (u32)from, len);
 
-	ret = spi_nor_lock_and_prep_rd(nor, from, len);
+	ret = spi_nor_lock_and_prep_rd(nor, from_lock, len_lock);
 	if (ret)
 		return ret;
 
@@ -1746,7 +1846,9 @@ static int spi_nor_read(struct mtd_info *mtd, loff_t from, size_t len,
 
 		addr = spi_nor_convert_addr(nor, addr);
 
+		spi_nor_lock_device(nor);
 		ret = spi_nor_read_data(nor, addr, len, buf);
+		spi_nor_unlock_device(nor);
 		if (ret == 0) {
 			/* We shouldn't see 0-length reads */
 			ret = -EIO;
@@ -1764,7 +1866,7 @@ static int spi_nor_read(struct mtd_info *mtd, loff_t from, size_t len,
 	ret = 0;
 
 read_err:
-	spi_nor_unlock_and_unprep_rd(nor, from, len);
+	spi_nor_unlock_and_unprep_rd(nor, from_lock, len_lock);
 
 	return ret;
 }
@@ -1809,11 +1911,16 @@ static int spi_nor_write(struct mtd_info *mtd, loff_t to, size_t len,
 
 		addr = spi_nor_convert_addr(nor, addr);
 
+		spi_nor_lock_device(nor);
+
 		ret = spi_nor_write_enable(nor);
-		if (ret)
+		if (ret) {
+			spi_nor_unlock_device(nor);
 			goto write_err;
+		}
 
 		ret = spi_nor_write_data(nor, addr, page_remain, buf + i);
+		spi_nor_unlock_device(nor);
 		if (ret < 0)
 			goto write_err;
 		written = ret;
@@ -3030,7 +3137,19 @@ int spi_nor_scan(struct spi_nor *nor, const char *name,
 
 	nor->info = info;
 
-	mutex_init(&nor->lock);
+	if (!spi_nor_parallel_locking(nor)) {
+		mutex_init(&nor->lock);
+	} else {
+		mutex_init(&nor->locks.device);
+		mutex_init(&nor->locks.pe_mode);
+		nor->locks.bank = kmalloc(sizeof(struct mutex) * nor->info->n_banks,
+					  GFP_KERNEL);
+		if (!nor->locks.bank)
+			return -ENOMEM;
+
+		for (i = 0; i < nor->info->n_banks; i++)
+			mutex_init(&nor->locks.bank[i]);
+	}
 
 	/* Init flash parameters based on flash_info struct and SFDP */
 	ret = spi_nor_init_params(nor);
diff --git a/include/linux/mtd/spi-nor.h b/include/linux/mtd/spi-nor.h
index 42218a1164f6..02c9a3cf924e 100644
--- a/include/linux/mtd/spi-nor.h
+++ b/include/linux/mtd/spi-nor.h
@@ -344,6 +344,9 @@ struct spi_nor_flash_parameter;
  * struct spi_nor - Structure for defining the SPI NOR layer
  * @mtd:		an mtd_info structure
  * @lock:		the lock for the read/write/erase/lock/unlock operations
+ * @locks.device:	the lock for the I/O bus
+ * @locks.pe_mode:	the lock for the program/erase mode for RWW operations
+ * @locks.bank:		the lock for every available bank
  * @dev:		pointer to an SPI device or an SPI NOR controller device
  * @spimem:		pointer to the SPI memory device
  * @bouncebuf:		bounce buffer used when the buffer passed by the MTD
@@ -374,7 +377,14 @@ struct spi_nor_flash_parameter;
  */
 struct spi_nor {
 	struct mtd_info		mtd;
-	struct mutex		lock;
+	union {
+		struct mutex	lock;
+		struct {
+			struct mutex device;
+			struct mutex pe_mode;
+			struct mutex *bank;
+		} locks;
+	};
 	struct device		*dev;
 	struct spi_mem		*spimem;
 	u8			*bouncebuf;
-- 
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v2 9/9] mtd: spi-nor: macronix: Add support for mx25uw51245g with RWW
  2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
                   ` (7 preceding siblings ...)
  2022-11-10 15:55 ` [PATCH v2 8/9] mtd: spi-nor: Enhance locking to support reads while writes Miquel Raynal
@ 2022-11-10 15:55 ` Miquel Raynal
  2022-11-20 15:41 ` [PATCH v2 0/9] mtd: spi-nor: read while write support Pratyush Yadav
  9 siblings, 0 replies; 17+ messages in thread
From: Miquel Raynal @ 2022-11-10 15:55 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni, Miquel Raynal

Describe this new part and provide the RWW flag for it.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
---
 drivers/mtd/spi-nor/macronix.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/mtd/spi-nor/macronix.c b/drivers/mtd/spi-nor/macronix.c
index d81a4cb2812b..e9b82afcd6c4 100644
--- a/drivers/mtd/spi-nor/macronix.c
+++ b/drivers/mtd/spi-nor/macronix.c
@@ -82,6 +82,9 @@ static const struct flash_info macronix_nor_parts[] = {
 	{ "mx25u51245g", INFO(0xc2253a, 0, 64 * 1024, 1024)
 		NO_SFDP_FLAGS(SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ)
 		FIXUP_FLAGS(SPI_NOR_4B_OPCODES) },
+	{ "mx25uw51245g", INFOB(0xc2813a, 0, 16 * 1024, 1024, 4)
+		NO_SFDP_FLAGS(SECT_4K | SPI_NOR_RWW)
+		FIXUP_FLAGS(SPI_NOR_4B_OPCODES) },
 	{ "mx25v8035f",  INFO(0xc22314, 0, 64 * 1024,  16)
 		NO_SFDP_FLAGS(SECT_4K | SPI_NOR_DUAL_READ |
 			      SPI_NOR_QUAD_READ) },
-- 
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 0/9] mtd: spi-nor: read while write support
  2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
                   ` (8 preceding siblings ...)
  2022-11-10 15:55 ` [PATCH v2 9/9] mtd: spi-nor: macronix: Add support for mx25uw51245g with RWW Miquel Raynal
@ 2022-11-20 15:41 ` Pratyush Yadav
  2022-11-23 17:19   ` Miquel Raynal
  9 siblings, 1 reply; 17+ messages in thread
From: Pratyush Yadav @ 2022-11-20 15:41 UTC (permalink / raw)
  To: Miquel Raynal
  Cc: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd, Julien Su, Jaime Liao,
	Thomas Petazzoni

Hi Miquel,

On 10/11/22 04:55PM, Miquel Raynal wrote:
> Hello folks,
> 
> Here is the follow-up of the RFC trying to bring a little bit of
> parallelism to support SPI-NOR Read While Write feature on parts
> supporting it and featuring several banks.
> 
> I have received some hardware to make it work, so since the RFC, the
> series has been updated to fix my mistakes, but the overall idea is the
> same.
> 
> There is nothing Macronix specific in the implementation, the operations
> and opcodes are exactly the same as before. The only difference being:
> we may consider the chip usable when it is in the busy state during a
> write or an erase. Any chip with an internal split allowing to perform
> parallel operations might possibly leverage the benefits of this
> implementation.
> 
> The first patches are just refactoring and preparation work, there is
> almost no functional change, it's just a way to prepare the introduction
> of the new locking mechanism and hopefully provide the cleanest and
> simplest diff possible for this new feature. The actual change is all
> contained in "mtd: spi-nor: Enhance locking to support reads while
> writes". The logic is described in the commit log and copy/pasted here
> for clarity:
> 
> "
> The following constraints have to be taken into account:
> 1#: A single operation can be performed in a given bank.
> 2#: Only a single program or erase operation can happen on the entire chip.

Is this a limitation of the chip you are working with? A chip with 
multiple banks can in theory support parallel erases and programs on 
each bank as well, right? If so then it might make sense to allow 
parallel erases and program operations too, and then allow for extra 
chip-specific constraints.

I have not yet read the code so I am not sure how complex implementing 
this would be. Just thinking out loud for now.

> 3#: The I/O bus is unique and thus is the most constrained resource, all
>     spi-nor operations requiring access to the spi bus (through the spi
>     controller) must be serialized until the bus exchanges are over. So
>     we must ensure a single operation can be "sent" at a time.
> 4#: Any other operation that would not be either a read or a write or an
>     erase is considered requiring access to the full chip and cannot be
>     parallelized, we then need to ensure the full chip is in the idle
>     state when this occurs.

Makes sense.

> 
> All these constraints can easily be managed with a proper locking model:
> 1#: Is protected by a per-bank mutex. Only a single operation can happen
>     in a specific bank at any times. If the bank mutex is not available,
>     the operation cannot start.
> 2#: Is handled by the pe_mode mutex which is acquired before any write
>     or erase, and is released only at the very end of the
>     operation. This way, no other destructive operation on the chip can
>     start during this time frame.
> 3#: A device-wide mutex is introduced in order to capture and serialize
>     bus accessed. This is the one being released "sooner" than before,
>     because we only need to protect the chip against other SPI accesses
>     during the I/O phase, which for the destructive operations is the
>     beginning of the operation (when we send the command cycles and
>     eventually the data), while the second part of the operation (the
>     erase delay or the programmation delay) is when we can do something
>     else with another bank.
> 4#: Is handled by the "generic" helpers which existed before, where
>     basically all the locks are taken before the operation can start,
>     and all locks are released once done.
> 
> As many devices still do not support this feature, the original lock is
> also kept in a union: either the feature is available and we initialize
> and use the new locks, or it is not and we keep using the previous
> logic.
> "
> 
> Here is now a benchmark with a Macronix MX25UW51245G with bank and RWW
> support:
> 
>      // Testing the two accesses in the same bank
>      $ flash_speed -b0 -k0 -c10 -d /dev/mtd0
>      [...]
>      testing read while write latency
>      read while write took 51ms, read ended after 51ms
> 
>      // Testing the two accesses within different banks
>      $ flash_speed -b0 -k4096 -c10 -d /dev/mtd0
>      [...]
>      testing read while write latency
>      read while write took 51ms, read ended after 20ms
> 
> Here is a branch with the mtd-utils patch bringing support for this
> additional "-k" parameter (for the second block to use during RWW
> testing), used to get the above result:
> https://github.com/miquelraynal/mtd-utils/compare/master...rww
> 
> Cheers,
> Miquèl
> 
> Miquel Raynal (9):
>   mtd: spi-nor: Create macros to define chip IDs and geometries
>   mtd: spi-nor: Introduce the concept of bank
>   mtd: spi-nor: Add a macro to define more banks
>   mtd: spi-nor: Reorder the preparation vs locking steps
>   mtd: spi-nor: Separate preparation and locking
>   mtd: spi-nor: Prepare the introduction of a new locking mechanism
>   mtd: spi-nor: Add a RWW flag
>   mtd: spi-nor: Enhance locking to support reads while writes
>   mtd: spi-nor: macronix: Add support for mx25uw51245g with RWW
> 
>  drivers/mtd/spi-nor/core.c     | 224 +++++++++++++++++++++++++++++----
>  drivers/mtd/spi-nor/core.h     |  61 +++++----
>  drivers/mtd/spi-nor/macronix.c |   3 +
>  include/linux/mtd/spi-nor.h    |  12 +-
>  4 files changed, 250 insertions(+), 50 deletions(-)
> 
> -- 
> 2.34.1
> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

-- 
Regards,
Pratyush Yadav

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 1/9] mtd: spi-nor: Create macros to define chip IDs and geometries
  2022-11-10 15:55 ` [PATCH v2 1/9] mtd: spi-nor: Create macros to define chip IDs and geometries Miquel Raynal
@ 2022-11-20 15:57   ` Pratyush Yadav
  0 siblings, 0 replies; 17+ messages in thread
From: Pratyush Yadav @ 2022-11-20 15:57 UTC (permalink / raw)
  To: Miquel Raynal
  Cc: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd, Julien Su, Jaime Liao,
	Thomas Petazzoni

On 10/11/22 04:55PM, Miquel Raynal wrote:
> The INFO() macro defines an ID array and a couple of geometry
> properties. Right now all its lines are duplicated twice because of the
> INFO6() macro (for extended IDs) and soon as well we will need to add a
> geometry parameter to include the number of banks.
> 
> In order to limit the code duplication, let's create a number of
> intermediate macros which will facilitate defining high-level INFOX()
> macros.
> 
> There is not functional change.

I think the INFO() macros in general are a bit convoluted, but this at 
least makes them a bit better.

Reviewed-by: Pratyush Yadav <pratyush@kernel.org>

> 
> Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
> ---
>  drivers/mtd/spi-nor/core.h | 43 ++++++++++++++++++--------------------
>  1 file changed, 20 insertions(+), 23 deletions(-)
> 
> diff --git a/drivers/mtd/spi-nor/core.h b/drivers/mtd/spi-nor/core.h
> index 85b0cf254e97..dc74c7be3e28 100644
> --- a/drivers/mtd/spi-nor/core.h
> +++ b/drivers/mtd/spi-nor/core.h
> @@ -527,33 +527,30 @@ struct flash_info {
>  	const struct spi_nor_fixups *fixups;
>  };
>  
> +#define SPI_NOR_ID_2ITEMS(_id) ((_id) >> 8) & 0xff, (_id) & 0xff
> +#define SPI_NOR_ID_3ITEMS(_id) ((_id) >> 16) & 0xff, SPI_NOR_ID_2ITEMS(_id)
> +
> +#define SPI_NOR_ID(_jedec_id, _ext_id)					\
> +	.id = { SPI_NOR_ID_3ITEMS(_jedec_id), SPI_NOR_ID_2ITEMS(_ext_id) }, \
> +	.id_len = !(_jedec_id) ? 0 : (3 + ((_ext_id) ? 2 : 0))
> +
> +#define SPI_NOR_ID6(_jedec_id, _ext_id)					\
> +	.id = { SPI_NOR_ID_3ITEMS(_jedec_id), SPI_NOR_ID_3ITEMS(_ext_id) }, \
> +	.id_len = 6
> +
> +#define SPI_NOR_GEOMETRY(_sector_size, _n_sectors)			\
> +	.sector_size = (_sector_size),					\
> +	.n_sectors = (_n_sectors),					\
> +	.page_size = 256
> +
>  /* Used when the "_ext_id" is two bytes at most */
>  #define INFO(_jedec_id, _ext_id, _sector_size, _n_sectors)		\
> -		.id = {							\
> -			((_jedec_id) >> 16) & 0xff,			\
> -			((_jedec_id) >> 8) & 0xff,			\
> -			(_jedec_id) & 0xff,				\
> -			((_ext_id) >> 8) & 0xff,			\
> -			(_ext_id) & 0xff,				\
> -			},						\
> -		.id_len = (!(_jedec_id) ? 0 : (3 + ((_ext_id) ? 2 : 0))),	\
> -		.sector_size = (_sector_size),				\
> -		.n_sectors = (_n_sectors),				\
> -		.page_size = 256,					\
> +	SPI_NOR_ID((_jedec_id), (_ext_id)),				\
> +	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors)),
>  
>  #define INFO6(_jedec_id, _ext_id, _sector_size, _n_sectors)		\
> -		.id = {							\
> -			((_jedec_id) >> 16) & 0xff,			\
> -			((_jedec_id) >> 8) & 0xff,			\
> -			(_jedec_id) & 0xff,				\
> -			((_ext_id) >> 16) & 0xff,			\
> -			((_ext_id) >> 8) & 0xff,			\
> -			(_ext_id) & 0xff,				\
> -			},						\
> -		.id_len = 6,						\
> -		.sector_size = (_sector_size),				\
> -		.n_sectors = (_n_sectors),				\
> -		.page_size = 256,					\
> +	SPI_NOR_ID6((_jedec_id), (_ext_id)),				\
> +	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors)),
>  
>  #define CAT25_INFO(_sector_size, _n_sectors, _page_size, _addr_nbytes)	\
>  		.sector_size = (_sector_size),				\
> -- 
> 2.34.1
> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

-- 
Regards,
Pratyush Yadav

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 2/9] mtd: spi-nor: Introduce the concept of bank
  2022-11-10 15:55 ` [PATCH v2 2/9] mtd: spi-nor: Introduce the concept of bank Miquel Raynal
@ 2022-11-20 16:11   ` Pratyush Yadav
  2022-11-21  8:26     ` Miquel Raynal
  0 siblings, 1 reply; 17+ messages in thread
From: Pratyush Yadav @ 2022-11-20 16:11 UTC (permalink / raw)
  To: Miquel Raynal
  Cc: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd, Julien Su, Jaime Liao,
	Thomas Petazzoni

On 10/11/22 04:55PM, Miquel Raynal wrote:
> SPI-NOR chips are made of pages, which gathered in small groups make
> (erase) sectors. Sectors, gathered together, make banks inside the
> chip. So far there was only one bank per device supported, but we are
> about to introduce support for new chips featuring several banks (up to
> 4 so far) where different operations may happen in parallel.
> 
> Let's allow describing these additional bank parameters.

Just to be sure, are the multiple banks still used via a single Chip 
Select, or do we need multi-CS support for this as well? I do remember 
seeing an RFC about multi-CS support from you some time back and I am 
not sure if that is related.

Reviewed-by: Pratyush Yadav <pratyush@kernel.org>

> 
> Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
> ---
>  drivers/mtd/spi-nor/core.c |  3 ++-
>  drivers/mtd/spi-nor/core.h | 16 +++++++++++-----
>  2 files changed, 13 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
> index f2c64006f8d7..38a57aac6754 100644
> --- a/drivers/mtd/spi-nor/core.c
> +++ b/drivers/mtd/spi-nor/core.c
> @@ -2539,7 +2539,8 @@ static void spi_nor_init_default_params(struct spi_nor *nor)
>  
>  	/* Set SPI NOR sizes. */
>  	params->writesize = 1;
> -	params->size = (u64)info->sector_size * info->n_sectors;
> +	params->bank_size = (u64)info->sector_size * info->n_sectors;
> +	params->size = params->bank_size * info->n_banks;
>  	params->page_size = info->page_size;
>  
>  	if (!(info->flags & SPI_NOR_NO_FR)) {
> diff --git a/drivers/mtd/spi-nor/core.h b/drivers/mtd/spi-nor/core.h
> index dc74c7be3e28..8a067d56c995 100644
> --- a/drivers/mtd/spi-nor/core.h
> +++ b/drivers/mtd/spi-nor/core.h
> @@ -336,7 +336,8 @@ struct spi_nor_otp {
>   * by the spi_nor_fixups hooks, or dynamically when parsing the JESD216
>   * Serial Flash Discoverable Parameters (SFDP) tables.
>   *
> - * @size:		the flash memory density in bytes.
> + * @bank_size:		the flash memory bank density in bytes.
> + * @size:		the total flash memory density in bytes.
>   * @writesize		Minimal writable flash unit size. Defaults to 1. Set to
>   *			ECC unit size for ECC-ed flashes.
>   * @page_size:		the page size of the SPI NOR flash memory.
> @@ -374,6 +375,7 @@ struct spi_nor_otp {
>   * @locking_ops:	SPI NOR locking methods.
>   */
>  struct spi_nor_flash_parameter {
> +	u64				bank_size;
>  	u64				size;
>  	u32				writesize;
>  	u32				page_size;
> @@ -434,7 +436,8 @@ struct spi_nor_fixups {
>   * @id_len:         the number of bytes of ID.
>   * @sector_size:    the size listed here is what works with SPINOR_OP_SE, which
>   *                  isn't necessarily called a "sector" by the vendor.
> - * @n_sectors:      the number of sectors.
> + * @n_sectors:      the number of sectors per bank.
> + * @n_banks:        the number of banks.
>   * @page_size:      the flash's page size.
>   * @addr_nbytes:    number of address bytes to send.
>   *
> @@ -493,6 +496,7 @@ struct flash_info {
>  	u8 id_len;
>  	unsigned sector_size;
>  	u16 n_sectors;
> +	u16 n_banks;
>  	u16 page_size;
>  	u8 addr_nbytes;
>  
> @@ -538,23 +542,25 @@ struct flash_info {
>  	.id = { SPI_NOR_ID_3ITEMS(_jedec_id), SPI_NOR_ID_3ITEMS(_ext_id) }, \
>  	.id_len = 6
>  
> -#define SPI_NOR_GEOMETRY(_sector_size, _n_sectors)			\
> +#define SPI_NOR_GEOMETRY(_sector_size, _n_sectors, _n_banks)		\
>  	.sector_size = (_sector_size),					\
>  	.n_sectors = (_n_sectors),					\
> +	.n_banks = (_n_banks),						\
>  	.page_size = 256
>  
>  /* Used when the "_ext_id" is two bytes at most */
>  #define INFO(_jedec_id, _ext_id, _sector_size, _n_sectors)		\
>  	SPI_NOR_ID((_jedec_id), (_ext_id)),				\
> -	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors)),
> +	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors), 1),
>  
>  #define INFO6(_jedec_id, _ext_id, _sector_size, _n_sectors)		\
>  	SPI_NOR_ID6((_jedec_id), (_ext_id)),				\
> -	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors)),
> +	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors), 1),
>  
>  #define CAT25_INFO(_sector_size, _n_sectors, _page_size, _addr_nbytes)	\
>  		.sector_size = (_sector_size),				\
>  		.n_sectors = (_n_sectors),				\
> +		.n_banks = 1,						\
>  		.page_size = (_page_size),				\
>  		.addr_nbytes = (_addr_nbytes),				\
>  		.flags = SPI_NOR_NO_ERASE | SPI_NOR_NO_FR,		\

-- 
Regards,
Pratyush Yadav

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 3/9] mtd: spi-nor: Add a macro to define more banks
  2022-11-10 15:55 ` [PATCH v2 3/9] mtd: spi-nor: Add a macro to define more banks Miquel Raynal
@ 2022-11-20 16:13   ` Pratyush Yadav
  0 siblings, 0 replies; 17+ messages in thread
From: Pratyush Yadav @ 2022-11-20 16:13 UTC (permalink / raw)
  To: Miquel Raynal
  Cc: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd, Julien Su, Jaime Liao,
	Thomas Petazzoni

On 10/11/22 04:55PM, Miquel Raynal wrote:
> Most of the chips on the market only feature a single bank. However, new
> chips may support more than a single bank, with the possibility to
> parallelize some operations. Let's introduce an INFOB() macro which also
> takes a n_bank parameter.
> 
> Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>

Reviewed-by: Pratyush Yadav <pratyush@kernel.org>

> ---
>  drivers/mtd/spi-nor/core.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/drivers/mtd/spi-nor/core.h b/drivers/mtd/spi-nor/core.h
> index 8a067d56c995..044d49d749e0 100644
> --- a/drivers/mtd/spi-nor/core.h
> +++ b/drivers/mtd/spi-nor/core.h
> @@ -553,6 +553,10 @@ struct flash_info {
>  	SPI_NOR_ID((_jedec_id), (_ext_id)),				\
>  	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors), 1),
>  
> +#define INFOB(_jedec_id, _ext_id, _sector_size, _n_sectors, _n_banks)	\
> +	SPI_NOR_ID((_jedec_id), (_ext_id)),				\
> +	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors), (_n_banks)),
> +
>  #define INFO6(_jedec_id, _ext_id, _sector_size, _n_sectors)		\
>  	SPI_NOR_ID6((_jedec_id), (_ext_id)),				\
>  	SPI_NOR_GEOMETRY((_sector_size), (_n_sectors), 1),

-- 
Regards,
Pratyush Yadav

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 2/9] mtd: spi-nor: Introduce the concept of bank
  2022-11-20 16:11   ` Pratyush Yadav
@ 2022-11-21  8:26     ` Miquel Raynal
  0 siblings, 0 replies; 17+ messages in thread
From: Miquel Raynal @ 2022-11-21  8:26 UTC (permalink / raw)
  To: Pratyush Yadav
  Cc: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd, Julien Su, Jaime Liao,
	Thomas Petazzoni

Hi Pratyush,

pratyush@kernel.org wrote on Sun, 20 Nov 2022 17:11:35 +0100:

> On 10/11/22 04:55PM, Miquel Raynal wrote:
> > SPI-NOR chips are made of pages, which gathered in small groups make
> > (erase) sectors. Sectors, gathered together, make banks inside the
> > chip. So far there was only one bank per device supported, but we are
> > about to introduce support for new chips featuring several banks (up to
> > 4 so far) where different operations may happen in parallel.
> > 
> > Let's allow describing these additional bank parameters.  
> 
> Just to be sure, are the multiple banks still used via a single Chip 
> Select, or do we need multi-CS support for this as well? I do remember 
> seeing an RFC about multi-CS support from you some time back and I am 
> not sure if that is related.

I confirm this is not related to the multi-CS binding I worked on
earlier. All the banks I am talking about are expected to be used via a
single CS.

> Reviewed-by: Pratyush Yadav <pratyush@kernel.org>

Thanks for the reviews!

Cheers,
Miquèl

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 0/9] mtd: spi-nor: read while write support
  2022-11-20 15:41 ` [PATCH v2 0/9] mtd: spi-nor: read while write support Pratyush Yadav
@ 2022-11-23 17:19   ` Miquel Raynal
  0 siblings, 0 replies; 17+ messages in thread
From: Miquel Raynal @ 2022-11-23 17:19 UTC (permalink / raw)
  To: Pratyush Yadav
  Cc: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd, Julien Su, Jaime Liao,
	Thomas Petazzoni

Hi Pratyush,

Jaime, your input is welcome on the topic below.

pratyush@kernel.org wrote on Sun, 20 Nov 2022 16:41:16 +0100:

> Hi Miquel,
> 
> On 10/11/22 04:55PM, Miquel Raynal wrote:
> > Hello folks,
> > 
> > Here is the follow-up of the RFC trying to bring a little bit of
> > parallelism to support SPI-NOR Read While Write feature on parts
> > supporting it and featuring several banks.
> > 
> > I have received some hardware to make it work, so since the RFC, the
> > series has been updated to fix my mistakes, but the overall idea is the
> > same.
> > 
> > There is nothing Macronix specific in the implementation, the operations
> > and opcodes are exactly the same as before. The only difference being:
> > we may consider the chip usable when it is in the busy state during a
> > write or an erase. Any chip with an internal split allowing to perform
> > parallel operations might possibly leverage the benefits of this
> > implementation.
> > 
> > The first patches are just refactoring and preparation work, there is
> > almost no functional change, it's just a way to prepare the introduction
> > of the new locking mechanism and hopefully provide the cleanest and
> > simplest diff possible for this new feature. The actual change is all
> > contained in "mtd: spi-nor: Enhance locking to support reads while
> > writes". The logic is described in the commit log and copy/pasted here
> > for clarity:
> > 
> > "
> > The following constraints have to be taken into account:
> > 1#: A single operation can be performed in a given bank.
> > 2#: Only a single program or erase operation can happen on the entire chip.
> 
> Is this a limitation of the chip you are working with? A chip with 
> multiple banks can in theory support parallel erases and programs on 
> each bank as well, right? If so then it might make sense to allow 
> parallel erases and program operations too, and then allow for extra 
> chip-specific constraints.

Yes this is a limitation of the chip family I had a chance to play
with, but there are two reasons why I would assume we won't need
a more parallelized scheme:

* I doubt any chip will ever be able to perform more than one erase or
  program operation at a time due to the current it might draw. A read
  is a rather non-expensive operation power-wise so I guess that is why
  you can do it in parallel of others, but two program or even worse,
  two erases, that might actually be too much (that is pure speculation
  on my side).

* The second reason is that the RWW feature serves one major purpose:
  accessing your filesystem while you update it. So the goal really is
  to be able to make an update, while still getting a rather good
  responsiveness from the system that runs on the same device during
  it. This actually only involves reading while performing any other
  operation. It does not improve the performances much, besides the
  read latencies.
 
Thanks,
Miquèl

> I have not yet read the code so I am not sure how complex implementing 
> this would be. Just thinking out loud for now.
> 
> > 3#: The I/O bus is unique and thus is the most constrained resource, all
> >     spi-nor operations requiring access to the spi bus (through the spi
> >     controller) must be serialized until the bus exchanges are over. So
> >     we must ensure a single operation can be "sent" at a time.
> > 4#: Any other operation that would not be either a read or a write or an
> >     erase is considered requiring access to the full chip and cannot be
> >     parallelized, we then need to ensure the full chip is in the idle
> >     state when this occurs.
> 
> Makes sense.
> 
> > 
> > All these constraints can easily be managed with a proper locking model:
> > 1#: Is protected by a per-bank mutex. Only a single operation can happen
> >     in a specific bank at any times. If the bank mutex is not available,
> >     the operation cannot start.
> > 2#: Is handled by the pe_mode mutex which is acquired before any write
> >     or erase, and is released only at the very end of the
> >     operation. This way, no other destructive operation on the chip can
> >     start during this time frame.
> > 3#: A device-wide mutex is introduced in order to capture and serialize
> >     bus accessed. This is the one being released "sooner" than before,
> >     because we only need to protect the chip against other SPI accesses
> >     during the I/O phase, which for the destructive operations is the
> >     beginning of the operation (when we send the command cycles and
> >     eventually the data), while the second part of the operation (the
> >     erase delay or the programmation delay) is when we can do something
> >     else with another bank.
> > 4#: Is handled by the "generic" helpers which existed before, where
> >     basically all the locks are taken before the operation can start,
> >     and all locks are released once done.
> > 
> > As many devices still do not support this feature, the original lock is
> > also kept in a union: either the feature is available and we initialize
> > and use the new locks, or it is not and we keep using the previous
> > logic.
> > "
> > 
> > Here is now a benchmark with a Macronix MX25UW51245G with bank and RWW
> > support:
> > 
> >      // Testing the two accesses in the same bank
> >      $ flash_speed -b0 -k0 -c10 -d /dev/mtd0
> >      [...]
> >      testing read while write latency
> >      read while write took 51ms, read ended after 51ms
> > 
> >      // Testing the two accesses within different banks
> >      $ flash_speed -b0 -k4096 -c10 -d /dev/mtd0
> >      [...]
> >      testing read while write latency
> >      read while write took 51ms, read ended after 20ms
> > 
> > Here is a branch with the mtd-utils patch bringing support for this
> > additional "-k" parameter (for the second block to use during RWW
> > testing), used to get the above result:
> > https://github.com/miquelraynal/mtd-utils/compare/master...rww
> > 
> > Cheers,
> > Miquèl
> > 
> > Miquel Raynal (9):
> >   mtd: spi-nor: Create macros to define chip IDs and geometries
> >   mtd: spi-nor: Introduce the concept of bank
> >   mtd: spi-nor: Add a macro to define more banks
> >   mtd: spi-nor: Reorder the preparation vs locking steps
> >   mtd: spi-nor: Separate preparation and locking
> >   mtd: spi-nor: Prepare the introduction of a new locking mechanism
> >   mtd: spi-nor: Add a RWW flag
> >   mtd: spi-nor: Enhance locking to support reads while writes
> >   mtd: spi-nor: macronix: Add support for mx25uw51245g with RWW
> > 
> >  drivers/mtd/spi-nor/core.c     | 224 +++++++++++++++++++++++++++++----
> >  drivers/mtd/spi-nor/core.h     |  61 +++++----
> >  drivers/mtd/spi-nor/macronix.c |   3 +
> >  include/linux/mtd/spi-nor.h    |  12 +-
> >  4 files changed, 250 insertions(+), 50 deletions(-)
> > 
> > -- 
> > 2.34.1
> > 
> > 
> > ______________________________________________________
> > Linux MTD discussion mailing list
> > http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v2 8/9] mtd: spi-nor: Enhance locking to support reads while writes
  2022-11-10 15:55 ` [PATCH v2 8/9] mtd: spi-nor: Enhance locking to support reads while writes Miquel Raynal
@ 2022-12-01 10:54   ` Miquel Raynal
  0 siblings, 0 replies; 17+ messages in thread
From: Miquel Raynal @ 2022-12-01 10:54 UTC (permalink / raw)
  To: Richard Weinberger, Vignesh Raghavendra, Tudor Ambarus,
	Pratyush Yadav, Michael Walle, linux-mtd
  Cc: Julien Su, Jaime Liao, Thomas Petazzoni

Hello,

miquel.raynal@bootlin.com wrote on Thu, 10 Nov 2022 16:55:12 +0100:

> On devices featuring several banks, the Read While Write (RWW) feature
> is here to improve the overall performance when performing parallel
> reads and writes at different locations (different banks). The following
> constraints have to be taken into account:
> 1#: A single operation can be performed in a given bank.
> 2#: Only a single program or erase operation can happen on the entire chip.
> 3#: The I/O bus is unique and thus is the most constrained resource, all
>     spi-nor operations requiring access to the spi bus (through the spi
>     controller) must be serialized until the bus exchanges are over. So
>     we must ensure a single operation can be "sent" at a time.
> 4#: Any other operation that would not be either a read or a write or an
>     erase is considered requiring access to the full chip and cannot be
>     parallelized, we then need to ensure the full chip is in the idle
>     state when this occurs.
> 
> All these constraints can easily be managed with a proper locking model:
> 1#: Is protected by a per-bank mutex. Only a single operation can happen
>     in a specific bank at any times. If the bank mutex is not available,
>     the operation cannot start.

I just discovered and fixed a bug in this version related to the erase
path, the indexes (at/len) may in some cases be updated by the
function, leading to the locks not being released as expected. I just
discovered that while playing with UBI. I will soon provide a v3 with
this fixed (I am testing now with io_paral which seems happy).

Thanks,
Miquèl

> 2#: Is handled by the pe_mode mutex which is acquired before any write
>     or erase, and is released only at the very end of the
>     operation. This way, no other destructive operation on the chip can
>     start during this time frame.
> 3#: A device-wide mutex is introduced in order to capture and serialize
>     bus accessed. This is the one being released "sooner" than before,
>     because we only need to protect the chip against other SPI accesses
>     during the I/O phase, which for the destructive operations is the
>     beginning of the operation (when we send the command cycles and
>     eventually the data), while the second part of the operation (the
>     erase delay or the programmation delay) is when we can do something
>     else with another bank.
> 4#: Is handled by the "generic" helpers which existed before, where
>     basically all the locks are taken before the operation can start,
>     and all locks are released once done.
> 
> As many devices still do not support this feature, the original lock is
> also kept in a union: either the feature is available and we initialize
> and use the new locks, or it is not and we keep using the previous
> logic.
> 
> Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
> ---
>  drivers/mtd/spi-nor/core.c  | 145 ++++++++++++++++++++++++++++++++----
>  include/linux/mtd/spi-nor.h |  12 ++-
>  2 files changed, 143 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
> index dfda350d5056..b467d5bf0f2a 100644
> --- a/drivers/mtd/spi-nor/core.c
> +++ b/drivers/mtd/spi-nor/core.c
> @@ -1086,6 +1086,61 @@ static void spi_nor_unprep(struct spi_nor *nor)
>  		nor->controller_ops->unprepare(nor);
>  }
>  
> +static bool spi_nor_parallel_locking(struct spi_nor *nor)
> +{
> +	return nor->info->n_banks > 1 && nor->info->no_sfdp_flags & SPI_NOR_RWW;
> +}
> +
> +static void spi_nor_lock_all_banks(struct spi_nor *nor)
> +{
> +	int i;
> +
> +	for (i = 0; i < nor->info->n_banks; i++)
> +		mutex_lock(&nor->locks.bank[i]);
> +}
> +
> +static void spi_nor_unlock_all_banks(struct spi_nor *nor)
> +{
> +	int i;
> +
> +	for (i = nor->info->n_banks; i >= 0; i--)
> +		mutex_unlock(&nor->locks.bank[i]);
> +}
> +
> +static void spi_nor_lock_banks(struct spi_nor *nor, loff_t start, size_t len)
> +{
> +	int first, last, i;
> +
> +	first = DIV_ROUND_DOWN_ULL(start, nor->params->bank_size);
> +	last = DIV_ROUND_DOWN_ULL(start + len - 1, nor->params->bank_size);
> +
> +	for (i = first; i <= last; i++)
> +		mutex_lock(&nor->locks.bank[i]);
> +}
> +
> +static void spi_nor_unlock_banks(struct spi_nor *nor, loff_t start, size_t len)
> +{
> +	int first, last, i;
> +
> +	first = DIV_ROUND_DOWN_ULL(start, nor->params->bank_size);
> +	last = DIV_ROUND_DOWN_ULL(start + len - 1, nor->params->bank_size);
> +
> +	for (i = last; i >= first; i--)
> +		mutex_unlock(&nor->locks.bank[i]);
> +}
> +
> +static void spi_nor_lock_device(struct spi_nor *nor)
> +{
> +	if (spi_nor_parallel_locking(nor))
> +		mutex_lock(&nor->locks.device);
> +}
> +
> +static void spi_nor_unlock_device(struct spi_nor *nor)
> +{
> +	if (spi_nor_parallel_locking(nor))
> +		mutex_unlock(&nor->locks.device);
> +}
> +
>  /* Generic helpers for internal locking and serialization */
>  int spi_nor_lock_and_prep(struct spi_nor *nor)
>  {
> @@ -1095,14 +1150,26 @@ int spi_nor_lock_and_prep(struct spi_nor *nor)
>  	if (ret)
>  		return ret;
>  
> -	mutex_lock(&nor->lock);
> +	if (!spi_nor_parallel_locking(nor)) {
> +		mutex_lock(&nor->lock);
> +	} else {
> +		spi_nor_lock_all_banks(nor);
> +		mutex_lock(&nor->locks.pe_mode);
> +		mutex_lock(&nor->locks.device);
> +	}
>  
>  	return 0;
>  }
>  
>  void spi_nor_unlock_and_unprep(struct spi_nor *nor)
>  {
> -	mutex_unlock(&nor->lock);
> +	if (!spi_nor_parallel_locking(nor)) {
> +		mutex_unlock(&nor->lock);
> +	} else {
> +		mutex_unlock(&nor->locks.device);
> +		mutex_unlock(&nor->locks.pe_mode);
> +		spi_nor_unlock_all_banks(nor);
> +	}
>  
>  	spi_nor_unprep(nor);
>  }
> @@ -1116,14 +1183,24 @@ static int spi_nor_lock_and_prep_pe(struct spi_nor *nor, loff_t start, size_t le
>  	if (ret)
>  		return ret;
>  
> -	mutex_lock(&nor->lock);
> +	if (!spi_nor_parallel_locking(nor)) {
> +		mutex_lock(&nor->lock);
> +	} else {
> +		spi_nor_lock_banks(nor, start, len);
> +		mutex_lock(&nor->locks.pe_mode);
> +	}
>  
>  	return 0;
>  }
>  
>  static void spi_nor_unlock_and_unprep_pe(struct spi_nor *nor, loff_t start, size_t len)
>  {
> -	mutex_unlock(&nor->lock);
> +	if (!spi_nor_parallel_locking(nor)) {
> +		mutex_unlock(&nor->lock);
> +	} else {
> +		mutex_unlock(&nor->locks.pe_mode);
> +		spi_nor_unlock_banks(nor, start, len);
> +	}
>  
>  	spi_nor_unprep(nor);
>  }
> @@ -1137,14 +1214,20 @@ static int spi_nor_lock_and_prep_rd(struct spi_nor *nor, loff_t start, size_t le
>  	if (ret)
>  		return ret;
>  
> -	mutex_lock(&nor->lock);
> +	if (!spi_nor_parallel_locking(nor))
> +		mutex_lock(&nor->lock);
> +	else
> +		spi_nor_lock_banks(nor, start, len);
>  
>  	return 0;
>  }
>  
>  static void spi_nor_unlock_and_unprep_rd(struct spi_nor *nor, loff_t start, size_t len)
>  {
> -	mutex_unlock(&nor->lock);
> +	if (!spi_nor_parallel_locking(nor))
> +		mutex_unlock(&nor->lock);
> +	else
> +		spi_nor_unlock_banks(nor, start, len);
>  
>  	spi_nor_unprep(nor);
>  }
> @@ -1451,11 +1534,16 @@ static int spi_nor_erase_multi_sectors(struct spi_nor *nor, u64 addr, u32 len)
>  			dev_vdbg(nor->dev, "erase_cmd->size = 0x%08x, erase_cmd->opcode = 0x%02x, erase_cmd->count = %u\n",
>  				 cmd->size, cmd->opcode, cmd->count);
>  
> +			spi_nor_lock_device(nor);
> +
>  			ret = spi_nor_write_enable(nor);
> -			if (ret)
> +			if (ret) {
> +				spi_nor_unlock_device(nor);
>  				goto destroy_erase_cmd_list;
> +			}
>  
>  			ret = spi_nor_erase_sector(nor, addr);
> +			spi_nor_unlock_device(nor);
>  			if (ret)
>  				goto destroy_erase_cmd_list;
>  
> @@ -1508,11 +1596,16 @@ static int spi_nor_erase(struct mtd_info *mtd, struct erase_info *instr)
>  	if (len == mtd->size && !(nor->flags & SNOR_F_NO_OP_CHIP_ERASE)) {
>  		unsigned long timeout;
>  
> +		spi_nor_lock_device(nor);
> +
>  		ret = spi_nor_write_enable(nor);
> -		if (ret)
> +		if (ret) {
> +			spi_nor_unlock_device(nor);
>  			goto erase_err;
> +		}
>  
>  		ret = spi_nor_erase_chip(nor);
> +		spi_nor_unlock_device(nor);
>  		if (ret)
>  			goto erase_err;
>  
> @@ -1537,11 +1630,16 @@ static int spi_nor_erase(struct mtd_info *mtd, struct erase_info *instr)
>  	/* "sector"-at-a-time erase */
>  	} else if (spi_nor_has_uniform_erase(nor)) {
>  		while (len) {
> +			spi_nor_lock_device(nor);
> +
>  			ret = spi_nor_write_enable(nor);
> -			if (ret)
> +			if (ret) {
> +				spi_nor_unlock_device(nor);
>  				goto erase_err;
> +			}
>  
>  			ret = spi_nor_erase_sector(nor, addr);
> +			spi_nor_unlock_device(nor);
>  			if (ret)
>  				goto erase_err;
>  
> @@ -1733,11 +1831,13 @@ static int spi_nor_read(struct mtd_info *mtd, loff_t from, size_t len,
>  			size_t *retlen, u_char *buf)
>  {
>  	struct spi_nor *nor = mtd_to_spi_nor(mtd);
> +	loff_t from_lock = from;
> +	size_t len_lock = len;
>  	ssize_t ret;
>  
>  	dev_dbg(nor->dev, "from 0x%08x, len %zd\n", (u32)from, len);
>  
> -	ret = spi_nor_lock_and_prep_rd(nor, from, len);
> +	ret = spi_nor_lock_and_prep_rd(nor, from_lock, len_lock);
>  	if (ret)
>  		return ret;
>  
> @@ -1746,7 +1846,9 @@ static int spi_nor_read(struct mtd_info *mtd, loff_t from, size_t len,
>  
>  		addr = spi_nor_convert_addr(nor, addr);
>  
> +		spi_nor_lock_device(nor);
>  		ret = spi_nor_read_data(nor, addr, len, buf);
> +		spi_nor_unlock_device(nor);
>  		if (ret == 0) {
>  			/* We shouldn't see 0-length reads */
>  			ret = -EIO;
> @@ -1764,7 +1866,7 @@ static int spi_nor_read(struct mtd_info *mtd, loff_t from, size_t len,
>  	ret = 0;
>  
>  read_err:
> -	spi_nor_unlock_and_unprep_rd(nor, from, len);
> +	spi_nor_unlock_and_unprep_rd(nor, from_lock, len_lock);
>  
>  	return ret;
>  }
> @@ -1809,11 +1911,16 @@ static int spi_nor_write(struct mtd_info *mtd, loff_t to, size_t len,
>  
>  		addr = spi_nor_convert_addr(nor, addr);
>  
> +		spi_nor_lock_device(nor);
> +
>  		ret = spi_nor_write_enable(nor);
> -		if (ret)
> +		if (ret) {
> +			spi_nor_unlock_device(nor);
>  			goto write_err;
> +		}
>  
>  		ret = spi_nor_write_data(nor, addr, page_remain, buf + i);
> +		spi_nor_unlock_device(nor);
>  		if (ret < 0)
>  			goto write_err;
>  		written = ret;
> @@ -3030,7 +3137,19 @@ int spi_nor_scan(struct spi_nor *nor, const char *name,
>  
>  	nor->info = info;
>  
> -	mutex_init(&nor->lock);
> +	if (!spi_nor_parallel_locking(nor)) {
> +		mutex_init(&nor->lock);
> +	} else {
> +		mutex_init(&nor->locks.device);
> +		mutex_init(&nor->locks.pe_mode);
> +		nor->locks.bank = kmalloc(sizeof(struct mutex) * nor->info->n_banks,
> +					  GFP_KERNEL);
> +		if (!nor->locks.bank)
> +			return -ENOMEM;
> +
> +		for (i = 0; i < nor->info->n_banks; i++)
> +			mutex_init(&nor->locks.bank[i]);
> +	}
>  
>  	/* Init flash parameters based on flash_info struct and SFDP */
>  	ret = spi_nor_init_params(nor);
> diff --git a/include/linux/mtd/spi-nor.h b/include/linux/mtd/spi-nor.h
> index 42218a1164f6..02c9a3cf924e 100644
> --- a/include/linux/mtd/spi-nor.h
> +++ b/include/linux/mtd/spi-nor.h
> @@ -344,6 +344,9 @@ struct spi_nor_flash_parameter;
>   * struct spi_nor - Structure for defining the SPI NOR layer
>   * @mtd:		an mtd_info structure
>   * @lock:		the lock for the read/write/erase/lock/unlock operations
> + * @locks.device:	the lock for the I/O bus
> + * @locks.pe_mode:	the lock for the program/erase mode for RWW operations
> + * @locks.bank:		the lock for every available bank
>   * @dev:		pointer to an SPI device or an SPI NOR controller device
>   * @spimem:		pointer to the SPI memory device
>   * @bouncebuf:		bounce buffer used when the buffer passed by the MTD
> @@ -374,7 +377,14 @@ struct spi_nor_flash_parameter;
>   */
>  struct spi_nor {
>  	struct mtd_info		mtd;
> -	struct mutex		lock;
> +	union {
> +		struct mutex	lock;
> +		struct {
> +			struct mutex device;
> +			struct mutex pe_mode;
> +			struct mutex *bank;
> +		} locks;
> +	};
>  	struct device		*dev;
>  	struct spi_mem		*spimem;
>  	u8			*bouncebuf;


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2022-12-01 10:55 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-10 15:55 [PATCH v2 0/9] mtd: spi-nor: read while write support Miquel Raynal
2022-11-10 15:55 ` [PATCH v2 1/9] mtd: spi-nor: Create macros to define chip IDs and geometries Miquel Raynal
2022-11-20 15:57   ` Pratyush Yadav
2022-11-10 15:55 ` [PATCH v2 2/9] mtd: spi-nor: Introduce the concept of bank Miquel Raynal
2022-11-20 16:11   ` Pratyush Yadav
2022-11-21  8:26     ` Miquel Raynal
2022-11-10 15:55 ` [PATCH v2 3/9] mtd: spi-nor: Add a macro to define more banks Miquel Raynal
2022-11-20 16:13   ` Pratyush Yadav
2022-11-10 15:55 ` [PATCH v2 4/9] mtd: spi-nor: Reorder the preparation vs locking steps Miquel Raynal
2022-11-10 15:55 ` [PATCH v2 5/9] mtd: spi-nor: Separate preparation and locking Miquel Raynal
2022-11-10 15:55 ` [PATCH v2 6/9] mtd: spi-nor: Prepare the introduction of a new locking mechanism Miquel Raynal
2022-11-10 15:55 ` [PATCH v2 7/9] mtd: spi-nor: Add a RWW flag Miquel Raynal
2022-11-10 15:55 ` [PATCH v2 8/9] mtd: spi-nor: Enhance locking to support reads while writes Miquel Raynal
2022-12-01 10:54   ` Miquel Raynal
2022-11-10 15:55 ` [PATCH v2 9/9] mtd: spi-nor: macronix: Add support for mx25uw51245g with RWW Miquel Raynal
2022-11-20 15:41 ` [PATCH v2 0/9] mtd: spi-nor: read while write support Pratyush Yadav
2022-11-23 17:19   ` Miquel Raynal

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).