[PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Kenneth Johansson]

This is needed if you try to use an already existing ubifs image that
is created for hardware that do not support subpage write.

It is not enough that you can select what nandchip to emulate as the
subpage support might not exist in the actual nand driver.

Signed-off-by: Kenneth Johansson <ken@kenjo.org>
---
 drivers/mtd/nand/raw/nandsim.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/mtd/nand/raw/nandsim.c b/drivers/mtd/nand/raw/nandsim.c
index c452819..3e9eaa2 100644
--- a/drivers/mtd/nand/raw/nandsim.c
+++ b/drivers/mtd/nand/raw/nandsim.c
@@ -110,6 +110,7 @@ static unsigned int overridesize = 0;
 static char *cache_file = NULL;
 static unsigned int bbt;
 static unsigned int bch;
+static unsigned int no_subpage;
 static u_char id_bytes[8] = {
 	[0] = CONFIG_NANDSIM_FIRST_ID_BYTE,
 	[1] = CONFIG_NANDSIM_SECOND_ID_BYTE,
@@ -142,6 +143,7 @@ module_param(overridesize,   uint, 0400);
 module_param(cache_file,     charp, 0400);
 module_param(bbt,	     uint, 0400);
 module_param(bch,	     uint, 0400);
+module_param(no_subpage,     uint, 0400);
 
 MODULE_PARM_DESC(id_bytes,       "The ID bytes returned by NAND Flash 'read ID' command");
 MODULE_PARM_DESC(first_id_byte,  "The first byte returned by NAND Flash 'read ID' command (manufacturer ID) (obsolete)");
@@ -177,6 +179,7 @@ MODULE_PARM_DESC(cache_file,     "File to use to cache nand pages instead of mem
 MODULE_PARM_DESC(bbt,		 "0 OOB, 1 BBT with marker in OOB, 2 BBT with marker in data area");
 MODULE_PARM_DESC(bch,		 "Enable BCH ecc and set how many bits should "
 				 "be correctable in 512-byte blocks");
+MODULE_PARM_DESC(no_subpage,	 "Disable use of subpage write");
 
 /* The largest possible page size */
 #define NS_LARGEST_PAGE_SIZE	4096
@@ -2260,6 +2263,10 @@ static int __init ns_init_module(void)
 	/* and 'badblocks' parameters to work */
 	chip->options   |= NAND_SKIP_BBTSCAN;
 
+	/* turn off subpage to be able to simulate using a nand controller without subpage support */
+	if (no_subpage)
+		chip->options   |= NAND_NO_SUBPAGE_WRITE;
+
 	switch (bbt) {
 	case 2:
 		 chip->bbt_options |= NAND_BBT_NO_OOB;
-- 
2.7.4

Re: [PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Kenneth Johansson]

So there was a comment from Richard on the original patch that got 
mangled so it did not even have a subject line.

The original intent for this patch was to make it possible to setup the 
simulator with the same parameters as some real hardware.
That is the command "mtdinfo /dev/mtdX" would show the same on the 
simulator as the real hardware.

It's true that you could change the vid offset with ubiattach but that 
require you know that the LEB missmatch error you get is fixed by a 
different vid offset and you need to understand what that value should be.
This way you only need to match the basic mtd parameters and the rest 
will just work automatically.


On 2018-12-22 13:40, Kenneth Johansson wrote:
> This is needed if you try to use an already existing ubifs image that
> is created for hardware that do not support subpage write.
>
> It is not enough that you can select what nandchip to emulate as the
> subpage support might not exist in the actual nand driver.
>
> Signed-off-by: Kenneth Johansson <ken@kenjo.org>
> ---
>   drivers/mtd/nand/raw/nandsim.c | 7 +++++++
>   1 file changed, 7 insertions(+)
>
> diff --git a/drivers/mtd/nand/raw/nandsim.c b/drivers/mtd/nand/raw/nandsim.c
> index c452819..3e9eaa2 100644
> --- a/drivers/mtd/nand/raw/nandsim.c
> +++ b/drivers/mtd/nand/raw/nandsim.c
> @@ -110,6 +110,7 @@ static unsigned int overridesize = 0;
>   static char *cache_file = NULL;
>   static unsigned int bbt;
>   static unsigned int bch;
> +static unsigned int no_subpage;
>   static u_char id_bytes[8] = {
>   	[0] = CONFIG_NANDSIM_FIRST_ID_BYTE,
>   	[1] = CONFIG_NANDSIM_SECOND_ID_BYTE,
> @@ -142,6 +143,7 @@ module_param(overridesize,   uint, 0400);
>   module_param(cache_file,     charp, 0400);
>   module_param(bbt,	     uint, 0400);
>   module_param(bch,	     uint, 0400);
> +module_param(no_subpage,     uint, 0400);
>   
>   MODULE_PARM_DESC(id_bytes,       "The ID bytes returned by NAND Flash 'read ID' command");
>   MODULE_PARM_DESC(first_id_byte,  "The first byte returned by NAND Flash 'read ID' command (manufacturer ID) (obsolete)");
> @@ -177,6 +179,7 @@ MODULE_PARM_DESC(cache_file,     "File to use to cache nand pages instead of mem
>   MODULE_PARM_DESC(bbt,		 "0 OOB, 1 BBT with marker in OOB, 2 BBT with marker in data area");
>   MODULE_PARM_DESC(bch,		 "Enable BCH ecc and set how many bits should "
>   				 "be correctable in 512-byte blocks");
> +MODULE_PARM_DESC(no_subpage,	 "Disable use of subpage write");
>   
>   /* The largest possible page size */
>   #define NS_LARGEST_PAGE_SIZE	4096
> @@ -2260,6 +2263,10 @@ static int __init ns_init_module(void)
>   	/* and 'badblocks' parameters to work */
>   	chip->options   |= NAND_SKIP_BBTSCAN;
>   
> +	/* turn off subpage to be able to simulate using a nand controller without subpage support */
> +	if (no_subpage)
> +		chip->options   |= NAND_NO_SUBPAGE_WRITE;
> +
>   	switch (bbt) {
>   	case 2:
>   		 chip->bbt_options |= NAND_BBT_NO_OOB;

Re: [PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Richard Weinberger]

Kenneth,

Am Samstag, 22. Dezember 2018, 13:53:39 CET schrieb Kenneth Johansson:
> So there was a comment from Richard on the original patch that got 
> mangled so it did not even have a subject line.
> 
> The original intent for this patch was to make it possible to setup the 
> simulator with the same parameters as some real hardware.
> That is the command "mtdinfo /dev/mtdX" would show the same on the 
> simulator as the real hardware.

Just to make sure: nandsim simulates a given NAND chip but the simulated
NAND flash controller will always be different.
Think of ECC layout.

In this case it may be good enough to fool your userspace, but don't
count on it.
 
> It's true that you could change the vid offset with ubiattach but that 
> require you know that the LEB missmatch error you get is fixed by a 
> different vid offset and you need to understand what that value should be.
> This way you only need to match the basic mtd parameters and the rest 
> will just work automatically.

I agree that adding a parameter to nandsim to control subpage behavior
is nice.
But please keep in mind that nandsim will never perfectly match your hardware.

Boris, unless you have objections I'm fine to control NAND_NO_SUBPAGE_WRITE
in nandsim using a module parameter.

Thanks,
//richard

Re: [PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Boris Brezillon]

Hi Richard,

On Thu, 03 Jan 2019 21:43:13 +0100
Richard Weinberger <richard@nod.at> wrote:

> > It's true that you could change the vid offset with ubiattach but that 
> > require you know that the LEB missmatch error you get is fixed by a 
> > different vid offset and you need to understand what that value should be.
> > This way you only need to match the basic mtd parameters and the rest 
> > will just work automatically.  
> 
> I agree that adding a parameter to nandsim to control subpage behavior
> is nice.
> But please keep in mind that nandsim will never perfectly match your hardware.
> 
> Boris, unless you have objections I'm fine to control NAND_NO_SUBPAGE_WRITE
> in nandsim using a module parameter.

Hm, I'd really like to have nandsim replaced by something more generic
at some point (mtdsim or MTD/NAND emulation in qemu using a virtio
iface), so adding new features/option to this driver is something I'd
like to avoid. This being said, this new MTD/NAND emulation
infrastructure is not there yet, and I don't think I'll have time to
work on it (I started working on mtdsim a while back, and someone took
over this work, but it stopped at some point).

To sum-up, if you think this is absolutely necessary (given the
discussion I'm not convinced it is) I'm okay to add this module param,
but I'd prefer if people were working on a solution to replace all
those emulation layers we have right now (mtdram, nandsim, ...).

Regards,

Boris

Re: [PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Boris Brezillon]

+Miquel

Kenneth, please Cc the NAND maintainers next time.

On Fri, 4 Jan 2019 09:53:02 +0100
Boris Brezillon <bbrezillon@kernel.org> wrote:

> Hi Richard,
> 
> On Thu, 03 Jan 2019 21:43:13 +0100
> Richard Weinberger <richard@nod.at> wrote:
> 
> > > It's true that you could change the vid offset with ubiattach but that 
> > > require you know that the LEB missmatch error you get is fixed by a 
> > > different vid offset and you need to understand what that value should be.
> > > This way you only need to match the basic mtd parameters and the rest 
> > > will just work automatically.    
> > 
> > I agree that adding a parameter to nandsim to control subpage behavior
> > is nice.
> > But please keep in mind that nandsim will never perfectly match your hardware.
> > 
> > Boris, unless you have objections I'm fine to control NAND_NO_SUBPAGE_WRITE
> > in nandsim using a module parameter.  
> 
> Hm, I'd really like to have nandsim replaced by something more generic
> at some point (mtdsim or MTD/NAND emulation in qemu using a virtio
> iface), so adding new features/option to this driver is something I'd
> like to avoid. This being said, this new MTD/NAND emulation
> infrastructure is not there yet, and I don't think I'll have time to
> work on it (I started working on mtdsim a while back, and someone took
> over this work, but it stopped at some point).
> 
> To sum-up, if you think this is absolutely necessary (given the
> discussion I'm not convinced it is) I'm okay to add this module param,
> but I'd prefer if people were working on a solution to replace all
> those emulation layers we have right now (mtdram, nandsim, ...).
> 
> Regards,
> 
> Boris

Re: [PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Kenneth Johansson]

On 2019-01-03 21:43, Richard Weinberger wrote:
> Kenneth,
>
> Am Samstag, 22. Dezember 2018, 13:53:39 CET schrieb Kenneth Johansson:
>> So there was a comment from Richard on the original patch that got
>> mangled so it did not even have a subject line.
>>
>> The original intent for this patch was to make it possible to setup the
>> simulator with the same parameters as some real hardware.
>> That is the command "mtdinfo /dev/mtdX" would show the same on the
>> simulator as the real hardware.
> Just to make sure: nandsim simulates a given NAND chip but the simulated
> NAND flash controller will always be different.
> Think of ECC layout.

Yes but ecc layout is generally not visible to layers on top of MTD in 
the same way that the smallest writeable block size(sub page) is. It 
just feels strange that the simulator could not simulate lack of subpage 
writes as it do exist on quite a few nand controllers. And in real life 
there is always going to be a controller in front of the nand chip so 
any simulator really should take that into account.

Re: [PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Marc Kleine-Budde]

[-- Attachment #1.1.1: Type: text/plain, Size: 2158 bytes --]

On 1/4/19 9:53 AM, Boris Brezillon wrote:
> Hi Richard,
> 
> On Thu, 03 Jan 2019 21:43:13 +0100
> Richard Weinberger <richard@nod.at> wrote:
> 
>>> It's true that you could change the vid offset with ubiattach but that 
>>> require you know that the LEB missmatch error you get is fixed by a 
>>> different vid offset and you need to understand what that value should be.
>>> This way you only need to match the basic mtd parameters and the rest 
>>> will just work automatically.  
>>
>> I agree that adding a parameter to nandsim to control subpage behavior
>> is nice.
>> But please keep in mind that nandsim will never perfectly match your hardware.
>>
>> Boris, unless you have objections I'm fine to control NAND_NO_SUBPAGE_WRITE
>> in nandsim using a module parameter.
> 
> Hm, I'd really like to have nandsim replaced by something more generic
> at some point (mtdsim or MTD/NAND emulation in qemu using a virtio
> iface), so adding new features/option to this driver is something I'd
> like to avoid. This being said, this new MTD/NAND emulation
> infrastructure is not there yet, and I don't think I'll have time to
> work on it (I started working on mtdsim a while back, and someone took
> over this work, but it stopped at some point).
> 
> To sum-up, if you think this is absolutely necessary (given the
> discussion I'm not convinced it is) I'm okay to add this module param,
> but I'd prefer if people were working on a solution to replace all
> those emulation layers we have right now (mtdram, nandsim, ...).

Sorry for picking up this old thread, but is there a change to get this patch
upstream? It works and solved my problem of mounting a NAND dump of an UBI/UBIFS
from an imx6 (2k page size/no sub pages).

I think it doesn't apply clearly to current linus/master, but rebasing is
trivial. I can send a v3.

regards,
Marc

-- 
Pengutronix e.K.                 | Marc Kleine-Budde           |
Embedded Linux                   | https://www.pengutronix.de  |
Vertretung West/Dortmund         | Phone: +49-231-2826-924     |
Amtsgericht Hildesheim, HRA 2686 | Fax:   +49-5121-206917-5555 |


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 144 bytes --]

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Richard Weinberger]

On Wed, Sep 23, 2020 at 5:08 PM Marc Kleine-Budde <mkl@pengutronix.de> wrote:
> Sorry for picking up this old thread, but is there a change to get this patch
> upstream? It works and solved my problem of mounting a NAND dump of an UBI/UBIFS
> from an imx6 (2k page size/no sub pages).
>
> I think it doesn't apply clearly to current linus/master, but rebasing is
> trivial. I can send a v3.

Since a nandsim replacement did not materialize yet, the chances are
good to get this merged. :-)
Before we do so, can you please explain why ubiattach -O does not work for you?
This is the way I deal with subpage "issues" when inspecting ubi/ubifs
images on nandsim.

-- 
Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Marc Kleine-Budde]

[-- Attachment #1.1.1: Type: text/plain, Size: 1345 bytes --]

On 9/24/20 8:31 AM, Richard Weinberger wrote:
> On Wed, Sep 23, 2020 at 5:08 PM Marc Kleine-Budde <mkl@pengutronix.de> wrote:
>> Sorry for picking up this old thread, but is there a change to get this patch
>> upstream? It works and solved my problem of mounting a NAND dump of an UBI/UBIFS
>> from an imx6 (2k page size/no sub pages).
>>
>> I think it doesn't apply clearly to current linus/master, but rebasing is
>> trivial. I can send a v3.
> 
> Since a nandsim replacement did not materialize yet, the chances are
> good to get this merged. :-)
> Before we do so, can you please explain why ubiattach -O does not work for you?

Doh! Thanks for the input, this is the option I needed to properly attach the
UBI! \o/

> This is the way I deal with subpage "issues" when inspecting ubi/ubifs
> images on nandsim.

One question (rather academic, I don't want to do that): Does the UBI/UBIFS do
sub-page writes on the nandsim? Is the resulting NAND image still compatible if
I write it to the original device, that doesn't do subpage writes?

Marc

-- 
Pengutronix e.K.                 | Marc Kleine-Budde           |
Embedded Linux                   | https://www.pengutronix.de  |
Vertretung West/Dortmund         | Phone: +49-231-2826-924     |
Amtsgericht Hildesheim, HRA 2686 | Fax:   +49-5121-206917-5555 |


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 144 bytes --]

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Doh! Thanks for the input, this is the option I needed to properly attach the
> UBI! \o/

:-)

>> This is the way I deal with subpage "issues" when inspecting ubi/ubifs
>> images on nandsim.
> 
> One question (rather academic, I don't want to do that): Does the UBI/UBIFS do
> sub-page writes on the nandsim? Is the resulting NAND image still compatible if
> I write it to the original device, that doesn't do subpage writes?

Since nandsim announces subpages, UBI will use them. Except you define your own
VID header offset.
UBIFS does not use subpages at all and UBI only for VID headers to be more storage
efficient.

So, ubiattach -O should completely fulfill your needs.

If all other parameters of the NAND match, and I guess so, otherwise ubiattach
would notice, the UBI image should also work on the target.
Modulo fastmap, if you have bad blocks.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH v2] mtd: rawnand: nandsim: Add support to disable subpage writes.

[Marc Kleine-Budde]

[-- Attachment #1.1.1: Type: text/plain, Size: 1170 bytes --]

On 9/24/20 12:28 PM, Richard Weinberger wrote:
>>> This is the way I deal with subpage "issues" when inspecting ubi/ubifs
>>> images on nandsim.
>>
>> One question (rather academic, I don't want to do that): Does the UBI/UBIFS do
>> sub-page writes on the nandsim? Is the resulting NAND image still compatible if
>> I write it to the original device, that doesn't do subpage writes?
> 
> Since nandsim announces subpages, UBI will use them. Except you define your own
> VID header offset.
> UBIFS does not use subpages at all and UBI only for VID headers to be more storage
> efficient.
> 
> So, ubiattach -O should completely fulfill your needs.

\o/

> If all other parameters of the NAND match, and I guess so, otherwise ubiattach
> would notice, the UBI image should also work on the target.
> Modulo fastmap, if you have bad blocks.

Thanks for the explanation!

Marc

-- 
Pengutronix e.K.                 | Marc Kleine-Budde           |
Embedded Linux                   | https://www.pengutronix.de  |
Vertretung West/Dortmund         | Phone: +49-231-2826-924     |
Amtsgericht Hildesheim, HRA 2686 | Fax:   +49-5121-206917-5555 |


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 144 bytes --]

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/