MTD: How to get actual image size from MTD partition

MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

Hi,

Our ARM32 Linux embedded system consists of these:
* Linux Kernel: 4.14
* Processor: Qualcomm Arm32 Cortex-A7
* Storage: NAND 512MB
* Platform: Simple busybox
* Filesystem: UBIFS, Squashfs
* Consists of nand raw partitions, squashfs ubi volumes.

My requirement:
To find the checksum of a real image in runtime which is flashed in an
MTD partition.

Problem:
Currently, to find the checksum, we are using:
$ md5sum /dev/mtd14
This returns the proper checksum of the entire partition.
But we wanted to find the checksum only for the actual image data
which will be used by our C utility to validate the image.
Here, we don't know the actual image size.
We only know the "partition-size" and "erasesize".

So, is there a mechanism to somehow find the image size at runtime?

Regards,
Pintu

MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

Hi,

Our ARM32 Linux embedded system consists of these:
* Linux Kernel: 4.14
* Processor: Qualcomm Arm32 Cortex-A7
* Storage: NAND 512MB
* Platform: Simple busybox
* Filesystem: UBIFS, Squashfs
* Consists of nand raw partitions, squashfs ubi volumes.

My requirement:
To find the checksum of a real image in runtime which is flashed in an
MTD partition.

Problem:
Currently, to find the checksum, we are using:
$ md5sum /dev/mtd14
This returns the proper checksum of the entire partition.
But we wanted to find the checksum only for the actual image data
which will be used by our C utility to validate the image.
Here, we don't know the actual image size.
We only know the "partition-size" and "erasesize".

So, is there a mechanism to somehow find the image size at runtime?

Regards,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

Hi,

Our ARM32 Linux embedded system consists of these:
* Linux Kernel: 4.14
* Processor: Qualcomm Arm32 Cortex-A7
* Storage: NAND 512MB
* Platform: Simple busybox
* Filesystem: UBIFS, Squashfs
* Consists of nand raw partitions, squashfs ubi volumes.

My requirement:
To find the checksum of a real image in runtime which is flashed in an
MTD partition.

Problem:
Currently, to find the checksum, we are using:
$ md5sum /dev/mtd14
This returns the proper checksum of the entire partition.
But we wanted to find the checksum only for the actual image data
which will be used by our C utility to validate the image.
Here, we don't know the actual image size.
We only know the "partition-size" and "erasesize".

So, is there a mechanism to somehow find the image size at runtime?

Regards,
Pintu

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

Pintu,

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> My requirement:
> To find the checksum of a real image in runtime which is flashed in an
> MTD partition.
> 
> Problem:
> Currently, to find the checksum, we are using:
> $ md5sum /dev/mtd14
> This returns the proper checksum of the entire partition.
> But we wanted to find the checksum only for the actual image data
> which will be used by our C utility to validate the image.
> Here, we don't know the actual image size.
> We only know the "partition-size" and "erasesize".
> 
> So, is there a mechanism to somehow find the image size at runtime?

not really, UBI manages the MTD and does wearleveling, auto growing of volumes, etc...
So as soon you attach the image once, it is changed and the checksum won't match.
It may work if you don't attach UBI and your flash program tool keeps track of
what pages it wrote.

Thanks,
//richard

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

Pintu,

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> My requirement:
> To find the checksum of a real image in runtime which is flashed in an
> MTD partition.
> 
> Problem:
> Currently, to find the checksum, we are using:
> $ md5sum /dev/mtd14
> This returns the proper checksum of the entire partition.
> But we wanted to find the checksum only for the actual image data
> which will be used by our C utility to validate the image.
> Here, we don't know the actual image size.
> We only know the "partition-size" and "erasesize".
> 
> So, is there a mechanism to somehow find the image size at runtime?

not really, UBI manages the MTD and does wearleveling, auto growing of volumes, etc...
So as soon you attach the image once, it is changed and the checksum won't match.
It may work if you don't attach UBI and your flash program tool keeps track of
what pages it wrote.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

Pintu,

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> My requirement:
> To find the checksum of a real image in runtime which is flashed in an
> MTD partition.
> 
> Problem:
> Currently, to find the checksum, we are using:
> $ md5sum /dev/mtd14
> This returns the proper checksum of the entire partition.
> But we wanted to find the checksum only for the actual image data
> which will be used by our C utility to validate the image.
> Here, we don't know the actual image size.
> We only know the "partition-size" and "erasesize".
> 
> So, is there a mechanism to somehow find the image size at runtime?

not really, UBI manages the MTD and does wearleveling, auto growing of volumes, etc...
So as soon you attach the image once, it is changed and the checksum won't match.
It may work if you don't attach UBI and your flash program tool keeps track of
what pages it wrote.

Thanks,
//richard

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Greg KH]

On Fri, Jul 16, 2021 at 12:12:41PM +0530, Pintu Agarwal wrote:
> Hi,
> 
> Our ARM32 Linux embedded system consists of these:
> * Linux Kernel: 4.14
> * Processor: Qualcomm Arm32 Cortex-A7
> * Storage: NAND 512MB
> * Platform: Simple busybox
> * Filesystem: UBIFS, Squashfs
> * Consists of nand raw partitions, squashfs ubi volumes.
> 
> My requirement:
> To find the checksum of a real image in runtime which is flashed in an
> MTD partition.

Try using the dm-verity module for ensuring that a block device really
is properly signed before mounting it.  That's what it was designed for
and is independent of the block device type.

good luck!

greg k-h

Re: MTD: How to get actual image size from MTD partition

[Greg KH]

On Fri, Jul 16, 2021 at 12:12:41PM +0530, Pintu Agarwal wrote:
> Hi,
> 
> Our ARM32 Linux embedded system consists of these:
> * Linux Kernel: 4.14
> * Processor: Qualcomm Arm32 Cortex-A7
> * Storage: NAND 512MB
> * Platform: Simple busybox
> * Filesystem: UBIFS, Squashfs
> * Consists of nand raw partitions, squashfs ubi volumes.
> 
> My requirement:
> To find the checksum of a real image in runtime which is flashed in an
> MTD partition.

Try using the dm-verity module for ensuring that a block device really
is properly signed before mounting it.  That's what it was designed for
and is independent of the block device type.

good luck!

greg k-h

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Greg KH]

On Fri, Jul 16, 2021 at 12:12:41PM +0530, Pintu Agarwal wrote:
> Hi,
> 
> Our ARM32 Linux embedded system consists of these:
> * Linux Kernel: 4.14
> * Processor: Qualcomm Arm32 Cortex-A7
> * Storage: NAND 512MB
> * Platform: Simple busybox
> * Filesystem: UBIFS, Squashfs
> * Consists of nand raw partitions, squashfs ubi volumes.
> 
> My requirement:
> To find the checksum of a real image in runtime which is flashed in an
> MTD partition.

Try using the dm-verity module for ensuring that a block device really
is properly signed before mounting it.  That's what it was designed for
and is independent of the block device type.

good luck!

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Greg KH" <greg@kroah.com>
> An: "Pintu Agarwal" <pintu.ping@gmail.com>
> CC: "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd" <linux-mtd@lists.infradead.org>, "linux-fsdevel"
> <linux-fsdevel@vger.kernel.org>, "Phillip Lougher" <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>,
> "Kernelnewbies" <kernelnewbies@kernelnewbies.org>, "richard" <richard@nod.at>
> Gesendet: Freitag, 16. Juli 2021 17:41:01
> Betreff: Re: MTD: How to get actual image size from MTD partition

> On Fri, Jul 16, 2021 at 12:12:41PM +0530, Pintu Agarwal wrote:
>> Hi,
>> 
>> Our ARM32 Linux embedded system consists of these:
>> * Linux Kernel: 4.14
>> * Processor: Qualcomm Arm32 Cortex-A7
>> * Storage: NAND 512MB
>> * Platform: Simple busybox
>> * Filesystem: UBIFS, Squashfs
>> * Consists of nand raw partitions, squashfs ubi volumes.
>> 
>> My requirement:
>> To find the checksum of a real image in runtime which is flashed in an
>> MTD partition.
> 
> Try using the dm-verity module for ensuring that a block device really
> is properly signed before mounting it.  That's what it was designed for
> and is independent of the block device type.

MTDs are not block devices. :-)

Thanks,
//richard

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Greg KH" <greg@kroah.com>
> An: "Pintu Agarwal" <pintu.ping@gmail.com>
> CC: "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd" <linux-mtd@lists.infradead.org>, "linux-fsdevel"
> <linux-fsdevel@vger.kernel.org>, "Phillip Lougher" <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>,
> "Kernelnewbies" <kernelnewbies@kernelnewbies.org>, "richard" <richard@nod.at>
> Gesendet: Freitag, 16. Juli 2021 17:41:01
> Betreff: Re: MTD: How to get actual image size from MTD partition

> On Fri, Jul 16, 2021 at 12:12:41PM +0530, Pintu Agarwal wrote:
>> Hi,
>> 
>> Our ARM32 Linux embedded system consists of these:
>> * Linux Kernel: 4.14
>> * Processor: Qualcomm Arm32 Cortex-A7
>> * Storage: NAND 512MB
>> * Platform: Simple busybox
>> * Filesystem: UBIFS, Squashfs
>> * Consists of nand raw partitions, squashfs ubi volumes.
>> 
>> My requirement:
>> To find the checksum of a real image in runtime which is flashed in an
>> MTD partition.
> 
> Try using the dm-verity module for ensuring that a block device really
> is properly signed before mounting it.  That's what it was designed for
> and is independent of the block device type.

MTDs are not block devices. :-)

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Greg KH" <greg@kroah.com>
> An: "Pintu Agarwal" <pintu.ping@gmail.com>
> CC: "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd" <linux-mtd@lists.infradead.org>, "linux-fsdevel"
> <linux-fsdevel@vger.kernel.org>, "Phillip Lougher" <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>,
> "Kernelnewbies" <kernelnewbies@kernelnewbies.org>, "richard" <richard@nod.at>
> Gesendet: Freitag, 16. Juli 2021 17:41:01
> Betreff: Re: MTD: How to get actual image size from MTD partition

> On Fri, Jul 16, 2021 at 12:12:41PM +0530, Pintu Agarwal wrote:
>> Hi,
>> 
>> Our ARM32 Linux embedded system consists of these:
>> * Linux Kernel: 4.14
>> * Processor: Qualcomm Arm32 Cortex-A7
>> * Storage: NAND 512MB
>> * Platform: Simple busybox
>> * Filesystem: UBIFS, Squashfs
>> * Consists of nand raw partitions, squashfs ubi volumes.
>> 
>> My requirement:
>> To find the checksum of a real image in runtime which is flashed in an
>> MTD partition.
> 
> Try using the dm-verity module for ensuring that a block device really
> is properly signed before mounting it.  That's what it was designed for
> and is independent of the block device type.

MTDs are not block devices. :-)

Thanks,
//richard

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:

> >> My requirement:
> >> To find the checksum of a real image in runtime which is flashed in an
> >> MTD partition.
> >
> > Try using the dm-verity module for ensuring that a block device really
> > is properly signed before mounting it.  That's what it was designed for
> > and is independent of the block device type.
>
> MTDs are not block devices. :-)
>
Is it possible to use dm-verity with squashfs ?
We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:

> >> My requirement:
> >> To find the checksum of a real image in runtime which is flashed in an
> >> MTD partition.
> >
> > Try using the dm-verity module for ensuring that a block device really
> > is properly signed before mounting it.  That's what it was designed for
> > and is independent of the block device type.
>
> MTDs are not block devices. :-)
>
Is it possible to use dm-verity with squashfs ?
We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:

> >> My requirement:
> >> To find the checksum of a real image in runtime which is flashed in an
> >> MTD partition.
> >
> > Try using the dm-verity module for ensuring that a block device really
> > is properly signed before mounting it.  That's what it was designed for
> > and is independent of the block device type.
>
> MTDs are not block devices. :-)
>
Is it possible to use dm-verity with squashfs ?
We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> An: "richard" <richard@nod.at>
> CC: "Greg KH" <greg@kroah.com>, "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd"
> <linux-mtd@lists.infradead.org>, "linux-fsdevel" <linux-fsdevel@vger.kernel.org>, "Phillip Lougher"
> <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>, "Kernelnewbies" <kernelnewbies@kernelnewbies.org>
> Gesendet: Montag, 19. Juli 2021 11:09:46
> Betreff: Re: MTD: How to get actual image size from MTD partition

> On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:
> 
>> >> My requirement:
>> >> To find the checksum of a real image in runtime which is flashed in an
>> >> MTD partition.
>> >
>> > Try using the dm-verity module for ensuring that a block device really
>> > is properly signed before mounting it.  That's what it was designed for
>> > and is independent of the block device type.
>>
>> MTDs are not block devices. :-)
>>
> Is it possible to use dm-verity with squashfs ?
> We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44

Well, if you emulate a block device using mtdblock, you can use dm-verity and friends.
Also consider using ubiblock. It offers better performance and wear leveling support.

Thanks,
//richard

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> An: "richard" <richard@nod.at>
> CC: "Greg KH" <greg@kroah.com>, "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd"
> <linux-mtd@lists.infradead.org>, "linux-fsdevel" <linux-fsdevel@vger.kernel.org>, "Phillip Lougher"
> <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>, "Kernelnewbies" <kernelnewbies@kernelnewbies.org>
> Gesendet: Montag, 19. Juli 2021 11:09:46
> Betreff: Re: MTD: How to get actual image size from MTD partition

> On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:
> 
>> >> My requirement:
>> >> To find the checksum of a real image in runtime which is flashed in an
>> >> MTD partition.
>> >
>> > Try using the dm-verity module for ensuring that a block device really
>> > is properly signed before mounting it.  That's what it was designed for
>> > and is independent of the block device type.
>>
>> MTDs are not block devices. :-)
>>
> Is it possible to use dm-verity with squashfs ?
> We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44

Well, if you emulate a block device using mtdblock, you can use dm-verity and friends.
Also consider using ubiblock. It offers better performance and wear leveling support.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> An: "richard" <richard@nod.at>
> CC: "Greg KH" <greg@kroah.com>, "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd"
> <linux-mtd@lists.infradead.org>, "linux-fsdevel" <linux-fsdevel@vger.kernel.org>, "Phillip Lougher"
> <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>, "Kernelnewbies" <kernelnewbies@kernelnewbies.org>
> Gesendet: Montag, 19. Juli 2021 11:09:46
> Betreff: Re: MTD: How to get actual image size from MTD partition

> On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:
> 
>> >> My requirement:
>> >> To find the checksum of a real image in runtime which is flashed in an
>> >> MTD partition.
>> >
>> > Try using the dm-verity module for ensuring that a block device really
>> > is properly signed before mounting it.  That's what it was designed for
>> > and is independent of the block device type.
>>
>> MTDs are not block devices. :-)
>>
> Is it possible to use dm-verity with squashfs ?
> We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44

Well, if you emulate a block device using mtdblock, you can use dm-verity and friends.
Also consider using ubiblock. It offers better performance and wear leveling support.

Thanks,
//richard

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Mon, 19 Jul 2021 at 14:58, Richard Weinberger <richard@nod.at> wrote:
>
> ----- Ursprüngliche Mail -----
> > Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> > An: "richard" <richard@nod.at>
> > CC: "Greg KH" <greg@kroah.com>, "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd"
> > <linux-mtd@lists.infradead.org>, "linux-fsdevel" <linux-fsdevel@vger.kernel.org>, "Phillip Lougher"
> > <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>, "Kernelnewbies" <kernelnewbies@kernelnewbies.org>
> > Gesendet: Montag, 19. Juli 2021 11:09:46
> > Betreff: Re: MTD: How to get actual image size from MTD partition
>
> > On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:
> >
> >> >> My requirement:
> >> >> To find the checksum of a real image in runtime which is flashed in an
> >> >> MTD partition.
> >> >
> >> > Try using the dm-verity module for ensuring that a block device really
> >> > is properly signed before mounting it.  That's what it was designed for
> >> > and is independent of the block device type.
> >>
> >> MTDs are not block devices. :-)
> >>
> > Is it possible to use dm-verity with squashfs ?
> > We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44
>
> Well, if you emulate a block device using mtdblock, you can use dm-verity and friends.
> Also consider using ubiblock. It offers better performance and wear leveling support.
>
Okay thank you.
We have tried dm-verity with squashfs (for our rootfs) but we are
facing some mounting issues.
[...]
[    4.697757] device-mapper: init: adding target '0 96160 verity 1
/dev/mtdblock34 /dev/mtdblock39 4096 4096 12020 8 sha256
d7b8a7d0c01b9aec888930841313a81603a50a2a7be44631c4c813197a50d681
aee087a5be3b982978c923f566a94613496b417f2af592639bc80d141e34dfe7'
[    4.704771] device-mapper: verity: sha256 using implementation
"sha256-generic"
[...]
[    4.727366] device-mapper: init: dm-0 is ready
[    4.912558] VFS: Cannot open root device "dm-0" or
unknown-block(253,0): error -5

The same works with ext4 emulation.
So, not sure if there are any changes missing w.r.t. squashfs on 4.14 kernel ?

Anyways, I will create a separate thread for dm-verity issue and keep
this thread still open for UBI image size issue.
We may use dm-verify for rootfs during booting, but still we need to
perform integrity check for other nand partitions and UBI volumes.

So, instead of calculating the checksum for the entire partition, is
it possible to perform checksum only based on the image size ?
Right now, we are still exploring what are the best possible
mechanisms available for this.

Thanks,
Pintu

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Mon, 19 Jul 2021 at 14:58, Richard Weinberger <richard@nod.at> wrote:
>
> ----- Ursprüngliche Mail -----
> > Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> > An: "richard" <richard@nod.at>
> > CC: "Greg KH" <greg@kroah.com>, "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd"
> > <linux-mtd@lists.infradead.org>, "linux-fsdevel" <linux-fsdevel@vger.kernel.org>, "Phillip Lougher"
> > <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>, "Kernelnewbies" <kernelnewbies@kernelnewbies.org>
> > Gesendet: Montag, 19. Juli 2021 11:09:46
> > Betreff: Re: MTD: How to get actual image size from MTD partition
>
> > On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:
> >
> >> >> My requirement:
> >> >> To find the checksum of a real image in runtime which is flashed in an
> >> >> MTD partition.
> >> >
> >> > Try using the dm-verity module for ensuring that a block device really
> >> > is properly signed before mounting it.  That's what it was designed for
> >> > and is independent of the block device type.
> >>
> >> MTDs are not block devices. :-)
> >>
> > Is it possible to use dm-verity with squashfs ?
> > We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44
>
> Well, if you emulate a block device using mtdblock, you can use dm-verity and friends.
> Also consider using ubiblock. It offers better performance and wear leveling support.
>
Okay thank you.
We have tried dm-verity with squashfs (for our rootfs) but we are
facing some mounting issues.
[...]
[    4.697757] device-mapper: init: adding target '0 96160 verity 1
/dev/mtdblock34 /dev/mtdblock39 4096 4096 12020 8 sha256
d7b8a7d0c01b9aec888930841313a81603a50a2a7be44631c4c813197a50d681
aee087a5be3b982978c923f566a94613496b417f2af592639bc80d141e34dfe7'
[    4.704771] device-mapper: verity: sha256 using implementation
"sha256-generic"
[...]
[    4.727366] device-mapper: init: dm-0 is ready
[    4.912558] VFS: Cannot open root device "dm-0" or
unknown-block(253,0): error -5

The same works with ext4 emulation.
So, not sure if there are any changes missing w.r.t. squashfs on 4.14 kernel ?

Anyways, I will create a separate thread for dm-verity issue and keep
this thread still open for UBI image size issue.
We may use dm-verify for rootfs during booting, but still we need to
perform integrity check for other nand partitions and UBI volumes.

So, instead of calculating the checksum for the entire partition, is
it possible to perform checksum only based on the image size ?
Right now, we are still exploring what are the best possible
mechanisms available for this.

Thanks,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Mon, 19 Jul 2021 at 14:58, Richard Weinberger <richard@nod.at> wrote:
>
> ----- Ursprüngliche Mail -----
> > Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> > An: "richard" <richard@nod.at>
> > CC: "Greg KH" <greg@kroah.com>, "linux-kernel" <linux-kernel@vger.kernel.org>, "linux-mtd"
> > <linux-mtd@lists.infradead.org>, "linux-fsdevel" <linux-fsdevel@vger.kernel.org>, "Phillip Lougher"
> > <phillip@squashfs.org.uk>, "Sean Nyekjaer" <sean@geanix.com>, "Kernelnewbies" <kernelnewbies@kernelnewbies.org>
> > Gesendet: Montag, 19. Juli 2021 11:09:46
> > Betreff: Re: MTD: How to get actual image size from MTD partition
>
> > On Fri, 16 Jul 2021 at 21:56, Richard Weinberger <richard@nod.at> wrote:
> >
> >> >> My requirement:
> >> >> To find the checksum of a real image in runtime which is flashed in an
> >> >> MTD partition.
> >> >
> >> > Try using the dm-verity module for ensuring that a block device really
> >> > is properly signed before mounting it.  That's what it was designed for
> >> > and is independent of the block device type.
> >>
> >> MTDs are not block devices. :-)
> >>
> > Is it possible to use dm-verity with squashfs ?
> > We are using squashfs for our rootfs which is an MTD block /dev/mtdblock44
>
> Well, if you emulate a block device using mtdblock, you can use dm-verity and friends.
> Also consider using ubiblock. It offers better performance and wear leveling support.
>
Okay thank you.
We have tried dm-verity with squashfs (for our rootfs) but we are
facing some mounting issues.
[...]
[    4.697757] device-mapper: init: adding target '0 96160 verity 1
/dev/mtdblock34 /dev/mtdblock39 4096 4096 12020 8 sha256
d7b8a7d0c01b9aec888930841313a81603a50a2a7be44631c4c813197a50d681
aee087a5be3b982978c923f566a94613496b417f2af592639bc80d141e34dfe7'
[    4.704771] device-mapper: verity: sha256 using implementation
"sha256-generic"
[...]
[    4.727366] device-mapper: init: dm-0 is ready
[    4.912558] VFS: Cannot open root device "dm-0" or
unknown-block(253,0): error -5

The same works with ext4 emulation.
So, not sure if there are any changes missing w.r.t. squashfs on 4.14 kernel ?

Anyways, I will create a separate thread for dm-verity issue and keep
this thread still open for UBI image size issue.
We may use dm-verify for rootfs during booting, but still we need to
perform integrity check for other nand partitions and UBI volumes.

So, instead of calculating the checksum for the entire partition, is
it possible to perform checksum only based on the image size ?
Right now, we are still exploring what are the best possible
mechanisms available for this.

Thanks,
Pintu

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> Okay thank you.
> We have tried dm-verity with squashfs (for our rootfs) but we are
> facing some mounting issues.
> [...]
> [    4.697757] device-mapper: init: adding target '0 96160 verity 1
> /dev/mtdblock34 /dev/mtdblock39 4096 4096 12020 8 sha256
> d7b8a7d0c01b9aec888930841313a81603a50a2a7be44631c4c813197a50d681
> aee087a5be3b982978c923f566a94613496b417f2af592639bc80d141e34dfe7'
> [    4.704771] device-mapper: verity: sha256 using implementation
> "sha256-generic"
> [...]
> [    4.727366] device-mapper: init: dm-0 is ready
> [    4.912558] VFS: Cannot open root device "dm-0" or
> unknown-block(253,0): error -5
> 
> The same works with ext4 emulation.
> So, not sure if there are any changes missing w.r.t. squashfs on 4.14 kernel ?

I don't know.

> Anyways, I will create a separate thread for dm-verity issue and keep
> this thread still open for UBI image size issue.
> We may use dm-verify for rootfs during booting, but still we need to
> perform integrity check for other nand partitions and UBI volumes.
> 
> So, instead of calculating the checksum for the entire partition, is
> it possible to perform checksum only based on the image size ?
> Right now, we are still exploring what are the best possible
> mechanisms available for this.

I still don't fully understand what you are trying to achieve.
Is it about cryptographic integrity of your storage or detecting
errors after the flashing process?

But let me advertise ubiblock a second time. If you place your squashfs on
a UBI static volume, UBI knows the exact length and you can checksum it
more easily.

Thanks,
//richard

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> Okay thank you.
> We have tried dm-verity with squashfs (for our rootfs) but we are
> facing some mounting issues.
> [...]
> [    4.697757] device-mapper: init: adding target '0 96160 verity 1
> /dev/mtdblock34 /dev/mtdblock39 4096 4096 12020 8 sha256
> d7b8a7d0c01b9aec888930841313a81603a50a2a7be44631c4c813197a50d681
> aee087a5be3b982978c923f566a94613496b417f2af592639bc80d141e34dfe7'
> [    4.704771] device-mapper: verity: sha256 using implementation
> "sha256-generic"
> [...]
> [    4.727366] device-mapper: init: dm-0 is ready
> [    4.912558] VFS: Cannot open root device "dm-0" or
> unknown-block(253,0): error -5
> 
> The same works with ext4 emulation.
> So, not sure if there are any changes missing w.r.t. squashfs on 4.14 kernel ?

I don't know.

> Anyways, I will create a separate thread for dm-verity issue and keep
> this thread still open for UBI image size issue.
> We may use dm-verify for rootfs during booting, but still we need to
> perform integrity check for other nand partitions and UBI volumes.
> 
> So, instead of calculating the checksum for the entire partition, is
> it possible to perform checksum only based on the image size ?
> Right now, we are still exploring what are the best possible
> mechanisms available for this.

I still don't fully understand what you are trying to achieve.
Is it about cryptographic integrity of your storage or detecting
errors after the flashing process?

But let me advertise ubiblock a second time. If you place your squashfs on
a UBI static volume, UBI knows the exact length and you can checksum it
more easily.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> Okay thank you.
> We have tried dm-verity with squashfs (for our rootfs) but we are
> facing some mounting issues.
> [...]
> [    4.697757] device-mapper: init: adding target '0 96160 verity 1
> /dev/mtdblock34 /dev/mtdblock39 4096 4096 12020 8 sha256
> d7b8a7d0c01b9aec888930841313a81603a50a2a7be44631c4c813197a50d681
> aee087a5be3b982978c923f566a94613496b417f2af592639bc80d141e34dfe7'
> [    4.704771] device-mapper: verity: sha256 using implementation
> "sha256-generic"
> [...]
> [    4.727366] device-mapper: init: dm-0 is ready
> [    4.912558] VFS: Cannot open root device "dm-0" or
> unknown-block(253,0): error -5
> 
> The same works with ext4 emulation.
> So, not sure if there are any changes missing w.r.t. squashfs on 4.14 kernel ?

I don't know.

> Anyways, I will create a separate thread for dm-verity issue and keep
> this thread still open for UBI image size issue.
> We may use dm-verify for rootfs during booting, but still we need to
> perform integrity check for other nand partitions and UBI volumes.
> 
> So, instead of calculating the checksum for the entire partition, is
> it possible to perform checksum only based on the image size ?
> Right now, we are still exploring what are the best possible
> mechanisms available for this.

I still don't fully understand what you are trying to achieve.
Is it about cryptographic integrity of your storage or detecting
errors after the flashing process?

But let me advertise ubiblock a second time. If you place your squashfs on
a UBI static volume, UBI knows the exact length and you can checksum it
more easily.

Thanks,
//richard

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Tue, 20 Jul 2021 at 12:10, Richard Weinberger <richard@nod.at> wrote:

> > Anyways, I will create a separate thread for dm-verity issue and keep
> > this thread still open for UBI image size issue.
> > We may use dm-verify for rootfs during booting, but still we need to
> > perform integrity check for other nand partitions and UBI volumes.
> >
> > So, instead of calculating the checksum for the entire partition, is
> > it possible to perform checksum only based on the image size ?
> > Right now, we are still exploring what are the best possible
> > mechanisms available for this.
>
> I still don't fully understand what you are trying to achieve.
> Is it about cryptographic integrity of your storage or detecting
> errors after the flashing process?
>
Yes, it is about md5 checksum verification for every partition to
check its integrity before updates.


> But let me advertise ubiblock a second time.
Sorry, I could not understand about the ubiblock request. Is it
possible to elaborate little more ?
We are already using squashfs on top of our UBI volumes (including
rootfs mounting).
This is the kernel command line we pass:
rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
Do we need to do something different for ubiblock ?

> If you place your squashfs on a UBI static volume, UBI knows the exact length and you can checksum it
> more easily.
Yes, we use squashfs on UBI volumes, but our volume type is still dynamic.
Also, you said, UBI knows the exact length, you mean the whole image length ?
How can we get this length at runtime ?
Also, how can we get the checksum of the entire UBI volume content
(ignoring the erased/empty/bad block content) ?

Or, you mean to say, the whole checksum logic is in-built inside the
UBI layer and users don't need to worry about the integrity at all ?


Thanks,
Pintu

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Tue, 20 Jul 2021 at 12:10, Richard Weinberger <richard@nod.at> wrote:

> > Anyways, I will create a separate thread for dm-verity issue and keep
> > this thread still open for UBI image size issue.
> > We may use dm-verify for rootfs during booting, but still we need to
> > perform integrity check for other nand partitions and UBI volumes.
> >
> > So, instead of calculating the checksum for the entire partition, is
> > it possible to perform checksum only based on the image size ?
> > Right now, we are still exploring what are the best possible
> > mechanisms available for this.
>
> I still don't fully understand what you are trying to achieve.
> Is it about cryptographic integrity of your storage or detecting
> errors after the flashing process?
>
Yes, it is about md5 checksum verification for every partition to
check its integrity before updates.


> But let me advertise ubiblock a second time.
Sorry, I could not understand about the ubiblock request. Is it
possible to elaborate little more ?
We are already using squashfs on top of our UBI volumes (including
rootfs mounting).
This is the kernel command line we pass:
rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
Do we need to do something different for ubiblock ?

> If you place your squashfs on a UBI static volume, UBI knows the exact length and you can checksum it
> more easily.
Yes, we use squashfs on UBI volumes, but our volume type is still dynamic.
Also, you said, UBI knows the exact length, you mean the whole image length ?
How can we get this length at runtime ?
Also, how can we get the checksum of the entire UBI volume content
(ignoring the erased/empty/bad block content) ?

Or, you mean to say, the whole checksum logic is in-built inside the
UBI layer and users don't need to worry about the integrity at all ?


Thanks,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Tue, 20 Jul 2021 at 12:10, Richard Weinberger <richard@nod.at> wrote:

> > Anyways, I will create a separate thread for dm-verity issue and keep
> > this thread still open for UBI image size issue.
> > We may use dm-verify for rootfs during booting, but still we need to
> > perform integrity check for other nand partitions and UBI volumes.
> >
> > So, instead of calculating the checksum for the entire partition, is
> > it possible to perform checksum only based on the image size ?
> > Right now, we are still exploring what are the best possible
> > mechanisms available for this.
>
> I still don't fully understand what you are trying to achieve.
> Is it about cryptographic integrity of your storage or detecting
> errors after the flashing process?
>
Yes, it is about md5 checksum verification for every partition to
check its integrity before updates.


> But let me advertise ubiblock a second time.
Sorry, I could not understand about the ubiblock request. Is it
possible to elaborate little more ?
We are already using squashfs on top of our UBI volumes (including
rootfs mounting).
This is the kernel command line we pass:
rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
Do we need to do something different for ubiblock ?

> If you place your squashfs on a UBI static volume, UBI knows the exact length and you can checksum it
> more easily.
Yes, we use squashfs on UBI volumes, but our volume type is still dynamic.
Also, you said, UBI knows the exact length, you mean the whole image length ?
How can we get this length at runtime ?
Also, how can we get the checksum of the entire UBI volume content
(ignoring the erased/empty/bad block content) ?

Or, you mean to say, the whole checksum logic is in-built inside the
UBI layer and users don't need to worry about the integrity at all ?


Thanks,
Pintu

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
>> But let me advertise ubiblock a second time.
> Sorry, I could not understand about the ubiblock request. Is it
> possible to elaborate little more ?
> We are already using squashfs on top of our UBI volumes (including
> rootfs mounting).
> This is the kernel command line we pass:
> rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> Do we need to do something different for ubiblock ?

From that command line I understand that you are *not* using squashfs on top of UBI.
You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
block device.
See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock

>> If you place your squashfs on a UBI static volume, UBI knows the exact length
>> and you can checksum it
>> more easily.
> Yes, we use squashfs on UBI volumes, but our volume type is still dynamic.
> Also, you said, UBI knows the exact length, you mean the whole image length ?
> How can we get this length at runtime ?

You need a static volume for that. If you update a static volume the length is
known by UBI.

> Also, how can we get the checksum of the entire UBI volume content
> (ignoring the erased/empty/bad block content) ?

Just read from the volume. /dev/ubiX_Y.

> Or, you mean to say, the whole checksum logic is in-built inside the
> UBI layer and users don't need to worry about the integrity at all ?

Static volumes have a crc32 checksum over the whole content.
Of course this offers no cryptographic integrity.
See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_overview

Thanks,
//richard

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
>> But let me advertise ubiblock a second time.
> Sorry, I could not understand about the ubiblock request. Is it
> possible to elaborate little more ?
> We are already using squashfs on top of our UBI volumes (including
> rootfs mounting).
> This is the kernel command line we pass:
> rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> Do we need to do something different for ubiblock ?

From that command line I understand that you are *not* using squashfs on top of UBI.
You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
block device.
See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock

>> If you place your squashfs on a UBI static volume, UBI knows the exact length
>> and you can checksum it
>> more easily.
> Yes, we use squashfs on UBI volumes, but our volume type is still dynamic.
> Also, you said, UBI knows the exact length, you mean the whole image length ?
> How can we get this length at runtime ?

You need a static volume for that. If you update a static volume the length is
known by UBI.

> Also, how can we get the checksum of the entire UBI volume content
> (ignoring the erased/empty/bad block content) ?

Just read from the volume. /dev/ubiX_Y.

> Or, you mean to say, the whole checksum logic is in-built inside the
> UBI layer and users don't need to worry about the integrity at all ?

Static volumes have a crc32 checksum over the whole content.
Of course this offers no cryptographic integrity.
See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_overview

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

----- Ursprüngliche Mail -----
>> But let me advertise ubiblock a second time.
> Sorry, I could not understand about the ubiblock request. Is it
> possible to elaborate little more ?
> We are already using squashfs on top of our UBI volumes (including
> rootfs mounting).
> This is the kernel command line we pass:
> rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> Do we need to do something different for ubiblock ?

From that command line I understand that you are *not* using squashfs on top of UBI.
You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
block device.
See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock

>> If you place your squashfs on a UBI static volume, UBI knows the exact length
>> and you can checksum it
>> more easily.
> Yes, we use squashfs on UBI volumes, but our volume type is still dynamic.
> Also, you said, UBI knows the exact length, you mean the whole image length ?
> How can we get this length at runtime ?

You need a static volume for that. If you update a static volume the length is
known by UBI.

> Also, how can we get the checksum of the entire UBI volume content
> (ignoring the erased/empty/bad block content) ?

Just read from the volume. /dev/ubiX_Y.

> Or, you mean to say, the whole checksum logic is in-built inside the
> UBI layer and users don't need to worry about the integrity at all ?

Static volumes have a crc32 checksum over the whole content.
Of course this offers no cryptographic integrity.
See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_overview

Thanks,
//richard

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
>
> ----- Ursprüngliche Mail -----
> >> But let me advertise ubiblock a second time.
> > Sorry, I could not understand about the ubiblock request. Is it
> > possible to elaborate little more ?
> > We are already using squashfs on top of our UBI volumes (including
> > rootfs mounting).
> > This is the kernel command line we pass:
> > rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> > And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> > Do we need to do something different for ubiblock ?
>
> From that command line I understand that you are *not* using squashfs on top of UBI.
> You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
> block device.
> See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
>
Okay, you mean to say, we should use this ?
ubi.mtd=5 ubi.block=0,0 root=/dev/ubiblock0_0
Instead of this:
root=/dev/mtdblock44 ubi.mtd=40,0,30

Okay I will discuss this internally and check..

> >> If you place your squashfs on a UBI static volume, UBI knows the exact length
> >> and you can checksum it
> >> more easily.
> > Yes, we use squashfs on UBI volumes, but our volume type is still dynamic.
> > Also, you said, UBI knows the exact length, you mean the whole image length ?
> > How can we get this length at runtime ?
>
> You need a static volume for that. If you update a static volume the length is
> known by UBI.
>
Thank you so much for your reply!

Sorry, I could not get this part. How static volume can give image len ?
You mean there is some interface available in kernel to get actual image len ?

> > Also, how can we get the checksum of the entire UBI volume content
> > (ignoring the erased/empty/bad block content) ?
>
> Just read from the volume. /dev/ubiX_Y.
>
I think this also will give the entire volume size, but we still don't know how
many pages have real data ?
For example:
Suppose, my raw partition/volume is of size 10MB
But my actual data inside it is of size ~3MB (may be split across?)
Then, how can we get the actual size of the data content ?
You mean to say: /dev/ubiX_Y should contain only data blocks ?

> > Or, you mean to say, the whole checksum logic is in-built inside the
> > UBI layer and users don't need to worry about the integrity at all ?
>

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
>
> ----- Ursprüngliche Mail -----
> >> But let me advertise ubiblock a second time.
> > Sorry, I could not understand about the ubiblock request. Is it
> > possible to elaborate little more ?
> > We are already using squashfs on top of our UBI volumes (including
> > rootfs mounting).
> > This is the kernel command line we pass:
> > rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> > And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> > Do we need to do something different for ubiblock ?
>
> From that command line I understand that you are *not* using squashfs on top of UBI.
> You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
> block device.
> See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
>
Okay, you mean to say, we should use this ?
ubi.mtd=5 ubi.block=0,0 root=/dev/ubiblock0_0
Instead of this:
root=/dev/mtdblock44 ubi.mtd=40,0,30

Okay I will discuss this internally and check..

> >> If you place your squashfs on a UBI static volume, UBI knows the exact length
> >> and you can checksum it
> >> more easily.
> > Yes, we use squashfs on UBI volumes, but our volume type is still dynamic.
> > Also, you said, UBI knows the exact length, you mean the whole image length ?
> > How can we get this length at runtime ?
>
> You need a static volume for that. If you update a static volume the length is
> known by UBI.
>
Thank you so much for your reply!

Sorry, I could not get this part. How static volume can give image len ?
You mean there is some interface available in kernel to get actual image len ?

> > Also, how can we get the checksum of the entire UBI volume content
> > (ignoring the erased/empty/bad block content) ?
>
> Just read from the volume. /dev/ubiX_Y.
>
I think this also will give the entire volume size, but we still don't know how
many pages have real data ?
For example:
Suppose, my raw partition/volume is of size 10MB
But my actual data inside it is of size ~3MB (may be split across?)
Then, how can we get the actual size of the data content ?
You mean to say: /dev/ubiX_Y should contain only data blocks ?

> > Or, you mean to say, the whole checksum logic is in-built inside the
> > UBI layer and users don't need to worry about the integrity at all ?
>

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
>
> ----- Ursprüngliche Mail -----
> >> But let me advertise ubiblock a second time.
> > Sorry, I could not understand about the ubiblock request. Is it
> > possible to elaborate little more ?
> > We are already using squashfs on top of our UBI volumes (including
> > rootfs mounting).
> > This is the kernel command line we pass:
> > rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> > And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> > Do we need to do something different for ubiblock ?
>
> From that command line I understand that you are *not* using squashfs on top of UBI.
> You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
> block device.
> See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
>
Okay, you mean to say, we should use this ?
ubi.mtd=5 ubi.block=0,0 root=/dev/ubiblock0_0
Instead of this:
root=/dev/mtdblock44 ubi.mtd=40,0,30

Okay I will discuss this internally and check..

> >> If you place your squashfs on a UBI static volume, UBI knows the exact length
> >> and you can checksum it
> >> more easily.
> > Yes, we use squashfs on UBI volumes, but our volume type is still dynamic.
> > Also, you said, UBI knows the exact length, you mean the whole image length ?
> > How can we get this length at runtime ?
>
> You need a static volume for that. If you update a static volume the length is
> known by UBI.
>
Thank you so much for your reply!

Sorry, I could not get this part. How static volume can give image len ?
You mean there is some interface available in kernel to get actual image len ?

> > Also, how can we get the checksum of the entire UBI volume content
> > (ignoring the erased/empty/bad block content) ?
>
> Just read from the volume. /dev/ubiX_Y.
>
I think this also will give the entire volume size, but we still don't know how
many pages have real data ?
For example:
Suppose, my raw partition/volume is of size 10MB
But my actual data inside it is of size ~3MB (may be split across?)
Then, how can we get the actual size of the data content ?
You mean to say: /dev/ubiX_Y should contain only data blocks ?

> > Or, you mean to say, the whole checksum logic is in-built inside the
> > UBI layer and users don't need to worry about the integrity at all ?
>

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

On Thu, Jul 22, 2021 at 1:11 PM Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
> >
> > ----- Ursprüngliche Mail -----
> > >> But let me advertise ubiblock a second time.
> > > Sorry, I could not understand about the ubiblock request. Is it
> > > possible to elaborate little more ?
> > > We are already using squashfs on top of our UBI volumes (including
> > > rootfs mounting).
> > > This is the kernel command line we pass:
> > > rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> > > And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> > > Do we need to do something different for ubiblock ?
> >
> > From that command line I understand that you are *not* using squashfs on top of UBI.
> > You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
> > block device.
> > See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
> >
> Okay, you mean to say, we should use this ?
> ubi.mtd=5 ubi.block=0,0 root=/dev/ubiblock0_0
> Instead of this:
> root=/dev/mtdblock44 ubi.mtd=40,0,30

Yes. But it is not only about a different command line. It is a
different concept.
You use a emulated block device on top of UBI, and not directly on top
of an MTD part.

> Sorry, I could not get this part. How static volume can give image len ?
> You mean there is some interface available in kernel to get actual image len ?

use the ubinfo tool. Static volumes know exactly how much they are filled.

> > > Also, how can we get the checksum of the entire UBI volume content
> > > (ignoring the erased/empty/bad block content) ?
> >
> > Just read from the volume. /dev/ubiX_Y.
> >
> I think this also will give the entire volume size, but we still don't know how
> many pages have real data ?

"ubiinfo /dev/ubiX_Y" will tell you if the volume is of type static.

> For example:
> Suppose, my raw partition/volume is of size 10MB
> But my actual data inside it is of size ~3MB (may be split across?)
> Then, how can we get the actual size of the data content ?

See above.

> You mean to say: /dev/ubiX_Y should contain only data blocks ?

Yes. An UBI volume contains only "user data".

-- 
Thanks,
//richard

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

On Thu, Jul 22, 2021 at 1:11 PM Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
> >
> > ----- Ursprüngliche Mail -----
> > >> But let me advertise ubiblock a second time.
> > > Sorry, I could not understand about the ubiblock request. Is it
> > > possible to elaborate little more ?
> > > We are already using squashfs on top of our UBI volumes (including
> > > rootfs mounting).
> > > This is the kernel command line we pass:
> > > rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> > > And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> > > Do we need to do something different for ubiblock ?
> >
> > From that command line I understand that you are *not* using squashfs on top of UBI.
> > You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
> > block device.
> > See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
> >
> Okay, you mean to say, we should use this ?
> ubi.mtd=5 ubi.block=0,0 root=/dev/ubiblock0_0
> Instead of this:
> root=/dev/mtdblock44 ubi.mtd=40,0,30

Yes. But it is not only about a different command line. It is a
different concept.
You use a emulated block device on top of UBI, and not directly on top
of an MTD part.

> Sorry, I could not get this part. How static volume can give image len ?
> You mean there is some interface available in kernel to get actual image len ?

use the ubinfo tool. Static volumes know exactly how much they are filled.

> > > Also, how can we get the checksum of the entire UBI volume content
> > > (ignoring the erased/empty/bad block content) ?
> >
> > Just read from the volume. /dev/ubiX_Y.
> >
> I think this also will give the entire volume size, but we still don't know how
> many pages have real data ?

"ubiinfo /dev/ubiX_Y" will tell you if the volume is of type static.

> For example:
> Suppose, my raw partition/volume is of size 10MB
> But my actual data inside it is of size ~3MB (may be split across?)
> Then, how can we get the actual size of the data content ?

See above.

> You mean to say: /dev/ubiX_Y should contain only data blocks ?

Yes. An UBI volume contains only "user data".

-- 
Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

On Thu, Jul 22, 2021 at 1:11 PM Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
> >
> > ----- Ursprüngliche Mail -----
> > >> But let me advertise ubiblock a second time.
> > > Sorry, I could not understand about the ubiblock request. Is it
> > > possible to elaborate little more ?
> > > We are already using squashfs on top of our UBI volumes (including
> > > rootfs mounting).
> > > This is the kernel command line we pass:
> > > rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> > > And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> > > Do we need to do something different for ubiblock ?
> >
> > From that command line I understand that you are *not* using squashfs on top of UBI.
> > You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
> > block device.
> > See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
> >
> Okay, you mean to say, we should use this ?
> ubi.mtd=5 ubi.block=0,0 root=/dev/ubiblock0_0
> Instead of this:
> root=/dev/mtdblock44 ubi.mtd=40,0,30

Yes. But it is not only about a different command line. It is a
different concept.
You use a emulated block device on top of UBI, and not directly on top
of an MTD part.

> Sorry, I could not get this part. How static volume can give image len ?
> You mean there is some interface available in kernel to get actual image len ?

use the ubinfo tool. Static volumes know exactly how much they are filled.

> > > Also, how can we get the checksum of the entire UBI volume content
> > > (ignoring the erased/empty/bad block content) ?
> >
> > Just read from the volume. /dev/ubiX_Y.
> >
> I think this also will give the entire volume size, but we still don't know how
> many pages have real data ?

"ubiinfo /dev/ubiX_Y" will tell you if the volume is of type static.

> For example:
> Suppose, my raw partition/volume is of size 10MB
> But my actual data inside it is of size ~3MB (may be split across?)
> Then, how can we get the actual size of the data content ?

See above.

> You mean to say: /dev/ubiX_Y should contain only data blocks ?

Yes. An UBI volume contains only "user data".

-- 
Thanks,
//richard

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

Hi Richard,

On Tue, 27 Jul 2021 at 18:16, Richard Weinberger
<richard.weinberger@gmail.com> wrote:
>
> On Thu, Jul 22, 2021 at 1:11 PM Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
> > >
> > > ----- Ursprüngliche Mail -----
> > > >> But let me advertise ubiblock a second time.
> > > > Sorry, I could not understand about the ubiblock request. Is it
> > > > possible to elaborate little more ?
> > > > We are already using squashfs on top of our UBI volumes (including
> > > > rootfs mounting).
> > > > This is the kernel command line we pass:
> > > > rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> > > > And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> > > > Do we need to do something different for ubiblock ?
> > >
> > > From that command line I understand that you are *not* using squashfs on top of UBI.
> > > You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
> > > block device.
> > > See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
> > >
[snip]

Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
instead of using ubiblock :-)

Can we patch either Kconfig or add some warn_once on mtdblock
usage, suggesting to use ubiblock instead?

I remember there was still some use case(s) for mtdblock but I can't remember
now what was it, perhaps we should document the expectations?
(Is that for JFFS2 to mount?)

Thanks,
Ezequiel

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

Hi Richard,

On Tue, 27 Jul 2021 at 18:16, Richard Weinberger
<richard.weinberger@gmail.com> wrote:
>
> On Thu, Jul 22, 2021 at 1:11 PM Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
> > >
> > > ----- Ursprüngliche Mail -----
> > > >> But let me advertise ubiblock a second time.
> > > > Sorry, I could not understand about the ubiblock request. Is it
> > > > possible to elaborate little more ?
> > > > We are already using squashfs on top of our UBI volumes (including
> > > > rootfs mounting).
> > > > This is the kernel command line we pass:
> > > > rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> > > > And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> > > > Do we need to do something different for ubiblock ?
> > >
> > > From that command line I understand that you are *not* using squashfs on top of UBI.
> > > You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
> > > block device.
> > > See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
> > >
[snip]

Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
instead of using ubiblock :-)

Can we patch either Kconfig or add some warn_once on mtdblock
usage, suggesting to use ubiblock instead?

I remember there was still some use case(s) for mtdblock but I can't remember
now what was it, perhaps we should document the expectations?
(Is that for JFFS2 to mount?)

Thanks,
Ezequiel

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

Hi Richard,

On Tue, 27 Jul 2021 at 18:16, Richard Weinberger
<richard.weinberger@gmail.com> wrote:
>
> On Thu, Jul 22, 2021 at 1:11 PM Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > On Thu, 22 Jul 2021 at 02:24, Richard Weinberger <richard@nod.at> wrote:
> > >
> > > ----- Ursprüngliche Mail -----
> > > >> But let me advertise ubiblock a second time.
> > > > Sorry, I could not understand about the ubiblock request. Is it
> > > > possible to elaborate little more ?
> > > > We are already using squashfs on top of our UBI volumes (including
> > > > rootfs mounting).
> > > > This is the kernel command line we pass:
> > > > rootfstype=squashfs root=/dev/mtdblock44 ubi.mtd=40,0,30
> > > > And CONFIG_MTD_UBI_BLOCK=y is already enabled in our kernel.
> > > > Do we need to do something different for ubiblock ?
> > >
> > > From that command line I understand that you are *not* using squashfs on top of UBI.
> > > You use mtdblock. ubiblock is a mechanism to turn an UBI volume into a read-only
> > > block device.
> > > See: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
> > >
[snip]

Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
instead of using ubiblock :-)

Can we patch either Kconfig or add some warn_once on mtdblock
usage, suggesting to use ubiblock instead?

I remember there was still some use case(s) for mtdblock but I can't remember
now what was it, perhaps we should document the expectations?
(Is that for JFFS2 to mount?)

Thanks,
Ezequiel

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

Ezequiel,

----- Ursprüngliche Mail -----
> [snip]
> 
> Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> instead of using ubiblock :-)
> 
> Can we patch either Kconfig or add some warn_once on mtdblock
> usage, suggesting to use ubiblock instead?

a hint in Kconfig makes IMHO sense. Do you want to send a patch?
A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
a good choice.
ubiblock is mostly useful for NAND flash.

> I remember there was still some use case(s) for mtdblock but I can't remember
> now what was it, perhaps we should document the expectations?
> (Is that for JFFS2 to mount?)

a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
JFFS2.
This limitation is gone.

Thanks,
//richard

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

Ezequiel,

----- Ursprüngliche Mail -----
> [snip]
> 
> Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> instead of using ubiblock :-)
> 
> Can we patch either Kconfig or add some warn_once on mtdblock
> usage, suggesting to use ubiblock instead?

a hint in Kconfig makes IMHO sense. Do you want to send a patch?
A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
a good choice.
ubiblock is mostly useful for NAND flash.

> I remember there was still some use case(s) for mtdblock but I can't remember
> now what was it, perhaps we should document the expectations?
> (Is that for JFFS2 to mount?)

a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
JFFS2.
This limitation is gone.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Richard Weinberger]

Ezequiel,

----- Ursprüngliche Mail -----
> [snip]
> 
> Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> instead of using ubiblock :-)
> 
> Can we patch either Kconfig or add some warn_once on mtdblock
> usage, suggesting to use ubiblock instead?

a hint in Kconfig makes IMHO sense. Do you want to send a patch?
A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
a good choice.
ubiblock is mostly useful for NAND flash.

> I remember there was still some use case(s) for mtdblock but I can't remember
> now what was it, perhaps we should document the expectations?
> (Is that for JFFS2 to mount?)

a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
JFFS2.
This limitation is gone.

Thanks,
//richard

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
>
> Ezequiel,
>
> ----- Ursprüngliche Mail -----
> > [snip]
> >
> > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > instead of using ubiblock :-)
> >
> > Can we patch either Kconfig or add some warn_once on mtdblock
> > usage, suggesting to use ubiblock instead?
>
> a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> a good choice.
> ubiblock is mostly useful for NAND flash.
>
> > I remember there was still some use case(s) for mtdblock but I can't remember
> > now what was it, perhaps we should document the expectations?
> > (Is that for JFFS2 to mount?)
>
> a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> JFFS2.
> This limitation is gone.
>

OK, let me try to cook a patch for you.

Eze

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
>
> Ezequiel,
>
> ----- Ursprüngliche Mail -----
> > [snip]
> >
> > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > instead of using ubiblock :-)
> >
> > Can we patch either Kconfig or add some warn_once on mtdblock
> > usage, suggesting to use ubiblock instead?
>
> a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> a good choice.
> ubiblock is mostly useful for NAND flash.
>
> > I remember there was still some use case(s) for mtdblock but I can't remember
> > now what was it, perhaps we should document the expectations?
> > (Is that for JFFS2 to mount?)
>
> a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> JFFS2.
> This limitation is gone.
>

OK, let me try to cook a patch for you.

Eze

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
>
> Ezequiel,
>
> ----- Ursprüngliche Mail -----
> > [snip]
> >
> > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > instead of using ubiblock :-)
> >
> > Can we patch either Kconfig or add some warn_once on mtdblock
> > usage, suggesting to use ubiblock instead?
>
> a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> a good choice.
> ubiblock is mostly useful for NAND flash.
>
> > I remember there was still some use case(s) for mtdblock but I can't remember
> > now what was it, perhaps we should document the expectations?
> > (Is that for JFFS2 to mount?)
>
> a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> JFFS2.
> This limitation is gone.
>

OK, let me try to cook a patch for you.

Eze

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
> >
> > Ezequiel,
> >
> > ----- Ursprüngliche Mail -----
> > > [snip]
> > >
> > > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > > instead of using ubiblock :-)
> > >
> > > Can we patch either Kconfig or add some warn_once on mtdblock
> > > usage, suggesting to use ubiblock instead?
> >
> > a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> > A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> > a good choice.
> > ubiblock is mostly useful for NAND flash.
> >
> > > I remember there was still some use case(s) for mtdblock but I can't remember
> > > now what was it, perhaps we should document the expectations?
> > > (Is that for JFFS2 to mount?)
> >
> > a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> > JFFS2.
> > This limitation is gone.
> >
>
> OK, let me try to cook a patch for you.
>

Dear Eze and Richard,

First of all, thank you so much for all your replies so far.
Sorry, I have limited knowledge about NAND, MTD, UBI layers, but my
current work involves all these so I am here.
But I will surely share this information to our internal team about
using ubiblock instead of mtdblock.

However, I still fail to understand the problem and consequences of
using mtdblock for rootfs instead of ubiblock.
Last time, for my squashfs test, when I tried to replace the command
line with ubiblock, I could not see any difference.
How to visibly see the difference so that I can easily understand and
explain the difference internally?
Or, is there a document available somewhere to highlight the
difference between the two?

BTW, we have few raw nand partitions and few ubi volumes [including
rootfs(squashfs), data(ubifs, rw)]
So, I guess we should use ubiblock for all ubi volumes?

Regarding Kconfig, I have few opinions.
Yes, adding more description in Kconfig is a good choice.
But I see that most of the time these kernel config options remains
(as default) and platform developers never cares about the
description.
So, how to better create awareness among them, not to make these mistakes?
One option is to capture these details as part of Kernel documentation.
Right now when I search I could not find anything.
Another option is to add a few (critical) warnings in kernel bootup
logs to give some hint to the developer that there is probably
something wrong.
This will directly be visible to all and some developers will not like
to ignore it.
Or, maybe adding both options is also good.


Thanks,
Pintu

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
> >
> > Ezequiel,
> >
> > ----- Ursprüngliche Mail -----
> > > [snip]
> > >
> > > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > > instead of using ubiblock :-)
> > >
> > > Can we patch either Kconfig or add some warn_once on mtdblock
> > > usage, suggesting to use ubiblock instead?
> >
> > a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> > A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> > a good choice.
> > ubiblock is mostly useful for NAND flash.
> >
> > > I remember there was still some use case(s) for mtdblock but I can't remember
> > > now what was it, perhaps we should document the expectations?
> > > (Is that for JFFS2 to mount?)
> >
> > a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> > JFFS2.
> > This limitation is gone.
> >
>
> OK, let me try to cook a patch for you.
>

Dear Eze and Richard,

First of all, thank you so much for all your replies so far.
Sorry, I have limited knowledge about NAND, MTD, UBI layers, but my
current work involves all these so I am here.
But I will surely share this information to our internal team about
using ubiblock instead of mtdblock.

However, I still fail to understand the problem and consequences of
using mtdblock for rootfs instead of ubiblock.
Last time, for my squashfs test, when I tried to replace the command
line with ubiblock, I could not see any difference.
How to visibly see the difference so that I can easily understand and
explain the difference internally?
Or, is there a document available somewhere to highlight the
difference between the two?

BTW, we have few raw nand partitions and few ubi volumes [including
rootfs(squashfs), data(ubifs, rw)]
So, I guess we should use ubiblock for all ubi volumes?

Regarding Kconfig, I have few opinions.
Yes, adding more description in Kconfig is a good choice.
But I see that most of the time these kernel config options remains
(as default) and platform developers never cares about the
description.
So, how to better create awareness among them, not to make these mistakes?
One option is to capture these details as part of Kernel documentation.
Right now when I search I could not find anything.
Another option is to add a few (critical) warnings in kernel bootup
logs to give some hint to the developer that there is probably
something wrong.
This will directly be visible to all and some developers will not like
to ignore it.
Or, maybe adding both options is also good.


Thanks,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
> >
> > Ezequiel,
> >
> > ----- Ursprüngliche Mail -----
> > > [snip]
> > >
> > > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > > instead of using ubiblock :-)
> > >
> > > Can we patch either Kconfig or add some warn_once on mtdblock
> > > usage, suggesting to use ubiblock instead?
> >
> > a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> > A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> > a good choice.
> > ubiblock is mostly useful for NAND flash.
> >
> > > I remember there was still some use case(s) for mtdblock but I can't remember
> > > now what was it, perhaps we should document the expectations?
> > > (Is that for JFFS2 to mount?)
> >
> > a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> > JFFS2.
> > This limitation is gone.
> >
>
> OK, let me try to cook a patch for you.
>

Dear Eze and Richard,

First of all, thank you so much for all your replies so far.
Sorry, I have limited knowledge about NAND, MTD, UBI layers, but my
current work involves all these so I am here.
But I will surely share this information to our internal team about
using ubiblock instead of mtdblock.

However, I still fail to understand the problem and consequences of
using mtdblock for rootfs instead of ubiblock.
Last time, for my squashfs test, when I tried to replace the command
line with ubiblock, I could not see any difference.
How to visibly see the difference so that I can easily understand and
explain the difference internally?
Or, is there a document available somewhere to highlight the
difference between the two?

BTW, we have few raw nand partitions and few ubi volumes [including
rootfs(squashfs), data(ubifs, rw)]
So, I guess we should use ubiblock for all ubi volumes?

Regarding Kconfig, I have few opinions.
Yes, adding more description in Kconfig is a good choice.
But I see that most of the time these kernel config options remains
(as default) and platform developers never cares about the
description.
So, how to better create awareness among them, not to make these mistakes?
One option is to capture these details as part of Kernel documentation.
Right now when I search I could not find anything.
Another option is to add a few (critical) warnings in kernel bootup
logs to give some hint to the developer that there is probably
something wrong.
This will directly be visible to all and some developers will not like
to ignore it.
Or, maybe adding both options is also good.


Thanks,
Pintu

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Thu, 29 Jul 2021 at 22:41, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
> > >
> > > Ezequiel,
> > >
> > > ----- Ursprüngliche Mail -----
> > > > [snip]
> > > >
> > > > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > > > instead of using ubiblock :-)
> > > >
> > > > Can we patch either Kconfig or add some warn_once on mtdblock
> > > > usage, suggesting to use ubiblock instead?
> > >
> > > a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> > > A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> > > a good choice.
> > > ubiblock is mostly useful for NAND flash.
> > >
> > > > I remember there was still some use case(s) for mtdblock but I can't remember
> > > > now what was it, perhaps we should document the expectations?
> > > > (Is that for JFFS2 to mount?)
> > >
> > > a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> > > JFFS2.
> > > This limitation is gone.
> > >

Hi,

Just a further follow-up on this discussion.
Whether to use /dev/mtdblock or /dev/ubiblock for rootfs (squashfs)
mounting during boot.

As suggested here:
Instead of using this in kernel command line:
[    0.000000] Kernel command line: ... rootfstype=squashfs
root=/dev/mtdblock44 ubi.mtd=40,0,30 ...

I used this:
[    0.000000] Kernel command line: ... rootfstype=squashfs
ubi.mtd=40,0,30 ubi.block=0,0 root=/dev/ubiblock0_0 ...

The device is booting fine with ubiblock as well.
But, per say, I could not find any visible difference.
I just observed a slight improvement in boot time, but I need to
double-check on this, with few more reboot cycles.

Apart from this what are the other visible benefits of using ubiblock
which can be explained to be management or internal team ?
I could not find any documentation explaining the difference, except this one:
http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock

Can someone also point me to the respective driver code in case of
using /dev/mtdblock and /dev/ubiblock ?
Apart from theory I also want to check the impact at the code level..

Thanks,
Pintu

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Thu, 29 Jul 2021 at 22:41, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
> > >
> > > Ezequiel,
> > >
> > > ----- Ursprüngliche Mail -----
> > > > [snip]
> > > >
> > > > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > > > instead of using ubiblock :-)
> > > >
> > > > Can we patch either Kconfig or add some warn_once on mtdblock
> > > > usage, suggesting to use ubiblock instead?
> > >
> > > a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> > > A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> > > a good choice.
> > > ubiblock is mostly useful for NAND flash.
> > >
> > > > I remember there was still some use case(s) for mtdblock but I can't remember
> > > > now what was it, perhaps we should document the expectations?
> > > > (Is that for JFFS2 to mount?)
> > >
> > > a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> > > JFFS2.
> > > This limitation is gone.
> > >

Hi,

Just a further follow-up on this discussion.
Whether to use /dev/mtdblock or /dev/ubiblock for rootfs (squashfs)
mounting during boot.

As suggested here:
Instead of using this in kernel command line:
[    0.000000] Kernel command line: ... rootfstype=squashfs
root=/dev/mtdblock44 ubi.mtd=40,0,30 ...

I used this:
[    0.000000] Kernel command line: ... rootfstype=squashfs
ubi.mtd=40,0,30 ubi.block=0,0 root=/dev/ubiblock0_0 ...

The device is booting fine with ubiblock as well.
But, per say, I could not find any visible difference.
I just observed a slight improvement in boot time, but I need to
double-check on this, with few more reboot cycles.

Apart from this what are the other visible benefits of using ubiblock
which can be explained to be management or internal team ?
I could not find any documentation explaining the difference, except this one:
http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock

Can someone also point me to the respective driver code in case of
using /dev/mtdblock and /dev/ubiblock ?
Apart from theory I also want to check the impact at the code level..

Thanks,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Thu, 29 Jul 2021 at 22:41, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
> > >
> > > Ezequiel,
> > >
> > > ----- Ursprüngliche Mail -----
> > > > [snip]
> > > >
> > > > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > > > instead of using ubiblock :-)
> > > >
> > > > Can we patch either Kconfig or add some warn_once on mtdblock
> > > > usage, suggesting to use ubiblock instead?
> > >
> > > a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> > > A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> > > a good choice.
> > > ubiblock is mostly useful for NAND flash.
> > >
> > > > I remember there was still some use case(s) for mtdblock but I can't remember
> > > > now what was it, perhaps we should document the expectations?
> > > > (Is that for JFFS2 to mount?)
> > >
> > > a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> > > JFFS2.
> > > This limitation is gone.
> > >

Hi,

Just a further follow-up on this discussion.
Whether to use /dev/mtdblock or /dev/ubiblock for rootfs (squashfs)
mounting during boot.

As suggested here:
Instead of using this in kernel command line:
[    0.000000] Kernel command line: ... rootfstype=squashfs
root=/dev/mtdblock44 ubi.mtd=40,0,30 ...

I used this:
[    0.000000] Kernel command line: ... rootfstype=squashfs
ubi.mtd=40,0,30 ubi.block=0,0 root=/dev/ubiblock0_0 ...

The device is booting fine with ubiblock as well.
But, per say, I could not find any visible difference.
I just observed a slight improvement in boot time, but I need to
double-check on this, with few more reboot cycles.

Apart from this what are the other visible benefits of using ubiblock
which can be explained to be management or internal team ?
I could not find any documentation explaining the difference, except this one:
http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock

Can someone also point me to the respective driver code in case of
using /dev/mtdblock and /dev/ubiblock ?
Apart from theory I also want to check the impact at the code level..

Thanks,
Pintu

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

Hi Pintu,

On Fri, 20 Aug 2021 at 15:25, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Thu, 29 Jul 2021 at 22:41, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia
> > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
> > > >
> > > > Ezequiel,
> > > >
> > > > ----- Ursprüngliche Mail -----
> > > > > [snip]
> > > > >
> > > > > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > > > > instead of using ubiblock :-)
> > > > >
> > > > > Can we patch either Kconfig or add some warn_once on mtdblock
> > > > > usage, suggesting to use ubiblock instead?
> > > >
> > > > a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> > > > A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> > > > a good choice.
> > > > ubiblock is mostly useful for NAND flash.
> > > >
> > > > > I remember there was still some use case(s) for mtdblock but I can't remember
> > > > > now what was it, perhaps we should document the expectations?
> > > > > (Is that for JFFS2 to mount?)
> > > >
> > > > a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> > > > JFFS2.
> > > > This limitation is gone.
> > > >
>
> Hi,
>
> Just a further follow-up on this discussion.
> Whether to use /dev/mtdblock or /dev/ubiblock for rootfs (squashfs)
> mounting during boot.
>
> As suggested here:
> Instead of using this in kernel command line:
> [    0.000000] Kernel command line: ... rootfstype=squashfs
> root=/dev/mtdblock44 ubi.mtd=40,0,30 ...
>
> I used this:
> [    0.000000] Kernel command line: ... rootfstype=squashfs
> ubi.mtd=40,0,30 ubi.block=0,0 root=/dev/ubiblock0_0 ...
>
> The device is booting fine with ubiblock as well.
> But, per say, I could not find any visible difference.
> I just observed a slight improvement in boot time, but I need to
> double-check on this, with few more reboot cycles.
>

That's a very good thing, it means we offered you a smooth transition :-)

> Apart from this what are the other visible benefits of using ubiblock
> which can be explained to be management or internal team ?
> I could not find any documentation explaining the difference, except this one:
> http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
>

I'm not a flash expert here. In any case, you are expected to do your own
research (just like we all did), design your own setup matching
your use-case, design tests based on your workload and access patterns, etc.

There are presentations on YouTube which discuss UBI, UBIFS and
NAND-based designs on Linux, as well as white papers discussing
NAND flashes challenges.

Having said that...

When you use UBI block, you are accessing the flash via the UBI layer.
This is IMO the best way to design your system, since UBI addresses
wear leveling and bad blocks, and offers atomic updates.

In other words, IMO it's best to expose the NAND through UBI
for both read-only and read-write access, using a single UBI device,
and then creating UBI volumes as needed. This will allow UBI
to spread wear leveling across the whole device, which is expected
to increase the flash lifetime.

For instance, just as some silly example, you could have something like this:

                               | RootFS SquashFS  |
                               | UBI block        | UBIFS User R-W area
------------------------------------------------------------------------
Kernel A | Kernel B | RootFS A | RootFS B         | User
------------------------------------------------------------------------
                                 UBIX
------------------------------------------------------------------------
                                 /dev/mtdX

This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
via SquashFS and there's a read-write user area. UBI is supporting all
the volumes, handling bad blocks and wear leveling.

> Can someone also point me to the respective driver code in case of
> using /dev/mtdblock and /dev/ubiblock ?
> Apart from theory I also want to check the impact at the code level..
>

You can find all the UBI code in drivers/mtd/ubi of course. The differences
between mtdblock and ubiblock are huge: one goes directly to the flash,
and the other uses UBI.

Good luck!
Ezequiel

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

Hi Pintu,

On Fri, 20 Aug 2021 at 15:25, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Thu, 29 Jul 2021 at 22:41, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia
> > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
> > > >
> > > > Ezequiel,
> > > >
> > > > ----- Ursprüngliche Mail -----
> > > > > [snip]
> > > > >
> > > > > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > > > > instead of using ubiblock :-)
> > > > >
> > > > > Can we patch either Kconfig or add some warn_once on mtdblock
> > > > > usage, suggesting to use ubiblock instead?
> > > >
> > > > a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> > > > A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> > > > a good choice.
> > > > ubiblock is mostly useful for NAND flash.
> > > >
> > > > > I remember there was still some use case(s) for mtdblock but I can't remember
> > > > > now what was it, perhaps we should document the expectations?
> > > > > (Is that for JFFS2 to mount?)
> > > >
> > > > a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> > > > JFFS2.
> > > > This limitation is gone.
> > > >
>
> Hi,
>
> Just a further follow-up on this discussion.
> Whether to use /dev/mtdblock or /dev/ubiblock for rootfs (squashfs)
> mounting during boot.
>
> As suggested here:
> Instead of using this in kernel command line:
> [    0.000000] Kernel command line: ... rootfstype=squashfs
> root=/dev/mtdblock44 ubi.mtd=40,0,30 ...
>
> I used this:
> [    0.000000] Kernel command line: ... rootfstype=squashfs
> ubi.mtd=40,0,30 ubi.block=0,0 root=/dev/ubiblock0_0 ...
>
> The device is booting fine with ubiblock as well.
> But, per say, I could not find any visible difference.
> I just observed a slight improvement in boot time, but I need to
> double-check on this, with few more reboot cycles.
>

That's a very good thing, it means we offered you a smooth transition :-)

> Apart from this what are the other visible benefits of using ubiblock
> which can be explained to be management or internal team ?
> I could not find any documentation explaining the difference, except this one:
> http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
>

I'm not a flash expert here. In any case, you are expected to do your own
research (just like we all did), design your own setup matching
your use-case, design tests based on your workload and access patterns, etc.

There are presentations on YouTube which discuss UBI, UBIFS and
NAND-based designs on Linux, as well as white papers discussing
NAND flashes challenges.

Having said that...

When you use UBI block, you are accessing the flash via the UBI layer.
This is IMO the best way to design your system, since UBI addresses
wear leveling and bad blocks, and offers atomic updates.

In other words, IMO it's best to expose the NAND through UBI
for both read-only and read-write access, using a single UBI device,
and then creating UBI volumes as needed. This will allow UBI
to spread wear leveling across the whole device, which is expected
to increase the flash lifetime.

For instance, just as some silly example, you could have something like this:

                               | RootFS SquashFS  |
                               | UBI block        | UBIFS User R-W area
------------------------------------------------------------------------
Kernel A | Kernel B | RootFS A | RootFS B         | User
------------------------------------------------------------------------
                                 UBIX
------------------------------------------------------------------------
                                 /dev/mtdX

This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
via SquashFS and there's a read-write user area. UBI is supporting all
the volumes, handling bad blocks and wear leveling.

> Can someone also point me to the respective driver code in case of
> using /dev/mtdblock and /dev/ubiblock ?
> Apart from theory I also want to check the impact at the code level..
>

You can find all the UBI code in drivers/mtd/ubi of course. The differences
between mtdblock and ubiblock are huge: one goes directly to the flash,
and the other uses UBI.

Good luck!
Ezequiel

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

Hi Pintu,

On Fri, 20 Aug 2021 at 15:25, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Thu, 29 Jul 2021 at 22:41, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > On Thu, 29 Jul 2021 at 17:33, Ezequiel Garcia
> > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > On Thu, 29 Jul 2021 at 08:45, Richard Weinberger <richard@nod.at> wrote:
> > > >
> > > > Ezequiel,
> > > >
> > > > ----- Ursprüngliche Mail -----
> > > > > [snip]
> > > > >
> > > > > Ouch, so surprised that after all these years someone is doing squashfs/mtdblock
> > > > > instead of using ubiblock :-)
> > > > >
> > > > > Can we patch either Kconfig or add some warn_once on mtdblock
> > > > > usage, suggesting to use ubiblock instead?
> > > >
> > > > a hint in Kconfig makes IMHO sense. Do you want to send a patch?
> > > > A warning is too much since on some tiny embedded system with NOR flash mtdblock is still
> > > > a good choice.
> > > > ubiblock is mostly useful for NAND flash.
> > > >
> > > > > I remember there was still some use case(s) for mtdblock but I can't remember
> > > > > now what was it, perhaps we should document the expectations?
> > > > > (Is that for JFFS2 to mount?)
> > > >
> > > > a long time ago mount didn't accept character devices, so you had to pass mtdblockX to mount
> > > > JFFS2.
> > > > This limitation is gone.
> > > >
>
> Hi,
>
> Just a further follow-up on this discussion.
> Whether to use /dev/mtdblock or /dev/ubiblock for rootfs (squashfs)
> mounting during boot.
>
> As suggested here:
> Instead of using this in kernel command line:
> [    0.000000] Kernel command line: ... rootfstype=squashfs
> root=/dev/mtdblock44 ubi.mtd=40,0,30 ...
>
> I used this:
> [    0.000000] Kernel command line: ... rootfstype=squashfs
> ubi.mtd=40,0,30 ubi.block=0,0 root=/dev/ubiblock0_0 ...
>
> The device is booting fine with ubiblock as well.
> But, per say, I could not find any visible difference.
> I just observed a slight improvement in boot time, but I need to
> double-check on this, with few more reboot cycles.
>

That's a very good thing, it means we offered you a smooth transition :-)

> Apart from this what are the other visible benefits of using ubiblock
> which can be explained to be management or internal team ?
> I could not find any documentation explaining the difference, except this one:
> http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock
>

I'm not a flash expert here. In any case, you are expected to do your own
research (just like we all did), design your own setup matching
your use-case, design tests based on your workload and access patterns, etc.

There are presentations on YouTube which discuss UBI, UBIFS and
NAND-based designs on Linux, as well as white papers discussing
NAND flashes challenges.

Having said that...

When you use UBI block, you are accessing the flash via the UBI layer.
This is IMO the best way to design your system, since UBI addresses
wear leveling and bad blocks, and offers atomic updates.

In other words, IMO it's best to expose the NAND through UBI
for both read-only and read-write access, using a single UBI device,
and then creating UBI volumes as needed. This will allow UBI
to spread wear leveling across the whole device, which is expected
to increase the flash lifetime.

For instance, just as some silly example, you could have something like this:

                               | RootFS SquashFS  |
                               | UBI block        | UBIFS User R-W area
------------------------------------------------------------------------
Kernel A | Kernel B | RootFS A | RootFS B         | User
------------------------------------------------------------------------
                                 UBIX
------------------------------------------------------------------------
                                 /dev/mtdX

This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
via SquashFS and there's a read-write user area. UBI is supporting all
the volumes, handling bad blocks and wear leveling.

> Can someone also point me to the respective driver code in case of
> using /dev/mtdblock and /dev/ubiblock ?
> Apart from theory I also want to check the impact at the code level..
>

You can find all the UBI code in drivers/mtd/ubi of course. The differences
between mtdblock and ubiblock are huge: one goes directly to the flash,
and the other uses UBI.

Good luck!
Ezequiel

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:

> In other words, IMO it's best to expose the NAND through UBI
> for both read-only and read-write access, using a single UBI device,
> and then creating UBI volumes as needed. This will allow UBI
> to spread wear leveling across the whole device, which is expected
> to increase the flash lifetime.
>
> For instance, just as some silly example, you could have something like this:
>
>                                | RootFS SquashFS  |
>                                | UBI block        | UBIFS User R-W area
> ------------------------------------------------------------------------
> Kernel A | Kernel B | RootFS A | RootFS B         | User
> ------------------------------------------------------------------------
>                                  UBIX
> ------------------------------------------------------------------------
>                                  /dev/mtdX
>
> This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> via SquashFS and there's a read-write user area. UBI is supporting all
> the volumes, handling bad blocks and wear leveling.
>
Dear Ezequiel,
Thank you so much for your reply.

This is exactly what we are also doing :)
In our system we have a mix of raw and ubi partitions.
The ubi partitioning is done almost exactly the same way.
Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
mount the rootfs.
Now, I understood we should change it to use /dev/ubiblock<id>
This might have several benefits, but one most important could be,
using ubiblock can handle bad-blocks/wear-leveling automatically,
whereas mtdblocks access the flash directly ?
I found some references for these..
So, this seems good for my proposal.

Another thing that is still open for us is:
How do we calculate the exact image size from a raw mtd partition ?
For example, support for one of the raw nand partitions, the size is
defined as 15MB but we flash the actual image of size only 2.5MB.
So, in the runtime how to determine the image size as ~2.5MB (at least
roughly) ?
Is it still possible ?


Thanks,
Pintu

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:

> In other words, IMO it's best to expose the NAND through UBI
> for both read-only and read-write access, using a single UBI device,
> and then creating UBI volumes as needed. This will allow UBI
> to spread wear leveling across the whole device, which is expected
> to increase the flash lifetime.
>
> For instance, just as some silly example, you could have something like this:
>
>                                | RootFS SquashFS  |
>                                | UBI block        | UBIFS User R-W area
> ------------------------------------------------------------------------
> Kernel A | Kernel B | RootFS A | RootFS B         | User
> ------------------------------------------------------------------------
>                                  UBIX
> ------------------------------------------------------------------------
>                                  /dev/mtdX
>
> This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> via SquashFS and there's a read-write user area. UBI is supporting all
> the volumes, handling bad blocks and wear leveling.
>
Dear Ezequiel,
Thank you so much for your reply.

This is exactly what we are also doing :)
In our system we have a mix of raw and ubi partitions.
The ubi partitioning is done almost exactly the same way.
Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
mount the rootfs.
Now, I understood we should change it to use /dev/ubiblock<id>
This might have several benefits, but one most important could be,
using ubiblock can handle bad-blocks/wear-leveling automatically,
whereas mtdblocks access the flash directly ?
I found some references for these..
So, this seems good for my proposal.

Another thing that is still open for us is:
How do we calculate the exact image size from a raw mtd partition ?
For example, support for one of the raw nand partitions, the size is
defined as 15MB but we flash the actual image of size only 2.5MB.
So, in the runtime how to determine the image size as ~2.5MB (at least
roughly) ?
Is it still possible ?


Thanks,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:

> In other words, IMO it's best to expose the NAND through UBI
> for both read-only and read-write access, using a single UBI device,
> and then creating UBI volumes as needed. This will allow UBI
> to spread wear leveling across the whole device, which is expected
> to increase the flash lifetime.
>
> For instance, just as some silly example, you could have something like this:
>
>                                | RootFS SquashFS  |
>                                | UBI block        | UBIFS User R-W area
> ------------------------------------------------------------------------
> Kernel A | Kernel B | RootFS A | RootFS B         | User
> ------------------------------------------------------------------------
>                                  UBIX
> ------------------------------------------------------------------------
>                                  /dev/mtdX
>
> This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> via SquashFS and there's a read-write user area. UBI is supporting all
> the volumes, handling bad blocks and wear leveling.
>
Dear Ezequiel,
Thank you so much for your reply.

This is exactly what we are also doing :)
In our system we have a mix of raw and ubi partitions.
The ubi partitioning is done almost exactly the same way.
Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
mount the rootfs.
Now, I understood we should change it to use /dev/ubiblock<id>
This might have several benefits, but one most important could be,
using ubiblock can handle bad-blocks/wear-leveling automatically,
whereas mtdblocks access the flash directly ?
I found some references for these..
So, this seems good for my proposal.

Another thing that is still open for us is:
How do we calculate the exact image size from a raw mtd partition ?
For example, support for one of the raw nand partitions, the size is
defined as 15MB but we flash the actual image of size only 2.5MB.
So, in the runtime how to determine the image size as ~2.5MB (at least
roughly) ?
Is it still possible ?


Thanks,
Pintu

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

Hi All,

On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
>
> > In other words, IMO it's best to expose the NAND through UBI
> > for both read-only and read-write access, using a single UBI device,
> > and then creating UBI volumes as needed. This will allow UBI
> > to spread wear leveling across the whole device, which is expected
> > to increase the flash lifetime.
> >
> > For instance, just as some silly example, you could have something like this:
> >
> >                                | RootFS SquashFS  |
> >                                | UBI block        | UBIFS User R-W area
> > ------------------------------------------------------------------------
> > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > ------------------------------------------------------------------------
> >                                  UBIX
> > ------------------------------------------------------------------------
> >                                  /dev/mtdX
> >
> > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > via SquashFS and there's a read-write user area. UBI is supporting all
> > the volumes, handling bad blocks and wear leveling.
> >
> Dear Ezequiel,
> Thank you so much for your reply.
>
> This is exactly what we are also doing :)
> In our system we have a mix of raw and ubi partitions.
> The ubi partitioning is done almost exactly the same way.
> Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> mount the rootfs.
> Now, I understood we should change it to use /dev/ubiblock<id>
> This might have several benefits, but one most important could be,
> using ubiblock can handle bad-blocks/wear-leveling automatically,
> whereas mtdblocks access the flash directly ?
> I found some references for these..
> So, this seems good for my proposal.
>
> Another thing that is still open for us is:
> How do we calculate the exact image size from a raw mtd partition ?
> For example, support for one of the raw nand partitions, the size is
> defined as 15MB but we flash the actual image of size only 2.5MB.
> So, in the runtime how to determine the image size as ~2.5MB (at least
> roughly) ?
> Is it still possible ?
>

I am happy to inform you that using "ubiblock" for squashfs mounting
seems very helpful for us.
We have seen almost the double performance boost when using ubiblock
for rootfs as well as other read-only volume mounting.

However, we have found few issues while defining the read only volume as STATIC.
With static volume we see that OTA update is failing during "fsync".
That is ota_fsync is failing from here:
https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
int status = fsync(fd);
if (status == -1 && errno == EIO)
*
{ have_eio_error = true; }
*
return status;
}

Is this the known issue with static volume?

For now we are using dynamic volume itself but the problem is that
with dynamic volume we cannot get the exact image size from:
$ cat /sys/class/ubi/ubi0_0/data_bytes
==> In case of dynamic volume this will return the total volume size.
==> Thus our md5 integrity check does not match exactly with the
flashed image size.

Is there an alternate way to handle this issue ?


Thanks,
Pintu

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

Hi All,

On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
>
> > In other words, IMO it's best to expose the NAND through UBI
> > for both read-only and read-write access, using a single UBI device,
> > and then creating UBI volumes as needed. This will allow UBI
> > to spread wear leveling across the whole device, which is expected
> > to increase the flash lifetime.
> >
> > For instance, just as some silly example, you could have something like this:
> >
> >                                | RootFS SquashFS  |
> >                                | UBI block        | UBIFS User R-W area
> > ------------------------------------------------------------------------
> > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > ------------------------------------------------------------------------
> >                                  UBIX
> > ------------------------------------------------------------------------
> >                                  /dev/mtdX
> >
> > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > via SquashFS and there's a read-write user area. UBI is supporting all
> > the volumes, handling bad blocks and wear leveling.
> >
> Dear Ezequiel,
> Thank you so much for your reply.
>
> This is exactly what we are also doing :)
> In our system we have a mix of raw and ubi partitions.
> The ubi partitioning is done almost exactly the same way.
> Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> mount the rootfs.
> Now, I understood we should change it to use /dev/ubiblock<id>
> This might have several benefits, but one most important could be,
> using ubiblock can handle bad-blocks/wear-leveling automatically,
> whereas mtdblocks access the flash directly ?
> I found some references for these..
> So, this seems good for my proposal.
>
> Another thing that is still open for us is:
> How do we calculate the exact image size from a raw mtd partition ?
> For example, support for one of the raw nand partitions, the size is
> defined as 15MB but we flash the actual image of size only 2.5MB.
> So, in the runtime how to determine the image size as ~2.5MB (at least
> roughly) ?
> Is it still possible ?
>

I am happy to inform you that using "ubiblock" for squashfs mounting
seems very helpful for us.
We have seen almost the double performance boost when using ubiblock
for rootfs as well as other read-only volume mounting.

However, we have found few issues while defining the read only volume as STATIC.
With static volume we see that OTA update is failing during "fsync".
That is ota_fsync is failing from here:
https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
int status = fsync(fd);
if (status == -1 && errno == EIO)
*
{ have_eio_error = true; }
*
return status;
}

Is this the known issue with static volume?

For now we are using dynamic volume itself but the problem is that
with dynamic volume we cannot get the exact image size from:
$ cat /sys/class/ubi/ubi0_0/data_bytes
==> In case of dynamic volume this will return the total volume size.
==> Thus our md5 integrity check does not match exactly with the
flashed image size.

Is there an alternate way to handle this issue ?


Thanks,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

Hi All,

On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
>
> > In other words, IMO it's best to expose the NAND through UBI
> > for both read-only and read-write access, using a single UBI device,
> > and then creating UBI volumes as needed. This will allow UBI
> > to spread wear leveling across the whole device, which is expected
> > to increase the flash lifetime.
> >
> > For instance, just as some silly example, you could have something like this:
> >
> >                                | RootFS SquashFS  |
> >                                | UBI block        | UBIFS User R-W area
> > ------------------------------------------------------------------------
> > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > ------------------------------------------------------------------------
> >                                  UBIX
> > ------------------------------------------------------------------------
> >                                  /dev/mtdX
> >
> > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > via SquashFS and there's a read-write user area. UBI is supporting all
> > the volumes, handling bad blocks and wear leveling.
> >
> Dear Ezequiel,
> Thank you so much for your reply.
>
> This is exactly what we are also doing :)
> In our system we have a mix of raw and ubi partitions.
> The ubi partitioning is done almost exactly the same way.
> Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> mount the rootfs.
> Now, I understood we should change it to use /dev/ubiblock<id>
> This might have several benefits, but one most important could be,
> using ubiblock can handle bad-blocks/wear-leveling automatically,
> whereas mtdblocks access the flash directly ?
> I found some references for these..
> So, this seems good for my proposal.
>
> Another thing that is still open for us is:
> How do we calculate the exact image size from a raw mtd partition ?
> For example, support for one of the raw nand partitions, the size is
> defined as 15MB but we flash the actual image of size only 2.5MB.
> So, in the runtime how to determine the image size as ~2.5MB (at least
> roughly) ?
> Is it still possible ?
>

I am happy to inform you that using "ubiblock" for squashfs mounting
seems very helpful for us.
We have seen almost the double performance boost when using ubiblock
for rootfs as well as other read-only volume mounting.

However, we have found few issues while defining the read only volume as STATIC.
With static volume we see that OTA update is failing during "fsync".
That is ota_fsync is failing from here:
https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
int status = fsync(fd);
if (status == -1 && errno == EIO)
*
{ have_eio_error = true; }
*
return status;
}

Is this the known issue with static volume?

For now we are using dynamic volume itself but the problem is that
with dynamic volume we cannot get the exact image size from:
$ cat /sys/class/ubi/ubi0_0/data_bytes
==> In case of dynamic volume this will return the total volume size.
==> Thus our md5 integrity check does not match exactly with the
flashed image size.

Is there an alternate way to handle this issue ?


Thanks,
Pintu

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> Hi All,
>
> On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > > In other words, IMO it's best to expose the NAND through UBI
> > > for both read-only and read-write access, using a single UBI device,
> > > and then creating UBI volumes as needed. This will allow UBI
> > > to spread wear leveling across the whole device, which is expected
> > > to increase the flash lifetime.
> > >
> > > For instance, just as some silly example, you could have something like this:
> > >
> > >                                | RootFS SquashFS  |
> > >                                | UBI block        | UBIFS User R-W area
> > > ------------------------------------------------------------------------
> > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > ------------------------------------------------------------------------
> > >                                  UBIX
> > > ------------------------------------------------------------------------
> > >                                  /dev/mtdX
> > >
> > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > the volumes, handling bad blocks and wear leveling.
> > >
> > Dear Ezequiel,
> > Thank you so much for your reply.
> >
> > This is exactly what we are also doing :)
> > In our system we have a mix of raw and ubi partitions.
> > The ubi partitioning is done almost exactly the same way.
> > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > mount the rootfs.
> > Now, I understood we should change it to use /dev/ubiblock<id>
> > This might have several benefits, but one most important could be,
> > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > whereas mtdblocks access the flash directly ?
> > I found some references for these..
> > So, this seems good for my proposal.
> >
> > Another thing that is still open for us is:
> > How do we calculate the exact image size from a raw mtd partition ?
> > For example, support for one of the raw nand partitions, the size is
> > defined as 15MB but we flash the actual image of size only 2.5MB.
> > So, in the runtime how to determine the image size as ~2.5MB (at least
> > roughly) ?
> > Is it still possible ?
> >
>
> I am happy to inform you that using "ubiblock" for squashfs mounting
> seems very helpful for us.
> We have seen almost the double performance boost when using ubiblock
> for rootfs as well as other read-only volume mounting.
>
> However, we have found few issues while defining the read only volume as STATIC.
> With static volume we see that OTA update is failing during "fsync".
> That is ota_fsync is failing from here:
> https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> int status = fsync(fd);
> if (status == -1 && errno == EIO)
> *
> { have_eio_error = true; }
> *
> return status;
> }
>
> Is this the known issue with static volume?
>

I don't know exactly how you are updating your volume,
the right way is using UBI_IOCVOLUP.

See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate

If you google around I'm sure you'll find some articles about this,
but I'm not sure if they'll go into details and subtleties.

There are probably a few different ways to do firmware upgrade
when you are on top of static volumes (and you want to be on top
of static volumes if it's read-only, because AFAIK they give you an
extra data-integrity guarantee).

One way, would be to have two static volumes A/B. The system
uses normally the A volume, and then you doUBI_IOCVOLUP
(or ubiupdatevol) to update the B volume. After the update is succesful
you run the atomic volume rename and flip A->B, B->A.

(If you don't have enough space to hold two A/B volumes....
 ... you'll have to find some other solution, I have no idea about that.)

Hope it helps,
Eze

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> Hi All,
>
> On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > > In other words, IMO it's best to expose the NAND through UBI
> > > for both read-only and read-write access, using a single UBI device,
> > > and then creating UBI volumes as needed. This will allow UBI
> > > to spread wear leveling across the whole device, which is expected
> > > to increase the flash lifetime.
> > >
> > > For instance, just as some silly example, you could have something like this:
> > >
> > >                                | RootFS SquashFS  |
> > >                                | UBI block        | UBIFS User R-W area
> > > ------------------------------------------------------------------------
> > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > ------------------------------------------------------------------------
> > >                                  UBIX
> > > ------------------------------------------------------------------------
> > >                                  /dev/mtdX
> > >
> > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > the volumes, handling bad blocks and wear leveling.
> > >
> > Dear Ezequiel,
> > Thank you so much for your reply.
> >
> > This is exactly what we are also doing :)
> > In our system we have a mix of raw and ubi partitions.
> > The ubi partitioning is done almost exactly the same way.
> > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > mount the rootfs.
> > Now, I understood we should change it to use /dev/ubiblock<id>
> > This might have several benefits, but one most important could be,
> > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > whereas mtdblocks access the flash directly ?
> > I found some references for these..
> > So, this seems good for my proposal.
> >
> > Another thing that is still open for us is:
> > How do we calculate the exact image size from a raw mtd partition ?
> > For example, support for one of the raw nand partitions, the size is
> > defined as 15MB but we flash the actual image of size only 2.5MB.
> > So, in the runtime how to determine the image size as ~2.5MB (at least
> > roughly) ?
> > Is it still possible ?
> >
>
> I am happy to inform you that using "ubiblock" for squashfs mounting
> seems very helpful for us.
> We have seen almost the double performance boost when using ubiblock
> for rootfs as well as other read-only volume mounting.
>
> However, we have found few issues while defining the read only volume as STATIC.
> With static volume we see that OTA update is failing during "fsync".
> That is ota_fsync is failing from here:
> https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> int status = fsync(fd);
> if (status == -1 && errno == EIO)
> *
> { have_eio_error = true; }
> *
> return status;
> }
>
> Is this the known issue with static volume?
>

I don't know exactly how you are updating your volume,
the right way is using UBI_IOCVOLUP.

See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate

If you google around I'm sure you'll find some articles about this,
but I'm not sure if they'll go into details and subtleties.

There are probably a few different ways to do firmware upgrade
when you are on top of static volumes (and you want to be on top
of static volumes if it's read-only, because AFAIK they give you an
extra data-integrity guarantee).

One way, would be to have two static volumes A/B. The system
uses normally the A volume, and then you doUBI_IOCVOLUP
(or ubiupdatevol) to update the B volume. After the update is succesful
you run the atomic volume rename and flip A->B, B->A.

(If you don't have enough space to hold two A/B volumes....
 ... you'll have to find some other solution, I have no idea about that.)

Hope it helps,
Eze

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> Hi All,
>
> On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > > In other words, IMO it's best to expose the NAND through UBI
> > > for both read-only and read-write access, using a single UBI device,
> > > and then creating UBI volumes as needed. This will allow UBI
> > > to spread wear leveling across the whole device, which is expected
> > > to increase the flash lifetime.
> > >
> > > For instance, just as some silly example, you could have something like this:
> > >
> > >                                | RootFS SquashFS  |
> > >                                | UBI block        | UBIFS User R-W area
> > > ------------------------------------------------------------------------
> > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > ------------------------------------------------------------------------
> > >                                  UBIX
> > > ------------------------------------------------------------------------
> > >                                  /dev/mtdX
> > >
> > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > the volumes, handling bad blocks and wear leveling.
> > >
> > Dear Ezequiel,
> > Thank you so much for your reply.
> >
> > This is exactly what we are also doing :)
> > In our system we have a mix of raw and ubi partitions.
> > The ubi partitioning is done almost exactly the same way.
> > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > mount the rootfs.
> > Now, I understood we should change it to use /dev/ubiblock<id>
> > This might have several benefits, but one most important could be,
> > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > whereas mtdblocks access the flash directly ?
> > I found some references for these..
> > So, this seems good for my proposal.
> >
> > Another thing that is still open for us is:
> > How do we calculate the exact image size from a raw mtd partition ?
> > For example, support for one of the raw nand partitions, the size is
> > defined as 15MB but we flash the actual image of size only 2.5MB.
> > So, in the runtime how to determine the image size as ~2.5MB (at least
> > roughly) ?
> > Is it still possible ?
> >
>
> I am happy to inform you that using "ubiblock" for squashfs mounting
> seems very helpful for us.
> We have seen almost the double performance boost when using ubiblock
> for rootfs as well as other read-only volume mounting.
>
> However, we have found few issues while defining the read only volume as STATIC.
> With static volume we see that OTA update is failing during "fsync".
> That is ota_fsync is failing from here:
> https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> int status = fsync(fd);
> if (status == -1 && errno == EIO)
> *
> { have_eio_error = true; }
> *
> return status;
> }
>
> Is this the known issue with static volume?
>

I don't know exactly how you are updating your volume,
the right way is using UBI_IOCVOLUP.

See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate

If you google around I'm sure you'll find some articles about this,
but I'm not sure if they'll go into details and subtleties.

There are probably a few different ways to do firmware upgrade
when you are on top of static volumes (and you want to be on top
of static volumes if it's read-only, because AFAIK they give you an
extra data-integrity guarantee).

One way, would be to have two static volumes A/B. The system
uses normally the A volume, and then you doUBI_IOCVOLUP
(or ubiupdatevol) to update the B volume. After the update is succesful
you run the atomic volume rename and flip A->B, B->A.

(If you don't have enough space to hold two A/B volumes....
 ... you'll have to find some other solution, I have no idea about that.)

Hope it helps,
Eze

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

Hi,


On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > Hi All,
> >
> > On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > >
> > > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > > In other words, IMO it's best to expose the NAND through UBI
> > > > for both read-only and read-write access, using a single UBI device,
> > > > and then creating UBI volumes as needed. This will allow UBI
> > > > to spread wear leveling across the whole device, which is expected
> > > > to increase the flash lifetime.
> > > >
> > > > For instance, just as some silly example, you could have something like this:
> > > >
> > > >                                | RootFS SquashFS  |
> > > >                                | UBI block        | UBIFS User R-W area
> > > > ------------------------------------------------------------------------
> > > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > > ------------------------------------------------------------------------
> > > >                                  UBIX
> > > > ------------------------------------------------------------------------
> > > >                                  /dev/mtdX
> > > >
> > > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > > the volumes, handling bad blocks and wear leveling.
> > > >
> > > Dear Ezequiel,
> > > Thank you so much for your reply.
> > >
> > > This is exactly what we are also doing :)
> > > In our system we have a mix of raw and ubi partitions.
> > > The ubi partitioning is done almost exactly the same way.
> > > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > > mount the rootfs.
> > > Now, I understood we should change it to use /dev/ubiblock<id>
> > > This might have several benefits, but one most important could be,
> > > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > > whereas mtdblocks access the flash directly ?
> > > I found some references for these..
> > > So, this seems good for my proposal.
> > >
> > > Another thing that is still open for us is:
> > > How do we calculate the exact image size from a raw mtd partition ?
> > > For example, support for one of the raw nand partitions, the size is
> > > defined as 15MB but we flash the actual image of size only 2.5MB.
> > > So, in the runtime how to determine the image size as ~2.5MB (at least
> > > roughly) ?
> > > Is it still possible ?
> > >
> >
> > I am happy to inform you that using "ubiblock" for squashfs mounting
> > seems very helpful for us.
> > We have seen almost the double performance boost when using ubiblock
> > for rootfs as well as other read-only volume mounting.
> >
> > However, we have found few issues while defining the read only volume as STATIC.
> > With static volume we see that OTA update is failing during "fsync".
> > That is ota_fsync is failing from here:
> > https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> > int status = fsync(fd);
> > if (status == -1 && errno == EIO)
> > *
> > { have_eio_error = true; }
> > *
> > return status;
> > }
> >
> > Is this the known issue with static volume?
> >
>
> I don't know exactly how you are updating your volume,
> the right way is using UBI_IOCVOLUP.
>
> See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
>
> If you google around I'm sure you'll find some articles about this,
> but I'm not sure if they'll go into details and subtleties.
>
> There are probably a few different ways to do firmware upgrade
> when you are on top of static volumes (and you want to be on top
> of static volumes if it's read-only, because AFAIK they give you an
> extra data-integrity guarantee).
>
> One way, would be to have two static volumes A/B. The system
> uses normally the A volume, and then you doUBI_IOCVOLUP
> (or ubiupdatevol) to update the B volume. After the update is succesful
> you run the atomic volume rename and flip A->B, B->A.
>
> (If you don't have enough space to hold two A/B volumes....
>  ... you'll have to find some other solution, I have no idea about that.)
>

Yes, this is what we are also doing exactly.
But, currently we are running into this issue right now:
1) The FOTA update is failing if we use static volume (building and
flashing the static image is fine)
    It works fine, if we use the volume as dynamic (even for RO type
with squashfs)
    But with dynamic type we end up using the entire volume as image size.

So, we wanted to know if there is any other way to get volume image
size at runtime ?


Thanks,
Pintu

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

Hi,


On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > Hi All,
> >
> > On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > >
> > > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > > In other words, IMO it's best to expose the NAND through UBI
> > > > for both read-only and read-write access, using a single UBI device,
> > > > and then creating UBI volumes as needed. This will allow UBI
> > > > to spread wear leveling across the whole device, which is expected
> > > > to increase the flash lifetime.
> > > >
> > > > For instance, just as some silly example, you could have something like this:
> > > >
> > > >                                | RootFS SquashFS  |
> > > >                                | UBI block        | UBIFS User R-W area
> > > > ------------------------------------------------------------------------
> > > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > > ------------------------------------------------------------------------
> > > >                                  UBIX
> > > > ------------------------------------------------------------------------
> > > >                                  /dev/mtdX
> > > >
> > > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > > the volumes, handling bad blocks and wear leveling.
> > > >
> > > Dear Ezequiel,
> > > Thank you so much for your reply.
> > >
> > > This is exactly what we are also doing :)
> > > In our system we have a mix of raw and ubi partitions.
> > > The ubi partitioning is done almost exactly the same way.
> > > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > > mount the rootfs.
> > > Now, I understood we should change it to use /dev/ubiblock<id>
> > > This might have several benefits, but one most important could be,
> > > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > > whereas mtdblocks access the flash directly ?
> > > I found some references for these..
> > > So, this seems good for my proposal.
> > >
> > > Another thing that is still open for us is:
> > > How do we calculate the exact image size from a raw mtd partition ?
> > > For example, support for one of the raw nand partitions, the size is
> > > defined as 15MB but we flash the actual image of size only 2.5MB.
> > > So, in the runtime how to determine the image size as ~2.5MB (at least
> > > roughly) ?
> > > Is it still possible ?
> > >
> >
> > I am happy to inform you that using "ubiblock" for squashfs mounting
> > seems very helpful for us.
> > We have seen almost the double performance boost when using ubiblock
> > for rootfs as well as other read-only volume mounting.
> >
> > However, we have found few issues while defining the read only volume as STATIC.
> > With static volume we see that OTA update is failing during "fsync".
> > That is ota_fsync is failing from here:
> > https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> > int status = fsync(fd);
> > if (status == -1 && errno == EIO)
> > *
> > { have_eio_error = true; }
> > *
> > return status;
> > }
> >
> > Is this the known issue with static volume?
> >
>
> I don't know exactly how you are updating your volume,
> the right way is using UBI_IOCVOLUP.
>
> See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
>
> If you google around I'm sure you'll find some articles about this,
> but I'm not sure if they'll go into details and subtleties.
>
> There are probably a few different ways to do firmware upgrade
> when you are on top of static volumes (and you want to be on top
> of static volumes if it's read-only, because AFAIK they give you an
> extra data-integrity guarantee).
>
> One way, would be to have two static volumes A/B. The system
> uses normally the A volume, and then you doUBI_IOCVOLUP
> (or ubiupdatevol) to update the B volume. After the update is succesful
> you run the atomic volume rename and flip A->B, B->A.
>
> (If you don't have enough space to hold two A/B volumes....
>  ... you'll have to find some other solution, I have no idea about that.)
>

Yes, this is what we are also doing exactly.
But, currently we are running into this issue right now:
1) The FOTA update is failing if we use static volume (building and
flashing the static image is fine)
    It works fine, if we use the volume as dynamic (even for RO type
with squashfs)
    But with dynamic type we end up using the entire volume as image size.

So, we wanted to know if there is any other way to get volume image
size at runtime ?


Thanks,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Pintu Agarwal]

Hi,


On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > Hi All,
> >
> > On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > >
> > > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > > In other words, IMO it's best to expose the NAND through UBI
> > > > for both read-only and read-write access, using a single UBI device,
> > > > and then creating UBI volumes as needed. This will allow UBI
> > > > to spread wear leveling across the whole device, which is expected
> > > > to increase the flash lifetime.
> > > >
> > > > For instance, just as some silly example, you could have something like this:
> > > >
> > > >                                | RootFS SquashFS  |
> > > >                                | UBI block        | UBIFS User R-W area
> > > > ------------------------------------------------------------------------
> > > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > > ------------------------------------------------------------------------
> > > >                                  UBIX
> > > > ------------------------------------------------------------------------
> > > >                                  /dev/mtdX
> > > >
> > > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > > the volumes, handling bad blocks and wear leveling.
> > > >
> > > Dear Ezequiel,
> > > Thank you so much for your reply.
> > >
> > > This is exactly what we are also doing :)
> > > In our system we have a mix of raw and ubi partitions.
> > > The ubi partitioning is done almost exactly the same way.
> > > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > > mount the rootfs.
> > > Now, I understood we should change it to use /dev/ubiblock<id>
> > > This might have several benefits, but one most important could be,
> > > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > > whereas mtdblocks access the flash directly ?
> > > I found some references for these..
> > > So, this seems good for my proposal.
> > >
> > > Another thing that is still open for us is:
> > > How do we calculate the exact image size from a raw mtd partition ?
> > > For example, support for one of the raw nand partitions, the size is
> > > defined as 15MB but we flash the actual image of size only 2.5MB.
> > > So, in the runtime how to determine the image size as ~2.5MB (at least
> > > roughly) ?
> > > Is it still possible ?
> > >
> >
> > I am happy to inform you that using "ubiblock" for squashfs mounting
> > seems very helpful for us.
> > We have seen almost the double performance boost when using ubiblock
> > for rootfs as well as other read-only volume mounting.
> >
> > However, we have found few issues while defining the read only volume as STATIC.
> > With static volume we see that OTA update is failing during "fsync".
> > That is ota_fsync is failing from here:
> > https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> > int status = fsync(fd);
> > if (status == -1 && errno == EIO)
> > *
> > { have_eio_error = true; }
> > *
> > return status;
> > }
> >
> > Is this the known issue with static volume?
> >
>
> I don't know exactly how you are updating your volume,
> the right way is using UBI_IOCVOLUP.
>
> See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
>
> If you google around I'm sure you'll find some articles about this,
> but I'm not sure if they'll go into details and subtleties.
>
> There are probably a few different ways to do firmware upgrade
> when you are on top of static volumes (and you want to be on top
> of static volumes if it's read-only, because AFAIK they give you an
> extra data-integrity guarantee).
>
> One way, would be to have two static volumes A/B. The system
> uses normally the A volume, and then you doUBI_IOCVOLUP
> (or ubiupdatevol) to update the B volume. After the update is succesful
> you run the atomic volume rename and flip A->B, B->A.
>
> (If you don't have enough space to hold two A/B volumes....
>  ... you'll have to find some other solution, I have no idea about that.)
>

Yes, this is what we are also doing exactly.
But, currently we are running into this issue right now:
1) The FOTA update is failing if we use static volume (building and
flashing the static image is fine)
    It works fine, if we use the volume as dynamic (even for RO type
with squashfs)
    But with dynamic type we end up using the entire volume as image size.

So, we wanted to know if there is any other way to get volume image
size at runtime ?


Thanks,
Pintu

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

On Mon, 8 Nov 2021 at 10:51, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> Hi,
>
>
> On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > >
> > > Hi All,
> > >
> > > On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > > >
> > > > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > > > <ezequiel@vanguardiasur.com.ar> wrote:
> > > >
> > > > > In other words, IMO it's best to expose the NAND through UBI
> > > > > for both read-only and read-write access, using a single UBI device,
> > > > > and then creating UBI volumes as needed. This will allow UBI
> > > > > to spread wear leveling across the whole device, which is expected
> > > > > to increase the flash lifetime.
> > > > >
> > > > > For instance, just as some silly example, you could have something like this:
> > > > >
> > > > >                                | RootFS SquashFS  |
> > > > >                                | UBI block        | UBIFS User R-W area
> > > > > ------------------------------------------------------------------------
> > > > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > > > ------------------------------------------------------------------------
> > > > >                                  UBIX
> > > > > ------------------------------------------------------------------------
> > > > >                                  /dev/mtdX
> > > > >
> > > > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > > > the volumes, handling bad blocks and wear leveling.
> > > > >
> > > > Dear Ezequiel,
> > > > Thank you so much for your reply.
> > > >
> > > > This is exactly what we are also doing :)
> > > > In our system we have a mix of raw and ubi partitions.
> > > > The ubi partitioning is done almost exactly the same way.
> > > > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > > > mount the rootfs.
> > > > Now, I understood we should change it to use /dev/ubiblock<id>
> > > > This might have several benefits, but one most important could be,
> > > > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > > > whereas mtdblocks access the flash directly ?
> > > > I found some references for these..
> > > > So, this seems good for my proposal.
> > > >
> > > > Another thing that is still open for us is:
> > > > How do we calculate the exact image size from a raw mtd partition ?
> > > > For example, support for one of the raw nand partitions, the size is
> > > > defined as 15MB but we flash the actual image of size only 2.5MB.
> > > > So, in the runtime how to determine the image size as ~2.5MB (at least
> > > > roughly) ?
> > > > Is it still possible ?
> > > >
> > >
> > > I am happy to inform you that using "ubiblock" for squashfs mounting
> > > seems very helpful for us.
> > > We have seen almost the double performance boost when using ubiblock
> > > for rootfs as well as other read-only volume mounting.
> > >
> > > However, we have found few issues while defining the read only volume as STATIC.
> > > With static volume we see that OTA update is failing during "fsync".
> > > That is ota_fsync is failing from here:
> > > https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> > > int status = fsync(fd);
> > > if (status == -1 && errno == EIO)
> > > *
> > > { have_eio_error = true; }
> > > *
> > > return status;
> > > }
> > >
> > > Is this the known issue with static volume?
> > >
> >
> > I don't know exactly how you are updating your volume,
> > the right way is using UBI_IOCVOLUP.
> >
> > See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
> >
> > If you google around I'm sure you'll find some articles about this,
> > but I'm not sure if they'll go into details and subtleties.
> >
> > There are probably a few different ways to do firmware upgrade
> > when you are on top of static volumes (and you want to be on top
> > of static volumes if it's read-only, because AFAIK they give you an
> > extra data-integrity guarantee).
> >
> > One way, would be to have two static volumes A/B. The system
> > uses normally the A volume, and then you doUBI_IOCVOLUP
> > (or ubiupdatevol) to update the B volume. After the update is succesful
> > you run the atomic volume rename and flip A->B, B->A.
> >
> > (If you don't have enough space to hold two A/B volumes....
> >  ... you'll have to find some other solution, I have no idea about that.)
> >
>
> Yes, this is what we are also doing exactly.
> But, currently we are running into this issue right now:
> 1) The FOTA update is failing if we use static volume (building and
> flashing the static image is fine)

Please add traces to find what syscall is failing, and provide more
details about it.

Thanks,
Ezequiel

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

On Mon, 8 Nov 2021 at 10:51, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> Hi,
>
>
> On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > >
> > > Hi All,
> > >
> > > On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > > >
> > > > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > > > <ezequiel@vanguardiasur.com.ar> wrote:
> > > >
> > > > > In other words, IMO it's best to expose the NAND through UBI
> > > > > for both read-only and read-write access, using a single UBI device,
> > > > > and then creating UBI volumes as needed. This will allow UBI
> > > > > to spread wear leveling across the whole device, which is expected
> > > > > to increase the flash lifetime.
> > > > >
> > > > > For instance, just as some silly example, you could have something like this:
> > > > >
> > > > >                                | RootFS SquashFS  |
> > > > >                                | UBI block        | UBIFS User R-W area
> > > > > ------------------------------------------------------------------------
> > > > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > > > ------------------------------------------------------------------------
> > > > >                                  UBIX
> > > > > ------------------------------------------------------------------------
> > > > >                                  /dev/mtdX
> > > > >
> > > > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > > > the volumes, handling bad blocks and wear leveling.
> > > > >
> > > > Dear Ezequiel,
> > > > Thank you so much for your reply.
> > > >
> > > > This is exactly what we are also doing :)
> > > > In our system we have a mix of raw and ubi partitions.
> > > > The ubi partitioning is done almost exactly the same way.
> > > > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > > > mount the rootfs.
> > > > Now, I understood we should change it to use /dev/ubiblock<id>
> > > > This might have several benefits, but one most important could be,
> > > > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > > > whereas mtdblocks access the flash directly ?
> > > > I found some references for these..
> > > > So, this seems good for my proposal.
> > > >
> > > > Another thing that is still open for us is:
> > > > How do we calculate the exact image size from a raw mtd partition ?
> > > > For example, support for one of the raw nand partitions, the size is
> > > > defined as 15MB but we flash the actual image of size only 2.5MB.
> > > > So, in the runtime how to determine the image size as ~2.5MB (at least
> > > > roughly) ?
> > > > Is it still possible ?
> > > >
> > >
> > > I am happy to inform you that using "ubiblock" for squashfs mounting
> > > seems very helpful for us.
> > > We have seen almost the double performance boost when using ubiblock
> > > for rootfs as well as other read-only volume mounting.
> > >
> > > However, we have found few issues while defining the read only volume as STATIC.
> > > With static volume we see that OTA update is failing during "fsync".
> > > That is ota_fsync is failing from here:
> > > https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> > > int status = fsync(fd);
> > > if (status == -1 && errno == EIO)
> > > *
> > > { have_eio_error = true; }
> > > *
> > > return status;
> > > }
> > >
> > > Is this the known issue with static volume?
> > >
> >
> > I don't know exactly how you are updating your volume,
> > the right way is using UBI_IOCVOLUP.
> >
> > See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
> >
> > If you google around I'm sure you'll find some articles about this,
> > but I'm not sure if they'll go into details and subtleties.
> >
> > There are probably a few different ways to do firmware upgrade
> > when you are on top of static volumes (and you want to be on top
> > of static volumes if it's read-only, because AFAIK they give you an
> > extra data-integrity guarantee).
> >
> > One way, would be to have two static volumes A/B. The system
> > uses normally the A volume, and then you doUBI_IOCVOLUP
> > (or ubiupdatevol) to update the B volume. After the update is succesful
> > you run the atomic volume rename and flip A->B, B->A.
> >
> > (If you don't have enough space to hold two A/B volumes....
> >  ... you'll have to find some other solution, I have no idea about that.)
> >
>
> Yes, this is what we are also doing exactly.
> But, currently we are running into this issue right now:
> 1) The FOTA update is failing if we use static volume (building and
> flashing the static image is fine)

Please add traces to find what syscall is failing, and provide more
details about it.

Thanks,
Ezequiel

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: MTD: How to get actual image size from MTD partition

[Ezequiel Garcia]

On Mon, 8 Nov 2021 at 10:51, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> Hi,
>
>
> On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > >
> > > Hi All,
> > >
> > > On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > > >
> > > > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > > > <ezequiel@vanguardiasur.com.ar> wrote:
> > > >
> > > > > In other words, IMO it's best to expose the NAND through UBI
> > > > > for both read-only and read-write access, using a single UBI device,
> > > > > and then creating UBI volumes as needed. This will allow UBI
> > > > > to spread wear leveling across the whole device, which is expected
> > > > > to increase the flash lifetime.
> > > > >
> > > > > For instance, just as some silly example, you could have something like this:
> > > > >
> > > > >                                | RootFS SquashFS  |
> > > > >                                | UBI block        | UBIFS User R-W area
> > > > > ------------------------------------------------------------------------
> > > > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > > > ------------------------------------------------------------------------
> > > > >                                  UBIX
> > > > > ------------------------------------------------------------------------
> > > > >                                  /dev/mtdX
> > > > >
> > > > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > > > the volumes, handling bad blocks and wear leveling.
> > > > >
> > > > Dear Ezequiel,
> > > > Thank you so much for your reply.
> > > >
> > > > This is exactly what we are also doing :)
> > > > In our system we have a mix of raw and ubi partitions.
> > > > The ubi partitioning is done almost exactly the same way.
> > > > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > > > mount the rootfs.
> > > > Now, I understood we should change it to use /dev/ubiblock<id>
> > > > This might have several benefits, but one most important could be,
> > > > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > > > whereas mtdblocks access the flash directly ?
> > > > I found some references for these..
> > > > So, this seems good for my proposal.
> > > >
> > > > Another thing that is still open for us is:
> > > > How do we calculate the exact image size from a raw mtd partition ?
> > > > For example, support for one of the raw nand partitions, the size is
> > > > defined as 15MB but we flash the actual image of size only 2.5MB.
> > > > So, in the runtime how to determine the image size as ~2.5MB (at least
> > > > roughly) ?
> > > > Is it still possible ?
> > > >
> > >
> > > I am happy to inform you that using "ubiblock" for squashfs mounting
> > > seems very helpful for us.
> > > We have seen almost the double performance boost when using ubiblock
> > > for rootfs as well as other read-only volume mounting.
> > >
> > > However, we have found few issues while defining the read only volume as STATIC.
> > > With static volume we see that OTA update is failing during "fsync".
> > > That is ota_fsync is failing from here:
> > > https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> > > int status = fsync(fd);
> > > if (status == -1 && errno == EIO)
> > > *
> > > { have_eio_error = true; }
> > > *
> > > return status;
> > > }
> > >
> > > Is this the known issue with static volume?
> > >
> >
> > I don't know exactly how you are updating your volume,
> > the right way is using UBI_IOCVOLUP.
> >
> > See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
> >
> > If you google around I'm sure you'll find some articles about this,
> > but I'm not sure if they'll go into details and subtleties.
> >
> > There are probably a few different ways to do firmware upgrade
> > when you are on top of static volumes (and you want to be on top
> > of static volumes if it's read-only, because AFAIK they give you an
> > extra data-integrity guarantee).
> >
> > One way, would be to have two static volumes A/B. The system
> > uses normally the A volume, and then you doUBI_IOCVOLUP
> > (or ubiupdatevol) to update the B volume. After the update is succesful
> > you run the atomic volume rename and flip A->B, B->A.
> >
> > (If you don't have enough space to hold two A/B volumes....
> >  ... you'll have to find some other solution, I have no idea about that.)
> >
>
> Yes, this is what we are also doing exactly.
> But, currently we are running into this issue right now:
> 1) The FOTA update is failing if we use static volume (building and
> flashing the static image is fine)

Please add traces to find what syscall is failing, and provide more
details about it.

Thanks,
Ezequiel

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies