All of lore.kernel.org
 help / color / mirror / Atom feed
From: Pintu Agarwal <pintu.ping@gmail.com>
To: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Cc: Richard Weinberger <richard@nod.at>,
	Kernelnewbies <kernelnewbies@kernelnewbies.org>,
	Greg KH <greg@kroah.com>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	linux-mtd <linux-mtd@lists.infradead.org>,
	Sean Nyekjaer <sean@geanix.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Phillip Lougher <phillip@squashfs.org.uk>
Subject: Re: MTD: How to get actual image size from MTD partition
Date: Mon, 8 Nov 2021 19:21:20 +0530	[thread overview]
Message-ID: <CAOuPNLiB5UogzjHqcwEf+HbLqB9QorQ1bU4qVdMSaSsE4taszw@mail.gmail.com> (raw)
In-Reply-To: <CAAEAJfDS3FK19xMs-7LcEjDe7Fx1XW6HZJGyb6Ff=zs2ZKHpJA@mail.gmail.com>

Hi,


On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > Hi All,
> >
> > On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > >
> > > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > > In other words, IMO it's best to expose the NAND through UBI
> > > > for both read-only and read-write access, using a single UBI device,
> > > > and then creating UBI volumes as needed. This will allow UBI
> > > > to spread wear leveling across the whole device, which is expected
> > > > to increase the flash lifetime.
> > > >
> > > > For instance, just as some silly example, you could have something like this:
> > > >
> > > >                                | RootFS SquashFS  |
> > > >                                | UBI block        | UBIFS User R-W area
> > > > ------------------------------------------------------------------------
> > > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > > ------------------------------------------------------------------------
> > > >                                  UBIX
> > > > ------------------------------------------------------------------------
> > > >                                  /dev/mtdX
> > > >
> > > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > > the volumes, handling bad blocks and wear leveling.
> > > >
> > > Dear Ezequiel,
> > > Thank you so much for your reply.
> > >
> > > This is exactly what we are also doing :)
> > > In our system we have a mix of raw and ubi partitions.
> > > The ubi partitioning is done almost exactly the same way.
> > > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > > mount the rootfs.
> > > Now, I understood we should change it to use /dev/ubiblock<id>
> > > This might have several benefits, but one most important could be,
> > > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > > whereas mtdblocks access the flash directly ?
> > > I found some references for these..
> > > So, this seems good for my proposal.
> > >
> > > Another thing that is still open for us is:
> > > How do we calculate the exact image size from a raw mtd partition ?
> > > For example, support for one of the raw nand partitions, the size is
> > > defined as 15MB but we flash the actual image of size only 2.5MB.
> > > So, in the runtime how to determine the image size as ~2.5MB (at least
> > > roughly) ?
> > > Is it still possible ?
> > >
> >
> > I am happy to inform you that using "ubiblock" for squashfs mounting
> > seems very helpful for us.
> > We have seen almost the double performance boost when using ubiblock
> > for rootfs as well as other read-only volume mounting.
> >
> > However, we have found few issues while defining the read only volume as STATIC.
> > With static volume we see that OTA update is failing during "fsync".
> > That is ota_fsync is failing from here:
> > https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> > int status = fsync(fd);
> > if (status == -1 && errno == EIO)
> > *
> > { have_eio_error = true; }
> > *
> > return status;
> > }
> >
> > Is this the known issue with static volume?
> >
>
> I don't know exactly how you are updating your volume,
> the right way is using UBI_IOCVOLUP.
>
> See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
>
> If you google around I'm sure you'll find some articles about this,
> but I'm not sure if they'll go into details and subtleties.
>
> There are probably a few different ways to do firmware upgrade
> when you are on top of static volumes (and you want to be on top
> of static volumes if it's read-only, because AFAIK they give you an
> extra data-integrity guarantee).
>
> One way, would be to have two static volumes A/B. The system
> uses normally the A volume, and then you doUBI_IOCVOLUP
> (or ubiupdatevol) to update the B volume. After the update is succesful
> you run the atomic volume rename and flip A->B, B->A.
>
> (If you don't have enough space to hold two A/B volumes....
>  ... you'll have to find some other solution, I have no idea about that.)
>

Yes, this is what we are also doing exactly.
But, currently we are running into this issue right now:
1) The FOTA update is failing if we use static volume (building and
flashing the static image is fine)
    It works fine, if we use the volume as dynamic (even for RO type
with squashfs)
    But with dynamic type we end up using the entire volume as image size.

So, we wanted to know if there is any other way to get volume image
size at runtime ?


Thanks,
Pintu

WARNING: multiple messages have this Message-ID (diff)
From: Pintu Agarwal <pintu.ping@gmail.com>
To: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Cc: Richard Weinberger <richard@nod.at>,
	Kernelnewbies <kernelnewbies@kernelnewbies.org>,
	 Greg KH <greg@kroah.com>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	 linux-mtd <linux-mtd@lists.infradead.org>,
	Sean Nyekjaer <sean@geanix.com>,
	 linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Phillip Lougher <phillip@squashfs.org.uk>
Subject: Re: MTD: How to get actual image size from MTD partition
Date: Mon, 8 Nov 2021 19:21:20 +0530	[thread overview]
Message-ID: <CAOuPNLiB5UogzjHqcwEf+HbLqB9QorQ1bU4qVdMSaSsE4taszw@mail.gmail.com> (raw)
In-Reply-To: <CAAEAJfDS3FK19xMs-7LcEjDe7Fx1XW6HZJGyb6Ff=zs2ZKHpJA@mail.gmail.com>

Hi,


On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > Hi All,
> >
> > On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > >
> > > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > > In other words, IMO it's best to expose the NAND through UBI
> > > > for both read-only and read-write access, using a single UBI device,
> > > > and then creating UBI volumes as needed. This will allow UBI
> > > > to spread wear leveling across the whole device, which is expected
> > > > to increase the flash lifetime.
> > > >
> > > > For instance, just as some silly example, you could have something like this:
> > > >
> > > >                                | RootFS SquashFS  |
> > > >                                | UBI block        | UBIFS User R-W area
> > > > ------------------------------------------------------------------------
> > > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > > ------------------------------------------------------------------------
> > > >                                  UBIX
> > > > ------------------------------------------------------------------------
> > > >                                  /dev/mtdX
> > > >
> > > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > > the volumes, handling bad blocks and wear leveling.
> > > >
> > > Dear Ezequiel,
> > > Thank you so much for your reply.
> > >
> > > This is exactly what we are also doing :)
> > > In our system we have a mix of raw and ubi partitions.
> > > The ubi partitioning is done almost exactly the same way.
> > > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > > mount the rootfs.
> > > Now, I understood we should change it to use /dev/ubiblock<id>
> > > This might have several benefits, but one most important could be,
> > > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > > whereas mtdblocks access the flash directly ?
> > > I found some references for these..
> > > So, this seems good for my proposal.
> > >
> > > Another thing that is still open for us is:
> > > How do we calculate the exact image size from a raw mtd partition ?
> > > For example, support for one of the raw nand partitions, the size is
> > > defined as 15MB but we flash the actual image of size only 2.5MB.
> > > So, in the runtime how to determine the image size as ~2.5MB (at least
> > > roughly) ?
> > > Is it still possible ?
> > >
> >
> > I am happy to inform you that using "ubiblock" for squashfs mounting
> > seems very helpful for us.
> > We have seen almost the double performance boost when using ubiblock
> > for rootfs as well as other read-only volume mounting.
> >
> > However, we have found few issues while defining the read only volume as STATIC.
> > With static volume we see that OTA update is failing during "fsync".
> > That is ota_fsync is failing from here:
> > https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> > int status = fsync(fd);
> > if (status == -1 && errno == EIO)
> > *
> > { have_eio_error = true; }
> > *
> > return status;
> > }
> >
> > Is this the known issue with static volume?
> >
>
> I don't know exactly how you are updating your volume,
> the right way is using UBI_IOCVOLUP.
>
> See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
>
> If you google around I'm sure you'll find some articles about this,
> but I'm not sure if they'll go into details and subtleties.
>
> There are probably a few different ways to do firmware upgrade
> when you are on top of static volumes (and you want to be on top
> of static volumes if it's read-only, because AFAIK they give you an
> extra data-integrity guarantee).
>
> One way, would be to have two static volumes A/B. The system
> uses normally the A volume, and then you doUBI_IOCVOLUP
> (or ubiupdatevol) to update the B volume. After the update is succesful
> you run the atomic volume rename and flip A->B, B->A.
>
> (If you don't have enough space to hold two A/B volumes....
>  ... you'll have to find some other solution, I have no idea about that.)
>

Yes, this is what we are also doing exactly.
But, currently we are running into this issue right now:
1) The FOTA update is failing if we use static volume (building and
flashing the static image is fine)
    It works fine, if we use the volume as dynamic (even for RO type
with squashfs)
    But with dynamic type we end up using the entire volume as image size.

So, we wanted to know if there is any other way to get volume image
size at runtime ?


Thanks,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

WARNING: multiple messages have this Message-ID (diff)
From: Pintu Agarwal <pintu.ping@gmail.com>
To: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Cc: Phillip Lougher <phillip@squashfs.org.uk>,
	Kernelnewbies <kernelnewbies@kernelnewbies.org>,
	Richard Weinberger <richard@nod.at>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Greg KH <greg@kroah.com>, Sean Nyekjaer <sean@geanix.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-mtd <linux-mtd@lists.infradead.org>
Subject: Re: MTD: How to get actual image size from MTD partition
Date: Mon, 8 Nov 2021 19:21:20 +0530	[thread overview]
Message-ID: <CAOuPNLiB5UogzjHqcwEf+HbLqB9QorQ1bU4qVdMSaSsE4taszw@mail.gmail.com> (raw)
In-Reply-To: <CAAEAJfDS3FK19xMs-7LcEjDe7Fx1XW6HZJGyb6Ff=zs2ZKHpJA@mail.gmail.com>

Hi,


On Fri, 29 Oct 2021 at 22:18, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Fri, 29 Oct 2021 at 13:13, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> > Hi All,
> >
> > On Mon, 30 Aug 2021 at 21:28, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> > >
> > > On Sun, 22 Aug 2021 at 19:51, Ezequiel Garcia
> > > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > > In other words, IMO it's best to expose the NAND through UBI
> > > > for both read-only and read-write access, using a single UBI device,
> > > > and then creating UBI volumes as needed. This will allow UBI
> > > > to spread wear leveling across the whole device, which is expected
> > > > to increase the flash lifetime.
> > > >
> > > > For instance, just as some silly example, you could have something like this:
> > > >
> > > >                                | RootFS SquashFS  |
> > > >                                | UBI block        | UBIFS User R-W area
> > > > ------------------------------------------------------------------------
> > > > Kernel A | Kernel B | RootFS A | RootFS B         | User
> > > > ------------------------------------------------------------------------
> > > >                                  UBIX
> > > > ------------------------------------------------------------------------
> > > >                                  /dev/mtdX
> > > >
> > > > This setup allows safe kernel and rootfs upgrading. The RootFS is read-only
> > > > via SquashFS and there's a read-write user area. UBI is supporting all
> > > > the volumes, handling bad blocks and wear leveling.
> > > >
> > > Dear Ezequiel,
> > > Thank you so much for your reply.
> > >
> > > This is exactly what we are also doing :)
> > > In our system we have a mix of raw and ubi partitions.
> > > The ubi partitioning is done almost exactly the same way.
> > > Only for the rootfs (squashfs) I see we were using /mtd/block<id> to
> > > mount the rootfs.
> > > Now, I understood we should change it to use /dev/ubiblock<id>
> > > This might have several benefits, but one most important could be,
> > > using ubiblock can handle bad-blocks/wear-leveling automatically,
> > > whereas mtdblocks access the flash directly ?
> > > I found some references for these..
> > > So, this seems good for my proposal.
> > >
> > > Another thing that is still open for us is:
> > > How do we calculate the exact image size from a raw mtd partition ?
> > > For example, support for one of the raw nand partitions, the size is
> > > defined as 15MB but we flash the actual image of size only 2.5MB.
> > > So, in the runtime how to determine the image size as ~2.5MB (at least
> > > roughly) ?
> > > Is it still possible ?
> > >
> >
> > I am happy to inform you that using "ubiblock" for squashfs mounting
> > seems very helpful for us.
> > We have seen almost the double performance boost when using ubiblock
> > for rootfs as well as other read-only volume mounting.
> >
> > However, we have found few issues while defining the read only volume as STATIC.
> > With static volume we see that OTA update is failing during "fsync".
> > That is ota_fsync is failing from here:
> > https://gerrit.pixelexperience.org/plugins/gitiles/bootable_recovery/+/ff6df890a2a01bf3bf56d3f430b17a5ef69055cf%5E%21/otafault/ota_io.cpp
> > int status = fsync(fd);
> > if (status == -1 && errno == EIO)
> > *
> > { have_eio_error = true; }
> > *
> > return status;
> > }
> >
> > Is this the known issue with static volume?
> >
>
> I don't know exactly how you are updating your volume,
> the right way is using UBI_IOCVOLUP.
>
> See http://www.linux-mtd.infradead.org/doc/ubi.html#L_volupdate
>
> If you google around I'm sure you'll find some articles about this,
> but I'm not sure if they'll go into details and subtleties.
>
> There are probably a few different ways to do firmware upgrade
> when you are on top of static volumes (and you want to be on top
> of static volumes if it's read-only, because AFAIK they give you an
> extra data-integrity guarantee).
>
> One way, would be to have two static volumes A/B. The system
> uses normally the A volume, and then you doUBI_IOCVOLUP
> (or ubiupdatevol) to update the B volume. After the update is succesful
> you run the atomic volume rename and flip A->B, B->A.
>
> (If you don't have enough space to hold two A/B volumes....
>  ... you'll have to find some other solution, I have no idea about that.)
>

Yes, this is what we are also doing exactly.
But, currently we are running into this issue right now:
1) The FOTA update is failing if we use static volume (building and
flashing the static image is fine)
    It works fine, if we use the volume as dynamic (even for RO type
with squashfs)
    But with dynamic type we end up using the entire volume as image size.

So, we wanted to know if there is any other way to get volume image
size at runtime ?


Thanks,
Pintu

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

  reply	other threads:[~2021-11-08 13:51 UTC|newest]

Thread overview: 69+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-16  6:42 MTD: How to get actual image size from MTD partition Pintu Agarwal
2021-07-16  6:42 ` Pintu Agarwal
2021-07-16  6:42 ` Pintu Agarwal
2021-07-16  7:16 ` Richard Weinberger
2021-07-16  7:16   ` Richard Weinberger
2021-07-16  7:16   ` Richard Weinberger
2021-07-16 15:41 ` Greg KH
2021-07-16 15:41   ` Greg KH
2021-07-16 15:41   ` Greg KH
2021-07-16 16:26   ` Richard Weinberger
2021-07-16 16:26     ` Richard Weinberger
2021-07-16 16:26     ` Richard Weinberger
2021-07-19  9:09     ` Pintu Agarwal
2021-07-19  9:09       ` Pintu Agarwal
2021-07-19  9:09       ` Pintu Agarwal
2021-07-19  9:28       ` Richard Weinberger
2021-07-19  9:28         ` Richard Weinberger
2021-07-19  9:28         ` Richard Weinberger
2021-07-20  6:17         ` Pintu Agarwal
2021-07-20  6:17           ` Pintu Agarwal
2021-07-20  6:17           ` Pintu Agarwal
2021-07-20  6:40           ` Richard Weinberger
2021-07-20  6:40             ` Richard Weinberger
2021-07-20  6:40             ` Richard Weinberger
2021-07-20  8:01             ` Pintu Agarwal
2021-07-20  8:01               ` Pintu Agarwal
2021-07-20  8:01               ` Pintu Agarwal
2021-07-21 20:54               ` Richard Weinberger
2021-07-21 20:54                 ` Richard Weinberger
2021-07-21 20:54                 ` Richard Weinberger
2021-07-22 11:10                 ` Pintu Agarwal
2021-07-22 11:10                   ` Pintu Agarwal
2021-07-22 11:10                   ` Pintu Agarwal
2021-07-27 21:16                   ` Richard Weinberger
2021-07-27 21:16                     ` Richard Weinberger
2021-07-27 21:16                     ` Richard Weinberger
2021-07-29 11:17                     ` Ezequiel Garcia
2021-07-29 11:17                       ` Ezequiel Garcia
2021-07-29 11:17                       ` Ezequiel Garcia
2021-07-29 11:45                       ` Richard Weinberger
2021-07-29 11:45                         ` Richard Weinberger
2021-07-29 11:45                         ` Richard Weinberger
2021-07-29 12:03                         ` Ezequiel Garcia
2021-07-29 12:03                           ` Ezequiel Garcia
2021-07-29 12:03                           ` Ezequiel Garcia
2021-07-29 17:11                           ` Pintu Agarwal
2021-07-29 17:11                             ` Pintu Agarwal
2021-07-29 17:11                             ` Pintu Agarwal
2021-08-20 18:24                             ` Pintu Agarwal
2021-08-20 18:24                               ` Pintu Agarwal
2021-08-20 18:24                               ` Pintu Agarwal
2021-08-22 14:21                               ` Ezequiel Garcia
2021-08-22 14:21                                 ` Ezequiel Garcia
2021-08-22 14:21                                 ` Ezequiel Garcia
2021-08-30 15:58                                 ` Pintu Agarwal
2021-08-30 15:58                                   ` Pintu Agarwal
2021-08-30 15:58                                   ` Pintu Agarwal
2021-10-29 16:12                                   ` Pintu Agarwal
2021-10-29 16:12                                     ` Pintu Agarwal
2021-10-29 16:12                                     ` Pintu Agarwal
2021-10-29 16:48                                     ` Ezequiel Garcia
2021-10-29 16:48                                       ` Ezequiel Garcia
2021-10-29 16:48                                       ` Ezequiel Garcia
2021-11-08 13:51                                       ` Pintu Agarwal [this message]
2021-11-08 13:51                                         ` Pintu Agarwal
2021-11-08 13:51                                         ` Pintu Agarwal
2021-11-12 13:58                                         ` Ezequiel Garcia
2021-11-12 13:58                                           ` Ezequiel Garcia
2021-11-12 13:58                                           ` Ezequiel Garcia

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAOuPNLiB5UogzjHqcwEf+HbLqB9QorQ1bU4qVdMSaSsE4taszw@mail.gmail.com \
    --to=pintu.ping@gmail.com \
    --cc=ezequiel@vanguardiasur.com.ar \
    --cc=greg@kroah.com \
    --cc=kernelnewbies@kernelnewbies.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mtd@lists.infradead.org \
    --cc=phillip@squashfs.org.uk \
    --cc=richard@nod.at \
    --cc=sean@geanix.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.