* [U-Boot] UBI/UBIFS complete integrity check @ 2017-10-31 15:01 Liam Beguin 2017-11-04 21:17 ` Lukasz Majewski 2017-11-05 8:37 ` Ladislav Michl 0 siblings, 2 replies; 8+ messages in thread From: Liam Beguin @ 2017-10-31 15:01 UTC (permalink / raw) To: u-boot Hi everyone, I'm currently using a UBIFS root file system (stored on SPI-NOR flash) and would like to perform a full integrity check before booting it. The rootfs is read-only and until now, I've been computing an md5sum on the whole mtd device from an initramfs and comparing it to a stored md5sum. If both md5sums don't match, I need to stop the boot process completely. If possible, I was hoping to drop initramfs and do the integrity check from U-Boot. I know UBI/UBIFS does a CRC-32 of the data it writes to flash but the intent here is to prevent booting an image where even a _single bit_ of flash may have been corrupted. My question is, does UBI/UBIFS have this kind of complete integrity check built-in? If not, can I take advantage of these CRC-32, to do something equivalent to my md5sum check from U-Boot. Thanks, Liam Beguin Xiphos Systems Corp. http://xiphos.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* [U-Boot] UBI/UBIFS complete integrity check 2017-10-31 15:01 [U-Boot] UBI/UBIFS complete integrity check Liam Beguin @ 2017-11-04 21:17 ` Lukasz Majewski 2017-11-06 16:34 ` Liam Beguin 2017-11-05 8:37 ` Ladislav Michl 1 sibling, 1 reply; 8+ messages in thread From: Lukasz Majewski @ 2017-11-04 21:17 UTC (permalink / raw) To: u-boot Hi Liam, > Hi everyone, > > I'm currently using a UBIFS root file system (stored on SPI-NOR flash) > and would like to perform a full integrity check before booting it. > The rootfs is read-only and until now, I've been computing an md5sum > on the whole mtd device from an initramfs and comparing it to a stored > md5sum. If both md5sums don't match, I need to stop the boot process > completely. > > If possible, I was hoping to drop initramfs and do the integrity check > from U-Boot. U-boot has support for crc32 and sha1 (256). It should be possible to do the integrity checking in it. If you have more SDRAM than SPI-NOR, then you can calculate sha1/crc32 of the whole memory. > I know UBI/UBIFS does a CRC-32 of the data it writes to > flash but the intent here is to prevent booting an image where > even a _single bit_ of flash may have been corrupted. Ok. I see. > > My question is, does UBI/UBIFS have this kind of complete integrity > check built-in? As fair as I'm aware - not. The only recent improvement was the "encryption/decryption" support. > If not, can I take advantage of these CRC-32, It may be hard to access UBI metadata (from PEB/LEB). > to do > something equivalent to my md5sum check from U-Boot. It may be possible to read the whole SPI-NOR Memory content to RAM, calculate crc32/sha1 and compare with some stored value (e.g. in u-boot envs). This all should be done with u-boot prompt. > Thanks, > > Liam Beguin > Xiphos Systems Corp. > http://xiphos.com > _______________________________________________ > U-Boot mailing list > U-Boot at lists.denx.de > https://lists.denx.de/listinfo/u-boot Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20171104/547e7711/attachment.sig> ^ permalink raw reply [flat|nested] 8+ messages in thread
* [U-Boot] UBI/UBIFS complete integrity check 2017-11-04 21:17 ` Lukasz Majewski @ 2017-11-06 16:34 ` Liam Beguin 2017-11-06 17:30 ` Lukasz Majewski 0 siblings, 1 reply; 8+ messages in thread From: Liam Beguin @ 2017-11-06 16:34 UTC (permalink / raw) To: u-boot Hi Lukasz, Thanks for taking the time to answer. On 11/04/2017 05:17 PM, Lukasz Majewski wrote: > Hi Liam, > >> Hi everyone, >> >> I'm currently using a UBIFS root file system (stored on SPI-NOR flash) >> and would like to perform a full integrity check before booting it. >> The rootfs is read-only and until now, I've been computing an md5sum >> on the whole mtd device from an initramfs and comparing it to a stored >> md5sum. If both md5sums don't match, I need to stop the boot process >> completely. >> >> If possible, I was hoping to drop initramfs and do the integrity check >> from U-Boot. > > U-boot has support for crc32 and sha1 (256). It should be possible to > do the integrity checking in it. > > If you have more SDRAM than SPI-NOR, then you can calculate sha1/crc32 > of the whole memory. > >> I know UBI/UBIFS does a CRC-32 of the data it writes to >> flash but the intent here is to prevent booting an image where >> even a _single bit_ of flash may have been corrupted. > > Ok. I see. > >> >> My question is, does UBI/UBIFS have this kind of complete integrity >> check built-in? > > As fair as I'm aware - not. The only recent improvement was the > "encryption/decryption" support I don't think I have enough time right now but would this integrity check be an interesting feature to add? > >> If not, can I take advantage of these CRC-32, > > It may be hard to access UBI metadata (from PEB/LEB). > >> to do >> something equivalent to my md5sum check from U-Boot. > > It may be possible to read the whole SPI-NOR Memory content to RAM, > calculate crc32/sha1 and compare with some stored value (e.g. in u-boot > envs). This all should be done with u-boot prompt. This was my backup plan. I should have enough RAM to do it. > >> Thanks, >> >> Liam Beguin >> Xiphos Systems Corp. >> http://xiphos.com >> _______________________________________________ >> U-Boot mailing list >> U-Boot at lists.denx.de >> https://lists.denx.de/listinfo/u-boot > > > > Best regards, > > Lukasz Majewski > > -- > > DENX Software Engineering GmbH, Managing Director: Wolfgang Denk > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de > Thanks, Liam Beguin Xiphos Systems Corp. http://xiphos.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* [U-Boot] UBI/UBIFS complete integrity check 2017-11-06 16:34 ` Liam Beguin @ 2017-11-06 17:30 ` Lukasz Majewski 0 siblings, 0 replies; 8+ messages in thread From: Lukasz Majewski @ 2017-11-06 17:30 UTC (permalink / raw) To: u-boot Hi Liam, > Hi Lukasz, > > Thanks for taking the time to answer. > > On 11/04/2017 05:17 PM, Lukasz Majewski wrote: > > Hi Liam, > > > >> Hi everyone, > >> > >> I'm currently using a UBIFS root file system (stored on SPI-NOR > >> flash) and would like to perform a full integrity check before > >> booting it. The rootfs is read-only and until now, I've been > >> computing an md5sum on the whole mtd device from an initramfs and > >> comparing it to a stored md5sum. If both md5sums don't match, I > >> need to stop the boot process completely. > >> > >> If possible, I was hoping to drop initramfs and do the integrity > >> check from U-Boot. > > > > U-boot has support for crc32 and sha1 (256). It should be possible > > to do the integrity checking in it. > > > > If you have more SDRAM than SPI-NOR, then you can calculate > > sha1/crc32 of the whole memory. > > > >> I know UBI/UBIFS does a CRC-32 of the data it writes to > >> flash but the intent here is to prevent booting an image where > >> even a _single bit_ of flash may have been corrupted. > > > > Ok. I see. > > > >> > >> My question is, does UBI/UBIFS have this kind of complete integrity > >> check built-in? > > > > As fair as I'm aware - not. The only recent improvement was the > > "encryption/decryption" support > > I don't think I have enough time right now but would this integrity > check be an interesting feature to add? It depends how "secure" your project needs to be... It is just one of the options to consider. > > > > >> If not, can I take advantage of these CRC-32, > > > > It may be hard to access UBI metadata (from PEB/LEB). > > > >> to do > >> something equivalent to my md5sum check from U-Boot. > > > > It may be possible to read the whole SPI-NOR Memory content to RAM, > > calculate crc32/sha1 and compare with some stored value (e.g. in > > u-boot envs). This all should be done with u-boot prompt. > > This was my backup plan. I should have enough RAM to do it. Ok. Good. > > > > >> Thanks, > >> > >> Liam Beguin > >> Xiphos Systems Corp. > >> http://xiphos.com > >> _______________________________________________ > >> U-Boot mailing list > >> U-Boot at lists.denx.de > >> https://lists.denx.de/listinfo/u-boot > > > > > > > > Best regards, > > > > Lukasz Majewski > > > > -- > > > > DENX Software Engineering GmbH, Managing Director: Wolfgang > > Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, > > Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: > > wd at denx.de > > Thanks, > > Liam Beguin > Xiphos Systems Corp. > http://xiphos.com > Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20171106/9dcffa00/attachment.sig> ^ permalink raw reply [flat|nested] 8+ messages in thread
* [U-Boot] UBI/UBIFS complete integrity check 2017-10-31 15:01 [U-Boot] UBI/UBIFS complete integrity check Liam Beguin 2017-11-04 21:17 ` Lukasz Majewski @ 2017-11-05 8:37 ` Ladislav Michl 2017-11-06 17:31 ` Liam Beguin 1 sibling, 1 reply; 8+ messages in thread From: Ladislav Michl @ 2017-11-05 8:37 UTC (permalink / raw) To: u-boot On Tue, Oct 31, 2017 at 11:01:21AM -0400, Liam Beguin wrote: > Hi everyone, > > I'm currently using a UBIFS root file system (stored on SPI-NOR flash) > and would like to perform a full integrity check before booting it. > The rootfs is read-only and until now, I've been computing an md5sum on > the whole mtd device from an initramfs and comparing it to a stored > md5sum. If both md5sums don't match, I need to stop the boot process > completely. Above doesn't sound right even in theory as UBI layer is free to correct bit-flips (unlikely on SPI-NOR) and shuffle eraseblocks around even if read only filesystem is sitting on top of it. See this faq: http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock So, if you are computing md5sum of underlaying mtd device you might get different checksum even for the same UBI content. > If possible, I was hoping to drop initramfs and do the integrity check > from U-Boot. I know UBI/UBIFS does a CRC-32 of the data it writes to > flash but the intent here is to prevent booting an image where > even a _single bit_ of flash may have been corrupted. > > My question is, does UBI/UBIFS have this kind of complete integrity > check built-in? If not, can I take advantage of these CRC-32, to do > something equivalent to my md5sum check from U-Boot. > Thanks, There is md5sum command, question is whenever you UBI volume fits into RAM to do calculation at once. > Liam Beguin > Xiphos Systems Corp. > http://xiphos.com > _______________________________________________ > U-Boot mailing list > U-Boot at lists.denx.de > https://lists.denx.de/listinfo/u-boot ^ permalink raw reply [flat|nested] 8+ messages in thread
* [U-Boot] UBI/UBIFS complete integrity check 2017-11-05 8:37 ` Ladislav Michl @ 2017-11-06 17:31 ` Liam Beguin 2017-11-06 17:57 ` Ladislav Michl 0 siblings, 1 reply; 8+ messages in thread From: Liam Beguin @ 2017-11-06 17:31 UTC (permalink / raw) To: u-boot Hi, Thanks for taking the time to answer. On 11/05/2017 03:37 AM, Ladislav Michl wrote: > On Tue, Oct 31, 2017 at 11:01:21AM -0400, Liam Beguin wrote: >> Hi everyone, >> >> I'm currently using a UBIFS root file system (stored on SPI-NOR flash) >> and would like to perform a full integrity check before booting it. >> The rootfs is read-only and until now, I've been computing an md5sum on >> the whole mtd device from an initramfs and comparing it to a stored >> md5sum. If both md5sums don't match, I need to stop the boot process >> completely. > > Above doesn't sound right even in theory as UBI layer is free to correct > bit-flips (unlikely on SPI-NOR) and shuffle eraseblocks around even > if read only filesystem is sitting on top of it. See this faq: > http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock> > So, if you are computing md5sum of underlaying mtd device you might get > different checksum even for the same UBI content. > I forgot to mention that the flash I use has a hardware lock which is enabled after the filesystem is first written (the flash is locked during boot). I'm quite confident this works as I've been using the md5sum mechanism for some time now. As the UBI layer is able to detect/fix bit-flips, what happens if a bit-flip is detected and UBI cannot write to flash? does it fail to attach? >> If possible, I was hoping to drop initramfs and do the integrity check >> from U-Boot. I know UBI/UBIFS does a CRC-32 of the data it writes to >> flash but the intent here is to prevent booting an image where >> even a _single bit_ of flash may have been corrupted. >> >> My question is, does UBI/UBIFS have this kind of complete integrity >> check built-in? If not, can I take advantage of these CRC-32, to do >> something equivalent to my md5sum check from U-Boot. >> Thanks, > > There is md5sum command, question is whenever you UBI volume fits > into RAM to do calculation at once. > >> Liam Beguin >> Xiphos Systems Corp. >> http://xiphos.com >> _______________________________________________ >> U-Boot mailing list >> U-Boot at lists.denx.de >> https://lists.denx.de/listinfo/u-boot Thanks, Liam Beguin Xiphos Systems Corp. http://xiphos.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* [U-Boot] UBI/UBIFS complete integrity check 2017-11-06 17:31 ` Liam Beguin @ 2017-11-06 17:57 ` Ladislav Michl 2017-11-06 18:29 ` Liam Beguin 0 siblings, 1 reply; 8+ messages in thread From: Ladislav Michl @ 2017-11-06 17:57 UTC (permalink / raw) To: u-boot On Mon, Nov 06, 2017 at 12:31:32PM -0500, Liam Beguin wrote: > Hi, > > Thanks for taking the time to answer. > > On 11/05/2017 03:37 AM, Ladislav Michl wrote: > > On Tue, Oct 31, 2017 at 11:01:21AM -0400, Liam Beguin wrote: > > > Hi everyone, > > > > > > I'm currently using a UBIFS root file system (stored on SPI-NOR flash) > > > and would like to perform a full integrity check before booting it. > > > The rootfs is read-only and until now, I've been computing an md5sum on > > > the whole mtd device from an initramfs and comparing it to a stored > > > md5sum. If both md5sums don't match, I need to stop the boot process > > > completely. > > > > Above doesn't sound right even in theory as UBI layer is free to correct > > bit-flips (unlikely on SPI-NOR) and shuffle eraseblocks around even > > if read only filesystem is sitting on top of it. See this faq: > > http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock> So, if you > > are computing md5sum of underlaying mtd device you might get > > different checksum even for the same UBI content. > > > > I forgot to mention that the flash I use has a hardware lock which is enabled > after the filesystem is first written (the flash is locked during boot). > I'm quite confident this works as I've been using the md5sum mechanism for > some time now. I see. Btw, have you considered using for example squashfs on top of static UBI volume? Given the fact your fs is read only you get: - CRC32 checksum of you volume for free - filesystem compression - you can easily compute mtd5sum of volume content - check filesystem consistency using already provided tools (probably overkill). > As the UBI layer is able to detect/fix bit-flips, what happens if a bit-flip is > detected and UBI cannot write to flash? does it fail to attach? No, it just logs it. It fails once there is enough bitflips that ECC is unable to correct them. But again, it is not case of NOR, I mentioned it just for completeness. > > > If possible, I was hoping to drop initramfs and do the integrity check > > > from U-Boot. I know UBI/UBIFS does a CRC-32 of the data it writes to > > > flash but the intent here is to prevent booting an image where > > > even a _single bit_ of flash may have been corrupted. Above mentioned solution should fail instantly in case of CRC error. Not sure if that's what you want as kernel already took over, so either watchdog will kick in or machine will reboot, if configured to do so after panics. > > > My question is, does UBI/UBIFS have this kind of complete integrity > > > check built-in? If not, can I take advantage of these CRC-32, to do > > > something equivalent to my md5sum check from U-Boot. > > > Thanks, > > > > There is md5sum command, question is whenever you UBI volume fits > > into RAM to do calculation at once. > > > > > Liam Beguin > > > Xiphos Systems Corp. > > > http://xiphos.com > > > _______________________________________________ > > > U-Boot mailing list > > > U-Boot at lists.denx.de > > > https://lists.denx.de/listinfo/u-boot > > Thanks, > > Liam Beguin > Xiphos Systems Corp. > http://xiphos.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* [U-Boot] UBI/UBIFS complete integrity check 2017-11-06 17:57 ` Ladislav Michl @ 2017-11-06 18:29 ` Liam Beguin 0 siblings, 0 replies; 8+ messages in thread From: Liam Beguin @ 2017-11-06 18:29 UTC (permalink / raw) To: u-boot On 11/06/2017 12:57 PM, Ladislav Michl wrote: > On Mon, Nov 06, 2017 at 12:31:32PM -0500, Liam Beguin wrote: >> Hi, >> >> Thanks for taking the time to answer. >> >> On 11/05/2017 03:37 AM, Ladislav Michl wrote: >>> On Tue, Oct 31, 2017 at 11:01:21AM -0400, Liam Beguin wrote: >>>> Hi everyone, >>>> >>>> I'm currently using a UBIFS root file system (stored on SPI-NOR flash) >>>> and would like to perform a full integrity check before booting it. >>>> The rootfs is read-only and until now, I've been computing an md5sum on >>>> the whole mtd device from an initramfs and comparing it to a stored >>>> md5sum. If both md5sums don't match, I need to stop the boot process >>>> completely. >>> >>> Above doesn't sound right even in theory as UBI layer is free to correct >>> bit-flips (unlikely on SPI-NOR) and shuffle eraseblocks around even >>> if read only filesystem is sitting on top of it. See this faq: >>> http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubiblock> So, if you >>> are computing md5sum of underlaying mtd device you might get >>> different checksum even for the same UBI content. >>> >> >> I forgot to mention that the flash I use has a hardware lock which is enabled >> after the filesystem is first written (the flash is locked during boot). >> I'm quite confident this works as I've been using the md5sum mechanism for >> some time now. > > I see. Btw, have you considered using for example squashfs on top of static UBI > volume? Given the fact your fs is read only you get: > - CRC32 checksum of you volume for free > - filesystem compression > - you can easily compute mtd5sum of volume content > - check filesystem consistency using already provided tools (probably overkill). > I have considered squashfs but on rare occasions, I need to be able to write to the rootfs. This involves disabling the hardware lock, remounting RW and updating the reference md5sum after writing. >> As the UBI layer is able to detect/fix bit-flips, what happens if a bit-flip is >> detected and UBI cannot write to flash? does it fail to attach? > > No, it just logs it. It fails once there is enough bitflips that ECC is unable > to correct them. But again, it is not case of NOR, I mentioned it just for > completeness. I understand that bit-flips are uncommon on NOR flash but in our case, they may be caused by the environment not the flash technology. > >>>> If possible, I was hoping to drop initramfs and do the integrity check >>>> from U-Boot. I know UBI/UBIFS does a CRC-32 of the data it writes to >>>> flash but the intent here is to prevent booting an image where >>>> even a _single bit_ of flash may have been corrupted. > > Above mentioned solution should fail instantly in case of CRC error. Not sure if > that's what you want as kernel already took over, so either watchdog will kick in > or machine will reboot, if configured to do so after panics. > Ideally, I'd like the attach to fail in U-Boot, before loading the kernel. Something remains obscure to me... When UBI scans the device and potentially warns of a bit-flip, has the _whole_ data been checked or was it only the information contained in the UBI headers? >>>> My question is, does UBI/UBIFS have this kind of complete integrity >>>> check built-in? If not, can I take advantage of these CRC-32, to do >>>> something equivalent to my md5sum check from U-Boot. >>>> Thanks, >>> >>> There is md5sum command, question is whenever you UBI volume fits >>> into RAM to do calculation at once. >>> >>>> Liam Beguin >>>> Xiphos Systems Corp. >>>> http://xiphos.com >>>> _______________________________________________ >>>> U-Boot mailing list >>>> U-Boot at lists.denx.de >>>> https://lists.denx.de/listinfo/u-boot >> >> Thanks, >> >> Liam Beguin >> Xiphos Systems Corp. >> http://xiphos.com ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2017-11-06 18:29 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-10-31 15:01 [U-Boot] UBI/UBIFS complete integrity check Liam Beguin 2017-11-04 21:17 ` Lukasz Majewski 2017-11-06 16:34 ` Liam Beguin 2017-11-06 17:30 ` Lukasz Majewski 2017-11-05 8:37 ` Ladislav Michl 2017-11-06 17:31 ` Liam Beguin 2017-11-06 17:57 ` Ladislav Michl 2017-11-06 18:29 ` Liam Beguin
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.