* Checksummed environments @ 2018-04-06 9:25 Kristian Amlie 2018-04-06 12:35 ` Daniel Kiper 0 siblings, 1 reply; 8+ messages in thread From: Kristian Amlie @ 2018-04-06 9:25 UTC (permalink / raw) To: grub-devel Hey, I work for Northern.tech, developing update software for embedded Linux devices. I have a question about GRUB's environment block: This block is not checksummed, and hence I reckon it can become corrupt if power is lost in the middle of a write. This is an important safety criterion for us, so we've been thinking of developing environment block checksumming as an extension to the existing save_env and load_env commands. The most likely approach will be to grab X amount of bytes at the end of the block and use these for the checksum. This would also allow us to fall back to an earlier environment file if the current one is corrupt, hence implementing redundancy. Is this something that the GRUB project would be interested in? We want to upstream this if possible, since we think many people may benefit from this. -- Kristian ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Checksummed environments 2018-04-06 9:25 Checksummed environments Kristian Amlie @ 2018-04-06 12:35 ` Daniel Kiper 2018-04-06 13:08 ` Kristian Amlie 0 siblings, 1 reply; 8+ messages in thread From: Daniel Kiper @ 2018-04-06 12:35 UTC (permalink / raw) To: kristian.amlie; +Cc: grub-devel On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: > Hey, I work for Northern.tech, developing update software for embedded > Linux devices. > > I have a question about GRUB's environment block: This block is not I am not sure what exactly you mean by "GRUB's environment block". Could you send me some references to the code? > checksummed, and hence I reckon it can become corrupt if power is lost > in the middle of a write. What about the other blocks? > This is an important safety criterion for us, so we've been thinking of > developing environment block checksumming as an extension to the > existing save_env and load_env commands. The most likely approach will > be to grab X amount of bytes at the end of the block and use these for > the checksum. Could you tell us more about that? > This would also allow us to fall back to an earlier environment file if > the current one is corrupt, hence implementing redundancy. > > Is this something that the GRUB project would be interested in? We want > to upstream this if possible, since we think many people may benefit > from this. Great! However, first of all we need some more info about that. Daniel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Checksummed environments 2018-04-06 12:35 ` Daniel Kiper @ 2018-04-06 13:08 ` Kristian Amlie 2018-04-10 21:09 ` Daniel Kiper 0 siblings, 1 reply; 8+ messages in thread From: Kristian Amlie @ 2018-04-06 13:08 UTC (permalink / raw) To: Daniel Kiper; +Cc: grub-devel On 06/04/18 14:35, Daniel Kiper wrote: > On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: >> Hey, I work for Northern.tech, developing update software for embedded >> Linux devices. >> >> I have a question about GRUB's environment block: This block is not > > I am not sure what exactly you mean by "GRUB's environment block". > Could you send me some references to the code? Of course, sorry if the context wasn't clear. I'm talking about the environment block mentioned here: https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block, which is used to store information from one boot to the next, using save_env and load_env. >> checksummed, and hence I reckon it can become corrupt if power is lost >> in the middle of a write. > > What about the other blocks? In theory, there is only one block, because it is written in-place, directly on disk, without changing any other filesystem blocks. Is that what you meant by "other blocks"? >> This is an important safety criterion for us, so we've been thinking of >> developing environment block checksumming as an extension to the >> existing save_env and load_env commands. The most likely approach will >> be to grab X amount of bytes at the end of the block and use these for >> the checksum. > > Could you tell us more about that? The idea comes from U-Boot [1], which uses a dedicated block on a storage device to store data, and uses a CRC32 checksum to make sure it is consistent. The motivation is to be able to detect that the block is corrupt, rather than accepting a block of data that may have incorrect data in if a write was interrupted midway by a powerloss. You can read more about it in the link. [2] https://www.denx.de/wiki/DULG/UBootEnvVariables -- Kristian > >> This would also allow us to fall back to an earlier environment file if >> the current one is corrupt, hence implementing redundancy. >> >> Is this something that the GRUB project would be interested in? We want >> to upstream this if possible, since we think many people may benefit >> from this. > > Great! However, first of all we need some more info about that. > > Daniel > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Checksummed environments 2018-04-06 13:08 ` Kristian Amlie @ 2018-04-10 21:09 ` Daniel Kiper 2018-04-12 8:33 ` Daniel Kiper 0 siblings, 1 reply; 8+ messages in thread From: Daniel Kiper @ 2018-04-10 21:09 UTC (permalink / raw) To: Kristian Amlie; +Cc: Daniel Kiper, grub-devel On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote: > On 06/04/18 14:35, Daniel Kiper wrote: > > On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: > >> Hey, I work for Northern.tech, developing update software for embedded > >> Linux devices. > >> > >> I have a question about GRUB's environment block: This block is not > > > > I am not sure what exactly you mean by "GRUB's environment block". > > Could you send me some references to the code? > > Of course, sorry if the context wasn't clear. I'm talking about the > environment block mentioned here: > https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block, > which is used to store information from one boot to the next, using > save_env and load_env. > > >> checksummed, and hence I reckon it can become corrupt if power is lost > >> in the middle of a write. > > > > What about the other blocks? > > In theory, there is only one block, because it is written in-place, > directly on disk, without changing any other filesystem blocks. Is that > what you meant by "other blocks"? > > >> This is an important safety criterion for us, so we've been thinking of > >> developing environment block checksumming as an extension to the > >> existing save_env and load_env commands. The most likely approach will > >> be to grab X amount of bytes at the end of the block and use these for > >> the checksum. > > > > Could you tell us more about that? > > The idea comes from U-Boot [1], which uses a dedicated block on a > storage device to store data, and uses a CRC32 checksum to make sure it > is consistent. The motivation is to be able to detect that the block is > corrupt, rather than accepting a block of data that may have incorrect > data in if a write was interrupted midway by a powerloss. You can read > more about it in the link. > > [2] https://www.denx.de/wiki/DULG/UBootEnvVariables I am OK with the idea. However, I think that the feature should have a kind of switch to turn it off/on. At first sight it looks that new environment variable is sufficient for it. Daniel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Checksummed environments 2018-04-10 21:09 ` Daniel Kiper @ 2018-04-12 8:33 ` Daniel Kiper 2018-04-12 8:35 ` Kristian Amlie 0 siblings, 1 reply; 8+ messages in thread From: Daniel Kiper @ 2018-04-12 8:33 UTC (permalink / raw) To: kristian.amlie; +Cc: dkiper, grub-devel On Tue, Apr 10, 2018 at 11:09:33PM +0200, Daniel Kiper wrote: > On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote: > > On 06/04/18 14:35, Daniel Kiper wrote: > > > On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: > > >> Hey, I work for Northern.tech, developing update software for embedded > > >> Linux devices. > > >> > > >> I have a question about GRUB's environment block: This block is not > > > > > > I am not sure what exactly you mean by "GRUB's environment block". > > > Could you send me some references to the code? > > > > Of course, sorry if the context wasn't clear. I'm talking about the > > environment block mentioned here: > > https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block, > > which is used to store information from one boot to the next, using > > save_env and load_env. > > > > >> checksummed, and hence I reckon it can become corrupt if power is lost > > >> in the middle of a write. > > > > > > What about the other blocks? > > > > In theory, there is only one block, because it is written in-place, > > directly on disk, without changing any other filesystem blocks. Is that > > what you meant by "other blocks"? > > > > >> This is an important safety criterion for us, so we've been thinking of > > >> developing environment block checksumming as an extension to the > > >> existing save_env and load_env commands. The most likely approach will > > >> be to grab X amount of bytes at the end of the block and use these for > > >> the checksum. > > > > > > Could you tell us more about that? > > > > The idea comes from U-Boot [1], which uses a dedicated block on a > > storage device to store data, and uses a CRC32 checksum to make sure it > > is consistent. The motivation is to be able to detect that the block is > > corrupt, rather than accepting a block of data that may have incorrect > > data in if a write was interrupted midway by a powerloss. You can read > > more about it in the link. > > > > [2] https://www.denx.de/wiki/DULG/UBootEnvVariables > > I am OK with the idea. However, I think that the feature should have > a kind of switch to turn it off/on. At first sight it looks that new > environment variable is sufficient for it. And/Or an argument for save_env/load_env... Daniel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Checksummed environments 2018-04-12 8:33 ` Daniel Kiper @ 2018-04-12 8:35 ` Kristian Amlie 2018-04-12 8:58 ` Daniel Kiper 0 siblings, 1 reply; 8+ messages in thread From: Kristian Amlie @ 2018-04-12 8:35 UTC (permalink / raw) To: Daniel Kiper; +Cc: grub-devel On 12/04/18 10:33, Daniel Kiper wrote: > On Tue, Apr 10, 2018 at 11:09:33PM +0200, Daniel Kiper wrote: >> On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote: >>> On 06/04/18 14:35, Daniel Kiper wrote: >>>> On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: >>>>> Hey, I work for Northern.tech, developing update software for embedded >>>>> Linux devices. >>>>> >>>>> I have a question about GRUB's environment block: This block is not >>>> >>>> I am not sure what exactly you mean by "GRUB's environment block". >>>> Could you send me some references to the code? >>> >>> Of course, sorry if the context wasn't clear. I'm talking about the >>> environment block mentioned here: >>> https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block, >>> which is used to store information from one boot to the next, using >>> save_env and load_env. >>> >>>>> checksummed, and hence I reckon it can become corrupt if power is lost >>>>> in the middle of a write. >>>> >>>> What about the other blocks? >>> >>> In theory, there is only one block, because it is written in-place, >>> directly on disk, without changing any other filesystem blocks. Is that >>> what you meant by "other blocks"? >>> >>>>> This is an important safety criterion for us, so we've been thinking of >>>>> developing environment block checksumming as an extension to the >>>>> existing save_env and load_env commands. The most likely approach will >>>>> be to grab X amount of bytes at the end of the block and use these for >>>>> the checksum. >>>> >>>> Could you tell us more about that? >>> >>> The idea comes from U-Boot [1], which uses a dedicated block on a >>> storage device to store data, and uses a CRC32 checksum to make sure it >>> is consistent. The motivation is to be able to detect that the block is >>> corrupt, rather than accepting a block of data that may have incorrect >>> data in if a write was interrupted midway by a powerloss. You can read >>> more about it in the link. >>> >>> [2] https://www.denx.de/wiki/DULG/UBootEnvVariables >> >> I am OK with the idea. However, I think that the feature should have >> a kind of switch to turn it off/on. At first sight it looks that new >> environment variable is sufficient for it. > > And/Or an argument for save_env/load_env... Yes, I'm fine with either. How about a variable that determines the default, and you can override it with flags? -- Kristian ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Checksummed environments 2018-04-12 8:35 ` Kristian Amlie @ 2018-04-12 8:58 ` Daniel Kiper 2018-04-12 9:09 ` Kristian Amlie 0 siblings, 1 reply; 8+ messages in thread From: Daniel Kiper @ 2018-04-12 8:58 UTC (permalink / raw) To: Kristian Amlie; +Cc: Daniel Kiper, grub-devel On Thu, Apr 12, 2018 at 10:35:06AM +0200, Kristian Amlie wrote: > On 12/04/18 10:33, Daniel Kiper wrote: > > On Tue, Apr 10, 2018 at 11:09:33PM +0200, Daniel Kiper wrote: > >> On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote: > >>> On 06/04/18 14:35, Daniel Kiper wrote: > >>>> On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: > >>>>> Hey, I work for Northern.tech, developing update software for embedded > >>>>> Linux devices. > >>>>> > >>>>> I have a question about GRUB's environment block: This block is not > >>>> > >>>> I am not sure what exactly you mean by "GRUB's environment block". > >>>> Could you send me some references to the code? > >>> > >>> Of course, sorry if the context wasn't clear. I'm talking about the > >>> environment block mentioned here: > >>> https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block, > >>> which is used to store information from one boot to the next, using > >>> save_env and load_env. > >>> > >>>>> checksummed, and hence I reckon it can become corrupt if power is lost > >>>>> in the middle of a write. > >>>> > >>>> What about the other blocks? > >>> > >>> In theory, there is only one block, because it is written in-place, > >>> directly on disk, without changing any other filesystem blocks. Is that > >>> what you meant by "other blocks"? > >>> > >>>>> This is an important safety criterion for us, so we've been thinking of > >>>>> developing environment block checksumming as an extension to the > >>>>> existing save_env and load_env commands. The most likely approach will > >>>>> be to grab X amount of bytes at the end of the block and use these for > >>>>> the checksum. > >>>> > >>>> Could you tell us more about that? > >>> > >>> The idea comes from U-Boot [1], which uses a dedicated block on a > >>> storage device to store data, and uses a CRC32 checksum to make sure it > >>> is consistent. The motivation is to be able to detect that the block is > >>> corrupt, rather than accepting a block of data that may have incorrect > >>> data in if a write was interrupted midway by a powerloss. You can read > >>> more about it in the link. > >>> > >>> [2] https://www.denx.de/wiki/DULG/UBootEnvVariables > >> > >> I am OK with the idea. However, I think that the feature should have > >> a kind of switch to turn it off/on. At first sight it looks that new > >> environment variable is sufficient for it. > > > > And/Or an argument for save_env/load_env... > > Yes, I'm fine with either. How about a variable that determines the > default, and you can override it with flags? Works for me. However, I would assume that this feature is by default off. Daniel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Checksummed environments 2018-04-12 8:58 ` Daniel Kiper @ 2018-04-12 9:09 ` Kristian Amlie 0 siblings, 0 replies; 8+ messages in thread From: Kristian Amlie @ 2018-04-12 9:09 UTC (permalink / raw) To: Daniel Kiper; +Cc: grub-devel On 12/04/18 10:58, Daniel Kiper wrote: > On Thu, Apr 12, 2018 at 10:35:06AM +0200, Kristian Amlie wrote: >> On 12/04/18 10:33, Daniel Kiper wrote: >>> On Tue, Apr 10, 2018 at 11:09:33PM +0200, Daniel Kiper wrote: >>>> On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote: >>>>> On 06/04/18 14:35, Daniel Kiper wrote: >>>>>> On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: >>>>>>> This is an important safety criterion for us, so we've been thinking of >>>>>>> developing environment block checksumming as an extension to the >>>>>>> existing save_env and load_env commands. The most likely approach will >>>>>>> be to grab X amount of bytes at the end of the block and use these for >>>>>>> the checksum. >>>>>> >>>>>> Could you tell us more about that? >>>>> >>>>> The idea comes from U-Boot [1], which uses a dedicated block on a >>>>> storage device to store data, and uses a CRC32 checksum to make sure it >>>>> is consistent. The motivation is to be able to detect that the block is >>>>> corrupt, rather than accepting a block of data that may have incorrect >>>>> data in if a write was interrupted midway by a powerloss. You can read >>>>> more about it in the link. >>>>> >>>>> [2] https://www.denx.de/wiki/DULG/UBootEnvVariables >>>> >>>> I am OK with the idea. However, I think that the feature should have >>>> a kind of switch to turn it off/on. At first sight it looks that new >>>> environment variable is sufficient for it. >>> >>> And/Or an argument for save_env/load_env... >> >> Yes, I'm fine with either. How about a variable that determines the >> default, and you can override it with flags? > > Works for me. However, I would assume that this feature is by default off. Of course, we need to maintain full backwards compatibility! :-) -- Kristian ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2018-04-12 9:09 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-04-06 9:25 Checksummed environments Kristian Amlie 2018-04-06 12:35 ` Daniel Kiper 2018-04-06 13:08 ` Kristian Amlie 2018-04-10 21:09 ` Daniel Kiper 2018-04-12 8:33 ` Daniel Kiper 2018-04-12 8:35 ` Kristian Amlie 2018-04-12 8:58 ` Daniel Kiper 2018-04-12 9:09 ` Kristian Amlie
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.