Checksummed environments

Checksummed environments

[Kristian Amlie]

Hey, I work for Northern.tech, developing update software for embedded
Linux devices.

I have a question about GRUB's environment block: This block is not
checksummed, and hence I reckon it can become corrupt if power is lost
in the middle of a write.

This is an important safety criterion for us, so we've been thinking of
developing environment block checksumming as an extension to the
existing save_env and load_env commands. The most likely approach will
be to grab X amount of bytes at the end of the block and use these for
the checksum.

This would also allow us to fall back to an earlier environment file if
the current one is corrupt, hence implementing redundancy.

Is this something that the GRUB project would be interested in? We want
to upstream this if possible, since we think many people may benefit
from this.

-- 
Kristian

Re: Checksummed environments

[Daniel Kiper]

On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote:
> Hey, I work for Northern.tech, developing update software for embedded
> Linux devices.
>
> I have a question about GRUB's environment block: This block is not

I am not sure what exactly you mean by "GRUB's environment block".
Could you send me some references to the code?

> checksummed, and hence I reckon it can become corrupt if power is lost
> in the middle of a write.

What about the other blocks?

> This is an important safety criterion for us, so we've been thinking of
> developing environment block checksumming as an extension to the
> existing save_env and load_env commands. The most likely approach will
> be to grab X amount of bytes at the end of the block and use these for
> the checksum.

Could you tell us more about that?

> This would also allow us to fall back to an earlier environment file if
> the current one is corrupt, hence implementing redundancy.
>
> Is this something that the GRUB project would be interested in? We want
> to upstream this if possible, since we think many people may benefit
> from this.

Great! However, first of all we need some more info about that.

Daniel

Re: Checksummed environments

[Kristian Amlie]

On 06/04/18 14:35, Daniel Kiper wrote:
> On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote:
>> Hey, I work for Northern.tech, developing update software for embedded
>> Linux devices.
>>
>> I have a question about GRUB's environment block: This block is not
> 
> I am not sure what exactly you mean by "GRUB's environment block".
> Could you send me some references to the code?

Of course, sorry if the context wasn't clear. I'm talking about the
environment block mentioned here:
https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block,
which is used to store information from one boot to the next, using
save_env and load_env.

>> checksummed, and hence I reckon it can become corrupt if power is lost
>> in the middle of a write.
> 
> What about the other blocks?

In theory, there is only one block, because it is written in-place,
directly on disk, without changing any other filesystem blocks. Is that
what you meant by "other blocks"?

>> This is an important safety criterion for us, so we've been thinking of
>> developing environment block checksumming as an extension to the
>> existing save_env and load_env commands. The most likely approach will
>> be to grab X amount of bytes at the end of the block and use these for
>> the checksum.
> 
> Could you tell us more about that?

The idea comes from U-Boot [1], which uses a dedicated block on a
storage device to store data, and uses a CRC32 checksum to make sure it
is consistent. The motivation is to be able to detect that the block is
corrupt, rather than accepting a block of data that may have incorrect
data in if a write was interrupted midway by a powerloss. You can read
more about it in the link.

[2] https://www.denx.de/wiki/DULG/UBootEnvVariables

-- 
Kristian

> 
>> This would also allow us to fall back to an earlier environment file if
>> the current one is corrupt, hence implementing redundancy.
>>
>> Is this something that the GRUB project would be interested in? We want
>> to upstream this if possible, since we think many people may benefit
>> from this.
> 
> Great! However, first of all we need some more info about that.
> 
> Daniel
> 

Re: Checksummed environments

[Daniel Kiper]

On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote:
> On 06/04/18 14:35, Daniel Kiper wrote:
> > On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote:
> >> Hey, I work for Northern.tech, developing update software for embedded
> >> Linux devices.
> >>
> >> I have a question about GRUB's environment block: This block is not
> >
> > I am not sure what exactly you mean by "GRUB's environment block".
> > Could you send me some references to the code?
>
> Of course, sorry if the context wasn't clear. I'm talking about the
> environment block mentioned here:
> https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block,
> which is used to store information from one boot to the next, using
> save_env and load_env.
>
> >> checksummed, and hence I reckon it can become corrupt if power is lost
> >> in the middle of a write.
> >
> > What about the other blocks?
>
> In theory, there is only one block, because it is written in-place,
> directly on disk, without changing any other filesystem blocks. Is that
> what you meant by "other blocks"?
>
> >> This is an important safety criterion for us, so we've been thinking of
> >> developing environment block checksumming as an extension to the
> >> existing save_env and load_env commands. The most likely approach will
> >> be to grab X amount of bytes at the end of the block and use these for
> >> the checksum.
> >
> > Could you tell us more about that?
>
> The idea comes from U-Boot [1], which uses a dedicated block on a
> storage device to store data, and uses a CRC32 checksum to make sure it
> is consistent. The motivation is to be able to detect that the block is
> corrupt, rather than accepting a block of data that may have incorrect
> data in if a write was interrupted midway by a powerloss. You can read
> more about it in the link.
>
> [2] https://www.denx.de/wiki/DULG/UBootEnvVariables

I am OK with the idea. However, I think that the feature should have
a kind of switch to turn it off/on. At first sight it looks that new
environment variable is sufficient for it.

Daniel

Re: Checksummed environments

[Daniel Kiper]

On Tue, Apr 10, 2018 at 11:09:33PM +0200, Daniel Kiper wrote:
> On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote:
> > On 06/04/18 14:35, Daniel Kiper wrote:
> > > On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote:
> > >> Hey, I work for Northern.tech, developing update software for embedded
> > >> Linux devices.
> > >>
> > >> I have a question about GRUB's environment block: This block is not
> > >
> > > I am not sure what exactly you mean by "GRUB's environment block".
> > > Could you send me some references to the code?
> >
> > Of course, sorry if the context wasn't clear. I'm talking about the
> > environment block mentioned here:
> > https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block,
> > which is used to store information from one boot to the next, using
> > save_env and load_env.
> >
> > >> checksummed, and hence I reckon it can become corrupt if power is lost
> > >> in the middle of a write.
> > >
> > > What about the other blocks?
> >
> > In theory, there is only one block, because it is written in-place,
> > directly on disk, without changing any other filesystem blocks. Is that
> > what you meant by "other blocks"?
> >
> > >> This is an important safety criterion for us, so we've been thinking of
> > >> developing environment block checksumming as an extension to the
> > >> existing save_env and load_env commands. The most likely approach will
> > >> be to grab X amount of bytes at the end of the block and use these for
> > >> the checksum.
> > >
> > > Could you tell us more about that?
> >
> > The idea comes from U-Boot [1], which uses a dedicated block on a
> > storage device to store data, and uses a CRC32 checksum to make sure it
> > is consistent. The motivation is to be able to detect that the block is
> > corrupt, rather than accepting a block of data that may have incorrect
> > data in if a write was interrupted midway by a powerloss. You can read
> > more about it in the link.
> >
> > [2] https://www.denx.de/wiki/DULG/UBootEnvVariables
>
> I am OK with the idea. However, I think that the feature should have
> a kind of switch to turn it off/on. At first sight it looks that new
> environment variable is sufficient for it.

And/Or an argument for save_env/load_env...

Daniel

Re: Checksummed environments

[Kristian Amlie]

On 12/04/18 10:33, Daniel Kiper wrote:
> On Tue, Apr 10, 2018 at 11:09:33PM +0200, Daniel Kiper wrote:
>> On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote:
>>> On 06/04/18 14:35, Daniel Kiper wrote:
>>>> On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote:
>>>>> Hey, I work for Northern.tech, developing update software for embedded
>>>>> Linux devices.
>>>>>
>>>>> I have a question about GRUB's environment block: This block is not
>>>>
>>>> I am not sure what exactly you mean by "GRUB's environment block".
>>>> Could you send me some references to the code?
>>>
>>> Of course, sorry if the context wasn't clear. I'm talking about the
>>> environment block mentioned here:
>>> https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block,
>>> which is used to store information from one boot to the next, using
>>> save_env and load_env.
>>>
>>>>> checksummed, and hence I reckon it can become corrupt if power is lost
>>>>> in the middle of a write.
>>>>
>>>> What about the other blocks?
>>>
>>> In theory, there is only one block, because it is written in-place,
>>> directly on disk, without changing any other filesystem blocks. Is that
>>> what you meant by "other blocks"?
>>>
>>>>> This is an important safety criterion for us, so we've been thinking of
>>>>> developing environment block checksumming as an extension to the
>>>>> existing save_env and load_env commands. The most likely approach will
>>>>> be to grab X amount of bytes at the end of the block and use these for
>>>>> the checksum.
>>>>
>>>> Could you tell us more about that?
>>>
>>> The idea comes from U-Boot [1], which uses a dedicated block on a
>>> storage device to store data, and uses a CRC32 checksum to make sure it
>>> is consistent. The motivation is to be able to detect that the block is
>>> corrupt, rather than accepting a block of data that may have incorrect
>>> data in if a write was interrupted midway by a powerloss. You can read
>>> more about it in the link.
>>>
>>> [2] https://www.denx.de/wiki/DULG/UBootEnvVariables
>>
>> I am OK with the idea. However, I think that the feature should have
>> a kind of switch to turn it off/on. At first sight it looks that new
>> environment variable is sufficient for it.
> 
> And/Or an argument for save_env/load_env...

Yes, I'm fine with either. How about a variable that determines the
default, and you can override it with flags?

-- 
Kristian

Re: Checksummed environments

[Daniel Kiper]

On Thu, Apr 12, 2018 at 10:35:06AM +0200, Kristian Amlie wrote:
> On 12/04/18 10:33, Daniel Kiper wrote:
> > On Tue, Apr 10, 2018 at 11:09:33PM +0200, Daniel Kiper wrote:
> >> On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote:
> >>> On 06/04/18 14:35, Daniel Kiper wrote:
> >>>> On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote:
> >>>>> Hey, I work for Northern.tech, developing update software for embedded
> >>>>> Linux devices.
> >>>>>
> >>>>> I have a question about GRUB's environment block: This block is not
> >>>>
> >>>> I am not sure what exactly you mean by "GRUB's environment block".
> >>>> Could you send me some references to the code?
> >>>
> >>> Of course, sorry if the context wasn't clear. I'm talking about the
> >>> environment block mentioned here:
> >>> https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block,
> >>> which is used to store information from one boot to the next, using
> >>> save_env and load_env.
> >>>
> >>>>> checksummed, and hence I reckon it can become corrupt if power is lost
> >>>>> in the middle of a write.
> >>>>
> >>>> What about the other blocks?
> >>>
> >>> In theory, there is only one block, because it is written in-place,
> >>> directly on disk, without changing any other filesystem blocks. Is that
> >>> what you meant by "other blocks"?
> >>>
> >>>>> This is an important safety criterion for us, so we've been thinking of
> >>>>> developing environment block checksumming as an extension to the
> >>>>> existing save_env and load_env commands. The most likely approach will
> >>>>> be to grab X amount of bytes at the end of the block and use these for
> >>>>> the checksum.
> >>>>
> >>>> Could you tell us more about that?
> >>>
> >>> The idea comes from U-Boot [1], which uses a dedicated block on a
> >>> storage device to store data, and uses a CRC32 checksum to make sure it
> >>> is consistent. The motivation is to be able to detect that the block is
> >>> corrupt, rather than accepting a block of data that may have incorrect
> >>> data in if a write was interrupted midway by a powerloss. You can read
> >>> more about it in the link.
> >>>
> >>> [2] https://www.denx.de/wiki/DULG/UBootEnvVariables
> >>
> >> I am OK with the idea. However, I think that the feature should have
> >> a kind of switch to turn it off/on. At first sight it looks that new
> >> environment variable is sufficient for it.
> >
> > And/Or an argument for save_env/load_env...
>
> Yes, I'm fine with either. How about a variable that determines the
> default, and you can override it with flags?

Works for me. However, I would assume that this feature is by default off.

Daniel

Re: Checksummed environments

[Kristian Amlie]

On 12/04/18 10:58, Daniel Kiper wrote:
> On Thu, Apr 12, 2018 at 10:35:06AM +0200, Kristian Amlie wrote:
>> On 12/04/18 10:33, Daniel Kiper wrote:
>>> On Tue, Apr 10, 2018 at 11:09:33PM +0200, Daniel Kiper wrote:
>>>> On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote:
>>>>> On 06/04/18 14:35, Daniel Kiper wrote:
>>>>>> On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote:
>>>>>>> This is an important safety criterion for us, so we've been thinking of
>>>>>>> developing environment block checksumming as an extension to the
>>>>>>> existing save_env and load_env commands. The most likely approach will
>>>>>>> be to grab X amount of bytes at the end of the block and use these for
>>>>>>> the checksum.
>>>>>>
>>>>>> Could you tell us more about that?
>>>>>
>>>>> The idea comes from U-Boot [1], which uses a dedicated block on a
>>>>> storage device to store data, and uses a CRC32 checksum to make sure it
>>>>> is consistent. The motivation is to be able to detect that the block is
>>>>> corrupt, rather than accepting a block of data that may have incorrect
>>>>> data in if a write was interrupted midway by a powerloss. You can read
>>>>> more about it in the link.
>>>>>
>>>>> [2] https://www.denx.de/wiki/DULG/UBootEnvVariables
>>>>
>>>> I am OK with the idea. However, I think that the feature should have
>>>> a kind of switch to turn it off/on. At first sight it looks that new
>>>> environment variable is sufficient for it.
>>>
>>> And/Or an argument for save_env/load_env...
>>
>> Yes, I'm fine with either. How about a variable that determines the
>> default, and you can override it with flags?
> 
> Works for me. However, I would assume that this feature is by default off.

Of course, we need to maintain full backwards compatibility! :-)

-- 
Kristian