[Buildroot] Verifying linux 5.4.x hashes

[Buildroot] Verifying linux 5.4.x hashes

[Ian Merin]

Hello, -- question about verifying linux kernel hashes.  I see in the linux.hash file there is an entry for the latest 5.4.x version, but I dont see any way to actually download and verify that 5.4.x version against the hash in linux.hash

What would be the method to have buildroot download the 'latest' 5.4.x kernel and also verify its hash against linux.hash?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20210528/3bc3c7cb/attachment.html>

[Buildroot] Verifying linux 5.4.x hashes

[Yann E. MORIN]

Ian, All,

On 2021-05-28 17:15 +0000, Ian Merin via buildroot spake thusly:
> Hello, -- question about verifying linux kernel hashes.  I see in the
> linux.hash file there is an entry for the latest 5.4.x version, but I
> dont see any way to actually download and verify that 5.4.x version
> against the hash in linux.hash

Here's a quick summary of our discussion on IRC:

  - the hash file is shared between linux and linux-headers
  - it is still possible to select a linux 5.4.x as linux-headers
  - hence we still ahve a 5.4.x entry even for linux
  - the hashes for custom version are not checked at all, becasue we
    can't have all the hashes of all the kernel versions

> What would be the method to have buildroot download the ???latest???
> 5.4.x kernel and also verify its hash against linux.hash?

And now a quick summary for that part;

 1. expand the hash-checking infra to accept custom hashes; that would
    impact:
        package/pkg-generic
        package/pkg-download
        support/download/dl-wrapper
        support/download/check-hash

 2. in linux/Config.in add a new entry for custom version:
        BR2_LINUX_KERNEL_CUSTOM_VERSION_HASHES="sha256:1234abcd sha512:abcd1234"

Note that I am not vey fond of the hash being set in the menuconfig, but
I don't have a definitive better idea.

One thing to consider, though: people that want to check custom versions
are probably already using a br2-external tree, so they could very well
set such hashes in their tree, e.g;

    br2-external/
        external/mk
        | include ......./hashes.mk
        `------------
        hashes.mk
        | LINUX_CUSTOM_HASHES = sha256:1234abcd sha512:abcd1234
        `------------

So they would be tracked in the VCS, and would apply transparently even
for configurations made from-scratch, even if you forgot to add it to
the configuraiton (becasue there is no need to add it to the
configuration anymore).

So, maybe that is another track to look at. I am not sure either but on
first glance, I think I'd prefer that...

Oh, and don't forget to update the manual accordingly! ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Verifying linux 5.4.x hashes

[Arnout Vandecappelle]

On 28/05/2021 21:55, Yann E. MORIN wrote:
> Ian, All,
> 
> On 2021-05-28 17:15 +0000, Ian Merin via buildroot spake thusly:
>> Hello, -- question about verifying linux kernel hashes.  I see in the
>> linux.hash file there is an entry for the latest 5.4.x version, but I
>> dont see any way to actually download and verify that 5.4.x version
>> against the hash in linux.hash
> 
> Here's a quick summary of our discussion on IRC:
> 
>   - the hash file is shared between linux and linux-headers
>   - it is still possible to select a linux 5.4.x as linux-headers
>   - hence we still ahve a 5.4.x entry even for linux
>   - the hashes for custom version are not checked at all, becasue we
>     can't have all the hashes of all the kernel versions
> 
>> What would be the method to have buildroot download the ???latest???
>> 5.4.x kernel and also verify its hash against linux.hash?
> 
> And now a quick summary for that part;
> 
>  1. expand the hash-checking infra to accept custom hashes; that would
>     impact:
>         package/pkg-generic
>         package/pkg-download
>         support/download/dl-wrapper
>         support/download/check-hash
> 
>  2. in linux/Config.in add a new entry for custom version:
>         BR2_LINUX_KERNEL_CUSTOM_VERSION_HASHES="sha256:1234abcd sha512:abcd1234"
> 
> Note that I am not vey fond of the hash being set in the menuconfig, but
> I don't have a definitive better idea.

 Why not? The kernel version itself is specified in the config file, so it makes
sense that the hash is there to. Compare to a normal package, where the version
and the hash are both specified in the package itself.


> One thing to consider, though: people that want to check custom versions
> are probably already using a br2-external tree, so they could very well
> set such hashes in their tree, e.g;
> 
>     br2-external/
>         external/mk
>         | include ......./hashes.mk
>         `------------
>         hashes.mk
>         | LINUX_CUSTOM_HASHES = sha256:1234abcd sha512:abcd1234
>         `------------
> 
> So they would be tracked in the VCS, and would apply transparently even
> for configurations made from-scratch, even if you forgot to add it to
> the configuraiton (becasue there is no need to add it to the
> configuration anymore).

 That doesn't work at all! You can have two different configs (with two
different kernel versions) in the same external, so you need to make the hash
specific for the config. An easy way to do that: make the hash part of the
config :-)


 Regards,
 Arnout


> 
> So, maybe that is another track to look at. I am not sure either but on
> first glance, I think I'd prefer that...
> 
> Oh, and don't forget to update the manual accordingly! ;-)
> 
> Regards,
> Yann E. MORIN.
> 

[Buildroot] Verifying linux 5.4.x hashes

[Yann E. MORIN]

Arnout, All,

On 2021-05-28 22:03 +0200, Arnout Vandecappelle spake thusly:
> On 28/05/2021 21:55, Yann E. MORIN wrote:
> > On 2021-05-28 17:15 +0000, Ian Merin via buildroot spake thusly:
> >> What would be the method to have buildroot download the ???latest???
> >> 5.4.x kernel and also verify its hash against linux.hash?
> > And now a quick summary for that part;
> > 
> >  1. expand the hash-checking infra to accept custom hashes; that would
> >     impact:
> >         package/pkg-generic
> >         package/pkg-download
> >         support/download/dl-wrapper
> >         support/download/check-hash
> > 
> >  2. in linux/Config.in add a new entry for custom version:
> >         BR2_LINUX_KERNEL_CUSTOM_VERSION_HASHES="sha256:1234abcd sha512:abcd1234"
> > 
> > Note that I am not vey fond of the hash being set in the menuconfig, but
> > I don't have a definitive better idea.
>  Why not? The kernel version itself is specified in the config file, so it makes
> sense that the hash is there to. Compare to a normal package, where the version
> and the hash are both specified in the package itself.
> > One thing to consider, though: people that want to check custom versions
> > are probably already using a br2-external tree, so they could very well
> > set such hashes in their tree, e.g;
>  That doesn't work at all! You can have two different configs (with two
> different kernel versions) in the same external, so you need to make the hash
> specific for the config. An easy way to do that: make the hash part of the
> config :-)

That is why a email client is not meant to write code: you can't test
it. ;-)

But more seriously, that is still doable with some hackery (which means:
don't do it):

    LINUX_CUSTOM_HASH_5.4.123 = sha256:1234abcd
    LINUX_CUSTOM_HASH_5.10.25 = sha256:1234abcd

and so on... Of course, that is still limiting to a set of know versions.
But in a project, the set of kernel versions to ever use is more often
than not very small, i.e. probably a single one, or just one per
suported board...

But OK, the hash in Config.in is more flexible, so yes, Ian: go with
that initial idea of yours.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [EXTERNAL] Re: Verifying linux 5.4.x hashes

[Ian Merin]

Hello,

Why wouldn't you be able to set a custom hash in a BR2-External directory?  

Isn't the issue of multiple kernel versions solvable in the same way it already works?

If you have say 5.4.123 and 5.10.25 why couldn't you have a file that looks like this:

Sha256 abc123 linux-5.4.123.tar.xz
Sha256 123abc linux-5.10.25.tar.xz




-----Original Message-----
From: Yann E. MORIN <yann.morin.1998@free.fr> 
Sent: Friday, May 28, 2021 4:17 PM
To: Arnout Vandecappelle <arnout@mind.be>
Cc: Ian Merin <Ian.Merin@entrust.com>; buildroot at busybox.net
Subject: [EXTERNAL] Re: [Buildroot] Verifying linux 5.4.x hashes

WARNING: This email originated outside of Entrust.
DO NOT CLICK links or attachments unless you trust the sender and know the content is safe.

______________________________________________________________________
Arnout, All,

On 2021-05-28 22:03 +0200, Arnout Vandecappelle spake thusly:
> On 28/05/2021 21:55, Yann E. MORIN wrote:
> > On 2021-05-28 17:15 +0000, Ian Merin via buildroot spake thusly:
> >> What would be the method to have buildroot download the ???latest???
> >> 5.4.x kernel and also verify its hash against linux.hash?
> > And now a quick summary for that part;
> > 
> >  1. expand the hash-checking infra to accept custom hashes; that would
> >     impact:
> >         package/pkg-generic
> >         package/pkg-download
> >         support/download/dl-wrapper
> >         support/download/check-hash
> > 
> >  2. in linux/Config.in add a new entry for custom version:
> >         BR2_LINUX_KERNEL_CUSTOM_VERSION_HASHES="sha256:1234abcd sha512:abcd1234"
> > 
> > Note that I am not vey fond of the hash being set in the menuconfig, 
> > but I don't have a definitive better idea.
>  Why not? The kernel version itself is specified in the config file, 
> so it makes sense that the hash is there to. Compare to a normal 
> package, where the version and the hash are both specified in the package itself.
> > One thing to consider, though: people that want to check custom 
> > versions are probably already using a br2-external tree, so they 
> > could very well set such hashes in their tree, e.g;
>  That doesn't work at all! You can have two different configs (with 
> two different kernel versions) in the same external, so you need to 
> make the hash specific for the config. An easy way to do that: make 
> the hash part of the config :-)

That is why a email client is not meant to write code: you can't test it. ;-)

But more seriously, that is still doable with some hackery (which means:
don't do it):

    LINUX_CUSTOM_HASH_5.4.123 = sha256:1234abcd
    LINUX_CUSTOM_HASH_5.10.25 = sha256:1234abcd

and so on... Of course, that is still limiting to a set of know versions.
But in a project, the set of kernel versions to ever use is more often than not very small, i.e. probably a single one, or just one per suported board...

But OK, the hash in Config.in is more flexible, so yes, Ian: go with that initial idea of yours.

Regards,
Yann E. MORIN.

--
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' 
| conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| https://urldefense.com/v3/__http://ymorin.is-a-geek.org/__;!!FJ-Y8qCqXTj2!OUuT-bsJ5X27mcQXNyq0D_2DViN3j2LNd6MuYUs2xLcgV6-WYtDstIbF2PqUkAI$  | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [EXTERNAL] Re: Verifying linux 5.4.x hashes

[Arnout Vandecappelle]

On 09/06/2021 16:28, Ian Merin wrote:
> Hello,
> 
> Why wouldn't you be able to set a custom hash in a BR2-External directory?

 There's no problem with that. However, there's a problem with Yann's proposal
of setting the hash in a variable.

> 
> Isn't the issue of multiple kernel versions solvable in the same way it already works?
> 
> If you have say 5.4.123 and 5.10.25 why couldn't you have a file that looks like this:
> 
> Sha256 abc123 linux-5.4.123.tar.xz
> Sha256 123abc linux-5.10.25.tar.xz

So instead of this, Yann proposed:

LINUX_CUSTOM_HASH = abc123

but then there's no way to have multiple, so he changed that to

LINUX_CUSTOM_HASH_5.4.123 = abc123

(which I find contrary to the usual way we do things in Buildroot).

 Adding a hash file in an external seems fine to me. The difficulty lies in
determining whether or not the hash is to be verified. We'd need to have an
option of checking the hash or not. And of course you need to make sure that the
hash really *is* checked - it's too easy to forget about checking the hash.

 Another alternative is to make it a config option:

BR2_LINUX_KERNEL_CUSTOM_HASH="abc123"

(since there's only one custom version possible per config). Although that way
is a bit more clunky from a user perspective, I think it's the most consistent
and also the easiest to implement.

 Regards,
 Arnout

> 
> 
> 
> 
> -----Original Message-----
> From: Yann E. MORIN <yann.morin.1998@free.fr> 
> Sent: Friday, May 28, 2021 4:17 PM
> To: Arnout Vandecappelle <arnout@mind.be>
> Cc: Ian Merin <Ian.Merin@entrust.com>; buildroot at busybox.net
> Subject: [EXTERNAL] Re: [Buildroot] Verifying linux 5.4.x hashes
> 
> WARNING: This email originated outside of Entrust.
> DO NOT CLICK links or attachments unless you trust the sender and know the content is safe.
> 
> ______________________________________________________________________
> Arnout, All,
> 
> On 2021-05-28 22:03 +0200, Arnout Vandecappelle spake thusly:
>> On 28/05/2021 21:55, Yann E. MORIN wrote:
>>> On 2021-05-28 17:15 +0000, Ian Merin via buildroot spake thusly:
>>>> What would be the method to have buildroot download the ???latest???
>>>> 5.4.x kernel and also verify its hash against linux.hash?
>>> And now a quick summary for that part;
>>>
>>>  1. expand the hash-checking infra to accept custom hashes; that would
>>>     impact:
>>>         package/pkg-generic
>>>         package/pkg-download
>>>         support/download/dl-wrapper
>>>         support/download/check-hash
>>>
>>>  2. in linux/Config.in add a new entry for custom version:
>>>         BR2_LINUX_KERNEL_CUSTOM_VERSION_HASHES="sha256:1234abcd sha512:abcd1234"
>>>
>>> Note that I am not vey fond of the hash being set in the menuconfig, 
>>> but I don't have a definitive better idea.
>>  Why not? The kernel version itself is specified in the config file, 
>> so it makes sense that the hash is there to. Compare to a normal 
>> package, where the version and the hash are both specified in the package itself.
>>> One thing to consider, though: people that want to check custom 
>>> versions are probably already using a br2-external tree, so they 
>>> could very well set such hashes in their tree, e.g;
>>  That doesn't work at all! You can have two different configs (with 
>> two different kernel versions) in the same external, so you need to 
>> make the hash specific for the config. An easy way to do that: make 
>> the hash part of the config :-)
> 
> That is why a email client is not meant to write code: you can't test it. ;-)
> 
> But more seriously, that is still doable with some hackery (which means:
> don't do it):
> 
>     LINUX_CUSTOM_HASH_5.4.123 = sha256:1234abcd
>     LINUX_CUSTOM_HASH_5.10.25 = sha256:1234abcd
> 
> and so on... Of course, that is still limiting to a set of know versions.
> But in a project, the set of kernel versions to ever use is more often than not very small, i.e. probably a single one, or just one per suported board...
> 
> But OK, the hash in Config.in is more flexible, so yes, Ian: go with that initial idea of yours.
> 
> Regards,
> Yann E. MORIN.
> 
> --
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' 
> | conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
> | https://urldefense.com/v3/__http://ymorin.is-a-geek.org/__;!!FJ-Y8qCqXTj2!OUuT-bsJ5X27mcQXNyq0D_2DViN3j2LNd6MuYUs2xLcgV6-WYtDstIbF2PqUkAI$  | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
> '------------------------------^-------^------------------^--------------------'
> 

[Buildroot] Verifying linux 5.4.x hashes

[Alexander Dahl]

Hello Yann,

On Fri, May 28, 2021 at 09:55:06PM +0200, Yann E. MORIN wrote:
> On 2021-05-28 17:15 +0000, Ian Merin via buildroot spake thusly:
> > Hello, -- question about verifying linux kernel hashes.  I see in the
> > linux.hash file there is an entry for the latest 5.4.x version, but I
> > dont see any way to actually download and verify that 5.4.x version
> > against the hash in linux.hash
> 
> Here's a quick summary of our discussion on IRC:
> 
>   - the hash file is shared between linux and linux-headers
>   - it is still possible to select a linux 5.4.x as linux-headers
>   - hence we still ahve a 5.4.x entry even for linux
>   - the hashes for custom version are not checked at all, becasue we
>     can't have all the hashes of all the kernel versions

Maybe not for non official version, but why not for all mainline
kernel versions?

    % git tag | grep -v rc | wc -l
    3025

This would be 3k lines of text currently, big compared to other
buildroot hashes files, but not that huge in general.  If one could
split it up for major releases, I would consider it maintainable,
that's just few hundred lines per kernel version max.

> > What would be the method to have buildroot download the ???latest???
> > 5.4.x kernel and also verify its hash against linux.hash?
> 
> And now a quick summary for that part;
> 
>  1. expand the hash-checking infra to accept custom hashes; that would
>     impact:
>         package/pkg-generic
>         package/pkg-download
>         support/download/dl-wrapper
>         support/download/check-hash
> 
>  2. in linux/Config.in add a new entry for custom version:
>         BR2_LINUX_KERNEL_CUSTOM_VERSION_HASHES="sha256:1234abcd sha512:abcd1234"
> 
> Note that I am not vey fond of the hash being set in the menuconfig, but
> I don't have a definitive better idea.

What about the above one?  Would be quite some work to setup, but once
in place it would be just adding a new hash to the file instead of
replacing the old one.

> One thing to consider, though: people that want to check custom versions
> are probably already using a br2-external tree, so they could very well
> set such hashes in their tree, e.g;

Would of course not apply to custom versions, for mainline only.  But
we all head for mainline first, anyways, don't we? ;-)

Greets
Alex

-- 
/"\ ASCII RIBBON | ?With the first link, the chain is forged. The first
\ / CAMPAIGN     | speech censured, the first thought forbidden, the
 X  AGAINST      | first freedom denied, chains us all irrevocably.?
/ \ HTML MAIL    | (Jean-Luc Picard, quoting Judge Aaron Satie)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20210528/a867ad05/attachment.asc>

[Buildroot] Verifying linux 5.4.x hashes

[Yann E. MORIN]

Alexander, All,

On 2021-05-28 22:29 +0200, Alexander Dahl spake thusly:
> On Fri, May 28, 2021 at 09:55:06PM +0200, Yann E. MORIN wrote:
> >   - the hashes for custom version are not checked at all, becasue we
> >     can't have all the hashes of all the kernel versions
> Maybe not for non official version, but why not for all mainline
> kernel versions?
>     % git tag | grep -v rc | wc -l
>     3025
> This would be 3k lines of text currently, big compared to other
> buildroot hashes files, but not that huge in general.  If one could
> split it up for major releases, I would consider it maintainable,
> that's just few hundred lines per kernel version max.

The problem is not really about the number of hashes. The problem is
about kernel relases that are made after we have done a relase.

For example, let's assume that we add all known hashes right now (and
update until we release. When we release Buildroot 2021.05, it is
frozen: it's not going to change, that's all about being a release.

A day after we release 2021.05 with all say) then-known 5.4.x versions
and thus up to and including 5.4.123, the kernel team (GKH) releases
5.4.124.

Then Buildroot 2021.05 no longer has a hash for all kernel releases.

That is the actual problem: kernel releases are going to be made after
we tag and release our own versions.

So, no, we can't have a hash for all kernel versions.

Besides, you are missing the (presumably much fewer) CIP-SLTS and
CIP-RT-SLTS versions, but that's a detail...

> Would of course not apply to custom versions, for mainline only.  But
> we all head for mainline first, anyways, don't we? ;-)

Muahahaha! :-]

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Verifying linux 5.4.x hashes

[Arnout Vandecappelle]

On 28/05/2021 19:15, Ian Merin via buildroot wrote:
> Hello, -- question about verifying linux kernel hashes.? I see in the linux.hash
> file there is an entry for the latest 5.4.x version, but I dont see any way to
> actually download and verify that 5.4.x version against the hash in linux.hash
> 
> ?
> 
> What would be the method to have buildroot download the ?latest? 5.4.x kernel
> and also verify its hash against linux.hash?

 There isn't such a method.

 The same hash file is used by both linux and linux-headers. linux-headers
(which is used by the toolchain) has a choice between the still-maintained
stable branches, while linux itself only has the "latest" choice. The idea is
that for the headers, the risk is extremely low when you update them within the
same stable series, while for the kernel itself, there is significant risk of
breakage when it's updated under the hood.

 But, to be honest, I think the value of that choice in linux-headers is pretty
limited. In all my projects, I use the exact same version as the kernel (either
with the "Same as kernel" option, or with a manually specified version when the
kernel is built seperately).


 Regards,
 Arnout