random.c backports for 5.18, 5.17, 5.15, and prior

random.c backports for 5.18, 5.17, 5.15, and prior

[Jason A. Donenfeld]

Hi Greg, Sasha,

I think we're finally at a good point to begin backporting the work I've
done on random.c during the last 6 months. I've been maintaining
branches for this incrementally as code has been merged into mainline,
in order to make this moment easier than otherwise.

Assuming that Linus merges my PR for 5.19 [1] today, all of these
patches are (or will be in a few hours) in Linus' tree. I've tried to
backport most of the general scaffolding and design of the current state
of random.c, while not backporting any new features or unusual
functionality changes that might invite trouble. So, for example, the
backports switch to using a cryptographic hash function, but they don't
have changes like warning when the cycle counter is zero, attempting to
use jitter on early uses of /dev/urandom, reseeding on suspend/hibernate
notifications, or the vmgenid driver. Hopefully that strikes an okay
balance between getting the core backported so that fixes are
backportable, but not going too far by backporting new "nice to have"
but unessential features.

In this git repo [2], there are three branches: linux-5.15.y,
linux-5.17.y, and linux-5.18.y, which contain backports for everything
up to and including [1].

You'll probably want to backport this to earlier kernels as well. Given
that there hasn't been overly much work on the rng in the last few
years, it shouldn't be too hard to take my 5.15.y branch and fill in the
missing pieces there to bring it back. Given how much changes, you could
probably just take every random.c change for backporting to before 5.15.

There is one snag, which is that some of the work I did during the 5.17
cycle depends on crypto I added for WireGuard, which landed in 5.6. So
for 5.4 and prior, that will need backports. Fortunately, I've already
done this in [3], in the branch backport-5.4.y, which I've kept up to
date for a few years now. This occasion might mark the perfect excuse
we've been waiting for to just backport WireGuard too to 5.4 (which
might make the Android work a bit easier also) :-D.

Let me know if you have any questions, and feel free to poke me on IRC.
And if all of the above sounds terrible to you, and you'd rather just
not take any of this into stable, I guess that's a valid path to take
too.

Regards,
Jason

[1] https://lore.kernel.org/lkml/20220522214457.37108-1-Jason@zx2c4.com/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/crng/random.git/
[3] https://git.kernel.org/pub/scm/linux/kernel/git/zx2c4/wireguard-linux.git/

Re: random.c backports for 5.18, 5.17, 5.15, and prior

[Greg KH]

On Mon, May 23, 2022 at 02:54:45PM +0200, Jason A. Donenfeld wrote:
> Hi Greg, Sasha,
> 
> I think we're finally at a good point to begin backporting the work I've
> done on random.c during the last 6 months. I've been maintaining
> branches for this incrementally as code has been merged into mainline,
> in order to make this moment easier than otherwise.
> 
> Assuming that Linus merges my PR for 5.19 [1] today, all of these
> patches are (or will be in a few hours) in Linus' tree. I've tried to
> backport most of the general scaffolding and design of the current state
> of random.c, while not backporting any new features or unusual
> functionality changes that might invite trouble. So, for example, the
> backports switch to using a cryptographic hash function, but they don't
> have changes like warning when the cycle counter is zero, attempting to
> use jitter on early uses of /dev/urandom, reseeding on suspend/hibernate
> notifications, or the vmgenid driver. Hopefully that strikes an okay
> balance between getting the core backported so that fixes are
> backportable, but not going too far by backporting new "nice to have"
> but unessential features.
> 
> In this git repo [2], there are three branches: linux-5.15.y,
> linux-5.17.y, and linux-5.18.y, which contain backports for everything
> up to and including [1].
> 
> You'll probably want to backport this to earlier kernels as well. Given
> that there hasn't been overly much work on the rng in the last few
> years, it shouldn't be too hard to take my 5.15.y branch and fill in the
> missing pieces there to bring it back. Given how much changes, you could
> probably just take every random.c change for backporting to before 5.15.
> 
> There is one snag, which is that some of the work I did during the 5.17
> cycle depends on crypto I added for WireGuard, which landed in 5.6. So
> for 5.4 and prior, that will need backports. Fortunately, I've already
> done this in [3], in the branch backport-5.4.y, which I've kept up to
> date for a few years now. This occasion might mark the perfect excuse
> we've been waiting for to just backport WireGuard too to 5.4 (which
> might make the Android work a bit easier also) :-D.
> 
> Let me know if you have any questions, and feel free to poke me on IRC.
> And if all of the above sounds terrible to you, and you'd rather just
> not take any of this into stable, I guess that's a valid path to take
> too.

Let me look at this later this week after we get this next round of
stable kernels out.  It's normally a nice calm period for the stable
kernels so this might not be that bad.

thanks,

greg k-h

Re: random.c backports for 5.18, 5.17, 5.15, and prior

[Jason A. Donenfeld]

On Mon, May 23, 2022 at 02:54:45PM +0200, Jason A. Donenfeld wrote:
> In this git repo [2], there are three branches: linux-5.15.y,
> linux-5.17.y, and linux-5.18.y, which contain backports for everything
> up to and including [1].

There's now a linux-5.10.y branch too.

Jason

Re: random.c backports for 5.18, 5.17, 5.15, and prior

[Jason A. Donenfeld]

On Mon, May 23, 2022 at 02:54:45PM +0200, Jason A. Donenfeld wrote:
> Assuming that Linus merges my PR for 5.19 [1] today, all of these
> patches are (or will be in a few hours) in Linus' tree.

This is now done, with [1], so everything should be properly upstream
now.

Jason

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ac2ab99072cce553c78f326ea22d72856f570d88

Re: random.c backports for 5.18, 5.17, 5.15, and prior

[Greg KH]

On Mon, May 23, 2022 at 03:36:02PM +0200, Greg KH wrote:
> On Mon, May 23, 2022 at 02:54:45PM +0200, Jason A. Donenfeld wrote:
> > Hi Greg, Sasha,
> > 
> > I think we're finally at a good point to begin backporting the work I've
> > done on random.c during the last 6 months. I've been maintaining
> > branches for this incrementally as code has been merged into mainline,
> > in order to make this moment easier than otherwise.
> > 
> > Assuming that Linus merges my PR for 5.19 [1] today, all of these
> > patches are (or will be in a few hours) in Linus' tree. I've tried to
> > backport most of the general scaffolding and design of the current state
> > of random.c, while not backporting any new features or unusual
> > functionality changes that might invite trouble. So, for example, the
> > backports switch to using a cryptographic hash function, but they don't
> > have changes like warning when the cycle counter is zero, attempting to
> > use jitter on early uses of /dev/urandom, reseeding on suspend/hibernate
> > notifications, or the vmgenid driver. Hopefully that strikes an okay
> > balance between getting the core backported so that fixes are
> > backportable, but not going too far by backporting new "nice to have"
> > but unessential features.
> > 
> > In this git repo [2], there are three branches: linux-5.15.y,
> > linux-5.17.y, and linux-5.18.y, which contain backports for everything
> > up to and including [1].
> > 
> > You'll probably want to backport this to earlier kernels as well. Given
> > that there hasn't been overly much work on the rng in the last few
> > years, it shouldn't be too hard to take my 5.15.y branch and fill in the
> > missing pieces there to bring it back. Given how much changes, you could
> > probably just take every random.c change for backporting to before 5.15.
> > 
> > There is one snag, which is that some of the work I did during the 5.17
> > cycle depends on crypto I added for WireGuard, which landed in 5.6. So
> > for 5.4 and prior, that will need backports. Fortunately, I've already
> > done this in [3], in the branch backport-5.4.y, which I've kept up to
> > date for a few years now. This occasion might mark the perfect excuse
> > we've been waiting for to just backport WireGuard too to 5.4 (which
> > might make the Android work a bit easier also) :-D.
> > 
> > Let me know if you have any questions, and feel free to poke me on IRC.
> > And if all of the above sounds terrible to you, and you'd rather just
> > not take any of this into stable, I guess that's a valid path to take
> > too.
> 
> Let me look at this later this week after we get this next round of
> stable kernels out.  It's normally a nice calm period for the stable
> kernels so this might not be that bad.

I get build errors in the 5.15 and 5.10 trees when applying these
patches.

Here's the 5.10 error:
In file included from ../crypto/testmgr.c:32:
../include/crypto/drbg.h:139:38: error: field 'random_ready' has incomplete type
  139 |         struct random_ready_callback random_ready;
      |                                      ^~~~~~~~~~~~


And the same error in 5.15.

So obviously I can't take them, I'm doing a simple 'make allmodconfig'
build for these trees on x86-64.

For 5.18 and 5.17, wow, lots of patches, but sure, let's see what
happens, I've queued them up now.

thanks,

greg k-h

Re: random.c backports for 5.18, 5.17, 5.15, and prior

[Jason A. Donenfeld]

Hi Greg,

On Thu, May 26, 2022 at 4:44 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> I get build errors in the 5.15 and 5.10 trees when applying these
> patches.
>
> Here's the 5.10 error:
> In file included from ../crypto/testmgr.c:32:
> ../include/crypto/drbg.h:139:38: error: field 'random_ready' has incomplete type
>   139 |         struct random_ready_callback random_ready;
>       |                                      ^~~~~~~~~~~~
>
>
> And the same error in 5.15.
>
> So obviously I can't take them, I'm doing a simple 'make allmodconfig'
> build for these trees on x86-64.

Sorry about that. I've fixed the broken backport (of "random: replace
custom notifier chain with standard one") and force pushed
linux-5.10.y and linux-5.15.y branches.

Jason

Re: random.c backports for 5.18, 5.17, 5.15, and prior

[Greg KH]

On Thu, May 26, 2022 at 05:15:12PM +0200, Jason A. Donenfeld wrote:
> Hi Greg,
> 
> On Thu, May 26, 2022 at 4:44 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> > I get build errors in the 5.15 and 5.10 trees when applying these
> > patches.
> >
> > Here's the 5.10 error:
> > In file included from ../crypto/testmgr.c:32:
> > ../include/crypto/drbg.h:139:38: error: field 'random_ready' has incomplete type
> >   139 |         struct random_ready_callback random_ready;
> >       |                                      ^~~~~~~~~~~~
> >
> >
> > And the same error in 5.15.
> >
> > So obviously I can't take them, I'm doing a simple 'make allmodconfig'
> > build for these trees on x86-64.
> 
> Sorry about that. I've fixed the broken backport (of "random: replace
> custom notifier chain with standard one") and force pushed
> linux-5.10.y and linux-5.15.y branches.

All now queued up.

These are big changes overall, but as you've broken them up into the
original bits, they seem semi-sane.  Let's see how the testing goes on
them...

thanks,

greg k-h

Re: random.c backports for 5.18, 5.17, 5.15, and prior

[Greg KH]

On Mon, May 23, 2022 at 03:36:02PM +0200, Greg KH wrote:
> On Mon, May 23, 2022 at 02:54:45PM +0200, Jason A. Donenfeld wrote:
> > Hi Greg, Sasha,
> > 
> > I think we're finally at a good point to begin backporting the work I've
> > done on random.c during the last 6 months. I've been maintaining
> > branches for this incrementally as code has been merged into mainline,
> > in order to make this moment easier than otherwise.
> > 
> > Assuming that Linus merges my PR for 5.19 [1] today, all of these
> > patches are (or will be in a few hours) in Linus' tree. I've tried to
> > backport most of the general scaffolding and design of the current state
> > of random.c, while not backporting any new features or unusual
> > functionality changes that might invite trouble. So, for example, the
> > backports switch to using a cryptographic hash function, but they don't
> > have changes like warning when the cycle counter is zero, attempting to
> > use jitter on early uses of /dev/urandom, reseeding on suspend/hibernate
> > notifications, or the vmgenid driver. Hopefully that strikes an okay
> > balance between getting the core backported so that fixes are
> > backportable, but not going too far by backporting new "nice to have"
> > but unessential features.
> > 
> > In this git repo [2], there are three branches: linux-5.15.y,
> > linux-5.17.y, and linux-5.18.y, which contain backports for everything
> > up to and including [1].
> > 
> > You'll probably want to backport this to earlier kernels as well. Given
> > that there hasn't been overly much work on the rng in the last few
> > years, it shouldn't be too hard to take my 5.15.y branch and fill in the
> > missing pieces there to bring it back. Given how much changes, you could
> > probably just take every random.c change for backporting to before 5.15.
> > 
> > There is one snag, which is that some of the work I did during the 5.17
> > cycle depends on crypto I added for WireGuard, which landed in 5.6. So
> > for 5.4 and prior, that will need backports. Fortunately, I've already
> > done this in [3], in the branch backport-5.4.y, which I've kept up to
> > date for a few years now. This occasion might mark the perfect excuse
> > we've been waiting for to just backport WireGuard too to 5.4 (which
> > might make the Android work a bit easier also) :-D.
> > 
> > Let me know if you have any questions, and feel free to poke me on IRC.
> > And if all of the above sounds terrible to you, and you'd rather just
> > not take any of this into stable, I guess that's a valid path to take
> > too.
> 
> Let me look at this later this week after we get this next round of
> stable kernels out.  It's normally a nice calm period for the stable
> kernels so this might not be that bad.

This is all now merged into 5.10 and newer.

For 5.4 I just don't think it's worth the trouble.  Devices based on
that old kernel aren't going to want to deal with the churn like this,
and being forced to add the WG code to it also seems like a lot of
unnecessary work for everyone (including yourself.)

But, I do know that a lot of Android devices are still relying on 5.4,
they already have WG in their kernels so overall this might be a smaller
diff for them?

I don't really know, do you have any people/companies/devices using 5.4
that would want this all added that you can show it is worth it for?

thanks,

greg k-h

Re: random.c backports for 5.18, 5.17, 5.15, and prior

[Jason A. Donenfeld]

Hey Greg,

On Mon, May 30, 2022 at 10:11:52AM +0200, Greg KH wrote:
> For 5.4 I just don't think it's worth the trouble.  Devices based on
> that old kernel aren't going to want to deal with the churn like this,
> and being forced to add the WG code to it also seems like a lot of
> unnecessary work for everyone (including yourself.)
> 
> But, I do know that a lot of Android devices are still relying on 5.4,
> they already have WG in their kernels so overall this might be a smaller
> diff for them?
> 
> I don't really know, do you have any people/companies/devices using 5.4
> that would want this all added that you can show it is worth it for?

I think if it's in 5.10, it makes sense to at least try to get it into
5.4 and below for the same reasons. I'm traveling over the next week or
so, but I think I'll attempt to do a straight backport of it (sans-wg)
like I did for 5.10. As mentioned, it's harder, but that doesn't mean
it's impossible. I might give up in exasperation or perhaps find it too
onerous. But hopefully I'll be able to reuse the work I did for the
Android wg backports. Anyway, no guarantees -- it's not a trivial walk
in the park -- but I'll give it a shot and let you know if I can make it
work.

Jason

Re: random.c backports for 5.18, 5.17, 5.15, and prior

[Jason A. Donenfeld]

Hi Greg,

On Mon, May 30, 2022 at 12:38:15PM +0200, Jason A. Donenfeld wrote:
> Hey Greg,
> 
> I think if it's in 5.10, it makes sense to at least try to get it into
> 5.4 and below for the same reasons. I'm traveling over the next week or
> so, but I think I'll attempt to do a straight backport of it (sans-wg)
> like I did for 5.10. As mentioned, it's harder, but that doesn't mean
> it's impossible. I might give up in exasperation or perhaps find it too
> onerous. But hopefully I'll be able to reuse the work I did for the
> Android wg backports. Anyway, no guarantees -- it's not a trivial walk
> in the park -- but I'll give it a shot and let you know if I can make it
> work.

I'm glad I tried, because that turned out to be really easy, and none of
the concerns I had about the crypto turned out to be valid at all. A lot
of the hairiness with the 5.6-era crypto code was the way that
lib/crypto/ interacted with kconfig and crypto/, and the way arch crypto
interacted with that. But for blake2s, there was just a single commit to
backport, which didn't need to interact with anything else, because
there was nothing prior in the kernel regarding blake2s. So it wound up
just being a boring lib/ commit, with no complications.

So with that out of the way, I succeeded in doing the remaining
backports. You can pull from
https://git.kernel.org/pub/scm/linux/kernel/git/crng/random.git/ the
following branches, with a linear series of commits on top of your
latest:

    - linux-4.9.y
    - linux-4.14.y
    - linux-4.19.y
    - linux-5.4.y

I've done an `allmodconfig` build test on these, and I've also booted a
system on each of them. They contain the fixes that have landed since
the previous tranche of backports, so that should bring all the
backports up to date with each other.

And that means that moving forward, a `Cc: stable@vger.kernel.org` tag
will hopefully apply evenly and without hassle to everything. More
globally, I noticed when doing these backports what had been already
backported and what hadn't, and it looks like a lot didn't easily apply
before and so was dropped without being reworked, so over time fixes
were lost. So I'm very happy to bring everything up to date finally.

Regards,
Jason

Re: random.c backports for 5.18, 5.17, 5.15, and prior

[Greg KH]

On Fri, Jun 10, 2022 at 11:05:56AM +0200, Jason A. Donenfeld wrote:
> Hi Greg,
> 
> On Mon, May 30, 2022 at 12:38:15PM +0200, Jason A. Donenfeld wrote:
> > Hey Greg,
> > 
> > I think if it's in 5.10, it makes sense to at least try to get it into
> > 5.4 and below for the same reasons. I'm traveling over the next week or
> > so, but I think I'll attempt to do a straight backport of it (sans-wg)
> > like I did for 5.10. As mentioned, it's harder, but that doesn't mean
> > it's impossible. I might give up in exasperation or perhaps find it too
> > onerous. But hopefully I'll be able to reuse the work I did for the
> > Android wg backports. Anyway, no guarantees -- it's not a trivial walk
> > in the park -- but I'll give it a shot and let you know if I can make it
> > work.
> 
> I'm glad I tried, because that turned out to be really easy, and none of
> the concerns I had about the crypto turned out to be valid at all. A lot
> of the hairiness with the 5.6-era crypto code was the way that
> lib/crypto/ interacted with kconfig and crypto/, and the way arch crypto
> interacted with that. But for blake2s, there was just a single commit to
> backport, which didn't need to interact with anything else, because
> there was nothing prior in the kernel regarding blake2s. So it wound up
> just being a boring lib/ commit, with no complications.
> 
> So with that out of the way, I succeeded in doing the remaining
> backports. You can pull from
> https://git.kernel.org/pub/scm/linux/kernel/git/crng/random.git/ the
> following branches, with a linear series of commits on top of your
> latest:
> 
>     - linux-4.9.y
>     - linux-4.14.y
>     - linux-4.19.y
>     - linux-5.4.y
> 
> I've done an `allmodconfig` build test on these, and I've also booted a
> system on each of them. They contain the fixes that have landed since
> the previous tranche of backports, so that should bring all the
> backports up to date with each other.
> 
> And that means that moving forward, a `Cc: stable@vger.kernel.org` tag
> will hopefully apply evenly and without hassle to everything. More
> globally, I noticed when doing these backports what had been already
> backported and what hadn't, and it looks like a lot didn't easily apply
> before and so was dropped without being reworked, so over time fixes
> were lost. So I'm very happy to bring everything up to date finally.

All now queued up, thanks.

greg k-h