Re: Bug#912087: openssh-server: Slow startup after the upgrade to 7.9p1

From: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
To: Kurt Roeckx <kurt@roeckx.be>,
	912087@bugs.debian.org,
	"Package Development List for OpenSSL packages." 
	<pkg-openssl-devel@alioth-lists.debian.net>,
	Theodore Ts'o <tytso@mit.edu>,
	linux-kernel@vger.kernel.org
Cc: "Bernhard Übelacker" <bernhardu@mailbox.org>,
	pkg-systemd-maintainers@lists.alioth.debian.org,
	debian-ssh@lists.debian.org, 912087-submitter@bugs.debian.org
Subject: Re: Bug#912087: openssh-server: Slow startup after the upgrade to 7.9p1
Date: Tue, 30 Oct 2018 01:18:08 +0100	[thread overview]
Message-ID: <20181030001807.7wailpm37mlinsli@breakpoint.cc> (raw)
In-Reply-To: <20181029223334.GH10011@roeckx.be>

On 2018-10-29 23:33:34 [+0100], Kurt Roeckx wrote:
> On Mon, Oct 29, 2018 at 09:58:20PM +0100, Sebastian Andrzej Siewior wrote:
> > On 2018-10-29 18:22:08 [+0100], Kurt Roeckx wrote:
> > > So I believe this is not an openssl issue, but something in the
> > > order that the kernel's RNG is initialized and openssh is started.
> > > Potentionally the RNG isn't initialized at all and you actually
> > > have to wait for the kernel to get it's random data from the slow
> > > way.
> > > 
> > > So I'm reassigning this to systemd and openssh-server, I have no
> > > idea where the problem really is.
> > 
> > I see it, too. So during boot someone invokes "sshd -t" which invokes
> 
> That's:
> ExecStartPre=/usr/sbin/sshd -t
> 
> > 	getrandom(, 32, 0)
> > and this blocks.
> 
> And did systemd-random-seed.service get run before that?

Yes, but it does not matter from what I can see in the code. On my
system this writes 512 to /dev/urandom at timestamp 11.670639. But sshd
does this:

  sshd-2638  [004] .......    22.445819: __x64_sys_getrandom: 1| 32 0
sshd asks for 32 bytes (flags = 0)

  sshd-2638  [004] .......    22.445824: __x64_sys_getrandom: 2
-> crng_ready() is not true so we wait_for_random_bytes()

  sshd-3164  [004] .......   117.577454: __x64_sys_getrandom: 3
-> "crng init done", sshd's getrandom() resumed.

The problem is that the entropy is added but the entropy count is not
increased. So we wait.

Using ioctl(/dev/urandom, RNDADDENTROPY, ) instead writting to
/dev/urandom would do the trick. Or using RNDADDTOENTCNT to increment
the entropy count after it was written. Those two are documented in
random(4). Or RNDRESEEDCRNG could be used to force crng to be reseeded.
It does also the job, too.

Ted, is there any best practise what to do with the seed which as
extrected from /dev/urandom on system shutdown? Using RNDADDTOENTCNT to
speed up init or just write to back to urandom and issue RNDRESEEDCRNG?

> Kurt

Sebastian