Re: Windows stuck in boot after WireGuard update (switch to WireGuardNT?)

From: "Jason A. Donenfeld" <Jason@zx2c4.com>
To: Oliver Freyermuth <freyermuth@physik.uni-bonn.de>,
	 Joshua Sjoding <joshua.sjoding@scjalliance.com>,
	Arlo Clauser <Arlo@starcubedesign.com>,
	 John-Paul Andreini <jandreini@geonerco.com>
Cc: WireGuard mailing list <wireguard@lists.zx2c4.com>
Subject: Re: Windows stuck in boot after WireGuard update (switch to WireGuardNT?)
Date: Wed, 13 Oct 2021 23:56:56 -0600	[thread overview]
Message-ID: <CAHmME9qSUqo4Wj8U4G3oznxnGB5_7oJkMULwSg4oVVgoRgQ05w@mail.gmail.com> (raw)
In-Reply-To: <CAHmME9rknanKEdk8L5PJxbg1OZejN59z5xuh5y54O7y+zWb1nw@mail.gmail.com>

Hi folks,

I assume that after >3 weeks of silence, the problem is fixed. That's good.

The root cause was that wgnt's initialization waited for winsock to
become available, but winsock was waiting for network interfaces like
wgnt to become available first, so they deadlocked for a half hour.
The solution, deployed over three weeks ago, was to defer wgnt's usage
of winsock until userspace requested it, which is necessarily after
everything else. This apparently works and is the correct fix.

But for whatever reason, the impact of this bug has really haunted me,
and so I didn't stop working on it or thinking about it even after it
was fixed correctly. The larger question is: why is a wgnt adapter
coming up at boot time at all? It's clearly not needed, as it is
something that anyway is controlled by a service. And indeed wgnt
adapters are supposed to be cleaned up when the service exits, but
this cleanup might not be committed if it happens during shutdown.
This is why sometimes the bug happened and sometimes it didn't.

So how can we solve the larger issue of never having wgnt adapters
exist at boot time, and always have them tied to the service lifetime?
It turns out Win8+ has an API for this called SwDevice, which
"unplugs" adapters when their owning process exits (or corresponding
handles close). This is used by various internal Windows components
like rasmans.dll and seems quite robust. It's also very different and
comes with its own gotchas, compared to what we were doing before, and
using it is no small undertaking. So over the last 3 weeks or so, I've
rewritten the entire userspace API portion of wgnt (and wintun) in
order to use this API where available (with ugly fallbacks for Win7).
The result is that the entire class of bugs, of which this bug here
was one, should be eliminated all together. And in the process, I've
made a lot of other things about adapter installation more reliable
and quicker too.

This work shipped with wgnt 0.10 and wintun 0.14, and is part of
wireguard for windows 0.4.11.

Regards,
Jason